pjandro/OmniRoute
1
0
Fork 0
mirror of https://github.com/diegosouzapw/OmniRoute.git synced 2026-07-26 09:52:11 +03:00
Code Issues Packages Projects Releases Wiki Activity

docs: populate wiki with 59 pages from project documentation 

Categories:
- Getting Started (11 pages): Setup, User Guide, Features, Docker, Electron, etc.
- Providers (4 pages): Reference, Free Tiers, Kiro Setup
- Routing (2 pages): Auto-Combo, Reasoning Replay
- Compression (5 pages): Guide, Engines, RTK, Language Packs, Rules
- Integrations (10 pages): MCP, A2A, OpenCode, Webhooks, Cloud Agents, etc.
- Architecture (5 pages): Overview, Codebase, Repository Map, Authz, Resilience
- Security (7 pages): Guardrails, Compliance, Error Sanitization, etc.
- Reference (5 pages): API, CLI Tools, Environment Variables
- Operations (7 pages): VM/Fly.io Deploy, Tunnels, Proxy, SQLite
- Meta (2 pages): Contributing, Comparison
diegosouzapw
2026-05-23 00:27:37 -03:00
parent 3f1d1e6f05
commit a5470ab85e
58 changed files with 19083 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files

276
A2A-Server.md Normal file

@@ -0,0 +1,276 @@
# OmniRoute A2A Server Documentation
> Agent-to-Agent Protocol v0.3 — OmniRoute as an intelligent routing agent
The A2A surface has two faces:
- **JSON-RPC 2.0** at `POST /a2a` (canonical entry point, defined in `src/app/a2a/route.ts`).
- **REST** under `/api/a2a/*` for dashboards and tooling (status, task list, cancel).
Tasks are tracked by `A2ATaskManager` (`src/lib/a2a/taskManager.ts`, default 5-minute TTL). Skills are dispatched via `A2A_SKILL_HANDLERS` in `src/lib/a2a/taskExecution.ts`.
## Agent Discovery
```bash
curl http://localhost:20128/.well-known/agent.json
```
Returns the Agent Card describing OmniRoute's capabilities, skills, and authentication requirements.
The Agent Card's `version` field is sourced from `process.env.npm_package_version` (see `src/app/.well-known/agent.json/route.ts:13`), so it stays auto-synced with `package.json` on every release.
---
## Authentication
All `/a2a` requests require an API key via the `Authorization` header:
```
Authorization: Bearer YOUR_OMNIROUTE_API_KEY
```
If no API key is configured on the server, authentication is bypassed.
## Enablement
A2A is controlled by the **Endpoints → A2A** toggle and is disabled by default. When disabled,
`GET /api/a2a/status` reports `status: "disabled"` and `online: false`; JSON-RPC calls to
`POST /a2a` return HTTP 503 with JSON-RPC error code `-32000`.
---
## JSON-RPC 2.0 Methods
### `message/send` — Synchronous Execution
Sends a message to a skill and waits for the complete response.
```bash
curl -X POST http://localhost:20128/a2a \
-H "Content-Type: application/json" \
-H "Authorization: Bearer YOUR_KEY" \
-d '{
"jsonrpc": "2.0",
"id": "1",
"method": "message/send",
"params": {
"skill": "smart-routing",
"messages": [{"role": "user", "content": "Write a hello world in Python"}],
"metadata": {"model": "auto", "combo": "fast-coding"}
}
}'
```
**Response:**
```json
{
"jsonrpc": "2.0",
"id": "1",
"result": {
"task": { "id": "uuid", "state": "completed" },
"artifacts": [{ "type": "text", "content": "..." }],
"metadata": {
"routing_explanation": "Selected claude-sonnet via provider \"anthropic\" (latency: 1200ms, cost: $0.003)",
"cost_envelope": { "estimated": 0.005, "actual": 0.003, "currency": "USD" },
"resilience_trace": [
{ "event": "primary_selected", "provider": "anthropic", "timestamp": "..." }
],
"policy_verdict": { "allowed": true, "reason": "within budget and quota limits" }
}
}
}
```
### `message/stream` — SSE Streaming
Same as `message/send` but returns Server-Sent Events for real-time streaming.
```bash
curl -N -X POST http://localhost:20128/a2a \
-H "Content-Type: application/json" \
-H "Authorization: Bearer YOUR_KEY" \
-d '{
"jsonrpc": "2.0",
"id": "1",
"method": "message/stream",
"params": {
"skill": "smart-routing",
"messages": [{"role": "user", "content": "Explain quantum computing"}]
}
}'
```
**SSE Events:**
```
data: {"jsonrpc":"2.0","method":"message/stream","params":{"task":{"id":"...","state":"working"},"chunk":{"type":"text","content":"..."}}}
: heartbeat 2026-03-03T17:00:00Z
data: {"jsonrpc":"2.0","method":"message/stream","params":{"task":{"id":"...","state":"completed"},"metadata":{...}}}
```
### `tasks/get` — Query Task Status
```bash
curl -X POST http://localhost:20128/a2a \
-H "Content-Type: application/json" \
-H "Authorization: Bearer YOUR_KEY" \
-d '{"jsonrpc":"2.0","id":"2","method":"tasks/get","params":{"taskId":"TASK_UUID"}}'
```
### `tasks/cancel` — Cancel a Task
```bash
curl -X POST http://localhost:20128/a2a \
-H "Content-Type: application/json" \
-H "Authorization: Bearer YOUR_KEY" \
-d '{"jsonrpc":"2.0","id":"3","method":"tasks/cancel","params":{"taskId":"TASK_UUID"}}'
```
---
## Available Skills
OmniRoute exposes 5 A2A skills wired in `src/lib/a2a/taskExecution.ts::A2A_SKILL_HANDLERS`. Each skill module lives in `src/lib/a2a/skills/`.
| Skill | ID | Description |
| :----------------- | :------------------- | :------------------------------------------------------------------------------------------ |
| Smart Routing | `smart-routing` | Routes a prompt through the optimal provider/combo using OmniRoute's combo engine + scoring |
| Quota Management | `quota-management` | Reports per-provider quota state, helps callers decide when to throttle/switch |
| Provider Discovery | `provider-discovery` | Lists installed providers with capabilities, free-tier flags, OAuth status |
| Cost Analysis | `cost-analysis` | Estimates cost of a request/conversation given the catalog + recent usage |
| Health Report | `health-report` | Aggregates circuit breaker, cooldown, lockout state per provider |
> Note: the Agent Card description currently advertises "36+ providers" (`src/app/.well-known/agent.json/route.ts:26` and `:55`). The actual catalog has grown to 180+ providers — the string should be updated in a follow-up change (tracked as a separate doc/code TODO; not modified here).
---
## REST API (auxiliary)
The JSON-RPC endpoint `/a2a` is the canonical A2A entry point. The REST endpoints below provide auxiliary access for dashboards and external tooling:
| Endpoint | Method | Description | Auth |
| :--------------------------- | :----- | :------------------------------- | :--------------------- |
| `/api/a2a/status` | GET | Server status, registered skills | (public) |
| `/api/a2a/tasks` | GET | List tasks with filters | management |
| `/api/a2a/tasks/[id]` | GET | Get task by ID | management |
| `/api/a2a/tasks/[id]/cancel` | POST | Cancel running task | management |
| `/.well-known/agent.json` | GET | Agent Card (A2A discovery) | (public, cached 3600s) |
---
## Adding a New Skill
1. **Create skill file:** `src/lib/a2a/skills/<your-skill>.ts`
Export an async function `(task: A2ATask) => Promise<{ artifacts, metadata }>`. Follow the shape of existing skills such as `smartRouting.ts`.
2. **Register handler:** in `src/lib/a2a/taskExecution.ts`, add an entry to `A2A_SKILL_HANDLERS`:
```typescript
export const A2A_SKILL_HANDLERS = {
// ...existing skills
"your-skill": async (task) => {
const skillModule = await import("./skills/yourSkill");
return skillModule.executeYourSkill(task);
},
};
```
3. **Expose in Agent Card:** in `src/app/.well-known/agent.json/route.ts`, append to the `skills` array:
```json
{
"id": "your-skill",
"name": "Your Skill",
"description": "Brief, intent-focused description",
"tags": ["routing", "quota"],
"examples": ["Sample natural-language invocation"]
}
```
4. **Write tests:** `tests/unit/a2a-<your-skill>.test.ts`. Cover happy path + error path.
5. **Document** the new skill in this file's `Available Skills` table.
---
## Task TTL
Tasks expire after `ttlMinutes` (default 5 min) — configured in the `A2ATaskManager` constructor at `src/lib/a2a/taskManager.ts:82`. To customize, fork the `A2ATaskManager` instantiation and pass a different value (e.g., `new A2ATaskManager(15)` for 15-minute TTL). A background interval sweeps expired tasks every 60 seconds.
---
## Task Lifecycle
```
submitted → working → completed
→ failed
→ cancelled
```
- Tasks expire after 5 minutes by default (see [Task TTL](#task-ttl))
- Terminal states: `completed`, `failed`, `cancelled`
- Event log tracks every state transition
---
## Error Codes
| Code | Meaning |
| :----- | :----------------------------- |
| -32700 | Parse error (invalid JSON) |
| -32600 | Invalid request / Unauthorized |
| -32601 | Method or skill not found |
| -32602 | Invalid params |
| -32603 | Internal error |
| -32000 | A2A endpoint is disabled |
---
## Integration Examples
### Python (requests)
```python
import requests
resp = requests.post("http://localhost:20128/a2a", json={
"jsonrpc": "2.0", "id": "1",
"method": "message/send",
"params": {
"skill": "smart-routing",
"messages": [{"role": "user", "content": "Hello"}]
}
}, headers={"Authorization": "Bearer YOUR_KEY"})
result = resp.json()["result"]
print(result["artifacts"][0]["content"])
print(result["metadata"]["routing_explanation"])
```
### TypeScript (fetch)
```typescript
const resp = await fetch("http://localhost:20128/a2a", {
method: "POST",
headers: {
"Content-Type": "application/json",
Authorization: "Bearer YOUR_KEY",
},
body: JSON.stringify({
jsonrpc: "2.0",
id: "1",
method: "message/send",
params: {
skill: "smart-routing",
messages: [{ role: "user", content: "Hello" }],
},
}),
});
const { result } = await resp.json();
console.log(result.metadata.routing_explanation);
```

867
API-Reference.md Normal file

@@ -0,0 +1,867 @@
# API Reference
🌐 **Languages:** 🇺🇸 [English](./API_REFERENCE.md) | 🇧🇷 [Português (Brasil)](../i18n/pt-BR/docs/reference/API_REFERENCE.md) | 🇪🇸 [Español](../i18n/es/docs/reference/API_REFERENCE.md) | 🇫🇷 [Français](../i18n/fr/docs/reference/API_REFERENCE.md) | 🇮🇹 [Italiano](../i18n/it/docs/reference/API_REFERENCE.md) | 🇷🇺 [Русский](../i18n/ru/docs/reference/API_REFERENCE.md) | 🇨🇳 [中文 (简体)](../i18n/zh-CN/docs/reference/API_REFERENCE.md) | 🇩🇪 [Deutsch](../i18n/de/docs/reference/API_REFERENCE.md) | 🇮🇳 [हिन्दी](../i18n/in/docs/reference/API_REFERENCE.md) | 🇹🇭 [ไทย](../i18n/th/docs/reference/API_REFERENCE.md) | 🇺🇦 [Українська](../i18n/uk-UA/docs/reference/API_REFERENCE.md) | 🇸🇦 [العربية](../i18n/ar/docs/reference/API_REFERENCE.md) | 🇯🇵 [日本語](../i18n/ja/docs/reference/API_REFERENCE.md) | 🇻🇳 [Tiếng Việt](../i18n/vi/docs/reference/API_REFERENCE.md) | 🇧🇬 [Български](../i18n/bg/docs/reference/API_REFERENCE.md) | 🇩🇰 [Dansk](../i18n/da/docs/reference/API_REFERENCE.md) | 🇫🇮 [Suomi](../i18n/fi/docs/reference/API_REFERENCE.md) | 🇮🇱 [עברית](../i18n/he/docs/reference/API_REFERENCE.md) | 🇭🇺 [Magyar](../i18n/hu/docs/reference/API_REFERENCE.md) | 🇮🇩 [Bahasa Indonesia](../i18n/id/docs/reference/API_REFERENCE.md) | 🇰🇷 [한국어](../i18n/ko/docs/reference/API_REFERENCE.md) | 🇲🇾 [Bahasa Melayu](../i18n/ms/docs/reference/API_REFERENCE.md) | 🇳🇱 [Nederlands](../i18n/nl/docs/reference/API_REFERENCE.md) | 🇳🇴 [Norsk](../i18n/no/docs/reference/API_REFERENCE.md) | 🇵🇹 [Português (Portugal)](../i18n/pt/docs/reference/API_REFERENCE.md) | 🇷🇴 [Română](../i18n/ro/docs/reference/API_REFERENCE.md) | 🇵🇱 [Polski](../i18n/pl/docs/reference/API_REFERENCE.md) | 🇸🇰 [Slovenčina](../i18n/sk/docs/reference/API_REFERENCE.md) | 🇸🇪 [Svenska](../i18n/sv/docs/reference/API_REFERENCE.md) | 🇵🇭 [Filipino](../i18n/phi/docs/reference/API_REFERENCE.md) | 🇨🇿 [Čeština](../i18n/cs/docs/reference/API_REFERENCE.md)
Complete reference for all OmniRoute API endpoints.
---
## Table of Contents
- [Chat Completions](#chat-completions)
- [Embeddings](#embeddings)
- [Image Generation](#image-generation)
- [List Models](#list-models)
- [Compatibility Endpoints](#compatibility-endpoints)
- [Files API](#files-api)
- [Batches API](#batches-api)
- [Search API](#search-api)
- [WebSocket Streaming](#websocket-streaming)
- [Quotas & Issues Reporting](#quotas--issues-reporting)
- [Semantic Cache](#semantic-cache)
- [Dashboard & Management](#dashboard--management)
- [Combo Management](#combo-management)
- [Webhooks](#webhooks)
- [Registered Keys (Auto-Management)](#registered-keys-auto-management)
- [Agents Protocol](#agents-protocol)
- [Management Proxies](#management-proxies)
- [Resilience (extended)](#resilience-extended)
- [Skills](#skills)
- [Memory](#memory)
- [MCP Server](#mcp-server)
- [A2A Server](#a2a-server)
- [Cloud, Evals & Assess](#cloud-evals--assess)
- [Request Processing](#request-processing)
- [Authentication](#authentication)
---
## Chat Completions
```bash
POST /v1/chat/completions
Authorization: Bearer your-api-key
Content-Type: application/json
{
"model": "cc/claude-opus-4-6",
"messages": [
{"role": "user", "content": "Write a function to..."}
],
"stream": true
}
```
### Custom Headers
| Header | Direction | Description |
| ------------------------ | --------- | ------------------------------------------------ |
| `X-OmniRoute-No-Cache` | Request | Set to `true` to bypass cache |
| `X-OmniRoute-Progress` | Request | Set to `true` for progress events |
| `X-Session-Id` | Request | Sticky session key for external session affinity |
| `x_session_id` | Request | Underscore variant also accepted (direct HTTP) |
| `Idempotency-Key` | Request | Dedup key (5s window) |
| `X-Request-Id` | Request | Alternative dedup key |
| `X-OmniRoute-Cache` | Response | `HIT` or `MISS` (non-streaming) |
| `X-OmniRoute-Idempotent` | Response | `true` if deduplicated |
| `X-OmniRoute-Progress` | Response | `enabled` if progress tracking on |
| `X-OmniRoute-Session-Id` | Response | Effective session ID used by OmniRoute |
> Nginx note: if you rely on underscore headers (for example `x_session_id`), enable `underscores_in_headers on;`.
---
## Embeddings
```bash
POST /v1/embeddings
Authorization: Bearer your-api-key
Content-Type: application/json
{
"model": "nebius/Qwen/Qwen3-Embedding-8B",
"input": "The food was delicious"
}
```
Available providers: Nebius, OpenAI, Mistral, Together AI, Fireworks, NVIDIA, **OpenRouter**, **GitHub Models**.
```bash
# List all embedding models
GET /v1/embeddings
```
---
## Image Generation
```bash
POST /v1/images/generations
Authorization: Bearer your-api-key
Content-Type: application/json
{
"model": "openai/gpt-image-2",
"prompt": "A beautiful sunset over mountains",
"size": "1024x1024"
}
```
Available providers: OpenAI (GPT Image 2), xAI (Grok Image), Together AI (FLUX), Fireworks AI, Nebius (FLUX), Hyperbolic, NanoBanana, **OpenRouter**, SD WebUI (local), ComfyUI (local).
```bash
# List all image models
GET /v1/images/generations
```
---
## List Models
```bash
GET /v1/models
Authorization: Bearer your-api-key
→ Returns all chat, embedding, and image models + combos in OpenAI format
```
---
## Compatibility Endpoints
| Method | Path | Format |
| ------ | --------------------------- | ------------------------------- |
| POST | `/v1/chat/completions` | OpenAI |
| POST | `/v1/messages` | Anthropic |
| POST | `/v1/responses` | OpenAI Responses |
| POST | `/v1/embeddings` | OpenAI |
| POST | `/v1/images/generations` | OpenAI Images |
| POST | `/v1/images/edits` | OpenAI Images (edit/inpaint) |
| POST | `/v1/videos/generations` | OpenAI-style video generation |
| POST | `/v1/music/generations` | OpenAI-style music generation |
| POST | `/v1/audio/transcriptions` | OpenAI Audio (STT) |
| POST | `/v1/audio/speech` | OpenAI TTS (returns audio body) |
| POST | `/v1/rerank` | Cohere/Voyage-style rerank |
| POST | `/v1/moderations` | OpenAI Moderations |
| GET | `/v1/models` | OpenAI |
| POST | `/v1/messages/count_tokens` | Anthropic |
| GET | `/v1beta/models` | Gemini |
| POST | `/v1beta/models/{...path}` | Gemini generateContent |
| POST | `/v1/api/chat` | Ollama |
All POST routes follow the same shape: `Bearer your-api-key` + Zod-validated JSON body (`v1RerankSchema`, `v1ModerationSchema`, `v1AudioSpeechSchema`, etc., see `src/shared/validation/schemas.ts`). 4xx is returned on schema failure.
```bash
# Rerank
POST /v1/rerank { "model": "cohere/rerank-3", "query": "...", "documents": ["..."] }
# Moderations
POST /v1/moderations { "model": "omni-moderation-latest", "input": "..." }
# TTS — returns audio/mpeg (or requested format) body
POST /v1/audio/speech { "model": "openai/tts-1", "input": "Hello", "voice": "alloy" }
# Image edit (multipart)
POST /v1/images/edits -F image=@input.png -F prompt="..." -F mask=@mask.png
# Video / music generation (provider-prefixed model id)
POST /v1/videos/generations { "model": "runway/gen-3", "prompt": "..." }
POST /v1/music/generations { "model": "suno/v3.5", "prompt": "..." }
```
### Dedicated Provider Routes
```bash
POST /v1/providers/{provider}/chat/completions
POST /v1/providers/{provider}/embeddings
POST /v1/providers/{provider}/images/generations
```
The provider prefix is auto-added if missing. Mismatched models return `400`.
---
## Files API
OpenAI-compatible files endpoint for batch input/output and file-purpose uploads.
| Method | Path | Description |
| ------ | ------------------------ | ------------------------------------------------------------------------------------------------------------- |
| POST | `/v1/files` | Upload a file (multipart: `file`, `purpose`, `expires_after[anchor]`, `expires_after[seconds]`) — 512 MiB max |
| GET | `/v1/files` | List files for the authenticated API key |
| GET | `/v1/files/[id]` | Retrieve a file's metadata |
| DELETE | `/v1/files/[id]` | Delete a file |
| GET | `/v1/files/[id]/content` | Stream the raw file body back |
**Auth:** Bearer API key — files are scoped per-API-key via `getApiKeyRequestScope`.
---
## Batches API
OpenAI-compatible batch processing.
| Method | Path | Description |
| ------ | ------------------------- | --------------------------------------------------------------------------------------------------------- |
| POST | `/v1/batches` | Create batch — body validated by `v1BatchCreateSchema` (`input_file_id`, `endpoint`, `completion_window`) |
| GET | `/v1/batches` | List batches |
| GET | `/v1/batches/[id]` | Retrieve batch status + `request_counts` |
| DELETE | `/v1/batches/[id]` | Delete a finished/failed batch |
| POST | `/v1/batches/[id]/cancel` | Cancel an in-progress batch |
**Auth:** Bearer API key. Batches are scoped per-API-key.
---
## Search API
Web/search provider abstraction (Tavily, Brave, Exa, Serper, etc.).
| Method | Path | Description |
| ------ | ---------------------- | ------------------------------------------------------------------------------------ |
| GET | `/v1/search` | List configured search providers + capabilities |
| POST | `/v1/search` | Run a search query — body validated by `v1SearchSchema`, supports caching/coalescing |
| GET | `/v1/search/analytics` | Per-provider hit/latency/cache stats |
**Auth:** Bearer API key (`extractApiKey` + `isValidApiKey`). Search policy enforced via `enforceApiKeyPolicy`.
---
## WebSocket Streaming
```bash
GET /v1/ws?handshake=1
```
Validates a WebSocket upgrade handshake and returns the wire protocol example messages (`request`, `cancel`). Actual WS frames are handled by the bundled WS server outside the Next.js route table.
**Auth:** Bearer API key during handshake.
---
## Quotas & Issues Reporting
| Method | Path | Description |
| ------ | ------------------- | ------------------------------------------------------------------------------------- |
| GET | `/v1/quotas/check` | Pre-validate quota for a `provider` + `accountId` before issuing a registered key |
| POST | `/v1/issues/report` | Report a quota/key issuance failure to GitHub (requires `GITHUB_ISSUES_REPO` + token) |
**Auth:** Bearer API key (`isAuthenticated`).
---
## Semantic Cache
```bash
# Get cache stats
GET /api/cache/stats
# Clear all caches
DELETE /api/cache/stats
```
Response example:
```json
{
"semanticCache": {
"memorySize": 42,
"memoryMaxSize": 500,
"dbSize": 128,
"hitRate": 0.65
},
"idempotency": {
"activeKeys": 3,
"windowMs": 5000
}
}
```
---
## Dashboard & Management
### Authentication
| Endpoint | Method | Description |
| ----------------------------- | ------- | --------------------- |
| `/api/auth/login` | POST | Login |
| `/api/auth/logout` | POST | Logout |
| `/api/settings/require-login` | GET/PUT | Toggle login required |
### Provider Management
| Endpoint | Method | Description |
| ---------------------------- | --------------------- | ---------------------------------------------- |
| `/api/providers` | GET/POST | List / create providers |
| `/api/providers/[id]` | GET/PUT/DELETE | Manage a provider |
| `/api/providers/[id]/test` | POST | Test provider connection |
| `/api/providers/[id]/models` | GET | List provider models |
| `/api/providers/validate` | POST | Validate provider config |
| `/api/provider-nodes*` | Various | Provider node management |
| `/api/provider-models` | GET/POST/PATCH/DELETE | Custom models (add, update, hide/show, delete) |
### OAuth Flows
| Endpoint | Method | Description |
| -------------------------------- | ------- | ----------------------- |
| `/api/oauth/[provider]/[action]` | Various | Provider-specific OAuth |
### Routing & Config
| Endpoint | Method | Description |
| --------------------- | -------- | ----------------------------- |
| `/api/models/alias` | GET/POST | Model aliases |
| `/api/models/catalog` | GET | All models by provider + type |
| `/api/combos*` | Various | Combo management |
| `/api/keys*` | Various | API key management |
| `/api/pricing` | GET | Model pricing |
### Usage & Analytics
| Endpoint | Method | Description |
| --------------------------- | ------ | -------------------- |
| `/api/usage/history` | GET | Usage history |
| `/api/usage/logs` | GET | Usage logs |
| `/api/usage/request-logs` | GET | Request-level logs |
| `/api/usage/[connectionId]` | GET | Per-connection usage |
### Settings
| Endpoint | Method | Description |
| ------------------------------- | ------------- | ------------------------- |
| `/api/settings` | GET/PUT/PATCH | General settings |
| `/api/settings/proxy` | GET/PUT | Network proxy config |
| `/api/settings/proxy/test` | POST | Test proxy connection |
| `/api/settings/ip-filter` | GET/PUT | IP allowlist/blocklist |
| `/api/settings/thinking-budget` | GET/PUT | Reasoning token budget |
| `/api/settings/system-prompt` | GET/PUT | Global system prompt |
| `/api/settings/compression` | GET/PUT | Global compression config |
### Context & Compression
| Endpoint | Method | Description |
| -------------------------------------- | -------------- | ------------------------------------------------------------------------ |
| `/api/compression/preview` | POST | Preview off/lite/standard/aggressive/ultra/RTK/stacked compression |
| `/api/compression/language-packs` | GET | List available Caveman language packs |
| `/api/compression/rules` | GET | List Caveman rule metadata |
| `/api/context/caveman/config` | GET/PUT | Caveman-specific settings alias |
| `/api/context/rtk/config` | GET/PUT | RTK-specific settings, including custom filters and raw-output retention |
| `/api/context/rtk/filters` | GET | RTK filter catalog and custom-filter diagnostics |
| `/api/context/rtk/test` | POST | Run RTK preview/test against a text payload |
| `/api/context/rtk/raw-output/[id]` | GET | Read retained redacted raw output by pointer id |
| `/api/context/combos` | GET/POST | Compression combo list/create |
| `/api/context/combos/[id]` | GET/PUT/DELETE | Compression combo detail/update/delete |
| `/api/context/combos/[id]/assignments` | GET/PUT | Assign compression combos to routing combos |
| `/api/context/analytics` | GET | Compression analytics alias |
### Monitoring
| Endpoint | Method | Description |
| ------------------------ | ---------- | ---------------------------------------------------------------------------------------------------- |
| `/api/sessions` | GET | Active session tracking |
| `/api/rate-limits` | GET | Per-account rate limits |
| `/api/monitoring/health` | GET | Health check + provider summary (`catalogCount`, `configuredCount`, `activeCount`, `monitoredCount`) |
| `/api/cache/stats` | GET/DELETE | Cache stats / clear |
### Backup & Export/Import
| Endpoint | Method | Description |
| --------------------------- | ------ | --------------------------------------- |
| `/api/db-backups` | GET | List available backups |
| `/api/db-backups` | PUT | Create a manual backup |
| `/api/db-backups` | POST | Restore from a specific backup |
| `/api/db-backups/export` | GET | Download database as .sqlite file |
| `/api/db-backups/import` | POST | Upload .sqlite file to replace database |
| `/api/db-backups/exportAll` | GET | Download full backup as .tar.gz archive |
### Cloud Sync
| Endpoint | Method | Description |
| ---------------------- | ------- | --------------------- |
| `/api/sync/cloud` | Various | Cloud sync operations |
| `/api/sync/initialize` | POST | Initialize sync |
| `/api/cloud/*` | Various | Cloud management |
### Tunnels
| Endpoint | Method | Description |
| -------------------------- | ------ | ----------------------------------------------------------------------- |
| `/api/tunnels/cloudflared` | GET | Read Cloudflare Quick Tunnel install/runtime status for the dashboard |
| `/api/tunnels/cloudflared` | POST | Enable or disable the Cloudflare Quick Tunnel (`action=enable/disable`) |
| `/api/tunnels/ngrok` | GET | Read ngrok Tunnel runtime status for the dashboard |
| `/api/tunnels/ngrok` | POST | Enable or disable the ngrok Tunnel (`action=enable/disable`) |
### CLI Tools
| Endpoint | Method | Description |
| ---------------------------------- | ------ | ------------------- |
| `/api/cli-tools/claude-settings` | GET | Claude CLI status |
| `/api/cli-tools/codex-settings` | GET | Codex CLI status |
| `/api/cli-tools/droid-settings` | GET | Droid CLI status |
| `/api/cli-tools/openclaw-settings` | GET | OpenClaw CLI status |
| `/api/cli-tools/runtime/[toolId]` | GET | Generic CLI runtime |
CLI responses include: `installed`, `runnable`, `command`, `commandPath`, `runtimeMode`, `reason`.
### ACP Agents
| Endpoint | Method | Description |
| ----------------- | ------ | -------------------------------------------------------- |
| `/api/acp/agents` | GET | List all detected agents (built-in + custom) with status |
| `/api/acp/agents` | POST | Add custom agent or refresh detection cache |
| `/api/acp/agents` | DELETE | Remove a custom agent by `id` query param |
GET response includes `agents[]` (id, name, binary, version, installed, protocol, isCustom) and `summary` (total, installed, notFound, builtIn, custom).
### Resilience & Rate Limits
| Endpoint | Method | Description |
| --------------------------------- | --------- | ------------------------------------------------------------------------------------ |
| `/api/resilience` | GET/PATCH | Get/update request queue, connection cooldown, provider breaker, and wait settings |
| `/api/resilience/reset` | POST | Reset provider circuit breakers |
| `/api/resilience/model-cooldowns` | GET | List active per-(provider, connection, model) lockouts, sorted by remaining time |
| `/api/resilience/model-cooldowns` | DELETE | Clear a model lockout — body `{provider, model}` or `{all: true}` to wipe everything |
| `/api/rate-limits` | GET | Per-account rate limit status |
| `/api/rate-limit` | GET | Global rate limit configuration |
> All four `/api/resilience/*` routes require **management auth** (`requireManagementAuth`). See [Resilience (extended)](#resilience-extended) for a full breakdown of provider breaker vs connection cooldown vs model lockout.
### Evals
| Endpoint | Method | Description |
| ------------ | -------- | --------------------------------- |
| `/api/evals` | GET/POST | List eval suites / run evaluation |
### Policies
| Endpoint | Method | Description |
| --------------- | --------------- | ----------------------- |
| `/api/policies` | GET/POST/DELETE | Manage routing policies |
### Compliance
| Endpoint | Method | Description |
| --------------------------- | ------ | ----------------------------- |
| `/api/compliance/audit-log` | GET | Compliance audit log (last N) |
### v1beta (Gemini-Compatible)
| Endpoint | Method | Description |
| -------------------------- | ------ | --------------------------------- |
| `/v1beta/models` | GET | List models in Gemini format |
| `/v1beta/models/{...path}` | POST | Gemini `generateContent` endpoint |
These endpoints mirror Gemini's API format for clients that expect native Gemini SDK compatibility.
### Internal / System APIs
| Endpoint | Method | Description |
| ------------------------ | ------ | ---------------------------------------------------- |
| `/api/init` | GET | Application initialization check (used on first run) |
| `/api/tags` | GET | Ollama-compatible model tags (for Ollama clients) |
| `/api/restart` | POST | Trigger graceful server restart |
| `/api/shutdown` | POST | Trigger graceful server shutdown |
| `/api/system/env/repair` | POST | Repair OAuth provider environment variables |
| `/api/system-info` | GET | Generate system diagnostics report |
> **Note:** These endpoints are used internally by the system or for Ollama client compatibility. They are not typically called by end users.
### OAuth Environment Repair _(v3.6.1+)_
```bash
POST /api/system/env/repair
Content-Type: application/json
{
"provider": "claude-code"
}
```
Repairs missing or corrupted OAuth environment variables for a specific provider. Returns:
```json
{
"success": true,
"repaired": ["CLAUDE_CODE_OAUTH_CLIENT_ID", "CLAUDE_CODE_OAUTH_CLIENT_SECRET"],
"backupPath": "/home/user/.omniroute/backups/env-repair-2026-04-11.bak"
}
```
---
## Audio Transcription
```bash
POST /v1/audio/transcriptions
Authorization: Bearer your-api-key
Content-Type: multipart/form-data
```
Transcribe audio files using Deepgram or AssemblyAI.
**Request:**
```bash
curl -X POST http://localhost:20128/v1/audio/transcriptions \
-H "Authorization: Bearer your-api-key" \
-F "file=@recording.mp3" \
-F "model=deepgram/nova-3"
```
**Response:**
```json
{
"text": "Hello, this is the transcribed audio content.",
"task": "transcribe",
"language": "en",
"duration": 12.5
}
```
**Supported providers:** `deepgram/nova-3`, `assemblyai/best`.
**Supported formats:** `mp3`, `wav`, `m4a`, `flac`, `ogg`, `webm`.
---
## Ollama Compatibility
For clients that use Ollama's API format:
```bash
# Chat endpoint (Ollama format)
POST /v1/api/chat
# Model listing (Ollama format)
GET /api/tags
```
Requests are automatically translated between Ollama and internal formats.
---
## Telemetry
```bash
# Get latency telemetry summary (p50/p95/p99 per provider)
GET /api/telemetry/summary
```
**Response:**
```json
{
"providers": {
"claudeCode": { "p50": 245, "p95": 890, "p99": 1200, "count": 150 },
"github": { "p50": 180, "p95": 620, "p99": 950, "count": 320 }
}
}
```
---
## Budget
```bash
# Get budget status for all API keys
GET /api/usage/budget
# Set or update a budget
POST /api/usage/budget
Content-Type: application/json
{
"apiKeyId": "key-123",
"dailyLimitUsd": 5.00,
"weeklyLimitUsd": 30.00,
"monthlyLimitUsd": 100.00,
"warningThreshold": 0.8,
"resetInterval": "monthly"
}
```
> **Schema notes** (`setBudgetSchema`): `apiKeyId` is required; at least one of `dailyLimitUsd`, `weeklyLimitUsd`, or `monthlyLimitUsd` must be greater than zero. Optional fields: `warningThreshold` (0–1), `resetInterval` (`daily` | `weekly` | `monthly`), `resetTime` (`HH:MM`). The legacy `{keyId, limit, period}` shape returns `400 Bad Request`.
## Request Processing
1. Client sends request to `/v1/*`
2. Route handler calls `handleChat`, `handleEmbedding`, `handleAudioTranscription`, or `handleImageGeneration`
3. Model is resolved (direct provider/model or alias/combo)
4. Credentials selected from local DB with account availability filtering
5. For chat: `handleChatCore` checks semantic/signature cache and resolves combo compression settings
6. Proactive compression runs before provider translation when enabled (`lite`, Caveman, RTK, or stacked)
7. Provider executor sends upstream request
8. Response translated back to client format (chat) or returned as-is (embeddings/images/audio)
9. Usage, compression analytics, and request logs are recorded
10. Fallback applies on errors according to combo rules
Full architecture reference: [`ARCHITECTURE.md`](../architecture/ARCHITECTURE.md)
---
## Combo Management
Higher-level routing combos (already summarized under `/api/combos*`) can also be mapped 1:1 from a model id pattern, allowing transparent redirection of an OpenAI-style model id to a combo.
| Method | Path | Description |
| ------ | -------------------------------- | ------------------------------------------------------------------------------ |
| GET | `/api/model-combo-mappings` | List all model→combo mappings |
| POST | `/api/model-combo-mappings` | Create mapping — body: `{pattern, comboId, priority?, enabled?, description?}` |
| GET | `/api/model-combo-mappings/[id]` | Retrieve a single mapping |
| PUT | `/api/model-combo-mappings/[id]` | Update fields of an existing mapping |
| DELETE | `/api/model-combo-mappings/[id]` | Remove a mapping |
**Auth:** management session/API key (`requireManagementAuth`).
---
## Webhooks
Outbound webhook subscriptions for OmniRoute events (request completion, quota exhaustion, key rotation, etc.).
| Method | Path | Description |
| ------ | ------------------------- | --------------------------------------------------------------------- |
| GET | `/api/webhooks` | List webhooks (secrets are masked to `<prefix>...`) |
| POST | `/api/webhooks` | Create webhook — body: `{url, events?: ["*"], secret?, description?}` |
| GET | `/api/webhooks/[id]` | Retrieve a webhook |
| PUT | `/api/webhooks/[id]` | Update url/events/secret/description |
| DELETE | `/api/webhooks/[id]` | Remove a webhook |
| POST | `/api/webhooks/[id]/test` | Send a test payload to the webhook URL and return delivery status |
**Auth:** management session/API key (`requireManagementAuth`).
---
## Registered Keys (Auto-Management)
Used by the auto-key management subsystem to issue and rotate API keys against a backing provider/account, with daily/hourly quotas.
| Method | Path | Description |
| ------ | ------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| GET | `/api/v1/registered-keys` | List registered keys (masked prefix only) |
| POST | `/api/v1/registered-keys` | Issue a new registered key — body: `{name, provider?, accountId?, idempotencyKey?, expiresAt?, dailyBudget?, hourlyBudget?}`. Returns the raw key **once**. Returns `429` on quota refusal. |
| GET | `/api/v1/registered-keys/[id]` | Retrieve a registered key's metadata (no raw material) |
| DELETE | `/api/v1/registered-keys/[id]` | Revoke a registered key |
| POST | `/api/v1/registered-keys/[id]/revoke` | Explicit revoke endpoint (same effect as DELETE) |
**Auth:** Bearer API key (`isAuthenticated`). See also `/v1/quotas/check` and `/v1/issues/report`.
---
## Agents Protocol
Cloud agent tasks (Claude Code, Codex Cloud, OpenHands, etc.) executed remotely on behalf of OmniRoute users.
| Method | Path | Description |
| ------ | ----------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------- |
| GET | `/api/v1/agents/tasks` | List tasks — optional `?provider=`, `?status=`, `?limit=` (1–500, default 50) |
| POST | `/api/v1/agents/tasks` | Create task — body validated by `CreateCloudAgentTaskSchema` (`providerId`, `prompt`, `source`, `options?`). Returns `201` with task envelope |
| DELETE | `/api/v1/agents/tasks?id=...` | Delete a task |
| GET | `/api/v1/agents/tasks/[id]` | Read task — synchronously refreshes status from the upstream cloud agent when an `external_id` is set |
| POST | `/api/v1/agents/tasks/[id]` | Discriminated action: `{action: "approve"}`, `{action: "message", message}`, or `{action: "cancel"}` |
| DELETE | `/api/v1/agents/tasks/[id]` | Delete a specific task by id |
> **Auth:** management auth required on every method (`requireCloudAgentManagementAuth`). Prior to v3.8.0 these were unauthenticated — see commit `588a0333` for the breaking change.
```bash
# Create a Claude Code cloud task
curl -X POST http://localhost:20128/api/v1/agents/tasks \
-H "Authorization: Bearer your-management-key" \
-H "Content-Type: application/json" \
-d '{"providerId":"claude-code-cloud","prompt":"Fix the failing test","source":{"repo":"...","branch":"..."}}'
```
---
## Management Proxies
Outbound HTTP(S)/SOCKS proxies that can be assigned to providers, accounts, or globally.
| Method | Path | Description |
| ------ | -------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------ |
| GET | `/api/v1/management/proxies` | List proxies (with `?id=` returns one; with `?id=&where_used=1` returns the assignment graph) |
| POST | `/api/v1/management/proxies` | Create proxy — body validated by `createProxyRegistrySchema` |
| PATCH | `/api/v1/management/proxies` | Update proxy — body validated by `updateProxyRegistrySchema` (requires `id`) |
| DELETE | `/api/v1/management/proxies?id=...&force=1` | Delete proxy (use `force=1` to detach assignments) |
| GET | `/api/v1/management/proxies/assignments` | List assignments — filterable by `proxy_id`, `scope`, `scope_id`; pass `resolve_connection_id=<id>` to resolve the active proxy for a connection |
| PUT | `/api/v1/management/proxies/assignments` | Assign — body validated by `proxyAssignmentSchema` (`{scope, scopeId?, proxyId?}`). Clears dispatcher cache |
| PUT | `/api/v1/management/proxies/bulk-assign` | Bulk-assign — body validated by `bulkProxyAssignmentSchema` (`{scope, scopeIds[], proxyId?}`) |
| GET | `/api/v1/management/proxies/health?hours=24` | Aggregate proxy health (success/fail counts, latency) over a window |
**Auth:** management session/API key on every route (`requireManagementAuth`).
> The task description's `POST /api/v1/management/proxies/[id]/assignments` and `POST /api/v1/management/proxies/[id]/health` are served by the flat `/assignments` and `/health` routes shown above — there are no per-id subroutes in the codebase.
---
## Resilience (extended)
OmniRoute exposes three independent temporary-failure mechanisms; the management endpoints below let operators read and override them:
| Scope | State storage | Read | Reset / clear |
| ------------------- | ------------------------------------------ | ----------------------------------------- | ------------------------------------------- |
| Provider breaker | `domain_circuit_breakers` + in-memory | `/api/monitoring/health` | `POST /api/resilience/reset` |
| Connection cooldown | `rateLimitedUntil` on provider connections | `/api/rate-limits`, `/api/providers/[id]` | (re-enables lazily; clear via provider PUT) |
| Model lockout | In-memory model-availability registry | `GET /api/resilience/model-cooldowns` | `DELETE /api/resilience/model-cooldowns` |
```bash
# Clear a single model lockout
curl -X DELETE http://localhost:20128/api/resilience/model-cooldowns \
-H "Cookie: auth_token=..." \
-H "Content-Type: application/json" \
-d '{"provider":"openai","model":"gpt-4o-mini"}'
# Wipe every lockout
curl -X DELETE http://localhost:20128/api/resilience/model-cooldowns \
-H "Cookie: auth_token=..." \
-d '{"all":true}'
```
Full conceptual reference and breaker defaults: see [`CLAUDE.md`](../../CLAUDE.md) → "Resilience Runtime State".
---
## Skills
Skill framework for extending OmniRoute with custom executable handlers, plus marketplace integrations.
| Method | Path | Description |
| ------ | --------------------------------- | -------------------------------------------------------------------------------------------------------------------------- |
| GET | `/api/skills` | List installed skills — filterable by `?q=`, `?mode=on\|off\|auto`, `?source=skillsmp\|skillssh\|local`, paginated |
| GET | `/api/skills/[id]` | Retrieve one skill |
| PUT | `/api/skills/[id]` | Update skill (name, description, mode, schema, handler, tags) |
| DELETE | `/api/skills/[id]` | Uninstall a skill |
| POST | `/api/skills/install` | Install a skill from a raw manifest — body: `{name, version, description, schema:{input, output}, handlerCode, apiKeyId?}` |
| GET | `/api/skills/executions` | List recent skill executions (audit trail with inputs/outputs/duration) |
| GET | `/api/skills/marketplace?q=...` | Search/popular list from the SkillsMP marketplace (requires `skillsmpApiKey` setting) |
| POST | `/api/skills/marketplace/install` | Install a skill by id from SkillsMP |
| GET | `/api/skills/skillssh?q=&limit=` | Search the skills.sh registry |
| POST | `/api/skills/skillssh/install` | Install a skill by id from skills.sh |
**Auth:** management session/API key. Marketplace search routes accept either management auth or a Bearer API key (`isAuthenticated`).
---
## Memory
Persistent conversational/factual memory store, scoped per API key / session.
| Method | Path | Description |
| ------ | -------------------- | ------------------------------------------------------------------------------------------------------------ |
| GET | `/api/memory` | List memories — `?apiKeyId=`, `?type=`, `?sessionId=`, `?q=`, with `offset/limit` or `page/limit` pagination |
| POST | `/api/memory` | Create memory — body validated by Zod: `{content, key, type?, sessionId?, apiKeyId?, metadata?, expiresAt?}` |
| GET | `/api/memory/[id]` | Retrieve one memory |
| DELETE | `/api/memory/[id]` | Delete a memory |
| GET | `/api/memory/health` | Memory subsystem health (DB connectivity, embeddings backend, vector index status) |
**Auth:** management session/API key (`requireManagementAuth`). `type` enum: `FACTUAL`, `EPISODIC`, `SEMANTIC`, `PROCEDURAL` (see `MemoryType` in `src/lib/memory/types.ts`).
---
## MCP Server
OmniRoute ships an embedded Model Context Protocol server with 3 transports (stdio, SSE, streamable-http) and scoped tools. The dashboard endpoints below read status/audit data and proxy the HTTP transports.
| Method | Path | Description |
| ------ | ---------------------- | ------------------------------------------------------------------------------------------------ | -------------------- |
| GET | `/api/mcp/status` | Heartbeat, transport, online state, last call, top tools, 24h success rate |
| GET | `/api/mcp/tools` | List of MCP tools with `name`, `description`, `scopes`, `phase`, `auditLevel`, `sourceEndpoints` |
| GET | `/api/mcp/sse` | Open SSE stream for the SSE transport (returns `503` if MCP disabled or transport mismatch) |
| POST | `/api/mcp/sse` | Send JSON-RPC frame on the SSE transport |
| GET | `/api/mcp/stream` | Open SSE side of the Streamable HTTP transport (server-initiated messages) |
| POST | `/api/mcp/stream` | Send JSON-RPC frame on the Streamable HTTP transport |
| DELETE | `/api/mcp/stream` | End a Streamable HTTP session |
| GET | `/api/mcp/audit` | Query audit log — `?limit=`, `?offset=`, `?tool=`, `?success=true | false`, `?apiKeyId=` |
| GET | `/api/mcp/audit/stats` | Aggregate audit stats (totals, success rate, avg duration, top tools) |
**Auth:** the `sse`/`stream` transports honor the MCP-specific auth surface (Bearer API key with `mcp` scope); the `status`/`tools`/`audit*` routes are readable from the dashboard (no extra auth required beyond reaching the dashboard host).
> Both HTTP transports are gated by `settings.mcpEnabled` and `settings.mcpTransport` — a transport mismatch returns `400`, an MCP disabled state returns `503`.
---
## A2A Server
OmniRoute exposes an A2A (Agent-to-Agent) JSON-RPC 2.0 endpoint plus a REST wrapper for inspection/dashboard use.
### JSON-RPC
```bash
POST /a2a
Authorization: Bearer your-api-key # optional unless OMNIROUTE_API_KEY is set
Content-Type: application/json
{
"jsonrpc": "2.0",
"id": 1,
"method": "message/send",
"params": {
"skill": "smart-routing",
"messages": [{"role": "user", "content": "Route this coding task"}]
}
}
```
Supported methods (all gated on `settings.a2aEnabled`):
| Method | Description |
| ---------------- | ------------------------------------------------------------------ |
| `message/send` | Synchronous skill execution; returns `{task, artifacts, metadata}` |
| `message/stream` | Streaming SSE execution of the same skill set |
| `tasks/get` | Fetch a task by `taskId` |
| `tasks/cancel` | Cancel a task by `taskId` |
Built-in skills: `smart-routing`, `quota-management`, `provider-discovery`, `cost-analysis`, `health-report`.
### Agent Card
```bash
GET /.well-known/agent.json
```
Returns the public A2A agent card (name, description, capabilities, skill catalog, auth scheme) — cached publicly for 1h. No auth required.
### REST helpers
| Method | Path | Description |
| ------ | ---------------------------- | --------------------------------------------------------------------------------------------------------------- |
| GET | `/api/a2a/status` | A2A enabled + task stats + cached agent card summary |
| GET | `/api/a2a/tasks` | List tasks — `?state=submitted\|working\|completed\|failed\|cancelled`, `?skill=`, `?limit=` (≤200), `?offset=` |
| POST | `/api/a2a/tasks` | (Not implemented as a REST helper — create via JSON-RPC `message/send`) |
| GET | `/api/a2a/tasks/[id]` | Retrieve one task |
| POST | `/api/a2a/tasks/[id]/cancel` | Cancel a task |
**Auth:** the REST helpers run without management auth (dashboard-readable); the JSON-RPC `/a2a` route uses Bearer `OMNIROUTE_API_KEY` if configured.
---
## Cloud, Evals & Assess
| Method | Path | Description |
| ------ | ------------------------------- | ------------------------------------------------------------------------------------------------- | ----------------------------- | ----------------------------------- |
| POST | `/api/cloud/auth` | Verify a Bearer key and return masked provider connections + model aliases for cloud sync clients |
| POST | `/api/cloud/credentials/update` | Update encrypted credentials for a cloud-synced provider |
| POST | `/api/cloud/model/resolve` | Resolve a logical model id to a concrete provider/model using the local routing table |
| GET | `/api/cloud/models/alias` | List model aliases as exposed to cloud sync |
| GET | `/api/assess` | Read latest assessment categorizations (per-provider/model) |
| POST | `/api/assess` | Run an assessment — body: `{scope: {type:"all"} | {type:"provider", providerId} | {type:"model", modelId}, trigger?}` |
| GET | `/api/evals` | List built-in eval suites + most recent runs |
| POST | `/api/evals` | Trigger an eval run |
| POST | `/api/evals/suites` | Create a custom eval suite — body validated by `evalSuiteSaveSchema` |
| GET | `/api/evals/suites/[id]` | Retrieve a custom eval suite |
**Auth:** `/api/cloud/auth` validates a Bearer key directly; the other `/api/cloud/*`, `/api/evals/*`, and `/api/assess` routes require management session/API key. `/api/assess` POST uses `validateBody` with a discriminated-union scope schema.
---
## Authentication
- Dashboard routes (`/dashboard/*`) use `auth_token` cookie
- Login uses saved password hash; fallback to `INITIAL_PASSWORD`
- `requireLogin` toggleable via `/api/settings/require-login`
- `/v1/*` routes optionally require Bearer API key when `REQUIRE_API_KEY=true`
> **Breaking change (v3.8.0)** — `/api/v1/agents/tasks/*` and the cooldown management endpoints now require **management auth** (dashboard `auth_token` cookie or a management-scoped API key). Clients that previously called these routes unauthenticated will receive `401 Unauthorized`. See commit `588a0333` (`fix(auth): require management auth for agent and cooldown APIs`).

277
Agent-Protocols-Guide.md Normal file

@@ -0,0 +1,277 @@
# Agent Protocols Guide
> **Source:** `src/lib/{a2a,acp,cloudAgent}/`, `src/app/api/{a2a,acp,cloud}/`, `src/app/api/v1/agents/`
> **Last updated:** 2026-05-13 — v3.8.0
OmniRoute exposes three different agent-related surfaces. They look similar at first glance but solve different problems. Use this page to pick the right one.
## TL;DR
| Surface | Best for | Transport | Standard |
| ----------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------ | --------------------------- | -------------------- |
| **A2A — Agent-to-Agent** | Cross-agent collaboration with peer agents that speak the A2A protocol | JSON-RPC 2.0 over HTTP | A2A v0.3 (open spec) |
| **ACP — CLI Agents Registry** | Detecting / registering / launching CLI coding agents installed on the user's machine (Cursor, Cline, Codex CLI, Claude Code, Aider, etc.) | HTTP REST | OmniRoute-specific |
| **Cloud Agents** | Submitting long-running coding tasks to external cloud services (Codex Cloud, Devin, Jules) | HTTP REST + DB-backed tasks | OmniRoute-specific |
The three are independent — pick any subset.
## Decision Tree
```
Do you need a cloud service to do work outside this machine (Codex Cloud / Devin / Jules)?
├─ YES → Cloud Agents (POST /api/v1/agents/tasks)
└─ NO → Continue
│
Do you have a peer agent that speaks A2A and wants to collaborate?
├─ YES → A2A (POST /a2a)
└─ NO → Continue
│
Do you need to list / configure CLI coding agents installed locally?
├─ YES → ACP (GET /api/acp/agents)
└─ NO → Use plain /v1/chat/completions
```
## 1. A2A — Agent-to-Agent
**Spec:** [A2A v0.3](https://a2a-protocol.org)
**OmniRoute endpoint:** `POST /a2a` (JSON-RPC 2.0)
**Agent Card:** `GET /.well-known/agent.json`
### When to use
- Building a multi-agent system where OmniRoute is one of the peers
- Exposing OmniRoute's routing intelligence (smart-routing, quota-management, etc.) to agents in frameworks like Google ADK or generic agent meshes
- Wrapping OmniRoute behind a standard discovery + invocation surface
### Methods
- `message/send` — submit a message, receive sync response
- `message/stream` — submit + receive SSE-streamed progress events
- `tasks/get` — read task by ID
- `tasks/cancel` — cancel a running task
### Built-in skills (5)
- `smart-routing` — route a prompt through the optimal combo
- `quota-management` — report per-provider quota state
- `provider-discovery` — list installed providers with capabilities
- `cost-analysis` — estimate cost of a request/conversation
- `health-report` — aggregate breaker/cooldown/lockout state per provider
### Deep dive
See [A2A-SERVER.md](./A2A-SERVER.md) for transport details, agent card structure, task TTL config, and the template for adding new skills.
## 2. ACP — CLI Agents Registry
**OmniRoute endpoint:** `GET /api/acp/agents`
**Source:** `src/lib/acp/{index,manager,registry}.ts`
### What it is
ACP is OmniRoute's **local CLI agent inventory**. It detects which coding CLIs are installed on the host (Cursor, Cline, Claude Code, Codex CLI, Continue, etc.), resolves their versions, and surfaces them to the dashboard so the user can wire each CLI to point at OmniRoute.
This is NOT an external protocol — it's an internal registry that powers the "CLI Tools" UI and the CLI fingerprint tracking (see [CLI-TOOLS.md](../reference/CLI-TOOLS.md)).
### What it does
- Probes the host for installed CLI binaries (uses `which` / `where` per OS)
- Reads each CLI's version (calls `<bin> --version`)
- Optionally accepts user-defined custom agents (binary path + version probe + spawn args)
- Persists custom agents in settings
- Returns the unified list to the dashboard
### REST API
| Endpoint | Method | Description | Auth |
| ----------------- | ------ | ------------------------------------------------------------- | ------- |
| `/api/acp/agents` | GET | List detected + custom agents (installed/total counts) | API key |
| `/api/acp/agents` | POST | Add/update/remove custom agent (action discriminator in body) | API key |
Body shape for POST (`customAgentBodySchema` in `src/app/api/acp/agents/route.ts`):
```json
{
"action": "add|update|remove",
"id": "cursor",
"name": "Cursor",
"binary": "/usr/local/bin/cursor",
"versionCommand": "--version",
"providerAlias": "cursor",
"spawnArgs": ["--api-base", "http://localhost:20128"],
"protocol": "stdio"
}
```
### Use cases
- Dashboard "CLI Tools" page lists what's installed and helps you point each at OmniRoute
- Custom agents let power users register internal/proprietary CLIs that OmniRoute doesn't know about by default
- Detection result fuels the `cli-tools` fingerprint matrix
### When NOT to use ACP
- ACP doesn't _run_ tasks. It only detects + configures CLIs. To actually invoke a CLI, you launch it yourself with the env vars OmniRoute provides (`OPENAI_BASE_URL`, `OPENAI_API_KEY`, etc.).
## 3. Cloud Agents
**OmniRoute endpoints:** `/api/v1/agents/tasks/*` (lifecycle) + `/api/cloud/*` (plumbing)
**Source:** `src/lib/cloudAgent/`
### What it is
A uniform interface over third-party cloud coding agents. You submit a prompt + repo URL, OmniRoute dispatches to the right cloud agent, polls status, returns results.
### Supported agents (3, all confirmed in `src/lib/cloudAgent/agents/`)
- `codex-cloud` — OpenAI Codex Cloud
- `devin` — Cognition Devin
- `jules` — Google Jules
### Lifecycle
```
POST /api/v1/agents/tasks
→ BaseAgent.createTask() per agent class
→ external service starts work
→ task row created in DB (cloud_agent_tasks)
↓
GET /api/v1/agents/tasks/[id]
→ lazy status sync from provider
→ returns current status + plan + activity log
↓
POST /api/v1/agents/tasks/[id] (action: "approve" | "message" | "cancel")
→ forwards to provider (or marks cancelled locally)
↓
DELETE /api/v1/agents/tasks/[id]
→ local cancel
```
### Auth
⚠️ **All `/api/v1/agents/tasks/*` endpoints require management auth** (commit `588a0333`). Bearer-only callers receive 401 since v3.8.0.
### Deep dive
See [CLOUD_AGENT.md](./CLOUD_AGENT.md) for the `CloudAgentBase` contract, per-agent specifics, schema details, and credential plumbing endpoints.
## Comparison: A2A vs Cloud Agents
Both have "long-running tasks" but at different layers:
| Aspect | A2A | Cloud Agents |
| ------------------ | --------------------------------------------------------------------------------- | ---------------------------------------- |
| Standard | Open A2A v0.3 | OmniRoute-specific |
| Where compute runs | Inside OmniRoute (uses configured combos) | External (Codex / Devin / Jules servers) |
| Task duration | Default TTL 5 min (configurable in `TaskManager`) | Minutes to hours |
| Repo-aware | No (passes prompts only) | Yes (repo URL + branch) |
| Use case | Cross-agent collab, smart routing as a service | Delegate "implement feature X in repo Y" |
| Auth | Optional `OMNIROUTE_API_KEY` for `/a2a`; management for `/api/a2a/*` REST helpers | Always management |
## Integration Examples
### Discover OmniRoute's A2A capabilities
```bash
curl http://localhost:20128/.well-known/agent.json
```
Returns the Agent Card with all 5 skills, transports, and version.
### Call OmniRoute as an A2A agent
```bash
curl -X POST http://localhost:20128/a2a \
-H "Content-Type: application/json" \
-d '{
"jsonrpc": "2.0",
"method": "message/send",
"params": {
"messages": [{"role": "user", "content": "Route this prompt"}],
"skillId": "smart-routing"
},
"id": 1
}'
```
### List installed CLI agents via ACP
```bash
curl http://localhost:20128/api/acp/agents \
-H "Authorization: Bearer <api-key>"
```
### Add a custom CLI agent
```bash
curl -X POST http://localhost:20128/api/acp/agents \
-H "Authorization: Bearer <api-key>" \
-H "Content-Type: application/json" \
-d '{
"action": "add",
"id": "my-custom-cli",
"name": "My Custom CLI",
"binary": "/opt/mycli/bin/mycli",
"versionCommand": "--version",
"providerAlias": "openai"
}'
```
### Submit a Cloud Agent task
```bash
curl -X POST http://localhost:20128/api/v1/agents/tasks \
-H "Cookie: auth_token=..." \
-H "Content-Type: application/json" \
-d '{
"agentId": "devin",
"prompt": "Implement feature X in repo Y",
"repo": "https://github.com/user/repo",
"branch": "main"
}'
```
### Poll cloud task status
```bash
curl http://localhost:20128/api/v1/agents/tasks/<task-id> \
-H "Cookie: auth_token=..."
```
## When to Use What
- **Chatbot / copilot frontend** → `/v1/chat/completions` (OpenAI-compat — not an agent protocol)
- **Multi-agent collaboration** → A2A
- **Listing local CLIs in the dashboard** → ACP
- **Delegating long-running coding tasks to cloud services** → Cloud Agents
## Internal Architecture
```
┌─────────────────────┐
│ OmniRoute Core │
└─────────────────────┘
↑ ↑ ↑
┌─────────┘ │ └─────────┐
│ │ │
┌───────┐ ┌─────────┐ ┌────────────┐
│ A2A │ │ ACP │ │ Cloud │
│ (/a2a)│ │ (/acp) │ │ Agents │
└───────┘ └─────────┘ │ (/v1/agents│
│ │ │ /tasks) │
↓ ↓ └────────────┘
External peer Local CLI │
agents that binaries on ↓
speak A2A v0.3 the host Codex Cloud,
Devin, Jules
```
## See Also
- [A2A-SERVER.md](./A2A-SERVER.md) — A2A deep dive
- [CLOUD_AGENT.md](./CLOUD_AGENT.md) — Cloud Agents deep dive
- [CLI-TOOLS.md](../reference/CLI-TOOLS.md) — External CLI integrations (uses ACP)
- [SKILLS.md](./SKILLS.md) — Skills framework (different from A2A skills — local execution sandbox)
- [API_REFERENCE.md](../reference/API_REFERENCE.md#agents-protocol) — endpoint reference
- Source: `src/lib/{a2a,acp,cloudAgent}/`

1139
Architecture.md Normal file

File diff suppressed because it is too large Load Diff

210
Authz-Guide.md Normal file

@@ -0,0 +1,210 @@
# Authorization Guide
> **Source of truth:** `src/server/authz/`, `src/shared/constants/publicApiRoutes.ts`, `src/lib/api/requireManagementAuth.ts`, `src/shared/utils/apiAuth.ts`
> **Last updated:** 2026-05-13 — v3.8.0
OmniRoute has a route-aware authorization pipeline that gates every API request. Classification is **deterministic** and **fail-closed** — anything that cannot be classified ends up as `MANAGEMENT` and demands a session or management-grade token. This page explains the model for engineers maintaining routes or designing new endpoints.
![AuthZ pipeline (3 route classes + policy evaluation)](../diagrams/exported/authz-pipeline.svg)
> Source: [diagrams/authz-pipeline.mmd](../diagrams/authz-pipeline.mmd)
## Two Auth Modes
### 1. API Key (Bearer)
Used for the OpenAI/Anthropic/Gemini-compatible client APIs and a few management routes when the key has the `manage` scope.
```
Authorization: Bearer <api-key>
```
Validated by `isValidApiKey()` / `extractApiKey()` in `src/sse/services/auth.ts` and re-exported through `src/shared/utils/apiAuth.ts`. The validator also accepts the `OMNIROUTE_API_KEY` / `ROUTER_API_KEY` env vars as persistent passthrough keys (issue #1350).
### 2. Dashboard Session (auth_token cookie)
For dashboard pages and admin operations.
```
Cookie: auth_token=<JWT signed with JWT_SECRET>
```
Verified by `isDashboardSessionAuthenticated()` in `src/shared/utils/apiAuth.ts`. The pipeline auto-refreshes the JWT when it has fewer than 7 days left in its 30-day lifetime.
Some management routes accept **either** mode: cookie OR `Bearer <key>` when the API key has the `manage` (or `admin`) scope. This is what enables the "configurable via API calls" workflow added in v3.8.
## Route Classes
`src/server/authz/types.ts` defines three classes; any route that cannot be classified deterministically falls back to `MANAGEMENT`.
| Class | Description | Auth required |
| ------------ | ---------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------- |
| `PUBLIC` | Explicitly safe routes — login, logout, status, init, health, onboarding bootstrap. | None |
| `CLIENT_API` | Model-serving endpoints — `/api/v1/*`, plus aliases `/v1/*`, `/chat/completions`, `/responses`, `/models`, `/codex/*`. | Bearer key (unless `REQUIRE_API_KEY != "true"`) |
| `MANAGEMENT` | Dashboard pages, settings, providers, keys, admin and diagnostics endpoints. | Dashboard session OR Bearer with `manage` scope |
## Pipeline
```
Incoming request → src/middleware.ts
→ runAuthzPipeline() in src/server/authz/pipeline.ts
1. Strip trusted internal headers (x-omniroute-auth-*, x-omniroute-route-class)
2. Generate request id, classify route via classifyRoute()
3. If pathname == "/" → redirect /dashboard
4. If draining (graceful shutdown) and /api/* → 503
5. If non-GET /api/* → checkBodySize() guard
6. If OPTIONS → CORS preflight 204
7. If options.enforce == false → pass-through with route-class headers
8. Otherwise: POLICIES[routeClass].evaluate(ctx)
- allow → stamp x-omniroute-auth-{kind,id,label,scopes} → NextResponse.next()
- reject → JSON error w/ correlation_id (dashboard pages → 302 /login)
```
Trusted internal headers (defined in `src/server/authz/headers.ts`) are **stripped from incoming requests** before classification — clients cannot pre-populate `x-omniroute-auth-*` to impersonate a subject.
### Policy contracts
Each route class has a policy in `src/server/authz/policies/`:
- **`publicPolicy`** (`policies/public.ts`) — always returns `allow({ kind: "anonymous", id: "anonymous" })`.
- **`clientApiPolicy`** (`policies/clientApi.ts`) — extracts Bearer, validates via `validateApiKey()`. Falls through to anonymous if `REQUIRE_API_KEY != "true"`. Allows dashboard-session GET on `/api/v1/models` (used by the dashboard model catalog).
- **`managementPolicy`** (`policies/management.ts`) — accepts dashboard session, internal model-sync requests (matched against `/api/providers/[name]/(sync-models|models)`), or skips entirely if `isAuthRequired()` returns false. Returns 403 (`AUTH_001`) when a Bearer token is present but invalid, 401 otherwise. Also enforces the route-guard tiers (LOCAL_ONLY / ALWAYS_PROTECTED) before any auth branch — see [Route Guard Tiers](../security/ROUTE_GUARD_TIERS.md). LOCAL_ONLY paths in `LOCAL_ONLY_MANAGE_SCOPE_BYPASS_PREFIXES` (today: `/api/mcp/`) may be accessed from non-loopback when the Bearer key carries the `manage` scope; all other LOCAL_ONLY paths remain strict-loopback regardless of scope.
A successful policy returns `AuthSubject` with `kind ∈ { client_api_key, dashboard_session, management_key, anonymous }`. Downstream handlers can read it via `assertAuth(request, "CLIENT_API")` in `src/server/authz/assertAuth.ts` instead of re-running auth logic.
## Public Routes List
`src/shared/constants/publicApiRoutes.ts` is the explicit allowlist:
```ts
PUBLIC_API_ROUTE_PREFIXES = [
"/api/auth/login",
"/api/auth/logout",
"/api/auth/status",
"/api/init",
"/api/v1/", // treated as CLIENT_API in classify, not as "no-auth public"
"/api/cloud/",
"/api/sync/bundle",
"/api/oauth/",
];
PUBLIC_READONLY_API_ROUTE_PREFIXES = ["/api/monitoring/health", "/api/settings/require-login"];
PUBLIC_READONLY_METHODS = new Set(["GET", "HEAD", "OPTIONS"]);
```
Read-only prefixes are public **only** for safe methods. Note: `classifyRoute()` excludes `/api/v1/*` from the PUBLIC fall-through — those are always `CLIENT_API` so the Bearer-key policy still applies.
## Adding a New Route
### Pattern 1 — Public client API endpoint (Bearer-auth)
Routes under `/api/v1/` are classified `CLIENT_API` automatically. The middleware enforces the Bearer check; route handlers don't need to redo it but can read the subject if useful.
```typescript
// src/app/api/v1/your-route/route.ts
import { NextRequest, NextResponse } from "next/server";
import { assertAuth } from "@/server/authz/assertAuth";
export async function POST(req: NextRequest) {
const subject = assertAuth(req, "CLIENT_API");
// subject.kind === "client_api_key" | "anonymous" | "dashboard_session"
// ... handler logic
}
```
### Pattern 2 — Management endpoint (session or Bearer + manage)
Use `requireManagementAuth()` from `src/lib/api/requireManagementAuth.ts`:
```typescript
import { requireManagementAuth } from "@/lib/api/requireManagementAuth";
export async function POST(request: Request) {
const rejection = await requireManagementAuth(request);
if (rejection) return rejection;
// ... handler logic
}
```
`requireManagementAuth()` returns `null` on success or a JSON error `Response`:
- 401 `AUTH_001` "Authentication required" — no credentials at all
- 403 — invalid Bearer **or** Bearer present but key lacks the `manage` / `admin` scope
`hasManageScope(scopes)` returns true for `"manage"` or `"admin"`.
### Pattern 3 — Adding to the public allowlist
Add the prefix to `PUBLIC_API_ROUTE_PREFIXES` (or `PUBLIC_READONLY_API_ROUTE_PREFIXES` for GET-only). Update unit tests at `tests/unit/public-api-routes.test.ts` and `tests/unit/authz/classify.test.ts`.
## Scopes
API keys carry a `scopes` array (stored as JSON in `api_keys.scopes`, see `src/lib/db/apiKeys.ts`).
### Management scope
- `manage` / `admin` — grants the key access to management API endpoints when sent as Bearer.
### MCP scopes (`src/shared/constants/mcpScopes.ts`)
Each MCP tool requires specific scopes via `MCP_TOOL_SCOPES`. Full list (`MCP_SCOPE_LIST`):
```
read:health, read:combos, write:combos, read:quota, read:usage,
read:models, execute:completions, execute:search, write:budget,
write:resilience, pricing:write, read:cache, write:cache,
read:compression, write:compression, read:proxies
```
Preset bundles (`MCP_SCOPE_PRESETS`): `readonly`, `full`, `monitor`, `agent`. Use `hasRequiredScopes(granted, toolName)` and `getMissingScopes()` for enforcement inside MCP handlers.
## Auth Required Toggle
`isAuthRequired()` in `src/shared/utils/apiAuth.ts` decides whether **any** auth is enforced for a request:
- `settings.requireLogin === false` → auth is globally disabled.
- No password configured **and** no `INITIAL_PASSWORD` env var → bootstrap mode allows the onboarding wizard and loopback requests, but exposed network requests still need credentials.
- Any DB error → fails closed (secure-by-default).
## Breaking Change — v3.8.0
The `/api/v1/agents/tasks/*` and `/api/resilience/model-cooldowns` endpoints **now require management auth** (commit `588a0333`). Clients previously sending a normal API key without the `manage` scope receive `403`. Migration: either issue the key the `manage` scope in the API Manager dashboard, or use a logged-in dashboard session.
## Behaviour Change — v3.8.2
`/api/mcp/*` (the remote MCP server) is still LOCAL_ONLY by default but now accepts non-loopback requests when the `Authorization: Bearer <api-key>` header carries the `manage` scope. The carve-out is gated explicitly per-path via `LOCAL_ONLY_MANAGE_SCOPE_BYPASS_PREFIXES` in `src/server/authz/routeGuard.ts`; the sibling LOCAL_ONLY prefix `/api/cli-tools/runtime/*` is intentionally NOT bypassable because it can spawn arbitrary subprocesses. Anonymous requests to `/api/mcp/*` from non-loopback continue to return `403 LOCAL_ONLY` — the default for any new LOCAL_ONLY path remains strict-loopback. See [Route Guard Tiers](../security/ROUTE_GUARD_TIERS.md#manage-scope-carve-out).
## Testing
- Unit tests: `tests/unit/authz/` — `classify.test.ts`, `pipeline.test.ts`, `client-api-policy.test.ts`, `management-policy.test.ts`, `public-policy.test.ts`.
- Public allowlist: `tests/unit/public-api-routes.test.ts`.
- Run focused: `node --import tsx/esm --test tests/unit/authz/classify.test.ts`.
## Debugging
The pipeline always stamps responses with:
```
x-request-id: <correlation id, echoed in error bodies>
x-omniroute-route-class: PUBLIC | CLIENT_API | MANAGEMENT
```
For authenticated requests the upstream (handler-side) request headers also include:
```
x-omniroute-auth-kind: client_api_key | dashboard_session | management_key | anonymous
x-omniroute-auth-id: key_<last-4> | "dashboard" | "anonymous"
x-omniroute-auth-label: (optional)
x-omniroute-auth-scopes: comma-separated list
```
Use `assertAuth(req, expectedClass)` inside handlers — it throws `AuthzAssertionError` with code `AUTHZ_NOT_INITIALIZED` if the middleware was bypassed (helpful for catching configuration regressions in tests).
## See Also
- [API_REFERENCE.md](../reference/API_REFERENCE.md) — auth marker per endpoint
- [COMPLIANCE.md](../security/COMPLIANCE.md) — audit log for auth events
- [MCP-SERVER.md](../frameworks/MCP-SERVER.md) — MCP scope enforcement details
- Source: `src/server/authz/`, `src/lib/api/requireManagementAuth.ts`

247
Auto-Combo.md Normal file

@@ -0,0 +1,247 @@
# OmniRoute Auto-Combo Engine
> Self-managing model chains with adaptive scoring + zero-config auto-routing
## Zero-Config Auto-Routing (`auto/` prefix)
> **NEW:** No combo creation required. Use `auto/` prefix directly in any client.
### Quick Examples
| Model ID | Variant | Behavior |
| -------------- | ------- | ------------------------------------------------------------------------ |
| `auto` | default | All connected providers, LKGP strategy, balanced weights |
| `auto/coding` | coding | Quality-first weights, suitable for code generation |
| `auto/fast` | fast | Low-latency weighted selection |
| `auto/cheap` | cheap | Cost-optimized routing (lowest cost first) |
| `auto/offline` | offline | Favors providers with highest quota availability |
| `auto/smart` | smart | Quality-first + higher exploration rate (10%) for better model discovery |
| `auto/lkgp` | lkgp | Explicit LKGP (same as default `auto`) |
**How to use:**
```bash
# Any IDE or CLI tool that supports OpenAI format
Base URL: http://localhost:20128/v1
API Key: <your-endpoint-key>
# In your code/config, set model to:
model: "auto" # balanced default
model: "auto/coding" # best for coding tasks
model: "auto/fast" # fastest available
model: "auto/cheap" # cheapest per token
```
**What happens:**
1. OmniRoute detects `auto/` prefix in `src/sse/handlers/chat.ts`
2. Queries all **active provider connections** from the database
3. Filters to those with valid credentials (API key or OAuth token)
4. Determines the model per connection (`connection.defaultModel` or provider's first model)
5. Builds a **virtual combo** in-memory (not stored in DB)
6. Routes using the selected variant's weight profile + LKGP strategy
**Key properties:**
- ✅ **Always-on:** No toggle, no combo creation, no configuration needed
- ✅ **Dynamic:** Reflects current connected providers automatically
- ✅ **Session stickiness:** LKGP ensures last successful provider is prioritized
- ✅ **Multi-account aware:** Each provider connection becomes a separate candidate
- ✅ **No DB writes:** Virtual combo exists only for the request, zero persistence overhead
**Behind the scenes:**
```txt
Request: { model: "auto/coding" }
↓
src/sse/handlers/chat.ts detects prefix
↓
createVirtualAutoCombo('coding') → candidatePool from active connections
↓
handleComboChat (same engine as persisted combos)
↓
Auto-scoring selects best provider/model per request
```
**Implementation files:**
| File | Purpose |
| --------------------------------------------------------- | ----------------------------------------- |
| `open-sse/services/autoCombo/autoPrefix.ts` | Prefix parser (`parseAutoPrefix`) |
| `open-sse/services/autoCombo/virtualFactory.ts` | Creates virtual `AutoComboConfig` objects |
| `open-sse/services/autoCombo/providerRegistryAccessor.ts` | Test hook for mocking provider registry |
| `src/sse/handlers/chat.ts` | Integration: auto prefix short-circuit |
| `src/shared/constants/providers.ts` | `SYSTEM_PROVIDERS.auto` system entry |
## How It Works (Persisted Auto-Combos)
The Auto-Combo Engine dynamically selects the best provider/model for each request using a **9-factor scoring function** (defined in `open-sse/services/autoCombo/scoring.ts` → `DEFAULT_WEIGHTS`). All weights sum to **1.0**.
![Auto-Combo 9-factor scoring](../diagrams/exported/auto-combo-9factor.svg)
> Source: [diagrams/auto-combo-9factor.mmd](../diagrams/auto-combo-9factor.mmd)
| Factor | Default Weight | Description |
| :----------------- | :------------- | :------------------------------------------------------------------------------------------------- |
| `health` | 0.22 | Health score from circuit breaker (CLOSED=1.0, HALF_OPEN=0.5, OPEN=0.0) |
| `quota` | 0.17 | Remaining quota / rate-limit headroom [0..1] |
| `costInv` | 0.17 | Inverse **blended** cost (60% input + 40% output token price, normalized) — cheaper = higher score |
| `latencyInv` | 0.13 | Inverse p95 latency normalized to pool — faster = higher score |
| `taskFit` | 0.08 | Task-type fitness (coding, review, planning, analysis, debugging, docs) |
| `specificityMatch` | 0.08 | Match between request specificity (manifest hint) and model tier |
| `stability` | 0.05 | Variance-based stability (low latency stdDev / error rate) |
| `tierPriority` | 0.05 | Account-tier priority — Ultra=1.0, Pro=0.67, Standard=0.33, Free=0.0 |
| `tierAffinity` | 0.05 | Affinity between the candidate's tier and the manifest-recommended tier |
**Sum:** `0.22 + 0.17 + 0.17 + 0.13 + 0.08 + 0.08 + 0.05 + 0.05 + 0.05 = 1.0` (validated by `validateWeights()`).
## Mode Packs
Four pre-defined weight profiles in `open-sse/services/autoCombo/modePacks.ts`. Each pack overrides the default weights to bias selection toward a specific goal. Below are the **full weight tables per pack** (each row sums to 1.0).
| Factor | ship-fast | cost-saver | quality-first | offline-friendly |
| :----------- | :-------- | :--------- | :------------ | :--------------- |
| quota | 0.15 | 0.15 | 0.10 | **0.40** |
| health | 0.30 | 0.20 | 0.20 | 0.30 |
| costInv | 0.05 | **0.40** | 0.05 | 0.10 |
| latencyInv | **0.35** | 0.05 | 0.05 | 0.05 |
| taskFit | 0.10 | 0.10 | **0.40** | 0.00 |
| stability | 0.00 | 0.05 | 0.15 | 0.10 |
| tierPriority | 0.05 | 0.05 | 0.05 | 0.05 |
Notes:
- `tierAffinity` and `specificityMatch` are not set in mode packs — `calculateScore()` treats them as `?? 0` when absent.
- Each pack's emphasis at a glance:
- **ship-fast** → latencyInv 0.35 + health 0.30 (low-latency, healthy connections)
- **cost-saver** → costInv 0.40 (cheapest tokens win)
- **quality-first** → taskFit 0.40 + stability 0.15 (best model for the task, consistent)
- **offline-friendly** → quota 0.40 + health 0.30 (max headroom regardless of speed/cost)
## All Routing Strategies
OmniRoute's combo engine supports **14 routing strategies** (declared in `src/shared/constants/routingStrategies.ts` → `ROUTING_STRATEGY_VALUES`). The Auto Combo engine itself is exposed under the `auto` strategy; the others are available for persisted combos.
| Strategy | Description |
| :------------------ | :----------------------------------------------------------------- |
| `priority` | First-target ordered list with explicit priority |
| `weighted` | Weighted random by per-target weight |
| `round-robin` | Cycle through targets in order |
| `context-relay` | Hand off context across targets (long conversations) |
| `fill-first` | Fill each target's quota before moving to next |
| `p2c` | Power-of-2-choices random load balancing |
| `random` | Uniform random selection |
| `least-used` | Pick target with lowest current load |
| `cost-optimized` | Minimize $ per request given catalog pricing |
| `reset-aware` ⭐ | Prioritize by quota reset time — short reset windows ranked higher |
| `strict-random` | Random without deduplication of repeats |
| `auto` | Use Auto Combo scoring (9-factor) — **recommended** |
| `lkgp` | Last-Known-Good Path (sticky route to last successful target) |
| `context-optimized` | Pick target with best fit for current context size |
⭐ = New in v3.8.0
## Virtual Auto-Combo Factory
The Auto Combo engine doesn't require pre-defined combos. Instead, `open-sse/services/autoCombo/virtualFactory.ts` builds candidates on-the-fly:
1. Pulls `getProviderConnections({ isActive: true })` (all enabled connections)
2. Filters to those with valid credentials (API key or non-expired OAuth token via `hasUsableOAuthToken()`)
3. Cross-references with `getProviderRegistry()` for model availability + pricing
4. For each tuple `(provider, model, connection)`, builds a `VirtualAutoComboCandidate`
5. Picks `connection.defaultModel` (or the registry's first model) as the dispatch target
6. Scores each candidate using the 9-factor `scorePool()` and the variant's weight pack
7. Returns the resulting in-memory `AutoComboConfig` for `handleComboChat()` — never persisted to DB
This means **adding a new provider with `auto/*` enabled automatically expands the candidate pool** — no manual combo editing needed. The virtual combo is rebuilt per request, so newly-added or newly-healthy connections are picked up immediately.
## Self-Healing
- **Temporary exclusion**: Score < 0.2 → excluded for 5 min (progressive backoff, max 30 min)
- **Circuit breaker awareness**: OPEN → auto-excluded; HALF_OPEN → probe requests
- **Incident mode**: >50% OPEN → disable exploration, maximize stability
- **Cooldown recovery**: After exclusion, first request is a "probe" with reduced timeout
## Bandit Exploration
5% of requests (configurable) are routed to random providers for exploration. Disabled in incident mode.
## API
There is **no dedicated `POST /api/combos/auto` endpoint** — Auto-Combo is consumed in two ways:
1. **Zero-config (recommended):** Send any chat completion request with `model: "auto"` or `model: "auto/<variant>"`. The virtual factory builds the combo per request — no persistence, no API calls needed.
2. **Persisted combo with `strategy: "auto"`:** Create a regular combo via `POST /api/combos` and set `strategy: "auto"` plus `config.auto.weights` / `config.auto.candidatePool`. The same scoring engine is used; the combo is stored in `combos` and reusable by ID.
```bash
# Zero-config usage (no combo creation)
curl -X POST http://localhost:20128/v1/chat/completions \
-H "Authorization: Bearer <key>" \
-H "Content-Type: application/json" \
-d '{"model":"auto/coding","messages":[{"role":"user","content":"Hello"}]}'
# Persisted auto combo via the regular combos endpoint
curl -X POST http://localhost:20128/api/combos \
-H "Content-Type: application/json" \
-d '{"id":"my-auto","name":"Auto Coder","strategy":"auto","config":{"auto":{"candidatePool":["anthropic","google","openai"],"weights":{"quota":0.15,"health":0.3,"costInv":0.05,"latencyInv":0.35,"taskFit":0.1,"stability":0,"tierPriority":0.05}}}}'
```
## Task Fitness
30+ models scored across 6 task types (`coding`, `review`, `planning`, `analysis`, `debugging`, `documentation`). Supports wildcard patterns (e.g., `*-coder` → high coding score).
## Auto Variants Recap
Including the bare `auto` (default) plus the 6 `AutoVariant` values declared in `autoPrefix.ts`, there are **7 invokable model IDs**:
`auto`, `auto/coding`, `auto/fast`, `auto/cheap`, `auto/offline`, `auto/smart`, `auto/lkgp`
(`AutoVariant` itself enumerates 6 values; the 7th option is "no variant" — bare `auto` — handled by `parseAutoPrefix()` as `variant: undefined`.)
## How tiers fit Auto-Combo
The 9-factor scoring function (`open-sse/services/autoCombo/scoring.ts`) treats tier
membership as one signal via the `tierPriority` weight. Default weights (from `DEFAULT_WEIGHTS`):
| Factor | Default weight | Notes |
| ------------------------ | -------------- | -------------------------------------------------------------- |
| Tier priority | 0.05 | Tier 1 premium → higher score |
| Latency (p50 inverse) | 0.35 | Fastest wins |
| Cost ($/1M inverse) | 0.20 | Cheapest **blended** price wins (60% input + 40% output ratio) |
| Recent health/error rate | 0.15 | Unhealthy deprioritized |
| Quota remaining | 0.10 | Near-exhausted deprioritized |
| Context window match | 0.08 | Penalizes short windows |
| Task fitness | 0.10 | Coding → coding-specialist models |
| Stability | 0.00 | Disabled by default |
Tier alone does **not** force Tier 1 first — if Tier 1 latency is bad or
cost-vs-quality is suboptimal, Tier 2 wins. To force tier ordering, use combo
strategy `priority` and arrange providers by tier.
To strongly favor Tier 1 (subscription), increase `tierPriority` weight:
```json
{
"strategy": "auto",
"config": { "auto": { "weights": { "tierPriority": 0.3, "costInv": 0.05 } } }
}
```
See `docs/marketing/TIERS.md` for tier definitions and provider classification.
## Files
| File | Purpose |
| :-------------------------------------------------------- | :------------------------------------------------------------------------- |
| `open-sse/services/autoCombo/scoring.ts` | 9-factor scoring function, `DEFAULT_WEIGHTS`, pool norm |
| `open-sse/services/autoCombo/taskFitness.ts` | Model × task fitness lookup |
| `open-sse/services/autoCombo/engine.ts` | Selection logic, bandit, budget cap |
| `open-sse/services/autoCombo/selfHealing.ts` | Exclusion, probes, incident mode |
| `open-sse/services/autoCombo/modePacks.ts` | 4 weight profiles (ship-fast, cost-saver, quality-first, offline-friendly) |
| `open-sse/services/autoCombo/autoPrefix.ts` | `auto/` prefix parser + 6 variants |
| `open-sse/services/autoCombo/virtualFactory.ts` | Builds in-memory `AutoComboConfig` from live connections |
| `open-sse/services/autoCombo/providerRegistryAccessor.ts` | Test hook for mocking provider registry |
| `src/shared/constants/routingStrategies.ts` | `ROUTING_STRATEGY_VALUES` (14 strategies) |
| `src/sse/handlers/chat.ts` | Integration: auto-prefix short-circuit |

43
CLI-Token-Auth.md Normal file

@@ -0,0 +1,43 @@
# CLI Machine-ID Token Authentication
OmniRoute's CLI uses a **machine-derived token** to authenticate to the local server without requiring an explicit API key. This enables zero-config local use while preserving security for remote access.
## How it works
1. **CLI side** (`bin/cli/utils/cliToken.mjs`): computes `SHA-256(machineId + salt).hex[0..32]` using [`node-machine-id`](https://github.com/automation-stack/node-machine-id) and injects the result as the `x-omniroute-cli-token` header on every `apiFetch` call.
2. **Server side** (`src/lib/middleware/cliTokenAuth.ts`): `isCliTokenAuthValid(request)` accepts the token only if:
- `OMNIROUTE_DISABLE_CLI_TOKEN` is not `"true"`
- The header is present and exactly 32 hex characters
- The originating IP is loopback (`127.0.0.1`, `::1`, `::ffff:127.0.0.1`)
- The token matches the server's own machine-derived hash (timing-safe compare)
3. `requireManagementAuth` and other route guards call `isCliTokenAuthValid` before checking API keys — so the CLI gets transparent localhost access without storing any credential.
## Threat model
| Scenario | Risk | Mitigation |
| ------------------------- | ---------------------------- | ------------------------------------------------------------------------------------------------------------------------------------ |
| Another user on same host | Could compute the same token | `machine-id` is per-device; on single-user desktops this is acceptable. Use `OMNIROUTE_DISABLE_CLI_TOKEN=true` in multi-user setups. |
| Token leak via logs | Logs may reveal the token | The header value is masked in audit logs (`x-omniroute-cli-token: ***`). |
| Replay attack | Token is static | Only accepted from `127.0.0.1`/`::1`. Rejected for any other `x-forwarded-for` IP. |
| Reuse on another machine | Machine-bound by design | `node-machine-id` reads `/etc/machine-id` (Linux), `IOPlatformUUID` (macOS), `MachineGuid` (Windows). Different per host. |
## Opt-out
Set `OMNIROUTE_DISABLE_CLI_TOKEN=true` in `.env` or the server environment to disable this mechanism entirely. All access then requires an explicit API key.
## Audit logging
Every request authenticated via CLI token is logged with `event: "cli_token_auth"`, the source IP, user-agent, path, and the first 8 characters of the machine-id hash (non-reversible).
## API key precedence
An explicit `Authorization: Bearer <key>` header (from `--api-key` or `OMNIROUTE_API_KEY`) always takes precedence over the CLI token and is evaluated first.
## Related files
- `bin/cli/utils/cliToken.mjs` — CLI token generation
- `src/lib/middleware/cliTokenAuth.ts` — server validation
- `src/lib/api/requireManagementAuth.ts` — integration into auth pipeline
- `tests/unit/cli-machine-token.test.ts` — unit tests

685
CLI-Tools.md Normal file

@@ -0,0 +1,685 @@
# CLI Tools — OmniRoute v3.8.0
Last updated: 2026-05-13
OmniRoute integrates with two categories of CLI tools:
1. **External CLI integrations** — third-party CLIs (Cursor, Cline, Codex, Claude Code, Qwen Code, Windsurf, Hermes, Amp, etc.) that you point at OmniRoute's local OpenAI-compatible endpoint.
2. **Internal OmniRoute CLI** — commands bundled with the `omniroute` binary for server lifecycle, setup, diagnostics, and provider management.
---
## How It Works
```
Claude / Codex / OpenCode / Cline / KiloCode / Continue / Cursor / Windsurf / Hermes / Amp / Qwen
│
▼ (all point to OmniRoute)
http://YOUR_SERVER:20128/v1
│
▼ (OmniRoute routes to the right provider)
Anthropic / OpenAI / Gemini / DeepSeek / Groq / Mistral / ...
```
**Benefits:**
- One API key to manage all tools
- Cost tracking across all CLIs in the dashboard
- Model switching without reconfiguring every tool
- Works locally and on remote servers (VPS, Docker, Akamai, Cloudflare Tunnel)
---
## 1. External CLI Integrations
### Source of Truth
The dashboard cards in `/dashboard/cli-tools` are generated from
`src/shared/constants/cliTools.ts`. The `omniroute setup` command can write
config files automatically for the scriptable tools.
### Current Catalog (v3.8.0)
| Tool | ID | Type / Config | Install / Access | Auth |
| ------------------ | ------------- | ------------------- | ------------------------------------ | ----------------------------------- |
| **Claude Code** | `claude` | env / settings.json | `npm i -g @anthropic-ai/claude-code` | API key (Anthropic gateway) |
| **OpenAI Codex** | `codex` | custom (toml) | `npm i -g @openai/codex` | API key (OpenAI) |
| **Factory Droid** | `droid` | custom | bundled / CLI | API key |
| **Open Claw** | `openclaw` | custom | bundled / CLI | API key |
| **Cursor** | `cursor` | guide (Cloud) | Cursor desktop app | API key (Cloud Endpoint) |
| **Windsurf** | `windsurf` | guide | Windsurf desktop IDE | API key (BYOK) |
| **Cline** | `cline` | custom / VS Code | `npm i -g cline` + VS Code ext | API key |
| **Kilo Code** | `kilo` | custom / VS Code | `npm i -g kilocode` + VS Code ext | API key |
| **Continue** | `continue` | guide (config.yaml) | VS Code extension | API key |
| **Antigravity** | `antigravity` | MITM | OmniRoute built-in | API key (MITM proxy) |
| **GitHub Copilot** | `copilot` | custom / VS Code | VS Code extension | API key (CLI fingerprint: `github`) |
| **OpenCode** | `opencode` | guide (json) | `npm i -g opencode-ai` | API key (OpenAI-compatible) |
| **Hermes** | `hermes` | guide (json) | install per docs | API key (OpenAI-compatible) |
| **Amp CLI** | `amp` | guide (env) | install per Sourcegraph docs | API key (OpenAI-compatible) |
| **Kiro AI** | `kiro` | MITM | Amazon Kiro IDE / CLI | API key (MITM proxy) |
| **Qwen Code** | `qwen` | guide (json/env) | `npm i -g @qwen-code/qwen-code` | API key (OpenAI-compatible) |
| **Custom CLI** | `custom` | custom-builder | any OpenAI-compatible client | API key |
> Notes:
>
> - "Web wrappers" like ChatGPT/Claude/Grok/Perplexity browser sessions are not
> listed here. OmniRoute can proxy them through the `chatgpt-web`,
> `claude-web`, `grok-web`, `perplexity-web`, `blackbox-web`,
> `muse-spark-web` provider connections, but those are **provider connections**
> (configured under `/dashboard/providers`), not CLI tools. They do not surface
> as cards under `/dashboard/cli-tools`.
> - Tools marked **MITM** (Antigravity, Kiro) intercept the desktop app traffic
> locally and require enabling the corresponding mitm endpoint in
> `/dashboard/settings`.
### CLI fingerprint sync (Agents + Settings)
`/dashboard/agents` and `Settings > CLI Fingerprint` use
`src/shared/constants/cliCompatProviders.ts`. This keeps provider IDs aligned
with the CLI cards and legacy IDs.
| CLI ID | Fingerprint Provider ID |
| --------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------- |
| `kilo` | `kilocode` |
| `copilot` | `github` |
| `claude` / `codex` / `antigravity` / `kiro` / `cursor` / `windsurf` / `cline` / `opencode` / `hermes` / `amp` / `qwen` / `droid` / `openclaw` | same ID |
Legacy IDs still accepted for compatibility: `copilot`, `kimi-coding`, `qwen`.
---
### Step 1 — Get an OmniRoute API Key
1. Open the OmniRoute dashboard → **API Manager** (`/dashboard/api-manager`)
2. Click **Create API Key**
3. Give it a name (e.g. `cli-tools`) and select all permissions
4. Copy the key — you'll need it for every CLI below
> Your key looks like: `sk-xxxxxxxxxxxxxxxx-xxxxxxxxx`
---
### Step 2 — Install CLI Tools
All npm-based tools require Node.js 20.20.2+, 22.22.2+ or 24.x:
```bash
# Claude Code (Anthropic)
npm install -g @anthropic-ai/claude-code
# OpenAI Codex
npm install -g @openai/codex
# OpenCode
npm install -g opencode-ai
# Cline
npm install -g cline
# KiloCode
npm install -g kilocode
# Qwen Code (Alibaba)
npm install -g @qwen-code/qwen-code
# Kiro CLI (Amazon — requires curl + unzip)
apt-get install -y unzip # on Debian/Ubuntu
curl -fsSL https://cli.kiro.dev/install | bash
export PATH="$HOME/.local/bin:$PATH" # add to ~/.bashrc
```
**Verify:**
```bash
claude --version # 2.x.x
codex --version # 0.x.x
opencode --version # x.x.x
cline --version # 2.x.x
kilocode --version # x.x.x (or: kilo --version)
qwen --version # x.x.x
kiro-cli --version # 1.x.x
```
---
### Step 3 — Set Global Environment Variables
Add to `~/.bashrc` (or `~/.zshrc`), then run `source ~/.bashrc`:
```bash
# OmniRoute Universal Endpoint
export OPENAI_BASE_URL="http://localhost:20128/v1"
export OPENAI_API_KEY="sk-your-omniroute-key"
export ANTHROPIC_BASE_URL="http://localhost:20128"
export ANTHROPIC_AUTH_TOKEN="sk-your-omniroute-key"
export GEMINI_BASE_URL="http://localhost:20128/v1"
export GEMINI_API_KEY="sk-your-omniroute-key"
```
> For a **remote server** replace `localhost:20128` with the server IP or domain,
> e.g. `http://192.168.0.15:20128`.
---
### Step 4 — Configure Each Tool
#### Claude Code
```bash
# Create ~/.claude/settings.json:
mkdir -p ~/.claude && cat > ~/.claude/settings.json << EOF
{
"env": {
"ANTHROPIC_BASE_URL": "http://localhost:20128",
"ANTHROPIC_AUTH_TOKEN": "sk-your-omniroute-key"
}
}
EOF
```
Use the unified Anthropic gateway root for Claude Code. Do not append `/v1` here.
**Test:** `claude "say hello"`
---
#### OpenAI Codex
```bash
mkdir -p ~/.codex && cat > ~/.codex/config.yaml << EOF
model: auto
apiKey: sk-your-omniroute-key
apiBaseUrl: http://localhost:20128/v1
EOF
```
**Test:** `codex "what is 2+2?"`
---
#### OpenCode
```bash
mkdir -p ~/.config/opencode && cat > ~/.config/opencode/opencode.json << EOF
{
"\$schema": "https://opencode.ai/config.json",
"provider": {
"omniroute": {
"npm": "@ai-sdk/openai-compatible",
"name": "OmniRoute",
"options": {
"baseURL": "http://localhost:20128/v1",
"apiKey": "sk-your-omniroute-key"
},
"models": {
"claude-sonnet-4-5": { "name": "claude-sonnet-4-5" },
"claude-sonnet-4-5-thinking": { "name": "claude-sonnet-4-5-thinking" },
"gemini-3-flash": { "name": "gemini-3-flash" }
}
}
}
}
EOF
```
**Test:** `opencode`
> Use `opencode run "your prompt" --model omniroute/claude-sonnet-4-5-thinking --variant high`
> to send thinking variants.
---
#### Cline (CLI or VS Code)
**CLI mode:**
```bash
mkdir -p ~/.cline/data && cat > ~/.cline/data/globalState.json << EOF
{
"apiProvider": "openai",
"openAiBaseUrl": "http://localhost:20128/v1",
"openAiApiKey": "sk-your-omniroute-key"
}
EOF
```
**VS Code mode:**
Cline extension settings → API Provider: `OpenAI Compatible` → Base URL: `http://localhost:20128/v1`
Or use the OmniRoute dashboard → **CLI Tools → Cline → Apply Config**.
---
#### KiloCode (CLI or VS Code)
**CLI mode:**
```bash
kilocode --api-base http://localhost:20128/v1 --api-key sk-your-omniroute-key
```
**VS Code settings:**
```json
{
"kilo-code.openAiBaseUrl": "http://localhost:20128/v1",
"kilo-code.apiKey": "sk-your-omniroute-key"
}
```
Or use the OmniRoute dashboard → **CLI Tools → KiloCode → Apply Config**.
---
#### Continue (VS Code Extension)
Edit `~/.continue/config.yaml`:
```yaml
models:
- name: OmniRoute
provider: openai
model: auto
apiBase: http://localhost:20128/v1
apiKey: sk-your-omniroute-key
default: true
```
Restart VS Code after editing.
---
#### Kiro CLI (Amazon)
```bash
# Login to your AWS/Kiro account:
kiro-cli login
# The CLI uses its own auth — OmniRoute is not needed as backend for Kiro CLI itself.
# Use kiro-cli alongside OmniRoute for other tools.
kiro-cli status
```
For the **Kiro IDE** desktop app, use the MITM endpoint exposed by OmniRoute
under `/dashboard/cli-tools → Kiro`.
---
#### Qwen Code (Alibaba)
Qwen Code supports OpenAI-compatible API endpoints via environment variables or `settings.json`.
> Qwen OAuth free tier was discontinued on 2026-04-15. Use OmniRoute with
> `bailian-coding-plan` / `alibaba` / `alibaba-cn` / `openrouter` / `anthropic` /
> `gemini` providers instead.
**Option 1: Environment variables (`~/.qwen/.env`)**
```bash
mkdir -p ~/.qwen && cat > ~/.qwen/.env << EOF
OPENAI_API_KEY="sk-your-omniroute-key"
OPENAI_BASE_URL="http://localhost:20128/v1"
OPENAI_MODEL="auto"
EOF
```
**Option 2: `settings.json` with `security.auth`**
```json
// ~/.qwen/settings.json
{
"security": {
"auth": {
"selectedType": "openai",
"apiKey": "sk-your-omniroute-key",
"baseUrl": "http://localhost:20128/v1"
}
},
"model": {
"name": "claude-sonnet-4-6"
}
}
```
**Option 3: Inline CLI flags**
```bash
OPENAI_BASE_URL="http://localhost:20128/v1" \
OPENAI_API_KEY="sk-your-omniroute-key" \
OPENAI_MODEL="auto" \
qwen
```
> For a **remote server** replace `localhost:20128` with the server IP or domain.
**Test:** `qwen "say hello"`
---
#### Cursor (Desktop App)
> **Note:** Cursor routes requests through its cloud. For OmniRoute integration,
> enable **Cloud Endpoint** in OmniRoute Settings and use your public domain URL.
Via GUI: **Settings → Models → OpenAI API Key**
- Base URL: `https://your-domain.com/v1`
- API Key: your OmniRoute key
---
#### Windsurf (Desktop IDE)
> Official Windsurf docs currently describe BYOK for select Claude models plus
> enterprise URL/token settings, not a generic custom OpenAI-compatible provider.
> Test BYOK behavior in your environment before relying on this integration.
1. Open AI Settings inside Windsurf.
2. Select **Add custom provider** (OpenAI-compatible).
3. Base URL: `http://localhost:20128/v1`
4. API Key: your OmniRoute key
5. Pick a model from the OmniRoute catalog.
---
#### Hermes
```json
// Hermes config file
{
"provider": {
"type": "openai",
"baseURL": "http://localhost:20128/v1",
"apiKey": "sk-your-omniroute-key",
"model": "claude-sonnet-4-6"
}
}
```
---
#### Amp CLI (Sourcegraph)
```bash
export OPENAI_API_KEY="sk-your-omniroute-key"
export OPENAI_BASE_URL="http://localhost:20128/v1"
amp --model "claude-sonnet-4-6"
# Suggested shorthand aliases you can map locally:
# g25p -> gemini/gemini-2.5-pro
# g25f -> gemini/gemini-2.5-flash
# cs45 -> cc/claude-sonnet-4-5-20250929
# g54 -> gemini/gemini-3.1-pro-high
```
---
### Dashboard Auto-Configuration
The OmniRoute dashboard automates configuration for most tools:
1. Go to `http://localhost:20128/dashboard/cli-tools`
2. Expand any tool card
3. Select your API key from the dropdown
4. Click **Apply Config** (if the tool is detected as installed)
5. Or copy the generated config snippet manually
---
### Built-in Agents: Droid & Open Claw
**Droid** and **Open Claw** are AI agents built directly into OmniRoute — no
installation needed. They run as internal routes and use OmniRoute's model
routing automatically.
- Access: `http://localhost:20128/dashboard/agents`
- Configure: same combos and providers as all other tools
- No API key or CLI install required
---
## 2. Internal OmniRoute CLI
The `omniroute` binary (installed via `npm install -g omniroute` or bundled
with the desktop app) provides commands beyond running the server. The full
matrix is implemented in:
- `bin/omniroute.mjs` — entry point, env loading, special-case dispatch (`--mcp`)
- `bin/cli/program.mjs` — Commander program builder
- `bin/cli/commands/<cmd>.mjs` — one file per command/group, registered in `registry.mjs`
- `bin/cli/output.mjs` — output formatters (json/jsonl/table/csv)
- `bin/cli/runtime.mjs` — withRuntime helper (server-first/db-fallback)
- `bin/cli/i18n.mjs` — t() helper with locales
### Server Lifecycle
```bash
omniroute # Start server (default port 20128)
omniroute --port 3000 # Override port
omniroute --no-open # Don't auto-open browser
omniroute --mcp # Start as MCP server (stdio transport)
omniroute serve # Same as `omniroute`
omniroute stop # Stop the running server
omniroute restart # Restart the server
omniroute dashboard # Open dashboard in default browser
omniroute open # Alias for `dashboard`
omniroute --version # Print version
omniroute --help # Show all commands
```
### Setup & Initialization
```bash
omniroute setup # Interactive setup wizard
omniroute setup --non-interactive # CI/automation mode (reads env vars + flags)
omniroute setup --password '<value>' # Set admin password directly
omniroute setup --add-provider \
--provider openai \
--api-key '<value>' \
--test-provider # Add and test a provider in one shot
```
Recognized environment variables for non-interactive setup:
| Var | Purpose |
| ----------------------------- | -------------------------------------------- |
| `OMNIROUTE_SETUP_PASSWORD` | Admin password (>=8 chars) |
| `OMNIROUTE_PROVIDER` | Provider id (e.g. `openai`, `anthropic`) |
| `OMNIROUTE_PROVIDER_NAME` | Display name for the connection |
| `OMNIROUTE_PROVIDER_BASE_URL` | Optional OpenAI-compatible base URL override |
| `OMNIROUTE_API_KEY` | Provider API key |
| `OMNIROUTE_DEFAULT_MODEL` | Optional default model |
| `DATA_DIR` | Override the OmniRoute data directory |
### Diagnostics
```bash
omniroute doctor # Check config, DB, ports, runtime, memory, liveness
omniroute doctor --json # Machine-readable JSON
omniroute doctor --no-liveness # Skip the HTTP health probe
omniroute doctor --host 0.0.0.0 # Override liveness host
omniroute doctor --liveness-url <url> # Full health endpoint URL override
```
The doctor runs these checks: `Config`, `Database`, `Storage/encryption`,
`Port availability`, `Node runtime`, `Native binary` (better-sqlite3),
`Memory`, and `Server liveness`. It exits non-zero if any check is `fail`.
### Provider Management
```bash
omniroute providers available # OmniRoute provider catalog
omniroute providers available --search openai # Filter catalog by id/name/alias/category
omniroute providers available --category api-key # Filter by category (api-key, oauth, free, ...)
omniroute providers available --json # Machine-readable JSON
omniroute providers list # Configured provider connections
omniroute providers list --json
omniroute providers test <id|name> # Test one configured connection
omniroute providers test-all # Test every active connection
omniroute providers validate # Local-only structural validation
```
> `providers available` reads the OmniRoute catalog; `providers list/test/test-all/validate`
> read the local SQLite database directly and do not require the server to be running.
### Recovery & Reset
```bash
omniroute reset-password # Reset the admin password (legacy alias still works)
omniroute reset-encrypted-columns # Show warning + dry-run for encrypted credential reset
omniroute reset-encrypted-columns --force # Actually null out encrypted credentials in SQLite
```
### Other subcommands
These assume a running OmniRoute server, unless noted otherwise:
```bash
omniroute status # Comprehensive runtime status
omniroute logs # Stream request logs (--json, --search, --follow)
omniroute config show # Display current configuration
omniroute provider list # List available providers (alias of providers list)
omniroute provider add # Register OmniRoute as a provider on a tool
omniroute keys add | list | remove # Manage API keys
omniroute models [provider] # List models (--json, --search)
omniroute combo list | switch | create | delete
omniroute backup # Snapshot config + DB
omniroute restore # Restore from a previous snapshot
omniroute health # Detailed health (breakers, cache, memory)
omniroute quota # Provider quota usage
omniroute cache # Cache status
omniroute cache clear # Clear semantic + signature caches
omniroute mcp status | restart # MCP server status / restart
omniroute a2a status | card # A2A server status / agent card
omniroute tunnel list | create | stop # Manage tunnels (cloudflare/tailscale/ngrok)
omniroute env show | get <k> | set <k> <v> # Inspect / set env vars (temporary)
omniroute test # Provider connectivity smoke test
omniroute update # Check for updates
omniroute completion # Generate shell completion
```
### Common flags
| Flag | Description |
| ------------------- | ------------------------------------------------------ |
| `--no-open` | Don't auto-open the browser on start |
| `--port <n>` | Override the API port (default 20128) |
| `--mcp` | Run as MCP server over stdio (for IDEs) |
| `--non-interactive` | CI mode (no prompts; reads from env/flags) |
| `--json` | Machine-readable JSON output (doctor, providers, etc.) |
| `--help`, `-h` | Show command-specific help |
| `--version`, `-v` | Print the installed version |
---
## Available API Endpoints
| Endpoint | Description | Use For |
| -------------------------- | ----------------------------- | --------------------------- |
| `/v1/chat/completions` | Standard chat (all providers) | All modern tools |
| `/v1/responses` | Responses API (OpenAI format) | Codex, agentic workflows |
| `/v1/completions` | Legacy text completions | Older tools using `prompt:` |
| `/v1/embeddings` | Text embeddings | RAG, search |
| `/v1/images/generations` | Image generation | GPT-Image, Flux, etc. |
| `/v1/audio/speech` | Text-to-speech | ElevenLabs, OpenAI TTS |
| `/v1/audio/transcriptions` | Speech-to-text | Deepgram, AssemblyAI |
---
## Troubleshooting
| Error | Cause | Fix |
| ------------------------------------------------- | --------------------------- | --------------------------------------------------------------------------- |
| `Connection refused` | OmniRoute not running | `omniroute serve` or `pm2 start omniroute` |
| `401 Unauthorized` | Wrong API key | Check in `/dashboard/api-manager` |
| `No combo configured` | No active routing combo | Set up in `/dashboard/combos` |
| `invalid model` | Model not in catalog | Use `auto` or check `/dashboard/providers` |
| CLI shows "not installed" | Binary not in PATH | Check `which <command>` |
| `kiro-cli: not found` | Not in PATH | `export PATH="$HOME/.local/bin:$PATH"` |
| `doctor` reports SQLite incompatible | Wrong native binary | `cd app && npm rebuild better-sqlite3` |
| `doctor` reports `STORAGE_ENCRYPTION_KEY` missing | Encrypted creds without key | Set `STORAGE_ENCRYPTION_KEY` or `omniroute reset-encrypted-columns --force` |
---
## Quick Setup Script (One Command)
```bash
cat > my-setup.sh <<'EOF'
#!/usr/bin/env bash
set -euo pipefail
# === Edit these ===
OMNIROUTE_URL="http://localhost:20128/v1"
OMNIROUTE_ANTHROPIC_URL="http://localhost:20128"
OMNIROUTE_KEY="sk-your-omniroute-key"
# ==================
# 1. Install the external CLIs
npm install -g \
@anthropic-ai/claude-code \
@openai/codex \
opencode-ai \
cline \
kilocode \
@qwen-code/qwen-code
# 2. Optional: Kiro CLI (needs unzip)
if ! command -v unzip >/dev/null 2>&1; then
sudo apt-get install -y unzip
fi
curl -fsSL https://cli.kiro.dev/install | bash
# 3. Write the per-tool config files
mkdir -p ~/.claude ~/.codex ~/.config/opencode ~/.continue ~/.qwen
cat > ~/.claude/settings.json <<JSON
{
"env": {
"ANTHROPIC_BASE_URL": "${OMNIROUTE_ANTHROPIC_URL}",
"ANTHROPIC_AUTH_TOKEN": "${OMNIROUTE_KEY}"
}
}
JSON
cat > ~/.codex/config.yaml <<YAML
model: auto
apiKey: ${OMNIROUTE_KEY}
apiBaseUrl: ${OMNIROUTE_URL}
YAML
cat > ~/.qwen/.env <<ENV
OPENAI_API_KEY="${OMNIROUTE_KEY}"
OPENAI_BASE_URL="${OMNIROUTE_URL}"
OPENAI_MODEL="auto"
ENV
# 4. Append global env vars (idempotent guard)
if ! grep -q "OmniRoute Universal Endpoint" ~/.bashrc 2>/dev/null; then
cat >> ~/.bashrc <<ENV
# OmniRoute Universal Endpoint
export OPENAI_BASE_URL="${OMNIROUTE_URL}"
export OPENAI_API_KEY="${OMNIROUTE_KEY}"
export ANTHROPIC_BASE_URL="${OMNIROUTE_ANTHROPIC_URL}"
export ANTHROPIC_AUTH_TOKEN="${OMNIROUTE_KEY}"
ENV
fi
# 5. Validate via the internal CLI
omniroute doctor || true
omniroute providers list || true
echo "All CLIs installed and configured for OmniRoute"
EOF
chmod +x my-setup.sh
./my-setup.sh
```

377
Cloud-Agent.md Normal file

@@ -0,0 +1,377 @@
# Cloud Agents
> **Source of truth:** `src/lib/cloudAgent/` and `src/app/api/v1/agents/tasks/`
> **Last updated:** 2026-05-13 — v3.8.0
OmniRoute orchestrates third-party cloud-hosted coding agents (Codex Cloud, Devin,
Jules) as long-running tasks. Each agent is wrapped behind a uniform interface so
clients can submit a prompt + repo URL and receive results without dealing with
provider-specific APIs.
A Cloud Agent task is **not** a regular chat completion. It is a durable, multi-step
unit of work that may take minutes to hours, can produce a Pull Request as its
artifact, and supports follow-up messages and (in some providers) plan approval gates.
![Cloud Agent task lifecycle](../diagrams/exported/cloud-agent-flow.svg)
> Source: [diagrams/cloud-agent-flow.mmd](../diagrams/cloud-agent-flow.mmd)
## Supported Agents
| Provider ID | Class | Source | Upstream Base URL | Plan Approval |
| ------------- | ----------------- | ------------------------------------ | --------------------------------------- | ------------- |
| `jules` | `JulesAgent` | `src/lib/cloudAgent/agents/jules.ts` | `https://jules.googleapis.com/v1alpha` | Yes |
| `devin` | `DevinAgent` | `src/lib/cloudAgent/agents/devin.ts` | `https://api.devin.ai/v1` | Yes |
| `codex-cloud` | `CodexCloudAgent` | `src/lib/cloudAgent/agents/codex.ts` | `https://api.openai.com/v1/codex/cloud` | No (auto) |
Registry: `src/lib/cloudAgent/registry.ts` — exports `getAgent(providerId)`,
`getAvailableAgents()`, and `isCloudAgentProvider(providerId)`. The registry is a
plain in-memory `Record<string, CloudAgentBase>` populated at module load.
## Architecture
```
Client (Dashboard / CLI / API)
→ POST /api/v1/agents/tasks (management auth required)
→ CreateCloudAgentTaskSchema validation (Zod)
→ registry.getAgent(providerId)
→ getCloudAgentCredentials(providerId)
└─ pulls from getProviderConnections({ provider, isActive: true })
(apiKey first, fallback to accessToken)
→ agent.createTask({ prompt, source, options }, credentials)
└─ HTTP POST to upstream provider API
└─ returns CloudAgentTask with internal id + externalId
→ insertCloudAgentTask(...) into cloud_agent_tasks (SQLite)
Polling (lazy sync on read):
GET /api/v1/agents/tasks/[id]
→ getCloudAgentTaskById(id)
→ agent.getStatus(externalId, credentials) // refreshes status + activities
→ updateCloudAgentTask(...) with new status, result, completed_at
→ return serialized task
Interactions:
POST /api/v1/agents/tasks/[id] body: { action: "approve" | "message" | "cancel" }
→ agent.approvePlan(externalId, credentials) for "approve"
→ agent.sendMessage(externalId, message, credentials) for "message"
→ status flips to "cancelled" for "cancel" (local-only)
```
Sync is **lazy**: status is refreshed from the upstream on every `GET /tasks/[id]`.
There is no background poller. Dashboards that need fresh state should poll the GET
endpoint at a sensible interval.
## `CloudAgentBase` Interface
Source: `src/lib/cloudAgent/baseAgent.ts`
```typescript
export interface AgentCredentials {
apiKey: string;
baseUrl?: string;
}
export interface CreateTaskParams {
prompt: string;
source: CloudAgentSource;
options: {
autoCreatePr?: boolean;
planApprovalRequired?: boolean;
environment?: Record<string, string>;
};
}
export interface GetStatusResult {
status: CloudAgentStatus;
externalId?: string;
result?: CloudAgentResult;
activities: CloudAgentActivity[];
error?: string;
}
export abstract class CloudAgentBase {
abstract readonly providerId: string;
abstract readonly baseUrl: string;
abstract createTask(p: CreateTaskParams, c: AgentCredentials): Promise<CloudAgentTask>;
abstract getStatus(externalId: string, c: AgentCredentials): Promise<GetStatusResult>;
abstract approvePlan(externalId: string, c: AgentCredentials): Promise<void>;
abstract sendMessage(
externalId: string,
message: string,
c: AgentCredentials
): Promise<CloudAgentActivity>;
abstract listSources(
c: AgentCredentials
): Promise<{ name: string; url: string; branch?: string }[]>;
protected mapStatus(raw: string): CloudAgentStatus; // heuristic upstream-string → enum
protected generateTaskId(): string; // `task_<ts>_<rand>`
protected generateActivityId(): string; // `act_<ts>_<rand>`
}
```
`CodexCloudAgent.approvePlan` intentionally throws — Codex Cloud auto-plans and has
no approval gate. `CodexCloudAgent.listSources` returns `[]`.
## Domain Types
Source: `src/lib/cloudAgent/types.ts`
```typescript
export const CLOUD_AGENT_STATUS = {
QUEUED: "queued",
RUNNING: "running",
AWAITING_APPROVAL: "awaiting_approval",
COMPLETED: "completed",
FAILED: "failed",
CANCELLED: "cancelled",
} as const;
export interface CloudAgentSource {
repoName: string;
repoUrl: string; // must be a valid URL
branch?: string;
}
export interface CloudAgentResult {
prUrl?: string;
prNumber?: number;
commitMessage?: string;
diffUrl?: string;
summary?: string;
duration?: number; // seconds, positive int
cost?: number; // positive float
}
export interface CloudAgentActivity {
id: string;
type: "plan" | "command" | "code_change" | "message" | "error" | "completion";
content: string;
timestamp: string; // ISO 8601
metadata?: Record<string, unknown>;
}
export interface CloudAgentTask {
id: string; // internal `task_...` id
providerId: "jules" | "devin" | "codex-cloud";
externalId?: string; // upstream provider's id
status: CloudAgentStatus;
prompt: string; // 1..10000 chars
source: CloudAgentSource;
options: {
autoCreatePr?: boolean;
planApprovalRequired?: boolean;
environment?: Record<string, string>;
};
result?: CloudAgentResult;
activities: CloudAgentActivity[];
error?: string;
createdAt: string;
updatedAt: string;
completedAt?: string;
}
```
Validation schemas (`CreateCloudAgentTaskSchema`, `UpdateCloudAgentTaskSchema`) are
exported alongside the types and are used by the route handlers.
## Database
Source: `src/lib/cloudAgent/db.ts` — table is created lazily via
`createCloudAgentTaskTable()` (also called from `src/lib/cloudAgent/index.ts` at
module import).
```sql
CREATE TABLE IF NOT EXISTS cloud_agent_tasks (
id TEXT PRIMARY KEY,
provider_id TEXT NOT NULL,
external_id TEXT,
status TEXT NOT NULL DEFAULT 'queued',
prompt TEXT NOT NULL,
source TEXT NOT NULL, -- JSON
options TEXT DEFAULT '{}', -- JSON
result TEXT, -- JSON
activities TEXT DEFAULT '[]', -- JSON
error TEXT,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
completed_at TEXT
);
CREATE INDEX IF NOT EXISTS idx_cloud_agent_tasks_provider ON cloud_agent_tasks(provider_id);
CREATE INDEX IF NOT EXISTS idx_cloud_agent_tasks_status ON cloud_agent_tasks(status);
CREATE INDEX IF NOT EXISTS idx_cloud_agent_tasks_created ON cloud_agent_tasks(created_at DESC);
```
`updateCloudAgentTask` enforces a **column whitelist** to prevent SQL injection:
`status`, `prompt`, `source`, `options`, `result`, `activities`, `error`,
`completed_at`. Any other key in the partial update is silently dropped.
## REST API — Task Lifecycle
**Auth:** All `/api/v1/agents/tasks*` endpoints require **management auth**
(`requireCloudAgentManagementAuth` wraps `requireManagementAuth` from
`src/lib/api/requireManagementAuth`). This is enforced after commit `588a0333`
(_"fix(auth): require management auth for agent and cooldown APIs"_).
| Method | Path | Purpose |
| ------- | ----------------------------- | ------------------------------------------------------ |
| OPTIONS | `/api/v1/agents/tasks` | CORS preflight |
| GET | `/api/v1/agents/tasks` | List tasks (filter: `provider`, `status`, `limit≤500`) |
| POST | `/api/v1/agents/tasks` | Create task (dispatches to upstream + persists) |
| DELETE | `/api/v1/agents/tasks?id=...` | Delete task by query id (does **not** cancel upstream) |
| OPTIONS | `/api/v1/agents/tasks/[id]` | CORS preflight |
| GET | `/api/v1/agents/tasks/[id]` | Read task + lazy-sync status from upstream |
| POST | `/api/v1/agents/tasks/[id]` | Action: `approve` / `message` / `cancel` |
| DELETE | `/api/v1/agents/tasks/[id]` | Delete task by path id |
### Create task
```bash
curl -X POST http://localhost:20128/api/v1/agents/tasks \
-H "Cookie: auth_token=..." \
-H "Content-Type: application/json" \
-d '{
"providerId": "devin",
"prompt": "Fix the bug in src/foo.ts where the parser returns null",
"source": {
"repoName": "user/repo",
"repoUrl": "https://github.com/user/repo",
"branch": "main"
},
"options": {
"autoCreatePr": true,
"planApprovalRequired": false
}
}'
```
Response `201`:
```json
{
"data": {
"id": "task_1731512345678_abc123def",
"providerId": "devin",
"externalId": "session_xyz",
"status": "queued",
"prompt": "...",
"source": { "repoName": "user/repo", "repoUrl": "...", "branch": "main" },
"options": { "autoCreatePr": true },
"createdAt": "2026-05-13T12:34:56.789Z"
}
}
```
### Approve a plan
```bash
curl -X POST http://localhost:20128/api/v1/agents/tasks/<id> \
-H "Cookie: auth_token=..." \
-H "Content-Type: application/json" \
-d '{"action":"approve"}'
```
### Send a follow-up message
```bash
curl -X POST http://localhost:20128/api/v1/agents/tasks/<id> \
-d '{"action":"message","message":"Also add a unit test for the parser"}'
```
### Cancel (local status only)
```bash
curl -X POST http://localhost:20128/api/v1/agents/tasks/<id> \
-d '{"action":"cancel"}'
```
`cancel` flips `status` to `"cancelled"` in the local DB but does **not** call the
upstream provider — there is no abort RPC in `CloudAgentBase`. To stop billing
upstream, terminate the task in the provider's own console.
## REST API — Cloud Provider Plumbing
These auxiliary endpoints under `src/app/api/cloud/` are used by remote clients
(the CLI, the Electron app, or sync workers) to read provider connection metadata
and resolve model aliases. They are authenticated with a **regular API key**
(via `validateApiKey`), not the management auth used by the task endpoints.
| Method | Path | Purpose |
| ------ | ------------------------------- | ------------------------------------------------------------------- |
| POST | `/api/cloud/auth` | Validate API key, return masked connection metadata + model aliases |
| PUT | `/api/cloud/credentials/update` | Refresh `accessToken` / `refreshToken` / `expiresAt` |
| POST | `/api/cloud/model/resolve` | Resolve a model alias to `{ provider, model }` |
| GET | `/api/cloud/models/alias` | List all model aliases |
| PUT | `/api/cloud/models/alias` | Set a model alias (and auto-sync to Cloud if enabled) |
`/api/cloud/auth` never returns raw `apiKey` / `accessToken` / `refreshToken`. It
returns `hasApiKey`, `hasAccessToken`, `hasRefreshToken`, and a masked preview
(`maskedApiKey`: first 4 + `****` + last 4).
## Credentials Resolution
`getCloudAgentCredentials(providerId)` in `src/lib/cloudAgent/api.ts`:
1. Loads active provider connections via `getProviderConnections({ provider: providerId, isActive: true })`.
2. For each connection, prefers `apiKey` (trimmed). Falls back to `accessToken`.
3. Returns the first non-empty token wrapped as `{ apiKey: token }`.
4. Returns `null` if no usable token is found — the API responds `400` with
`"No active credentials configured for cloud agent provider: <id>"`.
This means Cloud Agents reuse the same Provider Connection table as regular LLM
providers. To enable Jules, create an active connection with `provider: "jules"`
and a populated `apiKey`.
## Dashboard
Source: `src/app/(dashboard)/dashboard/cloud-agents/page.tsx`
A `"use client"` React page that:
- Lists tasks (polled via `GET /api/v1/agents/tasks`).
- Submits new tasks via a form that maps to `CreateCloudAgentTaskSchema`.
- Shows status badges (`queued`, `running`, `awaiting_approval`, `completed`,
`failed`, `cancelled`) and renders the `activities[]` timeline.
- Surfaces the `result.prUrl` / `commitMessage` / `summary` when `status === "completed"`.
## Integration with A2A
Cloud Agents can be exposed as A2A skills by registering an A2A skill that delegates
its `tasks/send` handler to `getAgent(...).createTask(...)` and translates A2A task
status events to the JSON-RPC 2.0 protocol. See [A2A-SERVER.md](./A2A-SERVER.md).
## Adding a New Cloud Agent
1. Create `src/lib/cloudAgent/agents/<name>.ts` extending `CloudAgentBase`.
2. Implement `createTask`, `getStatus`, `approvePlan` (or throw if N/A),
`sendMessage`, `listSources`. Use `this.mapStatus(...)` for status normalization.
3. Register in `src/lib/cloudAgent/registry.ts` under a stable `providerId`.
4. Extend the `providerId` literal union in `src/lib/cloudAgent/types.ts`
(`CloudAgentTask.providerId` and `CreateCloudAgentTaskSchema`).
5. Add the provider to `src/shared/constants/providers.ts` if it needs a connection
record. OAuth-based providers also need `src/lib/oauth/providers/`.
6. Add tests under `tests/unit/cloud-agent-*.test.ts`.
7. Update this doc and the dashboard's `CLOUD_AGENTS` constant.
## Configuration
| Env Var | Purpose |
| ---------------- | ----------------------------------------------------------- |
| `DATA_DIR` | Location of the SQLite database holding `cloud_agent_tasks` |
| `JWT_SECRET` | Required for management auth on task endpoints |
| `API_KEY_SECRET` | Required to encrypt provider connection credentials at rest |
No Cloud-Agent-specific env vars exist today — every secret lives in the
`provider_connections` table.
## See Also
- [A2A-SERVER.md](./A2A-SERVER.md)
- [API_REFERENCE.md](../reference/API_REFERENCE.md)
- [SKILLS.md](./SKILLS.md)
- [MEMORY.md](./MEMORY.md)
- Source: `src/lib/cloudAgent/`
- Routes: `src/app/api/v1/agents/tasks/`, `src/app/api/cloud/`
- Dashboard: `src/app/(dashboard)/dashboard/cloud-agents/page.tsx`

800
Codebase-Documentation.md Normal file

@@ -0,0 +1,800 @@
# OmniRoute Codebase Documentation
> **Version:** v3.8.0
> **Last updated:** 2026-05-13
> **Audience:** Engineers contributing to OmniRoute or building integrations on top of it.
>
> For high-level architecture diagrams and the reasoning behind each subsystem, read
> [ARCHITECTURE.md](./ARCHITECTURE.md). For deep dives on individual subsystems
> (Auto Combo, MCP server, A2A server, Skills, Memory, Cloud Agents, Resilience,
> Compression, etc.) see their dedicated files in this `docs/` directory.
This file describes **what exists in the repository today** so that a new engineer
can navigate the tree, understand the runtime layering, and know where to add code
without inventing new modules.
---
## 1. Tech Stack
| Concern | Choice |
| ------------- | ------------------------------------------------------------------------------------------------------------------------ | --- | ------------- | --- | ------------------------------------ |
| Web framework | **Next.js 16** (App Router, standalone output, no global middleware) |
| Language | **TypeScript 5.9+** — target `ES2022`, `module: esnext`, `moduleResolution: bundler`, `strict: false` |
| Runtime | **Node.js** `>=20.20.2 <21 | | >=22.22.2 <23 | | >=24.0.0 <27`(enforced via`engines`) |
| Database | **SQLite** via `better-sqlite3` (singleton, WAL journaling) |
| Desktop | **Electron 41** + `electron-builder` 26.10 (separate workspace at `electron/`) |
| Tests | **Node native test runner** (unit/integration), **Vitest** (MCP, autoCombo, cache), **Playwright** (e2e + protocols-e2e) |
| Build | Next.js standalone via `scripts/build/build-next-isolated.mjs` |
| Lint/format | ESLint flat config + Prettier (`lint-staged` via Husky pre-commit) |
| Module system | ESM everywhere (`"type": "module"`) |
| Workspaces | npm workspace — `open-sse` is the only sub-workspace |
Path aliases (`tsconfig.json`):
- `@/*` → `src/*`
- `@omniroute/open-sse` → `open-sse/index.ts`
- `@omniroute/open-sse/*` → `open-sse/*`
Default HTTP port: **`20128`** (API and dashboard share the same process). Data
directory is `DATA_DIR` env var, defaulting to `~/.omniroute/`.
---
## 2. Repository Layout
```
OmniRoute/
├── src/ Next.js application (App Router, libs, domain, server, shared)
├── open-sse/ Streaming engine workspace (@omniroute/open-sse)
├── electron/ Desktop wrapper (Electron 41 main + preload)
├── bin/ CLI entry points (omniroute, reset-password)
├── tests/ Unit, integration, e2e, protocols-e2e, translator, security, fixtures
├── scripts/ Build, sync, check, migration, and runtime helper scripts
├── docs/ Public documentation (this directory)
├── public/ Static assets, PWA manifest, service worker
├── config/ Runtime config samples
├── images/ Marketing/screenshot assets
├── _ideia/, _references/, _mono_repo/, _tasks/ Internal scratch / planning (not shipped)
├── CLAUDE.md Repo rules for Claude Code
├── AGENTS.md Deeper architecture reference for agents
├── package.json v3.8.0, workspace root
└── tsconfig.json Path aliases + core compiler options
```
---
## 3. `src/` — Next.js Application
```
src/
├── app/ App Router pages + API routes
├── lib/ Core libraries (DB, auth, OAuth, skills, memory, …)
├── domain/ Pure domain layer (policy, fallback, cost, lockout, …)
├── server/ Server-only modules (authz, cors, auth)
├── shared/ Types, constants, validation, contracts, utils (cross-boundary safe)
├── mitm/ Man-in-the-middle proxy helpers for CLI integration
├── models/ Local model metadata / aliasing
├── sse/ Legacy SSE handlers that still live under src/ (not open-sse/)
├── store/ Client-side state stores
├── middleware/ Route-level middleware utilities (not Next.js global middleware)
├── scripts/ In-tree scripts importable by app code
├── types/ Ambient and shared TS types
├── i18n/ Locale bundles
├── instrumentation.ts Next.js instrumentation hook
├── instrumentation-node.ts
├── server-init.ts Process-level bootstrap (env, DB, jobs, sync)
└── proxy.ts Top-level proxy bootstrap helper
```
### 3.1 `src/app/` — App Router
The App Router exposes both the dashboard UI and the public/management HTTP API.
There is **no global middleware** — interception is done per-route.
Top-level segments under `src/app/`:
| Path | Purpose |
| ----------------------------------------------------------------------------- | ----------------------------------------- |
| `api/` | All HTTP API routes (see breakdown below) |
| `a2a/` | A2A JSON-RPC 2.0 endpoint (`POST /a2a`) |
| `.well-known/agent.json/` | A2A Agent Card discovery document |
| `(dashboard)/` | Dashboard UI (route group, no URL prefix) |
| `auth/`, `login/`, `forgot-password/`, `callback/` | Auth flows |
| `landing/` | Marketing/landing page |
| `docs/` | Embedded API docs viewer |
| `status/`, `maintenance/`, `offline/` | Operational pages |
| `privacy/`, `terms/` | Legal pages |
| `400/`, `401/`, `403/`, `408/`, `429/`, `500/`, `502/`, `503/` | Static error pages |
| `error.tsx`, `global-error.tsx`, `not-found.tsx`, `forbidden/`, `loading.tsx` | Framework error/loading boundaries |
| `layout.tsx`, `page.tsx`, `globals.css`, `manifest.ts` | Root shell |
#### 3.1.1 `src/app/(dashboard)/dashboard/` — UI pages
`agents`, `analytics`, `api-manager`, `audit`, `auto-combo`, `batch`, `cache`,
`changelog`, `cli-tools`, `cloud-agents`, `combos`, `compression`, `context`,
`costs`, `endpoint`, `health`, `limits`, `logs`, `memory`, `onboarding`,
`playground`, `providers`, `search-tools`, `settings`, `skills`, `system`,
`translator`, `usage`, `webhooks`, plus root `page.tsx`, `HomePageClient.tsx`,
`BootstrapBanner.tsx`.
#### 3.1.2 `src/app/api/` — Top-level API groups
```
src/app/api/
├── a2a/{status, tasks}
├── acp/
├── admin/
├── analytics/
├── assess/
├── auth/
├── batches/
├── cache/
├── cli-tools/
├── cloud/{codex-responses-ws}
├── combos/
├── compliance/
├── compression/
├── context/
├── db/, db-backups/
├── evals/
├── fallback/
├── files/
├── health/
├── init/
├── internal/{concurrency}
├── keys/
├── logs/
├── mcp/{audit, sse, status, stream, tools}
├── memory/{health, [id]/, route.ts}
├── model-combo-mappings/
├── models/
├── monitoring/
├── oauth/
├── openapi/
├── policies/
├── pricing/
├── provider-metrics/, provider-models/, provider-nodes/
├── providers/
├── rate-limit/, rate-limits/
├── resilience/
├── restart/, shutdown/
├── search/
├── sessions/
├── settings/
├── skills/{executions, [id], install, marketplace, route.ts, skillssh}
├── storage/
├── sync/, synced-available-models/
├── system/
├── tags/
├── telemetry/
├── token-health/
├── translator/
├── tunnels/
├── upstream-proxy/
├── usage/
├── v1/ OpenAI-compatible public API
├── v1beta/ Gemini-style compat
├── version-manager/
└── webhooks/
```
#### 3.1.3 `src/app/api/v1/` — OpenAI-compatible public API
```
v1/
├── accounts/[id]/ account lookup
├── agents/tasks/[id]/, agents/tasks/ A2A-flavored task endpoints
├── api/ internal API helpers exposed under v1/api
├── audio/{speech, transcriptions}/ TTS + STT
├── batches/[id]/{cancel}, batches/ OpenAI Batches API
├── chat/completions/ Chat Completions (the main endpoint)
├── chatgpt-web/ ChatGPT-Web compat
├── completions/ Legacy text completions
├── embeddings/ Embeddings
├── files/[id]/, files/ Files API
├── _helpers/ Shared route helpers (no public URL)
├── images/{edits, generations}/ Image gen + edit
├── issues/ Triage helper endpoints
├── management/{proxies}/ Management-scoped routes inside v1
├── messages/{count_tokens}/ Anthropic-style messages compat
├── models/ Model listing (`route.ts`, `catalog.ts`)
├── moderations/ Moderation
├── music/ Music gen
├── providers/[provider]/ Per-provider operations
├── quotas/{check} Quota probes
├── registered-keys/ Registered key admin
├── rerank/ Reranking
├── responses/[...path]/ OpenAI Responses API (catch-all)
├── search/ Web search
├── videos/ Video gen
├── ws/ WebSocket bridge
└── route.ts Index handler
```
Every route file follows the same pattern:
```
Route → CORS preflight → Zod body validation → optional auth
→ API key policy enforcement → handler delegation (open-sse)
```
`v1beta/` is the Gemini-style compat surface (a thin wrapper that translates into
the same `open-sse/handlers/` pipeline).
### 3.2 `src/lib/` — Core libraries
Always import data, sync, OAuth, skill, memory, etc. through these modules. The
table groups the actual directories and notable top-level files.
| Module | Purpose |
| ----------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `a2a/` | A2A protocol server: `taskManager.ts`, `streaming.ts`, `taskExecution.ts`, `routingLogger.ts`, `skills/` (5 skills: cost analysis, health report, provider discovery, quota management, smart routing) |
| `acp/` | Agent-Control-Protocol: `index.ts`, `manager.ts`, `registry.ts` |
| `api/` | Internal API helpers: `requireManagementAuth.ts`, `requireCliToolsAuth.ts`, `errorResponse.ts` |
| `auth/` | `managementPassword.ts` (password reset / hashing) |
| `batches/` | OpenAI Batches API service (`service.ts`) |
| `catalog/` | OpenRouter catalog sync (`openrouterCatalog.ts`) |
| `cloudAgent/` | Cloud agent registry: `api.ts`, `baseAgent.ts`, `db.ts`, `index.ts`, `registry.ts`, `types.ts`, `agents/{codex, devin, jules}.ts` |
| `combos/` | Combo resolution helpers |
| `compliance/` | Audit + provider audit: `index.ts`, `providerAudit.ts` |
| `config/` | Runtime config glue |
| `db/` | SQLite domain modules (see §3.2.1) |
| `display/` | UI/display helpers used by API responses |
| `embeddings/` | Embedding service registry |
| `env/` | Env loading + introspection |
| `evals/` | Eval runtime |
| `guardrails/` | `piiMasker.ts`, `promptInjection.ts`, `visionBridge.ts`, `visionBridgeHelpers.ts`, `registry.ts`, `base.ts` |
| `jobs/` | Background jobs (`autoUpdate.ts`, …) |
| `memory/` | Persistent memory: `store.ts`, `cache.ts`, `retrieval.ts`, `summarization.ts`, `extraction.ts`, `injection.ts`, `qdrant.ts`, `settings.ts`, `verify.ts`, `schemas.ts`, `types.ts` |
| `monitoring/` | `observability.ts` |
| `oauth/` | OAuth providers (14): `antigravity`, `claude`, `cline`, `codex`, `cursor`, `gemini`, `github`, `gitlab-duo`, `kilocode`, `kimi-coding`, `kiro`, `qoder`, `qwen`, `windsurf` plus `services/`, `utils/{pkce, server, banner, codexAuthFile, ui}`, `constants/oauth.ts` |
| `plugins/` | Plugin loader (`index.ts`) |
| `promptCache/` | `prefixAnalyzer.ts`, `index.ts` |
| `providerModels/` | Managed model lifecycle: `modelDiscovery.ts`, `managedModelImport.ts`, `managedAvailableModels.ts`, `cursorAgent.ts` |
| `providers/` | Provider helpers: `catalog.ts`, `validation.ts`, `imageValidation.ts`, `claudeExtraUsage.ts`, `codexConnectionDefaults.ts`, `codexFastTier.ts`, `webCookieAuth.ts`, `managedAvailableModels.ts`, `requestDefaults.ts` |
| `resilience/` | `settings.ts` — settings for circuit breaker, cooldown, lockout |
| `runtime/` | Runtime feature detection |
| `search/` | `executeWebSearch.ts` |
| `skills/` | Skill framework: `registry.ts`, `executor.ts`, `interception.ts`, `injection.ts`, `sandbox.ts`, `custom.ts`, `hybrid.ts`, `builtins.ts`, `a2a.ts`, `providerSettings.ts`, `schemas.ts`, `skillssh.ts`, `types.ts`, plus `builtin/browser.ts` |
| `spend/` | `batchWriter.ts` (write-behind buffer) |
| `sync/` | `bundle.ts`, `tokens.ts` (Cloud Sync) |
| `system/` | System-level helpers |
| `translator/` | Top-level translator glue (delegates into `open-sse/translator/`) |
| `usage/` | Usage accounting: `costCalculator.ts`, `tokenAccounting.ts`, `usageHistory.ts`, `aggregateHistory.ts`, `usageStats.ts`, `callLogs.ts`, `callLogArtifacts.ts`, `fetcher.ts`, `providerLimits.ts`, `migrations.ts` |
| `versionManager/` | Auto-update + version manifest |
| `ws/` | WebSocket bridge |
| `zed-oauth/` | Zed editor OAuth flow |
Top-level files in `src/lib/`:
- `localDb.ts` — re-export layer only. **Never** add logic here.
- `proxyHealth.ts`, `proxyLogger.ts`, `tokenHealthCheck.ts`, `localHealthCheck.ts`
- `oneproxyRotator.ts`, `oneproxySync.ts`
- `apiBridgeServer.ts`, `cacheLayer.ts`, `semanticCache.ts`, `settingsCache.ts`
- `cloudSync.ts`, `initCloudSync.ts`
- `cloudflaredTunnel.ts`, `ngrokTunnel.ts`, `tailscaleTunnel.ts`
- `consoleInterceptor.ts`, `container.ts`, `gracefulShutdown.ts`, `idempotencyLayer.ts`
- `ipUtils.ts`, `logEnv.ts`, `logPayloads.ts`, `logRotation.ts`
- `modelAliasSeed.ts`, `modelCapabilities.ts`, `modelMetadataRegistry.ts`, `modelsDevSync.ts`
- `piiSanitizer.ts`, `pricingSync.ts`
- `apiKeyExposure.ts`, `cacheControlSettings.ts`, `dataPaths.ts`, `toolPolicy.ts`
- `translatorEvents.ts`, `usageDb.ts`, `usageAnalytics.ts`, `webhookDispatcher.ts`
#### 3.2.1 `src/lib/db/`
Singleton SQLite database (`getDbInstance()` in `core.ts`, WAL journaling).
**Never write raw SQL in routes or handlers** — go through these modules.
![Database schema overview (selected core tables)](../diagrams/exported/db-schema-overview.svg)
> Source: [diagrams/db-schema-overview.mmd](../diagrams/db-schema-overview.mmd)
Domain modules (each owns one or more tables): `apiKeys.ts`, `backup.ts`,
`batches.ts`, `cleanup.ts`, `cliToolState.ts`, `combos.ts`,
`commandCodeAuth.ts`, `compression.ts`, `compressionAnalytics.ts`,
`compressionCacheStats.ts`, `compressionCombos.ts`, `compressionScheduler.ts`,
`contextHandoffs.ts`, `core.ts`, `creditBalance.ts`, `databaseSettings.ts`,
`detailedLogs.ts`, `domainState.ts`, `encryption.ts`, `evals.ts`, `files.ts`,
`healthCheck.ts`, `jsonMigration.ts`, `migrationRunner.ts`,
`modelComboMappings.ts`, `models.ts`, `oneproxy.ts`, `prompts.ts`,
`providers.ts`, `providerLimits.ts`, `proxies.ts`, `quotaSnapshots.ts`,
`readCache.ts`, `reasoningCache.ts`, `registeredKeys.ts`, `secrets.ts`,
`sessionAccountAffinity.ts`, `settings.ts`, `stateReset.ts`, `stats.ts`,
`syncTokens.ts`, `tierConfig.ts`, `upstreamProxy.ts`, `versionManager.ts`,
`webhooks.ts`.
`migrations/` holds 55 versioned `.sql` files (idempotent, transactional) and is
executed by `migrationRunner.ts` at boot.
Tables created across the migrations (52 total):
`a`, `account_key_limits`, `api_keys`, `batches`, `call_logs`,
`combo_adaptation_state`, `combos`, `command_code_auth_sessions`,
`compression_analytics`, `compression_cache_stats`,
`compression_combo_assignments`, `compression_combos`, `context_handoffs`,
`daily_usage_summary`, `db_meta`, `domain_budgets`, `domain_circuit_breakers`,
`domain_cost_history`, `domain_fallback_chains`, `domain_lockout_state`,
`eval_cases`, `eval_runs`, `eval_suites`, `files`, `hourly_usage_summary`,
`key_value`, `mcp_tool_audit`, `memories`, `model_combo_mappings`,
`provider_connections`, `provider_key_limits`, `provider_nodes`,
`proxy_assignments`, `proxy_logs`, `proxy_registry`, `quota_snapshots`,
`reasoning_cache`, `registered_keys`, `request_detail_logs`,
`routing_decisions`, `semantic_cache`, `session_account_affinity`,
`skill_executions`, `skills`, `sync_tokens`, `tier_assignments`,
`tier_config`, `upstream_proxy_config`, `usage_history`, `version_manager`,
`webhooks` (plus FTS5 virtual tables for memory search).
### 3.3 `src/domain/` — Domain layer
Pure business logic, no I/O. Imported by routes and handlers.
| File | Purpose |
| ------------------------------------------ | ------------------------------------------------- |
| `policyEngine.ts` | Top-level policy resolver |
| `fallbackPolicy.ts` | Fallback decision tree |
| `costRules.ts` | Cost calculation rules |
| `lockoutPolicy.ts` | Model lockout decisions |
| `tagRouter.ts` | Tag-based routing |
| `comboResolver.ts` | Combo resolution from request → target list |
| `connectionModelRules.ts` | Per-connection model filters |
| `modelAvailability.ts` | Model availability check |
| `degradation.ts` | Degraded-mode transitions |
| `providerExpiration.ts` | Expired account/key detection |
| `quotaCache.ts` | Cached quota decisions |
| `responses.ts`, `omnirouteResponseMeta.ts` | Response shape helpers |
| `configAudit.ts` | Config change audit |
| `assessment/` | Model assessment (per RFC, partially implemented) |
| `types.ts` | Shared domain types |
### 3.4 `src/server/` — Server-only
Cannot be imported from client components.
```
server/
├── auth/loginGuard.ts
├── authz/
│ ├── classify.ts Classifies routes as public vs management
│ ├── assertAuth.ts Assertion helper
│ ├── context.ts Per-request authz context
│ ├── headers.ts
│ ├── pipeline.ts Authz pipeline
│ ├── policies/ Concrete policies
│ └── types.ts
└── cors/origins.ts CORS origin allowlist
```
### 3.5 `src/shared/` — Safe-to-share
Split into focused subdirectories:
- `constants/` — `providers.ts` (Zod-validated provider catalog), `models.ts`,
`modelSpecs.ts`, `modelCompat.ts`, `pricing.ts`, `cliTools.ts`,
`cliCompatProviders.ts`, `routingStrategies.ts`, `comboConfigMode.ts`,
`headers.ts`, `upstreamHeaders.ts` (denylist), `mcpScopes.ts`,
`errorCodes.ts`, `publicApiRoutes.ts`, `batch.ts`, `batchEndpoints.ts`,
`bodySize.ts`, `colors.ts`, `appConfig.ts`, `config.ts`,
`sidebarVisibility.ts`, `visionBridgeDefaults.ts`.
- `validation/` — `schemas.ts` (~80 Zod schemas), `compressionConfigSchemas.ts`,
`oneproxySchemas.ts`, `providerSchema.ts`, `settingsSchemas.ts`, `helpers.ts`.
- `contracts/` — public API contracts shipped to npm.
- `types/` — shared TS types.
- `utils/` — `circuitBreaker.ts`, `apiAuth.ts`, `apiKey.ts`, `apiKeyPolicy.ts`,
`apiResponse.ts`, `api.ts`, `classify429.ts`, `cliCompat.ts`, `clipboard.ts`,
`cloud.ts`, `cn.ts`, `cors.ts`, `costEstimator.ts`, `featureFlags.ts`,
`fetchTimeout.ts`, `formatting.ts`, `inputSanitizer.ts`, `logger.ts`,
`machine.ts`, `machineId.ts`, `maskEmail.ts`, `modelCatalogSearch.ts`,
`nodeRuntimeSupport.ts`, `parseApiKeys.ts`, `providerHints.ts`,
`providerModelAliases.ts`, `rateLimiter.ts`, `releaseNotes.ts`,
`a11yAudit.ts`, plus dashboard hooks/components under `services/`, `network/`,
`middleware/`, `schemas/`, `hooks/`, `components/`.
---
## 4. `open-sse/` — Streaming engine workspace
Separate npm workspace published as `@omniroute/open-sse`. Owns request
processing, executors, translators, services, transformer, and the MCP server.
```
open-sse/
├── index.ts Public exports
├── package.json Workspace manifest
├── tsconfig.json
├── types.d.ts
├── config/ Provider registries, header profiles, identity, …
├── handlers/ Request handlers (chat, embeddings, audio, image, …)
├── executors/ 38 provider-specific HTTP executors
├── translator/ Format conversion (OpenAI ↔ Claude ↔ Gemini ↔ Cursor ↔ Kiro)
├── transformer/ Responses API ↔ Chat Completions stream transformer
├── services/ 80+ service modules (combos, fallback, quotas, identity, …)
├── utils/ Streaming helpers, TLS client, AWS SigV4, proxy fetch, …
└── mcp-server/ MCP server (3 transports, 13 scopes, 42 tools)
```
### 4.1 `open-sse/handlers/`
| Handler | Purpose |
| ----------------------- | ------------------------------------------------------------------------ |
| `chatCore.ts` | Main chat pipeline (cache, rate limit, combo routing, executor dispatch) |
| `responsesHandler.ts` | OpenAI Responses API entry point |
| `embeddings.ts` | Embeddings |
| `imageGeneration.ts` | Image generation |
| `audioSpeech.ts` | Text-to-speech |
| `audioTranscription.ts` | Speech-to-text |
| `videoGeneration.ts` | Video generation |
| `musicGeneration.ts` | Music generation |
| `rerank.ts` | Reranking |
| `moderations.ts` | Moderation |
| `search.ts` | Web search |
| `sseParser.ts` | SSE event parser |
| `usageExtractor.ts` | Pull token counts out of upstream streams |
| `responseSanitizer.ts` | Strip provider-specific noise |
| `responseTranslator.ts` | Glue between provider response and translator layer |
### 4.2 `open-sse/executors/`
38 provider executors, each extending `BaseExecutor` (`base.ts`):
`antigravity`, `azure-openai`, `blackbox-web`, `chatgpt-web`, `cliproxyapi`,
`cloudflare-ai`, `codex`, `commandCode`, `cursor`, `default`, `devin-cli`,
`gemini-cli`, `github`, `gitlab`, `glm`, `grok-web`, `kie`, `kiro`,
`muse-spark-web`, `nlpcloud`, `opencode`, `perplexity-web`, `petals`,
`pollinations`, `puter`, `qoder`, `vertex`, `windsurf`, plus `claudeIdentity.ts`
(shared identity helper) and `index.ts` (registry).
> Note: providers not listed here are served by `default.ts` using the generic
> OpenAI-compatible executor. The full provider catalog (177+ entries) lives in
> `src/shared/constants/providers.ts`.
### 4.3 `open-sse/translator/`
Hub-and-spoke translation (OpenAI is the hub).
- **9 request translators** (`translator/request/`):
`antigravity-to-openai`, `claude-to-gemini`, `claude-to-openai`,
`gemini-to-openai`, `openai-responses`, `openai-to-claude`,
`openai-to-cursor`, `openai-to-gemini`, `openai-to-kiro`.
- **8 response translators** (`translator/response/`):
`claude-to-openai`, `cursor-to-openai`, `gemini-to-claude`, `gemini-to-openai`,
`kiro-to-openai`, `openai-responses`, `openai-to-antigravity`,
`openai-to-claude`.
- **9 helpers** (`translator/helpers/`):
`claudeHelper`, `geminiHelper`, `geminiToolsSanitizer`, `maxTokensHelper`,
`openaiHelper`, `responsesApiHelper`, `schemaCoercion`, `toolCallHelper`, plus
helper tests.
- **Image helpers** (`translator/image/sizeMapper.ts`).
- Top-level: `bootstrap.ts`, `formats.ts`, `registry.ts`, `index.ts`.
### 4.4 `open-sse/transformer/`
- `responsesTransformer.ts` — `TransformStream`-based Responses API ↔ Chat
Completions converter (used by the `responses/` route catch-all).
### 4.5 `open-sse/services/`
Highlights (full list under `open-sse/services/`):
| Concern | Files |
| ------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| Combo routing | `combo.ts` (14 strategies), `comboConfig.ts`, `comboMetrics.ts`, `comboManifestMetrics.ts`, `comboAgentMiddleware.ts` |
| Auto Combo engine | `autoCombo/` — `engine.ts`, `scoring.ts`, `taskFitness.ts`, `virtualFactory.ts`, `modePacks.ts`, `autoPrefix.ts`, `persistence.ts`, `providerDiversity.ts`, `providerRegistryAccessor.ts`, `routerStrategy.ts`, `selfHealing.ts`, `index.ts` |
| Resilience | `accountFallback.ts` (cooldown + lockout), `errorClassifier.ts`, `emergencyFallback.ts`, `rateLimitManager.ts`, `rateLimitSemaphore.ts`, `accountSemaphore.ts`, `accountSelector.ts` |
| Quotas | `quotaMonitor.ts`, `quotaPreflight.ts`, `bailianQuotaFetcher.ts`, `codexQuotaFetcher.ts`, `deepseekQuotaFetcher.ts`, `crofUsageFetcher.ts`, `antigravityCredits.ts` |
| Provider-specific shaping | `claudeCodeCCH.ts`, `claudeCodeCompatible.ts`, `claudeCodeConstraints.ts`, `claudeCodeExtraRemap.ts`, `claudeCodeFingerprint.ts`, `claudeCodeObfuscation.ts`, `claudeCodeToolRemapper.ts`, `cloudCodeHeaders.ts`, `cloudCodeThinking.ts`, `geminiCliHeaders.ts`, `geminiThoughtSignatureStore.ts`, `gigachatAuth.ts`, `antigravityHeaders.ts`, `antigravityHeaderScrub.ts`, `antigravityIdentity.ts`, `antigravityObfuscation.ts`, `antigravityVersion.ts`, `antigravity429Engine.ts`, `chatgptTlsClient.ts`, `chatgptImageCache.ts`, `cursorSessionManager.ts`, `qoderCli.ts`, `qwenThinking.ts`, `modelscopePolicy.ts` |
| Caching | `reasoningCache.ts`, `searchCache.ts`, `signatureCache.ts`, `requestDedup.ts` |
| Routing intelligence | `intentClassifier.ts`, `taskAwareRouter.ts`, `backgroundTaskDetector.ts`, `volumeDetector.ts`, `wildcardRouter.ts`, `workflowFSM.ts`, `specificityDetector.ts`, `specificityRules.ts`, `specificityTypes.ts` |
| Model handling | `modelCapabilities.ts`, `modelDeprecation.ts`, `modelFamilyFallback.ts`, `modelStrip.ts`, `model.ts`, `provider.ts`, `providerRequestDefaults.ts`, `providerCostData.ts`, `payloadRules.ts` |
| Compression | `compression/` — full compression engine wiring |
| Token + session | `tokenRefresh.ts`, `sessionManager.ts`, `apiKeyRotator.ts`, `contextManager.ts`, `contextHandoff.ts`, `systemPrompt.ts`, `roleNormalizer.ts`, `responsesInputSanitizer.ts`, `toolSchemaSanitizer.ts`, `toolLimitDetector.ts`, `thinkingBudget.ts` |
| Tier / manifest | `tierResolver.ts`, `tierConfig.ts`, `tierDefaults.json`, `tierTypes.ts`, `manifestAdapter.ts` |
| IP / network | `ipFilter.ts`, `webSearchFallback.ts` |
| Batches | `batchProcessor.ts` |
| Usage | `usage.ts` |
### 4.6 `open-sse/mcp-server/`
- **31 registered tools** wired in `server.ts` (12 scoped under `schemas/tools.ts`,
5 compression tools, 3 memory tools, 4 skills tools, plus advanced tools added
through `advancedTools.ts`).
- **3 transports**: stdio, HTTP Streamable, SSE.
- **13 scopes** declared in `src/shared/constants/mcpScopes.ts`.
- Audit table: `mcp_tool_audit` (populated by `audit.ts`).
- Files: `server.ts`, `index.ts`, `httpTransport.ts`, `audit.ts`, `scopeEnforcement.ts`,
`runtimeHeartbeat.ts`, `descriptionCompressor.ts`, `schemas/{tools, a2a, audit, index}.ts`,
`tools/{advancedTools, compressionTools, memoryTools, skillTools}.ts`,
plus tests under `__tests__/`.
- See [MCP-SERVER.md](../frameworks/MCP-SERVER.md) for the full tool catalog.
### 4.7 `open-sse/config/`
Provider registries (`providerRegistry.ts`, `providerModels.ts`,
`providerHeaderProfiles.ts`), per-format model registries (`audioRegistry.ts`,
`embeddingRegistry.ts`, `imageRegistry.ts`, `moderationRegistry.ts`,
`musicRegistry.ts`, `rerankRegistry.ts`, `searchRegistry.ts`, `videoRegistry.ts`),
identity helpers (`codexIdentity.ts`, `codexInstructions.ts`,
`anthropicHeaders.ts`, `antigravityUpstream.ts`, `antigravityModelAliases.ts`,
`cliFingerprints.ts`, `toolCloaking.ts`, `defaultThinkingSignature.ts`),
credential helpers (`credentialLoader.ts`, `codexClient.ts`), and cloud
adapters (`azureAi.ts`, `bedrock.ts`, `datarobot.ts`, `glmProvider.ts`,
`maritalk.ts`, `oci.ts`, `petals.ts`, `runway.ts`, `sap.ts`, `watsonx.ts`,
`ollamaModels.ts`, `errorConfig.ts`, `constants.ts`, `registryUtils.ts`).
### 4.8 `open-sse/utils/`
Streaming primitives and provider helpers: `stream.ts`, `streamHandler.ts`,
`streamHelpers.ts`, `streamPayloadCollector.ts`, `streamReadiness.ts`,
`sseHeartbeat.ts`, `proxyFetch.ts`, `proxyDispatcher.ts`, `tlsClient.ts`,
`networkProxy.ts`, `awsSigV4.ts`, `cacheControlPolicy.ts`,
`cursorChecksum.ts`, `cursorAgentProtobuf.ts`, `cursorVersionDetector.ts`,
`comfyuiClient.ts`, `kieTask.ts`, `bypassHandler.ts`, `aiSdkCompat.ts`,
`thinkTagParser.ts`, `urlSanitize.ts`, `usageTracking.ts`, `requestLogger.ts`,
`progressTracker.ts`, `cors.ts`, `error.ts`, `logger.ts`, `sleep.ts`,
`ollamaTransform.ts`.
---
## 5. `electron/` — Desktop wrapper
```
electron/
├── main.js Electron main process
├── preload.js Preload bridge (contextIsolation enabled)
├── types.d.ts
├── package.json electron-builder config, version 3.8.0
├── README.md
├── assets/ Build resources (icons, entitlements, …)
├── node_modules/ Dedicated node_modules (better-sqlite3, electron-updater)
└── dist-electron/ Build output (not committed)
```
Five npm scripts at the workspace root: `electron:dev`, `electron:build`,
`electron:build:{win,mac,linux}`, `electron:smoke:packaged`. Auto-update is via
`electron-updater` pointing at the GitHub release feed.
---
## 6. `bin/` — CLI
```
bin/
├── omniroute.mjs Main CLI entry (Node ESM)
├── reset-password.mjs Reset the management password from CLI
├── mcp-server.mjs MCP server launcher (stdio)
├── nodeRuntimeSupport.mjs Node version guard
└── cli/
├── program.mjs Commander program builder
├── runtime.mjs withRuntime helper (server-first/db-fallback)
├── output.mjs Output formatters (json/jsonl/table/csv)
├── i18n.mjs t() helper with locales
├── api.mjs API fetch helper
├── data-dir.mjs
├── encryption.mjs
├── sqlite.mjs
└── commands/
├── registry.mjs Command registration
├── setup.mjs
├── doctor.mjs
├── providers.mjs
└── ... (one file per command/group)
```
Two binaries are exposed in `package.json` → `bin`:
- `omniroute` → `bin/omniroute.mjs`
- `omniroute-reset-password` → `bin/reset-password.mjs`
---
## 7. `tests/`
| Directory | Type |
| ------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------ |
| `tests/unit/` | Unit tests via Node native test runner (506 files, plus `api/`, `auth/`, `authz/` subdirs) |
| `tests/integration/` | Cross-module + DB-state tests |
| `tests/e2e/` | Playwright UI tests |
| `tests/protocols-e2e/` | MCP/A2A protocol e2e |
| `tests/translator/` | Translator-specific tests |
| `tests/security/` | Security regressions |
| `tests/load/` | Load / stress tests |
| `tests/golden-set/` | Reference outputs for translator regressions |
| `tests/helpers/`, `tests/fixtures/`, `tests/manual/`, `tests/scratch_test.mjs` | Support |
Common commands:
| Command | What it runs |
| -------------------------------------------------------- | ---------------------------------------------------------------- |
| `npm run test:unit` | All `tests/unit/*.test.ts` via Node test runner (concurrency 10) |
| `npm run test:vitest` | Vitest suite (MCP, autoCombo, cache) |
| `npm run test:e2e` | Playwright UI suite |
| `npm run test:protocols:e2e` | MCP + A2A protocol e2e |
| `npm run test:coverage` | Coverage gate (≥60% lines/statements/functions/branches) |
| `node --import tsx/esm --test tests/unit/<file>.test.ts` | Single file run |
---
## 8. `scripts/`
Organized into 6 subfolders by purpose.
- **`scripts/build/`** — `build-next-isolated.mjs`, `prepublish.ts`,
`prepare-electron-standalone.mjs`, `pack-artifact-policy.ts`,
`validate-pack-artifact.ts`, `postinstall.mjs`, `postinstallSupport.mjs`,
`uninstall.mjs`, `bootstrap-env.mjs`, `runtime-env.mjs`,
`native-binary-compat.mjs`.
- **`scripts/dev/`** — `run-next.mjs`, `run-next-playwright.mjs`,
`run-standalone.mjs`, `standalone-server-ws.mjs`, `responses-ws-proxy.mjs`,
`v1-ws-bridge.mjs`, `smoke-electron-packaged.mjs`,
`run-playwright-tests.mjs`, `run-ecosystem-tests.mjs`,
`run-protocol-clients-tests.mjs`, `sync-env.mjs`, `healthcheck.mjs`,
`system-info.mjs`.
- **`scripts/check/`** — `check-cycles.mjs`, `check-docs-sync.mjs`,
`check-docs-counts-sync.mjs`, `check-env-doc-sync.mjs`,
`check-deprecated-versions.mjs`, `check-route-validation.mjs`,
`check-t11-any-budget.mjs`, `check-pr-test-policy.mjs`,
`check-supported-node-runtime.ts`, `test-report-summary.mjs`.
- **`scripts/docs/`** — `generate-docs-index.mjs`, `gen-provider-reference.ts`.
- **`scripts/i18n/`** — `generate-multilang.mjs`, `run-visual-qa.mjs`,
`generate-qa-checklist.mjs`, `apply-priority-overrides.mjs`,
`validate_translation.py`, `check_translations.py`, `i18n_autotranslate.py`,
`untranslatable-keys.json`.
- **`scripts/ad-hoc/`** — `cursor-tap.cjs`, `sync-cursor-models.mjs`,
`migrate-env.mjs`, `dbsetup.js`.
---
## 9. Request Pipeline (Summary)
![Request pipeline (/v1/chat/completions)](../diagrams/exported/request-pipeline.svg)
> Source: [diagrams/request-pipeline.mmd](../diagrams/request-pipeline.mmd)
```
Client request
→ /v1/chat/completions (route.ts)
CORS preflight check
Zod validation (chatCompletionsSchema in shared/validation/schemas.ts)
Auth (extractApiKey + isValidApiKey OR requireManagementAuth)
Policy engine (src/server/authz/pipeline.ts)
Guardrails (PII masker, prompt injection, vision bridge)
→ handleChatCore() (open-sse/handlers/chatCore.ts)
Cache check (semantic + read cache)
Rate limit (rateLimitManager, accountSemaphore)
Combo routing (if model resolves to a combo)
comboResolver → loop per target → handleSingleModel()
translateRequest() (open-sse/translator/request/*)
getExecutor(providerId).execute() (open-sse/executors/*)
fetch upstream → retry/backoff via accountFallback
translateResponse() (open-sse/translator/response/*)
SSE stream OR JSON response
If Responses API: TransformStream via open-sse/transformer/responsesTransformer.ts
→ Compliance audit (src/lib/compliance/)
→ Response to client
```
### Resilience runtime state (three mechanisms)
| Mechanism | Scope | Where |
| ------------------------ | ----------------------------- | ------------------------------------------------------------------------------------------------------------ |
| Provider circuit breaker | Whole provider | `src/shared/utils/circuitBreaker.ts`, persisted in `domain_circuit_breakers` |
| Connection cooldown | One account/key | `markAccountUnavailable()` in `src/sse/services/auth.ts`; consumed by `accountFallback.checkFallbackError()` |
| Model lockout | Provider + connection + model | `open-sse/services/accountFallback.ts`, persisted in `domain_lockout_state` |
See [RESILIENCE_GUIDE.md](./RESILIENCE_GUIDE.md) and the dedicated section in
[CLAUDE.md](../../CLAUDE.md).
---
## 10. How to Contribute
### Add a new provider
1. Register in `src/shared/constants/providers.ts` (Zod-validated at load).
2. Add an executor in `open-sse/executors/` if custom logic is required
(extend `BaseExecutor`).
3. Add a translator in `open-sse/translator/` if it does not speak OpenAI format.
4. If OAuth-based, add config under `src/lib/oauth/providers/` and
`src/lib/oauth/services/`.
5. Register models in `open-sse/config/providerRegistry.ts` (or the format-specific
registry under `open-sse/config/`).
6. Write tests under `tests/unit/`.
### Add a new API route
1. Create `src/app/api/your-route/route.ts`.
2. Follow the pattern: CORS → Zod body validation → auth → handler delegation.
3. If new request shape: add the Zod schema in `src/shared/validation/schemas.ts`.
4. If management-only: add the path to `src/shared/constants/publicApiRoutes.ts`
(denylist for the public API surface).
5. Add tests under `tests/unit/`.
6. Update `docs/reference/API_REFERENCE.md` and `docs/reference/openapi.yaml`.
### Add a new DB module
1. Create `src/lib/db/yourModule.ts` and import `getDbInstance()` from `./core.ts`.
2. Export CRUD functions for your domain.
3. If new tables: add a migration under `src/lib/db/migrations/`, numbered
sequentially, idempotent, transactional.
4. Re-export from `src/lib/localDb.ts` (re-export only — **no logic**).
5. Add tests under `tests/unit/`.
### Add a new MCP tool
1. Add the tool definition under `open-sse/mcp-server/tools/` (or extend
`open-sse/mcp-server/schemas/tools.ts`).
2. Assign the appropriate scope(s) in `src/shared/constants/mcpScopes.ts`.
3. Register the tool in `open-sse/mcp-server/server.ts`.
4. Add tests under `open-sse/mcp-server/__tests__/`.
5. Update [MCP-SERVER.md](../frameworks/MCP-SERVER.md).
### Add a new A2A skill
See [A2A-SERVER.md § Adding a New Skill](../frameworks/A2A-SERVER.md). Skills live in
`src/lib/a2a/skills/` and are registered through the A2A task manager.
---
## 11. Conventions
- **Code style**: 2-space indent, double quotes, 100 char width, semicolons,
`es5` trailing commas — enforced by Prettier via `lint-staged`.
- **Imports**: external → internal (`@/`, `@omniroute/open-sse`) → relative.
- **Naming**: files `camelCase` or `kebab-case`, components `PascalCase`,
constants `UPPER_SNAKE`.
- **ESLint**: `no-eval`, `no-implied-eval`, `no-new-func` = `error` everywhere;
`no-explicit-any` = `warn` in `open-sse/` and `tests/`, error elsewhere.
- **TypeScript**: `strict: false` (legacy posture). Prefer explicit types over
inference for cross-module boundaries.
- **Database**: never write raw SQL in routes or handlers — always go through
`src/lib/db/` modules. Never add logic to `src/lib/localDb.ts`.
- **Errors**: try/catch with specific error types, log with pino context. Never
silently swallow errors in SSE streams; use abort signals for cleanup.
- **Security**: never use `eval()` / `new Function()` / implied eval. Validate
all inputs with Zod. Encrypt credentials at rest (AES-256-GCM). Keep
`src/shared/constants/upstreamHeaders.ts` denylist aligned with the
sanitize/validation layer.
- **Commits**: Conventional Commits — `feat(scope): subject`. Allowed scopes:
`db`, `sse`, `oauth`, `dashboard`, `api`, `cli`, `docker`, `ci`, `mcp`,
`a2a`, `memory`, `skills`.
- **Branches**: prefixes `feat/`, `fix/`, `refactor/`, `docs/`, `test/`,
`chore/`. Never commit directly to `main`.
- **Husky**: pre-commit runs `lint-staged` + `check:docs-sync` +
`check:any-budget:t11`; pre-push runs `npm run test:unit`.
---
## 12. Hard Rules (from CLAUDE.md)
1. Never commit secrets or credentials.
2. Never add logic to `src/lib/localDb.ts`.
3. Never use `eval()` / `new Function()` / implied eval.
4. Never commit directly to `main`.
5. Never write raw SQL in routes — always go through `src/lib/db/` modules.
6. Never silently swallow errors in SSE streams.
7. Always validate inputs with Zod schemas.
8. Always include tests when changing production code.
9. Coverage must stay ≥ 60% (statements, lines, functions, branches).
---
## 13. See Also
- [ARCHITECTURE.md](./ARCHITECTURE.md) — high-level architecture and module
responsibilities.
- [API_REFERENCE.md](../reference/API_REFERENCE.md) — public + management API reference.
- [FEATURES.md](../guides/FEATURES.md) — feature matrix and version highlights.
- [RESILIENCE_GUIDE.md](./RESILIENCE_GUIDE.md) — circuit breaker, cooldown,
lockout deep dive.
- [AUTO-COMBO.md](../routing/AUTO-COMBO.md) — Auto Combo scoring and strategies.
- [MCP-SERVER.md](../frameworks/MCP-SERVER.md) — full MCP tool catalog + transports.
- [A2A-SERVER.md](../frameworks/A2A-SERVER.md) — A2A protocol skills and discovery.
- [COMPRESSION_GUIDE.md](../compression/COMPRESSION_GUIDE.md) — RTK + Caveman compression.
- [CLI-TOOLS.md](../reference/CLI-TOOLS.md) — CLI integrations.
- [ELECTRON_GUIDE.md](../guides/ELECTRON_GUIDE.md) (if present), [DOCKER_GUIDE.md](../guides/DOCKER_GUIDE.md), [FLY_IO_DEPLOYMENT_GUIDE.md](../ops/FLY_IO_DEPLOYMENT_GUIDE.md), [VM_DEPLOYMENT_GUIDE.md](../ops/VM_DEPLOYMENT_GUIDE.md), [TERMUX_GUIDE.md](../guides/TERMUX_GUIDE.md), [PWA_GUIDE.md](../guides/PWA_GUIDE.md) — deployment targets.
- [TROUBLESHOOTING.md](../guides/TROUBLESHOOTING.md) — common operational issues.
- [CONTRIBUTING.md](../../CONTRIBUTING.md) — contributor workflow.
- [CLAUDE.md](../../CLAUDE.md) — repo rules for Claude Code (the source of truth
for many of the conventions above).
- [AGENTS.md](../../AGENTS.md) — deeper architecture reference used by agents.

225
Compliance.md Normal file

@@ -0,0 +1,225 @@
# Compliance & Audit
> **Source of truth:** `src/lib/compliance/`, `src/app/api/compliance/`
> **Last updated:** 2026-05-13 — v3.8.0
OmniRoute records administrative actions, authentication events, provider
credential lifecycle changes, and MCP tool invocations to SQLite-backed audit
tables. This page covers what gets logged, where it lives, how long it is
retained, how API keys can opt out, and how to query the data.
The implementation lives in `src/lib/compliance/index.ts` (T-43 — "Compliance
Controls") and `src/lib/compliance/providerAudit.ts`. Audit writes never throw:
on any failure the call is silently swallowed so audit logging cannot break the
main request flow.
## What Gets Logged
### Administrative audit events (`audit_log`)
Every call to `logAuditEvent({ action, actor, target, details, ... })` produces
one row. Action strings follow a `domain.verb` (or `domain.verb.outcome`)
pattern. Confirmed in-tree action types include:
| Action | Source |
| ------------------------------------ | --------------------------------------- |
| `auth.login.success` | `src/app/api/auth/login/route.ts` |
| `auth.login.failed` | `src/app/api/auth/login/route.ts` |
| `auth.login.locked` | `src/app/api/auth/login/route.ts` |
| `auth.login.error` | `src/app/api/auth/login/route.ts` |
| `auth.login.misconfigured` | `src/app/api/auth/login/route.ts` |
| `auth.login.setup_required` | `src/app/api/auth/login/route.ts` |
| `auth.logout.success` | `src/app/api/auth/logout/route.ts` |
| `provider.credentials.created` | `src/app/api/providers/route.ts` |
| `provider.credentials.updated` | `src/app/api/providers/[id]/route.ts` |
| `provider.credentials.revoked` | `src/app/api/providers/[id]/route.ts` |
| `provider.credentials.batch_revoked` | `src/app/api/providers/route.ts` |
| `sync.token.created` | `src/app/api/sync/tokens/route.ts` |
| `sync.token.revoked` | `src/app/api/sync/tokens/[id]/route.ts` |
| `compliance.cleanup` | `src/lib/compliance/index.ts` |
Each entry captures `action`, `actor` (defaults to `"system"`), `target`,
`details`/`metadata` (JSON), `ip_address`, `resource_type`, `status`,
`request_id`, and `timestamp`. Sensitive keys (`apiKey`, `accessToken`,
`refreshToken`, `password`, anything matching `*token`/`*secret`/`*apikey`,
etc.) are recursively redacted to `"[redacted]"` before the row is written.
### MCP tool calls (`mcp_tool_audit`)
Every MCP tool invocation writes a row through
`open-sse/mcp-server/audit.ts`. Schema (from
`src/lib/db/migrations/002_mcp_a2a_tables.sql`):
| Column | Notes |
| ---------------- | ----------------------------------- |
| `id` | autoincrement |
| `tool_name` | MCP tool identifier |
| `input_hash` | sha256 of input (no payload stored) |
| `output_summary` | short, truncated summary |
| `duration_ms` | wall time |
| `api_key_id` | caller (nullable) |
| `success` | `1` / `0` |
| `error_code` | terminal error code on failure |
| `created_at` | ISO timestamp |
### Request / usage logs
These are operational telemetry (not strictly admin audit) but share the same
retention pipeline:
- `usage_history` — per-request usage roll-up
- `call_logs` — full per-request log (subject to row-cap, see below)
- `proxy_logs` — proxy traffic log (subject to row-cap)
- `request_detail_logs` — legacy detailed request log (still pruned if present)
## Storage Schema
`audit_log` is created lazily by `ensureAuditLogSchema()` on first use:
```sql
CREATE TABLE IF NOT EXISTS audit_log (
id INTEGER PRIMARY KEY AUTOINCREMENT,
timestamp TEXT NOT NULL DEFAULT (datetime('now')),
action TEXT NOT NULL,
actor TEXT NOT NULL DEFAULT 'system',
target TEXT,
details TEXT,
ip_address TEXT,
resource_type TEXT,
status TEXT,
request_id TEXT,
metadata TEXT
);
```
Indexes are created on `timestamp`, `action`, `actor`, `resource_type`,
`status`, and `request_id`. Missing columns on legacy DBs are added via
`ALTER TABLE` on demand.
## Retention & Cleanup
Two separate retention windows are honoured:
| Env var | Default | Applies to |
| --------------------------- | -------- | ----------------------------------------------------------------- |
| `APP_LOG_RETENTION_DAYS` | `7` | `audit_log`, `mcp_tool_audit` |
| `CALL_LOG_RETENTION_DAYS` | `7` | `usage_history`, `call_logs`, `proxy_logs`, `request_detail_logs` |
| `CALL_LOGS_TABLE_MAX_ROWS` | `100000` | Row-cap trim for `call_logs` |
| `PROXY_LOGS_TABLE_MAX_ROWS` | `100000` | Row-cap trim for `proxy_logs` |
`cleanupExpiredLogs()` runs the retention pass. It is invoked on server startup
from `src/server-init.ts` and `src/instrumentation-node.ts`. Each run logs a
`compliance.cleanup` audit event with the per-table delete counts. Proxy/call
log trimming is batched (`BATCH_SIZE = 5000`) to avoid long write locks.
Defaults are defined in `src/lib/logEnv.ts`
(`DEFAULT_APP_LOG_RETENTION_DAYS = 7`, `DEFAULT_CALL_LOG_RETENTION_DAYS = 7`).
## `noLog` Opt-Out (per API key)
API keys can be flagged so their downstream call traffic is not logged. The
flag lives on the `api_keys` table (`no_log INTEGER DEFAULT 0`) and is mirrored
into an in-memory set for hot-path lookups.
```bash
# Create a no-log key (management auth required)
curl -X POST http://localhost:20128/api/keys \
-H "Cookie: auth_token=..." \
-H "Content-Type: application/json" \
-d '{"name": "Privacy key", "noLog": true}'
```
Helpers (`src/lib/compliance/index.ts`):
- `setNoLog(apiKeyId, true|false)` — toggle the in-memory entry
- `isNoLog(apiKeyId)` — checked on the request path; falls back to a 30 s
cached read from `api_keys.no_log`
- `NO_LOG_API_KEY_IDS` (env, comma-separated) — preloaded into the in-memory
set on boot; useful when you cannot toggle the column directly
Administrative audit events (login, provider changes, MCP tool calls, etc.)
are **not** affected by `noLog` — only per-request traffic logging is opted
out.
## REST API
| Endpoint | Method | Description | Auth |
| --------------------------- | ------ | ------------------------------------------ | ---------- |
| `/api/compliance/audit-log` | `GET` | Paginated admin audit entries with filters | management |
| `/api/mcp/audit` | `GET` | Paginated MCP tool audit entries | (open-sse) |
| `/api/mcp/audit/stats` | `GET` | Aggregated MCP audit stats | (open-sse) |
No CSV export endpoint is shipped today — export from the dashboard or query
the SQLite database directly.
### Querying `/api/compliance/audit-log`
Supported query params (all optional, all use `LIKE %value%` matching for
text filters):
- `action`, `actor`, `target`, `resourceType` (or `resource_type`),
`status`, `requestId` (or `request_id`)
- `from` / `since`, `to` / `until` — ISO timestamps
- `limit` (default `50`, min `1`, max `500`)
- `offset` (default `0`, max `10_000`)
The response is a JSON array. Pagination metadata is returned in headers:
`x-total-count`, `x-page-limit`, `x-page-offset`.
```bash
curl "http://localhost:20128/api/compliance/audit-log?action=provider.credentials&from=2026-05-01" \
-H "Cookie: auth_token=..."
```
## Dashboard
The dashboard exposes audit data at **`/dashboard/audit`**
(`src/app/(dashboard)/dashboard/audit/page.tsx`). The page has two tabs:
- **Compliance** (`ComplianceTab.tsx`) — admin audit events from
`/api/compliance/audit-log`. Filters by event type, severity (info / warning
/ critical, derived from action + status), and date range. Severity is
computed client-side from the action/status strings.
- **MCP** (`McpAuditTab.tsx`) — MCP tool audit from `/api/mcp/audit`, with
filters by tool name and success/failure.
Both tabs paginate with page sizes of `50` (compliance) and `25` (MCP).
## Provider Credential Helpers
`src/lib/compliance/providerAudit.ts` provides shaping helpers used by the
provider-management routes when they emit credential events:
- `summarizeProviderConnectionForAudit(connection)` — strips `apiKey`,
`accessToken`, `refreshToken`, `idToken`, and
`providerSpecificData.consoleApiKey` before the connection snapshot is
written to `details`.
- `getProviderAuditTarget(connection)` — composes a stable
`"<provider>:<name|id>"` string for the `target` field.
- `extractProviderWarnings(...payloads)` — scans provider responses for
policy/safety warnings (`[sanitizer]`, `prompt injection detected`,
`content has been filtered`, `safety filter`, `policy violation`) and
surfaces up to 5 hits, each truncated to 400 chars.
## Best Practices
- Flag API keys handling PII (legal, medical, etc.) with `noLog: true`.
- Tune `APP_LOG_RETENTION_DAYS` / `CALL_LOG_RETENTION_DAYS` to meet your
retention policy. The 7-day defaults are conservative.
- Export the audit table off-platform (`sqlite3 dump`) on whatever cadence
your compliance program requires — no built-in archival exists.
- Track `auth.login.failed` and `auth.login.locked` counts for brute-force
detection.
- When adding new admin endpoints, call `logAuditEvent({ ... })` with a stable
`domain.verb.outcome` action string and pass the request context via
`getAuditRequestContext(request)` so IP and `requestId` are captured
automatically.
## See Also
- [`docs/security/GUARDRAILS.md`](./GUARDRAILS.md) — PII masking, prompt injection
- [`docs/frameworks/MCP-SERVER.md`](../frameworks/MCP-SERVER.md) — MCP tool catalog and scopes
- [`docs/reference/ENVIRONMENT.md`](../reference/ENVIRONMENT.md) — full env var reference
- Source: `src/lib/compliance/`, `src/app/api/compliance/`,
`src/app/api/mcp/audit/`, `src/lib/logEnv.ts`

216
Compression-Engines.md Normal file

@@ -0,0 +1,216 @@
# Compression Engines
OmniRoute compression is built around engine contracts. A mode can run one engine directly
(`caveman` or `rtk`) or a deterministic stacked pipeline that executes multiple engines in order.
## Modes
| Mode | Engine path | Intended input |
| ------------ | ---------------------------------- | -------------------------------------------- |
| `off` | none | Exact prompt preservation |
| `lite` | Caveman lite helpers | Low-risk always-on cleanup |
| `standard` | Caveman | Natural-language prompt condensation |
| `aggressive` | Caveman + history/tool summarizers | Long chat sessions |
| `ultra` | Caveman + pruning helpers | Context-limit recovery |
| `rtk` | RTK | Terminal, shell, build, test, and git output |
| `stacked` | Pipeline, default `rtk -> caveman` | Mixed tool logs and prose, max savings |
## Engine Registry
The registry lives in `open-sse/services/compression/engines/registry.ts`. Engines expose a shared
contract:
- `id`: stable engine id such as `caveman` or `rtk`
- `apply(text, config)`: legacy execution path used by stacked pipelines
- `compress(input, config)`: primary execution path returning text + stats
- `getConfigSchema()`: returns the JSON-Schema-like shape of valid config
- `validateConfig(config)`: returns `{ valid, errors[] }`
Registration uses `registerCompressionEngine(engine)` (or `registerEngine` for advanced cases),
which calls `assertValidEngine()` and `validateConfig(defaultConfig)` before accepting.
Use `unregisterCompressionEngine(id)` to remove an engine at runtime.
`strategySelector.ts` registers the built-in engines before compression runs. This lets preview,
runtime compression, stacked mode, tests, and future engines use the same execution path.
### MCP description compression (related)
A separate registry compresses MCP tool description metadata at registry-level — see
`open-sse/mcp-server/descriptionCompressor.ts` and [MCP-SERVER.md](../frameworks/MCP-SERVER.md). It reuses
Caveman rules but operates on tool metadata, not request payloads.
## Caveman
Caveman mode focuses on semantic condensation of normal prose:
- preserves code blocks, URLs, JSON, paths, and structured data
- removes filler, hedging, repeated context, and verbose connective phrasing
- supports language-aware file rule packs in `open-sse/services/compression/rules/`
- remains available through the legacy `standard`, `aggressive`, and `ultra` modes
The dashboard surface is `Dashboard -> Context & Cache -> Caveman`.
Caveman upstream reports `~75%` fewer output tokens, `65%` average output savings in benchmarks
with a `22-87%` range, and a `~46%` input-compression tool. OmniRoute uses the Caveman input-side
number when documenting stacked prompt/context savings; Caveman output mode remains a separate
response-behavior feature.
## RTK
RTK mode focuses on command and tool output:
- detects output classes such as `git status`, `git branch`, `git diff`, Vitest/Jest/Pytest,
Cargo/Go tests, TypeScript/Vite/Webpack builds, ESLint, npm audit/installs, Docker logs,
shell `find`/`grep`, stack traces, and generic logs
- applies 49 JSON filters from `open-sse/services/compression/engines/rtk/filters/`
- supports the RTK-style declarative pipeline: ANSI stripping, replace, match-output short-circuit,
strip/keep lines, per-line truncation, head/tail/max-line truncation, and on-empty fallback
- supports trust-gated project filters in `.rtk/filters.json` and global filters in
`DATA_DIR/rtk/filters.json`
- strips ANSI sequences, progress noise, repeated lines, and unhelpful boilerplate
- preserves actionable failures, warnings, summaries, changed files, and tail context
- can optionally retain redacted raw output for recovery/debugging through authenticated management
routes
The dashboard surface is `Dashboard -> Context & Cache -> RTK`.
Operational details for custom filters, trust, verify, and raw-output recovery live in
[`RTK_COMPRESSION.md`](./RTK_COMPRESSION.md).
RTK upstream reports `60-90%` savings for command-output compression. Its README example shows a
30-minute Claude Code session going from `~118,000` tokens to `~23,900`, or `79.7%` saved.
## Stacked Pipelines
Stacked mode runs pipeline steps in order. The default is:
```txt
rtk -> caveman
```
Use this for coding-agent sessions where a prompt combines command output with human or assistant
prose. RTK reduces noisy tool logs first, then Caveman compresses remaining natural language.
Pipeline steps are configured with `stackedPipeline` in compression settings or through compression
combos.
When both engines reduce the same eligible payload, savings compound:
```txt
combined = 1 - (1 - RTK savings) * (1 - Caveman input savings)
average = 1 - (1 - 0.80) * (1 - 0.46) = 89.2%
range = 1 - (1 - 0.60..0.90) * (1 - 0.46) = 78.4-94.6%
```
## MCP Accessibility Tree Filter
The MCP accessibility-tree smart filter is a post-execution compression layer that runs on MCP
**tool results**, not on prompts or context. It targets the verbose accessibility-tree and browser
snapshot payloads returned by tools like Playwright, computer-use, and browser-automation MCP
servers.
### What it does
1. **Noise stripping** — removes empty generic/text entries (`- generic:`, `- text: ""`)
2. **Sibling collapse** — when ≥ `collapseThreshold` (default 30) consecutive lines are structural
repeats, collapses them into the first `collapseKeepHead` (default 10) lines + a count summary +
the last `collapseKeepTail` (default 5) lines
3. **Ref preservation** — `[ref=eXX]` anchors required by Playwright/computer-use are never touched
4. **Hard truncation** — if the text after collapse still exceeds `maxTextChars` (default 50,000),
truncates with a navigation hint so the agent can continue working
### Engine location
```txt
open-sse/services/compression/engines/mcpAccessibility/
index.ts ← smartFilterText() entry point
collapseRepeated.ts ← sibling-collapse algorithm
constants.ts ← DEFAULT_MCP_ACCESSIBILITY_CONFIG
```
### Configuration
Controlled by `compression.mcpAccessibility` in global settings (migration 056). Default config:
```json
{
"enabled": true,
"maxTextChars": 50000,
"collapseThreshold": 30,
"collapseKeepHead": 10,
"collapseKeepTail": 5,
"minLengthToProcess": 2000
}
```
The filter is only applied to tool-result payloads whose `type` is `"text"` and whose length
exceeds `minLengthToProcess`. It does not affect prompt compression or request payloads.
### Expected savings
60–80% on browser snapshot tool results, depending on page complexity. The collapse algorithm
is O(n) in line count and adds negligible latency.
### This filter vs the compression engines above
| Aspect | Caveman / RTK / Stacked | MCP accessibility filter |
| ----------- | ------------------------- | -------------------------------------- |
| Target | Request prompts / context | MCP tool results |
| Trigger | Compression mode setting | `compression.mcpAccessibility.enabled` |
| Scope | All SSE messages | Tool results only |
| Ref anchors | N/A | Preserved unconditionally |
---
## Compression Combos
Compression combos are named compression profiles that can be assigned to routing combos:
- `compression_combos`: stores mode, pipeline, RTK config, language config, and default marker
- `compression_combo_assignments`: maps a compression combo to a routing combo
- runtime integration resolves an assigned compression combo before generic combo overrides
- analytics include `compression_combo_id` and `engine`
Dashboard surface: `Dashboard -> Context & Cache -> Compression Combos`.
## API Surface
| Route | Purpose |
| -------------------------------------- | ---------------------------------------------------------------- |
| `/api/settings/compression` | Global compression settings (includes `mcpAccessibility` config) |
| `/api/compression/preview` | Preview any compression mode |
| `/api/compression/language-packs` | List available Caveman language packs |
| `/api/context/caveman/config` | Caveman settings alias |
| `/api/context/rtk/config` | RTK defaults and settings |
| `/api/context/rtk/filters` | RTK filter catalog |
| `/api/context/rtk/test` | RTK preview/test endpoint |
| `/api/context/rtk/raw-output/[id]` | Authenticated redacted raw-output recovery |
| `/api/context/combos` | Compression combo CRUD |
| `/api/context/combos/[id]/assignments` | Routing-combo assignment CRUD |
| `/api/context/analytics` | Compression analytics alias |
Management routes require management authentication or API-key policy checks.
## MCP Tools
Compression exposes five MCP tools:
| Tool | Scope | Purpose |
| ----------------------------------- | ------------------- | -------------------------------- |
| `omniroute_compression_status` | `read:compression` | Settings, analytics, cache stats |
| `omniroute_compression_configure` | `write:compression` | Update global settings |
| `omniroute_set_compression_engine` | `write:compression` | Set mode and optional pipeline |
| `omniroute_list_compression_combos` | `read:compression` | List compression combos |
| `omniroute_compression_combo_stats` | `read:compression` | Read combo/engine analytics |
## Validation
The focused gates for this area are:
```bash
node --import tsx/esm --test tests/unit/compression/rtk-*.test.ts tests/unit/compression/pipeline-integration.test.ts tests/unit/compression/context-compression-api.test.ts
node --import tsx/esm --test tests/unit/compression/*.test.ts tests/golden-set/*.test.ts tests/integration/compression-pipeline.test.ts tests/unit/api/compression/compression-api.test.ts
node --import tsx/esm --test tests/unit/compression/mcpAccessibility*.test.ts
npm run typecheck:core
```

277
Compression-Guide.md Normal file

@@ -0,0 +1,277 @@
# 🗜️ Prompt Compression Guide — OmniRoute
> Save 15-95% on eligible context automatically. For a quick overview, see the [README Compression section](../README.md#%EF%B8%8F-prompt-compression--save-15-95-eligible-tokens-automatically).
## Overview
OmniRoute implements a modular prompt compression pipeline that runs **proactively** before requests hit upstream providers. This means your token savings happen transparently — no changes needed to your workflow.
```
Client Request
→ Compression Strategy Selector
→ Combo override? → Use combo setting
→ Auto-trigger threshold? → Use auto mode
→ Default mode? → Use global setting
→ Off? → Skip compression
→ Selected Compression Mode
→ Off: No compression
→ Lite: Safe whitespace/formatting cleanup (~15%)
→ Standard: Caveman-speak filler removal (~30%)
→ Aggressive: History aging + summarization (~50%)
→ Ultra: Heuristic pruning + code-block thinning (~75%)
→ RTK: Command-aware terminal/tool-output filtering (60-90% upstream range)
→ Stacked: Ordered multi-engine pipeline, usually RTK then Caveman (78-95% eligible range)
→ Compressed Request → Provider
```
---
## Compression Modes
### Off
No compression applied. All messages pass through unchanged.
### Lite Mode (~15% savings, <1ms latency)
The safest mode — zero semantic change, only formatting cleanup:
| Technique | Description |
| ------------------------ | ------------------------------------------------- |
| `collapseWhitespace` | Merge consecutive blank lines and trailing spaces |
| `dedupSystemPrompt` | Remove duplicate system messages |
| `compressToolResults` | Compress verbose tool/function outputs |
| `removeRedundantContent` | Strip repeated instructions |
| `replaceImageUrls` | Shorten base64 image data URIs |
**Best for:** Always-on usage, safety-critical workflows.
### Standard Mode (~30% savings)
Inspired by [Caveman](https://github.com/JuliusBrussee/caveman) — removes filler words and verbose phrasing while preserving meaning:
- Removes filler words ("please", "I think", "basically", "actually")
- Condenses verbose phrases ("in order to" → "to", "as a result of" → "because")
- Strips polite hedging ("Would you mind...", "If you could possibly...")
- 30+ regex rules tuned for coding prompts
**Best for:** Daily coding workflows, cost-conscious teams.
### Aggressive Mode (~50% savings)
Smart history management for long sessions:
- **Message Aging** — older messages get progressively compressed
- **Tool Result Summarization** — long tool outputs replaced with summaries
- **Structural Integrity Guards** — ensures `tool_use` + `tool_result` pairs stay consistent
- **Context Window Awareness** — respects per-model token limits
**Best for:** Extended debugging sessions, large codebases.
### Ultra Mode (~75% savings)
Maximum compression for token-critical scenarios:
- **Heuristic Pruning** — removes messages below relevance threshold
- **Code Block Thinning** — compresses repetitive code examples
- **Binary Search Truncation** — finds optimal cut point for context window
- All Aggressive mode features included
**Best for:** When you're hitting context limits repeatedly.
### RTK Mode (60-90% upstream range)
RTK mode is optimized for verbose tool outputs that appear in coding-agent sessions:
- Detects command/output classes such as `git status`, `git diff`, `git log`, test runners,
TypeScript/Vite/Webpack builds, ESLint/Biome/Prettier, npm audit/installs, Docker logs, infra
output, and generic shell output
- Applies JSON filter packs from `open-sse/services/compression/engines/rtk/filters/`
- Ships 49 built-in filters with inline verify samples
- Removes ANSI control sequences, progress bars, repeated lines, and non-actionable noise
- Preserves failures, errors, warnings, changed files, summaries, and the tail of long output
- Supports trust-gated project filters, global filters, and optional redacted raw-output recovery
**Best for:** Agent sessions with shell, build, test, git, grep, and file-output transcripts.
### Stacked Mode (78-95% eligible range)
Stacked mode runs multiple compression engines in a deterministic order. The default pipeline is:
```txt
RTK -> Caveman
```
That order keeps terminal/tool output compact first, then applies Caveman semantic condensation to
the remaining natural-language prompt. Stacked pipelines can be configured globally or through
compression combos assigned to routing combos.
**Best for:** Mixed context with large tool logs plus human instructions or assistant summaries.
---
## Upstream Savings Math
OmniRoute documents compression savings from two sources: upstream project benchmarks and
OmniRoute's own engine composition.
| Source | Upstream README number used here |
| ------- | --------------------------------------------------------------------------------------------------------------------- |
| Caveman | `~75%` fewer output tokens, `65%` benchmark average output savings, `22-87%` range, and `~46%` input compression tool |
| RTK | `60-90%` command-output savings; sample session `~118,000 -> ~23,900` tokens, or `79.7%` saved (`~80%`) |
For overlapping tool/context payloads, the default OmniRoute combo stacks the engines:
```txt
RTK -> Caveman
```
The combined savings are multiplicative, not additive:
```txt
combined = 1 - (1 - RTK savings) * (1 - Caveman input savings)
average = 1 - (1 - 0.80) * (1 - 0.46) = 89.2%
range = 1 - (1 - 0.60..0.90) * (1 - 0.46) = 78.4-94.6%
```
That `78-95%` number applies when both RTK and Caveman can reduce the same input/context payload.
Caveman response output mode is separate: when enabled, use Caveman's own output savings (`65%`
average, `~75%` headline, `22-87%` range). Total billing savings depend on your prompt/output mix.
---
## Token Savings Visualization
```
Without compression: 47K tokens sent to LLM
With Lite: 40K tokens sent (15% saved — safe, always-on)
With Standard: 33K tokens sent (30% saved — caveman-speak rules)
With Aggressive: 24K tokens sent (50% saved — aging + summarization)
With Ultra: 12K tokens sent (75% saved — heuristic pruning)
With RTK: 19K-5K tokens sent (60-90% saved on command/tool output)
With Stacked: 10K-2.5K tokens sent (78-95% eligible RTK+Caveman range)
```
---
## Configuration
### Dashboard
Navigate to `Dashboard → Context & Cache`:
- **Caveman** — mode selection, language packs, preview, and global defaults
- **RTK** — command-filter preview, RTK safety settings, and filter catalog
- **Compression Combos** — named engine pipelines assigned to routing combos
- **Auto-Trigger Threshold** — automatically engage compression when token count exceeds threshold
### Per-Combo Override
In `Dashboard → Context & Cache → Compression Combos`, assign a compression combo to a routing
combo:
```txt
Combo: "free-forever"
Compression Combo: "coding-agent-stack"
Pipeline: RTK -> Caveman
Targets:
1. gc/gemini-3-flash
2. if/kimi-k2-thinking
```
This lets you use stacked compression on free/coding providers while keeping lite mode on paid
subscriptions.
### API
```bash
# Get compression settings
curl http://localhost:20128/api/settings/compression
# Update compression settings
curl -X PUT http://localhost:20128/api/settings/compression \
-H "Content-Type: application/json" \
-d '{"defaultMode":"stacked","autoTriggerMode":"stacked","autoTriggerTokens":32000}'
# Preview a specific RTK/stacked payload
curl -X POST http://localhost:20128/api/compression/preview \
-H "Content-Type: application/json" \
-d '{"mode":"rtk","messages":[{"role":"tool","content":"npm test output here"}]}'
# List RTK filter packs
curl http://localhost:20128/api/context/rtk/filters
# Test RTK directly with optional command metadata
curl -X POST http://localhost:20128/api/context/rtk/test \
-H "Content-Type: application/json" \
-d '{"command":"npm test","text":"FAIL tests/example.test.ts\nError: boom"}'
```
---
## What Gets Protected
The compression engine **always preserves:**
- ✅ Code blocks (fenced and inline)
- ✅ URLs and file paths
- ✅ JSON structures and structured data
- ✅ Identifiers and protected technical tokens
- ✅ Mathematical expressions
- ✅ Tool/function call definitions
- ✅ System prompts (in lite mode)
RTK raw-output recovery redacts common API keys, bearer tokens, Slack tokens, AWS access keys,
passwords, tokens, and secrets before anything is persisted.
---
## Compression Stats
Every compressed request includes stats in the server logs:
```json
{
"originalTokens": 47200,
"compressedTokens": 40120,
"savingsPercent": 15.0,
"techniquesUsed": ["collapseWhitespace", "dedupSystemPrompt"],
"mode": "lite",
"engine": "caveman",
"compressionComboId": "coding-agent-stack",
"durationMs": 0.8,
"rtkRawOutputPointers": []
}
```
---
## Phase Roadmap
| Phase | Modes | Status |
| ------- | ------------------------------------ | ---------- |
| Phase 1 | Off, Lite | ✅ Shipped |
| Phase 2 | Standard, Aggressive, Ultra | ✅ Shipped |
| Phase 3 | RTK, Stacked, Compression Combos | ✅ Shipped |
| Phase 4 | Per-model adaptive, ML-based pruning | 🗓️ Planned |
---
## Acknowledgments
Standard mode compression rules are inspired by **[Caveman](https://github.com/JuliusBrussee/caveman)** by **[JuliusBrussee](https://github.com/JuliusBrussee)** (⭐ 51K+) — the viral "why use many token when few token do trick" project. Caveman reports `~75%` fewer output tokens, `65%` benchmark average output savings, a `22-87%` output range, and a `~46%` input-compression tool.
RTK mode is inspired by **[RTK - Rust Token Killer](https://github.com/rtk-ai/rtk)** by **[RTK AI](https://github.com/rtk-ai)** — the high-performance command-output compression project for terminal, build, test, git, and tool-output filtering. RTK reports `60-90%` savings, with its README sample session showing `~80%` saved.
---
## See Also
- [Environment Config](../reference/ENVIRONMENT.md) — Compression environment variables
- [Architecture Guide](../architecture/ARCHITECTURE.md) — Compression pipeline internals
- [User Guide](../guides/USER_GUIDE.md) — Getting started with compression
- [RTK Compression](./RTK_COMPRESSION.md) — RTK filters, trust model, verify gate, raw-output recovery
- [Compression Engines](./COMPRESSION_ENGINES.md) — Caveman, RTK, stacked, APIs, MCP, dashboard
- [Compression Rules Format](./COMPRESSION_RULES_FORMAT.md) — JSON rule-pack format
- [Compression Language Packs](./COMPRESSION_LANGUAGE_PACKS.md) — Language-specific Caveman rules

148
Compression-Language-Packs.md Normal file

@@ -0,0 +1,148 @@
# Compression Language Packs
Caveman compression can load language-specific rule packs in addition to the built-in English rules.
This keeps the core engine stable while allowing Portuguese, Spanish, German, French, Japanese, and
future language packs to evolve independently.
## Location
Language packs live under:
```txt
open-sse/services/compression/rules/<language>/
```
Current shipped packs (verified against `rules/` directory contents):
| Language | Directory | Rule categories present |
| ------------------- | -------------- | --------------------------------------------------- |
| English | `rules/en/` | `context`, `dedup`, `filler`, `structural`, `ultra` |
| Spanish | `rules/es/` | `context`, `dedup`, `filler`, `structural`, `ultra` |
| Portuguese (Brazil) | `rules/pt-BR/` | `context`, `filler`, `structural` |
| German | `rules/de/` | `context`, `filler`, `structural` |
| French | `rules/fr/` | `context`, `filler`, `structural` |
| Japanese | `rules/ja/` | `context`, `filler`, `structural` |
> **Parity note:** `en` and `es` packs have the full 5 categories; `pt-BR`, `de`, `fr`, `ja` ship 3 categories. The missing `dedup` and `ultra` categories silently fall back to the English built-ins. Contributions welcome to add `dedup.json` and `ultra.json` for the smaller packs.
>
> The canonical category list and per-category schema live in [`open-sse/services/compression/rules/_schema.json`](../../open-sse/services/compression/rules/_schema.json) (JSON Schema draft 2020-12).
## Language Detection
`languageDetector.ts` uses lightweight heuristics to infer the language from prompt text. The
configured default language is still respected, and detection can be disabled by config when exact
control is required.
Detection output is used only to choose rule packs. It does not change provider routing, locale
selection, or UI language.
## Config Shape
Compression settings can include:
```json
{
"languageConfig": {
"enabled": true,
"defaultLanguage": "en",
"autoDetect": true,
"enabledPacks": ["en", "pt-BR", "es", "de", "fr", "ja"]
},
"cavemanConfig": {
"language": "en",
"autoDetectLanguage": true,
"enabledLanguagePacks": ["en", "pt-BR", "es", "de", "fr", "ja"]
}
}
```
`languageConfig` controls dashboard/preview defaults. `cavemanConfig` is the runtime engine config
used when Caveman compresses message text.
## Adding a Language Pack
1. Create `open-sse/services/compression/rules/<language>/<pack>.json`.
2. Use the Caveman rule format from `docs/compression/COMPRESSION_RULES_FORMAT.md`.
3. Keep replacements conservative and avoid changing code, identifiers, URLs, or JSON.
4. Add or update tests for language selection and replacement behavior.
5. Expose new dashboard/i18n labels if the language appears in UI selectors.
## API
Available packs can be queried with:
```bash
curl http://localhost:20128/api/compression/language-packs
```
The preview endpoint accepts language config overrides:
```bash
curl -X POST http://localhost:20128/api/compression/preview \
-H "Content-Type: application/json" \
-d '{
"mode": "standard",
"text": "Por favor, eu gostaria que voce basicamente resumisse isso.",
"config": {
"languageConfig": {
"defaultLanguage": "pt-BR",
"autoDetect": true
}
}
}'
```
## SHARED_BOUNDARIES (v3.8.0)
All 6 language packs received a `SHARED_BOUNDARIES` clause in v3.8.0 that is applied at every
Caveman intensity (LITE, FULL, ULTRA). It instructs the engine to preserve these patterns verbatim,
regardless of surrounding filler removal:
| Pattern type | Example |
| -------------------------------- | -------------------------------------- |
| Fenced code blocks | ` ```python\n...\n``` ` |
| Inline code | `` `my_var` `` |
| URLs | `https://example.com/path` |
| File paths (absolute + relative) | `/etc/hosts`, `./src/index.ts` |
| Error headers | `Error:`, `TypeError:`, `SyntaxError:` |
| Stack trace lines | ` at functionName (file.ts:12:3)` |
These patterns are populated in `DEFAULT_CAVEMAN_CONFIG.preservePatterns` (previously `[]`). The
constant lives in `open-sse/services/compression/types.ts`.
### Why this matters
Without SHARED_BOUNDARIES, aggressive Caveman modes could strip content that looked like repetitive
prose but was actually a code snippet, file path, or error stack. SHARED_BOUNDARIES acts as a
language-agnostic safety net applied before filler rules run.
### Customizing preservePatterns
Additional patterns can be added at runtime via compression settings:
````json
{
"cavemanConfig": {
"preservePatterns": [
"```[\\s\\S]*?```",
"`[^`]+`",
"https?://\\S+",
"(?:/|\\./)[^\\s]+",
"\\b(?:Error|TypeError|SyntaxError|RangeError):",
"\\s+at\\s+\\S+\\s+\\(\\S+:\\d+:\\d+\\)"
]
}
}
````
Custom patterns extend (not replace) the 6 defaults.
---
## Operational Notes
- English built-in rules remain the fallback when a language pack is missing.
- Invalid built-in JSON packs fail validation so release assets do not silently degrade.
- Rule packs are data-only and should not import code or run arbitrary logic.
- The compression analytics layer records the selected mode and engine, not full prompt text.

190
Compression-Rules-Format.md Normal file

@@ -0,0 +1,190 @@
# Compression Rules Format
Compression rules are JSON files loaded at runtime. They are intentionally data-only so new
language packs and RTK command filters can be reviewed without changing engine code.
> **Canonical schema (source of truth):** [`open-sse/services/compression/rules/_schema.json`](../../open-sse/services/compression/rules/_schema.json) (JSON Schema draft 2020-12).
> The examples below are illustrative — when in doubt, validate your pack against `_schema.json`.
## Caveman Rule Packs
Caveman rule packs live under:
```txt
open-sse/services/compression/rules/<language>/<pack>.json
```
Each pack contains replacements that apply to normal prose after protected regions are isolated.
```json
{
"language": "en",
"category": "filler",
"rules": [
{
"name": "question_to_directive",
"pattern": "\\b(?:Can you explain why|Could you show me how)\\b\\s*",
"replacement": "Explain why ",
"replacementMap": {
"can you explain why": "Explain why ",
"could you show me how": "Show how "
},
"flags": "gi",
"context": "all",
"category": "context",
"minIntensity": "lite",
"description": "Convert verbose questions into direct requests."
}
]
}
```
### Caveman Fields
| Field | Required | Description |
| ------------------------ | -------- | ---------------------------------------------------------------- |
| `language` | yes | BCP-47-like language key such as `en`, `pt-BR`, `es` |
| `category` | yes | Pack category filename/category, for example `filler` or `dedup` |
| `rules` | yes | Array of regex replacement rules |
| `rules[].name` | yes | Stable rule name |
| `rules[].pattern` | yes | JavaScript regex source |
| `rules[].flags` | no | JavaScript regex flags; default `gi` |
| `rules[].replacement` | no | Replacement string or fallback when `replacementMap` misses |
| `rules[].replacementMap` | no | Match-specific replacements keyed by normalized matched text |
| `rules[].context` | no | `all`, `user`, `assistant`, or `system`; default `all` |
| `rules[].category` | no | `filler`, `context`, `structural`, `dedup`, `terse`, or `ultra` |
| `rules[].minIntensity` | no | `lite`, `full`, or `ultra`; default `lite` |
| `rules[].description` | no | Human-readable rule summary |
Use `flags` when case-sensitive matching matters, for example article removal before lowercase prose
without stripping `the OpenAI API`. Use `replacementMap` when one regex has multiple alternatives
that need different outputs; this keeps JSON rule packs data-only while preserving the behavior of
the richer built-in TypeScript replacement functions.
## RTK Filter Packs
RTK filters live under:
```txt
open-sse/services/compression/engines/rtk/filters/<filter>.json
```
Each filter describes how to recognize and compress a command-output family.
```json
{
"id": "test-vitest",
"label": "Vitest output",
"category": "test",
"priority": 92,
"match": {
"outputTypes": ["test-vitest"],
"commands": ["vitest", "npm test", "npm run test"],
"patterns": ["\\bFAIL\\b", "\\bPASS\\b", "\\bTest Files\\b"]
},
"rules": {
"stripAnsi": true,
"replace": [{ "pattern": "\\s+\\[[0-9]+ms\\]", "replacement": "" }],
"matchOutput": [
{ "pattern": "All tests passed", "message": "vitest: ok", "unless": "FAIL|Error:" }
],
"includePatterns": ["FAIL", "Error:", "Test Files", "Tests"],
"dropPatterns": ["^\\s*$", "Duration\\s+\\d+"],
"collapsePatterns": ["^\\s+at "],
"deduplicate": true,
"truncateLineAt": 240,
"maxLines": 160,
"headLines": 24,
"tailLines": 40,
"onEmpty": "vitest: ok",
"filterStderr": false
},
"preserve": {
"errorPatterns": ["FAIL", "Error:", "AssertionError"],
"summaryPatterns": ["Test Files", "Tests", "Snapshots"]
},
"tests": [
{
"name": "keeps failing tests",
"command": "vitest",
"input": "FAIL test/a.test.ts\\nError: boom\\nTest Files 1 failed",
"expected": "FAIL test/a.test.ts\\nError: boom\\nTest Files 1 failed"
}
]
}
```
### RTK Fields
| Field | Required | Description |
| -------------------------- | -------- | ------------------------------------------------------------------------------ |
| `id` | yes | Stable filter id |
| `label` | yes | Dashboard-readable name |
| `category` | yes | Filter family: git, test, build, shell, docker, package, infra, cloud, generic |
| `priority` | no | Higher priority wins when multiple filters match |
| `match.outputTypes` | no | Detector output ids that select this filter |
| `match.commands` | no | Command tokens that select this filter |
| `match.patterns` | no | Regex patterns that select this filter from output text |
| `rules.stripAnsi` | no | Remove ANSI escape sequences before regex stages |
| `rules.replace` | no | Ordered regex substitutions applied line by line |
| `rules.matchOutput` | no | Short-circuit output rules with optional `unless` guard |
| `rules.includePatterns` | no | Lines to prefer preserving |
| `rules.dropPatterns` | no | Lines to remove as noise |
| `rules.collapsePatterns` | no | Repeated matching lines that can be collapsed |
| `rules.deduplicate` | no | Collapse duplicate normalized lines |
| `rules.truncateLineAt` | no | Unicode-safe per-line character limit |
| `rules.maxLines` | no | Maximum retained lines before tail preservation |
| `rules.headLines` | no | Head lines retained during truncation |
| `rules.tailLines` | no | Tail lines retained for recent context |
| `rules.onEmpty` | no | Fallback message when filtering removes all content |
| `rules.filterStderr` | no | Normalize common stderr prefixes before later filtering stages |
| `preserve.errorPatterns` | no | Error lines that should survive truncation |
| `preserve.summaryPatterns` | no | Summary lines that should survive truncation |
| `tests[]` | no | Inline verification samples used by the RTK verify gate |
RTK applies declarative stages in this order: `stripAnsi`, `filterStderr`, `replace`,
`matchOutput`, `dropPatterns`/`includePatterns`, `truncateLineAt`, `headLines`/`tailLines`,
`maxLines`, and `onEmpty`.
Custom filters can be loaded from:
1. Project `.rtk/filters.json` files only after a matching `.rtk/trust.json` hash is present or
`trustProjectFilters` is enabled.
2. Global `DATA_DIR/rtk/filters.json`.
3. Built-in filters.
Project/global custom files may contain one filter object or an array of filter objects. Invalid
custom filters are skipped with diagnostics; invalid built-in filters fail validation.
Project trust file:
```json
{
"filtersSha256": "0123456789abcdef..."
}
```
The environment override `OMNIROUTE_RTK_TRUST_PROJECT_FILTERS=1` trusts project filters without a
hash and should be limited to controlled local development.
## Safety Rules
- Keep rules idempotent: running the same filter twice should not corrupt output.
- Preserve exact error text, file paths, line numbers, and command summaries where possible.
- Avoid rules that modify code blocks, JSON payloads, URLs, or secrets.
- Add unit coverage for new command families in detector/filter tests.
- Add `tests[]` samples to every built-in filter and to shared custom filters.
## Validation
Rule packs are validated before use. Built-in Caveman packs and built-in RTK filters fail fast
during validation so broken release assets are caught before shipment. Custom RTK filters are
skipped with diagnostics when parsing or trust validation fails.
Focused validation:
```bash
node --import tsx/esm --test tests/unit/compression/rule-loader.test.ts tests/unit/compression/language-packs.test.ts
node --import tsx/esm --test tests/unit/compression/rtk-verify.test.ts tests/unit/compression/rtk-dsl-pipeline.test.ts
```

319
Contributing.md Normal file

@@ -0,0 +1,319 @@
# Contributing to OmniRoute
Thank you for your interest in contributing! This guide covers everything you need to get started.
---
## Development Setup
### Prerequisites
- **Node.js** `>=22.22.3 <23`, or `>=24.0.0 <27` (recommended: 24 LTS)
- **npm** 10+
- **Git**
### Clone & Install
```bash
git clone https://github.com/diegosouzapw/OmniRoute.git
cd OmniRoute
npm install
```
### Environment Variables
```bash
# Create your .env from the template
cp .env.example .env
# Generate required secrets
echo "JWT_SECRET=$(openssl rand -base64 48)" >> .env
echo "API_KEY_SECRET=$(openssl rand -hex 32)" >> .env
```
Key variables for development:
| Variable | Development Default | Description |
| ---------------------- | ------------------------ | --------------------- |
| `PORT` | `20128` | Server port |
| `NEXT_PUBLIC_BASE_URL` | `http://localhost:20128` | Base URL for frontend |
| `JWT_SECRET` | (generate above) | JWT signing secret |
| `INITIAL_PASSWORD` | `CHANGEME` | First login password |
| `APP_LOG_LEVEL` | `info` | Log verbosity level |
### Dashboard Settings
The dashboard provides UI toggles for features that can also be configured via environment variables:
| Setting Location | Toggle | Description |
| ------------------- | ------------------ | ------------------------------ |
| Settings → Advanced | Debug Mode | Enable debug request logs (UI) |
| Settings → General | Sidebar Visibility | Show/hide sidebar sections |
These settings are stored in the database and persist across restarts, overriding env var defaults when set.
### Running Locally
```bash
# Development mode (hot reload)
npm run dev
# Production build
npm run build
npm run start
# Common port configuration
PORT=20128 NEXT_PUBLIC_BASE_URL=http://localhost:20128 npm run dev
```
Default URLs:
- **Dashboard**: `http://localhost:20128/dashboard`
- **API**: `http://localhost:20128/v1`
---
## Git Workflow
> ⚠️ **NEVER commit directly to `main`.** Always use feature branches.
```bash
git checkout -b feat/your-feature-name
# ... make changes ...
git commit -m "feat: describe your change"
git push -u origin feat/your-feature-name
# Open a Pull Request on GitHub
```
### Branch Naming
| Prefix | Purpose |
| ----------- | ------------------------- |
| `feat/` | New features |
| `fix/` | Bug fixes |
| `refactor/` | Code restructuring |
| `docs/` | Documentation changes |
| `test/` | Test additions/fixes |
| `chore/` | Tooling, CI, dependencies |
### Commit Messages
Follow [Conventional Commits](https://www.conventionalcommits.org/):
```
feat: add circuit breaker for provider calls
fix: resolve JWT secret validation edge case
docs: update SECURITY.md with PII protection
test: add observability unit tests
refactor(db): consolidate rate limit tables
```
Scopes (v3.8): `db`, `sse`, `oauth`, `dashboard`, `api`, `cli`, `docker`, `ci`, `mcp`, `a2a`, `memory`, `skills`, `cloud-agent`, `guardrails`, `compression`, `auto-combo`, `resilience`, `providers`, `executors`, `translator`, `domain`, `authz`.
---
## Running Tests
```bash
# All tests (unit + vitest + ecosystem + e2e)
npm run test:all
# Single test file (Node.js native test runner — most tests use this)
node --import tsx/esm --test tests/unit/your-file.test.ts
# Vitest (MCP server, autoCombo, cache)
npm run test:vitest
# E2E tests (requires Playwright)
npm run test:e2e
# Protocol clients E2E (MCP transports, A2A)
npm run test:protocols:e2e
# Ecosystem compatibility tests
npm run test:ecosystem
# Coverage gate: 75% statements/lines/functions, 70% branches
npm run test:coverage
npm run coverage:report
# Lint + format check
npm run lint
npm run check
```
Coverage notes:
- `npm run test:coverage` measures source coverage for the main unit test suite, excludes `tests/**`, and includes `open-sse/**`
- Pull requests must keep the coverage gate at **75%+** statements/lines/functions and **70%+** branches
- If a PR changes production code in `src/`, `open-sse/`, `electron/`, or `bin/`, it must add or update automated tests in the same PR
- `npm run coverage:report` prints the detailed file-by-file report from the latest coverage run
- `npm run test:coverage:legacy` preserves the older metric for historical comparison
- See `docs/ops/COVERAGE_PLAN.md` for the phased coverage improvement roadmap
### Pull Request Requirements
Before opening or merging a PR:
- Run `npm run test:unit`
- Run `npm run test:coverage`
- Ensure the coverage gate stays at **75%+** statements/lines/functions, **70%+** branches
- Include the changed or added test files in the PR description when production code changed
- Check the SonarQube result on the PR when the project secrets are configured in CI
Current test status: **122 unit test files** covering:
- Provider translators and format conversion
- Rate limiting, circuit breaker, and resilience
- Semantic cache, idempotency, progress tracking
- Database operations and schema (21 DB modules)
- OAuth flows and authentication
- API endpoint validation (Zod v4)
- MCP server tools and scope enforcement
- Memory and Skills systems
---
## Code Style
- **ESLint** — Run `npm run lint` before committing
- **Prettier** — Auto-formatted via `lint-staged` on commit (2 spaces, semicolons, double quotes, 100 char width, es5 trailing commas)
- **TypeScript** — All `src/` code uses `.ts`/`.tsx`; `open-sse/` uses `.ts`/`.js`; document with TSDoc (`@param`, `@returns`, `@throws`)
- **No `eval()`** — ESLint enforces `no-eval`, `no-implied-eval`, `no-new-func`
- **Zod validation** — Use Zod v4 schemas for all API input validation
- **Naming**: Files = camelCase/kebab-case, components = PascalCase, constants = UPPER_SNAKE
---
## Project Structure
```
src/ # TypeScript (.ts / .tsx)
├── app/ # Next.js 16 App Router
│ ├── (dashboard)/ # Dashboard pages (23 sections)
│ ├── api/ # API routes (51 directories)
│ └── login/ # Auth pages (.tsx)
├── domain/ # Policy engine (policyEngine, comboResolver, costRules, etc.)
├── lib/ # Core business logic (.ts)
│ ├── a2a/ # Agent-to-Agent v0.3 protocol server
│ ├── acp/ # Agent Communication Protocol registry
│ ├── compliance/ # Compliance policy engine
│ ├── db/ # SQLite database layer (21 modules + 16 migrations)
│ ├── memory/ # Persistent conversational memory
│ ├── oauth/ # OAuth providers, services, and utilities
│ ├── skills/ # Extensible skill framework
│ ├── usage/ # Usage tracking and cost calculation
│ └── localDb.ts # Re-export layer only — never add logic here
├── middleware/ # Request middleware (promptInjectionGuard)
├── mitm/ # MITM proxy (cert, DNS, target routing)
├── shared/
│ ├── components/ # React components (.tsx)
│ ├── constants/ # Provider definitions (177), MCP scopes, 14 routing strategies
│ ├── utils/ # Circuit breaker, sanitizer, auth helpers
│ └── validation/ # Zod v4 schemas
└── sse/ # SSE proxy pipeline
open-sse/ # @omniroute/open-sse workspace
├── executors/ # 14 provider-specific request executors
├── handlers/ # 11 request handlers (chat, responses, embeddings, images, etc.)
├── mcp-server/ # MCP server (25 tools, 3 transports, 10 scopes)
├── services/ # 36+ services (combo, autoCombo, rateLimitManager, etc.)
├── translator/ # Format translators (OpenAI ↔ Claude ↔ Gemini ↔ Responses ↔ Ollama)
├── transformer/ # Responses API transformer
└── utils/ # 22 utility modules (stream, TLS, proxy, logging)
electron/ # Electron desktop app (cross-platform)
tests/
├── unit/ # Node.js test runner (122 test files)
├── integration/ # Integration tests
├── e2e/ # Playwright tests
├── security/ # Security tests
├── translator/ # Translator-specific tests
└── load/ # Load tests
docs/ # Documentation
├── ARCHITECTURE.md # System architecture
├── API_REFERENCE.md # All endpoints
├── USER_GUIDE.md # Provider setup, CLI integration
├── TROUBLESHOOTING.md # Common issues
├── MCP-SERVER.md # MCP server (25 tools)
├── A2A-SERVER.md # A2A agent protocol
├── AUTO-COMBO.md # Auto-combo engine
├── CLI-TOOLS.md # CLI tools integration
├── COVERAGE_PLAN.md # Test coverage improvement plan
├── openapi.yaml # OpenAPI specification
└── adr/ # Architecture Decision Records
```
---
## Adding a New Provider
### Step 1: Register Provider Constants
Add to `src/shared/constants/providers.ts` — Zod-validated at module load.
### Step 2: Add Executor (if custom logic needed)
Create executor in `open-sse/executors/your-provider.ts` extending the base executor.
### Step 3: Add Translator (if non-OpenAI format)
Create request/response translators in `open-sse/translator/`.
### Step 4: Add OAuth Config (if OAuth-based)
Add OAuth credentials in `src/lib/oauth/constants/oauth.ts` and service in `src/lib/oauth/services/`.
If the upstream provider distributes a public OAuth client_id/secret or Firebase Web API key inside its public CLI / browser bundle, **do not** embed it as a string literal. Use `resolvePublicCred()` from `open-sse/utils/publicCreds.ts` and add a masked byte entry to `EMBEDDED_DEFAULTS`. The full mandatory workflow is documented in [`docs/security/PUBLIC_CREDS.md`](./docs/security/PUBLIC_CREDS.md).
Inside handlers/executors, error messages reaching the client must go through `buildErrorBody()` / `sanitizeErrorMessage()` from `open-sse/utils/error.ts` — never put raw `err.stack` or `err.message` in a Response body. See [`docs/security/ERROR_SANITIZATION.md`](./docs/security/ERROR_SANITIZATION.md).
### Step 5: Register Models
Add model definitions in `open-sse/config/providerRegistry.ts`.
### Step 6: Add Tests
Write unit tests in `tests/unit/` covering at minimum:
- Provider registration
- Request/response translation
- Error handling
---
## Pull Request Checklist
- [ ] Tests pass (`npm test`)
- [ ] Linting passes (`npm run lint`)
- [ ] Build succeeds (`npm run build`)
- [ ] TypeScript types added for new public functions and interfaces
- [ ] No hardcoded secrets or fallback values
- [ ] Public upstream credentials embedded via `resolvePublicCred()` (see [`docs/security/PUBLIC_CREDS.md`](./docs/security/PUBLIC_CREDS.md)), never as literals
- [ ] Error responses route through `buildErrorBody()` / `sanitizeErrorMessage()` — no raw stack traces in response bodies (see [`docs/security/ERROR_SANITIZATION.md`](./docs/security/ERROR_SANITIZATION.md))
- [ ] Shell commands (`exec` / `spawn`) pass runtime values via `env`, not via string interpolation
- [ ] All inputs validated with Zod schemas
- [ ] CHANGELOG updated (if user-facing change)
- [ ] Documentation updated (if applicable)
- [ ] No new CodeQL / Secret-Scanning alerts opened, or each one dismissed with technical justification referencing the relevant `docs/security/` doc
- [ ] Routes that spawn child processes (`/api/mcp/`, `/api/cli-tools/runtime/`) classified as `isLocalOnlyPath()` in `src/server/authz/routeGuard.ts` — see [Hard Rule #15](docs/security/ROUTE_GUARD_TIERS.md)
- [ ] No `Co-Authored-By` trailers in commit messages — commits must appear solely under the repository owner's Git identity (Hard Rule #16)
---
## Releasing
Releases are managed via the `/generate-release` workflow. When a new GitHub Release is created, the package is **automatically published to npm** via GitHub Actions.
---
## Getting Help
- **Architecture**: See [`docs/architecture/ARCHITECTURE.md`](docs/architecture/ARCHITECTURE.md)
- **API Reference**: See [`docs/reference/API_REFERENCE.md`](docs/reference/API_REFERENCE.md)
- **Security docs**: [`docs/security/CLI_TOKEN.md`](docs/security/CLI_TOKEN.md), [`docs/security/ROUTE_GUARD_TIERS.md`](docs/security/ROUTE_GUARD_TIERS.md), [`docs/security/ERROR_SANITIZATION.md`](docs/security/ERROR_SANITIZATION.md), [`docs/security/PUBLIC_CREDS.md`](docs/security/PUBLIC_CREDS.md)
- **Ops docs**: [`docs/ops/SQLITE_RUNTIME.md`](docs/ops/SQLITE_RUNTIME.md)
- **Issues**: [github.com/diegosouzapw/OmniRoute/issues](https://github.com/diegosouzapw/OmniRoute/issues)
- **ADRs**: See `docs/adr/` for architectural decision records

178
Coverage-Plan.md Normal file

@@ -0,0 +1,178 @@
# Test Coverage Plan
Last updated: 2026-05-13
> Status measured on 2026-05-13: lines 82.58%, statements 82.58%, functions 84.23%, branches 75.22%. Phases 1-5 are complete. Current focus is Phase 6 (>=85%) and Phase 7 (>=90%).
## Baseline
There are multiple coverage numbers depending on how the report is computed. For planning, only one of them is useful.
| Metric | Scope | Statements / Lines | Branches | Functions | Notes |
| -------------------- | ----------------------------------------------------- | -----------------: | -------: | --------: | --------------------------------------------------- |
| Legacy | Old `npm run test:coverage` | 79.42% | 75.15% | 67.94% | Inflated: counts test files and excludes `open-sse` |
| Diagnostic | Source-only, excluding tests and excluding `open-sse` | 68.16% | 63.55% | 64.06% | Useful only to isolate `src/**` |
| Recommended baseline | Source-only, excluding tests and including `open-sse` | 82.58% | 75.22% | 84.23% | This is the project-wide baseline to improve |
The recommended baseline is the number to optimize against.
## Rules
- Coverage targets apply to source files, not to `tests/**`.
- `open-sse/**` is part of the product and must remain in scope.
- New code should not reduce coverage in touched areas.
- Prefer testing behavior and branch outcomes over implementation details.
- Prefer temp SQLite databases and small fixtures over broad mocks for `src/lib/db/**`.
## Current command set
- `npm run test:coverage`
- Main source coverage gate for the unit test suite
- Generates `text-summary`, `html`, `json-summary`, and `lcov`
- `npm run coverage:report`
- Detailed file-by-file report from the latest run
- `npm run test:coverage:legacy`
- Historical comparison only
## Milestones
| Phase | Target | Focus | Status |
| ------- | ---------------------: | ------------------------------------------------- | ---------- |
| Phase 1 | 60% statements / lines | Quick wins and low-risk utility coverage | ✅ Done |
| Phase 2 | 65% statements / lines | DB and route foundations | ✅ Done |
| Phase 3 | 70% statements / lines | Provider validation and usage analytics | ✅ Done |
| Phase 4 | 75% statements / lines | `open-sse` translators and helpers | ✅ Done |
| Phase 5 | 80% statements / lines | `open-sse` handlers and executor branches | ✅ Done |
| Phase 6 | 85% statements / lines | Harder edge cases, branch debt, regression suites | In progress |
| Phase 7 | 90% statements / lines | Final sweep, gap closure, strict ratchet | Pending |
Branches and functions should ratchet upward with each phase, but the primary hard target is statements / lines.
## Priority hotspots
These files have the lowest line coverage today (< 60%) and offer the best return for Phases 6-7. Generated from `coverage/coverage-summary.json` on 2026-05-13:
| # | File | Lines % |
| --- | ----------------------------------------------------------------- | ------: |
| 1 | `open-sse/services/compression/validation.ts` | 7.87% |
| 2 | `src/app/api/v1/batches/route.ts` | 9.67% |
| 3 | `src/app/docs/components/FeedbackWidget.tsx` | 9.80% |
| 4 | `open-sse/services/compression/toolResultCompressor.ts` | 10.00% |
| 5 | `src/app/docs/components/DocCodeBlocks.tsx` | 10.63% |
| 6 | `open-sse/services/compression/engines/rtk/lineFilter.ts` | 10.96% |
| 7 | `open-sse/services/specificityRules.ts` | 11.28% |
| 8 | `src/mitm/systemCommands.ts` | 12.19% |
| 9 | `open-sse/services/compression/aggressive.ts` | 12.77% |
| 10 | `src/app/api/v1/batches/[id]/cancel/route.ts` | 12.98% |
| 11 | `open-sse/services/compression/progressiveAging.ts` | 13.26% |
| 12 | `open-sse/services/compression/engines/rtk/smartTruncate.ts` | 13.43% |
| 13 | `open-sse/services/compression/engines/rtk/deduplicator.ts` | 13.51% |
| 14 | `src/lib/cloudAgent/agents/jules.ts` | 13.52% |
| 15 | `open-sse/services/compression/lite.ts` | 14.46% |
| 16 | `src/app/api/v1/rerank/route.ts` | 14.94% |
| 17 | `open-sse/services/compression/preservation.ts` | 15.07% |
| 18 | `src/lib/cloudAgent/agents/codex.ts` | 15.54% |
| 19 | `open-sse/services/tierResolver.ts` | 16.66% |
| 20 | `src/app/docs/components/DocsLazyWrapper.tsx` | 16.66% |
Themes for Phases 6-7:
- `open-sse/services/compression/**` is the densest cluster of low coverage and dominates the remaining gap.
- Batch and rerank API routes (`src/app/api/v1/batches/**`, `src/app/api/v1/rerank/route.ts`) need handler-level tests.
- Cloud agent adapters (`src/lib/cloudAgent/agents/jules.ts`, `codex.ts`) and `tierResolver.ts` need scenario tests.
- Docs UI components and `src/mitm/systemCommands.ts` are lower priority but cheap branch wins.
## Execution checklist
### Phase 1: 56.95% -> 60%
- [x] Fix coverage metric so it reflects source code instead of test files
- [x] Keep a legacy coverage script for comparison
- [x] Record the baseline and hotspots in-repo
- [ ] Add focused tests for low-risk utilities:
- `src/shared/utils/upstreamError.ts`
- `src/shared/utils/fetchTimeout.ts`
- `src/lib/api/errorResponse.ts`
- `src/shared/utils/apiAuth.ts`
- `src/lib/display/names.ts`
- [ ] Add route tests for:
- `src/app/api/settings/require-login/route.ts`
- `src/app/api/providers/[id]/models/route.ts`
### Phase 2: 60% -> 65%
- [ ] Add DB-backed tests for:
- `src/lib/db/modelComboMappings.ts`
- `src/lib/db/settings.ts`
- `src/lib/db/registeredKeys.ts`
- [ ] Cover branch behavior in:
- `src/lib/providers/validation.ts`
- `src/app/api/v1/embeddings/route.ts`
- `src/app/api/v1/moderations/route.ts`
### Phase 3: 65% -> 70%
- [ ] Add usage analytics tests for:
- `src/lib/usage/usageHistory.ts`
- `src/lib/usage/usageStats.ts`
- `src/lib/usage/costCalculator.ts`
- [ ] Expand route coverage for proxy management and settings branches
### Phase 4: 70% -> 75%
- [ ] Cover translator helpers and central translation paths:
- `open-sse/translator/index.ts`
- `open-sse/translator/helpers/*`
- `open-sse/translator/request/*`
- `open-sse/translator/response/*`
### Phase 5: 75% -> 80%
- [ ] Add handler-level tests for:
- `open-sse/handlers/chatCore.ts`
- `open-sse/handlers/responsesHandler.js`
- `open-sse/handlers/imageGeneration.js`
- `open-sse/handlers/embeddings.js`
- [ ] Add executor branch coverage for provider-specific auth, retries, and endpoint overrides
### Phase 6: 80% -> 85%
- [ ] Merge more edge-case suites into the main coverage path
- [ ] Increase function coverage for DB modules with weak constructor/helper coverage
- [ ] Close branch gaps in `settings.ts`, `registeredKeys.ts`, `validation.ts`, and translator helpers
### Phase 7: 85% -> 90%
- [ ] Treat the remaining low-coverage files as blockers
- [ ] Add regression tests for every uncovered production bug fixed during the push to 90%
- [ ] Raise the coverage gate in CI only after the local baseline is stable for at least two consecutive runs
## Ratchet policy
Update `npm run test:coverage` thresholds only after the project actually exceeds the next milestone with a comfortable buffer.
**Current gate (as of 2026-05-13):** `npm run test:coverage` enforces **75 statements / 75 lines / 75 functions / 70 branches**. This is the conservative ratchet against the measured baseline (82.58% / 82.58% / 84.23% / 75.22%) and preserves headroom for transient flakiness.
For ad-hoc threshold checks against the latest report use:
```bash
node scripts/check/test-report-summary.mjs --threshold 75
```
Recommended ratchet sequence (order is `statements-lines / branches / functions`):
1. 55/60/55
2. 60/62/58
3. 65/64/62
4. 70/66/66
5. 75/70/72 <-- current gate (75/70/75)
6. 80/75/78
7. 85/80/84
8. 90/85/88
Next ratchet target is `80/75/78` once branch coverage holds above 78% for two consecutive runs.
## Known gap
The current coverage command measures the main Node unit suite and includes source reached from it, including `open-sse`. It does not yet merge Vitest coverage into a single unified report. That merge is worth doing later, but it is not a blocker for starting the 60% -> 80% climb.

231
Docker-Guide.md Normal file

@@ -0,0 +1,231 @@
# 🐳 Docker Guide — OmniRoute
> Complete Docker deployment reference. For a quick start, see the [README Docker section](../README.md#-docker).
## Table of Contents
- [Quick Run](#quick-run)
- [With Environment File](#with-environment-file)
- [Docker Compose](#docker-compose)
- [Available Profiles](#available-profiles)
- [Redis Sidecar](#redis-sidecar)
- [Production Compose](#production-compose)
- [Dockerfile Stages](#dockerfile-stages)
- [Critical Environment Variables](#critical-environment-variables)
- [Docker Compose with Caddy (HTTPS)](#docker-compose-with-caddy-https-auto-tls)
- [Cloudflare Quick Tunnel](#cloudflare-quick-tunnel)
- [Image Tags](#image-tags)
- [Important Notes](#important-notes)
---
## Quick Run
```bash
docker run -d \
--name omniroute \
--restart unless-stopped \
--stop-timeout 40 \
-p 20128:20128 \
-v omniroute-data:/app/data \
diegosouzapw/omniroute:latest
```
## With Environment File
```bash
# Copy and edit .env first
cp .env.example .env
docker run -d \
--name omniroute \
--restart unless-stopped \
--stop-timeout 40 \
--env-file .env \
-p 20128:20128 \
-v omniroute-data:/app/data \
diegosouzapw/omniroute:latest
```
## Docker Compose
```bash
# Base profile (no CLI tools)
docker compose --profile base up -d
# CLI profile (Claude Code, Codex, OpenClaw built-in)
docker compose --profile cli up -d
# Host profile (Linux-first; mounts host CLI binaries read-only)
docker compose --profile host up -d
# Combine CLI + CLIProxyAPI sidecar
docker compose --profile cli --profile cliproxyapi up -d
```
## Available Profiles
OmniRoute ships four Compose profiles. Pick the one that matches your environment.
| Profile | Service | When to use | Command |
| ---------------- | ---------------- | --------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------- |
| `base` (default) | `omniroute-base` | Headless server / minimal runtime, no provider CLIs bundled | `docker compose --profile base up -d` |
| `cli` | `omniroute-cli` | Agentic workflows that call `omniroute providers/setup/doctor` and bundled CLIs (Codex, Claude Code, Droid, OpenClaw) | `docker compose --profile cli up -d` |
| `host` | `omniroute-host` | Linux hosts that want `network_mode`-like access to host CLIs by mounting `~/.local/bin`, `~/.codex`, `~/.claude`, etc. read-only | `docker compose --profile host up -d` |
| `cliproxyapi` | `cliproxyapi` | Run the [CLIProxyAPI](https://github.com/router-for-me/CLIProxyAPI) sidecar on port `8317` for upstream CLI proxying | `docker compose --profile cliproxyapi up -d` |
> Multiple profiles can be combined: `docker compose --profile cli --profile cliproxyapi up -d`.
## Redis Sidecar
OmniRoute relies on Redis to back the distributed rate limiter and shared cache. The `redis` service is **always defined** in `docker-compose.yml` (it has no profile gate) and starts alongside any other profile.
| Detail | Value |
| -------------------- | --------------------------------- |
| Image | `redis:7-alpine` |
| Container name | `omniroute-redis` |
| Internal port | `6379` |
| Host port (override) | `REDIS_PORT` (defaults to `6379`) |
| Volume | `omniroute-redis-data` → `/data` |
| Healthcheck | `redis-cli ping` (10s interval) |
Related environment variables:
- `REDIS_URL` — connection string injected into the app (`redis://redis:6379` by default).
- `REDIS_PORT` — host-side port mapping for the Redis container.
**Disabling Redis** is not recommended (rate limiter will degrade to in-memory fallback). If you must, either remove/comment the `redis:` service block in `docker-compose.yml` or scale it to zero:
```bash
docker compose up -d --scale redis=0
```
## Production Compose
For an isolated production snapshot running alongside dev, use `docker-compose.prod.yml`.
| Detail | Value |
| ---------------------- | ---------------------------------------------------------------------------------- |
| File | `docker-compose.prod.yml` |
| Default dashboard port | `PROD_DASHBOARD_PORT=20130` (mapped to internal `${DASHBOARD_PORT:-20128}`) |
| Default API port | `PROD_API_PORT=20131` |
| Image | `omniroute:prod` (built from `runner-cli` target) |
| Redis container | `omniroute-redis-prod` (`redis:8.6.2`, dedicated `redis-prod-data` volume) |
| Data volume | `omniroute-prod-data` (named, persisted across rebuilds) |
| Healthchecks | `node healthcheck.mjs` + `redis-cli ping`, with `depends_on` gated on Redis health |
How to use:
```bash
# Build & start the production stack
docker compose -f docker-compose.prod.yml up -d --build
# Stream logs
docker compose -f docker-compose.prod.yml logs -f
# Tear down (keep volumes)
docker compose -f docker-compose.prod.yml down
```
The prod stack runs in parallel with the dev compose (different container names, ports, and volumes), so you can keep iterating locally while production stays up.
## Dockerfile Stages
The repository ships a multi-stage Dockerfile (`Dockerfile`). Three stages are exposed; pick the right `target` for your use case.
| Stage | Base image | Purpose |
| ------------- | -------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `builder` | `node:24.15.0-trixie-slim` | Installs deps (`npm ci --legacy-peer-deps`) and runs `npm run build -- --webpack` |
| `runner-base` | `node:24.15.0-trixie-slim` | Production runtime with the Next.js standalone output. **No provider CLIs bundled.** |
| `runner-cli` | `runner-base` | Adds `git`, `docker.io`, `docker-compose` and global CLIs: `@openai/codex`, `@anthropic-ai/claude-code`, `droid`, `openclaw`. **Pick this for agentic workflows.** |
Build a specific target manually:
```bash
docker build --target runner-base -t omniroute:base .
docker build --target runner-cli -t omniroute:cli .
```
Defaults exported by `runner-base`: `PORT=20128`, `HOSTNAME=0.0.0.0`, `NODE_OPTIONS=--max-old-space-size=256`, `DATA_DIR=/app/data`, `OMNIROUTE_MIGRATIONS_DIR=/app/migrations`.
## Critical Environment Variables
Beyond the defaults documented in [ENVIRONMENT.md](../reference/ENVIRONMENT.md), the following variables matter most when running under Docker:
| Variable | Purpose | Default |
| ----------------------------- | --------------------------------------------------------------------------------------------------- | ------------------------- |
| `OMNIROUTE_WS_BRIDGE_SECRET` | Shared secret for the WebSocket bridge. **Required in production** — set to a strong random string. | unset (must be provided) |
| `REDIS_URL` | Connection string for the rate limiter / cache backend | `redis://redis:6379` |
| `REDIS_PORT` | Host-side port for the bundled Redis container | `6379` |
| `AUTO_UPDATE_HOST_REPO_DIR` | Host path mounted into `cli` profile at `/workspace/omniroute` for self-update workflows | `.` (current directory) |
| `OMNIROUTE_MEMORY_MB` | Node heap ceiling (`NODE_OPTIONS=--max-old-space-size`) baked into the image | `256` (set in Dockerfile) |
| `DASHBOARD_PORT` / `API_PORT` | Override exposed ports for dashboard (20128) and API (20129) | `20128` / `20129` |
| `PROD_DASHBOARD_PORT` | Host-side dashboard port for `docker-compose.prod.yml` | `20130` |
| `CLIPROXYAPI_PORT` | Host-side port for the `cliproxyapi` sidecar | `8317` |
## Docker Compose with Caddy (HTTPS Auto-TLS)
OmniRoute can be securely exposed using Caddy's automatic SSL provisioning. Ensure your domain's DNS A record points to your server's IP.
```yaml
services:
omniroute:
image: diegosouzapw/omniroute:latest
container_name: omniroute
restart: unless-stopped
volumes:
- omniroute-data:/app/data
environment:
- PORT=20128
- NEXT_PUBLIC_BASE_URL=https://your-domain.com
caddy:
image: caddy:latest
container_name: caddy
restart: unless-stopped
ports:
- "80:80"
- "443:443"
command: caddy reverse-proxy --from https://your-domain.com --to http://omniroute:20128
volumes:
omniroute-data:
```
## Cloudflare Quick Tunnel
Dashboard support for Docker deployments includes a one-click **Cloudflare Quick Tunnel** on `Dashboard → Endpoints`. The first enable downloads `cloudflared` only when needed, starts a temporary tunnel to your current `/v1` endpoint, and shows the generated `https://*.trycloudflare.com/v1` URL directly below your normal public URL.
Endpoint tunnel panels (Cloudflare, Tailscale, ngrok) can be shown or hidden from `Settings → Appearance` without changing active tunnel state.
### Tunnel Notes
- Quick Tunnel URLs are temporary and change after every restart.
- Quick Tunnels are not auto-restored after an OmniRoute or container restart. Re-enable them from the dashboard when needed.
- Managed install currently supports Linux, macOS, and Windows on `x64` / `arm64`.
- Managed Quick Tunnels default to HTTP/2 transport to avoid noisy QUIC UDP buffer warnings in constrained container environments. Set `CLOUDFLARED_PROTOCOL=quic` or `auto` if you want a different transport.
- Docker images bundle system CA roots and pass them to managed `cloudflared`, which avoids TLS trust failures when the tunnel bootstraps inside the container.
- Set `CLOUDFLARED_BIN=/absolute/path/to/cloudflared` if you want OmniRoute to use an existing binary instead of downloading one.
## Image Tags
| Image | Tag | Size | Description |
| ------------------------ | -------- | ------ | --------------------- |
| `diegosouzapw/omniroute` | `latest` | ~250MB | Latest stable release |
| `diegosouzapw/omniroute` | `3.8.0` | ~250MB | Current version |
Multi-platform manifest: `linux/amd64` + `linux/arm64` native (Apple Silicon, AWS Graviton, Raspberry Pi). Docker selects the matching architecture automatically; pass `--platform linux/amd64` if you need to force AMD64 emulation on ARM hosts.
## Important Notes
- **SQLite WAL Mode:** `docker stop` should be allowed to finish so OmniRoute can checkpoint the latest changes back into `storage.sqlite`. The bundled Compose files already set a 40s stop grace period. If you run the image directly, keep `--stop-timeout 40`.
- **`DISABLE_SQLITE_AUTO_BACKUP`:** Set to `true` if backups are managed externally.
- **Data Persistence:** Always mount a volume to `/app/data` to persist your database, keys, and configurations across container restarts.
- **Port Configuration:** Override `PORT` environment variable to change the default `20128` port.
## See Also
- [VM Deployment Guide](../ops/VM_DEPLOYMENT_GUIDE.md) — VM + nginx + Cloudflare setup
- [Fly.io Deployment Guide](../ops/FLY_IO_DEPLOYMENT_GUIDE.md) — Deploy to Fly.io
- [Environment Config](../reference/ENVIRONMENT.md) — Complete `.env` reference

272
Electron-Guide.md Normal file

@@ -0,0 +1,272 @@
# Electron Desktop Guide
> **Source of truth:** `electron/` workspace
> **Last updated:** 2026-05-13 — v3.8.0
OmniRoute ships a cross-platform desktop app (Windows / macOS / Linux) built on
**Electron 41** + **electron-builder 26.10**. The desktop app spawns the Next.js
standalone server as a child process, points a `BrowserWindow` at it, and adds a
system tray, auto-updater, IPC bridge, and zero-config secret bootstrap.
## Architecture
```
┌──────────────────────────────────────────────┐
│ Electron main process (electron/main.js) │
│ ├─ Single-instance lock │
│ ├─ Child process: Next.js standalone server │
│ │ (spawned with Electron's Node runtime) │
│ ├─ BrowserWindow → http://localhost:PORT │
│ ├─ System tray + context menu │
│ ├─ Auto-update via electron-updater │
│ ├─ Content Security Policy (session headers) │
│ └─ Secret bootstrap (JWT / API_KEY_SECRET) │
└──────────────────────────────────────────────┘
↕ IPC bridge (electron/preload.js)
┌──────────────────────────────────────────────┐
│ Renderer (Next.js dashboard) │
│ window.electronAPI.* (contextIsolation) │
└──────────────────────────────────────────────┘
```
## Versions
Confirmed from `electron/package.json`:
| Package | Version |
| ------------------ | -------------------------- |
| `electron` | `^41.5.1` |
| `electron-builder` | `^26.10.0` |
| `electron-updater` | `^6.8.5` |
| `better-sqlite3` | `^12.9.0` |
| App version | `3.8.0` |
| App id | `online.omniroute.desktop` |
| Product name | `OmniRoute` |
## Scripts (root `package.json`)
| Script | Purpose |
| --------------------------------- | -------------------------------------------------------------------------- |
| `npm run electron:dev` | Starts `npm run dev` + waits for `localhost:20128` + launches Electron |
| `npm run electron:build` | Builds Next.js then runs `electron-builder` for the current OS |
| `npm run electron:build:win` | Builds Windows NSIS installer + portable (x64) |
| `npm run electron:build:mac` | Builds macOS DMG (Intel + Apple Silicon) |
| `npm run electron:build:linux` | Builds Linux AppImage + DEB (x64 + arm64) |
| `npm run electron:smoke:packaged` | Launches packaged binary and probes `/login` for HTTP 200, then shuts down |
The `electron/` workspace also exposes:
- `npm run prepare:bundle` — runs `scripts/build/prepare-electron-standalone.mjs`
- `npm run build:mac-x64` / `build:mac-arm64` — single-arch macOS builds
- `npm run pack` — directory-only build for local testing (no installer)
## Directory Layout
```
electron/
├── package.json # Electron deps + electron-builder config
├── main.js # Main process (24 KB — see annotations below)
├── preload.js # contextBridge IPC bridge
├── types.d.ts # AppInfo / ServerStatus / ElectronAPI types
├── README.md # In-workspace notes
├── assets/ # icon.png, icon.ico, icon.icns, tray-icon.png
└── dist-electron/ # electron-builder output (gitignored)
scripts/
├── build/
│ └── prepare-electron-standalone.mjs # Stages .next/electron-standalone bundle
└── dev/
└── smoke-electron-packaged.mjs # Post-build smoke test
```
Both `main.js` and `preload.js` are **CommonJS `.js` files**, not TypeScript. The
renderer-side typings live in `electron/types.d.ts`.
## IPC Bridge (`preload.js`)
The preload exposes a whitelisted API on `window.electronAPI` using `contextBridge`
with `contextIsolation: true` and `nodeIntegration: false`.
```javascript
const VALID_CHANNELS = {
invoke: [
"get-app-info",
"open-external",
"get-data-dir",
"restart-server",
"check-for-updates",
"download-update",
"install-update",
"get-app-version",
],
send: ["window-minimize", "window-maximize", "window-close"],
receive: ["server-status", "port-changed", "update-status"],
};
```
Exposed methods:
| Renderer call | Type |
| ----------------------------------------------------------------- | -------------------------- |
| `getAppInfo()` → `{ name, version, platform, isDev, port }` | invoke |
| `openExternal(url)` | invoke |
| `getDataDir()` | invoke |
| `restartServer()` | invoke |
| `getAppVersion()` | invoke |
| `checkForUpdates()` / `downloadUpdate()` / `installUpdate()` | invoke |
| `minimizeWindow()` / `maximizeWindow()` / `closeWindow()` | send |
| `onServerStatus(cb)` / `onPortChanged(cb)` / `onUpdateStatus(cb)` | receive (returns disposer) |
The receive helpers return a **disposer function** rather than relying on
`removeAllListeners` — this prevents listener accumulation when React components
remount.
## Server Lifecycle
`main.js` spawns the Next.js standalone bundle directly with the Electron Node
runtime to avoid native-module ABI mismatch with system Node:
```js
spawn(process.execPath, [serverScript], {
cwd: NEXT_SERVER_PATH,
env: { ...serverEnv, PORT, NODE_ENV: "production", ELECTRON_RUN_AS_NODE: "1", NODE_PATH },
stdio: "pipe",
});
```
Highlights:
- `waitForServer()` polls the URL up to 30 s before showing the window (no blank screen on cold start).
- `stdio: "pipe"` captures stdout/stderr; ready phrases (`Ready` / `listening`) emit `server-status: running` over IPC.
- `before-quit` waits up to 5 s for graceful SIGTERM (WAL checkpoint) then sends SIGKILL.
- Port switcher in the tray (`20128`, `3000`, `8080`) stops and restarts the server, then reloads the BrowserWindow.
## Zero-config Secret Bootstrap
On first launch, the main process auto-generates and persists missing secrets:
| Secret | Source |
| ------------------------ | ----------------------------------------------------------------------------------- |
| `JWT_SECRET` | `crypto.randomBytes(64).toString("hex")` |
| `STORAGE_ENCRYPTION_KEY` | `crypto.randomBytes(32).toString("hex")` (refuses if encrypted creds already exist) |
| `API_KEY_SECRET` | `crypto.randomBytes(32).toString("hex")` |
Persisted to `<DATA_DIR>/server.env`. `DATA_DIR` resolves to:
- Windows: `%APPDATA%\omniroute`
- Linux: `$XDG_CONFIG_HOME/omniroute` or `~/.omniroute`
- macOS: `~/.omniroute`
## Window & Tray
- `BrowserWindow`: 1400×900 (min 1024×700), `backgroundColor: "#0a0a0a"`.
- macOS: `titleBarStyle: "hiddenInset"`, traffic-light at `{ x: 16, y: 16 }`.
- Windows/Linux: native title bar.
- Close button minimizes to tray; the tray menu has **Open OmniRoute**, **Open Dashboard** (external browser), **Server Port** submenu, **Check for Updates**, **Quit**.
## Content Security Policy
Set via `session.defaultSession.webRequest.onHeadersReceived`. Notable directives:
- `frame-ancestors 'none'`, `object-src 'none'`, `child-src 'none'`
- `connect-src 'self' http://localhost:* http://127.0.0.1:* ws://localhost:* ws://127.0.0.1:* https://*.omniroute.online https://*.omniroute.dev`
- Dev mode adds `'unsafe-eval'` to `script-src` only
## Auto-update
Uses `electron-updater` with the GitHub provider (`diegosouzapw/OmniRoute`).
- `autoDownload = false`, `autoInstallOnAppQuit = true`
- Events forwarded to renderer via `update-status` IPC:
`checking`, `available`, `not-available`, `downloading` (with `percent`), `downloaded`, `error`
- `installUpdate()` kills the server then calls `autoUpdater.quitAndInstall()`
- Skipped in dev mode (`!app.isPackaged`)
## Build Pipeline
1. `npm run build` → Next.js standalone in `.next/standalone`.
2. `prepare-electron-standalone.mjs` → re-stages into `.next/electron-standalone` and rewrites absolute paths inside `server.js` + `required-server-files.json` so the bundle is relocatable.
3. `electron-builder` packages `main.js`, `preload.js`, `node_modules`, and `extraResources: { ../.next/electron-standalone → app }`.
### Build targets
| OS | Targets |
| ------- | ----------------------------------------- |
| Windows | NSIS installer + portable (x64) |
| macOS | DMG (Intel + arm64, drag-to-Applications) |
| Linux | AppImage + DEB (x64 + arm64) |
NSIS settings: `oneClick: false`, lets the user choose the install directory, creates Desktop and Start-Menu shortcuts.
## Smoke Testing Packaged Build
```bash
npm run electron:smoke:packaged
```
`scripts/dev/smoke-electron-packaged.mjs`:
- Auto-discovers the packaged binary in `electron/dist-electron/` for the current platform.
- Launches with isolated `HOME`/`APPDATA`/`XDG_*` directories so it doesn't touch developer data.
- Polls `http://127.0.0.1:20128/login` for HTTP 200 within 45 s.
- Watches stderr/stdout for fatal patterns (`Cannot find module`, `MODULE_NOT_FOUND`, `ERR_DLOPEN_FAILED`, `Failed to start server`, etc.).
- Waits 2 s of stable runtime after readiness, then issues SIGTERM and waits for the port to free.
- In CI, automatically passes `--no-sandbox --disable-gpu` (and `--disable-dev-shm-usage` on Linux).
Env overrides: `ELECTRON_SMOKE_APP_EXECUTABLE`, `ELECTRON_SMOKE_URL`, `ELECTRON_SMOKE_TIMEOUT_MS`, `ELECTRON_SMOKE_SETTLE_MS`, `ELECTRON_SMOKE_DATA_DIR`, `ELECTRON_SMOKE_KEEP_DATA`, `ELECTRON_SMOKE_STREAM_LOGS`.
## Code Signing
`electron/package.json` does **not** wire signing credentials directly. Pass them via env vars to `electron-builder`:
### macOS
```bash
export APPLE_ID=<email>
export APPLE_APP_SPECIFIC_PASSWORD=<password>
export APPLE_TEAM_ID=<id>
export CSC_LINK=path/to/cert.p12
export CSC_KEY_PASSWORD=<cert-password>
npm run electron:build:mac
```
### Windows
```bash
export CSC_LINK=path/to/cert.pfx
export CSC_KEY_PASSWORD=<cert-password>
npm run electron:build:win
```
### Linux
AppImage signing is optional — set `LINUX_GPG_KEY` if signing.
## Distribution
Artifacts land in `electron/dist-electron/`:
- `OmniRoute Setup X.Y.Z.exe`, `OmniRoute-X.Y.Z-portable.exe` (Windows)
- `OmniRoute-X.Y.Z-mac.dmg`, `OmniRoute-X.Y.Z-arm64-mac.dmg` (macOS)
- `OmniRoute-X.Y.Z.AppImage`, `omniroute-desktop_X.Y.Z_amd64.deb` (Linux)
Releases are published to GitHub Releases (`diegosouzapw/OmniRoute`), which is also where `electron-updater` checks for new versions.
## Troubleshooting
| Symptom | Fix |
| --------------------------------------------------------------- | --------------------------------------------------------------------------- |
| `Cannot find module 'better-sqlite3'` after Electron major bump | `cd electron && npm rebuild` |
| `ERR_DLOPEN_FAILED` for native module | Re-run `prepare:bundle` and verify ABI matches Electron's Node |
| Window appears blank on Linux | Confirm Next.js server actually bound to PORT (check `[Server]` logs) |
| macOS notarization stalls | Ensure `APPLE_*` vars are exported, not just in `.env` |
| Windows SmartScreen warning | Sign with EV cert, or users right-click → "Run anyway" |
| Smoke test fails with port-in-use | Stop any local dev server on 20128 before running `electron:smoke:packaged` |
## See Also
- [SETUP_GUIDE.md](./SETUP_GUIDE.md)
- [RELEASE_CHECKLIST.md](../ops/RELEASE_CHECKLIST.md)
- Source: `electron/main.js`, `electron/preload.js`, `electron/package.json`
- Helpers: `scripts/build/prepare-electron-standalone.mjs`, `scripts/dev/smoke-electron-packaged.mjs`

878
Environment.md Normal file

@@ -0,0 +1,878 @@
# Environment Variables Reference
> Complete reference for every environment variable recognized by OmniRoute.
> For a quick-start template, see [`.env.example`](../../.env.example).
> [!IMPORTANT]
> Every variable documented here must also appear in `.env.example`, and
> every variable in `.env.example` must appear here. `npm run check:env-doc-sync`
> enforces this on commit and in CI. To omit a variable on purpose, add it to
> the allowlist inside `scripts/check-env-doc-sync.mjs`.
---
## Table of Contents
- [1. Required Secrets](#1-required-secrets)
- [2. Storage & Database](#2-storage--database)
- [3. Network & Ports](#3-network--ports)
- [4. Security & Authentication](#4-security--authentication)
- [5. Input Sanitization & PII Protection](#5-input-sanitization--pii-protection)
- [6. Tool & Routing Policies](#6-tool--routing-policies)
- [7. URLs & Cloud Sync](#7-urls--cloud-sync)
- [8. Outbound Proxy](#8-outbound-proxy)
- [9. CLI Tool Integration](#9-cli-tool-integration)
- [10. Internal Agent & MCP Integrations](#10-internal-agent--mcp-integrations)
- [11. OAuth Provider Credentials](#11-oauth-provider-credentials)
- [12. Provider User-Agent Overrides](#12-provider-user-agent-overrides)
- [13. CLI Fingerprint Compatibility](#13-cli-fingerprint-compatibility)
- [14. API Key Providers](#14-api-key-providers)
- [15. Timeout Settings](#15-timeout-settings)
- [16. Logging](#16-logging)
- [17. Memory Optimization](#17-memory-optimization)
- [18. Pricing Sync](#18-pricing-sync)
- [19. Model Sync (Dev)](#19-model-sync-dev)
- [20. Provider-Specific Settings](#20-provider-specific-settings)
- [21. Proxy Health](#21-proxy-health)
- [22. Debugging](#22-debugging)
- [23. GitHub Integration](#23-github-integration)
- [24. Skills Sandbox (v3.8.0+)](#24-skills-sandbox-v380)
- [Deployment Scenarios](#deployment-scenarios)
- [Audit: Removed / Dead Variables](#audit-removed--dead-variables)
---
## 1. Required Secrets
These **must** be set before the first run. Without them, the application will either refuse to start or operate with insecure defaults.
| Variable | Required | Default | Source File | Description |
| ---------------------------- | -------------------- | ---------- | -------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `JWT_SECRET` | **Yes** | _(none)_ | `src/lib/auth` | Signs/verifies all dashboard session cookies (JWT). Generate with `openssl rand -base64 48`. |
| `API_KEY_SECRET` | **Yes** | _(none)_ | `src/lib/db/apiKeys.ts` | AES encryption key for API key values at rest in SQLite. Generate with `openssl rand -hex 32`. |
| `INITIAL_PASSWORD` | **Yes** | `CHANGEME` | Bootstrap script | Sets the initial admin dashboard password (matches `.env.example` default — kept obviously insecure to force a change). **Change before first use.** After login, change via Dashboard → Settings → Security. |
| `OMNIROUTE_WS_BRIDGE_SECRET` | **Yes** (production) | _(unset)_ | `src/app/api/internal/codex-responses-ws/route.ts` | Shared secret for the internal Codex Responses WebSocket bridge. Authenticates bridge requests between the Electron/browser WS relay and OmniRoute. ⚠️ **REQUIRED in production — when unset, all WS bridge requests are rejected.** Generate with `openssl rand -base64 32`. |
### Generation Commands
```bash
# Generate all four secrets at once:
echo "JWT_SECRET=$(openssl rand -base64 48)"
echo "API_KEY_SECRET=$(openssl rand -hex 32)"
echo "INITIAL_PASSWORD=$(openssl rand -base64 16)"
echo "OMNIROUTE_WS_BRIDGE_SECRET=$(openssl rand -base64 32)"
```
> [!CAUTION]
> Never commit `.env` files with real secrets to version control. The `.gitignore` already excludes `.env`, but verify before pushing.
---
## 2. Storage & Database
OmniRoute uses **SQLite** (via `better-sqlite3`) for all persistence. These variables control data location, encryption, and lifecycle.
| Variable | Default | Source File | Description |
| -------------------------------------- | -------------------- | ----------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------- |
| `DATA_DIR` | `~/.omniroute/` | `src/lib/db/core.ts` | Root directory for SQLite DB, backups, and data files. Override for Docker volumes or custom paths. |
| `STORAGE_ENCRYPTION_KEY` | _(empty = disabled)_ | `src/lib/db/encryption.ts` | AES key for full SQLite database encryption at rest. Generate with `openssl rand -hex 32`. |
| `STORAGE_ENCRYPTION_KEY_VERSION` | `v1` | `scripts/build/bootstrap-env.mjs`, `electron/main.js` | Version label for the encryption key. Increment when performing key rotation to support decryption of old backups. |
| `DISABLE_SQLITE_AUTO_BACKUP` | `false` | `src/lib/db/backup.ts` | When `true`, skips the automatic database backup that runs before migrations on every startup. |
| `OMNIROUTE_CRYPT_KEY` | _(unset)_ | `src/lib/db/encryption.ts` | **Legacy alias** for `STORAGE_ENCRYPTION_KEY`. Accepted as a fallback when the primary variable is absent. |
| `OMNIROUTE_API_KEY_BASE64` | _(unset)_ | `src/lib/db/encryption.ts` | **Legacy alias** (Base64-encoded form) accepted as a fallback. Decoded automatically before use. |
| `OMNIROUTE_DB_HEALTHCHECK_INTERVAL_MS` | _(unset)_ | `src/lib/db/core.ts` | Override the periodic SQLite healthcheck interval (ms). When unset, defaults are derived from `NODE_ENV`. |
| `OMNIROUTE_SKIP_DB_HEALTHCHECK` | `0` | `src/lib/db/core.ts`, `src/lib/db/healthCheck.ts` | Set to `1` to skip the DB healthcheck entirely on startup. Useful for short-lived tasks and integration tests. |
| `OMNIROUTE_FORCE_DB_HEALTHCHECK` | `0` | `src/lib/db/core.ts` | Set to `1` to force the DB healthcheck loop on, even when it would normally be skipped (e.g., short-lived tasks). |
| `OMNIROUTE_SKIP_POSTINSTALL` | `0` | `scripts/postinstall.mjs` | Set to `1` to skip the native-runtime warm-up during `npm install`. Useful in CI/headless installs where sqlite is already built. |
| `OMNIROUTE_MIGRATIONS_DIR` | _(auto-detect)_ | `src/lib/db/migrationRunner.ts` | Override the directory that the migration runner scans. Useful when shipping bundled migrations in custom builds. |
| `OMNIROUTE_SPEND_FLUSH_INTERVAL_MS` | _(default in code)_ | `src/lib/spend/batchWriter.ts` | Flush interval (ms) for the batched spend/cost writer. Lower values reduce write coalescing; higher values reduce DB contention. |
| `OMNIROUTE_SPEND_MAX_BUFFER_SIZE` | _(default in code)_ | `src/lib/spend/batchWriter.ts` | Max buffered spend entries before a forced flush. Raise on high-QPS deployments; lower when bounded memory matters more. |
### Scenarios
| Scenario | Configuration |
| --------------------- | -------------------------------------------------------------------------------- |
| **Local development** | Leave all defaults. DB lives at `~/.omniroute/omniroute.db`. |
| **Docker** | `DATA_DIR=/data` + mount a volume at `/data`. |
| **Encrypted at rest** | Set `STORAGE_ENCRYPTION_KEY` + keep backups of the key! Losing it = losing data. |
| **CI/Testing** | `DATA_DIR=/tmp/omniroute-test` — ephemeral, no encryption needed. |
---
## 3. Network & Ports
| Variable | Default | Source File | Description |
| ------------------------------- | ------------------------------- | -------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `PORT` | `20128` | `src/lib/runtime/ports.ts` | Primary port for both Dashboard UI and API endpoints (single-port mode). |
| `API_PORT` | _(unset)_ | `src/lib/runtime/ports.ts` | When set, serves the `/v1/*` proxy API on this separate port. |
| `API_HOST` | `0.0.0.0` | `src/lib/runtime/ports.ts` | Bind address for the API port. |
| `DASHBOARD_PORT` | _(unset)_ | `src/lib/runtime/ports.ts` | When set, serves the Dashboard UI on this separate port. |
| `PROD_DASHBOARD_PORT` | `20130` | `docker-compose.prod.yml` | Host-side published port for the Dashboard in Docker production mode. |
| `PROD_API_PORT` | `20131` | `docker-compose.prod.yml` | Host-side published port for the API in Docker production mode. |
| `OMNIROUTE_PORT` | _(unset)_ | `src/lib/runtime/ports.ts` | Takes precedence over `PORT` when running inside Electron or other wrappers. |
| `NODE_ENV` | `production` | Next.js core | Controls logging verbosity, caching, error detail exposure, and Next.js optimizations. |
| `OMNIROUTE_USE_TURBOPACK` | `1` (default in `.env.example`) | `package.json` / Next.js 16 | Toggles the Next.js 16 Turbopack bundler in `npm run dev` and `npm run build`. Set to `0` on Windows or when running into native binding incompatibilities. |
| `OMNIROUTE_SKIP_DB_HEALTHCHECK` | _(unset)_ | `src/lib/db/core.ts` / `src/lib/db/healthCheck.ts` | Set to `1` to skip the SQLite integrity health check on startup. Useful for faster boot on large databases. |
| `HOST` | `0.0.0.0` | `scripts/dev/run-next.mjs` | Bind address for the Next.js dev/start server. Overrides the default `0.0.0.0` when set. |
| `HOSTNAME` | `127.0.0.1` | `scripts/dev/run-next-playwright.mjs` | Bind address used by the Playwright runner when launching Next.js. Defaults to `127.0.0.1` for hermetic tests. |
### Port Modes
```
┌─────────────────────────── Single Port (default) ──────────────────────────┐
│ PORT=20128 │
│ → Dashboard: http://localhost:20128 │
│ → API: http://localhost:20128/v1/chat/completions │
└─────────────────────────────────────────────────────────────────────────────┘
┌─────────────────────────── Split Ports ─────────────────────────────────────┐
│ DASHBOARD_PORT=20128 │
│ API_PORT=20129 │
│ API_HOST=0.0.0.0 │
│ → Dashboard: http://localhost:20128 │
│ → API: http://0.0.0.0:20129/v1/chat/completions │
│ Use case: Expose API to LAN while restricting Dashboard to localhost. │
└─────────────────────────────────────────────────────────────────────────────┘
┌─────────────────────────── Docker Production ──────────────────────────────┐
│ PROD_DASHBOARD_PORT=443 PROD_API_PORT=8443 │
│ → Maps container ports to host ports in docker-compose.prod.yml. │
└─────────────────────────────────────────────────────────────────────────────┘
```
---
## 4. Security & Authentication
| Variable | Default | Source File | Description |
| --------------------------------------- | ----------------------- | ---------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `MACHINE_ID_SALT` | `endpoint-proxy-salt` | `src/lib/auth` | Salt combined with hardware identifiers for machine fingerprinting. Change per-deployment for isolation. |
| `OMNIROUTE_CLI_SALT` | `omniroute-cli-auth-v1` | `src/lib/machineToken.ts` | HMAC salt for deriving the local CLI auth token. Changing this value rotates all CLI tokens on the machine. See `docs/security/CLI_TOKEN.md`. |
| `AUTH_COOKIE_SECURE` | `false` | `src/lib/auth` | Sets the `Secure` flag on session cookies. **Must be `true`** when running behind HTTPS. |
| `REQUIRE_API_KEY` | `false` | API middleware | When `true`, all `/v1/*` proxy requests must include a valid API key. |
| `ALLOW_API_KEY_REVEAL` | `false` | Dashboard providers page | Allows revealing full API key values in the Dashboard UI. Security risk on shared instances. |
| `NO_LOG_API_KEY_IDS` | _(empty)_ | `src/lib/compliance/index.ts` | Comma-separated API key IDs that bypass request logging (GDPR compliance). |
| `DEFAULT_RATE_LIMIT_PER_DAY` | `1000` | `src/shared/utils/apiKeyPolicy.ts` | Fallback per-day request budget applied to API keys whose `rate_limits` column is null. Default (unset/empty/malformed) keeps the legacy 1000/day, 5000/week, 20000/month windows. Set explicitly to `0` to opt out (unlimited). Any positive integer N enables N/day, 5N/week, 20N/month. Zod-validated; invalid values log a warning and use the legacy default. |
| `MAX_BODY_SIZE_BYTES` | `10485760` (10 MB) | `src/shared/middleware/bodySizeGuard.ts` | Maximum allowed request body size. Rejects payloads exceeding this limit. |
| `CORS_ORIGIN` | `*` | Next.js middleware | CORS `Access-Control-Allow-Origin` value. Restrict for production. |
| `OUTBOUND_SSRF_GUARD_ENABLED` | `true` | `src/shared/network/outboundUrlGuard.ts` | Block provider calls targeting private/loopback/link-local IP ranges. Disable only in isolated test envs. |
| `OMNIROUTE_ALLOW_PRIVATE_PROVIDER_URLS` | `false` | `src/shared/network/outboundUrlGuard.ts` | Allow provider URLs pointing to private/local networks (localhost, 192.168.x.x, 10.x.x.x, etc.). **REQUIRED for self-hosted providers** (LM Studio, Ollama, vLLM, Llamafile, Triton, SearXNG). When `false`, the dashboard rejects validation of local URLs. |
### Hardening Checklist
```bash
# Production security minimum:
AUTH_COOKIE_SECURE=true # Requires HTTPS
REQUIRE_API_KEY=true # Authenticate all proxy calls
ALLOW_API_KEY_REVEAL=false # Never expose keys in UI
CORS_ORIGIN=https://your.domain.com
MAX_BODY_SIZE_BYTES=5242880 # 5 MB limit
```
---
## 5. Input Sanitization & PII Protection
OmniRoute provides a two-layer defense: request-side injection scanning and response-side PII stripping.
### Request-Side: Prompt Injection Guard
| Variable | Default | Source File | Description |
| ------------------------- | --------- | ---------------------------------------- | ------------------------------------------------------------------------------------------- |
| `INPUT_SANITIZER_ENABLED` | `false` | `src/middleware/promptInjectionGuard.ts` | Enable scanning of incoming messages for prompt injection patterns. |
| `INPUT_SANITIZER_MODE` | `warn` | `src/middleware/promptInjectionGuard.ts` | `warn` = log only, `block` = reject request with 400, `redact` = strip suspicious patterns. |
| `INJECTION_GUARD_MODE` | _(unset)_ | `src/middleware/promptInjectionGuard.ts` | Legacy alias for `INPUT_SANITIZER_MODE` — same behavior. |
| `PII_REDACTION_ENABLED` | `false` | `src/middleware/promptInjectionGuard.ts` | Detect PII (emails, phones, SSNs) in incoming requests. |
### Response-Side: PII Sanitizer
| Variable | Default | Source File | Description |
| -------------------------------- | -------- | ------------------------- | ----------------------------------------------------------------------- |
| `PII_RESPONSE_SANITIZATION` | `false` | `src/lib/piiSanitizer.ts` | Scan LLM responses for leaked PII before returning to client. |
| `PII_RESPONSE_SANITIZATION_MODE` | `redact` | `src/lib/piiSanitizer.ts` | `redact` = mask PII, `warn` = log only, `block` = drop entire response. |
### Scenarios
| Scenario | Configuration |
| ------------------------- | ---------------------------------------------------------------------------------------------------------------------------- |
| **Enterprise compliance** | `INPUT_SANITIZER_ENABLED=true`, `INPUT_SANITIZER_MODE=block`, `PII_REDACTION_ENABLED=true`, `PII_RESPONSE_SANITIZATION=true` |
| **Monitoring only** | `INPUT_SANITIZER_ENABLED=true`, `INPUT_SANITIZER_MODE=warn` — logs but never blocks |
| **Personal use** | Leave all disabled — zero overhead |
---
## 6. Tool & Routing Policies
| Variable | Default | Source File | Description |
| ----------------------------------- | ---------------------------- | ----------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------- |
| `TOOL_POLICY_MODE` | `disabled` | `src/lib/toolPolicy.ts` | Controls LLM tool/function-calling access. `allowlist` = only listed tools, `denylist` = all except listed, `disabled` = no restrictions. |
| `OMNIROUTE_PAYLOAD_RULES_PATH` | `./config/payloadRules.json` | `open-sse/services/payloadRules.ts` | Path to payload manipulation rules JSON file (per-model/protocol upstream tweaks). |
| `OMNIROUTE_PAYLOAD_RULES_RELOAD_MS` | `5000` | `open-sse/services/payloadRules.ts` | Reload interval (ms) for hot-reloading the payload rules file. Minimum `1000`. |
---
## 7. URLs & Cloud Sync
| Variable | Default | Source File | Description |
| --------------------------------------- | --------------------------------------------------------------- | ------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `BASE_URL` | `http://localhost:20128` | `src/lib/cloudSync.ts` | Server-side URL for internal sync jobs to call `/api/sync/cloud`. |
| `CLOUD_URL` | _(empty)_ | `src/lib/cloudSync.ts` | Cloud relay endpoint URL (premium feature). |
| `CLOUD_SYNC_TIMEOUT_MS` | `12000` | `src/lib/cloudSync.ts` | HTTP timeout for cloud sync requests. |
| `NEXT_PUBLIC_BASE_URL` | `http://localhost:20128` | OAuth, Dashboard, sync | Public-facing URL for OAuth redirect_uri, Dashboard links. **Must match your public URL behind reverse proxy.** |
| `NEXT_PUBLIC_CLOUD_URL` | _(empty)_ | Client-side | Client-side mirror of `CLOUD_URL`. |
| `NEXT_PUBLIC_APP_URL` | _(unset)_ | `src/shared/services/cloudSyncScheduler.ts` | Legacy fallback for `NEXT_PUBLIC_BASE_URL`. |
| `OMNIROUTE_PUBLIC_BASE_URL` | _(unset)_ | `open-sse/executors/chatgpt-web.ts` | Browser-facing OmniRoute origin used for image URLs in API responses (e.g., `/v1/chatgpt-web/image/<id>`). Set this when OpenWebUI or another relay reaches OmniRoute by an internal URL but the user's browser must fetch images from a LAN, tunnel, or public origin. Do **not** include `/v1`. |
| `OMNIROUTE_CGPT_WEB_IMAGE_TIMEOUT_MS` | `180000` (3 min) | `open-sse/executors/chatgpt-web.ts` | Max wait time for an async chatgpt-web image to land via the celsius WebSocket. Increase during upstream queue-deep windows. |
| `OMNIROUTE_CGPT_WEB_IMAGE_CACHE_MAX_MB` | `256` | `open-sse/services/chatgptImageCache.ts` | Total in-memory byte budget (MB) for the chatgpt-web image cache serving `/v1/chatgpt-web/image/<id>`. Lower on memory-constrained hosts; raise if image generation is heavy and clients race the 30-minute TTL. |
| `KIE_CALLBACK_URL` | _(unset)_ | `open-sse/utils/kieTask.ts` | Public callback URL for asynchronous kie.ai jobs. Highest-priority override before `OMNIROUTE_KIE_CALLBACK_URL` and `OMNIROUTE_PUBLIC_URL`. |
| `OMNIROUTE_KIE_CALLBACK_URL` | _(unset)_ | `open-sse/utils/kieTask.ts` | Alternate spelling of `KIE_CALLBACK_URL`. Falls back when the primary variable is unset. |
| `OMNIROUTE_PUBLIC_URL` | _(unset)_ | `open-sse/utils/kieTask.ts` | Public origin used to compose async callback URLs. Lowest-priority fallback for kie.ai callbacks; also used as a generic public URL for other relays. |
| `OMNIROUTE_CROF_USAGE_URL` | `https://crof.ai/usage_api/` | `open-sse/services/usage.ts` | CrofAI quota lookup endpoint used by the Usage page. Override for relays / test fixtures. |
| `OMNIROUTE_GEMINI_CLI_USAGE_URL` | `https://cloudcode-pa.googleapis.com/v1internal:loadCodeAssist` | `open-sse/services/usage.ts` | Gemini CLI quota lookup endpoint. Override for relays / test fixtures. |
| `OMNIROUTE_CODEWHISPERER_BASE_URL` | `https://codewhisperer.us-east-1.amazonaws.com` | `open-sse/services/usage.ts` | CodeWhisperer (AWS Kiro) usage limits endpoint. Override for relays / test fixtures. |
> [!IMPORTANT]
> When deploying behind a reverse proxy (nginx, Caddy), `NEXT_PUBLIC_BASE_URL` **must** be set to your public URL (e.g., `https://omniroute.example.com`). Without this, OAuth callbacks will fail because the redirect_uri won't match.
---
## 8. Outbound Proxy
Route upstream LLM provider calls through an HTTP or SOCKS5 proxy for egress control, geo-routing, or IP masking.
| Variable | Default | Source File | Description |
| --------------------------------- | --------- | -------------------- | ----------------------------------------------------------------------------------- |
| `ENABLE_SOCKS5_PROXY` | `true` | `open-sse/executors` | Enable SOCKS5 proxy agent for upstream calls. |
| `NEXT_PUBLIC_ENABLE_SOCKS5_PROXY` | `true` | Client-side | Client-side awareness of SOCKS5 availability. |
| `HTTP_PROXY` | _(unset)_ | Node.js standard | HTTP proxy for upstream calls. |
| `HTTPS_PROXY` | _(unset)_ | Node.js standard | HTTPS proxy for upstream calls. |
| `ALL_PROXY` | _(unset)_ | Node.js standard | Universal proxy (supports `socks5://`). |
| `NO_PROXY` | _(unset)_ | Node.js standard | Comma-separated hostnames/IPs to bypass the proxy. |
| `ENABLE_TLS_FINGERPRINT` | `false` | `open-sse/executors` | Spoof TLS fingerprint using wreq-js (mimics Chrome 124). Counters JA3/JA4 blocking. |
### Scenarios
| Scenario | Configuration |
| ----------------------------- | ------------------------------------------------------------------------------------------------------------------------- |
| **SOCKS5 through SSH tunnel** | `ALL_PROXY=socks5://127.0.0.1:7890`, `ENABLE_SOCKS5_PROXY=true` |
| **Corporate HTTP proxy** | `HTTP_PROXY=http://proxy.corp.com:3128`, `HTTPS_PROXY=http://proxy.corp.com:3128`, `NO_PROXY=localhost,internal.corp.com` |
| **Anti-fingerprint** | `ENABLE_TLS_FINGERPRINT=true` — requires `wreq-js` (included) |
---
## 9. CLI Tool Integration
Controls how OmniRoute discovers and launches CLI sidecars (Claude Code, Codex, etc.).
| Variable | Default | Source File | Description |
| ------------------------- | ---------- | ----------------------------------- | ---------------------------------------------------------------------------------- |
| `CLI_MODE` | `auto` | `src/shared/services/cliRuntime.ts` | `auto` = search system PATH; `manual` = use explicit paths only. |
| `CLI_EXTRA_PATHS` | _(unset)_ | `src/shared/services/cliRuntime.ts` | Additional PATH entries for CLI binary discovery (colon-separated). |
| `CLI_CONFIG_HOME` | _(unset)_ | `src/shared/services/cliRuntime.ts` | Override home directory for reading CLI configs (`~/.claude`, `~/.codex`). |
| `CLI_ALLOW_CONFIG_WRITES` | `false` | `src/shared/services/cliRuntime.ts` | Allow OmniRoute to write CLI config files (token refresh, session data). |
| `CLI_CLAUDE_BIN` | `claude` | `src/shared/services/cliRuntime.ts` | Custom path to Claude CLI binary. |
| `CLI_CODEX_BIN` | `codex` | `src/shared/services/cliRuntime.ts` | Custom path to Codex CLI binary. |
| `CLI_DROID_BIN` | `droid` | `src/shared/services/cliRuntime.ts` | Custom path to Droid CLI binary. |
| `CLI_OPENCLAW_BIN` | `openclaw` | `src/shared/services/cliRuntime.ts` | Custom path to OpenClaw CLI binary. |
| `CLI_CURSOR_BIN` | `agent` | `src/shared/services/cliRuntime.ts` | Custom path to Cursor agent binary. |
| `CLI_CLINE_BIN` | `cline` | `src/shared/services/cliRuntime.ts` | Custom path to Cline CLI binary. |
| `CLI_CONTINUE_BIN` | `cn` | `src/shared/services/cliRuntime.ts` | Custom path to Continue CLI binary. |
| `CLI_QODER_BIN` | `qoder` | `src/shared/services/cliRuntime.ts` | Custom path to Qoder CLI binary. |
| `CLI_QWEN_BIN` | `qwen` | `src/shared/services/cliRuntime.ts` | Custom path to the Qwen Code CLI binary. |
| `CLI_DEVIN_BIN` | `devin` | `open-sse/executors/devin-cli.ts` | Custom path to the Devin CLI binary (v3.8.0). Used by the Windsurf/Devin executor. |
### Docker Example
```bash
# Mount host binaries into the container and tell OmniRoute where they are:
CLI_EXTRA_PATHS=/host-cli/bin
CLI_CONFIG_HOME=/root
CLI_ALLOW_CONFIG_WRITES=true
CLI_CLAUDE_BIN=/host-cli/bin/claude
```
### CLI Binary (`omniroute`) helpers
These variables tune the `omniroute` CLI binary's own behavior (not the sidecar
detection above).
| Variable | Default | Source File | Description |
| --------------------------- | ---------- | --------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------- |
| `OMNIROUTE_LANG` | _(system)_ | `bin/cli/i18n.mjs` | Force CLI output language. BCP-47 locale (e.g. `en`, `pt-BR`). Overrides system locale env vars (LC_ALL, LC_MESSAGES). |
| `OMNIROUTE_SHOW_LOG` | _(unset)_ | `bin/cli/runtime/processSupervisor.mjs` | Set to `1` to forward server stdout/stderr to the terminal in supervised mode. Equivalent to `--log` flag on `omniroute serve`. |
| `OMNIROUTE_CLI_TOKEN` | _(unset)_ | `bin/cli/api.mjs` | Machine-auth token injected as `x-omniroute-cli-token` header. Auto-generated in task 8.12. |
| `OMNIROUTE_HTTP_TIMEOUT_MS` | `30000` | `bin/cli/api.mjs` | Per-attempt HTTP timeout (ms) for CLI → server requests. |
| `OMNIROUTE_VERBOSE` | `0` | `bin/cli/api.mjs` | Set to `1` to print retry/backoff diagnostics to stderr during CLI commands. |
| `OMNIROUTE_PLUGIN_PATH` | _(unset)_ | `bin/cli/plugins.mjs` | Custom directory for CLI plugin discovery (`omniroute-cmd-*` packages). Defaults to `~/.omniroute/plugins/` when unset. |
---
## 10. Internal Agent & MCP Integrations
| Variable | Default | Source File | Description |
| ----------------------------------------------- | ----------- | ----------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------- |
| `OMNIROUTE_BASE_URL` | auto-detect | `open-sse/mcp-server/server.ts` | Explicit URL for MCP/A2A tools to reach OmniRoute. Overrides localhost auto-detection. |
| `OMNIROUTE_API_KEY` | _(unset)_ | MCP/A2A modules | API key for internal MCP tool and A2A skill calls. |
| `OMNIROUTE_API_KEY_ID` | _(unset)_ | `open-sse/mcp-server/audit.ts` | Key ID for MCP audit log attribution. |
| `ROUTER_API_KEY` | _(unset)_ | Legacy | Legacy alias for `OMNIROUTE_API_KEY`. |
| `OMNIROUTE_MCP_ENFORCE_SCOPES` | `false` | `open-sse/mcp-server/server.ts` | Enforce scope-based access control on MCP tool calls. |
| `OMNIROUTE_MCP_SCOPES` | _(all)_ | `open-sse/mcp-server/server.ts` | Comma-separated scopes: `admin`, `combos`, `health`, `models`, `routing`, `budget`, `metrics`, `pricing`, `memory`, `skills`. |
| `OMNIROUTE_MCP_COMPRESS_DESCRIPTIONS` | enabled | `open-sse/mcp-server/descriptionCompressor.ts` | Compress MCP tool descriptions before serializing the manifest. Disable values: `0`, `false`, `off`. |
| `OMNIROUTE_MCP_DESCRIPTION_COMPRESSION` | `rtk` | `open-sse/mcp-server/descriptionCompressor.ts` | Compression algorithm/profile. Disable values: `0`, `false`, `off`. |
| `MODEL_SYNC_INTERVAL_HOURS` | `24` | `src/shared/services/modelSyncScheduler.ts` | Model catalog sync interval in hours. |
| `PROVIDER_LIMITS_SYNC_INTERVAL_MINUTES` | `70` | `src/server-init.ts` | Provider rate-limit and quota polling interval. |
| `OMNIROUTE_DISABLE_BACKGROUND_SERVICES` | `false` | `src/instrumentation-node.ts` | Disable all background services (sync, pricing, model refresh). Useful for CI/test. |
| `OMNIROUTE_ENABLE_RUNTIME_BACKGROUND_TASKS` | _(unset)_ | `src/lib/config/runtimeSettings.ts` | Force background tasks on under automated test detection. Set `1` to override the test heuristic. |
| `OMNIROUTE_BUDGET_RESET_JOB_INTERVAL_MS` | `600000` | `src/lib/jobs/budgetResetJob.ts` | Budget reset check cadence (ms). Floor `10000`. |
| `OMNIROUTE_REASONING_CACHE_CLEANUP_INTERVAL_MS` | `1800000` | `src/lib/jobs/reasoningCacheCleanupJob.ts` | Reasoning cache cleanup cadence (ms). Floor `60000`. |
| `OMNIROUTE_CONFIG_HOT_RELOAD_MS` | `5000` | `src/lib/config/hotReload.ts` | Polling interval (ms) for config hot-reload. Lower than `1000` is rejected. |
| `OMNIROUTE_DISABLE_REDIS_AUTH_CACHE` | _(enabled)_ | `src/lib/db/apiKeys.ts` | Set `1` to bypass the Redis-backed API-key auth cache (forces DB reads). |
| `OMNIROUTE_RTK_TRUST_PROJECT_FILTERS` | `0` | `open-sse/services/compression/engines/rtk/filterLoader.ts` | Trust user-managed RTK project filter rules without strict signature checks. |
| `OMNIROUTE_BOOTSTRAPPED` | `false` | `src/app/(dashboard)/dashboard/page.tsx` | Set `true` by bootstrap script after initial setup. Controls setup wizard visibility. |
| `OMNIROUTE_ALLOW_BODY_PROJECT_OVERRIDE` | `0` | `open-sse/executors/antigravity.ts` | Escape hatch: allow request body to override the Antigravity project field. |
| `ANTIGRAVITY_CREDITS` | _(unset)_ | `open-sse/services/antigravityCredits.ts` | Override Antigravity's advertised remaining credits (testing / forced values). |
### OAuth CLI Bridge (Internal)
| Variable | Default | Source File | Description |
| ------------------- | ----------- | ------------------------------- | ----------------------------------------- |
| `OMNIROUTE_SERVER` | auto-detect | `src/lib/oauth/config/index.ts` | Server URL for CLI↔OmniRoute auth bridge. |
| `OMNIROUTE_TOKEN` | _(unset)_ | `src/lib/oauth/config/index.ts` | Auth token for CLI bridge. |
| `OMNIROUTE_USER_ID` | `cli` | `src/lib/oauth/config/index.ts` | User ID for CLI bridge sessions. |
| `SERVER_URL` | _(unset)_ | `src/lib/oauth/config/index.ts` | Legacy alias for `OMNIROUTE_SERVER`. |
| `CLI_TOKEN` | _(unset)_ | `src/lib/oauth/config/index.ts` | Legacy alias for `OMNIROUTE_TOKEN`. |
| `CLI_USER_ID` | _(unset)_ | `src/lib/oauth/config/index.ts` | Legacy alias for `OMNIROUTE_USER_ID`. |
---
## 11. OAuth Provider Credentials
Built-in credentials for **localhost development**. For remote deployments, register your own at each provider's developer console.
| Variable | Provider | Notes |
| --------------------------------- | ----------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `CLAUDE_OAUTH_CLIENT_ID` | Claude Code (Anthropic) | Public client — no secret needed. |
| `CLAUDE_CODE_REDIRECT_URI` | Claude Code | Override redirect URI. Default: `https://platform.claude.com/oauth/code/callback` |
| `CODEX_OAUTH_CLIENT_ID` | Codex / OpenAI | Public client. |
| `GEMINI_OAUTH_CLIENT_ID` | Gemini (Google) | Requires matching `_SECRET`. |
| `GEMINI_OAUTH_CLIENT_SECRET` | Gemini (Google) | — |
| `GEMINI_CLI_OAUTH_CLIENT_ID` | Gemini CLI | Usually same as Gemini. |
| `GEMINI_CLI_OAUTH_CLIENT_SECRET` | Gemini CLI | — |
| `QWEN_OAUTH_CLIENT_ID` | Qwen (Alibaba) | Public client. |
| `KIMI_CODING_OAUTH_CLIENT_ID` | Kimi Coding (Moonshot) | Public client. |
| `ANTIGRAVITY_OAUTH_CLIENT_ID` | Antigravity (Google) | Requires matching `_SECRET`. |
| `ANTIGRAVITY_OAUTH_CLIENT_SECRET` | Antigravity (Google) | — |
| `GITHUB_OAUTH_CLIENT_ID` | GitHub Copilot | Public client. |
| `WINDSURF_FIREBASE_API_KEY` | Windsurf / Devin (v3.8) | Public Firebase Web API key used by Windsurf's Secure Token Service to refresh short-lived browser-flow tokens. Client-side credential (not a secret). Long-lived import tokens skip this entirely. Source: extracted from Devin CLI binary. |
| `WINDSURF_API_KEY` | Windsurf / Devin (v3.8) | API key fallback used by `open-sse/executors/devin-cli.ts` when no per-connection credential is available. Optional. |
| `CLI_DEVIN_BIN` | Devin CLI (v3.8) | Custom path to the Devin CLI binary (`devin`). Resolved by `open-sse/executors/devin-cli.ts`. |
| `GITLAB_DUO_OAUTH_CLIENT_ID` | GitLab Duo (v3.8) | OAuth client ID for GitLab Duo. Register an app at `https://gitlab.com/-/profile/applications` with redirect URI `<NEXT_PUBLIC_BASE_URL>/callback` and scopes `api, read_user, openid, profile, email`. Falls back to `GITLAB_OAUTH_CLIENT_ID`. |
| `GITLAB_DUO_OAUTH_CLIENT_SECRET` | GitLab Duo (v3.8) | OAuth client secret for GitLab Duo. Optional — PKCE flow does not require a secret. Falls back to `GITLAB_OAUTH_CLIENT_SECRET`. |
| `GITLAB_DUO_BASE_URL` | GitLab Duo (v3.8) | Override GitLab base URL (self-hosted GitLab). Defaults to `https://gitlab.com`. Falls back to `GITLAB_BASE_URL`. |
| `GITLAB_BASE_URL` | GitLab Duo (v3.8) | Legacy fallback for `GITLAB_DUO_BASE_URL`. Used when the `_DUO_` variant is unset. |
| `GITLAB_OAUTH_CLIENT_ID` | GitLab Duo (v3.8) | Legacy fallback for `GITLAB_DUO_OAUTH_CLIENT_ID` consumed by `src/lib/oauth/constants/oauth.ts`. |
| `GITLAB_OAUTH_CLIENT_SECRET` | GitLab Duo (v3.8) | Legacy fallback for `GITLAB_DUO_OAUTH_CLIENT_SECRET` consumed by `src/lib/oauth/constants/oauth.ts`. |
| `QODER_OAUTH_CLIENT_SECRET` | Qoder | — |
| `QODER_OAUTH_AUTHORIZE_URL` | Qoder | Set to enable Qoder OAuth. |
| `QODER_OAUTH_TOKEN_URL` | Qoder | — |
| `QODER_OAUTH_USERINFO_URL` | Qoder | — |
| `QODER_OAUTH_CLIENT_ID` | Qoder | — |
| `QODER_PERSONAL_ACCESS_TOKEN` | Qoder | Direct API key fallback (bypasses OAuth). |
| `QODER_CLI_WORKSPACE` | Qoder | Workspace ID for Qoder CLI. |
| `OMNIROUTE_QODER_WORKSPACE` | Qoder | Alias for `QODER_CLI_WORKSPACE`. |
| `BLACKBOX_WEB_VALIDATED_TOKEN` | Blackbox Web | Frontend `tk` token to send as `validated` on `/api/chat`. Required when Blackbox enforces token matching; otherwise OmniRoute falls back to a random UUID. See issue #2252. |
| `VISION_BRIDGE_BASE_URL` | Vision Bridge guardrail | OpenAI-compatible base URL for non-Anthropic vision-bridge calls. Defaults to the legacy OpenAI URL env or api.openai.com. Point at OmniRoute's `/v1` self-loop or any OpenAI-compat endpoint (Gemini OpenAI-compat, OpenRouter). Issue #2232. |
| `VISION_BRIDGE_API_KEY` | Vision Bridge guardrail | API key for the URL above. Overrides per-provider OpenAI / Google env vars for non-Anthropic vision-bridge calls. Anthropic models keep their dedicated Anthropic key path. Issue #2232. |
> [!WARNING]
> **Google OAuth** (Antigravity, Gemini CLI) credentials **only work on localhost**. For remote servers:
>
> 1. Go to [Google Cloud Console → Credentials](https://console.cloud.google.com/apis/credentials)
> 2. Create an OAuth 2.0 Client ID (type: "Web application")
> 3. Add your server URL as Authorized redirect URI
> 4. Replace the credential values in `.env`.
---
## 12. Provider User-Agent Overrides
Override the `User-Agent` header sent to each upstream provider. This is dynamically resolved at runtime by the executor base class:
```
process.env[`${PROVIDER_ID}_USER_AGENT`]
```
> **Source:** `open-sse/executors/base.ts` → `buildHeaders()`
| Variable | Default Value | When to Update |
| ------------------------ | --------------------------------------------- | ------------------------------------------------------------- |
| `CLAUDE_USER_AGENT` | `claude-cli/2.1.145 (external, cli)` | When Anthropic releases a new CLI version |
| `CODEX_USER_AGENT` | `codex-cli/0.132.0 (Windows 10.0.26200; x64)` | When OpenAI updates the Codex CLI |
| `CODEX_CLIENT_VERSION` | `0.131.0` | Override Codex client version independently of full UA string |
| `GITHUB_USER_AGENT` | `GitHubCopilotChat/0.45.1` | When GitHub Copilot Chat updates |
| `ANTIGRAVITY_USER_AGENT` | `antigravity/2.0.1 darwin/arm64` | When Antigravity IDE updates |
| `KIRO_USER_AGENT` | `AWS-SDK-JS/3.0.0 kiro-ide/1.0.0` | When Kiro IDE updates |
| `QODER_USER_AGENT` | `Qoder-Cli` | When Qoder CLI updates |
| `QWEN_USER_AGENT` | `QwenCode/0.15.9 (linux; x64)` | When Qwen Code updates |
| `CURSOR_USER_AGENT` | `Cursor/3.3` | When Cursor updates |
| `GEMINI_CLI_USER_AGENT` | `google-api-nodejs-client/10.3.0` | When Google API client updates |
> [!TIP]
> You can add User-Agent overrides for **any** provider using the pattern `{PROVIDER_ID}_USER_AGENT`. The executor dynamically constructs the env var name.
---
## 13. CLI Fingerprint Compatibility
When enabled, OmniRoute reorders HTTP headers and JSON body fields to match the exact signature of official CLI tools. This reduces the risk of account flagging while preserving your proxy IP.
**Source:** `open-sse/config/cliFingerprints.ts`, `open-sse/executors/base.ts`
### Per-Provider
| Variable | Activation | Effect |
| ------------------------ | ---------- | --------------------------------------- |
| `CLI_COMPAT_CODEX` | `=1` | Mimics Codex CLI request signature |
| `CLI_COMPAT_CLAUDE` | `=1` | Mimics Claude Code request signature |
| `CLI_COMPAT_GITHUB` | `=1` | Mimics GitHub Copilot request signature |
| `CLI_COMPAT_ANTIGRAVITY` | `=1` | Mimics Antigravity request signature |
| `CLI_COMPAT_CURSOR` | `=1` | Mimics Cursor request signature |
| `CLI_COMPAT_KIMI_CODING` | `=1` | Mimics Kimi Coding request signature |
| `CLI_COMPAT_KILOCODE` | `=1` | Mimics Kilo Code request signature |
| `CLI_COMPAT_CLINE` | `=1` | Mimics Cline request signature |
| `CLI_COMPAT_QWEN` | `=1` | Mimics Qwen Code request signature |
### Global
| Variable | Activation | Effect |
| ---------------- | ---------- | --------------------------------------------------------------- |
| `CLI_COMPAT_ALL` | `=1` | Enable fingerprint compatibility for **all** providers at once. |
### Kimi Coding CLI identity overrides
| Variable | Default | Source File | Description |
| ----------------------- | -------------------- | ---------------------------------------- | ------------------------------------------------------------ |
| `KIMI_CLI_VERSION` | `1.36.0` | `src/lib/oauth/providers/kimi-coding.ts` | Override the Kimi CLI version sent during OAuth/API calls. |
| `KIMI_CODING_DEVICE_ID` | _(captured default)_ | `src/lib/oauth/providers/kimi-coding.ts` | Override the captured Kimi device ID used in client headers. |
> [!NOTE]
> This feature works alongside the User-Agent overrides (§12). The fingerprint system handles header ordering and body field ordering, while User-Agent overrides handle the specific UA string. Both can be enabled independently.
---
## 14. API Key Providers
API keys for providers that use direct authentication. **Preferred setup:** Dashboard → Providers → Add API Key.
Setting via environment variables is an alternative for Docker or headless deployments.
Recognized pattern: `{PROVIDER_ID}_API_KEY`
| Variable | Provider |
| ------------------ | ---------- |
| `DEEPSEEK_API_KEY` | DeepSeek |
| `NVIDIA_API_KEY` | NVIDIA NIM |
> [!NOTE]
> Static `${PROVIDER}_API_KEY` entries for Groq, xAI, Mistral, Perplexity, Together AI, Fireworks, Cerebras, Cohere, Nebius, and Qianfan were removed in v3.8.0 because the runtime no longer reads them — those providers rely exclusively on Dashboard / `data/provider-credentials.json` / the encrypted DB. See the _Audit: Removed / Dead Variables_ section at the bottom of this document for the migration path.
> [!TIP]
> Keys set via the Dashboard are stored encrypted in SQLite and take precedence over environment variables.
---
## 15. Timeout Settings
All values are in **milliseconds**. Centralized resolution in `src/shared/utils/runtimeTimeouts.ts`.
### Timeout Hierarchy
```
REQUEST_TIMEOUT_MS (global override)
├─→ FETCH_TIMEOUT_MS (upstream provider calls, default: 600000)
│ ├─→ FETCH_HEADERS_TIMEOUT_MS (inherits from FETCH_TIMEOUT_MS)
│ ├─→ FETCH_BODY_TIMEOUT_MS (inherits from FETCH_TIMEOUT_MS)
│ ├─→ TLS_CLIENT_TIMEOUT_MS (inherits from FETCH_TIMEOUT_MS)
│ ├── FETCH_CONNECT_TIMEOUT_MS (independent, default: 30000)
│ └── FETCH_KEEPALIVE_TIMEOUT_MS (independent, default: 4000)
├─→ STREAM_IDLE_TIMEOUT_MS (inherits from REQUEST_TIMEOUT_MS, default: 600000)
└─→ API_BRIDGE_PROXY_TIMEOUT_MS (inherits from REQUEST_TIMEOUT_MS, default: 30000)
├─→ API_BRIDGE_SERVER_REQUEST_TIMEOUT_MS (derived, default: 300000)
├── API_BRIDGE_SERVER_HEADERS_TIMEOUT_MS (default: 60000)
├── API_BRIDGE_SERVER_KEEPALIVE_TIMEOUT_MS (default: 5000)
└── API_BRIDGE_SERVER_SOCKET_TIMEOUT_MS (default: 0 = disabled)
```
| Variable | Default | Description |
| ---------------------------------------- | -------------------- | ------------------------------------------------------------------------------------------- |
| `REQUEST_TIMEOUT_MS` | _(unset)_ | Global shortcut — overrides both `FETCH_TIMEOUT_MS` and `STREAM_IDLE_TIMEOUT_MS` defaults. |
| `FETCH_TIMEOUT_MS` | `600000` | Total HTTP request timeout for upstream provider calls. |
| `STREAM_IDLE_TIMEOUT_MS` | `600000` | Max silence between SSE chunks before aborting. Extended-thinking models rarely pause >90s. |
| `FETCH_HEADERS_TIMEOUT_MS` | = `FETCH_TIMEOUT_MS` | Time to receive response headers. |
| `FETCH_BODY_TIMEOUT_MS` | = `FETCH_TIMEOUT_MS` | Time to receive the full response body. |
| `FETCH_CONNECT_TIMEOUT_MS` | `30000` | TCP connection establishment timeout. |
| `FETCH_KEEPALIVE_TIMEOUT_MS` | `4000` | Keep-alive socket idle timeout. |
| `TLS_CLIENT_TIMEOUT_MS` | = `FETCH_TIMEOUT_MS` | TLS fingerprint proxy (wreq-js) timeout. |
| `API_BRIDGE_PROXY_TIMEOUT_MS` | `30000` | Proxy hop timeout for `/v1` bridge requests. |
| `API_BRIDGE_SERVER_REQUEST_TIMEOUT_MS` | `300000` | Overall server request timeout for the bridge. |
| `API_BRIDGE_SERVER_HEADERS_TIMEOUT_MS` | `60000` | Time to send response headers via the bridge. |
| `API_BRIDGE_SERVER_KEEPALIVE_TIMEOUT_MS` | `5000` | Bridge keep-alive idle timeout. |
| `API_BRIDGE_SERVER_SOCKET_TIMEOUT_MS` | `0` | Raw socket timeout (0 = disabled). |
| `SHUTDOWN_TIMEOUT_MS` | `30000` | Grace period on SIGTERM/SIGINT before force-exit. |
| `OMNIROUTE_DEFAULT_FETCH_TIMEOUT_MS` | `120000` | Fallback used by `src/shared/utils/fetchTimeout.ts` when `FETCH_TIMEOUT_MS` is unset. |
| `OMNIROUTE_CHATGPT_TLS_TIMEOUT_MS` | `60000` | Wire-level timeout for the bogdanfinn/tls-client koffi binding (`chatgptTlsClient.ts`). |
| `OMNIROUTE_CHATGPT_TLS_GRACE_MS` | `10000` | JS-side grace added on top of the wire timeout when the native binding is wedged. |
| `OMNIROUTE_CLAUDE_TLS_TIMEOUT_MS` | `60000` | Wire-level timeout for the bogdanfinn/tls-client koffi binding (`claudeTlsClient.ts`). |
| `OMNIROUTE_CLAUDE_TLS_GRACE_MS` | `10000` | JS-side grace added on top of the wire timeout when the native binding is wedged. |
| `OMNIROUTE_PPLX_TLS_TIMEOUT_MS` | `30000` | Wire-level timeout for the bogdanfinn/tls-client koffi binding (`perplexityTlsClient.ts`). |
| `OMNIROUTE_PPLX_TLS_GRACE_MS` | `10000` | JS-side grace added on top of the wire timeout when the native binding is wedged. |
### Circuit Breaker Thresholds
Provider-level circuit breaker tuning. Defaults reflect the scaled values used since v3.6 for 500+ connections.
| Variable | Default | Source File | Description |
| --------------------------------------------- | ------- | ------------------------------ | --------------------------------------------------------------------------- |
| `OMNIROUTE_CIRCUIT_BREAKER_OAUTH_THRESHOLD` | `8` | `open-sse/config/constants.ts` | Consecutive failure threshold for OAuth providers before the breaker trips. |
| `OMNIROUTE_CIRCUIT_BREAKER_OAUTH_RESET_MS` | `60000` | `open-sse/config/constants.ts` | Reset window (ms) for OAuth provider breaker. |
| `OMNIROUTE_CIRCUIT_BREAKER_API_KEY_THRESHOLD` | `12` | `open-sse/config/constants.ts` | Consecutive failure threshold for API-key providers. |
| `OMNIROUTE_CIRCUIT_BREAKER_API_KEY_RESET_MS` | `30000` | `open-sse/config/constants.ts` | Reset window (ms) for API-key provider breaker. |
| `OMNIROUTE_CIRCUIT_BREAKER_LOCAL_THRESHOLD` | `2` | `open-sse/config/constants.ts` | Consecutive failure threshold for local providers (Ollama, LM Studio, ...). |
| `OMNIROUTE_CIRCUIT_BREAKER_LOCAL_RESET_MS` | `15000` | `open-sse/config/constants.ts` | Reset window (ms) for local provider breaker. |
### Scenarios
| Scenario | Configuration |
| -------------------------------- | ------------------------------------------------------ |
| **Long-running code generation** | `REQUEST_TIMEOUT_MS=900000` (15 min) |
| **Fast-fail for production API** | `API_BRIDGE_PROXY_TIMEOUT_MS=10000` |
| **Extended thinking models** | `STREAM_IDLE_TIMEOUT_MS=300000` (5 min between chunks) |
---
## 16. Logging
The logging system writes to both stdout and rotated log files. All configuration is read by `src/lib/logEnv.ts`.
| Variable | Default | Description |
| ----------------------------------------- | -------------------------- | --------------------------------------------------------------------------------- |
| `APP_LOG_LEVEL` | `info` | Minimum log level: `debug`, `info`, `warn`, `error`. |
| `APP_LOG_FORMAT` | `text` | Output format: `text` (human-readable) or `json` (structured). |
| `APP_LOG_TO_FILE` | `true` | Write logs to file alongside stdout. |
| `APP_LOG_FILE_PATH` | `logs/application/app.log` | Log file path (relative to project root or `DATA_DIR`). |
| `APP_LOG_MAX_FILE_SIZE` | `50M` | Max file size before rotation. Accepts: `50M`, `1G`, `512K`, or plain bytes. |
| `APP_LOG_RETENTION_DAYS` | `7` | Days to keep rotated application log files. |
| `APP_LOG_MAX_FILES` | `20` | Maximum rotated log file backups. |
| `CALL_LOG_RETENTION_DAYS` | `7` | Days to keep request/call log entries in the database. |
| `CALL_LOG_MAX_ENTRIES` | `10000` | Max call log entries in the in-memory buffer. |
| `CALL_LOGS_TABLE_MAX_ROWS` | `100000` | Max rows in the `call_logs` SQLite table before pruning. |
| `CALL_LOG_PIPELINE_CAPTURE_STREAM_CHUNKS` | `true` | Store stream chunks in pipeline artifacts when `call_log_pipeline_enabled=true`. |
| `CALL_LOG_PIPELINE_MAX_SIZE_KB` | `512` | Max pipeline call log artifact size in KB when `call_log_pipeline_enabled=true`. |
| `PROXY_LOGS_TABLE_MAX_ROWS` | `100000` | Max rows in the `proxy_logs` SQLite table before pruning. |
| `APP_LOG_ROTATION_CHECK_INTERVAL_MS` | `60000` (1 min) | How often `src/lib/logRotation.ts` re-checks the active log file size. |
| `CHAT_LOG_TEXT_LIMIT` | `65536` | Max string length retained in chat log artifacts (default 64 KB). |
| `CHAT_LOG_ARRAY_TAIL_ITEMS` | `24` | Number of array items retained from the tail when truncating chat log payloads. |
| `CHAT_LOG_MAX_DEPTH` | `6` | Max nesting depth before chat log payloads are truncated. |
| `CHAT_LOG_MAX_OBJECT_KEYS` | `80` | Max object keys retained in chat log payloads (0 = unlimited). |
| `CHAT_DEBUG_FILE` | `false` | When true, `serializeArtifactForStorage` skips size-based truncation. Debug only. |
---
## 17. Memory Optimization
| Variable | Default | Description |
| -------------------------- | ------------------------------- | ---------------------------------------------------------------------- |
| `OMNIROUTE_MEMORY_MB` | `256` (Docker) / system default | V8 heap limit. Sets `--max-old-space-size`. |
| `PROMPT_CACHE_MAX_SIZE` | `50` | Max cached system prompt entries. |
| `PROMPT_CACHE_MAX_BYTES` | `2097152` (2 MB) | Max total prompt cache size. |
| `PROMPT_CACHE_TTL_MS` | `300000` (5 min) | Prompt cache entry TTL. |
| `SEMANTIC_CACHE_MAX_SIZE` | `100` | Max cached temperature=0 responses. |
| `SEMANTIC_CACHE_MAX_BYTES` | `4194304` (4 MB) | Max total semantic cache size. |
| `SEMANTIC_CACHE_TTL_MS` | `1800000` (30 min) | Semantic cache entry TTL. |
| `STREAM_HISTORY_MAX` | `50` | Max recent stream events in the Dashboard live view buffer. |
| `CONTEXT_LENGTH_DEFAULT` | `128000` | Global fallback max context length for models without explicit config. |
| `USAGE_TOKEN_BUFFER` | `100` | Extra token headroom reserved when tracking usage quotas. |
### Compression
| Variable | Default | Description |
| ------------------------------------- | ------- | ------------------------------------------------------------------------------------------------------------- |
| `OMNIROUTE_RTK_TRUST_PROJECT_FILTERS` | unset | Trust project `.rtk/filters.json` without a `.rtk/trust.json` hash. Use only in controlled local development. |
### Low-RAM Docker Example
```bash
OMNIROUTE_MEMORY_MB=128
PROMPT_CACHE_MAX_SIZE=20
PROMPT_CACHE_MAX_BYTES=524288 # 512 KB
SEMANTIC_CACHE_MAX_SIZE=25
SEMANTIC_CACHE_MAX_BYTES=1048576 # 1 MB
STREAM_HISTORY_MAX=10
```
---
## 18. Pricing Sync
Automatic model pricing data synchronization from external sources.
| Variable | Default | Source File | Description |
| ----------------------- | ------------- | ------------------------ | ----------------------------- |
| `PRICING_SYNC_ENABLED` | `false` | `src/lib/pricingSync.ts` | Opt-in periodic pricing sync. |
| `PRICING_SYNC_INTERVAL` | `86400` (24h) | `src/lib/pricingSync.ts` | Sync interval in seconds. |
| `PRICING_SYNC_SOURCES` | `litellm` | `src/lib/pricingSync.ts` | Comma-separated data sources. |
---
## 19. Model Sync (Dev)
| Variable | Default | Source File | Description |
| -------------------------- | ------------- | -------------------------- | -------------------------------------------------------- |
| `MODELS_DEV_SYNC_INTERVAL` | `86400` (24h) | `src/lib/modelsDevSync.ts` | Development-time model catalog sync interval in seconds. |
---
## 20. Provider-Specific Settings
| Variable | Default | Source File | Description |
| ----------------------------------------- | ------------------ | --------------------------------------------------------------------- | ------------------------------------------------------------------------------------- |
| `OPENROUTER_CATALOG_TTL_MS` | `86400000` (24h) | `src/lib/catalog/openrouterCatalog.ts` | OpenRouter model catalog cache TTL. |
| `NANOBANANA_POLL_TIMEOUT_MS` | `120000` | `open-sse/handlers/imageGeneration.ts` | Max wait for NanoBanana image generation jobs. |
| `NANOBANANA_POLL_INTERVAL_MS` | `2500` | `open-sse/handlers/imageGeneration.ts` | NanoBanana job polling frequency. |
| `AWS_REGION` | _(unset)_ | `src/lib/providers/validation.ts`, `open-sse/handlers/audioSpeech.ts` | Region used to construct AWS Bedrock endpoints (Kiro, audio). |
| `AWS_DEFAULT_REGION` | _(unset)_ | `src/lib/providers/validation.ts`, `open-sse/handlers/audioSpeech.ts` | Fallback when `AWS_REGION` is not set. |
| `CLOUDFLARE_ACCOUNT_ID` | _(unset)_ | `open-sse/executors/cloudflare-ai.ts` | Account ID for Cloudflare Workers AI. |
| `CLOUDFLARED_BIN` | auto-detect | `src/lib/cloudflaredTunnel.ts` | Custom path to `cloudflared` binary. |
| `SEARCH_CACHE_TTL_MS` | `300000` (5 min) | `open-sse/services/searchCache.ts` | TTL for search API (Perplexity, Brave, etc.) response caching. |
| `ALLOW_MULTI_CONNECTIONS_PER_COMPAT_NODE` | `false` | `src/app/api/providers/route.ts` | Allow multiple simultaneous connections per OpenAI-compatible provider. |
| `ENABLE_CC_COMPATIBLE_PROVIDER` | `false` | `src/shared/utils/featureFlags.ts` | Reveal the experimental CC-compatible provider UI for Claude Code-only relays. |
| `CLIPROXYAPI_HOST` | `127.0.0.1` | `open-sse/executors/cliproxyapi.ts` | CLIProxyAPI bridge host (legacy integration). |
| `CLIPROXYAPI_PORT` | `5544` | `open-sse/executors/cliproxyapi.ts` | CLIProxyAPI bridge port. |
| `CLIPROXYAPI_CONFIG_DIR` | `~/.cli-proxy-api` | `src/lib/versionManager/processManager.ts` | CLIProxyAPI config directory. |
| `LOCAL_HOSTNAMES` | _(empty)_ | `open-sse/config/providerRegistry.ts` | Comma-separated additional hostnames treated as "local" (Docker service names, etc.). |
`ENABLE_CC_COMPATIBLE_PROVIDER` is only for third-party relays that accept Claude Code clients
exclusively. OmniRoute rewrites requests so those relays accept them. If you only want to use
Claude Code CLI, or you are not sure what these relays are, keep this disabled and add a regular
Anthropic-compatible provider instead.
---
## 21. Proxy Health
| Variable | Default | Source File | Description |
| ---------------------------- | ---------------- | ---------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `PROXY_FAST_FAIL_TIMEOUT_MS` | `2000` | `src/lib/proxyHealth.ts` | Fast-fail health check timeout. |
| `PROXY_HEALTH_CACHE_TTL_MS` | `30000` | `src/lib/proxyHealth.ts` | Health check result cache TTL. |
| `RATE_LIMIT_MAX_WAIT_MS` | `120000` (2 min) | `open-sse/services/rateLimitManager.ts` | Max time to wait on a 429 before failing the request. |
| `RATE_LIMIT_AUTO_ENABLE` | _(unset)_ | `open-sse/services/rateLimitManager.ts` | Force the auto-enable rate limit safety net on/off regardless of the persisted Dashboard setting. Accepts `true`/`1`/`on` to force on, `false`/`0`/`off` to force off. |
| `HEALTHCHECK_STAGGER_MS` | `3000` | `src/lib/tokenHealthCheck.ts` | Stagger interval (ms) between provider token healthchecks at startup. |
| `REQUEST_RETRY` | `2` | `src/sse/services/cooldownAwareRetry.ts` | Number of automatic retries on model-scoped cooldown responses before returning error to client. |
| `MAX_RETRY_INTERVAL_SEC` | `30` | `src/sse/services/cooldownAwareRetry.ts` | Max backoff interval (seconds) between cooldown retries. Capped by this value regardless of upstream `Retry-After`. |
---
## 22. Debugging
> [!CAUTION]
> These variables produce **verbose output** and may leak sensitive data. **Never enable in production.**
| Variable | Default | Source File | Description |
| -------------------------------- | ------------------- | ------------------------------------------ | --------------------------------------------------------------------------------- |
| `CURSOR_DEBUG` | _(unset)_ | `open-sse/executors/cursor.ts` | Set `1` to enable verbose Cursor executor logs (decoded SSE chunks, etc.). |
| `CURSOR_STREAM_DEBUG` | _(unset)_ | `open-sse/executors/cursor.ts` | Backward-compatible alias of `CURSOR_DEBUG`. |
| `CURSOR_DUMP_FILE` | _(unset)_ | `open-sse/executors/cursor.ts` | Optional file path that receives raw decoded Cursor chunks when `CURSOR_DEBUG=1`. |
| `CURSOR_STREAM_TIMEOUT_MS` | `300000` | `open-sse/executors/cursor.ts` | Stream idle timeout (ms) for the Cursor executor. |
| `CURSOR_STATE_DB_PATH` | _(probed)_ | `open-sse/utils/cursorVersionDetector.ts` | Override the Cursor state DB lookup used for version detection. |
| `CURSOR_TOKEN` | _(unset)_ | `scripts/ad-hoc/cursor-tap.cjs` | Direct Cursor bearer token used by developer tooling. |
| `OMNIROUTE_LOG_REQUEST_SHAPE` | enabled (`!== "0"`) | `src/app/api/v1/chat/completions/route.ts` | Log content-type/length markers for large chat payloads. Set `"0"` to silence. |
| `DEBUG_RESPONSES_SSE_TO_JSON` | _(unset)_ | `open-sse/handlers/responseTranslator.ts` | Set `true` to log Responses API SSE→JSON translation details. |
| `NEXT_PUBLIC_OMNIROUTE_E2E_MODE` | _(unset)_ | E2E test harness | Set `true` to enable E2E test mode (relaxed auth, test hooks). |
---
## 23. GitHub Integration
Allow users to report issues directly from the Dashboard.
| Variable | Default | Source File | Description |
| --------------------- | --------- | --------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------- |
| `GITHUB_ISSUES_REPO` | _(unset)_ | `src/app/api/v1/issues/report/route.ts` | Repository in `owner/repo` format. |
| `GITHUB_ISSUES_TOKEN` | _(unset)_ | `src/app/api/v1/issues/report/route.ts` | GitHub Personal Access Token with `issues:write` scope. |
| `GITHUB_TOKEN` | _(unset)_ | issue triage / cloud agent helpers | Generic GitHub access token used as fallback for `GITHUB_ISSUES_TOKEN` and consumed by cloud agent helpers in `src/lib/cloudAgent/*`. |
---
## Deployment Scenarios
### Minimal Local Development
```bash
JWT_SECRET=$(openssl rand -base64 48)
API_KEY_SECRET=$(openssl rand -hex 32)
INITIAL_PASSWORD=dev123
PORT=20128
NODE_ENV=development
```
### Docker Production
```bash
JWT_SECRET=<generated>
API_KEY_SECRET=<generated>
INITIAL_PASSWORD=<generated>
STORAGE_ENCRYPTION_KEY=<generated>
DATA_DIR=/data
PORT=20128
API_PORT=20129
NODE_ENV=production
AUTH_COOKIE_SECURE=true
REQUIRE_API_KEY=true
NEXT_PUBLIC_BASE_URL=https://omniroute.example.com
BASE_URL=http://localhost:20128
OMNIROUTE_MEMORY_MB=512
CORS_ORIGIN=https://your-frontend.example.com
```
### Air-Gapped / CI
```bash
JWT_SECRET=test-jwt-secret-for-ci
API_KEY_SECRET=test-api-key-secret-for-ci
INITIAL_PASSWORD=testpass
NODE_ENV=production
OMNIROUTE_DISABLE_BACKGROUND_SERVICES=true
APP_LOG_TO_FILE=false
```
### VPS with Reverse Proxy (nginx + Cloudflare)
```bash
JWT_SECRET=<generated>
API_KEY_SECRET=<generated>
STORAGE_ENCRYPTION_KEY=<generated>
PORT=20128
AUTH_COOKIE_SECURE=true
REQUIRE_API_KEY=true
NEXT_PUBLIC_BASE_URL=https://omniroute.example.com
BASE_URL=http://127.0.0.1:20128
CORS_ORIGIN=https://omniroute.example.com
ENABLE_TLS_FINGERPRINT=true
CLI_COMPAT_ALL=1
```
---
## 24. Skills Sandbox (v3.8.0+)
Limits and safety knobs applied when the Skills framework (`src/lib/skills/`) executes user-defined automations in a sandboxed environment.
| Variable | Default | Source File | Description |
| --------------------------------- | --------------------------------------------- | ---------------------------- | ------------------------------------------------------------------------------------------------------------------ |
| `SKILLS_SANDBOX_TIMEOUT_MS` | `10000` (10 s) | `src/lib/skills/builtins.ts` | Per-execution wall-clock timeout for sandboxed skill code. Hard cap; anything longer is killed. |
| `SKILLS_EXECUTION_TIMEOUT_MS` | _(falls back to `SKILLS_SANDBOX_TIMEOUT_MS`)_ | `src/lib/skills/` | High-level skill orchestration timeout. Set higher than `SKILLS_SANDBOX_TIMEOUT_MS` to allow multi-step workflows. |
| `SKILLS_MAX_FILE_BYTES` | `1048576` (1 MB) | `src/lib/skills/builtins.ts` | Max bytes a skill may read from any single sandboxed file. |
| `SKILLS_MAX_HTTP_RESPONSE_BYTES` | `256000` (250 KB) | `src/lib/skills/builtins.ts` | Max bytes captured from any single HTTP response inside a skill. |
| `SKILLS_MAX_SANDBOX_OUTPUT_CHARS` | `100000` | `src/lib/skills/builtins.ts` | Hard cap on stdout/stderr characters returned from a sandbox invocation. |
| `SKILLS_SANDBOX_NETWORK_ENABLED` | `false` | `src/lib/skills/builtins.ts` | Set `1`/`true` to allow outbound network from inside the sandbox. Defaults to **isolated** for safety. |
| `SKILLS_ALLOWED_SANDBOX_IMAGES` | _(empty)_ | `src/lib/skills/builtins.ts` | Comma-separated allowlist of container images permitted for sandbox execution. Empty means built-in default only. |
| `SKILLS_SANDBOX_DOCKER_IMAGE` | _(built-in default)_ | `src/lib/skills/` | Container image used when spawning a Docker-backed sandbox. Override to pin a custom hardened base image. |
> [!CAUTION]
> Enabling `SKILLS_SANDBOX_NETWORK_ENABLED=true` opens an egress path from arbitrary skill code. Pair with `OUTBOUND_SSRF_GUARD_ENABLED=true` and a strict `CORS_ORIGIN`/proxy policy in shared deployments.
---
## 25. Provider Quotas, Tunnels, Backups & Misc Runtime
Provider quota endpoints, network tunnels (Tailscale, Ngrok, MITM debug proxy), the 1Proxy egress pool, database backups and small per-feature overrides referenced by the executor layer or scripts.
| Variable | Default | Source File | Description |
| -------------------------------- | ------------------------------------- | --------------------------------------------------- | --------------------------------------------------------------------------- |
| `REDIS_URL` | `redis://localhost:6379` | `src/shared/utils/rateLimiter.ts` | Redis connection string for the rate limiter backend. |
| `ALIBABA_CODING_PLAN_HOST` | _(production host)_ | `open-sse/services/bailianQuotaFetcher.ts` | Override the host used to fetch Alibaba Bailian coding-plan quotas. |
| `ALIBABA_CODING_PLAN_QUOTA_URL` | derived from host | `open-sse/services/bailianQuotaFetcher.ts` | Full quota URL override for Alibaba Bailian. |
| `CONTEXT_RESERVE_TOKENS` | `1024` | `open-sse/services/contextManager.ts` | Tokens reserved for completion output when computing prompt budgets. |
| `MODEL_ALIAS_COMPAT_ENABLED` | enabled | `open-sse/services/model.ts` | Toggle the legacy model-alias compatibility layer used by older clients. |
| `COMMAND_CODE_CALLBACK_PORT` | _(unset)_ | `src/app/api/providers/command-code/auth/shared.ts` | Local port used for OAuth-style callbacks from the Command Code CLI helper. |
| `MITM_LOCAL_PORT` | `443` | `src/mitm/server.cjs` | Local bind port for the MITM debug proxy. |
| `MITM_DISABLE_TLS_VERIFY` | `0` | `src/mitm/server.cjs` | Set `1` to disable upstream TLS verification (development only). |
| `ONEPROXY_ENABLED` | `true` | `src/lib/oneproxySync.ts` | Enable the 1Proxy egress pool sync. |
| `ONEPROXY_API_URL` | `https://1proxy-api.aitradepulse.com` | `src/lib/oneproxySync.ts` | 1Proxy service API URL override. |
| `ONEPROXY_MAX_PROXIES` | `500` | `src/lib/oneproxySync.ts` | Maximum proxies imported per sync. |
| `ONEPROXY_MIN_QUALITY_THRESHOLD` | `50` | `src/lib/oneproxySync.ts` | Minimum quality score for imported proxies. |
| `TAILSCALE_BIN` | _(auto-detect)_ | `src/lib/tailscaleTunnel.ts` | Explicit path to the `tailscale` binary. |
| `TAILSCALED_BIN` | _(auto-detect)_ | `src/lib/tailscaleTunnel.ts` | Explicit path to the `tailscaled` daemon binary. |
| `NGROK_AUTHTOKEN` | _(unset)_ | `src/lib/ngrokTunnel.ts` | Authenticates outbound ngrok tunnels. |
| `DB_BACKUP_MAX_FILES` | `20` | `src/lib/db/backup.ts` | Maximum SQLite backup files retained on disk. |
| `DB_BACKUP_RETENTION_DAYS` | `0` | `src/lib/db/backup.ts` | Maximum age (days) of retained backups. `0` disables age-based pruning. |
| `OMNIROUTE_TLS_PROXY_URL` | _(unset)_ | `open-sse/services/chatgptTlsClient.ts` | Override the TLS sidecar URL for tests. Production should leave unset. |
---
## 26. Test & E2E Harness
Used by `scripts/dev/run-next-playwright.mjs`, `scripts/dev/smoke-electron-packaged.mjs`,
`scripts/dev/run-ecosystem-tests.mjs`, and `scripts/build/uninstall.mjs`. Leave every
value below unset in production deployments.
| Variable | Default | Source File | Description |
| ------------------------------------- | -------------------------------- | ----------------------------------------- | ---------------------------------------------------------------------------------------- |
| `OMNIROUTE_E2E_BOOTSTRAP_MODE` | `auth` | `scripts/dev/run-next-playwright.mjs` | E2E bootstrap mode (`auth`, `fresh`, `reuse`) for the Playwright runner. |
| `OMNIROUTE_E2E_PASSWORD` | falls back to `INITIAL_PASSWORD` | `scripts/dev/run-next-playwright.mjs` | Admin password injected into the Playwright environment. |
| `OMNIROUTE_DISABLE_LOCAL_HEALTHCHECK` | `true` | `scripts/dev/run-next-playwright.mjs` | Disable the local healthcheck poll during Playwright runs. |
| `OMNIROUTE_DISABLE_TOKEN_HEALTHCHECK` | `true` | `scripts/dev/run-next-playwright.mjs` | Disable the OAuth token healthcheck loop during tests. |
| `OMNIROUTE_HIDE_HEALTHCHECK_LOGS` | `true` | `scripts/dev/run-next-playwright.mjs` | Silence healthcheck noise in Playwright stdout. |
| `OMNIROUTE_PLAYWRIGHT_SKIP_BUILD` | `0` | `scripts/dev/run-next-playwright.mjs` | Skip the Next.js production build before Playwright starts (CI optimization). |
| `OMNIROUTE_SKIP_UNINSTALL_HOOK` | `0` | `scripts/build/uninstall.mjs` | Skip the OmniRoute uninstall hook (used by CI to keep `node_modules` intact). |
| `ECOSYSTEM_SERVER_WAIT_MS` | `180000` | `scripts/dev/run-ecosystem-tests.mjs` | Wait time (ms) for the server to become healthy before running ecosystem/protocol tests. |
| `ELECTRON_SMOKE_URL` | `http://127.0.0.1:20128/login` | `scripts/dev/smoke-electron-packaged.mjs` | URL the Electron smoke harness expects the packaged app to serve. |
| `ELECTRON_SMOKE_TIMEOUT_MS` | `45000` | `scripts/dev/smoke-electron-packaged.mjs` | Total timeout (ms) before the smoke harness gives up. |
| `ELECTRON_SMOKE_SETTLE_MS` | `2000` | `scripts/dev/smoke-electron-packaged.mjs` | Settle window (ms) after the page loads. |
| `ELECTRON_SMOKE_APP_EXECUTABLE` | _(auto)_ | `scripts/dev/smoke-electron-packaged.mjs` | Explicit path to the packaged Electron executable. |
| `ELECTRON_SMOKE_DATA_DIR` | _(tmpdir)_ | `scripts/dev/smoke-electron-packaged.mjs` | Data directory for the Electron smoke run. |
| `ELECTRON_SMOKE_KEEP_DATA` | `0` | `scripts/dev/smoke-electron-packaged.mjs` | Set `1` to preserve the smoke data directory after the run. |
| `ELECTRON_SMOKE_STREAM_LOGS` | `0` | `scripts/dev/smoke-electron-packaged.mjs` | Set `1` to stream Electron logs to stdout during the run. |
| `CLI_DEVIN_BIN` | _(PATH lookup)_ | `open-sse/executors/devin-cli.ts` | Override the Devin CLI binary path. |
### Docs translation pipeline
Used by `scripts/i18n/run-translation.mjs` (the `npm run i18n:run` command).
All five variables are unset by default — set them in `.env` only on machines
that should be able to run the docs translator.
| Variable | Default | Source File | Description |
| ----------------------------------- | --------- | ---------------------------------- | ------------------------------------------------------------------------- |
| `OMNIROUTE_TRANSLATION_API_URL` | _(unset)_ | `scripts/i18n/run-translation.mjs` | OpenAI-compatible base URL for the translation backend. |
| `OMNIROUTE_TRANSLATION_API_KEY` | _(unset)_ | `scripts/i18n/run-translation.mjs` | Bearer token for the translation backend (never logged). |
| `OMNIROUTE_TRANSLATION_MODEL` | _(unset)_ | `scripts/i18n/run-translation.mjs` | Model id, e.g. `gpt-4o-mini` or `cx/gpt-5.4-mini`. |
| `OMNIROUTE_TRANSLATION_TIMEOUT_MS` | `60000` | `scripts/i18n/run-translation.mjs` | Per-request timeout in milliseconds. |
| `OMNIROUTE_TRANSLATION_CONCURRENCY` | `4` | `scripts/i18n/run-translation.mjs` | Parallel translation requests when running over multiple files / locales. |
---
## Audit: Removed / Dead Variables
The following variables appeared in previous versions of `.env.example` but have **no runtime references** in the current codebase. They have been removed:
| Variable | Reason |
| ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------- |
| `STORAGE_DRIVER=sqlite` | Never read by any source file. SQLite is the only supported driver — no selection needed. |
| `INSTANCE_NAME=omniroute` | Present in old docs/env templates but unused at runtime. May return in a future multi-instance feature. |
| `SQLITE_MAX_SIZE_MB=2048` | Not referenced in source code. Database size is not artificially limited. |
| `SQLITE_CLEAN_LEGACY_FILES=true` | Not referenced in source code. Legacy cleanup was likely removed. |
| `CLI_ROO_BIN` | Not registered in `src/shared/services/cliRuntime.ts`. |
| `CLI_KIMI_CODING_BIN` | Not registered in `src/shared/services/cliRuntime.ts` (Kimi Coding uses OAuth, not a CLI binary). |
| `IFLOW_OAUTH_CLIENT_ID` / `IFLOW_OAUTH_CLIENT_SECRET` | Not referenced anywhere in source code. |
| `CEREBRAS_API_KEY` / `COHERE_API_KEY` / `FIREWORKS_API_KEY` / `GROQ_API_KEY` / `MISTRAL_API_KEY` / `NEBIUS_API_KEY` / `PERPLEXITY_API_KEY` / `TOGETHER_API_KEY` / `XAI_API_KEY` | Removed in v3.8.0. The runtime no longer reads these env vars — credentials come from Dashboard / `data/provider-credentials.json` / encrypted DB. |
| `CURSOR_PROTOBUF_DEBUG` | Removed in v3.8.0. Cursor executor uses `CURSOR_DEBUG` / `CURSOR_STREAM_DEBUG` (see §22). |
| `CLI_COMPAT_KIRO` | Removed in v3.8.0. Kiro is in `CLI_COMPAT_OMITTED_PROVIDER_IDS` — its toggle has no effect. |
| `QIANFAN_API_KEY` | Removed alongside other unused provider API key stubs in v3.8.0. |
### Default Value Corrections
| Variable | Old `.env.example` Value | Actual Code Default | Fixed |
| ------------------------- | ------------------------ | ------------------- | ------------------------------------------------------ |
| `APP_LOG_RETENTION_DAYS` | `90` | `7` | ✅ Removed misleading value; documented `7` as default |
| `CALL_LOG_RETENTION_DAYS` | `90` | `7` | ✅ Removed misleading value; documented `7` as default |

172
Error-Sanitization.md Normal file

@@ -0,0 +1,172 @@
# Error Message Sanitization
> **Source of truth:** `open-sse/utils/error.ts` — `sanitizeErrorMessage`, `buildErrorBody`, `createErrorResult`
> **Tests:** `tests/unit/error-message-sanitization.test.ts`
> **Last updated:** 2026-05-14 — v3.8.0
> **Audience:** Any engineer touching error responses (HTTP routes, SSE streams, executors, MCP handlers).
> **Status:** **MANDATORY** for every code path that returns an error message to a client.
## Why this exists
CodeQL rule `js/stack-trace-exposure` (CWE-209) flags any code path where an error message originating from a runtime exception reaches an HTTP / SSE response without being sanitized. Stack traces and absolute file paths in production responses give attackers:
- Internal directory layout (`/srv/app/src/lib/...`) → reconnaissance for further attacks.
- Library / framework versions inferred from stack frames → targeted exploit selection.
- Sensitive runtime values that may be string-interpolated into errors (DB queries, config values).
The `sanitizeErrorMessage` helper in `open-sse/utils/error.ts` strips both classes of leakage:
1. Multi-line stack traces — only the first line (the actual error message) is kept.
2. Absolute paths (`/...*.{ts,js,tsx,jsx,mjs,cjs}[:line[:col]]` and `C:\...`) — replaced with `<path>`.
## The mandatory pattern
### 1. Building an error response (HTTP / API routes)
Use `buildErrorBody()` — sanitization is built-in:
```ts
import { buildErrorBody } from "@omniroute/open-sse/utils/error.ts";
export async function POST(req: Request) {
try {
// ... handler logic ...
} catch (err) {
return new Response(JSON.stringify(buildErrorBody(500, String(err))), {
status: 500,
headers: { "Content-Type": "application/json" },
});
}
}
```
Or, for the convenience wrappers in the same module:
```ts
import {
errorResponse, // one-shot Response object
writeStreamError, // SSE writer
createErrorResult, // { success: false, status, response, ... } shape
unavailableResponse, // adds Retry-After
providerCircuitOpenResponse,
modelCooldownResponse,
} from "@omniroute/open-sse/utils/error.ts";
```
All of these route through `buildErrorBody` and therefore through `sanitizeErrorMessage`. **You never need to call `sanitizeErrorMessage` manually** when using these helpers.
### 2. Custom error envelopes (rare)
When you can't use the helpers above (e.g. the response shape is dictated by an upstream protocol like Connect-RPC), import `sanitizeErrorMessage` directly:
```ts
import { sanitizeErrorMessage } from "@omniroute/open-sse/utils/error.ts";
const body = JSON.stringify({
error: {
message: sanitizeErrorMessage(rawMessage),
type: "invalid_request_error",
code: "",
},
});
```
This is the only sanctioned way to assemble a custom error body. See `open-sse/executors/cursor.ts::buildErrorResponse` for the reference implementation.
### 3. Logging vs. responding
`sanitizeErrorMessage` should **only** wrap the value that crosses the network boundary. Internal logs (`pino`, `console`) should keep the full message, including stack, so operators can debug. Pattern:
```ts
try {
// ...
} catch (err) {
log.error({ err }, "handler failed"); // full err with stack — internal log
return errorResponse(500, getErrorMessage(err)); // sanitized — sent to client
}
```
### 4. Forbidden patterns
❌ **Never** put raw exception output in a Response body:
```ts
// BAD: stack trace + file paths reach the client
return new Response(JSON.stringify({ error: { message: err.stack || err.message } }), {
status: 500,
});
```
❌ **Never** roll your own first-line splitter:
```ts
// BAD: forgets to strip absolute paths, may drift from the canonical helper
const safe = String(err).split("\n")[0];
```
❌ **Never** sanitize in the route and forget the SSE path. Anything that writes to a stream goes through `writeStreamError` (or its underlying `buildErrorBody`).
❌ **Never** include `process.cwd()`, `__filename`, `__dirname`, env-derived paths in error messages — they bypass the path regex and reveal the deployment topology.
## Coverage in CI
`tests/unit/error-message-sanitization.test.ts` enforces:
- Every route under `/api/model-combo-mappings/*` returns sanitized bodies on 4xx/5xx.
- `sanitizeErrorMessage` strips multi-line stack traces.
- `sanitizeErrorMessage` replaces POSIX and Windows absolute paths with `<path>`.
- `sanitizeErrorMessage` handles `null`/`undefined`/`Error` instance inputs safely.
- `buildErrorBody` never exposes stack traces in its `message` field.
When adding a new route or executor, copy the assertion pattern from this file. The coverage gate (`npm run test:coverage`) enforces ≥75% statements/lines/functions and ≥70% branches — error paths must be covered.
## Related controls
- `js/stack-trace-exposure` CodeQL alerts in `.github/security` should always be **either** fixed via these helpers **or** dismissed with a comment citing this doc.
- The `pino` redaction config (`src/lib/log/redaction.ts` — if present) handles structured log redaction separately. This doc covers only the response-message surface.
- Upstream-header denylist (`src/shared/constants/upstreamHeaders.ts`) covers header leakage — keep both files aligned when adding a new exfiltration concern.
## Upstream details passthrough
`buildErrorBody` accepts an optional third argument `upstreamDetails` (raw
parsed body from the upstream provider). When provided, it is sanitized by
`sanitizeUpstreamDetails` before inclusion in the response as `upstream_details`.
Sanitization rules applied to `upstreamDetails`:
1. String leaves: run through `sanitizeErrorMessage` (strips stacks + absolute paths).
2. Key blocklist: keys matching `/stack|trace|path|file|cwd|dir|password|secret|token|key/i`
are removed.
3. Depth cap: nesting beyond 4 levels is replaced with the string `"[truncated]"`.
4. Arrays are capped at 32 elements.
Only the seven upstream-error `createErrorResult` call sites in `chatCore.ts` pass
`upstreamErrorBody`. Internal OmniRoute errors (SSE parse failures, empty content,
guardrail blocks) do not include `upstream_details`.
Do NOT pass raw `err.stack`, `err.message`, or any string from a runtime exception to
`upstreamDetails`. Those must still go through `errorResponse` / `buildErrorBody(code, msg)`
without an upstream body.
## Known CodeQL limitation: custom sanitizers not recognized
The CodeQL query [`js/stack-trace-exposure`](https://codeql.github.com/codeql-query-help/javascript/js-stack-trace-exposure/) uses a fixed allowlist of sanitizer patterns (e.g. inline `.split("\n")[0]`, `String#replace` with specific regex shapes, access to `.message` on `Error`). It does **not** recognize indirection through a custom helper like our `sanitizeErrorMessage()`.
This means callsites that demonstrably sanitize via this module — for example `open-sse/utils/error.ts::errorResponse` and `open-sse/executors/cursor.ts::buildErrorResponse` — may continue to raise the alert even though the code is functionally safe. Precedent dismissals: `#224`, `#231` (May 2026), both marked `false positive` with technical justification.
**How to handle a new occurrence:**
1. Confirm the callsite actually routes the message through `sanitizeErrorMessage` / `buildErrorBody` / one of the wrappers documented above (read the call chain end-to-end — don't trust a comment).
2. Confirm `tests/unit/error-message-sanitization.test.ts` exercises the path (or add coverage).
3. Dismiss the alert via `gh api ... -X PATCH state=dismissed -f 'dismissed_reason=false positive'` referencing this doc.
4. Do **not** "fix" by inlining `.split("\n")[0]` everywhere — the helper is the single source of truth; duplicating the pattern weakens the sanitizer (loses path scrubbing, length cap, type coercion) for the appearance of placating the scanner.
Adopting opt-in features like CodeQL's [`@codeql/javascript-models` custom sanitizer config](https://codeql.github.com/docs/codeql-language-guides/customizing-library-models-for-javascript/) is the long-term fix; it lives outside this doc.
## References
- [CWE-209: Information Exposure Through an Error Message](https://cwe.mitre.org/data/definitions/209.html)
- [CodeQL `js/stack-trace-exposure`](https://codeql.github.com/codeql-query-help/javascript/js-stack-trace-exposure/)
- [OWASP: Error Handling Cheat Sheet](https://cheatsheetseries.owasp.org/cheatsheets/Error_Handling_Cheat_Sheet.html)
- Commit centralizing the helper: `1a39c31f` — _fix(security): mask public upstream creds + centralize error sanitization_

245
Evals.md Normal file

@@ -0,0 +1,245 @@
# Evaluations (Evals)
> **Source of truth:** `src/lib/evals/`, `src/lib/db/evals.ts`, `src/app/api/evals/`
> **Last updated:** 2026-05-13 — v3.8.0
OmniRoute ships a generic evaluation framework you can use to benchmark routing
configurations, single providers/models, or the bundled "golden set" suites.
Use it to verify routing changes, validate new providers, and gate releases
before promoting them to production traffic.
The framework is implemented as:
- A pure runner (`src/lib/evals/evalRunner.ts`) that registers in-memory
built-in suites, evaluates outputs against expected criteria, and aggregates
scorecards.
- A persistence layer (`src/lib/db/evals.ts`) for custom (user-defined) suites
and historical runs in SQLite.
- An orchestration layer (`src/lib/evals/runtime.ts`) that executes each case
by dispatching real calls to `POST /v1/chat/completions`, captures latency
and outputs, and persists the run.
- REST endpoints under `/api/evals/*` (management-auth only).
- A dashboard surface at `Dashboard → Usage → Evals` (`EvalsTab.tsx`).
## Concepts
### Suite
A suite is a named collection of test cases with a `description` and one or
more cases. Suites come from two sources:
| Source | Where defined | Mutable at runtime? |
| ---------- | --------------------------------------------- | ------------------- |
| `built-in` | Registered via `registerSuite()` at boot | No (code-defined) |
| `custom` | Stored in SQLite `eval_suites` + `eval_cases` | Yes (via API/UI) |
The current built-in suites (see `src/lib/evals/evalRunner.ts`):
- `golden-set` — 10 baseline cases across greeting/math/translation/safety
- `coding-proficiency` — Python/JS/SQL/TS/bug detection
- `reasoning-logic` — syllogisms, word problems, pattern recognition
- `multilingual` — translation and language detection
- `safety-guardrails` — PII, jailbreak, refusal, bias awareness
- `instruction-following` — JSON-only, numbered lists, language constraints
- `codex-comparison` — head-to-head coding tasks intended for compare mode
### Case
Each case carries:
| Field | Description |
| ---------- | ------------------------------------------------------------ |
| `id` | Stable identifier (used to key outputs and metrics) |
| `name` | Human-readable label |
| `model` | Default model when the run uses `suite-default` targeting |
| `input` | `{ messages, max_tokens? }` — sent to `/v1/chat/completions` |
| `expected` | `{ strategy, value }` — scoring rubric (see below) |
| `tags` | Optional labels (e.g. `safety`, `pii`, `jailbreak`) |
### Target
The same suite can be run against different targets. The target schema is
`evalTargetSchema` in `src/shared/validation/schemas.ts`:
| Target type | `id` | Behavior |
| --------------- | ---------- | --------------------------------------------------------------- |
| `suite-default` | `null` | Each case uses its built-in `model` field |
| `model` | model name | Force every case through one direct model (e.g. `gpt-4o`) |
| `combo` | combo name | Run every case through one combo (exercises the routing engine) |
For `model` and `combo`, the `id` field is required (enforced by Zod
`superRefine`). When `compareTarget` is provided, both targets must differ —
the runner persists both runs under the same `runGroupId` for A/B comparison.
## Scoring Rubrics
Implemented in `evaluateCase()` (evalRunner.ts):
| Strategy | Pass when… |
| ---------- | -------------------------------------------------------------------- |
| `exact` | `actualOutput === expected.value` |
| `contains` | `actualOutput.toLowerCase().includes(expected.value.toLowerCase())` |
| `regex` | `new RegExp(expected.value).test(actualOutput)` is truthy |
| `custom` | `expected.fn(actualOutput, evalCase)` returns truthy (built-in only) |
**Note:** Custom-function scoring is reserved for code-defined (built-in)
suites because functions cannot be serialized through the API. The
`evalCaseBuilderSchema` only accepts `contains | exact | regex` for
user-created suites.
There is no LLM-as-judge or embedding-based similarity scorer today — it would
be a clean extension point in `evaluateCase()`.
## Database Schema
Three tables (migrations `030_create_eval_runs.sql` and
`031_create_eval_suites.sql`):
| Table | Purpose |
| ------------- | ---------------------------------------------------------------------------------------------------------------------------- |
| `eval_suites` | Custom suite metadata (`id`, `name`, `description`) |
| `eval_cases` | Cases per suite — `input_json`, `expected_*`, `tags_json` |
| `eval_runs` | Historical runs — `pass_rate`, `total`, `passed`, `failed`, `avg_latency_ms`, `summary_json`, `results_json`, `outputs_json` |
Built-in suites are **not** stored in the DB. They live in memory and are
re-registered every time `evalRunner.ts` is imported.
## REST API
All endpoints require management auth (`requireManagementAuth`) — they are not
part of the public proxy surface.
| Endpoint | Method | Description |
| ----------------------------- | -------- | ------------------------------------------------------------- |
| `/api/evals` | `GET` | List suites + recent runs + scorecard + targets + keys |
| `/api/evals` | `POST` | Run a suite (single or compare) — schema `evalRunSuiteSchema` |
| `/api/evals/{suiteId}` | `GET` | Fetch one suite (built-in or custom) |
| `/api/evals/suites` | `POST` | Create a custom suite — schema `evalSuiteSaveSchema` |
| `/api/evals/suites/{suiteId}` | `GET` | Fetch a custom suite |
| `/api/evals/suites/{suiteId}` | `PUT` | Replace a custom suite (cases get re-inserted) |
| `/api/evals/suites/{suiteId}` | `DELETE` | Delete a custom suite and its cases |
### Running a suite
```bash
curl -X POST http://localhost:20128/api/evals \
-H "Cookie: auth_token=..." \
-H "Content-Type: application/json" \
-d '{
"suiteId": "golden-set",
"target": { "type": "combo", "id": "my-combo" },
"apiKeyId": "optional-api-key-uuid"
}'
```
Optional fields:
- `outputs` — `Record<caseId, string>` of pre-computed outputs. When provided,
the runner **skips dispatch** and only scores the cached outputs (useful for
offline evaluation).
- `compareTarget` — second target to run in parallel; both runs share a
generated `runGroupId` for head-to-head viewing.
- `apiKeyId` — internal API key used to authenticate the dispatched
`/v1/chat/completions` calls. Required when `REQUIRE_API_KEY` is enabled.
### Creating a custom suite
```bash
curl -X POST http://localhost:20128/api/evals/suites \
-H "Cookie: auth_token=..." \
-H "Content-Type: application/json" \
-d '{
"name": "Production smoke",
"description": "Quick sanity check before deploy",
"cases": [
{
"name": "JSON shape",
"model": "gpt-4o",
"input": { "messages": [{ "role": "user", "content": "Reply with {\"ok\": true}" }] },
"expected": { "strategy": "regex", "value": "\"ok\"\\s*:\\s*true" }
}
]
}'
```
## Dispatch Pipeline
`runEvalSuiteAgainstTarget()` (`src/lib/evals/runtime.ts`):
1. Resolves the suite (built-in or custom).
2. For each case, builds a `Request` to `/v1/chat/completions` with the case's
`messages`, the resolved `model`, `stream: false`, and `max_tokens: 512`
(or the case override).
3. Calls the chat handler directly (in-process — no extra HTTP hop).
4. Captures latency and extracts text from either `choices[0].message.content`
or the Responses-API `output[]` payload.
5. Scores all outputs via `runSuite()`, then persists via `saveEvalRun()`.
Cases run **sequentially**. There is no concurrency flag today.
## Dashboard
The UI lives at `Dashboard → Usage → Evals`
(`src/app/(dashboard)/dashboard/usage/components/EvalsTab.tsx`). From there you
can:
- Browse built-in and custom suites with case-by-case preview.
- Create/edit/delete custom suites with the case builder.
- Pick a target (suite defaults / model / combo), optionally a second
`compareTarget`, optionally an API key, then run on demand.
- Inspect run history, per-case pass/fail, latency, and captured outputs.
- See the rolling scorecard aggregated across the latest run per
`(suite, target)` scope.
## Relationship with the Auto-Assessment RFC
A separate, narrower assessment subsystem lives at `src/domain/assessment/`
(see also [AUTO-COMBO.md](../routing/AUTO-COMBO.md) for the live scoring engine).
That subsystem targets the Auto Combo engine — automatically scoring providers and
models so combos can self-heal when upstreams fail. It uses its own runner,
its own categorizer, and its own scoring logic.
The Evals framework documented here is the **broader, general-purpose
testing surface**. Prefer it for arbitrary regression suites, A/B comparisons,
and per-release smoke tests. Use the Auto-Assessment subsystem when you need
real-time provider health to influence routing decisions.
## CI Integration
There is no dedicated `eval:ci` npm script today. Two paths if you want to
gate releases on eval results:
- **HTTP path**: stand up the server, hit `POST /api/evals` with a known
`suiteId` + `target`, and assert `runs[].summary.passRate >= N` in the
response.
- **In-process path**: import `runEvalSuiteAgainstTarget()` from
`@/lib/evals/runtime` from a script, run against a test DB, and check the
returned `PersistedEvalRun.summary`.
Tests covering the route and history live at
`tests/unit/evals-route.test.ts` and `tests/unit/evals-history.test.ts`.
## Extension Points
Common changes and where to make them:
- **New scoring strategy** — extend the `switch (evalCase.expected.strategy)`
block in `evaluateCase()` (`evalRunner.ts`) and widen `EvalCaseStrategy` in
`src/lib/db/evals.ts` plus `evalCaseBuilderSchema` in `schemas.ts`.
- **New built-in suite** — define a suite object and call `registerSuite()` at
the bottom of `evalRunner.ts`. It will be auto-discovered by `listSuites()`.
- **Run with concurrency** — change the sequential `for` loop in
`runEvalSuiteAgainstTarget()` to a bounded `Promise.all` (no concurrency
control exists today).
- **Stream/tool-call cases** — currently the runner forces `stream: false`.
Streaming or tool-aware evaluation would require changes in `runtime.ts`
(capture and aggregate SSE chunks before scoring).
## See Also
- [USER_GUIDE.md](../guides/USER_GUIDE.md) — overall product walkthrough
- [ARCHITECTURE.md](../architecture/ARCHITECTURE.md) — request pipeline reference
- [AUTO-COMBO.md](../routing/AUTO-COMBO.md) — Auto Combo scoring engine (live runtime)
- Source: `src/lib/evals/`, `src/lib/db/evals.ts`, `src/app/api/evals/`
- UI: `src/app/(dashboard)/dashboard/usage/components/EvalsTab.tsx`

322
Features.md Normal file

@@ -0,0 +1,322 @@
# OmniRoute — Dashboard Features Gallery
🌐 **Main README translations:** 🇺🇸 [English](../README.md) | 🇧🇷 [Português (Brasil)](../i18n/pt-BR/README.md) | 🇪🇸 [Español](../i18n/es/README.md) | 🇫🇷 [Français](../i18n/fr/README.md) | 🇮🇹 [Italiano](../i18n/it/README.md) | 🇷🇺 [Русский](../i18n/ru/README.md) | 🇨🇳 [中文 (简体)](../i18n/zh-CN/README.md) | 🇩🇪 [Deutsch](../i18n/de/README.md) | 🇮🇳 [हिन्दी](../i18n/in/README.md) | 🇹🇭 [ไทย](../i18n/th/README.md) | 🇺🇦 [Українська](../i18n/uk-UA/README.md) | 🇸🇦 [العربية](../i18n/ar/README.md) | 🇯🇵 [日本語](../i18n/ja/README.md) | 🇻🇳 [Tiếng Việt](../i18n/vi/README.md) | 🇧🇬 [Български](../i18n/bg/README.md) | 🇩🇰 [Dansk](../i18n/da/README.md) | 🇫🇮 [Suomi](../i18n/fi/README.md) | 🇮🇱 [עברית](../i18n/he/README.md) | 🇭🇺 [Magyar](../i18n/hu/README.md) | 🇮🇩 [Bahasa Indonesia](../i18n/id/README.md) | 🇰🇷 [한국어](../i18n/ko/README.md) | 🇲🇾 [Bahasa Melayu](../i18n/ms/README.md) | 🇳🇱 [Nederlands](../i18n/nl/README.md) | 🇳🇴 [Norsk](../i18n/no/README.md) | 🇵🇹 [Português (Portugal)](../i18n/pt/README.md) | 🇷🇴 [Română](../i18n/ro/README.md) | 🇵🇱 [Polski](../i18n/pl/README.md) | 🇸🇰 [Slovenčina](../i18n/sk/README.md) | 🇸🇪 [Svenska](../i18n/sv/README.md) | 🇵🇭 [Filipino](../i18n/phi/README.md) | 🇨🇿 [Čeština](../i18n/cs/README.md)
Visual guide to every section of the OmniRoute dashboard.
> 📅 **Last updated:** 2026-05-13 — **v3.8.0**
---
## ✨ v3.8.0 Highlights
The v3.7.x → v3.8.0 cycle added zero-config auto routing, new providers, OAuth flows, deeper resilience, and a much richer CLI experience. Headline features below — full details further in the document and in linked specs.
- 🤖 **Auto Combo / Zero-config auto-routing** — use prefixes `auto/coding`, `auto/fast`, `auto/cheap`, `auto/offline`, `auto/smart`, `auto/lkgp`. Backed by a 9-factor scoring engine and 4 curated **mode packs** (ship-fast, cost-saver, quality-first, offline-friendly)
- 🆕 **Command Code provider** (#2199) — first-class registration with model catalog and quota tracking
- 🆕 **Z.AI provider** — new free-tier provider with quota labels
- 🎬 **KIE media expansion** — extended catalog including video generation models
- 🔐 **Windsurf + Devin CLI OAuth flows** (#2168) — end-to-end browser-based login
- 🆓 **9 new free providers** — LLM7, Lepton, Kluster, UncloseAI, BazaarLink, Completions, Enally, FreeTheAi, Command Code
- 🎯 **Manifest-aware tier routing W1–W4** — provider manifests drive weighted tier selection
- 🎨 **Cursor full OpenAI parity** — tool calls, streaming, session management end-to-end
- 📊 **Cursor Pro plan usage** — quota & cycle data surfaced in the provider-limits dashboard
- ⚡ **Service tier breakdown / Codex fast tier analytics** — per-tier consumption visibility
- 📌 **Per-session sticky routing** — Codex sessions pin to the same account between turns
- 🔊 **Inworld TTS enhancements** — voice catalogs, streaming, and latency improvements
- 🔑 **Kiro headless auth** — login via local `kiro-cli` SQLite store, no browser required
- 📉 **DeepSeek quota and limit monitoring** — daily/monthly usage exposed via dashboard
- 🔄 **Reset-aware routing strategy** — combos now prefer accounts whose quota window resets soonest
- ⏱️ **`fallbackDelayMs`** and **dynamic tool limit detection** — finer fallback timing + per-provider tool-count limits
- 🔧 **Background mode degradation (Responses API)** — falls back to synchronous mode with a structured warning when an upstream lacks background polling
- 🚦 **Per-provider 429 classification** + `useUpstream429BreakerHints` toggle — finer breaker behavior using upstream rate-limit hints
- 🩺 **Model cooldowns dashboard** — observe per-model lockouts and manually re-enable from the UI
- 🔒 **MITM dynamic Linux cert detection** — works across Debian/Ubuntu, Fedora/RHEL, Arch, and other distros
- 💻 **CLI enhancement suite** — 20+ commands including `omniroute providers`, `omniroute combos`, `omniroute doctor`, `omniroute setup`
- 🔍 **Qdrant embedding model discovery** — automatic vector-store model probe
- 🔑 **API Manager / Bearer keys with `manage` scope** — perform admin operations programmatically via API
- 🏥 **Combo target health analytics** + **structured combo builder** — per-target health & UI builder for assembling `(provider, model, connection)` steps
- 🤝 **GitLab Duo OAuth provider** — login with GitLab credentials
- 🧠 **Reasoning Replay Cache** — hybrid in-memory + SQLite persistence of reasoning traces
📚 **Related docs:** [Skills Framework](../frameworks/SKILLS.md) · [Memory System](../frameworks/MEMORY.md) · [Cloud Agents](../frameworks/CLOUD_AGENT.md) · [Webhooks](../frameworks/WEBHOOKS.md) · [Reasoning Replay Cache](../routing/REASONING_REPLAY.md)
---
## 🔌 Providers
Manage AI provider connections: OAuth providers (Claude Code, Codex, Gemini CLI), API key providers (Groq, DeepSeek, OpenRouter), and free providers (Qoder, Qwen, Kiro). Kiro accounts include credit balance tracking — remaining credits, total allowance, and renewal date visible in Dashboard → Usage.
![Providers Dashboard](../screenshots/01-providers.png)
---
## 🎨 Combos
Create model routing combos with 14 strategies: priority, weighted, fill-first, round-robin, p2c (power-of-two-choices), random, least-used, cost-optimized, reset-aware, strict-random, auto, lkgp (last-known-good-provider), context-optimized, and **context-relay**. Each combo chains multiple models with automatic fallback and includes quick templates and readiness checks.
Recent combo improvements:
- **Structured combo builder** — create each step by selecting provider, model, and exact account/connection
- **Repeated provider support** — reuse the same provider many times in one combo as long as the `(provider, model, connection)` tuple is unique
- **Combo target health** — analytics and health surfaces now distinguish individual combo targets/steps instead of collapsing everything into model strings
- **Composite tier ordering** — `defaultTier -> fallbackTier` now influences runtime execution/fallback order for top-level combo steps
![Combos Dashboard](../screenshots/02-combos.png)
---
## 📊 Analytics
Comprehensive usage analytics with token consumption, cost estimates, activity heatmaps, weekly distribution charts, and per-provider breakdowns.
![Analytics Dashboard](../screenshots/03-analytics.png)
---
## 🏥 System Health
Real-time monitoring: uptime, memory, version, latency percentiles (p50/p95/p99), cache statistics, provider circuit breaker states, active quota-monitored sessions, and combo target health.
![Health Dashboard](../screenshots/04-health.png)
---
## 🔧 Translator Playground
Four modes for debugging API translations: **Playground** (format converter), **Chat Tester** (live requests), **Test Bench** (batch tests), and **Live Monitor** (real-time stream).
![Translator Playground](../screenshots/05-translator.png)
---
## 🎮 Model Playground _(v2.0.9+)_
Test any model directly from the dashboard. Select provider, model, and endpoint, write prompts with Monaco Editor, stream responses in real-time, abort mid-stream, and view timing metrics.
---
## 🎨 Themes _(v2.0.5+)_
Customizable color themes for the entire dashboard. Choose from 7 preset colors (Coral, Blue, Red, Green, Violet, Orange, Cyan) or create a custom theme by picking any hex color. Supports light, dark, and system mode.
---
## ⚙️ Settings
Comprehensive settings panel with **7 tabs**:
- **General** — System storage, backup management (export/import database)
- **Appearance** — Theme selector (dark/light/system), color theme presets and custom colors, health log visibility, sidebar item visibility controls, Endpoint tunnel visibility controls
- **AI** — AI assistant features, default routing presets (Auto Combo `auto/coding`, `auto/fast`, `auto/cheap`, `auto/smart`), reasoning replay cache, and skill/memory toggles
- **Security** — API endpoint protection, custom provider blocking, IP filtering, session info
- **Routing** — Model aliases, background task degradation, manifest-aware tier routing (W1–W4), `fallbackDelayMs`, per-session sticky routing
- **Resilience** — Rate limit persistence, circuit breaker tuning, auto-disable banned accounts, provider expiration monitoring, **Context Relay** handoff threshold and summary model configuration, per-provider 429 classification & `useUpstream429BreakerHints` toggle, model cooldowns
- **Advanced** — Configuration overrides, configuration audit trail, fallback degradation mode, background mode degradation for Responses API
![Settings Dashboard](../screenshots/06-settings.png)
---
## 🔧 CLI Tools
One-click configuration for AI coding tools: Claude Code, Codex CLI, Gemini CLI, OpenClaw, Kilo Code, Antigravity, Cline, Continue, Cursor, and Factory Droid. Features automated config apply/reset, connection profiles, and model mapping.
![CLI Tools Dashboard](../screenshots/07-cli-tools.png)
---
## 🤖 CLI Agents _(v2.0.11+)_
Dashboard for discovering and managing CLI agents. Shows a grid of 18 built-in agents (Codex, Claude, Goose, Gemini CLI, OpenClaw, Aider, OpenCode, Cline, Qwen Code, ForgeCode, Amazon Q, Open Interpreter, Cursor CLI, Warp, **Windsurf**, **Devin CLI**, **Kimi Coding**, **Command Code**) with:
- **Installation status** — Installed / Not Found with version detection
- **Protocol badges** — stdio, HTTP, etc.
- **Custom agents** — Register any CLI tool via form (name, binary, version command, spawn args)
- **CLI Fingerprint Matching** — Per-provider toggle to match native CLI request signatures, reducing ban risk while preserving proxy IP
- **OAuth-backed agents** — Windsurf & Devin CLI now use browser OAuth flows for authentication (v3.8.0+)
---
## 🔗 Context Relay _(v3.5.5+)_
A combo strategy that preserves session continuity when account rotation happens mid-conversation. Before the active account is exhausted, OmniRoute generates a structured handoff summary in the background. After the next request resolves to a different account, the summary is injected as a system message so the new account continues with full context.
Configurable via combo-level or global settings:
- **Handoff Threshold** — Quota usage percentage that triggers summary generation (default 85%)
- **Max Messages For Summary** — How much recent history to condense
- **Summary Model** — Optional override model for generating the handoff summary
Currently supports Codex account rotation. See [Context Relay documentation](../architecture/ARCHITECTURE.md).
---
## 🗜️ Prompt Compression _(v3.7.9+)_
Context & Cache now exposes dedicated pages for Caveman, RTK, and Compression Combos:
- **Caveman** — language-aware rule packs, preview, output-mode controls, and analytics
- **RTK** — command-aware compression for shell, git, test, build, package, Docker, infra, JSON, and stack-trace output
- **Compression Combos** — named pipelines such as `rtk -> caveman` assigned to routing combos; the default stacked math reaches `~89%` average and `78-95%` eligible-context savings when both engines apply
- **Raw-output recovery** — optional redacted RTK raw-output pointers for debugging compressed failures
See [Compression Guide](../compression/COMPRESSION_GUIDE.md), [RTK Compression](../compression/RTK_COMPRESSION.md), and
[Compression Engines](../compression/COMPRESSION_ENGINES.md).
---
## 🛡️ Proxy Hardening _(v3.5.5+)_
Comprehensive proxy configuration enforcement across the entire request pipeline:
- **Token Health Check** — Background OAuth refresh now resolves proxy config per connection, preventing failures in proxy-required environments
- **API Key Validation** — Provider key validation (`POST /api/providers/validate`) routes through `runWithProxyContext`, honoring provider-level and global proxy settings
- **undici Dispatcher Fix** — Proxy dispatchers use undici's own fetch implementation instead of Node's built-in fetch, resolving `invalid onRequestStart method` errors on Node.js 22
- **Node.js Version Detection** — Login page proactively detects incompatible Node.js versions (24+) and displays a warning banner with instructions to use Node 22 LTS
---
## 📧 Email Privacy Masking _(v3.5.6+)_
OAuth account emails are now masked in the provider dashboard (e.g. `di*****@g****.com`) to prevent accidental exposure when sharing screenshots or recording demos. The full email address remains accessible via hover tooltip (`title` attribute).
---
## 👁️ Model Visibility Toggle _(v3.5.6+)_
The provider page model list now includes:
- **Real-time search/filter bar** — Quickly find specific models
- **Per-model visibility toggle** (👁 icon) — Hidden models are grayed out and excluded from the `/v1/models` catalog
- **Active-count badge** (`N/M active`) — Shows at a glance how many models are enabled vs total
---
## 🔧 OAuth Env Repair _(v3.6.1+)_
One-click "Repair env" action for OAuth providers that restores missing environment variables and fixes broken auth state. Accessible from `Dashboard → Providers → [OAuth Provider] → Repair env`. Automatically detects and repairs:
- Missing OAuth client credentials
- Corrupted env file entries
- Backup path sanitization
---
## 🗑️ Uninstall / Full Uninstall _(v3.6.2+)_
Clean removal scripts for all installation methods:
| Command | Action |
| ------------------------ | ----------------------------------------------------------------------------------- |
| `npm run uninstall` | Removes the system app but **keeps your DB and configurations** in `~/.omniroute`. |
| `npm run uninstall:full` | Removes the app AND permanently **erases all configurations, keys, and databases**. |
---
## 🖼️ Media _(v2.0.3+)_
Generate images, videos, and music from the dashboard. Supports OpenAI, xAI, Together, Hyperbolic, SD WebUI, ComfyUI, AnimateDiff, Stable Audio Open, and MusicGen.
---
## 📝 Request Logs
Real-time request logging with filtering by provider, model, account, and API key. Shows status codes, token usage, latency, and response details.
![Usage Logs](../screenshots/08-usage.png)
---
## 🌐 API Endpoint
Your unified API endpoint with capability breakdown: Chat Completions, Responses API, Embeddings, Image Generation, Reranking, Audio Transcription, Text-to-Speech, Moderations, and registered API keys. Cloudflare Quick Tunnel, Tailscale Funnel, ngrok Tunnel, and cloud proxy support are available for remote access.
![Endpoint Dashboard](../screenshots/09-endpoint.png)
---
## 🔑 API Key Management
Create, scope, and revoke API keys. Each key can be restricted to specific models/providers with full access or read-only permissions. Visual key management with usage tracking.
---
## 📋 Audit Log
Administrative action tracking with filtering by action type, actor, target, IP address, and timestamp. Full security event history.
---
## 🖥️ Desktop Application
Native Electron desktop app for Windows, macOS, and Linux. Run OmniRoute as a standalone application with system tray integration, offline support, auto-update, and one-click install.
Key features:
- Server readiness polling (no blank screen on cold start)
- System tray with port management
- Content Security Policy
- Single-instance lock
- Auto-update on restart
- Platform-conditional UI (macOS traffic lights, Windows/Linux default titlebar)
- Hardened Electron build packaging — symlinked `node_modules` in the standalone bundle is detected and rejected before packaging, preventing runtime dependency on the build machine (v2.5.5+)
- **Graceful shutdown** — Electron `before-quit` shuts down Next.js cleanly, preventing SQLite WAL database locks (v3.6.2+)
📖 See [`electron/README.md`](../../electron/README.md) for full documentation.
---
## 🌐 V1 WebSocket Bridge _(v3.6.6+)_
OmniRoute now supports **OpenAI-compatible WebSocket clients** via the `/v1/ws` upgrade endpoint. The custom `scripts/dev/v1-ws-bridge.mjs` server wraps Next.js and upgrades WS connections to full bidirectional streaming sessions. Authentication uses the same API key or session cookie as HTTP requests.
Key behaviours:
- WS upgrade validated by `src/lib/ws/handshake.ts` before the connection is established
- Streams terminated cleanly on session close or upstream error
- Works alongside the existing HTTP+SSE streaming path simultaneously
---
## 🔑 Sync Tokens & Config Bundle _(v3.6.6+)_
Multi-device and external operator access is now possible via **scoped sync tokens**:
- **`POST /api/sync/tokens`** — Issue a new sync token (scoped, with optional expiry)
- **`DELETE /api/sync/tokens/:id`** — Revoke a token
- **`GET /api/sync/bundle`** — Download a versioned, ETag-keyed JSON snapshot of all non-sensitive settings (passwords redacted)
The config bundle is built by `src/lib/sync/bundle.ts`. Consumers compare the `ETag` response header to detect changes without re-downloading the full payload.
---
## 🧠 GLM Thinking Preset _(v3.6.6+)_
**GLM Thinking (`glmt`)** is now a registered first-class provider: 65 536 max output tokens, 24 576 thinking budget, 900 s default timeout, Claude-compatible API format, and shared usage sync with the GLM family.
**Hybrid token counting** also lands in v3.6.6: when a Claude-compatible provider exposes `/messages/count_tokens`, OmniRoute calls it before large requests with graceful estimation fallback.
---
## 🛡️ Safe Outbound Fetch & SSRF Guard _(v3.6.6+)_
All provider validation and model discovery calls now go through a two-layer outbound guard:
1. **URL guard** (`src/shared/network/outboundUrlGuard.ts`) — Blocks private/loopback/link-local IP ranges before the socket is opened.
2. **Safe fetch wrapper** (`src/shared/network/safeOutboundFetch.ts`) — Applies the URL guard, normalises timeouts, and retries transient errors with exponential backoff.
Guard violations surface as HTTP 422 (`URL_GUARD_BLOCKED`) and are written to the compliance audit log via `providerAudit.ts`.
---
## 🔄 Cooldown-Aware Retries _(v3.6.6+)_
Chat requests now **automatically retry** when an upstream provider returns a model-scoped cooldown. Configurable via `REQUEST_RETRY` (default: 2) and `MAX_RETRY_INTERVAL_SEC` (default: 30 s). Rate-limit header learning improved across `x-ratelimit-reset-requests`, `x-ratelimit-reset-tokens`, and `Retry-After` — per-model cooldown state is visible in the Resilience dashboard.
---
## 📋 Compliance Audit v2 _(v3.6.6+)_
The audit log has been expanded with cursor-based pagination, request context enrichment (request ID, user agent, IP), structured auth events, provider CRUD events with diff context, and SSRF-blocked validation logging. New events emitted by `src/lib/compliance/providerAudit.ts`.

496
Fly-io-Deployment-Guide.md Normal file

@@ -0,0 +1,496 @@
# OmniRoute Fly.io 部署指南
本文档记录 OmniRoute 在 Fly.io 上的实际部署方法，适用于两类场景：
- 首次把当前项目部署到 Fly.io
- 后续代码更新后继续发布
- 新项目参考同样流程部署
本文基于当前项目已经验证通过的配置整理，应用名为 `omniroute`。
---
## 1. 部署目标
- 平台：Fly.io
- 部署方式：本地 `flyctl` 直接发布
- 运行方式：使用仓库内现有 `Dockerfile` 和 `fly.toml`
- 数据持久化：Fly Volume 挂载到 `/data`
- 访问地址：`https://omniroute.fly.dev/`
---
## 2. 当前项目关键配置
当前仓库中的 `fly.toml` 已确认包含以下关键项：
```toml
app = 'omniroute'
primary_region = 'sin'
[[mounts]]
source = 'data'
destination = '/data'
[processes]
app = 'node run-standalone.mjs'
[http_service]
internal_port = 20128
[env]
TZ = "Asia/Shanghai"
HOST = "0.0.0.0"
HOSTNAME = "0.0.0.0"
BIND = "0.0.0.0"
```
说明：
- `app = 'omniroute'` 决定实际部署到哪个 Fly 应用
- `destination = '/data'` 决定持久卷挂载目录
- 本项目必须让 `DATA_DIR=/data`，否则数据库和密钥会写到容器临时目录
---
## 3. 必备工具
### 3.1 安装 Fly CLI
Windows PowerShell：
```powershell
pwsh -Command "iwr https://fly.io/install.ps1 -useb | iex"
```
如果安装脚本在当前环境失败，也可以手动下载 `flyctl` 二进制并放到 `PATH` 中。
### 3.2 登录 Fly 账号
```powershell
flyctl auth login
```
### 3.3 检查登录状态
```powershell
flyctl auth whoami
flyctl version
```
---
## 4. 首次部署当前项目
### 4.1 获取代码并进入目录
```powershell
git clone https://github.com/diegosouzapw/OmniRoute.git
cd OmniRoute
```
### 4.2 确认应用名
打开 `fly.toml`，重点看这一行：
```toml
app = 'omniroute'
```
如果你准备部署到自己的新应用，可改成全局唯一名称，例如：
```toml
app = 'omniroute-yourname'
```
注意：
- 控制台里要看的是与 `fly.toml` 里 `app` 一致的应用
- 以前如果用过别的名字，例如 `oroute`，不要和 `omniroute` 混淆
### 4.3 创建应用
如果该应用尚不存在：
```powershell
flyctl apps create omniroute
```
如果你已经改成别的应用名，把 `omniroute` 替换成你的名字。
### 4.4 首次部署
```powershell
flyctl deploy
```
---
## 5. 必配参数
本项目在 Fly.io 上建议至少配置以下参数。
### 5.1 已验证使用的参数
这些参数已经在当前 `omniroute` 应用上实际部署：
- `API_KEY_SECRET`
- `DATA_DIR`
- `JWT_SECRET`
- `MACHINE_ID_SALT`
- `NEXT_PUBLIC_BASE_URL`
- `OMNIROUTE_WS_BRIDGE_SECRET` (生产环境必需 / required in production / obrigatório em produção — 用于 WebSocket 桥接鉴权 / used for WebSocket bridge authentication)
- `STORAGE_ENCRYPTION_KEY`
### 5.2 关于 `INITIAL_PASSWORD`
当前项目没有设置 `INITIAL_PASSWORD`，因为本次部署按需求不使用它。
如果不设置：
- 启动日志会提示默认密码是 `CHANGEME`
- 部署后应尽快在系统设置中修改登录密码
如果你希望无人值守初始化后台密码，也可以后续补：
- `INITIAL_PASSWORD`
---
## 6. 推荐参数说明
### 6.1 Secrets 中设置
建议放入 Fly Secrets：
| 变量名 | 是否推荐 | 说明 |
| ---------------------------- | --------------------------------- | ----------------------------------------------------------------------------------------- |
| `API_KEY_SECRET` | 必需 | API Key 生成与校验使用 |
| `JWT_SECRET` | 必需 | 登录态和 JWT 签名使用 |
| `OMNIROUTE_WS_BRIDGE_SECRET` | 生产必需 (required / obrigatório) | WebSocket 桥接鉴权密钥 (WebSocket bridge auth / chave de autenticação da ponte WebSocket) |
| `STORAGE_ENCRYPTION_KEY` | 强烈推荐 | 加密存储敏感连接信息 |
| `MACHINE_ID_SALT` | 推荐 | 生成稳定机器标识 |
| `INITIAL_PASSWORD` | 可选 | 首次部署时直接指定后台初始密码 |
| OAuth/API 私密凭证 | 按需 | 各类外部平台鉴权配置 |
### 6.2 当前项目推荐值
| 变量名 | 推荐值 |
| ---------------------- | --------------------------- |
| `DATA_DIR` | `/data` |
| `NEXT_PUBLIC_BASE_URL` | `https://omniroute.fly.dev` |
说明：
- `DATA_DIR=/data` 非常关键，必须与 Fly Volume 挂载点一致
- `NEXT_PUBLIC_BASE_URL` 用于调度器和前端回调等场景
### 6.3 OAuth 回调地址配置 (OAuth callback URL / URL de callback OAuth)
如果你需要在 Fly.io 部署上启用 OAuth 登录类的 provider（例如 Antigravity、Gemini、Cursor 等），必须确保以下两点：
(If you need to enable OAuth-based providers — e.g. Antigravity, Gemini, Cursor — on the Fly.io deployment, make sure of the following two points. / Se precisar habilitar providers via OAuth — p.ex. Antigravity, Gemini, Cursor — na implantação Fly.io, garanta os dois pontos abaixo.)
1. **设置 `NEXT_PUBLIC_BASE_URL` 指向你公开的 HTTPS 域名 (set `NEXT_PUBLIC_BASE_URL` to the public HTTPS domain / defina `NEXT_PUBLIC_BASE_URL` para o domínio HTTPS público)**
```powershell
flyctl secrets set NEXT_PUBLIC_BASE_URL=https://omniroute.fly.dev -a omniroute
```
如果你使用了自定义域名 (if using a custom domain / se usar um domínio personalizado)，请替换为对应域名 (e.g. `https://omniroute.yourdomain.com`)。
2. **在 provider 控制台配置回调 URL (configure the callback URL on the provider console / configure a URL de callback no painel do provider)**
通常格式为 (typical format / formato típico)：
```text
<NEXT_PUBLIC_BASE_URL>/api/oauth/<provider>/callback
```
例如 (e.g. / p.ex.)：
- `https://omniroute.fly.dev/api/oauth/gemini/callback`
- `https://omniroute.fly.dev/api/oauth/antigravity/callback`
- `https://omniroute.fly.dev/api/oauth/cursor/callback`
如果 `NEXT_PUBLIC_BASE_URL` 与 provider 控制台中注册的回调 URL 不一致，OAuth 流程会在浏览器回跳阶段失败 (mismatch between `NEXT_PUBLIC_BASE_URL` and the registered callback URL will cause OAuth to fail at the browser redirect step / divergência entre `NEXT_PUBLIC_BASE_URL` e a URL de callback registrada quebra o OAuth no redirect do navegador)。
---
## 7. 一键设置参数
下面命令会生成安全随机值，并把当前项目需要的参数一次性写入 Fly Secrets。
说明：
- 不包含 `INITIAL_PASSWORD`
- 适用于当前项目 `omniroute`
```powershell
$apiKeySecret = [Convert]::ToHexString((1..32 | ForEach-Object { Get-Random -Minimum 0 -Maximum 256 })).ToLower()
$jwtSecret = [Convert]::ToHexString((1..64 | ForEach-Object { Get-Random -Minimum 0 -Maximum 256 })).ToLower()
$machineIdSalt = [Convert]::ToHexString((1..32 | ForEach-Object { Get-Random -Minimum 0 -Maximum 256 })).ToLower()
$storageKey = [Convert]::ToHexString((1..32 | ForEach-Object { Get-Random -Minimum 0 -Maximum 256 })).ToLower()
$wsBridgeSecret = [Convert]::ToHexString((1..32 | ForEach-Object { Get-Random -Minimum 0 -Maximum 256 })).ToLower()
flyctl secrets set `
API_KEY_SECRET=$apiKeySecret `
JWT_SECRET=$jwtSecret `
MACHINE_ID_SALT=$machineIdSalt `
STORAGE_ENCRYPTION_KEY=$storageKey `
OMNIROUTE_WS_BRIDGE_SECRET=$wsBridgeSecret `
DATA_DIR=/data `
NEXT_PUBLIC_BASE_URL=https://omniroute.fly.dev `
-a omniroute
```
在 Linux / macOS 上，也可以直接用 `openssl rand -hex 32` 生成 (on Linux / macOS, you can also use `openssl rand -hex 32` / em Linux / macOS, também é possível usar `openssl rand -hex 32`)：
```bash
flyctl secrets set OMNIROUTE_WS_BRIDGE_SECRET=$(openssl rand -hex 32) -a omniroute
```
说明 (notes / observações)：
- `OMNIROUTE_WS_BRIDGE_SECRET` 在生产环境必需，缺失会导致 WebSocket 桥接握手失败 (required in production; missing it breaks WebSocket bridge handshake / obrigatório em produção; sem ele o handshake da ponte WebSocket falha)
如果你还要加初始密码：
```powershell
flyctl secrets set INITIAL_PASSWORD=你的强密码 -a omniroute
```
---
## 8. 查看当前参数
```powershell
flyctl secrets list -a omniroute
```
如果控制台 `Secrets` 页面没有显示你期待的变量，先检查：
- 看的应用是不是 `omniroute`
- `fly.toml` 的 `app` 是否和控制台应用一致
---
## 9. 后续更新发布
代码有更新后，发布步骤很简单：
```powershell
git pull
flyctl deploy
```
如果只更新参数，不改代码：
```powershell
flyctl secrets set KEY=value -a omniroute
```
Fly 会自动滚动更新机器。
### 9.1 跟踪原仓库更新并保留 fork 的 `fly.toml`
如果当前仓库是 fork，并且你要同步上游 `https://github.com/diegosouzapw/OmniRoute` 的更新，推荐按下面流程执行。
先确认远程：
```powershell
git remote -v
```
应至少包含：
- `origin` 指向你自己的 fork
- `upstream` 指向原仓库
如果没有 `upstream`，先添加：
```powershell
git remote add upstream https://github.com/diegosouzapw/OmniRoute.git
```
同步上游前，先抓取最新提交和标签：
```powershell
git fetch upstream --tags
```
查看当前版本和上游标签：
```powershell
git describe --tags --always
git show --no-patch --oneline v3.4.7
```
> 注 (note / nota)：当前项目版本为 `v3.8.0` (current project version is `v3.8.0` / a versão atual do projeto é `v3.8.0`)。下文中的 `v3.4.7` 仅为历史示例 (the `v3.4.7` references below are kept as historical examples only / as referências a `v3.4.7` abaixo são apenas exemplos históricos)；实际发布时请使用 `:latest` 或当前版本标签 (e.g. `:v3.8.0`) (use `:latest` or the current version tag — e.g. `:v3.8.0` — for actual releases / use `:latest` ou a tag da versão atual — p.ex. `:v3.8.0` — em releases reais)。
如果你想合并上游最新 `main`，并强制保留 fork 当前的 `fly.toml`，可按下面流程执行：
```powershell
git merge upstream/main
git checkout HEAD~1 -- fly.toml
git add -- fly.toml
git commit -m "chore(deploy): keep fork fly.toml"
git push origin main
```
说明：
- `git merge upstream/main` 用于同步原仓库最新代码
- `git checkout HEAD~1 -- fly.toml` 用于恢复合并前你 fork 自己的 `fly.toml`
- 如果上游没有改 `fly.toml`，这一步不会带来额外差异
- 如果上游改了 `fly.toml`，这一步能确保 Fly 应用名、挂载卷、区域等 fork 自定义部署配置不被覆盖
如果你明确只想对齐某个发布标签，例如 `v3.4.7`，也可以先确认标签是否已经包含在 `upstream/main`：
```powershell
git merge-base --is-ancestor v3.4.7 upstream/main
```
返回成功表示 `upstream/main` 已经包含该版本，直接合并 `upstream/main` 即可。
### 9.2 同步上游后的标准发布顺序
同步原仓库完成后，推荐按下面顺序发布：
1. `git fetch upstream --tags`
2. `git merge upstream/main`
3. 恢复 fork 的 `fly.toml`
4. `git push origin main`
5. `flyctl deploy`
6. `flyctl status -a omniroute`
7. `flyctl logs --no-tail -a omniroute`
这就是当前项目升级到 `v3.4.7` 时使用的实际流程 (示例为历史版本，当前实际版本是 `v3.8.0` / example refers to a historical version; the current actual version is `v3.8.0` / o exemplo refere-se a uma versão histórica; a versão atual é `v3.8.0`)。
---
## 10. 发布后检查
### 10.1 查看应用状态
```powershell
flyctl status -a omniroute
```
### 10.2 查看启动日志
```powershell
flyctl logs --no-tail -a omniroute
```
### 10.3 检查网站可访问
```powershell
try {
(Invoke-WebRequest -Uri "https://omniroute.fly.dev" -MaximumRedirection 5 -UseBasicParsing).StatusCode
} catch {
if ($_.Exception.Response) {
$_.Exception.Response.StatusCode.value__
} else {
throw
}
}
```
返回 `200` 说明站点已正常响应。
---
## 11. 成功标志
部署成功后，日志里应看到类似内容：
```text
[bootstrap] Secrets persisted to: /data/server.env
[DB] SQLite database ready: /data/storage.sqlite
```
这两个点很关键：
- `/data/server.env` 说明运行时密钥落到了持久卷
- `/data/storage.sqlite` 说明数据库写入持久卷
如果你看到的是 `/app/data/...`，说明 `DATA_DIR` 没配对，需要立即修正。
---
## 12. 常见问题
### 12.1 `Secrets` 页面是空的
通常有两种原因：
- 你还没执行 `flyctl secrets set`
- 你打开的是另一个应用，例如 `oroute`，不是 `omniroute`
### 12.2 `flyctl deploy` 报 `app not found`
先创建应用：
```powershell
flyctl apps create omniroute
```
### 12.3 `fly.toml` 解析失败
重点检查：
- 注释里是否有乱码字符
- TOML 引号和缩进是否正确
### 12.4 数据没有持久化
检查以下两点：
- `fly.toml` 中是否存在 `destination = '/data'`
- `DATA_DIR` 是否设置为 `/data`
### 12.5 不设置 `INITIAL_PASSWORD` 是否能跑
可以运行，但会回退到默认 `CHANGEME`。生产环境建议尽快修改后台密码。
---
## 13. 新项目复用建议
如果以后是新项目照着这份文档部署，最少改这几项：
1. 修改 `fly.toml` 里的 `app`
2. 修改 `NEXT_PUBLIC_BASE_URL`
3. 保持 `DATA_DIR=/data`
4. 重新生成 `API_KEY_SECRET`、`JWT_SECRET`、`MACHINE_ID_SALT`、`STORAGE_ENCRYPTION_KEY`
5. 首次部署后检查日志是否写入 `/data`
不要直接复用旧项目的密钥。
---
## 14. 当前项目的最小发布清单
当前项目后续最常用的命令如下：
```powershell
flyctl auth whoami
flyctl status -a omniroute
flyctl secrets list -a omniroute
flyctl deploy
flyctl logs --no-tail -a omniroute
```
如果只是正常发版，核心就是：
```powershell
flyctl deploy
```
如果是新环境首次部署，核心就是：
1. `flyctl auth login`
2. `flyctl apps create omniroute`
3. `flyctl secrets set ... -a omniroute`
4. `flyctl deploy`
5. `flyctl logs --no-tail -a omniroute`

234
Free-Tiers.md Normal file

@@ -0,0 +1,234 @@
# Free Tiers
> **Last consolidated:** 2026-05-13 — OmniRoute v3.8.2
> **Source of truth:** `src/shared/constants/providers.ts` (`FREE_PROVIDERS`, `OAUTH_PROVIDERS`, and `APIKEY_PROVIDERS` entries flagged with `hasFree: true` + `freeNote`)
This page lists providers with usable free tiers shipped in OmniRoute v3.8.2. The data is derived from the provider catalog. If a provider does not appear here, it either has no free tier in the catalog or its `hasFree` flag is `false`.
Add credentials from the dashboard (`/dashboard/providers/new`) — OmniRoute reads keys from the database, not from per-provider environment variables. The only env vars that influence provider behavior are listed in the [Environment Variables](#environment-variables) section.
---
## How free providers are wired
OmniRoute classifies providers into the following groups in `src/shared/constants/providers.ts`:
| Group | Auth | Example IDs |
| ------------------ | ------------------------------ | ------------------------------------------- |
| `FREE_PROVIDERS` | OAuth or vendor account | `qoder`, `gemini-cli`, `kiro`, `amazon-q` |
| `OAUTH_PROVIDERS` | OAuth | `claude`, `cursor`, `windsurf`, `devin-cli` |
| `APIKEY_PROVIDERS` | API key (with `hasFree: true`) | `groq`, `cerebras`, `mistral`, `gemini` |
A provider appears in the **free pool** when either:
- It is listed in the `FREE_PROVIDERS` map (and not flagged `deprecated: true`), or
- It is listed in `APIKEY_PROVIDERS` with `hasFree: true` and a `freeNote` string, or
- It is an OAuth provider whose vendor offers a free tier on top of OAuth sign-in.
---
## Quick reference (API key providers with `hasFree: true`)
| Provider | ID | Free tier note |
| ------------------ | --------------- | ---------------------------------------------------------------------------------------------------- |
| AgentRouter | `agentrouter` | $200 free credits on signup — multi-model routing gateway |
| AI21 Labs | `ai21` | $10 trial credits on signup (valid 3 months), no credit card required |
| AI/ML API | `aimlapi` | $0.025/day free credits — 200+ models via single endpoint |
| BazaarLink | `bazaarlink` | Free tier with `auto:free` routing — zero-cost inference, no credit card required |
| Baseten | `baseten` | $30 free trial credits for GPU inference |
| Blackbox AI | `blackbox` | Free tier: unlimited basic chat plus Minimax-M2.5, no credit card required |
| Bytez | `bytez` | $1 free credits, refreshes every 4 weeks |
| Cerebras | `cerebras` | Free: 1M tokens/day, 60K TPM — world's fastest inference |
| Cloudflare AI | `cloudflare-ai` | Free 10K Neurons/day: ~150 LLM responses or 500s Whisper audio |
| Cohere | `cohere` | Free Trial: 1,000 API calls/month for testing, no credit card required |
| Completions.me | `completions` | Free unlimited access to Claude, GPT, Gemini — no credit card, no rate limits |
| DeepInfra | `deepinfra` | Free signup credits for API testing and model exploration |
| DeepSeek | `deepseek` | 5M free tokens on signup — no credit card required |
| Enally AI | `enally` | Free for students and developers — no credit card, OTP verification |
| Fireworks AI | `fireworks` | $1 free starter credits on signup for API testing |
| FreeTheAi | `freetheai` | Community-run — free forever, no paid tiers, no credit card |
| Gemini (AI Studio) | `gemini` | Free forever: 1,500 req/day for Gemini 2.5 Flash — no credit card |
| GLHF Chat | `glhf` | Free tier for open-source model inference |
| Groq | `groq` | Free tier: 30 RPM / 14.4K RPD — no credit card |
| HuggingFace | `huggingface` | Free Inference API for thousands of models (Whisper, VITS, SDXL…) |
| Hyperbolic | `hyperbolic` | $1–5 trial credits on signup for serverless inference |
| Inference.net | `inference-net` | $25 free credits on signup plus research grants available |
| Jina AI | `jina-ai` | 10M free tokens on signup (non-commercial), no credit card required |
| Kluster AI | `kluster` | $5 free credits on signup — DeepSeek R1, Llama 4 Maverick/Scout, Qwen3 235B |
| Lepton AI | `lepton` | Free tier available — fast inference on custom hardware |
| LLM7.io | `llm7` | No signup required — 2 req/s, 20 RPM, 100 req/hr free tier |
| LongCat AI | `longcat` | 50M tokens/day (Flash-Lite) + 500K/day (Chat/Thinking) — 100% free while public beta |
| Mistral | `mistral` | Free Experiment tier: rate-limited access to all models, no credit card required |
| Modal | `modal` | $30/month free credits for new accounts |
| Morph | `morph` | Free tier: 250K credits/month, $0 |
| Nebius | `nebius` | ~$1 trial credits on signup for API testing |
| NLP Cloud | `nlpcloud` | Trial credits for new accounts |
| Nous Research | `nous-research` | Free tier: 50 RPM, 500,000 TPM — no credit card |
| Novita AI | `novita` | $0.50 trial credits on signup (valid about 1 year) |
| nScale | `nscale` | $5 free credits on signup for inference testing |
| NVIDIA NIM | `nvidia` | Free dev access: ~40 RPM, 70+ models (Kimi K2.5, GLM 4.7, DeepSeek V3.2…) |
| OpenRouter | `openrouter` | Free models at $0/token with `:free` suffix — 20 RPM / 200 RPD |
| Pollinations AI | `pollinations` | No API key required for free public endpoint. Optional Spore tier: ~0.01 pollen/hour |
| Predibase | `predibase` | $25 free trial credits (30-day validity) |
| PublicAI | `publicai` | Free community inference tier for testing |
| Puter AI | `puter` | 500+ models (GPT-5, Claude Opus 4, Gemini 3 Pro, Grok 4, DeepSeek V3…) — users pay via Puter account |
| Reka | `reka` | $10/month recurring free API credits |
| SambaNova | `sambanova` | $5 free credits on signup (30-day validity), no credit card required |
| Scaleway AI | `scaleway` | 1M free tokens for new accounts — EU/GDPR compliant (Paris), Qwen3 235B & Llama 70B |
| SiliconFlow | `siliconflow` | $1 free credits plus permanently free models after identity verification |
| Together AI | `together` | $25 signup credits + 3 permanently free models: Llama 3.3 70B, Vision, DeepSeek-R1 distill |
| UncloseAI | `uncloseai` | Free forever — no signup, no credit card. OpenAI-compatible endpoints |
| Voyage AI | `voyage-ai` | 200M free tokens for embeddings and reranking |
**Total: 48 API-key providers with `hasFree: true`.**
> All entries above are copied verbatim from the `freeNote` field in the provider catalog so they stay in sync with the code.
---
## OAuth-based free tiers
### Always-free OAuth providers (in `FREE_PROVIDERS`)
These providers are designed around a vendor OAuth flow and ship a free tier by default:
| Provider | ID | Notes |
| ---------- | ------------ | ----------------------------------------------------------------------------------------------------------- |
| Qoder AI | `qoder` | OAuth or Personal Access Token. Free tier on signup. |
| Gemini CLI | `gemini-cli` | Uses Gemini CLI OAuth / Cloud Code credentials. Pro models require an eligible Google account or paid plan. |
| Kiro AI | `kiro` | AWS Builder ID (Kiro Free tier). |
| Amazon Q | `amazon-q` | Same AWS Builder ID / refresh-token flow as Kiro, but kept as separate connections. |
### OAuth providers with vendor-controlled free tiers (in `OAUTH_PROVIDERS`)
The free-tier surface here depends entirely on each vendor's account plan, not on OmniRoute:
| Provider | ID | Auth hint |
| -------------------- | ------------- | --------------------------------------------------------------------------------------- |
| Claude Code | `claude` | OAuth via `platform.claude.com`. Free quota depends on your Anthropic account. |
| Antigravity | `antigravity` | Google OAuth (Antigravity). |
| OpenAI Codex | `codex` | OAuth via OpenAI (Codex CLI). Subject to ChatGPT plan free credits. |
| GitHub Copilot | `github` | OAuth via GitHub. Free for verified students; free trial otherwise. |
| GitLab Duo | `gitlab-duo` | OAuth (`ai_features + read_user`). Requires GitLab Duo entitlement. |
| Cursor IDE | `cursor` | Cursor OAuth. Free tier limits depend on Cursor plan. |
| Kimi Coding | `kimi-coding` | Moonshot OAuth. Free quota on Kimi Coding accounts. |
| Kilo Code | `kilocode` | Kilo OAuth — free auto-router available. |
| Cline | `cline` | Cline OAuth. |
| Windsurf (Devin CLI) | `windsurf` | Sign in at `windsurf.com`, paste your token. Free tier limits set by Windsurf. |
| Devin CLI (Official) | `devin-cli` | Uses the official Devin CLI binary or `WINDSURF_API_KEY`. Subject to Devin's free tier. |
---
## Deprecated / discontinued
### Qwen Code (`qwen`)
Marked `deprecated: true` in `FREE_PROVIDERS`. Discontinued **2026-04-15**.
> Qwen OAuth free tier was discontinued on 2026-04-15. Use `bailian-coding-plan`, `alibaba`, `alibaba-cn`, or `openrouter` providers with an API key instead.
Connections of type `qwen` will keep working until their tokens expire, but no new OAuth sign-ins are accepted upstream. Migrate to:
- `bailian-coding-plan` (Alibaba Coding Plan — Claude-compatible)
- `alibaba` (Alibaba — DashScope international)
- `alibaba-cn` (Alibaba (China) — DashScope China)
- `openrouter` (Qwen models exposed via OpenRouter)
---
## Command Code
`command-code` is a separate API-key provider for the Command Code agent (see `commandcode.ai`). It is not flagged with `hasFree: true` in the catalog, so it does not appear in the free table above, but it is included here because it ships in v3.8.0 alongside the free-tier providers:
- ID: `command-code`
- Endpoint: Command Code `/alpha/generate`
- Auth: Bearer API key, configured from the dashboard.
Check Command Code's website for the current free-tier policy.
---
## Environment variables
OmniRoute v3.8.2 does **not** read provider API keys from environment variables (with one exception below). Keys are stored in the encrypted SQLite database and configured from the dashboard. The env vars listed here are the only ones that affect free-tier behavior:
```bash
# Windsurf / Devin CLI — Firebase Web API key used by the Secure Token
# Service to refresh the Windsurf app. The default value ships in
# .env.example (it is a public Firebase Web API key extracted from the
# Devin CLI binary, not a real secret); override only if you mirror your
# own Windsurf token-refresh service.
WINDSURF_FIREBASE_API_KEY=<see .env.example>
# Optional fallback for the devin-cli executor when no connection key is set.
WINDSURF_API_KEY=
# Optional path to the official Devin CLI binary.
CLI_DEVIN_BIN=/usr/local/bin/devin
# OAuth client overrides (rarely needed — defaults shipped in code)
CODEX_OAUTH_CLIENT_ID=
GEMINI_OAUTH_CLIENT_ID=
GEMINI_OAUTH_CLIENT_SECRET=
GEMINI_CLI_OAUTH_CLIENT_ID=
GEMINI_CLI_OAUTH_CLIENT_SECRET=
QWEN_OAUTH_CLIENT_ID=
KIMI_CODING_OAUTH_CLIENT_ID=
GITHUB_OAUTH_CLIENT_ID=
GITLAB_DUO_OAUTH_CLIENT_ID=
GITLAB_DUO_OAUTH_CLIENT_SECRET=
QODER_OAUTH_CLIENT_SECRET=
QODER_PERSONAL_ACCESS_TOKEN=
# CLI sidecar binaries
CLI_CODEX_BIN=codex
CLI_CURSOR_BIN=agent
CLI_CLINE_BIN=cline
CLI_QODER_BIN=qoder
CLI_QWEN_BIN=qwen
```
For all other providers (Groq, Cerebras, Mistral, Gemini, Cohere, NVIDIA, OpenRouter, Together, Fireworks, Cloudflare AI, SambaNova, HuggingFace, SiliconFlow, Hyperbolic, Morph, LLM7, Lepton, Kluster, UncloseAI, BazaarLink, Completions, Enally, FreeTheAi, AgentRouter, Command Code, etc.), add the key from `/dashboard/providers/new`.
---
## How to use
1. Open `/dashboard/providers/new` and pick the provider you want.
2. Paste the API key (or complete the OAuth flow). For OAuth providers, follow the dashboard wizard.
3. The provider appears in your routing pool automatically and is eligible for combos and auto-routing.
4. Track usage at `/dashboard/usage` to see how close you are to free-tier limits.
### Suggested combos
| Goal | Strategy | Notes |
| ---------------------------- | -------------------- | --------------------------------------------------------------------- |
| Cheapest possible chat | `auto/cheap` | Prefers free / lowest-cost providers; falls back automatically. |
| Local-only routing | `auto/offline` | Routes only to local providers (Ollama, LM Studio, vLLM, …). |
| Redundancy across free tiers | combo `priority` | List Groq → Cerebras → Mistral → Gemini → NVIDIA → OpenRouter. |
| High RPM throughput | combo `round-robin` | Spreads requests across all configured free providers. |
| Best success rate | combo `lkgp` / `p2c` | Picks last-known-good provider or "power of two choices" rebalancing. |
### Tips
- Combine multiple free providers in a combo (`/dashboard/combos`) to maximize daily quota and route around outages.
- Use `omniroute doctor` to verify all configured free providers are reachable.
- Check provider health in `/dashboard/monitoring/health` — a provider with an open circuit breaker is skipped automatically.
- Free-tier limits change frequently; the `freeNote` strings reflect the limits as known at v3.8.0 ship date. Verify with each provider's official docs before relying on a specific number.
---
## Glossary
| Term | Meaning |
| ---------- | -------------------------------------------------------- |
| **RPM** | Requests per minute |
| **RPD** | Requests per day |
| **RPH** | Requests per hour |
| **RPS** | Requests per second |
| **TPM** | Tokens per minute |
| **TPD** | Tokens per day |
| **Neuron** | Cloudflare's compute unit (~1 output token) |
| **LKGP** | Last-known-good provider — auto-combo strategy |
| **P2C** | Power-of-two choices — auto-combo load balancer strategy |

1077
Gamification.md Normal file

File diff suppressed because it is too large Load Diff

264
Guardrails.md Normal file

@@ -0,0 +1,264 @@
# Guardrails
> **Source of truth:** `src/lib/guardrails/`
> **Last updated:** 2026-05-13 — v3.8.0
Guardrails enforce safety, policy, and content transformations at the boundary
between OmniRoute and upstream providers. Each guardrail can inspect (and
optionally reject, transform, or annotate) request payloads (`preCall`) and
upstream responses (`postCall`).
The system is **fail-open**: if a guardrail throws while executing, the registry
records the error and continues with the next guardrail rather than failing the
request. Blocking is an explicit decision (`block: true`), never an accident.
## Built-in Guardrails
The registry auto-loads three guardrails in priority order on import
(see `registry.ts` → `registerDefaultGuardrails()`):
| Priority | Name | Stage(s) | File |
| -------- | ------------------ | -------------- | -------------------- |
| `5` | `vision-bridge` | `preCall` | `visionBridge.ts` |
| `10` | `pii-masker` | `pre` + `post` | `piiMasker.ts` |
| `20` | `prompt-injection` | `preCall` | `promptInjection.ts` |
Lower priority numbers run **first**.
### Vision Bridge (`visionBridge.ts`)
Intercepts image-bearing requests aimed at **non-vision models** and replaces
the image parts with text descriptions produced by a configurable vision model
before the upstream call. This lets text-only providers transparently handle
multimodal payloads.
Flow:
1. Skip if the target model already supports vision (unless it appears in the
forced-bridge list `isVisionBridgeForcedModel`).
2. Extract image parts via `extractImageParts(messages)`. Skip if none.
3. Load runtime config from `getSettings()` (`visionBridgeEnabled`,
`visionBridgeModel`, `visionBridgePrompt`, `visionBridgeTimeout`,
`visionBridgeMaxImages`).
4. Cap images at `maxImages`, call the vision model **in parallel**
(`Promise.allSettled`), and inject `[Image N]: <description>` text parts
in their place — failed images become `[Image N]: (unavailable)`.
5. Return `modifiedPayload` + meta (`imagesProcessed`, `processingTimeMs`,
`visionModel`).
Defaults live in `src/shared/constants/visionBridgeDefaults.ts`. The guardrail
exposes a `deps` constructor option so tests can inject fake `getSettings` and
`callVisionModel` implementations.
### PII Masker (`piiMasker.ts`)
Runs on **both** stages.
- **`preCall`** clones the payload, walks `system`, `messages`, and `input`
arrays, and applies `processPII()` (from `@/shared/utils/inputSanitizer`) to
string `content`/`text` fields. When `PII_REDACTION_ENABLED=true` **and**
`INPUT_SANITIZER_MODE=redact`, detected PII is stripped/redacted in the
outbound payload. Otherwise the call records detection counts without
rewriting content.
- **`postCall`** deep-clones the response, runs `sanitizePIIResponse()` plus
the Responses-API-shape masker (`maskResponsesOutput` — covers
`output_text` and `output[].content[].text`). If any redaction occurs, the
modified response replaces the original.
The guardrail never blocks; it only annotates (`meta.detections`,
`meta.redacted`) or rewrites.
### Prompt Injection (`promptInjection.ts`)
Detects adversarial structures in user-supplied content and enforces the
configured policy. Behavior is driven by environment variables and constructor
options:
| Setting | Env var | Default | Effect |
| --------------- | ----------------------------------------------- | ------- | --------------------------------------- |
| Enabled | `INPUT_SANITIZER_ENABLED` | `true` | When `false`, guardrail short-circuits. |
| Mode | `INJECTION_GUARD_MODE` / `INPUT_SANITIZER_MODE` | `warn` | `block`, `warn`, or `log`. |
| Block threshold | `blockThreshold` option | `high` | Minimum severity required to block. |
Detection sources:
1. `sanitizeRequest()` from `@/shared/utils/inputSanitizer` (shared detector
set used elsewhere in the pipeline).
2. Built-in `DEFAULT_GUARD_PATTERNS` (currently `system_override_inline` and
`markdown_system_block`, both `high` severity).
3. Optional `customPatterns` passed via constructor options (strings, regex,
or `{ name, pattern, severity }` records).
When `mode === "block"` **and** at least one detection meets the severity
threshold, `preCall` returns `{ block: true, message: "Request rejected:
suspicious content detected" }`. In `warn`/`log` modes the guardrail logs but
allows the call. The shared helper `evaluatePromptInjection()` is also exported
for callers that need to evaluate prompts without going through the registry.
## Base Contract (`base.ts`)
```typescript
class BaseGuardrail {
enabled: boolean;
name: string;
priority: number;
constructor(name: string, options?: { enabled?: boolean; priority?: number });
async preCall(payload: unknown, context: GuardrailContext): Promise<GuardrailResult | void>;
async postCall(response: unknown, context: GuardrailContext): Promise<GuardrailResult | void>;
}
interface GuardrailResult<TValue = unknown> {
block?: boolean; // true short-circuits the chain
message?: string; // surfaced when blocking
meta?: Record<string, unknown> | null;
modifiedPayload?: TValue; // returned by preCall to rewrite the request
modifiedResponse?: TValue; // returned by postCall to rewrite the response
}
interface GuardrailContext {
apiKeyInfo?: Record<string, unknown> | null;
disabledGuardrails?: string[] | null;
endpoint?: string | null;
headers?: Headers | Record<string, unknown> | null;
log?: GuardrailLog | Console | null;
method?: string | null;
model?: string | null;
provider?: string | null;
sourceFormat?: string | null;
stream?: boolean;
targetFormat?: string | null;
}
```
A guardrail signals "no change" by returning either `void`, `{}`, or
`{ block: false }`. Returning a `modifiedPayload`/`modifiedResponse` replaces
the value flowing through the chain for downstream guardrails.
## Registry (`registry.ts`)
The singleton `guardrailRegistry` exposes:
- `register(guardrail)` — adds (or replaces by normalized name) a guardrail and
re-sorts by ascending `priority`.
- `clear()` / `list()` — administrative helpers.
- `runPreCallHooks(payload, context)` — iterates active guardrails, threads the
payload through `modifiedPayload`, and stops on the first `block: true`.
- `runPostCallHooks(response, context)` — same flow on the response side.
- `resetGuardrailsForTests({ registerDefaults })` — clears state and optionally
re-registers the defaults for clean test isolation.
Both runners return `{ blocked, payload|response, results, guardrail?, message? }`
where `results` is an array of `GuardrailExecutionResult` records that include
per-guardrail `blocked`, `skipped`, `modified`, `error`, and `meta` fields,
useful for tracing.
### Disabling Guardrails Per-Request
`resolveDisabledGuardrails({ apiKeyInfo, body, headers })` aggregates a
de-duplicated list of guardrail names that should be skipped for the current
request. Sources (all optional, all merged):
- `apiKeyInfo.disabledGuardrails`
- Request body `disabledGuardrails` (top-level)
- Request body `metadata.disabledGuardrails`
- Header `x-omniroute-disabled-guardrails` (or legacy
`x-disabled-guardrails`)
Values may be arrays of strings or a comma-separated string; names are
normalized to lowercase kebab-case (`pii_masker` → `pii-masker`). The result
is passed through `context.disabledGuardrails` to the registry, which skips
matching guardrails (`skipped: true` in `results`).
## Execution Order
For each request flowing through `src/sse/handlers/chat.ts` and
`open-sse/handlers/chatCore.ts`:
1. `resolveDisabledGuardrails(...)` builds the skip list from API key, body,
and headers.
2. `guardrailRegistry.runPreCallHooks(body, ctx)` runs guardrails in ascending
priority order:
- Disabled guardrails are recorded as `skipped`.
- Each guardrail's `preCall` may rewrite the payload via `modifiedPayload`.
- The first `block: true` short-circuits the chain and the handler returns
a guardrail rejection response.
3. The (potentially rewritten) payload flows into combo routing and upstream
dispatch.
4. After the response is assembled, `guardrailRegistry.runPostCallHooks(...)`
runs the same chain on the response. `block: true` here drops the upstream
response.
Guardrails that throw are recorded with `error: <message>` and logged via
`logger.warn`, but the chain continues — fail-open by design.
## Configuration
Environment variables read by the built-in guardrails:
| Variable | Used by | Effect |
| ------------------------------------- | -------------------------------- | ----------------------------------------------------- |
| `INPUT_SANITIZER_ENABLED` | `prompt-injection` | Set `false` to disable detection entirely. |
| `INPUT_SANITIZER_MODE` | `prompt-injection`, `pii-masker` | Shared mode: `warn`, `block`, `log`, or `redact`. |
| `INJECTION_GUARD_MODE` | `prompt-injection` | Legacy alias for `INPUT_SANITIZER_MODE`. |
| `PII_REDACTION_ENABLED` | `pii-masker` | When `true` + mode `redact`, request PII is stripped. |
| `PII_RESPONSE_SANITIZATION` / `_MODE` | `pii-masker` (downstream) | Controls response-side masker behavior. |
The Vision Bridge reads runtime config from the DB-backed settings store
(`getSettings()`), not env vars: `visionBridgeEnabled`, `visionBridgeModel`,
`visionBridgePrompt`, `visionBridgeTimeout`, `visionBridgeMaxImages`. Defaults
live in `src/shared/constants/visionBridgeDefaults.ts`.
## Custom Guardrails
```typescript
import { BaseGuardrail, guardrailRegistry } from "@/lib/guardrails";
class BudgetGuardrail extends BaseGuardrail {
constructor() {
super("budget", { priority: 50 });
}
async preCall(payload, ctx) {
if (ctx.apiKeyInfo?.budgetExceeded) {
return { block: true, message: "Daily budget exceeded" };
}
return { block: false };
}
}
guardrailRegistry.register(new BudgetGuardrail());
```
Steps:
1. Create `src/lib/guardrails/myGuardrail.ts` extending `BaseGuardrail`.
2. Implement `preCall` and/or `postCall`.
3. Either register at import time (push from `registerDefaultGuardrails`) or
call `guardrailRegistry.register(...)` at runtime — the registry replaces
any prior guardrail with the same normalized name.
4. Add tests under `tests/unit/` (existing examples:
`tests/unit/guardrails-registry.test.ts`,
`tests/unit/prompt-injection-guard.test.ts`,
`tests/unit/guardrails/visionBridge.test.ts`).
## Testing
Use `resetGuardrailsForTests()` between tests to start from a known state.
Pass `{ registerDefaults: false }` to start with an empty registry and
register only the guardrails under test. The Vision Bridge guardrail accepts
dependency injection (`deps.getSettings`, `deps.callVisionModel`) so tests can
exercise the full flow without DB or network access.
## See Also
- `src/lib/guardrails/` — implementation
- `src/shared/utils/inputSanitizer.ts` — shared detector that powers
prompt-injection and PII masking
- `src/shared/constants/visionBridgeDefaults.ts` — Vision Bridge defaults and
forced-bridge model list
- `docs/architecture/RESILIENCE_GUIDE.md` — orthogonal layer (circuit breaker, cooldowns)
- `docs/reference/ENVIRONMENT.md` — full env var reference

66
Home.md

@@ -1,2 +1,64 @@
Welcome to the OmniRoute wiki!
Coming Soon...
# 🚀 OmniRoute — The Free AI Gateway
**Never stop coding. Connect every AI tool to 212+ providers — 50+ free — through one endpoint.**
Plug Claude Code, Codex, Cursor, Cline, Copilot & Antigravity into FREE Claude / GPT / Gemini. Auto-fallback.
**RTK + Caveman compression saves 15–95% tokens. Never hit limits.**
---
## ✨ Highlights
| Feature | Description |
|---------|-------------|
| **212+ AI Providers** | OpenAI, Anthropic, Gemini, DeepSeek, Groq, xAI, Mistral, and many more |
| **50+ Free Tiers** | OAuth + free-tier providers for zero-cost AI access |
| **14 Routing Strategies** | Priority, weighted, round-robin, cost-optimized, context-relay, and more |
| **Token Compression** | RTK + Caveman engines save 15–95% tokens automatically |
| **MCP Server** | 37 tools with stdio/SSE/Streamable HTTP transports |
| **A2A Protocol** | JSON-RPC 2.0 agent-to-agent communication |
| **Electron App** | Cross-platform desktop app (Windows, macOS, Linux) |
| **40+ Languages** | Full i18n support across the dashboard |
## 🚀 Quick Start
```bash
# Install via npm
npx omniroute@latest
# Or via Docker
docker run -p 20128:20128 diegosouzapw/omniroute
```
Then open [http://localhost:20128](http://localhost:20128).
## 📚 Documentation
Use the **sidebar** to navigate through all documentation sections:
- **[Getting Started](Getting-Started)** — Installation, setup, first steps
- **[User Guide](User-Guide)** — Complete usage guide
- **[Features](Features)** — All features explained
- **[Architecture](Architecture)** — System design and internals
- **[API Reference](API-Reference)** — REST API documentation
- **[Providers](Provider-Reference)** — All 212+ supported providers
- **[Combos & Routing](Auto-Combo)** — Routing strategies
- **[Compression](Compression-Guide)** — Token compression pipeline
- **[MCP Server](MCP-Server)** — MCP tools and transports
- **[A2A Server](A2A-Server)** — Agent-to-agent protocol
- **[Security](Security)** — Auth, guardrails, compliance
- **[Deployment](VM-Deployment-Guide)** — Deploy to VPS/VM/Fly.io
- **[CLI & Tools](CLI-Tools)** — Command-line reference
- **[Environment](Environment)** — All environment variables
- **[Troubleshooting](Troubleshooting)** — Common issues & fixes
- **[Contributing](Contributing)** — How to contribute
## 🔗 Links
- [GitHub Repository](https://github.com/diegosouzapw/OmniRoute)
- [npm Package](https://www.npmjs.com/package/omniroute)
- [Docker Hub](https://hub.docker.com/r/diegosouzapw/omniroute)
- [Website](https://omniroute.online)
- [WhatsApp Community 🌍](https://chat.whatsapp.com/JI7cDQ1GyaiDHhVBpLxf8b?mode=gi_t)
- [WhatsApp Community 🇧🇷](https://chat.whatsapp.com/CeGCxdFzqBe5Uki288wOvf)

573
I18N.md Normal file

@@ -0,0 +1,573 @@
# i18n — Internationalization Guide
OmniRoute supports **30 languages** with full dashboard UI translation, translated documentation, and RTL support for Arabic and Hebrew.
🌐 **Languages:** 🇺🇸 [English](./I18N.md) | 🇧🇷 [Português (Brasil)](../i18n/pt-BR/docs/guides/I18N.md) | 🇪🇸 [Español](../i18n/es/docs/guides/I18N.md) | 🇫🇷 [Français](../i18n/fr/docs/guides/I18N.md) | 🇩🇪 [Deutsch](../i18n/de/docs/guides/I18N.md) | 🇮🇹 [Italiano](../i18n/it/docs/guides/I18N.md) | 🇷🇺 [Русский](../i18n/ru/docs/guides/I18N.md) | 🇨🇳 [中文 (简体)](../i18n/zh-CN/docs/guides/I18N.md) | 🇯🇵 [日本語](../i18n/ja/docs/guides/I18N.md) | 🇰🇷 [한국어](../i18n/ko/docs/guides/I18N.md) | 🇸🇦 [العربية](../i18n/ar/docs/guides/I18N.md) | 🇮🇳 [हिन्दी](../i18n/hi/docs/guides/I18N.md) | 🇹🇭 [ไทย](../i18n/th/docs/guides/I18N.md) | 🇹🇷 [Türkçe](../i18n/tr/docs/guides/I18N.md) | 🇺🇦 [Українська](../i18n/uk-UA/docs/guides/I18N.md) | 🇻🇳 [Tiếng Việt](../i18n/vi/docs/guides/I18N.md) | 🇧🇬 [Български](../i18n/bg/docs/guides/I18N.md) | 🇩🇰 [Dansk](../i18n/da/docs/guides/I18N.md) | 🇫🇮 [Suomi](../i18n/fi/docs/guides/I18N.md) | 🇮🇱 [עברית](../i18n/he/docs/guides/I18N.md) | 🇭🇺 [Magyar](../i18n/hu/docs/guides/I18N.md) | 🇮🇩 [Bahasa Indonesia](../i18n/id/docs/guides/I18N.md) | 🇲🇾 [Bahasa Melayu](../i18n/ms/docs/guides/I18N.md) | 🇳🇱 [Nederlands](../i18n/nl/docs/guides/I18N.md) | 🇳🇴 [Norsk](../i18n/no/docs/guides/I18N.md) | 🇵🇹 [Português (Portugal)](../i18n/pt/docs/guides/I18N.md) | 🇷🇴 [Română](../i18n/ro/docs/guides/I18N.md) | 🇵🇱 [Polski](../i18n/pl/docs/guides/I18N.md) | 🇸🇰 [Slovenčina](../i18n/sk/docs/guides/I18N.md) | 🇸🇪 [Svenska](../i18n/sv/docs/guides/I18N.md) | 🇵🇭 [Filipino](../i18n/phi/docs/guides/I18N.md) | 🇨🇿 [Čeština](../i18n/cs/docs/guides/I18N.md)
## Translation pipeline (recommended — v3.8.0)
OmniRoute uses a hash-based incremental translator for docs, backed by an
OpenAI-compatible LLM endpoint (typically `cx/gpt-5.4-mini` through OmniRoute
Cloud):
```bash
# Run translations (incremental — only touches changed sources)
npm run i18n:run
# Limit to one locale
npm run i18n:run -- --locale=pt-BR
# Specific files (comma-separated, repo-relative paths)
npm run i18n:run -- --files=CLAUDE.md,docs/architecture/ARCHITECTURE.md
# Force retranslate everything (expensive)
npm run i18n:run -- --force
# Preview what would happen (no API calls, no writes)
npm run i18n:run:dry
# CI gate — exits non-zero if state is drifting
npm run i18n:check
```
**Source of truth.** `config/i18n.json` lists every locale (UI + docs) plus
the RTL set and the `docsExcluded` codes. The runtime config in
`src/i18n/config.ts` is a thin adapter over that JSON.
**Backend.** Configured via env (set in `.env`, never committed):
| Variable | Purpose |
| ----------------------------------- | --------------------------------------- |
| `OMNIROUTE_TRANSLATION_API_URL` | OpenAI-compatible base URL, e.g. `…/v1` |
| `OMNIROUTE_TRANSLATION_API_KEY` | bearer token (kept out of logs) |
| `OMNIROUTE_TRANSLATION_MODEL` | model id, e.g. `cx/gpt-5.4-mini` |
| `OMNIROUTE_TRANSLATION_TIMEOUT_MS` | optional, default `60000` |
| `OMNIROUTE_TRANSLATION_CONCURRENCY` | optional, default `4` |
**State tracking.** `.i18n-state.json` (committed) keeps SHA-256 hashes per
source + per locale. Drift detection is automatic and deterministic — no API
calls in `i18n:check`.
**Output shape.** Each translated file gets a top-level `# <heading>
(<native>)` line, a `🌐 Languages: …` bar, an `---` separator, and the
translated body. That layout matches what `scripts/check/check-docs-sync.mjs`
already enforces for `llm.txt` and `CHANGELOG.md` mirrors.
### Legacy scripts (deprecated)
The older Python script (`scripts/i18n/i18n_autotranslate.py`) and the
Google-Translate-backed generator (`scripts/i18n/generate-multilang.mjs`)
still exist with a deprecation banner. They will be removed in v3.10. The
`messages` and `readme` modes of `generate-multilang.mjs` (UI strings + root
README variants) are not yet handled by the new pipeline and are still used.
## Quick Reference
| Task | Command |
| ----------------------- | ---------------------------------------------------------- |
| Translate docs (LLM) | `npm run i18n:run` (preferred — incremental, hash-based) |
| Translate UI strings | `node scripts/i18n/generate-multilang.mjs messages` |
| Check translation drift | `npm run i18n:check` |
| Validate a locale | `python3 scripts/i18n/validate_translation.py quick -l cs` |
| Check code keys | `python3 scripts/i18n/check_translations.py` |
| Generate QA report | `node scripts/i18n/generate-qa-checklist.mjs` |
| Visual QA (Playwright) | `node scripts/i18n/run-visual-qa.mjs` |
## Architecture
![Incremental hash-based i18n pipeline](../diagrams/exported/i18n-flow.svg)
> Source: [diagrams/i18n-flow.mmd](../diagrams/i18n-flow.mmd)
### Source of Truth
- **UI strings**: `src/i18n/messages/en.json` (English source, ~2800 keys)
- **Locale files**: `src/i18n/messages/{locale}.json` (30 translations)
- **Framework**: `next-intl` with cookie-based locale resolution
- **Config**: `src/i18n/config.ts` — defines all 30 locales, language names, flags
### Runtime Flow
1. User selects language → `NEXT_LOCALE` cookie set
2. `src/i18n/request.ts` resolves locale: cookie → `Accept-Language` header → fallback `en`
3. Dynamic import loads `messages/{locale}.json`
4. Components use `useTranslations("namespace")` and `t("key")`
### Supported Locales
| Code | Language | RTL | Google Translate Code |
| ------- | -------------------- | --- | --------------------- |
| `ar` | العربية | Yes | `ar` |
| `bg` | Български | No | `bg` |
| `cs` | Čeština | No | `cs` |
| `da` | Dansk | No | `da` |
| `de` | Deutsch | No | `de` |
| `es` | Español | No | `es` |
| `fi` | Suomi | No | `fi` |
| `fr` | Français | No | `fr` |
| `he` | עברית | Yes | `iw` |
| `hi` | हिन्दी | No | `hi` |
| `hu` | Magyar | No | `hu` |
| `id` | Bahasa Indonesia | No | `id` |
| `it` | Italiano | No | `it` |
| `ja` | 日本語 | No | `ja` |
| `ko` | 한국어 | No | `ko` |
| `ms` | Bahasa Melayu | No | `ms` |
| `nl` | Nederlands | No | `nl` |
| `no` | Norsk | No | `no` |
| `phi` | Filipino | No | `tl` |
| `pl` | Polski | No | `pl` |
| `pt` | Português (Portugal) | No | `pt` |
| `pt-BR` | Português (Brasil) | No | `pt` |
| `ro` | Română | No | `ro` |
| `ru` | Русский | No | `ru` |
| `sk` | Slovenčina | No | `sk` |
| `sv` | Svenska | No | `sv` |
| `th` | ไทย | No | `th` |
| `tr` | Türkçe | No | `tr` |
| `uk-UA` | Українська | No | `uk` |
| `vi` | Tiếng Việt | No | `vi` |
| `zh-CN` | 中文 (简体) | No | `zh-CN` |
## Adding a New Language
### 1. Register the Locale
Edit `src/i18n/config.ts`:
```ts
// Add to LOCALES array
"xx",
// Add to LANGUAGES array
{ code: "xx", label: "XX", name: "Language Name", flag: "🏳️" },
```
### 2. Add to Generator
Edit `scripts/i18n/generate-multilang.mjs` — add entry to `LOCALE_SPECS`:
```js
{
code: "xx",
googleTl: "xx",
label: "XX",
flag: "🏳️",
languageName: "Language Name",
readmeName: "Language Name",
docsName: "Language Name",
},
```
### 3. Generate Initial Translation
```bash
node scripts/i18n/generate-multilang.mjs messages
```
This creates `src/i18n/messages/xx.json` auto-translated from `en.json` via Google Translate.
### 4. Review & Fix Auto-Translations
Auto-translations are a starting point. Review manually for:
- Technical accuracy
- Context-appropriate terminology
- Proper handling of placeholders (`{count}`, `{value}`, etc.)
### 5. Validate
```bash
python3 scripts/i18n/validate_translation.py quick -l xx
python3 scripts/i18n/validate_translation.py diff common -l xx
```
### 6. Generate Translated Documentation
```bash
node scripts/i18n/generate-multilang.mjs docs
```
## Auto-Translation Pipeline
### generate-multilang.mjs (Google Translate)
**Primary auto-translation engine** — uses Google Translate free API to generate translations for UI strings, READMEs, and documentation.
```bash
node scripts/i18n/generate-multilang.mjs [messages|readme|docs|all]
```
| Mode | What it does |
| ---------- | ----------------------------------------------------------------------------- |
| `messages` | Translates missing keys in `src/i18n/messages/{locale}.json` from `en.json` |
| `readme` | Translates `README.md` into all locales as `README.{code}.md` in project root |
| `docs` | Translates `DOC_SOURCE_FILES` into `docs/i18n/{locale}/{docName}` |
| `all` | Runs all three modes |
**Features:**
- **Text protection**: Masks code blocks (` ``` `), inline code (`` ` ``), markdown links/images (`[text](url)`), HTML tags, tables, and ICU placeholders (`{count}`, `{value}`, `{total}`, etc.) before translation, then restores them
- **Chunked batching**: Joins multiple strings with `__OMNIROUTE_I18N_SEPARATOR__` delimiters to minimize API calls (max 1800 chars per request)
- **In-memory cache**: Avoids redundant API calls for repeated strings within a session
- **Retry logic**: Exponential backoff (up to 5 attempts with 300ms × attempt delay) for 429/5xx errors
- **Timeout**: 20 seconds per request
- **Skip existing**: If target file already exists, it is NOT overwritten
**Important behaviors:**
- `docs/i18n/README.md` is **regenerated** each run — it's an auto-generated index of all docs
- Root `README.{code}.md` files are only created if they don't exist (skips locales in `EXISTING_README_CODES`)
- Language bars (`🌐 **Languages:** ...`) are automatically inserted/updated in all translated docs
### i18n_autotranslate.py (LLM-based)
**Secondary translator** — uses any OpenAI-compatible LLM API (including OmniRoute itself) to translate existing `docs/i18n/` markdown files. Best for polishing or re-translating docs with better quality than Google Translate.
```bash
python3 scripts/i18n/i18n_autotranslate.py \
--api-url http://localhost:20128/v1 \
--api-key sk-your-key \
--model gpt-4o
```
**Features:**
- Scans `docs/i18n/` markdown files for English paragraphs
- Skips code blocks, tables, and already-translated content
- Sends paragraphs to LLM with technical translation system prompt
- Supports all 30 languages
## CLI i18n
The `omniroute` CLI has its own i18n layer separate from the Next.js dashboard.
### How it works
- Every user-facing string in CLI commands goes through `t("module.key", vars)` from `bin/cli/i18n.mjs`.
- Catalogs are JSON files in `bin/cli/locales/` — 42 ship out-of-the-box.
- Locale falls back to `en` for any missing key, so partial translations are valid.
- The source of truth for available locales is `config/i18n.json` (shared with the dashboard).
### Locale selection
Detection order (first match wins):
| Priority | Source | Example |
| -------- | ------------------------ | --------------------------------------- |
| 1 | `--lang` flag | `omniroute --lang de status` |
| 2 | `OMNIROUTE_LANG` env var | `OMNIROUTE_LANG=ja omniroute providers` |
| 3 | `LC_ALL` system env | auto-detected from terminal locale |
| 4 | `LC_MESSAGES` system env | auto-detected from terminal locale |
| 5 | `LANG` system env | auto-detected from terminal locale |
| 6 | Fallback | `en` |
Locale codes with underscores (`pt_BR`) are normalized to hyphen form (`pt-BR`).
Locale codes are validated against `/^[a-zA-Z0-9-]+$/` — path traversal is rejected.
### Saving a language preference
```bash
# Set language and save to ~/.omniroute/.env (persists across sessions)
omniroute config lang set pt-BR
# View current language
omniroute config lang get
# List all 42 available languages
omniroute config lang list
# JSON output
omniroute config lang list --output json
```
The saved preference is written atomically to `~/.omniroute/.env` and is loaded by the
CLI bootstrap before any command runs.
### One-time override
```bash
# Override for one command only (not persisted)
omniroute --lang de providers list
```
Note: the `--lang` flag does NOT write to the env file — it only affects the current
invocation. Use `config lang set` to persist.
### Available locales
42 locale files ship in `bin/cli/locales/`. Full translations: `en`, `pt-BR`.
Scaffold-only (all keys fall back to `en`): `bn`, `gu`, `he`, `in`, `mr`, `ms`, `phi`, `sw`, `ta`, `te`, `ur`.
All other 29 locales have `common` + `program` keys translated.
### Adding a new CLI locale
1. Add the locale entry to `config/i18n.json`.
2. Run `node bin/cli/scripts/generate-locales.mjs` — creates the locale file.
3. Translate the keys (or leave as `{}` for en-fallback scaffold).
4. PRs must add strings to `en.json` and `pt-BR.json`; other files are best-effort.
## Validation & QA
### validate_translation.py
**Translation validator** — compares any locale JSON against `en.json` and reports issues.
```bash
# Quick check (counts only)
python3 scripts/i18n/validate_translation.py quick -l cs
# Output:
# Missing: 0
# Untranslated: 0
# Ignored (UNTRANSLATABLE_KEYS): 236
# Detailed diff by category
python3 scripts/i18n/validate_translation.py diff common -l cs
python3 scripts/i18n/validate_translation.py diff settings -l cs
# Export to CSV
python3 scripts/i18n/validate_translation.py csv -l cs > report.csv
# Export to Markdown
python3 scripts/i18n/validate_translation.py md -l cs > report.md
# Full report (default)
python3 scripts/i18n/validate_translation.py -l cs
```
**Detects:**
- **Missing keys** — keys in `en.json` but not in locale file
- **Extra keys** — keys in locale file but not in `en.json`
- **Untranslated keys** — keys where locale value equals English source (excluding allowlist)
- **Placeholder mismatches** — ICU placeholders that don't match between source and translation
**Exit codes:**
| Code | Meaning |
|------|---------|
| 0 | OK |
| 1 | Generic error |
| 2 | Missing strings (hard error) |
| 3 | Untranslated warning (soft) |
**Environment:** Set `TRANSLATION_LANG=cs` or use `-l cs` flag.
### check_translations.py
**Code-to-JSON key checker** — scans `src/**/*.tsx` and `src/**/*.ts` for `useTranslations()` calls and verifies all referenced keys exist in `en.json`.
```bash
# Basic check
python3 scripts/i18n/check_translations.py
# Verbose output
python3 scripts/i18n/check_translations.py --verbose
# Auto-fix (adds missing keys to en.json)
python3 scripts/i18n/check_translations.py --fix
```
### generate-qa-checklist.mjs
**Static analysis QA** — scans Next.js page files for i18n risk metrics and generates a Markdown report.
```bash
node scripts/i18n/generate-qa-checklist.mjs
```
**Checks:**
- Fixed-width class usage (overflow risk)
- Directional left/right classes (RTL risk)
- Clipping-prone patterns
- Locale parity (missing/extra keys vs `en.json`)
- README language selector bars in priority locales (`es`, `fr`, `de`, `ja`, `ar`)
**Output:** `docs/reports/i18n-qa-checklist-{date}.md`
### run-visual-qa.mjs
**Visual QA via Playwright** — takes screenshots of all dashboard routes in multiple locales and viewports, then evaluates page health.
```bash
# Default: es, fr, de, ja, ar on localhost:20128
node scripts/i18n/run-visual-qa.mjs
# Custom base URL and locales
QA_BASE_URL=http://staging.example.com QA_LOCALES=de,fr node scripts/i18n/run-visual-qa.mjs
# Custom routes
QA_ROUTES=/dashboard/settings,/dashboard/providers node scripts/i18n/run-visual-qa.mjs
```
**Detects:**
- Text overflow
- Element clipping
- RTL layout mismatches
**Output:** `docs/reports/i18n-visual-qa-{date}.md` + JSON report
## Managing Untranslatable Keys
### untranslatable-keys.json
**File:** `scripts/i18n/untranslatable-keys.json`
Allowlist of keys that should remain identical to English source. Used by `validate_translation.py` to avoid false-positive "untranslated" warnings.
```json
{
"description": "Keys that should remain untranslated...",
"keys": [
"common.model",
"common.oauth",
"health.cpu",
...
]
}
```
**What belongs here:**
- Brand/product names: `landing.brandName`, `common.social-github`
- Technical terms/acronyms: `health.cpu`, `mcpDashboard.pid`, `settings.ai`
- ICU/format strings: `apiManager.modelsCount`, `health.millisecondsShort`
- Placeholder values: `providers.openaiBaseUrlPlaceholder`, `cliTools.baseUrlPlaceholder`
- Protocol names: `common.http`, `common.oauth`, `providers.oauth2Label`
- Navigation sections: `sidebar.primarySection`, `sidebar.cliSection`
**To add a key:** Edit the `keys` array in `scripts/i18n/untranslatable-keys.json` and re-run validation.
## CI Integration
### GitHub Actions (`.github/workflows/ci.yml`)
The CI pipeline validates all locales on every push and PR:
1. **`i18n-matrix` job** — dynamically discovers all locale files (excluding `en.json`)
2. **`i18n` job** — runs `validate_translation.py quick -l '<lang>'` for each locale in parallel
3. **`ci-summary` job** — aggregates results into a dashboard summary
```yaml
# i18n-matrix: discovers languages
LANGS=$(ls src/i18n/messages/*.json | xargs -n1 basename | sed 's/.json$//' | grep -v '^en$')
# i18n: validates each language
python3 scripts/i18n/validate_translation.py quick -l '${{ matrix.lang }}'
```
**Dashboard output:**
```
## 🌍 Translations
| Metric | Value |
|--------|------|
| Languages checked | 30 |
| Total untranslated | 0 |
✅ All translations complete
```
## File Structure
```
src/i18n/
├── config.ts # Locale definitions (30 locales, RTL config)
├── request.ts # Runtime locale resolution
└── messages/
├── en.json # Source of truth (~2800 keys)
├── cs.json # Czech translation
├── de.json # German translation
└── ... # 30 locale files total
scripts/
├── i18n/
│ ├── generate-multilang.mjs # Auto-translation engine (Google Translate, 888 lines)
│ ├── generate-qa-checklist.mjs # Static analysis QA
│ ├── run-visual-qa.mjs # Playwright visual QA
│ └── untranslatable-keys.json # Allowlist for validation (236 keys)
├── validate_translation.py # Translation validator
├── check_translations.py # Code-to-JSON key checker
└── i18n_autotranslate.py # LLM-based doc translator
.github/workflows/
└── ci.yml # i18n validation in CI matrix
docs/
├── I18N.md # This file — i18n toolchain documentation
├── i18n/
│ ├── README.md # Auto-generated language index
│ ├── cs/ # Czech docs
│ │ └── docs/
│ │ ├── I18N.md # Czech translation of this file
│ │ └── ...
│ ├── de/ # German docs
│ └── ... # 30 locale directories
└── reports/
├── i18n-qa-checklist-*.md # Static analysis reports
└── i18n-visual-qa-*.md # Visual QA reports
```
## Best Practices
### When Editing Translations
1. **Always edit `en.json` first** — it's the source of truth
2. **Run `generate-multilang.mjs messages`** to propagate new keys to all locales
3. **Review auto-translations** — Google Translate is a starting point, not final
4. **Validate before committing** — `python3 scripts/i18n/validate_translation.py quick -l <lang>`
5. **Update `untranslatable-keys.json`** if a key should remain in English
### Placeholder Safety
- ICU placeholders (`{count}`, `{value}`, `{total}`, `{seconds}`) must be preserved exactly
- Plural formats (`{count, plural, one {# model} other {# models}}`) must maintain structure
- The validator detects placeholder mismatches automatically
### Adding New Translation Keys in Code
```tsx
// Use namespaced keys
const t = useTranslations("settings");
t("cacheSettings"); // maps to settings.cacheSettings in JSON
// Run check_translations.py to verify keys exist
python3 scripts/i18n/check_translations.py --verbose
```
### RTL Considerations
- Arabic (`ar`) and Hebrew (`he`) are RTL locales
- Avoid hardcoded `left`/`right` CSS — use `start`/`end` logical properties
- Visual QA catches RTL layout mismatches via `run-visual-qa.mjs`
## Known Issues & History
### `in.json` → `hi.json` Fix
The generator originally used `code: "in"` (deprecated Google Translate code) for Hindi instead of the correct ISO 639-1 `hi`. This created an orphaned `in.json` duplicate of `hi.json`. Fixed by changing `code: "in"` to `code: "hi"` in `generate-multilang.mjs` and removing the orphaned file.
> ⚠️ **Audit (2026-05-13):** The `docs/i18n/in/` directory still exists on disk (full duplicate of `hi/`). Translation generator no longer writes to it, but the historical tree was not pruned. Safe to delete with `rm -rf docs/i18n/in/` after confirming no external links reference the old path.
### `docs/i18n/README.md` Is Auto-Generated
The `docs/i18n/README.md` file is completely regenerated by `generate-multilang.mjs docs`. Any manual edits will be lost. Use `docs/guides/I18N.md` (this file) for hand-written documentation that should persist.
### External Untranslatable Keys List
The `untranslatable-keys.json` allowlist was moved from an inline Python set in `validate_translation.py` to an external JSON file for easier maintenance. The validator loads it at runtime.
### `generate-multilang.mjs` Hindi Code Fix
The generator originally used `code: "in"` (deprecated Google Translate code) for Hindi instead of the correct ISO 639-1 `hi`. This was introduced in upstream commit `952b0b22c` by `diegosouzapw`. Fixed by changing `code: "in"` to `code: "hi"` in the `LOCALE_SPECS` array and removing the orphaned `in.json` file.
### `validate_translation.py` Ignored Count Output
The `quick` check now displays the count of ignored keys from `untranslatable-keys.json`:
```
Missing: 0
Untranslated: 0
Ignored (UNTRANSLATABLE_KEYS): <varies per release>
```

136
Kiro-Setup.md Normal file

@@ -0,0 +1,136 @@
# Kiro Setup Guide
This guide covers adding Kiro (AWS-hosted AI coding assistant) accounts to OmniRoute,
with a focus on running multiple accounts simultaneously without session conflicts.
---
## Background: Why Kiro Accounts Can Conflict
Kiro's backend uses AWS SSO OIDC client registrations to track active sessions.
The critical constraint: **each OIDC client registration supports only one active
session at a time**. When a second device or user authenticates using the same
registered client, the backend invalidates the first account's refresh token.
This is the same mechanism that causes problems when running `kiro-cli login` on a
machine where another Kiro account is already signed in — the new login revokes the
first account's token.
---
## How OmniRoute Solves This (v3.8.0+)
Starting with v3.8.0, OmniRoute calls `registerClient()` (AWS SSO OIDC) during every
Kiro connection import. This gives each OmniRoute connection its own dedicated OIDC
client registration. Because each client registration is independent, refreshing or
re-authenticating one account does not affect any other account's refresh token.
The isolation applies to all three import methods:
| Import method | Isolation status |
| --------------------------------------------- | ------------------------------------------------------------------------------------------------ |
| AWS Builder ID / IDC device-code flow | Isolated since the device-code flow was introduced |
| **Import Token** (manual refresh token paste) | Isolated from v3.8.0 |
| **Google / GitHub social login** | Isolated from v3.8.0 |
| **Auto-Import** (kiro-cli SQLite) | Isolated from v3.8.0 (SQLite path was already isolated; SSO-cache fallback is now also isolated) |
---
## Migration Note for Connections Created Before v3.8.0
Connections imported before v3.8.0 do not have a dedicated OIDC client registration
stored in `providerSpecificData`. These connections continue to work but use the shared
social-auth refresh endpoint, which means two such connections can still invalidate each
other.
**To gain isolation:** delete the old connection from **Dashboard → Providers** and
re-import it using any of the supported import flows. All newly created connections will
receive their own client registration automatically.
---
## Adding Two Kiro Accounts Side by Side
### Prerequisites
- OmniRoute v3.8.0 or later.
- A working Kiro account (email + password, Google, or GitHub login).
- Optionally a second Kiro account.
### Step 1: Import the first account
1. Open **Dashboard → Providers → Add Provider → Kiro**.
2. Choose one of:
- **Import Token** — paste a refresh token starting with `aorAAAAAG`.
- **Google / GitHub login** — complete the OAuth flow in the browser.
- **Auto-Import** — click the button; OmniRoute reads credentials from the
local kiro-cli database or `~/.aws/sso/cache`.
3. The connection is saved. OmniRoute automatically registers a dedicated OIDC client for it.
### Step 2: Import the second account
Repeat step 1 for the second account. Because each import creates a separate OIDC
client registration, the two connections are fully isolated.
### Step 3: Verify both connections are active
1. **Dashboard → Providers** — both Kiro connections should show **Active** status.
2. **Dashboard → Health** — both connections should pass their token health check.
### Step 4: Use a combo to route between accounts
Create a combo with both connections as targets to load-balance or fall back between them:
```
kiro/kiro-dev → kiro/kiro-pro
```
See [FEATURES.md](./FEATURES.md) and the routing documentation for combo configuration.
---
## Enterprise / IDC Users
For AWS IAM Identity Center (IDC) accounts, use the **AWS Builder ID / IDC device-code**
flow from **Dashboard → Providers → Kiro → Device Code**. The device-code flow has
always been fully isolated. No re-import is needed for these connections.
Enterprise users who operate in a non-default AWS region can specify the region when
importing via the Import Token API:
```bash
curl -X POST http://localhost:20128/api/oauth/kiro/import \
-H "Content-Type: application/json" \
-d '{"refreshToken": "aorAAAAAG...", "region": "eu-west-1"}'
```
The `region` field defaults to `us-east-1` when omitted.
---
## OIDC Client Expiry
AWS SSO OIDC public clients typically expire after 90 days
(`clientSecretExpiresAt`). OmniRoute stores this timestamp in `providerSpecificData`
for observability. If a connection stops refreshing after ~90 days, re-import the
connection to obtain a fresh OIDC client registration. Automatic re-registration on
expiry is tracked as a future improvement.
---
## Troubleshooting
### Second account keeps getting logged out
- Check both connections in **Dashboard → Providers** and confirm each shows a non-null
`clientId` in its raw JSON (visible via the info icon). If either connection is missing
`clientId`, it was imported before v3.8.0 — re-import it.
### Import fails with "Token validation failed"
- Ensure the refresh token starts with `aorAAAAAG`.
- Ensure OmniRoute can reach `https://oidc.us-east-1.amazonaws.com` (or the configured
region). If you are behind a corporate proxy, set a provider-level proxy in
**Dashboard → Settings → Proxies**.
For other issues, see the main [TROUBLESHOOTING.md](./TROUBLESHOOTING.md).

331
MCP-Server.md Normal file

@@ -0,0 +1,331 @@
# OmniRoute MCP Server Documentation
> Model Context Protocol server with 37 tools across routing, cache, compression, memory, skills, and proxy operations.
>
> Source of truth: `open-sse/mcp-server/schemas/tools.ts` (30 tools) + `open-sse/mcp-server/tools/memoryTools.ts` (3 tools) + `open-sse/mcp-server/tools/skillTools.ts` (4 tools). Tool registration and scope wiring lives in `open-sse/mcp-server/server.ts`.
![MCP tool inventory (37 tools by category)](../diagrams/exported/mcp-tools-37.svg)
> Source: [diagrams/mcp-tools-37.mmd](../diagrams/mcp-tools-37.mmd)
## Installation
OmniRoute MCP is built-in. Start it with:
```bash
omniroute --mcp
```
Or via the open-sse transport:
```bash
# HTTP streamable transport (port 20130)
omniroute --dev # MCP auto-starts on /mcp endpoint
```
## Transports
The MCP server exposes three transports, all backed by the same `createMcpServer()` factory:
| Transport | Where | When to use |
| :---------------- | :------------------------------------------ | :--------------------------------------------------- |
| `stdio` | `open-sse/mcp-server/server.ts` | IDE integrations (Claude Desktop, Cursor, etc.) |
| `sse` | `POST/GET /api/mcp/sse` via `httpTransport` | Browser/agent clients that need an event stream |
| `streamable-http` | `POST/GET/DELETE /api/mcp/stream` | Multi-session HTTP clients (`mcp-session-id` header) |
The active HTTP transport (`sse` or `streamable-http`) is selected by the `mcpTransport` setting. Switching transports closes existing sessions on the other transport.
### Remote access (manage-scope bypass)
`/api/mcp/*` is in the LOCAL_ONLY tier (`src/server/authz/routeGuard.ts`) — by default only loopback hosts (`localhost`, `127.0.0.1`, `::1`) can reach it. Since v3.8.2, non-loopback clients may connect if they present an `Authorization: Bearer <api-key>` whose key carries the `manage` scope. This is the only way to reach the remote MCP server through a tunnel, reverse proxy, or public hostname.
```bash
# Grant manage scope: open the dashboard API Manager and toggle
# "Management Access" on the key, or POST scopes:["manage"] when creating.
# Then connect from a remote MCP client:
curl -i \
-H "Host: your-public-host.example" \
-H "Authorization: Bearer sk-…" \
-H "Content-Type: application/json" \
-H "Accept: application/json, text/event-stream" \
-d '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-03-26","capabilities":{},"clientInfo":{"name":"my-client","version":"0"}}}' \
https://your-public-host.example/api/mcp/stream
```
A non-manage key (or no Bearer) returns `403 LOCAL_ONLY`. The sibling prefix `/api/cli-tools/runtime/*` is intentionally NOT bypassable — see [Route Guard Tiers — Manage-scope carve-out](../security/ROUTE_GUARD_TIERS.md#manage-scope-carve-out).
## IDE Configuration
See [MCP Client Configuration](../guides/SETUP_GUIDE.md#mcp-client-configuration) for Claude Desktop,
Cursor, Cline, and compatible MCP client setup.
---
## Essential Tools (8) — Phase 1
| Tool | Scopes | Description |
| :------------------------------ | :-------------------- | :------------------------------------------------------------ |
| `omniroute_get_health` | `read:health` | Uptime, memory, circuit breakers, rate limits, cache stats |
| `omniroute_list_combos` | `read:combos` | All configured combos with strategies (optional metrics) |
| `omniroute_get_combo_metrics` | `read:combos` | Performance metrics for a specific combo |
| `omniroute_switch_combo` | `write:combos` | Activate or deactivate a combo |
| `omniroute_check_quota` | `read:quota` | Quota used/total, percent remaining, reset time, token health |
| `omniroute_route_request` | `execute:completions` | Send a chat completion through OmniRoute routing |
| `omniroute_cost_report` | `read:usage` | Cost report by period (session/day/week/month) |
| `omniroute_list_models_catalog` | `read:models` | Full model catalog with capabilities, status, pricing |
## Phase 1 — Search
| Tool | Scopes | Description |
| :--------------------- | :--------------- | :--------------------------------------------------------------------------------------------------------------------------------- |
| `omniroute_web_search` | `execute:search` | Web search through OmniRoute search gateway (Serper/Brave/Perplexity/Exa/Tavily/Google PSE/Linkup/SearchAPI/SearXNG) with failover |
## Advanced Tools (11) — Phase 2
| Tool | Scopes | Description |
| :--------------------------------- | :----------------------------------- | :---------------------------------------------------------------------------------------- |
| `omniroute_simulate_route` | `read:health`, `read:combos` | Dry-run routing simulation with fallback tree |
| `omniroute_set_budget_guard` | `write:budget` | Session budget with degrade/block/alert action |
| `omniroute_set_routing_strategy` | `write:combos` | Update combo strategy at runtime (priority/weighted/auto/etc.) |
| `omniroute_set_resilience_profile` | `write:resilience` | Apply `aggressive` / `balanced` / `conservative` resilience preset |
| `omniroute_test_combo` | `execute:completions`, `read:combos` | Live test of every provider in a combo using a real upstream call |
| `omniroute_get_provider_metrics` | `read:health` | Per-provider metrics with p50/p95/p99 latency and circuit breaker state |
| `omniroute_best_combo_for_task` | `read:combos`, `read:health` | Recommend combo by task type with budget/latency constraints |
| `omniroute_explain_route` | `read:health`, `read:usage` | Explain why a request was routed to a provider (scoring factors + fallbacks) |
| `omniroute_get_session_snapshot` | `read:usage` | Full session snapshot: cost, tokens, top models/providers, errors, budget guard |
| `omniroute_db_health_check` | `read:health`, `write:resilience` | Diagnose (and optionally auto-repair) database drift like broken combo refs / orphan rows |
| `omniroute_sync_pricing` | `pricing:write` | Sync pricing data from external sources (LiteLLM); supports `dryRun` |
## Cache Tools (2)
| Tool | Scopes | Description |
| :---------------------- | :------------ | :-------------------------------------------------- |
| `omniroute_cache_stats` | `read:cache` | Semantic cache, prompt-cache, and idempotency stats |
| `omniroute_cache_flush` | `write:cache` | Flush cache globally or by signature/model |
## Compression Tools (5)
| Tool | Scopes | Description |
| :---------------------------------- | :------------------ | :----------------------------------------------------------------------------------------------------------------------- |
| `omniroute_compression_status` | `read:compression` | Compression settings, analytics summary, and cache-aware stats (includes `analytics.mcpDescriptionCompression` metadata) |
| `omniroute_compression_configure` | `write:compression` | Configure compression mode, threshold, target ratio, system-prompt preservation, MCP description compression toggle |
| `omniroute_set_compression_engine` | `write:compression` | Pick the active engine (off/caveman/rtk/stacked) and Caveman/RTK intensity |
| `omniroute_list_compression_combos` | `read:compression` | List named compression combos and their engine pipelines |
| `omniroute_compression_combo_stats` | `read:compression` | Analytics grouped by compression combo and engine |
`omniroute_compression_status` reports MCP description compression separately under
`analytics.mcpDescriptionCompression`. Those values are metadata-size estimates for MCP listable
descriptions (`tools`, `prompts`, `resources`, and `resourceTemplates`); they are not provider usage
receipts and are marked with `source: "mcp_metadata_estimate"`.
### MCP Accessibility Tree Filter (v3.8.0)
Separate from the 5 compression tools above, OmniRoute includes a post-execution filter that
compresses the **tool results** of MCP browser/accessibility tools before they are returned to the
agent. This filter is not itself a tool — it runs transparently on any tool result that contains
verbose accessibility-tree or browser-snapshot text (≥2000 chars).
Key behaviors:
- Collapses ≥30 consecutive repeated sibling lines into head + tail summary
- Preserves `[ref=eXX]` anchors required by Playwright/computer-use
- Hard-truncates oversized text (>50,000 chars) with a navigation hint
- Expected savings: **60–80%** on browser snapshot payloads
Configuration: `compression.mcpAccessibility` in global settings (migration 056).
Implementation: `open-sse/services/compression/engines/mcpAccessibility/`.
Full docs: [Compression Engines — MCP Accessibility Tree Filter](../compression/COMPRESSION_ENGINES.md#mcp-accessibility-tree-filter).
See [Compression Engines](../compression/COMPRESSION_ENGINES.md) and [RTK Compression](../compression/RTK_COMPRESSION.md) for
the runtime compression model behind these tools.
## 1Proxy Tools (3)
| Tool | Scopes | Description |
| :-------------------------- | :------------- | :-------------------------------------------------------------------------------------- |
| `omniroute_oneproxy_fetch` | `read:proxies` | Fetch free proxies from the 1proxy marketplace (protocol/country/quality/limit filters) |
| `omniroute_oneproxy_rotate` | `read:proxies` | Get the next available proxy by strategy (`random` / `quality` / `sequential`) |
| `omniroute_oneproxy_stats` | `read:proxies` | Pool stats, sync status, distribution by protocol and country |
## Memory Tools (3)
Defined in `open-sse/mcp-server/tools/memoryTools.ts`. Auth/scope is enforced through the standard MCP scope pipeline.
| Tool | Description |
| :------------------------ | :---------------------------------------------------------------------------------- |
| `omniroute_memory_search` | Search memories by query / type / API key with token-budget enforcement |
| `omniroute_memory_add` | Add a new memory entry (`factual` / `episodic` / `procedural` / `semantic`) |
| `omniroute_memory_clear` | Clear memories for an API key, optionally filtered by type or `olderThan` timestamp |
## Skill Tools (4)
Defined in `open-sse/mcp-server/tools/skillTools.ts`. Backed by `src/lib/skills/registry` + `src/lib/skills/executor`.
| Tool | Description |
| :---------------------------- | :-------------------------------------------------------------------------------- |
| `omniroute_skills_list` | List registered skills with optional filtering by API key, name, or enabled state |
| `omniroute_skills_enable` | Enable or disable a specific skill by ID |
| `omniroute_skills_execute` | Execute a skill with provided input and return the execution record |
| `omniroute_skills_executions` | List recent skill execution history |
## Related Frameworks (v3.8.0)
The MCP tool inventory above (37 tools = 30 base + 3 memory + 4 skills) is intentionally
scoped to runtime routing/cache/compression/memory/skills/proxy operations. Two adjacent
frameworks ship alongside the MCP server in v3.8.0 and are documented separately:
### Cloud Agents
Cloud Agents are out-of-process AI coding agents (codex-cloud, devin, jules) wired into
OmniRoute through the same connection model used for LLM providers. They are exposed via
their own REST surface (`/api/v1/agents/*`) and are **not** part of the MCP tool catalog
— calling a Cloud Agent does not consume an MCP scope.
- Implementation: `src/lib/cloudAgent/` (`registry.ts`, `agents/codex-cloud.ts`, `agents/devin.ts`, `agents/jules.ts`).
- Lifecycle: `createTask`, `getStatus`, `approvePlan`, `sendMessage`, `listSources`.
- Documentation: [docs/frameworks/CLOUD_AGENT.md](./CLOUD_AGENT.md).
### Guardrails
Guardrails are pre/post-execution filters (vision-bridge, pii-masker, prompt-injection)
applied inside the chat pipeline. They run before the MCP tool/route layer is reached
and emit structured violations to the audit pipeline; they are not invoked as MCP tools.
- Implementation: `src/lib/guardrails/`.
- Documentation: [docs/security/GUARDRAILS.md](../security/GUARDRAILS.md).
When debugging an MCP call that appears blocked, check both the MCP audit log
(`scope_denied:*` entries) and the guardrails audit trail — a request may be rejected by
a guardrail **before** it ever reaches the MCP scope enforcement layer.
---
## REST API Endpoints
| Endpoint | Method | Description | Auth |
| :--------------------- | :-------------------- | :-------------------------------------------------------------------------------------------------- | :------------------------- |
| `/api/mcp/status` | `GET` | Server status: heartbeat, HTTP transport state, audit activity summary | Management (session/admin) |
| `/api/mcp/tools` | `GET` | Tool catalog (name, description, scopes, phase, source endpoints) | Management |
| `/api/mcp/sse` | `GET` / `POST` | SSE transport endpoint (gated by `mcpEnabled` + `mcpTransport === "sse"`) | API key + scopes |
| `/api/mcp/stream` | `POST`/`GET`/`DELETE` | Streamable HTTP transport (uses `mcp-session-id` header; `DELETE` ends the session) | API key + scopes |
| `/api/mcp/audit` | `GET` | Audit log entries from `mcp_tool_audit` (filters: `limit`, `offset`, `tool`, `success`, `apiKeyId`) | Management |
| `/api/mcp/audit/stats` | `GET` | Aggregated audit stats (`totalCalls`, `successRate`, `avgDurationMs`, top tools) | Management |
Source files: `src/app/api/mcp/{status,tools,sse,stream,audit,audit/stats}/route.ts`.
Both SSE and Streamable HTTP transports are blocked until the MCP server is enabled in Settings (`mcpEnabled`) and the appropriate `mcpTransport` is selected. If the wrong transport is configured the route returns HTTP 400 with a hint to switch settings.
---
## Authentication & Scopes
MCP tools are authenticated through API key scopes. Scope enforcement is centralized in
`open-sse/mcp-server/scopeEnforcement.ts`. Each tool requires specific scopes:
| Scope | Tools |
| :-------------------- | :---------------------------------------------------------------------------------------------------------------- |
| `read:health` | `get_health`, `get_provider_metrics`, `simulate_route`, `explain_route`, `best_combo_for_task`, `db_health_check` |
| `read:combos` | `list_combos`, `get_combo_metrics`, `simulate_route`, `best_combo_for_task`, `test_combo` |
| `write:combos` | `switch_combo`, `set_routing_strategy` |
| `read:quota` | `check_quota` |
| `read:usage` | `cost_report`, `get_session_snapshot`, `explain_route` |
| `read:models` | `list_models_catalog` |
| `execute:completions` | `route_request`, `test_combo` |
| `execute:search` | `web_search` |
| `write:budget` | `set_budget_guard` |
| `write:resilience` | `set_resilience_profile`, `db_health_check` |
| `pricing:write` | `sync_pricing` |
| `read:cache` | `cache_stats` |
| `write:cache` | `cache_flush` |
| `read:compression` | `compression_status`, `list_compression_combos`, `compression_combo_stats` |
| `write:compression` | `compression_configure`, `set_compression_engine` |
| `read:proxies` | `oneproxy_fetch`, `oneproxy_rotate`, `oneproxy_stats` |
Wildcard scopes are supported: `read:*` grants all read-scopes, `*` grants full access.
Memory and Skill tools currently do not declare static scope requirements in their definitions; access is gated by the caller's API key and audited through the standard MCP audit pipeline.
---
## Environment Variables
| Variable | Default | Purpose |
| :-------------------------------------- | :--------------------------------- | :----------------------------------------------------------------------------------------------------------------------- |
| `OMNIROUTE_BASE_URL` | `http://localhost:20128` | Base URL the MCP server uses when calling OmniRoute internal APIs |
| `OMNIROUTE_API_KEY` | (empty) | API key forwarded as `Authorization: Bearer` to internal API calls |
| `OMNIROUTE_MCP_ENFORCE_SCOPES` | `false` (only `"true"` enables it) | When enabled, missing scopes deny tool calls and log `scope_denied:<reason>` in audit log |
| `OMNIROUTE_MCP_SCOPES` | (empty) | Comma-separated allowlist of scopes considered "available" by default (used when caller does not provide its own scopes) |
| `OMNIROUTE_MCP_COMPRESS_DESCRIPTIONS` | (unset = on) | When set to `0/false/off/no`, disables MCP description compression at registration time |
| `OMNIROUTE_MCP_DESCRIPTION_COMPRESSION` | (unset = on) | Alternate alias for the same toggle as above |
| `DATA_DIR` | `~/.omniroute` | Heartbeat file is written to `${DATA_DIR}/runtime/mcp-heartbeat.json` |
---
## Description Compression
MCP tool, prompt, and resource registries can compress descriptions at registration/list time to reduce the metadata footprint exposed to clients (and therefore the prompt context cost). The implementation lives in `open-sse/mcp-server/descriptionCompressor.ts` and is wired into the MCP server via `compressMcpRegistryMetadata` inside `createMcpServer()`.
- Compression runs over the description text using the Caveman ruleset (`getRulesForContext("all", "full")`) with preserved-block extraction (code spans, fenced blocks, etc.) so structural content is not altered.
- Toggle per-deployment via the `compression.mcpDescriptionCompressionEnabled` value in the `key_value` settings table (default: enabled) — exposed in the UI as **Analytics → MCP description compression**.
- Toggle process-wide via either `OMNIROUTE_MCP_COMPRESS_DESCRIPTIONS=false` or `OMNIROUTE_MCP_DESCRIPTION_COMPRESSION=false`.
- Realtime stats are surfaced via `omniroute_compression_status` under `analytics.mcpDescriptionCompression` and tagged `source: "mcp_metadata_estimate"` to disambiguate from real provider usage receipts.
---
## Runtime Heartbeat
The stdio transport persists liveness to `${DATA_DIR}/runtime/mcp-heartbeat.json` every 5 seconds. The dashboard (`/api/mcp/status`) reads this file plus PID liveness to derive `online`. HTTP transports report state from in-process `getMcpHttpStatus()` instead (no file write).
The heartbeat snapshot contains:
```json
{
"pid": 12345,
"startedAt": "2026-05-13T12:34:56.000Z",
"lastHeartbeatAt": "2026-05-13T12:35:01.000Z",
"version": "1.8.1",
"transport": "stdio",
"scopesEnforced": false,
"allowedScopes": [],
"toolCount": 37
}
```
---
## Audit Logging
Every tool call is logged to the SQLite `mcp_tool_audit` table by `open-sse/mcp-server/audit.ts`:
- Tool name, arguments (hashed/truncated as per per-tool `auditLevel`), result
- Duration in ms, success/failure flag, error message (when applicable)
- API key hash, timestamp
- Scope denials are logged as `scope_denied:<reason>` with the missing scope list
Use the dashboard or the `/api/mcp/audit` and `/api/mcp/audit/stats` REST endpoints to inspect recent calls.
---
## Files
| File | Purpose |
| :---------------------------------------------- | :--------------------------------------------------------------- |
| `open-sse/mcp-server/server.ts` | MCP server factory, stdio entry point, scoped tool registrations |
| `open-sse/mcp-server/httpTransport.ts` | SSE + Streamable HTTP transport (session management) |
| `open-sse/mcp-server/scopeEnforcement.ts` | Tool scope evaluation and caller resolution |
| `open-sse/mcp-server/audit.ts` | Tool call audit logging (`mcp_tool_audit`) |
| `open-sse/mcp-server/runtimeHeartbeat.ts` | stdio heartbeat writer (`mcp-heartbeat.json`) |
| `open-sse/mcp-server/descriptionCompressor.ts` | Description compression for tool / prompt / resource registries |
| `open-sse/mcp-server/schemas/tools.ts` | Zod schemas + tool registry (`MCP_TOOLS`, 30 entries) |
| `open-sse/mcp-server/tools/advancedTools.ts` | Phase 2 + cache + 1proxy tool handlers |
| `open-sse/mcp-server/tools/compressionTools.ts` | Compression tool handlers |
| `open-sse/mcp-server/tools/memoryTools.ts` | Memory tool definitions (3 tools) |
| `open-sse/mcp-server/tools/skillTools.ts` | Skill tool definitions (4 tools) |
| `src/app/api/mcp/status/route.ts` | `/api/mcp/status` endpoint |
| `src/app/api/mcp/tools/route.ts` | `/api/mcp/tools` endpoint |
| `src/app/api/mcp/sse/route.ts` | `/api/mcp/sse` SSE transport route |
| `src/app/api/mcp/stream/route.ts` | `/api/mcp/stream` Streamable HTTP transport route |
| `src/app/api/mcp/audit/route.ts` | `/api/mcp/audit` audit log query |
| `src/app/api/mcp/audit/stats/route.ts` | `/api/mcp/audit/stats` aggregated audit metrics |

322
Memory.md Normal file

@@ -0,0 +1,322 @@
# Memory System
> **Source of truth:** `src/lib/memory/` and `src/app/api/memory/`
> **Last updated:** 2026-05-13 — v3.8.0
OmniRoute provides persistent conversational memory keyed by API key (and
optionally session id). Memories are extracted automatically from LLM responses
via lightweight regex pattern matching and injected back into subsequent
requests as a leading system message (or first user message for providers that
reject the system role).
Memory is **scoped per API key**, not per user — every request authenticated
with the same API key shares the same memory pool, with optional further
scoping by `sessionId`.
## Architecture
```
Client → /v1/chat/completions (apiKeyInfo resolved upstream)
→ handleChatCore() [open-sse/handlers/chatCore.ts]
→ resolveMemoryOwnerId(apiKeyInfo) # extracts id
→ getMemorySettings() # cached settings
→ shouldInjectMemory(body, {enabled}) # gate
→ retrieveMemories(apiKeyId, config) # SQL + optional FTS5
→ injectMemory(body, memories, provider) # system or user message
→ upstream provider call
→ on response: extractFacts(text, apiKeyId, sessionId) # non-blocking
→ setImmediate → createMemory(fact) per match
```
The injection and extraction call-sites are wired in
`open-sse/handlers/chatCore.ts` (look for `retrieveMemories`, `injectMemory`,
and `extractFacts`).
## Storage Layers
### Primary: SQLite (`memories` table)
Created by migration `015_create_memories.sql`:
| Column | Type | Notes |
| --------------------------- | ------------------ | -------------------------------------------------------------------- |
| `id` | `TEXT PRIMARY KEY` | UUID generated via `crypto.randomUUID()` |
| `api_key_id` | `TEXT NOT NULL` | Owning API key |
| `session_id` | `TEXT` | Optional per-conversation scope |
| `type` | `TEXT NOT NULL` | One of `factual`, `episodic`, `procedural`, `semantic` |
| `key` | `TEXT` | Stable upsert key, e.g. `preference:i_prefer_python` |
| `content` | `TEXT NOT NULL` | The actual fact text |
| `metadata` | `TEXT` | JSON blob (category, extractedAt, source, ...) |
| `created_at` / `updated_at` | `TEXT` | ISO 8601 strings |
| `expires_at` | `TEXT` | Optional expiry; `NULL` means permanent |
| `memory_id` | `INTEGER UNIQUE` | Added by `023_fix_memory_fts_uuid.sql` to bridge UUIDs ↔ FTS5 rowids |
Indexes: `api_key_id`, `session_id`, `type`, `expires_at`, plus the unique
`memory_id` index.
**Upsert semantics**: `createMemory()` looks for an existing row with the same
`(api_key_id, key)` and updates it in place when found (merging `metadata` via
shallow spread). This keeps the table from growing unbounded for repeated
preference statements.
### Full-text Search (`memory_fts` virtual table)
`022_add_memory_fts5.sql` creates an FTS5 virtual table over `content` and
`key`. `023_fix_memory_fts_uuid.sql` fixes a real-world bug where the UUID
primary key did not join to FTS5's integer rowid — the migration adds the
`memory_id` column, recreates the FTS table, and wires triggers
(`memory_fts_ai`, `memory_fts_ad`, `memory_fts_au`) that keep FTS in sync on
INSERT, DELETE, and UPDATE.
Used by `retrieval.ts` for the `semantic` and `hybrid` strategies (see below).
The retrieval code guards with `hasTable("memory_fts")` and falls back to
chronological order if the FTS table is missing or the FTS query throws.
### Optional: Qdrant (vector store)
`src/lib/memory/qdrant.ts` implements an optional Qdrant integration for true
semantic memory:
- `upsertSemanticMemoryPoint()` — embed `key + content` with the configured
embedding model, ensure the collection exists (creates cosine-distance
vectors on first use), and upsert a point with payload `{memoryId,
apiKeyId, sessionId, key, content, metadata, createdAtUnix, expiresAtUnix}`.
- `searchSemanticMemory(query, topK, scope)` — embed the query, search the
collection filtered by `kind = "omniroute_memory"` and optionally by
`apiKeyId` / `sessionId`. Caps `topK` to `[1, 20]`.
- `deleteSemanticMemoryPoint(id)` — single point delete.
- `cleanupSemanticMemoryPoints({retentionDays})` — bulk delete points whose
`expiresAtUnix` is in the past or whose `createdAtUnix` is older than the
retention cutoff. Counts first so the dashboard can show actual numbers.
- `checkQdrantHealth()` — `GET /readyz` health probe with latency.
> **TODO**: The chat pipeline (`chatCore.ts`) and the in-tree `retrieveMemories()`
> implementation do not currently call `upsertSemanticMemoryPoint` or
> `searchSemanticMemory`. The Qdrant integration is feature-flagged via
> `qdrantEnabled` in settings, but at the time of writing the
> `searchSemanticMemory` results are not fused into retrieval — the
> `semantic`/`hybrid` retrieval strategies use SQLite FTS5 only. The settings UI
> in `dashboard/settings → MemorySkillsTab` exposes Qdrant config, health,
> search test, and cleanup, but the corresponding `/api/settings/qdrant`,
> `/api/settings/qdrant/health`, `/api/settings/qdrant/search`, and
> `/api/settings/qdrant/cleanup` routes are referenced from the UI but **not
> present** under `src/app/api/settings/qdrant/` (only `embedding-models/` is
> wired). Treat Qdrant as preview/optional plumbing.
## Memory Types
`MemoryType` (`src/lib/memory/types.ts`):
| Type | Used for |
| ------------ | ------------------------------------------------------------ |
| `factual` | Preferences, stable user facts, behavioral patterns |
| `episodic` | Decisions tied to a specific moment ("I chose Postgres") |
| `procedural` | Workflow / how-to memory (reserved; no auto-extractor today) |
| `semantic` | Reserved for vector-store entries |
`MemoryConfig` retrieval strategy is one of `exact`, `semantic`, or `hybrid`,
and scope is one of `session`, `apiKey`, or `global`. The default scope from
`getMemorySettings()` is `apiKey`.
## Fact Extraction (`extraction.ts`)
Extraction is **regex-based**, not LLM-based — it runs in-process with
`setImmediate()` so it never blocks the response stream:
- **Preference patterns** → `MemoryType.FACTUAL`
(e.g. `I prefer …`, `I really like …`, `my favorite is …`, `I hate …`)
- **Decision patterns** → `MemoryType.EPISODIC`
(e.g. `I'll use …`, `I chose …`, `I went with …`, `I'm going to adopt …`)
- **Pattern patterns** → `MemoryType.FACTUAL`
(e.g. `I usually …`, `I always …`, `I tend to …`)
Each match is sanitised (`trim`, whitespace-collapse, capped at 500 chars),
deduplicated within the batch via a stable `factKey(category, content)`, and
stored via `createMemory()` with metadata
`{category, extractedAt, source: "llm_response"}`. Input text is capped at
64 KiB (`MAX_EXTRACTION_TEXT_LENGTH`) — when longer, the **tail** of the text
is used so the most recent assistant content always participates.
`extractFactsFromText(text)` is exported for tests and returns the structured
facts without storing them.
## Retrieval (`retrieval.ts`)
`retrieveMemories(apiKeyId, config)` is the main entry point. It:
1. Normalises and validates the config through `MemoryConfigSchema`.
2. Returns `[]` immediately when `enabled` is false or `maxTokens <= 0`.
3. Clamps `maxTokens` to `[1, 8000]`.
4. Detects whether the modern `memories` table exists (vs the legacy `memory`
table) so older databases keep working.
5. Builds the base query with expiry guard
(`expires_at IS NULL OR datetime(expires_at) > datetime('now')`), optional
session scope, and optional `retentionDays` cutoff.
6. Branches on strategy:
- **`exact`** (default): chronological `ORDER BY created_at DESC LIMIT 100`.
- **`semantic`**: if `config.query` and `memory_fts` exists, JOIN
`memory_fts MATCH ?` and order by FTS rank; fall back to chronological
when FTS returns 0 rows.
- **`hybrid`**: union of FTS results (higher relevance) and the
chronological set, deduplicated by id.
7. Computes a keyword relevance score (`getRelevanceScore`) over
`content`, `key`, and `metadata` JSON when a query is provided. Rows with
zero score are filtered out.
8. Sorts by score desc, then `createdAt` desc.
9. Walks the ranked list and accepts entries while a running
`estimateTokens(content)` (≈ `length / 4`) stays under the budget. Always
returns at least one entry when any matched.
`estimateTokens` is exported and used by retrieval, summarisation, and the MCP
`omniroute_memory_search` tool.
## Injection (`injection.ts`)
`injectMemory(request, memories, provider)`:
1. Joins all memory contents into a single `Memory context: …` string.
2. Picks a strategy by provider name:
- **System message** (default for OpenAI, Anthropic, Gemini, …) — prepends
a `{role: "system", content: memoryText}` ahead of any existing system
messages so user system prompts still take precedence.
- **User message** (fallback) — for providers in
`PROVIDERS_WITHOUT_SYSTEM_MESSAGE`: `o1`, `o1-mini`, `o1-preview`,
`glm`, `glmt`, `glm-cn`, `zai`, `qianfan`. These reject the system role
and would 400 otherwise (cf. issue #1701 for GLM/Zhipu).
3. Logs the count, strategy, and model under `memory.injection.injected`.
`providerSupportsSystemMessage(provider)` is exported for callers that need to
make routing decisions of their own. Unknown providers default to `true`
(system role allowed) for safety.
## Settings (`settings.ts`)
Memory configuration is **stored in the DB settings table**, not in env vars.
`getMemorySettings()` reads from `getSettings()` and caches the result
in-process; `invalidateMemorySettingsCache()` is called by the settings PUT
route after writes.
| DB key | Type | Default | UI control |
| --------------------- | ------- | -------------------------------------------------- | ----------------------------------------------- |
| `memoryEnabled` | boolean | `true` | Memory on/off |
| `memoryMaxTokens` | integer | `2000` (range `0–16000`) | Token budget for injection |
| `memoryRetentionDays` | integer | `30` (range `1–365`) | Retention window |
| `memoryStrategy` | enum | `"hybrid"` (one of `recent`, `semantic`, `hybrid`) | Retrieval strategy |
| `skillsEnabled` | boolean | `false` | Toggles per-key skill injection (see SKILLS.md) |
Note: the UI strategy `"recent"` maps to the internal `"exact"` retrieval
strategy via `toMemoryRetrievalConfig()` (chronological order).
Qdrant-related DB keys (`qdrantEnabled`, `qdrantHost`, `qdrantPort`,
`qdrantApiKey`, `qdrantCollection` default `"omniroute_memory"`,
`qdrantEmbeddingModel` default `"openai/text-embedding-3-small"`) are read by
`normalizeQdrantConfig()` in `qdrant.ts`.
No `MEMORY_*` or `QDRANT_*` env vars exist today — everything is per-instance
DB settings. `OMNIROUTE_MEMORY_MB` (commented out in `.env.example`) is
unrelated and refers to Node heap sizing.
## Summarisation (`summarization.ts`)
`summarizeMemories(apiKeyId, sessionId?, maxTokens = 4000)` compacts older
content when the running token total over a key's memories exceeds the
budget. It iterates rows DESC by `created_at`, keeps rows that fit, and for
the rest replaces `content` in place with the first three sentences of the
original. `tokensSaved` is the difference in `estimateTokens` between old and
new content.
This routine is **available but not called automatically** in the current
chat pipeline — call it from a cron, an admin action, or
`MemoryConfig.autoSummarize` glue if you need ongoing compaction. The data
loss is one-way: original text is overwritten.
## REST API
All endpoints require management auth (`requireManagementAuth`).
| Method | Path | Description |
| -------- | ---------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `GET` | `/api/memory` | Paginated list with filters: `apiKeyId`, `type`, `sessionId`, `q`, `limit`, `page`, `offset`. Response includes `stats.total` and `stats.byType` |
| `POST` | `/api/memory` | Create entry (Zod-validated: `content`, `key`, optional `type`, `sessionId`, `apiKeyId`, `metadata`, `expiresAt`). Calls `createMemory()` which upserts on `(apiKeyId, key)` |
| `GET` | `/api/memory/[id]` | Fetch a single entry by UUID |
| `DELETE` | `/api/memory/[id]` | Delete an entry; returns 404 when missing |
| `GET` | `/api/memory/health` | Runs `verifyExtractionPipeline("health-check")` — round-trip create→list→delete to confirm the store is alive. Returns `{working, latencyMs, error?}` |
| `GET` | `/api/settings/memory` | Current normalised `MemorySettings` |
| `PUT` | `/api/settings/memory` | Update one or more of `enabled`, `maxTokens`, `retentionDays`, `strategy`, `skillsEnabled` |
The `/api/memory` list query supports either `page`-based pagination
(`parsePaginationParams`) **or** raw `offset` — when `offset` is present it
takes precedence and a derived `page` is computed for the response shape.
## MCP Tools (`open-sse/mcp-server/tools/memoryTools.ts`)
When the MCP server is enabled, three memory tools are registered:
- `omniroute_memory_search` — `{apiKeyId, query?, type?, maxTokens?, limit?}`
→ wraps `retrieveMemories()` with `retrievalStrategy: "exact"`, optionally
filters by `type`, and reports `totalTokens`.
- `omniroute_memory_add` — `{apiKeyId, sessionId?, type, key, content,
metadata?}` → wraps `createMemory()`.
- `omniroute_memory_clear` — `{apiKeyId, type?, olderThan?}` → lists matching
entries, optionally filters by created-before timestamp, then deletes each
via `deleteMemory()`.
See [MCP-SERVER.md](./MCP-SERVER.md) for transport and scope details.
## Dashboard
`src/app/(dashboard)/dashboard/memory/page.tsx` provides:
- Real-time list, search, and pagination (debounced 300 ms).
- Type filter (`factual` / `episodic` / `procedural` / `semantic` / all).
- Add-memory modal (key, content, type).
- Delete per row.
- JSON export of the current page; JSON import via file picker.
- A green/red health dot driven by `GET /api/memory/health`.
- Stat cards: `totalEntries`, `tokensUsed`, `hitRate` (the latter two come
from the API stats payload).
Memory and Qdrant settings live under
`/dashboard/settings → Memory & Skills` (`MemorySkillsTab.tsx`).
## Caching
`src/lib/memory/store.ts` keeps an in-process LRU-ish cache
(`MEMORY_CACHE_TTL = 5 min`, `MEMORY_MAX_CACHE_SIZE = 10 000`, with 20 %
oldest eviction) for `getMemory(id)` reads, plus a generic key/value
`memoryCache` layer (`src/lib/memory/cache.ts`) with `get`/`set`/`invalidate`
methods used by callers that want their own scoped cache (1 000-entry LRU,
default TTL 5 min).
## Privacy & Lifecycle
- Memory ownership is the API key id (`resolveMemoryOwnerId` in
`chatCore.ts`). Without an `apiKeyInfo.id` neither retrieval nor injection
nor extraction runs.
- Entries with a future `expires_at` are filtered out of retrieval; old
entries beyond `retentionDays` are excluded by the
`created_at >= cutoff` clause in `retrieveMemories`.
- For hard deletion, use `DELETE /api/memory/[id]` or `omniroute_memory_clear`.
- Extraction is fire-and-forget via `setImmediate`; failures are logged under
`memory.extraction.background.failed` and never surface to the caller.
- Verification round-trips (`verifyExtractionPipeline`) clean up their own
test entries in a `finally` block.
## See Also
- [SKILLS.md](./SKILLS.md) — the `skillsEnabled` setting injects tool
definitions alongside memory.
- [MCP-SERVER.md](./MCP-SERVER.md) — MCP transport / scopes.
- [API_REFERENCE.md](../reference/API_REFERENCE.md) — broader API surface.
- Source modules:
- `src/lib/memory/types.ts`, `schemas.ts`
- `src/lib/memory/store.ts`, `retrieval.ts`, `injection.ts`
- `src/lib/memory/extraction.ts`, `summarization.ts`, `verify.ts`
- `src/lib/memory/settings.ts`, `qdrant.ts`, `cache.ts`
- `src/lib/db/migrations/015_create_memories.sql`,
`022_add_memory_fts5.sql`, `023_fix_memory_fts_uuid.sql`
- `src/app/api/memory/route.ts`, `[id]/route.ts`, `health/route.ts`
- `src/app/api/settings/memory/route.ts`
- `open-sse/handlers/chatCore.ts` (injection / extraction wiring)
- `open-sse/mcp-server/tools/memoryTools.ts`

62
OmniRoute-vs-Alternatives.md Normal file

@@ -0,0 +1,62 @@
# OmniRoute vs Alternatives
Objective feature comparison vs popular open-source AI routers.
> **Methodology**: Public repos audited 2026-Q2. Versions as listed.
> Submit corrections via PR — we want this to be accurate.
| Feature | OmniRoute 3.8 | LiteLLM 1.x | OpenRouter (SaaS) | Portkey |
| -------------------------------------------------- | :------------------------: | :------------: | :---------------: | :---------: |
| **Providers** | **207+** | ~100 | ~50 | ~30 |
| **Self-hostable** | ✅ | ✅ | ❌ | ⚠ paid |
| **OAuth providers (Claude, Codex, Copilot, etc.)** | **15+** | partial | ❌ | ❌ |
| **Auto-fallback combos** | **14 strategies** | priority-based | tier-based | weighted |
| **Tier 1/2/3 fallback (subscription→cheap→free)** | ✅ + UI | manual | n/a | manual |
| **Token compression** | RTK (47 filters) + Caveman | none | none | none |
| **Built-in MCP server** | ✅ 37 tools, 13 scopes | ❌ | ❌ | ❌ |
| **A2A protocol** | ✅ 5 skills | ❌ | ❌ | ❌ |
| **Memory (FTS5 + vector)** | ✅ | ❌ | ❌ | ❌ |
| **Guardrails (PII, injection, vision)** | ✅ | partial | ❌ | ✅ paid |
| **Cloud agent integrations** | Codex, Devin, Jules | ❌ | ❌ | ❌ |
| **Circuit breaker per provider** | ✅ 3-state, lazy recovery | basic | ❌ | ✅ |
| **TLS fingerprint stealth (JA3/JA4)** | ✅ wreq-js | ❌ | ❌ | ❌ |
| **Eval framework** | ✅ built-in | ❌ | ❌ | ⚠ paid |
| **MITM proxy (intercepts Cursor/Antigravity)** | ✅ cross-platform | ❌ | ❌ | ❌ |
| **CLI with system tray (no Electron)** | ✅ | ❌ | n/a | n/a |
| **CLI machine-ID auto-auth** | ✅ | ❌ | n/a | n/a |
| **Dashboard** | Next.js 16 | basic | proprietary | proprietary |
| **i18n** | **40+ locales** | ❌ | ❌ | ⚠ |
| **Public agent skills (SKILL.md)** | ✅ 10 | ❌ | ❌ | ❌ |
| **Tunnel support (Cloudflared, Tailscale, Ngrok)** | ✅ | ❌ | n/a | n/a |
| **License** | MIT | MIT | proprietary | proprietary |
## When to choose OmniRoute
- You self-host and want **maximum provider coverage** (207+)
- You need a **built-in MCP server** (LLM tools, memory, skills exposed as tools)
- You need **A2A protocol** for agent-to-agent workflows
- You want **fingerprint stealth** (JA3/JA4) to avoid detection by upstream CAPTCHAs
- You need **enterprise features** (guardrails, evals, audit trail) without a SaaS bill
## When to choose LiteLLM
- You're **Python-first** and need tight integration with `litellm.completion()`
- You need **mature production deployment recipes** (k8s, Helm charts)
- Your team already runs Python microservices
## When to choose OpenRouter (SaaS)
- You don't want to self-host
- You're fine paying per-token at SaaS markup
- You need a **single payment method** across all providers
## When to choose Portkey
- You need a **commercial SLA** with uptime guarantees
- You prefer a **managed dashboard** without ops overhead
- You need **enterprise compliance** features out of the box
---
_Last updated: 2026-05-15. Submit corrections via PR to keep this table accurate._

160
OpenCode.md Normal file

@@ -0,0 +1,160 @@
# OpenCode Integration
> **Status:** Generally available.
> **Audience:** Operators wiring OpenCode to an OmniRoute deployment.
> **Source of truth (config schema):** `src/shared/services/opencodeConfig.ts`
> **Source of truth (npm package):** `@omniroute/opencode-provider/` (publishable workspace)
[OpenCode](https://opencode.ai) is an agentic CLI/desktop AI client. It reads its provider catalog from `~/.config/opencode/opencode.json` (or `opencode.jsonc`) and follows the schema at `https://opencode.ai/config.json`. OmniRoute exposes itself to OpenCode as one of those providers — every request flows through OmniRoute's standard OpenAI-compatible `/v1` surface, so OpenCode automatically benefits from Auto-Combo routing, circuit breakers, key policies, observability, etc.
There are **two supported integration paths**. Pick one — they generate the same config.
---
## Path 1 — CLI generator (no npm install)
Recommended for end users. Ships with OmniRoute. Writes `opencode.json` in place.
```bash
# After installing OmniRoute (npm i -g @omniroute/cli or local clone)
omniroute config opencode \
--baseUrl http://localhost:20128 \
--apiKey "$OMNIROUTE_API_KEY"
```
Behind the scenes the CLI calls `mergeOpenCodeConfigText()` (`src/shared/services/opencodeConfig.ts:104`), so an existing `opencode.json` keeps its other providers and comments. The OmniRoute entry is added/replaced atomically.
Resulting file (default model catalog):
```jsonc
{
"$schema": "https://opencode.ai/config.json",
"provider": {
"omniroute": {
"npm": "@ai-sdk/openai-compatible",
"name": "OmniRoute",
"options": {
"baseURL": "http://localhost:20128/v1",
"apiKey": "<your-key>",
},
"models": {
"claude-opus-4-5-thinking": { "name": "claude-opus-4-5-thinking" },
"claude-sonnet-4-5-thinking": { "name": "claude-sonnet-4-5-thinking" },
"gemini-3.1-pro-high": { "name": "gemini-3.1-pro-high" },
"gemini-3-flash": { "name": "gemini-3-flash" },
},
},
},
}
```
---
## Path 2 — npm package `@omniroute/opencode-provider`
Recommended when you're scripting the config from Node/TS (CI pipelines, monorepos, custom installer flows).
```bash
npm install --save-dev @omniroute/opencode-provider
```
```ts
import { writeFileSync } from "node:fs";
import { buildOmniRouteOpenCodeConfig } from "@omniroute/opencode-provider";
const config = buildOmniRouteOpenCodeConfig({
baseURL: "http://localhost:20128",
apiKey: process.env.OMNIROUTE_API_KEY ?? "sk_omniroute",
// Optional: override the model catalog exposed to OpenCode
models: ["auto", "claude-opus-4-7", "gpt-5.5"],
modelLabels: { auto: "Auto-Combo" },
});
writeFileSync("opencode.json", JSON.stringify(config, null, 2));
```
For a non-destructive merge against an existing file, replicate `mergeOpenCodeConfigText()` from `opencodeConfig.ts` or call the CLI generator.
See the [package README](../../@omniroute/opencode-provider/README.md) for the full API.
---
## What the runtime actually does
Both paths produce the same `provider.omniroute.npm: "@ai-sdk/openai-compatible"`. At runtime, OpenCode loads `@ai-sdk/openai-compatible` (already a transitive dependency of OpenCode) and configures it with `baseURL` + `apiKey`. From there:
```
OpenCode UI/agent
→ @ai-sdk/openai-compatible
→ HTTP POST {baseURL}/chat/completions (OmniRoute OpenAI surface)
→ OmniRoute /v1/chat/completions handler (open-sse/handlers/chatCore.ts)
→ combo routing / Auto-Combo / executor
→ upstream provider
```
The plugin never touches HTTP. It only emits configuration.
---
## Model catalog defaults
```ts
export const OMNIROUTE_DEFAULT_OPENCODE_MODELS = [
"claude-opus-4-5-thinking",
"claude-sonnet-4-5-thinking",
"gemini-3.1-pro-high",
"gemini-3-flash",
] as const;
```
You can override via `models: [...]`. Recommended additions:
- `"auto"` — surfaces OmniRoute's [Auto-Combo](../routing/AUTO-COMBO.md) zero-config router. Lets OpenCode pick "the best available model" without you hard-coding the catalog.
- `"<combo-name>"` — any combo you've defined in the dashboard; OmniRoute resolves it transparently.
---
## URL normalisation
The helper accepts both forms and emits exactly one `/v1`:
| Input | Output (`options.baseURL`) |
| ------------------------------ | --------------------------- |
| `http://localhost:20128` | `http://localhost:20128/v1` |
| `http://localhost:20128/` | `http://localhost:20128/v1` |
| `http://localhost:20128/v1` | `http://localhost:20128/v1` |
| `http://localhost:20128/v1///` | `http://localhost:20128/v1` |
This deduplication is **the most common breakage** seen in older configs. If you have an `opencode.json` from before v3.8.0 that points at `/v1/v1/...`, re-run the generator or call `createOmniRouteProvider` again.
---
## Authentication modes
| OmniRoute setting | Recommended `apiKey` value |
| ------------------------------------------- | ----------------------------------------------------- |
| `REQUIRE_API_KEY=false` (default for local) | `sk_omniroute` (literal placeholder) |
| `REQUIRE_API_KEY=true` | A real per-user API key from Dashboard → API Manager. |
For Anthropic-style clients that send `x-api-key` + `anthropic-version`, OmniRoute's `extractApiKey` also honours the key from `x-api-key`. OpenCode uses the OpenAI surface, so it'll always send `Authorization: Bearer ${apiKey}` — no Anthropic special-case applies here.
---
## Troubleshooting
| Symptom | Cause | Fix |
| ---------------------------------------------------- | ------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------- |
| `404` on every request with URL containing `/v1/v1/` | Stale config from pre-v3.8 plugin that double-suffixed `/v1`. | Regenerate via Path 1 or 2. |
| `401 Invalid API key` | OmniRoute has `REQUIRE_API_KEY=true` and the key is unknown. | Create the key in the dashboard, or set `REQUIRE_API_KEY=false` (local only) and use `sk_omniroute`. |
| Model list empty in OpenCode UI | All 4 default models are hidden in OmniRoute's provider visibility. | Pass `models: ["auto", ...]` to surface ones you've enabled. |
| OpenCode 500 with `cannot read property 'models'` | Older OpenCode (< 0.1.x) didn't accept inline `models`. | Upgrade OpenCode to a version that follows the v1 schema (`opencode.ai/config.json`). |
---
## See also
- [API reference](../reference/API_REFERENCE.md) — full OmniRoute REST surface
- [Auto-Combo](../routing/AUTO-COMBO.md) — what `model: "auto"` means
- [`@omniroute/opencode-provider` README](../../@omniroute/opencode-provider/README.md)
- Source: `src/shared/services/opencodeConfig.ts`, `src/lib/cli-helper/config-generator/opencode.ts`, `@omniroute/opencode-provider/src/index.ts`

187
PWA-Guide.md Normal file

@@ -0,0 +1,187 @@
# Progressive Web App (PWA) Guide
OmniRoute ships as a fully installable Progressive Web App. When you access the dashboard from any mobile browser — Android (Chrome) or iOS (Safari) — you can "Add to Home Screen" and get a native app-like experience with no app store required.
## What Is a PWA?
A Progressive Web App turns the OmniRoute web dashboard into something that looks and feels like a native mobile app. Once installed, it:
- Launches from your home screen with its own icon
- Opens fullscreen — no browser address bar or tab UI
- Works offline with a dedicated connectivity page
- Caches static assets for faster loading
- Supports both portrait and landscape orientations
## Installation
### Android (Chrome)
1. Open the OmniRoute dashboard in Chrome: `http://YOUR_IP:20128`
2. Chrome will show an **"Add OmniRoute to Home screen"** banner automatically, or:
- Tap the **⋮** menu (three dots) → **"Add to Home screen"** or **"Install app"**
3. Confirm the prompt
4. OmniRoute appears on your home screen as a standalone app
### iOS (Safari)
1. Open the OmniRoute dashboard in Safari: `http://YOUR_IP:20128`
2. Tap the **Share** button (box with arrow)
3. Scroll down and tap **"Add to Home Screen"**
4. Name it (defaults to "OmniRoute") and tap **Add**
5. OmniRoute appears on your home screen with the app icon
### Desktop (Chrome / Edge)
1. Open the OmniRoute dashboard
2. Click the **install icon** in the address bar (or ⋮ → "Install OmniRoute...")
3. Confirm the prompt
4. OmniRoute opens as a standalone window — no tabs, no address bar
## Features
### Fullscreen Experience
The manifest is configured with `display: "fullscreen"`, which means the installed app uses the entire screen — no browser chrome, no status bar overlap. This makes the dashboard feel truly native.
### Offline Support
OmniRoute includes a service worker (`sw.js`) that provides intelligent caching:
| Asset Type | Strategy | Behavior |
| ------------------------------------------------------- | ---------------------------------- | ---------------------------------------------------------------------------- |
| **App Shell** | Cache-first | `/`, `/offline`, manifest, and icons are pre-cached on install |
| **Static assets** (CSS, JS, images, fonts) | Network-first with cache fallback | Fetches fresh from the network; falls back to cache if offline |
| **Next.js bundles** (`/_next/`) | Network-first with cache update | Fetches from network and updates cache; serves cached version if offline |
| **Navigation requests** | Network-only with offline fallback | Always fetches from network; shows `/offline` page if network is unavailable |
| **API routes** (`/api/`, `/a2a`, `/dashboard/endpoint`) | Bypass (never cached) | Always goes directly to the server — never intercepted by the service worker |
### Offline Page
When the network is unavailable and a user navigates to a new page, the service worker serves a dedicated `/offline` page that:
- Displays a clear **"Connectivity Issue"** message
- Shows a live **online/offline status indicator** that updates in real time
- Provides a **"Retry Connection"** button to reload when connectivity returns
- Links to the **Status Page** for diagnostics
### App Icons
OmniRoute provides icons optimized for each platform:
| File | Size | Used By |
| ---------------------- | ---------------- | ------------------------------------- |
| `icon-512.png` | 512×512 | Android install prompt, splash screen |
| `apple-touch-icon.png` | 180×180 | iOS home screen icon |
| `icon-192.svg` | 192×192 (vector) | Android adaptive icon |
| `apple-touch-icon.svg` | 180×180 (vector) | Apple fallback |
| `favicon.svg` | Vector | Browser tabs |
| `favicon.ico` | Multi-size | Legacy browsers |
### Automatic Registration
The service worker is registered automatically via the `<PwaRegister />` component in the root layout. No user action is needed — the app becomes installable as soon as the browser detects the valid manifest and service worker.
## Technical Architecture
### Web App Manifest (`manifest.webmanifest`)
Generated by Next.js via `src/app/manifest.ts`:
```json
{
"name": "OmniRoute",
"short_name": "OmniRoute",
"description": "OmniRoute is an AI gateway for multi-provider LLMs. One endpoint for all your AI providers.",
"start_url": "/",
"scope": "/",
"display": "fullscreen",
"orientation": "any",
"background_color": "#0b0f1a",
"theme_color": "#0b0f1a",
"icons": [
{ "src": "/icon-512.png", "sizes": "512x512", "type": "image/png", "purpose": "any maskable" },
{ "src": "/apple-touch-icon.png", "sizes": "180x180", "type": "image/png" }
]
}
```
### Service Worker (`public/sw.js`)
A vanilla service worker (no framework dependencies) with:
- **Install phase**: Pre-caches the app shell (root, offline page, manifest, icons)
- **Activate phase**: Cleans up old cache versions and claims all clients
- **Fetch phase**: Intelligent routing based on request type (navigation, static asset, API)
- **Cache versioning**: `omniroute-pwa-v2` — bump this to force a fresh cache on update
### Layout Metadata (`src/app/layout.tsx`)
The root layout provides all the meta tags required for PWA compliance:
- `manifest` link to `/manifest.webmanifest`
- `apple-web-app-capable: true` for iOS standalone mode
- `apple-web-app-status-bar-style: black-translucent`
- `mobile-web-app-capable: yes` for Android Chrome
- `theme-color: #0b0f1a`
- `viewport-fit: cover` for edge-to-edge rendering
### Component: `PwaRegister`
Located at `src/shared/components/PwaRegister.tsx`, this client component:
1. Runs on mount (client-side only)
2. Checks for `serviceWorker` support in the browser
3. Registers `/sw.js` silently (errors are swallowed to avoid blocking the app)
4. Renders nothing (`return null`) — it's a side-effect-only component
## Use With Termux (Android)
When running OmniRoute on Android via Termux, the PWA works seamlessly:
1. Start OmniRoute in Termux: `npx omniroute`
2. Open Chrome on the same phone: `http://localhost:20128`
3. Install the PWA via "Add to Home Screen"
4. The PWA connects to the local Termux server — everything runs on-device
This combination means your Android phone is both the **server** (Termux) and the **client** (PWA) — a complete self-contained AI gateway.
## Use From Other Devices
Install the PWA on any device that has browser access to your OmniRoute server:
- **Another phone/tablet**: Navigate to `http://PHONE_IP:20128` and install the PWA
- **Laptop**: Open Chrome/Edge and install it as a desktop PWA
- **Smart TV with browser**: Access the dashboard fullscreen
## Customization
### Instance Name
The PWA title respects the **Instance Name** setting from `Dashboard → Settings`. If you rename your instance to "My AI Gateway", the installed PWA will show that name.
### Custom Favicon
If you upload a custom favicon via `Dashboard → Settings`, the PWA icon on desktop will reflect the custom icon. Mobile home screen icons use the pre-built `icon-512.png` and `apple-touch-icon.png` files.
## Limitations
- **No push notifications** — The service worker does not implement the Push API. Notifications are handled by the Electron app instead.
- **No background sync** — Offline actions are not queued for replay. The PWA is primarily a dashboard viewer.
- **iOS restrictions** — Safari on iOS does not support all PWA features (e.g., install prompts are manual, and background service workers are limited).
- **Cache size** — The service worker caches static assets only. Large response payloads from `/api/` routes are never cached.
- **Custom icons on mobile** — Changing the favicon in settings does not update the home screen icon on mobile (this requires regenerating the PWA icons).
## Files Reference
| File | Purpose |
| --------------------------------------- | ---------------------------------------------------------------- |
| `src/app/manifest.ts` | Next.js manifest route (generates `manifest.webmanifest`) |
| `public/sw.js` | Service worker with caching logic |
| `src/shared/components/PwaRegister.tsx` | Client component that registers the service worker |
| `src/app/offline/page.tsx` | Offline fallback page with live status indicator |
| `src/app/layout.tsx` | Root layout with PWA metadata (apple-web-app, theme-color, etc.) |
| `public/icon-512.png` | 512×512 PNG icon (Android, splash screen) |
| `public/apple-touch-icon.png` | 180×180 PNG icon (iOS home screen) |
| `public/icon-192.svg` | 192×192 SVG icon (Android adaptive) |
| `public/apple-touch-icon.svg` | 180×180 SVG icon (Apple fallback) |

268
Provider-Reference.md Normal file

@@ -0,0 +1,268 @@
# Provider Reference
> **Auto-generated** from `src/shared/constants/providers.ts` — do not edit by hand.
> Regenerate with: `npm run gen:provider-reference`
> **Last generated:** 2026-05-17
Total providers: **177**. See category breakdown below.
## Categories
- **Free** — free tier with API key (configured via dashboard)
- **OAuth** — sign-in flow handled by OmniRoute, no API key needed
- **Web cookie** — wraps the provider's web app via cookie auth
- **API key** — paid provider configured via API key (free credits may apply)
- **Local** — runs on the user's machine (Ollama, LM Studio, vLLM, etc.)
- **Search** — web search providers
- **Audio** — audio-only providers (TTS/STT)
- **Upstream proxy** — providers that proxy to other providers
- **Cloud agent** — long-running coding agents (Codex Cloud, Devin, Jules)
- **System** — OmniRoute-internal providers (loopback, etc.)
Additional tags: `image`, `video`, `aggregator`, `enterprise`, `embed/rerank`, `self-hosted`.
Use the dashboard at `/dashboard/providers` to enable, configure, and test each provider.
---
## Free Tier (OAuth-first or no-key) (5)
| ID | Alias | Name | Tags | Website | Notes |
| ------------ | ------------ | ---------- | ---- | ------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `amazon-q` | `aq` | Amazon Q | Free | [link](https://aws.amazon.com/q/developer/) | Uses the same AWS Builder ID or imported refresh-token flow as Kiro, but keeps Amazon Q connections separate. |
| `gemini-cli` | `gemini-cli` | Gemini CLI | Free | — | Uses Gemini CLI OAuth / Cloud Code credentials. Pro models require an eligible Google account or paid plan. |
| `kiro` | `kr` | Kiro AI | Free | — | — |
| `qoder` | `if` | Qoder AI | Free | — | — |
| `qwen` | `qw` | Qwen Code | Free | — | ⚠️ **DEPRECATED.** Qwen OAuth free tier was discontinued on 2026-04-15. Use 'bailian-coding-plan', 'alibaba', 'alibaba-cn', or 'openrouter' provider with API key instead. |
## OAuth Providers (11)
| ID | Alias | Name | Tags | Website | Notes |
| ------------- | ------------ | -------------------- | ----- | ------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `antigravity` | — | Antigravity | OAuth | — | — |
| `claude` | `cc` | Claude Code | OAuth | — | — |
| `cline` | `cl` | Cline | OAuth | — | — |
| `codex` | `cx` | OpenAI Codex | OAuth | — | — |
| `cursor` | `cu` | Cursor IDE | OAuth | — | — |
| `devin-cli` | `dv` | Devin CLI (Official) | OAuth | [link](https://cli.devin.ai) | Requires the Devin CLI binary. Run `devin auth login` to authenticate, or provide your WINDSURF_API_KEY. Install: https://cli.devin.ai |
| `github` | `gh` | GitHub Copilot | OAuth | — | — |
| `gitlab-duo` | `gitlab-duo` | GitLab Duo | OAuth | [link](https://docs.gitlab.com/user/duo_agent_platform/code_suggestions/) | OAuth application with ai_features + read_user scopes. Configure GITLAB_DUO_OAUTH_CLIENT_ID and optionally GITLAB_DUO_OAUTH_CLIENT_SECRET on this OmniRoute instance. |
| `kilocode` | `kc` | Kilo Code | OAuth | — | — |
| `kimi-coding` | `kmc` | Kimi Coding | OAuth | — | — |
| `windsurf` | `ws` | Windsurf (Devin CLI) | OAuth | [link](https://windsurf.com) | Sign in at windsurf.com to get your token. Visit windsurf.com/show-auth-token after logging in and paste it here, or use the device-code login flow. |
## Web Cookie Providers (7)
| ID | Alias | Name | Tags | Website | Notes |
| ---------------- | ---------- | --------------------------- | ---------- | --------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `blackbox-web` | `bb-web` | Blackbox Web (Subscription) | Web cookie | [link](https://app.blackbox.ai) | Paste your \_\_Secure-authjs.session-token value or full cookie header from app.blackbox.ai |
| `chatgpt-web` | `cgpt-web` | ChatGPT Web (Plus/Pro) | Web cookie | [link](https://chatgpt.com) | Paste your \_\_Secure-next-auth.session-token cookie value from chatgpt.com |
| `deepseek-web` | `ds-web` | DeepSeek Web | Web cookie | [link](https://chat.deepseek.com) | Paste your ds_session_id cookie from chat.deepseek.com |
| `grok-web` | `gw` | Grok Web (Subscription) | Web cookie | [link](https://grok.com) | Paste your sso= cookie value from grok.com |
| `muse-spark-web` | `ms-web` | Muse Spark Web (Meta AI) | Web cookie | [link](https://www.meta.ai) | Paste your abra_sess value or full cookie header from meta.ai |
| `perplexity-web` | `pplx-web` | Perplexity Web (Pro/Max) | Web cookie | [link](https://www.perplexity.ai) | Paste your \_\_Secure-next-auth.session-token cookie value from perplexity.ai |
| `t3-web` | `t3chat` | t3.chat (Pro/Free) | Web cookie | [link](https://t3.chat) | Pro: $8/mo, 50+ models. Free tier: limited models. Requires Cookie header + convex-session-id from DevTools. **Skeleton — endpoint URL not yet confirmed (TODO post-devtools-capture).** |
## API Key Providers (paid / paid-with-free-credits) (122)
| ID | Alias | Name | Tags | Website | Notes |
| --------------------- | -------------- | -------------------------- | --------------------- | -------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `agentrouter` | `agentrouter` | AgentRouter | API key, aggregator | [link](https://agentrouter.org) | $200 free credits on signup - multi-model routing gateway |
| `ai21` | `ai21` | AI21 Labs | API key | [link](https://www.ai21.com) | $10 trial credits on signup (valid 3 months), no credit card required |
| `aimlapi` | `aiml` | AI/ML API | API key, aggregator | [link](https://aimlapi.com) | $0.025/day free credits — 200+ models (GPT-4o, Claude, Gemini, Llama) via single endpoint |
| `alibaba` | `ali` | Alibaba | API key | [link](https://dashscope-intl.aliyuncs.com) | — |
| `alibaba-cn` | `ali-cn` | Alibaba (China) | API key | [link](https://dashscope.aliyuncs.com) | — |
| `anthropic` | `anthropic` | Anthropic | API key | [link](https://platform.claude.com) | — |
| `azure-ai` | `azure-ai` | Azure AI Foundry | API key, enterprise | [link](https://learn.microsoft.com/azure/ai-foundry) | Use your Azure AI Foundry key. Base URL can be https://<resource>.services.ai.azure.com/openai/v1/ or https://<resource>.openai.azure.com/openai/v1/. |
| `azure-openai` | `azure` | Azure OpenAI | API key, enterprise | [link](https://azure.microsoft.com/products/ai-services/openai-service) | Use your Azure OpenAI API key. Base URL should be your resource endpoint, for example https://my-resource.openai.azure.com. |
| `bailian-coding-plan` | `bcp` | Alibaba Coding Plan | API key | [link](https://www.alibabacloud.com/help/en/model-studio/coding-plan) | — |
| `baseten` | `baseten` | Baseten | API key | [link](https://baseten.co) | $30 free trial credits for GPU inference |
| `bazaarlink` | `bzl` | BazaarLink | API key | [link](https://bazaarlink.ai) | Free tier with auto:free routing — zero-cost inference, no credit card required |
| `bedrock` | `bedrock` | Amazon Bedrock | API key, enterprise | [link](https://aws.amazon.com/bedrock) | Use your Amazon Bedrock API key in Authorization: Bearer <key>. OmniRoute defaults to the OpenAI-compatible bedrock-mantle endpoint in us-east-1; set a regional base URL if your account uses another region or the bedrock-runtime /openai/v1 path. |
| `black-forest-labs` | `bfl` | Black Forest Labs | API key, image | [link](https://blackforestlabs.ai) | — |
| `blackbox` | `bb` | Blackbox AI | API key | [link](https://blackbox.ai) | Free tier: unlimited basic chat plus Minimax-M2.5, no credit card required |
| `bytez` | `bytez` | Bytez | API key | [link](https://bytez.com) | $1 free credits, refreshes every 4 weeks |
| `cablyai` | `cablyai` | CablyAI | API key, aggregator | [link](https://cablyai.com) | Bearer API key for the CablyAI OpenAI-compatible gateway. |
| `cerebras` | `cerebras` | Cerebras | API key | [link](https://inference.cerebras.ai) | Free: 1M tokens/day, 60K TPM — world's fastest inference |
| `chutes` | `chutes` | Chutes.ai | API key, aggregator | [link](https://chutes.ai) | Bearer API key for the Chutes OpenAI-compatible gateway. |
| `clarifai` | `clarifai` | Clarifai | API key, enterprise | [link](https://docs.clarifai.com) | Use your Clarifai PAT or app-specific API key. OmniRoute targets the OpenAI-compatible endpoint at https://api.clarifai.com/v2/ext/openai/v1 and authenticates with Authorization: Key <token>. |
| `cloudflare-ai` | `cf` | Cloudflare Workers AI | API key | [link](https://developers.cloudflare.com/workers-ai) | Requires API Token AND Account ID (found at dash.cloudflare.com) |
| `codestral` | `codestral` | Codestral | API key | [link](https://mistral.ai) | — |
| `cohere` | `cohere` | Cohere | API key | [link](https://cohere.com) | Free Trial: 1,000 API calls/month for testing, no credit card required |
| `command-code` | `cmd` | Command Code | API key | [link](https://commandcode.ai/) | Use a Command Code API key. Requests are sent to Command Code's /alpha/generate endpoint. |
| `completions` | `cpl` | Completions.me | API key | [link](https://completions.me) | Free unlimited access to Claude, GPT, Gemini — no credit card, no rate limits |
| `crof` | `crof` | CrofAI | API key | [link](https://crof.ai) | — |
| `databricks` | `databricks` | Databricks | API key, enterprise | [link](https://www.databricks.com) | — |
| `datarobot` | `datarobot` | DataRobot | API key, enterprise | [link](https://docs.datarobot.com) | Use your DataRobot API token. Optional Base URL can be the account root (for LLM Gateway) or a deployment URL under /api/v2/deployments/<id>. |
| `deepinfra` | `deepinfra` | DeepInfra | API key | [link](https://deepinfra.com) | Free signup credits for API testing and model exploration |
| `deepseek` | `ds` | DeepSeek | API key | [link](https://platform.deepseek.com) | 5M free tokens on signup - no credit card required |
| `empower` | `empower` | Empower | API key, aggregator | [link](https://docs.empower.dev) | Bearer API key for the Empower OpenAI-compatible endpoint. |
| `enally` | `enly` | Enally AI | API key | [link](https://ai.enally.in) | Free for students and developers — no credit card, OTP verification |
| `fal-ai` | `fal` | Fal.ai | API key, image | [link](https://fal.ai) | — |
| `featherless-ai` | `featherless` | Featherless AI | API key | [link](https://featherless.ai) | — |
| `fenayai` | `fenayai` | FenayAI | API key, aggregator | [link](https://fenayai.com) | Bearer API key for the FenayAI OpenAI-compatible gateway. |
| `fireworks` | `fireworks` | Fireworks AI | API key | [link](https://fireworks.ai) | $1 free starter credits on signup for API testing |
| `freetheai` | `fta` | FreeTheAi | API key | [link](https://freetheai.xyz) | Community-run — free forever, no paid tiers, no credit card |
| `friendliai` | `friendli` | FriendliAI | API key | [link](https://friendli.ai) | — |
| `galadriel` | `galadriel` | Galadriel | API key | [link](https://galadriel.com) | — |
| `gemini` | `gemini` | Gemini (Google AI Studio) | API key | [link](https://aistudio.google.com) | Free forever: 1,500 req/day for Gemini 2.5 Flash — no credit card, get key at aistudio.google.com |
| `getgoapi` | `ggo` | GoAPI | API key, aggregator | [link](https://api.getgoapi.com) | — |
| `gigachat` | `gigachat` | GigaChat (Sber) | API key | [link](https://developers.sber.ru) | — |
| `gitlab` | `gitlab` | GitLab Duo PAT | API key | [link](https://docs.gitlab.com/user/duo_agent_platform/code_suggestions/) | GitLab personal access token for the public Code Suggestions API. Configure a self-hosted base URL when not using gitlab.com. |
| `glhf` | `glhf` | GLHF Chat | API key, aggregator | [link](https://glhf.chat) | Bearer API key for the GLHF OpenAI-compatible gateway. |
| `glm` | `glm` | GLM Coding | API key | [link](https://z.ai/subscribe) | — |
| `glm-cn` | `glmcn` | GLM Coding (China) | API key | [link](https://open.bigmodel.cn) | — |
| `glmt` | `glmt` | GLM Thinking | API key | [link](https://open.bigmodel.cn) | — |
| `groq` | `groq` | Groq | API key | [link](https://groq.com) | Free tier: 30 RPM / 14.4K RPD — no credit card |
| `heroku` | `heroku` | Heroku AI | API key, enterprise | [link](https://www.heroku.com) | — |
| `huggingface` | `hf` | HuggingFace | API key | [link](https://huggingface.co) | Free Inference API for thousands of models (Whisper, VITS, SDXL…) |
| `hyperbolic` | `hyp` | Hyperbolic | API key | [link](https://hyperbolic.xyz) | $1-5 trial credits on signup for serverless inference |
| `inference-net` | `inet` | Inference.net | API key | [link](https://inference.net) | $25 free credits on signup plus research grants available |
| `jina-ai` | `jina` | Jina AI | API key, embed/rerank | [link](https://jina.ai) | Bearer API key for the Jina AI rerank API. |
| `kie` | `kie` | KIE.AI | API key | [link](https://kie.ai) | — |
| `kilo-gateway` | `kg` | Kilo Gateway | API key, aggregator | [link](https://kilo.ai) | — |
| `kimi` | `kimi` | Kimi | API key | [link](https://platform.moonshot.ai) | — |
| `kimi-coding-apikey` | `kmca` | Kimi Coding (API Key) | API key | [link](https://www.kimi.com/code) | — |
| `kluster` | `kluster` | Kluster AI | API key | [link](https://kluster.ai) | $5 free credits on signup - DeepSeek R1, Llama 4 Maverick/Scout, Qwen3 235B |
| `lambda-ai` | `lambda` | Lambda AI | API key | [link](https://lambda.ai) | — |
| `laozhang` | `lz` | LaoZhang AI | API key, aggregator | [link](https://api.laozhang.ai) | — |
| `lepton` | `lepton` | Lepton AI | API key | [link](https://lepton.ai) | Free tier available - fast inference on custom hardware |
| `llamagate` | `llamagate` | LlamaGate | API key | [link](https://llamagate.ai) | — |
| `llm7` | `llm7` | LLM7.io | API key | [link](https://llm7.io) | No signup required - 2 req/s, 20 RPM, 100 req/hr free tier |
| `longcat` | `lc` | LongCat AI | API key | [link](https://longcat.chat/platform/docs) | 50M tokens/day (Flash-Lite) + 500K/day (Chat/Thinking) — 100% free while public beta |
| `maritalk` | `maritalk` | Maritalk | API key | [link](https://www.maritaca.ai) | — |
| `meta-llama` | `meta` | Meta Llama API | API key | [link](https://llama.developer.meta.com) | — |
| `minimax` | `minimax` | Minimax Coding | API key | [link](https://www.minimax.io) | — |
| `minimax-cn` | `minimax-cn` | Minimax (China) | API key | [link](https://www.minimaxi.com) | — |
| `mistral` | `mistral` | Mistral | API key | [link](https://mistral.ai) | Free Experiment tier: rate-limited access to all models, no credit card required |
| `modal` | `mdl` | Modal | API key, enterprise | [link](https://modal.com/docs) | Use the bearer token that protects your Modal deployment, if enabled. Base URL should point to your OpenAI-compatible Modal app, for example https://<workspace>--<app>.modal.run/v1. |
| `moonshot` | `moonshot` | Moonshot AI | API key | [link](https://platform.moonshot.ai) | — |
| `morph` | `morph` | Morph | API key | [link](https://morphllm.com) | Free tier: 250K credits/month, $0 |
| `nanobanana` | `nb` | NanoBanana | API key, image | [link](https://nanobananaapi.ai) | — |
| `nanogpt` | `nanogpt` | NanoGPT | API key | [link](https://nano-gpt.com) | — |
| `nebius` | `nebius` | Nebius AI | API key | [link](https://nebius.com) | ~$1 trial credits on signup for API testing |
| `nlpcloud` | `nlpc` | NLP Cloud | API key | [link](https://docs.nlpcloud.com) | Use your NLP Cloud API key in Authorization: Token <key>. OmniRoute targets the chatbot endpoint on https://api.nlpcloud.io/v1/gpu/<model>/chatbot by default. |
| `nous-research` | `nous` | Nous Research | API key | [link](https://portal.nousresearch.com/help) | Use your Nous Portal API key. OmniRoute targets the official OpenAI-compatible inference endpoint at https://inference-api.nousresearch.com/v1. |
| `novita` | `novita` | Novita AI | API key, aggregator | [link](https://novita.ai) | $0.50 trial credits on signup (valid about 1 year) |
| `nscale` | `nscale` | nScale | API key | [link](https://nscale.com) | $5 free credits on signup for inference testing |
| `nvidia` | `nvidia` | NVIDIA NIM | API key | [link](https://build.nvidia.com) | Free dev access: ~40 RPM, 70+ models (Kimi K2.5, GLM 4.7, DeepSeek V3.2...) |
| `oci` | `oci` | OCI Generative AI | API key, enterprise | [link](https://www.oracle.com/artificial-intelligence/generative-ai) | Use your OCI Generative AI API key or IAM bearer token. Base URL can be https://inference.generativeai.<region>.oci.oraclecloud.com/openai/v1/. |
| `ollama-cloud` | `ollamacloud` | Ollama Cloud | API key | [link](https://ollama.com/settings/api-keys) | — |
| `openai` | `openai` | OpenAI | API key | [link](https://platform.openai.com) | — |
| `opencode-go` | `opencode-go` | OpenCode Go | API key | [link](https://opencode.ai/go) | — |
| `opencode-zen` | `opencode-zen` | OpenCode Zen | API key | [link](https://opencode.ai/zen) | — |
| `openrouter` | `openrouter` | OpenRouter | API key, aggregator | [link](https://openrouter.ai) | Free models at $0/token with :free suffix - 20 RPM / 200 RPD |
| `ovhcloud` | `ovh` | OVHcloud AI | API key | [link](https://www.ovhcloud.com) | — |
| `perplexity` | `pplx` | Perplexity | API key | [link](https://www.perplexity.ai) | — |
| `petals` | `petals` | Petals | API key | [link](https://chat.petals.dev) | No API key is required for the public research endpoint. Leave the field blank, or provide a bearer token if your self-hosted Petals gateway uses auth. |
| `piapi` | `pi` | PiAPI | API key, aggregator | [link](https://piapi.ai) | — |
| `poe` | `poe` | Poe | API key, aggregator | [link](https://creator.poe.com/api-reference) | Bearer API key for the Poe OpenAI-compatible API. |
| `pollinations` | `pol` | Pollinations AI | API key | [link](https://pollinations.ai) | No API key required for free public endpoint. Optional Spore tier: ~0.01 pollen/hour. |
| `predibase` | `predibase` | Predibase | API key | [link](https://predibase.com) | $25 free trial credits (30-day validity) |
| `publicai` | `publicai` | PublicAI | API key | [link](https://publicai.co) | Free community inference tier for testing |
| `puter` | `pu` | Puter AI | API key | [link](https://puter.com) | Get token at puter.com/dashboard → Copy Auth Token |
| `qianfan` | `qianfan` | Baidu Qianfan | API key | [link](https://cloud.baidu.com/product/wenxinworkshop) | — |
| `recraft` | `recraft` | Recraft | API key, image | [link](https://recraft.ai) | — |
| `reka` | `reka` | Reka | API key | [link](https://docs.reka.ai/chat/overview) | Use your Reka API key. OmniRoute supports the OpenAI-compatible base URL https://api.reka.ai/v1 and sends both Authorization and X-Api-Key headers for compatibility. |
| `runwayml` | `runway` | Runway | API key, video | [link](https://docs.dev.runwayml.com) | Use your Runway API key in Authorization: Bearer <key>. OmniRoute targets the current Runway API at https://api.dev.runwayml.com/v1 and sends the required X-Runway-Version header automatically. |
| `sambanova` | `samba` | SambaNova | API key | [link](https://sambanova.ai) | $5 free credits on signup (30-day validity), no credit card required |
| `sap` | `sap` | SAP Generative AI Hub | API key, enterprise | [link](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/generative-ai-hub-in-sap-ai-core) | Use your SAP AI Core bearer token. Base URL can be your AI_API_URL root or a deploymentUrl from Generative AI Hub. |
| `scaleway` | `scw` | Scaleway AI | API key | [link](https://www.scaleway.com/en/ai/generative-apis) | 1M free tokens for new accounts — EU/GDPR compliant (Paris), Qwen3 235B & Llama 70B |
| `siliconflow` | `siliconflow` | SiliconFlow | API key | [link](https://cloud.siliconflow.com) | $1 free credits plus permanently free models after identity verification |
| `snowflake` | `snowflake` | Snowflake Cortex | API key, enterprise | [link](https://www.snowflake.com) | — |
| `stability-ai` | `stability` | Stability AI | API key, image | [link](https://stability.ai) | — |
| `synthetic` | `synthetic` | Synthetic | API key, aggregator | [link](https://synthetic.new) | — |
| `thebai` | `thebai` | TheB.AI | API key, aggregator | [link](https://theb.ai) | Bearer API key for the TheB.AI OpenAI-compatible gateway. |
| `together` | `together` | Together AI | API key | [link](https://www.together.ai) | $25 signup credits + 3 permanently free models: Llama 3.3 70B, Vision, DeepSeek-R1 distill |
| `topaz` | `topaz` | Topaz | API key, image | [link](https://topazlabs.com) | — |
| `uncloseai` | `unc` | UncloseAI | API key | [link](https://uncloseai.com) | No auth required. API accepts any non-empty string as key for identification. |
| `upstage` | `upstage` | Upstage | API key | [link](https://www.upstage.ai) | — |
| `v0-vercel` | `v0` | v0 (Vercel) | API key | [link](https://v0.dev) | — |
| `venice` | `venice` | Venice.ai | API key | [link](https://venice.ai) | — |
| `vercel-ai-gateway` | `vag` | Vercel AI Gateway | API key, aggregator | [link](https://vercel.com/docs/ai-gateway) | — |
| `vertex` | `vertex` | Vertex AI | API key, enterprise | [link](https://cloud.google.com/vertex-ai) | Provide Service Account JSON or OAuth access_token |
| `vertex-partner` | `vp` | Vertex AI Partners | API key, enterprise | [link](https://cloud.google.com/vertex-ai) | Provide the same Service Account JSON used for Vertex AI partner models. |
| `volcengine` | `volcengine` | Volcengine | API key | [link](https://www.volcengine.com) | — |
| `voyage-ai` | `voyage` | Voyage AI | API key, embed/rerank | [link](https://www.voyageai.com) | Bearer API key for Voyage AI embeddings and rerank APIs. |
| `wandb` | `wandb` | Weights & Biases Inference | API key | [link](https://wandb.ai) | — |
| `watsonx` | `watsonx` | IBM watsonx.ai Gateway | API key, enterprise | [link](https://www.ibm.com/products/watsonx-ai) | Use your watsonx bearer token. Base URL can be https://<region>.ml.cloud.ibm.com/ml/gateway/v1/ or a self-managed /ml/gateway/v1 endpoint. |
| `xai` | `xai` | xAI (Grok) | API key | [link](https://x.ai) | — |
| `xiaomi-mimo` | `mimo` | Xiaomi MiMo | API key | [link](https://mimo.mi.com) | — |
| `zai` | `zai` | Z.AI | API key | [link](https://open.bigmodel.cn) | — |
## Local Providers (10)
| ID | Alias | Name | Tags | Website | Notes |
| --------------------- | ------------ | ------------------- | ------------------ | --------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------- |
| `comfyui` | `comfyui` | ComfyUI | Local | [link](https://github.com/comfyanonymous/ComfyUI) | No API key required. Configure the local ComfyUI base URL (default: http://localhost:8188). |
| `docker-model-runner` | `dmr` | Docker Model Runner | Local, self-hosted | [link](https://docs.docker.com/ai/model-runner/) | API key optional. Configure the local Docker Model Runner OpenAI-compatible base URL (default: http://localhost:12434/v1). |
| `lemonade` | `lemonade` | Lemonade Server | Local, self-hosted | [link](https://lemonade-server.ai) | API key optional. Configure the local Lemonade OpenAI-compatible base URL (default: http://localhost:13305/api/v1). |
| `llamafile` | `llamafile` | Llamafile | Local, self-hosted | [link](https://github.com/Mozilla-Ocho/llamafile) | API key optional. Configure the local Llamafile OpenAI-compatible base URL (default: http://127.0.0.1:8080/v1). |
| `lm-studio` | `lmstudio` | LM Studio | Local, self-hosted | [link](https://lmstudio.ai) | API key optional. Configure the local LM Studio OpenAI-compatible base URL (default: http://localhost:1234/v1). |
| `oobabooga` | `ooba` | oobabooga | Local, self-hosted | [link](https://github.com/oobabooga/text-generation-webui) | API key optional. Configure the local oobabooga OpenAI-compatible base URL (default: http://localhost:5000/v1). |
| `sdwebui` | `sdwebui` | SD WebUI | Local | [link](https://github.com/AUTOMATIC1111/stable-diffusion-webui) | No API key required. Configure the local WebUI base URL (default: http://localhost:7860). |
| `triton` | `triton` | NVIDIA Triton | Local, self-hosted | [link](https://developer.nvidia.com/triton-inference-server) | API key optional. Configure the Triton OpenAI-compatible base URL (default: http://localhost:8000/v1). |
| `vllm` | `vllm` | vLLM | Local, self-hosted | [link](https://github.com/vllm-project/vllm) | API key optional. Configure the local vLLM OpenAI-compatible base URL (default: http://localhost:8000/v1). |
| `xinference` | `xinference` | XInference | Local, self-hosted | [link](https://inference.readthedocs.io) | API key optional. Configure the local XInference OpenAI-compatible base URL (default: http://localhost:9997/v1). |
## Search Providers (11)
| ID | Alias | Name | Tags | Website | Notes |
| ------------------- | --------------- | -------------------------- | ------ | --------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------- |
| `brave-search` | `brave-search` | Brave Search | Search | [link](https://brave.com/search/api) | Subscription token from Brave Search API dashboard |
| `exa-search` | `exa-search` | Exa Search | Search | [link](https://exa.ai) | API key from dashboard.exa.ai |
| `google-pse-search` | `google-pse` | Google Programmable Search | Search | [link](https://developers.google.com/custom-search/v1/overview) | Requires a Google API key and your Programmable Search Engine ID (cx) |
| `linkup-search` | `linkup` | Linkup Search | Search | [link](https://docs.linkup.so) | Bearer API key from the Linkup dashboard |
| `ollama-search` | `ollama-search` | Ollama Search | Search | [link](https://ollama.com/settings/api-keys) | Same API key as Ollama Cloud (from ollama.com/settings/api-keys) |
| `perplexity-search` | `pplx-search` | Perplexity Search | Search | [link](https://docs.perplexity.ai/guides/search-quickstart) | Same API key as Perplexity (pplx-...) |
| `searchapi-search` | `searchapi` | SearchAPI | Search | [link](https://www.searchapi.io/docs) | API key from SearchAPI (query param or Bearer auth) |
| `searxng-search` | `searxng` | SearXNG Search | Search | [link](https://docs.searxng.org) | API key is optional. Set your SearXNG base URL. Some instances may require a bearer token for access. |
| `serper-search` | `serper-search` | Serper Search | Search | [link](https://serper.dev) | API key from serper.dev dashboard |
| `tavily-search` | `tavily-search` | Tavily Search | Search | [link](https://tavily.com) | API key from app.tavily.com (format: tvly-...) |
| `youcom-search` | `youcom-search` | You.com Search | Search | [link](https://you.com/docs/search/overview) | X-API-Key from the You.com platform dashboard |
## Audio-only Providers (7)
| ID | Alias | Name | Tags | Website | Notes |
| ------------ | ---------- | ---------- | ----- | ------------------------------------- | ----------------------------------------------------------------------------------------------- |
| `assemblyai` | `aai` | AssemblyAI | Audio | [link](https://assemblyai.com) | — |
| `aws-polly` | `polly` | AWS Polly | Audio | [link](https://aws.amazon.com/polly/) | Use AWS Secret Access Key as API key; set providerSpecificData.accessKeyId and optional region. |
| `cartesia` | `cartesia` | Cartesia | Audio | [link](https://cartesia.ai) | — |
| `deepgram` | `dg` | Deepgram | Audio | [link](https://deepgram.com) | — |
| `elevenlabs` | `el` | ElevenLabs | Audio | [link](https://elevenlabs.io) | — |
| `inworld` | `inworld` | Inworld | Audio | [link](https://inworld.ai) | — |
| `playht` | `playht` | PlayHT | Audio | [link](https://play.ht) | — |
## Upstream Proxy Providers (1)
| ID | Alias | Name | Tags | Website | Notes |
| ------------- | ----- | ----------- | -------------- | ---------------------------------------------------- | ----- |
| `cliproxyapi` | `cpa` | CLIProxyAPI | Upstream proxy | [link](https://github.com/router-for-me/CLIProxyAPI) | — |
## Cloud Agent Providers (3)
| ID | Alias | Name | Tags | Website | Notes |
| ------------- | ------------- | ------------ | ----------- | -------------------------------- | ----------------------------------------------------------- |
| `codex-cloud` | `codex-cloud` | Codex Cloud | Cloud agent | [link](https://openai.com/codex) | OpenAI API key with Codex Cloud task access. |
| `devin` | `devin` | Devin | Cloud agent | [link](https://devin.ai) | Devin API key for cloud agent sessions. |
| `jules` | `jules` | Google Jules | Cloud agent | [link](https://jules.google) | Jules API key for creating and managing cloud coding tasks. |
## System Providers (1)
| ID | Alias | Name | Tags | Website | Notes |
| ------ | ------ | ------------------ | ------ | ------- | ----- |
| `auto` | `auto` | Auto (Zero-Config) | System | — | — |
## Sources of truth
- Catalog: [`src/shared/constants/providers.ts`](../../src/shared/constants/providers.ts)
- Registry (per-model details): [`open-sse/config/providerRegistry.ts`](../../open-sse/config/providerRegistry.ts)
- Executors: [`open-sse/executors/`](../../open-sse/executors/) (31 files)
- Translators: [`open-sse/translator/`](../../open-sse/translator/)
## See Also
- [FREE_TIERS.md](./FREE_TIERS.md) — curated free-tier guide
- [USER_GUIDE.md](../guides/USER_GUIDE.md) — provider setup walkthrough
- [ARCHITECTURE.md](../architecture/ARCHITECTURE.md) — overall architecture

106
Providers.md Normal file

@@ -0,0 +1,106 @@
# Providers — Claude Web
## claude-web
Web-cookie-based provider for **Claude AI** (`claude.ai`) using session cookie authentication.
### How It Works
1. User pastes their `claude.ai` session cookies into the OmniRoute dashboard
2. `ClaudeWebExecutor` transforms OpenAI-format requests to Claude Web API format
3. Requests are sent via **`tls-client-node`** with **Chrome 124 TLS fingerprint** to bypass Cloudflare Turnstile
4. Responses are streamed back via SSE (`text/event-stream`)
### Required Cookies
| Cookie | Purpose | Source |
| -------------- | ------------------------------ | -------------------------------------- |
| `sessionKey` | Main authentication | `claude.ai` browser session |
| `routingHint` | Anthropic routing | `claude.ai` browser session |
| `cf_clearance` | Cloudflare Turnstile clearance | Auto-set by Cloudflare after challenge |
| `__cf_bm` | Cloudflare bot management | Auto-set by Cloudflare |
| `_cfuvid` | Cloudflare visitor ID | Auto-set by Cloudflare |
> **Note**: `cf_clearance` is bound to the TLS fingerprint of the browser that solved Cloudflare's Turnstile challenge. The `tls-client-node` library (via `claudeTlsClient.ts`) spoofs a Chrome 124 TLS handshake so the clearance token works from the OmniRoute server.
### API Reference
**Endpoint**: `POST /api/organizations/{orgId}/chat_conversations/{convId}/completion`
**Required Headers**:
```
accept: text/event-stream
anthropic-client-platform: web_claude_ai
anthropic-device-id: <uuid>
content-type: application/json
Referer: https://claude.ai/chat/{convId}
```
**Request Body**:
```json
{
"prompt": "user message",
"model": "claude-sonnet-4-6",
"timezone": "Asia/Jakarta",
"locale": "en-US",
"personalized_styles": [...],
"tools": [...],
"rendering_mode": "messages",
"create_conversation_params": {
"name": "",
"model": "claude-sonnet-4-6",
"is_temporary": false
}
}
```
### Architecture
```
User Cookies (claude.ai)
↓
OmniRoute Dashboard
↓
ClaudeWebExecutor (open-sse/executors/claude-web.ts)
↓ Request transformation (OpenAI → Claude Web format)
↓
tlsFetchClaude() (open-sse/services/claudeTlsClient.ts)
↓ Chrome 124 TLS fingerprint spoofing
↓
tls-client-node (Go native binding, koffi)
↓
claude.ai API
↓ SSE stream
```
### Files
| File | Purpose |
| ----------------------------------------------------- | -------------------------------------------- |
| `src/shared/constants/providers.ts` | Provider registration (WEB_COOKIE_PROVIDERS) |
| `src/lib/providers/wrappers/claudeWeb.ts` | Type definitions + cookie utilities |
| `open-sse/executors/claude-web.ts` | Executor implementation |
| `open-sse/executors/index.ts` | Executor registration |
| `open-sse/services/claudeTlsClient.ts` | TLS fingerprint spoofing via tls-client-node |
| `open-sse/services/__tests__/claudeTlsClient.test.ts` | TLS client tests |
| `tests/unit/claude-web.test.ts` | Executor tests |
### Testing
```bash
# Unit tests
node --import tsx/esm --test tests/unit/claude-web.test.ts
# TLS client tests
npx vitest run open-sse/services/__tests__/claudeTlsClient.test.ts
```
### Setup
1. Start OmniRoute: `omniroute`
2. Go to Dashboard → Providers → Add Provider
3. Select "Web Cookie" category
4. Choose "Claude Web"
5. Paste your full cookie header from `claude.ai` browser DevTools (Network tab → Copy as fetch → Cookie header)

601
Proxy-Guide.md Normal file

@@ -0,0 +1,601 @@
# 🌐 OmniRoute Proxy Guide
> **Bypass geographic blocks, protect your identity, and route AI traffic through any proxy — with zero configuration complexity.**
OmniRoute includes a full-featured proxy management system that lets you route upstream AI provider traffic through HTTP, HTTPS, or SOCKS5 proxies. Whether you're in a blocked region, need IP rotation, or want stealth fingerprinting — this guide covers everything.
---
## Table of Contents
- [Why Use Proxies?](#why-use-proxies)
- [Architecture Overview](#architecture-overview)
- [4-Level Proxy System](#4-level-proxy-system)
- [Proxy Registry (CRUD)](#proxy-registry-crud)
- [1proxy Free Marketplace](#1proxy-free-proxy-marketplace)
- [Proxy Rotation](#proxy-rotation)
- [Anti-Detection & Stealth](#anti-detection--stealth)
- [Upstream Proxy Modes](#upstream-proxy-modes)
- [Dashboard UI](#dashboard-ui)
- [API Reference](#api-reference)
- [Environment Variables](#environment-variables)
- [Troubleshooting](#troubleshooting)
---
## Why Use Proxies?
Many AI providers restrict access by geographic region. Developers in **Russia, China, Iran, Cuba, Turkey**, and other countries encounter errors like:
```
unsupported_country_region_territory
```
Even outside blocked regions, proxies are useful for:
| Use Case | Description |
| --------------------- | --------------------------------------------------------------- |
| **Geographic bypass** | Access OpenAI, Anthropic, Codex, Copilot from blocked countries |
| **IP rotation** | Distribute requests across multiple IPs to avoid rate limiting |
| **Privacy** | Hide your real IP from upstream providers |
| **Compliance** | Route traffic through specific jurisdictions |
| **Testing** | Simulate requests from different regions |
---
## Architecture Overview
```
┌───────────────────────────────────────────────────────────────┐
│ OmniRoute Server │
│ │
│ ┌─────────────┐ ┌──────────────┐ ┌──────────────────┐ │
│ │ Proxy │ │ Proxy │ │ Proxy │ │
│ │ Registry │───▶│ Dispatcher │───▶│ Fetch (undici) │ │
│ │ (SQLite) │ │ (cached) │ │ │ │
│ └─────────────┘ └──────────────┘ └────────┬─────────┘ │
│ ▲ │ │
│ │ ▼ │
│ ┌──────┴──────┐ ┌──────────────────┐ │
│ │ 1proxy Sync │ │ Upstream │ │
│ │ (free pool) │ │ Provider API │ │
│ └─────────────┘ └──────────────────┘ │
└───────────────────────────────────────────────────────────────┘
```
### Key Components
| Component | File | Role |
| -------------------- | -------------------------------------------- | ---------------------------------------------------------- |
| **Proxy Registry** | `src/lib/db/proxies.ts` | CRUD for proxy entries + scope assignments |
| **Proxy Dispatcher** | `open-sse/utils/proxyDispatcher.ts` | Creates `undici` ProxyAgent/SOCKS dispatchers with caching |
| **Proxy Fetch** | `open-sse/utils/proxyFetch.ts` | Wraps `fetch()` with proxy dispatcher injection |
| **Settings Route** | `src/app/api/settings/proxy/route.ts` | Legacy proxy config API (GET/PUT/DELETE) |
| **Management Route** | `src/app/api/v1/management/proxies/route.ts` | Registry CRUD API (GET/POST/PATCH/DELETE) |
| **1proxy DB** | `src/lib/db/oneproxy.ts` | Free proxy marketplace persistence |
| **1proxy Sync** | `src/lib/oneproxySync.ts` | Fetches proxies from 1proxy API |
| **1proxy Rotator** | `src/lib/oneproxyRotator.ts` | Rotation strategies (quality/random/sequential) |
---
## 4-Level Proxy System
OmniRoute supports proxy configuration at **four independent scopes**, resolved in priority order:
```
Priority Resolution Order (highest → lowest):
1. 🔵 Account/Connection Proxy → per API key / OAuth connection
2. 🟡 Provider Proxy → per provider (e.g., all OpenAI traffic)
3. 🟠 Combo Proxy → per combo/routing configuration
4. 🟢 Global Proxy → all traffic, all providers
```
### How Resolution Works
When OmniRoute sends a request to an upstream provider, it calls `resolveProxyForConnectionFromRegistry()` which checks each level in order:
1. **Account-level** — Is there a proxy assigned to this specific connection ID?
2. **Provider-level** — Is there a proxy assigned to this provider (e.g., `openai`)?
3. **Global-level** — Is there a global proxy configured?
4. **No proxy** — Direct connection to the provider.
The first match wins. This means you can set a global proxy as a fallback but override it for specific providers or connections.
### What Gets Proxied
| Traffic Type | Proxied? | Notes |
| -------------------- | -------- | --------------------------------------------- |
| Chat completions | ✅ | All `/v1/chat/completions` requests |
| Embeddings | ✅ | `/v1/embeddings` |
| Image generation | ✅ | `/v1/images/generations` |
| Audio (TTS/STT) | ✅ | `/v1/audio/*` |
| OAuth token exchange | ✅ | Solves `unsupported_country_region_territory` |
| Connection tests | ✅ | "Test Connection" button uses proxy |
| Token refresh | ✅ | Background OAuth renewal |
| Model sync | ✅ | Model listing and discovery |
---
## Proxy Registry (CRUD)
The proxy registry is a SQLite table (`proxy_registry`) that stores all your proxies. Each proxy has:
| Field | Type | Description |
| ---------- | ------- | ----------------------------------- |
| `id` | UUID | Unique identifier |
| `name` | String | Human-readable label |
| `type` | String | Protocol: `http`, `https`, `socks5` |
| `host` | String | Proxy hostname or IP |
| `port` | Integer | Port number |
| `username` | String | Auth username (encrypted at rest) |
| `password` | String | Auth password (encrypted at rest) |
| `region` | String | Geographic region label |
| `notes` | String | Free-text notes |
| `status` | String | `active` or `inactive` |
| `source` | String | `manual` or `oneproxy` |
### Creating a Proxy
**Via Dashboard:**
1. Go to **Settings → Proxy**
2. Click **Add Proxy**
3. Fill in the type, host, port, and optional auth credentials
4. Save
**Via API:**
```bash
curl -X POST http://localhost:20128/api/v1/management/proxies \
-H "Content-Type: application/json" \
-d '{
"name": "US Proxy",
"type": "http",
"host": "proxy.example.com",
"port": 8080,
"username": "user",
"password": "pass",
"region": "US"
}'
```
### Updating a Proxy
```bash
curl -X PATCH http://localhost:20128/api/v1/management/proxies \
-H "Content-Type: application/json" \
-d '{
"id": "proxy-uuid-here",
"host": "new-proxy.example.com",
"port": 9090
}'
```
> **Note:** Credentials are preserved unless you explicitly send non-empty replacements. Sending empty strings for `username`/`password` will keep the stored values.
### Deleting a Proxy
```bash
# Fails if proxy is assigned to any scope
curl -X DELETE "http://localhost:20128/api/v1/management/proxies?id=proxy-uuid"
# Force delete (removes assignments too)
curl -X DELETE "http://localhost:20128/api/v1/management/proxies?id=proxy-uuid&force=1"
```
### Listing Proxies
```bash
curl "http://localhost:20128/api/v1/management/proxies?limit=50&offset=0"
```
### Assigning Proxies to Scopes
```bash
# Assign to global scope
curl -X PUT http://localhost:20128/api/settings/proxy \
-H "Content-Type: application/json" \
-d '{"level": "global", "proxy": {"type":"http","host":"proxy.example.com","port":8080}}'
# Assign to a specific provider
curl -X PUT http://localhost:20128/api/settings/proxy \
-H "Content-Type: application/json" \
-d '{"level": "provider", "id": "openai", "proxy": {"type":"socks5","host":"socks.example.com","port":1080}}'
# Assign to a specific connection/key
curl -X PUT http://localhost:20128/api/settings/proxy \
-H "Content-Type: application/json" \
-d '{"level": "key", "id": "connection-uuid", "proxy": {"type":"http","host":"key-proxy.com","port":3128}}'
```
### Resolving Effective Proxy
Check which proxy would be used for a given connection:
```bash
curl "http://localhost:20128/api/settings/proxy?resolve=connection-uuid"
```
Returns the resolved proxy with its level (`account`, `provider`, or `global`) and source.
### Bulk Assignment
Assign one proxy to multiple providers or connections at once:
```bash
curl -X POST http://localhost:20128/api/v1/management/proxies/bulk-assign \
-H "Content-Type: application/json" \
-d '{
"scope": "provider",
"scopeIds": ["openai", "anthropic", "codex"],
"proxyId": "proxy-uuid"
}'
```
### Import/Export
Proxies are included in the **Backup/Restore** system. When you export your OmniRoute configuration:
1. Go to **Dashboard → Settings → Backup**
2. Click **Export** — proxy registry and assignments are included
3. To restore, click **Import** and upload the backup file
The proxy registry also supports **upsert by host+port** — if you import a proxy that already exists (same host and port), it updates instead of creating a duplicate.
### Legacy Migration
If you configured proxies in an older version (pre-registry), OmniRoute automatically migrates them:
```
Legacy key_value store → proxy_registry + proxy_assignments
```
This happens once on first startup after upgrade. Use `migrateLegacyProxyConfigToRegistry({ force: true })` to re-run.
---
## 1proxy Free Proxy Marketplace
> 🆕 **Contributed by [@oyi77](https://github.com/oyi77)** — PR [#1847](https://github.com/diegosouzapw/OmniRoute/pull/1847) (Issue [#1788](https://github.com/diegosouzapw/OmniRoute/issues/1788))
OmniRoute integrates with the **[1proxy](https://1proxy-api.aitradepulse.com)** community platform to provide access to **hundreds of free, validated proxies** from around the world. This is perfect for users who don't have their own proxy infrastructure.
### How It Works
```
┌─────────────┐ Sync ┌─────────────────┐ Rotate ┌──────────┐
│ 1proxy API │ ────────────▶ │ proxy_registry │ ────────────▶ │ Provider │
│ (external) │ up to 500 │ source=oneproxy │ by quality │ API │
└─────────────┘ proxies └─────────────────┘ └──────────┘
```
1. **Sync** — OmniRoute fetches validated proxies from the 1proxy API
2. **Store** — Proxies are saved in the same `proxy_registry` table with `source = 'oneproxy'`
3. **Filter** — Filter by protocol, country, quality score
4. **Rotate** — Pick the best proxy using quality, random, or sequential strategies
5. **Auto-degrade** — Failed proxies get their quality score reduced; below threshold → marked inactive
### Syncing Proxies
**Via Dashboard:**
1. Go to **Settings → 1proxy** tab
2. Click **"Sync Now"**
3. View stats: total proxies, active count, average quality, by-country breakdown
**Via API:**
```bash
# Trigger sync
curl -X POST http://localhost:20128/api/settings/oneproxy \
-H "Content-Type: application/json" \
-d '{}'
# Response:
# { "success": true, "added": 127, "updated": 45, "failed": 2, "total": 172 }
```
### Filtering Proxies
```bash
# Filter by protocol
curl "http://localhost:20128/api/settings/oneproxy?protocol=socks5"
# Filter by country
curl "http://localhost:20128/api/settings/oneproxy?countryCode=US"
# Filter by minimum quality score
curl "http://localhost:20128/api/settings/oneproxy?minQuality=80"
# Combine filters
curl "http://localhost:20128/api/settings/oneproxy?protocol=http&countryCode=DE&minQuality=70"
```
### Proxy Quality Scores
Each 1proxy proxy comes with metadata:
| Field | Description |
| --------------- | -------------------------------------------- |
| `qualityScore` | 0-100 rating from 1proxy validation |
| `latencyMs` | Measured network latency |
| `anonymity` | `transparent`, `anonymous`, or `elite` |
| `googleAccess` | Whether the proxy can access Google services |
| `countryCode` | Two-letter ISO country code |
| `lastValidated` | Timestamp of last validation |
Quality scores are dynamically adjusted:
- **Failed requests** reduce the score by 10 points
- **Score drops to ≤10** → proxy is marked `inactive`
- Inactive proxies are excluded from rotation
### Rotation Strategies
```bash
# Rotate by quality (best proxy first) — default
curl -X POST http://localhost:20128/api/settings/oneproxy/rotate \
-H "Content-Type: application/json" \
-d '{"strategy": "quality"}'
# Random rotation
curl -X POST http://localhost:20128/api/settings/oneproxy/rotate \
-d '{"strategy": "random"}'
# Sequential (least recently validated first)
curl -X POST http://localhost:20128/api/settings/oneproxy/rotate \
-d '{"strategy": "sequential"}'
```
### Circuit Breaker
The 1proxy sync has a built-in circuit breaker:
- After **5 consecutive sync failures**, further sync attempts are blocked
- Reset with: `resetOneproxyCircuitBreaker()` or restart the server
- Sync status is available at `GET /api/settings/oneproxy?action=status`
### Clearing 1proxy Proxies
```bash
# Delete a single 1proxy proxy
curl -X DELETE "http://localhost:20128/api/settings/oneproxy?id=proxy-uuid"
# Clear ALL 1proxy proxies (manual proxies are untouched)
curl -X DELETE "http://localhost:20128/api/settings/oneproxy?clearAll=1"
```
---
## Anti-Detection & Stealth
OmniRoute doesn't just route traffic through a proxy — it makes the traffic look legitimate:
### TLS Fingerprint Spoofing
Uses `wreq-js` to generate browser-like TLS fingerprints, bypassing bot detection systems that flag non-browser TLS handshakes.
### CLI Fingerprint Matching
The **CLI Fingerprint Toggle** (`Settings → Security`) reorders HTTP headers and JSON body fields to match the exact signature of native CLI binaries (Claude Code, Codex, etc.). This works **on top of** the proxy:
```
Your IP (blocked) → Proxy IP (US) → Provider API
+ TLS spoof
+ CLI fingerprint
```
You get both **IP masking** and **request authenticity** simultaneously.
### Proxy IP Preservation
Color-coded badges in the dashboard show which proxy level is active:
| Badge | Level | Meaning |
| ----- | ---------- | ----------------------------------------- |
| 🟢 | Global | All traffic goes through this proxy |
| 🟡 | Provider | Only this provider's traffic is proxied |
| 🔵 | Connection | This specific key/account uses this proxy |
The badge also shows the resolved proxy IP for verification.
---
## Upstream Proxy Modes
For providers that use the CLIProxyAPI pattern, OmniRoute supports three upstream proxy modes:
| Mode | Description |
| ------------- | -------------------------------------------------- |
| `native` | OmniRoute handles proxy routing directly (default) |
| `cliproxyapi` | Delegates to an external CLIProxyAPI instance |
| `fallback` | Tries native first, falls back to CLIProxyAPI |
Configure per-provider:
```bash
curl -X PUT "http://localhost:20128/api/upstream-proxy/openai" \
-H "Content-Type: application/json" \
-d '{"mode": "native", "enabled": true}'
```
---
## Dashboard UI
### Settings → Proxy Tab
- **Global proxy** configuration (set once for all traffic)
- **Per-provider proxy** overrides
- **Per-connection proxy** assignments
- **Connection test** through configured proxy
- **Color-coded badges** showing active proxy level
### Settings → 1proxy Tab
- **Sync Now** button to fetch free proxies
- **Stats cards**: Total, Active, Avg Quality, Last Sync
- **Filters**: Protocol, Country Code, Min Quality
- **Proxy table** with host, protocol, country, quality score, latency, anonymity, Google access
- **Sync status** panel with success/failure tracking and consecutive failure count
- **Clear All** to remove all 1proxy entries
---
## API Reference
### Proxy Settings API
| Method | Endpoint | Description |
| -------- | ---------------------------------------------- | ----------------------- |
| `GET` | `/api/settings/proxy` | Get full proxy config |
| `GET` | `/api/settings/proxy?level=global` | Get global proxy |
| `GET` | `/api/settings/proxy?level=provider&id=openai` | Get provider proxy |
| `GET` | `/api/settings/proxy?resolve=connectionId` | Resolve effective proxy |
| `PUT` | `/api/settings/proxy` | Update proxy config |
| `DELETE` | `/api/settings/proxy?level=provider&id=openai` | Remove proxy at level |
### Proxy Registry API
| Method | Endpoint | Description |
| -------- | ------------------------------------------------- | --------------------- |
| `GET` | `/api/v1/management/proxies` | List all proxies |
| `GET` | `/api/v1/management/proxies?id=uuid` | Get proxy by ID |
| `GET` | `/api/v1/management/proxies?id=uuid&where_used=1` | Get proxy assignments |
| `POST` | `/api/v1/management/proxies` | Create proxy |
| `PATCH` | `/api/v1/management/proxies` | Update proxy |
| `DELETE` | `/api/v1/management/proxies?id=uuid` | Delete proxy |
| `DELETE` | `/api/v1/management/proxies?id=uuid&force=1` | Force delete |
| `POST` | `/api/v1/management/proxies/bulk-assign` | Bulk assign |
| `GET` | `/api/v1/management/proxies/assignments` | List assignments |
| `GET` | `/api/v1/management/proxies/health` | Proxy health stats |
### Tunnels API
For exposing your OmniRoute instance to the public internet (Cloudflare/ngrok/Tailscale) instead of routing outbound through a proxy, see [TUNNELS_GUIDE.md](./TUNNELS_GUIDE.md). The tunnel REST API lives under `/api/tunnels/{cloudflared,ngrok,tailscale}/*` and is orthogonal to the outbound proxy chain documented above.
### 1proxy API
| Method | Endpoint | Description |
| -------- | -------------------------------------- | ----------------------- |
| `GET` | `/api/settings/oneproxy` | List 1proxy proxies |
| `GET` | `/api/settings/oneproxy?action=stats` | Get stats + sync status |
| `GET` | `/api/settings/oneproxy?action=status` | Get sync status only |
| `POST` | `/api/settings/oneproxy` | Trigger sync |
| `POST` | `/api/settings/oneproxy/rotate` | Rotate to next proxy |
| `DELETE` | `/api/settings/oneproxy?id=uuid` | Delete one |
| `DELETE` | `/api/settings/oneproxy?clearAll=1` | Clear all |
### Upstream Proxy API
| Method | Endpoint | Description |
| -------- | --------------------------------- | ---------------------------- |
| `GET` | `/api/upstream-proxy/:providerId` | Get upstream proxy config |
| `PUT` | `/api/upstream-proxy/:providerId` | Set upstream proxy mode |
| `DELETE` | `/api/upstream-proxy/:providerId` | Remove upstream proxy config |
---
## Environment Variables
| Variable | Default | Description |
| -------------------------------- | ------------------------------------- | -------------------------------------------------------------- |
| `ENABLE_SOCKS5_PROXY` | `true` | Enable SOCKS5 proxy support (default `true` in `.env.example`) |
| `ONEPROXY_ENABLED` | `true` | Enable 1proxy integration |
| `ONEPROXY_API_URL` | `https://1proxy-api.aitradepulse.com` | 1proxy API endpoint |
| `ONEPROXY_MAX_PROXIES` | `500` | Maximum proxies to sync |
| `ONEPROXY_MIN_QUALITY_THRESHOLD` | `50` | Minimum quality score to import |
---
## Troubleshooting
### "SOCKS5 proxy is disabled"
Set `ENABLE_SOCKS5_PROXY=true` in your `.env` file and restart.
### "socket hang up" errors through proxy
This is normal with cheap proxies that drop idle connections. OmniRoute already handles this by:
- Disabling keep-alive on proxy connections (`keepAliveTimeout: 1`)
- Disabling pipelining (`pipelining: 0`)
- Caching dispatchers to avoid repeated handshakes
If it persists, try a different proxy or use the 1proxy rotation feature.
### "unsupported_country_region_territory" during OAuth
Make sure the proxy is configured **before** starting the OAuth flow. OmniRoute routes OAuth token exchange through the configured proxy. Set a global or provider-level proxy first, then connect.
### Proxy not being used
Check the resolution order:
1. Verify with `GET /api/settings/proxy?resolve=your-connection-id`
2. Check if the proxy `status` is `active` (not `inactive`)
3. Ensure the proxy assignment scope matches your connection
### 1proxy sync failing
Check the sync status:
```bash
curl "http://localhost:20128/api/settings/oneproxy?action=status"
```
If `consecutiveFailures >= 5`, the circuit breaker has tripped. Restart the server to reset, or wait for manual reset.
---
## Database Schema
### `proxy_registry` Table
```sql
CREATE TABLE proxy_registry (
id TEXT PRIMARY KEY,
name TEXT NOT NULL,
type TEXT NOT NULL DEFAULT 'http',
host TEXT NOT NULL,
port INTEGER NOT NULL,
username TEXT DEFAULT '',
password TEXT DEFAULT '',
region TEXT,
notes TEXT,
status TEXT DEFAULT 'active',
source TEXT NOT NULL DEFAULT 'manual', -- 'manual' or 'oneproxy'
quality_score INTEGER, -- 0-100 (1proxy only)
latency_ms INTEGER, -- milliseconds (1proxy only)
anonymity TEXT, -- transparent/anonymous/elite
google_access INTEGER DEFAULT 0, -- can access Google? (1proxy)
last_validated TEXT, -- ISO timestamp (1proxy)
country_code TEXT, -- ISO 2-letter code (1proxy)
created_at TEXT NOT NULL,
updated_at TEXT NOT NULL
);
```
### `proxy_assignments` Table
```sql
CREATE TABLE proxy_assignments (
id INTEGER PRIMARY KEY AUTOINCREMENT,
proxy_id TEXT NOT NULL REFERENCES proxy_registry(id),
scope TEXT NOT NULL, -- 'global', 'provider', 'account', 'combo'
scope_id TEXT, -- provider ID, connection ID, or combo ID
created_at TEXT NOT NULL,
updated_at TEXT NOT NULL,
UNIQUE(scope, scope_id)
);
```
---
> 📖 **Related documentation:**
>
> - [User Guide](../guides/USER_GUIDE.md) — General setup and configuration
> - [API Reference](../reference/API_REFERENCE.md) — Full API documentation
> - [Environment Config](../reference/ENVIRONMENT.md) — All environment variables

137
Public-Creds.md Normal file

@@ -0,0 +1,137 @@
# Public Credentials Handling
> **Source of truth:** `open-sse/utils/publicCreds.ts`
> **Tests:** `tests/unit/publicCreds.test.ts`
> **Last updated:** 2026-05-14 — v3.8.0
> **Audience:** Engineers integrating providers that ship public OAuth client_id / client_secret / Firebase Web API keys in their public CLIs.
> **Status:** **MANDATORY** for all new code that embeds upstream identifiers.
## Why this exists
Some upstream providers (Gemini CLI, Antigravity CLI, Windsurf / Devin CLI, GitHub Copilot, and similar OAuth-native clients) ship credentials extracted from their **public binaries or web apps**. Google explicitly documents that these are not secrets:
- [OAuth 2.0 for native apps (PKCE)](https://developers.google.com/identity/protocols/oauth2/native-app) — OAuth client_id / client_secret for installed apps are public; PKCE provides the actual security.
- [Firebase API keys](https://firebase.google.com/docs/projects/api-keys) — Web client identifiers are public by design.
OmniRoute must embed these values so users who do not configure `.env` still get a working OAuth flow out of the box. Without an embedded fallback, the Gemini / Antigravity / Windsurf providers stop working for any user who follows the "just clone and run" path.
However, literal values like `AIzaSy…`, `GOCSPX-…`, `…apps.googleusercontent.com` are matched by **GitHub Secret Scanning**, **Semgrep**, and similar pattern scanners. Every release becomes a noisy stream of false positives, push protection blocks legitimate commits, and operators stop trusting the alert feed.
The `open-sse/utils/publicCreds.ts` helper solves both constraints at once:
- Embeds the public identifier as a **XOR-masked byte sequence** (no scanner pattern in source).
- Decodes at runtime via `decodePublicCred` / `resolvePublicCred`.
- Detects raw values that already follow well-known prefixes (`AIza`, `GOCSPX-`, `<digits>-<32hex>.apps.googleusercontent.com`, `Iv1.<hex>`) and passes them through unchanged, so users with raw values in their existing `.env` keep working with **zero migration**.
This is **obfuscation, not encryption.** Anyone reading the source can recover the value — which is fine because the value is public by design. The only goal is to avoid scanner regex matches.
## The mandatory pattern
### 1. Adding a new public credential
When you need to embed a new upstream-provided value that:
- comes from a public CLI / desktop app / browser bundle, **and**
- the upstream provider documents (or treats) it as a public client identifier, **and**
- a pattern scanner would otherwise match it (`AIza…`, `GOCSPX-…`, `<digits>-…apps.googleusercontent.com`, etc.),
…follow this checklist:
1. Generate the masked byte sequence:
```bash
node --import tsx/esm -e \
'import("./open-sse/utils/publicCreds.ts").then(m =>
console.log(JSON.stringify(Array.from(
Buffer.from(m.encodePublicCred("THE_PUBLIC_VALUE"), "base64")
))))'
```
2. Add a new entry to `EMBEDDED_DEFAULTS` in `open-sse/utils/publicCreds.ts` with a **neutral key name** (`<provider>_id`, `<provider>_alt`, `<provider>_fb`, etc.). Do **not** use names like `client_secret` or `api_key` in the helper — those words trigger Semgrep generic-secret rules.
3. Add a `keyof typeof EMBEDDED_DEFAULTS` to the public type union (it is inferred automatically).
4. In the consumer code, replace the hardcoded literal with:
```ts
// single env override
clientSecret: resolvePublicCred("provider_alt", "PROVIDER_OAUTH_CLIENT_SECRET"),
// multiple env aliases (first non-empty wins)
clientId: resolvePublicCredMulti("provider_id", [
"PROVIDER_CLI_OAUTH_CLIENT_ID",
"PROVIDER_OAUTH_CLIENT_ID",
]),
// no env override (always embedded default)
firebaseApiKey: resolvePublicCred("provider_fb"),
```
5. Remove the literal from `.env.example` (replace with comment-only documentation pointing readers here):
```dotenv
# ── Provider (Google / Firebase / etc.) ──
# Public OAuth credentials are baked into the code via
# open-sse/utils/publicCreds.ts. Set these vars only to use your own.
# PROVIDER_OAUTH_CLIENT_ID=
# PROVIDER_OAUTH_CLIENT_SECRET=
```
6. Update `tests/unit/publicCreds.test.ts` to add a shape assertion for the new key (verify format, not literal value — see existing tests for the pattern).
7. **Never** add `AIza…` / `GOCSPX-…` / `…apps.googleusercontent.com` literals to test files. Use the `FAKE_*` constants built from `.join("")` fragments (see existing tests).
### 2. Consumers
- **Read from `resolvePublicCred()` / `resolvePublicCredMulti()` only** — never call `decodePublicCredBytes()` directly outside the helper.
- The helper is intentionally cheap (linear byte XOR) and safe to call at module-load time; defaults are computed once.
- The env override always wins. If a user sets `PROVIDER_OAUTH_CLIENT_SECRET=GOCSPX-myown`, the helper passes that raw value straight through.
### 3. Forbidden patterns
❌ **Never** do any of the following in production code (`src/`, `open-sse/`, `electron/`, `bin/`):
```ts
// BAD: literal value triggers Secret Scanning + Semgrep
clientSecret: process.env.PROVIDER_OAUTH_CLIENT_SECRET || "GOCSPX-realvalue",
// BAD: base64 of the literal — GitHub still detects since Feb/2025
clientSecret: process.env.PROVIDER_OAUTH_CLIENT_SECRET ||
Buffer.from("R09DU1BYLXJlYWx2YWx1ZQ==", "base64").toString(),
// BAD: string concatenation that re-assembles the pattern at runtime
clientSecret: "GO" + "CS" + "PX-" + "realvalue",
// BAD: hex/ROT13 encoding — different obfuscation, same risk of detection
clientSecret: hexDecode("474f4353..."),
```
These all eventually trip a scanner. Use `resolvePublicCred()`.
❌ **Never** add literal credentials to `.env.example`. Users who need real upstream values can extract them from the public CLI themselves, or use their own OAuth registration.
❌ **Never** dismiss a new secret-scanning alert without first checking whether the credential should be moved to this helper.
## Related controls
- `RAW_VALUE_PATTERN` in `publicCreds.ts` enumerates the prefixes that trigger passthrough (retrocompat). Extend it only for documented public credential formats, never for proprietary secrets.
- `.env.example` lives in CI's `check-env-doc-sync` script — when you remove a var here, make sure the docs match.
- The `npm run test:vitest` and `node --import tsx/esm --test tests/unit/publicCreds.test.ts` suites must both stay green.
## When NOT to use this helper
This helper is **only** for credentials that are:
1. Distributed publicly by the upstream provider (CLI binary, browser bundle, official docs).
2. Documented or strongly implied to be non-confidential (PKCE-protected, Firebase Web key, similar).
For everything else — operator-issued tokens, per-tenant secrets, your own OAuth app's client_secret, encryption keys, JWT secrets, database passwords — use **env vars only** (`process.env.FOO`, `||` fallback to empty / explicit error). These belong in `.env` and the [encrypted credentials store](./COMPLIANCE.md), not in source.
## References
- [Google: OAuth 2.0 for native apps](https://developers.google.com/identity/protocols/oauth2/native-app)
- [Firebase: API keys for client identification](https://firebase.google.com/docs/projects/api-keys)
- [GitHub Secret Scanning supported secrets](https://docs.github.com/en/code-security/secret-scanning/introduction/supported-secret-scanning-patterns)
- [GitHub: base64 detection for tokens (Feb 2025)](https://github.blog/changelog/2025-02-14-secret-scanning-detects-base64-encoded-github-tokens/)
- Commit introducing this helper: `1a39c31f` — _fix(security): mask public upstream creds + centralize error sanitization_

235
RTK-Compression.md Normal file

@@ -0,0 +1,235 @@
# RTK Compression
RTK compression is OmniRoute's command-aware compression engine for terminal and tool output. It is
designed for coding-agent sessions where most context growth comes from test logs, build output,
package manager noise, shell transcripts, Docker output, git output, and stack traces.
RTK can run directly with `defaultMode: "rtk"` or as the first step in a stacked pipeline, usually:
```txt
rtk -> caveman
```
That order compresses noisy machine output first, then lets Caveman condense remaining prose.
Upstream RTK reports `60-90%` command-output savings. Its README sample session goes from
`~118,000` standard tokens to `~23,900` RTK tokens, which is `79.7%` saved (`~80%`). OmniRoute uses
that upstream average for the stacked savings calculation with Caveman input compression:
```txt
RTK average: 80% saved
Caveman input: 46% saved
Stacked: 1 - (1 - 0.80) * (1 - 0.46) = 89.2% saved
Range: 1 - (1 - 0.60..0.90) * (1 - 0.46) = 78.4-94.6%
```
## What It Compresses
The built-in catalog currently ships 49 filters across these categories:
| Category | Examples |
| --------- | ------------------------------------------------------------- |
| `git` | `git status`, `git branch`, `git diff`, `git log` |
| `test` | Vitest, Jest, Pytest, Playwright, Go tests, Cargo tests |
| `build` | TypeScript, ESLint, Biome, Prettier, Vite, Webpack, Turbo, Nx |
| `package` | `npm install`, `npm audit`, `pip`, `uv sync`, Poetry, Bundler |
| `shell` | `ls`, `find`, `grep`, generic shell logs |
| `docker` | `docker ps`, Docker logs |
| `infra` | Terraform, OpenTofu, `systemctl status` |
| `generic` | JSON output, stack traces, generic output fallback |
The detector in `open-sse/services/compression/engines/rtk/commandDetector.ts` classifies output
before filter selection. Filters can also match by command pattern or output regex when a command
class is not enough.
## Filter Resolution
RTK loads filters in this order:
1. Project filters from `.rtk/filters.json`, only when trusted.
2. Global filters from `DATA_DIR/rtk/filters.json`.
3. Built-in filters from `open-sse/services/compression/engines/rtk/filters/`.
Project filters are intentionally trust-gated because regex filters can change how tool output is
shown to agents. A project filter file is accepted when one of these is true:
- `rtkConfig.trustProjectFilters` is `true`.
- `OMNIROUTE_RTK_TRUST_PROJECT_FILTERS=1` is set.
- `.rtk/trust.json` contains the SHA-256 hash of `.rtk/filters.json`.
Trust file example:
```json
{
"filtersSha256": "0123456789abcdef..."
}
```
Custom filters can be one filter object or an array of filter objects. Invalid custom filters are
skipped and reported by `/api/context/rtk/filters` diagnostics. Invalid built-in filters fail fast.
## Filter DSL
Filters use the JSON schema described in [Compression Rules Format](./COMPRESSION_RULES_FORMAT.md).
The runtime applies these stages in order:
```txt
stripAnsi -> filterStderr -> replace -> matchOutput -> drop/include lines
-> truncateLineAt -> head/tail/maxLines -> onEmpty
```
Important fields:
| Field | Purpose |
| ---------------------------- | -------------------------------------------------------------- |
| `rules.stripAnsi` | Remove terminal color/control sequences before matching |
| `rules.filterStderr` | Normalize common stderr prefixes before matching/filtering |
| `rules.replace` | Apply ordered regex replacements |
| `rules.matchOutput` | Return a compact summary when output matches a known condition |
| `rules.matchOutput[].unless` | Skip the shortcut when an error/failure pattern is present |
| `rules.dropPatterns` | Remove noisy lines |
| `rules.includePatterns` | Prefer actionable lines |
| `rules.collapsePatterns` | Collapse repeated matching lines |
| `rules.truncateLineAt` | Unicode-safe per-line truncation |
| `rules.onEmpty` | Fallback message if all lines are filtered out |
| `tests[]` | Inline samples used by the verify gate |
Built-in filters are expected to include inline `tests[]` samples. Custom filters should include
them too, especially when they are shared across projects.
## Configuration
Global settings are available through `/api/settings/compression`. RTK-specific settings are also
available through `/api/context/rtk/config`.
```json
{
"defaultMode": "stacked",
"autoTriggerMode": "stacked",
"autoTriggerTokens": 32000,
"stackedPipeline": [
{ "engine": "rtk", "intensity": "standard" },
{ "engine": "caveman", "intensity": "full" }
],
"rtkConfig": {
"enabled": true,
"intensity": "standard",
"applyToToolResults": true,
"applyToCodeBlocks": false,
"applyToAssistantMessages": false,
"enabledFilters": [],
"disabledFilters": [],
"maxLinesPerResult": 120,
"maxCharsPerResult": 12000,
"deduplicateThreshold": 3,
"customFiltersEnabled": true,
"trustProjectFilters": false,
"rawOutputRetention": "never",
"rawOutputMaxBytes": 1048576
}
}
```
`enabledFilters` and `disabledFilters` use filter ids, for example `test-vitest` or `git-diff`.
## API
| Route | Method | Purpose |
| ---------------------------------- | ------ | -------------------------------------------- |
| `/api/context/rtk/config` | GET | Read RTK config |
| `/api/context/rtk/config` | PUT | Update RTK config |
| `/api/context/rtk/filters` | GET | List filter catalog and load diagnostics |
| `/api/context/rtk/test` | POST | Preview RTK compression for one text payload |
| `/api/context/rtk/raw-output/[id]` | GET | Read retained redacted raw output |
| `/api/compression/preview` | POST | Preview any compression mode |
RTK test payload:
```json
{
"command": "npm test",
"text": "FAIL tests/example.test.ts\nAssertionError: expected true\nTest Files 1 failed",
"config": {
"intensity": "standard"
}
}
```
Compression preview payload:
```json
{
"mode": "stacked",
"messages": [
{
"role": "tool",
"content": "FAIL tests/example.test.ts\nAssertionError: expected true\nTest Files 1 failed"
}
],
"config": {
"rtkConfig": {
"rawOutputRetention": "failures"
}
}
}
```
Management routes require dashboard management auth or the matching API-key policy.
## Raw Output Recovery
RTK normally returns only compressed text. For debugging, `rawOutputRetention` can retain redacted
raw output:
| Value | Behavior |
| ---------- | ------------------------------------------------------- |
| `never` | Do not retain raw output |
| `failures` | Retain only likely failure output |
| `always` | Retain every compressed RTK raw output, after redaction |
Retained files are written under:
```txt
DATA_DIR/rtk/raw-output/
```
Secrets are redacted before persistence, including common bearer tokens, API keys, Slack tokens,
AWS access keys, and assignment-style `token=...`, `secret=...`, `password=...` values. Analytics
stores only the pointer id, size, and hash metadata.
## Verify Gate
The focused verify gate runs built-in inline filter tests without shelling out to external commands:
```bash
node --import tsx/esm --test tests/unit/compression/rtk-verify.test.ts
```
The broader RTK gate is:
```bash
node --import tsx/esm --test \
tests/unit/compression/rtk-*.test.ts \
tests/unit/compression/pipeline-integration.test.ts \
tests/unit/compression/context-compression-api.test.ts
```
Run the broad compression gate before release:
```bash
node --import tsx/esm --test \
tests/unit/compression/*.test.ts \
tests/golden-set/*.test.ts \
tests/integration/compression-pipeline.test.ts \
tests/unit/api/compression/compression-api.test.ts
```
## Extending RTK
1. Add or update a filter JSON file.
2. Include at least one `tests[]` sample that proves the important behavior.
3. Add a fixture under `tests/unit/compression/fixtures/rtk/` for new command families.
4. Add command detection coverage when introducing a new output class.
5. Run the verify and broad RTK gates.
6. If the filter is project-local, commit `.rtk/filters.json` and refresh `.rtk/trust.json` only after review.

160
Reasoning-Replay.md Normal file

@@ -0,0 +1,160 @@
# Reasoning Replay Cache
> **Source of truth:** `src/lib/db/reasoningCache.ts`, `open-sse/services/reasoningCache.ts`
> **Last updated:** 2026-05-13 — v3.8.0
OmniRoute captures assistant `reasoning_content` produced by thinking-mode models and replays it transparently on multi-turn requests when the upstream provider requires it. This eliminates the HTTP 400 errors that strict providers raise when a client's conversation history is missing the prior turn's reasoning.
## Why This Exists
Several thinking-mode providers reject a follow-up turn unless the **previous assistant message includes the original `reasoning_content`**. The upstream returns 400 with messages like:
```
Param Incorrect: The reasoning_content in the thinking mode must be passed back to the API.
```
But typical clients (Cursor, Cline, Roo Code, OpenAI SDK) strip `reasoning_content` from the history they replay. OmniRoute restores it from a server-side cache so the request the upstream sees is consistent. Issue #1628 introduced the hybrid memory/SQLite persistence so the cache survives process restarts.
## Architecture
```
Turn N (assistant generates):
→ response contains reasoning_content + tool_calls
→ cacheReasoningFromAssistantMessage() writes (memory + DB), keyed by every tool_call.id
→ forward response to client (which may or may not retain reasoning)
Turn N+1 (client sends follow-up):
→ translator detects: requiresReasoningReplay(provider, model) === true
→ for each assistant message with tool_calls and no reasoning_content:
lookupReasoning(toolCalls[0].id) → memory → DB
hit → msg.reasoning_content = cached; recordReplay()
miss → msg.reasoning_content = "" (legacy fallback for older DeepSeek)
→ upstream sees consistent history → no 400
```
Capture happens in `open-sse/handlers/chatCore.ts` (two sites, around lines 4093 and 4380). Replay happens in `open-sse/translator/index.ts` after schema coercion but before dispatch.
## Storage — Hybrid Memory + SQLite
The hot path uses an in-memory `Map` (LRU-by-creation) backed by a SQLite table for crash recovery and dashboard visibility.
| Layer | Implementation | Purpose |
| ------ | ---------------------------------------------- | -------------------------------------- |
| Memory | `Map` in `open-sse/services/reasoningCache.ts` | Fast lookups, evicts oldest at 2000 |
| DB | `reasoning_cache` table (`src/lib/db/`) | Persists across restarts, drives stats |
Writes go to both. Reads consult memory first, then fall back to DB (DB hits are promoted back into memory). DB failures are non-fatal — the in-memory cache continues to serve the hot path.
**Defaults:**
- TTL: `2h` (`TTL_MS = 2 * 60 * 60 * 1000`)
- Max memory entries: `2000` (`MAX_MEMORY_ENTRIES`)
- Eviction: oldest `createdAt` first
## Database Schema
Migration: `src/lib/db/migrations/033_create_reasoning_cache.sql`
```sql
CREATE TABLE IF NOT EXISTS reasoning_cache (
tool_call_id TEXT PRIMARY KEY,
provider TEXT NOT NULL,
model TEXT NOT NULL,
reasoning TEXT NOT NULL,
char_count INTEGER NOT NULL DEFAULT 0,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
expires_at INTEGER NOT NULL
);
```
Indexes: `expires_at`, `provider`, `model`, `created_at`. `expires_at` is stored as Unix epoch seconds; the SELECT layer normalizes legacy text values via `EXPIRES_AT_EPOCH_SQL`.
## Provider / Model Detection
Replay is enabled when `requiresReasoningReplay(provider, model)` returns `true`. The function checks two lists in `open-sse/services/reasoningCache.ts`.
**Provider IDs (exact match, case-insensitive):**
- `deepseek`
- `opencode-go`
- `siliconflow`
- `nebius`
- `deepinfra`
- `sambanova`
- `fireworks`
- `together`
- `xiaomi-mimo`
**Model regex patterns (case-insensitive):**
- `/deepseek-r1/i`
- `/deepseek-reasoner/i`
- `/deepseek-chat/i`
- `/kimi-k2/i`
- `/qwq/i`
- `/qwen.*think/i`
- `/glm.*think/i`
- `/^mimo[-.]?v\d/i`
Adding a new strict provider/model means appending to one of these lists and writing a unit test asserting replay injection. The PR description should cite the exact upstream 400 string that motivated the change.
## REST API
The cache exposes two endpoints under `src/app/api/cache/reasoning/route.ts`. Both require management authentication (`isAuthenticated` from `@/shared/utils/apiAuth`).
| Method | Endpoint | Description |
| ------ | --------------------------------------------------------- | -------------------------------------------------------- |
| GET | `/api/cache/reasoning` | Stats + paginated entries |
| GET | `/api/cache/reasoning?provider=deepseek&model=...&limit=` | Filtered listing (`limit` clamped to `[1, 200]`) |
| DELETE | `/api/cache/reasoning` | Clear everything (memory + DB) and reset hit/miss counts |
| DELETE | `/api/cache/reasoning?provider=deepseek` | Clear only entries for one provider |
| DELETE | `/api/cache/reasoning?toolCallId=call_abc` | Delete a single entry |
**GET response shape:**
```json
{
"stats": {
"memoryEntries": 12,
"dbEntries": 47,
"totalEntries": 47,
"totalChars": 138291,
"hits": 84,
"misses": 6,
"replays": 81,
"replayRate": "90.0%",
"byProvider": { "deepseek": { "entries": 32, "chars": 98412 } },
"byModel": { "deepseek-reasoner": { "entries": 32, "chars": 98412 } },
"oldestEntry": "2026-05-13T10:00:00.000Z",
"newestEntry": "2026-05-13T11:42:11.000Z"
},
"entries": [
{
"toolCallId": "call_abc",
"provider": "deepseek",
"model": "deepseek-reasoner",
"reasoning": "...",
"charCount": 3128,
"createdAt": "...",
"expiresAt": "..."
}
]
}
```
## Operational Notes
- **Cleanup:** `cleanupReasoningCache()` purges expired memory entries and runs `DELETE FROM reasoning_cache WHERE expires_at <= unixepoch('now')`. Health-check workers call this periodically.
- **Crash recovery:** After a restart, memory is empty but the DB still holds unexpired entries. The first lookup for a given `tool_call_id` is a DB hit; subsequent lookups are memory hits.
- **No reasoning, no cache:** `cacheReasoningFromAssistantMessage` returns `0` when the assistant message has no `reasoning_content` / `reasoning` field, so non-thinking responses cost nothing.
- **Non-strict providers:** When `requiresReasoningReplay` is `false` and the target format is OpenAI, the translator **strips** any `reasoning_content` field from outgoing messages — OpenAI Chat Completions does not accept it.
## See Also
- [RESILIENCE_GUIDE.md](../architecture/RESILIENCE_GUIDE.md) — circuit breakers, cooldowns, model lockouts
- [TROUBLESHOOTING.md](../guides/TROUBLESHOOTING.md) — diagnosing upstream 400s
- Source: `src/lib/db/reasoningCache.ts`, `open-sse/services/reasoningCache.ts`, `open-sse/translator/index.ts`
- Migration: `src/lib/db/migrations/033_create_reasoning_cache.sql`
- API route: `src/app/api/cache/reasoning/route.ts`
- Original issue: #1628

233
Release-Checklist.md Normal file

@@ -0,0 +1,233 @@
# Release Checklist
> **Last updated:** 2026-05-13 — v3.8.0
> Streamlined release flow that leverages Claude Code skills for automation.
## TL;DR
```bash
# 1. Bump version + generate CHANGELOG (skill)
/version-bump-cc patch # or minor/major
# 2. Run quality gate locally
npm run check # lint + tests
npm run test:coverage # full coverage gate (75/75/75/70)
# 3. Build & smoke
npm run build
npm run test:e2e # optional but recommended
# 4. Generate release (skill)
/generate-release-cc
# 5. Deploy (skill)
/deploy-vps-both-cc # or akamai-cc / local-cc
# 6. Capture release evidences (skill)
/capture-release-evidences-cc
```
## Detailed Checklist
### Pre-release
- [ ] All PRs targeted to this release are merged to `release/vX.Y.0`
- [ ] All open Linear/issue items for this version are closed or pushed to next milestone
- [ ] CI green on `release/vX.Y.0` branch
- [ ] No `TODO(release)` markers in code: `grep -r "TODO(release)" src/ open-sse/`
- [ ] Docker base image up to date (currently `node:24.15.0-trixie-slim`)
### Version & Changelog
- [ ] Run `/version-bump-cc <patch|minor|major>` (Claude Code skill)
- Bumps `package.json`, `electron/package.json`
- Regenerates `CHANGELOG.md` from git commits since last tag
- Updates README.md badges
- [ ] Manually review CHANGELOG.md and clean up commit messages if needed
- [ ] Ensure the latest semver section in `CHANGELOG.md` equals `package.json` version
- [ ] Keep `## [Unreleased]` as the first changelog section for upcoming work
- [ ] Update `docs/reference/openapi.yaml` → `info.version` must equal `package.json` version
### Code Quality
- [ ] `npm run lint` — 0 errors (warnings are pre-existing)
- [ ] `npm run typecheck:core` — clean
- [ ] `npm run typecheck:noimplicit:core` — clean (strict)
- [ ] `npm run check:cycles` — no circular deps
- [ ] `npm run check:any-budget:t11` — within budget
- [ ] `npm run check:route-validation:t06` — clean
- [ ] `npm run check:node-runtime` — supported floor met (`>=20.20.2 <21`, `>=22.22.2 <23`, `>=24.0.0 <25`)
### Testing
- [ ] `npm run test:unit` — pass
- [ ] `npm run test:vitest` — pass (MCP server, autoCombo, cache)
- [ ] `npm run test:coverage` — gate 75/75/75/70 satisfied (statements/lines/functions/branches)
- [ ] `npm run test:integration` — pass (if changes touch DB / handlers)
- [ ] `npm run test:e2e` — pass (UI changes)
- [ ] `npm run test:protocols:e2e` — pass (MCP/A2A changes)
- [ ] `npm run test:ecosystem` — pass
### Hooks (Husky validated)
Husky hooks live in `.husky/` and run automatically on git operations.
- **pre-commit:** `npx lint-staged + node scripts/check/check-docs-sync.mjs + npm run check:any-budget:t11`
- **pre-push:** currently disabled (commented out). When re-enabled, runs `npm run test:unit`.
- Run `npm run test:unit` manually before pushing release branches.
If a hook fails: fix the underlying issue, don't bypass with `--no-verify`.
### Conventional Commits
All release-bound commits must follow `type(scope): subject` format.
**Valid types:** `feat`, `fix`, `refactor`, `docs`, `test`, `chore`, `perf`, `style`, `ci`
**Valid scopes:** `db`, `sse`, `oauth`, `dashboard`, `api`, `cli`, `docker`, `ci`, `mcp`, `a2a`, `memory`, `skills`, `cloud-agent`, `guardrails`, `compression`, `auto-combo`, `resilience`, `providers`, `executors`, `translator`, `domain`, `authz`
Breaking changes: add `BREAKING CHANGE:` footer or `!` after the scope (e.g. `feat(api)!: drop /v0`).
### Documentation
- [ ] `npm run check:docs-sync` passes (auto-run by pre-commit)
- [ ] `npm run check:docs-all` passes (umbrella: docs-sync + docs-counts + env-doc-sync + deprecated-versions + doc-links)
- [ ] `npm run check:env-doc-sync` exits 0 — code ↔ `.env.example` ↔ `docs/reference/ENVIRONMENT.md` env contract is intact
- [ ] `npm run check:doc-links` exits 0 — no broken internal markdown references after restructuring
- [ ] `docs/architecture/ARCHITECTURE.md` reviewed for storage/runtime drift
- [ ] `docs/guides/TROUBLESHOOTING.md` reviewed for env var and operational drift
- [ ] If `.env.example` changed: `docs/reference/ENVIRONMENT.md` updated
- [ ] If new feature has a UI: `docs/guides/USER_GUIDE.md` mentions it
- [ ] If new feature has API: `docs/reference/API_REFERENCE.md` + `docs/reference/openapi.yaml` updated
- [ ] If new feature is a module: dedicated `docs/<MODULE>.md` exists
- [ ] If breaking change: `docs/guides/TROUBLESHOOTING.md` has migration note
### i18n
- [ ] `npm run i18n:check` exits 0 — translation state (`.i18n-state.json`) in sync with source docs (no drifted sources in strict mode; warn-mode advisory is acceptable for last-minute doc touch-ups, but should be 0 before tagging)
- [ ] `npm run i18n:check-ui-coverage` exits 0 — every UI locale at or above the 80% coverage floor
- [ ] `npm run i18n:sync-ui:dry` reports 0 missing keys across all 40 locales
- [ ] If source English docs changed, run `npm run i18n:run` (requires `OMNIROUTE_TRANSLATION_API_KEY` in `.env`) before tagging
- [ ] Translation contributions can be deferred to next release if minor (track in CHANGELOG)
### Database Migrations
- [ ] If `src/lib/db/migrations/` has new files:
- [ ] Each migration is idempotent (`CREATE TABLE IF NOT EXISTS`, etc.)
- [ ] Migrations wrapped in transactions
- [ ] Numbered correctly (no gaps in sequence)
- [ ] Test on fresh install: delete `~/.omniroute/omniroute.db` and run `npm run dev`
- [ ] Test on existing install: backup DB, run migration, verify schema
- [ ] WAL files (`-wal`, `-shm`) handled correctly if migration rewrites tables
### Provider Catalog (Zod-validated)
- [ ] `src/shared/constants/providers.ts` Zod schema valid at load time
- [ ] All providers have required fields (`id`, `label`, `kind`, etc.)
- [ ] `freeNote` provided for new free providers
- [ ] OAuth providers have `oauthConfig` registered in `src/lib/oauth/constants/oauth.ts`
- [ ] If new provider added: corresponding executor in `open-sse/executors/`
- [ ] If non-OpenAI format: translator in `open-sse/translator/`
- [ ] Models registered in `open-sse/config/providerRegistry.ts`
- [ ] Unit tests in `tests/unit/` cover provider classification and routing
### Desktop (Electron)
If `electron/` changed:
- [ ] `npm run electron:smoke:packaged` passes
- [ ] Builds tested for at least one of `:win`, `:mac`, `:linux`
- [ ] Code signing certs not expired (if signing)
- [ ] `electron/package.json` version matches root `package.json`
- [ ] Auto-update channel pointer updated if releasing to `stable`
### Artifact Validation
- [ ] `npm run build:cli` succeeds
- [ ] `npm run check:pack-artifact` clean — no `app.__qa_backup`, `scripts/scratch`, `package-lock.json`, or other local residue
- [ ] `npm run build` produces a working standalone Next.js bundle
### Tagging & Release
- [ ] Run `/generate-release-cc` (Claude Code skill):
- Creates tag `vX.Y.Z`
- Pushes tag and branch
- Opens GitHub Release with changelog body
- Attaches Electron installers (if built)
- [ ] Or manually:
```bash
git tag -a vX.Y.Z -m "Release vX.Y.Z"
git push origin vX.Y.Z
gh release create vX.Y.Z --notes-from-tag
```
### Deploy
- [ ] Use deploy skill that matches target:
- `/deploy-vps-local-cc` — local VPS (192.168.0.15)
- `/deploy-vps-akamai-cc` — Akamai VPS (69.164.221.35)
- `/deploy-vps-both-cc` — both
- [ ] Smoke test deployed instance:
- Open `/dashboard/health` → check version string matches release
- Run a `/v1/chat/completions` request against a known provider
- Verify `/api/monitoring/health` returns `CLOSED` circuit breakers
- Confirm MCP transports respond (`/mcp` HTTP, `/mcp-sse` SSE)
### Post-release
- [ ] Run `/capture-release-evidences-cc` (Claude Code skill)
- Captures WebP screenshots/recordings of new features
- Attaches to release notes / blog post
- [ ] Update GitHub Discussions / Discord with release announcement
- [ ] Open milestone for next version
- [ ] If critical: pin discussion or post in `news.json` for in-app banner
## v3.8.0+ checks
Before shipping any v3.8.x release, verify these additional items:
- [ ] `omniroute --tray` boots on macOS (systray2 installed into `~/.omniroute/runtime/`)
- [ ] `omniroute --tray` boots on Linux (requires DISPLAY; graceful error if not set)
- [ ] `omniroute --tray` boots on Windows (PowerShell NotifyIcon, no extra binaries)
- [ ] `omniroute config tray enable` creates autostart entry; disable removes it
- [ ] `npm install -g omniroute@<this-version>` runs postinstall without fatal exit
- [ ] `omniroute status` works with no `.env` (CLI token path, loopback only)
- [ ] `curl http://localhost:20128/api/shutdown` returns 401 (always-protected route)
- [ ] `curl -H "host: evil.com" http://localhost:20128/api/mcp/sse` returns 401 (loopback guard)
- [ ] SQLite runtime resolves to `bundled` on first run (bundled binary valid for platform)
- [ ] SQLite runtime falls back to `runtime` when `node_modules/better-sqlite3` is deleted
- [ ] Smart MCP filter compresses real `playwright-mcp browser_snapshot` output (≥50% reduction)
- [ ] All 10 `skills/omniroute*/SKILL.md` files are publicly fetchable via raw GitHub URL
- [ ] Onboarding wizard shows "How It Works" tier tour step on fresh setup
- [ ] Home dashboard tier coverage widget shows configured/active counts
---
## Rollback
If release has critical issue:
1. `gh release edit vX.Y.Z --prerelease` (marks as not latest)
2. `git tag -d vX.Y.Z && git push --delete origin vX.Y.Z` (only if not yet adopted by users)
3. Or: hotfix on `release/vX.Y.0` → patch release `vX.Y.(Z+1)`
4. Communicate in GitHub Discussions and Discord immediately
## Hard Rules
- Never commit directly to `main`
- Never use `git push --force` to `main` or `release/*` branches
- Never skip Husky hooks (`--no-verify`)
- Never commit secrets, credentials, or `.env` files
- Coverage must stay ≥75/75/75/70 (statements/lines/functions/branches)
- Always include or update tests when changing production code in `src/`, `open-sse/`, `electron/`, or `bin/`
## Automated Sync Check
Run the docs sync guard locally before opening a PR:
```bash
npm run check:docs-sync
```
CI also runs this check in `.github/workflows/ci.yml` (lint job).

521
Repository-Map.md Normal file

@@ -0,0 +1,521 @@
# Repository Map
> **One-line description for every directory and root file.**
> Last updated: 2026-05-13 — OmniRoute v3.8.0
>
> Use this map to navigate the codebase quickly. For deep dives, follow links to dedicated docs.
## Top-level tree
```
OmniRoute/
├── src/ # Next.js 16 application (UI + API routes + libs + domain + server)
├── open-sse/ # Streaming engine workspace (handlers, executors, translator, MCP server)
├── electron/ # Desktop wrapper (Electron 41 + electron-builder 26.10)
├── bin/ # CLI entry point and command handlers
├── scripts/ # Build, check, sync, and one-off scripts
├── docs/ # Public documentation (you are here)
├── tests/ # All test suites (unit, integration, e2e, protocols-e2e)
├── public/ # Next.js static assets, PWA manifest, service worker, icons
├── config/ # Static config files
├── images/ # Marketing / README image assets
├── .github/ # GitHub Actions workflows + issue templates + PR template
├── .husky/ # Git hooks (pre-commit, pre-push)
├── .claude/ # Claude Code slash commands (project-scoped)
├── .agents/ # Codex / generic agent workflows + skills (mirror of .claude/)
├── .vscode/ # VS Code workspace settings
├── _ideia/ # Planning notes (informal; not shipped)
├── _mono_repo/ # Historic subprojects (cloud, site, vscode-extension)
├── _references/ # Read-only reference clones from related OSS projects
├── _tasks/ # Per-release task tracking files (informal)
├── .issues/ # Local issue cache (gitignored)
├── .playwright-mcp/ # Playwright MCP test artifacts
├── coverage/ # c8 coverage output (gitignored)
├── logs/ # Runtime logs (gitignored)
├── node_modules/ # Dependencies (gitignored)
├── package/ # npm pack staging area (build artifact)
├── .next/ # Next.js build output (gitignored)
└── (root files — see below)
```
---
## Root files
| File | Purpose |
| ------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------- |
| **README.md** | Marketing landing page + quick start + feature matrix (see also `llm.txt`) |
| **CHANGELOG.md** | Per-release changelog (auto-generated by `/version-bump-cc` skill) |
| **LICENSE** | MIT license text |
| **CLAUDE.md** | Project rules for Claude Code agents (hard rules, conventions, scenarios) |
| **AGENTS.md** | Same as CLAUDE.md but for non-Claude AI agents (Codex, Cursor, etc.) |
| **GEMINI.md** | Concise rules for Gemini-based agents (subset of CLAUDE.md) |
| **CONTRIBUTING.md** | Contributor guide: setup, conventional commits, testing, PR flow |
| **SECURITY.md** | Vulnerability reporting policy, supported versions, threat model |
| **CODE_OF_CONDUCT.md** | Contributor Covenant — community behavior expectations |
| **llm.txt** | Plain-text landing optimized for LLM crawlers (SEO for AI assistants) |
| **Tuto_Qdrant.md** | Tutorial for enabling Qdrant vector memory — **integration currently dormant** (see banner; primary memory docs in `docs/frameworks/MEMORY.md`) |
| **package.json** | npm manifest, scripts, dependencies, engines, c8 coverage gate |
| **package-lock.json** | Locked dependency tree |
| **tsconfig.json** | Root TypeScript config |
| **tsconfig.typecheck-core.json** | Typecheck config for `src/` core |
| **tsconfig.typecheck-noimplicit-core.json** | Strict (`noImplicitAny`) typecheck |
| **tsconfig.tsbuildinfo** | TS incremental build cache (gitignored) |
| **next.config.mjs** | Next.js 16 build configuration (standalone output) |
| **next-env.d.ts** | Next.js auto-generated env types |
| **eslint.config.mjs** | ESLint flat config (rules per project area) |
| **prettier.config.mjs** | Prettier formatting rules |
| **postcss.config.mjs** | PostCSS config for Tailwind/CSS pipeline |
| **playwright.config.ts** | Playwright E2E test config |
| **vitest.config.ts** | Vitest config (default suite) |
| **vitest.mcp.config.ts** | Vitest config for MCP server / autoCombo / cache suites |
| **sonar-project.properties** | SonarQube/SonarCloud config (code quality) |
| **Dockerfile** | Multi-stage Docker build (builder → runner-base → runner-cli) |
| **docker-compose.yml** | Dev compose with 4 profiles (base, cli, host, cliproxyapi) + redis sidecar |
| **docker-compose.prod.yml** | Production compose (port 20130, redis, named volumes) |
| **.dockerignore** | Files excluded from Docker context |
| **fly.toml** | Fly.io deployment config (region `sin`, port 20128, /data volume) |
| **.env.example** | Template env file (815 lines, auto-copied to `.env` on first install) |
| **.gitignore** | Git ignore patterns |
| **.npmignore** | npm publish exclusion list |
| **.npmrc** | npm config (registry, lockfile policy) |
| **.node-version** | Node version pin (used by nvm-compatible tools) |
| **.nvmrc** | Node version pin for nvm |
---
## `src/` — Next.js application
```
src/
├── app/ # App Router (pages + API routes + status pages + landing)
├── lib/ # Core libraries / domain modules (~50 subdirs + ~30 top-level files)
├── domain/ # Pure domain logic (policy engine, fallback, cost, lockout, comboResolver, assessment)
├── server/ # Server-only modules (authz pipeline, cors, auth middleware) — cannot import from client
├── shared/ # Shared between server and client where safe (constants, types, validation, contracts, utils)
├── i18n/ # next-intl config + per-locale message JSON (30+ locales)
├── middleware/ # Next.js middleware (request enrichment, locale detection)
├── mitm/ # MITM proxy helpers (Linux cert install, antigravity stealth)
├── models/ # Model adapter glue (legacy shim)
├── scripts/ # In-tree maintenance scripts (e.g., backfillAggregation)
├── sse/ # Legacy SSE handlers/services (chat.ts, chatHelpers.ts, services/auth.ts)
├── store/ # Legacy in-memory store (being phased out for src/lib/db)
├── types/ # Shared TS type files
├── instrumentation.ts # Next.js telemetry hook (browser + edge)
├── instrumentation-node.ts # Node-only instrumentation
├── server-init.ts # Server bootstrap (DB migrations, jobs, cleanup)
└── proxy.ts # HTTP-proxy entry shim
```
### `src/app/` — App Router (Next.js 16)
| Path | Purpose |
| ---------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `app/api/v1/` | Public OpenAI-compat API (~25 sub-routes: chat, completions, embeddings, files, batches, audio, images, videos, music, rerank, moderations, search, ws, agents, accounts, providers, etc.) |
| `app/api/v1beta/` | Gemini-style API endpoints |
| `app/api/` (non-v1) | Management/admin routes (~60 directories: providers, combos, settings, mcp, a2a, evals, memory, skills, webhooks, compliance, resilience, monitoring, tunnels, cli-tools, etc.) |
| `app/a2a/` | A2A JSON-RPC 2.0 entry point (`POST /a2a`) |
| `app/.well-known/agent.json/` | A2A Agent Card (discovery) |
| `app/(dashboard)/dashboard/` | Dashboard UI pages (~30 pages: providers, combos, settings, memory, skills, webhooks, evals, audit, batch, cache, costs, health, system, etc.) |
| `app/docs/` | Embedded documentation viewer (renders `docs/*.md`) |
| `app/landing/` | Marketing landing page |
| `app/login/`, `forgot-password/`, `forbidden/` | Auth-related pages |
| `app/{400,401,403,408,429,500,502,503}/` | HTTP error pages |
| `app/maintenance/`, `offline/`, `status/`, `privacy/`, `terms/`, `callback/` | Static/status pages |
| `app/layout.tsx`, `page.tsx`, `manifest.ts`, `globals.css` | Root layout, home, PWA manifest, global CSS |
| `app/error.tsx`, `global-error.tsx`, `not-found.tsx`, `loading.tsx` | Error boundaries |
### `src/lib/` — Core libraries (~50 modules)
| Module | Purpose |
| ---------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `a2a/` | A2A protocol task manager, skills (5), streaming |
| `acp/` | CLI Agent Registry (local CLI discovery — see `docs/frameworks/AGENT_PROTOCOLS_GUIDE.md`) |
| `api/` | Shared API helpers (`requireManagementAuth`, validation) |
| `auth/` | Session, password hashing, token validation |
| `batches/` | OpenAI Batches API handlers |
| `catalog/` | Provider catalog Zod validation + capability resolution |
| `cloudAgent/` | Cloud Agents (Codex Cloud, Devin, Jules) — see `docs/frameworks/CLOUD_AGENT.md` |
| `combos/` | Combo resolution + reorder helpers |
| `compliance/` | Audit log + provider audit — see `docs/security/COMPLIANCE.md` |
| `compression/` | Compression engine glue (engines live in `open-sse/services/compression/`) |
| `config/` | Runtime config helpers |
| `db/` | 45+ domain DB modules + 55 migrations (always go through here for SQLite) |
| `display/` | UI formatting helpers (cost, latency, etc.) |
| `embeddings/` | Embeddings service helpers |
| `env/` | Env variable parsing + validation |
| `evals/` | Eval framework (suites, runner, runtime) — see `docs/frameworks/EVALS.md` |
| `guardrails/` | PII masker, prompt injection, vision bridge — see `docs/security/GUARDRAILS.md` |
| `jobs/` | Background jobs (cron-like) |
| `memory/` | Conversational memory (SQLite FTS5 + Qdrant) — see `docs/frameworks/MEMORY.md` |
| `monitoring/` | Health checks, metrics emission |
| `oauth/` | OAuth flows for 14 providers (claude, codex, antigravity, cursor, github, gemini, kimi-coding, kilocode, cline, qwen, kiro, qoder, gitlab-duo, windsurf) |
| `plugins/` | Plugin registry |
| `promptCache/` | Anthropic-style prompt cache breakpoints |
| `skills/` | Skills framework (built-in + marketplace + SkillsSH) — see `docs/frameworks/SKILLS.md` |
| `webhookDispatcher.ts` | HMAC webhook delivery — see `docs/frameworks/WEBHOOKS.md` |
| `cloudflaredTunnel.ts`, `ngrokTunnel.ts` | Tunnel managers — see `docs/ops/TUNNELS_GUIDE.md` |
| `oneproxySync.ts`, `oneproxyRotator.ts` | 1proxy free proxy marketplace — see `docs/ops/PROXY_GUIDE.md` |
| `cloudSync.ts`, `initCloudSync.ts` | Optional cloud sync of state |
| `localDb.ts` | Re-export barrel for db modules (no logic — re-exports only) |
| `cacheLayer.ts`, `idempotencyLayer.ts` | Request caching + idempotency |
| (~30 more top-level files) | Specialized helpers (logEnv, modelsDevSync, piiSanitizer, etc.) |
### `src/db/` — Database (45+ modules + 55 migrations)
| Subdir | Purpose |
| ---------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `db/core.ts` | `getDbInstance()` singleton with WAL journaling |
| `db/migrations/` | 55 versioned SQL files (idempotent, transactional, numbered `001`..`055`) |
| `db/<domain>.ts` | One module per domain: providers, combos, apiKeys, users, sessions, usage, audit*log, webhooks, skills, memory_entries, cloud_agent_tasks, evals*\*, reasoning_cache, etc. |
### `src/domain/`
| Module | Purpose |
| ---------------------- | ----------------------------------------------------------------------- |
| `policy.ts` | Policy engine |
| `fallbackPolicy.ts` | Fallback decision tree |
| `costRules.ts` | Cost calculation rules |
| `lockoutPolicy.ts` | Model/connection lockout policy |
| `tagRouter.ts` | Tag-based routing |
| `comboResolver.ts` | Combo resolution (used by combo engine) |
| `modelAvailability.ts` | Per-model availability check |
| `assessment/` | Model assessment (Phase 1 of RFC-AUTO-ASSESSMENT — see `docs/archive/`) |
### `src/server/`
| Module | Purpose |
| -------- | ---------------------------------------------------------------------------------------------------- |
| `authz/` | Authorization pipeline: `classify` → `policies` → `enforce` — see `docs/architecture/AUTHZ_GUIDE.md` |
| `cors/` | CORS configuration |
| `auth/` | Session middleware |
### `src/shared/`
| Module | Purpose |
| -------------------------------- | ---------------------------------------------------------------------- |
| `constants/providers.ts` | **177 providers** with Zod validation (source of truth) |
| `constants/cliTools.ts` | External CLI tool registry |
| `constants/routingStrategies.ts` | **14 routing strategies** with priorities |
| `constants/publicApiRoutes.ts` | Routes that require Bearer (vs management) auth |
| `constants/upstreamHeaders.ts` | Header denylist for upstream requests |
| `validation/schemas.ts` | ~80 Zod schemas (single source of truth for API contracts) |
| `validation/helpers.ts` | Zod validation helpers (`validateBody`, etc.) |
| `types/` | Shared TS types |
| `contracts/` | Public API contracts (consumed by `files:` in `package.json`) |
| `utils/circuitBreaker.ts` | Provider circuit breaker (see `docs/architecture/RESILIENCE_GUIDE.md`) |
| `utils/apiAuth.ts` | API key validation, scope checking |
| `utils/fetchTimeout.ts` | Timeout/abort wrappers for upstream fetch |
---
## `open-sse/` — Streaming Engine Workspace
Separate npm workspace (`@omniroute/open-sse`). Handles request processing + provider execution.
```
open-sse/
├── handlers/ # 15 files (11 handlers + 4 helpers): chatCore, responsesHandler, embeddings, audio, image, video, music, rerank, moderations, search, etc.
├── executors/ # 31 provider-specific executors (extend BaseExecutor)
├── translator/ # Format converters (9 request, 8 response, 9 helpers)
├── transformer/ # Responses API ↔ Chat Completions (TransformStream)
├── services/ # ~80+ service modules (combo, accountFallback, autoCombo, reasoningCache, claude code/chatgpt stealth, modelDeprecation, taskAwareRouter, workflowFSM, etc.)
├── mcp-server/ # MCP server (37 tools, 3 transports, ~13 scopes)
├── config/ # Provider/model registries, header config, model aliases
├── utils/ # TLS client, proxy fetch/dispatcher, network helpers
├── index.ts # Workspace entry
├── package.json # Workspace manifest
├── tsconfig.json # Workspace TS config
└── types.d.ts # Workspace type declarations
```
### `open-sse/mcp-server/`
| Path | Purpose |
| --------------------------- | ------------------------------------------------------------------------------ |
| `server.ts` | MCP server lifecycle (stdio + HTTP transports) |
| `httpTransport.ts` | HTTP Streamable + SSE transports (`/api/mcp/sse`, `/api/mcp/stream`) |
| `audit.ts` | Audit logging to `mcp_tool_audit` table |
| `scopeEnforcement.ts` | Per-tool scope validation |
| `runtimeHeartbeat.ts` | Health heartbeat to `DATA_DIR/runtime/mcp-heartbeat.json` |
| `descriptionCompressor.ts` | Compress tool description metadata to save context |
| `schemas/tools.ts` | 30 base tool definitions + scopes |
| `tools/advancedTools.ts` | Advanced tool implementations |
| `tools/memoryTools.ts` | 3 memory tools (search/add/clear) |
| `tools/skillTools.ts` | 4 skill tools (list/enable/execute/executions) |
| `tools/compressionTools.ts` | 5 compression tools |
| `README.md` | Internal MCP server README (cross-linked from `docs/frameworks/MCP-SERVER.md`) |
---
## `electron/` — Desktop Wrapper
| File | Purpose |
| ---------------- | --------------------------------------------------------------------------------- |
| `main.js` | Electron main process (BrowserWindow, embedded Next.js server, tray, auto-update) |
| `preload.js` | IPC bridge (contextBridge → `window.omniroute`) |
| `package.json` | electron-builder config + Electron 41 + electron-builder 26.10 deps |
| `assets/` | App icons (Windows .ico, macOS .icns, Linux .png) |
| `dist-electron/` | Build output (gitignored) |
| `types.d.ts` | Type declarations for renderer bridge |
| `README.md` | Internal Electron README (see also `docs/guides/ELECTRON_GUIDE.md`) |
---
## `bin/` — CLI
| File | Purpose |
| ----------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------- |
| `omniroute.mjs` | Main CLI entry — `omniroute serve`, `omniroute setup`, `omniroute doctor`, `omniroute providers`, `omniroute combos`, etc. |
| `reset-password.mjs` | Standalone password reset CLI |
| `cli/commands/setup.mjs` | Interactive + non-interactive setup wizard |
| `cli/commands/doctor.mjs` | System health diagnostics (8+ checks) |
| `cli/commands/providers.mjs` | Provider list/test/validate |
| `cli/{args,data-dir,encryption,io,provider-catalog,provider-store,provider-test,settings-store,sqlite}.mjs` | CLI helper modules |
| `cli/tray/tray.ts` | System tray integration (cross-platform: NotifyIcon on Windows, systray2 on macOS/Linux) |
| `cli/tray/tray.ps1` | PowerShell NotifyIcon backend (Windows, zero new binaries) |
| `cli/tray/autostart.ts` | Cross-platform autostart (LaunchAgent / .desktop / registry) |
| `cli/runtime/sqliteRuntime.mjs` | 5-step SQLite driver resolution chain (bundled → runtime → lazy-install → node:sqlite → sql.js) |
| `cli/runtime/magicBytes.mjs` | Binary magic-byte validation (ELF / Mach-O / Mach-O fat / PE) |
| `cli/runtime/index.mjs` | `warmUpRuntimes()` — pre-resolves drivers at postinstall / first startup |
| `nodeRuntimeSupport.mjs` | Validate supported Node.js version on install |
---
## `skills/` — Public Agent Skills
| File | Purpose |
| ---------------------------- | ---------------------------------------------------------------------------------- |
| `skills/omniroute*/SKILL.md` | 10 skill manifests for external AI agents (Claude Desktop, ChatGPT, Cursor, Cline) |
---
## `scripts/` — Build & Check Scripts
| Script | Purpose |
| ----------------------------------- | -------------------------------------------------------------------------- |
| `run-next.mjs` | Dev/start runner with env hydration |
| `build-next-isolated.mjs` | Standalone build (Next.js 16 standalone) |
| `prepublish.ts` | Package preparation before `npm pack` |
| `postinstall.mjs` | Auto-create `.env` from `.env.example` on first install |
| `sync-env.mjs` | Re-sync `.env` keys with `.env.example` |
| `check-cycles.mjs` | Detect circular dependencies |
| `check-route-validation.mjs` | Validate all API routes have Zod validation |
| `check-t11-any-budget.mjs` | Enforce explicit `any` budget per file |
| `check-docs-sync.mjs` | Validate docs version sync (existing pre-commit) |
| **`check-env-doc-sync.mjs`** | NEW: cross-check env vars in code vs `.env.example` vs `ENVIRONMENT.md` |
| **`check-docs-counts-sync.mjs`** | NEW: validate counts (executors, strategies, OAuth, A2A skills) match docs |
| **`check-deprecated-versions.mjs`** | NEW: flag stale versions/dates in docs |
| `check-supported-node-runtime.ts` | Validate current Node version is supported |
| `check-pr-test-policy.mjs` | Enforce "tests required" rule on production code changes |
| **`gen-provider-reference.ts`** | NEW: auto-generate `docs/reference/PROVIDER_REFERENCE.md` from catalog |
| `generate-docs-index.mjs` | Build `src/app/docs/lib/docs-auto-generated.ts` from `docs/*.md` |
| `i18n/generate-multilang.mjs` | Translate UI strings + docs via Google Translate |
| `i18n_autotranslate.py` | LLM-based doc translation pipeline |
| `validate_translation.py` | Per-locale translation validation |
| `check_translations.py` | Code-side i18n key check |
| `run-playwright-tests.mjs` | Playwright E2E runner |
| `run-protocol-clients-tests.mjs` | MCP/A2A E2E runner |
| `run-ecosystem-tests.mjs` | Ecosystem (provider integration) tests |
| `test-report-summary.mjs` | Generate coverage summary markdown |
| `smoke-electron-packaged.mjs` | Smoke-test packaged Electron build |
| `native-binary-compat.mjs` | Validate native deps (`better-sqlite3`) match Electron's Node |
| `validate-pack-artifact.ts` | Validate npm pack output |
| `responses-ws-proxy.mjs` | WebSocket bridge for Codex Responses API |
| `v1-ws-bridge.mjs` | WebSocket bridge for `/api/v1/ws` endpoint |
| `standalone-server-ws.mjs` | Standalone WS server runner |
| `system-info.mjs` | Print system/runtime info for support |
| `healthcheck.mjs` | One-shot health check (used by Docker HEALTHCHECK) |
| `uninstall.mjs` | Clean uninstall script |
---
## `docs/` — Public Documentation (44 files + 4 subdirs)
### Top-level guides
| Doc | Purpose |
| --------------------------- | ------------------------------------------------------------------------------------- |
| `ARCHITECTURE.md` | High-level architecture, subsystem map, dashboard surface |
| `CODEBASE_DOCUMENTATION.md` | Engineering reference: directories, modules, conventions |
| `FEATURES.md` | Feature matrix with v3.8 highlights |
| `USER_GUIDE.md` | End-user manual (setup, models, combos, CLIs, audio, etc.) |
| `API_REFERENCE.md` | API endpoint reference with auth model |
| `openapi.yaml` | OpenAPI 3.0 spec (121 paths) |
| `SETUP_GUIDE.md` | Install methods (npm, npx, Docker, Electron, Termux, source) |
| `ENVIRONMENT.md` | All env vars (~219 used in code, ~810 lines `.env.example`) |
| `TROUBLESHOOTING.md` | Common errors + v3.8.0 known issues |
| `RELEASE_CHECKLIST.md` | Full release flow (skills, husky, conventional commits, deploy) |
| `COVERAGE_PLAN.md` | Coverage goals and current state |
| `FREE_TIERS.md` | Curated free-tier providers (48+ free + 11 OAuth) |
| `CLI-TOOLS.md` | External CLI integrations + Internal OmniRoute CLI |
| `I18N.md` | i18n architecture, adding a language, 30 locales |
| `UNINSTALL.md` | Clean uninstall steps |
| `PROVIDER_REFERENCE.md` | **Auto-generated** catalog of 177 providers (regen: `npm run gen:provider-reference`) |
### Subsystem deep-dives
| Doc | Purpose |
| -------------------------- | ------------------------------------------------------------------- |
| `MCP-SERVER.md` | MCP server: 37 tools, 3 transports, ~13 scopes, REST endpoints |
| `A2A-SERVER.md` | A2A v0.3: JSON-RPC, 5 skills, REST helpers, agent card |
| `AGENT_PROTOCOLS_GUIDE.md` | Unified guide: A2A vs ACP vs Cloud Agents |
| `CLOUD_AGENT.md` | Codex Cloud / Devin / Jules orchestration |
| `SKILLS.md` | Skills framework (built-in + marketplace + SkillsSH + sandbox) |
| `MEMORY.md` | Memory system (SQLite FTS5 + Qdrant) |
| `EVALS.md` | Eval framework (suites, runs, rubrics) |
| `GUARDRAILS.md` | PII masker, prompt injection, vision bridge |
| `COMPLIANCE.md` | Audit log, retention, noLog opt-out |
| `WEBHOOKS.md` | HMAC-signed webhook delivery |
| `REASONING_REPLAY.md` | Hybrid memory/SQLite cache for `reasoning_content` |
| `AUTHZ_GUIDE.md` | Authorization pipeline (`classify` → `policies` → `enforce`) |
| `RESILIENCE_GUIDE.md` | Circuit breaker + cooldown + model lockout |
| `STEALTH_GUIDE.md` | TLS fingerprinting (JA3/JA4), Claude Code CCH, MITM cert |
| `AUTO-COMBO.md` | Auto Combo engine (9-factor scoring, 4 mode packs, virtual factory) |
### Compression
| Doc | Purpose |
| ------------------------------- | ---------------------------------------- |
| `COMPRESSION_GUIDE.md` | Overview of compression modes + roadmap |
| `COMPRESSION_ENGINES.md` | Caveman + RTK engines, registry contract |
| `COMPRESSION_RULES_FORMAT.md` | Caveman rule pack JSON schema |
| `COMPRESSION_LANGUAGE_PACKS.md` | Per-language rule pack inventory |
| `RTK_COMPRESSION.md` | RTK declarative pipeline (49 filters) |
### Deployment
| Doc | Purpose |
| ---------------------------- | ----------------------------------------------------------------- |
| `DOCKER_GUIDE.md` | Docker build, profiles (base/cli/host/cliproxyapi), Redis sidecar |
| `VM_DEPLOYMENT_GUIDE.md` | Generic VM/VPS deployment (Ubuntu/Debian + nginx + systemd) |
| `FLY_IO_DEPLOYMENT_GUIDE.md` | Fly.io deployment (currently Chinese-only) |
| `TERMUX_GUIDE.md` | Android headless via Termux |
| `PWA_GUIDE.md` | Progressive Web App install + service worker |
| `ELECTRON_GUIDE.md` | Desktop app build + sign + distribute |
| `TUNNELS_GUIDE.md` | Cloudflared + ngrok + Tailscale Funnel |
| `PROXY_GUIDE.md` | 4-level outbound proxy + 1proxy marketplace |
### Subdirectories
| Subdir | Purpose |
| ------------------------- | ------------------------------------------------------------------------------------- |
| `docs/archive/` | Archived/historical docs (e.g., `RFC-AUTO-ASSESSMENT-DRAFT.md` — superseded by EVALS) |
| `docs/i18n/` | Localized doc translations (~40 locales) |
| `docs/screenshots/` | Image assets for guides |
| `docs/superpowers/plans/` | Implementation plans (generated by `superpowers:writing-plans` skill) |
---
## `tests/` — Test Suites
| Subdir | Type | Runner |
| ---------------------- | --------------------------------------- | --------------------------------------- |
| `tests/unit/` | Unit tests (~500 files, fastest) | Node native test runner |
| `tests/integration/` | Multi-module + DB integration tests | Node native test runner (concurrency 1) |
| `tests/e2e/` | UI + workflow E2E | Playwright |
| `tests/protocols-e2e/` | MCP + A2A real-client E2E | Custom protocol clients |
| `tests/ecosystem/` | Provider integration (network-touching) | Node native test runner |
---
## `public/` — Static Assets
| Path | Purpose |
| ------------------- | ---------------------------------------------------------------- |
| `public/` (root) | Favicons, robots.txt, manifest, service worker, marketing images |
| `public/providers/` | Provider logo PNG/SVG (used in dashboard) |
---
## `config/` — Static Configs
Shipped configuration templates and sample files (referenced by setup wizard).
---
## `.github/` — GitHub Integration
| Path | Purpose |
| ---------------------------------- | -------------------------------------------------------------- |
| `.github/workflows/` | GitHub Actions CI/CD workflows (lint, test, coverage, release) |
| `.github/ISSUE_TEMPLATE/` | Bug/feature issue templates |
| `.github/PULL_REQUEST_TEMPLATE.md` | PR template |
| `.github/dependabot.yml` | Dependency update config |
---
## `.husky/` — Git Hooks
| File | Purpose |
| ------------ | ----------------------------------------------------------------- |
| `pre-commit` | Runs `lint-staged + check-docs-sync + check:any-budget:t11` |
| `pre-push` | Currently disabled (commented). Run `npm run test:unit` manually. |
| `_/` | Husky internals |
---
## `.claude/` — Claude Code Slash Commands
| File | Purpose |
| ----------------------------------------------------------------- | -------------------------------------------------- |
| `commands/version-bump-cc.md` | `/version-bump-cc` — bump version + auto-changelog |
| `commands/generate-release-cc.md` | `/generate-release-cc` — full release workflow |
| `commands/deploy-vps-{local,akamai,both}-cc.md` | Deploy to VPS |
| `commands/capture-release-evidences-cc.md` | Browser-record new features as WebP |
| `commands/review-{prs,discussions}-cc.md` | Triage GitHub PRs/discussions |
| `commands/{issue-triage,resolve-issues,implement-features}-cc.md` | Issue workflows |
| `settings.local.json` | Per-project Claude Code settings |
---
## `.agents/` — Generic Agent Workflows (Codex / Cursor / etc.)
| Path | Purpose |
| ------------------------ | ------------------------------------------------------- |
| `workflows/*-ag.md` | 11 workflow definitions (mirror of `.claude/commands/`) |
| `skills/<name>/SKILL.md` | 9 skill definitions with Codex Execution Notes |
> **Note:** Workflows and commands are currently identical byte-by-byte. If `.agents/` is meant to target a different agent runtime (Codex), the variants need to diverge meaningfully.
---
## `_ideia/`, `_mono_repo/`, `_references/`, `_tasks/` — Out-of-tree
These underscore-prefixed directories hold non-shipping content:
- **`_ideia/`** — design notes (defer / notfit / viable categories)
- **`_mono_repo/`** — historic subprojects (omnirouteCloud, omnirouteSite, vscode-extension)
- **`_references/`** — read-only clones of related OSS projects (LiteLLM, 9router, ClawRouter, CLIProxyAPI, modelrelay, new-api, etc.) for cross-reference during development
- **`_tasks/`** — per-release task tracking files (informal)
Not included in `npm pack` output. See `.npmignore`.
---
## Generated / Gitignored
| Path | Purpose |
| ---------------------- | ----------------------------- |
| `node_modules/` | npm dependencies |
| `.next/` | Next.js build output |
| `coverage/` | c8 coverage reports |
| `logs/` | Runtime logs |
| `package/` | npm pack staging |
| `.playwright-mcp/` | Playwright MCP test artifacts |
| `.issues/` | Local issue cache |
| `tsconfig.tsbuildinfo` | TS incremental cache |
---
## Navigation tips
- **New contributor?** Read `CONTRIBUTING.md` → `CLAUDE.md` → `docs/architecture/ARCHITECTURE.md` → `docs/architecture/CODEBASE_DOCUMENTATION.md`.
- **Adding a provider?** Follow `docs/architecture/ARCHITECTURE.md § Adding a New Provider` + cross-check `docs/reference/PROVIDER_REFERENCE.md`.
- **Adding a route?** `docs/architecture/ARCHITECTURE.md § Adding a New API Route` + `src/shared/validation/schemas.ts`.
- **Adding an MCP tool?** `docs/frameworks/MCP-SERVER.md § Adding a Tool`.
- **Adding an A2A skill?** `docs/frameworks/A2A-SERVER.md § Adding a New Skill`.
- **Running locally?** `docs/guides/SETUP_GUIDE.md`.
- **Deploying?** `docs/guides/DOCKER_GUIDE.md` / `docs/ops/VM_DEPLOYMENT_GUIDE.md` / `docs/ops/FLY_IO_DEPLOYMENT_GUIDE.md`.
- **Releasing?** `docs/ops/RELEASE_CHECKLIST.md` (and `/generate-release-cc` Claude Code skill).

142
Resilience-Guide.md Normal file

@@ -0,0 +1,142 @@
# Resilience Guide
OmniRoute has three distinct but related resilience mechanisms. Each has a different scope and purpose. Keep them separate when debugging routing behavior.
![3-layer resilience model](../diagrams/exported/resilience-3layers.svg)
> Source: [diagrams/resilience-3layers.mmd](../diagrams/resilience-3layers.mmd)
## 1. Provider Circuit Breaker
**Scope:** entire provider (e.g., `glm`, `openai`, `anthropic`).
**Purpose:** stop sending traffic to a provider that is repeatedly failing at the upstream/service level.
**Implementation:**
- Core class: `src/shared/utils/circuitBreaker.ts`
- Wiring: `src/sse/handlers/chatHelpers.ts`, `src/sse/handlers/chat.ts`
- Status API: `GET /api/monitoring/health`
- Reset API: `POST /api/resilience/reset`
- Wrappers: `open-sse/services/accountFallback.ts`
- DB table: `domain_circuit_breakers`
**States:**
- `CLOSED` — normal traffic allowed
- `OPEN` — provider temporarily blocked; combo routing skips it
- `HALF_OPEN` — reset timeout elapsed; probe request allowed
**Defaults (`open-sse/config/constants.ts`):**
| Class | Threshold | Reset timeout |
| ------- | ---------- | ------------- |
| OAuth | 3 failures | 60s |
| API-key | 5 failures | 30s |
| Local | 2 failures | 15s |
**Trip codes:** only provider-level statuses `[408, 500, 502, 503, 504]`. Do NOT trip for account-level errors (most 401/403/429 — those belong to cooldown or lockout).
**Lazy recovery:** when `OPEN` expires, `getStatus()`, `canExecute()`, `getRetryAfterMs()` refresh state to `HALF_OPEN`. No background timer needed.
---
## 2. Connection Cooldown
**Scope:** single provider connection/account/key.
**Purpose:** skip one bad key while other connections for the same provider keep serving.
**Implementation:**
- Mark unavailable: `src/sse/services/auth.ts::markAccountUnavailable()`
- Selection: `getProviderCredentials*` in same file
- Cooldown calc: `open-sse/services/accountFallback.ts::checkFallbackError()`
- Settings: `src/lib/resilience/settings.ts`
**Fields per connection:**
- `rateLimitedUntil` — timestamp until cooldown expires
- `testStatus: "unavailable"`
- `lastError`, `lastErrorType`, `errorCode`
- `backoffLevel` — exponential backoff counter
**Default cooldowns:**
- OAuth base: 5s
- API-key base: 3s
- API-key 429: prefers upstream `Retry-After`/reset headers/parseable reset text
- Backoff: `baseCooldownMs * 2 ** failureIndex`
**Anti-thundering-herd guard:** prevents concurrent failures from over-extending cooldown or double-incrementing `backoffLevel`.
**Terminal states (NOT cooldowns):**
- `banned`
- `expired`
- `credits_exhausted`
These persist until credentials change or an operator resets them. Do not overwrite terminal states with transient cooldown state.
**Lazy recovery:** when `rateLimitedUntil` is past, connection becomes eligible again. On successful use, `clearAccountError()` clears all error fields.
---
## 3. Model Lockout
**Scope:** provider + connection + model triple.
**Purpose:** avoid disabling a whole connection when only one model is unavailable or quota-limited.
**Examples:**
- Per-model quota providers returning 429
- Local providers returning 404 for one missing model
- Provider-specific mode/model permission failures (e.g., Grok modes)
**Implementation:** `open-sse/services/accountFallback.ts` — `lockModel()`, `clearModelLock()`, `getAllModelLockouts()`.
### Model Cooldowns Dashboard (v3.8.0)
UI: Settings → Model Cooldowns (`src/app/(dashboard)/dashboard/settings/components/ModelCooldownsCard.tsx`)
Lists active lockouts with: provider, connection, model, reason, expiresAt. Operators can manually re-enable a model from the card.
**REST API:**
- `GET /api/resilience/model-cooldowns` — list active lockouts
- `DELETE /api/resilience/model-cooldowns` — manual re-enable. Body: `{provider, connection, model}`. Auth: management.
---
## Other Resilience Features
- **14 routing strategies** (priority, weighted, round-robin, context-relay, fill-first, p2c, random, least-used, cost-optimized, reset-aware, strict-random, auto, lkgp, context-optimized) — see [AUTO-COMBO.md](../routing/AUTO-COMBO.md).
- **Reset-aware routing** (v3.8.0) — prioritizes connections by quota reset time.
- **Background mode degradation** — Responses API `background: true` degraded to sync with warning.
- **Dynamic tool limit detection** — backs off providers when tool count limits hit.
---
## Debugging
- All keys for a provider skipped → check both circuit breaker state AND each connection's `rateLimitedUntil`/`testStatus`.
- Provider permanently excluded after reset window → code reading raw `state` instead of `getStatus()`/`canExecute()`.
- One key fails, others should work → prefer connection cooldown over circuit breaker.
- Only one model fails → prefer model lockout over connection cooldown.
- State should self-recover but doesn't → check for future timestamp + read path that refreshes expired state. Permanent statuses require manual changes.
---
## TLS Fingerprinting & Stealth
Provider-specific stealth (JA3/JA4, CCH, obfuscation) is separately documented — see [STEALTH_GUIDE.md](../security/STEALTH_GUIDE.md).
---
## See Also
- [Architecture Guide](./ARCHITECTURE.md) — System architecture and internals
- [User Guide](../guides/USER_GUIDE.md) — Providers, combos, CLI integration
- [Auto-Combo Engine](../routing/AUTO-COMBO.md) — 6-factor scoring, mode packs

135
Route-Guard-Tiers.md Normal file

@@ -0,0 +1,135 @@
# Route Guard Tiers
## Overview
All OmniRoute management API routes are classified into one of three protection
tiers. Classification is static, defined in `src/server/authz/routeGuard.ts`,
and evaluated before any other auth branch runs.
## Tiers
### Tier 1 — LOCAL_ONLY
**Enforced by:** `isLocalOnlyPath(path)` → loopback host check
**Bypass:** None by default. Narrow carve-out for paths in
`LOCAL_ONLY_MANAGE_SCOPE_BYPASS_PREFIXES` when the request carries a valid
API key with the `manage` scope (see [Manage-scope carve-out](#manage-scope-carve-out)).
These routes spawn child processes or execute runtime code. Exposing them to
non-loopback traffic would allow an attacker who obtained a valid JWT (e.g.,
via a Cloudflared/Ngrok tunnel) to trigger process spawning — a known CVE
class (GHSA-fhh6-4qxv-rpqj).
| Prefix | Reason | Bypassable by `manage`? |
| ------------------------- | -------------------------------------------------- | ----------------------- |
| `/api/mcp/` | MCP server — spawns stdio bridges and SSE handlers | Yes |
| `/api/cli-tools/runtime/` | CLI tool runtime — executes arbitrary plugin code | No (strict-loopback) |
**Response on violation:** `403 LOCAL_ONLY`
#### Manage-scope carve-out
A subset of LOCAL_ONLY paths MAY also be accessed from non-loopback if and
only if the request carries an `Authorization: Bearer <api-key>` whose
metadata includes the `manage` scope (or `admin`). The carve-out is gated
explicitly per-path via `LOCAL_ONLY_MANAGE_SCOPE_BYPASS_PREFIXES` so the
default for any new LOCAL_ONLY path remains strict-loopback. Unauthenticated
requests and requests with non-manage keys are still rejected with
`403 LOCAL_ONLY`.
Today the only bypassable prefix is `/api/mcp/`. `/api/cli-tools/runtime/`
is intentionally excluded because it can spawn arbitrary subprocesses, which
is the exact CVE class the LOCAL_ONLY tier exists to prevent.
| Request | Path | Result |
| ------------------------------------------- | -------------------------- | ------------------- |
| Non-loopback, no Bearer | `/api/mcp/*` | 403 LOCAL_ONLY |
| Non-loopback, Bearer with `manage` scope | `/api/mcp/*` | Allow |
| Non-loopback, Bearer without `manage` scope | `/api/mcp/*` | 403 LOCAL_ONLY |
| Non-loopback, Bearer with `manage` scope | `/api/cli-tools/runtime/*` | 403 LOCAL_ONLY |
| Loopback, any/no Bearer | any LOCAL_ONLY | Allow (gate passes) |
### Tier 2 — ALWAYS_PROTECTED
**Enforced by:** `isAlwaysProtectedPath(path)` → skip `requireLogin=false` bypass
**Bypass:** None when `requireLogin=false`; JWT always required
These routes are destructive or irreversible. Allowing them in a "no-password"
install would mean anyone on the same LAN could wipe the database or kill the
server process.
| Path | Reason |
| ------------------------ | --------------------------------- |
| `/api/shutdown` | Terminates the server process |
| `/api/settings/database` | Database export, import, and wipe |
**Response on violation:** `401 Authentication required`
### Tier 3 — MANAGEMENT (default)
All other management routes. Auth required unless `requireLogin=false` is
configured. CLI tokens can authenticate these routes (loopback + valid HMAC).
## Evaluation order
```
managementPolicy.evaluate(ctx)
1. isLocalOnlyPath(path)?
→ loopback → fall through
→ non-loopback, manage-scope Bearer
AND isLocalOnlyBypassableByManageScope → allow (management_key)
→ otherwise → reject 403 LOCAL_ONLY
2. isInternalModelSyncRequest(ctx)?
→ allow (system)
3. hasValidCliToken(headers)?
→ allow (cli) [loopback + timingSafeEqual HMAC check]
4. isAlwaysProtectedPath(path) or requireLogin=true?
→ isDashboardSessionAuthenticated?
→ allow (dashboard_session)
→ manage-scope Bearer on a non-bypassable path?
→ allow (management_key)
→ reject 401/403
5. requireLogin=false?
→ allow (anonymous)
```
Step 1's manage-scope branch is the only authenticated path that can satisfy a
LOCAL_ONLY route; the auth-backend failure mode returns 503 (not 403) so an
expired DB doesn't silently downgrade to "deny".
## Adding a new spawn-capable route
1. Add the path prefix to `LOCAL_ONLY_API_PREFIXES` in
`src/server/authz/routeGuard.ts`
2. Add a test in `tests/unit/authz/routeGuard.test.ts` asserting that
`isLocalOnlyPath()` returns true for the new prefix
3. **Never skip this step** — see Hard Rule #15 in `CLAUDE.md`
4. Decide: does this route ALSO belong in `LOCAL_ONLY_MANAGE_SCOPE_BYPASS_PREFIXES`?
Default answer is **no**. Only opt-in when the route is safe to expose to a
manage-scope holder (i.e. does NOT spawn arbitrary user-controlled code).
## Adding a manage-scope-bypassable path
1. Confirm the route does not execute user-supplied code or commands. If it
does, stop — this carve-out is the wrong tool.
2. Append the prefix to `LOCAL_ONLY_MANAGE_SCOPE_BYPASS_PREFIXES` in
`src/server/authz/routeGuard.ts`
3. Add coverage in `tests/unit/authz/management-policy.test.ts` for all four
request shapes: no Bearer (403), manage Bearer (allow), non-manage Bearer
(403), and the per-prefix regression that `/api/cli-tools/runtime/*` stays
strict-loopback even with a manage Bearer.
## Files
| File | Purpose |
| -------------------------------------------- | ------------------------------ |
| `src/server/authz/routeGuard.ts` | Constants and helper functions |
| `src/server/authz/policies/management.ts` | Evaluation logic |
| `tests/unit/authz/routeGuard.test.ts` | Unit tests for tier helpers |
| `tests/unit/authz/management-policy.test.ts` | Unit tests for evaluate() |
## See also
- `docs/security/CLI_TOKEN.md` — CLI machine-ID token
- `docs/architecture/AUTHZ_GUIDE.md` — full authorization pipeline
- `docs/frameworks/MCP-SERVER.md` — MCP server transports and scopes

78
SQLite-Runtime.md Normal file

@@ -0,0 +1,78 @@
# SQLite Runtime Resolution
OmniRoute resolves its SQLite driver at startup through a 5-step fallback chain:
1. **Bundled `better-sqlite3`** (via `dependencies` in `package.json`)
— fastest, native binary, installed by `npm install` when build tools are present.
2. **Runtime-installed `better-sqlite3`** (in `~/.omniroute/runtime/`)
— installed lazily on first run **OR** by `scripts/build/postinstall.mjs → scripts/postinstall.mjs`.
Validates native `.node` magic bytes (ELF / Mach-O / PE) before loading
to guard against corrupt or wrong-platform binaries.
3. **`node:sqlite`** (Node ≥22.5 stdlib) — no native build needed; used when
both better-sqlite3 paths fail. Limited feature set.
4. **`sql.js`** (WASM) — final fallback. Works everywhere but is slower
and writes data on an interval rather than synchronously.
## Why this complexity?
- **Windows EBUSY**: `npm install -g omniroute@latest` can fail if the previous
version's `better_sqlite3.node` is locked by a running process. The runtime
install in `~/.omniroute/runtime/` sidesteps the global npm cache.
- **No build tools**: Some environments (corporate Windows without VS Build
Tools, minimal Docker images) cannot compile `better-sqlite3`. The runtime
installer resolves a pre-built binary from the npm registry; the fallback
drivers ensure OmniRoute still boots even if that fails.
- **Air-gapped systems**: If the npm registry is unreachable, `node:sqlite`
or `sql.js` guarantee baseline functionality.
## Magic-byte validation
Before loading a runtime-installed `.node` file, OmniRoute reads the first 8
bytes and matches against known platform magics:
| Platform | Bytes (hex) | Label |
| --------------------- | ------------- | ----------- |
| Linux | `7F 45 4C 46` | `elf` |
| macOS 64-bit BE | `FE ED FA CF` | `macho` |
| macOS 64-bit LE | `CF FA ED FE` | `macho-le` |
| macOS fat (universal) | `CA FE BA BE` | `macho-fat` |
| Windows | `4D 5A` (MZ) | `pe` |
A mismatched magic → file is ignored, fallback continues to the next step.
## Checking the active driver
```typescript
import { getDriverInfo } from "@/lib/db/core";
const info = getDriverInfo();
// { source: "bundled" | "runtime" | "runtime-installed-now" | "node-sqlite" | "sql-js",
// kind: "better-sqlite3" | "node-sqlite" | "sql-js" }
```
## Manual control
```bash
# Skip postinstall warm-up (for fast CI installs)
OMNIROUTE_SKIP_POSTINSTALL=1 npm install -g omniroute
# Force-reinstall runtime better-sqlite3
rm -rf ~/.omniroute/runtime
omniroute # will reinstall on next start
# Check what driver is active
omniroute config db-info # (if CLI command exists)
```
## Reference
Implementation:
- `bin/cli/runtime/magicBytes.mjs` — binary magic-byte validation helpers
- `bin/cli/runtime/sqliteRuntime.mjs` — 5-step runtime resolver + lazy installer
- `bin/cli/runtime/index.mjs` — startup orchestrator (`warmUpRuntimes()`)
- `scripts/postinstall.mjs` — npm post-install hook (non-fatal warm-up)
- `src/lib/db/core.ts` — `ensureDbInitialized()` / `getDriverInfo()` exports

377
Setup-Guide.md Normal file

@@ -0,0 +1,377 @@
# 📖 Setup Guide — OmniRoute
> Complete setup reference for OmniRoute. For the quick version, see the [Quick Start in README](../README.md#-quick-start).
## Table of Contents
- [Install Methods](#install-methods)
- [CLI Tool Configuration](#cli-tool-configuration)
- [Protocol Setup (MCP + A2A)](#protocol-setup-mcp--a2a)
- [Timeout Configuration](#timeout-configuration)
- [Split-Port Mode](#split-port-mode)
- [Void Linux (xbps-src)](#void-linux-xbps-src-template)
- [Uninstalling](#uninstalling)
---
## Install Methods
### npm (recommended)
```bash
npm install -g omniroute
omniroute
```
Dashboard opens at `http://localhost:20128` and API base URL is `http://localhost:20128/v1`.
### pnpm
```bash
pnpm install -g omniroute
pnpm approve-builds -g # Select all packages → approve
omniroute
```
> **pnpm users:** `pnpm approve-builds -g` is required to enable native build scripts for `better-sqlite3` and `@swc/core`.
### Arch Linux (AUR)
```bash
yay -S omniroute-bin
systemctl --user enable --now omniroute.service
```
The [AUR package](https://aur.archlinux.org/packages/omniroute-bin) installs OmniRoute and provides a systemd user service.
### From Source
```bash
npm install
PORT=20128 DASHBOARD_PORT=20129 NEXT_PUBLIC_BASE_URL=http://localhost:20129 npm run dev
```
> **Note:** `npm install` auto-generates `.env` from `.env.example` on first run. Subsequent installs will not overwrite an existing `.env`, so customizations are preserved. To re-seed, delete `.env` before re-running.
### Docker
See the [Docker Guide](./DOCKER_GUIDE.md) for complete Docker setup including Compose profiles and Caddy HTTPS.
### Desktop App (Electron)
OmniRoute ships a desktop wrapper built on Electron 41 + electron-builder 26.10. Available scripts (workspace root):
```bash
npm run electron:dev # Run desktop with hot-reload
npm run electron:build # Build for current OS (auto-detected)
npm run electron:build:win # Windows installer (NSIS + portable)
npm run electron:build:mac # macOS (dmg + zip, arm64+x64)
npm run electron:build:linux # Linux (AppImage + deb + rpm)
npm run electron:smoke:packaged # Smoke-test packaged build
```
Releases of the desktop installers are attached to GitHub Releases. For the full Electron deep-dive (signing, IPC bridge, distros), see [`ELECTRON_GUIDE.md`](./ELECTRON_GUIDE.md) _(criado em fase posterior)_.
### Headless server (CI/automation)
For unattended setups (Docker, Kubernetes, CI), use:
```bash
omniroute setup --non-interactive
omniroute providers test-batch
```
Combined with env vars (`INITIAL_PASSWORD`, `OMNIROUTE_WS_BRIDGE_SECRET`, etc.), this lets you spin up an OmniRoute instance fully scriptable.
### CLI Options
| Command | Description |
| ----------------------- | -------------------------------------------------------------- |
| `omniroute` | Start server (`PORT=20128`, API and dashboard on same port) |
| `omniroute setup` | Guided CLI onboarding for password and first provider |
| `omniroute doctor` | Run local health checks without starting the server |
| `omniroute providers` | Discover, list, validate, and test providers from CLI |
| `omniroute config` | CLI tool configuration — list, get, set, validate configs |
| `omniroute status` | Offline status dashboard — version, DB, tools, config |
| `omniroute logs` | Stream usage logs from the API (supports `--follow`) |
| `omniroute update` | Check for or apply OmniRoute updates |
| `omniroute provider` | Manage provider connections — add, list, remove, test, default |
| `omniroute --port 3000` | Set canonical/API port to 3000 |
| `omniroute --mcp` | Start MCP server (stdio transport) |
| `omniroute --no-open` | Don't auto-open browser |
| `omniroute --help` | Show help |
Headless setup can be scripted with flags or environment variables:
```bash
omniroute setup --non-interactive --password "$OMNIROUTE_PASSWORD"
omniroute setup --non-interactive --add-provider --provider openai --api-key "$OPENAI_API_KEY"
omniroute setup --non-interactive --add-provider --provider openai --api-key "$OPENAI_API_KEY" --test-provider
```
Run local diagnostics without opening the dashboard:
```bash
omniroute doctor
omniroute doctor --json
omniroute doctor --no-liveness
```
Manage providers from SSH or scripts without opening the dashboard:
```bash
omniroute providers available
omniroute providers available --search openai
omniroute providers available --category api-key
omniroute providers list
omniroute providers test <id-or-name>
omniroute providers test-all
omniroute providers validate
```
---
## CLI Tool Configuration
### 1) Connect Providers and Create API Key
1. Open Dashboard → `Providers` and connect at least one provider (OAuth or API key).
2. Open Dashboard → `Endpoints` and create an API key.
3. (Optional) Open Dashboard → `Combos` and set your fallback chain.
### 2) Point Your Coding Tool
```txt
Base URL: http://localhost:20128/v1
API Key: [copy from Endpoint page]
Model: if/kimi-k2-thinking (or any provider/model prefix)
```
Works with Claude Code, Codex CLI, Gemini CLI, Cursor, Cline, OpenClaw, OpenCode, and OpenAI-compatible SDKs.
For detailed per-tool configuration (Claude Code, Codex CLI, Cursor, Cline, OpenClaw, Kilo Code, Copilot, and more), see the dedicated **[CLI Tools Guide](../reference/CLI-TOOLS.md)**.
---
## Protocol Setup (MCP + A2A)
### MCP Setup (Model Context Protocol)
Start MCP transport in stdio mode:
```bash
omniroute --mcp
```
Recommended validation flow:
```bash
# 1. Start MCP server
omniroute --mcp
# 2. From your MCP client, call:
omniroute_get_health # Should return system health
omniroute_list_combos # Should return active combos
# 3. Or run the full E2E suite:
npm run test:protocols:e2e
```
#### MCP Client Configuration
**Claude Code:**
```bash
claude mcp add-server omniroute --type http --url http://localhost:20128/api/mcp/stream
```
**Cursor / Cline:**
Add to your MCP settings:
```json
{
"mcpServers": {
"omniroute": {
"command": "omniroute",
"args": ["--mcp"],
"env": {}
}
}
}
```
**Full MCP documentation:** [MCP Server README](../../open-sse/mcp-server/README.md) — 37 tools, IDE configs, Python/TS/Go clients.
### A2A Setup (Agent-to-Agent Protocol)
Verify the Agent Card:
```bash
curl http://localhost:20128/.well-known/agent.json
```
Send a task:
```bash
curl -X POST http://localhost:20128/a2a \
-H 'content-type: application/json' \
-d '{"jsonrpc":"2.0","id":"quickstart","method":"message/send","params":{"skill":"quota-management","messages":[{"role":"user","content":"Give me a short quota summary."}]}}'
```
**Full A2A documentation:** [A2A Server README](../../src/lib/a2a/README.md) — JSON-RPC 2.0, skills, streaming, task lifecycle.
---
## Timeout Configuration
### Basic Timeouts
For most deployments, you only need these two variables:
| Variable | Default | Purpose |
| ------------------------ | ----------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------- |
| `REQUEST_TIMEOUT_MS` | `600000` | Shared baseline for upstream response-start timeout, hidden Undici timeouts, TLS fingerprint requests, and API bridge request/proxy timeouts |
| `STREAM_IDLE_TIMEOUT_MS` | inherits `REQUEST_TIMEOUT_MS` | Maximum gap between streaming chunks before OmniRoute aborts the SSE stream |
Backward compatibility is preserved: existing `FETCH_TIMEOUT_MS`, `API_BRIDGE_PROXY_TIMEOUT_MS`, and other per-layer timeout vars still work and override the shared baseline.
### Provider-Specific Notes
For Claude Code-compatible upstreams (`anthropic-compatible-cc-*`), OmniRoute derives the outbound `X-Stainless-Timeout` header from the resolved fetch timeout so provider-side read timeouts stay aligned with your env configuration.
For third-party Claude Code-compatible reverse proxies, OmniRoute keeps the default `anthropic-beta` set conservative and, when `Client Cache Control` is left on `Auto`, only forwards client-provided `cache_control` markers.
### Advanced Timeout Overrides
| Variable | Default | Purpose |
| ---------------------------------------- | ------------------------------------------ | -------------------------------------------------------------------- |
| `FETCH_TIMEOUT_MS` | inherits `REQUEST_TIMEOUT_MS` | Upstream response-start timeout used until response headers arrive |
| `FETCH_HEADERS_TIMEOUT_MS` | inherits `FETCH_TIMEOUT_MS` | Undici time limit for receiving upstream response headers |
| `FETCH_BODY_TIMEOUT_MS` | inherits `FETCH_TIMEOUT_MS` | Undici time limit between upstream body chunks (`0` disables it) |
| `FETCH_CONNECT_TIMEOUT_MS` | `30000` | Undici TCP connect timeout |
| `FETCH_KEEPALIVE_TIMEOUT_MS` | `4000` | Undici idle keep-alive socket timeout |
| `TLS_CLIENT_TIMEOUT_MS` | inherits `FETCH_TIMEOUT_MS` | Timeout for TLS fingerprint requests made through `wreq-js` |
| `API_BRIDGE_PROXY_TIMEOUT_MS` | inherits `REQUEST_TIMEOUT_MS` or `30000` | Timeout for `/v1` proxy forwarding from API port to dashboard port |
| `API_BRIDGE_SERVER_REQUEST_TIMEOUT_MS` | `max(API_BRIDGE_PROXY_TIMEOUT_MS, 300000)` | Incoming request timeout on the API bridge server |
| `API_BRIDGE_SERVER_HEADERS_TIMEOUT_MS` | `60000` | Incoming header timeout on the API bridge server |
| `API_BRIDGE_SERVER_KEEPALIVE_TIMEOUT_MS` | `5000` | Keep-alive timeout on the API bridge server |
| `API_BRIDGE_SERVER_SOCKET_TIMEOUT_MS` | `0` | Socket inactivity timeout on the API bridge server (`0` disables it) |
> **Note:** For streaming requests, `FETCH_TIMEOUT_MS` only covers connection setup / waiting for the first upstream response. Once the stream is active, OmniRoute will only abort on an actual stall (`STREAM_IDLE_TIMEOUT_MS`) or Undici body inactivity (`FETCH_BODY_TIMEOUT_MS`).
### Reverse Proxy Compatibility
If you run OmniRoute behind Nginx, Caddy, Cloudflare, or another reverse proxy, make sure the proxy timeouts are also higher than your OmniRoute stream/fetch timeouts.
---
## Split-Port Mode
Run API and Dashboard on separate ports for advanced scenarios (reverse proxy, container networking):
```bash
PORT=20128 DASHBOARD_PORT=20129 omniroute
# API: http://localhost:20128/v1
# Dashboard: http://localhost:20129
```
---
## Void Linux (xbps-src) Template
For Void Linux users, you can build a native package using `xbps-src`. Save this block as `srcpkgs/omniroute/template`:
```bash
# Template file for 'omniroute'
pkgname=omniroute
version=3.8.0
revision=1
hostmakedepends="nodejs python3 make"
depends="openssl"
short_desc="Universal AI gateway with smart routing for multiple LLM providers"
maintainer="zenobit <zenobit@disroot.org>"
license="MIT"
homepage="https://github.com/diegosouzapw/OmniRoute"
distfiles="https://github.com/diegosouzapw/OmniRoute/archive/refs/tags/v${version}.tar.gz"
# Regenerate the checksum for each release with:
# curl -L -o /tmp/omniroute.tar.gz "https://github.com/diegosouzapw/OmniRoute/archive/refs/tags/v${version}.tar.gz" && sha256sum /tmp/omniroute.tar.gz
checksum=PLACEHOLDER_REGENERATE_PER_RELEASE
system_accounts="_omniroute"
omniroute_homedir="/var/lib/omniroute"
export NODE_ENV=production
export npm_config_engine_strict=false
export npm_config_loglevel=error
export npm_config_fund=false
export npm_config_audit=false
do_build() {
local _gyp_arch
case "$XBPS_TARGET_MACHINE" in
aarch64*) _gyp_arch=arm64 ;;
armv7*|armv6*) _gyp_arch=arm ;;
i686*) _gyp_arch=ia32 ;;
*) _gyp_arch=x64 ;;
esac
NODE_ENV=development npm ci --ignore-scripts
npm run build
cp -r .next/static .next/standalone/.next/static
[ -d public ] && cp -r public .next/standalone/public || true
local _node_gyp=/usr/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js
(cd node_modules/better-sqlite3 && node "$_node_gyp" rebuild --arch="$_gyp_arch")
local _bs3_release=.next/standalone/node_modules/better-sqlite3/build/Release
mkdir -p "$_bs3_release"
cp node_modules/better-sqlite3/build/Release/better_sqlite3.node "$_bs3_release/"
rm -rf .next/standalone/node_modules/@img
for _mod in pino-abstract-transport split2 process-warning; do
cp -r "node_modules/$_mod" .next/standalone/node_modules/
done
}
do_check() {
npm run test:unit
}
do_install() {
vmkdir usr/lib/omniroute/.next
vcopy .next/standalone/. usr/lib/omniroute/.next/standalone
for _d in \
.next/standalone/.next/server/app/dashboard \
.next/standalone/.next/server/app/dashboard/settings \
.next/standalone/.next/server/app/dashboard/providers; do
touch "${DESTDIR}/usr/lib/omniroute/${_d}/.keep"
done
cat > "${WRKDIR}/omniroute" <<'EOF'
#!/bin/sh
export PORT="${PORT:-20128}"
export DATA_DIR="${DATA_DIR:-${XDG_DATA_HOME:-${HOME}/.local/share}/omniroute}"
export APP_LOG_TO_FILE="${APP_LOG_TO_FILE:-false}"
mkdir -p "${DATA_DIR}"
exec node /usr/lib/omniroute/.next/standalone/server.js "$@"
EOF
vbin "${WRKDIR}/omniroute"
}
post_install() {
vlicense LICENSE
}
```
---
## Uninstalling
| Command | Action |
| ------------------------ | ----------------------------------------------------------------------------------- |
| `npm run uninstall` | Removes the system app but **keeps your DB and configurations** in `~/.omniroute`. |
| `npm run uninstall:full` | Removes the app AND permanently **erases all configurations, keys, and databases**. |
> For detailed uninstall instructions across all methods, see [UNINSTALL.md](./UNINSTALL.md).

283
Skills.md Normal file

@@ -0,0 +1,283 @@
# Skills Framework
> **Source of truth:** `src/lib/skills/` and `src/app/api/skills/`
> **Last updated:** 2026-05-13 — v3.8.0
OmniRoute exposes an extensible Skills framework that lets language models (and operators) compose reusable capabilities — from filesystem reads and HTTP requests to sandboxed code execution and curated marketplace skills.
A skill is a versioned, schema-defined unit of work. OmniRoute can inject skills as tool definitions into outbound requests, intercept tool calls coming back from the model, run the matching handler, and feed the result back to the model so the conversation can continue. The model never sees the implementation — only the tool interface.
---
## Concepts
### Skill Sources
Three sources of skills coexist in the same registry:
1. **Built-in skills** (`src/lib/skills/builtins.ts`) — shipped with OmniRoute. Cover the common cases:
- `file_read`, `file_write` — per-API-key sandbox workspace under `<DATA_DIR>/skills/workspaces/<hashed-key>/`
- `http_request` — outbound HTTP through `safeOutboundFetch` with `guard: "public-only"`
- `web_search` — pluggable search provider with caching (`executeWebSearch`)
- `eval_code` — Docker-sandboxed `node` or `python` execution
- `execute_command` — Docker-sandboxed shell command
- `browser` — Playwright-backed scaffolding, disabled by default (`builtin/browser.ts`)
2. **SkillsMP** (the OmniRoute Marketplace) — fetched from `https://skillsmp.com/api/v1/skills/search`. Requires `skillsmpApiKey` in Settings.
3. **SkillsSH** (`skills.sh` community catalog) — fetched from `https://skills.sh/api/search`. No auth needed; SKILL.md content pulled from GitHub raw.
A single "active provider" controls which catalog the dashboard installs from (`src/lib/skills/providerSettings.ts`). Switch it under **Settings → Memory & Skills**. Default: `skillsmp`.
### Skill Identity
Skills are keyed by `name@version` in the in-memory registry (`src/lib/skills/registry.ts`). Version must be semver (`^\d+\.\d+\.\d+$`). `resolveVersion()` understands `^`, `~`, `>`, `>=`, `<`, `<=`, `==`, and exact-match constraints.
### Skill Mode
Each skill has a runtime mode that controls when it is injected:
| Mode | Behavior |
| ------ | ------------------------------------------------------------------------------------------ |
| `on` | Always injected as a tool definition |
| `off` | Never injected, never executable |
| `auto` | Scored against the incoming request; injected only if score ≥ `AUTO_MIN_SCORE` (default 3) |
`auto` is the default for marketplace-installed skills. `enabled=true` and `mode="off"` together mean "registered but inactive" — toggling `enabled` via the legacy column also bumps `mode` so older codepaths stay consistent (`src/app/api/skills/[id]/route.ts`).
### Status (executions)
Skill executions are tracked in the `skill_executions` table with the following statuses (`src/lib/skills/types.ts`):
```ts
enum SkillStatus {
PENDING = "pending",
RUNNING = "running",
SUCCESS = "success",
ERROR = "error",
TIMEOUT = "timeout",
}
```
### Registry Cache
`SkillRegistry` is a singleton with a 60-second TTL cache (`registry.ts:14`). `loadFromDatabase()` is idempotent and dedupes concurrent calls via `pendingLoad`. Any write (`register`/`unregister`/`unregisterById`) invalidates the cache. Look up versions via `getSkillVersions(name)` and `resolveVersion(name, constraint)`.
### Provider-Aware Injection
`injectSkills()` in `src/lib/skills/injection.ts` is the entry point that turns registered skills into provider-specific tool definitions:
- **OpenAI** — `{ type: "function", function: { name, description, parameters } }`
- **Anthropic** — `{ name, description, input_schema }`
- **Google (Gemini)** — `{ name, description, parameters }`
The tool name is encoded as `name@version` so the handler can pick the right version when the model calls it back.
### AUTO Scoring
When `mode="auto"`, each candidate skill is scored against the request context (`scoreAutoSkill()` in `injection.ts`):
| Signal | Points |
| ---------------------------------------------- | ------------ |
| Skill name appears verbatim in context | +6 |
| Each name token matches a context token | +2 |
| Each tag substring matches context | +3 |
| Each description token matches context | +1 |
| Background reason matches a name token | +2 per token |
| Background reason matches a tag | +2 per token |
| Provider hint in tags matches request provider | +2 / −2 |
Top `AUTO_MAX_SKILLS = 5` skills with `score >= AUTO_MIN_SCORE = 3` are injected. Ties are broken by `installCount` (desc), then alphabetical name (`injection.ts:225-235`).
### Tool Call Interception
`handleToolCallExecution()` in `src/lib/skills/interception.ts` is invoked by the chat handler after the upstream returns a tool-calling response:
1. `extractToolCalls()` reads provider-specific shapes (OpenAI `tool_calls` / Responses `function_call`, Anthropic `tool_use`, Gemini `functionCalls`).
2. Built-in tool aliases (e.g. `omniroute_web_search` → `web_search`) are resolved first. Built-in handlers run inline.
3. Anything else routes through `skillExecutor.execute(name@version, args, { apiKeyId, sessionId })`.
4. Results are spliced back into the response — `tool_results`, `function_call_output` items, or Anthropic `tool_result` blocks as appropriate.
`customSkillExecutionEnabled` in the execution context can be set to `false` to allow only built-in interception (used by request paths that explicitly disable user-defined handlers).
---
## Docker Sandbox
Non-builtin code paths (`eval_code`, `execute_command`) run inside Docker via `SandboxRunner` (`src/lib/skills/sandbox.ts`). Every container is launched with:
```
--rm --network none|bridge --cap-drop ALL
--security-opt no-new-privileges --pids-limit 100
--cpus <cpuLimit/1000> --memory <memoryLimit>m
--tmpfs /tmp:rw,noexec,nosuid,size=64m
--tmpfs /workspace:rw,noexec,nosuid,size=64m
--read-only (when readOnly=true)
```
Defaults (`SandboxRunner.DEFAULT_CONFIG`):
| Field | Default | Notes |
| ---------------- | --------------- | ---------------------------------------------------- |
| `cpuLimit` | 100 (= 0.1 CPU) | Divided by 1000 before passing to `--cpus` |
| `memoryLimit` | 256 MB | Hard limit |
| `timeout` | 30000 ms | Soft kill via `SIGTERM` + `docker kill` |
| `networkEnabled` | `false` | Becomes `--network none` |
| `readOnly` | `true` | Root FS read-only; `/tmp` and `/workspace` are tmpfs |
`SandboxRunner.kill(id)` and `killAll()` are exposed for shutdown; running containers are tracked in `runningContainers: Map<string, ChildProcess>`.
### Sandbox Env Vars
Configured via `process.env` in `src/lib/skills/builtins.ts`:
| Env Var | Default | Purpose |
| --------------------------------- | ---------------- | ------------------------------------------------------------------ |
| `SKILLS_MAX_FILE_BYTES` | `1048576` (1 MB) | Cap for `file_read` and `file_write` |
| `SKILLS_MAX_HTTP_RESPONSE_BYTES` | `256000` | Cap for `http_request` response body |
| `SKILLS_MAX_SANDBOX_OUTPUT_CHARS` | `100000` | Cap for stdout/stderr returned to the caller |
| `SKILLS_SANDBOX_TIMEOUT_MS` | `10000` | Default timeout for sandboxed commands; capped at 60 s |
| `SKILLS_SANDBOX_NETWORK_ENABLED` | `false` | Master gate for egress. Set `1` or `true` to allow per-call opt-in |
| `SKILLS_ALLOWED_SANDBOX_IMAGES` | (see below) | Comma-separated allowlist of Docker images |
Default allowed images: `alpine:3.20`, `node:22-alpine`, `python:3.12-alpine`. Any additions via `SKILLS_ALLOWED_SANDBOX_IMAGES` are merged with the defaults; unknown images are rejected by `normalizeImage()`.
> Note: there is no separate `SKILLS_EXECUTION_TIMEOUT_MS` env var. The non-sandbox handler timeout is hard-coded to 30 s in `SkillExecutor` (`executor.ts:13`) but can be overridden at runtime via `skillExecutor.setTimeout(ms)`.
### Workspace Isolation
`file_read` and `file_write` resolve every path relative to a per-API-key workspace at `<DATA_DIR>/skills/workspaces/<sha256(apiKeyId).slice(0,24)>/`. Path traversal (`..`) and forbidden segments (`.env`, `.git`, `.ssh`, `.omniroute`, `.codex`, `secrets`) are rejected before any disk I/O.
### HTTP Hardening
`http_request` (`builtins.ts:257`):
- Method allowlist: `GET, HEAD, POST, PUT, PATCH, DELETE`
- Blocked outbound headers: `host, connection, content-length, cookie, set-cookie, authorization, proxy-authorization`
- Redirects disabled (`allowRedirect: false`)
- Routed through `safeOutboundFetch` with `guard: "public-only"` (private/loopback ranges blocked)
- Response truncated at `SKILLS_MAX_HTTP_RESPONSE_BYTES`; client sees `truncated: true`
---
## Hybrid Executor (preview)
`src/lib/skills/hybrid.ts` defines a `HybridExecutor` that decides between `direct` (in-process) and `sandbox` execution per call, with an `autoUpgrade` retry path on timeout/memory errors. The wired-in `directExecutor` / `sandboxRunner` implementations are stubs (`executeDirect`, `executeInSandbox` return placeholder objects) — treat this module as a contract under construction. Real execution still goes through `skillExecutor` + `SandboxRunner`.
---
## Storage
Schema lives in two migrations:
- `src/lib/db/migrations/016_create_skills.sql` — base `skills` and `skill_executions` tables, with indexes on `(api_key_id, name)` and `(skill_id, status, created_at)`.
- `src/lib/db/migrations/027_skill_mode_and_metadata.sql` — adds `mode`, `source_provider`, `tags` (JSON), `install_count` to `skills`.
`skill_executions.status` is constrained at the database level: `CHECK(status IN ('pending', 'running', 'success', 'error', 'timeout'))`.
---
## REST API
All endpoints live under `src/app/api/skills/`. Management endpoints (`/api/skills`, `/api/skills/[id]`, `/api/skills/install`) require **management auth** via `requireManagementAuth()`. The marketplace/install flows use the lighter `isAuthenticated()` (session or API key).
| Endpoint | Method | Purpose |
| --------------------------------- | ------ | ------------------------------------------------------------------------ | --- | ------------------------ | -------- | ------------------ |
| `/api/skills` | GET | List registered skills. Supports `?q=`, `?mode=on | off | auto`, `?source=skillsmp | skillssh | local`, pagination |
| `/api/skills/[id]` | PUT | Update `enabled` or `mode` |
| `/api/skills/[id]` | DELETE | Unregister by id |
| `/api/skills/install` | POST | Install a custom skill (handler code + schema) |
| `/api/skills/marketplace` | GET | Search the SkillsMP catalog (returns popular defaults when `q` is empty) |
| `/api/skills/marketplace/install` | POST | Install a SkillsMP skill (requires active provider = `skillsmp`) |
| `/api/skills/skillssh` | GET | Search the skills.sh catalog (`?q=&limit=`, capped at 100) |
| `/api/skills/skillssh/install` | POST | Install a skills.sh skill (requires active provider = `skillssh`) |
| `/api/skills/executions` | GET | Paginated execution history (`?apiKeyId=`) |
| `/api/skills/executions` | POST | Execute a registered skill ad-hoc |
The `POST /api/skills/executions` endpoint returns HTTP `503` with `{ error: "Skills execution is disabled..." }` when `settings.skillsEnabled === false` (`executor.ts:42-45`). Operators can flip the master switch from **Settings → AI**.
### Example: install a custom skill
```bash
curl -X POST http://localhost:20128/api/skills/install \
-H "Authorization: Bearer $OMNIROUTE_MGMT_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"name": "reverse-text",
"version": "1.0.0",
"description": "Reverses a string",
"schema": {
"input": { "type": "object", "properties": { "text": { "type": "string" } }, "required": ["text"] },
"output": { "type": "object", "properties": { "reversed": { "type": "string" } } }
},
"handlerCode": "echo-handler",
"apiKeyId": "your-api-key-id"
}'
```
The `handlerCode` string is a **handler name lookup** — not executable code. The executor maps it via `skillExecutor.registerHandler(name, fn)` (`executor.ts:25`). Marketplace installs store the SKILL.md text in this field as documentation and route execution through model-generated tool calls. Arbitrary user-supplied source is not eval'd.
---
## MCP Tools
Four MCP tools wrap the skills surface (`open-sse/mcp-server/tools/skillTools.ts`). They are auto-registered when the MCP server boots.
| Tool | Description |
| ----------------------------- | ------------------------------------------------------------ |
| `omniroute_skills_list` | List skills, optional filters: `apiKeyId`, `name`, `enabled` |
| `omniroute_skills_enable` | Enable/disable a skill by `skillId` |
| `omniroute_skills_execute` | Execute a skill with an input payload |
| `omniroute_skills_executions` | Recent execution history (default 50, max 100) |
See [MCP-SERVER.md](./MCP-SERVER.md) for transport setup and scope assignments.
---
## A2A Integration
`src/lib/skills/a2a.ts` exports the `memory_aware_routing` A2A skill descriptor and a `registerA2ASkill(registry)` helper. Custom A2A skills live in `src/lib/a2a/skills/` and are dispatched via `A2A_SKILL_HANDLERS` (`src/lib/a2a/taskExecution.ts`). See [A2A-SERVER.md](./A2A-SERVER.md) for the full task lifecycle.
---
## Adding a New Built-in Skill
1. **Define the handler** in `src/lib/skills/builtins.ts` (or a sibling file under `src/lib/skills/builtin/`). Signature: `(input, { apiKeyId, sessionId }) => Promise<output>`.
2. **Sandboxed code path?** Call `sandboxRunner.run(image, command, env, sandboxConfig({...}))`. Use `normalizeImage()` against the allowlist.
3. **Filesystem path?** Always pass through `resolveWorkspacePath(input, context)` before touching disk.
4. **Network call?** Use `safeOutboundFetch` with `guard: "public-only"`; sanitize headers via `sanitizeHeaders()`.
5. **Register** by adding the entry to `builtinSkills` (or calling `registerBrowserSkill(executor)`-style at boot).
6. **Wire built-in tool aliases** (optional) in `BUILTIN_TOOL_ALIASES` (`interception.ts:23`) if the upstream model emits a different name.
7. **Tests** in `src/lib/skills/__tests__/` (Vitest).
---
## Adding a Custom (Non-Builtin) Skill
1. Register the handler at process startup:
```ts
skillExecutor.registerHandler("my-handler", async (input, ctx) => { ... });
```
2. Insert the skill via `POST /api/skills/install` (the `handlerCode` field must match the registered handler name).
3. Toggle `mode` to `on` or `auto` via `PUT /api/skills/[id]`.
---
## Operational Tips
- **Master switch:** `settings.skillsEnabled = false` blocks all execution and returns HTTP `503` on `/api/skills/executions`. The registry continues to load.
- **Lock down egress:** keep `SKILLS_SANDBOX_NETWORK_ENABLED` unset (default) for fully air-gapped sandboxing. Per-call `networkEnabled: true` still requires the master gate.
- **Allow specific images:** set `SKILLS_ALLOWED_SANDBOX_IMAGES="myorg/sandbox:1.0,node:22-alpine"` to extend the allowlist.
- **Audit executions:** `/dashboard/skills/executions` and `omniroute_skills_executions` both query `skill_executions`. Successful runs include `durationMs`; failures include `errorMessage`.
- **Cache invalidation:** call `skillRegistry.invalidateCache()` after manual DB edits; otherwise wait 60 s.
- **Anonymous workspace:** when `apiKeyId` is empty, all calls hash to the same `"anonymous"` workspace — share-aware code should always pass a real key.
---
## See Also
- [MCP-SERVER.md](./MCP-SERVER.md) — MCP tool registration and transports
- [A2A-SERVER.md](./A2A-SERVER.md) — A2A task lifecycle and skill dispatch
- [USER_GUIDE.md](../guides/USER_GUIDE.md#-skills-system) — user-facing introduction
- [ARCHITECTURE.md](../architecture/ARCHITECTURE.md) — request pipeline and component map
- Source: `src/lib/skills/`, `src/app/api/skills/`, `open-sse/mcp-server/tools/skillTools.ts`
- Tests: `src/lib/skills/__tests__/integration.test.ts`

273
Stealth-Guide.md Normal file

@@ -0,0 +1,273 @@
# Stealth Guide
> **Source of truth:** `open-sse/utils/tlsClient.ts`, `open-sse/services/{chatgptTlsClient,claudeCodeCCH,claudeCodeFingerprint,claudeCodeObfuscation,claudeCodeCompatible,antigravityObfuscation}.ts`, `open-sse/config/cliFingerprints.ts`, `src/mitm/`
> **Last updated:** 2026-05-13 — v3.8.0
> **Audience:** Engineers maintaining provider-specific stealth integrations.
OmniRoute integrates with providers whose edges actively fingerprint non-official clients (TLS JA3/JA4, header ordering, JSON body shape, integrity tokens). This page documents the stealth surfaces OmniRoute exposes and where they are implemented.
## Legal and Ethical Notice
Stealth features exist so OmniRoute can act as a compatibility layer between user-owned official accounts (Claude Code CLI, ChatGPT Desktop/Web, Antigravity, Cursor, etc.) and OmniRoute's unified API. They are **not** for evading fraud detection, sharing credentials, or violating provider Terms of Service. The maintainers expect operators to comply with the upstream ToS they signed when creating accounts.
---
## TLS Fingerprinting Layer
### `open-sse/utils/tlsClient.ts` — wreq-js (Chrome 124)
Lazy-loaded `wreq-js` session that impersonates **Chrome 124 on macOS**. Used as a generic JA3/JA4 wrapper for upstreams behind Cloudflare. Falls back to native fetch when `wreq-js` is not installed (`available = false`).
- Singleton session: `browser: "chrome_124", os: "macos"`
- Proxy resolution (priority): `HTTPS_PROXY` → `HTTP_PROXY` → `ALL_PROXY` (also lower-case)
- Timeout: `TLS_CLIENT_TIMEOUT_MS` (inherits from `FETCH_TIMEOUT_MS`, default 600000)
- `wreq-js` Response is fetch-compatible (`headers`, `text()`, `json()`, `clone()`, `body`).
### `open-sse/services/chatgptTlsClient.ts` — tls-client-node (Firefox 148)
Dedicated TLS impersonator for `chatgpt.com`. ChatGPT's Cloudflare config pins `cf_clearance` to JA3/JA4 + HTTP/2 SETTINGS frame ordering — undici's handshake gets `cf-mitigated: challenge` even with valid cookies.
- Profile: `firefox_148` (must match the Firefox 148 `User-Agent` sent)
- Mode: `runtimeMode: "native"` (koffi-loaded shared library; avoids managed sidecar HTTP)
- `withRandomTLSExtensionOrder: true`
- `tlsFetchChatGpt(url, options)` supports streaming (writes body to temp file, tailed as `ReadableStream`)
- Hang detection: `raceWithTimeout` + `TlsClientHangError` triggers `resetClientCache()` so the next call respawns the binding
- Proxy resolution (priority): per-call `proxyUrl` → `OMNIROUTE_TLS_PROXY_URL` → `HTTPS_PROXY`/`HTTP_PROXY`/`ALL_PROXY` (the native binding does **not** read these envs itself; it must be threaded through)
- Errors: `TlsClientUnavailableError` (binary missing), `TlsClientHangError` (binding deadlocked)
---
## Claude Code Stealth Bundle
When `cliCompatMode` is on, OmniRoute reshapes outgoing Claude requests so they are indistinguishable from `claude-cli` traffic. Three modules collaborate:
### `claudeCodeFingerprint.ts`
Computes the 3-char `cc_version` fingerprint embedded in the billing header:
```
SHA256(SALT + msg[4] + msg[7] + msg[20] + version)[:3]
```
- `FINGERPRINT_SALT = "59cf53e54c78"` (hardcoded; matches official client)
- Inputs: chars at index 4, 7, 20 of the first user message text + version string
- Output: 3-char hex prefix
### `claudeCodeCCH.ts` (Client Content Hash)
Server-side integrity check the official Claude Code CLI computes via Bun/Zig. OmniRoute reimplements with `xxhash-wasm`:
1. Serialize body with `cch=00000;` placeholder
2. `xxhash64(bytes, seed) & 0xFFFFF`
3. Zero-padded 5-char lowercase hex
4. Replace `cch=00000;` with the computed token
Constants:
- Seed: `0x6e52736ac806831e`
- Pattern: `/\bcch=([0-9a-f]{5});/`
### `claudeCodeObfuscation.ts`
Inserts a Unicode **zero-width joiner** (`U+200D`) after the first character of "sensitive" client names so upstream filters cannot grep them. Default word list:
```
opencode, open-code, cline, roo-cline, roo_cline, cursor, windsurf,
aider, continue.dev, copilot, avante, codecompanion
```
Applied to: `system` blocks, all `messages[].content`, and `tools[].description` / `tools[].function.description`. Operator-overridable via `setSensitiveWords()`.
### `claudeCodeCompatible.ts` — `anthropic-compatible-cc-*` providers
For third-party Anthropic relays that only accept "real Claude Code" traffic:
- `CLAUDE_CODE_COMPATIBLE_USER_AGENT = "claude-cli/2.1.146 (external, sdk-cli)"`
- `CLAUDE_CODE_COMPATIBLE_STAINLESS_PACKAGE_VERSION = "0.81.0"`
- `CLAUDE_CODE_COMPATIBLE_STAINLESS_RUNTIME_VERSION = "v24.3.0"`
- `anthropic-beta = "claude-code-20250219,interleaved-thinking-2025-05-14,effort-2025-11-24"`
- `CONTEXT_1M_BETA_HEADER = "context-1m-2025-08-07"` (Opus/Sonnet 4.x family)
- Default path: `/v1/messages?beta=true`
Sister modules in the same bundle:
- `claudeCodeConstraints.ts` — temperature + cache-control rules
- `claudeCodeToolRemapper.ts` — tool-name remapping
- `claudeCodeExtraRemap.ts` — extra payload normalization
---
## Antigravity Stealth
### `antigravityObfuscation.ts`
Same zero-width-joiner trick as Claude Code, but with an expanded word list that also masks: `claude code`, `claude-code`, `kilo code`, `kilocode`, **`omniroute`**. Mirrors ZeroGravity's `ZEROGRAVITY_SENSITIVE_WORDS` and CLIProxyAPI's cloak system.
### `antigravityHeaderScrub.ts`
Strips Stainless SDK markers (`x-stainless-lang`, `x-stainless-package-version`, `x-stainless-os`, `x-stainless-arch`, `x-stainless-runtime`, `x-stainless-runtime-version`, `x-stainless-timeout`, `x-stainless-retry-count`, `x-stainless-helper-method`) before forwarding.
### ⚠️ Risk: `ANTIGRAVITY_CREDITS=always` (account-ban hot spot)
`ANTIGRAVITY_CREDITS=always` (consumed by `open-sse/executors/antigravity.ts`) routes **every** request through Antigravity AI Credit Overages (paid Google credits) instead of letting Google's free-tier quota gate things. This is documented as a feature, but it is **the single most common ToS-violation report we see** — multiple Google Ultra accounts have been banned with `403 / "service disabled for ToS violation" / insufficient_quota` after running for a few hours with `=always`.
The upstream enforcement is on **Google's side**, not anything OmniRoute can prevent. The env var name and the existing docs make it sound like a safe knob to flip; it isn't.
**Why this draws abuse detection more aggressively than free-tier-only usage:**
- Sustained automated spend on a single Google account flags differently than free-tier hits-quota-and-stops.
- Credit overages have no rate ceiling, so a misconfigured client can burn through several hundred USD in minutes and look like API-key resale or bot traffic.
- Multiple OmniRoute users hitting overage credits in parallel from the same external IP compounds the signal.
**Recommended posture:**
1. **Default to `ANTIGRAVITY_CREDITS=retry`** — overages are used only when free-tier returns 429, not on every request. This is the safer of the two non-zero modes.
2. **Spread load across providers via Auto-Combo** (`model: "auto"` or `kr/glm/etc`-combo) instead of saturating a single Antigravity account.
3. **Set per-connection RPM limits** in the Antigravity provider's edit page (Dashboard → Providers → Antigravity → connection → rate limit). 30–60 RPM is a defensible upper bound for sustained use.
4. **Use distinct upstream IPs** per Antigravity account when possible (residential proxies aimed at the same account from many users compounds the abuse signal).
5. **If banned**: appeal via `support.google.com` → "Restore Workspace/Account access" with the exact `quota_exceeded` / `service disabled` response body Google sent. Restoration is not guaranteed.
This warning is also surfaced inline in the dashboard near the Antigravity provider edit screen when `ANTIGRAVITY_CREDITS` is set to `always` (or will be in v3.8.0; tracked separately).
Touch points:
- `open-sse/executors/antigravity.ts` — reads `process.env.ANTIGRAVITY_CREDITS`
- `src/lib/oauth/providers/antigravity.ts` — credential plumbing
- Original incident report: Discussion [#1183](https://github.com/diegosouzapw/OmniRoute/discussions/1183)
---
## CLI Fingerprint Registry — `open-sse/config/cliFingerprints.ts`
Per-provider table that pins **exact** header ordering and JSON body field ordering captured from mitmproxy traces of the official CLIs. Currently registered: `codex`, `claude`, plus runtime-derived profiles in `providerHeaderProfiles.ts` for `antigravity`, `qwen`, `github`.
```ts
interface CliFingerprint {
headerOrder: string[]; // case-sensitive
bodyFieldOrder: string[]; // top-level JSON keys
userAgent?: string | (() => string);
extraHeaders?: Record<string, string>;
}
```
Toggle per provider via env (see below). When disabled, headers/body keys appear in whatever order Node/JSON gave them — easy to fingerprint.
---
## MITM Proxy (Antigravity, Linux/macOS/Windows)
For CLIs whose binaries cannot be redirected via `OPENAI_BASE_URL`, OmniRoute runs a local TLS-terminating proxy. Endpoints live under `src/app/api/cli-tools/antigravity-mitm/`.
| Method | Endpoint | Purpose |
| ------ | --------------------------------------- | ------------------------------------------------ |
| GET | `/api/cli-tools/antigravity-mitm` | Status — running, pid, dnsConfigured, certExists |
| POST | `/api/cli-tools/antigravity-mitm` | Start MITM (requires `apiKey` + `sudoPassword`) |
| DELETE | `/api/cli-tools/antigravity-mitm` | Stop MITM |
| GET | `/api/cli-tools/antigravity-mitm/alias` | List model aliases |
| PUT | `/api/cli-tools/antigravity-mitm/alias` | Save model aliases for a tool |
Target intercepted host: **`daily-cloudcode-pa.googleapis.com`** (Antigravity's upstream).
### Start sequence (`src/mitm/manager.ts::startMitm`)
1. Generate self-signed cert via `selfsigned` (RSA-2048, SHA-256, 1y) — `cert/generate.ts`
2. Install cert to system trust store — `cert/install.ts`
3. Add hosts entry `127.0.0.1 daily-cloudcode-pa.googleapis.com` — `dns/dnsConfig.ts`
4. Spawn `src/mitm/server.cjs` with `ROUTER_API_KEY` + `MITM_LOCAL_PORT` (default `443`)
5. Persist PID to `<DATA_DIR>/mitm/.mitm.pid`
### Linux dynamic trust-store detection — `cert/install.ts`
`getLinuxCertConfig()` walks a priority list and picks the first existing directory:
| Distro family | Directory | Update command |
| ------------------------ | ------------------------------------------- | ------------------------ |
| Debian / Ubuntu | `/usr/local/share/ca-certificates` | `update-ca-certificates` |
| Arch / CachyOS / Manjaro | `/etc/ca-certificates/trust-source/anchors` | `update-ca-trust` |
| Fedora / RHEL / CentOS | `/etc/pki/ca-trust/source/anchors` | `update-ca-trust` |
| openSUSE | `/etc/pki/trust/anchors` | `update-ca-certificates` |
Cert filename: `omniroute-mitm.crt`. Fingerprint match via `getCertFingerprint()` (SHA-1 of DER).
Additionally, `updateNssDatabases()` installs into per-user NSS DBs when `certutil` is available: `~/.pki/nssdb`, `~/snap/chromium/.../nssdb`, all Firefox profiles (including snap), under the nickname **`OmniRoute MITM Root CA`**.
### macOS / Windows
- **macOS:** `security add-trusted-cert -d -r trustRoot -k /Library/Keychains/System.keychain`
- **Windows:** elevated PowerShell → `certutil -addstore Root`
### Auth
All MITM endpoints require management auth (`requireCliToolsAuth`). The sudo password is cached in module scope (never `globalThis`) and cleared on `stopMitm()`.
---
## User-Agent Overrides — env vars (`.env.example` section 12)
| Variable | Default |
| ------------------------ | --------------------------------------------- |
| `CLAUDE_USER_AGENT` | `claude-cli/2.1.146 (external, cli)` |
| `CODEX_USER_AGENT` | `codex-cli/0.132.0 (Windows 10.0.26200; x64)` |
| `GITHUB_USER_AGENT` | `GitHubCopilotChat/0.45.1` |
| `ANTIGRAVITY_USER_AGENT` | `antigravity/2.0.1 darwin/arm64` |
| `KIRO_USER_AGENT` | `AWS-SDK-JS/3.0.0 kiro-ide/1.0.0` |
| `QODER_USER_AGENT` | `Qoder-Cli` |
| `QWEN_USER_AGENT` | `QwenCode/0.15.9 (linux; x64)` |
| `CURSOR_USER_AGENT` | `Cursor/3.3` |
| `GEMINI_CLI_USER_AGENT` | `google-api-nodejs-client/10.3.0` |
Consumed by `open-sse/executors/base.ts::buildHeaders()` via dynamic lookup. **Bump these when providers release new CLI versions** — stale UA strings start getting rejected as outdated clients.
## CLI Compatibility Mode Toggles (`.env.example` section 13)
| Variable | Effect |
| -------------------------- | ------------------------------- |
| `CLI_COMPAT_CODEX=1` | Codex fingerprint |
| `CLI_COMPAT_CLAUDE=1` | claude-cli fingerprint |
| `CLI_COMPAT_GITHUB=1` | GitHub Copilot Chat fingerprint |
| `CLI_COMPAT_ANTIGRAVITY=1` | Antigravity fingerprint |
| `CLI_COMPAT_KIRO=1` | Kiro |
| `CLI_COMPAT_CURSOR=1` | Cursor |
| `CLI_COMPAT_KIMI_CODING=1` | Kimi Coding |
| `CLI_COMPAT_KILOCODE=1` | KiloCode |
| `CLI_COMPAT_CLINE=1` | Cline |
| `CLI_COMPAT_QWEN=1` | Qwen Code |
| `CLI_COMPAT_ALL=1` | Enable all of the above |
The provider IP is **always preserved** — the toggle only reshapes the request wire image, it does not switch IP egress.
---
## Inbound Header Sanitization
OmniRoute scrubs inbound client headers before forwarding so a request that arrives from Cursor doesn't leak `User-Agent: Cursor/X.Y.Z` to a Claude upstream. See `src/shared/constants/upstreamHeaders.ts` for the denylist, kept in lockstep with the Zod schemas and unit tests.
---
## Updating Fingerprints When a Provider Rotates
1. Capture official CLI traffic with `mitmproxy` (TLS interception + dump)
2. Extract JA3/JA4 and the literal header order
3. Update the relevant `CLI_FINGERPRINTS[...]` entry
4. Bump matching `*_USER_AGENT` default in `.env.example`
5. If TLS handshake itself changed: update `chatgptTlsClient.ts::CHATGPT_PROFILE` or wreq-js `browser:` option
6. Run `chatgptTlsClient.test.ts` and a manual canary against the live provider
7. Ship in a patch release; document in `CHANGELOG.md`
---
## Tests
- `open-sse/services/__tests__/chatgptTlsClient.test.ts` — proxy resolution priority, abort handling, hang recovery
- `tests/unit/anthropic-cache-fingerprint.test.ts` — fingerprint determinism
- `tests/unit/chatgpt-web.test.ts` — end-to-end stealth path for ChatGPT
---
## See Also
- [RESILIENCE_GUIDE.md](../architecture/RESILIENCE_GUIDE.md) — what happens when a stealth path gets a `403`
- [TROUBLESHOOTING.md](../guides/TROUBLESHOOTING.md)
- [ENVIRONMENT.md](../reference/ENVIRONMENT.md) — full env reference
- [CLI-TOOLS.md](../reference/CLI-TOOLS.md) — operator view of the MITM workflow

163
Termux-Guide.md Normal file

@@ -0,0 +1,163 @@
# Termux Headless Setup
OmniRoute can run as a headless server on Android through Termux. The Electron desktop app is not supported in Termux, but the web dashboard and OpenAI-compatible API work from the local browser or from other devices on the same network.
## Prerequisites
Install Termux from F-Droid or GitHub releases, then update packages and install the build tools required by native dependencies such as `better-sqlite3`.
```bash
pkg update
pkg upgrade
pkg install nodejs-lts python build-essential git
```
> **Node.js version:** OmniRoute requires Node `>=20.20.2 <21 || >=22.22.2 <23 || >=24.0.0 <27` (per `engines` in `package.json`). Termux's `nodejs-lts` typically ships Node 20 LTS, which is compatible. If `node --version` reports an older line, install `pkg install nodejs` (current) and verify the major matches a supported range.
If native package compilation fails, rerun the `pkg install` command above and then retry the OmniRoute install.
## Install
Run the latest published package directly:
```bash
npx -y omniroute@latest
```
You can also install it globally:
```bash
npm install -g omniroute
omniroute
```
## Run
Start OmniRoute in headless server mode:
```bash
omniroute
```
or:
```bash
npx omniroute
```
The dashboard listens on:
```text
http://localhost:20128
```
Open that URL in the Android browser. If you run clients inside Termux, use the same host and port as the OpenAI-compatible base URL.
## Background Execution
For a simple background process:
```bash
nohup omniroute > omniroute.log 2>&1 &
```
To stop it:
```bash
pkill -f omniroute
```
For automatic startup after device boot, install the Termux:Boot add-on and create a boot script:
```bash
mkdir -p ~/.termux/boot
cat > ~/.termux/boot/omniroute.sh <<'EOF'
#!/data/data/com.termux/files/usr/bin/sh
cd "$HOME"
nohup omniroute > "$HOME/omniroute.log" 2>&1 &
EOF
chmod +x ~/.termux/boot/omniroute.sh
```
Android battery optimization can stop long-running background processes. Disable battery optimization for Termux if the server is expected to stay online.
## Access From Other Devices
Find the phone IP address on the WiFi network:
```bash
ip addr show wlan0
```
Then open the dashboard from another device:
```text
http://PHONE_IP:20128
```
For example:
```text
http://192.168.1.50:20128
```
Keep the phone and client on the same trusted network. If you expose OmniRoute outside the phone, enable API keys and dashboard authentication.
## Data Directory
By default OmniRoute stores data under the Termux home directory, following the same server-side data path behavior used on Linux. To place the database somewhere explicit:
```bash
export DATA_DIR="$HOME/.omniroute"
omniroute
```
## Limitations
- Electron does not run in Termux.
- There is no system tray or desktop integration.
- This setup is server-only: use the browser dashboard.
- Native dependencies may need local compilation.
- Low-memory Android devices may need fewer concurrent requests.
- MITM/system certificate features may require Android-level trust-store work outside Termux.
## Troubleshooting
### better-sqlite3 Build Errors
Install the Termux build toolchain:
```bash
pkg install nodejs-lts python build-essential
```
Then rerun:
```bash
npx -y omniroute@latest
```
### Port Already In Use
Check what is listening on the default port:
```bash
ss -ltnp | grep 20128
```
Stop the old process:
```bash
pkill -f omniroute
```
### Dashboard Not Reachable From Another Device
Verify both devices are on the same WiFi network, then test from Termux:
```bash
curl http://localhost:20128
```
If local access works but LAN access does not, check Android hotspot/WiFi isolation and any firewall or VPN profile on the phone.

474
Troubleshooting.md Normal file

@@ -0,0 +1,474 @@
# Troubleshooting
🌐 **Languages:** 🇺🇸 [English](./TROUBLESHOOTING.md) | 🇧🇷 [Português (Brasil)](../i18n/pt-BR/docs/guides/TROUBLESHOOTING.md) | 🇪🇸 [Español](../i18n/es/docs/guides/TROUBLESHOOTING.md) | 🇫🇷 [Français](../i18n/fr/docs/guides/TROUBLESHOOTING.md) | 🇮🇹 [Italiano](../i18n/it/docs/guides/TROUBLESHOOTING.md) | 🇷🇺 [Русский](../i18n/ru/docs/guides/TROUBLESHOOTING.md) | 🇨🇳 [中文 (简体)](../i18n/zh-CN/docs/guides/TROUBLESHOOTING.md) | 🇩🇪 [Deutsch](../i18n/de/docs/guides/TROUBLESHOOTING.md) | 🇮🇳 [हिन्दी](../i18n/in/docs/guides/TROUBLESHOOTING.md) | 🇹🇭 [ไทย](../i18n/th/docs/guides/TROUBLESHOOTING.md) | 🇺🇦 [Українська](../i18n/uk-UA/docs/guides/TROUBLESHOOTING.md) | 🇸🇦 [العربية](../i18n/ar/docs/guides/TROUBLESHOOTING.md) | 🇯🇵 [日本語](../i18n/ja/docs/guides/TROUBLESHOOTING.md) | 🇻🇳 [Tiếng Việt](../i18n/vi/docs/guides/TROUBLESHOOTING.md) | 🇧🇬 [Български](../i18n/bg/docs/guides/TROUBLESHOOTING.md) | 🇩🇰 [Dansk](../i18n/da/docs/guides/TROUBLESHOOTING.md) | 🇫🇮 [Suomi](../i18n/fi/docs/guides/TROUBLESHOOTING.md) | 🇮🇱 [עברית](../i18n/he/docs/guides/TROUBLESHOOTING.md) | 🇭🇺 [Magyar](../i18n/hu/docs/guides/TROUBLESHOOTING.md) | 🇮🇩 [Bahasa Indonesia](../i18n/id/docs/guides/TROUBLESHOOTING.md) | 🇰🇷 [한국어](../i18n/ko/docs/guides/TROUBLESHOOTING.md) | 🇲🇾 [Bahasa Melayu](../i18n/ms/docs/guides/TROUBLESHOOTING.md) | 🇳🇱 [Nederlands](../i18n/nl/docs/guides/TROUBLESHOOTING.md) | 🇳🇴 [Norsk](../i18n/no/docs/guides/TROUBLESHOOTING.md) | 🇵🇹 [Português (Portugal)](../i18n/pt/docs/guides/TROUBLESHOOTING.md) | 🇷🇴 [Română](../i18n/ro/docs/guides/TROUBLESHOOTING.md) | 🇵🇱 [Polski](../i18n/pl/docs/guides/TROUBLESHOOTING.md) | 🇸🇰 [Slovenčina](../i18n/sk/docs/guides/TROUBLESHOOTING.md) | 🇸🇪 [Svenska](../i18n/sv/docs/guides/TROUBLESHOOTING.md) | 🇵🇭 [Filipino](../i18n/phi/docs/guides/TROUBLESHOOTING.md) | 🇨🇿 [Čeština](../i18n/cs/docs/guides/TROUBLESHOOTING.md)
Common problems and solutions for OmniRoute.
---
## Quick Fixes
| Problem | Solution |
| --------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------- |
| First login not working | Set `INITIAL_PASSWORD` in `.env` (no hardcoded default) |
| Dashboard opens on wrong port | Set `PORT=20128` and `NEXT_PUBLIC_BASE_URL=http://localhost:20128` |
| No logs written to disk | Set `APP_LOG_TO_FILE=true` and verify call log capture is enabled |
| EACCES: permission denied | Set `DATA_DIR=/path/to/writable/dir` to override `~/.omniroute` |
| Routing strategy not saving | Update to the latest v3.x release (Zod schema fix for settings persistence shipped in earlier versions) |
| Login crash / blank page | Check Node.js version — see [Node.js Compatibility](#nodejs-compatibility) below |
| `dlopen` / `slice is not valid mach-o file` (macOS) | Run `cd $(npm root -g)/omniroute/app && npm rebuild better-sqlite3 && omniroute` — see [macOS native module rebuild](#macos-native-module-rebuild) below |
| Proxy "fetch failed" | Ensure proxy config is set at the correct level — see [Proxy Issues](#proxy-issues) below |
---
## Node.js Compatibility
<a name="nodejs-compatibility"></a>
### Login page crashes or shows "Module self-registration" error
**Cause:** You are running a Node.js version outside OmniRoute's approved secure runtime floor. The most common case is running an older Node 20, 22, or 24 patch level that falls below the patched security floor OmniRoute requires.
**Symptoms:**
- Login page shows a blank screen or a server error
- Console shows `Error: Module did not self-register` or similar native binding errors
- The login page shows an **orange warning banner** with your Node version if the runtime is outside the supported secure policy
**Fix:**
1. Install a supported Node.js LTS release (recommended: Node.js 24.x):
```bash
nvm install 24
nvm use 24
```
2. Verify your version: `node --version` should show `v24.0.0` or newer on the 24.x LTS line
3. Reinstall OmniRoute: `npm install -g omniroute`
4. Restart: `omniroute`
> **Supported secure versions:** `>=20.20.2 <21`, `>=22.22.2 <23`, or `>=24.0.0 <27`. Node.js 24.x LTS (Krypton) and Node.js 26 are fully supported.
### macOS: `dlopen` / "slice is not valid mach-o file"
<a name="macos-native-module-rebuild"></a>
**Cause:** After a global `npm install -g omniroute`, the `better-sqlite3` native binary inside the package may have been compiled for a different architecture or Node.js ABI than what is running locally. This is common on macOS (both Apple Silicon and Intel) when the pre-built binary does not match your environment.
**Symptoms:**
- Server fails immediately on startup with a `dlopen` error
- Error contains `slice is not valid mach-o file`
- Full example:
```
dlopen(/Users/<user>/.nvm/versions/node/v24.14.1/lib/node_modules/omniroute/app/node_modules/better-sqlite3/build/Release/better_sqlite3.node, 0x0001): tried: '...' (slice is not valid mach-o file)
```
**Fix — rebuild for your local environment (no Node.js downgrade required):**
```bash
cd $(npm root -g)/omniroute/app
npm rebuild better-sqlite3
omniroute
```
> **Note:** This recompiles the native binding against your local Node.js version and CPU architecture, resolving the binary mismatch. The officially supported range is **`>=20.20.2 <21`, `>=22.22.2 <23`, or `>=24.0.0 <27`** (`engines` field in `package.json`). Node.js 24.x LTS (Krypton) and Node.js 26 are fully supported with `better-sqlite3` v12.x.
---
## Proxy Issues
<a name="proxy-issues"></a>
### Provider validation shows "fetch failed"
**Cause:** The API key validation endpoint (`POST /api/providers/validate`) was previously bypassing proxy configuration, causing failures in environments that require proxy routing.
**Fix (v3.5.5+):** This is now fixed. Provider validation routes through `runWithProxyContext`, honoring provider-level and global proxy settings automatically.
### Token health check fails with "fetch failed"
**Cause:** Background OAuth token refresh was not resolving proxy configuration per connection.
**Fix (v3.5.5+):** The token health check scheduler now resolves proxy config per connection before attempting refresh. Update to v3.5.5+.
### SOCKS5 proxy returns "invalid onRequestStart method"
**Cause:** On Node.js 22, the undici@8 dispatcher is incompatible with Node's built-in `fetch()` implementation.
**Fix (v3.5.5+):** OmniRoute now uses undici's own `fetch()` function when a proxy dispatcher is active, ensuring consistent behavior. Update to v3.5.5+.
---
## Provider Issues
### "Language model did not provide messages"
**Cause:** Provider quota exhausted.
**Fix:**
1. Check dashboard quota tracker
2. Use a combo with fallback tiers
3. Switch to cheaper/free tier
### Rate Limiting
**Cause:** Subscription quota exhausted.
**Fix:**
- Add fallback: `cc/claude-opus-4-6 → glm/glm-4.7 → if/kimi-k2-thinking`
- Use GLM/MiniMax as cheap backup
### OAuth Token Expired
OmniRoute auto-refreshes tokens. If issues persist:
1. Dashboard → Provider → Reconnect
2. Delete and re-add the provider connection
### Kiro multi-account: second account invalidates the first
**Cause:** Kiro's backend enforces a single active session per OIDC client registration.
When two accounts share the same registered client (connections imported before v3.8.0),
refreshing one account's token invalidates the other's refresh token.
**Fix (v3.8.0+):** Re-import affected connections.
Starting with v3.8.0, every new Kiro connection created via **Import Token**,
**Google/GitHub social login**, or **Auto-Import** automatically registers its own
dedicated OIDC client. The connection is therefore fully isolated and refreshing one
account has no effect on any other account.
Connections that were imported _before_ v3.8.0 do not carry a per-connection client
registration. Those connections continue to use the shared social-auth refresh endpoint.
To gain isolation, delete the old connection from Dashboard → Providers and re-add it
via any of the three import flows.
For full details and step-by-step instructions for adding two Kiro accounts side by side,
see [`docs/guides/KIRO_SETUP.md`](./KIRO_SETUP.md).
---
## Cloud Issues
### Cloud Sync Errors
1. Verify `BASE_URL` points to your running instance (e.g., `http://localhost:20128`)
2. Verify `CLOUD_URL` points to your cloud endpoint (e.g., `https://omniroute.dev`)
3. Keep `NEXT_PUBLIC_*` values aligned with server-side values
### Cloud `stream=false` Returns 500
**Symptom:** `Unexpected token 'd'...` on cloud endpoint for non-streaming calls.
**Cause:** Upstream returns SSE payload while client expects JSON.
**Workaround:** Use `stream=true` for cloud direct calls. Local runtime includes SSE→JSON fallback.
### Cloud Says Connected but "Invalid API key"
1. Create a fresh key from local dashboard (`/api/keys`)
2. Run cloud sync: Enable Cloud → Sync Now
3. Old/non-synced keys can still return `401` on cloud
---
## Docker Issues
### CLI Tool Shows Not Installed
1. Check runtime fields: `curl http://localhost:20128/api/cli-tools/runtime/codex | jq`
2. For portable mode: use image target `runner-cli` (bundled CLIs)
3. For host mount mode: set `CLI_EXTRA_PATHS` and mount host bin directory as read-only
4. If `installed=true` and `runnable=false`: binary was found but failed healthcheck
### Quick Runtime Validation
```bash
curl -s http://localhost:20128/api/cli-tools/codex-settings | jq '{installed,runnable,commandPath,runtimeMode,reason}'
curl -s http://localhost:20128/api/cli-tools/claude-settings | jq '{installed,runnable,commandPath,runtimeMode,reason}'
curl -s http://localhost:20128/api/cli-tools/openclaw-settings | jq '{installed,runnable,commandPath,runtimeMode,reason}'
```
---
## Cost Issues
### High Costs
1. Check usage stats in Dashboard → Usage
2. Switch primary model to GLM/MiniMax
3. Use free tier (Gemini CLI, Qoder) for non-critical tasks
4. Set cost budgets per API key: Dashboard → API Keys → Budget
---
## Debugging
### Enable Log Files
Set `APP_LOG_TO_FILE=true` in your `.env` file. Application logs are written under `logs/`.
Request artifacts are stored under `${DATA_DIR}/call_logs/` when the call log pipeline is
enabled in settings.
When pipeline capture is enabled, set `CALL_LOG_PIPELINE_CAPTURE_STREAM_CHUNKS=false` to omit
stream chunk payloads, or tune `CALL_LOG_PIPELINE_MAX_SIZE_KB` to change the artifact cap in KB.
### Check Provider Health
```bash
# Health dashboard
http://localhost:20128/dashboard/health
# API health check
curl http://localhost:20128/api/monitoring/health
```
### Runtime Storage
- Main state: `${DATA_DIR}/storage.sqlite` (providers, combos, aliases, keys, settings)
- Usage: SQLite tables in `storage.sqlite` (`usage_history`, `call_logs`, `proxy_logs`) + optional `${DATA_DIR}/call_logs/`
- Application logs: `<repo>/logs/...` (when `APP_LOG_TO_FILE=true`)
- Call log artifacts: `${DATA_DIR}/call_logs/YYYY-MM-DD/...` when the call log pipeline is enabled
---
## Circuit Breaker Issues
### Provider stuck in OPEN state
When a provider's circuit breaker is OPEN, requests are blocked until the cooldown expires.
**Fix:**
1. Go to **Dashboard → Settings → Resilience**
2. Check the circuit breaker card for the affected provider
3. Click **Reset All** to clear all breakers, or wait for the cooldown to expire
4. Verify the provider is actually available before resetting
### Provider keeps tripping the circuit breaker
If a provider repeatedly enters OPEN state:
1. Check **Dashboard → Health → Provider Health** for the failure pattern
2. Go to **Settings → Resilience → Provider Profiles** and increase the failure threshold
3. Check if the provider has changed API limits or requires re-authentication
4. Review latency telemetry — high latency may cause timeout-based failures
---
## Audio Transcription Issues
### "Unsupported model" error
- Ensure you're using the correct prefix: `deepgram/nova-3` or `assemblyai/best`
- Verify the provider is connected in **Dashboard → Providers**
### Transcription returns empty or fails
- Check supported audio formats: `mp3`, `wav`, `m4a`, `flac`, `ogg`, `webm`
- Verify file size is within provider limits (typically < 25MB)
- Check provider API key validity in the provider card
---
## Translator Debugging
Use **Dashboard → Translator** to debug format translation issues:
| Mode | When to Use |
| ---------------- | -------------------------------------------------------------------------------------------- |
| **Playground** | Compare input/output formats side by side — paste a failing request to see how it translates |
| **Chat Tester** | Send live messages and inspect the full request/response payload including headers |
| **Test Bench** | Run batch tests across format combinations to find which translations are broken |
| **Live Monitor** | Watch real-time request flow to catch intermittent translation issues |
### Common format issues
- **Thinking tags not appearing** — Check if the target provider supports thinking and the thinking budget setting
- **Tool calls dropping** — Some format translations may strip unsupported fields; verify in Playground mode
- **System prompt missing** — Claude and Gemini handle system prompts differently; check translation output
- **SDK returns raw string instead of object** — Resolved in v1.x; response sanitizer strips non-standard fields (`x_groq`, `usage_breakdown`, etc.) that cause OpenAI SDK Pydantic validation failures. If you still see this on v3.x+, please file an issue.
- **GLM/ERNIE rejects `system` role** — Resolved in v1.x; role normalizer automatically merges system messages into user messages for incompatible models. If you still see this on v3.x+, please file an issue.
- **`developer` role not recognized** — Resolved in v1.x; automatically converted to `system` for non-OpenAI providers. If you still see this on v3.x+, please file an issue.
- **`json_schema` not working with Gemini** — Resolved in v1.x; `response_format` is now converted to Gemini's `responseMimeType` + `responseSchema`. If you still see this on v3.x+, please file an issue.
---
## Resilience Settings
### Auto rate-limit not triggering
- Auto rate-limit only applies to API key providers (not OAuth/subscription)
- Verify **Settings → Resilience → Provider Profiles** has auto-rate-limit enabled
- Check if the provider returns `429` status codes or `Retry-After` headers
### Tuning exponential backoff
Provider profiles support these settings:
- **Base delay** — Initial wait time after first failure (default: 1s)
- **Max delay** — Maximum wait time cap (default: 30s)
- **Multiplier** — How much to increase delay per consecutive failure (default: 2x)
### Anti-thundering herd
When many concurrent requests hit a rate-limited provider, OmniRoute uses mutex + auto rate-limiting to serialize requests and prevent cascading failures. This is automatic for API key providers.
---
## Optional RAG / LLM failure taxonomy (16 problems)
Some OmniRoute users place the gateway in front of RAG or agent stacks. In those setups it is common to see a strange pattern: OmniRoute looks healthy (providers up, routing profiles ok, no rate limit alerts) but the final answer is still wrong.
In practice these incidents usually come from the downstream RAG pipeline, not from the gateway itself.
If you want a shared vocabulary to describe those failures you can use the WFGY ProblemMap, an external MIT license text resource that defines sixteen recurring RAG / LLM failure patterns. At a high level it covers:
- retrieval drift and broken context boundaries
- empty or stale indexes and vector stores
- embedding versus semantic mismatch
- prompt assembly and context window issues
- logic collapse and overconfident answers
- long chain and agent coordination failures
- multi agent memory and role drift
- deployment and bootstrap ordering problems
The idea is simple:
1. When you investigate a bad response, capture:
- user task and request
- route or provider combo in OmniRoute
- any RAG context used downstream (retrieved documents, tool calls, etc)
2. Map the incident to one or two WFGY ProblemMap numbers (`No.1` … `No.16`).
3. Store the number in your own dashboard, runbook, or incident tracker next to the OmniRoute logs.
4. Use the corresponding WFGY page to decide whether you need to change your RAG stack, retriever, or routing strategy.
Full text and concrete recipes live here (MIT license, text only):
[WFGY ProblemMap README](https://github.com/onestardao/WFGY/blob/main/ProblemMap/README.md)
You can ignore this section if you do not run RAG or agent pipelines behind OmniRoute.
---
## v3.8.0 Known Issues
Issues specific to the v3.8.0 release and their current workarounds. If a fix lands in a later patch, the entry will be updated or removed.
### Windsurf OAuth flow fails with 401
**Symptoms:**
- "401 unauthorized" while completing the Windsurf OAuth flow from the dashboard
- Windsurf provider card stays in "needs reconnection" state after the callback
**Causes:**
- `WINDSURF_FIREBASE_API_KEY` env var missing or empty
- `WINDSURF_API_KEY` misconfigured or pointing at a stale token
- Local firewall/proxy blocking the OAuth callback
**Fix:**
1. Verify both `WINDSURF_FIREBASE_API_KEY` and `WINDSURF_API_KEY` are set in `.env`
2. Restart OmniRoute so the new env values are picked up
3. Re-run the OAuth flow from **Dashboard → Providers → Windsurf → Reconnect**
### Devin CLI auth failures
**Symptoms:**
- "Devin CLI not found" or "auth failed" when invoking Devin-backed tools
- CLI runtime check reports `installed=false`
**Causes:**
- `CLI_DEVIN_BIN` points to a path that does not exist
- Devin CLI is not installed on the host
**Fix:**
1. Install the Devin CLI for your platform
2. Set `CLI_DEVIN_BIN=/usr/local/bin/devin` (or the real path) in `.env`
3. Restart OmniRoute and re-test from **Dashboard → CLI Tools**
### Model cooldown stuck (manual reset)
**Symptoms:**
- A model stays listed in cooldown even after the expiration time has passed
- Requests still skip the model in combo routing despite the timestamp being in the past
**Manual reset:**
- **Dashboard:** **Settings → Model Cooldowns** → click **Re-enable** on the affected card
- **API:** `DELETE /api/resilience/model-cooldowns` with management auth headers
### Command Code provider connection fails with 403
**Symptoms:**
- 403 when testing the Command Code provider connection
- The provider card shows "unauthorized" after a fresh add
**Cause:** The OAuth flow did not complete (callback not received or token not persisted).
**Fix:**
- Run `omniroute providers` from the CLI to re-trigger the OAuth flow, or
- Re-run OAuth from **Dashboard → Providers → Command Code → Reconnect**
### ModelScope returns aggressive 429 cooldowns
**Symptoms:**
- Very short or immediate cooldowns on ModelScope after a small burst of requests
- Combo routing skips ModelScope earlier than expected
**Cause:** ModelScope emits provider-specific `Retry-After` headers. v3.8.0 ships dedicated handling for those headers, so older versions misread them as generic rate-limit hints.
**Fix:**
- Ensure you are on v3.8.0 or later
- Verify the `useUpstream429BreakerHints` toggle is enabled under **Settings → Resilience**
### OMNIROUTE_WS_BRIDGE_SECRET missing in production
**Symptoms:**
- 401 on every Codex/Responses WebSocket bridge request when running on a remote production host
- WebSocket bridge handshake closes immediately after connect
**Cause:** The `OMNIROUTE_WS_BRIDGE_SECRET` env var is missing from the production environment.
**Fix:**
1. Generate a random secret: `openssl rand -hex 32`
2. Set `OMNIROUTE_WS_BRIDGE_SECRET=<random-secret>` in the production server env (and any client that talks to the bridge)
3. Restart OmniRoute
### Responses API: background mode degraded to synchronous
**Symptoms:**
- Warning logged: `background mode degraded to synchronous`
- A `background: true` request returns a normal synchronous response instead of a background job handle
**Cause:** v3.8.0 intentionally degrades `background: true` on the Responses API to synchronous execution while emitting a warning. Full async background execution is a future deliverable.
**Fix:**
- Adjust the client to call without `background`, or
- Wait for a later release that ships full async background mode (track the changelog)
---
## Still Stuck?
- **GitHub Issues**: [github.com/diegosouzapw/OmniRoute/issues](https://github.com/diegosouzapw/OmniRoute/issues)
- **Architecture**: See [`docs/architecture/ARCHITECTURE.md`](../architecture/ARCHITECTURE.md) for internal details
- **API Reference**: See [`docs/reference/API_REFERENCE.md`](../reference/API_REFERENCE.md) for all endpoints
- **Health Dashboard**: Check **Dashboard → Health** for real-time system status
- **Translator**: Use **Dashboard → Translator** to debug format issues

282
Tunnels-Guide.md Normal file

@@ -0,0 +1,282 @@
# Tunnels Guide
> **Source of truth:** `src/lib/{cloudflaredTunnel,ngrokTunnel,tailscaleTunnel}.ts`, `src/app/api/tunnels/`
> **Last updated:** 2026-05-13 — v3.8.0
OmniRoute can expose its local server (`http://localhost:20128`) to the public
internet via three tunnel backends. This is useful for:
- OAuth callbacks from cloud providers (Antigravity, Gemini, Cursor) that need a
publicly reachable redirect URL.
- Sharing your local instance with teammates without deploying a VM.
- Mobile, remote, or cross-network testing.
All three backends are managed in-process — OmniRoute starts/stops the underlying
binary or SDK from the dashboard or REST API. No reverse-proxy or systemd setup
is required.
## Backends at a glance
| Backend | Persistence | Cost | Setup |
| --------------------------- | ------------------------------------------------------ | ----------------- | ----------------------------------------------- |
| **Cloudflare Quick Tunnel** | Ephemeral (URL changes each restart) | Free | Zero — auto-installs `cloudflared` |
| **ngrok** | Stable while a paid plan or fixed domain is configured | Free tier + paid | Requires ngrok account + authtoken |
| **Tailscale Funnel** | Stable per node within your tailnet | Free for personal | Requires Tailscale install + login + Funnel ACL |
The implementations live in `src/lib/cloudflaredTunnel.ts`,
`src/lib/ngrokTunnel.ts`, and `src/lib/tailscaleTunnel.ts`. All three return a
common-shaped `status` object with `phase`, `running`, `publicUrl`, `apiUrl`,
`targetUrl`, and `lastError` fields, so the dashboard can render them uniformly.
## 1. Cloudflare Tunnel (Quick Tunnel)
`src/lib/cloudflaredTunnel.ts` runs `cloudflared tunnel --url
http://localhost:<apiPort>` as a child process and parses the assigned
`*.trycloudflare.com` URL from stdout.
Key behaviors:
- **Auto-install.** On first use, OmniRoute downloads the latest `cloudflared`
binary from the official GitHub releases (managed install lives under
`DATA_DIR/cloudflared/`). SHA256 of the downloaded asset is verified against the
release manifest before execution.
- **Quick-tunnel only.** The current implementation runs only the
`--url`-style quick tunnel. Named/persistent tunnels (`cloudflared tunnel
login` + `cloudflared tunnel route dns ...`) are not orchestrated by
OmniRoute. URLs are ephemeral and will change every restart.
- **Process supervision.** The cloudflared PID and resolved URL are persisted to
`cloudflared-state.json` so the dashboard can resume status across reloads.
### Enable / disable via REST
The endpoint uses an `{action: "enable" | "disable"}` body, not separate
`start`/`stop` paths. Management auth (admin session or admin API key) is
required.
```bash
# Enable
curl -X POST http://localhost:20128/api/tunnels/cloudflared \
-H "Content-Type: application/json" \
-H "Cookie: auth_token=..." \
-d '{"action":"enable"}'
# Status
curl http://localhost:20128/api/tunnels/cloudflared \
-H "Cookie: auth_token=..."
# Disable
curl -X POST http://localhost:20128/api/tunnels/cloudflared \
-H "Content-Type: application/json" \
-H "Cookie: auth_token=..." \
-d '{"action":"disable"}'
```
Or via dashboard: **Settings → Tunnels → Cloudflare**.
### Optional env vars
| Variable | Purpose |
| ---------------------------------------------------- | ------------------------------------------------------------------------------------- |
| `CLOUDFLARED_BIN` | Override the binary path. If set and valid, OmniRoute uses it instead of downloading. |
| `CLOUDFLARED_PROTOCOL` / `TUNNEL_TRANSPORT_PROTOCOL` | Transport protocol (default `http2`). |
## 2. ngrok
`src/lib/ngrokTunnel.ts` uses the **`@ngrok/ngrok` SDK** (in-process, no CLI
subprocess). The native module is imported lazily on first start so platforms
without prebuilt binaries do not break the app at boot.
### Prerequisites
1. Sign up at <https://ngrok.com>.
2. Copy your authtoken from the ngrok dashboard.
3. Provide it either via:
- `.env`: `NGROK_AUTHTOKEN=<token>`, or
- Dashboard: **Settings → Tunnels → ngrok**, or
- REST body (one-shot): `{"action":"enable","authToken":"<token>"}`.
If neither is configured, status returns `phase: "needs_auth"`.
### Enable / disable via REST
```bash
# Enable (uses NGROK_AUTHTOKEN from env)
curl -X POST http://localhost:20128/api/tunnels/ngrok \
-H "Content-Type: application/json" \
-H "Cookie: auth_token=..." \
-d '{"action":"enable"}'
# Enable with inline token
curl -X POST http://localhost:20128/api/tunnels/ngrok \
-H "Content-Type: application/json" \
-H "Cookie: auth_token=..." \
-d '{"action":"enable","authToken":"2abc..."}'
# Status
curl http://localhost:20128/api/tunnels/ngrok \
-H "Cookie: auth_token=..."
# Disable
curl -X POST http://localhost:20128/api/tunnels/ngrok \
-H "Content-Type: application/json" \
-H "Cookie: auth_token=..." \
-d '{"action":"disable"}'
```
The response includes the assigned `publicUrl` (e.g.
`https://abcd-1234.ngrok-free.app`). Custom domains, regions, and policy rules
must be configured in the ngrok dashboard — OmniRoute itself only forwards the
local target URL to the SDK.
## 3. Tailscale Funnel
`src/lib/tailscaleTunnel.ts` orchestrates the system `tailscale` CLI to expose
the local API port via **Funnel** (Tailscale's public-internet egress for serve).
It supports the full lifecycle: install, login, daemon start, enable, disable.
The implementation invokes `tailscale funnel --bg <port>` (background mode). The
public URL has the shape `https://<machine>.<tailnet>.ts.net/`.
### Prerequisites
1. Install Tailscale (or let OmniRoute do it — see `install` endpoint below).
2. Sign in (`tailscale login` or via OmniRoute's `login` endpoint).
3. Enable Funnel for your tailnet in the Tailscale admin console:
<https://login.tailscale.com/admin/settings/features>.
On Linux and macOS the daemon (`tailscaled`) requires `sudo` to control. The
POST endpoints accept an optional `sudoPassword` field which is forwarded to
OmniRoute's MITM password cache (`getCachedPassword` / `setCachedPassword`) for
the duration of the call. Windows uses the default service install at
`C:\Program Files\Tailscale\tailscale.exe`.
### REST endpoints
Tailscale has a richer surface than the other backends because installation,
login, daemon, and tunnel are separate concerns.
| Endpoint | Method | Purpose |
| ------------------------------------- | ------ | --------------------------------------------------------------- |
| `/api/tunnels/tailscale` | `GET` | Aggregated tunnel status (`phase`, `tunnelUrl`, `apiUrl`, etc.) |
| `/api/tunnels/tailscale/check` | `GET` | Lower-level check: installed? logged in? daemon running? |
| `/api/tunnels/tailscale/install` | `POST` | Install Tailscale (SSE-streamed progress events) — Linux/macOS |
| `/api/tunnels/tailscale/start-daemon` | `POST` | Start `tailscaled` on Linux/macOS |
| `/api/tunnels/tailscale/login` | `POST` | Begin login flow; returns `authUrl` to open in a browser |
| `/api/tunnels/tailscale/enable` | `POST` | Start the Funnel for the API port |
| `/api/tunnels/tailscale/disable` | `POST` | Stop the Funnel |
All Tailscale endpoints require management auth (see `routeUtils.ts ::
requireTailscaleAuth`).
Example enable:
```bash
curl -X POST http://localhost:20128/api/tunnels/tailscale/enable \
-H "Content-Type: application/json" \
-H "Cookie: auth_token=..." \
-d '{"sudoPassword":"<linux-pwd>","port":20128}'
```
If Funnel is not enabled in the admin console, the response includes
`funnelNotEnabled: true` plus an `enableUrl` to open in a browser.
### Optional env vars
| Variable | Purpose |
| --------------- | ------------------------------------ |
| `TAILSCALE_BIN` | Override the `tailscale` binary path |
## Endpoint summary
| Endpoint | Method | Body | Auth |
| ------------------------------------- | ------ | ----------------------------------- | ---------- |
| `/api/tunnels/cloudflared` | `GET` | — | management |
| `/api/tunnels/cloudflared` | `POST` | `{action: "enable" \| "disable"}` | management |
| `/api/tunnels/ngrok` | `GET` | — | management |
| `/api/tunnels/ngrok` | `POST` | `{action, authToken?}` | management |
| `/api/tunnels/tailscale` | `GET` | — | management |
| `/api/tunnels/tailscale/check` | `GET` | — | management |
| `/api/tunnels/tailscale/install` | `POST` | `{sudoPassword?}` (SSE) | management |
| `/api/tunnels/tailscale/start-daemon` | `POST` | `{sudoPassword?}` | management |
| `/api/tunnels/tailscale/login` | `POST` | `{hostname?}` | management |
| `/api/tunnels/tailscale/enable` | `POST` | `{sudoPassword?, hostname?, port?}` | management |
| `/api/tunnels/tailscale/disable` | `POST` | `{sudoPassword?}` | management |
There is no central `/api/settings/tunnels` endpoint — each backend is
independent.
## OAuth callback considerations
When you expose OmniRoute through a tunnel, the dashboard and OAuth flows must
build callback URLs against the **public** hostname, not `localhost`. Otherwise
the OAuth provider redirects the user back to a URL its servers cannot reach,
and the handshake fails.
Set:
```bash
NEXT_PUBLIC_BASE_URL=https://<your-tunnel-host>
```
and restart OmniRoute before initiating OAuth. For ephemeral Cloudflare Quick
Tunnels the URL changes after every restart, so prefer ngrok with a reserved
domain or Tailscale Funnel for production OAuth use.
## Health and monitoring
The dashboard surfaces tunnel state under **Settings → Tunnels**:
- Active backend(s) and current `phase` (`stopped`, `starting`, `running`,
`needs_auth`, `error`).
- The current public URL and the derived API URL (`<publicUrl>/v1`).
- The local target URL the tunnel is forwarding to.
- Last error message, if any.
For programmatic monitoring poll the per-backend `GET` endpoints. Running more
than one backend simultaneously is allowed; OmniRoute will track each
independently.
## Troubleshooting
### "cloudflared binary not found"
OmniRoute attempts to auto-install on first use. If the install is blocked
(restricted network, no GitHub access), download `cloudflared` manually from
<https://github.com/cloudflare/cloudflared/releases> and set
`CLOUDFLARED_BIN=/path/to/cloudflared`.
### "ngrok: authtoken required"
`phase: "needs_auth"` means no authtoken was found. Set `NGROK_AUTHTOKEN` in
`.env`, configure it via the dashboard, or pass `authToken` in the enable POST
body.
### "tailscale: funnel not enabled"
When the enable response includes `funnelNotEnabled: true`, Funnel is disabled
for your tailnet. Open the returned `enableUrl` (or the admin console feature
page) and toggle Funnel on.
### Tunnel URL changes break OAuth
Use ngrok with a reserved domain or Tailscale Funnel (both stable per-node).
Cloudflare Quick Tunnels are ephemeral by design and not recommended for
long-lived OAuth callbacks.
### Permission denied on Linux/macOS for Tailscale
`tailscaled` needs root. Provide `sudoPassword` to the relevant POST endpoint,
or run the daemon yourself (`sudo systemctl start tailscaled`).
## See also
- [PROXY_GUIDE.md](./PROXY_GUIDE.md) — outbound proxy (1proxy, SOCKS5, HTTP) for
egress traffic.
- [ENVIRONMENT.md](../reference/ENVIRONMENT.md) — full list of env vars including
`NEXT_PUBLIC_BASE_URL`.
- [FLY_IO_DEPLOYMENT_GUIDE.md](./FLY_IO_DEPLOYMENT_GUIDE.md),
[DOCKER_GUIDE.md](../guides/DOCKER_GUIDE.md) — alternatives to tunneling for stable
public hosting.
- Source: `src/lib/{cloudflaredTunnel,ngrokTunnel,tailscaleTunnel}.ts`,
`src/app/api/tunnels/`.

156
Uninstall.md Normal file

@@ -0,0 +1,156 @@
# OmniRoute — Uninstall Guide
🌐 **Languages:** 🇺🇸 [English](./UNINSTALL.md) | 🇧🇷 [Português (Brasil)](../i18n/pt-BR/docs/guides/UNINSTALL.md) | 🇪🇸 [Español](../i18n/es/docs/guides/UNINSTALL.md) | 🇫🇷 [Français](../i18n/fr/docs/guides/UNINSTALL.md) | 🇮🇹 [Italiano](../i18n/it/docs/guides/UNINSTALL.md) | 🇷🇺 [Русский](../i18n/ru/docs/guides/UNINSTALL.md) | 🇨🇳 [中文 (简体)](../i18n/zh-CN/docs/guides/UNINSTALL.md) | 🇩🇪 [Deutsch](../i18n/de/docs/guides/UNINSTALL.md) | 🇮🇳 [हिन्दी](../i18n/in/docs/guides/UNINSTALL.md) | 🇹🇭 [ไทย](../i18n/th/docs/guides/UNINSTALL.md) | 🇺🇦 [Українська](../i18n/uk-UA/docs/guides/UNINSTALL.md) | 🇸🇦 [العربية](../i18n/ar/docs/guides/UNINSTALL.md) | 🇯🇵 [日本語](../i18n/ja/docs/guides/UNINSTALL.md) | 🇻🇳 [Tiếng Việt](../i18n/vi/docs/guides/UNINSTALL.md) | 🇧🇬 [Български](../i18n/bg/docs/guides/UNINSTALL.md) | 🇩🇰 [Dansk](../i18n/da/docs/guides/UNINSTALL.md) | 🇫🇮 [Suomi](../i18n/fi/docs/guides/UNINSTALL.md) | 🇮🇱 [עברית](../i18n/he/docs/guides/UNINSTALL.md) | 🇭🇺 [Magyar](../i18n/hu/docs/guides/UNINSTALL.md) | 🇮🇩 [Bahasa Indonesia](../i18n/id/docs/guides/UNINSTALL.md) | 🇰🇷 [한국어](../i18n/ko/docs/guides/UNINSTALL.md) | 🇲🇾 [Bahasa Melayu](../i18n/ms/docs/guides/UNINSTALL.md) | 🇳🇱 [Nederlands](../i18n/nl/docs/guides/UNINSTALL.md) | 🇳🇴 [Norsk](../i18n/no/docs/guides/UNINSTALL.md) | 🇵🇹 [Português (Portugal)](../i18n/pt/docs/guides/UNINSTALL.md) | 🇷🇴 [Română](../i18n/ro/docs/guides/UNINSTALL.md) | 🇵🇱 [Polski](../i18n/pl/docs/guides/UNINSTALL.md) | 🇸🇰 [Slovenčina](../i18n/sk/docs/guides/UNINSTALL.md) | 🇸🇪 [Svenska](../i18n/sv/docs/guides/UNINSTALL.md) | 🇵🇭 [Filipino](../i18n/phi/docs/guides/UNINSTALL.md) | 🇨🇿 [Čeština](../i18n/cs/docs/guides/UNINSTALL.md)
This guide covers how to cleanly remove OmniRoute from your system.
---
## Quick Uninstall (v3.6.2+)
OmniRoute provides two built-in scripts for clean removal:
### Keep Your Data
```bash
npm run uninstall
```
This removes the OmniRoute application but **preserves** your database, configurations, API keys, and provider settings in `~/.omniroute/`. Use this if you plan to reinstall later and want to keep your setup.
### Full Removal
```bash
npm run uninstall:full
```
This removes the application **and permanently erases** all data:
- Database (`storage.sqlite`)
- Provider configurations and API keys
- Backup files
- Log files
- All files in the `~/.omniroute/` directory
> ⚠️ **Warning:** `npm run uninstall:full` is irreversible. All your provider connections, combos, API keys, and usage history will be permanently deleted.
---
## Manual Uninstall
### NPM Global Install
```bash
# Remove the global package
npm uninstall -g omniroute
# (Optional) Remove data directory
rm -rf ~/.omniroute
```
### pnpm Global Install
```bash
pnpm uninstall -g omniroute
rm -rf ~/.omniroute
```
### Docker
```bash
# Stop and remove the container
docker stop omniroute
docker rm omniroute
# Remove the volume (deletes all data)
docker volume rm omniroute-data
# (Optional) Remove the image
docker rmi diegosouzapw/omniroute:latest
```
### Docker Compose
```bash
# Stop and remove containers
docker compose down
# Also remove volumes (deletes all data)
docker compose down -v
```
### Electron Desktop App
**Windows:**
- Open `Settings → Apps → OmniRoute → Uninstall`
- Or run the NSIS uninstaller from the install directory
**macOS:**
- Drag `OmniRoute.app` from `/Applications` to Trash
- Remove data: `rm -rf ~/Library/Application Support/omniroute`
**Linux:**
- Remove the AppImage file
- Remove data: `rm -rf ~/.omniroute`
### Source Install (git clone)
```bash
# Remove the cloned directory
rm -rf /path/to/omniroute
# (Optional) Remove data directory
rm -rf ~/.omniroute
```
---
## Data Directories
OmniRoute stores data in the following locations by default:
| Platform | Default Path | Override |
| ------------- | ----------------------------- | ------------------------- |
| Linux | `~/.omniroute/` | `DATA_DIR` env var |
| macOS | `~/.omniroute/` | `DATA_DIR` env var |
| Windows | `%APPDATA%/omniroute/` | `DATA_DIR` env var |
| Docker | `/app/data/` (mounted volume) | `DATA_DIR` env var |
| XDG-compliant | `$XDG_CONFIG_HOME/omniroute/` | `XDG_CONFIG_HOME` env var |
### Files in the data directory
| File/Directory | Description |
| -------------------- | ------------------------------------------------- |
| `storage.sqlite` | Main database (providers, combos, settings, keys) |
| `storage.sqlite-wal` | SQLite write-ahead log (temporary) |
| `storage.sqlite-shm` | SQLite shared memory (temporary) |
| `call_logs/` | Request payload archives |
| `backups/` | Automatic database backups |
| `log.txt` | Legacy request log (optional) |
---
## Verify Complete Removal
After uninstalling, verify there are no remaining files:
```bash
# Check for global npm package
npm list -g omniroute 2>/dev/null
# Check for data directory
ls -la ~/.omniroute/ 2>/dev/null
# Check for running processes
pgrep -f omniroute
```
If any process is still running, stop it:
```bash
pkill -f omniroute
```

1219
User-Guide.md Normal file

File diff suppressed because it is too large Load Diff

406
VM-Deployment-Guide.md Normal file

@@ -0,0 +1,406 @@
# OmniRoute — Deployment Guide on VM with Cloudflare
🌐 **Languages:** 🇺🇸 [English](./VM_DEPLOYMENT_GUIDE.md) | 🇧🇷 [Português (Brasil)](../i18n/pt-BR/docs/ops/VM_DEPLOYMENT_GUIDE.md) | 🇪🇸 [Español](../i18n/es/docs/ops/VM_DEPLOYMENT_GUIDE.md) | 🇫🇷 [Français](../i18n/fr/docs/ops/VM_DEPLOYMENT_GUIDE.md) | 🇮🇹 [Italiano](../i18n/it/docs/ops/VM_DEPLOYMENT_GUIDE.md) | 🇷🇺 [Русский](../i18n/ru/docs/ops/VM_DEPLOYMENT_GUIDE.md) | 🇨🇳 [中文 (简体)](../i18n/zh-CN/docs/ops/VM_DEPLOYMENT_GUIDE.md) | 🇩🇪 [Deutsch](../i18n/de/docs/ops/VM_DEPLOYMENT_GUIDE.md) | 🇮🇳 [हिन्दी](../i18n/in/docs/ops/VM_DEPLOYMENT_GUIDE.md) | 🇹🇭 [ไทย](../i18n/th/docs/ops/VM_DEPLOYMENT_GUIDE.md) | 🇺🇦 [Українська](../i18n/uk-UA/docs/ops/VM_DEPLOYMENT_GUIDE.md) | 🇸🇦 [العربية](../i18n/ar/docs/ops/VM_DEPLOYMENT_GUIDE.md) | 🇯🇵 [日本語](../i18n/ja/docs/ops/VM_DEPLOYMENT_GUIDE.md) | 🇻🇳 [Tiếng Việt](../i18n/vi/docs/ops/VM_DEPLOYMENT_GUIDE.md) | 🇧🇬 [Български](../i18n/bg/docs/ops/VM_DEPLOYMENT_GUIDE.md) | 🇩🇰 [Dansk](../i18n/da/docs/ops/VM_DEPLOYMENT_GUIDE.md) | 🇫🇮 [Suomi](../i18n/fi/docs/ops/VM_DEPLOYMENT_GUIDE.md) | 🇮🇱 [עברית](../i18n/he/docs/ops/VM_DEPLOYMENT_GUIDE.md) | 🇭🇺 [Magyar](../i18n/hu/docs/ops/VM_DEPLOYMENT_GUIDE.md) | 🇮🇩 [Bahasa Indonesia](../i18n/id/docs/ops/VM_DEPLOYMENT_GUIDE.md) | 🇰🇷 [한국어](../i18n/ko/docs/ops/VM_DEPLOYMENT_GUIDE.md) | 🇲🇾 [Bahasa Melayu](../i18n/ms/docs/ops/VM_DEPLOYMENT_GUIDE.md) | 🇳🇱 [Nederlands](../i18n/nl/docs/ops/VM_DEPLOYMENT_GUIDE.md) | 🇳🇴 [Norsk](../i18n/no/docs/ops/VM_DEPLOYMENT_GUIDE.md) | 🇵🇹 [Português (Portugal)](../i18n/pt/docs/ops/VM_DEPLOYMENT_GUIDE.md) | 🇷🇴 [Română](../i18n/ro/docs/ops/VM_DEPLOYMENT_GUIDE.md) | 🇵🇱 [Polski](../i18n/pl/docs/ops/VM_DEPLOYMENT_GUIDE.md) | 🇸🇰 [Slovenčina](../i18n/sk/docs/ops/VM_DEPLOYMENT_GUIDE.md) | 🇸🇪 [Svenska](../i18n/sv/docs/ops/VM_DEPLOYMENT_GUIDE.md) | 🇵🇭 [Filipino](../i18n/phi/docs/ops/VM_DEPLOYMENT_GUIDE.md) | 🇨🇿 [Čeština](../i18n/cs/docs/ops/VM_DEPLOYMENT_GUIDE.md)
Complete guide to install and configure OmniRoute on a VM (VPS) with domain managed via Cloudflare.
---
## Prerequisites
| Item | Minimum | Recommended |
| ---------- | ------------------------ | ---------------- |
| **CPU** | 1 vCPU | 2 vCPU |
| **RAM** | 1 GB | 2 GB |
| **Disk** | 10 GB SSD | 25 GB SSD |
| **OS** | Ubuntu 22.04 LTS | Ubuntu 24.04 LTS |
| **Domain** | Registered on Cloudflare | — |
| **Docker** | Docker Engine 24+ | Docker 27+ |
**Tested providers**: Akamai (Linode), DigitalOcean, Vultr, Hetzner, AWS Lightsail.
---
## 1. Configure the VM
### 1.1 Create the instance
On your preferred VPS provider:
- Choose Ubuntu 24.04 LTS
- Select the minimum plan (1 vCPU / 1 GB RAM)
- Set a strong root password or configure SSH key
- Note the **public IP** (e.g., `203.0.113.10`)
### 1.2 Connect via SSH
```bash
ssh root@203.0.113.10
```
### 1.3 Update the system
```bash
apt update && apt upgrade -y
```
### 1.4 Install Docker
```bash
# Install dependencies
apt install -y ca-certificates curl gnupg
# Add official Docker repository
install -m 0755 -d /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | gpg --dearmor -o /etc/apt/keyrings/docker.gpg
chmod a+r /etc/apt/keyrings/docker.gpg
echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu $ (. /etc/os-release && echo "$VERSION_CODENAME") stable" | tee /etc/apt/sources.list.d/docker.list > /dev/null
apt update
apt install -y docker-ce docker-ce-cli containerd.io docker-compose-plugin
```
### 1.5 Install nginx
```bash
apt install -y nginx
```
### 1.6 Configure Firewall (UFW)
```bash
ufw default deny incoming
ufw default allow outgoing
ufw allow 22/tcp # SSH
ufw allow 80/tcp # HTTP (redirect)
ufw allow 443/tcp # HTTPS
ufw enable
```
> **Tip**: For maximum security, restrict ports 80 and 443 to Cloudflare IPs only. See the [Advanced Security](#advanced-security) section.
---
## 2. Install OmniRoute
### 2.1 Create configuration directory
```bash
mkdir -p /opt/omniroute
```
### 2.2 Create environment variables file
```bash
cat > /opt/omniroute/.env << 'EOF'
# === Security ===
JWT_SECRET=CHANGE-TO-A-UNIQUE-64-CHAR-SECRET-KEY
INITIAL_PASSWORD=YourSecurePassword123!
API_KEY_SECRET=REPLACE-WITH-ANOTHER-SECRET-KEY
STORAGE_ENCRYPTION_KEY=REPLACE-WITH-THIRD-SECRET-KEY
STORAGE_ENCRYPTION_KEY_VERSION=v1
MACHINE_ID_SALT=CHANGE-TO-A-UNIQUE-SALT
OMNIROUTE_WS_BRIDGE_SECRET=REPLACE-WITH-WS-BRIDGE-SECRET # REQUIRED em produção: usado pelo Codex Responses WS bridge
# === App ===
PORT=20128
NODE_ENV=production
HOSTNAME=0.0.0.0
DATA_DIR=/app/data
APP_LOG_TO_FILE=true
AUTH_COOKIE_SECURE=false
REQUIRE_API_KEY=false
# === Domain (change to your domain) ===
BASE_URL=https://llms.seudominio.com
NEXT_PUBLIC_BASE_URL=https://llms.seudominio.com
# === Cloud Sync (optional) ===
# CLOUD_URL=https://cloud.omniroute.online
# NEXT_PUBLIC_CLOUD_URL=https://cloud.omniroute.online
EOF
```
> ⚠️ **IMPORTANT**: Generate unique secret keys! Use `openssl rand -hex 32` for each key.
### 2.3 Start the container
```bash
docker pull diegosouzapw/omniroute:latest
docker run -d \
--name omniroute \
--restart unless-stopped \
--env-file /opt/omniroute/.env \
-p 20128:20128 \
-v omniroute-data:/app/data \
diegosouzapw/omniroute:latest
```
### 2.4 Verify that it is running
```bash
docker ps | grep omniroute
docker logs omniroute --tail 20
```
It should display: `[DB] SQLite database ready` and `listening on port 20128`.
---
## 3. Configure nginx (Reverse Proxy)
### 3.1 Generate SSL certificate (Cloudflare Origin)
In the Cloudflare dashboard:
1. Go to **SSL/TLS → Origin Server**
2. Click **Create Certificate**
3. Keep the defaults (15 years, \*.yourdomain.com)
4. Copy the **Origin Certificate** and the **Private Key**
```bash
mkdir -p /etc/nginx/ssl
# Paste the certificate
nano /etc/nginx/ssl/origin.crt
# Paste the private key
nano /etc/nginx/ssl/origin.key
chmod 600 /etc/nginx/ssl/origin.key
```
### 3.2 Nginx Configuration
```bash
cat > /etc/nginx/sites-available/omniroute << 'NGINX'
# Default server — blocks direct access via IP
server {
listen 80 default_server;
listen [::]:80 default_server;
listen 443 ssl default_server;
listen [::]:443 ssl default_server;
ssl_certificate /etc/nginx/ssl/origin.crt;
ssl_certificate_key /etc/nginx/ssl/origin.key;
server_name _;
return 444;
}
# OmniRoute — HTTPS
server {
listen 443 ssl;
listen [::]:443 ssl;
server_name llms.yourdomain.com; # Change to your domain
ssl_certificate /etc/nginx/ssl/origin.crt;
ssl_certificate_key /etc/nginx/ssl/origin.key;
ssl_protocols TLSv1.2 TLSv1.3;
client_max_body_size 100M;
location / {
proxy_pass http://127.0.0.1:20128;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
# WebSocket support
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
# SSE (Server-Sent Events) — streaming AI responses
proxy_buffering off;
proxy_cache off;
proxy_read_timeout 600s;
proxy_send_timeout 600s;
}
}
# HTTP → HTTPS redirect
server {
listen 80;
listen [::]:80;
server_name llms.yourdomain.com;
return 301 https://$server_name$request_uri;
}
NGINX
```
Keep reverse-proxy stream timeouts aligned with your OmniRoute timeout env vars. If you raise
`FETCH_TIMEOUT_MS` / `STREAM_IDLE_TIMEOUT_MS`, raise `proxy_read_timeout` / `proxy_send_timeout`
above the same threshold.
### 3.3 Enable and Test
```bash
# Remove default configuration
rm -f /etc/nginx/sites-enabled/default
# Enable OmniRoute
ln -sf /etc/nginx/sites-available/omniroute /etc/nginx/sites-enabled/omniroute
# Test and reload
nginx -t && systemctl reload nginx
```
---
## 4. Configure Cloudflare DNS
### 4.1 Add DNS record
In the Cloudflare dashboard → DNS:
| Type | Name | Content | Proxy |
| ---- | ------ | ---------------------- | ---------- |
| A | `llms` | `203.0.113.10` (VM IP) | ✅ Proxied |
### 4.2 Configure SSL
Under **SSL/TLS → Overview**:
- Mode: **Full (Strict)**
Under **SSL/TLS → Edge Certificates**:
- Always Use HTTPS: ✅ On
- Minimum TLS Version: TLS 1.2
- Automatic HTTPS Rewrites: ✅ On
### 4.3 Testing
```bash
curl -sI https://llms.seudominio.com/health
# Should return HTTP/2 200
```
---
## 5. Operations and Maintenance
### Upgrade to a new version
```bash
docker pull diegosouzapw/omniroute:latest
docker stop omniroute && docker rm omniroute
docker run -d --name omniroute --restart unless-stopped \
--env-file /opt/omniroute/.env \
-p 20128:20128 \
-v omniroute-data:/app/data \
diegosouzapw/omniroute:latest
```
### View logs
```bash
docker logs -f omniroute # Real-time stream
docker logs omniroute --tail 50 # Last 50 lines
```
### Manual database backup
```bash
# Copy data from the volume to the host
docker cp omniroute:/app/data ./backup-$(date +%F)
# Or compress the entire volume
docker run --rm -v omniroute-data:/data -v $(pwd):/backup \
alpine tar czf /backup/omniroute-data-$(date +%F).tar.gz /data
```
### Restore from backup
```bash
docker stop omniroute
docker run --rm -v omniroute-data:/data -v $(pwd):/backup \
alpine sh -c "rm -rf /data/* && tar xzf /backup/omniroute-data-YYYY-MM-DD.tar.gz -C /"
docker start omniroute
```
---
## 6. Advanced Security
### Restrict nginx to Cloudflare IPs
```bash
cat > /etc/nginx/cloudflare-ips.conf << 'CF'
# Cloudflare IPv4 ranges — update periodically
# https://www.cloudflare.com/ips-v4/
set_real_ip_from 173.245.48.0/20;
set_real_ip_from 103.21.244.0/22;
set_real_ip_from 103.22.200.0/22;
set_real_ip_from 103.31.4.0/22;
set_real_ip_from 141.101.64.0/18;
set_real_ip_from 108.162.192.0/18;
set_real_ip_from 190.93.240.0/20;
set_real_ip_from 188.114.96.0/20;
set_real_ip_from 197.234.240.0/22;
set_real_ip_from 198.41.128.0/17;
set_real_ip_from 162.158.0.0/15;
set_real_ip_from 104.16.0.0/13;
set_real_ip_from 104.24.0.0/14;
set_real_ip_from 172.64.0.0/13;
set_real_ip_from 131.0.72.0/22;
real_ip_header CF-Connecting-IP;
CF
```
Add the following to `nginx.conf` inside the `http {}` block:
```nginx
include /etc/nginx/cloudflare-ips.conf;
```
### Install fail2ban
```bash
apt install -y fail2ban
systemctl enable fail2ban
systemctl start fail2ban
# Check status
fail2ban-client status sshd
```
### Block direct access to the Docker port
```bash
# Prevent direct external access to port 20128
iptables -I DOCKER-USER -p tcp --dport 20128 -j DROP
iptables -I DOCKER-USER -i lo -p tcp --dport 20128 -j ACCEPT
# Persist the rules
apt install -y iptables-persistent
netfilter-persistent save
```
---
## 7. Deploy to Cloudflare Workers (Optional)
For remote access via Cloudflare Workers (without exposing the VM directly):
```bash
# In the local repository
cd omnirouteCloud
npm install
npx wrangler login
npx wrangler deploy
```
See also [TUNNELS_GUIDE.md](./TUNNELS_GUIDE.md) for the in-repo Cloudflare Tunnel walkthrough. The standalone `omnirouteCloud/` worker lives in a separate companion repo.
---
## Port Summary
| Port | Service | Access |
| ----- | ----------- | -------------------------- |
| 22 | SSH | Public (with fail2ban) |
| 80 | nginx HTTP | Redirect → HTTPS |
| 443 | nginx HTTPS | Via Cloudflare Proxy |
| 20128 | OmniRoute | Localhost only (via nginx) |

254
Webhooks.md Normal file

@@ -0,0 +1,254 @@
# Webhooks
> **Source of truth:** `src/lib/webhookDispatcher.ts`, `src/lib/db/webhooks.ts`, `src/app/api/webhooks/`
> **Last updated:** 2026-05-13 — v3.8.0
OmniRoute can fire HTTP webhooks on platform events. Use them to integrate with
Slack, PagerDuty, Datadog, internal alerting services, or any HTTP receiver.
The dispatcher signs each delivery with HMAC-SHA256, retries on transient
failures, tracks delivery health per webhook, and auto-disables endpoints that
keep failing.
## Supported Events
The `WebhookEvent` type (`src/lib/webhookDispatcher.ts`) currently models:
| Event | Fires when |
| -------------------- | --------------------------------------------------------- |
| `request.completed` | A proxied request completes successfully |
| `request.failed` | A proxied request fails after all retries/fallback |
| `provider.error` | A provider returns an error eligible for circuit-breaking |
| `provider.recovered` | A previously failing provider returns to a healthy state |
| `quota.exceeded` | An API key crosses a budget/quota threshold |
| `combo.switched` | A combo strategy switches its primary target |
| `test.ping` | Synthetic event used by the test endpoint |
Subscriptions accept the literal `"*"` to receive every event. Unknown event
names in `events` are ignored at dispatch time.
> Note: the dispatcher API is wired, but production call sites for some of the
> non-`test.ping` events are still landing. Check `grep dispatchEvent` to see
> which paths currently invoke the dispatcher in your release.
## Architecture
```
Caller (handler, service, monitor)
dispatchEvent(event, data) [src/lib/webhookDispatcher.ts]
-> getEnabledWebhooks() [src/lib/db/webhooks.ts]
-> filter by webhook.events
-> for each match (in parallel):
deliverWebhook(url, payload, secret)
build payload { event, timestamp, data }
sign body with HMAC-SHA256 (if secret present)
POST with 10s timeout
retry up to 3 times on 5xx / network error
recordWebhookDelivery(id, status, success)
-> disableWebhooksWithHighFailures(10)
```
Dispatch is fire-and-forget for the caller: `Promise.allSettled` swallows
per-webhook errors so one bad receiver cannot block the others.
## HMAC Signing
When a webhook has a `secret`, OmniRoute signs the JSON body and sends:
```
Content-Type: application/json
User-Agent: OmniRoute-Webhook/1.0
X-Webhook-Event: <event>
X-Webhook-Timestamp: <ISO-8601>
X-Webhook-Signature: sha256=<hex HMAC-SHA256(secret, body)>
```
> Header names use the `X-Webhook-*` prefix (not `X-OmniRoute-*`). The signature
> value is `sha256=<hex>` — verify the full prefix.
If `createWebhook` is called without a secret, the DB module generates one
(`whsec_<48 hex>`) so all webhooks are signed by default.
### Verifying on the receiver
```typescript
import { createHmac, timingSafeEqual } from "node:crypto";
function verify(rawBody: string, signature: string, secret: string) {
const expected = "sha256=" + createHmac("sha256", secret).update(rawBody).digest("hex");
const a = Buffer.from(expected);
const b = Buffer.from(signature);
return a.length === b.length && timingSafeEqual(a, b);
}
```
Always verify against the **raw** request body, before any JSON parsing.
## Retry & Failure Policy
`deliverWebhook(url, payload, secret, maxRetries = 3)`:
- 10 second timeout per attempt (`AbortController`).
- HTTP 2xx counts as success.
- HTTP 3xx/4xx counts as a non-retryable final status — recorded as delivered
with `success = res.ok`.
- HTTP 5xx and network errors are retried with exponential backoff:
`2^attempt * 1000 ms` (1s, 2s, 4s).
- After `maxRetries`, the delivery is recorded as failed.
- Each delivery updates `last_triggered_at`, `last_status`, and either resets
or increments `failure_count`.
- The dispatcher calls `disableWebhooksWithHighFailures(10)` after each fan-out,
so any webhook with `failure_count >= 10` is automatically disabled.
## Database
Table `webhooks` (migration `011_webhooks.sql`):
| Column | Type | Notes |
| ------------------- | ------- | --------------------------------------------- |
| `id` | TEXT PK | UUID |
| `url` | TEXT | Destination URL |
| `events` | TEXT | JSON array; default `["*"]` |
| `secret` | TEXT | HMAC secret (auto-generated if not given) |
| `enabled` | INT | 0/1; defaults to 1 |
| `description` | TEXT | Optional human label |
| `created_at` | TEXT | `datetime('now')` |
| `last_triggered_at` | TEXT | Updated on every delivery attempt |
| `last_status` | INT | HTTP status of the last attempt (0 = network) |
| `failure_count` | INT | Resets to 0 on success, +1 on failure |
There is **no separate `webhook_deliveries` table** in the current schema —
delivery history is aggregated on the `webhooks` row. If you need full audit
history, consume `request.completed` / `audit` style events from a downstream
log store.
## REST API
All endpoints require management auth (`requireManagementAuth`).
| Endpoint | Method | Description |
| ------------------------- | ------ | ------------------------------- |
| `/api/webhooks` | GET | List webhooks (secrets masked) |
| `/api/webhooks` | POST | Create webhook |
| `/api/webhooks/[id]` | GET | Webhook detail (full secret) |
| `/api/webhooks/[id]` | PUT | Update fields |
| `/api/webhooks/[id]` | DELETE | Remove |
| `/api/webhooks/[id]/test` | POST | Fire a `test.ping` (no retries) |
`GET /api/webhooks` masks the secret to `<first 10 chars>...` to avoid leaking
on listing pages. Use the `[id]` GET when you actually need the secret.
### Create webhook
```bash
curl -X POST http://localhost:20128/api/webhooks \
-H "Cookie: auth_token=..." \
-H "Content-Type: application/json" \
-d '{
"url": "https://hooks.slack.com/services/...",
"secret": "whsec_my_shared_secret",
"events": ["quota.exceeded", "provider.error"],
"description": "Slack alerts"
}'
```
If `secret` is omitted, the server generates a `whsec_<hex>` secret and returns
it in the response.
### Test webhook
```bash
curl -X POST http://localhost:20128/api/webhooks/<id>/test \
-H "Cookie: auth_token=..."
```
Returns `{ delivered, status, error }`. No retries are attempted — useful for
quickly validating that the receiver accepts the payload and signature.
## Dashboard
The dashboard page at `/dashboard/webhooks` (see
`src/app/(dashboard)/dashboard/webhooks/page.tsx`) provides:
- Create/edit webhooks with an event picker
- Status indicator (active / inactive / errored) based on `enabled`,
`failure_count`, and `last_status`
- One-click test delivery
- Manual enable/disable toggle
## Payload Examples
### request.completed
```json
{
"event": "request.completed",
"timestamp": "2026-05-13T20:30:00.123Z",
"data": {
"trace_id": "...",
"api_key_id": "...",
"provider": "openai",
"model": "gpt-5",
"status": 200,
"tokens_in": 142,
"tokens_out": 350,
"cost_usd": 0.0042
}
}
```
### provider.error
```json
{
"event": "provider.error",
"timestamp": "2026-05-13T20:31:00.000Z",
"data": {
"provider": "anthropic",
"status": 503,
"consecutive_failures": 5,
"circuit_state": "open"
}
}
```
### test.ping
```json
{
"event": "test.ping",
"timestamp": "2026-05-13T20:32:00.000Z",
"data": {
"message": "Test webhook delivery from OmniRoute",
"webhookId": "<uuid>"
}
}
```
Field shapes for non-`test.ping` events are defined by the call sites that emit
them; treat the `data` object as forward-compatible (add fields, don't depend on
absence).
## Best Practices
- **Verify the signature on every delivery** against the raw body — prevents
spoofed POSTs from anyone who guesses your webhook URL.
- **Respond 2xx within ~5 seconds** — the dispatcher times out at 10 s. Slow
receivers will eat retries and inflate `failure_count`.
- **Make handlers idempotent** — retries and at-least-once delivery semantics
mean duplicates are possible.
- **Subscribe minimally** — list only events you actually consume; `"*"` will
add cost on receivers you do not control.
- **Watch `failure_count`** — endpoints are auto-disabled at 10 consecutive
failures; reset by calling `PUT /api/webhooks/[id]` with `enabled: true`
after fixing the receiver.
- **Rotate secrets periodically** — `PUT` a new `secret`, deploy the new value
to the receiver, and confirm via the test endpoint.
## See Also
- [API_REFERENCE.md](../reference/API_REFERENCE.md) — full management API surface
- [RESILIENCE_GUIDE.md](../architecture/RESILIENCE_GUIDE.md) — circuit breaker / cooldown
semantics that drive `provider.error` / `provider.recovered`
- Source: `src/lib/webhookDispatcher.ts`, `src/lib/db/webhooks.ts`

3
_Footer.md

@@ -1 +1,2 @@
_Footer
---
**[OmniRoute](https://github.com/diegosouzapw/OmniRoute)** · [Website](https://omniroute.online) · [npm](https://www.npmjs.com/package/omniroute) · [Docker Hub](https://hub.docker.com/r/diegosouzapw/omniroute)

78
_Sidebar.md Normal file

@@ -0,0 +1,78 @@
### [🏠 Home](Home)
---
### 🚀 Getting Started
- [Setup Guide](Setup-Guide)
- [User Guide](User-Guide)
- [Features](Features)
- [Quick Start (Docker)](Docker-Guide)
- [Electron Desktop App](Electron-Guide)
- [Termux (Android)](Termux-Guide)
- [PWA Guide](PWA-Guide)
### 🌐 Providers
- [Provider Reference](Provider-Reference)
- [All Providers](Providers)
- [Free Tiers](Free-Tiers)
- [Kiro Setup](Kiro-Setup)
### 🎯 Routing & Combos
- [Auto-Combo](Auto-Combo)
- [Reasoning Replay](Reasoning-Replay)
### 🗜️ Compression
- [Compression Guide](Compression-Guide)
- [Compression Engines](Compression-Engines)
- [RTK Compression](RTK-Compression)
- [Language Packs](Compression-Language-Packs)
- [Rules Format](Compression-Rules-Format)
### 🔌 Integrations
- [MCP Server](MCP-Server)
- [A2A Server](A2A-Server)
- [Agent Protocols](Agent-Protocols-Guide)
- [OpenCode Plugin](OpenCode)
- [Webhooks](Webhooks)
- [Cloud Agents](Cloud-Agent)
- [Skills](Skills)
- [Memory](Memory)
- [Evals](Evals)
- [Gamification](Gamification)
### 🏗️ Architecture
- [Architecture Overview](Architecture)
- [Codebase Documentation](Codebase-Documentation)
- [Repository Map](Repository-Map)
- [Authorization Guide](Authz-Guide)
- [Resilience Guide](Resilience-Guide)
### 🔒 Security
- [Guardrails](Guardrails)
- [Compliance](Compliance)
- [Error Sanitization](Error-Sanitization)
- [Public Credentials](Public-Creds)
- [Route Guard Tiers](Route-Guard-Tiers)
- [Stealth Guide](Stealth-Guide)
- [CLI Token Auth](CLI-Token-Auth)
### 📋 Reference
- [API Reference](API-Reference)
- [CLI Tools](CLI-Tools)
- [Environment Variables](Environment)
### 🚀 Operations
- [VM Deployment](VM-Deployment-Guide)
- [Fly.io Deployment](Fly-io-Deployment-Guide)
- [Tunnels Guide](Tunnels-Guide)
- [Proxy Guide](Proxy-Guide)
- [SQLite Runtime](SQLite-Runtime)
- [Coverage Plan](Coverage-Plan)
- [Release Checklist](Release-Checklist)
### ℹ️ More
- [Troubleshooting](Troubleshooting)
- [i18n / Translations](I18N)
- [Uninstall](Uninstall)
- [Contributing](Contributing)
- [Comparison vs Alternatives](OmniRoute-vs-Alternatives)