mirror of
https://github.com/diegosouzapw/OmniRoute.git
synced 2026-08-05 23:02:10 +03:00
docs(guides): add dedicated setup, docker, compression, and resilience manuals
Break out long-form README content into focused documentation pages for core onboarding and operations topics. Refresh the README navigation to point readers to the new guides while keeping high-level product sections easier to scan.
This commit is contained in:
183
docs/COMPRESSION_GUIDE.md
Normal file
183
docs/COMPRESSION_GUIDE.md
Normal file
@@ -0,0 +1,183 @@
|
||||
# 🗜️ Prompt Compression Guide — OmniRoute
|
||||
|
||||
> Save 15-75% on token costs automatically. For a quick overview, see the [README Compression section](../README.md#%EF%B8%8F-prompt-compression--save-15-75-tokens-automatically).
|
||||
|
||||
## Overview
|
||||
|
||||
OmniRoute implements a modular prompt compression pipeline that runs **proactively** before requests hit upstream providers. This means your token savings happen transparently — no changes needed to your workflow.
|
||||
|
||||
```
|
||||
Client Request
|
||||
→ Compression Strategy Selector
|
||||
→ Combo override? → Use combo setting
|
||||
→ Auto-trigger threshold? → Use auto mode
|
||||
→ Default mode? → Use global setting
|
||||
→ Off? → Skip compression
|
||||
→ Selected Compression Mode
|
||||
→ Off: No compression
|
||||
→ Lite: Safe whitespace/formatting cleanup (~15%)
|
||||
→ Standard: Caveman-speak filler removal (~30%)
|
||||
→ Aggressive: History aging + summarization (~50%)
|
||||
→ Ultra: Heuristic pruning + code-block thinning (~75%)
|
||||
→ Compressed Request → Provider
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Compression Modes
|
||||
|
||||
### Off
|
||||
|
||||
No compression applied. All messages pass through unchanged.
|
||||
|
||||
### Lite Mode (~15% savings, <1ms latency)
|
||||
|
||||
The safest mode — zero semantic change, only formatting cleanup:
|
||||
|
||||
| Technique | Description |
|
||||
| ------------------------ | ------------------------------------------------- |
|
||||
| `collapseWhitespace` | Merge consecutive blank lines and trailing spaces |
|
||||
| `dedupSystemPrompt` | Remove duplicate system messages |
|
||||
| `compressToolResults` | Compress verbose tool/function outputs |
|
||||
| `removeRedundantContent` | Strip repeated instructions |
|
||||
| `replaceImageUrls` | Shorten base64 image data URIs |
|
||||
|
||||
**Best for:** Always-on usage, safety-critical workflows.
|
||||
|
||||
### Standard Mode (~30% savings)
|
||||
|
||||
Inspired by [Caveman](https://github.com/JuliusBrussee/caveman) — removes filler words and verbose phrasing while preserving meaning:
|
||||
|
||||
- Removes filler words ("please", "I think", "basically", "actually")
|
||||
- Condenses verbose phrases ("in order to" → "to", "as a result of" → "because")
|
||||
- Strips polite hedging ("Would you mind...", "If you could possibly...")
|
||||
- 30+ regex rules tuned for coding prompts
|
||||
|
||||
**Best for:** Daily coding workflows, cost-conscious teams.
|
||||
|
||||
### Aggressive Mode (~50% savings)
|
||||
|
||||
Smart history management for long sessions:
|
||||
|
||||
- **Message Aging** — older messages get progressively compressed
|
||||
- **Tool Result Summarization** — long tool outputs replaced with summaries
|
||||
- **Structural Integrity Guards** — ensures `tool_use` + `tool_result` pairs stay consistent
|
||||
- **Context Window Awareness** — respects per-model token limits
|
||||
|
||||
**Best for:** Extended debugging sessions, large codebases.
|
||||
|
||||
### Ultra Mode (~75% savings)
|
||||
|
||||
Maximum compression for token-critical scenarios:
|
||||
|
||||
- **Heuristic Pruning** — removes messages below relevance threshold
|
||||
- **Code Block Thinning** — compresses repetitive code examples
|
||||
- **Binary Search Truncation** — finds optimal cut point for context window
|
||||
- All Aggressive mode features included
|
||||
|
||||
**Best for:** When you're hitting context limits repeatedly.
|
||||
|
||||
---
|
||||
|
||||
## Token Savings Visualization
|
||||
|
||||
```
|
||||
Without compression: 47K tokens sent to LLM
|
||||
With Lite: 40K tokens sent (15% saved — safe, always-on)
|
||||
With Standard: 33K tokens sent (30% saved — caveman-speak rules)
|
||||
With Aggressive: 24K tokens sent (50% saved — aging + summarization)
|
||||
With Ultra: 12K tokens sent (75% saved — heuristic pruning)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Configuration
|
||||
|
||||
### Dashboard
|
||||
|
||||
Navigate to `Dashboard → Settings → Compression`:
|
||||
|
||||
- **Default Mode** — sets the system-wide compression mode
|
||||
- **Auto-Trigger Threshold** — automatically engage compression when token count exceeds threshold
|
||||
- **Per-Combo Override** — each combo can have its own compression mode
|
||||
|
||||
### Per-Combo Override
|
||||
|
||||
In `Dashboard → Combos → [Your Combo] → Advanced`, set compression mode per combo:
|
||||
|
||||
```txt
|
||||
Combo: "free-forever"
|
||||
Mode: Standard
|
||||
Targets:
|
||||
1. gc/gemini-3-flash
|
||||
2. if/kimi-k2-thinking
|
||||
```
|
||||
|
||||
This lets you use aggressive compression on free providers while keeping lite mode on paid subscriptions.
|
||||
|
||||
### API
|
||||
|
||||
```bash
|
||||
# Get compression settings
|
||||
curl http://localhost:20128/api/settings/compression
|
||||
|
||||
# Update compression settings
|
||||
curl -X PUT http://localhost:20128/api/settings/compression \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"defaultMode":"lite","autoTriggerThreshold":32000}'
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## What Gets Protected
|
||||
|
||||
The compression engine **always preserves:**
|
||||
|
||||
- ✅ Code blocks (fenced and inline)
|
||||
- ✅ URLs and file paths
|
||||
- ✅ JSON structures and structured data
|
||||
- ✅ API keys, tokens, and identifiers
|
||||
- ✅ Mathematical expressions
|
||||
- ✅ Tool/function call definitions
|
||||
- ✅ System prompts (in lite mode)
|
||||
|
||||
---
|
||||
|
||||
## Compression Stats
|
||||
|
||||
Every compressed request includes stats in the server logs:
|
||||
|
||||
```json
|
||||
{
|
||||
"originalTokens": 47200,
|
||||
"compressedTokens": 40120,
|
||||
"savingsPercent": 15.0,
|
||||
"techniquesUsed": ["collapseWhitespace", "dedupSystemPrompt"],
|
||||
"mode": "lite",
|
||||
"latencyMs": 0.8
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Phase Roadmap
|
||||
|
||||
| Phase | Modes | Status |
|
||||
| ------- | ------------------------------------ | ---------- |
|
||||
| Phase 1 | Off, Lite | ✅ Shipped |
|
||||
| Phase 2 | Standard, Aggressive, Ultra | ✅ Shipped |
|
||||
| Phase 3 | Per-model adaptive, ML-based pruning | 🗓️ Planned |
|
||||
|
||||
---
|
||||
|
||||
## Acknowledgments
|
||||
|
||||
Standard mode compression rules are inspired by **[Caveman](https://github.com/JuliusBrussee/caveman)** by **[JuliusBrussee](https://github.com/JuliusBrussee)** (⭐ 51K+) — the viral "why use many token when few token do trick" project.
|
||||
|
||||
---
|
||||
|
||||
## See Also
|
||||
|
||||
- [Environment Config](ENVIRONMENT.md) — Compression environment variables
|
||||
- [Architecture Guide](ARCHITECTURE.md) — Compression pipeline internals
|
||||
- [User Guide](USER_GUIDE.md) — Getting started with compression
|
||||
119
docs/DOCKER_GUIDE.md
Normal file
119
docs/DOCKER_GUIDE.md
Normal file
@@ -0,0 +1,119 @@
|
||||
# 🐳 Docker Guide — OmniRoute
|
||||
|
||||
> Complete Docker deployment reference. For a quick start, see the [README Docker section](../README.md#-docker).
|
||||
|
||||
## Table of Contents
|
||||
|
||||
- [Quick Run](#quick-run)
|
||||
- [With Environment File](#with-environment-file)
|
||||
- [Docker Compose](#docker-compose)
|
||||
- [Docker Compose with Caddy (HTTPS)](#docker-compose-with-caddy-https-auto-tls)
|
||||
- [Cloudflare Quick Tunnel](#cloudflare-quick-tunnel)
|
||||
- [Image Tags](#image-tags)
|
||||
- [Important Notes](#important-notes)
|
||||
|
||||
---
|
||||
|
||||
## Quick Run
|
||||
|
||||
```bash
|
||||
docker run -d \
|
||||
--name omniroute \
|
||||
--restart unless-stopped \
|
||||
--stop-timeout 40 \
|
||||
-p 20128:20128 \
|
||||
-v omniroute-data:/app/data \
|
||||
diegosouzapw/omniroute:latest
|
||||
```
|
||||
|
||||
## With Environment File
|
||||
|
||||
```bash
|
||||
# Copy and edit .env first
|
||||
cp .env.example .env
|
||||
|
||||
docker run -d \
|
||||
--name omniroute \
|
||||
--restart unless-stopped \
|
||||
--stop-timeout 40 \
|
||||
--env-file .env \
|
||||
-p 20128:20128 \
|
||||
-v omniroute-data:/app/data \
|
||||
diegosouzapw/omniroute:latest
|
||||
```
|
||||
|
||||
## Docker Compose
|
||||
|
||||
```bash
|
||||
# Base profile (no CLI tools)
|
||||
docker compose --profile base up -d
|
||||
|
||||
# CLI profile (Claude Code, Codex, OpenClaw built-in)
|
||||
docker compose --profile cli up -d
|
||||
```
|
||||
|
||||
## Docker Compose with Caddy (HTTPS Auto-TLS)
|
||||
|
||||
OmniRoute can be securely exposed using Caddy's automatic SSL provisioning. Ensure your domain's DNS A record points to your server's IP.
|
||||
|
||||
```yaml
|
||||
services:
|
||||
omniroute:
|
||||
image: diegosouzapw/omniroute:latest
|
||||
container_name: omniroute
|
||||
restart: unless-stopped
|
||||
volumes:
|
||||
- omniroute-data:/app/data
|
||||
environment:
|
||||
- PORT=20128
|
||||
- NEXT_PUBLIC_BASE_URL=https://your-domain.com
|
||||
|
||||
caddy:
|
||||
image: caddy:latest
|
||||
container_name: caddy
|
||||
restart: unless-stopped
|
||||
ports:
|
||||
- "80:80"
|
||||
- "443:443"
|
||||
command: caddy reverse-proxy --from https://your-domain.com --to http://omniroute:20128
|
||||
|
||||
volumes:
|
||||
omniroute-data:
|
||||
```
|
||||
|
||||
## Cloudflare Quick Tunnel
|
||||
|
||||
Dashboard support for Docker deployments includes a one-click **Cloudflare Quick Tunnel** on `Dashboard → Endpoints`. The first enable downloads `cloudflared` only when needed, starts a temporary tunnel to your current `/v1` endpoint, and shows the generated `https://*.trycloudflare.com/v1` URL directly below your normal public URL.
|
||||
|
||||
Endpoint tunnel panels (Cloudflare, Tailscale, ngrok) can be shown or hidden from `Settings → Appearance` without changing active tunnel state.
|
||||
|
||||
### Tunnel Notes
|
||||
|
||||
- Quick Tunnel URLs are temporary and change after every restart.
|
||||
- Quick Tunnels are not auto-restored after an OmniRoute or container restart. Re-enable them from the dashboard when needed.
|
||||
- Managed install currently supports Linux, macOS, and Windows on `x64` / `arm64`.
|
||||
- Managed Quick Tunnels default to HTTP/2 transport to avoid noisy QUIC UDP buffer warnings in constrained container environments. Set `CLOUDFLARED_PROTOCOL=quic` or `auto` if you want a different transport.
|
||||
- Docker images bundle system CA roots and pass them to managed `cloudflared`, which avoids TLS trust failures when the tunnel bootstraps inside the container.
|
||||
- Set `CLOUDFLARED_BIN=/absolute/path/to/cloudflared` if you want OmniRoute to use an existing binary instead of downloading one.
|
||||
|
||||
## Image Tags
|
||||
|
||||
| Image | Tag | Size | Description |
|
||||
| ------------------------ | -------- | ------ | --------------------- |
|
||||
| `diegosouzapw/omniroute` | `latest` | ~250MB | Latest stable release |
|
||||
| `diegosouzapw/omniroute` | `3.7.8` | ~250MB | Current version |
|
||||
|
||||
Multi-platform: AMD64 + ARM64 native (Apple Silicon, AWS Graviton, Raspberry Pi).
|
||||
|
||||
## Important Notes
|
||||
|
||||
- **SQLite WAL Mode:** `docker stop` should be allowed to finish so OmniRoute can checkpoint the latest changes back into `storage.sqlite`. The bundled Compose files already set a 40s stop grace period. If you run the image directly, keep `--stop-timeout 40`.
|
||||
- **`DISABLE_SQLITE_AUTO_BACKUP`:** Set to `true` if backups are managed externally.
|
||||
- **Data Persistence:** Always mount a volume to `/app/data` to persist your database, keys, and configurations across container restarts.
|
||||
- **Port Configuration:** Override `PORT` environment variable to change the default `20128` port.
|
||||
|
||||
## See Also
|
||||
|
||||
- [VM Deployment Guide](VM_DEPLOYMENT_GUIDE.md) — VM + nginx + Cloudflare setup
|
||||
- [Fly.io Deployment Guide](FLY_IO_DEPLOYMENT_GUIDE.md) — Deploy to Fly.io
|
||||
- [Environment Config](ENVIRONMENT.md) — Complete `.env` reference
|
||||
145
docs/RESILIENCE_GUIDE.md
Normal file
145
docs/RESILIENCE_GUIDE.md
Normal file
@@ -0,0 +1,145 @@
|
||||
# 🛡️ Resilience Guide — OmniRoute
|
||||
|
||||
> How OmniRoute keeps your AI coding workflow running when providers fail.
|
||||
|
||||
## Overview
|
||||
|
||||
OmniRoute implements a multi-layered resilience system that ensures zero downtime:
|
||||
|
||||
```
|
||||
Client Request
|
||||
→ Rate Limit Check (per-IP, per-connection)
|
||||
→ Combo Routing (13 strategies)
|
||||
→ Connection Selection (P2C, round-robin, etc.)
|
||||
→ Request Queue & Pacing
|
||||
→ Execute (provider-specific executor)
|
||||
→ On Failure:
|
||||
→ Connection Cooldown (exponential backoff)
|
||||
→ Circuit Breaker (provider-level)
|
||||
→ Wait For Cooldown (auto-retry)
|
||||
→ Next Combo Target (fallback chain)
|
||||
→ Response
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Request Queue & Pacing
|
||||
|
||||
Per-connection request buckets smooth bursts before they hit upstream rate caps.
|
||||
|
||||
Configure in `Dashboard → Settings → Resilience`:
|
||||
|
||||
| Setting | Default | Description |
|
||||
| --------------- | ------- | ------------------------------------ |
|
||||
| Queue Size | `10` | Max queued requests per connection |
|
||||
| Pacing Interval | `0ms` | Minimum gap between requests |
|
||||
| Max Concurrent | `5` | Simultaneous requests per connection |
|
||||
|
||||
---
|
||||
|
||||
## Connection Cooldown
|
||||
|
||||
A single connection cools down after retryable failures. Features:
|
||||
|
||||
- **Exponential Backoff** — progressively longer cooldowns after each failure
|
||||
- **`Retry-After` Header Support** — respects upstream hints
|
||||
- **Configurable Base/Max** — tune cooldown duration per use case
|
||||
- **Auto-Recovery** — connection automatically becomes available after cooldown expires
|
||||
|
||||
---
|
||||
|
||||
## Circuit Breaker
|
||||
|
||||
Provider-level protection against cascading failures:
|
||||
|
||||
1. **Connection-scoped `429` rate limits** stay in Connection Cooldown (don't trip the breaker)
|
||||
2. **Provider-wide transient errors** (5xx, network timeouts) increment the failure counter
|
||||
3. **Breaker trips** only after fallback is exhausted AND the provider still fails
|
||||
4. **Recovery** — breaker automatically moves to half-open state after timeout, tests with probe request
|
||||
|
||||
Configure thresholds in `Dashboard → Settings → Resilience`.
|
||||
|
||||
---
|
||||
|
||||
## Wait For Cooldown
|
||||
|
||||
Instead of immediately failing when all connections are in cooldown, OmniRoute can wait for the earliest connection to expire and retry:
|
||||
|
||||
- **Automatic** — server waits for the earliest cooldown to expire
|
||||
- **Transparent** — client sees a slightly delayed response instead of an error
|
||||
- **Configurable** — enable/disable per combo or globally
|
||||
|
||||
---
|
||||
|
||||
## Anti-Thundering Herd
|
||||
|
||||
When multiple concurrent requests hit a failing provider simultaneously:
|
||||
|
||||
- **Mutex Protection** — only one retry attempt at a time per connection
|
||||
- **Semaphore** — limits concurrent retry storms across connections
|
||||
- **Deduplication** — identical requests within 5s window are deduplicated
|
||||
|
||||
---
|
||||
|
||||
## Combo Fallback Chains
|
||||
|
||||
The primary resilience mechanism. Configure in `Dashboard → Combos`:
|
||||
|
||||
```txt
|
||||
Combo: "always-on"
|
||||
1. cc/claude-opus-4-7 ← Primary (subscription)
|
||||
2. cx/gpt-5.2-codex ← Secondary (subscription)
|
||||
3. glm/glm-4.7 ← Cheap backup ($0.5/1M)
|
||||
4. if/kimi-k2-thinking ← Free fallback (unlimited)
|
||||
```
|
||||
|
||||
When provider #1 fails (quota, rate, or health), OmniRoute automatically routes to #2, then #3, then #4 — with zero manual intervention.
|
||||
|
||||
### 13 Routing Strategies
|
||||
|
||||
| Strategy | Description |
|
||||
| ------------------- | ---------------------------------- |
|
||||
| `priority` | First available in order |
|
||||
| `weighted` | Weighted distribution |
|
||||
| `fill-first` | Fill primary before moving |
|
||||
| `round-robin` | Rotate through all targets |
|
||||
| `p2c` | Power-of-two choices (quota-aware) |
|
||||
| `random` | Random selection |
|
||||
| `least-used` | Least recently used |
|
||||
| `cost-optimized` | Cheapest available |
|
||||
| `strict-random` | True random (no tracking) |
|
||||
| `auto` | OmniRoute selects based on context |
|
||||
| `lkgp` | Last Known Good Provider |
|
||||
| `context-optimized` | Best for current context window |
|
||||
| `context-relay` | Session handoff during rotation |
|
||||
|
||||
---
|
||||
|
||||
## TLS Fingerprint Spoofing
|
||||
|
||||
OmniRoute makes proxied traffic look like legitimate browser/CLI requests:
|
||||
|
||||
- **Browser-like TLS** via `wreq-js` — prevents bot detection
|
||||
- **CLI Fingerprint Matching** — reorders headers and body fields to match native CLI binary signatures (Claude Code, Codex, etc.)
|
||||
- **Proxy IP Preservation** — stealth features work on top of proxy IP masking
|
||||
|
||||
---
|
||||
|
||||
## Health Dashboard
|
||||
|
||||
Monitor all resilience components in real-time at `Dashboard → Health`:
|
||||
|
||||
- **Uptime** — server uptime and last restart
|
||||
- **Provider Breaker States** — open/closed/half-open per provider
|
||||
- **Connection Cooldowns** — active cooldowns with expiry times
|
||||
- **Cache Stats** — signature + semantic cache hit rates
|
||||
- **Lockouts** — API key lockouts and IP bans
|
||||
- **Latency** — p50/p95/p99 percentiles
|
||||
|
||||
---
|
||||
|
||||
## See Also
|
||||
|
||||
- [Architecture Guide](ARCHITECTURE.md) — System architecture and internals
|
||||
- [User Guide](USER_GUIDE.md) — Providers, combos, CLI integration
|
||||
- [Auto-Combo Engine](AUTO-COMBO.md) — 6-factor scoring, mode packs
|
||||
311
docs/SETUP_GUIDE.md
Normal file
311
docs/SETUP_GUIDE.md
Normal file
@@ -0,0 +1,311 @@
|
||||
# 📖 Setup Guide — OmniRoute
|
||||
|
||||
> Complete setup reference for OmniRoute. For the quick version, see the [Quick Start in README](../README.md#-quick-start).
|
||||
|
||||
## Table of Contents
|
||||
|
||||
- [Install Methods](#install-methods)
|
||||
- [CLI Tool Configuration](#cli-tool-configuration)
|
||||
- [Protocol Setup (MCP + A2A)](#protocol-setup-mcp--a2a)
|
||||
- [Timeout Configuration](#timeout-configuration)
|
||||
- [Split-Port Mode](#split-port-mode)
|
||||
- [Void Linux (xbps-src)](#void-linux-xbps-src-template)
|
||||
- [Uninstalling](#uninstalling)
|
||||
|
||||
---
|
||||
|
||||
## Install Methods
|
||||
|
||||
### npm (recommended)
|
||||
|
||||
```bash
|
||||
npm install -g omniroute
|
||||
omniroute
|
||||
```
|
||||
|
||||
Dashboard opens at `http://localhost:20128` and API base URL is `http://localhost:20128/v1`.
|
||||
|
||||
### pnpm
|
||||
|
||||
```bash
|
||||
pnpm install -g omniroute
|
||||
pnpm approve-builds -g # Select all packages → approve
|
||||
omniroute
|
||||
```
|
||||
|
||||
> **pnpm users:** `pnpm approve-builds -g` is required to enable native build scripts for `better-sqlite3` and `@swc/core`.
|
||||
|
||||
### Arch Linux (AUR)
|
||||
|
||||
```bash
|
||||
yay -S omniroute-bin
|
||||
systemctl --user enable --now omniroute.service
|
||||
```
|
||||
|
||||
The [AUR package](https://aur.archlinux.org/packages/omniroute-bin) installs OmniRoute and provides a systemd user service.
|
||||
|
||||
### From Source
|
||||
|
||||
```bash
|
||||
cp .env.example .env
|
||||
npm install
|
||||
PORT=20128 DASHBOARD_PORT=20129 NEXT_PUBLIC_BASE_URL=http://localhost:20129 npm run dev
|
||||
```
|
||||
|
||||
### Docker
|
||||
|
||||
See the [Docker Guide](DOCKER_GUIDE.md) for complete Docker setup including Compose profiles and Caddy HTTPS.
|
||||
|
||||
### CLI Options
|
||||
|
||||
| Command | Description |
|
||||
| ----------------------- | ----------------------------------------------------------- |
|
||||
| `omniroute` | Start server (`PORT=20128`, API and dashboard on same port) |
|
||||
| `omniroute --port 3000` | Set canonical/API port to 3000 |
|
||||
| `omniroute --mcp` | Start MCP server (stdio transport) |
|
||||
| `omniroute --no-open` | Don't auto-open browser |
|
||||
| `omniroute --help` | Show help |
|
||||
|
||||
---
|
||||
|
||||
## CLI Tool Configuration
|
||||
|
||||
### 1) Connect Providers and Create API Key
|
||||
|
||||
1. Open Dashboard → `Providers` and connect at least one provider (OAuth or API key).
|
||||
2. Open Dashboard → `Endpoints` and create an API key.
|
||||
3. (Optional) Open Dashboard → `Combos` and set your fallback chain.
|
||||
|
||||
### 2) Point Your Coding Tool
|
||||
|
||||
```txt
|
||||
Base URL: http://localhost:20128/v1
|
||||
API Key: [copy from Endpoint page]
|
||||
Model: if/kimi-k2-thinking (or any provider/model prefix)
|
||||
```
|
||||
|
||||
Works with Claude Code, Codex CLI, Gemini CLI, Cursor, Cline, OpenClaw, OpenCode, and OpenAI-compatible SDKs.
|
||||
|
||||
For detailed per-tool configuration (Claude Code, Codex CLI, Cursor, Cline, OpenClaw, Kilo Code, Copilot, and more), see the dedicated **[CLI Tools Guide](CLI-TOOLS.md)**.
|
||||
|
||||
---
|
||||
|
||||
## Protocol Setup (MCP + A2A)
|
||||
|
||||
### MCP Setup (Model Context Protocol)
|
||||
|
||||
Start MCP transport in stdio mode:
|
||||
|
||||
```bash
|
||||
omniroute --mcp
|
||||
```
|
||||
|
||||
Recommended validation flow:
|
||||
|
||||
```bash
|
||||
# 1. Start MCP server
|
||||
omniroute --mcp
|
||||
|
||||
# 2. From your MCP client, call:
|
||||
omniroute_get_health # Should return system health
|
||||
omniroute_list_combos # Should return active combos
|
||||
|
||||
# 3. Or run the full E2E suite:
|
||||
npm run test:protocols:e2e
|
||||
```
|
||||
|
||||
#### MCP Client Configuration
|
||||
|
||||
**Claude Code:**
|
||||
|
||||
```bash
|
||||
claude mcp add-server omniroute --type http --url http://localhost:20128/api/mcp/stream
|
||||
```
|
||||
|
||||
**Cursor / Cline:**
|
||||
|
||||
Add to your MCP settings:
|
||||
|
||||
```json
|
||||
{
|
||||
"mcpServers": {
|
||||
"omniroute": {
|
||||
"command": "omniroute",
|
||||
"args": ["--mcp"],
|
||||
"env": {}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Full MCP documentation:** [MCP Server README](../open-sse/mcp-server/README.md) — 29 tools, IDE configs, Python/TS/Go clients.
|
||||
|
||||
### A2A Setup (Agent-to-Agent Protocol)
|
||||
|
||||
Verify the Agent Card:
|
||||
|
||||
```bash
|
||||
curl http://localhost:20128/.well-known/agent.json
|
||||
```
|
||||
|
||||
Send a task:
|
||||
|
||||
```bash
|
||||
curl -X POST http://localhost:20128/a2a \
|
||||
-H 'content-type: application/json' \
|
||||
-d '{"jsonrpc":"2.0","id":"quickstart","method":"message/send","params":{"skill":"quota-management","messages":[{"role":"user","content":"Give me a short quota summary."}]}}'
|
||||
```
|
||||
|
||||
**Full A2A documentation:** [A2A Server README](../src/lib/a2a/README.md) — JSON-RPC 2.0, skills, streaming, task lifecycle.
|
||||
|
||||
---
|
||||
|
||||
## Timeout Configuration
|
||||
|
||||
### Basic Timeouts
|
||||
|
||||
For most deployments, you only need these two variables:
|
||||
|
||||
| Variable | Default | Purpose |
|
||||
| ------------------------ | ----------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `REQUEST_TIMEOUT_MS` | `600000` | Shared baseline for upstream response-start timeout, hidden Undici timeouts, TLS fingerprint requests, and API bridge request/proxy timeouts |
|
||||
| `STREAM_IDLE_TIMEOUT_MS` | inherits `REQUEST_TIMEOUT_MS` | Maximum gap between streaming chunks before OmniRoute aborts the SSE stream |
|
||||
|
||||
Backward compatibility is preserved: existing `FETCH_TIMEOUT_MS`, `API_BRIDGE_PROXY_TIMEOUT_MS`, and other per-layer timeout vars still work and override the shared baseline.
|
||||
|
||||
### Provider-Specific Notes
|
||||
|
||||
For Claude Code-compatible upstreams (`anthropic-compatible-cc-*`), OmniRoute derives the outbound `X-Stainless-Timeout` header from the resolved fetch timeout so provider-side read timeouts stay aligned with your env configuration.
|
||||
|
||||
For third-party Claude Code-compatible reverse proxies, OmniRoute keeps the default `anthropic-beta` set conservative and, when `Client Cache Control` is left on `Auto`, only forwards client-provided `cache_control` markers.
|
||||
|
||||
### Advanced Timeout Overrides
|
||||
|
||||
| Variable | Default | Purpose |
|
||||
| ---------------------------------------- | ------------------------------------------ | -------------------------------------------------------------------- |
|
||||
| `FETCH_TIMEOUT_MS` | inherits `REQUEST_TIMEOUT_MS` | Upstream response-start timeout used until response headers arrive |
|
||||
| `FETCH_HEADERS_TIMEOUT_MS` | inherits `FETCH_TIMEOUT_MS` | Undici time limit for receiving upstream response headers |
|
||||
| `FETCH_BODY_TIMEOUT_MS` | inherits `FETCH_TIMEOUT_MS` | Undici time limit between upstream body chunks (`0` disables it) |
|
||||
| `FETCH_CONNECT_TIMEOUT_MS` | `30000` | Undici TCP connect timeout |
|
||||
| `FETCH_KEEPALIVE_TIMEOUT_MS` | `4000` | Undici idle keep-alive socket timeout |
|
||||
| `TLS_CLIENT_TIMEOUT_MS` | inherits `FETCH_TIMEOUT_MS` | Timeout for TLS fingerprint requests made through `wreq-js` |
|
||||
| `API_BRIDGE_PROXY_TIMEOUT_MS` | inherits `REQUEST_TIMEOUT_MS` or `30000` | Timeout for `/v1` proxy forwarding from API port to dashboard port |
|
||||
| `API_BRIDGE_SERVER_REQUEST_TIMEOUT_MS` | `max(API_BRIDGE_PROXY_TIMEOUT_MS, 300000)` | Incoming request timeout on the API bridge server |
|
||||
| `API_BRIDGE_SERVER_HEADERS_TIMEOUT_MS` | `60000` | Incoming header timeout on the API bridge server |
|
||||
| `API_BRIDGE_SERVER_KEEPALIVE_TIMEOUT_MS` | `5000` | Keep-alive timeout on the API bridge server |
|
||||
| `API_BRIDGE_SERVER_SOCKET_TIMEOUT_MS` | `0` | Socket inactivity timeout on the API bridge server (`0` disables it) |
|
||||
|
||||
> **Note:** For streaming requests, `FETCH_TIMEOUT_MS` only covers connection setup / waiting for the first upstream response. Once the stream is active, OmniRoute will only abort on an actual stall (`STREAM_IDLE_TIMEOUT_MS`) or Undici body inactivity (`FETCH_BODY_TIMEOUT_MS`).
|
||||
|
||||
### Reverse Proxy Compatibility
|
||||
|
||||
If you run OmniRoute behind Nginx, Caddy, Cloudflare, or another reverse proxy, make sure the proxy timeouts are also higher than your OmniRoute stream/fetch timeouts.
|
||||
|
||||
---
|
||||
|
||||
## Split-Port Mode
|
||||
|
||||
Run API and Dashboard on separate ports for advanced scenarios (reverse proxy, container networking):
|
||||
|
||||
```bash
|
||||
PORT=20128 DASHBOARD_PORT=20129 omniroute
|
||||
# API: http://localhost:20128/v1
|
||||
# Dashboard: http://localhost:20129
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Void Linux (xbps-src) Template
|
||||
|
||||
For Void Linux users, you can build a native package using `xbps-src`. Save this block as `srcpkgs/omniroute/template`:
|
||||
|
||||
```bash
|
||||
# Template file for 'omniroute'
|
||||
pkgname=omniroute
|
||||
version=3.4.1
|
||||
revision=1
|
||||
hostmakedepends="nodejs python3 make"
|
||||
depends="openssl"
|
||||
short_desc="Universal AI gateway with smart routing for multiple LLM providers"
|
||||
maintainer="zenobit <zenobit@disroot.org>"
|
||||
license="MIT"
|
||||
homepage="https://github.com/diegosouzapw/OmniRoute"
|
||||
distfiles="https://github.com/diegosouzapw/OmniRoute/archive/refs/tags/v${version}.tar.gz"
|
||||
checksum=009400afee90a9f32599d8fe734145cfd84098140b7287990183dde45ae2245b
|
||||
system_accounts="_omniroute"
|
||||
omniroute_homedir="/var/lib/omniroute"
|
||||
export NODE_ENV=production
|
||||
export npm_config_engine_strict=false
|
||||
export npm_config_loglevel=error
|
||||
export npm_config_fund=false
|
||||
export npm_config_audit=false
|
||||
|
||||
do_build() {
|
||||
local _gyp_arch
|
||||
case "$XBPS_TARGET_MACHINE" in
|
||||
aarch64*) _gyp_arch=arm64 ;;
|
||||
armv7*|armv6*) _gyp_arch=arm ;;
|
||||
i686*) _gyp_arch=ia32 ;;
|
||||
*) _gyp_arch=x64 ;;
|
||||
esac
|
||||
|
||||
NODE_ENV=development npm ci --ignore-scripts
|
||||
npm run build
|
||||
cp -r .next/static .next/standalone/.next/static
|
||||
[ -d public ] && cp -r public .next/standalone/public || true
|
||||
|
||||
local _node_gyp=/usr/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js
|
||||
(cd node_modules/better-sqlite3 && node "$_node_gyp" rebuild --arch="$_gyp_arch")
|
||||
|
||||
local _bs3_release=.next/standalone/node_modules/better-sqlite3/build/Release
|
||||
mkdir -p "$_bs3_release"
|
||||
cp node_modules/better-sqlite3/build/Release/better_sqlite3.node "$_bs3_release/"
|
||||
|
||||
rm -rf .next/standalone/node_modules/@img
|
||||
|
||||
for _mod in pino-abstract-transport split2 process-warning; do
|
||||
cp -r "node_modules/$_mod" .next/standalone/node_modules/
|
||||
done
|
||||
}
|
||||
|
||||
do_check() {
|
||||
npm run test:unit
|
||||
}
|
||||
|
||||
do_install() {
|
||||
vmkdir usr/lib/omniroute/.next
|
||||
vcopy .next/standalone/. usr/lib/omniroute/.next/standalone
|
||||
|
||||
for _d in \
|
||||
.next/standalone/.next/server/app/dashboard \
|
||||
.next/standalone/.next/server/app/dashboard/settings \
|
||||
.next/standalone/.next/server/app/dashboard/providers; do
|
||||
touch "${DESTDIR}/usr/lib/omniroute/${_d}/.keep"
|
||||
done
|
||||
|
||||
cat > "${WRKDIR}/omniroute" <<'EOF'
|
||||
#!/bin/sh
|
||||
export PORT="${PORT:-20128}"
|
||||
export DATA_DIR="${DATA_DIR:-${XDG_DATA_HOME:-${HOME}/.local/share}/omniroute}"
|
||||
export APP_LOG_TO_FILE="${APP_LOG_TO_FILE:-false}"
|
||||
mkdir -p "${DATA_DIR}"
|
||||
exec node /usr/lib/omniroute/.next/standalone/server.js "$@"
|
||||
EOF
|
||||
vbin "${WRKDIR}/omniroute"
|
||||
}
|
||||
|
||||
post_install() {
|
||||
vlicense LICENSE
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Uninstalling
|
||||
|
||||
| Command | Action |
|
||||
| ------------------------ | ----------------------------------------------------------------------------------- |
|
||||
| `npm run uninstall` | Removes the system app but **keeps your DB and configurations** in `~/.omniroute`. |
|
||||
| `npm run uninstall:full` | Removes the app AND permanently **erases all configurations, keys, and databases**. |
|
||||
|
||||
> For detailed uninstall instructions across all methods, see [UNINSTALL.md](UNINSTALL.md).
|
||||
Reference in New Issue
Block a user