mirror of
https://github.com/diegosouzapw/OmniRoute.git
synced 2026-08-01 12:52:11 +03:00
docs: track USER_GUIDE, API_REFERENCE, and TROUBLESHOOTING in git
These files were referenced in README but ignored by .gitignore. Added them to the exception list so they appear in the repository.
This commit is contained in:
3
.gitignore
vendored
3
.gitignore
vendored
@@ -56,6 +56,9 @@ docs/*
|
||||
!docs/ARCHITECTURE.md
|
||||
!docs/CODEBASE_DOCUMENTATION.md
|
||||
!docs/CONTRIBUTING.md
|
||||
!docs/USER_GUIDE.md
|
||||
!docs/API_REFERENCE.md
|
||||
!docs/TROUBLESHOOTING.md
|
||||
!docs/EXECUTION_CONTEXT_PROVIDER_SYNC.md
|
||||
!docs/TASK_NEBIUS_BACKEND_ENABLEMENT.md
|
||||
!docs/frontend-backend-provider-gap-report.md
|
||||
|
||||
428
docs/API_REFERENCE.md
Normal file
428
docs/API_REFERENCE.md
Normal file
@@ -0,0 +1,428 @@
|
||||
# API Reference
|
||||
|
||||
Complete reference for all OmniRoute API endpoints.
|
||||
|
||||
---
|
||||
|
||||
## Table of Contents
|
||||
|
||||
- [Chat Completions](#chat-completions)
|
||||
- [Embeddings](#embeddings)
|
||||
- [Image Generation](#image-generation)
|
||||
- [List Models](#list-models)
|
||||
- [Compatibility Endpoints](#compatibility-endpoints)
|
||||
- [Semantic Cache](#semantic-cache)
|
||||
- [Dashboard & Management](#dashboard--management)
|
||||
- [Request Processing](#request-processing)
|
||||
- [Authentication](#authentication)
|
||||
|
||||
---
|
||||
|
||||
## Chat Completions
|
||||
|
||||
```bash
|
||||
POST /v1/chat/completions
|
||||
Authorization: Bearer your-api-key
|
||||
Content-Type: application/json
|
||||
|
||||
{
|
||||
"model": "cc/claude-opus-4-6",
|
||||
"messages": [
|
||||
{"role": "user", "content": "Write a function to..."}
|
||||
],
|
||||
"stream": true
|
||||
}
|
||||
```
|
||||
|
||||
### Custom Headers
|
||||
|
||||
| Header | Direction | Description |
|
||||
| ------------------------ | --------- | --------------------------------- |
|
||||
| `X-OmniRoute-No-Cache` | Request | Set to `true` to bypass cache |
|
||||
| `X-OmniRoute-Progress` | Request | Set to `true` for progress events |
|
||||
| `Idempotency-Key` | Request | Dedup key (5s window) |
|
||||
| `X-Request-Id` | Request | Alternative dedup key |
|
||||
| `X-OmniRoute-Cache` | Response | `HIT` or `MISS` (non-streaming) |
|
||||
| `X-OmniRoute-Idempotent` | Response | `true` if deduplicated |
|
||||
| `X-OmniRoute-Progress` | Response | `enabled` if progress tracking on |
|
||||
|
||||
---
|
||||
|
||||
## Embeddings
|
||||
|
||||
```bash
|
||||
POST /v1/embeddings
|
||||
Authorization: Bearer your-api-key
|
||||
Content-Type: application/json
|
||||
|
||||
{
|
||||
"model": "nebius/Qwen/Qwen3-Embedding-8B",
|
||||
"input": "The food was delicious"
|
||||
}
|
||||
```
|
||||
|
||||
Available providers: Nebius, OpenAI, Mistral, Together AI, Fireworks, NVIDIA.
|
||||
|
||||
```bash
|
||||
# List all embedding models
|
||||
GET /v1/embeddings
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Image Generation
|
||||
|
||||
```bash
|
||||
POST /v1/images/generations
|
||||
Authorization: Bearer your-api-key
|
||||
Content-Type: application/json
|
||||
|
||||
{
|
||||
"model": "openai/dall-e-3",
|
||||
"prompt": "A beautiful sunset over mountains",
|
||||
"size": "1024x1024"
|
||||
}
|
||||
```
|
||||
|
||||
Available providers: OpenAI (DALL-E), xAI (Grok Image), Together AI (FLUX), Fireworks AI.
|
||||
|
||||
```bash
|
||||
# List all image models
|
||||
GET /v1/images/generations
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## List Models
|
||||
|
||||
```bash
|
||||
GET /v1/models
|
||||
Authorization: Bearer your-api-key
|
||||
|
||||
→ Returns all chat, embedding, and image models + combos in OpenAI format
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Compatibility Endpoints
|
||||
|
||||
| Method | Path | Format |
|
||||
| ------ | --------------------------- | ---------------------- |
|
||||
| POST | `/v1/chat/completions` | OpenAI |
|
||||
| POST | `/v1/messages` | Anthropic |
|
||||
| POST | `/v1/responses` | OpenAI Responses |
|
||||
| POST | `/v1/embeddings` | OpenAI |
|
||||
| POST | `/v1/images/generations` | OpenAI |
|
||||
| GET | `/v1/models` | OpenAI |
|
||||
| POST | `/v1/messages/count_tokens` | Anthropic |
|
||||
| GET | `/v1beta/models` | Gemini |
|
||||
| POST | `/v1beta/models/{...path}` | Gemini generateContent |
|
||||
| POST | `/v1/api/chat` | Ollama |
|
||||
|
||||
### Dedicated Provider Routes
|
||||
|
||||
```bash
|
||||
POST /v1/providers/{provider}/chat/completions
|
||||
POST /v1/providers/{provider}/embeddings
|
||||
POST /v1/providers/{provider}/images/generations
|
||||
```
|
||||
|
||||
The provider prefix is auto-added if missing. Mismatched models return `400`.
|
||||
|
||||
---
|
||||
|
||||
## Semantic Cache
|
||||
|
||||
```bash
|
||||
# Get cache stats
|
||||
GET /api/cache
|
||||
|
||||
# Clear all caches
|
||||
DELETE /api/cache
|
||||
```
|
||||
|
||||
Response example:
|
||||
|
||||
```json
|
||||
{
|
||||
"semanticCache": {
|
||||
"memorySize": 42,
|
||||
"memoryMaxSize": 500,
|
||||
"dbSize": 128,
|
||||
"hitRate": 0.65
|
||||
},
|
||||
"idempotency": {
|
||||
"activeKeys": 3,
|
||||
"windowMs": 5000
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Dashboard & Management
|
||||
|
||||
### Authentication
|
||||
|
||||
| Endpoint | Method | Description |
|
||||
| ----------------------------- | ------- | --------------------- |
|
||||
| `/api/auth/login` | POST | Login |
|
||||
| `/api/auth/logout` | POST | Logout |
|
||||
| `/api/settings/require-login` | GET/PUT | Toggle login required |
|
||||
|
||||
### Provider Management
|
||||
|
||||
| Endpoint | Method | Description |
|
||||
| ---------------------------- | --------------- | ------------------------ |
|
||||
| `/api/providers` | GET/POST | List / create providers |
|
||||
| `/api/providers/[id]` | GET/PUT/DELETE | Manage a provider |
|
||||
| `/api/providers/[id]/test` | POST | Test provider connection |
|
||||
| `/api/providers/[id]/models` | GET | List provider models |
|
||||
| `/api/providers/validate` | POST | Validate provider config |
|
||||
| `/api/provider-nodes*` | Various | Provider node management |
|
||||
| `/api/provider-models` | GET/POST/DELETE | Custom models |
|
||||
|
||||
### OAuth Flows
|
||||
|
||||
| Endpoint | Method | Description |
|
||||
| -------------------------------- | ------- | ----------------------- |
|
||||
| `/api/oauth/[provider]/[action]` | Various | Provider-specific OAuth |
|
||||
|
||||
### Routing & Config
|
||||
|
||||
| Endpoint | Method | Description |
|
||||
| --------------------- | -------- | ----------------------------- |
|
||||
| `/api/models/alias` | GET/POST | Model aliases |
|
||||
| `/api/models/catalog` | GET | All models by provider + type |
|
||||
| `/api/combos*` | Various | Combo management |
|
||||
| `/api/keys*` | Various | API key management |
|
||||
| `/api/pricing` | GET | Model pricing |
|
||||
|
||||
### Usage & Analytics
|
||||
|
||||
| Endpoint | Method | Description |
|
||||
| --------------------------- | ------ | -------------------- |
|
||||
| `/api/usage/history` | GET | Usage history |
|
||||
| `/api/usage/logs` | GET | Usage logs |
|
||||
| `/api/usage/request-logs` | GET | Request-level logs |
|
||||
| `/api/usage/[connectionId]` | GET | Per-connection usage |
|
||||
|
||||
### Settings
|
||||
|
||||
| Endpoint | Method | Description |
|
||||
| ------------------------------- | ------- | ---------------------- |
|
||||
| `/api/settings` | GET/PUT | General settings |
|
||||
| `/api/settings/proxy` | GET/PUT | Network proxy config |
|
||||
| `/api/settings/proxy/test` | POST | Test proxy connection |
|
||||
| `/api/settings/ip-filter` | GET/PUT | IP allowlist/blocklist |
|
||||
| `/api/settings/thinking-budget` | GET/PUT | Reasoning token budget |
|
||||
| `/api/settings/system-prompt` | GET/PUT | Global system prompt |
|
||||
|
||||
### Monitoring
|
||||
|
||||
| Endpoint | Method | Description |
|
||||
| ------------------------ | ---------- | ----------------------- |
|
||||
| `/api/sessions` | GET | Active session tracking |
|
||||
| `/api/rate-limits` | GET | Per-account rate limits |
|
||||
| `/api/monitoring/health` | GET | Health check |
|
||||
| `/api/cache` | GET/DELETE | Cache stats / clear |
|
||||
|
||||
### Cloud Sync
|
||||
|
||||
| Endpoint | Method | Description |
|
||||
| ---------------------- | ------- | --------------------- |
|
||||
| `/api/sync/cloud` | Various | Cloud sync operations |
|
||||
| `/api/sync/initialize` | POST | Initialize sync |
|
||||
| `/api/cloud/*` | Various | Cloud management |
|
||||
|
||||
### CLI Tools
|
||||
|
||||
| Endpoint | Method | Description |
|
||||
| ---------------------------------- | ------ | ------------------- |
|
||||
| `/api/cli-tools/claude-settings` | GET | Claude CLI status |
|
||||
| `/api/cli-tools/codex-settings` | GET | Codex CLI status |
|
||||
| `/api/cli-tools/droid-settings` | GET | Droid CLI status |
|
||||
| `/api/cli-tools/openclaw-settings` | GET | OpenClaw CLI status |
|
||||
| `/api/cli-tools/runtime/[toolId]` | GET | Generic CLI runtime |
|
||||
|
||||
CLI responses include: `installed`, `runnable`, `command`, `commandPath`, `runtimeMode`, `reason`.
|
||||
|
||||
### Resilience & Rate Limits
|
||||
|
||||
| Endpoint | Method | Description |
|
||||
| ----------------------- | ------- | ------------------------------- |
|
||||
| `/api/resilience` | GET/PUT | Get/update resilience profiles |
|
||||
| `/api/resilience/reset` | POST | Reset circuit breakers |
|
||||
| `/api/rate-limits` | GET | Per-account rate limit status |
|
||||
| `/api/rate-limit` | GET | Global rate limit configuration |
|
||||
|
||||
### Evals
|
||||
|
||||
| Endpoint | Method | Description |
|
||||
| ------------ | -------- | --------------------------------- |
|
||||
| `/api/evals` | GET/POST | List eval suites / run evaluation |
|
||||
|
||||
### Policies
|
||||
|
||||
| Endpoint | Method | Description |
|
||||
| --------------- | --------------- | ----------------------- |
|
||||
| `/api/policies` | GET/POST/DELETE | Manage routing policies |
|
||||
|
||||
### Compliance
|
||||
|
||||
| Endpoint | Method | Description |
|
||||
| --------------------------- | ------ | ----------------------------- |
|
||||
| `/api/compliance/audit-log` | GET | Compliance audit log (last N) |
|
||||
|
||||
### v1beta (Gemini-Compatible)
|
||||
|
||||
| Endpoint | Method | Description |
|
||||
| -------------------------- | ------ | --------------------------------- |
|
||||
| `/v1beta/models` | GET | List models in Gemini format |
|
||||
| `/v1beta/models/{...path}` | POST | Gemini `generateContent` endpoint |
|
||||
|
||||
These endpoints mirror Gemini's API format for clients that expect native Gemini SDK compatibility.
|
||||
|
||||
### Internal / System APIs
|
||||
|
||||
| Endpoint | Method | Description |
|
||||
| --------------- | ------ | ---------------------------------------------------- |
|
||||
| `/api/init` | GET | Application initialization check (used on first run) |
|
||||
| `/api/tags` | GET | Ollama-compatible model tags (for Ollama clients) |
|
||||
| `/api/restart` | POST | Trigger graceful server restart |
|
||||
| `/api/shutdown` | POST | Trigger graceful server shutdown |
|
||||
|
||||
> **Note:** These endpoints are used internally by the system or for Ollama client compatibility. They are not typically called by end users.
|
||||
|
||||
---
|
||||
|
||||
## Audio Transcription
|
||||
|
||||
```bash
|
||||
POST /v1/audio/transcriptions
|
||||
Authorization: Bearer your-api-key
|
||||
Content-Type: multipart/form-data
|
||||
```
|
||||
|
||||
Transcribe audio files using Deepgram or AssemblyAI.
|
||||
|
||||
**Request:**
|
||||
|
||||
```bash
|
||||
curl -X POST http://localhost:20128/v1/audio/transcriptions \
|
||||
-H "Authorization: Bearer your-api-key" \
|
||||
-F "file=@recording.mp3" \
|
||||
-F "model=deepgram/nova-3"
|
||||
```
|
||||
|
||||
**Response:**
|
||||
|
||||
```json
|
||||
{
|
||||
"text": "Hello, this is the transcribed audio content.",
|
||||
"task": "transcribe",
|
||||
"language": "en",
|
||||
"duration": 12.5
|
||||
}
|
||||
```
|
||||
|
||||
**Supported providers:** `deepgram/nova-3`, `assemblyai/best`.
|
||||
|
||||
**Supported formats:** `mp3`, `wav`, `m4a`, `flac`, `ogg`, `webm`.
|
||||
|
||||
---
|
||||
|
||||
## Ollama Compatibility
|
||||
|
||||
For clients that use Ollama's API format:
|
||||
|
||||
```bash
|
||||
# Chat endpoint (Ollama format)
|
||||
POST /v1/api/chat
|
||||
|
||||
# Model listing (Ollama format)
|
||||
GET /api/tags
|
||||
```
|
||||
|
||||
Requests are automatically translated between Ollama and internal formats.
|
||||
|
||||
---
|
||||
|
||||
## Telemetry
|
||||
|
||||
```bash
|
||||
# Get latency telemetry summary (p50/p95/p99 per provider)
|
||||
GET /api/telemetry/summary
|
||||
```
|
||||
|
||||
**Response:**
|
||||
|
||||
```json
|
||||
{
|
||||
"providers": {
|
||||
"claudeCode": { "p50": 245, "p95": 890, "p99": 1200, "count": 150 },
|
||||
"github": { "p50": 180, "p95": 620, "p99": 950, "count": 320 }
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Budget
|
||||
|
||||
```bash
|
||||
# Get budget status for all API keys
|
||||
GET /api/usage/budget
|
||||
|
||||
# Set or update a budget
|
||||
POST /api/usage/budget
|
||||
Content-Type: application/json
|
||||
|
||||
{
|
||||
"keyId": "key-123",
|
||||
"limit": 50.00,
|
||||
"period": "monthly"
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Model Availability
|
||||
|
||||
```bash
|
||||
# Get real-time model availability across all providers
|
||||
GET /api/models/availability
|
||||
|
||||
# Check availability for a specific model
|
||||
POST /api/models/availability
|
||||
Content-Type: application/json
|
||||
|
||||
{
|
||||
"model": "claude-sonnet-4-5-20250929"
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Request Processing
|
||||
|
||||
1. Client sends request to `/v1/*`
|
||||
2. Route handler calls `handleChat`, `handleEmbedding`, `handleAudioTranscription`, or `handleImageGeneration`
|
||||
3. Model is resolved (direct provider/model or alias/combo)
|
||||
4. Credentials selected from local DB with account availability filtering
|
||||
5. For chat: `handleChatCore` — format detection, translation, cache check, idempotency check
|
||||
6. Provider executor sends upstream request
|
||||
7. Response translated back to client format (chat) or returned as-is (embeddings/images/audio)
|
||||
8. Usage/logging recorded
|
||||
9. Fallback applies on errors according to combo rules
|
||||
|
||||
Full architecture reference: [`ARCHITECTURE.md`](ARCHITECTURE.md)
|
||||
|
||||
---
|
||||
|
||||
## Authentication
|
||||
|
||||
- Dashboard routes (`/dashboard/*`) use `auth_token` cookie
|
||||
- Login uses saved password hash; fallback to `INITIAL_PASSWORD`
|
||||
- `requireLogin` toggleable via `/api/settings/require-login`
|
||||
- `/v1/*` routes optionally require Bearer API key when `REQUIRE_API_KEY=true`
|
||||
211
docs/TROUBLESHOOTING.md
Normal file
211
docs/TROUBLESHOOTING.md
Normal file
@@ -0,0 +1,211 @@
|
||||
# Troubleshooting
|
||||
|
||||
Common problems and solutions for OmniRoute.
|
||||
|
||||
---
|
||||
|
||||
## Quick Fixes
|
||||
|
||||
| Problem | Solution |
|
||||
| ----------------------------- | ------------------------------------------------------------------ |
|
||||
| First login not working | Check `INITIAL_PASSWORD` in `.env` (default: `123456`) |
|
||||
| Dashboard opens on wrong port | Set `PORT=20128` and `NEXT_PUBLIC_BASE_URL=http://localhost:20128` |
|
||||
| No request logs under `logs/` | Set `ENABLE_REQUEST_LOGS=true` |
|
||||
|
||||
---
|
||||
|
||||
## Provider Issues
|
||||
|
||||
### "Language model did not provide messages"
|
||||
|
||||
**Cause:** Provider quota exhausted.
|
||||
|
||||
**Fix:**
|
||||
|
||||
1. Check dashboard quota tracker
|
||||
2. Use a combo with fallback tiers
|
||||
3. Switch to cheaper/free tier
|
||||
|
||||
### Rate Limiting
|
||||
|
||||
**Cause:** Subscription quota exhausted.
|
||||
|
||||
**Fix:**
|
||||
|
||||
- Add fallback: `cc/claude-opus-4-6 → glm/glm-4.7 → if/kimi-k2-thinking`
|
||||
- Use GLM/MiniMax as cheap backup
|
||||
|
||||
### OAuth Token Expired
|
||||
|
||||
OmniRoute auto-refreshes tokens. If issues persist:
|
||||
|
||||
1. Dashboard → Provider → Reconnect
|
||||
2. Delete and re-add the provider connection
|
||||
|
||||
---
|
||||
|
||||
## Cloud Issues
|
||||
|
||||
### Cloud Sync Errors
|
||||
|
||||
1. Verify `BASE_URL` points to your running instance (e.g., `http://localhost:20128`)
|
||||
2. Verify `CLOUD_URL` points to your cloud endpoint (e.g., `https://omniroute.dev`)
|
||||
3. Keep `NEXT_PUBLIC_*` values aligned with server-side values
|
||||
|
||||
### Cloud `stream=false` Returns 500
|
||||
|
||||
**Symptom:** `Unexpected token 'd'...` on cloud endpoint for non-streaming calls.
|
||||
|
||||
**Cause:** Upstream returns SSE payload while client expects JSON.
|
||||
|
||||
**Workaround:** Use `stream=true` for cloud direct calls. Local runtime includes SSE→JSON fallback.
|
||||
|
||||
### Cloud Says Connected but "Invalid API key"
|
||||
|
||||
1. Create a fresh key from local dashboard (`/api/keys`)
|
||||
2. Run cloud sync: Enable Cloud → Sync Now
|
||||
3. Old/non-synced keys can still return `401` on cloud
|
||||
|
||||
---
|
||||
|
||||
## Docker Issues
|
||||
|
||||
### CLI Tool Shows Not Installed
|
||||
|
||||
1. Check runtime fields: `curl http://localhost:20128/api/cli-tools/runtime/codex | jq`
|
||||
2. For portable mode: use image target `runner-cli` (bundled CLIs)
|
||||
3. For host mount mode: set `CLI_EXTRA_PATHS` and mount host bin directory as read-only
|
||||
4. If `installed=true` and `runnable=false`: binary was found but failed healthcheck
|
||||
|
||||
### Quick Runtime Validation
|
||||
|
||||
```bash
|
||||
curl -s http://localhost:20128/api/cli-tools/codex-settings | jq '{installed,runnable,commandPath,runtimeMode,reason}'
|
||||
curl -s http://localhost:20128/api/cli-tools/claude-settings | jq '{installed,runnable,commandPath,runtimeMode,reason}'
|
||||
curl -s http://localhost:20128/api/cli-tools/openclaw-settings | jq '{installed,runnable,commandPath,runtimeMode,reason}'
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Cost Issues
|
||||
|
||||
### High Costs
|
||||
|
||||
1. Check usage stats in Dashboard → Usage
|
||||
2. Switch primary model to GLM/MiniMax
|
||||
3. Use free tier (Gemini CLI, iFlow) for non-critical tasks
|
||||
4. Set cost budgets per API key: Dashboard → API Keys → Budget
|
||||
|
||||
---
|
||||
|
||||
## Debugging
|
||||
|
||||
### Enable Request Logs
|
||||
|
||||
Set `ENABLE_REQUEST_LOGS=true` in your `.env` file. Logs appear under `logs/` directory.
|
||||
|
||||
### Check Provider Health
|
||||
|
||||
```bash
|
||||
# Health dashboard
|
||||
http://localhost:20128/dashboard/health
|
||||
|
||||
# API health check
|
||||
curl http://localhost:20128/api/monitoring/health
|
||||
```
|
||||
|
||||
### Runtime Storage
|
||||
|
||||
- Main state: `${DATA_DIR}/db.json` (providers, combos, aliases, keys, settings)
|
||||
- Usage: `${DATA_DIR}/usage.json`, `${DATA_DIR}/log.txt`, `${DATA_DIR}/call_logs/`
|
||||
- Request logs: `<repo>/logs/...` (when `ENABLE_REQUEST_LOGS=true`)
|
||||
|
||||
---
|
||||
|
||||
## Circuit Breaker Issues
|
||||
|
||||
### Provider stuck in OPEN state
|
||||
|
||||
When a provider's circuit breaker is OPEN, requests are blocked until the cooldown expires.
|
||||
|
||||
**Fix:**
|
||||
|
||||
1. Go to **Dashboard → Settings → Resilience**
|
||||
2. Check the circuit breaker card for the affected provider
|
||||
3. Click **Reset All** to clear all breakers, or wait for the cooldown to expire
|
||||
4. Verify the provider is actually available before resetting
|
||||
|
||||
### Provider keeps tripping the circuit breaker
|
||||
|
||||
If a provider repeatedly enters OPEN state:
|
||||
|
||||
1. Check **Dashboard → Health → Provider Health** for the failure pattern
|
||||
2. Go to **Settings → Resilience → Provider Profiles** and increase the failure threshold
|
||||
3. Check if the provider has changed API limits or requires re-authentication
|
||||
4. Review latency telemetry — high latency may cause timeout-based failures
|
||||
|
||||
---
|
||||
|
||||
## Audio Transcription Issues
|
||||
|
||||
### "Unsupported model" error
|
||||
|
||||
- Ensure you're using the correct prefix: `deepgram/nova-3` or `assemblyai/best`
|
||||
- Verify the provider is connected in **Dashboard → Providers**
|
||||
|
||||
### Transcription returns empty or fails
|
||||
|
||||
- Check supported audio formats: `mp3`, `wav`, `m4a`, `flac`, `ogg`, `webm`
|
||||
- Verify file size is within provider limits (typically < 25MB)
|
||||
- Check provider API key validity in the provider card
|
||||
|
||||
---
|
||||
|
||||
## Translator Debugging
|
||||
|
||||
Use **Dashboard → Translator** to debug format translation issues:
|
||||
|
||||
| Mode | When to Use |
|
||||
| ---------------- | -------------------------------------------------------------------------------------------- |
|
||||
| **Playground** | Compare input/output formats side by side — paste a failing request to see how it translates |
|
||||
| **Chat Tester** | Send live messages and inspect the full request/response payload including headers |
|
||||
| **Test Bench** | Run batch tests across format combinations to find which translations are broken |
|
||||
| **Live Monitor** | Watch real-time request flow to catch intermittent translation issues |
|
||||
|
||||
### Common format issues
|
||||
|
||||
- **Thinking tags not appearing** — Check if the target provider supports thinking and the thinking budget setting
|
||||
- **Tool calls dropping** — Some format translations may strip unsupported fields; verify in Playground mode
|
||||
- **System prompt missing** — Claude and Gemini handle system prompts differently; check translation output
|
||||
|
||||
---
|
||||
|
||||
## Resilience Settings
|
||||
|
||||
### Auto rate-limit not triggering
|
||||
|
||||
- Auto rate-limit only applies to API key providers (not OAuth/subscription)
|
||||
- Verify **Settings → Resilience → Provider Profiles** has auto-rate-limit enabled
|
||||
- Check if the provider returns `429` status codes or `Retry-After` headers
|
||||
|
||||
### Tuning exponential backoff
|
||||
|
||||
Provider profiles support these settings:
|
||||
|
||||
- **Base delay** — Initial wait time after first failure (default: 1s)
|
||||
- **Max delay** — Maximum wait time cap (default: 30s)
|
||||
- **Multiplier** — How much to increase delay per consecutive failure (default: 2x)
|
||||
|
||||
### Anti-thundering herd
|
||||
|
||||
When many concurrent requests hit a rate-limited provider, OmniRoute uses mutex + auto rate-limiting to serialize requests and prevent cascading failures. This is automatic for API key providers.
|
||||
|
||||
---
|
||||
|
||||
## Still Stuck?
|
||||
|
||||
- **GitHub Issues**: [github.com/diegosouzapw/OmniRoute/issues](https://github.com/diegosouzapw/OmniRoute/issues)
|
||||
- **Architecture**: See [`docs/ARCHITECTURE.md`](ARCHITECTURE.md) for internal details
|
||||
- **API Reference**: See [`docs/API_REFERENCE.md`](API_REFERENCE.md) for all endpoints
|
||||
- **Health Dashboard**: Check **Dashboard → Health** for real-time system status
|
||||
- **Translator**: Use **Dashboard → Translator** to debug format issues
|
||||
639
docs/USER_GUIDE.md
Normal file
639
docs/USER_GUIDE.md
Normal file
@@ -0,0 +1,639 @@
|
||||
# User Guide
|
||||
|
||||
Complete guide for configuring providers, creating combos, integrating CLI tools, and deploying OmniRoute.
|
||||
|
||||
---
|
||||
|
||||
## Table of Contents
|
||||
|
||||
- [Pricing at a Glance](#-pricing-at-a-glance)
|
||||
- [Use Cases](#-use-cases)
|
||||
- [Provider Setup](#-provider-setup)
|
||||
- [CLI Integration](#-cli-integration)
|
||||
- [Deployment](#-deployment)
|
||||
- [Available Models](#-available-models)
|
||||
- [Advanced Features](#-advanced-features)
|
||||
|
||||
---
|
||||
|
||||
## 💰 Pricing at a Glance
|
||||
|
||||
| Tier | Provider | Cost | Quota Reset | Best For |
|
||||
| ------------------- | ----------------- | ----------- | ---------------- | -------------------- |
|
||||
| **💳 SUBSCRIPTION** | Claude Code (Pro) | $20/mo | 5h + weekly | Already subscribed |
|
||||
| | Codex (Plus/Pro) | $20-200/mo | 5h + weekly | OpenAI users |
|
||||
| | Gemini CLI | **FREE** | 180K/mo + 1K/day | Everyone! |
|
||||
| | GitHub Copilot | $10-19/mo | Monthly | GitHub users |
|
||||
| **🔑 API KEY** | DeepSeek | Pay per use | None | Cheap reasoning |
|
||||
| | Groq | Pay per use | None | Ultra-fast inference |
|
||||
| | xAI (Grok) | Pay per use | None | Grok 4 reasoning |
|
||||
| | Mistral | Pay per use | None | EU-hosted models |
|
||||
| | Perplexity | Pay per use | None | Search-augmented |
|
||||
| | Together AI | Pay per use | None | Open-source models |
|
||||
| | Fireworks AI | Pay per use | None | Fast FLUX images |
|
||||
| | Cerebras | Pay per use | None | Wafer-scale speed |
|
||||
| | Cohere | Pay per use | None | Command R+ RAG |
|
||||
| | NVIDIA NIM | Pay per use | None | Enterprise models |
|
||||
| **💰 CHEAP** | GLM-4.7 | $0.6/1M | Daily 10AM | Budget backup |
|
||||
| | MiniMax M2.1 | $0.2/1M | 5-hour rolling | Cheapest option |
|
||||
| | Kimi K2 | $9/mo flat | 10M tokens/mo | Predictable cost |
|
||||
| **🆓 FREE** | iFlow | $0 | Unlimited | 8 models free |
|
||||
| | Qwen | $0 | Unlimited | 3 models free |
|
||||
| | Kiro | $0 | Unlimited | Claude free |
|
||||
|
||||
**💡 Pro Tip:** Start with Gemini CLI (180K free/month) + iFlow (unlimited free) combo = $0 cost!
|
||||
|
||||
---
|
||||
|
||||
## 🎯 Use Cases
|
||||
|
||||
### Case 1: "I have Claude Pro subscription"
|
||||
|
||||
**Problem:** Quota expires unused, rate limits during heavy coding
|
||||
|
||||
```
|
||||
Combo: "maximize-claude"
|
||||
1. cc/claude-opus-4-6 (use subscription fully)
|
||||
2. glm/glm-4.7 (cheap backup when quota out)
|
||||
3. if/kimi-k2-thinking (free emergency fallback)
|
||||
|
||||
Monthly cost: $20 (subscription) + ~$5 (backup) = $25 total
|
||||
vs. $20 + hitting limits = frustration
|
||||
```
|
||||
|
||||
### Case 2: "I want zero cost"
|
||||
|
||||
**Problem:** Can't afford subscriptions, need reliable AI coding
|
||||
|
||||
```
|
||||
Combo: "free-forever"
|
||||
1. gc/gemini-3-flash (180K free/month)
|
||||
2. if/kimi-k2-thinking (unlimited free)
|
||||
3. qw/qwen3-coder-plus (unlimited free)
|
||||
|
||||
Monthly cost: $0
|
||||
Quality: Production-ready models
|
||||
```
|
||||
|
||||
### Case 3: "I need 24/7 coding, no interruptions"
|
||||
|
||||
**Problem:** Deadlines, can't afford downtime
|
||||
|
||||
```
|
||||
Combo: "always-on"
|
||||
1. cc/claude-opus-4-6 (best quality)
|
||||
2. cx/gpt-5.2-codex (second subscription)
|
||||
3. glm/glm-4.7 (cheap, resets daily)
|
||||
4. minimax/MiniMax-M2.1 (cheapest, 5h reset)
|
||||
5. if/kimi-k2-thinking (free unlimited)
|
||||
|
||||
Result: 5 layers of fallback = zero downtime
|
||||
Monthly cost: $20-200 (subscriptions) + $10-20 (backup)
|
||||
```
|
||||
|
||||
### Case 4: "I want FREE AI in OpenClaw"
|
||||
|
||||
**Problem:** Need AI assistant in messaging apps, completely free
|
||||
|
||||
```
|
||||
Combo: "openclaw-free"
|
||||
1. if/glm-4.7 (unlimited free)
|
||||
2. if/minimax-m2.1 (unlimited free)
|
||||
3. if/kimi-k2-thinking (unlimited free)
|
||||
|
||||
Monthly cost: $0
|
||||
Access via: WhatsApp, Telegram, Slack, Discord, iMessage, Signal...
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📖 Provider Setup
|
||||
|
||||
### 🔐 Subscription Providers
|
||||
|
||||
#### Claude Code (Pro/Max)
|
||||
|
||||
```bash
|
||||
Dashboard → Providers → Connect Claude Code
|
||||
→ OAuth login → Auto token refresh
|
||||
→ 5-hour + weekly quota tracking
|
||||
|
||||
Models:
|
||||
cc/claude-opus-4-6
|
||||
cc/claude-sonnet-4-5-20250929
|
||||
cc/claude-haiku-4-5-20251001
|
||||
```
|
||||
|
||||
**Pro Tip:** Use Opus for complex tasks, Sonnet for speed. OmniRoute tracks quota per model!
|
||||
|
||||
#### OpenAI Codex (Plus/Pro)
|
||||
|
||||
```bash
|
||||
Dashboard → Providers → Connect Codex
|
||||
→ OAuth login (port 1455)
|
||||
→ 5-hour + weekly reset
|
||||
|
||||
Models:
|
||||
cx/gpt-5.2-codex
|
||||
cx/gpt-5.1-codex-max
|
||||
```
|
||||
|
||||
#### Gemini CLI (FREE 180K/month!)
|
||||
|
||||
```bash
|
||||
Dashboard → Providers → Connect Gemini CLI
|
||||
→ Google OAuth
|
||||
→ 180K completions/month + 1K/day
|
||||
|
||||
Models:
|
||||
gc/gemini-3-flash-preview
|
||||
gc/gemini-2.5-pro
|
||||
```
|
||||
|
||||
**Best Value:** Huge free tier! Use this before paid tiers.
|
||||
|
||||
#### GitHub Copilot
|
||||
|
||||
```bash
|
||||
Dashboard → Providers → Connect GitHub
|
||||
→ OAuth via GitHub
|
||||
→ Monthly reset (1st of month)
|
||||
|
||||
Models:
|
||||
gh/gpt-5
|
||||
gh/claude-4.5-sonnet
|
||||
gh/gemini-3-pro
|
||||
```
|
||||
|
||||
### 💰 Cheap Providers
|
||||
|
||||
#### GLM-4.7 (Daily reset, $0.6/1M)
|
||||
|
||||
1. Sign up: [Zhipu AI](https://open.bigmodel.cn/)
|
||||
2. Get API key from Coding Plan
|
||||
3. Dashboard → Add API Key: Provider: `glm`, API Key: `your-key`
|
||||
|
||||
**Use:** `glm/glm-4.7` — **Pro Tip:** Coding Plan offers 3× quota at 1/7 cost! Reset daily 10:00 AM.
|
||||
|
||||
#### MiniMax M2.1 (5h reset, $0.20/1M)
|
||||
|
||||
1. Sign up: [MiniMax](https://www.minimax.io/)
|
||||
2. Get API key → Dashboard → Add API Key
|
||||
|
||||
**Use:** `minimax/MiniMax-M2.1` — **Pro Tip:** Cheapest option for long context (1M tokens)!
|
||||
|
||||
#### Kimi K2 ($9/month flat)
|
||||
|
||||
1. Subscribe: [Moonshot AI](https://platform.moonshot.ai/)
|
||||
2. Get API key → Dashboard → Add API Key
|
||||
|
||||
**Use:** `kimi/kimi-latest` — **Pro Tip:** Fixed $9/month for 10M tokens = $0.90/1M effective cost!
|
||||
|
||||
### 🆓 FREE Providers
|
||||
|
||||
#### iFlow (8 FREE models)
|
||||
|
||||
```bash
|
||||
Dashboard → Connect iFlow → OAuth login → Unlimited usage
|
||||
|
||||
Models: if/kimi-k2-thinking, if/qwen3-coder-plus, if/glm-4.7, if/minimax-m2, if/deepseek-r1
|
||||
```
|
||||
|
||||
#### Qwen (3 FREE models)
|
||||
|
||||
```bash
|
||||
Dashboard → Connect Qwen → Device code auth → Unlimited usage
|
||||
|
||||
Models: qw/qwen3-coder-plus, qw/qwen3-coder-flash
|
||||
```
|
||||
|
||||
#### Kiro (Claude FREE)
|
||||
|
||||
```bash
|
||||
Dashboard → Connect Kiro → AWS Builder ID or Google/GitHub → Unlimited
|
||||
|
||||
Models: kr/claude-sonnet-4.5, kr/claude-haiku-4.5
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🎨 Combos
|
||||
|
||||
### Example 1: Maximize Subscription → Cheap Backup
|
||||
|
||||
```
|
||||
Dashboard → Combos → Create New
|
||||
|
||||
Name: premium-coding
|
||||
Models:
|
||||
1. cc/claude-opus-4-6 (Subscription primary)
|
||||
2. glm/glm-4.7 (Cheap backup, $0.6/1M)
|
||||
3. minimax/MiniMax-M2.1 (Cheapest fallback, $0.20/1M)
|
||||
|
||||
Use in CLI: premium-coding
|
||||
```
|
||||
|
||||
### Example 2: Free-Only (Zero Cost)
|
||||
|
||||
```
|
||||
Name: free-combo
|
||||
Models:
|
||||
1. gc/gemini-3-flash-preview (180K free/month)
|
||||
2. if/kimi-k2-thinking (unlimited)
|
||||
3. qw/qwen3-coder-plus (unlimited)
|
||||
|
||||
Cost: $0 forever!
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🔧 CLI Integration
|
||||
|
||||
### Cursor IDE
|
||||
|
||||
```
|
||||
Settings → Models → Advanced:
|
||||
OpenAI API Base URL: http://localhost:20128/v1
|
||||
OpenAI API Key: [from omniroute dashboard]
|
||||
Model: cc/claude-opus-4-6
|
||||
```
|
||||
|
||||
### Claude Code
|
||||
|
||||
Edit `~/.claude/config.json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"anthropic_api_base": "http://localhost:20128/v1",
|
||||
"anthropic_api_key": "your-omniroute-api-key"
|
||||
}
|
||||
```
|
||||
|
||||
### Codex CLI
|
||||
|
||||
```bash
|
||||
export OPENAI_BASE_URL="http://localhost:20128"
|
||||
export OPENAI_API_KEY="your-omniroute-api-key"
|
||||
codex "your prompt"
|
||||
```
|
||||
|
||||
### OpenClaw
|
||||
|
||||
Edit `~/.openclaw/openclaw.json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"agents": {
|
||||
"defaults": {
|
||||
"model": { "primary": "omniroute/if/glm-4.7" }
|
||||
}
|
||||
},
|
||||
"models": {
|
||||
"providers": {
|
||||
"omniroute": {
|
||||
"baseUrl": "http://localhost:20128/v1",
|
||||
"apiKey": "your-omniroute-api-key",
|
||||
"api": "openai-completions",
|
||||
"models": [{ "id": "if/glm-4.7", "name": "glm-4.7" }]
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Or use Dashboard:** CLI Tools → OpenClaw → Auto-config
|
||||
|
||||
### Cline / Continue / RooCode
|
||||
|
||||
```
|
||||
Provider: OpenAI Compatible
|
||||
Base URL: http://localhost:20128/v1
|
||||
API Key: [from dashboard]
|
||||
Model: cc/claude-opus-4-6
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🚀 Deployment
|
||||
|
||||
### VPS Deployment
|
||||
|
||||
```bash
|
||||
git clone https://github.com/diegosouzapw/OmniRoute.git
|
||||
cd OmniRoute && npm install && npm run build
|
||||
|
||||
export JWT_SECRET="your-secure-secret-change-this"
|
||||
export INITIAL_PASSWORD="your-password"
|
||||
export DATA_DIR="/var/lib/omniroute"
|
||||
export PORT="20128"
|
||||
export HOSTNAME="0.0.0.0"
|
||||
export NODE_ENV="production"
|
||||
export NEXT_PUBLIC_BASE_URL="http://localhost:20128"
|
||||
export API_KEY_SECRET="endpoint-proxy-api-key-secret"
|
||||
|
||||
npm run start
|
||||
# Or: pm2 start npm --name omniroute -- start
|
||||
```
|
||||
|
||||
### Docker
|
||||
|
||||
```bash
|
||||
# Build image (default = runner-cli with codex/claude/droid preinstalled)
|
||||
docker build -t omniroute:cli .
|
||||
|
||||
# Portable mode (recommended)
|
||||
docker run -d --name omniroute -p 20128:20128 --env-file ./.env -v omniroute-data:/app/data omniroute:cli
|
||||
```
|
||||
|
||||
For host-integrated mode with CLI binaries, see the Docker section in the main docs.
|
||||
|
||||
### Environment Variables
|
||||
|
||||
| Variable | Default | Description |
|
||||
| --------------------- | ------------------------------------ | ------------------------------------------------------- |
|
||||
| `JWT_SECRET` | `omniroute-default-secret-change-me` | JWT signing secret (**change in production**) |
|
||||
| `INITIAL_PASSWORD` | `123456` | First login password |
|
||||
| `DATA_DIR` | `~/.omniroute` | Data directory (db, usage, logs) |
|
||||
| `PORT` | framework default | Service port (`20128` in examples) |
|
||||
| `HOSTNAME` | framework default | Bind host (Docker defaults to `0.0.0.0`) |
|
||||
| `NODE_ENV` | runtime default | Set `production` for deploy |
|
||||
| `BASE_URL` | `http://localhost:20128` | Server-side internal base URL |
|
||||
| `CLOUD_URL` | `https://omniroute.dev` | Cloud sync endpoint base URL |
|
||||
| `API_KEY_SECRET` | `endpoint-proxy-api-key-secret` | HMAC secret for generated API keys |
|
||||
| `REQUIRE_API_KEY` | `false` | Enforce Bearer API key on `/v1/*` |
|
||||
| `ENABLE_REQUEST_LOGS` | `false` | Enables request/response logs |
|
||||
| `AUTH_COOKIE_SECURE` | `false` | Force `Secure` auth cookie (behind HTTPS reverse proxy) |
|
||||
|
||||
For the full environment variable reference, see the [README](../README.md).
|
||||
|
||||
---
|
||||
|
||||
## 📊 Available Models
|
||||
|
||||
<details>
|
||||
<summary><b>View all available models</b></summary>
|
||||
|
||||
**Claude Code (`cc/`)** — Pro/Max: `cc/claude-opus-4-6`, `cc/claude-sonnet-4-5-20250929`, `cc/claude-haiku-4-5-20251001`
|
||||
|
||||
**Codex (`cx/`)** — Plus/Pro: `cx/gpt-5.2-codex`, `cx/gpt-5.1-codex-max`
|
||||
|
||||
**Gemini CLI (`gc/`)** — FREE: `gc/gemini-3-flash-preview`, `gc/gemini-2.5-pro`
|
||||
|
||||
**GitHub Copilot (`gh/`)**: `gh/gpt-5`, `gh/claude-4.5-sonnet`
|
||||
|
||||
**GLM (`glm/`)** — $0.6/1M: `glm/glm-4.7`
|
||||
|
||||
**MiniMax (`minimax/`)** — $0.2/1M: `minimax/MiniMax-M2.1`
|
||||
|
||||
**iFlow (`if/`)** — FREE: `if/kimi-k2-thinking`, `if/qwen3-coder-plus`, `if/deepseek-r1`
|
||||
|
||||
**Qwen (`qw/`)** — FREE: `qw/qwen3-coder-plus`, `qw/qwen3-coder-flash`
|
||||
|
||||
**Kiro (`kr/`)** — FREE: `kr/claude-sonnet-4.5`, `kr/claude-haiku-4.5`
|
||||
|
||||
**DeepSeek (`ds/`)**: `ds/deepseek-chat`, `ds/deepseek-reasoner`
|
||||
|
||||
**Groq (`groq/`)**: `groq/llama-3.3-70b-versatile`, `groq/llama-4-maverick-17b-128e-instruct`
|
||||
|
||||
**xAI (`xai/`)**: `xai/grok-4`, `xai/grok-4-0709-fast-reasoning`, `xai/grok-code-mini`
|
||||
|
||||
**Mistral (`mistral/`)**: `mistral/mistral-large-2501`, `mistral/codestral-2501`
|
||||
|
||||
**Perplexity (`pplx/`)**: `pplx/sonar-pro`, `pplx/sonar`
|
||||
|
||||
**Together AI (`together/`)**: `together/meta-llama/Llama-3.3-70B-Instruct-Turbo`
|
||||
|
||||
**Fireworks AI (`fireworks/`)**: `fireworks/accounts/fireworks/models/deepseek-v3p1`
|
||||
|
||||
**Cerebras (`cerebras/`)**: `cerebras/llama-3.3-70b`
|
||||
|
||||
**Cohere (`cohere/`)**: `cohere/command-r-plus-08-2024`
|
||||
|
||||
**NVIDIA NIM (`nvidia/`)**: `nvidia/nvidia/llama-3.3-70b-instruct`
|
||||
|
||||
</details>
|
||||
|
||||
---
|
||||
|
||||
## 🧩 Advanced Features
|
||||
|
||||
### Custom Models
|
||||
|
||||
Add any model ID to any provider without waiting for an app update:
|
||||
|
||||
```bash
|
||||
# Via API
|
||||
curl -X POST http://localhost:20128/api/provider-models \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"provider": "openai", "modelId": "gpt-4.5-preview", "modelName": "GPT-4.5 Preview"}'
|
||||
|
||||
# List: curl http://localhost:20128/api/provider-models?provider=openai
|
||||
# Remove: curl -X DELETE "http://localhost:20128/api/provider-models?provider=openai&model=gpt-4.5-preview"
|
||||
```
|
||||
|
||||
Or use Dashboard: **Providers → [Provider] → Custom Models**.
|
||||
|
||||
### Dedicated Provider Routes
|
||||
|
||||
Route requests directly to a specific provider with model validation:
|
||||
|
||||
```bash
|
||||
POST http://localhost:20128/v1/providers/openai/chat/completions
|
||||
POST http://localhost:20128/v1/providers/openai/embeddings
|
||||
POST http://localhost:20128/v1/providers/fireworks/images/generations
|
||||
```
|
||||
|
||||
The provider prefix is auto-added if missing. Mismatched models return `400`.
|
||||
|
||||
### Network Proxy Configuration
|
||||
|
||||
```bash
|
||||
# Set global proxy
|
||||
curl -X PUT http://localhost:20128/api/settings/proxy \
|
||||
-d '{"global": {"type":"http","host":"proxy.example.com","port":"8080"}}'
|
||||
|
||||
# Per-provider proxy
|
||||
curl -X PUT http://localhost:20128/api/settings/proxy \
|
||||
-d '{"providers": {"openai": {"type":"socks5","host":"proxy.example.com","port":"1080"}}}'
|
||||
|
||||
# Test proxy
|
||||
curl -X POST http://localhost:20128/api/settings/proxy/test \
|
||||
-d '{"proxy":{"type":"socks5","host":"proxy.example.com","port":"1080"}}'
|
||||
```
|
||||
|
||||
**Precedence:** Key-specific → Combo-specific → Provider-specific → Global → Environment.
|
||||
|
||||
### Model Catalog API
|
||||
|
||||
```bash
|
||||
curl http://localhost:20128/api/models/catalog
|
||||
```
|
||||
|
||||
Returns models grouped by provider with types (`chat`, `embedding`, `image`).
|
||||
|
||||
### Cloud Sync
|
||||
|
||||
- Sync providers, combos, and settings across devices
|
||||
- Automatic background sync with timeout + fail-fast
|
||||
- Prefer server-side `BASE_URL`/`CLOUD_URL` in production
|
||||
|
||||
### LLM Gateway Intelligence (Phase 9)
|
||||
|
||||
- **Semantic Cache** — Auto-caches non-streaming, temperature=0 responses (bypass with `X-OmniRoute-No-Cache: true`)
|
||||
- **Request Idempotency** — Deduplicates requests within 5s via `Idempotency-Key` or `X-Request-Id` header
|
||||
- **Progress Tracking** — Opt-in SSE `event: progress` events via `X-OmniRoute-Progress: true` header
|
||||
|
||||
---
|
||||
|
||||
### Translator Playground
|
||||
|
||||
Access via **Dashboard → Translator**. Debug and visualize how OmniRoute translates API requests between providers.
|
||||
|
||||
| Mode | Purpose |
|
||||
| ---------------- | -------------------------------------------------------------------------------------- |
|
||||
| **Playground** | Select source/target formats, paste a request, and see the translated output instantly |
|
||||
| **Chat Tester** | Send live chat messages through the proxy and inspect the full request/response cycle |
|
||||
| **Test Bench** | Run batch tests across multiple format combinations to verify translation correctness |
|
||||
| **Live Monitor** | Watch real-time translations as requests flow through the proxy |
|
||||
|
||||
**Use cases:**
|
||||
|
||||
- Debug why a specific client/provider combination fails
|
||||
- Verify that thinking tags, tool calls, and system prompts translate correctly
|
||||
- Compare format differences between OpenAI, Claude, Gemini, and Responses API formats
|
||||
|
||||
---
|
||||
|
||||
### Routing Strategies
|
||||
|
||||
Configure via **Dashboard → Settings → Routing**.
|
||||
|
||||
| Strategy | Description |
|
||||
| ------------------------------ | ------------------------------------------------------------------------------------------------ |
|
||||
| **Fill First** | Uses accounts in priority order — primary account handles all requests until unavailable |
|
||||
| **Round Robin** | Cycles through all accounts with a configurable sticky limit (default: 3 calls per account) |
|
||||
| **P2C (Power of Two Choices)** | Picks 2 random accounts and routes to the healthier one — balances load with awareness of health |
|
||||
|
||||
#### Wildcard Model Aliases
|
||||
|
||||
Create wildcard patterns to remap model names:
|
||||
|
||||
```
|
||||
Pattern: claude-sonnet-* → Target: cc/claude-sonnet-4-5-20250929
|
||||
Pattern: gpt-* → Target: gh/gpt-5.1-codex
|
||||
```
|
||||
|
||||
Wildcards support `*` (any characters) and `?` (single character).
|
||||
|
||||
#### Fallback Chains
|
||||
|
||||
Define global fallback chains that apply across all requests:
|
||||
|
||||
```
|
||||
Chain: production-fallback
|
||||
1. cc/claude-opus-4-6
|
||||
2. gh/gpt-5.1-codex
|
||||
3. glm/glm-4.7
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Resilience & Circuit Breakers
|
||||
|
||||
Configure via **Dashboard → Settings → Resilience**.
|
||||
|
||||
OmniRoute implements provider-level resilience with three components:
|
||||
|
||||
1. **Circuit Breaker** — Tracks failures per provider and automatically opens the circuit when a threshold is reached:
|
||||
- **CLOSED** (Healthy) — Requests flow normally
|
||||
- **OPEN** — Provider is temporarily blocked after repeated failures
|
||||
- **HALF_OPEN** — Testing if provider has recovered
|
||||
|
||||
2. **Provider Profiles** — Per-provider configuration for:
|
||||
- Failure threshold (how many failures before opening)
|
||||
- Cooldown duration
|
||||
- Rate limit detection sensitivity
|
||||
- Exponential backoff parameters
|
||||
|
||||
3. **Rate Limit Auto-Detection** — Monitors `429` and `Retry-After` headers to proactively avoid hitting provider rate limits.
|
||||
|
||||
**Pro Tip:** Use **Reset All** button to clear all circuit breakers and cooldowns when a provider recovers from an outage.
|
||||
|
||||
---
|
||||
|
||||
### Costs & Budget Management
|
||||
|
||||
Access via **Dashboard → Costs**.
|
||||
|
||||
| Tab | Purpose |
|
||||
| ----------- | ---------------------------------------------------------------------------------------- |
|
||||
| **Budget** | Set spending limits per API key with daily/weekly/monthly budgets and real-time tracking |
|
||||
| **Pricing** | View and edit model pricing entries — cost per 1K input/output tokens per provider |
|
||||
|
||||
```bash
|
||||
# API: Set a budget
|
||||
curl -X POST http://localhost:20128/api/usage/budget \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"keyId": "key-123", "limit": 50.00, "period": "monthly"}'
|
||||
|
||||
# API: Get current budget status
|
||||
curl http://localhost:20128/api/usage/budget
|
||||
```
|
||||
|
||||
**Cost Tracking:** Every request logs token usage and calculates cost using the pricing table. View breakdowns in **Dashboard → Usage** by provider, model, and API key.
|
||||
|
||||
---
|
||||
|
||||
### Audio Transcription
|
||||
|
||||
OmniRoute supports audio transcription via the OpenAI-compatible endpoint:
|
||||
|
||||
```bash
|
||||
POST /v1/audio/transcriptions
|
||||
Authorization: Bearer your-api-key
|
||||
Content-Type: multipart/form-data
|
||||
|
||||
# Example with curl
|
||||
curl -X POST http://localhost:20128/v1/audio/transcriptions \
|
||||
-H "Authorization: Bearer your-api-key" \
|
||||
-F "file=@audio.mp3" \
|
||||
-F "model=deepgram/nova-3"
|
||||
```
|
||||
|
||||
Available providers: **Deepgram** (`deepgram/`), **AssemblyAI** (`assemblyai/`).
|
||||
|
||||
Supported audio formats: `mp3`, `wav`, `m4a`, `flac`, `ogg`, `webm`.
|
||||
|
||||
---
|
||||
|
||||
### Combo Balancing Strategies
|
||||
|
||||
Configure per-combo balancing in **Dashboard → Combos → Create/Edit → Strategy**.
|
||||
|
||||
| Strategy | Description |
|
||||
| ------------------ | ------------------------------------------------------------------------ |
|
||||
| **Round-Robin** | Rotates through models sequentially |
|
||||
| **Priority** | Always tries the first model; falls back only on error |
|
||||
| **Random** | Picks a random model from the combo for each request |
|
||||
| **Weighted** | Routes proportionally based on assigned weights per model |
|
||||
| **Least-Used** | Routes to the model with the fewest recent requests (uses combo metrics) |
|
||||
| **Cost-Optimized** | Routes to the cheapest available model (uses pricing table) |
|
||||
|
||||
Global combo defaults can be set in **Dashboard → Settings → Routing → Combo Defaults**.
|
||||
|
||||
---
|
||||
|
||||
### Health Dashboard
|
||||
|
||||
Access via **Dashboard → Health**. Real-time system health overview with 6 cards:
|
||||
|
||||
| Card | What It Shows |
|
||||
| --------------------- | ----------------------------------------------------------- |
|
||||
| **System Status** | Uptime, version, memory usage, data directory |
|
||||
| **Provider Health** | Per-provider circuit breaker state (Closed/Open/Half-Open) |
|
||||
| **Rate Limits** | Active rate limit cooldowns per account with remaining time |
|
||||
| **Active Lockouts** | Providers temporarily blocked by the lockout policy |
|
||||
| **Signature Cache** | Deduplication cache stats (active keys, hit rate) |
|
||||
| **Latency Telemetry** | p50/p95/p99 latency aggregation per provider |
|
||||
|
||||
**Pro Tip:** The Health page auto-refreshes every 10 seconds. Use the circuit breaker card to identify which providers are experiencing issues.
|
||||
Reference in New Issue
Block a user