mirror of
https://github.com/diegosouzapw/OmniRoute.git
synced 2026-07-26 09:52:11 +03:00
docs: close critical documentation gaps (ACP, router strategies, APIs, compression) (#3438)
Integrated into release/v3.8.17
This commit is contained in:
@@ -4,6 +4,7 @@ var docs = defineDocs({
|
||||
dir: "docs",
|
||||
docs: {
|
||||
files: [
|
||||
"./getting-started/**/*.md",
|
||||
"./architecture/**/*.md",
|
||||
"./guides/**/*.md",
|
||||
"./reference/**/*.md",
|
||||
|
||||
334
docs/DOCUMENTATION_OVERHAUL_PLAN.md
Normal file
334
docs/DOCUMENTATION_OVERHAUL_PLAN.md
Normal file
@@ -0,0 +1,334 @@
|
||||
# OmniRoute Documentation Overhaul Plan
|
||||
|
||||
> Goal: Make all docs accessible to non-tech users while preserving technical depth for developers.
|
||||
> Strategy: Two-tier docs (User Guide + Technical Reference) with cross-links.
|
||||
|
||||
---
|
||||
|
||||
## Design Principles
|
||||
|
||||
1. **User Guide** (new files in `docs/guides/`)
|
||||
- Plain language, no jargon
|
||||
- "Why → What → How" structure
|
||||
- Tables for comparisons
|
||||
- Copy-paste examples
|
||||
- "What's next?" at the end
|
||||
|
||||
2. **Technical Reference** (existing files)
|
||||
- Keep as-is (already well-written)
|
||||
- Add "For Users" callout at top linking to user guide
|
||||
- Add "Prerequisites" section
|
||||
- Ensure consistent formatting
|
||||
|
||||
3. **Cross-Links**
|
||||
- User guides link to technical docs for "learn more"
|
||||
- Technical docs link to user guides for "quick start"
|
||||
|
||||
---
|
||||
|
||||
## Phase 1: User-Facing Quick Start Docs (Highest Impact)
|
||||
|
||||
### 1.1 `docs/guides/QUICK-START.md` — NEW
|
||||
**Replaces**: Condensed version of README.md
|
||||
**Content**:
|
||||
- Install (npm, Docker, source)
|
||||
- Connect a free provider (3 steps)
|
||||
- Point your IDE to OmniRoute
|
||||
- Verify it works
|
||||
- "What's next?" → link to other guides
|
||||
|
||||
### 1.2 `docs/guides/AUTO-COMBO-GUIDE.md` — NEW
|
||||
**Complements**: `docs/routing/AUTO-COMBO.md`
|
||||
**Content**:
|
||||
- What auto-combo does (30-second version)
|
||||
- Which `auto` should I use? (table)
|
||||
- How it picks providers (simple version)
|
||||
- Common questions (FAQ)
|
||||
- "Learn more" → link to technical reference
|
||||
|
||||
### 1.3 `docs/guides/PROVIDERS-GUIDE.md` — NEW
|
||||
**Complements**: `docs/reference/PROVIDER_REFERENCE.md`
|
||||
**Content**:
|
||||
- What is a provider? (analogy)
|
||||
- How to connect a provider (step-by-step)
|
||||
- Free vs paid providers (table)
|
||||
- OAuth vs API key (which do I need?)
|
||||
- Troubleshooting connection issues
|
||||
- "Learn more" → link to provider reference
|
||||
|
||||
### 1.4 `docs/guides/FREE-TIERS-GUIDE.md` — NEW
|
||||
**Complements**: `docs/reference/FREE_TIERS.md`
|
||||
**Content**:
|
||||
- What are free tiers? (analogy)
|
||||
- Best free providers (table with quotas)
|
||||
- How to stack free tiers for unlimited usage
|
||||
- Common questions (FAQ)
|
||||
- "Learn more" → link to free tiers reference
|
||||
|
||||
### 1.5 `docs/guides/TROUBLESHOOTING.md` — REWRITE
|
||||
**Current**: Technical, code-heavy
|
||||
**New**: "I see X → Do Y" format
|
||||
**Content**:
|
||||
- Problem → Solution table
|
||||
- Common error messages (with screenshots)
|
||||
- "Still stuck?" → link to community
|
||||
|
||||
---
|
||||
|
||||
## Phase 2: Refine Existing User Docs
|
||||
|
||||
### 2.1 `docs/guides/USER_GUIDE.md` — UPDATE
|
||||
**Changes**:
|
||||
- Add "What you'll learn" section at top
|
||||
- Remove jargon, explain terms
|
||||
- Add step-by-step screenshots
|
||||
- Add "Common mistakes" section
|
||||
|
||||
### 2.2 `docs/guides/SETUP_GUIDE.md` — UPDATE
|
||||
**Changes**:
|
||||
- Add "Prerequisites" section
|
||||
- Simplify commands (one-liners)
|
||||
- Add "Verify it works" step
|
||||
- Add "Troubleshooting" section
|
||||
|
||||
### 2.3 `docs/guides/FEATURES.md` — UPDATE
|
||||
**Changes**:
|
||||
- Add "Why this matters" for each feature
|
||||
- Add use-case examples
|
||||
- Add "How to enable" for each feature
|
||||
- Add screenshots
|
||||
|
||||
### 2.4 `docs/guides/DOCKER_GUIDE.md` — UPDATE
|
||||
**Changes**:
|
||||
- Add "Docker for beginners" section
|
||||
- Simplify commands
|
||||
- Add "Common Docker issues" section
|
||||
- Add "What's next?" section
|
||||
|
||||
### 2.5 `docs/guides/ELECTRON_GUIDE.md` — UPDATE
|
||||
**Changes**:
|
||||
- Add "What is Electron?" explanation
|
||||
- Add screenshots
|
||||
- Add "Common issues" section
|
||||
|
||||
### 2.6 `docs/guides/TERMUX_GUIDE.md` — UPDATE
|
||||
**Changes**:
|
||||
- Add "What is Termux?" explanation
|
||||
- Add step-by-step with screenshots
|
||||
- Add "Common issues" section
|
||||
|
||||
### 2.7 `docs/guides/PWA_GUIDE.md` — UPDATE
|
||||
**Changes**:
|
||||
- Add "What is a PWA?" explanation
|
||||
- Add step-by-step with screenshots
|
||||
- Add "Common issues" section
|
||||
|
||||
### 2.8 `docs/guides/I18N.md` — UPDATE
|
||||
**Changes**:
|
||||
- Add "How to change language" (user perspective)
|
||||
- Add "How to contribute translations" (contributor perspective)
|
||||
|
||||
### 2.9 `docs/guides/KIRO_SETUP.md` — UPDATE
|
||||
**Changes**:
|
||||
- Add "What is Kiro?" explanation
|
||||
- Add step-by-step with screenshots
|
||||
- Add "Common issues" section
|
||||
|
||||
### 2.10 `docs/guides/UNINSTALL.md` — UPDATE
|
||||
**Changes**:
|
||||
- Add "Why uninstall?" section (common reasons)
|
||||
- Add "Before you uninstall" checklist
|
||||
- Simplify commands
|
||||
|
||||
---
|
||||
|
||||
## Phase 3: Technical Docs (Add Cross-Links, Keep As-Is)
|
||||
|
||||
### 3.1 `docs/architecture/` (6 files)
|
||||
**Changes**: Add "For Users" callout at top → link to user guide
|
||||
- ARCHITECTURE.md
|
||||
- AUTHZ_GUIDE.md
|
||||
- CODEBASE_DOCUMENTATION.md
|
||||
- MONITORING_SECTIONS.md
|
||||
- REPOSITORY_MAP.md
|
||||
- RESILIENCE_GUIDE.md
|
||||
|
||||
### 3.2 `docs/frameworks/` (16 files)
|
||||
**Changes**: Add "For Users" callout at top → link to user guide
|
||||
- A2A-SERVER.md
|
||||
- AGENT-SKILLS.md
|
||||
- AGENTBRIDGE.md
|
||||
- AGENT_PROTOCOLS_GUIDE.md
|
||||
- CLOUD_AGENT.md
|
||||
- EMBEDDED-SERVICES.md
|
||||
- EVALS.md
|
||||
- GAMIFICATION.md
|
||||
- MCP-SERVER.md
|
||||
- MEMORY.md
|
||||
- OPENCODE.md
|
||||
- PLAYGROUND_STUDIO.md
|
||||
- SEARCH_TOOLS_STUDIO.md
|
||||
- SKILLS.md
|
||||
- TRAFFIC_INSPECTOR.md
|
||||
- WEBHOOKS.md
|
||||
|
||||
### 3.3 `docs/security/` (9 files)
|
||||
**Changes**: Add "For Users" callout at top → link to user guide
|
||||
- CLI_TOKEN.md
|
||||
- CLI_TOKEN_AUTH.md
|
||||
- COMPLIANCE.md
|
||||
- ERROR_SANITIZATION.md
|
||||
- GUARDRAILS.md
|
||||
- PUBLIC_CREDS.md
|
||||
- ROUTE_GUARD_TIERS.md
|
||||
- SOCKET_DEV_FINDINGS.md
|
||||
- STEALTH_GUIDE.md
|
||||
|
||||
### 3.4 `docs/ops/` (8 files)
|
||||
**Changes**: Add "For Users" callout at top → link to user guide
|
||||
- COVERAGE_PLAN.md
|
||||
- E2E_DASHBOARD_SHAKEDOWN_v3.8.0.md
|
||||
- FLY_IO_DEPLOYMENT_GUIDE.md
|
||||
- PROXY_GUIDE.md
|
||||
- RELEASE_CHECKLIST.md
|
||||
- SQLITE_RUNTIME.md
|
||||
- TUNNELS_GUIDE.md
|
||||
- VM_DEPLOYMENT_GUIDE.md
|
||||
|
||||
### 3.5 `docs/compression/` (5 files)
|
||||
**Changes**: Add "For Users" callout at top → link to user guide
|
||||
- COMPRESSION_ENGINES.md
|
||||
- COMPRESSION_GUIDE.md
|
||||
- COMPRESSION_LANGUAGE_PACKS.md
|
||||
- COMPRESSION_RULES_FORMAT.md
|
||||
- RTK_COMPRESSION.md
|
||||
|
||||
### 3.6 `docs/routing/` (3 files)
|
||||
**Changes**: Add "For Users" callout at top → link to user guide
|
||||
- AUTO-COMBO.md (→ link to AUTO-COMBO-GUIDE.md)
|
||||
- QUOTA_SHARE.md
|
||||
- REASONING_REPLAY.md
|
||||
|
||||
### 3.7 `docs/reference/` (5 files)
|
||||
**Changes**: Add "For Users" callout at top → link to user guide
|
||||
- API_REFERENCE.md
|
||||
- CLI-TOOLS.md
|
||||
- ENVIRONMENT.md
|
||||
- FREE_TIERS.md (→ link to FREE-TIERS-GUIDE.md)
|
||||
- PROVIDER_REFERENCE.md (→ link to PROVIDERS-GUIDE.md)
|
||||
|
||||
### 3.8 Other Docs (keep as-is)
|
||||
- `docs/comparison/OMNIROUTE_VS_ALTERNATIVES.md` — Already user-friendly
|
||||
- `docs/marketing/TIERS.md` — Already user-friendly
|
||||
- `docs/diagrams/README.md` — Keep as-is
|
||||
- `docs/dev/plugins.md` — Developer-only
|
||||
- `docs/plugins/PLUGIN_SDK.md` — Developer-only
|
||||
- `docs/providers/ZED-DOCKER.md` — Provider-specific
|
||||
- `docs/AGENTROUTER.md` — Provider-specific
|
||||
- `docs/PROVIDERS.md` — Provider-specific
|
||||
- `docs/README.md` — Keep as-is
|
||||
- `docs/SUBMIT_PR.md` — Contributor-only
|
||||
- `docs/releases/v3.8.0.md` — Release notes
|
||||
- `docs/research/` — Internal research
|
||||
- `docs/specs/` — Internal specs
|
||||
- `docs/openspec/` — Internal specs
|
||||
- `docs/superpowers/` — Internal plans
|
||||
|
||||
---
|
||||
|
||||
## Phase 4: Update Main README.md
|
||||
|
||||
**Changes**:
|
||||
- Add "Quick Start" section (3 steps)
|
||||
- Add "Which `auto` should I use?" table
|
||||
- Add "Free providers" table
|
||||
- Add "Common questions" section
|
||||
- Link to user guides
|
||||
|
||||
---
|
||||
|
||||
## Execution Order
|
||||
|
||||
### Week 1: Phase 1 (User-Facing Quick Start Docs)
|
||||
1. `docs/guides/QUICK-START.md` — NEW
|
||||
2. `docs/guides/AUTO-COMBO-GUIDE.md` — NEW
|
||||
3. `docs/guides/PROVIDERS-GUIDE.md` — NEW
|
||||
4. `docs/guides/FREE-TIERS-GUIDE.md` — NEW
|
||||
5. `docs/guides/TROUBLESHOOTING.md` — REWRITE
|
||||
|
||||
### Week 2: Phase 2 (Refine Existing User Docs)
|
||||
1. `docs/guides/USER_GUIDE.md` — UPDATE
|
||||
2. `docs/guides/SETUP_GUIDE.md` — UPDATE
|
||||
3. `docs/guides/FEATURES.md` — UPDATE
|
||||
4. `docs/guides/DOCKER_GUIDE.md` — UPDATE
|
||||
5. `docs/guides/ELECTRON_GUIDE.md` — UPDATE
|
||||
6. `docs/guides/TERMUX_GUIDE.md` — UPDATE
|
||||
7. `docs/guides/PWA_GUIDE.md` — UPDATE
|
||||
8. `docs/guides/I18N.md` — UPDATE
|
||||
9. `docs/guides/KIRO_SETUP.md` — UPDATE
|
||||
10. `docs/guides/UNINSTALL.md` — UPDATE
|
||||
|
||||
### Week 3: Phase 3 (Technical Docs Cross-Links)
|
||||
1. `docs/architecture/` — Add cross-links
|
||||
2. `docs/frameworks/` — Add cross-links
|
||||
3. `docs/security/` — Add cross-links
|
||||
4. `docs/ops/` — Add cross-links
|
||||
5. `docs/compression/` — Add cross-links
|
||||
6. `docs/routing/` — Add cross-links
|
||||
7. `docs/reference/` — Add cross-links
|
||||
|
||||
### Week 4: Phase 4 (Update Main README.md)
|
||||
1. Update README.md with user-friendly sections
|
||||
|
||||
---
|
||||
|
||||
## Success Metrics
|
||||
|
||||
- [ ] User can install OmniRoute in < 5 minutes (QUICK-START.md)
|
||||
- [ ] User can connect a provider in < 3 minutes (PROVIDERS-GUIDE.md)
|
||||
- [ ] User can use auto-combo in < 1 minute (AUTO-COMBO-GUIDE.md)
|
||||
- [ ] User can find free providers in < 2 minutes (FREE-TIERS-GUIDE.md)
|
||||
- [ ] User can troubleshoot common issues in < 5 minutes (TROUBLESHOOTING.md)
|
||||
- [ ] All technical docs link to user guides
|
||||
- [ ] All user guides link to technical docs
|
||||
|
||||
---
|
||||
|
||||
## File Naming Convention
|
||||
|
||||
- User guides: `docs/guides/[FEATURE]-GUIDE.md` (e.g., `AUTO-COMBO-GUIDE.md`)
|
||||
- Technical docs: Keep existing names (e.g., `docs/routing/AUTO-COMBO.md`)
|
||||
- Cross-links: "For Users" callout at top of technical docs
|
||||
|
||||
---
|
||||
|
||||
## Template for User Guides
|
||||
|
||||
```markdown
|
||||
# [Feature Name]: [One-Line Description]
|
||||
|
||||
## What It Does
|
||||
[2-3 sentences explaining what the feature does in plain language]
|
||||
|
||||
## Quick Start
|
||||
[Step-by-step instructions to get started]
|
||||
|
||||
## [Main Section]
|
||||
[Detailed explanation with tables, examples, screenshots]
|
||||
|
||||
## Common Questions
|
||||
[FAQ section with common questions and answers]
|
||||
|
||||
## What's Next?
|
||||
[Links to related guides and technical docs]
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Template for Technical Doc Cross-Links
|
||||
|
||||
```markdown
|
||||
> **For Users**: Looking for a quick start? See the [User Guide](../guides/[FEATURE]-GUIDE.md).
|
||||
|
||||
> **Prerequisites**: [List prerequisites]
|
||||
```
|
||||
@@ -22,9 +22,19 @@ How the system is put together — read these to understand the runtime, code la
|
||||
- [AUTHZ_GUIDE.md](architecture/AUTHZ_GUIDE.md) — authorization pipeline (route classifier + policy engine).
|
||||
- [RESILIENCE_GUIDE.md](architecture/RESILIENCE_GUIDE.md) — provider circuit breaker, connection cooldown, and model lockout.
|
||||
|
||||
## guides/
|
||||
## For Non-Tech Users
|
||||
|
||||
Task-focused walkthroughs for operators and end users.
|
||||
Simple guides for using OmniRoute — no technical background needed.
|
||||
|
||||
### getting-started/
|
||||
|
||||
- [QUICK-START.md](getting-started/QUICK-START.md) — install and run OmniRoute in 3 minutes.
|
||||
- [AUTO-COMBO-GUIDE.md](getting-started/AUTO-COMBO-GUIDE.md) — let OmniRoute pick the best AI for you.
|
||||
- [PROVIDERS-GUIDE.md](getting-started/PROVIDERS-GUIDE.md) — how to connect AI providers.
|
||||
- [FREE-TIERS-GUIDE.md](getting-started/FREE-TIERS-GUIDE.md) — get free AI with no credit card.
|
||||
- [TROUBLESHOOTING.md](getting-started/TROUBLESHOOTING.md) — fix common issues.
|
||||
|
||||
### guides/
|
||||
|
||||
- [SETUP_GUIDE.md](guides/SETUP_GUIDE.md) — first-time setup of OmniRoute.
|
||||
- [USER_GUIDE.md](guides/USER_GUIDE.md) — daily usage of the dashboard and API.
|
||||
@@ -32,12 +42,15 @@ Task-focused walkthroughs for operators and end users.
|
||||
- [ELECTRON_GUIDE.md](guides/ELECTRON_GUIDE.md) — desktop (Electron) builds.
|
||||
- [TERMUX_GUIDE.md](guides/TERMUX_GUIDE.md) — running on Android via Termux.
|
||||
- [PWA_GUIDE.md](guides/PWA_GUIDE.md) — installing the dashboard as a PWA.
|
||||
- [TROUBLESHOOTING.md](guides/TROUBLESHOOTING.md) — common issues and fixes.
|
||||
- [UNINSTALL.md](guides/UNINSTALL.md) — clean removal steps.
|
||||
- [I18N.md](guides/I18N.md) — translation and locale workflow.
|
||||
- [FEATURES.md](guides/FEATURES.md) — dashboard feature gallery.
|
||||
|
||||
## reference/
|
||||
## For Tech Users
|
||||
|
||||
Technical documentation for developers and contributors.
|
||||
|
||||
## architecture/
|
||||
|
||||
Lookup material — API surface, environment variables, CLI flags, provider catalog.
|
||||
|
||||
|
||||
@@ -271,6 +271,206 @@ RTK mode is inspired by **[RTK - Rust Token Killer](https://github.com/rtk-ai/rt
|
||||
|
||||
---
|
||||
|
||||
## Advanced Compression Systems
|
||||
|
||||
Beyond the 7 standard modes, OmniRoute includes several advanced compression
|
||||
systems that work automatically based on context.
|
||||
|
||||
### Cache-Aware Compression
|
||||
|
||||
Some providers (like Anthropic with prompt caching) support **prompt caching**,
|
||||
which lets them cache parts of the prompt to reduce costs and latency. When
|
||||
caching is enabled, aggressive compression can actually **hurt** performance
|
||||
because it changes the cached tokens, invalidating the cache.
|
||||
|
||||
The `cachingAware.ts` module solves this by **detecting caching context** and
|
||||
**adjusting the compression strategy** accordingly.
|
||||
|
||||
#### How it works
|
||||
|
||||
1. **Detect caching context** — Scans the request body for `cache_control` markers
|
||||
2. **Identify caching providers** — Checks if the target provider supports caching
|
||||
3. **Adjust strategy** — Downgrades `aggressive`/`ultra` to `standard` for caching providers
|
||||
4. **Skip system prompt** — System prompts are usually cached, so don't compress them
|
||||
5. **Use deterministic transformations** — Only use transformations that produce consistent output
|
||||
|
||||
#### Code example
|
||||
|
||||
```ts
|
||||
import { detectCachingContext, getCacheAwareStrategy } from "@omniroute/open-sse/services/compression/cachingAware";
|
||||
|
||||
const body = {
|
||||
model: "anthropic/claude-sonnet-4.5",
|
||||
messages: [{ role: "user", content: "Hello" }],
|
||||
cache_control: { type: "ephemeral" }, // ← Cache marker
|
||||
};
|
||||
|
||||
const ctx = detectCachingContext(body, { provider: "anthropic" });
|
||||
// → { hasCacheControl: true, provider: "anthropic", isCachingProvider: true }
|
||||
|
||||
const strategy = getCacheAwareStrategy("aggressive", ctx);
|
||||
// → { strategy: "standard", skipSystemPrompt: true, deterministicOnly: true }
|
||||
```
|
||||
|
||||
#### When to use
|
||||
|
||||
Cache-aware compression is **always on** — no configuration needed. It only kicks in
|
||||
when:
|
||||
- The request has `cache_control` markers
|
||||
- The target provider supports prompt caching (Anthropic, OpenAI, etc.)
|
||||
|
||||
### Progressive Aging
|
||||
|
||||
Long conversations accumulate many message turns, but older turns become less
|
||||
relevant. The `progressiveAging.ts` module **degrades messages by turn distance**:
|
||||
|
||||
- **Recent turns (0-3)**: Kept verbatim (full detail)
|
||||
- **Medium turns (4-8)**: Lite compression (whitespace, formatting cleanup)
|
||||
- **Old turns (9+)**: Caveman compression (filler removal, summarization)
|
||||
- **Very old turns (20+)**: Heavily summarized or dropped
|
||||
|
||||
#### Code example
|
||||
|
||||
```ts
|
||||
import { applyAging } from "@omniroute/open-sse/services/compression/progressiveAging";
|
||||
|
||||
const messages = [
|
||||
{ role: "system", content: "You are a helpful assistant" },
|
||||
{ role: "user", content: "What is 2+2?" },
|
||||
{ role: "assistant", content: "4" },
|
||||
// ... 50 more turns ...
|
||||
];
|
||||
|
||||
const { messages: aged, saved } = applyAging(messages, {
|
||||
verbatim: 3, // First 3 turns: verbatim
|
||||
light: 8, // Turns 4-8: lite compression
|
||||
moderate: 20, // Turns 9-20: caveman compression
|
||||
// Turns 21+: heavy summarization
|
||||
});
|
||||
|
||||
// saved = number of tokens saved
|
||||
```
|
||||
|
||||
#### When to use
|
||||
|
||||
Progressive aging is **always on** for `aggressive` and `ultra` modes. It's
|
||||
particularly effective for:
|
||||
- Long-running coding sessions
|
||||
- Multi-day conversations
|
||||
- Agentic workflows with many tool calls
|
||||
|
||||
### Caveman Output Mode
|
||||
|
||||
The `outputMode.ts` module injects **system prompt instructions** to make the
|
||||
model itself produce compressed, terse output (a "caveman" style).
|
||||
|
||||
#### How it works
|
||||
|
||||
Instead of compressing the input, this mode adds a system prompt like:
|
||||
> "Reply in minimal words. Skip pleasantries. Use short sentences."
|
||||
|
||||
This works particularly well for:
|
||||
- Code generation (terser output = fewer tokens)
|
||||
- Quick Q&A (no need for elaborate explanations)
|
||||
- Batch processing (maximize throughput)
|
||||
|
||||
#### When to use
|
||||
|
||||
Caveman output mode is **opt-in** — set it via the combo config:
|
||||
```json
|
||||
{
|
||||
"strategy": "auto",
|
||||
"config": {
|
||||
"auto": {
|
||||
"outputMode": "caveman"
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Tool Result Compression
|
||||
|
||||
The `toolResultCompressor.ts` module provides **5 specialized compression strategies**
|
||||
for tool results (function calls, agent outputs, search results, etc.):
|
||||
|
||||
1. **Search result compression** — Removes redundant results, keeps top-N
|
||||
2. **File read compression** — Truncates large files, preserves headers/imports
|
||||
3. **Code execution compression** — Keeps only essential stdout/stderr
|
||||
4. **Database query compression** — Limits rows, removes verbose metadata
|
||||
5. **API response compression** — Strips null fields, condenses arrays
|
||||
|
||||
#### When to use
|
||||
|
||||
Tool result compression is **always on** when tool calls are present. No
|
||||
configuration needed.
|
||||
|
||||
### Stacked Pipeline
|
||||
|
||||
The stacked mode runs **multiple engines in sequence** — usually RTK first
|
||||
(60-90% savings on tool output), then Caveman (30% additional savings on the
|
||||
remaining text). This achieves **78-95% total savings**.
|
||||
|
||||
#### How it works
|
||||
|
||||
```
|
||||
Input (1000 tokens)
|
||||
→ RTK (command-aware filter) → 200 tokens
|
||||
→ Caveman (filler removal) → 140 tokens
|
||||
→ Output (140 tokens, 86% savings)
|
||||
```
|
||||
|
||||
#### When to use
|
||||
|
||||
Use stacked mode for:
|
||||
- Tool-heavy workflows (agentic coding, research)
|
||||
- Cost-sensitive batch processing
|
||||
- When you need maximum token savings
|
||||
|
||||
Configure via combo:
|
||||
```json
|
||||
{
|
||||
"strategy": "auto",
|
||||
"config": {
|
||||
"auto": {
|
||||
"modePack": "stacked"
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Compression Combo Overrides
|
||||
|
||||
You can override the global compression mode **per combo** to fine-tune behavior
|
||||
for different use cases:
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "coding-combo",
|
||||
"strategy": "priority",
|
||||
"config": {
|
||||
"auto": {
|
||||
"weights": { "taskFit": 0.5 },
|
||||
"modePack": "quality-first"
|
||||
}
|
||||
},
|
||||
"compressionOverride": {
|
||||
"mode": "aggressive",
|
||||
"stackedPipelines": ["rtk", "caveman"],
|
||||
"preserveToolDefinitions": true
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
This is useful for:
|
||||
- **Coding combos**: Use `aggressive` mode for long sessions
|
||||
- **Quick Q&A combos**: Use `lite` mode for fast responses
|
||||
- **Tool-heavy combos**: Use `stacked` mode for max savings
|
||||
- **Production combos**: Use `cache-aware` mode for caching providers
|
||||
|
||||
---
|
||||
|
||||
## See Also
|
||||
|
||||
- [Environment Config](../reference/ENVIRONMENT.md) — Compression environment variables
|
||||
|
||||
560
docs/frameworks/ACP.md
Normal file
560
docs/frameworks/ACP.md
Normal file
@@ -0,0 +1,560 @@
|
||||
# ACP (Agent Client Protocol)
|
||||
|
||||
> **TL;DR**: ACP lets OmniRoute spawn CLI agents (like Claude Code, Codex, Gemini CLI) as child processes instead of using HTTP APIs. This gives you "CLI-as-backend" transport.
|
||||
|
||||
---
|
||||
|
||||
## What Is ACP?
|
||||
|
||||
ACP (Agent Client Protocol) is a **"CLI-as-backend" transport** for OmniRoute. Instead of intercepting HTTP API calls to AI providers, ACP **spawns CLI agents as child processes** and feeds prompts through their native interface.
|
||||
|
||||
### Why Use ACP?
|
||||
|
||||
| Benefit | Description |
|
||||
|---------|-------------|
|
||||
| **No API keys needed** | Uses your existing CLI authentication |
|
||||
| **Native protocol** | Uses each CLI's native input/output format |
|
||||
| **Auto-discovery** | Detects installed CLIs on your system |
|
||||
| **14 built-in agents** | Pre-configured for popular CLI tools |
|
||||
| **Custom agents** | Add your own CLI tools via settings |
|
||||
| **Process management** | Handles lifecycle (spawn, send, kill) |
|
||||
|
||||
---
|
||||
|
||||
## Supported CLI Agents
|
||||
|
||||
ACP supports **14 built-in CLI agents** out of the box:
|
||||
|
||||
| Agent ID | Display Name | Binary | Protocol |
|
||||
|----------|--------------|--------|----------|
|
||||
| `codex` | OpenAI Codex CLI | `codex` | stdio |
|
||||
| `claude` | Claude Code CLI | `claude` | stdio |
|
||||
| `goose` | Goose CLI | `goose` | stdio |
|
||||
| `gemini-cli` | Gemini CLI | `gemini` | stdio |
|
||||
| `openclaw` | OpenClaw | `openclaw` | stdio |
|
||||
| `aider` | Aider | `aider` | stdio |
|
||||
| `opencode` | OpenCode | `opencode` | stdio |
|
||||
| `cline` | Cline | `cline` | stdio |
|
||||
| `qwen-code` | Qwen Code | `qwen` | stdio |
|
||||
| `forge` | ForgeCode | `forge` | stdio |
|
||||
| `amazon-q` | Amazon Q Developer | `q` | stdio |
|
||||
| `interpreter` | Open Interpreter | `interpreter` | stdio |
|
||||
| `cursor-cli` | Cursor CLI | `cursor` | stdio |
|
||||
| `warp` | Warp AI | `warp` | stdio |
|
||||
|
||||
### Custom Agents
|
||||
|
||||
You can add your own CLI agents via settings. Custom agents support the same features as built-in agents.
|
||||
|
||||
---
|
||||
|
||||
## Quick Start
|
||||
|
||||
### Step 1: Install a CLI Agent
|
||||
|
||||
```bash
|
||||
# Example: Install Claude Code CLI
|
||||
npm install -g @anthropic-ai/claude-code
|
||||
|
||||
# Verify installation
|
||||
claude --version
|
||||
```
|
||||
|
||||
### Step 2: ACP Auto-Detection
|
||||
|
||||
ACP automatically detects installed CLI agents on your system. No configuration needed!
|
||||
|
||||
### Step 3: Use ACP Transport
|
||||
|
||||
Once detected, ACP can be used as a transport for any supported provider. OmniRoute will automatically use ACP when the CLI is available.
|
||||
|
||||
---
|
||||
|
||||
## How ACP Works
|
||||
|
||||
### Architecture
|
||||
|
||||
```
|
||||
┌─────────────────┐
|
||||
│ OmniRoute │
|
||||
│ (HTTP Proxy) │
|
||||
└────────┬────────┘
|
||||
│
|
||||
│ spawn()
|
||||
▼
|
||||
┌─────────────────┐
|
||||
│ Child Process │
|
||||
│ (CLI Agent) │
|
||||
│ │
|
||||
│ stdin ◄──────┤ Send prompt
|
||||
│ stdout ──────►│ Receive response
|
||||
│ stderr ──────►│ Receive errors
|
||||
└─────────────────┘
|
||||
```
|
||||
|
||||
### Process Lifecycle
|
||||
|
||||
1. **Spawn** — ACP creates a child process for the CLI agent
|
||||
2. **Send** — ACP writes prompts to the process's stdin
|
||||
3. **Receive** — ACP reads responses from stdout/stderr
|
||||
4. **Idle Detection** — ACP waits 2 seconds of inactivity before considering the response complete
|
||||
5. **Kill** — ACP terminates the process (SIGTERM, then SIGKILL after 5s)
|
||||
|
||||
### Communication Protocol
|
||||
|
||||
ACP uses **stdio** (standard input/output) for communication with CLI agents. The protocol is:
|
||||
|
||||
1. **Send prompt** — Write to stdin with a newline
|
||||
2. **Wait for response** — Read from stdout until idle (2s of no output)
|
||||
3. **Timeout** — Default 120 seconds (configurable)
|
||||
|
||||
---
|
||||
|
||||
## API Reference
|
||||
|
||||
### Registry Functions
|
||||
|
||||
#### `detectInstalledAgents()`
|
||||
|
||||
Detects all installed CLI agents on the system. Results are cached for 60 seconds.
|
||||
|
||||
```typescript
|
||||
import { detectInstalledAgents } from "@/lib/acp";
|
||||
|
||||
const agents = detectInstalledAgents();
|
||||
// Returns: CliAgentInfo[]
|
||||
|
||||
interface CliAgentInfo {
|
||||
id: string; // e.g., "codex", "claude"
|
||||
name: string; // Display name
|
||||
binary: string; // Binary name to spawn
|
||||
versionCommand: string; // Version detection command
|
||||
version: string | null; // Detected version (null if not installed)
|
||||
installed: boolean; // Whether the agent is installed
|
||||
providerAlias: string; // Provider ID in OmniRoute
|
||||
spawnArgs: string[]; // Arguments to pass when spawning
|
||||
protocol: "stdio" | "http"; // Communication protocol
|
||||
isCustom?: boolean; // Whether this is a user-defined custom agent
|
||||
}
|
||||
```
|
||||
|
||||
#### `getAvailableAgents()`
|
||||
|
||||
Gets only the agents that are installed and available for ACP.
|
||||
|
||||
```typescript
|
||||
import { getAvailableAgents } from "@/lib/acp";
|
||||
|
||||
const available = getAvailableAgents();
|
||||
// Returns: CliAgentInfo[] (only installed agents)
|
||||
```
|
||||
|
||||
#### `getAgentById(id)`
|
||||
|
||||
Gets a specific agent by ID.
|
||||
|
||||
```typescript
|
||||
import { getAgentById } from "@/lib/acp";
|
||||
|
||||
const agent = getAgentById("claude");
|
||||
// Returns: CliAgentInfo | undefined
|
||||
```
|
||||
|
||||
#### `setCustomAgents(agents)`
|
||||
|
||||
Sets custom agent definitions from settings.
|
||||
|
||||
```typescript
|
||||
import { setCustomAgents } from "@/lib/acp";
|
||||
|
||||
setCustomAgents([
|
||||
{
|
||||
id: "my-custom-cli",
|
||||
name: "My Custom CLI",
|
||||
binary: "mycli",
|
||||
versionCommand: "mycli --version",
|
||||
providerAlias: "my-provider",
|
||||
spawnArgs: [],
|
||||
protocol: "stdio",
|
||||
},
|
||||
]);
|
||||
```
|
||||
|
||||
### Manager Functions
|
||||
|
||||
#### `acpManager.spawn(agentId, binary, args, env)`
|
||||
|
||||
Spawns a new CLI agent process.
|
||||
|
||||
```typescript
|
||||
import { acpManager } from "@/lib/acp";
|
||||
|
||||
const session = acpManager.spawn(
|
||||
"claude",
|
||||
"claude",
|
||||
["--print", "--output-format", "json"],
|
||||
{ /* custom env vars */ }
|
||||
);
|
||||
// Returns: AcpSession
|
||||
```
|
||||
|
||||
**Allowed agent IDs**: `["claude", "codex", "gemini", "qwen"]`
|
||||
|
||||
#### `acpManager.sendPrompt(sessionId, prompt, timeoutMs)`
|
||||
|
||||
Sends a prompt to a CLI agent and collects the response.
|
||||
|
||||
```typescript
|
||||
import { acpManager } from "@/lib/acp";
|
||||
|
||||
const response = await acpManager.sendPrompt(
|
||||
"acp-claude-1234567890-abc123",
|
||||
"What is 2+2?",
|
||||
120000 // 2 minutes timeout
|
||||
);
|
||||
// Returns: Promise<string>
|
||||
```
|
||||
|
||||
#### `acpManager.kill(sessionId)`
|
||||
|
||||
Kills a session and cleans up.
|
||||
|
||||
```typescript
|
||||
import { acpManager } from "@/lib/acp";
|
||||
|
||||
const killed = acpManager.kill("acp-claude-1234567890-abc123");
|
||||
// Returns: boolean
|
||||
```
|
||||
|
||||
#### `acpManager.getActiveSessions()`
|
||||
|
||||
Gets all active sessions.
|
||||
|
||||
```typescript
|
||||
import { acpManager } from "@/lib/acp";
|
||||
|
||||
const sessions = acpManager.getActiveSessions();
|
||||
// Returns: AcpSession[]
|
||||
```
|
||||
|
||||
#### `acpManager.killAll()`
|
||||
|
||||
Kills all sessions.
|
||||
|
||||
```typescript
|
||||
import { acpManager } from "@/lib/acp";
|
||||
|
||||
acpManager.killAll();
|
||||
```
|
||||
|
||||
### Session Interface
|
||||
|
||||
```typescript
|
||||
interface AcpSession {
|
||||
id: string; // Unique session ID
|
||||
agentId: string; // Agent ID (e.g., "claude")
|
||||
process: ChildProcess; // Child process handle
|
||||
alive: boolean; // Whether the process is alive
|
||||
stdoutBuffer: string; // Accumulated stdout buffer
|
||||
stderrBuffer: string; // Accumulated stderr buffer
|
||||
createdAt: Date; // Created timestamp
|
||||
}
|
||||
```
|
||||
|
||||
### Events
|
||||
|
||||
The `AcpManager` extends `EventEmitter` and emits the following events:
|
||||
|
||||
#### `stdout`
|
||||
|
||||
Emitted when the CLI agent writes to stdout.
|
||||
|
||||
```typescript
|
||||
acpManager.on("stdout", ({ sessionId, data }) => {
|
||||
console.log(`[${sessionId}] stdout: ${data}`);
|
||||
});
|
||||
```
|
||||
|
||||
#### `stderr`
|
||||
|
||||
Emitted when the CLI agent writes to stderr.
|
||||
|
||||
```typescript
|
||||
acpManager.on("stderr", ({ sessionId, data }) => {
|
||||
console.error(`[${sessionId}] stderr: ${data}`);
|
||||
});
|
||||
```
|
||||
|
||||
#### `exit`
|
||||
|
||||
Emitted when the CLI agent process exits.
|
||||
|
||||
```typescript
|
||||
acpManager.on("exit", ({ sessionId, code, signal }) => {
|
||||
console.log(`[${sessionId}] exited with code ${code}, signal ${signal}`);
|
||||
});
|
||||
```
|
||||
|
||||
#### `error`
|
||||
|
||||
Emitted when the CLI agent process errors.
|
||||
|
||||
```typescript
|
||||
acpManager.on("error", ({ sessionId, error }) => {
|
||||
console.error(`[${sessionId}] error: ${error}`);
|
||||
});
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Configuration
|
||||
|
||||
### Environment Variables
|
||||
|
||||
ACP inherits all environment variables from the parent process and can be extended with custom env vars:
|
||||
|
||||
```typescript
|
||||
acpManager.spawn("claude", "claude", [], {
|
||||
ANTHROPIC_API_KEY: "sk-...",
|
||||
DEBUG: "true",
|
||||
});
|
||||
```
|
||||
|
||||
### Spawn Arguments
|
||||
|
||||
Each agent has default spawn arguments defined in the registry. You can override them:
|
||||
|
||||
```typescript
|
||||
acpManager.spawn("claude", "claude", ["--print", "--verbose"], {});
|
||||
```
|
||||
|
||||
### Timeouts
|
||||
|
||||
Default prompt timeout is **120 seconds** (2 minutes). You can override:
|
||||
|
||||
```typescript
|
||||
await acpManager.sendPrompt(sessionId, prompt, 300000); // 5 minutes
|
||||
```
|
||||
|
||||
### Detection Cache
|
||||
|
||||
Agent detection is cached for **60 seconds** to avoid expensive filesystem scans. Force refresh:
|
||||
|
||||
```typescript
|
||||
import { refreshAgentCache } from "@/lib/acp";
|
||||
|
||||
refreshAgentCache();
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Security
|
||||
|
||||
### Command Injection Prevention
|
||||
|
||||
ACP validates version commands to prevent command injection attacks:
|
||||
|
||||
```typescript
|
||||
const DISALLOWED_VERSION_COMMAND_CHARS = /[;&|<>`$\r\n]/;
|
||||
```
|
||||
|
||||
Version commands containing these characters are rejected:
|
||||
|
||||
- `;` — Command separator
|
||||
- `&` — Background process
|
||||
- `|` — Pipe
|
||||
- `<`, `>` — Redirection
|
||||
- `` ` `` — Command substitution
|
||||
- `$` — Variable expansion
|
||||
- `\r`, `\n` — Line breaks
|
||||
|
||||
### Binary Name Validation
|
||||
|
||||
ACP validates that the version command binary matches the expected binary name (unless it's a custom agent).
|
||||
|
||||
### Process Isolation
|
||||
|
||||
Each ACP session runs in its own child process. The process is killed when the session ends or times out.
|
||||
|
||||
---
|
||||
|
||||
## Performance
|
||||
|
||||
### Detection Performance
|
||||
|
||||
- **First call**: ~50-200ms (runs `version` command for each agent)
|
||||
- **Cached calls**: <1ms (returns from cache)
|
||||
- **Cache TTL**: 60 seconds
|
||||
|
||||
### Prompt Performance
|
||||
|
||||
- **Spawn**: ~50-100ms
|
||||
- **Send prompt**: ~10-50ms
|
||||
- **Wait for response**: Depends on CLI agent (typically 1-30 seconds)
|
||||
- **Kill**: ~5 seconds (SIGTERM) + immediate (SIGKILL)
|
||||
|
||||
### Resource Usage
|
||||
|
||||
- **Memory per session**: ~10-50MB (depends on CLI agent)
|
||||
- **CPU**: Minimal (I/O bound)
|
||||
- **Disk**: None
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### "Unknown agent" Error
|
||||
|
||||
**Problem**: `acpManager.spawn()` throws `Unknown agent: <id>`
|
||||
|
||||
**Solution**: Only 4 agents are allowed in `spawn()`:
|
||||
- `claude`
|
||||
- `codex`
|
||||
- `gemini`
|
||||
- `qwen`
|
||||
|
||||
Other agents must be spawned manually or via custom agent definitions.
|
||||
|
||||
### "Session not alive" Error
|
||||
|
||||
**Problem**: `acpManager.sendPrompt()` throws `Session ${sessionId} is not alive`
|
||||
|
||||
**Solution**: The session may have exited or been killed. Check session status:
|
||||
|
||||
```typescript
|
||||
const session = acpManager.getSession(sessionId);
|
||||
if (!session?.alive) {
|
||||
// Re-spawn the session
|
||||
acpManager.spawn("claude", "claude", [], {});
|
||||
}
|
||||
```
|
||||
|
||||
### "ACP timeout" Error
|
||||
|
||||
**Problem**: `acpManager.sendPrompt()` throws `ACP timeout after 120000ms`
|
||||
|
||||
**Solution**: Increase the timeout:
|
||||
|
||||
```typescript
|
||||
await acpManager.sendPrompt(sessionId, prompt, 300000); // 5 minutes
|
||||
```
|
||||
|
||||
### CLI Not Detected
|
||||
|
||||
**Problem**: `detectInstalledAgents()` doesn't find your CLI
|
||||
|
||||
**Solutions**:
|
||||
1. **Check PATH**: Ensure the CLI is in your system PATH
|
||||
2. **Check version command**: Run `claude --version` manually
|
||||
3. **Check permissions**: Ensure the CLI is executable
|
||||
4. **Custom agent**: Add a custom agent definition for non-standard CLIs
|
||||
|
||||
### Permission Denied
|
||||
|
||||
**Problem**: ACP can't execute the CLI
|
||||
|
||||
**Solutions**:
|
||||
1. **Check file permissions**: `chmod +x /usr/local/bin/claude`
|
||||
2. **Check ownership**: Ensure OmniRoute has read/execute permissions
|
||||
3. **Check SELinux/AppArmor**: May block process spawning
|
||||
|
||||
---
|
||||
|
||||
## Examples
|
||||
|
||||
### Example 1: Spawn and Use Claude Code
|
||||
|
||||
```typescript
|
||||
import { acpManager, detectInstalledAgents } from "@/lib/acp";
|
||||
|
||||
// Detect installed agents
|
||||
const agents = detectInstalledAgents();
|
||||
const claude = agents.find((a) => a.id === "claude");
|
||||
|
||||
if (claude?.installed) {
|
||||
// Spawn a new session
|
||||
const session = acpManager.spawn(
|
||||
"claude",
|
||||
claude.binary,
|
||||
["--print", "--output-format", "json"]
|
||||
);
|
||||
|
||||
// Send a prompt
|
||||
const response = await acpManager.sendPrompt(
|
||||
session.id,
|
||||
"Explain quantum computing in 100 words"
|
||||
);
|
||||
|
||||
console.log("Claude's response:", response);
|
||||
|
||||
// Clean up
|
||||
acpManager.kill(session.id);
|
||||
}
|
||||
```
|
||||
|
||||
### Example 2: Auto-Discovery with Fallback
|
||||
|
||||
```typescript
|
||||
import { acpManager, getAvailableAgents } from "@/lib/acp";
|
||||
|
||||
const available = getAvailableAgents();
|
||||
|
||||
// Try Claude first, fallback to Codex
|
||||
let agentId = "claude";
|
||||
if (!available.find((a) => a.id === "claude")) {
|
||||
if (available.find((a) => a.id === "codex")) {
|
||||
agentId = "codex";
|
||||
} else {
|
||||
throw new Error("No ACP-compatible CLI agent found");
|
||||
}
|
||||
}
|
||||
|
||||
const agent = available.find((a) => a.id === agentId)!;
|
||||
const session = acpManager.spawn(agentId, agent.binary, agent.spawnArgs);
|
||||
|
||||
const response = await acpManager.sendPrompt(session.id, "Hello!");
|
||||
|
||||
acpManager.kill(session.id);
|
||||
```
|
||||
|
||||
### Example 3: Custom Agent
|
||||
|
||||
```typescript
|
||||
import { setCustomAgents, detectInstalledAgents } from "@/lib/acp";
|
||||
|
||||
// Register a custom CLI agent
|
||||
setCustomAgents([
|
||||
{
|
||||
id: "my-llm-cli",
|
||||
name: "My LLM CLI",
|
||||
binary: "myllm",
|
||||
versionCommand: "myllm --version",
|
||||
providerAlias: "my-llm-provider",
|
||||
spawnArgs: ["--format", "json"],
|
||||
protocol: "stdio",
|
||||
},
|
||||
]);
|
||||
|
||||
// Now detectInstalledAgents() will include "my-llm-cli"
|
||||
const agents = detectInstalledAgents();
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## What's Next?
|
||||
|
||||
- **[API Reference](../reference/API_REFERENCE.md)** — REST API endpoints
|
||||
- **[Provider Reference](../reference/PROVIDER_REFERENCE.md)** — All 223 providers
|
||||
- **[MCP Server](./MCP-SERVER.md)** — Model Context Protocol integration
|
||||
- **[A2A Server](./A2A-SERVER.md)** — Agent-to-Agent protocol
|
||||
- **[Cloud Agent](./CLOUD_AGENT.md)** — Cloud-based agents
|
||||
|
||||
---
|
||||
|
||||
## Reference
|
||||
|
||||
- [AionUi Project](https://github.com/iOfficeAI/AionUi) — Inspiration for ACP auto-detection
|
||||
- [ACP Source Code](../../src/lib/acp/) — Implementation details
|
||||
- `manager.ts` — Process lifecycle management
|
||||
- `registry.ts` — Agent discovery and registration
|
||||
- `index.ts` — Public API exports
|
||||
@@ -4,6 +4,7 @@
|
||||
"MCP-SERVER",
|
||||
"A2A-SERVER",
|
||||
"AGENT_PROTOCOLS_GUIDE",
|
||||
"ACP",
|
||||
"CLOUD_AGENT",
|
||||
"EVALS",
|
||||
"GAMIFICATION",
|
||||
|
||||
210
docs/getting-started/AUTO-COMBO-GUIDE.md
Normal file
210
docs/getting-started/AUTO-COMBO-GUIDE.md
Normal file
@@ -0,0 +1,210 @@
|
||||
# Auto-Combo: Let OmniRoute Pick the Best AI for You
|
||||
|
||||
> **TL;DR**: Set your model to `auto` and OmniRoute automatically picks the best AI provider for each request. No configuration needed.
|
||||
|
||||
---
|
||||
|
||||
## What It Does
|
||||
|
||||
Instead of choosing a specific AI model (like GPT-4o or Claude), you can let OmniRoute **automatically pick the best one** for each request. It considers:
|
||||
|
||||
- **Health** — Is the provider working right now?
|
||||
- **Speed** — How fast is it?
|
||||
- **Cost** — How much does it cost?
|
||||
- **Quality** — Is it good at this type of task?
|
||||
- **Capacity** — Does it have quota remaining?
|
||||
|
||||
OmniRoute scores all your connected providers and picks the best one. If it fails, it automatically tries the next one.
|
||||
|
||||
---
|
||||
|
||||
## Quick Start
|
||||
|
||||
**Step 1**: Set your model to `auto` in your IDE or CLI:
|
||||
|
||||
```
|
||||
model: "auto"
|
||||
```
|
||||
|
||||
**Step 2**: That's it! OmniRoute handles the rest.
|
||||
|
||||
**Step 3** (optional): Use a variant for specific tasks:
|
||||
|
||||
```
|
||||
model: "auto/coding" # Best for code
|
||||
model: "auto/fast" # Fastest response
|
||||
model: "auto/cheap" # Cheapest option
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Which "auto" Should I Use?
|
||||
|
||||
| If you want... | Use this | Best for | How it works |
|
||||
|----------------|----------|----------|--------------|
|
||||
| **Best overall** | `auto` | General questions, chat | Balances speed, cost, and quality |
|
||||
| **Best code** | `auto/coding` | Writing code, debugging | Picks models good at coding tasks |
|
||||
| **Fastest response** | `auto/fast` | Quick answers, low latency | Prioritizes speed over everything |
|
||||
| **Cheapest option** | `auto/cheap` | Saving money | Picks the cheapest provider |
|
||||
| **Smartest model** | `auto/smart` | Complex tasks | Quality-first + explores new models |
|
||||
| **Most available** | `auto/offline` | When providers are busy | Picks providers with most capacity |
|
||||
|
||||
### Examples
|
||||
|
||||
```bash
|
||||
# General chat — balanced
|
||||
curl http://localhost:20128/v1/chat/completions \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"model":"auto","messages":[{"role":"user","content":"Hello!"}]}'
|
||||
|
||||
# Code generation — quality-first
|
||||
curl http://localhost:20128/v1/chat/completions \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"model":"auto/coding","messages":[{"role":"user","content":"Write a Python function"}]}'
|
||||
|
||||
# Quick answer — speed-first
|
||||
curl http://localhost:20128/v1/chat/completions \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"model":"auto/fast","messages":[{"role":"user","content":"What is 2+2?"}]}'
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## How It Works (Simple Version)
|
||||
|
||||
When you send a request with `model: "auto"`, OmniRoute:
|
||||
|
||||
1. **Looks at all your connected providers** — Every provider you've added (OpenAI, Anthropic, Google, etc.)
|
||||
2. **Scores each one** on 5 factors:
|
||||
- Is it working? (health)
|
||||
- Does it have capacity? (quota)
|
||||
- How much does it cost? (price)
|
||||
- How fast is it? (speed)
|
||||
- Is it good at this task? (quality)
|
||||
3. **Picks the best one** — The highest-scoring provider gets your request
|
||||
4. **Auto-recovers** — If it fails, OmniRoute tries the next one automatically
|
||||
|
||||
### The Scoring System
|
||||
|
||||
Each provider gets a score from 0 to 1. The higher the score, the better the fit.
|
||||
|
||||
| Factor | Weight | What it means |
|
||||
|--------|--------|---------------|
|
||||
| Health | 20% | Is the provider working? (circuit breaker state) |
|
||||
| Quota | 15% | Does it have capacity remaining? |
|
||||
| Cost | 15% | How expensive is it? (cheaper = higher score) |
|
||||
| Speed | 12% | How fast is it? (lower latency = higher score) |
|
||||
| Task Fit | 8% | Is it good at this type of task? |
|
||||
| Stability | 5% | Is it consistent? (low error rate) |
|
||||
| Tier | 5% | Account tier (Ultra > Pro > Free) |
|
||||
| Other | 20% | Context affinity, connection density, etc. |
|
||||
|
||||
### How Variants Change the Scoring
|
||||
|
||||
Each variant uses different weights:
|
||||
|
||||
| Variant | Prioritizes | Key Weights |
|
||||
|---------|-------------|-------------|
|
||||
| `auto` | Balanced | health=20%, quota=15%, cost=15% |
|
||||
| `auto/coding` | Quality | taskFit=37%, stability=15% |
|
||||
| `auto/fast` | Speed | latency=32%, health=28% |
|
||||
| `auto/cheap` | Cost | cost=37% |
|
||||
| `auto/smart` | Quality + Explore | taskFit=37%, exploration=10% |
|
||||
| `auto/offline` | Capacity | quota=37%, health=28% |
|
||||
|
||||
---
|
||||
|
||||
## How It Handles Failures
|
||||
|
||||
OmniRoute has **three layers of protection**:
|
||||
|
||||
### 1. Auto-Fallback
|
||||
If the best provider fails, OmniRoute automatically tries the next one. You don't need to do anything.
|
||||
|
||||
### 2. Self-Healing
|
||||
If a provider keeps failing:
|
||||
- **Score < 0.2** → Excluded for 5 minutes
|
||||
- **Circuit breaker open** → Auto-excluded
|
||||
- **More than 50% providers down** → Incident mode (no exploration)
|
||||
|
||||
### 3. Emergency Fallback
|
||||
If all providers fail, OmniRoute routes to stable free providers (like Kiro or Qoder) as a last resort.
|
||||
|
||||
---
|
||||
|
||||
## Multi-Account Support
|
||||
|
||||
If you have multiple accounts for the same provider (e.g., two OpenAI keys), OmniRoute treats each as a **separate candidate**. This means:
|
||||
|
||||
- Account A has quota remaining → use it
|
||||
- Account B is rate-limited → skip it
|
||||
- Account C is cheaper → prefer it
|
||||
|
||||
Each account is scored independently based on its own health, quota, and speed.
|
||||
|
||||
---
|
||||
|
||||
## Bandit Exploration
|
||||
|
||||
OmniRoute occasionally **explores** new providers to discover better options:
|
||||
|
||||
- **Default**: 5% of requests go to random providers
|
||||
- **Auto/smart**: 10% exploration rate
|
||||
- **Disabled** when more than 50% of providers are unhealthy
|
||||
|
||||
This helps OmniRoute learn which providers work best for your usage patterns.
|
||||
|
||||
---
|
||||
|
||||
## Common Questions
|
||||
|
||||
### "Will it always pick the most expensive model?"
|
||||
|
||||
**No.** Cost is only 15% of the score by default. A cheap, fast, healthy provider can beat an expensive one. Use `auto/cheap` if you want to prioritize cost even more.
|
||||
|
||||
### "What if a provider goes down?"
|
||||
|
||||
OmniRoute automatically skips it and tries the next one. If a provider keeps failing, it's excluded temporarily (5-30 minutes). You don't need to do anything.
|
||||
|
||||
### "Can I see which provider was used?"
|
||||
|
||||
Check the response headers — OmniRoute includes the provider and model used in each response.
|
||||
|
||||
### "Does it learn from my usage?"
|
||||
|
||||
Yes! The scoring system uses historical data (latency, error rates, success rates) to make better decisions over time.
|
||||
|
||||
### "What's the difference between `auto` and `auto/smart`?"
|
||||
|
||||
- `auto` — Balanced, 5% exploration
|
||||
- `auto/smart` — Quality-first (same weights as `auto/coding`), 10% exploration
|
||||
|
||||
Use `auto/smart` when you want the best quality and are okay with occasional exploration.
|
||||
|
||||
### "Can I force a specific provider?"
|
||||
|
||||
Yes! Use a combo with `priority` strategy instead of `auto`. See the [Technical Reference](../routing/AUTO-COMBO.md) for details.
|
||||
|
||||
### "How is this different from round-robin?"
|
||||
|
||||
Round-robin cycles through providers in order. Auto-combo **scores each provider** and picks the best one. It's smarter — it considers health, speed, cost, and quality.
|
||||
|
||||
---
|
||||
|
||||
## What's Next?
|
||||
|
||||
- **[Connect a Provider](./PROVIDERS-GUIDE.md)** — Add your first AI provider
|
||||
- **[Free Tiers Guide](./FREE-TIERS-GUIDE.md)** — Get free AI with no credit card
|
||||
- **[Troubleshooting](./TROUBLESHOOTING.md)** — Fix common issues
|
||||
- **[Technical Reference](../routing/AUTO-COMBO.md)** — Deep dive into the scoring algorithm
|
||||
|
||||
---
|
||||
|
||||
## Learn More
|
||||
|
||||
For developers and contributors, see the [Auto-Combo Technical Reference](../routing/AUTO-COMBO.md) for:
|
||||
- Full 12-factor scoring algorithm
|
||||
- Mode pack weight tables
|
||||
- Implementation file paths
|
||||
- API endpoints
|
||||
- Self-healing algorithm details
|
||||
270
docs/getting-started/FREE-TIERS-GUIDE.md
Normal file
270
docs/getting-started/FREE-TIERS-GUIDE.md
Normal file
@@ -0,0 +1,270 @@
|
||||
# Free Tiers Guide: Get Free AI Without a Credit Card
|
||||
|
||||
> **TL;DR**: OmniRoute aggregates free tiers from 50+ providers. Connect multiple free providers for unlimited free AI with automatic fallback.
|
||||
|
||||
---
|
||||
|
||||
## What Are Free Tiers?
|
||||
|
||||
Many AI providers offer **free usage** — no credit card required. Think of it like free samples at a grocery store. You can try the product without paying.
|
||||
|
||||
OmniRoute **aggregates** these free tiers into one endpoint. Instead of signing up for 10 different services, you connect them all to OmniRoute and use `model: "auto"` to automatically pick the best free option for each request.
|
||||
|
||||
---
|
||||
|
||||
## Best Free Providers (No Credit Card)
|
||||
|
||||
### Tier 1: Free Forever (Unlimited)
|
||||
|
||||
These providers are **always free** with no limits:
|
||||
|
||||
| Provider | Models | Quota | How to Connect |
|
||||
|----------|--------|-------|----------------|
|
||||
| **Kiro AI** | Claude Sonnet 4.5, Haiku 4.5, Opus 4.6 | 50 credits/month | No auth needed |
|
||||
| **OpenCode Free** | GPT-4o, Claude, Gemini | Unlimited | No auth needed |
|
||||
| **Pollinations** | GPT-5, Claude, Gemini, DeepSeek, Llama 4 | No key needed | No auth needed |
|
||||
| **LongCat** | LongCat-Flash-Lite | 50M tokens/day | No auth needed |
|
||||
| **Cloudflare AI** | 50+ models | 10K neurons/day | No auth needed |
|
||||
| **Qwen** | Qwen3-coder-plus/flash/next | Unlimited | No auth needed |
|
||||
| **Qoder** | Kimi-K2, DeepSeek-R1, Qwen3-coder | Unlimited | No auth needed |
|
||||
|
||||
### Tier 2: Free with Signup (Generous)
|
||||
|
||||
These providers give you **free credits** when you sign up:
|
||||
|
||||
| Provider | Free Credits | Models | How to Get |
|
||||
|----------|-------------|--------|------------|
|
||||
| **NVIDIA NIM** | ~40 RPM | 129 models | Sign up at build.nvidia.com |
|
||||
| **Cerebras** | 1M tokens/day | Qwen3 235B, GPT-OSS 120B | Sign up at cerebras.ai |
|
||||
| **DeepSeek** | 5M free tokens | DeepSeek V4 | Sign up at platform.deepseek.com |
|
||||
| **Groq** | 30 RPM free | Llama 4, Mixtral | Sign up at console.groq.com |
|
||||
| **OpenAI** | $5 free credits | GPT-5, GPT-4o | Sign up at platform.openai.com |
|
||||
| **Anthropic** | $5 free credits | Claude Opus 4.6, Sonnet 4.6 | Sign up at console.anthropic.com |
|
||||
| **Google** | 1,500 req/day | Gemini 2.5 Pro, Flash | Sign up at aistudio.google.com |
|
||||
|
||||
### Tier 3: Free with Limits (Specific Use Cases)
|
||||
|
||||
These providers have **free tiers** with specific limits:
|
||||
|
||||
| Provider | Free Limit | Models | Best For |
|
||||
|----------|-----------|--------|----------|
|
||||
| **Cerebras** | 1M tokens/day | Qwen3 235B | Fast inference |
|
||||
| **NVIDIA NIM** | ~40 RPM | 129 models | Variety |
|
||||
| **Groq** | 30 RPM | Llama 4, Mixtral | Speed |
|
||||
| **Cloudflare AI** | 10K neurons/day | 50+ models | Variety |
|
||||
|
||||
---
|
||||
|
||||
## How to Stack Free Tiers
|
||||
|
||||
The magic of OmniRoute is **stacking free tiers**. Instead of relying on one provider, you connect multiple free providers and let OmniRoute automatically pick the best one for each request.
|
||||
|
||||
### Example: Unlimited Free AI
|
||||
|
||||
Connect these 4 providers for **unlimited free AI**:
|
||||
|
||||
1. **Kiro AI** — 50 credits/month (Claude models)
|
||||
2. **OpenCode Free** — Unlimited (GPT models)
|
||||
3. **Pollinations** — No key needed (multiple models)
|
||||
4. **LongCat** — 50M tokens/day (backup)
|
||||
|
||||
Then use `model: "auto"` and OmniRoute will:
|
||||
- Try Kiro first (best quality)
|
||||
- If Kiro is busy → try OpenCode Free
|
||||
- If OpenCode Free is slow → try Pollinations
|
||||
- If all fail → use LongCat as backup
|
||||
|
||||
**Result**: Unlimited free AI with automatic fallback!
|
||||
|
||||
---
|
||||
|
||||
## How to Connect Free Providers
|
||||
|
||||
### Step 1: Open the Dashboard
|
||||
|
||||
Go to `http://localhost:20128` in your browser.
|
||||
|
||||
### Step 2: Go to Providers
|
||||
|
||||
Click **Providers** in the sidebar.
|
||||
|
||||
### Step 3: Click Add Provider
|
||||
|
||||
Click the **+ Add Provider** button.
|
||||
|
||||
### Step 4: Select a Free Provider
|
||||
|
||||
Browse the list and select one of these free providers:
|
||||
- **Kiro AI** — Free Claude models
|
||||
- **OpenCode Free** — Free GPT models
|
||||
- **Pollinations** — Free GPT-5, Claude, Gemini
|
||||
- **LongCat** — 50M tokens/day free
|
||||
- **Cloudflare AI** — 50+ models, 10K neurons/day
|
||||
|
||||
### Step 5: Click Connect
|
||||
|
||||
No API key needed — just click **Connect**.
|
||||
|
||||
### Step 6: Repeat
|
||||
|
||||
Connect 3-4 free providers for the best experience.
|
||||
|
||||
---
|
||||
|
||||
## Free Provider Details
|
||||
|
||||
### Kiro AI
|
||||
|
||||
- **Models**: Claude Sonnet 4.5, Haiku 4.5, Opus 4.6
|
||||
- **Quota**: 50 credits/month
|
||||
- **Auth**: No auth needed
|
||||
- **Best for**: High-quality Claude models
|
||||
|
||||
### OpenCode Free
|
||||
|
||||
- **Models**: GPT-4o, Claude, Gemini
|
||||
- **Quota**: Unlimited
|
||||
- **Auth**: No auth needed
|
||||
- **Best for**: General-purpose AI
|
||||
|
||||
### Pollinations
|
||||
|
||||
- **Models**: GPT-5, Claude, Gemini, DeepSeek, Llama 4
|
||||
- **Quota**: No key needed
|
||||
- **Auth**: No auth needed
|
||||
- **Best for**: Variety of models
|
||||
|
||||
### LongCat
|
||||
|
||||
- **Models**: LongCat-Flash-Lite
|
||||
- **Quota**: 50M tokens/day
|
||||
- **Auth**: No auth needed
|
||||
- **Best for**: High-volume usage
|
||||
|
||||
### Cloudflare AI
|
||||
|
||||
- **Models**: 50+ models
|
||||
- **Quota**: 10K neurons/day
|
||||
- **Auth**: No auth needed
|
||||
- **Best for**: Variety and reliability
|
||||
|
||||
### NVIDIA NIM
|
||||
|
||||
- **Models**: 129 models
|
||||
- **Quota**: ~40 RPM
|
||||
- **Auth**: Sign up at build.nvidia.com
|
||||
- **Best for**: Variety and speed
|
||||
|
||||
### Cerebras
|
||||
|
||||
- **Models**: Qwen3 235B, GPT-OSS 120B
|
||||
- **Quota**: 1M tokens/day
|
||||
- **Auth**: Sign up at cerebras.ai
|
||||
- **Best for**: Fast inference
|
||||
|
||||
### Qwen
|
||||
|
||||
- **Models**: Qwen3-coder-plus/flash/next
|
||||
- **Quota**: Unlimited
|
||||
- **Auth**: No auth needed
|
||||
- **Best for**: Coding tasks
|
||||
|
||||
### Qoder
|
||||
|
||||
- **Models**: Kimi-K2, DeepSeek-R1, Qwen3-coder
|
||||
- **Quota**: Unlimited
|
||||
- **Auth**: No auth needed
|
||||
- **Best for**: Coding tasks
|
||||
|
||||
---
|
||||
|
||||
## How OmniRoute Makes Free Tiers Better
|
||||
|
||||
### 1. Automatic Fallback
|
||||
|
||||
If one free provider is busy or down, OmniRoute automatically tries the next one. You don't need to do anything.
|
||||
|
||||
### 2. Smart Routing
|
||||
|
||||
OmniRoute picks the **best free provider** for each request based on:
|
||||
- Speed — Which provider is fastest right now?
|
||||
- Quality — Which provider is best for this task?
|
||||
- Capacity — Which provider has quota remaining?
|
||||
|
||||
### 3. Token Savings
|
||||
|
||||
OmniRoute's **compression** feature saves 15-95% of tokens. This means your free quota lasts **5-20x longer**.
|
||||
|
||||
### 4. Multi-Account Support
|
||||
|
||||
If you have multiple accounts for the same provider, OmniRoute treats each as a separate candidate. This doubles or triples your free quota.
|
||||
|
||||
---
|
||||
|
||||
## Free Tier Math
|
||||
|
||||
Let's calculate how much free AI you can get:
|
||||
|
||||
### Conservative Estimate (3 providers)
|
||||
|
||||
| Provider | Daily Quota | Monthly Quota |
|
||||
|----------|-------------|---------------|
|
||||
| Kiro AI | ~1.7 credits | 50 credits |
|
||||
| OpenCode Free | Unlimited | Unlimited |
|
||||
| Pollinations | Unlimited | Unlimited |
|
||||
|
||||
**Total**: Unlimited free AI
|
||||
|
||||
### Aggressive Estimate (7 providers)
|
||||
|
||||
| Provider | Daily Quota | Monthly Quota |
|
||||
|----------|-------------|---------------|
|
||||
| Kiro AI | ~1.7 credits | 50 credits |
|
||||
| OpenCode Free | Unlimited | Unlimited |
|
||||
| Pollinations | Unlimited | Unlimited |
|
||||
| LongCat | 50M tokens | 1.5B tokens |
|
||||
| Cloudflare AI | 10K neurons | 300K neurons |
|
||||
| NVIDIA NIM | ~40 RPM | ~1.7M requests |
|
||||
| Cerebras | 1M tokens | 30M tokens |
|
||||
|
||||
**Total**: ~1.9B+ free tokens/month (with compression: ~9.5B+ effective tokens)
|
||||
|
||||
---
|
||||
|
||||
## Common Questions
|
||||
|
||||
### "Is this really free?"
|
||||
|
||||
**Yes!** These are official free tiers from the providers. OmniRoute just makes it easier to use them all at once.
|
||||
|
||||
### "Will the free tier run out?"
|
||||
|
||||
Some providers have limits (like Kiro's 50 credits/month), but others are unlimited (like OpenCode Free and Pollinations). By connecting multiple providers, you always have a backup.
|
||||
|
||||
### "Can I use free providers for production?"
|
||||
|
||||
**Yes!** Many free providers are production-ready. However, for critical applications, consider adding a paid provider as a backup.
|
||||
|
||||
### "What's the catch?"
|
||||
|
||||
No catch! Providers offer free tiers to attract users. OmniRoute just makes it easier to use them all at once.
|
||||
|
||||
### "How do I get more free quota?"
|
||||
|
||||
1. Connect more free providers
|
||||
2. Use compression to save tokens (15-95% savings)
|
||||
3. Use `auto/cheap` to prioritize free/cheap providers
|
||||
4. Create multiple accounts for the same provider
|
||||
|
||||
### "Do free providers have worse quality?"
|
||||
|
||||
**Not necessarily!** Many free providers offer the same models as paid providers. For example, Kiro gives you access to Claude Sonnet 4.5 — the same model you'd get with a paid Anthropic subscription.
|
||||
|
||||
---
|
||||
|
||||
## What's Next?
|
||||
|
||||
- **[Auto-Combo Guide](./AUTO-COMBO-GUIDE.md)** — Let OmniRoute pick the best AI for you
|
||||
- **[Providers Guide](./PROVIDERS-GUIDE.md)** — Connect more providers
|
||||
- **[Troubleshooting](./TROUBLESHOOTING.md)** — Fix common issues
|
||||
- **[Free Tiers Reference](../reference/FREE_TIERS.md)** — Full list of free tiers
|
||||
220
docs/getting-started/PROVIDERS-GUIDE.md
Normal file
220
docs/getting-started/PROVIDERS-GUIDE.md
Normal file
@@ -0,0 +1,220 @@
|
||||
# Providers Guide: Connect AI Models to OmniRoute
|
||||
|
||||
> **TL;DR**: A provider is a connection to an AI service (like OpenAI, Anthropic, Google). You need at least one provider to use OmniRoute.
|
||||
|
||||
---
|
||||
|
||||
## What Is a Provider?
|
||||
|
||||
Think of a provider like a **phone carrier**. Just as you need a phone carrier to make calls, you need an AI provider to use AI models. OmniRoute is like a phone that works with **all carriers** — you can switch between them automatically.
|
||||
|
||||
### Types of Providers
|
||||
|
||||
| Type | What It Is | Examples | Cost |
|
||||
|------|-----------|----------|------|
|
||||
| **Free** | No payment required | Kiro, OpenCode Free, Pollinations | $0 |
|
||||
| **API Key** | You need an API key | OpenAI, Anthropic, Google | Pay per use |
|
||||
| **OAuth** | Login with your account | Claude Code, GitHub Copilot | Subscription |
|
||||
| **Web Cookie** | Uses your browser session | ChatGPT Web, Gemini Web | $0 (uses your account) |
|
||||
|
||||
---
|
||||
|
||||
## Quick Start: Connect Your First Provider
|
||||
|
||||
### Option A: Free Provider (No Credit Card)
|
||||
|
||||
1. Open the dashboard at `http://localhost:20128`
|
||||
2. Go to **Providers** → **Add Provider**
|
||||
3. Select one of these free providers:
|
||||
- **Kiro AI** — Free Claude models (no auth needed)
|
||||
- **OpenCode Free** — Free GPT models (no auth needed)
|
||||
- **Pollinations** — Free GPT-5, Claude, Gemini (no key needed)
|
||||
- **LongCat** — 50M tokens/day free
|
||||
- **Cloudflare AI** — 50+ models, 10K neurons/day
|
||||
4. Click **Connect**
|
||||
5. Done! You now have free AI access.
|
||||
|
||||
### Option B: API Key Provider (Paid)
|
||||
|
||||
1. Get an API key from the provider's website:
|
||||
- **OpenAI**: https://platform.openai.com/api-keys
|
||||
- **Anthropic**: https://console.anthropic.com/
|
||||
- **Google**: https://aistudio.google.com/apikey
|
||||
- **DeepSeek**: https://platform.deepseek.com/
|
||||
- **Groq**: https://console.groq.com/
|
||||
2. Open the dashboard at `http://localhost:20128`
|
||||
3. Go to **Providers** → **Add Provider**
|
||||
4. Select your provider
|
||||
5. Paste your API key
|
||||
6. Click **Connect**
|
||||
7. Done! You now have access to that provider's models.
|
||||
|
||||
### Option C: OAuth Provider (Subscription)
|
||||
|
||||
1. Open the dashboard at `http://localhost:20128`
|
||||
2. Go to **Providers** → **Add Provider**
|
||||
3. Select your provider (e.g., Claude Code, GitHub Copilot)
|
||||
4. Click **Connect with OAuth**
|
||||
5. Login with your account
|
||||
6. Done! You now have access to your subscription models.
|
||||
|
||||
---
|
||||
|
||||
## Best Free Providers
|
||||
|
||||
These providers offer **free access** with no credit card:
|
||||
|
||||
| Provider | Free Quota | Models | How to Connect |
|
||||
|----------|-----------|--------|----------------|
|
||||
| **Kiro AI** | 50 credits/month | Claude Sonnet 4.5, Haiku 4.5, Opus 4.6 | No auth needed |
|
||||
| **OpenCode Free** | Unlimited | GPT-4o, Claude, Gemini | No auth needed |
|
||||
| **Pollinations** | No key needed | GPT-5, Claude, Gemini, DeepSeek, Llama 4 | No auth needed |
|
||||
| **LongCat** | 50M tokens/day | LongCat-Flash-Lite | No auth needed |
|
||||
| **Cloudflare AI** | 10K neurons/day | 50+ models | No auth needed |
|
||||
| **NVIDIA NIM** | ~40 RPM | 129 models | API key needed |
|
||||
| **Cerebras** | 1M tokens/day | Qwen3 235B, GPT-OSS 120B | API key needed |
|
||||
| **Qwen** | Unlimited | Qwen3-coder-plus/flash/next | No auth needed |
|
||||
| **Qoder** | Unlimited | Kimi-K2, DeepSeek-R1, Qwen3-coder | No auth needed |
|
||||
|
||||
**Tip**: Connect multiple free providers for **unlimited free AI** with automatic fallback!
|
||||
|
||||
---
|
||||
|
||||
## Best Paid Providers
|
||||
|
||||
These providers offer **high-quality models** with API keys:
|
||||
|
||||
| Provider | Best Models | Cost | Free Tier |
|
||||
|----------|------------|------|-----------|
|
||||
| **OpenAI** | GPT-5, GPT-4o | $2.50-$10/1M tokens | $5 free credits |
|
||||
| **Anthropic** | Claude Opus 4.6, Sonnet 4.6 | $3-$15/1M tokens | $5 free credits |
|
||||
| **Google** | Gemini 2.5 Pro, Flash | $0.075-$1.25/1M tokens | 1,500 req/day free |
|
||||
| **DeepSeek** | DeepSeek V4 | $0.14-$0.28/1M tokens | 5M free tokens |
|
||||
| **Groq** | Llama 4, Mixtral | $0.05-$0.27/1M tokens | 30 RPM free |
|
||||
| **xAI** | Grok 3 | $0.30-$0.60/1M tokens | — |
|
||||
|
||||
---
|
||||
|
||||
## How to Connect a Provider (Step-by-Step)
|
||||
|
||||
### Step 1: Open the Dashboard
|
||||
|
||||
Go to `http://localhost:20128` in your browser.
|
||||
|
||||
### Step 2: Go to Providers
|
||||
|
||||
Click **Providers** in the sidebar.
|
||||
|
||||
### Step 3: Click Add Provider
|
||||
|
||||
Click the **+ Add Provider** button.
|
||||
|
||||
### Step 4: Select Your Provider
|
||||
|
||||
Browse the list or search for your provider. Click on it.
|
||||
|
||||
### Step 5: Enter Credentials
|
||||
|
||||
- **Free providers**: No credentials needed — just click **Connect**
|
||||
- **API key providers**: Paste your API key
|
||||
- **OAuth providers**: Click **Connect with OAuth** and login
|
||||
|
||||
### Step 6: Test the Connection
|
||||
|
||||
Click **Test Connection** to verify it works.
|
||||
|
||||
### Step 7: Done!
|
||||
|
||||
Your provider is now connected. You can use it with `model: "auto"` or specify the provider directly.
|
||||
|
||||
---
|
||||
|
||||
## Using Multiple Providers
|
||||
|
||||
OmniRoute works best with **multiple providers**. This gives you:
|
||||
|
||||
- **Automatic fallback** — If one provider fails, OmniRoute tries the next
|
||||
- **Cost optimization** — OmniRoute picks the cheapest provider for each request
|
||||
- **Speed optimization** — OmniRoute picks the fastest provider for each request
|
||||
- **Quality optimization** — OmniRoute picks the best provider for each task
|
||||
|
||||
### Recommended Setup
|
||||
|
||||
Connect at least **3 providers** for the best experience:
|
||||
|
||||
1. **One free provider** (Kiro, OpenCode Free, or Pollinations) — Always available
|
||||
2. **One fast provider** (Groq, Cerebras) — For quick responses
|
||||
3. **One quality provider** (OpenAI, Anthropic, Google) — For complex tasks
|
||||
|
||||
Then use `model: "auto"` and OmniRoute will automatically pick the best one for each request.
|
||||
|
||||
---
|
||||
|
||||
## Provider-Specific Setup
|
||||
|
||||
### OpenAI
|
||||
|
||||
1. Get API key: https://platform.openai.com/api-keys
|
||||
2. In OmniRoute: Providers → Add Provider → OpenAI
|
||||
3. Paste API key → Connect
|
||||
|
||||
### Anthropic
|
||||
|
||||
1. Get API key: https://console.anthropic.com/
|
||||
2. In OmniRoute: Providers → Add Provider → Anthropic
|
||||
3. Paste API key → Connect
|
||||
|
||||
### Google (Gemini)
|
||||
|
||||
1. Get API key: https://aistudio.google.com/apikey
|
||||
2. In OmniRoute: Providers → Add Provider → Gemini
|
||||
3. Paste API key → Connect
|
||||
|
||||
### DeepSeek
|
||||
|
||||
1. Get API key: https://platform.deepseek.com/
|
||||
2. In OmniRoute: Providers → Add Provider → DeepSeek
|
||||
3. Paste API key → Connect
|
||||
|
||||
### Groq
|
||||
|
||||
1. Get API key: https://console.groq.com/
|
||||
2. In OmniRoute: Providers → Add Provider → Groq
|
||||
3. Paste API key → Connect
|
||||
|
||||
---
|
||||
|
||||
## Common Questions
|
||||
|
||||
### "Do I need to pay to use OmniRoute?"
|
||||
|
||||
**No!** OmniRoute is free and open-source. You can use free providers (Kiro, OpenCode Free, Pollinations) without paying anything. You only pay if you choose to use paid providers.
|
||||
|
||||
### "Which provider should I start with?"
|
||||
|
||||
Start with **Kiro AI** — it's free, requires no API key, and gives you access to Claude models. Then add more providers as needed.
|
||||
|
||||
### "Can I use multiple providers at once?"
|
||||
|
||||
**Yes!** That's the whole point of OmniRoute. Connect multiple providers and use `model: "auto"` to let OmniRoute pick the best one for each request.
|
||||
|
||||
### "What if a provider goes down?"
|
||||
|
||||
OmniRoute automatically skips failed providers and tries the next one. You don't need to do anything.
|
||||
|
||||
### "How do I disconnect a provider?"
|
||||
|
||||
Go to Providers → click on the provider → click **Disconnect**.
|
||||
|
||||
### "Can I use my existing API keys?"
|
||||
|
||||
**Yes!** If you already have API keys for OpenAI, Anthropic, Google, etc., you can use them in OmniRoute. Just paste them when connecting the provider.
|
||||
|
||||
---
|
||||
|
||||
## What's Next?
|
||||
|
||||
- **[Auto-Combo Guide](./AUTO-COMBO-GUIDE.md)** — Let OmniRoute pick the best AI for you
|
||||
- **[Free Tiers Guide](./FREE-TIERS-GUIDE.md)** — Get free AI with no credit card
|
||||
- **[Troubleshooting](./TROUBLESHOOTING.md)** — Fix common issues
|
||||
- **[Provider Reference](../reference/PROVIDER_REFERENCE.md)** — Full list of 177 providers
|
||||
135
docs/getting-started/QUICK-START.md
Normal file
135
docs/getting-started/QUICK-START.md
Normal file
@@ -0,0 +1,135 @@
|
||||
# Quick Start: Get OmniRoute Running in 3 Minutes
|
||||
|
||||
> **TL;DR**: Install → Connect a free provider → Point your IDE to OmniRoute. Done.
|
||||
|
||||
---
|
||||
|
||||
## Step 1: Install OmniRoute
|
||||
|
||||
Choose your preferred method:
|
||||
|
||||
### Option A: npm (Recommended)
|
||||
|
||||
```bash
|
||||
npm install -g omniroute
|
||||
```
|
||||
|
||||
### Option B: Docker
|
||||
|
||||
```bash
|
||||
docker run -d --name omniroute -p 20128:20128 diegosouzapw/omniroute:latest
|
||||
```
|
||||
|
||||
### Option C: From Source
|
||||
|
||||
```bash
|
||||
git clone https://github.com/diegosouzapw/OmniRoute.git
|
||||
cd OmniRoute
|
||||
npm install
|
||||
npm run dev
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Step 2: Start OmniRoute
|
||||
|
||||
```bash
|
||||
omniroute
|
||||
```
|
||||
|
||||
OmniRoute starts at `http://localhost:20128`. The dashboard opens automatically.
|
||||
|
||||
---
|
||||
|
||||
## Step 3: Connect a Free Provider
|
||||
|
||||
You can use OmniRoute **without paying anything** by connecting a free provider.
|
||||
|
||||
### Option A: Kiro (Free Claude — No Credit Card)
|
||||
|
||||
1. Open the dashboard at `http://localhost:20128`
|
||||
2. Go to **Providers** → **Add Provider**
|
||||
3. Select **Kiro AI**
|
||||
4. Click **Connect** (no API key needed!)
|
||||
5. Done! You now have free access to Claude models.
|
||||
|
||||
### Option B: OpenCode Free (No Auth)
|
||||
|
||||
1. Open the dashboard at `http://localhost:20128`
|
||||
2. Go to **Providers** → **Add Provider**
|
||||
3. Select **OpenCode Free**
|
||||
4. Click **Connect** (no API key needed!)
|
||||
5. Done! You now have free access to multiple models.
|
||||
|
||||
### Option C: Pollinations (No Key Needed)
|
||||
|
||||
1. Open the dashboard at `http://localhost:20128`
|
||||
2. Go to **Providers** → **Add Provider**
|
||||
3. Select **Pollinations**
|
||||
4. Click **Connect** (no API key needed!)
|
||||
5. Done! You now have free access to GPT-5, Claude, Gemini, and more.
|
||||
|
||||
---
|
||||
|
||||
## Step 4: Point Your IDE to OmniRoute
|
||||
|
||||
In your IDE or CLI tool, set:
|
||||
|
||||
```
|
||||
Base URL: http://localhost:20128/v1
|
||||
API Key: [copy from Dashboard → Endpoints]
|
||||
Model: auto
|
||||
```
|
||||
|
||||
That's it! Your IDE now uses OmniRoute with automatic provider selection.
|
||||
|
||||
---
|
||||
|
||||
## Step 5: Verify It Works
|
||||
|
||||
```bash
|
||||
curl http://localhost:20128/v1/models -H "Authorization: Bearer YOUR_KEY"
|
||||
```
|
||||
|
||||
You should see your connected models listed.
|
||||
|
||||
---
|
||||
|
||||
## What's Next?
|
||||
|
||||
- **[Auto-Combo Guide](./AUTO-COMBO-GUIDE.md)** — Let OmniRoute pick the best AI for you
|
||||
- **[Providers Guide](./PROVIDERS-GUIDE.md)** — Connect more providers (free and paid)
|
||||
- **[Free Tiers Guide](./FREE-TIERS-GUIDE.md)** — Get free AI with no credit card
|
||||
- **[Troubleshooting](./TROUBLESHOOTING.md)** — Fix common issues
|
||||
|
||||
---
|
||||
|
||||
## Common Questions
|
||||
|
||||
### "Do I need an API key?"
|
||||
|
||||
**No!** You can use free providers (Kiro, OpenCode Free, Pollinations) without any API key. Just connect them in the dashboard.
|
||||
|
||||
### "What is `auto`?"
|
||||
|
||||
`auto` tells OmniRoute to automatically pick the best provider for each request. It considers speed, cost, quality, and availability. See the [Auto-Combo Guide](./AUTO-COMBO-GUIDE.md) for details.
|
||||
|
||||
### "How much does it cost?"
|
||||
|
||||
OmniRoute itself is **free and open-source**. You only pay for the providers you use. Many providers have free tiers — see the [Free Tiers Guide](./FREE-TIERS-GUIDE.md).
|
||||
|
||||
### "Can I use it with Claude Code / Cursor / Copilot?"
|
||||
|
||||
**Yes!** OmniRoute works with any tool that supports OpenAI format. Just set the base URL to `http://localhost:20128/v1`. See the [CLI Tools Guide](../reference/CLI-TOOLS.md) for specific setup instructions.
|
||||
|
||||
### "What if a provider goes down?"
|
||||
|
||||
OmniRoute automatically skips failed providers and tries the next one. You don't need to do anything. See the [Auto-Combo Guide](./AUTO-COMBO-GUIDE.md) for details.
|
||||
|
||||
---
|
||||
|
||||
## Need Help?
|
||||
|
||||
- **[Troubleshooting](./TROUBLESHOOTING.md)** — Common issues and fixes
|
||||
- **[Discord](https://discord.gg/hmexnhgE)** — Community support
|
||||
- **[GitHub Issues](https://github.com/diegosouzapw/OmniRoute/issues)** — Report bugs
|
||||
501
docs/getting-started/TROUBLESHOOTING.md
Normal file
501
docs/getting-started/TROUBLESHOOTING.md
Normal file
@@ -0,0 +1,501 @@
|
||||
---
|
||||
title: "Troubleshooting"
|
||||
version: 3.8.2
|
||||
lastUpdated: 2026-05-13
|
||||
---
|
||||
|
||||
# Troubleshooting
|
||||
|
||||
> **For Users**: Looking for quick fixes? See the [Quick Reference](#quick-reference) below.
|
||||
|
||||
🌐 **Languages:** 🇺🇸 [English](./TROUBLESHOOTING.md) | 🇧🇷 [Português (Brasil)](../i18n/pt-BR/docs/guides/TROUBLESHOOTING.md) | 🇪🇸 [Español](../i18n/es/docs/guides/TROUBLESHOOTING.md) | 🇫🇷 [Français](../i18n/fr/docs/guides/TROUBLESHOOTING.md) | 🇮🇹 [Italiano](../i18n/it/docs/guides/TROUBLESHOOTING.md) | 🇷🇺 [Русский](../i18n/ru/docs/guides/TROUBLESHOOTING.md) | 🇨🇳 [中文 (简体)](../i18n/zh-CN/docs/guides/TROUBLESHOOTING.md) | 🇩🇪 [Deutsch](../i18n/de/docs/guides/TROUBLESHOOTING.md) | 🇮🇳 [हिन्दी](../i18n/in/docs/guides/TROUBLESHOOTING.md) | 🇹🇭 [ไทย](../i18n/th/docs/guides/TROUBLESHOOTING.md) | 🇺🇦 [Українська](../i18n/uk-UA/docs/guides/TROUBLESHOOTING.md) | 🇸🇦 [العربية](../i18n/ar/docs/guides/TROUBLESHOOTING.md) | 🇯🇵 [日本語](../i18n/ja/docs/guides/TROUBLESHOOTING.md) | 🇻🇳 [Tiếng Việt](../i18n/vi/docs/guides/TROUBLESHOOTING.md) | 🇧🇬 [Български](../i18n/bg/docs/guides/TROUBLESHOOTING.md) | 🇩🇰 [Dansk](../i18n/da/docs/guides/TROUBLESHOOTING.md) | 🇫🇮 [Suomi](../i18n/fi/docs/guides/TROUBLESHOOTING.md) | 🇮🇱 [עברית](../i18n/he/docs/guides/TROUBLESHOOTING.md) | 🇭🇺 [Magyar](../i18n/hu/docs/guides/TROUBLESHOOTING.md) | 🇮🇩 [Bahasa Indonesia](../i18n/id/docs/guides/TROUBLESHOOTING.md) | 🇰🇷 [한국어](../i18n/ko/docs/guides/TROUBLESHOOTING.md) | 🇲🇾 [Bahasa Melayu](../i18n/ms/docs/guides/TROUBLESHOOTING.md) | 🇳🇱 [Nederlands](../i18n/nl/docs/guides/TROUBLESHOOTING.md) | 🇳🇴 [Norsk](../i18n/no/docs/guides/TROUBLESHOOTING.md) | 🇵🇹 [Português (Portugal)](../i18n/pt/docs/guides/TROUBLESHOOTING.md) | 🇷🇴 [Română](../i18n/ro/docs/guides/TROUBLESHOOTING.md) | 🇵🇱 [Polski](../i18n/pl/docs/guides/TROUBLESHOOTING.md) | 🇸🇰 [Slovenčina](../i18n/sk/docs/guides/TROUBLESHOOTING.md) | 🇸🇪 [Svenska](../i18n/sv/docs/guides/TROUBLESHOOTING.md) | 🇵🇭 [Filipino](../i18n/phi/docs/guides/TROUBLESHOOTING.md) | 🇨🇿 [Čeština](../i18n/cs/docs/guides/TROUBLESHOOTING.md)
|
||||
|
||||
Common problems and solutions for OmniRoute.
|
||||
|
||||
---
|
||||
|
||||
## Quick Reference
|
||||
|
||||
**New to OmniRoute?** Start here — these solve 90% of problems:
|
||||
|
||||
| I see this | What it means | What to do |
|
||||
|------------|--------------|------------|
|
||||
| "Can't connect" | OmniRoute isn't running | Run `omniroute` or `docker restart omniroute` |
|
||||
| "Invalid API key" | Your key is wrong or expired | Re-copy the key from the provider's website |
|
||||
| "Rate limit exceeded" | You're sending too many requests | Wait 1 minute, or use `model: "auto"` for automatic fallback |
|
||||
| "Quota exceeded" | You've used up your free/paid quota | Connect more providers, or use free providers (Kiro, Pollinations) |
|
||||
| "Slow responses" | Provider is busy or far away | Use `model: "auto/fast"` or connect a faster provider (Groq, Cerebras) |
|
||||
| "Wrong provider used" | `auto` picked a different provider | That's normal! `auto` picks the best one. Force a specific provider with `model: "openai/gpt-4o"` |
|
||||
| "502 Bad Gateway" | Provider is down | Wait and retry, or use `model: "auto"` to switch providers |
|
||||
| "401 Unauthorized" | Your credentials are wrong | Check your API key or re-authenticate with OAuth |
|
||||
| "429 Too Many Requests" | Rate limited | Wait 1 minute, or connect more providers |
|
||||
|
||||
**Still stuck?** See the [Quick Fixes](#quick-fixes) below, or ask on [Discord](https://discord.gg/hmexnhgE).
|
||||
|
||||
---
|
||||
|
||||
## Quick Fixes
|
||||
|
||||
| Problem | Solution |
|
||||
| --------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| First login not working | Set `INITIAL_PASSWORD` in `.env` (no hardcoded default) |
|
||||
| Dashboard opens on wrong port | Set `PORT=20128` and `NEXT_PUBLIC_BASE_URL=http://localhost:20128` |
|
||||
| No logs written to disk | Set `APP_LOG_TO_FILE=true` and verify call log capture is enabled |
|
||||
| EACCES: permission denied | Set `DATA_DIR=/path/to/writable/dir` to override `~/.omniroute` |
|
||||
| Routing strategy not saving | Update to the latest v3.x release (Zod schema fix for settings persistence shipped in earlier versions) |
|
||||
| Login crash / blank page | Check Node.js version — see [Node.js Compatibility](#nodejs-compatibility) below |
|
||||
| `dlopen` / `slice is not valid mach-o file` (macOS) | Run `cd $(npm root -g)/omniroute/app && npm rebuild better-sqlite3 && omniroute` — see [macOS native module rebuild](#macos-native-module-rebuild) below |
|
||||
| Proxy "fetch failed" | Ensure proxy config is set at the correct level — see [Proxy Issues](#proxy-issues) below |
|
||||
|
||||
---
|
||||
|
||||
## Node.js Compatibility
|
||||
|
||||
<a name="nodejs-compatibility"></a>
|
||||
|
||||
### Login page crashes or shows "Module self-registration" error
|
||||
|
||||
**Cause:** You are running a Node.js version outside OmniRoute's approved secure runtime floor. The most common case is running an older Node 20, 22, or 24 patch level that falls below the patched security floor OmniRoute requires.
|
||||
|
||||
**Symptoms:**
|
||||
|
||||
- Login page shows a blank screen or a server error
|
||||
- Console shows `Error: Module did not self-register` or similar native binding errors
|
||||
- The login page shows an **orange warning banner** with your Node version if the runtime is outside the supported secure policy
|
||||
|
||||
**Fix:**
|
||||
|
||||
1. Install a supported Node.js LTS release (recommended: Node.js 24.x):
|
||||
```bash
|
||||
nvm install 24
|
||||
nvm use 24
|
||||
```
|
||||
2. Verify your version: `node --version` should show `v24.0.0` or newer on the 24.x LTS line
|
||||
3. Reinstall OmniRoute: `npm install -g omniroute`
|
||||
4. Restart: `omniroute`
|
||||
|
||||
> **Supported secure versions:** `>=20.20.2 <21`, `>=22.22.2 <23`, or `>=24.0.0 <27`. Node.js 24.x LTS (Krypton) and Node.js 26 are fully supported.
|
||||
|
||||
### macOS: `dlopen` / "slice is not valid mach-o file"
|
||||
|
||||
<a name="macos-native-module-rebuild"></a>
|
||||
|
||||
**Cause:** After a global `npm install -g omniroute`, the `better-sqlite3` native binary inside the package may have been compiled for a different architecture or Node.js ABI than what is running locally. This is common on macOS (both Apple Silicon and Intel) when the pre-built binary does not match your environment.
|
||||
|
||||
**Symptoms:**
|
||||
|
||||
- Server fails immediately on startup with a `dlopen` error
|
||||
- Error contains `slice is not valid mach-o file`
|
||||
- Full example:
|
||||
|
||||
```
|
||||
dlopen(/Users/<user>/.nvm/versions/node/v24.14.1/lib/node_modules/omniroute/app/node_modules/better-sqlite3/build/Release/better_sqlite3.node, 0x0001): tried: '...' (slice is not valid mach-o file)
|
||||
```
|
||||
|
||||
**Fix — rebuild for your local environment (no Node.js downgrade required):**
|
||||
|
||||
```bash
|
||||
cd $(npm root -g)/omniroute/app
|
||||
npm rebuild better-sqlite3
|
||||
omniroute
|
||||
```
|
||||
|
||||
> **Note:** This recompiles the native binding against your local Node.js version and CPU architecture, resolving the binary mismatch. The officially supported range is **`>=20.20.2 <21`, `>=22.22.2 <23`, or `>=24.0.0 <27`** (`engines` field in `package.json`). Node.js 24.x LTS (Krypton) and Node.js 26 are fully supported with `better-sqlite3` v12.x.
|
||||
|
||||
---
|
||||
|
||||
## Proxy Issues
|
||||
|
||||
<a name="proxy-issues"></a>
|
||||
|
||||
### Provider validation shows "fetch failed"
|
||||
|
||||
**Cause:** The API key validation endpoint (`POST /api/providers/validate`) was previously bypassing proxy configuration, causing failures in environments that require proxy routing.
|
||||
|
||||
**Fix (v3.5.5+):** This is now fixed. Provider validation routes through `runWithProxyContext`, honoring provider-level and global proxy settings automatically.
|
||||
|
||||
### Token health check fails with "fetch failed"
|
||||
|
||||
**Cause:** Background OAuth token refresh was not resolving proxy configuration per connection.
|
||||
|
||||
**Fix (v3.5.5+):** The token health check scheduler now resolves proxy config per connection before attempting refresh. Update to v3.5.5+.
|
||||
|
||||
### SOCKS5 proxy returns "invalid onRequestStart method"
|
||||
|
||||
**Cause:** On Node.js 22, the undici@8 dispatcher is incompatible with Node's built-in `fetch()` implementation.
|
||||
|
||||
**Fix (v3.5.5+):** OmniRoute now uses undici's own `fetch()` function when a proxy dispatcher is active, ensuring consistent behavior. Update to v3.5.5+.
|
||||
|
||||
---
|
||||
|
||||
## Provider Issues
|
||||
|
||||
### "Language model did not provide messages"
|
||||
|
||||
**Cause:** Provider quota exhausted.
|
||||
|
||||
**Fix:**
|
||||
|
||||
1. Check dashboard quota tracker
|
||||
2. Use a combo with fallback tiers
|
||||
3. Switch to cheaper/free tier
|
||||
|
||||
### Rate Limiting
|
||||
|
||||
**Cause:** Subscription quota exhausted.
|
||||
|
||||
**Fix:**
|
||||
|
||||
- Add fallback: `cc/claude-opus-4-6 → glm/glm-4.7 → if/kimi-k2-thinking`
|
||||
- Use GLM/MiniMax as cheap backup
|
||||
|
||||
### OAuth Token Expired
|
||||
|
||||
OmniRoute auto-refreshes tokens. If issues persist:
|
||||
|
||||
1. Dashboard → Provider → Reconnect
|
||||
2. Delete and re-add the provider connection
|
||||
|
||||
### Kiro multi-account: second account invalidates the first
|
||||
|
||||
**Cause:** Kiro's backend enforces a single active session per OIDC client registration.
|
||||
When two accounts share the same registered client (connections imported before v3.8.0),
|
||||
refreshing one account's token invalidates the other's refresh token.
|
||||
|
||||
**Fix (v3.8.0+):** Re-import affected connections.
|
||||
Starting with v3.8.0, every new Kiro connection created via **Import Token**,
|
||||
**Google/GitHub social login**, or **Auto-Import** automatically registers its own
|
||||
dedicated OIDC client. The connection is therefore fully isolated and refreshing one
|
||||
account has no effect on any other account.
|
||||
|
||||
Connections that were imported _before_ v3.8.0 do not carry a per-connection client
|
||||
registration. Those connections continue to use the shared social-auth refresh endpoint.
|
||||
To gain isolation, delete the old connection from Dashboard → Providers and re-add it
|
||||
via any of the three import flows.
|
||||
|
||||
For full details and step-by-step instructions for adding two Kiro accounts side by side,
|
||||
see [`docs/guides/KIRO_SETUP.md`](../guides/KIRO_SETUP.md).
|
||||
|
||||
---
|
||||
|
||||
## Cloud Issues
|
||||
|
||||
### Cloud Sync Errors
|
||||
|
||||
1. Verify `BASE_URL` points to your running instance (e.g., `http://localhost:20128`)
|
||||
2. Verify `CLOUD_URL` points to your cloud endpoint (e.g., `https://omniroute.dev`)
|
||||
3. Keep `NEXT_PUBLIC_*` values aligned with server-side values
|
||||
|
||||
### Cloud `stream=false` Returns 500
|
||||
|
||||
**Symptom:** `Unexpected token 'd'...` on cloud endpoint for non-streaming calls.
|
||||
|
||||
**Cause:** Upstream returns SSE payload while client expects JSON.
|
||||
|
||||
**Workaround:** Use `stream=true` for cloud direct calls. Local runtime includes SSE→JSON fallback.
|
||||
|
||||
### Cloud Says Connected but "Invalid API key"
|
||||
|
||||
1. Create a fresh key from local dashboard (`/api/keys`)
|
||||
2. Run cloud sync: Enable Cloud → Sync Now
|
||||
3. Old/non-synced keys can still return `401` on cloud
|
||||
|
||||
---
|
||||
|
||||
## Docker Issues
|
||||
|
||||
### CLI Tool Shows Not Installed
|
||||
|
||||
1. Check runtime fields: `curl http://localhost:20128/api/cli-tools/runtime/codex | jq`
|
||||
2. For portable mode: use image target `runner-cli` (bundled CLIs)
|
||||
3. For host mount mode: set `CLI_EXTRA_PATHS` and mount host bin directory as read-only
|
||||
4. If `installed=true` and `runnable=false`: binary was found but failed healthcheck
|
||||
|
||||
### Quick Runtime Validation
|
||||
|
||||
```bash
|
||||
curl -s http://localhost:20128/api/cli-tools/codex-settings | jq '{installed,runnable,commandPath,runtimeMode,reason}'
|
||||
curl -s http://localhost:20128/api/cli-tools/claude-settings | jq '{installed,runnable,commandPath,runtimeMode,reason}'
|
||||
curl -s http://localhost:20128/api/cli-tools/openclaw-settings | jq '{installed,runnable,commandPath,runtimeMode,reason}'
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Cost Issues
|
||||
|
||||
### High Costs
|
||||
|
||||
1. Check usage stats in Dashboard → Usage
|
||||
2. Switch primary model to GLM/MiniMax
|
||||
3. Use free tier (Gemini CLI, Qoder) for non-critical tasks
|
||||
4. Set cost budgets per API key: Dashboard → API Keys → Budget
|
||||
|
||||
---
|
||||
|
||||
## Debugging
|
||||
|
||||
### Enable Log Files
|
||||
|
||||
Set `APP_LOG_TO_FILE=true` in your `.env` file. Application logs are written under `logs/`.
|
||||
Request artifacts are stored under `${DATA_DIR}/call_logs/` when the call log pipeline is
|
||||
enabled in settings.
|
||||
When pipeline capture is enabled, set `CALL_LOG_PIPELINE_CAPTURE_STREAM_CHUNKS=false` to omit
|
||||
stream chunk payloads, or tune `CALL_LOG_PIPELINE_MAX_SIZE_KB` to change the artifact cap in KB.
|
||||
|
||||
### Check Provider Health
|
||||
|
||||
```bash
|
||||
# Health dashboard
|
||||
http://localhost:20128/dashboard/health
|
||||
|
||||
# API health check
|
||||
curl http://localhost:20128/api/monitoring/health
|
||||
```
|
||||
|
||||
### Runtime Storage
|
||||
|
||||
- Main state: `${DATA_DIR}/storage.sqlite` (providers, combos, aliases, keys, settings)
|
||||
- Usage: SQLite tables in `storage.sqlite` (`usage_history`, `call_logs`, `proxy_logs`) + optional `${DATA_DIR}/call_logs/`
|
||||
- Application logs: `<repo>/logs/...` (when `APP_LOG_TO_FILE=true`)
|
||||
- Call log artifacts: `${DATA_DIR}/call_logs/YYYY-MM-DD/...` when the call log pipeline is enabled
|
||||
|
||||
---
|
||||
|
||||
## Circuit Breaker Issues
|
||||
|
||||
### Provider stuck in OPEN state
|
||||
|
||||
When a provider's circuit breaker is OPEN, requests are blocked until the cooldown expires.
|
||||
|
||||
**Fix:**
|
||||
|
||||
1. Go to **Dashboard → Settings → Resilience**
|
||||
2. Check the circuit breaker card for the affected provider
|
||||
3. Click **Reset All** to clear all breakers, or wait for the cooldown to expire
|
||||
4. Verify the provider is actually available before resetting
|
||||
|
||||
### Provider keeps tripping the circuit breaker
|
||||
|
||||
If a provider repeatedly enters OPEN state:
|
||||
|
||||
1. Check **Dashboard → Health → Provider Health** for the failure pattern
|
||||
2. Go to **Settings → Resilience → Provider Profiles** and increase the failure threshold
|
||||
3. Check if the provider has changed API limits or requires re-authentication
|
||||
4. Review latency telemetry — high latency may cause timeout-based failures
|
||||
|
||||
---
|
||||
|
||||
## Audio Transcription Issues
|
||||
|
||||
### "Unsupported model" error
|
||||
|
||||
- Ensure you're using the correct prefix: `deepgram/nova-3` or `assemblyai/best`
|
||||
- Verify the provider is connected in **Dashboard → Providers**
|
||||
|
||||
### Transcription returns empty or fails
|
||||
|
||||
- Check supported audio formats: `mp3`, `wav`, `m4a`, `flac`, `ogg`, `webm`
|
||||
- Verify file size is within provider limits (typically < 25MB)
|
||||
- Check provider API key validity in the provider card
|
||||
|
||||
---
|
||||
|
||||
## Translator Debugging
|
||||
|
||||
Use **Dashboard → Translator** to debug format translation issues:
|
||||
|
||||
| Mode | When to Use |
|
||||
| ---------------- | -------------------------------------------------------------------------------------------- |
|
||||
| **Playground** | Compare input/output formats side by side — paste a failing request to see how it translates |
|
||||
| **Chat Tester** | Send live messages and inspect the full request/response payload including headers |
|
||||
| **Test Bench** | Run batch tests across format combinations to find which translations are broken |
|
||||
| **Live Monitor** | Watch real-time request flow to catch intermittent translation issues |
|
||||
|
||||
### Common format issues
|
||||
|
||||
- **Thinking tags not appearing** — Check if the target provider supports thinking and the thinking budget setting
|
||||
- **Tool calls dropping** — Some format translations may strip unsupported fields; verify in Playground mode
|
||||
- **System prompt missing** — Claude and Gemini handle system prompts differently; check translation output
|
||||
- **SDK returns raw string instead of object** — Resolved in v1.x; response sanitizer strips non-standard fields (`x_groq`, `usage_breakdown`, etc.) that cause OpenAI SDK Pydantic validation failures. If you still see this on v3.x+, please file an issue.
|
||||
- **GLM/ERNIE rejects `system` role** — Resolved in v1.x; role normalizer automatically merges system messages into user messages for incompatible models. If you still see this on v3.x+, please file an issue.
|
||||
- **`developer` role not recognized** — Resolved in v1.x; automatically converted to `system` for non-OpenAI providers. If you still see this on v3.x+, please file an issue.
|
||||
- **`json_schema` not working with Gemini** — Resolved in v1.x; `response_format` is now converted to Gemini's `responseMimeType` + `responseSchema`. If you still see this on v3.x+, please file an issue.
|
||||
|
||||
---
|
||||
|
||||
## Resilience Settings
|
||||
|
||||
### Auto rate-limit not triggering
|
||||
|
||||
- Auto rate-limit only applies to API key providers (not OAuth/subscription)
|
||||
- Verify **Settings → Resilience → Provider Profiles** has auto-rate-limit enabled
|
||||
- Check if the provider returns `429` status codes or `Retry-After` headers
|
||||
|
||||
### Tuning exponential backoff
|
||||
|
||||
Provider profiles support these settings:
|
||||
|
||||
- **Base delay** — Initial wait time after first failure (default: 1s)
|
||||
- **Max delay** — Maximum wait time cap (default: 30s)
|
||||
- **Multiplier** — How much to increase delay per consecutive failure (default: 2x)
|
||||
|
||||
### Anti-thundering herd
|
||||
|
||||
When many concurrent requests hit a rate-limited provider, OmniRoute uses mutex + auto rate-limiting to serialize requests and prevent cascading failures. This is automatic for API key providers.
|
||||
|
||||
---
|
||||
|
||||
## Optional RAG / LLM failure taxonomy (16 problems)
|
||||
|
||||
Some OmniRoute users place the gateway in front of RAG or agent stacks. In those setups it is common to see a strange pattern: OmniRoute looks healthy (providers up, routing profiles ok, no rate limit alerts) but the final answer is still wrong.
|
||||
|
||||
In practice these incidents usually come from the downstream RAG pipeline, not from the gateway itself.
|
||||
|
||||
If you want a shared vocabulary to describe those failures you can use the WFGY ProblemMap, an external MIT license text resource that defines sixteen recurring RAG / LLM failure patterns. At a high level it covers:
|
||||
|
||||
- retrieval drift and broken context boundaries
|
||||
- empty or stale indexes and vector stores
|
||||
- embedding versus semantic mismatch
|
||||
- prompt assembly and context window issues
|
||||
- logic collapse and overconfident answers
|
||||
- long chain and agent coordination failures
|
||||
- multi agent memory and role drift
|
||||
- deployment and bootstrap ordering problems
|
||||
|
||||
The idea is simple:
|
||||
|
||||
1. When you investigate a bad response, capture:
|
||||
- user task and request
|
||||
- route or provider combo in OmniRoute
|
||||
- any RAG context used downstream (retrieved documents, tool calls, etc)
|
||||
2. Map the incident to one or two WFGY ProblemMap numbers (`No.1` … `No.16`).
|
||||
3. Store the number in your own dashboard, runbook, or incident tracker next to the OmniRoute logs.
|
||||
4. Use the corresponding WFGY page to decide whether you need to change your RAG stack, retriever, or routing strategy.
|
||||
|
||||
Full text and concrete recipes live here (MIT license, text only):
|
||||
|
||||
[WFGY ProblemMap README](https://github.com/onestardao/WFGY/blob/main/ProblemMap/README.md)
|
||||
|
||||
You can ignore this section if you do not run RAG or agent pipelines behind OmniRoute.
|
||||
|
||||
---
|
||||
|
||||
## v3.8.0 Known Issues
|
||||
|
||||
Issues specific to the v3.8.0 release and their current workarounds. If a fix lands in a later patch, the entry will be updated or removed.
|
||||
|
||||
### Windsurf OAuth flow fails with 401
|
||||
|
||||
**Symptoms:**
|
||||
|
||||
- "401 unauthorized" while completing the Windsurf OAuth flow from the dashboard
|
||||
- Windsurf provider card stays in "needs reconnection" state after the callback
|
||||
|
||||
**Causes:**
|
||||
|
||||
- `WINDSURF_FIREBASE_API_KEY` env var missing or empty
|
||||
- `WINDSURF_API_KEY` misconfigured or pointing at a stale token
|
||||
- Local firewall/proxy blocking the OAuth callback
|
||||
|
||||
**Fix:**
|
||||
|
||||
1. Verify both `WINDSURF_FIREBASE_API_KEY` and `WINDSURF_API_KEY` are set in `.env`
|
||||
2. Restart OmniRoute so the new env values are picked up
|
||||
3. Re-run the OAuth flow from **Dashboard → Providers → Windsurf → Reconnect**
|
||||
|
||||
### Devin CLI auth failures
|
||||
|
||||
**Symptoms:**
|
||||
|
||||
- "Devin CLI not found" or "auth failed" when invoking Devin-backed tools
|
||||
- CLI runtime check reports `installed=false`
|
||||
|
||||
**Causes:**
|
||||
|
||||
- `CLI_DEVIN_BIN` points to a path that does not exist
|
||||
- Devin CLI is not installed on the host
|
||||
|
||||
**Fix:**
|
||||
|
||||
1. Install the Devin CLI for your platform
|
||||
2. Set `CLI_DEVIN_BIN=/usr/local/bin/devin` (or the real path) in `.env`
|
||||
3. Restart OmniRoute and re-test from **Dashboard → CLI Tools**
|
||||
|
||||
### Model cooldown stuck (manual reset)
|
||||
|
||||
**Symptoms:**
|
||||
|
||||
- A model stays listed in cooldown even after the expiration time has passed
|
||||
- Requests still skip the model in combo routing despite the timestamp being in the past
|
||||
|
||||
**Manual reset:**
|
||||
|
||||
- **Dashboard:** **Settings → Model Cooldowns** → click **Re-enable** on the affected card
|
||||
- **API:** `DELETE /api/resilience/model-cooldowns` with management auth headers
|
||||
|
||||
### Command Code provider connection fails with 403
|
||||
|
||||
**Symptoms:**
|
||||
|
||||
- 403 when testing the Command Code provider connection
|
||||
- The provider card shows "unauthorized" after a fresh add
|
||||
|
||||
**Cause:** The OAuth flow did not complete (callback not received or token not persisted).
|
||||
|
||||
**Fix:**
|
||||
|
||||
- Run `omniroute providers` from the CLI to re-trigger the OAuth flow, or
|
||||
- Re-run OAuth from **Dashboard → Providers → Command Code → Reconnect**
|
||||
|
||||
### ModelScope returns aggressive 429 cooldowns
|
||||
|
||||
**Symptoms:**
|
||||
|
||||
- Very short or immediate cooldowns on ModelScope after a small burst of requests
|
||||
- Combo routing skips ModelScope earlier than expected
|
||||
|
||||
**Cause:** ModelScope emits provider-specific `Retry-After` headers. v3.8.0 ships dedicated handling for those headers, so older versions misread them as generic rate-limit hints.
|
||||
|
||||
**Fix:**
|
||||
|
||||
- Ensure you are on v3.8.0 or later
|
||||
- Verify the `useUpstream429BreakerHints` toggle is enabled under **Settings → Resilience**
|
||||
|
||||
### OMNIROUTE_WS_BRIDGE_SECRET missing in production
|
||||
|
||||
**Symptoms:**
|
||||
|
||||
- 401 on every Codex/Responses WebSocket bridge request when running on a remote production host
|
||||
- WebSocket bridge handshake closes immediately after connect
|
||||
|
||||
**Cause:** The `OMNIROUTE_WS_BRIDGE_SECRET` env var is missing from the production environment.
|
||||
|
||||
**Fix:**
|
||||
|
||||
1. Generate a random secret: `openssl rand -hex 32`
|
||||
2. Set `OMNIROUTE_WS_BRIDGE_SECRET=<random-secret>` in the production server env (and any client that talks to the bridge)
|
||||
3. Restart OmniRoute
|
||||
|
||||
### Responses API: background mode degraded to synchronous
|
||||
|
||||
**Symptoms:**
|
||||
|
||||
- Warning logged: `background mode degraded to synchronous`
|
||||
- A `background: true` request returns a normal synchronous response instead of a background job handle
|
||||
|
||||
**Cause:** v3.8.0 intentionally degrades `background: true` on the Responses API to synchronous execution while emitting a warning. Full async background execution is a future deliverable.
|
||||
|
||||
**Fix:**
|
||||
|
||||
- Adjust the client to call without `background`, or
|
||||
- Wait for a later release that ships full async background mode (track the changelog)
|
||||
|
||||
---
|
||||
|
||||
## Still Stuck?
|
||||
|
||||
- **GitHub Issues**: [github.com/diegosouzapw/OmniRoute/issues](https://github.com/diegosouzapw/OmniRoute/issues)
|
||||
- **Architecture**: See [`docs/architecture/ARCHITECTURE.md`](../architecture/ARCHITECTURE.md) for internal details
|
||||
- **API Reference**: See [`docs/reference/API_REFERENCE.md`](../reference/API_REFERENCE.md) for all endpoints
|
||||
- **Health Dashboard**: Check **Dashboard → Health** for real-time system status
|
||||
- **Translator**: Use **Dashboard → Translator** to debug format issues
|
||||
11
docs/getting-started/meta.json
Normal file
11
docs/getting-started/meta.json
Normal file
@@ -0,0 +1,11 @@
|
||||
{
|
||||
"title": "Getting Started",
|
||||
"description": "Get started with OmniRoute in minutes — no technical background needed",
|
||||
"pages": [
|
||||
"QUICK-START",
|
||||
"AUTO-COMBO-GUIDE",
|
||||
"PROVIDERS-GUIDE",
|
||||
"FREE-TIERS-GUIDE",
|
||||
"TROUBLESHOOTING"
|
||||
]
|
||||
}
|
||||
@@ -6,12 +6,38 @@ lastUpdated: 2026-05-13
|
||||
|
||||
# Troubleshooting
|
||||
|
||||
> **For Users**: Looking for quick fixes? See the [Quick Reference](#quick-reference) below.
|
||||
|
||||
🌐 **Languages:** 🇺🇸 [English](./TROUBLESHOOTING.md) | 🇧🇷 [Português (Brasil)](../i18n/pt-BR/docs/guides/TROUBLESHOOTING.md) | 🇪🇸 [Español](../i18n/es/docs/guides/TROUBLESHOOTING.md) | 🇫🇷 [Français](../i18n/fr/docs/guides/TROUBLESHOOTING.md) | 🇮🇹 [Italiano](../i18n/it/docs/guides/TROUBLESHOOTING.md) | 🇷🇺 [Русский](../i18n/ru/docs/guides/TROUBLESHOOTING.md) | 🇨🇳 [中文 (简体)](../i18n/zh-CN/docs/guides/TROUBLESHOOTING.md) | 🇩🇪 [Deutsch](../i18n/de/docs/guides/TROUBLESHOOTING.md) | 🇮🇳 [हिन्दी](../i18n/in/docs/guides/TROUBLESHOOTING.md) | 🇹🇭 [ไทย](../i18n/th/docs/guides/TROUBLESHOOTING.md) | 🇺🇦 [Українська](../i18n/uk-UA/docs/guides/TROUBLESHOOTING.md) | 🇸🇦 [العربية](../i18n/ar/docs/guides/TROUBLESHOOTING.md) | 🇯🇵 [日本語](../i18n/ja/docs/guides/TROUBLESHOOTING.md) | 🇻🇳 [Tiếng Việt](../i18n/vi/docs/guides/TROUBLESHOOTING.md) | 🇧🇬 [Български](../i18n/bg/docs/guides/TROUBLESHOOTING.md) | 🇩🇰 [Dansk](../i18n/da/docs/guides/TROUBLESHOOTING.md) | 🇫🇮 [Suomi](../i18n/fi/docs/guides/TROUBLESHOOTING.md) | 🇮🇱 [עברית](../i18n/he/docs/guides/TROUBLESHOOTING.md) | 🇭🇺 [Magyar](../i18n/hu/docs/guides/TROUBLESHOOTING.md) | 🇮🇩 [Bahasa Indonesia](../i18n/id/docs/guides/TROUBLESHOOTING.md) | 🇰🇷 [한국어](../i18n/ko/docs/guides/TROUBLESHOOTING.md) | 🇲🇾 [Bahasa Melayu](../i18n/ms/docs/guides/TROUBLESHOOTING.md) | 🇳🇱 [Nederlands](../i18n/nl/docs/guides/TROUBLESHOOTING.md) | 🇳🇴 [Norsk](../i18n/no/docs/guides/TROUBLESHOOTING.md) | 🇵🇹 [Português (Portugal)](../i18n/pt/docs/guides/TROUBLESHOOTING.md) | 🇷🇴 [Română](../i18n/ro/docs/guides/TROUBLESHOOTING.md) | 🇵🇱 [Polski](../i18n/pl/docs/guides/TROUBLESHOOTING.md) | 🇸🇰 [Slovenčina](../i18n/sk/docs/guides/TROUBLESHOOTING.md) | 🇸🇪 [Svenska](../i18n/sv/docs/guides/TROUBLESHOOTING.md) | 🇵🇭 [Filipino](../i18n/phi/docs/guides/TROUBLESHOOTING.md) | 🇨🇿 [Čeština](../i18n/cs/docs/guides/TROUBLESHOOTING.md)
|
||||
|
||||
Common problems and solutions for OmniRoute.
|
||||
|
||||
---
|
||||
|
||||
## Quick Reference
|
||||
|
||||
**New to OmniRoute?** Start here — these solve 90% of problems:
|
||||
|
||||
| I see this | What it means | What to do |
|
||||
|------------|--------------|------------|
|
||||
| "Can't connect" | OmniRoute isn't running | Run `omniroute` or `docker restart omniroute` |
|
||||
| "Invalid API key" | Your key is wrong or expired | Re-copy the key from the provider's website |
|
||||
| "Rate limit exceeded" | You're sending too many requests | Wait 1 minute, or use `model: "auto"` for automatic fallback |
|
||||
| "Quota exceeded" | You've used up your free/paid quota | Connect more providers, or use free providers (Kiro, Pollinations) |
|
||||
| "Slow responses" | Provider is busy or far away | Use `model: "auto/fast"` or connect a faster provider (Groq, Cerebras) |
|
||||
| "Wrong provider used" | `auto` picked a different provider | That's normal! `auto` picks the best one. Force a specific provider with `model: "openai/gpt-4o"` |
|
||||
| "502 Bad Gateway" | Provider is down | Wait and retry, or use `model: "auto"` to switch providers |
|
||||
| "401 Unauthorized" | Your credentials are wrong | Check your API key or re-authenticate with OAuth |
|
||||
| "429 Too Many Requests" | Rate limited | Wait 1 minute, or connect more providers |
|
||||
|
||||
**Still stuck?** See the [detailed troubleshooting](#detailed-troubleshooting) below, or ask on [Discord](https://discord.gg/hmexnhgE).
|
||||
|
||||
---
|
||||
|
||||
## Detailed Troubleshooting
|
||||
|
||||
---
|
||||
|
||||
## Quick Fixes
|
||||
|
||||
| Problem | Solution |
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
{
|
||||
"title": "Guides",
|
||||
"description": "Technical guides for operators and developers",
|
||||
"pages": [
|
||||
"SETUP_GUIDE",
|
||||
"USER_GUIDE",
|
||||
@@ -10,7 +11,6 @@
|
||||
"KIRO_SETUP",
|
||||
"PWA_GUIDE",
|
||||
"TERMUX_GUIDE",
|
||||
"TROUBLESHOOTING",
|
||||
"UNINSTALL"
|
||||
]
|
||||
}
|
||||
|
||||
@@ -1,12 +1,12 @@
|
||||
{
|
||||
"pages": [
|
||||
"guides",
|
||||
"architecture",
|
||||
"reference",
|
||||
"frameworks",
|
||||
"routing",
|
||||
"security",
|
||||
"compression",
|
||||
"ops"
|
||||
{
|
||||
"title": "For Non-Tech Users",
|
||||
"pages": ["getting-started", "guides"]
|
||||
},
|
||||
{
|
||||
"title": "For Tech Users",
|
||||
"pages": ["architecture", "reference", "frameworks", "routing", "security", "compression", "ops"]
|
||||
}
|
||||
]
|
||||
}
|
||||
|
||||
@@ -992,6 +992,341 @@ Returns the public A2A agent card (name, description, capabilities, skill catalo
|
||||
|
||||
---
|
||||
|
||||
## ACP (Agent Client Protocol) Management
|
||||
|
||||
The ACP framework lets you spawn CLI agents (Claude Code, Codex, Gemini CLI, etc.)
|
||||
as child processes. These endpoints manage ACP agent detection and custom agent
|
||||
registration.
|
||||
|
||||
| Method | Path | Description |
|
||||
| ------ | ----------------------- | ---------------------------------------------------------------------------------------- |
|
||||
| GET | `/api/acp/agents` | List all known CLI agents (built-in + custom) with installation status, version, binary |
|
||||
| POST | `/api/acp/agents` | Register a custom ACP agent — body: `{id, name, binary, versionCommand, providerAlias, spawnArgs, protocol}` |
|
||||
| DELETE | `/api/acp/agents/[id]` | Remove a custom ACP agent |
|
||||
| POST | `/api/acp/agents/refresh` | Force refresh of the agent detection cache (60s TTL) |
|
||||
|
||||
**Response example** (`GET /api/acp/agents`):
|
||||
|
||||
```json
|
||||
{
|
||||
"agents": [
|
||||
{
|
||||
"id": "claude",
|
||||
"name": "Claude Code CLI",
|
||||
"binary": "claude",
|
||||
"version": "1.0.45",
|
||||
"installed": true,
|
||||
"protocol": "stdio",
|
||||
"providerAlias": "claude",
|
||||
"isCustom": false
|
||||
},
|
||||
{
|
||||
"id": "my-custom-cli",
|
||||
"name": "My Custom CLI",
|
||||
"installed": false,
|
||||
"protocol": "stdio",
|
||||
"providerAlias": "my-provider",
|
||||
"isCustom": true
|
||||
}
|
||||
],
|
||||
"cacheTtlMs": 60000,
|
||||
"cacheAge": 1234
|
||||
}
|
||||
```
|
||||
|
||||
**Auth:** Requires management session (dashboard `auth_token` cookie) or a
|
||||
management-scoped API key.
|
||||
|
||||
See [ACP Framework](../frameworks/ACP.md) for full details.
|
||||
|
||||
---
|
||||
|
||||
## Analytics & Observability
|
||||
|
||||
Real-time analytics endpoints for monitoring routing, compression, and provider
|
||||
diversity. These power the `/dashboard/analytics/*` pages.
|
||||
|
||||
### Auto-routing analytics
|
||||
|
||||
| Method | Path | Description |
|
||||
| ------ | ----------------------------------- | -------------------------------------------------------------------------------------------- |
|
||||
| GET | `/api/analytics/auto-routing` | Aggregate auto-routing stats: total calls, strategy distribution, tier distribution, top providers |
|
||||
| GET | `/api/analytics/auto-routing?days=7` | Time-windowed stats (default 24h) |
|
||||
|
||||
**Response example**:
|
||||
|
||||
```json
|
||||
{
|
||||
"window": "24h",
|
||||
"totalCalls": 1234,
|
||||
"strategyBreakdown": {
|
||||
"rules": 800,
|
||||
"cost": 200,
|
||||
"latency": 150,
|
||||
"sla-aware": 50,
|
||||
"lkgp": 34
|
||||
},
|
||||
"tierBreakdown": {
|
||||
"ultra": 100,
|
||||
"pro": 500,
|
||||
"standard": 400,
|
||||
"free": 234
|
||||
},
|
||||
"topProviders": [
|
||||
{ "provider": "openai", "calls": 500, "avgLatencyMs": 850 },
|
||||
{ "provider": "anthropic", "calls": 300, "avgLatencyMs": 1200 }
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### Compression analytics
|
||||
|
||||
| Method | Path | Description |
|
||||
| ------ | --------------------------------- | ------------------------------------------------------------------------------------ |
|
||||
| GET | `/api/analytics/compression` | Aggregate compression stats: tokens saved, savings %, mode distribution, engine usage |
|
||||
|
||||
**Response example**:
|
||||
|
||||
```json
|
||||
{
|
||||
"window": "24h",
|
||||
"totalOriginalTokens": 5000000,
|
||||
"totalCompressedTokens": 3500000,
|
||||
"totalSavings": 1500000,
|
||||
"savingsPct": 30.0,
|
||||
"modeBreakdown": {
|
||||
"lite": 400,
|
||||
"standard": 600,
|
||||
"aggressive": 100,
|
||||
"ultra": 50,
|
||||
"rtk": 84
|
||||
},
|
||||
"engineBreakdown": {
|
||||
"caveman": 800,
|
||||
"rtk": 434
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Provider diversity tracking
|
||||
|
||||
| Method | Path | Description |
|
||||
| ------ | --------------------------------- | -------------------------------------------------------------------------------------------- |
|
||||
| GET | `/api/analytics/diversity` | Shannon entropy-based diversity tracking: prevents single points of failure by measuring provider spread |
|
||||
|
||||
**Response example**:
|
||||
|
||||
```json
|
||||
{
|
||||
"window": "24h",
|
||||
"shannonEntropy": 2.45,
|
||||
"maxEntropy": 3.17,
|
||||
"diversityRatio": 0.77,
|
||||
"providerUsage": {
|
||||
"openai": 0.40,
|
||||
"anthropic": 0.25,
|
||||
"google": 0.20,
|
||||
"kiro": 0.15
|
||||
},
|
||||
"warnings": [
|
||||
"OpenAI accounts for 40% of traffic — consider diversifying"
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
**Auth:** Requires management session or management-scoped API key.
|
||||
|
||||
---
|
||||
|
||||
## Admin Operations
|
||||
|
||||
Admin-only endpoints for operational management.
|
||||
|
||||
| Method | Path | Description |
|
||||
| ------ | ------------------------------- | ---------------------------------------------------------------------------------------------- |
|
||||
| GET | `/api/admin/concurrency` | Read current concurrency limits (global + per-provider) |
|
||||
| POST | `/api/admin/concurrency` | Update concurrency limits — body: `{global?: number, perProvider?: Record<string, number>}` |
|
||||
| GET | `/api/admin/circuit-breaker` | Read circuit breaker states for all providers |
|
||||
| POST | `/api/admin/circuit-breaker/reset` | Manually reset a circuit breaker — body: `{providerId}` |
|
||||
| GET | `/api/admin/rate-limits` | Read current rate limit configurations |
|
||||
| POST | `/api/admin/rate-limits` | Update rate limit configs — body: `{providerId, requestsPerMinute, tokensPerMinute}` |
|
||||
|
||||
**Auth:** Requires management session with admin scope.
|
||||
|
||||
---
|
||||
|
||||
## CLI Tools Management
|
||||
|
||||
Manage CLI tools that integrate with OmniRoute (antigravity, chipotle, commandCode,
|
||||
devin-cli, etc.). See [Provider Reference](./PROVIDER_REFERENCE.md) for the full list.
|
||||
|
||||
| Method | Path | Description |
|
||||
| ------ | --------------------------------- | -------------------------------------------------------------------------------------------- |
|
||||
| GET | `/api/cli-tools/all-statuses` | Status of all CLI tools (installed, version, last seen) |
|
||||
| GET | `/api/cli-tools/[id]/status` | Status of a specific CLI tool (id can be: antigravity, chipotle, commandCode, devin-cli, etc.) |
|
||||
| POST | `/api/cli-tools/apply` | Apply a CLI tool configuration to a provider connection |
|
||||
| GET | `/api/cli-tools/backups` | List CLI tool configuration backups |
|
||||
| POST | `/api/cli-tools/backups` | Create a backup of all CLI tool configurations |
|
||||
| POST | `/api/cli-tools/[id]/restore` | Restore a CLI tool from a backup |
|
||||
| GET | `/api/cli-tools/antigravity-mitm` | Antigravity MITM proxy status (the "antigravity-mitm" CLI tool) |
|
||||
| POST | `/api/cli-tools/antigravity-mitm/alias` | Configure antigravity-mitm aliases |
|
||||
|
||||
**Auth:** Requires management session.
|
||||
|
||||
---
|
||||
|
||||
## Agent Skills
|
||||
|
||||
Manage AI agent skills (similar to OpenAI's custom GPTs but for agents).
|
||||
|
||||
| Method | Path | Description |
|
||||
| ------ | --------------------------------- | -------------------------------------------------------------------------------------------- |
|
||||
| GET | `/api/agent-skills` | List all agent skills (built-in + custom) |
|
||||
| GET | `/api/agent-skills/[id]` | Get a specific agent skill |
|
||||
| POST | `/api/agent-skills` | Create a custom agent skill — body: `{name, description, prompt, model?, temperature?}` |
|
||||
| PUT | `/api/agent-skills/[id]` | Update a custom agent skill |
|
||||
| DELETE | `/api/agent-skills/[id]` | Delete a custom agent skill |
|
||||
| GET | `/api/agent-skills/[id]/raw` | Get raw prompt + metadata (no execution) |
|
||||
| POST | `/api/agent-skills/generate` | AI-generate a new skill from a natural language description |
|
||||
|
||||
**Auth:** Requires management session or management-scoped API key.
|
||||
|
||||
---
|
||||
|
||||
## Cache Management
|
||||
|
||||
Manage the semantic cache and reasoning cache.
|
||||
|
||||
| Method | Path | Description |
|
||||
| ------ | --------------------------------- | -------------------------------------------------------------------------------------------- |
|
||||
| GET | `/api/cache` | Cache overview: total entries, hit rate, size on disk |
|
||||
| GET | `/api/cache/entries` | List cached entries (with pagination) |
|
||||
| DELETE | `/api/cache/entries` | Delete cache entries (filter by query parameters) |
|
||||
| GET | `/api/cache/stats` | Detailed cache statistics (per-provider, per-model) |
|
||||
| GET | `/api/cache/reasoning` | Reasoning cache status (for reasoning replay) |
|
||||
| POST | `/api/cache/reasoning/clear` | Clear reasoning cache |
|
||||
| POST | `/api/cache/clear` | Clear all cache entries |
|
||||
|
||||
**Auth:** Requires management session.
|
||||
|
||||
---
|
||||
|
||||
## Memory System
|
||||
|
||||
Manage persistent memory (FTS5 + vector embeddings).
|
||||
|
||||
| Method | Path | Description |
|
||||
| ------ | --------------------------------- | -------------------------------------------------------------------------------------------- |
|
||||
| GET | `/api/memory` | List memory entries (filter by scope, type, search query) |
|
||||
| POST | `/api/memory` | Create a new memory entry — body: `{scope, type, content, metadata?}` |
|
||||
| GET | `/api/memory/[id]` | Get a specific memory entry |
|
||||
| PUT | `/api/memory/[id]` | Update a memory entry |
|
||||
| DELETE | `/api/memory/[id]` | Delete a memory entry |
|
||||
| GET | `/api/memory/search` | Search memory (FTS5 + vector) |
|
||||
| POST | `/api/memory/clear` | Clear memory entries (with filters) |
|
||||
| GET | `/api/memory/stats` | Memory statistics (total entries, embedding coverage, etc.) |
|
||||
|
||||
**Auth:** Requires management session or management-scoped API key.
|
||||
|
||||
---
|
||||
|
||||
## Webhooks
|
||||
|
||||
Manage webhook subscriptions for events.
|
||||
|
||||
| Method | Path | Description |
|
||||
| ------ | --------------------------------- | -------------------------------------------------------------------------------------------- |
|
||||
| GET | `/api/webhooks` | List all webhook subscriptions |
|
||||
| POST | `/api/webhooks` | Create a webhook subscription — body: `{url, events[], secret?, active?}` |
|
||||
| GET | `/api/webhooks/[id]` | Get a specific webhook subscription |
|
||||
| PUT | `/api/webhooks/[id]` | Update a webhook subscription |
|
||||
| DELETE | `/api/webhooks/[id]` | Delete a webhook subscription |
|
||||
| GET | `/api/webhooks/events` | List all available webhook event types |
|
||||
| GET | `/api/webhooks/[id]/deliveries` | List delivery history for a webhook (success/failure log) |
|
||||
| POST | `/api/webhooks/[id]/test` | Send a test event to a webhook |
|
||||
|
||||
**Auth:** Requires management session.
|
||||
|
||||
See [Webhooks Framework](../frameworks/WEBHOOKS.md) for full event types.
|
||||
|
||||
---
|
||||
|
||||
## Skills Framework
|
||||
|
||||
Manage Skills (the agentic extensions framework).
|
||||
|
||||
| Method | Path | Description |
|
||||
| ------ | --------------------------------- | -------------------------------------------------------------------------------------------- |
|
||||
| GET | `/api/skills` | List all installed skills (built-in + custom) |
|
||||
| POST | `/api/skills/install` | Install a skill from a local path or URL |
|
||||
| DELETE | `/api/skills/[id]` | Uninstall a skill |
|
||||
| POST | `/api/skills/[id]/enable` | Enable a disabled skill |
|
||||
| POST | `/api/skills/[id]/disable` | Disable an enabled skill |
|
||||
| POST | `/api/skills/[id]/execute` | Execute a skill with input |
|
||||
| GET | `/api/skills/[id]/executions` | List execution history for a skill |
|
||||
|
||||
**Auth:** Requires management session or management-scoped API key.
|
||||
|
||||
See [Skills Framework](../frameworks/SKILLS.md) for full details.
|
||||
|
||||
---
|
||||
|
||||
## Plugins
|
||||
|
||||
Manage OmniRoute plugins (third-party extensions).
|
||||
|
||||
| Method | Path | Description |
|
||||
| ------ | --------------------------------- | -------------------------------------------------------------------------------------------- |
|
||||
| GET | `/api/plugins` | List installed plugins |
|
||||
| POST | `/api/plugins/install` | Install a plugin from a local path or URL |
|
||||
| DELETE | `/api/plugins/[id]` | Uninstall a plugin |
|
||||
| POST | `/api/plugins/[id]/enable` | Enable a disabled plugin |
|
||||
| POST | `/api/plugins/[id]/disable` | Disable an enabled plugin |
|
||||
| GET | `/api/plugins/[id]/config` | Get plugin configuration |
|
||||
| PUT | `/api/plugins/[id]/config` | Update plugin configuration |
|
||||
|
||||
**Auth:** Requires management session.
|
||||
|
||||
See [Plugins Framework](../plugins/PLUGIN_SDK.md) for full details.
|
||||
|
||||
---
|
||||
|
||||
## Shadow Routing
|
||||
|
||||
Beta-test new providers without affecting production traffic.
|
||||
|
||||
| Method | Path | Description |
|
||||
| ------ | --------------------------------- | -------------------------------------------------------------------------------------------- |
|
||||
| GET | `/api/shadow` | List all shadow routing rules |
|
||||
| POST | `/api/shadow` | Create a shadow routing rule — body: `{providerId, shadowProviderId, trafficPct, duration?}` |
|
||||
| DELETE | `/api/shadow/[id]` | Delete a shadow routing rule |
|
||||
| GET | `/api/shadow/[id]/results` | Get shadow routing results (comparison metrics between real and shadow traffic) |
|
||||
| GET | `/api/shadow/metrics` | Aggregate shadow routing metrics across all rules |
|
||||
|
||||
**Auth:** Requires management session.
|
||||
|
||||
---
|
||||
|
||||
## Guardrails
|
||||
|
||||
Manage runtime guardrails (PII detection, prompt injection detection, vision bridging).
|
||||
|
||||
| Method | Path | Description |
|
||||
| ------ | --------------------------------- | -------------------------------------------------------------------------------------------- |
|
||||
| GET | `/api/guardrails` | List all guardrails and their status (enabled/disabled) |
|
||||
| POST | `/api/guardrails/[id]/enable` | Enable a guardrail |
|
||||
| POST | `/api/guardrails/[id]/disable` | Disable a guardrail |
|
||||
| GET | `/api/guardrails/logs` | Get guardrail trigger logs (PII detections, injection attempts, etc.) |
|
||||
| POST | `/api/guardrails/test` | Test a guardrail against a sample input |
|
||||
|
||||
**Auth:** Requires management session.
|
||||
|
||||
See [Security > Guardrails](../security/GUARDRAILS.md) for full details.
|
||||
|
||||
---
|
||||
|
||||
---
|
||||
|
||||
## Authentication
|
||||
|
||||
- Dashboard routes (`/dashboard/*`) use `auth_token` cookie
|
||||
|
||||
@@ -6,6 +6,8 @@ lastUpdated: 2026-06-05
|
||||
|
||||
# Free Tiers & Free-Token Budget
|
||||
|
||||
> **For Users**: Looking for a simple guide? See the [Free Tiers Guide](../getting-started/FREE-TIERS-GUIDE.md) for step-by-step instructions on getting free AI.
|
||||
|
||||
> **Last researched:** 2026-06-05 — per-provider web research of current free-tier quotas + ToS (98 providers).
|
||||
> **Source of truth (catalog):** `src/shared/constants/providers.ts` (`hasFree: true` + `freeNote`). The token-budget numbers below come from live web research and are an **approximation** — see [Methodology & caveats](#methodology--caveats).
|
||||
|
||||
|
||||
@@ -6,6 +6,8 @@ lastUpdated: 2026-06-06
|
||||
|
||||
# Provider Reference
|
||||
|
||||
> **For Users**: Looking for a simple guide? See the [Providers Guide](../getting-started/PROVIDERS-GUIDE.md) for step-by-step instructions.
|
||||
|
||||
> **Auto-generated** from `src/shared/constants/providers.ts` — do not edit by hand.
|
||||
> Regenerate with: `npm run gen:provider-reference`
|
||||
> **Last generated:** 2026-06-06
|
||||
|
||||
@@ -6,6 +6,8 @@ lastUpdated: 2026-05-13
|
||||
|
||||
# OmniRoute Auto-Combo Engine
|
||||
|
||||
> **For Users**: Looking for a quick start? See the [Auto-Combo User Guide](../getting-started/AUTO-COMBO-GUIDE.md) for simple explanations and examples.
|
||||
|
||||
> Self-managing model chains with adaptive scoring + zero-config auto-routing
|
||||
|
||||
## Zero-Config Auto-Routing (`auto/` prefix)
|
||||
@@ -107,11 +109,11 @@ Four pre-defined weight profiles in `open-sse/services/autoCombo/modePacks.ts`.
|
||||
|
||||
| Factor | ship-fast | cost-saver | quality-first | offline-friendly |
|
||||
| :----------- | :-------- | :--------- | :------------ | :--------------- |
|
||||
| quota | 0.15 | 0.15 | 0.10 | **0.40** |
|
||||
| health | 0.30 | 0.20 | 0.20 | 0.30 |
|
||||
| costInv | 0.05 | **0.40** | 0.05 | 0.10 |
|
||||
| latencyInv | **0.35** | 0.05 | 0.05 | 0.05 |
|
||||
| taskFit | 0.10 | 0.10 | **0.40** | 0.00 |
|
||||
| quota | 0.14 | 0.14 | 0.10 | **0.37** |
|
||||
| health | 0.28 | 0.19 | 0.18 | 0.28 |
|
||||
| costInv | 0.05 | **0.37** | 0.05 | 0.10 |
|
||||
| latencyInv | **0.32** | 0.05 | 0.05 | 0.05 |
|
||||
| taskFit | 0.10 | 0.10 | **0.37** | 0.00 |
|
||||
| stability | 0.00 | 0.05 | 0.15 | 0.10 |
|
||||
| tierPriority | 0.05 | 0.05 | 0.05 | 0.05 |
|
||||
|
||||
@@ -119,10 +121,10 @@ Notes:
|
||||
|
||||
- `tierAffinity` and `specificityMatch` are not set in mode packs — `calculateScore()` treats them as `?? 0` when absent.
|
||||
- Each pack's emphasis at a glance:
|
||||
- **ship-fast** → latencyInv 0.35 + health 0.30 (low-latency, healthy connections)
|
||||
- **cost-saver** → costInv 0.40 (cheapest tokens win)
|
||||
- **quality-first** → taskFit 0.40 + stability 0.15 (best model for the task, consistent)
|
||||
- **offline-friendly** → quota 0.40 + health 0.30 (max headroom regardless of speed/cost)
|
||||
- **ship-fast** → latencyInv 0.32 + health 0.28 (low-latency, healthy connections)
|
||||
- **cost-saver** → costInv 0.37 (cheapest tokens win)
|
||||
- **quality-first** → taskFit 0.37 + stability 0.15 (best model for the task, consistent)
|
||||
- **offline-friendly** → quota 0.37 + health 0.28 (max headroom regardless of speed/cost)
|
||||
|
||||
## All Routing Strategies
|
||||
|
||||
@@ -205,6 +207,227 @@ Persisted `strategy: "auto"` combos can set `config.routerStrategy` (or legacy
|
||||
cost SLOs
|
||||
- `lkgp` — last known good provider first
|
||||
|
||||
### Router strategies in detail
|
||||
|
||||
The auto-combo engine exposes 5 pluggable **RouterStrategy** implementations that
|
||||
you can swap via `config.routerStrategy` (or the legacy `config.auto.routerStrategy`).
|
||||
Each strategy picks one provider from the candidate pool, given a `RoutingContext`
|
||||
(task type, tool/vision hints, token estimate, optional SLA policy, optional
|
||||
last-known-good provider).
|
||||
|
||||
#### 1. `rules` (default) — 6-factor weighted scoring
|
||||
|
||||
Wraps the existing scoring engine. Filters out `OPEN` circuit-breaker
|
||||
candidates, then runs `scorePool()` with the current task type and `getTaskFitness()`,
|
||||
picking the top-scoring provider.
|
||||
|
||||
```ts
|
||||
class RulesStrategyImpl implements RouterStrategy {
|
||||
readonly name = "rules";
|
||||
readonly description =
|
||||
"6-factor weighted scoring: quota, health, cost, latency, taskFit, stability";
|
||||
|
||||
select(pool, context) {
|
||||
const eligible = pool.filter((c) => c.circuitBreakerState !== "OPEN");
|
||||
const ranked = scorePool(eligible.length > 0 ? eligible : pool, context.taskType, undefined, getTaskFitness);
|
||||
return { provider: ranked[0].provider, /* ... */ };
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**When to use**: Default. Use when you want a balanced trade-off across all signals.
|
||||
|
||||
**Alias**: `rules` (no alias)
|
||||
|
||||
---
|
||||
|
||||
#### 2. `cost` / `eco` — cheapest healthy provider
|
||||
|
||||
Sorts the candidate pool by `costPer1MTokens` (ascending) and picks the cheapest.
|
||||
Filters out `OPEN` candidates first.
|
||||
|
||||
```ts
|
||||
class CostStrategyImpl implements RouterStrategy {
|
||||
readonly name = "cost";
|
||||
readonly description = "Always selects cheapest available provider";
|
||||
|
||||
select(pool, context) {
|
||||
const healthy = pool.filter((c) => c.circuitBreakerState !== "OPEN");
|
||||
const sorted = [...healthy].sort((a, b) => a.costPer1MTokens - b.costPer1MTokens);
|
||||
return { provider: sorted[0].provider, /* ... */ };
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**When to use**: Cost-sensitive workloads, batch processing, or background jobs.
|
||||
|
||||
**Aliases**: `cost`, `eco`
|
||||
|
||||
---
|
||||
|
||||
#### 3. `latency` / `fast` — lowest p95 latency with reliability penalty
|
||||
|
||||
Sorts by `p95LatencyMs + (errorRate * 1000)`. The error-rate penalty ensures
|
||||
unreliable providers are ranked lower even if their nominal latency is low.
|
||||
|
||||
```ts
|
||||
class LatencyStrategyImpl implements RouterStrategy {
|
||||
readonly name = "latency";
|
||||
readonly description = "Prioritizes lowest p95 latency with reliability weighting";
|
||||
|
||||
select(pool, context) {
|
||||
const healthy = pool.filter((c) => c.circuitBreakerState !== "OPEN");
|
||||
const sorted = [...healthy].sort((a, b) =>
|
||||
(a.p95LatencyMs + a.errorRate * 1000) - (b.p95LatencyMs + b.errorRate * 1000)
|
||||
);
|
||||
return { provider: sorted[0].provider, /* ... */ };
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**When to use**: Latency-sensitive workloads like real-time chat, autocomplete, or
|
||||
interactive coding assistants.
|
||||
|
||||
**Aliases**: `latency`, `fast`
|
||||
|
||||
---
|
||||
|
||||
#### 4. `sla-aware` / `sla` — latency/error/cost SLO compliance
|
||||
|
||||
Scores each candidate by how well it satisfies the configured SLO policy:
|
||||
|
||||
| Factor | Weight | Formula |
|
||||
|--------|--------|---------|
|
||||
| Latency score | 35% | `threshold / max(value, ε)` |
|
||||
| Error score | 35% | `threshold / max(value, ε)` |
|
||||
| Health score | 15% | `1.0` (CLOSED) / `0.5` (HALF_OPEN) / `0.0` (OPEN) |
|
||||
| Cost score | 10% | `threshold / max(value, ε)` or inverse normalized |
|
||||
| Stability score | 5% | inverse normalized latency stddev |
|
||||
|
||||
When `hardConstraints: true`, candidates are sorted primarily by **violation score**
|
||||
(how far they exceed any SLO), then by composite score. Otherwise it's just
|
||||
the composite score.
|
||||
|
||||
```ts
|
||||
class SLAStrategyImpl implements RouterStrategy {
|
||||
readonly name = "sla-aware";
|
||||
readonly description = "Selects the provider most likely to satisfy latency, error-rate, and cost SLOs";
|
||||
|
||||
select(pool, context) {
|
||||
// ... scores each candidate against policy: { targetP95Ms, maxErrorRate, maxCostPer1MTokens, hardConstraints }
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**SLA fields** (set on the combo config):
|
||||
|
||||
```json
|
||||
{
|
||||
"strategy": "auto",
|
||||
"config": {
|
||||
"routerStrategy": "sla-aware",
|
||||
"slaTargetP95Ms": 1500,
|
||||
"slaMaxErrorRate": 0.05,
|
||||
"slaMaxCostPer1MTokens": 5,
|
||||
"slaHardConstraints": true
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**When to use**: Production workloads with strict latency, error-rate, or cost budgets.
|
||||
|
||||
**Aliases**: `sla-aware`, `sla`
|
||||
|
||||
---
|
||||
|
||||
#### 5. `lkgp` — last known good provider first
|
||||
|
||||
Tries the **last known good provider** (if set) first, then falls back to the
|
||||
`rules` strategy. Useful for session stickiness — the same provider handles
|
||||
follow-up requests in a conversation.
|
||||
|
||||
```ts
|
||||
class LKGPStrategyImpl implements RouterStrategy {
|
||||
readonly name = "lkgp";
|
||||
readonly description = "Tries last known good provider first, then falls back to rules";
|
||||
|
||||
select(pool, context) {
|
||||
if (context.lkgpEnabled === false) {
|
||||
return getStrategy("rules").select(pool, context);
|
||||
}
|
||||
|
||||
if (context.lastKnownGoodProvider) {
|
||||
const candidates = pool.filter(
|
||||
(c) => c.provider === context.lastKnownGoodProvider && c.circuitBreakerState !== "OPEN"
|
||||
);
|
||||
if (candidates.length > 0) {
|
||||
return { provider: candidates[0].provider, /* ... */ };
|
||||
}
|
||||
}
|
||||
|
||||
// Fallback to rules strategy
|
||||
return getStrategy("rules").select(pool, context);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**When to use**: Multi-turn conversations where you want the same provider to handle
|
||||
follow-up requests (e.g., for caching, context continuity, or pricing consistency).
|
||||
|
||||
**Alias**: `lkgp` (no alias)
|
||||
|
||||
---
|
||||
|
||||
### Custom router strategies
|
||||
|
||||
You can register your own `RouterStrategy` implementation via the public API:
|
||||
|
||||
```ts
|
||||
import { registerStrategy, type RouterStrategy } from "@omniroute/open-sse/services/autoCombo/routerStrategy";
|
||||
|
||||
class MyCustomStrategy implements RouterStrategy {
|
||||
readonly name = "my-custom";
|
||||
readonly description = "My custom routing strategy";
|
||||
|
||||
select(pool, context) {
|
||||
// Your routing logic here
|
||||
return {
|
||||
provider: pool[0].provider,
|
||||
model: pool[0].model,
|
||||
strategy: this.name,
|
||||
reason: "MyCustomStrategy: ...",
|
||||
candidatesConsidered: pool.length,
|
||||
finalScore: 1.0,
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
registerStrategy("my-custom", new MyCustomStrategy());
|
||||
```
|
||||
|
||||
Then use it:
|
||||
|
||||
```json
|
||||
{
|
||||
"strategy": "auto",
|
||||
"config": {
|
||||
"routerStrategy": "my-custom"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Router strategy selection guide
|
||||
|
||||
| Use case | Strategy | Reason |
|
||||
|---------|----------|--------|
|
||||
| Balanced workload | `rules` | Default — considers all factors |
|
||||
| Minimize cost | `cost` | Always picks cheapest |
|
||||
| Minimize latency | `latency` | Picks fastest reliable provider |
|
||||
| Strict SLOs | `sla-aware` | Filters by p95/error/cost thresholds |
|
||||
| Multi-turn chat | `lkgp` | Session stickiness |
|
||||
|
||||
SLA-aware fields:
|
||||
|
||||
```json
|
||||
|
||||
@@ -22,34 +22,40 @@ export const metadata: Metadata = {
|
||||
|
||||
const featuredLinks = [
|
||||
{
|
||||
href: "/docs/guides/setup-guide",
|
||||
title: "Setup Guide",
|
||||
href: "/docs/getting-started/quick-start",
|
||||
title: "Quick Start",
|
||||
icon: "rocket_launch",
|
||||
desc: "Get OmniRoute running in 3 minutes",
|
||||
},
|
||||
{
|
||||
href: "/docs/reference/api-reference",
|
||||
title: "API Reference",
|
||||
icon: "code",
|
||||
desc: "All endpoints with examples",
|
||||
href: "/docs/getting-started/auto-combo-guide",
|
||||
title: "Auto-Combo Guide",
|
||||
icon: "auto_awesome",
|
||||
desc: "Let OmniRoute pick the best AI for you",
|
||||
},
|
||||
{
|
||||
href: "/docs/compression/compression-guide",
|
||||
title: "Compression Guide",
|
||||
icon: "compress",
|
||||
desc: "Save 15-95% eligible tokens automatically",
|
||||
href: "/docs/getting-started/providers-guide",
|
||||
title: "Providers Guide",
|
||||
icon: "link",
|
||||
desc: "Connect AI providers in minutes",
|
||||
},
|
||||
];
|
||||
|
||||
const sections = [
|
||||
{ title: "Architecture", folder: "architecture" },
|
||||
{ title: "Guides", folder: "guides" },
|
||||
{ title: "Reference", folder: "reference" },
|
||||
{ title: "Frameworks", folder: "frameworks" },
|
||||
{ title: "Routing", folder: "routing" },
|
||||
{ title: "Security", folder: "security" },
|
||||
{ title: "Compression", folder: "compression" },
|
||||
{ title: "Operations", folder: "ops" },
|
||||
{
|
||||
title: "For Non-Tech Users",
|
||||
subtitle: "Get started quickly — no technical background needed",
|
||||
icon: "rocket_launch",
|
||||
color: "green",
|
||||
folders: ["getting-started", "guides"],
|
||||
},
|
||||
{
|
||||
title: "For Tech Users",
|
||||
subtitle: "Deep dive into architecture, APIs, and internals",
|
||||
icon: "code",
|
||||
color: "blue",
|
||||
folders: ["architecture", "reference", "frameworks", "routing", "security", "compression", "ops"],
|
||||
},
|
||||
];
|
||||
|
||||
export default function DocsHomePage() {
|
||||
@@ -92,13 +98,23 @@ export default function DocsHomePage() {
|
||||
|
||||
<div className="grid grid-cols-1 lg:grid-cols-2 gap-5 pb-12">
|
||||
{sections.map((section) => {
|
||||
const sectionPages = pages.filter((p) => p.url.startsWith(`/docs/${section.folder}/`));
|
||||
const sectionPages = pages.filter((p) =>
|
||||
section.folders.some((folder) => p.url.startsWith(`/docs/${folder}/`))
|
||||
);
|
||||
return (
|
||||
<div
|
||||
key={section.folder}
|
||||
key={section.title}
|
||||
className="border border-fd-border rounded-xl p-6 hover:border-fd-primary/30 transition-colors bg-fd-card/50"
|
||||
>
|
||||
<h2 className="text-base font-semibold text-fd-foreground mb-4">{section.title}</h2>
|
||||
<div className="flex items-center gap-3 mb-4">
|
||||
<span className="material-symbols-outlined text-2xl text-fd-primary">
|
||||
{section.icon}
|
||||
</span>
|
||||
<div>
|
||||
<h2 className="text-base font-semibold text-fd-foreground">{section.title}</h2>
|
||||
<p className="text-sm text-fd-muted-foreground">{section.subtitle}</p>
|
||||
</div>
|
||||
</div>
|
||||
<ul className="space-y-2.5">
|
||||
{sectionPages.map((page) => (
|
||||
<li key={page.url}>
|
||||
|
||||
Reference in New Issue
Block a user