mirror of
https://github.com/diegosouzapw/OmniRoute.git
synced 2026-09-14 02:42:24 +03:00
a86b9019a8
The weight table said stability accounts for "low latency stdDev / error rate". Grep errorRate in scoring.ts and you find it declared on ProviderCandidate and read nowhere — while combo.ts pulls 24 hours of usage history behind a ten-sample floor, falls back to real-time metrics, and hands every candidate an errorRate the scorer ignores. Two candidates, one failing 1% of calls and one failing 99%, scored identically at 0.459486. This declares reliability as a sixteenth factor: 1 - failureRate, using the same formula, field precedence and rate-bounding speedRanking.ts already applies, so a corrupt reading means "nothing observed" rather than "fails every call". It ships at weight 0, leaving the ranking unchanged to the digit — the honest default, since which weight this deserves is a product call backed by traffic the author does not have. Two declared-but-silent factors already ship (cacheAffinity, resetWindowAffinity), so the pattern is not new. The stability row now describes what that factor actually computes: latency variance. The rest is the mechanical 15 → 16 across nineteen documents and the forty-two llm.txt mirrors — sourced from check:docs-counts rather than a grep, the first real use of the gate #12316 extended. Protected-surface note: this PR touches AGENTS.md, llm.txt and its 42 mirrors, and skills/omni-combos-routing/SKILL.md. Every changed line in those 45 files is a digit substitution and nothing else — masking all digits makes the removed and added lines identical, with no sentence added, removed or reworded. Reviewed and approved on that basis before merging. Verified on the author's rebased head: check:docs-counts green (the gate that now enforces the count this PR moves), typecheck:core clean, and 71/71 focused tests across scoring-reliability-factor, combo-scoring-weights-schema-coverage, check-docs-counts-sync, lkgp-enabled-context, intelligent-routing-options and the combo-matrix auto integration suite. Thanks @maxmad64bis — shipping the factor at weight 0 and saying plainly that the weight is someone else's call is the right way to land this.
222 lines
8.0 KiB
Markdown
222 lines
8.0 KiB
Markdown
---
|
|
title: "Auto-Combo: Let OmniRoute Pick the Best AI for You"
|
|
version: 3.8.50
|
|
lastUpdated: 2026-08-06
|
|
---
|
|
|
|
# Auto-Combo: Let OmniRoute Pick the Best AI for You
|
|
|
|
> **TL;DR**: Set your model to `auto` and OmniRoute automatically picks the best AI provider for each request. No configuration needed.
|
|
|
|
---
|
|
|
|
## What It Does
|
|
|
|
Instead of choosing a specific AI model (like GPT-4o or Claude), you can let OmniRoute **automatically pick the best one** for each request. It considers:
|
|
|
|
- **Health** — Is the provider working right now?
|
|
- **Speed** — How fast is it?
|
|
- **Cost** — How much does it cost?
|
|
- **Quality** — Is it good at this type of task?
|
|
- **Capacity** — Does it have quota remaining?
|
|
|
|
OmniRoute scores all your connected providers and picks the best one. If it fails, it automatically tries the next one.
|
|
|
|
---
|
|
|
|
## Quick Start
|
|
|
|
**Step 1**: Set your model to `auto` in your IDE or CLI:
|
|
|
|
```
|
|
model: "auto"
|
|
```
|
|
|
|
**Step 2**: That's it! OmniRoute handles the rest.
|
|
|
|
**Step 3** (optional): Use a variant for specific tasks:
|
|
|
|
```
|
|
model: "auto/coding" # Best for code
|
|
model: "auto/fast" # Fastest response
|
|
model: "auto/cheap" # Cheapest option
|
|
```
|
|
|
|
---
|
|
|
|
## Which "auto" Should I Use?
|
|
|
|
| If you want... | Use this | Best for | How it works |
|
|
| -------------------- | -------------- | -------------------------- | ----------------------------------- |
|
|
| **Best overall** | `auto` | General questions, chat | Balances speed, cost, and quality |
|
|
| **Best code** | `auto/coding` | Writing code, debugging | Picks models good at coding tasks |
|
|
| **Fastest response** | `auto/fast` | Quick answers, low latency | Prioritizes speed over everything |
|
|
| **Cheapest option** | `auto/cheap` | Saving money | Picks the cheapest provider |
|
|
| **Smartest model** | `auto/smart` | Complex tasks | Quality-first + explores new models |
|
|
| **Most available** | `auto/offline` | When providers are busy | Picks providers with most capacity |
|
|
|
|
### Examples
|
|
|
|
```bash
|
|
# General chat — balanced
|
|
curl http://localhost:20128/v1/chat/completions \
|
|
-H "Content-Type: application/json" \
|
|
-d '{"model":"auto","messages":[{"role":"user","content":"Hello!"}]}'
|
|
|
|
# Code generation — quality-first
|
|
curl http://localhost:20128/v1/chat/completions \
|
|
-H "Content-Type: application/json" \
|
|
-d '{"model":"auto/coding","messages":[{"role":"user","content":"Write a Python function"}]}'
|
|
|
|
# Quick answer — speed-first
|
|
curl http://localhost:20128/v1/chat/completions \
|
|
-H "Content-Type: application/json" \
|
|
-d '{"model":"auto/fast","messages":[{"role":"user","content":"What is 2+2?"}]}'
|
|
```
|
|
|
|
---
|
|
|
|
## How It Works (Simple Version)
|
|
|
|
When you send a request with `model: "auto"`, OmniRoute:
|
|
|
|
1. **Looks at all your connected providers** — Every provider you've added (OpenAI, Anthropic, Google, etc.)
|
|
2. **Scores each one**, weighing among other things:
|
|
- Is it working? (health)
|
|
- Does it have capacity? (quota)
|
|
- How much does it cost? (price)
|
|
- How fast is it? (speed)
|
|
- Is it good at this task? (quality)
|
|
3. **Picks the best one** — The highest-scoring provider gets your request
|
|
4. **Auto-recovers** — If it fails, OmniRoute tries the next one automatically
|
|
|
|
### The Scoring System
|
|
|
|
Each provider gets a score from 0 to 1. The higher the score, the better the fit.
|
|
|
|
| Factor | Weight | What it means |
|
|
| --------- | ------ | ------------------------------------------------ |
|
|
| Health | 20% | Is the provider working? (circuit breaker state) |
|
|
| Quota | 15% | Does it have capacity remaining? |
|
|
| Cost | 15% | How expensive is it? (cheaper = higher score) |
|
|
| Speed | 12% | How fast is it? (lower latency = higher score) |
|
|
| Task Fit | 8% | Is it good at this type of task? |
|
|
| Stability | 5% | Is it consistent? (low error rate) |
|
|
| Tier | 5% | Account tier (Ultra > Pro > Free) |
|
|
| Other | 20% | Context affinity, connection density, etc. |
|
|
|
|
### How Variants Change the Scoring
|
|
|
|
Each variant uses different weights:
|
|
|
|
| Variant | Prioritizes | Key Weights |
|
|
| -------------- | ----------------- | ------------------------------- |
|
|
| `auto` | Balanced | health=20%, quota=15%, cost=15% |
|
|
| `auto/coding` | Quality | taskFit=37%, stability=15% |
|
|
| `auto/fast` | Speed | latency=32%, health=28% |
|
|
| `auto/cheap` | Cost | cost=37% |
|
|
| `auto/smart` | Quality + Explore | taskFit=37%, exploration=10% |
|
|
| `auto/offline` | Capacity | quota=37%, health=28% |
|
|
|
|
---
|
|
|
|
## How It Handles Failures
|
|
|
|
OmniRoute has **three layers of protection**:
|
|
|
|
### 1. Auto-Fallback
|
|
|
|
If the best provider fails, OmniRoute automatically tries the next one. You don't need to do anything.
|
|
|
|
### 2. Self-Healing
|
|
|
|
If a provider keeps failing:
|
|
|
|
- **Score < 0.2** → Excluded for 5 minutes
|
|
- **Circuit breaker open** → Auto-excluded
|
|
- **More than 50% providers down** → Incident mode (no exploration)
|
|
|
|
### 3. Emergency Fallback
|
|
|
|
If all providers fail, OmniRoute routes to stable free providers (like Kiro or Qoder) as a last resort.
|
|
|
|
---
|
|
|
|
## Multi-Account Support
|
|
|
|
If you have multiple accounts for the same provider (e.g., two OpenAI keys), OmniRoute treats each as a **separate candidate**. This means:
|
|
|
|
- Account A has quota remaining → use it
|
|
- Account B is rate-limited → skip it
|
|
- Account C is cheaper → prefer it
|
|
|
|
Each account is scored independently based on its own health, quota, and speed.
|
|
|
|
---
|
|
|
|
## Bandit Exploration
|
|
|
|
OmniRoute occasionally **explores** new providers to discover better options:
|
|
|
|
- **Default**: 5% of requests go to random providers
|
|
- **Auto/smart**: 10% exploration rate
|
|
- **Disabled** when more than 50% of providers are unhealthy
|
|
|
|
This helps OmniRoute learn which providers work best for your usage patterns.
|
|
|
|
---
|
|
|
|
## Common Questions
|
|
|
|
### "Will it always pick the most expensive model?"
|
|
|
|
**No.** Cost is only 15% of the score by default. A cheap, fast, healthy provider can beat an expensive one. Use `auto/cheap` if you want to prioritize cost even more.
|
|
|
|
### "What if a provider goes down?"
|
|
|
|
OmniRoute automatically skips it and tries the next one. If a provider keeps failing, it's excluded temporarily (5-30 minutes). You don't need to do anything.
|
|
|
|
### "Can I see which provider was used?"
|
|
|
|
Check the response headers — OmniRoute includes the provider and model used in each response.
|
|
|
|
### "Does it learn from my usage?"
|
|
|
|
Yes! The scoring system uses historical data (latency, error rates, success rates) to make better decisions over time.
|
|
|
|
### "What's the difference between `auto` and `auto/smart`?"
|
|
|
|
- `auto` — Balanced, 5% exploration
|
|
- `auto/smart` — Quality-first (same weights as `auto/coding`), 10% exploration
|
|
|
|
Use `auto/smart` when you want the best quality and are okay with occasional exploration.
|
|
|
|
### "Can I force a specific provider?"
|
|
|
|
Yes! Use a combo with `priority` strategy instead of `auto`, then send the combo's **exact name** as the `model` field (e.g. `model: "my-combo"` — not `auto`). See the [Technical Reference](../routing/AUTO-COMBO.md) for details.
|
|
|
|
### "How is this different from round-robin?"
|
|
|
|
Round-robin cycles through providers in order. Auto-combo **scores each provider** and picks the best one. It's smarter — it considers health, speed, cost, and quality.
|
|
|
|
---
|
|
|
|
## What's Next?
|
|
|
|
- **[Connect a Provider](./PROVIDERS-GUIDE.md)** — Add your first AI provider
|
|
- **[Free Tiers Guide](./FREE-TIERS-GUIDE.md)** — Get free AI with no credit card
|
|
- **[Troubleshooting](../guides/TROUBLESHOOTING.md)** — Fix common issues
|
|
- **[Technical Reference](../routing/AUTO-COMBO.md)** — Deep dive into the scoring algorithm
|
|
|
|
---
|
|
|
|
## Learn More
|
|
|
|
For developers and contributors, see the [Auto-Combo Technical Reference](../routing/AUTO-COMBO.md) for:
|
|
|
|
- Full 16-factor scoring algorithm
|
|
- Mode pack weight tables
|
|
- Implementation file paths
|
|
- API endpoints
|
|
- Self-healing algorithm details
|