mirror of
https://github.com/diegosouzapw/OmniRoute.git
synced 2026-09-19 05:12:16 +03:00
docs: Chaos Mode setup guide + weighted strategy semantics (#12250)
Gap surfaced by OmniCopilot#16: the Chaos Mode dashboard page, its per-key chaosModeEnabled permission and both dispatch endpoints had no setup doc at all (only the auto/chaos table line existed), and AUTO-COMBO.md never stated that weighted is a proportional draw where zero-weight steps are never drawn. New docs/guides/CHAOS-MODE.md (registered in meta.json + docs/README.md) and a 'weighted semantics' subsection under the strategy table, both written from the code (chaosConfig.ts, chaosExecutor.ts, both routes, targetSorters.ts, targetResolution.ts). check:docs-all exits 0.
This commit is contained in:
Diego Rodrigues de Sa e Souza
committed by
GitHub
GitHub
parent
6c93e74f26
commit
aa2aec5e59
@@ -30,6 +30,7 @@ Simple guides for using OmniRoute — no technical background needed.
|
||||
- [USER_GUIDE.md](guides/USER_GUIDE.md) — daily usage of the dashboard and API.
|
||||
- [THINKING_BUDGET.md](guides/THINKING_BUDGET.md) — thinking/reasoning budget modes (passthrough vs auto-strip).
|
||||
- [FEATURES.md](guides/FEATURES.md) — dashboard feature gallery.
|
||||
- [CHAOS-MODE.md](guides/CHAOS-MODE.md) — multi-model parallel/collaborative execution (setup, permissions, API).
|
||||
- [TIERS.md](guides/TIERS.md) — OmniRoute tiers explained (user guide).
|
||||
- [USAGE_QUOTA_GUIDE.md](guides/USAGE_QUOTA_GUIDE.md) — usage, quota & spend tracking.
|
||||
- [COST_TRACKING.md](guides/COST_TRACKING.md) — cost and spend tracking.
|
||||
|
||||
109
docs/guides/CHAOS-MODE.md
Normal file
109
docs/guides/CHAOS-MODE.md
Normal file
@@ -0,0 +1,109 @@
|
||||
---
|
||||
title: "Chaos Mode"
|
||||
version: 3.8.51
|
||||
lastUpdated: 2026-09-01
|
||||
---
|
||||
|
||||
# Chaos Mode
|
||||
|
||||
> **Dashboard:** **Chaos Mode** (sidebar) → `/dashboard/chaos`
|
||||
> **API:** `GET` / `PUT` `/api/chaos/config` · `POST /api/chaos/run` (dashboard session) · `POST /api/skills/collect/chaos` (API key)
|
||||
> **Source:** `src/lib/chaos/chaosExecutor.ts`, `src/lib/chaos/chaosConfig.ts`
|
||||
|
||||
Chaos Mode sends **one task to several providers at once** — every participating provider
|
||||
contributes one model instance, and you get all the answers side by side (or chained). It is a
|
||||
multi-model execution surface, not a routing strategy: your normal `/v1/chat/completions`
|
||||
traffic is never affected by it.
|
||||
|
||||
**Disambiguation — three different things ship with "chaos" in the name:**
|
||||
|
||||
| Thing | What it is | Where documented |
|
||||
| ------------------- | -------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------- |
|
||||
| **Chaos Mode** | The dashboard page + API described here: fan one task out to many providers (parallel or collaborative). | This guide |
|
||||
| `auto/chaos` | An Auto-Combo model id with fault-injection scoring weights, for resilience testing. Nothing to configure. | [AUTO-COMBO.md](../routing/AUTO-COMBO.md) |
|
||||
| Chaos combo config | A persisted combo with `config.chaos.enabled` fans out to a panel with an optional judge model (API-only). | `open-sse/services/autoCombo/chaosEngine.ts` |
|
||||
|
||||
## Setup
|
||||
|
||||
1. Open **Dashboard → Chaos Mode** (`/dashboard/chaos`).
|
||||
2. Turn it **on** — Chaos Mode ships **disabled by default** (`enabled: false` in
|
||||
`src/lib/chaos/chaosConfig.ts`). While disabled, `POST /api/chaos/run` answers
|
||||
`400 — "Chaos Mode is not enabled. Enable it in Dashboard → Chaos Mode."`.
|
||||
3. Pick the participants and defaults (persisted per instance via the settings store):
|
||||
|
||||
| Field | Meaning | Default / limits |
|
||||
| ------------------- | ------------------------------------------------------------------- | --------------------------------------- |
|
||||
| `enabled` | Master switch | `false` |
|
||||
| `defaultMode` | `parallel` or `collaborative` (see below) | `parallel` |
|
||||
| `providerOverrides` | Per-provider participation (`providerId`, optional `modelId`, `enabled`) | empty = every active provider, max 200 |
|
||||
| `systemPrompt` | Override for the built-in Chaos system prompt | optional, max 10 000 chars |
|
||||
| `timeoutMs` | Max time per model call | `120000` (5 000–600 000) |
|
||||
| `maxTokens` | `max_tokens` per model call | `4096` (256–128 000) |
|
||||
|
||||
4. Run a **test from the page itself** — the results panel shows each provider's answer,
|
||||
status and duration.
|
||||
|
||||
## Execution modes
|
||||
|
||||
- **`parallel`** — every model gets the same task simultaneously; you receive all answers
|
||||
independently.
|
||||
- **`collaborative`** — models run **in a chain**: each one sees the previous model's output and
|
||||
is asked to refine, extend, critique or offer an alternative. The response's `summary` field
|
||||
concatenates the successful outputs in chain order (parallel runs have no `summary`).
|
||||
|
||||
## API
|
||||
|
||||
### `POST /api/chaos/run` — dashboard session
|
||||
|
||||
Cookie-authenticated (the management session — see
|
||||
[MANAGEMENT-AUTH.md](MANAGEMENT-AUTH.md)); used by the dashboard page.
|
||||
|
||||
```jsonc
|
||||
// body
|
||||
{
|
||||
"task": "Compare approaches to X", // required
|
||||
"providers": ["glm", "kimi"], // optional filter
|
||||
"mode": "parallel", // optional — overrides defaultMode
|
||||
"systemPrompt": "…", // optional override
|
||||
"maxTokens": 4096 // optional override
|
||||
}
|
||||
```
|
||||
|
||||
### `POST /api/skills/collect/chaos` — API key
|
||||
|
||||
Bearer-token variant for external callers. The key must carry the **Chaos Mode permission**
|
||||
(`chaosModeEnabled`), which is **off by default** — enable it per key in
|
||||
**Dashboard → API Manager → edit key → permissions → Chaos Mode**. Same body as above.
|
||||
|
||||
```bash
|
||||
curl -X POST http://localhost:20128/api/skills/collect/chaos \
|
||||
-H "Authorization: Bearer $OMNIROUTE_API_KEY" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"task":"Compare approaches to X","mode":"parallel"}'
|
||||
```
|
||||
|
||||
Both endpoints return the same shape:
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"task": "…",
|
||||
"mode": "parallel",
|
||||
"startedAt": "2026-09-01T00:00:00.000Z",
|
||||
"totalProviders": 3,
|
||||
"totalResults": 3,
|
||||
"models": [
|
||||
{ "providerId": "glm", "providerName": "GLM", "modelId": "glm-4.7",
|
||||
"status": "success", "content": "…", "durationMs": 3210 }
|
||||
],
|
||||
"summary": "…" // collaborative mode only
|
||||
}
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
- **`400 Chaos Mode is not enabled`** — step 2 above: the global switch is off.
|
||||
- **API key gets rejected on `/api/skills/collect/chaos`** — the key lacks the per-key
|
||||
`chaosModeEnabled` permission (off by default; this is a setting, not an error).
|
||||
- **A provider you expected is missing from the results** — check `providerOverrides` on the
|
||||
Chaos Mode page (a disabled override excludes it) and whether the provider connection is
|
||||
active.
|
||||
@@ -8,6 +8,7 @@
|
||||
"DOCKER_GUIDE",
|
||||
"ELECTRON_GUIDE",
|
||||
"FEATURES",
|
||||
"CHAOS-MODE",
|
||||
"FREE_PROVIDER_RANKINGS",
|
||||
"COST_TRACKING",
|
||||
"I18N",
|
||||
|
||||
@@ -289,6 +289,26 @@ OmniRoute's combo engine supports **19 routing strategies** (declared in `src/sh
|
||||
|
||||
⭐ = New in v3.8.0 · 🧬 = New in v3.8.36
|
||||
|
||||
### `weighted` semantics
|
||||
|
||||
`weighted` is a **proportional random draw per request**
|
||||
(`open-sse/services/combo/targetSorters.ts` → `selectWeightedTarget`), not an equalizer:
|
||||
|
||||
- Each request draws **one** step with probability `weight / totalWeight`; the remaining steps
|
||||
are ordered by descending weight as the fallback chain for that request.
|
||||
- A step whose weight is `0` (or missing) is **never drawn** while any other step has a
|
||||
weight > 0 — it can only serve as a fallback after the drawn step fails. Only when **all**
|
||||
weights are 0 does selection become uniform.
|
||||
- Steps whose targets are all unavailable — provider circuit breaker `OPEN`, connection
|
||||
cooldown, model lockout — are removed from the draw before it happens
|
||||
(`open-sse/services/combo/targetResolution.ts`), so a single healthy step can temporarily
|
||||
win every request.
|
||||
- `stickyWeightedLimit` (combo config, default `1` = off) pins the drawn step for that many
|
||||
consecutive successes before re-drawing.
|
||||
|
||||
For strict rotation use `round-robin`; equal weights on `weighted` give statistical — not
|
||||
strict — balance.
|
||||
|
||||
## Fusion Strategy
|
||||
|
||||
`fusion` is the one strategy that does **not** pick a single target. It fans the prompt
|
||||
|
||||
Reference in New Issue
Block a user