mirror of
https://github.com/diegosouzapw/OmniRoute.git
synced 2026-08-08 00:02:20 +03:00
docs: feature-documentation catch-up (v3.8.20 → v3.8.30) (#4391)
One-time reconciliation of the docs with every user-facing feature shipped since
v3.8.20 (we had never done a dedicated pass, so debt had accumulated):
- README: new '✨ What's New' section (curated v3.8.20→v3.8.30 highlights).
- New guides: CLI-INTEGRATIONS (all setup-*/launch commands), MITM-TPROXY-DECRYPT
(transparent-decrypt epic), CONTEXT_EDITING (delegated Anthropic clear_tool_uses).
- Refreshed: AUTO-COMBO (auto/<category>:<tier> + Arena-ELO), API_REFERENCE
(x-omniroute-no-memory), MEMORY (int8 quantization + off-by-default), RESILIENCE
(model-lockout success-decay), RTK, AGENTBRIDGE, TRAFFIC_INSPECTOR, GUARDRAILS,
CLOUD_AGENT, ENVIRONMENT, SETUP_GUIDE, CLI-TOOLS, MCP-SERVER.
- Regenerated PROVIDER_REFERENCE (231 providers); synced the count in README/CLAUDE/AGENTS.
- Allowlisted external-tool env vars (OPENAI_API_BASE, PROMPTFOO_PROVIDER_KEY) and the
STREAM_RECOVERY config-object name in the docs-accuracy gates.
All claims source-verified; check:docs-all (sync/counts/env/links/fabricated) passes.
Going forward this runs every release via generate-release step 6b.
This commit is contained in:
Diego Rodrigues de Sa e Souza
committed by
GitHub
GitHub
parent
7ef11eb94e
commit
a56c76a330
@@ -3,12 +3,12 @@
|
||||
## Project
|
||||
|
||||
Unified AI proxy/router — route any LLM through one endpoint. Multi-provider support
|
||||
with **227 provider entries** (OpenAI, Anthropic, Gemini, DeepSeek, Groq, xAI, Mistral, Fireworks,
|
||||
with **231 provider entries** (OpenAI, Anthropic, Gemini, DeepSeek, Groq, xAI, Mistral, Fireworks,
|
||||
Cohere, NVIDIA, Cerebras, Pollinations, Puter, Cloudflare AI, HuggingFace, DeepInfra,
|
||||
SambaNova, Meta Llama API, Moonshot AI, AI21 Labs, Databricks, Snowflake, and many more)
|
||||
with **MCP Server** (87 tools), **A2A v0.3 Protocol**, and **Electron desktop app**.
|
||||
|
||||
> **Live counts (v3.8.24)**: providers 227 · MCP tools 87 · MCP scopes 30 · A2A skills 6 ·
|
||||
> **Live counts (v3.8.31)**: providers 231 · MCP tools 87 · MCP scopes 30 · A2A skills 6 ·
|
||||
> open-sse services 115 · routing strategies 15 · auto-combo scoring factors 9 ·
|
||||
> DB modules 83 · DB migrations 97 · base tables 17 · search providers 11 ·
|
||||
> i18n locales 42. **Refresh with `npm run check:docs-all`.**
|
||||
|
||||
@@ -26,6 +26,10 @@ _In development — bullets added per PR; finalized at release._
|
||||
- **fix(db): cascade-delete orphaned model aliases when a provider is removed** — deleting a custom provider removed its connections and node but left behind the imported model-alias rows (stored as `key=<alias>`, `value="<providerId>/<model>"`). Those stale aliases then blocked re-importing the same provider — the import dedup treated them as "already exists", so no new models appeared. A new `deleteModelAliasesForProvider(providerId)` DB helper drops every alias whose stored value begins with `<providerId>/` (leaving other providers and user-defined settings aliases untouched), and the provider-node DELETE handler now calls it after removing the connections and node, so a fresh import is unblocked. (thanks @nguyenvanhuy0612)
|
||||
- **fix(api): persist `max_input_tokens` / `max_output_tokens` when adding a custom model** — `POST /api/provider-models` silently dropped the per-model token limits set in the "add custom model" form: the handler destructured the rest of the body but never read `max_input_tokens` / `max_output_tokens`, and `addCustomModel()` had no parameter for them, so the values were thrown away on write. The DB layer (`inputTokenLimit` / `outputTokenLimit`) and the `/v1/models` catalog already round-trip these fields — only the write path was missing. The validation schema now accepts the two optional limits, the handler forwards them, and `addCustomModel()` persists them so a custom model's context/output window survives into the catalog. (thanks @codename-zen)
|
||||
|
||||
### 📝 Maintenance
|
||||
|
||||
- **docs: one-time feature-documentation catch-up (v3.8.20 → v3.8.30)** — reconciled the docs with every user-facing feature shipped since v3.8.20: a new README **✨ What's New** section; new guides for [CLI integrations](docs/guides/CLI-INTEGRATIONS.md), [MITM TPROXY transparent decrypt](docs/security/MITM-TPROXY-DECRYPT.md) and [delegated Anthropic Context Editing](docs/compression/CONTEXT_EDITING.md); refreshed AUTO-COMBO (`auto/<category>:<tier>` + Arena-ELO), API_REFERENCE (`x-omniroute-no-memory`), MEMORY (int8 quantization, off-by-default), RESILIENCE (model-lockout success-decay), RTK, AGENTBRIDGE, TRAFFIC_INSPECTOR, GUARDRAILS, CLOUD_AGENT, ENVIRONMENT; regenerated PROVIDER_REFERENCE (231 providers) and synced the provider count in README/CLAUDE/AGENTS. Going forward this runs every release (generate-release step 6b). ([#4391](https://github.com/diegosouzapw/OmniRoute/pull/4391))
|
||||
|
||||
---
|
||||
|
||||
## [3.8.30] — 2026-06-20
|
||||
|
||||
@@ -35,7 +35,7 @@ For full test matrix, see `CONTRIBUTING.md` → "Running Tests". For deep archit
|
||||
|
||||
## Project at a Glance
|
||||
|
||||
**OmniRoute** — unified AI proxy/router. One endpoint, 227 LLM providers, auto-fallback.
|
||||
**OmniRoute** — unified AI proxy/router. One endpoint, 231 LLM providers, auto-fallback.
|
||||
|
||||
| Layer | Location | Purpose |
|
||||
| ------------- | ----------------------- | -------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
|
||||
40
README.md
40
README.md
@@ -6,7 +6,7 @@
|
||||
|
||||
# 🚀 OmniRoute — The Free AI Gateway
|
||||
|
||||
### Never stop coding. Connect every AI tool to **227 providers** — **50+ free** — through one endpoint.
|
||||
### Never stop coding. Connect every AI tool to **231 providers** — **50+ free** — through one endpoint.
|
||||
|
||||
**Plug Claude Code, Codex, Cursor, Cline, Copilot & Antigravity into FREE Claude / GPT / Gemini. Auto-fallback.**
|
||||
<br/>
|
||||
@@ -19,8 +19,8 @@
|
||||
|
||||
<br/>
|
||||
|
||||
[](#-227-ai-providers--50-free)
|
||||
[](#-227-ai-providers--50-free)
|
||||
[](#-231-ai-providers--50-free)
|
||||
[](#-231-ai-providers--50-free)
|
||||
[](docs/reference/FREE_TIERS.md)
|
||||
[](#%EF%B8%8F-save-1595-tokens--automatically)
|
||||
[](#-combos--the-flagship)
|
||||
@@ -59,7 +59,7 @@
|
||||
|
||||
<br/>
|
||||
|
||||
[**🚀 Quick Start**](#-quick-start) • [**🎯 Combos**](#-combos--the-flagship) • [**🌐 Providers**](#-227-ai-providers--50-free) • [**🔌 CLI & MCP**](#-full-cli--a2a--mcp) • [**🗜️ Compression**](#%EF%B8%8F-save-1595-tokens--automatically) • [**🌍 Website**](https://omniroute.online)
|
||||
[**🚀 Quick Start**](#-quick-start) • [**🎯 Combos**](#-combos--the-flagship) • [**🌐 Providers**](#-231-ai-providers--50-free) • [**🔌 CLI & MCP**](#-full-cli--a2a--mcp) • [**🗜️ Compression**](#%EF%B8%8F-save-1595-tokens--automatically) • [**🌍 Website**](https://omniroute.online)
|
||||
|
||||
[💥 The Promise](#-the-promise) • [🤔 Why](#-why-omniroute) • [🏆 What Sets Apart](#-what-sets-omniroute-apart) • [🤖 Compatible CLIs](#-compatible-clis--coding-agents) • [🖥️ Where It Runs](#%EF%B8%8F-where-omniroute-runs--anywhere) • [🔒 Private](#-private--local-first) • [🎬 In Action](#-omniroute-in-action) • [📚 Explore More](#-explore-more) • [📧 Support](#-support--community)
|
||||
|
||||
@@ -137,11 +137,11 @@
|
||||
|
||||
</div>
|
||||
|
||||
> One endpoint. **227 providers.** Never stop building — and let OmniRoute pick the cheapest one that works.
|
||||
> One endpoint. **231 providers.** Never stop building — and let OmniRoute pick the cheapest one that works.
|
||||
|
||||
<table>
|
||||
<tr>
|
||||
<td width="33%" valign="top"><b>🚫 Never hit limits</b><br/><sub>Auto-fallback across 227 providers in milliseconds. Quota out? Next provider takes over — zero downtime.</sub></td>
|
||||
<td width="33%" valign="top"><b>🚫 Never hit limits</b><br/><sub>Auto-fallback across 231 providers in milliseconds. Quota out? Next provider takes over — zero downtime.</sub></td>
|
||||
<td width="33%" valign="top"><b>💸 Save up to 95% tokens</b><br/><sub>RTK + Caveman stacked compression cuts 15–95% of eligible tokens (~89% avg on tool-heavy sessions).</sub></td>
|
||||
<td width="33%" valign="top"><b>🆓 $0 to start</b><br/><sub>50+ providers with a free tier, 11 free <i>forever</i> (Kiro, Qoder, Pollinations, LongCat…). No card needed.</sub></td>
|
||||
</tr>
|
||||
@@ -264,7 +264,7 @@ Result: 4 layers of fallback = zero downtime
|
||||
|
||||
| Feature | OmniRoute | Other routers |
|
||||
| -------------------------------------- | ----------------------------------------------------------- | ------------- |
|
||||
| 🌐 Providers | **227** | 20–100 |
|
||||
| 🌐 Providers | **231** | 20–100 |
|
||||
| 🆓 Free providers | **50+ (11 free forever)** | 1–5 |
|
||||
| 🔀 Routing strategies | **15** (priority, weighted, cost-optimized, context-relay…) | 1–3 |
|
||||
| 🗜️ Token compression | **RTK + Caveman stacked (15–95%)** | None / 20–40% |
|
||||
@@ -283,6 +283,26 @@ Result: 4 layers of fallback = zero downtime
|
||||
|
||||
<div align="center">
|
||||
|
||||
# ✨ What's New
|
||||
|
||||
</div>
|
||||
|
||||
> Recent highlights from **v3.8.20 → v3.8.30**. Full history in [`CHANGELOG.md`](CHANGELOG.md).
|
||||
|
||||
- **🤖 One-command CLI/agent setup** — a dedicated `setup-*` command configures each coding tool to route through OmniRoute (Claude Code, Codex, Cline, Continue, Cursor, Roo Code, Kilo Code, Crush, Goose, Qwen Code, Aider, OpenCode, Gemini CLI); `omniroute launch` / `omniroute launch-codex` are zero-config launchers. → [CLI Integrations](docs/guides/CLI-INTEGRATIONS.md)
|
||||
- **🛰️ Remote mode** — drive a remote OmniRoute from any machine with scoped access tokens (`omniroute connect` / `omniroute contexts` / `omniroute tokens`). → [Remote Mode](docs/guides/REMOTE-MODE.md)
|
||||
- **🧭 Smarter auto-routing** — OpenRouter-style `auto/<category>:<tier>` combos (e.g. `auto/coding:fast`, `auto/reasoning:pro`), live Arena-ELO + models.dev model intelligence, and per-step account allowlists. → [Auto-Combo](docs/routing/AUTO-COMBO.md)
|
||||
- **🗜️ Pluggable compression** — an async compression pipeline with Compression Studios, a stable LLMLingua-2 ONNX engine, RTK, and delegated Anthropic Context Editing. → [Compression](docs/compression/COMPRESSION_ENGINES.md)
|
||||
- **🕵️ Transparent MITM decrypt (TPROXY)** — capture & translate traffic from CLIs that ignore proxy env vars, with a per-SNI certificate authority and a trust-store installer. → [MITM/TPROXY](docs/security/MITM-TPROXY-DECRYPT.md)
|
||||
- **💸 Cost telemetry everywhere** — `X-OmniRoute-*` cost/usage headers on every endpoint (including media), a non-token cost engine, a cache-HIT `X-OmniRoute-Cost-Saved` header, and per-key USD spend quotas. → [API Reference](docs/reference/API_REFERENCE.md)
|
||||
- **🧠 Memory you control** — opt-in int8 vector quantization (Qdrant + sqlite-vec), memory off by default, and a per-request `x-omniroute-no-memory` header. → [Memory](docs/frameworks/MEMORY.md)
|
||||
- **🛡️ Security** — a prompt-injection guard across every LLM route (backed by a red-team suite), plus a free DuckDuckGo last-resort web search. → [Guardrails](docs/security/GUARDRAILS.md)
|
||||
- **🤝 More providers & agents** — Cursor Cloud Agent (a 4th cloud agent), a refreshed 231-provider catalog (OrcaRouter, Wafer AI, OpenAdapter, dit.ai, TokenRouter, …), and Vertex AI media generation (speech / transcription / music / video). → [Providers](docs/reference/PROVIDER_REFERENCE.md)
|
||||
|
||||
<br/>
|
||||
|
||||
<div align="center">
|
||||
|
||||
# 🤖 Compatible CLIs & Coding Agents
|
||||
|
||||
> One config — `http://localhost:20128/v1` — and **every** AI IDE or CLI runs on free & low-cost models.
|
||||
@@ -320,11 +340,11 @@ Result: 4 layers of fallback = zero downtime
|
||||
|
||||
<div align="center">
|
||||
|
||||
# 🌐 227 AI Providers — 50+ Free
|
||||
# 🌐 231 AI Providers — 50+ Free
|
||||
|
||||
</div>
|
||||
|
||||
> The most complete catalog of any open-source router: **227 providers**, **50+ with a free tier**, **11 free forever**.
|
||||
> The most complete catalog of any open-source router: **231 providers**, **50+ with a free tier**, **11 free forever**.
|
||||
|
||||
<div align="center">
|
||||
|
||||
@@ -767,7 +787,7 @@ Compression: aggressive (~50%) → double your free quota · Cost: $0/mo
|
||||
**Will I be charged by OmniRoute?** No — it's free, open-source software on your machine. You only pay paid providers directly. OmniRoute has no billing system.
|
||||
**Are FREE providers really unlimited?** Yes — Kiro, Qoder, Pollinations, LongCat, Cloudflare. No catch.
|
||||
**Will compression hurt quality?** No — it only compresses the **input**; code, URLs, JSON are always protected.
|
||||
**Does it work where AI is blocked?** Yes — 3-level proxy + 1proxy marketplace reach all 227 providers.
|
||||
**Does it work where AI is blocked?** Yes — 3-level proxy + 1proxy marketplace reach all 231 providers.
|
||||
|
||||
📖 [User Guide](docs/guides/USER_GUIDE.md) · [API Reference](docs/reference/API_REFERENCE.md) · [Environment Config](docs/reference/ENVIRONMENT.md)
|
||||
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
title: "Resilience Guide"
|
||||
version: 3.8.18
|
||||
lastUpdated: 2026-06-09
|
||||
version: 3.8.31
|
||||
lastUpdated: 2026-06-20
|
||||
---
|
||||
|
||||
# Resilience Guide
|
||||
@@ -116,6 +116,47 @@ Lists active lockouts with: provider, connection, model, reason, expiresAt. Oper
|
||||
- `GET /api/resilience/model-cooldowns` — list active lockouts
|
||||
- `DELETE /api/resilience/model-cooldowns` — manual re-enable. Body: `{provider, connection, model}`. Auth: management.
|
||||
|
||||
### Lockout settings UI + success-decay recovery (v3.8.23)
|
||||
|
||||
Model lockout went from always-on hardcoded behavior to a fully configurable,
|
||||
opt-in feature with its own settings card and a self-healing recovery path.
|
||||
|
||||
**Settings card:** Settings → Model Lockout
|
||||
(`src/app/(dashboard)/dashboard/settings/components/ModelLockoutCard.tsx`).
|
||||
This is **distinct** from the read-only `ModelCooldownsCard` above (which only
|
||||
*lists* active lockouts) — the new card *configures the parameters*. Defaults
|
||||
live in `DEFAULT_MODEL_LOCKOUT_SETTINGS`
|
||||
(`src/lib/resilience/modelLockoutSettings.ts`):
|
||||
|
||||
| Setting | Default | Meaning |
|
||||
| ----------------------- | -------------------------------- | --------------------------------------------------------------- |
|
||||
| `enabled` | `false` | Master toggle — model lockout is **off by default**. |
|
||||
| `errorCodes` | `[403, 404, 429, 502, 503, 504]` | Upstream statuses that count as a model-scoped failure. |
|
||||
| `baseCooldownMs` | `120_000` (120 s) | Initial lockout duration for the first failure. |
|
||||
| `maxCooldownMs` | `1_800_000` (30 min) | Cap on the escalated cooldown. |
|
||||
| `maxBackoffSteps` | `10` | Max exponential-backoff escalation steps. |
|
||||
| `useExponentialBackoff` | `true` | Whether repeated failures escalate the cooldown exponentially. |
|
||||
|
||||
Settings persist through the normal settings store and validate via the
|
||||
resilience settings schema; the card clamps `baseCooldownMs`/`maxCooldownMs`
|
||||
(with `maxCooldownMs ≥ baseCooldownMs`) and `maxBackoffSteps`.
|
||||
|
||||
**Success-decay recovery:** recovery is **not** purely timer expiry. A healthy
|
||||
response walks the model's failure count back down so a model that recovered
|
||||
mid-window stops escalating (and clears) before its timer would. On a successful
|
||||
combo target, `open-sse/services/combo.ts` calls `decayModelFailureCount()`
|
||||
(`open-sse/services/accountFallback.ts`), which **halves** the stored
|
||||
`failureCount` (`Math.floor(failureCount / 2)`); when it reaches `0` the lockout
|
||||
entry is deleted entirely. The counterpart `recordModelLockoutFailure()`
|
||||
increments the count (and escalates the cooldown) on failures within the
|
||||
escalation window. This success-decay is in addition to plain timer expiry —
|
||||
either path can re-enable a model.
|
||||
|
||||
**State:** lockouts are held **in-memory** (per-process `Map`s of
|
||||
`ModelLockoutEntry` keyed by `provider:connectionId:model`), not persisted to
|
||||
the DB — they are lost on restart. The *settings* are persisted; the active
|
||||
lockout *state* is ephemeral.
|
||||
|
||||
---
|
||||
|
||||
## Other Resilience Features
|
||||
|
||||
195
docs/compression/CONTEXT_EDITING.md
Normal file
195
docs/compression/CONTEXT_EDITING.md
Normal file
@@ -0,0 +1,195 @@
|
||||
---
|
||||
title: "Delegated Context Editing (Anthropic)"
|
||||
version: 3.8.31
|
||||
lastUpdated: 2026-06-20
|
||||
---
|
||||
|
||||
# Delegated Context Editing (Anthropic)
|
||||
|
||||
Delegated **Context Editing** is a Claude-only context-management feature. Unlike OmniRoute's local
|
||||
compression engines (Caveman, RTK, LLMLingua, stacked pipelines) — which rewrite the request body
|
||||
*before* it leaves the proxy — Context Editing asks the **provider** to clear stale
|
||||
tool-use / tool-result blocks from its own running context window. OmniRoute only attaches a body
|
||||
parameter (`context_management.edits[]`); Claude does the actual clearing against its own tokenizer.
|
||||
|
||||
This is a delegated capability by nature: other providers reject the parameter, so OmniRoute scopes
|
||||
it strictly to Claude and Claude-Code-compatible relays.
|
||||
|
||||
Source of truth: `open-sse/config/contextEditing.ts` (strategy ids, body injection, telemetry
|
||||
extraction), `open-sse/executors/base.ts` (injection gate + 400-fallback), and
|
||||
`open-sse/services/compression/types.ts` (config shape + default).
|
||||
|
||||
## What `clear_tool_uses` does
|
||||
|
||||
OmniRoute injects a single edit into the outbound Anthropic Messages body:
|
||||
|
||||
```json
|
||||
{
|
||||
"context_management": {
|
||||
"edits": [
|
||||
{
|
||||
"type": "clear_tool_uses_20250919",
|
||||
"trigger": { "type": "input_tokens", "value": 100000 },
|
||||
"keep": { "type": "tool_uses", "value": 3 }
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
- `type: "clear_tool_uses_20250919"` — the dated Anthropic strategy id (`CLEAR_TOOL_USES_STRATEGY`).
|
||||
- `trigger.value: 100000` — once the request's input tokens exceed this threshold, Claude begins
|
||||
clearing old tool-use/result pairs (`CONTEXT_EDITING_DEFAULT_TRIGGER_TOKENS`, Anthropic's default).
|
||||
- `keep.value: 3` — the N most recent tool-use/result pairs are kept untouched
|
||||
(`CONTEXT_EDITING_DEFAULT_KEEP_TOOL_USES`).
|
||||
|
||||
The beta is advertised via the `anthropic-beta: context-management-2025-06-27` header, which
|
||||
OmniRoute already emits on Claude requests.
|
||||
|
||||
Injection is performed by `applyContextEditingToBody()` and is **idempotent**: if a `clear_tool_uses`
|
||||
edit already exists on the body (added by a previous call or supplied by the client), the body is
|
||||
left as-is. If a `clear_thinking_20251015` edit is also present, OmniRoute stable-sorts the
|
||||
`clear_thinking` edit to the front, because Anthropic requires `clear_thinking` to precede
|
||||
`clear_tool_uses` in the `edits[]` array.
|
||||
|
||||
## The per-combo enable toggle
|
||||
|
||||
Context Editing is **off by default** and opt-in. The toggle is a single boolean carried in the
|
||||
compression config:
|
||||
|
||||
- Setting key: `contextEditing.enabled` (camelCase — **not** `context_editing` / `context-editing`).
|
||||
- Type: `ContextEditingConfig { enabled: boolean }` in
|
||||
`open-sse/services/compression/types.ts`.
|
||||
- Default: `DEFAULT_CONTEXT_EDITING_CONFIG = { enabled: false }`.
|
||||
- Zod schema: `contextEditingConfigSchema` in `src/shared/validation/compressionConfigSchemas.ts`.
|
||||
- Storage: persisted with the rest of the compression settings (normalized in
|
||||
`src/lib/db/compression.ts`).
|
||||
|
||||
In the dashboard the toggle lives in the compression hub
|
||||
(`src/app/(dashboard)/dashboard/context/combos/CompressionHub.tsx`) and writes
|
||||
`{ contextEditing: { enabled: … } }` back through `saveSettings()`. Because it rides on the
|
||||
compression-settings object, it composes with the per-combo compression profile rather than being a
|
||||
fully independent surface — the config carries only the on/off flag; all thresholds (`trigger`,
|
||||
`keep`) are the constants documented above.
|
||||
|
||||
## Claude-only gating
|
||||
|
||||
Injection only happens for genuine Claude or Claude-Code-compatible relays. The gate in
|
||||
`open-sse/executors/base.ts` is:
|
||||
|
||||
```ts
|
||||
if (
|
||||
(this.provider === "claude" || isClaudeCodeCompatible(this.provider)) &&
|
||||
contextEditing?.enabled &&
|
||||
!contextEditingDisabled
|
||||
) {
|
||||
applyContextEditingToBody(transformedBody, { enabled: true });
|
||||
}
|
||||
```
|
||||
|
||||
- `this.provider === "claude"` — real Anthropic key/OAuth.
|
||||
- `isClaudeCodeCompatible(this.provider)` — relays whose provider id starts with the
|
||||
`anthropic-compatible-cc-` prefix (they advertise Claude Code compatibility, so they are the relays
|
||||
most likely to accept the beta). See `open-sse/services/provider.ts`.
|
||||
|
||||
Deliberately **excluded**:
|
||||
|
||||
- `claude-web` — a browser relay with a `create_conversation_params` request shape that never sees
|
||||
`context_management`.
|
||||
- Generic `anthropic-compatible-*` relays (without the `-cc-` prefix) — third-party endpoints with
|
||||
uncertain beta support.
|
||||
|
||||
Non-Claude providers never receive the `context_management` parameter even when the toggle is on.
|
||||
|
||||
## The 400-fallback / relay coverage
|
||||
|
||||
A Claude-compatible relay may advertise the beta but still reject the `context_management` parameter
|
||||
with an HTTP 400. To degrade gracefully instead of failing the request, the executor strips the
|
||||
parameter and retries the same URL **once**:
|
||||
|
||||
```ts
|
||||
if (
|
||||
response.status === HTTP_STATUS.BAD_REQUEST &&
|
||||
contextEditing?.enabled &&
|
||||
!contextEditingDisabled &&
|
||||
transformedBody?.context_management !== undefined
|
||||
) {
|
||||
const errText = await response.clone().text().catch(() => "");
|
||||
if (/context[_-]management|context editing/i.test(errText)) {
|
||||
contextEditingDisabled = true;
|
||||
delete transformedBody.context_management;
|
||||
let retryBody = JSON.stringify(transformedBody);
|
||||
if (isClaudeCodeCompatible(this.provider) || this.provider === "claude") {
|
||||
retryBody = await signRequestBody(retryBody);
|
||||
}
|
||||
response = await fetch(url, { ...fetchOptions, body: retryBody });
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Behavior:
|
||||
|
||||
1. Fires only on a `400` while context editing is enabled and the body actually carries
|
||||
`context_management`.
|
||||
2. The 400 body is read via a `clone()` so the original response stays intact for the non-matching
|
||||
path.
|
||||
3. The error text must match `/context[_-]management|context editing/i` — an unrelated 400 (e.g.
|
||||
`max_tokens must be >= 1`) does **not** trigger the fallback; the original error propagates.
|
||||
4. On a match it sets `contextEditingDisabled = true` (which suppresses re-injection if a fresh
|
||||
`transformedBody` is later built for a retry/fallback URL), deletes `context_management`,
|
||||
re-signs the body for Claude / Claude-Code-compatible relays (`signRequestBody`), and retries the
|
||||
same URL once.
|
||||
|
||||
Genuine Claude carries the beta in `ANTHROPIC_BETA_BASE` and does not hit this fallback path.
|
||||
|
||||
## `applied_edits` telemetry
|
||||
|
||||
After a Claude response, OmniRoute records how much context the provider actually cleared. This is
|
||||
**not** streamed — it is extracted from the non-streaming response body, best-effort, and never
|
||||
affects the response (telemetry failures are swallowed).
|
||||
|
||||
- Extraction: `extractContextEditingTelemetry(responseBody)` in `open-sse/config/contextEditing.ts`.
|
||||
It probes `applied_edits` in three locations (defensive over the response shape):
|
||||
- `context_management.applied_edits`
|
||||
- `usage.context_management.applied_edits`
|
||||
- `usage.applied_edits`
|
||||
- Per-edit fields read from each entry: `cleared_input_tokens` and `cleared_tool_uses`
|
||||
(snake_case, Anthropic-native), with `clearedInputTokens` / `clearedToolUses` camelCase fallbacks.
|
||||
- Returns `null` when no `applied_edits` array is found or nothing was actually cleared.
|
||||
|
||||
The receipt shape is `ContextEditingTelemetry { editCount, clearedInputTokens, clearedToolUses }`.
|
||||
Recording happens in `open-sse/handlers/chatCore.ts` (gated to `provider === "claude"`) via
|
||||
`recordContextEditingTelemetry()` (`src/lib/db/compressionAnalytics.ts`), which writes a compression
|
||||
analytics row tagged:
|
||||
|
||||
- `mode: "context-editing"`
|
||||
- `engine: "context-editing"`
|
||||
- `tokens_saved` / `original_tokens` = the cleared input-token count
|
||||
- `request_id` suffixed with `::context-editing`
|
||||
|
||||
So delegated clearing shows up in compression analytics alongside the local engines, under the
|
||||
`context-editing` engine label, and is distinguishable from RTK/Caveman/LLMLingua savings.
|
||||
|
||||
## Relationship to the local compression engines
|
||||
|
||||
| Aspect | Local engines (Caveman / RTK / LLMLingua / stacked) | Delegated Context Editing |
|
||||
| ----------------- | --------------------------------------------------- | ------------------------------------------ |
|
||||
| Where it runs | In OmniRoute, before the request leaves the proxy | In the provider (Claude), server-side |
|
||||
| What it edits | Prompt / context / tool-result text | Old tool-use / tool-result blocks |
|
||||
| Provider scope | All providers | `claude` + `anthropic-compatible-cc-*` only |
|
||||
| Toggle | Compression mode settings | `contextEditing.enabled` |
|
||||
| Failure mode | Fail-open (original text) | 400-fallback: strip param, retry once |
|
||||
| Savings telemetry | `engine: <engine id>` | `engine: "context-editing"` |
|
||||
|
||||
The two are complementary: local engines compress the bytes OmniRoute sends; Context Editing lets
|
||||
Claude prune the running context across turns. They can be enabled together.
|
||||
|
||||
## See Also
|
||||
|
||||
- [COMPRESSION_ENGINES.md](./COMPRESSION_ENGINES.md) — engine registry and the local compression
|
||||
engines
|
||||
- [RTK_COMPRESSION.md](./RTK_COMPRESSION.md) — command/tool-output compression
|
||||
- [../frameworks/MCP-SERVER.md](../frameworks/MCP-SERVER.md) — MCP description compression and
|
||||
tool-cardinality reduction
|
||||
- Source: `open-sse/config/contextEditing.ts`, `open-sse/executors/base.ts`,
|
||||
`open-sse/services/compression/types.ts`, `src/lib/db/compressionAnalytics.ts`
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
title: "RTK Compression"
|
||||
version: 3.8.2
|
||||
lastUpdated: 2026-05-13
|
||||
version: 3.8.31
|
||||
lastUpdated: 2026-06-20
|
||||
---
|
||||
|
||||
# RTK Compression
|
||||
@@ -96,6 +96,7 @@ Important fields:
|
||||
| `rules.dropPatterns` | Remove noisy lines |
|
||||
| `rules.includePatterns` | Prefer actionable lines |
|
||||
| `rules.collapsePatterns` | Collapse repeated matching lines |
|
||||
| `rules.deduplicate` | Per-filter opt-in: collapse consecutive duplicate lines |
|
||||
| `rules.truncateLineAt` | Unicode-safe per-line truncation |
|
||||
| `rules.onEmpty` | Fallback message if all lines are filtered out |
|
||||
| `tests[]` | Inline samples used by the verify gate |
|
||||
@@ -103,6 +104,57 @@ Important fields:
|
||||
Built-in filters are expected to include inline `tests[]` samples. Custom filters should include
|
||||
them too, especially when they are shared across projects.
|
||||
|
||||
## Line Deduplication (two layers)
|
||||
|
||||
RTK collapses duplicate lines at two independent layers:
|
||||
|
||||
1. **Per-filter `deduplicate` (opt-in, default `false`).** A filter can set `rules.deduplicate: true`
|
||||
to collapse consecutive duplicate lines *within that filter's matched output*, before truncation.
|
||||
This runs inside `lineFilter.ts`. For legacy filters, it is auto-enabled when the filter defines
|
||||
`collapsePatterns`. Schema: `deduplicate: z.boolean().default(false)` in
|
||||
`open-sse/services/compression/engines/rtk/filterSchema.ts`.
|
||||
2. **Engine-wide `deduplicateThreshold` (default `3`).** After all filters run, the engine collapses
|
||||
any run of `>= deduplicateThreshold` identical consecutive lines across the whole result
|
||||
(`deduplicateRepeatedLines`, applied in `engines/rtk/index.ts`). The value is bounded to 2–100 on
|
||||
normalization.
|
||||
|
||||
The per-filter pass runs first (inside the filter), the engine-wide pass runs last (over the joined
|
||||
output), so the two compose without double-counting.
|
||||
|
||||
## Line Grouping (`enableGrouping`)
|
||||
|
||||
When `rtkConfig.enableGrouping` is `true` (default `false`), RTK runs an additional `groupSimilarLines`
|
||||
pass over the post-dedup result that collapses runs of *near-equivalent* (not byte-identical)
|
||||
consecutive lines. `rtkConfig.groupingThreshold` (default `3`) is the minimum run length that triggers
|
||||
grouping. This is the structural counterpart to `deduplicateThreshold`: dedup handles exact repeats,
|
||||
grouping handles "the same shape with small differences". Both flags are part of the `rtkConfig` JSON
|
||||
persisted in the `key_value` table (see Configuration above), so the setting survives restarts.
|
||||
|
||||
## Code Comment Stripping (`stripCodeComments` / `preserveDocstrings`)
|
||||
|
||||
When `rtkConfig.applyToCodeBlocks` is enabled, RTK can also strip comments from fenced code blocks:
|
||||
|
||||
- `stripCodeComments` (default `false`) — opt-in. When `true`, RTK removes comments from JavaScript
|
||||
and TypeScript fenced blocks. The flag was historically read but never applied, so the default stays
|
||||
at "preserve" to avoid a silent production change.
|
||||
- `preserveDocstrings` (default `true`) — when stripping comments, JSDoc/`/** … */` block comments are
|
||||
kept (they carry API documentation worth more than the bytes they cost). Set to `false` to strip
|
||||
those too.
|
||||
|
||||
Comment removal is implemented in `open-sse/services/compression/engines/rtk/codeStripper.ts`. It uses
|
||||
the **TypeScript parser** (not a regex) so that string, template, and regex literals are never mistaken
|
||||
for comments, and it bails out entirely when JSX is detected (so JSX expression-container comments are
|
||||
never corrupted). Comment stripping currently applies to **JavaScript and TypeScript only** — other
|
||||
languages in the stripper's `CodeLanguage` set (Python, Rust, Go, Ruby, Java) have empty-line and
|
||||
whitespace collapse but no comment removal. The stripped-block run is tagged `rtk:code-strip` in
|
||||
`rulesApplied`.
|
||||
|
||||
> **Note — GCF / tabular encoding is a separate engine.** RTK does **not** contain the "GCF"
|
||||
> (Graph Compact Format) tabular/columnar JSON encoder. That encoder — which replaced an older
|
||||
> `omni-tabular` encoder — lives in the **headroom** engine
|
||||
> (`open-sse/services/compression/engines/headroom/`, with the vendored codec under
|
||||
> `headroom/gcf/`). It is unrelated to the RTK filter pipeline documented here.
|
||||
|
||||
## Configuration
|
||||
|
||||
Global settings are available through `/api/settings/compression`. RTK-specific settings are also
|
||||
@@ -131,13 +183,32 @@ available through `/api/context/rtk/config`.
|
||||
"customFiltersEnabled": true,
|
||||
"trustProjectFilters": false,
|
||||
"rawOutputRetention": "never",
|
||||
"rawOutputMaxBytes": 1048576
|
||||
"rawOutputMaxBytes": 1048576,
|
||||
"enableGrouping": false,
|
||||
"groupingThreshold": 3,
|
||||
"stripCodeComments": false,
|
||||
"preserveDocstrings": true
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
`enabledFilters` and `disabledFilters` use filter ids, for example `test-vitest` or `git-diff`.
|
||||
|
||||
The full `rtkConfig` shape is defined by `RtkConfig` / `DEFAULT_RTK_CONFIG` in
|
||||
`open-sse/services/compression/types.ts`. The whole object is persisted as a single JSON value in
|
||||
the SQLite `key_value` table under `namespace = "compression"`, `key = "rtkConfig"`
|
||||
(`src/lib/db/compression.ts`), and normalized on read by `normalizeRtkConfig`. So every field below
|
||||
— including `enableGrouping`, `groupingThreshold`, `stripCodeComments`, and `preserveDocstrings` —
|
||||
round-trips through the same store and survives a restart.
|
||||
|
||||
| Key | Default | Purpose |
|
||||
| ---------------------- | ------- | ----------------------------------------------------------------------------- |
|
||||
| `deduplicateThreshold` | `3` | Engine-wide: min consecutive identical lines to collapse (bounded 2–100) |
|
||||
| `enableGrouping` | `false` | Opt-in: collapse runs of near-equivalent consecutive lines |
|
||||
| `groupingThreshold` | `3` | Min consecutive similar-line run that triggers grouping |
|
||||
| `stripCodeComments` | `false` | Opt-in: remove comments from fenced code blocks (needs `applyToCodeBlocks`) |
|
||||
| `preserveDocstrings` | `true` | When stripping comments, keep JSDoc/`/** … */` blocks |
|
||||
|
||||
## API
|
||||
|
||||
| Route | Method | Purpose |
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
title: "AgentBridge"
|
||||
version: 3.8.6
|
||||
lastUpdated: 2026-05-28
|
||||
version: 3.8.31
|
||||
lastUpdated: 2026-06-20
|
||||
---
|
||||
|
||||
# AgentBridge
|
||||
@@ -10,7 +10,7 @@ AgentBridge is OmniRoute's MITM (Man-in-the-Middle) proxy that intercepts HTTPS
|
||||
|
||||
**Dashboard location:** `/dashboard/tools/agent-bridge`
|
||||
**Sidebar group:** Tools (after Cloud Agents)
|
||||
**See also:** [`TRAFFIC_INSPECTOR.md`](./TRAFFIC_INSPECTOR.md) — monitor all intercepted traffic in real-time.
|
||||
**See also:** [`TRAFFIC_INSPECTOR.md`](./TRAFFIC_INSPECTOR.md) — monitor all intercepted traffic in real-time; [`docs/security/MITM-TPROXY-DECRYPT.md`](../security/MITM-TPROXY-DECRYPT.md) — the Linux TPROXY transparent-decrypt capture mode driven by the `/api/tools/agent-bridge/tproxy` route.
|
||||
|
||||
---
|
||||
|
||||
@@ -225,6 +225,64 @@ Wildcard `*` maps any unrecognized model to the specified target. Persisted in `
|
||||
|
||||
AgentBridge intercepts credentials (OAuth tokens, API keys) that the IDE uses to authenticate with upstream providers. These are **masked before logging** (see §2.7) but are visible to OmniRoute's MITM layer. First activation of each agent shows a dismissible risk notice modal.
|
||||
|
||||
### 3.6 Maintenance & Diagnostics
|
||||
|
||||
The dashboard exposes a **Maintenance & Diagnostics** card (`AgentBridgeMaintenanceCard`, in `src/app/(dashboard)/dashboard/tools/agent-bridge/components/`) that surfaces operational MITM routes which previously had no UI. Its subtitle: *"Self-test the capture pipeline, undo leftover system state, and move your setup between machines."* The card client helpers live in `src/lib/inspector/agentBridgeMaintenanceApi.ts`.
|
||||
|
||||
| Button | Route | What it does |
|
||||
|--------|-------|--------------|
|
||||
| **Diagnose** | `GET /api/tools/agent-bridge/diagnose` | Runs the capture-pipeline self-test and shows a per-check report (✓/✗ + remediation hint). |
|
||||
| **Repair** | `POST /api/tools/agent-bridge/repair` | Undoes orphaned MITM system state (DNS spoof entries, root CA, system proxy) left behind by a crash or SIGKILL. Idempotent — reports "Nothing to repair" when state is clean. |
|
||||
| **Remove CA** | `DELETE /api/tools/agent-bridge/cert` | Untrusts and removes the MITM root CA from the OS trust store (explicit, idempotent). Shown only when the CA is currently trusted; requires an inline "Remove CA?" confirmation. |
|
||||
| **Export config** | `GET /api/tools/agent-bridge/config` | Downloads the portable config JSON (see §3.7). |
|
||||
| **Import config** | `POST /api/tools/agent-bridge/config` | Uploads a previously-exported config JSON (see §3.7). |
|
||||
|
||||
**Diagnostics checks** (`summarizeDiagnostics()` in `src/mitm/inspector/diagnostics.ts`). The route runs the effectful probe for each and feeds the booleans into the pure summarizer; a single `healthy` verdict plus a per-failure hint is returned:
|
||||
|
||||
| Check name | What it verifies | Hint on failure |
|
||||
|------------|------------------|-----------------|
|
||||
| `server-running` | The MITM server process is active | "The MITM server is not running. Start it from the AgentBridge tab." |
|
||||
| `server-reachable` | The MITM server accepts connections on its port (TCP probe) | "The MITM server is not accepting connections on its port. Check that the port is free and that you have privileges to bind it." |
|
||||
| `cert-exists` | The MITM certificate has been generated on disk | "No MITM certificate has been generated yet. Generate one from the AgentBridge tab." |
|
||||
| `cert-trusted` | The MITM root CA is in the OS trust store | "The MITM root CA is not trusted by the OS store, so TLS interception will fail. Trust the certificate from the AgentBridge tab." |
|
||||
| `dns-configured` | Target hostnames are spoofed in `/etc/hosts` | "Target hostnames are not spoofed in /etc/hosts, so traffic never reaches the proxy. Enable DNS for the agent(s) you want to capture." |
|
||||
|
||||
**Orphaned-state banner:** when the page detects state left behind by a crash (DNS spoof / CA / system proxy), the card shows an amber banner — *"A previous session left system state behind (DNS spoof, CA, or system proxy). Run Repair to clean it up."* — and highlights the **Repair** button. `Repair` is the application-layer analogue of ProxyBridge's `--cleanup` flag (it delegates to `repairMitm()` in `src/mitm/manager.ts`).
|
||||
|
||||
> The MITM root CA is kept installed across stop/start to avoid repeated sudo
|
||||
> prompts (the same behavior as mitmproxy/Charles), so removing it is an explicit
|
||||
> **Remove CA** action rather than something that happens automatically on stop.
|
||||
|
||||
### 3.7 Portable config import/export
|
||||
|
||||
AgentBridge can serialize the **operator-tunable** state into a versioned JSON blob so a setup can be replicated across machines. The serializer is `src/lib/inspector/configPortability.ts` (`exportConfig()` / `importConfig()`), validated by `AgentBridgeConfigSchema`.
|
||||
|
||||
The export includes exactly three pieces (built-in defaults are intentionally **NOT** exported, so importing never duplicates or fights them):
|
||||
|
||||
| Field | Source | Notes |
|
||||
|-------|--------|-------|
|
||||
| `bypassPatterns` | user-defined bypass patterns (`agent_bridge_bypass`) | default bank/gov/okta patterns are excluded |
|
||||
| `customHosts` | Traffic Inspector custom hosts (`inspector_custom_hosts`) | each: `{ host, kind: "llm"\|"app"\|"custom", label? }` |
|
||||
| `agentMappings` | per-agent model mappings (`agent_bridge_mappings`) | `{ [agentId]: [{ source, target }] }` for every agent that has mappings |
|
||||
|
||||
```jsonc
|
||||
// GET /api/tools/agent-bridge/config
|
||||
{
|
||||
"version": 1,
|
||||
"bypassPatterns": ["*.internal.example.com"],
|
||||
"customHosts": [{ "host": "api.example.com", "kind": "llm", "label": null }],
|
||||
"agentMappings": { "copilot": [{ "source": "gpt-4o", "target": "claude-sonnet-4.7" }] }
|
||||
}
|
||||
```
|
||||
|
||||
**Import behavior** (`POST /api/tools/agent-bridge/config`): bypass patterns and per-agent mappings **replace wholesale**; custom hosts are added **idempotently** (`INSERT OR IGNORE`). The response reports how many of each were applied:
|
||||
|
||||
```jsonc
|
||||
{ "ok": true, "bypassPatterns": 1, "customHosts": 1, "agents": 1 }
|
||||
```
|
||||
|
||||
What is **NOT** in the config: server running state, cert paths, per-agent DNS state, upstream CA path, and TPROXY settings — those are host/runtime state, not portable preferences.
|
||||
|
||||
---
|
||||
|
||||
## §4 Per-agent reference
|
||||
@@ -364,18 +422,31 @@ Base path: `/api/tools/agent-bridge/`
|
||||
|
||||
| Method | Path | Description |
|
||||
|--------|------|-------------|
|
||||
| GET | `/api/tools/agent-bridge/agents` | List all 9 agents with current state |
|
||||
| GET | `/api/tools/agent-bridge/state` | Global server state (running, port, cert info) |
|
||||
| POST | `/api/tools/agent-bridge/server` | Start/stop/restart server (`action: "start"\|"stop"\|"restart"\|"trust-cert"\|"regenerate-cert"`) |
|
||||
| GET | `/api/tools/agent-bridge/agents/{id}` | State of one agent (dns_enabled, cert_trusted, etc.) |
|
||||
| GET | `/api/tools/agent-bridge/state` | Global server state + per-agent detection/status |
|
||||
| GET | `/api/tools/agent-bridge/agents` | List registered agents (id, name, hosts, viability, state) |
|
||||
| GET | `/api/tools/agent-bridge/agents/{id}` | State of one agent (target config + detection + stored state) |
|
||||
| PATCH | `/api/tools/agent-bridge/agents/{id}` | Update `setup_completed` for agent |
|
||||
| GET | `/api/tools/agent-bridge/agents/{id}/detect` | Run detection probe for agent (`installed`, `version?`, `path?`) |
|
||||
| POST | `/api/tools/agent-bridge/agents/{id}/dns` | Enable/disable DNS for agent (`{enabled: boolean}`) |
|
||||
| GET | `/api/tools/agent-bridge/agents/{id}/mappings` | Model mappings for agent |
|
||||
| PUT | `/api/tools/agent-bridge/agents/{id}/mappings` | Update model mappings |
|
||||
| GET | `/api/tools/agent-bridge/bypass` | List bypass patterns |
|
||||
| PUT | `/api/tools/agent-bridge/bypass` | Update bypass patterns |
|
||||
| POST | `/api/tools/agent-bridge/cert` | Download or regenerate CA cert |
|
||||
| PUT | `/api/tools/agent-bridge/agents/{id}/mappings` | Replace model mappings |
|
||||
| POST | `/api/tools/agent-bridge/server` | Start/stop/restart server (`action: "start"\|"stop"\|"restart"\|"trust-cert"\|"regenerate-cert"`) |
|
||||
| GET | `/api/tools/agent-bridge/cert` | Cert status (`exists`, `trusted`, `path`) |
|
||||
| POST | `/api/tools/agent-bridge/cert` | Trust (install) the MITM root CA |
|
||||
| DELETE | `/api/tools/agent-bridge/cert` | Untrust (remove) the MITM root CA — idempotent (see §3.6) |
|
||||
| POST | `/api/tools/agent-bridge/cert/regenerate` | Regenerate the self-signed MITM cert |
|
||||
| GET | `/api/tools/agent-bridge/cert/download` | Stream the PEM cert for download |
|
||||
| GET | `/api/tools/agent-bridge/bypass` | List bypass patterns (`default` + `user`) |
|
||||
| POST | `/api/tools/agent-bridge/bypass` | Replace user-defined bypass patterns wholesale |
|
||||
| DELETE | `/api/tools/agent-bridge/bypass?pattern=...` | Remove a single user-defined bypass pattern |
|
||||
| GET | `/api/tools/agent-bridge/diagnose` | Capture-pipeline self-test (see §3.6) |
|
||||
| POST | `/api/tools/agent-bridge/repair` | Undo orphaned MITM system state (see §3.6) |
|
||||
| GET | `/api/tools/agent-bridge/config` | Export portable config JSON (see §3.7) |
|
||||
| POST | `/api/tools/agent-bridge/config` | Import portable config JSON (see §3.7) |
|
||||
| GET | `/api/tools/agent-bridge/upstream-ca` | Get configured upstream CA path |
|
||||
| POST | `/api/tools/agent-bridge/upstream-ca` | Set upstream CA cert path |
|
||||
| POST | `/api/tools/agent-bridge/upstream-ca` | Validate + persist upstream CA path |
|
||||
| POST | `/api/tools/agent-bridge/upstream-ca/test` | Validate-only (dry-run) an upstream CA path — does not persist |
|
||||
| GET / POST / DELETE | `/api/tools/agent-bridge/tproxy` | TPROXY transparent-decrypt capture mode — see [`docs/security/MITM-TPROXY-DECRYPT.md`](../security/MITM-TPROXY-DECRYPT.md) |
|
||||
|
||||
Full OpenAPI schemas: `docs/reference/openapi.yaml` → tag `AgentBridge`.
|
||||
|
||||
|
||||
@@ -1,13 +1,13 @@
|
||||
---
|
||||
title: "Cloud Agents"
|
||||
version: 3.8.2
|
||||
lastUpdated: 2026-05-13
|
||||
version: 3.8.31
|
||||
lastUpdated: 2026-06-20
|
||||
---
|
||||
|
||||
# Cloud Agents
|
||||
|
||||
> **Source of truth:** `src/lib/cloudAgent/` and `src/app/api/v1/agents/tasks/`
|
||||
> **Last updated:** 2026-05-13 — v3.8.0
|
||||
> **Last updated:** 2026-06-20 — v3.8.31 (frontmatter refresh; 4 agents incl. cursor-cloud)
|
||||
|
||||
OmniRoute orchestrates third-party cloud-hosted coding agents (Codex Cloud, Cursor,
|
||||
Devin, Jules) as long-running tasks. Each agent is wrapped behind a uniform interface so
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
title: "OmniRoute MCP Server Documentation"
|
||||
version: 3.8.8
|
||||
lastUpdated: 2026-05-30
|
||||
version: 3.8.31
|
||||
lastUpdated: 2026-06-20
|
||||
---
|
||||
|
||||
# OmniRoute MCP Server Documentation
|
||||
@@ -310,6 +310,8 @@ Wildcard scopes are supported: `read:*` grants all read-scopes, `*` grants full
|
||||
| `OMNIROUTE_MCP_SCOPES` | (empty) | Comma-separated allowlist of scopes considered "available" by default (used when caller does not provide its own scopes) |
|
||||
| `OMNIROUTE_MCP_COMPRESS_DESCRIPTIONS` | (unset = on) | When set to `0/false/off/no`, disables MCP description compression at registration time |
|
||||
| `OMNIROUTE_MCP_DESCRIPTION_COMPRESSION` | (unset = on) | Alternate alias for the same toggle as above |
|
||||
| `MCP_TOOL_DENY` | (unset = no filter) | Comma-separated tool names to drop from `tools/list` (tool-cardinality reduction — see below) |
|
||||
| `MCP_TOOL_ALLOW` | (unset = no filter) | Comma-separated tool names to keep exclusively (allow-list mode — see below) |
|
||||
| `DATA_DIR` | `~/.omniroute` | Heartbeat file is written to `${DATA_DIR}/runtime/mcp-heartbeat.json` |
|
||||
|
||||
---
|
||||
@@ -325,6 +327,33 @@ MCP tool, prompt, and resource registries can compress descriptions at registrat
|
||||
|
||||
---
|
||||
|
||||
## Tool Cardinality Reduction (F4.3)
|
||||
|
||||
Description compression shrinks each tool's metadata; **tool-cardinality reduction** goes one step further by reducing *how many* tools are announced at all. Advertising fewer tools in the `tools/list` manifest cuts the per-request token cost the client's model pays for the tool catalog ("layer 5" compression). The implementation is a pure, stateless filter in `open-sse/mcp-server/toolCardinality.ts` (`reduceToolManifest`), wired into the registration loop in `createMcpServer()` (`open-sse/mcp-server/server.ts`).
|
||||
|
||||
**Opt-in, off by default.** The filter only runs when at least one of two environment variables is set; with neither set, all 87 tools are announced unchanged.
|
||||
|
||||
| Variable | Mode |
|
||||
| :--------------- | :-------------------------------------------------------------------------------------- |
|
||||
| `MCP_TOOL_DENY` | Blacklist — comma-separated tool names that are always dropped from `tools/list` |
|
||||
| `MCP_TOOL_ALLOW` | Allow-list — comma-separated tool names; only these survive, everything else is dropped |
|
||||
|
||||
`deny` takes priority over `allow`. Names are comma-separated, trimmed, and empty entries are ignored. Examples:
|
||||
|
||||
```bash
|
||||
# Drop two tools from the catalog
|
||||
MCP_TOOL_DENY="omniroute_get_health,omniroute_list_combos" omniroute --mcp
|
||||
|
||||
# Announce only the routing + quota tools (allow-list mode)
|
||||
MCP_TOOL_ALLOW="omniroute_route_request,omniroute_check_quota" omniroute --mcp
|
||||
```
|
||||
|
||||
**How filtered tools are removed:** registration always succeeds; a tool the profile rejects is then `.disable()`d on the MCP SDK handle, so it never appears in `tools/list` but the wiring stays intact (clean enable/disable, no re-registration). The profile parser is `readMcpToolProfileFromEnv(process.env)`, which returns `null` (no filtering) when both vars are empty.
|
||||
|
||||
The richer `ToolProfile` shape behind `reduceToolManifest` also supports scope-intersection filtering (`allowScopes`, with `read:*`-style wildcard matching) and a deterministic `maxTools` cap, but those two knobs need the full manifest at registration time and are **not** exposed through the environment variables today (a `tools/list`-level hook is a tracked follow-up). `estimateManifestTokens()` is available to compare the manifest token cost before and after reduction.
|
||||
|
||||
---
|
||||
|
||||
## Runtime Heartbeat
|
||||
|
||||
The stdio transport persists liveness to `${DATA_DIR}/runtime/mcp-heartbeat.json` every 5 seconds. The dashboard (`/api/mcp/status`) reads this file plus PID liveness to derive `online`. HTTP transports report state from in-process `getMcpHttpStatus()` instead (no file write).
|
||||
|
||||
@@ -1,13 +1,13 @@
|
||||
---
|
||||
title: "Memory System"
|
||||
version: 3.8.6
|
||||
lastUpdated: 2026-05-28
|
||||
version: 3.8.31
|
||||
lastUpdated: 2026-06-20
|
||||
---
|
||||
|
||||
# Memory System
|
||||
|
||||
> **Source of truth:** `src/lib/memory/` and `src/app/api/memory/`
|
||||
> **Last updated:** 2026-05-28 — v3.8.6 (plan 21 — Memory Engine Redesign)
|
||||
> **Last updated:** 2026-06-20 — v3.8.31 (off-by-default + int8 quantization catch-up)
|
||||
|
||||
OmniRoute provides persistent conversational memory keyed by API key (and
|
||||
optionally session id). Memories are extracted automatically from LLM responses
|
||||
@@ -15,6 +15,18 @@ via lightweight regex pattern matching and injected back into subsequent
|
||||
requests as a leading system message (or first user message for providers that
|
||||
reject the system role).
|
||||
|
||||
> **Memory is OFF by default (v3.8.30+).** `DEFAULT_MEMORY_SETTINGS.enabled` is
|
||||
> now `false` (`src/lib/memory/settings.ts`). Enabling memory injects up to
|
||||
> `maxTokens` (~2k) of retrieved context into **every** chat request, which is
|
||||
> billed — a surprising cost for new installs and for clients that manage their
|
||||
> own context. Opt in explicitly under **Settings → Memory** (the
|
||||
> `MemorySkillsTab` shows a token-cost warning callout when memory is enabled).
|
||||
> A client can opt a single request out with the `x-omniroute-no-memory`
|
||||
> request header (`true`/`1`/`yes`) — see the request-header table in
|
||||
> [API_REFERENCE.md](../reference/API_REFERENCE.md). A no-memory request sets
|
||||
> `memoryOwnerId = null`, which disables **both** memory and skill injection for
|
||||
> that request (`open-sse/handlers/chatCore/headers.ts::isNoMemoryRequested`).
|
||||
|
||||
Memory is **scoped per API key**, not per user — every request authenticated
|
||||
with the same API key shares the same memory pool, with optional further
|
||||
scoping by `sessionId`.
|
||||
@@ -235,6 +247,30 @@ routes under `src/app/api/settings/qdrant/` are all wired as of v3.8.6:
|
||||
| `/api/settings/qdrant/cleanup` | `POST` | Remove expired / old points |
|
||||
| `/api/settings/qdrant/embedding-models` | `GET` | List available embedding models |
|
||||
|
||||
### Vector quantization (int8 — opt-in, both backends)
|
||||
|
||||
Both vector backends support **opt-in int8 quantization** to cut the memory
|
||||
footprint of stored vectors (~4× smaller than Float32) at a small recall cost.
|
||||
Default is **off** on both — vectors stay full-precision unless explicitly
|
||||
enabled.
|
||||
|
||||
| Backend | Setting | Type | Default | Where read |
|
||||
| ------------ | -------------------------------- | ----------------------------- | -------- | --------------------------------------------------- |
|
||||
| Qdrant | `qdrantQuantization` (DB key) | `"none" \| "int8" \| "binary"` | `"none"` | `src/lib/memory/qdrant.ts::normalizeQdrantConfig()` |
|
||||
| sqlite-vec | `MEMORY_VEC_QUANTIZATION` (env) | `"none" \| "int8"` | `"none"` | `src/lib/memory/vectorStore.ts::requestedVecQuantization()` |
|
||||
|
||||
- **Qdrant** is configured per-instance via the `qdrantQuantization` setting
|
||||
key (exposed as the `quantization` field on `PUT /api/settings/qdrant`). When
|
||||
`"int8"`, `buildQuantizationConfig()` requests scalar quantization
|
||||
(`always_ram`, quantile `0.99`) and searches enable `rescore: true` so the
|
||||
full-precision vectors refine the int8 candidate set.
|
||||
- **sqlite-vec** quantization is **environment-only** (not a DB setting): set
|
||||
`MEMORY_VEC_QUANTIZATION=int8` to store the local vectors as an `int8[dim]`
|
||||
column via `vec_quantize_int8(?, 'unit')`. The chosen mode is folded into the
|
||||
`embedding_signature` (an `:int8` suffix), so switching modes triggers a full
|
||||
reindex of the `vec_memories` table — the same lazy-backfill path used when
|
||||
the embedding model changes.
|
||||
|
||||
## Memory Types
|
||||
|
||||
`MemoryType` (`src/lib/memory/types.ts`):
|
||||
@@ -332,7 +368,7 @@ route after writes.
|
||||
|
||||
| DB key | Type | Default | UI control |
|
||||
| --------------------- | ------- | -------------------------------------------------- | ----------------------------------------------- |
|
||||
| `memoryEnabled` | boolean | `true` | Memory on/off |
|
||||
| `memoryEnabled` | boolean | `false` (off by default since v3.8.30) | Memory on/off |
|
||||
| `memoryMaxTokens` | integer | `2000` (range `0–16000`) | Token budget for injection |
|
||||
| `memoryRetentionDays` | integer | `30` (range `1–365`) | Retention window |
|
||||
| `memoryStrategy` | enum | `"hybrid"` (one of `recent`, `semantic`, `hybrid`) | Retrieval strategy |
|
||||
@@ -373,6 +409,7 @@ Six optional env vars tune the engine's runtime behaviour (documented in `.env.e
|
||||
| `MEMORY_STATIC_CACHE_DIR` | `<DATA_DIR>/embeddings` | Where to store downloaded models |
|
||||
| `MEMORY_VEC_TOP_K` | `20` | Default top-K for vector search |
|
||||
| `MEMORY_RRF_K` | `60` | RRF k constant for hybrid search |
|
||||
| `MEMORY_VEC_QUANTIZATION` | `none` | Set to `int8` to store local sqlite-vec vectors quantized (~4× smaller; opt-in). Mode change forces a reindex. |
|
||||
|
||||
## Summarisation (`summarization.ts`)
|
||||
|
||||
|
||||
@@ -1,12 +1,12 @@
|
||||
---
|
||||
title: "Traffic Inspector"
|
||||
version: 3.8.6
|
||||
lastUpdated: 2026-05-28
|
||||
version: 3.8.31
|
||||
lastUpdated: 2026-06-20
|
||||
---
|
||||
|
||||
# Traffic Inspector
|
||||
|
||||
Traffic Inspector is OmniRoute's built-in HTTPS traffic debugger — a Charles Proxy / mitmweb / HTTP Toolkit-like tool that is **LLM-aware** and **agent-aware**. It lives at `/dashboard/tools/traffic-inspector` and receives live traffic from up to 4 simultaneous capture sources.
|
||||
Traffic Inspector is OmniRoute's built-in HTTPS traffic debugger — a Charles Proxy / mitmweb / HTTP Toolkit-like tool that is **LLM-aware** and **agent-aware**. It lives at `/dashboard/tools/traffic-inspector` and receives live traffic from up to 5 simultaneous capture sources.
|
||||
|
||||
**Dashboard location:** `/dashboard/tools/traffic-inspector`
|
||||
**Sidebar group:** Tools (after AgentBridge)
|
||||
@@ -42,7 +42,7 @@ The `TrafficBuffer` (`src/mitm/inspector/buffer.ts`) is a shared in-memory ring
|
||||
|
||||
## §2 Capture modes
|
||||
|
||||
Traffic Inspector supports **4 simultaneous capture sources**. Each is independently toggleable.
|
||||
Traffic Inspector supports **5 simultaneous capture sources**. Each is independently toggleable. The `source` field on every `InterceptedRequest` (`src/mitm/inspector/types.ts`) is one of `"agent-bridge"`, `"custom-host"`, `"http-proxy"`, `"system-proxy"`, or `"tproxy"`.
|
||||
|
||||
### Mode 1 — AgentBridge (default, always on)
|
||||
|
||||
@@ -99,6 +99,17 @@ export HTTPS_PROXY=http://127.0.0.1:8080
|
||||
- Dashboard shows "Reverting system proxy" prompt if user navigates away while active
|
||||
- UI shows `⚠ Advanced` badge + explicit confirmation checkbox
|
||||
|
||||
### Mode 5 — TPROXY transparent decrypt (Linux, root, opt-in)
|
||||
|
||||
**Source:** Kernel TPROXY + policy routing (`src/mitm/tproxy/`)
|
||||
**Mechanism:** Marks new local outbound TCP connections to a target port (default `443`) in `mangle OUTPUT`, an `ip rule` reroutes the marked packets to local delivery, and `mangle PREROUTING`'s `TPROXY` target hands them to a transparent (**IP_TRANSPARENT**) listener (default port `8443`). The listener terminates TLS with a leaf certificate issued **per SNI hostname on demand** by a dynamic CA, captures the decrypted exchange, and forwards the request re-encrypted to the original destination.
|
||||
**Reach:** **Arbitrary** destination hosts on the target port — no `/etc/hosts` spoof, no `HTTP_PROXY` env, no system-wide proxy mutation. The intercepted process needs no config change, but must trust the dynamic CA.
|
||||
**Note:** `source` = `"tproxy"`
|
||||
|
||||
**Requirements:** Linux only (**IP_TRANSPARENT** is Linux-only), the **CAP_NET_ADMIN** capability (root), and a native N-API addon that must be built with a C toolchain (`npm run build:native:tproxy`). When unavailable, the dashboard toggle is disabled with the tooltip "TPROXY decrypt requires Linux + root + the native addon". The firewall rules apply/revert transactionally (a crash never leaves a `mangle` rule behind) and flush on reboot. An SO_MARK-based anti-loop keeps the proxy's own re-encrypted forward from being re-intercepted.
|
||||
|
||||
This is a substantial subsystem with its own dedicated operator guide — see **[`docs/security/MITM-TPROXY-DECRYPT.md`](../security/MITM-TPROXY-DECRYPT.md)** for the full firewall recipe, the per-SNI dynamic CA + trust-store installer, the local-only route, anti-loop details, and the configuration schema. The toggle is driven by `GET / POST / DELETE /api/tools/agent-bridge/tproxy` (note: the route lives under the AgentBridge prefix, not the Traffic Inspector prefix).
|
||||
|
||||
### Capture mode comparison
|
||||
|
||||
| Mode | Setup | Sudo? | Reach | Notes |
|
||||
@@ -107,6 +118,7 @@ export HTTPS_PROXY=http://127.0.0.1:8080
|
||||
| 2. Custom Hosts | Per-host input | Yes (hosts file) | Any app using that host | Persisted in DB |
|
||||
| 3. HTTP_PROXY | `export HTTPS_PROXY=...` | No | Apps respecting env | Port 8080, no TLS decrypt by default |
|
||||
| 4. System-wide | Toggle + confirm | Yes | All apps on machine | Auto-disable in 30 min |
|
||||
| 5. TPROXY decrypt | Toggle (Linux + native addon) | Yes (root + CA install) | Any host on the target port | Decrypts arbitrary hosts; off by default — see [MITM-TPROXY-DECRYPT.md](../security/MITM-TPROXY-DECRYPT.md) |
|
||||
|
||||
---
|
||||
|
||||
@@ -166,7 +178,8 @@ export HTTPS_PROXY=http://127.0.0.1:8080
|
||||
| Host filter | Substring match on `host` field |
|
||||
| Agent filter | Dropdown: All / per-agent |
|
||||
| Status filter | All / 2xx / 3xx / 4xx / 5xx / error |
|
||||
| Source filter | All / agent-bridge / custom-host / http-proxy / system-proxy |
|
||||
| Source filter | All / agent-bridge / custom-host / http-proxy / system-proxy / tproxy |
|
||||
| **Live** filter | Show only in-flight (open) requests — `liveOnly` toggle (see §4.6) |
|
||||
|
||||
### 3.5 Resizable panels
|
||||
|
||||
@@ -245,6 +258,47 @@ interface LlmMetadata {
|
||||
}
|
||||
```
|
||||
|
||||
### 4.6 Live in-flight request filter
|
||||
|
||||
The request `status` field is `number | "in-flight" | "error"` — an entry is
|
||||
pushed as `"in-flight"` the moment the request starts and **updated in place**
|
||||
when the response (or error) arrives. The toolbar's **"Live"** toggle
|
||||
(`liveOnly`, i18n key `trafficInspector.liveOnly`) restricts the list to entries
|
||||
whose `status === "in-flight"`, letting you watch open connections in real time.
|
||||
|
||||
The filter is a pure, client-side predicate in
|
||||
`src/lib/inspector/matchesTrafficFilter.ts`:
|
||||
|
||||
```ts
|
||||
if (f.liveOnly && req.status !== "in-flight") return false;
|
||||
```
|
||||
|
||||
The toggle state lives in `useTrafficFilters` (the inspector dashboard hooks) and
|
||||
combines with the other filters (profile, host, agent, source, status, context).
|
||||
|
||||
### 4.7 Process attribution (Linux)
|
||||
|
||||
On Linux, each intercepted request can be attributed to the **originating local
|
||||
process**. Two optional fields are added to `InterceptedRequest`:
|
||||
|
||||
```ts
|
||||
pid?: number; // originating process id (Linux only)
|
||||
processName?: string; // originating process name (Linux only)
|
||||
```
|
||||
|
||||
`src/mitm/inspector/processAttribution.ts` maps the connection's *client*
|
||||
ephemeral port to a PID + name by:
|
||||
|
||||
1. Reading `/proc/net/tcp` and `/proc/net/tcp6` to find the socket inode for the
|
||||
port (`parseProcNetTcpForInode`, a pure fixture-testable parser).
|
||||
2. Scanning `/proc/<pid>/fd/` for a symlink to `socket:[<inode>]`.
|
||||
3. Reading the process name from `/proc/<pid>/comm`.
|
||||
|
||||
A 1-second TTL cache bounds the procfs scan cost under load. Attribution is
|
||||
**best-effort** — any failure resolves to `null` and never blocks capture. On
|
||||
macOS/Windows the function returns `null` (stub; `lsof`/`GetExtendedTcpTable`
|
||||
support is a follow-up).
|
||||
|
||||
---
|
||||
|
||||
## §5 Sessions
|
||||
@@ -396,10 +450,15 @@ Base path: `/api/tools/traffic-inspector/`
|
||||
|
||||
| Method | Path | Description |
|
||||
|--------|------|-------------|
|
||||
| GET | `/capture-modes` | State of all 4 capture modes |
|
||||
| GET | `/capture-modes` | State of the AgentBridge / custom-hosts / HTTP_PROXY / system-proxy modes + the `tls-intercept` toggle |
|
||||
| POST | `/capture-modes/http-proxy` | Start/stop HTTP_PROXY listener (`{action: "start"\|"stop"}`) |
|
||||
| POST | `/capture-modes/system-proxy` | Apply/revert system-wide proxy (`{action: "apply"\|"revert"}`) |
|
||||
| POST | `/capture-modes/tls-intercept` | Toggle HTTPS body decryption in proxy mode |
|
||||
| POST | `/capture-modes/tls-intercept` | Toggle HTTPS body decryption in proxy mode (`{enabled: boolean}`) |
|
||||
|
||||
> **TPROXY decrypt** (capture mode 5) is driven by a **separate** route under the
|
||||
> AgentBridge prefix — `GET / POST / DELETE /api/tools/agent-bridge/tproxy` — not
|
||||
> under `/api/tools/traffic-inspector/`. See
|
||||
> [`docs/security/MITM-TPROXY-DECRYPT.md`](../security/MITM-TPROXY-DECRYPT.md).
|
||||
|
||||
### Sessions
|
||||
|
||||
|
||||
208
docs/guides/CLI-INTEGRATIONS.md
Normal file
208
docs/guides/CLI-INTEGRATIONS.md
Normal file
@@ -0,0 +1,208 @@
|
||||
---
|
||||
title: "CLI Integrations — point any coding CLI at OmniRoute"
|
||||
version: 3.8.31
|
||||
lastUpdated: 2026-06-20
|
||||
---
|
||||
|
||||
# CLI Integrations
|
||||
|
||||
OmniRoute ships a family of `setup-*` commands that configure a coding
|
||||
CLI (Codex, Claude Code, OpenCode, Cline, …) to use OmniRoute as its backend — so
|
||||
the tool talks to **one** endpoint and OmniRoute routes to the right provider with
|
||||
auto-fallback. Each command reads the **live** model catalog from a running
|
||||
OmniRoute (local or remote) and writes the tool's own config file on **your**
|
||||
machine. The API key is referenced by env var wherever the tool supports it, so the
|
||||
secret is never written to disk (the exceptions are noted below).
|
||||
|
||||
There are also two launchers — `omniroute launch` (Claude Code) and
|
||||
`omniroute launch-codex` (Codex) — that spawn the CLI with the right env injected,
|
||||
without writing any config at all.
|
||||
|
||||
For the one-time, hand-written base setup of the two richest integrations, see the
|
||||
per-tool deep dives:
|
||||
|
||||
- [Claude Code configuration](./CLAUDE-CODE-CONFIGURATION.md)
|
||||
- [Codex CLI configuration](./CODEX-CLI-CONFIGURATION.md)
|
||||
- [Remote Mode](./REMOTE-MODE.md) — drive a remote OmniRoute (VPS / Tailnet) from your laptop
|
||||
|
||||
---
|
||||
|
||||
## Master table
|
||||
|
||||
Every command honours the **active context** (set with `omniroute connect`, see
|
||||
[Remote Mode](./REMOTE-MODE.md)) or explicit `--remote <url> --api-key <key>` flags.
|
||||
"Local vs remote" below means: with no flags it targets `http://localhost:20128`;
|
||||
with `--remote` (or an active remote context) it fetches the catalog from that
|
||||
server and writes the config locally.
|
||||
|
||||
| Command | Tool | What it writes | Key flags | Local vs remote |
|
||||
|---------|------|----------------|-----------|-----------------|
|
||||
| `omniroute setup-codex` | OpenAI Codex CLI | `~/.codex/<name>.config.toml` — one profile per matched model (`codex --profile <name>`) | `--remote` `--api-key` `--only` `--dry-run` `--port` `--codex-home` | Both |
|
||||
| `omniroute setup-claude` | Claude Code | `~/.claude/profiles/<name>/settings.json` — one profile per matched model (`CLAUDE_CONFIG_DIR`) | `--remote` `--api-key` `--only` `--dry-run` `--port` `--claude-home` | Both |
|
||||
| `omniroute setup-opencode` | OpenCode (openai-compatible) | `~/.config/opencode/opencode.json` — `omniroute` provider with every catalog model (`opencode -m omniroute/<model>`) | `--remote` `--api-key` `--only` `--model` `--dry-run` `--port` | Both |
|
||||
| `omniroute setup-cline` | Cline | `~/.cline/data/{globalState,secrets}.json` (CLI mode) + prints VS Code extension settings | `--remote` `--api-key` `--model` `--yes` `--dry-run` `--port` `--cline-dir` | Both |
|
||||
| `omniroute setup-kilo` | Kilo Code | `~/.local/share/kilo/auth.json` (CLI) + merges `kilocode.*` into VS Code `settings.json` if present | `--remote` `--api-key` `--model` `--yes` `--dry-run` `--port` `--auth-path` `--vscode-settings` | Both |
|
||||
| `omniroute setup-continue` | Continue / `cn` CLI | `~/.continue/config.yaml` — `provider: openai` models, key via `${{ secrets.OMNIROUTE_API_KEY }}` | `--remote` `--api-key` `--only` `--dry-run` `--port` `--config-path` | Both |
|
||||
| `omniroute setup-cursor` | Cursor | Nothing — prints the in-app steps (Cursor config is opaque SQLite) | `--remote` `--api-key` `--only` `--port` | Both |
|
||||
| `omniroute setup-roo` | Roo Code | `~/.omniroute/roo-settings.json` (import doc) + sets `roo-cline.autoImportSettingsPath` if a VS Code `settings.json` exists | `--remote` `--api-key` `--model` `--yes` `--dry-run` `--port` `--import-path` `--vscode-settings` | Both |
|
||||
| `omniroute setup-crush` | Crush | `~/.config/crush/crush.json` — `openai-compat` provider, key via `$OMNIROUTE_API_KEY` | `--remote` `--api-key` `--only` `--dry-run` `--port` `--config-path` | Both |
|
||||
| `omniroute setup-goose` | Goose | `~/.config/goose/config.yaml` (`GOOSE_PROVIDER`/`OPENAI_HOST`/`GOOSE_MODEL`) + prints env recipe | `--remote` `--api-key` `--model` `--yes` `--dry-run` `--port` `--config-path` | Both |
|
||||
| `omniroute setup-qwen` | Qwen Code | `~/.qwen/settings.json` — openai `modelProvider`, key via `envKey` (`OMNIROUTE_API_KEY`) | `--remote` `--api-key` `--model` `--yes` `--dry-run` `--port` `--config-path` | Both |
|
||||
| `omniroute setup-aider` | Aider | `~/.aider.conf.yml` (`openai-api-base` + `model: openai/<id>`) + prints env recipe | `--remote` `--api-key` `--model` `--yes` `--dry-run` `--port` `--config-path` | Both |
|
||||
| `omniroute setup-gemini` | Gemini CLI (native) | `~/.gemini/settings.json` (`model`) + prints env recipe; base URL is env-only | `--remote` `--api-key` `--model` `--yes` `--dry-run` `--port` `--config-path` | Both |
|
||||
| `omniroute launch` | Claude Code | Nothing — spawns `claude` with `ANTHROPIC_BASE_URL`/`ANTHROPIC_AUTH_TOKEN` injected | `--remote` `--api-key` `--token` `--profile` `--port` | Both |
|
||||
| `omniroute launch-codex` | OpenAI Codex CLI | Nothing — spawns `codex` with the `omniroute` provider injected via `-c` flags | `--remote` `--api-key` `--profile` (`-p`) `--port` | Both |
|
||||
|
||||
Notes on flags (verified in the command source):
|
||||
|
||||
- `--remote <url>` — fetch the catalog from a remote OmniRoute (overrides `--port`
|
||||
and the active context). `--api-key <key>` supplies the credential for that
|
||||
server (defaults to the `OMNIROUTE_API_KEY` env var, or the active context's token).
|
||||
- `--only <patterns>` — comma-separated substrings; keep only model IDs that match
|
||||
(e.g. `--only glm,kimi`). Available on `setup-codex`, `setup-claude`,
|
||||
`setup-opencode`, `setup-continue`, `setup-cursor`, `setup-crush`.
|
||||
- `--dry-run` — print exactly what would be written without touching the
|
||||
filesystem. Available on every `setup-*` command **except** `setup-cursor`
|
||||
(which never writes a file).
|
||||
- `--model <id>` — required (or picked interactively) for the tools that have no
|
||||
model auto-discovery: Cline, Kilo, Roo, Goose, Qwen, Aider, Gemini. Those tools
|
||||
also accept `--yes` for non-interactive runs (which then requires `--model`).
|
||||
`setup-opencode` takes `--model` to set the default top-level model.
|
||||
- `--port <port>` — local OmniRoute port (default `20128`, ignored when `--remote`
|
||||
is set). Present on all `setup-*` and both launchers.
|
||||
- The two launchers (`launch`, `launch-codex`) accept `--profile <name>` to select
|
||||
a profile written by `setup-claude` / `setup-codex`, plus pass-through args for
|
||||
the underlying `claude` / `codex` binary.
|
||||
|
||||
> `setup-opencode` is the **lightweight openai-compatible** OpenCode integration.
|
||||
> There is also a richer plugin integration — `omniroute setup opencode` — which
|
||||
> installs `@omniroute/opencode-plugin`. They are different commands; the table
|
||||
> above documents `setup-opencode`.
|
||||
|
||||
---
|
||||
|
||||
## Local usage
|
||||
|
||||
With OmniRoute running on `localhost:20128`, just run the setup command for your
|
||||
tool. The catalog is fetched from the local server.
|
||||
|
||||
```bash
|
||||
# Codex: write a profile per matched model into ~/.codex/
|
||||
omniroute setup-codex
|
||||
codex --profile glm52 # use a generated profile
|
||||
|
||||
# Claude Code: write per-model profiles, then launch one
|
||||
omniroute setup-claude
|
||||
omniroute launch --profile glm52
|
||||
|
||||
# OpenCode: write the openai-compatible provider with all catalog models
|
||||
omniroute setup-opencode
|
||||
export OMNIROUTE_API_KEY=sk-... # referenced via {env:OMNIROUTE_API_KEY}, never on disk
|
||||
opencode -m omniroute/glm/glm-5.2 "..."
|
||||
|
||||
# Tools without auto-discovery need an explicit model:
|
||||
omniroute setup-aider --model glm/glm-5.2
|
||||
omniroute setup-qwen --model kmc/kimi-k2.7
|
||||
|
||||
# Preview without writing anything:
|
||||
omniroute setup-continue --dry-run
|
||||
```
|
||||
|
||||
Launch without writing any config at all (env-injection only):
|
||||
|
||||
```bash
|
||||
omniroute launch # Claude Code → local OmniRoute
|
||||
omniroute launch-codex # Codex CLI → local OmniRoute
|
||||
omniroute launch-codex --profile glm52
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Remote usage
|
||||
|
||||
Point any setup command at a remote OmniRoute with `--remote` + `--api-key`. The
|
||||
catalog is fetched from the remote; the config is written on your local machine.
|
||||
|
||||
```bash
|
||||
# OpenCode against a remote VPS, keep only glm/kimi models
|
||||
omniroute setup-opencode --remote http://192.168.0.15:20128 --api-key oma_live_xxx \
|
||||
--only glm,kimi
|
||||
opencode -m omniroute/glm/glm-5.2 "..." # export OMNIROUTE_API_KEY first
|
||||
|
||||
# Codex profiles from a remote catalog
|
||||
omniroute setup-codex --remote http://192.168.0.15:20128 --api-key oma_live_xxx
|
||||
|
||||
# Launch a CLI straight against the remote
|
||||
omniroute launch --remote http://192.168.0.15:20128 --api-key oma_live_xxx
|
||||
omniroute launch-codex --remote http://192.168.0.15:20128 --api-key oma_live_xxx
|
||||
```
|
||||
|
||||
Instead of passing `--remote`/`--api-key` every time, log in once and let the
|
||||
**active context** supply them automatically:
|
||||
|
||||
```bash
|
||||
omniroute connect 192.168.0.15 # mints a scoped token, stores the context
|
||||
omniroute setup-codex # ← now uses the remote catalog
|
||||
omniroute setup-opencode # ← same
|
||||
omniroute launch # ← Claude Code against the remote
|
||||
```
|
||||
|
||||
See [Remote Mode](./REMOTE-MODE.md) for contexts, scopes, and token management.
|
||||
|
||||
---
|
||||
|
||||
## Base URL conventions (which tools want `/v1`)
|
||||
|
||||
OmniRoute exposes the OpenAI surface at `/v1`, the Anthropic surface at the root,
|
||||
and a native Gemini surface at `/v1beta`. Each integration is wired to the form its
|
||||
tool expects (verified in the command source):
|
||||
|
||||
| Integration | Base URL written | `/v1`? |
|
||||
|-------------|------------------|--------|
|
||||
| `setup-cline` (`openAiBaseUrl`) | root | No — Cline appends `/v1/chat/completions` |
|
||||
| `setup-goose` (`OPENAI_HOST`) | root | No — Goose appends the path |
|
||||
| `setup-aider` (`OPENAI_API_BASE`) | root | No — LiteLLM appends `/v1/chat/completions` |
|
||||
| `setup-kilo`, `setup-roo`, `setup-continue`, `setup-crush`, `setup-qwen`, `setup-cursor` | with `/v1` | Yes |
|
||||
| `setup-claude` (`ANTHROPIC_BASE_URL`), `launch` | root | No — Claude Code appends `/v1/messages` |
|
||||
| `setup-codex`, `launch-codex` (`model_providers.omniroute.base_url`) | with `/v1` | Yes |
|
||||
| `setup-gemini` (`GOOGLE_GEMINI_BASE_URL`) | root | No — the genai SDK appends `/v1beta` |
|
||||
|
||||
> Gemini CLI caveat: a cached Google login can make the CLI ignore
|
||||
> `GOOGLE_GEMINI_BASE_URL`. Run it logged-out / API-key-only so the base URL takes
|
||||
> effect (`setup-gemini` prints this warning too).
|
||||
|
||||
---
|
||||
|
||||
## Keeping native deps on update: `--include=optional`
|
||||
|
||||
When you update with `omniroute update` (after confirming, or with `--apply`),
|
||||
OmniRoute runs the install with `--include=optional` baked in:
|
||||
|
||||
```bash
|
||||
npm install -g omniroute@latest --include=optional
|
||||
```
|
||||
|
||||
This is **not** a flag you pass to `omniroute update` — it is always applied by the
|
||||
updater. It guarantees the `optionalDependencies` (`better-sqlite3`, `keytar`,
|
||||
`tls-client`, the LLMLingua SLM stack) survive the update even if your npm config
|
||||
has `omit=optional` set, which would otherwise silently drop the native SQLite
|
||||
driver and OS-keyring binding. To preview the exact command without applying:
|
||||
|
||||
```bash
|
||||
omniroute update --dry-run
|
||||
# [DRY RUN] Would run: npm install -g omniroute@latest --include=optional
|
||||
```
|
||||
|
||||
Other `omniroute update` flags (verified in source): `--check` (exit 1 if
|
||||
outdated), `--apply` (install without prompting), `--changelog`, `--no-backup`,
|
||||
`--yes`.
|
||||
|
||||
---
|
||||
|
||||
## See also
|
||||
|
||||
- [Claude Code configuration](./CLAUDE-CODE-CONFIGURATION.md) — the deeper Claude Code guide
|
||||
- [Codex CLI configuration](./CODEX-CLI-CONFIGURATION.md) — the one-time `[model_providers.omniroute]` base setup
|
||||
- [Remote Mode](./REMOTE-MODE.md) — contexts, scoped access tokens, driving a remote server
|
||||
- [CLI Tools reference](../reference/CLI-TOOLS.md) — the full catalog of supported tools + dashboard pages
|
||||
- [Setup Guide](./SETUP_GUIDE.md) — install methods and first-run onboarding
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
title: "📖 Setup Guide — OmniRoute"
|
||||
version: 3.8.2
|
||||
lastUpdated: 2026-05-13
|
||||
version: 3.8.31
|
||||
lastUpdated: 2026-06-20
|
||||
---
|
||||
|
||||
# 📖 Setup Guide — OmniRoute
|
||||
@@ -164,6 +164,35 @@ Ollama Tags URL: http://localhost:20128/api/v1/vscode/YOUR_KEY/api/tags
|
||||
|
||||
Works with Claude Code, Codex CLI, Gemini CLI, Cursor, Cline, OpenClaw, OpenCode, and OpenAI-compatible SDKs.
|
||||
|
||||
#### Auto-configure with `setup-*`
|
||||
|
||||
Instead of pasting the base URL and key by hand, let OmniRoute write each tool's
|
||||
own config from the live model catalog. One command per tool:
|
||||
|
||||
```bash
|
||||
omniroute setup-codex # ~/.codex/<name>.config.toml profiles
|
||||
omniroute setup-claude # ~/.claude/profiles/<name>/settings.json
|
||||
omniroute setup-opencode # ~/.config/opencode/opencode.json (openai-compatible)
|
||||
omniroute setup-cline # Cline CLI + VS Code extension settings
|
||||
omniroute setup-kilo # Kilo Code
|
||||
omniroute setup-continue # ~/.continue/config.yaml (Continue / cn)
|
||||
omniroute setup-cursor # prints Cursor's in-app steps
|
||||
omniroute setup-roo # Roo Code import + autoImport pointer
|
||||
omniroute setup-crush # ~/.config/crush/crush.json
|
||||
omniroute setup-goose # ~/.config/goose/config.yaml
|
||||
omniroute setup-qwen # ~/.qwen/settings.json
|
||||
omniroute setup-aider # ~/.aider.conf.yml
|
||||
omniroute setup-gemini # Gemini CLI (native /v1beta endpoint)
|
||||
```
|
||||
|
||||
Each accepts `--remote <url> --api-key <key>` to configure a local tool against a
|
||||
**remote** OmniRoute, plus `--dry-run` to preview. The launchers
|
||||
`omniroute launch` (Claude Code) and `omniroute launch-codex` (Codex) spawn the CLI
|
||||
with the right env injected, writing no config at all.
|
||||
|
||||
For the full table (what each command writes, every flag, local vs remote, base-URL
|
||||
`/v1` conventions), see **[CLI Integrations](./CLI-INTEGRATIONS.md)**.
|
||||
|
||||
For detailed per-tool configuration (Claude Code, Codex CLI, Cursor, Cline, OpenClaw, Kilo Code, Copilot, and more), see the dedicated **[CLI Tools Guide](../reference/CLI-TOOLS.md)**.
|
||||
|
||||
---
|
||||
|
||||
@@ -63,6 +63,7 @@ Content-Type: application/json
|
||||
| Header | Direction | Description |
|
||||
| ------------------------ | --------- | ------------------------------------------------ |
|
||||
| `X-OmniRoute-No-Cache` | Request | Set to `true` to bypass cache |
|
||||
| `x-omniroute-no-memory` | Request | Set to `true` to skip memory + skills injection for this request (mirrors no-cache; avoids the per-call token/cost overhead) |
|
||||
| `X-OmniRoute-Progress` | Request | Set to `true` for progress events |
|
||||
| `X-Session-Id` | Request | Sticky session key for external session affinity |
|
||||
| `x_session_id` | Request | Underscore variant also accepted (direct HTTP) |
|
||||
|
||||
@@ -1,12 +1,12 @@
|
||||
---
|
||||
title: "CLI Tools — OmniRoute v3.8.6"
|
||||
version: 3.8.6
|
||||
lastUpdated: 2026-05-28
|
||||
title: "CLI Tools — OmniRoute"
|
||||
version: 3.8.31
|
||||
lastUpdated: 2026-06-20
|
||||
---
|
||||
|
||||
# CLI Tools — OmniRoute v3.8.6
|
||||
# CLI Tools — OmniRoute
|
||||
|
||||
Last updated: 2026-05-28
|
||||
Last updated: 2026-06-20
|
||||
|
||||
OmniRoute integrates with three categories of CLI tools spread across three dedicated dashboard pages:
|
||||
|
||||
@@ -45,6 +45,33 @@ ACP Agents (reverse spawn flow):
|
||||
|
||||
---
|
||||
|
||||
## Auto-configure with `setup-*`
|
||||
|
||||
You do not have to write each tool's config by hand. OmniRoute ships a `setup-*`
|
||||
command per supported CLI that reads the **live** model catalog from a running
|
||||
OmniRoute (local or remote) and writes the tool's own config on your machine:
|
||||
|
||||
```bash
|
||||
omniroute setup-codex omniroute setup-claude omniroute setup-opencode
|
||||
omniroute setup-cline omniroute setup-kilo omniroute setup-continue
|
||||
omniroute setup-cursor omniroute setup-roo omniroute setup-crush
|
||||
omniroute setup-goose omniroute setup-qwen omniroute setup-aider
|
||||
omniroute setup-gemini
|
||||
```
|
||||
|
||||
Each accepts `--remote <url> --api-key <key>` (configure a local tool against a
|
||||
remote OmniRoute), `--dry-run` (preview without writing), and `--port`. Tools
|
||||
without model auto-discovery (Cline, Kilo, Roo, Goose, Qwen, Aider, Gemini) take
|
||||
`--model <id>` (and `--yes` for non-interactive runs). The launchers
|
||||
`omniroute launch` (Claude Code) and `omniroute launch-codex` (Codex) spawn the CLI
|
||||
with the right env injected and write no config at all.
|
||||
|
||||
> **Full reference:** the master table — what each command writes, every flag,
|
||||
> local vs remote, and which tools want a `/v1` suffix — lives in
|
||||
> **[CLI Integrations](../guides/CLI-INTEGRATIONS.md)**.
|
||||
|
||||
---
|
||||
|
||||
## Source of Truth
|
||||
|
||||
The unified catalog lives in `src/shared/constants/cliTools.ts` as `CLI_TOOLS: Record<string, CliCatalogEntry>`.
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
title: "Environment Variables Reference"
|
||||
version: 3.8.2
|
||||
lastUpdated: 2026-05-13
|
||||
version: 3.8.31
|
||||
lastUpdated: 2026-06-20
|
||||
---
|
||||
|
||||
# Environment Variables Reference
|
||||
@@ -766,12 +766,34 @@ Anthropic-compatible provider instead.
|
||||
| `PROVIDER_COOLDOWN_ENABLED` | _(unset → off)_ | `open-sse/services/providerCooldownTracker.ts` | Opt-in global cross-request provider/connection cooldown tracking. OFF by default (overlaps Connection Cooldown / Provider Circuit Breaker). Accepts `true`/`1`/`on` to enable. |
|
||||
| `PROVIDER_COOLDOWN_MIN_MS` | `5000` | `open-sse/services/providerCooldownTracker.ts` | Minimum cooldown (ms) before a failed provider/connection is retried. Scaled exponentially with consecutive failures. Only used when `PROVIDER_COOLDOWN_ENABLED`. |
|
||||
| `PROVIDER_COOLDOWN_MAX_MS` | `300000` (5 min) | `open-sse/services/providerCooldownTracker.ts` | Maximum cooldown (ms) cap before a failed provider/connection is retried regardless. Only used when `PROVIDER_COOLDOWN_ENABLED`. |
|
||||
| `STREAM_RECOVERY_ENABLED` | _(unset → off)_ | `open-sse/services/streamRecovery.ts` | Opt-in transparent recovery of truncated upstream streams (free-claude-code port). Holds the opening SSE window so an early cutoff is retried invisibly; OFF by default (adds time-to-first-token latency). Accepts `true`/`1`/`on` to enable. |
|
||||
| `STREAM_RECOVERY_MIDSTREAM_ENABLED` | _(unset → off)_ | `open-sse/services/streamRecovery.ts` | Opt-in mid-stream continuation (Fase 4.4): after a post-commit truncation, re-request with the partial text as an assistant prefill and stitch the missing suffix (plain-text OpenAI-compatible streams only, never with a tool call in flight). OFF by default — the recovered tail arrives as one burst. Independent of `STREAM_RECOVERY_ENABLED`. Accepts `true`/`1`/`on` to enable. |
|
||||
| `STREAM_RECOVERY_ENABLED` | _(unset → off)_ | `src/lib/resilience/settings.ts` (seed) → `open-sse/services/streamRecovery.ts` (logic) | **What:** transparent recovery of truncated upstream streams (free-claude-code port). Holds the opening SSE window up to `STREAM_RECOVERY.HOLDBACK_MS` (750 ms) so a *pre-commit* cutoff — one that happens before any byte reaches the client — is re-opened and retried invisibly. **When to enable:** flaky/upstreams that frequently 0-byte-truncate at stream start; leave OFF if you cannot afford up to 750 ms of added time-to-first-token on every stream. Accepts `true`/`1`/`on`. Seeds the persisted Resilience setting; the Dashboard setting wins once set. |
|
||||
| `STREAM_RECOVERY_MIDSTREAM_ENABLED` | _(unset → off)_ | `src/lib/resilience/settings.ts` (seed) → `open-sse/services/streamRecovery.ts` (logic) | **What:** mid-stream continuation (Fase 4.4) — after a *post-commit* truncation (bytes already reached the client), re-request with the partial text as an assistant prefill and stitch the missing suffix. Plain-text OpenAI-compatible streams only; never fires with a tool call in flight. **When to enable:** long generations that get cut mid-answer and you accept the recovered tail arriving as one burst rather than token-by-token. Independent of `STREAM_RECOVERY_ENABLED` (different risk profile). Accepts `true`/`1`/`on`. |
|
||||
| `HEALTHCHECK_STAGGER_MS` | `3000` | `src/lib/tokenHealthCheck.ts` | Stagger interval (ms) between provider token healthchecks at startup. |
|
||||
| `REQUEST_RETRY` | `2` | `src/sse/services/cooldownAwareRetry.ts` | Number of automatic retries on model-scoped cooldown responses before returning error to client. |
|
||||
| `MAX_RETRY_INTERVAL_SEC` | `30` | `src/sse/services/cooldownAwareRetry.ts` | Max backoff interval (seconds) between cooldown retries. Capped by this value regardless of upstream `Retry-After`. |
|
||||
|
||||
### Stream-recovery tuning constants (not env vars)
|
||||
|
||||
The two `STREAM_RECOVERY_*` flags above are the only operator-facing toggles. The
|
||||
recovery behavior is otherwise tuned by hardcoded constants in
|
||||
`open-sse/config/constants.ts` (`STREAM_RECOVERY`), shown here for reference —
|
||||
changing them requires a code edit, not an env var:
|
||||
|
||||
- `STREAM_RECOVERY.HOLDBACK_MS = 750` — how long the opening SSE window is held
|
||||
so an early truncation can be retried before any byte is committed to the client.
|
||||
- `STREAM_RECOVERY.BUFFER_MAX_BYTES = 65536` — hard cap on the held window; commit
|
||||
(flush + passthrough) as soon as this many bytes accumulate, regardless of the timer.
|
||||
- `STREAM_RECOVERY.EARLY_RETRY_MAX = 4` — max transparent re-opens of the upstream
|
||||
stream while the holdback is still uncommitted.
|
||||
|
||||
> **Per-provider sliding-window rate limit (no env var):** the FCC-ported
|
||||
> per-provider sliding-window rate-limit *fallback* exists in code
|
||||
> (`open-sse/services/providerDefaultRateLimit.ts`, wired through
|
||||
> `open-sse/services/rateLimitManager.ts`) but ships with an **empty default map**
|
||||
> and has **no operator env var** today — it is enabled only via a test hook /
|
||||
> code edit. It is intentionally not listed in the table above. The per-`(token, IP)`
|
||||
> relay limiter that *does* have a knob is `RELAY_IP_PER_MINUTE` (§3 Network & Ports).
|
||||
|
||||
---
|
||||
|
||||
## 22. Debugging
|
||||
|
||||
@@ -1,16 +1,16 @@
|
||||
---
|
||||
title: "Provider Reference"
|
||||
version: 3.8.27
|
||||
lastUpdated: 2026-06-17
|
||||
version: 3.8.31
|
||||
lastUpdated: 2026-06-20
|
||||
---
|
||||
|
||||
# Provider Reference
|
||||
|
||||
> **Auto-generated** from `src/shared/constants/providers.ts` — do not edit by hand.
|
||||
> Regenerate with: `npm run gen:provider-reference`
|
||||
> **Last generated:** 2026-06-17
|
||||
> **Last generated:** 2026-06-20
|
||||
|
||||
Total providers: **227**. See category breakdown below.
|
||||
Total providers: **231**. See category breakdown below.
|
||||
|
||||
## Categories
|
||||
|
||||
@@ -33,277 +33,281 @@ Use the dashboard at `/dashboard/providers` to enable, configure, and test each
|
||||
|
||||
## OAuth Providers (19)
|
||||
|
||||
| ID | Alias | Name | Tags | Website | Notes |
|
||||
| ------------- | ------------ | -------------------- | ----- | ------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `agy` | `agy` | Antigravity CLI | OAuth | [link](https://antigravity.google) | Import your Antigravity CLI (`agy`) login (paste/upload its token file), auto-detect a local CLI login, or sign in with Google. Shares the Antigravity backend (incl. Claude models). |
|
||||
| `amazon-q` | `aq` | Amazon Q | OAuth | [link](https://aws.amazon.com/q/developer/) | Uses the same AWS Builder ID or imported refresh-token flow as Kiro, but keeps Amazon Q connections separate. |
|
||||
| `antigravity` | — | Antigravity | OAuth | — | — |
|
||||
| `claude` | `cc` | Claude Code | OAuth | — | — |
|
||||
| `cline` | `cl` | Cline | OAuth | — | — |
|
||||
| `codex` | `cx` | OpenAI Codex | OAuth | — | — |
|
||||
| `cursor` | `cu` | Cursor IDE | OAuth | — | — |
|
||||
| `devin-cli` | `dv` | Devin CLI (Official) | OAuth | [link](https://cli.devin.ai) | Requires the Devin CLI binary. Run `devin auth login` to authenticate, or provide your WINDSURF_API_KEY. Install: https://cli.devin.ai |
|
||||
| `gemini-cli` | `gemini-cli` | Gemini CLI | OAuth | — | Uses Gemini CLI OAuth / Cloud Code credentials. Pro models require an eligible Google account or paid plan. |
|
||||
| `github` | `gh` | GitHub Copilot | OAuth | — | — |
|
||||
| `gitlab-duo` | `gitlab-duo` | GitLab Duo | OAuth | [link](https://docs.gitlab.com/user/duo_agent_platform/code_suggestions/) | OAuth application with ai_features + read_user scopes. Configure GITLAB_DUO_OAUTH_CLIENT_ID and optionally GITLAB_DUO_OAUTH_CLIENT_SECRET on this OmniRoute instance. |
|
||||
| `kilocode` | `kc` | Kilo Code | OAuth | — | — |
|
||||
| `kimi-coding` | `kmc` | Kimi Coding | OAuth | — | — |
|
||||
| `kiro` | `kr` | Kiro AI | OAuth | — | Free tier: 50 credits/month (~25K–100K tokens). ⚠️ Kiro ToS prohibits third-party proxy/harness use. |
|
||||
| `qoder` | `if` | Qoder AI | OAuth | — | — |
|
||||
| `qwen` | `qw` | Qwen Code | OAuth | — | ⚠️ **DEPRECATED.** Qwen OAuth free tier was discontinued on 2026-04-15. Use 'bailian-coding-plan', 'alibaba', 'alibaba-cn', or 'openrouter' provider with API key instead. |
|
||||
| `trae` | `tr` | Trae | OAuth | [link](https://trae.ai) | Trae is an AI-native IDE by ByteDance (SOLO remote agent). Authorize via trae.ai in the popup, or sign in at solo.trae.ai and paste the Cloud-IDE-JWT (sent as 'Authorization: Cloud-IDE-JWT <token>', ~14-day lifetime) as the access token; web_id/biz_user_id/user_unique_id/scope/tenant/region propagate via providerSpecificData. No headless refresh for pasted tokens — re-paste on expiry. |
|
||||
| `windsurf` | `ws` | Windsurf (Devin CLI) | OAuth | [link](https://windsurf.com) | In the Windsurf / VS Code IDE, open the command palette and run `Windsurf: Provide Auth Token` (or click the Jupyter "Get Windsurf Authentication Token" button), then copy the shown token and paste it here. Note: opening windsurf.com/show-auth-token directly only renders a "Redirecting" page — the IDE must initiate the flow (it adds a `?state=...` param) for the token to appear. |
|
||||
| `zed` | `zd` | Zed IDE | OAuth | [link](https://zed.dev) | Zed stores LLM provider credentials (OpenAI, Anthropic, Google, Mistral, xAI) in the OS keychain. Use the Import button below to discover and import them automatically. |
|
||||
| ID | Alias | Name | Tags | Website | Notes |
|
||||
|----|-------|------|------|---------|-------|
|
||||
| `agy` | `agy` | Antigravity CLI | OAuth | [link](https://antigravity.google) | Import your Antigravity CLI (`agy`) login (paste/upload its token file), auto-detect a local CLI login, or sign in with Google. Shares the Antigravity backend (incl. Claude models). |
|
||||
| `amazon-q` | `aq` | Amazon Q | OAuth | [link](https://aws.amazon.com/q/developer/) | Uses the same AWS Builder ID or imported refresh-token flow as Kiro, but keeps Amazon Q connections separate. |
|
||||
| `antigravity` | — | Antigravity | OAuth | — | — |
|
||||
| `claude` | `cc` | Claude Code | OAuth | — | — |
|
||||
| `cline` | `cl` | Cline | OAuth | — | — |
|
||||
| `codex` | `cx` | OpenAI Codex | OAuth | — | — |
|
||||
| `cursor` | `cu` | Cursor IDE | OAuth | — | — |
|
||||
| `devin-cli` | `dv` | Devin CLI (Official) | OAuth | [link](https://cli.devin.ai) | Requires the Devin CLI binary. Run `devin auth login` to authenticate, or provide your WINDSURF_API_KEY. Install: https://cli.devin.ai |
|
||||
| `gemini-cli` | `gemini-cli` | Gemini CLI | OAuth | — | Uses Gemini CLI OAuth / Cloud Code credentials. Pro models require an eligible Google account or paid plan. |
|
||||
| `github` | `gh` | GitHub Copilot | OAuth | — | — |
|
||||
| `gitlab-duo` | `gitlab-duo` | GitLab Duo | OAuth | [link](https://docs.gitlab.com/user/duo_agent_platform/code_suggestions/) | OAuth application with ai_features + read_user scopes. Configure GITLAB_DUO_OAUTH_CLIENT_ID and optionally GITLAB_DUO_OAUTH_CLIENT_SECRET on this OmniRoute instance. |
|
||||
| `kilocode` | `kc` | Kilo Code | OAuth | — | — |
|
||||
| `kimi-coding` | `kmc` | Kimi Coding | OAuth | — | — |
|
||||
| `kiro` | `kr` | Kiro AI | OAuth | — | Free tier: 50 credits/month (~25K–100K tokens). ⚠️ Kiro ToS prohibits third-party proxy/harness use. |
|
||||
| `qoder` | `if` | Qoder AI | OAuth | — | — |
|
||||
| `qwen` | `qw` | Qwen Code | OAuth | — | ⚠️ **DEPRECATED.** Qwen OAuth free tier was discontinued on 2026-04-15. Use 'bailian-coding-plan', 'alibaba', 'alibaba-cn', or 'openrouter' provider with API key instead. |
|
||||
| `trae` | `tr` | Trae | OAuth | [link](https://trae.ai) | Trae is an AI-native IDE by ByteDance (SOLO remote agent). Authorize via trae.ai in the popup, or sign in at solo.trae.ai and paste the Cloud-IDE-JWT (sent as 'Authorization: Cloud-IDE-JWT <token>', ~14-day lifetime) as the access token; web_id/biz_user_id/user_unique_id/scope/tenant/region propagate via providerSpecificData. No headless refresh for pasted tokens — re-paste on expiry. |
|
||||
| `windsurf` | `ws` | Windsurf (Devin CLI) | OAuth | [link](https://windsurf.com) | In the Windsurf / VS Code IDE, open the command palette and run `Windsurf: Provide Auth Token` (or click the Jupyter "Get Windsurf Authentication Token" button), then copy the shown token and paste it here. Note: opening windsurf.com/show-auth-token directly only renders a "Redirecting" page — the IDE must initiate the flow (it adds a `?state=...` param) for the token to appear. |
|
||||
| `zed` | `zd` | Zed IDE | OAuth | [link](https://zed.dev) | Zed stores LLM provider credentials (OpenAI, Anthropic, Google, Mistral, xAI) in the OS keychain. Use the Import button below to discover and import them automatically. |
|
||||
|
||||
## Web Cookie Providers (22)
|
||||
|
||||
| ID | Alias | Name | Tags | Website | Notes |
|
||||
| ----------------- | ------------- | ---------------------------- | ---------- | -------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `adapta-web` | `adp-web` | Adapta.org (Adapta One Web) | Web cookie | [link](https://agent.adapta.one) | Paste your \_\_client cookie value from .clerk.agent.adapta.one (DevTools → Application → Cookies) |
|
||||
| `blackbox-web` | `bb-web` | Blackbox Web (Subscription) | Web cookie | [link](https://app.blackbox.ai) | Paste your \_\_Secure-authjs.session-token value or full cookie header from app.blackbox.ai |
|
||||
| `chatgpt-web` | `cgpt-web` | ChatGPT Web (Plus/Pro) | Web cookie | [link](https://chatgpt.com) | Paste your \_\_Secure-next-auth.session-token cookie value from chatgpt.com |
|
||||
| `claude-web` | `cw` | Claude Web | Web cookie | [link](https://claude.ai) | Paste your session cookie from claude.ai |
|
||||
| `copilot-web` | `copilot` | Microsoft Copilot Web | Web cookie | [link](https://copilot.microsoft.com) | Paste your access_token from copilot.microsoft.com (or export a .har file from DevTools while logged in) |
|
||||
| `deepseek-web` | `ds-web` | DeepSeek Web | Web cookie | [link](https://chat.deepseek.com) | Paste your userToken from chat.deepseek.com — DevTools → Application → Local Storage → userToken |
|
||||
| `doubao-web` | `db` | Doubao Web (ByteDance) | Web cookie | [link](https://www.doubao.com) | Paste your session cookie from doubao.com (DevTools → Application → Cookies) |
|
||||
| `gemini-business` | `gembiz` | Gemini Business (Enterprise) | Web cookie | [link](https://business.gemini.google) | From your enterprise account: open business.gemini.google/home/cid/{your-cid}, then copy **Secure-1PSID and **Secure-1PSIDTS cookies from DevTools → Application → Cookies. Paste as a cookie header below. |
|
||||
| `gemini-web` | `gweb` | Gemini Web (Free) | Web cookie | [link](https://gemini.google.com) | Paste your **Secure-1PSID cookie value from gemini.google.com. Optionally add **Secure-1PSIDTS separated by semicolon. |
|
||||
| `grok-web` | `gw` | Grok Web (Subscription) | Web cookie | [link](https://grok.com) | Paste the full grok.com cookie line from DevTools → Application → Cookies. Include both `sso` and `sso-rw` (e.g. `sso=...; sso-rw=...`) — Grok's anti-bot rejects `sso` on its own. |
|
||||
| `huggingchat` | `huggingchat` | HuggingChat (Free) | Web cookie | [link](https://huggingface.co/chat) | Paste your hf-chat cookie value from huggingface.co/chat (DevTools → Application → Cookies → hf-chat). Optional — works without auth for basic use. |
|
||||
| `inner-ai` | `in-ai` | Inner.ai (Subscription) | Web cookie | [link](https://app.innerai.com) | Paste your token cookie and email separated by a space: open DevTools → Application → Cookies → .innerai.com, copy the token value, then append a space and your Inner.ai login email. Example: eyJhbG... user@example.com |
|
||||
| `kimi-web` | `kimi-web` | Kimi Web (Moonshot AI) | Web cookie | [link](https://kimi.moonshot.cn) | Paste your session cookie from kimi.moonshot.cn (DevTools → Application → Cookies) |
|
||||
| `lmarena` | `lma` | LMArena (Free) | Web cookie | [link](https://lmarena.ai) | Paste your session cookie from lmarena.ai (DevTools → Application → Cookies). Optional — works with free tier for basic comparisons. |
|
||||
| `muse-spark-web` | `ms-web` | Muse Spark Web (Meta AI) | Web cookie | [link](https://www.meta.ai) | Paste your abra_sess value or full cookie header from meta.ai |
|
||||
| `perplexity-web` | `pplx-web` | Perplexity Web (Pro/Max) | Web cookie | [link](https://www.perplexity.ai) | Paste your \_\_Secure-next-auth.session-token cookie value from perplexity.ai |
|
||||
| `phind` | `ph` | Phind (Free) | Web cookie | [link](https://www.phind.com) | Paste your session cookie from phind.com (DevTools → Application → Cookies). Optional — works with free tier. |
|
||||
| `poe-web` | `poe` | Poe Web (Subscription) | Web cookie | [link](https://poe.com) | Paste your p-b cookie value from poe.com (DevTools → Application → Cookies → p-b) |
|
||||
| `qwen-web` | `qwen-web` | Qwen Web (Free) | Web cookie | [link](https://chat.qwen.ai) | Open chat.qwen.ai, log in, then open DevTools → Application → Local Storage → copy the "token" value (or use tongyi_sso_ticket cookie as Bearer token). |
|
||||
| `t3-web` | `t3chat` | t3.chat (Pro/Free) | Web cookie | [link](https://t3.chat) | Open t3.chat in your browser, log in, then open DevTools → Application → Local Storage → https://t3.chat. Copy the value of 'convex-session-id'. Also open DevTools → Network, copy the Cookie header from any request. Paste both values here. See provider setup docs for a step-by-step guide. |
|
||||
| `v0-vercel-web` | `v0` | v0 Vercel Web (Code Gen) | Web cookie | [link](https://v0.dev) | Paste your session cookie from v0.dev (DevTools → Application → Cookies) |
|
||||
| `venice-web` | `ven` | Venice Web (Privacy) | Web cookie | [link](https://venice.ai) | Paste your session cookie from venice.ai (DevTools → Application → Cookies) |
|
||||
| ID | Alias | Name | Tags | Website | Notes |
|
||||
|----|-------|------|------|---------|-------|
|
||||
| `adapta-web` | `adp-web` | Adapta.org (Adapta One Web) | Web cookie | [link](https://agent.adapta.one) | Paste your __client cookie value from .clerk.agent.adapta.one (DevTools → Application → Cookies) |
|
||||
| `blackbox-web` | `bb-web` | Blackbox Web (Subscription) | Web cookie | [link](https://app.blackbox.ai) | Paste your __Secure-authjs.session-token value or full cookie header from app.blackbox.ai |
|
||||
| `chatgpt-web` | `cgpt-web` | ChatGPT Web (Plus/Pro) | Web cookie | [link](https://chatgpt.com) | Paste your __Secure-next-auth.session-token cookie value from chatgpt.com |
|
||||
| `claude-web` | `cw` | Claude Web | Web cookie | [link](https://claude.ai) | Paste your session cookie from claude.ai |
|
||||
| `copilot-web` | `copilot` | Microsoft Copilot Web | Web cookie | [link](https://copilot.microsoft.com) | Paste your access_token from copilot.microsoft.com (or export a .har file from DevTools while logged in) |
|
||||
| `deepseek-web` | `ds-web` | DeepSeek Web | Web cookie | [link](https://chat.deepseek.com) | Paste your userToken from chat.deepseek.com — DevTools → Application → Local Storage → userToken |
|
||||
| `doubao-web` | `db` | Doubao Web (ByteDance) | Web cookie | [link](https://www.doubao.com) | Paste your session cookie from doubao.com (DevTools → Application → Cookies) |
|
||||
| `gemini-business` | `gembiz` | Gemini Business (Enterprise) | Web cookie | [link](https://business.gemini.google) | From your enterprise account: open business.gemini.google/home/cid/{your-cid}, then copy __Secure-1PSID and __Secure-1PSIDTS cookies from DevTools → Application → Cookies. Paste as a cookie header below. |
|
||||
| `gemini-web` | `gweb` | Gemini Web (Free) | Web cookie | [link](https://gemini.google.com) | Paste your __Secure-1PSID cookie value from gemini.google.com. Optionally add __Secure-1PSIDTS separated by semicolon. |
|
||||
| `grok-web` | `gw` | Grok Web (Subscription) | Web cookie | [link](https://grok.com) | Paste the full grok.com cookie line from DevTools → Application → Cookies. Include both `sso` and `sso-rw` (e.g. `sso=...; sso-rw=...`) — Grok's anti-bot rejects `sso` on its own. |
|
||||
| `huggingchat` | `huggingchat` | HuggingChat (Free) | Web cookie | [link](https://huggingface.co/chat) | Paste your hf-chat cookie value from huggingface.co/chat (DevTools → Application → Cookies → hf-chat). Optional — works without auth for basic use. |
|
||||
| `inner-ai` | `in-ai` | Inner.ai (Subscription) | Web cookie | [link](https://app.innerai.com) | Paste your token cookie and email separated by a space: open DevTools → Application → Cookies → .innerai.com, copy the token value, then append a space and your Inner.ai login email. Example: eyJhbG... user@example.com |
|
||||
| `kimi-web` | `kimi-web` | Kimi Web (Moonshot AI) | Web cookie | [link](https://kimi.moonshot.cn) | Paste your session cookie from kimi.moonshot.cn (DevTools → Application → Cookies) |
|
||||
| `lmarena` | `lma` | LMArena (Free) | Web cookie | [link](https://lmarena.ai) | Paste the full Cookie header from lmarena.ai (DevTools → Network → request → Cookie). The session is now split across arena-auth-prod-v1.0, .1, … — copy the whole header. Optional — works with free tier for basic comparisons. |
|
||||
| `muse-spark-web` | `ms-web` | Muse Spark Web (Meta AI) | Web cookie | [link](https://www.meta.ai) | Paste your abra_sess value or full cookie header from meta.ai |
|
||||
| `perplexity-web` | `pplx-web` | Perplexity Web (Pro/Max) | Web cookie | [link](https://www.perplexity.ai) | Paste your __Secure-next-auth.session-token cookie value from perplexity.ai |
|
||||
| `phind` | `ph` | Phind (Free) | Web cookie | [link](https://www.phind.com) | ⚠️ **DEPRECATED.** Phind shut down its API (2026-01); the /api/chat endpoint no longer serves (sweep 2026-06-19). |
|
||||
| `poe-web` | `poe` | Poe Web (Subscription) | Web cookie | [link](https://poe.com) | Paste your p-b cookie value from poe.com (DevTools → Application → Cookies → p-b) |
|
||||
| `qwen-web` | `qwen-web` | Qwen Web (Free) | Web cookie | [link](https://chat.qwen.ai) | Open chat.qwen.ai, log in, then open DevTools → Application → Local Storage → copy the "token" value (or use tongyi_sso_ticket cookie as Bearer token). |
|
||||
| `t3-web` | `t3chat` | t3.chat (Pro/Free) | Web cookie | [link](https://t3.chat) | Open t3.chat in your browser, log in, then open DevTools → Application → Local Storage → https://t3.chat. Copy the value of 'convex-session-id'. Also open DevTools → Network, copy the Cookie header from any request. Paste both values here. See provider setup docs for a step-by-step guide. |
|
||||
| `v0-vercel-web` | `v0` | v0 Vercel Web (Code Gen) | Web cookie | [link](https://v0.dev) | Paste your session cookie from v0.dev (DevTools → Application → Cookies) |
|
||||
| `venice-web` | `ven` | Venice Web (Privacy) | Web cookie | [link](https://venice.ai) | Paste your session cookie from venice.ai (DevTools → Application → Cookies) |
|
||||
|
||||
## API Key Providers (paid / paid-with-free-credits) (153)
|
||||
## API Key Providers (paid / paid-with-free-credits) (157)
|
||||
|
||||
| ID | Alias | Name | Tags | Website | Notes |
|
||||
| --------------------- | -------------- | ------------------------------- | --------------------- | -------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `360ai` | `360ai` | 360 AI | API key | [link](https://ai.360.cn) | Get API key at ai.360.cn |
|
||||
| `agentrouter` | `agentrouter` | AgentRouter | API key, aggregator | [link](https://agentrouter.org) | $200 free credits on signup - multi-model routing gateway |
|
||||
| `ai21` | `ai21` | AI21 Labs | API key | [link](https://www.ai21.com) | $10 trial credits on signup (valid 3 months), no credit card required |
|
||||
| `aimlapi` | `aiml` | AI/ML API | API key, aggregator | [link](https://aimlapi.com) | $0.025/day free credits — 200+ models (GPT-4o, Claude, Gemini, Llama) via single endpoint |
|
||||
| `alibaba` | `ali` | Alibaba | API key | [link](https://dashscope-intl.aliyuncs.com) | — |
|
||||
| `alibaba-cn` | `ali-cn` | Alibaba (China) | API key | [link](https://dashscope.aliyuncs.com) | — |
|
||||
| `anthropic` | `anthropic` | Anthropic | API key | [link](https://platform.claude.com) | — |
|
||||
| `api-airforce` | `af` | Api.airforce | API key | [link](https://api.airforce) | 55 free tier models including Grok-3, Claude 3.7, Qwen3, Kimi-K2, Gemini 2.5 Flash, DeepSeek-V3 |
|
||||
| `arcee-ai` | `arcee` | Arcee AI | API key | [link](https://arcee.ai) | Get API key at arcee.ai |
|
||||
| `azure-ai` | `azure-ai` | Azure AI Foundry | API key, enterprise | [link](https://learn.microsoft.com/azure/ai-foundry) | Use your Azure AI Foundry key. Base URL can be https://<resource>.services.ai.azure.com/openai/v1/ or https://<resource>.openai.azure.com/openai/v1/. |
|
||||
| `azure-openai` | `azure` | Azure OpenAI | API key, enterprise | [link](https://azure.microsoft.com/products/ai-services/openai-service) | Use your Azure OpenAI API key. Base URL should be your resource endpoint, for example https://my-resource.openai.azure.com. |
|
||||
| `baichuan` | `baichuan` | Baichuan | API key | [link](https://baichuan.com) | Get API key at platform.baichuan-ai.com |
|
||||
| `baidu` | `baidu` | Baidu (ERNIE) | API key | [link](https://yiyan.baidu.com) | Get API key at console.bce.baidu.com |
|
||||
| `bailian-coding-plan` | `bcp` | Alibaba Coding Plan | API key | [link](https://www.alibabacloud.com/help/en/model-studio/coding-plan) | — |
|
||||
| `baseten` | `baseten` | Baseten | API key | [link](https://baseten.co) | $30 free trial credits for GPU inference |
|
||||
| `bazaarlink` | `bzl` | BazaarLink | API key | [link](https://bazaarlink.ai) | Free tier with auto:free routing — zero-cost inference, no credit card required |
|
||||
| `bedrock` | `bedrock` | Amazon Bedrock | API key, enterprise | [link](https://aws.amazon.com/bedrock) | Use your Amazon Bedrock API key and configure the AWS region where your models are enabled (for example eu-west-2). OmniRoute calls Bedrock's native Converse API directly. |
|
||||
| `black-forest-labs` | `bfl` | Black Forest Labs | API key, image | [link](https://blackforestlabs.ai) | — |
|
||||
| `blackbox` | `bb` | Blackbox AI | API key | [link](https://blackbox.ai) | Free tier: unlimited basic chat plus Minimax-M2.5, no credit card required |
|
||||
| `bluesminds` | `bm` | BluesMinds | API key | [link](https://www.bluesminds.com) | Free daily pi credits — supports 200+ models including GPT-4o, GPT-4.1, Claude Sonnet 4.5, Gemini 2.0 Flash, DeepSeek V4, Qwen, Kimi K2 |
|
||||
| `byteplus` | `bpm` | BytePlus ModelArk | API key | [link](https://console.byteplus.com/ark) | — |
|
||||
| `bytez` | `bytez` | Bytez | API key | [link](https://bytez.com) | $1 free credits, refreshes every 4 weeks |
|
||||
| `cablyai` | `cablyai` | CablyAI | API key, aggregator | [link](https://cablyai.com) | Bearer API key for the CablyAI OpenAI-compatible gateway. |
|
||||
| `cerebras` | `cerebras` | Cerebras | API key | [link](https://inference.cerebras.ai) | Free Trial: 1M tokens/day, 30K TPM, 5 RPM — no credit card. |
|
||||
| `chutes` | `chutes` | Chutes.ai | API key, aggregator | [link](https://chutes.ai) | Bearer API key for the Chutes OpenAI-compatible gateway. |
|
||||
| `clarifai` | `clarifai` | Clarifai | API key, enterprise | [link](https://docs.clarifai.com) | Use your Clarifai PAT or app-specific API key. OmniRoute targets the OpenAI-compatible endpoint at https://api.clarifai.com/v2/ext/openai/v1 and authenticates with Authorization: Key <token>. |
|
||||
| `cloudflare-ai` | `cf` | Cloudflare Workers AI | API key | [link](https://developers.cloudflare.com/workers-ai) | Requires API Token AND Account ID (found at dash.cloudflare.com) |
|
||||
| `codestral` | `codestral` | Codestral | API key | [link](https://mistral.ai) | — |
|
||||
| `cohere` | `cohere` | Cohere | API key | [link](https://cohere.com) | Free Trial: 1,000 API calls/month for testing, no credit card required |
|
||||
| `command-code` | `cmd` | Command Code | API key | [link](https://commandcode.ai/) | Use a Command Code API key. Requests are sent to Command Code's /alpha/generate endpoint. |
|
||||
| `coze` | `coze` | Coze | API key | [link](https://coze.com) | Get API key at coze.com/open/api |
|
||||
| `crof` | `crof` | CrofAI | API key | [link](https://crof.ai) | — |
|
||||
| `databricks` | `databricks` | Databricks | API key, enterprise | [link](https://www.databricks.com) | — |
|
||||
| `datarobot` | `datarobot` | DataRobot | API key, enterprise | [link](https://docs.datarobot.com) | Use your DataRobot API token. Optional Base URL can be the account root (for LLM Gateway) or a deployment URL under /api/v2/deployments/<id>. |
|
||||
| `deepinfra` | `deepinfra` | DeepInfra | API key | [link](https://deepinfra.com) | Free signup credits for API testing and model exploration |
|
||||
| `deepseek` | `ds` | DeepSeek | API key | [link](https://platform.deepseek.com) | 5M free tokens on signup - no credit card required |
|
||||
| `dify` | `dify` | Dify | API key | [link](https://dify.ai) | Get API key from your Dify instance. |
|
||||
| `doubao` | `doubao` | Doubao | API key | [link](https://doubao.com) | Get API key at console.volcengine.com |
|
||||
| `empower` | `empower` | Empower | API key, aggregator | [link](https://docs.empower.dev) | Bearer API key for the Empower OpenAI-compatible endpoint. |
|
||||
| `fal-ai` | `fal` | Fal.ai | API key, image | [link](https://fal.ai) | — |
|
||||
| `featherless-ai` | `featherless` | Featherless AI | API key | [link](https://featherless.ai) | Free tier available — no credit card required |
|
||||
| `fenayai` | `fenayai` | FenayAI | API key, aggregator | [link](https://fenayai.com) | Bearer API key for the FenayAI OpenAI-compatible gateway. |
|
||||
| `firecrawl` | `fc` | Firecrawl | API key | [link](https://firecrawl.dev) | — |
|
||||
| `fireworks` | `fireworks` | Fireworks AI | API key | [link](https://fireworks.ai) | $1 free starter credits on signup for API testing |
|
||||
| `freeaiapikey` | `faik` | FreeAIAPIKey | API key | [link](https://freeaiapikey.com) | — |
|
||||
| `freemodel-dev` | `fmd` | FreeModel.dev | API key | [link](https://freemodel.dev) | $300 free credits on signup — no credit card required. Access GPT-5.4 and GPT-5.5 (OpenAI's latest flagship models) through an OpenAI-compatible API. |
|
||||
| `friendliai` | `friendli` | FriendliAI | API key | [link](https://friendli.ai) | Free tier for serverless inference — no credit card required |
|
||||
| `galadriel` | `galadriel` | Galadriel | API key | [link](https://galadriel.com) | — |
|
||||
| `gemini` | `gemini` | Gemini (Google AI Studio) | API key | [link](https://aistudio.google.com) | Free forever: 1,500 req/day for Gemini 2.5 Flash — no credit card, get key at aistudio.google.com |
|
||||
| `getgoapi` | `ggo` | GoAPI | API key, aggregator | [link](https://api.getgoapi.com) | — |
|
||||
| `gigachat` | `gigachat` | GigaChat (Sber) | API key | [link](https://developers.sber.ru) | — |
|
||||
| `github-models` | `ghm` | GitHub Models | API key | [link](https://github.com/marketplace/models) | Create a GitHub PAT with 'models: read' scope at github.com/settings/tokens |
|
||||
| `gitlab` | `gitlab` | GitLab Duo PAT | API key | [link](https://docs.gitlab.com/user/duo_agent_platform/code_suggestions/) | GitLab personal access token for the public Code Suggestions API. Configure a self-hosted base URL when not using gitlab.com. |
|
||||
| `gitlawb` | `glb` | Gitlawb Opengateway (MiMo) | API key | [link](https://opengateway.gitlawb.com) | Free tier available — no credit card required |
|
||||
| `gitlawb-gmi` | `glb-gmi` | Gitlawb Opengateway (GMI Cloud) | API key | [link](https://opengateway.gitlawb.com) | Free tier available — no credit card required |
|
||||
| `glhf` | `glhf` | GLHF Chat | API key, aggregator | [link](https://glhf.chat) | Bearer API key for the GLHF OpenAI-compatible gateway. |
|
||||
| `glm` | `glm` | GLM Coding | API key | [link](https://z.ai/subscribe) | — |
|
||||
| `glm-cn` | `glmcn` | GLM Coding (China) | API key | [link](https://open.bigmodel.cn) | — |
|
||||
| `glmt` | `glmt` | GLM Thinking | API key | [link](https://open.bigmodel.cn) | — |
|
||||
| `groq` | `groq` | Groq | API key | [link](https://groq.com) | Free tier: 30 RPM / 14.4K RPD — no credit card |
|
||||
| `hackclub` | `hc` | Hackclub AI | API key, aggregator | [link](https://ai.hackclub.com) | Sign in with your Hack Club account at ai.hackclub.com. |
|
||||
| `haiper` | `hp` | Haiper | API key, video | [link](https://haiper.ai) | Get API key at haiper.ai/haiper-api |
|
||||
| `heroku` | `heroku` | Heroku AI | API key, enterprise | [link](https://www.heroku.com) | — |
|
||||
| `huggingchat` | `huggingchat` | HuggingChat | API key | [link](https://huggingface.co/chat) | No API key required for basic access. |
|
||||
| `huggingface` | `hf` | HuggingFace | API key | [link](https://huggingface.co) | Free Inference API for thousands of models (Whisper, VITS, SDXL…) |
|
||||
| `hyperbolic` | `hyp` | Hyperbolic | API key | [link](https://hyperbolic.xyz) | $1-5 trial credits on signup for serverless inference |
|
||||
| `ideogram` | `ideo` | Ideogram | API key | [link](https://ideogram.ai) | Get API key at ideogram.ai/docs/api |
|
||||
| `iflytek` | `iflytek` | iFlytek Spark | API key | [link](https://xinghuo.xfyun.cn) | Get API key at console.xfyun.cn |
|
||||
| `inclusionai` | `inclusion` | InclusionAI | API key | [link](https://inclusionai.com) | Get API key at inclusionai.com |
|
||||
| `inference-net` | `inet` | Inference.net | API key | [link](https://inference.net) | $25 free credits on signup plus research grants available |
|
||||
| `jina-ai` | `jina` | Jina AI | API key, embed/rerank | [link](https://jina.ai) | Bearer API key for the Jina AI rerank API. |
|
||||
| `jina-reader` | `jr` | Jina Reader | API key | [link](https://jina.ai/reader) | — |
|
||||
| `kie` | `kie` | KIE.AI | API key | [link](https://kie.ai) | — |
|
||||
| `kilo-gateway` | `kg` | Kilo Gateway | API key, aggregator | [link](https://kilo.ai) | — |
|
||||
| `kimi` | `kimi` | Kimi | API key | [link](https://platform.moonshot.ai) | — |
|
||||
| `kimi-coding-apikey` | `kmca` | Kimi Coding (API Key) | API key | [link](https://www.kimi.com/code) | — |
|
||||
| `kluster` | `kluster` | Kluster AI | API key | [link](https://kluster.ai) | $5 free credits on signup - DeepSeek R1, Llama 4 Maverick/Scout, Qwen3 235B |
|
||||
| `lambda-ai` | `lambda` | Lambda AI | API key | [link](https://lambda.ai) | — |
|
||||
| `laozhang` | `lz` | LaoZhang AI | API key, aggregator | [link](https://api.laozhang.ai) | — |
|
||||
| `leonardo` | `leo` | Leonardo AI | API key, video | [link](https://leonardo.ai) | Get API key at leonardo.ai/developer |
|
||||
| `liquid` | `liquid` | Liquid AI | API key | [link](https://liquid.ai) | Get API key at liquid.ai |
|
||||
| `llamagate` | `llamagate` | LlamaGate | API key | [link](https://llamagate.ai) | — |
|
||||
| `llm7` | `llm7` | LLM7.io | API key | [link](https://llm7.io) | No signup required - 2 req/s, 20 RPM, 100 req/hr free tier |
|
||||
| `longcat` | `lc` | LongCat AI | API key | [link](https://longcat.chat/platform/docs) | Free: 5M tokens/day on LongCat-2.0-Preview (Flash models retired 2026-05-29); up to 120M/day via feedback. |
|
||||
| `maritalk` | `maritalk` | Maritalk | API key | [link](https://www.maritaca.ai) | — |
|
||||
| `meta-llama` | `meta` | Meta Llama API | API key | [link](https://llama.developer.meta.com) | — |
|
||||
| `minimax` | `minimax` | Minimax Coding | API key, video | [link](https://www.minimax.io) | — |
|
||||
| `minimax-cn` | `minimax-cn` | Minimax (China) | API key | [link](https://www.minimaxi.com) | — |
|
||||
| `mistral` | `mistral` | Mistral | API key | [link](https://mistral.ai) | Free Experiment tier: rate-limited access to all models, no credit card required |
|
||||
| `modal` | `mdl` | Modal | API key, enterprise | [link](https://modal.com/docs) | Use the bearer token that protects your Modal deployment, if enabled. Base URL should point to your OpenAI-compatible Modal app, for example https://<workspace>--<app>.modal.run/v1. |
|
||||
| `monsterapi` | `monster` | MonsterAPI | API key | [link](https://monsterapi.ai) | Get API key at monsterapi.ai |
|
||||
| `moonshot` | `moonshot` | Moonshot AI | API key | [link](https://platform.moonshot.ai) | — |
|
||||
| `morph` | `morph` | Morph | API key | [link](https://morphllm.com) | Free tier: 250K credits/month, $0 |
|
||||
| `nanogpt` | `nanogpt` | NanoGPT | API key | [link](https://nano-gpt.com) | — |
|
||||
| `nebius` | `nebius` | Nebius AI | API key | [link](https://nebius.com) | ~$1 trial credits on signup for API testing |
|
||||
| `nlpcloud` | `nlpc` | NLP Cloud | API key | [link](https://docs.nlpcloud.com) | Use your NLP Cloud API key in Authorization: Token <key>. OmniRoute targets the chatbot endpoint on https://api.nlpcloud.io/v1/gpu/<model>/chatbot by default. |
|
||||
| `nomic` | `nomic` | Nomic | API key | [link](https://nomic.ai) | Get API key at atlas.nomic.ai |
|
||||
| `nous-research` | `nous` | Nous Research | API key | [link](https://portal.nousresearch.com/help) | Use your Nous Portal API key. OmniRoute targets the official OpenAI-compatible inference endpoint at https://inference-api.nousresearch.com/v1. |
|
||||
| `novita` | `novita` | Novita AI | API key, aggregator | [link](https://novita.ai) | $0.50 trial credits on signup (valid about 1 year) |
|
||||
| `nscale` | `nscale` | nScale | API key | [link](https://nscale.com) | $5 free credits on signup for inference testing |
|
||||
| `nvidia` | `nvidia` | NVIDIA NIM | API key | [link](https://build.nvidia.com) | Free dev access: ~40 RPM, 70+ models (Kimi K2.5, GLM 4.7, DeepSeek V3.2...) |
|
||||
| `oci` | `oci` | OCI Generative AI | API key, enterprise | [link](https://www.oracle.com/artificial-intelligence/generative-ai) | Use your OCI Generative AI API key or IAM bearer token. Base URL can be https://inference.generativeai.<region>.oci.oraclecloud.com/openai/v1/. |
|
||||
| `ollama-cloud` | `ollamacloud` | Ollama Cloud | API key | [link](https://ollama.com/settings/api-keys) | — |
|
||||
| `openai` | `openai` | OpenAI | API key | [link](https://platform.openai.com) | — |
|
||||
| `opencode-go` | `opencode-go` | OpenCode Go | API key | [link](https://opencode.ai/go) | — |
|
||||
| `opencode-zen` | `opencode-zen` | OpenCode Zen | API key | [link](https://opencode.ai/zen) | — |
|
||||
| `openrouter` | `openrouter` | OpenRouter | API key, aggregator | [link](https://openrouter.ai) | Free models at $0/token with :free suffix - 20 RPM / 200 RPD |
|
||||
| `orcarouter` | `orcarouter` | OrcaRouter | API key | [link](https://www.orcarouter.ai) | — |
|
||||
| `ovhcloud` | `ovh` | OVHcloud AI | API key | [link](https://www.ovhcloud.com) | — |
|
||||
| `perplexity` | `pplx` | Perplexity | API key | [link](https://www.perplexity.ai) | — |
|
||||
| `phind` | `phind` | Phind | API key | [link](https://phind.com) | Get API key at phind.com |
|
||||
| `piapi` | `pi` | PiAPI | API key, aggregator | [link](https://piapi.ai) | — |
|
||||
| `poe` | `poe` | Poe | API key, aggregator | [link](https://creator.poe.com/api-reference) | Bearer API key for the Poe OpenAI-compatible API. |
|
||||
| `pollinations` | `pol` | Pollinations AI | API key, video | [link](https://pollinations.ai) | No API key required for free public endpoint. Optional Spore tier: ~0.01 pollen/hour. |
|
||||
| `predibase` | `predibase` | Predibase | API key | [link](https://predibase.com) | $25 free trial credits (30-day validity) |
|
||||
| `publicai` | `publicai` | PublicAI | API key | [link](https://publicai.co) | Requires an API key — one-time signup credit, then paid |
|
||||
| `puter` | `pu` | Puter AI | API key | [link](https://puter.com) | Get token at puter.com/dashboard → Copy Auth Token |
|
||||
| `qianfan` | `qianfan` | Baidu Qianfan | API key | [link](https://cloud.baidu.com/product/wenxinworkshop) | — |
|
||||
| `recraft` | `recraft` | Recraft | API key, image | [link](https://recraft.ai) | — |
|
||||
| `reka` | `reka` | Reka | API key | [link](https://docs.reka.ai/chat/overview) | Use your Reka API key. OmniRoute supports the OpenAI-compatible base URL https://api.reka.ai/v1 and sends both Authorization and X-Api-Key headers for compatibility. |
|
||||
| `runwayml` | `runway` | Runway | API key, video | [link](https://docs.dev.runwayml.com) | Use your Runway API key in Authorization: Bearer <key>. OmniRoute targets the current Runway API at https://api.dev.runwayml.com/v1 and sends the required X-Runway-Version header automatically. |
|
||||
| `sambanova` | `samba` | SambaNova | API key | [link](https://sambanova.ai) | $5 free credits on signup (30-day validity), no credit card required |
|
||||
| `sap` | `sap` | SAP Generative AI Hub | API key, enterprise | [link](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/generative-ai-hub-in-sap-ai-core) | Use your SAP AI Core bearer token. Base URL can be your AI_API_URL root or a deploymentUrl from Generative AI Hub. |
|
||||
| `scaleway` | `scw` | Scaleway AI | API key | [link](https://www.scaleway.com/en/ai/generative-apis) | 1M free tokens for new accounts — EU/GDPR compliant (Paris), Qwen3 235B & Llama 70B |
|
||||
| `sensenova` | `sensenova` | SenseNova | API key | [link](https://platform.sensenova.cn) | Get API key at platform.sensenova.cn |
|
||||
| `siliconflow` | `siliconflow` | SiliconFlow | API key | [link](https://cloud.siliconflow.com) | $1 free credits plus permanently free models after identity verification |
|
||||
| `snowflake` | `snowflake` | Snowflake Cortex | API key, enterprise | [link](https://www.snowflake.com) | — |
|
||||
| `sparkdesk` | `sparkdesk` | SparkDesk | API key | [link](https://xinghuo.xfyun.cn) | Get API key at console.xfyun.cn |
|
||||
| `stability-ai` | `stability` | Stability AI | API key, image | [link](https://stability.ai) | — |
|
||||
| `stepfun` | `stepfun` | StepFun | API key | [link](https://stepfun.com) | Get API key at platform.stepfun.com |
|
||||
| `suno` | `suno` | Suno | API key | [link](https://suno.ai) | Paste session cookie from suno.ai (Clerk auth) |
|
||||
| `synthetic` | `synthetic` | Synthetic | API key, aggregator | [link](https://synthetic.new) | — |
|
||||
| `tencent` | `tencent` | Tencent Hunyuan | API key | [link](https://hunyuan.tencent.com) | Get API key at console.cloud.tencent.com |
|
||||
| `thebai` | `thebai` | TheB.AI | API key, aggregator | [link](https://theb.ai) | Bearer API key for the TheB.AI OpenAI-compatible gateway. |
|
||||
| `together` | `together` | Together AI | API key, video | [link](https://www.together.ai) | $25 signup credits + 3 permanently free models: Llama 3.3 70B, Vision, DeepSeek-R1 distill |
|
||||
| `topaz` | `topaz` | Topaz | API key, image | [link](https://topazlabs.com) | — |
|
||||
| `udio` | `udio` | Udio | API key | [link](https://udio.com) | Paste session cookie from udio.com (Supabase auth) |
|
||||
| `uncloseai` | `unc` | UncloseAI | API key | [link](https://uncloseai.com) | No auth required. API accepts any non-empty string as key for identification. |
|
||||
| `upstage` | `upstage` | Upstage | API key | [link](https://www.upstage.ai) | — |
|
||||
| `v0-vercel` | `v0` | v0 (Vercel) | API key | [link](https://v0.dev) | — |
|
||||
| `venice` | `venice` | Venice.ai | API key | [link](https://venice.ai) | — |
|
||||
| `vercel-ai-gateway` | `vag` | Vercel AI Gateway | API key, aggregator | [link](https://vercel.com/docs/ai-gateway) | — |
|
||||
| `vertex` | `vertex` | Vertex AI | API key, enterprise | [link](https://cloud.google.com/vertex-ai) | Provide Service Account JSON or OAuth access_token |
|
||||
| `vertex-partner` | `vp` | Vertex AI Partners | API key, enterprise | [link](https://cloud.google.com/vertex-ai) | Provide the same Service Account JSON used for Vertex AI partner models. |
|
||||
| `volcengine` | `volcengine` | Volcengine | API key | [link](https://www.volcengine.com) | — |
|
||||
| `voyage-ai` | `voyage` | Voyage AI | API key, embed/rerank | [link](https://www.voyageai.com) | Bearer API key for Voyage AI embeddings and rerank APIs. |
|
||||
| `wandb` | `wandb` | Weights & Biases Inference | API key | [link](https://wandb.ai) | — |
|
||||
| `watsonx` | `watsonx` | IBM watsonx.ai Gateway | API key, enterprise | [link](https://www.ibm.com/products/watsonx-ai) | Use your watsonx bearer token. Base URL can be https://<region>.ml.cloud.ibm.com/ml/gateway/v1/ or a self-managed /ml/gateway/v1 endpoint. |
|
||||
| `xai` | `xai` | xAI (Grok) | API key | [link](https://x.ai) | — |
|
||||
| `xiaomi-mimo` | `mimo` | Xiaomi MiMo | API key | [link](https://mimo.mi.com) | — |
|
||||
| `yi` | `yi` | Yi (01.AI) | API key | [link](https://01.ai) | Get API key at platform.lingyiwanwu.com |
|
||||
| `zai` | `zai` | Z.AI | API key | [link](https://open.bigmodel.cn) | — |
|
||||
| `zenmux` | `zm` | ZenMux | API key | [link](https://zenmux.ai) | Use your ZenMux API key in Authorization: Bearer <key>. ZenMux is fully OpenAI-compatible. Base URL: https://zenmux.ai/api/v1. |
|
||||
| ID | Alias | Name | Tags | Website | Notes |
|
||||
|----|-------|------|------|---------|-------|
|
||||
| `360ai` | `360ai` | 360 AI | API key | [link](https://ai.360.cn) | Get API key at ai.360.cn |
|
||||
| `agentrouter` | `agentrouter` | AgentRouter | API key, aggregator | [link](https://agentrouter.org) | $200 free credits on signup - multi-model routing gateway |
|
||||
| `ai21` | `ai21` | AI21 Labs | API key | [link](https://www.ai21.com) | $10 trial credits on signup (valid 3 months), no credit card required |
|
||||
| `aimlapi` | `aiml` | AI/ML API | API key, aggregator | [link](https://aimlapi.com) | Free tier paused (2026) — AI/ML API is now pay-as-you-go only (min $20 top-up); no recurring free credits. |
|
||||
| `alibaba` | `ali` | Alibaba | API key | [link](https://dashscope-intl.aliyuncs.com) | — |
|
||||
| `alibaba-cn` | `ali-cn` | Alibaba (China) | API key | [link](https://dashscope.aliyuncs.com) | — |
|
||||
| `anthropic` | `anthropic` | Anthropic | API key | [link](https://platform.claude.com) | — |
|
||||
| `api-airforce` | `af` | Api.airforce | API key | [link](https://api.airforce) | 55 free tier models including Grok-3, Claude 3.7, Qwen3, Kimi-K2, Gemini 2.5 Flash, DeepSeek-V3 |
|
||||
| `arcee-ai` | `arcee` | Arcee AI | API key | [link](https://arcee.ai) | Get API key at arcee.ai |
|
||||
| `azure-ai` | `azure-ai` | Azure AI Foundry | API key, enterprise | [link](https://learn.microsoft.com/azure/ai-foundry) | Use your Azure AI Foundry key. Base URL can be https://<resource>.services.ai.azure.com/openai/v1/ or https://<resource>.openai.azure.com/openai/v1/. |
|
||||
| `azure-openai` | `azure` | Azure OpenAI | API key, enterprise | [link](https://azure.microsoft.com/products/ai-services/openai-service) | Use your Azure OpenAI API key. Base URL should be your resource endpoint, for example https://my-resource.openai.azure.com. |
|
||||
| `baichuan` | `baichuan` | Baichuan | API key | [link](https://baichuan.com) | Get API key at platform.baichuan-ai.com |
|
||||
| `baidu` | `baidu` | Baidu (ERNIE) | API key | [link](https://yiyan.baidu.com) | Get API key at console.bce.baidu.com |
|
||||
| `bailian-coding-plan` | `bcp` | Alibaba Coding Plan | API key | [link](https://www.alibabacloud.com/help/en/model-studio/coding-plan) | — |
|
||||
| `baseten` | `baseten` | Baseten | API key | [link](https://baseten.co) | $30 free trial credits for GPU inference |
|
||||
| `bazaarlink` | `bzl` | BazaarLink | API key | [link](https://bazaarlink.ai) | Free tier with auto:free routing — zero-cost inference, no credit card required |
|
||||
| `bedrock` | `bedrock` | Amazon Bedrock | API key, enterprise | [link](https://aws.amazon.com/bedrock) | Use your Amazon Bedrock API key and configure the AWS region where your models are enabled (for example eu-west-2). OmniRoute calls Bedrock's native Converse API directly. |
|
||||
| `black-forest-labs` | `bfl` | Black Forest Labs | API key, image | [link](https://blackforestlabs.ai) | — |
|
||||
| `blackbox` | `bb` | Blackbox AI | API key | [link](https://blackbox.ai) | Free tier: unlimited basic chat plus Minimax-M2.5, no credit card required |
|
||||
| `bluesminds` | `bm` | BluesMinds | API key | [link](https://www.bluesminds.com) | Free daily pi credits — supports 200+ models including GPT-4o, GPT-4.1, Claude Sonnet 4.5, Gemini 2.0 Flash, DeepSeek V4, Qwen, Kimi K2 |
|
||||
| `byteplus` | `bpm` | BytePlus ModelArk | API key | [link](https://console.byteplus.com/ark) | — |
|
||||
| `bytez` | `bytez` | Bytez | API key | [link](https://bytez.com) | $1 free credits, refreshes every 4 weeks |
|
||||
| `cablyai` | `cablyai` | CablyAI | API key, aggregator | [link](https://cablyai.com) | Bearer API key for the CablyAI OpenAI-compatible gateway. |
|
||||
| `cerebras` | `cerebras` | Cerebras | API key | [link](https://inference.cerebras.ai) | Free Trial: 1M tokens/day, 30K TPM, 5 RPM — no credit card. |
|
||||
| `chutes` | `chutes` | Chutes.ai | API key, aggregator | [link](https://chutes.ai) | Bearer API key for the Chutes OpenAI-compatible gateway. |
|
||||
| `clarifai` | `clarifai` | Clarifai | API key, enterprise | [link](https://docs.clarifai.com) | Use your Clarifai PAT or app-specific API key. OmniRoute targets the OpenAI-compatible endpoint at https://api.clarifai.com/v2/ext/openai/v1 and authenticates with Authorization: Key <token>. |
|
||||
| `cloudflare-ai` | `cf` | Cloudflare Workers AI | API key | [link](https://developers.cloudflare.com/workers-ai) | Requires API Token AND Account ID (found at dash.cloudflare.com) |
|
||||
| `codestral` | `codestral` | Codestral | API key | [link](https://mistral.ai) | — |
|
||||
| `cohere` | `cohere` | Cohere | API key | [link](https://cohere.com) | Free Trial: 1,000 API calls/month for testing, no credit card required |
|
||||
| `command-code` | `cmd` | Command Code | API key | [link](https://commandcode.ai/) | Use a Command Code API key. Requests are sent to Command Code's /alpha/generate endpoint. |
|
||||
| `coze` | `coze` | Coze | API key | [link](https://coze.com) | Get API key at coze.com/open/api |
|
||||
| `crof` | `crof` | CrofAI | API key | [link](https://crof.ai) | — |
|
||||
| `databricks` | `databricks` | Databricks | API key, enterprise | [link](https://www.databricks.com) | — |
|
||||
| `datarobot` | `datarobot` | DataRobot | API key, enterprise | [link](https://docs.datarobot.com) | Use your DataRobot API token. Optional Base URL can be the account root (for LLM Gateway) or a deployment URL under /api/v2/deployments/<id>. |
|
||||
| `deepinfra` | `deepinfra` | DeepInfra | API key | [link](https://deepinfra.com) | Free signup credits for API testing and model exploration |
|
||||
| `deepseek` | `ds` | DeepSeek | API key | [link](https://platform.deepseek.com) | 5M free tokens on signup - no credit card required |
|
||||
| `dify` | `dify` | Dify | API key | [link](https://dify.ai) | Get API key from your Dify instance. |
|
||||
| `dit` | `dai` | DIT.ai | API key | [link](https://dit.ai) | Use your dit.ai API key in Authorization: Bearer <key>. Fully OpenAI-compatible — a drop-in replacement, just change the base URL to https://api.dit.ai/v1. |
|
||||
| `doubao` | `doubao` | Doubao | API key | [link](https://doubao.com) | Get API key at console.volcengine.com |
|
||||
| `empower` | `empower` | Empower | API key, aggregator | [link](https://docs.empower.dev) | Bearer API key for the Empower OpenAI-compatible endpoint. |
|
||||
| `fal-ai` | `fal` | Fal.ai | API key, image | [link](https://fal.ai) | — |
|
||||
| `featherless-ai` | `featherless` | Featherless AI | API key | [link](https://featherless.ai) | Free tier available — no credit card required |
|
||||
| `fenayai` | `fenayai` | FenayAI | API key, aggregator | [link](https://fenayai.com) | Bearer API key for the FenayAI OpenAI-compatible gateway. |
|
||||
| `firecrawl` | `fc` | Firecrawl | API key | [link](https://firecrawl.dev) | — |
|
||||
| `fireworks` | `fireworks` | Fireworks AI | API key | [link](https://fireworks.ai) | $1 free starter credits on signup for API testing |
|
||||
| `freeaiapikey` | `faik` | FreeAIAPIKey | API key | [link](https://freeaiapikey.com) | — |
|
||||
| `freemodel-dev` | `fmd` | FreeModel.dev | API key | [link](https://freemodel.dev) | $300 free credits on signup — no credit card required. Access GPT-5.4 and GPT-5.5 (OpenAI's latest flagship models) through an OpenAI-compatible API. |
|
||||
| `friendliai` | `friendli` | FriendliAI | API key | [link](https://friendli.ai) | Free tier for serverless inference — no credit card required |
|
||||
| `galadriel` | `galadriel` | Galadriel | API key | [link](https://galadriel.com) | ⚠️ **DEPRECATED.** api.galadriel.ai no longer resolves (sweep 2026-06-19); the inference API appears discontinued. |
|
||||
| `gemini` | `gemini` | Gemini (Google AI Studio) | API key | [link](https://aistudio.google.com) | Free forever: 1,500 req/day for Gemini 2.5 Flash — no credit card, get key at aistudio.google.com |
|
||||
| `getgoapi` | `ggo` | GoAPI | API key, aggregator | [link](https://api.getgoapi.com) | — |
|
||||
| `gigachat` | `gigachat` | GigaChat (Sber) | API key | [link](https://developers.sber.ru) | — |
|
||||
| `github-models` | `ghm` | GitHub Models | API key | [link](https://github.com/marketplace/models) | Create a GitHub PAT with 'models: read' scope at github.com/settings/tokens |
|
||||
| `gitlab` | `gitlab` | GitLab Duo PAT | API key | [link](https://docs.gitlab.com/user/duo_agent_platform/code_suggestions/) | GitLab personal access token for the public Code Suggestions API. Configure a self-hosted base URL when not using gitlab.com. |
|
||||
| `gitlawb` | `glb` | Gitlawb Opengateway (MiMo) | API key | [link](https://opengateway.gitlawb.com) | Free MiMo (xiaomi/mimo-v2.5) revoked 2026-05 — Opengateway is now a pay-as-you-go credit gateway; no recurring free model. |
|
||||
| `gitlawb-gmi` | `glb-gmi` | Gitlawb Opengateway (GMI Cloud) | API key | [link](https://opengateway.gitlawb.com) | Free Nemotron promo ended 2026-06 — the GMI Cloud route is now pay-as-you-go credit only. |
|
||||
| `glhf` | `glhf` | GLHF Chat | API key, aggregator | [link](https://glhf.chat) | ⚠️ **DEPRECATED.** glhf.chat shut down (2026); its api.laf.run gateway no longer serves the catalog (sweep 2026-06-19). |
|
||||
| `glm` | `glm` | GLM Coding | API key | [link](https://z.ai/subscribe) | — |
|
||||
| `glm-cn` | `glmcn` | GLM Coding (China) | API key | [link](https://open.bigmodel.cn) | — |
|
||||
| `glmt` | `glmt` | GLM Thinking | API key | [link](https://open.bigmodel.cn) | — |
|
||||
| `groq` | `groq` | Groq | API key | [link](https://groq.com) | Free tier: 30 RPM / 14.4K RPD — no credit card |
|
||||
| `hackclub` | `hc` | Hackclub AI | API key, aggregator | [link](https://ai.hackclub.com) | Sign in with your Hack Club account at ai.hackclub.com. |
|
||||
| `haiper` | `hp` | Haiper | API key, video | [link](https://haiper.ai) | Get API key at haiper.ai/haiper-api |
|
||||
| `heroku` | `heroku` | Heroku AI | API key, enterprise | [link](https://www.heroku.com) | — |
|
||||
| `huggingchat` | `huggingchat` | HuggingChat | API key | [link](https://huggingface.co/chat) | No API key required for basic access. |
|
||||
| `huggingface` | `hf` | HuggingFace | API key | [link](https://huggingface.co) | Free Inference API for thousands of models (Whisper, VITS, SDXL…) |
|
||||
| `hyperbolic` | `hyp` | Hyperbolic | API key | [link](https://hyperbolic.xyz) | $1-5 trial credits on signup for serverless inference |
|
||||
| `ideogram` | `ideo` | Ideogram | API key | [link](https://ideogram.ai) | Get API key at ideogram.ai/docs/api |
|
||||
| `iflytek` | `iflytek` | iFlytek Spark | API key | [link](https://xinghuo.xfyun.cn) | Get API key at console.xfyun.cn |
|
||||
| `inclusionai` | `inclusion` | InclusionAI | API key | [link](https://inclusionai.com) | ⚠️ **DEPRECATED.** api.inclusionai.tech no longer resolves (sweep 2026-06-19); the inference API appears discontinued. |
|
||||
| `inference-net` | `inet` | Inference.net | API key | [link](https://inference.net) | $25 free credits on signup plus research grants available |
|
||||
| `jina-ai` | `jina` | Jina AI | API key, embed/rerank | [link](https://jina.ai) | Bearer API key for the Jina AI rerank API. |
|
||||
| `jina-reader` | `jr` | Jina Reader | API key | [link](https://jina.ai/reader) | — |
|
||||
| `kie` | `kie` | KIE.AI | API key | [link](https://kie.ai) | — |
|
||||
| `kilo-gateway` | `kg` | Kilo Gateway | API key, aggregator | [link](https://kilo.ai) | — |
|
||||
| `kimi` | `kimi` | Kimi | API key | [link](https://platform.moonshot.ai) | — |
|
||||
| `kimi-coding-apikey` | `kmca` | Kimi Coding (API Key) | API key | [link](https://www.kimi.com/code) | — |
|
||||
| `kluster` | `kluster` | Kluster AI | API key | [link](https://kluster.ai) | ⚠️ **DEPRECATED.** kluster.ai shut down (2026-06-09); api.kluster.ai no longer resolves (sweep 2026-06-19). Use another OpenAI-compatible provider. |
|
||||
| `lambda-ai` | `lambda` | Lambda AI | API key | [link](https://lambda.ai) | — |
|
||||
| `laozhang` | `lz` | LaoZhang AI | API key, aggregator | [link](https://api.laozhang.ai) | — |
|
||||
| `leonardo` | `leo` | Leonardo AI | API key, video | [link](https://leonardo.ai) | Get API key at leonardo.ai/developer |
|
||||
| `liquid` | `liquid` | Liquid AI | API key | [link](https://liquid.ai) | Get API key at liquid.ai |
|
||||
| `llamagate` | `llamagate` | LlamaGate | API key | [link](https://llamagate.ai) | — |
|
||||
| `llm7` | `llm7` | LLM7.io | API key | [link](https://llm7.io) | No signup required - 2 req/s, 20 RPM, 100 req/hr free tier |
|
||||
| `longcat` | `lc` | LongCat AI | API key | [link](https://longcat.chat/platform/docs) | Free: 5M tokens/day on LongCat-2.0-Preview (Flash models retired 2026-05-29); up to 120M/day via feedback. |
|
||||
| `maritalk` | `maritalk` | Maritalk | API key | [link](https://www.maritaca.ai) | — |
|
||||
| `meta-llama` | `meta` | Meta Llama API | API key | [link](https://llama.developer.meta.com) | — |
|
||||
| `minimax` | `minimax` | Minimax Coding | API key, video | [link](https://www.minimax.io) | — |
|
||||
| `minimax-cn` | `minimax-cn` | Minimax (China) | API key | [link](https://www.minimaxi.com) | — |
|
||||
| `mistral` | `mistral` | Mistral | API key | [link](https://mistral.ai) | Free Experiment tier: rate-limited access to all models, no credit card required |
|
||||
| `modal` | `mdl` | Modal | API key, enterprise | [link](https://modal.com/docs) | Use the bearer token that protects your Modal deployment, if enabled. Base URL should point to your OpenAI-compatible Modal app, for example https://<workspace>--<app>.modal.run/v1. |
|
||||
| `monsterapi` | `monster` | MonsterAPI | API key | [link](https://monsterapi.ai) | Get API key at monsterapi.ai |
|
||||
| `moonshot` | `moonshot` | Moonshot AI | API key | [link](https://platform.moonshot.ai) | — |
|
||||
| `morph` | `morph` | Morph | API key | [link](https://morphllm.com) | Free tier: 250K credits/month, $0 |
|
||||
| `nanogpt` | `nanogpt` | NanoGPT | API key | [link](https://nano-gpt.com) | — |
|
||||
| `nebius` | `nebius` | Nebius AI | API key | [link](https://nebius.com) | ~$1 trial credits on signup for API testing |
|
||||
| `nlpcloud` | `nlpc` | NLP Cloud | API key | [link](https://docs.nlpcloud.com) | Use your NLP Cloud API key in Authorization: Token <key>. OmniRoute targets the chatbot endpoint on https://api.nlpcloud.io/v1/gpu/<model>/chatbot by default. |
|
||||
| `nomic` | `nomic` | Nomic | API key | [link](https://nomic.ai) | Get API key at atlas.nomic.ai |
|
||||
| `nous-research` | `nous` | Nous Research | API key | [link](https://portal.nousresearch.com/help) | Use your Nous Portal API key. OmniRoute targets the official OpenAI-compatible inference endpoint at https://inference-api.nousresearch.com/v1. |
|
||||
| `novita` | `novita` | Novita AI | API key, aggregator | [link](https://novita.ai) | $0.50 trial credits on signup (valid about 1 year) |
|
||||
| `nscale` | `nscale` | nScale | API key | [link](https://nscale.com) | $5 free credits on signup for inference testing |
|
||||
| `nvidia` | `nvidia` | NVIDIA NIM | API key | [link](https://build.nvidia.com) | Free dev access: ~40 RPM, 70+ models (Kimi K2.5, GLM 4.7, DeepSeek V3.2...) |
|
||||
| `oci` | `oci` | OCI Generative AI | API key, enterprise | [link](https://www.oracle.com/artificial-intelligence/generative-ai) | Use your OCI Generative AI API key or IAM bearer token. Base URL can be https://inference.generativeai.<region>.oci.oraclecloud.com/openai/v1/. |
|
||||
| `ollama-cloud` | `ollamacloud` | Ollama Cloud | API key | [link](https://ollama.com/settings/api-keys) | — |
|
||||
| `openadapter` | `oad` | OpenAdapter | API key | [link](https://openadapter.dev) | Use your OpenAdapter API key in Authorization: Bearer sk-cv-<key>. Fully OpenAI-compatible. API base URL: https://api.openadapter.in/v1. |
|
||||
| `openai` | `openai` | OpenAI | API key | [link](https://platform.openai.com) | — |
|
||||
| `opencode-go` | `opencode-go` | OpenCode Go | API key | [link](https://opencode.ai/go) | — |
|
||||
| `opencode-zen` | `opencode-zen` | OpenCode Zen | API key | [link](https://opencode.ai/zen) | — |
|
||||
| `openrouter` | `openrouter` | OpenRouter | API key, aggregator | [link](https://openrouter.ai) | Free models at $0/token with :free suffix - 20 RPM / 200 RPD |
|
||||
| `orcarouter` | `orcarouter` | OrcaRouter | API key | [link](https://www.orcarouter.ai) | — |
|
||||
| `ovhcloud` | `ovh` | OVHcloud AI | API key | [link](https://www.ovhcloud.com) | — |
|
||||
| `perplexity` | `pplx` | Perplexity | API key | [link](https://www.perplexity.ai) | — |
|
||||
| `phind` | `phind` | Phind | API key | [link](https://phind.com) | Get API key at phind.com |
|
||||
| `piapi` | `pi` | PiAPI | API key, aggregator | [link](https://piapi.ai) | — |
|
||||
| `poe` | `poe` | Poe | API key, aggregator | [link](https://creator.poe.com/api-reference) | Bearer API key for the Poe OpenAI-compatible API. |
|
||||
| `pollinations` | `pol` | Pollinations AI | API key, video | [link](https://pollinations.ai) | Free keyless tier: openai, openai-fast, openai-large, qwen-coder, mistral, deepseek, grok, gemini-flash-lite-3.1, perplexity-fast, perplexity-reasoning. Premium models (claude, gemini, midijourney) require a Pollinations API key from enter.pollinations.ai. |
|
||||
| `predibase` | `predibase` | Predibase | API key | [link](https://predibase.com) | ⚠️ **DEPRECATED.** serving.app.predibase.com no longer resolves (sweep 2026-06-19); the managed serving API appears discontinued. |
|
||||
| `publicai` | `publicai` | PublicAI | API key | [link](https://publicai.co) | Requires an API key — one-time signup credit, then paid |
|
||||
| `puter` | `pu` | Puter AI | API key | [link](https://puter.com) | Get token at puter.com/dashboard → Copy Auth Token |
|
||||
| `qianfan` | `qianfan` | Baidu Qianfan | API key | [link](https://cloud.baidu.com/product/wenxinworkshop) | — |
|
||||
| `recraft` | `recraft` | Recraft | API key, image | [link](https://recraft.ai) | — |
|
||||
| `reka` | `reka` | Reka | API key | [link](https://docs.reka.ai/chat/overview) | Use your Reka API key. OmniRoute supports the OpenAI-compatible base URL https://api.reka.ai/v1 and sends both Authorization and X-Api-Key headers for compatibility. |
|
||||
| `runwayml` | `runway` | Runway | API key, video | [link](https://docs.dev.runwayml.com) | Use your Runway API key in Authorization: Bearer <key>. OmniRoute targets the current Runway API at https://api.dev.runwayml.com/v1 and sends the required X-Runway-Version header automatically. |
|
||||
| `sambanova` | `samba` | SambaNova | API key | [link](https://sambanova.ai) | $5 free credits on signup (30-day validity), no credit card required |
|
||||
| `sap` | `sap` | SAP Generative AI Hub | API key, enterprise | [link](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/generative-ai-hub-in-sap-ai-core) | Use your SAP AI Core bearer token. Base URL can be your AI_API_URL root or a deploymentUrl from Generative AI Hub. |
|
||||
| `scaleway` | `scw` | Scaleway AI | API key | [link](https://www.scaleway.com/en/ai/generative-apis) | 1M free tokens for new accounts — EU/GDPR compliant (Paris), Qwen3 235B & Llama 70B |
|
||||
| `sensenova` | `sensenova` | SenseNova | API key | [link](https://platform.sensenova.cn) | Get API key at platform.sensenova.cn |
|
||||
| `siliconflow` | `siliconflow` | SiliconFlow | API key | [link](https://cloud.siliconflow.com) | $1 free credits plus permanently free models after identity verification |
|
||||
| `snowflake` | `snowflake` | Snowflake Cortex | API key, enterprise | [link](https://www.snowflake.com) | — |
|
||||
| `sparkdesk` | `sparkdesk` | SparkDesk | API key | [link](https://xinghuo.xfyun.cn) | Get API key at console.xfyun.cn |
|
||||
| `stability-ai` | `stability` | Stability AI | API key, image | [link](https://stability.ai) | — |
|
||||
| `stepfun` | `stepfun` | StepFun | API key | [link](https://stepfun.com) | Get API key at platform.stepfun.com |
|
||||
| `suno` | `suno` | Suno | API key | [link](https://suno.ai) | Paste session cookie from suno.ai (Clerk auth) |
|
||||
| `synthetic` | `synthetic` | Synthetic | API key, aggregator | [link](https://synthetic.new) | — |
|
||||
| `tencent` | `tencent` | Tencent Hunyuan | API key | [link](https://hunyuan.tencent.com) | Get API key at console.cloud.tencent.com |
|
||||
| `thebai` | `thebai` | TheB.AI | API key, aggregator | [link](https://theb.ai) | Bearer API key for the TheB.AI OpenAI-compatible gateway. |
|
||||
| `together` | `together` | Together AI | API key, video | [link](https://www.together.ai) | $25 signup credits + 3 permanently free models: Llama 3.3 70B, Vision, DeepSeek-R1 distill |
|
||||
| `tokenrouter` | `trk` | TokenRouter | API key | [link](https://tokenrouter.com) | Use your TokenRouter API key in Authorization: Bearer <key>. Fully OpenAI-compatible. API base URL: https://api.tokenrouter.com/v1. |
|
||||
| `topaz` | `topaz` | Topaz | API key, image | [link](https://topazlabs.com) | — |
|
||||
| `udio` | `udio` | Udio | API key | [link](https://udio.com) | Paste session cookie from udio.com (Supabase auth) |
|
||||
| `uncloseai` | `unc` | UncloseAI | API key | [link](https://uncloseai.com) | No auth required. API accepts any non-empty string as key for identification. |
|
||||
| `upstage` | `upstage` | Upstage | API key | [link](https://www.upstage.ai) | — |
|
||||
| `v0-vercel` | `v0` | v0 (Vercel) | API key | [link](https://v0.dev) | — |
|
||||
| `venice` | `venice` | Venice.ai | API key | [link](https://venice.ai) | — |
|
||||
| `vercel-ai-gateway` | `vag` | Vercel AI Gateway | API key, aggregator | [link](https://vercel.com/docs/ai-gateway) | — |
|
||||
| `vertex` | `vertex` | Vertex AI | API key, enterprise | [link](https://cloud.google.com/vertex-ai) | Provide Service Account JSON or OAuth access_token |
|
||||
| `vertex-partner` | `vp` | Vertex AI Partners | API key, enterprise | [link](https://cloud.google.com/vertex-ai) | Provide the same Service Account JSON used for Vertex AI partner models. |
|
||||
| `volcengine` | `volcengine` | Volcengine | API key | [link](https://www.volcengine.com) | — |
|
||||
| `voyage-ai` | `voyage` | Voyage AI | API key, embed/rerank | [link](https://www.voyageai.com) | Bearer API key for Voyage AI embeddings and rerank APIs. |
|
||||
| `wafer` | `wafer` | Wafer AI | API key | [link](https://wafer.ai) | — |
|
||||
| `wandb` | `wandb` | Weights & Biases Inference | API key | [link](https://wandb.ai) | — |
|
||||
| `watsonx` | `watsonx` | IBM watsonx.ai Gateway | API key, enterprise | [link](https://www.ibm.com/products/watsonx-ai) | Use your watsonx bearer token. Base URL can be https://<region>.ml.cloud.ibm.com/ml/gateway/v1/ or a self-managed /ml/gateway/v1 endpoint. |
|
||||
| `xai` | `xai` | xAI (Grok) | API key | [link](https://x.ai) | — |
|
||||
| `xiaomi-mimo` | `mimo` | Xiaomi MiMo | API key | [link](https://mimo.mi.com) | — |
|
||||
| `yi` | `yi` | Yi (01.AI) | API key | [link](https://01.ai) | Get API key at platform.lingyiwanwu.com |
|
||||
| `zai` | `zai` | Z.AI | API key | [link](https://open.bigmodel.cn) | — |
|
||||
| `zenmux` | `zm` | ZenMux | API key | [link](https://zenmux.ai) | Use your ZenMux API key in Authorization: Bearer <key>. ZenMux is fully OpenAI-compatible. Base URL: https://zenmux.ai/api/v1. |
|
||||
|
||||
## Local Providers (11)
|
||||
|
||||
| ID | Alias | Name | Tags | Website | Notes |
|
||||
| --------------------- | ------------ | ------------------- | ------------------ | --------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `comfyui` | `comfyui` | ComfyUI | Local | [link](https://github.com/comfyanonymous/ComfyUI) | No API key required. Configure the local ComfyUI base URL (default: http://localhost:8188). |
|
||||
| `docker-model-runner` | `dmr` | Docker Model Runner | Local, self-hosted | [link](https://docs.docker.com/ai/model-runner/) | API key optional. Configure the local Docker Model Runner OpenAI-compatible base URL (default: http://localhost:12434/v1). |
|
||||
| `lemonade` | `lemonade` | Lemonade Server | Local, self-hosted | [link](https://lemonade-server.ai) | API key optional. Configure the local Lemonade OpenAI-compatible base URL (default: http://localhost:13305/api/v1). |
|
||||
| `llama-cpp` | `llamacpp` | llama.cpp | Local, self-hosted | [link](https://github.com/ggml-org/llama.cpp) | API key optional (use any value, e.g. sk-no-key-required). Configure the llama-server OpenAI-compatible base URL (default: http://127.0.0.1:8080/v1). Note: if Llamafile is also installed, both default to port 8080 — run only one at a time or override the port. |
|
||||
| `llamafile` | `llamafile` | Llamafile | Local, self-hosted | [link](https://github.com/Mozilla-Ocho/llamafile) | API key optional. Configure the local Llamafile OpenAI-compatible base URL (default: http://127.0.0.1:8080/v1). |
|
||||
| `lm-studio` | `lmstudio` | LM Studio | Local, self-hosted | [link](https://lmstudio.ai) | API key optional. Configure the local LM Studio OpenAI-compatible base URL (default: http://localhost:1234/v1). |
|
||||
| `oobabooga` | `ooba` | oobabooga | Local, self-hosted | [link](https://github.com/oobabooga/text-generation-webui) | API key optional. Configure the local oobabooga OpenAI-compatible base URL (default: http://localhost:5000/v1). |
|
||||
| `sdwebui` | `sdwebui` | SD WebUI | Local | [link](https://github.com/AUTOMATIC1111/stable-diffusion-webui) | No API key required. Configure the local WebUI base URL (default: http://localhost:7860). |
|
||||
| `triton` | `triton` | NVIDIA Triton | Local, self-hosted | [link](https://developer.nvidia.com/triton-inference-server) | API key optional. Configure the Triton OpenAI-compatible base URL (default: http://localhost:8000/v1). |
|
||||
| `vllm` | `vllm` | vLLM | Local, self-hosted | [link](https://github.com/vllm-project/vllm) | API key optional. Configure the local vLLM OpenAI-compatible base URL (default: http://localhost:8000/v1). |
|
||||
| `xinference` | `xinference` | XInference | Local, self-hosted | [link](https://inference.readthedocs.io) | API key optional. Configure the local XInference OpenAI-compatible base URL (default: http://localhost:9997/v1). |
|
||||
| ID | Alias | Name | Tags | Website | Notes |
|
||||
|----|-------|------|------|---------|-------|
|
||||
| `comfyui` | `comfyui` | ComfyUI | Local | [link](https://github.com/comfyanonymous/ComfyUI) | No API key required. Configure the local ComfyUI base URL (default: http://localhost:8188). |
|
||||
| `docker-model-runner` | `dmr` | Docker Model Runner | Local, self-hosted | [link](https://docs.docker.com/ai/model-runner/) | API key optional. Configure the local Docker Model Runner OpenAI-compatible base URL (default: http://localhost:12434/v1). |
|
||||
| `lemonade` | `lemonade` | Lemonade Server | Local, self-hosted | [link](https://lemonade-server.ai) | API key optional. Configure the local Lemonade OpenAI-compatible base URL (default: http://localhost:13305/api/v1). |
|
||||
| `llama-cpp` | `llamacpp` | llama.cpp | Local, self-hosted | [link](https://github.com/ggml-org/llama.cpp) | API key optional (use any value, e.g. sk-no-key-required). Configure the llama-server OpenAI-compatible base URL (default: http://127.0.0.1:8080/v1). Note: if Llamafile is also installed, both default to port 8080 — run only one at a time or override the port. |
|
||||
| `llamafile` | `llamafile` | Llamafile | Local, self-hosted | [link](https://github.com/Mozilla-Ocho/llamafile) | API key optional. Configure the local Llamafile OpenAI-compatible base URL (default: http://127.0.0.1:8080/v1). |
|
||||
| `lm-studio` | `lmstudio` | LM Studio | Local, self-hosted | [link](https://lmstudio.ai) | API key optional. Configure the local LM Studio OpenAI-compatible base URL (default: http://localhost:1234/v1). |
|
||||
| `oobabooga` | `ooba` | oobabooga | Local, self-hosted | [link](https://github.com/oobabooga/text-generation-webui) | API key optional. Configure the local oobabooga OpenAI-compatible base URL (default: http://localhost:5000/v1). |
|
||||
| `sdwebui` | `sdwebui` | SD WebUI | Local | [link](https://github.com/AUTOMATIC1111/stable-diffusion-webui) | No API key required. Configure the local WebUI base URL (default: http://localhost:7860). |
|
||||
| `triton` | `triton` | NVIDIA Triton | Local, self-hosted | [link](https://developer.nvidia.com/triton-inference-server) | API key optional. Configure the Triton OpenAI-compatible base URL (default: http://localhost:8000/v1). |
|
||||
| `vllm` | `vllm` | vLLM | Local, self-hosted | [link](https://github.com/vllm-project/vllm) | API key optional. Configure the local vLLM OpenAI-compatible base URL (default: http://localhost:8000/v1). |
|
||||
| `xinference` | `xinference` | XInference | Local, self-hosted | [link](https://inference.readthedocs.io) | API key optional. Configure the local XInference OpenAI-compatible base URL (default: http://localhost:9997/v1). |
|
||||
|
||||
## Search Providers (11)
|
||||
|
||||
| ID | Alias | Name | Tags | Website | Notes |
|
||||
| ------------------- | --------------- | -------------------------- | ------ | --------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------- |
|
||||
| `brave-search` | `brave-search` | Brave Search | Search | [link](https://brave.com/search/api) | Subscription token from Brave Search API dashboard |
|
||||
| `exa-search` | `exa-search` | Exa Search | Search | [link](https://exa.ai) | API key from dashboard.exa.ai |
|
||||
| `google-pse-search` | `google-pse` | Google Programmable Search | Search | [link](https://developers.google.com/custom-search/v1/overview) | Requires a Google API key and your Programmable Search Engine ID (cx) |
|
||||
| `linkup-search` | `linkup` | Linkup Search | Search | [link](https://docs.linkup.so) | Bearer API key from the Linkup dashboard |
|
||||
| `ollama-search` | `ollama-search` | Ollama Search | Search | [link](https://ollama.com/settings/api-keys) | Same API key as Ollama Cloud (from ollama.com/settings/api-keys) |
|
||||
| `perplexity-search` | `pplx-search` | Perplexity Search | Search | [link](https://docs.perplexity.ai/guides/search-quickstart) | Same API key as Perplexity (pplx-...) |
|
||||
| `searchapi-search` | `searchapi` | SearchAPI | Search | [link](https://www.searchapi.io/docs) | API key from SearchAPI (query param or Bearer auth) |
|
||||
| `searxng-search` | `searxng` | SearXNG Search | Search | [link](https://docs.searxng.org) | API key is optional. Set your SearXNG base URL. Some instances may require a bearer token for access. |
|
||||
| `serper-search` | `serper-search` | Serper Search | Search | [link](https://serper.dev) | API key from serper.dev dashboard |
|
||||
| `tavily-search` | `tavily-search` | Tavily Search | Search | [link](https://tavily.com) | API key from app.tavily.com (format: tvly-...) |
|
||||
| `youcom-search` | `youcom-search` | You.com Search | Search | [link](https://you.com/docs/search/overview) | X-API-Key from the You.com platform dashboard |
|
||||
| ID | Alias | Name | Tags | Website | Notes |
|
||||
|----|-------|------|------|---------|-------|
|
||||
| `brave-search` | `brave-search` | Brave Search | Search | [link](https://brave.com/search/api) | Subscription token from Brave Search API dashboard |
|
||||
| `exa-search` | `exa-search` | Exa Search | Search | [link](https://exa.ai) | API key from dashboard.exa.ai |
|
||||
| `google-pse-search` | `google-pse` | Google Programmable Search | Search | [link](https://developers.google.com/custom-search/v1/overview) | Requires a Google API key and your Programmable Search Engine ID (cx) |
|
||||
| `linkup-search` | `linkup` | Linkup Search | Search | [link](https://docs.linkup.so) | Bearer API key from the Linkup dashboard |
|
||||
| `ollama-search` | `ollama-search` | Ollama Search | Search | [link](https://ollama.com/settings/api-keys) | Same API key as Ollama Cloud (from ollama.com/settings/api-keys) |
|
||||
| `perplexity-search` | `pplx-search` | Perplexity Search | Search | [link](https://docs.perplexity.ai/guides/search-quickstart) | Same API key as Perplexity (pplx-...) |
|
||||
| `searchapi-search` | `searchapi` | SearchAPI | Search | [link](https://www.searchapi.io/docs) | API key from SearchAPI (query param or Bearer auth) |
|
||||
| `searxng-search` | `searxng` | SearXNG Search | Search | [link](https://docs.searxng.org) | API key is optional. Set your SearXNG base URL. Some instances may require a bearer token for access. |
|
||||
| `serper-search` | `serper-search` | Serper Search | Search | [link](https://serper.dev) | API key from serper.dev dashboard |
|
||||
| `tavily-search` | `tavily-search` | Tavily Search | Search | [link](https://tavily.com) | API key from app.tavily.com (format: tvly-...) |
|
||||
| `youcom-search` | `youcom-search` | You.com Search | Search | [link](https://you.com/docs/search/overview) | X-API-Key from the You.com platform dashboard |
|
||||
|
||||
## Audio-only Providers (7)
|
||||
|
||||
| ID | Alias | Name | Tags | Website | Notes |
|
||||
| ------------ | ---------- | ---------- | ----- | ------------------------------------- | ----------------------------------------------------------------------------------------------- |
|
||||
| `assemblyai` | `aai` | AssemblyAI | Audio | [link](https://assemblyai.com) | — |
|
||||
| `aws-polly` | `polly` | AWS Polly | Audio | [link](https://aws.amazon.com/polly/) | Use AWS Secret Access Key as API key; set providerSpecificData.accessKeyId and optional region. |
|
||||
| `cartesia` | `cartesia` | Cartesia | Audio | [link](https://cartesia.ai) | — |
|
||||
| `deepgram` | `dg` | Deepgram | Audio | [link](https://deepgram.com) | — |
|
||||
| `elevenlabs` | `el` | ElevenLabs | Audio | [link](https://elevenlabs.io) | — |
|
||||
| `inworld` | `inworld` | Inworld | Audio | [link](https://inworld.ai) | — |
|
||||
| `playht` | `playht` | PlayHT | Audio | [link](https://play.ht) | — |
|
||||
| ID | Alias | Name | Tags | Website | Notes |
|
||||
|----|-------|------|------|---------|-------|
|
||||
| `assemblyai` | `aai` | AssemblyAI | Audio | [link](https://assemblyai.com) | — |
|
||||
| `aws-polly` | `polly` | AWS Polly | Audio | [link](https://aws.amazon.com/polly/) | Use AWS Secret Access Key as API key; set providerSpecificData.accessKeyId and optional region. |
|
||||
| `cartesia` | `cartesia` | Cartesia | Audio | [link](https://cartesia.ai) | — |
|
||||
| `deepgram` | `dg` | Deepgram | Audio | [link](https://deepgram.com) | — |
|
||||
| `elevenlabs` | `el` | ElevenLabs | Audio | [link](https://elevenlabs.io) | — |
|
||||
| `inworld` | `inworld` | Inworld | Audio | [link](https://inworld.ai) | — |
|
||||
| `playht` | `playht` | PlayHT | Audio | [link](https://play.ht) | — |
|
||||
|
||||
## Upstream Proxy Providers (2)
|
||||
|
||||
| ID | Alias | Name | Tags | Website | Notes |
|
||||
| ------------- | ----- | ----------- | -------------- | ---------------------------------------------------- | ----- |
|
||||
| `9router` | `nr` | 9router | Upstream proxy | [link](https://www.npmjs.com/package/9router) | — |
|
||||
| `cliproxyapi` | `cpa` | CLIProxyAPI | Upstream proxy | [link](https://github.com/router-for-me/CLIProxyAPI) | — |
|
||||
| ID | Alias | Name | Tags | Website | Notes |
|
||||
|----|-------|------|------|---------|-------|
|
||||
| `9router` | `nr` | 9router | Upstream proxy | [link](https://www.npmjs.com/package/9router) | — |
|
||||
| `cliproxyapi` | `cpa` | CLIProxyAPI | Upstream proxy | [link](https://github.com/router-for-me/CLIProxyAPI) | — |
|
||||
|
||||
## Cloud Agent Providers (3)
|
||||
|
||||
| ID | Alias | Name | Tags | Website | Notes |
|
||||
| ------------- | ------------- | ------------ | ----------- | -------------------------------- | ----------------------------------------------------------- |
|
||||
| `codex-cloud` | `codex-cloud` | Codex Cloud | Cloud agent | [link](https://openai.com/codex) | OpenAI API key with Codex Cloud task access. |
|
||||
| `devin` | `devin` | Devin | Cloud agent | [link](https://devin.ai) | Devin API key for cloud agent sessions. |
|
||||
| `jules` | `jules` | Google Jules | Cloud agent | [link](https://jules.google) | Jules API key for creating and managing cloud coding tasks. |
|
||||
| ID | Alias | Name | Tags | Website | Notes |
|
||||
|----|-------|------|------|---------|-------|
|
||||
| `codex-cloud` | `codex-cloud` | Codex Cloud | Cloud agent | [link](https://openai.com/codex) | OpenAI API key with Codex Cloud task access. |
|
||||
| `devin` | `devin` | Devin | Cloud agent | [link](https://devin.ai) | Devin API key for cloud agent sessions. |
|
||||
| `jules` | `jules` | Google Jules | Cloud agent | [link](https://jules.google) | Jules API key for creating and managing cloud coding tasks. |
|
||||
|
||||
## System Providers (1)
|
||||
|
||||
| ID | Alias | Name | Tags | Website | Notes |
|
||||
| ------ | ------ | ------------------ | ------ | ------- | ----- |
|
||||
| `auto` | `auto` | Auto (Zero-Config) | System | — | — |
|
||||
| ID | Alias | Name | Tags | Website | Notes |
|
||||
|----|-------|------|------|---------|-------|
|
||||
| `auto` | `auto` | Auto (Zero-Config) | System | — | — |
|
||||
|
||||
## Sources of truth
|
||||
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
title: "OmniRoute Auto-Combo Engine"
|
||||
version: 3.8.2
|
||||
lastUpdated: 2026-05-13
|
||||
version: 3.8.31
|
||||
lastUpdated: 2026-06-20
|
||||
---
|
||||
|
||||
# OmniRoute Auto-Combo Engine
|
||||
@@ -26,6 +26,25 @@ lastUpdated: 2026-05-13
|
||||
| `auto/smart` | smart | Quality-first + higher exploration rate (10%) for better model discovery |
|
||||
| `auto/lkgp` | lkgp | Explicit LKGP (same as default `auto`) |
|
||||
|
||||
### Category × Tier Composition (`auto/<category>:<tier>`)
|
||||
|
||||
OpenRouter-style suffixes separate **what kind of route** (category) from **how to optimize it** (tier), so you can compose them freely (#4235 Phase B, `open-sse/services/autoCombo/suffixComposition.ts`):
|
||||
|
||||
- **Categories** (filter the candidate pool by capability): `coding` · `reasoning` · `vision` · `chat` · `multimodal`. `vision`/`multimodal` keep vision-capable models; `reasoning` keeps reasoning/thinking models.
|
||||
- **Tiers** (pick the scoring weights / pool filter): `fast` (ship-fast) · `cheap` (alias `floor`, cost-saver) · `reliable` (circuit-breaker health + latency stability) · `free` / `pro` (filter the pool by model tier via `classifyTier` — free-tier vs. premium).
|
||||
|
||||
| Example | Resolves to |
|
||||
| ---------------------- | ----------------------------------------------------------------- |
|
||||
| `auto/coding:fast` | coding pool, low-latency weights |
|
||||
| `auto/coding:cheap` | coding pool, cost-optimized (alias `auto/coding:floor`) |
|
||||
| `auto/reasoning:pro` | reasoning/thinking models only, premium tier |
|
||||
| `auto/vision` | vision-capable models (no tier → balanced weights) |
|
||||
| `auto/multimodal:free` | multimodal-capable models, free tier only |
|
||||
|
||||
Any valid `auto/<category>[:<tier>]` resolves on demand; a curated subset is advertised in `/v1/models` and the dashboard (`AUTO_SUFFIX_VARIANTS` in `open-sse/services/autoCombo/builtinCatalog.ts`). Filtering is **fail-open** — if a constraint matches no connected models, the full pool is used so routing never breaks. The core scorer (`combo.ts`) is unchanged; the category/tier filter is applied in `buildAutoCandidates`.
|
||||
|
||||
> **Live model intelligence:** auto-routing fitness is informed by live **Arena ELO** rankings + **models.dev** tier data when the `ARENA_ELO_SYNC_ENABLED` flag is on (falls back to the static fitness map otherwise).
|
||||
|
||||
**How to use:**
|
||||
|
||||
```bash
|
||||
|
||||
@@ -1,13 +1,13 @@
|
||||
---
|
||||
title: "Guardrails"
|
||||
version: 3.8.2
|
||||
lastUpdated: 2026-05-13
|
||||
version: 3.8.31
|
||||
lastUpdated: 2026-06-20
|
||||
---
|
||||
|
||||
# Guardrails
|
||||
|
||||
> **Source of truth:** `src/lib/guardrails/`
|
||||
> **Last updated:** 2026-05-13 — v3.8.0
|
||||
> **Last updated:** 2026-06-20 — v3.8.31 (injection-guard coverage + 16 KB scan bound + red-team)
|
||||
|
||||
Guardrails enforce safety, policy, and content transformations at the boundary
|
||||
between OmniRoute and upstream providers. Each guardrail can inspect (and
|
||||
@@ -109,6 +109,14 @@ suspicious content detected" }`. In `warn`/`log` modes the guardrail logs but
|
||||
allows the call. The shared helper `evaluatePromptInjection()` is also exported
|
||||
for callers that need to evaluate prompts without going through the registry.
|
||||
|
||||
**Scan bound (v3.8.20):** the detector only inspects the **first 16 KB** of
|
||||
joined prompt text — `MAX_INJECTION_SCAN_BYTES = 16 * 1024` (16 384 bytes) in
|
||||
`src/shared/utils/inputSanitizer.ts`. Both `detectInjection()` and
|
||||
`evaluatePromptInjection()` `slice(0, MAX_INJECTION_SCAN_BYTES)` before running
|
||||
the pattern loop. Injection directives sit near the top of an input, so this
|
||||
caps regex CPU/GC on multi-hundred-KB payloads without weakening detection (cf.
|
||||
#3932, #4041).
|
||||
|
||||
## Base Contract (`base.ts`)
|
||||
|
||||
```typescript
|
||||
@@ -294,3 +302,20 @@ A extração de texto (`extractMessageContents`) cobre `messages`/`input`/`promp
|
||||
o corpus OWASP-LLM em `INJECTION_GUARD_MODE=block`; garak roda probes (skip sem secret).
|
||||
`moderations` é incluída por consistência — operadores em block-mode podem isentá-la via
|
||||
`resolveDisabledGuardrails`.
|
||||
|
||||
The nightly workflow (`.github/workflows/nightly-llm-security.yml`, cron + manual
|
||||
dispatch) has two jobs:
|
||||
|
||||
- **`promptfoo-guard` (blocking)** — runs `promptfoo eval -c promptfooconfig.yaml`
|
||||
with `INJECTION_GUARD_MODE=block`. Each adversarial case (e.g. "ignore all
|
||||
previous instructions…", DAN-style jailbreaks) asserts the response carries
|
||||
`error.code === "SECURITY_001"`, i.e. the guard actually rejected the request.
|
||||
- **`garak` (advisory)** — runs garak `--probes promptinject,dan,leakreplay`
|
||||
against a local OmniRoute instance (`http://localhost:20128/v1`). Gated on a
|
||||
provider secret (`PROMPTFOO_PROVIDER_KEY`); skips gracefully and is suffixed
|
||||
`|| true`, so it reports without failing CI.
|
||||
|
||||
Coverage of the guard helper (`createInjectionGuard` / `withInjectionGuard`)
|
||||
spans every prompt-bearing `/v1` route; prompt text is pulled from
|
||||
`messages`/`input`/`prompt`/`query`+`documents`/`instructions`/`system` by
|
||||
`extractMessageContents()` in `src/shared/utils/inputSanitizer.ts`.
|
||||
|
||||
381
docs/security/MITM-TPROXY-DECRYPT.md
Normal file
381
docs/security/MITM-TPROXY-DECRYPT.md
Normal file
@@ -0,0 +1,381 @@
|
||||
---
|
||||
title: "MITM TPROXY Transparent Decrypt"
|
||||
version: 3.8.31
|
||||
lastUpdated: 2026-06-20
|
||||
---
|
||||
|
||||
# MITM TPROXY Transparent Decrypt
|
||||
|
||||
TPROXY transparent decrypt is OmniRoute's **5th capture mode** for the
|
||||
[Traffic Inspector](../frameworks/TRAFFIC_INSPECTOR.md) / [AgentBridge](../frameworks/AGENTBRIDGE.md)
|
||||
MITM stack. It intercepts and **decrypts** local outbound HTTPS traffic on Linux
|
||||
using kernel TPROXY + policy routing — **without** spoofing `/etc/hosts` and
|
||||
**without** mutating OS-wide system-proxy settings. It is headless-friendly
|
||||
(no DNS edits to clean up) and the firewall rules auto-flush on reboot.
|
||||
|
||||
Unlike the other capture modes, TPROXY needs no per-host setup: it transparently
|
||||
intercepts **arbitrary** destination hosts on a target port, terminates TLS with
|
||||
a leaf certificate it issues on the fly per SNI hostname, captures the decrypted
|
||||
exchange, and re-encrypts the request to the original destination.
|
||||
|
||||
> **Linux-only, root-only, opt-in.** This mode requires Linux, a native addon
|
||||
> built with a C toolchain, and the **CAP_NET_ADMIN** capability (typically root). It is gated
|
||||
> behind the loopback-only AgentBridge API and disabled by default. A trusted
|
||||
> MITM CA that can sign any host is a powerful capability — see [§6 Security](#6-security).
|
||||
|
||||
**Source:** `src/mitm/tproxy/`
|
||||
**API route:** `GET / POST / DELETE /api/tools/agent-bridge/tproxy`
|
||||
**Dashboard toggle:** Traffic Inspector → capture-modes toolbar → **"TPROXY Decrypt"** ⚠
|
||||
**See also:** [`docs/frameworks/TRAFFIC_INSPECTOR.md`](../frameworks/TRAFFIC_INSPECTOR.md),
|
||||
[`docs/frameworks/AGENTBRIDGE.md`](../frameworks/AGENTBRIDGE.md)
|
||||
|
||||
---
|
||||
|
||||
## §1 What it is and when to use it
|
||||
|
||||
The other four capture modes each have a limitation:
|
||||
|
||||
| Mode | How traffic is steered | Limitation |
|
||||
|------|------------------------|------------|
|
||||
| AgentBridge | `/etc/hosts` DNS spoof of a fixed host set | only the registered IDE-agent hosts |
|
||||
| Custom Hosts | `/etc/hosts` DNS spoof per host | one entry per host; sudo to edit hosts |
|
||||
| HTTP_PROXY | `HTTP_PROXY`/`HTTPS_PROXY` env | only apps that honor the env var |
|
||||
| System-wide proxy | OS proxy settings | mutates global state; needs revert |
|
||||
|
||||
TPROXY transparent decrypt steers traffic at the **kernel** layer instead. It
|
||||
marks new local outbound TCP connections to a target port (default `443`) in the
|
||||
`mangle OUTPUT` chain, an `ip rule` reroutes the marked packets to local delivery,
|
||||
and on re-entry the `mangle PREROUTING` `TPROXY` target hands them to an
|
||||
**IP_TRANSPARENT** listener — which then terminates TLS and captures the plaintext.
|
||||
|
||||
Use it when you want to capture and decrypt traffic from a process that:
|
||||
|
||||
- talks to a host AgentBridge does not register, and
|
||||
- does not honor `HTTP_PROXY`, and
|
||||
- you do not want to disturb with a system-wide proxy change.
|
||||
|
||||
Because interception happens in the kernel, the originating process needs **no
|
||||
configuration change** — but the process must trust the dynamic CA OmniRoute
|
||||
installs (see [§4](#4-the-per-sni-dynamic-ca-and-trust-store-installer)).
|
||||
|
||||
---
|
||||
|
||||
## §2 Requirements
|
||||
|
||||
| Requirement | Detail |
|
||||
|-------------|--------|
|
||||
| **OS** | Linux only — **IP_TRANSPARENT** is a Linux-only socket option. The loader returns "unavailable" on every other platform. |
|
||||
| **Privilege** | The **CAP_NET_ADMIN** capability to create the transparent socket and apply `iptables`/`ip` rules — in practice, run as root. |
|
||||
| **Native addon** | A tiny N-API addon (`src/mitm/tproxy/native/transparent.c`) must be built or shipped as a prebuild. See [§3](#3-the-native-ip_transparent-addon). |
|
||||
| **Kernel modules** | `iptables` with the `TPROXY`, `mangle`, and `mark` match support (validated against kernel 6.8.0). |
|
||||
|
||||
**Graceful degradation:** if any requirement is missing (non-Linux, no toolchain,
|
||||
addon not built), the addon loader (`src/mitm/tproxy/transparentSocket.ts::loadTransparentAddon`)
|
||||
returns `null` rather than throwing. The capture-mode status then reports
|
||||
`available: false`, the dashboard toggle is **disabled** with the tooltip
|
||||
"TPROXY decrypt requires Linux + root + the native addon", and the rest of
|
||||
OmniRoute keeps working.
|
||||
|
||||
---
|
||||
|
||||
## §3 The native IP_TRANSPARENT addon
|
||||
|
||||
Node's `net` module cannot `setsockopt(IP_TRANSPARENT)` *before* `bind()`, which
|
||||
TPROXY requires (otherwise the kernel drops the redirected packets). The addon
|
||||
(`src/mitm/tproxy/native/transparent.c`, built via `binding.gyp`) is a small N-API
|
||||
module exposing three functions, consumed through `transparentSocket.ts`:
|
||||
|
||||
| Addon function | Socket work | Used for |
|
||||
|----------------|-------------|----------|
|
||||
| `createTransparentListener(ip, port)` | `socket()` + **SO_REUSEADDR** + **IP_TRANSPARENT** + `bind()` + `listen()`, returns the raw fd | the transparent capture listener (Node adopts the fd via `server.listen({ fd })`) |
|
||||
| `setSocketMark(fd, mark)` | `setsockopt` **SO_MARK** on an existing fd | anti-loop (mark the proxy's own sockets) |
|
||||
| `connectMarked(ip, port, mark)` | `socket()` + **SO_MARK** **before** a non-blocking `connect()`, returns fd | the re-encrypted upstream forward (the SYN carries the mark) |
|
||||
|
||||
The original destination is read from `socket.localAddress`/`localPort` — TPROXY
|
||||
preserves it, so there is no **SO_ORIGINAL_DST**/NAT lookup.
|
||||
|
||||
### Building the addon
|
||||
|
||||
```bash
|
||||
npm run build:native:tproxy # cd src/mitm/tproxy/native && node-gyp rebuild
|
||||
# -> native/build/Release/transparent.node
|
||||
```
|
||||
|
||||
- During `npm run build`, `scripts/build/build-tproxy-native.mjs` runs `node-gyp
|
||||
rebuild`. It is **Linux-only and non-fatal** — a missing toolchain just leaves
|
||||
the capture mode unavailable.
|
||||
- `assembleStandalone.mjs` copies `build/Release/transparent.node` into the
|
||||
standalone bundle; `transparentSocket.ts` resolves it both module-relative and
|
||||
cwd-relative (`<cwd>/src/mitm/tproxy/native/...`).
|
||||
- `build/` and `prebuilds/` are git-ignored — the binary is **built, never
|
||||
committed**.
|
||||
|
||||
The loader probes, in priority order:
|
||||
`native/build/Release/transparent.node`, then `native/prebuilds/transparent.node`
|
||||
(both module-relative and under `<cwd>/src/mitm/tproxy/`).
|
||||
|
||||
---
|
||||
|
||||
## §4 The per-SNI dynamic CA and trust-store installer
|
||||
|
||||
The static AgentBridge MITM cert works only because AgentBridge DNS-spoofs a
|
||||
**fixed** host set. TPROXY intercepts **arbitrary** hosts, so the listener must
|
||||
present a valid leaf for whatever SNI the client requests.
|
||||
|
||||
### Dynamic CA (`src/mitm/tproxy/dynamicCert.ts`)
|
||||
|
||||
`DynamicCertStore` runs a local CA (built on the `selfsigned` dependency) that:
|
||||
|
||||
- Generates a long-lived CA via `generateMitmCa()` (CN `"OmniRoute MITM CA"`,
|
||||
10-year validity, `basicConstraints CA=true` + `keyUsage keyCertSign,cRLSign`,
|
||||
2048-bit RSA / SHA-256).
|
||||
- Issues a **leaf per SNI hostname on demand** via `issueLeafCert()` (1-year
|
||||
validity, `subjectAltName` = the SNI host) and caches one `tls.SecureContext`
|
||||
per hostname.
|
||||
- Exposes `createSNICallback()` for the TLS-terminating server (see [§5](#5-how-decrypt-and-capture-work)).
|
||||
- Can be constructed with an `existingCa` to keep the CA stable across restarts
|
||||
(so the trust store does not need re-installing).
|
||||
|
||||
The CA private key **never leaves the machine**.
|
||||
|
||||
### Trust-store installer (`src/mitm/tproxy/caTrust.ts`)
|
||||
|
||||
The intercepted client must trust the dynamic CA, so starting the capture mode
|
||||
installs the CA cert into the OS trust store under a **dedicated slot** —
|
||||
`omniroute-tproxy-ca.crt` (constant `TPROXY_CA_CERT_NAME`) — kept separate from
|
||||
the static MITM cert's slot (`omniroute-mitm.crt`) so the two never clobber each
|
||||
other.
|
||||
|
||||
`installTproxyCa(caPem, sudoPassword?)` detects the distro's anchor directory
|
||||
(in order: Debian-style first) and runs the matching refresh command:
|
||||
|
||||
| Anchor directory | Refresh command |
|
||||
|------------------|-----------------|
|
||||
| `/usr/local/share/ca-certificates` | `update-ca-certificates` |
|
||||
| `/etc/ca-certificates/trust-source/anchors` | `update-ca-trust` |
|
||||
| `/etc/pki/ca-trust/source/anchors` | `update-ca-trust` |
|
||||
| `/etc/pki/trust/anchors` | `update-ca-certificates` |
|
||||
|
||||
Install stages the PEM to a temp file, then (privileged) `mkdir -p` the anchor
|
||||
dir, `cp` the staged file into it, and runs the refresh command. `uninstallTproxyCa()`
|
||||
removes the dedicated slot only (leaving the static MITM cert untouched) and
|
||||
refreshes — a no-op on non-Linux.
|
||||
|
||||
All privileged commands run via `execFileWithPassword` (`src/mitm/systemCommands.ts`)
|
||||
— `spawn` with **arg arrays, no shell, no string interpolation** (Hard Rule #13).
|
||||
When the process is root (e.g. the VPS) the target runs directly and no password
|
||||
is needed; on a non-root desktop the `sudoPassword` is passed via `sudo -S` on stdin.
|
||||
|
||||
> The desktop's `sudoPassword` is supplied in the POST body to authorize the
|
||||
> trust-store install; it is ignored entirely when the process is root.
|
||||
|
||||
---
|
||||
|
||||
## §5 How decrypt and capture work
|
||||
|
||||
The pipeline (all under `src/mitm/tproxy/`):
|
||||
|
||||
```
|
||||
local app ──TCP/443──▶ mangle OUTPUT marks the conn (fwmark)
|
||||
ip rule → local route table → lo
|
||||
mangle PREROUTING TPROXY → IP_TRANSPARENT listener (port 8443)
|
||||
│ captureMode.ts: reads orig dest from socket.localAddress
|
||||
▼
|
||||
tlsCapture.ts:
|
||||
1. TLS-terminate the CLIENT with a per-SNI leaf (dynamicCert)
|
||||
2. internal http.Server parses the decrypted plaintext
|
||||
3. capture → globalTrafficBuffer.push() with source: "tproxy"
|
||||
(sanitizeHeaders + maskSecret applied)
|
||||
4. forward RE-encrypted to the original destination
|
||||
over a bypass-marked socket (connectMarked, anti-loop)
|
||||
│
|
||||
▼
|
||||
original upstream (api.example.com)
|
||||
```
|
||||
|
||||
- **TLS termination** (`createTlsCaptureServer`): wraps the raw intercepted
|
||||
socket in a server-side `tls.TLSSocket` using the dynamic CA's SNI callback,
|
||||
then hands the decrypted stream to an internal `http.Server` (the standard MITM
|
||||
termination trick). Socket lifetimes are bounded by `MITM_IDLE_TIMEOUT_MS` so a
|
||||
hung tunnel cannot exhaust file descriptors.
|
||||
- **Capture** (`handleDecryptedRequest`): pushes an `InterceptedRequest` with
|
||||
`source: "tproxy"`, status starting `"in-flight"`, headers run through
|
||||
`sanitizeHeaders()` and bodies through `maskSecret()` before they enter the
|
||||
buffer. The entry is then updated with the response, sizes, and latency.
|
||||
- **Re-encrypted forward** (`createForward` / `realForward`): re-encrypts to the
|
||||
original destination. `rejectUnauthorized` defaults to **`true`** (secure by
|
||||
default) — the upstream cert is verified against the SNI/Host the client
|
||||
requested, so the proxy rejects exactly what the original client would.
|
||||
|
||||
### Anti-loop (SO_MARK)
|
||||
|
||||
Because the rules mark new local outbound connections, the proxy's **own**
|
||||
re-encrypted forward would normally be re-intercepted — an infinite loop. The
|
||||
forward path defends against this with a bypass socket mark (**SO_MARK**):
|
||||
|
||||
- `realForward` opens its upstream socket via `connectMarked(ip, port, DEFAULT_BYPASS_MARK)`
|
||||
— `DEFAULT_BYPASS_MARK = 0x539` — which sets the **SO_MARK** **before** `connect()`,
|
||||
so the forward's SYN carries the bypass mark.
|
||||
- The `mangle OUTPUT` rule excludes connections already carrying the bypass mark
|
||||
(`-m mark ! --mark <bypassMark>`), so the proxy's forward is **not** re-marked
|
||||
and does not re-enter TPROXY.
|
||||
|
||||
> Implementation note: the bypass-marked socket must be installed on the agent's
|
||||
> `createConnection` (`https.request({ createConnection })` is silently ignored
|
||||
> when an agent is present), or the forward would open an unmarked socket and the
|
||||
> loop would return. This was the e2e-validated anti-loop fix.
|
||||
|
||||
---
|
||||
|
||||
## §6 Security
|
||||
|
||||
| Control | Detail |
|
||||
|---------|--------|
|
||||
| **Loopback-only API** | `/api/tools/agent-bridge/tproxy` is covered by the `/api/tools/agent-bridge/` prefix in `LOCAL_ONLY_API_PREFIXES` (`src/server/authz/routeGuard.ts`). Loopback enforcement runs **before** auth (Hard Rules #15 + #17) — a leaked JWT over a tunnel cannot start TPROXY capture, which applies `iptables` rules and installs a trust-store CA via child processes. |
|
||||
| **Dedicated CA slot** | The dynamic CA installs to `omniroute-tproxy-ca.crt`, never clobbering the static MITM cert. |
|
||||
| **CA key never leaves the host** | `DynamicCertStore` holds the CA key in memory; it is not exported. |
|
||||
| **Secret masking** | `maskSecret()` on request/response bodies and `sanitizeHeaders()` on headers run **before** `globalTrafficBuffer.push()`. |
|
||||
| **No shell interpolation** | All `iptables`/`ip`/trust-store commands run via `execFile`/`execFileWithPassword` with arg arrays (Hard Rule #13). |
|
||||
| **Upstream cert verification** | The re-encrypted forward verifies the upstream cert by default (`rejectUnauthorized: true`). |
|
||||
| **Error sanitization** | The route's error responses go through `sanitizeErrorMessage()` (Hard Rule #12). |
|
||||
|
||||
**The MITM CA is a powerful capability.** A CA trusted by the OS that can sign any
|
||||
host means anything OmniRoute intercepts can be decrypted. It is gated behind the
|
||||
explicit, local-only TPROXY capture mode, off by default, and the trust-store
|
||||
entry is removed when you stop the mode.
|
||||
|
||||
---
|
||||
|
||||
## §7 Transactional firewall apply / revert
|
||||
|
||||
A crash must never leave a `mangle` rule or stale route behind. The command builder
|
||||
(`src/mitm/tproxy/commands.ts`) and runner (`src/mitm/tproxy/setup.ts`) guarantee
|
||||
**revert is the exact inverse of apply, in reverse order**.
|
||||
|
||||
`applyTproxy(cfg)` runs the apply commands in order; on **any** failure it runs a
|
||||
best-effort full `revertTproxy(cfg)` and rethrows — so the firewall is either
|
||||
fully applied or fully reverted, never half-applied. `revertTproxy(cfg)` runs the
|
||||
inverse commands in reverse order and swallows failures (idempotent — safe to call
|
||||
unconditionally, e.g. from the AgentBridge `repairMitm()` cleanup).
|
||||
|
||||
`validateTproxyConfig(cfg)` runs before any command: ports must be `1–65535`,
|
||||
`mark`/`routeTable`/`bypassMark` must be positive integers, and `bypassMark` must
|
||||
differ from `mark` (anti-loop).
|
||||
|
||||
### Apply commands (in order)
|
||||
|
||||
```bash
|
||||
ip rule add fwmark <mark> lookup <routeTable>
|
||||
ip route add local 0.0.0.0/0 dev lo table <routeTable>
|
||||
iptables -t mangle -A OUTPUT -p tcp --dport <dport> -m mark ! --mark <bypassMark> -j MARK --set-mark <mark>
|
||||
iptables -t mangle -A PREROUTING -p tcp --dport <dport> -m mark --mark <mark> -j TPROXY --on-port <onPort> --tproxy-mark <mark>
|
||||
```
|
||||
|
||||
Revert deletes them in reverse: `PREROUTING -D`, `OUTPUT -D`, `ip route del`, `ip rule del`.
|
||||
|
||||
> The recipe is **OUTPUT-based** because the MITM use case is *local* outbound
|
||||
> traffic (apps on the same host), which TPROXY in `PREROUTING` alone does not
|
||||
> see — `PREROUTING` only sees forwarded traffic. The `OUTPUT` chain marks new
|
||||
> local connections, the `ip rule` reroutes them to local delivery (`lo`), and
|
||||
> `PREROUTING` then assigns them to the transparent listener.
|
||||
|
||||
---
|
||||
|
||||
## §8 Configuration
|
||||
|
||||
The start request (`POST /api/tools/agent-bridge/tproxy`) accepts the following
|
||||
fields, validated by `StartTproxyBodySchema` (`tproxy/route.ts`). All are optional
|
||||
and fall back to their defaults:
|
||||
|
||||
| Field | Type | Default | Notes |
|
||||
|-------|------|---------|-------|
|
||||
| **dport** | int (1–65535) | `443` | Destination TCP port to transparently intercept |
|
||||
| **mark** | int (≥1) | `0x2333` | Firewall mark set on `OUTPUT`, matched by the `ip rule` + `PREROUTING` |
|
||||
| **onPort** | int (1–65535) | `8443` | Port the transparent (**IP_TRANSPARENT**) listener binds |
|
||||
| **routeTable** | int (≥1) | `233` | Policy-routing table id holding the `local 0.0.0.0/0` route |
|
||||
| **bypassMark** | int (≥1, ≠ `mark`) | `0x539` | The bypass socket mark (**SO_MARK**) the proxy sets on its own upstream conns; excluded in `OUTPUT` (anti-loop) |
|
||||
| **sudoPassword** | string | — | Non-root desktops only: authorizes the trust-store install; ignored when root |
|
||||
|
||||
There are **no environment variables** for TPROXY — all configuration is via the
|
||||
POST body or the defaults above.
|
||||
|
||||
---
|
||||
|
||||
## §9 Enabling from the Traffic Inspector
|
||||
|
||||
1. Open the **Traffic Inspector** (`/dashboard/tools/traffic-inspector`).
|
||||
2. In the capture-modes toolbar, find the **"TPROXY Decrypt"** ⚠ button
|
||||
(`src/app/(dashboard)/dashboard/tools/traffic-inspector/components/CaptureModesToolbar.tsx`).
|
||||
- If it is **disabled** with the tooltip "TPROXY decrypt requires Linux + root +
|
||||
the native addon", the native addon is unavailable on this host (non-Linux,
|
||||
no toolchain, or addon not built). See [§2](#2-requirements) and [§3](#3-the-native-ip_transparent-addon).
|
||||
3. Click the button. It calls `POST /api/tools/agent-bridge/tproxy` via
|
||||
`startTproxyCaptureMode()` (`src/lib/inspector/tproxyCaptureApi.ts`), which:
|
||||
builds the dynamic CA, opens the transparent listener, applies the firewall
|
||||
rules, and installs the CA in the OS trust store.
|
||||
4. When running, the toggle turns amber and shows the live intercept count
|
||||
(`· <interceptCount>`). Intercepted requests appear in the request list with
|
||||
`source: "tproxy"`.
|
||||
5. Click again to stop — `DELETE /api/tools/agent-bridge/tproxy` via
|
||||
`stopTproxyCaptureMode()` closes the listener, uninstalls the CA, and reverts
|
||||
the firewall rules.
|
||||
|
||||
The capture-mode status (running / available / intercept count / listener port) comes
|
||||
from `GET /api/tools/agent-bridge/tproxy` (`getCaptureStatus()` in
|
||||
`src/mitm/tproxy/captureManager.ts`). Only **one** TPROXY session runs at a time —
|
||||
starting a second rejects with "TPROXY capture mode is already running".
|
||||
|
||||
---
|
||||
|
||||
## §10 Troubleshooting
|
||||
|
||||
### Toggle is disabled
|
||||
|
||||
The native addon is not loadable. Confirm: you are on Linux, you built the addon
|
||||
(`npm run build:native:tproxy`), and the process can load `transparent.node`.
|
||||
`isTransparentSocketAvailable()` gates the toggle; `GET /api/tools/agent-bridge/tproxy`
|
||||
returns `available: false` when the addon is missing.
|
||||
|
||||
### Nothing is captured
|
||||
|
||||
- Confirm the intercepted process actually connects to the configured `dport`
|
||||
(default `443`).
|
||||
- Confirm the process trusts the dynamic CA. The CA is installed under
|
||||
`omniroute-tproxy-ca.crt`; apps with their own trust store (Firefox/Chrome NSS)
|
||||
may need the cert added there too.
|
||||
- Run the AgentBridge **Diagnose** self-test (see
|
||||
[`AGENTBRIDGE.md`](../frameworks/AGENTBRIDGE.md)) for cert-trusted / server
|
||||
health checks.
|
||||
|
||||
### Stale firewall rules after a crash
|
||||
|
||||
`revertTproxy()` is the exact inverse of apply and is idempotent. Stopping the
|
||||
mode reverts the rules; if OmniRoute was killed mid-session, use the AgentBridge
|
||||
**Repair** action (`POST /api/tools/agent-bridge/repair`) to undo orphaned system
|
||||
state (DNS spoof, root CA, system proxy). The TPROXY `mangle` rules and route also
|
||||
flush automatically on reboot.
|
||||
|
||||
### Infinite loop / the proxy intercepts its own forward
|
||||
|
||||
This is the anti-loop case. Confirm `bypassMark` differs from `mark` (validation
|
||||
enforces this) and that the forward uses `connectMarked` (it does in `realForward`).
|
||||
See [§5 Anti-loop](#anti-loop-so_mark).
|
||||
|
||||
---
|
||||
|
||||
## §11 Source map
|
||||
|
||||
| File | Responsibility |
|
||||
|------|----------------|
|
||||
| `src/mitm/tproxy/commands.ts` | Pure `iptables`/`ip` apply + revert command builder; `validateTproxyConfig` |
|
||||
| `src/mitm/tproxy/setup.ts` | Transactional `applyTproxy` / `revertTproxy` runner (rollback on failure) |
|
||||
| `src/mitm/tproxy/transparentSocket.ts` | Native-addon loader (`loadTransparentAddon`), `createTransparentListenerFd`, `connectMarked`, `setSocketMark`, `isTransparentSocketAvailable` |
|
||||
| `src/mitm/tproxy/native/transparent.c` | N-API addon: `createTransparentListener` (IP_TRANSPARENT), `setSocketMark`, `connectMarked` |
|
||||
| `src/mitm/tproxy/native/binding.gyp` | node-gyp build manifest |
|
||||
| `src/mitm/tproxy/dynamicCert.ts` | `DynamicCertStore` — per-SNI dynamic CA + leaf cache |
|
||||
| `src/mitm/tproxy/caTrust.ts` | OS trust-store install/uninstall (`installTproxyCa` / `uninstallTproxyCa`, dedicated slot) |
|
||||
| `src/mitm/tproxy/tlsCapture.ts` | TLS-terminating decrypt engine + re-encrypted anti-loop forward |
|
||||
| `src/mitm/tproxy/captureMode.ts` | Transparent-listener orchestration; reads orig dest from `socket.localAddress` |
|
||||
| `src/mitm/tproxy/captureManager.ts` | Singleton lifecycle: `startCaptureMode` / `stopCaptureMode` / `getCaptureStatus` |
|
||||
| `src/app/api/tools/agent-bridge/tproxy/route.ts` | `GET` / `POST` / `DELETE` route (LOCAL_ONLY) |
|
||||
| `src/lib/inspector/tproxyCaptureApi.ts` | Client fetch helpers (`fetchTproxyStatus` / `startTproxyCaptureMode` / `stopTproxyCaptureMode`) |
|
||||
@@ -177,6 +177,10 @@ const DOC_ONLY_ALLOWLIST = new Set([
|
||||
"IFLOW_OAUTH_CLIENT_SECRET",
|
||||
// Source-code constants accidentally captured by the doc regex.
|
||||
"CLI_COMPAT_OMITTED_PROVIDER_IDS",
|
||||
// The stream-recovery tuning object in open-sse/config/constants.ts (`STREAM_RECOVERY.HOLDBACK_MS`
|
||||
// etc.) — documented for reference; the real operator-facing env vars are STREAM_RECOVERY_ENABLED /
|
||||
// STREAM_RECOVERY_MIDSTREAM_ENABLED (both in .env.example). The bare prefix is not an env var.
|
||||
"STREAM_RECOVERY",
|
||||
// Sample default values that look like SHOUTY_NAMES (not env vars).
|
||||
"CHANGEME",
|
||||
// Legacy aliases — present in docs as "would be aliases" but read-only
|
||||
|
||||
@@ -108,6 +108,8 @@ const ENV_VAR_ALLOWLIST = new Set([
|
||||
"CODEX_HOME", // Codex CLI's own config-home env var (CODEX-CLI-CONFIGURATION.md)
|
||||
"GEMINI_API_KEY", // Gemini CLI's own API-key env var, set by `omniroute setup-gemini` (REMOTE-MODE.md)
|
||||
"GOOGLE_GEMINI_BASE_URL", // Gemini CLI's own base-URL env var, set by `omniroute setup-gemini` (REMOTE-MODE.md)
|
||||
"OPENAI_API_BASE", // legacy OpenAI base-URL env var some downstream tools (e.g. Aider) read (CLI-INTEGRATIONS.md)
|
||||
"PROMPTFOO_PROVIDER_KEY", // promptfoo's own provider-key env var, used by the red-team suite (GUARDRAILS.md)
|
||||
"REDIS_PORT", // docker-compose host-port override (DOCKER_GUIDE.md)
|
||||
"AUTO_UPDATE_HOST_REPO_DIR", // docker-compose self-update mount (DOCKER_GUIDE.md)
|
||||
"LINUX_GPG_KEY", // electron AppImage signing key, CI/build only (ELECTRON_GUIDE.md)
|
||||
|
||||
Reference in New Issue
Block a user