Compare commits
109 Commits
fix/10785-
...
dependabot
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
eadbb3f544 | ||
|
|
118840131d | ||
|
|
3fed9e837a | ||
|
|
1f04e73a1c | ||
|
|
f66c986dbd | ||
|
|
3669df2ca5 | ||
|
|
590cbbe7b1 | ||
|
|
c7e264e1a6 | ||
|
|
699be22e1e | ||
|
|
7756aef970 | ||
|
|
050c7c0021 | ||
|
|
c535df9076 | ||
|
|
5d9998eb31 | ||
|
|
19741775ee | ||
|
|
b668d91364 | ||
|
|
4ec080dc19 | ||
|
|
0458c5ac4c | ||
|
|
9603ec1bf1 | ||
|
|
6d043674c2 | ||
|
|
bed4d24049 | ||
|
|
018badc3b3 | ||
|
|
87719f2381 | ||
|
|
0a1f1d42ee | ||
|
|
eb6f319712 | ||
|
|
bc9090ba65 | ||
|
|
d9cb4f5f5d | ||
|
|
ce6249cbb7 | ||
|
|
9935f80971 | ||
|
|
e968d11b1c | ||
|
|
7afafcecc9 | ||
|
|
c79faa45fb | ||
|
|
871832820f | ||
|
|
d87b97a786 | ||
|
|
25ba4f2a34 | ||
|
|
c40ff16a1d | ||
|
|
0b51a242ce | ||
|
|
7f90af645c | ||
|
|
e6801bace1 | ||
|
|
82ed31d27a | ||
|
|
dacf4c3c1a | ||
|
|
362c5acbfe | ||
|
|
7fd82eb146 | ||
|
|
6f28688b04 | ||
|
|
12d0acbe06 | ||
|
|
61051a1460 | ||
|
|
1fb466a1ee | ||
|
|
2c84ce19df | ||
|
|
e5b7c40d11 | ||
|
|
053c64d380 | ||
|
|
3112304db6 | ||
|
|
a280bfc112 | ||
|
|
d99701d6b3 | ||
|
|
80b517edea | ||
|
|
a72dc25c04 | ||
|
|
621f30a188 | ||
|
|
ff9a4c2fbd | ||
|
|
f52fa9dc85 | ||
|
|
424b950856 | ||
|
|
01b3828278 | ||
|
|
998c3c2129 | ||
|
|
b59a88b7eb | ||
|
|
54b39690e5 | ||
|
|
22e46a0875 | ||
|
|
2a10d16114 | ||
|
|
06315c445c | ||
|
|
31031f93ef | ||
|
|
c6a0d09bcd | ||
|
|
b052c91014 | ||
|
|
bbcfb730ca | ||
|
|
db7c3abaf6 | ||
|
|
567b9db04d | ||
|
|
8bc2f0f10c | ||
|
|
8c4a219746 | ||
|
|
7288fa0dd7 | ||
|
|
d14a4d2da1 | ||
|
|
2acafd9c9e | ||
|
|
9d2240eab7 | ||
|
|
9eddafff60 | ||
|
|
7c6bf32186 | ||
|
|
56b9d00335 | ||
|
|
2f7315882b | ||
|
|
77d75022d6 | ||
|
|
6f08a089e7 | ||
|
|
8b52596d7c | ||
|
|
6767f27011 | ||
|
|
7ac6bbba37 | ||
|
|
84d7e33c26 | ||
|
|
80a59c0ae5 | ||
|
|
8122f6b71c | ||
|
|
5e508147c4 | ||
|
|
a352c23bad | ||
|
|
4821f9ffdb | ||
|
|
f4772500bc | ||
|
|
b43ad73166 | ||
|
|
bb98e9a345 | ||
|
|
ff8b7b172f | ||
|
|
0bfaaa4929 | ||
|
|
4c15c05f9b | ||
|
|
8cd248b4f5 | ||
|
|
05a37634c2 | ||
|
|
62f6e87869 | ||
|
|
74c54828fc | ||
|
|
f060117464 | ||
|
|
9fc3b29217 | ||
|
|
e9dd87ad77 | ||
|
|
0f13fe4221 | ||
|
|
5089c17b44 | ||
|
|
bc6129bcb2 | ||
|
|
142ae93498 |
37
.env.example
@@ -229,6 +229,15 @@ PORT=20128
|
||||
# unaffected by this dev-only flag).
|
||||
OMNIROUTE_USE_TURBOPACK=1
|
||||
|
||||
# Disable systemd sd_notify (Type=notify / WatchdogSec=) even when running
|
||||
# under a systemd unit with NOTIFY_SOCKET set.
|
||||
# Used by: scripts/dev/systemd-notify.mjs. Set to 1 to disable.
|
||||
# OMNIROUTE_DISABLE_SD_NOTIFY=1
|
||||
|
||||
# Injected by systemd when running under a service unit (sd_notify protocol).
|
||||
# Read by scripts/dev/systemd-notify.mjs — never set this yourself.
|
||||
# NOTIFY_SOCKET=/run/systemd/notify
|
||||
|
||||
# Skip the SQLite integrity health check on startup (faster boot on large DBs).
|
||||
# Used by: src/lib/db/core.ts, src/lib/db/healthCheck.ts. Set to 1 to skip.
|
||||
# OMNIROUTE_SKIP_DB_HEALTHCHECK=1
|
||||
@@ -882,7 +891,7 @@ NEXT_PUBLIC_ENABLE_SOCKS5_PROXY=true
|
||||
# MODEL_SYNC_INTERVAL_HOURS=24
|
||||
|
||||
# Provider limits sync interval in minutes (rate limit windows, quotas).
|
||||
# Used by: src/server-init.ts — polls provider health endpoints.
|
||||
# Used by: src/lib/usage/providerLimits.ts — polls provider health endpoints.
|
||||
# Default: 70
|
||||
PROVIDER_LIMITS_SYNC_INTERVAL_MINUTES=70
|
||||
|
||||
@@ -1352,6 +1361,14 @@ CURSOR_USER_AGENT="Cursor/3.4"
|
||||
# FETCH_BODY_TIMEOUT_MS=600000 # Time to receive full response body
|
||||
# FETCH_CONNECT_TIMEOUT_MS=30000 # TCP connection establishment (default: 30s)
|
||||
# FETCH_KEEPALIVE_TIMEOUT_MS=4000 # Keep-alive socket idle timeout (default: 4s)
|
||||
# OMNIROUTE_DIRECT_HEADERS_TIMEOUT_MS=30000 # Bounded response-start window per direct
|
||||
# # (no-proxy) attempt (#10214). A silently-dropped
|
||||
# # pooled keep-alive socket surfaces no transport
|
||||
# # error, so without this bound a direct request can
|
||||
# # stall until undici's headersTimeout (600s) or the
|
||||
# # caller's deadline; on expiry the request retries
|
||||
# # once on a fresh no-keep-alive socket. 0 disables
|
||||
# # the bound (default: 30000 = 30s).
|
||||
|
||||
# Default timeout (ms) for src/shared/utils/fetchTimeout.ts. Acts as the
|
||||
# fallback when FETCH_TIMEOUT_MS is unset. Default: 120000 (2 min).
|
||||
@@ -1406,6 +1423,14 @@ CURSOR_USER_AGENT="Cursor/3.4"
|
||||
# OMNIROUTE_PPLX_TLS_TIMEOUT_MS=30000
|
||||
# OMNIROUTE_PPLX_TLS_GRACE_MS=10000
|
||||
|
||||
# ── Perplexity web: built-in-search hint ──
|
||||
# Used by: open-sse/executors/perplexity-web/protocol.ts — appends "You have
|
||||
# built-in web search. Answer questions directly using search results." to the
|
||||
# caller's system message. Off by default: Perplexity's answer engine searches
|
||||
# anyway, and for coding clients the sentence leaks into replies as
|
||||
# meta-commentary. Set to 1/true/yes/on to restore the old behavior.
|
||||
# OMNIROUTE_PPLX_SEARCH_HINT=0
|
||||
|
||||
# ── Grok web TLS sidecar (Chrome-fingerprinted client) ──
|
||||
# Used by: open-sse/services/grokTlsClient.ts — wire-level timeout for the
|
||||
# bogdanfinn/tls-client koffi binding and the JS-side grace window layered on
|
||||
@@ -1979,6 +2004,16 @@ APP_LOG_TO_FILE=true
|
||||
# Reachability probe target for the scheduler and the auto-test endpoint.
|
||||
# Point it at an internal/self-hosted URL to avoid the public default.
|
||||
# PROXY_HEALTH_TEST_URL=https://httpbin.org/ip
|
||||
# Probes started at once per batch, for the scheduler and the auto-test endpoint.
|
||||
# Floored at 1 and capped at 50. Default: 10.
|
||||
# PROXY_HEALTH_TEST_CONCURRENCY=10
|
||||
# Delay in ms between two probe departures inside a batch. Without it the whole batch
|
||||
# leaves at once and a shared egress IP can trip a rate-limited target. 0 disables the
|
||||
# spacing; capped at 5000. Default: 100.
|
||||
# PROXY_HEALTH_TEST_STAGGER_MS=100
|
||||
# Set "false" to stop probing the real host of a proxy's assigned provider (GET /models,
|
||||
# no API key) and always use the generic target above instead. Default: enabled.
|
||||
# PROXY_HEALTH_USE_PROVIDER_TARGET=true
|
||||
# Set "true" to let the scheduler auto-remove proxies after repeated failures.
|
||||
# PROXY_AUTO_REMOVE=false
|
||||
# Consecutive failures before an auto-remove fires. Default: 3.
|
||||
|
||||
14
.github/dependabot.yml
vendored
@@ -50,13 +50,13 @@ updates:
|
||||
# bumps; majors here need their own PR and a deliberate migration review.
|
||||
- dependency-name: "ioredis"
|
||||
update-types: ["version-update:semver-major"]
|
||||
# @huggingface/transformers is HARD-PINNED at 3.5.2 (exact, no caret) — FROZEN.
|
||||
# It is load-bearing for the LLMLingua ONNX compression engine (open-sse/services/
|
||||
# compression/engines/llmlingua/ — worker.ts pins @huggingface/transformers@3.5.2)
|
||||
# and for local memory embeddings (src/lib/memory/embedding/transformersLocal.ts),
|
||||
# and was VPS-validated at 3.5.2 (#4014). 4.x breaks both, and even 3.x minors must
|
||||
# be re-validated on the VPS — so freeze ALL auto-bumps (no update-types = ignore
|
||||
# every version). Migrate it intentionally, not via dependabot (#4050).
|
||||
# @huggingface/transformers is VPS-validated at ^4.2.0 (migrated intentionally in
|
||||
# #9962). It is load-bearing for the LLMLingua ONNX compression engine (open-sse/
|
||||
# services/compression/engines/llmlingua/ — @atjsh/llmlingua-2@2.0.5 peers on
|
||||
# "@huggingface/transformers": "^3.5.2 || ^4.0.0") and for local memory embeddings
|
||||
# (src/lib/memory/embedding/transformersLocal.ts). Further majors must be re-validated
|
||||
# on the VPS — so keep auto-bumps frozen (no update-types = ignore every version).
|
||||
# Migrate it intentionally, not via dependabot (#4050).
|
||||
- dependency-name: "@huggingface/transformers"
|
||||
|
||||
- package-ecosystem: "github-actions"
|
||||
|
||||
2
.github/workflows/codeql.yml
vendored
@@ -26,6 +26,6 @@ jobs:
|
||||
with:
|
||||
languages: javascript-typescript
|
||||
queries: security-extended
|
||||
- uses: github/codeql-action/analyze@5595ccaf912efad79be6eef63a5619ff05969be3 # v4.37.6
|
||||
- uses: github/codeql-action/analyze@ff2f1c621b7f889edc0d3c761ac2e6a3f8cdb0dd # v4.37.7
|
||||
with:
|
||||
category: "/language:javascript-typescript"
|
||||
|
||||
64
.github/workflows/radar-export.yml
vendored
Normal file
@@ -0,0 +1,64 @@
|
||||
# Publica o export estável do catálogo consumido pelo OmniRoute Radar numa URL
|
||||
# fixa (asset de release `radar-export-latest`), para o servidor privado do Radar
|
||||
# (1 GB RAM, nunca clona/builda o OmniRoute) baixá-lo via `RADAR_EXPORT_URL` em
|
||||
# vez de depender do snapshot gravado no deploy. Fonte: scripts/release/radar-export.mjs.
|
||||
#
|
||||
# A URL estável resultante (definir em RADAR_EXPORT_URL no .env do radar-server):
|
||||
# https://github.com/diegosouzapw/OmniRoute/releases/download/radar-export-latest/export-omniroute.json
|
||||
name: Radar Export
|
||||
|
||||
on:
|
||||
workflow_dispatch: # o operador pode publicar sob demanda (de qualquer ref)
|
||||
push:
|
||||
branches: [main] # produção: só o catálogo do main clobra o asset estável
|
||||
paths:
|
||||
- open-sse/config/freeModelCatalog.data.ts
|
||||
- open-sse/config/freeModelCatalog.ts
|
||||
- open-sse/config/providerRegistry.ts
|
||||
- open-sse/config/providers/**
|
||||
- scripts/release/radar-export.mjs
|
||||
- .github/workflows/radar-export.yml
|
||||
schedule:
|
||||
- cron: "17 6 * * 1" # semanal (segunda 06:17 UTC): mantém geradoEm/proveniência frescos
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
concurrency:
|
||||
group: radar-export-${{ github.ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
env:
|
||||
CI_NODE_VERSION: "24"
|
||||
|
||||
jobs:
|
||||
publish-export:
|
||||
runs-on: ubuntu-latest
|
||||
permissions:
|
||||
contents: write # gh release upload — clobra o asset estável do export
|
||||
steps:
|
||||
- uses: actions/checkout@v7
|
||||
with:
|
||||
persist-credentials: false # publish usa GH_TOKEN via gh release, não a credencial do checkout
|
||||
- uses: actions/setup-node@v7
|
||||
with:
|
||||
node-version: ${{ env.CI_NODE_VERSION }}
|
||||
cache: npm
|
||||
- run: npm ci
|
||||
- name: Generate catalog export with provenance
|
||||
run: node --import tsx/esm scripts/release/radar-export.mjs "$RUNNER_TEMP/export-omniroute.json"
|
||||
- name: Publish to the stable release asset
|
||||
env:
|
||||
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
TAG="radar-export-latest"
|
||||
# Cria o release estável na primeira vez; nas seguintes só re-anexa o asset.
|
||||
if ! gh release view "$TAG" --repo "$GITHUB_REPOSITORY" >/dev/null 2>&1; then
|
||||
gh release create "$TAG" \
|
||||
--repo "$GITHUB_REPOSITORY" \
|
||||
--title "Radar catalog export (rolling)" \
|
||||
--notes "Export estável do catálogo OmniRoute para o Radar. Atualizado automaticamente; NÃO é um release de versão do produto." \
|
||||
--latest=false
|
||||
fi
|
||||
gh release upload "$TAG" "$RUNNER_TEMP/export-omniroute.json" --repo "$GITHUB_REPOSITORY" --clobber
|
||||
@@ -1294,10 +1294,15 @@ export function mapRawModelToModelV2(
|
||||
// `(providerID, modelID)`. If the raw id is already provider-prefixed
|
||||
// (e.g. `cc/claude-opus-4-7` from the `cc` Claude Code alias, or
|
||||
// `nvidia/llama-3-70b` from a provider that ships prefixed ids), leave
|
||||
// it as-is — double-prefixing breaks OC's lookup. Otherwise prefix with
|
||||
// the resolved `providerId` so a bare key like `claude-opus-4` parses as
|
||||
// `(omniroute, claude-opus-4)` and the credentials resolve correctly.
|
||||
id: raw.id.includes("/") ? raw.id : `${ctx.providerId}/${raw.id}`,
|
||||
// it as-is — double-prefixing breaks OC's lookup. Bare **combo** ids
|
||||
// (`owned_by: "combo"`, e.g. `gpt-5.6-sol`) must also stay unprefixed:
|
||||
// OpenCode looks up `-m <plugin>/<combo>` as model id `<combo>` under
|
||||
// the plugin provider (#10345). Other bare ids still prefix with
|
||||
// `providerId` so credentials resolve as `(omniroute, model)`.
|
||||
id:
|
||||
raw.id.includes("/") || raw.owned_by === "combo"
|
||||
? raw.id
|
||||
: `${ctx.providerId}/${raw.id}`,
|
||||
/**
|
||||
* Display name. Falls back to raw.id when no enrichment is available;
|
||||
* the caller (`createOmniRouteProviderHook`) overlays
|
||||
|
||||
@@ -0,0 +1,34 @@
|
||||
import test from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
|
||||
import { mapRawModelToModelV2 } from "../src/index.ts";
|
||||
|
||||
test("mapRawModelToModelV2: bare combo ids stay unprefixed (#10345)", () => {
|
||||
const combo = mapRawModelToModelV2(
|
||||
{
|
||||
id: "gpt-5.6-sol",
|
||||
owned_by: "combo",
|
||||
context_length: 272000,
|
||||
max_output_tokens: 8192,
|
||||
},
|
||||
{ providerId: "omniroute", baseURL: "https://or.example.com/v1" }
|
||||
);
|
||||
assert.equal(combo.id, "gpt-5.6-sol");
|
||||
assert.equal(combo.providerID, "omniroute");
|
||||
|
||||
const slashed = mapRawModelToModelV2(
|
||||
{
|
||||
id: "cx/gpt-5.6-sol",
|
||||
owned_by: "combo",
|
||||
context_length: 272000,
|
||||
},
|
||||
{ providerId: "omniroute", baseURL: "https://or.example.com/v1" }
|
||||
);
|
||||
assert.equal(slashed.id, "cx/gpt-5.6-sol");
|
||||
|
||||
const ordinary = mapRawModelToModelV2(
|
||||
{ id: "claude-primary", context_length: 200000 },
|
||||
{ providerId: "omniroute", baseURL: "https://or.example.com/v1" }
|
||||
);
|
||||
assert.equal(ordinary.id, "omniroute/claude-primary");
|
||||
});
|
||||
@@ -46,7 +46,7 @@ Repository map and Reference Documentation sections below.
|
||||
|
||||
## Project at a Glance
|
||||
|
||||
**OmniRoute** — unified AI proxy/router. One endpoint, 342 LLM providers, auto-fallback.
|
||||
**OmniRoute** — unified AI proxy/router. One endpoint, 346 LLM providers, auto-fallback.
|
||||
|
||||
| Layer | Location | Purpose |
|
||||
| ------------- | ----------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
|
||||
@@ -173,7 +173,7 @@ COPY . ./
|
||||
RUN --mount=type=cache,id=s/92ca8a61-c1ba-421f-a389-d48ac7258c2d-next-cache,target=/app/.build/next/cache \
|
||||
mkdir -p /app/data \
|
||||
&& npm run build \
|
||||
&& node --input-type=module -e "import { createRequire } from 'node:module'; import { pathToFileURL } from 'node:url'; const standaloneRoot = '/app/.build/next/standalone/node_modules/'; const require = createRequire('/app/.build/next/standalone/package.json'); for (const pkg of ['@atjsh/llmlingua-2', '@huggingface/transformers', '@tensorflow/tfjs', 'js-tiktoken']) { const resolved = require.resolve(pkg); if (!resolved.startsWith(standaloneRoot)) throw new Error(pkg + ' resolved outside standalone: ' + resolved); await import(pathToFileURL(resolved).href); } const onnxRuntime = require.resolve('onnxruntime-node'); if (!onnxRuntime.startsWith(standaloneRoot)) throw new Error('onnxruntime-node resolved outside standalone: ' + onnxRuntime); await import(pathToFileURL(onnxRuntime).href);"
|
||||
&& node --input-type=module -e "import { createRequire } from 'node:module'; import { pathToFileURL } from 'node:url'; const standaloneRoot = '/app/.build/next/standalone/node_modules/'; const require = createRequire('/app/.build/next/standalone/package.json'); for (const pkg of ['@atjsh/llmlingua-2', '@huggingface/transformers', 'js-tiktoken']) { const resolved = require.resolve(pkg); if (!resolved.startsWith(standaloneRoot)) throw new Error(pkg + ' resolved outside standalone: ' + resolved); await import(pathToFileURL(resolved).href); } const onnxRuntime = require.resolve('onnxruntime-node'); if (!onnxRuntime.startsWith(standaloneRoot)) throw new Error('onnxruntime-node resolved outside standalone: ' + onnxRuntime); await import(pathToFileURL(onnxRuntime).href);"
|
||||
|
||||
# ── Runner base ────────────────────────────────────────────────────────────
|
||||
FROM base AS runner-base
|
||||
|
||||
16
README.md
@@ -7,7 +7,7 @@
|
||||
|
||||
# 🚀 OmniRoute — The Free AI Gateway
|
||||
|
||||
<img src="./docs/diagrams/readme-hero.svg" width="100%" alt="OmniRoute — Never stop coding. Every AI tool → 342 providers — 90+ free — through one endpoint. Claude Code, Codex, Cursor, Cline, Copilot & Antigravity into FREE Claude / GPT / Gemini with auto-fallback. RTK + Caveman stacked compression saves 15–95% tokens (~89% avg) — never hit limits. 342 AI providers · 90+ free tiers · ~1.51B free tokens/mo · 19 routing strategies · $0 to start."/>
|
||||
<img src="./docs/diagrams/readme-hero.svg" width="100%" alt="OmniRoute — Never stop coding. Every AI tool → 346 providers — 90+ free — through one endpoint. Claude Code, Codex, Cursor, Cline, Copilot & Antigravity into FREE Claude / GPT / Gemini with auto-fallback. RTK + Caveman stacked compression saves 15–95% tokens (~89% avg) — never hit limits. 346 AI providers · 90+ free tiers · ~1.51B free tokens/mo · 19 routing strategies · $0 to start."/>
|
||||
|
||||
</div>
|
||||
|
||||
@@ -101,7 +101,7 @@
|
||||
<tr>
|
||||
<td align="right"><b>⚙️ Features</b></td>
|
||||
<td align="center"><a href="#-combos--the-flagship">🎯 Combos</a></td>
|
||||
<td align="center"><a href="#-342-ai-providers--90-free">🌐 Providers</a></td>
|
||||
<td align="center"><a href="#-346-ai-providers--90-free">🌐 Providers</a></td>
|
||||
<td align="center"><a href="#-full-cli--a2a--mcp">🔌 CLI & MCP</a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
@@ -210,7 +210,7 @@ curl http://localhost:20128/v1/chat/completions \
|
||||
|
||||
</div>
|
||||
|
||||
<img src="./docs/diagrams/promise-pillars.svg" width="100%" alt="The Promise — One endpoint. 342 providers. Never stop building — OmniRoute picks the cheapest one that works. Six pillars: Never hit limits (auto-fallback across 342 providers in milliseconds, zero downtime) · Save up to 95% tokens (RTK + Caveman stacked compression cuts 15–95%, ~89% avg on tool-heavy sessions) · $0 to start (90+ free tiers, 56 free forever — no card needed) · Every tool works (33 coding agents through one config) · One endpoint (OpenAI ↔ Claude ↔ Gemini ↔ Responses API at /v1) · Production-grade (circuit breakers, TLS stealth, MCP 109 tools, A2A, memory, guardrails, evals — 25,000+ tests)."/>
|
||||
<img src="./docs/diagrams/promise-pillars.svg" width="100%" alt="The Promise — One endpoint. 346 providers. Never stop building — OmniRoute picks the cheapest one that works. Six pillars: Never hit limits (auto-fallback across 346 providers in milliseconds, zero downtime) · Save up to 95% tokens (RTK + Caveman stacked compression cuts 15–95%, ~89% avg on tool-heavy sessions) · $0 to start (90+ free tiers, 56 free forever — no card needed) · Every tool works (33 coding agents through one config) · One endpoint (OpenAI ↔ Claude ↔ Gemini ↔ Responses API at /v1) · Production-grade (circuit breakers, TLS stealth, MCP 109 tools, A2A, memory, guardrails, evals — 25,000+ tests)."/>
|
||||
|
||||
<br/>
|
||||
<br/>
|
||||
@@ -461,7 +461,7 @@ All **19** strategies — mix & match per combo step:
|
||||
|
||||
</div>
|
||||
|
||||
<img src="./docs/diagrams/comparison-table.svg" width="100%" alt="What sets OmniRoute apart — comparison table vs 9router, OpenRouter, CLIProxyAPI and LiteLLM across 13 capabilities. OmniRoute: 342 providers, 90+ free providers built-in, 19 routing strategies, 12-engine token compression, built-in MCP server with 109 tools, A2A agent protocol, persistent memory, guardrails, cloud agents, TLS fingerprint stealth, Desktop/Termux/PWA, 43 i18n UI locales, 100% MIT self-hosted. OmniRoute is the only one with the full set; competitors show a mix of checks, partials and crosses. Verified from each project's docs."/>
|
||||
<img src="./docs/diagrams/comparison-table.svg" width="100%" alt="What sets OmniRoute apart — comparison table vs 9router, OpenRouter, CLIProxyAPI and LiteLLM across 13 capabilities. OmniRoute: 346 providers, 90+ free providers built-in, 19 routing strategies, 12-engine token compression, built-in MCP server with 109 tools, A2A agent protocol, persistent memory, guardrails, cloud agents, TLS fingerprint stealth, Desktop/Termux/PWA, 43 i18n UI locales, 100% MIT self-hosted. OmniRoute is the only one with the full set; competitors show a mix of checks, partials and crosses. Verified from each project's docs."/>
|
||||
|
||||
<sub>📊 Full methodology & per-feature detail vs 9router, OpenRouter, CLIProxyAPI & LiteLLM → [`docs/comparison/OMNIROUTE_VS_ALTERNATIVES.md`](docs/comparison/OMNIROUTE_VS_ALTERNATIVES.md)</sub>
|
||||
|
||||
@@ -559,7 +559,7 @@ the current catalog at **[radar.omniroute.online/planos](https://radar.omniroute
|
||||
- **🖼️ New endpoints** — `/v1/ocr` (Mistral OCR) and `/v1/audio/translations` (Whisper-style) round out the media surface. → [API Reference](docs/reference/API_REFERENCE.md)
|
||||
- **🎨 Image / video / audio generation** — one API for media: xAI Grok Imagine & Novita AI video, ComfyUI, Freepik, Adobe Firefly, Microsoft Designer, Segmind, EdgeTTS. → [API Reference](docs/reference/API_REFERENCE.md)
|
||||
- **🌍 Deployment & ops** — reverse-proxy `basePath`, browser-language auto-detect, per-key device tracking, root-less MITM trust, zh-TW localization. → [Environment](docs/reference/ENVIRONMENT.md)
|
||||
- **🤝 More providers & agents** — Cursor Cloud Agent, Grok Build (xAI) with browser + OAuth login, Ollama first-class card, Claude Opus 5 & Sonnet 5, Kimi official partnership (Code/Web/Moonshot), Zed, Requesty, SenseNova, Yuanbao, Agnes AI… and a refreshed **342-provider catalog**. → [Providers](docs/reference/PROVIDER_REFERENCE.md)
|
||||
- **🤝 More providers & agents** — Cursor Cloud Agent, Grok Build (xAI) with browser + OAuth login, Ollama first-class card, Claude Opus 5 & Sonnet 5, Kimi official partnership (Code/Web/Moonshot), Zed, Requesty, SenseNova, Yuanbao, Agnes AI… and a refreshed **346-provider catalog**. → [Providers](docs/reference/PROVIDER_REFERENCE.md)
|
||||
- **📡 Routing transparency** — every response carries an `X-OmniRoute-Decision` header naming the strategy/provider/latency that served it, a new `cache-optimized` combo strategy + Auto-Combo `cacheAffinity` factor route repeat requests back to the connection holding the cached prefix, and a read-only `/v1/auto-combo/{channel}/candidates` endpoint exposes an `auto/*` channel's live candidate pool. → [Auto-Combo](docs/routing/AUTO-COMBO.md)
|
||||
- **⚡ Local performance & infra** — one-click local Redis, Cloudflare Workers / Deno Deploy relay deployers, Bifrost & Mux as supervised embedded services. → [Embedded Services](docs/frameworks/EMBEDDED-SERVICES.md)
|
||||
|
||||
@@ -642,11 +642,11 @@ of your shell history. → [CLI Integrations](docs/guides/CLI-INTEGRATIONS.md)
|
||||
|
||||
<div align="center">
|
||||
|
||||
## 🌐 342 AI Providers — 90+ Free
|
||||
## 🌐 346 AI Providers — 90+ Free
|
||||
|
||||
</div>
|
||||
|
||||
> The most complete catalog of any open-source router: **342 providers**, **90+ with a free tier**, **56 free forever**.
|
||||
> The most complete catalog of any open-source router: **346 providers**, **90+ with a free tier**, **56 free forever**.
|
||||
|
||||
<div align="center">
|
||||
|
||||
@@ -988,6 +988,8 @@ docker run -d --name omniroute --restart unless-stopped --stop-timeout 40 \
|
||||
-p 127.0.0.1:20128:20128 -v omniroute-data:/app/data diegosouzapw/omniroute:latest
|
||||
```
|
||||
|
||||
`:latest` follows the highest **published** stable SemVer. It does not track git `main`. Pin `:X.Y.Z` for GitOps. See [Docker Release Channels](docs/guides/DOCKER_GUIDE.md#release-channels).
|
||||
|
||||
> **Pre-release Docker channel:** `diegosouzapw/omniroute:next` and
|
||||
> `diegosouzapw/omniroute:next-web` follow the current default `release/v*`
|
||||
> branch. These mutable tags are intended only for testing unreleased fixes and
|
||||
|
||||
@@ -52,6 +52,19 @@ function resolveUrl(path, opts) {
|
||||
return `${getBaseUrl(opts)}${path.startsWith("/") ? path : `/${path}`}`;
|
||||
}
|
||||
|
||||
/** The machine-derived token is valid only for the local loopback server. */
|
||||
export function isLoopbackUrl(value) {
|
||||
try {
|
||||
const hostname = new URL(value).hostname.replace(/^\[|\]$/g, "").toLowerCase();
|
||||
if (hostname === "localhost" || hostname === "::1") return true;
|
||||
if (/^127(?:\.[0-9]{1,3}){3}$/.test(hostname)) return true;
|
||||
if (/^::ffff:(?:127\.|7f[0-9a-f]{2}:)/i.test(hostname)) return true;
|
||||
return false;
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
export async function buildHeaders(opts) {
|
||||
const headers = new Headers(opts.headers || {});
|
||||
if (!headers.has("accept")) headers.set("accept", "application/json");
|
||||
@@ -87,10 +100,17 @@ export async function buildHeaders(opts) {
|
||||
if (auth && !headers.has("authorization")) {
|
||||
headers.set("authorization", `Bearer ${auth}`);
|
||||
}
|
||||
// Inject machine-id derived CLI token; env var override for testing.
|
||||
const cliToken = opts.cliToken ?? process.env.OMNIROUTE_CLI_TOKEN ?? (await getCliToken());
|
||||
if (cliToken && !headers.has(CLI_TOKEN_HEADER)) {
|
||||
headers.set(CLI_TOKEN_HEADER, cliToken);
|
||||
// Inject the machine-derived credential only for an explicit local loopback
|
||||
// destination. Remote contexts and absolute remote URLs use scoped access
|
||||
// tokens and must never receive this machine-bound local credential.
|
||||
const destinationUrl = opts.destinationUrl ?? getBaseUrl(opts);
|
||||
if (!isLoopbackUrl(destinationUrl)) {
|
||||
headers.delete(CLI_TOKEN_HEADER);
|
||||
} else {
|
||||
const cliToken = opts.cliToken ?? process.env.OMNIROUTE_CLI_TOKEN ?? (await getCliToken());
|
||||
if (cliToken && !headers.has(CLI_TOKEN_HEADER)) {
|
||||
headers.set(CLI_TOKEN_HEADER, cliToken);
|
||||
}
|
||||
}
|
||||
if (opts.idempotencyKey && !headers.has("idempotency-key")) {
|
||||
headers.set("idempotency-key", opts.idempotencyKey);
|
||||
@@ -195,8 +215,12 @@ function fetchOnce(url, init, timeoutMs) {
|
||||
export async function apiFetch(path, opts = {}) {
|
||||
const method = String(opts.method || "GET").toUpperCase();
|
||||
const url = resolveUrl(path, opts);
|
||||
const headers = await buildHeaders(opts);
|
||||
const headers = await buildHeaders({ ...opts, destinationUrl: url });
|
||||
const body = serializeBody(opts.body, headers);
|
||||
// Undici preserves custom headers across cross-origin redirects. A local server
|
||||
// redirect must never turn the loopback machine credential into an outbound
|
||||
// secret, so fail redirects whenever this header is present.
|
||||
const redirect = headers.has(CLI_TOKEN_HEADER) ? "error" : opts.redirect;
|
||||
const timeout =
|
||||
opts.timeout ?? (Number.parseInt(process.env.OMNIROUTE_HTTP_TIMEOUT_MS || "", 10) || 30000);
|
||||
const maxAttempts = opts.retry === false ? 1 : (opts.retryMax ?? RETRY_DEFAULTS.maxAttempts);
|
||||
@@ -205,7 +229,7 @@ export async function apiFetch(path, opts = {}) {
|
||||
let lastErr;
|
||||
for (let attempt = 1; attempt <= maxAttempts; attempt++) {
|
||||
try {
|
||||
const res = await fetchOnce(url, { method, headers, body }, timeout);
|
||||
const res = await fetchOnce(url, { method, headers, body, redirect }, timeout);
|
||||
if (res.ok) return enrichResponse(res, opts);
|
||||
if (attempt < maxAttempts && shouldRetryStatus(res.status, method, opts)) {
|
||||
const delay = computeBackoff(attempt, res.headers.get("retry-after"));
|
||||
|
||||
@@ -4,7 +4,9 @@ import os from "node:os";
|
||||
import path from "node:path";
|
||||
import { createDecipheriv, scryptSync } from "node:crypto";
|
||||
import { fileURLToPath, pathToFileURL } from "node:url";
|
||||
import { isLoopbackUrl } from "../api.mjs";
|
||||
import { resolveDataDir, resolveStoragePath } from "../data-dir.mjs";
|
||||
import { getCliToken, CLI_TOKEN_HEADER } from "../utils/cliToken.mjs";
|
||||
import { printHeading } from "../io.mjs";
|
||||
import { t } from "../i18n.mjs";
|
||||
import { readDatabaseHealth, readEncryptedCredentialSamples } from "../sqlite.mjs";
|
||||
@@ -378,11 +380,11 @@ function checkMemory() {
|
||||
});
|
||||
}
|
||||
|
||||
async function fetchWithTimeout(url) {
|
||||
async function fetchWithTimeout(url, options = {}) {
|
||||
const controller = new AbortController();
|
||||
const timeout = setTimeout(() => controller.abort(), CHECK_TIMEOUT_MS);
|
||||
try {
|
||||
return await fetch(url, { signal: controller.signal });
|
||||
return await fetch(url, { ...options, signal: controller.signal });
|
||||
} finally {
|
||||
clearTimeout(timeout);
|
||||
}
|
||||
@@ -471,6 +473,98 @@ async function checkServerLiveness(options = {}) {
|
||||
);
|
||||
}
|
||||
|
||||
export async function checkMachineTokenAuth(options = {}) {
|
||||
if (process.env.OMNIROUTE_DISABLE_CLI_TOKEN === "true") {
|
||||
return warn("CLI machine token", "CLI machine-token authentication is disabled", {
|
||||
derived: false,
|
||||
accepted: false,
|
||||
disabled: true,
|
||||
tokenExposed: false,
|
||||
});
|
||||
}
|
||||
|
||||
let url;
|
||||
try {
|
||||
const parsed = new URL(resolveLivenessUrl(options));
|
||||
if (
|
||||
!["http:", "https:"].includes(parsed.protocol) ||
|
||||
parsed.username ||
|
||||
parsed.password ||
|
||||
!isLoopbackUrl(parsed.toString())
|
||||
) {
|
||||
return warn(
|
||||
"CLI machine token",
|
||||
"Machine-token probes are limited to HTTP(S) loopback endpoints",
|
||||
{ derived: false, accepted: false, tokenExposed: false }
|
||||
);
|
||||
}
|
||||
parsed.pathname = "/api/cli/whoami";
|
||||
parsed.search = "";
|
||||
parsed.hash = "";
|
||||
url = parsed.toString();
|
||||
} catch {
|
||||
return warn("CLI machine token", "Could not resolve the management endpoint", {
|
||||
derived: false,
|
||||
accepted: false,
|
||||
tokenExposed: false,
|
||||
});
|
||||
}
|
||||
|
||||
const token = await getCliToken();
|
||||
if (!token) {
|
||||
return fail(
|
||||
"CLI machine token",
|
||||
"Could not derive a machine token; verify the node-machine-id runtime is installed",
|
||||
{ derived: false, accepted: false, tokenExposed: false }
|
||||
);
|
||||
}
|
||||
|
||||
try {
|
||||
const response = await fetchWithTimeout(url, {
|
||||
headers: { [CLI_TOKEN_HEADER]: token },
|
||||
redirect: "error",
|
||||
});
|
||||
if (response.ok) {
|
||||
return ok("CLI machine token", "Server accepted the local machine token", {
|
||||
url,
|
||||
status: response.status,
|
||||
derived: true,
|
||||
accepted: true,
|
||||
tokenExposed: false,
|
||||
});
|
||||
}
|
||||
if (response.status === 401 || response.status === 403) {
|
||||
return warn(
|
||||
"CLI machine token",
|
||||
"Server rejected the local machine token; if the CLI and server are on different hosts or container boundaries, run `omniroute connect <host> --key <oma_live_...>`",
|
||||
{
|
||||
url,
|
||||
status: response.status,
|
||||
derived: true,
|
||||
accepted: false,
|
||||
containerBoundaryLikely: true,
|
||||
tokenExposed: false,
|
||||
}
|
||||
);
|
||||
}
|
||||
return warn("CLI machine token", `Machine-token probe returned HTTP ${response.status}`, {
|
||||
url,
|
||||
status: response.status,
|
||||
derived: true,
|
||||
accepted: false,
|
||||
tokenExposed: false,
|
||||
});
|
||||
} catch {
|
||||
return warn("CLI machine token", "Machine-token endpoint could not be reached", {
|
||||
url,
|
||||
status: 0,
|
||||
derived: true,
|
||||
accepted: false,
|
||||
tokenExposed: false,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
export async function collectDoctorChecks(context = {}, options = {}) {
|
||||
const rootDir =
|
||||
context.rootDir || path.resolve(path.dirname(fileURLToPath(import.meta.url)), "..", "..", "..");
|
||||
@@ -488,6 +582,7 @@ export async function collectDoctorChecks(context = {}, options = {}) {
|
||||
|
||||
if (!options.skipLiveness) {
|
||||
checks.push(await checkServerLiveness(options));
|
||||
checks.push(await checkMachineTokenAuth(options));
|
||||
}
|
||||
|
||||
// CLI tool health checks
|
||||
|
||||
@@ -121,7 +121,16 @@ function writeLinuxSystemdUnit(cliPath) {
|
||||
"Wants=network-online.target",
|
||||
"",
|
||||
"[Service]",
|
||||
"Type=simple",
|
||||
// Type=notify + WatchdogSec: the server sends READY=1 once listening and
|
||||
// WATCHDOG=1 every 60s; if its event loop ever blocks (frozen process),
|
||||
// the pings stop and systemd kills+restarts the service. NotifyAccess=all
|
||||
// because the pings come from the server child, not the serve supervisor.
|
||||
// Foreground serve only: `--daemon` escapes the cgroup and would break
|
||||
// the notify handshake.
|
||||
"Type=notify",
|
||||
"NotifyAccess=all",
|
||||
"WatchdogSec=180",
|
||||
"TimeoutStartSec=300",
|
||||
`ExecStart=${buildServeExecLine(cliPath, { tray: false })}`,
|
||||
"Restart=on-failure",
|
||||
"RestartSec=5",
|
||||
|
||||
@@ -12,25 +12,39 @@ function getActiveSalt() {
|
||||
return process.env.OMNIROUTE_CLI_SALT || BUILTIN_DEFAULT_SALT;
|
||||
}
|
||||
|
||||
export async function getCliToken() {
|
||||
const salt = getActiveSalt();
|
||||
if (_cached !== null && _cachedSalt === salt) return _cached;
|
||||
export function deriveCliToken(machineIdModule, salt) {
|
||||
try {
|
||||
// node-machine-id is CommonJS: under `await import()` its exports land on
|
||||
// `.default`, so destructuring `machineIdSync` off the namespace yields
|
||||
// undefined and calling it throws — which the catch below turned into an
|
||||
// empty token, silently disabling CLI auth for every management request.
|
||||
// Same resolution order as src/lib/machineToken.ts.
|
||||
const mod = await import("node-machine-id");
|
||||
const machineIdSync = mod.machineIdSync ?? mod.default?.machineIdSync;
|
||||
if (typeof machineIdSync !== "function") throw new Error("machine-id API unavailable");
|
||||
const machineIdSync =
|
||||
machineIdModule?.machineIdSync || machineIdModule?.default?.machineIdSync;
|
||||
if (typeof machineIdSync !== "function") return "";
|
||||
// machineIdSync(true) returns the original unhashed hardware ID — mirrors
|
||||
// getMachineTokenSync() in src/lib/machineToken.ts (#10148 cliToken hardening).
|
||||
const mid = machineIdSync(true);
|
||||
_cached = crypto.createHmac("sha256", mid).update(salt).digest("hex");
|
||||
const rawId = machineIdSync(true);
|
||||
if (!rawId) return "";
|
||||
return crypto.createHmac("sha256", rawId).update(salt).digest("hex");
|
||||
} catch {
|
||||
return "";
|
||||
}
|
||||
}
|
||||
|
||||
export async function getCliToken() {
|
||||
const salt = getActiveSalt();
|
||||
if (_cached !== null && _cachedSalt === salt) return _cached;
|
||||
try {
|
||||
const imported = await import("node-machine-id");
|
||||
const token = deriveCliToken(imported, salt);
|
||||
if (!token) {
|
||||
// Swallowing here changes control flow (every management call goes out
|
||||
// unauthenticated and 401s), so leave a breadcrumb rather than failing mute.
|
||||
console.debug("[CLI_TOKEN] machine-id resolution failed, CLI auth disabled");
|
||||
}
|
||||
_cached = token;
|
||||
} catch (e) {
|
||||
// Swallowing here changes control flow (every management call goes out
|
||||
// unauthenticated and 401s), so leave a breadcrumb rather than failing mute.
|
||||
console.debug("[CLI_TOKEN] machine-id resolution failed, CLI auth disabled:", e);
|
||||
_cached = "";
|
||||
}
|
||||
|
||||
@@ -119,6 +119,9 @@ function loadEnvFile() {
|
||||
addEnvPath(join(ROOT, ".env"));
|
||||
}
|
||||
|
||||
const keyOrigin = new Map();
|
||||
const shadowed = new Map();
|
||||
|
||||
for (const envPath of envPaths) {
|
||||
try {
|
||||
if (existsSync(envPath)) {
|
||||
@@ -131,19 +134,31 @@ function loadEnvFile() {
|
||||
const key = trimmed.slice(0, eqIdx).trim();
|
||||
if (process.env[key] === undefined) {
|
||||
process.env[key] = parseEnvValue(trimmed.slice(eqIdx + 1));
|
||||
keyOrigin.set(key, envPath);
|
||||
} else if (!shadowed.has(key)) {
|
||||
// The line is inert: something set this key first. Report it once
|
||||
// per key, whether the winner was an earlier file or the process
|
||||
// environment (#6194: a shell's own HOSTNAME beat the .env and the
|
||||
// server bound to the wrong address in silence).
|
||||
shadowed.set(key, { winner: keyOrigin.get(key) ?? null, loser: envPath });
|
||||
}
|
||||
}
|
||||
}
|
||||
loadedEnvPaths.push(envPath);
|
||||
}
|
||||
} catch {
|
||||
// Ignore errors reading env files.
|
||||
} catch (err) {
|
||||
console.warn(` \x1b[33m⚠ Could not read ${envPath}: ${err?.message ?? err}\x1b[0m`);
|
||||
}
|
||||
}
|
||||
|
||||
for (const envPath of loadedEnvPaths) {
|
||||
console.log(` \x1b[2m📋 Loaded env from ${envPath}\x1b[0m`);
|
||||
}
|
||||
|
||||
for (const [key, { winner, loser }] of shadowed) {
|
||||
const setter = winner ? winner : "the environment";
|
||||
console.warn(` \x1b[33m⚠ ${key} in ${loser} is ignored, ${setter} set it first\x1b[0m`);
|
||||
}
|
||||
}
|
||||
|
||||
loadEnvFile();
|
||||
|
||||
1
changelog.d/features/10303-healthz-event-loop-lag.md
Normal file
@@ -0,0 +1 @@
|
||||
- **feat(resilience):** warn when `/healthz` is served under event-loop lag ≥200ms so a slow 200 is visible as sick, not healthy ([#10303](https://github.com/diegosouzapw/OmniRoute/issues/10303))
|
||||
1
changelog.d/features/10316-livez-endpoint.md
Normal file
@@ -0,0 +1 @@
|
||||
- **feat(docker):** add `GET`/`HEAD` `/livez` as a process-alive probe, distinct from `/healthz` readiness ([#10316](https://github.com/diegosouzapw/OmniRoute/issues/10316))
|
||||
1
changelog.d/features/10587-ogg-speech-alias.md
Normal file
@@ -0,0 +1 @@
|
||||
- **feat(providers):** accept `response_format=ogg` on `/v1/audio/speech` as an alias for the existing Opus/Ogg encoder ([#10587](https://github.com/diegosouzapw/OmniRoute/issues/10587))
|
||||
1
changelog.d/features/10662-systemd-notify.md
Normal file
@@ -0,0 +1 @@
|
||||
- feat(server): emit systemd sd_notify READY/WATCHDOG/STOPPING (generated unit becomes Type=notify with WatchdogSec=180) so a frozen server process is killed and restarted by systemd instead of lingering undetected
|
||||
2
changelog.d/features/10668-newapi-gateway-protocols.md
Normal file
@@ -0,0 +1,2 @@
|
||||
- **feat(providers):** add the TabiToken NewAPI gateway (`tabitoken`) and teach the existing HCNSec entry (`hcnsec`) the three further protocols it actually serves. TabiToken leaves the NewAPI pricing endpoint public, so its catalog is read from the host rather than guessed: four Claude models, each reporting the Anthropic and OpenAI protocols. HCNSec shipped OpenAI-only; probing the host showed `/v1/messages`, `/v1/responses` and the Gemini `/v1beta` path all reach its token layer, so each is now declared as an alternate format — with its default format, base URL, auth scheme and regional catalog classification untouched. ([#10668](https://github.com/diegosouzapw/OmniRoute/pull/10668)) — thanks @yawar-aquil
|
||||
- **feat(sse):** allow an alternate protocol to build its own upstream URL. `AlternateFormat` gained an optional `urlBuilder`, because the Gemini protocol carries the model inside the path (`{base}/{model}:generateContent`) and the existing `chatPath`/`urlSuffix` fields are constants that cannot express it. The route builder is extracted as `buildGeminiGenerateContentUrl` and shared with the native `gemini` provider so the two consumers cannot drift on the `?alt=sse` streaming suffix. ([#10668](https://github.com/diegosouzapw/OmniRoute/pull/10668)) — thanks @yawar-aquil
|
||||
1
changelog.d/features/10670-call-logs-error-type.md
Normal file
@@ -0,0 +1 @@
|
||||
- **feat(call_logs):** persist the per-call error family in `call_logs.error_type` and expose a failure breakdown (`errorBreakdown`) in the usage analytics endpoint, reusing the existing production classifier ([#10670](https://github.com/diegosouzapw/OmniRoute/issues/10670))
|
||||
1
changelog.d/features/10677-egress-sharing-summary.md
Normal file
@@ -0,0 +1 @@
|
||||
- **feat(proxy):** the proxy-health sweep and `GET /api/settings/proxies/egress` now report an anonymous summary of egress-IP sharing — how many rotation groups share an egress IP and the largest number of accounts behind one IP — computed from persisted `proxy_logs` over a 24h window. No IPs and no account identities by default; `PROXY_LOG_INCLUDE_IPS=true` restores raw details. ([#10677](https://github.com/diegosouzapw/OmniRoute/issues/10677))
|
||||
1
changelog.d/features/10869-combo-patch-verb.md
Normal file
@@ -0,0 +1 @@
|
||||
- feat(api): accept PATCH on /api/combos/[id], the verb the OpenAPI spec already documents (#10869)
|
||||
1
changelog.d/features/10896-glm-5.3.md
Normal file
@@ -0,0 +1 @@
|
||||
- **feat(sse):** add GLM-5.3 support (`glm-5.3`, `glm-5.3-high`, `glm-5.3-low`) across the z.ai first-party providers, mapping the upstream `reasoning_effort` request parameter to the existing 5.2 tier UX ([#10896](https://github.com/diegosouzapw/OmniRoute/pull/10896)) — thanks @phuongddx
|
||||
1
changelog.d/features/10897-home-recent-requests.md
Normal file
@@ -0,0 +1 @@
|
||||
- **feat(home):** add a live **Recent Requests** panel beside the home Provider Topology (polls `GET /api/usage/call-logs?excludeTests=1` every ~3s, gated by the topology appearance toggle + page visibility). `excludeTests` is now an allowlist of real provider inference (`/v1/%` or `/api/v1/%`), applied before `LIMIT`, so connection-test/model-sync/management rows can never leak into the feed ([#10897](https://github.com/diegosouzapw/OmniRoute/pull/10897), extracted from [#8450](https://github.com/diegosouzapw/OmniRoute/pull/8450)) — thanks @nguyenha935
|
||||
@@ -0,0 +1,2 @@
|
||||
- **feat(credential-health):** pace the credential health sweep per connection via `provider_connections.healthCheckInterval` (minutes, 0 = never), with `CREDENTIAL_HEALTH_CHECK_INTERVAL` as the global default ([#8443](https://github.com/diegosouzapw/OmniRoute/issues/8443))
|
||||
- **behavior change:** `healthCheckInterval` is a shared column — it paces both the OAuth token refresh and the credential health sweep, and `0` disables both. The connection editor defaults it to 60, so configured OAuth connections are now credential-checked at 60min instead of the previous ~10min (aligned with the probe-volume goal of #8443)
|
||||
1
changelog.d/features/cursor-agent-image-provider.md
Normal file
@@ -0,0 +1 @@
|
||||
- feat(sse): add Cursor plan image generation via Agent CLI (`IMAGE_PROVIDERS.cursor`, format `cursor-agent-image`), reusing the chat Cursor OAuth connection
|
||||
@@ -0,0 +1 @@
|
||||
- fix(domain): stop treating an unreported Antigravity quota fraction (`fractionReported:false`) as 0% remaining in `quotaCache.ts`, which was falsely marking every fresh/newly-connected account as exhausted and blocking multi-account rotation (#10095)
|
||||
@@ -0,0 +1 @@
|
||||
- **fix(sse):** Responses-passthrough `response.completed` snapshots now drop `phase:"commentary"` items the same way live SSE frames already do, so the terminal `response.output` array no longer echoes internal commentary text that was already suppressed from the stream (#10156).
|
||||
@@ -0,0 +1 @@
|
||||
- **fix(routing):** keep approximate Combo context estimates advisory so requests reach concrete targets instead of returning a pre-dispatch 400 ([#10162](https://github.com/diegosouzapw/OmniRoute/pull/10162)) — thanks @xz-dev
|
||||
1
changelog.d/fixes/10345-bare-combo-opencode-ids.md
Normal file
@@ -0,0 +1 @@
|
||||
- **fix(opencode-plugin):** publish bare combo model ids without the plugin provider prefix so OpenCode can select them ([#10345](https://github.com/diegosouzapw/OmniRoute/issues/10345))
|
||||
1
changelog.d/fixes/10346-empty-pool-warn-once.md
Normal file
@@ -0,0 +1 @@
|
||||
- **fix(backend):** log `auto/<family> matched no connected models` once per process per label instead of every minute ([#10346](https://github.com/diegosouzapw/OmniRoute/issues/10346))
|
||||
1
changelog.d/fixes/10353-memory-heap-conflict-warn.md
Normal file
@@ -0,0 +1 @@
|
||||
- **fix(docker):** warn at boot when `OMNIROUTE_MEMORY_MB` disagrees with `NODE_OPTIONS --max-old-space-size`, and document that the standalone/Docker launcher appends `OMNIROUTE_MEMORY_MB` last ([#10353](https://github.com/diegosouzapw/OmniRoute/issues/10353))
|
||||
@@ -0,0 +1 @@
|
||||
- **fix(antigravity):** automatically rotate to a sibling account when one is BYOP (GCP Project ID required, `gcp_project_required` 422) — the account is excluded from selection for 24h and the request succeeds via another account instead of failing fast; the actionable 422 is surfaced only when no sibling exists (follow-up to the #10424 BYOP fast-fail) ([#10470](https://github.com/diegosouzapw/OmniRoute/pull/10470)) — thanks @rqzbeh
|
||||
@@ -0,0 +1 @@
|
||||
- **fix(network):** direct (no-proxy) egress now bounds each attempt's response-start window (default 30s, `OMNIROUTE_DIRECT_HEADERS_TIMEOUT_MS`) and retries once on a fresh no-keep-alive socket, so a silently-dropped pooled keep-alive connection can no longer stall direct providers (opencode-go, command-code) until a service restart ([#10214](https://github.com/diegosouzapw/OmniRoute/issues/10214))
|
||||
1
changelog.d/fixes/10536-llmlingua-2-2.0.5-drop-tfjs.md
Normal file
@@ -0,0 +1 @@
|
||||
- **fix(deps):** upgrade `@atjsh/llmlingua-2` from 2.0.3 to 2.0.5 and remove `@tensorflow/tfjs` from the LLMLingua SLM stack — 2.0.5 adds official Transformers.js v4 support (peers `@huggingface/transformers` at `^3.5.2 || ^4.0.0`) and 2.0.4+ no longer requires TensorFlow.js, restoring compatibility with OmniRoute's Transformers.js v4 while dropping the largest single contributor to the optional runtime footprint ([#10536](https://github.com/diegosouzapw/OmniRoute/issues/10536))
|
||||
@@ -0,0 +1 @@
|
||||
- fix(dashboard): route the Playground's ChatTab "Send" through the endpoint actually selected in StudioConfigPane (`search`, `web.fetch`, etc.) instead of always POSTing to `/api/v1/chat/completions`, fixing the false "No active credentials for provider" 404 when testing search-only providers (#10592)
|
||||
1
changelog.d/fixes/10597-combo-log-error-body.md
Normal file
@@ -0,0 +1 @@
|
||||
- **fix(sse):** Include the redacted upstream error body in the per-target COMBO failure log (`Model X failed, trying next`) so operators can triage a 400/500 without reproducing the request ([#10597](https://github.com/diegosouzapw/OmniRoute/issues/10597))
|
||||
1
changelog.d/fixes/10686-combo-quota-token-limit-await.md
Normal file
@@ -0,0 +1 @@
|
||||
- **Combo routing:** await each connection's token limit before reserving quota. The old lookup treated the `Promise` as a connection and dropped `rateLimitOverrides.tpm` ([#10686](https://github.com/diegosouzapw/OmniRoute/pull/10686)).
|
||||
1
changelog.d/fixes/10735-search-provider-named-errors.md
Normal file
@@ -0,0 +1 @@
|
||||
- **fix(search):** name `/v1/search` 502s with provider id and sanitized Node cause code, without hostnames ([#10735](https://github.com/diegosouzapw/OmniRoute/issues/10735))
|
||||
1
changelog.d/fixes/10765-rtk-unconditional-stats-cpu.md
Normal file
@@ -0,0 +1 @@
|
||||
- fix(compression): skip the expensive `createCompressionStats()` pass in RTK when no message was actually compressed, matching every sibling stacked engine (#10765)
|
||||
1
changelog.d/fixes/10788-ollama-cloud-effort-tiers.md
Normal file
@@ -0,0 +1 @@
|
||||
- **fix(open-sse):** declare `supportedThinkingEfforts` (`low`/`medium`/`high`/`max`) on Ollama Cloud's `glm-5.1`, `glm-5.2`, `deepseek-v4-pro` and `deepseek-v4-flash` registry entries so the catalog's `appendSyncedEffortVariants()` pass — which only synthesizes selectable `-low`/`-high`/`-max` model ids from an already-populated `capabilities.effort_tiers` — can expose an effort selector for these reasoning-capable models, matching what `gpt-oss:20b`/`gpt-oss:120b` already had (#10788)
|
||||
1
changelog.d/fixes/10792-double-transport-retry-scope.md
Normal file
@@ -0,0 +1 @@
|
||||
- **fix(resilience):** scope the same-account transport retry (#9708) out of emergency-fallback and combo hops — it was retrying the free fallback model and combo targets too, doubling upstream calls and corrupting the terminal error status on those paths.
|
||||
@@ -0,0 +1 @@
|
||||
- **fix(providers):** Keep NVIDIA timeout probes and generic Antigravity/AGY HTTP 400 probes from poisoning credential health while preserving explicit Google geo-block handling ([#10799](https://github.com/diegosouzapw/OmniRoute/pull/10799)) — thanks @Zartharas
|
||||
1
changelog.d/fixes/10815-kiro-oauth-profilearn-dedup.md
Normal file
@@ -0,0 +1 @@
|
||||
- fix(db): disambiguate `createProviderConnection()`'s OAuth email dedup by `providerSpecificData.profileArn` in addition to `username`, so adding a second Kiro/AWS profile with the same email creates a new connection instead of silently merging into the first (#10815)
|
||||
1
changelog.d/fixes/10832-unprefixed-dalle3.md
Normal file
@@ -0,0 +1 @@
|
||||
- **fix(images):** register OpenAI `dall-e-3` in the image registry so unprefixed `dall-e-3` (and `openai/dall-e-3`) route to OpenAI Images instead of Microsoft Designer Web, and so the chat catalog no longer lists `openai/dall-e-3` as a 128k chat model ([#10832](https://github.com/diegosouzapw/OmniRoute/issues/10832))
|
||||
1
changelog.d/fixes/10843-outbound-guard-mapped-ipv4.md
Normal file
@@ -0,0 +1 @@
|
||||
- **fix(security):** Outbound URL guard now resolves IPv4-mapped IPv6 literals to their embedded address, so `[::ffff:169.254.169.254]` is refused by the unconditional cloud-metadata block like its dotted spelling; `[::]` is refused alongside `0.0.0.0` ([#10843](https://github.com/diegosouzapw/OmniRoute/pull/10843)) — thanks @ntdat812
|
||||
1
changelog.d/fixes/10848-image-scan-cookie-bridge.md
Normal file
@@ -0,0 +1 @@
|
||||
- fix(config): exclude cookie-auth image bridges (chatgpt-web, gemini-web) from the unprefixed model scan so a bare id never silently binds to an unofficial web bridge (#10848)
|
||||
1
changelog.d/fixes/10849-search-provider-opaque-400.md
Normal file
@@ -0,0 +1 @@
|
||||
- fix(api): POST /v1/search now replies with a named `Unknown search provider: <id>` error (and field-named validation messages) instead of an opaque `Invalid request` for unrecognized or short-alias provider ids like `brave`/`serper` (#10849)
|
||||
1
changelog.d/fixes/10853-i18n-disabled-mistranslation.md
Normal file
@@ -0,0 +1 @@
|
||||
- **fix(i18n):** The "Disabled" status no longer renders as the noun for a person with a disability in Japanese, Spanish, Hindi, Polish, Telugu, Urdu and both Chinese locales — 24 strings now use each catalog's existing wording (ja 無効, es Deshabilitado, hi अक्षम, pl Wyłączone, te నిలిపివేయబడింది, ur غیر فعال, zh-CN 已禁用, zh-TW 已停用) ([#10812](https://github.com/diegosouzapw/OmniRoute/issues/10812), [#10853](https://github.com/diegosouzapw/OmniRoute/pull/10853)) — thanks @ntdat812
|
||||
@@ -0,0 +1 @@
|
||||
- **fix(catalog):** `/v1/models` no longer advertises the built-in `auto/*` ids while auto routing is disabled — they were listed but rejected at request time with `Auto routing is disabled` ([#10831](https://github.com/diegosouzapw/OmniRoute/issues/10831), [#10857](https://github.com/diegosouzapw/OmniRoute/pull/10857)) — thanks @ntdat812
|
||||
1
changelog.d/fixes/10858-base64-file-token-estimate.md
Normal file
@@ -0,0 +1 @@
|
||||
- **fix(context):** Base64 file payloads (OpenAI `file` parts, Responses `input_file`, Claude `document` blocks) are budgeted like the Gemini `inlineData` path instead of being counted as prompt text — a ~1MB PDF estimated at 350k tokens and was rejected on the context limit before reaching the provider's document pipeline ([#10840](https://github.com/diegosouzapw/OmniRoute/issues/10840), [#10858](https://github.com/diegosouzapw/OmniRoute/pull/10858)) — thanks @ntdat812
|
||||
1
changelog.d/fixes/10860-mcp-upstream-fetch-timeout.md
Normal file
@@ -0,0 +1 @@
|
||||
- **fix(mcp):** MCP tool calls that wait on a model provider no longer abort after 10 seconds. `omniRouteFetch` applied a single hardcoded `AbortSignal.timeout(10000)` to every internal hop, and `omniroute_route_request` — which posts to `/v1/chat/completions` and waits on the upstream provider, plus auto-combo candidate probing before a provider is even chosen — passed no signal of its own, so it inherited it. Any route slower than 10s failed from the MCP side while the identical request succeeded through the REST API. `omniroute_web_search` and `omniroute_web_fetch` in the same file already carried an explicit 60s signal, so that value is now shared by all three provider-bound calls instead of being repeated as a literal, while management reads (health, resilience, rate limits, combos, quota, usage) keep their fast-fail 10s budget so a stalled local endpoint still cannot hold a tool call open. Both budgets are overridable through `OMNIROUTE_MCP_FETCH_TIMEOUT_MS` and `OMNIROUTE_MCP_UPSTREAM_TIMEOUT_MS`, replacing the reported workaround of patching the compiled `dist/.build/next/server/chunks/*.js`; a malformed or non-positive override falls back to the default rather than disabling the timeout
|
||||
@@ -0,0 +1 @@
|
||||
- **fix(providers):** importing models with an expired API key now surfaces the credential error instead of reporting "No new models were added". The Import button posts to `/api/providers/{id}/sync-models`, which self-fetches the models route; that route does not fail on an upstream 401 but degrades to a catalog it already has, preferring the cache and using the local catalog only when there is no cache. A provider that imported successfully once therefore has a cache, so an expired key produced `{ source: "cache", warning: "Models probe failed (401) — using cached catalog" }` with HTTP 200 — and the #5460/#5465 degradation guard only recognised the `local_catalog` branch, so model-sync accepted it as a successful discovery, found every cached model already imported, and returned the empty-diff result. Retest does not go through this path, which is why it failed correctly and made the import look like a genuine "nothing to do". The existing rule — a degraded discovery must not be persisted as the synced catalog — is now applied to the branch it missed rather than special-casing 401/403, discriminating on the warning the fallback builder always attaches (an ordinary non-refresh cache hit attaches none, and model-sync always requests `refresh=true`). `isDegradedLocalCatalog` keeps its exact meaning and its existing tests
|
||||
1
changelog.d/fixes/10866-combo-empty-models.md
Normal file
@@ -0,0 +1 @@
|
||||
- fix(api): reject a combo update that removes every model, and store the copilot's combo targets where the router reads them (#10866)
|
||||
1
changelog.d/fixes/10868-proxy-echo-ipv4-fallback.md
Normal file
@@ -0,0 +1 @@
|
||||
- **fix(proxy):** proxy "Test connection" no longer reports an IPv4-only SOCKS5/SSH proxy as dead. #1255 moved every egress probe from `api.ipify.org` to `api64.ipify.org` so proxies with IPv6 egress could be tested, but `api64` is IPv6-first: a tunnel with no IPv6 route has nothing to connect to, so the probe hung until the caller's deadline and a proxy that was carrying live LLM traffic came back as a failure. Swapping the target to `api4` fixes that case and re-breaks the one #1255 fixed, so the probe now tries the targets in order instead — `api64` first, so a proxy with working IPv6 answers on the first attempt and keeps the exact behaviour #1255 introduced, including which of its addresses is reported (the egress IP is used as an identity to detect accounts of one rotation group sharing an address, so the attempts are sequential rather than raced). The attempts split the budget each call site already enforced, so no probe can take longer than it could before, and each attempt gets its own `AbortController` so exhausting the budget on an unreachable target does not abort the next one. `OMNIROUTE_PROXY_ECHO_URL` pins a single target — including a self-hosted echo — replacing the workaround of rewriting the compiled bundle after every upgrade. The relay branch of the test route still targets `api64` through `x-relay-target`, since that request egresses from the relay worker rather than the operator's tunnel
|
||||
1
changelog.d/fixes/10870-cli-env-collision.md
Normal file
@@ -0,0 +1 @@
|
||||
- fix(cli): warn when a .env line never takes effect, and stop swallowing an unreadable .env (#10870)
|
||||
@@ -0,0 +1 @@
|
||||
- **fix(db):** Remove stale MiMoCode provider configuration, including the legacy `mcode` alias, left after provider retirement while preserving historical usage and call logs ([#10873](https://github.com/diegosouzapw/OmniRoute/pull/10873)) — thanks @Zartharas
|
||||
@@ -0,0 +1 @@
|
||||
- **fix(sse):** `getResetAwareProvider()` and the auto-combo quota lookup in `combo.ts` now canonicalize the provider id via `resolveProviderId()` before calling `getQuotaFetcher()`, so a fetcher registered under a provider's canonical id (e.g. `ollama-cloud`, `codex`) is found for combo targets stored under an alias spelling (e.g. `ollamacloud`, `cx`) instead of silently degrading reset-aware/reset-window/auto quota-aware routing to plain priority ordering (#10877)
|
||||
@@ -0,0 +1 @@
|
||||
- **fix(provider-health):** Keep unsupported 404/405 validation probes neutral so they do not poison stored credential health or scheduler failure state, while still honoring per-connection health-check pacing ([#10878](https://github.com/diegosouzapw/OmniRoute/pull/10878)) — thanks @Zartharas
|
||||
@@ -0,0 +1 @@
|
||||
- **fix(antigravity):** map Gemini 3.7 Flash tier ids (`gemini-3.7-flash-high/medium/low`, bare `gemini-3.7-flash`) to the upstream `gemini-3.7-flash-tiered` model id Google's Cloud Code endpoint expects, and configure per-tier thinking budgets ([#10882](https://github.com/diegosouzapw/OmniRoute/pull/10882)) — thanks @adevwithpurpose
|
||||
1
changelog.d/fixes/10887-memory-mcp-tools.md
Normal file
@@ -0,0 +1 @@
|
||||
- **fix(memory):** enable agent memory save/update via MCP tools (`memory_save`/`update`/`search`/`delete` builtins with per-provider schemas, `apiKeyId` optional with caller-principal fallback) and gate server-side memory builtin injection to non-stream requests only ([#10887](https://github.com/diegosouzapw/OmniRoute/pull/10887)) — thanks @Egorich-print
|
||||
1
changelog.d/fixes/10902-pplx-search-hint-optin.md
Normal file
@@ -0,0 +1 @@
|
||||
- **fix(perplexity-web):** make the built-in-search hint appended to every system message opt-in via `OMNIROUTE_PPLX_SEARCH_HINT` (off by default) — Perplexity's answer engine searches anyway, and the hint leaked into replies as meta-commentary for coding clients ([#10902](https://github.com/diegosouzapw/OmniRoute/pull/10902), extracted from [#8634](https://github.com/diegosouzapw/OmniRoute/pull/8634)) — thanks @danscMax
|
||||
@@ -0,0 +1 @@
|
||||
- **fix(images):** retry Codex image generation on a sibling ChatGPT account when the requested model isn't entitled on the current account, instead of failing the request outright ([#8307](https://github.com/diegosouzapw/OmniRoute/pull/8307)).
|
||||
1
changelog.d/fixes/claude-to-gemini-consecutive-roles.md
Normal file
@@ -0,0 +1 @@
|
||||
- **fix(translator):** merge consecutive same-role contents in direct Claude to Gemini request translation to prevent upstream HTTP 400 errors
|
||||
@@ -0,0 +1 @@
|
||||
- **fix(catalog):** derive combo reasoning-effort tiers from the exact runtime-selectable connection scope, intersecting dynamic, pinned, allowlisted, and compatible provider-node evidence while failing closed on unknown capabilities.
|
||||
1
changelog.d/fixes/minimax-music-generation-dispatch.md
Normal file
@@ -0,0 +1 @@
|
||||
- **fix(sse):** MiniMax music models now generate audio instead of failing with `Unsupported music format: minimax-music` — the provider entry was registered in the music registry (and advertised by `/v1/models`), but `handleMusicGeneration` had no branch for its format, so every `minimax/*` music request fell through the dispatch chain to a 400. Adds the missing dispatch: a single synchronous POST with the `base_resp` envelope check (a non-zero `status_code` arrives on HTTP 200 too), `data.status` handling (an unfinished generation is reported instead of polled — the operation has no task id and no query endpoint), `url` and `hex` output formats (hex normalized to base64), `mp3`/`wav`/`pcm` containers via `audio_setting`, and the regional endpoint through the per-connection base-URL override, which is also the only host that accepts `aigc_watermark`. The registry entry gains the generation and cover model ids it was missing and drops a query URL that does not exist for this operation. Regression guard: `tests/unit/minimax-music-generation.test.ts` (9 tests).
|
||||
@@ -0,0 +1 @@
|
||||
- **docs(docker):** spell out that `:latest` tracks the highest **published** stable SemVer (not git `main`), and that GitOps should pin `X.Y.Z` ([#10317](https://github.com/diegosouzapw/OmniRoute/issues/10317))
|
||||
@@ -0,0 +1 @@
|
||||
- **docs(backend):** document that memory extraction, skills injection, and token refresh share the request event loop, plus dashboard kill switches ([#10349](https://github.com/diegosouzapw/OmniRoute/issues/10349))
|
||||
@@ -0,0 +1 @@
|
||||
- **docs(docker):** document default SQLite as single-replica / HA-unsupported, including Recreate and HEALTHCHECK session blast radius ([#10350](https://github.com/diegosouzapw/OmniRoute/issues/10350))
|
||||
@@ -0,0 +1 @@
|
||||
- **docs(backend):** document that pre-write SQLite backups (including models.dev pricing) are throttled to once per 60 minutes and can be disabled with `DISABLE_SQLITE_AUTO_BACKUP` ([#10351](https://github.com/diegosouzapw/OmniRoute/issues/10351))
|
||||
1
changelog.d/maintenance/10780-server-init-dead-code.md
Normal file
@@ -0,0 +1 @@
|
||||
- chore(startup): remove `src/server-init.ts` (183 lines, never imported — the boot path is `src/instrumentation-node.ts`) and correct four `"called from server-init.ts"` comments left pointing at the dead entry point (#10780)
|
||||
1
changelog.d/maintenance/10875-combos-id-verb-coverage.md
Normal file
@@ -0,0 +1 @@
|
||||
- **docs(openapi):** document the `GET` and `PUT` operations on `/api/combos/{id}`, and add an operation-level coverage floor so a missing verb can no longer hide behind a path that already counts as covered ([#10875](https://github.com/diegosouzapw/OmniRoute/pull/10875))
|
||||
1
changelog.d/maintenance/7786-management-auth-guide.md
Normal file
@@ -0,0 +1 @@
|
||||
- **docs(auth):** distinguish dashboard sessions, `oma_live_…` Access Tokens, manage-scoped API keys, and inference keys ([#7786](https://github.com/diegosouzapw/OmniRoute/issues/7786))
|
||||
1
changelog.d/maintenance/env-doc-sync-adhoc-bot.md
Normal file
@@ -0,0 +1 @@
|
||||
- **chore(ci):** ignore ad-hoc `BOT_TOKEN`/`BOT_URL` in env-doc-sync (scripts/ad-hoc mesh helpers, not runtime config)
|
||||
@@ -20,7 +20,6 @@
|
||||
"@stryker-mutator/tap-runner",
|
||||
"@swc/helpers",
|
||||
"@tailwindcss/postcss",
|
||||
"@tensorflow/tfjs",
|
||||
"@testing-library/jest-dom",
|
||||
"@testing-library/react",
|
||||
"@toon-format/toon",
|
||||
|
||||
@@ -54,11 +54,6 @@
|
||||
"count": 3
|
||||
}
|
||||
},
|
||||
"open-sse/handlers/chatCore/codexFailover.ts": {
|
||||
"no-restricted-imports": {
|
||||
"count": 1
|
||||
}
|
||||
},
|
||||
"open-sse/handlers/chatCore/comboContextCache.ts": {
|
||||
"no-restricted-imports": {
|
||||
"count": 1
|
||||
@@ -1023,11 +1018,6 @@
|
||||
"count": 1
|
||||
}
|
||||
},
|
||||
"src/lib/api/modelTestRunner.ts": {
|
||||
"no-restricted-imports": {
|
||||
"count": 1
|
||||
}
|
||||
},
|
||||
"src/lib/api/proxyRegistryRouteHandlers.ts": {
|
||||
"no-restricted-imports": {
|
||||
"count": 1
|
||||
@@ -1083,11 +1073,6 @@
|
||||
"count": 1
|
||||
}
|
||||
},
|
||||
"src/lib/embeddings/service.ts": {
|
||||
"no-restricted-imports": {
|
||||
"count": 1
|
||||
}
|
||||
},
|
||||
"src/lib/evals/runtime.ts": {
|
||||
"no-restricted-imports": {
|
||||
"count": 1
|
||||
@@ -2934,11 +2919,6 @@
|
||||
"count": 83
|
||||
}
|
||||
},
|
||||
"tests/unit/responses-parse-once-4041.test.ts": {
|
||||
"@typescript-eslint/no-explicit-any": {
|
||||
"count": 6
|
||||
}
|
||||
},
|
||||
"tests/unit/responses-translation-fixes.test.ts": {
|
||||
"@typescript-eslint/no-explicit-any": {
|
||||
"count": 35
|
||||
|
||||
@@ -304,6 +304,8 @@
|
||||
"_rebaseline_2026_07_27_3850_relax_filesize_cap": "OWNER-APPROVED TEMPORARY relax for v3.8.50-3.8.54 PREPARE phase (docs/ROADMAP.md). cap 800->900 (+100), testCap 800->900 (+100). Targets: decompose-existing-frozen unchanged (frozen still only-shrink); this only relaxes the cap for NEW files in the decompose/extract-while-PREPARE phase (.51='executor registry in-place' and .52='combo.ts decomposition' create new leaf modules above 800). RE-TIGHTENING MANDATORY in v3.8.51: cap target 850 = 850 once decomposition wave stabilizes. SUPERSEDED by _rebaseline_2026_07_27_3850_relax_filesize_cap_v2_20pct (v1 +20% buffer) — retained for audit. Tracked via same roadmap issue.",
|
||||
"_rebaseline_2026_07_27_v3849_train1h": "Merge-train 1H (31 PRs) — owner-approved 2026-07-27. Two distinct causes, kept separate on purpose: (1) GENUINE irreducible growth at existing chokepoints — providerLimits/auth (#8632 Kimi quota-reset recovery), rateLimitManager (#8616 idle wedged limiters), models-catalog-route.test (#8610 OpenCode Go effort aliases); (2) COLLISION with #8585, which banked shrinks measured on the pre-train release tip while 30 sibling PRs in the SAME train grew those files again — chat/accountFallback (#8628), chatCore (#8613), videoGeneration (#8581), imageGeneration. The zero-headroom frozen entries cannot absorb either. Ceilings re-pinned to the post-merge tip; #8612 (also in this train) automates shrink-banking so this self-inflicted drift stops recurring. Detail: src/lib/usage/providerLimits.ts 1006->1013 (#8632); src/sse/services/auth.ts 2492->2508 (#8632); open-sse/services/rateLimitManager.ts 1014->1060 (#8616); src/sse/handlers/chat.ts 1842->1845 (#8628); open-sse/handlers/chatCore.ts 4939->4955 (#8613); open-sse/handlers/imageGeneration.ts 3100->3101 ((sem PR — teto do #8585)); open-sse/handlers/videoGeneration.ts 1038->1063 (#8581); open-sse/services/accountFallback.ts 1965->1966 (#8628); tests/unit/models-catalog-route.test.ts 1608->1636 (#8610)",
|
||||
"frozen": {
|
||||
"_rebaseline_2026_08_20_10878_10799_provider_health_probes": "PRs #10878 (unsupported OpenAI-like validation probes stay neutral) + #10799 (preserve credential health on inconclusive NVIDIA-timeout/Antigravity-400 probes) own growth: src/app/api/providers/[id]/test/route.ts 946->1025 (+79, sum of both boarded together). Both add narrowly-scoped classification branches at the existing test-route dispatch chokepoint (unsupported-capability skip, credential-inconclusive detection) rather than new files, mirroring the prior 2026_06_27_5193 rebaseline of the same file. Covered by tests/unit/provider-validation-unsupported-neutral.test.ts + tests/unit/provider-health-inconclusive-probes.test.ts.",
|
||||
"src/app/api/providers/[id]/test/route.ts": 1025,
|
||||
"_rebaseline_2026_06_22_4644_deepseek_web_tools": "PR #4644 (BugsBag/robust deepseek-web tool-call parsing): open-sse/executors/deepseek-web.ts 1117->1125 (+8). The new agentic tool-call path emits surrounding text + reasoning before tool_calls and swaps to the dedicated deepseekWebTools.ts parser; the +8 lines are cohesive wiring at the existing transformSSE chokepoint (the parser itself lives in the new deepseekWebTools.ts file, already under cap). The PR's own fast-gate (PR->release) does not run check:file-size, so this surfaced only at release reconcile. Covered by tests/unit/deepseek-web-tools-variants.test.ts + deepseek-web-tools-execute.test.ts.",
|
||||
"_rebaseline_2026_06_23_4712_deepseek_web_tool_results": "PR for #4712 (deepseek-web drops role:tool): open-sse/executors/deepseek-web.ts 1125->1148 (+23). messagesToPrompt() now folds role:\"tool\" results into the single-prompt transcript (recovering the tool name from the preceding assistant tool_calls by tool_call_id) instead of silently dropping them; the lines are cohesive wiring inside the existing function. Covered by tests/unit/deepseek-web-tool-result-prompt-4712.test.ts.",
|
||||
"_rebaseline_2026_06_24_headroom_strategy": "Headroom-aware connection selection (dario technique): combo.ts 3168->3180 (+12 = a new `else if (strategy === \"headroom\")` dispatch branch in handleComboChat that delegates to orderTargetsByHeadroom + its log line, plus the import). The actual logic lives OUT of the god-file: the pure ranker rankByHeadroom/computeHeadroom is the new leaf open-sse/services/combo/headroomRanking.ts (91 LOC, <cap) and the async orderer orderTargetsByHeadroom is appended to the existing open-sse/services/combo/quotaStrategies.ts (<cap) next to its sibling reset-aware/reset-window orderers (reuses their connection-expansion machinery). headroom = 1 - max(util_5h, util_7d) from getSaturation (src/lib/quota/saturationSignals.ts), prefers the connection with the most free capacity. Only the dispatch wiring is irreducible at the existing combo strategy chokepoint (mirrors the reset-aware/reset-window/context-optimized branches); not extractable without hiding the call site. fill-first stays default; all existing strategies untouched. Covered by tests/unit/combo-headroom-ranking.test.ts (pure helper) + tests/unit/combo-headroom-strategy.test.ts (orderer, saturation injected). Structural shrink of combo.ts tracked in #3501.",
|
||||
@@ -442,14 +444,15 @@
|
||||
"src/app/(dashboard)/dashboard/providers/[id]/components/modals/AddApiKeyModal.tsx": 1062,
|
||||
"src/app/(dashboard)/dashboard/providers/[id]/hooks/useProviderConnections.ts": 1051,
|
||||
"src/shared/components/ModelSelectModal.tsx": 1138,
|
||||
"src/shared/constants/providers/apikey/gateways.ts": 1268,
|
||||
"src/shared/constants/providers/apikey/gateways.ts": 1283,
|
||||
"open-sse/vendor/codex-chatgpt-web/bridge.ts": 1387,
|
||||
"_rebaseline_2026_08_11_v3850_merge_storm_provider_registry": "DRIFT do merge-storm 2026-08-11 (99 PRs mergeados no release/v3.8.50). AddApiKeyModal.tsx (PR #8949 ChatGPT Web provider) e useProviderConnections.ts/ModelSelectModal.tsx (PRs #9011 combo test-all, #9499 image combos) = UI nova legitima acima do cap; gateways.ts = god-file de catalogo de providers que cresceu com PRs #9009/#9421/#9468/#9594 (qualquer split arriscaria corromper o merge de novo — o proprio PR #9421 quebrou o arquivo); bridge.ts (PR #8949) = ponte Chromium vendored; proxyFetch.ts 1207->1220 = drift herdado de merges. Owner autorizou rebaseline com anotacao (2026-08-11).",
|
||||
"src/lib/modelCapabilities.ts": 1006,
|
||||
"src/app/(dashboard)/dashboard/providers/[id]/providerPageHelpers.ts": 1014,
|
||||
"open-sse/config/imageRegistry.ts": 1019,
|
||||
"open-sse/config/imageRegistry.ts": 1034,
|
||||
"src/sse/handlers/chatHelpers.ts": 1017,
|
||||
"src/shared/middleware/chatBodyAdmission.ts": 1005
|
||||
"src/shared/middleware/chatBodyAdmission.ts": 1005,
|
||||
"_rebaseline_2026_08_20_10668_tabitoken_gateway": "#10668 (yawar-aquil) own catalog growth: src/shared/constants/providers/apikey/gateways.ts 1268->1283 (+15, entirely this PR diff -- one new tabitoken gateway entry, data lines only; base moved from 1255 to 1268 via other merges since the PR forked). Not combination drift: reproducible on the PR branch alone, so the WS5.5 release-captain rule does not apply. Extraction is not available -- the file is pure data (own header: \"Pure data; merged by apikey/index.ts via spread\") and already split into 6 family files under apikey/. Same precedent as _rebaseline_2026_08_14_imagetotext_servicekinds (#10275/#10291, gateways.ts 1250->1255, data lines only) and _rebaseline_2026_08_11_v3850_merge_storm_provider_registry (owner-authorized for this same file)."
|
||||
},
|
||||
"_rebaseline_base_2026_08_10_proxyfetch": "Base-red fix (green-prs sweep, issue #9985): open-sse/utils/proxyFetch.ts 1207 > cap 1000 — new proxied-TLS fetch helper introduced by the Fal reference-image work. Owner-authorized quick rebaseline to green; structural slim tracked for v3.9.0.",
|
||||
"_rebaseline_2026_07_27_v3849_train2": "Merge-train 2 (7 PRs) — owner-approved 2026-07-27. Single entry: chatCore.ts 4955->5006 (#8595, Responses multi-turn image compaction before the context hard-reject). Genuine irreducible growth at the existing compaction chokepoint in handleChatCore — the PR adds a last-resort retry against the concrete budget plus the estimateFinalInputTokens helper, both wired at the pre-existing call site rather than a new branch. Covered by tests/unit/8560-responses-image-compaction.test.ts (4 tests).",
|
||||
@@ -615,5 +618,7 @@
|
||||
"_rebaseline_2026_08_12_modelcapabilities_snapshot_routing": "Base-reds round 3 (#9985): modelCapabilities.ts crossed the new-file cap at 1006 (+~10) when the context/max-input-token override lookups were routed through the #9199 bulk snapshot (fixing 323 per-model SQLite reads per catalog prepare — auto-combo-context-advertising guard); cohesive change at the existing resolution chokepoints, not extractable. Covered by tests/unit/auto-combo-context-advertising.test.ts + model-capability-resolution-snapshot-9199.test.ts.",
|
||||
"_rebaseline_2026_08_14_imagetotext_servicekinds": "Image-to-Text category (#10275/#10291): gateways.ts grew 1250→1255 by data lines only — the serviceKinds: [\"llm\", \"imageToText\"] declarations on the openrouter and chutes catalog entries, plus the 3-line comment recording why chutes needs no static dots.ocr entry (passthroughModels discovery). No new logic or branching; the file is a provider catalog of declarative metadata. Splitting a catalog for five lines would be worse than the growth (semantic-families rule).",
|
||||
"_rebaseline_2026_08_18_imageregistry_merge_train": "merge-train 2026-08-18 (owner-authorized, /merge-prs batch of 84): open-sse/config/imageRegistry.ts crossed the 1000-line new-file cap for the first time purely from combining three independent, already-legitimate provider registrations boarded in the same local merge-train — #10542 (aihorde optional-key image catalog), #10494 (gemini-web image generation), #10594 (freepik/magnific provider rename + validation). 996 on release tip -> 1019 on the train tip. Each PR individually adds a small, additive IMAGE_PROVIDERS registry entry at the existing chokepoint; none crosses the cap alone. Not modularized as part of this train's gate fix (out of scope for a merge reconciliation, not a feature change). Covered by each PR's own focused tests (aihorde-image-catalog/generation, gemini-web image tests, freepik/magnific provider tests).",
|
||||
"_rebaseline_2026_08_20_v3850_merge_train_batch1": "Merge-train batch1 (2026-08-19/20, 30 PRs boarded onto release/v3.8.50): gateways.ts 1255->1268 = PR #10722 (Token Kiosk OpenAI-compatible provider gateway catalog entry, +13 declarative lines, same god-file no-split rationale as prior gateways.ts rebaselines); chatHelpers.ts (uncapped, not previously frozen) new 1017 = PR #10797 (relay/bifrost error normalization, +23/-2, own-PR growth, existing file already near cap from accumulated chokepoint wiring per its own rebaseline history above); chatBodyAdmission.ts (uncapped) new 1005 = pre-existing base-red on the pure release tip (1004>1000 before this train boarded anything, no PR in this batch touches this file) — frozen here at its current size, not authorizing further growth. Owner-authorized rebaseline (2026-08-19 merge-prs session)."
|
||||
"_rebaseline_2026_08_20_v3850_merge_train_batch1": "Merge-train batch1 (2026-08-19/20, 30 PRs boarded onto release/v3.8.50): gateways.ts 1255->1268 = PR #10722 (Token Kiosk OpenAI-compatible provider gateway catalog entry, +13 declarative lines, same god-file no-split rationale as prior gateways.ts rebaselines); chatHelpers.ts (uncapped, not previously frozen) new 1017 = PR #10797 (relay/bifrost error normalization, +23/-2, own-PR growth, existing file already near cap from accumulated chokepoint wiring per its own rebaseline history above); chatBodyAdmission.ts (uncapped) new 1005 = pre-existing base-red on the pure release tip (1004>1000 before this train boarded anything, no PR in this batch touches this file) — frozen here at its current size, not authorizing further growth. Owner-authorized rebaseline (2026-08-19 merge-prs session).",
|
||||
"_rebaseline_2026_08_20_8338_cursor_image_provider": "PR (reimplementation of #8338, @valvesss): imageRegistry.ts 1019->1033 = new cursor IMAGE_PROVIDERS entry (Cursor plan image generation via Agent CLI), +14 lines of declarative provider metadata. Same god-registry no-split rationale as prior imageRegistry/gateways rebaselines.",
|
||||
"_rebaseline_2026_08_20_imageregistry_1034": "imageRegistry.ts 1033->1034: +1 line drift between #10842 (cursor image provider, froze at 1033) and its actual merged state on release (measured 1034) — trivial rebaseline, not a new feature."
|
||||
}
|
||||
@@ -166,7 +166,8 @@
|
||||
"dedicatedGate": true
|
||||
},
|
||||
"zizmorFindings": {
|
||||
"value": 190,
|
||||
"value": 192,
|
||||
"_rebaseline_2026_08_20_radar_export_workflow": "190 -> 192 (+2). Workflow novo `.github/workflows/radar-export.yml` (passo 10 do go-live do Radar: publica o export estável do catálogo como asset de release para o servidor privado baixar via RADAR_EXPORT_URL). Os +2 são unpinned-uses @vN: actions/checkout@v7 + actions/setup-node@v7 — a MESMA convenção deliberada de todos os workflows (ver _scanner_harden_workflows_2026_06_16); fixar por SHA só este violaria a convenção. O findings artipacked do checkout foi CORRIGIDO com `persist-credentials: false` (o job publica via GH_TOKEN em `gh release`, não usa a credencial do checkout). Nenhuma classe nova de template-injection / cache-poisoning / dangerous-triggers. Medido local com zizmor 1.25.2 via `node scripts/check/check-workflows.mjs --ratchet` = 191; +1 do delta conhecido do runner (ver _rebaseline_2026_07_28_ci_runner_delta: o runner enxerga 1 unpinned-uses @vN a mais que o devbox no mesmo commit; a baseline segue o runner) => 192.",
|
||||
"_rebaseline_2026_07_20_aliasresolver_hook_split_7808": "175 -> 176 (+1). Companion to PR #7808 (CodeQL js/incomplete-url-substring-sanitization fix in bin/aliasResolver.mjs). The +1 is NOT caused by this PR's code changes (bin/* is not a workflow file) — it is a pre-existing drift that surfaced because the ratchet gate runs on this PR's CI: the zizmor scanner version on the GitHub runner gained a new rule (or extended an existing one) since the v3.8.49 baseline was seeded on 2026-07-17. Breakdown: the new finding is an unpinned-uses @vN class item on one of the existing workflows (same deliberate convention as _scanner_harden_workflows_2026_06_16 — @vN is intentional, SHA-pinning only this one would violate the convention). No new template-injection/artipacked/cache-poisoning/dangerous-triggers classes introduced. Measured by the Quality Gates (Extended) job on run 29713001401 = 176, baseline was 175. Note: by the time this landed on release/v3.8.49, the baseline was already at 176 via _rebaseline_2026_07_17_combo_recovery_hints — this entry is kept as historical record; no further bump applied.",
|
||||
"_rebaseline_2026_07_17_v3849_release": "169 -> 175 (+6). Cycle workflow drift (v3.8.48/v3.8.49): npm-publish.yml (new, WS1.3 #7092), electron-release.yml, nightly-compat.yml, nightly-release-green.yml, CI restructures (#7501 full-history base fetch, #7355 main-green, #7202 merge-queue gates, Trunk/Codecov). Breakdown vs v3.8.47: +3 unpinned-uses (@vN convention, deliberate per _scanner_harden_workflows_2026_06_16), +2 cache-poisoning (artifact upload/cache in the OWN electron-release/npm-publish RELEASE workflows -- operator-controlled, not fork-PR exploitable), +1 excessive-permissions (nightly-compat.yml permissions:issues). No new template-injection/artipacked/dangerous-triggers. Measured with zizmor 1.25.2 via `node scripts/check/check-workflows.mjs --ratchet` = 175 on da3a0be69.",
|
||||
"direction": "down",
|
||||
|
||||
350
docs/architecture/ADAPTIVE_ROUTING.md
Normal file
@@ -0,0 +1,350 @@
|
||||
---
|
||||
title: "Adaptive Routing: Routing Events, Quality Feedback & Explainability"
|
||||
version: 3.8.50
|
||||
lastUpdated: 2026-08-20
|
||||
---
|
||||
|
||||
# Adaptive Routing: Routing Events, Quality Feedback & Explainability
|
||||
|
||||
This document describes the feedback-driven adaptive routing foundation added to
|
||||
OmniRoute. It is deliberately small: it introduces a typed routing-outcome
|
||||
channel, an online quality signal that feeds the existing auto-combo scorer, an
|
||||
optional OpenTelemetry exporter, and an explainability endpoint. It does **not**
|
||||
replace the existing resilience stack (circuit breaker, connection cooldown,
|
||||
model lockout, health matrix, autopilot) — it complements it.
|
||||
|
||||
## 1. Architectural context
|
||||
|
||||
OmniRoute is a data plane with a **request hot path** and a **control/intelligence
|
||||
plane**. The hot path must stay fast, memory-efficient, asynchronous, resilient and
|
||||
predictable. Evaluation, quality scoring, experiments and historical analysis belong
|
||||
to the control plane.
|
||||
|
||||
```
|
||||
AI Agent / IDE
|
||||
│
|
||||
▼
|
||||
┌─────────────────────┐
|
||||
│ OmniRoute │ data plane (fast, sync, in-memory)
|
||||
│ routing / failover │
|
||||
│ health / guardrail │
|
||||
│ cache / streaming │
|
||||
└──────────┬──────────┘
|
||||
│ RoutingEvent (fire-and-forget, ~0.2µs)
|
||||
▼
|
||||
┌─────────────────────┐
|
||||
│ Feedback sinks │ control plane (async, best-effort)
|
||||
│ quality tracker │
|
||||
│ OTel exporter │
|
||||
│ explain store │
|
||||
└──────────┬──────────┘
|
||||
▼ quality score
|
||||
auto-combo scorer
|
||||
```
|
||||
|
||||
### What was already there (audited, not duplicated)
|
||||
|
||||
| Concept | Existing implementation |
|
||||
| ----------------------------------- | -------------------------------------------------------------------------------------------------- |
|
||||
| Availability (can we send traffic?) | Circuit breaker (CLOSED/DEGRADED/OPEN/HALF_OPEN, DB-persisted), connection cooldown, model lockout |
|
||||
| Health reporting | `providerHealthMatrix.ts`, `providerHealthAutopilot.ts` |
|
||||
| Shadow traffic | `open-sse/services/combo/shadowRouting.ts` |
|
||||
| Guardrails | `src/lib/guardrails/` (pre/post hooks) |
|
||||
| Exact cache | `src/lib/semanticCache.ts` (signature-based) |
|
||||
| Evaluators / eval-driven routing | `src/lib/evals/`, `open-sse/services/evalRouting.ts` |
|
||||
| Combo decision explainability | `open-sse/services/combo/decisionTrace.ts` |
|
||||
| Dashboard real-time events | `src/lib/events/eventBus.ts` (UI notification channel, `unknown` payloads, 100-entry history) |
|
||||
|
||||
The routing-event layer is **not** a re-implementation of `eventBus`: that bus is
|
||||
the dashboard's real-time notification channel (typed _event names_, opaque
|
||||
payloads, UI consumers). `RoutingEvent` is a typed _outcome_ struct
|
||||
(latency/tokens/cost/outcome/finish-reason) consumed by the control plane's
|
||||
feedback sinks (quality tracker, OTel exporter, explain store).
|
||||
|
||||
### What was missing (added here)
|
||||
|
||||
1. A **typed routing-outcome event + sink abstraction** (`RoutingEvent` /
|
||||
`RoutingEventSink`). `decisionTrace` is combo-scoped and in-memory-only;
|
||||
`comboMetrics` are cumulative counters; `call_logs` is raw async persistence.
|
||||
None is a typed, sink-based outcome channel that a quality tracker, an OTel
|
||||
exporter, or a Future-AGI-style evaluator can subscribe to.
|
||||
2. An **online quality signal** (EWMA) for output quality — the scorer previously
|
||||
proxied "quality" only through static task fitness and opt-in eval pass-rates.
|
||||
3. An **optional, dependency-free OTel exporter** using GenAI semantic conventions.
|
||||
4. An **explainability endpoint** returning the real routing decisions + quality state.
|
||||
|
||||
## 2. Routing Events (feedback foundation)
|
||||
|
||||
Files: `open-sse/services/routing/events.ts`, `.../index.ts`
|
||||
|
||||
A `RoutingEvent` carries only routing metadata:
|
||||
|
||||
```ts
|
||||
interface RoutingEvent {
|
||||
requestId: string;
|
||||
provider: string;
|
||||
model: string;
|
||||
strategy: string; // "auto" | "priority" | "direct" | ...
|
||||
latencyMs: number;
|
||||
ttftMs: number | null;
|
||||
inputTokens: number | null;
|
||||
outputTokens: number | null;
|
||||
cost: number | null;
|
||||
retries: number;
|
||||
fallbackUsed: boolean;
|
||||
outcome: RoutingOutcome; // allowlisted union
|
||||
status: number | null;
|
||||
finishReason: string | null;
|
||||
connectionId: string | null;
|
||||
ts: number;
|
||||
}
|
||||
```
|
||||
|
||||
`RoutingEventSink` is a `Send+Sync`-style trait in TypeScript:
|
||||
|
||||
```ts
|
||||
interface RoutingEventSink {
|
||||
readonly name: string;
|
||||
record(event: RoutingEvent): void; // must be O(1), no sync I/O
|
||||
}
|
||||
```
|
||||
|
||||
The hot path calls `emitRoutingEvent(event)` once per completed request
|
||||
(the streaming-completion callback, the non-streaming success path, and the
|
||||
malformed-200 failure path in `handleChatCore`). Dispatch is synchronous fan-out
|
||||
to registered sinks, but each sink only enqueues/updates in-memory state. **No
|
||||
synchronous database writes, no network I/O on the hot path.**
|
||||
|
||||
Default sinks:
|
||||
|
||||
- `MemoryRoutingEventStore` — bounded (500) ring buffer, newest-first, for the
|
||||
explain endpoint.
|
||||
- `QualityTracker` consumer — updates the EWMA quality estimate.
|
||||
- `OtlpHttpsEventSink` — optional, enabled only when `OMNIROUTE_OTEL_ENDPOINT`
|
||||
(or `OTEL_EXPORTER_OTLP_ENDPOINT`) is set.
|
||||
|
||||
### Measured overhead (honest comparison)
|
||||
|
||||
`npm run bench:routing-events` on this workstation (100k iterations; sub-µs ops
|
||||
measured as aggregate µs/op because per-op percentiles are below
|
||||
`performance.now()` timer resolution):
|
||||
|
||||
| Scenario | µs/op | ops/s |
|
||||
| --------------------------------- | ------ | ------ |
|
||||
| baseline (scoring only) | ~0.045 | ~22 M |
|
||||
| baseline + RoutingEvent (2 sinks) | ~0.168 | ~5.9 M |
|
||||
| baseline + event + OTel enqueue | ~0.163 | ~6.1 M |
|
||||
| concurrent (8 interleaved bursts) | ~0.18 | — |
|
||||
|
||||
The event-dispatch delta over baseline scoring is ~0.12 µs/request; the OTel sink
|
||||
only enqueues (O(1) buffer push), adding nothing measurable. These numbers are
|
||||
machine-specific and relative — not a production guarantee. The v1 "~0.2 µs"
|
||||
figure was an aggregate estimate; this methodology separates the scoring baseline
|
||||
from the event-dispatch cost.
|
||||
|
||||
## 3. Quality Signal (feedback-driven provider state)
|
||||
|
||||
Files: `open-sse/services/routing/quality.ts`
|
||||
|
||||
v2 separates **operational** from **semantic** quality:
|
||||
|
||||
- **Operational** — derived from the routing hot path (HTTP 4xx/5xx, connection
|
||||
failures, 429s, malformed responses, stream interruptions, `finish_reason=length`,
|
||||
zero-output successes, latency/TTFT EWMA). A 200 is NOT treated as semantic
|
||||
quality.
|
||||
- **Semantic** — the actual value of the generated output. ONLY ever produced by
|
||||
an evaluator via `setSemanticQuality()`. It is `null` until one provides it and
|
||||
never leaks into the operational score.
|
||||
|
||||
Per-(provider, model) state (EWMA + bounded counters):
|
||||
|
||||
- `successEwma` — EWMA (α=0.2) of outcome success.
|
||||
- `latencyEwma` / `ttftEwma` — EWMA of latency (α=0.1).
|
||||
- `samples`, `anomalies`, `rateLimited`, `semantic`, `semanticConfidence`.
|
||||
- `recencyMs` — how recently the model was last observed.
|
||||
|
||||
### Confidence / sample awareness
|
||||
|
||||
`confidence = clamp01(samples / 50)`, and the score returned to the scorer is
|
||||
blended toward the neutral midpoint:
|
||||
|
||||
```
|
||||
score = 0.5 + confidence * (operational - 0.5)
|
||||
```
|
||||
|
||||
Consequences (verified by tests):
|
||||
|
||||
- A cold provider (0 samples) scores **0.5** — not unfairly penalized, but
|
||||
unable to dominate a provider with thousands of solid observations.
|
||||
- A provider with 7 lucky successes is pulled toward 0.5 (never dominates from
|
||||
optimistic initialization).
|
||||
- A provider with 50+ samples converges to its true operational score.
|
||||
- Degradation and recovery are gradual (EWMA), and one isolated failure does
|
||||
not destroy a healthy provider.
|
||||
|
||||
`ProviderQuality` exposes `{ operational, semantic, confidence, samples, anomalies,
|
||||
rateLimited, successEwma, latencyEwmaMs, ttftEwmaMs, recencyMs }`.
|
||||
|
||||
This feeds the auto-combo scorer as the `quality` scoring factor:
|
||||
|
||||
- `ScoringFactors.quality` / `ScoringWeights.quality` in
|
||||
`open-sse/services/autoCombo/scoring.ts`.
|
||||
- `DEFAULT_WEIGHTS`: `health` 0.1905 → 0.1605, `quality` 0.03. Sum stays 1.0.
|
||||
- `buildAutoCandidates` populates `candidate.quality` from the tracker; candidates
|
||||
without data default to neutral **0.5** (a cold candidate is neither boosted nor
|
||||
penalized).
|
||||
|
||||
The closed loop:
|
||||
|
||||
```
|
||||
RoutingEvent → QualityTracker → getQualityScore → auto-combo quality factor
|
||||
↑ │
|
||||
└────── request outcome (handleChatCore) ←────────────┘
|
||||
```
|
||||
|
||||
### Hard exclusion vs soft penalty
|
||||
|
||||
The quality signal is a **soft adaptive preference** only. Hard exclusion stays
|
||||
with the existing resilience stack: circuit breaker OPEN, quota exhausted,
|
||||
auth failure, model lockout — none of these are affected by the quality score.
|
||||
A provider whose quality score dips temporarily is de-preferenced, never
|
||||
hard-disabled.
|
||||
|
||||
## 3b. Canonical stream timing (TTFT / ITL)
|
||||
|
||||
Files: `open-sse/utils/streamTiming.ts`
|
||||
|
||||
`createStreamTiming()` is the single instrumentation seam for the streaming path,
|
||||
wired into `createSSEStream` (open-sse/utils/stream.ts):
|
||||
|
||||
- `markByte()` — first upstream chunk received.
|
||||
- `markForward()` — first chunk forwarded to the client (used for TTFT).
|
||||
- `markInterrupted()` — stream timeout/abort/error before a clean finish.
|
||||
- `ttft()` = first-forwarded-SSE-chunk latency. **This is NOT token-level TTFT** —
|
||||
a single SSE chunk may carry zero/one/many tokens. Documented precisely.
|
||||
- `avgItlMs()` = mean inter-chunk gap (a chunk-latency proxy for ITL).
|
||||
|
||||
TTFT/ITL/interrupted flow into the `RoutingEvent` (`ttftMs`, `itlMs`) and are
|
||||
exported as GenAI/OmniRoute span attributes by the OTel sink.
|
||||
|
||||
## 4. OpenTelemetry / GenAI observability
|
||||
|
||||
Files: `open-sse/services/routing/otel.ts`
|
||||
|
||||
- Dependency-free OTLP/HTTP JSON exporter (uses global `fetch`, no
|
||||
`@opentelemetry/*` SDK).
|
||||
- Spans follow GenAI semantic conventions (`gen_ai.provider.name`,
|
||||
`gen_ai.request.model`, `gen_ai.usage.input_tokens/output_tokens`,
|
||||
`gen_ai.completion.finish_reason`, `gen_ai.system`) plus OmniRoute routing
|
||||
attributes (outcome, status, ttft, retries, fallback).
|
||||
- `record()` only enqueues into a bounded buffer (O(1)); a background timer
|
||||
flushes via `POST {endpoint}/v1/traces` asynchronously. Under overload the
|
||||
oldest events are dropped (`dropped` counter) — never backpressure the data
|
||||
plane.
|
||||
- **Disabled unless configured.** `OMNIROUTE_OTEL_ENDPOINT` (or
|
||||
`OTEL_EXPORTER_OTLP_ENDPOINT`) must be set; otherwise the sink is not
|
||||
registered and zero OTel code runs.
|
||||
|
||||
## 5. Explainability
|
||||
|
||||
- `GET /v1/explain/routing` returns the recent `RoutingEvent`s (the real
|
||||
decisions, newest first) and the per-provider/model quality snapshot.
|
||||
- Auth mirrors `/v1/combos` (Bearer API key or dashboard session; anonymous on
|
||||
single-user local deployments with `REQUIRE_API_KEY=false`).
|
||||
- Combo-level per-invocation traces remain available via the existing
|
||||
`decisionTrace.ts` (header `X-OmniRoute-Combo-Trace`).
|
||||
- Safety: events carry only routing metadata, never prompts/bodies/credentials.
|
||||
|
||||
## 6. Evaluation-plane integration (Future AGI readiness)
|
||||
|
||||
OmniRoute treats Future AGI (or any evaluator) as a **potential
|
||||
intelligence/evaluation backend, not a dependency**. The seams:
|
||||
|
||||
- A `RoutingEventSink` can forward events to an evaluator asynchronously.
|
||||
- The `MemoryRoutingEventStore` + quality snapshot give an evaluator the raw
|
||||
decision stream.
|
||||
- A future `Evaluator` (deterministic, local judge, HTTP, WASM) would consume
|
||||
events/traces and return a `QualityScore` that feeds the same
|
||||
`getQualityScore`/quality-factor path.
|
||||
- Existing eval-driven routing (`open-sse/services/evalRouting.ts`) already
|
||||
re-orders combo targets by `eval_runs` pass-rates when enabled.
|
||||
|
||||
No evaluation runs synchronously on the request path, and the gateway operates
|
||||
fully with the evaluator absent.
|
||||
|
||||
## 7. Final architectural review
|
||||
|
||||
1. **What remains on the synchronous hot path?** Routing/scoring, guardrail
|
||||
pre-checks, cache lookup, and one `emitRoutingEvent` fan-out (~0.12 µs over
|
||||
baseline scoring) to in-memory sinks.
|
||||
2. **What moved to asynchronous processing?** OTel export (timer + fetch),
|
||||
`call_logs`/usage persistence, semantic-cache writes, quality is in-memory
|
||||
and O(1) (no async needed).
|
||||
3. **How does a routing outcome become feedback?** `handleChatCore` emits a
|
||||
`RoutingEvent` → `QualityTracker` updates EWMA state → `getQualityScore`
|
||||
feeds the auto-combo `quality` factor.
|
||||
4. **How does quality influence future routing?** A low quality score reduces
|
||||
the weighted score of that provider/model in `scoreAutoTargets`, so degraded
|
||||
models are gradually de-preferenced and recover as their EWMA improves.
|
||||
5. **How can Future AGI integrate without becoming a dependency?** Via the
|
||||
`RoutingEventSink` interface / a future `Evaluator` adapter — no hardcoded
|
||||
dependency.
|
||||
6. **What happens when the evaluator is unavailable?** Routing is unaffected;
|
||||
quality falls back to neutral (1.0) for models with no observed signal.
|
||||
7. **What happens when telemetry is unavailable?** The OTel sink simply isn't
|
||||
registered; the rest of the routing layer runs unchanged.
|
||||
8. **What happens under overload?** The OTel buffer drops oldest events; quality
|
||||
and the ring buffer are bounded by construction; no backpressure.
|
||||
9. **How does provider state recover after degradation?** EWMA re-converges as
|
||||
successes accumulate; warmup keeps cold models neutral; the circuit breaker
|
||||
independently recovers via HALF_OPEN probes.
|
||||
10. **Which proposed features were intentionally NOT implemented, and why?**
|
||||
- Shadow traffic / experiments — already implemented
|
||||
(`combo/shadowRouting.ts`); not re-built.
|
||||
- Guardrails — already implemented (`src/lib/guardrails/`); not duplicated.
|
||||
- Semantic cache — already implemented (`src/lib/semanticCache.ts`); not
|
||||
duplicated.
|
||||
- A full experiment-management platform, dataset tooling, prompt-optimization
|
||||
platform, vector DB, or mandatory external OTel infrastructure — out of
|
||||
scope for a lean data plane.
|
||||
- A Rust `RoutingEvent` struct — the data plane is TypeScript; the TS type
|
||||
is the adapted equivalent.
|
||||
|
||||
## 8. Configuration reference
|
||||
|
||||
| Variable | Default | Effect |
|
||||
| ----------------------------- | ----------- | ------------------------------------------------------------------------------- |
|
||||
| `OMNIROUTE_OTEL_ENDPOINT` | unset | When set, enables the OTLP/HTTP traces exporter (e.g. `http://collector:4318`). |
|
||||
| `OTEL_EXPORTER_OTLP_ENDPOINT` | unset | Fallback alias for the OTLP endpoint. |
|
||||
| `OTEL_SERVICE_NAME` | `omniroute` | `service.name` resource attribute. |
|
||||
|
||||
## 9. Tests
|
||||
|
||||
- `tests/unit/routing-events.test.ts` — event normalization, status
|
||||
classification, bounded ring buffer, sink fan-out + isolation.
|
||||
- `tests/unit/routing-quality.test.ts` — EWMA warmup, failure/success recovery,
|
||||
anomaly penalties, 429 transient handling, snapshot, reset.
|
||||
- `tests/unit/routing-scoring-quality.test.ts` — weight integrity, neutral
|
||||
default, quality factor ranking.
|
||||
- `tests/unit/routing-otel.test.ts` — enable gating, GenAI span payload, async
|
||||
flush, drop-under-overload.
|
||||
- `tests/unit/routing-events-concurrency.test.ts` — thousands of events, ring
|
||||
buffer boundedness, throwing-sink isolation, interleaved async bursts,
|
||||
reset-during-inserts.
|
||||
- `tests/unit/routing-adaptive-e2e.test.ts` — deterministic end-to-end loop via
|
||||
the real `scoreAutoTargets` scorer: healthy → degrade → recover → blip, plus
|
||||
cold-start and lucky-cold-provider scenarios.
|
||||
- `tests/unit/stream-timing.test.ts` — TTFT (first-forwarded-chunk), ITL,
|
||||
first-byte vs first-forward, interruption, malformed/empty chunk safety.
|
||||
|
||||
## 10. Pre-existing issues status (Phase 18)
|
||||
|
||||
| Issue | Status | Notes |
|
||||
| ----------------------------------------------------- | ------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `omniglyph` export mismatch | **FIXED (environmental)** | `node_modules` was out of sync with `package-lock.json` (installed 1.3.1 vs locked 1.4.0). Running `npm install omniglyph@1.4.0` restored the locked version; type errors dropped to 0. Manifests unchanged. |
|
||||
| Stale `getKnownContextOverflow` tests | **KNOWN — not fixed** | `combo-context-overflow-compression-probe.test.ts` imports a function that no longer exists in `open-sse/services/combo.ts` (only comments reference it). Fixing requires re-implementing or re-writing those tests — unrelated architectural churn. |
|
||||
| `combo-runtime-unit-concurrency.test.ts` DB isolation | **KNOWN — not fixed** | Test-harness SQLite-isolation assertion fails when run directly; fails identically on the base branch. |
|
||||
| i18n `llm.txt` drift | **KNOWN — not fixed** | `docs/i18n/*/llm.txt` differ from root; pre-existing, blocks the docs-sync pre-commit gate. |
|
||||
|
||||
Environmental vs code issues are kept distinct; no unrelated failures are hidden
|
||||
behind changed test filters.
|
||||
@@ -89,7 +89,6 @@ src/
|
||||
├── i18n/ Locale bundles
|
||||
├── instrumentation.ts Next.js instrumentation hook
|
||||
├── instrumentation-node.ts
|
||||
├── server-init.ts Process-level bootstrap (env, DB, jobs, sync)
|
||||
└── proxy.ts Top-level proxy bootstrap helper
|
||||
```
|
||||
|
||||
|
||||
@@ -133,7 +133,6 @@ src/
|
||||
├── types/ # Shared TS type files
|
||||
├── instrumentation.ts # Next.js telemetry hook (browser + edge)
|
||||
├── instrumentation-node.ts # Node-only instrumentation
|
||||
├── server-init.ts # Server bootstrap (DB migrations, jobs, cleanup)
|
||||
└── proxy.ts # HTTP-proxy entry shim
|
||||
```
|
||||
|
||||
|
||||
@@ -12,6 +12,7 @@
|
||||
"ROUTER_BACKENDS",
|
||||
"admission-lanes",
|
||||
"cluster-decisions",
|
||||
"persistence-backend-boundary"
|
||||
"persistence-backend-boundary",
|
||||
"ADAPTIVE_ROUTING"
|
||||
]
|
||||
}
|
||||
|
||||
@@ -28,12 +28,12 @@ The `omniglyph` engine (package `omniglyph`, 1.4.0+) accepts a named semantic pr
|
||||
globally through `omniglyph.profile` in the compression settings or per step through the
|
||||
stacked pipeline's step config:
|
||||
|
||||
| Profile | Boundary |
|
||||
| -------------- | --------------------------------------------------------------------------- |
|
||||
| `aggressive` | Default. The policy the published receipts measured — images system, tool docs and dense history |
|
||||
| `balanced` | Keeps live state native, protects the last 8 turns, collapses older closed history |
|
||||
| `coding-safe` | Keeps authority, tool schemas and live tool output native, protects the last 12 turns |
|
||||
| `passthrough` | Routes without transforming; the engine is skipped |
|
||||
| Profile | Boundary |
|
||||
| ------------- | ------------------------------------------------------------------------------------------------ |
|
||||
| `aggressive` | Default. The policy the published receipts measured — images system, tool docs and dense history |
|
||||
| `balanced` | Keeps live state native, protects the last 8 turns, collapses older closed history |
|
||||
| `coding-safe` | Keeps authority, tool schemas and live tool output native, protects the last 12 turns |
|
||||
| `passthrough` | Routes without transforming; the engine is skipped |
|
||||
|
||||
The profile is a **ceiling, not a floor**: `mergeCompressionProfileOptions` in the package
|
||||
refuses to let a caller override reopen a lossy lane the profile closed, so a per-step
|
||||
@@ -170,22 +170,22 @@ override points it at a local copy instead (offline / air-gapped installs).
|
||||
|
||||
### Optional dependencies & on-demand install
|
||||
|
||||
The prunable LLMLingua runtime peer stack is **optional**. Three packages are declared as
|
||||
The prunable LLMLingua runtime peer stack is **optional**. Two packages are declared as
|
||||
`optionalDependencies` in `package.json` and kept **external** by the production build
|
||||
(`scripts/build/prepublish.ts` does not bundle them):
|
||||
|
||||
| Package | Version (pin) | Notes |
|
||||
| -------------------- | ------------- | ---------------------------------------------- |
|
||||
| `@atjsh/llmlingua-2` | `2.0.3` | Entry package; declares the others as peers |
|
||||
| `@tensorflow/tfjs` | `4.22.0` | Heaviest dep — dominates the ~800 MB footprint |
|
||||
| `js-tiktoken` | `^1.0.20` | Tokenizer |
|
||||
| Package | Version (pin) | Notes |
|
||||
| -------------------- | ------------- | ------------------------------------------- |
|
||||
| `@atjsh/llmlingua-2` | `2.0.5` | Entry package; declares the others as peers |
|
||||
| `js-tiktoken` | `^1.0.20` | Tokenizer |
|
||||
|
||||
`@huggingface/transformers` is pinned at `3.5.2` as an **optional** dependency (shared with
|
||||
the local embeddings path and also traced into the standalone bundle). Keeping it optional prevents
|
||||
`onnxruntime-node` CUDA provider postinstall failures on CUDA 11 hosts from aborting the whole
|
||||
OmniRoute install; when the optional stack is absent, LLMLingua still fail-opens. Only the three
|
||||
packages above are prunable SLM peers. A standard `npm install` (dev) installs the optional stack
|
||||
automatically unless optional dependencies are omitted.
|
||||
`@huggingface/transformers` is pinned at `^4.2.0` (shared with the local embeddings path and
|
||||
also traced into the standalone bundle); `@atjsh/llmlingua-2@2.0.5` peers on it with
|
||||
`"^3.5.2 || ^4.0.0"`, so both Transformers.js v3 and v4 are supported. Since 2.0.4,
|
||||
`@atjsh/llmlingua-2` no longer requires `@tensorflow/tfjs`, which removed the largest single
|
||||
contributor (TensorFlow.js) from the SLM stack. Only the two packages above are prunable SLM
|
||||
peers. A standard `npm install` (dev) installs the optional stack automatically unless optional
|
||||
dependencies are omitted.
|
||||
|
||||
**Why on-demand:** the npm-published package, the standalone bundle, and the Docker image
|
||||
ship **without** these deps to stay slim. When they are absent, the worker's dependency
|
||||
@@ -195,11 +195,12 @@ error logged). To activate it in a pruned environment, install the optional stac
|
||||
|
||||
```bash
|
||||
# pin to the versions declared in package.json optionalDependencies
|
||||
npm install @atjsh/llmlingua-2@2.0.3 @tensorflow/tfjs@4.22.0 js-tiktoken
|
||||
npm install @atjsh/llmlingua-2@2.0.5 js-tiktoken
|
||||
```
|
||||
|
||||
Roughly **~800 MB** total: the TensorFlow.js + transformers runtimes dominate; the
|
||||
TinyBERT model adds ~57 MB downloaded at first use (not via npm).
|
||||
The `@tensorflow/tfjs` removal (2.0.4+) eliminates the previously dominant ~800 MB
|
||||
contributor — the remaining footprint is the transformers.js + onnxruntime-node runtimes,
|
||||
plus the TinyBERT model (~57 MB) downloaded at first use (not via npm).
|
||||
|
||||
Per environment:
|
||||
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
<svg viewBox="0 0 1200 350" xmlns="http://www.w3.org/2000/svg" role="img" aria-label="Animated terminal demoing the OmniRoute CLI: omniroute providers list (342 providers registered, anthropic, codex, glm, kimi shown active), omniroute combo list (always-on priority, cost-saver, fusion-panel, context-relay) and omniroute health (healthy, 18412 requests in 24h, p95 412ms, circuit breakers 24 closed, 1 half-open, 0 open), cycling over the 80+ command surface: providers, oauth, keys, combo, nodes, models, cache, compression, cost, usage, quota, health, resilience, telemetry, logs, audit, mcp, a2a, cloud, memory, skills, eval, doctor, repl, tunnel, backup, sync, webhooks, policy, pricing, translator, simulate and more.">
|
||||
<svg viewBox="0 0 1200 350" xmlns="http://www.w3.org/2000/svg" role="img" aria-label="Animated terminal demoing the OmniRoute CLI: omniroute providers list (343 providers registered, anthropic, codex, glm, kimi shown active), omniroute combo list (always-on priority, cost-saver, fusion-panel, context-relay) and omniroute health (healthy, 18412 requests in 24h, p95 412ms, circuit breakers 24 closed, 1 half-open, 0 open), cycling over the 80+ command surface: providers, oauth, keys, combo, nodes, models, cache, compression, cost, usage, quota, health, resilience, telemetry, logs, audit, mcp, a2a, cloud, memory, skills, eval, doctor, repl, tunnel, backup, sync, webhooks, policy, pricing, translator, simulate and more.">
|
||||
<desc>Compact animated terminal cycling three real OmniRoute CLI commands with a typewriter effect and a scrolling subcommand ticker; the first frame shows the completed providers-list screen.</desc>
|
||||
<defs><clipPath id="tickerClip"><rect x="12" y="304" width="1176" height="40"/></clipPath><clipPath id="tw0"><rect x="64" y="46" height="26" width="0"><animate attributeName="width" calcMode="discrete" values="0;31;61;92;122;153;184;214;245;245" keyTimes="0;0.012;0.018;0.024;0.030;0.036;0.042;0.048;0.054;1" dur="18s" repeatCount="indefinite"/></rect></clipPath><clipPath id="tw1"><rect x="64" y="46" height="26" width="0"><animate attributeName="width" calcMode="discrete" values="0;26;51;76;102;128;153;178;204;204" keyTimes="0;0.345;0.351;0.357;0.363;0.369;0.375;0.381;0.387;1" dur="18s" repeatCount="indefinite"/></rect></clipPath><clipPath id="tw2"><rect x="64" y="46" height="26" width="0"><animate attributeName="width" calcMode="discrete" values="0;20;41;61;82;102;122;143;163;163" keyTimes="0;0.678;0.684;0.690;0.696;0.702;0.708;0.714;0.720;1" dur="18s" repeatCount="indefinite"/></rect></clipPath></defs>
|
||||
<rect width="1200" height="350" fill="#0d1117"/>
|
||||
|
||||
|
Before Width: | Height: | Size: 12 KiB After Width: | Height: | Size: 12 KiB |
@@ -1,4 +1,4 @@
|
||||
<svg viewBox="0 0 1200 780" xmlns="http://www.w3.org/2000/svg" role="img" aria-label="Comparison table: OmniRoute versus 9router, OpenRouter, CLIProxyAPI and LiteLLM across 13 capabilities. OmniRoute is the only one with the full set: 342 providers, 90+ free providers built-in, 19 routing strategies, 12-engine token compression, a built-in MCP server with 109 tools, A2A protocol, persistent memory, guardrails, cloud agents, TLS fingerprint stealth, desktop/Termux/PWA, 43 UI locales and 100% MIT self-hosted. 9router has free providers, RTK compression and translation but no MCP, A2A, memory, guardrails, cloud agents or stealth. OpenRouter is a hosted SaaS with 400+ models, guardrails and a hosted MCP but is not self-hosted and lacks A2A, memory, cloud agents and stealth. CLIProxyAPI is a light OAuth proxy with two routing strategies. LiteLLM has 100+ providers, A2A and extensive guardrails but no memory, compression, free tier, stealth or cloud agents.">
|
||||
<svg viewBox="0 0 1200 780" xmlns="http://www.w3.org/2000/svg" role="img" aria-label="Comparison table: OmniRoute versus 9router, OpenRouter, CLIProxyAPI and LiteLLM across 13 capabilities. OmniRoute is the only one with the full set: 343 providers, 90+ free providers built-in, 19 routing strategies, 12-engine token compression, a built-in MCP server with 109 tools, A2A protocol, persistent memory, guardrails, cloud agents, TLS fingerprint stealth, desktop/Termux/PWA, 43 UI locales and 100% MIT self-hosted. 9router has free providers, RTK compression and translation but no MCP, A2A, memory, guardrails, cloud agents or stealth. OpenRouter is a hosted SaaS with 400+ models, guardrails and a hosted MCP but is not self-hosted and lacks A2A, memory, cloud agents and stealth. CLIProxyAPI is a light OAuth proxy with two routing strategies. LiteLLM has 100+ providers, A2A and extensive guardrails but no memory, compression, free tier, stealth or cloud agents.">
|
||||
<desc>Static-header comparison table where each capability row fades in top to bottom; the OmniRoute column is highlighted and shows a check or a leading value in every row, while competitors show a mix of checks, partials and crosses.</desc>
|
||||
<defs>
|
||||
<pattern id="gC" width="32" height="32" patternUnits="userSpaceOnUse"><path d="M 32 0 L 0 0 0 32" fill="none" stroke="#ffffff" stroke-opacity="0.05" stroke-width="1"/></pattern>
|
||||
|
||||
|
Before Width: | Height: | Size: 13 KiB After Width: | Height: | Size: 13 KiB |
@@ -1,4 +1,4 @@
|
||||
<svg viewBox="0 0 1200 540" xmlns="http://www.w3.org/2000/svg" role="img" aria-label="The OmniRoute promise: one endpoint, 342 providers — never stop building, OmniRoute picks the cheapest one that works. Six pillars. Never hit limits: auto-fallback across 342 providers in milliseconds, quota out means the next provider takes over with zero downtime. Save up to 95 percent of tokens: RTK plus Caveman stacked compression cuts 15 to 95 percent of eligible tokens, about 89 percent average on tool-heavy sessions. Zero dollars to start: 90+ providers with a free tier, 56 free forever — Qoder, Pollinations, Cloudflare, SiliconFlow — no card needed. Every tool works: 33 coding agents including Claude Code, Codex, Cursor, Cline, Copilot and Antigravity through one config. One endpoint: OpenAI, Claude, Gemini and Responses API translation — point any tool at /v1 and it just works. Production-grade: circuit breakers, TLS stealth, MCP with 109 tools, A2A, memory, guardrails, evals — 25,000+ tests.">
|
||||
<svg viewBox="0 0 1200 540" xmlns="http://www.w3.org/2000/svg" role="img" aria-label="The OmniRoute promise: one endpoint, 343 providers — never stop building, OmniRoute picks the cheapest one that works. Six pillars. Never hit limits: auto-fallback across 343 providers in milliseconds, quota out means the next provider takes over with zero downtime. Save up to 95 percent of tokens: RTK plus Caveman stacked compression cuts 15 to 95 percent of eligible tokens, about 89 percent average on tool-heavy sessions. Zero dollars to start: 90+ providers with a free tier, 56 free forever — Qoder, Pollinations, Cloudflare, SiliconFlow — no card needed. Every tool works: 33 coding agents including Claude Code, Codex, Cursor, Cline, Copilot and Antigravity through one config. One endpoint: OpenAI, Claude, Gemini and Responses API translation — point any tool at /v1 and it just works. Production-grade: circuit breakers, TLS stealth, MCP with 109 tools, A2A, memory, guardrails, evals — 25,000+ tests.">
|
||||
<desc>Animated promise card: six pillar tiles fade in in reading order, then a soft colored border highlight sweeps from tile to tile in a continuous cycle.</desc>
|
||||
<defs>
|
||||
<pattern id="gridPaperP" width="32" height="32" patternUnits="userSpaceOnUse">
|
||||
@@ -21,7 +21,7 @@
|
||||
<line x1="150" y1="53" x2="1160" y2="53" stroke="#232b38" stroke-width="1.5"/>
|
||||
</g>
|
||||
<g>
|
||||
<text x="40" y="100" font-family="Inter, 'Segoe UI', Arial, Helvetica, system-ui, sans-serif" font-size="23" font-weight="600" fill="#c9d1d9">One endpoint. <tspan fill="#a78bfa" font-weight="800">342 providers.</tspan> Never stop building — OmniRoute picks <tspan fill="#7ee787" font-weight="700">the cheapest one that works</tspan>.</text>
|
||||
<text x="40" y="100" font-family="Inter, 'Segoe UI', Arial, Helvetica, system-ui, sans-serif" font-size="23" font-weight="600" fill="#c9d1d9">One endpoint. <tspan fill="#a78bfa" font-weight="800">343 providers.</tspan> Never stop building — OmniRoute picks <tspan fill="#7ee787" font-weight="700">the cheapest one that works</tspan>.</text>
|
||||
</g>
|
||||
|
||||
<g font-family="Inter, 'Segoe UI', Arial, Helvetica, system-ui, sans-serif">
|
||||
@@ -38,7 +38,7 @@
|
||||
<line x1="3.9" y1="3.9" x2="18.1" y2="18.1"/>
|
||||
</g>
|
||||
<text x="102" y="170" font-size="18" font-weight="800" fill="#74b9ff">Never hit limits</text>
|
||||
<text x="66" y="204" font-size="13.5" fill="#a1a1aa">Auto-fallback across 342 providers in</text>
|
||||
<text x="66" y="204" font-size="13.5" fill="#a1a1aa">Auto-fallback across 343 providers in</text>
|
||||
<text x="66" y="226" font-size="13.5" fill="#a1a1aa">milliseconds. Quota out? The next provider</text>
|
||||
<text x="66" y="248" font-size="13.5" fill="#a1a1aa">takes over — zero downtime.</text>
|
||||
</g>
|
||||
|
||||
|
Before Width: | Height: | Size: 10 KiB After Width: | Height: | Size: 10 KiB |
@@ -1,4 +1,4 @@
|
||||
<svg viewBox="0 0 1200 548" xmlns="http://www.w3.org/2000/svg" role="img" aria-label="OmniRoute hero: Never stop coding. Every AI tool to 342 providers — 90+ free — through one endpoint. Claude Code, Codex, Cursor, Cline, Copilot and Antigravity into free Claude, GPT and Gemini with auto-fallback. RTK + Caveman stacked compression saves 15 to 95 percent of tokens — about 89 percent average on tool-heavy sessions — so you never hit limits. Stats: 342 AI providers, 90+ free tiers, about 1.51B free tokens per month, 15 to 95 percent token savings, 19 routing strategies, zero dollars to start.">
|
||||
<svg viewBox="0 0 1200 548" xmlns="http://www.w3.org/2000/svg" role="img" aria-label="OmniRoute hero: Never stop coding. Every AI tool to 343 providers — 90+ free — through one endpoint. Claude Code, Codex, Cursor, Cline, Copilot and Antigravity into free Claude, GPT and Gemini with auto-fallback. RTK + Caveman stacked compression saves 15 to 95 percent of tokens — about 89 percent average on tool-heavy sessions — so you never hit limits. Stats: 343 AI providers, 90+ free tiers, about 1.51B free tokens per month, 15 to 95 percent token savings, 19 routing strategies, zero dollars to start.">
|
||||
<desc>Animated hero card: a pulse travels the divider line and a compression bar demo repeatedly shrinks a prompt by up to 95 percent; all headline content is static and readable on the first frame.</desc>
|
||||
<defs>
|
||||
<pattern id="gridPaperH" width="32" height="32" patternUnits="userSpaceOnUse">
|
||||
@@ -28,7 +28,7 @@
|
||||
<text x="48" y="138" font-family="Inter, 'Segoe UI', Arial, Helvetica, system-ui, sans-serif" font-size="60" font-weight="800" fill="#e9edf3">Never stop coding<tspan fill="#a855f7">.</tspan></text>
|
||||
|
||||
<!-- subheadline -->
|
||||
<text x="48" y="184" font-family="Inter, 'Segoe UI', Arial, Helvetica, system-ui, sans-serif" font-size="25" font-weight="600" fill="#c9d1d9">Every AI tool → <tspan fill="#a78bfa" font-weight="800">342 providers</tspan> — <tspan fill="#7ee787" font-weight="800">90+ free</tspan> — through one endpoint.</text>
|
||||
<text x="48" y="184" font-family="Inter, 'Segoe UI', Arial, Helvetica, system-ui, sans-serif" font-size="25" font-weight="600" fill="#c9d1d9">Every AI tool → <tspan fill="#a78bfa" font-weight="800">343 providers</tspan> — <tspan fill="#7ee787" font-weight="800">90+ free</tspan> — through one endpoint.</text>
|
||||
|
||||
<!-- plug line -->
|
||||
<text x="48" y="222" font-family="Inter, 'Segoe UI', Arial, Helvetica, system-ui, sans-serif" font-size="16.5" fill="#a1a1aa">Claude Code · Codex · Cursor · Cline · Copilot · Antigravity  →  <tspan fill="#7ee787" font-weight="700">FREE</tspan> Claude / GPT / Gemini · auto-fallback</text>
|
||||
|
||||
|
Before Width: | Height: | Size: 7.3 KiB After Width: | Height: | Size: 7.3 KiB |
@@ -347,6 +347,8 @@ per-key path take precedence once it is. stdio has no per-caller identity (see
|
||||
| `OMNIROUTE_MCP_SCOPES` | (empty) | Comma-separated allowlist of scopes considered "available" by default (used when caller does not provide its own scopes) |
|
||||
| `OMNIROUTE_MCP_COMPRESS_DESCRIPTIONS` | (unset = on) | When set to `0/false/off/no`, disables MCP description compression at registration time |
|
||||
| `OMNIROUTE_MCP_DESCRIPTION_COMPRESSION` | (unset = on) | Alternate alias for the same toggle as above |
|
||||
| `OMNIROUTE_MCP_FETCH_TIMEOUT_MS` | `10000` | Abort budget for internal management reads (health, resilience, combos, quota, usage) |
|
||||
| `OMNIROUTE_MCP_UPSTREAM_TIMEOUT_MS` | `60000` | Abort budget for hops that wait on a provider (`route_request`, `web_search`, `web_fetch`) |
|
||||
| `MCP_TOOL_DENY` | (unset = no filter) | Comma-separated tool names to drop from `tools/list` (tool-cardinality reduction — see below) |
|
||||
| `MCP_TOOL_ALLOW` | (unset = no filter) | Comma-separated tool names to keep exclusively (allow-list mode — see below) |
|
||||
| `DATA_DIR` | `~/.omniroute` | Heartbeat file is written to `${DATA_DIR}/runtime/mcp-heartbeat.json` |
|
||||
|
||||
@@ -52,6 +52,8 @@ safely retry only the failures after a partial result.
|
||||
- **Pollinations** — Free GPT-5, Claude, Gemini (no key needed)
|
||||
- **LongCat** — 10M tokens free (one-time grant, requires account + KYC)
|
||||
- **Cloudflare AI** — 50+ models, 10K neurons/day
|
||||
- **MLX Gemma 26B** — Local Apple Silicon model (~38.5 tok/s, ~15.9GB RAM)
|
||||
- **MLX Qwen 3.8 27B** — Local Apple Silicon model (~9.1 tok/s, ~13.1GB RAM)
|
||||
4. Click **Connect**
|
||||
5. Done! You now have free AI access.
|
||||
|
||||
@@ -79,6 +81,94 @@ safely retry only the failures after a partial result.
|
||||
5. Login with your account
|
||||
6. Done! You now have access to your subscription models.
|
||||
|
||||
### Option D: Local MLX Models (Apple Silicon)
|
||||
|
||||
For Apple Silicon Macs with unified memory, OmniRoute supports connecting to local MLX models running via `mlx-lm.server` as regular OpenAI-compatible local providers.
|
||||
|
||||
#### Prerequisites
|
||||
|
||||
- **Apple Silicon Mac** (M1/M2/M3/M4) with 24GB+ unified memory recommended
|
||||
- **uv** package manager: `curl -LsSf https://astral.sh/uv/install.sh | sh`
|
||||
- **mlx-lm**: `uv pip install mlx-lm`
|
||||
|
||||
#### Quick Start
|
||||
|
||||
1. **Install dependencies**:
|
||||
|
||||
```bash
|
||||
# Install uv if not already installed
|
||||
curl -LsSf https://astral.sh/uv/install.sh | sh
|
||||
|
||||
# Install mlx-lm
|
||||
uv pip install mlx-lm
|
||||
```
|
||||
|
||||
2. **Start MLX servers manually** (in separate terminals):
|
||||
|
||||
```bash
|
||||
# Terminal 1: Gemma 4 26B A4B IT-QAT (port 11435)
|
||||
uv run mlx_lm.server --model mlx-community/gemma-4-26B-A4B-it-qat-q4_0-mlx-aligned --port 11435 --host 127.0.0.1
|
||||
|
||||
# Terminal 2: Qwen 3.8 27B MLX Mixed (port 11436)
|
||||
uv run mlx_lm.server --model maglun/Qwen3.8-27B-MLX-Mixed-3.80bpw --port 11436 --host 127.0.0.1
|
||||
```
|
||||
|
||||
3. **Connect in OmniRoute Dashboard**:
|
||||
- Go to **Providers** → **Add Provider**
|
||||
- Select **MLX Gemma 26B** or **MLX Qwen 3.8 27B**
|
||||
- Click **Connect** (no API key needed)
|
||||
|
||||
4. **Use with OpenCode**:
|
||||
```bash
|
||||
# Configure OpenCode to use OmniRoute
|
||||
opencode config set api.base_url http://localhost:20128/v1
|
||||
opencode config set api.key <your-omniroute-api-key>
|
||||
|
||||
# Use MLX models
|
||||
opencode run --model mlx-gemma/gemma-4-26b
|
||||
opencode run --model mlx-qwen/qwen3.8-27b
|
||||
```
|
||||
|
||||
#### Memory Management
|
||||
|
||||
**Important**: With 24GB unified memory, only **one large MLX model can run at a time**.
|
||||
|
||||
- Gemma 26B: ~15.9GB peak memory
|
||||
- Qwen 3.8 27B: ~13.1GB peak memory
|
||||
|
||||
You must manage this manually:
|
||||
|
||||
- Run only one MLX server at a time, or
|
||||
- Run both on separate machines, or
|
||||
- Stop one before starting the other
|
||||
|
||||
OmniRoute does not automatically manage MLX server processes — it only routes requests to the OpenAI-compatible endpoints you configure.
|
||||
|
||||
#### Tool Calling Support
|
||||
|
||||
Both models support OpenAI-compatible tool calling. Test with:
|
||||
|
||||
```bash
|
||||
curl -X POST http://localhost:20128/v1/chat/completions \
|
||||
-H "Authorization: Bearer <key>" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"model": "mlx-gemma/gemma-4-26b",
|
||||
"messages": [{"role": "user", "content": "What is 2+2? Use the calculator tool."}],
|
||||
"tools": [{"type": "function", "function": {"name": "calculator", "description": "Calculate", "parameters": {"type": "object", "properties": {"expression": {"type": "string"}}, "required": ["expression"]}}}]
|
||||
}'
|
||||
```
|
||||
|
||||
#### Troubleshooting
|
||||
|
||||
| Issue | Solution |
|
||||
| ------------------ | ----------------------------------------------------------------------------- |
|
||||
| Server won't start | Check `uv run mlx_lm.server --help` and verify model IDs |
|
||||
| Out of memory | Ensure only one model runs; close other apps; check Activity Monitor |
|
||||
| Connection refused | Verify server is running on correct port (11435/11436) |
|
||||
| Slow responses | First request loads model into memory (~30-60s); subsequent requests are fast |
|
||||
| Tool calling fails | Ensure model supports tools; check OmniRoute logs for translation errors |
|
||||
|
||||
---
|
||||
|
||||
## Best Free Providers
|
||||
@@ -239,3 +329,7 @@ Go to Providers → click on the provider → click **Disconnect**.
|
||||
- **[Free Tiers Guide](./FREE-TIERS-GUIDE.md)** — Get free AI with no credit card
|
||||
- **[Troubleshooting](../guides/TROUBLESHOOTING.md)** — Fix common issues
|
||||
- **[Provider Reference](../reference/PROVIDER_REFERENCE.md)** — Full list of 226 providers
|
||||
|
||||
## Cursor images
|
||||
|
||||
Cursor plan images use `IMAGE_PROVIDERS.cursor` (`cursor-agent-image`). See [CURSOR_IMAGE.md](../providers/CURSOR_IMAGE.md).
|
||||
|
||||
@@ -26,6 +26,8 @@ npm install -g omniroute
|
||||
docker run -d --name omniroute -p 20128:20128 diegosouzapw/omniroute:latest
|
||||
```
|
||||
|
||||
`:latest` is the highest **published** stable SemVer. It does **not** track git `main`. Pin `diegosouzapw/omniroute:X.Y.Z` for GitOps. See [Image Tags / Release Channels](../guides/DOCKER_GUIDE.md#release-channels).
|
||||
|
||||
### Option C: From Source
|
||||
|
||||
```bash
|
||||
|
||||
@@ -22,6 +22,7 @@ lastUpdated: 2026-06-28
|
||||
- [Docker Compose with Caddy (HTTPS)](#docker-compose-with-caddy-https-auto-tls)
|
||||
- [Cloudflare Quick Tunnel](#cloudflare-quick-tunnel)
|
||||
- [Image Tags](#image-tags)
|
||||
- [Availability: default SQLite is single-replica](#availability-default-sqlite-is-single-replica)
|
||||
- [Important Notes](#important-notes)
|
||||
|
||||
---
|
||||
@@ -341,13 +342,15 @@ For orchestrators (Kubernetes, Nomad, etc.):
|
||||
|
||||
| Probe | Prefer | Avoid |
|
||||
| --- | --- | --- |
|
||||
| Liveness | TCP on the main port (`PORT`, default `20128`), or soft HTTP `/healthz` | `/api/monitoring/health` as liveness |
|
||||
| Liveness | HTTP `GET /livez`, or TCP on the main port (`PORT`, default `20128`) | `/api/monitoring/health` as liveness |
|
||||
| Readiness | HTTP `GET /healthz` | Tight timeouts that treat event-loop busy as dead |
|
||||
| Deep / blackbox | `/api/monitoring/health` | — |
|
||||
|
||||
`/healthz` only reports process lifecycle (`ok` / `starting` / `stopping`). It still
|
||||
runs on the same Node event loop as request handling, so CPU-bound catalog or
|
||||
compression work can delay it — busy ≠ dead. Full probe guidance:
|
||||
`/healthz` reports process lifecycle (`ok` / `starting` / `stopping`). `/livez` is
|
||||
process-alive only (200 whenever the handler can run; it does not wait for
|
||||
readiness). Both still run on the same Node event loop as request handling, so
|
||||
CPU-bound catalog or compression work can delay them — busy ≠ dead. Prefer TCP
|
||||
liveness if HTTP probes time out. Full probe guidance:
|
||||
[Monitoring guide — Kubernetes probe recommendations](../ops/MONITORING_GUIDE.md#kubernetes-probe-recommendations).
|
||||
|
||||
## Docker Compose with Caddy (HTTPS Auto-TLS)
|
||||
@@ -409,8 +412,8 @@ Endpoint tunnel panels (Cloudflare, Tailscale, ngrok) can be shown or hidden fro
|
||||
|
||||
| Image | Tag | Size | Description |
|
||||
| ------------------------ | -------- | ------ | --------------------- |
|
||||
| `diegosouzapw/omniroute` | `latest` | ~250MB | Latest stable release |
|
||||
| `diegosouzapw/omniroute` | `3.8.0` | ~250MB | Current version |
|
||||
| `diegosouzapw/omniroute` | `latest` | ~250MB | Highest **published** stable SemVer (not git `main`) |
|
||||
| `diegosouzapw/omniroute` | `3.8.0` | ~250MB | Pin this class of tag for GitOps |
|
||||
|
||||
Multi-platform manifest: `linux/amd64` + `linux/arm64` native (Apple Silicon, AWS Graviton, Raspberry Pi). Docker selects the matching architecture automatically; pass `--platform linux/amd64` if you need to force AMD64 emulation on ARM hosts.
|
||||
|
||||
@@ -421,7 +424,7 @@ OmniRoute publishes separate Docker channels for stable releases, active release
|
||||
| Channel | Source | Mutability | Recommended use |
|
||||
| ------------------------------- | ----------------------------------- | --------------------------- | ----------------------------------------------------------------------------------------------- |
|
||||
| `:<version>` / `:<version>-web` | Signed/versioned release | Immutable | Production deployments that pin an exact release |
|
||||
| `:latest` / `:latest-web` | Highest stable release | Mutable stable pointer | Production deployments that intentionally follow stable releases |
|
||||
| `:latest` / `:latest-web` | Highest **published** stable SemVer | Mutable stable pointer | Follows stable releases **after** a SemVer publish job — does **not** track `main` or unreleased `release/v*` commits |
|
||||
| `:next` / `:next-web` | Current default `release/v*` branch | Mutable pre-release pointer | Testing fixes that have landed on the active release branch but are not yet in a stable release |
|
||||
| `:main` / `:main-web` | `main` branch | Mutable development pointer | Development and integration testing only |
|
||||
|
||||
@@ -465,6 +468,37 @@ docker compose up -d
|
||||
|
||||
A release-branch build can never move `latest`; only an eligible stable semantic version may promote the stable pointer. The `next` images retain the release image inspection and blocking CRITICAL-vulnerability gate.
|
||||
|
||||
**`latest` is not a currency guarantee for git.** Merged fixes on `main` or on the active `release/v*` branch are **not** in `:latest` until a stable SemVer image is published and the publish job promotes `:latest` (same digest as that SemVer). If `latest` looks frozen while GitHub already shows the fix, pull `:next` to test the release branch or wait for the SemVer tag.
|
||||
|
||||
| You want | Use |
|
||||
| --- | --- |
|
||||
| GitOps / production that must not drift | Pin `:X.Y.Z` (or the image digest) |
|
||||
| Follow published stables and accept a recreate on each release | `:latest` |
|
||||
| Test unreleased `release/v*` commits | `:next` (not production) |
|
||||
| Test `main` | `:main` (not production) |
|
||||
|
||||
## Availability: default SQLite is single-replica
|
||||
|
||||
Stock Docker / Kubernetes OmniRoute is **one Node process + one SQLite writer**. High availability is **not supported** on that topology.
|
||||
|
||||
| Constraint | Consequence |
|
||||
| --- | --- |
|
||||
| Single writer | Do **not** run multiple replicas against the same SQLite file. That corrupts the DB. |
|
||||
| Recreate / restart / HEALTHCHECK kill | **Full outage** of in-flight SSE, dashboard sessions, and in-memory state. Every connected client drops. |
|
||||
| Same event loop as `/healthz` | A busy catalog or compression tick can delay probes; a short timeout then restarts the **only** replica. |
|
||||
|
||||
**Probe matrix** (see also [Kubernetes probe recommendations](../ops/MONITORING_GUIDE.md#kubernetes-probe-recommendations)):
|
||||
|
||||
| Probe | Target | Do not use |
|
||||
| --- | --- | --- |
|
||||
| Liveness | TCP on `PORT` (default `20128`), or soft HTTP `/healthz` | `/api/monitoring/health` |
|
||||
| Readiness | HTTP `GET /healthz` | Tight timeouts that treat event-loop busy as dead |
|
||||
| Deep / humans | `/api/monitoring/health` | Automated kubelet liveness |
|
||||
|
||||
**Upgrades:** expect every session to drop. Drain clients if you can; there is no rolling update on default SQLite. Compose `restart: unless-stopped` plus Docker `HEALTHCHECK` will also replace the only process when the container is Unhealthy — same blast radius.
|
||||
|
||||
External Postgres / multi-writer HA is **not** a documented stock path. If you need HA, keep a single replica or run a topology the project has tested and documented separately.
|
||||
|
||||
## Important Notes
|
||||
|
||||
- **SQLite WAL Mode:** `docker stop` should be allowed to finish so OmniRoute can checkpoint the latest changes back into `storage.sqlite`. The bundled Compose files already set a 40s stop grace period. If you run the image directly, keep `--stop-timeout 40`.
|
||||
|
||||
@@ -169,7 +169,7 @@ stale data.
|
||||
The sync runs **on by default**:
|
||||
|
||||
- It runs once at server startup and then on a periodic timer
|
||||
(`src/lib/arenaEloSync.ts`, wired from `src/server-init.ts`).
|
||||
(`src/lib/arenaEloSync.ts`, wired from `src/instrumentation-node.ts`).
|
||||
- It is **non-blocking and never fatal** — if the upstream fetch fails, OmniRoute keeps
|
||||
running and the rankings simply show the last good data (or an empty state).
|
||||
|
||||
|
||||
@@ -1,47 +1,159 @@
|
||||
---
|
||||
title: "Management Authentication"
|
||||
version: 3.8.50
|
||||
lastUpdated: 2026-08-05
|
||||
lastUpdated: 2026-08-20
|
||||
---
|
||||
|
||||
# Management Authentication
|
||||
|
||||
OmniRoute uses four distinct credential families for management access. This guide
|
||||
distinguishes them by purpose, scope, and locality.
|
||||
OmniRoute has **four credential families** that can authorize management routes.
|
||||
They are not interchangeable. Inference API keys (`sk-…`) do **not** manage the
|
||||
server unless they were explicitly granted `manage` or `admin` scope.
|
||||
|
||||
| Credential | Scope | Locality | Use Case |
|
||||
|-------------------------|--------------------|---------------|-----------------------------------|
|
||||
| Dashboard JWT session | Full management | Localhost | Web dashboard login |
|
||||
| CLI machine-id token | Full management | Per-machine | `omniroute` CLI commands |
|
||||
| Scoped `oma_` token | Configurable scope | External | Automation / CI / API access |
|
||||
| Manage-scope API key | `manage` scope | External | Management API calls |
|
||||
Canonical implementation: `src/lib/api/requireManagementAuth.ts`.
|
||||
|
||||
## Dashboard JWT Session
|
||||
| Credential | Typical form | Created where | Intended use | Management capability |
|
||||
|---|---|---|---|---|
|
||||
| Dashboard session | `auth_token` cookie | Dashboard login | Browser UI | Full dashboard management, subject to CSRF, locality, and always-protected-route rules |
|
||||
| Local CLI machine token | internal / local | CLI bootstrap (`omniroute` on the same machine) | Local CLI | Local management only |
|
||||
| Scoped Access Token | `oma_live_…` | **Settings → Access Tokens** or `omniroute connect` | Remote CLI and management API | Must satisfy the route's required `read`, `write`, or `admin` scope |
|
||||
| Inference API key | `sk-…` (and other API-key prefixes) | **API Manager / API Keys** | `/v1/*` inference | **None** unless the key metadata includes `manage` or `admin` |
|
||||
|
||||
Generated on dashboard login (`/api/auth/login`). Stored in HTTP-only cookie.
|
||||
Valid for the session duration. Cannot be used from external hosts.
|
||||
`oma_` credentials are management/CLI credentials. They are **not** inference API keys.
|
||||
|
||||
## CLI Machine-ID Token
|
||||
If login/API-key auth is disabled for the server, some management routes may
|
||||
accept unauthenticated calls. Local-only and always-protected routes still apply
|
||||
their own rules. Presenting one of these credentials is therefore not universally
|
||||
mandatory, and possessing one is not universally sufficient without the required
|
||||
scope and route locality.
|
||||
|
||||
Created by `omniroute auth login` on first use. Stored in `~/.omniroute/auth.json`.
|
||||
Used by the CLI for all management operations. Tied to the machine identity.
|
||||
Related: [Remote Mode](./REMOTE-MODE.md) (how `oma_live_…` is minted for a remote CLI).
|
||||
|
||||
## Scoped `oma_` Access Token
|
||||
---
|
||||
|
||||
Created via dashboard or CLI with configurable scopes (e.g., `manage`, `read`).
|
||||
Format: `oma_<random-hex>`. Used for programmatic access from external systems.
|
||||
## Scope matrices
|
||||
|
||||
## Manage-Scope API Key
|
||||
These two scope vocabularies are **different**. Do not mix them.
|
||||
|
||||
Standard API key with the `manage` scope enabled. Created in dashboard API Keys page.
|
||||
Used for management API calls from external hosts.
|
||||
### Access Token scopes (`oma_live_…`)
|
||||
|
||||
## Header Examples
|
||||
| Scope | Typical operations |
|
||||
|---|---|
|
||||
| `read` | List/status GETs that the token is allowed to see |
|
||||
| `write` | Mutations (create/update/delete) below admin |
|
||||
| `admin` | Full remote CLI / connect token (password bootstrap defaults here) |
|
||||
|
||||
```
|
||||
Authorization: Bearer oma_abc123def456
|
||||
Authorization: Bearer <standard-api-key-with-manage-scope>
|
||||
Cookie: omniroute_session=<jwt-token>
|
||||
A token with `read` cannot call a `write` route. Runtime message shape:
|
||||
`Access token scope '<have>' is insufficient; '<need>' required.`
|
||||
|
||||
### API-key management scopes
|
||||
|
||||
| Scope | Meaning |
|
||||
|---|---|
|
||||
| (none) | Inference only. Management routes return 403. |
|
||||
| `manage` | Management API (same gate as `requireManagementAuth` API-key branch) |
|
||||
| `admin` | Also satisfies `hasManageScope` (treated as management-capable) |
|
||||
|
||||
Enable `manage` on the key in the API Keys / API Manager UI. Do not reuse a
|
||||
chat client key for automation unless you deliberately granted that scope.
|
||||
|
||||
---
|
||||
|
||||
## How to create and revoke
|
||||
|
||||
### Dashboard session
|
||||
|
||||
1. Open `/login`, sign in with the management password (`INITIAL_PASSWORD` on first boot).
|
||||
2. Cookie `auth_token` is HttpOnly. Browser dashboard uses it automatically.
|
||||
3. Log out via `/api/auth/logout`. There is no long-lived secret to copy.
|
||||
|
||||
### Local CLI machine token
|
||||
|
||||
1. Run `omniroute` on the **same host** as the server (loopback).
|
||||
2. The CLI bootstraps a machine-id token under `~/.omniroute/` (chmod 600).
|
||||
3. This does **not** work from another machine. Use an Access Token for remote CLI.
|
||||
|
||||
### Scoped Access Token (`oma_live_…`)
|
||||
|
||||
1. Dashboard: **Settings → Access Tokens** → create (name + scope). **The secret is shown once.**
|
||||
2. Or CLI: `omniroute connect <host>` (password → token). See [Remote Mode](./REMOTE-MODE.md).
|
||||
3. Header: `Authorization: Bearer oma_live_…`
|
||||
4. Revoke from the same Access Tokens page (or delete the CLI context).
|
||||
5. Server stores only a hash. Treat the plaintext like a password.
|
||||
|
||||
### Manage-scoped API key
|
||||
|
||||
1. Dashboard: **API Manager / API Keys** → create or edit a key → enable `manage` (or `admin`).
|
||||
2. Header: `Authorization: Bearer sk-…` (the key's actual prefix).
|
||||
3. Revoke or strip `manage` in the same UI.
|
||||
4. Least privilege for automation that is not the CLI: prefer a `read` Access Token for GET-only jobs; use `manage` on an API key only when the caller must also speak `/v1` and management.
|
||||
|
||||
---
|
||||
|
||||
## Header format
|
||||
|
||||
```http
|
||||
Authorization: Bearer oma_live_<secret>
|
||||
Authorization: Bearer sk-<secret>
|
||||
Cookie: auth_token=<dashboard-jwt>
|
||||
```
|
||||
|
||||
See `docs/reference/API_REFERENCE.md` for endpoint-specific auth requirements.
|
||||
Do not put management credentials in the URL path or query string. Management
|
||||
auth is header/cookie only.
|
||||
|
||||
---
|
||||
|
||||
## Copy-paste examples
|
||||
|
||||
Read-only (list providers). Use a `read` Access Token:
|
||||
|
||||
```bash
|
||||
curl -sS "$OMNIROUTE_URL/api/providers" \
|
||||
-H "Authorization: Bearer oma_live_<read-token>"
|
||||
```
|
||||
|
||||
Modifying (create a provider connection). Use `write`/`admin` Access Token or a
|
||||
manage-scoped API key:
|
||||
|
||||
```bash
|
||||
curl -sS -X POST "$OMNIROUTE_URL/api/providers" \
|
||||
-H "Authorization: Bearer oma_live_<write-or-admin-token>" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"provider":"openai","apiKey":"<upstream-key>"}'
|
||||
```
|
||||
|
||||
Inference (not management). Ordinary API key, no `manage` required:
|
||||
|
||||
```bash
|
||||
curl -sS "$OMNIROUTE_URL/v1/models" \
|
||||
-H "Authorization: Bearer sk-<inference-key>"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Current runtime errors (do not echo secrets)
|
||||
|
||||
| Situation | Typical status | Message (sanitized) |
|
||||
|---|---|---|
|
||||
| No credential | 401 | `Authentication required` |
|
||||
| Invalid/expired `oma_live_…` | 401 | `Invalid or expired access token` |
|
||||
| Valid API key without `manage`/`admin` | 403 | `API key lacks 'manage' scope. Enable it in the API Keys dashboard.` |
|
||||
| Invalid ordinary API key on a management route | 403 | `Invalid management token` |
|
||||
| Access Token scope too low | 403 | `Access token scope '<have>' is insufficient; '<need>' required.` |
|
||||
|
||||
"Invalid management token" means the bearer was **not** accepted as a management
|
||||
credential. It does **not** tell you which family to mint. Use the table above:
|
||||
inference keys need `manage` scope; remote CLI needs `oma_live_…`; the dashboard
|
||||
uses the session cookie.
|
||||
|
||||
---
|
||||
|
||||
## Recommended least-privilege choice
|
||||
|
||||
| Caller | Use |
|
||||
|---|---|
|
||||
| Browser | Dashboard session |
|
||||
| CLI on the server host | Machine token |
|
||||
| CLI on a laptop talking to a remote server | `oma_live_…` from `omniroute connect` |
|
||||
| CI / scripts (management only) | `oma_live_…` with the smallest scope that works |
|
||||
| CI that must call both `/v1` and `/api` | API key with `manage` **or** two credentials |
|
||||
|
||||
@@ -142,22 +142,22 @@ wskazuje zamiast tego lokalną kopię (instalacje offline / air-gapped).
|
||||
|
||||
### Opcjonalne zależności i instalacja on-demand
|
||||
|
||||
Przycinany stos peerów runtime LLMLingua jest **opcjonalny**. Trzy pakiety są zadeklarowane jako
|
||||
Przycinany stos peerów runtime LLMLingua jest **opcjonalny**. Dwa pakiety są zadeklarowane jako
|
||||
`optionalDependencies` w `package.json` i utrzymywane jako **external** przez build produkcyjny
|
||||
(`scripts/build/prepublish.ts` ich nie bundluje):
|
||||
|
||||
| Package | Version (pin) | Notes |
|
||||
| -------------------- | ------------- | ------------------------------------------------- |
|
||||
| `@atjsh/llmlingua-2` | `2.0.3` | Pakiet wejściowy; deklaruje pozostałe jako peery |
|
||||
| `@tensorflow/tfjs` | `4.22.0` | Najcięższa zależność — dominuje footprint ~800 MB |
|
||||
| `js-tiktoken` | `^1.0.20` | Tokenizer |
|
||||
| Package | Version (pin) | Notes |
|
||||
| -------------------- | ------------- | ------------------------------------------- |
|
||||
| `@atjsh/llmlingua-2` | `2.0.5` | Pakiet wejściowy; deklaruje pozostałe jako peery |
|
||||
| `js-tiktoken` | `^1.0.20` | Tokenizer |
|
||||
|
||||
`@huggingface/transformers` jest pinowany na `3.5.2` jako **opcjonalna** zależność (współdzielona ze
|
||||
ścieżką lokalnych embeddings i również śledzona do standalone bundle). Utrzymanie jej jako optional
|
||||
zapobiega awariom postinstall providera CUDA `onnxruntime-node` na hostach CUDA 11, które przerywałyby
|
||||
całą instalację OmniRoute; gdy opcjonalny stos jest nieobecny, LLMLingua nadal fail-openuje. Tylko trzy
|
||||
powyższe pakiety to przycinane peery SLM. Standardowe `npm install` (dev) instaluje opcjonalny stos
|
||||
automatycznie, o ile opcjonalne zależności nie zostaną pominięte.
|
||||
`@huggingface/transformers` jest pinowany na `^4.2.0` (współdzielony ze ścieżką lokalnych embeddings
|
||||
i również śledzony do standalone bundle); `@atjsh/llmlingua-2@2.0.5` peeruje na nim przez
|
||||
`"^3.5.2 || ^4.0.0"`, więc obsługiwane są zarówno Transformers.js v3, jak i v4. Od 2.0.4
|
||||
`@atjsh/llmlingua-2` nie wymaga już `@tensorflow/tfjs`, co usunęło największy pojedynczy wkład
|
||||
(TensorFlow.js) ze stosu SLM. Tylko dwa powyższe pakiety to przycinane peery SLM. Standardowe
|
||||
`npm install` (dev) instaluje opcjonalny stos automatycznie, o ile opcjonalne zależności nie zostaną
|
||||
pominięte.
|
||||
|
||||
**Dlaczego on-demand:** pakiet publikowany w npm, standalone bundle i obraz Docker
|
||||
dostarczane są **bez** tych zależności, aby pozostać lekkie. Gdy ich brakuje, bramka zależności
|
||||
@@ -167,11 +167,12 @@ logowanego błędu). Aby aktywować go w przyciętym środowisku, zainstaluj opc
|
||||
|
||||
```bash
|
||||
# pin to the versions declared in package.json optionalDependencies
|
||||
npm install @atjsh/llmlingua-2@2.0.3 @tensorflow/tfjs@4.22.0 js-tiktoken
|
||||
npm install @atjsh/llmlingua-2@2.0.5 js-tiktoken
|
||||
```
|
||||
|
||||
Łącznie mniej więcej **~800 MB**: dominują runtime’y TensorFlow.js + transformers; model
|
||||
TinyBERT dodaje ~57 MB pobierane przy pierwszym użyciu (nie przez npm).
|
||||
Usunięcie `@tensorflow/tfjs` (2.0.4+) eliminuje wcześniej dominujący wkład ~800 MB — pozostały
|
||||
footprint to runtime’y transformers.js + onnxruntime-node oraz model TinyBERT (~57 MB) pobierany
|
||||
przy pierwszym użyciu (nie przez npm).
|
||||
|
||||
Per środowisko:
|
||||
|
||||
|
||||
@@ -326,13 +326,11 @@ Przed wypuszczeniem dowolnego wydania v3.8.x zweryfikuj te dodatkowe pozycje:
|
||||
- [ ] `npm install -g omniroute@<this-version>` uruchamia postinstall bez fatalnego wyjścia
|
||||
- [ ] Ścieżka update zachowuje optional deps: `omniroute update --apply` i auto-updater
|
||||
uruchamiają `npm install -g … --include=optional`, żeby `optionalDependencies` (better-sqlite3,
|
||||
keytar, tls-client oraz stack SLM llmlingua: `@atjsh/llmlingua-2`,
|
||||
`@huggingface/transformers@3.5.2`, `@tensorflow/tfjs`, `js-tiktoken`) przeżyły update.
|
||||
`@huggingface/transformers` zostaje optional, żeby jego postinstall providera CUDA `onnxruntime-node`
|
||||
nie mógł przerwać instalacji na hostach CUDA 11. Tier ultra `modelPath` SLM potrzebuje też
|
||||
keytar, tls-client oraz stack SLM llmlingua: `@atjsh/llmlingua-2@2.0.5`,
|
||||
`js-tiktoken`) przeżyły update. Tier ultra `modelPath` SLM potrzebuje też
|
||||
modelu tinybert, auto-pobieranego do `${DATA_DIR}/models/llmlingua` przy pierwszym użyciu. Postinstall
|
||||
(`scripts/build/colocateOptionals.mjs`) następnie ko-lokuje opcjonalne zamknięcie SLM do
|
||||
`dist/node_modules`, żeby worker rozwiązywał JEDNĄ opcjonalną instancję `@huggingface/transformers` 3.5.2
|
||||
`dist/node_modules`, żeby worker rozwiązywał JEDNĄ instancję `@huggingface/transformers` ^4.2.0
|
||||
— standalone trace bundluje tylko transformers, nie dynamicznie importowane
|
||||
optionals, więc bez tego worker załadowałby llmlingua-2 przeciw transformers z roota
|
||||
i tier SLM cicho fail-openowałby.
|
||||
|
||||
@@ -275,14 +275,12 @@ npm run build:release
|
||||
- [ ] `npm install -g omniroute@<this-version>` 运行 postinstall 无致命退出
|
||||
- [ ] 更新路径保留可选依赖:`omniroute update --apply` 以及自动更新器
|
||||
运行 `npm install -g … --include=optional` 以确保 `optionalDependencies`(better-sqlite3、
|
||||
keytar、tls-client 以及 llmlingua SLM 栈:`@atjsh/llmlingua-2`、
|
||||
`@huggingface/transformers@3.5.2`、`@tensorflow/tfjs`、`js-tiktoken`)在更新后仍然存在。
|
||||
`@huggingface/transformers` 保持为可选依赖,这样其 `onnxruntime-node` CUDA provider postinstall
|
||||
不会在 CUDA 11 主机上中断安装。Ultra 模式的 `modelPath` SLM 层还需要
|
||||
keytar、tls-client 以及 llmlingua SLM 栈:`@atjsh/llmlingua-2@2.0.5`、
|
||||
`js-tiktoken`)在更新后仍然存在。Ultra 模式的 `modelPath` SLM 层还需要
|
||||
tinybert 模型,首次使用时自动下载到 `${DATA_DIR}/models/llmlingua`。postinstall
|
||||
(`scripts/build/colocateOptionals.mjs`)随后将 SLM 可选依赖闭包共置到
|
||||
`dist/node_modules`,使 Worker 解析单一的 `@huggingface/transformers` 3.5.2
|
||||
可选实例 — standalone trace 仅打包 transformers,不包含动态导入的
|
||||
`dist/node_modules`,使 Worker 解析单一的 `@huggingface/transformers` ^4.2.0
|
||||
实例 — standalone trace 仅打包 transformers,不包含动态导入的
|
||||
可选依赖,否则 Worker 会基于根目录的 transformers 加载 llmlingua-2,
|
||||
SLM 层将静默失效。
|
||||
- [ ] `omniroute status` 在无 `.env` 的情况下正常工作(CLI Token 路径,仅 loopback)
|
||||
|
||||
@@ -322,14 +322,12 @@ npm run build:release
|
||||
- [ ] `npm install -g omniroute@<此版本>` 執行 postinstall 而不會致命退出
|
||||
- [ ] 更新路徑保留選擇性依賴:`omniroute update --apply` 和自動更新器
|
||||
執行 `npm install -g … --include=optional`,因此 `optionalDependencies`(better-sqlite3、
|
||||
keytar、tls-client,以及 llmlingua SLM 堆疊:`@atjsh/llmlingua-2`、
|
||||
`@huggingface/transformers@3.5.2`、`@tensorflow/tfjs`、`js-tiktoken`)在更新後仍會保留。
|
||||
`@huggingface/transformers` 維持選擇性,因此其 `onnxruntime-node` CUDA 提供者的 postinstall
|
||||
不會在 CUDA 11 主機上中斷安裝。Ultra `modelPath` SLM 層還需要
|
||||
keytar、tls-client,以及 llmlingua SLM 堆疊:`@atjsh/llmlingua-2@2.0.5`、
|
||||
`js-tiktoken`)在更新後仍會保留。Ultra `modelPath` SLM 層還需要
|
||||
tinybert 模型,會在首次使用時自動下載到 `${DATA_DIR}/models/llmlingua`。Postinstall
|
||||
(`scripts/build/colocateOptionals.mjs`)接著將 SLM 選擇性閉包複製到
|
||||
`dist/node_modules`,使工作者解析到**單一** `@huggingface/transformers` 3.5.2
|
||||
選擇性實例——獨立追蹤僅捆綁 transformers,而非動態匯入的
|
||||
`dist/node_modules`,使工作者解析到**單一** `@huggingface/transformers` ^4.2.0
|
||||
實例——獨立追蹤僅捆綁 transformers,而非動態匯入的
|
||||
選擇性套件,因此若無此步驟,工作者會載入 llmlingua-2 並使用根目錄的 transformers,
|
||||
導致 SLM 層靜默地失敗但仍保持運作。
|
||||
- [ ] `omniroute status` 在無 `.env` 的情況下正常運作(僅限 CLI 權杖路徑,迴環介面)
|
||||
|
||||
@@ -2069,6 +2069,39 @@ paths:
|
||||
description: Created combo
|
||||
|
||||
/api/combos/{id}:
|
||||
get:
|
||||
tags: [Combos]
|
||||
summary: Get combo by ID
|
||||
parameters:
|
||||
- $ref: "#/components/parameters/ResourceId"
|
||||
responses:
|
||||
"200":
|
||||
description: Combo details
|
||||
"404":
|
||||
description: Combo not found
|
||||
put:
|
||||
tags: [Combos]
|
||||
summary: Update combo
|
||||
description: >-
|
||||
Partial update: the body is merged onto the stored combo, so a field left out keeps
|
||||
its current value. An array that IS sent replaces the stored one outright.
|
||||
parameters:
|
||||
- $ref: "#/components/parameters/ResourceId"
|
||||
requestBody:
|
||||
required: true
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
type: object
|
||||
responses:
|
||||
"200":
|
||||
description: Updated combo
|
||||
"400":
|
||||
description: Invalid body, or the resulting combo fails validation
|
||||
"404":
|
||||
description: Combo not found
|
||||
"409":
|
||||
description: Name already taken, or the combo is quota-share managed
|
||||
patch:
|
||||
tags: [Combos]
|
||||
summary: Update combo
|
||||
@@ -7329,12 +7362,18 @@ components:
|
||||
BearerAuth:
|
||||
type: http
|
||||
scheme: bearer
|
||||
description: API key obtained from the OmniRoute dashboard
|
||||
description: >
|
||||
Two bearer families are accepted. Inference API keys (typically `sk-…`)
|
||||
authorize `/v1/*`. Management routes also accept `oma_live_…` Access Tokens
|
||||
(Settings → Access Tokens / `omniroute connect`) and API keys whose metadata
|
||||
includes `manage` or `admin` scope. See docs/guides/MANAGEMENT-AUTH.md.
|
||||
Bearer credentials are accepted on management routes that use this scheme;
|
||||
they are not rejected solely for being Bearer.
|
||||
ManagementSessionAuth:
|
||||
type: apiKey
|
||||
in: cookie
|
||||
name: auth_token
|
||||
description: Dashboard management session cookie for protected management routes
|
||||
description: Dashboard management session cookie (auth_token) for protected management routes. Distinct from Bearer Access Tokens and API keys. See docs/guides/MANAGEMENT-AUTH.md.
|
||||
|
||||
parameters:
|
||||
ResourceId:
|
||||
|
||||
@@ -453,26 +453,47 @@ Run monthly during low-traffic windows. (WAL mode reduces the need, but doesn't
|
||||
|
||||
`src/lib/db/healthCheck.ts` provides **DB-level health diagnostics**:
|
||||
|
||||
````bash
|
||||
GET /api/db/health
|
||||
Both verbs require authentication (`401` otherwise). `GET` diagnoses only; `POST` runs the
|
||||
same check with `autoRepair` enabled.
|
||||
|
||||
Returns:
|
||||
```bash
|
||||
GET /api/db/health # diagnose
|
||||
POST /api/db/health # diagnose + repair
|
||||
```
|
||||
|
||||
The response is the `DbHealthCheckResult` produced by `runDbHealthCheck()`
|
||||
(`src/lib/db/healthCheck.ts`):
|
||||
|
||||
```json
|
||||
{
|
||||
"status": "healthy",
|
||||
"checks": {
|
||||
"writable": { "status": "pass" },
|
||||
"integrity": { "status": "pass", "result": "ok" },
|
||||
"foreign_keys": { "status": "pass", "violations": 0 },
|
||||
"orphaned_artifacts": { "status": "warn", "count": 12 },
|
||||
"table_sizes": {
|
||||
"usage_history": { "rows": 12345, "size_mb": 12.3 },
|
||||
"call_logs": { "rows": 567, "size_mb": 2.1 }
|
||||
"isHealthy": false,
|
||||
"issues": [
|
||||
{
|
||||
"type": "broken_reference",
|
||||
"table": "domain_budgets",
|
||||
"description": "Domain budgets referenced API keys that no longer exist.",
|
||||
"count": 2
|
||||
}
|
||||
}
|
||||
],
|
||||
"repairedCount": 0,
|
||||
"backupCreated": false,
|
||||
"autoRepair": false,
|
||||
"checkedAt": "2026-08-18T09:00:00.000Z",
|
||||
"driver": { "name": "better-sqlite3", "degraded": false }
|
||||
}
|
||||
````
|
||||
```
|
||||
|
||||
| Field | Meaning |
|
||||
| ----------------- | ---------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `isHealthy` | `true` when `issues` is empty. `driver` never influences it. |
|
||||
| `issues[].type` | One of `integrity_check_failed`, `broken_reference`, `stale_snapshot`, `invalid_state`. |
|
||||
| `repairedCount` | Rows repaired during this run; always `0` when `autoRepair` is false. |
|
||||
| `backupCreated` | Whether a backup was taken before repairing. |
|
||||
| `checkedAt` | ISO timestamp shared by the run and by any repair note it writes. |
|
||||
| `driver.name` | SQLite driver serving the checked database. |
|
||||
| `driver.degraded` | `true` when writes are not durably backed by the database file — the `sql.js` WASM fallback (whole-file persistence) or an in-memory database. |
|
||||
|
||||
The same payload is returned by the `omniroute_db_health_check` MCP tool.
|
||||
|
||||
Run `PRAGMA integrity_check` to detect corruption:
|
||||
|
||||
|
||||
@@ -157,13 +157,13 @@ Response:
|
||||
|
||||
### Kubernetes probe recommendations
|
||||
|
||||
OmniRoute is a **single Node process** (one event loop). Stock Docker `HEALTHCHECK` targets `/api/monitoring/health` — that is **too heavy** for kubelet liveness intervals.
|
||||
OmniRoute is a **single Node process** (one event loop). Stock Docker `HEALTHCHECK` targets lightweight `/healthz`. `/api/monitoring/health` is **too heavy** for kubelet liveness intervals.
|
||||
|
||||
| Probe | Recommended target | Notes |
|
||||
| --- | --- | --- |
|
||||
| **Startup** | HTTP `GET /healthz` with a long `failureThreshold` (or large `startPeriod`) | Cold start + SQLite migration can exceed a few seconds |
|
||||
| **Readiness** | HTTP `GET /healthz` | Remove endpoints while starting/stopping; still flaps if the loop is CPU-blocked |
|
||||
| **Liveness** | **TCP** on the main service port (`PORT`, default `20128`), **or** HTTP `/healthz` with soft thresholds | Do **not** kill the pod on short event-loop stalls; busy ≠ dead |
|
||||
| **Readiness** | HTTP `GET /healthz` | Lifecycle `ok` / `starting` / `stopping` (200 vs 503). Still flaps if the loop is CPU-blocked. A **200 in multiple seconds is not healthy** (#10303) — it means the event loop was starved before the 3-byte handler ran |
|
||||
| **Liveness** | HTTP `GET /livez`, **or TCP** on the main service port (`PORT`, default `20128`) | `/livez` is process-alive only (always 200 if the handler runs). It still shares the event loop — busy ≠ dead, and it does not detect event-loop starvation (#10303) any better than TCP does. Prefer **TCP** if HTTP probes time out under catalog/compression load; do **not** kill the pod on short event-loop stalls either way |
|
||||
| **Deep health** | `GET /api/monitoring/health` from an external checker | Not for kubelet `livenessProbe` / tight `readinessProbe` |
|
||||
|
||||
Example shape (adjust thresholds to your cold-start and compression load):
|
||||
@@ -186,17 +186,27 @@ readinessProbe:
|
||||
timeoutSeconds: 2
|
||||
failureThreshold: 6
|
||||
livenessProbe:
|
||||
tcpSocket:
|
||||
httpGet:
|
||||
path: /livez
|
||||
port: http
|
||||
periodSeconds: 10
|
||||
timeoutSeconds: 3
|
||||
failureThreshold: 6
|
||||
# Under event-loop stall HTTP /livez can still time out. TCP is the
|
||||
# conservative alternative:
|
||||
# tcpSocket:
|
||||
# port: http
|
||||
```
|
||||
|
||||
**Do not** point kubelet **liveness** at `/api/monitoring/health`. That path does real DB/monitoring work and will false-positive under load.
|
||||
|
||||
Related: [#10052](https://github.com/diegosouzapw/OmniRoute/issues/10052) (probes while the event loop is busy), [#9685](https://github.com/diegosouzapw/OmniRoute/issues/9685) / [#10055](https://github.com/diegosouzapw/OmniRoute/pull/10055) (catalog pricing hog), [#10117](https://github.com/diegosouzapw/OmniRoute/issues/10117) (compression token-count hog).
|
||||
|
||||
|
||||
### Optional request-path work (memory, skills, token refresh)
|
||||
|
||||
Memory extraction, skills injection, and OAuth token refresh share the **main Node event loop** with `/healthz`. They are dashboard-toggle features (`memoryEnabled`, `skillsEnabled`), not a worker pool. See [Environment — event-loop cost](../reference/ENVIRONMENT.md#event-loop-cost-of-memory-skills-and-token-refresh-10349).
|
||||
|
||||
### Provider Health
|
||||
|
||||
> **No REST endpoint.** Provider health data is available via the MCP tool `observability_snapshot` or the dashboard `/dashboard/providers` page.
|
||||
|
||||