mirror of
https://github.com/diegosouzapw/OmniRoute.git
synced 2026-07-26 09:52:11 +03:00
cc17b304ab
* fix(sse): Gemini TPM classification, combo-cooldown-wait for auto/quota-share, and target-timeout floor
Gemini TPM/RPM 429s were misclassified as QUOTA_EXHAUSTED because
sanitizeErrorMessage() truncates to the first line, hiding Google's
metric name and retry hint on lines 2-3. Added a rawMessage field
(internal-only, never reaches the client) and classifyGeminiQuotaMetricFromText()
to classify from the untruncated text, reordered ahead of the generic
credits/daily-quota checks.
Widened comboCooldownWaitEnabled (wait out a short transient cooldown
instead of crystallizing a 429/503) from quota-share-only to also cover
auto-strategy combos, and raised the wait ceiling to 65s/130s-budget/90s-cap
to match Gemini's ~60s TPM/RPM windows.
The per-target timeout (DEFAULT_COMBO_TARGET_TIMEOUT_MS, 120s) was shorter
than the new 130s cooldown-wait budget, so a target could get cut off
mid-wait with a synthetic 524 instead of completing the retry. Added
resolveComboTargetTimeoutMsForCombo()/isComboCooldownWaitEligible() in
comboConfig.ts to raise the per-target floor to budgetMs+buffer only for
wait-eligible strategies (auto/quota-share), verified live: a 12-request
concurrent burst against a TPM-exhausted combo went from 2/12 succeeding
(10 x 524) to 12/12 succeeding with zero 503/524.
Also: liveGeminiShared.ts's sendAndValidate now fails fast on a 503
instead of retrying past it, and the health dashboard + request logger
surface TPM stats alongside RPM/RPD.
Co-authored-by: Markus Hartung <markus.hartung@gmail.com>
* fix(sse): combo-exhausted rejection logs now capture request body + attempted models
recordRejectedRequestUsage() (the fast path for combo requests that never
reach handleChatCore, e.g. all targets locked by resilience cooldown)
hardcoded provider: "-" and never passed a request body to saveCallLog(),
so /dashboard/logs entries for these failures were nearly useless for
debugging: no way to see the client's request or which models were tried.
- recordRejectedRequestUsage() now accepts requestBody and persists it
through the existing saveCallLog() artifact mechanism (same path
handleChatCore's own logging uses).
- Added summarizeComboAttemptedModels(), which reads the combo's own model
list (always available, unlike the response's combo-diagnostics headers —
a model-level resilience-lockout skip never touches the
exhaustedProviders/exhaustedConnections sets those headers are built
from) to populate a real "provider" value instead of "-".
- Wired both into the call site in src/sse/handlers/chat.ts.
NOTE: unrelated to the Gemini TPM/combo-cooldown-wait fix on this branch —
landed here per operator request, to be split into its own branch/PR.
Co-authored-by: Markus Hartung <markus.hartung@gmail.com>
* feat(sse): synthetic streaming keep-alive event + 5-minute Gemini cooldown-wait ceiling
Many clients enforce a first-SSE-byte timeout, which made it unsafe to wait
out a longer upstream rate-limit cooldown on a streaming request — the
client would abandon the connection before any bytes arrived. This landed
in two parts:
1. Synthetic startup "thinking" event (OpenAI chat/completions format):
the already-existing withEarlyStreamKeepalive wrapper (open-sse/utils/
earlyStreamKeepalive.ts, wired into /v1/chat/completions, /v1/messages,
/v1/responses since #2544) opens the SSE stream immediately once a
request runs past its threshold, but only ever sent empty/no-op
keepalive frames. Added a `startupFrame` option (defaults to
`keepaliveFrame` — zero behavior change unless a route opts in) so the
very first frame can carry real content instead. Wired
OPENAI_STARTUP_THINKING_FRAME (a reasoning_content delta: "OmniRoute:
got request, sending to provider") into /v1/chat/completions only —
Claude Messages and Responses API formats both require a preceding
envelope event (message_start / response.created) that a synthetic
pre-dispatch frame can't safely fabricate without risking a duplicate
envelope once the real stream arrives, so those two routes keep their
existing (safe, proven) keepalive frames unchanged.
2. Raised the "wait out a known cooldown, then retry" ceiling to 5 minutes
for both retry mechanisms, now that a client-side first-byte timeout is
no longer a risk on the (opted-in) route:
- comboCooldownWait (auto/quota-share combos, open-sse/services/combo.ts):
maxWaitMs hard clamp raised 90s -> 300s (src/lib/resilience/settings/
normalize.ts); defaults raised to maxWaitMs:90s/maxAttempts:5/
budgetMs:300s. comboConfig.ts's resolveComboTargetTimeoutMsForCombo
already derives the per-target timeout floor from budgetMs, so it
tracks the new ceiling with no further changes.
- waitForCooldown (direct, non-combo model requests, src/sse/handlers/
chat.ts): this mechanism had NO cumulative cap before — only a
per-wait cap (maxRetryWaitMs) and a retry count (maxRetries), so
maxRetries x maxRetryWaitMs could exceed 5 minutes with no ceiling.
Added a budgetMs field (mirrors comboCooldownWait) to
WaitForCooldownSettings/CooldownAwareRetrySettings, threaded a
requestRetryBudgetLeftMs tracker through chat.ts's requestAttemptLoop
(mirrors combo.ts's comboCooldownBudgetLeftMs), and made
getCooldownAwareRetryDecision refuse to wait once the cumulative
budget is exhausted even if the single wait is under maxRetryWaitMs.
Co-authored-by: Markus Hartung <markus.hartung@gmail.com>
* fix(sse): extend the synthetic keep-alive thinking event to /v1/responses
Live incident (OpenClaw, log id 1784407081908-cbc24f): a /v1/responses
request to gemini/gemma-4-31b-it took 56s to produce a first byte and the
client disconnected (499 request_signal_aborted) — the same client-first-byte-
timeout problem the previous commit fixed for /v1/chat/completions, but
/v1/responses only had the generic bare-comment keepalive (no content), so it
wasn't covered.
Added RESPONSES_STARTUP_THINKING_FRAME: a self-contained synthetic reasoning
item (response.output_item.added -> reasoning_summary_part.added ->
reasoning_summary_text.delta -> reasoning_summary_part.done), opened AND
closed within this one frame rather than left dangling — it never carries a
response_id, so it can't collide with the real upstream response's own
independent response.created lifecycle that follows. Mirrors the abbreviated
delta+part.done close pattern open-sse/utils/stream.ts's own
emitSyntheticResponsesReasoningSummary already uses for real mid-stream
reasoning content.
Wired into src/app/api/v1/responses/route.ts via the startupFrame option
added in the previous commit.
Co-authored-by: Markus Hartung <markus.hartung@gmail.com>
* fix(sse): combo cooldown-wait vars reset every setTry, crystallizing a bogus 503 instead of waiting
Live incident (log id 1784416706646-51): a request to the "default" combo
(strategy=auto, maxSetRetries=3) hit a real Gemini TPM 429 on both gemma-4
targets, correctly classified as a short 40s rate_limit lockout — then
crystallized a 503 "all upstream accounts are inactive" in 6.9s instead of
ever reaching the cooldown-aware wait.
Root cause: `lastError`/`earliestRetryAfter`/`lastStatus` were declared with
`let` INSIDE the `for (setTry...)` loop body, so they reset to null at the
start of every set-try. When both targets lock out on setTry 0, every
subsequent setTry (1..maxSetRetries) pre-skips both targets via the
isModelLocked check with no real dispatch — so on the FINAL setTry (the only
one whose values the post-loop decision reads, since it's gated behind
`if (setTry < maxSetRetries) continue`), lastStatus was null, hitting the
"!lastStatus" branch (ALL_ACCOUNTS_INACTIVE 503) and completely bypassing the
comboCooldownWaitEnabled / earliestRetryAfter wait logic — even though a
real 429 with a known ~40s retry-after WAS observed on setTry 0.
This bug predates today's Gemini TPM work (any combo with maxSetRetries > 0
whose targets all lock out on the first pass was affected) but was masked in
existing tests: the "auto strategy (2 models...)" regression test uses
maxSetRetries: 0, so it only ever runs ONE setTry iteration and never
exercises the reset-on-retry path. It also explains why the dedicated
12-concurrent-request burst test passed cleanly — with concurrent requests,
timing variance meant some request's FINAL setTry iteration still had a live
target to dispatch to, giving lastStatus/earliestRetryAfter fresh data. A
single isolated request has no such luck.
Fix: hoist lastError/earliestRetryAfter/lastStatus to just inside
dispatchWithCooldownRetry, before the setTry loop, so they persist across
set-tries (still reset fresh on each recursive dispatchWithCooldownRetry()
call after a wait, which is correct). recordedAttempts/fallbackCount/
exhaustedProviders etc. are intentionally left per-iteration (unrelated to
this bug).
New regression test in tests/unit/combo-quota-share-cooldown-wait.test.ts
reproduces the exact live scenario (2 targets, both lock out on setTry 0,
maxSetRetries: 3) — confirmed red (503) against the pre-fix code, green
(200, waits and retries) against the fix.
Co-authored-by: Markus Hartung <markus.hartung@gmail.com>
* test(sse): extend live Gemini workload to Responses API + add large-context TPM test
Two additions to the live Gemini test suite, both live-verified against the
dev instance:
1. sendAndValidate() (tests/integration/liveGeminiShared.ts) now accepts an
apiFormat: "chat" | "responses" parameter, building the Responses-API
request shape (input array, max_output_tokens) and parsing its SSE events
(response.output_text.delta / response.reasoning_summary_text.delta /
response.completed) via the new readResponsesSSEStream(). Wired into two
new tests in live-gemini-workload.test.ts ([30]/[31]), mirroring the
existing Chat Completions streaming coverage. Verified live: 24/25 + 5/5
payloads succeeded end-to-end through the new code path (the one failure
was a ~300s test-client fetch timeout unrelated to the Responses API code
itself — a separate, not-yet-addressed test-harness limitation).
2. genHugeContextMessage() builds a single message large enough (~4
chars/token estimate) to approach or exceed Gemini's free-tier TPM ceiling
(16000 input tokens/min for gemma-4) by itself. Every other prompt
generator in this file tops out around 1-2k tokens — nowhere near that
ceiling — so none of the existing workload tests ever exercised a REAL TPM
429, only RPM-style rate limiting. tests/integration/gemini-large-context-tpm.test.ts
sends two ~12-13k-token requests back-to-back (comfortably exceeding
16000/min together) to exercise the full path against production Gemini:
TPM classification, the comboCooldownWait retry, and the synthetic
keep-alive frame on a genuinely slow request.
Co-authored-by: Markus Hartung <markus.hartung@gmail.com>
* fix(sse): abandoned combo target dispatch now observes its own per-target timeout, fixing a permanent "pending" dashboard leak
Live incident (dashboard log id 1784418258231-14961a, reported as "an ongoing
request even though there's already a 200"): a combo target dispatch
abandoned by comboTargetTimeoutMs (open-sse/services/combo/targetTimeoutRunner.ts)
left a permanent phantom "pending" entry in the dashboard, even after the
overall combo request had already succeeded via a different retry.
Root cause: chatCore.ts's createStreamController — and everything downstream
that depends on it (withRateLimit's Promise.race against Bottleneck,
acquireAccountSemaphore) — only ever watches clientRawRequest.signal, which
is the ORIGINAL client's request signal (set once via buildClientRawRequest
and reused unchanged across every target dispatch in a combo). It has no
connection to targetTimeoutRunner.ts's OWN AbortController
(target.modelAbortSignal), which is what actually fires when
comboTargetTimeoutMs (300s) elapses. src/sse/handlers/chat.ts's
handleSingleModel bridge between combo.ts and handleSingleModelChat received
`target.modelAbortSignal` but silently dropped it — never forwarded it
anywhere. So when a target got abandoned (e.g. stuck inside a wedged
Bottleneck rate-limiter queue, see the WEDGED force-reset log line from the
same incident), its per-target timeout fired and let the COMBO move on and
retry successfully elsewhere — but the abandoned dispatch's own promise
chain never learned it had been superseded, so it hung forever waiting on a
signal that was never going to fire, and trackPendingRequest(false) (the
finalize call) never ran.
Fix: thread target.modelAbortSignal through as a new modelAbortSignal
runtimeOption, and merge it into clientRawRequest.signal (via the existing
mergeAbortSignals helper from open-sse/executors/base.ts) right before
dispatch, so an abandoned target's own promise chain now observes its abort
and can reach its cleanup path — new resolveDispatchClientRawRequest() makes
this mechanically testable in isolation.
Co-authored-by: Markus Hartung <markus.hartung@gmail.com>
* fix(sse): combo cooldown-wait state recording, rate-limit wedge recovery, OpenAI-format SSE error frames
Five related fixes surfaced by live incidents (dashboard log ids 1784457764961-73,
1784465227489-a2cbc0, 1784504040241-6f8b9a) while validating the Gemini TPM/cooldown-wait
work on this branch against real OpenClaw traffic:
- combo.ts: the model-lockout bail-out branches in dispatchWithCooldownRetry never
recorded lastStatus, so once every target in a set hit an existing lockout the final
check crystallized a bogus ALL_ACCOUNTS_INACTIVE 503 instead of reaching the
cooldown-wait decision, even with a real 429 + short retry-after observed.
- combo.ts/combo/types.ts: the "all credentials cooling down" pre-dispatch rejection
(buildModelCooldownBody) nests its retry hint as error.retry_after/reset_seconds, not
the top-level retryAfter every other 429 shape uses — combo's extraction only read the
latter, so earliestRetryAfter stayed null for this shape even after lastStatus was fixed.
- rateLimitManager.ts: the wedge-recovery watchdog used disconnect(), which releases the
heartbeat timer but never rejects jobs already QUEUED on that instance — orphaned
dispatches hung until the outer ~300s per-target timeout, well past real clients'
patience. Switched to stop({ dropWaitingJobs: true }), safe because the wedge condition
already requires RUNNING===0 && EXECUTING===0.
- earlyStreamKeepalive.ts: the in-band error frame emitted after committing to a 200 SSE
stream was hardcoded to Anthropic's `event: error` convention for every route, including
the OpenAI-format ones (/v1/chat/completions, /v1/responses) where that framing is
either invisible or malformed to a plain data-line parser. Added per-route
OPENAI_CHAT_ERROR_FRAME / OPENAI_RESPONSES_ERROR_FRAME and wired them in.
- chatCore.ts: persisted a synthetic clientResponse error body even when the client had
already disconnected (AbortError) before that body was ever computed — misleading the
dashboard into showing "what the client received" for a response that was never sent.
Also: RequestLoggerDetail.tsx — Provider/Client Event Stream panes lost their collapse
toggle when StreamSection replaced the collapsible PayloadSection (692d6be80, unifying
active/finished request views) without carrying the toggle over.
Each fix has a TDD regression test with a confirmed red-before-green cycle.
Co-authored-by: Markus Hartung <markus.hartung@gmail.com>
* test(sse): free-tier model + gemma-4 TPM-ceiling benchmark harness
Adds a live benchmark comparing free models OmniRoute exposes across
configured providers plus previously-unexercised no-auth providers
(felo-web, aihorde, opencode, duckduckgo-web — none need a connection
row, they were just never tried). Reuses liveGeminiShared.ts's SSE
parsers and CASE_BUILDERS instead of duplicating them.
Also adds a targeted TPM-stress test firing back-to-back large-context
prompts at the gemma-4-31b model across its 3 free hosts (Gemini,
NVIDIA, AI Horde) to isolate whether the documented 16k-tokens/minute
free-tier ceiling is Gemini-specific enforcement or an inherent
model property.
FORCE_TOOL_CHOICE_REQUIRED is a test-only, default-off env flag added
to liveGeminiShared.ts and live-gemini-agentic-loop.test.ts for an
earlier live A/B comparison of tool_choice: required vs unset — kept
as a reusable knob for future runs.
Co-Authored-By: Markus Hartung <markus.hartung@gmail.com>
* test(sse): benchmark for the 2026-07-22 newly-enabled provider batch
Adds NEWLY_ENABLED_MODELS to freeModelBenchmarkShared.ts (Mistral
Leanstral, OpenRouter's live "free"-tagged roster, OpenCode Zen's
current free models — refetched live from
https://opencode.ai/zen/v1/models since the static catalog had
drifted) and a dedicated workload benchmark test for them.
Co-Authored-By: Markus Hartung <markus.hartung@gmail.com>
* test(sse): sync geminiRateLimitTracker tests with e74a1722b's corrected Gemma 4 limits
e74a1722b updated geminiRateLimits.json's gemma-4-* entries from the stale
15/1500/-1 (rpm/rpd/tpm) to the real published free-tier values
16000/14400/16000, but never updated the tests asserting the old numbers.
Surfaced by running the full test:unit suite as a post-rebase sanity check.
Co-Authored-By: Markus Hartung <markus.hartream@gmail.com>
* chore(quality): file-size baseline for own-growth (#8213)
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
---------
Co-authored-by: Markus Hartung <markus.hartung@gmail.com>
Co-authored-by: Markus Hartung <markus.hartream@gmail.com>
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
🚀 OmniRoute — The Free AI Gateway
💰 ~1.53B Free Tokens / Month
Stacking free tiers by hand is painful — dozens of SDKs, dozens of rate limits, and no idea how much you actually have. OmniRoute aggregates the documented free tiers of 43 provider pools / 460+ models into one honest number and shows it live on the dashboard (
/dashboard/free-tiers).
Animated summary of the live
/dashboard/free-tierspage. Full methodology (pool dedupe, credit tiers, provider terms): docs/reference/FREE_TIERS.md.These figures are re-audited every two weeks against the live catalog and move both ways — a provider ends a free tier and the number drops; a new one lands and it climbs. We publish what the catalog actually computes, never a rounded-up best case. A CI gate (
check:docs-counts) fails the build if this headline drifts from the code.
⭐ Star the repo if OMNIROUTE helped you save money and make your work easier.
💬 Join the community
Questions, provider tips, roadmap & support → Discord · Telegram · WhatsApp 🌍 Global / 🇧🇷 Brasil
🧩 Available
🚀 Quick Start • 🎯 Combos • 🌐 Providers • 🔌 CLI & MCP • 🗜️ Compression • 🌍 Website
💥 The Promise • 🤔 Why • 🏆 What Sets Apart • 🤖 Compatible CLIs • 🖥️ Where It Runs • 🔒 Private • 🎬 In Action • 📸 Screenshots • 📧 Support
🌐 In 43 languages
| 🇺🇸 | 🇧🇷 | 🇵🇹 | 🇪🇸 | 🇫🇷 | 🇮🇹 | 🇩🇪 | 🇳🇱 | 🇷🇺 | 🇺🇦 | 🇵🇱 | 🇨🇿 | 🇸🇰 | 🇷🇴 | 🇭🇺 |
| 🇧🇬 | 🇩🇰 | 🇫🇮 | 🇳🇴 | 🇸🇪 | 🇨🇳 | 🇹🇼 | 🇯🇵 | 🇰🇷 | 🇹🇭 | 🇻🇳 | 🇮🇩 | 🇲🇾 | 🇵🇭 | |
| 🇮🇳 | 🇮🇳 | 🇮🇳 | 🇮🇳 | 🇮🇳 | 🇮🇳 | 🇧🇩 | 🇵🇰 | 🇮🇷 | 🇸🇦 | 🇮🇱 | 🇹🇷 | 🇦🇿 | 🇹🇿 |
🆓 Works the second you install it — no keys, no config
Install → point your tool at the endpoint → it already answers. OmniRoute ships with keyless free providers (OpenCode Free, Felo) already wired into the
autocombo, so a fresh install responds out of the box — no API key, no signup, no configuration.
# Fresh install, zero credentials — `auto` already works:
curl http://localhost:20128/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{"model":"auto","messages":[{"role":"user","content":"Hello!"}]}'
- ✅
autoresponds immediately — OmniRoute builds a virtual combo from the built-in keyless free providers and routes to a healthy one, with no setup. - ➕ Add more providers anytime — drop in a Claude / GPT / Gemini key (or any of the 278 providers) from the dashboard and they join the
autopool automatically. - 🎛️ Build your own free combos — chain your free tiers with any of the 19 routing strategies so you never run out of quota.
Prefer a specific free backend? Call it directly, e.g. oc/… (OpenCode Free) or felo/… (Felo). Then graduate to auto and let OmniRoute pick.
💥 The Promise
🤔 Why OmniRoute?
🎯 Combos — The Flagship
A combo is a chain of models OmniRoute routes across automatically. Quota runs out, a provider fails, or costs spike — the combo silently slides to the next model. This is what makes OmniRoute unbreakable. 🛡️
⚡ Zero-config — just use auto
No combo to create. Set your model to auto (or a variant) and OmniRoute builds a virtual combo from your connected providers, scored live:
| Model ID | What it optimizes for |
|---|---|
auto |
🎯 Balanced default (LKGP — sticks to your last good provider) |
auto/coding |
🧑💻 Quality-first weights for code generation |
auto/fast |
⚡ Lowest latency first |
auto/cheap |
💰 Cheapest per token first |
auto/offline |
🔋 Most quota / rate-limit headroom first |
auto/smart |
🔭 Quality-first + 10% exploration to discover better models |
🔀 Or build your own — 19 routing strategies
All 19 strategies — mix & match per combo step:
| # | Strategy | What it does |
|---|---|---|
| 1 | priority |
First-target ordered list — drain each before the next 🥇 |
| 2 | fill-first |
Fill each target's quota fully before moving on |
| 3 | weighted |
Weighted random by per-target weight |
| 4 | round-robin |
Cycle through targets in order |
| 5 | p2c |
Power-of-two-choices random load balancing |
| 6 | least-used |
Pick the target with the lowest current load |
| 7 | random |
Uniform random pick (deduplicated) |
| 8 | strict-random |
Random without de-duplicating repeats 🎲 |
| 9 | cost-optimized |
Minimize $ per request from live catalog pricing 💸 |
| 10 | headroom |
Pick the target with the most remaining quota |
| 11 | reset-window |
Prefer the target whose quota window resets soonest |
| 12 | reset-aware |
Rank by quota reset time — short windows first 📊 |
| 13 | context-relay |
Hand off context across targets for long conversations 🧠 |
| 14 | context-optimized |
Pick the best fit for the current context size |
| 15 | cache-optimized |
Pin each reusable prompt prefix to the same account — maximize prompt-cache hits 🎯 |
| 16 | lkgp |
Last-Known-Good Path — sticky to the last successful target |
| 17 | auto |
12-factor live scoring across every connection 🤖 |
| 18 | fusion |
Fan out to a panel of models + a judge synthesizes one answer 🧬 |
| 19 | pipeline |
Chain steps — each target's output feeds the next one 🔗 |
The Auto-Combo engine scores every candidate on 12 factors (health, quota, cost, latency, success rate, freshness…) — see docs/routing/AUTO-COMBO.md.
⚖️ Quota-Share — split one subscription across a team ✨ NEW
Running several keys against the same upstream account (one Codex Pro plan, one Kimi key, one GLM Coding seat)? A burst on one key can burn the whole 5-hour / hourly quota and lock everyone else out. Quota-Share distributes a provider's time-based quota fairly across the keys in a pool — and it's work-conserving, so an idle member's slice is lent out instead of wasted.
| Knob | What it controls |
|---|---|
| ⚖️ Allocation weight | each key's slice of the pool — e.g. 50 / 30 / 20 |
| 📐 Dimensions | track % · requests · tokens · $, per 5h / 7d / per-model window |
| 🚦 Policy | hard (block over share) · soft (deprioritize) · burst (use idle headroom) |
| 🧱 Cap | absolute ceiling per key, independent of mode |
Enforced in the hot path before the request leaves OmniRoute, with per-(key, model) caps + session stickiness for prompt-cache integrity (now with a per-combo / global disable toggle). 📖 Quota Sharing Engine
🧱 Resilience is built in (3 independent layers)
📖 Auto-Combo Engine · Resilience Guide
🏆 What Sets OmniRoute Apart
| Feature | OmniRoute | Other routers |
|---|---|---|
| 🌐 Providers | 278 | 20–100 |
| 🆓 Free providers | 90+ (40+ free forever) | 1–5 |
| 🔀 Routing strategies | 19 strategies | 1–3 |
| 🗜️ Token compression | RTK + Caveman stacked (15–95%) | None / 20–40% |
| 🧰 Built-in MCP server | 104 tools, 3 transports, 31 scopes | Rare |
| 🤝 A2A agent protocol | 6 skills, JSON-RPC 2.0 | None |
| 🧠 Memory (FTS5 + vector) | Yes | Rare |
| 🛡️ Guardrails (PII, injection, vision) | Yes | Rare |
| ☁️ Cloud agents | Codex, Cursor, Devin, Jules | None |
| 🥷 TLS fingerprint stealth | JA3/JA4 via wreq-js | None |
| 🖥️ Multi-platform | Web · Desktop · Termux · PWA | Web only |
| 🌍 i18n | 43 locales | 0–4 |
📊 Detailed comparison vs LiteLLM, OpenRouter & Portkey → docs/comparison/OMNIROUTE_VS_ALTERNATIVES.md
✨ What's New
Recent highlights from v3.8.20 → v3.8.49. Full history in
CHANGELOG.md.
- 🗜️ Compression hardening — default-on inflation guard, Caveman packs for DE / FR / JA + Chinese (wényán), RTK filters for Gradle & .NET. → Compression
- 💸 Honest flat-rate cost — subscription / coding-plan providers read $0 in cost analytics; budget, quota & routing keep estimating. → API Reference
- ⚖️ Quota-Share routing — split a shared account's quota fairly across pooled keys, work-conserving so idle slices are lent out. → Resilience Guide
- 🤖 One-command CLI/agent setup —
setup-*configures 12+ coding tools;omniroute launch/launch-codexare zero-config. → CLI Integrations - 🛰️ Remote mode — drive a remote OmniRoute with scoped tokens (
connect/contexts/tokens) + anantigravityOAuth helper for VPS installs. → Remote Mode - 🧭 Smarter auto-routing —
auto/<category>:<tier>combos, Fusion (model panel + judge), task-aware routing, per-request model / mode / USD-budget overrides. → Auto-Combo - 🗜️ Pluggable compression — 12 composable engines + Compression Studios: LLMLingua-2, two-tier Ultra, omniglyph, per-step fidelity gate, GCF v3.2, drag-reorder editor. → Compression
- 🕵️ Transparent MITM decrypt (TPROXY) — capture CLIs that ignore proxy env vars, with a per-SNI CA + trust-store installer. → MITM/TPROXY
- 💸 Cost telemetry everywhere —
X-OmniRoute-*cost/usage headers on every endpoint, cache-HIT savings header, per-key USD spend quotas. → API Reference - 🧠 Memory you control — off by default, opt-in int8 vector quantization + typed decay, per-request
x-omniroute-no-memory. → Memory - 🛡️ Security — prompt-injection guard on every LLM route (red-team suite) + free DuckDuckGo last-resort web search. → Guardrails
- 🖼️ New endpoints —
/v1/ocr(Mistral OCR) and/v1/audio/translations(Whisper-style) round out the media surface. → API Reference - 🌍 Deployment & ops — reverse-proxy
basePath, browser-language auto-detect, per-key device tracking, root-less MITM trust, zh-TW localization. → Environment - 🤝 More providers & agents — Cursor Cloud Agent, Grok Build (xAI), Ollama first-class card, Claude Sonnet 5, Zed, Requesty, SenseNova, Yuanbao… and a refreshed 250-provider catalog. → Providers
- ⚡ Local performance & infra — one-click local Redis, Cloudflare Workers / Deno Deploy relay deployers, Bifrost & Mux as supervised embedded services. → Embedded Services
🤖 Compatible CLIs & Coding Agents
One config —
http://localhost:20128/v1— and every AI IDE or CLI runs on free & low-cost models.
 Claude Code |
 Codex CLI |
 Cline |
 Kilo Code |
  Roo Code |
 Continue |
 Aider |
ForgeCode |
jcode |
 DeepSeek TUI |
CodeWhale |
 OpenCode |
 Factory Droid |
 Copilot CLI |
 Cursor CLI |
Smelt |
Pi |
 Grok Build |
 Hermes Agent |
 OpenClaw |
 Goose |
Open Interpreter |
Warp AI |
Agent Deck |
＋ also works with · Kiro · Command Code · Antigravity · Windsurf · AMP · any OpenAI-compatible tool
📖 Per-tool setup for all 33 tools (25 CLI Code's + 8 CLI Agents) → docs/reference/CLI-TOOLS.md · 🧩 OpenCode plugin → @omniroute/opencode-provider
🌐 278 AI Providers — 90+ Free
The most complete catalog of any open-source router: 278 providers, 90+ with a free tier, 40+ free forever.
🏢 Every major lab — through one endpoint
 OpenAI |
 Anthropic |
 Gemini |
 xAI Grok |
 DeepSeek |
 Mistral |
 Qwen |
 Meta Llama |
 Groq |
 NVIDIA |
 MiniMax |
 Cohere |
 Perplexity |
 HuggingFace |
 Together |
 Fireworks |
 Cloudflare |
 Baidu |
…and 220+ more — every icon resolves live from the dashboard's provider catalog. 📖 Provider Reference
🆓 Free Forever — $0, no card
 AgentRouter GPT-5, Claude, Gemini $100 free credits |
 Qoder AI Kimi-K2, DeepSeek-R1 Unlimited FREE |
 Pollinations GPT-5, Claude, Llama 4 No key needed |
 LongCat LongCat-2.0 10M tokens one-time (KYC) 🔑 |
Cloudflare AI 50+ models 10K neurons/day |
NVIDIA NIM 129 models ~40 RPM free |
 Cerebras Qwen3 235B 1M tokens/day |
📖 Full machine-readable catalog → docs/reference/PROVIDER_REFERENCE.md
🖥️ Where OmniRoute Runs — Anywhere
Same app, your machine, your rules. From a global npm install to your phone via Termux.
| Platform | Install | Highlights |
|---|---|---|
| 📦 npm (global) | npm install -g omniroute |
One command, any OS |
| 🐳 Docker | docker run … diegosouzapw/omniroute |
Multi-arch AMD64 + ARM64 |
| 🖥️ Desktop (Electron) | npm run electron:build |
Native window + system tray — Windows / macOS / Linux |
| 💪 ARM | native arm64 |
Raspberry Pi, ARM servers, Apple Silicon |
| 📱 Android (Termux) | pkg install nodejs && npx -y omniroute |
Runs on your phone, 24/7, no root |
| 📲 PWA | "Add to Home Screen" | Fullscreen, offline, installable from browser |
| 🧩 OpenCode plugin | @omniroute/opencode-provider |
Native OpenCode integration |
| 🛠️ From source | npm install && npm run dev |
Hack on it, contribute |
📖 Docker Guide · Desktop · Termux · PWA · OpenCode
🔒 Private & Local-First
📖 Authorization · Guardrails · Compliance
🔌 Full CLI + A2A & MCP
Beyond the server, OmniRoute is a full command-line cockpit with 80+ commands, plus open agent protocols so an AI agent can drive it on its own.
⌨️ A real CLI (not just start)
omniroute # serve gateway + dashboard (port 20128)
omniroute chat # interactive TUI chat client (slash: /model /combo /skill /memory)
omniroute setup # guided first-run wizard
omniroute doctor # diagnose providers, ports, native deps
🛰️ Remote mode — run the CLI here, OmniRoute on a VPS
OmniRoute on a server? Drive it from your laptop with the same CLI. Log in once with a scoped access token; every command then targets the remote.
omniroute connect 192.168.0.15 # password → scoped token, saved as a context
omniroute models list # ← runs against the REMOTE server
omniroute configure codex # ← picks a remote model, writes a local Codex profile
omniroute tokens create --name ci --scope read # mint narrower tokens for other machines
omniroute contexts use default # ← switch back to the local server
Tokens are scoped read / write / admin; process-spawning routes stay loopback-only.
📖 Remote Mode
🤝 Connect an agent — and it controls OmniRoute itself
Expose OmniRoute over MCP or A2A and any capable agent gets the keys to the whole gateway — routing, providers, combos, cache, compression, memory — autonomously. HTTP endpoints below are served under http://localhost:20128.
| Protocol | Endpoint | Use it for |
|---|---|---|
| 🧰 MCP (stdio) | omniroute --mcp |
Plug into Claude Desktop, Cursor, any MCP client |
| 🌊 MCP (HTTP) | /api/mcp/stream |
Remote MCP — 104 tools, 31 scopes, full audit trail |
| 📡 MCP (SSE) | /api/mcp/sse |
Streaming MCP transport |
| 🤝 A2A | /.well-known/agent.json |
Agent-to-agent, JSON-RPC 2.0 + SSE, 6 skills |
# Give Claude Code the full OmniRoute toolset over MCP:
claude mcp add-server omniroute --type http --url http://localhost:20128/api/mcp/stream
📖 MCP Server · A2A Server · Agent Protocols
🗜️ Save 15–95% Tokens — Automatically
Why use many tokens when few tokens do the trick? Every request passes through OmniRoute's compression pipeline transparently — no client changes. It's now a stack of 12 composable engines that run in order and mix & match per routing combo — building on ideas from RTK, Caveman (⭐ 90K+), LLMLingua-2, and Troglodita (PT-BR).
🧱 The 12-engine stack
Engines run in pipeline order; each is independently toggleable and configurable per combo:
| # | Engine | What it does |
|---|---|---|
| 1 | Session-Dedup | Drops content repeated across turns (content-addressed, cross-turn) |
| 2 | CCR | Archives large blocks behind retrieve markers, fetched on demand |
| 3 | Lite | Whitespace + image-URL trimming (latency-light baseline) |
| 4 | RTK | Smart tool-result filtering, dedup & truncation (command-aware) |
| 5 | Responses Tool Output | Lossless-first JSON + bounded diagnostic compression for shell/patch/search/build outputs (Responses API) |
| 6 | Headroom | Lossless tabular compaction of JSON arrays (~30%) via a vendored GCF codec |
| 7 | Relevance | Extractive sentence scoring against the last user query |
| 8 | Caveman | Rule-based prose compression (~65–75% on output) |
| 9 | Aggressive | Summarization + progressive aging of old turns |
| 10 | LLMLingua-2 | ML semantic pruning via MobileBERT ONNX — code-safe, async |
| 11 | Ultra | Heuristic token pruning with an optional small-model (SLM) tier |
| 12 | OmniGlyph | Experimental context-as-image encoding routed to Claude Fable 5 (most aggressive; opt-in) |
Code blocks, URLs and structured data are always preserved byte-perfect. One-click presets combine the engines:
| Mode | Savings | Best for |
|---|---|---|
| 🪶 Lite | ~15% | Always-on safe default |
| 🪨 Standard (Caveman) | ~30% | Daily coding |
| ⚡ Aggressive | ~50% | Long tool-heavy sessions |
| 🔥 Ultra | ~75% | Maximum savings |
| 🧰 RTK | 60–90% | Shell/test/build/git output |
| 🔗 Stacked (RTK → Caveman) | 78–95% | Mixed prompts + tool logs |
Real example — Standard mode:
Before (69 tokens): "The reason your React component is re-rendering is likely because you're creating a new object reference on each render cycle. When you pass an inline object as a prop, React's shallow comparison sees it as a different object every time, which triggers a re-render. I would recommend using useMemo to memoize the object."
After (19 tokens): "New object ref each render. Inline object prop = new ref = re-render. Wrap in useMemo."
Same answer. 72% fewer tokens. Zero accuracy loss. ✅
PT-BR example — Troglodita mode:
Antes (42 tokens): "O problema é que o componente está re-renderizando porque uma nova referência de objeto está sendo criada em cada ciclo de renderização. Eu recomendaria usar useMemo."
Depois (12 tokens): "Re-render: ref nova cada ciclo (objeto inline recriado). Usar
useMemo."Mesma resposta. ~70% menos tokens. Precisão técnica intacta. ✅
📖 How it works — pipeline, architecture & savings math
Default stacked combo runs RTK → Caveman. When both act on the same tool/context payload, savings compound:
combined = 1 − (1 − RTK) × (1 − Caveman_input)
average = 1 − (1 − 0.80) × (1 − 0.46) = 89.2%
range = 78.4 – 94.6%
Code blocks, URLs, JSON and structured data are always protected by the preservation engine.
🎚️ Beyond the engines — output styles, the adaptive dial & per-request control
The 12 engines above shrink what goes in. Three more layers shape how, when, and what comes out:
- 🪄 Output Styles (output-axis steering) — inject deterministic, cache-safe response-shaping instructions; combinable, each at
lite/full/ultraintensity. Adding a style is a one-line registry entry:- Terse prose — drop filler / articles / hedging; keep technical substance exact.
- Less code — "lazy senior dev" YAGNI: smallest working change, no unrequested scaffolding.
- Terse CJK (文言) — classical-Chinese ultra-terse style (locale-gated to
zh).
- 🎯 Adaptive context-budget (the dial) — instead of one on/off token threshold, escalate the cheapest, most-lossless engines only as far as needed to fit the model's context window. Policy:
reserve-output(default, model-aware) ·percentage·absolute. Mode:floor(guarantee fit) ·replace-autotrigger(your explicit choice wins) ·off(legacy threshold). - 🎛️ Where compression is decided (precedence, high → low) — per-request
x-omniroute-compressionheader › routing-combo override › active named profile › adaptive / auto-trigger › panel default › off. The applied plan echoes back in theX-OmniRoute-Compression: <mode>; source=<source>response header.
Auto-trigger by token threshold, flip on the adaptive dial, pin a named profile, set a one-off per request, or assign a pipeline per routing combo — whichever fits the workload. An opt-in offline eval harness (npm run eval:compression) scores fidelity vs. savings on a pinned corpus before you promote a change.
📖 COMPRESSION_GUIDE.md · RTK_COMPRESSION.md · COMPRESSION_ENGINES.md
⚡ Quick Start
1) Install & run
npm install -g omniroute
omniroute
💡 See
npm warn ERESOLVEor peer-dep warnings? They're harmless.
Dashboard at http://localhost:20128 · API at http://localhost:20128/v1.
2) Connect a FREE provider (no signup)
Dashboard → Providers → connect Kiro AI (free Claude, ~50 credits/month per account) or OpenCode Free (no auth) → done.
3) Point your coding tool
Base URL: http://localhost:20128/v1
API Key: [copy from Dashboard → Endpoints]
Model: auto (zero-config smart routing — or any provider/model)
4) Verify it's working
curl http://localhost:20128/v1/models -H "Authorization: Bearer YOUR_KEY"
You should see your connected models listed. 🎉 That's it — start coding, and OmniRoute auto-routes & falls back for you.
If your client cannot send custom headers, OmniRoute also exposes tokenized compatibility aliases:
OpenAI catalog: http://localhost:20128/vscode/YOUR_KEY/
OpenAI models: http://localhost:20128/vscode/YOUR_KEY/models
OpenAI chat: http://localhost:20128/vscode/YOUR_KEY/chat/completions
OpenAI responses: http://localhost:20128/vscode/YOUR_KEY/responses
Ollama chat: http://localhost:20128/vscode/YOUR_KEY/api/chat
Ollama tags: http://localhost:20128/vscode/YOUR_KEY/api/tags
Use these only for clients that cannot attach Authorization: Bearer .... Header auth remains the preferred mode.
📦 More install methods — Docker, source, pnpm, Arch
🐳 Docker
docker run -d --name omniroute --restart unless-stopped --stop-timeout 40 \
-p 127.0.0.1:20128:20128 -v omniroute-data:/app/data diegosouzapw/omniroute:latest
🛠️ From source
cp .env.example .env && npm install
PORT=20128 npm run dev
📦 pnpm
pnpm add -g omniroute@latest --allow-build=better-sqlite3 --allow-build=@swc/core && omniroute
🐧 Arch Linux (AUR)
yay -S omniroute-bin && systemctl --user enable --now omniroute.service
🔧 Nix (Flake)
# Using Nix flakes
nix develop
npm run dev
# Or using devbox
devbox run npm run dev
📖 Docker Guide — Compose profiles, Caddy HTTPS, Cloudflare tunnels.
🦭 Podman
# 1. Build the image
podman build --target runner-base -t omniroute:base .
# 2. Fix data directory permissions for rootless Podman
mkdir -p data && podman unshare chown 1000:1000 ./data
# 3. Set runtime in .env, then run (see contrib/podman/ for Quadlet)
echo "CONTAINER_HOST=podman" >> .env
podman compose --profile base up -d
📖 Podman Guide — Quadlet setup, podman-compose, Quadlet.
⚡ Faster / leaner install (skip the native build)
The native SQLite engine (better-sqlite3) is an optional dependency, so a global
install never blocks on compiling from source: it uses a prebuilt binary when one matches
your platform/Node, and otherwise falls back transparently to a pure-JS engine
(node:sqlite on Node 22+, else the bundled sql.js WASM) — no build tools required.
To skip the post-install native warm-up entirely (CI, headless, or slow machines):
OMNIROUTE_SKIP_POSTINSTALL=1 npm install -g omniroute # CI=1 also skips it
For the fastest installs prefer pnpm (content-addressed store + hard links — see above).
For a dashboard-free, headless runtime use the Docker base profile (above) or the
Termux guide. The CLI and the web dashboard are served by the
same process on one port, so there is no separate CLI-only package today.
🎬 OmniRoute in Action
 🇧🇷 Português Guia completo |
 🇺🇸 English Complete walkthrough |
 🇷🇺 Русский Полное руководство |
🎬 Made a video about OmniRoute? Open an issue or discussion with the link — we'll feature it here.
📸 Dashboard Screenshots
Providers |
Combos |
Analytics |
Health |
Translator |
Settings |
CLI Tools |
Usage Logs |
📧 Support & Community
💬 Chat with the community — Discord, Telegram & WhatsApp (🌍 / 🇧🇷) links are at the top of this README.
- 🌍 Website: omniroute.online
- 🐙 GitHub: github.com/diegosouzapw/OmniRoute
- 🐛 Issues: report a bug (attach
npm run system-infooutput) - 🤝 Contributing: see CONTRIBUTING.md, Branching & Release Model, or pick a
good first issue
🛠️ Tech Stack
- Runtime: Node.js 22.x or 24.x LTS (24 LTS recommended) —
>=22.22.2 <23 || >=24.0.0 <27 - Language: TypeScript 6.0 — 100% TypeScript across
src/andopen-sse/(zeroanyin core modules since v2.0) - Framework: Next.js 16 + React 19 + Tailwind CSS 4
- Database: better-sqlite3 (SQLite) + LowDB (JSON legacy) — domain state, proxy logs, MCP audit, routing decisions, memory, skills
- Schemas: Zod (MCP tool I/O validation, API contracts)
- Protocols: MCP (stdio/HTTP) + A2A v0.3 (JSON-RPC 2.0 + SSE)
- Streaming: Server-Sent Events (SSE) + WebSocket bridge (
/v1/ws) - Auth: OAuth 2.0 (PKCE) + JWT + API Keys + MCP Scoped Authorization
- Testing: Node.js test runner + Vitest (25,000+ test cases across 3,300+ files — unit, integration, E2E, security, ecosystem)
- Platforms: Desktop (Electron), Android (Termux), PWA (any browser)
- CI/CD: GitHub Actions (auto npm publish + Docker Hub on release)
- Website: omniroute.online
- Package: npmjs.com/package/omniroute
- Docker: hub.docker.com/r/diegosouzapw/omniroute
- Resilience: Circuit breaker, exponential backoff, anti-thundering herd, TLS spoofing, auto-combo self-healing
📖 Documentation
📘 Getting Started
| Document | Description |
|---|---|
| User Guide | Providers, combos, CLI integration, deployment |
| Setup Guide | Full install methods, CLI tool configs, protocol setup, timeout tuning |
| CLI Tools Guide | Per-tool setup for Claude Code, Codex, Cursor, Cline, OpenClaw, Kilo, Copilot |
| Remote Mode | Drive a remote OmniRoute (VPS) from your laptop CLI via scoped access tokens |
| Claude Code Config | Point Claude Code at OmniRoute (local/remote) with launch + per-model profiles |
| Quick Start | 3-step install → connect → configure |
🔧 Operations & Deployment
| Document | Description |
|---|---|
| Docker Guide | Docker run, Compose profiles, Caddy HTTPS, tunnels, image tags |
| Podman Guide | Quadlet systemd integration, podman-compose, SELinux |
| VM Deployment | Complete guide: VM + nginx + Cloudflare setup |
| Fly.io Deployment | Deploy to Fly.io with persistent storage |
| Termux Guide | Run OmniRoute on Android via Termux |
| PWA Guide | Progressive Web App install, caching, architecture |
| Uninstall Guide | Clean removal for all install methods |
| Environment Config | Complete .env variables and references |
🧠 Features & Architecture
| Document | Description |
|---|---|
| Architecture | System architecture, data flow, and internals |
| Compression Guide | 7-option pipeline: off / lite / standard / aggressive / ultra / RTK / stacked |
| RTK Compression | Command-output compression, filters, trust, verify, raw-output recovery |
| Compression Engines | Caveman, RTK, stacked pipelines, dashboard/API/MCP surfaces |
| Compression Rules Format | JSON rule-pack schemas for Caveman and RTK filters |
| Compression Language Packs | Language detection and Caveman rule-pack authoring |
| Resilience Guide | Circuit breakers, cooldowns, queue, anti-thundering herd, TLS spoofing |
| Auto-Combo Engine | 12-factor scoring, mode packs, self-healing |
| Proxy Guide | 3-level proxy system, 1proxy marketplace, registry CRUD |
| Free Tiers | 25+ free API providers consolidated directory |
| Features Gallery | Visual dashboard tour with screenshots |
| Codebase Documentation | Beginner-friendly codebase walkthrough |
🤖 Protocols & APIs
| Document | Description |
|---|---|
| API Reference | All endpoints with examples |
| OpenAPI Spec | OpenAPI 3.0 specification |
| MCP Server | 104 MCP tools, IDE configs, Python/TS/Go clients |
| MCP Server Guide | MCP installation, transports, and tool reference |
| A2A Server | JSON-RPC 2.0 protocol, skills, streaming, task mgmt |
| A2A Server Guide | A2A agent card, tasks, skills, and streaming |
📋 Project & Quality
| Document | Description |
|---|---|
| Contributing | Development setup and guidelines |
| Branching & Release Model | Where PRs target (release/*), what main and tags mean |
| Changelog | Full per-version release history |
| Security Policy | Vulnerability reporting and security practices |
| i18n Guide | 40+ language support, translation workflow, RTL |
| Release Checklist | Pre-release validation steps |
| Coverage Plan | Test coverage strategy and 25,000+ test suite |
⭐ Top Contributors
OmniRoute is shaped by a passionate open-source community. These individuals have made exceptional contributions that directly impact the quality, stability, and reach of the project. Thank you.
 oyi77 🥇 213 commits • +114K lines Analytics engine, SQL aggregations, proxy marketplace, test coverage |
 R.D. & Randi 🥈 108 commits • +38K lines Endpoints page, tunnel integrations, Docker workflows, A2A status, compression UI |
 Chris Staley 🥉 70 commits • +1.8K lines SSE stream hardening, Responses API, Gemini pagination, test regression fixes |
 zenobit 🏅 62 commits • +22K lines CI/CD pipeline, i18n for 33 languages, Void Linux package, platform fixes |
 Jan Leon 🏅 58 commits • +22K lines Reasoning-effort routing, proxy controls, quota visibility, Live Zone compression |
 backryun 🏅 53 commits • +70K lines Provider catalog curation — Perplexity, Kimi, Cerebras, Copilot, LMArena refreshes |
 Chirag Singhal 🏅 46 commits • +4.8K lines Error sanitization, MITM prefill fix, fusion judge, breaker/429 correctness |
 kfiramar 🏅 38 commits • +1.7K lines Codex websocket + passthrough, auth/onboarding, Electron hardening, DB migrations |
 Benson K B 🏅 28 commits • +9.2K lines Electron desktop app, auto-updater, release build workflows, cross-platform CI |
 Hernan J. Ardila 🏅 25 commits • +174K lines Zero-latency combos, vision-bridge auto-routing, catalog context-length, resilience 429 hints |
🙏 These contributors' features, bug fixes, and infrastructure improvements are a core part of what makes OmniRoute reliable and feature-rich. Every pull request, every test case, and every i18n translation file matters. Open source is built by people like them.
👥 500+ Contributors
How to Contribute
- Fork the repository
- Branch from the active
release/vX.Y.Ztip (notmain) — see Branching & Release Model - Create your feature branch (
git checkout -b feat/amazing-feature) - Commit your changes (
git commit -m 'feat: add amazing feature') - Push to the branch (
git push origin feat/amazing-feature) - Open a Pull Request with base = that
release/vX.Y.Zbranch
See CONTRIBUTING.md for detailed guidelines.
Releasing a New Version
# Create a release — npm publish happens automatically
gh release create v3.8.2 --title "v3.8.2" --generate-notes
📊 Stars
🌍 StarMapper
🙏 Acknowledgments
OmniRoute stands on the shoulders of giants. It started as a fork of 9router and a TypeScript port of the Go project CLIProxyAPI — and from there, every subsystem below was inspired by an open-source project that got there first. Each one shaped a concrete piece of OmniRoute. This is our thank-you to all of them. 🙏
⭐ star counts as of July 2026 — go give these projects a star.
🧬 Lineage & gateway
| Project | ⭐ | How it inspired OmniRoute |
|---|---|---|
| 9router · decolua | 22.7k | The original project this fork is built on — extended here with multi-modal APIs and a full TypeScript rewrite. |
| CLIProxyAPI · router-for-me | 43.6k | The Go implementation that inspired this JavaScript / TypeScript port. |
| LiteLLM · BerriAI | 54.0k | The AI gateway whose public pricing dataset feeds our cost-tracking sync and whose provider-normalization model informed our routing. |
🗜️ Context & token compression — engines
| Project | ⭐ | How it inspired OmniRoute |
|---|---|---|
| Caveman · JuliusBrussee | 90.8k | The viral "why use many token when few token do trick" project — its caveman-speak philosophy powers our standard compression mode and 30+ filler/condensation rules. |
| RTK – Rust Token Killer · rtk-ai | 71.8k | High-performance command-output compression — inspired our RTK engine, JSON filter DSL, raw-output recovery and the stacked RTK → Caveman pipeline. |
| headroom · headroomlabs-ai | 60.1k | Reversible context-compression (SmartCrusher) — inspired our headroom engine and the ccr retrieve-marker pattern. |
| LLMLingua · Microsoft | 6.5k | Prompt-compression research (LLMLingua / LLMLingua-2) — inspired our async, code-safe, fail-open llmlingua engine. |
| llmlingua-2-js · atjsh | 30 | The JS/ONNX port (MobileBERT / XLM-RoBERTa) used as the worker-thread backend for our LLMLingua engine. |
| Troglodita · Lenine Júnior | 26 | PT-BR token compression — powers our pt-BR language pack: pleonasm reduction and filler removal tuned for Brazilian-Portuguese grammar. |
| ponytail · DietrichGebert | 86.0k | The viral "lazy senior dev" YAGNI-coder skill — inspired our less-code Output Style: smallest-working-change steering that cuts generated code (the output-axis sibling to Caveman's terse prose). |
🧩 Compact formats, token research & code-aware tooling
| Project | ⭐ | How it inspired OmniRoute |
|---|---|---|
| TOON · toon-format | 24.9k | Token-Oriented Object Notation — its columnar, header-plus-rows model shaped our tabular compaction stage. |
| GCF – Graph Compact Format · Blackwell Systems | 22 | First inspired our tabular compaction stage; now its zero-dependency, lossless generic-profile encoder is vendored directly as the Headroom codec (MIT, SPDX-marked), current with GCF spec v3.2. |
| token-optimizer-mcp · ooples | 444 | Brotli/SQLite cache + per-session context-delta — inspired our session-dedup engine. |
| token-savior · Mibayy | 1.1k | Bash-output compaction + MCP profiles — inspired our compression bail-out discipline and MCP tool-manifest reduction. |
| token-saver · ppgranger | 117 | Content-aware, per-file-type output compression with failure-aware bail-out — validated our per-type dispatch and minimum-gain skip. |
| token-optimizer · alexgreensh | 1.7k | "Find the ghost tokens" — its offload + recoverable-handle pattern informed our CCR offload thinking. |
| TokenMizer · Shweta-Mishra-ai | 16 | A session-graph + cross-turn line-dedup blueprint that informed our session-dedup design. |
| OmniCompress · jessefreitas | 3 | Rust columnar-JSON + content-addressed retrieve + cross-message dedup — validated our headroom/ccr/session-dedup engine design and the cache-stable "compressed form is position-independent" invariant. |
| mcp-compressor · Atlassian Labs | 98 | MCP tool-schema/description compression — informed our MCP tool-manifest cardinality reduction. |
| RepoMapper · pdavis68 | 187 | Aider-style repo-map ranking — informed our repo-map / retrieval-ranking exploration. |
| quiet-shell-mcp · mrsimpson | 4 | Declarative shell-output reduction over MCP — validated our declarative bash-output compaction. |
| ts-morph · David Sherret | 6.1k | TypeScript Compiler API toolkit — inspired our parser-based comment removal that preserves string, template and regex literals. |
🧠 Memory & RAG
| Project | ⭐ | How it inspired OmniRoute |
|---|---|---|
| Mem0 · mem0ai | 61.2k | Universal memory layer — its proxy-as-write/read-boundary model shaped our memory architecture. |
| Letta (MemGPT) · letta-ai | 23.9k | Stateful agents with tiered memory — inspired our Context Control & Recovery (CCR) tiered model. |
| WFGY · onestardao | 1.8k | The ProblemMap taxonomy of 16 recurring RAG/LLM failure modes — the shared vocabulary in our troubleshooting guide. |
🛰️ Traffic inspection, MITM & transparent proxy
| Project | ⭐ | How it inspired OmniRoute |
|---|---|---|
| llm-interceptor · chouzz | 49 | MITM interception/analysis of coding-assistant ↔ LLM traffic — our Traffic Inspector ports its SSE merge, conversation normalization, host passthrough and secret masking (MIT). |
| ProxyBridge · InterceptSuite | 5.5k | Transparent per-process proxy routing — inspired our crash-safe MITM teardown, socket idle-timeouts, /proc process attribution and TPROXY capture. |
📚 Model data, observability & UI
| Project | ⭐ | How it inspired OmniRoute |
|---|---|---|
| models.dev · SST / OpenCode | 6.0k | Open database of AI model specs, pricing and capabilities — synced natively into our model catalog. |
| React Flow / xyflow · xyflow | 37.7k | The node-based graph library powering our real-time Compression Studio and Combo/Routing Studio. |
| LangGraph · LangChain | 37.6k | LangGraph Studio's live workflow-graph visualization inspired our Studios' real-time cascade view. |
| Langfuse · Langfuse | 31.4k | Its trace → span → generation observability model shaped our Compression Studio waterfall. |
| Kiali · Kiali | 3.6k | Istio service-mesh observability — inspired our circuit-breaker badges and error-edge visuals in the Routing/Combo Studio. |
| lobe-icons · LobeHub | 2.2k | AI/LLM brand logos that render the provider icons across our dashboard. |
🛡️ Security
| Project | ⭐ | How it inspired OmniRoute |
|---|---|---|
| awesome-secure-defaults · tldrsec | 710 | A curated list of secure-by-default libraries that guides our security choices (Helmet.js, DOMPurify, ssrf-req-filter, safe-regex, Google Tink). |
🧭 Complementary tools
| Project | How it composes with OmniRoute |
|---|---|
| CodeWebChat · robertpiosik | Editor-side companion — VS Code + browser extension that autofills 15+ chatbot web UIs with editor context. Owns the free-web-UI rail alongside OmniRoute's API rail; can point its API mode at OmniRoute. |
🤝 Supported by our Open Source Friends
Want to join as an Open Source Friend? These are the companies that back open source and help keep OmniRoute moving — and we say publicly where every token they give us goes. Reach out: diegosouza.pw@outlook.com
Kimi Moonshot AI 
|
Thanks to Kimi (Moonshot AI), our founding Open Source Friend, for backing this project! Kimi is the AI lab behind the open-weight K2 and K3 model families — Kimi K3 delivers a 1M-token context window, native vision and frontier-level coding at a fraction of closed-model prices, and works out of the box with Claude Code, Codex and every coding tool OmniRoute serves.
What Kimi's support powers: Kimi's API credits power OmniRoute's AI-validated release pipeline — the merge validation powered by Kimi K3 stage that reviews every pull request before it ships — plus day-to-day feature development. First-class Kimi support ships on both rails: the direct Kimi API ( kimi-k3) and the Kimi Code coding plan (OAuth and API key). OmniRoute is also the first Brazilian open-source project in Kimi's support program. Get a Kimi API key →
|
Links tagged aff=omniroute are partner links. They fund the project at no extra cost to you.
💖 Sponsors
A heartfelt thank-you to the people who fund OmniRoute out of their own pocket. Every contribution keeps the project free, independent and moving.
Professor Igor Morais Vasconcelos · longtao · and others who prefer to stay private 💛
Want to support OmniRoute? Become a sponsor → Every dollar goes straight into keeping the project free and independent.
❤️ Support
OmniRoute is free and open source, built and maintained in the open. If it saves you time or money, consider supporting development:
- ⭐ Star the repo — it genuinely helps visibility
- 💖 GitHub Sponsors — fund ongoing maintenance and new providers
- 🐛 Report bugs and share feedback in Discussions
📄 License
MIT License - see LICENSE for details.
⬆ Back to top · Built with ❤️ for the open-source AI community.
OmniRoute v3.8.49 · Node ≥22.22.2 · MIT License · omniroute.online