pjandro/OmniRoute
1
0
Fork 0
mirror of https://github.com/diegosouzapw/OmniRoute.git synced 2026-07-31 04:12:10 +03:00
Code Issues Packages Projects Releases Wiki Activity

docs(compression): document RTK+Caveman stacked savings ranges

Browse Source 
Refresh the compression docs and README to describe the default
stacked pipeline in terms of eligible-context savings instead of the
older generic token-saving range.

Add upstream RTK and Caveman benchmark references, explain the
multiplicative savings math behind the stacked default, and update
feature summaries plus package metadata to match the revised
positioning.
This commit is contained in:
diegosouzapw
2026-05-03 15:59:28 -03:00
parent d8baf4434d
commit 6424c82570
6 changed files with 113 additions and 28 deletions
Download Patch File Download Diff File Expand all files Collapse all files

58
README.md
View File

@@ -2,7 +2,7 @@
# 🚀 OmniRoute — The Free AI Gateway
### Never stop coding. Save 15-75% tokens with prompt compression + auto-fallback to **FREE & low-cost AI models**.
### Never stop coding. Save 15-95% eligible tokens with RTK+Caveman compression + auto-fallback to **FREE & low-cost AI models**.
_The most complete open-source AI proxy — **one endpoint**, **160+ providers**, **13 routing strategies**, zero downtime. Multi-platform: **Web**, **Desktop (Electron)**, **Mobile (PWA + Termux)**. Fully extensible via **MCP Server (37 tools)**, **A2A Protocol**, and **Memory/Skills** systems. Available in **40+ languages**._
@@ -20,7 +20,7 @@ _The most complete open-source AI proxy — **one endpoint**, **160+ providers**
<a href="https://trendshift.io/repositories/23589" target="_blank"><img src="https://trendshift.io/api/badge/repositories/23589" alt="diegosouzapw%2FOmniRoute | Trendshift" style="width: 250px; height: 55px;" width="250" height="55"/></a>
[🚀 Quick Start](#-quick-start) • [💡 Features](#-key-features) • [🗜️ Compression](#%EF%B8%8F-prompt-compression--save-15-75-tokens-automatically) • [💰 Pricing](#-pricing-at-a-glance) • [🎯 Use Cases](#-use-cases--ready-made-combo-playbooks) • [🌍 Proxy](#-bypass-geographic-blocks--use-ai-from-any-country) • [❓ FAQ](#-frequently-asked-questions) • [📖 Docs](#-documentation) • [💬 WhatsApp](https://chat.whatsapp.com/JI7cDQ1GyaiDHhVBpLxf8b?mode=gi_t)
[🚀 Quick Start](#-quick-start) • [💡 Features](#-key-features) • [🗜️ Compression](#%EF%B8%8F-prompt-compression--save-15-95-eligible-tokens-automatically) • [💰 Pricing](#-pricing-at-a-glance) • [🎯 Use Cases](#-use-cases--ready-made-combo-playbooks) • [🌍 Proxy](#-bypass-geographic-blocks--use-ai-from-any-country) • [❓ FAQ](#-frequently-asked-questions) • [📖 Docs](#-documentation) • [💬 WhatsApp](https://chat.whatsapp.com/JI7cDQ1GyaiDHhVBpLxf8b?mode=gi_t)
</div>
@@ -219,7 +219,7 @@ _Connect any AI-powered IDE or CLI tool through OmniRoute — free API gateway f
**OmniRoute solves all of this:**
✅ **Prompt Compression** — auto-compress prompts & tool outputs, save 15-75% tokens per request
✅ **Prompt Compression** — auto-compress prompts & tool outputs, save 15-95% eligible tokens per request with RTK+Caveman stacked mode
✅ **Maximize subscriptions** — track quota, use every bit before reset
✅ **Auto fallback** — Subscription → API Key → Cheap → Free, zero downtime
✅ **Multi-account** — round-robin between accounts per provider
@@ -410,7 +410,7 @@ Alibaba · Amazon Q · AssemblyAI · Baidu Qianfan · Baseten · Black Forest La
↓
┌──────────────────────────────────────────────────┐
│ OmniRoute (Smart Router) │
│ • 🗜️ Prompt Compression (save 15-75% tokens) │
│ • 🗜️ Prompt Compression (save 15-95% eligible) │
│ • Format translation (OpenAI ↔ Claude ↔ Gemini) │
│ • Quota tracking + Embeddings + Images │
│ • Auto token refresh + Rate limit management │
@@ -424,14 +424,14 @@ Alibaba · Amazon Q · AssemblyAI · Baidu Qianfan · Baseten · Black Forest La
│ ↓ budget limit
└─→ [Tier 4: FREE] Qoder, Qwen, Kiro (unlimited)
Result: Never stop coding, minimal cost + 15-75% token savings
Result: Never stop coding, minimal cost + 15-95% eligible token savings
```
---
## 🗜️ Prompt Compression — Save 15-75% Tokens Automatically
## 🗜️ Prompt Compression — Save 15-95% Eligible Tokens Automatically
> **Why use many token when few token do trick?** OmniRoute's built-in compression pipeline reduces token usage on every request — before it even reaches the provider. Inspired by [Caveman](https://github.com/JuliusBrussee/caveman) (⭐ 51K+).
> **Why use many token when few token do trick?** OmniRoute's built-in compression pipeline reduces token usage before requests reach the provider. It combines ideas from [RTK - Rust Token Killer](https://github.com/rtk-ai/rtk) and [Caveman](https://github.com/JuliusBrussee/caveman) (⭐ 51K+).
### How It Works
@@ -442,12 +442,12 @@ Every request passes through the compression pipeline **transparently** — no c
│ Client sends │────▶│ OmniRoute Compression │────▶│ Provider │
│ full prompt │ │ Pipeline (7 options) │ │ receives │
│ (10,000 tok) │ │ │ │ compressed │
│ │ │ 🪶 Lite ........... ~15% │ │ (2,500 tok) │
│ │ │ 🪶 Lite ........... ~15% │ │ (~1,080 tok)│
│ │ │ 🪨 Standard ....... ~30% │ │ │
│ │ │ ⚡ Aggressive ..... ~50% │ │ 💰 75% saved │
│ │ │ ⚡ Aggressive ..... ~50% │ │ 💰 up to 95%│
│ │ │ 🔥 Ultra .......... ~75% │ │ │
│ │ │ 🧰 RTK ............ 20-70% │ │ │
│ │ │ 🔗 Stacked ........ 30-80% │ │ │
│ │ │ 🧰 RTK ............ 60-90% │ │ │
│ │ │ 🔗 Stacked ........ 78-95% │ │ │
└──────────────────┘ └─────────────────────────────┘ └──────────────┘
```
@@ -460,8 +460,36 @@ Every request passes through the compression pipeline **transparently** — no c
| **🪨 Standard (Caveman)** | ~30% | 30+ regex rules: filler removal, context condensation, structural compression, multi-turn dedup | Daily coding with Claude/Codex |
| **⚡ Aggressive** | ~50% | All standard + progressive message aging + tool result summarization + LLM-based compression | Long sessions with many tool calls |
| **🔥 Ultra** | ~75% | All aggressive + heuristic token pruning + stopword removal + score-based filtering | Maximum savings when tokens are scarce |
| **🧰 RTK** | 20-70% | 49 command-aware filters, RTK-style JSON DSL, verify gate, trust-gated custom filters | Shell/test/build/git output in agents |
| **🔗 Stacked** | 30-80% | Ordered engine pipeline, usually RTK first then Caveman | Mixed prompts with tool logs + prose |
| **🧰 RTK** | 60-90% | 49 command-aware filters, RTK-style JSON DSL, verify gate, trust-gated custom filters | Shell/test/build/git output in agents |
| **🔗 Stacked** | 78-95% | RTK first, then Caveman input condensation; ~89% with upstream average math | Mixed prompts with tool logs + prose |
### RTK + Caveman Savings Math
These numbers are based on the upstream project READMEs under `_references/_outros`:
| Source | Upstream claim used by OmniRoute docs |
| ------- | ------------------------------------------------------------------------------------------------------------------- |
| Caveman | `~75%` fewer output tokens; benchmark average `65%` output savings, range `22-87%`; `~46%` input compression tool |
| RTK | `60-90%` command-output token savings; sample session `~118,000 -> ~23,900` tokens, which is `79.7%` saved (`~80%`) |
For the default stacked compression combo, OmniRoute runs:
```txt
RTK -> Caveman
```
When both engines can act on the same tool/context payload, the savings compound:
```txt
combined = 1 - (1 - RTK savings) * (1 - Caveman input savings)
average = 1 - (1 - 0.80) * (1 - 0.46) = 89.2%
range = 1 - (1 - 0.60..0.90) * (1 - 0.46) = 78.4-94.6%
```
Caveman output mode is separate from prompt compression. When enabled for responses, use Caveman's
own upstream output numbers: `65%` average, `~75%` headline, `22-87%` observed range. Total bill
savings depend on the prompt/output mix, but coding-agent sessions are often tool-context heavy, so
the `RTK -> Caveman` combo is the best default for maximum context savings.
### Before & After (Standard/Caveman Mode)
@@ -516,7 +544,7 @@ Auto-trigger: set `autoTriggerTokens` to automatically enable compression when a
Compression combos can also assign a named compression pipeline to routing combos, so a coding combo can use RTK + Caveman while a paid subscription combo stays on lite mode.
> 🪨 **Fun fact:** The standard/caveman mode is inspired by [Caveman](https://github.com/JuliusBrussee/caveman) — the viral project that proved "caveman speak" cuts 65% of tokens while keeping 100% technical accuracy. OmniRoute takes this further with a **7-option pipeline** that goes from gentle whitespace cleanup through RTK tool-output filters and stacked multi-engine compression.
> 🪨 **Fun fact:** The standard/caveman mode is inspired by [Caveman](https://github.com/JuliusBrussee/caveman) — the viral project that reports 65% average output-token savings while keeping technical accuracy. OmniRoute takes this further with a **7-option pipeline** and a default `RTK -> Caveman` combo that can reach ~89% average savings on eligible tool/context payloads.
📖 **Full compression documentation:** [`docs/COMPRESSION_GUIDE.md`](docs/COMPRESSION_GUIDE.md) • [`docs/RTK_COMPRESSION.md`](docs/RTK_COMPRESSION.md) • [`docs/COMPRESSION_ENGINES.md`](docs/COMPRESSION_ENGINES.md) • [`docs/COMPRESSION_RULES_FORMAT.md`](docs/COMPRESSION_RULES_FORMAT.md) • [`docs/COMPRESSION_LANGUAGE_PACKS.md`](docs/COMPRESSION_LANGUAGE_PACKS.md)
@@ -941,7 +969,7 @@ Then in `/dashboard/media` → **Transcription** tab: upload any audio or video
| ---------------------------------------------------------------------------------------------------- | -------------------------------- |
| 🧠 **Smart 4-Tier Fallback** — Subscription → API → Cheap → Free | Never stop coding, zero downtime |
| 🔄 **Format Translation** — OpenAI ↔ Claude ↔ Gemini ↔ Responses API | Works with ANY CLI tool |
| 🗜️ **Prompt Compression** — 7 options including Caveman, RTK, and stacked pipelines | Save 15-75% tokens automatically |
| 🗜️ **Prompt Compression** — 7 options including Caveman, RTK, and stacked pipelines | Save 15-95% eligible tokens |
| 🤖 **MCP Server** — 37 tools, 3 transports (stdio/SSE/HTTP), 10 scopes | IDE/agent tool integration |
| 🛡️ **Resilience Engine** — circuit breakers, cooldowns, TLS spoofing, anti-thundering herd | Auto-recovery from any failure |
| 🎵 **10 Multi-Modal APIs** — chat, embed, images, video, music, TTS, STT, moderation, rerank, search | One endpoint for everything |

18
docs/COMPRESSION_ENGINES.md
View File

@@ -13,7 +13,7 @@ OmniRoute compression is built around engine contracts. A mode can run one engin
| `aggressive` | Caveman + history/tool summarizers | Long chat sessions |
| `ultra` | Caveman + pruning helpers | Context-limit recovery |
| `rtk` | RTK | Terminal, shell, build, test, and git output |
| `stacked` | Pipeline, default `rtk -> caveman` | Mixed tool logs and prose |
| `stacked` | Pipeline, default `rtk -> caveman` | Mixed tool logs and prose, max savings |
## Engine Registry
@@ -39,6 +39,11 @@ Caveman mode focuses on semantic condensation of normal prose:
The dashboard surface is `Dashboard -> Context & Cache -> Caveman`.
Caveman upstream reports `~75%` fewer output tokens, `65%` average output savings in benchmarks
with a `22-87%` range, and a `~46%` input-compression tool. OmniRoute uses the Caveman input-side
number when documenting stacked prompt/context savings; Caveman output mode remains a separate
response-behavior feature.
## RTK
RTK mode focuses on command and tool output:
@@ -61,6 +66,9 @@ The dashboard surface is `Dashboard -> Context & Cache -> RTK`.
Operational details for custom filters, trust, verify, and raw-output recovery live in
[`RTK_COMPRESSION.md`](RTK_COMPRESSION.md).
RTK upstream reports `60-90%` savings for command-output compression. Its README example shows a
30-minute Claude Code session going from `~118,000` tokens to `~23,900`, or `79.7%` saved.
## Stacked Pipelines
Stacked mode runs pipeline steps in order. The default is:
@@ -75,6 +83,14 @@ prose. RTK reduces noisy tool logs first, then Caveman compresses remaining natu
Pipeline steps are configured with `stackedPipeline` in compression settings or through compression
combos.
When both engines reduce the same eligible payload, savings compound:
```txt
combined = 1 - (1 - RTK savings) * (1 - Caveman input savings)
average = 1 - (1 - 0.80) * (1 - 0.46) = 89.2%
range = 1 - (1 - 0.60..0.90) * (1 - 0.46) = 78.4-94.6%
```
## Compression Combos
Compression combos are named compression profiles that can be assigned to routing combos:

48
docs/COMPRESSION_GUIDE.md
View File

@@ -1,6 +1,6 @@
# 🗜️ Prompt Compression Guide — OmniRoute
> Save 15-75% on token costs automatically. For a quick overview, see the [README Compression section](../README.md#%EF%B8%8F-prompt-compression--save-15-75-tokens-automatically).
> Save 15-95% on eligible context automatically. For a quick overview, see the [README Compression section](../README.md#%EF%B8%8F-prompt-compression--save-15-95-eligible-tokens-automatically).
## Overview
@@ -19,8 +19,8 @@ Client Request
→ Standard: Caveman-speak filler removal (~30%)
→ Aggressive: History aging + summarization (~50%)
→ Ultra: Heuristic pruning + code-block thinning (~75%)
→ RTK: Command-aware terminal/tool-output filtering (20-70%)
→ Stacked: Ordered multi-engine pipeline, usually RTK then Caveman (30-80%)
→ RTK: Command-aware terminal/tool-output filtering (60-90% upstream range)
→ Stacked: Ordered multi-engine pipeline, usually RTK then Caveman (78-95% eligible range)
→ Compressed Request → Provider
```
@@ -79,7 +79,7 @@ Maximum compression for token-critical scenarios:
**Best for:** When you're hitting context limits repeatedly.
### RTK Mode (20-70% savings)
### RTK Mode (60-90% upstream range)
RTK mode is optimized for verbose tool outputs that appear in coding-agent sessions:
@@ -94,7 +94,7 @@ RTK mode is optimized for verbose tool outputs that appear in coding-agent sessi
**Best for:** Agent sessions with shell, build, test, git, grep, and file-output transcripts.
### Stacked Mode (30-80% savings)
### Stacked Mode (78-95% eligible range)
Stacked mode runs multiple compression engines in a deterministic order. The default pipeline is:
@@ -110,6 +110,36 @@ compression combos assigned to routing combos.
---
## Upstream Savings Math
OmniRoute documents compression savings from two sources: upstream project benchmarks and
OmniRoute's own engine composition.
| Source | Upstream README number used here |
| ------- | --------------------------------------------------------------------------------------------------------------------- |
| Caveman | `~75%` fewer output tokens, `65%` benchmark average output savings, `22-87%` range, and `~46%` input compression tool |
| RTK | `60-90%` command-output savings; sample session `~118,000 -> ~23,900` tokens, or `79.7%` saved (`~80%`) |
For overlapping tool/context payloads, the default OmniRoute combo stacks the engines:
```txt
RTK -> Caveman
```
The combined savings are multiplicative, not additive:
```txt
combined = 1 - (1 - RTK savings) * (1 - Caveman input savings)
average = 1 - (1 - 0.80) * (1 - 0.46) = 89.2%
range = 1 - (1 - 0.60..0.90) * (1 - 0.46) = 78.4-94.6%
```
That `78-95%` number applies when both RTK and Caveman can reduce the same input/context payload.
Caveman response output mode is separate: when enabled, use Caveman's own output savings (`65%`
average, `~75%` headline, `22-87%` range). Total billing savings depend on your prompt/output mix.
---
## Token Savings Visualization
```
@@ -118,8 +148,8 @@ With Lite: 40K tokens sent (15% saved — safe, always-on)
With Standard: 33K tokens sent (30% saved — caveman-speak rules)
With Aggressive: 24K tokens sent (50% saved — aging + summarization)
With Ultra: 12K tokens sent (75% saved — heuristic pruning)
With RTK: varies by output (20-70% saved — command-aware filters)
With Stacked: varies by pipeline (30-80% saved — multi-engine execution)
With RTK: 19K-5K tokens sent (60-90% saved on command/tool output)
With Stacked: 10K-2.5K tokens sent (78-95% eligible RTK+Caveman range)
```
---
@@ -229,9 +259,9 @@ Every compressed request includes stats in the server logs:
## Acknowledgments
Standard mode compression rules are inspired by **[Caveman](https://github.com/JuliusBrussee/caveman)** by **[JuliusBrussee](https://github.com/JuliusBrussee)** (⭐ 51K+) — the viral "why use many token when few token do trick" project.
Standard mode compression rules are inspired by **[Caveman](https://github.com/JuliusBrussee/caveman)** by **[JuliusBrussee](https://github.com/JuliusBrussee)** (⭐ 51K+) — the viral "why use many token when few token do trick" project. Caveman reports `~75%` fewer output tokens, `65%` benchmark average output savings, a `22-87%` output range, and a `~46%` input-compression tool.
RTK mode is inspired by **[RTK - Rust Token Killer](https://github.com/rtk-ai/rtk)** by **[RTK AI](https://github.com/rtk-ai)** — the high-performance command-output compression project for terminal, build, test, git, and tool-output filtering.
RTK mode is inspired by **[RTK - Rust Token Killer](https://github.com/rtk-ai/rtk)** by **[RTK AI](https://github.com/rtk-ai)** — the high-performance command-output compression project for terminal, build, test, git, and tool-output filtering. RTK reports `60-90%` savings, with its README sample session showing `~80%` saved.
---

2
docs/FEATURES.md
View File

@@ -119,7 +119,7 @@ Context & Cache now exposes dedicated pages for Caveman, RTK, and Compression Co
- **Caveman** — language-aware rule packs, preview, output-mode controls, and analytics
- **RTK** — command-aware compression for shell, git, test, build, package, Docker, infra, JSON, and stack-trace output
- **Compression Combos** — named pipelines such as `rtk -> caveman` assigned to routing combos
- **Compression Combos** — named pipelines such as `rtk -> caveman` assigned to routing combos; the default stacked math reaches `~89%` average and `78-95%` eligible-context savings when both engines apply
- **Raw-output recovery** — optional redacted RTK raw-output pointers for debugging compressed failures
See [Compression Guide](COMPRESSION_GUIDE.md), [RTK Compression](RTK_COMPRESSION.md), and

13
docs/RTK_COMPRESSION.md
View File

@@ -12,9 +12,20 @@ rtk -> caveman
That order compresses noisy machine output first, then lets Caveman condense remaining prose.
Upstream RTK reports `60-90%` command-output savings. Its README sample session goes from
`~118,000` standard tokens to `~23,900` RTK tokens, which is `79.7%` saved (`~80%`). OmniRoute uses
that upstream average for the stacked savings calculation with Caveman input compression:
```txt
RTK average: 80% saved
Caveman input: 46% saved
Stacked: 1 - (1 - 0.80) * (1 - 0.46) = 89.2% saved
Range: 1 - (1 - 0.60..0.90) * (1 - 0.46) = 78.4-94.6%
```
## What It Compresses
The built-in catalog currently ships 39 filters across these categories:
The built-in catalog currently ships 49 filters across these categories:
| Category | Examples |
| --------- | ------------------------------------------------------------- |

2
package.json
View File

@@ -1,7 +1,7 @@
{
"name": "omniroute",
"version": "3.7.9",
"description": "Smart AI Router with auto fallback — route to FREE & cheap models, zero downtime. Works with Cursor, Cline, Claude Desktop, Codex, and any OpenAI-compatible tool.",
"description": "Unified AI router with 160+ providers, RTK+Caveman compression, auto fallback, MCP/A2A, desktop, PWA, and OpenAI-compatible APIs.",
"type": "module",
"bin": {
"omniroute": "bin/omniroute.mjs",