docs(readme): document prompt compression savings and routing flow

Highlight automatic prompt compression as a core capability with estimated token savings, supported compression modes, and a visual request pipeline. Update the architecture overview to reflect broader format support, rate limit handling, and the cost-saving impact alongside fallback routing.
2026-07-31 04:12:10 +03:00 · 2026-05-01 13:47:26 -03:00
parent cf703138f2
commit f8a23f41a6
1 changed files with 94 additions and 8 deletions
--- a/README.md
+++ b/README.md
@@ -2,7 +2,7 @@

 # 🚀 OmniRoute — The Free AI Gateway

-### Never stop coding. Smart routing to **FREE & low-cost AI models** with automatic fallback.
+### Never stop coding. Save 15-75% tokens with prompt compression + auto-fallback to **FREE & low-cost AI models**.

 _The most complete open-source AI proxy — **one endpoint**, **160+ providers**, **13 routing strategies**, zero downtime. Multi-platform: **Web**, **Desktop (Electron)**, **Mobile (PWA + Termux)**. Fully extensible via **MCP Server (29 tools)**, **A2A Protocol**, and **Memory/Skills** systems. Available in **40+ languages**._

@@ -245,12 +245,13 @@ This generates a `system-info.txt` with your Node.js version, OmniRoute version,
 └──────┬──────┘
       │ http://localhost:20128/v1
       ↓
-┌─────────────────────────────────────────┐
-│           OmniRoute (Smart Router)        │
-│  • Format translation (OpenAI ↔ Claude) │
-│  • Quota tracking + Embeddings + Images │
-│  • Auto token refresh                   │
-└──────┬──────────────────────────────────┘
+┌──────────────────────────────────────────────────┐
+│              OmniRoute (Smart Router)             │
+│  • 🗜️ Prompt Compression (save 15-75% tokens)    │
+│  • Format translation (OpenAI ↔ Claude ↔ Gemini) │
+│  • Quota tracking + Embeddings + Images          │
+│  • Auto token refresh + Rate limit management    │
+└──────┬───────────────────────────────────────────┘
       │
       ├─→ [Tier 1: SUBSCRIPTION] Claude Code, Codex, Gemini CLI
       │   ↓ quota exhausted
@@ -260,11 +261,94 @@ This generates a `system-info.txt` with your Node.js version, OmniRoute version,
       │   ↓ budget limit
       └─→ [Tier 4: FREE] Qoder, Qwen, Kiro (unlimited)

-Result: Never stop coding, minimal cost
+Result: Never stop coding, minimal cost + 15-75% token savings
 ```

 ---

+## 🗜️ Prompt Compression — Save 15-75% Tokens Automatically
+
+> **Why use many token when few token do trick?** OmniRoute's built-in compression pipeline reduces token usage on every request — before it even reaches the provider. Inspired by [Caveman](https://github.com/JuliusBrussee/caveman) (⭐ 51K+).
+
+### How It Works
+
+Every request passes through the compression pipeline **transparently** — no client changes needed:
+
+```
+┌──────────────────┐     ┌─────────────────────────────┐     ┌──────────────┐
+│   Client sends   │────▶│  OmniRoute Compression      │────▶│  Provider    │
+│   full prompt    │     │  Pipeline (5 modes)          │     │  receives    │
+│   (10,000 tok)   │     │                              │     │  compressed  │
+│                  │     │  🪶 Lite ........... ~15%     │     │  (2,500 tok) │
+│                  │     │  🪨 Standard ....... ~30%     │     │              │
+│                  │     │  ⚡ Aggressive ..... ~50%     │     │  💰 75% saved │
+│                  │     │  🔥 Ultra .......... ~75%     │     │              │
+└──────────────────┘     └─────────────────────────────┘     └──────────────┘
+```
+
+### 5 Compression Modes
+
+| Mode                      | Savings | Technique                                                                                       | Best For                               |
+| ------------------------- | ------- | ----------------------------------------------------------------------------------------------- | -------------------------------------- |
+| **Off**                   | 0%      | No compression                                                                                  | When you need exact prompts            |
+| **🪶 Lite**               | ~15%    | Whitespace collapse, dedup system prompts, image URL shortening                                 | Always-on safe default                 |
+| **🪨 Standard (Caveman)** | ~30%    | 30+ regex rules: filler removal, context condensation, structural compression, multi-turn dedup | Daily coding with Claude/Codex         |
+| **⚡ Aggressive**         | ~50%    | All standard + progressive message aging + tool result summarization + LLM-based compression    | Long sessions with many tool calls     |
+| **🔥 Ultra**              | ~75%    | All aggressive + heuristic token pruning + stopword removal + score-based filtering             | Maximum savings when tokens are scarce |
+
+### Before & After (Standard/Caveman Mode)
+
+**🗣️ Before compression (69 tokens):**
+
+> "The reason your React component is re-rendering is likely because you're creating a new object reference on each render cycle. When you pass an inline object as a prop, React's shallow comparison sees it as a different object every time, which triggers a re-render. I would recommend using useMemo to memoize the object."
+
+**🪨 After compression (19 tokens):**
+
+> "New object ref each render. Inline object prop = new ref = re-render. Wrap in useMemo."
+
+**Same answer. 72% less tokens. Zero accuracy loss.**
+
+### Architecture
+
+```
+Request Body
+  │
+  ├─ strategySelector.ts ─── Picks mode (config / combo override / auto-trigger)
+  │
+  ├─ lite.ts ─────────────── Whitespace, dedup, image URLs, redundant content
+  ├─ caveman.ts ──────────── 30+ regex rules via cavemanRules.ts
+  │   └─ preservation.ts ─── Protects code blocks, URLs, JSON from compression
+  ├─ aggressive.ts ───────── Summarizer + tool result compressor + progressive aging
+  │   ├─ summarizer.ts ───── Rule-based message summarization
+  │   ├─ toolResultCompressor.ts ── file/grep/shell/JSON/error compression
+  │   └─ progressiveAging.ts ──── Older messages → shorter summaries
+  └─ ultra.ts ────────────── Heuristic token scoring + pruning
+      └─ ultraHeuristic.ts ─ Stopword detection, score thresholds, force-preserve
+```
+
+### Configuration
+
+```
+Dashboard → Settings → Compression → Pick your mode
+```
+
+Or per-combo override:
+
+```json
+{
+  "comboOverrides": {
+    "my-coding-combo": "standard",
+    "my-cheap-combo": "ultra"
+  }
+}
+```
+
+Auto-trigger: set `autoTriggerTokens` to automatically enable compression when a request exceeds a token threshold.
+
+> 🪨 **Fun fact:** The standard/caveman mode is inspired by [Caveman](https://github.com/JuliusBrussee/caveman) — the viral project that proved "caveman speak" cuts 65% of tokens while keeping 100% technical accuracy. OmniRoute takes this further with a **5-mode pipeline** that goes from gentle whitespace cleanup all the way to aggressive heuristic pruning.
+
+---
+
 ## 🎯 What OmniRoute Solves — 30 Real Pain Points & Use Cases

 > **Every developer using AI tools faces these problems daily.** OmniRoute was built to solve them all — from cost overruns to regional blocks, from broken OAuth flows to protocol operations and enterprise observability.
@@ -2795,6 +2879,8 @@ Special thanks to **[9router](https://github.com/decolua/9router)** by **[decolu

 Special thanks to **[CLIProxyAPI](https://github.com/router-for-me/CLIProxyAPI)** — the original Go implementation that inspired this JavaScript port.

+Special thanks to **[Caveman](https://github.com/JuliusBrussee/caveman)** by **[JuliusBrussee](https://github.com/JuliusBrussee)** (⭐ 51K+) — the viral "why use many token when few token do trick" project whose caveman-speak compression philosophy inspired OmniRoute's standard compression mode and 30+ filler/condensation regex rules.
+
 ---

 ## 📄 License