pjandro/OmniRoute
1
0
Fork 0
mirror of https://github.com/diegosouzapw/OmniRoute.git synced 2026-08-05 06:42:12 +03:00
Code Issues Packages Projects Releases Wiki Activity

docs(compression): document RTK+Caveman stacked savings ranges

Browse Source 
Refresh the compression docs and README to describe the default
stacked pipeline in terms of eligible-context savings instead of the
older generic token-saving range.

Add upstream RTK and Caveman benchmark references, explain the
multiplicative savings math behind the stacked default, and update
feature summaries plus package metadata to match the revised
positioning.
This commit is contained in:
diegosouzapw
2026-05-03 15:59:28 -03:00
parent d8baf4434d
commit 6424c82570
6 changed files with 113 additions and 28 deletions
Download Patch File Download Diff File Expand all files Collapse all files

18
docs/COMPRESSION_ENGINES.md
View File

@@ -13,7 +13,7 @@ OmniRoute compression is built around engine contracts. A mode can run one engin
| `aggressive` | Caveman + history/tool summarizers | Long chat sessions |
| `ultra` | Caveman + pruning helpers | Context-limit recovery |
| `rtk` | RTK | Terminal, shell, build, test, and git output |
| `stacked` | Pipeline, default `rtk -> caveman` | Mixed tool logs and prose |
| `stacked` | Pipeline, default `rtk -> caveman` | Mixed tool logs and prose, max savings |
## Engine Registry
@@ -39,6 +39,11 @@ Caveman mode focuses on semantic condensation of normal prose:
The dashboard surface is `Dashboard -> Context & Cache -> Caveman`.
Caveman upstream reports `~75%` fewer output tokens, `65%` average output savings in benchmarks
with a `22-87%` range, and a `~46%` input-compression tool. OmniRoute uses the Caveman input-side
number when documenting stacked prompt/context savings; Caveman output mode remains a separate
response-behavior feature.
## RTK
RTK mode focuses on command and tool output:
@@ -61,6 +66,9 @@ The dashboard surface is `Dashboard -> Context & Cache -> RTK`.
Operational details for custom filters, trust, verify, and raw-output recovery live in
[`RTK_COMPRESSION.md`](RTK_COMPRESSION.md).
RTK upstream reports `60-90%` savings for command-output compression. Its README example shows a
30-minute Claude Code session going from `~118,000` tokens to `~23,900`, or `79.7%` saved.
## Stacked Pipelines
Stacked mode runs pipeline steps in order. The default is:
@@ -75,6 +83,14 @@ prose. RTK reduces noisy tool logs first, then Caveman compresses remaining natu
Pipeline steps are configured with `stackedPipeline` in compression settings or through compression
combos.
When both engines reduce the same eligible payload, savings compound:
```txt
combined = 1 - (1 - RTK savings) * (1 - Caveman input savings)
average = 1 - (1 - 0.80) * (1 - 0.46) = 89.2%
range = 1 - (1 - 0.60..0.90) * (1 - 0.46) = 78.4-94.6%
```
## Compression Combos
Compression combos are named compression profiles that can be assigned to routing combos:

48
docs/COMPRESSION_GUIDE.md
View File

@@ -1,6 +1,6 @@
# 🗜️ Prompt Compression Guide — OmniRoute
> Save 15-75% on token costs automatically. For a quick overview, see the [README Compression section](../README.md#%EF%B8%8F-prompt-compression--save-15-75-tokens-automatically).
> Save 15-95% on eligible context automatically. For a quick overview, see the [README Compression section](../README.md#%EF%B8%8F-prompt-compression--save-15-95-eligible-tokens-automatically).
## Overview
@@ -19,8 +19,8 @@ Client Request
→ Standard: Caveman-speak filler removal (~30%)
→ Aggressive: History aging + summarization (~50%)
→ Ultra: Heuristic pruning + code-block thinning (~75%)
→ RTK: Command-aware terminal/tool-output filtering (20-70%)
→ Stacked: Ordered multi-engine pipeline, usually RTK then Caveman (30-80%)
→ RTK: Command-aware terminal/tool-output filtering (60-90% upstream range)
→ Stacked: Ordered multi-engine pipeline, usually RTK then Caveman (78-95% eligible range)
→ Compressed Request → Provider
```
@@ -79,7 +79,7 @@ Maximum compression for token-critical scenarios:
**Best for:** When you're hitting context limits repeatedly.
### RTK Mode (20-70% savings)
### RTK Mode (60-90% upstream range)
RTK mode is optimized for verbose tool outputs that appear in coding-agent sessions:
@@ -94,7 +94,7 @@ RTK mode is optimized for verbose tool outputs that appear in coding-agent sessi
**Best for:** Agent sessions with shell, build, test, git, grep, and file-output transcripts.
### Stacked Mode (30-80% savings)
### Stacked Mode (78-95% eligible range)
Stacked mode runs multiple compression engines in a deterministic order. The default pipeline is:
@@ -110,6 +110,36 @@ compression combos assigned to routing combos.
---
## Upstream Savings Math
OmniRoute documents compression savings from two sources: upstream project benchmarks and
OmniRoute's own engine composition.
| Source | Upstream README number used here |
| ------- | --------------------------------------------------------------------------------------------------------------------- |
| Caveman | `~75%` fewer output tokens, `65%` benchmark average output savings, `22-87%` range, and `~46%` input compression tool |
| RTK | `60-90%` command-output savings; sample session `~118,000 -> ~23,900` tokens, or `79.7%` saved (`~80%`) |
For overlapping tool/context payloads, the default OmniRoute combo stacks the engines:
```txt
RTK -> Caveman
```
The combined savings are multiplicative, not additive:
```txt
combined = 1 - (1 - RTK savings) * (1 - Caveman input savings)
average = 1 - (1 - 0.80) * (1 - 0.46) = 89.2%
range = 1 - (1 - 0.60..0.90) * (1 - 0.46) = 78.4-94.6%
```
That `78-95%` number applies when both RTK and Caveman can reduce the same input/context payload.
Caveman response output mode is separate: when enabled, use Caveman's own output savings (`65%`
average, `~75%` headline, `22-87%` range). Total billing savings depend on your prompt/output mix.
---
## Token Savings Visualization
```
@@ -118,8 +148,8 @@ With Lite: 40K tokens sent (15% saved — safe, always-on)
With Standard: 33K tokens sent (30% saved — caveman-speak rules)
With Aggressive: 24K tokens sent (50% saved — aging + summarization)
With Ultra: 12K tokens sent (75% saved — heuristic pruning)
With RTK: varies by output (20-70% saved — command-aware filters)
With Stacked: varies by pipeline (30-80% saved — multi-engine execution)
With RTK: 19K-5K tokens sent (60-90% saved on command/tool output)
With Stacked: 10K-2.5K tokens sent (78-95% eligible RTK+Caveman range)
```
---
@@ -229,9 +259,9 @@ Every compressed request includes stats in the server logs:
## Acknowledgments
Standard mode compression rules are inspired by **[Caveman](https://github.com/JuliusBrussee/caveman)** by **[JuliusBrussee](https://github.com/JuliusBrussee)** (⭐ 51K+) — the viral "why use many token when few token do trick" project.
Standard mode compression rules are inspired by **[Caveman](https://github.com/JuliusBrussee/caveman)** by **[JuliusBrussee](https://github.com/JuliusBrussee)** (⭐ 51K+) — the viral "why use many token when few token do trick" project. Caveman reports `~75%` fewer output tokens, `65%` benchmark average output savings, a `22-87%` output range, and a `~46%` input-compression tool.
RTK mode is inspired by **[RTK - Rust Token Killer](https://github.com/rtk-ai/rtk)** by **[RTK AI](https://github.com/rtk-ai)** — the high-performance command-output compression project for terminal, build, test, git, and tool-output filtering.
RTK mode is inspired by **[RTK - Rust Token Killer](https://github.com/rtk-ai/rtk)** by **[RTK AI](https://github.com/rtk-ai)** — the high-performance command-output compression project for terminal, build, test, git, and tool-output filtering. RTK reports `60-90%` savings, with its README sample session showing `~80%` saved.
---

2
docs/FEATURES.md
View File

@@ -119,7 +119,7 @@ Context & Cache now exposes dedicated pages for Caveman, RTK, and Compression Co
- **Caveman** — language-aware rule packs, preview, output-mode controls, and analytics
- **RTK** — command-aware compression for shell, git, test, build, package, Docker, infra, JSON, and stack-trace output
- **Compression Combos** — named pipelines such as `rtk -> caveman` assigned to routing combos
- **Compression Combos** — named pipelines such as `rtk -> caveman` assigned to routing combos; the default stacked math reaches `~89%` average and `78-95%` eligible-context savings when both engines apply
- **Raw-output recovery** — optional redacted RTK raw-output pointers for debugging compressed failures
See [Compression Guide](COMPRESSION_GUIDE.md), [RTK Compression](RTK_COMPRESSION.md), and

13
docs/RTK_COMPRESSION.md
View File

@@ -12,9 +12,20 @@ rtk -> caveman
That order compresses noisy machine output first, then lets Caveman condense remaining prose.
Upstream RTK reports `60-90%` command-output savings. Its README sample session goes from
`~118,000` standard tokens to `~23,900` RTK tokens, which is `79.7%` saved (`~80%`). OmniRoute uses
that upstream average for the stacked savings calculation with Caveman input compression:
```txt
RTK average: 80% saved
Caveman input: 46% saved
Stacked: 1 - (1 - 0.80) * (1 - 0.46) = 89.2% saved
Range: 1 - (1 - 0.60..0.90) * (1 - 0.46) = 78.4-94.6%
```
## What It Compresses
The built-in catalog currently ships 39 filters across these categories:
The built-in catalog currently ships 49 filters across these categories:
| Category | Examples |
| --------- | ------------------------------------------------------------- |