pjandro/OmniRoute
1
0
Fork 0
mirror of https://github.com/diegosouzapw/OmniRoute.git synced 2026-08-03 05:45:04 +03:00
Code Issues Packages Projects Releases Wiki Activity
Files
223e67a4fbb20ce153dc367b6cd6fcba02c32d71
OmniRoute/docs/compression/COMPRESSION_ENGINES.md
Diego Rodrigues de Sa e Souza 5ac4637816 docs: reorganize docs/ + accuracy audit + drop Node 20 (rebased onto current release tip) (#5262) 
* docs: move superpowers/research artifacts to isolated _tasks repo + docs tree cleanup

- Move docs/superpowers/{plans,specs} and docs/research/* into the gitignored,
  separately-versioned _tasks/ repo; untrack the two tracked research design docs.
- Add CLAUDE.md "Planning & Research Artifacts" section overriding the superpowers
  default save paths (docs/... -> _tasks/...); align REPOSITORY_MAP and
  DOCUMENTATION_OVERHAUL_PLAN with the new convention.
- Drop 4 now-obsolete /api/discovery/* entries from check-docs-symbols allowlist
  (stale-enforcement) and refresh code/spec path comments to _tasks/...
- Sweeps in concurrent docs-tree restructuring (root-level provider/guide docs,
  compression spec cleanup, .mcp.json.example removal).

* docs: reorganize docs/ tree + fix stale facts across ~26 docs

Phase A — reorganization:
- Move 7 orphan root docs into subfolders (providers/ created; TIERS+USAGE_QUOTA→guides/;
  plugins+PLUGIN_SDK→frameworks/); delete 8 obsolete/redundant docs (SUBMIT_PR superseded
  by CONTRIBUTING; DOCUMENTATION_OVERHAUL_PLAN; INCIDENT_RESPONSE/PERF_BUDGETS/THREAT_MODEL;
  3 ops snapshots). Rebuild README index (was missing ~40 files) + per-folder meta.json nav.
- Clean 14 dangling doc-path references in bin/ ops scripts, scripts/, workflow, tests;
  fix the dockerignore-docs-coverage required-docs path (PROVIDERS→providers/CLAUDE_WEB).

Phase B — content accuracy (verified against code, not the audit summary):
- Functional: ENVIRONMENT flag defaults (INPUT_SANITIZER/MCP_ENFORCE_SCOPES=true,
  COMPRESS_DESCRIPTIONS=false, dynamic heap); MCP-SERVER notion tool names (omniroute_*→
  notion_*) + counts 87→94; coverage gate 75/70→60/60/60/60 (RELEASE_CHECKLIST, COVERAGE_PLAN,
  ERROR_SANITIZATION, CONTRIBUTING); pre-push hook description; regenerate PROVIDER_REFERENCE (237).
- Count drift: providers 237, executors 70, migrations 106, db modules 94, oauth 19,
  strategies 17, MCP 94, flags 38, TS 6.0, open-sse ~900/services 294 across architecture/
  frameworks/ops docs; AUTO-COMBO 9→12 factors w/ correct DEFAULT_WEIGHTS; REASONING +2
  patterns; STEALTH UA defaults; AGENT_PROTOCOLS +cursor-cloud/list-capabilities;
  LANGUAGE_PACKS +id pack.
- Kept Node 20 (runtime guard accepts 20.20.2+; only engines is stricter) and MCP scopes=13
  (mcpScopes.ts) — both were correct in the docs; corrected only the attribution.

* docs: finish content refresh — compression engines, CLI_TOKEN merge, metadata sweep

- Compression: document the additional built-in engines (CCR, headroom, ionizer,
  session-dedup) in COMPRESSION_ENGINES; clarify LLMLingua-2 is the ultra-mode SLM
  backend + cross-ref the extra engines in EXTENDING_COMPRESSION; add the id
  (Indonesian) language pack to LANGUAGE_PACKS.
- AUTO-COMBO: replace the orphan 'How tiers fit' weight table (stale weights) with a
  pointer to the canonical 12-factor DEFAULT_WEIGHTS table.
- Security: merge CLI_TOKEN_AUTH.md (legacy 32-char SHA-256 format) into CLI_TOKEN.md
  as a 'Legacy format — still accepted' section (server accepts both HMAC + legacy),
  delete CLI_TOKEN_AUTH.md, drop it from the index + security nav.
- Metadata: bump stale frontmatter (version/lastUpdated) to 3.8.40/2026-06-28 across the
  doc set audited this pass, and normalize the in-body 'Last updated' header lines to match.

* fix(runtime): drop Node 20 from supported range + align all docs/diagrams/counts

- Node minimum is now 22 (aligned with package.json engines). SUPPORTED_NODE_RANGE in
  src/shared/utils/nodeRuntimeSupport.ts (and the bin/ mirror) drops the 20.x line →
  '>=22.22.2 <23 || >=24.0.0 <27'; getNodeRuntimeSupport now rejects Node 20 as
  unsupported-major. Test updated (TDD): node-runtime-support.test.ts asserts Node 20
  rejected. Docs aligned (TROUBLESHOOTING ×2, TERMUX, RELEASE_CHECKLIST, CODEBASE,
  CLI-TOOLS, README, llm.txt + 42 i18n llm.txt mirrors, skills/cli-serve).
- Diagrams regenerated: mcp-tools-87 -> mcp-tools-94 (34 base + pool 6 = 94) and
  auto-combo-9factor -> auto-combo-12factor (correct DEFAULT_WEIGHTS); SVGs re-rendered
  via mermaid-cli; doc refs + diagrams/README updated; fixed a pre-existing broken
  resilience-3layers image path.
- CLAUDE.md + AGENTS.md aligned to real counts (237 providers, 94 MCP tools / 34 base,
  106 migrations, 94 db modules, 12-factor auto-combo, 17 strategies); README provider
  count 231 -> 237; executor count corrected to 68 (provider executors, excl base/index)
  and OAuth to 18 across architecture docs. check:docs-all now passes (0 strict drift,
  0 broken links); removed dead .mcp.json.example doc link.

* fix(services): update installer Node hint to >=22.22.2 (aligned with dropped Node 20)

* docs: realign counts to current release tip after rebase

The release tip advanced while this work was in flight (Gemini CLI provider/executor
removed by #5246, plus other PRs). Re-counted against the current code and updated:
providers 237->236, executors 68->67, OAuth modules 18->17, open-sse services 294->298;
regenerated PROVIDER_REFERENCE.md (236). check:docs-all passes (0 strict drift).

* docs(changelog) + i18n: record Node 20 drop + fix nodeIncompatibleHint

- CHANGELOG: add [3.8.40] entries for the Node 20.x removal (runtime) and the docs
  reorganization/accuracy audit.
- i18n: nodeIncompatibleHint across all 42 locales no longer lists Node 20.x as
  supported (ASCII + CJK full-width variants), aligned with the dropped Node 20.

* fix(docs): repair CI breakages from the doc moves

- test: cli-plugin-system asserted docs/dev/plugins.md exists; the file moved to
  docs/frameworks/PLUGINS.md — point the test at the new path (Unit fast-path 2/2 fix).
- frontmatter: PLUGINS.md and PLUGIN_SDK.md moved into the fumadocs-indexed
  docs/frameworks/ which requires a 'title' frontmatter; the missing frontmatter
  failed the Next.js MDX build (dast-smoke 'invalid frontmatter'). Added frontmatter
  to both, plus the providers/ docs (consistency; that folder is not indexed).
2026-06-28 22:07:18 -03:00

331 lines
18 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
---
title: "Compression Engines"
version: 3.8.40
lastUpdated: 2026-06-28
---
# Compression Engines
OmniRoute compression is built around engine contracts. A mode can run one engine directly
(`caveman` or `rtk`) or a deterministic stacked pipeline that executes multiple engines in order.
## Modes
| Mode | Engine path | Intended input |
| ------------ | ---------------------------------- | -------------------------------------------- |
| `off` | none | Exact prompt preservation |
| `lite` | Caveman lite helpers | Low-risk always-on cleanup |
| `standard` | Caveman | Natural-language prompt condensation |
| `aggressive` | Caveman + history/tool summarizers | Long chat sessions |
| `ultra` | Caveman + pruning helpers | Context-limit recovery |
| `rtk` | RTK | Terminal, shell, build, test, and git output |
| `stacked` | Pipeline, default `rtk -> caveman` | Mixed tool logs and prose, max savings |
## Engine Registry
The registry lives in `open-sse/services/compression/engines/registry.ts`. Engines expose a shared
contract:
- `id`: stable engine id such as `caveman` or `rtk`
- `apply(text, config)`: legacy execution path used by stacked pipelines
- `compress(input, config)`: primary execution path returning text + stats
- `getConfigSchema()`: returns the JSON-Schema-like shape of valid config
- `validateConfig(config)`: returns `{ valid, errors[] }`
Registration uses `registerCompressionEngine(engine)` (or `registerEngine` for advanced cases),
which calls `assertValidEngine()` and `validateConfig(defaultConfig)` before accepting.
Use `unregisterCompressionEngine(id)` to remove an engine at runtime.
`strategySelector.ts` registers the built-in engines before compression runs. This lets preview,
runtime compression, stacked mode, tests, and future engines use the same execution path.
### MCP description compression (related)
A separate registry compresses MCP tool description metadata at registry-level — see
`open-sse/mcp-server/descriptionCompressor.ts` and [MCP-SERVER.md](../frameworks/MCP-SERVER.md). It reuses
Caveman rules but operates on tool metadata, not request payloads.
### Additional built-in engines
Beyond Caveman, RTK, and LLMLingua-2, the registry ships several specialized lossless /
structural engines (used by stacked pipelines, the playground, and tests):
| Engine | Id | What it does |
| ------------- | --------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| CCR | `ccr` | Content-Compress-Retrieve (H4): replaces large contiguous text blocks with content-addressed references, so repeated/large blocks are sent once and referenced thereafter. |
| headroom | `headroom` | SmartCrusher (H3 + N5): lossless tabular compaction of homogeneous JSON-array payloads into a columnar `[N rows]` form. |
| ionizer | `ionizer` | Head/middle/tail row sampling for very large homogeneous blocks, storing the elided middle as a CCR content-addressed reference. |
| session-dedup | `session-dedup` | Content-addressed cross-turn deduplication (TokenMizer-inspired): elides text already seen in earlier turns of the same session. |
## Caveman
Caveman mode focuses on semantic condensation of normal prose:
- preserves code blocks, URLs, JSON, paths, and structured data
- removes filler, hedging, repeated context, and verbose connective phrasing
- supports language-aware file rule packs in `open-sse/services/compression/rules/`
- remains available through the legacy `standard`, `aggressive`, and `ultra` modes
The dashboard surface is `Dashboard -> Context & Cache -> Caveman`.
Caveman upstream reports `~75%` fewer output tokens, `65%` average output savings in benchmarks
with a `22-87%` range, and a `~46%` input-compression tool. OmniRoute uses the Caveman input-side
number when documenting stacked prompt/context savings; Caveman output mode remains a separate
response-behavior feature.
## RTK
RTK mode focuses on command and tool output:
- detects output classes such as `git status`, `git branch`, `git diff`, Vitest/Jest/Pytest,
Cargo/Go tests, TypeScript/Vite/Webpack builds, ESLint, npm audit/installs, Docker logs,
shell `find`/`grep`, stack traces, and generic logs
- applies 49 JSON filters from `open-sse/services/compression/engines/rtk/filters/`
- supports the RTK-style declarative pipeline: ANSI stripping, replace, match-output short-circuit,
strip/keep lines, per-line truncation, head/tail/max-line truncation, and on-empty fallback
- supports trust-gated project filters in `.rtk/filters.json` and global filters in
`DATA_DIR/rtk/filters.json`
- strips ANSI sequences, progress noise, repeated lines, and unhelpful boilerplate
- preserves actionable failures, warnings, summaries, changed files, and tail context
- can optionally retain redacted raw output for recovery/debugging through authenticated management
routes
The dashboard surface is `Dashboard -> Context & Cache -> RTK`.
Operational details for custom filters, trust, verify, and raw-output recovery live in
[`RTK_COMPRESSION.md`](./RTK_COMPRESSION.md).
RTK upstream reports `60-90%` savings for command-output compression. Its README example shows a
30-minute Claude Code session going from `~118,000` tokens to `~23,900`, or `79.7%` saved.
## LLMLingua-2 (Semantic Pruning)
LLMLingua-2 mode performs **semantic token pruning** on prose using a small ONNX token
classifier, complementing the rule-based Caveman and RTK engines:
- compresses prose in non-system messages only; fenced code blocks and other preserved
constructs are never altered
- runs the `@atjsh/llmlingua-2` backend (ONNX via `@huggingface/transformers`) in a
worker thread, so model inference never blocks the request event loop
- is **stackable** (`stackPriority` 35): in a stacked pipeline it runs after the
structural engines (CCR, session-dedup, headroom, Caveman) but before `ultra`, since
semantic pruning is most effective on already-structurally-compressed text — e.g.
`rtk -> caveman -> llmlingua`
- **fail-opens on any error** (missing optional deps, worker spawn, model load, inference,
or timeout) → the original text is returned unchanged, never an error
Engine location: `open-sse/services/compression/engines/llmlingua/`. The dashboard surface
is `Dashboard -> Context & Cache -> LLMLingua`.
### Models
The default model is **TinyBERT** (`atjsh/llmlingua-2-js-tinybert-meetingbank`, ~57 MB,
fast). A higher-accuracy **BERT-base** model (`Arcoldd/llmlingua4j-bert-base-onnx`,
~710 MB) is available via the engine config `model` field. `@huggingface/transformers`
downloads the selected model lazily from the HuggingFace Hub into
`${DATA_DIR}/models/llmlingua` on the first call (`modelStore.ts`); a `modelPath` config
override points it at a local copy instead (offline / air-gapped installs).
### Optional dependencies & on-demand install
The prunable LLMLingua runtime peer stack is **optional**. Three packages are declared as
`optionalDependencies` in `package.json` and kept **external** by the production build
(`scripts/build/prepublish.ts` does not bundle them):
| Package | Version (pin) | Notes |
| -------------------- | ------------- | ---------------------------------------------- |
| `@atjsh/llmlingua-2` | `2.0.3` | Entry package; declares the others as peers |
| `@tensorflow/tfjs` | `4.22.0` | Heaviest dep — dominates the ~800 MB footprint |
| `js-tiktoken` | `^1.0.20` | Tokenizer |
`@huggingface/transformers` is pinned at `3.5.2` as an **optional** dependency (shared with
the local embeddings path and also traced into the standalone bundle). Keeping it optional prevents
`onnxruntime-node` CUDA provider postinstall failures on CUDA 11 hosts from aborting the whole
OmniRoute install; when the optional stack is absent, LLMLingua still fail-opens. Only the three
packages above are prunable SLM peers. A standard `npm install` (dev) installs the optional stack
automatically unless optional dependencies are omitted.
**Why on-demand:** the npm-published package, the standalone bundle, and the Docker image
ship **without** these deps to stay slim. When they are absent, the worker's dependency
gate (a `@atjsh/llmlingua-2` resolve probe in `worker.ts`) fails and the engine
**fail-opens silently** — selecting LLMLingua becomes a no-op (text returned unchanged, no
error logged). To activate it in a pruned environment, install the optional stack:
```bash
# pin to the versions declared in package.json optionalDependencies
npm install @atjsh/llmlingua-2@2.0.3 @tensorflow/tfjs@4.22.0 js-tiktoken
```
Roughly **~800 MB** total: the TensorFlow.js + transformers runtimes dominate; the
TinyBERT model adds ~57 MB downloaded at first use (not via npm).
Per environment:
- **Dev / `npm install`** — installed automatically unless you passed `--omit=optional`
(or `--no-optional`). No action needed.
- **Global npm (`npm i -g omniroute`) / standalone** — run the install command above inside
the installed package directory, or reinstall without omitting optional deps.
- **Docker** — add the install command in a derived image layer; the published image
ships slim by design.
- **VPS (PM2)** — install into the app's `node_modules`, then restart the process so the
worker re-probes the gate.
**Verify it is active:** with LLMLingua selected, real prose actually shrinks (the engine
stops fail-opening), and the first request triggers the model download into
`${DATA_DIR}/models/llmlingua`. The gate intentionally probes only `@atjsh/llmlingua-2` —
the other peers are ESM-only and `require.resolve` throws on them even when present — so
the worker still fail-opens if any peer is genuinely missing at `import()` time.
## Stacked Pipelines
Stacked mode runs pipeline steps in order. The default is:
```txt
rtk -> caveman
```
Use this for coding-agent sessions where a prompt combines command output with human or assistant
prose. RTK reduces noisy tool logs first, then Caveman compresses remaining natural language.
Pipeline steps are configured with `stackedPipeline` in compression settings or through compression
combos.
When both engines reduce the same eligible payload, savings compound:
```txt
combined = 1 - (1 - RTK savings) * (1 - Caveman input savings)
average = 1 - (1 - 0.80) * (1 - 0.46) = 89.2%
range = 1 - (1 - 0.60..0.90) * (1 - 0.46) = 78.4-94.6%
```
## MCP Accessibility Tree Filter
The MCP accessibility-tree smart filter is a post-execution compression layer that runs on MCP
**tool results**, not on prompts or context. It targets the verbose accessibility-tree and browser
snapshot payloads returned by tools like Playwright, computer-use, and browser-automation MCP
servers.
### What it does
1. **Noise stripping** — removes empty generic/text entries (`- generic:`, `- text: ""`)
2. **Sibling collapse** — when ≥ `collapseThreshold` (default 30) consecutive lines are structural
repeats, collapses them into the first `collapseKeepHead` (default 10) lines + a count summary +
the last `collapseKeepTail` (default 5) lines
3. **Ref preservation** — `[ref=eXX]` anchors required by Playwright/computer-use are never touched
4. **Hard truncation** — if the text after collapse still exceeds `maxTextChars` (default 50,000),
truncates with a navigation hint so the agent can continue working
### Engine location
```txt
open-sse/services/compression/engines/mcpAccessibility/
index.ts ← smartFilterText() entry point
collapseRepeated.ts ← sibling-collapse algorithm
constants.ts ← DEFAULT_MCP_ACCESSIBILITY_CONFIG
```
### Configuration
Controlled by `compression.mcpAccessibility` in global settings (migration 056). Default config:
```json
{
"enabled": true,
"maxTextChars": 50000,
"collapseThreshold": 30,
"collapseKeepHead": 10,
"collapseKeepTail": 5,
"minLengthToProcess": 2000
}
```
The filter is only applied to tool-result payloads whose `type` is `"text"` and whose length
exceeds `minLengthToProcess`. It does not affect prompt compression or request payloads.
### Expected savings
60–80% on browser snapshot tool results, depending on page complexity. The collapse algorithm
is O(n) in line count and adds negligible latency.
### This filter vs the compression engines above
| Aspect | Caveman / RTK / Stacked | MCP accessibility filter |
| ----------- | ------------------------- | -------------------------------------- |
| Target | Request prompts / context | MCP tool results |
| Trigger | Compression mode setting | `compression.mcpAccessibility.enabled` |
| Scope | All SSE messages | Tool results only |
| Ref anchors | N/A | Preserved unconditionally |
---
## Compression Combos
Compression combos are named compression profiles that can be assigned to routing combos:
- `compression_combos`: stores mode, pipeline, RTK config, language config, and default marker
- `compression_combo_assignments`: maps a compression combo to a routing combo
- runtime integration resolves an assigned compression combo before generic combo overrides
- analytics include `compression_combo_id` and `engine`
Dashboard surface: `Dashboard -> Context & Cache -> Compression Combos`.
## API Surface
| Route | Purpose |
| -------------------------------------- | ---------------------------------------------------------------- |
| `/api/settings/compression` | Global compression settings (includes `mcpAccessibility` config) |
| `/api/compression/preview` | Preview any compression mode |
| `/api/compression/language-packs` | List available Caveman language packs |
| `/api/context/caveman/config` | Caveman settings alias |
| `/api/context/rtk/config` | RTK defaults and settings |
| `/api/context/rtk/filters` | RTK filter catalog |
| `/api/context/rtk/test` | RTK preview/test endpoint |
| `/api/context/rtk/raw-output/[id]` | Authenticated redacted raw-output recovery |
| `/api/context/combos` | Compression combo CRUD |
| `/api/context/combos/[id]/assignments` | Routing-combo assignment CRUD |
| `/api/context/analytics` | Compression analytics alias |
Management routes require management authentication or API-key policy checks.
## MCP Tools
Compression exposes five MCP tools:
| Tool | Scope | Purpose |
| ----------------------------------- | ------------------- | -------------------------------- |
| `omniroute_compression_status` | `read:compression` | Settings, analytics, cache stats |
| `omniroute_compression_configure` | `write:compression` | Update global settings |
| `omniroute_set_compression_engine` | `write:compression` | Set mode and optional pipeline |
| `omniroute_list_compression_combos` | `read:compression` | List compression combos |
| `omniroute_compression_combo_stats` | `read:compression` | Read combo/engine analytics |
## Known limitations
- **LLMLingua-2 (SLM) requires co-located optional deps.** The worker only runs in a
production build when `@atjsh/llmlingua-2` + peers are co-located into
`dist/node_modules` (see `scripts/build/colocateOptionals.mjs`, #4286). Without them the
engine fail-opens (returns the original text). Worker resolution no longer depends on
`import.meta.url` (it dies in the standalone bundle) — it anchors on the runtime
cwd / `argv[1]`.
- **Caveman language packs `de` / `fr` / `ja` are partial.** They ship `context` +
`filler` + `structural` rules but no `dedup` / `ultra` packs, so `ultra` intensity is
no stronger than `full` for those languages (they use only their own rules — there is no
silent fall-back to the English `dedup`/`ultra` rules, which would mangle foreign text).
`en` / `es` / `id` / `pt-BR` are complete. Contributions of `dedup.json` + `ultra.json`
for the partial packs are welcome.
- **Stacked telemetry only lists engines that compressed.** A stacked-pipeline step whose
engine ran but produced 0 % savings returns `stats:null` and so does not appear in
`engineBreakdown` — indistinguishable from a step that was skipped. Distinguishing
"ran, 0 %" from "skipped" would require a breakdown-model change and is deferred.
## Validation
The focused gates for this area are:
```bash
node --import tsx/esm --test tests/unit/compression/rtk-*.test.ts tests/unit/compression/pipeline-integration.test.ts tests/unit/compression/context-compression-api.test.ts
node --import tsx/esm --test tests/unit/compression/*.test.ts tests/golden-set/*.test.ts tests/integration/compression-pipeline.test.ts tests/unit/api/compression/compression-api.test.ts
node --import tsx/esm --test tests/unit/compression/mcpAccessibility*.test.ts
npm run typecheck:core
```
Reference in New Issue View Git Blame Copy Permalink