pjandro/OmniRoute
1
0
Fork 0
mirror of https://github.com/diegosouzapw/OmniRoute.git synced 2026-08-03 05:45:04 +03:00
Code Issues Packages Projects Releases Wiki Activity
Files
1dc4e0781d0e91aa12964b4e7f61609f45f63891
OmniRoute/docs/compression/COMPRESSION_ENGINES.md
Diego Rodrigues de Sa e Souza 5d89fa84e7 fix(compression): end-to-end audit — fixes across the whole compression flow (#4323) 
* fix(compression): SLM worker resolves deps+worker file without import.meta.url (B-SLM)

The Next.js standalone bundle (webpack) replaces createRequire(import.meta.url)
with a stub that always throws MODULE_NOT_FOUND, and freezes import.meta.url to the
build-machine path. So depsAvailable() was always false (the worker never spawned)
and resolveWorkerFile() anchored on a path absent at runtime — the SLM silently
fell back to the aggressive summarizer in production. Confirmed by inspecting
dist/.build/next/server/chunks/26410.js (stub module 215743 + frozen file:// path).

Replace both with filesystem probing from runtime anchors (process.cwd(),
process.argv[1]) that survive the bundle. Necessary complement to #4286 (deps
co-location) for the SLM to actually engage in prod; still fail-open without it.
VPS live validation deferred (Rule #18); local resolver regression tests added.

* fix(compression): ultra heuristic preserves code blocks / inline code / URLs (B-ULTRA-CODE)

ultra.ts called pruneByScore on raw text with no tombstoning, so the token pruner
dropped low-score code tokens (`b)`, `{`, `+`) inside fenced blocks while leaving the
fence markers intact — output that looked like valid code but was syntactically
destroyed. caveman + llmlingua both extract/restore preserved blocks first; ultra was
the only pruning engine that didn't.

Add pruneProseOnly(): extractPreservedBlocks tombstones fenced code, inline code,
URLs, CONST_CASE, versions; only the prose between placeholders is pruned; preserved
blocks are re-stitched verbatim.

* fix(compression): GCF round-trips values containing the inline-array pattern [..]: (B-GCF-QUOTE)

A value like `ERR[404]: Not Found` / `[Speaker 1]: Hello` nested one level deep was
emitted bare and re-parsed by the decoder as an inline-array header → it threw
`count_mismatch` (or silently decoded wrong), losing the whole block. headroomEngine
.apply() ships such blobs in prod, so this was a reachable lossless violation.

Two complementary fixes, both per SPEC §2.4:
- encode: needsQuote() now quotes strings matching `[`…`]``:` (spec compliance / other
  decoders).
- decode: the inline-array branch only fires when the bracket is in the KEY position
  (no `=` before it), so a quoted `note="ERR[404]: …"` value falls through to key=value.

* fix(compression): aggressive fidelity — keep text blocks, compress Anthropic tool_result, don't corrupt JSON (B-AGG-*)

Three fidelity fixes in the aggressive path (each TDD, aggressive-fidelity.test.ts):
- B-AGG-TEXTDROP: replaceTextContent dropped 2nd+ text blocks unconditionally; now a
  trailing block is dropped only when its text is already subsumed by newText, else kept.
- B-AGG-ANTHROPIC-TR: tool-result compression only fired for OpenAI role:tool messages;
  now Anthropic-shape tool_result content blocks (inside user messages) are compressed
  too, preserving tool_use_id + block structure.
- B-AGG-JSONTAG: the [COMPRESSED:aging:*] prefix corrupted JSON/code payloads; pure JSON
  is now kept verbatim+untagged (stays parseable), fenced blocks get the tag on a
  preceding line.

* fix(compression): accessibility collapse preserves [ref] anchors + fires on interleaved trees (B-MCPA11Y-*)

- B-MCPA11Y-ANCHORS: collapseRepeated silently dropped the omitted middle siblings'
  [ref=eNN] anchors (the agent could no longer click them); now every omitted ref is
  kept alongside the collapse notice. Wires the previously-dead preserveRefPattern.
  Invariant: extractRefs(input) ⊆ extractRefs(output).
- B-MCPA11Y-COLLAPSE: noise removal blanked lines (replace→""), and a blank line broke
  the sibling run so collapse never fired on realistic interleaved trees; noise lines
  are now deleted, and the sibling walk skips stray blanks.

* fix(compression): rtk intensity scales the line budget (B-RTK-INTENSITY)

The intensity knob only set smartTruncate's preserveHead/Tail (16↔24), which rarely
fired because the matched filter capped lines first — so minimal/standard/aggressive
produced byte-identical output on filter-matched tool output. effectiveMaxLines() now
scales the effective line budget (minimal 1.5x, standard 1x, aggressive 0.5x) at both
the per-filter and engine-level truncation sites. Both go through smartTruncate with
priorityPatterns, so error/failure lines survive at every intensity (tested).

* fix(compression): robust language detection + auto-detect honors the detected pack (B-LANG-*)

- B-LANG-DETECTOR: detector was first-match-wins on a single keyword, and some hints are
  English-ambiguous ("configuration" in fr, "error" in es) → English text misclassified.
  Now score-based (count native-keyword hits, highest wins), and the two English-ambiguous
  words are removed from the hint lists, so a lone shared word never misclassifies while
  sparse-keyword languages (id) still detect on a single native word.
- B-LANG-DORMANT: with autoDetectLanguage on but enabledPacks ["en"], detected non-English
  text fell back to the English pack, whose `articles` rule deletes foreign articles
  (pt-BR "a"/"o"). Auto-detect now uses the detected pack directly (it always has rules);
  enabledPacks still gates manual selection.

* fix(compression): mode selection enables its engine + align stacked allowlist (B-MODE-ENGINE-DECOUPLE, B-PIPELINE-DIVERGENCE)

- B-MODE-ENGINE-DECOUPLE: picking the standard/rtk MODE now runs caveman/rtk regardless of
  the per-engine enabled flag — the mode selection is the enable signal (the per-engine flag
  still gates stacked pipeline steps). Previously an operator who picked a mode but left the
  engine toggle off got silent 0% compression.
- B-PIPELINE-DIVERGENCE: the global stackedPipeline normalizer stripped
  session-dedup/ccr/headroom/llmlingua (engines the combo path accepts via KNOWN_ENGINE_IDS).
  The allowlist now matches, so the global setting can use all registered engines.

* docs(compression): correct SLM "stable" claim + document partial packs / stacked telemetry limits

- The llmlingua `stable:true` comment claimed the bundle walk-up + deps-gate were
  "confirmed against the live install" — that was wrong (webpack froze import.meta.url and
  stubbed createRequire, so the worker never spawned in prod). Corrected to reflect B-SLM.
- COMPRESSION_ENGINES.md: add a Known limitations section (SLM dep co-location requirement,
  partial de/fr/ja packs, no-op engines absent from engineBreakdown).

* fix(compression): cast normalized engine id to CompressionPipelineStep['engine'] (typecheck)
2026-06-19 21:51:12 -03:00

316 lines
16 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
---
title: "Compression Engines"
version: 3.8.2
lastUpdated: 2026-06-17
---
# Compression Engines
OmniRoute compression is built around engine contracts. A mode can run one engine directly
(`caveman` or `rtk`) or a deterministic stacked pipeline that executes multiple engines in order.
## Modes
| Mode | Engine path | Intended input |
| ------------ | ---------------------------------- | -------------------------------------------- |
| `off` | none | Exact prompt preservation |
| `lite` | Caveman lite helpers | Low-risk always-on cleanup |
| `standard` | Caveman | Natural-language prompt condensation |
| `aggressive` | Caveman + history/tool summarizers | Long chat sessions |
| `ultra` | Caveman + pruning helpers | Context-limit recovery |
| `rtk` | RTK | Terminal, shell, build, test, and git output |
| `stacked` | Pipeline, default `rtk -> caveman` | Mixed tool logs and prose, max savings |
## Engine Registry
The registry lives in `open-sse/services/compression/engines/registry.ts`. Engines expose a shared
contract:
- `id`: stable engine id such as `caveman` or `rtk`
- `apply(text, config)`: legacy execution path used by stacked pipelines
- `compress(input, config)`: primary execution path returning text + stats
- `getConfigSchema()`: returns the JSON-Schema-like shape of valid config
- `validateConfig(config)`: returns `{ valid, errors[] }`
Registration uses `registerCompressionEngine(engine)` (or `registerEngine` for advanced cases),
which calls `assertValidEngine()` and `validateConfig(defaultConfig)` before accepting.
Use `unregisterCompressionEngine(id)` to remove an engine at runtime.
`strategySelector.ts` registers the built-in engines before compression runs. This lets preview,
runtime compression, stacked mode, tests, and future engines use the same execution path.
### MCP description compression (related)
A separate registry compresses MCP tool description metadata at registry-level — see
`open-sse/mcp-server/descriptionCompressor.ts` and [MCP-SERVER.md](../frameworks/MCP-SERVER.md). It reuses
Caveman rules but operates on tool metadata, not request payloads.
## Caveman
Caveman mode focuses on semantic condensation of normal prose:
- preserves code blocks, URLs, JSON, paths, and structured data
- removes filler, hedging, repeated context, and verbose connective phrasing
- supports language-aware file rule packs in `open-sse/services/compression/rules/`
- remains available through the legacy `standard`, `aggressive`, and `ultra` modes
The dashboard surface is `Dashboard -> Context & Cache -> Caveman`.
Caveman upstream reports `~75%` fewer output tokens, `65%` average output savings in benchmarks
with a `22-87%` range, and a `~46%` input-compression tool. OmniRoute uses the Caveman input-side
number when documenting stacked prompt/context savings; Caveman output mode remains a separate
response-behavior feature.
## RTK
RTK mode focuses on command and tool output:
- detects output classes such as `git status`, `git branch`, `git diff`, Vitest/Jest/Pytest,
Cargo/Go tests, TypeScript/Vite/Webpack builds, ESLint, npm audit/installs, Docker logs,
shell `find`/`grep`, stack traces, and generic logs
- applies 49 JSON filters from `open-sse/services/compression/engines/rtk/filters/`
- supports the RTK-style declarative pipeline: ANSI stripping, replace, match-output short-circuit,
strip/keep lines, per-line truncation, head/tail/max-line truncation, and on-empty fallback
- supports trust-gated project filters in `.rtk/filters.json` and global filters in
`DATA_DIR/rtk/filters.json`
- strips ANSI sequences, progress noise, repeated lines, and unhelpful boilerplate
- preserves actionable failures, warnings, summaries, changed files, and tail context
- can optionally retain redacted raw output for recovery/debugging through authenticated management
routes
The dashboard surface is `Dashboard -> Context & Cache -> RTK`.
Operational details for custom filters, trust, verify, and raw-output recovery live in
[`RTK_COMPRESSION.md`](./RTK_COMPRESSION.md).
RTK upstream reports `60-90%` savings for command-output compression. Its README example shows a
30-minute Claude Code session going from `~118,000` tokens to `~23,900`, or `79.7%` saved.
## LLMLingua-2 (Semantic Pruning)
LLMLingua-2 mode performs **semantic token pruning** on prose using a small ONNX token
classifier, complementing the rule-based Caveman and RTK engines:
- compresses prose in non-system messages only; fenced code blocks and other preserved
constructs are never altered
- runs the `@atjsh/llmlingua-2` backend (ONNX via `@huggingface/transformers`) in a
worker thread, so model inference never blocks the request event loop
- is **stackable** (`stackPriority` 35): in a stacked pipeline it runs after the
structural engines (CCR, session-dedup, headroom, Caveman) but before `ultra`, since
semantic pruning is most effective on already-structurally-compressed text — e.g.
`rtk -> caveman -> llmlingua`
- **fail-opens on any error** (missing optional deps, worker spawn, model load, inference,
or timeout) → the original text is returned unchanged, never an error
Engine location: `open-sse/services/compression/engines/llmlingua/`. The dashboard surface
is `Dashboard -> Context & Cache -> LLMLingua`.
### Models
The default model is **TinyBERT** (`atjsh/llmlingua-2-js-tinybert-meetingbank`, ~57 MB,
fast). A higher-accuracy **BERT-base** model (`Arcoldd/llmlingua4j-bert-base-onnx`,
~710 MB) is available via the engine config `model` field. `@huggingface/transformers`
downloads the selected model lazily from the HuggingFace Hub into
`${DATA_DIR}/models/llmlingua` on the first call (`modelStore.ts`); a `modelPath` config
override points it at a local copy instead (offline / air-gapped installs).
### Optional dependencies & on-demand install
The LLMLingua runtime stack is **optional**. Three packages are declared as
`optionalDependencies` in `package.json` and kept **external** by the production build
(`scripts/build/prepublish.ts` does not bundle them):
| Package | Version (pin) | Notes |
| -------------------- | ------------- | ---------------------------------------------- |
| `@atjsh/llmlingua-2` | `2.0.3` | Entry package; declares the others as peers |
| `@tensorflow/tfjs` | `4.22.0` | Heaviest dep — dominates the ~800 MB footprint |
| `js-tiktoken` | `^1.0.20` | Tokenizer |
`@huggingface/transformers` is pinned at `3.5.2` as a **regular** dependency (shared with
the local embeddings path), so it always ships — only the three packages above are
prunable. A standard `npm install` (dev) installs them automatically.
**Why on-demand:** the npm-published package, the standalone bundle, and the Docker image
ship **without** these deps to stay slim. When they are absent, the worker's dependency
gate (a `@atjsh/llmlingua-2` resolve probe in `worker.ts`) fails and the engine
**fail-opens silently** — selecting LLMLingua becomes a no-op (text returned unchanged, no
error logged). To activate it in a pruned environment, install the optional stack:
```bash
# pin to the versions declared in package.json optionalDependencies
npm install @atjsh/llmlingua-2@2.0.3 @tensorflow/tfjs@4.22.0 js-tiktoken
```
Roughly **~800 MB** total: the TensorFlow.js + transformers runtimes dominate; the
TinyBERT model adds ~57 MB downloaded at first use (not via npm).
Per environment:
- **Dev / `npm install`** — installed automatically unless you passed `--omit=optional`
(or `--no-optional`). No action needed.
- **Global npm (`npm i -g omniroute`) / standalone** — run the install command above inside
the installed package directory, or reinstall without omitting optional deps.
- **Docker** — add the install command in a derived image layer; the published image
ships slim by design.
- **VPS (PM2)** — install into the app's `node_modules`, then restart the process so the
worker re-probes the gate.
**Verify it is active:** with LLMLingua selected, real prose actually shrinks (the engine
stops fail-opening), and the first request triggers the model download into
`${DATA_DIR}/models/llmlingua`. The gate intentionally probes only `@atjsh/llmlingua-2` —
the other peers are ESM-only and `require.resolve` throws on them even when present — so
the worker still fail-opens if any peer is genuinely missing at `import()` time.
## Stacked Pipelines
Stacked mode runs pipeline steps in order. The default is:
```txt
rtk -> caveman
```
Use this for coding-agent sessions where a prompt combines command output with human or assistant
prose. RTK reduces noisy tool logs first, then Caveman compresses remaining natural language.
Pipeline steps are configured with `stackedPipeline` in compression settings or through compression
combos.
When both engines reduce the same eligible payload, savings compound:
```txt
combined = 1 - (1 - RTK savings) * (1 - Caveman input savings)
average = 1 - (1 - 0.80) * (1 - 0.46) = 89.2%
range = 1 - (1 - 0.60..0.90) * (1 - 0.46) = 78.4-94.6%
```
## MCP Accessibility Tree Filter
The MCP accessibility-tree smart filter is a post-execution compression layer that runs on MCP
**tool results**, not on prompts or context. It targets the verbose accessibility-tree and browser
snapshot payloads returned by tools like Playwright, computer-use, and browser-automation MCP
servers.
### What it does
1. **Noise stripping** — removes empty generic/text entries (`- generic:`, `- text: ""`)
2. **Sibling collapse** — when ≥ `collapseThreshold` (default 30) consecutive lines are structural
repeats, collapses them into the first `collapseKeepHead` (default 10) lines + a count summary +
the last `collapseKeepTail` (default 5) lines
3. **Ref preservation** — `[ref=eXX]` anchors required by Playwright/computer-use are never touched
4. **Hard truncation** — if the text after collapse still exceeds `maxTextChars` (default 50,000),
truncates with a navigation hint so the agent can continue working
### Engine location
```txt
open-sse/services/compression/engines/mcpAccessibility/
index.ts ← smartFilterText() entry point
collapseRepeated.ts ← sibling-collapse algorithm
constants.ts ← DEFAULT_MCP_ACCESSIBILITY_CONFIG
```
### Configuration
Controlled by `compression.mcpAccessibility` in global settings (migration 056). Default config:
```json
{
"enabled": true,
"maxTextChars": 50000,
"collapseThreshold": 30,
"collapseKeepHead": 10,
"collapseKeepTail": 5,
"minLengthToProcess": 2000
}
```
The filter is only applied to tool-result payloads whose `type` is `"text"` and whose length
exceeds `minLengthToProcess`. It does not affect prompt compression or request payloads.
### Expected savings
60–80% on browser snapshot tool results, depending on page complexity. The collapse algorithm
is O(n) in line count and adds negligible latency.
### This filter vs the compression engines above
| Aspect | Caveman / RTK / Stacked | MCP accessibility filter |
| ----------- | ------------------------- | -------------------------------------- |
| Target | Request prompts / context | MCP tool results |
| Trigger | Compression mode setting | `compression.mcpAccessibility.enabled` |
| Scope | All SSE messages | Tool results only |
| Ref anchors | N/A | Preserved unconditionally |
---
## Compression Combos
Compression combos are named compression profiles that can be assigned to routing combos:
- `compression_combos`: stores mode, pipeline, RTK config, language config, and default marker
- `compression_combo_assignments`: maps a compression combo to a routing combo
- runtime integration resolves an assigned compression combo before generic combo overrides
- analytics include `compression_combo_id` and `engine`
Dashboard surface: `Dashboard -> Context & Cache -> Compression Combos`.
## API Surface
| Route | Purpose |
| -------------------------------------- | ---------------------------------------------------------------- |
| `/api/settings/compression` | Global compression settings (includes `mcpAccessibility` config) |
| `/api/compression/preview` | Preview any compression mode |
| `/api/compression/language-packs` | List available Caveman language packs |
| `/api/context/caveman/config` | Caveman settings alias |
| `/api/context/rtk/config` | RTK defaults and settings |
| `/api/context/rtk/filters` | RTK filter catalog |
| `/api/context/rtk/test` | RTK preview/test endpoint |
| `/api/context/rtk/raw-output/[id]` | Authenticated redacted raw-output recovery |
| `/api/context/combos` | Compression combo CRUD |
| `/api/context/combos/[id]/assignments` | Routing-combo assignment CRUD |
| `/api/context/analytics` | Compression analytics alias |
Management routes require management authentication or API-key policy checks.
## MCP Tools
Compression exposes five MCP tools:
| Tool | Scope | Purpose |
| ----------------------------------- | ------------------- | -------------------------------- |
| `omniroute_compression_status` | `read:compression` | Settings, analytics, cache stats |
| `omniroute_compression_configure` | `write:compression` | Update global settings |
| `omniroute_set_compression_engine` | `write:compression` | Set mode and optional pipeline |
| `omniroute_list_compression_combos` | `read:compression` | List compression combos |
| `omniroute_compression_combo_stats` | `read:compression` | Read combo/engine analytics |
## Known limitations
- **LLMLingua-2 (SLM) requires co-located optional deps.** The worker only runs in a
production build when `@atjsh/llmlingua-2` + peers are co-located into
`dist/node_modules` (see `scripts/build/colocateOptionals.mjs`, #4286). Without them the
engine fail-opens (returns the original text). Worker resolution no longer depends on
`import.meta.url` (it dies in the standalone bundle) — it anchors on the runtime
cwd / `argv[1]`.
- **Caveman language packs `de` / `fr` / `ja` are partial.** They ship `context` +
`filler` + `structural` rules but no `dedup` / `ultra` packs, so `ultra` intensity is
no stronger than `full` for those languages (they use only their own rules — there is no
silent fall-back to the English `dedup`/`ultra` rules, which would mangle foreign text).
`en` / `es` / `id` / `pt-BR` are complete. Contributions of `dedup.json` + `ultra.json`
for the partial packs are welcome.
- **Stacked telemetry only lists engines that compressed.** A stacked-pipeline step whose
engine ran but produced 0 % savings returns `stats:null` and so does not appear in
`engineBreakdown` — indistinguishable from a step that was skipped. Distinguishing
"ran, 0 %" from "skipped" would require a breakdown-model change and is deferred.
## Validation
The focused gates for this area are:
```bash
node --import tsx/esm --test tests/unit/compression/rtk-*.test.ts tests/unit/compression/pipeline-integration.test.ts tests/unit/compression/context-compression-api.test.ts
node --import tsx/esm --test tests/unit/compression/*.test.ts tests/golden-set/*.test.ts tests/integration/compression-pipeline.test.ts tests/unit/api/compression/compression-api.test.ts
node --import tsx/esm --test tests/unit/compression/mcpAccessibility*.test.ts
npm run typecheck:core
```
Reference in New Issue View Git Blame Copy Permalink