From 133dd0026dc87eca5b08f9e92b798e5cacb20056 Mon Sep 17 00:00:00 2001 From: diegosouzapw Date: Fri, 15 May 2026 03:26:55 -0300 Subject: [PATCH] docs: fill documentation gaps for v3.8.0 features MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - COMPRESSION_ENGINES.md: add MCP accessibility-tree filter section with config reference, algorithm description, and comparison table - COMPRESSION_LANGUAGE_PACKS.md: document SHARED_BOUNDARIES clause (6 patterns × 6 languages × 3 intensities, preservePatterns defaults) - MCP-SERVER.md: add accessibility-tree filter note in Compression Tools - CONTRIBUTING.md: fix coverage gate (60%→75/70), add Hard Rules #15/#16 to PR checklist, add links to new security/ops docs --- CONTRIBUTING.md | 10 ++- docs/compression/COMPRESSION_ENGINES.md | 87 ++++++++++++++++--- .../compression/COMPRESSION_LANGUAGE_PACKS.md | 47 ++++++++++ docs/frameworks/MCP-SERVER.md | 18 ++++ 4 files changed, 146 insertions(+), 16 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index f2db935482..aa20f27f18 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -133,7 +133,7 @@ npm run test:protocols:e2e # Ecosystem compatibility tests npm run test:ecosystem -# Coverage (60% min statements/lines/functions/branches) +# Coverage gate: 75% statements/lines/functions, 70% branches npm run test:coverage npm run coverage:report @@ -145,7 +145,7 @@ npm run check Coverage notes: - `npm run test:coverage` measures source coverage for the main unit test suite, excludes `tests/**`, and includes `open-sse/**` -- Pull requests must keep the overall coverage gate at **60% or higher** for statements, lines, functions, and branches +- Pull requests must keep the coverage gate at **75%+** statements/lines/functions and **70%+** branches - If a PR changes production code in `src/`, `open-sse/`, `electron/`, or `bin/`, it must add or update automated tests in the same PR - `npm run coverage:report` prints the detailed file-by-file report from the latest coverage run - `npm run test:coverage:legacy` preserves the older metric for historical comparison @@ -157,7 +157,7 @@ Before opening or merging a PR: - Run `npm run test:unit` - Run `npm run test:coverage` -- Ensure the coverage gate stays at **60%+** for all metrics +- Ensure the coverage gate stays at **75%+** statements/lines/functions, **70%+** branches - Include the changed or added test files in the PR description when production code changed - Check the SonarQube result on the PR when the project secrets are configured in CI @@ -298,6 +298,8 @@ Write unit tests in `tests/unit/` covering at minimum: - [ ] CHANGELOG updated (if user-facing change) - [ ] Documentation updated (if applicable) - [ ] No new CodeQL / Secret-Scanning alerts opened, or each one dismissed with technical justification referencing the relevant `docs/security/` doc +- [ ] Routes that spawn child processes (`/api/mcp/`, `/api/cli-tools/runtime/`) classified as `isLocalOnlyPath()` in `src/server/authz/routeGuard.ts` — see [Hard Rule #15](docs/security/ROUTE_GUARD_TIERS.md) +- [ ] No `Co-Authored-By` trailers in commit messages — commits must appear solely under the repository owner's Git identity (Hard Rule #16) --- @@ -311,5 +313,7 @@ Releases are managed via the `/generate-release` workflow. When a new GitHub Rel - **Architecture**: See [`docs/architecture/ARCHITECTURE.md`](docs/architecture/ARCHITECTURE.md) - **API Reference**: See [`docs/reference/API_REFERENCE.md`](docs/reference/API_REFERENCE.md) +- **Security docs**: [`docs/security/CLI_TOKEN.md`](docs/security/CLI_TOKEN.md), [`docs/security/ROUTE_GUARD_TIERS.md`](docs/security/ROUTE_GUARD_TIERS.md), [`docs/security/ERROR_SANITIZATION.md`](docs/security/ERROR_SANITIZATION.md), [`docs/security/PUBLIC_CREDS.md`](docs/security/PUBLIC_CREDS.md) +- **Ops docs**: [`docs/ops/SQLITE_RUNTIME.md`](docs/ops/SQLITE_RUNTIME.md) - **Issues**: [github.com/diegosouzapw/OmniRoute/issues](https://github.com/diegosouzapw/OmniRoute/issues) - **ADRs**: See `docs/adr/` for architectural decision records diff --git a/docs/compression/COMPRESSION_ENGINES.md b/docs/compression/COMPRESSION_ENGINES.md index 5dcdaa75b4..a5efb02093 100644 --- a/docs/compression/COMPRESSION_ENGINES.md +++ b/docs/compression/COMPRESSION_ENGINES.md @@ -108,6 +108,66 @@ average = 1 - (1 - 0.80) * (1 - 0.46) = 89.2% range = 1 - (1 - 0.60..0.90) * (1 - 0.46) = 78.4-94.6% ``` +## MCP Accessibility Tree Filter + +The MCP accessibility-tree smart filter is a post-execution compression layer that runs on MCP +**tool results**, not on prompts or context. It targets the verbose accessibility-tree and browser +snapshot payloads returned by tools like Playwright, computer-use, and browser-automation MCP +servers. + +### What it does + +1. **Noise stripping** — removes empty generic/text entries (`- generic:`, `- text: ""`) +2. **Sibling collapse** — when ≥ `collapseThreshold` (default 30) consecutive lines are structural + repeats, collapses them into the first `collapseKeepHead` (default 10) lines + a count summary + + the last `collapseKeepTail` (default 5) lines +3. **Ref preservation** — `[ref=eXX]` anchors required by Playwright/computer-use are never touched +4. **Hard truncation** — if the text after collapse still exceeds `maxTextChars` (default 50,000), + truncates with a navigation hint so the agent can continue working + +### Engine location + +```txt +open-sse/services/compression/engines/mcpAccessibility/ + index.ts ← smartFilterText() entry point + collapseRepeated.ts ← sibling-collapse algorithm + constants.ts ← DEFAULT_MCP_ACCESSIBILITY_CONFIG +``` + +### Configuration + +Controlled by `compression.mcpAccessibility` in global settings (migration 056). Default config: + +```json +{ + "enabled": true, + "maxTextChars": 50000, + "collapseThreshold": 30, + "collapseKeepHead": 10, + "collapseKeepTail": 5, + "minLengthToProcess": 2000 +} +``` + +The filter is only applied to tool-result payloads whose `type` is `"text"` and whose length +exceeds `minLengthToProcess`. It does not affect prompt compression or request payloads. + +### Expected savings + +60–80% on browser snapshot tool results, depending on page complexity. The collapse algorithm +is O(n) in line count and adds negligible latency. + +### This filter vs the compression engines above + +| Aspect | Caveman / RTK / Stacked | MCP accessibility filter | +| ----------- | ------------------------- | -------------------------------------- | +| Target | Request prompts / context | MCP tool results | +| Trigger | Compression mode setting | `compression.mcpAccessibility.enabled` | +| Scope | All SSE messages | Tool results only | +| Ref anchors | N/A | Preserved unconditionally | + +--- + ## Compression Combos Compression combos are named compression profiles that can be assigned to routing combos: @@ -121,19 +181,19 @@ Dashboard surface: `Dashboard -> Context & Cache -> Compression Combos`. ## API Surface -| Route | Purpose | -| -------------------------------------- | ------------------------------------------ | -| `/api/settings/compression` | Global compression settings | -| `/api/compression/preview` | Preview any compression mode | -| `/api/compression/language-packs` | List available Caveman language packs | -| `/api/context/caveman/config` | Caveman settings alias | -| `/api/context/rtk/config` | RTK defaults and settings | -| `/api/context/rtk/filters` | RTK filter catalog | -| `/api/context/rtk/test` | RTK preview/test endpoint | -| `/api/context/rtk/raw-output/[id]` | Authenticated redacted raw-output recovery | -| `/api/context/combos` | Compression combo CRUD | -| `/api/context/combos/[id]/assignments` | Routing-combo assignment CRUD | -| `/api/context/analytics` | Compression analytics alias | +| Route | Purpose | +| -------------------------------------- | ---------------------------------------------------------------- | +| `/api/settings/compression` | Global compression settings (includes `mcpAccessibility` config) | +| `/api/compression/preview` | Preview any compression mode | +| `/api/compression/language-packs` | List available Caveman language packs | +| `/api/context/caveman/config` | Caveman settings alias | +| `/api/context/rtk/config` | RTK defaults and settings | +| `/api/context/rtk/filters` | RTK filter catalog | +| `/api/context/rtk/test` | RTK preview/test endpoint | +| `/api/context/rtk/raw-output/[id]` | Authenticated redacted raw-output recovery | +| `/api/context/combos` | Compression combo CRUD | +| `/api/context/combos/[id]/assignments` | Routing-combo assignment CRUD | +| `/api/context/analytics` | Compression analytics alias | Management routes require management authentication or API-key policy checks. @@ -156,5 +216,6 @@ The focused gates for this area are: ```bash node --import tsx/esm --test tests/unit/compression/rtk-*.test.ts tests/unit/compression/pipeline-integration.test.ts tests/unit/compression/context-compression-api.test.ts node --import tsx/esm --test tests/unit/compression/*.test.ts tests/golden-set/*.test.ts tests/integration/compression-pipeline.test.ts tests/unit/api/compression/compression-api.test.ts +node --import tsx/esm --test tests/unit/compression/mcpAccessibility*.test.ts npm run typecheck:core ``` diff --git a/docs/compression/COMPRESSION_LANGUAGE_PACKS.md b/docs/compression/COMPRESSION_LANGUAGE_PACKS.md index 0fa53029d8..cedfed4106 100644 --- a/docs/compression/COMPRESSION_LANGUAGE_PACKS.md +++ b/docs/compression/COMPRESSION_LANGUAGE_PACKS.md @@ -98,6 +98,53 @@ curl -X POST http://localhost:20128/api/compression/preview \ }' ``` +## SHARED_BOUNDARIES (v3.8.0) + +All 6 language packs received a `SHARED_BOUNDARIES` clause in v3.8.0 that is applied at every +Caveman intensity (LITE, FULL, ULTRA). It instructs the engine to preserve these patterns verbatim, +regardless of surrounding filler removal: + +| Pattern type | Example | +| -------------------------------- | -------------------------------------- | +| Fenced code blocks | ` ```python\n...\n``` ` | +| Inline code | `` `my_var` `` | +| URLs | `https://example.com/path` | +| File paths (absolute + relative) | `/etc/hosts`, `./src/index.ts` | +| Error headers | `Error:`, `TypeError:`, `SyntaxError:` | +| Stack trace lines | ` at functionName (file.ts:12:3)` | + +These patterns are populated in `DEFAULT_CAVEMAN_CONFIG.preservePatterns` (previously `[]`). The +constant lives in `open-sse/services/compression/types.ts`. + +### Why this matters + +Without SHARED_BOUNDARIES, aggressive Caveman modes could strip content that looked like repetitive +prose but was actually a code snippet, file path, or error stack. SHARED_BOUNDARIES acts as a +language-agnostic safety net applied before filler rules run. + +### Customizing preservePatterns + +Additional patterns can be added at runtime via compression settings: + +````json +{ + "cavemanConfig": { + "preservePatterns": [ + "```[\\s\\S]*?```", + "`[^`]+`", + "https?://\\S+", + "(?:/|\\./)[^\\s]+", + "\\b(?:Error|TypeError|SyntaxError|RangeError):", + "\\s+at\\s+\\S+\\s+\\(\\S+:\\d+:\\d+\\)" + ] + } +} +```` + +Custom patterns extend (not replace) the 6 defaults. + +--- + ## Operational Notes - English built-in rules remain the fallback when a language pack is missing. diff --git a/docs/frameworks/MCP-SERVER.md b/docs/frameworks/MCP-SERVER.md index 176f7d755b..115adab6a5 100644 --- a/docs/frameworks/MCP-SERVER.md +++ b/docs/frameworks/MCP-SERVER.md @@ -105,6 +105,24 @@ Cursor, Cline, and compatible MCP client setup. descriptions (`tools`, `prompts`, `resources`, and `resourceTemplates`); they are not provider usage receipts and are marked with `source: "mcp_metadata_estimate"`. +### MCP Accessibility Tree Filter (v3.8.0) + +Separate from the 5 compression tools above, OmniRoute includes a post-execution filter that +compresses the **tool results** of MCP browser/accessibility tools before they are returned to the +agent. This filter is not itself a tool — it runs transparently on any tool result that contains +verbose accessibility-tree or browser-snapshot text (≥2000 chars). + +Key behaviors: + +- Collapses ≥30 consecutive repeated sibling lines into head + tail summary +- Preserves `[ref=eXX]` anchors required by Playwright/computer-use +- Hard-truncates oversized text (>50,000 chars) with a navigation hint +- Expected savings: **60–80%** on browser snapshot payloads + +Configuration: `compression.mcpAccessibility` in global settings (migration 056). +Implementation: `open-sse/services/compression/engines/mcpAccessibility/`. +Full docs: [Compression Engines — MCP Accessibility Tree Filter](../compression/COMPRESSION_ENGINES.md#mcp-accessibility-tree-filter). + See [Compression Engines](../compression/COMPRESSION_ENGINES.md) and [RTK Compression](../compression/RTK_COMPRESSION.md) for the runtime compression model behind these tools.