mirror of https://github.com/diegosouzapw/OmniRoute.git synced 2026-07-26 09:52:11 +03:00

Files

diegosouzapw 20cb648ea9 docs(docs): expand v3.8.0 guides and add provider reference generator

Refresh the documentation set for v3.8.0 with new guides covering
authz, agent protocols, cloud agents, compliance, electron, evals,
guardrails, memory, skills, stealth, tunnels, webhooks, and more.

Add an auto-generated provider reference plus a
`gen:provider-reference` script to keep provider catalog docs aligned
with `src/shared/constants/providers.ts`.

2026-05-13 03:47:41 -03:00

3.8 KiB

Raw Blame History

Compression Language Packs

Caveman compression can load language-specific rule packs in addition to the built-in English rules. This keeps the core engine stable while allowing Portuguese, Spanish, German, French, Japanese, and future language packs to evolve independently.

Location

Language packs live under:

open-sse/services/compression/rules/<language>/

Current shipped packs (verified against rules/ directory contents):

Language	Directory	Rule categories present
English	`rules/en/`	`context`, `dedup`, `filler`, `structural`, `ultra`
Spanish	`rules/es/`	`context`, `dedup`, `filler`, `structural`, `ultra`
Portuguese (Brazil)	`rules/pt-BR/`	`context`, `filler`, `structural`
German	`rules/de/`	`context`, `filler`, `structural`
French	`rules/fr/`	`context`, `filler`, `structural`
Japanese	`rules/ja/`	`context`, `filler`, `structural`

Parity note: en and es packs have the full 5 categories; pt-BR, de, fr, ja ship 3 categories. The missing dedup and ultra categories silently fall back to the English built-ins. Contributions welcome to add dedup.json and ultra.json for the smaller packs.

The canonical category list and per-category schema live in open-sse/services/compression/rules/_schema.json (JSON Schema draft 2020-12).

Language Detection

languageDetector.ts uses lightweight heuristics to infer the language from prompt text. The configured default language is still respected, and detection can be disabled by config when exact control is required.

Detection output is used only to choose rule packs. It does not change provider routing, locale selection, or UI language.

Config Shape

Compression settings can include:

{
  "languageConfig": {
    "enabled": true,
    "defaultLanguage": "en",
    "autoDetect": true,
    "enabledPacks": ["en", "pt-BR", "es", "de", "fr", "ja"]
  },
  "cavemanConfig": {
    "language": "en",
    "autoDetectLanguage": true,
    "enabledLanguagePacks": ["en", "pt-BR", "es", "de", "fr", "ja"]
  }
}

languageConfig controls dashboard/preview defaults. cavemanConfig is the runtime engine config used when Caveman compresses message text.

Adding a Language Pack

Create open-sse/services/compression/rules/<language>/<pack>.json.
Use the Caveman rule format from docs/COMPRESSION_RULES_FORMAT.md.
Keep replacements conservative and avoid changing code, identifiers, URLs, or JSON.
Add or update tests for language selection and replacement behavior.
Expose new dashboard/i18n labels if the language appears in UI selectors.

API

Available packs can be queried with:

curl http://localhost:20128/api/compression/language-packs

The preview endpoint accepts language config overrides:

curl -X POST http://localhost:20128/api/compression/preview \
  -H "Content-Type: application/json" \
  -d '{
    "mode": "standard",
    "text": "Por favor, eu gostaria que voce basicamente resumisse isso.",
    "config": {
      "languageConfig": {
        "defaultLanguage": "pt-BR",
        "autoDetect": true
      }
    }
  }'

Operational Notes

English built-in rules remain the fallback when a language pack is missing.
Invalid built-in JSON packs fail validation so release assets do not silently degrade.
Rule packs are data-only and should not import code or run arbitrary logic.
The compression analytics layer records the selected mode and engine, not full prompt text.

3.8 KiB Raw Blame History