mirror of
https://github.com/diegosouzapw/OmniRoute.git
synced 2026-07-31 12:22:14 +03:00
0acb8d0aeb
The compression "ultra" SLM tier (#4257) runs @atjsh/llmlingua-2 + transformers + tfjs + js-tiktoken in a worker thread shipped under dist/. These are optionalDependencies installed into the ROOT node_modules on --include=optional, but the Next.js standalone trace bundles ONLY @huggingface/transformers (3.5.2, pinned) into dist/node_modules — not the dynamically-imported optionals. Result: the worker resolves transformers from dist/node_modules (3.5.2) for its env config but resolves @atjsh/llmlingua-2 from the ROOT, whose own transformers import hits a DIFFERENT instance. The cacheDir config never reaches the instance llmlingua-2 uses, so the local model never loads and the SLM tier silently fails-open (and on a root transformers 4.x, llmlingua-2 throws on the tokenizer API change). Fix: postinstall co-locates the SLM optional closure from the root node_modules into dist/node_modules (no-clobber, so the pinned dist transformers/onnxruntime stay), so the worker resolves a single 3.5.2 instance and the local model loads. VPS-validated (Rule #18): the co-located layout produced real 54.8% compression (11520->5203 chars) via real ONNX inference on the production host, both the default and the #4257 modelPath code paths. - scripts/build/colocateOptionals.mjs: closure walk (deps+optionalDeps, skips the transformers peer) + no-clobber co-location; idempotent + fail-soft - wired into scripts/build/postinstall.mjs next to ensureSwcHelpers - registered in package.json files + pack-artifact allow/required lists - tests/unit/colocate-optionals.test.ts: closure, no-clobber, idempotence, gates - docs/ops/RELEASE_CHECKLIST.md: note the auto co-location
title, version, lastUpdated
| title | version | lastUpdated |
|---|---|---|
| OmniRoute Documentation | 3.8.24 | 2026-06-13 |
OmniRoute Documentation
Navigable index of the OmniRoute documentation set. Topics are grouped by intent so you can find what you need quickly.
Looking for the project overview, install steps, or release notes? See the root README.md, CHANGELOG.md, and CONTRIBUTING.md.
architecture/
How the system is put together — read these to understand the runtime, code layout, and resilience model.
- ARCHITECTURE.md — high-level system architecture (request pipeline, layers, modules).
- CODEBASE_DOCUMENTATION.md — engineering reference for the codebase.
- REPOSITORY_MAP.md — directory-by-directory navigation guide.
- AUTHZ_GUIDE.md — authorization pipeline (route classifier + policy engine).
- RESILIENCE_GUIDE.md — provider circuit breaker, connection cooldown, and model lockout.
For Non-Tech Users
Simple guides for using OmniRoute — no technical background needed.
getting-started/
- QUICK-START.md — install and run OmniRoute in 3 minutes.
- AUTO-COMBO-GUIDE.md — let OmniRoute pick the best AI for you.
- PROVIDERS-GUIDE.md — how to connect AI providers.
- FREE-TIERS-GUIDE.md — get free AI with no credit card.
- TROUBLESHOOTING.md — fix common issues.
guides/
- SETUP_GUIDE.md — first-time setup of OmniRoute.
- USER_GUIDE.md — daily usage of the dashboard and API.
- DOCKER_GUIDE.md — running OmniRoute under Docker.
- ELECTRON_GUIDE.md — desktop (Electron) builds.
- TERMUX_GUIDE.md — running on Android via Termux.
- PWA_GUIDE.md — installing the dashboard as a PWA.
- UNINSTALL.md — clean removal steps.
- I18N.md — translation and locale workflow.
- FEATURES.md — dashboard feature gallery.
For Tech Users
Technical documentation for developers and contributors.
reference/
Lookup material — API surface, environment variables, CLI flags, provider catalog.
- API_REFERENCE.md — REST API endpoints and shapes.
- PROVIDER_REFERENCE.md — auto-generated provider catalog.
- openapi.yaml — OpenAPI 3.1 spec for the public API.
- ENVIRONMENT.md — environment variables reference.
- CLI-TOOLS.md — bundled CLI commands.
- FREE_TIERS.md — free-tier LLM provider directory.
frameworks/
Pluggable subsystems exposed to clients, agents, and operators.
- MCP-SERVER.md — Model Context Protocol server.
- A2A-SERVER.md — Agent-to-Agent (A2A) JSON-RPC server.
- AGENT_PROTOCOLS_GUIDE.md — A2A / ACP / Cloud agent overview.
- CLOUD_AGENT.md — cloud agent runtime and providers.
- SKILLS.md — Skills framework (sandboxed extension).
- MEMORY.md — persistent memory (FTS5 + Qdrant).
- WEBHOOKS.md — webhook events and dispatch.
- EVALS.md — eval suites.
routing/
Combo routing, scoring, and replay.
- AUTO-COMBO.md — Auto-Combo (9-factor scoring, 15 strategies).
- REASONING_REPLAY.md — reasoning replay flow.
security/
Guardrails, compliance, stealth, and the mandatory patterns for handling public credentials and error messages.
- GUARDRAILS.md — PII, prompt injection, vision guardrails.
- COMPLIANCE.md — audit trails and compliance.
- STEALTH_GUIDE.md — TLS / fingerprint stealth.
- PUBLIC_CREDS.md — mandatory pattern for embedding public upstream OAuth client_id/secret + Firebase Web keys without tripping secret scanners.
- ERROR_SANITIZATION.md — mandatory pattern for routing every error response through
sanitizeErrorMessageto prevent stack-trace exposure.
compression/
Prompt compression engines, rules, and language packs.
- COMPRESSION_GUIDE.md — top-level compression overview.
- COMPRESSION_ENGINES.md — available compression engines.
- COMPRESSION_RULES_FORMAT.md — rule file format.
- COMPRESSION_LANGUAGE_PACKS.md — language packs.
- RTK_COMPRESSION.md — RTK engine deep dive.
ops/
Release, deployment, proxies, tunnels, coverage.
- RELEASE_CHECKLIST.md — release flow checklist.
- COVERAGE_PLAN.md — test coverage plan.
- FLY_IO_DEPLOYMENT_GUIDE.md — Fly.io deployment.
- VM_DEPLOYMENT_GUIDE.md — generic VM deployment.
- PROXY_GUIDE.md — upstream proxy configuration.
- TUNNELS_GUIDE.md — Cloudflare tunnel and friends.
diagrams/
Mermaid sources and exported SVG/PNG diagrams referenced from the docs above. Populated incrementally — see diagrams/README.md.
i18n/
Translated mirrors of the documentation in 42 locales. See i18n/README.md for the supported language list.
screenshots/
Static screenshots used by the dashboard and the README. Not part of the doc body.
Auto-generated artifacts
- reference/PROVIDER_REFERENCE.md is generated by
scripts/docs/gen-provider-reference.tsfromsrc/shared/constants/providers.ts. Do not edit by hand. - The
/docsUI is backed by Fumadocs MDX source generation from the subfolders above.