mirror of
https://github.com/diegosouzapw/OmniRoute.git
synced 2026-08-18 21:22:28 +03:00
d49ccdaaf1
* fix(sse): gate structural chat admission shedding on real heap pressure Closes #10183, Closes #10268 3.8.49 (#9654/#9940) replaced the 3.8.48 heap-ratio shed (heapUsed/heapLimit >= 0.75) in chatBodyAdmission.ts with an unconditional CHAT_MAX_HEAVY_IN_FLIGHT=1 structural lease. A second concurrent "structurally heavy" chat request (>=200 messages, >=64 tools, or >=32k estimated tokens — routine for coding-agent fan-out like Hermes/Cursor/Claude Code) was hard-rejected with a retryable HTTP 503 chat_admission_busy/structure_limit regardless of actual heap pressure, even on a host with ample free RAM. Restore the heap-conditional gate as an ADDITIONAL check layered on top of (not a replacement for) the #9654 bounded-concurrency / per-connection-lane protection: when heavyweight capacity is busy, only enter the bounded-wait/shed path when a live heap-pressure probe (heapUsed / v8 heap_size_limit >= OMNIROUTE_CHAT_ADMISSION_HEAP_SHED_RATIO, default 0.75) confirms real pressure. A healthy heap now admits the second heavy request immediately via a no-op lease instead of parking or shedding it. The probe is injectable via admitChatStructure({ heapPressureCheck }) for deterministic tests. Regression tests: - tests/unit/bug-10183-admission-heavy-healthy-heap.test.ts (new, permanent): healthy-heap 2nd heavy request now admitted (was RED); genuinely pressured heap still sheds it. - tests/unit/probe-10268-structural-503.test.ts (promoted to permanent): the exact reported 503 chat_admission_busy shape is still produced under real heap pressure, and the same fan-out is admitted on a healthy heap. - tests/unit/chat-body-admission.test.ts, tests/unit/chat-body-admission-queue.test.ts, tests/unit/per-connection-admission-9654.test.ts updated to inject heapPressureCheck: () => true where they exercise the busy/shed path, preserving #9654/#4380 coverage. Gates run: npm run typecheck:core (clean), eslint --suppressions-location config/quality/eslint-suppressions.json on changed files (clean), scripts/check/check-file-size.mjs (OK), scripts/check/check-test-discovery.mjs (OK), focused admission suite (68/68 passing) and npm run test:unit (in progress at commit time under heavy shared-devbox contention from a 13-way parallel session fan-out; no admission-related failures observed through 1873 lines of output, the sole failure seen was a pre-existing unrelated proxy/search timeout consistent with known load-induced flakiness, not a regression from this change). ⚠️ base-red inherited: #9985 — ESLint errors (2) from #10250 * docs(env): document OMNIROUTE_CHAT_ADMISSION_HEAP_SHED_RATIO (#10183, #10268) * fix(sse): bound the healthy-heap admission fast path (#10437) The #10183/#10268 fix admitted a busy heavyweight request immediately whenever the heap was healthy, via an unconditional no-op lease with no bound of its own -- an unlimited number of "healthy heap" requests could pile in ahead of the heap-pressure shed path, defeating the point of admission control. Adds an independent, bounded healthy-heap headroom budget (CHAT_ADMISSION_HEALTHY_HEADROOM, tryAcquireHealthyHeadroom()) that the healthy-heap fast path draws from; once exhausted, requests fall through to the same bounded-wait/shed path used under real heap pressure, which is otherwise unchanged. Also fixes a pre-existing gap in per-connection-admission-9654.test.ts's shared-budget test, which needed an explicit heapPressureCheck override to keep exercising the #10110 invariant now that a healthy heap gets bounded headroom instead of an outright reject. * docs(env): document OMNIROUTE_CHAT_ADMISSION_HEALTHY_HEADROOM in .env.example Documented in docs/reference/ENVIRONMENT.md but missing from .env.example, caught by the env-doc-sync gate when combined with other PRs in the release merge-train. --------- Co-authored-by: adevwithpurpose <adevwithpurpose@users.noreply.github.com>
title, version, lastUpdated
| title | version | lastUpdated |
|---|---|---|
| OmniRoute Documentation | 3.8.40 | 2026-06-28 |
OmniRoute Documentation
Navigable index of the OmniRoute documentation set. Topics are grouped by intent so you can find what you need quickly.
Looking for the project overview, install steps, or release notes? See the root README.md, ROADMAP.md, CHANGELOG.md, and CONTRIBUTING.md.
For Non-Tech Users
Simple guides for using OmniRoute — no technical background needed.
getting-started/
- QUICK-START.md — install and run OmniRoute in 3 minutes.
- AUTO-COMBO-GUIDE.md — let OmniRoute pick the best AI for you.
- PROVIDERS-GUIDE.md — how to connect AI providers.
- FREE-TIERS-GUIDE.md — get free AI with no credit card.
- WEB-COOKIE-GUIDE.md — web cookie providers (session-credential setup).
guides/
- SETUP_GUIDE.md — first-time setup of OmniRoute.
- USER_GUIDE.md — daily usage of the dashboard and API.
- THINKING_BUDGET.md — thinking/reasoning budget modes (passthrough vs auto-strip).
- FEATURES.md — dashboard feature gallery.
- TIERS.md — OmniRoute tiers explained (user guide).
- USAGE_QUOTA_GUIDE.md — usage, quota & spend tracking.
- COST_TRACKING.md — cost and spend tracking.
- FREE_PROVIDER_RANKINGS.md — free provider rankings (Arena ELO).
- DOCKER_GUIDE.md — running OmniRoute under Docker.
- ELECTRON_GUIDE.md — desktop (Electron) builds.
- TERMUX_GUIDE.md — running on Android via Termux.
- PWA_GUIDE.md — installing the dashboard as a PWA.
- REMOTE-MODE.md — exposing OmniRoute remotely + scoped tokens.
- CLI-INTEGRATIONS.md — master table of
setup-*CLI integrations. - CLAUDE-CODE-CONFIGURATION.md — Claude Code CLI with OmniRoute.
- CODEX-CLI-CONFIGURATION.md — Codex CLI with OmniRoute.
- KIRO_SETUP.md — Kiro setup.
- ANTIGRAVITY-ONBOARDING.md — Antigravity (Google One AI) onboarding.
- MANAGEMENT-AUTH.md — management authentication.
- I18N.md — translation and locale workflow.
- TROUBLESHOOTING.md — detailed troubleshooting reference.
- UNINSTALL.md — clean removal steps.
For Tech Users
Technical documentation for developers and contributors.
architecture/
How the system is put together — read these to understand the runtime, code layout, and resilience model.
- ARCHITECTURE.md — high-level system architecture (request pipeline, layers, modules).
- CODEBASE_DOCUMENTATION.md — engineering reference for the codebase.
- REPOSITORY_MAP.md — directory-by-directory navigation guide.
- AUTHZ_GUIDE.md — authorization pipeline (route classifier + policy engine).
- RESILIENCE_GUIDE.md — provider circuit breaker, connection cooldown, and model lockout.
- QUALITY_GATES.md — quality-gate scripts and CI jobs inventory.
- MONITORING_SECTIONS.md — monitoring/costs dashboard navigation.
- cluster-decisions.md — optional sidecar/cluster profile decisions.
- DESIGN_SYSTEM.md — design system & visual identity.
- ROUTER_BACKENDS.md — router backends & embedded services architecture contract (ADR).
- admission-lanes.md — the two admission-lane systems and what gates each.
- persistence-backend-boundary.md — pluggable persistence boundary (ADR).
reference/
Lookup material — API surface, environment variables, CLI flags, provider catalog.
- API_REFERENCE.md — REST API endpoints and shapes.
- PROVIDER_REFERENCE.md — auto-generated provider catalog (do not edit by hand).
- PROVIDER_PLUGIN_MANIFEST.md — sidecar-safe provider plugin contract for Bifrost and CLIProxyAPI migration.
- openapi.yaml — OpenAPI spec for the public API.
- ENVIRONMENT.md — environment variables reference.
- FEATURE_FLAGS.md — feature flags and their defaults.
- CLI-TOOLS.md — bundled CLI commands.
- FREE_TIERS.md — free-tier LLM provider directory.
- FREE_PROXIES_API.md — free proxies API.
- RELAY_BACKEND_STRATEGY.md — relay backend strategy.
- RELAY_TROUBLESHOOTING.md — relay troubleshooting.
frameworks/
Pluggable subsystems exposed to clients, agents, and operators.
- MCP-SERVER.md — Model Context Protocol server.
- A2A-SERVER.md — Agent-to-Agent (A2A) JSON-RPC server.
- ACP.md — Agent Client Protocol.
- AGENT_PROTOCOLS_GUIDE.md — A2A / ACP / Cloud agent overview.
- AGENTBRIDGE.md — IDE agent bridge.
- AGENT-SKILLS.md — agent skills catalog.
- CLOUD_AGENT.md — cloud agent runtime and providers.
- SKILLS.md — Skills framework (sandboxed extension).
- MEMORY.md — persistent memory (FTS5 + Qdrant).
- WEBHOOKS.md — webhook events and dispatch.
- EVALS.md — eval suites.
- GAMIFICATION.md — gamification & leaderboard system.
- EMBEDDED-SERVICES.md — embedded sidecar services (9Router, CLIProxyAPI).
- NOTION_CONTEXT.md — Notion context source.
- OBSIDIAN_CONTEXT.md — Obsidian context source.
- LOCAL_CORPUS_CONTEXT.md — local corpus context source (approved directory exposed to MCP).
- OPENCODE.md — OpenCode integration.
- OPEN_SSE_ARCHITECTURE.md — open-sse streaming engine internals.
- PLAYGROUND_STUDIO.md — Playground Studio UI.
- SEARCH_TOOLS_STUDIO.md — Search Tools Studio UI.
- TRAFFIC_INSPECTOR.md — traffic inspector (MITM).
- PLUGINS.md — CLI plugin system overview.
- PLUGIN_SDK.md — plugin SDK reference.
- PLUGIN_MARKETPLACE.md — plugin marketplace.
- RADAR.md — Radar free-model catalog overlay (optional, off by default).
routing/
Combo routing, scoring, and replay.
- AUTO-COMBO.md — Auto-Combo (multi-factor scoring, 19 strategies).
- QUOTA_SHARE.md — quota sharing engine.
- REASONING_REPLAY.md — reasoning replay cache.
- REASONING_ROUTING.md — reasoning routing rules (effort/budget rule engine).
security/
Guardrails, compliance, stealth, and the mandatory patterns for handling public credentials and error messages.
- GUARDRAILS.md — PII, prompt injection, vision guardrails.
- COMPLIANCE.md — audit trails and compliance.
- STEALTH_GUIDE.md — TLS / fingerprint stealth.
- PUBLIC_CREDS.md — mandatory pattern for embedding public upstream OAuth client_id/secret + Firebase Web keys without tripping secret scanners.
- ERROR_SANITIZATION.md — mandatory pattern for routing every error response through
sanitizeErrorMessageto prevent stack-trace exposure. - ROUTE_GUARD_TIERS.md — route-guard classification tiers.
- CLI_TOKEN.md — CLI machine-ID token (HMAC + legacy SHA-256) auth.
- EGRESS_POLICY.md — egress IP family (IPv4/IPv6) policy.
- BAN_DETECTION.md — account-ban / banned-keyword detection.
- AGENTROUTER_WAF.md — agentrouter.org WAF.
- CORS.md — CORS configuration & security.
- MITM-TPROXY-DECRYPT.md — transparent MITM decrypt.
- SUPPLY_CHAIN.md — supply-chain gates (SLSA, SBOM, Trivy, osv-scanner, Scorecard).
- SOCKET_DEV_FINDINGS.md — supply-chain finding attestations.
compression/
Prompt compression engines, rules, and language packs.
- COMPRESSION_GUIDE.md — top-level compression overview.
- COMPRESSION_ENGINES.md — available compression engines.
- COMPRESSION_RULES_FORMAT.md — rule file format.
- COMPRESSION_LANGUAGE_PACKS.md — language packs.
- RTK_COMPRESSION.md — RTK engine deep dive.
- CONTEXT_EDITING.md — delegated context editing (Anthropic).
- EXTENDING_COMPRESSION.md — adding a custom compression engine.
providers/
Provider-specific integration guides.
- CLAUDE_WEB.md — Claude Web (cookie-auth) provider.
- CHATGPT_WEB.md — ChatGPT Web (Plus/Pro + Codex) providers.
- ALIBABA-QWEN-PROVIDER-FAMILIES.md — Alibaba and Qwen provider families.
- AGENTROUTER.md — AgentRouter setup.
- ZED-DOCKER.md — Zed IDE integration under Docker.
- CURSOR-DOCKER.md — Cursor model listing under Docker.
comparison/
- OMNIROUTE_VS_ALTERNATIVES.md — how OmniRoute compares to alternatives.
ops/
Release, deployment, proxies, tunnels, coverage, database, monitoring.
- RELEASE_CHECKLIST.md — release flow checklist.
- RELEASE_GREEN.md — keeping the PR queue and release branch green.
- BRANCHING_MODEL.md — branching & release model.
- MERGE_TRAIN.md — merge queue & manual merge-train runbook.
- HOMOLOGATION.md — homologation suite (
npm run homolog). - QUALITY_GATE_PLAYBOOK.md — quality-gate playbook.
- RUNNER_BOX.md — self-hosted runner box operations.
- BRANCH_PROTECTION_MAIN.md —
mainbranch protection. - CONTRIBUTION_GOLDEN_PATH.md — contribution golden path (focused checks per change type).
- COVERAGE_PLAN.md — test coverage plan.
- DATABASE_GUIDE.md — DB schema and operations.
- SQLITE_RUNTIME.md — SQLite driver resolution chain.
- REDIS_PRODUCTION_CONFIG.md — Redis production configuration.
- MONITORING_GUIDE.md — monitoring & observability.
- FLY_IO_DEPLOYMENT_GUIDE.md — Fly.io deployment.
- VM_DEPLOYMENT_GUIDE.md — generic VM deployment.
- PROXY_GUIDE.md — upstream proxy configuration.
- TUNNELS_GUIDE.md — Cloudflare tunnel and friends.
diagrams/
Mermaid sources and exported SVG/PNG diagrams referenced from the docs above. See diagrams/README.md.
i18n/
Translated mirrors of the documentation in 43 locales. See i18n/README.md for the supported language list.
screenshots/
Static screenshots used by the dashboard and the README. Not part of the doc body.
Auto-generated artifacts
- reference/PROVIDER_REFERENCE.md is generated by
scripts/docs/gen-provider-reference.tsfromsrc/shared/constants/providers.ts. Do not edit by hand. - The
/docsUI is backed by Fumadocs MDX source generation from the subfolders above.