mirror of
https://github.com/diegosouzapw/OmniRoute.git
synced 2026-09-19 05:12:16 +03:00
0fbd8854c5
* feat(api): route /v1/rerank to remote provider nodes behind RERANK_REMOTE_PROVIDER_NODES POST /v1/rerank only ever dispatched to provider nodes whose base URL hostname was localhost, 127.0.0.1, or 172.16.0.0/12 — a filter hardcoded in the route. A rerank node on any other host (a LAN box or Tailscale peer running TEI, Infinity, vLLM, …) was silently dropped and the request fell through to "Invalid rerank model", even though the same node served /v1/embeddings without complaint and had already passed the provider outbound URL policy at creation time. The memory engine's rerank step calls this route over loopback, so `rerankProviderModel` could not reach such a node either. Mirror the audio routes (#3963): loopback nodes stay always-eligible and unchanged; remote nodes are opt-in via a new `RERANK_REMOTE_PROVIDER_NODES` feature flag (default off — routing to a remote host changes egress identity) AND must pass the provider outbound URL policy (`getProviderOutboundGuard()`, `public-only` deployments never route to private hosts. - src/shared/network/loopbackNodeHost.ts: one pure definition of the loopback host set, replacing three copies (rerank route, audioRegistry, localHealthCheck). The shared version also rejects `user@host` URLs, which the audio copy did not. - src/shared/network/providerNodeHost.ts: policy-aware remote-node eligibility that mirrors guardProviderNodeBaseUrl() on the creation path. - src/app/api/v1/_shared/rerankProviderNodes.ts: pure, testable selection step + loader, modelled on audioProviderNodes.ts. - Feature flag definition, FEATURE_FLAGS.md / ENVIRONMENT.md / .env.example rows, API_REFERENCE.md and MEMORY.md notes, changelog fragment. - tests/unit/rerank-remote-provider-nodes.test.ts covers the host classification, the three policy modes, the selection step, and the route end-to-end (flag off → 400 without contacting the node; flag on → forwarded to <base>/v1/rerank with the node credential; flag on + strict policy → still excluded). Feature-flag count test bumped to 56. * chore(changelog): name the #13732 fragment * fix(api): adapt TEI/Infinity request and response shapes on the /v1/rerank node path The provider-node branch of POST /v1/rerank already fell back from <base>/v1/rerank to <base>/rerank on 404 "for Infinity / TEI", but it kept sending the Cohere body and returned the upstream JSON verbatim. Against Hugging Face text-embeddings-inference that could never work: TEI requires the candidate list as `texts` (HTTP 422 otherwise), takes `return_text`, and answers a bare `[{index, score, text?}]` with no `results` envelope and `score` instead of `relevance_score`. Thin gateways in front of TEI/Infinity commonly emit `score` too. Either way the memory engine's applyRerank(), which reads `results[].relevance_score`, ended up with undefined scores. Add two pure adapters in src/app/api/v1/_shared/rerankLocalNodeShapes.ts: - buildLocalRerankRequestBody(): one upstream body carrying both spellings (`documents` + `texts`, `return_documents` + `return_text`). TEI's request struct is not deny_unknown_fields and the OpenAI-shaped servers (vLLM, llama.cpp, Infinity, oMLX) ignore extras, so a single body serves all. - normalizeLocalRerankResponse(): folds `{results:[…]}`, Voyage-style `{data:[…]}`, and TEI's bare array into the Cohere envelope, backfilling `relevance_score` from `score`, sorting by score, honouring `top_n`, attaching `document.text` when requested, dropping malformed entries, and preserving other top-level fields (`model`, `usage`, …). The route now uses both on the primary and fallback fetch. Cloud registry providers are untouched (they go through open-sse/handlers/rerank.ts). tests/unit/rerank-local-node-shapes.test.ts covers the adapters and the route end-to-end: 404 → /rerank with `texts`, bare TEI array normalized and top_n-capped; `score`-only gateway → `relevance_score` for the client. * chore(changelog): name the #13733 fragment * refactor(api): split the local rerank response normalizer into per-entry helpers The complexity ratchet (new-code mode) flagged normalizeLocalRerankResponse at 18/15 on both metrics. Pull the per-entry validation and the document resolution into toCohereResult() / resolveResultDocument(); behaviour and tests are unchanged. --------- Co-authored-by: seanford <seanford@users.noreply.github.com> Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
title, version, lastUpdated
| title | version | lastUpdated |
|---|---|---|
| OmniRoute Documentation | 3.8.40 | 2026-06-28 |
OmniRoute Documentation
Navigable index of the OmniRoute documentation set. Topics are grouped by intent so you can find what you need quickly.
Looking for the project overview, install steps, or release notes? See the root README.md, ROADMAP.md, CHANGELOG.md, and CONTRIBUTING.md.
For Non-Tech Users
Simple guides for using OmniRoute — no technical background needed.
getting-started/
- QUICK-START.md — install and run OmniRoute in 3 minutes.
- AUTO-COMBO-GUIDE.md — let OmniRoute pick the best AI for you.
- PROVIDERS-GUIDE.md — how to connect AI providers.
- FREE-TIERS-GUIDE.md — get free AI with no credit card.
- WEB-COOKIE-GUIDE.md — web cookie providers (session-credential setup).
guides/
- SETUP_GUIDE.md — first-time setup of OmniRoute.
- USER_GUIDE.md — daily usage of the dashboard and API.
- THINKING_BUDGET.md — thinking/reasoning budget modes (passthrough vs auto-strip).
- FEATURES.md — dashboard feature gallery.
- CHAOS-MODE.md — multi-model parallel/collaborative execution (setup, permissions, API).
- TIERS.md — OmniRoute tiers explained (user guide).
- USAGE_QUOTA_GUIDE.md — usage, quota & spend tracking.
- COST_TRACKING.md — cost and spend tracking.
- FREE_PROVIDER_RANKINGS.md — free provider rankings (Arena ELO).
- DOCKER_GUIDE.md — running OmniRoute under Docker, including runtime RAM for coding agents.
- ELECTRON_GUIDE.md — desktop (Electron) builds.
- TERMUX_GUIDE.md — running on Android via Termux.
- PWA_GUIDE.md — installing the dashboard as a PWA.
- REMOTE-MODE.md — exposing OmniRoute remotely + scoped tokens.
- CLI-INTEGRATIONS.md — master table of
setup-*CLI integrations. - OPENCODE-V2-PLUGIN.md — installing and configuring the OpenCode v2 plugin.
- CLAUDE-CODE-CONFIGURATION.md — Claude Code CLI with OmniRoute.
- CODEX-CLI-CONFIGURATION.md — Codex CLI with OmniRoute.
- KIRO_SETUP.md — Kiro setup.
- ANTIGRAVITY-ONBOARDING.md — Antigravity (Google One AI) onboarding.
- MANAGEMENT-AUTH.md — management authentication.
- I18N.md — translation and locale workflow.
- TROUBLESHOOTING.md — detailed troubleshooting reference.
- UNINSTALL.md — clean removal steps.
For Tech Users
Technical documentation for developers and contributors.
architecture/
How the system is put together — read these to understand the runtime, code layout, and resilience model.
- ARCHITECTURE.md — high-level system architecture (request pipeline, layers, modules).
- CODEBASE_DOCUMENTATION.md — engineering reference for the codebase.
- REPOSITORY_MAP.md — directory-by-directory navigation guide.
- AUTHZ_GUIDE.md — authorization pipeline (route classifier + policy engine).
- RESILIENCE_GUIDE.md — provider circuit breaker, connection cooldown, and model lockout.
- QUALITY_GATES.md — quality-gate scripts and CI jobs inventory.
- MONITORING_SECTIONS.md — monitoring/costs dashboard navigation.
- cluster-decisions.md — optional sidecar/cluster profile decisions.
- DESIGN_SYSTEM.md — design system & visual identity.
- ROUTER_BACKENDS.md — router backends & embedded services architecture contract (ADR).
- admission-lanes.md — the two admission-lane systems and what gates each.
- persistence-backend-boundary.md — pluggable persistence boundary (ADR).
reference/
Lookup material — API surface, environment variables, CLI flags, provider catalog.
- API_REFERENCE.md — REST API endpoints and shapes.
- PROVIDER_REFERENCE.md — auto-generated provider catalog (do not edit by hand).
- REMOVED_PROVIDERS.md — providers removed at their operator's request; never reintroduce without written permission.
- PROVIDER_PLUGIN_MANIFEST.md — sidecar-safe provider plugin contract for Bifrost and CLIProxyAPI migration.
- openapi.yaml — OpenAPI spec for the public API.
- ENVIRONMENT.md — environment variables reference.
- FEATURE_FLAGS.md — feature flags and their defaults.
- CLI-TOOLS.md — bundled CLI commands.
- FREE_TIERS.md — free-tier LLM provider directory.
- FREE_PROXIES_API.md — free proxies API.
- RELAY_BACKEND_STRATEGY.md — relay backend strategy.
- RELAY_TROUBLESHOOTING.md — relay troubleshooting.
frameworks/
Pluggable subsystems exposed to clients, agents, and operators.
- MCP-SERVER.md — Model Context Protocol server.
- A2A-SERVER.md — Agent-to-Agent (A2A) JSON-RPC server.
- ACP.md — Agent Client Protocol.
- AGENT_PROTOCOLS_GUIDE.md — A2A / ACP / Cloud agent overview.
- AGENTBRIDGE.md — IDE agent bridge.
- AGENT-SKILLS.md — agent skills catalog.
- CLOUD_AGENT.md — cloud agent runtime and providers.
- SKILLS.md — Skills framework (sandboxed extension).
- MEMORY.md — persistent memory (FTS5 + Qdrant).
- WEBHOOKS.md — webhook events and dispatch.
- EVALS.md — eval suites.
- GAMIFICATION.md — gamification & leaderboard system.
- EMBEDDED-SERVICES.md — embedded sidecar services (9Router, CLIProxyAPI).
- NOTION_CONTEXT.md — Notion context source.
- OBSIDIAN_CONTEXT.md — Obsidian context source.
- LOCAL_CORPUS_CONTEXT.md — local corpus context source (approved directory exposed to MCP).
- OPENCODE.md — OpenCode integration.
- OPEN_SSE_ARCHITECTURE.md — open-sse streaming engine internals.
- PLAYGROUND_STUDIO.md — Playground Studio UI.
- SEARCH_TOOLS_STUDIO.md — Search Tools Studio UI.
- TRAFFIC_INSPECTOR.md — traffic inspector (MITM).
- PLUGINS.md — CLI plugin system overview.
- PLUGIN_SDK.md — plugin SDK reference.
- PLUGIN_MARKETPLACE.md — plugin marketplace.
- RADAR.md — Radar free-model catalog overlay (optional, off by default).
routing/
Combo routing, scoring, and replay.
- AUTO-COMBO.md — Auto-Combo (multi-factor scoring, 19 strategies).
- QUOTA_SHARE.md — quota sharing engine.
- REASONING_REPLAY.md — reasoning replay cache.
- REASONING_ROUTING.md — reasoning routing rules (effort/budget rule engine).
security/
Guardrails, compliance, stealth, and the mandatory patterns for handling public credentials and error messages.
- GUARDRAILS.md — PII, prompt injection, vision guardrails.
- COMPLIANCE.md — audit trails and compliance.
- STEALTH_GUIDE.md — TLS / fingerprint stealth.
- PUBLIC_CREDS.md — mandatory pattern for embedding public upstream OAuth client_id/secret + Firebase Web keys without tripping secret scanners.
- ERROR_SANITIZATION.md — mandatory pattern for routing every error response through
sanitizeErrorMessageto prevent stack-trace exposure. - ROUTE_GUARD_TIERS.md — route-guard classification tiers.
- CLI_TOKEN.md — CLI machine-ID token (HMAC + legacy SHA-256) auth.
- EGRESS_POLICY.md — egress IP family (IPv4/IPv6) policy.
- BAN_DETECTION.md — account-ban / banned-keyword detection.
- AGENTROUTER_WAF.md — agentrouter.org WAF.
- CORS.md — CORS configuration & security.
- MITM-TPROXY-DECRYPT.md — transparent MITM decrypt.
- SUPPLY_CHAIN.md — supply-chain gates (SLSA, SBOM, Trivy, osv-scanner, Scorecard).
- SOCKET_DEV_FINDINGS.md — supply-chain finding attestations.
compression/
Prompt compression engines, rules, and language packs.
- COMPRESSION_GUIDE.md — top-level compression overview.
- COMPRESSION_ENGINES.md — available compression engines.
- COMPRESSION_RULES_FORMAT.md — rule file format.
- COMPRESSION_LANGUAGE_PACKS.md — language packs.
- RTK_COMPRESSION.md — RTK engine deep dive.
- CONTEXT_EDITING.md — delegated context editing (Anthropic).
- EXTENDING_COMPRESSION.md — adding a custom compression engine.
providers/
Provider-specific integration guides.
- CLAUDE_WEB.md — Claude Web (cookie-auth) provider.
- CHATGPT_WEB.md — ChatGPT Web (Codex) provider and common-provider retirement note.
- COPILOT-M365.md — Microsoft 365 Copilot (BizChat) provider.
- ALIBABA-QWEN-PROVIDER-FAMILIES.md — Alibaba and Qwen provider families.
- AGENTROUTER.md — AgentRouter setup.
- ZED-DOCKER.md — Zed IDE integration under Docker.
- CURSOR-DOCKER.md — Cursor model listing under Docker.
comparison/
- OMNIROUTE_VS_ALTERNATIVES.md — how OmniRoute compares to alternatives.
ops/
Release, deployment, proxies, tunnels, coverage, database, monitoring.
- RELEASE_CHECKLIST.md — release flow checklist.
- RELEASE_GREEN.md — keeping the PR queue and release branch green.
- BRANCHING_MODEL.md — branching & release model.
- MERGE_TRAIN.md — merge queue & manual merge-train runbook.
- HOMOLOGATION.md — homologation suite (
npm run homolog). - QUALITY_GATE_PLAYBOOK.md — quality-gate playbook.
- RUNNER_BOX.md — self-hosted runner box operations.
- BRANCH_PROTECTION_MAIN.md —
mainbranch protection. - CONTRIBUTION_GOLDEN_PATH.md — contribution golden path (focused checks per change type).
- COVERAGE_PLAN.md — test coverage plan.
- DATABASE_GUIDE.md — DB schema and operations.
- SQLITE_RUNTIME.md — SQLite driver resolution chain.
- REDIS_PRODUCTION_CONFIG.md — Redis production configuration.
- MONITORING_GUIDE.md — monitoring & observability.
- FLY_IO_DEPLOYMENT_GUIDE.md — Fly.io deployment.
- VM_DEPLOYMENT_GUIDE.md — generic VM deployment.
- PROXY_GUIDE.md — upstream proxy configuration.
- TUNNELS_GUIDE.md — Cloudflare tunnel and friends.
diagrams/
Mermaid sources and exported SVG/PNG diagrams referenced from the docs above. See diagrams/README.md.
i18n/
Translated mirrors of the documentation in 65 locales (plus the English originals — 66 languages in total). See i18n/README.md for the supported language list.
screenshots/
Static screenshots used by the dashboard and the README. Not part of the doc body.
Auto-generated artifacts
- reference/PROVIDER_REFERENCE.md is generated by
scripts/docs/gen-provider-reference.tsfromsrc/shared/constants/providers.ts. Do not edit by hand. - The
/docsUI is backed by Fumadocs MDX source generation from the subfolders above.