pjandro/OmniRoute
1
0
Fork 0
mirror of https://github.com/diegosouzapw/OmniRoute.git synced 2026-09-19 21:32:20 +03:00
Code Issues Packages Projects Releases Wiki Activity

feat(routing): deterministic routing strategies for self-hosted entry (RIC-740) (#13611)

Browse Source 
* feat(routing): self-hosted unified OpenAI-compatible entry (RIC-738)

Divert /v1/chat/completions through the self-hosted provider adapters when
OMNIROUTE_SELF_HOSTED_PROVIDERS / OMNIROUTE_SELF_HOSTED_PROVIDERS_FILE is set:
one OpenAI-compatible contract in, auto-route to the selected provider
(x-omniroute-provider header, provider/model prefix, or first provider),
standard OpenAI error shape out. Optional OMNIROUTE_SELF_HOSTED_API_KEY guards
the entry (D5 reserved); unset = open loopback route. Upstream credentials stay
runtime-only and are stripped from echoed responses.

Brings in the provider-adapters baseline from sibling branch (RIC-737) that
this entry depends on. Includes 21 passing unit tests (provider selection,
model-prefix forwarding, header hygiene, auth, error normalization, SSE
passthrough, fall-through/misconfig), docs, env example, changelog fragment.

* feat(routing): deterministic routing strategies for self-hosted entry (RIC-740)

Add the M2 deterministic routing strategy engine (D3 可审计路由) to the
self-hosted unified entry: a declarative `strategy:` block expressing five
explainable, non-predictive policies — blacklist/whitelist hard filters,
cooldown circuit breaker, cost-priority, latency-aware ordering, and an
explicit fallback chain. The ordered candidate list is the fallback chain:
a failed primary (network or non-2xx) falls through to the next candidate and
each failure feeds the breaker. Every response carries an
x-omniroute-route-decision header answering "why this model / why not that
one". A pinned provider rejected by a hard filter returns 400 (never a silent
re-route); no eligible providers returns 503 with the full explainable
decision. No ML/predict dependency.

Covers the RIC-740 acceptance: 5 strategy types with unit tests + HTTP
fault-injection tests (primary down -> fallback works), config matching docs,
and no predict/ML deps. Adds docs, .env.example entries, and a changelog
fragment.

* refactor(routing): reduce complexity-ratchet violations in new self-hosted routing files

Extract cost/id validation, pin-blocked resolution, ordering, and env/file
source resolution into small helpers so routingStrategies.ts and
selfHostedEntry.ts stay under the complexity-ratchets cap. No behavior
change — the same 51 routing-strategies/self-hosted-entry/provider-adapters
tests pass unmodified.

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>

* docs(routing): document the 5 self-hosted env vars in ENVIRONMENT.md

check:env-doc-sync failed because OMNIROUTE_SELF_HOSTED_PROVIDERS(_FILE),
OMNIROUTE_SELF_HOSTED_API_KEY and OMNIROUTE_SELF_HOSTED_STRATEGY(_FILE)
were present in .env.example but missing from
docs/reference/ENVIRONMENT.md. Add them under "6. Tool & Routing Policies".

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>

---------

Co-authored-by: Ant Rich <ant@richants.com>
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
Co-authored-by: luyuehm <luyuehm@users.noreply.github.com>
This commit is contained in:
luyuehm
2026-09-18 22:59:15 +08:00
committed by GitHub
parent 3ebea07278
commit 5a82da7084
13 changed files with 2282 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

5
docs/reference/ENVIRONMENT.md
View File

@@ -310,6 +310,11 @@ OmniRoute provides a two-layer defense: request-side injection scanning and resp
| `OMNIROUTE_PREFER_CLAUDE_CODE_FOR_UNPREFIXED_CLAUDE_MODELS` | `false` | `open-sse/services/model.ts` | Opt-in: route bare `claude-*` model IDs from Claude Code clients through the Claude Code OAuth account instead of requiring a provider prefix. Explicit provider prefixes still win. Also configurable via a dashboard toggle on the Claude provider page. |
| `COMBO_CONCURRENCY_PER_MODEL` | `3` | `open-sse/services/comboConfig.ts` | Per-model concurrency cap for round-robin combos (#9100). The round-robin combo semaphore was hard-capped at 3 concurrent requests per model with no override, serializing higher-concurrency traffic behind that cap. Validated to `>= 1`, clamped to `<= 32`. |
| `DISABLE_CONTEXT_WINDOW_CHECKS` | `false` | `open-sse/handlers/chatCore.ts` | Dangerous opt-in that skips OmniRoute's local context-window / max-input-token check for direct single-model requests. Upstream providers still enforce their actual limits; prompt compression and the model's own output-token cap remain active. Effective precedence is Feature Flags DB override > environment variable > default; no restart is required. |
| `OMNIROUTE_SELF_HOSTED_PROVIDERS` | _(unset)_ | `open-sse/services/selfHostedEntry.ts` | Inline YAML `providers:` document (RIC-738, D4). When set (with or without a `strategy:` block), `/v1/chat/completions` diverts to the self-hosted unified OpenAI-compatible entry instead of the cloud pipeline. Unset (the default): the route falls straight through to the existing cloud pipeline. See `docs/routing/SELF_HOSTED_OPENAI_ENTRY.md`. |
| `OMNIROUTE_SELF_HOSTED_PROVIDERS_FILE` | _(unset)_ | `open-sse/services/selfHostedEntry.ts` | Path to a YAML file holding the same `providers:` document as `OMNIROUTE_SELF_HOSTED_PROVIDERS`, for deployments that prefer a file over an inline env var. Either one activates the self-hosted entry. |
| `OMNIROUTE_SELF_HOSTED_API_KEY` | _(unset — open route)_ | `open-sse/services/selfHostedEntry.ts` | Optional shared API key for the unified self-hosted entry (D5 scaffold, reserved for the per-key quota system). When set, requests must include `Authorization: Bearer <key>`. Unset: the route is open, matching the existing self-hosted local-provider pattern (loopback/trusted-network deployment). |
| `OMNIROUTE_SELF_HOSTED_STRATEGY` | _(unset)_ | `open-sse/services/routingStrategies.ts` | Inline YAML `strategy:` document for the deterministic routing engine (M2/RIC-740, D3) — blacklist/whitelist, cooldown breaker, cost-priority, latency-aware, fallback chain. Overrides an inline `strategy:` block nested inside `OMNIROUTE_SELF_HOSTED_PROVIDERS`, per-key. See `docs/routing/DETERMINISTIC_ROUTING.md`. |
| `OMNIROUTE_SELF_HOSTED_STRATEGY_FILE` | _(unset)_ | `open-sse/services/routingStrategies.ts` | Path to a YAML file holding the same `strategy:` document as `OMNIROUTE_SELF_HOSTED_STRATEGY`, for deployments that prefer a file over an inline env var. |
| `OMNIROUTE_DISABLE_CONVERSATION_TRACKING` | _(unset)_ | `open-sse/services/conversationTracker.ts` | Set `1` to stop collecting conversation history. `resolveConversationId()` returns an untracked result before it reads SQLite or parses message history, and client-supplied session IDs are covered too. Routing-session handling is unchanged and existing records are not deleted. For deployments that do not use the dashboard's conversation view and want the turn table to stop growing. |
---