pjandro/OmniRoute
1
0
Fork 0
mirror of https://github.com/diegosouzapw/OmniRoute.git synced 2026-09-20 05:42:19 +03:00
Code Issues Packages Projects Releases Wiki Activity

feat(routing): deterministic routing strategies for self-hosted entry (RIC-740) (#13611)

Browse Source 
* feat(routing): self-hosted unified OpenAI-compatible entry (RIC-738)

Divert /v1/chat/completions through the self-hosted provider adapters when
OMNIROUTE_SELF_HOSTED_PROVIDERS / OMNIROUTE_SELF_HOSTED_PROVIDERS_FILE is set:
one OpenAI-compatible contract in, auto-route to the selected provider
(x-omniroute-provider header, provider/model prefix, or first provider),
standard OpenAI error shape out. Optional OMNIROUTE_SELF_HOSTED_API_KEY guards
the entry (D5 reserved); unset = open loopback route. Upstream credentials stay
runtime-only and are stripped from echoed responses.

Brings in the provider-adapters baseline from sibling branch (RIC-737) that
this entry depends on. Includes 21 passing unit tests (provider selection,
model-prefix forwarding, header hygiene, auth, error normalization, SSE
passthrough, fall-through/misconfig), docs, env example, changelog fragment.

* feat(routing): deterministic routing strategies for self-hosted entry (RIC-740)

Add the M2 deterministic routing strategy engine (D3 可审计路由) to the
self-hosted unified entry: a declarative `strategy:` block expressing five
explainable, non-predictive policies — blacklist/whitelist hard filters,
cooldown circuit breaker, cost-priority, latency-aware ordering, and an
explicit fallback chain. The ordered candidate list is the fallback chain:
a failed primary (network or non-2xx) falls through to the next candidate and
each failure feeds the breaker. Every response carries an
x-omniroute-route-decision header answering "why this model / why not that
one". A pinned provider rejected by a hard filter returns 400 (never a silent
re-route); no eligible providers returns 503 with the full explainable
decision. No ML/predict dependency.

Covers the RIC-740 acceptance: 5 strategy types with unit tests + HTTP
fault-injection tests (primary down -> fallback works), config matching docs,
and no predict/ML deps. Adds docs, .env.example entries, and a changelog
fragment.

* refactor(routing): reduce complexity-ratchet violations in new self-hosted routing files

Extract cost/id validation, pin-blocked resolution, ordering, and env/file
source resolution into small helpers so routingStrategies.ts and
selfHostedEntry.ts stay under the complexity-ratchets cap. No behavior
change — the same 51 routing-strategies/self-hosted-entry/provider-adapters
tests pass unmodified.

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>

* docs(routing): document the 5 self-hosted env vars in ENVIRONMENT.md

check:env-doc-sync failed because OMNIROUTE_SELF_HOSTED_PROVIDERS(_FILE),
OMNIROUTE_SELF_HOSTED_API_KEY and OMNIROUTE_SELF_HOSTED_STRATEGY(_FILE)
were present in .env.example but missing from
docs/reference/ENVIRONMENT.md. Add them under "6. Tool & Routing Policies".

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>

---------

Co-authored-by: Ant Rich <ant@richants.com>
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
Co-authored-by: luyuehm <luyuehm@users.noreply.github.com>
This commit is contained in:
luyuehm
2026-09-18 22:59:15 +08:00
committed by GitHub
parent 3ebea07278
commit 5a82da7084
13 changed files with 2282 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

11
src/app/api/v1/chat/completions/route.ts
View File

@@ -5,6 +5,7 @@ import { handleChat } from "@/sse/handlers/chat";
import { generateRequestId } from "@/shared/utils/requestId";
import { resolveIncomingCorrelationId } from "@/shared/utils/correlationPreserve.ts";
import { errorResponse } from "@omniroute/open-sse/utils/error.ts";
import { handleSelfHostedCompletions } from "@omniroute/open-sse/services/selfHostedEntry.ts";
import { initTranslators } from "@omniroute/open-sse/translator/index.ts";
import { createInjectionGuard } from "@/middleware/promptInjectionGuard";
import { acceptHeaderForcesStream } from "@omniroute/open-sse/utils/aiSdkCompat.ts";
@@ -158,6 +159,16 @@ export async function POST(request) {
);
}
// Self-hosted unified entry (D4 — RIC-738): when a provider config is
// present, divert BEFORE the cloud-only model retirement/alias checks so
// self-hosted model ids (`local/llama3`, `ollama/qwen2`, ...) never trip
// cloud-peer 410s or alias rewrites. Config-absent requests proceed to the
// normal cloud pipeline unchanged.
const selfHostedResponse = await handleSelfHostedCompletions(request, parsedBody);
if (selfHostedResponse) {
return finishAdmission(selfHostedResponse);
}
try {
assertCommonChatGptWebModelAvailable(parsedBody.model);
} catch (error) {