diff --git a/.env.example b/.env.example index 1f292cf0de..6609161e86 100644 --- a/.env.example +++ b/.env.example @@ -1784,7 +1784,7 @@ QUOTA_STORE_DRIVER=sqlite # sqlite | redis # Per-request timeout when proxying to the Bifrost gateway. Default: 30000 (30s). # BIFROST_TIMEOUT_MS= # Alias for BIFROST_API_KEY (used by scripts that read the env via -# OMNIROUTE_*). Falls back to BIFROST_API_KEY when unset. +# OMNIROUTE_*). BIFROST_API_KEY takes precedence when both are set. # OMNIROUTE_BIFROST_KEY= # Relay backend selection for the OpenAI-compatible relay endpoint: # ts | bifrost | auto. "ts" (default when Bifrost is not configured) uses the @@ -1850,6 +1850,8 @@ QUOTA_STORE_DRIVER=sqlite # sqlite | redis # gateway as a separate sidecar (helps in 3+ replica deployments where you want # provider rotation centralised). See docs/architecture/cluster-decisions.md § # "Bifrost (bifrost profile)". +# Set OMNIROUTE_RELAY_BACKEND=auto to use this sidecar when healthy, or +# OMNIROUTE_RELAY_BACKEND=bifrost to require it without TS fallback. # BIFROST_BASE_URL=http://bifrost:8080 # BIFROST_API_KEY= # BIFROST_STREAMING_ENABLED=true diff --git a/docs/reference/ENVIRONMENT.md b/docs/reference/ENVIRONMENT.md index 8c2f7043e9..ca165776e2 100644 --- a/docs/reference/ENVIRONMENT.md +++ b/docs/reference/ENVIRONMENT.md @@ -1004,7 +1004,7 @@ Provider quota endpoints, network tunnels (Tailscale, Ngrok, MITM debug proxy), | `BIFROST_API_KEY` | _(unset)_ | `src/app/api/v1/relay/chat/completions/bifrost/route.ts` | API key for the Bifrost gateway (sent as `Authorization: Bearer ...`). If unset, the route expects the request to carry a valid OmniRoute API key; this key is for gateway-side auth only. | | `BIFROST_STREAMING_ENABLED` | `true` | `src/app/api/v1/relay/chat/completions/bifrost/route.ts` | When true, the Bifrost sidecar route streams responses back via SSE through the gateway rather than the TS streaming executor. Set to `0` to force non-streaming JSON responses through the gateway. | | `BIFROST_TIMEOUT_MS` | `30000` | `src/app/api/v1/relay/chat/completions/bifrost/route.ts` | Per-request timeout when proxying to the Bifrost gateway (ms). On timeout the route returns the TS relay path via the `X-Bifrost-Fallback` header. | -| `OMNIROUTE_BIFROST_KEY` | _(unset)_ | `src/app/api/v1/relay/chat/completions/bifrost/route.ts` | Alias for `BIFROST_API_KEY` (used by scripts that read the env via `OMNIROUTE_*`). Falls back to `BIFROST_API_KEY` when unset. | +| `OMNIROUTE_BIFROST_KEY` | _(unset)_ | `src/app/api/v1/relay/chat/completions/bifrost/route.ts` | Alias for `BIFROST_API_KEY` (used by scripts that read the env via `OMNIROUTE_*`). `BIFROST_API_KEY` takes precedence when both are set. | | `OMNIROUTE_RELAY_BACKEND` | `ts` / `auto` | `src/app/api/v1/relay/chat/completions/routingBackend.ts` | Relay backend for `/api/v1/relay/chat/completions`: `ts \| bifrost \| auto`. `ts` = TypeScript relay (default when Bifrost unconfigured); `auto` selects Bifrost when `BIFROST_BASE_URL` is set and `BIFROST_ENABLED` ≠ `0`, with automatic TS fallback if the sidecar is unreachable; `bifrost` forces Bifrost (strict, no fallback). Auth/rate-limit/injection-guard/allowlist always run in the Next route first. Responses carry `X-Routing-Backend` / `X-Routing-Fallback`. | | `RELAY_ROUTING_BACKEND` | _(unset)_ | `src/app/api/v1/relay/chat/completions/routingBackend.ts` | Accepted alias for `OMNIROUTE_RELAY_BACKEND` (same `ts \| bifrost \| auto` values). `OMNIROUTE_RELAY_BACKEND` takes precedence when both are set. | | `OMNIROUTE_TLS_CERT` | _(unset)_ | `bin/cli/commands/serve.mjs` | Path to a PEM TLS certificate to serve `omniroute serve` over HTTPS (equivalent to `--tls-cert`). Must be paired with `OMNIROUTE_TLS_KEY`; the standalone server then terminates TLS on the same listener (`wss://` works unchanged). Unset → plain HTTP. Providing only one of cert/key, or an unreadable path, logs a warning and stays HTTP. |