mirror of
https://github.com/diegosouzapw/OmniRoute.git
synced 2026-08-18 21:22:28 +03:00
e3bca29bbc
* fix(docker): real image tags + complete OMNIROUTE_BASE_PATH runtime patcher
Three docker issues fixed:
1. Images that do not exist:
- bifrost: ghcr.io/maximhq/bifrost:1.5.21 never existed (1.5.x tops at
v1.5.16, all tags carry the v prefix) -> ghcr.io/maximhq/bifrost:v1.6.11
- cliproxyapi: ghcr.io/router-for-me/* is not publicly pullable (403);
the official prebuilt image is docker.io/eceasy/cli-proxy-api, where
the pinned v6.9.7 exists -> docker.io/eceasy/cli-proxy-api:v6.9.7
- Verified still-current: redis:8.6.5-alpine (already on Redis 8 since
#9065; ioredis 5.10 is RESP2/3-compatible, no modules used) and
qdrant:v1.12.4 -- both exist, unchanged.
2. OMNIROUTE_BASE_PATH ignored on prebuilt images (root cause):
Next 16 (webpack and Turbopack) app-router renders SSR asset URLs from
assetPrefix ALONE; basePath only affects routing. The runtime patcher
(ensure-docker-base-path) rewrote basePath literals only, so a prebuilt
root-path image patched to /omniroute served the page but every
/_next/static shell reference stayed unprefixed (404 behind a subpath
proxy), the RSC flight-payload chunk refs came from client-reference
manifests baked with unprefixed paths, and the Turbopack client process
shim ships an empty env object so the client never learns the subpath.
Extended patch-standalone-base-path.mjs to also rewrite:
- assetPrefix literals (mirrors the subpath for SSR asset URLs)
- the NEXT_PUBLIC_OMNIROUTE_BASE_PATH env mirror in the inline config
- the client process.env shim (.env={}) with the two basePath keys
- every baked "/_next/static URL (manifests, media imports, .html pages)
next.config.mjs now mirrors basePath into assetPrefix so REBUILT images
bake prefixed assets too. E2E-verified on the published main-web image:
HTML under /omniroute now has 16/16 prefixed JS srcs and 82/82 prefixed
flight refs (was 13/9 + ~150 unprefixed), prefixed assets return 200.
* chore(changelog): fragment for #10482 (docker images + basepath patcher)
* chore(changelog): bullet-form fragment for #10482
* Merge branch 'release/v3.8.50' into fix/docker-compose-images-and-basepath
* test(fix): refresh expired alibaba quota sample validity and onnxruntime pin for v3.8.50 base
- alibaba-free-tier-quota-fetcher.test.ts: sample quotaValidityPeriod
(2026-08-16 16:00 UTC) is in the past, making every quota entry classify
as expired/not_capable; bump to 2028-01-01 UTC so the text/merge
classification tests exercise the intended path again.
- optional-transformers-dependency.test.ts: onnxruntime-node pin assertion
updated from ~1.24.3 to ~1.27.0 to match package.json (bumped by #10403);
the regular-not-optional intent is unchanged.
---------
Co-authored-by: Rouzbeh <rqzbeh@users.noreply.github.com>
102 lines
8.6 KiB
Markdown
102 lines
8.6 KiB
Markdown
---
|
||
title: "Cluster Decisions"
|
||
version: 3.8.40
|
||
lastUpdated: 2026-06-28
|
||
---
|
||
|
||
# Cluster Decisions — Optional Sidecar Profiles
|
||
|
||
**Status:** proposal (awaiting @diegosouzapw review)
|
||
**Date:** 2026-06-20
|
||
**Refs:** [#3932](https://github.com/diegosouzapw/OmniRoute/issues/3932), PR #4381
|
||
|
||
## TL;DR
|
||
|
||
Two opt-in compose profiles (`memory`, `bifrost`) for the existing 8-service deployment in [`docker-compose.yml`](../../docker-compose.yml). Default-up behaviour is **unchanged**: 3 × `omniroute` replicas + Caddy + Redis + CliproxyAPI. The two new profiles add Qdrant and Bifrost as optional sidecars, gated by `docker compose --profile <name> up`. **No existing service is removed or replaced.**
|
||
|
||
## Why this is conservative
|
||
|
||
OmniRoute's existing deployment shape is already lean and proven:
|
||
|
||
- **`redis:7-alpine`** handles the rate-limit/cache workload at production scale.
|
||
- **SQLite + sqlite-vec + FTS5** cover local memory + vector + text-search (see [`src/lib/memory/vectorStore.ts:108`](../../src/lib/memory/vectorStore.ts)).
|
||
- **Caddy** is already the LB + TLS terminator ([`docker-compose.yml`](../../docker-compose.yml)).
|
||
- **Bifrost** is already integrated as the Tier-1 router in [`src/app/api/v1/relay/chat/completions/bifrost/route.ts`](../../src/app/api/v1/relay/chat/completions/bifrost/route.ts) (sidecar proxy with kill switch via `BIFROST_ENABLED` env var — set `=0` to bypass the sidecar and fall through to the TS path).
|
||
|
||
The two profiles here are **scale-out options for deployments that hit the SQLite ceiling** — not migrations. Both are default-off.
|
||
|
||
## The two profiles
|
||
|
||
### `memory` — Qdrant Vector Memory Sidecar
|
||
|
||
**When to flip on:**
|
||
|
||
- > 1M embeddings per deployment (sqlite-vec starts to slow at scale).
|
||
- Multi-replica deployment that needs shared vector state across `omniroute-1/2/3`.
|
||
- You already have an external Qdrant cluster (Qdrant Cloud, on-prem).
|
||
|
||
**What it adds:**
|
||
|
||
| Service | Image | Ports | Notes |
|
||
| -------- | ----------------------- | ----------- | ----------------------------------------------------- |
|
||
| `qdrant` | `qdrant/qdrant:v1.12.4` | `6333` HTTP | HNSW index; persistent volume `omniroute_qdrant_data` |
|
||
|
||
**Activation:** flip `qdrantEnabled = true` in the Settings UI **or** set `QDRANT_HOST=qdrant` env. See [`src/lib/memory/qdrant.ts:60`](../../src/lib/memory/qdrant.ts) for the precedence rules (settings table → env var → default).
|
||
|
||
**Env vars:** `QDRANT_HOST`, `QDRANT_PORT`, `QDRANT_API_KEY`, `QDRANT_COLLECTION`, `QDRANT_VECTOR_SIZE`, `QDRANT_HNSW_EF_CONSTRUCT` (see `.env.example` lines 1672-1683).
|
||
|
||
### `bifrost` — Bifrost Tier-1 Router Sidecar
|
||
|
||
**When to flip on:**
|
||
|
||
- You run ≥3 `omniroute` replicas and want provider rotation centralised in a single Go process.
|
||
- You want a single audit/logging surface for upstream-provider requests across all replicas.
|
||
- You want horizontal scaling of the Tier-1 routing layer independent of the OmniRoute replicas.
|
||
|
||
**What it adds:**
|
||
|
||
| Service | Image | Ports | Notes |
|
||
| --------- | --------------------------------- | ------ | ----------------------------------------------------------------------- |
|
||
| `bifrost` | `ghcr.io/maximhq/bifrost:v1.6.11` | `8080` | Go-based Tier-1 router; persistent logs volume `omniroute_bifrost_logs` |
|
||
|
||
**Activation:** set `BIFROST_BASE_URL=http://bifrost:8080` in `.env.example`. The existing sidecar proxy route at [`src/app/api/v1/relay/chat/completions/bifrost/route.ts`](../../src/app/api/v1/relay/chat/completions/bifrost/route.ts) (added in PR #4381) will pick this up automatically.
|
||
|
||
**Env vars:** `BIFROST_BASE_URL`, `BIFROST_API_KEY`, `BIFROST_STREAMING_ENABLED`, `BIFROST_TIMEOUT_MS` (see `.env.example` lines 1685-1695).
|
||
|
||
## What this PR explicitly does NOT do
|
||
|
||
The original issue thread floated a larger cluster rewrite. After auditing the actual workload shape, the following are **rejected** for the reasons given:
|
||
|
||
| Component | Verdict | Reason |
|
||
| ------------------------------------ | -------- | ------------------------------------------------------------------------------------------------------ |
|
||
| **Dragonfly** | **DROP** | `redis:7-alpine` is already fine for the rate-limit workload at production scale; no ceiling to break. |
|
||
| **NATS** | **DROP** | Each `omniroute` replica is a single Node.js process; no multi-process pub/sub workload exists. |
|
||
| **PostgreSQL** | **DROP** | SQLite + sqlite-vec + FTS5 cover all 3 use cases; 97 migrations + Electron packaging block migration. |
|
||
| **Neo4j** | **DROP** | Routing is a 5-table join; recursive CTE on SQLite is sufficient. |
|
||
| **MinIO** | **DROP** | No multi-MB blob workload; images/audio are passthrough proxies. |
|
||
| **pgvector / pg_ai / pg_textsearch** | **DROP** | Same SQLite-ceiling reason as PostgreSQL; pgvector ecosystem fragmented. |
|
||
| **HAProxy / Envoy** | **DROP** | Caddy already does LB + TLS; both were explicitly rejected as Tier-1 routers (see `AGENTS.md`). |
|
||
|
||
If a future use case proves out one of these, this doc is the place to amend.
|
||
|
||
## 4-week rollout (if approved)
|
||
|
||
1. **Wk 1** — Land this PR + verification of opt-in profiles with a 3-replica compose stack.
|
||
2. **Wk 2** — Bifrost full activation for OpenAI/Claude/Gemini/Ollama (4 of 14+ providers) using the sidecar proxy route at [`src/app/api/v1/relay/chat/completions/bifrost/route.ts`](../../src/app/api/v1/relay/chat/completions/bifrost/route.ts) (gated by `BIFROST_ENABLED`, kill-switchable at runtime).
|
||
3. **Wk 3** — Qdrant memory profile enabled in a single test deployment; measure latency delta vs sqlite-vec.
|
||
4. **Wk 4** — Observability healthchecks (`docker compose ps` exit codes + `wget` smoke tests); 71-pillar refresh per ADR-041.
|
||
|
||
## Files changed in this PR
|
||
|
||
| File | Change |
|
||
| ---------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||
| `docker-compose.yml` | +30 lines: `memory` profile (Qdrant), `bifrost` profile (Bifrost), persistent volumes, healthchecks. |
|
||
| `.env.example` | +24 lines: `QDRANT_*` (6 vars), `BIFROST_*` (4 vars). |
|
||
| `docs/reference/ENVIRONMENT.md` | +6 rows in section 25 for the `QDRANT_*` env vars. |
|
||
| `src/lib/memory/qdrant.ts` | +33 lines: env-var fallback chain (settings → env → default) for `QDRANT_HOST`/`QDRANT_PORT`/`QDRANT_API_KEY`/`QDRANT_COLLECTION`/`QDRANT_VECTOR_SIZE`/`QDRANT_HNSW_EF_CONSTRUCT`/`QDRANT_EMBEDDING_MODEL`. |
|
||
| `src/lib/memory/__tests__/qdrant-wiring.test.ts` | +88 lines: 9 new test cases pinning the env-var fallback precedence. |
|
||
| `docs/architecture/cluster-decisions.md` (this file) | NEW — decision record for the opt-in profiles. |
|
||
| `AGENTS.md` | +1 line: pointer to this doc in the reference documentation table. |
|
||
|
||
**Net touched code:** 4 production files (`docker-compose.yml`, `qdrant.ts`, `.env.example`, `ENVIRONMENT.md`), 1 test file (`qdrant-wiring.test.ts`), 2 doc files (`cluster-decisions.md`, `AGENTS.md`).
|