mirror of https://github.com/diegosouzapw/OmniRoute.git synced 2026-08-18 21:22:28 +03:00

Files

Rouzbeh† e3bca29bbc fix(docker): real image tags (bifrost/cliproxyapi) + complete OMNIROUTE_BASE_PATH runtime patcher (#10482 )

* fix(docker): real image tags + complete OMNIROUTE_BASE_PATH runtime patcher

Three docker issues fixed:

1. Images that do not exist:
   - bifrost: ghcr.io/maximhq/bifrost:1.5.21 never existed (1.5.x tops at
     v1.5.16, all tags carry the v prefix) -> ghcr.io/maximhq/bifrost:v1.6.11
   - cliproxyapi: ghcr.io/router-for-me/* is not publicly pullable (403);
     the official prebuilt image is docker.io/eceasy/cli-proxy-api, where
     the pinned v6.9.7 exists -> docker.io/eceasy/cli-proxy-api:v6.9.7
   - Verified still-current: redis:8.6.5-alpine (already on Redis 8 since
     #9065; ioredis 5.10 is RESP2/3-compatible, no modules used) and
     qdrant:v1.12.4 -- both exist, unchanged.

2. OMNIROUTE_BASE_PATH ignored on prebuilt images (root cause):
   Next 16 (webpack and Turbopack) app-router renders SSR asset URLs from
   assetPrefix ALONE; basePath only affects routing. The runtime patcher
   (ensure-docker-base-path) rewrote basePath literals only, so a prebuilt
   root-path image patched to /omniroute served the page but every
   /_next/static shell reference stayed unprefixed (404 behind a subpath
   proxy), the RSC flight-payload chunk refs came from client-reference
   manifests baked with unprefixed paths, and the Turbopack client process
   shim ships an empty env object so the client never learns the subpath.
   Extended patch-standalone-base-path.mjs to also rewrite:
   - assetPrefix literals (mirrors the subpath for SSR asset URLs)
   - the NEXT_PUBLIC_OMNIROUTE_BASE_PATH env mirror in the inline config
   - the client process.env shim (.env={}) with the two basePath keys
   - every baked "/_next/static URL (manifests, media imports, .html pages)
   next.config.mjs now mirrors basePath into assetPrefix so REBUILT images
   bake prefixed assets too. E2E-verified on the published main-web image:
   HTML under /omniroute now has 16/16 prefixed JS srcs and 82/82 prefixed
   flight refs (was 13/9 + ~150 unprefixed), prefixed assets return 200.

* chore(changelog): fragment for #10482 (docker images + basepath patcher)

* chore(changelog): bullet-form fragment for #10482

* Merge branch 'release/v3.8.50' into fix/docker-compose-images-and-basepath

* test(fix): refresh expired alibaba quota sample validity and onnxruntime pin for v3.8.50 base

- alibaba-free-tier-quota-fetcher.test.ts: sample quotaValidityPeriod
  (2026-08-16 16:00 UTC) is in the past, making every quota entry classify
  as expired/not_capable; bump to 2028-01-01 UTC so the text/merge
  classification tests exercise the intended path again.
- optional-transformers-dependency.test.ts: onnxruntime-node pin assertion
  updated from ~1.24.3 to ~1.27.0 to match package.json (bumped by #10403);
  the regular-not-optional intent is unchanged.

---------

Co-authored-by: Rouzbeh <rqzbeh@users.noreply.github.com>

2026-08-17 08:24:27 -03:00

8.6 KiB

Raw Blame History

title, version, lastUpdated

title	version	lastUpdated
Cluster Decisions	3.8.40	2026-06-28

Cluster Decisions — Optional Sidecar Profiles

Status: proposal (awaiting @diegosouzapw review) Date: 2026-06-20 Refs: #3932, PR #4381

TL;DR

Two opt-in compose profiles (memory, bifrost) for the existing 8-service deployment in docker-compose.yml. Default-up behaviour is unchanged: 3 × omniroute replicas + Caddy + Redis + CliproxyAPI. The two new profiles add Qdrant and Bifrost as optional sidecars, gated by docker compose --profile <name> up. No existing service is removed or replaced.

Why this is conservative

OmniRoute's existing deployment shape is already lean and proven:

redis:7-alpine handles the rate-limit/cache workload at production scale.
SQLite + sqlite-vec + FTS5 cover local memory + vector + text-search (see src/lib/memory/vectorStore.ts:108).
Caddy is already the LB + TLS terminator (docker-compose.yml).
Bifrost is already integrated as the Tier-1 router in src/app/api/v1/relay/chat/completions/bifrost/route.ts (sidecar proxy with kill switch via BIFROST_ENABLED env var — set =0 to bypass the sidecar and fall through to the TS path).

The two profiles here are scale-out options for deployments that hit the SQLite ceiling — not migrations. Both are default-off.

The two profiles

`memory` — Qdrant Vector Memory Sidecar

When to flip on:

1M embeddings per deployment (sqlite-vec starts to slow at scale).
Multi-replica deployment that needs shared vector state across omniroute-1/2/3.
You already have an external Qdrant cluster (Qdrant Cloud, on-prem).

What it adds:

Service	Image	Ports	Notes
`qdrant`	`qdrant/qdrant:v1.12.4`	`6333` HTTP	HNSW index; persistent volume `omniroute_qdrant_data`

Activation: flip qdrantEnabled = true in the Settings UI or set QDRANT_HOST=qdrant env. See src/lib/memory/qdrant.ts:60 for the precedence rules (settings table → env var → default).

Env vars: QDRANT_HOST, QDRANT_PORT, QDRANT_API_KEY, QDRANT_COLLECTION, QDRANT_VECTOR_SIZE, QDRANT_HNSW_EF_CONSTRUCT (see .env.example lines 1672-1683).

`bifrost` — Bifrost Tier-1 Router Sidecar

When to flip on:

You run ≥3 omniroute replicas and want provider rotation centralised in a single Go process.
You want a single audit/logging surface for upstream-provider requests across all replicas.
You want horizontal scaling of the Tier-1 routing layer independent of the OmniRoute replicas.

What it adds:

Service	Image	Ports	Notes
`bifrost`	`ghcr.io/maximhq/bifrost:v1.6.11`	`8080`	Go-based Tier-1 router; persistent logs volume `omniroute_bifrost_logs`

Activation: set BIFROST_BASE_URL=http://bifrost:8080 in .env.example. The existing sidecar proxy route at src/app/api/v1/relay/chat/completions/bifrost/route.ts (added in PR #4381) will pick this up automatically.

Env vars: BIFROST_BASE_URL, BIFROST_API_KEY, BIFROST_STREAMING_ENABLED, BIFROST_TIMEOUT_MS (see .env.example lines 1685-1695).

What this PR explicitly does NOT do

The original issue thread floated a larger cluster rewrite. After auditing the actual workload shape, the following are rejected for the reasons given:

Component	Verdict	Reason
Dragonfly	DROP	`redis:7-alpine` is already fine for the rate-limit workload at production scale; no ceiling to break.
NATS	DROP	Each `omniroute` replica is a single Node.js process; no multi-process pub/sub workload exists.
PostgreSQL	DROP	SQLite + sqlite-vec + FTS5 cover all 3 use cases; 97 migrations + Electron packaging block migration.
Neo4j	DROP	Routing is a 5-table join; recursive CTE on SQLite is sufficient.
MinIO	DROP	No multi-MB blob workload; images/audio are passthrough proxies.
pgvector / pg_ai / pg_textsearch	DROP	Same SQLite-ceiling reason as PostgreSQL; pgvector ecosystem fragmented.
HAProxy / Envoy	DROP	Caddy already does LB + TLS; both were explicitly rejected as Tier-1 routers (see `AGENTS.md`).

If a future use case proves out one of these, this doc is the place to amend.

4-week rollout (if approved)

Wk 1 — Land this PR + verification of opt-in profiles with a 3-replica compose stack.
Wk 2 — Bifrost full activation for OpenAI/Claude/Gemini/Ollama (4 of 14+ providers) using the sidecar proxy route at src/app/api/v1/relay/chat/completions/bifrost/route.ts (gated by BIFROST_ENABLED, kill-switchable at runtime).
Wk 3 — Qdrant memory profile enabled in a single test deployment; measure latency delta vs sqlite-vec.
Wk 4 — Observability healthchecks (docker compose ps exit codes + wget smoke tests); 71-pillar refresh per ADR-041.

Files changed in this PR

File	Change
`docker-compose.yml`	+30 lines: `memory` profile (Qdrant), `bifrost` profile (Bifrost), persistent volumes, healthchecks.
`.env.example`	+24 lines: `QDRANT_` (6 vars), `BIFROST_` (4 vars).
`docs/reference/ENVIRONMENT.md`	+6 rows in section 25 for the `QDRANT_*` env vars.
`src/lib/memory/qdrant.ts`	+33 lines: env-var fallback chain (settings → env → default) for `QDRANT_HOST`/`QDRANT_PORT`/`QDRANT_API_KEY`/`QDRANT_COLLECTION`/`QDRANT_VECTOR_SIZE`/`QDRANT_HNSW_EF_CONSTRUCT`/`QDRANT_EMBEDDING_MODEL`.
`src/lib/memory/__tests__/qdrant-wiring.test.ts`	+88 lines: 9 new test cases pinning the env-var fallback precedence.
`docs/architecture/cluster-decisions.md` (this file)	NEW — decision record for the opt-in profiles.
`AGENTS.md`	+1 line: pointer to this doc in the reference documentation table.

Net touched code: 4 production files (docker-compose.yml, qdrant.ts, .env.example, ENVIRONMENT.md), 1 test file (qdrant-wiring.test.ts), 2 doc files (cluster-decisions.md, AGENTS.md).

8.6 KiB Raw Blame History Unescape Escape