Compare commits
1 Commits
security/v
...
docs/dedup
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
18fceaf8d4 |
@@ -46,7 +46,7 @@ Repository map and Reference Documentation sections below.
|
||||
|
||||
## Project at a Glance
|
||||
|
||||
**OmniRoute** — unified AI proxy/router. One endpoint, 357 LLM providers, auto-fallback.
|
||||
**OmniRoute** — unified AI proxy/router. One endpoint, 356 LLM providers, auto-fallback.
|
||||
|
||||
| Layer | Location | Purpose |
|
||||
| ------------- | ----------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
|
||||
36
Dockerfile
@@ -184,29 +184,19 @@ ENV NODE_OPTIONS="--max-old-space-size=${OMNIROUTE_BUILD_MEMORY_MB}"
|
||||
# silently leaving no standalone bundle. Next derives the worker count from
|
||||
# CIRCLE_NODE_TOTAL (workers = N-1). (#10060)
|
||||
#
|
||||
# Lowered 8 → 3 (7 workers → 2) in #11419, then 3 → 2 (2 workers → 1) in #7518.
|
||||
# Every page-data worker inherits NODE_OPTIONS above, so the ceiling is per
|
||||
# PROCESS, not per build: 7 workers on a 16 GB GitHub runner (ubuntu-24.04 /
|
||||
# ubuntu-24.04-arm, 4 vCPU) exhausted the host and buildkit failed the whole
|
||||
# step with `ResourceExhausted: ... cannot allocate memory`. The compile phase
|
||||
# always finished ("✓ Compiled successfully in 4.2min"); the kernel killed the
|
||||
# build right after "Collecting page data using N workers".
|
||||
#
|
||||
# #11419's first fix (8 → 3) modeled the per-worker peak as an INFERENCE
|
||||
# (2560 MB, guessed from "7 workers didn't fit") and assumed the parent
|
||||
# process's RSS tracked the V8 heap ceiling. Both assumptions were wrong: a
|
||||
# live VPS reproduction (issue #7518, dmesg OOM-killer report) measured the
|
||||
# real per-process RSS directly at ~4.5 GB, independent of the NODE_OPTIONS
|
||||
# heap flag (Turbopack itself is native/Rust, outside the V8 heap) — and it
|
||||
# applies to the parent process too, not just workers. 2 workers (3 processes
|
||||
# × 4.5 GB = 13.5 GB) still didn't fit the 12.288 GB (75%) budget on a 16 GB
|
||||
# runner, matching the still-live publish failures after #11419 merged. 1
|
||||
# worker (2 processes × 4.5 GB = 9 GB) fits with headroom to spare.
|
||||
# tests/unit/docker-build-memory-budget.test.ts does the arithmetic against
|
||||
# the measured figure and fails if either knob is raised past what a 16 GB
|
||||
# runner holds. Override for a big builder: `--build-arg
|
||||
# OMNIROUTE_BUILD_WORKERS=8`.
|
||||
ARG OMNIROUTE_BUILD_WORKERS=2
|
||||
# Lowered 8 → 3 (7 workers → 2). Every page-data worker inherits NODE_OPTIONS
|
||||
# above, so the ceiling is per PROCESS, not per build: 7 workers on a 16 GB
|
||||
# GitHub runner (ubuntu-24.04 / ubuntu-24.04-arm, 4 vCPU) exhausted the host and
|
||||
# buildkit failed the whole step with `ResourceExhausted: ... cannot allocate
|
||||
# memory`. The compile phase always finished ("✓ Compiled successfully in
|
||||
# 4.2min"); the kernel killed the build right after "Collecting page data using
|
||||
# 7 workers". It was intermittent for a while and went 100% on 2026-08-22, which
|
||||
# is what a threshold being crossed by ordinary codebase growth looks like.
|
||||
# tests/unit/docker-build-memory-budget.test.ts does the arithmetic and fails if
|
||||
# either knob is raised past what a 16 GB runner holds. 2 workers also stops
|
||||
# oversubscribing the runner's 4 vCPU, which 7 did. Override for a big builder:
|
||||
# `--build-arg OMNIROUTE_BUILD_WORKERS=8`.
|
||||
ARG OMNIROUTE_BUILD_WORKERS=3
|
||||
ENV CIRCLE_NODE_TOTAL=${OMNIROUTE_BUILD_WORKERS}
|
||||
|
||||
COPY . ./
|
||||
|
||||
10
README.md
@@ -7,7 +7,7 @@
|
||||
|
||||
# 🚀 OmniRoute — The Free AI Gateway
|
||||
|
||||
<img src="./docs/diagrams/readme-hero.svg" width="100%" alt="OmniRoute — Never stop coding. Every AI tool → 357 providers — 90+ free — through one endpoint. Claude Code, Codex, Cursor, Cline, Copilot & Antigravity into FREE Claude / GPT / Gemini with auto-fallback. RTK + Caveman stacked compression saves 15–95% tokens (~89% avg) — never hit limits. 357 AI providers · 90+ free tiers · ~1.51B free tokens/mo · 19 routing strategies · $0 to start."/>
|
||||
<img src="./docs/diagrams/readme-hero.svg" width="100%" alt="OmniRoute — Never stop coding. Every AI tool → 356 providers — 90+ free — through one endpoint. Claude Code, Codex, Cursor, Cline, Copilot & Antigravity into FREE Claude / GPT / Gemini with auto-fallback. RTK + Caveman stacked compression saves 15–95% tokens (~89% avg) — never hit limits. 356 AI providers · 90+ free tiers · ~1.51B free tokens/mo · 19 routing strategies · $0 to start."/>
|
||||
|
||||
</div>
|
||||
|
||||
@@ -210,7 +210,7 @@ curl http://localhost:20128/v1/chat/completions \
|
||||
|
||||
</div>
|
||||
|
||||
<img src="./docs/diagrams/promise-pillars.svg" width="100%" alt="The Promise — One endpoint and 357 providers. Automatic fallback keeps routing while another healthy target is available. Six pillars: resilient fallback across 357 providers · up to 95% token savings on eligible workloads · $0 to start with 90+ free tiers and 56 recurring/keyless free-forever providers · 35 CLI/agent integrations through one config · OpenAI, Claude, Gemini and Responses API compatibility at /v1 · production controls including circuit breakers, TLS stealth, MCP 110 tools, A2A, memory, guardrails, evals and 39,000+ static test declarations across 5,100+ tracked test files."/>
|
||||
<img src="./docs/diagrams/promise-pillars.svg" width="100%" alt="The Promise — One endpoint and 356 providers. Automatic fallback keeps routing while another healthy target is available. Six pillars: resilient fallback across 356 providers · up to 95% token savings on eligible workloads · $0 to start with 90+ free tiers and 56 recurring/keyless free-forever providers · 35 CLI/agent integrations through one config · OpenAI, Claude, Gemini and Responses API compatibility at /v1 · production controls including circuit breakers, TLS stealth, MCP 110 tools, A2A, memory, guardrails, evals and 39,000+ static test declarations across 5,100+ tracked test files."/>
|
||||
|
||||
<br/>
|
||||
<br/>
|
||||
@@ -461,7 +461,7 @@ All **19** strategies — mix & match per combo step:
|
||||
|
||||
</div>
|
||||
|
||||
<img src="./docs/diagrams/comparison-table.svg" width="100%" alt="What sets OmniRoute apart — a dated feature snapshot vs 9router, OpenRouter, CLIProxyAPI and LiteLLM across 13 capabilities. OmniRoute: 357 providers, 90+ free tiers built in, 19 routing strategies, 12-engine token compression, built-in MCP server with 110 tools, A2A agent protocol, persistent memory, guardrails, cloud agents, TLS fingerprint stealth, Desktop/Termux/PWA and 43 i18n UI locales. OmniRoute is MIT-licensed and self-hostable. Competitor capabilities and counts may change; see the linked methodology."/>
|
||||
<img src="./docs/diagrams/comparison-table.svg" width="100%" alt="What sets OmniRoute apart — a dated feature snapshot vs 9router, OpenRouter, CLIProxyAPI and LiteLLM across 13 capabilities. OmniRoute: 356 providers, 90+ free tiers built in, 19 routing strategies, 12-engine token compression, built-in MCP server with 110 tools, A2A agent protocol, persistent memory, guardrails, cloud agents, TLS fingerprint stealth, Desktop/Termux/PWA and 43 i18n UI locales. OmniRoute is MIT-licensed and self-hostable. Competitor capabilities and counts may change; see the linked methodology."/>
|
||||
|
||||
<sub>📊 Full methodology & per-feature detail vs 9router, OpenRouter, CLIProxyAPI & LiteLLM → [`docs/comparison/OMNIROUTE_VS_ALTERNATIVES.md`](docs/comparison/OMNIROUTE_VS_ALTERNATIVES.md)</sub>
|
||||
|
||||
@@ -559,7 +559,7 @@ the current catalog at **[radar.omniroute.online/planos](https://radar.omniroute
|
||||
- **🖼️ New endpoints** — `/v1/ocr` (Mistral OCR) and `/v1/audio/translations` (Whisper-style) round out the media surface. → [API Reference](docs/reference/API_REFERENCE.md)
|
||||
- **🎨 Image / video / audio generation** — one API for media: xAI Grok Imagine & Novita AI video, ComfyUI, Freepik, Adobe Firefly, Microsoft Designer, Segmind, EdgeTTS. → [API Reference](docs/reference/API_REFERENCE.md)
|
||||
- **🌍 Deployment & ops** — reverse-proxy `basePath`, browser-language auto-detect, per-key device tracking, root-less MITM trust, zh-TW localization. → [Environment](docs/reference/ENVIRONMENT.md)
|
||||
- **🤝 More providers & agents** — Cursor Cloud Agent, Grok Build (xAI) with browser + OAuth login, Ollama first-class card, Claude Opus 5 & Sonnet 5, Kimi official partnership (Code/Web/Moonshot), Zed, Requesty, SenseNova, Yuanbao, Agnes AI… and a refreshed **357-provider catalog**. → [Providers](docs/reference/PROVIDER_REFERENCE.md)
|
||||
- **🤝 More providers & agents** — Cursor Cloud Agent, Grok Build (xAI) with browser + OAuth login, Ollama first-class card, Claude Opus 5 & Sonnet 5, Kimi official partnership (Code/Web/Moonshot), Zed, Requesty, SenseNova, Yuanbao, Agnes AI… and a refreshed **356-provider catalog**. → [Providers](docs/reference/PROVIDER_REFERENCE.md)
|
||||
- **📡 Routing transparency** — every response carries an `X-OmniRoute-Decision` header naming the strategy/provider/latency that served it, a new `cache-optimized` combo strategy + Auto-Combo `cacheAffinity` factor route repeat requests back to the connection holding the cached prefix, and a read-only `/v1/auto-combo/{channel}/candidates` endpoint exposes an `auto/*` channel's live candidate pool. → [Auto-Combo](docs/routing/AUTO-COMBO.md)
|
||||
- **⚡ Local performance & infra** — one-click local Redis, Cloudflare Workers / Deno Deploy relay deployers, Bifrost & Mux as supervised embedded services. → [Embedded Services](docs/frameworks/EMBEDDED-SERVICES.md)
|
||||
|
||||
@@ -642,7 +642,7 @@ of your shell history. → [CLI Integrations](docs/guides/CLI-INTEGRATIONS.md)
|
||||
|
||||
<div align="center">
|
||||
|
||||
## 🌐 357 AI Providers — 154 Catalog-Marked Free
|
||||
## 🌐 356 AI Providers — 154 Catalog-Marked Free
|
||||
|
||||
</div>
|
||||
|
||||
|
||||
@@ -1 +0,0 @@
|
||||
- **fix(kie):** correct 12 more KIE Market catalog ids that were sent to `createTask` unchanged but diverge from KIE's documented upstream `model` values — GPT Image 2 T2I/I2I (drops the `gpt/` prefix), GPT Image 1.5 T2I/I2I (`gpt-image/` namespace), Seedream 5.0 Lite T2I/I2I (drops the `.0`), all 4 Flux 2 variants (`flux-2/` namespace, generic variant renamed `flex`), and Wan 2.7 Image / Image Pro (dash instead of dot) — each verified individually against the literal example request published on docs.kie.ai. `#11326`'s "everything else already matches" claim was wrong a second time (#11296); `z-image/4.0-*`/`z-image/4.5-*` and `flux/kontext` remain open, documented as unresolved in `KIE_MARKET_UPSTREAM_MODEL_IDS`'s comment pending further verification.
|
||||
@@ -1 +0,0 @@
|
||||
- **fix(dashboard):** `useApiKeySave.handleSaveApiKey` no longer forces a full upstream `/models` catalog sync on every non-curated provider connection save — callers can now pass `skipModelSync: true` to opt out, so a workflow that only wants to add one manual model no longer floods the provider's available-models list with hundreds/thousands of synced entries. The flag is a client-side intent signal only and is stripped before the connection payload is POSTed to `/api/providers`; default behavior (full sync on save) is unchanged when the flag is omitted (#11324)
|
||||
@@ -1 +0,0 @@
|
||||
- **fix(combos):** the combo builder's precision-select, global-model-search, and manual-entry flows now serialize a model step's `model` string using the provider's already-computed routing-alias prefix (e.g. `oc/`) instead of rebuilding it from the raw canonical `providerId`, fixing the no-auth "OpenCode Free" provider (`opencode`) being routed to the unrelated paid "OpenCode Zen" provider (`opencode-zen`) because `opencode` doubles as a manual routing-prefix override ([#11433](https://github.com/diegosouzapw/OmniRoute/issues/11433)).
|
||||
@@ -1 +0,0 @@
|
||||
- fix(ui): let AnySearch use the normal provider-icon fallback when LobeHub has no matching icon (#11449)
|
||||
@@ -1 +0,0 @@
|
||||
- **fix(combo):** attach the same combo-diagnostics trace (`poolSize`/`attemptOrder`/`excluded`/`terminalReason`, plus `x-omniroute-combo-*` headers) to the round-robin strategy's and the nested pipeline/fusion runtime-unit loop's "Maximum combo retry limit reached" 503 that the priority-strategy path already attaches for the identical terminal condition — previously those two paths returned a bare, contextless 503 ([#11462](https://github.com/diegosouzapw/OmniRoute/issues/11462)).
|
||||
@@ -1 +0,0 @@
|
||||
- **fix(db):** dedupe the raw `[Encryption] Decryption failed...` log line emitted by the lazy-decrypt views (`createLazyRowProxy`/`createLazyConnectionView`), which power `getProviderConnections()` and were re-triggering that line on every CredentialHealth/model-sync cycle for the same corrupt or stale-key credential — a fresh Proxy over a fresh row on every cycle meant the per-proxy memoization never suppressed it, unlike the dedup `decryptConnectionFields()` already had since [#9927](https://github.com/diegosouzapw/OmniRoute/issues/9927). Now shares that dedupe tracking so the line logs at most once per credential ([#11500](https://github.com/diegosouzapw/OmniRoute/issues/11500)).
|
||||
@@ -1 +0,0 @@
|
||||
- **fix(sse):** stop dropping resolved `thoughtSignature` values on parallel (multi tool-call) turns sent to Gemini 3.x — the claude→gemini and openai→gemini translators previously kept the signature only on the *first* function call of a message, causing Gemini to reject subsequent calls in the same turn with HTTP 400 "Function call is missing a thought_signature"; each function call now keeps its own resolved signature ([#11510](https://github.com/diegosouzapw/OmniRoute/issues/11510)).
|
||||
@@ -1 +0,0 @@
|
||||
- **fix(sse):** cap the upstream headers-wait phase for STREAMING requests to a client-realistic ceiling (110s, under Codex's own ~120s hard client-abort window) instead of the flat 10-minute `FETCH_TIMEOUT_MS` default — that default was 5x longer than the body-phase readiness watchdog's own adaptive bound, so a request whose upstream never returned any response at all (not even headers, e.g. a stalled NVIDIA target behind a tool-heavy Responses→Chat translation) kept the client connection alive on keepalives only, guaranteeing the client's own patience ran out first with an opaque 499 instead of OmniRoute detecting and failing the stall fast. Non-streaming requests are unaffected — they keep the existing flat default (`open-sse/utils/fetchStartTimeoutPolicy.ts`) (#11526)
|
||||
@@ -1 +0,0 @@
|
||||
- **fix(security):** Video Bridge transcript text is now omitted from OmniRoute-owned retained request, response, usage, error, stream, continuation, handoff, and Memory surfaces while the live bridge request remains intact. Server-generated descriptions are matched by bounded SHA-256/length identities emitted only by a successful Video Bridge rewrite; ordinary fields named `transcript` and caller-forged description prose remain untouched. Traversal is cycle-safe and bounded, and hostile getters, proxies, or budget overflow fail closed to a constant omission marker instead of leaking content or breaking the request. Custom plugins and guardrails remain privileged processors of the live payload and must secure any sinks they create themselves. ([#11658](https://github.com/diegosouzapw/OmniRoute/issues/11658))
|
||||
@@ -1 +0,0 @@
|
||||
- **fix(docker):** re-derive the Docker build's worker-pool memory budget from the MEASURED ~4.5 GB per-process RSS (the issue owner's own VPS dmesg OOM-killer reproduction) instead of the stale 2560 MB/worker inference #11419 shipped, and lower `OMNIROUTE_BUILD_WORKERS` 3 → 2 so 1 parent + 1 page-data worker (2 processes × 4.5 GB = 9 GB) fits the 12.288 GB (75%) budget on a 16 GB GitHub Actions runner — the previous default (1 parent + 2 workers = 13.5 GB) still overcommitted the runner and kept "Publish to Docker Hub" failing with `cannot allocate memory` after #11419 merged (#7518).
|
||||
@@ -3433,7 +3433,7 @@
|
||||
"count": 15
|
||||
},
|
||||
"@typescript-eslint/no-unused-vars": {
|
||||
"count": 1
|
||||
"count": 2
|
||||
}
|
||||
},
|
||||
"tests/integration/traffic-inspector-error-sanitization.test.ts": {
|
||||
@@ -3680,6 +3680,11 @@
|
||||
"count": 15
|
||||
}
|
||||
},
|
||||
"tests/unit/authz/probe-9033-repro.test.ts": {
|
||||
"@typescript-eslint/no-unused-vars": {
|
||||
"count": 1
|
||||
}
|
||||
},
|
||||
"tests/unit/autoCombo/tieredRotation.test.ts": {
|
||||
"@typescript-eslint/no-unused-vars": {
|
||||
"count": 3
|
||||
|
||||
@@ -188,15 +188,14 @@
|
||||
"dedicatedGate": true
|
||||
},
|
||||
"bundleSize": {
|
||||
"value": 8653,
|
||||
"value": 8461,
|
||||
"direction": "down",
|
||||
"dedicatedGate": true,
|
||||
"_rebaseline_2026_07_07_v3846_release_close": "5601->6534 (+933). v3.8.46 release close: gzip of the 4 bin/*.mjs entrypoints (size-limit + @size-limit/file) grew from this cycle's feature/fix merges pulled transitively into the CLI entrypoints (new providers, combo pipeline strategy #6396, effort/thinking standardization #6241, catalog cache-invalidation #6408). Measured 6534 locally via `check:bundle-size --ratchet` (deterministic gzip, matches CI). Legitimate cycle growth; shrink is separate debt.",
|
||||
"_rebaseline_2026_07_19_7808_codeql_alias_resolver_hook": "6534->6762 (+228). PR #7808 (CodeQL js/incomplete-url-substring-sanitization fix): the ESM loader hook source moved out of the inline `HOOK_SOURCE` template literal in bin/aliasResolver.mjs into a real file bin/aliasResolverHook.mjs, loaded via pathToFileURL() instead of a dynamically-built `data:text/javascript,...` URL. The new file is now counted by size-limit as a 5th bin/*.mjs entrypoint. Net +228 = the hook's gzip size (previously hidden inside aliasResolver.mjs because the template literal was compressed away). Security-driven; no shrink opportunity.",
|
||||
"_rebaseline_2026_07_28_v3849_release_preflight": "6762 -> 7666 (+904). Fechamento do ciclo v3.8.49: gzip dos entrypoints bin/*.mjs (size-limit + @size-limit/file) cresceu com o que os merges do ciclo puxam transitivamente para o CLI (novos provedores — 271->290, seletor de protocolo por conexão #8861, catálogos de busca #8814, resiliência). Crescimento legítimo de ciclo, medido localmente com `npm run check:bundle-size` = 7666 (gzip determinístico, bate com o CI). Encolher é dívida separada.",
|
||||
"_rebaseline_2026_08_09_v3850_release_close": "7666 -> 8045 (+379 gzip bytes, +4.9%). Release v3.8.50 close reconciliation measured twice with the real size-limit + @size-limit/file path on tip e0ce95c592. Per-entry measurements remain below their absolute budgets: omniroute.mjs 4380/15000, mcp-server.mjs 1195/5000, nodeRuntimeSupport.mjs 887/8000, reset-password.mjs 1583/6000. The growth accumulated through legitimate CLI/runtime work in this cycle, including global-install ESM alias resolution, Termux cache preparation, and MCP stdio startup hardening; no entrypoint is near its absolute ceiling. The direction:down ratchet stays blocking from this exact measured tip.",
|
||||
"_rebaseline_2026_08_24_ci_green_gates_f95b03d7": "8045 -> 8461 (+416 gzip bytes, +5.2%). CI run 32786966560 (release PR #8875, tip f95b03d7) measured bundleSize=8461 via check:bundle-size --ratchet, above the 8045 baseline left at the v3.8.50 close. The growth comes from the post-freeze back-merge cycle landing in the CLI entrypoints (Synthetic + Kilo Gateway providers, kilo-gateway routing surface). Re-baseline per the gate's own instruction (legitimate cycle growth); shrinking the entrypoints remains separate debt; direction:down ratchet stays blocking from this measured tip.",
|
||||
"_rebaseline_2026_08_27_v3851_volatile_env_warning_11437": "8461 -> 8653 (+192 gzip bytes, +2.3%). Exact paired size-limit measurements on the VPS compared f95b03d709 with release/v3.8.51: only bin/omniroute.mjs changed, 4700 -> 4892; the other three entries remained 1195/983/1583. The growth originates in 943b9aaa84 (#11437), which warns users before a package-local .env is lost on the next global install. The CLI entry remains 4892/15000 bytes (32.6% of its absolute budget). Legitimate bug-fix growth; shrinking stays separate debt and direction:down remains blocking from this measured tip."
|
||||
"_rebaseline_2026_08_24_ci_green_gates_f95b03d7": "8045 -> 8461 (+416 gzip bytes, +5.2%). CI run 32786966560 (release PR #8875, tip f95b03d7) measured bundleSize=8461 via check:bundle-size --ratchet, above the 8045 baseline left at the v3.8.50 close. The growth comes from the post-freeze back-merge cycle landing in the CLI entrypoints (Synthetic + Kilo Gateway providers, kilo-gateway routing surface). Re-baseline per the gate's own instruction (legitimate cycle growth); shrinking the entrypoints remains separate debt; direction:down ratchet stays blocking from this measured tip."
|
||||
},
|
||||
"openapiBreaking": {
|
||||
"value": 4,
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
<svg viewBox="0 0 1200 350" xmlns="http://www.w3.org/2000/svg" role="img" aria-label="Animated terminal demoing the OmniRoute CLI: omniroute providers list (357 providers registered, anthropic, codex, glm, kimi shown active), omniroute combo list (always-on priority, cost-saver, fusion-panel, context-relay) and omniroute health (healthy, 18412 requests in 24h, p95 412ms, circuit breakers 24 closed, 1 half-open, 0 open), cycling over 85 top-level commands: providers, oauth, keys, combo, nodes, models, cache, compression, cost, usage, quota, health, resilience, telemetry, logs, audit, mcp, a2a, cloud, memory, skills, eval, doctor, repl, tunnel, backup, sync, webhooks, policy, pricing, translator, simulate and more.">
|
||||
<svg viewBox="0 0 1200 350" xmlns="http://www.w3.org/2000/svg" role="img" aria-label="Animated terminal demoing the OmniRoute CLI: omniroute providers list (356 providers registered, anthropic, codex, glm, kimi shown active), omniroute combo list (always-on priority, cost-saver, fusion-panel, context-relay) and omniroute health (healthy, 18412 requests in 24h, p95 412ms, circuit breakers 24 closed, 1 half-open, 0 open), cycling over 85 top-level commands: providers, oauth, keys, combo, nodes, models, cache, compression, cost, usage, quota, health, resilience, telemetry, logs, audit, mcp, a2a, cloud, memory, skills, eval, doctor, repl, tunnel, backup, sync, webhooks, policy, pricing, translator, simulate and more.">
|
||||
<desc>Compact animated terminal cycling three real OmniRoute CLI commands with a typewriter effect and a scrolling subcommand ticker; the first frame shows the completed providers-list screen.</desc>
|
||||
<defs><clipPath id="tickerClip"><rect x="12" y="304" width="1176" height="40"/></clipPath><clipPath id="tw0"><rect x="64" y="46" height="26" width="0"><animate attributeName="width" calcMode="discrete" values="0;31;61;92;122;153;184;214;245;245" keyTimes="0;0.012;0.018;0.024;0.030;0.036;0.042;0.048;0.054;1" dur="18s" repeatCount="indefinite"/></rect></clipPath><clipPath id="tw1"><rect x="64" y="46" height="26" width="0"><animate attributeName="width" calcMode="discrete" values="0;26;51;76;102;128;153;178;204;204" keyTimes="0;0.348;0.351;0.357;0.363;0.369;0.375;0.381;0.387;1" dur="18s" repeatCount="indefinite"/></rect></clipPath><clipPath id="tw2"><rect x="64" y="46" height="26" width="0"><animate attributeName="width" calcMode="discrete" values="0;20;41;61;82;102;122;143;163;163" keyTimes="0;0.678;0.684;0.690;0.696;0.702;0.708;0.714;0.720;1" dur="18s" repeatCount="indefinite"/></rect></clipPath></defs>
|
||||
<rect width="1200" height="350" fill="#0d1117"/>
|
||||
|
||||
|
Before Width: | Height: | Size: 12 KiB After Width: | Height: | Size: 12 KiB |
@@ -1,4 +1,4 @@
|
||||
<svg viewBox="0 0 1200 780" xmlns="http://www.w3.org/2000/svg" role="img" aria-label="Comparison table: OmniRoute versus 9router, OpenRouter, CLIProxyAPI and LiteLLM across 13 capabilities. OmniRoute is the only one with the full set: 357 providers, 90+ free providers built-in, 19 routing strategies, 12-engine token compression, a built-in MCP server with 110 tools, A2A protocol, persistent memory, guardrails, cloud agents, TLS fingerprint stealth, desktop/Termux/PWA, 43 UI locales and 100% MIT self-hosted. 9router has free providers, RTK compression and translation but no MCP, A2A, memory, guardrails, cloud agents or stealth. OpenRouter is a hosted SaaS with 400+ models, guardrails and a hosted MCP but is not self-hosted and lacks A2A, memory, cloud agents and stealth. CLIProxyAPI is a light OAuth proxy with two routing strategies. LiteLLM has 100+ providers, A2A and extensive guardrails but no memory, compression, free tier, stealth or cloud agents.">
|
||||
<svg viewBox="0 0 1200 780" xmlns="http://www.w3.org/2000/svg" role="img" aria-label="Comparison table: OmniRoute versus 9router, OpenRouter, CLIProxyAPI and LiteLLM across 13 capabilities. OmniRoute is the only one with the full set: 356 providers, 90+ free providers built-in, 19 routing strategies, 12-engine token compression, a built-in MCP server with 110 tools, A2A protocol, persistent memory, guardrails, cloud agents, TLS fingerprint stealth, desktop/Termux/PWA, 43 UI locales and 100% MIT self-hosted. 9router has free providers, RTK compression and translation but no MCP, A2A, memory, guardrails, cloud agents or stealth. OpenRouter is a hosted SaaS with 400+ models, guardrails and a hosted MCP but is not self-hosted and lacks A2A, memory, cloud agents and stealth. CLIProxyAPI is a light OAuth proxy with two routing strategies. LiteLLM has 100+ providers, A2A and extensive guardrails but no memory, compression, free tier, stealth or cloud agents.">
|
||||
<desc>Static-header comparison table where each capability row fades in top to bottom; the OmniRoute column is highlighted and shows a check or a leading value in every row, while competitors show a mix of checks, partials and crosses.</desc>
|
||||
<defs>
|
||||
<pattern id="gC" width="32" height="32" patternUnits="userSpaceOnUse"><path d="M 32 0 L 0 0 0 32" fill="none" stroke="#ffffff" stroke-opacity="0.05" stroke-width="1"/></pattern>
|
||||
|
||||
|
Before Width: | Height: | Size: 13 KiB After Width: | Height: | Size: 13 KiB |
@@ -1,4 +1,4 @@
|
||||
<svg viewBox="0 0 1200 540" xmlns="http://www.w3.org/2000/svg" role="img" aria-label="The OmniRoute promise: one endpoint and 357 providers. Six pillars. Resilient fallback: automatic routing continues while another healthy target is available. Save up to 95 percent of eligible tokens: RTK plus Caveman stacked compression averages about 89 percent on tool-heavy sessions. Zero dollars to start: 90+ providers with a free tier and 56 recurring or keyless free-forever providers. Every tool works: 35 CLI and agent integration records, including Claude Code, Codex, Cursor, Cline, Copilot and Antigravity, through one config. One endpoint: OpenAI, Claude, Gemini and Responses API translation at /v1. Production controls: circuit breakers, TLS stealth, MCP with 110 tools, A2A, memory, guardrails, evals, and 39,000+ static test declarations across 5,100+ tracked test files.">
|
||||
<svg viewBox="0 0 1200 540" xmlns="http://www.w3.org/2000/svg" role="img" aria-label="The OmniRoute promise: one endpoint and 356 providers. Six pillars. Resilient fallback: automatic routing continues while another healthy target is available. Save up to 95 percent of eligible tokens: RTK plus Caveman stacked compression averages about 89 percent on tool-heavy sessions. Zero dollars to start: 90+ providers with a free tier and 56 recurring or keyless free-forever providers. Every tool works: 35 CLI and agent integration records, including Claude Code, Codex, Cursor, Cline, Copilot and Antigravity, through one config. One endpoint: OpenAI, Claude, Gemini and Responses API translation at /v1. Production controls: circuit breakers, TLS stealth, MCP with 110 tools, A2A, memory, guardrails, evals, and 39,000+ static test declarations across 5,100+ tracked test files.">
|
||||
<desc>Animated promise card: six pillar tiles fade in in reading order, then a soft colored border highlight sweeps from tile to tile in a continuous cycle.</desc>
|
||||
<defs>
|
||||
<pattern id="gridPaperP" width="32" height="32" patternUnits="userSpaceOnUse">
|
||||
@@ -21,7 +21,7 @@
|
||||
<line x1="150" y1="53" x2="1160" y2="53" stroke="#232b38" stroke-width="1.5"/>
|
||||
</g>
|
||||
<g>
|
||||
<text x="40" y="100" font-family="Inter, 'Segoe UI', Arial, Helvetica, system-ui, sans-serif" font-size="23" font-weight="600" fill="#c9d1d9">One endpoint. <tspan fill="#a78bfa" font-weight="800">357 providers.</tspan> Never stop building — OmniRoute picks <tspan fill="#7ee787" font-weight="700">the cheapest one that works</tspan>.</text>
|
||||
<text x="40" y="100" font-family="Inter, 'Segoe UI', Arial, Helvetica, system-ui, sans-serif" font-size="23" font-weight="600" fill="#c9d1d9">One endpoint. <tspan fill="#a78bfa" font-weight="800">356 providers.</tspan> Never stop building — OmniRoute picks <tspan fill="#7ee787" font-weight="700">the cheapest one that works</tspan>.</text>
|
||||
</g>
|
||||
|
||||
<g font-family="Inter, 'Segoe UI', Arial, Helvetica, system-ui, sans-serif">
|
||||
@@ -38,7 +38,7 @@
|
||||
<line x1="3.9" y1="3.9" x2="18.1" y2="18.1"/>
|
||||
</g>
|
||||
<text x="102" y="170" font-size="18" font-weight="800" fill="#74b9ff">Never hit limits</text>
|
||||
<text x="66" y="204" font-size="13.5" fill="#a1a1aa">Auto-fallback across 357 providers in</text>
|
||||
<text x="66" y="204" font-size="13.5" fill="#a1a1aa">Auto-fallback across 356 providers in</text>
|
||||
<text x="66" y="226" font-size="13.5" fill="#a1a1aa">milliseconds. Quota out? The next provider</text>
|
||||
<text x="66" y="248" font-size="13.5" fill="#a1a1aa">takes over while a healthy target remains.</text>
|
||||
</g>
|
||||
|
||||
|
Before Width: | Height: | Size: 10 KiB After Width: | Height: | Size: 10 KiB |
@@ -1,4 +1,4 @@
|
||||
<svg viewBox="0 0 1200 548" xmlns="http://www.w3.org/2000/svg" role="img" aria-label="OmniRoute hero: Never stop coding. Every AI tool to 357 providers — 90+ free — through one endpoint. Claude Code, Codex, Cursor, Cline, Copilot and Antigravity into free Claude, GPT and Gemini with auto-fallback. RTK + Caveman stacked compression saves 15 to 95 percent of tokens — about 89 percent average on tool-heavy sessions — so you never hit limits. Stats: 357 AI providers, 90+ free tiers, about 1.51B free tokens per month, 15 to 95 percent token savings, 19 routing strategies, zero dollars to start.">
|
||||
<svg viewBox="0 0 1200 548" xmlns="http://www.w3.org/2000/svg" role="img" aria-label="OmniRoute hero: Never stop coding. Every AI tool to 356 providers — 90+ free — through one endpoint. Claude Code, Codex, Cursor, Cline, Copilot and Antigravity into free Claude, GPT and Gemini with auto-fallback. RTK + Caveman stacked compression saves 15 to 95 percent of tokens — about 89 percent average on tool-heavy sessions — so you never hit limits. Stats: 356 AI providers, 90+ free tiers, about 1.51B free tokens per month, 15 to 95 percent token savings, 19 routing strategies, zero dollars to start.">
|
||||
<desc>Animated hero card: a pulse travels the divider line and a compression bar demo repeatedly shrinks a prompt by up to 95 percent; all headline content is static and readable on the first frame.</desc>
|
||||
<defs>
|
||||
<pattern id="gridPaperH" width="32" height="32" patternUnits="userSpaceOnUse">
|
||||
@@ -28,7 +28,7 @@
|
||||
<text x="48" y="138" font-family="Inter, 'Segoe UI', Arial, Helvetica, system-ui, sans-serif" font-size="60" font-weight="800" fill="#e9edf3">Never stop coding<tspan fill="#a855f7">.</tspan></text>
|
||||
|
||||
<!-- subheadline -->
|
||||
<text x="48" y="184" font-family="Inter, 'Segoe UI', Arial, Helvetica, system-ui, sans-serif" font-size="25" font-weight="600" fill="#c9d1d9">Every AI tool → <tspan fill="#a78bfa" font-weight="800">357 providers</tspan> — <tspan fill="#7ee787" font-weight="800">90+ free</tspan> — through one endpoint.</text>
|
||||
<text x="48" y="184" font-family="Inter, 'Segoe UI', Arial, Helvetica, system-ui, sans-serif" font-size="25" font-weight="600" fill="#c9d1d9">Every AI tool → <tspan fill="#a78bfa" font-weight="800">356 providers</tspan> — <tspan fill="#7ee787" font-weight="800">90+ free</tspan> — through one endpoint.</text>
|
||||
|
||||
<!-- plug line -->
|
||||
<text x="48" y="222" font-family="Inter, 'Segoe UI', Arial, Helvetica, system-ui, sans-serif" font-size="16.5" fill="#a1a1aa">Claude Code · Codex · Cursor · Cline · Copilot · Antigravity  →  <tspan fill="#7ee787" font-weight="700">FREE</tspan> Claude / GPT / Gemini · auto-fallback</text>
|
||||
|
||||
|
Before Width: | Height: | Size: 7.3 KiB After Width: | Height: | Size: 7.3 KiB |
@@ -226,21 +226,16 @@ Three build args control what the `builder` stage costs. They are build-time onl
|
||||
| --------------------------- | ------- | ----------------------------------------------------------------------------------- |
|
||||
| `OMNIROUTE_USE_TURBOPACK` | `1` | `0` builds with webpack instead. Lower peak memory, slower. |
|
||||
| `OMNIROUTE_BUILD_MEMORY_MB` | `6144` | V8 heap ceiling (`--max-old-space-size`) for the spawned `next build`. |
|
||||
| `OMNIROUTE_BUILD_WORKERS` | `2` | Feeds `CIRCLE_NODE_TOTAL`; Next derives `workers = N - 1` for page-data collection. |
|
||||
| `OMNIROUTE_BUILD_WORKERS` | `3` | Feeds `CIRCLE_NODE_TOTAL`; Next derives `workers = N - 1` for page-data collection. |
|
||||
|
||||
`OMNIROUTE_BUILD_WORKERS` is the one to raise on a big builder and the one to
|
||||
suspect when a constrained build dies **after** `✓ Compiled successfully`. Each
|
||||
page-data worker is its own process, and so is the parent `next build` itself;
|
||||
a live VPS reproduction (issue #7518) measured each process's peak RSS at
|
||||
~4.5 GB independent of the `NODE_OPTIONS` heap flag (Turbopack compiles in
|
||||
native/Rust memory outside the V8 heap). The default of `2` (→ 1 worker, 2
|
||||
processes total) is sized for the 16 GB / 4 vCPU GitHub-hosted runners the
|
||||
publish pipeline uses. At `8` (→ 7 workers) that runner ran out of memory and
|
||||
buildkit failed the step with `ResourceExhausted: ... cannot allocate memory`;
|
||||
`3` (→ 2 workers) still didn't fit once the per-process RSS was measured
|
||||
directly instead of inferred. `tests/unit/docker-build-memory-budget.test.ts`
|
||||
does the arithmetic against the measured figure and fails if either knob
|
||||
outgrows the runner.
|
||||
page-data worker is its own process and inherits `NODE_OPTIONS`, so the heap
|
||||
ceiling is per process, not per build: the default of `3` (→ 2 workers) is sized
|
||||
for the 16 GB / 4 vCPU GitHub-hosted runners the publish pipeline uses. At `8`
|
||||
(→ 7 workers) that runner ran out of memory and buildkit failed the step with
|
||||
`ResourceExhausted: ... cannot allocate memory`. `tests/unit/docker-build-memory-budget.test.ts`
|
||||
does the arithmetic and fails if either knob outgrows the runner.
|
||||
|
||||
Turbopack compiles in native Rust memory that lives **outside** the V8 heap, so
|
||||
`OMNIROUTE_BUILD_MEMORY_MB` does not bound it. On a host with a memory ceiling the
|
||||
|
||||
@@ -4,7 +4,8 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 357 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 356 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -281,7 +282,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
## Key Features (v3.8.50)
|
||||
|
||||
### Core Proxy
|
||||
- **357 AI providers** with automatic format translation
|
||||
- **356 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
@@ -479,7 +480,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **357-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **356-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 110 tools / 33 scopes** (canonical + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
|
||||
@@ -4,7 +4,8 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 357 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 356 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -281,7 +282,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
## Key Features (v3.8.50)
|
||||
|
||||
### Core Proxy
|
||||
- **357 AI providers** with automatic format translation
|
||||
- **356 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
@@ -479,7 +480,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **357-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **356-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 110 tools / 33 scopes** (canonical + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
|
||||
@@ -4,7 +4,8 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 357 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 356 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -281,7 +282,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
## Key Features (v3.8.50)
|
||||
|
||||
### Core Proxy
|
||||
- **357 AI providers** with automatic format translation
|
||||
- **356 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
@@ -479,7 +480,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **357-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **356-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 110 tools / 33 scopes** (canonical + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
|
||||
@@ -4,7 +4,8 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 357 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 356 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -281,7 +282,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
## Key Features (v3.8.50)
|
||||
|
||||
### Core Proxy
|
||||
- **357 AI providers** with automatic format translation
|
||||
- **356 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
@@ -479,7 +480,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **357-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **356-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 110 tools / 33 scopes** (canonical + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
|
||||
@@ -4,7 +4,8 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 357 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 356 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -281,7 +282,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
## Key Features (v3.8.50)
|
||||
|
||||
### Core Proxy
|
||||
- **357 AI providers** with automatic format translation
|
||||
- **356 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
@@ -479,7 +480,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **357-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **356-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 110 tools / 33 scopes** (canonical + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
|
||||
@@ -4,7 +4,8 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 357 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 356 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -281,7 +282,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
## Key Features (v3.8.50)
|
||||
|
||||
### Core Proxy
|
||||
- **357 AI providers** with automatic format translation
|
||||
- **356 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
@@ -479,7 +480,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **357-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **356-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 110 tools / 33 scopes** (canonical + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
|
||||
@@ -4,7 +4,8 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 357 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 356 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -281,7 +282,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
## Key Features (v3.8.50)
|
||||
|
||||
### Core Proxy
|
||||
- **357 AI providers** with automatic format translation
|
||||
- **356 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
@@ -479,7 +480,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **357-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **356-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 110 tools / 33 scopes** (canonical + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
|
||||
@@ -4,7 +4,8 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 357 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 356 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -281,7 +282,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
## Key Features (v3.8.50)
|
||||
|
||||
### Core Proxy
|
||||
- **357 AI providers** with automatic format translation
|
||||
- **356 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
@@ -479,7 +480,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **357-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **356-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 110 tools / 33 scopes** (canonical + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
|
||||
@@ -4,7 +4,8 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 357 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 356 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -281,7 +282,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
## Key Features (v3.8.50)
|
||||
|
||||
### Core Proxy
|
||||
- **357 AI providers** with automatic format translation
|
||||
- **356 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
@@ -479,7 +480,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **357-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **356-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 110 tools / 33 scopes** (canonical + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
|
||||
@@ -4,7 +4,8 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 357 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 356 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -281,7 +282,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
## Key Features (v3.8.50)
|
||||
|
||||
### Core Proxy
|
||||
- **357 AI providers** with automatic format translation
|
||||
- **356 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
@@ -479,7 +480,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **357-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **356-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 110 tools / 33 scopes** (canonical + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
|
||||
@@ -4,7 +4,8 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 357 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 356 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -281,7 +282,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
## Key Features (v3.8.50)
|
||||
|
||||
### Core Proxy
|
||||
- **357 AI providers** with automatic format translation
|
||||
- **356 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
@@ -479,7 +480,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **357-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **356-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 110 tools / 33 scopes** (canonical + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
|
||||
@@ -4,7 +4,8 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 357 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 356 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -281,7 +282,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
## Key Features (v3.8.50)
|
||||
|
||||
### Core Proxy
|
||||
- **357 AI providers** with automatic format translation
|
||||
- **356 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
@@ -479,7 +480,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **357-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **356-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 110 tools / 33 scopes** (canonical + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
|
||||
@@ -4,7 +4,8 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 357 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 356 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -281,7 +282,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
## Key Features (v3.8.50)
|
||||
|
||||
### Core Proxy
|
||||
- **357 AI providers** with automatic format translation
|
||||
- **356 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
@@ -479,7 +480,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **357-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **356-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 110 tools / 33 scopes** (canonical + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
|
||||
@@ -4,7 +4,8 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 357 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 356 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -281,7 +282,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
## Key Features (v3.8.50)
|
||||
|
||||
### Core Proxy
|
||||
- **357 AI providers** with automatic format translation
|
||||
- **356 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
@@ -479,7 +480,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **357-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **356-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 110 tools / 33 scopes** (canonical + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
|
||||
@@ -4,7 +4,8 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 357 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 356 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -281,7 +282,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
## Key Features (v3.8.50)
|
||||
|
||||
### Core Proxy
|
||||
- **357 AI providers** with automatic format translation
|
||||
- **356 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
@@ -479,7 +480,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **357-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **356-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 110 tools / 33 scopes** (canonical + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
|
||||
@@ -4,7 +4,8 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 357 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 356 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -281,7 +282,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
## Key Features (v3.8.50)
|
||||
|
||||
### Core Proxy
|
||||
- **357 AI providers** with automatic format translation
|
||||
- **356 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
@@ -479,7 +480,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **357-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **356-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 110 tools / 33 scopes** (canonical + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
|
||||
@@ -4,7 +4,8 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 357 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 356 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -281,7 +282,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
## Key Features (v3.8.50)
|
||||
|
||||
### Core Proxy
|
||||
- **357 AI providers** with automatic format translation
|
||||
- **356 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
@@ -479,7 +480,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **357-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **356-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 110 tools / 33 scopes** (canonical + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
|
||||
@@ -4,7 +4,8 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 357 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 356 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -281,7 +282,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
## Key Features (v3.8.50)
|
||||
|
||||
### Core Proxy
|
||||
- **357 AI providers** with automatic format translation
|
||||
- **356 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
@@ -479,7 +480,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **357-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **356-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 110 tools / 33 scopes** (canonical + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
|
||||
@@ -4,7 +4,8 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 357 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 356 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -281,7 +282,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
## Key Features (v3.8.50)
|
||||
|
||||
### Core Proxy
|
||||
- **357 AI providers** with automatic format translation
|
||||
- **356 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
@@ -479,7 +480,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **357-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **356-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 110 tools / 33 scopes** (canonical + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
|
||||
@@ -4,7 +4,8 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 357 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 356 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -281,7 +282,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
## Key Features (v3.8.50)
|
||||
|
||||
### Core Proxy
|
||||
- **357 AI providers** with automatic format translation
|
||||
- **356 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
@@ -479,7 +480,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **357-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **356-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 110 tools / 33 scopes** (canonical + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
|
||||
@@ -4,7 +4,8 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 357 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 356 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -281,7 +282,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
## Key Features (v3.8.50)
|
||||
|
||||
### Core Proxy
|
||||
- **357 AI providers** with automatic format translation
|
||||
- **356 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
@@ -479,7 +480,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **357-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **356-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 110 tools / 33 scopes** (canonical + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
|
||||
@@ -4,7 +4,8 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 357 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 356 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -281,7 +282,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
## Key Features (v3.8.50)
|
||||
|
||||
### Core Proxy
|
||||
- **357 AI providers** with automatic format translation
|
||||
- **356 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
@@ -479,7 +480,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **357-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **356-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 110 tools / 33 scopes** (canonical + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
|
||||
@@ -4,7 +4,8 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 357 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 356 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -281,7 +282,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
## Key Features (v3.8.50)
|
||||
|
||||
### Core Proxy
|
||||
- **357 AI providers** with automatic format translation
|
||||
- **356 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
@@ -479,7 +480,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **357-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **356-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 110 tools / 33 scopes** (canonical + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
|
||||
@@ -4,7 +4,8 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 357 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 356 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -281,7 +282,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
## Key Features (v3.8.50)
|
||||
|
||||
### Core Proxy
|
||||
- **357 AI providers** with automatic format translation
|
||||
- **356 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
@@ -479,7 +480,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **357-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **356-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 110 tools / 33 scopes** (canonical + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
|
||||
@@ -4,7 +4,8 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 357 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 356 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -281,7 +282,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
## Key Features (v3.8.50)
|
||||
|
||||
### Core Proxy
|
||||
- **357 AI providers** with automatic format translation
|
||||
- **356 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
@@ -479,7 +480,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **357-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **356-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 110 tools / 33 scopes** (canonical + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
|
||||
@@ -4,7 +4,8 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 357 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 356 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -281,7 +282,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
## Key Features (v3.8.50)
|
||||
|
||||
### Core Proxy
|
||||
- **357 AI providers** with automatic format translation
|
||||
- **356 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
@@ -479,7 +480,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **357-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **356-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 110 tools / 33 scopes** (canonical + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
|
||||
@@ -4,7 +4,8 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 357 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 356 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -281,7 +282,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
## Key Features (v3.8.50)
|
||||
|
||||
### Core Proxy
|
||||
- **357 AI providers** with automatic format translation
|
||||
- **356 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
@@ -479,7 +480,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **357-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **356-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 110 tools / 33 scopes** (canonical + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
|
||||
@@ -4,7 +4,8 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 357 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 356 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -281,7 +282,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
## Key Features (v3.8.50)
|
||||
|
||||
### Core Proxy
|
||||
- **357 AI providers** with automatic format translation
|
||||
- **356 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
@@ -479,7 +480,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **357-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **356-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 110 tools / 33 scopes** (canonical + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
|
||||
@@ -4,7 +4,8 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 357 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 356 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -281,7 +282,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
## Key Features (v3.8.50)
|
||||
|
||||
### Core Proxy
|
||||
- **357 AI providers** with automatic format translation
|
||||
- **356 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
@@ -479,7 +480,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **357-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **356-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 110 tools / 33 scopes** (canonical + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
|
||||
@@ -4,7 +4,8 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 357 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 356 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -281,7 +282,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
## Key Features (v3.8.50)
|
||||
|
||||
### Core Proxy
|
||||
- **357 AI providers** with automatic format translation
|
||||
- **356 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
@@ -479,7 +480,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **357-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **356-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 110 tools / 33 scopes** (canonical + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
|
||||
@@ -4,7 +4,8 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 357 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 356 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -281,7 +282,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
## Key Features (v3.8.50)
|
||||
|
||||
### Core Proxy
|
||||
- **357 AI providers** with automatic format translation
|
||||
- **356 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
@@ -479,7 +480,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **357-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **356-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 110 tools / 33 scopes** (canonical + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
|
||||
@@ -4,7 +4,8 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 357 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 356 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -281,7 +282,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
## Key Features (v3.8.50)
|
||||
|
||||
### Core Proxy
|
||||
- **357 AI providers** with automatic format translation
|
||||
- **356 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
@@ -479,7 +480,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **357-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **356-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 110 tools / 33 scopes** (canonical + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
|
||||
@@ -4,7 +4,8 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 357 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 356 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -281,7 +282,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
## Key Features (v3.8.50)
|
||||
|
||||
### Core Proxy
|
||||
- **357 AI providers** with automatic format translation
|
||||
- **356 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
@@ -479,7 +480,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **357-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **356-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 110 tools / 33 scopes** (canonical + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
|
||||
@@ -4,7 +4,8 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 357 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 356 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -281,7 +282,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
## Key Features (v3.8.50)
|
||||
|
||||
### Core Proxy
|
||||
- **357 AI providers** with automatic format translation
|
||||
- **356 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
@@ -479,7 +480,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **357-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **356-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 110 tools / 33 scopes** (canonical + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
|
||||
@@ -4,7 +4,8 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 357 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 356 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -281,7 +282,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
## Key Features (v3.8.50)
|
||||
|
||||
### Core Proxy
|
||||
- **357 AI providers** with automatic format translation
|
||||
- **356 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
@@ -479,7 +480,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **357-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **356-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 110 tools / 33 scopes** (canonical + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
|
||||
@@ -4,7 +4,8 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 357 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 356 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -281,7 +282,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
## Key Features (v3.8.50)
|
||||
|
||||
### Core Proxy
|
||||
- **357 AI providers** with automatic format translation
|
||||
- **356 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
@@ -479,7 +480,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **357-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **356-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 110 tools / 33 scopes** (canonical + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
|
||||
@@ -4,7 +4,8 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 357 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 356 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -281,7 +282,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
## Key Features (v3.8.50)
|
||||
|
||||
### Core Proxy
|
||||
- **357 AI providers** with automatic format translation
|
||||
- **356 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
@@ -479,7 +480,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **357-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **356-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 110 tools / 33 scopes** (canonical + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
|
||||
@@ -4,7 +4,8 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 357 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 356 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -281,7 +282,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
## Key Features (v3.8.50)
|
||||
|
||||
### Core Proxy
|
||||
- **357 AI providers** with automatic format translation
|
||||
- **356 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
@@ -479,7 +480,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **357-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **356-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 110 tools / 33 scopes** (canonical + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
|
||||
@@ -4,7 +4,8 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 357 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 356 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -281,7 +282,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
## Key Features (v3.8.50)
|
||||
|
||||
### Core Proxy
|
||||
- **357 AI providers** with automatic format translation
|
||||
- **356 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
@@ -479,7 +480,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **357-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **356-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 110 tools / 33 scopes** (canonical + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
|
||||
@@ -4,7 +4,8 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 357 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 356 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -281,7 +282,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
## Key Features (v3.8.50)
|
||||
|
||||
### Core Proxy
|
||||
- **357 AI providers** with automatic format translation
|
||||
- **356 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
@@ -479,7 +480,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **357-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **356-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 110 tools / 33 scopes** (canonical + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
|
||||
@@ -4,7 +4,8 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 357 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 356 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -281,7 +282,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
## Key Features (v3.8.50)
|
||||
|
||||
### Core Proxy
|
||||
- **357 AI providers** with automatic format translation
|
||||
- **356 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
@@ -479,7 +480,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **357-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **356-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 110 tools / 33 scopes** (canonical + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
|
||||
@@ -4,7 +4,8 @@
|
||||
|
||||
---
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 357 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 356 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -281,7 +282,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
## Key Features (v3.8.50)
|
||||
|
||||
### Core Proxy
|
||||
- **357 AI providers** with automatic format translation
|
||||
- **356 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
@@ -479,7 +480,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **357-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **356-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 110 tools / 33 scopes** (canonical + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
|
||||
@@ -1,16 +1,16 @@
|
||||
---
|
||||
title: "Provider Reference"
|
||||
version: 3.8.51
|
||||
lastUpdated: 2026-08-27
|
||||
lastUpdated: 2026-08-26
|
||||
---
|
||||
|
||||
# Provider Reference
|
||||
|
||||
> **Auto-generated** from `src/shared/constants/providers.ts` — do not edit by hand.
|
||||
> Regenerate with: `npm run gen:provider-reference`
|
||||
> **Last generated:** 2026-08-27
|
||||
> **Last generated:** 2026-08-26
|
||||
|
||||
Total providers: **357**. See category breakdown below.
|
||||
Total providers: **356**. See category breakdown below.
|
||||
|
||||
## Categories
|
||||
|
||||
@@ -381,11 +381,10 @@ Use the dashboard at `/dashboard/providers` to enable, configure, and test each
|
||||
| `vllm` | `vllm` | vLLM | Local, self-hosted | [link](https://github.com/vllm-project/vllm) | API key optional. Configure the local vLLM OpenAI-compatible base URL (default: http://localhost:8000/v1). |
|
||||
| `xinference` | `xinference` | XInference | Local, self-hosted | [link](https://inference.readthedocs.io) | API key optional. Configure the local XInference OpenAI-compatible base URL (default: http://localhost:9997/v1). |
|
||||
|
||||
## Search Providers (17)
|
||||
## Search Providers (16)
|
||||
|
||||
| ID | Alias | Name | Tags | Website | Notes |
|
||||
|----|-------|------|------|---------|-------|
|
||||
| `anysearch-search` | `anysearch` | AnySearch | Search | [link](https://anysearch.com) | Optional API key from anysearch.com (as_sk_...) - free 1000/day; keyless tier has lower limits |
|
||||
| `brave-search` | `brave-search` | Brave Search | Search | [link](https://brave.com/search/api) | Subscription token from Brave Search API dashboard |
|
||||
| `context7` | `context7` | Context7 (library docs) | Search | [link](https://context7.com) | API key optional (ctx7sk-...) — anonymous tier works without a key; a key raises the rate limit |
|
||||
| `exa-search` | `exa-search` | Exa Search | Search | [link](https://exa.ai) | API key from dashboard.exa.ai |
|
||||
@@ -399,6 +398,7 @@ Use the dashboard at `/dashboard/providers` to enable, configure, and test each
|
||||
| `searxng-search` | `searxng` | SearXNG Search | Search | [link](https://docs.searxng.org) | API key is optional. Set your SearXNG base URL. Some instances may require a bearer token for access. |
|
||||
| `serper-search` | `serper-search` | Serper Search | Search | [link](https://serper.dev) | API key from serper.dev dashboard |
|
||||
| `tavily-search` | `tavily-search` | Tavily Search | Search | [link](https://tavily.com) | API key from app.tavily.com (format: tvly-...) |
|
||||
| `anysearch-search` | `anysearch` | AnySearch | Search | [link](https://anysearch.com/docs) | Optional API key (as_sk_...). Free public web search for agents; 1000 req/day per key, shared with extract. Fallback-only. |
|
||||
| `x-search` | `x_search` | X Search (Grok) | Search | [link](https://docs.x.ai/developers/tools/x-search) | SuperGrok OAuth (xai-oauth) or xAI API key. This is Grok X Search, not the X Developer MCP. |
|
||||
| `xquik-search` | `xquik` | Xquik X Search | Search | [link](https://docs.xquik.com) | Xquik API key (xq_...). Search is metered per returned post; the catalog estimate uses 5 results. |
|
||||
| `youcom-search` | `youcom-search` | You.com Search | Search | [link](https://you.com/business/api/) | X-API-Key from the You.com platform dashboard |
|
||||
@@ -445,7 +445,7 @@ Use the dashboard at `/dashboard/providers` to enable, configure, and test each
|
||||
|
||||
- Catalog: [`src/shared/constants/providers.ts`](../../src/shared/constants/providers.ts)
|
||||
- Registry (per-model details): [`open-sse/config/providerRegistry.ts`](../../open-sse/config/providerRegistry.ts)
|
||||
- Executors: [`open-sse/executors/`](../../open-sse/executors/) (112 implementations)
|
||||
- Executors: [`open-sse/executors/`](../../open-sse/executors/) (111 implementations)
|
||||
- Translators: [`open-sse/translator/`](../../open-sse/translator/)
|
||||
|
||||
## See Also
|
||||
|
||||
@@ -1,13 +1,13 @@
|
||||
---
|
||||
title: "Guardrails"
|
||||
version: 3.8.51
|
||||
lastUpdated: 2026-08-26
|
||||
version: 3.8.50
|
||||
lastUpdated: 2026-08-24
|
||||
---
|
||||
|
||||
# Guardrails
|
||||
|
||||
> **Source of truth:** `src/lib/guardrails/`
|
||||
> **Last updated:** 2026-08-26 — v3.8.51 (Video Bridge transcript-retention boundary)
|
||||
> **Last updated:** 2026-08-24 — v3.8.50 (Video Bridge visual dedup hardening + focused captions)
|
||||
|
||||
Guardrails enforce safety, policy, and content transformations at the boundary
|
||||
between OmniRoute and upstream providers. Each guardrail can inspect (and
|
||||
@@ -439,61 +439,6 @@ are rendered as untrusted observations alongside the frame captions. Invalid,
|
||||
out-of-range, or provenance-free text is rejected rather than mixed into the
|
||||
caption stream.
|
||||
|
||||
#### Transcript retention boundary
|
||||
|
||||
Transcript text remains available to the live bridge and upstream request, but it is not retained
|
||||
by OmniRoute's logging and memory surfaces. `videoTranscriptLogRedaction.ts` recognizes
|
||||
`transcript` and `audioTranscript` only inside supported video carriers. A field with either name
|
||||
in an ordinary tool argument or application object is preserved, and delimiter-shaped caller
|
||||
prose is not trusted as a bridge result.
|
||||
|
||||
When the Video Bridge produces a description containing validated cues, it emits only a
|
||||
SHA-256/length identity in guardrail metadata. The request pipeline accepts those identities only
|
||||
from a successful, payload-modifying `video-bridge` result and uses them to omit the exact generated
|
||||
description segment after translation or concatenation. Raw transcript text is never placed in the
|
||||
metadata.
|
||||
|
||||
The retention policy covers raw, converted, and provider request logs; provider and client response
|
||||
bodies; detailed pipeline artifacts; call logs, active/pending usage, rejected requests, proxy/error
|
||||
diagnostics, SSE chunks, and Memory extraction. Response bodies and stream chunks for a sensitive
|
||||
request are omitted because an upstream may echo request text. Safe operational metadata such as
|
||||
status, timing, provider, model, headers-presence, token counts, and the fact that redaction occurred
|
||||
remains available. The constant retained marker is `[omitted: video transcript]`.
|
||||
|
||||
Sensitivity is resolved before the guardrail chain from the original client body and resolved again
|
||||
after the chain from the original body, processed body, and trusted description identities. A
|
||||
guardrail that removes or replaces the raw carrier therefore cannot clear the request-scoped bit.
|
||||
Explicit video parts remain sensitive when malformed, and nested carrier objects are inspected under
|
||||
the same bounds. Rejected-request persistence also inspects the body itself, so forgetting to pass the
|
||||
bit at one caller does not expose a raw carrier.
|
||||
|
||||
The request-scoped bit protects OmniRoute-owned application paths in addition to structured payload
|
||||
clones. Native priority, round-robin, pinned, and fusion diagnostics; quality-rejection and live-event
|
||||
logs; terminal combo errors; proxy fast-fail logs; guardrail-registry logs; plugin-dispatcher failure
|
||||
logs; and stream lifecycle callbacks retain only the constant marker when their detail could echo the
|
||||
request. Transcript-sensitive requests do not generate Context Relay or Universal Handoff summaries,
|
||||
and Memory extraction is skipped. Daily-quota routing state uses the server-resolved model rather than
|
||||
promoting a model token parsed from transcript-sensitive upstream text. These rules affect retention,
|
||||
not the live request or the functional response returned to the same client.
|
||||
|
||||
Inspection is bounded by depth, aggregate entries, trusted identities, description-prefix
|
||||
occurrences, candidate hashes, and total hashed text. Cycles and binary views are handled without
|
||||
retaining their contents. If a bound is exceeded, or an enumerable getter/proxy throws during
|
||||
inspection, the retained copy fails closed to the omission marker; the live request is not changed.
|
||||
|
||||
Privacy-redacted Responses artifacts cannot safely reconstruct complete `previous_response_id`
|
||||
history. `responsesContinuationStore.ts` therefore rejects an artifact carrying the trusted
|
||||
pipeline-level redaction marker and asks the client to resend full history instead of replaying a
|
||||
silently incomplete conversation.
|
||||
|
||||
Custom guardrails and plugins are privileged, in-process processors: they intentionally receive the
|
||||
live payload so they can inspect, transform, or block it. OmniRoute supplies the server-owned
|
||||
`videoTranscriptSensitive` bit, wraps `GuardrailContext.log`, and protects errors emitted by its native
|
||||
plugin dispatcher. It cannot control a third-party processor that independently writes the live body
|
||||
to `console`, a file, a database, or the network. Operators must audit such code and treat its own
|
||||
sinks as outside the OmniRoute-owned retention boundary; clearing or ignoring the bit does not make
|
||||
the transcript safe to persist.
|
||||
|
||||
An advanced caller may provide an already-authorized `audioTranscript` track
|
||||
for the same video. The fusion seam runs visual and audio observations under
|
||||
one deadline and abort signal, orders them on a common timeline, collapses
|
||||
|
||||
6
llm.txt
@@ -1,6 +1,6 @@
|
||||
# OmniRoute
|
||||
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 357 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
> OmniRoute is a free, open-source AI Gateway that acts as a universal API proxy for multi-provider LLMs. It provides smart routing, automatic fallback, load balancing, and format translation across 356 AI providers — all through a single OpenAI-compatible endpoint. Includes a built-in MCP Server (110 tools), A2A v0.3 protocol, Memory/Skills systems, Cloud Agents (codex, cursor, devin, jules), Guardrails framework, and an Electron desktop app.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -277,7 +277,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
## Key Features (v3.8.50)
|
||||
|
||||
### Core Proxy
|
||||
- **357 AI providers** with automatic format translation
|
||||
- **356 AI providers** with automatic format translation
|
||||
- **Provider categories**: Free (90+ free tiers), OAuth, API Key, Self-Hosted, Custom (OpenAI/Anthropic-compatible)
|
||||
- **18 routing strategies**: priority, weighted, round-robin, fill-first, p2c, random, least-used, cost-optimized, reset-aware, reset-window, headroom, strict-random, auto, lkgp, context-optimized, context-relay, fusion, pipeline
|
||||
- **4-tier fallback**: Subscription → API Key → Cheap → Free
|
||||
@@ -475,7 +475,7 @@ diagnostics) plus **memory**, **skill**, **agentSkill**, **githubSkill**, **pool
|
||||
|
||||
## v3.8.x Highlights
|
||||
|
||||
- **357-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **356-provider catalog** with 90+ free tiers, one-click account imports, and bulk key add
|
||||
- **19 routing strategies** — including `fusion` (parallel panel + judge synthesis), `pipeline`, `reset-aware`, `reset-window`, `headroom`, and `context-relay`
|
||||
- **14-factor Auto-Combo scoring** with bandit exploration and progressive cooldown
|
||||
- **MCP server expanded to 110 tools / 33 scopes** (canonical + memory/skill/agentSkill/githubSkill/pool/notion/obsidian/localCorpus/gamification/plugin modules)
|
||||
|
||||
@@ -1,6 +1,5 @@
|
||||
import { HTTP_STATUS, FETCH_TIMEOUT_MS } from "../config/constants.ts";
|
||||
import { getRegistryEntry } from "../config/providerRegistry.ts";
|
||||
import { resolveFetchStartTimeout } from "../utils/fetchStartTimeoutPolicy.ts";
|
||||
import {
|
||||
resolveAlternateFormat,
|
||||
type AlternateFormat,
|
||||
@@ -202,10 +201,6 @@ export type ExecuteInput = {
|
||||
* this to apply client-format-aware policies such as `</think>` close-marker
|
||||
* suppression. */
|
||||
clientResponseFormat?: string | null;
|
||||
/** True when upstream diagnostics may echo a video transcript. Executors must
|
||||
* preserve operational responses/errors while omitting those echoes from
|
||||
* retained logs. */
|
||||
videoTranscriptSensitive?: boolean;
|
||||
/** Callback to persist tokens that are proactively refreshed during execution.
|
||||
* Accepts a partial credentials patch (e.g. `{ accessToken, refreshToken }` or
|
||||
* `{ testStatus: "expired", isActive: false }`); the caller merges into the
|
||||
@@ -907,24 +902,9 @@ export class BaseExecutor {
|
||||
clampNestedThinkingBudget(transformedBody, thinkingBudgetClampedMax);
|
||||
}
|
||||
|
||||
// Timeout only covers response start; stream stalls are handled downstream.
|
||||
// #11526: streaming requests cap the headers-wait phase to a client-realistic
|
||||
// ceiling (see fetchStartTimeoutPolicy.ts) — non-streaming keeps the flat default.
|
||||
// Declared outside the try/catch below so the catch's TIMEOUT log (on the
|
||||
// error path) reports the same effective value the fetch actually used.
|
||||
const fetchStartTimeoutPolicy = resolveFetchStartTimeout({
|
||||
baseTimeoutMs: this.getTimeoutMs(),
|
||||
stream,
|
||||
});
|
||||
const fetchStartTimeoutMs = fetchStartTimeoutPolicy.timeoutMs;
|
||||
if (fetchStartTimeoutPolicy.capped) {
|
||||
log?.debug?.(
|
||||
"TIMEOUT",
|
||||
`fetch-start timeout capped ${fetchStartTimeoutPolicy.baseTimeoutMs}ms -> ${fetchStartTimeoutMs}ms (streaming)`
|
||||
);
|
||||
}
|
||||
|
||||
try {
|
||||
// Timeout only covers response start; stream stalls are handled downstream.
|
||||
const fetchStartTimeoutMs = this.getTimeoutMs();
|
||||
const fetchWithStartTimeout = async (requestUrl: string, requestOptions: RequestInit) => {
|
||||
// GHSA-4f49: guard here (not only next to the first buildUrl) so retries
|
||||
// and fallback URLs are validated too, before any bytes leave the host.
|
||||
@@ -1733,7 +1713,7 @@ export class BaseExecutor {
|
||||
// Distinguish timeout errors from other abort errors
|
||||
const err = error instanceof Error ? error : new Error(String(error));
|
||||
if (err.name === "TimeoutError") {
|
||||
log?.warn?.("TIMEOUT", `Fetch timeout after ${fetchStartTimeoutMs}ms on ${url}`);
|
||||
log?.warn?.("TIMEOUT", `Fetch timeout after ${this.getTimeoutMs()}ms on ${url}`);
|
||||
}
|
||||
lastError = err;
|
||||
if (!skipUpstreamRetry && urlIndex + 1 < fallbackCount) {
|
||||
|
||||
@@ -1,5 +1,4 @@
|
||||
import { randomUUID } from "node:crypto";
|
||||
import { redactVideoTranscriptSensitiveText } from "../../src/lib/guardrails/videoTranscriptLogRedaction.ts";
|
||||
import type { KeyHealth } from "../services/apiKeyRotator.ts";
|
||||
|
||||
import { DefaultExecutor } from "./default.ts";
|
||||
@@ -46,11 +45,6 @@ function asRecord(value: unknown): JsonRecord | null {
|
||||
return value && typeof value === "object" && !Array.isArray(value) ? (value as JsonRecord) : null;
|
||||
}
|
||||
|
||||
function retainGlmDiagnostic(error: unknown, input: ExecuteInput): string {
|
||||
const message = error instanceof Error ? error.message : String(error);
|
||||
return redactVideoTranscriptSensitiveText(message, input.videoTranscriptSensitive === true);
|
||||
}
|
||||
|
||||
function getEffectiveKey(credentials: ProviderCredentials): string {
|
||||
const extraKeys = (credentials.providerSpecificData?.extraApiKeys as string[] | undefined) ?? [];
|
||||
if (credentials.apiKey && credentials.connectionId && extraKeys.length > 0) {
|
||||
@@ -471,7 +465,6 @@ export class GlmExecutor extends DefaultExecutor {
|
||||
timeoutMs: STREAM_READINESS_TIMEOUT_MS,
|
||||
provider: this.provider,
|
||||
model: input.model,
|
||||
redactUpstreamDiagnosticForLog: input.videoTranscriptSensitive === true,
|
||||
log: input.log,
|
||||
});
|
||||
response = readiness.response;
|
||||
@@ -566,7 +559,7 @@ export class GlmExecutor extends DefaultExecutor {
|
||||
if (!isRetryableGlmFallbackError(error)) throw error;
|
||||
input.log?.debug?.(
|
||||
"GLM_FALLBACK",
|
||||
`${primaryTransport} error (${retainGlmDiagnostic(error, input)}); trying ${fallbackTransport}`
|
||||
`${primaryTransport} error (${error instanceof Error ? error.message : String(error)}); trying ${fallbackTransport}`
|
||||
);
|
||||
}
|
||||
|
||||
@@ -579,7 +572,7 @@ export class GlmExecutor extends DefaultExecutor {
|
||||
if (!primaryResult) throw error;
|
||||
input.log?.debug?.(
|
||||
"GLM_FALLBACK",
|
||||
`${fallbackTransport} fallback failed (${retainGlmDiagnostic(error, input)}); returning primary response`
|
||||
`${fallbackTransport} fallback failed (${error instanceof Error ? error.message : String(error)}); returning primary response`
|
||||
);
|
||||
}
|
||||
|
||||
|
||||
@@ -166,10 +166,6 @@ import {
|
||||
runWithCasGuard,
|
||||
} from "../services/tokenRefresh.ts";
|
||||
import { createRequestLogger } from "../utils/requestLogger.ts";
|
||||
import {
|
||||
redactVideoTranscriptSensitiveText,
|
||||
resolveVideoTranscriptLogSensitivity,
|
||||
} from "@/lib/guardrails/videoTranscriptLogRedaction";
|
||||
import { createPreparedRequestLogger, runWithCapture } from "../utils/providerRequestLogging.ts";
|
||||
import { summarizeToolSources } from "../utils/toolSources.ts";
|
||||
import { applyResponsesPreviousResponseIdPolicy } from "../utils/responsesStatePolicy.ts";
|
||||
@@ -528,21 +524,7 @@ export async function handleChatCore({
|
||||
skipResourcePressureGuard = false,
|
||||
reasoningTransportFallback = "drop",
|
||||
managedLease = null,
|
||||
videoTranscriptSensitive = false,
|
||||
videoTranscriptDescriptionFingerprints = [],
|
||||
}) {
|
||||
videoTranscriptSensitive =
|
||||
videoTranscriptSensitive ||
|
||||
resolveVideoTranscriptLogSensitivity({
|
||||
rawRequestBody: clientRawRequest?.body,
|
||||
processedBody: body,
|
||||
trustedDescriptionFingerprints: videoTranscriptDescriptionFingerprints,
|
||||
});
|
||||
const retainedErrorTextForLog = (error: unknown): string =>
|
||||
redactVideoTranscriptSensitiveText(
|
||||
error instanceof Error ? error.message : String(error),
|
||||
videoTranscriptSensitive
|
||||
);
|
||||
let { provider, model, extendedContext } = modelInfo;
|
||||
const resilienceSettings = resolveResilienceSettings(cachedSettings);
|
||||
if (!skipResourcePressureGuard) {
|
||||
@@ -655,7 +637,6 @@ export async function handleChatCore({
|
||||
apiKeyInfo,
|
||||
headers: clientRawRequest?.headers,
|
||||
log,
|
||||
videoTranscriptSensitive,
|
||||
});
|
||||
if (pluginGate.blocked === true) {
|
||||
return {
|
||||
@@ -740,7 +721,6 @@ export async function handleChatCore({
|
||||
effectiveServiceTier,
|
||||
startTime,
|
||||
log,
|
||||
videoTranscriptSensitive,
|
||||
});
|
||||
if (idempotencyHit) {
|
||||
return idempotencyHit;
|
||||
@@ -922,8 +902,6 @@ export async function handleChatCore({
|
||||
stage: "registered",
|
||||
correlationId,
|
||||
sessionTag: conversationId || null,
|
||||
videoTranscriptSensitive,
|
||||
videoTranscriptDescriptionFingerprints,
|
||||
}) || generateRequestId();
|
||||
|
||||
// Initialize rate limit settings from persisted DB (once, lazy)
|
||||
@@ -1056,8 +1034,6 @@ export async function handleChatCore({
|
||||
noLogEnabled,
|
||||
correlationId,
|
||||
modelPinned,
|
||||
videoTranscriptSensitive,
|
||||
videoTranscriptDescriptionFingerprints,
|
||||
// Resolved conversationId (open-sse/services/conversationTracker.ts) wins when
|
||||
// present — it's populated for every request now, not just ones where the
|
||||
// client explicitly sent x-omniroute-session-id. The raw header remains a
|
||||
@@ -1180,17 +1156,8 @@ export async function handleChatCore({
|
||||
model,
|
||||
provider: provider || undefined,
|
||||
connectionId: connectionId || credentials?.connectionId || undefined,
|
||||
videoTranscriptSensitive,
|
||||
videoTranscriptDescriptionFingerprints,
|
||||
});
|
||||
const pendingScope = {
|
||||
id: pendingRequestId,
|
||||
model,
|
||||
provider,
|
||||
connectionId: pendingConnId,
|
||||
videoTranscriptSensitive,
|
||||
videoTranscriptDescriptionFingerprints,
|
||||
};
|
||||
const pendingScope = { id: pendingRequestId, model, provider, connectionId: pendingConnId };
|
||||
const providerRequestCapture = createPreparedRequestLogger(reqLogger, pendingScope);
|
||||
// 0. Log client raw request (before format conversion)
|
||||
if (clientRawRequest) {
|
||||
@@ -1236,7 +1203,6 @@ export async function handleChatCore({
|
||||
apiKeyId: apiKeyInfo?.id ?? undefined,
|
||||
cacheDefaultMode: (apiKeyInfo as { cacheDefaultMode?: "legacy" | "bypass" } | null)
|
||||
?.cacheDefaultMode,
|
||||
videoTranscriptSensitive,
|
||||
});
|
||||
if (cacheHit) {
|
||||
return cacheHit;
|
||||
@@ -1516,7 +1482,8 @@ export async function handleChatCore({
|
||||
} catch (err) {
|
||||
log?.debug?.(
|
||||
"COMPRESSION",
|
||||
"Combo compression override lookup skipped: " + retainedErrorTextForLog(err)
|
||||
"Combo compression override lookup skipped: " +
|
||||
(err instanceof Error ? err.message : String(err))
|
||||
);
|
||||
}
|
||||
}
|
||||
@@ -1525,7 +1492,10 @@ export async function handleChatCore({
|
||||
const { listCompressionCombos } = await import("../../src/lib/db/compressionCombos.ts");
|
||||
namedCombos = buildNamedComboLookup(listCompressionCombos());
|
||||
} catch (err) {
|
||||
log?.debug?.("COMPRESSION", "Named combos load skipped: " + retainedErrorTextForLog(err));
|
||||
log?.debug?.(
|
||||
"COMPRESSION",
|
||||
"Named combos load skipped: " + (err instanceof Error ? err.message : String(err))
|
||||
);
|
||||
}
|
||||
// Phase 3: per-request override. Unknown values fall through in the resolver (never error).
|
||||
const compressionHeader = resolveCompressionHeader(clientRawRequest?.headers ?? null);
|
||||
@@ -1573,7 +1543,8 @@ export async function handleChatCore({
|
||||
} catch (err) {
|
||||
log?.debug?.(
|
||||
"COMPRESSION",
|
||||
"Default compression combo lookup skipped: " + retainedErrorTextForLog(err)
|
||||
"Default compression combo lookup skipped: " +
|
||||
(err instanceof Error ? err.message : String(err))
|
||||
);
|
||||
}
|
||||
}
|
||||
@@ -1615,7 +1586,10 @@ export async function handleChatCore({
|
||||
}
|
||||
}
|
||||
} catch (err) {
|
||||
log?.debug?.("COMPRESSION", "Output styles skipped: " + retainedErrorTextForLog(err));
|
||||
log?.debug?.(
|
||||
"COMPRESSION",
|
||||
"Output styles skipped: " + (err instanceof Error ? err.message : String(err))
|
||||
);
|
||||
}
|
||||
}
|
||||
const compressionInputBody = body as Record<string, unknown>;
|
||||
@@ -1925,7 +1899,8 @@ export async function handleChatCore({
|
||||
} catch (err) {
|
||||
log?.warn?.(
|
||||
"COMPRESSION",
|
||||
"Compression pipeline error (non-fatal): " + retainedErrorTextForLog(err)
|
||||
"Compression pipeline error (non-fatal): " +
|
||||
(err instanceof Error ? err.message : String(err))
|
||||
);
|
||||
}
|
||||
// --- End Modular Compression Pipeline ---
|
||||
@@ -1979,10 +1954,7 @@ export async function handleChatCore({
|
||||
`Combo context limit: ${resolved.limit} (source=${resolved.source})`
|
||||
);
|
||||
} catch (err) {
|
||||
log?.warn?.(
|
||||
"CONTEXT",
|
||||
"Failed to resolve combo limits for compression: " + retainedErrorTextForLog(err)
|
||||
);
|
||||
log?.warn?.("CONTEXT", "Failed to resolve combo limits for compression: " + err);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -2460,21 +2432,13 @@ export async function handleChatCore({
|
||||
try {
|
||||
const { runOnError } = await import("@/lib/plugins/hooks");
|
||||
await runOnError(
|
||||
{
|
||||
requestId: traceId,
|
||||
body,
|
||||
model,
|
||||
provider,
|
||||
apiKeyInfo,
|
||||
metadata: {},
|
||||
videoTranscriptSensitive,
|
||||
},
|
||||
{ requestId: traceId, body, model, provider, apiKeyInfo, metadata: {} },
|
||||
error instanceof Error ? error : new Error(String(error))
|
||||
);
|
||||
} catch (pluginErr) {
|
||||
log?.debug?.(
|
||||
"PLUGIN",
|
||||
`onError hook error (non-fatal): ${retainedErrorTextForLog(pluginErr)}`
|
||||
`onError hook error (non-fatal): ${pluginErr instanceof Error ? pluginErr.message : String(pluginErr)}`
|
||||
);
|
||||
}
|
||||
|
||||
@@ -2486,13 +2450,7 @@ export async function handleChatCore({
|
||||
const message = error?.message || "Invalid request";
|
||||
const errorType = typeof error?.errorType === "string" ? error.errorType : null;
|
||||
|
||||
log?.warn?.(
|
||||
"TRANSLATE",
|
||||
`Request translation failed: ${redactVideoTranscriptSensitiveText(
|
||||
message,
|
||||
videoTranscriptSensitive
|
||||
)}`
|
||||
);
|
||||
log?.warn?.("TRANSLATE", `Request translation failed: ${message}`);
|
||||
|
||||
if (errorType) {
|
||||
trackPendingRequest(model, provider, connectionId, false);
|
||||
@@ -2586,7 +2544,8 @@ export async function handleChatCore({
|
||||
// must never turn an otherwise valid translated request into a 500.
|
||||
log?.warn?.(
|
||||
"COMPRESSION",
|
||||
"Post-translation OmniGlyph skipped: " + retainedErrorTextForLog(error)
|
||||
"Post-translation OmniGlyph skipped: " +
|
||||
(error instanceof Error ? error.message : String(error))
|
||||
);
|
||||
}
|
||||
}
|
||||
@@ -2902,7 +2861,7 @@ export async function handleChatCore({
|
||||
}).catch((err: unknown): EnforceDecision => {
|
||||
log?.warn?.(
|
||||
"QUOTA_SHARE",
|
||||
`enforceQuotaShare failed; fail-open: ${retainedErrorTextForLog(err)}`
|
||||
`enforceQuotaShare failed; fail-open: ${err instanceof Error ? err.message : String(err)}`
|
||||
);
|
||||
return { kind: "allow" as const };
|
||||
});
|
||||
@@ -2944,7 +2903,7 @@ export async function handleChatCore({
|
||||
// Outer fail-open guard — should not be reached (inner .catch covers it)
|
||||
log?.warn?.(
|
||||
"QUOTA_SHARE",
|
||||
`[quotaShare] enforceQuotaShare unexpected error; fail-open: ${retainedErrorTextForLog(err)}`
|
||||
`[quotaShare] enforceQuotaShare unexpected error; fail-open: ${err instanceof Error ? err.message : String(err)}`
|
||||
);
|
||||
}
|
||||
}
|
||||
@@ -2956,7 +2915,7 @@ export async function handleChatCore({
|
||||
} catch (err) {
|
||||
log?.warn?.(
|
||||
"QUOTA_SHARE",
|
||||
`[quotaShare] could not set soft penalty on candidate: ${retainedErrorTextForLog(err)}`
|
||||
`[quotaShare] could not set soft penalty on candidate: ${err instanceof Error ? err.message : String(err)}`
|
||||
);
|
||||
}
|
||||
}
|
||||
@@ -2990,8 +2949,6 @@ export async function handleChatCore({
|
||||
let onPipelineStreamError: streamFailure.PipelineStreamErrorHandler | null = null;
|
||||
let onClientDisconnectFinalize:
|
||||
((event: { reason: string; duration: number }) => boolean) | null = null;
|
||||
const redactStreamDiagnosticsForLog =
|
||||
videoTranscriptSensitive || reqLogger.isVideoTranscriptSensitive();
|
||||
|
||||
// Create stream controller for disconnect detection
|
||||
const streamController = createStreamController({
|
||||
@@ -3023,7 +2980,6 @@ export async function handleChatCore({
|
||||
clientAbortSignal: clientRawRequest?.signal,
|
||||
allowCompletedToolHandoffGrace: isCodexResponsesEcho,
|
||||
clientDisconnectGracePeriodMs: STREAM_DISCONNECT_GRACE_PERIOD_MS,
|
||||
redactStreamDiagnosticsForLog,
|
||||
});
|
||||
|
||||
const dedupRequestBody = { ...translatedBody, model: `${provider}/${model}`, stream };
|
||||
@@ -3173,7 +3129,6 @@ export async function handleChatCore({
|
||||
userAgent
|
||||
),
|
||||
clientResponseFormat,
|
||||
videoTranscriptSensitive,
|
||||
onCredentialsRefreshed,
|
||||
skipUpstreamRetry,
|
||||
contextEditing: { enabled: contextEditingEnabled },
|
||||
@@ -3208,7 +3163,7 @@ export async function handleChatCore({
|
||||
invalidateCodexQuotaCache(String(attemptConnectionId));
|
||||
}
|
||||
} catch (err) {
|
||||
const errMessage = retainedErrorTextForLog(err);
|
||||
const errMessage = err instanceof Error ? err.message : String(err);
|
||||
log?.debug?.("CODEX", `Failed to persist codex quota state: ${errMessage}`);
|
||||
}
|
||||
}
|
||||
@@ -3490,7 +3445,6 @@ export async function handleChatCore({
|
||||
userAgent
|
||||
),
|
||||
clientResponseFormat,
|
||||
videoTranscriptSensitive,
|
||||
onCredentialsRefreshed,
|
||||
skipUpstreamRetry,
|
||||
contextEditing: { enabled: contextEditingEnabled },
|
||||
@@ -3723,9 +3677,7 @@ export async function handleChatCore({
|
||||
} catch (err) {
|
||||
// Fail-open at Tier 2: Tier 1 already enforced the model/global limit pre-dispatch.
|
||||
// A transient counter read error here must not break an otherwise-valid request.
|
||||
log?.warn?.("TOKEN_LIMIT", "Tier 2 token-limit check failed; allowing request", {
|
||||
error: retainedErrorTextForLog(err),
|
||||
});
|
||||
log?.warn?.("TOKEN_LIMIT", "Tier 2 token-limit check failed; allowing request", { err });
|
||||
}
|
||||
}
|
||||
|
||||
@@ -3744,9 +3696,7 @@ export async function handleChatCore({
|
||||
);
|
||||
}
|
||||
} catch (err) {
|
||||
log?.warn?.("GEMINI_RATE_LIMIT", "Pre-dispatch TPM check failed; allowing request", {
|
||||
error: retainedErrorTextForLog(err),
|
||||
});
|
||||
log?.warn?.("GEMINI_RATE_LIMIT", "Pre-dispatch TPM check failed; allowing request", { err });
|
||||
}
|
||||
}
|
||||
|
||||
@@ -3896,12 +3846,7 @@ export async function handleChatCore({
|
||||
failureStatus,
|
||||
upstreamErrorCode || (error instanceof Error && error.name ? error.name : "upstream_error")
|
||||
);
|
||||
console.log(
|
||||
`${COLORS.red}[ERROR] ${redactVideoTranscriptSensitiveText(
|
||||
failureMessage,
|
||||
videoTranscriptSensitive
|
||||
)}${COLORS.reset}`
|
||||
);
|
||||
console.log(`${COLORS.red}[ERROR] ${failureMessage}${COLORS.reset}`);
|
||||
if (stream && upstreamErrorCode) {
|
||||
const result = createStreamingErrorResult(
|
||||
failureStatus,
|
||||
@@ -4038,7 +3983,6 @@ export async function handleChatCore({
|
||||
upstreamExtraHeaders: buildUpstreamHeadersForExecute(retryModelId),
|
||||
clientHeaders: buildExecutorClientHeaders(clientRawRequest?.headers, userAgent),
|
||||
clientResponseFormat,
|
||||
videoTranscriptSensitive,
|
||||
onCredentialsRefreshed,
|
||||
skipUpstreamRetry: isCombo,
|
||||
contextEditing: { enabled: contextEditingEnabled },
|
||||
@@ -4068,13 +4012,9 @@ export async function handleChatCore({
|
||||
// executor throw). Don't swallow — the operator-visible signal "the user
|
||||
// saw 401 even though auth was actually fixed" is much more confusing
|
||||
// than the original 401 alone. Surface at error level with sanitization.
|
||||
const retainedRetryError = redactVideoTranscriptSensitiveText(
|
||||
sanitizeErrorMessage(retryErr),
|
||||
videoTranscriptSensitive
|
||||
);
|
||||
log?.error?.(
|
||||
"TOKEN",
|
||||
`${provider?.toUpperCase()} | retry after refresh failed: ${retainedRetryError}`
|
||||
`${provider?.toUpperCase()} | retry after refresh failed: ${sanitizeErrorMessage(retryErr)}`
|
||||
);
|
||||
}
|
||||
} else {
|
||||
@@ -4191,11 +4131,6 @@ export async function handleChatCore({
|
||||
|
||||
if (signatureRecovery.succeeded) break providerFailure;
|
||||
|
||||
const retainedProviderMessage = redactVideoTranscriptSensitiveText(
|
||||
message,
|
||||
videoTranscriptSensitive
|
||||
);
|
||||
|
||||
// #10281 — tiny-budget reasoning probes (e.g. Claude Code's `/model` check
|
||||
// sends `max_tokens: 1`): the model burns the whole budget on thinking, and
|
||||
// some upstreams (e.g. api.cline.bot for deepseek-v4-flash) answer the empty
|
||||
@@ -4218,7 +4153,7 @@ export async function handleChatCore({
|
||||
});
|
||||
log?.warn?.(
|
||||
"PROBE",
|
||||
`Reasoning probe (max_tokens < ${REASONING_BUFFER_MIN_TRIGGER}) answered with truncated 200 — upstream reported "${retainedProviderMessage}"`
|
||||
`Reasoning probe (max_tokens < ${REASONING_BUFFER_MIN_TRIGGER}) answered with truncated 200 — upstream reported "${message}"`
|
||||
);
|
||||
break providerFailure;
|
||||
}
|
||||
@@ -4247,7 +4182,7 @@ export async function handleChatCore({
|
||||
{
|
||||
testStatus: "banned",
|
||||
isActive: false,
|
||||
lastError: retainedProviderMessage,
|
||||
lastError: message,
|
||||
lastErrorType: errorType,
|
||||
errorCode: String(statusCode),
|
||||
},
|
||||
@@ -4278,7 +4213,7 @@ export async function handleChatCore({
|
||||
) {
|
||||
await updateProviderConnection(errorConnectionId, {
|
||||
lastErrorType: errorType,
|
||||
lastError: retainedProviderMessage,
|
||||
lastError: message,
|
||||
errorCode: statusCode,
|
||||
});
|
||||
console.warn(
|
||||
@@ -4291,7 +4226,7 @@ export async function handleChatCore({
|
||||
{
|
||||
testStatus: "deactivated",
|
||||
isActive: false,
|
||||
lastError: retainedProviderMessage,
|
||||
lastError: message,
|
||||
lastErrorType: errorType,
|
||||
errorCode: String(statusCode),
|
||||
},
|
||||
@@ -4315,7 +4250,7 @@ export async function handleChatCore({
|
||||
errorConnectionId,
|
||||
{
|
||||
testStatus: "credits_exhausted",
|
||||
lastError: retainedProviderMessage,
|
||||
lastError: message,
|
||||
lastErrorType: errorType,
|
||||
errorCode: String(statusCode),
|
||||
},
|
||||
@@ -4363,7 +4298,7 @@ export async function handleChatCore({
|
||||
rateLimitedUntil: kimiRateLimitResetAt,
|
||||
backoffLevel: 0,
|
||||
lastErrorType: PROVIDER_ERROR_TYPES.RATE_LIMITED,
|
||||
lastError: retainedProviderMessage,
|
||||
lastError: message,
|
||||
errorCode: statusCode,
|
||||
});
|
||||
console.warn(
|
||||
@@ -4393,7 +4328,7 @@ export async function handleChatCore({
|
||||
errorConnectionId,
|
||||
{
|
||||
testStatus: "credits_exhausted",
|
||||
lastError: retainedProviderMessage,
|
||||
lastError: message,
|
||||
lastErrorType: errorType,
|
||||
errorCode: String(statusCode),
|
||||
},
|
||||
@@ -4409,14 +4344,14 @@ export async function handleChatCore({
|
||||
// Normal 401 (token/session auth issue): keep account active for refresh/re-auth.
|
||||
await updateProviderConnection(errorConnectionId, {
|
||||
lastErrorType: errorType,
|
||||
lastError: retainedProviderMessage,
|
||||
lastError: message,
|
||||
errorCode: statusCode,
|
||||
});
|
||||
} else if (errorType === PROVIDER_ERROR_TYPES.OAUTH_INVALID_TOKEN) {
|
||||
// OAuth 401 with invalid credentials - token refresh can recover
|
||||
await updateProviderConnection(errorConnectionId, {
|
||||
lastErrorType: errorType,
|
||||
lastError: retainedProviderMessage,
|
||||
lastError: message,
|
||||
errorCode: statusCode,
|
||||
});
|
||||
console.warn(
|
||||
@@ -4426,7 +4361,7 @@ export async function handleChatCore({
|
||||
// Cloud Code 403 with stale project: not a ban, keep account active.
|
||||
await updateProviderConnection(errorConnectionId, {
|
||||
lastErrorType: errorType,
|
||||
lastError: retainedProviderMessage,
|
||||
lastError: message,
|
||||
errorCode: statusCode,
|
||||
});
|
||||
console.warn(
|
||||
@@ -4442,7 +4377,7 @@ export async function handleChatCore({
|
||||
const geoCooldownMs = COOLDOWN_MS.geoBlocked ?? 24 * 60 * 60 * 1000;
|
||||
await updateProviderConnection(errorConnectionId, {
|
||||
lastErrorType: errorType,
|
||||
lastError: retainedProviderMessage,
|
||||
lastError: message,
|
||||
errorCode: statusCode,
|
||||
});
|
||||
// T-PROBE: the 24h exclusion is a routing mutation — a probe must
|
||||
@@ -4467,7 +4402,7 @@ export async function handleChatCore({
|
||||
const byopCooldownMs = COOLDOWN_MS.gcpProjectRequired ?? 24 * 60 * 60 * 1000;
|
||||
await updateProviderConnection(errorConnectionId, {
|
||||
lastErrorType: errorType,
|
||||
lastError: retainedProviderMessage,
|
||||
lastError: message,
|
||||
errorCode: statusCode,
|
||||
});
|
||||
try {
|
||||
@@ -4513,13 +4448,7 @@ export async function handleChatCore({
|
||||
}).catch(() => {});
|
||||
|
||||
const errMsg = formatProviderError(new Error(message), provider, model, statusCode);
|
||||
const retainedErrMsg = formatProviderError(
|
||||
new Error(retainedProviderMessage),
|
||||
provider,
|
||||
model,
|
||||
statusCode
|
||||
);
|
||||
console.log(`${COLORS.red}[ERROR] ${retainedErrMsg}${COLORS.reset}`);
|
||||
console.log(`${COLORS.red}[ERROR] ${errMsg}${COLORS.reset}`);
|
||||
|
||||
// Log Antigravity retry time if available
|
||||
if (retryAfterMs && provider === "antigravity") {
|
||||
@@ -4846,11 +4775,12 @@ export async function handleChatCore({
|
||||
}
|
||||
}
|
||||
} catch (retryErr) {
|
||||
const retainedRetryError = redactVideoTranscriptSensitiveText(
|
||||
retryErr instanceof Error ? retryErr.message : String(retryErr),
|
||||
videoTranscriptSensitive
|
||||
log?.warn?.(
|
||||
"RETRY",
|
||||
`clinepass retry failed: ${
|
||||
retryErr instanceof Error ? retryErr.message : String(retryErr)
|
||||
}`
|
||||
);
|
||||
log?.warn?.("RETRY", `clinepass retry failed: ${retainedRetryError}`);
|
||||
}
|
||||
}
|
||||
if (envError) {
|
||||
@@ -5069,7 +4999,6 @@ export async function handleChatCore({
|
||||
cacheReasoningFromAssistantMessage(msg, provider, model, {
|
||||
scope: reasoningCacheScope,
|
||||
historyMessages: Array.isArray(historyMessages) ? historyMessages : [],
|
||||
videoTranscriptSensitive,
|
||||
});
|
||||
}
|
||||
} catch {
|
||||
@@ -5121,18 +5050,12 @@ export async function handleChatCore({
|
||||
);
|
||||
|
||||
if (memoryOwnerId && memorySettings?.enabled && memorySettings.maxTokens > 0) {
|
||||
const requestMemoryText = extractMemoryTextFromRequestBody(
|
||||
body as Record<string, unknown>,
|
||||
videoTranscriptSensitive,
|
||||
{ trustedDescriptionFingerprints: videoTranscriptDescriptionFingerprints }
|
||||
);
|
||||
const requestMemoryText = extractMemoryTextFromRequestBody(body as Record<string, unknown>);
|
||||
if (requestMemoryText) {
|
||||
extractFacts(requestMemoryText, memoryOwnerId, pipelineSessionId);
|
||||
}
|
||||
|
||||
const memoryText = videoTranscriptSensitive
|
||||
? ""
|
||||
: extractMemoryTextFromResponse(memoryExtractionResponse);
|
||||
const memoryText = extractMemoryTextFromResponse(memoryExtractionResponse);
|
||||
if (memoryText) {
|
||||
extractFacts(memoryText, memoryOwnerId, pipelineSessionId);
|
||||
}
|
||||
@@ -5172,7 +5095,6 @@ export async function handleChatCore({
|
||||
provider,
|
||||
responsePayloadFormat,
|
||||
clientResponseFormat,
|
||||
videoTranscriptSensitive,
|
||||
});
|
||||
const postCallGuardrails = await guardrailRegistry.runPostCallHooks(
|
||||
translatedResponse,
|
||||
@@ -5214,10 +5136,7 @@ export async function handleChatCore({
|
||||
}
|
||||
log?.warn?.(
|
||||
"GUARDRAIL",
|
||||
`Response blocked by ${postCallGuardrails.guardrail || "guardrail"}: ${redactVideoTranscriptSensitiveText(
|
||||
guardrailMessage,
|
||||
videoTranscriptSensitive
|
||||
)}`
|
||||
`Response blocked by ${postCallGuardrails.guardrail || "guardrail"}: ${guardrailMessage}`
|
||||
);
|
||||
finalizePendingScope(pendingScope, {
|
||||
providerResponse: responseBody,
|
||||
@@ -5320,7 +5239,6 @@ export async function handleChatCore({
|
||||
apiKeyId: apiKeyInfo?.id ?? undefined,
|
||||
usage,
|
||||
log,
|
||||
videoTranscriptSensitive,
|
||||
});
|
||||
|
||||
// ── Phase 9.2: Save for idempotency ──
|
||||
@@ -5400,7 +5318,6 @@ export async function handleChatCore({
|
||||
apiKeyInfo,
|
||||
headers: clientRawRequest?.headers,
|
||||
response: { status: 200, data: translatedResponse },
|
||||
videoTranscriptSensitive,
|
||||
});
|
||||
|
||||
// Routing event (feedback foundation) — fire-and-forget, cheap.
|
||||
@@ -5476,7 +5393,6 @@ export async function handleChatCore({
|
||||
provider,
|
||||
model,
|
||||
log,
|
||||
redactUpstreamDiagnosticForLog: videoTranscriptSensitive,
|
||||
});
|
||||
if (streamReadiness.ok === false) {
|
||||
const { response: failureResponse, reason } = streamReadiness;
|
||||
@@ -5608,7 +5524,6 @@ export async function handleChatCore({
|
||||
cacheReasoningFromAssistantMessage(msg, provider, model, {
|
||||
scope: reasoningCacheScope,
|
||||
historyMessages: Array.isArray(historyMessages) ? historyMessages : [],
|
||||
videoTranscriptSensitive,
|
||||
});
|
||||
}
|
||||
} catch {
|
||||
@@ -5641,7 +5556,6 @@ export async function handleChatCore({
|
||||
status: normalizedStreamStatus,
|
||||
error: streamError,
|
||||
errorCode: streamErrorCode,
|
||||
videoTranscriptSensitive,
|
||||
});
|
||||
|
||||
// Track cache token metrics for streaming responses
|
||||
@@ -5764,20 +5678,14 @@ export async function handleChatCore({
|
||||
memorySettings.maxTokens > 0 &&
|
||||
streamStatus === 200
|
||||
) {
|
||||
const requestMemoryText = extractMemoryTextFromRequestBody(
|
||||
body as Record<string, unknown>,
|
||||
videoTranscriptSensitive,
|
||||
{ trustedDescriptionFingerprints: videoTranscriptDescriptionFingerprints }
|
||||
);
|
||||
const requestMemoryText = extractMemoryTextFromRequestBody(body as Record<string, unknown>);
|
||||
if (requestMemoryText) {
|
||||
extractFacts(requestMemoryText, memoryOwnerId, pipelineSessionId);
|
||||
}
|
||||
|
||||
const streamedMemoryText = videoTranscriptSensitive
|
||||
? ""
|
||||
: extractMemoryTextFromResponse(
|
||||
(streamResponseBody ?? null) as Record<string, unknown> | null
|
||||
);
|
||||
const streamedMemoryText = extractMemoryTextFromResponse(
|
||||
(streamResponseBody ?? null) as Record<string, unknown> | null
|
||||
);
|
||||
if (streamedMemoryText) {
|
||||
extractFacts(streamedMemoryText, memoryOwnerId, pipelineSessionId);
|
||||
}
|
||||
@@ -5794,7 +5702,6 @@ export async function handleChatCore({
|
||||
apiKeyId: apiKeyInfo?.id ?? undefined,
|
||||
streamUsage,
|
||||
log,
|
||||
videoTranscriptSensitive,
|
||||
});
|
||||
|
||||
// Plugin onStreamComplete hook — fire-and-forget, fail-open (#9571)
|
||||
@@ -5863,8 +5770,7 @@ export async function handleChatCore({
|
||||
// openai-responses → openai translation still wants the namespace identity
|
||||
// map for #7936-style round-trip closure when the client also speaks
|
||||
// Responses (Codex CLI).
|
||||
requestToolIdentityMap,
|
||||
redactStreamDiagnosticsForLog
|
||||
requestToolIdentityMap
|
||||
);
|
||||
} else if (needsTranslation(targetFormat, clientResponseFormat)) {
|
||||
// Standard translation for other providers
|
||||
@@ -5894,8 +5800,7 @@ export async function handleChatCore({
|
||||
clientResponseFormat,
|
||||
}),
|
||||
customToolNames,
|
||||
requestToolIdentityMap,
|
||||
redactStreamDiagnosticsForLog
|
||||
requestToolIdentityMap
|
||||
);
|
||||
} else {
|
||||
log?.debug?.("STREAM", `Standard passthrough mode`);
|
||||
@@ -5910,8 +5815,7 @@ export async function handleChatCore({
|
||||
apiKeyInfo,
|
||||
handleStreamFailure,
|
||||
clientResponseFormat,
|
||||
requestToolIdentityMap,
|
||||
redactStreamDiagnosticsForLog
|
||||
requestToolIdentityMap
|
||||
);
|
||||
}
|
||||
|
||||
@@ -5923,7 +5827,6 @@ export async function handleChatCore({
|
||||
clientRawRequestHeaders: clientRawRequest?.headers,
|
||||
clientResponseFormat,
|
||||
echoModel,
|
||||
redactStreamDiagnosticsForLog,
|
||||
responseHeaders,
|
||||
});
|
||||
|
||||
@@ -5939,7 +5842,6 @@ export async function handleChatCore({
|
||||
apiKeyInfo,
|
||||
headers: clientRawRequest?.headers,
|
||||
response: { status: 200, streamed: true },
|
||||
videoTranscriptSensitive,
|
||||
});
|
||||
|
||||
return {
|
||||
|
||||
@@ -15,10 +15,6 @@ import { logAuditEvent } from "@/lib/compliance";
|
||||
import { emit } from "@/lib/events/eventBus";
|
||||
import type { RequestCompletedPayload, RequestFailedPayload } from "@/lib/events/types";
|
||||
import { saveCallLog } from "@/lib/usageDb";
|
||||
import {
|
||||
VIDEO_TRANSCRIPT_LOG_OMISSION_MARKER,
|
||||
VIDEO_TRANSCRIPT_REDACTION_PIPELINE_KEY,
|
||||
} from "@/lib/guardrails/videoTranscriptLogRedaction";
|
||||
import { FORMATS } from "../../translator/formats.ts";
|
||||
import { takeEarlyKeepaliveBytes } from "../../utils/earlyKeepaliveByteBuffer.ts";
|
||||
import { cloneBoundedChatLogPayload, truncateForLog } from "./logTruncation.ts";
|
||||
@@ -72,13 +68,7 @@ export type PersistAttemptLogsContext = {
|
||||
model: string | null | undefined;
|
||||
skillRequestId: string;
|
||||
detailedLoggingEnabled: boolean;
|
||||
reqLogger:
|
||||
| {
|
||||
getPipelinePayloads?: () => Record<string, unknown> | undefined;
|
||||
isVideoTranscriptSensitive?: () => boolean;
|
||||
}
|
||||
| null
|
||||
| undefined;
|
||||
reqLogger: { getPipelinePayloads?: () => Record<string, unknown> | undefined } | null | undefined;
|
||||
pendingRequestId: unknown;
|
||||
clientRawRequest: { endpoint?: string } | null | undefined;
|
||||
requestedModel: unknown;
|
||||
@@ -99,34 +89,12 @@ export type PersistAttemptLogsContext = {
|
||||
* explicitly present (never synthesized from skillRequestId) — persisted as call_logs.session_tag
|
||||
* for per-session cost attribution. */
|
||||
sessionTag?: string | null;
|
||||
/** Trusted request state derived from the Video Bridge guardrail result. */
|
||||
videoTranscriptSensitive?: boolean;
|
||||
/** Exact SHA-256 identities of transcript descriptions generated by the guardrail. */
|
||||
videoTranscriptDescriptionFingerprints?: readonly string[];
|
||||
};
|
||||
|
||||
function toConnectionId(value: unknown): string | null {
|
||||
return typeof value === "string" && value.trim().length > 0 ? value.trim() : null;
|
||||
}
|
||||
|
||||
function omitSensitivePipelineResponse(value: unknown): Record<string, unknown> {
|
||||
const record =
|
||||
value && typeof value === "object" && !Array.isArray(value)
|
||||
? (value as Record<string, unknown>)
|
||||
: {};
|
||||
return {
|
||||
...(typeof record.timestamp === "string" ? { timestamp: record.timestamp } : {}),
|
||||
...(typeof record.status === "number" && Number.isFinite(record.status)
|
||||
? { status: record.status }
|
||||
: {}),
|
||||
...(record.headers !== undefined ? { headers: VIDEO_TRANSCRIPT_LOG_OMISSION_MARKER } : {}),
|
||||
...(record.statusText !== undefined
|
||||
? { statusText: VIDEO_TRANSCRIPT_LOG_OMISSION_MARKER }
|
||||
: {}),
|
||||
body: VIDEO_TRANSCRIPT_LOG_OMISSION_MARKER,
|
||||
};
|
||||
}
|
||||
|
||||
function buildAccountRotationMeta(
|
||||
provider: string | null | undefined,
|
||||
initialConnectionId: string | null,
|
||||
@@ -236,8 +204,6 @@ export function persistAttemptLogs(args: PersistAttemptLogsArgs, ctx: PersistAtt
|
||||
correlationId,
|
||||
modelPinned,
|
||||
sessionTag,
|
||||
videoTranscriptSensitive,
|
||||
videoTranscriptDescriptionFingerprints,
|
||||
} = ctx;
|
||||
const initialConnectionId = toConnectionId(connectionId);
|
||||
const finalConnectionId = toConnectionId(credentials?.connectionId) || initialConnectionId;
|
||||
@@ -246,21 +212,8 @@ export function persistAttemptLogs(args: PersistAttemptLogsArgs, ctx: PersistAtt
|
||||
initialConnectionId,
|
||||
finalConnectionId
|
||||
);
|
||||
const transcriptSensitive =
|
||||
videoTranscriptSensitive === true || reqLogger?.isVideoTranscriptSensitive?.() === true;
|
||||
const descriptionLogContext = {
|
||||
trustedDescriptionFingerprints: videoTranscriptDescriptionFingerprints ?? [],
|
||||
};
|
||||
|
||||
const detectedProviderWarnings = extractProviderWarnings(
|
||||
providerResponse,
|
||||
clientResponse,
|
||||
responseBody
|
||||
);
|
||||
const providerWarnings =
|
||||
transcriptSensitive && detectedProviderWarnings.length > 0
|
||||
? [VIDEO_TRANSCRIPT_LOG_OMISSION_MARKER]
|
||||
: detectedProviderWarnings;
|
||||
const providerWarnings = extractProviderWarnings(providerResponse, clientResponse, responseBody);
|
||||
if (providerWarnings.length > 0) {
|
||||
logAuditEvent({
|
||||
action: "provider.warning",
|
||||
@@ -274,7 +227,6 @@ export function persistAttemptLogs(args: PersistAttemptLogsArgs, ctx: PersistAtt
|
||||
model,
|
||||
connectionId: finalConnectionId,
|
||||
httpStatus: status,
|
||||
warningCount: detectedProviderWarnings.length,
|
||||
warnings: providerWarnings,
|
||||
},
|
||||
});
|
||||
@@ -321,51 +273,8 @@ export function persistAttemptLogs(args: PersistAttemptLogsArgs, ctx: PersistAtt
|
||||
};
|
||||
}
|
||||
}
|
||||
if (transcriptSensitive) {
|
||||
pipelinePayloads[VIDEO_TRANSCRIPT_REDACTION_PIPELINE_KEY] = true;
|
||||
for (const requestStage of [
|
||||
"clientRawRequest",
|
||||
"openaiRequest",
|
||||
"providerRequest",
|
||||
] as const) {
|
||||
if (pipelinePayloads[requestStage] !== undefined) {
|
||||
pipelinePayloads[requestStage] = cloneBoundedChatLogPayload(
|
||||
pipelinePayloads[requestStage],
|
||||
0,
|
||||
requestStage !== "clientRawRequest" ? descriptionLogContext : {}
|
||||
) as Record<string, unknown>;
|
||||
}
|
||||
}
|
||||
if (pipelinePayloads.providerResponse !== undefined || providerResponse !== undefined) {
|
||||
pipelinePayloads.providerResponse = omitSensitivePipelineResponse(
|
||||
pipelinePayloads.providerResponse ?? providerResponse
|
||||
);
|
||||
}
|
||||
if (pipelinePayloads.clientResponse !== undefined || clientResponse !== undefined) {
|
||||
pipelinePayloads.clientResponse = omitSensitivePipelineResponse(
|
||||
pipelinePayloads.clientResponse ?? clientResponse
|
||||
);
|
||||
}
|
||||
if (pipelinePayloads.error !== undefined || error) {
|
||||
const errorRecord =
|
||||
pipelinePayloads.error && typeof pipelinePayloads.error === "object"
|
||||
? pipelinePayloads.error
|
||||
: {};
|
||||
pipelinePayloads.error = {
|
||||
...(typeof errorRecord.timestamp === "string"
|
||||
? { timestamp: errorRecord.timestamp }
|
||||
: {}),
|
||||
message: VIDEO_TRANSCRIPT_LOG_OMISSION_MARKER,
|
||||
};
|
||||
}
|
||||
delete pipelinePayloads.streamChunks;
|
||||
}
|
||||
}
|
||||
|
||||
const responseBodyForLog = transcriptSensitive
|
||||
? VIDEO_TRANSCRIPT_LOG_OMISSION_MARKER
|
||||
: responseBody;
|
||||
|
||||
saveCallLog({
|
||||
id: pendingRequestId,
|
||||
method: "POST",
|
||||
@@ -381,12 +290,10 @@ export function persistAttemptLogs(args: PersistAttemptLogsArgs, ctx: PersistAtt
|
||||
attachLogMeta(truncateForLog(body as Record<string, unknown>), {
|
||||
...accountRotationMeta,
|
||||
claudePromptCache: claudeCacheMeta,
|
||||
}),
|
||||
0,
|
||||
descriptionLogContext
|
||||
})
|
||||
),
|
||||
responseBody: cloneBoundedChatLogPayload(
|
||||
attachLogMeta(truncateForLog(responseBodyForLog as Record<string, unknown>), {
|
||||
attachLogMeta(truncateForLog(responseBody as Record<string, unknown>), {
|
||||
...accountRotationMeta,
|
||||
claudePromptCache: claudeCacheMeta
|
||||
? {
|
||||
@@ -398,7 +305,7 @@ export function persistAttemptLogs(args: PersistAttemptLogsArgs, ctx: PersistAtt
|
||||
claudePromptCacheUsage: claudeCacheUsageMeta,
|
||||
})
|
||||
),
|
||||
error: transcriptSensitive && error ? VIDEO_TRANSCRIPT_LOG_OMISSION_MARKER : error || null,
|
||||
error: error || null,
|
||||
sourceFormat,
|
||||
targetFormat,
|
||||
comboName,
|
||||
@@ -425,7 +332,7 @@ export function persistAttemptLogs(args: PersistAttemptLogsArgs, ctx: PersistAtt
|
||||
const lifecycle = resolveRequestLifecycleEvent({
|
||||
traceId,
|
||||
status,
|
||||
error: transcriptSensitive && error ? VIDEO_TRANSCRIPT_LOG_OMISSION_MARKER : error,
|
||||
error,
|
||||
model,
|
||||
provider,
|
||||
comboName,
|
||||
|
||||
@@ -17,10 +17,8 @@
|
||||
*/
|
||||
|
||||
import { getExecutor } from "../../executors/index.ts";
|
||||
import type { ExecuteInput } from "../../executors/base.ts";
|
||||
import { isCliproxyapiDeepModeEnabled } from "../../executors/cliproxyapi.ts";
|
||||
import { isDarioDeepModeEnabled } from "../../executors/dario.ts";
|
||||
import { redactVideoTranscriptSensitiveText } from "../../../src/lib/guardrails/videoTranscriptLogRedaction.ts";
|
||||
import { getCachedSettings } from "@/lib/db/readCache";
|
||||
import { getUpstreamProxyConfigCached } from "./comboContextCache.ts";
|
||||
import type { FallbackBackend } from "@/lib/db/upstreamProxy";
|
||||
@@ -50,11 +48,6 @@ function parseFallbackCodes(raw: unknown): number[] | null {
|
||||
return parsed.length > 0 ? parsed : null;
|
||||
}
|
||||
|
||||
function retainExecutorDiagnostic(error: unknown, input: ExecuteInput): string {
|
||||
const message = error instanceof Error ? error.message : String(error);
|
||||
return redactVideoTranscriptSensitiveText(message, input.videoTranscriptSensitive === true);
|
||||
}
|
||||
|
||||
/**
|
||||
* Reads the CLIProxyAPI-related settings shared by both the direct
|
||||
* `mode: "cliproxyapi"` passthrough leg and the `mode: "fallback"` retry leg:
|
||||
@@ -162,20 +155,25 @@ export async function resolveExecutorWithProxy(
|
||||
const isRetryableStatus = (s: number) => fallbackCodes.includes(s) || s === 0;
|
||||
|
||||
const wrapper = Object.create(nativeExec);
|
||||
wrapper.execute = async (input: ExecuteInput) => {
|
||||
wrapper.execute = async (input: {
|
||||
model: string;
|
||||
body: unknown;
|
||||
stream: boolean;
|
||||
credentials: unknown;
|
||||
signal?: AbortSignal | null;
|
||||
log?: unknown;
|
||||
upstreamExtraHeaders?: Record<string, string> | null;
|
||||
}) => {
|
||||
let result;
|
||||
try {
|
||||
result = await nativeExec.execute(input);
|
||||
} catch (err) {
|
||||
const errMsg = retainExecutorDiagnostic(err, input);
|
||||
log?.info?.(
|
||||
"UPSTREAM_PROXY",
|
||||
`${prov} native error (${errMsg}), retrying via ${backendLabel}`
|
||||
);
|
||||
const errMsg = err instanceof Error ? err.message : String(err);
|
||||
log?.info?.("UPSTREAM_PROXY", `${prov} native error (${errMsg}), retrying via ${backendLabel}`);
|
||||
try {
|
||||
return await proxyExec.execute(input);
|
||||
} catch (proxyErr) {
|
||||
const proxyMsg = retainExecutorDiagnostic(proxyErr, input);
|
||||
const proxyMsg = proxyErr instanceof Error ? proxyErr.message : String(proxyErr);
|
||||
log?.error?.("UPSTREAM_PROXY", `${prov} ${backendLabel} fallback also failed: ${proxyMsg}`);
|
||||
throw proxyErr;
|
||||
}
|
||||
@@ -191,7 +189,7 @@ export async function resolveExecutorWithProxy(
|
||||
try {
|
||||
return await proxyExec.execute(input);
|
||||
} catch (proxyErr) {
|
||||
const proxyMsg = retainExecutorDiagnostic(proxyErr, input);
|
||||
const proxyMsg = proxyErr instanceof Error ? proxyErr.message : String(proxyErr);
|
||||
log?.error?.("UPSTREAM_PROXY", `${prov} ${backendLabel} fallback also failed: ${proxyMsg}`);
|
||||
throw proxyErr;
|
||||
}
|
||||
|
||||
@@ -124,7 +124,6 @@ export async function checkIdempotencyCache({
|
||||
effectiveServiceTier,
|
||||
startTime,
|
||||
log,
|
||||
videoTranscriptSensitive,
|
||||
}: {
|
||||
clientRawRequest: IdempotencyRequest;
|
||||
provider: string;
|
||||
@@ -133,9 +132,7 @@ export async function checkIdempotencyCache({
|
||||
effectiveServiceTier: EffectiveServiceTier | null | undefined;
|
||||
startTime: number;
|
||||
log: LoggerLike;
|
||||
videoTranscriptSensitive: boolean;
|
||||
}): Promise<{ hit: { success: true; response: Response } | null; idempotencyKey: string | null }> {
|
||||
if (videoTranscriptSensitive) return { hit: null, idempotencyKey: null };
|
||||
// NEXA fusion-idempotency fix: namespace the raw header key (see composeIdempotencyKey).
|
||||
const rawIdempotencyKey = getIdempotencyKey(clientRawRequest?.headers);
|
||||
const idempotencyKey = composeIdempotencyKey({
|
||||
|
||||
@@ -5,10 +5,6 @@ import {
|
||||
getChatLogMaxObjectKeys,
|
||||
getChatLogMaxBodyBytes,
|
||||
} from "@/lib/logEnv";
|
||||
import {
|
||||
omitVideoTranscriptForLog,
|
||||
type VideoTranscriptLogContext,
|
||||
} from "@/lib/guardrails/videoTranscriptLogRedaction";
|
||||
import { estimateSizeFast } from "../../utils/estimateSize.ts";
|
||||
|
||||
export const MEMORY_EXTRACTION_TEXT_LIMIT = 64 * 1024;
|
||||
@@ -26,7 +22,7 @@ export function truncateChatLogText(value: string): string {
|
||||
return `${head}\n[...truncated ${value.length - limit} chars...]\n${tail}`;
|
||||
}
|
||||
|
||||
function cloneBoundedChatLogPayloadValue(value: unknown, depth = 0): unknown {
|
||||
export function cloneBoundedChatLogPayload(value: unknown, depth = 0): unknown {
|
||||
if (value === null || value === undefined) return value;
|
||||
if (typeof value === "string") return truncateChatLogText(value);
|
||||
if (typeof value !== "object") return value;
|
||||
@@ -36,7 +32,7 @@ function cloneBoundedChatLogPayloadValue(value: unknown, depth = 0): unknown {
|
||||
|
||||
if (Array.isArray(value)) {
|
||||
const retained = value.length > maxTailItems ? value.slice(-maxTailItems) : value;
|
||||
const cloned = retained.map((item) => cloneBoundedChatLogPayloadValue(item, depth + 1));
|
||||
const cloned = retained.map((item) => cloneBoundedChatLogPayload(item, depth + 1));
|
||||
if (value.length > maxTailItems) {
|
||||
return [
|
||||
{
|
||||
@@ -51,11 +47,10 @@ function cloneBoundedChatLogPayloadValue(value: unknown, depth = 0): unknown {
|
||||
}
|
||||
|
||||
const result: Record<string, unknown> = {};
|
||||
const record = value as Record<string, unknown>;
|
||||
const entries = Object.entries(record);
|
||||
const entries = Object.entries(value as Record<string, unknown>);
|
||||
const maxKeys = getChatLogMaxObjectKeys();
|
||||
for (const [key, item] of maxKeys > 0 ? entries.slice(0, maxKeys) : entries) {
|
||||
result[key] = cloneBoundedChatLogPayloadValue(item, depth + 1);
|
||||
result[key] = cloneBoundedChatLogPayload(item, depth + 1);
|
||||
}
|
||||
if (maxKeys > 0 && entries.length > maxKeys) {
|
||||
result._omniroute_truncated_keys = entries.length - maxKeys;
|
||||
@@ -63,16 +58,6 @@ function cloneBoundedChatLogPayloadValue(value: unknown, depth = 0): unknown {
|
||||
return result;
|
||||
}
|
||||
|
||||
export function cloneBoundedChatLogPayload(
|
||||
value: unknown,
|
||||
depth = 0,
|
||||
descriptionContext: VideoTranscriptLogContext = {}
|
||||
): unknown {
|
||||
const transcriptSafeValue =
|
||||
depth === 0 ? omitVideoTranscriptForLog(value, descriptionContext) : value;
|
||||
return cloneBoundedChatLogPayloadValue(transcriptSafeValue, depth);
|
||||
}
|
||||
|
||||
/**
|
||||
* Truncate a large object for logging. If its JSON representation exceeds
|
||||
* getChatLogMaxBodyBytes() (default 1MB; CHAT_LOG_MAX_BODY_KB env override),
|
||||
|
||||
@@ -1,39 +1,5 @@
|
||||
import {
|
||||
containsVideoTranscriptForLog,
|
||||
omitVideoTranscriptDerivedTextForMemory,
|
||||
type VideoTranscriptLogContext,
|
||||
} from "../../../src/lib/guardrails/videoTranscriptLogRedaction.ts";
|
||||
import { capMemoryExtractionText, MEMORY_EXTRACTION_TEXT_LIMIT } from "./logTruncation.ts";
|
||||
|
||||
function normalizeMemoryInputText(value: unknown, context: VideoTranscriptLogContext = {}): string {
|
||||
if (typeof value !== "string") return "";
|
||||
return omitVideoTranscriptDerivedTextForMemory(value, context).trim();
|
||||
}
|
||||
|
||||
function extractMemoryTextPart(
|
||||
part: Record<string, unknown>,
|
||||
transcriptSensitive: boolean,
|
||||
context: VideoTranscriptLogContext
|
||||
): string {
|
||||
try {
|
||||
const rawText = typeof part?.text === "string" ? part.text : "";
|
||||
if (!rawText) return "";
|
||||
|
||||
const retainedText = normalizeMemoryInputText(rawText, context);
|
||||
if (!retainedText) return "";
|
||||
if (
|
||||
transcriptSensitive &&
|
||||
containsVideoTranscriptForLog(part, context) &&
|
||||
retainedText === rawText.trim()
|
||||
) {
|
||||
return "";
|
||||
}
|
||||
return retainedText;
|
||||
} catch {
|
||||
return "";
|
||||
}
|
||||
}
|
||||
|
||||
export function extractMemoryTextFromResponse(
|
||||
response: Record<string, unknown> | null | undefined
|
||||
): string {
|
||||
@@ -63,16 +29,9 @@ export function extractMemoryTextFromResponse(
|
||||
}
|
||||
|
||||
export function extractMemoryTextFromRequestBody(
|
||||
body: Record<string, unknown> | null | undefined,
|
||||
videoTranscriptSensitive = false,
|
||||
context: VideoTranscriptLogContext = {}
|
||||
body: Record<string, unknown> | null | undefined
|
||||
): string {
|
||||
if (!body || typeof body !== "object") return "";
|
||||
// Re-check the structured body at the sink boundary. The explicit bit covers
|
||||
// processed requests whose raw carrier was already replaced; trusted hashes
|
||||
// identify only descriptions emitted by a modified Video Bridge guardrail.
|
||||
const transcriptSensitive =
|
||||
videoTranscriptSensitive || containsVideoTranscriptForLog(body, context);
|
||||
|
||||
const messages = Array.isArray(body.messages) ? body.messages : null;
|
||||
if (messages && messages.length > 0) {
|
||||
@@ -80,16 +39,18 @@ export function extractMemoryTextFromRequestBody(
|
||||
const msg = messages[i] as Record<string, unknown>;
|
||||
if (msg?.role !== "user") continue;
|
||||
|
||||
const messageText = normalizeMemoryInputText(msg.content, context);
|
||||
if (messageText) {
|
||||
return capMemoryExtractionText(messageText);
|
||||
if (typeof msg.content === "string" && msg.content.trim().length > 0) {
|
||||
return capMemoryExtractionText(msg.content.trim());
|
||||
}
|
||||
|
||||
if (Array.isArray(msg.content)) {
|
||||
const text = msg.content
|
||||
.map((part: Record<string, unknown>) =>
|
||||
extractMemoryTextPart(part, transcriptSensitive, context)
|
||||
)
|
||||
.map((part: Record<string, unknown>) => {
|
||||
if (typeof part?.text === "string") return part.text.trim();
|
||||
if (part?.type === "input_text" && typeof part?.text === "string")
|
||||
return part.text.trim();
|
||||
return "";
|
||||
})
|
||||
.filter(Boolean)
|
||||
.join("\n")
|
||||
.trim();
|
||||
@@ -107,15 +68,17 @@ export function extractMemoryTextFromRequestBody(
|
||||
if (role && role !== "user") continue;
|
||||
if (itemType && itemType !== "message") continue;
|
||||
|
||||
const itemText = normalizeMemoryInputText(item?.content, context);
|
||||
if (itemText) {
|
||||
return capMemoryExtractionText(itemText);
|
||||
if (typeof item?.content === "string" && item.content.trim()) {
|
||||
return capMemoryExtractionText(item.content.trim());
|
||||
}
|
||||
if (Array.isArray(item?.content)) {
|
||||
const text = item.content
|
||||
.map((part: Record<string, unknown>) =>
|
||||
extractMemoryTextPart(part, transcriptSensitive, context)
|
||||
)
|
||||
.map((part: Record<string, unknown>) => {
|
||||
if (typeof part?.text === "string") return part.text.trim();
|
||||
if (part?.type === "input_text" && typeof part?.text === "string")
|
||||
return part.text.trim();
|
||||
return "";
|
||||
})
|
||||
.filter(Boolean)
|
||||
.join("\n")
|
||||
.trim();
|
||||
@@ -133,14 +96,15 @@ export function extractMemoryTextFromRequestBody(
|
||||
if (role && role !== "user") return "";
|
||||
if (itemType && itemType !== "message") return "";
|
||||
|
||||
if (typeof item?.content === "string") {
|
||||
return normalizeMemoryInputText(item.content, context);
|
||||
}
|
||||
if (typeof item?.content === "string") return item.content.trim();
|
||||
if (Array.isArray(item?.content)) {
|
||||
return item.content
|
||||
.map((part: Record<string, unknown>) =>
|
||||
extractMemoryTextPart(part, transcriptSensitive, context)
|
||||
)
|
||||
.map((part: Record<string, unknown>) => {
|
||||
if (typeof part?.text === "string") return part.text.trim();
|
||||
if (part?.type === "input_text" && typeof part?.text === "string")
|
||||
return part.text.trim();
|
||||
return "";
|
||||
})
|
||||
.filter(Boolean)
|
||||
.join("\n")
|
||||
.trim();
|
||||
|
||||
@@ -9,8 +9,6 @@
|
||||
* byte-identical to the previous inline block.
|
||||
*/
|
||||
|
||||
import { redactVideoTranscriptSensitiveText } from "@/lib/guardrails/videoTranscriptLogRedaction";
|
||||
|
||||
type LoggerLike =
|
||||
{ info?: (...args: unknown[]) => void; debug?: (...args: unknown[]) => void } | null | undefined;
|
||||
|
||||
@@ -27,7 +25,6 @@ export async function runPluginOnRequestHook(args: {
|
||||
apiKeyInfo: unknown;
|
||||
headers?: Record<string, string | string[] | undefined>;
|
||||
log?: LoggerLike;
|
||||
videoTranscriptSensitive?: boolean;
|
||||
}): Promise<PluginOnRequestGate> {
|
||||
try {
|
||||
const { runOnRequest } = await import("@/lib/plugins/hooks");
|
||||
@@ -39,7 +36,6 @@ export async function runPluginOnRequestHook(args: {
|
||||
apiKeyInfo: args.apiKeyInfo,
|
||||
headers: args.headers,
|
||||
metadata: {},
|
||||
videoTranscriptSensitive: args.videoTranscriptSensitive === true,
|
||||
};
|
||||
const pluginResult = await runOnRequest(pluginCtx);
|
||||
if (pluginResult?.blocked) {
|
||||
@@ -59,11 +55,10 @@ export async function runPluginOnRequestHook(args: {
|
||||
}
|
||||
return { blocked: false, body: pluginResult?.body };
|
||||
} catch (pluginErr) {
|
||||
const retainedPluginError = redactVideoTranscriptSensitiveText(
|
||||
pluginErr instanceof Error ? pluginErr.message : String(pluginErr),
|
||||
args.videoTranscriptSensitive === true
|
||||
args.log?.debug?.(
|
||||
"PLUGIN",
|
||||
`onRequest hook error (non-fatal): ${pluginErr instanceof Error ? pluginErr.message : String(pluginErr)}`
|
||||
);
|
||||
args.log?.debug?.("PLUGIN", `onRequest hook error (non-fatal): ${retainedPluginError}`);
|
||||
return { blocked: false };
|
||||
}
|
||||
}
|
||||
|
||||
@@ -26,7 +26,6 @@ export async function runPluginOnResponseHook(args: {
|
||||
apiKeyInfo: unknown;
|
||||
headers?: Record<string, string | string[] | undefined>;
|
||||
response: PluginOnResponsePayload;
|
||||
videoTranscriptSensitive?: boolean;
|
||||
}): Promise<void> {
|
||||
try {
|
||||
const { runOnResponse } = await import("@/lib/plugins/hooks");
|
||||
@@ -39,7 +38,6 @@ export async function runPluginOnResponseHook(args: {
|
||||
apiKeyInfo: args.apiKeyInfo,
|
||||
headers: args.headers,
|
||||
metadata: {},
|
||||
videoTranscriptSensitive: args.videoTranscriptSensitive === true,
|
||||
},
|
||||
args.response
|
||||
).catch(() => {});
|
||||
|
||||
@@ -36,7 +36,6 @@ export function buildPostCallGuardrailContext(
|
||||
provider: string | null | undefined;
|
||||
responsePayloadFormat: unknown;
|
||||
clientResponseFormat: unknown;
|
||||
videoTranscriptSensitive?: boolean;
|
||||
},
|
||||
resolveDisabledGuardrails: typeof defaultResolveDisabled = defaultResolveDisabled
|
||||
): GuardrailContext {
|
||||
@@ -58,6 +57,5 @@ export function buildPostCallGuardrailContext(
|
||||
sourceFormat: optionalString(args.responsePayloadFormat),
|
||||
stream: false,
|
||||
targetFormat: optionalString(args.clientResponseFormat),
|
||||
videoTranscriptSensitive: args.videoTranscriptSensitive === true,
|
||||
};
|
||||
}
|
||||
|
||||
@@ -1,4 +1,8 @@
|
||||
import { generateSignature, getCachedResponse, isCacheableForRead } from "@/lib/semanticCache";
|
||||
import {
|
||||
generateSignature,
|
||||
getCachedResponse,
|
||||
isCacheableForRead,
|
||||
} from "@/lib/semanticCache";
|
||||
import { calculateCost } from "@/lib/usage/costCalculator";
|
||||
import { trackPendingRequest } from "@/lib/usageDb";
|
||||
import { synthesizeOpenAiSseFromJson } from "../../utils/jsonToSse.ts";
|
||||
@@ -21,7 +25,6 @@ export async function checkSemanticCache({
|
||||
persistAttemptLogs,
|
||||
apiKeyId,
|
||||
cacheDefaultMode,
|
||||
videoTranscriptSensitive,
|
||||
}: {
|
||||
semanticCacheEnabled: boolean;
|
||||
// Only the fields this read path actually touches are named; everything else
|
||||
@@ -39,9 +42,7 @@ export async function checkSemanticCache({
|
||||
persistAttemptLogs: (args: unknown) => void;
|
||||
apiKeyId?: string | null;
|
||||
cacheDefaultMode?: "legacy" | "bypass" | null;
|
||||
videoTranscriptSensitive: boolean;
|
||||
}) {
|
||||
if (videoTranscriptSensitive) return null;
|
||||
// Per-key bypass: skip cache lookup entirely when the API key opts out.
|
||||
if (cacheDefaultMode === "bypass") return null;
|
||||
if (semanticCacheEnabled && isCacheableForRead(body, clientRawRequest?.headers)) {
|
||||
|
||||
@@ -50,12 +50,10 @@ export function storeSemanticCacheResponse(
|
||||
apiKeyId?: string;
|
||||
usage?: UsageLike;
|
||||
log?: LoggerLike;
|
||||
videoTranscriptSensitive: boolean;
|
||||
},
|
||||
deps: SemanticCacheStoreDeps = DEFAULT_DEPS
|
||||
): void {
|
||||
if (
|
||||
args.videoTranscriptSensitive ||
|
||||
!args.enabled ||
|
||||
!deps.isCacheableForWrite(args.body, args.headers) ||
|
||||
!deps.isSmallEnoughForSemanticCache(args.translatedResponse)
|
||||
|
||||
@@ -68,7 +68,6 @@ export function assembleStreamingPipeline(
|
||||
clientRawRequestHeaders: HeadersLike;
|
||||
clientResponseFormat: Parameters<typeof defaultShape>[0];
|
||||
echoModel: string | null | undefined;
|
||||
redactStreamDiagnosticsForLog?: boolean;
|
||||
responseHeaders: Record<string, string>;
|
||||
},
|
||||
deps: StreamingPipelineDeps = DEFAULT_DEPS
|
||||
@@ -84,8 +83,7 @@ export function assembleStreamingPipeline(
|
||||
let piiStream = deps.pipeWithDisconnect(
|
||||
args.providerResponse,
|
||||
args.transformStream,
|
||||
args.streamController,
|
||||
{ redactStreamDiagnosticsForLog: args.redactStreamDiagnosticsForLog }
|
||||
args.streamController
|
||||
);
|
||||
if (typeof args.createPiiTransform === "function") {
|
||||
piiStream = piiStream.pipeThrough((args.createPiiTransform as () => TransformStream)());
|
||||
|
||||
@@ -49,7 +49,6 @@ interface StreamingCacheArgs {
|
||||
apiKeyId?: string;
|
||||
streamUsage?: Record<string, unknown> | null;
|
||||
log?: LoggerLike;
|
||||
videoTranscriptSensitive: boolean;
|
||||
}
|
||||
|
||||
function streamTokensSaved(streamUsage: Record<string, unknown> | null | undefined): number {
|
||||
@@ -88,7 +87,6 @@ export function storeStreamingSemanticCacheResponse(
|
||||
deps: StreamingSemanticCacheStoreDeps = DEFAULT_DEPS
|
||||
): void {
|
||||
if (
|
||||
args.videoTranscriptSensitive ||
|
||||
!args.enabled ||
|
||||
args.streamStatus !== 200 ||
|
||||
!args.streamResponseBody ||
|
||||
|
||||
@@ -92,65 +92,19 @@ interface KieImageOptions {
|
||||
}
|
||||
|
||||
// KIE Market catalog ids are namespaced for OmniRoute's catalog
|
||||
// (`<vendor>/<model>`), but the KIE Market createTask API expects
|
||||
// (`google-imagen/<model>`), but the KIE Market createTask API expects
|
||||
// vendor-specific upstream ids that do not follow a single consistent
|
||||
// pattern. Every entry below was confirmed individually against the literal
|
||||
// example request JSON published on docs.kie.ai (never inferred by pattern —
|
||||
// see #11326's false "everything else already matches" claim and #11296's
|
||||
// follow-up correction):
|
||||
// - google-imagen: nano-banana-2 and nano-banana-pro drop the vendor
|
||||
// namespace entirely; nano-banana and nano-banana-edit use a `google/`
|
||||
// prefix instead of `google-imagen/` (docs.kie.ai/market/google/*).
|
||||
// - gpt: gpt-image-2-* drops the `gpt/` namespace entirely
|
||||
// (docs.kie.ai/market/gpt/gpt-image-2-*); gpt-image-1.5-* uses a
|
||||
// `gpt-image/` namespace instead of `gpt/gpt-image-1.5-`, and keeps the
|
||||
// dot in "1.5" (docs.kie.ai/market/gpt-image/1-5-*).
|
||||
// - seedream: 5.0-lite-* drops the ".0" — real id is `5-lite-*`
|
||||
// (docs.kie.ai/market/seedream/5-lite-text-to-image); seedream 4.5 (T2I
|
||||
// and edit) already matches byte-for-byte.
|
||||
// - flux: `flux/2-*` uses a `flux-2/` namespace (dash, not slash); the
|
||||
// generic (non-"pro") variant is named `flex` upstream, not `2`
|
||||
// (docs.kie.ai/market/flux2/pro-*, .../flex-*).
|
||||
// - wan: `wan/2.7-*` keeps the dot in our catalog, but KIE's documented
|
||||
// enum uses a dash — real id is `wan/2-7-*`
|
||||
// (docs.kie.ai/market/wan/2-7-image[-pro]).
|
||||
// - ideogram (v3-text-to-image, v3-edit, v3-remix), qwen, qwen2, and
|
||||
// grok-imagine already match byte-for-byte
|
||||
// (docs.kie.ai/market/{ideogram,qwen,qwen2,grok-imagine}/*).
|
||||
// ideogram/v3-reframe has no dedicated docs.kie.ai page as of this sweep
|
||||
// (its 3 siblings above are all direct id matches, so it is assumed
|
||||
// correct by pattern, not independently confirmed).
|
||||
// Two catalog entries remain UNRESOLVED after this sweep and are
|
||||
// deliberately left untouched pending a follow-up (see #11296 discussion):
|
||||
// - z-image/4.0-text-to-image and z-image/4.5-text-to-image: the only
|
||||
// documented Z-Image Market page (docs.kie.ai/market/z-image/z-image)
|
||||
// shows a single fixed `model` enum value `"z-image"` with no
|
||||
// version-specific id or "version" input field found — unclear whether
|
||||
// both catalog ids should collapse to the same upstream call.
|
||||
// - flux/kontext: no `docs.kie.ai/market/flux2/kontext` (or similar)
|
||||
// Market page exists; Flux Kontext is documented under the separate
|
||||
// `/flux-kontext-api/*` docs tree with its own endpoint
|
||||
// (`POST /api/v1/flux/kontext/generate`, models `flux-kontext-pro`/
|
||||
// `flux-kontext-max`), not the Market `createTask` flow this map feeds.
|
||||
// This entry may be miscatalogued as `isMarket: true` and need a
|
||||
// dedicated reroute rather than an id rewrite.
|
||||
// pattern (confirmed against docs.kie.ai/market/google/* — see #11225,
|
||||
// #11296): nano-banana-2 and nano-banana-pro drop the vendor namespace
|
||||
// entirely, while nano-banana and nano-banana-edit use a `google/` prefix
|
||||
// instead of `google-imagen/`. Every other KIE Market namespace (seedream,
|
||||
// flux, ideogram, qwen, wan, grok-imagine, gpt) already matches its real
|
||||
// upstream id byte-for-byte, so this map stays scoped to google-imagen.
|
||||
export const KIE_MARKET_UPSTREAM_MODEL_IDS: ReadonlyMap<string, string> = new Map([
|
||||
["google-imagen/nano-banana", "google/nano-banana"],
|
||||
["google-imagen/nano-banana-2", "nano-banana-2"],
|
||||
["google-imagen/nano-banana-pro", "nano-banana-pro"],
|
||||
["google-imagen/nano-banana-edit", "google/nano-banana-edit"],
|
||||
["gpt/gpt-image-2-text-to-image", "gpt-image-2-text-to-image"],
|
||||
["gpt/gpt-image-2-image-to-image", "gpt-image-2-image-to-image"],
|
||||
["gpt/gpt-image-1.5-text-to-image", "gpt-image/1.5-text-to-image"],
|
||||
["gpt/gpt-image-1.5-image-to-image", "gpt-image/1.5-image-to-image"],
|
||||
["seedream/5.0-lite-text-to-image", "seedream/5-lite-text-to-image"],
|
||||
["seedream/5.0-lite-image-to-image", "seedream/5-lite-image-to-image"],
|
||||
["flux/2-pro-text-to-image", "flux-2/pro-text-to-image"],
|
||||
["flux/2-pro-image-to-image", "flux-2/pro-image-to-image"],
|
||||
["flux/2-text-to-image", "flux-2/flex-text-to-image"],
|
||||
["flux/2-image-to-image", "flux-2/flex-image-to-image"],
|
||||
["wan/2.7-image", "wan/2-7-image"],
|
||||
["wan/2.7-image-pro", "wan/2-7-image-pro"],
|
||||
]);
|
||||
|
||||
export function resolveKieMarketUpstreamModelId(publicModelId: string): string {
|
||||
|
||||
@@ -63,10 +63,6 @@ import {
|
||||
} from "../../src/lib/db/contextHandoffs.ts";
|
||||
import { extractSessionAffinityKey } from "@/sse/services/auth";
|
||||
import { getHiddenModelsByProvider } from "@/models";
|
||||
import {
|
||||
containsVideoTranscriptForLog,
|
||||
redactVideoTranscriptSensitiveText,
|
||||
} from "@/lib/guardrails/videoTranscriptLogRedaction";
|
||||
import { resolveModelLockoutSettings } from "../../src/lib/resilience/modelLockoutSettings";
|
||||
import { fetchCodexQuota } from "./codexQuotaFetcher.ts";
|
||||
import { evaluateQuotaCutoff, getQuotaFetcher, type QuotaInfo } from "./quotaPreflight.ts";
|
||||
@@ -706,15 +702,7 @@ export async function resolveTargetTimeoutMsForTarget(
|
||||
*/
|
||||
export async function handleComboChat(options: HandleComboChatOptions): Promise<Response> {
|
||||
const traceInvocationId = options.invocationId ?? createInvocationId();
|
||||
// Re-check a still-structured body at the service boundary. The explicit bit remains
|
||||
// authoritative when the caller already ran Video Bridge and replaced the raw carrier.
|
||||
const videoTranscriptSensitive =
|
||||
options.videoTranscriptSensitive === true || containsVideoTranscriptForLog(options.body);
|
||||
const response = await handleComboChatInner({
|
||||
...options,
|
||||
invocationId: traceInvocationId,
|
||||
videoTranscriptSensitive,
|
||||
});
|
||||
const response = await handleComboChatInner({ ...options, invocationId: traceInvocationId });
|
||||
response.headers.set("X-OmniRoute-Combo-Trace", traceInvocationId);
|
||||
const trace = getComboTrace(traceInvocationId);
|
||||
options.log.info(
|
||||
@@ -744,8 +732,6 @@ async function handleComboChatInner({
|
||||
sourceFormat = null,
|
||||
endpointPath = null,
|
||||
requestHeaders = null,
|
||||
videoTranscriptSensitive = false,
|
||||
videoTranscriptDescriptionFingerprints = [],
|
||||
invocationId,
|
||||
}: HandleComboChatOptions): Promise<Response> {
|
||||
const comboCtx = createComboContext({ body, combo, settings, relayOptions, log });
|
||||
@@ -797,7 +783,6 @@ async function handleComboChatInner({
|
||||
handleSingleModelWithTimeout,
|
||||
log,
|
||||
hiddenModelsByProvider,
|
||||
videoTranscriptSensitive,
|
||||
});
|
||||
if (pinnedDispatch) return pinnedDispatch;
|
||||
}
|
||||
@@ -820,8 +805,6 @@ async function handleComboChatInner({
|
||||
signal,
|
||||
apiKeyAllowedConnections,
|
||||
hiddenModelsByProvider,
|
||||
videoTranscriptSensitive,
|
||||
videoTranscriptDescriptionFingerprints,
|
||||
perTargetAdmission,
|
||||
deferContextOverflowWhenCompressible,
|
||||
compressionExclusions,
|
||||
@@ -878,8 +861,6 @@ async function handleComboChatInner({
|
||||
signal,
|
||||
apiKeyAllowedConnections,
|
||||
hiddenModelsByProvider,
|
||||
videoTranscriptSensitive,
|
||||
videoTranscriptDescriptionFingerprints,
|
||||
perTargetAdmission,
|
||||
deferContextOverflowWhenCompressible,
|
||||
compressionExclusions,
|
||||
@@ -908,7 +889,6 @@ async function handleComboChatInner({
|
||||
allCombos,
|
||||
signal,
|
||||
hiddenModelsByProvider,
|
||||
videoTranscriptSensitive,
|
||||
clientManagedResponsesContext,
|
||||
deferContextOverflowWhenCompressible,
|
||||
compressionExclusions,
|
||||
@@ -1701,13 +1681,9 @@ async function handleComboChatInner({
|
||||
releaseQualityClone(qualityClone, result, quality);
|
||||
if (!quality.valid) {
|
||||
releaseRejectedQualityResponse(qualityClone, result);
|
||||
const retainedQualityReason = redactVideoTranscriptSensitiveText(
|
||||
quality.reason || "upstream response failed quality validation",
|
||||
videoTranscriptSensitive
|
||||
);
|
||||
log.warn(
|
||||
"COMBO",
|
||||
`Model ${modelStr} returned 200 but failed quality check: ${retainedQualityReason}`
|
||||
`Model ${modelStr} returned 200 but failed quality check: ${quality.reason}`
|
||||
);
|
||||
// #6692: a quality-rejected 200 never marks the connection row
|
||||
// unhealthy, so the sticky pin's lazy headroom recheck would never
|
||||
@@ -1760,7 +1736,7 @@ async function handleComboChatInner({
|
||||
targetIndex: i,
|
||||
provider,
|
||||
model: modelStr,
|
||||
error: `Quality: ${retainedQualityReason}`,
|
||||
error: `Quality: ${quality.reason}`,
|
||||
latencyMs: Date.now() - startTime,
|
||||
});
|
||||
observeFailure(false, target.executionKey);
|
||||
@@ -1893,8 +1869,6 @@ async function handleComboChatInner({
|
||||
prevModel,
|
||||
currModel: modelStr,
|
||||
universalConfig: universalHandoffConfig,
|
||||
videoTranscriptSensitive,
|
||||
trustedDescriptionFingerprints: videoTranscriptDescriptionFingerprints,
|
||||
handleSingleModel: handleSingleModelWithTimeout,
|
||||
});
|
||||
}
|
||||
@@ -1946,8 +1920,6 @@ async function handleComboChatInner({
|
||||
model: modelStr,
|
||||
expiresAt: resetCandidates[0] || null,
|
||||
config: relayConfig,
|
||||
videoTranscriptSensitive,
|
||||
trustedDescriptionFingerprints: videoTranscriptDescriptionFingerprints,
|
||||
handleSingleModel: handleSingleModelWithTimeout,
|
||||
});
|
||||
}
|
||||
@@ -2471,9 +2443,7 @@ async function handleComboChatInner({
|
||||
}
|
||||
log.warn("COMBO", `Model ${modelStr} failed, trying next`, {
|
||||
status: result.status,
|
||||
errorBody: redactConnectionLabel(
|
||||
redactVideoTranscriptSensitiveText(errorText, videoTranscriptSensitive)
|
||||
),
|
||||
errorBody: redactConnectionLabel(errorText),
|
||||
});
|
||||
|
||||
// #5976: per-model-quota providers (Gemini, GitHub, etc.) multiplex models
|
||||
@@ -2554,11 +2524,7 @@ async function handleComboChatInner({
|
||||
}
|
||||
})().catch((err) => {
|
||||
const logError = log.error ?? log.warn;
|
||||
logError(
|
||||
"COMBO",
|
||||
`Speculative task error for target ${i}`,
|
||||
videoTranscriptSensitive ? redactVideoTranscriptSensitiveText(String(err), true) : err
|
||||
);
|
||||
logError("COMBO", `Speculative task error for target ${i}`, err);
|
||||
// G2 (silent-stop fix): never leave the speculative loop waiting on an
|
||||
// unresolved globalPromise. If a task throws unexpectedly (outside
|
||||
// executeTarget's error handling) and no other task succeeds, the post-loop
|
||||
@@ -2779,13 +2745,9 @@ async function handleComboChatInner({
|
||||
});
|
||||
|
||||
if (decision.wait) {
|
||||
const retainedComboFailure = redactVideoTranscriptSensitiveText(
|
||||
msg,
|
||||
videoTranscriptSensitive
|
||||
);
|
||||
log.info(
|
||||
"COMBO",
|
||||
`${strategy} cooldown wait: ${retainedComboFailure} — waiting ${Math.ceil(
|
||||
`${strategy} cooldown wait: ${msg} — waiting ${Math.ceil(
|
||||
decision.waitMs / 1000
|
||||
)}s (reason=${decision.reason ?? "?"}) then retrying (attempt ${
|
||||
comboCooldownAttempt + 1
|
||||
@@ -2811,13 +2773,7 @@ async function handleComboChatInner({
|
||||
// a config-class status like 403/422).
|
||||
if (earliestRetryAfter && isRetryAfterEligibleStatus(status)) {
|
||||
const retryHuman = formatRetryAfter(toRetryAfterDisplayValue(earliestRetryAfter));
|
||||
log.warn(
|
||||
"COMBO",
|
||||
`All models failed | ${redactVideoTranscriptSensitiveText(
|
||||
msg,
|
||||
videoTranscriptSensitive
|
||||
)} (${retryHuman})`
|
||||
);
|
||||
log.warn("COMBO", `All models failed | ${msg} (${retryHuman})`);
|
||||
return withQuotaExhaustionClassification(
|
||||
unavailableResponse(status, msg, earliestRetryAfter, retryHuman),
|
||||
observedFailure ? allObservedFailuresQuota : null
|
||||
@@ -2828,10 +2784,7 @@ async function handleComboChatInner({
|
||||
// `try-auto` recovery action via buildRecoveryHint so the OC plugin can show "→ Try
|
||||
// model: auto" instead of an opaque 5xx. We pass the upstream retry-after seconds to
|
||||
// the hint so the client can render a precise "wait Ns and retry" message.
|
||||
log.warn(
|
||||
"COMBO",
|
||||
`All models failed | ${redactVideoTranscriptSensitiveText(msg, videoTranscriptSensitive)}`
|
||||
);
|
||||
log.warn("COMBO", `All models failed | ${msg}`);
|
||||
const { pinClearedNow } = recordComboFailure(effectiveSessionId, combo.name);
|
||||
if (pinClearedNow) {
|
||||
log.info(
|
||||
@@ -2935,7 +2888,6 @@ async function handleRoundRobinCombo({
|
||||
requestHeaders = null,
|
||||
relayOptions,
|
||||
perTargetAdmission = null,
|
||||
videoTranscriptSensitive = false,
|
||||
}: HandleRoundRobinOptions): Promise<Response> {
|
||||
const config = settings
|
||||
? resolveComboConfig(combo, settings)
|
||||
@@ -3334,20 +3286,7 @@ async function handleRoundRobinCombo({
|
||||
"COMBO-RR",
|
||||
`Maximum combo attempts (${maxGlobalAttempts}) exceeded. Terminating loop to prevent runaway requests.`
|
||||
);
|
||||
return errorResponseWithComboDiagnostics(503, "Maximum combo retry limit reached", {
|
||||
poolSize: modelCount,
|
||||
attempted: globalAttempts,
|
||||
excluded: [
|
||||
...[...exhaustedProviders].map((p) => ({ provider: p, reason: "exhausted" })),
|
||||
...[...exhaustedConnections].map((c) => formatExhaustedConnectionKey(String(c))),
|
||||
],
|
||||
attemptOrder: rrOutcomes.map((o) => ({
|
||||
provider: o.model.split("/")[0] || "unknown",
|
||||
model: o.model,
|
||||
})),
|
||||
terminalReason: "max_attempts_exceeded",
|
||||
recovery: buildRecoveryHint("max_attempts_exceeded"),
|
||||
});
|
||||
return errorResponse(503, "Maximum combo retry limit reached");
|
||||
}
|
||||
if (retry > 0) {
|
||||
log.info(
|
||||
@@ -3448,13 +3387,9 @@ async function handleRoundRobinCombo({
|
||||
releaseQualityClone(rrClone, result, quality);
|
||||
if (!quality.valid) {
|
||||
releaseRejectedQualityResponse(rrClone, result);
|
||||
const retainedQualityReason = redactVideoTranscriptSensitiveText(
|
||||
quality.reason || "upstream response failed quality validation",
|
||||
videoTranscriptSensitive
|
||||
);
|
||||
log.warn(
|
||||
"COMBO-RR",
|
||||
`${modelStr} returned 200 but failed quality check: ${retainedQualityReason}`
|
||||
`${modelStr} returned 200 but failed quality check: ${quality.reason}`
|
||||
);
|
||||
// #6692: same rationale as handleComboChat's quality-fail branch —
|
||||
// a quality-rejected 200 never marks the connection row unhealthy,
|
||||
@@ -3806,9 +3741,7 @@ async function handleRoundRobinCombo({
|
||||
if (offset > 0) fallbackCount++;
|
||||
log.warn("COMBO-RR", `${modelStr} failed, trying next model`, {
|
||||
status: result.status,
|
||||
errorBody: redactConnectionLabel(
|
||||
redactVideoTranscriptSensitiveText(errorText, videoTranscriptSensitive)
|
||||
),
|
||||
errorBody: redactConnectionLabel(errorText),
|
||||
});
|
||||
|
||||
if (
|
||||
@@ -3861,11 +3794,7 @@ async function handleRoundRobinCombo({
|
||||
} catch (err) {
|
||||
// G4: unexpected exception in the round-robin loop must never crash the
|
||||
// request silently — surface a 500 instead of hanging the client.
|
||||
log.error?.(
|
||||
"COMBO-RR",
|
||||
"Unexpected error in round-robin loop",
|
||||
videoTranscriptSensitive ? redactVideoTranscriptSensitiveText(String(err), true) : err
|
||||
);
|
||||
log.error?.("COMBO-RR", "Unexpected error in round-robin loop", err);
|
||||
return errorResponse(500, "Unexpected error in round-robin combo");
|
||||
} finally {
|
||||
if (rrLoopSafetyTimer) {
|
||||
@@ -3961,20 +3890,11 @@ async function handleRoundRobinCombo({
|
||||
|
||||
if (earliestRetryAfter && isRetryAfterEligibleStatus(status)) {
|
||||
const retryHuman = formatRetryAfter(toRetryAfterDisplayValue(earliestRetryAfter));
|
||||
log.warn(
|
||||
"COMBO-RR",
|
||||
`All models failed | ${redactVideoTranscriptSensitiveText(
|
||||
msg,
|
||||
videoTranscriptSensitive
|
||||
)} (${retryHuman})`
|
||||
);
|
||||
log.warn("COMBO-RR", `All models failed | ${msg} (${retryHuman})`);
|
||||
return unavailableResponse(status, msg, earliestRetryAfter, retryHuman);
|
||||
}
|
||||
|
||||
log.warn(
|
||||
"COMBO-RR",
|
||||
`All models failed | ${redactVideoTranscriptSensitiveText(msg, videoTranscriptSensitive)}`
|
||||
);
|
||||
log.warn("COMBO-RR", `All models failed | ${msg}`);
|
||||
return new Response(JSON.stringify({ error: { message: msg } }), {
|
||||
status,
|
||||
headers: { "Content-Type": "application/json" },
|
||||
|
||||
@@ -13,7 +13,6 @@
|
||||
* Extracted from combo.ts as a pure move (#3501). No behaviour change.
|
||||
*/
|
||||
import { getCachedProviderConnections } from "../../../src/lib/db/readCache";
|
||||
import { redactVideoTranscriptSensitiveText } from "../../../src/lib/guardrails/videoTranscriptLogRedaction";
|
||||
import { getCircuitBreaker } from "../../../src/shared/utils/circuitBreaker";
|
||||
import { fisherYatesShuffle, getNextFromDeck } from "../../../src/shared/utils/shuffleDeck";
|
||||
import { handleFusionChat, type FusionTuning } from "../fusion.ts";
|
||||
@@ -85,8 +84,6 @@ type PreludeBaseOptionArgs = {
|
||||
signal?: AbortSignal | null;
|
||||
apiKeyAllowedConnections?: string[] | null;
|
||||
hiddenModelsByProvider?: HiddenModelsByProvider;
|
||||
videoTranscriptSensitive?: boolean;
|
||||
videoTranscriptDescriptionFingerprints?: readonly string[];
|
||||
clientManagedResponsesContext?: boolean;
|
||||
/** #9654 Wave 2: per-target lane-aware admission probe (see HandleComboChatOptions). */
|
||||
perTargetAdmission?: PerTargetAdmissionHook | null;
|
||||
@@ -114,8 +111,6 @@ function buildBaseOptions(a: PreludeBaseOptionArgs): HandleComboChatOptions {
|
||||
signal: a.signal,
|
||||
apiKeyAllowedConnections: a.apiKeyAllowedConnections,
|
||||
hiddenModelsByProvider: a.hiddenModelsByProvider,
|
||||
videoTranscriptSensitive: a.videoTranscriptSensitive,
|
||||
videoTranscriptDescriptionFingerprints: a.videoTranscriptDescriptionFingerprints,
|
||||
invocationId: a.invocationId,
|
||||
clientManagedResponsesContext: a.clientManagedResponsesContext,
|
||||
perTargetAdmission: a.perTargetAdmission,
|
||||
@@ -226,16 +221,8 @@ async function evaluatePinnedResponse(args: {
|
||||
clientRequestedStream: boolean;
|
||||
config: ComboSetupConfig;
|
||||
log: ComboLogger;
|
||||
videoTranscriptSensitive?: boolean;
|
||||
}): Promise<Response | null> {
|
||||
const {
|
||||
pinnedResult,
|
||||
pinnedModel,
|
||||
clientRequestedStream,
|
||||
config,
|
||||
log,
|
||||
videoTranscriptSensitive = false,
|
||||
} = args;
|
||||
const { pinnedResult, pinnedModel, clientRequestedStream, config, log } = args;
|
||||
if (pinnedResult.ok) {
|
||||
let pinnedClone: Response;
|
||||
try {
|
||||
@@ -252,13 +239,9 @@ async function evaluatePinnedResponse(args: {
|
||||
releaseQualityClone(pinnedClone, pinnedResult, pinnedQuality);
|
||||
if (pinnedQuality.valid) return pinnedResult;
|
||||
releaseRejectedQualityResponse(pinnedClone, pinnedResult);
|
||||
const retainedReason = redactVideoTranscriptSensitiveText(
|
||||
pinnedQuality.reason || "upstream response failed quality validation",
|
||||
videoTranscriptSensitive
|
||||
);
|
||||
log.warn(
|
||||
"COMBO",
|
||||
`Pinned model ${pinnedModel} returned 200 but failed quality check: ${retainedReason}, falling through to combo retry/fallback`
|
||||
`Pinned model ${pinnedModel} returned 200 but failed quality check: ${pinnedQuality.reason}, falling through to combo retry/fallback`
|
||||
);
|
||||
return null;
|
||||
}
|
||||
@@ -288,7 +271,6 @@ export async function tryPinnedModelDispatch(args: {
|
||||
handleSingleModelWithTimeout: HandleSingleModel;
|
||||
log: ComboLogger;
|
||||
hiddenModelsByProvider?: HiddenModelsByProvider;
|
||||
videoTranscriptSensitive?: boolean;
|
||||
}): Promise<Response | null> {
|
||||
const {
|
||||
body,
|
||||
@@ -300,7 +282,6 @@ export async function tryPinnedModelDispatch(args: {
|
||||
handleSingleModelWithTimeout,
|
||||
log,
|
||||
hiddenModelsByProvider,
|
||||
videoTranscriptSensitive = false,
|
||||
} = args;
|
||||
// The pin is read from session_model_history (a PRIOR turn) and may name a
|
||||
// model that has since been removed from this combo, or a provider whose
|
||||
@@ -360,13 +341,9 @@ export async function tryPinnedModelDispatch(args: {
|
||||
modelPinned: true,
|
||||
} as SingleModelTarget);
|
||||
} catch (pinErr) {
|
||||
const retainedPinError = redactVideoTranscriptSensitiveText(
|
||||
pinErr instanceof Error ? pinErr.message : String(pinErr),
|
||||
videoTranscriptSensitive
|
||||
);
|
||||
log.warn(
|
||||
"COMBO",
|
||||
`Pinned model ${pinnedModel} threw error: ${retainedPinError}, falling through to combo retry/fallback`
|
||||
`Pinned model ${pinnedModel} threw error: ${pinErr instanceof Error ? pinErr.message : String(pinErr)}, falling through to combo retry/fallback`
|
||||
);
|
||||
}
|
||||
if (pinnedResult) {
|
||||
@@ -376,7 +353,6 @@ export async function tryPinnedModelDispatch(args: {
|
||||
clientRequestedStream,
|
||||
config,
|
||||
log,
|
||||
videoTranscriptSensitive,
|
||||
});
|
||||
if (accepted) return accepted;
|
||||
}
|
||||
@@ -419,8 +395,6 @@ export async function tryFusionDispatch(args: {
|
||||
signal?: AbortSignal | null;
|
||||
apiKeyAllowedConnections?: string[] | null;
|
||||
hiddenModelsByProvider?: HiddenModelsByProvider;
|
||||
videoTranscriptSensitive?: boolean;
|
||||
videoTranscriptDescriptionFingerprints?: readonly string[];
|
||||
perTargetAdmission?: PerTargetAdmissionHook | null;
|
||||
deferContextOverflowWhenCompressible?: boolean;
|
||||
compressionExclusions?: import("../compression/exclusions.ts").CompressionExclusions;
|
||||
@@ -550,7 +524,6 @@ export async function tryFusionDispatch(args: {
|
||||
perTargetAdmission: args.perTargetAdmission,
|
||||
judgeModel,
|
||||
tuning: fusionTuning,
|
||||
videoTranscriptSensitive: args.videoTranscriptSensitive,
|
||||
});
|
||||
}
|
||||
|
||||
@@ -703,8 +676,6 @@ export async function tryRuntimeUnitDispatch(args: {
|
||||
signal?: AbortSignal | null;
|
||||
apiKeyAllowedConnections?: string[] | null;
|
||||
hiddenModelsByProvider?: HiddenModelsByProvider;
|
||||
videoTranscriptSensitive?: boolean;
|
||||
videoTranscriptDescriptionFingerprints?: readonly string[];
|
||||
perTargetAdmission?: PerTargetAdmissionHook | null;
|
||||
deferContextOverflowWhenCompressible?: boolean;
|
||||
compressionExclusions?: import("../compression/exclusions.ts").CompressionExclusions;
|
||||
|
||||
@@ -5,8 +5,7 @@
|
||||
* @changes
|
||||
* - [2026-07-24] [Composer] - Skip execute-mode units at concurrency cap before dispatch
|
||||
*/
|
||||
import { errorResponse, errorResponseWithComboDiagnostics } from "../../utils/error.ts";
|
||||
import type { ComboDiagnostics } from "../../utils/error.ts";
|
||||
import { errorResponse } from "../../utils/error.ts";
|
||||
import { recordComboRequest } from "../comboMetrics.ts";
|
||||
import { resolveDelayMs } from "./comboPredicates.ts";
|
||||
import { isRuntimeUnitAtConcurrencyCap } from "./runtimeUnitCapacity.ts";
|
||||
@@ -217,17 +216,6 @@ export async function executeRuntimeUnitCombo(args: {
|
||||
};
|
||||
const finalFailure = (response: Response): Response =>
|
||||
withQuotaExhaustionClassification(response, observedFailure ? allObservedFailuresQuota : null);
|
||||
// #11462: attempts already made this loop, tracked for the attempt-budget-exceeded
|
||||
// diagnostics trace below (mirrors the poolSize/attemptOrder shape combo.ts already
|
||||
// attaches for the priority/round-robin strategies).
|
||||
const attemptedUnits: Array<{ provider: string; model: string }> = [];
|
||||
const buildAttemptBudgetDiag = (): ComboDiagnostics => ({
|
||||
poolSize: orderedUnits.length,
|
||||
attempted: args.nesting.attemptBudget.count,
|
||||
excluded: [],
|
||||
attemptOrder: attemptedUnits,
|
||||
terminalReason: "max_attempts_exceeded",
|
||||
});
|
||||
|
||||
for (const unit of orderedUnits) {
|
||||
const protectedPriorityUnit =
|
||||
@@ -259,21 +247,13 @@ export async function executeRuntimeUnitCombo(args: {
|
||||
}
|
||||
args.nesting.attemptBudget.count += 1;
|
||||
if (args.nesting.attemptBudget.count > args.nesting.attemptBudget.limit) {
|
||||
lastResponse = errorResponseWithComboDiagnostics(
|
||||
503,
|
||||
"Maximum combo retry limit reached",
|
||||
buildAttemptBudgetDiag()
|
||||
);
|
||||
lastResponse = errorResponse(503, "Maximum combo retry limit reached");
|
||||
await observeFailure(lastResponse, unit);
|
||||
return { response: finalFailure(lastResponse), unit };
|
||||
}
|
||||
if (retry > 0) {
|
||||
await new Promise((resolve) => setTimeout(resolve, retryDelayMs));
|
||||
}
|
||||
attemptedUnits.push({
|
||||
provider: unit.kind === "model" ? unit.provider : "combo-ref",
|
||||
model: unitDisplayName(unit),
|
||||
});
|
||||
args.log.info(
|
||||
"COMBO",
|
||||
`Trying ${unit.kind} ${unitDisplayName(unit)}${retry > 0 ? ` (retry ${retry})` : ""}`
|
||||
|
||||
@@ -114,10 +114,6 @@ export type HandleComboChatOptions = {
|
||||
apiKeyAllowedConnections?: string[] | null;
|
||||
nesting?: ComboNestingContext | null;
|
||||
hiddenModelsByProvider?: HiddenModelsByProvider;
|
||||
/** Request-scoped retention bit derived before guardrails can replace a video carrier. */
|
||||
videoTranscriptSensitive?: boolean;
|
||||
/** Exact bounded identities emitted by a modified Video Bridge guardrail. */
|
||||
videoTranscriptDescriptionFingerprints?: readonly string[];
|
||||
/** Native Responses clients (for example Codex CLI/Desktop) manage compaction themselves. */
|
||||
clientManagedResponsesContext?: boolean;
|
||||
/**
|
||||
|
||||
@@ -5,11 +5,6 @@ import {
|
||||
type HandoffPayload,
|
||||
upsertHandoff,
|
||||
} from "../../src/lib/db/contextHandoffs.ts";
|
||||
import {
|
||||
containsVideoTranscriptForLog,
|
||||
omitVideoTranscriptForLog,
|
||||
type VideoTranscriptLogContext,
|
||||
} from "../../src/lib/guardrails/videoTranscriptLogRedaction.ts";
|
||||
import { estimateTokens } from "./contextManager.ts";
|
||||
import { stripMarkdownCodeFence } from "../utils/aiSdkCompat.ts";
|
||||
|
||||
@@ -48,23 +43,6 @@ export type MessageLike = {
|
||||
content?: unknown;
|
||||
};
|
||||
|
||||
type HandoffTranscriptContext = VideoTranscriptLogContext & {
|
||||
videoTranscriptSensitive?: boolean;
|
||||
};
|
||||
|
||||
function retainHandoffMessages(
|
||||
messages: MessageLike[],
|
||||
context: HandoffTranscriptContext
|
||||
): MessageLike[] | null {
|
||||
const source = Array.isArray(messages) ? messages : [];
|
||||
const transcriptSensitive =
|
||||
context.videoTranscriptSensitive === true || containsVideoTranscriptForLog(source, context);
|
||||
if (!transcriptSensitive) return source;
|
||||
|
||||
const retained = omitVideoTranscriptForLog(source, context);
|
||||
return Array.isArray(retained) ? (retained as MessageLike[]) : null;
|
||||
}
|
||||
|
||||
export interface ContextRelayConfig {
|
||||
handoffModel?: string;
|
||||
handoffThreshold?: number;
|
||||
@@ -467,10 +445,6 @@ export function maybeGenerateHandoff(options: {
|
||||
model: string;
|
||||
expiresAt: string | null;
|
||||
config?: ContextRelayConfig | null;
|
||||
/** Trusted request bit for carriers already replaced by the Video Bridge guardrail. */
|
||||
videoTranscriptSensitive?: boolean;
|
||||
/** Exact bounded identities emitted by a modified Video Bridge guardrail. */
|
||||
trustedDescriptionFingerprints?: readonly string[];
|
||||
handleSingleModel: (body: Record<string, unknown>, modelStr: string) => Promise<Response>;
|
||||
}): void {
|
||||
if (!options.sessionId || !options.connectionId) return;
|
||||
@@ -480,12 +454,6 @@ export function maybeGenerateHandoff(options: {
|
||||
if (options.percentUsed < relayConfig.handoffThreshold) return;
|
||||
if (options.percentUsed >= HANDOFF_EXHAUSTION_THRESHOLD) return;
|
||||
|
||||
const retainedMessages = retainHandoffMessages(options.messages, {
|
||||
videoTranscriptSensitive: options.videoTranscriptSensitive,
|
||||
trustedDescriptionFingerprints: options.trustedDescriptionFingerprints,
|
||||
});
|
||||
if (!retainedMessages) return;
|
||||
|
||||
cleanupExpiredHandoffs();
|
||||
if (hasActiveHandoff(options.sessionId, options.comboName)) return;
|
||||
const inflightKey = getInflightKey(options.sessionId, options.comboName);
|
||||
@@ -495,7 +463,6 @@ export function maybeGenerateHandoff(options: {
|
||||
setImmediate(() => {
|
||||
generateHandoffAsync({
|
||||
...options,
|
||||
messages: retainedMessages,
|
||||
sessionId: options.sessionId as string,
|
||||
connectionId: options.connectionId as string,
|
||||
config: relayConfig,
|
||||
@@ -723,10 +690,6 @@ export function maybeGenerateUniversalHandoff(options: {
|
||||
prevModel: string | null;
|
||||
currModel: string;
|
||||
universalConfig: UniversalHandoffConfig;
|
||||
/** Trusted request bit for carriers already replaced by the Video Bridge guardrail. */
|
||||
videoTranscriptSensitive?: boolean;
|
||||
/** Exact bounded identities emitted by a modified Video Bridge guardrail. */
|
||||
trustedDescriptionFingerprints?: readonly string[];
|
||||
handleSingleModel: (body: Record<string, unknown>, modelStr: string) => Promise<Response>;
|
||||
}): void {
|
||||
const decision = shouldGenerateUniversalHandoff({
|
||||
@@ -740,12 +703,6 @@ export function maybeGenerateUniversalHandoff(options: {
|
||||
if (decision !== "generate") return;
|
||||
if (!options.sessionId) return;
|
||||
|
||||
const retainedMessages = retainHandoffMessages(options.messages, {
|
||||
videoTranscriptSensitive: options.videoTranscriptSensitive,
|
||||
trustedDescriptionFingerprints: options.trustedDescriptionFingerprints,
|
||||
});
|
||||
if (!retainedMessages) return;
|
||||
|
||||
const inflightKey = getInflightKey(options.sessionId, options.comboName);
|
||||
if (inflightHandoffGenerations.has(inflightKey)) return;
|
||||
inflightHandoffGenerations.add(inflightKey);
|
||||
@@ -756,7 +713,7 @@ export function maybeGenerateUniversalHandoff(options: {
|
||||
generateUniversalHandoffAsync({
|
||||
sessionId: options.sessionId as string,
|
||||
comboName: options.comboName,
|
||||
messages: retainedMessages,
|
||||
messages: options.messages,
|
||||
prevModel: options.prevModel || "unknown",
|
||||
currModel: options.currModel,
|
||||
handoffModel: options.universalConfig.handoffModel || options.currModel,
|
||||
|
||||
@@ -20,7 +20,6 @@
|
||||
*/
|
||||
import { errorResponse, sanitizeErrorMessage } from "../utils/error.ts";
|
||||
import { extractTextContent } from "../translator/helpers/geminiHelper.ts";
|
||||
import { redactVideoTranscriptSensitiveText } from "../../src/lib/guardrails/videoTranscriptLogRedaction.ts";
|
||||
import type { PerTargetAdmissionHook } from "./admission/types.ts";
|
||||
import type { ComboLogger, HandleSingleModel, ResolvedComboTarget } from "./combo/types.ts";
|
||||
|
||||
@@ -232,8 +231,6 @@ export type HandleFusionChatOptions = {
|
||||
tuning?: FusionTuning | null;
|
||||
/** #9654 Wave 2: per-target lane-aware admission probe (see HandleComboChatOptions). */
|
||||
perTargetAdmission?: PerTargetAdmissionHook | null;
|
||||
/** Request-scoped bit set before Video Bridge can replace the raw carrier. */
|
||||
videoTranscriptSensitive?: boolean;
|
||||
};
|
||||
|
||||
function getFusionModelString(model: FusionModel): string {
|
||||
@@ -279,7 +276,6 @@ export async function handleFusionChat({
|
||||
judgeTarget,
|
||||
tuning,
|
||||
perTargetAdmission,
|
||||
videoTranscriptSensitive = false,
|
||||
}: HandleFusionChatOptions): Promise<Response> {
|
||||
const panel = Array.isArray(models) ? models.filter(Boolean) : [];
|
||||
if (panel.length === 0) {
|
||||
@@ -404,10 +400,7 @@ export async function handleFusionChat({
|
||||
}
|
||||
if (sentinel.__error) {
|
||||
log.warn("FUSION", `Panel ${model} threw`, {
|
||||
error: redactVideoTranscriptSensitiveText(
|
||||
sanitizeErrorMessage(sentinel.__error as Error),
|
||||
videoTranscriptSensitive
|
||||
),
|
||||
error: sanitizeErrorMessage(sentinel.__error as Error),
|
||||
});
|
||||
failures.push({ model, reason: "threw" });
|
||||
continue;
|
||||
@@ -435,10 +428,7 @@ export async function handleFusionChat({
|
||||
}
|
||||
} catch (e) {
|
||||
log.warn("FUSION", `Panel ${model} unparseable`, {
|
||||
error: redactVideoTranscriptSensitiveText(
|
||||
sanitizeErrorMessage(e as Error),
|
||||
videoTranscriptSensitive
|
||||
),
|
||||
error: sanitizeErrorMessage(e as Error),
|
||||
});
|
||||
failures.push({ model, reason: "unparseable" });
|
||||
}
|
||||
|
||||
@@ -140,7 +140,6 @@ type AssistantMessageLike = {
|
||||
type AssistantMessageCacheContext = {
|
||||
scope?: string;
|
||||
historyMessages?: AssistantMessageLike[];
|
||||
videoTranscriptSensitive?: boolean;
|
||||
};
|
||||
|
||||
type ToolCallLike = {
|
||||
@@ -335,7 +334,6 @@ export function cacheReasoningFromAssistantMessage(
|
||||
model: string,
|
||||
context?: AssistantMessageCacheContext
|
||||
): number {
|
||||
if (context?.videoTranscriptSensitive === true) return 0;
|
||||
if (!message || message.role !== "assistant") {
|
||||
return 0;
|
||||
}
|
||||
|
||||
@@ -141,8 +141,16 @@ function extractSystemContent(body: Record<string, unknown>): unknown {
|
||||
*
|
||||
* It is a PLAINTEXT prefix rather than digest input, matching
|
||||
* `semanticCache.generateSignature` (#3740): the id is an internal namespace
|
||||
* key, not a credential, and keeping it out of the digest avoids the
|
||||
* false-positive CodeQL js/insufficient-password-hash on a cache/dedup key.
|
||||
* key, not a credential, and a namespace you can read off the key is worth more
|
||||
* than one you cannot when debugging a dedup collision.
|
||||
*
|
||||
* It does NOT dodge the CodeQL js/insufficient-password-hash false positive,
|
||||
* which the #3740 comment claims for its own version and which this comment
|
||||
* claimed too until alert #874 was raised on the `createHash` below anyway.
|
||||
* Once an API-key-derived value reaches this file at all, the query flags the
|
||||
* sibling digest regardless of what actually goes into it. Dismissed per HR#14;
|
||||
* expect it to come back on any edit here, and do not "fix" it with a KDF —
|
||||
* that would break the determinism dedup depends on.
|
||||
*
|
||||
* Omitting `tenantId` keeps the un-namespaced hash. Keyless local-first
|
||||
* deployments have no tenant boundary to preserve, and every such install would
|
||||
|
||||
@@ -136,6 +136,7 @@ export function claudeToGeminiRequest(model, body, stream, credentials = null) {
|
||||
const omittedToolCallIds = new Set<string>();
|
||||
for (const msg of body.messages) {
|
||||
const parts = [];
|
||||
let shouldUseEmbeddedSignature = true;
|
||||
|
||||
if (Array.isArray(msg.content)) {
|
||||
for (const block of msg.content) {
|
||||
@@ -159,15 +160,15 @@ export function claudeToGeminiRequest(model, body, stream, credentials = null) {
|
||||
break;
|
||||
}
|
||||
|
||||
// #11510: each functionCall part carries its OWN resolved
|
||||
// thoughtSignature — a parallel (multi tool_use) turn can have a
|
||||
// real, individually-valid signature per tool call, and Gemini
|
||||
// 3.x rejects the request if any functionCall in the turn is
|
||||
// missing one. Previously only the first functionCall of the
|
||||
// message kept its signature; this dropped valid signatures for
|
||||
// every subsequent parallel tool call in the same turn.
|
||||
const embeddedThoughtSignature = shouldUseEmbeddedSignature
|
||||
? signatureForToolCall
|
||||
: undefined;
|
||||
if (embeddedThoughtSignature) {
|
||||
shouldUseEmbeddedSignature = false;
|
||||
}
|
||||
|
||||
parts.push({
|
||||
...(signatureForToolCall ? { thoughtSignature: signatureForToolCall } : {}),
|
||||
...(embeddedThoughtSignature ? { thoughtSignature: embeddedThoughtSignature } : {}),
|
||||
functionCall: {
|
||||
...(stripFunctionCallId ? {} : { id: block.id }),
|
||||
name: sanitizeToolName(block.name),
|
||||
|
||||
@@ -383,6 +383,7 @@ function openaiToGeminiBase(
|
||||
if (toolCalls && Array.isArray(toolCalls)) {
|
||||
const toolCallIds: string[] = [];
|
||||
const resolvedSignatures = new Map<string, string>();
|
||||
let firstPersistedSignature: string | undefined;
|
||||
for (const tc of toolCalls) {
|
||||
const id = tc.id as string;
|
||||
const resolved = resolveGeminiThoughtSignature(
|
||||
@@ -391,9 +392,11 @@ function openaiToGeminiBase(
|
||||
);
|
||||
if (typeof resolved === "string" && resolved.length > 0) {
|
||||
resolvedSignatures.set(id, resolved);
|
||||
firstPersistedSignature ??= resolved;
|
||||
}
|
||||
}
|
||||
|
||||
let shouldUseEmbeddedSignature = !parts.some((p) => p.thoughtSignature);
|
||||
const signaturelessToolCallMode = toolNameOptions.signaturelessToolCallMode;
|
||||
const stringifySignaturelessToolCalls = signaturelessToolCallMode === "text";
|
||||
const contextualizeSignaturelessToolResponses =
|
||||
@@ -430,14 +433,13 @@ function openaiToGeminiBase(
|
||||
}
|
||||
|
||||
const args = tryParseJSON(fn.arguments || "{}");
|
||||
// #11510: each functionCall part carries its OWN resolved
|
||||
// thoughtSignature — a parallel (multi tool_calls) turn can have a
|
||||
// real, individually-valid signature per tool call, and Gemini 3.x
|
||||
// rejects the request if any functionCall in the turn is missing
|
||||
// one. Previously only the first functionCall of the message kept
|
||||
// its signature; this dropped valid signatures for every
|
||||
// subsequent parallel tool call in the same turn.
|
||||
const embeddedThoughtSignature = signatureForToolCall;
|
||||
const embeddedThoughtSignature = shouldUseEmbeddedSignature
|
||||
? firstPersistedSignature || signatureForToolCall
|
||||
: undefined;
|
||||
|
||||
if (embeddedThoughtSignature) {
|
||||
shouldUseEmbeddedSignature = false;
|
||||
}
|
||||
|
||||
// Gemini expects the signature on the functionCall part itself.
|
||||
// If we are in a mode where missing signatures cause 400s (and we couldn't find one),
|
||||
|
||||
@@ -1,52 +0,0 @@
|
||||
// #11526: the fetch-start (headers-wait) phase had no ceiling comparable to a
|
||||
// real client's patience for STREAMING requests — it inherited the flat,
|
||||
// non-adaptive FETCH_TIMEOUT_MS (default 600_000ms / 10 minutes), five times
|
||||
// longer than Codex's own ~120s hard client-abort window. When an upstream
|
||||
// never returns a response at all (not even headers), OmniRoute kept the
|
||||
// connection open with nothing but keepalives, guaranteeing the client gave
|
||||
// up first with an opaque 499 instead of OmniRoute detecting the stall and
|
||||
// failing fast/over within a client-realistic window.
|
||||
//
|
||||
// This mirrors the adaptive philosophy of streamReadinessPolicy.ts's
|
||||
// resolveStreamReadinessTimeout (which already protects the BODY phase, after
|
||||
// headers arrive) but inverted: instead of bumping a small base timeout up for
|
||||
// heavy payloads, it caps an oversized base timeout down for the HEADERS
|
||||
// phase of streaming requests specifically. Non-streaming requests are left
|
||||
// on the existing flat default — providers that are legitimately slow to
|
||||
// accept a connection (but not streaming SSE) are unaffected.
|
||||
|
||||
export type FetchStartTimeoutPolicyInput = {
|
||||
baseTimeoutMs: number;
|
||||
/** Only streaming requests are capped — non-streaming keeps the flat default. */
|
||||
stream?: boolean | null;
|
||||
capMs?: number;
|
||||
};
|
||||
|
||||
export type FetchStartTimeoutPolicyResult = {
|
||||
timeoutMs: number;
|
||||
baseTimeoutMs: number;
|
||||
/** True when the base timeout was reduced by the streaming cap. */
|
||||
capped: boolean;
|
||||
};
|
||||
|
||||
// Codex's documented hard client-abort window for a stalled turn (nothing but
|
||||
// keepalives in flight) is ~120s. Keep the cap safely under that so OmniRoute's
|
||||
// own headers-phase watchdog always fires before the client gives up on its own.
|
||||
export const CODEX_CLIENT_ABORT_MS = 120_000;
|
||||
export const DEFAULT_FETCH_START_TIMEOUT_CAP_MS = 110_000;
|
||||
|
||||
export function resolveFetchStartTimeout(
|
||||
input: FetchStartTimeoutPolicyInput
|
||||
): FetchStartTimeoutPolicyResult {
|
||||
const baseTimeoutMs = Math.max(0, Math.floor(input.baseTimeoutMs || 0));
|
||||
if (baseTimeoutMs <= 0 || !input.stream) {
|
||||
return { timeoutMs: baseTimeoutMs, baseTimeoutMs, capped: false };
|
||||
}
|
||||
|
||||
const capMs = Math.max(0, Math.floor(input.capMs ?? DEFAULT_FETCH_START_TIMEOUT_CAP_MS));
|
||||
if (capMs <= 0 || baseTimeoutMs <= capMs) {
|
||||
return { timeoutMs: baseTimeoutMs, baseTimeoutMs, capped: false };
|
||||
}
|
||||
|
||||
return { timeoutMs: capMs, baseTimeoutMs, capped: true };
|
||||
}
|
||||
@@ -1,11 +1,5 @@
|
||||
import { getPendingById } from "@/lib/usage/usageHistory";
|
||||
import { getChatLogMaxDepth, getChatLogArrayTailItems } from "@/lib/logEnv";
|
||||
import {
|
||||
containsVideoTranscriptForLog,
|
||||
omitVideoTranscriptForLog,
|
||||
type VideoTranscriptLogContext,
|
||||
VIDEO_TRANSCRIPT_LOG_OMISSION_MARKER,
|
||||
} from "@/lib/guardrails/videoTranscriptLogRedaction";
|
||||
import { sanitizeErrorMessage } from "./error.ts";
|
||||
|
||||
type JsonRecord = Record<string, unknown>;
|
||||
@@ -18,7 +12,6 @@ type HeaderInput =
|
||||
| undefined;
|
||||
|
||||
export type RequestPipelinePayloads = {
|
||||
_omnirouteVideoTranscriptRedacted?: true;
|
||||
routeDecision?: JsonRecord;
|
||||
clientRawRequest?: JsonRecord;
|
||||
openaiRequest?: JsonRecord;
|
||||
@@ -51,7 +44,6 @@ type RequestLogger = {
|
||||
appendConvertedChunk: (chunk: string) => void;
|
||||
logError: (error: unknown, requestBody?: unknown) => void;
|
||||
getPipelinePayloads: () => RequestPipelinePayloads | null;
|
||||
isVideoTranscriptSensitive: () => boolean;
|
||||
};
|
||||
|
||||
type RequestLoggerOptions = {
|
||||
@@ -63,13 +55,6 @@ type RequestLoggerOptions = {
|
||||
model?: string;
|
||||
provider?: string;
|
||||
connectionId?: string | null;
|
||||
/**
|
||||
* Server-derived from a recognized video carrier, successful bridge result, or bounded
|
||||
* unknown-sensitive detector overflow; never from delimiter-shaped prose alone.
|
||||
*/
|
||||
videoTranscriptSensitive?: boolean;
|
||||
/** Exact SHA-256 identities of transcript descriptions generated by the guardrail. */
|
||||
videoTranscriptDescriptionFingerprints?: readonly string[];
|
||||
};
|
||||
|
||||
const DEFAULT_MAX_STREAM_CHUNK_BYTES = 128 * 1024;
|
||||
@@ -174,7 +159,7 @@ function truncateLogString(value: string, maxLength = MAX_LOG_STRING_LENGTH): st
|
||||
* recursing into an object's values, enabling the per-field exemption above.
|
||||
* Top-level arrays (no key context) remain subject to truncation.
|
||||
*/
|
||||
function cloneBoundedForLogValue(value: unknown, depth = 0, key: string | null = null): unknown {
|
||||
export function cloneBoundedForLog(value: unknown, depth = 0, key: string | null = null): unknown {
|
||||
if (value === null || value === undefined) return value;
|
||||
if (typeof value === "string") return truncateLogString(value);
|
||||
if (typeof value !== "object") return value;
|
||||
@@ -193,15 +178,12 @@ function cloneBoundedForLogValue(value: unknown, depth = 0, key: string | null =
|
||||
// item and rewrite originalLength with the truncated length (25 instead of the true 800), so
|
||||
// the log would misreport how much was cut. Keep the original marker, re-bound only the tail.
|
||||
if (isTruncatedArrayMarker(value[0])) {
|
||||
return [
|
||||
value[0],
|
||||
...value.slice(1).map((item) => cloneBoundedForLogValue(item, depth + 1, null)),
|
||||
];
|
||||
return [value[0], ...value.slice(1).map((item) => cloneBoundedForLog(item, depth + 1))];
|
||||
}
|
||||
const exempt = key === "tools";
|
||||
const shouldTruncate = !exempt && value.length > MAX_LOG_ARRAY_ITEMS;
|
||||
const source = shouldTruncate ? value.slice(-MAX_LOG_ARRAY_ITEMS) : value;
|
||||
const mapped = source.map((item) => cloneBoundedForLogValue(item, depth + 1, null));
|
||||
const mapped = source.map((item) => cloneBoundedForLog(item, depth + 1));
|
||||
if (shouldTruncate) {
|
||||
return [
|
||||
{
|
||||
@@ -224,7 +206,7 @@ function cloneBoundedForLogValue(value: unknown, depth = 0, key: string | null =
|
||||
([k]) => !(carried > 0 && k === TRUNCATED_KEYS_MARKER)
|
||||
);
|
||||
for (const [k, item] of entries.slice(0, MAX_LOG_OBJECT_KEYS)) {
|
||||
result[k] = cloneBoundedForLogValue(item, depth + 1, k);
|
||||
result[k] = cloneBoundedForLog(item, depth + 1, k);
|
||||
}
|
||||
const dropped = Math.max(0, entries.length - MAX_LOG_OBJECT_KEYS) + carried;
|
||||
if (dropped > 0) {
|
||||
@@ -233,17 +215,6 @@ function cloneBoundedForLogValue(value: unknown, depth = 0, key: string | null =
|
||||
return result;
|
||||
}
|
||||
|
||||
export function cloneBoundedForLog(
|
||||
value: unknown,
|
||||
depth = 0,
|
||||
key: string | null = null,
|
||||
descriptionContext: VideoTranscriptLogContext = {}
|
||||
): unknown {
|
||||
const transcriptSafeValue =
|
||||
depth === 0 ? omitVideoTranscriptForLog(value, descriptionContext) : value;
|
||||
return cloneBoundedForLogValue(transcriptSafeValue, depth, key);
|
||||
}
|
||||
|
||||
function appendBoundedChunk(
|
||||
chunks: string[],
|
||||
bytes: { value: number; truncated: boolean },
|
||||
@@ -306,7 +277,7 @@ function compactPipelinePayloads(
|
||||
continue;
|
||||
}
|
||||
|
||||
(result as Record<string, unknown>)[key] = value;
|
||||
result[key as keyof RequestPipelinePayloads] = value;
|
||||
}
|
||||
|
||||
return hasOwnValues(result) ? result : null;
|
||||
@@ -327,7 +298,6 @@ function makeStreamChunkMethods(options: RequestLoggerOptions, captureChunks: bo
|
||||
? Number(options.maxStreamChunkItems)
|
||||
: DEFAULT_MAX_STREAM_CHUNK_ITEMS;
|
||||
let pendingPushed = false;
|
||||
let videoTranscriptSensitive = options.videoTranscriptSensitive === true;
|
||||
|
||||
const push = () => {
|
||||
if (pendingPushed) return;
|
||||
@@ -360,7 +330,7 @@ function makeStreamChunkMethods(options: RequestLoggerOptions, captureChunks: bo
|
||||
};
|
||||
|
||||
const append = (arr: string[], bytes: { value: number; truncated: boolean }, chunk: string) => {
|
||||
if (!captureChunks || videoTranscriptSensitive) return;
|
||||
if (!captureChunks) return;
|
||||
push();
|
||||
const ts = new Date().toISOString().slice(11, 23);
|
||||
appendBoundedChunk(arr, bytes, `[${ts}] ${chunk}`, maxBytes, maxItems);
|
||||
@@ -378,17 +348,6 @@ function makeStreamChunkMethods(options: RequestLoggerOptions, captureChunks: bo
|
||||
appendConvertedChunk(chunk: string) {
|
||||
append(streamChunks.client, streamChunkBytes.client, chunk);
|
||||
},
|
||||
suppressVideoTranscriptChunks() {
|
||||
videoTranscriptSensitive = true;
|
||||
for (const chunks of Object.values(streamChunks)) chunks.splice(0, chunks.length);
|
||||
for (const state of Object.values(streamChunkBytes)) {
|
||||
state.value = 0;
|
||||
state.truncated = false;
|
||||
}
|
||||
},
|
||||
isVideoTranscriptSensitive() {
|
||||
return videoTranscriptSensitive;
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
@@ -403,58 +362,26 @@ export async function createRequestLogger(
|
||||
// so that active requests always have real-time stream data available via
|
||||
// the /api/logs/active endpoint.
|
||||
const chunkMethods = makeStreamChunkMethods(options, captureStreamChunks);
|
||||
const descriptionLogContext: VideoTranscriptLogContext = {
|
||||
trustedDescriptionFingerprints: options.videoTranscriptDescriptionFingerprints ?? [],
|
||||
};
|
||||
const suppressTranscriptChunksIfNeeded = (
|
||||
descriptionContext: VideoTranscriptLogContext,
|
||||
...values: unknown[]
|
||||
): void => {
|
||||
if (values.some((value) => containsVideoTranscriptForLog(value, descriptionContext))) {
|
||||
chunkMethods.suppressVideoTranscriptChunks();
|
||||
}
|
||||
};
|
||||
const cloneRequestBody = (
|
||||
value: unknown,
|
||||
descriptionContext: VideoTranscriptLogContext
|
||||
): unknown => cloneBoundedForLog(value, 0, null, descriptionContext);
|
||||
const cloneResponseBody = (value: unknown): unknown =>
|
||||
chunkMethods.isVideoTranscriptSensitive()
|
||||
? VIDEO_TRANSCRIPT_LOG_OMISSION_MARKER
|
||||
: cloneBoundedForLog(value);
|
||||
|
||||
if (options.enabled === false) {
|
||||
let routeDecision: JsonRecord | null = null;
|
||||
return {
|
||||
sessionPath: null,
|
||||
logClientRawRequest(_endpoint, body) {
|
||||
suppressTranscriptChunksIfNeeded({}, body);
|
||||
},
|
||||
logClientRawRequest() {},
|
||||
logRouteDecision(decision) {
|
||||
routeDecision = cloneBoundedForLog(decision) as JsonRecord;
|
||||
},
|
||||
logOpenAIRequest(body) {
|
||||
suppressTranscriptChunksIfNeeded(descriptionLogContext, body);
|
||||
},
|
||||
logTargetRequest(_url, _headers, body) {
|
||||
suppressTranscriptChunksIfNeeded(descriptionLogContext, body);
|
||||
},
|
||||
logProviderResponse(_status, _statusText, _headers, body) {
|
||||
void body;
|
||||
},
|
||||
logOpenAIRequest() {},
|
||||
logTargetRequest() {},
|
||||
logProviderResponse() {},
|
||||
appendProviderChunk: chunkMethods.appendProviderChunk,
|
||||
appendOpenAIChunk: chunkMethods.appendOpenAIChunk,
|
||||
logConvertedResponse(body) {
|
||||
void body;
|
||||
},
|
||||
logConvertedResponse() {},
|
||||
appendConvertedChunk: chunkMethods.appendConvertedChunk,
|
||||
logError(_error, requestBody) {
|
||||
suppressTranscriptChunksIfNeeded(descriptionLogContext, requestBody);
|
||||
},
|
||||
logError() {},
|
||||
getPipelinePayloads() {
|
||||
return routeDecision ? { routeDecision } : null;
|
||||
},
|
||||
isVideoTranscriptSensitive: chunkMethods.isVideoTranscriptSensitive,
|
||||
};
|
||||
}
|
||||
|
||||
@@ -466,12 +393,11 @@ export async function createRequestLogger(
|
||||
sessionPath: null,
|
||||
|
||||
logClientRawRequest(endpoint, body, headers = {}) {
|
||||
suppressTranscriptChunksIfNeeded({}, body);
|
||||
payloads.clientRawRequest = {
|
||||
timestamp: new Date().toISOString(),
|
||||
endpoint,
|
||||
headers: maskSensitiveHeaders(headers),
|
||||
body: cloneRequestBody(body, {}),
|
||||
body: cloneBoundedForLog(body),
|
||||
};
|
||||
},
|
||||
|
||||
@@ -480,20 +406,18 @@ export async function createRequestLogger(
|
||||
},
|
||||
|
||||
logOpenAIRequest(body) {
|
||||
suppressTranscriptChunksIfNeeded(descriptionLogContext, body);
|
||||
payloads.openaiRequest = {
|
||||
timestamp: new Date().toISOString(),
|
||||
body: cloneRequestBody(body, descriptionLogContext),
|
||||
body: cloneBoundedForLog(body),
|
||||
};
|
||||
},
|
||||
|
||||
logTargetRequest(url, headers, body) {
|
||||
suppressTranscriptChunksIfNeeded(descriptionLogContext, body);
|
||||
payloads.providerRequest = {
|
||||
timestamp: new Date().toISOString(),
|
||||
url,
|
||||
headers: maskSensitiveHeaders(headers),
|
||||
body: cloneRequestBody(body, descriptionLogContext),
|
||||
body: cloneBoundedForLog(body),
|
||||
};
|
||||
},
|
||||
|
||||
@@ -503,7 +427,7 @@ export async function createRequestLogger(
|
||||
status,
|
||||
statusText,
|
||||
headers: maskSensitiveHeaders(headers),
|
||||
body: cloneResponseBody(body),
|
||||
body: cloneBoundedForLog(body),
|
||||
};
|
||||
},
|
||||
|
||||
@@ -512,25 +436,21 @@ export async function createRequestLogger(
|
||||
logConvertedResponse(body) {
|
||||
payloads.clientResponse = {
|
||||
timestamp: new Date().toISOString(),
|
||||
body: cloneResponseBody(body),
|
||||
body: cloneBoundedForLog(body),
|
||||
};
|
||||
},
|
||||
appendConvertedChunk: chunkMethods.appendConvertedChunk,
|
||||
|
||||
logError(error, requestBody = null) {
|
||||
suppressTranscriptChunksIfNeeded(descriptionLogContext, requestBody);
|
||||
payloads.error = {
|
||||
timestamp: new Date().toISOString(),
|
||||
error: chunkMethods.isVideoTranscriptSensitive()
|
||||
? VIDEO_TRANSCRIPT_LOG_OMISSION_MARKER
|
||||
: sanitizeErrorMessage(error instanceof Error ? error.message : String(error)),
|
||||
requestBody: cloneRequestBody(requestBody, descriptionLogContext),
|
||||
error: sanitizeErrorMessage(error instanceof Error ? error.message : String(error)),
|
||||
requestBody: cloneBoundedForLog(requestBody),
|
||||
};
|
||||
},
|
||||
|
||||
getPipelinePayloads() {
|
||||
return compactPipelinePayloads(payloads);
|
||||
},
|
||||
isVideoTranscriptSensitive: chunkMethods.isVideoTranscriptSensitive,
|
||||
};
|
||||
}
|
||||
|
||||
@@ -1,7 +1,6 @@
|
||||
import { VIDEO_TRANSCRIPT_LOG_OMISSION_MARKER } from "@/lib/guardrails/videoTranscriptLogRedaction";
|
||||
import { trackPendingRequest, appendRequestLog } from "@/lib/usageDb";
|
||||
import { translateResponse, initState } from "../translator/index.ts";
|
||||
import { FORMATS } from "../translator/formats.ts";
|
||||
import { trackPendingRequest, appendRequestLog } from "@/lib/usageDb";
|
||||
import {
|
||||
extractUsage,
|
||||
hasValidUsage,
|
||||
@@ -178,8 +177,6 @@ type StreamOptions = {
|
||||
* codex-compatible `namespace` + `name` fields.
|
||||
*/
|
||||
requestToolIdentityMap?: Map<string, { namespace: string; name: string }> | null;
|
||||
/** Omit request-sensitive transcript text from retained stream diagnostics only. */
|
||||
redactStreamDiagnosticsForLog?: boolean;
|
||||
};
|
||||
|
||||
type TranslateState = ReturnType<typeof initState> & {
|
||||
@@ -657,10 +654,7 @@ export function createSSEStream(options: StreamOptions = {}) {
|
||||
dropResponsesCommentary,
|
||||
customToolNames = new Set<string>(),
|
||||
requestToolIdentityMap = null,
|
||||
redactStreamDiagnosticsForLog = false,
|
||||
} = options;
|
||||
const retainDiagnosticForLog = (value: unknown): unknown =>
|
||||
redactStreamDiagnosticsForLog ? VIDEO_TRANSCRIPT_LOG_OMISSION_MARKER : value;
|
||||
const signatureNamespace = connectionId;
|
||||
// Request-body-size metric (for monitoring payload size distribution & correlation with TTFT).
|
||||
// The size is JSON-serialised byte count; stored as a performance mark detail so monitoring
|
||||
@@ -1011,10 +1005,7 @@ export function createSSEStream(options: StreamOptions = {}) {
|
||||
try {
|
||||
failureHandled = onFailure({ status: 502, message: msg, code: "empty_response" }) === true;
|
||||
} catch (e) {
|
||||
console.debug(
|
||||
`[STREAM] onFailure callback error (empty_response):`,
|
||||
retainDiagnosticForLog(e)
|
||||
);
|
||||
console.debug(`[STREAM] onFailure callback error (empty_response):`, e);
|
||||
}
|
||||
}
|
||||
if (decrementPendingRequest && !failureHandled) {
|
||||
@@ -1208,10 +1199,7 @@ export function createSSEStream(options: StreamOptions = {}) {
|
||||
type: "timeout_error",
|
||||
}) === true;
|
||||
} catch (e) {
|
||||
console.debug(
|
||||
`[STREAM] onFailure callback error (idle_timeout):`,
|
||||
retainDiagnosticForLog(e)
|
||||
);
|
||||
console.debug(`[STREAM] onFailure callback error (idle_timeout):`, e);
|
||||
}
|
||||
}
|
||||
if (!failureHandled) {
|
||||
@@ -1653,7 +1641,10 @@ export function createSSEStream(options: StreamOptions = {}) {
|
||||
isResponsesCommentaryMessageItem
|
||||
).items
|
||||
: passthroughResponsesOutputItems;
|
||||
const backfilled = backfillResponsesCompletedOutput(parsed, backfillCandidates);
|
||||
const backfilled = backfillResponsesCompletedOutput(
|
||||
parsed,
|
||||
backfillCandidates
|
||||
);
|
||||
const usageNormalized = normalizeUsage(parsed);
|
||||
if (
|
||||
stripped ||
|
||||
@@ -2049,7 +2040,7 @@ export function createSSEStream(options: StreamOptions = {}) {
|
||||
try {
|
||||
failureHandled = onFailure(failurePayload) === true;
|
||||
} catch (e) {
|
||||
console.debug(`[STREAM] onFailure callback error:`, retainDiagnosticForLog(e));
|
||||
console.debug(`[STREAM] onFailure callback error:`, e);
|
||||
}
|
||||
}
|
||||
clearIdleTimer();
|
||||
@@ -2633,10 +2624,7 @@ export function createSSEStream(options: StreamOptions = {}) {
|
||||
),
|
||||
});
|
||||
} catch (e) {
|
||||
console.debug(
|
||||
`[STREAM] onComplete callback error (${model || "unknown"}):`,
|
||||
retainDiagnosticForLog(e)
|
||||
);
|
||||
console.debug(`[STREAM] onComplete callback error (${model || "unknown"}):`, e);
|
||||
}
|
||||
} else {
|
||||
clearPendingRequestFromStream();
|
||||
@@ -2724,10 +2712,7 @@ export function createSSEStream(options: StreamOptions = {}) {
|
||||
type: err.type,
|
||||
}) === true;
|
||||
} catch (e) {
|
||||
console.debug(
|
||||
`[STREAM] onFailure callback error (${model || "unknown"}):`,
|
||||
retainDiagnosticForLog(e)
|
||||
);
|
||||
console.debug(`[STREAM] onFailure callback error (${model || "unknown"}):`, e);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -2755,7 +2740,7 @@ export function createSSEStream(options: StreamOptions = {}) {
|
||||
} catch (e) {
|
||||
console.debug(
|
||||
`[STREAM] onComplete callback error in error path (${model || "unknown"}):`,
|
||||
retainDiagnosticForLog(e)
|
||||
e
|
||||
);
|
||||
}
|
||||
}
|
||||
@@ -2960,18 +2945,14 @@ export function createSSEStream(options: StreamOptions = {}) {
|
||||
} catch (e) {
|
||||
console.debug(
|
||||
`[STREAM] onComplete callback error in flush (${model || "unknown"}):`,
|
||||
retainDiagnosticForLog(e)
|
||||
e
|
||||
);
|
||||
}
|
||||
} else {
|
||||
clearPendingRequestFromStream();
|
||||
}
|
||||
} catch (error) {
|
||||
const diagnostic = error instanceof Error ? error.message : error;
|
||||
console.log(
|
||||
`[STREAM] Error in flush (${model || "unknown"}):`,
|
||||
retainDiagnosticForLog(diagnostic)
|
||||
);
|
||||
console.log(`[STREAM] Error in flush (${model || "unknown"}):`, error.message || error);
|
||||
}
|
||||
},
|
||||
cancel(reason) {
|
||||
@@ -3001,8 +2982,7 @@ export function createSSETransformStreamWithLogger(
|
||||
copilotCompatibleReasoning = false,
|
||||
suppressThinkClose = false,
|
||||
customToolNames: ReadonlySet<string> = new Set(),
|
||||
requestToolIdentityMap: Map<string, { namespace: string; name: string }> | null = null,
|
||||
redactStreamDiagnosticsForLog = false
|
||||
requestToolIdentityMap: Map<string, { namespace: string; name: string }> | null = null
|
||||
) {
|
||||
return createSSEStream({
|
||||
mode: STREAM_MODE.TRANSLATE,
|
||||
@@ -3021,7 +3001,6 @@ export function createSSETransformStreamWithLogger(
|
||||
suppressThinkClose,
|
||||
customToolNames,
|
||||
requestToolIdentityMap,
|
||||
redactStreamDiagnosticsForLog,
|
||||
});
|
||||
}
|
||||
|
||||
@@ -3036,8 +3015,7 @@ export function createPassthroughStreamWithLogger(
|
||||
apiKeyInfo: unknown = null,
|
||||
onFailure: ((payload: StreamFailurePayload) => boolean | void | Promise<void>) | null = null,
|
||||
clientResponseFormat: string | null = null,
|
||||
requestToolIdentityMap: Map<string, { namespace: string; name: string }> | null = null,
|
||||
redactStreamDiagnosticsForLog = false
|
||||
requestToolIdentityMap: Map<string, { namespace: string; name: string }> | null = null
|
||||
) {
|
||||
return createSSEStream({
|
||||
mode: STREAM_MODE.PASSTHROUGH,
|
||||
@@ -3052,7 +3030,6 @@ export function createPassthroughStreamWithLogger(
|
||||
onFailure,
|
||||
clientResponseFormat,
|
||||
requestToolIdentityMap,
|
||||
redactStreamDiagnosticsForLog,
|
||||
});
|
||||
}
|
||||
|
||||
|
||||
@@ -92,7 +92,6 @@ export function finalizeStreamRequestLog({
|
||||
status,
|
||||
error,
|
||||
errorCode,
|
||||
videoTranscriptSensitive,
|
||||
onWarn,
|
||||
}: {
|
||||
pendingRequestId: string;
|
||||
@@ -104,7 +103,6 @@ export function finalizeStreamRequestLog({
|
||||
status: number;
|
||||
error?: string | null;
|
||||
errorCode?: string | null;
|
||||
videoTranscriptSensitive: boolean;
|
||||
onWarn?: (error: unknown) => void;
|
||||
}) {
|
||||
try {
|
||||
@@ -114,7 +112,6 @@ export function finalizeStreamRequestLog({
|
||||
status,
|
||||
error: error || null,
|
||||
errorCode: errorCode || null,
|
||||
videoTranscriptSensitive,
|
||||
});
|
||||
if (!completedById) {
|
||||
finalizeMostRecentPendingRequest(model, provider, connectionId, {
|
||||
@@ -123,7 +120,6 @@ export function finalizeStreamRequestLog({
|
||||
status,
|
||||
error: error || null,
|
||||
errorCode: errorCode || null,
|
||||
videoTranscriptSensitive,
|
||||
});
|
||||
}
|
||||
} catch (error) {
|
||||
|
||||
@@ -1,4 +1,3 @@
|
||||
import { VIDEO_TRANSCRIPT_LOG_OMISSION_MARKER } from "@/lib/guardrails/videoTranscriptLogRedaction";
|
||||
import { trackPendingRequest } from "@/lib/usageDb";
|
||||
import { STREAM_IDLE_TIMEOUT_MS } from "../config/constants.ts";
|
||||
import { FORMATS } from "../translator/formats.ts";
|
||||
@@ -40,7 +39,6 @@ type StreamControllerOptions = {
|
||||
clientAbortSignal?: AbortSignal | null;
|
||||
allowCompletedToolHandoffGrace?: boolean;
|
||||
clientDisconnectGracePeriodMs?: number;
|
||||
redactStreamDiagnosticsForLog?: boolean;
|
||||
};
|
||||
|
||||
type StreamController = ReturnType<typeof createStreamController>;
|
||||
@@ -245,7 +243,6 @@ export function createStreamController({
|
||||
clientAbortSignal,
|
||||
allowCompletedToolHandoffGrace = false,
|
||||
clientDisconnectGracePeriodMs = 0,
|
||||
redactStreamDiagnosticsForLog = false,
|
||||
}: StreamControllerOptions = {}) {
|
||||
const abortController = new AbortController();
|
||||
const startTime = Date.now();
|
||||
@@ -256,9 +253,6 @@ export function createStreamController({
|
||||
let pendingRequestCleared = false;
|
||||
let cleanupClientAbortSignal: (() => void) | null = null;
|
||||
|
||||
const retainDiagnosticForLog = (value: unknown): unknown =>
|
||||
redactStreamDiagnosticsForLog ? VIDEO_TRANSCRIPT_LOG_OMISSION_MARKER : value;
|
||||
|
||||
const logStream = (status) => {
|
||||
const duration = Date.now() - startTime;
|
||||
const p = provider?.toUpperCase() || "UNKNOWN";
|
||||
@@ -285,7 +279,7 @@ export function createStreamController({
|
||||
} catch (e) {
|
||||
console.error(
|
||||
`[${getTimeString()}] [streamHandler] trackPendingRequest decrement failed — counter may drift`,
|
||||
retainDiagnosticForLog(e)
|
||||
e
|
||||
);
|
||||
}
|
||||
};
|
||||
@@ -323,7 +317,7 @@ export function createStreamController({
|
||||
disconnected = true;
|
||||
cleanupClientAbortListener();
|
||||
|
||||
logStream(`disconnect: ${String(retainDiagnosticForLog(reason))}`);
|
||||
logStream(`disconnect: ${reason}`);
|
||||
|
||||
// Decrement pending request counter — the TransformStream flush() won't
|
||||
// fire when the client aborts mid-stream, so we must clean up here.
|
||||
@@ -396,7 +390,7 @@ export function createStreamController({
|
||||
duration: Date.now() - startTime,
|
||||
}) === true;
|
||||
} catch (e) {
|
||||
console.debug(`[STREAM-HANDLER] onError callback error:`, retainDiagnosticForLog(e));
|
||||
console.debug(`[STREAM-HANDLER] onError callback error:`, e);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -412,7 +406,7 @@ export function createStreamController({
|
||||
}
|
||||
|
||||
if (error instanceof Error) {
|
||||
logStream(`error: ${String(retainDiagnosticForLog(error.message))}`);
|
||||
logStream(`error: ${error.message}`);
|
||||
return;
|
||||
}
|
||||
logStream("error: unknown");
|
||||
@@ -851,11 +845,9 @@ export function pipeWithDisconnect(
|
||||
providerResponse: Response,
|
||||
transformStream: TransformStream<Uint8Array, Uint8Array>,
|
||||
streamController: StreamController,
|
||||
opts: { redactStreamDiagnosticsForLog?: boolean; stallTimeoutMs?: number } = {}
|
||||
opts: { stallTimeoutMs?: number } = {}
|
||||
) {
|
||||
const stallTimeoutMs = opts.stallTimeoutMs ?? DEFAULT_STREAM_STALL_TIMEOUT_MS;
|
||||
const retainDiagnosticForLog = (value: unknown): unknown =>
|
||||
opts.redactStreamDiagnosticsForLog ? VIDEO_TRANSCRIPT_LOG_OMISSION_MARKER : value;
|
||||
|
||||
// Watchdog disabled — preserve legacy behavior verbatim.
|
||||
if (!stallTimeoutMs || stallTimeoutMs <= 0) {
|
||||
@@ -895,10 +887,7 @@ export function pipeWithDisconnect(
|
||||
try {
|
||||
streamController.handleError?.(stallError);
|
||||
} catch (e) {
|
||||
console.debug(
|
||||
`[STREAM-HANDLER] stall watchdog handleError failed:`,
|
||||
retainDiagnosticForLog(e)
|
||||
);
|
||||
console.debug(`[STREAM-HANDLER] stall watchdog handleError failed:`, e);
|
||||
}
|
||||
// Error the pipeline so the downstream reader unblocks. createDisconnect-
|
||||
// AwareStream's catch block translates this into buildStreamErrorChunks
|
||||
@@ -906,16 +895,13 @@ export function pipeWithDisconnect(
|
||||
try {
|
||||
upstreamTapController?.error(stallError);
|
||||
} catch (e) {
|
||||
console.debug(
|
||||
`[STREAM-HANDLER] stall watchdog upstream tap error failed:`,
|
||||
retainDiagnosticForLog(e)
|
||||
);
|
||||
console.debug(`[STREAM-HANDLER] stall watchdog upstream tap error failed:`, e);
|
||||
}
|
||||
// Abort the underlying fetch so upstream releases the connection.
|
||||
try {
|
||||
streamController.abort?.();
|
||||
} catch (e) {
|
||||
console.debug(`[STREAM-HANDLER] stall watchdog abort failed:`, retainDiagnosticForLog(e));
|
||||
console.debug(`[STREAM-HANDLER] stall watchdog abort failed:`, e);
|
||||
}
|
||||
}, stallTimeoutMs);
|
||||
};
|
||||
|
||||
@@ -474,8 +474,6 @@ export async function ensureStreamReadiness(
|
||||
provider?: string | null;
|
||||
model?: string | null;
|
||||
log?: StreamReadinessLogger | null;
|
||||
/** Keep the diagnostic transient while retaining only its existence in logs. */
|
||||
redactUpstreamDiagnosticForLog?: boolean;
|
||||
}
|
||||
): Promise<StreamReadinessResult> {
|
||||
if (!response.body || options.timeoutMs <= 0) return { ok: true, response };
|
||||
@@ -570,13 +568,9 @@ export async function ensureStreamReadiness(
|
||||
const reason = upstreamDiagnostic
|
||||
? `${classificationReason}: ${upstreamDiagnostic}`
|
||||
: classificationReason;
|
||||
const retainedReason =
|
||||
upstreamDiagnostic && options.redactUpstreamDiagnosticForLog
|
||||
? `${classificationReason}: [upstream diagnostic omitted]`
|
||||
: reason;
|
||||
options.log?.warn?.(
|
||||
"STREAM",
|
||||
`${retainedReason} (${options.provider || "provider"}/${options.model || "unknown"})`
|
||||
`${reason} (${options.provider || "provider"}/${options.model || "unknown"})`
|
||||
);
|
||||
return {
|
||||
ok: false,
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
{
|
||||
"name": "omniroute",
|
||||
"version": "3.8.51",
|
||||
"description": "Unified AI router with 357 providers, RTK+Caveman compression, auto fallback, MCP/A2A, desktop, PWA, and OpenAI-compatible APIs.",
|
||||
"description": "Unified AI router with 356 providers, RTK+Caveman compression, auto fallback, MCP/A2A, desktop, PWA, and OpenAI-compatible APIs.",
|
||||
"type": "module",
|
||||
"bin": {
|
||||
"omniroute": "bin/omniroute.mjs",
|
||||
|
||||
@@ -47,7 +47,7 @@ curl https://localhost:20128/api/v1/search \
|
||||
|
||||
Run a unified search
|
||||
|
||||
Searches the web, news, or X through a configured provider. Set `provider` to `xquik-search` to use Xquik for X search. The aliases `xquik` and `xquik_search` resolve to the same provider. AnySearch (`anysearch-search`, aliases `anysearch` / `anysearch_search`) provides free fallback-only web search.
|
||||
Searches the web, news, or X through a configured provider. Set `provider` to `xquik-search` to use Xquik for X search. The aliases `xquik` and `xquik_search` resolve to the same provider.
|
||||
|
||||
```bash
|
||||
curl -X POST https://localhost:20128/api/v1/search \
|
||||
|
||||
@@ -2177,9 +2177,6 @@ function ComboFormModal({ isOpen, combo, onClose, onSave, activeProviders, combo
|
||||
builderConnectionId !== COMBO_BUILDER_AUTO_CONNECTION ? builderConnectionId : null,
|
||||
connectionLabel: selectedBuilderConnection?.label || null,
|
||||
allowedConnectionIds: builderEffectiveAllowedConnectionIds,
|
||||
// #11433: use the already-corrected routing prefix (e.g. "oc" for
|
||||
// OpenCode Free) instead of letting it default to the raw providerId.
|
||||
modelPrefix: parseQualifiedModel(selectedBuilderModel.qualifiedModel)?.providerId,
|
||||
})
|
||||
: null;
|
||||
const builderHasDuplicate =
|
||||
@@ -2504,9 +2501,6 @@ function ComboFormModal({ isOpen, combo, onClose, onSave, activeProviders, combo
|
||||
builderConnectionId !== COMBO_BUILDER_AUTO_CONNECTION ? builderConnectionId : null,
|
||||
connectionLabel: selectedBuilderConnection?.label || null,
|
||||
allowedConnectionIds: builderEffectiveAllowedConnectionIds,
|
||||
// #11433: use the already-corrected routing prefix (e.g. "oc" for
|
||||
// OpenCode Free) instead of letting it default to the raw providerId.
|
||||
modelPrefix: parseQualifiedModel(selectedBuilderModel.qualifiedModel)?.providerId,
|
||||
});
|
||||
|
||||
if (hasExactModelStepDuplicate(models, nextStep)) {
|
||||
|
||||