Commit Graph

523 Commits

Author SHA1 Message Date
oyi77
fec585e44b feat(cache): persistent metrics, cache entry browser, settings UI, MCP tools, prefix analyzer
Implements remaining features from #813:

Phase 1 - Persistent Metrics:
- Add cache_metrics table for persistent hit/miss tracking
- Semantic cache stats now survive server restarts

Phase 2 - Cache Entry Browser:
- /api/cache/entries endpoint with search, pagination, delete
- CacheEntriesTab component for browsing cached entries

Phase 3 - Settings UI:
- CacheSettingsTab for semantic/prompt cache configuration
- /api/settings/cache-config endpoint

Phase 4 - Prefix Analyzer:
- src/lib/promptCache/prefixAnalyzer.ts for intelligent caching
- Analyzes message arrays to find stable prefixes

Phase 5 - Provider Support:
- Added deepseek to CACHING_PROVIDERS

Phase 6 - MCP Tools:
- omniroute_cache_stats tool
- omniroute_cache_flush tool

Phase 7 - Retention:
- cleanOldMetrics() for auto-purge of old entries

Closes #813
2026-03-31 02:41:30 +07:00
oyi77
ae1a0f411b feat(cache): fix cache page to display prompt cache metrics and trend data
Closes #813
2026-03-31 00:53:18 +07:00
Diego Rodrigues de Sa e Souza
6ad0910790 Merge pull request #810 from oyi77/main
feat(settings): add debug toggle and sidebar visibility toggle
2026-03-30 07:07:54 -03:00
oyi77
35f96d4a40 feat(settings): add debug toggle and sidebar visibility toggle
feat(ui): replace hide-sidebar toggle with dynamic visibility toggle
2026-03-30 15:15:02 +07:00
R.D.
179c03e79d Isolate cloudflared runtime environment 2026-03-29 22:30:07 -04:00
diegosouzapw
d69e7ec850 chore(release): v3.3.3 — Core UI bugfixes and AutoUpdate repairs 2026-03-29 21:18:07 -03:00
diegosouzapw
1c070d16a6 fix: add missing i18n keys for windsurf/copilot and apply fetch timeout to streaming requests (#748, #769)
- Add windsurf and copilot entries to toolDescriptions in all 33 locale files
  to fix MISSING_MESSAGE errors on the CLI Tools page (#748)
- Apply FETCH_TIMEOUT_MS to streaming requests' initial fetch() call to prevent
  300s TCP default timeout causing silent failures on long-running requests (#769)
- Previously only non-streaming requests had timeout protection; streaming requests
  relied solely on stream idle detection which doesn't cover initial connection hangs
2026-03-29 19:43:21 -03:00
Diego Rodrigues de Sa e Souza
7837fcc657 Merge pull request #772 from rdself/coder/cloudflared-tunnel-endpoint
Add Cloudflare Quick Tunnel controls to endpoint page
2026-03-29 19:37:03 -03:00
Diego Rodrigues de Sa e Souza
f9690d40d3 Merge pull request #778 from christopher-s/glm-coding-audit
feat: GLM Coding provider enhancements and fixes
2026-03-29 19:36:52 -03:00
Diego Rodrigues de Sa e Souza
5de6cd77dc Merge pull request #773 from rdself/codex/fix-combo-live-test-no-cache
Bypass semantic cache in combo live tests
2026-03-29 19:36:40 -03:00
Diego Rodrigues de Sa e Souza
aa5ab55b14 Merge pull request #777 from diegosouzapw/release/v3.3.1
chore(release): v3.3.1 — bug fixes and schema adjustments
2026-03-29 19:36:11 -03:00
Chris Staley
b812d6efb8 fix: use relative paths in audit doc and correct Indonesian translations in in.json 2026-03-29 15:17:39 -06:00
Chris Staley
6736806361 i18n: add proper translations for model import dialog keys in all locales
Add allModelsAlreadyImported, noNewModelsToImport, and
skippingExistingModels translations for all 32 non-English locales.
2026-03-29 15:13:07 -06:00
Chris Staley
8e17756bf8 fix: filter registry models from auto-sync to prevent duplicates
The model sync scheduler and sync-models endpoint were blindly
replacing custom models with all fetched models, including ones
already in the built-in registry. Now filters out registry models
before saving to custom models.
2026-03-29 15:13:07 -06:00
Chris Staley
0b133fe55e fix: skip duplicate models during Import from /models
Compare fetched models against existing custom models AND built-in
registry models before posting. Only new models trigger
POST /api/provider-models calls. Shows skip count in import progress
when some models already exist. Adds i18n keys for all locales.
2026-03-29 15:13:07 -06:00
Chris Staley
d01266c642 fix: remove glm-4.7-flashx from GLM Coding provider (429 insufficient balance)
Live-tested all GLM Coding models against the /api/coding/paas/v4
endpoint. glm-4.7-flashx returns 429 "Insufficient balance or no
resource package" and is not listed on the /models endpoint.

All other models (glm-5.1, glm-5, glm-5-turbo, glm-4.7, glm-4.7-flash,
glm-4.6v, glm-4.6, glm-4.5v, glm-4.5, glm-4.5-air) return 200.
2026-03-29 15:13:07 -06:00
Chris Staley
fe3f9c86d5 fix: use GLM Coding API endpoints for model import with region-aware URLs
The "Import from /models" button was using the wrong Z.AI API surface
(Anthropic-compatible /api/anthropic/v1/models with x-api-key auth).
Switched to the correct Coding API endpoints with Authorization: Bearer
auth, matching the pattern used by the quota/usage tracking code.

- International: https://api.z.ai/api/coding/paas/v4/models
- China: https://open.bigmodel.cn/api/coding/paas/v4/models
- Auth: Authorization: Bearer <token> (not x-api-key)
- Region sourced from providerSpecificData.apiRegion

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-29 15:13:07 -06:00
Chris Staley
0f4a7b2405 fix: add GLM-4.7-FlashX model and correct GLM-4.7 tool calling support
- Add missing glm-4.7-flashx variant to provider registry (confirmed in
  Z.AI official GLM-4.7 overview docs as one of three variants)
- Remove glm-4.7/glm4.7 from tool calling denylist — official docs
  explicitly show GLM-4.7 supporting function calling with tools param
- Add estimated pricing for glm-4.7-flashx ($0.3/$1.1) between free
  Flash and standard 4.7 tiers

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-29 15:13:07 -06:00
diegosouzapw
6e9c97fbff chore(release): v3.3.1 — bug fixes and schema adjustments 2026-03-29 16:39:10 -03:00
Paijo
f0912feefb feat: auto-disable permanently banned provider accounts (with Settings toggle) (#765)
* feat: auto-disable banned accounts setting with UI toggle

Add a configurable setting to automatically disable provider accounts
that return permanent/terminal errors (403 banned, ToS violation, etc.)

Changes:
- open-sse/services/accountFallback.ts: extend ACCOUNT_DEACTIVATED_SIGNALS
  with AG-specific ban messages ('verify your account', 'service disabled
  for violation')
- src/app/api/settings/auto-disable-accounts/route.ts: new GET/PUT endpoint
  for the setting (enabled bool + threshold int)
- src/shared/validation/schemas.ts: updateAutoDisableAccountsSchema
- src/sse/services/auth.ts: in markAccountUnavailable(), capture result.permanent
  from checkFallbackError() and — when autoDisableBannedAccounts is enabled and
  backoffLevel >= threshold — set isActive=false on the connection

Default: disabled (backward-compatible). Enable via Settings UI or PUT
/api/settings/auto-disable-accounts { "enabled": true, "threshold": 3 }

Fixes: antigravity accounts with 403/Verify-your-account errors being
retried indefinitely in the rotation pool.

Co-authored-by: oyi77 <oyi77@users.noreply.github.com>

* fix: address reviewer comments for auto-disable (use getCachedSettings, immediate disable on permanent bans)

---------

Co-authored-by: oyi77 <oyi77@github.com>
Co-authored-by: oyi77 <oyi77@users.noreply.github.com>
2026-03-29 16:12:17 -03:00
R.D.
cf36972969 Harden cloudflared child process environment 2026-03-29 14:25:07 -04:00
R.D.
40862f26e6 Bypass semantic cache in combo live tests 2026-03-29 14:21:39 -04:00
R.D.
3cb34ad827 Add Cloudflare Quick Tunnel controls to endpoint page 2026-03-29 14:04:50 -04:00
Randi
8af9bd1ac3 Force real upstream combo live tests (#759) 2026-03-29 13:21:53 -03:00
Gorchakov-Pressure
93bbe8e7a8 feat(combo): response quality validation, circuit breaker fix, Cursor 4.6 models (#762)
- Add `validateResponseQuality()` to detect empty/invalid 200 responses from
  upstream providers in combo routing. Non-streaming responses with empty body,
  invalid JSON, or missing content/tool_calls now trigger circuit breaker
  failure and fallback to the next model instead of being returned to the client.

- Add missing `breaker._onSuccess()` calls in both priority and round-robin
  combo paths. Previously failures accumulated without reset, causing premature
  circuit breaker trips on healthy models.

- Update Cursor provider registry with Claude 4.6 model IDs (opus-high,
  sonnet-high, haiku, opus + thinking variants). Keep 4.5 IDs for backward
  compatibility.

- Update free-stack preset: replace duplicate qw/qwen3-coder-plus with
  if/deepseek-v3.2 for better model diversity.

- Add paid-premium combo template for round-robin load distribution across
  paid subscription providers (Cursor, Antigravity).

Made-with: Cursor
2026-03-29 13:21:48 -03:00
Diego Rodrigues de Sa e Souza
46acd16999 chore(release): v3.2.8 — Docker Auto-Update & Analytics Fixes (#755)
* chore(release): v3.2.8 — Docker auto-update UI and cache analytics fixes

* fix(sse): remove race condition in cache metrics tracking (#758)

- Remove in-memory metrics tracking (currentMetrics, trackCacheMetrics, updateCacheMetrics)
- Cache metrics now computed on-the-fly from usage_history table (single source of truth)
- Fixes CRITICAL issue from code review: concurrent requests overwriting metrics
- Fixes WARNING: duplicate metric tracking logic in streaming/non-streaming paths

Ref: PR #752 (merged before this fix was included)

* fix: handle allRateLimited credentials & forward extra body keys in embeddings/images routes (#757)

* fix: handle allRateLimited credentials in embeddings and images routes

When getProviderCredentials() returns an allRateLimited object (truthy,
but without apiKey/accessToken), the embeddings and images routes
incorrectly passed it to handlers as valid credentials. The handlers
then sent upstream requests without Authorization headers, causing
401 errors from providers (e.g. NVIDIA NIM).

This only manifested under concurrent requests: a chat/completions
call could trigger rate limiting on a provider account, and a
simultaneous embeddings request would receive the allRateLimited
sentinel — but treat it as valid credentials.

The chat pipeline already handled this case correctly. This commit
adds the same allRateLimited guard to all affected routes:
- POST /v1/embeddings
- POST /v1/providers/{provider}/embeddings
- POST /v1/images/generations
- POST /v1/providers/{provider}/images/generations

Also adds a defense-in-depth guard in the embeddings handler itself:
if no auth token is available for a non-local provider, return 401
immediately instead of sending an unauthenticated request upstream.

Made-with: Cursor

* fix(embeddings): forward extra body keys to upstream providers

The embeddings handler only forwarded model, input, dimensions, and
encoding_format to upstream providers, silently dropping any additional
fields. This broke asymmetric embedding APIs (e.g. NVIDIA NIM
nv-embedqa-e5-v5) that require input_type, and other providers
expecting user or truncate parameters.

Add a KNOWN_FIELDS exclusion set and forward all unrecognized body
keys to the upstream request, matching the passthrough pattern used
by the chat pipeline's DefaultExecutor.transformRequest().

Made-with: Cursor

* fix(auth): redirect and unconditional 401 on disabled requireLogin + fix test cases

* fix(build): remove legacy proxy.ts causing Next.js build collision

* fix(build): revert middleware.ts rename to proxy.ts because of Next.js Edge constraints

---------

Co-authored-by: diegosouzapw <diegosouzapw@users.noreply.github.com>
Co-authored-by: tombii <tombii@users.noreply.github.com>
Co-authored-by: Gorchakov-Pressure <117600961+Gorchakov-Pressure@users.noreply.github.com>
2026-03-29 13:09:38 -03:00
diegosouzapw
5ad2c6abf6 Fix merge conflicts 2026-03-29 11:26:17 -03:00
Diego Rodrigues de Sa e Souza
d5781d60bd Merge pull request #752 from tombii/feat/preserve-client-cache-control
feat: preserve client cache_control with deterministic routing + metrics dashboard
2026-03-29 11:23:22 -03:00
tombii
319018f055 test: fix cache metrics tests with usage_history table
- Add usage_history table creation in test setup
- Simplify byStrategy query to avoid non-existent combo_strategy column
- Update test assertions to work with existing test data
2026-03-29 16:05:32 +02:00
LASTHXH
394b986ccb Fix CLI tools status endpoint crash and add droid detection support
1. Fixed crash in /api/cli-tools/status when statuses[toolId] is undefined
   - Added null check before accessing statuses[toolId] properties
   - Prevents "Cannot set property of undefined" error

2. Added support for droid.exe detection in ~/bin directory
   - Added ~/bin and ~/.local/bin to EXPECTED_PARENT_PATHS
   - Added droid.exe variant to toolBins for Windows
   - Added specific path check for droid in ~/bin/droid.exe

These fixes resolve issues where CLI tools (Claude Code, Codex, Droid)
were showing as "not installed" even when properly installed.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-29 17:44:37 +05:00
tombii
26f7b36ce4 feat: add cache control settings and token-based metrics
Settings:
- Add `alwaysPreserveClientCache` setting with modes: auto/always/never
- UI toggle in Dashboard > Settings > Routing tab
- Auto mode preserves cache_control for Claude Code clients with deterministic routing

Metrics:
- Track prompt cache token usage (input, cached, creation)
- Display cache reuse ratio (cached/input tokens)
- Breakdown by provider and routing strategy
- Shows tokens saved and estimated cost savings

API Endpoints:
- GET /api/settings/cache-metrics - retrieve metrics
- DELETE /api/settings/cache-metrics - reset metrics

Files:
- open-sse/utils/cacheControlPolicy.ts: CacheControlMetrics interface, trackCacheMetrics, updateCacheTokenMetrics
- open-sse/handlers/chatCore.ts: Track cache tokens from provider responses
- src/lib/db/settings.ts: Database functions for metrics persistence
- src/lib/cacheControlSettings.ts: Cached settings accessor
- src/app/(dashboard)/dashboard/settings/components/CacheStatsCard.tsx: Metrics dashboard UI
- tests/unit/*.test.mjs: Unit tests (41 tests pass)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-29 14:37:55 +02:00
cai kerui
f0daad10ce Add Docker-aware dashboard auto-update flow 2026-03-29 20:25:14 +09:00
tombii
0bc557fb8b feat(sse): preserve client cache_control for Claude Code with deterministic routing
Adds intelligent cache control preservation for Claude Code clients:

- New cacheControlPolicy.ts module with detection logic:
  - isClaudeCodeClient(): Detects Claude Code via User-Agent
  - providerSupportsCaching(): Checks provider (claude, anthropic, zai, qwen)
  - isDeterministicStrategy(): Identifies priority/cost-optimized strategies
  - shouldPreserveCacheControl(): Main policy decision

- Cache control is preserved when:
  1. Client is Claude Code (detected via User-Agent)
  2. Provider supports prompt caching
  3. Request routing is deterministic:
     - Single model requests (always)
     - Combo with priority or cost-optimized strategy only

- Updated translator to accept preserveCacheControl option
- Updated chatCore and chat handler to propagate combo strategy
- Added comprehensive unit tests (24 tests)

Non-deterministic combo strategies (weighted, round-robin, random, etc.)
continue to use OmniRoute's managed caching strategy.

Refs: #cache-control-preservation
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-29 12:24:44 +02:00
diegosouzapw
b84e79362e docs: sync v3.2.6 features across 30 languages (README + FEATURES) 2026-03-29 05:32:01 -03:00
Randi
d352b6b509 Scope API key reveal to Api Manager (#740) 2026-03-29 04:30:11 -03:00
Randi
fcc48cc738 Add full sidebar visibility controls (#739) 2026-03-29 04:30:08 -03:00
Randi
ec06a345cc Require live responses for combo tests (#735) 2026-03-29 04:30:06 -03:00
Randi
7690b364e7 fix detailed log summaries for streamed responses (#734) 2026-03-29 04:30:03 -03:00
diegosouzapw
d23b19c466 fix(core): prevent emergency fallback from masking original errors 2026-03-29 01:46:05 -03:00
diegosouzapw
11a8adc71c Merge branch 'feat/issue-659-mobile-ui' 2026-03-28 23:12:26 -03:00
diegosouzapw
b9c7fd879f fix(core): resolve routing schemas, CLI streaming leaks, and thinking tag extraction 2026-03-28 23:11:22 -03:00
Diego Rodrigues de Sa e Souza
2fc4c7ea33 Merge pull request #728 from rdself/codex/normalize-provider-limits-labels
normalize provider limits labels
2026-03-28 23:06:16 -03:00
R.D.
538028c150 normalize provider limits quota labels 2026-03-28 21:17:07 -04:00
R.D.
04de492019 fix: add four-stage request log payloads 2026-03-28 22:09:27 -03:00
diegosouzapw
94a00cb6d6 feat: improve dashboard layout for smaller screens (#659) 2026-03-28 21:53:07 -03:00
diegosouzapw
f1cddba938 feat: add global fallback provider support (#689)
When all combo models are exhausted (502/503), OmniRoute now checks for
a globalFallbackModel setting and attempts one last request through it
before returning the error. Settings stored in key_value table, no
migration needed.
2026-03-28 21:10:29 -03:00
Chris
877cfa0071 feat: add GLM Coding usage/quota tracking with Z.AI session quota (#698)
* feat: add GLM Coding usage/quota tracking with Z.AI session quota

Add GLM to the usage tracking pipeline: usage API route, Z.AI quota
fetcher (TOKENS_LIMIT percentage-based), quota parser, and Provider
Limits UI. Adds API region dropdown (International/China) to Add/Edit
connection modals. Displays session quota with plan level.

* fix: address PR review feedback for GLM usage tracking

- Remove explicit `any` types from getGlmUsage (fix lint budget)
- Fix empty string fallback for plan level
- Remove duplicate `case "glm"` in quota parser (identical to default)
- Skip OAuth refresh flow for GLM (API key auth) in usage route

* fix: upgrade path-to-regexp to fix ReDoS vulnerability (GHSA-j3q9-mxjg-w52f, GHSA-27v5-c462-wpq7)

---------

Co-authored-by: Chris Staley <christopher-s@users.noreply.github.com>
2026-03-28 20:39:24 -03:00
Paijo
e6f0a780b7 feat(dashboard): add Cache Management page with stats, hit rate, and targeted invalidation (#701)
Adds a new /dashboard/cache page that surfaces the existing but UI-less
semantic cache infrastructure.

Changes:
- New page: src/app/(dashboard)/dashboard/cache/page.tsx
  - Live stats: memory entries, DB entries, cache hits, tokens saved
  - Hit rate progress bar with color coding (green/yellow/red)
  - Hits/Misses/Total breakdown
  - Idempotency layer stats (active dedup keys + window)
  - Cache behavior info panel
  - Clear All button
  - Auto-refresh every 10s
- Enhanced API: src/app/api/cache/route.ts
  - DELETE ?model=<name> — invalidate by model
  - DELETE ?signature=<hex> — invalidate single entry
  - DELETE ?staleMs=<ms> — invalidate entries older than N ms
  - DELETE (no params) — clear all (existing behavior)
- Sidebar: added Cache nav item (icon: cached)
- i18n: added cache + sidebar.cache keys for all 31 supported locales

No new dependencies. All functionality builds on existing semanticCache.ts,
cacheLayer.ts, and idempotencyLayer.ts modules.

Co-authored-by: oyi77 <oyi77@users.noreply.github.com>
2026-03-28 20:39:20 -03:00
Randi
dd9de2efa9 fix: harden combo fallback and health checks (#704) 2026-03-28 20:39:16 -03:00
Randi
f6b0811f78 [codex] fix provider limits ui (#718)
* fix provider limits ui

* restore remaining quota progress bars

* address provider limits review feedback
2026-03-28 20:39:06 -03:00