Commit Graph

160 Commits

Author SHA1 Message Date
oyi77
f99c90dc85 feat(cache): add OpenAI prompt_cache_key and Gemini cachedContent support
Provider-specific caching enhancements:

OpenAI:
- Auto-generate prompt_cache_key from message prefix hash
- Key format: omni-{prefix_hash_32chars}
- Preserves client-provided keys (doesn't override)

Gemini:
- Preserve cachedContent ID if provided by client
- Enables explicit Gemini caching for long prompts
- cachedContentTokenCount already tracked in responses
2026-03-31 04:50:50 +07:00
diegosouzapw
76e135077b Resolve merge conflicts with main natively built Prompt Cache UI 2026-03-30 18:20:19 -03:00
Diego Rodrigues de Sa e Souza
6078cd2eab Merge pull request #829 from rdself/coder/fix-cloudflared-startup
Fix cloudflared quick tunnel startup in Docker
2026-03-30 18:18:03 -03:00
Diego Rodrigues de Sa e Souza
3482dade71 Merge pull request #828 from rdself/coder/fix-combo-test-false-negative
Fix combo test false negatives and parallelize model probes
2026-03-30 18:18:00 -03:00
oyi77
6780485051 feat(cache): persistent metrics, cache entry browser, settings UI, MCP tools, prefix analyzer
Implements remaining features from #813:

Phase 1 - Persistent Metrics:
- Add cache_metrics table for persistent hit/miss tracking
- Semantic cache stats now survive server restarts

Phase 2 - Cache Entry Browser:
- /api/cache/entries endpoint with search, pagination, delete
- CacheEntriesTab component for browsing cached entries

Phase 3 - Settings UI:
- CacheSettingsTab for semantic/prompt cache configuration
- /api/settings/cache-config endpoint

Phase 4 - Prefix Analyzer:
- src/lib/promptCache/prefixAnalyzer.ts for intelligent caching
- Analyzes message arrays to find stable prefixes

Phase 5 - Provider Support:
- Added deepseek to CACHING_PROVIDERS

Phase 6 - MCP Tools:
- omniroute_cache_stats tool
- omniroute_cache_flush tool

Phase 7 - Retention:
- cleanOldMetrics() for auto-purge of old entries

Closes #813
2026-03-30 17:32:45 -03:00
oyi77
d043e7a242 feat(cache): fix cache page to display prompt cache metrics and trend data
Closes #813
2026-03-30 17:32:45 -03:00
R.D.
b492c5ac1a Fix cloudflared startup TLS handling 2026-03-30 16:31:07 -04:00
R.D.
03a860dd6f Fix combo smoke tests for reasoning responses 2026-03-30 16:23:53 -04:00
tombii
c6eadc504b fix(usage): include cache tokens in input token counts
- Fix getLoggedInputTokens to return full prompt_tokens (input + cache_read + cache_creation)
- Fix usageExtractor for non-streaming Claude responses to calculate total correctly
- Add formatUsageLog helper to show CR=<cache_read> in logs
- Add migration 012 to fix historical token counts in usage_history
- Move prompt cache metrics from Settings to /dashboard/cache page

Per Claude API docs:
Total input tokens = input_tokens + cache_creation_input_tokens + cache_read_input_tokens

Fixes issue where totalInputTokens (71k) was less than totalCacheCreationTokens (1.35M).

Tested:
- All 1134 unit tests pass
- Cache metrics API returns correct totals
- Migration is idempotent and tracked in _omniroute_migrations
- Logs show cache read tokens: 'in=6055 | out=211 | CR=22399'

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-30 19:24:26 +02:00
Diego Rodrigues de Sa e Souza
db3753d611 Merge PR 811: fix UI fallbacks and Electron release workflow
fix: UI fallbacks and Electron release workflow
2026-03-30 11:04:02 -03:00
diegosouzapw
5ad687c6d8 fix(ui/ci): use ProviderIcon for Provider header breadcrumbs and add permissions to electron-release.yml (#745, #761)
- Use ProviderIcon for internal .png paths solving SVG provider 404 images (#745).
- Add id-token: write and packages: write permissions to .github/workflows/electron-release.yml to fix permissions denied failure when calling the reusable workflow npm-publish.yml (#761).
- Fix tests and ESM resolution for autoUpdate.ts override logic.
2026-03-30 07:38:30 -03:00
R.D.
179c03e79d Isolate cloudflared runtime environment 2026-03-29 22:30:07 -04:00
diegosouzapw
d69e7ec850 chore(release): v3.3.3 — Core UI bugfixes and AutoUpdate repairs 2026-03-29 21:18:07 -03:00
Diego Rodrigues de Sa e Souza
7837fcc657 Merge pull request #772 from rdself/coder/cloudflared-tunnel-endpoint
Add Cloudflare Quick Tunnel controls to endpoint page
2026-03-29 19:37:03 -03:00
R.D.
cf36972969 Harden cloudflared child process environment 2026-03-29 14:25:07 -04:00
R.D.
40862f26e6 Bypass semantic cache in combo live tests 2026-03-29 14:21:39 -04:00
R.D.
3cb34ad827 Add Cloudflare Quick Tunnel controls to endpoint page 2026-03-29 14:04:50 -04:00
diegosouzapw
5ad2c6abf6 Fix merge conflicts 2026-03-29 11:26:17 -03:00
tombii
319018f055 test: fix cache metrics tests with usage_history table
- Add usage_history table creation in test setup
- Simplify byStrategy query to avoid non-existent combo_strategy column
- Update test assertions to work with existing test data
2026-03-29 16:05:32 +02:00
tombii
26f7b36ce4 feat: add cache control settings and token-based metrics
Settings:
- Add `alwaysPreserveClientCache` setting with modes: auto/always/never
- UI toggle in Dashboard > Settings > Routing tab
- Auto mode preserves cache_control for Claude Code clients with deterministic routing

Metrics:
- Track prompt cache token usage (input, cached, creation)
- Display cache reuse ratio (cached/input tokens)
- Breakdown by provider and routing strategy
- Shows tokens saved and estimated cost savings

API Endpoints:
- GET /api/settings/cache-metrics - retrieve metrics
- DELETE /api/settings/cache-metrics - reset metrics

Files:
- open-sse/utils/cacheControlPolicy.ts: CacheControlMetrics interface, trackCacheMetrics, updateCacheTokenMetrics
- open-sse/handlers/chatCore.ts: Track cache tokens from provider responses
- src/lib/db/settings.ts: Database functions for metrics persistence
- src/lib/cacheControlSettings.ts: Cached settings accessor
- src/app/(dashboard)/dashboard/settings/components/CacheStatsCard.tsx: Metrics dashboard UI
- tests/unit/*.test.mjs: Unit tests (41 tests pass)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-29 14:37:55 +02:00
cai kerui
f0daad10ce Add Docker-aware dashboard auto-update flow 2026-03-29 20:25:14 +09:00
Randi
d352b6b509 Scope API key reveal to Api Manager (#740) 2026-03-29 04:30:11 -03:00
Randi
fcc48cc738 Add full sidebar visibility controls (#739) 2026-03-29 04:30:08 -03:00
Randi
ec06a345cc Require live responses for combo tests (#735) 2026-03-29 04:30:06 -03:00
Randi
7690b364e7 fix detailed log summaries for streamed responses (#734) 2026-03-29 04:30:03 -03:00
Diego Rodrigues de Sa e Souza
2fc4c7ea33 Merge pull request #728 from rdself/codex/normalize-provider-limits-labels
normalize provider limits labels
2026-03-28 23:06:16 -03:00
R.D.
538028c150 normalize provider limits quota labels 2026-03-28 21:17:07 -04:00
R.D.
04de492019 fix: add four-stage request log payloads 2026-03-28 22:09:27 -03:00
Randi
dd9de2efa9 fix: harden combo fallback and health checks (#704) 2026-03-28 20:39:16 -03:00
Randi
eba9d854a9 fix model auto-sync startup and auth (#719) 2026-03-28 20:39:02 -03:00
diegosouzapw
65edddd62e refactor(open-sse): remove unused imports from translator/index.ts
remove unused imports coerceToolSchemas and sanitizeToolDescriptions from translator/index.ts to satisfy lint and prevent unused import issues
2026-03-28 17:26:55 -03:00
diegosouzapw
7abea2020c Merge feature-tests: schema coercion, tool sanitization, Codex auth export, enhanced test suite 2026-03-28 16:27:32 -03:00
diegosouzapw
98470e8551 Merge fix/issue-711: provider max_tokens cap + upstream sync tasks (#711) 2026-03-28 16:08:12 -03:00
diegosouzapw
00f59b95ae fix: protocol clients e2e dev mode singleton and auth (#710) 2026-03-28 15:52:29 -03:00
diegosouzapw
8595964ab8 feat/fix: implement upstream sync tasks 1-7 2026-03-28 14:48:57 -03:00
diegosouzapw
922dae8546 feat: add Codex auth.json export and apply-local buttons for CLI integration
- Add codexAuthFile.ts utility: builds Codex auth.json payload from OAuth connection
  (id_token, access_token, refresh_token, account_id) with auto-refresh if expired
- Add POST /api/providers/[id]/codex-auth/export: downloads auth.json file
- Add POST /api/providers/[id]/codex-auth/apply-local: writes auth.json to local CLI path
- Add 'Apply auth' and 'Export auth' buttons to ConnectionRow (Codex provider only)
- Add i18n keys for en and pt-BR
2026-03-28 13:28:06 -03:00
diegosouzapw
55325773dc feat(open-sse): add schema coercion and tool sanitization
- introduce open-sse/translator/helpers/schemaCoercion.ts to coerce
  numeric JSON Schema fields encoded as strings
- wire coerceToolSchemas and sanitizeToolDescriptions into translator
  pipeline; ensure tool descriptions are sanitized
- inject empty reasoning content for tool calls when target is OpenAI
  format
- update qwen base URL to DashScope-compatible endpoint
- extend antigravity static catalog with Gemini 3.1 pro preview models and
  update Gemini model specs with preview aliases
- implement call log max cap caching with TTL; expose invalidateCallLogsMaxCache
  and invalidate on settings PATCH
- add tests: call-log-cap.test.mjs and tool-request-sanitization.test.mjs;
  extend tests for Windsurf integration and gemini previews
- update CLI runtime and tools to include Windsurf as a guide-only tool
- add maxCallLogs to validation schemas (settings and updateSettings)
- add Czech README (README.cs.md) to repository
2026-03-28 12:33:13 -03:00
diegosouzapw
ba4b496306 Merge PR #666: Add Claude prompt cache logging and exclude cache reads
Includes fixes applied during review:
- Removed duplicate imports in chatCore.ts
- Fixed stray translatedBody argument (stream boolean bug)
- Fixed truncated test file
- Fixed usageExtractor cached_tokens fallback

Closes #688, Closes #640
2026-03-28 01:53:25 -03:00
diegosouzapw
f380d44697 fix(core): hidden models flag, antigravity streaming, and i18n translation sync (#681, #684, #685) 2026-03-27 16:17:28 -03:00
cai kerui
ae8d2ac2e1 Merge branch 'main' into codex/claude-cache-log-accounting 2026-03-27 17:25:38 +09:00
cai kerui
93beb068a3 Add Claude prompt cache logging and exclude cache reads 2026-03-27 15:14:54 +09:00
Diego Rodrigues de Sa e Souza
36fbd3d018 Merge pull request #647 from brendandebeasi/fix/expired-token-retry-healthcheck
Thanks @brendandebeasi for this excellent contribution! 🎉 The bounded retry with exponential backoff is exactly the right approach for expired connections. Merged and will be included in v3.1.1.
2026-03-26 15:49:52 -03:00
Brendan DeBeasi
2392006246 fix: retry expired connections in token health check instead of permanently skipping
Connections marked as 'expired' were permanently skipped by the health check scheduler (line 176: if testStatus === "expired" return). A single transient refresh failure could permanently disable auto-refresh, requiring manual re-authentication.

Replace the hard skip with a bounded retry mechanism: up to 3 attempts with exponential backoff (5min, 10min, 20min). On success, the connection is fully restored to active. On exhaustion, it remains expired (same as before). The existing circuit breaker (5 failures → 30min pause) provides additional protection.

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-opencode)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
2026-03-26 11:11:49 -07:00
diegosouzapw
3976c79e12 fix: convert reasoning_effort to Claude thinking format & proactive token refresh (#627, #631)
- OpenAI-to-Claude translator now maps reasoning_effort (low/medium/high/max)
  to Claude's thinking.budget_tokens. Fixes clients like OpenCode sending
  reasoning_effort via @ai-sdk/openai-compatible losing thinking configuration.
- Ensures max_tokens > budget_tokens for all thinking configs.
- Token health check now proactively refreshes tokens within 5 min of expiry,
  regardless of the configured health check interval — addresses Qwen OAuth
  token refresh failures between scheduled checks.
2026-03-26 08:59:21 -03:00
diegosouzapw
2602cd9ab2 chore(release): v3.0.8 — fix translation failures for OpenAI-format providers (#632) 2026-03-25 23:30:35 -03:00
Ivan
58df1c06ee fix: correct LongCat API base URL path 2026-03-25 18:14:19 +03:00
Diego Rodrigues de Sa e Souza
57b3319ac0 Merge pull request #597 from rdself/feat/auto-sync-models
feat: add per-provider auto-sync for model lists
2026-03-25 10:47:30 -03:00
Diego Rodrigues de Sa e Souza
33fee5dcc5 fix: strip proxy_ prefix in non-streaming Claude responses & fix LongCat validation (#605, #592) (#607)
- fix(translator): pass toolNameMap to translateNonStreamingResponse so Claude
  OAuth proxy_ prefix is correctly stripped from tool_use block names in
  non-streaming responses (was only stripped in streaming path)
- fix(validation): add LongCat specialty validator that probes /chat/completions
  directly, bypassing the /v1/models endpoint that LongCat does not expose (#592)

Co-authored-by: diegosouzapw <diegosouzapw@users.noreply.github.com>
2026-03-25 08:16:46 -03:00
R.D.
6eda0f4d00 feat: add per-provider auto-sync for model lists
- Add POST /api/providers/[id]/sync-models endpoint that fetches models
  from a provider's /models API and replaces the full custom models list,
  preserving per-model compatibility overrides
- Rewrite modelSyncScheduler to dynamically discover connections with
  autoSync enabled in providerSpecificData instead of a hardcoded list
- Add replaceCustomModels() to db/models.ts for full list replacement
  while preserving existing compat flags
- Log each model sync operation to call_logs for visibility in the
  Logs page
- Add Auto-Sync toggle button next to "Import from /models" in the
  provider detail page UI
- Add en/zh-CN i18n translations for auto-sync strings

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-24 21:16:09 -04:00
diegosouzapw
9248ab4dfd fix(ci): route validation, CodeQL alerts, Docker workflow
- Add Zod schemas + validateBody() to 5 routes missing validation:
  model-combo-mappings (POST, PUT), webhooks (POST, PUT), openapi/try (POST)
- Fix 6 polynomial-redos CodeQL alerts in provider.ts and chatCore.ts
  by replacing (?:^|/) alternation patterns with segment-based matching
- Fix insecure-randomness in acp/manager.ts (crypto.randomUUID)
- Fix shell-command-injection in prepublish.mjs (JSON.stringify)
- Upgrade docker/setup-buildx-action from v3 to v4 (Node.js 20 deprecation)

CI check:route-validation:t06 PASS (176/176 routes validated)
Tests: 926/926 pass
2026-03-24 16:08:02 -03:00