2 Commits

Author	SHA1	Message	Date
tombii	26f7b36ce4	feat: add cache control settings and token-based metrics ... Settings: - Add `alwaysPreserveClientCache` setting with modes: auto/always/never - UI toggle in Dashboard > Settings > Routing tab - Auto mode preserves cache_control for Claude Code clients with deterministic routing Metrics: - Track prompt cache token usage (input, cached, creation) - Display cache reuse ratio (cached/input tokens) - Breakdown by provider and routing strategy - Shows tokens saved and estimated cost savings API Endpoints: - GET /api/settings/cache-metrics - retrieve metrics - DELETE /api/settings/cache-metrics - reset metrics Files: - open-sse/utils/cacheControlPolicy.ts: CacheControlMetrics interface, trackCacheMetrics, updateCacheTokenMetrics - open-sse/handlers/chatCore.ts: Track cache tokens from provider responses - src/lib/db/settings.ts: Database functions for metrics persistence - src/lib/cacheControlSettings.ts: Cached settings accessor - src/app/(dashboard)/dashboard/settings/components/CacheStatsCard.tsx: Metrics dashboard UI - tests/unit/*.test.mjs: Unit tests (41 tests pass) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>	2026-03-29 14:37:55 +02:00
tombii	0bc557fb8b	feat(sse): preserve client cache_control for Claude Code with deterministic routing ... Adds intelligent cache control preservation for Claude Code clients: - New cacheControlPolicy.ts module with detection logic: - isClaudeCodeClient(): Detects Claude Code via User-Agent - providerSupportsCaching(): Checks provider (claude, anthropic, zai, qwen) - isDeterministicStrategy(): Identifies priority/cost-optimized strategies - shouldPreserveCacheControl(): Main policy decision - Cache control is preserved when: 1. Client is Claude Code (detected via User-Agent) 2. Provider supports prompt caching 3. Request routing is deterministic: - Single model requests (always) - Combo with priority or cost-optimized strategy only - Updated translator to accept preserveCacheControl option - Updated chatCore and chat handler to propagate combo strategy - Added comprehensive unit tests (24 tests) Non-deterministic combo strategies (weighted, round-robin, random, etc.) continue to use OmniRoute's managed caching strategy. Refs: #cache-control-preservation Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>	2026-03-29 12:24:44 +02:00