tombii
|
26f7b36ce4
|
feat: add cache control settings and token-based metrics
Settings:
- Add `alwaysPreserveClientCache` setting with modes: auto/always/never
- UI toggle in Dashboard > Settings > Routing tab
- Auto mode preserves cache_control for Claude Code clients with deterministic routing
Metrics:
- Track prompt cache token usage (input, cached, creation)
- Display cache reuse ratio (cached/input tokens)
- Breakdown by provider and routing strategy
- Shows tokens saved and estimated cost savings
API Endpoints:
- GET /api/settings/cache-metrics - retrieve metrics
- DELETE /api/settings/cache-metrics - reset metrics
Files:
- open-sse/utils/cacheControlPolicy.ts: CacheControlMetrics interface, trackCacheMetrics, updateCacheTokenMetrics
- open-sse/handlers/chatCore.ts: Track cache tokens from provider responses
- src/lib/db/settings.ts: Database functions for metrics persistence
- src/lib/cacheControlSettings.ts: Cached settings accessor
- src/app/(dashboard)/dashboard/settings/components/CacheStatsCard.tsx: Metrics dashboard UI
- tests/unit/*.test.mjs: Unit tests (41 tests pass)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
|
2026-03-29 14:37:55 +02:00 |
|