mirror of
https://github.com/diegosouzapw/OmniRoute.git
synced 2026-07-31 04:12:10 +03:00
aa85fa02bb
Adds GET /api/usage/cache-health and a Cache Health tab under /dashboard/analytics, both backed by a pure summary over the cache columns already present in call_logs. Motivated by a production diagnosis where the aggregate ratio was actively misleading. The window read 24.8M cached tokens and wrote 9.5M — a write/read of 0.385, which reads as merely mediocre. The actual shape was very different: the median call wrote 848 tokens while 18% of the calls carried 94% of every written token, and two models in the same window sat at 0.13 (Sonnet) and 0.51 (Opus). Averaging hid all three facts. So the summary reports what the average cannot: the distribution (p50/p90/p99), the concentration (how few calls carry how much of the write), and the per-model split. The heavy-write threshold is relative to the window (10x the median, floored at 1024) because a cutoff tuned for 130k-token conversations reports nothing at all on 2k-token ones; 1024 is the minimum Anthropic bills for cache creation, below which a write carries no signal. Calls that neither read nor wrote are counted separately from thrash — a route that does not cache is an absence of caching, not a sick cache — and only successful calls are summarized, since a 4xx/5xx never reached the provider cache and would dilute the ratio. Tests cover the summary (8) and the route (6, against a real SQLite so the WHERE clause itself is exercised), including that an internal failure answers 500 without leaking the stack trace, the SQL text or a table name.