mirror of
https://github.com/diegosouzapw/OmniRoute.git
synced 2026-08-12 10:12:11 +03:00
2b2d947faf
* fix(cache): add latency marker + per-key bypass for semantic cache
Semantic cache silently corrupts latency measurements: a 10s upstream
call served from cache looks like 19ms. Three fixes:
A. Latency marker: cache HIT responses now carry
X-OmniRoute-Cache-Latency: synthetic so measurement tools can
distinguish real vs cached latency.
B. Per-key bypass: new apiKeys.cacheDefaultMode ('legacy' | 'bypass')
lets latency-sensitive clients opt out of cache reads entirely.
- DB column + migration (134)
- rowParser parseCacheDefaultMode
- API create default + PATCH update
- checkSemanticCache returns null on bypass
C. Type safety: ApiKeyRow/ApiKeyView/params updated, superRefine
guard includes cacheDefaultMode.
Cache write path intentionally unchanged: apiKeyId is already in the
cache signature (semanticCache.ts:140), so per-key isolation prevents
cross-key pollution.
Changed test files:
- tests/unit/chatcore-semantic-cache.test.ts (3 new tests)
Signed-off-by: Minxi Hou <houminxi@gmail.com>
* docs: document semantic cache latency impact + bypass configuration
---------
Signed-off-by: Minxi Hou <houminxi@gmail.com>