* feat(cache): implement dual-layer semantic caching with in-memory and redis vector stores
Implements production-grade, configurable semantic caching for OmniRoute.
- Dual-layer architecture: Layer 1 exact hash match (0 embedding latency) + Layer 2 vector cosine similarity search.
- Backends: in-memory vector store with L2 normalization, LRU and TTL + Redis vector store adapter with fail-open fallback.
- Embedding generation: conversation history normalization, system prompt exclusion, timeout protection.
- Streaming support: serializable SSE stream synthesis ending in data: [DONE]\n\n.
- Request overrides and telemetry headers: X-OmniRoute-Cache (HIT (exact) | HIT (semantic) | MISS), X-OmniRoute-Cache-Similarity, X-OmniRoute-Savings-Tokens, Cache-Control: no-cache, x-omniroute-no-cache, x-omniroute-cache-threshold, x-omniroute-cache-type, x-omniroute-cache-no-store, x-omniroute-cache-key.
- Unit test coverage across dual-layer search, eviction, redis resilience, and streaming replay.
* feat(cache): add default embedding client and live integration test for Lemonade server and Redis
- Add embeddingBaseUrl and embeddingApiKey configuration options to SemanticCacheConfig.
- Implement createDefaultEmbeddingGenerator for automatic OpenAI-compatible embedding integration.
- Add live verification script scripts/ad-hoc/test-semantic-cache-lemonade.ts.
- Add network-aware integration test tests/integration/semantic-cache-lemonade.test.ts for Lemonade harrier-oss-v1-0.6b and Redis vector store.
* fix(cache): address CodeRabbit review recommendations on PR #1
- Multi-tenant partition isolation: support null sentinel in StoreFilter so anonymous requests cannot match authenticated entries.
- Provider propagation: pass routed provider to non-streaming and streaming cache writes.
- Embedding resilience: race generator with timeout promise in generateEmbeddingWithTimeout to guard against uncooperative generators.
- Memory store consistency: replace older entries with identical hash on insert, and safe-guard hashToId deletion in removeEntry.
- Redis store consistency: prune expired/missing entries from candidate sets during search/stats, and delete hash mapping conditionally.
- Token telemetry: use managerResult.tokensSaved when cached response lacks usage data.
- Clova batch timeout: add AbortSignal.timeout(FETCH_TIMEOUT_MS) to fetchClovaEmbeddingBatch.
---------
Co-authored-by: John Smith <you@example.com>
d8c69b87e3