diegosouzapw
|
b08fb31a28
|
feat(gateway): Phase 9 — LLM Gateway Intelligence
9.1 — Semantic Cache
- New: src/lib/semanticCache.js — Two-tier cache (in-memory LRU + SQLite)
- Signature = SHA-256(model + normalized messages + temperature + top_p)
- Only caches non-streaming, temperature=0 requests
- X-OmniRoute-No-Cache: true header bypass
- Response headers: X-OmniRoute-Cache: HIT/MISS
- DB table: semantic_cache with indexes on signature and model
- New: src/app/api/cache/route.js — GET stats, DELETE clear
9.2 — Request Idempotency
- New: src/lib/idempotencyLayer.js — In-memory 5s dedup window
- Reads Idempotency-Key or X-Request-Id headers
- Returns cached response with X-OmniRoute-Idempotent: true header
- Ephemeral by design (no SQLite)
9.3 — Progress Tracking in Streaming
- New: open-sse/utils/progressTracker.js
- Emits SSE 'event: progress' with tokens_generated + elapsed_ms
- Opt-in via X-OmniRoute-Progress: true header
- Supports AbortSignal cancellation
- Final event includes done: true
Integration:
- chatCore.js: idempotency check → cache check → provider call → cache store → idempotency save
- Streaming path: optional progress transform chain
- DB: semantic_cache table added to db/core.js schema
Tests: 320 pass (+25 new) | Build: success
|
2026-02-15 11:04:51 -03:00 |
|