diegosouzapw b08fb31a28 feat(gateway): Phase 9 — LLM Gateway Intelligence ...

9.1 — Semantic Cache
  - New: src/lib/semanticCache.js — Two-tier cache (in-memory LRU + SQLite)
  - Signature = SHA-256(model + normalized messages + temperature + top_p)
  - Only caches non-streaming, temperature=0 requests
  - X-OmniRoute-No-Cache: true header bypass
  - Response headers: X-OmniRoute-Cache: HIT/MISS
  - DB table: semantic_cache with indexes on signature and model
  - New: src/app/api/cache/route.js — GET stats, DELETE clear

9.2 — Request Idempotency
  - New: src/lib/idempotencyLayer.js — In-memory 5s dedup window
  - Reads Idempotency-Key or X-Request-Id headers
  - Returns cached response with X-OmniRoute-Idempotent: true header
  - Ephemeral by design (no SQLite)

9.3 — Progress Tracking in Streaming
  - New: open-sse/utils/progressTracker.js
  - Emits SSE 'event: progress' with tokens_generated + elapsed_ms
  - Opt-in via X-OmniRoute-Progress: true header
  - Supports AbortSignal cancellation
  - Final event includes done: true

Integration:
  - chatCore.js: idempotency check → cache check → provider call → cache store → idempotency save
  - Streaming path: optional progress transform chain
  - DB: semantic_cache table added to db/core.js schema

Tests: 320 pass (+25 new) | Build: success

2026-02-15 11:04:51 -03:00

2.8 KiB

Raw Blame History

View Raw