mirror of
https://github.com/diegosouzapw/OmniRoute.git
synced 2026-07-31 04:12:10 +03:00
7db3ba615b
* fix(sse): stop thrashing the provider prompt cache for caching-aware clients - shouldPreserveCacheControl: preserve client cache_control markers for every combo strategy. The deterministic-strategy gate forced marker rewrites whose per-request breakpoint positions are not stable turn-over-turn, thrashing the upstream prompt cache (observed in production as ~200k cache_write tokens per turn on quota-share combos). Preserving is never worse: on a stable target the client's breakpoints advance deterministically; on a target switch both approaches miss equally. - prepareClaudeRequest: translator-path opt-in fallback — when preserve-mode has nothing to preserve (client sent no cache_control anywhere), apply the standard heuristic so requests never ship with zero cache breakpoints. The claude-code-compatible relay path keeps its no-supplement contract. - anthropic-beta: forward the client-negotiated context-1m-2025-08-07 through the allowlist merge so a /model <id>[1m] client keeps its long-context negotiation behind the proxy (never forced when the client did not send it). * fix(sse): surface cache tokens in non-streaming OpenAI usage + finalize pending on quota-share block - translateNonStreamingResponse (claude→openai): fold cache_read into prompt_tokens and expose prompt_tokens_details.cached_tokens / cache_creation_tokens, mirroring the streaming contract (#1426/#2215). Non-streaming OpenAI clients behind a cached Claude upstream previously saw prompt_tokens=<uncached remainder> (e.g. 23 for a ~9k request) with no cache visibility. - chatCore quota-share block: finalize the pending-request slot before returning the policy 429 — the path never reaches upstream and the orphaned pending lingered as a status-0 call-log row until the reaper swept it. Validated live on the staging box (repro: blocked key → orphan row; after: clean). --------- Co-authored-by: diegosouzapw <diegosouzapw24@gmail.com>