pjandro/OmniRoute
1
0
Fork 0
mirror of https://github.com/diegosouzapw/OmniRoute.git synced 2026-08-23 15:42:12 +03:00
Code Issues Packages Projects Releases Wiki Activity

fix(sse): guard reasoning-cache write by the same predicate its readers use (#10978)

Browse Source 
⭐5 — Cache de reasoning-replay escrevia em toda resposta com reasoning_content, mesmo quando nenhum read-path jamais consumiria (install sem provider de replay). Guard com requiresReasoningReplay() nos dois write-sites, superset seguro do que os readers checam. Testes cobrindo o predicate isoladamente e o wiring real via handleChatCore.
This commit is contained in:
Dizzle
2026-08-21 18:58:56 +02:00
committed by GitHub
parent bc0a272bfc
commit 1f4bde1817
4 changed files with 246 additions and 10 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
docs/routing/REASONING_REPLAY.md
View File

@@ -26,7 +26,8 @@ But typical clients (Cursor, Cline, Roo Code, OpenAI SDK) strip `reasoning_conte
```
Turn N (assistant generates):
→ response contains reasoning_content + tool_calls
→ cacheReasoningFromAssistantMessage() writes (memory + DB), keyed by every tool_call.id
→ if requiresReasoningReplay(provider, model): cacheReasoningFromAssistantMessage()
writes (memory + DB), keyed by every tool_call.id
→ forward response to client (which may or may not retain reasoning)
Turn N+1 (client sends follow-up):
@@ -157,6 +158,7 @@ The cache exposes two endpoints under `src/app/api/cache/reasoning/route.ts`. Bo
- **Cleanup:** `cleanupReasoningCache()` purges expired memory entries and runs `DELETE FROM reasoning_cache WHERE expires_at <= unixepoch('now')`. Health-check workers call this periodically.
- **Crash recovery:** After a restart, memory is empty but the DB still holds unexpired entries. The first lookup for a given `tool_call_id` is a DB hit; subsequent lookups are memory hits.
- **No reasoning, no cache:** `cacheReasoningFromAssistantMessage` returns `0` when the assistant message has no `reasoning_content` / `reasoning` field, so non-thinking responses cost nothing.
- **Write is gated too:** both call sites in `chatCore.ts` (non-streaming and streaming) only call `cacheReasoningFromAssistantMessage()` when `requiresReasoningReplay(provider, model)` is `true` — the same predicate the read side checks. Installs that never touch a replay provider stop paying for the write, the index update, and the try/catch on every reasoning-bearing response.
- **Non-strict providers:** When `requiresReasoningReplay` is `false` and the target format is OpenAI, the translator **strips** any `reasoning_content` field from outgoing messages — OpenAI Chat Completions does not accept it.
## See Also