fix(sse): replay reasoning for Responses-API targets on plain turns and Anthropic clients (#13031)

* fix(sse): replay reasoning for Responses-API targets on plain turns and Anthropic clients

DeepSeek thinking mode requires the reasoning of every prior assistant turn
to be passed back once the request carries `tools`, including turns that
made no tool call. Since #10540 routed opencode-go/deepseek-v4-* to
`/responses`, the reasoning replay cache had two gaps on Responses-API
targets, and clients that drop `reasoning_content` hit intermittent
`400 The reasoning_text in the thinking mode must be passed back`.

1. Plain (non-tool-call) turns are keyed on a digest of the normalized
   OpenAI transcript. Both capture sites used `translatedBody.messages` as
   the history, which a Responses body (`input`) does not carry, so the
   write-time digest never matched the read side. translateRequest now
   reports the pivot transcript it digested via `onReasoningReplayHistory`,
   and the streaming / non-streaming capture sites digest that transcript.
2. The Responses replay pass was gated on `sourceFormat === "openai"`, so
   Anthropic Messages clients (Claude -> OpenAI -> Responses) got no replay
   at all. The pass now runs on the OpenAI pivot for every source format,
   right before the Responses conversion discards `messages`.

The reported transcript is a shallow snapshot of the digested fields only
and travels through a callback, not the body, so nothing new reaches the
upstream payload.

* docs(changelog): add fragment for #13031

* fix(sse): guard the Responses capture sites and skip plain-turn writes with no history

Review follow-ups for #13031:

- Add tests/unit/chatcore-reasoning-cache-write-guard-responses.test.ts:
  runs the real handleChatCore against a mocked opencode-go/deepseek-v4-flash
  Responses upstream (JSON and SSE), then asserts the next turn's upstream
  body carries the replayed `reasoning` input item. Removing either capture
  site fallback turns both cases red.
- Project the reported transcript down to the digested fields only
  (tool_calls keep type/name/arguments, ids are dropped) and document that
  `content` is shared by reference.
- Skip the plain-turn cache write when the history is empty: a real request
  always has a prior user turn, so an empty history means the transcript
  could not be recovered and a one-message digest can never match.
- Changelog wording: the pre-fix write digested only the assistant message.

* test(sse): select the /responses dispatch by URL in the Responses replay guard

Review follow-ups for #13031: the guard picks the upstream body by URL
(`/responses`) and asserts exactly one such dispatch per turn instead of
taking the last fetch, the streaming case asserts the same body shape as the
non-streaming one, and the `historyMessages` doc on
NonStreamingClientTranslateInput names the Responses-shaped fallback.

* docs(routing): name the replay-history hand-off without tripping the hook heuristic

The fabricated-docs gate treats any `onXxx` token in prose as a plugin hook
name and flagged `onReasoningReplayHistory` (a translateRequest option, not a
hook). Point at the option's home file instead.

* chore(quality): freeze chatCore.ts at 6159 for the Responses replay wiring

check:file-size in PR mode caps a frozen file at max(frozen, base). The rebase onto
the v3.8.51 tip (cde49c937) leaves chatCore.ts at 6159 lines against a 6146 ceiling:
the onReasoningReplayHistory callback on both Responses-capable translateRequest call
sites, reasoningReplayHistory on both non-streaming leg inputs, and the historyMessages
fallback at the streaming cache write. Record the growth with a justification key, as
#13033 did for the same file.

---------

Co-authored-by: jmche <jmche@users.noreply.github.com>
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
This commit is contained in:
James
2026-09-18 23:59:06 +09:00
committed by GitHub
parent 6768b14b54
commit 3ebea07278
9 changed files with 672 additions and 25 deletions

View File

@@ -0,0 +1 @@
- **fix(sse):** reasoning replay now works for Chat Completions and Anthropic Messages clients on Responses-API reasoning targets such as `opencode-go/deepseek-v4-flash`: plain (non-tool-call) assistant turns are captured against the same normalized transcript the read side digests (the Responses body carries `input`, not `messages`, so the write side digested only the assistant message instead of the full transcript and every replay missed), and the replay pass runs on the OpenAI pivot for every source format, so Anthropic Messages clients are replayed too. Fixes the intermittent `400 The reasoning_text in the thinking mode must be passed back to the API` from Console Go for clients that drop `reasoning_content` ([#13031](https://github.com/diegosouzapw/OmniRoute/pull/13031)) — thanks @jmche

View File

@@ -41,6 +41,8 @@ Turn N+1 (client sends follow-up):
Capture happens in `open-sse/handlers/chatCore.ts` (two sites, at the two `cacheReasoningFromAssistantMessage` call sites). Replay happens in `open-sse/translator/index.ts` after schema coercion but before dispatch.
Plain (non-tool-call) assistant turns are keyed differently: `buildAssistantMessageCacheKey()` digests the session scope plus the normalized OpenAI-format transcript up to that turn, because DeepSeek requires the reasoning of _every_ prior turn once `tools` is present. For Responses-API targets (for example `opencode-go/deepseek-v4-flash`, routed to `/responses`) the upstream body carries `input`, not `messages`, so `translateRequest()` (`open-sse/translator/index.ts`) reports the pivot transcript it digested through a callback option and the capture sites digest that same transcript. The Responses replay pass runs on the OpenAI pivot for every source format, so Anthropic Messages clients (Claude → OpenAI → Responses) are replayed too.
## Storage — Hybrid Memory + SQLite
The hot path uses an in-memory `Map` (LRU-by-creation) backed by a SQLite table for crash recovery and dashboard visibility.

View File

@@ -1070,6 +1070,11 @@ export async function handleChatCore({
const reasoningCacheScope = reasoningReplaySessionKey
? `api-key:${String(apiKeyInfo?.id ?? "local")}\x1f${String(reasoningReplaySessionKey)}`
: null;
// Normalized OpenAI transcript the reasoning replay pass digested for a
// Responses-API target (reported by translateRequest). A Responses body has
// `input`, not `messages`, so the replay-cache write side would otherwise digest
// an empty history and never match the read side for plain assistant turns.
let reasoningReplayHistory: unknown[] | null = null;
// persistAttemptLogs extracted to chatCore/attemptLogging.ts (#3501); bind the per-request context
// once so the 16 call sites keep passing only the per-attempt args (byte-identical).
const persistAttemptLogs = (args: PersistAttemptLogsArgs) =>
@@ -2555,6 +2560,9 @@ export async function handleChatCore({
signatureNamespace: connectionId,
copilotClient: copilotCompatibleReasoning,
reasoningCacheScope,
onReasoningReplayHistory: (messages) => {
reasoningReplayHistory = messages;
},
...(preCompressionBody ? { preCompressionBody } : {}),
}
);
@@ -5042,6 +5050,7 @@ export async function handleChatCore({
toolNameMap,
requestToolIdentityMap,
reasoningCacheScope,
reasoningReplayHistory,
clientHeaders: clientRawRequest?.headers ?? null,
isClaudeCodeCompatible,
log,
@@ -5185,6 +5194,9 @@ export async function handleChatCore({
signatureNamespace: connectionId,
copilotClient: copilotCompatibleReasoning,
reasoningCacheScope,
onReasoningReplayHistory: (messages) => {
reasoningReplayHistory = messages;
},
}
);
return runNonStreamingProviderLeg(
@@ -5210,6 +5222,7 @@ export async function handleChatCore({
toolNameMap,
requestToolIdentityMap,
reasoningCacheScope,
reasoningReplayHistory,
clientHeaders: clientRawRequest?.headers ?? null,
isClaudeCodeCompatible,
log,
@@ -5858,8 +5871,11 @@ export async function handleChatCore({
const choices = cacheStreamBody.choices as
{ message?: Record<string, unknown> }[] | undefined;
const msg = choices?.[0]?.message;
const historyMessages = (translatedBody as { messages?: unknown[] } | null | undefined)
?.messages;
// Responses-shaped bodies carry `input`, not `messages` — use the pivot
// transcript translateRequest reported so plain-turn keys match the read side.
const historyMessages =
(translatedBody as { messages?: unknown[] } | null | undefined)?.messages ??
reasoningReplayHistory;
if (requiresReasoningReplay({ provider, model })) {
cacheReasoningFromAssistantMessage(msg, provider, model, {
scope: reasoningCacheScope,

View File

@@ -90,6 +90,10 @@ export interface ProviderLegInput {
toolNameMap?: Map<string, string> | null;
requestToolIdentityMap?: Map<string, { namespace?: string; name: string }> | null;
reasoningCacheScope?: string | null;
/** Normalized OpenAI transcript reported by translateRequest for Responses-API
* targets (their body has `input`, not `messages`) — the replay-cache write
* side must digest the same transcript the read side keyed plain turns on. */
reasoningReplayHistory?: unknown[] | null;
clientHeaders?: Headers | Record<string, unknown> | null;
isClaudeCodeCompatible?: boolean;
sleep?: (ms: number) => Promise<void>;
@@ -281,8 +285,10 @@ function finishOk(
provider: params.provider,
model: params.model,
requestBody: params.requestBody,
historyMessages: (input.translatedBody as { messages?: unknown[] } | null | undefined)
?.messages,
historyMessages:
(input.translatedBody as { messages?: unknown[] } | null | undefined)?.messages ??
input.reasoningReplayHistory ??
null,
responseToolNameMap,
requestToolIdentityMap: input.requestToolIdentityMap ?? null,
reasoningCacheScope: input.reasoningCacheScope ?? null,

View File

@@ -359,7 +359,11 @@ export function cacheReasoningFromAssistantMessage(
if (toolCallIds.length === 0) {
const scope = context?.scope?.trim();
const historyMessages = context?.historyMessages;
if (!scope || !Array.isArray(historyMessages)) return 0;
// A real request always has at least one prior message (the user turn), so an
// empty history means the caller could not recover the transcript the read
// side keys on (e.g. a Responses-shaped body with `input` and no reported
// pivot). Writing a one-message digest then can never match — skip it.
if (!scope || !Array.isArray(historyMessages) || historyMessages.length === 0) return 0;
const messages = [...historyMessages, message];
const cacheKey = buildAssistantMessageCacheKey(scope, messages, messages.length - 1);

View File

@@ -204,6 +204,35 @@ function requiresReasoningContentPresence(provider: unknown, model: unknown): bo
return normalizedProvider === "xiaomi-mimo" || /(^|\/)mimo/i.test(normalizedModel);
}
/**
* Projects the pivot transcript down to what `buildAssistantMessageCacheKey`
* digests (`role`, `name`, `content`, and `tool_calls[].{type, function.name,
* function.arguments}`). The caller keeps the result for the whole request, so
* nothing the digest ignores is retained: `reasoning_content` is dropped (the
* write side receives the upstream reasoning separately) and tool-call ids are
* dropped. `content` is shared by reference — the digest only reads it, and the
* Responses conversion that follows re-references content parts without mutating
* them.
*/
function snapshotReasoningReplayHistory(
messages: Array<Record<string, unknown>>
): Array<Record<string, unknown>> {
return messages.map((message) => {
const record = message && typeof message === "object" ? message : {};
const snapshot: Record<string, unknown> = { role: record.role };
if (record.name !== undefined) snapshot.name = record.name;
if (record.content !== undefined) snapshot.content = record.content;
if (Array.isArray(record.tool_calls)) {
snapshot.tool_calls = record.tool_calls.map((toolCall) => {
const call = (toolCall ?? {}) as Record<string, unknown>;
const fn = (call.function ?? {}) as Record<string, unknown>;
return { type: call.type, function: { name: fn.name, arguments: fn.arguments } };
});
}
return snapshot;
});
}
type OpenAIReplayOptions = {
canReplayReasoningOnly: boolean;
requiresExplicitReasoningReplay: boolean;
@@ -320,6 +349,12 @@ export function translateRequest(
signatureNamespace?: string | null;
preCompressionBody?: Record<string, unknown> | null;
reasoningCacheScope?: string | null;
/** Receives the normalized OpenAI-format transcript the reasoning replay pass
* digested for a Responses-API target. A Responses body carries `input`, not
* `messages`, so the caller cannot recover that transcript from the returned
* body; the replay cache keys plain (non-tool-call) assistant turns on exactly
* this transcript, and the write side must digest the same one (#1682). */
onReasoningReplayHistory?: (messages: Array<Record<string, unknown>>) => void;
/** UA-detected GitHub Copilot client. Forwarded to translators via the
* transient `_copilotClient` credential flag (see openai-responses → openai). */
copilotClient?: boolean;
@@ -414,25 +449,6 @@ export function translateRequest(
result.messages = hoistLeadingSystemMessage(result.messages, provider);
}
if (
sourceFormat === FORMATS.OPENAI &&
targetFormat === FORMATS.OPENAI_RESPONSES &&
isReasoner &&
Array.isArray(result.messages)
) {
const messages = result.messages as Array<Record<string, unknown>>;
const replayOptions: OpenAIReplayOptions = {
canReplayReasoningOnly: isReasoningOnlyReplayTarget(normalizedProvider, normalizedModel),
requiresExplicitReasoningReplay,
provider: normalizedProvider,
model: normalizedModel,
reasoningCacheScope: options?.reasoningCacheScope,
};
for (let messageIndex = 0; messageIndex < messages.length; messageIndex += 1) {
replayOpenAIReasoningMessage(messages, messageIndex, replayOptions);
}
}
// If same format, skip translation steps
if (sourceFormat !== targetFormat) {
// Check for direct translation path first (e.g., Claude → Gemini)
@@ -491,6 +507,36 @@ export function translateRequest(
}
}
// Reasoning replay for Responses-API targets runs on the OpenAI pivot, before
// the Responses conversion discards `messages`. It used to be gated on
// `sourceFormat === "openai"`, which left Anthropic Messages clients (Claude →
// OpenAI → Responses) with no replay at all: the generic pass further down only
// sees `result.messages`, and a Responses body has none. The pivot is the same
// transcript the replay cache keys plain turns on, so report it to the caller
// for the write side (#1682 — DeepSeek requires every prior turn's reasoning
// once `tools` is present). Known divergence: a `_ensureUserTurn` synthetic
// user turn appended by step 1 is part of this transcript but not of the
// client's next request, so that (tool-loop-only) shape keys a plain turn
// the next read cannot match — it degrades to a cache miss, never a wrong hit.
if (
targetFormat === FORMATS.OPENAI_RESPONSES &&
isReasoner &&
Array.isArray(result.messages)
) {
const messages = result.messages as Array<Record<string, unknown>>;
const replayOptions: OpenAIReplayOptions = {
canReplayReasoningOnly: isReasoningOnlyReplayTarget(normalizedProvider, normalizedModel),
requiresExplicitReasoningReplay,
provider: normalizedProvider,
model: normalizedModel,
reasoningCacheScope: options?.reasoningCacheScope,
};
for (let messageIndex = 0; messageIndex < messages.length; messageIndex += 1) {
replayOpenAIReasoningMessage(messages, messageIndex, replayOptions);
}
options?.onReasoningReplayHistory?.(snapshotReasoningReplayHistory(messages));
}
// Step 2: openai -> target (if target is not openai)
if (targetFormat !== FORMATS.OPENAI) {
const fromOpenAI = getRequestTranslator(FORMATS.OPENAI, targetFormat);

View File

@@ -185,7 +185,9 @@ export interface NonStreamingClientTranslateInput {
/**
* Transcript used for no-tool_calls reasoning replay (#1628).
* Must be the client-translated Chat `messages` (parent: `translatedBody.messages`),
* not `finalBody` — Responses-shaped `finalBody` has `input`, not `messages`.
* not `finalBody` — Responses-shaped `finalBody` has `input`, not `messages`. For a
* Responses-shaped body the parent passes the pivot transcript `translateRequest`
* reports through `onReasoningReplayHistory` instead.
*/
historyMessages?: unknown[] | null;
responseToolNameMap: Map<string, string> | null;

View File

@@ -0,0 +1,268 @@
// Integration guard for the reasoning-cache write path on a Responses-API target.
//
// tests/unit/chatcore-reasoning-cache-write-guard.test.ts proves the capture sites
// fire for a Chat-format upstream (xiaomi-mimo) and deliberately avoids DeepSeek
// because its wire format is openai-responses. This file covers exactly that lane:
// opencode-go/deepseek-v4-flash is dispatched to `/responses`, so the translated
// upstream body carries `input`, not `messages`, and the plain-turn cache key must
// be built from the pivot transcript `translateRequest` reports instead.
//
// Same convention as the sibling file: mock fetch, call the real handleChatCore,
// assert real behavior — here the observable behavior is the SECOND turn's upstream
// request body: a client that replays history without `reasoning_content` must
// still produce a `reasoning` input item ahead of the assistant message, or
// DeepSeek answers `400 The reasoning_text in the thinking mode must be passed back`.
import { test } from "node:test";
import assert from "node:assert/strict";
import fs from "node:fs";
import os from "node:os";
import path from "node:path";
const TEST_DATA_DIR = fs.mkdtempSync(
path.join(os.tmpdir(), "omniroute-chatcore-reasoning-cache-write-guard-responses-")
);
process.env.DATA_DIR = TEST_DATA_DIR;
const { handleChatCore } = await import("../../open-sse/handlers/chatCore.ts");
const { clearReasoningCacheAll } = await import("../../open-sse/services/reasoningCache.ts");
const core = await import("../../src/lib/db/core.ts");
type JsonRecord = Record<string, unknown>;
const PROVIDER = "opencode-go";
const MODEL = "deepseek-v4-flash";
const REASONING = "The user only wants a greeting, so no tool is needed.";
const TOOLS = [
{
type: "function",
function: {
name: "read_file",
description: "Read a file",
parameters: { type: "object", properties: { path: { type: "string" } } },
},
},
];
function noopLog() {
return { debug() {}, info() {}, warn() {}, error() {} };
}
function completedResponse() {
return {
id: "resp_reasoning_guard",
object: "response",
status: "completed",
model: MODEL,
output: [
{
type: "reasoning",
id: "rs_reasoning_guard",
summary: [],
content: [{ type: "reasoning_text", text: REASONING }],
},
{
type: "message",
id: "msg_reasoning_guard",
role: "assistant",
status: "completed",
content: [{ type: "output_text", text: "Hi!", annotations: [] }],
},
],
usage: { input_tokens: 1, output_tokens: 1, total_tokens: 2 },
};
}
function nonStreamingUpstreamResponse() {
return new Response(JSON.stringify(completedResponse()), {
status: 200,
headers: { "content-type": "application/json" },
});
}
function streamingUpstreamResponse() {
const completed = completedResponse();
const events: Array<[string, JsonRecord]> = [
[
"response.created",
{ type: "response.created", response: { ...completed, status: "in_progress", output: [] } },
],
[
"response.output_item.added",
{
type: "response.output_item.added",
output_index: 0,
item: { type: "reasoning", id: "rs_reasoning_guard", summary: [], content: [] },
},
],
[
"response.reasoning_text.delta",
{
type: "response.reasoning_text.delta",
item_id: "rs_reasoning_guard",
output_index: 0,
content_index: 0,
delta: REASONING,
},
],
[
"response.output_item.done",
{ type: "response.output_item.done", output_index: 0, item: completed.output[0] },
],
[
"response.output_item.added",
{
type: "response.output_item.added",
output_index: 1,
item: {
type: "message",
id: "msg_reasoning_guard",
role: "assistant",
status: "in_progress",
content: [],
},
},
],
[
"response.output_text.delta",
{
type: "response.output_text.delta",
item_id: "msg_reasoning_guard",
output_index: 1,
content_index: 0,
delta: "Hi!",
},
],
[
"response.output_item.done",
{ type: "response.output_item.done", output_index: 1, item: completed.output[1] },
],
["response.completed", { type: "response.completed", response: completed }],
];
const sseBody = events
.map(([event, data]) => `event: ${event}\ndata: ${JSON.stringify(data)}\n\n`)
.join("");
return new Response(sseBody, {
status: 200,
headers: { "content-type": "text/event-stream" },
});
}
async function drain(result: { success?: boolean; response?: Response }) {
if (!result.success || !result.response) return;
if (result.response.body) {
const reader = result.response.body.getReader();
for (;;) {
const { done } = await reader.read();
if (done) break;
}
} else {
try {
await result.response.text();
} catch {}
}
await new Promise((resolve) => setImmediate(resolve));
}
/**
* Runs one client turn through the real handleChatCore against a mocked upstream and
* returns the JSON body OmniRoute sent upstream.
*/
async function runTurn(
session: string,
messages: unknown[],
stream: boolean,
upstream: () => Response
): Promise<JsonRecord> {
const originalFetch = globalThis.fetch;
const sent: Array<{ url: string; body: JsonRecord }> = [];
globalThis.fetch = (async (input: unknown, init?: { body?: unknown }) => {
const url = String((input as { url?: unknown })?.url ?? input);
if (typeof init?.body === "string") {
sent.push({ url, body: JSON.parse(init.body) as JsonRecord });
}
return upstream();
}) as typeof fetch;
try {
const body = { model: MODEL, messages, tools: TOOLS, stream };
const result = await handleChatCore({
body,
modelInfo: { provider: PROVIDER, model: MODEL, extendedContext: false },
credentials: { apiKey: "sk-test", providerSpecificData: {} },
log: noopLog(),
clientRawRequest: {
endpoint: "/v1/chat/completions",
body,
headers: new Headers({ accept: stream ? "text/event-stream" : "application/json" }),
},
userAgent: "unit-test",
sessionAffinityKey: session,
} as never);
await drain(result as { success?: boolean; response?: Response });
} finally {
globalThis.fetch = originalFetch;
}
// Select the Responses-lane dispatch by URL rather than "last fetch", so an
// unrelated POST inside the handler window can neither satisfy nor break this.
const responsesCalls = sent.filter((call) => /\/responses(\?|$)/.test(call.url));
assert.equal(responsesCalls.length, 1, `exactly one /responses dispatch (saw ${sent.length})`);
return responsesCalls[0].body;
}
function reasoningTexts(input: unknown): string[] {
return (Array.isArray(input) ? input : [])
.filter((item) => (item as JsonRecord)?.type === "reasoning")
.map((item) =>
((item as JsonRecord).content as Array<{ text?: string }>)
.map((part) => part.text ?? "")
.join("")
);
}
const TURN_1 = [
{ role: "system", content: "You are helpful." },
{ role: "user", content: "Say hi first, no tools." },
];
const TURN_2 = [
...TURN_1,
// The client replays the plain assistant turn WITHOUT reasoning_content.
{ role: "assistant", content: "Hi!" },
{ role: "user", content: "Now read README.md" },
];
test.after(() => {
try {
core.resetDbInstance();
} catch {}
try {
clearReasoningCacheAll();
} catch {}
fs.rmSync(TEST_DATA_DIR, { recursive: true, force: true, maxRetries: 5, retryDelay: 100 });
});
test("non-streaming: a plain turn captured from a Responses upstream is replayed on the next turn", async () => {
clearReasoningCacheAll();
const session = "reasoning-cache-write-guard-responses-nonstream";
const first = await runTurn(session, TURN_1, false, nonStreamingUpstreamResponse);
assert.ok(Array.isArray(first.input), "turn 1 went upstream as a Responses body");
assert.equal(first.messages, undefined);
const second = await runTurn(session, TURN_2, false, nonStreamingUpstreamResponse);
const input = second.input as JsonRecord[];
assert.deepEqual(reasoningTexts(input), [REASONING]);
const reasoningIndex = input.findIndex((item) => item.type === "reasoning");
const assistantIndex = input.findIndex(
(item) => item.type === "message" && item.role === "assistant"
);
assert.ok(reasoningIndex >= 0 && reasoningIndex < assistantIndex);
});
test("streaming: a plain turn captured from a Responses SSE upstream is replayed on the next turn", async () => {
clearReasoningCacheAll();
const session = "reasoning-cache-write-guard-responses-stream";
const first = await runTurn(session, TURN_1, true, streamingUpstreamResponse);
assert.ok(Array.isArray(first.input), "turn 1 went upstream as a Responses body");
assert.equal(first.messages, undefined);
const second = await runTurn(session, TURN_2, false, nonStreamingUpstreamResponse);
assert.deepEqual(reasoningTexts(second.input), [REASONING]);
});

View File

@@ -0,0 +1,302 @@
/**
* Reasoning replay for Responses-API targets (opencode-go/deepseek-v4-flash after #10540).
*
* DeepSeek's thinking mode requires the reasoning of EVERY prior assistant turn to be
* passed back whenever the request carries `tools` — including turns that made no tool
* call. OmniRoute's replay cache re-injects that reasoning for clients that drop it.
*
* Two gaps existed once the model moved from Chat Completions to `/responses`:
*
* 1. Plain (non-tool-call) turns are cached under a digest of the normalized OpenAI
* transcript. chatCore fed the write side `translatedBody.messages`, which a
* Responses-shaped body (`input`) does not have, so the write-time digest covered
* only the assistant message while the read side digested the full transcript —
* every plain-turn replay missed.
* 2. The Responses replay pass only ran for `sourceFormat === "openai"`. Anthropic
* Messages clients pivot Claude → OpenAI → Responses, and the generic replay pass
* runs after the Responses conversion (no `messages` left), so those clients got no
* replay at all, not even for tool-call turns.
*
* `translateRequest` now reports the normalized pivot transcript it digested through
* `onReasoningReplayHistory`, and chatCore uses that as the write-side history for
* Responses targets. The replay pass runs on the pivot for every source format.
*/
import { describe, it, before, after } from "node:test";
import assert from "node:assert/strict";
import { mkdtempSync } from "node:fs";
import { join } from "node:path";
import { tmpdir } from "node:os";
process.env.DATA_DIR = mkdtempSync(join(tmpdir(), "omniroute-reasoning-history-"));
process.env.API_KEY_SECRET = process.env.API_KEY_SECRET || "reasoning-history-test-secret";
import {
cacheReasoningFromAssistantMessage,
clearReasoningCacheAll,
} from "../../open-sse/services/reasoningCache.ts";
import { translateRequest } from "../../open-sse/translator/index.ts";
import { FORMATS } from "../../open-sse/translator/formats.ts";
import { clearModelsDevCapabilities } from "../../src/lib/modelsDevSync.ts";
type JsonRecord = Record<string, unknown>;
const PROVIDER = "opencode-go";
const MODEL = "deepseek-v4-flash";
const SCOPE = "api-key:local\x1finput:sha256:reasoning-history";
const PLAIN_REASONING = "The user only wants a greeting; no tool is needed.";
const TOOL_REASONING = "The user asked for the file contents, so read it first.";
const OPENAI_TOOLS = [
{
type: "function",
function: {
name: "read_file",
description: "Read a file",
parameters: { type: "object", properties: { path: { type: "string" } } },
},
},
];
const CLAUDE_TOOLS = [
{
name: "read_file",
description: "Read a file",
input_schema: { type: "object", properties: { path: { type: "string" } } },
},
];
/**
* Mirrors the chatCore write path: translate the request that produced the assistant
* turn, keep the transcript `translateRequest` reports, then capture the upstream
* response's reasoning against that transcript.
*/
function translateAndCapture(
sourceFormat: string,
body: JsonRecord,
assistantResponse: JsonRecord
): { translated: JsonRecord; history: unknown[] | null; keysWritten: number } {
let history: unknown[] | null = null;
const translated = translateRequest(
sourceFormat,
FORMATS.OPENAI_RESPONSES,
MODEL,
structuredClone(body),
false,
null,
PROVIDER,
null,
{
reasoningCacheScope: SCOPE,
onReasoningReplayHistory: (messages: unknown[]) => {
history = messages;
},
}
) as JsonRecord;
const historyMessages = Array.isArray(translated.messages)
? (translated.messages as unknown[])
: history;
const keysWritten = cacheReasoningFromAssistantMessage(
assistantResponse as Parameters<typeof cacheReasoningFromAssistantMessage>[0],
PROVIDER,
MODEL,
{
scope: SCOPE,
historyMessages: Array.isArray(historyMessages) ? historyMessages : [],
}
);
return { translated, history, keysWritten };
}
function translateFollowUp(sourceFormat: string, body: JsonRecord): JsonRecord[] {
const translated = translateRequest(
sourceFormat,
FORMATS.OPENAI_RESPONSES,
MODEL,
structuredClone(body),
false,
null,
PROVIDER,
null,
{ reasoningCacheScope: SCOPE }
) as { input: JsonRecord[] };
return translated.input;
}
function reasoningTexts(input: JsonRecord[]): string[] {
return input
.filter((item) => item.type === "reasoning")
.map((item) =>
(item.content as Array<{ text?: string }>).map((part) => part.text ?? "").join("")
);
}
describe("reasoning replay history for Responses targets (opencode-go/deepseek-v4-flash)", () => {
before(() => {
clearModelsDevCapabilities();
clearReasoningCacheAll();
});
after(() => {
clearReasoningCacheAll();
clearModelsDevCapabilities();
});
it("replays a plain assistant turn for an OpenAI Chat client", () => {
clearReasoningCacheAll();
const turn1 = {
model: MODEL,
tools: OPENAI_TOOLS,
messages: [
{ role: "system", content: "You are helpful." },
{ role: "user", content: "Say hi first, no tools." },
],
};
const { translated, history, keysWritten } = translateAndCapture(FORMATS.OPENAI, turn1, {
role: "assistant",
content: "Hi!",
reasoning_content: PLAIN_REASONING,
});
assert.equal(translated.messages, undefined, "Responses body carries input, not messages");
assert.ok(Array.isArray(history), "translateRequest reports the transcript it digested");
assert.equal(keysWritten, 1);
// Turn 2: the client replays history without reasoning_content (default OpenAI client).
const input = translateFollowUp(FORMATS.OPENAI, {
model: MODEL,
tools: OPENAI_TOOLS,
messages: [
...turn1.messages,
{ role: "assistant", content: "Hi!" },
{ role: "user", content: "Now read README.md" },
],
});
assert.deepEqual(reasoningTexts(input), [PLAIN_REASONING]);
const reasoningIndex = input.findIndex((item) => item.type === "reasoning");
const assistantIndex = input.findIndex(
(item) => item.type === "message" && item.role === "assistant"
);
assert.ok(reasoningIndex < assistantIndex, "reasoning precedes the assistant message");
});
it("replays a tool-call turn for an Anthropic Messages client", () => {
clearReasoningCacheAll();
const toolUseId = "call_read_readme_1";
// Turn 1 response (already translated to Chat shape by the response translator)
// is captured under the tool_call id, as chatCore does.
cacheReasoningFromAssistantMessage(
{
role: "assistant",
content: "",
reasoning_content: TOOL_REASONING,
tool_calls: [
{ id: toolUseId, type: "function", function: { name: "read_file", arguments: "{}" } },
],
},
PROVIDER,
MODEL,
{ scope: SCOPE, historyMessages: [] }
);
// Turn 2: Claude client sends tool_use / tool_result history without a thinking block.
const input = translateFollowUp(FORMATS.CLAUDE, {
model: MODEL,
max_tokens: 1024,
tools: CLAUDE_TOOLS,
messages: [
{ role: "user", content: "Read README.md" },
{
role: "assistant",
content: [
{ type: "tool_use", id: toolUseId, name: "read_file", input: { path: "README.md" } },
],
},
{
role: "user",
content: [{ type: "tool_result", tool_use_id: toolUseId, content: "# README" }],
},
],
});
assert.deepEqual(reasoningTexts(input), [TOOL_REASONING]);
const reasoningIndex = input.findIndex((item) => item.type === "reasoning");
const functionCallIndex = input.findIndex((item) => item.type === "function_call");
assert.ok(reasoningIndex < functionCallIndex, "reasoning precedes the function_call");
});
it("replays a plain assistant turn for an Anthropic Messages client", () => {
clearReasoningCacheAll();
const turn1 = {
model: MODEL,
max_tokens: 1024,
tools: CLAUDE_TOOLS,
system: "You are helpful.",
messages: [{ role: "user", content: "Say hi first, no tools." }],
};
const { history, keysWritten } = translateAndCapture(FORMATS.CLAUDE, turn1, {
role: "assistant",
content: "Hi!",
reasoning_content: PLAIN_REASONING,
});
assert.ok(Array.isArray(history));
assert.equal(keysWritten, 1);
const input = translateFollowUp(FORMATS.CLAUDE, {
...turn1,
messages: [
...turn1.messages,
{ role: "assistant", content: [{ type: "text", text: "Hi!" }] },
{ role: "user", content: "Now read README.md" },
],
});
assert.deepEqual(reasoningTexts(input), [PLAIN_REASONING]);
});
it("keeps the reported transcript out of the upstream body and off Chat targets", () => {
let responsesHistory: unknown[] | null = null;
const responsesBody = translateRequest(
FORMATS.OPENAI,
FORMATS.OPENAI_RESPONSES,
MODEL,
{ model: MODEL, tools: OPENAI_TOOLS, messages: [{ role: "user", content: "hi" }] },
false,
null,
PROVIDER,
null,
{
reasoningCacheScope: SCOPE,
onReasoningReplayHistory: (messages: unknown[]) => {
responsesHistory = messages;
},
}
) as JsonRecord;
assert.ok(Array.isArray(responsesHistory));
const serialized = JSON.parse(JSON.stringify(responsesBody)) as JsonRecord;
assert.equal(serialized.messages, undefined);
assert.ok(
!Object.keys(serialized).some((key) => /replay|history/i.test(key)),
"no replay bookkeeping leaks into the upstream payload"
);
let chatHistoryCalls = 0;
const chatBody = translateRequest(
FORMATS.OPENAI,
FORMATS.OPENAI,
MODEL,
{ model: MODEL, tools: OPENAI_TOOLS, messages: [{ role: "user", content: "hi" }] },
false,
null,
PROVIDER,
null,
{
reasoningCacheScope: SCOPE,
onReasoningReplayHistory: () => {
chatHistoryCalls += 1;
},
}
) as JsonRecord;
assert.ok(Array.isArray(chatBody.messages), "Chat targets keep messages[] for the write side");
assert.equal(chatHistoryCalls, 0, "Chat targets do not need the side channel");
});
});