pjandro/OmniRoute
1
0
Fork 0
mirror of https://github.com/diegosouzapw/OmniRoute.git synced 2026-07-26 09:52:11 +03:00
Code Issues Packages Projects Releases Wiki Activity

fix(sse): Gemini malformed function-call handling + tool_choice translation (#8211)

Browse Source 
* fix(sse): synthesize tool_calls for Gemini's malformed function-call abort reasons

Live incident (dashboard log id 1784489701456-d8c0e9): Gemini terminates a
stream with finishReason MALFORMED_FUNCTION_CALL/UNEXPECTED_TOOL_CALL when its
own parser rejects an attempted tool call — there's no real functionCall part,
only a human-readable finishMessage. gemini-to-openai.ts passed this through
raw as finish_reason (9router#2462's fix, correctly keeping it off a clean
"stop"/Claude end_turn), but a raw "malformed_function_call" isn't one of
OpenAI's 5 documented finish_reason values, so a real OpenAI-format client
(OpenClaw) has no handling for it at all and silently never notices the turn
failed — confirmed live via tests/integration/live-gemini-workload.test.ts's
[28] streaming case after the Gemini TPM/rebase work on this branch.

Fix: synthesize a tool_calls entry (arguments carry the error code + Gemini's
finishMessage, valid JSON) and finish_reason: "tool_calls" instead, routing
the failure into the ordinary "tool call arguments didn't parse" path every
OpenAI-compatible agent loop already handles. Defers to a real tool call if
one already completed earlier in the same turn — the real call wins, no
synthetic entry piles on top of it.

Tests (TDD, each confirmed red-before-green):
- 5 new unit tests in the existing 9router#2462 regression file, covering the
  synthesis itself, UNEXPECTED_TOOL_CALL, the real-tool-call-wins edge case,
  and no-regression on a clean STOP.
- New fixture (tests/fixtures/translation/gemini-malformed-function-call-stream.json):
  the real 6-chunk event series from the live incident, sanitized (personal
  paths/URLs replaced with generic placeholders, structure preserved exactly).
- New integration test chains the real translator into the real Responses API
  transformer using that same fixture, proving correct behavior on BOTH
  /v1/chat/completions and /v1/responses from one shared ground-truth event
  series.

Co-authored-by: Markus Hartung <markus.hartung@gmail.com>

* fix(sse): don't drop a malformed tool-call failure when it lands beside a real one

Live incident (dashboard log id 1784589106014-2a42f8), analyzing why the
prior malformed-function-call fix (3568c7259) still wasn't reaching the
client in this case: Gemini can emit a REAL, valid functionCall AND finish
the SAME candidate with MALFORMED_FUNCTION_CALL — the model attempted
multiple tool calls in one turn (here: a real status-check call plus a
malformed "exec"+"cron" multi-call attempt), one parsed cleanly and the
other didn't.

The first fix version skipped synthesizing a failure signal whenever a real
tool call already existed (state.toolCalls.size > 0), on the assumption
that meant the model was retrying a LATER, separate attempt after an
earlier one already succeeded. That's indistinguishable, from the
translator's state, from this same-turn case — so it silently discarded
the malformed attempt's information entirely: the client saw the real call
succeed and never learned the other tool calls were attempted and rejected.

Fix: always synthesize the failure entry when a malformed abort reason is
seen, appending it alongside any real tool call rather than skipping it.
Multiple tool_calls in one response is normal, well-supported OpenAI
behavior (parallel tool calls), so this adds the failure as an additional
entry instead of replacing or hiding the real one.

Tests (TDD, confirmed red-before-green):
- Rewrote the unit test that encoded the old (wrong) assumption to assert
  both the real and synthesized calls are present.
- New fixture (gemini-malformed-function-call-parallel-real-call-stream.json):
  the real event series from this incident, sanitized.
- New integration tests (same file as 3568c7259's) prove both
  /v1/chat/completions and /v1/responses surface both tool calls correctly
  from this fixture.

Co-authored-by: Markus Hartung <markus.hartung@gmail.com>

* feat(sse): honor tool_choice when translating OpenAI requests to Gemini

Investigating a live report that gemini-3.1-flash-lite frequently narrates
an intended tool call in plain text instead of actually emitting one
(dashboard log id 1784591483850-49c408 — 9 raw provider chunks, all plain
text, zero functionCall parts, clean finishReason STOP): body.tool_choice
was never read anywhere in the OpenAI->Gemini request translator.
result.toolConfig was unconditionally hardcoded to
{ functionCallingConfig: { mode: "VALIDATED" } } whenever tools were
present, regardless of what the caller sent. VALIDATED lets the model
respond with plain text OR a schema-validated function call at its own
discretion — it never forces a call the way OpenAI's tool_choice:
"required" (Gemini's ANY mode) does, so a caller had no way to compel a
tool call even when explicitly requesting one.

Added convertOpenAIToolChoiceToGemini(), mirroring the existing
convertOpenAIToolChoice() in openai-to-claude.ts for the same OpenAI
tool_choice shapes (string "auto"/"none"/"required", or
{type:"function",function:{name}} to force one specific tool):
  - unset/"auto"        -> VALIDATED (unchanged default, no regression)
  - "required"/"any"    -> ANY (forces a call)
  - "none"              -> NONE (disables function calling)
  - {type:"function",...} -> ANY + allowedFunctionNames: [name]

Wired into both Gemini request paths: the direct/base translator
(openaiToGeminiBase) and the Antigravity/Cloud Code envelope
(wrapInCloudCodeEnvelope), which now reuses the base translator's already-
computed toolConfig instead of re-deriving its own hardcoded VALIDATED.

This unblocks (but does not itself resolve) the live question — a
tool_choice: "required" A/B test against gemini-3.1-flash-lite follows to
confirm ANY mode actually changes the narrate-vs-act behavior in practice.

Also updates the T11 any-budget allowlist for this file: the "any" string
comparisons (tool_choice value "any", not a TypeScript type) are the same
documented false-positive pattern already carved out for executors/base.ts.

Tests (TDD, confirmed red-before-green): 9 new unit tests covering all
tool_choice shapes on both the direct and Antigravity/Cloud Code paths,
plus the no-tools and unset-default no-regression cases.

Co-authored-by: Markus Hartung <markus.hartung@gmail.com>

* chore(quality): file-size baseline for own-growth (#8211)

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>

---------

Co-authored-by: Markus Hartung <markus.hartung@gmail.com>
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
This commit is contained in:
Markus Hartung
2026-07-23 10:03:44 +02:00
committed by GitHub
parent 40eb5a87a7
commit 4ea08f520b
10 changed files with 930 additions and 14 deletions
Download Patch File Download Diff File Expand all files Collapse all files

98
tests/fixtures/translation/gemini-malformed-function-call-parallel-real-call-stream.json vendored Normal file
View File

@@ -0,0 +1,98 @@
{
"name": "gemini-malformed-function-call-parallel-real-call-live-incident",
"description": "Real raw Gemini streaming response chunks captured from a live incident (dashboard log id 1784589106014-2a42f8, 2026-07-20) — a follow-up to gemini-malformed-function-call-stream.json. Unlike that fixture (no real functionCall at all), THIS incident shows Gemini emitting a REAL, valid functionCall in the same turn where it ALSO attempts a separate malformed multi-call and finishes the candidate with finishReason MALFORMED_FUNCTION_CALL. An earlier version of the fix skipped synthesizing a failure signal whenever a real tool call already existed, which silently discarded the malformed attempt's information in exactly this case. Content sanitized: personal file paths replaced with generic placeholders. Tool name genericized (real incident used a self-referential 'check status of this agent' tool named after the client itself). Structure (chunk count, field shapes, thoughtSignature split across chunks) preserved exactly as captured.",
"chunks": [
{
"candidates": [
{
"content": {
"parts": [
{
"text": "**Re-evaluating the approach**\n\nI've hit a roadblock with the status request; the previous attempt was unsuccessful and subsequently aborted. I'm now reassessing the current strategy, and considering how to proceed. I'll need to explore alternative methods, or perhaps inform the user of the limitations I'm facing at this time.\n\n\n",
"thought": true
}
],
"role": "model"
},
"index": 0
}
],
"usageMetadata": {
"promptTokenCount": 34579,
"totalTokenCount": 34579,
"promptTokensDetails": [{ "modality": "TEXT", "tokenCount": 34579 }],
"serviceTier": "standard"
},
"modelVersion": "gemini-3.1-flash-lite",
"responseId": "resp-fixture-parallel-001"
},
{
"candidates": [
{
"content": {
"parts": [
{
"functionCall": {
"name": "check_status",
"args": { "message": "status" },
"id": "zgtP7IsV"
},
"thoughtSignature": "FIXTURE_PLACEHOLDER_THOUGHT_SIGNATURE_OPAQUE_BASE64_BLOB_TRUNCATED_FOR_READABILITY=="
}
],
"role": "model"
},
"index": 0
}
],
"usageMetadata": {
"promptTokenCount": 34579,
"candidatesTokenCount": 15,
"totalTokenCount": 35035,
"promptTokensDetails": [{ "modality": "TEXT", "tokenCount": 34579 }],
"thoughtsTokenCount": 441,
"serviceTier": "standard"
},
"modelVersion": "gemini-3.1-flash-lite",
"responseId": "resp-fixture-parallel-001"
},
{
"candidates": [
{
"content": { "parts": [{ "text": "" }], "role": "model" },
"index": 0
}
],
"usageMetadata": {
"promptTokenCount": 34579,
"candidatesTokenCount": 58,
"totalTokenCount": 35078,
"promptTokensDetails": [{ "modality": "TEXT", "tokenCount": 34579 }],
"thoughtsTokenCount": 441,
"serviceTier": "standard"
},
"modelVersion": "gemini-3.1-flash-lite",
"responseId": "resp-fixture-parallel-001"
},
{
"candidates": [
{
"content": { "parts": [{ "text": "" }], "role": "model" },
"finishReason": "MALFORMED_FUNCTION_CALL",
"index": 0,
"finishMessage": "Malformed function call: call:default_api:exec{command: df -h /home/user/workspace }\n call:default_api:cron{action: status } }"
}
],
"usageMetadata": {
"promptTokenCount": 34579,
"candidatesTokenCount": 58,
"totalTokenCount": 35078,
"promptTokensDetails": [{ "modality": "TEXT", "tokenCount": 34579 }],
"thoughtsTokenCount": 441,
"serviceTier": "standard"
},
"modelVersion": "gemini-3.1-flash-lite",
"responseId": "resp-fixture-parallel-001"
}
]
}

144
tests/fixtures/translation/gemini-malformed-function-call-stream.json vendored Normal file
View File

@@ -0,0 +1,144 @@
{
"name": "gemini-malformed-function-call-live-incident",
"description": "Real raw Gemini streaming response chunks captured from a live incident (dashboard log id 1784489701456-d8c0e9, 2026-07-19) where the model attempted to chain multiple tool calls into one malformed invocation and Gemini's own parser rejected it with finishReason MALFORMED_FUNCTION_CALL instead of a real functionCall part. Content has been sanitized: real file paths, personal automation project details, and job-search site URLs were replaced with generic placeholders. Structure (chunk count, field shapes, thought-then-empty-transition-then-malformed-finish sequence) is preserved exactly as captured, including the long opaque thoughtSignature blob (its exact bytes are irrelevant — only its presence/shape matters — so it has been shortened for fixture readability).",
"chunks": [
{
"candidates": [
{
"content": {
"parts": [
{
"text": "**Initiating the Workflow**\n\nI've initiated the workflow, focusing on running `run_search.py`. The preliminary step is complete. Now, I'm fetching the initial target sites, specifically `site-a.example`, `site-b.example`, and `site-c.example`, as specified to begin the search.\n\n\n",
"thought": true
}
],
"role": "model"
},
"index": 0
}
],
"usageMetadata": {
"promptTokenCount": 24512,
"totalTokenCount": 24512,
"promptTokensDetails": [{ "modality": "TEXT", "tokenCount": 24512 }],
"serviceTier": "standard"
},
"modelVersion": "gemini-3.1-flash-lite",
"responseId": "resp-fixture-malformed-001"
},
{
"candidates": [
{
"content": {
"parts": [
{
"text": "**Expanding the Search Parameters**\n\nI'm now expanding the search to include `site-d.example` and `site-e.example` in my `web_fetch` operation. My next steps are to filter the results against the state file and score them based on matching criteria. I will then post high matches to the results channel, update the state, and log the results.\n\n\n",
"thought": true
}
],
"role": "model"
},
"index": 0
}
],
"usageMetadata": {
"promptTokenCount": 24512,
"totalTokenCount": 24512,
"promptTokensDetails": [{ "modality": "TEXT", "tokenCount": 24512 }],
"serviceTier": "standard"
},
"modelVersion": "gemini-3.1-flash-lite",
"responseId": "resp-fixture-malformed-001"
},
{
"candidates": [
{
"content": {
"parts": [
{
"text": "**Analyzing the Results**\n\nI'm now analyzing the output from the initial web fetch. I will check the data against processed_ids, filter for specific categories, and proceed with scoring based on these parameters. I'll then prepare to post high matches to the specified results channel, update the search state, and log the results.\n\n\n",
"thought": true
}
],
"role": "model"
},
"index": 0
}
],
"usageMetadata": {
"promptTokenCount": 24512,
"totalTokenCount": 24512,
"promptTokensDetails": [{ "modality": "TEXT", "tokenCount": 24512 }],
"serviceTier": "standard"
},
"modelVersion": "gemini-3.1-flash-lite",
"responseId": "resp-fixture-malformed-001"
},
{
"candidates": [
{
"content": { "parts": [{ "text": "" }], "role": "model" },
"index": 0
}
],
"usageMetadata": {
"promptTokenCount": 24512,
"candidatesTokenCount": 167,
"totalTokenCount": 25738,
"promptTokensDetails": [{ "modality": "TEXT", "tokenCount": 24512 }],
"thoughtsTokenCount": 1059,
"serviceTier": "standard"
},
"modelVersion": "gemini-3.1-flash-lite",
"responseId": "resp-fixture-malformed-001"
},
{
"candidates": [
{
"content": {
"parts": [
{
"text": "",
"thoughtSignature": "FIXTURE_PLACEHOLDER_THOUGHT_SIGNATURE_OPAQUE_BASE64_BLOB_TRUNCATED_FOR_READABILITY_the_real_value_is_several_kb_of_base64_and_carries_no_meaning_for_this_test=="
}
],
"role": "model"
},
"index": 0
}
],
"usageMetadata": {
"promptTokenCount": 24512,
"candidatesTokenCount": 167,
"totalTokenCount": 25738,
"cachedContentTokenCount": 19504,
"promptTokensDetails": [{ "modality": "TEXT", "tokenCount": 24512 }],
"thoughtsTokenCount": 1059,
"serviceTier": "standard"
},
"modelVersion": "gemini-3.1-flash-lite",
"responseId": "resp-fixture-malformed-001"
},
{
"candidates": [
{
"content": { "parts": [{ "text": "" }], "role": "model" },
"finishReason": "MALFORMED_FUNCTION_CALL",
"index": 0,
"finishMessage": "Malformed function call: call:default_api:exec{command: cd /home/user/workspace && venv/bin/python run_search.py , web_fetch \"https://site-a.example/jobs\", web_fetch \"https://site-b.example/careers\", web_fetch \"https://site-c.example/jobs\", web_fetch \"https://site-d.example/jobs\", web_fetch \"https://site-e.example/careers\" ,reason: Executing the search workflow: running the scraper and fetching the target sites. }"
}
],
"usageMetadata": {
"promptTokenCount": 24512,
"candidatesTokenCount": 167,
"totalTokenCount": 25738,
"cachedContentTokenCount": 19504,
"promptTokensDetails": [{ "modality": "TEXT", "tokenCount": 24512 }],
"thoughtsTokenCount": 1059,
"serviceTier": "standard"
},
"modelVersion": "gemini-3.1-flash-lite",
"responseId": "resp-fixture-malformed-001"
}
]
}

282
tests/integration/gemini-malformed-function-call-stream.test.ts Normal file
View File

@@ -0,0 +1,282 @@
/**
* tests/integration/gemini-malformed-function-call-stream.test.ts
*
* Chains the real Gemini -> OpenAI translator (open-sse/translator/response/
* gemini-to-openai.ts) with the real Responses API transformer (open-sse/
* transformer/responsesTransformer.ts) against the exact (sanitized) event
* series captured from a live incident — see tests/fixtures/translation/
* gemini-malformed-function-call-stream.json for the full sanitized fixture
* and incident description (dashboard log id 1784489701456-d8c0e9).
*
* Gemini's own parser rejected an attempted tool call mid-turn and terminated
* the stream with finishReason: MALFORMED_FUNCTION_CALL — a value that isn't
* one of OpenAI's 5 documented finish_reason values (stop, length, tool_calls,
* content_filter, function_call). Passed through raw, a real OpenAI-format
* client (OpenClaw) has no handling for it at all and silently never notices
* the turn failed. The fix (open-sse/translator/response/gemini-to-openai.ts)
* synthesizes a tool_calls entry and finish_reason: "tool_calls" instead,
* routing the failure into the ordinary "tool call arguments didn't parse"
* path every OpenAI-compatible agent loop already handles.
*
* This test proves the fix holds for BOTH client-facing surfaces that share
* this same translator output: Chat Completions (consumes the translator's
* chunks directly) and the Responses API (consumes the same chunks through
* the transformer chain) — using the identical real event series for both,
* so a regression in either surface is caught against the same ground truth.
*/
import test from "node:test";
import assert from "node:assert/strict";
import fs from "node:fs";
import path from "node:path";
import { fileURLToPath } from "node:url";
const { geminiToOpenAIResponse } =
await import("../../open-sse/translator/response/gemini-to-openai.ts");
const { createResponsesApiTransformStream } =
await import("../../open-sse/transformer/responsesTransformer.ts");
const FIXTURES_DIR = path.resolve(
path.dirname(fileURLToPath(import.meta.url)),
"../fixtures/translation"
);
const FIXTURE_PATH = path.join(FIXTURES_DIR, "gemini-malformed-function-call-stream.json");
// Follow-up live incident (log id 1784589106014-2a42f8): a REAL functionCall and a
// MALFORMED_FUNCTION_CALL finish arriving in the SAME turn — see the fixture's own
// "description" field for the full incident writeup.
const PARALLEL_FIXTURE_PATH = path.join(
FIXTURES_DIR,
"gemini-malformed-function-call-parallel-real-call-stream.json"
);
type GeminiChunk = Record<string, unknown>;
type OpenAIChunk = {
choices: Array<{
delta: {
reasoning_content?: string;
tool_calls?: Array<{
id: string;
type: string;
function: { name: string; arguments: string };
}>;
};
finish_reason: string | null;
}>;
usage?: { prompt_tokens: number; completion_tokens: number; total_tokens: number };
};
type ResponsesFunctionCallItem = {
type: "function_call";
call_id: string;
name: string;
arguments: string;
};
type ResponsesEventData = { type?: string; item?: ResponsesFunctionCallItem };
function asResponsesEventData(data: unknown): ResponsesEventData | null {
return data && typeof data === "object" ? (data as ResponsesEventData) : null;
}
function loadFixtureChunks(fixturePath: string): GeminiChunk[] {
const fixture = JSON.parse(fs.readFileSync(fixturePath, "utf8"));
assert.ok(Array.isArray(fixture.chunks) && fixture.chunks.length > 0, "fixture must have chunks");
return fixture.chunks;
}
/** Real Gemini -> OpenAI translation, exactly as the live streaming pipeline runs it. */
function translateFixtureToOpenAIChunks(fixturePath: string = FIXTURE_PATH): OpenAIChunk[] {
const state: Record<string, unknown> = {
functionIndex: 0,
toolCalls: new Map(),
messageId: "fixture-msg",
model: "gemini-3.1-flash-lite",
};
const openaiChunks: OpenAIChunk[] = [];
for (const chunk of loadFixtureChunks(fixturePath)) {
const results = geminiToOpenAIResponse(chunk, state);
if (results) openaiChunks.push(...(results as OpenAIChunk[]));
}
return openaiChunks;
}
/** Feeds already-translated OpenAI chat chunks through the real Responses API transformer. */
async function transformToResponsesEvents(
openaiChunks: OpenAIChunk[]
): Promise<Array<{ event: string | null; data: unknown }>> {
const stream = createResponsesApiTransformStream();
const writer = stream.writable.getWriter();
const reader = stream.readable.getReader();
const encoder = new TextEncoder();
const decoder = new TextDecoder();
const output: string[] = [];
const readerTask = (async () => {
while (true) {
const { value, done } = await reader.read();
if (done) break;
output.push(decoder.decode(value));
}
})();
for (const chunk of openaiChunks) {
await writer.write(encoder.encode(`data: ${JSON.stringify(chunk)}\n\n`));
}
await writer.close();
await readerTask;
return output
.join("")
.trim()
.split("\n\n")
.filter(Boolean)
.map((entry) => {
const lines = entry.split("\n");
const eventLine = lines.find((l) => l.startsWith("event: "));
const dataLine = lines.find((l) => l.startsWith("data: "));
const rawData = dataLine ? dataLine.slice("data: ".length) : null;
return {
event: eventLine ? eventLine.slice("event: ".length) : null,
data: rawData && rawData !== "[DONE]" ? JSON.parse(rawData) : rawData,
};
});
}
// ── Chat Completions surface ────────────────────────────────────────────────
test("Chat Completions: malformed-tool-call stream ends in a client-usable tool_calls, not a raw error string", () => {
const openaiChunks = translateFixtureToOpenAIChunks();
assert.ok(openaiChunks.length > 0, "expected at least one translated chunk");
// The reasoning ("thought") content from the live incident's planning steps
// must still come through untouched — this fix only changes the terminal
// finish_reason/tool_calls handling, not the reasoning stream.
const reasoningDeltas = openaiChunks
.map((c) => c.choices?.[0]?.delta?.reasoning_content)
.filter((v): v is string => typeof v === "string" && v.length > 0);
assert.ok(reasoningDeltas.length >= 3, "expected the 3 planning/thought deltas to survive");
assert.match(reasoningDeltas[0], /Initiating the Workflow/);
const toolCallChunk = openaiChunks.find((c) => c.choices?.[0]?.delta?.tool_calls);
assert.ok(toolCallChunk, "expected a synthesized tool_calls delta chunk");
const toolCall = toolCallChunk!.choices[0].delta.tool_calls[0];
assert.equal(toolCall.type, "function");
assert.equal(typeof toolCall.id, "string");
const parsedArgs = JSON.parse(toolCall.function.arguments);
assert.equal(parsedArgs.error, "MALFORMED_FUNCTION_CALL");
assert.match(parsedArgs.message, /Malformed function call/);
// Sanitized fixture content, not the real incident's personal paths/URLs.
assert.doesNotMatch(parsedArgs.message, /\/home\/ping\//);
assert.doesNotMatch(parsedArgs.message, /academicwork|senterprise|hiq\.se/);
const finalChunk = openaiChunks.at(-1)!;
assert.equal(
finalChunk.choices[0].finish_reason,
"tool_calls",
"finish_reason must be the standard OpenAI value a real client can act on"
);
// Never leak the raw Gemini string as the terminal signal.
assert.notEqual(finalChunk.choices[0].finish_reason, "malformed_function_call");
// Usage from the fixture's final chunk must still flow through untouched.
assert.equal(finalChunk.usage?.prompt_tokens, 24512);
assert.ok((finalChunk.usage?.completion_tokens ?? 0) > 0);
});
// ── Responses API surface (same fixture, chained through the transformer) ──
test("Responses API: malformed-tool-call stream ends in a real function_call output item, not response.failed", async () => {
const openaiChunks = translateFixtureToOpenAIChunks();
const events = await transformToResponsesEvents(openaiChunks);
const eventTypes = events.map((e) => e.event).filter(Boolean);
assert.ok(eventTypes.includes("response.output_item.added"), "expected an output item to open");
const functionCallDone = events.find(
(e) =>
e.event === "response.output_item.done" &&
asResponsesEventData(e.data)?.item?.type === "function_call"
);
assert.ok(functionCallDone, "expected a completed function_call output item");
const funcItem = asResponsesEventData(functionCallDone!.data)!.item!;
assert.equal(typeof funcItem.call_id, "string");
const parsedArgs = JSON.parse(funcItem.arguments);
assert.equal(parsedArgs.error, "MALFORMED_FUNCTION_CALL");
const completed = events.find((e) => e.event === "response.completed");
assert.ok(completed, "expected a normal response.completed terminal event, not an error");
assert.notEqual(asResponsesEventData(completed!.data)?.type, "response.failed");
// The transformer must never surface this as an in-band error event either —
// that would be the same class of "client sees nothing actionable" failure
// this whole fix exists to prevent, just on the Responses API side.
assert.ok(
!eventTypes.includes("error") && !eventTypes.includes("response.failed"),
`expected no error-shaped event, got: ${eventTypes.join(", ")}`
);
const doneMarker = events.at(-1);
assert.equal(doneMarker?.data, "[DONE]");
});
test("both surfaces agree: exactly one synthesized tool call, sourced from the same translated chunks", async () => {
const openaiChunks = translateFixtureToOpenAIChunks();
const toolCallChunks = openaiChunks.filter((c) => c.choices?.[0]?.delta?.tool_calls);
assert.equal(toolCallChunks.length, 1, "expected exactly one synthesized tool_calls chunk");
const events = await transformToResponsesEvents(openaiChunks);
const functionCallItems = events.filter(
(e) =>
e.event === "response.output_item.done" &&
asResponsesEventData(e.data)?.item?.type === "function_call"
);
assert.equal(functionCallItems.length, 1, "expected exactly one function_call output item");
});
// ── Second fixture: a REAL tool call arriving in the SAME turn as a malformed one ──
//
// See tests/fixtures/translation/gemini-malformed-function-call-parallel-real-call-stream.json
// for the full incident writeup (log id 1784589106014-2a42f8). This is the case the
// first version of the fix got wrong: it skipped synthesizing anything whenever a real
// tool call already existed, silently discarding the malformed attempt's information.
// Both the real call and the synthesized failure must reach the client.
test("Chat Completions: a real tool call alongside a malformed one keeps BOTH — the failure is not silently dropped", () => {
const openaiChunks = translateFixtureToOpenAIChunks(PARALLEL_FIXTURE_PATH);
const toolCallChunks = openaiChunks.filter((c) => c.choices?.[0]?.delta?.tool_calls);
assert.equal(toolCallChunks.length, 2, "expected the real call AND the synthesized failure");
const realCall = toolCallChunks[0].choices[0].delta.tool_calls![0];
assert.equal(realCall.function.name, "check_status");
const syntheticCall = toolCallChunks[1].choices[0].delta.tool_calls![0];
assert.equal(syntheticCall.function.name, "malformed_tool_call");
const parsedArgs = JSON.parse(syntheticCall.function.arguments);
assert.equal(parsedArgs.error, "MALFORMED_FUNCTION_CALL");
assert.match(parsedArgs.message, /exec/);
const finalChunk = openaiChunks.at(-1)!;
assert.equal(finalChunk.choices[0].finish_reason, "tool_calls");
});
test("Responses API: a real tool call alongside a malformed one produces TWO function_call output items", async () => {
const openaiChunks = translateFixtureToOpenAIChunks(PARALLEL_FIXTURE_PATH);
const events = await transformToResponsesEvents(openaiChunks);
const functionCallItems = events
.filter(
(e) =>
e.event === "response.output_item.done" &&
asResponsesEventData(e.data)?.item?.type === "function_call"
)
.map((e) => asResponsesEventData(e.data)!.item!);
assert.equal(
functionCallItems.length,
2,
"expected both the real and synthesized function_call items"
);
assert.ok(functionCallItems.some((item) => item.name === "check_status"));
assert.ok(functionCallItems.some((item) => item.name === "malformed_tool_call"));
const completed = events.find((e) => e.event === "response.completed");
assert.ok(completed, "expected a normal response.completed terminal event, not an error");
});

175
tests/unit/gemini-malformed-function-call-finish-reason-2462.test.ts
View File

@@ -1,6 +1,19 @@
import test from "node:test";
import assert from "node:assert/strict";
type OpenAIToolCallChunk = {
choices: Array<{
delta: {
tool_calls?: Array<{
id: string;
type: string;
function: { name: string; arguments: string };
}>;
};
finish_reason: string | null;
}>;
};
// Upstream: decolua/9router#2462 sub-bug #2 (@anhdiepmmk).
//
// Gemini/Antigravity aborts a turn mid tool-call with finishReason
@@ -18,15 +31,12 @@ import assert from "node:assert/strict";
// abort/error finish reasons, while a genuine clean STOP still maps to
// "end_turn" (no regression).
const { geminiToOpenAIResponse } = await import(
"../../open-sse/translator/response/gemini-to-openai.ts"
);
const { openaiToClaudeResponse } = await import(
"../../open-sse/translator/response/openai-to-claude.ts"
);
const { geminiToClaudeResponse } = await import(
"../../open-sse/translator/response/gemini-to-claude.ts"
);
const { geminiToOpenAIResponse } =
await import("../../open-sse/translator/response/gemini-to-openai.ts");
const { openaiToClaudeResponse } =
await import("../../open-sse/translator/response/openai-to-claude.ts");
const { geminiToClaudeResponse } =
await import("../../open-sse/translator/response/gemini-to-claude.ts");
// Direct Gemini -> Claude translator (the path Claude Code hits through an
// antigravity/Gemini-routed model — sourceFormat=CLAUDE, targetFormat=GEMINI —
@@ -105,6 +115,153 @@ test("Gemini UNEXPECTED_TOOL_CALL does not surface as a clean Claude end_turn",
assert.notEqual(stopReason, "end_turn");
});
// ─── OpenAI hub follow-up (live incident, dashboard log id 1784489701456-d8c0e9) ──
//
// The tests above only assert the raw "malformed_function_call" string doesn't get
// misread by the DOWNSTREAM Claude translation step. But passing that raw,
// non-standard string through as finish_reason to a real OpenAI-format client is
// itself the bug: it's not one of OpenAI's 5 documented finish_reason values (stop,
// length, tool_calls, content_filter, function_call), so a client like OpenClaw has
// no handling for it at all and silently never notices the turn failed — worse than
// "honest but unrecognized," it's invisible. These tests assert the OpenAI hub now
// synthesizes a real tool_calls entry and finish_reason: "tool_calls" instead, so the
// failure routes into the ordinary "tool call arguments didn't parse" path every
// OpenAI-compatible agent loop already has to handle.
function runGeminiToOpenAI(geminiChunk: unknown) {
const state: Record<string, unknown> = {
functionIndex: 0,
toolCalls: new Map(),
messageId: "test-msg",
model: "gemini-test",
};
return geminiToOpenAIResponse(geminiChunk, state) || [];
}
test("Gemini MALFORMED_FUNCTION_CALL synthesizes a tool_calls entry with finish_reason tool_calls", () => {
const events = runGeminiToOpenAI({
responseId: "resp-malformed",
modelVersion: "gemini-2.5-pro",
candidates: [
{
content: { parts: [{ text: "" }] },
finishReason: "MALFORMED_FUNCTION_CALL",
finishMessage: "Malformed function call: call:default_api:exec{command: ...}",
index: 0,
},
],
}) as OpenAIToolCallChunk[];
const toolCallChunk = events.find((e) => e.choices?.[0]?.delta?.tool_calls);
assert.ok(toolCallChunk, "expected a chunk carrying a synthesized tool_calls delta");
const toolCall = toolCallChunk.choices[0].delta.tool_calls[0];
assert.equal(toolCall.type, "function");
assert.ok(toolCall.function?.name, "synthesized tool call must have a name");
// Arguments must be valid, parseable JSON (that's the whole point — it routes into
// the client's normal "tool args didn't parse *cleanly*" error path, not a crash).
const parsedArgs = JSON.parse(toolCall.function.arguments);
assert.equal(parsedArgs.error, "MALFORMED_FUNCTION_CALL");
assert.match(parsedArgs.message, /Malformed function call/);
const finalChunk = events.at(-1);
assert.equal(
finalChunk?.choices?.[0]?.finish_reason,
"tool_calls",
"finish_reason must be the standard OpenAI value, not the raw Gemini string"
);
});
test("Gemini UNEXPECTED_TOOL_CALL also synthesizes a tool_calls entry", () => {
const events = runGeminiToOpenAI({
responseId: "resp-unexpected",
modelVersion: "gemini-2.5-pro",
candidates: [
{
content: { parts: [{ text: "" }] },
finishReason: "UNEXPECTED_TOOL_CALL",
index: 0,
},
],
}) as OpenAIToolCallChunk[];
const finalChunk = events.at(-1);
assert.equal(finalChunk?.choices?.[0]?.finish_reason, "tool_calls");
});
// Follow-up live incident (log id 1784589106014-2a42f8): Gemini can emit a REAL,
// valid functionCall AND finish the SAME candidate with MALFORMED_FUNCTION_CALL —
// the model attempted multiple tool calls in one turn (here: a real "openclaw" call
// plus a malformed "exec"+"cron" multi-call attempt), one parsed cleanly and the
// other didn't. An earlier version of this fix skipped synthesis whenever a real
// tool call already existed, on the assumption that meant a LATER, separate retry —
// but that's indistinguishable from this same-turn case, and skipping silently
// discarded the "exec"/"cron" failure entirely: the client saw "openclaw" succeed
// and never learned the other calls were attempted and rejected.
test("a REAL tool call alongside MALFORMED_FUNCTION_CALL in the same turn keeps BOTH — the failure is never silently dropped", () => {
const state: Record<string, unknown> = {
functionIndex: 0,
toolCalls: new Map(),
messageId: "test-msg-2",
model: "gemini-test",
};
geminiToOpenAIResponse(
{
responseId: "resp-real-then-malformed",
modelVersion: "gemini-2.5-pro",
candidates: [
{
content: { parts: [{ functionCall: { name: "read_file", args: { path: "a.txt" } } }] },
index: 0,
},
],
},
state
);
const finalEvents = geminiToOpenAIResponse(
{
responseId: "resp-real-then-malformed",
modelVersion: "gemini-2.5-pro",
candidates: [
{
content: { parts: [{ text: "" }] },
finishReason: "MALFORMED_FUNCTION_CALL",
finishMessage: "Malformed function call: call:default_api:exec{command: df -h}",
index: 0,
},
],
},
state
) as OpenAIToolCallChunk[];
const synthesizedChunk = finalEvents.find((e) => e.choices?.[0]?.delta?.tool_calls);
assert.ok(synthesizedChunk, "expected a second, synthesized tool_calls delta");
const synthesizedCall = synthesizedChunk!.choices[0].delta.tool_calls![0];
assert.equal(synthesizedCall.function.name, "malformed_tool_call");
assert.match(JSON.parse(synthesizedCall.function.arguments).message, /exec/);
const finalChunk = finalEvents.at(-1)!;
assert.equal(finalChunk.choices[0].finish_reason, "tool_calls");
assert.equal(
(state.toolCalls as Map<number, unknown>).size,
2,
"both the real read_file call AND the synthesized malformed-call entry must be present"
);
});
test("Gemini clean STOP with no tool calls is unaffected (no regression)", () => {
const events = runGeminiToOpenAI({
responseId: "resp-clean",
modelVersion: "gemini-2.5-pro",
candidates: [{ content: { parts: [{ text: "All done." }] }, finishReason: "STOP", index: 0 }],
}) as OpenAIToolCallChunk[];
const finalChunk = events.at(-1);
assert.equal(finalChunk?.choices?.[0]?.finish_reason, "stop");
assert.equal(finalChunk?.choices?.[0]?.delta?.tool_calls, undefined);
});
test("Gemini clean STOP still maps to Claude end_turn (no regression)", () => {
const { claudeEvents } = runGeminiToClaude({
responseId: "resp-clean",

116
tests/unit/gemini-tool-choice.test.ts Normal file
View File

@@ -0,0 +1,116 @@
import test from "node:test";
import assert from "node:assert/strict";
// tool_choice was never read anywhere in the OpenAI -> Gemini request translator —
// result.toolConfig was unconditionally hardcoded to { mode: "VALIDATED" } whenever
// tools were present, regardless of what the caller sent. VALIDATED allows the model
// to respond with plain text OR a (schema-validated) function call at its own
// discretion; it never FORCES a call the way OpenAI's tool_choice: "required" (Gemini's
// ANY mode) does. Investigating a live report that gemini-3.1-flash-lite frequently
// narrates an intended tool call in plain text instead of emitting one (dashboard log
// id 1784591483850-49c408, zero functionCall parts, finishReason STOP, nothing but
// prose) surfaced that tool_choice: "required" had no way to reach Gemini at all —
// this is the fix, and the mechanism the live test compares against a baseline.
const { openaiToGeminiRequest, openaiToCloudCodeGeminiRequest } =
await import("../../open-sse/translator/request/openai-to-gemini.ts");
type ToolConfigResult = {
toolConfig?: { functionCallingConfig: { mode: string; allowedFunctionNames?: string[] } };
};
const SAMPLE_TOOLS = [
{
type: "function",
function: { name: "run_command", description: "Run a shell command", parameters: {} },
},
];
function baseBody(toolChoice?: unknown) {
return {
messages: [{ role: "user", content: "hi" }],
tools: SAMPLE_TOOLS,
...(toolChoice !== undefined ? { tool_choice: toolChoice } : {}),
};
}
test("no tool_choice (unset) keeps the existing VALIDATED default — no behavior change", () => {
const result = openaiToGeminiRequest("gemini-2.5-pro", baseBody(), false) as ToolConfigResult;
assert.equal(result.toolConfig?.functionCallingConfig.mode, "VALIDATED");
});
test('tool_choice: "auto" maps to VALIDATED (same as unset)', () => {
const result = openaiToGeminiRequest(
"gemini-2.5-pro",
baseBody("auto"),
false
) as ToolConfigResult;
assert.equal(result.toolConfig?.functionCallingConfig.mode, "VALIDATED");
});
test('tool_choice: "required" maps to Gemini ANY mode (forces a function call)', () => {
const result = openaiToGeminiRequest(
"gemini-2.5-pro",
baseBody("required"),
false
) as ToolConfigResult;
assert.equal(result.toolConfig?.functionCallingConfig.mode, "ANY");
});
test('tool_choice: "any" (OpenAI-compatible alias) also maps to ANY', () => {
const result = openaiToGeminiRequest(
"gemini-2.5-pro",
baseBody("any"),
false
) as ToolConfigResult;
assert.equal(result.toolConfig?.functionCallingConfig.mode, "ANY");
});
test('tool_choice: "none" maps to Gemini NONE mode (disables function calling)', () => {
const result = openaiToGeminiRequest(
"gemini-2.5-pro",
baseBody("none"),
false
) as ToolConfigResult;
assert.equal(result.toolConfig?.functionCallingConfig.mode, "NONE");
});
test("tool_choice forcing a specific function maps to ANY with allowedFunctionNames", () => {
const result = openaiToGeminiRequest(
"gemini-2.5-pro",
baseBody({ type: "function", function: { name: "run_command" } }),
false
) as ToolConfigResult;
assert.equal(result.toolConfig?.functionCallingConfig.mode, "ANY");
assert.deepEqual(result.toolConfig?.functionCallingConfig.allowedFunctionNames, ["run_command"]);
});
test("no tools present: toolConfig is not set regardless of tool_choice", () => {
const result = openaiToGeminiRequest(
"gemini-2.5-pro",
{ messages: [{ role: "user", content: "hi" }], tool_choice: "required" },
false
) as ToolConfigResult;
assert.equal(result.toolConfig, undefined);
});
// Antigravity / Cloud Code envelope path (wrapInCloudCodeEnvelope) previously
// re-derived its own hardcoded VALIDATED independently of the base translator —
// it now reuses whatever openaiToGeminiBase already computed from tool_choice.
test("Antigravity/Cloud Code path also honors tool_choice: required", () => {
const result = openaiToCloudCodeGeminiRequest(
"gemini-2.5-pro",
baseBody("required"),
false
) as ToolConfigResult;
assert.equal(result.toolConfig?.functionCallingConfig.mode, "ANY");
});
test("Antigravity/Cloud Code path defaults to VALIDATED when tool_choice is unset (no regression)", () => {
const result = openaiToCloudCodeGeminiRequest(
"gemini-2.5-pro",
baseBody(),
false
) as ToolConfigResult;
assert.equal(result.toolConfig?.functionCallingConfig.mode, "VALIDATED");
});