feat(providers): Gemini tool-calling end-to-end on /v1beta (#6222) (#6394)

Gemini tool-calling end-to-end on /v1beta (#6222) (net +1/-0, tests OK). Integrated into release/v3.8.46.
This commit is contained in:
Diego Rodrigues de Sa e Souza
2026-07-06 19:31:12 -03:00
committed by GitHub
parent f5147d00f9
commit 18fa0b2651
5 changed files with 595 additions and 42 deletions

View File

@@ -8,6 +8,7 @@
### ✨ New Features
- **feat(providers):** end-to-end **tool/function calling on the native Gemini `/v1beta` endpoint** ([#6222](https://github.com/diegosouzapw/OmniRoute/issues/6222)) — both directions of the Gemini↔OpenAI conversion now preserve tool data (previously silently dropped). Request side: `convertGeminiToInternal` (extracted to its own testable module) maps `tools[].functionDeclarations` → OpenAI `tools`, prior `functionCall` parts → assistant `tool_calls`, and `functionResponse` parts → `tool`-role messages. Response side: `convertOpenAIResponseToGemini` emits `parts[].functionCall {name,args}` from `message.tool_calls`, and the streaming `openAIChunkToGeminiChunk` accumulates fragmented `tool_calls` deltas by index into complete `functionCall` parts. The non-Gemini client paths (Claude, OpenAI-Responses) already preserved tool calls — this closes the gap specific to the native Gemini surface. Regression guard: `tests/unit/v1beta-gemini-tool-calling-6222.test.ts` (6, incl. a streaming SSE round-trip).
- **feat(providers):** copilot-m365-web **enterprise / work tier** support ([#6334](https://github.com/diegosouzapw/OmniRoute/issues/6334)) — mirrors the EDU-tier pattern (#6210): `M365ConnectionParams` gains an `agent` field, a new opt-in `M365_ENTERPRISE_OVERRIDES` preset (`agent=work`, `scenario=officeweb`, `licenseType=Premium`) applies via `providerSpecificData.tier="enterprise"` (alias `"work"`), and `agent` is also overridable directly via `providerSpecificData.agent`. `buildWsUrl` was hardcoding `agent="web"` (the one enterprise-distinguishing param with no override path), so a Premium work account handshook then returned an empty stream. The individual and EDU paths are untouched. Kilo's dup flag vs #6210 (EDU tier) was a false positive — different tier. Regression guard: `tests/unit/copilot-m365-enterprise-6334.test.ts` (7). End-to-end confirmation on a real Premium work account is a live-VPS validation follow-up (Hard Rule #18). (thanks @Forcerecon)
- **feat(api):** standardized, provider-agnostic **`effort` + `thinking` request params** ([#6241](https://github.com/diegosouzapw/OmniRoute/issues/6241)) — a thin standardization layer over the existing mature per-provider reasoning plumbing (no provider mapper touched). `providerChatCompletionSchema` gains a canonical `effort` (reusing the shared `none/low/medium/high/xhigh` vocabulary — the UI tiers `extra`/`max` collapse onto `xhigh`) and a boolean `thinking`. A pure `normalizeReasoningRequest` (wired once in `src/sse/handlers/chat.ts`, before any reasoning field is read) folds them onto the fields the translators already consume (`reasoning_effort` / `reasoning.effort` / `thinking`), so they fan out to Anthropic / Gemini / xAI / Responses — an explicit client `reasoning_effort` / object-shaped `thinking` always wins (backward-compatible). `/models` additively exposes `supportsThinking` + `effort_tiers` so the frontend can render the toggles (UI component is a follow-up). Regression guard: `tests/unit/effort-thinking-standardization-6241.test.ts` (12). (thanks @Iammilansoni, @shabeer)
- **feat(combo):** new **`pipeline` (sequential) combo strategy** ([#6297](https://github.com/diegosouzapw/OmniRoute/issues/6297)) — the 18th routing strategy runs targets **in order**, threading each step's output into the next step's input, with an optional per-step `prompt` (system instruction); only the final step's response is returned. Distinct from `fusion` (parallel fan-out + judge). Implemented as a self-contained `open-sse/services/pipeline.ts` (sibling to `fusion.ts`), dispatched from `combo.ts`; the step list reuses `combo.models` order and reads an optional `prompt` off each target (backward-compatible — ignored by every other strategy). Intermediate steps run non-streaming with tools stripped (complete prose to thread forward); the final step keeps the client's `stream` flag + tools. A failing/empty/unparseable intermediate step fails the whole pipeline explicitly via a sanitized error (never silently swallowed). Kilo's dup flag vs #563 was a false positive (that's model→chain selection; this is a sequential chain). Regression guard: `tests/unit/combo-pipeline-strategy.test.ts` (5). (thanks @ofekbetzalel)

View File

@@ -37,10 +37,18 @@ export const OPENAI_TO_GEMINI_FINISH_REASON: Record<string, string> = {
content_filter: "SAFETY",
};
interface OpenAIToolCallDelta {
index?: number;
id?: string;
type?: string;
function?: { name?: string; arguments?: string };
}
interface OpenAIChoiceDelta {
content?: string | null;
reasoning_content?: string | null;
role?: string;
tool_calls?: OpenAIToolCallDelta[];
}
interface OpenAIChoice {
@@ -63,9 +71,31 @@ interface OpenAIStreamChunk {
model?: string;
}
interface GeminiFunctionCall {
name: string;
args: Record<string, unknown>;
}
interface GeminiFunctionResponse {
name: string;
response: Record<string, unknown>;
}
interface GeminiPart {
text: string;
text?: string;
thought?: boolean;
functionCall?: GeminiFunctionCall;
functionResponse?: GeminiFunctionResponse;
}
/**
* Per-stream mutable accumulator for OpenAI streamed tool calls. OpenAI emits
* a tool call's `arguments` as partial JSON fragments across several delta
* chunks, keyed by `index`; we buffer them here and flush a complete
* `functionCall` part once `finish_reason` arrives.
*/
export interface GeminiToolCallState {
toolCallAccum?: Record<number, { id: string; name: string; arguments: string }>;
}
interface GeminiCandidate {
@@ -97,7 +127,8 @@ interface GeminiStreamChunk {
*/
export function openAIChunkToGeminiChunk(
parsed: OpenAIStreamChunk,
fallbackModel: string
fallbackModel: string,
state?: GeminiToolCallState
): GeminiStreamChunk | null {
const choice = parsed.choices?.[0];
if (!choice) return null;
@@ -112,6 +143,35 @@ export function openAIChunkToGeminiChunk(
parts.push({ text: String(delta.content) });
}
// Accumulate streamed tool-call fragments (OpenAI streams partial JSON
// `arguments` across chunks, keyed by index). Requires a caller-supplied
// per-stream `state`; nothing is emitted until finish_reason arrives.
if (state && Array.isArray(delta.tool_calls)) {
const accum = (state.toolCallAccum ??= {});
for (const tc of delta.tool_calls) {
const idx = tc.index ?? 0;
const entry = (accum[idx] ??= { id: "", name: "", arguments: "" });
if (tc.id) entry.id = tc.id;
if (tc.function?.name) entry.name += tc.function.name;
if (tc.function?.arguments) entry.arguments += tc.function.arguments;
}
}
// On finish, flush accumulated tool calls as complete functionCall parts.
if (choice.finish_reason && state?.toolCallAccum) {
for (const key of Object.keys(state.toolCallAccum)) {
const entry = state.toolCallAccum[Number(key)];
if (!entry.name) continue;
let args: Record<string, unknown> = {};
try {
args = JSON.parse(entry.arguments || "{}") as Record<string, unknown>;
} catch {
args = {};
}
parts.push({ functionCall: { name: entry.name, args } });
}
}
// Skip pure role-only deltas with no content and no finish signal.
if (parts.length === 0 && !choice.finish_reason) return null;
@@ -168,6 +228,9 @@ export function transformOpenAISSEToGeminiSSE(upstreamResponse: Response, model:
// chunk so we never JSON.parse a half-event.
let buffer = "";
// Per-stream tool-call accumulator (shared across transform + flush).
const toolCallState: GeminiToolCallState = {};
const transform = new TransformStream<Uint8Array, Uint8Array>({
transform(chunk, controller) {
buffer += decoder.decode(chunk, { stream: true });
@@ -192,7 +255,7 @@ export function transformOpenAISSEToGeminiSSE(upstreamResponse: Response, model:
continue;
}
const geminiChunk = openAIChunkToGeminiChunk(parsed, model);
const geminiChunk = openAIChunkToGeminiChunk(parsed, model, toolCallState);
if (!geminiChunk) continue;
controller.enqueue(encoder.encode("data: " + JSON.stringify(geminiChunk) + "\r\n\r\n"));
@@ -212,7 +275,7 @@ export function transformOpenAISSEToGeminiSSE(upstreamResponse: Response, model:
} catch {
return;
}
const geminiChunk = openAIChunkToGeminiChunk(parsed, model);
const geminiChunk = openAIChunkToGeminiChunk(parsed, model, toolCallState);
if (!geminiChunk) return;
controller.enqueue(encoder.encode("data: " + JSON.stringify(geminiChunk) + "\r\n\r\n"));
},
@@ -228,10 +291,17 @@ export function transformOpenAISSEToGeminiSSE(upstreamResponse: Response, model:
});
}
interface OpenAIToolCall {
id?: string;
type?: string;
function?: { name?: string; arguments?: string };
}
interface OpenAIMessage {
content?: string | null;
reasoning_content?: string | null;
role?: string;
tool_calls?: OpenAIToolCall[];
}
interface OpenAINonStreamChoice {
@@ -311,7 +381,22 @@ export async function convertOpenAIResponseToGemini(
if (message.reasoning_content) {
parts.push({ text: String(message.reasoning_content), thought: true });
}
parts.push({ text: String(message.content ?? "") });
// Text content — only emit a text part when there is actual text, so a
// pure tool-call response isn't padded with an empty-string part.
const toolCalls = Array.isArray(message.tool_calls) ? message.tool_calls : [];
if (message.content || toolCalls.length === 0) {
parts.push({ text: String(message.content ?? "") });
}
// Tool calls → functionCall parts (arguments JSON string → object).
for (const tc of toolCalls) {
let args: Record<string, unknown> = {};
try {
args = JSON.parse(tc.function?.arguments || "{}") as Record<string, unknown>;
} catch {
args = {};
}
parts.push({ functionCall: { name: tc.function?.name || "", args } });
}
const finishReason = OPENAI_TO_GEMINI_FINISH_REASON[finish_reason ?? "stop"] ?? "STOP";

View File

@@ -0,0 +1,211 @@
/**
* Convert a native Gemini `generateContent` request body into the internal
* OpenAI Chat Completions shape consumed by `handleChat`.
*
* Extracted from the route handler so the (pure) conversion can be unit-tested
* without importing the full chat-handler graph (which keeps timers alive and
* hangs the node:test runner). See feature #6222.
*
* Tool/function calling is preserved in the request direction:
* - `tools[].functionDeclarations` → OpenAI `tools[{type:"function",...}]`
* - prior `functionCall` parts → assistant `tool_calls`
* - `functionResponse` parts → `tool`-role messages
*
* Mirrors the shapes already used by the request translator
* `open-sse/translator/request/gemini-to-openai.ts`.
*/
interface GeminiFunctionCall {
name?: string;
args?: Record<string, unknown>;
id?: string;
}
interface GeminiFunctionResponse {
name?: string;
id?: string;
response?: { result?: unknown } & Record<string, unknown>;
}
interface GeminiPart {
text?: string;
functionCall?: GeminiFunctionCall;
functionResponse?: GeminiFunctionResponse;
[key: string]: unknown;
}
interface GeminiContent {
role?: string;
parts?: GeminiPart[];
}
interface GeminiTool {
functionDeclarations?: Array<{
name?: string;
description?: string;
parameters?: unknown;
}>;
}
interface GeminiGenerateBody {
systemInstruction?: { parts?: GeminiPart[] };
contents?: GeminiContent[];
tools?: GeminiTool[];
generationConfig?: {
maxOutputTokens?: number;
temperature?: number;
topP?: number;
};
}
interface InternalMessage {
role: string;
content?: string | null;
tool_calls?: Array<{
id: string;
type: "function";
function: { name: string; arguments: string };
}>;
tool_call_id?: string;
}
interface InternalTool {
type: "function";
function: { name: string; description: string; parameters: unknown };
}
export interface InternalChatBody {
model: string;
messages: InternalMessage[];
stream: boolean;
max_tokens?: number;
temperature?: number;
top_p?: number;
tools?: InternalTool[];
}
let toolCallSeq = 0;
function newToolCallId(): string {
toolCallSeq += 1;
return `call_${Date.now()}_${toolCallSeq}_${Math.random().toString(36).slice(2, 8)}`;
}
/**
* Convert a single Gemini `content` entry into one internal message.
*
* `functionResponse` parts become a `tool` message; `functionCall` parts become
* an assistant message carrying `tool_calls`; otherwise a plain text message.
* Returns `null` when the content has nothing to contribute.
*/
function convertContent(content: GeminiContent): InternalMessage | null {
const parts = content.parts;
if (!parts || !Array.isArray(parts)) return null;
// A functionResponse turn maps to a `tool` role message.
for (const part of parts) {
if (part.functionResponse) {
const fr = part.functionResponse;
const payload =
fr.response && "result" in fr.response ? fr.response.result : fr.response ?? {};
return {
role: "tool",
tool_call_id: fr.id || fr.name || "",
content: JSON.stringify(payload ?? {}),
};
}
}
const textSegments: string[] = [];
const toolCalls: InternalMessage["tool_calls"] = [];
for (const part of parts) {
if (typeof part.text === "string") {
textSegments.push(part.text);
}
if (part.functionCall) {
toolCalls.push({
id: part.functionCall.id || newToolCallId(),
type: "function",
function: {
name: part.functionCall.name || "",
arguments: JSON.stringify(part.functionCall.args || {}),
},
});
}
}
const text = textSegments.join("\n");
if (toolCalls.length > 0) {
const msg: InternalMessage = { role: "assistant" };
if (text) msg.content = text;
msg.tool_calls = toolCalls;
return msg;
}
const role = content.role === "model" ? "assistant" : "user";
return { role, content: text };
}
/**
* Convert Gemini request format to OpenAI/internal format.
*
* @param geminiBody parsed Gemini request body
* @param model resolved model string (e.g. "gemini/gemini-pro")
* @param stream whether to stream (derived from URL action suffix)
*/
export function convertGeminiToInternal(
geminiBody: GeminiGenerateBody,
model: string,
stream: boolean
): InternalChatBody {
const messages: InternalMessage[] = [];
// Convert system instruction
if (geminiBody.systemInstruction) {
const systemText =
geminiBody.systemInstruction.parts?.map((p) => p.text ?? "").join("\n") || "";
if (systemText) {
messages.push({ role: "system", content: systemText });
}
}
// Convert contents to messages (text + tool calls + tool responses)
if (geminiBody.contents) {
for (const content of geminiBody.contents) {
const converted = convertContent(content);
if (converted) messages.push(converted);
}
}
const result: InternalChatBody = {
model,
messages,
stream,
max_tokens: geminiBody.generationConfig?.maxOutputTokens,
temperature: geminiBody.generationConfig?.temperature,
top_p: geminiBody.generationConfig?.topP,
};
// Convert tool declarations → OpenAI tools.
if (Array.isArray(geminiBody.tools)) {
const tools: InternalTool[] = [];
for (const tool of geminiBody.tools) {
if (!tool.functionDeclarations) continue;
for (const func of tool.functionDeclarations) {
tools.push({
type: "function",
function: {
name: func.name || "",
description: func.description || "",
parameters: func.parameters || { type: "object", properties: {} },
},
});
}
}
if (tools.length > 0) result.tools = tools;
}
return result;
}

View File

@@ -7,6 +7,7 @@ import {
import { sanitizeErrorMessage } from "@omniroute/open-sse/utils/error";
import { v1betaGeminiGenerateSchema } from "@/shared/validation/schemas";
import { isValidationFailure, validateBody } from "@/shared/validation/helpers";
import { convertGeminiToInternal } from "./convertGeminiToInternal";
let initialized = false;
@@ -132,40 +133,3 @@ export async function POST(request, { params }) {
);
}
}
/**
* Convert Gemini request format to OpenAI/internal format.
*
* @param geminiBody parsed Gemini request body
* @param model resolved model string (e.g. "gemini/gemini-pro")
* @param stream whether to stream (derived from URL action suffix)
*/
function convertGeminiToInternal(geminiBody, model, stream) {
const messages = [];
// Convert system instruction
if (geminiBody.systemInstruction) {
const systemText = geminiBody.systemInstruction.parts?.map((p) => p.text).join("\n") || "";
if (systemText) {
messages.push({ role: "system", content: systemText });
}
}
// Convert contents to messages
if (geminiBody.contents) {
for (const content of geminiBody.contents) {
const role = content.role === "model" ? "assistant" : "user";
const text = content.parts?.map((p) => p.text).join("\n") || "";
messages.push({ role, content: text });
}
}
return {
model,
messages,
stream,
max_tokens: geminiBody.generationConfig?.maxOutputTokens,
temperature: geminiBody.generationConfig?.temperature,
top_p: geminiBody.generationConfig?.topP,
};
}

View File

@@ -0,0 +1,292 @@
import test from "node:test";
import assert from "node:assert/strict";
// Feature #6222 — Gemini tool/function calling end-to-end on /v1beta.
// Covers the three converters that previously dropped tool calls:
// 1. Request: convertGeminiToInternal (sibling of route.ts)
// 2. Non-stream response: convertOpenAIResponseToGemini
// 3. Stream response: openAIChunkToGeminiChunk / transformOpenAISSEToGeminiSSE
//
// The request converter lives in its own module (not route.ts) so it can be
// unit-tested without importing the chat-handler graph, which keeps timers
// alive and hangs the node:test runner.
const { convertGeminiToInternal } = await import(
"../../src/app/api/v1beta/models/[...path]/convertGeminiToInternal.ts"
);
const {
openAIChunkToGeminiChunk,
transformOpenAISSEToGeminiSSE,
convertOpenAIResponseToGemini,
} = await import("../../open-sse/translator/response/openai-to-gemini-sse.ts");
// ---------------------------------------------------------------------------
// 1. Request converter
// ---------------------------------------------------------------------------
test("request: tools[].functionDeclarations → OpenAI tools", () => {
const geminiBody = {
contents: [{ role: "user", parts: [{ text: "What is the weather in Paris?" }] }],
tools: [
{
functionDeclarations: [
{
name: "get_weather",
description: "Get the current weather for a city",
parameters: {
type: "object",
properties: { city: { type: "string" } },
required: ["city"],
},
},
],
},
],
};
const out = convertGeminiToInternal(geminiBody, "gemini/gemini-pro", false);
assert.ok(Array.isArray(out.tools), "tools should be an array");
assert.equal(out.tools.length, 1);
assert.deepEqual(out.tools[0], {
type: "function",
function: {
name: "get_weather",
description: "Get the current weather for a city",
parameters: {
type: "object",
properties: { city: { type: "string" } },
required: ["city"],
},
},
});
// Existing text mapping preserved.
const userMsg = out.messages.find((m) => m.role === "user");
assert.ok(userMsg);
assert.equal(userMsg.content, "What is the weather in Paris?");
});
test("request: prior functionCall part → assistant tool_calls", () => {
const geminiBody = {
contents: [
{ role: "user", parts: [{ text: "Weather in Paris?" }] },
{
role: "model",
parts: [{ functionCall: { name: "get_weather", args: { city: "Paris" } } }],
},
],
};
const out = convertGeminiToInternal(geminiBody, "gemini/gemini-pro", false);
const assistantMsg = out.messages.find((m) => m.role === "assistant");
assert.ok(assistantMsg, "assistant message should exist");
assert.ok(Array.isArray(assistantMsg.tool_calls), "assistant should carry tool_calls");
assert.equal(assistantMsg.tool_calls.length, 1);
assert.equal(assistantMsg.tool_calls[0].type, "function");
assert.equal(assistantMsg.tool_calls[0].function.name, "get_weather");
assert.deepEqual(
JSON.parse(assistantMsg.tool_calls[0].function.arguments),
{ city: "Paris" }
);
});
test("request: functionResponse part → tool role message", () => {
const geminiBody = {
contents: [
{ role: "user", parts: [{ text: "Weather in Paris?" }] },
{
role: "model",
parts: [{ functionCall: { name: "get_weather", args: { city: "Paris" } } }],
},
{
role: "user",
parts: [
{
functionResponse: {
name: "get_weather",
response: { result: { tempC: 18 } },
},
},
],
},
],
};
const out = convertGeminiToInternal(geminiBody, "gemini/gemini-pro", false);
const toolMsg = out.messages.find((m) => m.role === "tool");
assert.ok(toolMsg, "tool message should exist");
assert.equal(toolMsg.tool_call_id, "get_weather");
assert.deepEqual(JSON.parse(toolMsg.content), { tempC: 18 });
});
// ---------------------------------------------------------------------------
// 2. Non-stream response converter
// ---------------------------------------------------------------------------
function makeJsonResponse(obj: unknown): Response {
return new Response(JSON.stringify(obj), {
status: 200,
headers: { "Content-Type": "application/json" },
});
}
test("non-stream: message.tool_calls → parts[].functionCall {name,args}", async () => {
const openaiResponse = {
model: "gemini-pro",
choices: [
{
message: {
role: "assistant",
content: null,
tool_calls: [
{
id: "call_1",
type: "function",
function: {
name: "get_weather",
arguments: '{"city":"Paris"}',
},
},
],
},
finish_reason: "tool_calls",
},
],
usage: { prompt_tokens: 10, completion_tokens: 5, total_tokens: 15 },
};
const geminiResp = await convertOpenAIResponseToGemini(
makeJsonResponse(openaiResponse),
"gemini/gemini-pro"
);
const body = (await geminiResp.json()) as {
candidates: Array<{
content: { parts: Array<Record<string, unknown>> };
finishReason: string;
}>;
};
const parts = body.candidates[0].content.parts;
const fcPart = parts.find((p) => "functionCall" in p) as
| { functionCall: { name: string; args: Record<string, unknown> } }
| undefined;
assert.ok(fcPart, "should emit a functionCall part");
assert.equal(fcPart.functionCall.name, "get_weather");
// args must be parsed to an object, NOT left as a JSON string.
assert.deepEqual(fcPart.functionCall.args, { city: "Paris" });
assert.equal(body.candidates[0].finishReason, "STOP");
});
// ---------------------------------------------------------------------------
// 3. Stream converter — fragmented tool_calls accumulate
// ---------------------------------------------------------------------------
test("stream (unit): fragmented tool_calls accumulate into one functionCall", () => {
const state = {} as Record<string, unknown>;
// Chunk 1: opens the tool call with name + partial args.
const c1 = openAIChunkToGeminiChunk(
{
choices: [
{
delta: {
tool_calls: [
{
index: 0,
id: "call_1",
function: { name: "get_weather", arguments: '{"ci' },
},
],
},
finish_reason: null,
},
],
},
"gemini/gemini-pro",
state
);
assert.equal(c1, null, "intermediate tool-call chunk emits nothing");
// Chunk 2: continuation of args.
const c2 = openAIChunkToGeminiChunk(
{
choices: [
{ delta: { tool_calls: [{ index: 0, function: { arguments: 'ty":"Paris"}' } }] }, finish_reason: null },
],
},
"gemini/gemini-pro",
state
);
assert.equal(c2, null, "second fragment still emits nothing");
// Final chunk with finish_reason — emit the accumulated functionCall.
const c3 = openAIChunkToGeminiChunk(
{ choices: [{ delta: {}, finish_reason: "tool_calls" }] },
"gemini/gemini-pro",
state
);
assert.ok(c3, "final chunk should emit");
const parts = c3!.candidates[0].content.parts as Array<Record<string, unknown>>;
const fcPart = parts.find((p) => "functionCall" in p) as
| { functionCall: { name: string; args: Record<string, unknown> } }
| undefined;
assert.ok(fcPart, "final chunk carries the functionCall part");
assert.equal(fcPart.functionCall.name, "get_weather");
assert.deepEqual(fcPart.functionCall.args, { city: "Paris" });
assert.equal(c3!.candidates[0].finishReason, "STOP");
});
test("stream (e2e): SSE with fragmented tool_calls → Gemini functionCall", async () => {
const events = [
'data: {"choices":[{"delta":{"role":"assistant","tool_calls":[{"index":0,"id":"call_1","function":{"name":"get_weather","arguments":"{\\"ci"}}]},"finish_reason":null}]}',
'data: {"choices":[{"delta":{"tool_calls":[{"index":0,"function":{"arguments":"ty\\":\\"Paris\\"}"}}]},"finish_reason":null}]}',
'data: {"choices":[{"delta":{},"finish_reason":"tool_calls"}],"usage":{"prompt_tokens":10,"completion_tokens":5,"total_tokens":15},"model":"gemini-pro"}',
"data: [DONE]",
];
const body = events.map((e) => e + "\n\n").join("");
const upstream = new Response(
new ReadableStream<Uint8Array>({
start(controller) {
controller.enqueue(new TextEncoder().encode(body));
controller.close();
},
}),
{ status: 200, headers: { "Content-Type": "text/event-stream" } }
);
const geminiResp = transformOpenAISSEToGeminiSSE(upstream, "gemini/gemini-pro");
const reader = geminiResp.body!.getReader();
const decoder = new TextDecoder();
let raw = "";
for (;;) {
const { done, value } = await reader.read();
if (done) break;
raw += decoder.decode(value, { stream: true });
}
raw += decoder.decode();
const chunks: Array<Record<string, unknown>> = [];
for (const line of raw.split("\n")) {
const trimmed = line.endsWith("\r") ? line.slice(0, -1) : line;
if (!trimmed.startsWith("data:")) continue;
const data = trimmed.slice(5).trim();
if (!data) continue;
chunks.push(JSON.parse(data));
}
// Find the functionCall part anywhere in the emitted stream.
let fc: { name: string; args: Record<string, unknown> } | undefined;
for (const ch of chunks) {
const parts =
(ch.candidates as Array<{ content: { parts: Array<Record<string, unknown>> } }>)?.[0]?.content
?.parts ?? [];
for (const p of parts) {
if ("functionCall" in p) fc = (p as { functionCall: typeof fc }).functionCall;
}
}
assert.ok(fc, "stream should emit a functionCall part");
assert.equal(fc!.name, "get_weather");
assert.deepEqual(fc!.args, { city: "Paris" });
});