Files
OmniRoute/tests/unit/openai-responses-single-text-content-string.test.ts
Markus Hartung 6e1e5c9a45 fix(sse): tool-incapable provider handling (AI Horde + Responses content-collapse scoping) (#8212)
* fix(sse): collapse single-text-part Responses-API content to a plain string

Every /v1/responses request — even the simplest single-string input —
got 500'd by AI Horde's Aphrodite-backed facade. Root cause:
normalizeResponsesInputForChat() always wraps a plain string input as
`content: [{ type: "input_text", text: value }]` (a one-element array),
and openaiResponsesToOpenAIRequest() mapped that straight through to
`content: [{ type: "text", text: value }]` on the Chat Completions side
— an array. That's spec-valid (OpenAI's own API accepts both shapes),
but strict/naive OpenAI-compatible backends like AI Horde's only
implement the plain-string form and reject the array form outright.

A single-text-part array and a plain string are semantically
identical, so collapse is safe. Real multi-part messages (text+image,
text+file) are left untouched.

Regression test: tests/unit/openai-responses-single-text-content-string.test.ts
(RED before the fix — every collapsed-content assertion failed with an
object instead of a string; GREEN after).

Also adds a deeper AI Horde load-test suite (sequential/concurrent/
cross-model/sustained-throughput/new-capable-model-candidates) that
surfaced this bug via real live traffic after Behemoth-X-123B was
temporarily added to the "default" combo for evaluation.

Co-Authored-By: Markus Hartung <markus.hartung@gmail.com>

* fix(sse): unsupportedParams provider-level fallback for aihorde's live-discovered models

Real OpenClaw traffic against the newly-added Behemoth-X-123B combo
target kept 500ing on every attempt even after the Responses-API
content-array fix landed. The pipeline artifact showed why: `tools`
was still present, unstripped, in the request actually sent to AI
Horde's Aphrodite backend.

Root cause: `unsupportedParams: ["tools", "tool_choice",
"parallel_tool_calls"]` was only declared on the 3 models statically
listed in the aihorde registry entry (Cydonia-24B, Skyfall-31B,
google/gemma-4-31b). AI Horde uses `passthroughModels: true` — its
live worker roster changes constantly — so Behemoth-X-123B, like every
other dynamically-discovered aihorde model, had no model-specific
unsupportedParams entry, and getUnsupportedParams() returned [] for
it. But "the workers run raw text-completion backends" (no tool
calling) is true of every model AI Horde serves, not just the 3
catalogued ones.

Adds a provider-level `unsupportedParams` fallback on RegistryEntry,
checked by getUnsupportedParams() after the per-model lookup misses.
Set on the aihorde entry so it covers its entire live-discovered
roster, present and future, without needing a static per-model catalog
entry for each one.

Regression test: tests/unit/aihorde-tools-unsupported-provider-fallback.test.ts
(RED before the fix — Behemoth-X and deepseek-v4-flash both returned
[] instead of the stripped param list; GREEN after, with a control
case confirming the fallback doesn't leak to unrelated providers).

Co-Authored-By: Markus Hartung <markus.hartung@gmail.com>

* fix(sse): flatten leftover tool-call history when stripping unsupported tools

Third bug in the same AI Horde/Behemoth-X saga: even after tools/
tool_choice were correctly stripped from the live request (previous
fix), real combo traffic still 500'd. The conversation history itself
carried a prior turn's role:"assistant" tool_calls and role:"tool"
result messages, left over from before the combo failed over from a
tool-capable model (Gemini) to a non-tool-capable one (AI Horde). Its
raw completion backend doesn't understand those message shapes at all,
independent of whether live `tools` is present — confirmed by
reproducing with a role:"tool" message and NO tools param at all.

flattenToolHistory() (open-sse/utils/flattenToolHistory.ts) already
existed for exactly this, fully unit-tested — it just had zero call
sites anywhere in the request pipeline. Extracts the unsupported-params
strip into a small testable module
(open-sse/handlers/chatCore/unsupportedParamsStrip.ts, following the
existing chatCore god-file decomposition pattern e.g.
executorClientHeaders.ts) that now also flattens tool-call history
whenever "tools" was among the stripped params.

Regression test: tests/unit/chatcore-unsupported-params-strip.test.ts
(RED before the fix — the flattening test failed with the raw
tool_calls array still present; GREEN after). All 434 existing
chatcore-*.test.ts tests still pass.

Co-Authored-By: Markus Hartung <markus.hartung@gmail.com>

* fix(sse): gate tool-history flattening on unsupported, not on stripped-this-request

The previous commit's flattening only fired when "tools" was actually
present-and-stripped on THIS request. A second live reproduction
against AI Horde had no live `tools` param at all — only stale
tool_calls/tool-result messages inherited from before a combo
failover — and still 500'd, because that condition never triggered.

A model that can't do tool calling can't do it whether or not the
current request happens to carry a `tools` array. Gate on the
unsupported-params list itself (unsupported.includes("tools")) instead
of the subset that was actually present-and-deleted this time.

Regression test added to the same file (RED before — the no-live-tools
case left tool_calls/role:"tool" untouched; GREEN after). All 435
chatcore-*.test.ts still pass.

Co-Authored-By: Markus Hartung <markus.hartung@gmail.com>

* fix(sse): skip tool-incapable combo targets, error clearly on direct requests

Two complementary fixes for a model that structurally can't do tool
calling at all (e.g. AI Horde's raw completion backends) rather than
silently degrading — following up on the earlier strip/flatten fix,
which stopped the crashes but let a tool-incapable target still get
selected and return a 200 that narrates a fake tool call in prose
instead of erroring or being skipped.

1. Root cause, combo routing: getResolvedModelCapabilities()'s
   `supportsTools` resolution only checked per-model registry entries,
   synced capabilities, and static specs — none of which exist for a
   dynamically-discovered model (AI Horde's passthroughModels roster
   changes as workers come and go). It fell through to
   heuristicToolCalling(), which optimistically defaults to `true` for
   any unrecognized model (TOOL_CALLING_UNSUPPORTED_PATTERNS is empty).
   Added a provider-level fallback reusing the same unsupportedParams
   signal the request-time strip already relies on. This makes the
   EXISTING filterTargetsByRequestCompatibility (comboStructure.ts) —
   which already correctly excludes non-tool-capable targets when a
   request requires tools — actually work for these models; no combo.ts
   changes were needed, it was only ever fed bad capability data.

2. Direct/pinned requests: filterTargetsByRequestCompatibility only
   protects combo routing. A direct request naming an exact
   tool-incapable model has no other target to fail over to — added
   checkToolCallingRequiredButUnsupported (chatCore/toolCallingRequiredCheck.ts),
   gated on isCombo: false, returning a clear 400 instead of a 200 that
   silently can't do what was asked.

Regression tests (both RED before, GREEN after):
- tests/unit/model-capabilities-provider-unsupported-tools.test.ts
- tests/unit/chatcore-tool-calling-required-check.test.ts
All 463 chatcore-*/model-capabilities-*.test.ts and 31 combo
compatibility-filter tests still pass.

Co-Authored-By: Markus Hartung <markus.hartung@gmail.com>

* fix(sse): correct handleChatCore return shape for the tool-calling-blocked error

handleChatCore's documented contract is `{ success, response, status,
error }`, not a raw Response — returning `new Response(...)` directly
(copied from a different early-return whose surrounding context turned
out not to share this function's top-level contract) produced "No
response is returned from route handler ... Expected a Response object
but received 'undefined'" and a bare 500 with an empty body, caught
immediately when verifying the previous commit live.

Uses createErrorResult() (already used by the adjacent
translation-failure branch a few lines up) instead of hand-building the
Response, matching the same pattern already established in this
function for early error returns.

Co-Authored-By: Markus Hartung <markus.hartung@gmail.com>

* fix(sse): scope Responses single-text-content collapse to providers that need it

The single-text-part content array -> plain string collapse (added for AI
Horde's Aphrodite facade, which 500s on the array form) was applied
unconditionally to every provider, silently breaking the standard OpenAI
array-shaped content contract that other providers and existing tests
depend on. Added RegistryEntry.requiresPlainStringContent, gated the
collapse on it (true only for aihorde), and threaded modelInfo.provider
through responsesHandler -> responsesApiHelper -> the translator so the
real /v1/responses call site can identify the provider.

Co-Authored-By: Markus Hartung <markus.hartream@gmail.com>

---------

Co-authored-by: Markus Hartung <markus.hartung@gmail.com>
Co-authored-by: Markus Hartung <markus.hartream@gmail.com>
2026-07-23 05:03:53 -03:00

104 lines
4.1 KiB
TypeScript

import test from "node:test";
import assert from "node:assert/strict";
// Live incident: a real request through /v1/responses to a strict OpenAI-compatible
// upstream (AI Horde's Aphrodite-backed facade, oai.aihorde.net) got a 500 Internal
// Server Error for every message, including the simplest possible single-string
// input. Root cause: normalizeResponsesInputForChat() always wraps a plain string
// input as `content: [{ type: "input_text", text: value }]` (array of ONE part),
// and openaiResponsesToOpenAIRequest() maps that straight through to
// `content: [{ type: "text", text: value }]` — an array — never collapsing it back
// to a plain string. That's spec-valid per OpenAI's own API (which accepts both
// shapes), but several strict/naive OpenAI-compatible backends only implement the
// plain-string form for single-part text messages and reject the array form
// outright. A single-text-part array and a plain string are semantically
// identical, so collapsing is safe and doesn't affect real multi-part (text+image,
// text+file) messages, which must stay arrays.
const { openaiResponsesToOpenAIRequest } =
await import("../../open-sse/translator/request/openai-responses.ts");
type ChatMsg = { role: string; content?: unknown };
test("Responses -> OpenAI: plain string input collapses to string content, not a single-part array", () => {
const result = openaiResponsesToOpenAIRequest(
"aihorde/aphrodite/TheDrummer/Behemoth-X-123B-v2.1",
{ input: "Say OK." },
false,
{}
) as { messages: ChatMsg[] };
const userMsg = result.messages.find((m) => m.role === "user");
assert.ok(userMsg, "expected a user message");
assert.equal(typeof userMsg!.content, "string", "expected a plain string, got an array/object");
assert.equal(userMsg!.content, "Say OK.");
});
test("Responses -> OpenAI: single-part input_text array input also collapses to a string", () => {
const result = openaiResponsesToOpenAIRequest(
"aihorde/aphrodite/TheDrummer/Behemoth-X-123B-v2.1",
{
input: [
{ type: "message", role: "user", content: [{ type: "input_text", text: "hi there" }] },
],
},
false,
{}
) as { messages: ChatMsg[] };
const userMsg = result.messages.find((m) => m.role === "user");
assert.equal(typeof userMsg!.content, "string");
assert.equal(userMsg!.content, "hi there");
});
test("Responses -> OpenAI: multi-turn conversation collapses every single-text-part message", () => {
const result = openaiResponsesToOpenAIRequest(
"aihorde/aphrodite/TheDrummer/Behemoth-X-123B-v2.1",
{
input: [
{ type: "message", role: "user", content: [{ type: "input_text", text: "question one" }] },
{
type: "message",
role: "assistant",
content: [{ type: "output_text", text: "answer one" }],
},
{ type: "message", role: "user", content: [{ type: "input_text", text: "question two" }] },
],
},
false,
{}
) as { messages: ChatMsg[] };
const turnMessages = result.messages.filter((m) => m.role === "user" || m.role === "assistant");
assert.equal(turnMessages.length, 3);
for (const m of turnMessages) {
assert.equal(typeof m.content, "string", `expected string content for role ${m.role}`);
}
assert.equal(turnMessages[0].content, "question one");
assert.equal(turnMessages[1].content, "answer one");
assert.equal(turnMessages[2].content, "question two");
});
test("Responses -> OpenAI: real multi-part content (text + image) is NOT collapsed", () => {
const result = openaiResponsesToOpenAIRequest(
"aihorde/aphrodite/TheDrummer/Behemoth-X-123B-v2.1",
{
input: [
{
type: "message",
role: "user",
content: [
{ type: "input_text", text: "what is this?" },
{ type: "input_image", image_url: "https://example.com/cat.png" },
],
},
],
},
false,
{}
) as { messages: ChatMsg[] };
const userMsg = result.messages.find((m) => m.role === "user");
assert.ok(Array.isArray(userMsg!.content), "multi-part content must stay an array");
assert.equal((userMsg!.content as unknown[]).length, 2);
});