pjandro/OmniRoute
1
0
Fork 0
mirror of https://github.com/diegosouzapw/OmniRoute.git synced 2026-08-01 21:02:12 +03:00
Code Issues Packages Projects Releases Wiki Activity
Files
aa85fa02bb37df8f5756f1991ccd142012b9732e
OmniRoute/tests/unit/executor-qwen-web.test.ts
Hernan Javier Ardila Sanchez 3be0a5d290 fix: combo input-bound, Responses->Chat image strip, qwen-web toolCalling, empty-response exhaustion (#8476) 
* test(tail): retire stale i18n __MISSING__ repro + fix qianfan website URL

Base-red slice 6, rebased onto the advanced release/v3.8.49 (91fd5f9). The oauth
grok-cli #7610 guard was already fixed on the base by #8027 (it reads the warning
from grokCliAuthJson.ts) — dropped from this slice to avoid a conflicting duplicate.
Remaining two, still red on the current base:

- i18n #7258: the "focused repro" asserted zh-TW.json STILL carries raw __MISSING__:
  placeholders. That backlog was filled (the "no locale has a raw __MISSING__: leaf"
  invariant is the durable guard); retired the now-inverted repro.
- qianfan: Baidu renamed the product page (product/wenxinworkshop -> product-s/
  qianfan_home); updated the expected website URL.

Validated (clean env): i18n 4/0, qianfan 5/0; oauth-modal-grok 2/0 already green on base.

* fix(resilience): short-circuit combo on input-bound failures (context_length_exceeded) (#8375)

isInputBoundRequestFailure() predicate detects deterministic input-bound
errors (context_length_exceeded/context_window_exceeded). The combo loop
propagates the original 400 immediately instead of burning MAX_GLOBAL_ATTEMPTS
retrying identical oversized inputs against every account.

Test: combo-input-bound-failure-8375.test.ts (1 test, 2 assertions)

* fix(resilience): add early-exit in combo dispatcher for input-bound failures (#8375)

When isInputBoundRequestFailure detects context_length_exceeded,
the combo loop returns {ok:false, response} immediately instead of
re-dispatching the oversized request.

Test: node --import tsx/esm --test tests/unit/combo-input-bound-failure-8375.test.ts
- 1 test, 2 assertions, 0 fail

* fix(translator): strip input_image from tool outputs in Responses->Chat downgrade (#8459)

toolOutputContentToString() extracts input_text/output_text parts and
replaces input_image with a placeholder instead of JSON.stringify'ing
the content-part array (which embedded raw ~52KB base64 as inert text).

Applied to both function_call_output and custom_tool_call_output branches.

Existing translator tests: 88/88 pass.
New tests: 4/4 pass.

* fix(providers): set qwen-web toolCalling to false — web-cookie provider has no native function calling (#8437)

qwen-web is a web-cookie provider that emulates tools via synthetic system
prompt text and <tool> XML parsing, never sending a native tools[] field
upstream. The filterTargetsByRequestCompatibility gate filters out non-tool-
calling targets when the request carries tools, but qwen-web's registry entry
had toolCalling=true, so the filter let it through and a tool-using session
failing over to qwen-web would silently degrade to text-only chat with
'Tool X does not exists' errors.

Sibling web-cookie providers (chatgpt-web, yuanbao-web, claude-web, etc.)
all correctly set toolCalling: false — qwen-web was an outlier introduced
in PR #7874.

Verification:
- LSP diagnostics: clean
- Pattern matches chatgpt-web, yuanbao-web, and other web-cookie providers

* fix(backend): empty upstream response mislabeled as exhausted_connection (#8397)

isEmptyContentFailure guard only matched '/empty content/i' but the actual
error text from detectMalformedNonStream is 'returned an empty response
(no usable choices/output)' — which lacks the word 'content'. Expanded
regex to also match '/empty response/i' so these transient upstream glitches
don't get classified as connection-level exhaustion in combo diagnostics.

Test: 28 existing combo-target-exhaustion tests pass (no new test needed)

* test(#8397): add regression test for empty-response 502 not marking provider/connection exhausted

* test(qwen-web): align registry snapshot with toolCalling:false

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>

* fix(resilience): scope #8375 input-bound short-circuit to homogeneous remainders

The isInputBoundFailure short-circuit (context_length_exceeded /
context_window_exceeded) fired unconditionally on the first target, aborting
the whole combo even when later targets are a different model with a larger
context window — regressing the intentional heterogeneous-combo fallback that
isContextOverflow400 (#6637) protects. Reproduced with a 2-target combo
(small-context model fails, larger-context model would have succeeded): the
combo never reached target 2.

Scope the short-circuit to remainders where every remaining target shares the
same modelStr as the one that just failed — the "retrying will fail
identically" premise for context_length_exceeded only holds within a
homogeneous same-model pool.

Rebaselines open-sse/services/combo.ts's frozen file-size cap (3642->3679)
for this PR's own combo.ts growth (config/quality/file-size-baseline.json).

Refs #8375

Co-authored-by: diegosouzapw <diegosouza.pw@gmail.com>

---------

Co-authored-by: Probe Test <probe@example.com>
Co-authored-by: herjarsa <herjarsa@users.noreply.github.com>
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
Co-authored-by: ikelvingo <im.kelvinwong@gmail.com>
Co-authored-by: diegosouzapw <diegosouza.pw@gmail.com>
2026-07-26 03:53:32 -03:00

378 lines
15 KiB
TypeScript
Raw Blame History

import { describe, it, beforeEach, afterEach } from "node:test";
import assert from "node:assert/strict";
import type { RegistryModel } from "../../open-sse/config/providers/shared.ts";
const mod = await import("../../open-sse/executors/qwen-web.ts");
const { REGISTRY } = await import("../../open-sse/config/providerRegistry.ts");
const { FREE_MODEL_BUDGETS } = await import("../../open-sse/config/freeModelCatalog.data.ts");
type FetchCall = { url: string; init: any };
const realFetch = globalThis.fetch;
let calls: FetchCall[] = [];
/** Build an SSE Response from an array of v2 "phase" delta events. */
function sseResponse(events: Array<Record<string, unknown>>): Response {
const encoder = new TextEncoder();
const stream = new ReadableStream({
start(controller) {
for (const ev of events) {
controller.enqueue(encoder.encode(`data: ${JSON.stringify(ev)}\n\n`));
}
controller.enqueue(encoder.encode("data: [DONE]\n\n"));
controller.close();
},
});
return new Response(stream, {
status: 200,
headers: { "content-type": "text/event-stream" },
});
}
function chatCreatedResponse(id = "chat-abc"): Response {
return new Response(JSON.stringify({ success: true, data: { id } }), {
status: 200,
headers: { "content-type": "application/json" },
});
}
/** The 504 + HTML page Alibaba's gateway returns for the retired v1 endpoint
* and for WAF-blocked requests. */
function wafHtmlResponse(status = 504): Response {
return new Response(
"<html>\n<head><title>504 Gateway Time-out</title></head>\n<body>\n" +
"<center><h1>504 Gateway Time-out</h1></center>\n<hr><center>alibaba-ga</center>\n" +
'<meta name="aliyun_waf_aa" content="ff926c7f07e45e2e487a29a6197d3460">\n</body>\n</html>',
{ status, headers: { "content-type": "text/html; charset=utf-8" } }
);
}
beforeEach(() => {
calls = [];
});
afterEach(() => {
globalThis.fetch = realFetch;
});
describe("QwenWebExecutor (v2 migration)", () => {
it("can be instantiated", () => {
assert.ok(new mod.QwenWebExecutor());
});
it("uses the v2 two-step flow: chats/new then chat/completions?chat_id=", async () => {
globalThis.fetch = (async (url: any, init: any = {}) => {
calls.push({ url: String(url), init });
if (String(url).includes("/api/v2/chats/new")) return chatCreatedResponse("chat-xyz");
return sseResponse([
{ choices: [{ delta: { phase: "answer", content: "Hello", status: "typing" } }] },
{ choices: [{ delta: { phase: "answer", content: " world", status: "finished" } }] },
]);
}) as any;
const executor = new mod.QwenWebExecutor();
const result = await executor.execute({
model: "qwen3.7-max",
body: { messages: [{ role: "user", content: "hi" }] },
stream: false,
credentials: { apiKey: "token=jwt-tok; cna=abc; ssxmod_itna=1-xyz" },
signal: null,
} as any);
assert.equal(calls.length, 2, "should make exactly two upstream calls");
assert.match(calls[0].url, /\/api\/v2\/chats\/new$/);
assert.equal(calls[0].init.method, "POST");
assert.match(calls[1].url, /\/api\/v2\/chat\/completions\?chat_id=chat-xyz/);
assert.equal(calls[1].init.method, "POST");
// chats/new payload shape
const newBody = JSON.parse(calls[0].init.body);
assert.deepEqual(newBody.models, ["qwen3.7-max"]);
assert.equal(newBody.chat_type, "t2t");
assert.equal(newBody.chat_mode, "normal");
// completion payload references the created chat_id
const compBody = JSON.parse(calls[1].init.body);
assert.equal(compBody.chat_id, "chat-xyz");
assert.equal(compBody.model, "qwen3.7-max");
assert.equal(compBody.messages[0].role, "user");
assert.equal(compBody.messages[0].content, "hi");
assert.equal(compBody.messages[0].feature_config.thinking_enabled, false);
const json = (await result.response.json()) as any;
assert.equal(json.choices[0].message.content, "Hello world");
});
it("replays the full cookie jar and the extracted bearer token on every call", async () => {
globalThis.fetch = (async (url: any, init: any = {}) => {
calls.push({ url: String(url), init });
if (String(url).includes("/api/v2/chats/new")) return chatCreatedResponse();
return sseResponse([
{ choices: [{ delta: { phase: "answer", content: "ok", status: "finished" } }] },
]);
}) as any;
const cookieBlob = "token=jwt-secret; cna=CNA1; ssxmod_itna=1-AAA; ssxmod_itna2=1-BBB";
const executor = new mod.QwenWebExecutor();
await executor.execute({
model: "qwen3.7-plus",
body: { messages: [{ role: "user", content: "hi" }] },
stream: false,
credentials: { apiKey: cookieBlob },
signal: null,
} as any);
for (const call of calls) {
const headers = call.init.headers as Record<string, string>;
const cookie = headers.Cookie || headers.cookie || "";
assert.match(cookie, /cna=CNA1/, "full cookie jar must be replayed");
assert.match(cookie, /ssxmod_itna=1-AAA/, "WAF cookies must be replayed");
const auth = headers.Authorization || headers.authorization || "";
assert.equal(auth, "Bearer jwt-secret", "bearer token extracted from token= cookie");
}
});
it("sends the anti-bot headers required by the v2 endpoint", async () => {
globalThis.fetch = (async (url: any, init: any = {}) => {
calls.push({ url: String(url), init });
if (String(url).includes("/api/v2/chats/new")) return chatCreatedResponse();
return sseResponse([
{ choices: [{ delta: { phase: "answer", content: "ok", status: "finished" } }] },
]);
}) as any;
const executor = new mod.QwenWebExecutor();
await executor.execute({
model: "qwen3.7-plus",
body: { messages: [{ role: "user", content: "hi" }] },
stream: false,
credentials: { apiKey: "token=t; cna=c" },
signal: null,
} as any);
const headers = calls[0].init.headers as Record<string, string>;
assert.ok(headers["bx-v"], "bx-v header present");
assert.ok(headers["bx-umidtoken"], "bx-umidtoken header present");
assert.equal(headers.source || headers.Source, "web", "source: web header present");
});
it("sends the Qwen SPA build 'version' header on the v2 chat completion request", async () => {
globalThis.fetch = (async (url: any, init: any = {}) => {
calls.push({ url: String(url), init });
if (String(url).includes("/api/v2/chats/new")) return chatCreatedResponse();
return sseResponse([
{ choices: [{ delta: { phase: "answer", content: "ok", status: "finished" } }] },
]);
}) as any;
const executor = new mod.QwenWebExecutor();
await executor.execute({
model: "qwen3.7-plus",
body: { messages: [{ role: "user", content: "hi" }] },
stream: false,
credentials: { apiKey: "token=t; cna=c" },
signal: null,
} as any);
// Without the `version` header the v2 endpoint short-circuits with a
// Bad_Request envelope before ever reaching the model router — see
// open-sse/executors/qwen-web.ts::QWEN_SPA_VERSION.
const completionCall = calls.find((call) => call.url.includes("/api/v2/chat/completions"));
assert.ok(completionCall, "chat/completions call must have been made");
const headers = completionCall!.init.headers as Record<string, string>;
assert.equal(headers.version, "0.2.66", "SPA build version header present");
});
it("maps the thinking phase to reasoning_content, not the answer content", async () => {
globalThis.fetch = (async (url: any) => {
if (String(url).includes("/api/v2/chats/new")) return chatCreatedResponse();
return sseResponse([
{ choices: [{ delta: { phase: "think", content: "let me think", status: "typing" } }] },
{ choices: [{ delta: { phase: "think", content: "...", status: "finished" } }] },
{ choices: [{ delta: { phase: "answer", content: "Final answer", status: "finished" } }] },
]);
}) as any;
const executor = new mod.QwenWebExecutor();
const result = await executor.execute({
model: "qwen3.7-max",
body: { messages: [{ role: "user", content: "hi" }] },
stream: false,
credentials: { apiKey: "token=t; cna=c" },
signal: null,
} as any);
const json = (await result.response.json()) as any;
assert.equal(json.choices[0].message.content, "Final answer");
assert.ok(
!String(json.choices[0].message.content).includes("let me think"),
"thinking content must not leak into the answer"
);
});
it("classifies the retired-v1 / WAF 504 HTML page as a clear auth error (not raw HTML)", async () => {
globalThis.fetch = (async (url: any) => {
if (String(url).includes("/api/v2/chats/new")) return wafHtmlResponse(504);
return chatCreatedResponse();
}) as any;
const executor = new mod.QwenWebExecutor();
const result = await executor.execute({
model: "qwen3.7-max",
body: { messages: [{ role: "user", content: "hi" }] },
stream: false,
credentials: { apiKey: "token=stale; cna=c" },
signal: null,
} as any);
assert.ok([401, 403].includes(result.response.status), "should map to an auth status");
const json = (await result.response.json()) as any;
const msg = String(json.error?.message || "");
assert.ok(!msg.includes("<html"), "raw HTML must not be returned to the client");
assert.match(msg, /session|expired|WAF|re-?login|cookie/i, "actionable error message");
});
it("streams answer-phase content as OpenAI chat.completion.chunk deltas", async () => {
globalThis.fetch = (async (url: any) => {
if (String(url).includes("/api/v2/chats/new")) return chatCreatedResponse();
return sseResponse([
{ choices: [{ delta: { phase: "answer", content: "Hi", status: "typing" } }] },
{ choices: [{ delta: { phase: "answer", content: " there", status: "finished" } }] },
]);
}) as any;
const executor = new mod.QwenWebExecutor();
const result = await executor.execute({
model: "qwen3.7-max",
body: { messages: [{ role: "user", content: "hi" }] },
stream: true,
credentials: { apiKey: "token=t; cna=c" },
signal: null,
} as any);
const text = await result.response.text();
assert.match(text, /chat\.completion\.chunk/);
assert.match(text, /"content":"Hi"/);
assert.match(text, /"content":" there"/);
assert.match(text, /data: \[DONE\]/);
});
it("accepts a bare token (back-compat) without a cookie jar", async () => {
globalThis.fetch = (async (url: any, init: any = {}) => {
calls.push({ url: String(url), init });
if (String(url).includes("/api/v2/chats/new")) return chatCreatedResponse();
return sseResponse([
{ choices: [{ delta: { phase: "answer", content: "ok", status: "finished" } }] },
]);
}) as any;
const executor = new mod.QwenWebExecutor();
await executor.execute({
model: "qwen3.7-plus",
body: { messages: [{ role: "user", content: "hi" }] },
stream: false,
credentials: { apiKey: "barejwttoken" },
signal: null,
} as any);
const headers = calls[0].init.headers as Record<string, string>;
assert.equal(headers.Authorization || headers.authorization, "Bearer barejwttoken");
});
it("registry points at the v2 endpoint and the current model catalog", () => {
const provider = (REGISTRY as any)["qwen-web"];
assert.ok(provider, "qwen-web must be registered");
assert.match(
provider.baseUrl,
/\/api\/v2\/chat\/completions$/,
"registry must use v2 endpoint"
);
const ids = provider.models.map((m: any) => m.id);
assert.deepEqual(ids.sort(), [
"qwen3.6-plus",
"qwen3.7-max",
"qwen3.7-plus",
"qwen3.8-max-preview",
]);
const qwen38 = provider.models.find(
(model: RegistryModel) => model.id === "qwen3.8-max-preview"
);
assert.deepEqual(qwen38, {
id: "qwen3.8-max-preview",
name: "Qwen3.8 Max Preview",
toolCalling: false,
supportsReasoning: true,
supportsVision: true,
contextLength: 1_000_000,
maxOutputTokens: 65_536,
});
const qwen37Max = provider.models.find((model: RegistryModel) => model.id === "qwen3.7-max");
assert.equal(qwen37Max.supportsVision, false);
});
it("free-model catalog lists the current qwen-web ids (not the retired ones)", () => {
const qwenModels = (FREE_MODEL_BUDGETS as any[]).filter((m) => m.provider === "qwen-web");
const ids = qwenModels.map((m) => m.modelId);
assert.ok(ids.includes("qwen3.8-max-preview"), "catalog must list qwen3.8-max-preview");
assert.ok(ids.includes("qwen3.7-max"), "catalog must list qwen3.7-max");
assert.ok(!ids.includes("qwen-plus"), "retired qwen-plus must be gone");
assert.ok(
qwenModels.every((m) => m.freeType !== "discontinued"),
"qwen-web is no longer discontinued after the v2 migration"
);
});
it("passes qwen3.8-max-preview through unchanged", async () => {
globalThis.fetch = (async (url: string | URL | Request, init: RequestInit = {}) => {
calls.push({ url: String(url), init });
if (String(url).includes("/api/v2/chats/new")) return chatCreatedResponse();
return sseResponse([
{ choices: [{ delta: { phase: "answer", content: "ok", status: "finished" } }] },
]);
}) as typeof globalThis.fetch;
const executor = new mod.QwenWebExecutor();
await executor.execute({
model: "qwen3.8-max-preview",
body: {
model: "qwen3.8-max-preview",
messages: [{ role: "user", content: "hi" }],
},
stream: false,
credentials: { apiKey: "token=t; cna=c" },
signal: null,
});
const newBody = JSON.parse(calls[0].init.body);
const completionBody = JSON.parse(calls[1].init.body);
assert.deepEqual(newBody.models, ["qwen3.8-max-preview"]);
assert.equal(completionBody.model, "qwen3.8-max-preview");
assert.equal(completionBody.messages[0].feature_config.thinking_enabled, true);
assert.equal(completionBody.messages[0].feature_config.auto_thinking, true);
});
it("maps legacy model ids to the current upstream catalog", async () => {
globalThis.fetch = (async (url: any, init: any = {}) => {
calls.push({ url: String(url), init });
if (String(url).includes("/api/v2/chats/new")) return chatCreatedResponse();
return sseResponse([
{ choices: [{ delta: { phase: "answer", content: "ok", status: "finished" } }] },
]);
}) as any;
const executor = new mod.QwenWebExecutor();
await executor.execute({
model: "qwen3-max",
body: { messages: [{ role: "user", content: "hi" }] },
stream: false,
credentials: { apiKey: "token=t; cna=c" },
signal: null,
} as any);
const newBody = JSON.parse(calls[0].init.body);
assert.match(newBody.models[0], /^qwen3\.[67]-/, "legacy qwen3-max maps to a current model id");
});
});
Reference in New Issue View Git Blame Copy Permalink