Add Alibaba-family media model support (#8266)

* Add Alibaba-family media models

* chore(quality): rebaseline imageRegistry+cognitive for #8266 media own-growth

---------

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
This commit is contained in:
backryun
2026-07-23 23:12:28 +09:00
committed by GitHub
parent 2670674107
commit 869d08ff8b
21 changed files with 2351 additions and 76 deletions

View File

@@ -54,11 +54,6 @@
"count": 3
}
},
"open-sse/handlers/imageGeneration.ts": {
"@typescript-eslint/no-explicit-any": {
"count": 3
}
},
"open-sse/handlers/musicGeneration.ts": {
"@typescript-eslint/no-explicit-any": {
"count": 1
@@ -1845,11 +1840,6 @@
"count": 3
}
},
"tests/unit/prompt-required-routes.test.ts": {
"@typescript-eslint/no-explicit-any": {
"count": 2
}
},
"tests/unit/provider-connection-apikey-dedup.test.ts": {
"@typescript-eslint/no-explicit-any": {
"count": 1
@@ -2430,4 +2420,4 @@
"count": 5
}
}
}
}

View File

@@ -159,7 +159,8 @@
"_rebaseline_2026_07_02_5816_qoder": "PR #5816 (@AgentKiller45, qoder PAT via qodercli): qoderCli.ts 666->989, new-above-cap frozen (owner-approved baseline freeze). The growth is the legitimate PAT job-token exchange + quota parsing CLI transport (the pure-JS Cosy path 500'd on every PAT request); extracting the spawn/parse helpers now would just add indirection to a contributor PR mid-merge. Test frozen also raised for this PR's coverage growth: providers-page-utils.test.ts 1052->1092. Additionally clears an inherited base-red from the already-merged #5933 (codex json_schema->text.format): translator-openai-responses-req.test.ts 1097->1172 (+75 regression tests, no offending branch left). All remain frozen (cannot grow further); release captain's rebaseline-at-release supersedes.",
"open-sse/services/qoderCli.ts": 989,
"_rebaseline_pr1043_minimax_tts": "Upstream port decolua/9router#1043 (toanalien) own growth: audioSpeech.ts 965->1061 (+96). Adds MiniMax T2A v2 TTS dispatch (handleMinimaxSpeech + hexToBytes helper) — provider entry was already in audioRegistry (format: minimax-tts) but no handler existed, falling through to the OpenAI-compatible default that fails (T2A has custom shape + hex-encoded audio + base_resp envelope). New branch sits next to the other inline provider branches (xiaomi-mimo, coqui, tortoise, aws-polly) — extracting would just create indirection. Covered by tests/unit/minimax-tts-1043.test.ts (3 tests, GREEN: success, base_resp error, invalid-hex).",
"open-sse/config/imageRegistry.ts": 821,
"_rebaseline_2026_07_23_8266_alibaba_media": "#8266 (@backryun) own growth: imageRegistry.ts 821->979 (+158) — Alibaba-family media models (Qwen image/video, Bailian, Wan) added to the image/video registry. Registry model data, not extractable logic; frozen at new size.",
"open-sse/config/imageRegistry.ts": 979,
"open-sse/config/providerRegistry.ts": 4731,
"open-sse/executors/antigravity.ts": 1813,
"open-sse/executors/base.ts": 1540,

View File

@@ -123,7 +123,7 @@
"_rebaseline_2026_06_26_v3837_release": "343->345. v3.8.37 cycle drift surfaced by the release-green pre-flight (the Quality Ratchet does NOT run on PR->release fast-gates, so warnings/complexity accrued unmeasured across this cycle's 76 commits — provider adds DGrid/Pioneer/xAI, headroom proxy lifecycle #4649, ~50 SSE/translator fixes, Engine Combos #5062). Trust-but-verify: this release-finalize working tree touches ONLY CHANGELOG.md, docs/i18n/*/CHANGELOG.md mirrors, and these baselines — 0 production-code change, so all drift is inherited cycle drift (`any` warn-allowed in open-sse/ + tests/). Tighten via --require-tighten next cycle."
},
"cognitiveComplexity": {
"value": 950,
"value": 951,
"_rebaseline_2026_07_10_gcf_v3_2": "885->888 (+3). PR feat/headroom-gcf-v3.2-nested-flattening: own growth from re-vendoring the GCF (Headroom) codec to spec v3.2 (nested flattening). The new over-threshold functions are the vendored v3.2 flatten/unflatten walk in open-sse/services/compression/engines/headroom/gcf/{generic,decode_generic}.ts. Imported third-party code kept byte-faithful to upstream gcf-typescript; measured 888 with the update vs 885 on the pristine origin/release/v3.8.47 tip. Guarded by tests/unit/compression/headroom-smartcrusher.test.ts (deep-nested case). Structural shrink belongs upstream in gcf.",
"_rebaseline_2026_07_12_v3847_mergetrain_burst": "885->890 (+5). v3.8.47 /merge-prs merge-train batch (23 merge-ready PRs) inherited drift: cognitive-complexity does NOT run on PR->release fast-gates, so incidental growth accrued unmeasured across the batch. Measured 890 on the combined merge-train tip (5d980352d) vs 885 on the pristine release tip 1b7a9150e. #6838 (headroom gcf codec re-vendor) accounts for +3 (its own baseline bump to 888, superseded here by this later 890 rebaseline which already covers it); the remaining +2 is parallel-batch drift across the other 22 PRs. Owner-approved rebaseline (merge-burst reconciliation, same class as the v3.8.46/v3.8.44 notes below). Tighten via --update next cycle.",
"_rebaseline_2026_07_09_6587_kiro_api_key_auth": "884->885 (+1). PR #6587 (@strangersp) own growth: open-sse/services/usage/kiro.ts gains ONE new over-threshold function — getKiroUsage grew from a single fetch to a 3-endpoint fallback chain (codewhisperer-get / codewhisperer-post / q-get) with per-attempt auth-header selection (tokentype: API_KEY vs Bearer-only), needed so usage/quota lookups work for the new long-lived-API-key auth path in addition to the existing OAuth path (measured: 0 violations on release tip -> 1 violation, complexity 33, at open-sse/services/usage/kiro.ts). Covered by tests/unit/kiro-iam-profilearn-usage.test.ts (tokentype header selection, friendly auth-expired/rejected-token messages). Cohesive multi-endpoint-fallback logic at an existing usage chokepoint; not extractable without splitting the fallback loop mid-merge. Structural shrink tracked in #3501.",
@@ -146,7 +146,8 @@
"_rebaseline_2026_07_06_v3845_release_close": "867->877 (+10). v3.8.45 cycle drift measured by check:release-green (hermetic) on release tip 5ecca12aa5 during the /generate-release Phase 0 pre-flight. Inherited from the cycle's merge burst (cognitive-complexity does not run on PR->release fast-gates); the captain's pre-flight fixes are gate/test/workflow changes (complexity-neutral). Tighten via --update next cycle.",
"_rebaseline_2026_07_07_6519_chirag_fallback_reasons": "882->883 (+1). PR #6519 (@chirag127, #6461): the preview route's fallbackReasons dedup loop adds one function over the cognitive threshold. Owner-approved rebaseline (contributor own-growth). Tighten via --update next cycle.",
"_rebaseline_2026_07_08_6556_inherited_drift": "883->884 (+1). PR #6556 (omniglyph engine): drift herdado do merge burst da base (cognitive-complexity nao roda no fast-path PR->release). Trust-but-verify: check:cognitive-complexity mede 884 IDENTICO na base pura origin/release/v3.8.47 e neste HEAD — o PR e cognitive-net-zero (runCompressionAsync extraido para engines/omniglyphSingleMode.ts e o page client dividido em section components na mesma rodada). Tighten via --update next cycle.",
"_rebaseline_2026_07_19_v3849_fix_sweep_cluster": "890->900 (owner-approved, 2026-07-19). Same /fix-prs cluster as the complexity note: the clean-by-cyclomatic subset still bumped cognitive-complexity (measured 896 on the 9-PR combined probe tip vs 890 baseline). Raised to 900 = 896 + 4 headroom, matching the complexity rebaseline. Tighten via --update next cycle."
"_rebaseline_2026_07_19_v3849_fix_sweep_cluster": "890->900 (owner-approved, 2026-07-19). Same /fix-prs cluster as the complexity note: the clean-by-cyclomatic subset still bumped cognitive-complexity (measured 896 on the 9-PR combined probe tip vs 890 baseline). Raised to 900 = 896 + 4 headroom, matching the complexity rebaseline. Tighten via --update next cycle.",
"_note_8266": "2026-07-23: +1 own-growth from #8266 (@backryun) Alibaba media provider dispatch wiring"
},
"typeCoveragePct": {
"value": 92.17,

View File

@@ -71,9 +71,36 @@ creating another provider card or changing saved provider IDs.
provider page.
6. Keep custom endpoint overrides available for advanced and workspace-specific deployments.
Token Plan image-generation models are intentionally outside this first chat-provider slice. The
official quick start states that they use dedicated endpoints, so they require the image provider
pipeline instead of being advertised as chat models.
Token Plan media models remain outside the chat catalog because they use dedicated endpoints.
`wan2.7-image` and `wan2.7-image-pro` are registered in `open-sse/config/imageRegistry.ts`;
`happyhorse-1.1-i2v`, `happyhorse-1.1-t2v`, and `happyhorse-1.1-r2v` are registered in
`open-sse/config/videoRegistry.ts`. Both registries reuse the `qwen-cloud-token-plan` connection
identity and its region-specific credentials.
Regular Qwen Cloud has separate pay-as-you-go image and video catalogs under the `qwen-cloud`
identity. Its image models are `wan2.7-image-pro`, `wan2.7-image`, `qwen-image-3.0-pro`,
`qwen-image-2.0-pro-2026-06-22`, `qwen-image-2.0-2026-03-03`, and `z-image-turbo`. Its video
models are `happyhorse-1.1-i2v`, `happyhorse-1.1-t2v`, `happyhorse-1.1-r2v`,
`happyhorse-1.0-video-edit`, `wan2.7-t2v`, `wan2.7-i2v`, `wan2.7-r2v-2026-06-12`, and
`wan2.7-videoedit`. These models reuse only the regular `qwen-cloud` connection and regional
DashScope media endpoint. The Token Plan and pay-as-you-go registries do not inherit or merge
each other's model lists.
The `bailian-coding-plan` identity also has its own media allowlists:
`wan2.7-image`, `wan2.7-image-pro`, `qwen-image-2.0`, and `qwen-image-2.0-pro` for image
generation, plus
`happyhorse-1.1-i2v`, `happyhorse-1.1-t2v`, and `happyhorse-1.1-r2v` for video
generation. It uses only the Bailian Coding Plan connection and regional Coding Plan endpoint.
Its image and video registries do not inherit from or merge with either Qwen Cloud provider.
The regular `alibaba` identity has separate image and video allowlists. Its image models are
`qwen-image-3.0-pro`, `qwen-image-2.0-pro-2026-06-22`, `qwen-image-2.0`,
`z-image-turbo`, and `wan2.6-t2i`. Its added video models are `happyhorse-1.1-i2v`,
`happyhorse-1.1-t2v`, `happyhorse-1.1-r2v`, `happyhorse-1.0-video-edit`,
`wan2.7-i2v-2026-04-25`, `wan2.6-i2v-flash`, `wan2.7-t2v-2026-06-12`,
`wan2.7-r2v-2026-06-12`, and `wan2.7-videoedit`. These models use only the Alibaba Model Studio
connection and its selected regional media endpoint; they are not added to any Coding Plan or Qwen
Cloud registry.
The Qwen Cloud Token Plan chat catalog follows the Individual plan's exact-string text allowlist:
`qwen3.8-max-preview`, `qwen3.7-max`, `qwen3.7-plus`, `qwen3.6-flash`, `glm-5.2`, and

View File

@@ -140,6 +140,31 @@ function resolveAliasImageRequired(alias, modelConfig) {
}
export const IMAGE_PROVIDERS: Record<string, ImageProviderConfig> = {
"qwen-cloud-token-plan": {
id: "qwen-cloud-token-plan",
alias: "qct",
baseUrl:
"https://token-plan.ap-southeast-1.maas.aliyuncs.com/api/v1/services/aigc/multimodal-generation/generation",
authType: "apikey",
authHeader: "bearer",
format: "qwen-token-plan-image",
models: [
{
id: "wan2.7-image",
name: "Wan 2.7 Image",
inputModalities: ["text", "image"],
},
{
id: "wan2.7-image-pro",
name: "Wan 2.7 Image Pro",
inputModalities: ["text", "image"],
},
],
// Both models share 1K/2K support. The Pro model also accepts explicit 4K
// dimensions, which callers can still pass through the permissive request schema.
supportedSizes: ["1024x1024", "2048x2048"],
},
openai: {
id: "openai",
baseUrl: "https://api.openai.com/v1/images/generations",
@@ -187,7 +212,8 @@ export const IMAGE_PROVIDERS: Record<string, ImageProviderConfig> = {
"microsoft-designer-web": {
id: "microsoft-designer-web",
alias: "msdesigner",
baseUrl: "https://designerapp.officeapps.live.com/designerapp/DallE.ashx?action=GetDallEImagesCogSci",
baseUrl:
"https://designerapp.officeapps.live.com/designerapp/DallE.ashx?action=GetDallEImagesCogSci",
authType: "apikey",
authHeader: "bearer",
format: "designer-web",
@@ -652,9 +678,21 @@ export const IMAGE_PROVIDERS: Record<string, ImageProviderConfig> = {
authHeader: "bearer",
format: "adobe-firefly-image",
models: [
{ id: "nano-banana-pro", name: "Firefly Gemini 3.0 (Nano Banana Pro)", inputModalities: ["text", "image"] },
{ id: "nano-banana", name: "Firefly Gemini 2.5 (Nano Banana)", inputModalities: ["text", "image"] },
{ id: "nano-banana-2", name: "Firefly Gemini 3.1 (Nano Banana 2)", inputModalities: ["text", "image"] },
{
id: "nano-banana-pro",
name: "Firefly Gemini 3.0 (Nano Banana Pro)",
inputModalities: ["text", "image"],
},
{
id: "nano-banana",
name: "Firefly Gemini 2.5 (Nano Banana)",
inputModalities: ["text", "image"],
},
{
id: "nano-banana-2",
name: "Firefly Gemini 3.1 (Nano Banana 2)",
inputModalities: ["text", "image"],
},
{ id: "gpt-image-2", name: "Firefly GPT Image 2", inputModalities: ["text", "image"] },
{ id: "gpt-image", name: "Firefly GPT Image 2", inputModalities: ["text", "image"] },
{ id: "gpt-image-1.5", name: "Firefly GPT Image 1.5", inputModalities: ["text", "image"] },
@@ -662,11 +700,131 @@ export const IMAGE_PROVIDERS: Record<string, ImageProviderConfig> = {
{ id: "flux-pro", name: "Firefly Flux 1.1 Pro", inputModalities: ["text", "image"] },
{ id: "flux-ultra", name: "Firefly Flux 1.1 Ultra", inputModalities: ["text", "image"] },
{ id: "seedream-4", name: "Firefly Seedream 4.0", inputModalities: ["text", "image"] },
{ id: "seedream-5-lite", name: "Firefly Seedream 5.0 Lite", inputModalities: ["text", "image"] },
{ id: "runway-gen4-image", name: "Firefly Runway Gen-4 Image", inputModalities: ["text", "image"] },
{
id: "seedream-5-lite",
name: "Firefly Seedream 5.0 Lite",
inputModalities: ["text", "image"],
},
{
id: "runway-gen4-image",
name: "Firefly Runway Gen-4 Image",
inputModalities: ["text", "image"],
},
],
supportedSizes: ["1:1", "16:9", "9:16", "4:3", "3:4", "1024x1024", "1792x1024", "1024x1792"],
},
// Keep Bailian Coding Plan after existing duplicate model owners so adding
// explicit `bailian-coding-plan/` and `bcp/` routes does not change
// historical bare-model routing.
"bailian-coding-plan": {
id: "bailian-coding-plan",
alias: "bcp",
baseUrl:
"https://coding-intl.dashscope.aliyuncs.com/api/v1/services/aigc/multimodal-generation/generation",
authType: "apikey",
authHeader: "bearer",
format: "bailian-coding-plan-image",
models: [
{
id: "wan2.7-image",
name: "Wan 2.7 Image",
inputModalities: ["text", "image"],
},
{
id: "wan2.7-image-pro",
name: "Wan 2.7 Image Pro",
inputModalities: ["text", "image"],
},
{
id: "qwen-image-2.0",
name: "Qwen Image 2.0",
inputModalities: ["text", "image"],
},
{
id: "qwen-image-2.0-pro",
name: "Qwen Image 2.0 Pro",
inputModalities: ["text", "image"],
},
],
supportedSizes: ["1024x1024", "2048x2048"],
},
// Keep Alibaba after existing duplicate model owners so adding explicit
// `alibaba/` and `ali/` routes does not change historical bare-model routing.
alibaba: {
id: "alibaba",
alias: "ali",
baseUrl:
"https://dashscope-intl.aliyuncs.com/api/v1/services/aigc/multimodal-generation/generation",
authType: "apikey",
authHeader: "bearer",
format: "alibaba-image",
models: [
{
id: "qwen-image-3.0-pro",
name: "Qwen Image 3.0 Pro",
inputModalities: ["text", "image"],
},
{
id: "qwen-image-2.0-pro-2026-06-22",
name: "Qwen Image 2.0 Pro (2026-06-22)",
inputModalities: ["text", "image"],
},
{
id: "qwen-image-2.0",
name: "Qwen Image 2.0",
inputModalities: ["text", "image"],
},
{ id: "z-image-turbo", name: "Z-Image Turbo" },
{ id: "wan2.6-t2i", name: "Wan 2.6 T2I" },
],
supportedSizes: ["1024x1024", "1280x1280", "2048x2048"],
},
// Keep regular Qwen Cloud isolated from Alibaba, Bailian Coding Plan, and
// Qwen Cloud Token Plan. Explicit `qwen-cloud/` or `qwc/` routes use only
// the regular Qwen Cloud connection and its regional DashScope endpoint.
"qwen-cloud": {
id: "qwen-cloud",
alias: "qwc",
baseUrl:
"https://dashscope-intl.aliyuncs.com/api/v1/services/aigc/multimodal-generation/generation",
authType: "apikey",
authHeader: "bearer",
format: "qwen-cloud-image",
models: [
{
id: "wan2.7-image-pro",
name: "Wan 2.7 Image Pro",
inputModalities: ["text", "image"],
},
{
id: "wan2.7-image",
name: "Wan 2.7 Image",
inputModalities: ["text", "image"],
},
{
id: "qwen-image-3.0-pro",
name: "Qwen Image 3.0 Pro",
inputModalities: ["text", "image"],
},
{
id: "qwen-image-2.0-pro-2026-06-22",
name: "Qwen Image 2.0 Pro (2026-06-22)",
inputModalities: ["text", "image"],
},
{
id: "qwen-image-2.0-2026-03-03",
name: "Qwen Image 2.0 (2026-03-03)",
inputModalities: ["text", "image"],
},
{ id: "z-image-turbo", name: "Z-Image Turbo" },
],
// 1K/2K are shared by the whole catalog. Wan 2.7 Image Pro callers can
// still pass supported 4K dimensions through the permissive request schema.
supportedSizes: ["1024x1024", "2048x2048"],
},
};
/**

View File

@@ -27,6 +27,36 @@ interface VideoProvider {
}
export const VIDEO_PROVIDERS: Record<string, VideoProvider> = {
"qwen-cloud-token-plan": {
id: "qwen-cloud-token-plan",
alias: "qct",
baseUrl: "https://token-plan.ap-southeast-1.maas.aliyuncs.com/api/v1",
statusUrl: "https://token-plan.ap-southeast-1.maas.aliyuncs.com/api/v1/tasks",
authType: "apikey",
authHeader: "bearer",
format: "dashscope-video",
models: [
{ id: "happyhorse-1.1-i2v", name: "HappyHorse 1.1 I2V" },
{ id: "happyhorse-1.1-t2v", name: "HappyHorse 1.1 T2V" },
{ id: "happyhorse-1.1-r2v", name: "HappyHorse 1.1 R2V" },
],
},
"bailian-coding-plan": {
id: "bailian-coding-plan",
alias: "bcp",
baseUrl: "https://coding-intl.dashscope.aliyuncs.com/api/v1",
statusUrl: "https://coding-intl.dashscope.aliyuncs.com/api/v1/tasks",
authType: "apikey",
authHeader: "bearer",
format: "dashscope-video",
models: [
{ id: "happyhorse-1.1-i2v", name: "HappyHorse 1.1 I2V" },
{ id: "happyhorse-1.1-t2v", name: "HappyHorse 1.1 T2V" },
{ id: "happyhorse-1.1-r2v", name: "HappyHorse 1.1 R2V" },
],
},
vertex: {
id: "vertex",
baseUrl: "https://us-central1-aiplatform.googleapis.com/v1",
@@ -221,7 +251,37 @@ export const VIDEO_PROVIDERS: Record<string, VideoProvider> = {
authType: "apikey",
authHeader: "bearer",
format: "dashscope-video",
models: [{ id: "wan2.7-t2v", name: "Wan 2.7 T2V" }],
models: [
{ id: "happyhorse-1.1-i2v", name: "HappyHorse 1.1 I2V" },
{ id: "happyhorse-1.1-t2v", name: "HappyHorse 1.1 T2V" },
{ id: "happyhorse-1.1-r2v", name: "HappyHorse 1.1 R2V" },
{ id: "happyhorse-1.0-video-edit", name: "HappyHorse 1.0 Video Edit" },
{ id: "wan2.7-i2v-2026-04-25", name: "Wan 2.7 I2V (2026-04-25)" },
{ id: "wan2.6-i2v-flash", name: "Wan 2.6 I2V Flash" },
{ id: "wan2.7-t2v-2026-06-12", name: "Wan 2.7 T2V (2026-06-12)" },
{ id: "wan2.7-r2v-2026-06-12", name: "Wan 2.7 R2V (2026-06-12)" },
{ id: "wan2.7-videoedit", name: "Wan 2.7 Video Edit" },
],
},
"qwen-cloud": {
id: "qwen-cloud",
alias: "qwc",
baseUrl: "https://dashscope-intl.aliyuncs.com/api/v1",
statusUrl: "https://dashscope-intl.aliyuncs.com/api/v1/tasks",
authType: "apikey",
authHeader: "bearer",
format: "dashscope-video",
models: [
{ id: "happyhorse-1.1-i2v", name: "HappyHorse 1.1 I2V" },
{ id: "happyhorse-1.1-t2v", name: "HappyHorse 1.1 T2V" },
{ id: "happyhorse-1.1-r2v", name: "HappyHorse 1.1 R2V" },
{ id: "happyhorse-1.0-video-edit", name: "HappyHorse 1.0 Video Edit" },
{ id: "wan2.7-t2v", name: "Wan 2.7 T2V" },
{ id: "wan2.7-i2v", name: "Wan 2.7 I2V" },
{ id: "wan2.7-r2v-2026-06-12", name: "Wan 2.7 R2V (2026-06-12)" },
{ id: "wan2.7-videoedit", name: "Wan 2.7 Video Edit" },
],
},
// Segmind video generation (#6656). Same `POST /v1/{model}` REST shape as

View File

@@ -24,6 +24,7 @@ import { kieExecutor } from "../executors/kie.ts";
import { mapImageSize } from "../translator/image/sizeMapper.ts";
import { getCodexClientVersion, getCodexUserAgent } from "../config/codexClient.ts";
import { ChatGptWebExecutor } from "../executors/chatgpt-web.ts";
import type { ExecutorLog, ProviderCredentials } from "../executors/base.ts";
import { getChatGptImage, findChatGptImageBySha256 } from "../services/chatgptImageCache.ts";
import { createHash } from "node:crypto";
import { saveCallLog } from "@/lib/usageDb";
@@ -73,6 +74,7 @@ import { handleSegmindImageGeneration } from "./imageGeneration/providers/segmin
import { handleDesignerWebImageGeneration } from "./imageGeneration/providers/designerWeb.ts";
import { handleMinimaxImageGeneration } from "./imageGeneration/providers/minimax.ts";
import { handleAdobeFireflyImageGeneration } from "./imageGeneration/providers/adobeFirefly.ts";
import { handleAlibabaImageGeneration } from "./imageGeneration/providers/alibabaImage.ts";
import {
applyPollinationsAnonymousFallback,
reportPollinationsAnonOutcome,
@@ -621,6 +623,22 @@ export async function handleImageGeneration({
});
}
if (
providerConfig.format === "alibaba-image" ||
providerConfig.format === "qwen-cloud-image" ||
providerConfig.format === "qwen-token-plan-image" ||
providerConfig.format === "bailian-coding-plan-image"
) {
return handleAlibabaImageGeneration({
model,
provider,
providerConfig,
body,
credentials,
log,
});
}
return handleOpenAIImageGeneration({ model, provider, providerConfig, body, credentials, log });
}
@@ -1256,11 +1274,11 @@ export async function handleImageEdit({
}: {
provider: string;
model: string;
body: Record<string, any>;
body: Record<string, unknown>;
imageBytes: Buffer;
imageMime?: string; // accepted for symmetry with route layer; not used
credentials: any;
log: any;
credentials: ProviderCredentials | null | undefined;
log: ExecutorLog | null | undefined;
signal?: AbortSignal | null;
clientHeaders?: Record<string, string> | null;
}) {

View File

@@ -0,0 +1,202 @@
import { resolveAlibabaProviderMediaBaseUrl } from "@/shared/constants/alibabaProviderRegions";
import { isJsonObject } from "../../../utils/kieTask.ts";
import { sanitizeErrorMessage } from "../../../utils/error.ts";
interface AlibabaImageOptions {
model: string;
provider: string;
providerConfig: {
baseUrl: string;
models?: Array<{ id: string }>;
};
body: Record<string, unknown> & {
prompt?: unknown;
size?: unknown;
n?: unknown;
image?: unknown;
image_url?: unknown;
imageUrls?: unknown;
image_urls?: unknown;
parameters?: unknown;
};
credentials?: {
apiKey?: string;
accessToken?: string;
providerSpecificData?: unknown;
} | null;
log?: {
info: (scope: string, message: string) => void;
error: (scope: string, message: string) => void;
} | null;
}
function collectImageUrls(body: AlibabaImageOptions["body"]): string[] {
const values = [body.image, body.image_url, body.imageUrls, body.image_urls].flatMap((value) =>
Array.isArray(value) ? value : [value]
);
const urls = new Set<string>();
for (const value of values) {
if (typeof value === "string" && value.trim()) {
urls.add(value.trim());
continue;
}
if (!isJsonObject(value)) continue;
const url =
typeof value.url === "string"
? value.url
: typeof value.image_url === "string"
? value.image_url
: null;
if (url?.trim()) urls.add(url.trim());
}
return [...urls];
}
function normalizeImageSize(value: unknown): string | undefined {
if (typeof value !== "string" || !value.trim()) return undefined;
const size = value.trim();
return /^\d+x\d+$/i.test(size) ? size.replace(/x/i, "*") : size;
}
function getAlibabaImageUrls(payload: unknown): string[] {
if (!isJsonObject(payload) || !isJsonObject(payload.output)) return [];
const choices = Array.isArray(payload.output.choices) ? payload.output.choices : [];
const urls = new Set<string>();
for (const choice of choices) {
if (!isJsonObject(choice) || !isJsonObject(choice.message)) continue;
const content = Array.isArray(choice.message.content) ? choice.message.content : [];
for (const item of content) {
if (!isJsonObject(item) || typeof item.image !== "string" || !item.image.trim()) continue;
urls.add(item.image.trim());
}
}
return [...urls];
}
function getAlibabaImageError(payload: unknown, fallback: string): string {
if (!isJsonObject(payload)) return fallback;
const message =
typeof payload.message === "string"
? payload.message
: typeof payload.code === "string"
? payload.code
: fallback;
return sanitizeErrorMessage(message) || fallback;
}
export async function handleAlibabaImageGeneration({
model,
provider,
providerConfig,
body,
credentials,
log,
}: AlibabaImageOptions) {
const isRegisteredModel = providerConfig.models?.some((entry) => entry.id === model) === true;
if (!isRegisteredModel) {
return {
success: false as const,
status: 400,
error: `Unsupported ${provider} image model: ${model}`,
};
}
const token = credentials?.apiKey || credentials?.accessToken;
if (!token) {
return {
success: false as const,
status: 401,
error: `${provider} API key is required`,
};
}
const prompt = typeof body.prompt === "string" ? body.prompt.trim() : "";
const content = [
...collectImageUrls(body).map((image) => ({ image })),
...(prompt ? [{ text: prompt }] : []),
];
const parameters: Record<string, unknown> = isJsonObject(body.parameters)
? { ...body.parameters }
: {};
const size = normalizeImageSize(body.size);
if (size) parameters.size = size;
if (Number.isInteger(body.n) && Number(body.n) > 0) parameters.n = Number(body.n);
for (const key of [
"negative_prompt",
"prompt_extend",
"watermark",
"seed",
"enable_sequential",
"thinking_mode",
"color_palette",
] as const) {
if (body[key] !== undefined) parameters[key] = body[key];
}
const mediaBaseUrl = resolveAlibabaProviderMediaBaseUrl(
provider,
credentials.providerSpecificData,
providerConfig.baseUrl
);
const url = mediaBaseUrl.endsWith("/services/aigc/multimodal-generation/generation")
? mediaBaseUrl
: `${mediaBaseUrl}/services/aigc/multimodal-generation/generation`;
log?.info?.("IMAGE", `${provider}/${model} (alibaba-image)`);
try {
const response = await fetch(url, {
method: "POST",
headers: {
Authorization: `Bearer ${token}`,
"Content-Type": "application/json",
},
body: JSON.stringify({
model,
input: {
messages: [{ role: "user", content }],
},
...(Object.keys(parameters).length > 0 ? { parameters } : {}),
}),
});
const payload = await response.json().catch(() => ({}));
if (!response.ok) {
return {
success: false as const,
status: response.status || 502,
error: getAlibabaImageError(payload, "Alibaba image generation failed"),
};
}
const urls = getAlibabaImageUrls(payload);
if (urls.length === 0) {
return {
success: false as const,
status: 502,
error: "Alibaba image generation returned no images",
};
}
return {
success: true as const,
data: {
created: Math.floor(Date.now() / 1000),
data: urls.map((url) => ({ url })),
},
};
} catch (error: unknown) {
log?.error?.("IMAGE", `Alibaba image generation failed: ${sanitizeErrorMessage(error)}`);
return {
success: false as const,
status: 502,
error: sanitizeErrorMessage(error) || "Alibaba image generation failed",
};
}
}

View File

@@ -1,11 +1,14 @@
import { isJsonObject } from "../../utils/kieTask.ts";
import { saveCallLog } from "@/lib/usageDb";
import { resolveAlibabaProviderMediaBaseUrl } from "@/shared/constants/alibabaProviderRegions";
import { sanitizeErrorMessage } from "../../utils/error.ts";
/**
* Alibaba (DashScope) Wan video generation: create async task → poll → MP4.
* Targets wan2.7-t2v on the DashScope intl region. Reuses the stored alibaba
* provider Bearer apiKey — no separate credential flow.
* Alibaba-family video generation: create async task → poll → MP4.
*
* Provider identity remains authoritative for credentials and regional endpoints.
* Bailian Coding Plan, regular Qwen Cloud, and Qwen Cloud Token Plan intentionally
* share wire helpers only; their model lists and connections stay isolated.
*/
export async function handleDashscopeVideoGeneration({
model,
@@ -17,17 +20,35 @@ export async function handleDashscopeVideoGeneration({
}: {
model: string;
provider: string;
providerConfig: { baseUrl: string; statusUrl?: string };
providerConfig: {
baseUrl: string;
statusUrl?: string;
models?: Array<{ id: string }>;
};
body: Record<string, unknown> & {
prompt?: unknown;
negative_prompt?: unknown;
size?: unknown;
aspect_ratio?: unknown;
ratio?: unknown;
resolution?: unknown;
duration?: unknown;
image?: unknown;
image_url?: unknown;
imageUrls?: unknown;
image_urls?: unknown;
reference_images?: unknown;
media?: unknown;
prompt_extend?: unknown;
watermark?: unknown;
timeout_ms?: unknown;
poll_interval_ms?: unknown;
};
credentials?: { apiKey?: string; accessToken?: string } | null;
credentials?: {
apiKey?: string;
accessToken?: string;
providerSpecificData?: unknown;
} | null;
log?: {
info: (scope: string, message: string) => void;
error: (scope: string, message: string) => void;
@@ -37,29 +58,51 @@ export async function handleDashscopeVideoGeneration({
const timeoutMs = Number(body.timeout_ms) > 0 ? Number(body.timeout_ms) : 300000;
const pollIntervalMs = Number(body.poll_interval_ms) > 0 ? Number(body.poll_interval_ms) : 2500;
const token = credentials?.apiKey || credentials?.accessToken;
const baseUrl = providerConfig.baseUrl.replace(/\/$/, "");
const statusUrl = (providerConfig.statusUrl || `${baseUrl}/tasks`).replace(/\/$/, "");
const isAlibabaManagedMediaProvider =
provider === "alibaba" ||
provider === "bailian-coding-plan" ||
provider === "qwen-cloud" ||
provider === "qwen-cloud-token-plan";
const isRegisteredAlibabaMediaModel =
!isAlibabaManagedMediaProvider ||
providerConfig.models?.some((entry) => entry.id === model) === true;
if (!isRegisteredAlibabaMediaModel) {
return {
success: false,
status: 400,
error: `Unsupported ${provider} video model: ${model}`,
};
}
const baseUrl = (
isAlibabaManagedMediaProvider
? resolveAlibabaProviderMediaBaseUrl(
provider,
credentials?.providerSpecificData,
providerConfig.baseUrl
)
: providerConfig.baseUrl
).replace(/\/$/, "");
const statusUrl = (
isAlibabaManagedMediaProvider
? `${baseUrl}/tasks`
: providerConfig.statusUrl || `${baseUrl}/tasks`
).replace(/\/$/, "");
const prompt = typeof body.prompt === "string" ? body.prompt : String(body.prompt ?? "");
if (!token) {
return { success: false, status: 401, error: "Alibaba DashScope API key is required" };
}
const sizeParam = normalizeDashscopeSize(body.size, body.aspect_ratio);
const parameters: Record<string, unknown> = {};
if (sizeParam) parameters.size = sizeParam;
if (body.duration != null) parameters.duration = Number(body.duration);
const payload = {
model,
input: {
prompt,
...(typeof body.negative_prompt === "string"
? { negative_prompt: body.negative_prompt }
: {}),
},
parameters,
};
const payload =
provider === "qwen-cloud" || provider === "alibaba"
? buildAlibabaMediaPayload(provider, model, prompt, body)
: provider === "qwen-cloud-token-plan" || provider === "bailian-coding-plan"
? buildHappyHorsePayload(model, prompt, body)
: buildLegacyDashscopePayload(model, prompt, body);
if ("error" in payload) {
return { success: false, status: 400, error: payload.error };
}
if (log) {
log.info(
@@ -154,6 +197,353 @@ export async function handleDashscopeVideoGeneration({
}
}
function buildLegacyDashscopePayload(model: string, prompt: string, body: Record<string, unknown>) {
const sizeParam = normalizeDashscopeSize(body.size, body.aspect_ratio);
const parameters: Record<string, unknown> = {};
if (sizeParam) parameters.size = sizeParam;
if (body.duration != null) parameters.duration = Number(body.duration);
return {
model,
input: {
prompt,
...(typeof body.negative_prompt === "string"
? { negative_prompt: body.negative_prompt }
: {}),
},
parameters,
};
}
function buildHappyHorsePayload(
model: string,
prompt: string,
body: Record<string, unknown>
): Record<string, unknown> | { error: string } {
const input = buildDashscopeInput(prompt, body);
const media = collectDashscopeMedia(body);
if (model.endsWith("-i2v")) {
const firstFrame =
media.find((item) => item.type === "first_frame") ||
media.find((item) => item.type === "reference_image");
if (!firstFrame) {
return { error: `Image input is required for video model: ${model}` };
}
input.media = [{ type: "first_frame", url: firstFrame.url }];
} else if (model.endsWith("-r2v")) {
const referenceImages = media
.filter((item) => item.type === "first_frame" || item.type === "reference_image")
.map((item) => ({ type: "reference_image", url: item.url }));
if (referenceImages.length === 0) {
return { error: `Reference image input is required for video model: ${model}` };
}
input.media = referenceImages;
}
return {
model,
input,
parameters: buildModernDashscopeParameters(body, !model.endsWith("-i2v")),
};
}
function buildAlibabaMediaPayload(
provider: string,
model: string,
prompt: string,
body: Record<string, unknown>
): Record<string, unknown> | { error: string } {
if (
model === "happyhorse-1.1-i2v" ||
model === "happyhorse-1.1-t2v" ||
model === "happyhorse-1.1-r2v"
) {
return buildHappyHorsePayload(model, prompt, body);
}
if (model === "happyhorse-1.0-video-edit" || model === "wan2.7-videoedit") {
return buildModernDashscopeVideoEditPayload(model, prompt, body);
}
if (model === "wan2.7-t2v" || model === "wan2.7-t2v-2026-06-12") {
return buildModernDashscopeTextToVideoPayload(model, prompt, body);
}
if (model === "wan2.7-i2v" || model === "wan2.7-i2v-2026-04-25" || model === "wan2.6-i2v-flash") {
return buildModernDashscopeImageToVideoPayload(model, prompt, body);
}
if (model === "wan2.7-r2v-2026-06-12") {
return buildModernDashscopeReferenceToVideoPayload(model, prompt, body);
}
return { error: `Unsupported ${provider} video model: ${model}` };
}
function buildModernDashscopeTextToVideoPayload(
model: string,
prompt: string,
body: Record<string, unknown>
) {
const input = buildDashscopeInput(prompt, body);
const audio = collectDashscopeMedia(body).find((item) => item.type === "driving_audio");
if (audio) input.audio_url = audio.url;
return {
model,
input,
parameters: buildModernDashscopeParameters(body, true),
};
}
function buildModernDashscopeImageToVideoPayload(
model: string,
prompt: string,
body: Record<string, unknown>
): Record<string, unknown> | { error: string } {
const input = buildDashscopeInput(prompt, body);
const media = collectDashscopeMedia(body);
const firstFrame =
media.find((item) => item.type === "first_frame") ||
media.find((item) => item.type === "reference_image");
if (!firstFrame) {
return { error: `Image input is required for video model: ${model}` };
}
input.media = [
{ type: "first_frame", url: firstFrame.url },
...media
.filter((item) => item.type === "last_frame" || item.type === "driving_audio")
.map((item) => copyDashscopeMediaItem(item)),
];
return {
model,
input,
parameters: buildModernDashscopeParameters(body, false),
};
}
function buildModernDashscopeReferenceToVideoPayload(
model: string,
prompt: string,
body: Record<string, unknown>
): Record<string, unknown> | { error: string } {
const input = buildDashscopeInput(prompt, body);
const references = collectDashscopeMedia(body).flatMap((item) => {
if (
item.type === "first_frame" ||
item.type === "last_frame" ||
item.type === "reference_image"
) {
return [copyDashscopeMediaItem(item, "reference_image")];
}
if (item.type === "video" || item.type === "first_clip" || item.type === "reference_video") {
return [copyDashscopeMediaItem(item, "reference_video")];
}
return [];
});
if (references.length === 0) {
return { error: `Reference image or video input is required for video model: ${model}` };
}
input.media = references;
return {
model,
input,
parameters: buildModernDashscopeParameters(body, true),
};
}
function buildModernDashscopeVideoEditPayload(
model: string,
prompt: string,
body: Record<string, unknown>
): Record<string, unknown> | { error: string } {
const input = buildDashscopeInput(prompt, body);
const media = collectDashscopeMedia(body);
const sourceVideo = media.find(
(item) => item.type === "video" || item.type === "first_clip" || item.type === "reference_video"
);
if (!sourceVideo) {
return { error: `Video input is required for video model: ${model}` };
}
input.media = [
{ type: "video", url: sourceVideo.url },
...media
.filter((item) => item.type === "first_frame" || item.type === "reference_image")
.map((item) => copyDashscopeMediaItem(item, "reference_image")),
];
return {
model,
input,
parameters: buildModernDashscopeParameters(body, true),
};
}
type DashscopeMediaType =
| "first_frame"
| "last_frame"
| "first_clip"
| "reference_image"
| "reference_video"
| "video"
| "driving_audio";
type DashscopeMediaItem = {
type: DashscopeMediaType;
url: string;
reference_voice?: string;
};
const DASHSCOPE_MEDIA_TYPES = new Set<DashscopeMediaType>([
"first_frame",
"last_frame",
"first_clip",
"reference_image",
"reference_video",
"video",
"driving_audio",
]);
function buildDashscopeInput(prompt: string, body: Record<string, unknown>) {
return {
...(prompt ? { prompt } : {}),
...(typeof body.negative_prompt === "string" && body.negative_prompt.trim()
? { negative_prompt: body.negative_prompt.trim() }
: {}),
} as Record<string, unknown>;
}
function buildModernDashscopeParameters(
body: Record<string, unknown>,
includeRatio: boolean
): Record<string, unknown> {
const parameters = isJsonObject(body.parameters) ? { ...body.parameters } : {};
const resolution = normalizeHappyHorseResolution(body.resolution, body.size);
if (resolution) parameters.resolution = resolution;
const ratio = normalizeHappyHorseRatio(body.ratio, body.aspect_ratio, body.size);
if (ratio && includeRatio) parameters.ratio = ratio;
if (body.duration != null && Number.isFinite(Number(body.duration))) {
parameters.duration = Number(body.duration);
}
if (typeof body.prompt_extend === "boolean") parameters.prompt_extend = body.prompt_extend;
if (typeof body.watermark === "boolean") parameters.watermark = body.watermark;
if (Number.isInteger(body.seed) && Number(body.seed) >= 0) parameters.seed = Number(body.seed);
if (typeof body.shot_type === "string" && body.shot_type.trim()) {
parameters.shot_type = body.shot_type.trim();
}
return parameters;
}
function copyDashscopeMediaItem(
item: DashscopeMediaItem,
type: DashscopeMediaType = item.type
): DashscopeMediaItem {
return {
type,
url: item.url,
...(item.reference_voice ? { reference_voice: item.reference_voice } : {}),
};
}
function collectDashscopeMedia(body: Record<string, unknown>): DashscopeMediaItem[] {
if (Array.isArray(body.media)) {
const explicit = body.media.filter(isJsonObject).flatMap((item) => {
const normalized = toDashscopeMediaItem(item, "reference_image");
return normalized ? [normalized] : [];
});
if (explicit.length > 0) return dedupeDashscopeMedia(explicit);
}
const media: DashscopeMediaItem[] = [];
const imageCandidates = [body.image, body.image_url, body.imageUrls, body.image_urls].flatMap(
(value) => (Array.isArray(value) ? value : [value])
);
imageCandidates.forEach((value, index) => {
const item = toDashscopeMediaItem(value, index === 0 ? "first_frame" : "reference_image");
if (item) media.push(item);
});
addDashscopeMedia(media, body.reference_images, "reference_image");
addDashscopeMedia(media, [body.video, body.video_url, body.videoUrls, body.video_urls], "video");
addDashscopeMedia(media, body.reference_videos, "reference_video");
addDashscopeMedia(media, [body.audio, body.audio_url], "driving_audio");
return dedupeDashscopeMedia(media);
}
function addDashscopeMedia(media: DashscopeMediaItem[], value: unknown, type: DashscopeMediaType) {
const values = Array.isArray(value) ? value : [value];
for (const candidate of values.flatMap((item) => (Array.isArray(item) ? item : [item]))) {
const normalized = toDashscopeMediaItem(candidate, type);
if (normalized) media.push(normalized);
}
}
function toDashscopeMediaItem(
value: unknown,
fallbackType: DashscopeMediaType
): DashscopeMediaItem | null {
if (typeof value === "string" && value.trim()) {
return { type: fallbackType, url: value.trim() };
}
if (!isJsonObject(value)) return null;
const url = [value.url, value.image_url, value.video_url, value.audio_url].find(
(candidate) => typeof candidate === "string" && candidate.trim()
);
if (typeof url !== "string") return null;
const type =
typeof value.type === "string" && DASHSCOPE_MEDIA_TYPES.has(value.type as DashscopeMediaType)
? (value.type as DashscopeMediaType)
: fallbackType;
return {
type,
url: url.trim(),
...(typeof value.reference_voice === "string" && value.reference_voice.trim()
? { reference_voice: value.reference_voice.trim() }
: {}),
};
}
function dedupeDashscopeMedia(media: DashscopeMediaItem[]): DashscopeMediaItem[] {
const seen = new Set<string>();
return media.filter((item) => {
const key = `${item.type}\0${item.url}`;
if (seen.has(key)) return false;
seen.add(key);
return true;
});
}
function normalizeHappyHorseResolution(resolution: unknown, size: unknown): string | undefined {
if (typeof resolution === "string" && /^(720|1080)p$/i.test(resolution.trim())) {
return resolution.trim().toUpperCase();
}
if (typeof size !== "string") return undefined;
const match = size.trim().match(/^(\d+)[x*](\d+)$/i);
if (!match) return undefined;
return Math.max(Number(match[1]), Number(match[2])) >= 1920 ? "1080P" : "720P";
}
function normalizeHappyHorseRatio(
ratio: unknown,
aspectRatio: unknown,
size: unknown
): string | undefined {
const explicit = [ratio, aspectRatio].find(
(value) => typeof value === "string" && /^\d+:\d+$/.test(value.trim())
);
if (typeof explicit === "string") return explicit.trim();
if (typeof size !== "string") return undefined;
const match = size.trim().match(/^(\d+)[x*](\d+)$/i);
if (!match) return undefined;
const width = Number(match[1]);
const height = Number(match[2]);
if (width === height) return "1:1";
if (width * 9 === height * 16) return "16:9";
if (width * 16 === height * 9) return "9:16";
if (width * 3 === height * 4) return "4:3";
if (width * 4 === height * 3) return "3:4";
return undefined;
}
// Map OmniRoute size/aspect_ratio → Alibaba DashScope "WxH" (1280*720).
// Accepts "1280*720", "1280x720", or a ratio "16:9". Returns undefined if unparseable
// (then omitted from the payload so DashScope applies its own default).

View File

@@ -10,7 +10,7 @@ import {
AUDIO_SPEECH_PROVIDERS,
AUDIO_TRANSCRIPTION_PROVIDERS,
} from "@omniroute/open-sse/config/audioRegistry.ts";
import { AI_PROVIDERS } from "@/shared/constants/providers";
import { toProviderModels, type ProviderModelGroup } from "./mediaProviderModels";
type Modality = "image" | "video" | "music" | "speech" | "transcription";
type GenerationResult = {
@@ -19,27 +19,6 @@ type GenerationResult = {
timestamp: number;
audioUrl?: string;
};
type MediaModelConfig = { id: string; name: string };
type MediaProviderConfig = { id: string; models: MediaModelConfig[] };
type ProviderModelGroup = {
id: string;
name: string;
models: { id: string; name: string }[];
};
const PROVIDER_METADATA = AI_PROVIDERS as Record<string, { name?: string }>;
function toProviderModels(registry: Record<string, MediaProviderConfig>): ProviderModelGroup[] {
return Object.entries(registry).map(([providerId, config]) => ({
id: providerId,
name: PROVIDER_METADATA[providerId]?.name || config.id || providerId,
models: config.models.map((model) => ({
id: model.id.startsWith(`${providerId}/`) ? model.id : `${providerId}/${model.id}`,
name: model.name,
})),
}));
}
const IMAGE_PROVIDER_MODELS = toProviderModels(IMAGE_PROVIDERS);
const VIDEO_PROVIDER_MODELS = toProviderModels(VIDEO_PROVIDERS);
const MUSIC_PROVIDER_MODELS = toProviderModels(MUSIC_PROVIDERS);

View File

@@ -0,0 +1,37 @@
import { AI_PROVIDERS } from "@/shared/constants/providers";
type MediaModelConfig = { id: string; name: string };
type MediaProviderConfig = { id: string; models: MediaModelConfig[] };
export type ProviderModelGroup = {
id: string;
name: string;
models: { id: string; name: string }[];
};
const PROVIDER_METADATA = AI_PROVIDERS as Record<string, { name?: string }>;
/**
* Sort dropdown labels without changing registry insertion order, which can
* affect bare-model routing precedence.
*/
export function toProviderModels(
registry: Record<string, MediaProviderConfig>
): ProviderModelGroup[] {
return Object.entries(registry)
.map(([providerId, config]) => ({
id: providerId,
name: PROVIDER_METADATA[providerId]?.name || config.id || providerId,
models: config.models.map((model) => ({
id: model.id.startsWith(`${providerId}/`) ? model.id : `${providerId}/${model.id}`,
name: model.name,
})),
}))
.sort((left, right) => {
const byName = left.name.localeCompare(right.name, "en", {
sensitivity: "base",
numeric: true,
});
return byName || left.id.localeCompare(right.id, "en");
});
}

View File

@@ -65,16 +65,28 @@ async function postHandler(request, context) {
}
const body = parsed.body;
const startTime = Date.now();
const parsedModel = parseVideoModel(body.model);
const promptError = promptRequiredResponse(body);
if (promptError) return promptError;
const promptOptional =
(parsedModel.model === "happyhorse-1.1-i2v" &&
(parsedModel.provider === "alibaba" ||
parsedModel.provider === "bailian-coding-plan" ||
parsedModel.provider === "qwen-cloud-token-plan" ||
parsedModel.provider === "qwen-cloud")) ||
(parsedModel.provider === "qwen-cloud" && parsedModel.model === "wan2.7-i2v") ||
(parsedModel.provider === "alibaba" &&
(parsedModel.model === "wan2.7-i2v-2026-04-25" || parsedModel.model === "wan2.6-i2v-flash"));
if (!promptOptional) {
const promptError = promptRequiredResponse(body);
if (promptError) return promptError;
}
// Enforce API key policies (model restrictions + budget limits)
const policy = await enforceApiKeyPolicy(request, body.model);
if (policy.rejection) return policy.rejection;
// Parse model to get provider
const { provider } = parseVideoModel(body.model);
const { provider } = parsedModel;
if (!provider) {
return errorResponse(
HTTP_STATUS.BAD_REQUEST,

View File

@@ -144,3 +144,19 @@ export function resolveAlibabaProviderModelsUrl(
.replace(/\/(?:chat\/completions|messages|models)$/i, "");
return baseUrl ? `${baseUrl}/models` : "";
}
/**
* Resolve the dedicated Alibaba-family media API root from the connection's
* regional OpenAI/Anthropic-compatible endpoint.
*/
export function resolveAlibabaProviderMediaBaseUrl(
providerId: string,
providerSpecificData?: unknown,
fallback = ""
): string {
return resolveAlibabaProviderBaseUrl(providerId, providerSpecificData, fallback)
.trim()
.replace(/\/+$/, "")
.replace(/\/compatible-mode\/v1(?:\/(?:chat\/completions|models))?$/i, "/api/v1")
.replace(/\/apps\/anthropic(?:\/v1)?(?:\/messages)?$/i, "/api/v1");
}

View File

@@ -0,0 +1,161 @@
import test from "node:test";
import assert from "node:assert/strict";
import { mkdtempSync } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
process.env.DATA_DIR = mkdtempSync(join(tmpdir(), "omniroute-alibaba-image-media-"));
const { IMAGE_PROVIDERS, parseImageModel } = await import("../../open-sse/config/imageRegistry.ts");
const { resolveProviderServiceKinds } = await import("../../open-sse/config/mediaServiceKinds.ts");
const { handleImageGeneration } = await import("../../open-sse/handlers/imageGeneration.ts");
const { resolveAlibabaProviderMediaBaseUrl } =
await import("../../src/shared/constants/alibabaProviderRegions.ts");
function jsonResponse(body, status = 200) {
return new Response(JSON.stringify(body), {
status,
headers: { "content-type": "application/json" },
});
}
test("Alibaba owns an independent five-model image catalog", () => {
assert.deepEqual(
IMAGE_PROVIDERS.alibaba.models.map((model) => model.id),
[
"qwen-image-3.0-pro",
"qwen-image-2.0-pro-2026-06-22",
"qwen-image-2.0",
"z-image-turbo",
"wan2.6-t2i",
]
);
assert.notEqual(IMAGE_PROVIDERS.alibaba.models, IMAGE_PROVIDERS["bailian-coding-plan"].models);
assert.notEqual(IMAGE_PROVIDERS.alibaba.models, IMAGE_PROVIDERS["qwen-cloud-token-plan"].models);
assert.deepEqual(parseImageModel("ali/qwen-image-3.0-pro"), {
provider: "alibaba",
model: "qwen-image-3.0-pro",
});
assert.deepEqual(resolveProviderServiceKinds("alibaba", ["llm"]).sort(), [
"image",
"llm",
"video",
]);
});
test("Alibaba registration preserves existing bare duplicate-model routing", () => {
assert.deepEqual(parseImageModel("z-image-turbo"), {
provider: "nanogpt",
model: "z-image-turbo",
});
assert.deepEqual(parseImageModel("qwen-image-2.0"), {
provider: "lmarena",
model: "qwen-image-2.0",
});
assert.deepEqual(parseImageModel("qwen-image-3.0-pro"), {
provider: "alibaba",
model: "qwen-image-3.0-pro",
});
});
test("Alibaba image URLs follow only the selected Alibaba region", () => {
assert.equal(
resolveAlibabaProviderMediaBaseUrl("alibaba", { region: "global-sg" }),
"https://dashscope-intl.aliyuncs.com/api/v1"
);
assert.equal(
resolveAlibabaProviderMediaBaseUrl("alibaba", { region: "china-beijing" }),
"https://dashscope.aliyuncs.com/api/v1"
);
assert.notEqual(
resolveAlibabaProviderMediaBaseUrl("alibaba", { region: "global-sg" }),
resolveAlibabaProviderMediaBaseUrl("bailian-coding-plan", { region: "global-sg" })
);
assert.notEqual(
resolveAlibabaProviderMediaBaseUrl("alibaba", { region: "global-sg" }),
resolveAlibabaProviderMediaBaseUrl("qwen-cloud-token-plan", { region: "global-sg" })
);
});
test("All five Alibaba image models use the Alibaba multimodal endpoint and key", async () => {
const originalFetch = globalThis.fetch;
const captures = [];
globalThis.fetch = async (url, options = {}) => {
captures.push({
url: String(url),
headers: options.headers,
body: JSON.parse(String(options.body || "{}")),
});
return jsonResponse({
output: {
choices: [
{
message: {
content: [{ image: "https://cdn.example.com/alibaba-image.png" }],
},
},
],
},
});
};
try {
for (const model of IMAGE_PROVIDERS.alibaba.models.map((entry) => entry.id)) {
const result = await handleImageGeneration({
body: {
model: `alibaba/${model}`,
prompt: "A cinematic horse portrait",
size: "2048x2048",
n: 1,
negative_prompt: "blurry",
prompt_extend: true,
watermark: false,
seed: 42,
},
credentials: {
apiKey: "alibaba-key",
providerSpecificData: { region: "china-beijing" },
},
log: null,
});
assert.equal(result.success, true);
}
assert.equal(captures.length, 5);
for (const capture of captures) {
assert.equal(
capture.url,
"https://dashscope.aliyuncs.com/api/v1/services/aigc/multimodal-generation/generation"
);
assert.equal(capture.headers.Authorization, "Bearer alibaba-key");
assert.deepEqual(capture.body.input.messages[0].content, [
{ text: "A cinematic horse portrait" },
]);
assert.deepEqual(capture.body.parameters, {
size: "2048*2048",
n: 1,
negative_prompt: "blurry",
prompt_extend: true,
watermark: false,
seed: 42,
});
}
} finally {
globalThis.fetch = originalFetch;
}
});
test("Alibaba rejects image models outside its own allowlist", async () => {
const result = await handleImageGeneration({
body: {
model: "alibaba/wan2.7-image-pro",
prompt: "wrong catalog",
},
credentials: { apiKey: "alibaba-key" },
log: null,
});
assert.equal(result.status, 400);
assert.match(result.error, /unsupported alibaba image model/i);
});

View File

@@ -0,0 +1,258 @@
import test from "node:test";
import assert from "node:assert/strict";
import { mkdtempSync } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
process.env.DATA_DIR = mkdtempSync(join(tmpdir(), "omniroute-alibaba-video-media-"));
const { VIDEO_PROVIDERS, parseVideoModel } = await import("../../open-sse/config/videoRegistry.ts");
const { resolveProviderServiceKinds } = await import("../../open-sse/config/mediaServiceKinds.ts");
const { handleVideoGeneration } = await import("../../open-sse/handlers/videoGeneration.ts");
const ADDED_ALIBABA_VIDEO_MODELS = [
"happyhorse-1.1-i2v",
"happyhorse-1.1-t2v",
"happyhorse-1.1-r2v",
"happyhorse-1.0-video-edit",
"wan2.7-i2v-2026-04-25",
"wan2.6-i2v-flash",
"wan2.7-t2v-2026-06-12",
"wan2.7-r2v-2026-06-12",
"wan2.7-videoedit",
];
function jsonResponse(body, status = 200) {
return new Response(JSON.stringify(body), {
status,
headers: { "content-type": "application/json" },
});
}
function immediateTimeout(callback, _ms, ...args) {
if (typeof callback === "function") callback(...args);
return 0;
}
async function captureAlibabaRequest(body, region = "global-sg") {
const originalFetch = globalThis.fetch;
const originalSetTimeout = globalThis.setTimeout;
let captured;
globalThis.setTimeout = immediateTimeout;
globalThis.fetch = async (url, options = {}) => {
const stringUrl = String(url);
if (stringUrl.endsWith("/services/aigc/video-generation/video-synthesis")) {
captured = {
url: stringUrl,
headers: options.headers,
body: JSON.parse(String(options.body || "{}")),
};
return jsonResponse({ output: { task_id: "alibaba-video-task", task_status: "PENDING" } });
}
if (stringUrl.endsWith("/tasks/alibaba-video-task")) {
return jsonResponse({
output: {
task_status: "SUCCEEDED",
video_url: "https://cdn.example.com/alibaba-video.mp4",
},
});
}
throw new Error(`Unexpected URL: ${stringUrl}`);
};
try {
const result = await handleVideoGeneration({
body,
credentials: {
apiKey: "alibaba-video-key",
providerSpecificData: { region },
},
log: null,
});
return { captured, result };
} finally {
globalThis.fetch = originalFetch;
globalThis.setTimeout = originalSetTimeout;
}
}
test("Alibaba exposes the nine added video models without changing Qwen Cloud lists", () => {
const alibabaModels = VIDEO_PROVIDERS.alibaba.models.map((model) => model.id);
assert.deepEqual(alibabaModels, ADDED_ALIBABA_VIDEO_MODELS);
assert.equal(
VIDEO_PROVIDERS["qwen-cloud"].models.some(
(model) =>
model.id === "wan2.7-i2v-2026-04-25" ||
model.id === "wan2.6-i2v-flash" ||
model.id === "wan2.7-t2v-2026-06-12"
),
false
);
assert.deepEqual(parseVideoModel("ali/wan2.6-i2v-flash"), {
provider: "alibaba",
model: "wan2.6-i2v-flash",
});
assert.deepEqual(resolveProviderServiceKinds("alibaba", ["llm"]).sort(), [
"image",
"llm",
"video",
]);
});
test("Alibaba HappyHorse I2V uses only the Alibaba key and selected regional endpoint", async () => {
const { captured, result } = await captureAlibabaRequest(
{
model: "alibaba/happyhorse-1.1-i2v",
image_url: "https://cdn.example.com/horse.png",
resolution: "720p",
duration: 5,
},
"china-beijing"
);
assert.equal(
captured.url,
"https://dashscope.aliyuncs.com/api/v1/services/aigc/video-generation/video-synthesis"
);
assert.equal(captured.headers.Authorization, "Bearer alibaba-video-key");
assert.deepEqual(captured.body.input.media, [
{ type: "first_frame", url: "https://cdn.example.com/horse.png" },
]);
assert.deepEqual(captured.body.parameters, {
resolution: "720P",
duration: 5,
});
assert.equal(result.success, true);
});
test("Alibaba Wan I2V variants map first-frame and driving-audio media", async () => {
for (const model of ["wan2.7-i2v-2026-04-25", "wan2.6-i2v-flash"]) {
const { captured } = await captureAlibabaRequest({
model: `alibaba/${model}`,
image_url: "https://cdn.example.com/first.png",
audio_url: "https://cdn.example.com/voice.mp3",
duration: 10,
});
assert.equal(captured.body.model, model);
assert.deepEqual(captured.body.input.media, [
{ type: "first_frame", url: "https://cdn.example.com/first.png" },
{ type: "driving_audio", url: "https://cdn.example.com/voice.mp3" },
]);
assert.equal(captured.body.parameters.duration, 10);
assert.equal("ratio" in captured.body.parameters, false);
}
});
test("Alibaba dated Wan T2V maps modern resolution, ratio, and audio input", async () => {
const { captured } = await captureAlibabaRequest({
model: "alibaba/wan2.7-t2v-2026-06-12",
prompt: "A cinematic horse race",
audio_url: "https://cdn.example.com/score.mp3",
size: "1920x1080",
duration: 10,
prompt_extend: true,
});
assert.deepEqual(captured.body.input, {
prompt: "A cinematic horse race",
audio_url: "https://cdn.example.com/score.mp3",
});
assert.deepEqual(captured.body.parameters, {
resolution: "1080P",
ratio: "16:9",
duration: 10,
prompt_extend: true,
});
});
test("Alibaba Wan R2V preserves ordered image, video, and voice references", async () => {
const { captured } = await captureAlibabaRequest({
model: "alibaba/wan2.7-r2v-2026-06-12",
prompt: "Video 1 greets Image 1",
media: [
{
type: "reference_video",
url: "https://cdn.example.com/actor.mp4",
reference_voice: "https://cdn.example.com/actor.mp3",
},
{
type: "reference_image",
url: "https://cdn.example.com/prop.png",
},
],
ratio: "16:9",
});
assert.deepEqual(captured.body.input.media, [
{
type: "reference_video",
url: "https://cdn.example.com/actor.mp4",
reference_voice: "https://cdn.example.com/actor.mp3",
},
{
type: "reference_image",
url: "https://cdn.example.com/prop.png",
},
]);
assert.equal(captured.body.parameters.ratio, "16:9");
});
test("Alibaba video-edit models require and map a source video", async () => {
for (const model of ["happyhorse-1.0-video-edit", "wan2.7-videoedit"]) {
const { captured } = await captureAlibabaRequest({
model: `alibaba/${model}`,
prompt: "Replace the jacket with the reference image",
video_url: "https://cdn.example.com/source.mp4",
reference_images: ["https://cdn.example.com/jacket.png"],
resolution: "1080P",
});
assert.deepEqual(captured.body.input.media, [
{ type: "video", url: "https://cdn.example.com/source.mp4" },
{ type: "reference_image", url: "https://cdn.example.com/jacket.png" },
]);
assert.equal(captured.body.parameters.resolution, "1080P");
}
});
test("Alibaba media-specific video models reject missing input locally", async () => {
for (const [model, errorPattern] of [
["happyhorse-1.1-i2v", /image input is required/i],
["happyhorse-1.1-r2v", /image input is required/i],
["happyhorse-1.0-video-edit", /video input is required/i],
["wan2.7-i2v-2026-04-25", /image input is required/i],
["wan2.6-i2v-flash", /image input is required/i],
["wan2.7-r2v-2026-06-12", /reference image or video input is required/i],
["wan2.7-videoedit", /video input is required/i],
]) {
const result = await handleVideoGeneration({
body: {
model: `alibaba/${model}`,
prompt: "missing media",
},
credentials: { apiKey: "alibaba-video-key" },
log: null,
});
assert.equal(result.success, false);
assert.equal(result.status, 400);
assert.match(result.error, errorPattern);
}
});
test("Alibaba rejects video models outside its own allowlist", async () => {
const result = await handleVideoGeneration({
body: {
model: "alibaba/wan2.7-t2v",
prompt: "not part of the Alibaba allowlist",
},
credentials: { apiKey: "alibaba-video-key" },
log: null,
});
assert.equal(result.success, false);
assert.equal(result.status, 400);
assert.match(result.error, /unsupported alibaba video model/i);
});

View File

@@ -0,0 +1,233 @@
import test from "node:test";
import assert from "node:assert/strict";
import { mkdtempSync } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
process.env.DATA_DIR = mkdtempSync(join(tmpdir(), "omniroute-bailian-coding-plan-media-"));
const { IMAGE_PROVIDERS, parseImageModel } = await import("../../open-sse/config/imageRegistry.ts");
const { VIDEO_PROVIDERS, parseVideoModel } = await import("../../open-sse/config/videoRegistry.ts");
const { resolveProviderServiceKinds } = await import("../../open-sse/config/mediaServiceKinds.ts");
const { handleImageGeneration } = await import("../../open-sse/handlers/imageGeneration.ts");
const { handleVideoGeneration } = await import("../../open-sse/handlers/videoGeneration.ts");
const { resolveAlibabaProviderMediaBaseUrl } =
await import("../../src/shared/constants/alibabaProviderRegions.ts");
function jsonResponse(body, status = 200) {
return new Response(JSON.stringify(body), {
status,
headers: { "content-type": "application/json" },
});
}
function immediateTimeout(callback, _ms, ...args) {
if (typeof callback === "function") callback(...args);
return 0;
}
test("Bailian Coding Plan owns an independent seven-model media catalog", () => {
assert.deepEqual(
IMAGE_PROVIDERS["bailian-coding-plan"].models.map((model) => model.id),
["wan2.7-image", "wan2.7-image-pro", "qwen-image-2.0", "qwen-image-2.0-pro"]
);
assert.deepEqual(
VIDEO_PROVIDERS["bailian-coding-plan"].models.map((model) => model.id),
["happyhorse-1.1-i2v", "happyhorse-1.1-t2v", "happyhorse-1.1-r2v"]
);
assert.notEqual(
IMAGE_PROVIDERS["bailian-coding-plan"].models,
IMAGE_PROVIDERS["qwen-cloud-token-plan"].models
);
assert.notEqual(
VIDEO_PROVIDERS["bailian-coding-plan"].models,
VIDEO_PROVIDERS["qwen-cloud-token-plan"].models
);
assert.notEqual(
IMAGE_PROVIDERS["bailian-coding-plan"].models,
IMAGE_PROVIDERS["qwen-cloud"].models
);
assert.deepEqual(parseImageModel("bcp/wan2.7-image-pro"), {
provider: "bailian-coding-plan",
model: "wan2.7-image-pro",
});
assert.deepEqual(parseImageModel("bcp/qwen-image-2.0-pro"), {
provider: "bailian-coding-plan",
model: "qwen-image-2.0-pro",
});
assert.deepEqual(parseVideoModel("bcp/happyhorse-1.1-t2v"), {
provider: "bailian-coding-plan",
model: "happyhorse-1.1-t2v",
});
assert.deepEqual(parseImageModel("wan2.7-image"), {
provider: "qwen-cloud-token-plan",
model: "wan2.7-image",
});
assert.deepEqual(parseVideoModel("happyhorse-1.1-t2v"), {
provider: "qwen-cloud-token-plan",
model: "happyhorse-1.1-t2v",
});
assert.deepEqual(resolveProviderServiceKinds("bailian-coding-plan", ["llm"]).sort(), [
"image",
"llm",
"video",
]);
});
test("Bailian Coding Plan media URLs are isolated from both Qwen Cloud providers", () => {
const bailianUrl = resolveAlibabaProviderMediaBaseUrl("bailian-coding-plan", {
region: "global-sg",
});
assert.equal(bailianUrl, "https://coding-intl.dashscope.aliyuncs.com/api/v1");
assert.equal(
resolveAlibabaProviderMediaBaseUrl("bailian-coding-plan", {
region: "china-beijing",
}),
"https://coding.dashscope.aliyuncs.com/api/v1"
);
assert.notEqual(
bailianUrl,
resolveAlibabaProviderMediaBaseUrl("qwen-cloud", { region: "global-sg" })
);
assert.notEqual(
bailianUrl,
resolveAlibabaProviderMediaBaseUrl("qwen-cloud-token-plan", { region: "global-sg" })
);
});
test("Bailian image models use only the Coding Plan endpoint and key", async () => {
const originalFetch = globalThis.fetch;
let captured;
globalThis.fetch = async (url, options = {}) => {
captured = {
url: String(url),
headers: options.headers,
body: JSON.parse(String(options.body || "{}")),
};
return jsonResponse({
output: {
choices: [
{
message: {
content: [{ image: "https://cdn.example.com/bailian-wan.png" }],
},
},
],
},
});
};
try {
for (const model of [
"wan2.7-image",
"wan2.7-image-pro",
"qwen-image-2.0",
"qwen-image-2.0-pro",
]) {
const result = await handleImageGeneration({
body: {
model: `bailian-coding-plan/${model}`,
prompt: "A watercolor horse",
size: "2048x2048",
},
credentials: {
apiKey: "bailian-plan-key",
providerSpecificData: { region: "china-beijing" },
},
log: null,
});
assert.equal(
captured.url,
"https://coding.dashscope.aliyuncs.com/api/v1/services/aigc/multimodal-generation/generation"
);
assert.equal(captured.headers.Authorization, "Bearer bailian-plan-key");
assert.equal(captured.body.model, model);
assert.deepEqual(captured.body.parameters, { size: "2048*2048" });
assert.equal(result.success, true);
}
} finally {
globalThis.fetch = originalFetch;
}
});
test("HappyHorse video generation uses only the Bailian Coding Plan endpoint and key", async () => {
const originalFetch = globalThis.fetch;
const originalSetTimeout = globalThis.setTimeout;
let captured;
globalThis.setTimeout = immediateTimeout;
globalThis.fetch = async (url, options = {}) => {
const stringUrl = String(url);
if (stringUrl.endsWith("/services/aigc/video-generation/video-synthesis")) {
captured = {
url: stringUrl,
headers: options.headers,
body: JSON.parse(String(options.body || "{}")),
};
return jsonResponse({ output: { task_id: "bailian-task", task_status: "PENDING" } });
}
if (stringUrl.endsWith("/tasks/bailian-task")) {
return jsonResponse({
output: {
task_status: "SUCCEEDED",
video_url: "https://cdn.example.com/bailian-happyhorse.mp4",
},
});
}
throw new Error(`Unexpected URL: ${stringUrl}`);
};
try {
const result = await handleVideoGeneration({
body: {
model: "bailian-coding-plan/happyhorse-1.1-i2v",
image_url: "https://cdn.example.com/first-frame.png",
duration: 5,
},
credentials: {
apiKey: "bailian-plan-key",
providerSpecificData: { region: "global-sg" },
},
log: null,
});
assert.equal(
captured.url,
"https://coding-intl.dashscope.aliyuncs.com/api/v1/services/aigc/video-generation/video-synthesis"
);
assert.equal(captured.headers.Authorization, "Bearer bailian-plan-key");
assert.deepEqual(captured.body.input.media, [
{ type: "first_frame", url: "https://cdn.example.com/first-frame.png" },
]);
assert.equal(result.success, true);
} finally {
globalThis.fetch = originalFetch;
globalThis.setTimeout = originalSetTimeout;
}
});
test("Bailian Coding Plan rejects models outside its own media allowlists", async () => {
const imageResult = await handleImageGeneration({
body: {
model: "bailian-coding-plan/qwen-image-3.0-pro",
prompt: "wrong catalog",
},
credentials: { apiKey: "bailian-plan-key" },
log: null,
});
const videoResult = await handleVideoGeneration({
body: {
model: "bailian-coding-plan/wan2.7-videoedit",
prompt: "wrong catalog",
},
credentials: { apiKey: "bailian-plan-key" },
log: null,
});
assert.equal(imageResult.status, 400);
assert.match(imageResult.error, /unsupported bailian-coding-plan image model/i);
assert.equal(videoResult.status, 400);
assert.match(videoResult.error, /unsupported bailian-coding-plan video model/i);
});

View File

@@ -0,0 +1,58 @@
import test from "node:test";
import assert from "node:assert/strict";
import { IMAGE_PROVIDERS } from "../../open-sse/config/imageRegistry.ts";
import { VIDEO_PROVIDERS } from "../../open-sse/config/videoRegistry.ts";
import { toProviderModels } from "../../src/app/(dashboard)/dashboard/cache/media/mediaProviderModels.ts";
function assertAlphabetical(names: string[]) {
const sorted = [...names].sort((left, right) =>
left.localeCompare(right, "en", { sensitivity: "base", numeric: true })
);
assert.deepEqual(names, sorted);
}
test("media page sorts every provider dropdown alphabetically by display name", () => {
const imageProviders = toProviderModels(IMAGE_PROVIDERS);
const videoProviders = toProviderModels(VIDEO_PROVIDERS);
assertAlphabetical(imageProviders.map((provider) => provider.name));
assertAlphabetical(videoProviders.map((provider) => provider.name));
assert.equal(
imageProviders.some((provider) => provider.id === "alibaba"),
true
);
assert.equal(
imageProviders.some((provider) => provider.id === "qwen-cloud"),
true
);
assert.equal(
videoProviders.some((provider) => provider.id === "alibaba"),
true
);
assert.equal(
videoProviders.some((provider) => provider.id === "qwen-cloud"),
true
);
});
test("media page provider mapping preserves provider-scoped model IDs", () => {
const groups = toProviderModels({
example: {
id: "example",
models: [
{ id: "plain-model", name: "Plain Model" },
{ id: "example/scoped-model", name: "Scoped Model" },
],
},
});
assert.deepEqual(groups[0], {
id: "example",
name: "example",
models: [
{ id: "example/plain-model", name: "Plain Model" },
{ id: "example/scoped-model", name: "Scoped Model" },
],
});
});

View File

@@ -11,6 +11,8 @@ const core = await import("../../src/lib/db/core.ts");
const musicRoute = await import("../../src/app/api/v1/music/generations/route.ts");
const videoRoute = await import("../../src/app/api/v1/videos/generations/route.ts");
type ErrorResponseBody = { error: { message: string } };
test.after(() => {
core.resetDbInstance();
fs.rmSync(TEST_DATA_DIR, { recursive: true, force: true });
@@ -26,12 +28,40 @@ test("v1 video generation POST rejects requests without a prompt", async () => {
}),
})
);
const body = (await response.json()) as any;
const body = (await response.json()) as ErrorResponseBody;
assert.equal(response.status, 400);
assert.match(body.error.message, /Prompt is required/);
});
test("Alibaba plan and Qwen Cloud I2V models allow media-only requests", async () => {
for (const model of [
"alibaba/happyhorse-1.1-i2v",
"alibaba/wan2.7-i2v-2026-04-25",
"alibaba/wan2.6-i2v-flash",
"bailian-coding-plan/happyhorse-1.1-i2v",
"qwen-cloud/happyhorse-1.1-i2v",
"qwen-cloud/wan2.7-i2v",
"qwen-cloud-token-plan/happyhorse-1.1-i2v",
]) {
const response = await videoRoute.POST(
new Request("http://localhost/api/v1/videos/generations", {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({
model,
image_url: "https://cdn.example.com/input.png",
}),
})
);
const body = (await response.json()) as ErrorResponseBody;
assert.equal(response.status, 400);
assert.doesNotMatch(body.error.message, /Prompt is required/);
assert.match(body.error.message, /No credentials for video provider/);
}
});
test("v1 music generation POST rejects requests without a prompt", async () => {
const response = await musicRoute.POST(
new Request("http://localhost/api/v1/music/generations", {
@@ -42,7 +72,7 @@ test("v1 music generation POST rejects requests without a prompt", async () => {
}),
})
);
const body = (await response.json()) as any;
const body = (await response.json()) as ErrorResponseBody;
assert.equal(response.status, 400);
assert.match(body.error.message, /Prompt is required/);

View File

@@ -0,0 +1,120 @@
import test from "node:test";
import assert from "node:assert/strict";
import { mkdtempSync } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
process.env.DATA_DIR = mkdtempSync(join(tmpdir(), "omniroute-qwen-cloud-image-"));
const { IMAGE_PROVIDERS, parseImageModel } = await import("../../open-sse/config/imageRegistry.ts");
const { resolveProviderServiceKinds } = await import("../../open-sse/config/mediaServiceKinds.ts");
const { handleImageGeneration } = await import("../../open-sse/handlers/imageGeneration.ts");
function jsonResponse(body, status = 200) {
return new Response(JSON.stringify(body), {
status,
headers: { "content-type": "application/json" },
});
}
test("Qwen Cloud owns an independent six-model image catalog", () => {
assert.deepEqual(
IMAGE_PROVIDERS["qwen-cloud"].models.map((model) => model.id),
[
"wan2.7-image-pro",
"wan2.7-image",
"qwen-image-3.0-pro",
"qwen-image-2.0-pro-2026-06-22",
"qwen-image-2.0-2026-03-03",
"z-image-turbo",
]
);
assert.notEqual(
IMAGE_PROVIDERS["qwen-cloud"].models,
IMAGE_PROVIDERS["qwen-cloud-token-plan"].models
);
assert.notEqual(
IMAGE_PROVIDERS["qwen-cloud"].models,
IMAGE_PROVIDERS["bailian-coding-plan"].models
);
assert.notEqual(IMAGE_PROVIDERS["qwen-cloud"].models, IMAGE_PROVIDERS.alibaba.models);
assert.deepEqual(parseImageModel("qwc/qwen-image-3.0-pro"), {
provider: "qwen-cloud",
model: "qwen-image-3.0-pro",
});
assert.deepEqual(resolveProviderServiceKinds("qwen-cloud", ["llm"]).sort(), [
"image",
"llm",
"video",
]);
});
test("Qwen Cloud images use only the selected Qwen Cloud endpoint and key", async () => {
const originalFetch = globalThis.fetch;
const captures = [];
globalThis.fetch = async (url, options = {}) => {
captures.push({
url: String(url),
headers: options.headers,
body: JSON.parse(String(options.body || "{}")),
});
return jsonResponse({
output: {
choices: [
{
message: {
content: [{ image: "https://cdn.example.com/qwen-cloud-image.png" }],
},
},
],
},
});
};
try {
for (const model of IMAGE_PROVIDERS["qwen-cloud"].models.map((entry) => entry.id)) {
const result = await handleImageGeneration({
body: {
model: `qwen-cloud/${model}`,
prompt: "A horse above the clouds",
size: "2048x2048",
},
credentials: {
apiKey: "qwen-cloud-key",
providerSpecificData: { region: "china-beijing" },
},
log: null,
});
assert.equal(result.success, true);
}
assert.equal(captures.length, 6);
for (const capture of captures) {
assert.equal(
capture.url,
"https://dashscope.aliyuncs.com/api/v1/services/aigc/multimodal-generation/generation"
);
assert.equal(capture.headers.Authorization, "Bearer qwen-cloud-key");
assert.deepEqual(capture.body.input.messages[0].content, [
{ text: "A horse above the clouds" },
]);
}
} finally {
globalThis.fetch = originalFetch;
}
});
test("Qwen Cloud rejects image models outside its own allowlist", async () => {
const result = await handleImageGeneration({
body: {
model: "qwen-cloud/qwen-image-2.0",
prompt: "wrong catalog",
},
credentials: { apiKey: "qwen-cloud-key" },
log: null,
});
assert.equal(result.status, 400);
assert.match(result.error, /unsupported qwen-cloud image model/i);
});

View File

@@ -0,0 +1,242 @@
import test from "node:test";
import assert from "node:assert/strict";
import { mkdtempSync } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
process.env.DATA_DIR = mkdtempSync(join(tmpdir(), "omniroute-qwen-token-plan-media-"));
const { IMAGE_PROVIDERS, parseImageModel } = await import("../../open-sse/config/imageRegistry.ts");
const { VIDEO_PROVIDERS, parseVideoModel } = await import("../../open-sse/config/videoRegistry.ts");
const { resolveProviderServiceKinds } = await import("../../open-sse/config/mediaServiceKinds.ts");
const { handleImageGeneration } = await import("../../open-sse/handlers/imageGeneration.ts");
const { handleVideoGeneration } = await import("../../open-sse/handlers/videoGeneration.ts");
const { resolveAlibabaProviderMediaBaseUrl } =
await import("../../src/shared/constants/alibabaProviderRegions.ts");
function jsonResponse(body, status = 200) {
return new Response(JSON.stringify(body), {
status,
headers: { "content-type": "application/json" },
});
}
function immediateTimeout(callback, _ms, ...args) {
if (typeof callback === "function") callback(...args);
return 0;
}
test("Qwen Cloud Token Plan registers Wan image and HappyHorse video models", () => {
assert.deepEqual(
IMAGE_PROVIDERS["qwen-cloud-token-plan"].models.map((model) => model.id),
["wan2.7-image", "wan2.7-image-pro"]
);
assert.deepEqual(
VIDEO_PROVIDERS["qwen-cloud-token-plan"].models.map((model) => model.id),
["happyhorse-1.1-i2v", "happyhorse-1.1-t2v", "happyhorse-1.1-r2v"]
);
assert.deepEqual(parseImageModel("qct/wan2.7-image-pro"), {
provider: "qwen-cloud-token-plan",
model: "wan2.7-image-pro",
});
assert.deepEqual(parseVideoModel("qct/happyhorse-1.1-t2v"), {
provider: "qwen-cloud-token-plan",
model: "happyhorse-1.1-t2v",
});
assert.deepEqual(resolveProviderServiceKinds("qwen-cloud-token-plan", ["llm"]).sort(), [
"image",
"llm",
"video",
]);
});
test("Qwen Cloud Token Plan media URLs follow the saved region", () => {
assert.equal(
resolveAlibabaProviderMediaBaseUrl("qwen-cloud-token-plan", { region: "global-sg" }),
"https://token-plan.ap-southeast-1.maas.aliyuncs.com/api/v1"
);
assert.equal(
resolveAlibabaProviderMediaBaseUrl("qwen-cloud-token-plan", {
region: "china-beijing",
}),
"https://token-plan.cn-beijing.maas.aliyuncs.com/api/v1"
);
});
test("Wan image generation uses the Token Plan multimodal endpoint and normalizes URLs", async () => {
const originalFetch = globalThis.fetch;
let captured;
globalThis.fetch = async (url, options = {}) => {
captured = {
url: String(url),
headers: options.headers,
body: JSON.parse(String(options.body || "{}")),
};
return jsonResponse({
output: {
choices: [
{
message: {
content: [
{ image: "https://cdn.example.com/wan-1.png" },
{ image: "https://cdn.example.com/wan-2.png" },
],
},
},
],
},
});
};
try {
const result = await handleImageGeneration({
body: {
model: "qwen-cloud-token-plan/wan2.7-image-pro",
prompt: "A watercolor horse",
size: "2048x2048",
n: 2,
},
credentials: {
apiKey: "token-plan-key",
providerSpecificData: { region: "china-beijing" },
},
log: null,
});
assert.equal(
captured.url,
"https://token-plan.cn-beijing.maas.aliyuncs.com/api/v1/services/aigc/multimodal-generation/generation"
);
assert.equal(captured.headers.Authorization, "Bearer token-plan-key");
assert.equal(captured.body.model, "wan2.7-image-pro");
assert.deepEqual(captured.body.input.messages[0].content, [{ text: "A watercolor horse" }]);
assert.deepEqual(captured.body.parameters, { size: "2048*2048", n: 2 });
assert.equal(result.success, true);
assert.deepEqual(
result.data.data.map((item) => item.url),
["https://cdn.example.com/wan-1.png", "https://cdn.example.com/wan-2.png"]
);
} finally {
globalThis.fetch = originalFetch;
}
});
async function captureHappyHorseRequest(body) {
const originalFetch = globalThis.fetch;
const originalSetTimeout = globalThis.setTimeout;
let captured;
globalThis.setTimeout = immediateTimeout;
globalThis.fetch = async (url, options = {}) => {
const stringUrl = String(url);
if (stringUrl.endsWith("/services/aigc/video-generation/video-synthesis")) {
captured = {
url: stringUrl,
headers: options.headers,
body: JSON.parse(String(options.body || "{}")),
};
return jsonResponse({ output: { task_id: "happyhorse-task", task_status: "PENDING" } });
}
if (stringUrl.endsWith("/tasks/happyhorse-task")) {
return jsonResponse({
output: {
task_status: "SUCCEEDED",
video_url: "https://cdn.example.com/happyhorse.mp4",
},
});
}
throw new Error(`Unexpected URL: ${stringUrl}`);
};
try {
const result = await handleVideoGeneration({
body,
credentials: {
apiKey: "token-plan-key",
providerSpecificData: { region: "global-sg" },
},
log: null,
});
return { captured, result };
} finally {
globalThis.fetch = originalFetch;
globalThis.setTimeout = originalSetTimeout;
}
}
test("HappyHorse T2V uses resolution, ratio, and async Token Plan polling", async () => {
const { captured, result } = await captureHappyHorseRequest({
model: "qwen-cloud-token-plan/happyhorse-1.1-t2v",
prompt: "A horse running through clouds",
size: "1920x1080",
duration: 8,
watermark: false,
});
assert.equal(
captured.url,
"https://token-plan.ap-southeast-1.maas.aliyuncs.com/api/v1/services/aigc/video-generation/video-synthesis"
);
assert.deepEqual(captured.body.input, { prompt: "A horse running through clouds" });
assert.deepEqual(captured.body.parameters, {
resolution: "1080P",
ratio: "16:9",
duration: 8,
watermark: false,
});
assert.equal(result.success, true);
assert.equal(result.data.data[0].url, "https://cdn.example.com/happyhorse.mp4");
});
test("HappyHorse I2V maps an image URL to a first-frame media item", async () => {
const { captured } = await captureHappyHorseRequest({
model: "qwen-cloud-token-plan/happyhorse-1.1-i2v",
prompt: "The horse starts running",
image_url: "https://cdn.example.com/horse.png",
resolution: "720p",
aspect_ratio: "16:9",
duration: 5,
});
assert.deepEqual(captured.body.input.media, [
{ type: "first_frame", url: "https://cdn.example.com/horse.png" },
]);
assert.deepEqual(captured.body.parameters, {
resolution: "720P",
duration: 5,
});
});
test("HappyHorse R2V maps reference images in order", async () => {
const { captured } = await captureHappyHorseRequest({
model: "qwen-cloud-token-plan/happyhorse-1.1-r2v",
prompt: "[Image 1] rides beside [Image 2]",
reference_images: ["https://cdn.example.com/horse.png", "https://cdn.example.com/rider.png"],
ratio: "4:3",
duration: 6,
});
assert.deepEqual(captured.body.input.media, [
{ type: "reference_image", url: "https://cdn.example.com/horse.png" },
{ type: "reference_image", url: "https://cdn.example.com/rider.png" },
]);
assert.equal(captured.body.parameters.ratio, "4:3");
});
test("HappyHorse I2V and R2V reject missing image input before calling upstream", async () => {
for (const model of ["happyhorse-1.1-i2v", "happyhorse-1.1-r2v"]) {
const result = await handleVideoGeneration({
body: {
model: `qwen-cloud-token-plan/${model}`,
prompt: "missing input",
},
credentials: { apiKey: "token-plan-key" },
log: null,
});
assert.equal(result.success, false);
assert.equal(result.status, 400);
assert.match(result.error, /image input is required/i);
}
});

View File

@@ -0,0 +1,282 @@
import test from "node:test";
import assert from "node:assert/strict";
import { mkdtempSync } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
process.env.DATA_DIR = mkdtempSync(join(tmpdir(), "omniroute-qwen-cloud-video-"));
const { VIDEO_PROVIDERS, parseVideoModel } = await import("../../open-sse/config/videoRegistry.ts");
const { resolveProviderServiceKinds } = await import("../../open-sse/config/mediaServiceKinds.ts");
const { handleVideoGeneration } = await import("../../open-sse/handlers/videoGeneration.ts");
const { resolveAlibabaProviderMediaBaseUrl } =
await import("../../src/shared/constants/alibabaProviderRegions.ts");
function jsonResponse(body, status = 200) {
return new Response(JSON.stringify(body), {
status,
headers: { "content-type": "application/json" },
});
}
function immediateTimeout(callback, _ms, ...args) {
if (typeof callback === "function") callback(...args);
return 0;
}
async function captureQwenCloudRequest(body, region = "global-sg") {
const originalFetch = globalThis.fetch;
const originalSetTimeout = globalThis.setTimeout;
let captured;
globalThis.setTimeout = immediateTimeout;
globalThis.fetch = async (url, options = {}) => {
const stringUrl = String(url);
if (stringUrl.endsWith("/services/aigc/video-generation/video-synthesis")) {
captured = {
url: stringUrl,
headers: options.headers,
body: JSON.parse(String(options.body || "{}")),
};
return jsonResponse({ output: { task_id: "qwen-cloud-task", task_status: "PENDING" } });
}
if (stringUrl.endsWith("/tasks/qwen-cloud-task")) {
return jsonResponse({
output: {
task_status: "SUCCEEDED",
video_url: "https://cdn.example.com/qwen-cloud.mp4",
},
});
}
throw new Error(`Unexpected URL: ${stringUrl}`);
};
try {
const result = await handleVideoGeneration({
body,
credentials: {
apiKey: "qwen-cloud-key",
providerSpecificData: { region },
},
log: null,
});
return { captured, result };
} finally {
globalThis.fetch = originalFetch;
globalThis.setTimeout = originalSetTimeout;
}
}
test("Qwen Cloud and Token Plan keep independent video model lists", () => {
assert.deepEqual(
VIDEO_PROVIDERS["qwen-cloud"].models.map((model) => model.id),
[
"happyhorse-1.1-i2v",
"happyhorse-1.1-t2v",
"happyhorse-1.1-r2v",
"happyhorse-1.0-video-edit",
"wan2.7-t2v",
"wan2.7-i2v",
"wan2.7-r2v-2026-06-12",
"wan2.7-videoedit",
]
);
assert.deepEqual(
VIDEO_PROVIDERS["qwen-cloud-token-plan"].models.map((model) => model.id),
["happyhorse-1.1-i2v", "happyhorse-1.1-t2v", "happyhorse-1.1-r2v"]
);
assert.equal(
VIDEO_PROVIDERS["qwen-cloud"].models.some((model) => model.id === "wan2.1-vace-plus"),
false
);
assert.deepEqual(parseVideoModel("qwc/wan2.7-videoedit"), {
provider: "qwen-cloud",
model: "wan2.7-videoedit",
});
assert.deepEqual(parseVideoModel("qct/happyhorse-1.1-t2v"), {
provider: "qwen-cloud-token-plan",
model: "happyhorse-1.1-t2v",
});
assert.deepEqual(resolveProviderServiceKinds("qwen-cloud", ["llm"]).sort(), [
"image",
"llm",
"video",
]);
});
test("Qwen Cloud media URLs follow its own saved region", () => {
assert.equal(
resolveAlibabaProviderMediaBaseUrl("qwen-cloud", { region: "global-sg" }),
"https://dashscope-intl.aliyuncs.com/api/v1"
);
assert.equal(
resolveAlibabaProviderMediaBaseUrl("qwen-cloud", { region: "china-beijing" }),
"https://dashscope.aliyuncs.com/api/v1"
);
assert.notEqual(
resolveAlibabaProviderMediaBaseUrl("qwen-cloud", { region: "global-sg" }),
resolveAlibabaProviderMediaBaseUrl("qwen-cloud-token-plan", { region: "global-sg" })
);
});
test("Qwen Cloud HappyHorse I2V uses Qwen Cloud credentials and endpoint", async () => {
const { captured, result } = await captureQwenCloudRequest({
model: "qwen-cloud/happyhorse-1.1-i2v",
image_url: "https://cdn.example.com/horse.png",
resolution: "720p",
duration: 5,
});
assert.equal(
captured.url,
"https://dashscope-intl.aliyuncs.com/api/v1/services/aigc/video-generation/video-synthesis"
);
assert.equal(captured.headers.Authorization, "Bearer qwen-cloud-key");
assert.deepEqual(captured.body.input.media, [
{ type: "first_frame", url: "https://cdn.example.com/horse.png" },
]);
assert.deepEqual(captured.body.parameters, {
resolution: "720P",
duration: 5,
});
assert.equal(result.success, true);
});
test("Qwen Cloud Wan T2V maps modern resolution, ratio, and audio input", async () => {
const { captured } = await captureQwenCloudRequest({
model: "qwen-cloud/wan2.7-t2v",
prompt: "A cinematic horse race",
audio_url: "https://cdn.example.com/score.mp3",
size: "1920x1080",
duration: 10,
prompt_extend: true,
watermark: false,
});
assert.deepEqual(captured.body.input, {
prompt: "A cinematic horse race",
audio_url: "https://cdn.example.com/score.mp3",
});
assert.deepEqual(captured.body.parameters, {
resolution: "1080P",
ratio: "16:9",
duration: 10,
prompt_extend: true,
watermark: false,
});
});
test("Qwen Cloud Wan I2V maps first-frame and driving-audio media", async () => {
const { captured } = await captureQwenCloudRequest({
model: "qwen-cloud/wan2.7-i2v",
image_url: "https://cdn.example.com/first.png",
audio_url: "https://cdn.example.com/voice.mp3",
duration: 10,
});
assert.deepEqual(captured.body.input.media, [
{ type: "first_frame", url: "https://cdn.example.com/first.png" },
{ type: "driving_audio", url: "https://cdn.example.com/voice.mp3" },
]);
assert.equal(captured.body.parameters.duration, 10);
assert.equal("ratio" in captured.body.parameters, false);
});
test("Qwen Cloud Wan R2V preserves ordered image, video, and voice references", async () => {
const { captured } = await captureQwenCloudRequest({
model: "qwen-cloud/wan2.7-r2v-2026-06-12",
prompt: "Video 1 greets Image 1",
media: [
{
type: "reference_video",
url: "https://cdn.example.com/actor.mp4",
reference_voice: "https://cdn.example.com/actor.mp3",
},
{
type: "reference_image",
url: "https://cdn.example.com/prop.png",
},
],
ratio: "16:9",
duration: 10,
});
assert.deepEqual(captured.body.input.media, [
{
type: "reference_video",
url: "https://cdn.example.com/actor.mp4",
reference_voice: "https://cdn.example.com/actor.mp3",
},
{
type: "reference_image",
url: "https://cdn.example.com/prop.png",
},
]);
assert.equal(captured.body.parameters.ratio, "16:9");
});
test("Qwen Cloud video-edit models require and map a source video", async () => {
for (const model of ["happyhorse-1.0-video-edit", "wan2.7-videoedit"]) {
const { captured } = await captureQwenCloudRequest({
model: `qwen-cloud/${model}`,
prompt: "Replace the jacket with the reference image",
video_url: "https://cdn.example.com/source.mp4",
reference_images: ["https://cdn.example.com/jacket.png"],
resolution: "1080P",
});
assert.deepEqual(captured.body.input.media, [
{ type: "video", url: "https://cdn.example.com/source.mp4" },
{ type: "reference_image", url: "https://cdn.example.com/jacket.png" },
]);
assert.equal(captured.body.parameters.resolution, "1080P");
}
});
test("Qwen Cloud media-specific models reject missing required inputs locally", async () => {
for (const [model, errorPattern] of [
["happyhorse-1.1-i2v", /image input is required/i],
["happyhorse-1.1-r2v", /image input is required/i],
["happyhorse-1.0-video-edit", /video input is required/i],
["wan2.7-i2v", /image input is required/i],
["wan2.7-r2v-2026-06-12", /reference image or video input is required/i],
["wan2.7-videoedit", /video input is required/i],
]) {
const result = await handleVideoGeneration({
body: {
model: `qwen-cloud/${model}`,
prompt: "missing media",
},
credentials: { apiKey: "qwen-cloud-key" },
log: null,
});
assert.equal(result.success, false);
assert.equal(result.status, 400);
assert.match(result.error, errorPattern);
}
});
test("Qwen provider prefixes cannot cross their registered model lists", async () => {
const qwenCloudResult = await handleVideoGeneration({
body: {
model: "qwen-cloud/wan2.7-image",
prompt: "wrong registry",
},
credentials: { apiKey: "qwen-cloud-key" },
log: null,
});
const tokenPlanResult = await handleVideoGeneration({
body: {
model: "qwen-cloud-token-plan/wan2.7-videoedit",
prompt: "wrong registry",
},
credentials: { apiKey: "token-plan-key" },
log: null,
});
assert.equal(qwenCloudResult.status, 400);
assert.match(qwenCloudResult.error, /unsupported qwen-cloud video model/i);
assert.equal(tokenPlanResult.status, 400);
assert.match(tokenPlanResult.error, /unsupported qwen-cloud-token-plan video model/i);
});