fix(providers): resolve combo names on /v1/audio/speech and /v1/videos/generations (#10471)

* fix(providers): resolve combo names on /v1/audio/speech and /v1/videos/generations

`GET /v1/models` advertises combos with `owned_by: combo`, and chat, embeddings,
transcriptions (#9134) and images (#8986, #9239) all resolve those names. Speech
and video did not: both rejected a combo name at model validation, before any
resolution could happen.

    POST /v1/audio/speech        {"model":"my-combo","input":"hi"}
    -> 400 Invalid speech model: my-combo. Use format: provider/model

    POST /v1/videos/generations  {"model":"my-combo","prompt":"a cube"}
    -> 400 Invalid video model: my-combo. Use format: provider/model

A client picking a model out of /v1/models therefore could not tell which
entries the catalogue would actually accept, and callers ended up hardcoding
vendor ids for these two routes while using combo names everywhere else.

Both routes now mirror the images route: detect a combo name before the
provider lookup and divert to a strategy executor. The two new executors follow
imageCombo — expand targets with resolveComboTargets(), filter to targets the
route can actually serve, walk them in priority order, and return the first
success or the last failure, with 400/401/403 treated as terminal.

Two details differ from the image strategy:

Speech filters at model level rather than provider level. parseSpeechModel()
resolves a provider prefix without checking that the model behind it can speak,
so `openai/gpt-4o` would otherwise be accepted as a target and fail only once
dispatched. The filter now checks the provider's own model list, and keeps
targets from dynamic provider nodes that do not enumerate models.

Speech also returns the handler's Response untouched instead of building a JSON
body, because that route streams audio; only the ADD-only meta headers are
attached, exactly as the direct path does. The failure branch is the only place
the body is read.

successfulMediaGenerationResponse() gains optional `strategy` and
`fallbackAttempts` so the video strategy can report them the way imageCombo
does, rather than duplicating the cost calculation. Both are omitted on the
direct single-model path, where neither is meaningful.

Tests mirror tests/unit/combo/image-combo.test.ts for both routes: combo not
found, no capable targets, empty combo, and targets present with no provider
connection. 16/16 pass across the three combo test files.

* fix(providers): preserve local overrides, custom models and per-target prompt rules through video combo dispatch

executeVideoCombo() diverged from the direct /v1/videos/generations route in
three ways: it dropped the ComfyUI-style local-override credential lookup for
authType:"none" targets, its capability filter only matched the built-in
video registry (skipping custom OpenAI-compatible provider nodes tagged with
the "videos" endpoint), and the route validated the prompt against the
unresolved combo name before combo targets were expanded — rejecting
prompt-optional I2V targets that never got the chance to opt out.

Extracts the shared resolution rules (resolveVideoModelTarget,
isVideoPromptOptional, resolveLocalOverrideCredentials) into
src/app/api/v1/_shared/videoModelResolution.ts so the direct route and the
combo executor apply identical rules, moves the combo-name diversion ahead of
the prompt-required check so validation runs against the real resolved
target, and adds per-target prompt validation inside the combo loop so a
missing prompt only rules out that target instead of the whole combo.

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>

---------

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
This commit is contained in:
sha367
2026-08-18 17:50:31 +04:00
committed by GitHub
parent a55daacc49
commit 09680013de
9 changed files with 1016 additions and 53 deletions

View File

@@ -112,6 +112,8 @@ export async function successfulMediaGenerationResponse({
model,
startTime,
duration,
strategy,
fallbackAttempts,
}: {
result: { data: unknown };
billingMode: "audio" | "video";
@@ -119,6 +121,11 @@ export async function successfulMediaGenerationResponse({
model: string;
startTime: number;
duration: unknown;
// Set by combo execution so the response reports which strategy picked the
// target and how many earlier targets were skipped. Omitted on the direct
// single-model path, where neither is meaningful.
strategy?: string;
fallbackAttempts?: number;
}) {
const seconds = Number(duration) || 0;
const costUsd = await calculateModalCost(billingMode, provider, model, { seconds });
@@ -129,6 +136,8 @@ export async function successfulMediaGenerationResponse({
costUsd,
latencyMs: Date.now() - startTime,
requestId: generateRequestId(),
...(strategy ? { strategy } : {}),
...(fallbackAttempts !== undefined ? { fallbackAttempts } : {}),
});
return new Response(JSON.stringify(result.data), {

View File

@@ -0,0 +1,81 @@
import { getAllCustomModels } from "@/lib/db/models";
import { parseVideoModel } from "@omniroute/open-sse/config/videoRegistry.ts";
import { getProviderCredentialsWithQuotaPreflight } from "@/sse/services/auth";
import { isAllRateLimitedCredentials } from "@/app/api/v1/_shared/rateLimit";
export type VideoModelTarget = {
provider: string | null;
model: string | null;
isCustomModel: boolean;
};
/**
* Resolve a `provider/model` string to a video target, checking the built-in
* video registry first and falling back to custom OpenAI-compatible provider
* nodes tagged with the "videos" endpoint (mirrors the direct
* /v1/videos/generations route's custom-model scan). Shared by the direct
* route and the combo executor so both paths cover the same target set.
*/
export async function resolveVideoModelTarget(
modelStr: string | null | undefined
): Promise<VideoModelTarget> {
const parsed = parseVideoModel(modelStr ?? null);
if (parsed.provider) {
return { provider: parsed.provider, model: parsed.model, isCustomModel: false };
}
if (!modelStr) {
return { provider: null, model: null, isCustomModel: false };
}
try {
const customModelsMap = (await getAllCustomModels()) as Record<string, unknown>;
for (const [providerId, models] of Object.entries(customModelsMap)) {
if (!Array.isArray(models)) continue;
for (const model of models as Array<{ id?: string; supportedEndpoints?: unknown }>) {
if (!model?.id || !Array.isArray(model.supportedEndpoints)) continue;
if (!model.supportedEndpoints.includes("videos")) continue;
const fullId = `${providerId}/${model.id}`;
if (fullId === modelStr) {
return { provider: providerId, model: model.id, isCustomModel: true };
}
}
}
} catch {
// registry read failure — fall through to "not found"
}
return { provider: null, model: null, isCustomModel: false };
}
/**
* I2V models that accept an omitted prompt (the source image already carries
* enough context). Mirrors the allow-list the direct route applies before
* promptRequiredResponse(); kept here so combo targets get identical rules.
*/
export function isVideoPromptOptional(parsed: { provider: string | null; model: string | null }) {
return (
(parsed.model === "happyhorse-1.1-i2v" &&
(parsed.provider === "alibaba" ||
parsed.provider === "bailian-coding-plan" ||
parsed.provider === "qwen-cloud-token-plan" ||
parsed.provider === "qwen-cloud")) ||
(parsed.provider === "qwen-cloud" && parsed.model === "wan2.7-i2v") ||
(parsed.provider === "alibaba" &&
(parsed.model === "wan2.7-i2v-2026-04-25" || parsed.model === "wan2.6-i2v-flash"))
);
}
/**
* #6928: best-effort per-connection base-URL override lookup for local no-auth
* media providers (ComfyUI). Returns null instead of failing when no connection
* exists — local providers must keep working with zero configuration. Shared
* by the direct route and the combo executor so a configured ComfyUI base URL
* is honored the same way regardless of which path dispatched the request.
*/
export async function resolveLocalOverrideCredentials(provider: string) {
const localCredentials = await getProviderCredentialsWithQuotaPreflight(provider);
return localCredentials && !isAllRateLimitedCredentials(localCredentials)
? localCredentials
: null;
}

View File

@@ -55,6 +55,19 @@ async function postHandler(request, context) {
const policy = await enforceApiKeyPolicy(request, body.model);
if (policy.rejection) return policy.rejection;
// Detect a combo name and divert to full speech combo execution, mirroring
// the images route. Checks before parseSpeechModel so a combo name is never
// rejected as an invalid `provider/model` id — /v1/models advertises these
// names, so refusing them here made the catalogue dishonest.
if (body.model && typeof body.model === "string" && !body.model.includes("/")) {
const { getComboByName } = await import("@/lib/db/combos");
const combo = await getComboByName(body.model);
if (combo) {
const { executeSpeechCombo } = await import("@omniroute/open-sse/services/speechCombo");
return executeSpeechCombo(body.model, body, { request, policy }, startTime);
}
}
// Provider nodes eligible for speech: this route's own audio type plus general
// chat/responses gateways. Remote hosts are opt-in (default OFF).
const dynamicProviders = await resolveDynamicAudioProviders("/audio/speech", "audio-speech");

View File

@@ -1,12 +1,11 @@
import { handleVideoGeneration } from "@omniroute/open-sse/handlers/videoGeneration.ts";
import { resolveVideoCredentialProvider } from "@omniroute/open-sse/handlers/videoGeneration/googleFlow.ts";
import { withInjectionGuard } from "@/middleware/promptInjectionGuard";
import { getAllCustomModels } from "@/lib/db/models";
import {
getProviderCredentialsWithQuotaPreflight,
clearRecoveredProviderState,
} from "@/sse/services/auth";
import { parseVideoModel, getVideoProvider } from "@omniroute/open-sse/config/videoRegistry.ts";
import { getVideoProvider } from "@omniroute/open-sse/config/videoRegistry.ts";
import { errorResponse } from "@omniroute/open-sse/utils/error.ts";
import { HTTP_STATUS } from "@omniroute/open-sse/config/constants.ts";
import * as log from "@/sse/utils/logger";
@@ -25,6 +24,11 @@ import {
} from "@/app/api/v1/_shared/mediaGenerationRoute";
import type { MediaGenerationResultLike } from "@/app/api/v1/_shared/mediaGenerationRoute";
import { getSpecialtyModelsResponse } from "@/app/api/v1/_shared/specialtyCatalog";
import {
isVideoPromptOptional,
resolveLocalOverrideCredentials,
resolveVideoModelTarget,
} from "@/app/api/v1/_shared/videoModelResolution";
export const dynamic = "force-dynamic";
@@ -46,18 +50,6 @@ export async function GET(request?: Request) {
);
}
/**
* #6928: best-effort per-connection base-URL override lookup for local no-auth
* media providers (ComfyUI). Returns null instead of failing when no connection
* exists — local providers must keep working with zero configuration.
*/
async function resolveLocalOverrideCredentials(provider) {
const localCredentials = await getProviderCredentialsWithQuotaPreflight(provider);
return localCredentials && !isAllRateLimitedCredentials(localCredentials)
? localCredentials
: null;
}
/**
* POST /v1/videos/generations — generate videos
*/
@@ -68,52 +60,30 @@ async function postHandler(request, context) {
}
const body = parsed.body;
const startTime = Date.now();
const parsedModel = parseVideoModel(body.model);
const promptOptional =
(parsedModel.model === "happyhorse-1.1-i2v" &&
(parsedModel.provider === "alibaba" ||
parsedModel.provider === "bailian-coding-plan" ||
parsedModel.provider === "qwen-cloud-token-plan" ||
parsedModel.provider === "qwen-cloud")) ||
(parsedModel.provider === "qwen-cloud" && parsedModel.model === "wan2.7-i2v") ||
(parsedModel.provider === "alibaba" &&
(parsedModel.model === "wan2.7-i2v-2026-04-25" || parsedModel.model === "wan2.6-i2v-flash"));
if (!promptOptional) {
const promptError = promptRequiredResponse(body);
if (promptError) return promptError;
}
// Enforce API key policies (model restrictions + budget limits)
const policy = await enforceApiKeyPolicy(request, body.model);
if (policy.rejection) return policy.rejection;
// Parse model to get provider
let { provider, model: requestedModel } = parsedModel;
let isCustomModel = false;
if (!provider) {
// Custom OpenAI-compatible provider nodes (mirrors images route): scan the
// dynamic model registry for a matching `${nodeId}/${modelId}` entry.
try {
const customModelsMap = (await getAllCustomModels()) as Record<string, any>;
for (const [providerId, models] of Object.entries(customModelsMap)) {
if (!Array.isArray(models)) continue;
for (const model of models) {
if (!model?.id || !Array.isArray(model.supportedEndpoints)) continue;
if (!model.supportedEndpoints.includes("videos")) continue;
const fullId = `${providerId}/${model.id}`;
if (fullId === body.model) {
provider = providerId;
requestedModel = model.id;
isCustomModel = true;
break;
}
}
}
} catch {
// registry read failure — fall through to invalid-model error below
// Detect a combo name and divert to full video combo execution, mirroring
// the images route. Checks before the provider lookup — and before the
// prompt-required check below — so a combo name is never rejected as an
// invalid `provider/model` id or against the wrong model's prompt rules:
// /v1/models advertises these names, and prompt requirements depend on the
// resolved target, which for a combo is only known after expansion.
if (body.model && typeof body.model === "string" && !body.model.includes("/")) {
const { getComboByName } = await import("@/lib/db/combos");
const combo = await getComboByName(body.model);
if (combo) {
const { executeVideoCombo } = await import("@omniroute/open-sse/services/videoCombo");
return executeVideoCombo(body.model, body, { request, policy }, startTime, log);
}
}
// Parse model to get provider — checks the built-in registry, then custom
// OpenAI-compatible provider nodes tagged with the "videos" endpoint.
const resolvedTarget = await resolveVideoModelTarget(body.model);
const { provider, model: requestedModel, isCustomModel } = resolvedTarget;
if (!provider) {
return errorResponse(
HTTP_STATUS.BAD_REQUEST,
@@ -121,6 +91,11 @@ async function postHandler(request, context) {
);
}
if (!isVideoPromptOptional(resolvedTarget)) {
const promptError = promptRequiredResponse(body);
if (promptError) return promptError;
}
// Check provider config for auth bypass
const providerConfig = getVideoProvider(provider);