mirror of
https://github.com/diegosouzapw/OmniRoute.git
synced 2026-08-19 05:32:19 +03:00
fix(providers): resolve combo names on /v1/audio/speech and /v1/videos/generations (#10471)
* fix(providers): resolve combo names on /v1/audio/speech and /v1/videos/generations `GET /v1/models` advertises combos with `owned_by: combo`, and chat, embeddings, transcriptions (#9134) and images (#8986, #9239) all resolve those names. Speech and video did not: both rejected a combo name at model validation, before any resolution could happen. POST /v1/audio/speech {"model":"my-combo","input":"hi"} -> 400 Invalid speech model: my-combo. Use format: provider/model POST /v1/videos/generations {"model":"my-combo","prompt":"a cube"} -> 400 Invalid video model: my-combo. Use format: provider/model A client picking a model out of /v1/models therefore could not tell which entries the catalogue would actually accept, and callers ended up hardcoding vendor ids for these two routes while using combo names everywhere else. Both routes now mirror the images route: detect a combo name before the provider lookup and divert to a strategy executor. The two new executors follow imageCombo — expand targets with resolveComboTargets(), filter to targets the route can actually serve, walk them in priority order, and return the first success or the last failure, with 400/401/403 treated as terminal. Two details differ from the image strategy: Speech filters at model level rather than provider level. parseSpeechModel() resolves a provider prefix without checking that the model behind it can speak, so `openai/gpt-4o` would otherwise be accepted as a target and fail only once dispatched. The filter now checks the provider's own model list, and keeps targets from dynamic provider nodes that do not enumerate models. Speech also returns the handler's Response untouched instead of building a JSON body, because that route streams audio; only the ADD-only meta headers are attached, exactly as the direct path does. The failure branch is the only place the body is read. successfulMediaGenerationResponse() gains optional `strategy` and `fallbackAttempts` so the video strategy can report them the way imageCombo does, rather than duplicating the cost calculation. Both are omitted on the direct single-model path, where neither is meaningful. Tests mirror tests/unit/combo/image-combo.test.ts for both routes: combo not found, no capable targets, empty combo, and targets present with no provider connection. 16/16 pass across the three combo test files. * fix(providers): preserve local overrides, custom models and per-target prompt rules through video combo dispatch executeVideoCombo() diverged from the direct /v1/videos/generations route in three ways: it dropped the ComfyUI-style local-override credential lookup for authType:"none" targets, its capability filter only matched the built-in video registry (skipping custom OpenAI-compatible provider nodes tagged with the "videos" endpoint), and the route validated the prompt against the unresolved combo name before combo targets were expanded — rejecting prompt-optional I2V targets that never got the chance to opt out. Extracts the shared resolution rules (resolveVideoModelTarget, isVideoPromptOptional, resolveLocalOverrideCredentials) into src/app/api/v1/_shared/videoModelResolution.ts so the direct route and the combo executor apply identical rules, moves the combo-name diversion ahead of the prompt-required check so validation runs against the real resolved target, and adds per-target prompt validation inside the combo loop so a missing prompt only rules out that target instead of the whole combo. Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com> --------- Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
This commit is contained in:
@@ -112,6 +112,8 @@ export async function successfulMediaGenerationResponse({
|
||||
model,
|
||||
startTime,
|
||||
duration,
|
||||
strategy,
|
||||
fallbackAttempts,
|
||||
}: {
|
||||
result: { data: unknown };
|
||||
billingMode: "audio" | "video";
|
||||
@@ -119,6 +121,11 @@ export async function successfulMediaGenerationResponse({
|
||||
model: string;
|
||||
startTime: number;
|
||||
duration: unknown;
|
||||
// Set by combo execution so the response reports which strategy picked the
|
||||
// target and how many earlier targets were skipped. Omitted on the direct
|
||||
// single-model path, where neither is meaningful.
|
||||
strategy?: string;
|
||||
fallbackAttempts?: number;
|
||||
}) {
|
||||
const seconds = Number(duration) || 0;
|
||||
const costUsd = await calculateModalCost(billingMode, provider, model, { seconds });
|
||||
@@ -129,6 +136,8 @@ export async function successfulMediaGenerationResponse({
|
||||
costUsd,
|
||||
latencyMs: Date.now() - startTime,
|
||||
requestId: generateRequestId(),
|
||||
...(strategy ? { strategy } : {}),
|
||||
...(fallbackAttempts !== undefined ? { fallbackAttempts } : {}),
|
||||
});
|
||||
|
||||
return new Response(JSON.stringify(result.data), {
|
||||
|
||||
81
src/app/api/v1/_shared/videoModelResolution.ts
Normal file
81
src/app/api/v1/_shared/videoModelResolution.ts
Normal file
@@ -0,0 +1,81 @@
|
||||
import { getAllCustomModels } from "@/lib/db/models";
|
||||
import { parseVideoModel } from "@omniroute/open-sse/config/videoRegistry.ts";
|
||||
import { getProviderCredentialsWithQuotaPreflight } from "@/sse/services/auth";
|
||||
import { isAllRateLimitedCredentials } from "@/app/api/v1/_shared/rateLimit";
|
||||
|
||||
export type VideoModelTarget = {
|
||||
provider: string | null;
|
||||
model: string | null;
|
||||
isCustomModel: boolean;
|
||||
};
|
||||
|
||||
/**
|
||||
* Resolve a `provider/model` string to a video target, checking the built-in
|
||||
* video registry first and falling back to custom OpenAI-compatible provider
|
||||
* nodes tagged with the "videos" endpoint (mirrors the direct
|
||||
* /v1/videos/generations route's custom-model scan). Shared by the direct
|
||||
* route and the combo executor so both paths cover the same target set.
|
||||
*/
|
||||
export async function resolveVideoModelTarget(
|
||||
modelStr: string | null | undefined
|
||||
): Promise<VideoModelTarget> {
|
||||
const parsed = parseVideoModel(modelStr ?? null);
|
||||
if (parsed.provider) {
|
||||
return { provider: parsed.provider, model: parsed.model, isCustomModel: false };
|
||||
}
|
||||
|
||||
if (!modelStr) {
|
||||
return { provider: null, model: null, isCustomModel: false };
|
||||
}
|
||||
|
||||
try {
|
||||
const customModelsMap = (await getAllCustomModels()) as Record<string, unknown>;
|
||||
for (const [providerId, models] of Object.entries(customModelsMap)) {
|
||||
if (!Array.isArray(models)) continue;
|
||||
for (const model of models as Array<{ id?: string; supportedEndpoints?: unknown }>) {
|
||||
if (!model?.id || !Array.isArray(model.supportedEndpoints)) continue;
|
||||
if (!model.supportedEndpoints.includes("videos")) continue;
|
||||
const fullId = `${providerId}/${model.id}`;
|
||||
if (fullId === modelStr) {
|
||||
return { provider: providerId, model: model.id, isCustomModel: true };
|
||||
}
|
||||
}
|
||||
}
|
||||
} catch {
|
||||
// registry read failure — fall through to "not found"
|
||||
}
|
||||
|
||||
return { provider: null, model: null, isCustomModel: false };
|
||||
}
|
||||
|
||||
/**
|
||||
* I2V models that accept an omitted prompt (the source image already carries
|
||||
* enough context). Mirrors the allow-list the direct route applies before
|
||||
* promptRequiredResponse(); kept here so combo targets get identical rules.
|
||||
*/
|
||||
export function isVideoPromptOptional(parsed: { provider: string | null; model: string | null }) {
|
||||
return (
|
||||
(parsed.model === "happyhorse-1.1-i2v" &&
|
||||
(parsed.provider === "alibaba" ||
|
||||
parsed.provider === "bailian-coding-plan" ||
|
||||
parsed.provider === "qwen-cloud-token-plan" ||
|
||||
parsed.provider === "qwen-cloud")) ||
|
||||
(parsed.provider === "qwen-cloud" && parsed.model === "wan2.7-i2v") ||
|
||||
(parsed.provider === "alibaba" &&
|
||||
(parsed.model === "wan2.7-i2v-2026-04-25" || parsed.model === "wan2.6-i2v-flash"))
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* #6928: best-effort per-connection base-URL override lookup for local no-auth
|
||||
* media providers (ComfyUI). Returns null instead of failing when no connection
|
||||
* exists — local providers must keep working with zero configuration. Shared
|
||||
* by the direct route and the combo executor so a configured ComfyUI base URL
|
||||
* is honored the same way regardless of which path dispatched the request.
|
||||
*/
|
||||
export async function resolveLocalOverrideCredentials(provider: string) {
|
||||
const localCredentials = await getProviderCredentialsWithQuotaPreflight(provider);
|
||||
return localCredentials && !isAllRateLimitedCredentials(localCredentials)
|
||||
? localCredentials
|
||||
: null;
|
||||
}
|
||||
@@ -55,6 +55,19 @@ async function postHandler(request, context) {
|
||||
const policy = await enforceApiKeyPolicy(request, body.model);
|
||||
if (policy.rejection) return policy.rejection;
|
||||
|
||||
// Detect a combo name and divert to full speech combo execution, mirroring
|
||||
// the images route. Checks before parseSpeechModel so a combo name is never
|
||||
// rejected as an invalid `provider/model` id — /v1/models advertises these
|
||||
// names, so refusing them here made the catalogue dishonest.
|
||||
if (body.model && typeof body.model === "string" && !body.model.includes("/")) {
|
||||
const { getComboByName } = await import("@/lib/db/combos");
|
||||
const combo = await getComboByName(body.model);
|
||||
if (combo) {
|
||||
const { executeSpeechCombo } = await import("@omniroute/open-sse/services/speechCombo");
|
||||
return executeSpeechCombo(body.model, body, { request, policy }, startTime);
|
||||
}
|
||||
}
|
||||
|
||||
// Provider nodes eligible for speech: this route's own audio type plus general
|
||||
// chat/responses gateways. Remote hosts are opt-in (default OFF).
|
||||
const dynamicProviders = await resolveDynamicAudioProviders("/audio/speech", "audio-speech");
|
||||
|
||||
@@ -1,12 +1,11 @@
|
||||
import { handleVideoGeneration } from "@omniroute/open-sse/handlers/videoGeneration.ts";
|
||||
import { resolveVideoCredentialProvider } from "@omniroute/open-sse/handlers/videoGeneration/googleFlow.ts";
|
||||
import { withInjectionGuard } from "@/middleware/promptInjectionGuard";
|
||||
import { getAllCustomModels } from "@/lib/db/models";
|
||||
import {
|
||||
getProviderCredentialsWithQuotaPreflight,
|
||||
clearRecoveredProviderState,
|
||||
} from "@/sse/services/auth";
|
||||
import { parseVideoModel, getVideoProvider } from "@omniroute/open-sse/config/videoRegistry.ts";
|
||||
import { getVideoProvider } from "@omniroute/open-sse/config/videoRegistry.ts";
|
||||
import { errorResponse } from "@omniroute/open-sse/utils/error.ts";
|
||||
import { HTTP_STATUS } from "@omniroute/open-sse/config/constants.ts";
|
||||
import * as log from "@/sse/utils/logger";
|
||||
@@ -25,6 +24,11 @@ import {
|
||||
} from "@/app/api/v1/_shared/mediaGenerationRoute";
|
||||
import type { MediaGenerationResultLike } from "@/app/api/v1/_shared/mediaGenerationRoute";
|
||||
import { getSpecialtyModelsResponse } from "@/app/api/v1/_shared/specialtyCatalog";
|
||||
import {
|
||||
isVideoPromptOptional,
|
||||
resolveLocalOverrideCredentials,
|
||||
resolveVideoModelTarget,
|
||||
} from "@/app/api/v1/_shared/videoModelResolution";
|
||||
|
||||
export const dynamic = "force-dynamic";
|
||||
|
||||
@@ -46,18 +50,6 @@ export async function GET(request?: Request) {
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* #6928: best-effort per-connection base-URL override lookup for local no-auth
|
||||
* media providers (ComfyUI). Returns null instead of failing when no connection
|
||||
* exists — local providers must keep working with zero configuration.
|
||||
*/
|
||||
async function resolveLocalOverrideCredentials(provider) {
|
||||
const localCredentials = await getProviderCredentialsWithQuotaPreflight(provider);
|
||||
return localCredentials && !isAllRateLimitedCredentials(localCredentials)
|
||||
? localCredentials
|
||||
: null;
|
||||
}
|
||||
|
||||
/**
|
||||
* POST /v1/videos/generations — generate videos
|
||||
*/
|
||||
@@ -68,52 +60,30 @@ async function postHandler(request, context) {
|
||||
}
|
||||
const body = parsed.body;
|
||||
const startTime = Date.now();
|
||||
const parsedModel = parseVideoModel(body.model);
|
||||
|
||||
const promptOptional =
|
||||
(parsedModel.model === "happyhorse-1.1-i2v" &&
|
||||
(parsedModel.provider === "alibaba" ||
|
||||
parsedModel.provider === "bailian-coding-plan" ||
|
||||
parsedModel.provider === "qwen-cloud-token-plan" ||
|
||||
parsedModel.provider === "qwen-cloud")) ||
|
||||
(parsedModel.provider === "qwen-cloud" && parsedModel.model === "wan2.7-i2v") ||
|
||||
(parsedModel.provider === "alibaba" &&
|
||||
(parsedModel.model === "wan2.7-i2v-2026-04-25" || parsedModel.model === "wan2.6-i2v-flash"));
|
||||
if (!promptOptional) {
|
||||
const promptError = promptRequiredResponse(body);
|
||||
if (promptError) return promptError;
|
||||
}
|
||||
|
||||
// Enforce API key policies (model restrictions + budget limits)
|
||||
const policy = await enforceApiKeyPolicy(request, body.model);
|
||||
if (policy.rejection) return policy.rejection;
|
||||
|
||||
// Parse model to get provider
|
||||
let { provider, model: requestedModel } = parsedModel;
|
||||
let isCustomModel = false;
|
||||
if (!provider) {
|
||||
// Custom OpenAI-compatible provider nodes (mirrors images route): scan the
|
||||
// dynamic model registry for a matching `${nodeId}/${modelId}` entry.
|
||||
try {
|
||||
const customModelsMap = (await getAllCustomModels()) as Record<string, any>;
|
||||
for (const [providerId, models] of Object.entries(customModelsMap)) {
|
||||
if (!Array.isArray(models)) continue;
|
||||
for (const model of models) {
|
||||
if (!model?.id || !Array.isArray(model.supportedEndpoints)) continue;
|
||||
if (!model.supportedEndpoints.includes("videos")) continue;
|
||||
const fullId = `${providerId}/${model.id}`;
|
||||
if (fullId === body.model) {
|
||||
provider = providerId;
|
||||
requestedModel = model.id;
|
||||
isCustomModel = true;
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
} catch {
|
||||
// registry read failure — fall through to invalid-model error below
|
||||
// Detect a combo name and divert to full video combo execution, mirroring
|
||||
// the images route. Checks before the provider lookup — and before the
|
||||
// prompt-required check below — so a combo name is never rejected as an
|
||||
// invalid `provider/model` id or against the wrong model's prompt rules:
|
||||
// /v1/models advertises these names, and prompt requirements depend on the
|
||||
// resolved target, which for a combo is only known after expansion.
|
||||
if (body.model && typeof body.model === "string" && !body.model.includes("/")) {
|
||||
const { getComboByName } = await import("@/lib/db/combos");
|
||||
const combo = await getComboByName(body.model);
|
||||
if (combo) {
|
||||
const { executeVideoCombo } = await import("@omniroute/open-sse/services/videoCombo");
|
||||
return executeVideoCombo(body.model, body, { request, policy }, startTime, log);
|
||||
}
|
||||
}
|
||||
|
||||
// Parse model to get provider — checks the built-in registry, then custom
|
||||
// OpenAI-compatible provider nodes tagged with the "videos" endpoint.
|
||||
const resolvedTarget = await resolveVideoModelTarget(body.model);
|
||||
const { provider, model: requestedModel, isCustomModel } = resolvedTarget;
|
||||
if (!provider) {
|
||||
return errorResponse(
|
||||
HTTP_STATUS.BAD_REQUEST,
|
||||
@@ -121,6 +91,11 @@ async function postHandler(request, context) {
|
||||
);
|
||||
}
|
||||
|
||||
if (!isVideoPromptOptional(resolvedTarget)) {
|
||||
const promptError = promptRequiredResponse(body);
|
||||
if (promptError) return promptError;
|
||||
}
|
||||
|
||||
// Check provider config for auth bypass
|
||||
const providerConfig = getVideoProvider(provider);
|
||||
|
||||
|
||||
Reference in New Issue
Block a user