Compare commits

...

6 Commits

Author SHA1 Message Date
Minxi Hou
8822b18f70 fix(providers): compare the untrimmed credential, and cover the guard's branches
The guard trimmed the incoming value before comparing it, which catches a paste
carrying whitespace the password does not have. It missed the mirror case:
neither the login route nor the set-password route trims, so a dashboard
password may itself begin or end with a space, and an autofill reproducing it
exactly was trimmed into a value that no longer matched the stored hash. The
write then went through, which is the state this guard exists to prevent. Both
forms are compared now, the second only when the first fails on a string that
differs, so an ordinary key still costs a single bcrypt round.

Two branches carried no coverage and both are load-bearing. The catch that logs
and allows is the only path that lets a write through; a stored hash bcrypt
cannot parse reaches it without needing a mock, since the shape check accepts an
impossible cost factor that the comparison then rejects. The early return is
what keeps a token renewal -- a write carrying tokens but no apiKey -- from
paying for a settings read and a bcrypt round every time it fires, and the same
unparseable hash makes that path observable, so an absent warning is proof the
return happened.

The narrower scope is deliberate and now says so in the code: the OAuth tokens
arrive from a provider's token endpoint rather than from a form, so extending
the comparison to them would charge every renewal for a field no autofill can
reach.

Signed-off-by: Minxi Hou <houminxi@gmail.com>
2026-08-09 01:41:18 -03:00
Minxi Hou
ded204b277 fix(providers): refuse to store the dashboard password as a connection API key
A browser autofilled the management password into a connection's API-key field.
The resulting credential authenticates against nothing, so every request routed
through that connection came back 401, and because the field looks like any
other password input the same autofill fired again while the connection was
being repaired by hand.

The refusal belongs on the write path rather than in the form. Twenty routes
create or update connections and all of them funnel through
createProviderConnection and updateProviderConnection, so one check there covers
every entry point including a future one. The two other places that write
api_key are left alone on purpose: one re-encrypts rows that already exist and
the other is the one-time db.json import, and neither takes a value an operator
just typed.

Update checks the incoming value, never the merged one. A connection that
already holds the password has to stay editable or the operator cannot repair
the exact state this prevents, and re-checking the merged value would spend a
bcrypt round on every unrelated field edit.

Only a real match blocks the write. An unreadable settings row or a throwing
bcrypt call logs and allows, because a guard against one specific mistake must
not turn into a way to lock out every connection write.

Signed-off-by: Minxi Hou <houminxi@gmail.com>
2026-08-09 01:41:18 -03:00
Diego Rodrigues de Sa e Souza
aae408f585 Merge pull request #9296 from artickc/fix/adobe-firefly-model-capabilities
fix(adobe-firefly): sync discovered models and capabilities
2026-08-09 01:09:01 -03:00
diegosouzapw
3e1c31c606 fix(adobe-firefly): retain Topaz catalog models 2026-08-09 00:21:52 -03:00
diegosouzapw
2e12ee89f7 Merge remote-tracking branch 'origin/release/v3.8.50' into fix/merge-pr-9296
# Conflicts:
#	open-sse/config/imageRegistry.ts
#	open-sse/handlers/imageGeneration/providers/adobeFirefly.ts
#	open-sse/services/adobeFireflyClient.ts
#	tests/unit/adobe-firefly.test.ts
2026-08-09 00:06:06 -03:00
artickc
723ce0b166 fix(adobe-firefly): sync models and media capabilities 2026-08-03 17:22:08 +03:00
16 changed files with 1880 additions and 988 deletions

View File

@@ -1,4 +1,5 @@
{
"_rebaseline_2026_08_09_9296_adobe_media_capabilities": "PR #9296 (artickc, fix/adobe-firefly-model-capabilities) own growth: src/app/api/v1/models/catalog.ts 1590->1597 (+7). The image and video catalog serializers now expose the already-normalized Adobe Firefly discovery capability data (media_capabilities, plus the existing video modality/size fields) at their only response-emission chokepoints. The discovery parser and capability normalization remain in open-sse/services/adobeFireflyModels.ts; extracting these seven serialization fields would obscure the catalog contract. Covered by tests/unit/adobe-firefly.test.ts and tests/unit/image-upscale.test.ts.",
"_rebaseline_2026_07_24_8470_hyperagent_sticky_thread": "PR #8470 (artickc, fix/hyperagent-tool-loop-thread-sticky) own growth: open-sse/executors/hyperagent.ts 936->1025 (wc -l; check-file-size.mjs counts via split(\"\\n\").length so the gate sees 937->1026, +89, crosses the 1000 cap). Fixes a real bug where a reverse-conversion proxy (text-Intent/JSON to Claude Code native tool_calls) rewrites assistant messages between agentic tool-loop turns, breaking HyperAgents conversation-prefix fingerprint and cold-starting the thread mid tool-loop. Adds Anthropic tool_use/tool_result flattening to extractMessageText() plus a new rootUserFingerprint()/root-key lookup tier in resolveHyperAgentThreadBinding()/storeHyperAgentThreadAfterTurn() so the thread stays sticky across the tool loop. Cohesive additions inside the existing single-file executor; not extractable without splitting the executor mid-request-flow. Covered by tests/unit/executor-hyperagent.test.ts (19/19, +5 new cases for tool_result/tool_use flattening + root-key stickiness). Pre-merge review flagged a cross-conversation root-key collision risk (tracked in the PRs own mandatory pre-merge checklist, not yet addressed) — unrelated to this file-size ratchet, tracked separately by /fix-prs.",
"_rebaseline_2026_07_25_8494_capability_filter_fail_closed": "PR #8494 (fix/capability-filters-fail-closed, #8488) own growth: open-sse/services/combo.ts 3640->3693 (+53) adds a fail-closed guard after filterTargetsByRequestCompatibility() — when every eligible target is excluded by request-capability filtering (vision/tools/etc) instead of quota/health, the combo now returns an explicit `capability_mismatch` 400 (describeCapabilityFilterExhaustion, imported from combo/comboStructure.ts) rather than silently falling through to a generic no-targets error, plus a `compatFilterFailOpen` escape hatch (combo config OR settings) mirrored at both the main/auto and round-robin call sites for symmetry. combo/comboStructure.ts (previously under cap, un-frozen) grows 794->918 (+124) — new home for describeCapabilityFilterExhaustion + providerSupportsEmulatedToolCalling (#5240 emulated tool-calling exemption so fail-closed does not regress prompt-emulation-only combos like all-chatgpt-web). Irreducible orchestration wiring at the existing filter chokepoint (same precedent as #7301's universal-cooldown-retry generalization). Companion test tests/unit/combo-routing-engine.test.ts 3409->3449 (+40, fail-closed/fail-open coverage across both call sites) also rebaselined. Covered by tests/unit/8488-capability-filter-fail-closed.test.ts (new) + 95/95 passing across both files. Structural shrink of combo.ts tracked in #3501.",
"_rebaseline_2026_07_25_8499_ts7_result_union_predicates": "PR #8499 (backryun, chore/ts7-types-executor-scattered) own growth: muse-spark-web.ts 1396->1405 (+9, irreducible). Under this workspace's `strictNullChecks: false`, the boolean-literal discriminant on `GraphqlResult` (`{ ok: true } | { ok: false; error: string }`) narrows the positive `.ok===true` branch but leaves `!result.ok` at the full union under TS7, making `.error` unreachable to the checker at the two call sites (warmup, mode-switch). Fixed by adding a single `isGraphqlFailure()` type-predicate helper (doc comment + 3-line body) reused at both call sites instead of duplicating the predicate inline — not extractable to a shared module without splitting a single-file executor's local narrowing helper out of its own file. Covered by the existing muse-spark-web executor test suite (no behavior change, pure narrowing fix).",
@@ -387,7 +388,7 @@
"src/app/(dashboard)/dashboard/usage/components/EvalsTab.tsx": 2148,
"src/app/(dashboard)/dashboard/usage/components/ProviderLimits/index.tsx": 1119,
"src/app/api/providers/[id]/models/route.ts": 2361,
"src/app/api/v1/models/catalog.ts": 1590,
"src/app/api/v1/models/catalog.ts": 1597,
"src/lib/db/apiKeys.ts": 1529,
"src/lib/db/core.ts": 1639,
"src/lib/db/migrationRunner.ts": 1094,
@@ -536,7 +537,7 @@
"src/app/(dashboard)/dashboard/usage/components/EvalsTab.tsx": "2148",
"src/app/(dashboard)/dashboard/usage/components/ProviderLimits/index.tsx": "1119",
"src/app/api/providers/[id]/models/route.ts": "2361",
"src/app/api/v1/models/catalog.ts": "1590",
"src/app/api/v1/models/catalog.ts": "1597",
"src/lib/tokenHealthCheck.ts": "1053",
"src/lib/db/apiKeys.ts": "1529",
"src/lib/db/core.ts": "1639",

View File

@@ -12,6 +12,10 @@ import { FREEPIK_IMAGE_PROVIDER } from "./providers/registry/freepik/index.ts";
import { STABILITY_AI_IMAGE_MODELS } from "./providers/registry/stability-ai/imageModels.ts";
import { GEMINI_IMAGEN_PROVIDER } from "./providers/registry/gemini/imageModels.ts";
import { CHEAPERINFERENCE_IMAGE_PROVIDER } from "./providers/registry/cheaperinference/imageModels.ts";
import {
ADOBE_FIREFLY_IMAGE_ROUTING_ALIASES,
toRegistryImageModels,
} from "../services/adobeFireflyModels.ts";
interface ImageModelEntry {
id: string;
@@ -22,6 +26,8 @@ interface ImageModelEntry {
imageRequired?: boolean;
description?: string;
isMarket?: boolean;
supportedSizes?: string[];
mediaCapabilities?: Record<string, unknown>;
}
interface ImageProviderConfig {
@@ -35,6 +41,7 @@ interface ImageProviderConfig {
authHeader: string;
format: string;
models: ImageModelEntry[];
routingAliases?: readonly string[];
supportedSizes: string[];
}
@@ -46,6 +53,7 @@ interface ImageModelAliasEntry {
inputModalities?: string[];
imageRequired?: boolean;
description?: string;
mediaCapabilities?: Record<string, unknown>;
}
interface ImageCatalogModelEntry {
@@ -55,6 +63,7 @@ interface ImageCatalogModelEntry {
supportedSizes: string[];
inputModalities: string[];
description?: string;
mediaCapabilities?: Record<string, unknown>;
}
const IMAGE_MODEL_ALIASES: Record<string, ImageModelAliasEntry> = {
@@ -678,55 +687,9 @@ export const IMAGE_PROVIDERS: Record<string, ImageProviderConfig> = {
authType: "apikey",
authHeader: "bearer",
format: "adobe-firefly-image",
models: [
{
id: "nano-banana-pro",
name: "Firefly Gemini 3.0 (Nano Banana Pro)",
inputModalities: ["text", "image"],
},
{
id: "nano-banana",
name: "Firefly Gemini 2.5 (Nano Banana)",
inputModalities: ["text", "image"],
},
{
id: "nano-banana-2",
name: "Firefly Gemini 3.1 (Nano Banana 2)",
inputModalities: ["text", "image"],
},
{ id: "gpt-image-2", name: "Firefly GPT Image 2", inputModalities: ["text", "image"] },
{ id: "gpt-image", name: "Firefly GPT Image 2", inputModalities: ["text", "image"] },
{ id: "gpt-image-1.5", name: "Firefly GPT Image 1.5", inputModalities: ["text", "image"] },
{ id: "flux-2", name: "Firefly Flux 2", inputModalities: ["text", "image"] },
{ id: "flux-pro", name: "Firefly Flux 1.1 Pro", inputModalities: ["text", "image"] },
{ id: "flux-ultra", name: "Firefly Flux 1.1 Ultra", inputModalities: ["text", "image"] },
{ id: "seedream-4", name: "Firefly Seedream 4.0", inputModalities: ["text", "image"] },
{
id: "seedream-5-lite",
name: "Firefly Seedream 5.0 Lite",
inputModalities: ["text", "image"],
},
{
id: "runway-gen4-image",
name: "Firefly Runway Gen-4 Image",
inputModalities: ["text", "image"],
},
// Topaz Labs upscalers (inputMediaUseCase: ["upscaling"]).
// Served by firefly-3p /v2/3p-images/upsample — see config/upscaleRegistry.ts.
{
id: "topaz-standard",
name: "Firefly Topaz Upscale (Standard)",
inputModalities: ["image"],
imageRequired: true,
},
{
id: "topaz-bloom",
name: "Firefly Topaz Bloom (Creative Upscale)",
inputModalities: ["image"],
imageRequired: true,
},
],
supportedSizes: ["1:1", "16:9", "9:16", "4:3", "3:4", "1024x1024", "1792x1024", "1024x1792"],
models: toRegistryImageModels(),
routingAliases: ADOBE_FIREFLY_IMAGE_ROUTING_ALIASES,
supportedSizes: [],
},
// Cheaper Inference (OSS-sponsor gateway). Declared AFTER adobe-firefly on
@@ -887,7 +850,7 @@ export function parseImageModel(modelStr) {
// No provider prefix — try to find the model in every provider
for (const [providerId, config] of Object.entries(IMAGE_PROVIDERS)) {
if (config.models.some((m) => m.id === modelStr)) {
if (config.routingAliases?.includes(modelStr) || config.models.some((m) => m.id === modelStr)) {
return { provider: providerId, model: modelStr };
}
}
@@ -906,9 +869,10 @@ function imageProviderCatalogEntries(
id: `${providerId}/${model.id}`,
name: model.name,
provider: providerId,
supportedSizes: config.supportedSizes,
supportedSizes: model.supportedSizes || config.supportedSizes,
inputModalities: model.inputModalities || ["text"],
description: model.description || undefined,
mediaCapabilities: model.mediaCapabilities,
}));
}

View File

@@ -5,14 +5,17 @@
* Supports local providers plus hosted task-based APIs such as Runway.
*/
import { parseModelFromRegistry, getAllModelsFromRegistry } from "./registryUtils.ts";
import { parseModelFromRegistry } from "./registryUtils.ts";
import { RUNWAYML_SUPPORTED_VIDEO_MODELS } from "./runway.ts";
import { SEGMIND_VIDEO_MODELS } from "./providers/registry/segmind/videoModels.ts";
import { toRegistryVideoModels } from "../services/adobeFireflyModels.ts";
interface VideoModel {
id: string;
name: string;
isMarket?: boolean;
supportedSizes?: string[];
mediaCapabilities?: Record<string, unknown>;
}
interface VideoProvider {
@@ -326,8 +329,7 @@ export const VIDEO_PROVIDERS: Record<string, VideoProvider> = {
},
// Adobe Firefly (unofficial) — same IMS/cookie credential as the image entry.
// Async 3P video generate + poll (Sora 2, Veo 3.1, Kling …). Fallback list
// from models/discovery capture (adobe/get_models.txt).
// Exact async video models and capabilities from the verified discovery snapshot.
"adobe-firefly": {
id: "adobe-firefly",
alias: "firefly",
@@ -335,18 +337,7 @@ export const VIDEO_PROVIDERS: Record<string, VideoProvider> = {
authType: "apikey",
authHeader: "bearer",
format: "adobe-firefly-video",
models: [
{ id: "sora-2", name: "Firefly Sora 2" },
{ id: "sora-2-pro", name: "Firefly Sora 2 Pro" },
{ id: "veo-3.1", name: "Firefly Veo 3.1" },
{ id: "veo-3.1-fast", name: "Firefly Veo 3.1 Fast" },
{ id: "veo-3.1-ref", name: "Firefly Veo 3.1 Reference" },
{ id: "kling-3", name: "Firefly Kling v3 Standard I2V" },
{ id: "kling-v3-t2v", name: "Firefly Kling v3 Standard T2V" },
{ id: "kling-v3-pro-i2v", name: "Firefly Kling v3 Pro I2V" },
{ id: "luma-ray3", name: "Firefly Ray3" },
{ id: "runway-gen4-turbo", name: "Firefly Runway Gen-4 Video" },
],
models: toRegistryVideoModels(),
},
};
@@ -368,5 +359,17 @@ export function parseVideoModel(modelStr: string | null) {
* Get all video models as a flat list
*/
export function getAllVideoModels() {
return getAllModelsFromRegistry(VIDEO_PROVIDERS);
return Object.entries(VIDEO_PROVIDERS).flatMap(([providerId, config]) =>
[providerId, config.alias]
.filter((prefix): prefix is string => Boolean(prefix))
.flatMap((prefix) =>
config.models.map((model) => ({
id: `${prefix}/${model.id}`,
name: model.name,
provider: providerId,
supportedSizes: model.supportedSizes || [],
mediaCapabilities: model.mediaCapabilities,
}))
)
);
}

View File

@@ -16,11 +16,11 @@ import {
AdobeFireflyError,
adobeFireflyGenerateImage,
adobeFireflyImageTimeoutMs,
adobeFireflyMaxImageRefs,
resolveAdobeAccessToken,
resolveAdobeSourceImageIds,
resolveAdobeSourceImageReferences,
resolveAdobeImageModel,
} from "../../../services/adobeFireflyClient.ts";
import { getAdobeReferenceUploadLimit } from "../../../services/adobeFireflyModels.ts";
import { isAdobeFireflyUpscaleModel } from "../../../services/adobeFireflyUpscale.ts";
import { handleAdobeFireflyImageUpscale } from "../../imageUpscale/adobeFirefly.ts";
@@ -90,7 +90,8 @@ export async function handleAdobeFireflyImageGeneration({
// Keep the raw credential blob for Cookie + sherlockToken (x-arp-session-id).
// JWT may be embedded in the same paste as cookies (HAR / multi-line).
const psd = (credentials as { providerSpecificData?: { cookie?: string } })?.providerSpecificData;
const psd = (credentials as { providerSpecificData?: { cookie?: string } })
?.providerSpecificData;
const sessionCookie =
(typeof psd?.cookie === "string" && psd.cookie.trim()) ||
(typeof credentials?.apiKey === "string" && credentials.apiKey.trim()) ||
@@ -98,15 +99,11 @@ export async function handleAdobeFireflyImageGeneration({
? credentials.accessToken
: undefined);
// Cap uploads by model family. gpt-image: 2 subject refs max (34+ stalls colligo → 504).
// nano: 4 general refs for multi-panel composition.
const { id: resolvedId } = resolveAdobeImageModel(model);
const maxRefs = adobeFireflyMaxImageRefs(resolvedId);
const sourceImageIds = await resolveAdobeSourceImageIds({
const { spec } = resolveAdobeImageModel(model);
const references = await resolveAdobeSourceImageReferences({
accessToken,
body,
max: maxRefs,
max: getAdobeReferenceUploadLimit(spec, "image"),
sessionCookie,
prompt,
fetchImpl,
@@ -121,13 +118,13 @@ export async function handleAdobeFireflyImageGeneration({
: undefined;
const timeoutMs = adobeFireflyImageTimeoutMs({
timeoutMs: explicitTimeout,
refCount: sourceImageIds.length,
refCount: references.length,
});
log?.info?.(
"IMAGE",
`${provider}/${model} (adobe-firefly) | prompt: "${prompt.slice(0, 60)}${prompt.length > 60 ? "..." : ""}"` +
(sourceImageIds.length ? ` | refs: ${sourceImageIds.length}/${maxRefs}` : "") +
(references.length ? ` | refs: ${references.length}` : "") +
` | pollTimeoutMs=${timeoutMs}`
);
@@ -139,9 +136,8 @@ export async function handleAdobeFireflyImageGeneration({
aspectRatio: body.aspect_ratio ?? body.aspectRatio ?? body.size,
quality: body.quality,
seed: Number.isFinite(seed as number) ? (seed as number) : undefined,
negativePrompt:
typeof body.negative_prompt === "string" ? body.negative_prompt : undefined,
sourceImageIds: sourceImageIds.length ? sourceImageIds : undefined,
negativePrompt: typeof body.negative_prompt === "string" ? body.negative_prompt : undefined,
references: references.length ? references : undefined,
sessionCookie,
timeoutMs,
fetchImpl,

View File

@@ -10,9 +10,10 @@ import {
AdobeFireflyError,
adobeFireflyGenerateVideo,
resolveAdobeAccessToken,
resolveAdobeSourceImageIds,
resolveAdobeSourceImageReferences,
resolveAdobeVideoModel,
} from "../../services/adobeFireflyClient.ts";
import { getAdobeReferenceUploadLimit } from "../../services/adobeFireflyModels.ts";
function normalizePositiveNumber(value: unknown, fallback: number): number {
const n = Number(value);
@@ -55,7 +56,8 @@ export async function handleAdobeFireflyVideoGeneration({
? Number(body.seed)
: undefined;
// Keep raw paste for Cookie + sherlockToken (x-arp-session-id).
const psd = (credentials as { providerSpecificData?: { cookie?: string } })?.providerSpecificData;
const psd = (credentials as { providerSpecificData?: { cookie?: string } })
?.providerSpecificData;
const sessionCookie =
(typeof psd?.cookie === "string" && psd.cookie.trim()) ||
(typeof credentials?.apiKey === "string" && credentials.apiKey.trim()) ||
@@ -63,13 +65,11 @@ export async function handleAdobeFireflyVideoGeneration({
? credentials.accessToken
: undefined);
// Kling i2v / Veo ref / Sora frame: upload reference images first.
const { id: videoModelId } = resolveAdobeVideoModel(String(model));
const maxFrames = videoModelId.includes("kling") || videoModelId.includes("sora") ? 2 : 3;
const sourceImageIds = await resolveAdobeSourceImageIds({
const { spec } = resolveAdobeVideoModel(String(model));
const references = await resolveAdobeSourceImageReferences({
accessToken,
body,
max: maxFrames,
max: getAdobeReferenceUploadLimit(spec, "image"),
sessionCookie,
prompt,
fetchImpl,
@@ -79,7 +79,7 @@ export async function handleAdobeFireflyVideoGeneration({
log?.info?.(
"VIDEO",
`${provider}/${model} (adobe-firefly) | prompt: "${prompt.slice(0, 60)}${prompt.length > 60 ? "..." : ""}"` +
(sourceImageIds.length ? ` | frames: ${sourceImageIds.length}` : "")
(references.length ? ` | refs: ${references.length}` : "")
);
const result = await adobeFireflyGenerateVideo({
@@ -99,7 +99,7 @@ export async function handleAdobeFireflyVideoGeneration({
? body.negativePrompt
: undefined,
generateAudio: body.generate_audio !== false && body.generateAudio !== false,
sourceImageIds: sourceImageIds.length ? sourceImageIds : undefined,
references: references.length ? references : undefined,
sessionCookie,
timeoutMs,
fetchImpl,

File diff suppressed because it is too large Load Diff

File diff suppressed because one or more lines are too long

View File

@@ -1,328 +1,590 @@
/**
* Adobe Firefly model catalog: live discovery + static fallback from browser capture.
* Adobe Firefly model discovery and normalized media capabilities.
*
* Live: POST firefly-3p.ff.adobe.io/v2/models/discovery (needs valid IMS token).
* Fallback: curated rows from adobe/get_models.txt (2026-07 Firefly SPA capture) so
* Media/Models still list usable ids when discovery fails or credentials are missing.
* The live discovery schema is authoritative. The generated snapshot is used only
* when a request cannot perform authenticated discovery (for example /v1/models).
*/
import {
type AdobeFireflyDiscoveredModel,
discoverAdobeFireflyModels,
resolveAdobeAccessToken,
} from "./adobeFireflyClient.ts";
import { ADOBE_FIREFLY_DISCOVERY_SNAPSHOT } from "./adobeFireflyModelSnapshot.ts";
export type AdobeFireflyModality = "image" | "video" | "audio" | "unknown";
export interface AdobeFireflyDiscoveredModel {
modelId: string;
modelVersion: string;
displayName: string;
modality: AdobeFireflyModality;
enabled: boolean;
providerName?: string;
releaseReadiness?: string;
healthStatus?: string;
inputMediaUseCases: string[];
requestSchema?: Record<string, unknown>;
backingModel?: string;
}
export interface AdobeFireflyReferenceInputCapability {
mediaType: string;
usageType: string;
minItems: number;
maxItems: number | null;
maxFileSizeBytes: number | null;
}
export interface AdobeFireflyMediaCapabilities {
inputMediaUseCases: string[];
schemaProperties: string[];
requiredProperties: string[];
referenceInputs: AdobeFireflyReferenceInputCapability[];
maxReferenceItems: number | null;
supportedSizes: string[];
supportedAspectRatios: string[];
supportedResolutions: string[];
supportedDurations: number[];
durationMin: number | null;
durationMax: number | null;
durationDefault: number | null;
outputCountMin: number | null;
outputCountMax: number | null;
promptMaxLength: number | null;
releaseReadiness: string;
healthStatus: string;
}
export interface AdobeFireflyCatalogModel {
/** OpenAI-style id without provider prefix, e.g. nano-banana-pro or flux-fluxPro */
/** Stable API id without the provider prefix. */
id: string;
name: string;
modality: "image" | "video";
/** Upstream wire modelId for generate-async */
upstreamModelId: string;
/** Upstream wire modelVersion for generate-async */
upstreamModelVersion: string;
inputModalities?: string[];
providerName: string;
backingModel: string;
inputModalities: string[];
capabilities: AdobeFireflyMediaCapabilities;
}
/**
* Static fallback built from adobe/get_models.txt discovery response.
* Friendly aliases first (Media page defaults), then popular upstream families.
*/
export const ADOBE_FIREFLY_FALLBACK_MODELS: AdobeFireflyCatalogModel[] = [
// ── Friendly aliases (handler resolveAdobeImageModel / resolveAdobeVideoModel) ──
{
id: "nano-banana-pro",
name: "Gemini 3.0 (Nano Banana Pro)",
modality: "image",
upstreamModelId: "gemini-flash",
upstreamModelVersion: "nano-banana-2",
inputModalities: ["text", "image"],
},
{
id: "nano-banana",
name: "Gemini 2.5 (Nano Banana)",
modality: "image",
upstreamModelId: "gemini-flash",
upstreamModelVersion: "nano-banana",
inputModalities: ["text", "image"],
},
{
id: "nano-banana-2",
name: "Gemini 3.1 (Nano Banana 2)",
modality: "image",
upstreamModelId: "gemini-flash",
upstreamModelVersion: "nano-banana-3",
inputModalities: ["text", "image"],
},
{
id: "gpt-image-2",
name: "GPT Image 2",
modality: "image",
upstreamModelId: "gpt-image",
upstreamModelVersion: "2",
inputModalities: ["text", "image"],
},
{
id: "gpt-image",
name: "GPT Image 2",
modality: "image",
upstreamModelId: "gpt-image",
upstreamModelVersion: "2",
inputModalities: ["text", "image"],
},
{
id: "gpt-image-1.5",
name: "GPT Image 1.5",
modality: "image",
upstreamModelId: "gpt-image",
upstreamModelVersion: "1.5",
inputModalities: ["text", "image"],
},
{
id: "sora-2",
name: "Sora 2",
modality: "video",
upstreamModelId: "sora",
upstreamModelVersion: "sora-2",
},
{
id: "sora-2-pro",
name: "Sora 2 Pro",
modality: "video",
upstreamModelId: "sora",
upstreamModelVersion: "sora-2-pro",
},
{
id: "veo-3.1",
name: "Veo 3.1",
modality: "video",
upstreamModelId: "veo",
upstreamModelVersion: "3.1-generate",
},
{
id: "veo-3.1-fast",
name: "Veo 3.1 Fast",
modality: "video",
upstreamModelId: "veo",
upstreamModelVersion: "3.1-fast-generate",
},
{
id: "veo-3.1-ref",
name: "Veo 3.1 Reference",
modality: "video",
upstreamModelId: "veo",
upstreamModelVersion: "3.1-generate",
},
{
id: "kling-3",
name: "Kling Video v3 Standard Image to Video",
modality: "video",
upstreamModelId: "kling",
upstreamModelVersion: "kling_v3_standard_i2v",
},
// ── Additional image families from discovery capture ──
{
id: "flux-2",
name: "Flux 2",
modality: "image",
upstreamModelId: "flux",
upstreamModelVersion: "2",
inputModalities: ["text", "image"],
},
{
id: "flux-pro",
name: "Flux 1.1 Pro",
modality: "image",
upstreamModelId: "flux",
upstreamModelVersion: "fluxPro",
inputModalities: ["text", "image"],
},
{
id: "flux-ultra",
name: "Flux 1.1 Ultra",
modality: "image",
upstreamModelId: "flux",
upstreamModelVersion: "fluxUltra",
inputModalities: ["text", "image"],
},
{
id: "seedream-4",
name: "Seedream 4.0",
modality: "image",
upstreamModelId: "seedream",
upstreamModelVersion: "seedream_v4",
inputModalities: ["text", "image"],
},
{
id: "seedream-5-lite",
name: "Seedream 5.0 Lite",
modality: "image",
upstreamModelId: "seedream",
upstreamModelVersion: "seedream_v5_lite",
inputModalities: ["text", "image"],
},
{
id: "runway-gen4-image",
name: "Runway Gen-4 Image",
modality: "image",
upstreamModelId: "runway-gen4-image",
upstreamModelVersion: "gen4_image",
inputModalities: ["text", "image"],
},
// ── Additional video families ──
{
id: "kling-v3-t2v",
name: "Kling Video v3 Standard Text to Video",
modality: "video",
upstreamModelId: "kling",
upstreamModelVersion: "kling_v3_standard_t2v",
},
{
id: "kling-v3-pro-i2v",
name: "Kling Video v3 Pro Image to Video",
modality: "video",
upstreamModelId: "kling",
upstreamModelVersion: "kling_v3_pro_i2v",
},
{
id: "luma-ray3",
name: "Ray3",
modality: "video",
upstreamModelId: "luma",
upstreamModelVersion: "3.0-ray",
},
{
id: "runway-gen4-turbo",
name: "Runway Gen-4 Video",
modality: "video",
upstreamModelId: "runway",
upstreamModelVersion: "gen4_turbo",
},
];
export interface AdobeFireflyImageModelSpec extends AdobeFireflyCatalogModel {
modality: "image";
/** Payload dialect observed for this model family. */
family: "gemini" | "gpt-image" | "generic";
}
/** Stable slug for upstream modelId + modelVersion (catalog id when not a friendly alias). */
export interface AdobeFireflyVideoModelSpec extends AdobeFireflyCatalogModel {
modality: "video";
defaultDuration: number;
defaultResolution: string;
}
interface MergedObjectSchema {
properties: Record<string, Record<string, unknown>>;
required: string[];
}
function asRecord(value: unknown): Record<string, unknown> {
return value && typeof value === "object" && !Array.isArray(value)
? (value as Record<string, unknown>)
: {};
}
function asStringArray(value: unknown): string[] {
return Array.isArray(value)
? value.map((item) => String(item)).filter((item) => item.length > 0)
: [];
}
function finiteInteger(value: unknown): number | null {
return Number.isInteger(value) ? (value as number) : null;
}
/** Merge object properties/required keys contributed through JSON Schema allOf. */
export function mergeAdobeObjectSchema(schema: unknown): MergedObjectSchema {
const merged: MergedObjectSchema = { properties: {}, required: [] };
const visit = (value: unknown) => {
const node = asRecord(value);
const properties = asRecord(node.properties);
for (const [key, property] of Object.entries(properties)) {
merged.properties[key] = asRecord(property);
}
merged.required.push(...asStringArray(node.required));
if (Array.isArray(node.allOf)) node.allOf.forEach(visit);
};
visit(schema);
merged.required = [...new Set(merged.required)];
return merged;
}
function schemaBranches(schema: unknown): Record<string, unknown>[] {
const root = asRecord(schema);
if (Object.keys(root).length === 0) return [];
return [
root,
...(Array.isArray(root.anyOf) ? root.anyOf.map(asRecord) : []),
...(Array.isArray(root.oneOf) ? root.oneOf.map(asRecord) : []),
];
}
function enumStrings(schema: unknown): string[] {
return [
...new Set(
schemaBranches(schema)
.flatMap((branch) => (Array.isArray(branch.enum) ? branch.enum : []))
.filter((value): value is string => typeof value === "string")
),
];
}
function integerBranch(schema: unknown): Record<string, unknown> {
return schemaBranches(schema).find((branch) => branch.type === "integer") || {};
}
/** Stable, collision-resistant public id for an exact upstream model/version pair. */
export function slugifyAdobeModel(modelId: string, modelVersion: string): string {
const mid = String(modelId || "")
.trim()
.toLowerCase()
.replace(/[^a-z0-9]+/g, "-")
.replace(/^-|-$/g, "");
const ver = String(modelVersion || "")
.trim()
.toLowerCase()
.replace(/[^a-z0-9.]+/g, "-")
.replace(/^-|-$/g, "");
if (!ver || ver === "default" || ver === mid) return mid || "model";
return `${mid}-${ver}`;
const slug = (value: string, allowDot = false) =>
String(value || "")
.trim()
.toLowerCase()
.replace(allowDot ? /[^a-z0-9.]+/g : /[^a-z0-9]+/g, "-")
.replace(/^-|-$/g, "");
const family = slug(modelId);
// Adobe still uses `kling_v3_omni*` internally, while discovery exposes these
// products to users as Kling O3. Never leak the obsolete/internal "omni" name
// into the public API catalog; the untouched upstream version stays in the spec.
const publicVersion =
family === "kling" ? modelVersion.replace(/^kling_v3_omni/i, "kling_o3") : modelVersion;
const version = slug(publicVersion, true);
if (!version || version === "default" || version === family) return family || "model";
return `${family}-${version}`;
}
/** Map discovery rows → catalog entries (image/video only). */
export function mapDiscoveredToCatalog(
rows: AdobeFireflyDiscoveredModel[]
): AdobeFireflyCatalogModel[] {
const out: AdobeFireflyCatalogModel[] = [];
const seen = new Set<string>();
/** Parse POST /v2/models/discovery without discarding its resolved request schema. */
export function parseAdobeModelsDiscovery(body: unknown): AdobeFireflyDiscoveredModel[] {
const root = asRecord(body);
const families = Array.isArray(root.models) ? root.models : [];
const rows: AdobeFireflyDiscoveredModel[] = [];
// Prefer friendly aliases when upstream matches known fallback rows.
for (const fb of ADOBE_FIREFLY_FALLBACK_MODELS) {
const hit = rows.find(
(r) =>
r.modelId === fb.upstreamModelId &&
r.modelVersion === fb.upstreamModelVersion &&
(r.modality === fb.modality || r.modality === "unknown")
);
if (hit && !seen.has(fb.id)) {
seen.add(fb.id);
out.push({
...fb,
name: hit.displayName || fb.name,
for (const familyValue of families) {
const family = asRecord(familyValue);
const modelId = String(family.modelId || "").trim();
if (!modelId) continue;
for (const [modelVersion, versionValue] of Object.entries(asRecord(family.modelVersions))) {
const version = asRecord(versionValue);
if (version.enabled === false) continue;
const outputModalities = asStringArray(version.outputModality).map((item) =>
item.toLowerCase()
);
const modality: AdobeFireflyModality = outputModalities.includes("image")
? "image"
: outputModalities.includes("video")
? "video"
: outputModalities.includes("audio")
? "audio"
: "unknown";
rows.push({
modelId,
modelVersion,
displayName: String(
version.modelDisplayName || version.modelCaiDisplayName || modelVersion
),
modality,
enabled: version.enabled !== false,
providerName:
typeof family.acModelFamilyProviderDisplayName === "string"
? family.acModelFamilyProviderDisplayName
: undefined,
releaseReadiness:
typeof version.releaseReadiness === "string" ? version.releaseReadiness : undefined,
healthStatus: typeof version.healthStatus === "string" ? version.healthStatus : undefined,
inputMediaUseCases: asStringArray(version.inputMediaUseCase),
requestSchema: asRecord(version.requestSchema),
backingModel:
typeof version.bksGenerationModel === "string" ? version.bksGenerationModel : undefined,
});
}
}
return rows;
}
function normalizeCapabilities(row: AdobeFireflyDiscoveredModel): AdobeFireflyMediaCapabilities {
const schema = mergeAdobeObjectSchema(row.requestSchema);
const referenceSchema = asRecord(schema.properties.referenceBlobs);
const referenceInputs: AdobeFireflyReferenceInputCapability[] = [];
const mediaCapabilities = Array.isArray(referenceSchema["x-capabilities"])
? referenceSchema["x-capabilities"]
: [];
for (const mediaValue of mediaCapabilities) {
const media = asRecord(mediaValue);
const maxFileSizeBytes = finiteInteger(media.maxFileSizeBytes);
const usageConstraints = Array.isArray(media.usageConstraints) ? media.usageConstraints : [];
for (const usageValue of usageConstraints) {
const usage = asRecord(usageValue);
if (usage.deprecated === true) continue;
const usageType = String(usage.usageType || "");
const mediaType = String(media.mediaType || "");
if (!usageType || !mediaType) continue;
referenceInputs.push({
mediaType,
usageType,
minItems: finiteInteger(usage.minItems) ?? 0,
maxItems: finiteInteger(usage.maxItems),
maxFileSizeBytes,
});
}
}
for (const r of rows) {
if (r.modality !== "image" && r.modality !== "video") continue;
const id = slugifyAdobeModel(r.modelId, r.modelVersion);
if (seen.has(id)) continue;
// Skip if already covered by a friendly alias with same upstream
if (
out.some(
(o) =>
o.upstreamModelId === r.modelId && o.upstreamModelVersion === r.modelVersion
const supportedSizes = [
...new Set(
schemaBranches(schema.properties.size)
.flatMap((branch) => (Array.isArray(branch.enum) ? branch.enum : []))
.map(asRecord)
.filter((size) => finiteInteger(size.width) !== null && finiteInteger(size.height) !== null)
.map((size) => `${size.width}x${size.height}`)
),
];
const supportedAspectRatios = [
...new Set(
schemaBranches(schema.properties.generationSettings).flatMap((branch) =>
enumStrings(asRecord(asRecord(branch.properties).aspectRatio))
)
) {
continue;
}
seen.add(id);
out.push({
id,
name: r.displayName || id,
modality: r.modality,
upstreamModelId: r.modelId,
upstreamModelVersion: r.modelVersion,
inputModalities: r.modality === "image" ? ["text", "image"] : ["text"],
});
}
return out;
}
export function getAdobeFireflyFallbackCatalog(modality?: "image" | "video"): AdobeFireflyCatalogModel[] {
if (!modality) return [...ADOBE_FIREFLY_FALLBACK_MODELS];
return ADOBE_FIREFLY_FALLBACK_MODELS.filter((m) => m.modality === modality);
}
/**
* Live discovery when credentials resolve; otherwise static fallback from get_models capture.
*/
export async function resolveAdobeFireflyCatalog(opts: {
credentials?: {
apiKey?: string;
accessToken?: string;
providerSpecificData?: Record<string, unknown> | null;
} | null;
modality?: "image" | "video";
fetchImpl?: typeof fetch;
}): Promise<{ models: AdobeFireflyCatalogModel[]; source: "api" | "fallback" }> {
const fetchImpl = opts.fetchImpl || fetch;
try {
if (opts.credentials) {
const token = await resolveAdobeAccessToken(opts.credentials, fetchImpl);
const discovered = await discoverAdobeFireflyModels(token, fetchImpl);
let catalog = mapDiscoveredToCatalog(discovered);
if (opts.modality) catalog = catalog.filter((m) => m.modality === opts.modality);
if (catalog.length > 0) return { models: catalog, source: "api" };
}
} catch {
// fall through to static catalog
}
),
];
const duration = integerBranch(schema.properties.duration);
const outputCount = integerBranch(schema.properties.n);
const prompt =
schemaBranches(schema.properties.prompt).find((branch) => branch.type === "string") || {};
return {
models: getAdobeFireflyFallbackCatalog(opts.modality),
source: "fallback",
inputMediaUseCases: [...row.inputMediaUseCases],
schemaProperties: Object.keys(schema.properties),
requiredProperties: [...schema.required],
referenceInputs,
maxReferenceItems: finiteInteger(referenceSchema.maxItems),
supportedSizes,
supportedAspectRatios,
supportedResolutions: enumStrings(schema.properties.resolution),
supportedDurations: [
...new Set(
schemaBranches(schema.properties.duration)
.flatMap((branch) => (Array.isArray(branch.enum) ? branch.enum : []))
.filter((value): value is number => Number.isInteger(value))
),
],
durationMin: finiteInteger(duration.minimum),
durationMax: finiteInteger(duration.maximum),
durationDefault: finiteInteger(duration.default),
outputCountMin: finiteInteger(outputCount.minimum),
outputCountMax: finiteInteger(outputCount.maximum),
promptMaxLength: finiteInteger(prompt.maxLength),
releaseReadiness: row.releaseReadiness || "",
healthStatus: row.healthStatus || "",
};
}
/** Registry-shaped models for imageRegistry / videoRegistry. */
export function toRegistryImageModels(
models: AdobeFireflyCatalogModel[] = getAdobeFireflyFallbackCatalog("image")
): Array<{ id: string; name: string; inputModalities?: string[] }> {
return models
.filter((m) => m.modality === "image")
.map((m) => ({
id: m.id,
name: m.name.startsWith("Firefly ") ? m.name : `Firefly ${m.name}`,
inputModalities: m.inputModalities || ["text", "image"],
}));
function isCallableGenerationModel(row: AdobeFireflyDiscoveredModel): boolean {
if (row.modality !== "image" && row.modality !== "video") return false;
if (!mergeAdobeObjectSchema(row.requestSchema).properties.prompt) return false;
const excluded = new Set(["upscaling", "sharpening", "denoising"]);
return !row.inputMediaUseCases.some((value) => excluded.has(value.toLowerCase()));
}
export function toRegistryVideoModels(
models: AdobeFireflyCatalogModel[] = getAdobeFireflyFallbackCatalog("video")
): Array<{ id: string; name: string }> {
return models
.filter((m) => m.modality === "video")
.map((m) => ({
id: m.id,
name: m.name.startsWith("Firefly ") ? m.name : `Firefly ${m.name}`,
}));
function deriveInputModalities(capabilities: AdobeFireflyMediaCapabilities): string[] {
return ["text", ...new Set(capabilities.referenceInputs.map((reference) => reference.mediaType))];
}
function semanticCatalogKey(model: AdobeFireflyCatalogModel): string {
return JSON.stringify({
backingModel: model.backingModel,
name: model.name,
modality: model.modality,
capabilities: model.capabilities,
});
}
/** Normalize and de-duplicate callable image/video rows from live discovery. */
export function mapDiscoveredToCatalog(
rows: AdobeFireflyDiscoveredModel[]
): AdobeFireflyCatalogModel[] {
const output: AdobeFireflyCatalogModel[] = [];
const seen = new Set<string>();
for (const row of rows) {
if (!isCallableGenerationModel(row)) continue;
const capabilities = normalizeCapabilities(row);
const model: AdobeFireflyCatalogModel = {
id: slugifyAdobeModel(row.modelId, row.modelVersion),
name: row.displayName,
modality: row.modality as "image" | "video",
upstreamModelId: row.modelId,
upstreamModelVersion: row.modelVersion,
providerName: row.providerName || "",
backingModel: row.backingModel || "",
inputModalities: deriveInputModalities(capabilities),
capabilities,
};
const key = semanticCatalogKey(model);
if (seen.has(key)) continue;
seen.add(key);
output.push(model);
}
return output;
}
function snapshotCatalog(): AdobeFireflyCatalogModel[] {
return ADOBE_FIREFLY_DISCOVERY_SNAPSHOT.map((model) => {
const capabilities: AdobeFireflyMediaCapabilities = {
inputMediaUseCases: [...model.inputMediaUseCases],
schemaProperties: [...model.schemaProperties],
requiredProperties: [...model.requiredProperties],
referenceInputs: model.referenceInputs.map((reference) => ({ ...reference })),
maxReferenceItems: model.maxReferenceItems,
supportedSizes: [...model.supportedSizes],
supportedAspectRatios: [...model.supportedAspectRatios],
supportedResolutions: [...model.supportedResolutions],
supportedDurations: [...model.supportedDurations],
durationMin: model.durationMin,
durationMax: model.durationMax,
durationDefault: model.durationDefault,
outputCountMin: model.outputCountMin,
outputCountMax: model.outputCountMax,
promptMaxLength: model.promptMaxLength,
releaseReadiness: model.releaseReadiness,
healthStatus: model.healthStatus,
};
return {
id: model.id,
name: model.name,
modality: model.modality,
upstreamModelId: model.upstreamModelId,
upstreamModelVersion: model.upstreamModelVersion,
providerName: model.providerName,
backingModel: model.backingModel,
inputModalities: deriveInputModalities(capabilities),
capabilities,
};
});
}
export const ADOBE_FIREFLY_FALLBACK_MODELS: AdobeFireflyCatalogModel[] = snapshotCatalog();
export function getAdobeFireflyFallbackCatalog(
modality?: "image" | "video"
): AdobeFireflyCatalogModel[] {
return ADOBE_FIREFLY_FALLBACK_MODELS.filter((model) => !modality || model.modality === modality);
}
function imageFamily(model: AdobeFireflyCatalogModel): AdobeFireflyImageModelSpec["family"] {
if (model.upstreamModelId === "gemini-flash") return "gemini";
if (model.upstreamModelId === "gpt-image" || model.upstreamModelId === "gpt-4o-image") {
return "gpt-image";
}
return "generic";
}
export const ADOBE_FIREFLY_IMAGE_MODELS: Record<string, AdobeFireflyImageModelSpec> =
Object.fromEntries(
getAdobeFireflyFallbackCatalog("image").map((model) => [
model.id,
{ ...model, modality: "image" as const, family: imageFamily(model) },
])
);
function defaultDuration(model: AdobeFireflyCatalogModel): number {
const caps = model.capabilities;
return caps.durationDefault ?? caps.supportedDurations[0] ?? caps.durationMin ?? 5;
}
function defaultResolution(model: AdobeFireflyCatalogModel): string {
if (model.capabilities.supportedSizes.some((value) => value.includes("1920x1080"))) {
return "1080p";
}
return "720p";
}
export const ADOBE_FIREFLY_VIDEO_MODELS: Record<string, AdobeFireflyVideoModelSpec> =
Object.fromEntries(
getAdobeFireflyFallbackCatalog("video").map((model) => [
model.id,
{
...model,
modality: "video" as const,
defaultDuration: defaultDuration(model),
defaultResolution: defaultResolution(model),
},
])
);
const LEGACY_MODEL_ALIASES: Record<string, string> = {
"nano-banana": "gemini-flash-nano-banana",
"nano-banana-pro": "gemini-flash-nano-banana-2",
"nano-banana-2": "gemini-flash-nano-banana-3",
"gpt-image": "gpt-image-2",
"gpt-image-2": "gpt-image-2",
"gpt-image-1.5": "gpt-image-1.5",
"flux-2": "flux-2",
"flux-pro": "flux-fluxpro",
"flux-ultra": "flux-fluxultra",
"seedream-4": "seedream-seedream-v4",
"seedream-5-lite": "seedream-seedream-v5-lite",
"runway-gen4-image": "runway-gen4-image",
"veo-3.1": "veo-3.1-generate",
"veo-3.1-fast": "veo-3.1-fast-generate",
"luma-ray3": "luma-3.0-ray",
"runway-gen4-turbo": "runway-gen4-turbo",
// Backward compatibility only; the catalog advertises the exact discovered id.
"kling-3": "kling-kling-v3-standard-i2v",
};
// Preserve established API aliases when (and only when) they resolve to a model
// that is present in the verified discovery snapshot. These keys are not listed.
for (const [alias, target] of Object.entries(LEGACY_MODEL_ALIASES)) {
const imageTarget = ADOBE_FIREFLY_IMAGE_MODELS[target];
if (imageTarget) ADOBE_FIREFLY_IMAGE_MODELS[alias] = imageTarget;
const videoTarget = ADOBE_FIREFLY_VIDEO_MODELS[target];
if (videoTarget) ADOBE_FIREFLY_VIDEO_MODELS[alias] = videoTarget;
}
/** Backward-compatible request ids. Kept out of every advertised model catalog. */
export const ADOBE_FIREFLY_IMAGE_ROUTING_ALIASES = Object.freeze(
Object.entries(LEGACY_MODEL_ALIASES)
.filter(([, target]) => Boolean(ADOBE_FIREFLY_IMAGE_MODELS[target]))
.map(([alias]) => alias)
);
function normalizeRequestedId(model: string): string {
return String(model || "")
.trim()
.toLowerCase()
.replace(/^adobe-firefly\//, "")
.replace(/^firefly\//, "");
}
function resolveCatalogId(model: string): string {
const requested = normalizeRequestedId(model);
return LEGACY_MODEL_ALIASES[requested] || requested;
}
export function resolveAdobeImageModel(model: string): {
id: string;
spec: AdobeFireflyImageModelSpec;
} {
const id = resolveCatalogId(model);
const spec = ADOBE_FIREFLY_IMAGE_MODELS[id];
if (!spec) {
throw new Error(
`Unknown Adobe Firefly image model: ${normalizeRequestedId(model) || "(empty)"}`
);
}
return { id, spec };
}
export function resolveAdobeVideoModel(model: string): {
id: string;
spec: AdobeFireflyVideoModelSpec;
} {
const id = resolveCatalogId(model);
const spec = ADOBE_FIREFLY_VIDEO_MODELS[id];
if (!spec) {
throw new Error(
`Unknown Adobe Firefly video model: ${normalizeRequestedId(model) || "(empty)"}`
);
}
return { id, spec };
}
export function toRegistryImageModels(): Array<{
id: string;
name: string;
inputModalities: string[];
imageRequired?: boolean;
supportedSizes: string[];
mediaCapabilities: Record<string, unknown>;
}> {
const generated = getAdobeFireflyFallbackCatalog("image").map((model) => ({
id: model.id,
name: `Firefly ${model.name}`,
inputModalities: model.inputModalities,
supportedSizes: model.capabilities.supportedSizes,
mediaCapabilities: toAdobeMediaCapabilitiesApi(model),
}));
// Upscaling uses a distinct Firefly endpoint and is not returned by the image
// generation discovery schema. Keep its two supported Topaz models visible in
// the same provider catalog so image clients can select them deliberately.
return [
...generated,
{
id: "topaz-standard",
name: "Firefly Topaz Upscale (Standard)",
inputModalities: ["image"],
imageRequired: true,
supportedSizes: [],
mediaCapabilities: { input_media_use_cases: ["upscaling"] },
},
{
id: "topaz-bloom",
name: "Firefly Topaz Bloom (Creative Upscale)",
inputModalities: ["image"],
imageRequired: true,
supportedSizes: [],
mediaCapabilities: { input_media_use_cases: ["upscaling"] },
},
];
}
export function toRegistryVideoModels(): Array<{
id: string;
name: string;
supportedSizes: string[];
mediaCapabilities: Record<string, unknown>;
}> {
return getAdobeFireflyFallbackCatalog("video").map((model) => ({
id: model.id,
name: `Firefly ${model.name}`,
supportedSizes: model.capabilities.supportedSizes,
mediaCapabilities: toAdobeMediaCapabilitiesApi(model),
}));
}
/** JSON-safe extension emitted by /v1/models. */
export function toAdobeMediaCapabilitiesApi(
model: AdobeFireflyCatalogModel
): Record<string, unknown> {
const caps = model.capabilities;
return {
upstream_model_id: model.upstreamModelId,
upstream_model_version: model.upstreamModelVersion,
provider_name: model.providerName,
release_readiness: caps.releaseReadiness,
health_status: caps.healthStatus,
input_media_use_cases: caps.inputMediaUseCases,
reference_inputs: caps.referenceInputs.map((reference) => ({
media_type: reference.mediaType,
usage_type: reference.usageType,
min_items: reference.minItems,
max_items: reference.maxItems,
max_file_size_bytes: reference.maxFileSizeBytes,
})),
max_reference_items: caps.maxReferenceItems,
supported_sizes: caps.supportedSizes,
supported_aspect_ratios: caps.supportedAspectRatios,
supported_resolutions: caps.supportedResolutions,
supported_durations: caps.supportedDurations,
duration_min: caps.durationMin,
duration_max: caps.durationMax,
duration_default: caps.durationDefault,
output_count_min: caps.outputCountMin,
output_count_max: caps.outputCountMax,
prompt_max_length: caps.promptMaxLength,
};
}
export function getAdobeReferenceUploadLimit(
model: AdobeFireflyCatalogModel,
mediaType: string
): number {
if (model.capabilities.maxReferenceItems !== null) {
return Math.max(1, Math.min(32, model.capabilities.maxReferenceItems));
}
const declaredTotal = model.capabilities.referenceInputs
.filter((reference) => reference.mediaType === mediaType)
.reduce((total, reference) => total + (reference.maxItems ?? 0), 0);
return Math.max(1, Math.min(32, declaredTotal || 1));
}

View File

@@ -0,0 +1,207 @@
#!/usr/bin/env node
import fs from "node:fs";
import path from "node:path";
import { createHash } from "node:crypto";
function usage() {
console.error(
"Usage: node scripts/dev/generate-adobe-firefly-snapshot.mjs <discovery.json> <output.ts>"
);
process.exit(2);
}
const [, , inputArg, outputArg] = process.argv;
if (!inputArg || !outputArg) usage();
const inputPath = path.resolve(inputArg);
const outputPath = path.resolve(outputArg);
const inputBytes = fs.readFileSync(inputPath);
const sourceHash = createHash("sha256").update(inputBytes).digest("hex");
const root = JSON.parse(inputBytes.toString("utf8"));
function mergeObjectSchema(schema) {
const merged = { properties: {}, required: [] };
const visit = (node) => {
if (!node || typeof node !== "object") return;
if (node.properties && typeof node.properties === "object") {
Object.assign(merged.properties, node.properties);
}
if (Array.isArray(node.required)) merged.required.push(...node.required);
if (Array.isArray(node.allOf)) node.allOf.forEach(visit);
};
visit(schema);
merged.required = [...new Set(merged.required)];
return merged;
}
function branches(schema) {
if (!schema || typeof schema !== "object") return [];
return [schema, ...(schema.anyOf || []), ...(schema.oneOf || [])];
}
function stringEnums(schema) {
return [
...new Set(
branches(schema)
.flatMap((branch) => (Array.isArray(branch.enum) ? branch.enum : []))
.filter((value) => typeof value === "string")
),
];
}
function integerSchema(schema) {
return branches(schema).find((branch) => branch.type === "integer") || {};
}
function publicModelId(modelId, modelVersion) {
const slug = (value, allowDot = false) =>
String(value || "")
.trim()
.toLowerCase()
.replace(allowDot ? /[^a-z0-9.]+/g : /[^a-z0-9]+/g, "-")
.replace(/^-|-$/g, "");
const family = slug(modelId);
const publicVersion =
family === "kling" ? String(modelVersion).replace(/^kling_v3_omni/i, "kling_o3") : modelVersion;
const version = slug(publicVersion, true);
if (!version || version === "default" || version === family) return family || "model";
return `${family}-${version}`;
}
function normalizeModel(family, modelVersion, version) {
const schema = mergeObjectSchema(version.requestSchema);
const properties = schema.properties;
const referenceSchema = properties.referenceBlobs || {};
const referenceInputs = [];
for (const media of referenceSchema["x-capabilities"] || []) {
for (const usage of media.usageConstraints || []) {
if (usage.deprecated === true) continue;
referenceInputs.push({
mediaType: String(media.mediaType || ""),
usageType: String(usage.usageType || ""),
minItems: Number.isInteger(usage.minItems) ? usage.minItems : 0,
maxItems: Number.isInteger(usage.maxItems) ? usage.maxItems : null,
maxFileSizeBytes: Number.isInteger(media.maxFileSizeBytes) ? media.maxFileSizeBytes : null,
});
}
}
const supportedSizes = [
...new Set(
branches(properties.size)
.flatMap((branch) => (Array.isArray(branch.enum) ? branch.enum : []))
.filter(
(size) =>
size &&
Number.isInteger(size.width) &&
size.width > 0 &&
Number.isInteger(size.height) &&
size.height > 0
)
.map((size) => `${size.width}x${size.height}`)
),
];
const supportedAspectRatios = [
...new Set(
branches(properties.generationSettings).flatMap((branch) =>
stringEnums(branch?.properties?.aspectRatio)
)
),
];
const duration = integerSchema(properties.duration);
const supportedDurations = [
...new Set(
branches(properties.duration)
.flatMap((branch) => (Array.isArray(branch.enum) ? branch.enum : []))
.filter(Number.isInteger)
),
];
const prompt = branches(properties.prompt).find((branch) => branch.type === "string") || {};
const outputCount = integerSchema(properties.n);
return {
id: publicModelId(family.modelId, modelVersion),
name: String(version.modelDisplayName || version.modelCaiDisplayName || modelVersion),
modality: version.outputModality[0],
upstreamModelId: family.modelId,
upstreamModelVersion: modelVersion,
providerName: String(family.acModelFamilyProviderDisplayName || ""),
releaseReadiness: String(version.releaseReadiness || ""),
healthStatus: String(version.healthStatus || ""),
inputMediaUseCases: (version.inputMediaUseCase || []).map(String),
schemaProperties: Object.keys(properties),
requiredProperties: schema.required,
referenceInputs,
maxReferenceItems: Number.isInteger(referenceSchema.maxItems) ? referenceSchema.maxItems : null,
supportedSizes,
supportedAspectRatios,
supportedResolutions: stringEnums(properties.resolution),
supportedDurations,
durationMin: Number.isInteger(duration.minimum) ? duration.minimum : null,
durationMax: Number.isInteger(duration.maximum) ? duration.maximum : null,
durationDefault: Number.isInteger(duration.default) ? duration.default : null,
outputCountMin: Number.isInteger(outputCount.minimum) ? outputCount.minimum : null,
outputCountMax: Number.isInteger(outputCount.maximum) ? outputCount.maximum : null,
promptMaxLength: Number.isInteger(prompt.maxLength) ? prompt.maxLength : null,
backingModel: String(version.bksGenerationModel || ""),
};
}
const rawModels = [];
for (const family of Array.isArray(root.models) ? root.models : []) {
for (const [modelVersion, version] of Object.entries(family.modelVersions || {})) {
if (!version || version.enabled === false) continue;
const modality = Array.isArray(version.outputModality)
? version.outputModality.map((value) => String(value).toLowerCase())[0]
: "";
if (modality !== "image" && modality !== "video") continue;
const schema = mergeObjectSchema(version.requestSchema);
if (!schema.properties.prompt) continue;
const useCases = (version.inputMediaUseCase || []).map((value) => String(value).toLowerCase());
if (useCases.some((value) => ["upscaling", "sharpening", "denoising"].includes(value))) {
continue;
}
rawModels.push(normalizeModel(family, modelVersion, version));
}
}
// Discovery currently repeats a few exact aliases (for example flux/fluxPro and
// fluxPro/1.1). Keep the first canonical wire pair and suppress duplicate cards.
const seen = new Set();
const models = [];
for (const model of rawModels) {
const semanticKey = JSON.stringify({
backingModel: model.backingModel,
name: model.name,
modality: model.modality,
schemaProperties: model.schemaProperties,
requiredProperties: model.requiredProperties,
referenceInputs: model.referenceInputs,
maxReferenceItems: model.maxReferenceItems,
supportedSizes: model.supportedSizes,
supportedAspectRatios: model.supportedAspectRatios,
supportedResolutions: model.supportedResolutions,
supportedDurations: model.supportedDurations,
durationMin: model.durationMin,
durationMax: model.durationMax,
});
if (seen.has(semanticKey)) continue;
seen.add(semanticKey);
models.push(model);
}
const source = `/**
* Generated from Adobe Firefly POST /v2/models/discovery with resolveSchema=true.
* Source SHA-256: ${sourceHash}
* Regenerate with scripts/dev/generate-adobe-firefly-snapshot.mjs; do not edit by hand.
* The generated literal stays compact to satisfy the repository's line-count gate.
*/
// prettier-ignore
export const ADOBE_FIREFLY_DISCOVERY_SNAPSHOT = ${JSON.stringify(models)} as const;
`;
fs.mkdirSync(path.dirname(outputPath), { recursive: true });
fs.writeFileSync(outputPath, source, "utf8");
console.log(`Wrote ${models.length} models to ${outputPath}`);

View File

@@ -0,0 +1,73 @@
import {
discoverAdobeFireflyModels,
resolveAdobeAccessToken,
} from "@omniroute/open-sse/services/adobeFireflyClient.ts";
import {
getAdobeFireflyFallbackCatalog,
mapDiscoveredToCatalog,
toAdobeMediaCapabilitiesApi,
type AdobeFireflyCatalogModel,
} from "@omniroute/open-sse/services/adobeFireflyModels.ts";
import { sanitizeErrorMessage } from "@omniroute/open-sse/utils/error";
type AdobeProviderData = { cookie?: unknown; access_token?: unknown; accessToken?: unknown };
interface AdobeProviderModelsResult {
models: Array<Record<string, unknown>>;
source: "api" | "local_catalog";
warning?: string;
}
function toModelResponse(model: AdobeFireflyCatalogModel): Record<string, unknown> {
const endpoint = model.modality === "image" ? "images" : "videos";
return {
id: model.id,
name: model.name,
owned_by: "adobe-firefly",
apiFormat: endpoint,
supportedEndpoints: [endpoint],
type: model.modality,
input_modalities: model.inputModalities,
output_modalities: [model.modality],
supported_sizes: model.capabilities.supportedSizes,
media_capabilities: toAdobeMediaCapabilitiesApi(model),
};
}
function fallback(warning: string): AdobeProviderModelsResult {
return {
models: getAdobeFireflyFallbackCatalog().map(toModelResponse),
source: "local_catalog",
warning,
};
}
export async function getAdobeModels(
apiKey: string | undefined,
accessToken: string | undefined,
providerData: unknown,
fetchImpl: typeof fetch = fetch
): Promise<AdobeProviderModelsResult> {
const providerSpecificData =
providerData && typeof providerData === "object" ? (providerData as AdobeProviderData) : {};
try {
const token = await resolveAdobeAccessToken(
{
apiKey,
accessToken,
providerSpecificData,
},
fetchImpl
);
const models = mapDiscoveredToCatalog(await discoverAdobeFireflyModels(token, fetchImpl));
return models.length > 0
? { models: models.map(toModelResponse), source: "api" }
: fallback("Adobe Firefly discovery returned no callable image or video models");
} catch (error) {
return fallback(
`Adobe Firefly discovery unavailable: ${sanitizeErrorMessage(
error instanceof Error ? error.message : String(error)
)}`
);
}
}

View File

@@ -84,10 +84,8 @@ import {
isAutoFetchModelsEnabled,
persistDiscoveredModels,
} from "@/lib/providerModels/modelDiscovery";
import {
buildProviderModelsUrl,
getDiscoveryClientVersionOptions,
} from "./discoveryClientVersion";
import { buildProviderModelsUrl, getDiscoveryClientVersionOptions } from "./discoveryClientVersion";
import { getAdobeModels } from "./adobeFireflyDiscovery";
import {
parseGeminiModelsList,
type GeminiDiscoveryModel,
@@ -422,10 +420,7 @@ export async function GET(
// #6267 — a models-endpoint redirect (307/308) is not a fixable-config
// error. safeOutboundFetch throws REDIRECT_BLOCKED which
// getSafeOutboundFetchErrorStatus maps to 503, but unlike the other 503
// cases (URL_GUARD_BLOCKED / INVALID_URL, which are genuinely
// unrecoverable and stay hard errors) a blocked redirect should degrade to
// the local/cached catalog OmniRoute ships instead of surfacing a raw 503.
// General fix — covers any config-driven provider that 307s (e.g. qwen-web).
// Redirect blocks degrade to the local/cached catalog; invalid URLs remain hard errors.
if (error instanceof SafeOutboundFetchError && error.code === "REDIRECT_BLOCKED") {
return buildDiscoveryFallbackResponse(warnings);
}
@@ -434,6 +429,11 @@ export async function GET(
return buildDiscoveryFallbackResponse(warnings);
};
if (provider === "adobe-firefly") {
const discovery = await getAdobeModels(apiKey, accessToken, connection.providerSpecificData);
return buildResponse({ provider, connectionId, ...discovery });
}
const maybeReturnCachedDiscovery = () => {
if (!refresh && cachedDiscoveryModels.length > 0) {
return buildCachedDiscoveryResponse();

View File

@@ -1113,6 +1113,7 @@ async function buildUnifiedModelsResponseCore(
input_modalities: imgModel.inputModalities || ["text"],
output_modalities: ["image"],
...(imgModel.description ? { description: imgModel.description } : {}),
...(imgModel.mediaCapabilities ? { media_capabilities: imgModel.mediaCapabilities } : {}),
});
}
@@ -1178,6 +1179,12 @@ async function buildUnifiedModelsResponseCore(
created: timestamp,
owned_by: videoModel.provider,
type: "video",
supported_sizes: videoModel.supportedSizes,
input_modalities: ["text"],
output_modalities: ["video"],
...(videoModel.mediaCapabilities
? { media_capabilities: videoModel.mediaCapabilities }
: {}),
});
}

View File

@@ -19,7 +19,12 @@ import {
} from "@omniroute/open-sse/services/apiKeyRotator.ts";
import { invalidateReasoningRoutingRuleCache } from "./reasoningRoutingRules";
import { normalizeProviderSpecificData } from "@/lib/providers/requestDefaults";
import { bumpProxyConfigGeneration } from "./settings";
import { bumpProxyConfigGeneration, getSettings } from "./settings";
import {
getStoredManagementPassword,
isBcryptHash,
verifyManagementPassword,
} from "@/lib/auth/managementPassword";
import { webSessionCredentialKey, parseProviderSpecificData } from "./webSessionDedup";
import { pickCodexConnectionForUser } from "@/lib/oauth/utils/codexConnectionSelection";
import { reconcileCodexUsageHistory } from "./providers/usageIdentityReconciliation";
@@ -40,6 +45,69 @@ type JsonRecord = Record<string, unknown>;
const CONNECTION_CREDENTIAL_FIELDS = ["apiKey", "accessToken", "refreshToken", "idToken"] as const;
/** Thrown when a write would store the dashboard login password as a provider credential. */
export class ManagementPasswordAsCredentialError extends Error {
readonly code = "MANAGEMENT_PASSWORD_AS_CREDENTIAL" as const;
constructor() {
super(
"That value is the dashboard login password, not a provider API key. Storing it would " +
"send it upstream on every request routed through this connection."
);
this.name = "ManagementPasswordAsCredentialError";
}
}
/**
* Refuse to store the dashboard login password as a connection API key.
*
* A browser that autofills the management password into the API-key field
* produces a connection whose credential authenticates against nothing, and
* every request routed through it comes back 401. Rejecting it in the form
* would not be enough: the same autofill fires again while an operator is
* repairing the connection by hand, so the refusal has to sit on the write
* path that all of those forms funnel into.
*
* Only an actual match blocks the write. A settings row that cannot be read,
* or a bcrypt call that throws, logs and allows -- a guard against one specific
* operator mistake must not become a way to lock out every connection write.
*
* Deliberately narrower than CONNECTION_CREDENTIAL_FIELDS. The OAuth tokens
* arrive from a provider's token endpoint, and the refresh path writes them
* back through updateProviderConnection on every renewal, so checking them
* would put a bcrypt round on a renewal path to defend a field no autofill
* reaches. apiKey is the only credential an operator types into a form.
*/
async function assertApiKeyIsNotManagementPassword(apiKey: unknown): Promise<void> {
if (typeof apiKey !== "string") return;
const trimmed = apiKey.trim();
if (!trimmed) return;
try {
const settings = (await getSettings()) as JsonRecord;
const stored = getStoredManagementPassword(settings);
// Only a stored bcrypt hash is comparable. A fresh install that has never
// bootstrapped a password has nothing to collide with.
if (!isBcryptHash(stored)) return;
// Both forms of the value, because neither the login route nor the
// set-password route trims: a paste carries whitespace the password does
// not have, and a password is allowed to carry whitespace of its own. The
// second comparison only runs when the first fails on a different string.
const matches =
(await verifyManagementPassword(trimmed, stored)) ||
(trimmed !== apiKey && (await verifyManagementPassword(apiKey, stored)));
if (!matches) return;
} catch (err) {
console.warn(
"[Providers] could not check the credential against the dashboard password:",
err instanceof Error ? err.message : String(err)
);
return;
}
throw new ManagementPasswordAsCredentialError();
}
interface StatementLike<TRow = unknown> {
all: (...params: unknown[]) => TRow[];
get: (...params: unknown[]) => TRow | undefined;
@@ -282,6 +350,7 @@ function findExistingCookieConnection(
}
export async function createProviderConnection(data: JsonRecord) {
await assertApiKeyIsNotManagementPassword(data.apiKey);
const db = getDbInstance() as unknown as DbLike;
const now = new Date().toISOString();
const normalizedProviderSpecificData = normalizeProviderSpecificData(
@@ -730,6 +799,12 @@ export async function updateProviderConnection(id: string, data: JsonRecord) {
const existing = db.prepare("SELECT * FROM provider_connections WHERE id = ?").get(id);
if (!existing) return null;
// The incoming value only. A connection that already holds the password has
// to stay editable, or an operator cannot repair the one this guard exists
// to prevent -- and re-checking the merged value would spend a bcrypt round
// on every unrelated field edit.
await assertApiKeyIsNotManagementPassword(data.apiKey);
const merged: JsonRecord = {
...toRecord(rowToCamel(existing)),
...data,

View File

@@ -0,0 +1,90 @@
import { test } from "node:test";
import assert from "node:assert";
import {
ADOBE_FIREFLY_VIDEO_MODELS,
extractAdobeSourceImageReferences,
normalizeAdobeReferenceBlobs,
} from "../../open-sse/services/adobeFireflyClient.ts";
import { getAdobeModels } from "../../src/app/api/providers/[id]/models/adobeFireflyDiscovery.ts";
function userImsJwt(): string {
const payload = Buffer.from(
JSON.stringify({
user_id: "test@AdobeID",
type: "access_token",
client_id: "clio-playground-web",
})
).toString("base64url");
return `eyJhbGciOiJSUzI1NiJ9.${payload}.${"sig".padEnd(40, "x")}`;
}
test("reference validation enforces discovered roles, counts, and frame order", () => {
const kling = ADOBE_FIREFLY_VIDEO_MODELS["kling-3"];
assert.deepEqual(
normalizeAdobeReferenceBlobs(kling, [
{ id: "frame-a", mediaType: "image", usage: "frame" },
{ id: "frame-b", mediaType: "image", usage: "frame" },
]),
[
{ id: "frame-a", usage: "frame", order: 1 },
{ id: "frame-b", usage: "frame", order: 2 },
]
);
assert.throws(
() => normalizeAdobeReferenceBlobs(kling, [{ id: "bad", mediaType: "image", usage: "mask" }]),
/does not support image references with usage 'mask'/
);
assert.throws(
() =>
normalizeAdobeReferenceBlobs(kling, [
{ id: "frame-a", usage: "frame" },
{ id: "frame-b", usage: "frame" },
{ id: "frame-c", usage: "frame" },
]),
/at most 2 frame image reference/
);
});
test("structured references skip malformed entries and preserve explicit roles", () => {
assert.deepEqual(
extractAdobeSourceImageReferences({
adobe_reference_inputs: [
null,
{ media_type: "video", source: "ignored" },
{ media_type: "image", source: "data:image/png;base64,AAAA", usage: "frame", order: 2 },
],
}),
[{ source: "data:image/png;base64,AAAA", usage: "frame", order: 2 }]
);
});
test("provider discovery adapter returns live capabilities and verified fallback", async () => {
const live = await getAdobeModels(undefined, userImsJwt(), {}, async () =>
Response.json({
models: [
{
modelId: "firefly-image",
acModelFamilyProviderDisplayName: "Adobe",
modelVersions: {
image5: {
enabled: true,
outputModality: ["image"],
modelDisplayName: "Firefly Image 5",
requestSchema: { type: "object", properties: { prompt: { type: "string" } } },
},
},
},
],
})
);
assert.equal(live.source, "api");
assert.equal(live.models[0].id, "firefly-image-image5");
assert.ok(live.models[0].media_capabilities);
const fallback = await getAdobeModels(undefined, userImsJwt(), {}, async () => {
throw new Error("offline");
});
assert.equal(fallback.source, "local_catalog");
assert.equal(fallback.models.length, 52);
assert.match(fallback.warning || "", /discovery unavailable/);
});

View File

@@ -78,6 +78,11 @@ test("adobe-firefly is registered in IMAGE_PROVIDERS with adobe-firefly-image fo
assert.equal(entry.format, "adobe-firefly-image");
assert.match(entry.baseUrl, /firefly-3p\.ff\.adobe\.io/);
assert.ok(Array.isArray(entry.models) && entry.models.length >= 4);
assert.equal(
entry.models.some((model: { id: string }) => model.id === "nano-banana-pro"),
false,
"routing-only compatibility aliases must not be advertised as discovered models"
);
});
test("adobe-firefly is registered in VIDEO_PROVIDERS with adobe-firefly-video format", () => {
@@ -154,20 +159,25 @@ test("normalizeAdobeOutputResolution maps quality tiers", () => {
assert.equal(normalizeAdobeOutputResolution(undefined, undefined), "2K");
});
test("resolveAdobeImageModel maps catalog and long model ids", () => {
assert.equal(resolveAdobeImageModel("nano-banana-pro").id, "nano-banana-pro");
assert.equal(resolveAdobeImageModel("adobe-firefly/nano-banana-2").id, "nano-banana-2");
assert.equal(resolveAdobeImageModel("firefly-nano-banana-pro-2k-16x9").id, "nano-banana-pro");
assert.equal(resolveAdobeImageModel("gpt-image").id, "gpt-image");
test("resolveAdobeImageModel maps valid aliases to exact discovery ids", () => {
assert.equal(resolveAdobeImageModel("nano-banana-pro").id, "gemini-flash-nano-banana-2");
assert.equal(
resolveAdobeImageModel("adobe-firefly/nano-banana-2").id,
"gemini-flash-nano-banana-3"
);
assert.equal(resolveAdobeImageModel("gpt-image").id, "gpt-image-2");
assert.throws(
() => resolveAdobeImageModel("invented-image-model"),
/Unknown Adobe Firefly image model/
);
assert.ok(ADOBE_FIREFLY_IMAGE_MODELS["nano-banana-pro"].upstreamModelVersion);
});
test("resolveAdobeVideoModel maps sora/veo/kling families", () => {
assert.equal(resolveAdobeVideoModel("sora-2").id, "sora-2");
assert.equal(resolveAdobeVideoModel("firefly-sora2-pro-8s-16x9").id, "sora-2-pro");
assert.equal(resolveAdobeVideoModel("veo-3.1-fast").id, "veo-3.1-fast");
assert.equal(resolveAdobeVideoModel("kling-3").id, "kling-3");
assert.ok(ADOBE_FIREFLY_VIDEO_MODELS["sora-2"].defaultDuration > 0);
test("resolveAdobeVideoModel maps only discovered video models", () => {
assert.equal(resolveAdobeVideoModel("veo-3.1-fast").id, "veo-3.1-fast-generate");
assert.equal(resolveAdobeVideoModel("kling-3").id, "kling-kling-v3-standard-i2v");
assert.throws(() => resolveAdobeVideoModel("sora-2"), /Unknown Adobe Firefly video model/);
assert.ok(ADOBE_FIREFLY_VIDEO_MODELS["veo-3.1"].defaultDuration > 0);
});
test("buildAdobeImagePayload produces nano and gpt-image shapes", () => {
@@ -265,41 +275,12 @@ test("buildAdobeImagePayload attaches referenceBlobs like live adobe_atach_image
sourceImageIds: ["aaaaaaaa-bbbb-4ccc-8ddd-eeeeeeeeeeee"],
});
assert.deepEqual(gpt.referenceBlobs, [
{ id: "aaaaaaaa-bbbb-4ccc-8ddd-eeeeeeeeeeee", usage: "subject" },
{ id: "aaaaaaaa-bbbb-4ccc-8ddd-eeeeeeeeeeee", usage: "source" },
]);
assert.equal((gpt.generationMetadata as Record<string, unknown>).module, "image2image");
// gpt-image: only first 2 subject refs survive (extra screenshots hang colligo).
const gptMany = buildAdobeImagePayload({
prompt: "edit me",
aspectRatio: "1:1",
outputResolution: "1K",
modelSpec: ADOBE_FIREFLY_IMAGE_MODELS["gpt-image-2"],
sourceImageIds: ["id-1", "id-2", "id-3", "id-4", "id-5"],
});
assert.deepEqual(gptMany.referenceBlobs, [
{ id: "id-1", usage: "subject" },
{ id: "id-2", usage: "subject" },
]);
// nano keeps up to 4 general refs for multi-panel composition.
const nanoMany = buildAdobeImagePayload({
prompt: "compose",
aspectRatio: "16:9",
outputResolution: "2K",
modelSpec: ADOBE_FIREFLY_IMAGE_MODELS["nano-banana-2"],
sourceImageIds: ["a", "b", "c", "d", "e"],
});
assert.equal((nanoMany.referenceBlobs as unknown[]).length, 4);
assert.equal((nanoMany.referenceBlobs as Array<{ usage: string }>)[0].usage, "general");
});
test("adobeFireflyMaxImageRefs + adaptive image timeout", () => {
assert.equal(adobeFireflyMaxImageRefs("gpt-image-2"), 2);
assert.equal(adobeFireflyMaxImageRefs("adobe-firefly/gpt-image"), 2);
assert.equal(adobeFireflyMaxImageRefs("nano-banana-2"), 4);
assert.equal(adobeFireflyMaxImageRefs("flux-2"), 2);
test("adobeFireflyImageTimeoutMs scales boundedly with reference count", () => {
assert.equal(adobeFireflyImageTimeoutMs({ refCount: 0 }), DEFAULT_IMAGE_TIMEOUT_MS);
assert.equal(
adobeFireflyImageTimeoutMs({ refCount: 2 }),
@@ -381,16 +362,7 @@ test("resolveAdobeSourceImageIds uploads data URLs then returns blob ids", async
assert.equal(ADOBE_FIREFLY_IMAGE_UPLOAD_URL.includes("storage/image"), true);
});
test("buildAdobeVideoPayload produces sora and veo shapes", () => {
const sora = buildAdobeVideoPayload({
prompt: "ocean waves",
aspectRatio: "16:9",
duration: 8,
modelSpec: ADOBE_FIREFLY_VIDEO_MODELS["sora-2"],
});
assert.equal(sora.modelId, "sora");
assert.equal(sora.duration, 8);
test("buildAdobeVideoPayload follows discovered fields and reference roles", () => {
const veo = buildAdobeVideoPayload({
prompt: "city flyover",
aspectRatio: "9:16",
@@ -399,12 +371,30 @@ test("buildAdobeVideoPayload produces sora and veo shapes", () => {
});
assert.equal(veo.modelId, "veo");
assert.equal(veo.modelVersion, "3.1-generate");
assert.equal(
(veo.modelSpecificPayload as Record<string, Record<string, unknown>>).parameters
.durationSeconds,
6
);
assert.equal(veo.duration, 6);
assert.equal(veo.generateAudio, true);
const kling = buildAdobeVideoPayload({
prompt: "ocean waves",
aspectRatio: "16:9",
duration: 5,
modelSpec: ADOBE_FIREFLY_VIDEO_MODELS["kling-3"],
sourceImageIds: ["aaaaaaaa-bbbb-4ccc-8ddd-eeeeeeeeeeee"],
});
assert.equal(kling.modelVersion, "kling_v3_standard_i2v");
assert.deepEqual(kling.referenceBlobs, [
{ id: "aaaaaaaa-bbbb-4ccc-8ddd-eeeeeeeeeeee", usage: "frame", order: 1 },
]);
assert.throws(
() =>
buildAdobeVideoPayload({
prompt: "bad duration",
aspectRatio: "16:9",
duration: 5,
modelSpec: ADOBE_FIREFLY_VIDEO_MODELS["veo-3.1"],
}),
/supports duration/
);
});
test("extractAdobeResultLink prefers x-override-status-link then links.result", () => {
@@ -539,7 +529,7 @@ test("adobe-firefly is in USAGE_SUPPORTED_PROVIDERS for Limits", () => {
assert.ok(USAGE_SUPPORTED_PROVIDERS.includes("firefly"));
});
test("parseAdobeModelsDiscovery extracts image/video versions", () => {
test("parseAdobeModelsDiscovery preserves schemas and maps exact ids", () => {
const rows = parseAdobeModelsDiscovery({
models: [
{
@@ -550,16 +540,44 @@ test("parseAdobeModelsDiscovery extracts image/video versions", () => {
outputModality: ["image"],
modelDisplayName: "Gemini 3.0 (Nano Banana Pro)",
healthStatus: "HEALTHY",
inputMediaUseCase: ["editing"],
bksGenerationModel: "firefly_3p:external:gemini_flash_2",
requestSchema: {
type: "object",
properties: {
prompt: { type: "string" },
referenceBlobs: {
maxItems: 14,
"x-capabilities": [
{
mediaType: "image",
usageConstraints: [{ usageType: "general", minItems: 0, maxItems: 14 }],
maxFileSizeBytes: 104857600,
},
],
},
},
},
},
},
},
{
modelId: "sora",
modelId: "veo",
modelVersions: {
"sora-2": {
"3.1-generate": {
enabled: true,
outputModality: ["video"],
modelDisplayName: "Sora 2",
modelDisplayName: "Veo 3.1",
requestSchema: {
allOf: [
{
properties: {
prompt: { type: "string" },
duration: { anyOf: [{ type: "integer", enum: [4, 6, 8] }] },
},
},
],
},
},
},
},
@@ -569,14 +587,35 @@ test("parseAdobeModelsDiscovery extracts image/video versions", () => {
assert.equal(rows[0].modality, "image");
assert.equal(rows[1].modality, "video");
const catalog = mapDiscoveredToCatalog(rows);
assert.ok(catalog.some((m) => m.id === "nano-banana-pro"));
assert.ok(catalog.some((m) => m.id === "sora-2"));
assert.ok(catalog.some((m) => m.id === "gemini-flash-nano-banana-2"));
assert.ok(catalog.some((m) => m.id === "veo-3.1-generate"));
assert.equal(catalog[0].capabilities.referenceInputs[0].maxItems, 14);
assert.deepEqual(catalog[1].capabilities.supportedDurations, [4, 6, 8]);
});
test("fallback catalog has image and video entries from get_models capture", () => {
assert.ok(ADOBE_FIREFLY_FALLBACK_MODELS.length >= 10);
assert.ok(getAdobeFireflyFallbackCatalog("image").length >= 4);
assert.ok(getAdobeFireflyFallbackCatalog("video").length >= 4);
test("fallback catalog is the verified discovery snapshot without invented Sora", () => {
assert.equal(ADOBE_FIREFLY_FALLBACK_MODELS.length, 52);
assert.equal(getAdobeFireflyFallbackCatalog("image").length, 17);
assert.equal(getAdobeFireflyFallbackCatalog("video").length, 35);
assert.equal(
ADOBE_FIREFLY_FALLBACK_MODELS.some((model) => model.id.includes("sora")),
false
);
assert.equal(
ADOBE_FIREFLY_FALLBACK_MODELS.some(
(model) => model.id.includes("kling") && model.id.includes("omni")
),
false
);
assert.ok(ADOBE_FIREFLY_FALLBACK_MODELS.some((model) => model.id === "kling-kling-o3"));
assert.equal(
ADOBE_FIREFLY_IMAGE_MODELS["nano-banana-pro"].capabilities.referenceInputs[0].maxItems,
14
);
assert.equal(
ADOBE_FIREFLY_IMAGE_MODELS["gpt-image"].capabilities.referenceInputs[0].maxItems,
16
);
});
test("extractAdobeAccountIdFromToken reads user_id claim", () => {
@@ -716,7 +755,7 @@ test("adobeFireflyGenerateVideo submit+poll happy path (mocked)", async () => {
const result = await adobeFireflyGenerateVideo({
accessToken: "tok",
prompt: "drone over forest",
model: "sora-2",
model: "veo-3.1",
duration: 4,
aspectRatio: "16:9",
fetchImpl: fetchImpl as typeof fetch,
@@ -727,7 +766,7 @@ test("adobeFireflyGenerateVideo submit+poll happy path (mocked)", async () => {
test("handleAdobeFireflyVideoGeneration returns 400 without prompt", async () => {
const result = await handleAdobeFireflyVideoGeneration({
model: "sora-2",
model: "veo-3.1",
provider: "adobe-firefly",
body: {},
credentials: { apiKey: "aaa.bbb.ccc" },

View File

@@ -0,0 +1,260 @@
/**
* The dashboard login password must never be storable as a provider API key.
*
* A browser autofilling the management password into the API-key field created
* connections that 401 every request routed through them, and the same autofill
* fired again while the connection was being repaired by hand. These assert the
* refusal sits on the write path rather than in any one form.
*/
import { after, beforeEach, describe, it } from "node:test";
import assert from "node:assert/strict";
import fs from "node:fs";
import os from "node:os";
import path from "node:path";
const TEST_DATA_DIR = fs.mkdtempSync(path.join(os.tmpdir(), "omniroute-apikey-guard-"));
process.env.DATA_DIR = TEST_DATA_DIR;
process.env.DISABLE_SQLITE_AUTO_BACKUP = "true";
const core = await import("../../src/lib/db/core.ts");
const providersDb = await import("../../src/lib/db/providers.ts");
const settingsDb = await import("../../src/lib/db/settings.ts");
const mgmt = await import("../../src/lib/auth/managementPassword.ts");
const DASHBOARD_PASSWORD = "correct-horse-battery-staple";
/** getStoredManagementPassword reads `settings.password`, holding a bcrypt hash. */
async function storeDashboardPassword(plaintext: string) {
await settingsDb.updateSettings({ password: await mgmt.hashManagementPassword(plaintext) });
}
beforeEach(() => {
core.resetDbInstance();
fs.rmSync(TEST_DATA_DIR, { recursive: true, force: true });
fs.mkdirSync(TEST_DATA_DIR, { recursive: true });
});
after(() => {
core.resetDbInstance();
fs.rmSync(TEST_DATA_DIR, { recursive: true, force: true });
});
describe("management password as a provider credential", () => {
it("create is refused when the apiKey is the dashboard password", async () => {
await storeDashboardPassword(DASHBOARD_PASSWORD);
await assert.rejects(
() =>
providersDb.createProviderConnection({
provider: "openai",
authType: "apikey",
name: "autofilled",
apiKey: DASHBOARD_PASSWORD,
isActive: true,
}),
(err: Error) => {
assert.equal(err.name, "ManagementPasswordAsCredentialError");
assert.ok(
!err.message.includes(DASHBOARD_PASSWORD),
"the refusal must not echo the password back"
);
return true;
}
);
const rows = await providersDb.getProviderConnections({ provider: "openai" });
assert.equal(rows.length, 0, "nothing may be persisted when the guard fires");
});
it("create is refused on surrounding whitespace, which a paste carries", async () => {
await storeDashboardPassword(DASHBOARD_PASSWORD);
await assert.rejects(
() =>
providersDb.createProviderConnection({
provider: "openai",
authType: "apikey",
name: "pasted",
apiKey: ` ${DASHBOARD_PASSWORD} `,
isActive: true,
}),
/dashboard login password/
);
});
it("a password that carries its own whitespace is caught too", async () => {
// Neither the login route nor the set-password route trims, so this is a
// password an operator can really have. Comparing only the trimmed value
// would miss it and store the password the guard exists to reject.
const padded = ` ${DASHBOARD_PASSWORD} `;
await storeDashboardPassword(padded);
await assert.rejects(
() =>
providersDb.createProviderConnection({
provider: "openai",
authType: "apikey",
name: "autofilled",
apiKey: padded,
isActive: true,
}),
/dashboard login password/
);
});
it("a real API key is stored normally", async () => {
await storeDashboardPassword(DASHBOARD_PASSWORD);
const conn = await providersDb.createProviderConnection({
provider: "openai",
authType: "apikey",
name: "genuine",
apiKey: "sk-a-real-provider-key",
isActive: true,
});
assert.ok(conn?.id);
});
it("update is refused too, which is where the repair attempt gets re-infected", async () => {
await storeDashboardPassword(DASHBOARD_PASSWORD);
const conn = await providersDb.createProviderConnection({
provider: "openai",
authType: "apikey",
name: "genuine",
apiKey: "sk-a-real-provider-key",
isActive: true,
});
assert.ok(conn?.id);
await assert.rejects(
() => providersDb.updateProviderConnection(conn.id as string, { apiKey: DASHBOARD_PASSWORD }),
/dashboard login password/
);
const stored = await providersDb.getProviderConnectionById(conn.id as string);
assert.equal(stored?.apiKey, "sk-a-real-provider-key", "the good key must survive the refusal");
});
it("an update that does not carry an apiKey is untouched by the guard", async () => {
await storeDashboardPassword(DASHBOARD_PASSWORD);
const conn = await providersDb.createProviderConnection({
provider: "openai",
authType: "apikey",
name: "before",
apiKey: "sk-a-real-provider-key",
isActive: true,
});
assert.ok(conn?.id);
const updated = await providersDb.updateProviderConnection(conn.id as string, {
name: "after",
});
assert.equal(updated?.name, "after");
});
it("a connection already holding the password stays editable, so it can be repaired", async () => {
// Seed the bad state the way the incident produced it: the password was
// stored before the guard existed. Write it with no dashboard password
// configured, then configure one.
const conn = await providersDb.createProviderConnection({
provider: "openai",
authType: "apikey",
name: "poisoned",
apiKey: DASHBOARD_PASSWORD,
isActive: true,
});
assert.ok(conn?.id, "no dashboard password configured yet, so the write goes through");
await storeDashboardPassword(DASHBOARD_PASSWORD);
const repaired = await providersDb.updateProviderConnection(conn.id as string, {
apiKey: "sk-the-actual-key",
});
assert.equal(repaired?.apiKey, "sk-the-actual-key");
});
it("no dashboard password configured means nothing to collide with", async () => {
const conn = await providersDb.createProviderConnection({
provider: "openai",
authType: "apikey",
name: "fresh install",
apiKey: DASHBOARD_PASSWORD,
isActive: true,
});
assert.ok(conn?.id);
});
it("an OAuth connection carrying no apiKey never reaches the bcrypt round", async () => {
// Storing a hash bcrypt cannot parse turns the expensive path into an
// observable one: reaching it throws and the catch logs. Silence is then
// proof the guard returned before touching settings at all, which is what
// keeps a token renewal -- a write that carries tokens but no apiKey --
// from paying for a database read and a bcrypt round on every renewal.
await settingsDb.updateSettings({ password: `$2a$99$${"a".repeat(53)}` });
const warnings: string[] = [];
const realWarn = console.warn;
console.warn = (...args: unknown[]) => {
warnings.push(args.map((a) => String(a)).join(" "));
};
try {
const conn = await providersDb.createProviderConnection({
provider: "claude",
authType: "oauth",
name: "oauth",
accessToken: DASHBOARD_PASSWORD,
isActive: true,
});
assert.ok(conn?.id, "the guard covers apiKey only");
} finally {
console.warn = realWarn;
}
assert.equal(
warnings.filter((w) => w.includes("could not check the credential")).length,
0,
"a write carrying no apiKey must not reach settings or bcrypt"
);
});
it("a check that cannot complete allows the write instead of blocking it", async () => {
// isBcryptHash validates shape, not semantics, so a stored hash carrying an
// impossible cost factor passes it and then makes bcrypt.compare throw. That
// reaches the same branch as an unreadable settings row, without a mock.
const unparseable = `$2a$99$${"a".repeat(53)}`;
// Asserted rather than assumed. Should bcrypt ever resolve instead of
// throwing here, the guard would take its no-match return and this test
// would quietly stop covering the catch branch, so name the reason first.
await assert.rejects(() => mgmt.verifyManagementPassword("anything", unparseable));
await settingsDb.updateSettings({ password: unparseable });
const warnings: string[] = [];
const realWarn = console.warn;
console.warn = (...args: unknown[]) => {
warnings.push(args.map((a) => String(a)).join(" "));
};
try {
const conn = await providersDb.createProviderConnection({
provider: "openai",
authType: "apikey",
name: "unverifiable",
apiKey: DASHBOARD_PASSWORD,
isActive: true,
});
assert.ok(conn?.id, "one broken settings row must not block every connection write");
} finally {
console.warn = realWarn;
}
assert.ok(
warnings.some((w) => w.includes("could not check the credential")),
"failing open silently would hide that the guard stopped guarding"
);
});
});