Files
OmniRoute/tests/unit/gemini-web-image-account-fallback.test.ts
Abhishek4512009 885cd8c411 feat(gemini-web): expose image generation through /v1/images/generations (closes #10466) (#10494)
* feat(providers): add Cloudflare AI Playground as No Auth provider (closes #10389)

Reverse-engineered access to the free, anonymous Cloudflare AI Playground:
chat runs over a PartySocket WebSocket speaking Cloudflare's cf_agent RPC
protocol with zero credentials (no account, no API key, no cookies). The
WS upgrade is gated on a browser-grade TLS fingerprint, so the executor
drives a headless Chromium via Playwright and speaks the protocol from
inside the page context.

- registry entry: cloudflare-playground (alias cfp), authType none,
  curated 20-model catalog (GLM 5.2, Kimi K2.7 Code, DeepSeek V4 Pro,
  gpt-oss-120B, Llama 3.3 70B, Qwen2.5 Coder 32B, ...) captured from the
  live getModels RPC (2026-08-15)
- executor: cf_agent frame stream -> OpenAI SSE translation, id-filtered
  parser (RPC done:true frames cannot kill the stream), in-band upstream
  errors mapped to HTTP 429/502, abort + timeout handling, clean errors
- noauth UI entry with reverse-engineered-endpoint notice
- tests: 12 unit tests using real captured frames (incl. the 3021
  rate-limit error) + fake transport; ESLint clean; open-sse typecheck clean

* fix(providers): define __name helper in page context before evaluate

Bundlers with keepNames (esbuild/tsx, webpack) inject a __name() call into
serialized function bodies. page.evaluate(openPlaygroundSession) therefore
threw ReferenceError: __name is not defined in real browser sessions.
Define the helper on window before evaluating the session opener.

* fix(providers): sync docs counts, golden snapshots and add reasoning_content support for cloudflare-playground

* chore: remove ad-hoc cfp-shim debug script per review feedback

The standalone shim duplicated the executor's frame-parsing and transport
logic and is superseded by open-sse/executors/cloudflare-playground.ts.
Requested in PR #10442 review.

* feat(gemini-web): expose image generation through /v1/images/generations (closes #10466)

Adds a gemini-web image-generation path following the chatgpt-web precedent:

- imageRegistry: gemini-web provider entry (format gemini-web, cookie auth)
  with the nano-banana-web model. The -web suffix keeps the bare
  nano-banana id owned by adobe-firefly (operator decision 2026-07-31).
- gemini-web executor: new parseStreamResponseImages() extracts generated
  image URLs from the StreamGenerate candidate extension block
  (inner[4][0][12][7][0], url at entry[0][3][3] — string or list form),
  dedupes cumulative frames, upgrades to =s2048, and deliberately skips
  web-search thumbnails at [12][1]. Image mode (x_gemini_web_image_mode)
  captures every StreamGenerate frame, resolves on first image, and gets
  a 90s window; chat mode is byte-for-byte unchanged.
- handlers/imageGeneration/providers/geminiWeb.ts: drives the executor in
  image mode with an explicit generation directive prompt (the web UI
  otherwise answers with web-search images), caps n at 4, returns URLs or
  b64_json (downloads the public googleusercontent asset), and surfaces
  refusal text when no image was produced.
- Dispatch branch on format gemini-web in handleImageGeneration.

Tests: 21 new tests with fixtures built from the documented frame layout
(string/list url forms, cumulative-frame dedupe, web-image exclusion,
size-directive handling, refusal visibility, n-cap, b64_json, registry
wiring incl. the bare nano-banana → adobe-firefly regression guard).
Adjacent suites: gemini-web (6 files), chatgpt-web image, image handler,
route, registry, adobe-firefly, freepik, designer — all green.
ESLint clean on touched files (2 pre-existing any warnings unchanged);
tsc -p open-sse 0 errors.

* fix(media): close browser leak, surface timeout errors, and fall back accounts for gemini-web images

Addresses pre-merge review findings on #10494 (closes #10466):

- cloudflare-playground executor: close the launched browser on EVERY
  non-success start() path, including the detected Cloudflare "Attention
  Required" challenge branch (was leaking a Chromium process per blocked
  request).
- cloudflare-playground executor: a streaming chat timeout now emits an
  explicit timeout_error SSE chunk before [DONE] instead of silently
  completing, so a client can no longer mistake an empty/partial timed-out
  stream for a successful answer. Timeout duration is now injectable for
  deterministic tests.
- gemini-web image handler + imageCredentialRetry: classify the underlying
  GeminiWebExecutor's expired/blocked-session failure modes (400/500, per
  its own Playwright timeout/catch-all branches) as retryable, so
  executeImageWithCredentialFallback advances to the next eligible account
  instead of only doing so on a plain 401.

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>

* docs: regenerate provider counts after merging release/v3.8.50 (341 -> 342)

The previous merge commit resolved all 51 auto-generated-file conflicts by
taking release/v3.8.50's content, which still said 341 providers. Merging in
this branch's Cloudflare Playground provider brings the live catalog to 342,
so npm run check:docs-counts-sync now flags stale claims. Fix:

- docs/reference/PROVIDER_REFERENCE.md: regenerated via
  `npm run gen:provider-reference`.
- README.md/AGENTS.md/llm.txt/package.json description: 341 -> 342.
- docs/diagrams/{readme-hero,promise-pillars,comparison-table,cli-terminal}.svg:
  341 -> 342 in the embedded "NNN providers" text (targeted replace, matched
  against the exact pattern check-docs-counts-sync.mjs validates).

check:docs-counts-sync and check:changelog-integrity are both clean after
this commit.

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>

* docs(env): document CLOUDFLARE_PLAYGROUND_CHROME_PATH

Used by open-sse/executors/cloudflare-playground.ts but missing from
.env.example and docs/reference/ENVIRONMENT.md, caught by the
env-doc-sync gate when combined with other PRs in the release
merge-train.

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>

---------

Co-authored-by: user.email <freakymustard67@gmail.com>
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
2026-08-18 11:43:16 -03:00

175 lines
7.4 KiB
TypeScript

// #10494: Gemini Web image-generation account fallback gap.
//
// #10466's acceptance criteria require that "expired or blocked sessions
// return a clear session/provider error and can fall back normally inside an
// image Combo." The gemini-web image handler passed the executor's raw HTTP
// status straight through to executeImageWithCredentialFallback, whose retry
// loop only advances to the next account on a plain HTTP 401 — but the
// underlying GeminiWebExecutor's browser-automation catch paths surface an
// expired/blocked session as 400 (Playwright selector/click timeout — "the
// session is so expired it lands on a different page", #9407) or 500 (the
// generic automation-failure catch-all), never 401. So expired/blocked
// Gemini Web sessions never triggered account fallback.
//
// Covers:
// - isExpiredOrBlockedGeminiWebSession() classification (unit).
// - A multi-account regression: first account fails with a classified
// status, the retry loop advances to a second account, which succeeds.
// - An invalid-session test that drives the REAL GeminiWebExecutor (Playwright
// launch mocked, same technique as tests/unit/gemini-web.test.ts) so the
// classified status is the executor's actual status code, not a synthetic
// one, and confirms the handler marks it retryable end to end.
import test from "node:test";
import assert from "node:assert/strict";
import { mkdtempSync, rmSync } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
const TEST_DATA_DIR = mkdtempSync(join(tmpdir(), "omniroute-geminiweb-image-fallback-"));
process.env.DATA_DIR = TEST_DATA_DIR;
const { isExpiredOrBlockedGeminiWebSession, handleGeminiWebImageGeneration } = await import(
"../../open-sse/handlers/imageGeneration/providers/geminiWeb.ts"
);
const { executeImageWithCredentialFallback } = await import(
"../../src/sse/services/imageCredentialRetry.ts"
);
const { GeminiWebExecutor } = await import("../../open-sse/executors/gemini-web.ts");
const core = await import("../../src/lib/db/core.ts");
test.after(() => {
core.resetDbInstance();
rmSync(TEST_DATA_DIR, { recursive: true, force: true });
});
// ── Classification (unit) ───────────────────────────────────────────────────
test("isExpiredOrBlockedGeminiWebSession classifies 400/500 as retryable, everything else as not", () => {
assert.equal(isExpiredOrBlockedGeminiWebSession(400), true);
assert.equal(isExpiredOrBlockedGeminiWebSession(500), true);
assert.equal(isExpiredOrBlockedGeminiWebSession(401), false, "handled by the plain 401 path");
assert.equal(
isExpiredOrBlockedGeminiWebSession(503),
false,
"missing-Playwright-browser is a host/config problem, not a per-account issue"
);
assert.equal(isExpiredOrBlockedGeminiWebSession(502), false);
assert.equal(isExpiredOrBlockedGeminiWebSession(200), false);
});
// ── Multi-account regression: 2 accounts, first classified-fails, second succeeds ──
test("executeImageWithCredentialFallback: expired/blocked (400) on account 1 falls back to account 2", async () => {
const attempts: string[] = [];
const accountA = { connectionId: "conn-a", apiKey: "cookie-a" };
const accountB = { connectionId: "conn-b", apiKey: "cookie-b" };
const execution = await executeImageWithCredentialFallback({
provider: "gemini-web",
requestedModel: "gemini-2.5-pro",
credentials: accountA,
// Simulates the real handler path: geminiWeb.ts sets retryable via
// saveImageErrorResult when the executor status is classified as an
// expired/blocked session (400/500), not just a plain 401.
execute: async (creds) => {
attempts.push(creds.connectionId);
if (creds.connectionId === "conn-a") {
return { success: false, status: 400, error: "session expired", retryable: true };
}
return { success: true, data: { created: 1, data: [{ url: "https://example/img.png" }] } };
},
selectNextCredentials: async () => accountB,
});
assert.deepEqual(attempts, ["conn-a", "conn-b"], "must try both accounts in order");
assert.equal(execution.result.success, true);
assert.equal(execution.credentials.connectionId, "conn-b");
});
test("executeImageWithCredentialFallback: a non-retryable 400 (e.g. bad prompt) does NOT burn a second account", async () => {
const attempts: string[] = [];
const accountA = { connectionId: "conn-a", apiKey: "cookie-a" };
const execution = await executeImageWithCredentialFallback({
provider: "gemini-web",
requestedModel: "gemini-2.5-pro",
credentials: accountA,
execute: async (creds) => {
attempts.push(creds.connectionId);
return { success: false, status: 400, error: "Prompt is required" }; // retryable unset
},
selectNextCredentials: async () => {
throw new Error("must not be called for a non-retryable failure");
},
});
assert.deepEqual(attempts, ["conn-a"]);
assert.equal(execution.result.success, false);
assert.equal(execution.result.status, 400);
});
// ── Invalid-session test against the REAL executor's actual status code ────
test("handler classifies the REAL GeminiWebExecutor's session-expired 400 as retryable", async () => {
const playwright = await import("playwright");
const originalLaunch = playwright.chromium.launch;
// Mirrors tests/unit/gemini-web.test.ts's pattern for a fake page whose
// waitForSelector() times out — the exact path (#9407) that makes the
// real executor return a 400 tagged "the session is so expired it lands
// on a different page".
playwright.chromium.launch = (async () =>
({
newContext: async () => ({
addCookies: async () => {},
newPage: async () => ({
on: () => {},
goto: async () => {},
waitForTimeout: async () => {},
waitForSelector: async () => {
const err = new Error("Timeout 10000ms exceeded while waiting for selector");
err.name = "TimeoutError";
throw err;
},
}),
}),
close: async () => {},
}) as unknown as ReturnType<typeof playwright.chromium.launch>) as typeof playwright.chromium.launch;
try {
const executor = new GeminiWebExecutor();
const direct = await executor.execute({
model: "gemini-2.5-pro",
body: { messages: [{ role: "user", content: "hi" }], x_gemini_web_image_mode: true },
stream: false,
credentials: { apiKey: "expired-session-cookie" },
signal: AbortSignal.timeout(10000),
log: null,
});
// Confirm the REAL executor really does surface this as 400 (not a
// synthetic status invented by the test).
assert.equal(direct.response.status, 400, "sanity: executor's real session-expired status");
const res = await handleGeminiWebImageGeneration({
model: "gemini-2.5-pro",
provider: "gemini-web",
body: { prompt: "a kitten" },
credentials: { apiKey: "expired-session-cookie", connectionId: "conn-real" },
log: null,
signal: null,
clientHeaders: {},
executorFactory: () => new GeminiWebExecutor(),
});
assert.equal(res.success, false);
assert.equal(res.status, 400);
assert.equal(
(res as { retryable?: boolean }).retryable,
true,
"the handler must mark the real executor's session-expired status as retryable"
);
} finally {
playwright.chromium.launch = originalLaunch;
}
});