mirror of
https://github.com/diegosouzapw/OmniRoute.git
synced 2026-08-23 15:42:12 +03:00
* feat(providers): add Cloudflare AI Playground as No Auth provider (closes #10389) Reverse-engineered access to the free, anonymous Cloudflare AI Playground: chat runs over a PartySocket WebSocket speaking Cloudflare's cf_agent RPC protocol with zero credentials (no account, no API key, no cookies). The WS upgrade is gated on a browser-grade TLS fingerprint, so the executor drives a headless Chromium via Playwright and speaks the protocol from inside the page context. - registry entry: cloudflare-playground (alias cfp), authType none, curated 20-model catalog (GLM 5.2, Kimi K2.7 Code, DeepSeek V4 Pro, gpt-oss-120B, Llama 3.3 70B, Qwen2.5 Coder 32B, ...) captured from the live getModels RPC (2026-08-15) - executor: cf_agent frame stream -> OpenAI SSE translation, id-filtered parser (RPC done:true frames cannot kill the stream), in-band upstream errors mapped to HTTP 429/502, abort + timeout handling, clean errors - noauth UI entry with reverse-engineered-endpoint notice - tests: 12 unit tests using real captured frames (incl. the 3021 rate-limit error) + fake transport; ESLint clean; open-sse typecheck clean * fix(providers): define __name helper in page context before evaluate Bundlers with keepNames (esbuild/tsx, webpack) inject a __name() call into serialized function bodies. page.evaluate(openPlaygroundSession) therefore threw ReferenceError: __name is not defined in real browser sessions. Define the helper on window before evaluating the session opener. * fix(providers): sync docs counts, golden snapshots and add reasoning_content support for cloudflare-playground * chore: remove ad-hoc cfp-shim debug script per review feedback The standalone shim duplicated the executor's frame-parsing and transport logic and is superseded by open-sse/executors/cloudflare-playground.ts. Requested in PR #10442 review. * feat(gemini-web): expose image generation through /v1/images/generations (closes #10466) Adds a gemini-web image-generation path following the chatgpt-web precedent: - imageRegistry: gemini-web provider entry (format gemini-web, cookie auth) with the nano-banana-web model. The -web suffix keeps the bare nano-banana id owned by adobe-firefly (operator decision 2026-07-31). - gemini-web executor: new parseStreamResponseImages() extracts generated image URLs from the StreamGenerate candidate extension block (inner[4][0][12][7][0], url at entry[0][3][3] — string or list form), dedupes cumulative frames, upgrades to =s2048, and deliberately skips web-search thumbnails at [12][1]. Image mode (x_gemini_web_image_mode) captures every StreamGenerate frame, resolves on first image, and gets a 90s window; chat mode is byte-for-byte unchanged. - handlers/imageGeneration/providers/geminiWeb.ts: drives the executor in image mode with an explicit generation directive prompt (the web UI otherwise answers with web-search images), caps n at 4, returns URLs or b64_json (downloads the public googleusercontent asset), and surfaces refusal text when no image was produced. - Dispatch branch on format gemini-web in handleImageGeneration. Tests: 21 new tests with fixtures built from the documented frame layout (string/list url forms, cumulative-frame dedupe, web-image exclusion, size-directive handling, refusal visibility, n-cap, b64_json, registry wiring incl. the bare nano-banana → adobe-firefly regression guard). Adjacent suites: gemini-web (6 files), chatgpt-web image, image handler, route, registry, adobe-firefly, freepik, designer — all green. ESLint clean on touched files (2 pre-existing any warnings unchanged); tsc -p open-sse 0 errors. * fix(media): close browser leak, surface timeout errors, and fall back accounts for gemini-web images Addresses pre-merge review findings on #10494 (closes #10466): - cloudflare-playground executor: close the launched browser on EVERY non-success start() path, including the detected Cloudflare "Attention Required" challenge branch (was leaking a Chromium process per blocked request). - cloudflare-playground executor: a streaming chat timeout now emits an explicit timeout_error SSE chunk before [DONE] instead of silently completing, so a client can no longer mistake an empty/partial timed-out stream for a successful answer. Timeout duration is now injectable for deterministic tests. - gemini-web image handler + imageCredentialRetry: classify the underlying GeminiWebExecutor's expired/blocked-session failure modes (400/500, per its own Playwright timeout/catch-all branches) as retryable, so executeImageWithCredentialFallback advances to the next eligible account instead of only doing so on a plain 401. Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com> * docs: regenerate provider counts after merging release/v3.8.50 (341 -> 342) The previous merge commit resolved all 51 auto-generated-file conflicts by taking release/v3.8.50's content, which still said 341 providers. Merging in this branch's Cloudflare Playground provider brings the live catalog to 342, so npm run check:docs-counts-sync now flags stale claims. Fix: - docs/reference/PROVIDER_REFERENCE.md: regenerated via `npm run gen:provider-reference`. - README.md/AGENTS.md/llm.txt/package.json description: 341 -> 342. - docs/diagrams/{readme-hero,promise-pillars,comparison-table,cli-terminal}.svg: 341 -> 342 in the embedded "NNN providers" text (targeted replace, matched against the exact pattern check-docs-counts-sync.mjs validates). check:docs-counts-sync and check:changelog-integrity are both clean after this commit. Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com> * docs(env): document CLOUDFLARE_PLAYGROUND_CHROME_PATH Used by open-sse/executors/cloudflare-playground.ts but missing from .env.example and docs/reference/ENVIRONMENT.md, caught by the env-doc-sync gate when combined with other PRs in the release merge-train. Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com> --------- Co-authored-by: user.email <freakymustard67@gmail.com> Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
175 lines
7.4 KiB
TypeScript
175 lines
7.4 KiB
TypeScript
// #10494: Gemini Web image-generation account fallback gap.
|
|
//
|
|
// #10466's acceptance criteria require that "expired or blocked sessions
|
|
// return a clear session/provider error and can fall back normally inside an
|
|
// image Combo." The gemini-web image handler passed the executor's raw HTTP
|
|
// status straight through to executeImageWithCredentialFallback, whose retry
|
|
// loop only advances to the next account on a plain HTTP 401 — but the
|
|
// underlying GeminiWebExecutor's browser-automation catch paths surface an
|
|
// expired/blocked session as 400 (Playwright selector/click timeout — "the
|
|
// session is so expired it lands on a different page", #9407) or 500 (the
|
|
// generic automation-failure catch-all), never 401. So expired/blocked
|
|
// Gemini Web sessions never triggered account fallback.
|
|
//
|
|
// Covers:
|
|
// - isExpiredOrBlockedGeminiWebSession() classification (unit).
|
|
// - A multi-account regression: first account fails with a classified
|
|
// status, the retry loop advances to a second account, which succeeds.
|
|
// - An invalid-session test that drives the REAL GeminiWebExecutor (Playwright
|
|
// launch mocked, same technique as tests/unit/gemini-web.test.ts) so the
|
|
// classified status is the executor's actual status code, not a synthetic
|
|
// one, and confirms the handler marks it retryable end to end.
|
|
import test from "node:test";
|
|
import assert from "node:assert/strict";
|
|
import { mkdtempSync, rmSync } from "node:fs";
|
|
import { tmpdir } from "node:os";
|
|
import { join } from "node:path";
|
|
|
|
const TEST_DATA_DIR = mkdtempSync(join(tmpdir(), "omniroute-geminiweb-image-fallback-"));
|
|
process.env.DATA_DIR = TEST_DATA_DIR;
|
|
|
|
const { isExpiredOrBlockedGeminiWebSession, handleGeminiWebImageGeneration } = await import(
|
|
"../../open-sse/handlers/imageGeneration/providers/geminiWeb.ts"
|
|
);
|
|
const { executeImageWithCredentialFallback } = await import(
|
|
"../../src/sse/services/imageCredentialRetry.ts"
|
|
);
|
|
const { GeminiWebExecutor } = await import("../../open-sse/executors/gemini-web.ts");
|
|
const core = await import("../../src/lib/db/core.ts");
|
|
|
|
test.after(() => {
|
|
core.resetDbInstance();
|
|
rmSync(TEST_DATA_DIR, { recursive: true, force: true });
|
|
});
|
|
|
|
// ── Classification (unit) ───────────────────────────────────────────────────
|
|
|
|
test("isExpiredOrBlockedGeminiWebSession classifies 400/500 as retryable, everything else as not", () => {
|
|
assert.equal(isExpiredOrBlockedGeminiWebSession(400), true);
|
|
assert.equal(isExpiredOrBlockedGeminiWebSession(500), true);
|
|
assert.equal(isExpiredOrBlockedGeminiWebSession(401), false, "handled by the plain 401 path");
|
|
assert.equal(
|
|
isExpiredOrBlockedGeminiWebSession(503),
|
|
false,
|
|
"missing-Playwright-browser is a host/config problem, not a per-account issue"
|
|
);
|
|
assert.equal(isExpiredOrBlockedGeminiWebSession(502), false);
|
|
assert.equal(isExpiredOrBlockedGeminiWebSession(200), false);
|
|
});
|
|
|
|
// ── Multi-account regression: 2 accounts, first classified-fails, second succeeds ──
|
|
|
|
test("executeImageWithCredentialFallback: expired/blocked (400) on account 1 falls back to account 2", async () => {
|
|
const attempts: string[] = [];
|
|
const accountA = { connectionId: "conn-a", apiKey: "cookie-a" };
|
|
const accountB = { connectionId: "conn-b", apiKey: "cookie-b" };
|
|
|
|
const execution = await executeImageWithCredentialFallback({
|
|
provider: "gemini-web",
|
|
requestedModel: "gemini-2.5-pro",
|
|
credentials: accountA,
|
|
// Simulates the real handler path: geminiWeb.ts sets retryable via
|
|
// saveImageErrorResult when the executor status is classified as an
|
|
// expired/blocked session (400/500), not just a plain 401.
|
|
execute: async (creds) => {
|
|
attempts.push(creds.connectionId);
|
|
if (creds.connectionId === "conn-a") {
|
|
return { success: false, status: 400, error: "session expired", retryable: true };
|
|
}
|
|
return { success: true, data: { created: 1, data: [{ url: "https://example/img.png" }] } };
|
|
},
|
|
selectNextCredentials: async () => accountB,
|
|
});
|
|
|
|
assert.deepEqual(attempts, ["conn-a", "conn-b"], "must try both accounts in order");
|
|
assert.equal(execution.result.success, true);
|
|
assert.equal(execution.credentials.connectionId, "conn-b");
|
|
});
|
|
|
|
test("executeImageWithCredentialFallback: a non-retryable 400 (e.g. bad prompt) does NOT burn a second account", async () => {
|
|
const attempts: string[] = [];
|
|
const accountA = { connectionId: "conn-a", apiKey: "cookie-a" };
|
|
|
|
const execution = await executeImageWithCredentialFallback({
|
|
provider: "gemini-web",
|
|
requestedModel: "gemini-2.5-pro",
|
|
credentials: accountA,
|
|
execute: async (creds) => {
|
|
attempts.push(creds.connectionId);
|
|
return { success: false, status: 400, error: "Prompt is required" }; // retryable unset
|
|
},
|
|
selectNextCredentials: async () => {
|
|
throw new Error("must not be called for a non-retryable failure");
|
|
},
|
|
});
|
|
|
|
assert.deepEqual(attempts, ["conn-a"]);
|
|
assert.equal(execution.result.success, false);
|
|
assert.equal(execution.result.status, 400);
|
|
});
|
|
|
|
// ── Invalid-session test against the REAL executor's actual status code ────
|
|
|
|
test("handler classifies the REAL GeminiWebExecutor's session-expired 400 as retryable", async () => {
|
|
const playwright = await import("playwright");
|
|
const originalLaunch = playwright.chromium.launch;
|
|
|
|
// Mirrors tests/unit/gemini-web.test.ts's pattern for a fake page whose
|
|
// waitForSelector() times out — the exact path (#9407) that makes the
|
|
// real executor return a 400 tagged "the session is so expired it lands
|
|
// on a different page".
|
|
playwright.chromium.launch = (async () =>
|
|
({
|
|
newContext: async () => ({
|
|
addCookies: async () => {},
|
|
newPage: async () => ({
|
|
on: () => {},
|
|
goto: async () => {},
|
|
waitForTimeout: async () => {},
|
|
waitForSelector: async () => {
|
|
const err = new Error("Timeout 10000ms exceeded while waiting for selector");
|
|
err.name = "TimeoutError";
|
|
throw err;
|
|
},
|
|
}),
|
|
}),
|
|
close: async () => {},
|
|
}) as unknown as ReturnType<typeof playwright.chromium.launch>) as typeof playwright.chromium.launch;
|
|
|
|
try {
|
|
const executor = new GeminiWebExecutor();
|
|
const direct = await executor.execute({
|
|
model: "gemini-2.5-pro",
|
|
body: { messages: [{ role: "user", content: "hi" }], x_gemini_web_image_mode: true },
|
|
stream: false,
|
|
credentials: { apiKey: "expired-session-cookie" },
|
|
signal: AbortSignal.timeout(10000),
|
|
log: null,
|
|
});
|
|
// Confirm the REAL executor really does surface this as 400 (not a
|
|
// synthetic status invented by the test).
|
|
assert.equal(direct.response.status, 400, "sanity: executor's real session-expired status");
|
|
|
|
const res = await handleGeminiWebImageGeneration({
|
|
model: "gemini-2.5-pro",
|
|
provider: "gemini-web",
|
|
body: { prompt: "a kitten" },
|
|
credentials: { apiKey: "expired-session-cookie", connectionId: "conn-real" },
|
|
log: null,
|
|
signal: null,
|
|
clientHeaders: {},
|
|
executorFactory: () => new GeminiWebExecutor(),
|
|
});
|
|
|
|
assert.equal(res.success, false);
|
|
assert.equal(res.status, 400);
|
|
assert.equal(
|
|
(res as { retryable?: boolean }).retryable,
|
|
true,
|
|
"the handler must mark the real executor's session-expired status as retryable"
|
|
);
|
|
} finally {
|
|
playwright.chromium.launch = originalLaunch;
|
|
}
|
|
});
|