fix(grok-web): forward cf_clearance/__cf_bm + honest IP-reputation message (#5350) (#5358)

Integrated into release/v3.8.41 (#5242/#5350 fixes). Tests green per PR.

CHANGELOG.md

@@ -10,6 +10,10 @@ _In development — bullets added per PR; finalized at release._
 
 ### 🔧 Bug Fixes
 
+- **grok-web:** forward the Cloudflare clearance cookies and stop mislabeling IP-reputation blocks as a bad cookie. "Check cookie" returned `Invalid SSO cookie` even with a valid, complete browser session — but the cookie parser was never the problem (it robustly extracts `sso`/`sso-rw` from a full DevTools header). Two real gaps fixed: **(1)** `buildGrokCookieHeader` now forwards `cf_clearance` and `__cf_bm` when pasted (it dropped them before; AIClient2API forwards them too) — strictly additive, a bare `sso` blob still yields exactly `sso=…`; **(2)** when the user supplied a `cf_clearance`, a 401 / invalid-credentials-403 from grok.com is now surfaced as an IP-reputation/anti-bot block (cf_clearance is IP+TLS+UA-pinned and can't be replayed from a different machine) instead of the misleading "Invalid SSO cookie — re-paste". A bare cookie with no clearance still gets the re-paste hint. Regression guards in `web-cookie-auth.test.ts` + `provider-validation-specialty.test.ts`. ([#5350](https://github.com/diegosouzapw/OmniRoute/issues/5350) — thanks @SeaXen)
+
+### 🔧 Bug Fixes
+
 - **opencode/observability:** make OpenCode Free account/proxy rotation visible and fix two real defects surfaced alongside it. **(1)** the per-request rotation selection log (`dispatch via account … through proxy …`) was `debug` (hidden at default `APP_LOG_LEVEL=info`) — promoted to `info` so the shuffle/cooldown lifecycle is auditable (token stays masked). **(2)** `[ProxyEgress]` reported `proxy=direct` even when an account proxy was applied, because the egress logger ran outside the executor's nested proxy context — the effective applied proxy is now captured (via an applied-proxy sink threaded through the proxy AsyncLocalStorage) and reflected in the egress log. **(3)** `[callLogs] too many SQL variables` — `deleteCallLogRowsByIds` deleted up to 5000 ids in one `IN (…)`, exceeding SQLite's ~999 bound-param cap and aborting log trimming/retention; ids are now chunked (≤500 per statement). Regression guards: `tests/unit/call-log-trim-sql-vars-5217.test.ts`, `apply-executor-proxy-info-5217.test.ts`, extended `opencode-proxy-rotation-4954.test.ts`. The Proxy Pool dropdown (by-id) UI (Gap 1) is a follow-up requiring browser validation. ([#5217](https://github.com/diegosouzapw/OmniRoute/issues/5217) — thanks @daniij)
 - **chatgpt-web:** wire tool/function calling into the `chatgpt-web` provider. It was the only web-session executor that never read `body.tools` — both response builders hardcoded `finish_reason:"stop"` and emitted only content, so tool calls were silently dropped (the model answered in prose). It now uses the shared `webTools` prompt-emulation shim (a `<tool>`-contract system message + `<tool>{…}</tool>` response parsing) exactly like its 9 sibling executors (qwen-web, perplexity-web, …) — it was simply omitted from the #3259 rollout. Tool mode buffers and emits `tool_calls` + `finish_reason:"tool_calls"` (gated off the image-gen path); plain chat is unchanged. Regression guard: `tests/unit/chatgpt-web-tools-5240.test.ts`. ([#5240](https://github.com/diegosouzapw/OmniRoute/issues/5240) — thanks @Rougler)
 - **oauth/dashboard:** fix the persistent/false Antigravity "Token Expired" badge (continuation of #3679/#3850). Two causes: **(1)** new OAuth connections never set `tokenExpiresAt` (only `expiresAt`), so the dashboard badge — which prefers `tokenExpiresAt || expiresAt` — fell back to the original grant clock and could flash a false "Token Expired" until the first background refresh. Creation now mirrors `expiresAt` into `tokenExpiresAt` across all 5 OAuth create paths (a shared `buildOAuthConnectionCreatePayload`), consistent with every refresh path which already writes both. **(2)** when a refresh-capable connection has no usable refresh token, the health-check sweep silently skipped it, leaving `testStatus="active"` forever while the cosmetic badge showed expired; it now surfaces a terminal `testStatus="expired"` ("needs re-auth"), tightly gated so it never clobbers non-refresh providers or already-terminal/cooldown states. Regression guards: `tests/unit/oauth-connection-tokenexpiresat-5326.test.ts`, `tests/unit/token-health-no-refresh-token-expired-5326.test.ts`. ([#5326](https://github.com/diegosouzapw/OmniRoute/issues/5326))

src/lib/providers/validation/webProvidersA.ts

@@ -182,6 +182,17 @@ export function isGrokAntiBotBlock(body: string | null | undefined): boolean {
   return false;
 }
 
+// Shared IP-reputation / anti-bot guidance (#3474, #5350). The request was rejected
+// before (or independently of) auth — the cookie itself is likely fine. cf_clearance
+// is pinned to the IP + TLS fingerprint + User-Agent that earned it and cannot be
+// replayed from a different machine/IP, so an auth-shaped rejection after a
+// cf_clearance was supplied is almost always this block, not a bad cookie.
+const GROK_IP_REPUTATION_GUIDANCE =
+  "Your sso cookie is likely fine — this is an IP-reputation block on the request, not an " +
+  "auth failure. cf_clearance is pinned to the IP + TLS fingerprint + User-Agent that earned " +
+  "it and cannot be replayed from a different machine/IP. Retry from a residential IP or " +
+  "configure a proxy for grok-web.";
+
 export async function validateGrokWebProvider({ apiKey, providerSpecificData = {} }: any) {
   try {
     const token = extractCookieValue(apiKey, "sso");
@@ -291,7 +302,17 @@ export async function validateGrokWebProvider({ apiKey, providerSpecificData = {
       return { valid: true, error: null };
     }
 
+    // Did the user actually supply a cf_clearance cookie? Detect it from the raw
+    // input blob via a real cookie-pair match — NOT extractCookieValue, which
+    // returns the whole bare value for any name when the input has no ";" (#5350).
+    const suppliedCfClearance = /(?:^|;\s*)cf_clearance=[^;\s]+/.test(String(apiKey || ""));
+
     if (response.status === 401) {
+      // With a cf_clearance supplied, a 401 is almost always an IP-reputation block
+      // (the clearance can't be replayed from a different machine), not a bad cookie.
+      if (suppliedCfClearance) {
+        return { valid: false, error: `Grok returned 401. ${GROK_IP_REPUTATION_GUIDANCE}` };
+      }
       return {
         valid: false,
         error: "Invalid SSO cookie — re-paste from grok.com DevTools → Cookies → sso",
@@ -304,8 +325,14 @@ export async function validateGrokWebProvider({ apiKey, providerSpecificData = {
       // messaging — a misleading "invalid cookie" verdict on an IP-reputation
       // block (issue #3474) sends users chasing a cookie that is actually fine.
       //
-      // 1. Auth-shaped → the cookie/session is the problem; re-paste it.
+      // 1. Auth-shaped → the cookie/session is the problem; re-paste it. But when a
+      //    cf_clearance was supplied, this is almost always an IP-reputation block the
+      //    edge surfaced as an auth failure — the clearance can't be replayed from a
+      //    different machine, so re-pasting the cookie will not help (#5350).
       if (/invalid-credentials|unauthenticated|unauthorized/i.test(errorDetail)) {
+        if (suppliedCfClearance) {
+          return { valid: false, error: `Grok returned 403. ${GROK_IP_REPUTATION_GUIDANCE}` };
+        }
         return {
           valid: false,
           error: "Invalid SSO cookie — re-paste from grok.com DevTools → Cookies → sso",
@@ -319,10 +346,7 @@ export async function validateGrokWebProvider({ apiKey, providerSpecificData = {
       if (isCloudflareChallenge(errorDetail) || isGrokAntiBotBlock(errorDetail)) {
         return {
           valid: false,
-          error:
-            "Grok returned 403 (anti-bot/Cloudflare block). Your sso cookie is likely fine — " +
-            "this is an IP-reputation block on the request, not an auth failure. Retry from a " +
-            "residential IP or configure a proxy for grok-web.",
+          error: `Grok returned 403 (anti-bot/Cloudflare block). ${GROK_IP_REPUTATION_GUIDANCE}`,
         };
       }
       // 3. Structured upstream error (e.g. probe model renamed) → surface the body

src/lib/providers/webCookieAuth.ts

@@ -49,6 +49,12 @@ export function extractCookieValue(rawValue: string, cookieName: string): string
  * only appended when it appears as a real cookie pair in the input, so a bare
  * `sso` value (no `;`/`=`) is never mistaken for an `sso-rw` value.
  *
+ * The Cloudflare cookies `cf_clearance` and `__cf_bm` are forwarded the same
+ * way when present (#5350) — Cloudflare on grok.com expects the same clearance
+ * the browser earned, and AIClient2API forwards them too. Like `sso-rw`, each is
+ * appended only when it appears as a real cookie pair, so a bare `sso` blob
+ * still produces exactly `sso=<value>` (no phantom cf keys).
+ *
  * Returns "" when no `sso` value can be extracted.
  */
 export function buildGrokCookieHeader(rawValue: string): string {
@@ -56,9 +62,11 @@ export function buildGrokCookieHeader(rawValue: string): string {
   if (!sso) return "";
 
   const parts = [`sso=${sso}`];
-  if (/(?:^|;\s*)sso-rw=/.test(rawValue)) {
-    const ssoRw = extractCookieValue(rawValue, "sso-rw");
-    if (ssoRw) parts.push(`sso-rw=${ssoRw}`);
+  for (const name of ["sso-rw", "cf_clearance", "__cf_bm"]) {
+    if (new RegExp("(?:^|;\\s*)" + name + "=").test(rawValue)) {
+      const value = extractCookieValue(rawValue, name);
+      if (value) parts.push(`${name}=${value}`);
+    }
   }
   return parts.join("; ");
 }

tests/unit/provider-validation-specialty.test.ts

@@ -447,7 +447,10 @@ test("grok-web validator: full DevTools cookie blob is parsed for the sso value"
   const result = await validateProviderApiKey({ provider: "grok-web", apiKey: blob });
 
   assert.equal(result.valid, true);
-  assert.equal(capturedCookie, "sso=eyJTARGET.abc.def");
+  // #5350 — the outbound cookie now forwards the Cloudflare cookies too.
+  assert.match(capturedCookie, /(?:^|;\s*)sso=eyJTARGET\.abc\.def(?:;|$)/);
+  assert.match(capturedCookie, /(?:^|;\s*)cf_clearance=baz(?:;|$)/);
+  assert.match(capturedCookie, /(?:^|;\s*)__cf_bm=bar(?:;|$)/);
 });
 
 test("grok-web validator: empty/missing sso in input returns 'Missing sso cookie'", async () => {
@@ -594,6 +597,45 @@ test("grok-web validator: 403 with credential-rejection body is treated as auth-
   assert.match(result.error || "", /Invalid SSO cookie/i);
 });
 
+// #5350 — when the user DID supply a cf_clearance, an auth-shaped 401 / invalid-credentials 403
+// is almost always an IP-reputation block (cf_clearance is IP+TLS+UA-pinned and cannot be
+// replayed from a different machine), NOT a bad cookie. Surface the IP guidance instead of the
+// misleading "Invalid SSO cookie" verdict.
+test("grok-web validator: 401 WITH a cf_clearance maps to IP-reputation guidance, not 'invalid cookie' (#5350)", async () => {
+  __setGrokTlsFetchOverride(async () => {
+    return { status: 401, headers: new Headers(), text: "Unauthorized", body: null };
+  });
+
+  const blob = "sso=eyJTARGET.abc.def; sso-rw=RW; cf_clearance=CF; __cf_bm=BM";
+  const result = await validateProviderApiKey({ provider: "grok-web", apiKey: blob });
+  assert.equal(result.valid, false);
+  assert.match(result.error || "", /residential IP|proxy/i);
+  assert.doesNotMatch(result.error || "", /invalid SSO cookie/i);
+});
+
+test("grok-web validator: invalid-credentials 403 WITH a cf_clearance maps to IP-reputation guidance (#5350)", async () => {
+  __setGrokTlsFetchOverride(async () => {
+    return {
+      status: 403,
+      headers: new Headers(),
+      text: JSON.stringify({
+        error: {
+          code: 16,
+          message: "Failed to look up session ID. [WKE=unauthenticated:invalid-credentials]",
+          details: [],
+        },
+      }),
+      body: null,
+    };
+  });
+
+  const blob = "sso=eyJTARGET.abc.def; cf_clearance=CF";
+  const result = await validateProviderApiKey({ provider: "grok-web", apiKey: blob });
+  assert.equal(result.valid, false);
+  assert.match(result.error || "", /residential IP|proxy/i);
+  assert.doesNotMatch(result.error || "", /invalid SSO cookie/i);
+});
+
 test("grok-web validator: TLS client unavailable surfaces actionable error", async () => {
   __setGrokTlsFetchOverride(async () => {
     const { TlsClientUnavailableError } = await import("../../open-sse/services/grokTlsClient.ts");

tests/unit/web-cookie-auth.test.ts

@@ -71,9 +71,30 @@ test("buildGrokCookieHeader: single sso= pair emits only sso", () => {
   assert.equal(buildGrokCookieHeader("sso=eyJ0eXAi.abc"), "sso=eyJ0eXAi.abc");
 });
 
-test("buildGrokCookieHeader: full cookie blob forwards both sso and sso-rw", () => {
+test("buildGrokCookieHeader: full cookie blob forwards sso, sso-rw and cf_clearance", () => {
   const blob = "cf_clearance=zzz; sso=AAA.bbb; sso-rw=CCC.ddd; other=1";
-  assert.equal(buildGrokCookieHeader(blob), "sso=AAA.bbb; sso-rw=CCC.ddd");
+  assert.equal(buildGrokCookieHeader(blob), "sso=AAA.bbb; sso-rw=CCC.ddd; cf_clearance=zzz");
+});
+
+// #5350 — forward the Cloudflare cookies (cf_clearance + __cf_bm) when the pasted
+// blob carries them, matching the real browser request (parity with AIClient2API).
+test("buildGrokCookieHeader: forwards sso, sso-rw, cf_clearance and __cf_bm (order-independent)", () => {
+  const blob = "i18nextLng=en; __cf_bm=BM; sso=SSO; sso-rw=RW; cf_clearance=CF; x-userid=U";
+  const header = buildGrokCookieHeader(blob);
+  assert.match(header, /(?:^|;\s*)sso=SSO(?:;|$)/);
+  assert.match(header, /(?:^|;\s*)sso-rw=RW(?:;|$)/);
+  assert.match(header, /(?:^|;\s*)cf_clearance=CF(?:;|$)/);
+  assert.match(header, /(?:^|;\s*)__cf_bm=BM(?:;|$)/);
+});
+
+test("buildGrokCookieHeader: bare sso emits no phantom cf_clearance/__cf_bm keys", () => {
+  assert.equal(buildGrokCookieHeader("sso=SSO"), "sso=SSO");
+  assert.doesNotMatch(buildGrokCookieHeader("sso=SSO"), /cf_clearance|__cf_bm/);
+});
+
+test("buildGrokCookieHeader: forwards __cf_bm even when cf_clearance is absent", () => {
+  const blob = "foo=1; __cf_bm=BM; sso=AAA.bbb";
+  assert.equal(buildGrokCookieHeader(blob), "sso=AAA.bbb; __cf_bm=BM");
 });
 
 test("buildGrokCookieHeader: order-independent — sso-rw before sso in the blob", () => {