diff --git a/changelog.d/features/claude-low-priority-mode.md b/changelog.d/features/claude-low-priority-mode.md new file mode 100644 index 0000000000..aa3aa2c414 --- /dev/null +++ b/changelog.d/features/claude-low-priority-mode.md @@ -0,0 +1 @@ +- **feat(sse):** Claude OAuth connections can opt in (per account, Edit connection → Claude section) to Claude Code's lower-priority lane and once-a-week session-limit reset. After the first 5-hour usage-wall 429 carrying `anthropic-ratelimit-unified-slow-offer: treatment`, OmniRoute retries the same account with `anthropic-usage-limit: slow` and keeps sending it until the window resets — the account keeps serving past the limit instead of being cooled down (slot_busy/529 wait the server's `slow-retry-after`, bounded by `slow-max-wait`). With auto-reset on, the wall first tries `POST /api/organizations/{org}/reset_rate_limits` (`juniper_tide`) and retries at full speed when the server grants it. Both default off; nothing is sent before the limit is hit. diff --git a/docs/architecture/RESILIENCE_GUIDE.md b/docs/architecture/RESILIENCE_GUIDE.md index df5896e1f0..eff68abb15 100644 --- a/docs/architecture/RESILIENCE_GUIDE.md +++ b/docs/architecture/RESILIENCE_GUIDE.md @@ -108,6 +108,66 @@ These persist until credentials change or an operator resets them. Do not overwr **Lazy recovery:** when `rateLimitedUntil` is past, connection becomes eligible again. On successful use, `clearAccountError()` clears all error fields. +### Claude OAuth usage wall: lower-priority lane + session-limit reset + +**Scope:** one Claude subscription (OAuth) connection. Both features are **opt-in per +connection** (Edit connection → Claude section → `lowPriorityMode` / `autoLimitReset` in +`providerSpecificData`, both default off) and mirror Claude Code's `/low-priority` and +`/limit-reset` commands (wire contract captured from Claude Code 2.1.263). + +**Implementation:** + +- State machine + response classification: `open-sse/services/claudeLowPriority.ts` +- Reset status/claim client: `open-sse/services/claudeLimitReset.ts` +- Executor hook (header injection + same-account retry): `open-sse/executors/base.ts::execute()` +- Opt-in persistence: `src/lib/providers/requestDefaults.ts::normalizeProviderSpecificData()` + +**Trigger:** the 5-hour usage wall — a `429` whose headers carry +`anthropic-ratelimit-unified-status: rejected` and, when the account is eligible, +`anthropic-ratelimit-unified-slow-offer: treatment`. Nothing is sent before that first wall +429; a burst 429 without unified headers goes through the normal cooldown path. + +**Lower-priority lane** (`lowPriorityMode`): + +- On the wall 429 the executor accepts the offer and immediately retries the **same** + account with `anthropic-usage-limit: slow`; the lane stays active until the announced + `anthropic-ratelimit-unified-reset` (+60s grace) and every request in that window carries + the header. The intercepted 429 never reaches `handleChatCore`, so the connection is + **not** put in cooldown and is not rotated away. +- `anthropic-ratelimit-unified-slow-status` on later responses: `active` / `not_needed` + keep the lane; `slot_busy` (429) or a `529` wait the server's + `anthropic-ratelimit-unified-slow-retry-after` (default 20s, clamp 5–600s, ±30% jitter) + and retry, bounded by `anthropic-ratelimit-unified-slow-max-wait` (default 20 min, clamp + 1 min–6 h) — past that the lane ends and a 10-minute cool-off blocks re-acceptance. The + wait is additionally capped by what is left of the request's own upstream-start timeout + (`resolveFetchStartTimeout`, 10 min by default) minus a 5 s margin: without that cap the + 20-minute default max-wait would outlive the request and the sleep would be aborted + mid-wait, surfacing a `TimeoutError` instead of the graceful `max_wait` end + cool-off. +- `weekly_limit` / `budget_exhausted` / `off` / `ineligible`, a 5h-window rollover, or + `ineligible` + `anthropic-ratelimit-unified-overage-in-use: true` (which ends it as + `extra_usage` on any status, since paid overage now covers the wall) end the lane; the + response then flows to the normal cooldown path. `budget_exhausted` is remembered until + the announced budget reset (≤ 8 days). +- The wall check runs after the executor's own 400-driven intra-attempt retries (context + editing, thinking/effort clamps, param auto-learn), so a wall 429 that only surfaces on + one of those retries is still intercepted instead of reaching the cooldown path. +- State is in-memory per connection (a restart costs one extra wall 429 to re-accept). + +**Session-limit reset** (`autoLimitReset`, tried before the lane when both are on): + +- `GET https://api.anthropic.com/api/oauth/usage?at_wall=1&skip_spend=1` → `juniper_tide` + block; when `arm: "reset"` and `available: true`, + `POST https://api.anthropic.com/api/organizations/{orgUUID}/reset_rate_limits` with + `{ "program": "juniper_tide" }` (organization UUID from + `providerSpecificData.organizationUUID`, bootstrap fallback). +- `result: reset|not_limited` → the request is retried at full speed (no slow header). + `already_used` / `not_offered` memoise `next_available_at` (default one week); any + failure backs off 15 minutes. The reset is once a week and still counts toward the + weekly limit. + +Regression guards: `tests/unit/claude-low-priority-mode.test.ts`, +`tests/unit/claude-limit-reset.test.ts`, `tests/unit/claude-low-priority-executor.test.ts`. + ### Session affinity (#7274) **Scope:** one client session (`X-Session-Id` / `x-codex-session-id` / `x-omniroute-session` header) pinned to one connection, for **any** provider. diff --git a/open-sse/config/anthropicHeaders.ts b/open-sse/config/anthropicHeaders.ts index afbfaceead..32fd4b61cf 100644 --- a/open-sse/config/anthropicHeaders.ts +++ b/open-sse/config/anthropicHeaders.ts @@ -186,3 +186,37 @@ export const CLAUDE_CLI_USER_AGENT = getClaudeCodeUserAgent("cli"); export { getClaudeCodeUserAgent }; export const CLAUDE_CLI_STAINLESS_PACKAGE_VERSION = CLAUDE_CODE_SDK_PACKAGE_VERSION; export const CLAUDE_CLI_STAINLESS_RUNTIME_VERSION = CLAUDE_CODE_RUNTIME_VERSION; + +/** + * Merge a Claude-Code-shaped header set over the outbound headers, dropping any + * case variant of the same name first — undici would otherwise concatenate the two + * into a single rejected value (issue #1454). + */ +export function mergeCcHeaders( + headers: Record, + ccHeaders: Record +): void { + const ccKeysLower = new Set(Object.keys(ccHeaders).map((k) => k.toLowerCase())); + for (const key of Object.keys(headers)) { + if (ccKeysLower.has(key.toLowerCase())) delete headers[key]; + } + Object.assign(headers, ccHeaders); +} + +/** + * Stainless SDK metadata for the Claude wire image. OS/arch follow the host running + * the signed binary; the runtime version is pinned to the captured CLI, not OmniRoute's + * Node. Mutates `headers`. + */ +export function applyStainlessHeaders( + headers: Record, + parts: { arch: string; os: string } +): void { + headers["X-Stainless-Arch"] = parts.arch; + headers["X-Stainless-Lang"] = "js"; + headers["X-Stainless-OS"] = parts.os; + headers["X-Stainless-Runtime"] = "node"; + headers["X-Stainless-Runtime-Version"] = CLAUDE_CLI_STAINLESS_RUNTIME_VERSION; + headers["X-Stainless-Retry-Count"] = "0"; + delete headers["X-Stainless-Os"]; +} diff --git a/open-sse/executors/base.ts b/open-sse/executors/base.ts index dc65898295..7f07fa56a3 100644 --- a/open-sse/executors/base.ts +++ b/open-sse/executors/base.ts @@ -6,8 +6,9 @@ import { type AlternateFormat, } from "../config/providers/alternateFormats.ts"; import { - CLAUDE_CLI_STAINLESS_RUNTIME_VERSION, + applyStainlessHeaders, getClaudeCliBillingVersion, + mergeCcHeaders, mergeClientAnthropicBeta, normalizeAnthropicHeaderVariants, } from "../config/anthropicHeaders.ts"; @@ -41,6 +42,7 @@ import { isFreeVariantModel, } from "../services/openrouterFreeWindow.ts"; import { gateOutboundRequest } from "../services/wafRateLimit.ts"; +import { ClaudeUsageLimitGuard } from "./claudeUsageLimit.ts"; import type { PoolConfig } from "../services/sessionPool/types.ts"; import type { Session } from "../services/sessionPool/session.ts"; import { SessionPool } from "../services/sessionPool/sessionPool.ts"; @@ -98,6 +100,7 @@ import { selectBetaFlags, stainlessArch, stainlessOS, + stripClaudeSystemPrefixBlocks, stripProxyToolPrefix, } from "./claudeIdentity.ts"; import { withForcedResponsesUpstream } from "./forceResponsesUpstream.ts"; @@ -708,6 +711,8 @@ export class BaseExecutor { let activeCredentials = credentials; // Track per-URL intra-retry attempts to avoid infinite loops const retryAttemptsByUrl: Record = {}; + // Claude OAuth usage wall (opt-in per connection): see ./claudeUsageLimit.ts. + const claudeUsageLimit = new ClaudeUsageLimitGuard(this.provider, log); // Probe-origin dispatches must not consume a refresh-token rotation — // routing state untouched; the reactive 401/403 path is probe-guarded @@ -1154,18 +1159,7 @@ export class BaseExecutor { // Strip any pre-existing billing/sentinel before re-prepending — keeps // retries idempotent and avoids stacking that breaks prompt-cache prefix // matching (see issue #1712). - for (let i = sysBlocks.length - 1; i >= 0; i--) { - const t = sysBlocks[i]?.text; - if (typeof t === "string" && t.startsWith("x-anthropic-billing-header:")) { - sysBlocks.splice(i, 1); - } - } - for (let i = sysBlocks.length - 1; i >= 0; i--) { - const t = sysBlocks[i]?.text; - if (typeof t === "string" && t.startsWith(SENTINEL)) { - sysBlocks.splice(i, 1); - } - } + stripClaudeSystemPrefixBlocks(sysBlocks, SENTINEL); sysBlocks.unshift({ type: "text", text: billingLine }, { type: "text", text: SENTINEL }); tb.system = sysBlocks; normalizeCacheControlTtl(tb); @@ -1247,29 +1241,14 @@ export class BaseExecutor { "X-Claude-Code-Session-Id": sessionId, }; - // Drop case variants of the same header name before merging — undici - // would otherwise concatenate them (issue #1454). - const ccKeysLower = new Set(Object.keys(ccHeaders).map((k) => k.toLowerCase())); - for (const key of Object.keys(headers)) { - if (ccKeysLower.has(key.toLowerCase())) delete headers[key]; - } - Object.assign(headers, ccHeaders); + mergeCcHeaders(headers, ccHeaders); if (usesCcWireImage(this.provider) && usesClaudeCodeProtocol) { delete headers["Authorization"]; headers["x-api-key"] = activeCredentials?.apiKey || activeCredentials?.accessToken || ""; } delete headers["X-Stainless-Helper-Method"]; - - // OS/arch follow the host running the signed binary. Runtime version - // is pinned to the captured CLI wire image, not OmniRoute's Node. - headers["X-Stainless-Arch"] = stainlessArch(); - headers["X-Stainless-Lang"] = "js"; - headers["X-Stainless-OS"] = stainlessOS(); - headers["X-Stainless-Runtime"] = "node"; - headers["X-Stainless-Runtime-Version"] = CLAUDE_CLI_STAINLESS_RUNTIME_VERSION; - headers["X-Stainless-Retry-Count"] = "0"; - delete headers["X-Stainless-Os"]; + applyStainlessHeaders(headers, { arch: stainlessArch(), os: stainlessOS() }); } // selectBetaFlags() above always includes redact-thinking for an // "opaque" client (no client-negotiated anthropic-beta) — correct @@ -1401,6 +1380,8 @@ export class BaseExecutor { // Enforce peer tracing after all configurable headers have been merged so // operator/provider metadata cannot accidentally erase the loop guard. applyPeerTraceHeader(finalHeaders, clientHeaders, url); + // Rides `anthropic-usage-limit: slow` once this account accepted the offer. + const claudeSentSlow = claudeUsageLimit.applyHeader(finalHeaders, activeCredentials); const serializedBody = prl.parseBody(bodyString); // #4307 — Preserve the non-enumerable tool-name cloak/remap reverse map // (`_toolNameMap`, set on the live `transformedBody` by @@ -1652,6 +1633,21 @@ export class BaseExecutor { } } + // Claude OAuth usage wall: accept the slow-lane offer / claim the weekly + // session-limit reset and retry the SAME account instead of surfacing the 429 + // (which would cool the connection down). Runs AFTER every 400-driven retry + // above so it classifies the FINAL response of this attempt. + const claudeRetry = await claudeUsageLimit.shouldRetry(response, url, { + credentials: activeCredentials, + signal, + budgetMs: fetchStartTimeoutMs, + sentSlow: claudeSentSlow, + }); + if (claudeRetry) { + urlIndex--; // re-run this urlIndex (header injection sees the new lane state) + continue; + } + // Intra-URL retry: agentrouter.org WAF returns 400 content-blocked // intermittently (burst-sensitive, recovers after cooldown). Retry the // same URL with exponential backoff before falling through to the diff --git a/open-sse/executors/claudeIdentity.ts b/open-sse/executors/claudeIdentity.ts index 2190158782..8ccb325267 100644 --- a/open-sse/executors/claudeIdentity.ts +++ b/open-sse/executors/claudeIdentity.ts @@ -469,3 +469,21 @@ export function stripProxyToolPrefix(body: Record): void { } } } + +/** + * Drop any previously injected billing header / Claude Code sentinel block from a + * `system` array, so re-prepending them on a retry stays idempotent instead of stacking + * (issue #1712 — stacking breaks prompt-cache prefix matching). Mutates `sysBlocks`. + */ +export function stripClaudeSystemPrefixBlocks( + sysBlocks: Array>, + sentinel: string +): void { + for (let i = sysBlocks.length - 1; i >= 0; i--) { + const text = sysBlocks[i]?.text; + if (typeof text !== "string") continue; + if (text.startsWith("x-anthropic-billing-header:") || text.startsWith(sentinel)) { + sysBlocks.splice(i, 1); + } + } +} diff --git a/open-sse/executors/claudeUsageLimit.ts b/open-sse/executors/claudeUsageLimit.ts new file mode 100644 index 0000000000..b861c812a0 --- /dev/null +++ b/open-sse/executors/claudeUsageLimit.ts @@ -0,0 +1,146 @@ +/** + * claudeUsageLimit.ts — executor-side glue for the Claude OAuth usage wall. + * + * Keeps `BaseExecutor.execute()` free of the lower-priority lane's mechanics: this guard + * owns the per-request wait accounting, the header injection, the abort-aware sleep and the + * decision logging. The decision itself is the pure state machine in + * `open-sse/services/claudeLowPriority.ts`; the weekly session-limit reset lives in + * `open-sse/services/claudeLimitReset.ts`. + * + * Both behaviors are opt-in per connection (`providerSpecificData.lowPriorityMode` / + * `autoLimitReset`) and only ever act on a real 5-hour usage wall — see the module header of + * claudeLowPriority.ts for the wire contract and the lifecycle. + */ + +import { attemptClaudeLimitReset } from "../services/claudeLimitReset.ts"; +import { + CLAUDE_USAGE_LIMIT_HEADER, + CLAUDE_USAGE_LIMIT_SLOW, + createClaudeLowPriorityWait, + handleClaudeUsageLimitResponse, + isClaudeLowPriorityActive, + readClaudeUsageLimitConfig, + resolveClaudeUsageLimitKey, + type ClaudeLowPriorityWait, +} from "../services/claudeLowPriority.ts"; +import type { ExecutorLog, ProviderCredentials } from "./base.ts"; + +/** Safety margin kept between the last lane wait and the request's own upstream timeout. */ +export const CLAUDE_USAGE_LIMIT_WAIT_MARGIN_MS = 5_000; + +type GuardResponse = { status: number; headers: Headers }; + +export type ClaudeUsageLimitRetryInput = { + credentials?: ProviderCredentials | null; + signal?: AbortSignal | null; + /** The request's upstream-start timeout; 0/undefined means "unbounded". */ + budgetMs?: number; + /** Whether THIS request went out carrying the slow header (see `sentSlow` in the service). */ + sentSlow: boolean; +}; + +/** True for a native Claude connection authenticated with a subscription OAuth token. */ +function isClaudeOAuth(provider: string, credentials?: ProviderCredentials | null): boolean { + return ( + provider === "claude" && + typeof credentials?.accessToken === "string" && + credentials.accessToken.startsWith("sk-ant-oat") && + !credentials?.apiKey + ); +} + +/** Sleep that rejects as soon as the request is aborted, so a lane wait never outlives it. */ +function abortableSleep(delayMs: number, signal?: AbortSignal | null): Promise { + return new Promise((resolve, reject) => { + const timer = setTimeout(() => { + signal?.removeEventListener("abort", onAbort); + resolve(); + }, delayMs); + const onAbort = () => { + clearTimeout(timer); + reject(signal?.reason ?? new DOMException("Aborted", "AbortError")); + }; + if (signal?.aborted) return onAbort(); + signal?.addEventListener("abort", onAbort, { once: true }); + }); +} + +/** One instance per `execute()` call: the wait window spans the intra-URL retries. */ +export class ClaudeUsageLimitGuard { + private readonly wait: ClaudeLowPriorityWait = createClaudeLowPriorityWait(); + private readonly startedAtMs = Date.now(); + private key: string | null = null; + + constructor( + private readonly provider: string, + private readonly log?: ExecutorLog | null + ) {} + + /** + * Stamp `anthropic-usage-limit: slow` when this connection's lane is active. Returns + * whether the header went out, which the response side needs to tell a lane verdict from + * a header-less sibling's outcome. + */ + applyHeader(headers: Record, credentials?: ProviderCredentials | null): boolean { + this.key = isClaudeOAuth(this.provider, credentials) + ? resolveClaudeUsageLimitKey(credentials ?? {}) + : null; + if (this.key === null || !isClaudeLowPriorityActive(this.key)) return false; + headers[CLAUDE_USAGE_LIMIT_HEADER] = CLAUDE_USAGE_LIMIT_SLOW; + return true; + } + + /** + * Classify an upstream response. Resolves true when the caller must retry the SAME + * account (the sleep, if any, has already happened) instead of surfacing the response. + */ + async shouldRetry( + response: GuardResponse, + url: string, + input: ClaudeUsageLimitRetryInput + ): Promise { + if (this.key === null) return false; + const key = this.key; + const credentials = input.credentials; + const decision = await handleClaudeUsageLimitResponse({ + key, + config: readClaudeUsageLimitConfig(credentials?.providerSpecificData), + response, + wait: this.wait, + sentSlow: input.sentSlow, + waitCeilingMs: this.waitCeilingMs(input.budgetMs), + claimLimitReset: () => + attemptClaudeLimitReset({ + key, + accessToken: credentials?.accessToken ?? "", + providerSpecificData: credentials?.providerSpecificData, + log: this.log, + }).then((attempt) => attempt.reset), + }); + + if (decision.kind === "ended") { + this.log?.info?.("CLAUDE_LOW_PRIORITY", `lane ended (${decision.reason}) on ${url}`); + return false; + } + if (decision.kind !== "retry") return false; + + this.log?.info?.( + "CLAUDE_LOW_PRIORITY", + `${decision.via} on ${url} — retrying same account in ${decision.delayMs}ms` + ); + if (decision.delayMs > 0) await abortableSleep(decision.delayMs, input.signal); + return true; + } + + /** + * What is left of the request's upstream timeout, minus a safety margin. Without this the + * server-announced max-wait (20 min by default, up to 6 h) outlives the request and the + * sleep is aborted mid-wait, surfacing a TimeoutError instead of the graceful `max_wait` + * end plus its cool-off. + */ + private waitCeilingMs(budgetMs?: number): number | undefined { + if (!budgetMs || budgetMs <= 0) return undefined; + const elapsed = Date.now() - this.startedAtMs; + return Math.max(0, budgetMs - elapsed - CLAUDE_USAGE_LIMIT_WAIT_MARGIN_MS); + } +} diff --git a/open-sse/services/claudeLimitReset.ts b/open-sse/services/claudeLimitReset.ts new file mode 100644 index 0000000000..de323dfc9f --- /dev/null +++ b/open-sse/services/claudeLimitReset.ts @@ -0,0 +1,374 @@ +/** + * claudeLimitReset.ts — Claude OAuth once-a-week session-limit reset. + * + * OmniRoute counterpart of Claude Code's hidden `/limit-reset` command (wire contract + * captured from Claude Code 2.1.263, program name `juniper_tide`). At the 5-hour usage + * wall a subscription account may be allowed to reset its session window once per week; + * the reset still counts toward the weekly limit. + * + * Wire contract: + * Status GET https://api.anthropic.com/api/oauth/usage?at_wall=1&skip_spend=1 + * → body.juniper_tide = { eligible, ineligible_reason, in_experiment, + * arm: "control"|"reset", available, next_available_at, weekly_resets_at, + * resets_per_week } + * Claim POST https://api.anthropic.com/api/organizations/{orgUUID}/reset_rate_limits + * body { program: "juniper_tide" } + * → { result: "reset"|"already_used"|"not_limited"|"ineligible"|"unavailable", + * next_available_at, weekly_resets_at } + * + * Both calls use the OAuth bearer token (scope `user:profile` for the status read — the + * scope OmniRoute already requests). The organization UUID comes from the connection's + * `providerSpecificData.organizationUUID` (persisted at OAuth provisioning) with the + * `/api/claude_cli/bootstrap` lookup as fallback. + * + * A per-connection memo (in-memory) remembers "not before" so a wall that cannot be reset + * (already used this week, not in the experiment, …) does not re-query on every request. + */ + +import { fetchClaudeBootstrap, getClaudeCodeVersion } from "../executors/claudeIdentity.ts"; +import { setBoundedEntry } from "./claudeLowPriority.ts"; + +type JsonRecord = Record; +type FetchLike = typeof fetch; + +export const CLAUDE_LIMIT_RESET_PROGRAM = "juniper_tide"; +export const CLAUDE_LIMIT_RESET_STATUS_URL = + "https://api.anthropic.com/api/oauth/usage?at_wall=1&skip_spend=1"; +export const CLAUDE_LIMIT_RESET_STATUS_TIMEOUT_MS = 5_000; +export const CLAUDE_LIMIT_RESET_CLAIM_TIMEOUT_MS = 25_000; +/** Back-off after a failed/unavailable claim before the next wall may re-query. */ +export const CLAUDE_LIMIT_RESET_RETRY_BACKOFF_MS = 15 * 60_000; +/** Fallback "next available" horizon when the server does not announce one. */ +export const CLAUDE_LIMIT_RESET_WEEK_MS = 7 * 86_400_000; + +export function claudeLimitResetClaimUrl(organizationUuid: string): string { + return `https://api.anthropic.com/api/organizations/${encodeURIComponent(organizationUuid)}/reset_rate_limits`; +} + +export type ClaudeLimitResetArm = "control" | "reset"; + +export type ClaudeLimitResetStatus = { + eligible: boolean; + ineligibleReason: string | null; + inExperiment: boolean; + arm: ClaudeLimitResetArm | null; + available: boolean; + nextAvailableAt: string | null; + weeklyResetsAt: string | null; + resetsPerWeek: number; +}; + +export type ClaudeLimitResetResult = + | "reset" + | "already_used" + | "not_limited" + | "ineligible" + | "unavailable" + | "rate_limited" + | "auth_error" + | "error"; + +export type ClaudeLimitResetClaim = { + result: ClaudeLimitResetResult; + nextAvailableAt: string | null; + weeklyResetsAt: string | null; +}; + +function asRecord(value: unknown): JsonRecord { + return value && typeof value === "object" && !Array.isArray(value) ? (value as JsonRecord) : {}; +} + +function stringOrNull(value: unknown): string | null { + return typeof value === "string" && value.trim() !== "" ? value : null; +} + +function oauthHeaders(accessToken: string): Record { + // Same shape as the existing /api/oauth/usage poller (usage/claude.ts): axios-style + // `claude-code/` UA, not the Stainless `claude-cli/…` one. + return { + Accept: "application/json, text/plain, */*", + Authorization: `Bearer ${accessToken}`, + "Content-Type": "application/json", + "User-Agent": `claude-code/${getClaudeCodeVersion()}`, + "anthropic-beta": "oauth-2025-04-20", + }; +} + +/** Parse the `juniper_tide` block of a `/api/oauth/usage?at_wall=1` body. Null when absent/malformed. */ +export function parseClaudeLimitResetStatus(usageBody: unknown): ClaudeLimitResetStatus | null { + const block = asRecord(usageBody).juniper_tide; + if (block === undefined || block === null) return null; + const r = asRecord(block); + if (typeof r.eligible !== "boolean") return null; + const arm = r.arm === "control" || r.arm === "reset" ? r.arm : null; + const resetsPerWeek = + typeof r.resets_per_week === "number" && Number.isFinite(r.resets_per_week) + ? r.resets_per_week + : 1; + return { + eligible: r.eligible, + ineligibleReason: stringOrNull(r.ineligible_reason), + inExperiment: r.in_experiment === true, + arm, + available: r.available === true, + nextAvailableAt: stringOrNull(r.next_available_at), + weeklyResetsAt: stringOrNull(r.weekly_resets_at), + resetsPerWeek, + }; +} + +/** Parse the `reset_rate_limits` response body. Unknown/malformed → `unavailable`. */ +export function parseClaudeLimitResetClaim(body: unknown): ClaudeLimitResetClaim { + const r = asRecord(body); + const raw = r.result; + const result: ClaudeLimitResetResult = + raw === "reset" || + raw === "already_used" || + raw === "not_limited" || + raw === "ineligible" || + raw === "unavailable" + ? raw + : "unavailable"; + return { + result, + nextAvailableAt: stringOrNull(r.next_available_at), + weeklyResetsAt: stringOrNull(r.weekly_resets_at), + }; +} + +async function fetchWithTimeout( + fetchImpl: FetchLike, + url: string, + init: RequestInit, + timeoutMs: number +): Promise { + const ctrl = new AbortController(); + const timer = setTimeout(() => ctrl.abort(), timeoutMs); + try { + return await fetchImpl(url, { ...init, signal: ctrl.signal }); + } finally { + clearTimeout(timer); + } +} + +/** GET the at-wall usage snapshot and extract the reset offer. Null on any failure. */ +export async function fetchClaudeLimitResetStatus( + accessToken: string, + fetchImpl: FetchLike = fetch +): Promise { + try { + const res = await fetchWithTimeout( + fetchImpl, + CLAUDE_LIMIT_RESET_STATUS_URL, + { method: "GET", headers: oauthHeaders(accessToken) }, + CLAUDE_LIMIT_RESET_STATUS_TIMEOUT_MS + ); + if (!res.ok) return null; + const body: unknown = await res.json().catch(() => null); + return parseClaudeLimitResetStatus(body); + } catch { + return null; + } +} + +/** POST the reset claim for one organization. Never throws. */ +export async function claimClaudeLimitReset( + accessToken: string, + organizationUuid: string, + fetchImpl: FetchLike = fetch +): Promise { + try { + const res = await fetchWithTimeout( + fetchImpl, + claudeLimitResetClaimUrl(organizationUuid), + { + method: "POST", + headers: oauthHeaders(accessToken), + body: JSON.stringify({ program: CLAUDE_LIMIT_RESET_PROGRAM }), + }, + CLAUDE_LIMIT_RESET_CLAIM_TIMEOUT_MS + ); + if (res.status === 429) + return { result: "rate_limited", nextAvailableAt: null, weeklyResetsAt: null }; + if (res.status === 401 || res.status === 403) { + return { result: "auth_error", nextAvailableAt: null, weeklyResetsAt: null }; + } + if (!res.ok) return { result: "error", nextAvailableAt: null, weeklyResetsAt: null }; + const body: unknown = await res.json().catch(() => null); + return parseClaudeLimitResetClaim(body); + } catch { + return { result: "error", nextAvailableAt: null, weeklyResetsAt: null }; + } +} + +/** Organization UUID: persisted `providerSpecificData` first, bootstrap endpoint as fallback. */ +export async function resolveClaudeOrganizationUuid( + providerSpecificData: unknown, + accessToken: string +): Promise { + const psd = asRecord(providerSpecificData); + const persisted = stringOrNull(psd.organizationUUID) ?? stringOrNull(psd.organization_uuid); + if (persisted) return persisted; + const bootstrap = await fetchClaudeBootstrap(accessToken).catch(() => null); + return bootstrap?.organization_uuid ?? null; +} + +/** + * After a granted reset, sibling requests that hit the (now stale) wall in the same breath + * are told "reset: true" for this long instead of re-querying the server. + */ +export const CLAUDE_LIMIT_RESET_RECENT_WINDOW_MS = 60_000; + +/** FIFO cap for the per-connection memos — see CLAUDE_LOW_PRIORITY_CACHE_LIMIT for the why. */ +export const CLAUDE_LIMIT_RESET_CACHE_LIMIT = 10_000; + +const notBefore = new Map(); +const recentResetUntil = new Map(); +const inflight = new Map>(); + +function memoise(map: Map, key: string, value: number): void { + setBoundedEntry(map, key, value, CLAUDE_LIMIT_RESET_CACHE_LIMIT); +} + +function parseIsoMs(value: string | null): number | undefined { + if (!value) return undefined; + const ms = Date.parse(value); + return Number.isFinite(ms) ? ms : undefined; +} + +export type ClaudeLimitResetAttempt = { + reset: boolean; + outcome: + | "reset" + | "not_limited" + | "recent_reset" + | "memo_skip" + | "no_status" + | "not_offered" + | "no_organization" + | ClaudeLimitResetResult; + nextAvailableAt: string | null; +}; + +/** + * Auto-claim at the wall (opt-in per connection). Resolves `reset: true` only when the + * server confirmed the session window was reset (or reports the account is not limited), + * i.e. the caller may immediately retry at full speed. Every other outcome memoises a + * "not before" so the next wall does not re-query immediately. + */ +export async function attemptClaudeLimitReset(opts: { + key: string; + accessToken: string; + providerSpecificData?: unknown; + now?: number; + fetchImpl?: FetchLike; + log?: { + info?: (tag: string, msg: string) => void; + warn?: (tag: string, msg: string) => void; + } | null; +}): Promise { + const now = opts.now ?? Date.now(); + const recentUntil = recentResetUntil.get(opts.key); + if (recentUntil !== undefined && now < recentUntil) { + return { reset: true, outcome: "recent_reset", nextAvailableAt: null }; + } + const skipUntil = notBefore.get(opts.key); + if (skipUntil !== undefined && now < skipUntil) { + return { reset: false, outcome: "memo_skip", nextAvailableAt: null }; + } + // Parallel requests on one connection hit the wall together: run a single status+claim + // round trip and hand every caller the same verdict (no duplicate POST reset_rate_limits). + const pending = inflight.get(opts.key); + if (pending) return pending; + const run = runClaudeLimitResetAttempt(opts, now).finally(() => { + if (inflight.get(opts.key) === run) inflight.delete(opts.key); + }); + inflight.set(opts.key, run); + return run; +} + +/** Memoise "do not re-query before": the announced instant when it is in the future, else `fallbackMs` from now. */ +function memoiseNotBefore( + key: string, + announcedAt: string | null, + now: number, + fallbackMs: number +) { + const next = parseIsoMs(announcedAt); + memoise(notBefore, key, next !== undefined && next > now ? next : now + fallbackMs); +} + +/** The status half: is a reset actually on offer for this account right now? */ +async function resolveLimitResetOffer( + opts: Parameters[0], + now: number, + fetchImpl: FetchLike +): Promise { + const status = await fetchClaudeLimitResetStatus(opts.accessToken, fetchImpl); + if (!status) { + memoise(notBefore, opts.key, now + CLAUDE_LIMIT_RESET_RETRY_BACKOFF_MS); + return { reset: false, outcome: "no_status", nextAvailableAt: null }; + } + if (status.eligible && status.arm === "reset" && status.available) return null; + memoiseNotBefore(opts.key, status.nextAvailableAt, now, CLAUDE_LIMIT_RESET_RETRY_BACKOFF_MS); + opts.log?.info?.( + "CLAUDE_LIMIT_RESET", + `not offered (eligible=${status.eligible} arm=${status.arm ?? "-"} available=${status.available} reason=${status.ineligibleReason ?? "-"})` + ); + return { reset: false, outcome: "not_offered", nextAvailableAt: status.nextAvailableAt }; +} + +/** The claim half: POST the reset and memoise the outcome. */ +async function runLimitResetClaim( + opts: Parameters[0], + now: number, + fetchImpl: FetchLike, + organizationUuid: string +): Promise { + const claim = await claimClaudeLimitReset(opts.accessToken, organizationUuid, fetchImpl); + const granted = claim.result === "reset" || claim.result === "not_limited"; + const spent = granted || claim.result === "already_used"; + memoiseNotBefore( + opts.key, + claim.nextAvailableAt, + now, + spent ? CLAUDE_LIMIT_RESET_WEEK_MS : CLAUDE_LIMIT_RESET_RETRY_BACKOFF_MS + ); + if (granted) memoise(recentResetUntil, opts.key, now + CLAUDE_LIMIT_RESET_RECENT_WINDOW_MS); + opts.log?.info?.( + "CLAUDE_LIMIT_RESET", + granted + ? `${claim.result} — next reset available ${claim.nextAvailableAt ?? "in a week"}` + : `claim result=${claim.result}` + ); + return { reset: granted, outcome: claim.result, nextAvailableAt: claim.nextAvailableAt }; +} + +async function runClaudeLimitResetAttempt( + opts: Parameters[0], + now: number +): Promise { + const fetchImpl = opts.fetchImpl ?? fetch; + + const notOffered = await resolveLimitResetOffer(opts, now, fetchImpl); + if (notOffered) return notOffered; + + const orgUuid = await resolveClaudeOrganizationUuid(opts.providerSpecificData, opts.accessToken); + if (!orgUuid) { + memoise(notBefore, opts.key, now + CLAUDE_LIMIT_RESET_RETRY_BACKOFF_MS); + opts.log?.warn?.( + "CLAUDE_LIMIT_RESET", + "no organization UUID for this connection; cannot claim" + ); + return { reset: false, outcome: "no_organization", nextAvailableAt: null }; + } + + return runLimitResetClaim(opts, now, fetchImpl, orgUuid); +} + +/** Test-only: forget every memoised "not before" / recent reset / in-flight attempt. */ +export function _resetClaudeLimitResetMemo(): void { + notBefore.clear(); + recentResetUntil.clear(); + inflight.clear(); +} diff --git a/open-sse/services/claudeLowPriority.ts b/open-sse/services/claudeLowPriority.ts new file mode 100644 index 0000000000..6e07de2451 --- /dev/null +++ b/open-sse/services/claudeLowPriority.ts @@ -0,0 +1,650 @@ +/** + * claudeLowPriority.ts — Claude OAuth "lower-priority lane" after the 5-hour usage wall. + * + * OmniRoute counterpart of Claude Code's `/low-priority` command (wire contract captured + * from Claude Code 2.1.263). A Claude subscription account that hits its 5-hour usage + * limit gets a 429 carrying `anthropic-ratelimit-unified-slow-offer: treatment`. Accepting + * the offer means re-sending the request — and every following request until the window + * resets — with `anthropic-usage-limit: slow`. Anthropic then serves the account at lower + * priority instead of rejecting it, so the connection keeps working past the wall (the + * weekly limit still applies). + * + * Wire contract (headers are lowercase on the wire; values verbatim): + * Request `anthropic-usage-limit: slow` + * Wall 429 `anthropic-ratelimit-unified-slow-offer: treatment|control` + * `anthropic-ratelimit-unified-reset: ` (5h window reset) + * `anthropic-ratelimit-unified-slow-retry-after` / `…-slow-max-wait` (seconds) + * Any resp `anthropic-ratelimit-unified-slow-status: active|not_needed|slot_busy| + * weekly_limit|budget_exhausted|ineligible|off` + * `anthropic-ratelimit-unified-slow-budget-utilization` (0..1) + * `anthropic-ratelimit-unified-slow-budget-reset` / `…-7d-reset` (epoch seconds) + * `anthropic-ratelimit-unified-5h-reset` (epoch seconds — window rollover) + * `anthropic-ratelimit-unified-overage-in-use: true|false` + * + * Lifecycle (per connection, in-memory): + * idle ──429 + offer=treatment + lowPriorityMode──▶ active(until unified-reset + 60s) + * active: 2xx keeps it active (`not_needed` = served at standard priority, still active); + * 429 `slot_busy` / 529 → wait `retry-after` (jittered ±30%) and retry the same + * account, up to `max-wait`, then end + 10-minute cool-off; + * `weekly_limit` / `budget_exhausted` / `off` / `ineligible` → end, the 429 flows + * to the normal cooldown path; 5h window rollover → end. + * + * State is deliberately not persisted: after a restart the next wall 429 re-activates the + * lane at the cost of one extra round trip. The executor never surfaces the intercepted 429 + * to chatCore, so an account riding the slow lane is NOT put in connection cooldown. + * + * Pure module (no fetch, no timers) — the executor owns the sleep; the session-limit reset + * claim (`claudeLimitReset.ts`) is injected as a callback so this file stays unit-testable. + */ + +type JsonRecord = Record; + +export const CLAUDE_USAGE_LIMIT_HEADER = "anthropic-usage-limit"; +export const CLAUDE_USAGE_LIMIT_SLOW = "slow"; + +export const CLAUDE_UNIFIED_HEADERS = Object.freeze({ + status: "anthropic-ratelimit-unified-status", + reset: "anthropic-ratelimit-unified-reset", + reset5h: "anthropic-ratelimit-unified-5h-reset", + reset7d: "anthropic-ratelimit-unified-7d-reset", + representativeClaim: "anthropic-ratelimit-unified-representative-claim", + overageStatus: "anthropic-ratelimit-unified-overage-status", + overageInUse: "anthropic-ratelimit-unified-overage-in-use", + slowOffer: "anthropic-ratelimit-unified-slow-offer", + slowStatus: "anthropic-ratelimit-unified-slow-status", + slowRetryAfter: "anthropic-ratelimit-unified-slow-retry-after", + slowMaxWait: "anthropic-ratelimit-unified-slow-max-wait", + slowBudgetUtilization: "anthropic-ratelimit-unified-slow-budget-utilization", + slowBudgetReset: "anthropic-ratelimit-unified-slow-budget-reset", +}); + +/** Defaults + clamps mirror Claude Code 2.1.263 exactly. */ +export const CLAUDE_LOW_PRIORITY_DEFAULTS = Object.freeze({ + retryAfterMs: 20_000, + retryAfterMinMs: 5_000, + retryAfterMaxMs: 600_000, + maxWaitMs: 1_200_000, + maxWaitMinMs: 60_000, + maxWaitMaxMs: 21_600_000, + /** ±30% jitter on every wait so parallel requests do not stampede the slot. */ + jitter: 0.3, + /** Active lane survives the announced reset by this much before lazy expiry. */ + resetGraceMs: 60_000, + /** A 5h-reset header this far past the accepted reset means the window rolled over. */ + rolloverToleranceSeconds: 60, + /** After giving up on `max-wait`, do not auto-accept a new offer for this long. */ + cooloffMs: 10 * 60_000, + /** `budget_exhausted` remembers "spent" until the announced reset, capped at 8 days. */ + budgetSpentCapSeconds: 8 * 86_400, +}); + +export type ClaudeSlowOffer = "treatment" | "control"; +export type ClaudeSlowStatus = + | "active" + | "not_needed" + | "slot_busy" + | "weekly_limit" + | "budget_exhausted" + | "ineligible" + | "off" + | "unrecognized"; + +export type ClaudeLowPriorityEndReason = + | "reset" + | "weekly" + | "budget" + | "off" + | "ineligible" + | "wall" + | "max_wait" + | "extra_usage" + | "operator"; + +export type ClaudeUsageLimitConfig = { + /** Opt-in: accept the slow-lane offer on the 5h wall (`anthropic-usage-limit: slow`). */ + lowPriorityMode: boolean; + /** Opt-in: claim the once-a-week session-limit reset on the 5h wall (`/limit-reset`). */ + autoLimitReset: boolean; +}; + +export type ClaudeLowPriorityWait = { + current: { sinceMs: number; attempts: number; nextTryAtMs: number } | null; +}; + +export type ClaudeUsageLimitDecision = + | { kind: "none" } + | { + kind: "retry"; + delayMs: number; + via: "limit-reset" | "low-priority-accepted" | "slot-busy" | "capacity-busy"; + } + | { kind: "ended"; reason: ClaudeLowPriorityEndReason }; + +export type ClaudeLowPrioritySnapshot = { + active: boolean; + resetsAtSeconds: number | null; + acceptedAtMs: number | null; + retryAfterMs: number; + maxWaitMs: number; + requestsServed: number; + requestsServedStandard: number; + budgetUtilization: number | null; + budgetSpentUntilSeconds: number | null; + coolingOffUntilMs: number | null; +}; + +type Phase = { phase: "idle" } | { phase: "active"; resetsAtSeconds: number; acceptedAtMs: number }; + +type Entry = { + state: Phase; + retryAfterMs: number; + maxWaitMs: number; + requestsServed: number; + requestsServedStandard: number; + budgetUtilization?: number; + budgetSpentUntilSeconds?: number; + coolingOffUntilMs?: number; +}; + +type HeaderSource = Headers | Record | null | undefined; + +/** + * FIFO cap on the per-connection state map. The key is the connection id, but falls back to + * the access token for callers without one — and OAuth tokens rotate on every refresh, so an + * uncapped map would grow for the process lifetime. Same bound and eviction policy as the + * identity caches in `open-sse/executors/claudeIdentity.ts`. + */ +export const CLAUDE_LOW_PRIORITY_CACHE_LIMIT = 10_000; + +const entries = new Map(); +const D = CLAUDE_LOW_PRIORITY_DEFAULTS; + +/** Insert with FIFO eviction once the map reaches `max`. JS Maps preserve insertion order. */ +export function setBoundedEntry(map: Map, key: K, value: V, max: number): void { + if (!map.has(key) && map.size >= max) { + const oldest = map.keys().next().value as K | undefined; + if (oldest !== undefined) map.delete(oldest); + } + map.set(key, value); +} + +function asRecord(value: unknown): JsonRecord { + return value && typeof value === "object" && !Array.isArray(value) ? (value as JsonRecord) : {}; +} + +/** Read the two per-connection opt-ins from `providerSpecificData` (both default off). */ +export function readClaudeUsageLimitConfig(providerSpecificData: unknown): ClaudeUsageLimitConfig { + const psd = asRecord(providerSpecificData); + return { + lowPriorityMode: psd.lowPriorityMode === true, + autoLimitReset: psd.autoLimitReset === true, + }; +} + +/** Stable per-connection state key: connection id, else the token itself (same seed rule as identity). */ +export function resolveClaudeUsageLimitKey(credentials: { + connectionId?: string | null; + accessToken?: string | null; +}): string { + return credentials.connectionId || credentials.accessToken || "anon"; +} + +export function getClaudeHeader(headers: HeaderSource, name: string): string | undefined { + if (!headers) return undefined; + if (typeof (headers as Headers).get === "function") { + const v = (headers as Headers).get(name); + return v === null ? undefined : v; + } + const record = headers as Record; + const direct = record[name]; + if (typeof direct === "string") return direct; + const lower = name.toLowerCase(); + for (const key of Object.keys(record)) { + if (key.toLowerCase() === lower && typeof record[key] === "string") return record[key]; + } + return undefined; +} + +export function parseClaudeSlowOffer(headers: HeaderSource): ClaudeSlowOffer | undefined { + const v = getClaudeHeader(headers, CLAUDE_UNIFIED_HEADERS.slowOffer); + return v === "treatment" || v === "control" ? v : undefined; +} + +export function parseClaudeSlowStatus(headers: HeaderSource): ClaudeSlowStatus | undefined { + const v = getClaudeHeader(headers, CLAUDE_UNIFIED_HEADERS.slowStatus); + if (v === undefined) return undefined; + switch (v) { + case "active": + case "not_needed": + case "slot_busy": + case "weekly_limit": + case "budget_exhausted": + case "ineligible": + case "off": + return v; + default: + return "unrecognized"; + } +} + +/** Non-negative finite number header (seconds / ratios); undefined when absent or malformed. */ +export function parseClaudeHeaderNumber(headers: HeaderSource, name: string): number | undefined { + const raw = getClaudeHeader(headers, name); + if (raw === undefined || raw.trim() === "") return undefined; + const n = Number(raw); + return Number.isFinite(n) && n >= 0 ? n : undefined; +} + +/** + * True when a 429 is the account's unified usage wall (5h/7d subscription window), as + * opposed to a per-minute burst limit: `anthropic-ratelimit-unified-status: rejected`. + */ +export function isClaudeUsageWall(headers: HeaderSource): boolean { + return getClaudeHeader(headers, CLAUDE_UNIFIED_HEADERS.status) === "rejected"; +} + +/** The window the wall 429 blames (`five_hour`, `seven_day`, …), when announced. */ +export function parseClaudeRepresentativeClaim(headers: HeaderSource): string | undefined { + return getClaudeHeader(headers, CLAUDE_UNIFIED_HEADERS.representativeClaim) || undefined; +} + +function clamp(value: number, min: number, max: number): number { + return Math.min(max, Math.max(min, Math.round(value))); +} + +function freshEntry(): Entry { + return { + state: { phase: "idle" }, + retryAfterMs: D.retryAfterMs, + maxWaitMs: D.maxWaitMs, + requestsServed: 0, + requestsServedStandard: 0, + }; +} + +function entryFor(key: string): Entry { + let e = entries.get(key); + if (!e) { + e = freshEntry(); + setBoundedEntry(entries, key, e, CLAUDE_LOW_PRIORITY_CACHE_LIMIT); + } + return e; +} + +function expireIfPastReset(key: string, e: Entry, now: number): void { + if (e.state.phase !== "active") return; + if (now >= e.state.resetsAtSeconds * 1000 + D.resetGraceMs) endEntry(key, e, "reset", now); +} + +function endEntry(key: string, e: Entry, reason: ClaudeLowPriorityEndReason, now: number): void { + if (e.state.phase !== "active") return; + if (reason === "max_wait") e.coolingOffUntilMs = now + D.cooloffMs; + e.state = { phase: "idle" }; + e.requestsServed = 0; + e.requestsServedStandard = 0; + entries.set(key, e); +} + +function applyWaitHints(e: Entry, headers: HeaderSource): void { + const retry = parseClaudeHeaderNumber(headers, CLAUDE_UNIFIED_HEADERS.slowRetryAfter); + if (retry !== undefined) + e.retryAfterMs = clamp(retry * 1000, D.retryAfterMinMs, D.retryAfterMaxMs); + const maxWait = parseClaudeHeaderNumber(headers, CLAUDE_UNIFIED_HEADERS.slowMaxWait); + if (maxWait !== undefined) e.maxWaitMs = clamp(maxWait * 1000, D.maxWaitMinMs, D.maxWaitMaxMs); + const util = parseClaudeHeaderNumber(headers, CLAUDE_UNIFIED_HEADERS.slowBudgetUtilization); + if (util !== undefined) e.budgetUtilization = Math.min(1, util); +} + +function recordBudgetSpent(e: Entry, headers: HeaderSource, now: number): void { + const reset = + parseClaudeHeaderNumber(headers, CLAUDE_UNIFIED_HEADERS.slowBudgetReset) ?? + parseClaudeHeaderNumber(headers, CLAUDE_UNIFIED_HEADERS.reset7d); + if (reset !== undefined && reset > 0) { + e.budgetSpentUntilSeconds = Math.round(Math.min(reset, now / 1000 + D.budgetSpentCapSeconds)); + } +} + +function isBudgetSpent(e: Entry, now: number): boolean { + return e.budgetSpentUntilSeconds !== undefined && now < e.budgetSpentUntilSeconds * 1000; +} + +function isCoolingOff(e: Entry, now: number): boolean { + return e.coolingOffUntilMs !== undefined && now < e.coolingOffUntilMs; +} + +export function isClaudeLowPriorityActive(key: string, now: number = Date.now()): boolean { + const e = entries.get(key); + if (!e) return false; + expireIfPastReset(key, e, now); + return e.state.phase === "active"; +} + +export function getClaudeLowPrioritySnapshot( + key: string, + now: number = Date.now() +): ClaudeLowPrioritySnapshot { + const e = entries.get(key) ?? freshEntry(); + expireIfPastReset(key, e, now); + return { + active: e.state.phase === "active", + resetsAtSeconds: e.state.phase === "active" ? e.state.resetsAtSeconds : null, + acceptedAtMs: e.state.phase === "active" ? e.state.acceptedAtMs : null, + retryAfterMs: e.retryAfterMs, + maxWaitMs: e.maxWaitMs, + requestsServed: e.requestsServed, + requestsServedStandard: e.requestsServedStandard, + budgetUtilization: e.budgetUtilization ?? null, + budgetSpentUntilSeconds: e.budgetSpentUntilSeconds ?? null, + coolingOffUntilMs: e.coolingOffUntilMs ?? null, + }; +} + +export function endClaudeLowPriority( + key: string, + reason: ClaudeLowPriorityEndReason = "operator", + now: number = Date.now() +): boolean { + const e = entries.get(key); + if (!e || e.state.phase !== "active") return false; + endEntry(key, e, reason, now); + return true; +} + +/** + * Accept the slow-lane offer carried by a wall 429. Returns true when the lane became + * active (the caller must immediately retry the same account — the retry carries + * `anthropic-usage-limit: slow`). Mirrors Claude Code's acceptance gate: offer must be + * `treatment`, the announced reset must be in the future, and neither the weekly + * slow-lane budget nor the post-`max_wait` cool-off may be in effect. + */ +export function tryActivateClaudeLowPriority( + key: string, + headers: HeaderSource, + now: number = Date.now() +): boolean { + const e = entryFor(key); + expireIfPastReset(key, e, now); + if (e.state.phase === "active") return false; + if (parseClaudeSlowOffer(headers) !== "treatment") return false; + if (isBudgetSpent(e, now) || isCoolingOff(e, now)) return false; + const resetsAt = + parseClaudeHeaderNumber(headers, CLAUDE_UNIFIED_HEADERS.reset) ?? + parseClaudeHeaderNumber(headers, CLAUDE_UNIFIED_HEADERS.reset5h); + if (resetsAt === undefined || resetsAt * 1000 <= now) return false; + e.retryAfterMs = D.retryAfterMs; + e.maxWaitMs = D.maxWaitMs; + applyWaitHints(e, headers); + e.budgetUtilization = undefined; + e.requestsServed = 0; + e.requestsServedStandard = 0; + e.state = { phase: "active", resetsAtSeconds: resetsAt, acceptedAtMs: now }; + return true; +} + +/** `ineligible` + paid overage now covering the wall — the lane is no longer the thing serving us. */ +function isOverageTakeover(status: ClaudeSlowStatus | undefined, headers: HeaderSource): boolean { + return ( + status === "ineligible" && + getClaudeHeader(headers, CLAUDE_UNIFIED_HEADERS.overageInUse) === "true" + ); +} + +function mapEndReason( + status: ClaudeSlowStatus | undefined, + headers: HeaderSource +): ClaudeLowPriorityEndReason | null { + // Overage takeover wins over the plain `ineligible` mapping on every status, so a wall + // 429 that also announces paid overage ends the lane as `extra_usage` (the state the + // rest of the codebase keys on — see src/lib/providers/claudeExtraUsage.ts), not as a + // generic ineligibility. + if (isOverageTakeover(status, headers)) return "extra_usage"; + switch (status) { + case "weekly_limit": + return "weekly"; + case "budget_exhausted": + return "budget"; + case "off": + return "off"; + case "ineligible": + return "ineligible"; + case "slot_busy": + return null; + default: { + // A 429 with unified rate-limit headers but no slow verdict is a real wall + // (e.g. the weekly window closed) — the lane cannot help any more. + const unified = + getClaudeHeader(headers, CLAUDE_UNIFIED_HEADERS.representativeClaim) || + getClaudeHeader(headers, CLAUDE_UNIFIED_HEADERS.overageStatus); + return unified ? "wall" : null; + } + } +} + +function waitOrGiveUp( + key: string, + e: Entry, + wait: ClaudeLowPriorityWait, + via: "slot-busy" | "capacity-busy", + now: number, + random: () => number, + waitCeilingMs?: number +): ClaudeUsageLimitDecision { + if (e.state.phase !== "active") return { kind: "none" }; + const acceptedAtMs = e.state.acceptedAtMs; + const current = + wait.current && wait.current.sinceMs >= acceptedAtMs + ? wait.current + : { sinceMs: now, attempts: 0, nextTryAtMs: now }; + // The caller's own budget (the request's upstream timeout) caps the server-announced + // max-wait: sleeping past it would be aborted mid-wait, surfacing a hard TimeoutError + // instead of the graceful max_wait end + cool-off. + const effectiveMaxWaitMs = + waitCeilingMs === undefined ? e.maxWaitMs : Math.min(e.maxWaitMs, Math.max(0, waitCeilingMs)); + const waitedMs = now - current.sinceMs; + if (waitedMs >= effectiveMaxWaitMs) { + wait.current = null; + endEntry(key, e, "max_wait", now); + return { kind: "ended", reason: "max_wait" }; + } + const jitter = 1 + (random() * 2 - 1) * D.jitter; + const delayMs = Math.min( + Math.max(0, Math.round(e.retryAfterMs * jitter)), + effectiveMaxWaitMs - waitedMs + ); + wait.current = { + sinceMs: current.sinceMs, + attempts: current.attempts + 1, + nextTryAtMs: now + delayMs, + }; + return { kind: "retry", delayMs, via }; +} + +/** Error-status half of the observation: terminal verdict, slot/capacity wait, or nothing. */ +function observeErrorResponse( + ctx: ObserveContext, + status: number, + slowStatus: ClaudeSlowStatus | undefined +): ClaudeUsageLimitDecision { + const { key, entry, headers, wait, now, random, waitCeilingMs } = ctx; + if (status === 429) { + const reason = mapEndReason(slowStatus, headers); + if (reason) { + if (reason === "budget") recordBudgetSpent(entry, headers, now); + endEntry(key, entry, reason, now); + return { kind: "ended", reason }; + } + if (slowStatus === "slot_busy") { + return waitOrGiveUp(key, entry, wait, "slot-busy", now, random, waitCeilingMs); + } + return { kind: "none" }; + } + return waitOrGiveUp(key, entry, wait, "capacity-busy", now, random, waitCeilingMs); +} + +/** Success half: window rollover, served counters, or the overage takeover. */ +function observeSuccessResponse( + ctx: ObserveContext, + slowStatus: ClaudeSlowStatus | undefined +): ClaudeUsageLimitDecision { + const { key, entry, headers, wait, now } = ctx; + if (entry.state.phase !== "active") return { kind: "none" }; + + // Window rollover announced by the server → the wall is gone, drop the header. + const reset5h = parseClaudeHeaderNumber(headers, CLAUDE_UNIFIED_HEADERS.reset5h); + if ( + reset5h !== undefined && + reset5h >= entry.state.resetsAtSeconds + D.rolloverToleranceSeconds + ) { + endEntry(key, entry, "reset", now); + return { kind: "ended", reason: "reset" }; + } + + if (slowStatus === "active") { + entry.requestsServed += 1; + wait.current = null; + } else if (slowStatus === "not_needed") { + entry.requestsServedStandard += 1; + wait.current = null; + } else if (isOverageTakeover(slowStatus, headers)) { + endEntry(key, entry, "extra_usage", now); + return { kind: "ended", reason: "extra_usage" }; + } + return { kind: "none" }; +} + +type ObserveContext = { + key: string; + entry: Entry; + headers: HeaderSource; + wait: ClaudeLowPriorityWait; + now: number; + random: () => number; + waitCeilingMs?: number; +}; + +/** + * Observe an upstream response for an ACTIVE lane: keeps counters/hints fresh, ends the + * lane on terminal verdicts, and asks for a same-account retry on `slot_busy` (429) or + * capacity (529) while inside `max-wait`. + */ +export function observeClaudeLowPriorityResponse( + key: string, + response: { status: number; headers: HeaderSource }, + wait: ClaudeLowPriorityWait, + now: number = Date.now(), + random: () => number = Math.random, + waitCeilingMs?: number +): ClaudeUsageLimitDecision { + const entry = entries.get(key); + if (!entry) return { kind: "none" }; + expireIfPastReset(key, entry, now); + if (entry.state.phase !== "active") return { kind: "none" }; + + const headers = response.headers; + const slowStatus = parseClaudeSlowStatus(headers); + applyWaitHints(entry, headers); + const ctx: ObserveContext = { key, entry, headers, wait, now, random, waitCeilingMs }; + + const isCapacityWait = + response.status === 529 && (slowStatus === "active" || slowStatus === undefined); + if (response.status === 429 || isCapacityWait) { + return observeErrorResponse(ctx, response.status, slowStatus); + } + return observeSuccessResponse(ctx, slowStatus); +} + +/** + * One call per upstream response, from the Claude OAuth executor. Decides whether the + * executor should retry the same account (`retry`) instead of surfacing the response. + * + * Idle + wall 429: try the once-a-week session-limit reset first (restores full speed), + * then accept the slow-lane offer. Active: delegate to observeClaudeLowPriorityResponse. + */ +export async function handleClaudeUsageLimitResponse(opts: { + key: string; + config: ClaudeUsageLimitConfig; + response: { status: number; headers: HeaderSource }; + wait: ClaudeLowPriorityWait; + /** Attempts the `/limit-reset` claim; resolves true when the 5h window was reset. */ + claimLimitReset?: () => Promise; + /** + * Whether THIS request went out with `anthropic-usage-limit: slow`. Two requests on the + * same connection can hit the wall together: the first one activates the lane, the second + * one's 429 (built while the lane was still idle, so without the header) must not be read + * as a verdict on the lane — it just joins it. Defaults to "matches the lane state". + */ + sentSlow?: boolean; + /** + * The caller's remaining budget for THIS request (its upstream timeout). Caps the + * server-announced max-wait so the lane ends gracefully instead of being aborted mid-sleep. + */ + waitCeilingMs?: number; + now?: number; + random?: () => number; +}): Promise { + const now = opts.now ?? Date.now(); + const { key, config, response, wait } = opts; + const headers = response.headers; + const atWall = isClaudeUsageWall(headers) || parseClaudeSlowOffer(headers) !== undefined; + + if (isClaudeLowPriorityActive(key, now)) { + // Raced activation (see `sentSlow`): a header-less sibling's outcome is not lane + // telemetry. Re-send it on the lane if it hit the wall, otherwise ignore it. + if (opts.sentSlow === false) { + const rejoin = response.status === 429 && atWall; + return rejoin + ? { kind: "retry", delayMs: 0, via: "low-priority-accepted" } + : { kind: "none" }; + } + return observeClaudeLowPriorityResponse( + key, + response, + wait, + now, + opts.random, + opts.waitCeilingMs + ); + } + + const optedIn = config.lowPriorityMode || config.autoLimitReset; + if (response.status !== 429 || !optedIn || !atWall) return { kind: "none" }; + + if (await shouldClaimLimitReset(config, headers, opts.claimLimitReset)) { + return { kind: "retry", delayMs: 0, via: "limit-reset" }; + } + + if (config.lowPriorityMode && tryActivateClaudeLowPriority(key, headers, now)) { + wait.current = null; + return { kind: "retry", delayMs: 0, via: "low-priority-accepted" }; + } + + return { kind: "none" }; +} + +/** + * Opt-in weekly session-limit reset, attempted before the slow lane. Only for a wall that + * blames the 5-hour window (or names no window at all); a failing claim is never fatal. + */ +async function shouldClaimLimitReset( + config: ClaudeUsageLimitConfig, + headers: HeaderSource, + claimLimitReset: (() => Promise) | undefined +): Promise { + if (!config.autoLimitReset || !claimLimitReset) return false; + const claim = parseClaudeRepresentativeClaim(headers); + if (claim !== undefined && claim !== "five_hour") return false; + try { + return await claimLimitReset(); + } catch { + return false; + } +} + +export function createClaudeLowPriorityWait(): ClaudeLowPriorityWait { + return { current: null }; +} + +/** Test-only: drop every connection's lane state. */ +export function _resetClaudeLowPriorityState(): void { + entries.clear(); +} diff --git a/src/app/(dashboard)/dashboard/providers/[id]/components/modals/ClaudeConnectionFields.tsx b/src/app/(dashboard)/dashboard/providers/[id]/components/modals/ClaudeConnectionFields.tsx new file mode 100644 index 0000000000..71d64562fd --- /dev/null +++ b/src/app/(dashboard)/dashboard/providers/[id]/components/modals/ClaudeConnectionFields.tsx @@ -0,0 +1,51 @@ +"use client"; + +import { useTranslations } from "next-intl"; +import { Toggle } from "@/shared/components"; + +type ClaudeConnectionFieldsProps = { + values: { + blockExtraUsage: boolean; + lowPriorityMode: boolean; + autoLimitReset: boolean; + }; + /** The usage-wall options only exist for subscription (OAuth) connections. */ + showUsageWallOptions: boolean; + onChange: (patch: Partial) => void; +}; + +/** + * Per-connection Claude options. `blockExtraUsage` steers fallback away from + * pay-as-you-go overage; the two usage-wall toggles mirror Claude Code's + * `/low-priority` and `/limit-reset` (see open-sse/services/claudeLowPriority.ts). + */ +export default function ClaudeConnectionFields(props: ClaudeConnectionFieldsProps) { + const t = useTranslations("providers"); + + return ( +
+ props.onChange({ blockExtraUsage: checked })} + label={t("blockClaudeExtraUsageLabel")} + description={t("blockClaudeExtraUsageDescription")} + /> + {props.showUsageWallOptions && ( + <> + props.onChange({ lowPriorityMode: checked })} + label={t("claudeLowPriorityModeLabel")} + description={t("claudeLowPriorityModeDescription")} + /> + props.onChange({ autoLimitReset: checked })} + label={t("claudeAutoLimitResetLabel")} + description={t("claudeAutoLimitResetDescription")} + /> + + )} +
+ ); +} diff --git a/src/app/(dashboard)/dashboard/providers/[id]/components/modals/EditConnectionModal.tsx b/src/app/(dashboard)/dashboard/providers/[id]/components/modals/EditConnectionModal.tsx index 347c5ec3e5..aaeebd57c3 100644 --- a/src/app/(dashboard)/dashboard/providers/[id]/components/modals/EditConnectionModal.tsx +++ b/src/app/(dashboard)/dashboard/providers/[id]/components/modals/EditConnectionModal.tsx @@ -20,7 +20,6 @@ import { maskEmail } from "@/shared/utils/maskEmail"; import useEmailPrivacyStore from "@/store/emailPrivacyStore"; import { useNotificationStore } from "@/store/notificationStore"; import { type CodexServiceTier } from "@/lib/providers/requestDefaults"; -import { isClaudeExtraUsageBlockEnabled } from "@/lib/providers/claudeExtraUsage"; import { resolveDashboardProviderInfo } from "../../../providerPageUtils"; import { isBaseUrlConfigurableProvider, @@ -52,6 +51,8 @@ import { useOpenRouterPresetControl } from "../OpenRouterPresetInput"; import WebSessionCredentialGuide from "../WebSessionCredentialGuide"; import HarImportButton from "../HarImportButton"; import CcCompatibleRequestDefaultsFields from "./CcCompatibleRequestDefaultsFields"; +import ClaudeConnectionFields from "./ClaudeConnectionFields"; +import { claudeConnectionFieldPatch, claudeConnectionFieldValues } from "./claudeConnectionFields"; import { CodexConnectionFields } from "./CodexFingerprintFields"; import { assignEditApiKeyProviderSpecificData } from "./connectionProviderSpecificData"; import { isM365TierCapableProvider, normalizeM365TierValue, type M365TierValue } from "./m365Tier"; @@ -152,10 +153,7 @@ export default function EditConnectionModal({ ccCompatibleSummarizeThinking: false, cloudCodeProjectId: "", antigravityClientProfile: "ide", - blockExtraUsage: - provider === "claude" - ? isClaudeExtraUsageBlockEnabled(provider, connectionProviderSpecificData) - : false, + ...claudeConnectionFieldValues(provider, connectionProviderSpecificData), passthroughModels: connectionProviderSpecificData?.passthroughModels === true, disableCooling: connectionProviderSpecificData?.disableCooling === true, importFreeModelsOnly: connectionProviderSpecificData?.importFreeModelsOnly === true, @@ -393,10 +391,7 @@ export default function EditConnectionModal({ antigravityClientProfile: normalizeAntigravityClientProfileSetting( connection.providerSpecificData?.clientProfile ), - blockExtraUsage: isClaudeExtraUsageBlockEnabled( - effectiveProvider, - connection.providerSpecificData - ), + ...claudeConnectionFieldValues(effectiveProvider, connection.providerSpecificData), passthroughModels: connection?.providerSpecificData?.passthroughModels === true, disableCooling: connection?.providerSpecificData?.disableCooling === true, importFreeModelsOnly: connection?.providerSpecificData?.importFreeModelsOnly === true, @@ -694,7 +689,7 @@ export default function EditConnectionModal({ excludedModels: parseExcludedModelsInput(formData.excludedModels), }; if (isClaude) { - updates.providerSpecificData.blockExtraUsage = formData.blockExtraUsage; + Object.assign(updates.providerSpecificData, claudeConnectionFieldPatch(formData)); } if (isCodex) { updates.providerSpecificData.requestDefaults = { @@ -839,14 +834,11 @@ export default function EditConnectionModal({ /> )} {isClaude && ( -
- setFormData({ ...formData, blockExtraUsage: checked })} - label={t("blockClaudeExtraUsageLabel")} - description={t("blockClaudeExtraUsageDescription")} - /> -
+ setFormData({ ...formData, ...patch })} + /> )} {(isCcCompatible || openRouterPreset.input) && (
diff --git a/src/app/(dashboard)/dashboard/providers/[id]/components/modals/claudeConnectionFields.ts b/src/app/(dashboard)/dashboard/providers/[id]/components/modals/claudeConnectionFields.ts new file mode 100644 index 0000000000..d7eb9f972c --- /dev/null +++ b/src/app/(dashboard)/dashboard/providers/[id]/components/modals/claudeConnectionFields.ts @@ -0,0 +1,34 @@ +import { isClaudeExtraUsageBlockEnabled } from "@/lib/providers/claudeExtraUsage"; + +export type ClaudeConnectionFieldValues = { + blockExtraUsage: boolean; + lowPriorityMode: boolean; + autoLimitReset: boolean; +}; + +/** + * Per-connection Claude form fields read out of `providerSpecificData`. Both usage-wall + * opt-ins default to off; `blockExtraUsage` defaults to on for Claude (see + * `isClaudeExtraUsageBlockEnabled`). Shared by the modal's two initialization sites. + */ +export function claudeConnectionFieldValues( + provider: string | null | undefined, + providerSpecificData: Record | null | undefined +): ClaudeConnectionFieldValues { + return { + blockExtraUsage: isClaudeExtraUsageBlockEnabled(provider, providerSpecificData), + lowPriorityMode: providerSpecificData?.lowPriorityMode === true, + autoLimitReset: providerSpecificData?.autoLimitReset === true, + }; +} + +/** The same three fields on their way back into `providerSpecificData` on save. */ +export function claudeConnectionFieldPatch( + values: ClaudeConnectionFieldValues +): ClaudeConnectionFieldValues { + return { + blockExtraUsage: values.blockExtraUsage, + lowPriorityMode: values.lowPriorityMode, + autoLimitReset: values.autoLimitReset, + }; +} diff --git a/src/i18n/messages/en.json b/src/i18n/messages/en.json index f0d08510c3..18433ec10a 100644 --- a/src/i18n/messages/en.json +++ b/src/i18n/messages/en.json @@ -5798,6 +5798,10 @@ "grokWebCookieHint": "Grok Web Cookie Hint", "blockClaudeExtraUsageDescription": "When enabled, OmniRoute marks this Claude connection unavailable as soon as the usage API reports queued extra usage, so fallback switches to another connection instead of continuing on pay-as-you-go extra billing.", "blockClaudeExtraUsageLabel": "Block extra Claude usage", + "claudeLowPriorityModeLabel": "Continue at lower priority after the 5-hour limit", + "claudeLowPriorityModeDescription": "When this account hits its 5-hour usage limit and Anthropic offers the lower-priority lane, OmniRoute accepts it and keeps sending requests with anthropic-usage-limit: slow until the window resets, instead of cooling the connection down. The weekly limit still applies and responses may wait for spare capacity.", + "claudeAutoLimitResetLabel": "Auto-reset the session limit (once a week)", + "claudeAutoLimitResetDescription": "When this account hits its 5-hour usage limit and a session-limit reset is available, OmniRoute claims it automatically and retries at full speed. Uses the account's once-a-week reset and still counts toward the weekly limit.", "disableCoolingDescription": "Skip the transient cooldown so this connection stays eligible even after recoverable errors (terminal states like banned/expired still apply).", "disableCoolingLabel": "Disable cooldown for this connection", "bulkPasteAdded": "{count, plural, one {1 key added} other {# keys added}}", diff --git a/src/i18n/messages/it.json b/src/i18n/messages/it.json index ee02c94831..d0a6f84f35 100644 --- a/src/i18n/messages/it.json +++ b/src/i18n/messages/it.json @@ -5795,6 +5795,10 @@ "grokWebCookieHint": "Suggerimento per il cookie web di Grok", "blockClaudeExtraUsageDescription": "Se abilitato, OmniRoute segna questa connessione Claude come non disponibile non appena l'API di utilizzo segnala extra usage in coda, in modo che il fallback passi a un'altra connessione invece di continuare con una fatturazione extra pay-as-you-go.", "blockClaudeExtraUsageLabel": "Blocca l'extra usage di Claude", + "claudeLowPriorityModeLabel": "Continua a priorità ridotta dopo il limite delle 5 ore", + "claudeLowPriorityModeDescription": "Quando questo account raggiunge il limite di utilizzo delle 5 ore e Anthropic offre la corsia a priorità ridotta, OmniRoute la accetta e continua a inviare le richieste con anthropic-usage-limit: slow fino al reset della finestra, invece di mettere la connessione in cooldown. Il limite settimanale resta valido e le risposte possono attendere capacità disponibile.", + "claudeAutoLimitResetLabel": "Reset automatico del limite di sessione (una volta a settimana)", + "claudeAutoLimitResetDescription": "Quando questo account raggiunge il limite di utilizzo delle 5 ore ed è disponibile un reset del limite di sessione, OmniRoute lo richiede automaticamente e riprova a piena velocità. Consuma il reset settimanale dell'account e conta comunque verso il limite settimanale.", "disableCoolingDescription": "Salta il cooldown transitorio in modo che questa connessione rimanga idonea anche dopo errori ripristinabili (gli stati terminali come bannato/scaduto si applicano ancora).", "disableCoolingLabel": "Disabilita il cooldown per questa connessione", "bulkPasteAdded": "{count, plural, one {1 chiave aggiunta} other {# chiavi aggiunte}}", diff --git a/src/i18n/messages/pt-BR.json b/src/i18n/messages/pt-BR.json index 6710a83964..2d47bd31e1 100644 --- a/src/i18n/messages/pt-BR.json +++ b/src/i18n/messages/pt-BR.json @@ -5795,6 +5795,10 @@ "grokWebCookieHint": "Cole o cookie sso do grok.com. Um valor completo `sso=...` também funciona.", "blockClaudeExtraUsageDescription": "Quando habilitado, o OmniRoute marca esta conta do Claude Code como indisponível assim que a API de usage reporta `extra_usage.queued`, para que o fallback troque para outra conta antes de continuar com cobranças extras pay-as-you-go.", "blockClaudeExtraUsageLabel": "Bloquear Claude Extra Usage", + "claudeLowPriorityModeLabel": "Continuar em prioridade menor após o limite de 5 horas", + "claudeLowPriorityModeDescription": "Quando esta conta atinge o limite de uso de 5 horas e a Anthropic oferece a faixa de prioridade menor, o OmniRoute aceita e segue enviando as requisições com anthropic-usage-limit: slow até a janela reiniciar, em vez de colocar a conexão em cooldown. O limite semanal continua valendo e as respostas podem aguardar capacidade livre.", + "claudeAutoLimitResetLabel": "Reset automático do limite de sessão (uma vez por semana)", + "claudeAutoLimitResetDescription": "Quando esta conta atinge o limite de uso de 5 horas e há um reset de limite de sessão disponível, o OmniRoute o solicita automaticamente e repete a requisição em velocidade normal. Consome o reset semanal da conta e ainda conta para o limite semanal.", "disableCoolingDescription": "Ignore o tempo de espera transitório para que esta conexão permaneça elegível mesmo após erros recuperáveis (estados terminais como banido/expirado ainda se aplicam).", "disableCoolingLabel": "Desativar o tempo de espera para esta conexão", "bulkPasteAdded": "{count, plural, one {1 chave adicionada} other {# chaves adicionadas}}", diff --git a/src/lib/providers/requestDefaults.ts b/src/lib/providers/requestDefaults.ts index 301ad61f75..b8288dc9b6 100644 --- a/src/lib/providers/requestDefaults.ts +++ b/src/lib/providers/requestDefaults.ts @@ -215,6 +215,15 @@ export function normalizeProviderSpecificData( delete normalized.disableCooling; } + // Claude OAuth usage-wall opt-ins (open-sse/services/claudeLowPriority.ts) — only + // persist real booleans; both default to off when absent. + if ("lowPriorityMode" in normalized && typeof normalized.lowPriorityMode !== "boolean") { + delete normalized.lowPriorityMode; + } + if ("autoLimitReset" in normalized && typeof normalized.autoLimitReset !== "boolean") { + delete normalized.autoLimitReset; + } + if ("peakHourProtection" in normalized) { const peakHourProtection = normalizePeakHourProtection(normalized.peakHourProtection); if (peakHourProtection) { diff --git a/tests/unit/claude-limit-reset.test.ts b/tests/unit/claude-limit-reset.test.ts new file mode 100644 index 0000000000..3551cd4a88 --- /dev/null +++ b/tests/unit/claude-limit-reset.test.ts @@ -0,0 +1,347 @@ +/** + * Claude OAuth once-a-week session-limit reset (open-sse/services/claudeLimitReset.ts). + * + * Wire contract captured from Claude Code 2.1.263 (`/limit-reset`, program `juniper_tide`): + * status via GET /api/oauth/usage?at_wall=1&skip_spend=1 → body.juniper_tide, claim via + * POST /api/organizations/{org}/reset_rate_limits { program: "juniper_tide" }. + */ +import test from "node:test"; +import assert from "node:assert/strict"; + +import { + CLAUDE_LIMIT_RESET_RECENT_WINDOW_MS, + CLAUDE_LIMIT_RESET_RETRY_BACKOFF_MS, + CLAUDE_LIMIT_RESET_STATUS_URL, + _resetClaudeLimitResetMemo, + attemptClaudeLimitReset, + claudeLimitResetClaimUrl, + parseClaudeLimitResetClaim, + parseClaudeLimitResetStatus, +} from "../../open-sse/services/claudeLimitReset.ts"; + +const NOW = 1_800_000_000_000; +const TOKEN = "sk-ant-oat-test-token"; + +type Call = { url: string; method: string; headers: Record; body: unknown }; + +function mockFetch(routes: Record Response | Promise>): { + calls: Call[]; + fetchImpl: typeof fetch; +} { + const calls: Call[] = []; + const fetchImpl = (async (input: unknown, init: RequestInit = {}) => { + const url = String(input); + calls.push({ + url, + method: init.method ?? "GET", + headers: (init.headers as Record) ?? {}, + body: init.body ? JSON.parse(String(init.body)) : undefined, + }); + const route = Object.entries(routes).find(([prefix]) => url.startsWith(prefix)); + if (!route) throw new Error(`unexpected fetch ${url}`); + return route[1](); + }) as typeof fetch; + return { calls, fetchImpl }; +} + +const json = (body: unknown, status = 200) => + new Response(JSON.stringify(body), { status, headers: { "Content-Type": "application/json" } }); + +const CLAIM_PREFIX = "https://api.anthropic.com/api/organizations/"; + +test.beforeEach(() => _resetClaudeLimitResetMemo()); + +test("parseClaudeLimitResetStatus reads the juniper_tide block and tolerates absence", () => { + assert.equal(parseClaudeLimitResetStatus({}), null); + assert.equal( + parseClaudeLimitResetStatus({ juniper_tide: { arm: "reset" } }), + null, + "eligible is required" + ); + assert.deepEqual( + parseClaudeLimitResetStatus({ + juniper_tide: { + eligible: true, + in_experiment: true, + arm: "reset", + available: true, + next_available_at: null, + weekly_resets_at: "2026-09-14T00:00:00Z", + resets_per_week: 1, + }, + }), + { + eligible: true, + ineligibleReason: null, + inExperiment: true, + arm: "reset", + available: true, + nextAvailableAt: null, + weeklyResetsAt: "2026-09-14T00:00:00Z", + resetsPerWeek: 1, + } + ); + assert.equal( + parseClaudeLimitResetStatus({ + juniper_tide: { eligible: false, ineligible_reason: "tier", arm: "bogus" }, + })?.arm, + null + ); +}); + +test("parseClaudeLimitResetClaim maps unknown results to unavailable", () => { + assert.deepEqual( + parseClaudeLimitResetClaim({ result: "reset", next_available_at: "2026-09-15T00:00:00Z" }), + { + result: "reset", + nextAvailableAt: "2026-09-15T00:00:00Z", + weeklyResetsAt: null, + } + ); + assert.equal(parseClaudeLimitResetClaim({ result: "???" }).result, "unavailable"); + assert.equal(parseClaudeLimitResetClaim(null).result, "unavailable"); +}); + +test("claim URL encodes the organization UUID", () => { + assert.equal( + claudeLimitResetClaimUrl("org 1"), + "https://api.anthropic.com/api/organizations/org%201/reset_rate_limits" + ); +}); + +test("attempt: available → claims with program juniper_tide, OAuth bearer, then memoises the week", async () => { + const { calls, fetchImpl } = mockFetch({ + [CLAUDE_LIMIT_RESET_STATUS_URL]: () => + json({ + juniper_tide: { eligible: true, in_experiment: true, arm: "reset", available: true }, + }), + [CLAIM_PREFIX]: () => json({ result: "reset", next_available_at: "2026-09-15T00:00:00Z" }), + }); + + const first = await attemptClaudeLimitReset({ + key: "c1", + accessToken: TOKEN, + providerSpecificData: { organizationUUID: "org-uuid-1" }, + now: NOW, + fetchImpl, + }); + assert.deepEqual(first, { + reset: true, + outcome: "reset", + nextAvailableAt: "2026-09-15T00:00:00Z", + }); + assert.equal(calls.length, 2); + assert.equal(calls[0].method, "GET"); + assert.equal(calls[0].url, CLAUDE_LIMIT_RESET_STATUS_URL); + assert.equal(calls[0].headers.Authorization, `Bearer ${TOKEN}`); + assert.equal(calls[0].headers["anthropic-beta"], "oauth-2025-04-20"); + assert.equal(calls[1].method, "POST"); + assert.equal(calls[1].url, claudeLimitResetClaimUrl("org-uuid-1")); + assert.deepEqual(calls[1].body, { program: "juniper_tide" }); + + // Same wall again → memo skip, no network. + const second = await attemptClaudeLimitReset({ + key: "c1", + accessToken: TOKEN, + now: NOW + 60_000, + fetchImpl, + }); + assert.deepEqual(second, { reset: false, outcome: "memo_skip", nextAvailableAt: null }); + assert.equal(calls.length, 2); +}); + +test("attempt: not offered (already used this week) → no claim, memo until next_available_at", async () => { + const nextAt = new Date(NOW + 3 * 86_400_000).toISOString(); + const { calls, fetchImpl } = mockFetch({ + [CLAUDE_LIMIT_RESET_STATUS_URL]: () => + json({ + juniper_tide: { eligible: true, arm: "reset", available: false, next_available_at: nextAt }, + }), + }); + const r = await attemptClaudeLimitReset({ key: "c1", accessToken: TOKEN, now: NOW, fetchImpl }); + assert.deepEqual(r, { reset: false, outcome: "not_offered", nextAvailableAt: nextAt }); + assert.equal(calls.length, 1); + // Still skipped right before next_available_at, re-queried right after. + assert.equal( + ( + await attemptClaudeLimitReset({ + key: "c1", + accessToken: TOKEN, + now: NOW + 3 * 86_400_000 - 1, + fetchImpl, + }) + ).outcome, + "memo_skip" + ); + assert.equal( + ( + await attemptClaudeLimitReset({ + key: "c1", + accessToken: TOKEN, + now: NOW + 3 * 86_400_000 + 1, + fetchImpl, + }) + ).outcome, + "not_offered" + ); +}); + +test("attempt: control arm / not eligible never claims", async () => { + const { calls, fetchImpl } = mockFetch({ + [CLAUDE_LIMIT_RESET_STATUS_URL]: () => + json({ juniper_tide: { eligible: true, arm: "control", available: true } }), + }); + const r = await attemptClaudeLimitReset({ key: "c1", accessToken: TOKEN, now: NOW, fetchImpl }); + assert.equal(r.outcome, "not_offered"); + assert.equal(calls.length, 1); +}); + +test("attempt: status endpoint failure backs off 15 minutes", async () => { + const { calls, fetchImpl } = mockFetch({ + [CLAUDE_LIMIT_RESET_STATUS_URL]: () => json({ error: "nope" }, 500), + }); + assert.equal( + (await attemptClaudeLimitReset({ key: "c1", accessToken: TOKEN, now: NOW, fetchImpl })).outcome, + "no_status" + ); + assert.equal( + ( + await attemptClaudeLimitReset({ + key: "c1", + accessToken: TOKEN, + now: NOW + CLAUDE_LIMIT_RESET_RETRY_BACKOFF_MS - 1, + fetchImpl, + }) + ).outcome, + "memo_skip" + ); + assert.equal( + ( + await attemptClaudeLimitReset({ + key: "c1", + accessToken: TOKEN, + now: NOW + CLAUDE_LIMIT_RESET_RETRY_BACKOFF_MS, + fetchImpl, + }) + ).outcome, + "no_status" + ); + assert.equal(calls.length, 2); +}); + +test("attempt: claim already_used → not reset, memo for a week", async () => { + const { fetchImpl } = mockFetch({ + [CLAUDE_LIMIT_RESET_STATUS_URL]: () => + json({ juniper_tide: { eligible: true, arm: "reset", available: true } }), + [CLAIM_PREFIX]: () => json({ result: "already_used" }), + }); + const r = await attemptClaudeLimitReset({ + key: "c1", + accessToken: TOKEN, + providerSpecificData: { organizationUUID: "org-1" }, + now: NOW, + fetchImpl, + }); + assert.deepEqual(r, { reset: false, outcome: "already_used", nextAvailableAt: null }); + assert.equal( + ( + await attemptClaudeLimitReset({ + key: "c1", + accessToken: TOKEN, + now: NOW + 6 * 86_400_000, + fetchImpl, + }) + ).outcome, + "memo_skip" + ); +}); + +test("attempt: claim 401/403 → auth_error, 429 → rate_limited (both non-fatal)", async () => { + for (const [status, outcome] of [ + [401, "auth_error"], + [403, "auth_error"], + [429, "rate_limited"], + ] as const) { + _resetClaudeLimitResetMemo(); + const { fetchImpl } = mockFetch({ + [CLAUDE_LIMIT_RESET_STATUS_URL]: () => + json({ juniper_tide: { eligible: true, arm: "reset", available: true } }), + [CLAIM_PREFIX]: () => json({}, status), + }); + const r = await attemptClaudeLimitReset({ + key: "c1", + accessToken: TOKEN, + providerSpecificData: { organizationUUID: "org-1" }, + now: NOW, + fetchImpl, + }); + assert.equal(r.reset, false, String(status)); + assert.equal(r.outcome, outcome, String(status)); + } +}); + +test("attempt: concurrent wall hits share one status+claim round trip (no duplicate POST)", async () => { + const { calls, fetchImpl } = mockFetch({ + [CLAUDE_LIMIT_RESET_STATUS_URL]: () => + json({ juniper_tide: { eligible: true, arm: "reset", available: true } }), + [CLAIM_PREFIX]: () => json({ result: "reset", next_available_at: "2026-09-15T00:00:00Z" }), + }); + const opts = { + key: "c1", + accessToken: TOKEN, + providerSpecificData: { organizationUUID: "org-1" }, + now: NOW, + fetchImpl, + }; + const [a, b, c] = await Promise.all([ + attemptClaudeLimitReset(opts), + attemptClaudeLimitReset(opts), + attemptClaudeLimitReset(opts), + ]); + assert.equal(a.reset, true); + assert.deepEqual(b, a); + assert.deepEqual(c, a); + assert.equal(calls.filter((x) => x.method === "POST").length, 1, "exactly one claim"); + assert.equal(calls.length, 2); +}); + +test("attempt: right after a granted reset, a stale sibling wall is answered reset:true without network", async () => { + const { calls, fetchImpl } = mockFetch({ + [CLAUDE_LIMIT_RESET_STATUS_URL]: () => + json({ juniper_tide: { eligible: true, arm: "reset", available: true } }), + [CLAIM_PREFIX]: () => json({ result: "reset" }), + }); + const base = { + key: "c1", + accessToken: TOKEN, + providerSpecificData: { organizationUUID: "org-1" }, + fetchImpl, + }; + assert.equal((await attemptClaudeLimitReset({ ...base, now: NOW })).outcome, "reset"); + const sibling = await attemptClaudeLimitReset({ ...base, now: NOW + 5_000 }); + assert.deepEqual(sibling, { reset: true, outcome: "recent_reset", nextAvailableAt: null }); + assert.equal(calls.length, 2, "no extra network for the sibling"); + // Past the recent-reset window the weekly memo takes over (the reset is spent). + assert.equal( + (await attemptClaudeLimitReset({ ...base, now: NOW + CLAUDE_LIMIT_RESET_RECENT_WINDOW_MS + 1 })) + .outcome, + "memo_skip" + ); +}); + +test("attempt: not_limited (window already reset server-side) counts as reset → retry at full speed", async () => { + const { fetchImpl } = mockFetch({ + [CLAUDE_LIMIT_RESET_STATUS_URL]: () => + json({ juniper_tide: { eligible: true, arm: "reset", available: true } }), + [CLAIM_PREFIX]: () => json({ result: "not_limited" }), + }); + const r = await attemptClaudeLimitReset({ + key: "c1", + accessToken: TOKEN, + providerSpecificData: { organization_uuid: "org-snake" }, + now: NOW, + fetchImpl, + }); + assert.equal(r.reset, true); + assert.equal(r.outcome, "not_limited"); +}); diff --git a/tests/unit/claude-low-priority-executor.test.ts b/tests/unit/claude-low-priority-executor.test.ts new file mode 100644 index 0000000000..4ea2bb31ee --- /dev/null +++ b/tests/unit/claude-low-priority-executor.test.ts @@ -0,0 +1,244 @@ +/** + * End-to-end wiring of the Claude OAuth lower-priority lane and the session-limit reset + * through `BaseExecutor.execute()` (mirrors the fetch-capture pattern of + * context-editing-executor-injection.test.ts). + * + * Proves, on the real outbound request: + * - no `anthropic-usage-limit` header before the account hits its 5-hour wall; + * - on the wall 429 (slow-offer: treatment) with `lowPriorityMode` on, the executor + * retries the SAME account with `anthropic-usage-limit: slow` and returns the 200 — + * chatCore never sees the 429, so the connection is not cooled down; + * - later requests on that connection carry the header from the first attempt; + * - an opt-out connection surfaces the 429 untouched, header never sent; + * - with `autoLimitReset` on, the wall triggers status + claim and retries at full speed. + */ +import test from "node:test"; +import assert from "node:assert/strict"; + +import { DefaultExecutor } from "../../open-sse/executors/default.ts"; +import { _resetClaudeLowPriorityState } from "../../open-sse/services/claudeLowPriority.ts"; +import { + CLAUDE_LIMIT_RESET_STATUS_URL, + _resetClaudeLimitResetMemo, +} from "../../open-sse/services/claudeLimitReset.ts"; + +type Captured = { url: string; headers: Record }; + +const NOW_S = Math.floor(Date.now() / 1000); + +function wall429(extra: Record = {}): Response { + return new Response( + JSON.stringify({ type: "error", error: { type: "rate_limit_error", message: "usage limit" } }), + { + status: 429, + headers: { + "Content-Type": "application/json", + "anthropic-ratelimit-unified-status": "rejected", + "anthropic-ratelimit-unified-reset": String(NOW_S + 3600), + "anthropic-ratelimit-unified-representative-claim": "five_hour", + "anthropic-ratelimit-unified-slow-offer": "treatment", + "anthropic-ratelimit-unified-slow-retry-after": "20", + "anthropic-ratelimit-unified-slow-max-wait": "1200", + ...extra, + }, + } + ); +} + +function ok(headers: Record = {}): Response { + return new Response(JSON.stringify({ id: "msg_1", type: "message", content: [] }), { + status: 200, + headers: { "Content-Type": "application/json", ...headers }, + }); +} + +/** + * Sequenced fetch mock: `/v1/messages` POSTs consume `messages` in order; any other URL + * (identity bootstrap, usage/status, reset claim) is answered by `others` or an empty 200. + */ +function mockFetch( + messages: Array<() => Response>, + others: Record Response> = {} +): { calls: Captured[]; restore: () => void } { + const calls: Captured[] = []; + const original = globalThis.fetch; + let i = 0; + globalThis.fetch = (async (input: unknown, init: RequestInit = {}) => { + const url = String(input); + const headers = { ...((init.headers as Record) ?? {}) }; + if (url.includes("/v1/messages")) { + calls.push({ url, headers }); + const next = messages[Math.min(i, messages.length - 1)]; + i++; + return next(); + } + const route = Object.entries(others).find(([prefix]) => url.startsWith(prefix)); + if (route) return route[1](); + return new Response("{}", { status: 200, headers: { "Content-Type": "application/json" } }); + }) as typeof fetch; + return { calls, restore: () => void (globalThis.fetch = original) }; +} + +function lower(headers: Record): Record { + return Object.fromEntries(Object.entries(headers).map(([k, v]) => [k.toLowerCase(), v])); +} + +function run( + connectionId: string, + providerSpecificData: Record, + contextEditingEnabled = false +) { + return new DefaultExecutor("claude").execute({ + model: "claude-opus-4-8", + body: { model: "claude-opus-4-8", messages: [{ role: "user", content: "hi" }], max_tokens: 8 }, + stream: false, + credentials: { + connectionId, + accessToken: `sk-ant-oat-${connectionId}`, + providerSpecificData, + }, + clientHeaders: { "user-agent": "Cursor/1.0" }, + contextEditing: { enabled: contextEditingEnabled }, + // Combo-style: the generic 2×2s intra-URL 429 retry is skipped so the test only + // exercises the lane's own retry (which runs regardless of this flag). + skipUpstreamRetry: true, + }); +} + +test.beforeEach(() => { + _resetClaudeLowPriorityState(); + _resetClaudeLimitResetMemo(); +}); + +test("wall 429 + lowPriorityMode → same-account retry with anthropic-usage-limit: slow, 200 returned", async () => { + const { calls, restore } = mockFetch([ + () => wall429(), + () => ok({ "anthropic-ratelimit-unified-slow-status": "active" }), + ]); + try { + const result = await run("conn-lowpri", { lowPriorityMode: true }); + assert.equal(result.response.status, 200, "the intercepted 429 never reaches chatCore"); + assert.equal(calls.length, 2); + assert.equal( + lower(calls[0].headers)["anthropic-usage-limit"], + undefined, + "not sent before the wall" + ); + assert.equal(lower(calls[1].headers)["anthropic-usage-limit"], "slow", "sent on the retry"); + assert.equal(lower(calls[1].headers)["anthropic-dispatch-id"], undefined); + + // Next request on the same connection rides the lane from its first attempt. + const again = await run("conn-lowpri", { lowPriorityMode: true }); + assert.equal(again.response.status, 200); + assert.equal(calls.length, 3); + assert.equal(lower(calls[2].headers)["anthropic-usage-limit"], "slow"); + } finally { + restore(); + } +}); + +test("wall 429 without the opt-in → 429 surfaced untouched, header never sent", async () => { + const { calls, restore } = mockFetch([() => wall429(), () => ok()]); + try { + const result = await run("conn-optout", {}); + assert.equal(result.response.status, 429); + assert.equal(calls.length, 1); + assert.equal(lower(calls[0].headers)["anthropic-usage-limit"], undefined); + } finally { + restore(); + } +}); + +test("control arm offer is not accepted even with the opt-in", async () => { + const { calls, restore } = mockFetch([ + () => wall429({ "anthropic-ratelimit-unified-slow-offer": "control" }), + () => ok(), + ]); + try { + const result = await run("conn-control", { lowPriorityMode: true }); + assert.equal(result.response.status, 429); + assert.equal(calls.length, 1); + } finally { + restore(); + } +}); + +test("lane state is per connection: another account still sees its own wall", async () => { + const { calls, restore } = mockFetch([() => wall429(), () => ok(), () => wall429(), () => ok()]); + try { + await run("conn-a", { lowPriorityMode: true }); + assert.equal(calls.length, 2); + // conn-b: first attempt has no header (idle), hits the wall, accepts, retries. + const b = await run("conn-b", { lowPriorityMode: true }); + assert.equal(b.response.status, 200); + assert.equal(lower(calls[2].headers)["anthropic-usage-limit"], undefined); + assert.equal(lower(calls[3].headers)["anthropic-usage-limit"], "slow"); + } finally { + restore(); + } +}); + +test("a wall 429 that only surfaces after a 400-driven intra-attempt retry is still intercepted", async () => { + // The context-editing 400 fallback re-fetches the same URL and REPLACES `response`. + // The usage-wall check must classify that final response, otherwise the offer is missed + // and the 429 reaches chatCore, cooling the connection down — the exact opposite of + // what lowPriorityMode is for. + const badRequest = () => + new Response(JSON.stringify({ error: { message: "context_management not supported" } }), { + status: 400, + headers: { "Content-Type": "application/json" }, + }); + const { calls, restore } = mockFetch([ + badRequest, // 1st: 400 → context-editing fallback re-fetches + () => wall429(), // 2nd: the wall shows up only here + () => ok({ "anthropic-ratelimit-unified-slow-status": "active" }), // 3rd: lane retry + ]); + try { + const result = await run("conn-400-then-wall", { lowPriorityMode: true }, true); + assert.equal(result.response.status, 200, "the late wall 429 was intercepted, not surfaced"); + assert.equal(calls.length, 3); + assert.equal(lower(calls[1].headers)["anthropic-usage-limit"], undefined); + assert.equal(lower(calls[2].headers)["anthropic-usage-limit"], "slow"); + } finally { + restore(); + } +}); + +test("wall 429 + autoLimitReset → status + claim, then full-speed retry without the slow header", async () => { + const claimUrl = "https://api.anthropic.com/api/organizations/org-uuid-1/reset_rate_limits"; + const hits: string[] = []; + const { calls, restore } = mockFetch([() => wall429(), () => ok()], { + [CLAUDE_LIMIT_RESET_STATUS_URL]: () => { + hits.push("status"); + return new Response( + JSON.stringify({ + juniper_tide: { eligible: true, in_experiment: true, arm: "reset", available: true }, + }), + { status: 200, headers: { "Content-Type": "application/json" } } + ); + }, + [claimUrl]: () => { + hits.push("claim"); + return new Response(JSON.stringify({ result: "reset", next_available_at: null }), { + status: 200, + headers: { "Content-Type": "application/json" }, + }); + }, + }); + try { + const result = await run("conn-reset", { + autoLimitReset: true, + organizationUUID: "org-uuid-1", + }); + assert.equal(result.response.status, 200); + assert.deepEqual(hits, ["status", "claim"]); + assert.equal(calls.length, 2); + assert.equal( + lower(calls[1].headers)["anthropic-usage-limit"], + undefined, + "full speed, no slow lane" + ); + } finally { + restore(); + } +}); diff --git a/tests/unit/claude-low-priority-mode.test.ts b/tests/unit/claude-low-priority-mode.test.ts new file mode 100644 index 0000000000..0cd4f34d50 --- /dev/null +++ b/tests/unit/claude-low-priority-mode.test.ts @@ -0,0 +1,544 @@ +/** + * Claude OAuth lower-priority lane — pure state machine (open-sse/services/claudeLowPriority.ts). + * + * Mirrors the Claude Code 2.1.263 /low-priority contract: the lane is accepted only on a + * 5-hour wall 429 that carries `anthropic-ratelimit-unified-slow-offer: treatment`, is + * opt-in per connection, sends `anthropic-usage-limit: slow` only while active, waits on + * slot_busy/529 inside slow-max-wait, and ends on terminal verdicts / window rollover. + */ +import test from "node:test"; +import assert from "node:assert/strict"; + +import { + CLAUDE_LOW_PRIORITY_DEFAULTS, + _resetClaudeLowPriorityState, + createClaudeLowPriorityWait, + getClaudeLowPrioritySnapshot, + handleClaudeUsageLimitResponse, + isClaudeLowPriorityActive, + observeClaudeLowPriorityResponse, + parseClaudeSlowOffer, + parseClaudeSlowStatus, + readClaudeUsageLimitConfig, + resolveClaudeUsageLimitKey, + tryActivateClaudeLowPriority, +} from "../../open-sse/services/claudeLowPriority.ts"; + +const NOW = 1_800_000_000_000; // fixed clock (ms) +const RESET_AT = Math.floor(NOW / 1000) + 3600; // 5h window resets in one hour + +function wall429Headers(extra: Record = {}): Record { + return { + "anthropic-ratelimit-unified-status": "rejected", + "anthropic-ratelimit-unified-reset": String(RESET_AT), + "anthropic-ratelimit-unified-representative-claim": "five_hour", + "anthropic-ratelimit-unified-slow-offer": "treatment", + "anthropic-ratelimit-unified-slow-retry-after": "20", + "anthropic-ratelimit-unified-slow-max-wait": "1200", + ...extra, + }; +} + +const BOTH_ON = { lowPriorityMode: true, autoLimitReset: false }; +const OFF = { lowPriorityMode: false, autoLimitReset: false }; + +test.beforeEach(() => _resetClaudeLowPriorityState()); + +test("readClaudeUsageLimitConfig: both opt-ins default to off and only accept real booleans", () => { + assert.deepEqual(readClaudeUsageLimitConfig(undefined), OFF); + assert.deepEqual(readClaudeUsageLimitConfig({}), OFF); + assert.deepEqual(readClaudeUsageLimitConfig({ lowPriorityMode: "true" }), OFF); + assert.deepEqual(readClaudeUsageLimitConfig({ lowPriorityMode: true, autoLimitReset: true }), { + lowPriorityMode: true, + autoLimitReset: true, + }); +}); + +test("resolveClaudeUsageLimitKey prefers the connection id over the token", () => { + assert.equal(resolveClaudeUsageLimitKey({ connectionId: "c1", accessToken: "t" }), "c1"); + assert.equal(resolveClaudeUsageLimitKey({ accessToken: "sk-ant-oat-x" }), "sk-ant-oat-x"); + assert.equal(resolveClaudeUsageLimitKey({}), "anon"); +}); + +test("header parsers accept Headers objects and plain records (case-insensitive)", () => { + const h = new Headers({ "Anthropic-Ratelimit-Unified-Slow-Offer": "treatment" }); + assert.equal(parseClaudeSlowOffer(h), "treatment"); + assert.equal( + parseClaudeSlowOffer({ "Anthropic-Ratelimit-Unified-Slow-Offer": "control" }), + "control" + ); + assert.equal( + parseClaudeSlowOffer({ "anthropic-ratelimit-unified-slow-offer": "bogus" }), + undefined + ); + assert.equal( + parseClaudeSlowStatus({ "anthropic-ratelimit-unified-slow-status": "slot_busy" }), + "slot_busy" + ); + assert.equal( + parseClaudeSlowStatus({ "anthropic-ratelimit-unified-slow-status": "???" }), + "unrecognized" + ); + assert.equal(parseClaudeSlowStatus({}), undefined); +}); + +test("lane is idle until the first wall 429: no header before the limit is hit", () => { + assert.equal(isClaudeLowPriorityActive("c1", NOW), false); + assert.equal(getClaudeLowPrioritySnapshot("c1", NOW).active, false); +}); + +test("activation requires the treatment offer AND a future reset", () => { + assert.equal( + tryActivateClaudeLowPriority( + "c1", + wall429Headers({ "anthropic-ratelimit-unified-slow-offer": "control" }), + NOW + ), + false + ); + assert.equal( + tryActivateClaudeLowPriority("c1", { "anthropic-ratelimit-unified-status": "rejected" }, NOW), + false + ); + assert.equal( + tryActivateClaudeLowPriority( + "c1", + wall429Headers({ "anthropic-ratelimit-unified-reset": String(Math.floor(NOW / 1000) - 5) }), + NOW + ), + false + ); + assert.equal(tryActivateClaudeLowPriority("c1", wall429Headers(), NOW), true); + assert.equal(isClaudeLowPriorityActive("c1", NOW), true); + // Re-activating an active lane is a no-op. + assert.equal(tryActivateClaudeLowPriority("c1", wall429Headers(), NOW), false); + const snap = getClaudeLowPrioritySnapshot("c1", NOW); + assert.equal(snap.resetsAtSeconds, RESET_AT); + assert.equal(snap.retryAfterMs, 20_000); + assert.equal(snap.maxWaitMs, 1_200_000); +}); + +test("retry-after / max-wait hints are clamped to Claude Code's bounds", () => { + tryActivateClaudeLowPriority( + "c1", + wall429Headers({ + "anthropic-ratelimit-unified-slow-retry-after": "1", // below 5s floor + "anthropic-ratelimit-unified-slow-max-wait": "999999", // above 6h ceiling + }), + NOW + ); + const snap = getClaudeLowPrioritySnapshot("c1", NOW); + assert.equal(snap.retryAfterMs, CLAUDE_LOW_PRIORITY_DEFAULTS.retryAfterMinMs); + assert.equal(snap.maxWaitMs, CLAUDE_LOW_PRIORITY_DEFAULTS.maxWaitMaxMs); +}); + +test("lane expires lazily 60s after the announced reset", () => { + tryActivateClaudeLowPriority("c1", wall429Headers(), NOW); + const justBefore = RESET_AT * 1000 + CLAUDE_LOW_PRIORITY_DEFAULTS.resetGraceMs - 1; + assert.equal(isClaudeLowPriorityActive("c1", justBefore), true); + assert.equal(isClaudeLowPriorityActive("c1", justBefore + 1), false); +}); + +test("handle: idle + wall 429 + opt-in → accept offer and retry immediately (no cooldown)", async () => { + const wait = createClaudeLowPriorityWait(); + const decision = await handleClaudeUsageLimitResponse({ + key: "c1", + config: BOTH_ON, + response: { status: 429, headers: wall429Headers() }, + wait, + now: NOW, + }); + assert.deepEqual(decision, { kind: "retry", delayMs: 0, via: "low-priority-accepted" }); + assert.equal(isClaudeLowPriorityActive("c1", NOW), true); +}); + +test("handle: opt-out connection ignores the offer (429 flows to the normal cooldown path)", async () => { + const decision = await handleClaudeUsageLimitResponse({ + key: "c1", + config: OFF, + response: { status: 429, headers: wall429Headers() }, + wait: createClaudeLowPriorityWait(), + now: NOW, + }); + assert.deepEqual(decision, { kind: "none" }); + assert.equal(isClaudeLowPriorityActive("c1", NOW), false); +}); + +test("handle: a burst 429 without unified wall headers is not the usage wall", async () => { + const decision = await handleClaudeUsageLimitResponse({ + key: "c1", + config: BOTH_ON, + response: { status: 429, headers: { "retry-after": "3" } }, + wait: createClaudeLowPriorityWait(), + now: NOW, + }); + assert.deepEqual(decision, { kind: "none" }); +}); + +test("handle: limit-reset is tried before the slow lane and wins when it succeeds", async () => { + let claims = 0; + const decision = await handleClaudeUsageLimitResponse({ + key: "c1", + config: { lowPriorityMode: true, autoLimitReset: true }, + response: { status: 429, headers: wall429Headers() }, + wait: createClaudeLowPriorityWait(), + claimLimitReset: async () => { + claims++; + return true; + }, + now: NOW, + }); + assert.equal(claims, 1); + assert.deepEqual(decision, { kind: "retry", delayMs: 0, via: "limit-reset" }); + assert.equal(isClaudeLowPriorityActive("c1", NOW), false, "full-speed reset → no slow lane"); +}); + +test("handle: failed limit-reset falls back to the slow lane; a throwing claim is contained", async () => { + const d1 = await handleClaudeUsageLimitResponse({ + key: "c1", + config: { lowPriorityMode: true, autoLimitReset: true }, + response: { status: 429, headers: wall429Headers() }, + wait: createClaudeLowPriorityWait(), + claimLimitReset: async () => false, + now: NOW, + }); + assert.deepEqual(d1, { kind: "retry", delayMs: 0, via: "low-priority-accepted" }); + _resetClaudeLowPriorityState(); + const d2 = await handleClaudeUsageLimitResponse({ + key: "c1", + config: { lowPriorityMode: true, autoLimitReset: true }, + response: { status: 429, headers: wall429Headers() }, + wait: createClaudeLowPriorityWait(), + claimLimitReset: async () => { + throw new Error("network"); + }, + now: NOW, + }); + assert.deepEqual(d2, { kind: "retry", delayMs: 0, via: "low-priority-accepted" }); +}); + +test("handle: limit-reset is skipped when the wall blames a non-5h window", async () => { + let claims = 0; + await handleClaudeUsageLimitResponse({ + key: "c1", + config: { lowPriorityMode: false, autoLimitReset: true }, + response: { + status: 429, + headers: wall429Headers({ "anthropic-ratelimit-unified-representative-claim": "seven_day" }), + }, + wait: createClaudeLowPriorityWait(), + claimLimitReset: async () => { + claims++; + return true; + }, + now: NOW, + }); + assert.equal(claims, 0); +}); + +test("active: 2xx with slow-status active/not_needed keeps the lane and counts requests", () => { + tryActivateClaudeLowPriority("c1", wall429Headers(), NOW); + const wait = createClaudeLowPriorityWait(); + assert.deepEqual( + observeClaudeLowPriorityResponse( + "c1", + { status: 200, headers: { "anthropic-ratelimit-unified-slow-status": "active" } }, + wait, + NOW + 1000 + ), + { kind: "none" } + ); + assert.deepEqual( + observeClaudeLowPriorityResponse( + "c1", + { status: 200, headers: { "anthropic-ratelimit-unified-slow-status": "not_needed" } }, + wait, + NOW + 2000 + ), + { kind: "none" } + ); + const snap = getClaudeLowPrioritySnapshot("c1", NOW + 2000); + assert.equal(snap.active, true); + assert.equal(snap.requestsServed, 1); + assert.equal(snap.requestsServedStandard, 1); +}); + +test("active: slot_busy 429 → jittered retry-after wait, same account, until max-wait then cool-off", () => { + tryActivateClaudeLowPriority("c1", wall429Headers(), NOW); + const wait = createClaudeLowPriorityWait(); + const busy = { status: 429, headers: { "anthropic-ratelimit-unified-slow-status": "slot_busy" } }; + + const d1 = observeClaudeLowPriorityResponse("c1", busy, wait, NOW, () => 0.5); // jitter factor 1.0 + assert.deepEqual(d1, { kind: "retry", delayMs: 20_000, via: "slot-busy" }); + const d2 = observeClaudeLowPriorityResponse("c1", busy, wait, NOW + 20_000, () => 1); // +30% + assert.deepEqual(d2, { kind: "retry", delayMs: 26_000, via: "slot-busy" }); + assert.equal(wait.current?.attempts, 2); + + // Past slow-max-wait (20 min) → give up: lane ends, 10-minute cool-off blocks re-acceptance. + const late = NOW + CLAUDE_LOW_PRIORITY_DEFAULTS.maxWaitMs; + const d3 = observeClaudeLowPriorityResponse("c1", busy, wait, late); + assert.deepEqual(d3, { kind: "ended", reason: "max_wait" }); + assert.equal(isClaudeLowPriorityActive("c1", late), false); + assert.equal( + tryActivateClaudeLowPriority( + "c1", + wall429Headers({ + "anthropic-ratelimit-unified-reset": String(Math.floor(late / 1000) + 3600), + }), + late + 1000 + ), + false + ); + assert.equal( + tryActivateClaudeLowPriority( + "c1", + wall429Headers({ + "anthropic-ratelimit-unified-reset": String(Math.floor(late / 1000) + 3600), + }), + late + CLAUDE_LOW_PRIORITY_DEFAULTS.cooloffMs + ), + true + ); +}); + +test("active: 529 while the lane is active is a capacity wait, not a failure", () => { + tryActivateClaudeLowPriority("c1", wall429Headers(), NOW); + const wait = createClaudeLowPriorityWait(); + const d = observeClaudeLowPriorityResponse( + "c1", + { status: 529, headers: {} }, + wait, + NOW, + () => 0.5 + ); + assert.deepEqual(d, { kind: "retry", delayMs: 20_000, via: "capacity-busy" }); +}); + +test("active: terminal verdicts end the lane and let the 429 reach the cooldown path", () => { + for (const [status, reason] of [ + ["weekly_limit", "weekly"], + ["budget_exhausted", "budget"], + ["off", "off"], + ["ineligible", "ineligible"], + ] as const) { + _resetClaudeLowPriorityState(); + tryActivateClaudeLowPriority("c1", wall429Headers(), NOW); + const d = observeClaudeLowPriorityResponse( + "c1", + { status: 429, headers: { "anthropic-ratelimit-unified-slow-status": status } }, + createClaudeLowPriorityWait(), + NOW + ); + assert.deepEqual(d, { kind: "ended", reason }, status); + assert.equal(isClaudeLowPriorityActive("c1", NOW), false, status); + } +}); + +test("active: ineligible + overage-in-use ends as extra_usage on a 429 too, not plain ineligible", () => { + // The wall verdict normally arrives ON the 429, so the overage takeover must win over + // the generic `ineligible` mapping there — extra_usage is the state the rest of the + // codebase keys on (src/lib/providers/claudeExtraUsage.ts). + tryActivateClaudeLowPriority("c1", wall429Headers(), NOW); + const d = observeClaudeLowPriorityResponse( + "c1", + { + status: 429, + headers: { + "anthropic-ratelimit-unified-slow-status": "ineligible", + "anthropic-ratelimit-unified-overage-in-use": "true", + }, + }, + createClaudeLowPriorityWait(), + NOW + ); + assert.deepEqual(d, { kind: "ended", reason: "extra_usage" }); + assert.equal(isClaudeLowPriorityActive("c1", NOW), false); +}); + +test("active: overage-in-use false or absent keeps the plain ineligible verdict", () => { + for (const extra of [{}, { "anthropic-ratelimit-unified-overage-in-use": "false" }]) { + _resetClaudeLowPriorityState(); + tryActivateClaudeLowPriority("c1", wall429Headers(), NOW); + const d = observeClaudeLowPriorityResponse( + "c1", + { + status: 429, + headers: { "anthropic-ratelimit-unified-slow-status": "ineligible", ...extra }, + }, + createClaudeLowPriorityWait(), + NOW + ); + assert.deepEqual(d, { kind: "ended", reason: "ineligible" }); + } +}); + +test("active: waitCeilingMs caps the wait so the caller's own timeout cannot abort it mid-sleep", () => { + tryActivateClaudeLowPriority("c1", wall429Headers(), NOW); // retry-after 20s, max-wait 20min + const wait = createClaudeLowPriorityWait(); + const busy = { status: 429, headers: { "anthropic-ratelimit-unified-slow-status": "slot_busy" } }; + + // Ceiling below the server's retry-after: the sleep is clamped to what is left. + const d1 = observeClaudeLowPriorityResponse("c1", busy, wait, NOW, () => 0.5, 8_000); + assert.deepEqual(d1, { kind: "retry", delayMs: 8_000, via: "slot-busy" }); + + // Ceiling reached → graceful max_wait end (+ cool-off), never a mid-sleep abort. + const d2 = observeClaudeLowPriorityResponse("c1", busy, wait, NOW + 8_000, () => 0.5, 8_000); + assert.deepEqual(d2, { kind: "ended", reason: "max_wait" }); + assert.equal(isClaudeLowPriorityActive("c1", NOW + 8_000), false); + + // A ceiling of 0 (budget already spent) gives up immediately instead of sleeping. + _resetClaudeLowPriorityState(); + tryActivateClaudeLowPriority("c2", wall429Headers(), NOW); + const d3 = observeClaudeLowPriorityResponse( + "c2", + busy, + createClaudeLowPriorityWait(), + NOW, + () => 0.5, + 0 + ); + assert.deepEqual(d3, { kind: "ended", reason: "max_wait" }); +}); + +test("active: no ceiling keeps the server-announced max-wait semantics", () => { + tryActivateClaudeLowPriority("c1", wall429Headers(), NOW); + const wait = createClaudeLowPriorityWait(); + const busy = { status: 429, headers: { "anthropic-ratelimit-unified-slow-status": "slot_busy" } }; + assert.deepEqual( + observeClaudeLowPriorityResponse("c1", busy, wait, NOW, () => 0.5), + { + kind: "retry", + delayMs: 20_000, + via: "slot-busy", + } + ); +}); + +test("active: budget_exhausted remembers the spent budget until its announced reset", () => { + tryActivateClaudeLowPriority("c1", wall429Headers(), NOW); + const budgetReset = Math.floor(NOW / 1000) + 2 * 86_400; + observeClaudeLowPriorityResponse( + "c1", + { + status: 429, + headers: { + "anthropic-ratelimit-unified-slow-status": "budget_exhausted", + "anthropic-ratelimit-unified-slow-budget-reset": String(budgetReset), + }, + }, + createClaudeLowPriorityWait(), + NOW + ); + assert.equal(getClaudeLowPrioritySnapshot("c1", NOW).budgetSpentUntilSeconds, budgetReset); + const fresh = wall429Headers({ "anthropic-ratelimit-unified-reset": String(budgetReset + 3600) }); + assert.equal( + tryActivateClaudeLowPriority("c1", fresh, NOW + 60_000), + false, + "budget spent → no re-accept" + ); + assert.equal(tryActivateClaudeLowPriority("c1", fresh, budgetReset * 1000 + 1), true); +}); + +test("active: a 429 wall verdict without slow status (weekly window closed) ends the lane", () => { + tryActivateClaudeLowPriority("c1", wall429Headers(), NOW); + const d = observeClaudeLowPriorityResponse( + "c1", + { + status: 429, + headers: { + "anthropic-ratelimit-unified-status": "rejected", + "anthropic-ratelimit-unified-representative-claim": "seven_day", + }, + }, + createClaudeLowPriorityWait(), + NOW + ); + assert.deepEqual(d, { kind: "ended", reason: "wall" }); +}); + +test("active: a plain 429 without unified headers is left to the generic retry path", () => { + tryActivateClaudeLowPriority("c1", wall429Headers(), NOW); + const d = observeClaudeLowPriorityResponse( + "c1", + { status: 429, headers: {} }, + createClaudeLowPriorityWait(), + NOW + ); + assert.deepEqual(d, { kind: "none" }); + assert.equal(isClaudeLowPriorityActive("c1", NOW), true); +}); + +test("active: 5h window rollover announced on a 2xx ends the lane (header dropped next request)", () => { + tryActivateClaudeLowPriority("c1", wall429Headers(), NOW); + const d = observeClaudeLowPriorityResponse( + "c1", + { status: 200, headers: { "anthropic-ratelimit-unified-5h-reset": String(RESET_AT + 60) } }, + createClaudeLowPriorityWait(), + NOW + ); + assert.deepEqual(d, { kind: "ended", reason: "reset" }); + assert.equal(isClaudeLowPriorityActive("c1", NOW), false); +}); + +test("active: ineligible + overage-in-use → extra usage now covers the wall, lane ends", () => { + tryActivateClaudeLowPriority("c1", wall429Headers(), NOW); + const d = observeClaudeLowPriorityResponse( + "c1", + { + status: 200, + headers: { + "anthropic-ratelimit-unified-slow-status": "ineligible", + "anthropic-ratelimit-unified-overage-in-use": "true", + }, + }, + createClaudeLowPriorityWait(), + NOW + ); + assert.deepEqual(d, { kind: "ended", reason: "extra_usage" }); +}); + +test("raced activation: a header-less sibling's wall 429 joins the lane instead of ending it", async () => { + // Request A activated the lane; request B was built while the lane was idle (no header) + // and now comes back with the same wall 429 — it must NOT be read as a lane verdict. + tryActivateClaudeLowPriority("c1", wall429Headers(), NOW); + const wait = createClaudeLowPriorityWait(); + const d = await handleClaudeUsageLimitResponse({ + key: "c1", + config: BOTH_ON, + response: { status: 429, headers: wall429Headers() }, + wait, + sentSlow: false, + now: NOW + 10, + }); + assert.deepEqual(d, { kind: "retry", delayMs: 0, via: "low-priority-accepted" }); + assert.equal(isClaudeLowPriorityActive("c1", NOW + 10), true, "lane survives the sibling 429"); + + // A header-less 2xx while active is not lane telemetry either. + const d2 = await handleClaudeUsageLimitResponse({ + key: "c1", + config: BOTH_ON, + response: { status: 200, headers: { "anthropic-ratelimit-unified-slow-status": "active" } }, + wait, + sentSlow: false, + now: NOW + 20, + }); + assert.deepEqual(d2, { kind: "none" }); + assert.equal(getClaudeLowPrioritySnapshot("c1", NOW + 20).requestsServed, 0); + + // With the header sent (default), the same wall 429 without a slow verdict IS a wall → end. + const d3 = await handleClaudeUsageLimitResponse({ + key: "c1", + config: BOTH_ON, + response: { status: 429, headers: wall429Headers() }, + wait, + sentSlow: true, + now: NOW + 30, + }); + assert.deepEqual(d3, { kind: "ended", reason: "wall" }); +}); + +test("state is per connection", () => { + tryActivateClaudeLowPriority("c1", wall429Headers(), NOW); + assert.equal(isClaudeLowPriorityActive("c1", NOW), true); + assert.equal(isClaudeLowPriorityActive("c2", NOW), false); +});