diff --git a/CHANGELOG.md b/CHANGELOG.md index f61ead0edb..f7587c9b01 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,7 @@ --- +## [3.8.22] — TBD ## [3.8.23] — TBD --- @@ -28,6 +29,7 @@ - **Provider-detail god-component decomposition — Phase 2b (remaining shared helpers→leaf)** ([#3501]): extended `providers/[id]/providerPageHelpers.ts` with all remaining pure helpers needed by the heavy modals (`AddApiKeyModal`/`EditConnectionModal`) before they can be extracted. Moved 22 symbols: web-session credential label/hint/check/title helpers; upstream-headers helpers (`upstreamHeadersRecordsEqual`, `headerRowsToRecord`, `effectiveUpstreamHeadersForProtocol`, `anyUpstreamHeadersBadge`, `getProtoSlice`) plus their `HeaderDraftRow`/`CompatModelRow`/`CompatModelMap`/`CompatByProtocolMap` types; Codex consts and helpers (`CODEX_REASONING_STRENGTH_OPTIONS`, `CODEX_ACCOUNT_SERVICE_TIER_VALUES`, `CODEX_GLOBAL_SERVICE_MODE_VALUES`, `getCodexServiceTierLabel`, `normalizeCodexLimitPolicy`, `getCodexRequestDefaults`, `getClaudeCodeCompatibleRequestDefaults`); misc helpers (`compatProtocolLabelKey`, `extractCommandCodeCredentialInput`, `normalizeAndValidateHttpBaseUrl`, `SILICONFLOW_ENDPOINTS`, `CommandCodeAuthFlowState`). New transitive imports wired into the leaf: `MODEL_COMPAT_PROTOCOL_KEYS` (`@/shared/constants/modelCompat`), `CodexServiceTier`/`getCodexRequestDefaults`/`getClaudeCodeCompatibleRequestDefaults` (`@/lib/providers/requestDefaults`), `CodexGlobalServiceMode` (`@/lib/providers/codexFastTier`), `WebSessionCredentialRequirement` (`./webSessionCredentials`). `ProviderDetailPageClient.tsx`: 10,288 → 9,980 LOC. Leaf module: 589 LOC (acyclic). 25-assertion unit test suite passes; smoke test 3/3; no import cycles. Co-authored with @oyi77. - **Provider-detail god-component decomposition — Phase 2 (helpers→lib)** ([#3501]): extracted the pure shared helpers — `ProviderMessageTranslator`/`LocalProviderMetadata` types, `providerText`/`providerCountText`/`readBooleanToggle`, and the provider base-URL + routing-tag/excluded-model parse/format block — into a new leaf `providers/[id]/providerPageHelpers.ts` (imports only `@/shared`, so the client and modals share them with no import cycle). `ProviderDetailPageClient.tsx`: 10,435 → 10,288 LOC. Unblocks extracting the heavier `AddApiKeyModal`/`EditConnectionModal` (which depend on these helpers) without cycling. The Phase 0 smoke test caught a missing transitive import (`isSelfHostedChatProvider`) at mount — now wired + locked by a new helpers unit test (12 assertions). Co-authored with @oyi77. +- **#3500 fully resolved** — Hard Rule #5 (no raw SQL in route handlers): all 13 internal offenders migrated to `src/lib/db/` modules across slices (call_logs, usage_history/daily_usage_summary, community_servers, usage_logs, semantic_cache, proxy_logs, skills UPDATE, db-backups). The gate's `KNOWN_RAW_SQL` set is renamed to `EXTERNAL_DB_ALLOWED` (with a back-compat alias) and now holds only the **2 external-DB reads** (`oauth/cursor/auto-import`, `oauth/kiro/auto-import`) — these open *another app's* SQLite to import credentials, so by design they cannot live in OmniRoute's `db/` domain. The gate still blocks any NEW raw SQL against OmniRoute's DB. - **#3500 fully resolved** — Hard Rule #5 (no raw SQL in route handlers): all 13 internal offenders migrated to `src/lib/db/` modules across slices (call*logs, usage_history/daily_usage_summary, community_servers, usage_logs, semantic_cache, proxy_logs, skills UPDATE, db-backups). The gate's `KNOWN_RAW_SQL` set is renamed to `EXTERNAL_DB_ALLOWED` (with a back-compat alias) and now holds only the **2 external-DB reads** (`oauth/cursor/auto-import`, `oauth/kiro/auto-import`) — these open \_another app's* SQLite to import credentials, so by design they cannot live in OmniRoute's `db/` domain. The gate still blocks any NEW raw SQL against OmniRoute's DB. - **chore(db-gate):** reclassify `KNOWN_UNEXPORTED` → `INTENTIONALLY_INTERNAL` in `scripts/check/check-db-rules.mjs` ([#3499]): a full audit of all 25 db modules confirmed each is consumed via direct/dynamic import per Hard Rule #2 ("Never barrel-import from localDb.ts"). The old framing labelled them as "debt", which was misleading — they are the correct pattern. The gate's blocking behaviour is unchanged (a NEW unexported module still fails); only the name, comments, and per-module justifications were updated to reflect audited truth. Four modules flagged `DEAD?` (`compressionScheduler`, `discovery`, `pluginMetrics`, `prompts`) have zero production importers and are documented as schema-reserved. A new regression-guard test (`tests/unit/check-db-rules-classification.test.ts`) asserts every non-dead module in the set has ≥1 real importer, so a future consumer removal surfaces as a test failure requiring explicit reclassification. - **refactor(db): move `call_logs` aggregations into `callLogStats` db module** ([#3500]): extracted raw SQL from three route handlers (`/api/provider-metrics`, `/api/search/stats`, `/api/v1/search/analytics`) into a new `src/lib/db/callLogStats.ts` domain module (`getProviderMetrics`, `getSearchProviderStats`, `getRecentSearchLogs`, `getSearchAggregateStats`, `getSearchProviderCounts`). First slice of #3500 (call_logs cluster). Behavior unchanged; the three routes are removed from `KNOWN_RAW_SQL` in the gate. Validated with TDD unit tests (6 assertions seeding an in-memory SQLite fixture). @@ -35,6 +37,7 @@ - **refactor(db): move `community_servers` auth look-up into `gamification` db module** ([#3500]): extracted raw SQL from two federation route handlers (`/api/gamification/federation/leaderboard`, `/api/gamification/federation/score`) into a new `getConnectedServerByKeyHash(apiKeyHash)` function in `src/lib/db/gamification.ts`. Third slice of #3500 (gamification federation cluster). Behavior unchanged; the two routes are removed from `KNOWN_RAW_SQL` in the gate. Validated with TDD unit tests (3 assertions seeding a temp SQLite fixture). +- **refactor(db): move `skills UPDATE` + `db-backups` SQL into db modules** ([#3500]): fifth slice of #3500. Extracted the dynamic `UPDATE skills SET …` from `src/app/api/skills/[id]/route.ts` into a new `src/lib/db/skills.ts` module (`updateSkill(id, patch)`). The dynamic SET clause is injection-safe: column names are validated against a hard-coded allowlist of known writable columns before being interpolated; unknown keys are silently ignored. Extended `src/lib/db/backup.ts` with three new functions: `exportAllSummaryRows()` (multi-table SELECT for key_value / combos / provider_connections / api_keys, used by exportAll), `getTableNamesFromAdapter()` (sqlite_master introspection via an adapter arg, used by import validation), and `countImportedRows()` (post-import COUNT(*) per table). The backup domain module is the correct home for sqlite_master introspection — it is not "raw SQL in a route" once moved there. `KNOWN_RAW_SQL` drops by 3 (from 8 → 5). Validated with 11 TDD unit tests (`tests/unit/db-backups-skills-3500.test.ts`). - **refactor(db): move `skills UPDATE` + `db-backups` SQL into db modules** ([#3500]): fifth slice of #3500. Extracted the dynamic `UPDATE skills SET …` from `src/app/api/skills/[id]/route.ts` into a new `src/lib/db/skills.ts` module (`updateSkill(id, patch)`). The dynamic SET clause is injection-safe: column names are validated against a hard-coded allowlist of known writable columns before being interpolated; unknown keys are silently ignored. Extended `src/lib/db/backup.ts` with three new functions: `exportAllSummaryRows()` (multi-table SELECT for key_value / combos / provider_connections / api_keys, used by exportAll), `getTableNamesFromAdapter()` (sqlite_master introspection via an adapter arg, used by import validation), and `countImportedRows()` (post-import COUNT(\*) per table). The backup domain module is the correct home for sqlite_master introspection — it is not "raw SQL in a route" once moved there. `KNOWN_RAW_SQL` drops by 3 (from 8 → 5). Validated with 11 TDD unit tests (`tests/unit/db-backups-skills-3500.test.ts`). - **refactor(db): move `usage_logs`/`semantic_cache`/`proxy_logs` SQL into db modules** ([#3500]): extracted raw `db.prepare(...)` SQL from three route handlers (`/api/analytics/auto-routing` → `usageLogs.ts`; `/api/cache/entries` → `semanticCache.ts`; `/api/logs/export` → `proxyLogs.ts`) into new `src/lib/db/` domain modules. New exports: `getAutoRoutingTotalCount`, `getAutoRoutingVariantBreakdown`, `getAutoRoutingTopProviders` (usage_logs), `listSemanticCacheEntries`, `deleteSemanticCacheBySignature`, `deleteSemanticCacheByModel` (semantic_cache), and `exportProxyLogsSince` (proxy_logs). Fourth slice of #3500. `KNOWN_RAW_SQL` drops from 8 → 5. Validated with 13 TDD unit tests (`tests/unit/db-logs-cache-3500.test.ts`) seeding temp SQLite fixtures. diff --git a/open-sse/config/geminiRateLimits.json b/open-sse/config/geminiRateLimits.json new file mode 100644 index 0000000000..a1ae15dc7f --- /dev/null +++ b/open-sse/config/geminiRateLimits.json @@ -0,0 +1,34 @@ +{ + "gemini-2.5-flash": { "rpm": 5, "rpd": 20, "tpm": 250000 }, + "gemini-2.5-pro": { "rpm": 0, "rpd": 0, "tpm": 0 }, + "gemini-2-flash": { "rpm": 0, "rpd": 0, "tpm": 0 }, + "gemini-2-flash-lite": { "rpm": 0, "rpd": 0, "tpm": 0 }, + "gemini-2.5-flash-tts": { "rpm": 3, "rpd": 10, "tpm": 10000 }, + "gemini-2.5-pro-tts": { "rpm": 0, "rpd": 0, "tpm": 0 }, + "imagen-4-generate": { "rpm": -1, "rpd": 25, "tpm": -1 }, + "imagen-4-ultra-generate": { "rpm": -1, "rpd": 25, "tpm": -1 }, + "imagen-4-fast-generate": { "rpm": -1, "rpd": 25, "tpm": -1 }, + "gemma-4-26b-it": { "rpm": 15, "rpd": 1500, "tpm": -1 }, + "gemma-4-31b-it": { "rpm": 15, "rpd": 1500, "tpm": -1 }, + "gemini-embedding-exp-03-07": { "rpm": 100, "rpd": 1000, "tpm": 30000 }, + "gemini-3.5-flash": { "rpm": 5, "rpd": 20, "tpm": 250000 }, + "gemini-3.1-flash-lite": { "rpm": 15, "rpd": 500, "tpm": 250000 }, + "gemini-3.1-pro": { "rpm": 0, "rpd": 0, "tpm": 0 }, + "gemini-2.5-flash-lite": { "rpm": 10, "rpd": 20, "tpm": 250000 }, + "nano-banana-gemini-2.5-flash-preview-image": { "rpm": 0, "rpd": 0, "tpm": 0 }, + "nano-banana-pro-gemini-3-pro-image": { "rpm": 0, "rpd": 0, "tpm": 0 }, + "nano-banana-2-gemini-3.1-flash-image": { "rpm": 0, "rpd": 0, "tpm": 0 }, + "lyria-3-clip": { "rpm": 0, "rpd": 0, "tpm": 0 }, + "lyria-3-pro": { "rpm": 0, "rpd": 0, "tpm": 0 }, + "veo-3-generate": { "rpm": 0, "rpd": 0, "tpm": -1 }, + "veo-3-fast-generate": { "rpm": 0, "rpd": 0, "tpm": -1 }, + "veo-3-lite-generate": { "rpm": 0, "rpd": 0, "tpm": -1 }, + "gemini-3.1-flash-tts": { "rpm": 3, "rpd": 10, "tpm": 10000 }, + "gemini-robotics-er-1.5-preview": { "rpm": 10, "rpd": 20, "tpm": 250000 }, + "gemini-robotics-er-1.6-preview": { "rpm": 5, "rpd": 20, "tpm": 250000 }, + "computer-use-preview": { "rpm": 0, "rpd": 0, "tpm": 0 }, + "gemini-embedding-exp-04-07": { "rpm": 100, "rpd": 1000, "tpm": 30000 }, + "gemini-3.5-live-translate": { "rpm": -1, "rpd": -1, "tpm": 20000 }, + "gemini-2.5-flash-native-audio-dialog": { "rpm": -1, "rpd": -1, "tpm": 1000000 }, + "gemini-3-flash-live": { "rpm": -1, "rpd": -1, "tpm": 65000 } +} diff --git a/open-sse/handlers/chatCore.ts b/open-sse/handlers/chatCore.ts index 873bb3fd82..c53209cafb 100644 --- a/open-sse/handlers/chatCore.ts +++ b/open-sse/handlers/chatCore.ts @@ -233,6 +233,7 @@ import { getModelScopeRetryDelayMs, isModelScopeProvider, } from "../services/modelscopePolicy.ts"; +import { incrementRequestCount } from "../services/geminiRateLimitTracker.ts"; const MEMORY_EXTRACTION_TEXT_LIMIT = 64 * 1024; @@ -3797,6 +3798,12 @@ export async function handleChatCore({ }); const res = normalizeExecutorResult(rawExecutorResult); trace("post_executor", { status: res?.response?.status }); + + // Track Gemini RPM + RPD request counts for 429 classification + if (provider === "gemini") { + incrementRequestCount(modelToCall); + } + updatePendingRequest(model, provider, connectionId, { stage: "provider_response_started", }); diff --git a/open-sse/services/accountFallback.ts b/open-sse/services/accountFallback.ts index ad0b2a344e..20dac5e01e 100644 --- a/open-sse/services/accountFallback.ts +++ b/open-sse/services/accountFallback.ts @@ -28,6 +28,7 @@ import { type FailureKind, } from "../../src/shared/utils/classify429"; import { resolveUseUpstream429BreakerHints } from "../../src/shared/utils/providerHints"; +import { isRpdExhausted, isRpmExhausted } from "./geminiRateLimitTracker.ts"; export type ProviderProfile = { baseCooldownMs: number; @@ -150,9 +151,6 @@ export const CREDITS_EXHAUSTED_SIGNALS = [ "credits exhausted", "out of credits", "payment required", - "resource has been exhausted", - "resource_exhausted", - "check quota", "free tier of the model has been exhausted", ]; @@ -933,8 +931,9 @@ export function parseRetryFromErrorText(errorText: unknown): number | null { // 2026-05-17T10:00:00Z" or "Please wait until 2026-05-17T10:00:00.000Z"). // Convert to a future-duration in milliseconds if it parses. const isoMatch = - /\b(?:try again at|wait until|reset(?:s)? at|available at|retry after)\s+(\d{4}-\d{2}-\d{2}[Tt ]\d{2}:\d{2}(?::\d{2})?(?:\.\d+)?(?:Z|[+-]\d{2}:?\d{2})?)/i - .exec(msg); + /\b(?:try again at|wait until|reset(?:s)? at|available at|retry after)\s+(\d{4}-\d{2}-\d{2}[Tt ]\d{2}:\d{2}(?::\d{2})?(?:\.\d+)?(?:Z|[+-]\d{2}:?\d{2})?)/i.exec( + msg + ); if (isoMatch) { const parsedTs = Date.parse(isoMatch[1]); if (Number.isFinite(parsedTs)) { @@ -1021,10 +1020,8 @@ export function classifyErrorText(errorText: unknown): RateLimitReasonValue { const configuredRule = matchErrorRuleByText(errorText); if (configuredRule?.reason) return configuredRule.reason; if (lower.includes("rate_limit")) return RateLimitReason.RATE_LIMIT_EXCEEDED; - if ( - lower.includes("resource exhausted") || - lower.includes("high demand") - ) return RateLimitReason.MODEL_CAPACITY; + if (lower.includes("resource exhausted") || lower.includes("high demand")) + return RateLimitReason.MODEL_CAPACITY; if ( lower.includes("unauthorized") || lower.includes("invalid api key") || @@ -1414,6 +1411,19 @@ export function checkFallbackError( } } + // Gemini-specific: use known published RPM/RPD limits to distinguish 429 types. + // Gemini returns the same error body for both, so we use per-model request + // counters to decide: if daily count >= RPD → quota_exhausted (midnight lockout); + // if minute count >= RPM → rate_limit_exceeded (exponential backoff). + if (provider === "gemini" && status === HTTP_STATUS.RATE_LIMITED && _model) { + if (isRpdExhausted(_model)) { + return buildRetryableFallback(RateLimitReason.QUOTA_EXHAUSTED); + } + if (isRpmExhausted(_model)) { + return buildRetryableFallback(RateLimitReason.RATE_LIMIT_EXCEEDED); + } + } + const configuredRule = isRateLimitStatus && !preserveQuota429 ? matchErrorRuleByStatus(status) diff --git a/open-sse/services/geminiRateLimitTracker.ts b/open-sse/services/geminiRateLimitTracker.ts new file mode 100644 index 0000000000..7ede8a4f88 --- /dev/null +++ b/open-sse/services/geminiRateLimitTracker.ts @@ -0,0 +1,145 @@ +/** + * In-memory request counters for Gemini models — tracks both RPD (daily) + * and RPM (sliding 60s window) so that 429 responses can be classified + * as either quota_exhausted (RPD hit) or rate_limit_exceeded (RPM hit). + * + * Gemini returns identical error bodies for both types, so we rely on + * published per-model limits from geminiRateLimits.json to distinguish them. + * + * Counters are incremented on every Gemini request so that once usage + * reaches the published limit, subsequent 429s are correctly classified. + */ + +import geminiLimits from "../config/geminiRateLimits.json"; + +// ── RPD (daily) state ──────────────────────────────────────────────────────── + +interface DailyCount { + date: string; // "YYYY-MM-DD" + count: number; +} + +const dailyCounts = new Map(); + +// ── RPM (sliding 60s window) state ─────────────────────────────────────────── + +const minuteWindows = new Map(); + +// ── Helpers ────────────────────────────────────────────────────────────────── + +function toDateKey(): string { + return new Date().toISOString().slice(0, 10); +} + +function stripModelPrefix(modelId: string): string { + // Only strip the "gemini/" provider prefix, never "gemini-" which is part + // of the actual model name (e.g. "gemini-2.5-flash", "gemini-3.5-live-translate"). + return modelId.replace(/^gemini\//, "").trim(); +} + +function lookupValue(modelId: string, field: "rpm" | "rpd"): number { + if (!modelId) return 0; + const key = stripModelPrefix(modelId); + const entry = (geminiLimits as Record>)[key]; + if (!entry) { + for (const [knownKey, knownEntry] of Object.entries(geminiLimits)) { + if (key.endsWith(knownKey) || knownKey.endsWith(key)) { + const val = knownEntry[field]; + return typeof val === "number" && val > 0 ? val : 0; + } + } + return 0; + } + const val = entry[field]; + return typeof val === "number" && val > 0 ? val : 0; +} + +// ── RPD exports ────────────────────────────────────────────────────────────── + +export function getModelRpd(modelId: string): number { + return lookupValue(modelId, "rpd"); +} + +export function incrementDailyRequestCount(modelId: string): void { + if (!modelId) return; + const key = stripModelPrefix(modelId); + const today = toDateKey(); + const existing = dailyCounts.get(key); + if (existing && existing.date === today) { + existing.count++; + } else { + dailyCounts.set(key, { date: today, count: 1 }); + } +} + +export function getDailyRequestCount(modelId: string): number { + if (!modelId) return 0; + const key = stripModelPrefix(modelId); + const today = toDateKey(); + const entry = dailyCounts.get(key); + if (entry && entry.date === today) return entry.count; + return 0; +} + +export function isRpdExhausted(modelId: string): boolean { + const rpd = getModelRpd(modelId); + if (rpd <= 0) return false; + return getDailyRequestCount(modelId) >= rpd; +} + +// ── RPM exports ────────────────────────────────────────────────────────────── + +export function getModelRpm(modelId: string): number { + return lookupValue(modelId, "rpm"); +} + +/** Prune timestamps older than 60 seconds from a model's window. */ +function pruneMinuteWindow(key: string): void { + const now = Date.now(); + const cutoff = now - 60_000; + const timestamps = minuteWindows.get(key); + if (!timestamps) return; + // Keep only timestamps >= cutoff + let i = 0; + while (i < timestamps.length && timestamps[i] < cutoff) i++; + if (i > 0) { + minuteWindows.set(key, timestamps.slice(i)); + } +} + +export function incrementMinuteRequestCount(modelId: string): void { + if (!modelId) return; + const key = stripModelPrefix(modelId); + pruneMinuteWindow(key); + const timestamps = minuteWindows.get(key) ?? []; + timestamps.push(Date.now()); + minuteWindows.set(key, timestamps); +} + +export function getMinuteRequestCount(modelId: string): number { + if (!modelId) return 0; + const key = stripModelPrefix(modelId); + pruneMinuteWindow(key); + return minuteWindows.get(key)?.length ?? 0; +} + +export function isRpmExhausted(modelId: string): boolean { + const rpm = getModelRpm(modelId); + if (rpm <= 0) return false; + return getMinuteRequestCount(modelId) >= rpm; +} + +// ── Increment both (convenience) ───────────────────────────────────────────── + +/** Increment both daily and minute counters for a Gemini request. */ +export function incrementRequestCount(modelId: string): void { + incrementDailyRequestCount(modelId); + incrementMinuteRequestCount(modelId); +} + +// ── Reset (testing) ────────────────────────────────────────────────────────── + +export function resetCounters(): void { + dailyCounts.clear(); + minuteWindows.clear(); +} diff --git a/open-sse/tsconfig.json b/open-sse/tsconfig.json index 6a35a3d3e1..f64585be7e 100644 --- a/open-sse/tsconfig.json +++ b/open-sse/tsconfig.json @@ -7,6 +7,7 @@ "checkJs": true, "noEmit": true, "allowImportingTsExtensions": true, + "resolveJsonModule": true, "skipLibCheck": true, "esModuleInterop": true, "strict": false, diff --git a/src/sse/services/auth.ts b/src/sse/services/auth.ts index a5fce790cb..515f605a19 100644 --- a/src/sse/services/auth.ts +++ b/src/sse/services/auth.ts @@ -31,7 +31,7 @@ import { recordModelLockoutFailure, } from "@omniroute/open-sse/services/accountFallback.ts"; import { isLocalProvider } from "@omniroute/open-sse/config/providerRegistry.ts"; -import { COOLDOWN_MS } from "@omniroute/open-sse/config/constants.ts"; +import { COOLDOWN_MS, RateLimitReason } from "@omniroute/open-sse/config/constants.ts"; import { preflightQuota, isQuotaPreflightEnabled, @@ -42,7 +42,7 @@ import { classifyProviderError, PROVIDER_ERROR_TYPES, } from "@omniroute/open-sse/services/errorClassifier.ts"; -import { looksLikeQuotaExhausted } from "@/shared/utils/classify429"; + import { getCodexModelScope } from "@omniroute/open-sse/executors/codex.ts"; import { getProviderById, @@ -1799,7 +1799,7 @@ export async function markAccountUnavailable( const reason = status === 404 ? "not_found" - : status === 429 && looksLikeQuotaExhausted(errorText) + : status === 429 && fallbackResult.reason === RateLimitReason.QUOTA_EXHAUSTED ? "quota_exhausted" : status === 429 ? "rate_limited" diff --git a/tests/integration/gemini-live-429-classification.test.ts b/tests/integration/gemini-live-429-classification.test.ts new file mode 100644 index 0000000000..f8635c79ac --- /dev/null +++ b/tests/integration/gemini-live-429-classification.test.ts @@ -0,0 +1,152 @@ +/** + * Gemini 429 classification integration tests. + * + * Tests the end-to-end classification path for Gemini rate-limit errors + * through OmniRoute. Sends bursts of requests to try to trigger published + * RPM/RPD limits, then verifies the classification is correct. + * + * The tests are "best effort" — if rate limits aren't triggered (Gemini + * may be more generous in practice), the test logs a warning and passes + * rather than failing. The unit tests in account-fallback-service.test.ts + * provide the definitive coverage of classification logic. + * + * Env vars: + * OMNIROUTE_URL — base URL (default http://localhost:20128) + * OMNIROUTE_API_KEY — API key for auth (REQUIRED) + * TEST_GEMINI_RPM_MODEL — RPM model (default gemini/gemma-4-31b-it) + * TEST_GEMINI_RPD_MODEL — RPD model (default gemini/gemini-2.5-flash) + */ + +import test from "node:test"; +import assert from "node:assert/strict"; + +const API_KEY = process.env.OMNIROUTE_API_KEY; +const BASE_URL = process.env.OMNIROUTE_URL || "http://localhost:20128"; +const RPM_MODEL = process.env.TEST_GEMINI_RPM_MODEL || "gemini/gemma-4-31b-it"; +const RPD_MODEL = process.env.TEST_GEMINI_RPD_MODEL || "gemini/gemini-2.5-flash"; + +const skip = !API_KEY ? "OMNIROUTE_API_KEY not set — skipping live test" : undefined; + +async function chat(model: string, content: string) { + const res = await fetch(`${BASE_URL}/api/v1/chat/completions`, { + method: "POST", + headers: { "Content-Type": "application/json", Authorization: `Bearer ${API_KEY}` }, + body: JSON.stringify({ model, stream: false, messages: [{ role: "user", content }] }), + }); + return { status: res.status, body: await res.text() }; +} + +// ── Test 1: RPM burst ──────────────────────────────────────────────────────── + +test( + "Gemma 4 RPM burst: try to hit 15 RPM, verify 429 classification if triggered", + { skip }, + async () => { + const BURST = 30; + console.error(`\n[RPM] Sending ${BURST} concurrent requests to ${RPM_MODEL} (15 RPM)...`); + const fetches = Array.from({ length: BURST }, (_, i) => + chat(RPM_MODEL, `Count to 3. Only numbers. Request ${i}.`) + ); + const results = await Promise.all(fetches); + + const statuses = results.map((r) => r.status); + const successes = results.filter((r) => r.status === 200); + const rateLimited = results.filter((r) => r.status === 429); + + console.error( + `[RPM] ${successes.length} success, ${rateLimited.length} 429 (statuses: ${statuses.join(",")})` + ); + + assert.ok(successes.length > 0, "expected at least one successful request"); + + if (rateLimited.length > 0) { + for (const r of rateLimited) { + assert.equal( + r.body.includes("quota_exhausted"), + false, + `RPM 429 should NOT be quota_exhausted: ${r.body.slice(0, 300)}` + ); + assert.ok( + r.body.includes("cooling down") || r.body.includes("rate_limit"), + `RPM 429 should mention cooldown: ${r.body.slice(0, 300)}` + ); + } + } else { + console.error("[RPM] No 429s received (Gemini may have higher effective RPM for this key)"); + console.error( + "[RPM] Classification logic verified by unit tests in account-fallback-service.test.ts" + ); + } + } +); + +test("Gemma 4 RPM recovery: after 65s, requests should succeed again", { skip }, async () => { + // First send a burst to ensure any cooldown from the previous test has cleared + const warmup = await chat(RPM_MODEL, "ping"); + if (warmup.status === 429) { + console.error("[RPM recovery] Previous test left model in cooldown, waiting 65s..."); + await new Promise((r) => setTimeout(r, 65_000)); + } else { + console.error("[RPM recovery] Model is healthy, skipping wait"); + } + + const results: Array<{ status: number }> = []; + for (let i = 0; i < 3; i++) { + results.push(await chat(RPM_MODEL, `Hello ${i}.`)); + await new Promise((r) => setTimeout(r, 500)); + } + + const successes = results.filter((r) => r.status === 200); + console.error(`[RPM recovery] ${successes.length}/3 success`); + assert.ok( + successes.length >= 1, + `expected at least 1 recovery, got: ${results.map((r) => r.status).join(",")}` + ); +}); + +// ── Test 2: RPD burst ──────────────────────────────────────────────────────── + +test( + "Gemini 2.5 Flash RPD burst: try to hit 20 RPD, verify quota_exhausted if triggered", + { skip }, + async () => { + const BURST = 30; + console.error(`\n[RPD] Sending ${BURST} concurrent requests to ${RPD_MODEL} (20 RPD)...`); + const fetches = Array.from({ length: BURST }, (_, i) => + chat(RPD_MODEL, `Count to 5. Only numbers. Request ${i}.`) + ); + const results = await Promise.all(fetches); + const statuses = results.map((r) => r.status); + const successes = results.filter((r) => r.status === 200); + const rateLimited = results.filter((r) => r.status === 429); + + console.error( + `[RPD] ${successes.length} success, ${rateLimited.length} 429 (statuses: ${statuses.join(",")})` + ); + + assert.ok(successes.length > 0, "expected at least one successful request"); + + const quotaExhausted = rateLimited.filter((r) => + r.body.toLowerCase().includes("quota_exhausted") + ); + + if (quotaExhausted.length > 0) { + console.error(`[RPD] ${quotaExhausted.length} quota_exhausted responses ✓`); + } else if (rateLimited.length > 0) { + // Check that non-quota-exhausted 429s are still rate_limit, not some other error + for (const r of rateLimited) { + assert.equal( + r.body.includes("quota_exhausted"), + false, + `RPM 429 should not be quota_exhausted: ${r.body.slice(0, 200)}` + ); + } + console.error("[RPD] 429s present but none are quota_exhausted (RPD not yet hit)"); + } else { + console.error( + "[RPD] No 429s received (daily quota may not have been reached, or limits are higher)" + ); + console.error("[RPD] Classification logic verified by unit tests"); + } + } +); diff --git a/tests/integration/gemini-rate-limit-classification.test.ts b/tests/integration/gemini-rate-limit-classification.test.ts new file mode 100644 index 0000000000..bbeb7cbb6e --- /dev/null +++ b/tests/integration/gemini-rate-limit-classification.test.ts @@ -0,0 +1,253 @@ +/** + * Gemini rate-limit classification integration tests. + * + * Tests the full integration between geminiRateLimitTracker (in-memory + * daily/minute counters) and accountFallback.checkFallbackError (429 + * classification). No live Gemini API key needed — the tracker counters + * are incremented directly and a synthetic 429 error is passed to + * checkFallbackError. + * + * This validates that the whole pipeline works: + * incrementRequestCount → isRpdExhausted / isRpmExhausted → checkFallbackError + * + * Covers three classification outcomes: + * - RPM exhausted → RATE_LIMIT_EXCEEDED (exponential backoff) + * - RPD exhausted → QUOTA_EXHAUSTED (midnight lockout) + * - Neither exhausted → falls through to generic 429 (RATE_LIMIT_EXCEEDED) + */ + +import test from "node:test"; +import assert from "node:assert/strict"; + +const { checkFallbackError } = await import("../../open-sse/services/accountFallback.ts"); +const { RateLimitReason } = await import("../../open-sse/config/constants.ts"); +const { + incrementRequestCount, + getDailyRequestCount, + getMinuteRequestCount, + isRpdExhausted, + isRpmExhausted, + resetCounters, +} = await import("../../open-sse/services/geminiRateLimitTracker.ts"); + +const PROFILE = { + baseCooldownMs: 125, + useUpstreamRetryHints: false, + maxBackoffSteps: 3, + failureThreshold: 60, + degradationThreshold: 40, + resetTimeoutMs: 5000, + transientCooldown: 125, + rateLimitCooldown: 125, + maxBackoffLevel: 3, + circuitBreakerThreshold: 60, + circuitBreakerReset: 5000, + providerFailureThreshold: 5, + providerFailureWindowMs: 300000, + providerCooldownMs: 60000, +}; + +const GEMINI_429_BODY = "Resource has been exhausted (e.g. check quota)."; + +test.beforeEach(() => { + resetCounters(); +}); + +// ── Scenario 1: RPM exhausted, RPD not exhausted → RATE_LIMIT_EXCEEDED ──────── + +test("Gemini 2.5 Flash 5 RPM hit: 429 classifies as RATE_LIMIT_EXCEEDED (not QUOTA_EXHAUSTED)", () => { + // gemini-2.5-flash: RPM=5, RPD=20 + for (let i = 0; i < 5; i++) incrementRequestCount("gemini-2.5-flash"); + assert.equal(isRpmExhausted("gemini-2.5-flash"), true); + assert.equal(isRpdExhausted("gemini-2.5-flash"), false); + + const result = checkFallbackError( + 429, + GEMINI_429_BODY, + 0, + "gemini-2.5-flash", + "gemini", + null, + PROFILE + ); + + assert.equal(result.shouldFallback, true); + assert.equal(result.reason, RateLimitReason.RATE_LIMIT_EXCEEDED); + assert.ok(result.cooldownMs > 0, "cooldownMs should be positive"); +}); + +// ── Scenario 2: RPD exhausted → QUOTA_EXHAUSTED ─────────────────────────────── + +test("Gemini 2.5 Flash 20 RPD hit: 429 classifies as QUOTA_EXHAUSTED", () => { + for (let i = 0; i < 20; i++) incrementRequestCount("gemini-2.5-flash"); + assert.equal(isRpdExhausted("gemini-2.5-flash"), true); + + const result = checkFallbackError( + 429, + GEMINI_429_BODY, + 0, + "gemini-2.5-flash", + "gemini", + null, + PROFILE + ); + + assert.equal(result.shouldFallback, true); + assert.equal(result.reason, RateLimitReason.QUOTA_EXHAUSTED); + assert.ok(result.cooldownMs > 0, "cooldownMs should be positive"); +}); + +// ── Scenario 3: Neither RPM nor RPD exhausted → falls through to generic 429 ── + +test("Gemini 2.5 Flash 3 requests (below both): 429 falls through to generic RATE_LIMIT_EXCEEDED", () => { + for (let i = 0; i < 3; i++) incrementRequestCount("gemini-2.5-flash"); + assert.equal(isRpmExhausted("gemini-2.5-flash"), false); + assert.equal(isRpdExhausted("gemini-2.5-flash"), false); + + const result = checkFallbackError( + 429, + GEMINI_429_BODY, + 0, + "gemini-2.5-flash", + "gemini", + null, + PROFILE + ); + + assert.equal(result.shouldFallback, true); + assert.equal(result.reason, RateLimitReason.RATE_LIMIT_EXCEEDED); + assert.ok(result.cooldownMs > 0, "cooldownMs should be positive"); +}); + +// ── Scenario 4: Both limits exhausted → RPD takes priority → QUOTA_EXHAUSTED ── + +test("Gemini 2.5 Flash both RPM and RPD hit: RPD check runs first → QUOTA_EXHAUSTED", () => { + for (let i = 0; i < 25; i++) incrementRequestCount("gemini-2.5-flash"); + assert.equal(isRpmExhausted("gemini-2.5-flash"), true); + assert.equal(isRpdExhausted("gemini-2.5-flash"), true); + + const result = checkFallbackError( + 429, + GEMINI_429_BODY, + 0, + "gemini-2.5-flash", + "gemini", + null, + PROFILE + ); + + // RPD check is first in the if-chain, so it takes priority + assert.equal(result.reason, RateLimitReason.QUOTA_EXHAUSTED); +}); + +// ── Scenario 5: Gemma 4 — 15 RPM hit, 1500 RPD not hit → RATE_LIMIT_EXCEEDED ─ + +test("Gemma 4 15 RPM hit (RPD=1500 untouched): 429 classifies as RATE_LIMIT_EXCEEDED", () => { + for (let i = 0; i < 15; i++) incrementRequestCount("gemini/gemma-4-31b-it"); + assert.equal(isRpmExhausted("gemini/gemma-4-31b-it"), true); + assert.equal(isRpdExhausted("gemini/gemma-4-31b-it"), false); + + const result = checkFallbackError( + 429, + GEMINI_429_BODY, + 0, + "gemini/gemma-4-31b-it", + "gemini", + null, + PROFILE + ); + + assert.equal(result.reason, RateLimitReason.RATE_LIMIT_EXCEEDED); +}); + +// ── Scenario 6: Non-Gemini provider bypasses the Gemini-specific check ───────── + +test("Non-Gemini provider: tracker state is irrelevant, 429 goes through generic path", () => { + for (let i = 0; i < 30; i++) incrementRequestCount("gemini-2.5-flash"); + assert.equal(isRpdExhausted("gemini-2.5-flash"), true); + + // Provider is "openai" — Gemini-specific check is skipped + const result = checkFallbackError( + 429, + GEMINI_429_BODY, + 0, + "gemini-2.5-flash", + "openai", + null, + PROFILE + ); + + // Falls through to generic 429 handling → RATE_LIMIT_EXCEEDED + assert.equal(result.shouldFallback, true); + assert.equal(result.reason, RateLimitReason.RATE_LIMIT_EXCEEDED); +}); + +// ── Scenario 7: Reset clears state → no longer exhausted ────────────────────── + +test("resetCounters clears both RPM and RPD exhaustion", () => { + for (let i = 0; i < 20; i++) incrementRequestCount("gemini-2.5-flash"); + assert.equal(isRpmExhausted("gemini-2.5-flash"), true); + assert.equal(isRpdExhausted("gemini-2.5-flash"), true); + + resetCounters(); + + assert.equal(getDailyRequestCount("gemini-2.5-flash"), 0); + assert.equal(getMinuteRequestCount("gemini-2.5-flash"), 0); + assert.equal(isRpmExhausted("gemini-2.5-flash"), false); + assert.equal(isRpdExhausted("gemini-2.5-flash"), false); + + // Generic 429 path (no model-specific early return) + const result = checkFallbackError( + 429, + GEMINI_429_BODY, + 0, + "gemini-2.5-flash", + "gemini", + null, + PROFILE + ); + + assert.equal(result.reason, RateLimitReason.RATE_LIMIT_EXCEEDED); +}); + +// ── Scenario 8: RPD exhaustion with Gemma 4 (high RPD, never hit with 15 RPM) ─ + +test("Gemma 4 1500 RPD exhaustion overrides RPM classification", () => { + // Pump 1500 daily requests to exhaust RPD + for (let i = 0; i < 1500; i++) incrementRequestCount("gemini/gemma-4-31b-it"); + assert.equal(isRpdExhausted("gemini/gemma-4-31b-it"), true); + assert.equal(isRpmExhausted("gemini/gemma-4-31b-it"), true); // 1500 >> 15 RPM + + const result = checkFallbackError( + 429, + GEMINI_429_BODY, + 0, + "gemini/gemma-4-31b-it", + "gemini", + null, + PROFILE + ); + + // RPD check runs first + assert.equal(result.reason, RateLimitReason.QUOTA_EXHAUSTED); +}); + +// ── Scenario 9: Unknown model (no RPM/RPD in JSON) → generic 429 path ───────── + +test("Unknown Gemini model without published limits falls through to generic 429", () => { + incrementRequestCount("gemini/unknown-model"); + assert.equal(isRpmExhausted("gemini/unknown-model"), false); + assert.equal(isRpdExhausted("gemini/unknown-model"), false); + + const result = checkFallbackError( + 429, + GEMINI_429_BODY, + 0, + "gemini/unknown-model", + "gemini", + null, + PROFILE + ); + + assert.equal(result.reason, RateLimitReason.RATE_LIMIT_EXCEEDED); +}); diff --git a/tests/unit/account-fallback-service.test.ts b/tests/unit/account-fallback-service.test.ts index 3b52caa560..6911616945 100644 --- a/tests/unit/account-fallback-service.test.ts +++ b/tests/unit/account-fallback-service.test.ts @@ -30,6 +30,8 @@ const { isProviderFailureCode, getProvidersInCooldown, getProviderBreakerState, + isCreditsExhausted, + CREDITS_EXHAUSTED_SIGNALS, } = accountFallback; const { selectAccount } = accountSelector; @@ -1115,6 +1117,182 @@ test("checkFallbackError ignores structured error with unrelated code on 400", ( assert.equal(result.shouldFallback, false); }); +// ─── Gemini RPM 429 Classification (CREDITS_EXHAUSTED_SIGNALS fix) ───── + +test("isCreditsExhausted returns false for Gemini RPM 429 body text", () => { + const geminiRpmText = "Resource has been exhausted (e.g. check quota)."; + assert.equal(isCreditsExhausted(geminiRpmText), false); +}); + +test("isCreditsExhausted returns true for actual credits-exhausted signals", () => { + assert.equal(isCreditsExhausted("insufficient_quota"), true); + assert.equal(isCreditsExhausted("credits exhausted"), true); + assert.equal(isCreditsExhausted("payment required"), true); + assert.equal(isCreditsExhausted("free tier of the model has been exhausted"), true); + assert.equal(isCreditsExhausted("exceeded your current usage quota"), true); +}); + +test("CREDITS_EXHAUSTED_SIGNALS no longer contains generic gRPC resource-exhausted patterns", () => { + // These patterns were removed because they falsely matched Gemini RPM 429 errors + assert.equal(CREDITS_EXHAUSTED_SIGNALS.includes("resource has been exhausted"), false); + assert.equal(CREDITS_EXHAUSTED_SIGNALS.includes("resource_exhausted"), false); + assert.equal(CREDITS_EXHAUSTED_SIGNALS.includes("check quota"), false); +}); + +test("checkFallbackError classifies Gemini RPM 429 as RATE_LIMIT_EXCEEDED (not QUOTA_EXHAUSTED)", () => { + // provider=null → preserveQuota429=true → text quota checks run + // isCreditsExhausted must NOT match Gemini's "Resource has been exhausted" + const result = checkFallbackError( + 429, + "Resource has been exhausted (e.g. check quota).", + 0, + null, + null, + null, + makeProfile() + ); + assert.equal(result.shouldFallback, true); + assert.equal(result.reason, RateLimitReason.RATE_LIMIT_EXCEEDED); + assert.equal(result.creditsExhausted, undefined); + assert.equal(result.dailyQuotaExhausted, undefined); + assert.ok(result.cooldownMs > 0, "cooldownMs should be positive"); +}); + +test("checkFallbackError classifies Gemini RPM 429 as RATE_LIMIT_EXCEEDED for API-key provider", () => { + // provider="gemini" → preserveQuota429=false → status-based rule applies + const result = checkFallbackError( + 429, + "Resource has been exhausted (e.g. check quota).", + 0, + null, + "gemini", + null, + makeProfile() + ); + assert.equal(result.shouldFallback, true); + assert.equal(result.reason, RateLimitReason.RATE_LIMIT_EXCEEDED); + assert.equal(result.cooldownMs, 125); // makeProfile().baseCooldownMs +}); + +test("checkFallbackError still classifies genuine OAuth quota-exhausted text as QUOTA_EXHAUSTED", () => { + // Regression: OAuth providers must still get QUOTA_EXHAUSTED for actual quota messages + const result = checkFallbackError( + 429, + "Coding Plan hour quota has been exceeded", + 0, + null, + "codex", + null, + makeProfile() + ); + assert.equal(result.shouldFallback, true); + assert.equal(result.reason, RateLimitReason.QUOTA_EXHAUSTED); +}); + +test("checkFallbackError preserves daily-quota exhaustion for non-429 status codes", () => { + // Non-429 status codes with daily quota text must still be QUOTA_EXHAUSTED + const result = checkFallbackError( + 402, + "You have exceeded today's quota, please try again tomorrow" + ); + assert.equal(result.shouldFallback, true); + assert.equal(result.reason, RateLimitReason.QUOTA_EXHAUSTED); + assert.equal(result.dailyQuotaExhausted, true); +}); + +// ─── Gemini 429 → Model Lockout: rate_limited (not quota_exhausted) ──── + +test("Gemini RPM 429: recordModelLockoutFailure uses exponential backoff for rate_limited reason", () => { + const originalNow = Date.now; + const now = 1_700_000_000_000; + Date.now = () => now; + const provider = "gemini"; + const connectionId = "test-conn-gemini-rpm"; + const model = "gemini/gemma-4-31b-it"; + + try { + clearModelLock(provider, connectionId, model); + + const profile = makeProfile({ + baseCooldownMs: 5000, + transientCooldown: 5000, + rateLimitCooldown: 5000, + }); + + // auth.ts flow: 429 + fallbackResult.reason=RATE_LIMIT_EXCEEDED + // → reason="rate_limited" → recordModelLockoutFailure + const first = recordModelLockoutFailure( + provider, + connectionId, + model, + "rate_limited", + 429, + 0, + profile + ); + assert.equal(first.failureCount, 1); + assert.equal(first.cooldownMs, 5000, "first failure: 5s base cooldown"); + + const second = recordModelLockoutFailure( + provider, + connectionId, + model, + "rate_limited", + 429, + 0, + profile + ); + assert.equal(second.failureCount, 2); + assert.equal(second.cooldownMs, 10000, "second failure: 10s exponential backoff"); + + assert.equal(isModelLocked(provider, connectionId, model), true); + clearModelLock(provider, connectionId, model); + } finally { + Date.now = originalNow; + clearModelLock("gemini", "test-conn-gemini-rpm", "gemini/gemma-4-31b-it"); + } +}); + +test("Gemini RPD (quota_exhausted) still triggers midnight lockout in recordModelLockoutFailure", () => { + // Regression: real daily quota exhaustion must still produce midnight reset + const originalNow = Date.now; + const testDate = new Date(); + testDate.setHours(12, 0, 0, 0); + const now = testDate.getTime(); + Date.now = () => now; + const provider = "gemini"; + const connectionId = "test-conn-gemini-rpd"; + const model = "gemini/gemma-4-31b-it"; + + try { + clearModelLock(provider, connectionId, model); + const profile = makeProfile(); + const result = recordModelLockoutFailure( + provider, + connectionId, + model, + "quota_exhausted", + 429, + 0, + profile + ); + + // Must lock until midnight, NOT exponential backoff + const tomorrow = new Date(now); + tomorrow.setDate(tomorrow.getDate() + 1); + tomorrow.setHours(0, 0, 0, 0); + const expected = tomorrow.getTime() - now; + assert.ok( + Math.abs(result.cooldownMs - expected) <= 300_000, + `cooldown should be until tomorrow (expected ~${expected}, got ${result.cooldownMs})` + ); + clearModelLock(provider, connectionId, model); + } finally { + Date.now = originalNow; + clearModelLock("gemini", "test-conn-gemini-rpd", "gemini/gemma-4-31b-it"); + } +}); + // ─── G-02: X-Omni-Fallback-Hint: connection_cooldown ───────────────────────── // When 9router executor signals a supervisor-not-running 503, checkFallbackError // must return 5s cooldown with skipProviderBreaker:true — not trip the circuit breaker. diff --git a/tests/unit/services/geminiRateLimitTracker.test.ts b/tests/unit/services/geminiRateLimitTracker.test.ts new file mode 100644 index 0000000000..a9d7deb5c3 --- /dev/null +++ b/tests/unit/services/geminiRateLimitTracker.test.ts @@ -0,0 +1,323 @@ +import test from "node:test"; +import assert from "node:assert/strict"; +import { + getModelRpd, + getModelRpm, + incrementRequestCount, + getDailyRequestCount, + getMinuteRequestCount, + isRpdExhausted, + isRpmExhausted, + resetCounters, +} from "../../../open-sse/services/geminiRateLimitTracker.ts"; + +test.beforeEach(() => { + resetCounters(); +}); + +// ── getModelRpd ────────────────────────────────────────────────────────────── + +test("getModelRpd returns known RPD for exact model match", () => { + assert.equal(getModelRpd("gemini-2.5-flash"), 20); +}); + +test("getModelRpd returns known RPD for model with gemini/ prefix", () => { + assert.equal(getModelRpd("gemini/gemini-2.5-flash"), 20); +}); + +test("getModelRpd returns 0 for model with zero RPD (Gemini 2.5 Pro)", () => { + assert.equal(getModelRpd("gemini-2.5-pro"), 0); +}); + +test("getModelRpd returns 0 for unknown model", () => { + assert.equal(getModelRpd("gemini/fake-model-not-in-list"), 0); +}); + +test("getModelRpd returns 0 for empty string", () => { + assert.equal(getModelRpd(""), 0); +}); + +test("getModelRpd matches gemma-4-31b-it via suffix fallback", () => { + // The JSON has "gemma-4-31b-it", and callers may pass "gemma-4-31b-it" + assert.equal(getModelRpd("gemma-4-31b-it"), 1500); +}); + +test("getModelRpd strips gemma- prefix correctly for gemma models", () => { + // stripModelPrefix("gemini/gemma-4-31b-it") → "gemma-4-31b-it" + assert.equal(getModelRpd("gemini/gemma-4-31b-it"), 1500); +}); + +test("getModelRpd handles image-generation models (no RPM value, -1)", () => { + // RPD is 25 for imagen models; RPM is -1 in the JSON + assert.equal(getModelRpd("imagen-4-generate"), 25); +}); + +test("getModelRpd handles models with unlimited RPD (-1)", () => { + // gemini-3.5-live-translate has rpd: -1 + assert.equal(getModelRpd("gemini-3.5-live-translate"), 0); +}); + +test("getModelRpd returns 0 for null input", () => { + assert.equal(getModelRpd(null as unknown as string), 0); +}); + +test("getModelRpd returns 0 for undefined input", () => { + assert.equal(getModelRpd(undefined as unknown as string), 0); +}); + +// ── incrementRequestCount / getDailyRequestCount ───────────────────────────── + +test("incrementRequestCount starts at 1 for first request", () => { + incrementRequestCount("gemini-2.5-flash"); + assert.equal(getDailyRequestCount("gemini-2.5-flash"), 1); +}); + +test("incrementRequestCount increments sequentially", () => { + incrementRequestCount("gemini-2.5-flash"); + incrementRequestCount("gemini-2.5-flash"); + incrementRequestCount("gemini-2.5-flash"); + assert.equal(getDailyRequestCount("gemini-2.5-flash"), 3); +}); + +test("incrementRequestCount treats gemini/ prefix and bare name as same model", () => { + incrementRequestCount("gemini-2.5-flash"); + incrementRequestCount("gemini/gemini-2.5-flash"); + assert.equal(getDailyRequestCount("gemini-2.5-flash"), 2); + assert.equal(getDailyRequestCount("gemini/gemini-2.5-flash"), 2); +}); + +test("models have independent counters", () => { + incrementRequestCount("gemini-2.5-flash"); + incrementRequestCount("gemini-2.5-flash"); + incrementRequestCount("gemma-4-31b-it"); + assert.equal(getDailyRequestCount("gemini-2.5-flash"), 2); + assert.equal(getDailyRequestCount("gemma-4-31b-it"), 1); +}); + +test("getDailyRequestCount returns 0 for model with no requests", () => { + assert.equal(getDailyRequestCount("gemini-2.5-flash"), 0); +}); + +test("incrementRequestCount does nothing for empty model ID", () => { + incrementRequestCount(""); + assert.equal(getDailyRequestCount("gemini-2.5-flash"), 0); +}); + +test("resetCounters clears all state between tests", () => { + incrementRequestCount("gemini-2.5-flash"); + incrementRequestCount("gemma-4-31b-it"); + resetCounters(); + assert.equal(getDailyRequestCount("gemini-2.5-flash"), 0); + assert.equal(getDailyRequestCount("gemma-4-31b-it"), 0); +}); + +// ── isRpdExhausted ─────────────────────────────────────────────────────────── + +test("isRpdExhausted returns false when count is below RPD limit", () => { + // gemini-2.5-flash has RPD=20 + for (let i = 0; i < 19; i++) incrementRequestCount("gemini-2.5-flash"); + assert.equal(isRpdExhausted("gemini-2.5-flash"), false); +}); + +test("isRpdExhausted returns true when count equals RPD limit", () => { + // gemini-2.5-flash has RPD=20 + for (let i = 0; i < 20; i++) incrementRequestCount("gemini-2.5-flash"); + assert.equal(isRpdExhausted("gemini-2.5-flash"), true); +}); + +test("isRpdExhausted returns true when count exceeds RPD limit", () => { + for (let i = 0; i < 25; i++) incrementRequestCount("gemini-2.5-flash"); + assert.equal(isRpdExhausted("gemini-2.5-flash"), true); +}); + +test("isRpdExhausted returns false for model with RPD=0", () => { + // gemini-2.5-pro has rpd=0 + incrementRequestCount("gemini-2.5-pro"); + assert.equal(isRpdExhausted("gemini-2.5-pro"), false); +}); + +test("isRpdExhausted returns false for unknown model", () => { + incrementRequestCount("gemini/unknown-model"); + assert.equal(isRpdExhausted("gemini/unknown-model"), false); +}); + +test("isRpdExhausted returns false for model with unlimited RPD (-1, no data)", () => { + // gemini-3.5-live-translate has rpd: -1 + incrementRequestCount("gemini-3.5-live-translate"); + assert.equal(isRpdExhausted("gemini-3.5-live-translate"), false); +}); + +test("isRpdExhausted works with gemini/ prefix for the model", () => { + for (let i = 0; i < 20; i++) incrementRequestCount("gemini/gemini-2.5-flash"); + assert.equal(isRpdExhausted("gemini/gemini-2.5-flash"), true); +}); + +// ── Integration: Gemma 4 RPM scenario ──────────────────────────────────────── + +test("Gemma 4: 15 RPM hits never trigger quota_exhausted (RPD=1500)", () => { + // Gemma 4 has RPD=1500, so 15 RPM hits should not trigger RPD exhaustion + for (let i = 0; i < 15; i++) incrementRequestCount("gemini/gemma-4-31b-it"); + assert.equal(isRpdExhausted("gemini/gemma-4-31b-it"), false); + assert.equal(getDailyRequestCount("gemini/gemma-4-31b-it"), 15); +}); + +test("Gemma 4: RPD exhaustion requires 1500 requests", () => { + for (let i = 0; i < 1499; i++) incrementRequestCount("gemini/gemma-4-31b-it"); + assert.equal(isRpdExhausted("gemini/gemma-4-31b-it"), false); + + incrementRequestCount("gemini/gemma-4-31b-it"); + assert.equal(isRpdExhausted("gemini/gemma-4-31b-it"), true); +}); + +// ── Integration: Gemini 2.5 Flash RPD scenario ─────────────────────────────── + +test("Gemini 2.5 Flash: first 19 requests do NOT exhaust RPD, 20th does", () => { + for (let i = 0; i < 19; i++) incrementRequestCount("gemini-2.5-flash"); + assert.equal(isRpdExhausted("gemini-2.5-flash"), false, "19 requests < 20 RPD"); + + incrementRequestCount("gemini-2.5-flash"); + assert.equal(isRpdExhausted("gemini-2.5-flash"), true, "20 requests = 20 RPD"); +}); + +test("Gemini 2.5 Flash: excess requests stay exhausted", () => { + for (let i = 0; i < 22; i++) incrementRequestCount("gemini-2.5-flash"); + assert.equal(isRpdExhausted("gemini-2.5-flash"), true); + assert.equal(getDailyRequestCount("gemini-2.5-flash"), 22); +}); + +// ── getModelRpm ─────────────────────────────────────────────────────────────── + +test("getModelRpm returns known RPM for exact model match", () => { + assert.equal(getModelRpm("gemini-2.5-flash"), 5); +}); + +test("getModelRpm returns known RPM for model with gemini/ prefix", () => { + assert.equal(getModelRpm("gemini/gemini-2.5-flash"), 5); +}); + +test("getModelRpm returns 0 for model with zero RPM", () => { + assert.equal(getModelRpm("gemini-2.5-pro"), 0); +}); + +test("getModelRpm returns 0 for unknown model", () => { + assert.equal(getModelRpm("gemini/fake-model-not-in-list"), 0); +}); + +test("getModelRpm returns 0 for empty string", () => { + assert.equal(getModelRpm(""), 0); +}); + +test("getModelRpm returns 0 for models with RPM=-1 (imagen)", () => { + assert.equal(getModelRpm("imagen-4-generate"), 0); +}); + +test("getModelRpm returns 0 for null input", () => { + assert.equal(getModelRpm(null as unknown as string), 0); +}); + +// ── incrementMinuteRequestCount / getMinuteRequestCount ─────────────────────── + +test("getMinuteRequestCount returns 0 before first request", () => { + assert.equal(getMinuteRequestCount("gemini-2.5-flash"), 0); +}); + +test("incrementMinuteRequestCount starts at 1", () => { + incrementRequestCount("gemini-2.5-flash"); + assert.equal(getMinuteRequestCount("gemini-2.5-flash"), 1); +}); + +test("incrementMinuteRequestCount increments with each call", () => { + incrementRequestCount("gemini-2.5-flash"); + incrementRequestCount("gemini-2.5-flash"); + incrementRequestCount("gemini-2.5-flash"); + assert.equal(getMinuteRequestCount("gemini-2.5-flash"), 3); +}); + +test("incrementMinuteRequestCount normalizes gemini/ prefix", () => { + incrementRequestCount("gemini-2.5-flash"); + incrementRequestCount("gemini/gemini-2.5-flash"); + assert.equal(getMinuteRequestCount("gemini-2.5-flash"), 2); +}); + +test("minute counters are independent per model", () => { + incrementRequestCount("gemini-2.5-flash"); + incrementRequestCount("gemini-2.5-flash"); + incrementRequestCount("gemma-4-31b-it"); + assert.equal(getMinuteRequestCount("gemini-2.5-flash"), 2); + assert.equal(getMinuteRequestCount("gemma-4-31b-it"), 1); +}); + +test("incrementRequestCount does nothing for empty model ID", () => { + incrementRequestCount(""); + assert.equal(getMinuteRequestCount("gemini-2.5-flash"), 0); +}); + +test("resetCounters clears minute windows", () => { + incrementRequestCount("gemini-2.5-flash"); + incrementRequestCount("gemma-4-31b-it"); + resetCounters(); + assert.equal(getMinuteRequestCount("gemini-2.5-flash"), 0); + assert.equal(getMinuteRequestCount("gemma-4-31b-it"), 0); +}); + +// ── isRpmExhausted ──────────────────────────────────────────────────────────── + +test("isRpmExhausted returns false below RPM limit", () => { + // gemini-2.5-flash has RPM=5 + for (let i = 0; i < 4; i++) incrementRequestCount("gemini-2.5-flash"); + assert.equal(isRpmExhausted("gemini-2.5-flash"), false); +}); + +test("isRpmExhausted returns true at RPM limit", () => { + for (let i = 0; i < 5; i++) incrementRequestCount("gemini-2.5-flash"); + assert.equal(isRpmExhausted("gemini-2.5-flash"), true); +}); + +test("isRpmExhausted returns true above RPM limit", () => { + for (let i = 0; i < 8; i++) incrementRequestCount("gemini-2.5-flash"); + assert.equal(isRpmExhausted("gemini-2.5-flash"), true); +}); + +test("isRpmExhausted returns false for model with RPM=0", () => { + incrementRequestCount("gemini-2.5-pro"); + assert.equal(isRpmExhausted("gemini-2.5-pro"), false); +}); + +test("isRpmExhausted returns false for unknown model", () => { + incrementRequestCount("gemini/unknown-model"); + assert.equal(isRpmExhausted("gemini/unknown-model"), false); +}); + +test("isRpmExhausted returns false for model with RPM=-1 (imagen)", () => { + incrementRequestCount("imagen-4-generate"); + assert.equal(isRpmExhausted("imagen-4-generate"), false); +}); + +test("isRpmExhausted works with gemini/ prefix", () => { + for (let i = 0; i < 5; i++) incrementRequestCount("gemini/gemini-2.5-flash"); + assert.equal(isRpmExhausted("gemini/gemini-2.5-flash"), true); +}); + +// ── Integration: RPM + RPD work independently ───────────────────────────────── + +test("RPM and RPD limits are tracked independently", () => { + // Gemma 4: RPM=15, RPD=1500 + // After 15 requests, RPM is exhausted but RPD is not + for (let i = 0; i < 15; i++) incrementRequestCount("gemini/gemma-4-31b-it"); + assert.equal(isRpmExhausted("gemini/gemma-4-31b-it"), true); + assert.equal(isRpdExhausted("gemini/gemma-4-31b-it"), false); +}); + +test("Gemini 2.5 Flash: RPM=5 always hits before RPD=20", () => { + // After 5 requests, RPM is exhausted; after 20, RPD is exhausted + for (let i = 0; i < 5; i++) incrementRequestCount("gemini-2.5-flash"); + assert.equal(isRpmExhausted("gemini-2.5-flash"), true); + assert.equal(isRpdExhausted("gemini-2.5-flash"), false); + + incrementRequestCount("gemini-2.5-flash"); // 6 + incrementRequestCount("gemini-2.5-flash"); // 7 + incrementRequestCount("gemini-2.5-flash"); // 8 + + // Still at RPD=8 which is < 20 + assert.equal(isRpdExhausted("gemini-2.5-flash"), false); +});