mirror of
https://github.com/diegosouzapw/OmniRoute.git
synced 2026-07-31 12:22:14 +03:00
feat: gemini - handle known ratelimits (#3686)
Integrated into release/v3.8.23
This commit is contained in:
@@ -4,6 +4,7 @@
|
||||
|
||||
---
|
||||
|
||||
## [3.8.22] — TBD
|
||||
## [3.8.23] — TBD
|
||||
|
||||
---
|
||||
@@ -28,6 +29,7 @@
|
||||
- **Provider-detail god-component decomposition — Phase 2b (remaining shared helpers→leaf)** ([#3501]): extended `providers/[id]/providerPageHelpers.ts` with all remaining pure helpers needed by the heavy modals (`AddApiKeyModal`/`EditConnectionModal`) before they can be extracted. Moved 22 symbols: web-session credential label/hint/check/title helpers; upstream-headers helpers (`upstreamHeadersRecordsEqual`, `headerRowsToRecord`, `effectiveUpstreamHeadersForProtocol`, `anyUpstreamHeadersBadge`, `getProtoSlice`) plus their `HeaderDraftRow`/`CompatModelRow`/`CompatModelMap`/`CompatByProtocolMap` types; Codex consts and helpers (`CODEX_REASONING_STRENGTH_OPTIONS`, `CODEX_ACCOUNT_SERVICE_TIER_VALUES`, `CODEX_GLOBAL_SERVICE_MODE_VALUES`, `getCodexServiceTierLabel`, `normalizeCodexLimitPolicy`, `getCodexRequestDefaults`, `getClaudeCodeCompatibleRequestDefaults`); misc helpers (`compatProtocolLabelKey`, `extractCommandCodeCredentialInput`, `normalizeAndValidateHttpBaseUrl`, `SILICONFLOW_ENDPOINTS`, `CommandCodeAuthFlowState`). New transitive imports wired into the leaf: `MODEL_COMPAT_PROTOCOL_KEYS` (`@/shared/constants/modelCompat`), `CodexServiceTier`/`getCodexRequestDefaults`/`getClaudeCodeCompatibleRequestDefaults` (`@/lib/providers/requestDefaults`), `CodexGlobalServiceMode` (`@/lib/providers/codexFastTier`), `WebSessionCredentialRequirement` (`./webSessionCredentials`). `ProviderDetailPageClient.tsx`: 10,288 → 9,980 LOC. Leaf module: 589 LOC (acyclic). 25-assertion unit test suite passes; smoke test 3/3; no import cycles. Co-authored with @oyi77.
|
||||
- **Provider-detail god-component decomposition — Phase 2 (helpers→lib)** ([#3501]): extracted the pure shared helpers — `ProviderMessageTranslator`/`LocalProviderMetadata` types, `providerText`/`providerCountText`/`readBooleanToggle`, and the provider base-URL + routing-tag/excluded-model parse/format block — into a new leaf `providers/[id]/providerPageHelpers.ts` (imports only `@/shared`, so the client and modals share them with no import cycle). `ProviderDetailPageClient.tsx`: 10,435 → 10,288 LOC. Unblocks extracting the heavier `AddApiKeyModal`/`EditConnectionModal` (which depend on these helpers) without cycling. The Phase 0 smoke test caught a missing transitive import (`isSelfHostedChatProvider`) at mount — now wired + locked by a new helpers unit test (12 assertions). Co-authored with @oyi77.
|
||||
|
||||
- **#3500 fully resolved** — Hard Rule #5 (no raw SQL in route handlers): all 13 internal offenders migrated to `src/lib/db/` modules across slices (call_logs, usage_history/daily_usage_summary, community_servers, usage_logs, semantic_cache, proxy_logs, skills UPDATE, db-backups). The gate's `KNOWN_RAW_SQL` set is renamed to `EXTERNAL_DB_ALLOWED` (with a back-compat alias) and now holds only the **2 external-DB reads** (`oauth/cursor/auto-import`, `oauth/kiro/auto-import`) — these open *another app's* SQLite to import credentials, so by design they cannot live in OmniRoute's `db/` domain. The gate still blocks any NEW raw SQL against OmniRoute's DB.
|
||||
- **#3500 fully resolved** — Hard Rule #5 (no raw SQL in route handlers): all 13 internal offenders migrated to `src/lib/db/` modules across slices (call*logs, usage_history/daily_usage_summary, community_servers, usage_logs, semantic_cache, proxy_logs, skills UPDATE, db-backups). The gate's `KNOWN_RAW_SQL` set is renamed to `EXTERNAL_DB_ALLOWED` (with a back-compat alias) and now holds only the **2 external-DB reads** (`oauth/cursor/auto-import`, `oauth/kiro/auto-import`) — these open \_another app's* SQLite to import credentials, so by design they cannot live in OmniRoute's `db/` domain. The gate still blocks any NEW raw SQL against OmniRoute's DB.
|
||||
- **chore(db-gate):** reclassify `KNOWN_UNEXPORTED` → `INTENTIONALLY_INTERNAL` in `scripts/check/check-db-rules.mjs` ([#3499]): a full audit of all 25 db modules confirmed each is consumed via direct/dynamic import per Hard Rule #2 ("Never barrel-import from localDb.ts"). The old framing labelled them as "debt", which was misleading — they are the correct pattern. The gate's blocking behaviour is unchanged (a NEW unexported module still fails); only the name, comments, and per-module justifications were updated to reflect audited truth. Four modules flagged `DEAD?` (`compressionScheduler`, `discovery`, `pluginMetrics`, `prompts`) have zero production importers and are documented as schema-reserved. A new regression-guard test (`tests/unit/check-db-rules-classification.test.ts`) asserts every non-dead module in the set has ≥1 real importer, so a future consumer removal surfaces as a test failure requiring explicit reclassification.
|
||||
- **refactor(db): move `call_logs` aggregations into `callLogStats` db module** ([#3500]): extracted raw SQL from three route handlers (`/api/provider-metrics`, `/api/search/stats`, `/api/v1/search/analytics`) into a new `src/lib/db/callLogStats.ts` domain module (`getProviderMetrics`, `getSearchProviderStats`, `getRecentSearchLogs`, `getSearchAggregateStats`, `getSearchProviderCounts`). First slice of #3500 (call_logs cluster). Behavior unchanged; the three routes are removed from `KNOWN_RAW_SQL` in the gate. Validated with TDD unit tests (6 assertions seeding an in-memory SQLite fixture).
|
||||
@@ -35,6 +37,7 @@
|
||||
|
||||
- **refactor(db): move `community_servers` auth look-up into `gamification` db module** ([#3500]): extracted raw SQL from two federation route handlers (`/api/gamification/federation/leaderboard`, `/api/gamification/federation/score`) into a new `getConnectedServerByKeyHash(apiKeyHash)` function in `src/lib/db/gamification.ts`. Third slice of #3500 (gamification federation cluster). Behavior unchanged; the two routes are removed from `KNOWN_RAW_SQL` in the gate. Validated with TDD unit tests (3 assertions seeding a temp SQLite fixture).
|
||||
|
||||
- **refactor(db): move `skills UPDATE` + `db-backups` SQL into db modules** ([#3500]): fifth slice of #3500. Extracted the dynamic `UPDATE skills SET …` from `src/app/api/skills/[id]/route.ts` into a new `src/lib/db/skills.ts` module (`updateSkill(id, patch)`). The dynamic SET clause is injection-safe: column names are validated against a hard-coded allowlist of known writable columns before being interpolated; unknown keys are silently ignored. Extended `src/lib/db/backup.ts` with three new functions: `exportAllSummaryRows()` (multi-table SELECT for key_value / combos / provider_connections / api_keys, used by exportAll), `getTableNamesFromAdapter()` (sqlite_master introspection via an adapter arg, used by import validation), and `countImportedRows()` (post-import COUNT(*) per table). The backup domain module is the correct home for sqlite_master introspection — it is not "raw SQL in a route" once moved there. `KNOWN_RAW_SQL` drops by 3 (from 8 → 5). Validated with 11 TDD unit tests (`tests/unit/db-backups-skills-3500.test.ts`).
|
||||
- **refactor(db): move `skills UPDATE` + `db-backups` SQL into db modules** ([#3500]): fifth slice of #3500. Extracted the dynamic `UPDATE skills SET …` from `src/app/api/skills/[id]/route.ts` into a new `src/lib/db/skills.ts` module (`updateSkill(id, patch)`). The dynamic SET clause is injection-safe: column names are validated against a hard-coded allowlist of known writable columns before being interpolated; unknown keys are silently ignored. Extended `src/lib/db/backup.ts` with three new functions: `exportAllSummaryRows()` (multi-table SELECT for key_value / combos / provider_connections / api_keys, used by exportAll), `getTableNamesFromAdapter()` (sqlite_master introspection via an adapter arg, used by import validation), and `countImportedRows()` (post-import COUNT(\*) per table). The backup domain module is the correct home for sqlite_master introspection — it is not "raw SQL in a route" once moved there. `KNOWN_RAW_SQL` drops by 3 (from 8 → 5). Validated with 11 TDD unit tests (`tests/unit/db-backups-skills-3500.test.ts`).
|
||||
- **refactor(db): move `usage_logs`/`semantic_cache`/`proxy_logs` SQL into db modules** ([#3500]): extracted raw `db.prepare(...)` SQL from three route handlers (`/api/analytics/auto-routing` → `usageLogs.ts`; `/api/cache/entries` → `semanticCache.ts`; `/api/logs/export` → `proxyLogs.ts`) into new `src/lib/db/` domain modules. New exports: `getAutoRoutingTotalCount`, `getAutoRoutingVariantBreakdown`, `getAutoRoutingTopProviders` (usage_logs), `listSemanticCacheEntries`, `deleteSemanticCacheBySignature`, `deleteSemanticCacheByModel` (semantic_cache), and `exportProxyLogsSince` (proxy_logs). Fourth slice of #3500. `KNOWN_RAW_SQL` drops from 8 → 5. Validated with 13 TDD unit tests (`tests/unit/db-logs-cache-3500.test.ts`) seeding temp SQLite fixtures.
|
||||
|
||||
|
||||
34
open-sse/config/geminiRateLimits.json
Normal file
34
open-sse/config/geminiRateLimits.json
Normal file
@@ -0,0 +1,34 @@
|
||||
{
|
||||
"gemini-2.5-flash": { "rpm": 5, "rpd": 20, "tpm": 250000 },
|
||||
"gemini-2.5-pro": { "rpm": 0, "rpd": 0, "tpm": 0 },
|
||||
"gemini-2-flash": { "rpm": 0, "rpd": 0, "tpm": 0 },
|
||||
"gemini-2-flash-lite": { "rpm": 0, "rpd": 0, "tpm": 0 },
|
||||
"gemini-2.5-flash-tts": { "rpm": 3, "rpd": 10, "tpm": 10000 },
|
||||
"gemini-2.5-pro-tts": { "rpm": 0, "rpd": 0, "tpm": 0 },
|
||||
"imagen-4-generate": { "rpm": -1, "rpd": 25, "tpm": -1 },
|
||||
"imagen-4-ultra-generate": { "rpm": -1, "rpd": 25, "tpm": -1 },
|
||||
"imagen-4-fast-generate": { "rpm": -1, "rpd": 25, "tpm": -1 },
|
||||
"gemma-4-26b-it": { "rpm": 15, "rpd": 1500, "tpm": -1 },
|
||||
"gemma-4-31b-it": { "rpm": 15, "rpd": 1500, "tpm": -1 },
|
||||
"gemini-embedding-exp-03-07": { "rpm": 100, "rpd": 1000, "tpm": 30000 },
|
||||
"gemini-3.5-flash": { "rpm": 5, "rpd": 20, "tpm": 250000 },
|
||||
"gemini-3.1-flash-lite": { "rpm": 15, "rpd": 500, "tpm": 250000 },
|
||||
"gemini-3.1-pro": { "rpm": 0, "rpd": 0, "tpm": 0 },
|
||||
"gemini-2.5-flash-lite": { "rpm": 10, "rpd": 20, "tpm": 250000 },
|
||||
"nano-banana-gemini-2.5-flash-preview-image": { "rpm": 0, "rpd": 0, "tpm": 0 },
|
||||
"nano-banana-pro-gemini-3-pro-image": { "rpm": 0, "rpd": 0, "tpm": 0 },
|
||||
"nano-banana-2-gemini-3.1-flash-image": { "rpm": 0, "rpd": 0, "tpm": 0 },
|
||||
"lyria-3-clip": { "rpm": 0, "rpd": 0, "tpm": 0 },
|
||||
"lyria-3-pro": { "rpm": 0, "rpd": 0, "tpm": 0 },
|
||||
"veo-3-generate": { "rpm": 0, "rpd": 0, "tpm": -1 },
|
||||
"veo-3-fast-generate": { "rpm": 0, "rpd": 0, "tpm": -1 },
|
||||
"veo-3-lite-generate": { "rpm": 0, "rpd": 0, "tpm": -1 },
|
||||
"gemini-3.1-flash-tts": { "rpm": 3, "rpd": 10, "tpm": 10000 },
|
||||
"gemini-robotics-er-1.5-preview": { "rpm": 10, "rpd": 20, "tpm": 250000 },
|
||||
"gemini-robotics-er-1.6-preview": { "rpm": 5, "rpd": 20, "tpm": 250000 },
|
||||
"computer-use-preview": { "rpm": 0, "rpd": 0, "tpm": 0 },
|
||||
"gemini-embedding-exp-04-07": { "rpm": 100, "rpd": 1000, "tpm": 30000 },
|
||||
"gemini-3.5-live-translate": { "rpm": -1, "rpd": -1, "tpm": 20000 },
|
||||
"gemini-2.5-flash-native-audio-dialog": { "rpm": -1, "rpd": -1, "tpm": 1000000 },
|
||||
"gemini-3-flash-live": { "rpm": -1, "rpd": -1, "tpm": 65000 }
|
||||
}
|
||||
@@ -233,6 +233,7 @@ import {
|
||||
getModelScopeRetryDelayMs,
|
||||
isModelScopeProvider,
|
||||
} from "../services/modelscopePolicy.ts";
|
||||
import { incrementRequestCount } from "../services/geminiRateLimitTracker.ts";
|
||||
|
||||
const MEMORY_EXTRACTION_TEXT_LIMIT = 64 * 1024;
|
||||
|
||||
@@ -3797,6 +3798,12 @@ export async function handleChatCore({
|
||||
});
|
||||
const res = normalizeExecutorResult(rawExecutorResult);
|
||||
trace("post_executor", { status: res?.response?.status });
|
||||
|
||||
// Track Gemini RPM + RPD request counts for 429 classification
|
||||
if (provider === "gemini") {
|
||||
incrementRequestCount(modelToCall);
|
||||
}
|
||||
|
||||
updatePendingRequest(model, provider, connectionId, {
|
||||
stage: "provider_response_started",
|
||||
});
|
||||
|
||||
@@ -28,6 +28,7 @@ import {
|
||||
type FailureKind,
|
||||
} from "../../src/shared/utils/classify429";
|
||||
import { resolveUseUpstream429BreakerHints } from "../../src/shared/utils/providerHints";
|
||||
import { isRpdExhausted, isRpmExhausted } from "./geminiRateLimitTracker.ts";
|
||||
|
||||
export type ProviderProfile = {
|
||||
baseCooldownMs: number;
|
||||
@@ -150,9 +151,6 @@ export const CREDITS_EXHAUSTED_SIGNALS = [
|
||||
"credits exhausted",
|
||||
"out of credits",
|
||||
"payment required",
|
||||
"resource has been exhausted",
|
||||
"resource_exhausted",
|
||||
"check quota",
|
||||
"free tier of the model has been exhausted",
|
||||
];
|
||||
|
||||
@@ -933,8 +931,9 @@ export function parseRetryFromErrorText(errorText: unknown): number | null {
|
||||
// 2026-05-17T10:00:00Z" or "Please wait until 2026-05-17T10:00:00.000Z").
|
||||
// Convert to a future-duration in milliseconds if it parses.
|
||||
const isoMatch =
|
||||
/\b(?:try again at|wait until|reset(?:s)? at|available at|retry after)\s+(\d{4}-\d{2}-\d{2}[Tt ]\d{2}:\d{2}(?::\d{2})?(?:\.\d+)?(?:Z|[+-]\d{2}:?\d{2})?)/i
|
||||
.exec(msg);
|
||||
/\b(?:try again at|wait until|reset(?:s)? at|available at|retry after)\s+(\d{4}-\d{2}-\d{2}[Tt ]\d{2}:\d{2}(?::\d{2})?(?:\.\d+)?(?:Z|[+-]\d{2}:?\d{2})?)/i.exec(
|
||||
msg
|
||||
);
|
||||
if (isoMatch) {
|
||||
const parsedTs = Date.parse(isoMatch[1]);
|
||||
if (Number.isFinite(parsedTs)) {
|
||||
@@ -1021,10 +1020,8 @@ export function classifyErrorText(errorText: unknown): RateLimitReasonValue {
|
||||
const configuredRule = matchErrorRuleByText(errorText);
|
||||
if (configuredRule?.reason) return configuredRule.reason;
|
||||
if (lower.includes("rate_limit")) return RateLimitReason.RATE_LIMIT_EXCEEDED;
|
||||
if (
|
||||
lower.includes("resource exhausted") ||
|
||||
lower.includes("high demand")
|
||||
) return RateLimitReason.MODEL_CAPACITY;
|
||||
if (lower.includes("resource exhausted") || lower.includes("high demand"))
|
||||
return RateLimitReason.MODEL_CAPACITY;
|
||||
if (
|
||||
lower.includes("unauthorized") ||
|
||||
lower.includes("invalid api key") ||
|
||||
@@ -1414,6 +1411,19 @@ export function checkFallbackError(
|
||||
}
|
||||
}
|
||||
|
||||
// Gemini-specific: use known published RPM/RPD limits to distinguish 429 types.
|
||||
// Gemini returns the same error body for both, so we use per-model request
|
||||
// counters to decide: if daily count >= RPD → quota_exhausted (midnight lockout);
|
||||
// if minute count >= RPM → rate_limit_exceeded (exponential backoff).
|
||||
if (provider === "gemini" && status === HTTP_STATUS.RATE_LIMITED && _model) {
|
||||
if (isRpdExhausted(_model)) {
|
||||
return buildRetryableFallback(RateLimitReason.QUOTA_EXHAUSTED);
|
||||
}
|
||||
if (isRpmExhausted(_model)) {
|
||||
return buildRetryableFallback(RateLimitReason.RATE_LIMIT_EXCEEDED);
|
||||
}
|
||||
}
|
||||
|
||||
const configuredRule =
|
||||
isRateLimitStatus && !preserveQuota429
|
||||
? matchErrorRuleByStatus(status)
|
||||
|
||||
145
open-sse/services/geminiRateLimitTracker.ts
Normal file
145
open-sse/services/geminiRateLimitTracker.ts
Normal file
@@ -0,0 +1,145 @@
|
||||
/**
|
||||
* In-memory request counters for Gemini models — tracks both RPD (daily)
|
||||
* and RPM (sliding 60s window) so that 429 responses can be classified
|
||||
* as either quota_exhausted (RPD hit) or rate_limit_exceeded (RPM hit).
|
||||
*
|
||||
* Gemini returns identical error bodies for both types, so we rely on
|
||||
* published per-model limits from geminiRateLimits.json to distinguish them.
|
||||
*
|
||||
* Counters are incremented on every Gemini request so that once usage
|
||||
* reaches the published limit, subsequent 429s are correctly classified.
|
||||
*/
|
||||
|
||||
import geminiLimits from "../config/geminiRateLimits.json";
|
||||
|
||||
// ── RPD (daily) state ────────────────────────────────────────────────────────
|
||||
|
||||
interface DailyCount {
|
||||
date: string; // "YYYY-MM-DD"
|
||||
count: number;
|
||||
}
|
||||
|
||||
const dailyCounts = new Map<string, DailyCount>();
|
||||
|
||||
// ── RPM (sliding 60s window) state ───────────────────────────────────────────
|
||||
|
||||
const minuteWindows = new Map<string, number[]>();
|
||||
|
||||
// ── Helpers ──────────────────────────────────────────────────────────────────
|
||||
|
||||
function toDateKey(): string {
|
||||
return new Date().toISOString().slice(0, 10);
|
||||
}
|
||||
|
||||
function stripModelPrefix(modelId: string): string {
|
||||
// Only strip the "gemini/" provider prefix, never "gemini-" which is part
|
||||
// of the actual model name (e.g. "gemini-2.5-flash", "gemini-3.5-live-translate").
|
||||
return modelId.replace(/^gemini\//, "").trim();
|
||||
}
|
||||
|
||||
function lookupValue(modelId: string, field: "rpm" | "rpd"): number {
|
||||
if (!modelId) return 0;
|
||||
const key = stripModelPrefix(modelId);
|
||||
const entry = (geminiLimits as Record<string, Record<string, number>>)[key];
|
||||
if (!entry) {
|
||||
for (const [knownKey, knownEntry] of Object.entries(geminiLimits)) {
|
||||
if (key.endsWith(knownKey) || knownKey.endsWith(key)) {
|
||||
const val = knownEntry[field];
|
||||
return typeof val === "number" && val > 0 ? val : 0;
|
||||
}
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
const val = entry[field];
|
||||
return typeof val === "number" && val > 0 ? val : 0;
|
||||
}
|
||||
|
||||
// ── RPD exports ──────────────────────────────────────────────────────────────
|
||||
|
||||
export function getModelRpd(modelId: string): number {
|
||||
return lookupValue(modelId, "rpd");
|
||||
}
|
||||
|
||||
export function incrementDailyRequestCount(modelId: string): void {
|
||||
if (!modelId) return;
|
||||
const key = stripModelPrefix(modelId);
|
||||
const today = toDateKey();
|
||||
const existing = dailyCounts.get(key);
|
||||
if (existing && existing.date === today) {
|
||||
existing.count++;
|
||||
} else {
|
||||
dailyCounts.set(key, { date: today, count: 1 });
|
||||
}
|
||||
}
|
||||
|
||||
export function getDailyRequestCount(modelId: string): number {
|
||||
if (!modelId) return 0;
|
||||
const key = stripModelPrefix(modelId);
|
||||
const today = toDateKey();
|
||||
const entry = dailyCounts.get(key);
|
||||
if (entry && entry.date === today) return entry.count;
|
||||
return 0;
|
||||
}
|
||||
|
||||
export function isRpdExhausted(modelId: string): boolean {
|
||||
const rpd = getModelRpd(modelId);
|
||||
if (rpd <= 0) return false;
|
||||
return getDailyRequestCount(modelId) >= rpd;
|
||||
}
|
||||
|
||||
// ── RPM exports ──────────────────────────────────────────────────────────────
|
||||
|
||||
export function getModelRpm(modelId: string): number {
|
||||
return lookupValue(modelId, "rpm");
|
||||
}
|
||||
|
||||
/** Prune timestamps older than 60 seconds from a model's window. */
|
||||
function pruneMinuteWindow(key: string): void {
|
||||
const now = Date.now();
|
||||
const cutoff = now - 60_000;
|
||||
const timestamps = minuteWindows.get(key);
|
||||
if (!timestamps) return;
|
||||
// Keep only timestamps >= cutoff
|
||||
let i = 0;
|
||||
while (i < timestamps.length && timestamps[i] < cutoff) i++;
|
||||
if (i > 0) {
|
||||
minuteWindows.set(key, timestamps.slice(i));
|
||||
}
|
||||
}
|
||||
|
||||
export function incrementMinuteRequestCount(modelId: string): void {
|
||||
if (!modelId) return;
|
||||
const key = stripModelPrefix(modelId);
|
||||
pruneMinuteWindow(key);
|
||||
const timestamps = minuteWindows.get(key) ?? [];
|
||||
timestamps.push(Date.now());
|
||||
minuteWindows.set(key, timestamps);
|
||||
}
|
||||
|
||||
export function getMinuteRequestCount(modelId: string): number {
|
||||
if (!modelId) return 0;
|
||||
const key = stripModelPrefix(modelId);
|
||||
pruneMinuteWindow(key);
|
||||
return minuteWindows.get(key)?.length ?? 0;
|
||||
}
|
||||
|
||||
export function isRpmExhausted(modelId: string): boolean {
|
||||
const rpm = getModelRpm(modelId);
|
||||
if (rpm <= 0) return false;
|
||||
return getMinuteRequestCount(modelId) >= rpm;
|
||||
}
|
||||
|
||||
// ── Increment both (convenience) ─────────────────────────────────────────────
|
||||
|
||||
/** Increment both daily and minute counters for a Gemini request. */
|
||||
export function incrementRequestCount(modelId: string): void {
|
||||
incrementDailyRequestCount(modelId);
|
||||
incrementMinuteRequestCount(modelId);
|
||||
}
|
||||
|
||||
// ── Reset (testing) ──────────────────────────────────────────────────────────
|
||||
|
||||
export function resetCounters(): void {
|
||||
dailyCounts.clear();
|
||||
minuteWindows.clear();
|
||||
}
|
||||
@@ -7,6 +7,7 @@
|
||||
"checkJs": true,
|
||||
"noEmit": true,
|
||||
"allowImportingTsExtensions": true,
|
||||
"resolveJsonModule": true,
|
||||
"skipLibCheck": true,
|
||||
"esModuleInterop": true,
|
||||
"strict": false,
|
||||
|
||||
@@ -31,7 +31,7 @@ import {
|
||||
recordModelLockoutFailure,
|
||||
} from "@omniroute/open-sse/services/accountFallback.ts";
|
||||
import { isLocalProvider } from "@omniroute/open-sse/config/providerRegistry.ts";
|
||||
import { COOLDOWN_MS } from "@omniroute/open-sse/config/constants.ts";
|
||||
import { COOLDOWN_MS, RateLimitReason } from "@omniroute/open-sse/config/constants.ts";
|
||||
import {
|
||||
preflightQuota,
|
||||
isQuotaPreflightEnabled,
|
||||
@@ -42,7 +42,7 @@ import {
|
||||
classifyProviderError,
|
||||
PROVIDER_ERROR_TYPES,
|
||||
} from "@omniroute/open-sse/services/errorClassifier.ts";
|
||||
import { looksLikeQuotaExhausted } from "@/shared/utils/classify429";
|
||||
|
||||
import { getCodexModelScope } from "@omniroute/open-sse/executors/codex.ts";
|
||||
import {
|
||||
getProviderById,
|
||||
@@ -1799,7 +1799,7 @@ export async function markAccountUnavailable(
|
||||
const reason =
|
||||
status === 404
|
||||
? "not_found"
|
||||
: status === 429 && looksLikeQuotaExhausted(errorText)
|
||||
: status === 429 && fallbackResult.reason === RateLimitReason.QUOTA_EXHAUSTED
|
||||
? "quota_exhausted"
|
||||
: status === 429
|
||||
? "rate_limited"
|
||||
|
||||
152
tests/integration/gemini-live-429-classification.test.ts
Normal file
152
tests/integration/gemini-live-429-classification.test.ts
Normal file
@@ -0,0 +1,152 @@
|
||||
/**
|
||||
* Gemini 429 classification integration tests.
|
||||
*
|
||||
* Tests the end-to-end classification path for Gemini rate-limit errors
|
||||
* through OmniRoute. Sends bursts of requests to try to trigger published
|
||||
* RPM/RPD limits, then verifies the classification is correct.
|
||||
*
|
||||
* The tests are "best effort" — if rate limits aren't triggered (Gemini
|
||||
* may be more generous in practice), the test logs a warning and passes
|
||||
* rather than failing. The unit tests in account-fallback-service.test.ts
|
||||
* provide the definitive coverage of classification logic.
|
||||
*
|
||||
* Env vars:
|
||||
* OMNIROUTE_URL — base URL (default http://localhost:20128)
|
||||
* OMNIROUTE_API_KEY — API key for auth (REQUIRED)
|
||||
* TEST_GEMINI_RPM_MODEL — RPM model (default gemini/gemma-4-31b-it)
|
||||
* TEST_GEMINI_RPD_MODEL — RPD model (default gemini/gemini-2.5-flash)
|
||||
*/
|
||||
|
||||
import test from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
|
||||
const API_KEY = process.env.OMNIROUTE_API_KEY;
|
||||
const BASE_URL = process.env.OMNIROUTE_URL || "http://localhost:20128";
|
||||
const RPM_MODEL = process.env.TEST_GEMINI_RPM_MODEL || "gemini/gemma-4-31b-it";
|
||||
const RPD_MODEL = process.env.TEST_GEMINI_RPD_MODEL || "gemini/gemini-2.5-flash";
|
||||
|
||||
const skip = !API_KEY ? "OMNIROUTE_API_KEY not set — skipping live test" : undefined;
|
||||
|
||||
async function chat(model: string, content: string) {
|
||||
const res = await fetch(`${BASE_URL}/api/v1/chat/completions`, {
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json", Authorization: `Bearer ${API_KEY}` },
|
||||
body: JSON.stringify({ model, stream: false, messages: [{ role: "user", content }] }),
|
||||
});
|
||||
return { status: res.status, body: await res.text() };
|
||||
}
|
||||
|
||||
// ── Test 1: RPM burst ────────────────────────────────────────────────────────
|
||||
|
||||
test(
|
||||
"Gemma 4 RPM burst: try to hit 15 RPM, verify 429 classification if triggered",
|
||||
{ skip },
|
||||
async () => {
|
||||
const BURST = 30;
|
||||
console.error(`\n[RPM] Sending ${BURST} concurrent requests to ${RPM_MODEL} (15 RPM)...`);
|
||||
const fetches = Array.from({ length: BURST }, (_, i) =>
|
||||
chat(RPM_MODEL, `Count to 3. Only numbers. Request ${i}.`)
|
||||
);
|
||||
const results = await Promise.all(fetches);
|
||||
|
||||
const statuses = results.map((r) => r.status);
|
||||
const successes = results.filter((r) => r.status === 200);
|
||||
const rateLimited = results.filter((r) => r.status === 429);
|
||||
|
||||
console.error(
|
||||
`[RPM] ${successes.length} success, ${rateLimited.length} 429 (statuses: ${statuses.join(",")})`
|
||||
);
|
||||
|
||||
assert.ok(successes.length > 0, "expected at least one successful request");
|
||||
|
||||
if (rateLimited.length > 0) {
|
||||
for (const r of rateLimited) {
|
||||
assert.equal(
|
||||
r.body.includes("quota_exhausted"),
|
||||
false,
|
||||
`RPM 429 should NOT be quota_exhausted: ${r.body.slice(0, 300)}`
|
||||
);
|
||||
assert.ok(
|
||||
r.body.includes("cooling down") || r.body.includes("rate_limit"),
|
||||
`RPM 429 should mention cooldown: ${r.body.slice(0, 300)}`
|
||||
);
|
||||
}
|
||||
} else {
|
||||
console.error("[RPM] No 429s received (Gemini may have higher effective RPM for this key)");
|
||||
console.error(
|
||||
"[RPM] Classification logic verified by unit tests in account-fallback-service.test.ts"
|
||||
);
|
||||
}
|
||||
}
|
||||
);
|
||||
|
||||
test("Gemma 4 RPM recovery: after 65s, requests should succeed again", { skip }, async () => {
|
||||
// First send a burst to ensure any cooldown from the previous test has cleared
|
||||
const warmup = await chat(RPM_MODEL, "ping");
|
||||
if (warmup.status === 429) {
|
||||
console.error("[RPM recovery] Previous test left model in cooldown, waiting 65s...");
|
||||
await new Promise((r) => setTimeout(r, 65_000));
|
||||
} else {
|
||||
console.error("[RPM recovery] Model is healthy, skipping wait");
|
||||
}
|
||||
|
||||
const results: Array<{ status: number }> = [];
|
||||
for (let i = 0; i < 3; i++) {
|
||||
results.push(await chat(RPM_MODEL, `Hello ${i}.`));
|
||||
await new Promise((r) => setTimeout(r, 500));
|
||||
}
|
||||
|
||||
const successes = results.filter((r) => r.status === 200);
|
||||
console.error(`[RPM recovery] ${successes.length}/3 success`);
|
||||
assert.ok(
|
||||
successes.length >= 1,
|
||||
`expected at least 1 recovery, got: ${results.map((r) => r.status).join(",")}`
|
||||
);
|
||||
});
|
||||
|
||||
// ── Test 2: RPD burst ────────────────────────────────────────────────────────
|
||||
|
||||
test(
|
||||
"Gemini 2.5 Flash RPD burst: try to hit 20 RPD, verify quota_exhausted if triggered",
|
||||
{ skip },
|
||||
async () => {
|
||||
const BURST = 30;
|
||||
console.error(`\n[RPD] Sending ${BURST} concurrent requests to ${RPD_MODEL} (20 RPD)...`);
|
||||
const fetches = Array.from({ length: BURST }, (_, i) =>
|
||||
chat(RPD_MODEL, `Count to 5. Only numbers. Request ${i}.`)
|
||||
);
|
||||
const results = await Promise.all(fetches);
|
||||
const statuses = results.map((r) => r.status);
|
||||
const successes = results.filter((r) => r.status === 200);
|
||||
const rateLimited = results.filter((r) => r.status === 429);
|
||||
|
||||
console.error(
|
||||
`[RPD] ${successes.length} success, ${rateLimited.length} 429 (statuses: ${statuses.join(",")})`
|
||||
);
|
||||
|
||||
assert.ok(successes.length > 0, "expected at least one successful request");
|
||||
|
||||
const quotaExhausted = rateLimited.filter((r) =>
|
||||
r.body.toLowerCase().includes("quota_exhausted")
|
||||
);
|
||||
|
||||
if (quotaExhausted.length > 0) {
|
||||
console.error(`[RPD] ${quotaExhausted.length} quota_exhausted responses ✓`);
|
||||
} else if (rateLimited.length > 0) {
|
||||
// Check that non-quota-exhausted 429s are still rate_limit, not some other error
|
||||
for (const r of rateLimited) {
|
||||
assert.equal(
|
||||
r.body.includes("quota_exhausted"),
|
||||
false,
|
||||
`RPM 429 should not be quota_exhausted: ${r.body.slice(0, 200)}`
|
||||
);
|
||||
}
|
||||
console.error("[RPD] 429s present but none are quota_exhausted (RPD not yet hit)");
|
||||
} else {
|
||||
console.error(
|
||||
"[RPD] No 429s received (daily quota may not have been reached, or limits are higher)"
|
||||
);
|
||||
console.error("[RPD] Classification logic verified by unit tests");
|
||||
}
|
||||
}
|
||||
);
|
||||
253
tests/integration/gemini-rate-limit-classification.test.ts
Normal file
253
tests/integration/gemini-rate-limit-classification.test.ts
Normal file
@@ -0,0 +1,253 @@
|
||||
/**
|
||||
* Gemini rate-limit classification integration tests.
|
||||
*
|
||||
* Tests the full integration between geminiRateLimitTracker (in-memory
|
||||
* daily/minute counters) and accountFallback.checkFallbackError (429
|
||||
* classification). No live Gemini API key needed — the tracker counters
|
||||
* are incremented directly and a synthetic 429 error is passed to
|
||||
* checkFallbackError.
|
||||
*
|
||||
* This validates that the whole pipeline works:
|
||||
* incrementRequestCount → isRpdExhausted / isRpmExhausted → checkFallbackError
|
||||
*
|
||||
* Covers three classification outcomes:
|
||||
* - RPM exhausted → RATE_LIMIT_EXCEEDED (exponential backoff)
|
||||
* - RPD exhausted → QUOTA_EXHAUSTED (midnight lockout)
|
||||
* - Neither exhausted → falls through to generic 429 (RATE_LIMIT_EXCEEDED)
|
||||
*/
|
||||
|
||||
import test from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
|
||||
const { checkFallbackError } = await import("../../open-sse/services/accountFallback.ts");
|
||||
const { RateLimitReason } = await import("../../open-sse/config/constants.ts");
|
||||
const {
|
||||
incrementRequestCount,
|
||||
getDailyRequestCount,
|
||||
getMinuteRequestCount,
|
||||
isRpdExhausted,
|
||||
isRpmExhausted,
|
||||
resetCounters,
|
||||
} = await import("../../open-sse/services/geminiRateLimitTracker.ts");
|
||||
|
||||
const PROFILE = {
|
||||
baseCooldownMs: 125,
|
||||
useUpstreamRetryHints: false,
|
||||
maxBackoffSteps: 3,
|
||||
failureThreshold: 60,
|
||||
degradationThreshold: 40,
|
||||
resetTimeoutMs: 5000,
|
||||
transientCooldown: 125,
|
||||
rateLimitCooldown: 125,
|
||||
maxBackoffLevel: 3,
|
||||
circuitBreakerThreshold: 60,
|
||||
circuitBreakerReset: 5000,
|
||||
providerFailureThreshold: 5,
|
||||
providerFailureWindowMs: 300000,
|
||||
providerCooldownMs: 60000,
|
||||
};
|
||||
|
||||
const GEMINI_429_BODY = "Resource has been exhausted (e.g. check quota).";
|
||||
|
||||
test.beforeEach(() => {
|
||||
resetCounters();
|
||||
});
|
||||
|
||||
// ── Scenario 1: RPM exhausted, RPD not exhausted → RATE_LIMIT_EXCEEDED ────────
|
||||
|
||||
test("Gemini 2.5 Flash 5 RPM hit: 429 classifies as RATE_LIMIT_EXCEEDED (not QUOTA_EXHAUSTED)", () => {
|
||||
// gemini-2.5-flash: RPM=5, RPD=20
|
||||
for (let i = 0; i < 5; i++) incrementRequestCount("gemini-2.5-flash");
|
||||
assert.equal(isRpmExhausted("gemini-2.5-flash"), true);
|
||||
assert.equal(isRpdExhausted("gemini-2.5-flash"), false);
|
||||
|
||||
const result = checkFallbackError(
|
||||
429,
|
||||
GEMINI_429_BODY,
|
||||
0,
|
||||
"gemini-2.5-flash",
|
||||
"gemini",
|
||||
null,
|
||||
PROFILE
|
||||
);
|
||||
|
||||
assert.equal(result.shouldFallback, true);
|
||||
assert.equal(result.reason, RateLimitReason.RATE_LIMIT_EXCEEDED);
|
||||
assert.ok(result.cooldownMs > 0, "cooldownMs should be positive");
|
||||
});
|
||||
|
||||
// ── Scenario 2: RPD exhausted → QUOTA_EXHAUSTED ───────────────────────────────
|
||||
|
||||
test("Gemini 2.5 Flash 20 RPD hit: 429 classifies as QUOTA_EXHAUSTED", () => {
|
||||
for (let i = 0; i < 20; i++) incrementRequestCount("gemini-2.5-flash");
|
||||
assert.equal(isRpdExhausted("gemini-2.5-flash"), true);
|
||||
|
||||
const result = checkFallbackError(
|
||||
429,
|
||||
GEMINI_429_BODY,
|
||||
0,
|
||||
"gemini-2.5-flash",
|
||||
"gemini",
|
||||
null,
|
||||
PROFILE
|
||||
);
|
||||
|
||||
assert.equal(result.shouldFallback, true);
|
||||
assert.equal(result.reason, RateLimitReason.QUOTA_EXHAUSTED);
|
||||
assert.ok(result.cooldownMs > 0, "cooldownMs should be positive");
|
||||
});
|
||||
|
||||
// ── Scenario 3: Neither RPM nor RPD exhausted → falls through to generic 429 ──
|
||||
|
||||
test("Gemini 2.5 Flash 3 requests (below both): 429 falls through to generic RATE_LIMIT_EXCEEDED", () => {
|
||||
for (let i = 0; i < 3; i++) incrementRequestCount("gemini-2.5-flash");
|
||||
assert.equal(isRpmExhausted("gemini-2.5-flash"), false);
|
||||
assert.equal(isRpdExhausted("gemini-2.5-flash"), false);
|
||||
|
||||
const result = checkFallbackError(
|
||||
429,
|
||||
GEMINI_429_BODY,
|
||||
0,
|
||||
"gemini-2.5-flash",
|
||||
"gemini",
|
||||
null,
|
||||
PROFILE
|
||||
);
|
||||
|
||||
assert.equal(result.shouldFallback, true);
|
||||
assert.equal(result.reason, RateLimitReason.RATE_LIMIT_EXCEEDED);
|
||||
assert.ok(result.cooldownMs > 0, "cooldownMs should be positive");
|
||||
});
|
||||
|
||||
// ── Scenario 4: Both limits exhausted → RPD takes priority → QUOTA_EXHAUSTED ──
|
||||
|
||||
test("Gemini 2.5 Flash both RPM and RPD hit: RPD check runs first → QUOTA_EXHAUSTED", () => {
|
||||
for (let i = 0; i < 25; i++) incrementRequestCount("gemini-2.5-flash");
|
||||
assert.equal(isRpmExhausted("gemini-2.5-flash"), true);
|
||||
assert.equal(isRpdExhausted("gemini-2.5-flash"), true);
|
||||
|
||||
const result = checkFallbackError(
|
||||
429,
|
||||
GEMINI_429_BODY,
|
||||
0,
|
||||
"gemini-2.5-flash",
|
||||
"gemini",
|
||||
null,
|
||||
PROFILE
|
||||
);
|
||||
|
||||
// RPD check is first in the if-chain, so it takes priority
|
||||
assert.equal(result.reason, RateLimitReason.QUOTA_EXHAUSTED);
|
||||
});
|
||||
|
||||
// ── Scenario 5: Gemma 4 — 15 RPM hit, 1500 RPD not hit → RATE_LIMIT_EXCEEDED ─
|
||||
|
||||
test("Gemma 4 15 RPM hit (RPD=1500 untouched): 429 classifies as RATE_LIMIT_EXCEEDED", () => {
|
||||
for (let i = 0; i < 15; i++) incrementRequestCount("gemini/gemma-4-31b-it");
|
||||
assert.equal(isRpmExhausted("gemini/gemma-4-31b-it"), true);
|
||||
assert.equal(isRpdExhausted("gemini/gemma-4-31b-it"), false);
|
||||
|
||||
const result = checkFallbackError(
|
||||
429,
|
||||
GEMINI_429_BODY,
|
||||
0,
|
||||
"gemini/gemma-4-31b-it",
|
||||
"gemini",
|
||||
null,
|
||||
PROFILE
|
||||
);
|
||||
|
||||
assert.equal(result.reason, RateLimitReason.RATE_LIMIT_EXCEEDED);
|
||||
});
|
||||
|
||||
// ── Scenario 6: Non-Gemini provider bypasses the Gemini-specific check ─────────
|
||||
|
||||
test("Non-Gemini provider: tracker state is irrelevant, 429 goes through generic path", () => {
|
||||
for (let i = 0; i < 30; i++) incrementRequestCount("gemini-2.5-flash");
|
||||
assert.equal(isRpdExhausted("gemini-2.5-flash"), true);
|
||||
|
||||
// Provider is "openai" — Gemini-specific check is skipped
|
||||
const result = checkFallbackError(
|
||||
429,
|
||||
GEMINI_429_BODY,
|
||||
0,
|
||||
"gemini-2.5-flash",
|
||||
"openai",
|
||||
null,
|
||||
PROFILE
|
||||
);
|
||||
|
||||
// Falls through to generic 429 handling → RATE_LIMIT_EXCEEDED
|
||||
assert.equal(result.shouldFallback, true);
|
||||
assert.equal(result.reason, RateLimitReason.RATE_LIMIT_EXCEEDED);
|
||||
});
|
||||
|
||||
// ── Scenario 7: Reset clears state → no longer exhausted ──────────────────────
|
||||
|
||||
test("resetCounters clears both RPM and RPD exhaustion", () => {
|
||||
for (let i = 0; i < 20; i++) incrementRequestCount("gemini-2.5-flash");
|
||||
assert.equal(isRpmExhausted("gemini-2.5-flash"), true);
|
||||
assert.equal(isRpdExhausted("gemini-2.5-flash"), true);
|
||||
|
||||
resetCounters();
|
||||
|
||||
assert.equal(getDailyRequestCount("gemini-2.5-flash"), 0);
|
||||
assert.equal(getMinuteRequestCount("gemini-2.5-flash"), 0);
|
||||
assert.equal(isRpmExhausted("gemini-2.5-flash"), false);
|
||||
assert.equal(isRpdExhausted("gemini-2.5-flash"), false);
|
||||
|
||||
// Generic 429 path (no model-specific early return)
|
||||
const result = checkFallbackError(
|
||||
429,
|
||||
GEMINI_429_BODY,
|
||||
0,
|
||||
"gemini-2.5-flash",
|
||||
"gemini",
|
||||
null,
|
||||
PROFILE
|
||||
);
|
||||
|
||||
assert.equal(result.reason, RateLimitReason.RATE_LIMIT_EXCEEDED);
|
||||
});
|
||||
|
||||
// ── Scenario 8: RPD exhaustion with Gemma 4 (high RPD, never hit with 15 RPM) ─
|
||||
|
||||
test("Gemma 4 1500 RPD exhaustion overrides RPM classification", () => {
|
||||
// Pump 1500 daily requests to exhaust RPD
|
||||
for (let i = 0; i < 1500; i++) incrementRequestCount("gemini/gemma-4-31b-it");
|
||||
assert.equal(isRpdExhausted("gemini/gemma-4-31b-it"), true);
|
||||
assert.equal(isRpmExhausted("gemini/gemma-4-31b-it"), true); // 1500 >> 15 RPM
|
||||
|
||||
const result = checkFallbackError(
|
||||
429,
|
||||
GEMINI_429_BODY,
|
||||
0,
|
||||
"gemini/gemma-4-31b-it",
|
||||
"gemini",
|
||||
null,
|
||||
PROFILE
|
||||
);
|
||||
|
||||
// RPD check runs first
|
||||
assert.equal(result.reason, RateLimitReason.QUOTA_EXHAUSTED);
|
||||
});
|
||||
|
||||
// ── Scenario 9: Unknown model (no RPM/RPD in JSON) → generic 429 path ─────────
|
||||
|
||||
test("Unknown Gemini model without published limits falls through to generic 429", () => {
|
||||
incrementRequestCount("gemini/unknown-model");
|
||||
assert.equal(isRpmExhausted("gemini/unknown-model"), false);
|
||||
assert.equal(isRpdExhausted("gemini/unknown-model"), false);
|
||||
|
||||
const result = checkFallbackError(
|
||||
429,
|
||||
GEMINI_429_BODY,
|
||||
0,
|
||||
"gemini/unknown-model",
|
||||
"gemini",
|
||||
null,
|
||||
PROFILE
|
||||
);
|
||||
|
||||
assert.equal(result.reason, RateLimitReason.RATE_LIMIT_EXCEEDED);
|
||||
});
|
||||
@@ -30,6 +30,8 @@ const {
|
||||
isProviderFailureCode,
|
||||
getProvidersInCooldown,
|
||||
getProviderBreakerState,
|
||||
isCreditsExhausted,
|
||||
CREDITS_EXHAUSTED_SIGNALS,
|
||||
} = accountFallback;
|
||||
|
||||
const { selectAccount } = accountSelector;
|
||||
@@ -1115,6 +1117,182 @@ test("checkFallbackError ignores structured error with unrelated code on 400", (
|
||||
assert.equal(result.shouldFallback, false);
|
||||
});
|
||||
|
||||
// ─── Gemini RPM 429 Classification (CREDITS_EXHAUSTED_SIGNALS fix) ─────
|
||||
|
||||
test("isCreditsExhausted returns false for Gemini RPM 429 body text", () => {
|
||||
const geminiRpmText = "Resource has been exhausted (e.g. check quota).";
|
||||
assert.equal(isCreditsExhausted(geminiRpmText), false);
|
||||
});
|
||||
|
||||
test("isCreditsExhausted returns true for actual credits-exhausted signals", () => {
|
||||
assert.equal(isCreditsExhausted("insufficient_quota"), true);
|
||||
assert.equal(isCreditsExhausted("credits exhausted"), true);
|
||||
assert.equal(isCreditsExhausted("payment required"), true);
|
||||
assert.equal(isCreditsExhausted("free tier of the model has been exhausted"), true);
|
||||
assert.equal(isCreditsExhausted("exceeded your current usage quota"), true);
|
||||
});
|
||||
|
||||
test("CREDITS_EXHAUSTED_SIGNALS no longer contains generic gRPC resource-exhausted patterns", () => {
|
||||
// These patterns were removed because they falsely matched Gemini RPM 429 errors
|
||||
assert.equal(CREDITS_EXHAUSTED_SIGNALS.includes("resource has been exhausted"), false);
|
||||
assert.equal(CREDITS_EXHAUSTED_SIGNALS.includes("resource_exhausted"), false);
|
||||
assert.equal(CREDITS_EXHAUSTED_SIGNALS.includes("check quota"), false);
|
||||
});
|
||||
|
||||
test("checkFallbackError classifies Gemini RPM 429 as RATE_LIMIT_EXCEEDED (not QUOTA_EXHAUSTED)", () => {
|
||||
// provider=null → preserveQuota429=true → text quota checks run
|
||||
// isCreditsExhausted must NOT match Gemini's "Resource has been exhausted"
|
||||
const result = checkFallbackError(
|
||||
429,
|
||||
"Resource has been exhausted (e.g. check quota).",
|
||||
0,
|
||||
null,
|
||||
null,
|
||||
null,
|
||||
makeProfile()
|
||||
);
|
||||
assert.equal(result.shouldFallback, true);
|
||||
assert.equal(result.reason, RateLimitReason.RATE_LIMIT_EXCEEDED);
|
||||
assert.equal(result.creditsExhausted, undefined);
|
||||
assert.equal(result.dailyQuotaExhausted, undefined);
|
||||
assert.ok(result.cooldownMs > 0, "cooldownMs should be positive");
|
||||
});
|
||||
|
||||
test("checkFallbackError classifies Gemini RPM 429 as RATE_LIMIT_EXCEEDED for API-key provider", () => {
|
||||
// provider="gemini" → preserveQuota429=false → status-based rule applies
|
||||
const result = checkFallbackError(
|
||||
429,
|
||||
"Resource has been exhausted (e.g. check quota).",
|
||||
0,
|
||||
null,
|
||||
"gemini",
|
||||
null,
|
||||
makeProfile()
|
||||
);
|
||||
assert.equal(result.shouldFallback, true);
|
||||
assert.equal(result.reason, RateLimitReason.RATE_LIMIT_EXCEEDED);
|
||||
assert.equal(result.cooldownMs, 125); // makeProfile().baseCooldownMs
|
||||
});
|
||||
|
||||
test("checkFallbackError still classifies genuine OAuth quota-exhausted text as QUOTA_EXHAUSTED", () => {
|
||||
// Regression: OAuth providers must still get QUOTA_EXHAUSTED for actual quota messages
|
||||
const result = checkFallbackError(
|
||||
429,
|
||||
"Coding Plan hour quota has been exceeded",
|
||||
0,
|
||||
null,
|
||||
"codex",
|
||||
null,
|
||||
makeProfile()
|
||||
);
|
||||
assert.equal(result.shouldFallback, true);
|
||||
assert.equal(result.reason, RateLimitReason.QUOTA_EXHAUSTED);
|
||||
});
|
||||
|
||||
test("checkFallbackError preserves daily-quota exhaustion for non-429 status codes", () => {
|
||||
// Non-429 status codes with daily quota text must still be QUOTA_EXHAUSTED
|
||||
const result = checkFallbackError(
|
||||
402,
|
||||
"You have exceeded today's quota, please try again tomorrow"
|
||||
);
|
||||
assert.equal(result.shouldFallback, true);
|
||||
assert.equal(result.reason, RateLimitReason.QUOTA_EXHAUSTED);
|
||||
assert.equal(result.dailyQuotaExhausted, true);
|
||||
});
|
||||
|
||||
// ─── Gemini 429 → Model Lockout: rate_limited (not quota_exhausted) ────
|
||||
|
||||
test("Gemini RPM 429: recordModelLockoutFailure uses exponential backoff for rate_limited reason", () => {
|
||||
const originalNow = Date.now;
|
||||
const now = 1_700_000_000_000;
|
||||
Date.now = () => now;
|
||||
const provider = "gemini";
|
||||
const connectionId = "test-conn-gemini-rpm";
|
||||
const model = "gemini/gemma-4-31b-it";
|
||||
|
||||
try {
|
||||
clearModelLock(provider, connectionId, model);
|
||||
|
||||
const profile = makeProfile({
|
||||
baseCooldownMs: 5000,
|
||||
transientCooldown: 5000,
|
||||
rateLimitCooldown: 5000,
|
||||
});
|
||||
|
||||
// auth.ts flow: 429 + fallbackResult.reason=RATE_LIMIT_EXCEEDED
|
||||
// → reason="rate_limited" → recordModelLockoutFailure
|
||||
const first = recordModelLockoutFailure(
|
||||
provider,
|
||||
connectionId,
|
||||
model,
|
||||
"rate_limited",
|
||||
429,
|
||||
0,
|
||||
profile
|
||||
);
|
||||
assert.equal(first.failureCount, 1);
|
||||
assert.equal(first.cooldownMs, 5000, "first failure: 5s base cooldown");
|
||||
|
||||
const second = recordModelLockoutFailure(
|
||||
provider,
|
||||
connectionId,
|
||||
model,
|
||||
"rate_limited",
|
||||
429,
|
||||
0,
|
||||
profile
|
||||
);
|
||||
assert.equal(second.failureCount, 2);
|
||||
assert.equal(second.cooldownMs, 10000, "second failure: 10s exponential backoff");
|
||||
|
||||
assert.equal(isModelLocked(provider, connectionId, model), true);
|
||||
clearModelLock(provider, connectionId, model);
|
||||
} finally {
|
||||
Date.now = originalNow;
|
||||
clearModelLock("gemini", "test-conn-gemini-rpm", "gemini/gemma-4-31b-it");
|
||||
}
|
||||
});
|
||||
|
||||
test("Gemini RPD (quota_exhausted) still triggers midnight lockout in recordModelLockoutFailure", () => {
|
||||
// Regression: real daily quota exhaustion must still produce midnight reset
|
||||
const originalNow = Date.now;
|
||||
const testDate = new Date();
|
||||
testDate.setHours(12, 0, 0, 0);
|
||||
const now = testDate.getTime();
|
||||
Date.now = () => now;
|
||||
const provider = "gemini";
|
||||
const connectionId = "test-conn-gemini-rpd";
|
||||
const model = "gemini/gemma-4-31b-it";
|
||||
|
||||
try {
|
||||
clearModelLock(provider, connectionId, model);
|
||||
const profile = makeProfile();
|
||||
const result = recordModelLockoutFailure(
|
||||
provider,
|
||||
connectionId,
|
||||
model,
|
||||
"quota_exhausted",
|
||||
429,
|
||||
0,
|
||||
profile
|
||||
);
|
||||
|
||||
// Must lock until midnight, NOT exponential backoff
|
||||
const tomorrow = new Date(now);
|
||||
tomorrow.setDate(tomorrow.getDate() + 1);
|
||||
tomorrow.setHours(0, 0, 0, 0);
|
||||
const expected = tomorrow.getTime() - now;
|
||||
assert.ok(
|
||||
Math.abs(result.cooldownMs - expected) <= 300_000,
|
||||
`cooldown should be until tomorrow (expected ~${expected}, got ${result.cooldownMs})`
|
||||
);
|
||||
clearModelLock(provider, connectionId, model);
|
||||
} finally {
|
||||
Date.now = originalNow;
|
||||
clearModelLock("gemini", "test-conn-gemini-rpd", "gemini/gemma-4-31b-it");
|
||||
}
|
||||
});
|
||||
|
||||
// ─── G-02: X-Omni-Fallback-Hint: connection_cooldown ─────────────────────────
|
||||
// When 9router executor signals a supervisor-not-running 503, checkFallbackError
|
||||
// must return 5s cooldown with skipProviderBreaker:true — not trip the circuit breaker.
|
||||
|
||||
323
tests/unit/services/geminiRateLimitTracker.test.ts
Normal file
323
tests/unit/services/geminiRateLimitTracker.test.ts
Normal file
@@ -0,0 +1,323 @@
|
||||
import test from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import {
|
||||
getModelRpd,
|
||||
getModelRpm,
|
||||
incrementRequestCount,
|
||||
getDailyRequestCount,
|
||||
getMinuteRequestCount,
|
||||
isRpdExhausted,
|
||||
isRpmExhausted,
|
||||
resetCounters,
|
||||
} from "../../../open-sse/services/geminiRateLimitTracker.ts";
|
||||
|
||||
test.beforeEach(() => {
|
||||
resetCounters();
|
||||
});
|
||||
|
||||
// ── getModelRpd ──────────────────────────────────────────────────────────────
|
||||
|
||||
test("getModelRpd returns known RPD for exact model match", () => {
|
||||
assert.equal(getModelRpd("gemini-2.5-flash"), 20);
|
||||
});
|
||||
|
||||
test("getModelRpd returns known RPD for model with gemini/ prefix", () => {
|
||||
assert.equal(getModelRpd("gemini/gemini-2.5-flash"), 20);
|
||||
});
|
||||
|
||||
test("getModelRpd returns 0 for model with zero RPD (Gemini 2.5 Pro)", () => {
|
||||
assert.equal(getModelRpd("gemini-2.5-pro"), 0);
|
||||
});
|
||||
|
||||
test("getModelRpd returns 0 for unknown model", () => {
|
||||
assert.equal(getModelRpd("gemini/fake-model-not-in-list"), 0);
|
||||
});
|
||||
|
||||
test("getModelRpd returns 0 for empty string", () => {
|
||||
assert.equal(getModelRpd(""), 0);
|
||||
});
|
||||
|
||||
test("getModelRpd matches gemma-4-31b-it via suffix fallback", () => {
|
||||
// The JSON has "gemma-4-31b-it", and callers may pass "gemma-4-31b-it"
|
||||
assert.equal(getModelRpd("gemma-4-31b-it"), 1500);
|
||||
});
|
||||
|
||||
test("getModelRpd strips gemma- prefix correctly for gemma models", () => {
|
||||
// stripModelPrefix("gemini/gemma-4-31b-it") → "gemma-4-31b-it"
|
||||
assert.equal(getModelRpd("gemini/gemma-4-31b-it"), 1500);
|
||||
});
|
||||
|
||||
test("getModelRpd handles image-generation models (no RPM value, -1)", () => {
|
||||
// RPD is 25 for imagen models; RPM is -1 in the JSON
|
||||
assert.equal(getModelRpd("imagen-4-generate"), 25);
|
||||
});
|
||||
|
||||
test("getModelRpd handles models with unlimited RPD (-1)", () => {
|
||||
// gemini-3.5-live-translate has rpd: -1
|
||||
assert.equal(getModelRpd("gemini-3.5-live-translate"), 0);
|
||||
});
|
||||
|
||||
test("getModelRpd returns 0 for null input", () => {
|
||||
assert.equal(getModelRpd(null as unknown as string), 0);
|
||||
});
|
||||
|
||||
test("getModelRpd returns 0 for undefined input", () => {
|
||||
assert.equal(getModelRpd(undefined as unknown as string), 0);
|
||||
});
|
||||
|
||||
// ── incrementRequestCount / getDailyRequestCount ─────────────────────────────
|
||||
|
||||
test("incrementRequestCount starts at 1 for first request", () => {
|
||||
incrementRequestCount("gemini-2.5-flash");
|
||||
assert.equal(getDailyRequestCount("gemini-2.5-flash"), 1);
|
||||
});
|
||||
|
||||
test("incrementRequestCount increments sequentially", () => {
|
||||
incrementRequestCount("gemini-2.5-flash");
|
||||
incrementRequestCount("gemini-2.5-flash");
|
||||
incrementRequestCount("gemini-2.5-flash");
|
||||
assert.equal(getDailyRequestCount("gemini-2.5-flash"), 3);
|
||||
});
|
||||
|
||||
test("incrementRequestCount treats gemini/ prefix and bare name as same model", () => {
|
||||
incrementRequestCount("gemini-2.5-flash");
|
||||
incrementRequestCount("gemini/gemini-2.5-flash");
|
||||
assert.equal(getDailyRequestCount("gemini-2.5-flash"), 2);
|
||||
assert.equal(getDailyRequestCount("gemini/gemini-2.5-flash"), 2);
|
||||
});
|
||||
|
||||
test("models have independent counters", () => {
|
||||
incrementRequestCount("gemini-2.5-flash");
|
||||
incrementRequestCount("gemini-2.5-flash");
|
||||
incrementRequestCount("gemma-4-31b-it");
|
||||
assert.equal(getDailyRequestCount("gemini-2.5-flash"), 2);
|
||||
assert.equal(getDailyRequestCount("gemma-4-31b-it"), 1);
|
||||
});
|
||||
|
||||
test("getDailyRequestCount returns 0 for model with no requests", () => {
|
||||
assert.equal(getDailyRequestCount("gemini-2.5-flash"), 0);
|
||||
});
|
||||
|
||||
test("incrementRequestCount does nothing for empty model ID", () => {
|
||||
incrementRequestCount("");
|
||||
assert.equal(getDailyRequestCount("gemini-2.5-flash"), 0);
|
||||
});
|
||||
|
||||
test("resetCounters clears all state between tests", () => {
|
||||
incrementRequestCount("gemini-2.5-flash");
|
||||
incrementRequestCount("gemma-4-31b-it");
|
||||
resetCounters();
|
||||
assert.equal(getDailyRequestCount("gemini-2.5-flash"), 0);
|
||||
assert.equal(getDailyRequestCount("gemma-4-31b-it"), 0);
|
||||
});
|
||||
|
||||
// ── isRpdExhausted ───────────────────────────────────────────────────────────
|
||||
|
||||
test("isRpdExhausted returns false when count is below RPD limit", () => {
|
||||
// gemini-2.5-flash has RPD=20
|
||||
for (let i = 0; i < 19; i++) incrementRequestCount("gemini-2.5-flash");
|
||||
assert.equal(isRpdExhausted("gemini-2.5-flash"), false);
|
||||
});
|
||||
|
||||
test("isRpdExhausted returns true when count equals RPD limit", () => {
|
||||
// gemini-2.5-flash has RPD=20
|
||||
for (let i = 0; i < 20; i++) incrementRequestCount("gemini-2.5-flash");
|
||||
assert.equal(isRpdExhausted("gemini-2.5-flash"), true);
|
||||
});
|
||||
|
||||
test("isRpdExhausted returns true when count exceeds RPD limit", () => {
|
||||
for (let i = 0; i < 25; i++) incrementRequestCount("gemini-2.5-flash");
|
||||
assert.equal(isRpdExhausted("gemini-2.5-flash"), true);
|
||||
});
|
||||
|
||||
test("isRpdExhausted returns false for model with RPD=0", () => {
|
||||
// gemini-2.5-pro has rpd=0
|
||||
incrementRequestCount("gemini-2.5-pro");
|
||||
assert.equal(isRpdExhausted("gemini-2.5-pro"), false);
|
||||
});
|
||||
|
||||
test("isRpdExhausted returns false for unknown model", () => {
|
||||
incrementRequestCount("gemini/unknown-model");
|
||||
assert.equal(isRpdExhausted("gemini/unknown-model"), false);
|
||||
});
|
||||
|
||||
test("isRpdExhausted returns false for model with unlimited RPD (-1, no data)", () => {
|
||||
// gemini-3.5-live-translate has rpd: -1
|
||||
incrementRequestCount("gemini-3.5-live-translate");
|
||||
assert.equal(isRpdExhausted("gemini-3.5-live-translate"), false);
|
||||
});
|
||||
|
||||
test("isRpdExhausted works with gemini/ prefix for the model", () => {
|
||||
for (let i = 0; i < 20; i++) incrementRequestCount("gemini/gemini-2.5-flash");
|
||||
assert.equal(isRpdExhausted("gemini/gemini-2.5-flash"), true);
|
||||
});
|
||||
|
||||
// ── Integration: Gemma 4 RPM scenario ────────────────────────────────────────
|
||||
|
||||
test("Gemma 4: 15 RPM hits never trigger quota_exhausted (RPD=1500)", () => {
|
||||
// Gemma 4 has RPD=1500, so 15 RPM hits should not trigger RPD exhaustion
|
||||
for (let i = 0; i < 15; i++) incrementRequestCount("gemini/gemma-4-31b-it");
|
||||
assert.equal(isRpdExhausted("gemini/gemma-4-31b-it"), false);
|
||||
assert.equal(getDailyRequestCount("gemini/gemma-4-31b-it"), 15);
|
||||
});
|
||||
|
||||
test("Gemma 4: RPD exhaustion requires 1500 requests", () => {
|
||||
for (let i = 0; i < 1499; i++) incrementRequestCount("gemini/gemma-4-31b-it");
|
||||
assert.equal(isRpdExhausted("gemini/gemma-4-31b-it"), false);
|
||||
|
||||
incrementRequestCount("gemini/gemma-4-31b-it");
|
||||
assert.equal(isRpdExhausted("gemini/gemma-4-31b-it"), true);
|
||||
});
|
||||
|
||||
// ── Integration: Gemini 2.5 Flash RPD scenario ───────────────────────────────
|
||||
|
||||
test("Gemini 2.5 Flash: first 19 requests do NOT exhaust RPD, 20th does", () => {
|
||||
for (let i = 0; i < 19; i++) incrementRequestCount("gemini-2.5-flash");
|
||||
assert.equal(isRpdExhausted("gemini-2.5-flash"), false, "19 requests < 20 RPD");
|
||||
|
||||
incrementRequestCount("gemini-2.5-flash");
|
||||
assert.equal(isRpdExhausted("gemini-2.5-flash"), true, "20 requests = 20 RPD");
|
||||
});
|
||||
|
||||
test("Gemini 2.5 Flash: excess requests stay exhausted", () => {
|
||||
for (let i = 0; i < 22; i++) incrementRequestCount("gemini-2.5-flash");
|
||||
assert.equal(isRpdExhausted("gemini-2.5-flash"), true);
|
||||
assert.equal(getDailyRequestCount("gemini-2.5-flash"), 22);
|
||||
});
|
||||
|
||||
// ── getModelRpm ───────────────────────────────────────────────────────────────
|
||||
|
||||
test("getModelRpm returns known RPM for exact model match", () => {
|
||||
assert.equal(getModelRpm("gemini-2.5-flash"), 5);
|
||||
});
|
||||
|
||||
test("getModelRpm returns known RPM for model with gemini/ prefix", () => {
|
||||
assert.equal(getModelRpm("gemini/gemini-2.5-flash"), 5);
|
||||
});
|
||||
|
||||
test("getModelRpm returns 0 for model with zero RPM", () => {
|
||||
assert.equal(getModelRpm("gemini-2.5-pro"), 0);
|
||||
});
|
||||
|
||||
test("getModelRpm returns 0 for unknown model", () => {
|
||||
assert.equal(getModelRpm("gemini/fake-model-not-in-list"), 0);
|
||||
});
|
||||
|
||||
test("getModelRpm returns 0 for empty string", () => {
|
||||
assert.equal(getModelRpm(""), 0);
|
||||
});
|
||||
|
||||
test("getModelRpm returns 0 for models with RPM=-1 (imagen)", () => {
|
||||
assert.equal(getModelRpm("imagen-4-generate"), 0);
|
||||
});
|
||||
|
||||
test("getModelRpm returns 0 for null input", () => {
|
||||
assert.equal(getModelRpm(null as unknown as string), 0);
|
||||
});
|
||||
|
||||
// ── incrementMinuteRequestCount / getMinuteRequestCount ───────────────────────
|
||||
|
||||
test("getMinuteRequestCount returns 0 before first request", () => {
|
||||
assert.equal(getMinuteRequestCount("gemini-2.5-flash"), 0);
|
||||
});
|
||||
|
||||
test("incrementMinuteRequestCount starts at 1", () => {
|
||||
incrementRequestCount("gemini-2.5-flash");
|
||||
assert.equal(getMinuteRequestCount("gemini-2.5-flash"), 1);
|
||||
});
|
||||
|
||||
test("incrementMinuteRequestCount increments with each call", () => {
|
||||
incrementRequestCount("gemini-2.5-flash");
|
||||
incrementRequestCount("gemini-2.5-flash");
|
||||
incrementRequestCount("gemini-2.5-flash");
|
||||
assert.equal(getMinuteRequestCount("gemini-2.5-flash"), 3);
|
||||
});
|
||||
|
||||
test("incrementMinuteRequestCount normalizes gemini/ prefix", () => {
|
||||
incrementRequestCount("gemini-2.5-flash");
|
||||
incrementRequestCount("gemini/gemini-2.5-flash");
|
||||
assert.equal(getMinuteRequestCount("gemini-2.5-flash"), 2);
|
||||
});
|
||||
|
||||
test("minute counters are independent per model", () => {
|
||||
incrementRequestCount("gemini-2.5-flash");
|
||||
incrementRequestCount("gemini-2.5-flash");
|
||||
incrementRequestCount("gemma-4-31b-it");
|
||||
assert.equal(getMinuteRequestCount("gemini-2.5-flash"), 2);
|
||||
assert.equal(getMinuteRequestCount("gemma-4-31b-it"), 1);
|
||||
});
|
||||
|
||||
test("incrementRequestCount does nothing for empty model ID", () => {
|
||||
incrementRequestCount("");
|
||||
assert.equal(getMinuteRequestCount("gemini-2.5-flash"), 0);
|
||||
});
|
||||
|
||||
test("resetCounters clears minute windows", () => {
|
||||
incrementRequestCount("gemini-2.5-flash");
|
||||
incrementRequestCount("gemma-4-31b-it");
|
||||
resetCounters();
|
||||
assert.equal(getMinuteRequestCount("gemini-2.5-flash"), 0);
|
||||
assert.equal(getMinuteRequestCount("gemma-4-31b-it"), 0);
|
||||
});
|
||||
|
||||
// ── isRpmExhausted ────────────────────────────────────────────────────────────
|
||||
|
||||
test("isRpmExhausted returns false below RPM limit", () => {
|
||||
// gemini-2.5-flash has RPM=5
|
||||
for (let i = 0; i < 4; i++) incrementRequestCount("gemini-2.5-flash");
|
||||
assert.equal(isRpmExhausted("gemini-2.5-flash"), false);
|
||||
});
|
||||
|
||||
test("isRpmExhausted returns true at RPM limit", () => {
|
||||
for (let i = 0; i < 5; i++) incrementRequestCount("gemini-2.5-flash");
|
||||
assert.equal(isRpmExhausted("gemini-2.5-flash"), true);
|
||||
});
|
||||
|
||||
test("isRpmExhausted returns true above RPM limit", () => {
|
||||
for (let i = 0; i < 8; i++) incrementRequestCount("gemini-2.5-flash");
|
||||
assert.equal(isRpmExhausted("gemini-2.5-flash"), true);
|
||||
});
|
||||
|
||||
test("isRpmExhausted returns false for model with RPM=0", () => {
|
||||
incrementRequestCount("gemini-2.5-pro");
|
||||
assert.equal(isRpmExhausted("gemini-2.5-pro"), false);
|
||||
});
|
||||
|
||||
test("isRpmExhausted returns false for unknown model", () => {
|
||||
incrementRequestCount("gemini/unknown-model");
|
||||
assert.equal(isRpmExhausted("gemini/unknown-model"), false);
|
||||
});
|
||||
|
||||
test("isRpmExhausted returns false for model with RPM=-1 (imagen)", () => {
|
||||
incrementRequestCount("imagen-4-generate");
|
||||
assert.equal(isRpmExhausted("imagen-4-generate"), false);
|
||||
});
|
||||
|
||||
test("isRpmExhausted works with gemini/ prefix", () => {
|
||||
for (let i = 0; i < 5; i++) incrementRequestCount("gemini/gemini-2.5-flash");
|
||||
assert.equal(isRpmExhausted("gemini/gemini-2.5-flash"), true);
|
||||
});
|
||||
|
||||
// ── Integration: RPM + RPD work independently ─────────────────────────────────
|
||||
|
||||
test("RPM and RPD limits are tracked independently", () => {
|
||||
// Gemma 4: RPM=15, RPD=1500
|
||||
// After 15 requests, RPM is exhausted but RPD is not
|
||||
for (let i = 0; i < 15; i++) incrementRequestCount("gemini/gemma-4-31b-it");
|
||||
assert.equal(isRpmExhausted("gemini/gemma-4-31b-it"), true);
|
||||
assert.equal(isRpdExhausted("gemini/gemma-4-31b-it"), false);
|
||||
});
|
||||
|
||||
test("Gemini 2.5 Flash: RPM=5 always hits before RPD=20", () => {
|
||||
// After 5 requests, RPM is exhausted; after 20, RPD is exhausted
|
||||
for (let i = 0; i < 5; i++) incrementRequestCount("gemini-2.5-flash");
|
||||
assert.equal(isRpmExhausted("gemini-2.5-flash"), true);
|
||||
assert.equal(isRpdExhausted("gemini-2.5-flash"), false);
|
||||
|
||||
incrementRequestCount("gemini-2.5-flash"); // 6
|
||||
incrementRequestCount("gemini-2.5-flash"); // 7
|
||||
incrementRequestCount("gemini-2.5-flash"); // 8
|
||||
|
||||
// Still at RPD=8 which is < 20
|
||||
assert.equal(isRpdExhausted("gemini-2.5-flash"), false);
|
||||
});
|
||||
Reference in New Issue
Block a user