feat(sse): add LLM Gateway DevPass quota tracking (#12462)

* feat(sse): add LLM Gateway DevPass quota tracking

Surface the LLM Gateway DevPass allowance (GET /v1/key) in OmniRoute's
quota telemetry, mirroring the OpenRouter API-key fetcher pattern.

- llmgatewayQuotaFetcher.ts: fetch + parse the DevPass /v1/key response
  (decimal-string USD values), exposing two windows — monthly plan
  credits and the 7-day premium-model window — with a 45s TTL cache.
  Pay-as-you-go keys (devPlan "none") and 401/403 fail open (no quota).
- Register in chat.ts before registerGenericQuotaFetchers + register the
  named windows for the dashboard cutoff modal.
- usage/llmgateway.ts leaf + usage.ts dispatch case so the Limits page
  renders the monthly + weekly premium rows.
- Add "llmgateway" to USAGE_FETCHER_PROVIDERS, USAGE_SUPPORTED_PROVIDERS,
  PROVIDER_LIMITS_APIKEY_PROVIDERS, and the dashboard label/order map.
- tests: 21 cases covering the parser, auth fail-open, cache TTL, window
  exhaustion, preflight proceed/block, registration, and the usage leaf.

* docs(sse): add changelog fragment + codebase-doc entry for llmgateway quota

* refactor(sse): register llmgateway quota via quotaTrackersBatch

Move the LLM Gateway fetcher registration out of chat.ts (a frozen
file-size-baseline chokepoint) into quotaTrackersBatch.ts, the dedicated
side-effect module that exists precisely so new fetchers don't grow
chat.ts. The batch import runs at module load, before
registerGenericQuotaFetchers(), so the bespoke fetcher still wins over
the generic path. Fixes the file-size gate (chat.ts must not grow).

---------

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
This commit is contained in:
Pixma
2026-09-18 16:58:05 +02:00
committed by GitHub
parent f1eabd8885
commit 01b2467d61
11 changed files with 797 additions and 16 deletions

View File

@@ -0,0 +1 @@
- **feat(sse):** track LLM Gateway DevPass quota — the `llmgateway` provider now reads its monthly plan-credit and weekly premium-model allowance from `GET /v1/key` and surfaces both windows in Dashboard Limits and quota-aware preflight ([#12462](https://github.com/diegosouzapw/OmniRoute/pull/12462)).

View File

@@ -519,21 +519,21 @@ Hub-and-spoke translation (OpenAI is the hub).
Highlights (full list under `open-sse/services/`):
| Concern | Files |
| -------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Combo routing | `combo.ts` (19 strategies), `comboConfig.ts`, `comboMetrics.ts`, `comboManifestMetrics.ts`, `comboAgentMiddleware.ts` |
| Auto Combo engine | `autoCombo/``engine.ts`, `scoring.ts`, `taskFitness.ts`, `virtualFactory.ts`, `modePacks.ts`, `autoPrefix.ts`, `persistence.ts`, `providerDiversity.ts`, `providerRegistryAccessor.ts`, `routerStrategy.ts`, `selfHealing.ts`, `index.ts` |
| Resilience | `accountFallback.ts` (cooldown + lockout), `errorClassifier.ts`, `requestRejectedStreak.ts`, `emergencyFallback.ts`, `rateLimitManager.ts`, `rateLimitSemaphore.ts`, `accountSemaphore.ts`, `accountSelector.ts` |
| Quotas | `quotaMonitor.ts`, `quotaPreflight.ts`, `bailianQuotaFetcher.ts`, `codexQuotaFetcher.ts`, `deepseekQuotaFetcher.ts`, `openrouterQuotaFetcher.ts`, `openrouterFreeWindow.ts`, `crofUsageFetcher.ts`, `antigravityCredits.ts` |
| Caching | `reasoningCache.ts`, `searchCache.ts`, `signatureCache.ts`, `requestDedup.ts` |
| Routing intelligence | `intentClassifier.ts`, `taskAwareRouter.ts`, `backgroundTaskDetector.ts`, `volumeDetector.ts`, `wildcardRouter.ts`, `workflowFSM.ts`, `specificityDetector.ts`, `specificityRules.ts`, `specificityTypes.ts` |
| Model handling | `modelCapabilities.ts`, `modelDeprecation.ts`, `modelFamilyFallback.ts`, `modelStrip.ts`, `model.ts`, `provider.ts`, `providerRequestDefaults.ts`, `providerCostData.ts`, `payloadRules.ts` |
| Compression | `compression/` — full compression engine wiring |
| Token + session | `tokenRefresh.ts`, `sessionManager.ts`, `apiKeyRotator.ts`, `contextManager.ts`, `contextHandoff.ts`, `systemPrompt.ts`, `roleNormalizer.ts`, `responsesInputSanitizer.ts`, `toolSchemaSanitizer.ts`, `toolLimitDetector.ts`, `thinkingBudget.ts` |
| Tier / manifest | `tierResolver.ts`, `tierConfig.ts`, `tierDefaults.json`, `tierTypes.ts`, `manifestAdapter.ts` |
| IP / network | `ipFilter.ts`, `webSearchFallback.ts` |
| Batches | `batchProcessor.ts` |
| Usage | `usage.ts` |
| Concern | Files |
| -------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Combo routing | `combo.ts` (19 strategies), `comboConfig.ts`, `comboMetrics.ts`, `comboManifestMetrics.ts`, `comboAgentMiddleware.ts` |
| Auto Combo engine | `autoCombo/``engine.ts`, `scoring.ts`, `taskFitness.ts`, `virtualFactory.ts`, `modePacks.ts`, `autoPrefix.ts`, `persistence.ts`, `providerDiversity.ts`, `providerRegistryAccessor.ts`, `routerStrategy.ts`, `selfHealing.ts`, `index.ts` |
| Resilience | `accountFallback.ts` (cooldown + lockout), `errorClassifier.ts`, `requestRejectedStreak.ts`, `emergencyFallback.ts`, `rateLimitManager.ts`, `rateLimitSemaphore.ts`, `accountSemaphore.ts`, `accountSelector.ts` |
| Quotas | `quotaMonitor.ts`, `quotaPreflight.ts`, `bailianQuotaFetcher.ts`, `codexQuotaFetcher.ts`, `deepseekQuotaFetcher.ts`, `openrouterQuotaFetcher.ts`, `openrouterFreeWindow.ts`, `llmgatewayQuotaFetcher.ts`, `crofUsageFetcher.ts`, `antigravityCredits.ts` |
| Caching | `reasoningCache.ts`, `searchCache.ts`, `signatureCache.ts`, `requestDedup.ts` |
| Routing intelligence | `intentClassifier.ts`, `taskAwareRouter.ts`, `backgroundTaskDetector.ts`, `volumeDetector.ts`, `wildcardRouter.ts`, `workflowFSM.ts`, `specificityDetector.ts`, `specificityRules.ts`, `specificityTypes.ts` |
| Model handling | `modelCapabilities.ts`, `modelDeprecation.ts`, `modelFamilyFallback.ts`, `modelStrip.ts`, `model.ts`, `provider.ts`, `providerRequestDefaults.ts`, `providerCostData.ts`, `payloadRules.ts` |
| Compression | `compression/` — full compression engine wiring |
| Token + session | `tokenRefresh.ts`, `sessionManager.ts`, `apiKeyRotator.ts`, `contextManager.ts`, `contextHandoff.ts`, `systemPrompt.ts`, `roleNormalizer.ts`, `responsesInputSanitizer.ts`, `toolSchemaSanitizer.ts`, `toolLimitDetector.ts`, `thinkingBudget.ts` |
| Tier / manifest | `tierResolver.ts`, `tierConfig.ts`, `tierDefaults.json`, `tierTypes.ts`, `manifestAdapter.ts` |
| IP / network | `ipFilter.ts`, `webSearchFallback.ts` |
| Batches | `batchProcessor.ts` |
| Usage | `usage.ts` |
### 4.6 `open-sse/mcp-server/`

View File

@@ -0,0 +1,325 @@
/**
* llmgatewayQuotaFetcher.ts — LLM Gateway DevPass Quota Fetcher
*
* Implements QuotaFetcher for the "llmgateway" provider (quotaPreflight.ts +
* quotaMonitor.ts).
*
* LLM Gateway exposes one official, documented monitoring endpoint
* (https://docs.llmgateway.io/developers/devpass-usage):
*
* GET https://api.llmgateway.io/v1/key
* -> { data: { label, usage, limit, devPlan,
* devPlanCreditsUsed, devPlanCreditsLimit, devPlanCreditsRemaining,
* devPlanPremiumWeeklyLimit, devPlanPremiumCreditsUsed,
* devPlanPremiumWeekResetsAt } }
*
* Authenticated with the connection's own gateway API key (`llmgtwy_…`) as a
* Bearer token — no dashboard session needed. Every USD value comes back as a
* decimal STRING and must be parsed before doing math (per the upstream docs).
*
* DevPass has two independent allowance windows, both surfaced here:
* - monthly: the plan credit cycle (devPlanCreditsUsed / …Limit / …Remaining) —
* this is the ~$87/mo DevPass allowance.
* - premium (weekly): the premium-model window (devPlanPremiumWeeklyLimit /
* …CreditsUsed / …WeekResetsAt). The window starts on the first premium
* request and lasts 7 days; when it expires the endpoint returns "0.00"
* used and a null reset (full allowance available again).
*
* Pay-as-you-go keys return devPlan "none" and zero for every DevPass field —
* there is no subscription allowance to track, so we return null (no quota) and
* let routing/preflight treat the connection as unlimited.
*
* Graceful "unknown" on any fetch failure — quota tracking must never block
* routing (mirrors deepseekQuotaFetcher.ts / openrouterQuotaFetcher.ts). A 401
* (invalid/inactive key) or 403 (publishable/session key, which cannot read
* org-level plan state) drops the cache and returns null.
*
* Cache: in-memory TTL (45s) keyed by connectionId, so combo preflight/monitor
* polling doesn't hammer the upstream on every request.
*
* Registration: call registerLlmgatewayQuotaFetcher() once at server startup.
*/
import {
registerQuotaFetcher,
registerQuotaWindows,
type QuotaInfo,
type QuotaWindowInfo,
} from "./quotaPreflight.ts";
import { registerMonitorFetcher } from "./quotaMonitor.ts";
import { throttleQuotaFetch } from "./quotaFetchThrottle.ts";
const LLMGATEWAY_CONFIG = {
baseUrl: "https://api.llmgateway.io/v1",
keyPath: "/key",
};
// Cache TTL — same 45s window as the OpenRouter fetcher.
const CACHE_TTL_MS = 45_000;
// Canonical window names surfaced to the dashboard cutoff modal + usage leaf.
export const LLMGATEWAY_WINDOW_MONTHLY = "devpass_monthly";
export const LLMGATEWAY_WINDOW_PREMIUM_WEEKLY = "devpass_premium_weekly";
export interface LlmgatewayQuota extends QuotaInfo {
/** Plan tier: "lite" | "pro" | "max" | "none". */
devPlan: string;
/** Monthly plan credit cycle (USD). */
monthlyUsed: number;
monthlyLimit: number | null;
monthlyRemaining: number | null;
/** Weekly premium-model window (USD). */
premiumWeeklyLimit: number | null;
premiumUsed: number;
premiumRemaining: number | null;
premiumResetAt: string | null;
}
interface CacheEntry {
quota: LlmgatewayQuota;
fetchedAt: number;
}
const quotaCache = new Map<string, CacheEntry>();
const _cacheCleanup = setInterval(() => {
const now = Date.now();
for (const [key, entry] of quotaCache) {
if (now - entry.fetchedAt > CACHE_TTL_MS * 5) {
quotaCache.delete(key);
}
}
}, 5 * 60_000);
if (typeof _cacheCleanup === "object" && "unref" in _cacheCleanup) {
(_cacheCleanup as { unref?: () => void }).unref?.();
}
// ─── Helpers ─────────────────────────────────────────────────────────────────
function toRecord(value: unknown): Record<string, unknown> {
return value && typeof value === "object" && !Array.isArray(value)
? (value as Record<string, unknown>)
: {};
}
/**
* Parse a DevPass USD value. Upstream returns decimal STRINGS ("31.42"), but we
* also accept plain numbers defensively. Returns null for absent/blank/invalid.
*/
function toNullableNumber(value: unknown): number | null {
if (typeof value === "number" && Number.isFinite(value)) return value;
if (typeof value === "string" && value.trim().length > 0) {
const parsed = Number(value);
if (Number.isFinite(parsed)) return parsed;
}
return null;
}
function toFiniteNumber(value: unknown, fallback = 0): number {
const n = toNullableNumber(value);
return n === null ? fallback : n;
}
function toIsoOrNull(value: unknown): string | null {
if (typeof value !== "string" || value.trim().length === 0) return null;
const date = new Date(value);
if (Number.isNaN(date.getTime()) || date.getTime() <= 0) return null;
return date.toISOString();
}
// ─── Response Parser ─────────────────────────────────────────────────────────
export interface LlmgatewayKeyFields {
devPlan: string;
monthlyUsed: number;
monthlyLimit: number | null;
monthlyRemaining: number | null;
premiumWeeklyLimit: number | null;
premiumUsed: number;
premiumResetAt: string | null;
}
/**
* Parse the `GET /v1/key` response body. Returns null when the payload doesn't
* carry a recognizable `data` object (e.g. an unexpected shape).
*/
export function parseLlmgatewayKeyResponse(data: unknown): LlmgatewayKeyFields | null {
const outer = toRecord(data);
const inner = "data" in outer ? toRecord(outer.data) : outer;
if (Object.keys(inner).length === 0) return null;
const devPlan =
typeof inner.devPlan === "string" && inner.devPlan.trim().length > 0 ? inner.devPlan : "none";
return {
devPlan,
monthlyUsed: toFiniteNumber(inner.devPlanCreditsUsed, 0),
monthlyLimit: toNullableNumber(inner.devPlanCreditsLimit),
monthlyRemaining: toNullableNumber(inner.devPlanCreditsRemaining),
premiumWeeklyLimit: toNullableNumber(inner.devPlanPremiumWeeklyLimit),
premiumUsed: toFiniteNumber(inner.devPlanPremiumCreditsUsed, 0),
premiumResetAt: toIsoOrNull(inner.devPlanPremiumWeekResetsAt),
};
}
function percentUsedFrom(used: number, limit: number | null): number {
if (limit === null || limit <= 0) return 0;
const pct = used / limit;
return Math.min(1, Math.max(0, pct));
}
/**
* Build the QuotaInfo from parsed fields. Returns null for pay-as-you-go keys
* (`devPlan === "none"`) — there is no subscription allowance to enforce, so the
* connection is treated as unlimited by preflight/routing.
*/
export function buildLlmgatewayQuota(key: LlmgatewayKeyFields): LlmgatewayQuota | null {
if (key.devPlan === "none") return null;
const monthlyPercent = percentUsedFrom(key.monthlyUsed, key.monthlyLimit);
const premiumPercent = percentUsedFrom(key.premiumUsed, key.premiumWeeklyLimit);
const monthlyRemaining =
key.monthlyRemaining ??
(key.monthlyLimit !== null ? Math.max(0, key.monthlyLimit - key.monthlyUsed) : null);
const premiumRemaining =
key.premiumWeeklyLimit !== null ? Math.max(0, key.premiumWeeklyLimit - key.premiumUsed) : null;
const monthlyExhausted = key.monthlyLimit !== null && monthlyPercent >= 1;
const premiumExhausted = key.premiumWeeklyLimit !== null && premiumPercent >= 1;
const windows: Record<string, QuotaWindowInfo> = {};
const windowMonthly: QuotaWindowInfo | undefined =
key.monthlyLimit !== null ? { percentUsed: monthlyPercent, resetAt: null } : undefined;
const windowWeekly: QuotaWindowInfo | undefined =
key.premiumWeeklyLimit !== null
? { percentUsed: premiumPercent, resetAt: key.premiumResetAt }
: undefined;
if (windowMonthly) windows[LLMGATEWAY_WINDOW_MONTHLY] = windowMonthly;
if (windowWeekly) windows[LLMGATEWAY_WINDOW_PREMIUM_WEEKLY] = windowWeekly;
// Legacy single-signal percentUsed = worst of the two windows.
const worstPercent = Math.max(monthlyPercent, premiumPercent);
return {
used: worstPercent * 100,
total: 100,
percentUsed: worstPercent,
resetAt: key.premiumResetAt,
limitReached: monthlyExhausted || premiumExhausted,
windows: Object.keys(windows).length > 0 ? windows : undefined,
windowWeekly,
windowMonthly,
devPlan: key.devPlan,
monthlyUsed: key.monthlyUsed,
monthlyLimit: key.monthlyLimit,
monthlyRemaining,
premiumWeeklyLimit: key.premiumWeeklyLimit,
premiumUsed: key.premiumUsed,
premiumRemaining,
premiumResetAt: key.premiumResetAt,
};
}
// ─── Core Fetcher ────────────────────────────────────────────────────────────
async function fetchJson(
url: string,
apiKey: string
): Promise<{ status: number; data: unknown } | null> {
try {
const response = await fetch(url, {
method: "GET",
headers: {
Authorization: `Bearer ${apiKey}`,
Accept: "application/json",
},
signal: AbortSignal.timeout(8_000),
});
if (!response.ok) return { status: response.status, data: null };
const data = await response.json();
return { status: response.status, data };
} catch {
return null;
}
}
/**
* Fetch current DevPass quota for an LLM Gateway connection.
*
* @param connectionId - Connection ID from the DB (used for cache keying)
* @param connection - Optional connection object with apiKey
* @returns LlmgatewayQuota, or null on no credentials / fetch failure / auth
* rejection / pay-as-you-go key
*/
export async function fetchLlmgatewayQuota(
connectionId: string,
connection?: Record<string, unknown>
): Promise<QuotaInfo | null> {
const cached = quotaCache.get(connectionId);
if (cached && Date.now() - cached.fetchedAt < CACHE_TTL_MS) {
return cached.quota;
}
const apiKey =
typeof connection?.apiKey === "string" && connection.apiKey.trim().length > 0
? connection.apiKey
: null;
if (!apiKey) return null;
try {
await throttleQuotaFetch();
const keyResult = await fetchJson(
`${LLMGATEWAY_CONFIG.baseUrl}${LLMGATEWAY_CONFIG.keyPath}`,
apiKey
);
// Auth-rejected (invalid key = 401, publishable/session key = 403) or a
// network failure — fail open and drop any stale cache.
if (!keyResult || keyResult.status !== 200) {
quotaCache.delete(connectionId);
return null;
}
const fields = parseLlmgatewayKeyResponse(keyResult.data);
if (!fields) {
quotaCache.delete(connectionId);
return null;
}
const quota = buildLlmgatewayQuota(fields);
if (!quota) {
// Pay-as-you-go key: no subscription allowance to track.
quotaCache.delete(connectionId);
return null;
}
quotaCache.set(connectionId, { quota, fetchedAt: Date.now() });
return quota;
} catch {
// Network error, timeout, etc. — fail open (graceful "unknown").
return null;
}
}
// ─── Invalidation ────────────────────────────────────────────────────────────
export function invalidateLlmgatewayQuotaCache(connectionId: string): void {
quotaCache.delete(connectionId);
}
// ─── Registration ─────────────────────────────────────────────────────────────
/**
* Register the LLM Gateway DevPass quota fetcher with the preflight and monitor
* systems, plus its named windows for the dashboard cutoff modal. Call this once
* at server startup (before registerGenericQuotaFetchers so the bespoke fetcher
* wins over the generic path).
*/
export function registerLlmgatewayQuotaFetcher(): void {
registerQuotaFetcher("llmgateway", fetchLlmgatewayQuota);
registerMonitorFetcher("llmgateway", fetchLlmgatewayQuota);
registerQuotaWindows("llmgateway", [LLMGATEWAY_WINDOW_MONTHLY, LLMGATEWAY_WINDOW_PREMIUM_WEEKLY]);
}

View File

@@ -1,6 +1,7 @@
/**
* quotaTrackersBatch.ts — startup registration for batch quota trackers
* (AgentRouter, v0-vercel, freemodel-dev, grok-cli, xai-oauth, firecrawl).
* (AgentRouter, v0-vercel, freemodel-dev, grok-cli, xai-oauth, firecrawl,
* llmgateway).
*
* Kept in a dedicated module (rather than adding more inline calls to
* `src/sse/handlers/chat.ts`, which is a frozen file at its LOC baseline) so the
@@ -13,6 +14,7 @@ import { registerFreeModelQuotaFetcher } from "./freeModelQuotaFetcher.ts";
import { registerGrokCliQuotaFetcher } from "./grokCliQuotaFetcher.ts";
import { registerXaiOauthQuotaFetcher } from "./xaiOauthQuotaFetcher.ts";
import { registerFirecrawlQuotaFetcher } from "./firecrawlQuotaFetcher.ts";
import { registerLlmgatewayQuotaFetcher } from "./llmgatewayQuotaFetcher.ts";
export function registerQuotaTrackersBatch(): void {
registerAgentrouterQuotaFetcher();
@@ -21,6 +23,7 @@ export function registerQuotaTrackersBatch(): void {
registerGrokCliQuotaFetcher();
registerXaiOauthQuotaFetcher();
registerFirecrawlQuotaFetcher();
registerLlmgatewayQuotaFetcher();
}
// Side-effect registration at module load, mirroring the sibling

View File

@@ -49,6 +49,7 @@ import { getKiroUsage, buildKiroUsageResult, discoverKiroProfileArn } from "./us
export { buildKiroUsageResult, discoverKiroProfileArn } from "./usage/kiro.ts";
import { getAdobeFireflyUsage } from "./usage/adobeFirefly.ts";
import { getOpenrouterUsage } from "./usage/openrouter.ts";
import { getLlmgatewayUsage } from "./usage/llmgateway.ts";
import { getOllamaCloudUsage } from "./opencodeOllamaUsage.ts";
import { getCodeBuddyCnUsage } from "./usage/codebuddy-cn.ts";
import { getPromptQlUsage } from "./usage/promptql.ts";
@@ -179,6 +180,8 @@ export async function getUsageForProvider(
return await getMoonshotOpenPlatformUsage(connection);
case "openrouter":
return await getOpenrouterUsage(id || "", apiKey || "", providerSpecificData);
case "llmgateway":
return await getLlmgatewayUsage(id || "", apiKey || "");
case "opencode":
case "opencode-zen":
return await getOpencodeUsage(id || "", apiKey || "");

View File

@@ -58,6 +58,8 @@ export const USAGE_FETCHER_PROVIDERS = [
"vertex-partner",
"codebuddy-cn",
"openrouter",
// LLM Gateway DevPass allowance (GET /v1/key → monthly + weekly premium)
"llmgateway",
// PromptQL playground credits (data.pro.ql.app getCreditSummary)
"promptql",
"pql",

View File

@@ -0,0 +1,95 @@
/**
* usage/llmgateway.ts — LLM Gateway DevPass usage-dashboard builder
*
* Extracted as a leaf module (not inlined in usage.ts) so the god-file stays
* flat: this owns turning an LlmgatewayQuota into the `UsageQuota` rows the
* Dashboard → Usage / Limits page renders, mirroring getOpenrouterUsage's
* pattern.
*
* DevPass exposes two allowance windows:
* - monthly plan credits (the ~$87/mo DevPass allowance)
* - weekly premium-model credits (7-day rolling window)
* Each maps to one UsageQuota row. Pay-as-you-go keys (devPlan "none") have no
* allowance, so the fetcher returns null and we surface a plain connected state.
*/
import {
fetchLlmgatewayQuota,
type LlmgatewayQuota,
LLMGATEWAY_WINDOW_MONTHLY,
LLMGATEWAY_WINDOW_PREMIUM_WEEKLY,
} from "../llmgatewayQuotaFetcher.ts";
import { type UsageQuota } from "./quota.ts";
function remainingPct(used: number, limit: number | null): number {
if (limit === null || limit <= 0) return 100;
return Math.max(0, Math.round((1 - used / limit) * 100));
}
function buildMonthlyQuota(quota: LlmgatewayQuota): UsageQuota | null {
if (quota.monthlyLimit === null) return null;
return {
used: quota.monthlyUsed,
total: quota.monthlyLimit,
remaining: quota.monthlyRemaining ?? undefined,
remainingPercentage: remainingPct(quota.monthlyUsed, quota.monthlyLimit),
resetAt: null,
unlimited: false,
displayName: "DevPass credits (monthly)",
currency: "USD",
};
}
function buildPremiumQuota(quota: LlmgatewayQuota): UsageQuota | null {
if (quota.premiumWeeklyLimit === null) return null;
return {
used: quota.premiumUsed,
total: quota.premiumWeeklyLimit,
remaining: quota.premiumRemaining ?? undefined,
remainingPercentage: remainingPct(quota.premiumUsed, quota.premiumWeeklyLimit),
resetAt: quota.premiumResetAt,
unlimited: false,
displayName: "Premium credits (weekly)",
currency: "USD",
};
}
function planLabel(devPlan: string): string {
const tier = devPlan.charAt(0).toUpperCase() + devPlan.slice(1);
return `LLM Gateway DevPass (${tier})`;
}
/**
* LLM Gateway Usage — reads the DevPass allowance from GET /v1/key and returns
* the monthly + weekly premium windows as dashboard quota rows.
*/
export async function getLlmgatewayUsage(connectionId: string, apiKey: string) {
if (!apiKey) {
return { message: "LLM Gateway API key not available. Add a key to view usage." };
}
const connection = { apiKey };
const quota = (await fetchLlmgatewayQuota(connectionId, connection)) as LlmgatewayQuota | null;
if (!quota) {
// No DevPass allowance (pay-as-you-go key), unreachable endpoint, or an
// invalid/publishable key — surface a plain connected state.
return {
plan: "LLM Gateway (pay-as-you-go)",
quotas: {},
message: "LLM Gateway connected. No DevPass allowance on this key.",
};
}
const quotas: Record<string, UsageQuota> = {};
const monthly = buildMonthlyQuota(quota);
const premium = buildPremiumQuota(quota);
if (monthly) quotas[LLMGATEWAY_WINDOW_MONTHLY] = monthly;
if (premium) quotas[LLMGATEWAY_WINDOW_PREMIUM_WEEKLY] = premium;
return {
plan: planLabel(quota.devPlan),
quotas,
devPlan: quota.devPlan,
};
}

View File

@@ -80,6 +80,8 @@ export const USAGE_SUPPORTED_PROVIDERS: readonly string[] = [
"kilocode",
// OpenRouter key limits + account credits (GET /api/v1/key + /api/v1/credits)
"openrouter",
// LLM Gateway DevPass allowance (GET /v1/key → monthly + weekly premium)
"llmgateway",
// Devin CLI agentic quota (Codeium seat-management GetUserStatus, protobuf)
"devin-cli",
];

View File

@@ -18,6 +18,7 @@ export const PROVIDER_LABEL: Record<string, string> = {
"xai-oauth": "xAI OAuth (Grok)",
xao: "xAI OAuth (Grok)",
"grok-cli": "Grok Build",
llmgateway: "LLM Gateway",
};
export const PROVIDER_ORDER: Record<string, number> = {
@@ -38,6 +39,7 @@ export const PROVIDER_ORDER: Record<string, number> = {
"xai-oauth": 16,
xao: 16,
"grok-cli": 17,
llmgateway: 18,
};
export const TIER_FILTERS = [

View File

@@ -91,6 +91,8 @@ const PROVIDER_LIMITS_APIKEY_PROVIDERS = new Set([
"agentrouter",
// OpenRouter API key → /key limits + /credits account balance
"openrouter",
// LLM Gateway API key (llmgtwy_…) → GET /v1/key DevPass allowance
"llmgateway",
]);
const DEFAULT_PROVIDER_LIMITS_SYNC_INTERVAL_MINUTES = 70;
const PROVIDER_LIMITS_AUTO_SYNC_SETTING_KEY = "provider_limits_auto_sync_last_run";

View File

@@ -0,0 +1,346 @@
import test from "node:test";
import assert from "node:assert/strict";
import {
buildLlmgatewayQuota,
fetchLlmgatewayQuota,
invalidateLlmgatewayQuotaCache,
parseLlmgatewayKeyResponse,
registerLlmgatewayQuotaFetcher,
type LlmgatewayQuota,
LLMGATEWAY_WINDOW_MONTHLY,
LLMGATEWAY_WINDOW_PREMIUM_WEEKLY,
} from "../../open-sse/services/llmgatewayQuotaFetcher.ts";
import {
getQuotaFetcher,
getQuotaWindows,
preflightQuota,
} from "../../open-sse/services/quotaPreflight.ts";
import { clearQuotaMonitors } from "../../open-sse/services/quotaMonitor.ts";
import { getLlmgatewayUsage } from "../../open-sse/services/usage/llmgateway.ts";
const originalFetch = globalThis.fetch;
const PRO_KEY_RESPONSE = {
data: {
label: "My coding tool",
usage: "31.42",
limit: null,
devPlan: "pro",
devPlanCreditsUsed: "25",
devPlanCreditsLimit: "237",
devPlanCreditsRemaining: "212.00",
devPlanPremiumWeeklyLimit: "35.55",
devPlanPremiumCreditsUsed: "5.00",
devPlanPremiumWeekResetsAt: "2026-08-28T12:00:00.000Z",
},
};
function keyResponse(overrides: Record<string, unknown>) {
return {
data: { ...PRO_KEY_RESPONSE.data, ...overrides },
};
}
test.afterEach(() => {
globalThis.fetch = originalFetch;
clearQuotaMonitors();
});
// ─── Parser ──────────────────────────────────────────────────────────────────
test("parseLlmgatewayKeyResponse parses decimal-string DevPass fields", () => {
const fields = parseLlmgatewayKeyResponse(PRO_KEY_RESPONSE);
assert.ok(fields);
assert.equal(fields.devPlan, "pro");
assert.equal(fields.monthlyUsed, 25);
assert.equal(fields.monthlyLimit, 237);
assert.equal(fields.monthlyRemaining, 212);
assert.equal(fields.premiumWeeklyLimit, 35.55);
assert.equal(fields.premiumUsed, 5);
assert.equal(fields.premiumResetAt, "2026-08-28T12:00:00.000Z");
});
test("parseLlmgatewayKeyResponse returns null on empty/unexpected shape", () => {
assert.equal(parseLlmgatewayKeyResponse({}), null);
assert.equal(parseLlmgatewayKeyResponse({ data: {} }), null);
assert.equal(parseLlmgatewayKeyResponse(null), null);
});
test("parseLlmgatewayKeyResponse defaults devPlan to none when missing", () => {
const fields = parseLlmgatewayKeyResponse({ data: { devPlanCreditsUsed: "0" } });
assert.ok(fields);
assert.equal(fields.devPlan, "none");
});
test("parseLlmgatewayKeyResponse treats expired premium window as null reset", () => {
const fields = parseLlmgatewayKeyResponse(
keyResponse({ devPlanPremiumCreditsUsed: "0.00", devPlanPremiumWeekResetsAt: null })
);
assert.ok(fields);
assert.equal(fields.premiumUsed, 0);
assert.equal(fields.premiumResetAt, null);
});
// ─── buildLlmgatewayQuota ──────────────────────────────────────────────────────
test("buildLlmgatewayQuota returns null for pay-as-you-go keys (devPlan none)", () => {
const fields = parseLlmgatewayKeyResponse({
data: {
devPlan: "none",
devPlanCreditsUsed: "0",
devPlanCreditsLimit: "0",
devPlanCreditsRemaining: "0",
devPlanPremiumWeeklyLimit: "0",
devPlanPremiumCreditsUsed: "0",
devPlanPremiumWeekResetsAt: null,
},
});
assert.ok(fields);
assert.equal(buildLlmgatewayQuota(fields), null);
});
test("buildLlmgatewayQuota exposes monthly + weekly windows with correct percentUsed", () => {
const fields = parseLlmgatewayKeyResponse(PRO_KEY_RESPONSE);
assert.ok(fields);
const quota = buildLlmgatewayQuota(fields);
assert.ok(quota);
// monthly: 25/237 ≈ 0.1055 ; premium: 5/35.55 ≈ 0.1406 → worst = premium
assert.ok(Math.abs(quota.windows![LLMGATEWAY_WINDOW_MONTHLY].percentUsed - 25 / 237) < 1e-9);
assert.ok(
Math.abs(quota.windows![LLMGATEWAY_WINDOW_PREMIUM_WEEKLY].percentUsed - 5 / 35.55) < 1e-9
);
assert.ok(Math.abs(quota.percentUsed - 5 / 35.55) < 1e-9);
assert.equal(quota.limitReached, false);
assert.equal(quota.monthlyRemaining, 212);
assert.ok(Math.abs((quota.premiumRemaining ?? 0) - (35.55 - 5)) < 1e-9);
});
test("buildLlmgatewayQuota marks limitReached when monthly credits exhausted", () => {
const fields = parseLlmgatewayKeyResponse(
keyResponse({
devPlanCreditsUsed: "237",
devPlanCreditsLimit: "237",
devPlanCreditsRemaining: "0",
})
);
assert.ok(fields);
const quota = buildLlmgatewayQuota(fields);
assert.ok(quota);
assert.equal(quota.limitReached, true);
assert.equal(quota.windows![LLMGATEWAY_WINDOW_MONTHLY].percentUsed, 1);
});
test("buildLlmgatewayQuota marks limitReached when weekly premium exhausted", () => {
const fields = parseLlmgatewayKeyResponse(keyResponse({ devPlanPremiumCreditsUsed: "35.55" }));
assert.ok(fields);
const quota = buildLlmgatewayQuota(fields);
assert.ok(quota);
assert.equal(quota.limitReached, true);
});
// ─── fetchLlmgatewayQuota ──────────────────────────────────────────────────────
test("fetchLlmgatewayQuota returns null when no API key exists", async () => {
const quota = await fetchLlmgatewayQuota(`missing-${Date.now()}`);
assert.equal(quota, null);
});
test("fetchLlmgatewayQuota parses a live /v1/key response with Bearer auth", async () => {
const connectionId = `llmgw-live-${Date.now()}`;
const calls: Array<{ url: unknown; init: unknown }> = [];
globalThis.fetch = async (url, init) => {
calls.push({ url, init });
return new Response(JSON.stringify(PRO_KEY_RESPONSE), {
status: 200,
headers: { "content-type": "application/json" },
});
};
const quota = (await fetchLlmgatewayQuota(connectionId, {
apiKey: "llmgtwy_test",
})) as LlmgatewayQuota | null;
assert.equal(calls.length, 1);
assert.equal(String(calls[0].url), "https://api.llmgateway.io/v1/key");
assert.equal(
(calls[0].init as RequestInit).headers!["Authorization" as never],
"Bearer llmgtwy_test"
);
assert.equal(quota?.devPlan, "pro");
assert.equal(quota?.monthlyLimit, 237);
invalidateLlmgatewayQuotaCache(connectionId);
});
test("fetchLlmgatewayQuota returns null on 401 (invalid key)", async () => {
const connectionId = `llmgw-401-${Date.now()}`;
globalThis.fetch = async () => new Response(null, { status: 401 });
const quota = await fetchLlmgatewayQuota(connectionId, { apiKey: "bad" });
assert.equal(quota, null);
});
test("fetchLlmgatewayQuota returns null on 403 (publishable/session key)", async () => {
const connectionId = `llmgw-403-${Date.now()}`;
globalThis.fetch = async () => new Response(null, { status: 403 });
const quota = await fetchLlmgatewayQuota(connectionId, { apiKey: "publishable" });
assert.equal(quota, null);
});
test("fetchLlmgatewayQuota returns null for pay-as-you-go key", async () => {
const connectionId = `llmgw-payg-${Date.now()}`;
globalThis.fetch = async () =>
new Response(
JSON.stringify({
data: {
devPlan: "none",
devPlanCreditsUsed: "0",
devPlanCreditsLimit: "0",
devPlanCreditsRemaining: "0",
devPlanPremiumWeeklyLimit: "0",
devPlanPremiumCreditsUsed: "0",
devPlanPremiumWeekResetsAt: null,
},
}),
{ status: 200, headers: { "content-type": "application/json" } }
);
const quota = await fetchLlmgatewayQuota(connectionId, { apiKey: "payg" });
assert.equal(quota, null);
});
test("fetchLlmgatewayQuota caches results within TTL", async () => {
const connectionId = `llmgw-cache-${Date.now()}`;
const calls: number[] = [];
globalThis.fetch = async () => {
calls.push(1);
return new Response(JSON.stringify(PRO_KEY_RESPONSE), {
status: 200,
headers: { "content-type": "application/json" },
});
};
const first = await fetchLlmgatewayQuota(connectionId, { apiKey: "llmgtwy_test" });
const second = await fetchLlmgatewayQuota(connectionId, { apiKey: "llmgtwy_test" });
assert.equal(calls.length, 1);
assert.deepEqual(first, second);
invalidateLlmgatewayQuotaCache(connectionId);
await fetchLlmgatewayQuota(connectionId, { apiKey: "llmgtwy_test" });
assert.equal(calls.length, 2);
invalidateLlmgatewayQuotaCache(connectionId);
});
test("fetchLlmgatewayQuota returns null on network error (fail-open)", async () => {
const connectionId = `llmgw-net-${Date.now()}`;
globalThis.fetch = async () => {
throw new Error("Network error");
};
const quota = await fetchLlmgatewayQuota(connectionId, { apiKey: "llmgtwy_test" });
assert.equal(quota, null);
});
// ─── Registration + preflight ──────────────────────────────────────────────────
test("registerLlmgatewayQuotaFetcher wires preflight + monitor + windows", () => {
registerLlmgatewayQuotaFetcher();
assert.ok(getQuotaFetcher("llmgateway"));
const windows = getQuotaWindows("llmgateway");
assert.deepEqual([...windows], [LLMGATEWAY_WINDOW_MONTHLY, LLMGATEWAY_WINDOW_PREMIUM_WEEKLY]);
});
test("preflight proceeds when DevPass has headroom", async () => {
const connectionId = `llmgw-proceed-${Date.now()}`;
registerLlmgatewayQuotaFetcher();
globalThis.fetch = async () =>
new Response(JSON.stringify(PRO_KEY_RESPONSE), {
status: 200,
headers: { "content-type": "application/json" },
});
const preflight = await preflightQuota("llmgateway", connectionId, {
apiKey: "llmgtwy_test",
providerSpecificData: { quotaPreflightEnabled: true },
});
assert.equal(preflight.proceed, true);
invalidateLlmgatewayQuotaCache(connectionId);
});
test("preflight blocks when a DevPass window is exhausted", async () => {
const connectionId = `llmgw-block-${Date.now()}`;
registerLlmgatewayQuotaFetcher();
globalThis.fetch = async () =>
new Response(
JSON.stringify(
keyResponse({
devPlanCreditsUsed: "237",
devPlanCreditsLimit: "237",
devPlanCreditsRemaining: "0",
})
),
{ status: 200, headers: { "content-type": "application/json" } }
);
const preflight = await preflightQuota("llmgateway", connectionId, {
apiKey: "llmgtwy_test",
providerSpecificData: { quotaPreflightEnabled: true },
});
assert.equal(preflight.proceed, false);
assert.equal(preflight.reason, "quota_exhausted");
invalidateLlmgatewayQuotaCache(connectionId);
});
// ─── Usage leaf ────────────────────────────────────────────────────────────────
test("getLlmgatewayUsage returns monthly + weekly quota rows", async () => {
const connectionId = `llmgw-usage-${Date.now()}`;
globalThis.fetch = async () =>
new Response(JSON.stringify(PRO_KEY_RESPONSE), {
status: 200,
headers: { "content-type": "application/json" },
});
const usage = await getLlmgatewayUsage(connectionId, "llmgtwy_test");
assert.equal(usage.plan, "LLM Gateway DevPass (Pro)");
const monthly = usage.quotas?.[LLMGATEWAY_WINDOW_MONTHLY];
const premium = usage.quotas?.[LLMGATEWAY_WINDOW_PREMIUM_WEEKLY];
assert.ok(monthly);
assert.equal(monthly.total, 237);
assert.equal(monthly.currency, "USD");
assert.ok(premium);
assert.equal(premium.resetAt, "2026-08-28T12:00:00.000Z");
invalidateLlmgatewayQuotaCache(connectionId);
});
test("getLlmgatewayUsage surfaces pay-as-you-go state without quotas", async () => {
const connectionId = `llmgw-usage-payg-${Date.now()}`;
globalThis.fetch = async () =>
new Response(
JSON.stringify({
data: {
devPlan: "none",
devPlanCreditsUsed: "0",
devPlanCreditsLimit: "0",
devPlanCreditsRemaining: "0",
devPlanPremiumWeeklyLimit: "0",
devPlanPremiumCreditsUsed: "0",
devPlanPremiumWeekResetsAt: null,
},
}),
{ status: 200, headers: { "content-type": "application/json" } }
);
const usage = await getLlmgatewayUsage(connectionId, "payg");
assert.equal(usage.plan, "LLM Gateway (pay-as-you-go)");
assert.deepEqual(usage.quotas, {});
});
test("getLlmgatewayUsage returns a message when no API key given", async () => {
const usage = await getLlmgatewayUsage(`llmgw-nokey-${Date.now()}`, "");
assert.ok(usage.message);
assert.equal(usage.quotas, undefined);
});