Merge pull request #20 from diegosouzapw/feature/v0.2.0-release

feat: v0.2.0 — advanced routing services, cost analytics, pricing overhaul
This commit is contained in:
Diego Rodrigues de Sa e Souza
2026-02-14 13:51:18 -03:00
committed by GitHub
50 changed files with 4892 additions and 187 deletions

1
.gitignore vendored
View File

@@ -72,3 +72,4 @@ open-sse/test/*
test-results/
playwright-report/
blob-report/
cloud/

View File

@@ -8,15 +8,77 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
## [Unreleased]
---
## [0.2.0] — 2026-02-14
Major feature release: advanced routing services, security hardening, cost analytics dashboard, and pricing management overhaul.
### Added
- Model selector with autocomplete in Chat Tester and Test Bench modes — prevents "model does not exist" errors by letting users choose available models (`c25230c`)
- OpenAPI specification at `docs/openapi.yaml` covering all 89 API endpoints (`7abda4d`)
- Enhanced `restart.sh` with clean build, health check, graceful shutdown (Ctrl+C), and real-time log tailing (`0db8f3d`)
#### Open-SSE Services
- **Account Selector** — intelligent provider account selection with priority and load-balancing strategies (`accountSelector.js`)
- **Context Manager** — request context tracking and lifecycle management (`contextManager.js`)
- **IP Filter** — allowlist/blocklist IP filtering with CIDR support (`ipFilter.js`)
- **Session Manager** — persistent session tracking across requests (`sessionManager.js`)
- **Signature Cache** — request signature caching for deduplication (`signatureCache.js`)
- **System Prompt** — global system prompt injection into all chat completions (`systemPrompt.js`)
- **Thinking Budget** — token budget management for reasoning models (`thinkingBudget.js`)
- **Wildcard Router** — pattern-based model routing with glob matching (`wildcardRouter.js`)
- Enhanced **Rate Limit Manager** with sliding-window algorithm and per-key quotas
#### Dashboard Settings
- **IP Filter** settings tab — configure allowed/blocked IPs from the UI (`IPFilterSection.js`)
- **System Prompt** settings tab — set global system prompt injection (`SystemPromptTab.js`)
- **Thinking Budget** settings tab — configure reasoning token budgets (`ThinkingBudgetTab.js`)
- **Pricing Tab** — full-page redesign with provider-centric organization, inline editing, search/filter, and save/reset per provider (`PricingTab.js`)
- **Rate Limit Status** component on Usage page (`RateLimitStatus.js`)
- **Sessions Tab** on Usage page — view and manage active sessions (`SessionsTab.js`)
#### Usage & Cost Analytics
- **Cost stat card** (amber accent) prominently displayed in analytics top row
- **Provider Cost Donut** — new chart showing cost distribution across providers
- **Daily Cost Trend** — cost line overlay (amber) on token trend chart with secondary Y-axis
- **Model Table Cost column** — sortable cost column in model breakdown table
- Cost-aware tooltip formatting throughout analytics charts
#### Pricing API
- `/api/pricing/models` endpoint — serves merged model catalog from 3 sources: registry, custom models (DB), and pricing-only models
- Custom model badge in pricing page for user-imported models
- `/api/rate-limits` endpoint for rate limit configuration
- `/api/sessions` endpoint for session management
- `/api/settings/ip-filter`, `/api/settings/system-prompt`, `/api/settings/thinking-budget` endpoints
#### Cloudflare Worker
- Cloud worker module for edge deployment (`cloud/`)
#### Tests
- Unit tests for account selector, context manager, IP filter, enhanced rate limiting, session manager, signature cache, system prompt, thinking budget, and wildcard router (9 new test files)
#### Documentation
- OpenAPI specification at `docs/openapi.yaml` covering all 89 API endpoints
- Enhanced `restart.sh` with clean build, health check, graceful shutdown (Ctrl+C), and real-time log tailing
- Updated architecture documentation and codebase docs with new services and API routes
- Model selector with autocomplete in Chat Tester and Test Bench modes
### Fixed
- Server port collision (EADDRINUSE) during restart — now kills port before `next start` (`e4c5c0c`)
- Server port collision (EADDRINUSE) during restart — now kills port before `next start`
- Icon rendering corrected from `material-symbols-rounded` to `material-symbols-outlined`
- Pricing page only showed hardcoded registry models — now includes custom/imported models
### Changed
- Usage analytics layout reorganized: donuts separated into logical groupings, bottom stats simplified from 6 to 4 cards
- Daily trend chart upgraded from `BarChart` to `ComposedChart` with dual Y-axes
- Routing tab updated with new service integrations
---

View File

@@ -146,6 +146,12 @@ Default URLs:
| 💾 **Cloud Sync** | Sync config across devices | Same setup everywhere |
| 📊 **Usage Analytics** | Track tokens, cost, trends over time | Optimize spending |
| 🌐 **Deploy Anywhere** | Localhost, VPS, Docker, Cloudflare Workers | Flexible deployment options |
| 🛡️ **IP Allowlist/Blocklist** | Restrict API access by IP address | Security for exposed deployments |
| 🧠 **Thinking Budget** | Control reasoning token budget per model | Optimize cost vs quality |
| 💬 **System Prompt Injection** | Global system prompt for all requests | Consistent behavior across models |
| 📊 **Session Tracking** | Track active sessions with fingerprinting | Monitor connected clients |
| ⚡ **Rate Limiting** | Per-account request rate management | Prevent abuse and quota waste |
| 💰 **Model Pricing** | Per-model cost tracking and calculation | Precise usage cost analytics |
<details>
<summary><b>📖 Feature Details</b></summary>
@@ -1217,6 +1223,11 @@ Expected behavior from recent validation:
- Routing config: `/api/models/alias`, `/api/combos*`, `/api/keys*`, `/api/pricing`
- Usage/logs: `/api/usage/history`, `/api/usage/logs`, `/api/usage/request-logs`, `/api/usage/[connectionId]`
- Cloud sync: `/api/sync/cloud`, `/api/sync/initialize`, `/api/cloud/*`
- IP filter: `/api/settings/ip-filter` (GET/PUT) — Allowlist/blocklist management
- Thinking budget: `/api/settings/thinking-budget` (GET/PUT) — Reasoning token budget config
- System prompt: `/api/settings/system-prompt` (GET/PUT) — Global system prompt injection
- Sessions: `/api/sessions` (GET) — Active session tracking
- Rate limits: `/api/rate-limits` (GET) — Per-account rate limit status
- CLI helpers: `/api/cli-tools/claude-settings`, `/api/cli-tools/codex-settings`, `/api/cli-tools/droid-settings`, `/api/cli-tools/openclaw-settings`
- Generic runtime status: `/api/cli-tools/runtime/[toolId]` (covers `claude`, `codex`, `droid`, `openclaw`, `cursor`, `cline`, `roo`, `continue`)
- CLI `GET` responses expose runtime fields: `installed`, `runnable`, `command`, `commandPath`, `runtimeMode`, `reason`

View File

@@ -1,6 +1,6 @@
# OmniRoute Architecture
_Last updated: 2026-02-09_
_Last updated: 2026-02-14_
## Executive Summary
@@ -20,6 +20,12 @@ Core capabilities:
- Local persistence for providers, keys, aliases, combos, settings, pricing
- Usage/cost tracking and request logging
- Optional cloud sync for multi-device/state sync
- IP allowlist/blocklist for API access control
- Thinking budget management (passthrough/auto/custom/adaptive)
- Global system prompt injection
- Session tracking and fingerprinting
- Per-account enhanced rate limiting
- Signature-based request deduplication cache
Primary runtime model:
@@ -129,6 +135,11 @@ Management domains:
- Usage: `src/app/api/usage/*`
- Sync/cloud: `src/app/api/sync/*`, `src/app/api/cloud/*`
- CLI tooling helpers: `src/app/api/cli-tools/*`
- IP filter: `src/app/api/settings/ip-filter` (GET/PUT)
- Thinking budget: `src/app/api/settings/thinking-budget` (GET/PUT)
- System prompt: `src/app/api/settings/system-prompt` (GET/PUT)
- Sessions: `src/app/api/sessions` (GET)
- Rate limits: `src/app/api/rate-limits` (GET)
## 2) SSE + Translation Core
@@ -149,13 +160,24 @@ Main flow modules:
- Image generation handler: `open-sse/handlers/imageGeneration.js`
- Image provider registry: `open-sse/config/imageRegistry.js`
Services (business logic):
- Account selection/scoring: `open-sse/services/accountSelector.js`
- Context lifecycle management: `open-sse/services/contextManager.js`
- IP filter enforcement: `open-sse/services/ipFilter.js`
- Session tracking: `open-sse/services/sessionManager.js`
- Request deduplication: `open-sse/services/signatureCache.js`
- System prompt injection: `open-sse/services/systemPrompt.js`
- Thinking budget management: `open-sse/services/thinkingBudget.js`
- Wildcard model routing: `open-sse/services/wildcardRouter.js`
## 3) Persistence Layer
Primary state DB:
- `src/lib/localDb.js`
- file: `${DATA_DIR}/db.json` (or `$XDG_CONFIG_HOME/omniroute/db.json` when set, else `~/.omniroute/db.json`)
- entities: providerConnections, providerNodes, modelAliases, combos, apiKeys, settings, pricing, **customModels**, **proxyConfig**
- entities: providerConnections, providerNodes, modelAliases, combos, apiKeys, settings, pricing, **customModels**, **proxyConfig**, **ipFilter**, **thinkingBudget**, **systemPrompt**
Usage DB:
@@ -399,6 +421,24 @@ erDiagram
string global
json providers
}
IP_FILTER {
string mode
string[] allowlist
string[] blocklist
}
THINKING_BUDGET {
string mode
number customBudget
string effortLevel
}
SYSTEM_PROMPT {
boolean enabled
string prompt
string position
}
```
Physical storage files:
@@ -459,6 +499,11 @@ flowchart LR
- `src/app/api/usage/*`: usage and logs APIs
- `src/app/api/sync/*` + `src/app/api/cloud/*`: cloud sync and cloud-facing helpers
- `src/app/api/cli-tools/*`: local CLI config writers/checkers
- `src/app/api/settings/ip-filter`: IP allowlist/blocklist (GET/PUT)
- `src/app/api/settings/thinking-budget`: thinking token budget config (GET/PUT)
- `src/app/api/settings/system-prompt`: global system prompt (GET/PUT)
- `src/app/api/sessions`: active session listing (GET)
- `src/app/api/rate-limits`: per-account rate limit status (GET)
### Routing and Execution Core

View File

@@ -268,6 +268,14 @@ Business logic that supports the handlers and executors.
| `tokenRefresh.js` | OAuth token refresh for **every provider**: Google (Gemini, Antigravity), Claude, Codex, Qwen, iFlow, GitHub (OAuth + Copilot dual-token), Kiro (AWS SSO OIDC + Social Auth). Includes in-flight promise deduplication cache and retry with exponential backoff. |
| `combo.js` | **Combo models**: chains of fallback models. If model A fails with a fallback-eligible error, try model B, then C, etc. Returns actual upstream status codes. |
| `usage.js` | Fetches quota/usage data from provider APIs (GitHub Copilot quotas, Antigravity model quotas, Codex rate limits, Kiro usage breakdowns, Claude settings). |
| `accountSelector.js` | Smart account selection with scoring algorithm: considers priority, health status, round-robin position, and cooldown state to pick the optimal account for each request. |
| `contextManager.js` | Request context lifecycle management: creates and tracks per-request context objects with metadata (request ID, timestamps, provider info) for debugging and logging. |
| `ipFilter.js` | IP-based access control: supports allowlist and blocklist modes. Validates client IP against configured rules before processing API requests. |
| `sessionManager.js` | Session tracking with client fingerprinting: tracks active sessions using hashed client identifiers, monitors request counts, and provides session metrics. |
| `signatureCache.js` | Request signature-based deduplication cache: prevents duplicate requests by caching recent request signatures and returning cached responses for identical requests within a time window. |
| `systemPrompt.js` | Global system prompt injection: prepends or appends a configurable system prompt to all requests, with per-provider compatibility handling. |
| `thinkingBudget.js` | Reasoning token budget management: supports passthrough, auto (strip thinking config), custom (fixed budget), and adaptive (complexity-scaled) modes for controlling thinking/reasoning tokens. |
| `wildcardRouter.js` | Wildcard model pattern routing: resolves wildcard patterns (e.g., `*/claude-*`) to concrete provider/model pairs based on availability and priority. |
#### Token Refresh Deduplication
@@ -460,6 +468,11 @@ logs/
| `/v1/providers/[provider]/chat/completions` | POST | Dedicated per-provider chat completions with model validation |
| `/v1/providers/[provider]/embeddings` | POST | Dedicated per-provider embeddings with model validation |
| `/v1/providers/[provider]/images/generations` | POST | Dedicated per-provider image generation with model validation |
| `/api/settings/ip-filter` | GET/PUT | IP allowlist/blocklist management |
| `/api/settings/thinking-budget` | GET/PUT | Reasoning token budget configuration (passthrough/auto/custom/adaptive) |
| `/api/settings/system-prompt` | GET/PUT | Global system prompt injection for all requests |
| `/api/sessions` | GET | Active session tracking and metrics |
| `/api/rate-limits` | GET | Per-account rate limit status |
---

View File

@@ -1,5 +1,6 @@
/** @type {import('next').NextConfig} */
const nextConfig = {
turbopack: {},
output: "standalone",
transpilePackages: ["@omniroute/open-sse"],
allowedDevOrigins: ["192.168.*"],

View File

@@ -113,6 +113,21 @@ export const BACKOFF_CONFIG = {
maxLevel: 15, // Cap backoff level
};
// Configurable backoff steps for rate limits (Phase 1 — enhanced rate limiting)
// Used for per-model lockouts with increasing severity
export const BACKOFF_STEPS_MS = [60_000, 120_000, 300_000, 600_000, 1_200_000];
// 1min → 2min → 5min → 10min → 20min
// Structured error classification for rate limiting decisions
export const RateLimitReason = {
QUOTA_EXHAUSTED: 'quota_exhausted', // Daily/monthly quota depleted
RATE_LIMIT_EXCEEDED: 'rate_limit_exceeded', // RPM/RPD limits hit
MODEL_CAPACITY: 'model_capacity', // Model overloaded (529, 503)
SERVER_ERROR: 'server_error', // 5xx errors
AUTH_ERROR: 'auth_error', // 401, 403
UNKNOWN: 'unknown',
};
// Error-based cooldown times (aligned with CLIProxyAPI)
export const COOLDOWN_MS = {
unauthorized: 2 * 60 * 1000, // 401 → 30 min

View File

@@ -1,4 +1,213 @@
import { COOLDOWN_MS, BACKOFF_CONFIG, HTTP_STATUS } from "../config/constants.js";
import { COOLDOWN_MS, BACKOFF_CONFIG, BACKOFF_STEPS_MS, RateLimitReason, HTTP_STATUS } from "../config/constants.js";
// ─── Per-Model Lockout Tracking ─────────────────────────────────────────────
// In-memory map: "provider:connectionId:model" → { reason, until, lockedAt }
const modelLockouts = new Map();
// Auto-cleanup expired lockouts every 15 seconds
const _cleanupTimer = setInterval(() => {
const now = Date.now();
for (const [key, entry] of modelLockouts) {
if (now > entry.until) modelLockouts.delete(key);
}
}, 15_000);
_cleanupTimer.unref(); // Don't prevent process exit
/**
* Lock a specific model on a specific account
* @param {string} provider
* @param {string} connectionId
* @param {string} model
* @param {string} reason - from RateLimitReason
* @param {number} cooldownMs
*/
export function lockModel(provider, connectionId, model, reason, cooldownMs) {
if (!model) return; // No model → skip model-level locking
const key = `${provider}:${connectionId}:${model}`;
modelLockouts.set(key, {
reason,
until: Date.now() + cooldownMs,
lockedAt: Date.now(),
});
}
/**
* Check if a specific model on a specific account is locked
* @returns {boolean}
*/
export function isModelLocked(provider, connectionId, model) {
if (!model) return false;
const key = `${provider}:${connectionId}:${model}`;
const entry = modelLockouts.get(key);
if (!entry) return false;
if (Date.now() > entry.until) {
modelLockouts.delete(key);
return false;
}
return true;
}
/**
* Get model lockout info (for debugging/dashboard)
*/
export function getModelLockoutInfo(provider, connectionId, model) {
if (!model) return null;
const key = `${provider}:${connectionId}:${model}`;
const entry = modelLockouts.get(key);
if (!entry || Date.now() > entry.until) return null;
return {
reason: entry.reason,
remainingMs: entry.until - Date.now(),
lockedAt: new Date(entry.lockedAt).toISOString(),
};
}
/**
* Get all active model lockouts (for dashboard)
*/
export function getAllModelLockouts() {
const now = Date.now();
const active = [];
for (const [key, entry] of modelLockouts) {
if (now <= entry.until) {
const [provider, connectionId, model] = key.split(":");
active.push({ provider, connectionId, model, reason: entry.reason, remainingMs: entry.until - now });
}
}
return active;
}
// ─── Retry-After Parsing ────────────────────────────────────────────────────
/**
* Parse retry-after information from JSON error response bodies.
* Providers embed retry info in different formats.
*
* @param {string|object} responseBody - Raw response body or parsed JSON
* @returns {{ retryAfterMs: number|null, reason: string }}
*/
export function parseRetryAfterFromBody(responseBody) {
let body;
try {
body = typeof responseBody === "string" ? JSON.parse(responseBody) : responseBody;
} catch {
return { retryAfterMs: null, reason: RateLimitReason.UNKNOWN };
}
if (!body) return { retryAfterMs: null, reason: RateLimitReason.UNKNOWN };
// Gemini: { error: { details: [{ retryDelay: "33s" }] } }
const details = body.error?.details || body.details || [];
for (const detail of Array.isArray(details) ? details : []) {
if (detail.retryDelay) {
return { retryAfterMs: parseDelayString(detail.retryDelay), reason: RateLimitReason.RATE_LIMIT_EXCEEDED };
}
}
// OpenAI: "Please retry after 20s" in message
const msg = body.error?.message || body.message || "";
const retryMatch = msg.match(/retry\s+after\s+(\d+)\s*s/i);
if (retryMatch) {
return { retryAfterMs: parseInt(retryMatch[1], 10) * 1000, reason: RateLimitReason.RATE_LIMIT_EXCEEDED };
}
// Anthropic: error type classification
const errorType = body.error?.type || body.type || "";
if (errorType === "rate_limit_error") {
return { retryAfterMs: null, reason: RateLimitReason.RATE_LIMIT_EXCEEDED };
}
// Classify by error message keywords
const reason = classifyErrorText(msg || errorType);
return { retryAfterMs: null, reason };
}
/**
* Parse delay strings like "33s", "2m", "1h", "1500ms"
*/
function parseDelayString(value) {
if (!value) return null;
const str = String(value).trim();
const msMatch = str.match(/^(\d+)\s*ms$/i);
if (msMatch) return parseInt(msMatch[1], 10);
const secMatch = str.match(/^(\d+)\s*s$/i);
if (secMatch) return parseInt(secMatch[1], 10) * 1000;
const minMatch = str.match(/^(\d+)\s*m$/i);
if (minMatch) return parseInt(minMatch[1], 10) * 60 * 1000;
const hrMatch = str.match(/^(\d+)\s*h$/i);
if (hrMatch) return parseInt(hrMatch[1], 10) * 3600 * 1000;
// Bare number → seconds
const num = parseInt(str, 10);
return isNaN(num) ? null : num * 1000;
}
// ─── Error Classification ───────────────────────────────────────────────────
/**
* Classify error text into RateLimitReason
*/
export function classifyErrorText(errorText) {
if (!errorText) return RateLimitReason.UNKNOWN;
const lower = String(errorText).toLowerCase();
if (lower.includes("quota exceeded") || lower.includes("quota depleted") || lower.includes("billing")) {
return RateLimitReason.QUOTA_EXHAUSTED;
}
if (lower.includes("rate limit") || lower.includes("too many requests") || lower.includes("rate_limit")) {
return RateLimitReason.RATE_LIMIT_EXCEEDED;
}
if (lower.includes("capacity") || lower.includes("overloaded") || lower.includes("resource exhausted")) {
return RateLimitReason.MODEL_CAPACITY;
}
if (lower.includes("unauthorized") || lower.includes("invalid api key") || lower.includes("authentication")) {
return RateLimitReason.AUTH_ERROR;
}
if (lower.includes("server error") || lower.includes("internal error")) {
return RateLimitReason.SERVER_ERROR;
}
return RateLimitReason.UNKNOWN;
}
/**
* Classify HTTP status + error text into RateLimitReason
*/
export function classifyError(status, errorText) {
// Text classification takes priority (more specific)
const textReason = classifyErrorText(errorText);
if (textReason !== RateLimitReason.UNKNOWN) return textReason;
// Fall back to status code
if (status === HTTP_STATUS.UNAUTHORIZED || status === HTTP_STATUS.FORBIDDEN) {
return RateLimitReason.AUTH_ERROR;
}
if (status === HTTP_STATUS.PAYMENT_REQUIRED) {
return RateLimitReason.QUOTA_EXHAUSTED;
}
if (status === HTTP_STATUS.RATE_LIMITED) {
return RateLimitReason.RATE_LIMIT_EXCEEDED;
}
if (status === HTTP_STATUS.SERVICE_UNAVAILABLE || status === 529) {
return RateLimitReason.MODEL_CAPACITY;
}
if (status >= 500) {
return RateLimitReason.SERVER_ERROR;
}
return RateLimitReason.UNKNOWN;
}
// ─── Configurable Backoff ───────────────────────────────────────────────────
/**
* Get backoff duration from configurable steps.
* @param {number} failureCount - Number of consecutive failures
* @returns {number} Duration in ms
*/
export function getBackoffDuration(failureCount) {
const idx = Math.min(failureCount, BACKOFF_STEPS_MS.length - 1);
return BACKOFF_STEPS_MS[idx];
}
// ─── Original API (Backward Compatible) ────────────────────────────────────
/**
* Calculate exponential backoff cooldown for rate limits (429)
@@ -16,20 +225,21 @@ export function getQuotaCooldown(backoffLevel = 0) {
* @param {number} status - HTTP status code
* @param {string} errorText - Error message text
* @param {number} backoffLevel - Current backoff level for exponential backoff
* @returns {{ shouldFallback: boolean, cooldownMs: number, newBackoffLevel?: number }}
* @param {string} [model] - Optional model name for model-level lockout
* @returns {{ shouldFallback: boolean, cooldownMs: number, newBackoffLevel?: number, reason?: string }}
*/
export function checkFallbackError(status, errorText, backoffLevel = 0) {
export function checkFallbackError(status, errorText, backoffLevel = 0, model = null) {
// Check error message FIRST - specific patterns take priority over status codes
if (errorText) {
const errorStr = typeof errorText === "string" ? errorText : JSON.stringify(errorText);
const lowerError = errorStr.toLowerCase();
if (lowerError.includes("no credentials")) {
return { shouldFallback: true, cooldownMs: COOLDOWN_MS.notFound };
return { shouldFallback: true, cooldownMs: COOLDOWN_MS.notFound, reason: RateLimitReason.AUTH_ERROR };
}
if (lowerError.includes("request not allowed")) {
return { shouldFallback: true, cooldownMs: COOLDOWN_MS.requestNotAllowed };
return { shouldFallback: true, cooldownMs: COOLDOWN_MS.requestNotAllowed, reason: RateLimitReason.RATE_LIMIT_EXCEEDED };
}
// Rate limit keywords - exponential backoff
@@ -41,24 +251,26 @@ export function checkFallbackError(status, errorText, backoffLevel = 0) {
lowerError.includes("overloaded")
) {
const newLevel = Math.min(backoffLevel + 1, BACKOFF_CONFIG.maxLevel);
const reason = classifyErrorText(errorStr);
return {
shouldFallback: true,
cooldownMs: getQuotaCooldown(backoffLevel),
newBackoffLevel: newLevel,
reason,
};
}
}
if (status === HTTP_STATUS.UNAUTHORIZED) {
return { shouldFallback: true, cooldownMs: COOLDOWN_MS.unauthorized };
return { shouldFallback: true, cooldownMs: COOLDOWN_MS.unauthorized, reason: RateLimitReason.AUTH_ERROR };
}
if (status === HTTP_STATUS.PAYMENT_REQUIRED || status === HTTP_STATUS.FORBIDDEN) {
return { shouldFallback: true, cooldownMs: COOLDOWN_MS.paymentRequired };
return { shouldFallback: true, cooldownMs: COOLDOWN_MS.paymentRequired, reason: RateLimitReason.QUOTA_EXHAUSTED };
}
if (status === HTTP_STATUS.NOT_FOUND) {
return { shouldFallback: true, cooldownMs: COOLDOWN_MS.notFound };
return { shouldFallback: true, cooldownMs: COOLDOWN_MS.notFound, reason: RateLimitReason.UNKNOWN };
}
// 429 - Rate limit with exponential backoff
@@ -68,10 +280,11 @@ export function checkFallbackError(status, errorText, backoffLevel = 0) {
shouldFallback: true,
cooldownMs: getQuotaCooldown(backoffLevel),
newBackoffLevel: newLevel,
reason: RateLimitReason.RATE_LIMIT_EXCEEDED,
};
}
// Transient errors
// Transient / server errors — DON'T increase backoff level
const transientStatuses = [
HTTP_STATUS.NOT_ACCEPTABLE,
HTTP_STATUS.REQUEST_TIMEOUT,
@@ -81,18 +294,20 @@ export function checkFallbackError(status, errorText, backoffLevel = 0) {
HTTP_STATUS.GATEWAY_TIMEOUT,
];
if (transientStatuses.includes(status)) {
return { shouldFallback: true, cooldownMs: COOLDOWN_MS.transient };
return { shouldFallback: true, cooldownMs: COOLDOWN_MS.transient, reason: RateLimitReason.SERVER_ERROR };
}
// 400 Bad Request - don't fallback (same request will fail on all accounts)
if (status === HTTP_STATUS.BAD_REQUEST) {
return { shouldFallback: false, cooldownMs: 0 };
return { shouldFallback: false, cooldownMs: 0, reason: RateLimitReason.UNKNOWN };
}
// All other errors - fallback with transient cooldown
return { shouldFallback: true, cooldownMs: COOLDOWN_MS.transient };
return { shouldFallback: true, cooldownMs: COOLDOWN_MS.transient, reason: RateLimitReason.UNKNOWN };
}
// ─── Account State Management ───────────────────────────────────────────────
/**
* Check if account is currently unavailable (cooldown not expired)
*/
@@ -110,8 +325,6 @@ export function getUnavailableUntil(cooldownMs) {
/**
* Get the earliest rateLimitedUntil from a list of accounts
* @param {Array} accounts - Array of account objects with rateLimitedUntil
* @returns {string|null} Earliest rateLimitedUntil ISO string, or null
*/
export function getEarliestRateLimitedUntil(accounts) {
let earliest = null;
@@ -128,8 +341,6 @@ export function getEarliestRateLimitedUntil(accounts) {
/**
* Format rateLimitedUntil to human-readable "reset after Xm Ys"
* @param {string} rateLimitedUntil - ISO timestamp
* @returns {string} e.g. "reset after 2m 30s"
*/
export function formatRetryAfter(rateLimitedUntil) {
if (!rateLimitedUntil) return "";
@@ -163,9 +374,6 @@ export function filterAvailableAccounts(accounts, excludeId = null) {
/**
* Reset account state when request succeeds
* Clears cooldown and resets backoff level to 0
* @param {object} account - Account object
* @returns {object} Updated account with reset state
*/
export function resetAccountState(account) {
if (!account) return account;
@@ -180,22 +388,32 @@ export function resetAccountState(account) {
/**
* Apply error state to account
* @param {object} account - Account object
* @param {number} status - HTTP status code
* @param {string} errorText - Error message
* @returns {object} Updated account with error state
*/
export function applyErrorState(account, status, errorText) {
if (!account) return account;
const backoffLevel = account.backoffLevel || 0;
const { cooldownMs, newBackoffLevel } = checkFallbackError(status, errorText, backoffLevel);
const { cooldownMs, newBackoffLevel, reason } = checkFallbackError(status, errorText, backoffLevel);
return {
...account,
rateLimitedUntil: cooldownMs > 0 ? getUnavailableUntil(cooldownMs) : null,
backoffLevel: newBackoffLevel ?? backoffLevel,
lastError: { status, message: errorText, timestamp: new Date().toISOString() },
lastError: { status, message: errorText, timestamp: new Date().toISOString(), reason },
status: "error",
};
}
/**
* Get account health score (0-100) for P2C selection (Phase 9)
* @param {object} account
* @returns {number} score 0 = unhealthy, 100 = perfectly healthy
*/
export function getAccountHealth(account) {
if (!account) return 0;
let score = 100;
score -= (account.backoffLevel || 0) * 10;
if (account.lastError) score -= 20;
if (account.rateLimitedUntil && isAccountUnavailable(account.rateLimitedUntil)) score -= 30;
return Math.max(0, score);
}

View File

@@ -0,0 +1,74 @@
/**
* Quota-Aware Account Selection (P2C) — Phase 9
*
* Power of Two Choices: pick 2 random accounts, select the healthier one.
* Uses account health scores from accountFallback.js.
*/
import { getAccountHealth } from "./accountFallback.js";
/**
* P2C selection: pick 2 random candidates, return the healthier one.
* Falls back to random if only 1 candidate.
*
* @param {Array} accounts - Available account objects
* @param {string} [model] - Model name (for model-specific health check)
* @returns {object|null} Selected account
*/
export function selectAccountP2C(accounts, model = null) {
if (!accounts || accounts.length === 0) return null;
if (accounts.length === 1) return accounts[0];
// Pick 2 random distinct indices
const i = Math.floor(Math.random() * accounts.length);
let j = Math.floor(Math.random() * (accounts.length - 1));
if (j >= i) j++; // Ensure distinct
const a = accounts[i];
const b = accounts[j];
const healthA = getAccountHealth(a, model);
const healthB = getAccountHealth(b, model);
return healthA >= healthB ? a : b;
}
/**
* Select account with strategy support.
* Integrates P2C as a new strategy alongside existing fill-first and round-robin.
*
* @param {Array} accounts - Available accounts
* @param {string} strategy - "fill-first" | "round-robin" | "p2c" | "random"
* @param {object} [state] - Strategy state (e.g., lastIndex for round-robin)
* @param {string} [model] - Model name
* @returns {{ account: object|null, state: object }}
*/
export function selectAccount(accounts, strategy = "fill-first", state = {}, model = null) {
if (!accounts || accounts.length === 0) {
return { account: null, state };
}
switch (strategy) {
case "p2c":
return { account: selectAccountP2C(accounts, model), state };
case "random":
return {
account: accounts[Math.floor(Math.random() * accounts.length)],
state,
};
case "round-robin": {
const lastIndex = state.lastIndex ?? -1;
const nextIndex = (lastIndex + 1) % accounts.length;
return {
account: accounts[nextIndex],
state: { ...state, lastIndex: nextIndex },
};
}
case "fill-first":
default:
return { account: accounts[0], state };
}
}

View File

@@ -0,0 +1,203 @@
/**
* Context Manager — Phase 4
*
* Pre-flight context compression to prevent "prompt too long" errors.
* 3 layers: trim tool messages, compress thinking, aggressive purification.
*/
// Default token limits per provider (rough estimates based on model context windows)
const DEFAULT_LIMITS = {
claude: 200000,
openai: 128000,
gemini: 1000000,
default: 128000,
};
// Rough chars-per-token ratio for quick estimation
const CHARS_PER_TOKEN = 4;
/**
* Estimate token count from text length
*/
export function estimateTokens(text) {
if (!text) return 0;
const str = typeof text === "string" ? text : JSON.stringify(text);
return Math.ceil(str.length / CHARS_PER_TOKEN);
}
/**
* Get token limit for a provider/model combination
*/
export function getTokenLimit(provider, model = null) {
// Check if model has a known limit
if (model) {
const lower = model.toLowerCase();
if (lower.includes("claude")) return DEFAULT_LIMITS.claude;
if (lower.includes("gemini")) return DEFAULT_LIMITS.gemini;
if (lower.includes("gpt") || lower.includes("o1") || lower.includes("o3") || lower.includes("o4")) return DEFAULT_LIMITS.openai;
}
return DEFAULT_LIMITS[provider] || DEFAULT_LIMITS.default;
}
/**
* Apply context compression to request body.
* Operates in 3 layers of increasing aggressiveness:
*
* Layer 1: Trim tool_result messages (truncate long outputs)
* Layer 2: Compress thinking blocks (remove from history, keep last)
* Layer 3: Aggressive purification (drop old messages until fitting)
*
* @param {object} body - Request body with messages[]
* @param {object} options - { provider?, model?, maxTokens?, reserveTokens? }
* @returns {{ body: object, compressed: boolean, stats: object }}
*/
export function compressContext(body, options = {}) {
if (!body || !body.messages || !Array.isArray(body.messages)) {
return { body, compressed: false, stats: {} };
}
const provider = options.provider || "default";
const maxTokens = options.maxTokens || getTokenLimit(provider, body.model || options.model);
const reserveTokens = options.reserveTokens || 16000; // Reserve for response
const targetTokens = maxTokens - reserveTokens;
let messages = [...body.messages];
let currentTokens = estimateTokens(JSON.stringify(messages));
const stats = { original: currentTokens, layers: [] };
// Already fits
if (currentTokens <= targetTokens) {
return { body, compressed: false, stats: { original: currentTokens, final: currentTokens } };
}
// Layer 1: Trim tool_result/tool messages
messages = trimToolMessages(messages, 2000); // Max 2000 chars per tool result
currentTokens = estimateTokens(JSON.stringify(messages));
stats.layers.push({ name: "trim_tools", tokens: currentTokens });
if (currentTokens <= targetTokens) {
return {
body: { ...body, messages },
compressed: true,
stats: { ...stats, final: currentTokens },
};
}
// Layer 2: Compress thinking blocks (remove from non-last assistant messages)
messages = compressThinking(messages);
currentTokens = estimateTokens(JSON.stringify(messages));
stats.layers.push({ name: "compress_thinking", tokens: currentTokens });
if (currentTokens <= targetTokens) {
return {
body: { ...body, messages },
compressed: true,
stats: { ...stats, final: currentTokens },
};
}
// Layer 3: Aggressive purification — drop oldest messages keeping system + last N pairs
messages = purifyHistory(messages, targetTokens);
currentTokens = estimateTokens(JSON.stringify(messages));
stats.layers.push({ name: "purify_history", tokens: currentTokens });
return {
body: { ...body, messages },
compressed: true,
stats: { ...stats, final: currentTokens },
};
}
// ─── Layer 1: Trim Tool Messages ────────────────────────────────────────────
function trimToolMessages(messages, maxChars) {
return messages.map((msg) => {
if (msg.role === "tool" && typeof msg.content === "string" && msg.content.length > maxChars) {
return {
...msg,
content: msg.content.slice(0, maxChars) + "\n... [truncated]",
};
}
// Handle array content (Claude format with tool_result blocks)
if (msg.role === "user" && Array.isArray(msg.content)) {
return {
...msg,
content: msg.content.map((block) => {
if (block.type === "tool_result" && typeof block.content === "string" && block.content.length > maxChars) {
return { ...block, content: block.content.slice(0, maxChars) + "\n... [truncated]" };
}
return block;
}),
};
}
return msg;
});
}
// ─── Layer 2: Compress Thinking Blocks ──────────────────────────────────────
function compressThinking(messages) {
// Find last assistant message index
let lastAssistantIdx = -1;
for (let i = messages.length - 1; i >= 0; i--) {
if (messages[i].role === "assistant") {
lastAssistantIdx = i;
break;
}
}
return messages.map((msg, i) => {
if (msg.role !== "assistant") return msg;
if (i === lastAssistantIdx) return msg; // Keep thinking in last assistant msg
// Remove thinking blocks from content array
if (Array.isArray(msg.content)) {
const filtered = msg.content.filter((block) => block.type !== "thinking");
if (filtered.length === 0) {
return { ...msg, content: "[thinking compressed]" };
}
return { ...msg, content: filtered };
}
// Remove thinking XML tags from string content
if (typeof msg.content === "string") {
const cleaned = msg.content
.replace(/<thinking>[\s\S]*?<\/thinking>/g, "")
.replace(/<antThinking>[\s\S]*?<\/antThinking>/g, "")
.trim();
return { ...msg, content: cleaned || "[thinking compressed]" };
}
return msg;
});
}
// ─── Layer 3: Aggressive Purification ───────────────────────────────────────
function purifyHistory(messages, targetTokens) {
// Keep system message(s) and the last N message pairs
const system = messages.filter((m) => m.role === "system" || m.role === "developer");
const nonSystem = messages.filter((m) => m.role !== "system" && m.role !== "developer");
// Binary search for how many messages to keep from the end
let keep = nonSystem.length;
while (keep > 2) {
const candidate = [...system, ...nonSystem.slice(-keep)];
const tokens = estimateTokens(JSON.stringify(candidate));
if (tokens <= targetTokens) break;
keep = Math.max(2, Math.floor(keep * 0.7)); // Drop 30% each iteration
}
const result = [...system, ...nonSystem.slice(-keep)];
// Add summary of dropped messages
if (keep < nonSystem.length) {
const dropped = nonSystem.length - keep;
result.splice(system.length, 0, {
role: "system",
content: `[Context compressed: ${dropped} earlier messages removed to fit context window]`,
});
}
return result;
}

View File

@@ -0,0 +1,249 @@
/**
* IP Filter Middleware — Phase 6
*
* IP-based access control with blacklist, whitelist, priority modes, and temporary bans.
*/
// In-memory IP lists
let _config = {
enabled: false,
mode: "blacklist", // "blacklist" | "whitelist" | "whitelist-priority"
blacklist: new Set(),
whitelist: new Set(),
tempBans: new Map(), // IP → { until: timestamp, reason: string }
};
/**
* Configure the IP filter
*/
export function configureIPFilter(config) {
if (config.enabled !== undefined) _config.enabled = config.enabled;
if (config.mode) _config.mode = config.mode;
if (config.blacklist) _config.blacklist = new Set(config.blacklist);
if (config.whitelist) _config.whitelist = new Set(config.whitelist);
}
/**
* Get current IP filter config (for API)
*/
export function getIPFilterConfig() {
return {
enabled: _config.enabled,
mode: _config.mode,
blacklist: Array.from(_config.blacklist),
whitelist: Array.from(_config.whitelist),
tempBans: Array.from(_config.tempBans.entries()).map(([ip, info]) => ({
ip,
until: new Date(info.until).toISOString(),
reason: info.reason,
remainingMs: Math.max(0, info.until - Date.now()),
})),
};
}
/**
* Check if an IP is allowed
* @param {string} ip
* @returns {{ allowed: boolean, reason?: string }}
*/
export function checkIP(ip) {
if (!_config.enabled) return { allowed: true };
if (!ip) return { allowed: true };
const normalizedIP = normalizeIP(ip);
// Check temp bans first (highest priority)
const ban = _config.tempBans.get(normalizedIP);
if (ban) {
if (Date.now() < ban.until) {
return { allowed: false, reason: `Temporarily banned: ${ban.reason}` };
}
_config.tempBans.delete(normalizedIP); // Expired
}
switch (_config.mode) {
case "whitelist":
// Only whitelisted IPs allowed
if (!matchesAny(normalizedIP, _config.whitelist)) {
return { allowed: false, reason: "IP not in whitelist" };
}
return { allowed: true };
case "whitelist-priority":
// Whitelist overrides blacklist
if (matchesAny(normalizedIP, _config.whitelist)) {
return { allowed: true };
}
if (matchesAny(normalizedIP, _config.blacklist)) {
return { allowed: false, reason: "IP blacklisted" };
}
return { allowed: true };
case "blacklist":
default:
// Blacklisted IPs blocked
if (matchesAny(normalizedIP, _config.blacklist)) {
return { allowed: false, reason: "IP blacklisted" };
}
return { allowed: true };
}
}
/**
* Temporarily ban an IP
*/
export function tempBanIP(ip, durationMs, reason = "Automated ban") {
const normalizedIP = normalizeIP(ip);
_config.tempBans.set(normalizedIP, {
until: Date.now() + durationMs,
reason,
});
}
/**
* Remove a temporary ban
*/
export function removeTempBan(ip) {
_config.tempBans.delete(normalizeIP(ip));
}
/**
* Add IP to blacklist
*/
export function addToBlacklist(ip) {
_config.blacklist.add(normalizeIP(ip));
}
/**
* Remove IP from blacklist
*/
export function removeFromBlacklist(ip) {
_config.blacklist.delete(normalizeIP(ip));
}
/**
* Add IP to whitelist
*/
export function addToWhitelist(ip) {
_config.whitelist.add(normalizeIP(ip));
}
/**
* Remove IP from whitelist
*/
export function removeFromWhitelist(ip) {
_config.whitelist.delete(normalizeIP(ip));
}
/**
* Express/Next.js middleware factory
*/
export function createIPFilterMiddleware() {
return (req, res, next) => {
const ip = extractClientIP(req);
const { allowed, reason } = checkIP(ip);
if (!allowed) {
const statusCode = 403;
if (res.status) {
// Express
return res.status(statusCode).json({ error: reason || "Access denied" });
}
// Raw response
res.writeHead(statusCode, { "Content-Type": "application/json" });
return res.end(JSON.stringify({ error: reason || "Access denied" }));
}
if (next) next();
};
}
/**
* For Next.js App Router — check IP from request object
*/
export function checkRequestIP(request) {
const ip =
request.headers?.get?.("x-forwarded-for")?.split(",")[0].trim() ||
request.headers?.get?.("x-real-ip") ||
request.headers?.get?.("cf-connecting-ip") ||
request.ip ||
"unknown";
return checkIP(ip);
}
// ─── Internal Helpers ───────────────────────────────────────────────────────
function normalizeIP(ip) {
if (!ip) return "";
// Remove IPv6 prefix from IPv4-mapped addresses
return ip.replace(/^::ffff:/, "").trim();
}
function matchesAny(ip, ipSet) {
// Direct match
if (ipSet.has(ip)) return true;
// CIDR match
for (const entry of ipSet) {
if (entry.includes("/") && matchesCIDR(ip, entry)) return true;
// Wildcard match (e.g., "192.168.*.*")
if (entry.includes("*") && matchesWildcard(ip, entry)) return true;
}
return false;
}
function matchesCIDR(ip, cidr) {
try {
const [range, bits] = cidr.split("/");
const mask = parseInt(bits, 10);
if (isNaN(mask) || mask < 0 || mask > 32) return false;
const ipNum = ipToNum(ip);
const rangeNum = ipToNum(range);
if (ipNum === null || rangeNum === null) return false;
const maskBits = (-1 << (32 - mask)) >>> 0;
return (ipNum & maskBits) === (rangeNum & maskBits);
} catch {
return false;
}
}
function ipToNum(ip) {
const parts = ip.split(".");
if (parts.length !== 4) return null;
let num = 0;
for (const p of parts) {
const n = parseInt(p, 10);
if (isNaN(n) || n < 0 || n > 255) return null;
num = (num << 8) | n;
}
return num >>> 0;
}
function matchesWildcard(ip, pattern) {
const ipParts = ip.split(".");
const patParts = pattern.split(".");
if (ipParts.length !== 4 || patParts.length !== 4) return false;
return ipParts.every((part, i) => patParts[i] === "*" || part === patParts[i]);
}
function extractClientIP(req) {
return (
req.headers?.["x-forwarded-for"]?.split(",")[0].trim() ||
req.headers?.["x-real-ip"] ||
req.headers?.["cf-connecting-ip"] ||
req.socket?.remoteAddress ||
req.ip ||
"unknown"
);
}
/**
* Reset config (for testing)
*/
export function resetIPFilter() {
_config = {
enabled: false,
mode: "blacklist",
blacklist: new Set(),
whitelist: new Set(),
tempBans: new Map(),
};
}

View File

@@ -8,6 +8,7 @@
*/
import Bottleneck from "bottleneck";
import { parseRetryAfterFromBody, lockModel } from "./accountFallback.js";
// Store limiters keyed by "provider:connectionId" (and optionally ":model")
const limiters = new Map();
@@ -325,3 +326,39 @@ export function getAllRateLimitStatus() {
}
return result;
}
/**
* Update rate limiter based on API response body (JSON error responses).
* Providers embed retry info in JSON payloads in different formats.
* Should be called alongside updateFromHeaders for 4xx/5xx responses.
*
* @param {string} provider - Provider ID
* @param {string} connectionId - Connection ID
* @param {string|object} responseBody - Response body (string or parsed JSON)
* @param {number} status - HTTP status code
* @param {string} model - Model name (for per-model lockouts)
*/
export function updateFromResponseBody(provider, connectionId, responseBody, status, model = null) {
if (!enabledConnections.has(connectionId)) return;
const { retryAfterMs, reason } = parseRetryAfterFromBody(responseBody);
if (retryAfterMs && retryAfterMs > 0) {
const limiter = getLimiter(provider, connectionId, null);
console.log(
`🚫 [RATE-LIMIT] ${provider}:${connectionId.slice(0, 8)} — body-parsed retry: ${Math.ceil(retryAfterMs / 1000)}s (${reason})`
);
limiter.updateSettings({
reservoir: 0,
reservoirRefreshAmount: 60,
reservoirRefreshInterval: retryAfterMs,
});
// Also apply model-level lockout if model is known
if (model) {
lockModel(provider, connectionId, model, reason, retryAfterMs);
}
}
}

View File

@@ -0,0 +1,179 @@
/**
* Session Fingerprinting — Phase 5
*
* Generates stable session IDs for sticky routing,
* prompt caching, and per-session tracking.
*/
import { createHash } from "node:crypto";
// In-memory session store with metadata
// key: sessionId → { createdAt, lastActive, requestCount, connectionId? }
const sessions = new Map();
// Auto-cleanup sessions older than 30 minutes
const SESSION_TTL_MS = 30 * 60 * 1000;
const _cleanupTimer = setInterval(() => {
const now = Date.now();
for (const [key, entry] of sessions) {
if (now - entry.lastActive > SESSION_TTL_MS) sessions.delete(key);
}
}, 60_000);
_cleanupTimer.unref();
/**
* Generate a stable session fingerprint from request characteristics.
* Same client + same conversation → same session ID.
*
* Fingerprint factors:
* - System prompt hash (stable per conversation/tool)
* - First user message hash (stable per conversation)
* - Model name
* - Provider (optional)
* - Tools signature (sorted tool names)
*
* @param {object} body - Request body
* @param {object} [options] - Extra context
* @returns {string} Session ID (hex hash)
*/
export function generateSessionId(body, options = {}) {
const parts = [];
// Model contributes to fingerprint
if (body.model) parts.push(`model:${body.model}`);
// Provider binding
if (options.provider) parts.push(`provider:${options.provider}`);
// System prompt hash (first 32 chars of system content)
const systemPrompt = extractSystemPrompt(body);
if (systemPrompt) {
parts.push(`sys:${hashShort(systemPrompt)}`);
}
// First user message hash (identifies the conversation)
const firstUser = extractFirstUserMessage(body);
if (firstUser) {
parts.push(`user0:${hashShort(firstUser)}`);
}
// Tools signature (sorted names)
if (body.tools && Array.isArray(body.tools) && body.tools.length > 0) {
const toolNames = body.tools
.map((t) => t.name || t.function?.name || "")
.filter(Boolean)
.sort()
.join(",");
if (toolNames) parts.push(`tools:${hashShort(toolNames)}`);
}
// Connection ID for sticky routing
if (options.connectionId) parts.push(`conn:${options.connectionId}`);
if (parts.length === 0) return null;
const fingerprint = parts.join("|");
return createHash("sha256").update(fingerprint).digest("hex").slice(0, 16);
}
/**
* Touch or create a session
*/
export function touchSession(sessionId, connectionId = null) {
if (!sessionId) return;
const existing = sessions.get(sessionId);
if (existing) {
existing.lastActive = Date.now();
existing.requestCount++;
if (connectionId) existing.connectionId = connectionId;
} else {
sessions.set(sessionId, {
createdAt: Date.now(),
lastActive: Date.now(),
requestCount: 1,
connectionId,
});
}
}
/**
* Get session info (for sticky routing decisions)
*/
export function getSessionInfo(sessionId) {
if (!sessionId) return null;
const entry = sessions.get(sessionId);
if (!entry) return null;
if (Date.now() - entry.lastActive > SESSION_TTL_MS) {
sessions.delete(sessionId);
return null;
}
return { ...entry };
}
/**
* Get the bound connection for a session (sticky routing)
*/
export function getSessionConnection(sessionId) {
const info = getSessionInfo(sessionId);
return info?.connectionId || null;
}
/**
* Get session count (for dashboard)
*/
export function getActiveSessionCount() {
return sessions.size;
}
/**
* Get all active sessions (for dashboard)
*/
export function getActiveSessions() {
const now = Date.now();
const result = [];
for (const [id, entry] of sessions) {
if (now - entry.lastActive <= SESSION_TTL_MS) {
result.push({ sessionId: id, ...entry, ageMs: now - entry.createdAt });
}
}
return result;
}
/**
* Clear all sessions (for testing)
*/
export function clearSessions() {
sessions.clear();
}
// ─── Internal Helpers ───────────────────────────────────────────────────────
function hashShort(text) {
return createHash("sha256").update(text).digest("hex").slice(0, 8);
}
function extractSystemPrompt(body) {
// Claude format: body.system
if (body.system) {
return typeof body.system === "string" ? body.system : JSON.stringify(body.system);
}
// OpenAI format: messages[0].role === "system"
if (body.messages && Array.isArray(body.messages)) {
const sys = body.messages.find((m) => m.role === "system" || m.role === "developer");
if (sys) {
return typeof sys.content === "string" ? sys.content : JSON.stringify(sys.content);
}
}
return null;
}
function extractFirstUserMessage(body) {
const messages = body.messages || body.input || [];
if (!Array.isArray(messages)) return null;
for (const msg of messages) {
if (msg.role === "user") {
return typeof msg.content === "string" ? msg.content : JSON.stringify(msg.content);
}
}
return null;
}

View File

@@ -0,0 +1,171 @@
/**
* Signature Cache — Phase 3
*
* Dynamic 3-layer cache for thinking signatures (tool, model family, session).
* Replaces hardcoded thinking signature patterns with adaptive detection.
*/
// 3-layer cache: tool → model family → session
// Each layer stores patterns detected from responses
const layers = {
tool: new Map(), // e.g. "cursor" → Set of signature patterns
family: new Map(), // e.g. "claude-sonnet" → Set of signature patterns
session: new Map(), // e.g. sessionId → Set of signature patterns
};
// Known default signatures (bootstrap — will be supplemented by learning)
const DEFAULT_SIGNATURES = [
"<antThinking>",
"</antThinking>",
"<thinking>",
"</thinking>",
"<internal_thought>",
"</internal_thought>",
];
// Max entries per cache layer to prevent unbounded growth
const MAX_ENTRIES_PER_LAYER = 500;
const MAX_PATTERNS_PER_KEY = 50;
/**
* Get all matching signatures for a given context.
* Checks all 3 layers and merges results with defaults.
*
* @param {object} context - { tool?, modelFamily?, sessionId? }
* @returns {string[]} Array of unique signature patterns
*/
export function getSignatures(context = {}) {
const patterns = new Set(DEFAULT_SIGNATURES);
// Layer 1: Tool (e.g., "cursor", "cline", "antigravity")
if (context.tool && layers.tool.has(context.tool)) {
for (const p of layers.tool.get(context.tool)) patterns.add(p);
}
// Layer 2: Model family (e.g., "claude-sonnet", "claude-opus")
if (context.modelFamily && layers.family.has(context.modelFamily)) {
for (const p of layers.family.get(context.modelFamily)) patterns.add(p);
}
// Layer 3: Session-specific
if (context.sessionId && layers.session.has(context.sessionId)) {
for (const p of layers.session.get(context.sessionId)) patterns.add(p);
}
return Array.from(patterns);
}
/**
* Add a discovered signature pattern to the cache.
*
* @param {string} pattern - The signature pattern (e.g., "<antThinking>")
* @param {object} context - { tool?, modelFamily?, sessionId? }
*/
export function addSignature(pattern, context = {}) {
if (!pattern || typeof pattern !== "string") return;
const addToLayer = (layer, key) => {
if (!key) return;
if (!layer.has(key)) {
if (layer.size >= MAX_ENTRIES_PER_LAYER) {
// Evict oldest entry
const firstKey = layer.keys().next().value;
layer.delete(firstKey);
}
layer.set(key, new Set());
}
const set = layer.get(key);
if (set.size < MAX_PATTERNS_PER_KEY) {
set.add(pattern);
}
};
addToLayer(layers.tool, context.tool);
addToLayer(layers.family, context.modelFamily);
addToLayer(layers.session, context.sessionId);
}
/**
* Detect signatures in a text chunk from streaming response.
* Auto-learns new patterns and adds them to cache.
*
* @param {string} text - Streaming text to scan
* @param {object} context - { tool?, modelFamily?, sessionId? }
* @returns {{ found: string[], cleaned: string }} Detected tags and cleaned text
*/
export function detectAndLearn(text, context = {}) {
if (!text || typeof text !== "string") return { found: [], cleaned: text };
const found = [];
let cleaned = text;
// Check all known signatures
const known = getSignatures(context);
for (const sig of known) {
if (cleaned.includes(sig)) {
found.push(sig);
cleaned = cleaned.split(sig).join("");
}
}
// Auto-detect new XML-like thinking tags
const tagRegex = /<\/?([a-zA-Z_][a-zA-Z0-9_]*(?:Thinking|thinking|thought|Thought|internal_thought))>/g;
let match;
while ((match = tagRegex.exec(text)) !== null) {
const tag = match[0];
if (!known.includes(tag)) {
found.push(tag);
addSignature(tag, context);
cleaned = cleaned.split(tag).join("");
}
}
return { found, cleaned: cleaned.trim() || cleaned };
}
/**
* Extract model family from model name
* "claude-sonnet-4-20250514" → "claude-sonnet"
* "gpt-4o-2024-08-06" → "gpt-4o"
*/
export function getModelFamily(model) {
if (!model) return null;
// Remove date suffixes and version numbers
const cleaned = model
.replace(/-\d{4}-\d{2}-\d{2}$/, "") // Remove YYYY-MM-DD suffix
.replace(/-\d{8,}$/, "") // Remove YYYYMMDD suffix
.replace(/-\d+(\.\d+)*$/, "") // Remove version suffix like -4
.replace(/@.*$/, ""); // Remove @latest etc.
// Keep meaningful prefix
return cleaned || model;
}
/**
* Get cache stats (for dashboard)
*/
export function getCacheStats() {
return {
tool: {
entries: layers.tool.size,
patterns: Array.from(layers.tool.values()).reduce((sum, s) => sum + s.size, 0),
},
family: {
entries: layers.family.size,
patterns: Array.from(layers.family.values()).reduce((sum, s) => sum + s.size, 0),
},
session: {
entries: layers.session.size,
patterns: Array.from(layers.session.values()).reduce((sum, s) => sum + s.size, 0),
},
defaultCount: DEFAULT_SIGNATURES.length,
};
}
/**
* Clear all cache layers (for testing)
*/
export function clearCache() {
layers.tool.clear();
layers.family.clear();
layers.session.clear();
}

View File

@@ -0,0 +1,66 @@
/**
* System Prompt Injection — Phase 10
*
* Injects a global system prompt into all requests at proxy level.
*/
// In-memory config
let _config = {
enabled: false,
prompt: "",
};
/**
* Set system prompt config
*/
export function setSystemPromptConfig(config) {
_config = { ..._config, ...config };
}
/**
* Get system prompt config
*/
export function getSystemPromptConfig() {
return { ..._config };
}
/**
* Inject system prompt into request body.
*
* @param {object} body - Request body
* @param {string} [promptText] - Override prompt text
* @returns {object} Modified body
*/
export function injectSystemPrompt(body, promptText = null) {
const text = promptText || _config.prompt;
if (!text || !_config.enabled) return body;
if (!body || typeof body !== "object") return body;
if (body._skipSystemPrompt) return body;
const result = { ...body };
// OpenAI/Claude format (messages[])
if (result.messages && Array.isArray(result.messages)) {
const sysIdx = result.messages.findIndex((m) => m.role === "system" || m.role === "developer");
result.messages = [...result.messages];
if (sysIdx >= 0) {
// Prepend to existing system message
const msg = { ...result.messages[sysIdx] };
msg.content = text + "\n\n" + (msg.content || "");
result.messages[sysIdx] = msg;
} else {
result.messages = [{ role: "system", content: text }, ...result.messages];
}
}
// Claude format (system field)
if (result.system !== undefined) {
if (typeof result.system === "string") {
result.system = text + "\n\n" + result.system;
} else if (Array.isArray(result.system)) {
result.system = [{ type: "text", text }, ...result.system];
}
}
return result;
}

View File

@@ -0,0 +1,186 @@
/**
* Thinking Budget Control — Phase 2
*
* Provides proxy-level control over AI thinking/reasoning budgets.
* Modes: auto, passthrough, custom, adaptive
*/
// Thinking budget modes
export const ThinkingMode = {
AUTO: "auto", // Let provider decide (remove client's budget)
PASSTHROUGH: "passthrough", // No changes (current behavior)
CUSTOM: "custom", // Set fixed budget
ADAPTIVE: "adaptive", // Scale based on request complexity
};
// Effort → budget token mapping
export const EFFORT_BUDGETS = {
none: 0,
low: 1024,
medium: 10240,
high: 131072,
};
// Default config (passthrough = backward compatible)
export const DEFAULT_THINKING_CONFIG = {
mode: ThinkingMode.PASSTHROUGH,
customBudget: 10240,
effortLevel: "medium",
};
// In-memory config (loaded from DB on startup, or default)
let _config = { ...DEFAULT_THINKING_CONFIG };
/**
* Set the thinking budget config (called from settings API or startup)
*/
export function setThinkingBudgetConfig(config) {
_config = { ...DEFAULT_THINKING_CONFIG, ...config };
}
/**
* Get current thinking budget config
*/
export function getThinkingBudgetConfig() {
return { ..._config };
}
/**
* Apply thinking budget control to a request body.
* Called before format-specific translation.
*
* @param {object} body - Request body (any format)
* @param {object} [config] - Override config (defaults to stored config)
* @returns {object} Modified body
*/
export function applyThinkingBudget(body, config = null) {
const cfg = config || _config;
if (!body || typeof body !== "object") return body;
switch (cfg.mode) {
case ThinkingMode.AUTO:
return stripThinkingConfig(body);
case ThinkingMode.PASSTHROUGH:
return body; // No changes
case ThinkingMode.CUSTOM:
return setCustomBudget(body, cfg.customBudget);
case ThinkingMode.ADAPTIVE:
return applyAdaptiveBudget(body, cfg);
default:
return body;
}
}
/**
* AUTO mode: strip all thinking configuration, let provider decide
*/
function stripThinkingConfig(body) {
const result = { ...body };
// Claude format
delete result.thinking;
// OpenAI format
delete result.reasoning_effort;
delete result.reasoning;
// Gemini format
if (result.generationConfig) {
result.generationConfig = { ...result.generationConfig };
delete result.generationConfig.thinking_config;
delete result.generationConfig.thinkingConfig;
}
return result;
}
/**
* CUSTOM mode: set exact budget tokens
*/
function setCustomBudget(body, budget) {
const result = { ...body };
// If body already has thinking config in Claude format, update it
if (result.thinking || hasThinkingCapableModel(result)) {
result.thinking = {
type: budget > 0 ? "enabled" : "disabled",
budget_tokens: budget,
};
}
// OpenAI reasoning_effort mapping
if (result.reasoning_effort !== undefined || result.reasoning !== undefined) {
if (budget <= 0) {
delete result.reasoning_effort;
delete result.reasoning;
} else if (budget <= 1024) {
result.reasoning_effort = "low";
} else if (budget <= 10240) {
result.reasoning_effort = "medium";
} else {
result.reasoning_effort = "high";
}
}
// Gemini thinking_config
if (result.generationConfig?.thinking_config || result.generationConfig?.thinkingConfig) {
result.generationConfig = {
...result.generationConfig,
thinking_config: { thinking_budget: budget },
};
}
return result;
}
/**
* ADAPTIVE mode: scale budget based on request complexity
*/
function applyAdaptiveBudget(body, cfg) {
const messages = body.messages || body.input || [];
const messageCount = messages.length;
const tools = body.tools || [];
const toolCount = tools.length;
// Get last user message length
let lastMsgLength = 0;
for (let i = messages.length - 1; i >= 0; i--) {
const msg = messages[i];
if (msg.role === "user") {
lastMsgLength = typeof msg.content === "string"
? msg.content.length
: JSON.stringify(msg.content || "").length;
break;
}
}
// Calculate multiplier
let multiplier = 1.0;
if (messageCount > 10) multiplier += 0.5;
if (toolCount > 3) multiplier += 0.5;
if (lastMsgLength > 2000) multiplier += 0.3;
const baseBudget = EFFORT_BUDGETS[cfg.effortLevel] || EFFORT_BUDGETS.medium;
const budget = Math.min(Math.ceil(baseBudget * multiplier), 131072);
return setCustomBudget(body, budget);
}
/**
* Check if model name suggests thinking capability
*/
function hasThinkingCapableModel(body) {
const model = body.model || "";
return (
model.includes("claude") ||
model.includes("o1") ||
model.includes("o3") ||
model.includes("o4") ||
model.includes("gemini") ||
model.includes("thinking")
);
}

View File

@@ -0,0 +1,117 @@
/**
* Wildcard Model Routing — Phase 8
*
* Glob-style wildcard matching for model aliases with specificity ranking.
* Integrates into the existing model resolution pipeline.
*/
/**
* Match a model name against a pattern with glob wildcards.
* Supports * (any sequence) and ? (single char).
*
* @param {string} model - Model name to match
* @param {string} pattern - Pattern with wildcards
* @returns {boolean}
*/
export function wildcardMatch(model, pattern) {
if (!model || !pattern) return false;
if (pattern === "*") return true;
if (pattern === model) return true;
// Convert glob pattern to regex
const regexStr = pattern
.replace(/[.+^${}()|[\]\\]/g, "\\$&")
.replace(/\*/g, ".*")
.replace(/\?/g, ".");
const regex = new RegExp(`^${regexStr}$`, "i");
return regex.test(model);
}
/**
* Calculate specificity score for a pattern.
* Higher score = more specific match.
* Used to rank multiple matching patterns.
*
* @param {string} pattern
* @returns {number} Specificity score (higher = more specific)
*/
export function getSpecificity(pattern) {
if (!pattern) return 0;
let score = 0;
// Exact segments (no wildcards) contribute most
const segments = pattern.split(/[/*?]/);
for (const seg of segments) {
if (seg.length > 0) score += seg.length * 10;
}
// Wildcards reduce specificity
const wildcardCount = (pattern.match(/\*/g) || []).length;
const questionCount = (pattern.match(/\?/g) || []).length;
score -= wildcardCount * 50;
score -= questionCount * 5;
// Longer patterns tend to be more specific
score += pattern.length;
return score;
}
/**
* Find the best matching alias for a model name from a list of alias entries.
* Returns the most specific match.
*
* @param {string} model - Model name to resolve
* @param {Array<{ pattern: string, target: string, [key: string]: any }>} aliases - Alias entries
* @returns {{ pattern: string, target: string, specificity: number } | null}
*/
export function resolveWildcardAlias(model, aliases) {
if (!model || !aliases || !Array.isArray(aliases)) return null;
const matches = [];
for (const alias of aliases) {
const pattern = alias.pattern || alias.alias || alias.from;
const target = alias.target || alias.model || alias.to;
if (!pattern || !target) continue;
if (wildcardMatch(model, pattern)) {
matches.push({
pattern,
target,
specificity: getSpecificity(pattern),
...alias,
});
}
}
if (matches.length === 0) return null;
// Sort by specificity (highest first)
matches.sort((a, b) => b.specificity - a.specificity);
return matches[0];
}
/**
* Resolve model through wildcard aliases, falling back to exact match.
* Can be integrated into existing model resolution.
*
* @param {string} model - Requested model name
* @param {Map|Object} exactAliases - Exact model alias map
* @param {Array} wildcardAliases - Wildcard pattern aliases
* @returns {string} Resolved model name
*/
export function resolveModel(model, exactAliases = {}, wildcardAliases = []) {
if (!model) return model;
// 1. Check exact aliases first (fastest)
if (exactAliases instanceof Map) {
if (exactAliases.has(model)) return exactAliases.get(model);
} else if (exactAliases[model]) {
return exactAliases[model];
}
// 2. Check wildcard aliases
const match = resolveWildcardAlias(model, wildcardAliases);
if (match) return match.target;
// 3. Return original
return model;
}

View File

@@ -3,6 +3,7 @@ import { ensureToolCallIds, fixMissingToolResponses } from "./helpers/toolCallHe
import { prepareClaudeRequest } from "./helpers/claudeHelper.js";
import { filterToOpenAIFormat } from "./helpers/openaiHelper.js";
import { normalizeThinkingConfig } from "../services/provider.js";
import { applyThinkingBudget } from "../services/thinkingBudget.js";
// Registry for translators.
// NOTE: translator modules import this file and call register() at module-load time.
@@ -119,6 +120,9 @@ export function translateRequest(
) {
let result = body;
// Phase 2: Apply thinking budget control before normalization
result = applyThinkingBudget(result);
// Normalize thinking config: remove if lastMessage is not user
normalizeThinkingConfig(result);

View File

@@ -1,6 +1,6 @@
{
"name": "omniroute",
"version": "0.1.0",
"version": "0.2.0",
"description": "Smart AI Router with auto fallback — route to FREE & cheap models, zero downtime. Works with Cursor, Cline, Claude Desktop, Codex, and any OpenAI-compatible tool.",
"type": "module",
"bin": {

View File

@@ -0,0 +1,217 @@
"use client";
import { useState, useEffect } from "react";
import { Card, Button, Input, Toggle } from "@/shared/components";
const MODES = [
{ value: "disabled", label: "Disabled", icon: "block" },
{ value: "blacklist", label: "Blacklist", icon: "do_not_disturb" },
{ value: "whitelist", label: "Whitelist", icon: "verified_user" },
{ value: "whitelist-priority", label: "WL Priority", icon: "priority_high" },
];
export default function IPFilterSection() {
const [config, setConfig] = useState({ enabled: false, mode: "blacklist", blacklist: [], whitelist: [], tempBans: [] });
const [loading, setLoading] = useState(true);
const [newIP, setNewIP] = useState("");
const [listTarget, setListTarget] = useState("blacklist"); // "blacklist" | "whitelist"
useEffect(() => {
loadConfig();
}, []);
const loadConfig = async () => {
try {
const res = await fetch("/api/settings/ip-filter");
if (res.ok) setConfig(await res.json());
} catch {} finally {
setLoading(false);
}
};
const updateConfig = async (updates) => {
try {
const res = await fetch("/api/settings/ip-filter", {
method: "PUT",
headers: { "Content-Type": "application/json" },
body: JSON.stringify(updates),
});
if (res.ok) setConfig(await res.json());
} catch {}
};
const toggleEnabled = () => updateConfig({ enabled: !config.enabled });
const setMode = (mode) => {
if (mode === "disabled") {
updateConfig({ enabled: false });
} else {
updateConfig({ enabled: true, mode });
}
};
const addIP = () => {
if (!newIP.trim()) return;
const key = listTarget === "blacklist" ? "addBlacklist" : "addWhitelist";
updateConfig({ [key]: newIP.trim() });
setNewIP("");
};
const removeIP = (ip, list) => {
const key = list === "blacklist" ? "removeBlacklist" : "removeWhitelist";
updateConfig({ [key]: ip });
};
const removeBan = (ip) => updateConfig({ removeBan: ip });
const activeMode = !config.enabled ? "disabled" : config.mode;
return (
<Card>
<div className="flex items-center gap-3 mb-5">
<div className="p-2 rounded-lg bg-red-500/10 text-red-500">
<span className="material-symbols-outlined text-[20px]" aria-hidden="true">
security
</span>
</div>
<div className="flex-1">
<h3 className="text-lg font-semibold">IP Access Control</h3>
<p className="text-sm text-text-muted">Block or allow specific IP addresses</p>
</div>
</div>
{/* Mode selector */}
<div className="grid grid-cols-4 gap-2 mb-5">
{MODES.map((m) => (
<button
key={m.value}
onClick={() => setMode(m.value)}
disabled={loading}
className={`flex flex-col items-center gap-1.5 p-3 rounded-lg border text-center transition-all ${
activeMode === m.value
? "border-red-500/50 bg-red-500/5 ring-1 ring-red-500/20"
: "border-border/50 hover:border-border hover:bg-surface/30"
}`}
>
<span className={`material-symbols-outlined text-[20px] ${
activeMode === m.value ? "text-red-400" : "text-text-muted"
}`}>{m.icon}</span>
<span className={`text-xs font-medium ${activeMode === m.value ? "text-red-400" : "text-text-muted"}`}>
{m.label}
</span>
</button>
))}
</div>
{config.enabled && (
<div className="flex flex-col gap-4">
{/* Add IP */}
<div className="flex gap-2 items-end">
<div className="flex-1">
<Input
label="Add IP Address"
placeholder="192.168.1.0/24 or 10.0.*.*"
value={newIP}
onChange={(e) => setNewIP(e.target.value)}
onKeyDown={(e) => e.key === "Enter" && addIP()}
/>
</div>
<div className="flex gap-1 pb-[2px]">
<Button
size="sm"
variant={listTarget === "blacklist" ? "danger" : "secondary"}
onClick={() => { setListTarget("blacklist"); if (newIP.trim()) addIP(); }}
>
+ Block
</Button>
<Button
size="sm"
variant={listTarget === "whitelist" ? "primary" : "secondary"}
onClick={() => { setListTarget("whitelist"); if (newIP.trim()) addIP(); }}
>
+ Allow
</Button>
</div>
</div>
{/* Blacklist */}
{config.blacklist.length > 0 && (
<div>
<p className="text-xs font-semibold text-text-muted uppercase tracking-wider mb-2">
Blocked ({config.blacklist.length})
</p>
<div className="flex flex-wrap gap-1.5">
{config.blacklist.map((ip) => (
<span
key={ip}
className="inline-flex items-center gap-1.5 px-2.5 py-1 rounded-md text-xs font-mono
bg-red-500/10 text-red-400 border border-red-500/20"
>
{ip}
<button onClick={() => removeIP(ip, "blacklist")} className="hover:text-red-300">
<span className="material-symbols-outlined text-[14px]">close</span>
</button>
</span>
))}
</div>
</div>
)}
{/* Whitelist */}
{config.whitelist.length > 0 && (
<div>
<p className="text-xs font-semibold text-text-muted uppercase tracking-wider mb-2">
Allowed ({config.whitelist.length})
</p>
<div className="flex flex-wrap gap-1.5">
{config.whitelist.map((ip) => (
<span
key={ip}
className="inline-flex items-center gap-1.5 px-2.5 py-1 rounded-md text-xs font-mono
bg-emerald-500/10 text-emerald-400 border border-emerald-500/20"
>
{ip}
<button onClick={() => removeIP(ip, "whitelist")} className="hover:text-emerald-300">
<span className="material-symbols-outlined text-[14px]">close</span>
</button>
</span>
))}
</div>
</div>
)}
{/* Temp Bans */}
{config.tempBans.length > 0 && (
<div>
<p className="text-xs font-semibold text-text-muted uppercase tracking-wider mb-2">
Temporary Bans ({config.tempBans.length})
</p>
<div className="flex flex-col gap-1.5">
{config.tempBans.map((ban) => (
<div
key={ban.ip}
className="flex items-center justify-between px-3 py-2 rounded-lg
bg-orange-500/5 border border-orange-500/20 text-sm"
>
<div className="flex items-center gap-2">
<span className="font-mono text-xs text-orange-400">{ban.ip}</span>
<span className="text-xs text-text-muted"> {ban.reason}</span>
</div>
<div className="flex items-center gap-2">
<span className="text-xs text-text-muted tabular-nums">
{Math.ceil(ban.remainingMs / 60000)}m left
</span>
<button onClick={() => removeBan(ban.ip)} className="text-text-muted hover:text-orange-400">
<span className="material-symbols-outlined text-[16px]">delete</span>
</button>
</div>
</div>
))}
</div>
</div>
)}
</div>
)}
</Card>
);
}

View File

@@ -0,0 +1,530 @@
"use client";
import { useState, useEffect, useCallback, useMemo } from "react";
import { Card } from "@/shared/components";
const PRICING_FIELDS = ["input", "output", "cached", "reasoning", "cache_creation"];
const FIELD_LABELS = {
input: "Input",
output: "Output",
cached: "Cached",
reasoning: "Reasoning",
cache_creation: "Cache Write",
};
export default function PricingTab() {
const [catalog, setCatalog] = useState({});
const [pricingData, setPricingData] = useState({});
const [loading, setLoading] = useState(true);
const [saving, setSaving] = useState(false);
const [saveStatus, setSaveStatus] = useState("");
const [selectedProvider, setSelectedProvider] = useState(null);
const [expandedProviders, setExpandedProviders] = useState(new Set());
const [searchQuery, setSearchQuery] = useState("");
const [editedProviders, setEditedProviders] = useState(new Set());
// Load catalog + pricing
useEffect(() => {
loadData();
}, []);
const loadData = async () => {
setLoading(true);
try {
const [catalogRes, pricingRes] = await Promise.all([
fetch("/api/pricing/models"),
fetch("/api/pricing"),
]);
if (catalogRes.ok) setCatalog(await catalogRes.json());
if (pricingRes.ok) setPricingData(await pricingRes.json());
} catch (error) {
console.error("Failed to load pricing data:", error);
} finally {
setLoading(false);
}
};
// All providers sorted by model count (desc)
const allProviders = useMemo(() => {
const providers = Object.entries(catalog)
.map(([alias, info]) => ({
alias,
...info,
pricedModels: pricingData[alias]
? Object.keys(pricingData[alias]).length
: 0,
}))
.sort((a, b) => b.modelCount - a.modelCount);
return providers;
}, [catalog, pricingData]);
// Filter providers by search
const filteredProviders = useMemo(() => {
if (!searchQuery.trim()) return allProviders;
const q = searchQuery.toLowerCase();
return allProviders.filter(
(p) =>
p.alias.toLowerCase().includes(q) ||
p.id.toLowerCase().includes(q) ||
p.models.some(
(m) =>
m.id.toLowerCase().includes(q) ||
m.name.toLowerCase().includes(q)
)
);
}, [allProviders, searchQuery]);
// Stats
const stats = useMemo(() => {
const totalModels = allProviders.reduce((s, p) => s + p.modelCount, 0);
const pricedCount = Object.values(pricingData).reduce(
(s, models) => s + Object.keys(models).length,
0
);
return {
providers: allProviders.length,
totalModels,
pricedCount,
};
}, [allProviders, pricingData]);
const toggleProvider = useCallback((alias) => {
setExpandedProviders((prev) => {
const next = new Set(prev);
if (next.has(alias)) next.delete(alias);
else next.add(alias);
return next;
});
}, []);
const handlePricingChange = useCallback(
(provider, model, field, value) => {
const numValue = parseFloat(value);
if (isNaN(numValue) || numValue < 0) return;
setPricingData((prev) => {
const next = { ...prev };
if (!next[provider]) next[provider] = {};
if (!next[provider][model])
next[provider][model] = { input: 0, output: 0, cached: 0, reasoning: 0, cache_creation: 0 };
next[provider][model] = { ...next[provider][model], [field]: numValue };
return next;
});
setEditedProviders((prev) => new Set(prev).add(provider));
},
[]
);
const saveProvider = useCallback(
async (providerAlias) => {
setSaving(true);
setSaveStatus("");
try {
const providerPricing = pricingData[providerAlias] || {};
const response = await fetch("/api/pricing", {
method: "PATCH",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({ [providerAlias]: providerPricing }),
});
if (response.ok) {
setSaveStatus(`${providerAlias.toUpperCase()} saved`);
setEditedProviders((prev) => {
const next = new Set(prev);
next.delete(providerAlias);
return next;
});
setTimeout(() => setSaveStatus(""), 3000);
} else {
const err = await response.json();
setSaveStatus(`❌ Error: ${err.error}`);
}
} catch (error) {
setSaveStatus(`❌ Save failed: ${error.message}`);
} finally {
setSaving(false);
}
},
[pricingData]
);
const resetProvider = useCallback(
async (providerAlias) => {
if (
!confirm(
`Reset all pricing for ${providerAlias.toUpperCase()} to defaults?`
)
)
return;
try {
const response = await fetch(
`/api/pricing?provider=${providerAlias}`,
{ method: "DELETE" }
);
if (response.ok) {
const updated = await response.json();
setPricingData(updated);
setSaveStatus(`🔄 ${providerAlias.toUpperCase()} reset to defaults`);
setEditedProviders((prev) => {
const next = new Set(prev);
next.delete(providerAlias);
return next;
});
setTimeout(() => setSaveStatus(""), 3000);
}
} catch (error) {
setSaveStatus(`❌ Reset failed: ${error.message}`);
}
},
[]
);
const selectProviderFilter = useCallback((alias) => {
setSelectedProvider((prev) => (prev === alias ? null : alias));
}, []);
// Which providers to display in the main area
const displayProviders = useMemo(() => {
if (selectedProvider) {
return filteredProviders.filter((p) => p.alias === selectedProvider);
}
return filteredProviders;
}, [filteredProviders, selectedProvider]);
if (loading) {
return (
<div className="flex items-center justify-center py-16">
<div className="text-text-muted animate-pulse">
Loading pricing data...
</div>
</div>
);
}
return (
<div className="flex flex-col gap-4">
{/* Header + Stats */}
<div className="flex items-start justify-between flex-wrap gap-4">
<div>
<h2 className="text-xl font-bold">Model Pricing</h2>
<p className="text-text-muted text-sm mt-1">
Configure cost rates per model All rates in{" "}
<strong>$/1M tokens</strong>
</p>
</div>
<div className="flex gap-3 text-sm">
<div className="bg-bg-subtle rounded-lg px-3 py-2 text-center">
<div className="text-text-muted text-xs font-semibold">
Providers
</div>
<div className="text-lg font-bold">{stats.providers}</div>
</div>
<div className="bg-bg-subtle rounded-lg px-3 py-2 text-center">
<div className="text-text-muted text-xs font-semibold">
Registry
</div>
<div className="text-lg font-bold">{stats.totalModels}</div>
</div>
<div className="bg-bg-subtle rounded-lg px-3 py-2 text-center">
<div className="text-text-muted text-xs font-semibold">Priced</div>
<div className="text-lg font-bold text-success">
{stats.pricedCount}
</div>
</div>
</div>
</div>
{/* Save Status */}
{saveStatus && (
<div className="px-3 py-2 rounded-lg bg-bg-subtle border border-border text-sm">
{saveStatus}
</div>
)}
{/* Search + Provider Filter */}
<div className="flex gap-3 items-center flex-wrap">
<div className="relative flex-1 min-w-[200px]">
<span className="material-symbols-outlined absolute left-3 top-1/2 -translate-y-1/2 text-text-muted text-lg">
search
</span>
<input
type="text"
placeholder="Search providers or models..."
value={searchQuery}
onChange={(e) => setSearchQuery(e.target.value)}
className="w-full pl-10 pr-3 py-2 bg-bg-base border border-border rounded-lg focus:outline-none focus:border-primary text-sm"
/>
</div>
{selectedProvider && (
<button
onClick={() => setSelectedProvider(null)}
className="px-3 py-2 text-xs bg-primary/10 text-primary border border-primary/20 rounded-lg hover:bg-primary/20 transition-colors flex items-center gap-1"
>
<span className="material-symbols-outlined text-sm">close</span>
{selectedProvider.toUpperCase()} Show All
</button>
)}
</div>
{/* Provider Pills (quick filter) */}
<div className="flex flex-wrap gap-1.5">
{allProviders.map((p) => (
<button
key={p.alias}
onClick={() => selectProviderFilter(p.alias)}
className={`px-2.5 py-1 rounded-md text-xs font-medium transition-all ${
selectedProvider === p.alias
? "bg-primary text-white shadow-sm"
: editedProviders.has(p.alias)
? "bg-yellow-500/15 text-yellow-400 border border-yellow-500/30"
: "bg-bg-subtle text-text-muted hover:bg-bg-hover border border-transparent"
}`}
>
{p.alias.toUpperCase()}{" "}
<span className="opacity-60">({p.modelCount})</span>
</button>
))}
</div>
{/* Provider Sections */}
<div className="flex flex-col gap-2">
{displayProviders.map((provider) => (
<ProviderSection
key={provider.alias}
provider={provider}
pricingData={pricingData[provider.alias] || {}}
isExpanded={expandedProviders.has(provider.alias)}
isEdited={editedProviders.has(provider.alias)}
onToggle={() => toggleProvider(provider.alias)}
onPricingChange={(model, field, value) =>
handlePricingChange(provider.alias, model, field, value)
}
onSave={() => saveProvider(provider.alias)}
onReset={() => resetProvider(provider.alias)}
saving={saving}
/>
))}
{displayProviders.length === 0 && (
<div className="text-center py-12 text-text-muted">
No providers match your search.
</div>
)}
</div>
{/* Info Box */}
<Card className="p-4 mt-2">
<h3 className="text-sm font-semibold mb-2">
<span className="material-symbols-outlined text-sm align-middle mr-1">
info
</span>
How Pricing Works
</h3>
<div className="text-xs text-text-muted space-y-1">
<p>
<strong>Input</strong>: tokens sent to the model {" "}
<strong>Output</strong>: tokens generated {" "}
<strong>Cached</strong>: reused input (~50% of input rate) {" "}
<strong>Reasoning</strong>: thinking tokens (falls back to Output) {" "}
<strong>Cache Write</strong>: creating cache entries (falls back to
Input)
</p>
<p>
Cost = (input × input_rate) + (output × output_rate) + (cached ×
cached_rate) per million tokens.
</p>
</div>
</Card>
</div>
);
}
// ── Provider Section (collapsible) ──────────────────────────────────────
function ProviderSection({
provider,
pricingData,
isExpanded,
isEdited,
onToggle,
onPricingChange,
onSave,
onReset,
saving,
}) {
const pricedCount = Object.keys(pricingData).length;
const authBadge =
provider.authType === "oauth"
? "OAuth"
: provider.authType === "apikey"
? "API Key"
: provider.authType;
return (
<div
className={`border rounded-lg overflow-hidden transition-colors ${
isEdited
? "border-yellow-500/40 bg-yellow-500/5"
: "border-border"
}`}
>
{/* Header (click to expand) */}
<button
onClick={onToggle}
className="w-full flex items-center justify-between px-4 py-3 hover:bg-bg-hover/50 transition-colors text-left"
>
<div className="flex items-center gap-3">
<span
className={`material-symbols-outlined text-lg transition-transform ${
isExpanded ? "rotate-90" : ""
}`}
>
chevron_right
</span>
<div>
<span className="font-semibold text-sm">
{provider.id.charAt(0).toUpperCase() + provider.id.slice(1)}
</span>
<span className="text-text-muted text-xs ml-2">
({provider.alias.toUpperCase()})
</span>
</div>
<span className="px-1.5 py-0.5 bg-bg-subtle text-text-muted text-[10px] rounded uppercase font-semibold">
{authBadge}
</span>
<span className="px-1.5 py-0.5 bg-bg-subtle text-text-muted text-[10px] rounded uppercase font-semibold">
{provider.format}
</span>
</div>
<div className="flex items-center gap-3">
{isEdited && (
<span className="text-yellow-500 text-xs font-medium">
unsaved
</span>
)}
<span className="text-text-muted text-xs">
{pricedCount}/{provider.modelCount} priced
</span>
<div className="w-16 h-1.5 bg-bg-subtle rounded-full overflow-hidden">
<div
className="h-full bg-primary rounded-full transition-all"
style={{
width: `${
provider.modelCount > 0
? Math.round((pricedCount / provider.modelCount) * 100)
: 0
}%`,
}}
/>
</div>
</div>
</button>
{/* Expanded: models table */}
{isExpanded && (
<div className="border-t border-border">
{/* Actions bar */}
<div className="flex items-center justify-between px-4 py-2 bg-bg-subtle/50">
<span className="text-xs text-text-muted">
{provider.modelCount} models {" "}
{pricedCount} with pricing configured
</span>
<div className="flex items-center gap-2">
<button
onClick={(e) => {
e.stopPropagation();
onReset();
}}
className="px-2.5 py-1 text-[11px] text-red-400 hover:bg-red-500/10 rounded border border-red-500/20 transition-colors"
>
Reset Defaults
</button>
<button
onClick={(e) => {
e.stopPropagation();
onSave();
}}
disabled={saving || !isEdited}
className="px-2.5 py-1 text-[11px] bg-primary text-white rounded hover:bg-primary/90 transition-colors disabled:opacity-40"
>
{saving ? "Saving..." : "Save Provider"}
</button>
</div>
</div>
{/* Table */}
<div className="overflow-x-auto">
<table className="w-full text-sm">
<thead className="text-[11px] text-text-muted uppercase bg-bg-subtle/30">
<tr>
<th className="px-4 py-2 text-left font-semibold">Model</th>
{PRICING_FIELDS.map((field) => (
<th
key={field}
className="px-2 py-2 text-right font-semibold w-24"
>
{FIELD_LABELS[field]}
</th>
))}
</tr>
</thead>
<tbody className="divide-y divide-border/50">
{provider.models.map((model) => (
<ModelRow
key={model.id}
model={model}
pricing={pricingData[model.id]}
onPricingChange={(field, value) =>
onPricingChange(model.id, field, value)
}
/>
))}
</tbody>
</table>
</div>
</div>
)}
</div>
);
}
// ── Model Row ────────────────────────────────────────────────────────────
function ModelRow({ model, pricing, onPricingChange }) {
const hasPricing = pricing && Object.values(pricing).some((v) => v > 0);
return (
<tr className="hover:bg-bg-hover/30 group">
<td className="px-4 py-1.5">
<div className="flex items-center gap-2">
<span
className={`w-1.5 h-1.5 rounded-full ${
hasPricing ? "bg-success" : "bg-text-muted/30"
}`}
/>
<span className="font-medium text-xs">{model.name}</span>
{model.custom && (
<span className="px-1 py-0.5 text-[8px] font-bold bg-blue-500/15 text-blue-400 border border-blue-500/20 rounded uppercase">
custom
</span>
)}
<span className="text-text-muted text-[10px] opacity-0 group-hover:opacity-100 transition-opacity">
{model.id}
</span>
</div>
</td>
{PRICING_FIELDS.map((field) => (
<td key={field} className="px-2 py-1.5">
<input
type="number"
step="0.01"
min="0"
value={pricing?.[field] || 0}
onChange={(e) => onPricingChange(field, e.target.value)}
className="w-full px-2 py-1 text-right text-xs bg-transparent border border-transparent hover:border-border focus:border-primary focus:bg-bg-base rounded transition-colors outline-none tabular-nums"
/>
</td>
))}
</tr>
);
}

View File

@@ -1,105 +1,190 @@
"use client";
import { useState, useEffect } from "react";
import { Card, Input, Toggle } from "@/shared/components";
import { Card, Input, Toggle, Button } from "@/shared/components";
const STRATEGIES = [
{ value: "fill-first", label: "Fill First", desc: "Use accounts in priority order", icon: "vertical_align_top" },
{ value: "round-robin", label: "Round Robin", desc: "Cycle through all accounts", icon: "loop" },
{ value: "p2c", label: "P2C", desc: "Pick 2 random, use the healthier one", icon: "balance" },
];
export default function RoutingTab() {
const [settings, setSettings] = useState({ fallbackStrategy: "fill-first" });
const [loading, setLoading] = useState(true);
const [aliases, setAliases] = useState([]);
const [newPattern, setNewPattern] = useState("");
const [newTarget, setNewTarget] = useState("");
useEffect(() => {
fetch("/api/settings")
.then((res) => res.json())
.then((data) => {
setSettings(data);
setAliases(data.wildcardAliases || []);
setLoading(false);
})
.catch(() => setLoading(false));
}, []);
const updateFallbackStrategy = async (strategy) => {
const updateSetting = async (patch) => {
try {
const res = await fetch("/api/settings", {
method: "PATCH",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({ fallbackStrategy: strategy }),
body: JSON.stringify(patch),
});
if (res.ok) {
setSettings((prev) => ({ ...prev, fallbackStrategy: strategy }));
setSettings((prev) => ({ ...prev, ...patch }));
}
} catch (err) {
console.error("Failed to update settings:", err);
}
};
const updateStickyLimit = async (limit) => {
const numLimit = parseInt(limit);
if (isNaN(numLimit) || numLimit < 1) return;
try {
const res = await fetch("/api/settings", {
method: "PATCH",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({ stickyRoundRobinLimit: numLimit }),
});
if (res.ok) {
setSettings((prev) => ({ ...prev, stickyRoundRobinLimit: numLimit }));
}
} catch (err) {
console.error("Failed to update sticky limit:", err);
}
const addAlias = async () => {
if (!newPattern.trim() || !newTarget.trim()) return;
const updated = [...aliases, { pattern: newPattern.trim(), target: newTarget.trim() }];
await updateSetting({ wildcardAliases: updated });
setAliases(updated);
setNewPattern("");
setNewTarget("");
};
const removeAlias = async (idx) => {
const updated = aliases.filter((_, i) => i !== idx);
await updateSetting({ wildcardAliases: updated });
setAliases(updated);
};
return (
<Card>
<div className="flex items-center gap-3 mb-4">
<div className="p-2 rounded-lg bg-blue-500/10 text-blue-500">
<span className="material-symbols-outlined text-[20px]" aria-hidden="true">
route
</span>
</div>
<h3 className="text-lg font-semibold">Routing Strategy</h3>
</div>
<div className="flex flex-col gap-4">
<div className="flex items-center justify-between">
<div>
<p className="font-medium">Round Robin</p>
<p className="text-sm text-text-muted">Cycle through accounts to distribute load</p>
<div className="flex flex-col gap-6">
{/* Strategy Selection */}
<Card>
<div className="flex items-center gap-3 mb-4">
<div className="p-2 rounded-lg bg-blue-500/10 text-blue-500">
<span className="material-symbols-outlined text-[20px]" aria-hidden="true">
route
</span>
</div>
<Toggle
checked={settings.fallbackStrategy === "round-robin"}
onChange={() =>
updateFallbackStrategy(
settings.fallbackStrategy === "round-robin" ? "fill-first" : "round-robin"
)
}
disabled={loading}
/>
<h3 className="text-lg font-semibold">Routing Strategy</h3>
</div>
<div className="grid grid-cols-3 gap-2 mb-4">
{STRATEGIES.map((s) => (
<button
key={s.value}
onClick={() => updateSetting({ fallbackStrategy: s.value })}
disabled={loading}
className={`flex flex-col items-center gap-2 p-4 rounded-lg border text-center transition-all ${
settings.fallbackStrategy === s.value
? "border-blue-500/50 bg-blue-500/5 ring-1 ring-blue-500/20"
: "border-border/50 hover:border-border hover:bg-surface/30"
}`}
>
<span
className={`material-symbols-outlined text-[24px] ${
settings.fallbackStrategy === s.value ? "text-blue-400" : "text-text-muted"
}`}
>
{s.icon}
</span>
<div>
<p className={`text-sm font-medium ${settings.fallbackStrategy === s.value ? "text-blue-400" : ""}`}>
{s.label}
</p>
<p className="text-xs text-text-muted mt-0.5">{s.desc}</p>
</div>
</button>
))}
</div>
{settings.fallbackStrategy === "round-robin" && (
<div className="flex items-center justify-between pt-2 border-t border-border/50">
<div className="flex items-center justify-between pt-3 border-t border-border/30">
<div>
<p className="font-medium">Sticky Limit</p>
<p className="text-sm text-text-muted">Calls per account before switching</p>
<p className="text-sm font-medium">Sticky Limit</p>
<p className="text-xs text-text-muted">Calls per account before switching</p>
</div>
<Input
type="number"
min="1"
max="10"
value={settings.stickyRoundRobinLimit || 3}
onChange={(e) => updateStickyLimit(e.target.value)}
onChange={(e) => updateSetting({ stickyRoundRobinLimit: parseInt(e.target.value) })}
disabled={loading}
className="w-20 text-center"
/>
</div>
)}
<p className="text-xs text-text-muted italic pt-2 border-t border-border/50">
{settings.fallbackStrategy === "round-robin"
? `Currently distributing requests across all available accounts with ${settings.stickyRoundRobinLimit || 3} calls per account.`
: "Currently using accounts in priority order (Fill First)."}
<p className="text-xs text-text-muted italic pt-3 border-t border-border/30 mt-3">
{settings.fallbackStrategy === "round-robin" &&
`Distributing requests across accounts with ${settings.stickyRoundRobinLimit || 3} calls per account.`}
{settings.fallbackStrategy === "fill-first" && "Using accounts in priority order (Fill First)."}
{settings.fallbackStrategy === "p2c" &&
"Power of Two Choices: picks 2 random accounts and routes to the healthier one."}
</p>
</div>
</Card>
</Card>
{/* Wildcard Aliases */}
<Card>
<div className="flex items-center gap-3 mb-4">
<div className="p-2 rounded-lg bg-purple-500/10 text-purple-500">
<span className="material-symbols-outlined text-[20px]" aria-hidden="true">
alt_route
</span>
</div>
<div>
<h3 className="text-lg font-semibold">Model Aliases</h3>
<p className="text-sm text-text-muted">Wildcard patterns to remap model names Use * and ?</p>
</div>
</div>
{aliases.length > 0 && (
<div className="flex flex-col gap-1.5 mb-4">
{aliases.map((a, i) => (
<div
key={i}
className="flex items-center justify-between px-3 py-2 rounded-lg bg-surface/30 border border-border/20"
>
<div className="flex items-center gap-3 text-sm">
<span className="font-mono text-purple-400">{a.pattern}</span>
<span className="material-symbols-outlined text-[14px] text-text-muted">arrow_forward</span>
<span className="font-mono text-text-main">{a.target}</span>
</div>
<button
onClick={() => removeAlias(i)}
className="text-text-muted hover:text-red-400 transition-colors"
>
<span className="material-symbols-outlined text-[16px]">close</span>
</button>
</div>
))}
</div>
)}
<div className="flex gap-2 items-end">
<div className="flex-1">
<Input
label="Pattern"
placeholder="claude-sonnet-*"
value={newPattern}
onChange={(e) => setNewPattern(e.target.value)}
/>
</div>
<div className="flex-1">
<Input
label="Target Model"
placeholder="claude-sonnet-4-20250514"
value={newTarget}
onChange={(e) => setNewTarget(e.target.value)}
/>
</div>
<Button size="sm" variant="primary" onClick={addAlias} className="mb-[2px]">
+ Add
</Button>
</div>
</Card>
</div>
);
}

View File

@@ -2,6 +2,7 @@
import { useState, useEffect } from "react";
import { Card, Button, Input, Toggle } from "@/shared/components";
import IPFilterSection from "./IPFilterSection";
export default function SecurityTab() {
const [settings, setSettings] = useState({ requireLogin: false, hasPassword: false });
@@ -69,79 +70,82 @@ export default function SecurityTab() {
};
return (
<Card>
<div className="flex items-center gap-3 mb-4">
<div className="p-2 rounded-lg bg-primary/10 text-primary">
<span className="material-symbols-outlined text-[20px]" aria-hidden="true">
shield
</span>
</div>
<h3 className="text-lg font-semibold">Security</h3>
</div>
<div className="flex flex-col gap-4">
<div className="flex items-center justify-between">
<div>
<p className="font-medium">Require login</p>
<p className="text-sm text-text-muted">
When ON, dashboard requires password. When OFF, access without login.
</p>
<div className="flex flex-col gap-6">
<Card>
<div className="flex items-center gap-3 mb-4">
<div className="p-2 rounded-lg bg-primary/10 text-primary">
<span className="material-symbols-outlined text-[20px]" aria-hidden="true">
shield
</span>
</div>
<Toggle
checked={settings.requireLogin === true}
onChange={() => updateRequireLogin(!settings.requireLogin)}
disabled={loading}
/>
<h3 className="text-lg font-semibold">Security</h3>
</div>
{settings.requireLogin === true && (
<form
onSubmit={handlePasswordChange}
className="flex flex-col gap-4 pt-4 border-t border-border/50"
>
{settings.hasPassword && (
<Input
label="Current Password"
type="password"
placeholder="Enter current password"
value={passwords.current}
onChange={(e) => setPasswords({ ...passwords, current: e.target.value })}
required
/>
)}
<div className="grid grid-cols-1 md:grid-cols-2 gap-4">
<Input
label="New Password"
type="password"
placeholder="Enter new password"
value={passwords.new}
onChange={(e) => setPasswords({ ...passwords, new: e.target.value })}
required
/>
<Input
label="Confirm New Password"
type="password"
placeholder="Confirm new password"
value={passwords.confirm}
onChange={(e) => setPasswords({ ...passwords, confirm: e.target.value })}
required
/>
</div>
{passStatus.message && (
<p
className={`text-sm ${passStatus.type === "error" ? "text-red-500" : "text-green-500"}`}
>
{passStatus.message}
<div className="flex flex-col gap-4">
<div className="flex items-center justify-between">
<div>
<p className="font-medium">Require login</p>
<p className="text-sm text-text-muted">
When ON, dashboard requires password. When OFF, access without login.
</p>
)}
<div className="pt-2">
<Button type="submit" variant="primary" loading={passLoading}>
{settings.hasPassword ? "Update Password" : "Set Password"}
</Button>
</div>
</form>
)}
</div>
</Card>
<Toggle
checked={settings.requireLogin === true}
onChange={() => updateRequireLogin(!settings.requireLogin)}
disabled={loading}
/>
</div>
{settings.requireLogin === true && (
<form
onSubmit={handlePasswordChange}
className="flex flex-col gap-4 pt-4 border-t border-border/50"
>
{settings.hasPassword && (
<Input
label="Current Password"
type="password"
placeholder="Enter current password"
value={passwords.current}
onChange={(e) => setPasswords({ ...passwords, current: e.target.value })}
required
/>
)}
<div className="grid grid-cols-1 md:grid-cols-2 gap-4">
<Input
label="New Password"
type="password"
placeholder="Enter new password"
value={passwords.new}
onChange={(e) => setPasswords({ ...passwords, new: e.target.value })}
required
/>
<Input
label="Confirm New Password"
type="password"
placeholder="Confirm new password"
value={passwords.confirm}
onChange={(e) => setPasswords({ ...passwords, confirm: e.target.value })}
required
/>
</div>
{passStatus.message && (
<p
className={`text-sm ${passStatus.type === "error" ? "text-red-500" : "text-green-500"}`}
>
{passStatus.message}
</p>
)}
<div className="pt-2">
<Button type="submit" variant="primary" loading={passLoading}>
{settings.hasPassword ? "Update Password" : "Set Password"}
</Button>
</div>
</form>
)}
</div>
</Card>
<IPFilterSection />
</div>
);
}

View File

@@ -0,0 +1,104 @@
"use client";
import { useState, useEffect } from "react";
import { Card, Toggle } from "@/shared/components";
export default function SystemPromptTab() {
const [config, setConfig] = useState({ enabled: false, prompt: "" });
const [loading, setLoading] = useState(true);
const [status, setStatus] = useState("");
const [debounceTimer, setDebounceTimer] = useState(null);
useEffect(() => {
fetch("/api/settings/system-prompt")
.then((res) => res.json())
.then((data) => {
setConfig(data);
setLoading(false);
})
.catch(() => setLoading(false));
}, []);
const save = async (updates) => {
const newConfig = { ...config, ...updates };
setConfig(newConfig);
setStatus("");
try {
const res = await fetch("/api/settings/system-prompt", {
method: "PUT",
headers: { "Content-Type": "application/json" },
body: JSON.stringify(newConfig),
});
if (res.ok) {
setStatus("saved");
setTimeout(() => setStatus(""), 2000);
}
} catch {
setStatus("error");
}
};
const handlePromptChange = (text) => {
setConfig((prev) => ({ ...prev, prompt: text }));
if (debounceTimer) clearTimeout(debounceTimer);
setDebounceTimer(
setTimeout(() => {
save({ prompt: text });
}, 800)
);
};
return (
<Card>
<div className="flex items-center gap-3 mb-5">
<div className="p-2 rounded-lg bg-amber-500/10 text-amber-500">
<span className="material-symbols-outlined text-[20px]" aria-hidden="true">
edit_note
</span>
</div>
<div className="flex-1">
<h3 className="text-lg font-semibold">Global System Prompt</h3>
<p className="text-sm text-text-muted">Injected into all requests at proxy level</p>
</div>
<div className="flex items-center gap-3">
{status === "saved" && (
<span className="text-xs font-medium text-emerald-500 flex items-center gap-1">
<span className="material-symbols-outlined text-[14px]">check_circle</span> Saved
</span>
)}
<Toggle
checked={config.enabled}
onChange={() => save({ enabled: !config.enabled })}
disabled={loading}
/>
</div>
</div>
{config.enabled && (
<div className="flex flex-col gap-3">
<div className="relative">
<textarea
value={config.prompt}
onChange={(e) => handlePromptChange(e.target.value)}
placeholder="Enter system prompt to inject into all requests..."
rows={5}
className="w-full px-4 py-3 rounded-lg border border-border/50 bg-surface/30 text-sm
placeholder:text-text-muted/50 resize-y min-h-[120px]
focus:outline-none focus:ring-1 focus:ring-amber-500/30 focus:border-amber-500/50
transition-colors"
disabled={loading}
/>
<div className="absolute bottom-2 right-3 text-xs text-text-muted/60 tabular-nums">
{config.prompt.length} chars
</div>
</div>
<p className="text-xs text-text-muted/70 flex items-center gap-1.5">
<span className="material-symbols-outlined text-[14px]">info</span>
This prompt is prepended to the system message of every request. Use for global instructions,
safety guidelines, or response formatting rules. Send <code className="px-1 py-0.5 rounded bg-surface/50">_skipSystemPrompt: true</code> in a request to bypass.
</p>
</div>
)}
</Card>
);
}

View File

@@ -0,0 +1,188 @@
"use client";
import { useState, useEffect } from "react";
import { Card, Button, Select } from "@/shared/components";
const MODES = [
{
value: "passthrough",
label: "Passthrough",
desc: "No changes — client controls thinking budget",
icon: "arrow_forward",
},
{
value: "auto",
label: "Auto",
desc: "Strip all thinking config — let provider decide",
icon: "auto_awesome",
},
{
value: "custom",
label: "Custom",
desc: "Set a fixed token budget for all requests",
icon: "tune",
},
{
value: "adaptive",
label: "Adaptive",
desc: "Scale budget based on request complexity",
icon: "trending_up",
},
];
const EFFORTS = [
{ value: "none", label: "None (0 tokens)" },
{ value: "low", label: "Low (1K tokens)" },
{ value: "medium", label: "Medium (10K tokens)" },
{ value: "high", label: "High (128K tokens)" },
];
export default function ThinkingBudgetTab() {
const [config, setConfig] = useState({
mode: "passthrough",
customBudget: 10240,
effortLevel: "medium",
});
const [loading, setLoading] = useState(true);
const [saving, setSaving] = useState(false);
const [status, setStatus] = useState("");
useEffect(() => {
fetch("/api/settings/thinking-budget")
.then((res) => res.json())
.then((data) => {
setConfig(data);
setLoading(false);
})
.catch(() => setLoading(false));
}, []);
const save = async (updates) => {
const newConfig = { ...config, ...updates };
setConfig(newConfig);
setSaving(true);
setStatus("");
try {
const res = await fetch("/api/settings/thinking-budget", {
method: "PUT",
headers: { "Content-Type": "application/json" },
body: JSON.stringify(newConfig),
});
if (res.ok) {
setStatus("saved");
setTimeout(() => setStatus(""), 2000);
} else {
setStatus("error");
}
} catch {
setStatus("error");
} finally {
setSaving(false);
}
};
return (
<Card>
<div className="flex items-center gap-3 mb-5">
<div className="p-2 rounded-lg bg-violet-500/10 text-violet-500">
<span className="material-symbols-outlined text-[20px]" aria-hidden="true">
psychology
</span>
</div>
<div>
<h3 className="text-lg font-semibold">Thinking Budget</h3>
<p className="text-sm text-text-muted">Control AI reasoning token usage across all requests</p>
</div>
{status === "saved" && (
<span className="ml-auto text-xs font-medium text-emerald-500 flex items-center gap-1">
<span className="material-symbols-outlined text-[14px]">check_circle</span> Saved
</span>
)}
</div>
{/* Mode selector */}
<div className="grid grid-cols-2 gap-2 mb-5">
{MODES.map((m) => (
<button
key={m.value}
onClick={() => save({ mode: m.value })}
disabled={loading || saving}
className={`flex items-start gap-3 p-3 rounded-lg border text-left transition-all ${
config.mode === m.value
? "border-violet-500/50 bg-violet-500/5 ring-1 ring-violet-500/20"
: "border-border/50 hover:border-border hover:bg-surface/30"
}`}
>
<span
className={`material-symbols-outlined text-[20px] mt-0.5 ${
config.mode === m.value ? "text-violet-500" : "text-text-muted"
}`}
>
{m.icon}
</span>
<div className="min-w-0">
<p className={`text-sm font-medium ${config.mode === m.value ? "text-violet-400" : ""}`}>
{m.label}
</p>
<p className="text-xs text-text-muted mt-0.5 leading-relaxed">{m.desc}</p>
</div>
</button>
))}
</div>
{/* Custom budget slider */}
{config.mode === "custom" && (
<div className="p-4 rounded-lg bg-surface/30 border border-border/30 mb-4">
<div className="flex items-center justify-between mb-3">
<p className="text-sm font-medium">Token Budget</p>
<span className="text-sm font-mono tabular-nums text-violet-400">
{config.customBudget.toLocaleString()} tokens
</span>
</div>
<input
type="range"
min="0"
max="131072"
step="1024"
value={config.customBudget}
onChange={(e) => save({ customBudget: parseInt(e.target.value) })}
className="w-full accent-violet-500"
/>
<div className="flex justify-between text-xs text-text-muted mt-1">
<span>Off</span>
<span>1K</span>
<span>10K</span>
<span>64K</span>
<span>128K</span>
</div>
</div>
)}
{/* Adaptive effort level */}
{config.mode === "adaptive" && (
<div className="p-4 rounded-lg bg-surface/30 border border-border/30">
<p className="text-sm font-medium mb-3">Base Effort Level</p>
<p className="text-xs text-text-muted mb-3">
Adaptive mode scales from this base level based on message count, tool usage, and prompt length.
</p>
<div className="grid grid-cols-4 gap-2">
{EFFORTS.map((e) => (
<button
key={e.value}
onClick={() => save({ effortLevel: e.value })}
disabled={loading || saving}
className={`px-3 py-2 rounded-lg text-xs font-medium border transition-all ${
config.effortLevel === e.value
? "border-violet-500/50 bg-violet-500/10 text-violet-400"
: "border-border/50 text-text-muted hover:border-border"
}`}
>
{e.value.charAt(0).toUpperCase() + e.value.slice(1)}
</button>
))}
</div>
</div>
)}
</Card>
);
}

View File

@@ -9,11 +9,16 @@ import RoutingTab from "./components/RoutingTab";
import ComboDefaultsTab from "./components/ComboDefaultsTab";
import ProxyTab from "./components/ProxyTab";
import AppearanceTab from "./components/AppearanceTab";
import ThinkingBudgetTab from "./components/ThinkingBudgetTab";
import SystemPromptTab from "./components/SystemPromptTab";
import PricingTab from "./components/PricingTab";
const tabs = [
{ id: "general", label: "General", icon: "settings" },
{ id: "ai", label: "AI", icon: "smart_toy" },
{ id: "security", label: "Security", icon: "shield" },
{ id: "routing", label: "Routing", icon: "route" },
{ id: "pricing", label: "Pricing", icon: "payments" },
{ id: "advanced", label: "Advanced", icon: "tune" },
];
@@ -62,6 +67,13 @@ export default function SettingsPage() {
</>
)}
{activeTab === "ai" && (
<div className="flex flex-col gap-6">
<ThinkingBudgetTab />
<SystemPromptTab />
</div>
)}
{activeTab === "security" && <SecurityTab />}
{activeTab === "routing" && (
@@ -71,6 +83,8 @@ export default function SettingsPage() {
</div>
)}
{activeTab === "pricing" && <PricingTab />}
{activeTab === "advanced" && <ProxyTab />}
</div>

View File

@@ -0,0 +1,114 @@
"use client";
import { useState, useEffect, useCallback } from "react";
import { Card } from "@/shared/components";
export default function RateLimitStatus() {
const [data, setData] = useState({ lockouts: [], cacheStats: null });
const [loading, setLoading] = useState(true);
const load = useCallback(async () => {
try {
const res = await fetch("/api/rate-limits");
if (res.ok) setData(await res.json());
} catch {} finally {
setLoading(false);
}
}, []);
useEffect(() => {
load();
const interval = setInterval(load, 10000);
return () => clearInterval(interval);
}, [load]);
const formatMs = (ms) => {
if (ms < 1000) return `${ms}ms`;
if (ms < 60000) return `${Math.ceil(ms / 1000)}s`;
return `${Math.ceil(ms / 60000)}m`;
};
return (
<div className="flex flex-col gap-4">
{/* Model Lockouts */}
<Card>
<div className="flex items-center gap-3 mb-4">
<div className="p-2 rounded-lg bg-orange-500/10 text-orange-500">
<span className="material-symbols-outlined text-[20px]" aria-hidden="true">
lock_clock
</span>
</div>
<div className="flex-1">
<h3 className="text-lg font-semibold">Model Lockouts</h3>
<p className="text-sm text-text-muted">Per-model rate limit locks Auto-refresh 10s</p>
</div>
{data.lockouts.length > 0 && (
<span className="px-2.5 py-1 rounded-full text-xs font-semibold bg-orange-500/10 text-orange-400 border border-orange-500/20">
{data.lockouts.length} locked
</span>
)}
</div>
{data.lockouts.length === 0 ? (
<div className="text-center py-6 text-text-muted">
<span className="material-symbols-outlined text-[32px] mb-2 block opacity-40">
lock_open
</span>
<p className="text-sm">No models currently locked</p>
</div>
) : (
<div className="flex flex-col gap-2">
{data.lockouts.map((lock, i) => (
<div
key={i}
className="flex items-center justify-between px-3 py-2.5 rounded-lg
bg-orange-500/5 border border-orange-500/15"
>
<div className="flex items-center gap-3">
<span className="material-symbols-outlined text-[16px] text-orange-400">lock</span>
<div>
<p className="text-sm font-medium">{lock.model}</p>
<p className="text-xs text-text-muted">
Account: <span className="font-mono">{lock.accountId?.slice(0, 12) || "N/A"}</span>
{lock.reason && <> {lock.reason}</>}
</p>
</div>
</div>
<span className="text-xs font-mono tabular-nums text-orange-400">
{formatMs(lock.remainingMs)} left
</span>
</div>
))}
</div>
)}
</Card>
{/* Signature Cache Stats */}
{data.cacheStats && (
<Card>
<div className="flex items-center gap-3 mb-4">
<div className="p-2 rounded-lg bg-indigo-500/10 text-indigo-500">
<span className="material-symbols-outlined text-[20px]" aria-hidden="true">
database
</span>
</div>
<h3 className="text-lg font-semibold">Signature Cache</h3>
</div>
<div className="grid grid-cols-4 gap-3">
{[
{ label: "Defaults", value: data.cacheStats.defaultCount, color: "text-text-muted" },
{ label: "Tool", value: `${data.cacheStats.tool.entries}/${data.cacheStats.tool.patterns}`, color: "text-blue-400" },
{ label: "Family", value: `${data.cacheStats.family.entries}/${data.cacheStats.family.patterns}`, color: "text-purple-400" },
{ label: "Session", value: `${data.cacheStats.session.entries}/${data.cacheStats.session.patterns}`, color: "text-cyan-400" },
].map(({ label, value, color }) => (
<div key={label} className="text-center p-3 rounded-lg bg-surface/30 border border-border/30">
<p className={`text-lg font-bold tabular-nums ${color}`}>{value}</p>
<p className="text-xs text-text-muted mt-0.5">{label}</p>
</div>
))}
</div>
</Card>
)}
</div>
);
}

View File

@@ -0,0 +1,99 @@
"use client";
import { useState, useEffect, useCallback } from "react";
import { Card } from "@/shared/components";
export default function SessionsTab() {
const [data, setData] = useState({ count: 0, sessions: [] });
const [loading, setLoading] = useState(true);
const loadSessions = useCallback(async () => {
try {
const res = await fetch("/api/sessions");
if (res.ok) setData(await res.json());
} catch {} finally {
setLoading(false);
}
}, []);
useEffect(() => {
loadSessions();
const interval = setInterval(loadSessions, 5000);
return () => clearInterval(interval);
}, [loadSessions]);
const formatAge = (ms) => {
if (ms < 60000) return `${Math.floor(ms / 1000)}s`;
if (ms < 3600000) return `${Math.floor(ms / 60000)}m`;
return `${Math.floor(ms / 3600000)}h`;
};
return (
<Card>
<div className="flex items-center gap-3 mb-5">
<div className="p-2 rounded-lg bg-cyan-500/10 text-cyan-500">
<span className="material-symbols-outlined text-[20px]" aria-hidden="true">
fingerprint
</span>
</div>
<div className="flex-1">
<h3 className="text-lg font-semibold">Active Sessions</h3>
<p className="text-sm text-text-muted">Tracked via request fingerprinting Auto-refresh 5s</p>
</div>
<div className="flex items-center gap-2">
<span className="flex items-center gap-1.5 px-3 py-1.5 rounded-full bg-cyan-500/10 border border-cyan-500/20">
<span className="w-2 h-2 rounded-full bg-cyan-500 animate-pulse" />
<span className="text-sm font-semibold tabular-nums text-cyan-400">
{data.count}
</span>
</span>
</div>
</div>
{data.sessions.length === 0 ? (
<div className="text-center py-8 text-text-muted">
<span className="material-symbols-outlined text-[40px] mb-2 block opacity-40">
fingerprint
</span>
<p className="text-sm">No active sessions</p>
<p className="text-xs mt-1">Sessions appear as requests flow through the proxy</p>
</div>
) : (
<div className="overflow-x-auto">
<table className="w-full text-sm">
<thead>
<tr className="border-b border-border/30">
<th className="text-left py-2 px-3 text-xs font-semibold text-text-muted uppercase tracking-wider">Session</th>
<th className="text-left py-2 px-3 text-xs font-semibold text-text-muted uppercase tracking-wider">Age</th>
<th className="text-right py-2 px-3 text-xs font-semibold text-text-muted uppercase tracking-wider">Requests</th>
<th className="text-left py-2 px-3 text-xs font-semibold text-text-muted uppercase tracking-wider">Connection</th>
</tr>
</thead>
<tbody>
{data.sessions.map((s) => (
<tr key={s.sessionId} className="border-b border-border/10 hover:bg-surface/20 transition-colors">
<td className="py-2.5 px-3">
<span className="font-mono text-xs px-2 py-1 rounded bg-surface/40 text-text-muted">
{s.sessionId.slice(0, 12)}
</span>
</td>
<td className="py-2.5 px-3 text-text-muted tabular-nums">{formatAge(s.ageMs)}</td>
<td className="py-2.5 px-3 text-right">
<span className="font-semibold tabular-nums">{s.requestCount}</span>
</td>
<td className="py-2.5 px-3">
{s.connectionId ? (
<span className="text-xs font-mono text-cyan-400">{s.connectionId.slice(0, 10)}</span>
) : (
<span className="text-text-muted/40"></span>
)}
</td>
</tr>
))}
</tbody>
</table>
</div>
)}
</Card>
);
}

View File

@@ -9,6 +9,8 @@ import {
SegmentedControl,
} from "@/shared/components";
import ProviderLimits from "./components/ProviderLimits";
import SessionsTab from "./components/SessionsTab";
import RateLimitStatus from "./components/RateLimitStatus";
export default function UsagePage() {
const [activeTab, setActiveTab] = useState("overview");
@@ -21,6 +23,7 @@ export default function UsagePage() {
{ value: "logs", label: "Logger" },
{ value: "proxy-logs", label: "Proxy" },
{ value: "limits", label: "Limits" },
{ value: "sessions", label: "Sessions" },
]}
value={activeTab}
onChange={setActiveTab}
@@ -35,10 +38,14 @@ export default function UsagePage() {
{activeTab === "logs" && <RequestLoggerV2 />}
{activeTab === "proxy-logs" && <ProxyLogger />}
{activeTab === "limits" && (
<Suspense fallback={<CardSkeleton />}>
<ProviderLimits />
</Suspense>
<div className="flex flex-col gap-6">
<Suspense fallback={<CardSkeleton />}>
<ProviderLimits />
</Suspense>
<RateLimitStatus />
</div>
)}
{activeTab === "sessions" && <SessionsTab />}
</div>
);
}

View File

@@ -0,0 +1,120 @@
import { NextResponse } from "next/server";
import { REGISTRY } from "@omniroute/open-sse/config/providerRegistry.js";
import { getAllCustomModels, getPricing } from "@/lib/localDb.js";
/**
* GET /api/pricing/models
* Returns the full model catalog merged from three sources:
* 1. providerRegistry (hardcoded)
* 2. customModels (DB — user-added or imported via /models)
* 3. pricing data (DB — models with pricing configured but not in sources 1/2)
*/
export async function GET() {
try {
const catalog = {};
// ── 1. Registry models (hardcoded) ──────────────────────────────
for (const entry of Object.values(REGISTRY)) {
const alias = entry.alias || entry.id;
if (!entry.models || entry.models.length === 0) continue;
catalog[alias] = {
id: entry.id,
alias,
name: entry.id.charAt(0).toUpperCase() + entry.id.slice(1),
authType: entry.authType || "unknown",
format: entry.format || "openai",
models: entry.models.map((m) => ({
id: m.id,
name: m.name || m.id,
custom: false,
})),
};
}
// ── 2. Custom models (DB) ───────────────────────────────────────
let customModelsMap = {};
try {
customModelsMap = await getAllCustomModels();
} catch {
/* DB may not be ready */
}
for (const [providerId, models] of Object.entries(customModelsMap)) {
// Resolve alias — check if a registry entry maps this providerId
let alias = providerId;
for (const entry of Object.values(REGISTRY)) {
if (entry.id === providerId) {
alias = entry.alias || entry.id;
break;
}
}
if (!catalog[alias]) {
catalog[alias] = {
id: providerId,
alias,
name: providerId.charAt(0).toUpperCase() + providerId.slice(1),
authType: "unknown",
format: "openai",
models: [],
};
}
const existingIds = new Set(catalog[alias].models.map((m) => m.id));
for (const model of models) {
if (!existingIds.has(model.id)) {
catalog[alias].models.push({
id: model.id,
name: model.name || model.id,
custom: true,
});
existingIds.add(model.id);
}
}
}
// ── 3. Pricing-only models (DB) ─────────────────────────────────
let pricingData = {};
try {
pricingData = await getPricing();
} catch {
/* DB may not be ready */
}
for (const [providerAlias, models] of Object.entries(pricingData)) {
if (!catalog[providerAlias]) {
catalog[providerAlias] = {
id: providerAlias,
alias: providerAlias,
name: providerAlias.charAt(0).toUpperCase() + providerAlias.slice(1),
authType: "unknown",
format: "openai",
models: [],
};
}
const existingIds = new Set(catalog[providerAlias].models.map((m) => m.id));
for (const modelId of Object.keys(models)) {
if (!existingIds.has(modelId)) {
catalog[providerAlias].models.push({
id: modelId,
name: modelId,
custom: true,
});
existingIds.add(modelId);
}
}
}
// Add modelCount to each entry
for (const entry of Object.values(catalog)) {
entry.modelCount = entry.models.length;
}
return NextResponse.json(catalog);
} catch (error) {
console.error("Error fetching model catalog:", error);
return NextResponse.json({ error: "Failed to fetch model catalog" }, { status: 500 });
}
}

View File

@@ -0,0 +1,15 @@
import { NextResponse } from "next/server";
import {
getAllModelLockouts,
} from "@omniroute/open-sse/services/accountFallback.js";
import { getCacheStats } from "@omniroute/open-sse/services/signatureCache.js";
export async function GET() {
try {
const lockouts = getAllModelLockouts();
const cacheStats = getCacheStats();
return NextResponse.json({ lockouts, cacheStats });
} catch (error) {
return NextResponse.json({ error: error.message }, { status: 500 });
}
}

View File

@@ -0,0 +1,15 @@
import { NextResponse } from "next/server";
import {
getActiveSessions,
getActiveSessionCount,
} from "@omniroute/open-sse/services/sessionManager.js";
export async function GET() {
try {
const sessions = getActiveSessions();
const count = getActiveSessionCount();
return NextResponse.json({ count, sessions });
} catch (error) {
return NextResponse.json({ error: error.message }, { status: 500 });
}
}

View File

@@ -0,0 +1,46 @@
import { NextResponse } from "next/server";
import {
configureIPFilter,
getIPFilterConfig,
addToBlacklist,
removeFromBlacklist,
addToWhitelist,
removeFromWhitelist,
tempBanIP,
removeTempBan,
} from "@omniroute/open-sse/services/ipFilter.js";
export async function GET() {
try {
return NextResponse.json(getIPFilterConfig());
} catch (error) {
return NextResponse.json({ error: error.message }, { status: 500 });
}
}
export async function PUT(request) {
try {
const body = await request.json();
// Configure entire filter
if (body.enabled !== undefined || body.mode || body.blacklist || body.whitelist) {
configureIPFilter(body);
}
// Add/remove individual IPs
if (body.addBlacklist) addToBlacklist(body.addBlacklist);
if (body.removeBlacklist) removeFromBlacklist(body.removeBlacklist);
if (body.addWhitelist) addToWhitelist(body.addWhitelist);
if (body.removeWhitelist) removeFromWhitelist(body.removeWhitelist);
// Temp bans
if (body.tempBan) {
tempBanIP(body.tempBan.ip, body.tempBan.durationMs || 3600000, body.tempBan.reason || "Manual ban");
}
if (body.removeBan) removeTempBan(body.removeBan);
return NextResponse.json(getIPFilterConfig());
} catch (error) {
return NextResponse.json({ error: error.message }, { status: 500 });
}
}

View File

@@ -0,0 +1,31 @@
import { NextResponse } from "next/server";
import {
setSystemPromptConfig,
getSystemPromptConfig,
} from "@omniroute/open-sse/services/systemPrompt.js";
import { updateSettings } from "@/lib/localDb";
export async function GET() {
try {
return NextResponse.json(getSystemPromptConfig());
} catch (error) {
return NextResponse.json({ error: error.message }, { status: 500 });
}
}
export async function PUT(request) {
try {
const body = await request.json();
if (body.prompt !== undefined && typeof body.prompt !== "string") {
return NextResponse.json({ error: "prompt must be a string" }, { status: 400 });
}
setSystemPromptConfig(body);
await updateSettings({ systemPrompt: body });
return NextResponse.json(getSystemPromptConfig());
} catch (error) {
return NextResponse.json({ error: error.message }, { status: 500 });
}
}

View File

@@ -0,0 +1,62 @@
import { NextResponse } from "next/server";
import { getSettings, updateSettings } from "@/lib/localDb";
import {
setThinkingBudgetConfig,
getThinkingBudgetConfig,
ThinkingMode,
} from "@omniroute/open-sse/services/thinkingBudget.js";
export async function GET() {
try {
const config = getThinkingBudgetConfig();
return NextResponse.json(config);
} catch (error) {
return NextResponse.json({ error: error.message }, { status: 500 });
}
}
export async function PUT(request) {
try {
const body = await request.json();
// Validate mode
const validModes = Object.values(ThinkingMode);
if (body.mode && !validModes.includes(body.mode)) {
return NextResponse.json(
{ error: `Invalid mode. Must be one of: ${validModes.join(", ")}` },
{ status: 400 }
);
}
// Validate customBudget
if (body.customBudget !== undefined) {
const budget = parseInt(body.customBudget, 10);
if (isNaN(budget) || budget < 0 || budget > 131072) {
return NextResponse.json(
{ error: "customBudget must be between 0 and 131072" },
{ status: 400 }
);
}
body.customBudget = budget;
}
// Validate effortLevel
const validEfforts = ["none", "low", "medium", "high"];
if (body.effortLevel && !validEfforts.includes(body.effortLevel)) {
return NextResponse.json(
{ error: `Invalid effortLevel. Must be one of: ${validEfforts.join(", ")}` },
{ status: 400 }
);
}
// Apply config in-memory
setThinkingBudgetConfig(body);
// Persist to settings DB
await updateSettings({ thinkingBudget: body });
return NextResponse.json(getThinkingBudgetConfig());
} catch (error) {
return NextResponse.json({ error: error.message }, { status: 500 });
}
}

View File

@@ -14,6 +14,7 @@ import {
MostActiveDay7d,
WeeklySquares7d,
ModelTable,
ProviderCostDonut,
} from "./analytics";
// ============================================================================
@@ -83,8 +84,8 @@ export default function UsageAnalytics() {
</div>
</div>
{/* Summary Cards */}
<div className="grid grid-cols-2 md:grid-cols-6 gap-3">
{/* Summary Cards — 7 columns: tokens, input, output, cost, accounts, keys, models */}
<div className="grid grid-cols-2 md:grid-cols-7 gap-3">
<StatCard
icon="generating_tokens"
label="Total Tokens"
@@ -103,6 +104,12 @@ export default function UsageAnalytics() {
value={fmt(s.completionTokens)}
color="text-emerald-500"
/>
<StatCard
icon="payments"
label="Est. Cost"
value={fmtCost(s.totalCost)}
color="text-amber-500"
/>
<StatCard icon="group" label="Accounts" value={s.uniqueAccounts || 0} />
<StatCard icon="vpn_key" label="API Keys" value={s.uniqueApiKeys || 0} />
<StatCard icon="model_training" label="Models" value={s.uniqueModels || 0} />
@@ -119,35 +126,30 @@ export default function UsageAnalytics() {
</div>
</div>
{/* Token Trend + Account Donut */}
{/* Token & Cost Trend + Provider Cost Donut */}
<div className="grid grid-cols-1 md:grid-cols-2 gap-4">
<DailyTrendChart dailyTrend={analytics?.dailyTrend} />
<AccountDonut byAccount={analytics?.byAccount} />
<ProviderCostDonut byProvider={analytics?.byProvider} />
</div>
{/* API Key Graph + Table */}
{/* Account Donut + API Key Donut */}
<div className="grid grid-cols-1 md:grid-cols-2 gap-4">
<AccountDonut byAccount={analytics?.byAccount} />
<ApiKeyDonut byApiKey={analytics?.byApiKey} />
<ApiKeyTable byApiKey={analytics?.byApiKey} />
</div>
{/* API Key Table */}
<ApiKeyTable byApiKey={analytics?.byApiKey} />
{/* Model Breakdown Table */}
<ModelTable byModel={analytics?.byModel} summary={s} />
{/* Bottom Stats */}
<div className="grid grid-cols-2 md:grid-cols-6 gap-3">
<div className="grid grid-cols-2 md:grid-cols-4 gap-3">
<Card className="px-4 py-3 text-center">
<span className="text-xs text-text-muted uppercase font-semibold">Requests</span>
<div className="text-lg font-bold mt-1">{fmtFull(s.totalRequests)}</div>
</Card>
<Card className="px-4 py-3 text-center">
<span className="text-xs text-text-muted uppercase font-semibold">Models</span>
<div className="text-lg font-bold mt-1">{s.uniqueModels}</div>
</Card>
<Card className="px-4 py-3 text-center">
<span className="text-xs text-text-muted uppercase font-semibold">Accounts</span>
<div className="text-lg font-bold mt-1">{s.uniqueAccounts}</div>
</Card>
<Card className="px-4 py-3 text-center">
<span className="text-xs text-text-muted uppercase font-semibold">Streak</span>
<div className="text-lg font-bold mt-1">{s.streak || 0}d</div>
@@ -155,7 +157,6 @@ export default function UsageAnalytics() {
<Card className="px-4 py-3 text-center">
<span className="text-xs text-text-muted uppercase font-semibold">Total Tokens</span>
<div className="text-lg font-bold mt-1">{fmt(s.totalTokens)}</div>
<span className="text-[10px] text-text-muted">Est. {fmtCost(s.totalCost)}</span>
</Card>
<Card className="px-4 py-3 text-center">
<span className="text-xs text-text-muted uppercase font-semibold">Usage Cost</span>

View File

@@ -9,7 +9,19 @@ import {
fmtCost,
formatApiKeyLabel as maskApiKeyLabel,
} from "@/shared/utils/formatting";
import { BarChart, Bar, XAxis, Tooltip, ResponsiveContainer, Cell, PieChart, Pie } from "recharts";
import {
BarChart,
ComposedChart,
Bar,
Line,
XAxis,
YAxis,
Tooltip,
ResponsiveContainer,
Cell,
PieChart,
Pie,
} from "recharts";
// ── Custom Tooltip for dark theme ──────────────────────────────────────────
@@ -214,9 +226,12 @@ export function DailyTrendChart({ dailyTrend }) {
date: d.date.slice(5),
Input: d.promptTokens,
Output: d.completionTokens,
Cost: d.cost || 0,
}));
}, [dailyTrend]);
const hasCost = useMemo(() => chartData.some((d) => d.Cost > 0), [chartData]);
if (!chartData.length) {
return (
<Card className="p-4">
@@ -231,10 +246,10 @@ export function DailyTrendChart({ dailyTrend }) {
return (
<Card className="p-4 flex-1">
<h3 className="text-sm font-semibold text-text-muted uppercase tracking-wider mb-3">
Token Trend
Token &amp; Cost Trend
</h3>
<ResponsiveContainer width="100%" height={128}>
<BarChart data={chartData} margin={{ top: 0, right: 0, left: 0, bottom: 0 }}>
<ResponsiveContainer width="100%" height={140}>
<ComposedChart data={chartData} margin={{ top: 0, right: hasCost ? 40 : 0, left: 0, bottom: 0 }}>
<XAxis
dataKey="date"
tick={{ fontSize: 9, fill: "var(--text-muted)" }}
@@ -242,8 +257,19 @@ export function DailyTrendChart({ dailyTrend }) {
tickLine={false}
interval={Math.max(Math.floor(chartData.length / 6), 0)}
/>
{hasCost && (
<YAxis
yAxisId="cost"
orientation="right"
tick={{ fontSize: 8, fill: "#f59e0b" }}
axisLine={false}
tickLine={false}
tickFormatter={(v) => `$${v.toFixed(2)}`}
width={36}
/>
)}
<Tooltip
content={<DarkTooltip formatter={fmt} />}
content={<CostTooltip />}
cursor={{ fill: "rgba(255,255,255,0.04)" }}
/>
<Bar
@@ -262,7 +288,18 @@ export function DailyTrendChart({ dailyTrend }) {
radius={[3, 3, 0, 0]}
animationDuration={600}
/>
</BarChart>
{hasCost && (
<Line
yAxisId="cost"
type="monotone"
dataKey="Cost"
stroke="#f59e0b"
strokeWidth={2}
dot={false}
animationDuration={600}
/>
)}
</ComposedChart>
</ResponsiveContainer>
<div className="flex items-center gap-4 mt-2 text-[10px] text-text-muted">
<span className="flex items-center gap-1">
@@ -271,11 +308,39 @@ export function DailyTrendChart({ dailyTrend }) {
<span className="flex items-center gap-1">
<span className="w-2 h-2 rounded-full bg-emerald-500/70" /> Output
</span>
{hasCost && (
<span className="flex items-center gap-1">
<span className="w-2 h-2 rounded-full bg-amber-500/70" /> Cost ($)
</span>
)}
</div>
</Card>
);
}
// ── Cost-aware Tooltip ─────────────────────────────────────────────────────
function CostTooltip({ active, payload, label }) {
if (!active || !payload?.length) return null;
return (
<div className="rounded-lg border border-white/10 bg-surface px-3 py-2 text-xs shadow-lg">
{label && <div className="font-semibold text-text-main mb-1">{label}</div>}
{payload.map((entry, i) => (
<div key={i} className="flex items-center gap-1.5 text-text-muted">
<span
className="w-2 h-2 rounded-full shrink-0"
style={{ backgroundColor: entry.color }}
/>
<span>{entry.name}:</span>
<span className="font-mono font-medium text-text-main">
{entry.name === "Cost" ? fmtCost(entry.value) : fmt(entry.value)}
</span>
</div>
))}
</div>
);
}
// ── AccountDonut (Recharts) ────────────────────────────────────────────────
export function AccountDonut({ byAccount }) {
@@ -834,6 +899,12 @@ export function ModelTable({ byModel, summary }) {
>
Total <SortIndicator active={sortBy === "totalTokens"} sortOrder={sortOrder} />
</th>
<th
className="px-4 py-2.5 text-right cursor-pointer group"
onClick={() => toggleSort("cost")}
>
Cost <SortIndicator active={sortBy === "cost"} sortOrder={sortOrder} />
</th>
<th className="px-4 py-2.5 text-right w-36">Share</th>
</tr>
</thead>
@@ -864,6 +935,9 @@ export function ModelTable({ byModel, summary }) {
<td className="px-4 py-2.5 text-right font-mono font-semibold">
{fmt(m.totalTokens)}
</td>
<td className="px-4 py-2.5 text-right font-mono text-amber-500">
{fmtCost(m.cost)}
</td>
<td className="px-4 py-2.5 text-right">
<div className="flex items-center gap-2 justify-end">
<div className="w-16 h-1.5 rounded-full bg-white/[0.06] overflow-hidden">
@@ -911,3 +985,85 @@ export function UsageDetail({ summary }) {
</Card>
);
}
// ── ProviderCostDonut ──────────────────────────────────────────────────────
const PROVIDER_COLORS = [
"#f59e0b", "#ef4444", "#8b5cf6", "#10b981", "#06b6d4",
"#ec4899", "#f97316", "#6366f1", "#14b8a6", "#a855f7",
];
export function ProviderCostDonut({ byProvider }) {
const data = useMemo(() => byProvider || [], [byProvider]);
const hasData = data.length > 0 && data.some((p) => p.cost > 0);
const pieData = useMemo(() => {
return data
.filter((item) => item.cost > 0)
.sort((a, b) => b.cost - a.cost)
.slice(0, 8)
.map((item, i) => ({
name: item.provider,
value: item.cost,
fill: PROVIDER_COLORS[i % PROVIDER_COLORS.length],
}));
}, [data]);
if (!hasData) {
return (
<Card className="p-4 flex-1">
<h3 className="text-sm font-semibold text-text-muted uppercase tracking-wider mb-3">
Cost by Provider
</h3>
<div className="text-center text-text-muted text-sm py-8">No cost data</div>
</Card>
);
}
return (
<Card className="p-4 flex-1">
<h3 className="text-sm font-semibold text-text-muted uppercase tracking-wider mb-3">
Cost by Provider
</h3>
<div className="flex items-center gap-4">
<ResponsiveContainer width={120} height={120}>
<PieChart>
<Pie
data={pieData}
dataKey="value"
nameKey="name"
cx="50%"
cy="50%"
innerRadius={28}
outerRadius={55}
paddingAngle={1}
animationDuration={600}
>
{pieData.map((entry, i) => (
<Cell key={i} fill={entry.fill} stroke="none" />
))}
</Pie>
<Tooltip content={<DarkTooltip formatter={fmtCost} />} />
</PieChart>
</ResponsiveContainer>
<div className="flex flex-col gap-1 min-w-0 flex-1">
{pieData.map((seg, i) => (
<div key={i} className="flex items-center justify-between gap-2 text-xs">
<div className="flex items-center gap-1.5 min-w-0">
<span
className="w-2 h-2 rounded-full shrink-0"
style={{ backgroundColor: seg.fill }}
/>
<span className="truncate text-text-main capitalize">{seg.name}</span>
</div>
<span className="font-mono font-medium text-amber-500 shrink-0">
{fmtCost(seg.value)}
</span>
</div>
))}
</div>
</div>
</Card>
);
}

View File

@@ -11,4 +11,5 @@ export {
WeeklySquares7d,
ModelTable,
UsageDetail,
ProviderCostDonut,
} from "./charts";

View File

@@ -10,6 +10,8 @@ import {
getEarliestRateLimitedUntil,
formatRetryAfter,
checkFallbackError,
isModelLocked,
lockModel,
} from "@omniroute/open-sse/services/accountFallback.js";
import * as log from "../utils/logger.js";
@@ -185,35 +187,46 @@ export async function getProviderCredentials(provider, excludeConnectionId = nul
/**
* Mark account as unavailable — reads backoffLevel from DB, calculates cooldown with exponential backoff, saves new level
* @param {string} connectionId
* @param {number} status - HTTP status code
* @param {string} errorText - Error message
* @param {string|null} provider
* @param {string|null} model - Model name for per-model lockout
* @returns {{ shouldFallback: boolean, cooldownMs: number }}
*/
export async function markAccountUnavailable(connectionId, status, errorText, provider = null) {
export async function markAccountUnavailable(connectionId, status, errorText, provider = null, model = null) {
// Read current connection to get backoffLevel
const connections = await getProviderConnections({ provider });
const conn = connections.find((c) => c.id === connectionId);
const backoffLevel = conn?.backoffLevel || 0;
const { shouldFallback, cooldownMs, newBackoffLevel } = checkFallbackError(
const { shouldFallback, cooldownMs, newBackoffLevel, reason } = checkFallbackError(
status,
errorText,
backoffLevel
backoffLevel,
model
);
if (!shouldFallback) return { shouldFallback: false, cooldownMs: 0 };
const rateLimitedUntil = getUnavailableUntil(cooldownMs);
const reason = typeof errorText === "string" ? errorText.slice(0, 100) : "Provider error";
const errorMsg = typeof errorText === "string" ? errorText.slice(0, 100) : "Provider error";
await updateProviderConnection(connectionId, {
rateLimitedUntil,
testStatus: "unavailable",
lastError: reason,
lastError: errorMsg,
errorCode: status,
lastErrorAt: new Date().toISOString(),
backoffLevel: newBackoffLevel ?? backoffLevel,
});
if (provider && status && reason) {
console.error(`${provider} [${status}]: ${reason}`);
// Per-model lockout: lock the specific model if known
if (provider && model && cooldownMs > 0) {
lockModel(provider, connectionId, model, reason || "unknown", cooldownMs);
}
if (provider && status && errorMsg) {
console.error(`${provider} [${status}]: ${errorMsg}`);
}
return { shouldFallback: true, cooldownMs };

View File

@@ -0,0 +1,81 @@
import test from "node:test";
import assert from "node:assert/strict";
const {
selectAccountP2C,
selectAccount,
} = await import("../../open-sse/services/accountSelector.js");
// ─── selectAccountP2C ───────────────────────────────────────────────────────
test("selectAccountP2C: returns null for empty array", () => {
assert.equal(selectAccountP2C([]), null);
assert.equal(selectAccountP2C(null), null);
});
test("selectAccountP2C: returns single account", () => {
const acct = { id: "a1" };
assert.equal(selectAccountP2C([acct]), acct);
});
test("selectAccountP2C: returns one of the candidates", () => {
const accounts = [
{ id: "a1" },
{ id: "a2" },
{ id: "a3" },
];
const selected = selectAccountP2C(accounts);
assert.ok(accounts.includes(selected));
});
test("selectAccountP2C: prefers healthier account over many runs", () => {
// Account with error should be selected less often
const healthy = { id: "healthy" };
const degraded = { id: "degraded", error: true, rateLimited: true, backoffLevel: 3 };
const accounts = [healthy, degraded];
let healthyCount = 0;
for (let i = 0; i < 100; i++) {
const selected = selectAccountP2C(accounts);
if (selected.id === "healthy") healthyCount++;
}
// Should strongly prefer healthy account
assert.ok(healthyCount > 60, `Expected healthy to win >60/100 times, got ${healthyCount}`);
});
// ─── selectAccount strategies ───────────────────────────────────────────────
test("selectAccount: fill-first returns first", () => {
const accounts = [{ id: "first" }, { id: "second" }];
const { account } = selectAccount(accounts, "fill-first");
assert.equal(account.id, "first");
});
test("selectAccount: round-robin cycles", () => {
const accounts = [{ id: "a" }, { id: "b" }, { id: "c" }];
let state = {};
const results = [];
for (let i = 0; i < 6; i++) {
const { account, state: newState } = selectAccount(accounts, "round-robin", state);
results.push(account.id);
state = newState;
}
assert.deepEqual(results, ["a", "b", "c", "a", "b", "c"]);
});
test("selectAccount: random returns valid account", () => {
const accounts = [{ id: "x" }, { id: "y" }];
const { account } = selectAccount(accounts, "random");
assert.ok(accounts.includes(account));
});
test("selectAccount: p2c returns valid account", () => {
const accounts = [{ id: "p1" }, { id: "p2" }, { id: "p3" }];
const { account } = selectAccount(accounts, "p2c");
assert.ok(accounts.includes(account));
});
test("selectAccount: empty accounts returns null", () => {
const { account } = selectAccount([], "fill-first");
assert.equal(account, null);
});

View File

@@ -0,0 +1,119 @@
import test from "node:test";
import assert from "node:assert/strict";
const {
compressContext,
estimateTokens,
getTokenLimit,
} = await import("../../open-sse/services/contextManager.js");
// ─── estimateTokens ─────────────────────────────────────────────────────────
test("estimateTokens: estimates from string", () => {
assert.equal(estimateTokens("hello"), 2); // 5/4 = 2
assert.ok(estimateTokens("a".repeat(100)) === 25);
});
test("estimateTokens: handles null", () => {
assert.equal(estimateTokens(null), 0);
assert.equal(estimateTokens(""), 0);
});
// ─── getTokenLimit ──────────────────────────────────────────────────────────
test("getTokenLimit: detects claude", () => {
assert.equal(getTokenLimit("claude", "claude-sonnet-4"), 200000);
});
test("getTokenLimit: detects gemini", () => {
assert.equal(getTokenLimit("gemini", "gemini-2.5-pro"), 1000000);
});
test("getTokenLimit: default fallback", () => {
assert.equal(getTokenLimit("unknown"), 128000);
});
// ─── compressContext ────────────────────────────────────────────────────────
test("compressContext: returns unchanged if fits", () => {
const body = {
model: "claude-sonnet-4",
messages: [
{ role: "system", content: "You are helpful." },
{ role: "user", content: "Hello" },
],
};
const result = compressContext(body);
assert.equal(result.compressed, false);
});
test("compressContext: handles null/empty body", () => {
assert.equal(compressContext(null).compressed, false);
assert.equal(compressContext({}).compressed, false);
assert.equal(compressContext({ messages: null }).compressed, false);
});
test("compressContext: Layer 1 — trims long tool messages", () => {
const longContent = "x".repeat(10000);
const body = {
model: "test",
messages: [
{ role: "user", content: "run tool" },
{ role: "tool", content: longContent, tool_call_id: "t1" },
{ role: "user", content: "done?" },
],
};
// Use very tight limit to force compression
const result = compressContext(body, { maxTokens: 500, reserveTokens: 100 });
assert.ok(result.compressed);
const toolMsg = result.body.messages.find((m) => m.role === "tool");
assert.ok(toolMsg.content.length < longContent.length);
assert.ok(toolMsg.content.includes("[truncated]"));
});
test("compressContext: Layer 2 — compresses thinking in old messages", () => {
const body = {
model: "test",
messages: [
{ role: "user", content: "q1" },
{
role: "assistant",
content: [
{ type: "thinking", thinking: "lots of thinking here ".repeat(500) },
{ type: "text", text: "answer1" },
],
},
{ role: "user", content: "q2" },
{
role: "assistant",
content: [
{ type: "thinking", thinking: "more thinking" },
{ type: "text", text: "answer2" },
],
},
],
};
const result = compressContext(body, { maxTokens: 2000, reserveTokens: 500 });
// First assistant should have thinking removed
const firstAssistant = result.body.messages.find((m) => m.role === "assistant");
if (Array.isArray(firstAssistant.content)) {
const hasThinking = firstAssistant.content.some((b) => b.type === "thinking");
assert.equal(hasThinking, false);
}
});
test("compressContext: Layer 3 — drops old messages to fit", () => {
const messages = [
{ role: "system", content: "You are helpful" },
...Array.from({ length: 100 }, (_, i) => [
{ role: "user", content: `Message ${i}: ${"content ".repeat(50)}` },
{ role: "assistant", content: `Response ${i}: ${"answer ".repeat(50)}` },
]).flat(),
];
const body = { model: "test", messages };
const result = compressContext(body, { maxTokens: 3000, reserveTokens: 500 });
assert.ok(result.compressed);
assert.ok(result.body.messages.length < messages.length);
// System message preserved
assert.equal(result.body.messages[0].role, "system");
});

View File

@@ -0,0 +1,123 @@
import test from "node:test";
import assert from "node:assert/strict";
const {
checkIP,
configureIPFilter,
tempBanIP,
removeTempBan,
addToBlacklist,
removeFromBlacklist,
addToWhitelist,
removeFromWhitelist,
getIPFilterConfig,
resetIPFilter,
} = await import("../../open-sse/services/ipFilter.js");
test.beforeEach(() => resetIPFilter());
// ─── Disabled ───────────────────────────────────────────────────────────────
test("disabled: allows all IPs", () => {
assert.equal(checkIP("1.2.3.4").allowed, true);
});
// ─── Blacklist Mode ─────────────────────────────────────────────────────────
test("blacklist: blocks blacklisted IP", () => {
configureIPFilter({ enabled: true, mode: "blacklist", blacklist: ["1.2.3.4"] });
assert.equal(checkIP("1.2.3.4").allowed, false);
assert.equal(checkIP("5.6.7.8").allowed, true);
});
test("blacklist: CIDR match", () => {
configureIPFilter({ enabled: true, mode: "blacklist", blacklist: ["192.168.1.0/24"] });
assert.equal(checkIP("192.168.1.100").allowed, false);
assert.equal(checkIP("192.168.2.1").allowed, true);
});
test("blacklist: wildcard match", () => {
configureIPFilter({ enabled: true, mode: "blacklist", blacklist: ["10.0.*.*"] });
assert.equal(checkIP("10.0.1.1").allowed, false);
assert.equal(checkIP("10.1.0.1").allowed, true);
});
// ─── Whitelist Mode ─────────────────────────────────────────────────────────
test("whitelist: only allows listed IPs", () => {
configureIPFilter({ enabled: true, mode: "whitelist", whitelist: ["1.2.3.4"] });
assert.equal(checkIP("1.2.3.4").allowed, true);
assert.equal(checkIP("5.6.7.8").allowed, false);
});
test("whitelist: CIDR match", () => {
configureIPFilter({ enabled: true, mode: "whitelist", whitelist: ["10.0.0.0/8"] });
assert.equal(checkIP("10.255.255.255").allowed, true);
assert.equal(checkIP("11.0.0.1").allowed, false);
});
// ─── Whitelist Priority Mode ────────────────────────────────────────────────
test("whitelist-priority: whitelist overrides blacklist", () => {
configureIPFilter({
enabled: true,
mode: "whitelist-priority",
blacklist: ["192.168.1.0/24"],
whitelist: ["192.168.1.100"],
});
assert.equal(checkIP("192.168.1.100").allowed, true); // Whitelisted
assert.equal(checkIP("192.168.1.50").allowed, false); // Blacklisted
assert.equal(checkIP("10.0.0.1").allowed, true); // Neither
});
// ─── Temporary Bans ─────────────────────────────────────────────────────────
test("tempBanIP: bans temporarily", () => {
configureIPFilter({ enabled: true, mode: "blacklist" });
tempBanIP("5.5.5.5", 60000, "abuse");
assert.equal(checkIP("5.5.5.5").allowed, false);
assert.ok(checkIP("5.5.5.5").reason.includes("banned"));
});
test("removeTempBan: removes ban", () => {
configureIPFilter({ enabled: true, mode: "blacklist" });
tempBanIP("5.5.5.5", 60000, "abuse");
removeTempBan("5.5.5.5");
assert.equal(checkIP("5.5.5.5").allowed, true);
});
// ─── Dynamic List Management ────────────────────────────────────────────────
test("addToBlacklist/removeFromBlacklist: dynamic updates", () => {
configureIPFilter({ enabled: true, mode: "blacklist" });
addToBlacklist("9.9.9.9");
assert.equal(checkIP("9.9.9.9").allowed, false);
removeFromBlacklist("9.9.9.9");
assert.equal(checkIP("9.9.9.9").allowed, true);
});
test("addToWhitelist/removeFromWhitelist: dynamic updates", () => {
configureIPFilter({ enabled: true, mode: "whitelist" });
addToWhitelist("1.1.1.1");
assert.equal(checkIP("1.1.1.1").allowed, true);
removeFromWhitelist("1.1.1.1");
assert.equal(checkIP("1.1.1.1").allowed, false);
});
// ─── IPv6 Normalization ─────────────────────────────────────────────────────
test("normalizes ::ffff: prefix", () => {
configureIPFilter({ enabled: true, mode: "blacklist", blacklist: ["1.2.3.4"] });
assert.equal(checkIP("::ffff:1.2.3.4").allowed, false);
});
// ─── Config API ─────────────────────────────────────────────────────────────
test("getIPFilterConfig: returns serializable config", () => {
configureIPFilter({ enabled: true, mode: "whitelist", whitelist: ["1.2.3.4"] });
const config = getIPFilterConfig();
assert.equal(config.enabled, true);
assert.equal(config.mode, "whitelist");
assert.ok(Array.isArray(config.whitelist));
assert.ok(config.whitelist.includes("1.2.3.4"));
});

View File

@@ -0,0 +1,273 @@
import test from "node:test";
import assert from "node:assert/strict";
const {
checkFallbackError,
parseRetryAfterFromBody,
classifyError,
classifyErrorText,
lockModel,
isModelLocked,
getModelLockoutInfo,
getAllModelLockouts,
getQuotaCooldown,
getBackoffDuration,
getAccountHealth,
isAccountUnavailable,
getUnavailableUntil,
formatRetryAfter,
filterAvailableAccounts,
resetAccountState,
applyErrorState,
} = await import("../../open-sse/services/accountFallback.js");
const { RateLimitReason, BACKOFF_STEPS_MS } = await import("../../open-sse/config/constants.js");
// ─── parseRetryAfterFromBody Tests ──────────────────────────────────────────
test("parseRetryAfterFromBody: parses Gemini retryDelay format", () => {
const body = {
error: {
code: 429,
message: "Resource has been exhausted",
details: [{ "@type": "google.rpc.RetryInfo", retryDelay: "33s" }],
},
};
const result = parseRetryAfterFromBody(body);
assert.equal(result.retryAfterMs, 33000);
assert.equal(result.reason, RateLimitReason.RATE_LIMIT_EXCEEDED);
});
test("parseRetryAfterFromBody: parses OpenAI retry message format", () => {
const body = {
error: {
message: "Rate limit reached. Please retry after 20s.",
type: "rate_limit_error",
},
};
const result = parseRetryAfterFromBody(body);
assert.equal(result.retryAfterMs, 20000);
assert.equal(result.reason, RateLimitReason.RATE_LIMIT_EXCEEDED);
});
test("parseRetryAfterFromBody: classifies Anthropic rate_limit_error", () => {
const body = {
type: "error",
error: { type: "rate_limit_error", message: "Too many requests" },
};
const result = parseRetryAfterFromBody(body);
assert.equal(result.reason, RateLimitReason.RATE_LIMIT_EXCEEDED);
});
test("parseRetryAfterFromBody: handles string input", () => {
const body = JSON.stringify({
error: { details: [{ retryDelay: "10s" }] },
});
const result = parseRetryAfterFromBody(body);
assert.equal(result.retryAfterMs, 10000);
});
test("parseRetryAfterFromBody: handles invalid JSON", () => {
const result = parseRetryAfterFromBody("not json");
assert.equal(result.retryAfterMs, null);
assert.equal(result.reason, RateLimitReason.UNKNOWN);
});
test("parseRetryAfterFromBody: handles null/undefined", () => {
assert.equal(parseRetryAfterFromBody(null).retryAfterMs, null);
assert.equal(parseRetryAfterFromBody(undefined).retryAfterMs, null);
});
// ─── classifyError Tests ────────────────────────────────────────────────────
test("classifyError: 429 → RATE_LIMIT_EXCEEDED", () => {
assert.equal(classifyError(429, ""), RateLimitReason.RATE_LIMIT_EXCEEDED);
});
test("classifyError: 401 → AUTH_ERROR", () => {
assert.equal(classifyError(401, ""), RateLimitReason.AUTH_ERROR);
});
test("classifyError: 402 → QUOTA_EXHAUSTED", () => {
assert.equal(classifyError(402, ""), RateLimitReason.QUOTA_EXHAUSTED);
});
test("classifyError: 503 → MODEL_CAPACITY", () => {
assert.equal(classifyError(503, ""), RateLimitReason.MODEL_CAPACITY);
});
test("classifyError: text overrides status code", () => {
// 500 normally → SERVER_ERROR, but quota text → QUOTA_EXHAUSTED
assert.equal(classifyError(500, "quota exceeded"), RateLimitReason.QUOTA_EXHAUSTED);
});
test("classifyErrorText: handles various patterns", () => {
assert.equal(classifyErrorText("rate limit reached"), RateLimitReason.RATE_LIMIT_EXCEEDED);
assert.equal(classifyErrorText("too many requests"), RateLimitReason.RATE_LIMIT_EXCEEDED);
assert.equal(classifyErrorText("capacity exceeded"), RateLimitReason.MODEL_CAPACITY);
assert.equal(classifyErrorText("overloaded"), RateLimitReason.MODEL_CAPACITY);
assert.equal(classifyErrorText("unauthorized"), RateLimitReason.AUTH_ERROR);
assert.equal(classifyErrorText("random error"), RateLimitReason.UNKNOWN);
});
// ─── Per-Model Lockout Tests ────────────────────────────────────────────────
test("lockModel + isModelLocked: locks specific model", () => {
lockModel("claude", "conn1", "claude-sonnet-4", "rate_limit_exceeded", 5000);
assert.equal(isModelLocked("claude", "conn1", "claude-sonnet-4"), true);
});
test("isModelLocked: different model not locked", () => {
lockModel("claude", "conn2", "claude-sonnet-4", "rate_limit_exceeded", 5000);
assert.equal(isModelLocked("claude", "conn2", "claude-haiku-4"), false);
});
test("isModelLocked: returns false when no model specified", () => {
assert.equal(isModelLocked("claude", "conn1", null), false);
assert.equal(isModelLocked("claude", "conn1", undefined), false);
});
test("getModelLockoutInfo: returns lockout details", () => {
lockModel("openai", "conn3", "gpt-4o", "quota_exhausted", 10000);
const info = getModelLockoutInfo("openai", "conn3", "gpt-4o");
assert.ok(info);
assert.equal(info.reason, "quota_exhausted");
assert.ok(info.remainingMs > 0);
});
test("getAllModelLockouts: returns active lockouts", () => {
lockModel("test-provider", "conn-test", "test-model", "test", 10000);
const lockouts = getAllModelLockouts();
const found = lockouts.find((l) => l.model === "test-model");
assert.ok(found);
assert.equal(found.provider, "test-provider");
});
// ─── checkFallbackError Tests ────────────────────────────────────────────────
test("checkFallbackError: backward compatible without model param", () => {
const result = checkFallbackError(429, "Rate limit hit", 0);
assert.equal(result.shouldFallback, true);
assert.ok(result.cooldownMs > 0);
assert.equal(result.newBackoffLevel, 1);
assert.equal(result.reason, RateLimitReason.RATE_LIMIT_EXCEEDED);
});
test("checkFallbackError: 400 does not trigger fallback", () => {
const result = checkFallbackError(400, "bad request");
assert.equal(result.shouldFallback, false);
});
test("checkFallbackError: server error has reason", () => {
const result = checkFallbackError(500, "internal server error");
assert.equal(result.shouldFallback, true);
assert.equal(result.reason, RateLimitReason.SERVER_ERROR);
});
test("checkFallbackError: transient errors don't increase backoff level", () => {
const result = checkFallbackError(502, "", 5);
assert.equal(result.shouldFallback, true);
assert.equal(result.newBackoffLevel, undefined);
});
// ─── Backoff Steps Tests ────────────────────────────────────────────────────
test("getBackoffDuration: follows step sequence", () => {
assert.equal(getBackoffDuration(0), BACKOFF_STEPS_MS[0]); // 60s
assert.equal(getBackoffDuration(1), BACKOFF_STEPS_MS[1]); // 120s
assert.equal(getBackoffDuration(2), BACKOFF_STEPS_MS[2]); // 300s
assert.equal(getBackoffDuration(4), BACKOFF_STEPS_MS[4]); // 1200s
});
test("getBackoffDuration: caps at max step", () => {
assert.equal(getBackoffDuration(100), BACKOFF_STEPS_MS[BACKOFF_STEPS_MS.length - 1]);
});
// ─── Exponential backoff (original) Tests ───────────────────────────────────
test("getQuotaCooldown: exponential progression", () => {
assert.equal(getQuotaCooldown(0), 1000); // 1s
assert.equal(getQuotaCooldown(1), 2000); // 2s
assert.equal(getQuotaCooldown(3), 8000); // 8s
assert.ok(getQuotaCooldown(20) <= 120000); // Capped at 2min
});
// ─── Account Health Tests ───────────────────────────────────────────────────
test("getAccountHealth: healthy account = 100", () => {
assert.equal(getAccountHealth({ backoffLevel: 0 }), 100);
});
test("getAccountHealth: degraded by backoff level", () => {
assert.equal(getAccountHealth({ backoffLevel: 5 }), 50);
});
test("getAccountHealth: degraded by error + rateLimited", () => {
const score = getAccountHealth({
backoffLevel: 3,
lastError: { message: "something" },
rateLimitedUntil: new Date(Date.now() + 60000).toISOString(),
});
assert.equal(score, 20); // 100 - 30 - 20 - 30
});
test("getAccountHealth: null account = 0", () => {
assert.equal(getAccountHealth(null), 0);
});
// ─── Account State Tests ────────────────────────────────────────────────────
test("resetAccountState: clears all error state", () => {
const reset = resetAccountState({
rateLimitedUntil: "2030-01-01",
backoffLevel: 5,
lastError: "something",
status: "error",
});
assert.equal(reset.rateLimitedUntil, null);
assert.equal(reset.backoffLevel, 0);
assert.equal(reset.lastError, null);
assert.equal(reset.status, "active");
});
test("applyErrorState: applies cooldown and reason", () => {
const result = applyErrorState({ backoffLevel: 0 }, 429, "rate limit hit");
assert.ok(result.rateLimitedUntil);
assert.equal(result.backoffLevel, 1);
assert.equal(result.status, "error");
assert.ok(result.lastError.reason);
});
// ─── Utility Tests ──────────────────────────────────────────────────────────
test("isAccountUnavailable: false for null", () => {
assert.equal(isAccountUnavailable(null), false);
});
test("isAccountUnavailable: true for future timestamp", () => {
assert.equal(isAccountUnavailable(new Date(Date.now() + 60000).toISOString()), true);
});
test("isAccountUnavailable: false for past timestamp", () => {
assert.equal(isAccountUnavailable(new Date(Date.now() - 1000).toISOString()), false);
});
test("formatRetryAfter: formats correctly", () => {
const future = new Date(Date.now() + 150000).toISOString(); // 2.5 min
const formatted = formatRetryAfter(future);
assert.match(formatted, /reset after \d+m/);
});
test("filterAvailableAccounts: filters out rate-limited", () => {
const accounts = [
{ id: "a", rateLimitedUntil: null },
{ id: "b", rateLimitedUntil: new Date(Date.now() + 60000).toISOString() },
{ id: "c", rateLimitedUntil: new Date(Date.now() - 1000).toISOString() },
];
const available = filterAvailableAccounts(accounts);
assert.equal(available.length, 2); // a and c (expired)
assert.deepEqual(
available.map((a) => a.id),
["a", "c"]
);
});

View File

@@ -0,0 +1,149 @@
import test from "node:test";
import assert from "node:assert/strict";
const {
generateSessionId,
touchSession,
getSessionInfo,
getSessionConnection,
getActiveSessionCount,
getActiveSessions,
clearSessions,
} = await import("../../open-sse/services/sessionManager.js");
// Reset between tests
test.beforeEach(() => clearSessions());
// ─── Session ID Generation ──────────────────────────────────────────────────
test("generateSessionId: produces stable ID for same request", () => {
const body = {
model: "claude-sonnet-4-20250514",
messages: [
{ role: "system", content: "You are helpful." },
{ role: "user", content: "hello" },
],
};
const id1 = generateSessionId(body);
const id2 = generateSessionId(body);
assert.equal(id1, id2);
assert.equal(id1.length, 16);
});
test("generateSessionId: different model = different ID", () => {
const body1 = { model: "claude-sonnet-4-20250514", messages: [{ role: "user", content: "hi" }] };
const body2 = { model: "gpt-4o", messages: [{ role: "user", content: "hi" }] };
assert.notEqual(generateSessionId(body1), generateSessionId(body2));
});
test("generateSessionId: different system prompt = different ID", () => {
const body1 = {
model: "claude-sonnet-4-20250514",
messages: [{ role: "system", content: "A" }, { role: "user", content: "hi" }],
};
const body2 = {
model: "claude-sonnet-4-20250514",
messages: [{ role: "system", content: "B" }, { role: "user", content: "hi" }],
};
assert.notEqual(generateSessionId(body1), generateSessionId(body2));
});
test("generateSessionId: tools contribute to fingerprint", () => {
const body1 = {
model: "claude-sonnet-4-20250514",
messages: [{ role: "user", content: "hi" }],
tools: [{ name: "search" }],
};
const body2 = {
model: "claude-sonnet-4-20250514",
messages: [{ role: "user", content: "hi" }],
tools: [{ name: "different" }],
};
assert.notEqual(generateSessionId(body1), generateSessionId(body2));
});
test("generateSessionId: same tools in different order = same ID", () => {
const body1 = {
model: "m1",
messages: [{ role: "user", content: "hi" }],
tools: [{ name: "a" }, { name: "b" }],
};
const body2 = {
model: "m1",
messages: [{ role: "user", content: "hi" }],
tools: [{ name: "b" }, { name: "a" }],
};
assert.equal(generateSessionId(body1), generateSessionId(body2));
});
test("generateSessionId: Claude body.system format", () => {
const body = {
model: "claude-sonnet-4-20250514",
system: "You are helpful.",
messages: [{ role: "user", content: "hi" }],
};
const id = generateSessionId(body);
assert.ok(id);
assert.equal(id.length, 16);
});
test("generateSessionId: empty body returns null", () => {
assert.equal(generateSessionId({}), null);
});
test("generateSessionId: provider option changes ID", () => {
const body = { model: "m1", messages: [{ role: "user", content: "hi" }] };
const id1 = generateSessionId(body, { provider: "claude" });
const id2 = generateSessionId(body, { provider: "openai" });
assert.notEqual(id1, id2);
});
// ─── Session Tracking ───────────────────────────────────────────────────────
test("touchSession + getSessionInfo: creates and updates session", () => {
touchSession("abc123", "conn1");
const info = getSessionInfo("abc123");
assert.ok(info);
assert.equal(info.requestCount, 1);
assert.equal(info.connectionId, "conn1");
touchSession("abc123");
const updated = getSessionInfo("abc123");
assert.equal(updated.requestCount, 2);
});
test("getSessionConnection: returns bound connection", () => {
touchSession("sess1", "conn-x");
assert.equal(getSessionConnection("sess1"), "conn-x");
});
test("getSessionConnection: null for unknown session", () => {
assert.equal(getSessionConnection("nonexistent"), null);
});
test("getActiveSessionCount: tracks count", () => {
assert.equal(getActiveSessionCount(), 0);
touchSession("s1");
touchSession("s2");
assert.equal(getActiveSessionCount(), 2);
});
test("getActiveSessions: returns session list", () => {
touchSession("s1", "c1");
touchSession("s2", "c2");
const all = getActiveSessions();
assert.equal(all.length, 2);
assert.ok(all[0].sessionId);
assert.ok(all[0].ageMs >= 0);
});
test("clearSessions: empties store", () => {
touchSession("s1");
clearSessions();
assert.equal(getActiveSessionCount(), 0);
});
test("touchSession with null sessionId: no-op", () => {
touchSession(null);
assert.equal(getActiveSessionCount(), 0);
});

View File

@@ -0,0 +1,127 @@
import test from "node:test";
import assert from "node:assert/strict";
const {
getSignatures,
addSignature,
detectAndLearn,
getModelFamily,
getCacheStats,
clearCache,
} = await import("../../open-sse/services/signatureCache.js");
test.beforeEach(() => clearCache());
// ─── getSignatures ──────────────────────────────────────────────────────────
test("getSignatures: returns defaults when empty cache", () => {
const sigs = getSignatures();
assert.ok(sigs.includes("<antThinking>"));
assert.ok(sigs.includes("</antThinking>"));
assert.ok(sigs.includes("<thinking>"));
assert.ok(sigs.length >= 6);
});
test("getSignatures: merges tool-layer patterns", () => {
addSignature("<cursorThinking>", { tool: "cursor" });
const sigs = getSignatures({ tool: "cursor" });
assert.ok(sigs.includes("<cursorThinking>"));
assert.ok(sigs.includes("<antThinking>")); // defaults still present
});
test("getSignatures: merges family-layer patterns", () => {
addSignature("<deepThought>", { modelFamily: "claude-sonnet" });
const sigs = getSignatures({ modelFamily: "claude-sonnet" });
assert.ok(sigs.includes("<deepThought>"));
});
test("getSignatures: merges session-layer patterns", () => {
addSignature("<sessThink>", { sessionId: "abc123" });
const sigs = getSignatures({ sessionId: "abc123" });
assert.ok(sigs.includes("<sessThink>"));
// Another session should NOT see it
const other = getSignatures({ sessionId: "other" });
assert.ok(!other.includes("<sessThink>"));
});
// ─── addSignature ───────────────────────────────────────────────────────────
test("addSignature: ignores null/empty", () => {
addSignature(null, { tool: "test" });
addSignature("", { tool: "test" });
const stats = getCacheStats();
assert.equal(stats.tool.entries, 0);
});
test("addSignature: adds to multiple layers", () => {
addSignature("<multiTag>", { tool: "t", modelFamily: "f", sessionId: "s" });
assert.ok(getSignatures({ tool: "t" }).includes("<multiTag>"));
assert.ok(getSignatures({ modelFamily: "f" }).includes("<multiTag>"));
assert.ok(getSignatures({ sessionId: "s" }).includes("<multiTag>"));
});
// ─── detectAndLearn ─────────────────────────────────────────────────────────
test("detectAndLearn: finds known signatures", () => {
const result = detectAndLearn("<antThinking>I think...</antThinking> Hello!", {});
assert.ok(result.found.includes("<antThinking>"));
assert.ok(result.found.includes("</antThinking>"));
assert.ok(!result.cleaned.includes("<antThinking>"));
});
test("detectAndLearn: auto-learns new thinking tags", () => {
const text = "<customThinking>some thought</customThinking> answer";
const result = detectAndLearn(text, { tool: "test-tool" });
assert.ok(result.found.includes("<customThinking>"));
// Should now be in cache
const sigs = getSignatures({ tool: "test-tool" });
assert.ok(sigs.includes("<customThinking>"));
});
test("detectAndLearn: handles null/empty input", () => {
assert.deepEqual(detectAndLearn(null), { found: [], cleaned: null });
assert.deepEqual(detectAndLearn(""), { found: [], cleaned: "" });
});
test("detectAndLearn: preserves non-thinking text", () => {
const result = detectAndLearn("Hello world, no thinking tags here", {});
assert.equal(result.found.length, 0);
assert.equal(result.cleaned, "Hello world, no thinking tags here");
});
// ─── getModelFamily ─────────────────────────────────────────────────────────
test("getModelFamily: extracts family from versioned model", () => {
assert.equal(getModelFamily("claude-sonnet-4-20250514"), "claude-sonnet");
assert.equal(getModelFamily("gpt-4o-2024-08-06"), "gpt-4o");
});
test("getModelFamily: returns null for empty", () => {
assert.equal(getModelFamily(null), null);
assert.equal(getModelFamily(""), null);
});
test("getModelFamily: handles simple names", () => {
assert.equal(getModelFamily("claude-sonnet-4"), "claude-sonnet");
assert.equal(getModelFamily("gpt-4o"), "gpt-4o");
});
// ─── getCacheStats ──────────────────────────────────────────────────────────
test("getCacheStats: returns structure", () => {
const stats = getCacheStats();
assert.ok("tool" in stats);
assert.ok("family" in stats);
assert.ok("session" in stats);
assert.ok("defaultCount" in stats);
assert.equal(stats.defaultCount, 6);
});
test("getCacheStats: counts accurately", () => {
addSignature("<a>", { tool: "t1" });
addSignature("<b>", { tool: "t1" });
addSignature("<c>", { tool: "t2" });
const stats = getCacheStats();
assert.equal(stats.tool.entries, 2); // t1, t2
assert.equal(stats.tool.patterns, 3); // a, b under t1 + c under t2
});

View File

@@ -0,0 +1,96 @@
import test from "node:test";
import assert from "node:assert/strict";
const {
injectSystemPrompt,
setSystemPromptConfig,
getSystemPromptConfig,
} = await import("../../open-sse/services/systemPrompt.js");
// ─── Config ─────────────────────────────────────────────────────────────────
test("default config: disabled", () => {
const config = getSystemPromptConfig();
assert.equal(config.enabled, false);
assert.equal(config.prompt, "");
});
// ─── Injection ──────────────────────────────────────────────────────────────
test("injectSystemPrompt: disabled → no change", () => {
setSystemPromptConfig({ enabled: false, prompt: "system" });
const body = { messages: [{ role: "user", content: "hi" }] };
const result = injectSystemPrompt(body);
assert.deepEqual(result, body);
});
test("injectSystemPrompt: adds system message when none exists", () => {
setSystemPromptConfig({ enabled: true, prompt: "You are an AI assistant." });
const body = { messages: [{ role: "user", content: "hi" }] };
const result = injectSystemPrompt(body);
assert.equal(result.messages[0].role, "system");
assert.ok(result.messages[0].content.includes("You are an AI assistant."));
assert.equal(result.messages.length, 2);
});
test("injectSystemPrompt: prepends to existing system message", () => {
setSystemPromptConfig({ enabled: true, prompt: "GLOBAL:" });
const body = {
messages: [
{ role: "system", content: "Original prompt" },
{ role: "user", content: "hi" },
],
};
const result = injectSystemPrompt(body);
assert.ok(result.messages[0].content.startsWith("GLOBAL:"));
assert.ok(result.messages[0].content.includes("Original prompt"));
assert.equal(result.messages.length, 2);
});
test("injectSystemPrompt: Claude body.system field", () => {
setSystemPromptConfig({ enabled: true, prompt: "GLOBAL:" });
const body = {
system: "Claude prompt",
messages: [{ role: "user", content: "hi" }],
};
const result = injectSystemPrompt(body);
assert.ok(result.system.startsWith("GLOBAL:"));
assert.ok(result.system.includes("Claude prompt"));
});
test("injectSystemPrompt: Claude array system field", () => {
setSystemPromptConfig({ enabled: true, prompt: "GLOBAL:" });
const body = {
system: [{ type: "text", text: "Claude prompt" }],
messages: [{ role: "user", content: "hi" }],
};
const result = injectSystemPrompt(body);
assert.ok(Array.isArray(result.system));
assert.equal(result.system[0].text, "GLOBAL:");
assert.equal(result.system.length, 2);
});
test("injectSystemPrompt: _skipSystemPrompt bypasses", () => {
setSystemPromptConfig({ enabled: true, prompt: "GLOBAL:" });
const body = {
_skipSystemPrompt: true,
messages: [{ role: "user", content: "hi" }],
};
const result = injectSystemPrompt(body);
assert.deepEqual(result, body);
});
test("injectSystemPrompt: with explicit promptText override", () => {
setSystemPromptConfig({ enabled: true, prompt: "default" });
const body = { messages: [{ role: "user", content: "hi" }] };
const result = injectSystemPrompt(body, "custom override");
assert.ok(result.messages[0].content.includes("custom override"));
});
test("injectSystemPrompt: null body returns as-is", () => {
setSystemPromptConfig({ enabled: true, prompt: "test" });
assert.equal(injectSystemPrompt(null), null);
});
// Reset
test.after(() => setSystemPromptConfig({ enabled: false, prompt: "" }));

View File

@@ -0,0 +1,161 @@
import test from "node:test";
import assert from "node:assert/strict";
const {
applyThinkingBudget,
setThinkingBudgetConfig,
getThinkingBudgetConfig,
ThinkingMode,
EFFORT_BUDGETS,
DEFAULT_THINKING_CONFIG,
} = await import("../../open-sse/services/thinkingBudget.js");
// ─── Config Management ──────────────────────────────────────────────────────
test("default config is passthrough", () => {
const config = getThinkingBudgetConfig();
assert.equal(config.mode, ThinkingMode.PASSTHROUGH);
});
test("setThinkingBudgetConfig updates config", () => {
setThinkingBudgetConfig({ mode: ThinkingMode.AUTO });
assert.equal(getThinkingBudgetConfig().mode, ThinkingMode.AUTO);
// Reset
setThinkingBudgetConfig(DEFAULT_THINKING_CONFIG);
});
// ─── PASSTHROUGH Mode ───────────────────────────────────────────────────────
test("PASSTHROUGH: body unchanged", () => {
setThinkingBudgetConfig({ mode: ThinkingMode.PASSTHROUGH });
const body = {
model: "claude-sonnet-4-20250514",
messages: [{ role: "user", content: "hello" }],
thinking: { type: "enabled", budget_tokens: 8192 },
};
const result = applyThinkingBudget(body);
assert.deepEqual(result, body);
setThinkingBudgetConfig(DEFAULT_THINKING_CONFIG);
});
// ─── AUTO Mode ──────────────────────────────────────────────────────────────
test("AUTO: strips Claude thinking config", () => {
setThinkingBudgetConfig({ mode: ThinkingMode.AUTO });
const body = {
model: "claude-sonnet-4-20250514",
messages: [{ role: "user", content: "hello" }],
thinking: { type: "enabled", budget_tokens: 8192 },
};
const result = applyThinkingBudget(body);
assert.equal(result.thinking, undefined);
setThinkingBudgetConfig(DEFAULT_THINKING_CONFIG);
});
test("AUTO: strips OpenAI reasoning_effort", () => {
setThinkingBudgetConfig({ mode: ThinkingMode.AUTO });
const body = {
model: "o3-mini",
messages: [{ role: "user", content: "hello" }],
reasoning_effort: "high",
};
const result = applyThinkingBudget(body);
assert.equal(result.reasoning_effort, undefined);
setThinkingBudgetConfig(DEFAULT_THINKING_CONFIG);
});
test("AUTO: strips Gemini thinking_config", () => {
setThinkingBudgetConfig({ mode: ThinkingMode.AUTO });
const body = {
model: "gemini-2.5-pro",
generationConfig: { thinking_config: { thinking_budget: 8192 } },
};
const result = applyThinkingBudget(body);
assert.equal(result.generationConfig.thinking_config, undefined);
setThinkingBudgetConfig(DEFAULT_THINKING_CONFIG);
});
// ─── CUSTOM Mode ────────────────────────────────────────────────────────────
test("CUSTOM: sets Claude budget", () => {
setThinkingBudgetConfig({ mode: ThinkingMode.CUSTOM, customBudget: 4096 });
const body = {
model: "claude-sonnet-4-20250514",
messages: [{ role: "user", content: "hello" }],
thinking: { type: "enabled", budget_tokens: 8192 },
};
const result = applyThinkingBudget(body);
assert.equal(result.thinking.budget_tokens, 4096);
setThinkingBudgetConfig(DEFAULT_THINKING_CONFIG);
});
test("CUSTOM: sets OpenAI reasoning_effort from budget", () => {
setThinkingBudgetConfig({ mode: ThinkingMode.CUSTOM, customBudget: 131072 });
const body = {
model: "o3-mini",
messages: [{ role: "user", content: "hello" }],
reasoning_effort: "low",
};
const result = applyThinkingBudget(body);
assert.equal(result.reasoning_effort, "high");
setThinkingBudgetConfig(DEFAULT_THINKING_CONFIG);
});
test("CUSTOM: budget 0 disables Claude thinking", () => {
setThinkingBudgetConfig({ mode: ThinkingMode.CUSTOM, customBudget: 0 });
const body = {
model: "claude-sonnet-4-20250514",
messages: [{ role: "user", content: "hello" }],
thinking: { type: "enabled", budget_tokens: 8192 },
};
const result = applyThinkingBudget(body);
assert.equal(result.thinking.type, "disabled");
setThinkingBudgetConfig(DEFAULT_THINKING_CONFIG);
});
// ─── ADAPTIVE Mode ──────────────────────────────────────────────────────────
test("ADAPTIVE: simple request gets base budget", () => {
setThinkingBudgetConfig({ mode: ThinkingMode.ADAPTIVE, effortLevel: "medium" });
const body = {
model: "claude-sonnet-4-20250514",
messages: [{ role: "user", content: "hello" }],
thinking: { type: "enabled", budget_tokens: 8192 },
};
const result = applyThinkingBudget(body);
assert.equal(result.thinking.budget_tokens, EFFORT_BUDGETS.medium);
setThinkingBudgetConfig(DEFAULT_THINKING_CONFIG);
});
test("ADAPTIVE: complex request (many messages + tools) gets higher budget", () => {
setThinkingBudgetConfig({ mode: ThinkingMode.ADAPTIVE, effortLevel: "medium" });
const messages = Array.from({ length: 15 }, (_, i) => ({
role: i % 2 === 0 ? "user" : "assistant",
content: "x".repeat(3000),
}));
const tools = Array.from({ length: 5 }, (_, i) => ({ name: `tool${i}` }));
const body = {
model: "claude-sonnet-4-20250514",
messages,
tools,
thinking: { type: "enabled", budget_tokens: 1000 },
};
const result = applyThinkingBudget(body);
// multiplier = 1.0 + 0.5 (msgs>10) + 0.5 (tools>3) + 0.3 (lastMsg>2000) = 2.3
assert.ok(result.thinking.budget_tokens > EFFORT_BUDGETS.medium);
setThinkingBudgetConfig(DEFAULT_THINKING_CONFIG);
});
// ─── Edge Cases ─────────────────────────────────────────────────────────────
test("null/undefined body returns as-is", () => {
assert.equal(applyThinkingBudget(null), null);
assert.equal(applyThinkingBudget(undefined), undefined);
});
test("EFFORT_BUDGETS has expected keys", () => {
assert.ok(EFFORT_BUDGETS.none === 0);
assert.ok(EFFORT_BUDGETS.low > 0);
assert.ok(EFFORT_BUDGETS.medium > EFFORT_BUDGETS.low);
assert.ok(EFFORT_BUDGETS.high > EFFORT_BUDGETS.medium);
});

View File

@@ -0,0 +1,102 @@
import test from "node:test";
import assert from "node:assert/strict";
const {
wildcardMatch,
getSpecificity,
resolveWildcardAlias,
resolveModel,
} = await import("../../open-sse/services/wildcardRouter.js");
// ─── wildcardMatch ──────────────────────────────────────────────────────────
test("wildcardMatch: exact match", () => {
assert.equal(wildcardMatch("gpt-4o", "gpt-4o"), true);
});
test("wildcardMatch: star catches all", () => {
assert.equal(wildcardMatch("anything", "*"), true);
});
test("wildcardMatch: prefix star", () => {
assert.equal(wildcardMatch("claude-sonnet-4-20250514", "claude-*"), true);
assert.equal(wildcardMatch("gpt-4o", "claude-*"), false);
});
test("wildcardMatch: middle star", () => {
assert.equal(wildcardMatch("claude-sonnet-4-20250514", "claude-*-4*"), true);
assert.equal(wildcardMatch("claude-haiku-4-20250514", "claude-*-4*"), true);
});
test("wildcardMatch: question mark single char", () => {
assert.equal(wildcardMatch("gpt-4o", "gpt-?o"), true);
assert.equal(wildcardMatch("gpt-4o", "gpt-??"), true);
assert.equal(wildcardMatch("gpt-4o", "gpt-???"), false);
});
test("wildcardMatch: case insensitive", () => {
assert.equal(wildcardMatch("GPT-4o", "gpt-*"), true);
});
test("wildcardMatch: null/empty", () => {
assert.equal(wildcardMatch(null, "*"), false);
assert.equal(wildcardMatch("model", null), false);
});
// ─── getSpecificity ─────────────────────────────────────────────────────────
test("getSpecificity: more specific > less specific", () => {
assert.ok(getSpecificity("claude-sonnet-4") > getSpecificity("claude-*"));
assert.ok(getSpecificity("claude-sonnet-*") > getSpecificity("claude-*"));
assert.ok(getSpecificity("*") < getSpecificity("claude-*"));
});
test("getSpecificity: exact > wildcard", () => {
assert.ok(getSpecificity("claude-sonnet-4-20250514") > getSpecificity("claude-sonnet-4-*"));
});
// ─── resolveWildcardAlias ───────────────────────────────────────────────────
test("resolveWildcardAlias: picks most specific match", () => {
const aliases = [
{ pattern: "claude-*", target: "generic-claude" },
{ pattern: "claude-sonnet-*", target: "sonnet-claude" },
{ pattern: "claude-sonnet-4-*", target: "sonnet4-claude" },
];
const result = resolveWildcardAlias("claude-sonnet-4-20250514", aliases);
assert.equal(result.target, "sonnet4-claude");
});
test("resolveWildcardAlias: returns null for no match", () => {
const aliases = [{ pattern: "gpt-*", target: "openai" }];
assert.equal(resolveWildcardAlias("claude-sonnet-4", aliases), null);
});
test("resolveWildcardAlias: handles empty/null", () => {
assert.equal(resolveWildcardAlias("model", null), null);
assert.equal(resolveWildcardAlias("model", []), null);
assert.equal(resolveWildcardAlias(null, []), null);
});
// ─── resolveModel ───────────────────────────────────────────────────────────
test("resolveModel: exact alias takes priority", () => {
const exact = { "my-claude": "claude-sonnet-4-20250514" };
const wildcards = [{ pattern: "my-*", target: "generic" }];
assert.equal(resolveModel("my-claude", exact, wildcards), "claude-sonnet-4-20250514");
});
test("resolveModel: falls back to wildcard", () => {
const exact = {};
const wildcards = [{ pattern: "my-*", target: "wildcard-match" }];
assert.equal(resolveModel("my-model", exact, wildcards), "wildcard-match");
});
test("resolveModel: returns original if no match", () => {
assert.equal(resolveModel("unknown", {}, []), "unknown");
});
test("resolveModel: works with Map for exact aliases", () => {
const exact = new Map([["alias1", "target1"]]);
assert.equal(resolveModel("alias1", exact, []), "target1");
});

View File

@@ -34,5 +34,5 @@
".next/types/**/*.ts",
".next/dev/types/**/*.ts"
],
"exclude": ["node_modules", "open-sse"]
"exclude": ["node_modules", "open-sse", "antigravity-manager-analysis"]
}