feat(release): v1.7.3 — model deprecation, background degradation, rate limit persistence, thinking improvements, circuit breaker

Features:
- Model Deprecation Auto-Forward (10+ built-in aliases + custom via UI)
- Background Task Smart Degradation (19 patterns, degradation map)
- Rate Limit Persistence (SQLite, 60s debounce, 24h staleness)
- thinkingLevel string → budget conversion (high/medium/low/none)
- Claude -thinking model auto-injection
- Gemini 3.0/3.1 model registry distinction
- Token Refresh Circuit Breaker (5 failures → 30min cooldown)

Tests: 561 total (40+ new), 0 failures
This commit is contained in:
diegosouzapw
2026-03-01 21:42:39 -03:00
parent dc90211222
commit 848cc1415f
19 changed files with 1638 additions and 49 deletions

View File

@@ -7,6 +7,37 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
---
## [1.7.3] — 2026-03-01
### ✨ New Features
- **Model Deprecation Auto-Forward** — New `modelDeprecation.ts` service with 10+ built-in aliases for legacy Gemini, Claude, and OpenAI models. Deprecated model IDs (e.g., `gemini-pro`, `claude-2`) are automatically forwarded to their current replacements. Custom aliases configurable via new Settings → Routing → Model Aliases UI tab with full CRUD API (`/api/settings/model-aliases`)
- **Background Task Smart Degradation** — New `backgroundTaskDetector.ts` service detects background/utility requests (title generation, summarization, etc.) via 19 system prompt patterns and `X-Request-Priority` header, and automatically reroutes them to cheaper models. Configurable degradation map and detection patterns via new Settings → Routing → Background Degradation UI tab. Disabled by default (opt-in)
- **Rate Limit Persistence** — Learned rate limits from API response headers are now persisted to SQLite with 60-second debouncing and restored on startup (24h staleness filter). Rate limits survive server restarts instead of being lost in memory
- **thinkingLevel String Conversion** — `applyThinkingBudget()` now handles string-based `thinkingLevel` inputs (`"high"`, `"medium"`, `"low"`, `"none"`) by converting them to numeric token budgets. Supports `thinkingLevel`, `thinking_level`, and Gemini's `generationConfig.thinkingConfig.thinkingLevel` fields
- **Claude -thinking Model Auto-Injection** — Models ending with `-thinking` suffix (e.g., `claude-opus-4-6-thinking`) automatically get thinking parameters injected to prevent API errors. `hasThinkingCapableModel()` updated to recognize these suffixes
- **Gemini 3.0/3.1 Model Registry** — Updated provider registry to explicitly distinguish Gemini 3.1 (Pro, Flash) from 3.0 Preview variants across `gemini`, `gemini-cli`, and `antigravity` providers with clear naming conventions
- **Token Refresh Circuit Breaker** — Per-provider circuit breaker in `refreshWithRetry()`: 5 consecutive failures trigger a 30-minute cooldown to prevent infinite retry loops. Added 30-second timeout wrapper per refresh attempt. Exported `isProviderBlocked()` and `getCircuitBreakerStatus()` for diagnostics
### 🧪 Tests
- **40+ new unit tests** across 3 files: `model-deprecation.test.mjs` (14 tests), `background-task-detector.test.mjs` (14 tests), extended `thinking-budget.test.mjs` (+13 tests). Total suite: **561 tests, 0 failures**
### 📁 New Files
| File | Purpose |
| ---------------------------------------------------------------------- | --------------------------------------------------------------------- |
| `open-sse/services/modelDeprecation.ts` | Model deprecation alias resolver with built-in + custom aliases |
| `open-sse/services/backgroundTaskDetector.ts` | Background task detection with pattern matching and model degradation |
| `src/app/api/settings/model-aliases/route.ts` | CRUD API for model alias management |
| `src/app/api/settings/background-degradation/route.ts` | API for background degradation config |
| `src/app/(dashboard)/settings/components/ModelAliasesTab.tsx` | Settings UI for model alias management |
| `src/app/(dashboard)/settings/components/BackgroundDegradationTab.tsx` | Settings UI for background degradation |
| `tests/unit/model-deprecation.test.mjs` | 14 unit tests for model deprecation |
| `tests/unit/background-task-detector.test.mjs` | 14 unit tests for background task detection |
---
## [1.7.2] — 2026-03-01
### ✨ New Features

View File

@@ -607,6 +607,8 @@ When minimized, OmniRoute lives in your system tray with quick actions:
| 🧩 **Custom Models** | Add any model ID to any provider |
| 🌐 **Wildcard Router** | Route `provider/*` patterns to any provider dynamically |
| 🧠 **Thinking Budget** | Passthrough, auto, custom, and adaptive modes for reasoning models |
| 🔀 **Model Aliases** | Auto-forward deprecated model IDs to current replacements (built-in + custom) |
| ⚡ **Background Degradation** | Auto-route background tasks (titles, summaries) to cheaper models |
| 💬 **System Prompt Injection** | Global system prompt applied across all requests |
| 📄 **Responses API** | Full OpenAI Responses API (`/v1/responses`) support for Codex |
@@ -634,6 +636,8 @@ When minimized, OmniRoute lives in your system tray with quick actions:
| 🔒 **TLS Fingerprint Spoofing** | Bypass TLS-based bot detection via wreq-js |
| 🌐 **IP Filtering** | Allowlist/blocklist for API access control |
| 📊 **Editable Rate Limits** | Configurable RPM, min gap, and max concurrent at system level |
| 💾 **Rate Limit Persistence** | Learned limits survive restarts via SQLite with 60s debounce + 24h staleness |
| 🔄 **Token Refresh Resilience** | Per-provider circuit breaker (5 fails→30min) + 30s timeout per attempt |
| 🛡 **API Endpoint Protection** | Auth gating + provider blocking for the `/models` endpoint |
| 🔒 **Proxy Visibility** | Color-coded badges: 🟢 global, 🟡 provider, 🔵 per-connection with IP display |
| 🌐 **3-Level Proxy Config** | Configure proxies at global, per-provider, or per-connection level |

View File

@@ -574,19 +574,21 @@ Quando minimizado, o OmniRoute fica na bandeja do sistema com ações rápidas:
### 🧠 Roteamento e Inteligência
| Funcionalidade | O que Faz |
| ----------------------------------------- | ------------------------------------------------------------------------------- |
| 🎯 **Fallback Inteligente 4 Tiers** | Auto-roteamento: Assinatura → API Key → Barato → Gratuito |
| 📊 **Rastreamento de Cota em Tempo Real** | Contagem de tokens ao vivo + countdown de reset por provedor |
| 🔄 **Tradução de Formato** | OpenAI ↔ Claude ↔ Gemini ↔ Cursor ↔ Kiro transparente |
| 👥 **Suporte Multi-Conta** | Múltiplas contas por provedor com seleção inteligente |
| 🔄 **Renovação Automática de Token** | Tokens OAuth renovam automaticamente com retry |
| 🎨 **Combos Personalizados** | 6 estratégias: fill-first, round-robin, p2c, random, least-used, cost-optimized |
| 🧩 **Modelos Personalizados** | Adicione qualquer ID de modelo a qualquer provedor |
| 🌐 **Roteador Wildcard** | Roteie padrões `provider/*` para qualquer provedor dinamicamente |
| 🧠 **Budget de Raciocínio** | Modos passthrough, auto, custom e adaptativo para modelos de raciocínio |
| 💬 **Injeção de System Prompt** | System prompt global aplicado em todas as requisições |
| 📄 **API Responses** | Suporte completo à API Responses da OpenAI (`/v1/responses`) para Codex |
| Funcionalidade | O que Faz |
| ----------------------------------------- | ---------------------------------------------------------------------------------- |
| 🎯 **Fallback Inteligente 4 Tiers** | Auto-roteamento: Assinatura → API Key → Barato → Gratuito |
| 📊 **Rastreamento de Cota em Tempo Real** | Contagem de tokens ao vivo + countdown de reset por provedor |
| 🔄 **Tradução de Formato** | OpenAI ↔ Claude ↔ Gemini ↔ Cursor ↔ Kiro transparente |
| 👥 **Suporte Multi-Conta** | Múltiplas contas por provedor com seleção inteligente |
| 🔄 **Renovação Automática de Token** | Tokens OAuth renovam automaticamente com retry |
| 🎨 **Combos Personalizados** | 6 estratégias: fill-first, round-robin, p2c, random, least-used, cost-optimized |
| 🧩 **Modelos Personalizados** | Adicione qualquer ID de modelo a qualquer provedor |
| 🌐 **Roteador Wildcard** | Roteie padrões `provider/*` para qualquer provedor dinamicamente |
| 🧠 **Budget de Raciocínio** | Modos passthrough, auto, custom e adaptativo para modelos de raciocínio |
| <EFBFBD> **Aliases de Modelo** | Redireciona IDs de modelos depreciados para substitutos atuais (built-in + custom) |
| **Degradação em Background** | Redireciona tarefas em background (títulos, resumos) para modelos mais baratos |
| <20>💬 **Injeção de System Prompt** | System prompt global aplicado em todas as requisições |
| 📄 **API Responses** | Suporte completo à API Responses da OpenAI (`/v1/responses`) para Codex |
### 🎵 APIs Multi-Modal
@@ -603,18 +605,20 @@ Quando minimizado, o OmniRoute fica na bandeja do sistema com ações rápidas:
### 🛡️ Resiliência e Segurança
| Funcionalidade | O que Faz |
| ---------------------------------- | --------------------------------------------------------------------------- |
| 🔌 **Circuit Breaker** | Auto-abertura/fechamento por provedor com limites configuráveis |
| 🛡️ **Anti-Thundering Herd** | Mutex + semáforo rate-limit para provedores com API key |
| 🧠 **Cache Semântico** | Cache de duas camadas (assinatura + semântico) reduz custo e latência |
| ⚡ **Idempotência de Requisição** | Janela de dedup de 5s para requisições duplicadas |
| 🔒 **Spoofing de Fingerprint TLS** | Bypass de detecção de bot via TLS com wreq-js |
| 🌐 **Filtragem de IP** | Allowlist/blocklist para controle de acesso à API |
| 📊 **Rate Limits Editáveis** | RPM, gap mínimo e concorrência máxima configuráveis |
| 🛡 **Proteção de Endpoint API** | Gateway de Auth + bloqueio de provedores para o endpoint `/models` |
| 🔒 **Visibilidade de Proxy** | Badges coloridos: 🟢 global, 🟡 provedor, 🔵 por-conexão com exibição de IP |
| 🌐 **Proxy em 3 Níveis** | Configure proxies em nível global, por provedor ou por conexão |
| Funcionalidade | O que Faz |
| ----------------------------------- | ----------------------------------------------------------------------------- |
| 🔌 **Circuit Breaker** | Auto-abertura/fechamento por provedor com limites configuráveis |
| 🛡️ **Anti-Thundering Herd** | Mutex + semáforo rate-limit para provedores com API key |
| 🧠 **Cache Semântico** | Cache de duas camadas (assinatura + semântico) reduz custo e latência |
| ⚡ **Idempotência de Requisição** | Janela de dedup de 5s para requisições duplicadas |
| 🔒 **Spoofing de Fingerprint TLS** | Bypass de detecção de bot via TLS com wreq-js |
| 🌐 **Filtragem de IP** | Allowlist/blocklist para controle de acesso à API |
| 📊 **Rate Limits Editáveis** | RPM, gap mínimo e concorrência máxima configuráveis |
| 💾 **Persistência de Rate Limits** | Limites aprendidos persistem via SQLite com debounce de 60s + 24h de validade |
| 🔄 **Resiliência de Token Refresh** | Circuit breaker por provedor (5 falhas→30min) + timeout de 30s por tentativa |
| 🛡 **Proteção de Endpoint API** | Gateway de Auth + bloqueio de provedores para o endpoint `/models` |
| 🔒 **Visibilidade de Proxy** | Badges coloridos: 🟢 global, 🟡 provedor, 🔵 por-conexão com exibição de IP |
| 🌐 **Proxy em 3 Níveis** | Configure proxies em nível global, por provedor ou por conexão |
### 📊 Observabilidade e Analytics

View File

@@ -48,7 +48,7 @@ Four modes for debugging API translations: **Playground** (format converter), **
## ⚙️ Settings
General settings, system storage, backup management (export/import database), appearance (dark/light mode), security (includes API endpoint protection and custom provider blocking), routing, resilience, and advanced configuration.
General settings, system storage, backup management (export/import database), appearance (dark/light mode), security (includes API endpoint protection and custom provider blocking), routing (model aliases, background task degradation), resilience (rate limit persistence), and advanced configuration.
![Settings Dashboard](screenshots/06-settings.png)

View File

@@ -111,7 +111,10 @@ export const REGISTRY: Record<string, RegistryEntry> = {
clientSecretDefault: "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",
},
models: [
{ id: "gemini-3-pro-preview", name: "Gemini 3 Pro Preview" },
{ id: "gemini-3.1-pro", name: "Gemini 3.1 Pro" },
{ id: "gemini-3.1-flash", name: "Gemini 3.1 Flash" },
{ id: "gemini-3-pro-preview", name: "Gemini 3.0 Pro Preview" },
{ id: "gemini-3-flash-preview", name: "Gemini 3.0 Flash Preview" },
{ id: "gemini-2.5-pro", name: "Gemini 2.5 Pro" },
{ id: "gemini-2.5-flash", name: "Gemini 2.5 Flash" },
{ id: "gemini-2.5-flash-lite", name: "Gemini 2.5 Flash Lite" },
@@ -137,8 +140,10 @@ export const REGISTRY: Record<string, RegistryEntry> = {
clientSecretDefault: "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",
},
models: [
{ id: "gemini-3-flash-preview", name: "Gemini 3 Flash Preview" },
{ id: "gemini-3-pro-preview", name: "Gemini 3 Pro Preview" },
{ id: "gemini-3.1-pro", name: "Gemini 3.1 Pro" },
{ id: "gemini-3.1-flash", name: "Gemini 3.1 Flash" },
{ id: "gemini-3-flash-preview", name: "Gemini 3.0 Flash Preview" },
{ id: "gemini-3-pro-preview", name: "Gemini 3.0 Pro Preview" },
{ id: "gemini-2.5-pro", name: "Gemini 2.5 Pro" },
{ id: "gemini-2.5-flash", name: "Gemini 2.5 Flash" },
{ id: "gemini-2.5-flash-lite", name: "Gemini 2.5 Flash Lite" },
@@ -268,7 +273,8 @@ export const REGISTRY: Record<string, RegistryEntry> = {
{ id: "claude-sonnet-4-6", name: "Claude Sonnet 4.6" },
{ id: "gemini-3.1-pro-high", name: "Gemini 3.1 Pro High" },
{ id: "gemini-3.1-pro-low", name: "Gemini 3.1 Pro Low" },
{ id: "gemini-3-flash", name: "Gemini 3 Flash" },
{ id: "gemini-3.1-flash", name: "Gemini 3.1 Flash" },
{ id: "gemini-3-flash", name: "Gemini 3.0 Flash" },
{ id: "gpt-oss-120b-medium", name: "GPT OSS 120B Medium" },
],
},

View File

@@ -0,0 +1,187 @@
/**
* Background Task Detector — Feature 3
*
* Detects when CLI tools send "background" requests (title generation,
* summarization, short descriptions) and provides model degradation
* recommendations to save premium model quota.
*
* Detection heuristics:
* - System prompt patterns indicating background/utility tasks
* - Very short conversations with summary-like system prompts
* - X-Request-Priority header
*/
// ── Configuration ───────────────────────────────────────────────────────────
interface DegradationConfig {
enabled: boolean;
degradationMap: Record<string, string>; // original → cheaper model
detectionPatterns: string[]; // regex patterns for system prompt matching
stats: {
detected: number;
tokensSaved: number;
};
}
const DEFAULT_DETECTION_PATTERNS = [
"generate a title",
"generate title",
"create a title",
"create a short",
"summarize this",
"summarize the",
"write a brief",
"write a summary",
"one-line summary",
"one line summary",
"short description",
"brief description",
"conversation title",
"chat title",
"name this conversation",
"name this chat",
"title for this",
"suggest a title",
"label this",
];
const DEFAULT_DEGRADATION_MAP: Record<string, string> = {
// Premium → Cheap alternatives
"claude-opus-4-6": "gemini-2.5-flash",
"claude-opus-4-6-thinking": "gemini-2.5-flash",
"claude-opus-4-5-20251101": "gemini-2.5-flash",
"claude-sonnet-4-5-20250929": "gemini-2.5-flash",
"claude-sonnet-4-20250514": "gemini-2.5-flash",
"claude-sonnet-4": "gemini-2.5-flash",
"gemini-3.1-pro": "gemini-3.1-flash",
"gemini-3.1-pro-high": "gemini-3.1-flash",
"gemini-3-pro-preview": "gemini-3-flash-preview",
"gemini-2.5-pro": "gemini-2.5-flash",
"gpt-4o": "gpt-4o-mini",
"gpt-5": "gpt-5-mini",
"gpt-5.1": "gpt-5-mini",
"gpt-5.1-codex": "gpt-5.1-codex-mini",
};
// ── State ───────────────────────────────────────────────────────────────────
let _config: DegradationConfig = {
enabled: false, // Disabled by default — user must opt in
degradationMap: { ...DEFAULT_DEGRADATION_MAP },
detectionPatterns: [...DEFAULT_DETECTION_PATTERNS],
stats: { detected: 0, tokensSaved: 0 },
};
// ── Config Management ───────────────────────────────────────────────────────
/**
* Set the background degradation config (called from settings API or startup).
*/
export function setBackgroundDegradationConfig(config: Partial<DegradationConfig>): void {
_config = {
..._config,
...config,
stats: _config.stats, // preserve stats across config changes
};
}
/**
* Get current background degradation config.
*/
export function getBackgroundDegradationConfig(): DegradationConfig {
return {
..._config,
degradationMap: { ..._config.degradationMap },
detectionPatterns: [..._config.detectionPatterns],
stats: { ..._config.stats },
};
}
/**
* Reset stats counters.
*/
export function resetStats(): void {
_config.stats = { detected: 0, tokensSaved: 0 };
}
// ── Detection ───────────────────────────────────────────────────────────────
/**
* Check if a request is a background/utility task.
*
* @param {object} body - Request body
* @param {object} [headers] - Request headers (optional)
* @returns {boolean} True if the request looks like a background task
*/
export function isBackgroundTask(
body: any,
headers: Record<string, string> | null = null
): boolean {
if (!body || typeof body !== "object") return false;
// 1. Check explicit header
if (headers) {
const priority =
headers["x-request-priority"] || headers["X-Request-Priority"] || headers["x-initiator"];
if (priority === "background" || priority === "Background") return true;
}
// 2. Check system prompt for background task patterns
const messages = body.messages || body.input || [];
if (!Array.isArray(messages) || messages.length === 0) return false;
// Find system message
const systemMsg = messages.find((m: any) => m.role === "system" || m.role === "developer");
if (!systemMsg) return false;
const systemContent =
typeof systemMsg.content === "string" ? systemMsg.content.toLowerCase() : "";
if (!systemContent) return false;
// Check against detection patterns
const matched = _config.detectionPatterns.some((pattern) =>
systemContent.includes(pattern.toLowerCase())
);
if (!matched) return false;
// 3. Additional heuristic: background tasks typically have very few messages
// (system + 1-2 user messages)
const userMessages = messages.filter((m: any) => m.role === "user");
if (userMessages.length > 3) return false; // Too many turns for a background task
return true;
}
/**
* Get the degraded (cheaper) model for a given model.
*
* @param {string} originalModel - The original model ID
* @returns {string} The cheaper model or original if no mapping exists
*/
export function getDegradedModel(originalModel: string): string {
if (!originalModel) return originalModel;
const degraded = _config.degradationMap[originalModel];
if (degraded) {
_config.stats.detected++;
return degraded;
}
return originalModel;
}
/**
* Get default degradation map (for UI reset).
*/
export function getDefaultDegradationMap(): Record<string, string> {
return { ...DEFAULT_DEGRADATION_MAP };
}
/**
* Get default detection patterns (for UI reset).
*/
export function getDefaultDetectionPatterns(): string[] {
return [...DEFAULT_DETECTION_PATTERNS];
}

View File

@@ -0,0 +1,125 @@
/**
* Model Deprecation Auto-Forward — Feature 2
*
* Maps deprecated model IDs to their replacements so user configs
* don't break when providers rename or retire models.
*
* Supports both built-in aliases (static) and custom aliases (persisted via Settings API).
*/
// ── Built-in Deprecation Aliases ────────────────────────────────────────────
// These are known renames/retirements across providers.
// Format: deprecated ID → current ID
const BUILT_IN_ALIASES: Record<string, string> = {
// Gemini legacy → current
"gemini-pro": "gemini-2.5-pro",
"gemini-pro-vision": "gemini-2.5-pro",
"gemini-1.5-pro": "gemini-2.5-pro",
"gemini-1.5-flash": "gemini-2.5-flash",
"gemini-1.0-pro": "gemini-2.5-pro",
"gemini-2.0-flash": "gemini-2.5-flash",
// Claude legacy → current
"claude-3-opus-20240229": "claude-opus-4-20250514",
"claude-3-sonnet-20240229": "claude-sonnet-4-20250514",
"claude-3-haiku-20240307": "claude-3-5-sonnet-20241022",
"claude-3-5-sonnet-latest": "claude-sonnet-4-20250514",
"claude-3-5-haiku-latest": "claude-3-5-sonnet-20241022",
// OpenAI legacy → current
"gpt-4-turbo-preview": "gpt-4-turbo",
"gpt-4-0125-preview": "gpt-4-turbo",
"gpt-4-1106-preview": "gpt-4-turbo",
"gpt-3.5-turbo-0125": "gpt-3.5-turbo",
};
// ── Custom Aliases (persisted via Settings API) ─────────────────────────────
let _customAliases: Record<string, string> = {};
/**
* Set custom aliases (called from settings API or startup).
*/
export function setCustomAliases(aliases: Record<string, string>): void {
_customAliases = { ...aliases };
}
/**
* Get current custom aliases.
*/
export function getCustomAliases(): Record<string, string> {
return { ..._customAliases };
}
/**
* Get the full alias map (built-in + custom).
* Custom aliases take precedence over built-in.
*/
export function getAllAliases(): Record<string, string> {
return { ...BUILT_IN_ALIASES, ..._customAliases };
}
/**
* Resolve a model alias to its current ID.
* Custom aliases override built-in ones.
*
* @param {string} modelId - The model ID to resolve
* @returns {string} The resolved model ID, or the original if not deprecated
*/
export function resolveModelAlias(modelId: string): string {
if (!modelId) return modelId;
// Check custom aliases first (higher priority)
if (_customAliases[modelId]) return _customAliases[modelId];
// Then check built-in
if (BUILT_IN_ALIASES[modelId]) return BUILT_IN_ALIASES[modelId];
return modelId;
}
/**
* Get a deprecation notice if the model is deprecated.
*
* @param {string} modelId - The model ID to check
* @returns {string | null} Deprecation message or null if not deprecated
*/
export function getDeprecationNotice(modelId: string): string | null {
if (!modelId) return null;
const resolved = resolveModelAlias(modelId);
if (resolved === modelId) return null;
return `Model "${modelId}" is deprecated. Forwarding to "${resolved}".`;
}
/**
* Check if a model is deprecated.
*/
export function isDeprecated(modelId: string): boolean {
return getDeprecationNotice(modelId) !== null;
}
/**
* Add a custom alias.
*/
export function addCustomAlias(from: string, to: string): void {
_customAliases[from] = to;
}
/**
* Remove a custom alias.
*/
export function removeCustomAlias(from: string): boolean {
if (_customAliases[from]) {
delete _customAliases[from];
return true;
}
return false;
}
/**
* Get the built-in aliases (read-only reference).
*/
export function getBuiltInAliases(): Record<string, string> {
return { ...BUILT_IN_ALIASES };
}

View File

@@ -19,6 +19,11 @@ const limiters = new Map();
// Store connections that have rate limit protection enabled
const enabledConnections = new Set();
// Store learned limits for persistence (debounced)
const learnedLimits: Record<string, any> = {};
let persistTimer: ReturnType<typeof setTimeout> | null = null;
const PERSIST_DEBOUNCE_MS = 60_000; // Debounce persistence to every 60s max
// Track initialization
let initialized = false;
@@ -82,6 +87,9 @@ export async function initializeRateLimits() {
`🛡️ [RATE-LIMIT] Loaded ${explicitCount} explicit + ${autoCount} auto-enabled (API key) protection(s)`
);
}
// Load persisted learned limits
await loadPersistedLimits();
} catch (err) {
console.error("[RATE-LIMIT] Failed to load settings:", err.message);
}
@@ -314,6 +322,9 @@ export function updateFromHeaders(provider, connectionId, headers, status, model
}
limiter.updateSettings(updates);
// Persist learned limits (debounced)
recordLearnedLimit(provider, connectionId, { limit, remaining, minTime: updates.minTime });
}
}
@@ -360,6 +371,82 @@ export function getAllRateLimitStatus() {
return result;
}
/**
* Get all learned limits (for dashboard display).
*/
export function getLearnedLimits() {
return { ...learnedLimits };
}
// ─── Persistence ────────────────────────────────────────────────────────────
/**
* Record a learned limit for debounced persistence.
*/
function recordLearnedLimit(provider: string, connectionId: string, limits: any) {
const key = `${provider}:${connectionId}`;
learnedLimits[key] = {
...limits,
provider,
connectionId,
lastUpdated: Date.now(),
};
// Debounce: save at most once per PERSIST_DEBOUNCE_MS
if (!persistTimer) {
persistTimer = setTimeout(async () => {
persistTimer = null;
try {
const { updateSettings } = await import("@/lib/db/settings");
await updateSettings({ learnedRateLimits: JSON.stringify(learnedLimits) });
console.log(
`💾 [RATE-LIMIT] Persisted learned limits for ${Object.keys(learnedLimits).length} provider(s)`
);
} catch (err) {
console.error("[RATE-LIMIT] Failed to persist learned limits:", err.message);
}
}, PERSIST_DEBOUNCE_MS);
}
}
/**
* Load persisted learned limits on startup.
*/
async function loadPersistedLimits() {
try {
const { getSettings } = await import("@/lib/db/settings");
const settings = await getSettings();
const raw = settings?.learnedRateLimits;
if (!raw) return;
const parsed = JSON.parse(raw);
let count = 0;
for (const [key, data] of Object.entries<any>(parsed)) {
// Skip stale entries (older than 24h)
if (data.lastUpdated && Date.now() - data.lastUpdated > 24 * 60 * 60 * 1000) continue;
learnedLimits[key] = data;
// Apply to limiter if it exists and has rate limit enabled
if (data.connectionId && enabledConnections.has(data.connectionId)) {
const limiter = limiters.get(key);
if (limiter && data.limit) {
const minTime = data.minTime || Math.max(0, Math.floor(60000 / data.limit) - 10);
limiter.updateSettings({ minTime });
count++;
}
}
}
if (count > 0) {
console.log(`📥 [RATE-LIMIT] Restored ${count} learned rate limit(s) from persistence`);
}
} catch (err) {
console.error("[RATE-LIMIT] Failed to load persisted limits:", err.message);
}
}
/**
* Update rate limiter based on API response body (JSON error responses).
* Providers embed retry info in JSON payloads in different formats.

View File

@@ -21,6 +21,15 @@ export const EFFORT_BUDGETS = {
high: 131072,
};
// thinkingLevel string → budget token mapping
// Used when clients send string-based thinking levels (e.g., VS Code Copilot)
export const THINKING_LEVEL_MAP = {
none: 0,
low: 1024,
medium: 10240,
high: 131072,
};
// Default config (passthrough = backward compatible)
export const DEFAULT_THINKING_CONFIG = {
mode: ThinkingMode.PASSTHROUGH,
@@ -45,10 +54,81 @@ export function getThinkingBudgetConfig() {
return { ..._config };
}
/**
* Normalize thinkingLevel string fields into numeric budget.
* Handles: body.thinkingLevel, body.thinking_level,
* and Gemini's generationConfig.thinkingConfig.thinkingLevel
*
* @param {object} body - Request body
* @returns {object} Body with string thinkingLevel converted to numeric budget
*/
export function normalizeThinkingLevel(body) {
if (!body || typeof body !== "object") return body;
const result = { ...body };
// Handle top-level thinkingLevel or thinking_level string fields
const levelStr = result.thinkingLevel || result.thinking_level;
if (typeof levelStr === "string" && THINKING_LEVEL_MAP[levelStr] !== undefined) {
const budget = THINKING_LEVEL_MAP[levelStr];
// Convert to Claude thinking format as canonical representation
result.thinking = {
type: budget > 0 ? "enabled" : "disabled",
budget_tokens: budget,
};
delete result.thinkingLevel;
delete result.thinking_level;
}
// Handle Gemini's generationConfig.thinkingConfig.thinkingLevel
const geminiLevel =
result.generationConfig?.thinkingConfig?.thinkingLevel ||
result.generationConfig?.thinking_config?.thinkingLevel;
if (typeof geminiLevel === "string" && THINKING_LEVEL_MAP[geminiLevel] !== undefined) {
const budget = THINKING_LEVEL_MAP[geminiLevel];
result.generationConfig = {
...result.generationConfig,
thinking_config: { thinking_budget: budget },
};
// Clean up camelCase variant if it was the source
if (result.generationConfig.thinkingConfig) {
delete result.generationConfig.thinkingConfig;
}
}
return result;
}
/**
* Ensure models with -thinking suffix have thinking config injected.
* Prevents 400 errors from Claude API when thinking params are missing.
*
* @param {object} body - Request body
* @returns {object} Body with thinking config auto-injected if needed
*/
export function ensureThinkingConfig(body) {
if (!body || typeof body !== "object") return body;
const model = body.model || "";
// Only auto-inject for models with -thinking suffix
if (!model.endsWith("-thinking")) return body;
// If thinking config already present, don't override
if (body.thinking) return body;
const result = { ...body };
result.thinking = {
type: "enabled",
budget_tokens: EFFORT_BUDGETS.medium, // 10240 default
};
return result;
}
/**
* Apply thinking budget control to a request body.
* Called before format-specific translation.
*
* Pipeline: normalizeThinkingLevel → ensureThinkingConfig → mode processing
*
* @param {object} body - Request body (any format)
* @param {object} [config] - Override config (defaults to stored config)
* @returns {object} Modified body
@@ -57,21 +137,27 @@ export function applyThinkingBudget(body, config = null) {
const cfg = config || _config;
if (!body || typeof body !== "object") return body;
// Pre-processing: convert string thinkingLevel to numeric budget
let processed = normalizeThinkingLevel(body);
// Pre-processing: auto-inject thinking config for -thinking suffix models
processed = ensureThinkingConfig(processed);
switch (cfg.mode) {
case ThinkingMode.AUTO:
return stripThinkingConfig(body);
return stripThinkingConfig(processed);
case ThinkingMode.PASSTHROUGH:
return body; // No changes
return processed;
case ThinkingMode.CUSTOM:
return setCustomBudget(body, cfg.customBudget);
return setCustomBudget(processed, cfg.customBudget);
case ThinkingMode.ADAPTIVE:
return applyAdaptiveBudget(body, cfg);
return applyAdaptiveBudget(processed, cfg);
default:
return body;
return processed;
}
}
@@ -151,9 +237,10 @@ function applyAdaptiveBudget(body, cfg) {
for (let i = messages.length - 1; i >= 0; i--) {
const msg = messages[i];
if (msg.role === "user") {
lastMsgLength = typeof msg.content === "string"
? msg.content.length
: JSON.stringify(msg.content || "").length;
lastMsgLength =
typeof msg.content === "string"
? msg.content.length
: JSON.stringify(msg.content || "").length;
break;
}
}
@@ -173,7 +260,7 @@ function applyAdaptiveBudget(body, cfg) {
/**
* Check if model name suggests thinking capability
*/
function hasThinkingCapableModel(body) {
export function hasThinkingCapableModel(body) {
const model = body.model || "";
return (
model.includes("claude") ||
@@ -181,6 +268,7 @@ function hasThinkingCapableModel(body) {
model.includes("o3") ||
model.includes("o4") ||
model.includes("gemini") ||
model.endsWith("-thinking") ||
model.includes("thinking")
);
}

View File

@@ -323,9 +323,13 @@ export async function refreshQwenToken(refreshToken, log) {
}
if (errorCode === "invalid_request") {
log?.error?.("TOKEN_REFRESH", "Qwen refresh token is invalid or expired. Re-authentication required.", {
status: response.status,
});
log?.error?.(
"TOKEN_REFRESH",
"Qwen refresh token is invalid or expired. Re-authentication required.",
{
status: response.status,
}
);
return { error: "invalid_request" };
}
@@ -720,8 +724,11 @@ export function supportsTokenRefresh(provider) {
* Callers should stop retrying and request re-authentication.
*/
export function isUnrecoverableRefreshError(result) {
return result && typeof result === "object" &&
(result.error === "refresh_token_reused" || result.error === "invalid_request");
return (
result &&
typeof result === "object" &&
(result.error === "refresh_token_reused" || result.error === "invalid_request")
);
}
/**
@@ -841,12 +848,103 @@ export async function getAllAccessTokens(userInfo, log) {
/**
* Refresh token with retry and exponential backoff
* Retries on failure with increasing delay: 1s, 2s, 3s...
*
* Includes:
* - Per-provider circuit breaker (5 consecutive failures → 30min pause)
* - 30s timeout per refresh attempt to prevent hanging connections
*
* @param {function} refreshFn - Async function that returns token or null
* @param {number} maxRetries - Max retry attempts (default 3)
* @param {object} log - Logger instance (optional)
* @param {string} provider - Provider ID for circuit breaker tracking (optional)
* @returns {Promise<object|null>} Token result or null if all retries fail
*/
export async function refreshWithRetry(refreshFn, maxRetries = 3, log = null) {
// ─── Circuit Breaker State ──────────────────────────────────────────────────
const _circuitBreaker: Record<string, { failures: number; blockedUntil: number }> = {};
const CIRCUIT_BREAKER_THRESHOLD = 5; // consecutive failures before tripping
const CIRCUIT_BREAKER_COOLDOWN = 30 * 60 * 1000; // 30 minutes
const REFRESH_TIMEOUT_MS = 30_000; // 30s max per refresh attempt
/**
* Check if a provider is circuit-breaker blocked.
*/
export function isProviderBlocked(provider: string): boolean {
const state = _circuitBreaker[provider];
if (!state) return false;
if (state.blockedUntil > Date.now()) return true;
// Cooldown expired — reset
delete _circuitBreaker[provider];
return false;
}
/**
* Get circuit breaker status for all providers (for diagnostics).
*/
export function getCircuitBreakerStatus(): Record<string, any> {
const result: Record<string, any> = {};
for (const [provider, state] of Object.entries(_circuitBreaker)) {
result[provider] = {
failures: state.failures,
blocked: state.blockedUntil > Date.now(),
blockedUntil:
state.blockedUntil > Date.now() ? new Date(state.blockedUntil).toISOString() : null,
remainingMs: Math.max(0, state.blockedUntil - Date.now()),
};
}
return result;
}
/**
* Record a successful refresh — resets circuit breaker for provider.
*/
function recordSuccess(provider: string) {
if (_circuitBreaker[provider]) {
delete _circuitBreaker[provider];
}
}
/**
* Record a failed refresh — increments circuit breaker counter.
*/
function recordFailure(provider: string, log: any = null) {
if (!_circuitBreaker[provider]) {
_circuitBreaker[provider] = { failures: 0, blockedUntil: 0 };
}
_circuitBreaker[provider].failures++;
if (_circuitBreaker[provider].failures >= CIRCUIT_BREAKER_THRESHOLD) {
_circuitBreaker[provider].blockedUntil = Date.now() + CIRCUIT_BREAKER_COOLDOWN;
log?.error?.(
"TOKEN_REFRESH",
`🔴 Circuit breaker tripped for ${provider}: ${CIRCUIT_BREAKER_THRESHOLD} consecutive failures. ` +
`Blocked for ${CIRCUIT_BREAKER_COOLDOWN / 60000}min. Provider needs re-authentication.`
);
}
}
/**
* Execute a function with a timeout.
*/
async function withTimeout<T>(fn: () => Promise<T>, timeoutMs: number): Promise<T | null> {
return Promise.race([
fn(),
new Promise<null>((resolve) => setTimeout(() => resolve(null), timeoutMs)),
]);
}
export async function refreshWithRetry(
refreshFn,
maxRetries = 3,
log = null,
provider = "unknown"
) {
// Circuit breaker check
if (isProviderBlocked(provider)) {
log?.warn?.("TOKEN_REFRESH", `⚡ Circuit breaker active for ${provider}, skipping refresh`);
return null;
}
for (let attempt = 0; attempt < maxRetries; attempt++) {
if (attempt > 0) {
const delay = attempt * 1000;
@@ -855,13 +953,18 @@ export async function refreshWithRetry(refreshFn, maxRetries = 3, log = null) {
}
try {
const result = await refreshFn();
if (result) return result;
const result = await withTimeout(refreshFn, REFRESH_TIMEOUT_MS);
if (result) {
recordSuccess(provider);
return result;
}
} catch (error) {
log?.warn?.("TOKEN_REFRESH", `Attempt ${attempt + 1}/${maxRetries} failed: ${error.message}`);
}
}
log?.error?.("TOKEN_REFRESH", `All ${maxRetries} retry attempts failed`);
// All retries exhausted — record failure for circuit breaker
recordFailure(provider, log);
log?.error?.("TOKEN_REFRESH", `All ${maxRetries} retry attempts failed for ${provider}`);
return null;
}

View File

@@ -1,6 +1,6 @@
{
"name": "omniroute",
"version": "1.7.2",
"version": "1.7.3",
"description": "Smart AI Router with auto fallback — route to FREE & cheap models, zero downtime. Works with Cursor, Cline, Claude Desktop, Codex, and any OpenAI-compatible tool.",
"type": "module",
"bin": {

View File

@@ -0,0 +1,261 @@
"use client";
import { useState, useEffect } from "react";
import { Card } from "@/shared/components";
import { useTranslations } from "next-intl";
export default function BackgroundDegradationTab() {
const [config, setConfig] = useState({
enabled: false,
degradationMap: {},
detectionPatterns: [],
stats: { detected: 0, tokensSaved: 0 },
});
const [loading, setLoading] = useState(true);
const [saving, setSaving] = useState(false);
const [status, setStatus] = useState("");
const [newFrom, setNewFrom] = useState("");
const [newTo, setNewTo] = useState("");
const [newPattern, setNewPattern] = useState("");
const t = useTranslations("settings");
useEffect(() => {
fetch("/api/settings/background-degradation")
.then((res) => res.json())
.then((data) => {
setConfig(data);
setLoading(false);
})
.catch(() => setLoading(false));
}, []);
const save = async (updates) => {
const newConfig = { ...config, ...updates };
setConfig(newConfig);
setSaving(true);
setStatus("");
try {
const { stats, ...persistable } = newConfig;
const res = await fetch("/api/settings/background-degradation", {
method: "PUT",
headers: { "Content-Type": "application/json" },
body: JSON.stringify(persistable),
});
if (res.ok) {
const data = await res.json();
setConfig(data);
setStatus("saved");
setTimeout(() => setStatus(""), 2000);
} else {
setStatus("error");
}
} catch {
setStatus("error");
} finally {
setSaving(false);
}
};
const addMapping = () => {
if (!newFrom.trim() || !newTo.trim()) return;
const map = { ...config.degradationMap, [newFrom.trim()]: newTo.trim() };
save({ degradationMap: map });
setNewFrom("");
setNewTo("");
};
const removeMapping = (key) => {
const map = { ...config.degradationMap };
delete map[key];
save({ degradationMap: map });
};
const addPattern = () => {
if (!newPattern.trim()) return;
const patterns = [...config.detectionPatterns, newPattern.trim()];
save({ detectionPatterns: patterns });
setNewPattern("");
};
const removePattern = (idx) => {
const patterns = config.detectionPatterns.filter((_, i) => i !== idx);
save({ detectionPatterns: patterns });
};
const mapEntries = Object.entries(config.degradationMap || {}) as [string, string][];
return (
<Card>
<div className="flex items-center gap-3 mb-5">
<div className="p-2 rounded-lg bg-sky-500/10 text-sky-500">
<span className="material-symbols-outlined text-[20px]" aria-hidden="true">
speed
</span>
</div>
<div className="flex-1">
<h3 className="text-lg font-semibold">
{t("backgroundDegradationTitle") || "Background Task Degradation"}
</h3>
<p className="text-sm text-text-muted">
{t("backgroundDegradationDesc") ||
"Auto-redirect background requests (titles, summaries) to cheaper models"}
</p>
</div>
{status === "saved" && (
<span className="text-xs font-medium text-emerald-500 flex items-center gap-1">
<span className="material-symbols-outlined text-[14px]">check_circle</span>{" "}
{t("saved") || "Saved"}
</span>
)}
</div>
{/* Toggle */}
<div className="flex items-center justify-between p-4 rounded-lg bg-surface/30 border border-border/30 mb-4">
<div>
<p className="text-sm font-medium">
{t("enableDegradation") || "Enable Background Degradation"}
</p>
<p className="text-xs text-text-muted mt-0.5">
{t("enableDegradationHint") ||
"Automatically use cheaper models for background utility tasks"}
</p>
</div>
<button
onClick={() => save({ enabled: !config.enabled })}
disabled={loading || saving}
className={`relative inline-flex h-6 w-11 items-center rounded-full transition-colors ${
config.enabled ? "bg-sky-500" : "bg-white/10"
}`}
>
<span
className={`inline-block h-4 w-4 rounded-full bg-white transition-transform ${
config.enabled ? "translate-x-6" : "translate-x-1"
}`}
/>
</button>
</div>
{/* Stats */}
{config.stats && config.stats.detected > 0 && (
<div className="flex items-center gap-4 p-3 rounded-lg bg-sky-500/5 border border-sky-500/20 mb-4">
<div className="flex items-center gap-2">
<span className="material-symbols-outlined text-[16px] text-sky-400">analytics</span>
<span className="text-xs text-text-muted">
{t("tasksDetected") || "Tasks detected"}:
</span>
<span className="text-sm font-mono font-semibold text-sky-400">
{config.stats.detected}
</span>
</div>
</div>
)}
{config.enabled && (
<>
{/* Degradation Map */}
<div className="mb-4">
<p className="text-xs font-medium text-text-muted uppercase tracking-wider mb-2">
{t("degradationMap") || "Model Degradation Map"}
</p>
{/* Add new mapping */}
<div className="flex items-center gap-2 mb-3">
<input
type="text"
placeholder={t("premiumModel") || "Premium model"}
value={newFrom}
onChange={(e) => setNewFrom(e.target.value)}
className="flex-1 px-3 py-2 rounded-lg text-sm bg-surface border border-border/50 focus:border-sky-500/50 focus:outline-none"
/>
<span className="text-text-muted text-lg"></span>
<input
type="text"
placeholder={t("cheapModel") || "Cheap model"}
value={newTo}
onChange={(e) => setNewTo(e.target.value)}
className="flex-1 px-3 py-2 rounded-lg text-sm bg-surface border border-border/50 focus:border-sky-500/50 focus:outline-none"
/>
<button
onClick={addMapping}
disabled={saving || !newFrom.trim() || !newTo.trim()}
className="px-3 py-2 rounded-lg text-sm font-medium bg-sky-500/10 text-sky-500 hover:bg-sky-500/20 disabled:opacity-50 transition-all"
>
{t("add") || "Add"}
</button>
</div>
{/* Existing mappings */}
{mapEntries.length > 0 && (
<div className="rounded-lg border border-border/30 divide-y divide-border/20 max-h-48 overflow-y-auto">
{mapEntries.map(([from, to]) => (
<div key={from} className="flex items-center gap-3 px-4 py-2">
<code className="text-xs text-orange-400/80 flex-1 truncate">{from}</code>
<span className="material-symbols-outlined text-[14px] text-text-muted">
arrow_forward
</span>
<code className="text-xs text-sky-400/80 flex-1 truncate">{to}</code>
<button
onClick={() => removeMapping(from)}
disabled={saving}
className="p-1 rounded hover:bg-red-500/10 text-text-muted hover:text-red-400 transition-all"
>
<span className="material-symbols-outlined text-[16px]">close</span>
</button>
</div>
))}
</div>
)}
</div>
{/* Detection Patterns */}
<details className="group">
<summary className="text-xs font-medium text-text-muted uppercase tracking-wider cursor-pointer flex items-center gap-1 mb-2">
<span className="material-symbols-outlined text-[14px] group-open:rotate-90 transition-transform">
chevron_right
</span>
{t("detectionPatterns") || "Detection Patterns"} (
{config.detectionPatterns?.length || 0})
</summary>
{/* Add new pattern */}
<div className="flex items-center gap-2 mb-3">
<input
type="text"
placeholder={t("newPattern") || 'e.g. "generate a title"'}
value={newPattern}
onChange={(e) => setNewPattern(e.target.value)}
className="flex-1 px-3 py-2 rounded-lg text-sm bg-surface border border-border/50 focus:border-sky-500/50 focus:outline-none"
/>
<button
onClick={addPattern}
disabled={saving || !newPattern.trim()}
className="px-3 py-2 rounded-lg text-sm font-medium bg-sky-500/10 text-sky-500 hover:bg-sky-500/20 disabled:opacity-50 transition-all"
>
{t("add") || "Add"}
</button>
</div>
{/* Existing patterns */}
<div className="flex flex-wrap gap-2">
{(config.detectionPatterns || []).map((pattern, idx) => (
<span
key={idx}
className="inline-flex items-center gap-1 px-2.5 py-1 rounded-full text-xs bg-sky-500/10 text-sky-400 border border-sky-500/20"
>
{pattern}
<button
onClick={() => removePattern(idx)}
className="hover:text-red-400 transition-colors"
disabled={saving}
>
<span className="material-symbols-outlined text-[12px]">close</span>
</button>
</span>
))}
</div>
</details>
</>
)}
</Card>
);
}

View File

@@ -0,0 +1,179 @@
"use client";
import { useState, useEffect } from "react";
import { Card } from "@/shared/components";
import { useTranslations } from "next-intl";
export default function ModelAliasesTab() {
const [builtIn, setBuiltIn] = useState({});
const [custom, setCustom] = useState({});
const [loading, setLoading] = useState(true);
const [saving, setSaving] = useState(false);
const [status, setStatus] = useState("");
const [newFrom, setNewFrom] = useState("");
const [newTo, setNewTo] = useState("");
const t = useTranslations("settings");
useEffect(() => {
fetch("/api/settings/model-aliases")
.then((res) => res.json())
.then((data) => {
setBuiltIn(data.builtIn || {});
setCustom(data.custom || {});
setLoading(false);
})
.catch(() => setLoading(false));
}, []);
const addAlias = async () => {
if (!newFrom.trim() || !newTo.trim()) return;
setSaving(true);
try {
const res = await fetch("/api/settings/model-aliases", {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({ from: newFrom.trim(), to: newTo.trim() }),
});
if (res.ok) {
const data = await res.json();
setCustom(data.custom);
setNewFrom("");
setNewTo("");
setStatus("saved");
setTimeout(() => setStatus(""), 2000);
}
} catch {
setStatus("error");
} finally {
setSaving(false);
}
};
const removeAlias = async (from) => {
setSaving(true);
try {
const res = await fetch("/api/settings/model-aliases", {
method: "DELETE",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({ from }),
});
if (res.ok) {
const data = await res.json();
setCustom(data.custom);
setStatus("saved");
setTimeout(() => setStatus(""), 2000);
}
} catch {
setStatus("error");
} finally {
setSaving(false);
}
};
const builtInEntries = Object.entries(builtIn);
const customEntries = Object.entries(custom);
return (
<Card>
<div className="flex items-center gap-3 mb-5">
<div className="p-2 rounded-lg bg-amber-500/10 text-amber-500">
<span className="material-symbols-outlined text-[20px]" aria-hidden="true">
swap_horiz
</span>
</div>
<div>
<h3 className="text-lg font-semibold">{t("modelAliasesTitle") || "Model Aliases"}</h3>
<p className="text-sm text-text-muted">
{t("modelAliasesDesc") || "Auto-forward deprecated model IDs to their replacements"}
</p>
</div>
{status === "saved" && (
<span className="ml-auto text-xs font-medium text-emerald-500 flex items-center gap-1">
<span className="material-symbols-outlined text-[14px]">check_circle</span>{" "}
{t("saved") || "Saved"}
</span>
)}
</div>
{/* Add custom alias */}
<div className="p-4 rounded-lg bg-surface/30 border border-border/30 mb-4">
<p className="text-sm font-medium mb-3">
{t("addCustomAlias") || "Add Custom Alias"}
</p>
<div className="flex items-center gap-2">
<input
type="text"
placeholder={t("deprecatedModelId") || "Deprecated model ID"}
value={newFrom}
onChange={(e) => setNewFrom(e.target.value)}
className="flex-1 px-3 py-2 rounded-lg text-sm bg-surface border border-border/50 focus:border-amber-500/50 focus:outline-none"
/>
<span className="text-text-muted text-lg"></span>
<input
type="text"
placeholder={t("newModelId") || "New model ID"}
value={newTo}
onChange={(e) => setNewTo(e.target.value)}
className="flex-1 px-3 py-2 rounded-lg text-sm bg-surface border border-border/50 focus:border-amber-500/50 focus:outline-none"
/>
<button
onClick={addAlias}
disabled={saving || !newFrom.trim() || !newTo.trim()}
className="px-4 py-2 rounded-lg text-sm font-medium bg-amber-500/10 text-amber-500 hover:bg-amber-500/20 disabled:opacity-50 transition-all"
>
{t("add") || "Add"}
</button>
</div>
</div>
{/* Custom aliases */}
{customEntries.length > 0 && (
<div className="mb-4">
<p className="text-xs font-medium text-text-muted uppercase tracking-wider mb-2">
{t("customAliases") || "Custom Aliases"}
</p>
<div className="rounded-lg border border-border/30 divide-y divide-border/20">
{customEntries.map(([from, to]) => (
<div key={from} className="flex items-center gap-3 px-4 py-2.5">
<code className="text-xs text-red-400/80 flex-1 truncate">{from}</code>
<span className="material-symbols-outlined text-[14px] text-text-muted">
arrow_forward
</span>
<code className="text-xs text-emerald-400/80 flex-1 truncate">{to}</code>
<button
onClick={() => removeAlias(from)}
disabled={saving}
className="p-1 rounded hover:bg-red-500/10 text-text-muted hover:text-red-400 transition-all"
>
<span className="material-symbols-outlined text-[16px]">close</span>
</button>
</div>
))}
</div>
</div>
)}
{/* Built-in aliases (collapsed by default) */}
<details className="group">
<summary className="text-xs font-medium text-text-muted uppercase tracking-wider cursor-pointer flex items-center gap-1 mb-2">
<span className="material-symbols-outlined text-[14px] group-open:rotate-90 transition-transform">
chevron_right
</span>
{t("builtInAliases") || "Built-in Aliases"} ({builtInEntries.length})
</summary>
<div className="rounded-lg border border-border/30 divide-y divide-border/20 max-h-60 overflow-y-auto">
{builtInEntries.map(([from, to]) => (
<div key={from} className="flex items-center gap-3 px-4 py-2 opacity-60">
<code className="text-xs text-red-400/60 flex-1 truncate">{from}</code>
<span className="material-symbols-outlined text-[14px] text-text-muted">
arrow_forward
</span>
<code className="text-xs text-emerald-400/60 flex-1 truncate">{to}</code>
<span className="material-symbols-outlined text-[14px] text-text-muted">lock</span>
</div>
))}
</div>
</details>
</Card>
);
}

View File

@@ -13,6 +13,8 @@ import ProxyTab from "./components/ProxyTab";
import AppearanceTab from "./components/AppearanceTab";
import ThinkingBudgetTab from "./components/ThinkingBudgetTab";
import SystemPromptTab from "./components/SystemPromptTab";
import ModelAliasesTab from "./components/ModelAliasesTab";
import BackgroundDegradationTab from "./components/BackgroundDegradationTab";
import CacheStatsCard from "./components/CacheStatsCard";
import ResilienceTab from "./components/ResilienceTab";
@@ -94,6 +96,8 @@ export default function SettingsPage() {
<div className="flex flex-col gap-6">
<RoutingTab />
<ComboDefaultsTab />
<ModelAliasesTab />
<BackgroundDegradationTab />
</div>
)}

View File

@@ -0,0 +1,60 @@
import { NextResponse } from "next/server";
import {
getBackgroundDegradationConfig,
setBackgroundDegradationConfig,
resetStats,
} from "@omniroute/open-sse/services/backgroundTaskDetector.ts";
import { updateSettings } from "@/lib/db/settings";
/**
* GET /api/settings/background-degradation
* Returns the current background degradation configuration.
*/
export async function GET() {
try {
return NextResponse.json(getBackgroundDegradationConfig());
} catch (error) {
console.error("[API ERROR] /api/settings/background-degradation GET:", error);
return NextResponse.json({ error: "Failed to get config" }, { status: 500 });
}
}
/**
* PUT /api/settings/background-degradation
* Update the background degradation configuration.
* Body: { enabled?: boolean, degradationMap?: {...}, detectionPatterns?: [...] }
*/
export async function PUT(request) {
try {
const config = await request.json();
setBackgroundDegradationConfig(config);
// Persist to database (excluding stats)
const { stats, ...persistable } = getBackgroundDegradationConfig();
await updateSettings({ backgroundDegradation: JSON.stringify(persistable) });
return NextResponse.json({ success: true, ...getBackgroundDegradationConfig() });
} catch (error) {
console.error("[API ERROR] /api/settings/background-degradation PUT:", error);
return NextResponse.json({ error: "Failed to update config" }, { status: 500 });
}
}
/**
* POST /api/settings/background-degradation
* Reset stats counters.
* Body: { action: "reset-stats" }
*/
export async function POST(request) {
try {
const { action } = await request.json();
if (action === "reset-stats") {
resetStats();
return NextResponse.json({ success: true, stats: getBackgroundDegradationConfig().stats });
}
return NextResponse.json({ error: "Unknown action" }, { status: 400 });
} catch (error) {
console.error("[API ERROR] /api/settings/background-degradation POST:", error);
return NextResponse.json({ error: "Failed to execute action" }, { status: 500 });
}
}

View File

@@ -0,0 +1,90 @@
import { NextResponse } from "next/server";
import {
getAllAliases,
getCustomAliases,
getBuiltInAliases,
setCustomAliases,
addCustomAlias,
removeCustomAlias,
} from "@omniroute/open-sse/services/modelDeprecation.ts";
import { getSettings, updateSettings } from "@/lib/db/settings";
/**
* GET /api/settings/model-aliases
* Returns the full alias map, separated into built-in and custom.
*/
export async function GET() {
try {
return NextResponse.json({
builtIn: getBuiltInAliases(),
custom: getCustomAliases(),
all: getAllAliases(),
});
} catch (error) {
console.error("[API ERROR] /api/settings/model-aliases GET:", error);
return NextResponse.json({ error: "Failed to get model aliases" }, { status: 500 });
}
}
/**
* PUT /api/settings/model-aliases
* Update the custom aliases map.
* Body: { aliases: { "old-model": "new-model", ... } }
*/
export async function PUT(request) {
try {
const { aliases } = await request.json();
if (!aliases || typeof aliases !== "object") {
return NextResponse.json({ error: "Missing or invalid 'aliases' object" }, { status: 400 });
}
setCustomAliases(aliases);
await updateSettings({ modelAliases: JSON.stringify(aliases) });
return NextResponse.json({ success: true, custom: getCustomAliases() });
} catch (error) {
console.error("[API ERROR] /api/settings/model-aliases PUT:", error);
return NextResponse.json({ error: "Failed to update model aliases" }, { status: 500 });
}
}
/**
* POST /api/settings/model-aliases
* Add a single custom alias.
* Body: { from: "old-model", to: "new-model" }
*/
export async function POST(request) {
try {
const { from, to } = await request.json();
if (!from || !to) {
return NextResponse.json({ error: "Missing 'from' or 'to'" }, { status: 400 });
}
addCustomAlias(from, to);
await updateSettings({ modelAliases: JSON.stringify(getCustomAliases()) });
return NextResponse.json({ success: true, custom: getCustomAliases() });
} catch (error) {
console.error("[API ERROR] /api/settings/model-aliases POST:", error);
return NextResponse.json({ error: "Failed to add alias" }, { status: 500 });
}
}
/**
* DELETE /api/settings/model-aliases
* Remove a custom alias.
* Body: { from: "old-model" }
*/
export async function DELETE(request) {
try {
const { from } = await request.json();
if (!from) {
return NextResponse.json({ error: "Missing 'from'" }, { status: 400 });
}
const removed = removeCustomAlias(from);
if (!removed) {
return NextResponse.json({ error: "Alias not found" }, { status: 404 });
}
await updateSettings({ modelAliases: JSON.stringify(getCustomAliases()) });
return NextResponse.json({ success: true, custom: getCustomAliases() });
} catch (error) {
console.error("[API ERROR] /api/settings/model-aliases DELETE:", error);
return NextResponse.json({ error: "Failed to remove alias" }, { status: 500 });
}
}

View File

@@ -0,0 +1,130 @@
import test from "node:test";
import assert from "node:assert/strict";
const {
isBackgroundTask,
getDegradedModel,
setBackgroundDegradationConfig,
getBackgroundDegradationConfig,
getDefaultDegradationMap,
getDefaultDetectionPatterns,
resetStats,
} = await import("../../open-sse/services/backgroundTaskDetector.ts");
// ─── isBackgroundTask ───────────────────────────────────────────────────────
test("isBackgroundTask: returns true for title generation pattern", () => {
setBackgroundDegradationConfig({ enabled: true });
const body = {
model: "claude-sonnet-4",
messages: [
{ role: "system", content: "Generate a title for this conversation" },
{ role: "user", content: "How to deploy a Next.js app" },
],
};
assert.equal(isBackgroundTask(body), true);
});
test("isBackgroundTask: returns true for summarize pattern", () => {
const body = {
model: "claude-sonnet-4",
messages: [
{ role: "system", content: "Summarize this conversation briefly" },
{ role: "user", content: "We discussed deployment techniques" },
],
};
assert.equal(isBackgroundTask(body), true);
});
test("isBackgroundTask: returns false for normal chat", () => {
const body = {
model: "claude-sonnet-4",
messages: [
{ role: "system", content: "You are a helpful coding assistant" },
{ role: "user", content: "Help me write a function" },
],
};
assert.equal(isBackgroundTask(body), false);
});
test("isBackgroundTask: returns false for many-turn conversations", () => {
const messages = [
{ role: "system", content: "Generate a title" },
...Array.from({ length: 10 }, (_, i) => ({
role: i % 2 === 0 ? "user" : "assistant",
content: `Message ${i}`,
})),
];
const body = { model: "claude-sonnet-4", messages };
assert.equal(isBackgroundTask(body), false); // Too many turns
});
test("isBackgroundTask: detects X-Request-Priority header", () => {
const body = {
model: "claude-sonnet-4",
messages: [{ role: "user", content: "hello" }],
};
const headers = { "x-request-priority": "background" };
assert.equal(isBackgroundTask(body, headers), true);
});
test("isBackgroundTask: returns false for null/undefined body", () => {
assert.equal(isBackgroundTask(null), false);
assert.equal(isBackgroundTask(undefined), false);
});
test("isBackgroundTask: returns false for empty messages", () => {
assert.equal(isBackgroundTask({ messages: [] }), false);
});
// ─── getDegradedModel ───────────────────────────────────────────────────────
test("getDegradedModel: returns cheaper model from map", () => {
resetStats();
assert.equal(getDegradedModel("claude-opus-4-6"), "gemini-2.5-flash");
assert.equal(getDegradedModel("gemini-2.5-pro"), "gemini-2.5-flash");
assert.equal(getDegradedModel("gpt-4o"), "gpt-4o-mini");
});
test("getDegradedModel: returns original if no mapping exists", () => {
assert.equal(getDegradedModel("some-unknown-model"), "some-unknown-model");
});
test("getDegradedModel: handles null/empty", () => {
assert.equal(getDegradedModel(""), "");
assert.equal(getDegradedModel(null), null);
});
test("getDegradedModel: increments stats counter", () => {
resetStats();
getDegradedModel("claude-opus-4-6"); // known mapping
const config = getBackgroundDegradationConfig();
assert.equal(config.stats.detected, 1);
});
// ─── Config Management ──────────────────────────────────────────────────────
test("getBackgroundDegradationConfig: returns config copy", () => {
const config = getBackgroundDegradationConfig();
assert.ok(typeof config.enabled === "boolean");
assert.ok(typeof config.degradationMap === "object");
assert.ok(Array.isArray(config.detectionPatterns));
});
test("setBackgroundDegradationConfig: updates config", () => {
setBackgroundDegradationConfig({ enabled: true });
assert.equal(getBackgroundDegradationConfig().enabled, true);
setBackgroundDegradationConfig({ enabled: false }); // reset
});
test("getDefaultDegradationMap: returns non-empty map", () => {
const map = getDefaultDegradationMap();
assert.ok(Object.keys(map).length > 0);
assert.ok(map["claude-opus-4-6"]);
});
test("getDefaultDetectionPatterns: returns non-empty array", () => {
const patterns = getDefaultDetectionPatterns();
assert.ok(patterns.length > 0);
assert.ok(patterns.includes("generate a title"));
});

View File

@@ -0,0 +1,105 @@
import test from "node:test";
import assert from "node:assert/strict";
const {
resolveModelAlias,
getDeprecationNotice,
isDeprecated,
setCustomAliases,
getCustomAliases,
addCustomAlias,
removeCustomAlias,
getAllAliases,
getBuiltInAliases,
} = await import("../../open-sse/services/modelDeprecation.ts");
// ─── resolveModelAlias ──────────────────────────────────────────────────────
test("resolveModelAlias: returns original for non-deprecated model", () => {
assert.equal(resolveModelAlias("claude-opus-4-6"), "claude-opus-4-6");
});
test("resolveModelAlias: resolves deprecated Gemini model", () => {
assert.equal(resolveModelAlias("gemini-pro"), "gemini-2.5-pro");
assert.equal(resolveModelAlias("gemini-1.5-pro"), "gemini-2.5-pro");
assert.equal(resolveModelAlias("gemini-1.5-flash"), "gemini-2.5-flash");
});
test("resolveModelAlias: resolves deprecated Claude model", () => {
assert.equal(resolveModelAlias("claude-3-opus-20240229"), "claude-opus-4-20250514");
assert.equal(resolveModelAlias("claude-3-5-sonnet-latest"), "claude-sonnet-4-20250514");
});
test("resolveModelAlias: resolves deprecated OpenAI model", () => {
assert.equal(resolveModelAlias("gpt-4-turbo-preview"), "gpt-4-turbo");
assert.equal(resolveModelAlias("gpt-3.5-turbo-0125"), "gpt-3.5-turbo");
});
test("resolveModelAlias: handles null/empty", () => {
assert.equal(resolveModelAlias(""), "");
assert.equal(resolveModelAlias(null), null);
assert.equal(resolveModelAlias(undefined), undefined);
});
// ─── getDeprecationNotice ───────────────────────────────────────────────────
test("getDeprecationNotice: returns message for deprecated model", () => {
const notice = getDeprecationNotice("gemini-pro");
assert.ok(notice);
assert.ok(notice.includes("deprecated"));
assert.ok(notice.includes("gemini-2.5-pro"));
});
test("getDeprecationNotice: returns null for non-deprecated model", () => {
assert.equal(getDeprecationNotice("claude-opus-4-6"), null);
});
test("getDeprecationNotice: returns null for empty/null", () => {
assert.equal(getDeprecationNotice(""), null);
assert.equal(getDeprecationNotice(null), null);
});
// ─── isDeprecated ───────────────────────────────────────────────────────────
test("isDeprecated: true for deprecated model", () => {
assert.equal(isDeprecated("gemini-pro"), true);
});
test("isDeprecated: false for current model", () => {
assert.equal(isDeprecated("claude-opus-4-6"), false);
});
// ─── Custom Aliases ─────────────────────────────────────────────────────────
test("custom aliases override built-in", () => {
setCustomAliases({ "gemini-pro": "gemini-3.1-pro" });
assert.equal(resolveModelAlias("gemini-pro"), "gemini-3.1-pro"); // custom wins
setCustomAliases({}); // reset
});
test("addCustomAlias and removeCustomAlias", () => {
addCustomAlias("my-old-model", "my-new-model");
assert.equal(resolveModelAlias("my-old-model"), "my-new-model");
const removed = removeCustomAlias("my-old-model");
assert.equal(removed, true);
assert.equal(resolveModelAlias("my-old-model"), "my-old-model");
});
test("removeCustomAlias: returns false for non-existent", () => {
assert.equal(removeCustomAlias("nonexistent"), false);
});
test("getAllAliases: includes both built-in and custom", () => {
addCustomAlias("test-from", "test-to");
const all = getAllAliases();
assert.ok(all["gemini-pro"]); // built-in
assert.equal(all["test-from"], "test-to"); // custom
removeCustomAlias("test-from"); // cleanup
});
test("getBuiltInAliases: returns built-in aliases", () => {
const builtIn = getBuiltInAliases();
assert.ok(builtIn["gemini-pro"]);
assert.ok(builtIn["claude-3-opus-20240229"]);
});

View File

@@ -8,6 +8,10 @@ const {
ThinkingMode,
EFFORT_BUDGETS,
DEFAULT_THINKING_CONFIG,
THINKING_LEVEL_MAP,
normalizeThinkingLevel,
ensureThinkingConfig,
hasThinkingCapableModel,
} = await import("../../open-sse/services/thinkingBudget.ts");
// ─── Config Management ──────────────────────────────────────────────────────
@@ -159,3 +163,124 @@ test("EFFORT_BUDGETS has expected keys", () => {
assert.ok(EFFORT_BUDGETS.medium > EFFORT_BUDGETS.low);
assert.ok(EFFORT_BUDGETS.high > EFFORT_BUDGETS.medium);
});
// ─── thinkingLevel String Conversion (Feature 4) ────────────────────────────
test("THINKING_LEVEL_MAP has all expected levels", () => {
assert.equal(THINKING_LEVEL_MAP.none, 0);
assert.equal(THINKING_LEVEL_MAP.low, 1024);
assert.equal(THINKING_LEVEL_MAP.medium, 10240);
assert.equal(THINKING_LEVEL_MAP.high, 131072);
});
test("normalizeThinkingLevel: converts thinkingLevel 'high' to budget", () => {
const body = {
model: "claude-sonnet-4",
thinkingLevel: "high",
messages: [{ role: "user", content: "hello" }],
};
const result = normalizeThinkingLevel(body);
assert.equal(result.thinking.type, "enabled");
assert.equal(result.thinking.budget_tokens, 131072);
assert.equal(result.thinkingLevel, undefined);
});
test("normalizeThinkingLevel: converts thinking_level 'low' to budget", () => {
const body = {
model: "claude-sonnet-4",
thinking_level: "low",
messages: [{ role: "user", content: "hello" }],
};
const result = normalizeThinkingLevel(body);
assert.equal(result.thinking.type, "enabled");
assert.equal(result.thinking.budget_tokens, 1024);
assert.equal(result.thinking_level, undefined);
});
test("normalizeThinkingLevel: converts 'none' to disabled", () => {
const body = { model: "claude-sonnet-4", thinkingLevel: "none" };
const result = normalizeThinkingLevel(body);
assert.equal(result.thinking.type, "disabled");
assert.equal(result.thinking.budget_tokens, 0);
});
test("normalizeThinkingLevel: converts Gemini thinkingConfig.thinkingLevel", () => {
const body = {
model: "gemini-2.5-pro",
generationConfig: {
thinkingConfig: { thinkingLevel: "high" },
},
};
const result = normalizeThinkingLevel(body);
assert.equal(result.generationConfig.thinking_config.thinking_budget, 131072);
assert.equal(result.generationConfig.thinkingConfig, undefined);
});
test("normalizeThinkingLevel: ignores unknown string values", () => {
const body = { model: "claude-sonnet-4", thinkingLevel: "ultra" };
const result = normalizeThinkingLevel(body);
assert.equal(result.thinking, undefined); // not converted
assert.equal(result.thinkingLevel, "ultra"); // preserved
});
// ─── -thinking Suffix Auto-Injection (Feature 5) ────────────────────────────
test("ensureThinkingConfig: auto-injects for -thinking suffix model", () => {
const body = {
model: "claude-opus-4-6-thinking",
messages: [{ role: "user", content: "hello" }],
};
const result = ensureThinkingConfig(body);
assert.equal(result.thinking.type, "enabled");
assert.equal(result.thinking.budget_tokens, EFFORT_BUDGETS.medium);
});
test("ensureThinkingConfig: does NOT override existing thinking config", () => {
const body = {
model: "claude-opus-4-6-thinking",
thinking: { type: "enabled", budget_tokens: 50000 },
messages: [{ role: "user", content: "hello" }],
};
const result = ensureThinkingConfig(body);
assert.equal(result.thinking.budget_tokens, 50000); // preserved
});
test("ensureThinkingConfig: does nothing for non-thinking models", () => {
const body = {
model: "claude-sonnet-4",
messages: [{ role: "user", content: "hello" }],
};
const result = ensureThinkingConfig(body);
assert.equal(result.thinking, undefined);
});
test("hasThinkingCapableModel: matches -thinking suffix", () => {
assert.ok(hasThinkingCapableModel({ model: "claude-opus-4-6-thinking" }));
assert.ok(hasThinkingCapableModel({ model: "kimi-k2-thinking" }));
assert.ok(hasThinkingCapableModel({ model: "custom-model-thinking" }));
});
test("applyThinkingBudget: thinkingLevel 'high' + PASSTHROUGH = converts and passes through", () => {
setThinkingBudgetConfig({ mode: ThinkingMode.PASSTHROUGH });
const body = {
model: "claude-sonnet-4",
thinkingLevel: "high",
messages: [{ role: "user", content: "hello" }],
};
const result = applyThinkingBudget(body);
assert.equal(result.thinking.budget_tokens, 131072);
assert.equal(result.thinkingLevel, undefined);
setThinkingBudgetConfig(DEFAULT_THINKING_CONFIG);
});
test("applyThinkingBudget: -thinking model without config + PASSTHROUGH = auto-inject", () => {
setThinkingBudgetConfig({ mode: ThinkingMode.PASSTHROUGH });
const body = {
model: "claude-opus-4-6-thinking",
messages: [{ role: "user", content: "hello" }],
};
const result = applyThinkingBudget(body);
assert.equal(result.thinking.type, "enabled");
assert.equal(result.thinking.budget_tokens, EFFORT_BUDGETS.medium);
setThinkingBudgetConfig(DEFAULT_THINKING_CONFIG);
});