mirror of
https://github.com/diegosouzapw/OmniRoute.git
synced 2026-07-26 09:52:11 +03:00
Compare commits
9 Commits
v3.0.0-rc.
...
v3.0.0-rc.
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
0546d06c0a | ||
|
|
5337111990 | ||
|
|
bb06f8eb0c | ||
|
|
e47740e02e | ||
|
|
d9ff0035f5 | ||
|
|
7a7f3be0d2 | ||
|
|
91e45fbe95 | ||
|
|
7d7e9da28c | ||
|
|
24a9739604 |
1
.gitignore
vendored
1
.gitignore
vendored
@@ -89,6 +89,7 @@ docs/*
|
||||
!docs/MCP-SERVER.md
|
||||
!docs/CLI-TOOLS.md
|
||||
|
||||
|
||||
# open-sse tests
|
||||
open-sse/test/*
|
||||
|
||||
|
||||
79
CHANGELOG.md
79
CHANGELOG.md
@@ -6,9 +6,47 @@
|
||||
|
||||
---
|
||||
|
||||
## [3.0.0-rc.5] — 2026-03-22 _(What's New vs v2.9.5 — will be released as v3.0.0)_
|
||||
## [3.0.0-rc.10] — 2026-03-23
|
||||
|
||||
> **Upgrade from v2.9.5:** 16 issues resolved · 2 community PRs merged · 2 new providers · 7 new API endpoints · 3 new features · DB migration 008 · 832 tests passing.
|
||||
### 🔧 Bug Fixes
|
||||
|
||||
- **#509 / #508** — Electron build regression: downgraded Next.js from `16.1.x` to `16.0.10` to eliminate Turbopack module-hashing instability that caused blank screens in the Electron desktop bundle.
|
||||
- **Unit test fixes** — Corrected two stale test assertions (`nanobanana-image-handler` aspect ratio/resolution, `thinking-budget` Gemini `thinkingConfig` field mapping) that had drifted after recent implementation changes.
|
||||
- **#541** — Responded to user feedback about installation complexity; no code changes required.
|
||||
|
||||
---
|
||||
|
||||
## [3.0.0-rc.9] — 2026-03-23
|
||||
|
||||
### ✨ New Features
|
||||
|
||||
- **T29** — Vertex AI SA JSON Executor: implemented using the `jose` library to handle JWT/Service Account auth, along with configurable regions in the UI and automatic partner model URL building.
|
||||
- **T42** — Image generation aspect ratio mapping: created `sizeMapper` logic for generic OpenAI formats (`size`), added native `imagen3` handling, and updated NanoBanana endpoints to utilize mapped aspect ratios automatically.
|
||||
- **T38** — Centralized model specifications: `modelSpecs.ts` created for limits and parameters per model.
|
||||
|
||||
### 🔧 Improvements
|
||||
|
||||
- **T40** — OpenCode CLI tools integration: native `opencode-zen` and `opencode-go` integration completed in earlier PR.
|
||||
|
||||
---
|
||||
|
||||
## [3.0.0-rc.8] — 2026-03-23
|
||||
|
||||
### 🔧 Bug Fixes & Improvements (Fallback, Quota & Budget)
|
||||
|
||||
- **T24** — `503` cooldown await fix + `406` mapping: mapped `406 Not Acceptable` to `503 Service Unavailable` with proper cooldown intervals.
|
||||
- **T25** — Provider validation fallback: graceful fallback to standard validation models when a specific `validationModelId` is not present.
|
||||
- **T36** — `403` vs `429` provider handling refinement: extracted into `errorClassifier.ts` to properly segregate hard permissions failures (`403`) from rate limits (`429`).
|
||||
- **T39** — Endpoint Fallback for `fetchAvailableModels`: implemented a tri-tier mechanism (`/models` -> `/v1/models` -> local generic catalog) + `list_models_catalog` MCP tool updates to reflect `source` and `warning`.
|
||||
- **T33** — Thinking level to budget conversion: translates qualitative thinking levels into precise budget allocations.
|
||||
- **T41** — Background task auto redirect: routes heavy background evaluation tasks to flash/efficient models automatically.
|
||||
- **T23** — Intelligent quota reset fallback: accurately extracts `x-ratelimit-reset` / `retry-after` header values or maps static cooldowns.
|
||||
|
||||
---
|
||||
|
||||
## [3.0.0-rc.7] — 2026-03-23 _(What's New vs v2.9.5 — will be released as v3.0.0)_
|
||||
|
||||
> **Upgrade from v2.9.5:** 16 issues resolved · 2 community PRs merged · 2 new providers · 7 new API endpoints · 3 new features · DB migration 008+009 · 832 tests passing · 15 sub2api gap improvements (T01–T15 complete).
|
||||
|
||||
### 🆕 New Providers
|
||||
|
||||
@@ -111,6 +149,43 @@ OmniRoute now automatically refreshes model lists for connected providers every
|
||||
|
||||
---
|
||||
|
||||
## [3.0.0-rc.7] - 2026-03-23
|
||||
|
||||
### 🔧 Improvements (sub2api Gap Analysis — T05, T08, T09, T13, T14)
|
||||
|
||||
- **T05** — Rate-limit DB persistence: `setConnectionRateLimitUntil()`, `isConnectionRateLimited()`, `getRateLimitedConnections()` in `providers.ts`. The existing `rate_limited_until` column is now exposed as a dedicated API — OAuth token refresh must NOT touch this field to prevent rate-limit loops.
|
||||
- **T08** — Per-API-key session limit: `max_sessions INTEGER DEFAULT 0` added to `api_keys` via auto-migration. `sessionManager.ts` gains `registerKeySession()`, `unregisterKeySession()`, `checkSessionLimit()`, and `getActiveSessionCountForKey()`. Callers in `chatCore.js` can enforce the limit and decrement on `req.close`.
|
||||
- **T09** — Codex vs Spark rate-limit scopes: `getCodexModelScope()` and `getCodexRateLimitKey()` in `codex.ts`. Standard models (`gpt-5.x-codex`, `codex-mini`) get scope `"codex"`; spark models (`codex-spark*`) get scope `"spark"`. Rate-limit keys should be `${accountId}:${scope}` so exhausting one pool doesn't block the other.
|
||||
- **T13** — Stale quota display fix: `getEffectiveQuotaUsage(used, resetAt)` returns `0` when the reset window has passed; `formatResetCountdown(resetAt)` returns a human-readable countdown string (e.g. `"2h 35m"`). Both exported from `providers.ts` + `localDb.ts` for dashboard consumption.
|
||||
- **T14** — Proxy fast-fail: new `src/lib/proxyHealth.ts` with `isProxyReachable(proxyUrl, timeoutMs=2000)` (TCP check, ≤2s instead of 30s timeout), `getCachedProxyHealth()`, `invalidateProxyHealth()`, and `getAllProxyHealthStatuses()`. Results cached 30s by default; configurable via `PROXY_FAST_FAIL_TIMEOUT_MS` / `PROXY_HEALTH_CACHE_TTL_MS`.
|
||||
|
||||
### 🧪 Tests
|
||||
|
||||
- Test suite: **832 tests, 0 failures**
|
||||
|
||||
---
|
||||
|
||||
## [3.0.0-rc.6] - 2026-03-23
|
||||
|
||||
### 🔧 Bug Fixes & Improvements (sub2api Gap Analysis — T01–T15)
|
||||
|
||||
- **T01** — `requested_model` column in `call_logs` (migration 009): track which model the client originally requested vs the actual routed model. Enables fallback rate analytics.
|
||||
- **T02** — Strip empty text blocks from nested `tool_result.content`: prevents Anthropic 400 errors (`text content blocks must be non-empty`) when Claude Code chains tool results.
|
||||
- **T03** — Parse `x-codex-5h-*` / `x-codex-7d-*` headers: `parseCodexQuotaHeaders()` + `getCodexResetTime()` extract Codex quota windows for precise cooldown scheduling instead of generic 5-min fallback.
|
||||
- **T04** — `X-Session-Id` header for external sticky routing: `extractExternalSessionId()` in `sessionManager.ts` reads `x-session-id` / `x-omniroute-session` headers with `ext:` prefix to avoid collision with internal SHA-256 session IDs. Nginx-compatible (hyphenated header).
|
||||
- **T06** — Account deactivated → permanent block: `isAccountDeactivated()` in `accountFallback.ts` detects 401 deactivation signals and applies a 1-year cooldown to prevent retrying permanently dead accounts.
|
||||
- **T07** — X-Forwarded-For IP validation: new `src/lib/ipUtils.ts` with `extractClientIp()` and `getClientIpFromRequest()` — skips `unknown`/non-IP entries in `X-Forwarded-For` chains (Nginx/proxy-forwarded requests).
|
||||
- **T10** — Credits exhausted → distinct fallback: `isCreditsExhausted()` in `accountFallback.ts` returns 1h cooldown with `creditsExhausted` flag, distinct from generic 429 rate limiting.
|
||||
- **T11** — `max` reasoning effort → 131072 budget tokens: `EFFORT_BUDGETS` and `THINKING_LEVEL_MAP` updated; reverse mapping now returns `"max"` for full-budget responses. Unit test updated.
|
||||
- **T12** — MiniMax M2.7 pricing entries added: `minimax-m2.7`, `MiniMax-M2.7`, `minimax-m2.7-highspeed` added to pricing table (sub2api PR #1120). M2.5/GLM-4.7/GLM-5/Kimi pricing already existed.
|
||||
- **T15** — Array content normalization: `normalizeContentToString()` helper in `openai-to-claude.ts` correctly collapses array-formatted system/tool messages to string before sending to Anthropic.
|
||||
|
||||
### 🧪 Tests
|
||||
|
||||
- Test suite: **832 tests, 0 failures** (unchanged from rc.5)
|
||||
|
||||
---
|
||||
|
||||
## [3.0.0-rc.5] - 2026-03-22
|
||||
|
||||
### ✨ New Features
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
openapi: 3.1.0
|
||||
info:
|
||||
title: OmniRoute API
|
||||
version: 3.0.0-rc.5
|
||||
version: 3.0.0-rc.10
|
||||
description: |
|
||||
OmniRoute is a local-first AI API proxy router. It provides an OpenAI-compatible
|
||||
endpoint that routes requests to multiple AI providers with load balancing,
|
||||
|
||||
@@ -6,6 +6,8 @@
|
||||
* is auto-generated from this registry.
|
||||
*/
|
||||
|
||||
import { platform, arch } from "os";
|
||||
|
||||
// ── Types ─────────────────────────────────────────────────────────────────
|
||||
|
||||
export interface RegistryModel {
|
||||
@@ -96,6 +98,32 @@ const KIMI_CODING_SHARED = {
|
||||
] as RegistryModel[],
|
||||
} as const;
|
||||
|
||||
function mapStainlessOs() {
|
||||
switch (platform()) {
|
||||
case "darwin":
|
||||
return "MacOS";
|
||||
case "win32":
|
||||
return "Windows";
|
||||
case "linux":
|
||||
return "Linux";
|
||||
default:
|
||||
return `Other::${platform()}`;
|
||||
}
|
||||
}
|
||||
|
||||
function mapStainlessArch() {
|
||||
switch (arch()) {
|
||||
case "x64":
|
||||
return "x64";
|
||||
case "arm64":
|
||||
return "arm64";
|
||||
case "ia32":
|
||||
return "x86";
|
||||
default:
|
||||
return `other::${arch()}`;
|
||||
}
|
||||
}
|
||||
|
||||
// ── Registry ──────────────────────────────────────────────────────────────
|
||||
|
||||
export const REGISTRY: Record<string, RegistryEntry> = {
|
||||
@@ -112,19 +140,19 @@ export const REGISTRY: Record<string, RegistryEntry> = {
|
||||
headers: {
|
||||
"Anthropic-Version": "2023-06-01",
|
||||
"Anthropic-Beta":
|
||||
"claude-code-20250219,oauth-2025-04-20,interleaved-thinking-2025-05-14,fine-grained-tool-streaming-2025-05-14,context-management-2025-06-27",
|
||||
"claude-code-20250219,oauth-2025-04-20,interleaved-thinking-2025-05-14,fine-grained-tool-streaming-2025-05-14,context-management-2025-06-27,prompt-caching-scope-2026-01-05",
|
||||
"Anthropic-Dangerous-Direct-Browser-Access": "true",
|
||||
"User-Agent": "claude-cli/1.0.83 (external, cli)",
|
||||
"User-Agent": "claude-cli/2.1.63 (external, cli)",
|
||||
"X-App": "cli",
|
||||
"X-Stainless-Helper-Method": "stream",
|
||||
"X-Stainless-Retry-Count": "0",
|
||||
"X-Stainless-Runtime-Version": "v24.3.0",
|
||||
"X-Stainless-Package-Version": "0.55.1",
|
||||
"X-Stainless-Package-Version": "0.74.0",
|
||||
"X-Stainless-Runtime": "node",
|
||||
"X-Stainless-Lang": "js",
|
||||
"X-Stainless-Arch": "arm64",
|
||||
"X-Stainless-Os": "MacOS",
|
||||
"X-Stainless-Timeout": "60",
|
||||
"X-Stainless-Arch": mapStainlessArch(),
|
||||
"X-Stainless-Os": mapStainlessOs(),
|
||||
"X-Stainless-Timeout": "600",
|
||||
},
|
||||
oauth: {
|
||||
clientIdEnv: "CLAUDE_OAUTH_CLIENT_ID",
|
||||
@@ -159,6 +187,8 @@ export const REGISTRY: Record<string, RegistryEntry> = {
|
||||
clientSecretDefault: "",
|
||||
},
|
||||
models: [
|
||||
{ id: "gemini-3.1-pro-high", name: "Gemini 3.1 Pro High" },
|
||||
{ id: "gemini-3.1-pro-low", name: "Gemini 3.1 Pro Low" },
|
||||
{ id: "gemini-3.1-pro", name: "Gemini 3.1 Pro" },
|
||||
{ id: "gemini-3-1-pro", name: "Gemini 3.1 Pro (Alt ID)" },
|
||||
{ id: "gemini-3.1-pro-preview", name: "Gemini 3.1 Pro Preview" },
|
||||
@@ -191,6 +221,8 @@ export const REGISTRY: Record<string, RegistryEntry> = {
|
||||
clientSecretDefault: "",
|
||||
},
|
||||
models: [
|
||||
{ id: "gemini-3.1-pro-high", name: "Gemini 3.1 Pro High" },
|
||||
{ id: "gemini-3.1-pro-low", name: "Gemini 3.1 Pro Low" },
|
||||
{ id: "gemini-3.1-pro", name: "Gemini 3.1 Pro" },
|
||||
{ id: "gemini-3-1-pro", name: "Gemini 3.1 Pro (Alt ID)" },
|
||||
{ id: "gemini-3.1-pro-preview", name: "Gemini 3.1 Pro Preview" },
|
||||
@@ -322,7 +354,11 @@ export const REGISTRY: Record<string, RegistryEntry> = {
|
||||
alias: "ag",
|
||||
format: "antigravity",
|
||||
executor: "antigravity",
|
||||
baseUrls: ["https://daily-cloudcode-pa.googleapis.com", "https://cloudcode-pa.googleapis.com"],
|
||||
baseUrls: [
|
||||
"https://daily-cloudcode-pa.googleapis.com",
|
||||
"https://daily-cloudcode-pa.sandbox.googleapis.com",
|
||||
"https://cloudcode-pa.googleapis.com",
|
||||
],
|
||||
urlBuilder: (base, model, stream) => {
|
||||
const path = stream
|
||||
? "/v1internal:streamGenerateContent?alt=sse"
|
||||
@@ -332,7 +368,7 @@ export const REGISTRY: Record<string, RegistryEntry> = {
|
||||
authType: "oauth",
|
||||
authHeader: "bearer",
|
||||
headers: {
|
||||
"User-Agent": "antigravity/1.104.0 darwin/arm64",
|
||||
"User-Agent": `antigravity/1.107.0 ${platform()}/${arch()}`,
|
||||
},
|
||||
oauth: {
|
||||
clientIdEnv: "ANTIGRAVITY_OAUTH_CLIENT_ID",
|
||||
@@ -361,9 +397,9 @@ export const REGISTRY: Record<string, RegistryEntry> = {
|
||||
authHeader: "bearer",
|
||||
headers: {
|
||||
"copilot-integration-id": "vscode-chat",
|
||||
"editor-version": "vscode/1.107.1",
|
||||
"editor-plugin-version": "copilot-chat/0.26.7",
|
||||
"user-agent": "GitHubCopilotChat/0.26.7",
|
||||
"editor-version": "vscode/1.110.0",
|
||||
"editor-plugin-version": "copilot-chat/0.38.0",
|
||||
"user-agent": "GitHubCopilotChat/0.38.0",
|
||||
"openai-intent": "conversation-panel",
|
||||
"x-github-api-version": "2025-04-01",
|
||||
"x-vscode-user-agent-library-version": "electron-fetch",
|
||||
@@ -1146,7 +1182,7 @@ export const REGISTRY: Record<string, RegistryEntry> = {
|
||||
alias: "vertex",
|
||||
// Vertex AI uses Google's generateContent format (same as Gemini)
|
||||
format: "gemini",
|
||||
executor: "default",
|
||||
executor: "vertex",
|
||||
// URL uses {project_id} and {region} from providerSpecificData — handled by custom executor or fallback
|
||||
// Default to us-central1 / generic endpoint; users configure project via providerSpecificData
|
||||
baseUrl: "https://us-central1-aiplatform.googleapis.com/v1/projects",
|
||||
@@ -1241,6 +1277,68 @@ export const REGISTRY: Record<string, RegistryEntry> = {
|
||||
],
|
||||
},
|
||||
|
||||
puter: {
|
||||
id: "puter",
|
||||
alias: "pu",
|
||||
format: "openai",
|
||||
executor: "puter",
|
||||
// OpenAI-compatible gateway with 500+ models (GPT, Claude, Gemini, Grok, DeepSeek, Qwen…)
|
||||
// Auth: Bearer <puter_auth_token> from puter.com/dashboard → Copy Auth Token
|
||||
// Model IDs use provider/model-name format for non-OpenAI models.
|
||||
// Only chat completions (incl. streaming) are available via REST.
|
||||
// Image gen, TTS, STT, video are puter.js SDK-only (browser).
|
||||
baseUrl: "https://api.puter.com/puterai/openai/v1/chat/completions",
|
||||
authType: "apikey",
|
||||
authHeader: "bearer",
|
||||
models: [
|
||||
// OpenAI — use bare IDs
|
||||
{ id: "gpt-4o-mini", name: "GPT-4o Mini (🆓 Puter)" },
|
||||
{ id: "gpt-4o", name: "GPT-4o (Puter)" },
|
||||
{ id: "gpt-4.1", name: "GPT-4.1 (Puter)" },
|
||||
{ id: "gpt-4.1-mini", name: "GPT-4.1 Mini (Puter)" },
|
||||
{ id: "gpt-5-nano", name: "GPT-5 Nano (Puter)" },
|
||||
{ id: "gpt-5-mini", name: "GPT-5 Mini (Puter)" },
|
||||
{ id: "gpt-5", name: "GPT-5 (Puter)" },
|
||||
{ id: "o3-mini", name: "OpenAI o3-mini (Puter)" },
|
||||
{ id: "o3", name: "OpenAI o3 (Puter)" },
|
||||
{ id: "o4-mini", name: "OpenAI o4-mini (Puter)" },
|
||||
// Anthropic Claude — use bare IDs (confirmed working)
|
||||
{ id: "claude-haiku-4-5", name: "Claude Haiku 4.5 (Puter)" },
|
||||
{ id: "claude-sonnet-4-5", name: "Claude Sonnet 4.5 (Puter)" },
|
||||
{ id: "claude-opus-4-5", name: "Claude Opus 4.5 (Puter)" },
|
||||
{ id: "claude-sonnet-4", name: "Claude Sonnet 4 (Puter)" },
|
||||
{ id: "claude-opus-4", name: "Claude Opus 4 (Puter)" },
|
||||
// Google Gemini — use google/ prefix (confirmed working)
|
||||
{ id: "google/gemini-2.0-flash", name: "Gemini 2.0 Flash (Puter)" },
|
||||
{ id: "google/gemini-2.5-flash", name: "Gemini 2.5 Flash (Puter)" },
|
||||
{ id: "google/gemini-2.5-pro", name: "Gemini 2.5 Pro (Puter)" },
|
||||
{ id: "google/gemini-3-flash", name: "Gemini 3 Flash (Puter)" },
|
||||
{ id: "google/gemini-3-pro", name: "Gemini 3 Pro (Puter)" },
|
||||
// DeepSeek — use deepseek/ prefix (confirmed working)
|
||||
{ id: "deepseek/deepseek-chat", name: "DeepSeek Chat (Puter)" },
|
||||
{ id: "deepseek/deepseek-r1", name: "DeepSeek R1 (Puter)" },
|
||||
{ id: "deepseek/deepseek-v3.2", name: "DeepSeek V3.2 (Puter)" },
|
||||
// xAI Grok — use x-ai/ prefix
|
||||
{ id: "x-ai/grok-3", name: "Grok 3 (Puter)" },
|
||||
{ id: "x-ai/grok-3-mini", name: "Grok 3 Mini (Puter)" },
|
||||
{ id: "x-ai/grok-4", name: "Grok 4 (Puter)" },
|
||||
{ id: "x-ai/grok-4-fast", name: "Grok 4 Fast (Puter)" },
|
||||
// Meta Llama — bare IDs (confirmed ✅)
|
||||
{ id: "llama-4-scout", name: "Llama 4 Scout (Puter)" },
|
||||
{ id: "llama-4-maverick", name: "Llama 4 Maverick (Puter)" },
|
||||
{ id: "llama-3.3-70b-instruct", name: "Llama 3.3 70B (Puter)" },
|
||||
// Mistral — bare IDs (confirmed ✅)
|
||||
{ id: "mistral-small-latest", name: "Mistral Small (Puter)" },
|
||||
{ id: "mistral-medium-latest", name: "Mistral Medium (Puter)" },
|
||||
{ id: "open-mistral-nemo", name: "Mistral Nemo (Puter)" },
|
||||
// Qwen — use qwen/ prefix (confirmed ✅)
|
||||
{ id: "qwen/qwen3-235b-a22b", name: "Qwen3 235B (Puter)" },
|
||||
{ id: "qwen/qwen3-32b", name: "Qwen3 32B (Puter)" },
|
||||
{ id: "qwen/qwen3-coder", name: "Qwen3 Coder 480B (Puter)" },
|
||||
],
|
||||
passthroughModels: true, // 500+ models available — users can type any Puter model ID
|
||||
},
|
||||
|
||||
"cloudflare-ai": {
|
||||
id: "cloudflare-ai",
|
||||
alias: "cf",
|
||||
|
||||
@@ -3,6 +3,112 @@ import { CODEX_DEFAULT_INSTRUCTIONS } from "../config/codexInstructions.ts";
|
||||
import { PROVIDERS } from "../config/constants.ts";
|
||||
import { refreshCodexToken } from "../services/tokenRefresh.ts";
|
||||
|
||||
// ─── T09: Codex vs Spark Scope-Aware Rate Limiting ────────────────────────
|
||||
// Codex has two independent quota pools: "codex" (standard) and "spark" (premium).
|
||||
// Exhausting one should NOT block requests to the other.
|
||||
// Ref: sub2api PR #1129 (feat(openai): split codex spark rate limiting from codex)
|
||||
|
||||
/**
|
||||
* Maps model name substrings to their rate-limit scope.
|
||||
* Checked in order — first match wins.
|
||||
*/
|
||||
const CODEX_SCOPE_PATTERNS: Array<{ pattern: string; scope: "codex" | "spark" }> = [
|
||||
{ pattern: "codex-spark", scope: "spark" },
|
||||
{ pattern: "spark", scope: "spark" },
|
||||
{ pattern: "codex", scope: "codex" },
|
||||
{ pattern: "gpt-5", scope: "codex" }, // gpt-5.2-codex, gpt-5.3-codex, etc.
|
||||
];
|
||||
|
||||
/**
|
||||
* T09: Determine the rate-limit scope for a Codex model.
|
||||
* Use this key as the suffix for per-scope rate limit state:
|
||||
* `${accountId}:${getModelScope(model)}`
|
||||
*
|
||||
* @param model - The Codex model ID (e.g. "gpt-5.3-codex", "codex-spark-mini")
|
||||
* @returns "codex" | "spark"
|
||||
*/
|
||||
export function getCodexModelScope(model: string): "codex" | "spark" {
|
||||
const lower = model.toLowerCase();
|
||||
for (const { pattern, scope } of CODEX_SCOPE_PATTERNS) {
|
||||
if (lower.includes(pattern)) return scope;
|
||||
}
|
||||
return "codex"; // default scope
|
||||
}
|
||||
|
||||
/**
|
||||
* T09: Get the scope-keyed rate limit identifier for an account+model combination.
|
||||
* Use this as the key for rateLimitState maps to ensure scope isolation.
|
||||
*/
|
||||
export function getCodexRateLimitKey(accountId: string, model: string): string {
|
||||
return `${accountId}:${getCodexModelScope(model)}`;
|
||||
}
|
||||
|
||||
/**
|
||||
* T03: Parsed quota snapshot from Codex response headers.
|
||||
* Codex includes per-account usage windows that allow precise reset scheduling.
|
||||
* Ref: sub2api PR #357 (feat(oauth): persist usage snapshots and window cooldown)
|
||||
*/
|
||||
export interface CodexQuotaSnapshot {
|
||||
usage5h: number; // tokens used in 5h window
|
||||
limit5h: number; // token limit for 5h window
|
||||
resetAt5h: string | null; // ISO timestamp when 5h window resets
|
||||
usage7d: number; // tokens used in 7d window
|
||||
limit7d: number; // token limit for 7d window
|
||||
resetAt7d: string | null; // ISO timestamp when 7d window resets
|
||||
}
|
||||
|
||||
/**
|
||||
* T03: Parse Codex-specific quota headers from a provider response.
|
||||
* Returns null if none of the relevant headers are present.
|
||||
*
|
||||
* Extracts:
|
||||
* x-codex-5h-usage / x-codex-5h-limit / x-codex-5h-reset-at
|
||||
* x-codex-7d-usage / x-codex-7d-limit / x-codex-7d-reset-at
|
||||
*/
|
||||
export function parseCodexQuotaHeaders(headers: Headers): CodexQuotaSnapshot | null {
|
||||
const usage5h = headers.get("x-codex-5h-usage");
|
||||
const limit5h = headers.get("x-codex-5h-limit");
|
||||
const resetAt5h = headers.get("x-codex-5h-reset-at");
|
||||
const usage7d = headers.get("x-codex-7d-usage");
|
||||
const limit7d = headers.get("x-codex-7d-limit");
|
||||
const resetAt7d = headers.get("x-codex-7d-reset-at");
|
||||
|
||||
// Return null if none of the quota headers are present (not a quota-aware response)
|
||||
if (!usage5h && !limit5h && !resetAt5h && !usage7d && !limit7d && !resetAt7d) {
|
||||
return null;
|
||||
}
|
||||
|
||||
return {
|
||||
usage5h: usage5h ? parseFloat(usage5h) : 0,
|
||||
limit5h: limit5h ? parseFloat(limit5h) : Infinity,
|
||||
resetAt5h: resetAt5h ?? null,
|
||||
usage7d: usage7d ? parseFloat(usage7d) : 0,
|
||||
limit7d: limit7d ? parseFloat(limit7d) : Infinity,
|
||||
resetAt7d: resetAt7d ?? null,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* T03: Get the soonest quota reset time from a CodexQuotaSnapshot.
|
||||
* 7d window takes priority (wider window, harder limit) but we use whichever
|
||||
* is further in the future to avoid releasing the block too early.
|
||||
*
|
||||
* @returns Unix timestamp (ms) of the soonest effective reset, or null
|
||||
*/
|
||||
export function getCodexResetTime(quota: CodexQuotaSnapshot): number | null {
|
||||
const times: number[] = [];
|
||||
if (quota.resetAt7d) {
|
||||
const t = new Date(quota.resetAt7d).getTime();
|
||||
if (!isNaN(t) && t > Date.now()) times.push(t);
|
||||
}
|
||||
if (quota.resetAt5h) {
|
||||
const t = new Date(quota.resetAt5h).getTime();
|
||||
if (!isNaN(t) && t > Date.now()) times.push(t);
|
||||
}
|
||||
if (times.length === 0) return null;
|
||||
return Math.max(...times); // Use furthest-out reset to avoid premature unblock
|
||||
}
|
||||
|
||||
// Ordered list of effort levels from lowest to highest
|
||||
const EFFORT_ORDER = ["none", "low", "medium", "high", "xhigh"] as const;
|
||||
type EffortLevel = (typeof EFFORT_ORDER)[number];
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
import { BaseExecutor } from "./base.ts";
|
||||
import { BaseExecutor, ExecuteInput } from "./base.ts";
|
||||
import { PROVIDERS, OAUTH_ENDPOINTS } from "../config/constants.ts";
|
||||
import { getModelTargetFormat } from "../config/providerModels.ts";
|
||||
|
||||
@@ -19,15 +19,82 @@ export class GithubExecutor extends BaseExecutor {
|
||||
return this.config.baseUrl;
|
||||
}
|
||||
|
||||
injectResponseFormat(messages: any[], responseFormat: any) {
|
||||
if (!responseFormat) return messages;
|
||||
|
||||
let formatInstruction = "";
|
||||
if (responseFormat.type === "json_object") {
|
||||
formatInstruction =
|
||||
"Respond only with valid JSON. Do not include any text before or after the JSON object.";
|
||||
} else if (responseFormat.type === "json_schema" && responseFormat.json_schema) {
|
||||
formatInstruction = `Respond only with valid JSON matching this schema:\n${JSON.stringify(
|
||||
responseFormat.json_schema.schema,
|
||||
null,
|
||||
2
|
||||
)}\nDo not include any text before or after the JSON.`;
|
||||
}
|
||||
|
||||
if (!formatInstruction) return messages;
|
||||
|
||||
const systemIdx = messages.findIndex((m: any) => m.role === "system");
|
||||
if (systemIdx >= 0) {
|
||||
return messages.map((m: any, i: number) =>
|
||||
i === systemIdx ? { ...m, content: `${m.content}\n\n${formatInstruction}` } : m
|
||||
);
|
||||
}
|
||||
|
||||
return [{ role: "system", content: formatInstruction }, ...messages];
|
||||
}
|
||||
|
||||
transformRequest(model: string, body: any, stream: boolean, credentials: any): any {
|
||||
const modifiedBody = JSON.parse(JSON.stringify(body));
|
||||
if (modifiedBody.response_format && model.toLowerCase().includes("claude")) {
|
||||
modifiedBody.messages = this.injectResponseFormat(
|
||||
modifiedBody.messages,
|
||||
modifiedBody.response_format
|
||||
);
|
||||
delete modifiedBody.response_format;
|
||||
}
|
||||
return modifiedBody;
|
||||
}
|
||||
|
||||
async execute(input: ExecuteInput) {
|
||||
const result = await super.execute(input);
|
||||
if (!result || !result.response?.body) return result;
|
||||
|
||||
const isStreaming = input.stream === true;
|
||||
if (isStreaming) {
|
||||
const decoder = new TextDecoder();
|
||||
const transformStream = new TransformStream({
|
||||
transform(chunk, controller) {
|
||||
const text = decoder.decode(chunk, { stream: true });
|
||||
if (text.includes("data: [DONE]")) {
|
||||
return;
|
||||
}
|
||||
controller.enqueue(chunk);
|
||||
},
|
||||
});
|
||||
|
||||
const newResponse = new Response(result.response.body.pipeThrough(transformStream), {
|
||||
status: result.response.status,
|
||||
statusText: result.response.statusText,
|
||||
headers: result.response.headers, // Headers class carries over correctly
|
||||
});
|
||||
result.response = newResponse;
|
||||
}
|
||||
|
||||
return result;
|
||||
}
|
||||
|
||||
buildHeaders(credentials, stream = true) {
|
||||
const token = credentials.copilotToken || credentials.accessToken;
|
||||
return {
|
||||
Authorization: `Bearer ${token}`,
|
||||
"Content-Type": "application/json",
|
||||
"copilot-integration-id": "vscode-chat",
|
||||
"editor-version": "vscode/1.107.1",
|
||||
"editor-plugin-version": "copilot-chat/0.26.7",
|
||||
"user-agent": "GitHubCopilotChat/0.26.7",
|
||||
"editor-version": "vscode/1.110.0",
|
||||
"editor-plugin-version": "copilot-chat/0.38.0",
|
||||
"user-agent": "GitHubCopilotChat/0.38.0",
|
||||
"openai-intent": "conversation-panel",
|
||||
"x-github-api-version": "2025-04-01",
|
||||
"x-request-id":
|
||||
@@ -44,7 +111,7 @@ export class GithubExecutor extends BaseExecutor {
|
||||
headers: {
|
||||
Authorization: `token ${githubAccessToken}`,
|
||||
"User-Agent": "GithubCopilot/1.0",
|
||||
"Editor-Version": "vscode/1.100.0",
|
||||
"Editor-Version": "vscode/1.110.0",
|
||||
"Editor-Plugin-Version": "copilot/1.300.0",
|
||||
Accept: "application/json",
|
||||
},
|
||||
|
||||
@@ -9,6 +9,8 @@ import { DefaultExecutor } from "./default.ts";
|
||||
import { PollinationsExecutor } from "./pollinations.ts";
|
||||
import { CloudflareAIExecutor } from "./cloudflare-ai.ts";
|
||||
import { OpencodeExecutor } from "./opencode.ts";
|
||||
import { PuterExecutor } from "./puter.ts";
|
||||
import { VertexExecutor } from "./vertex.ts";
|
||||
|
||||
const executors = {
|
||||
antigravity: new AntigravityExecutor(),
|
||||
@@ -25,6 +27,9 @@ const executors = {
|
||||
cf: new CloudflareAIExecutor(), // Alias
|
||||
"opencode-zen": new OpencodeExecutor("opencode-zen"),
|
||||
"opencode-go": new OpencodeExecutor("opencode-go"),
|
||||
puter: new PuterExecutor(),
|
||||
pu: new PuterExecutor(), // Alias
|
||||
vertex: new VertexExecutor(),
|
||||
};
|
||||
|
||||
const defaultCache = new Map();
|
||||
@@ -51,3 +56,5 @@ export { DefaultExecutor } from "./default.ts";
|
||||
export { PollinationsExecutor } from "./pollinations.ts";
|
||||
export { CloudflareAIExecutor } from "./cloudflare-ai.ts";
|
||||
export { OpencodeExecutor } from "./opencode.ts";
|
||||
export { PuterExecutor } from "./puter.ts";
|
||||
export { VertexExecutor } from "./vertex.ts";
|
||||
|
||||
59
open-sse/executors/puter.ts
Normal file
59
open-sse/executors/puter.ts
Normal file
@@ -0,0 +1,59 @@
|
||||
import { BaseExecutor } from "./base.ts";
|
||||
import { PROVIDERS } from "../config/constants.ts";
|
||||
|
||||
/**
|
||||
* PuterExecutor — OpenAI-compatible proxy for Puter AI.
|
||||
*
|
||||
* Puter exposes 500+ models (GPT, Claude, Gemini, Grok, DeepSeek, Qwen, Mistral...)
|
||||
* through a single OpenAI-compatible REST endpoint.
|
||||
*
|
||||
* Endpoint: https://api.puter.com/puterai/openai/v1/chat/completions
|
||||
* Auth: Bearer <puter_auth_token> (from puter.com/dashboard → Copy Auth Token)
|
||||
* Docs: https://docs.puter.com/AI/
|
||||
*
|
||||
* Model ID examples:
|
||||
* OpenAI: "gpt-4o-mini", "gpt-4o", "gpt-4.1"
|
||||
* Claude: "claude-sonnet-4-5", "claude-opus-4", "claude-haiku-4-5"
|
||||
* Gemini: "google/gemini-2.0-flash", "google/gemini-2.5-pro"
|
||||
* DeepSeek: "deepseek/deepseek-chat", "deepseek/deepseek-r1"
|
||||
* Grok: "x-ai/grok-3", "x-ai/grok-4"
|
||||
* Mistral: "mistralai/mistral-small-3.2"
|
||||
* Meta: "meta-llama/llama-3.3-70b-instruct"
|
||||
*
|
||||
* Note: Image generation, TTS, STT, and video are puter.js SDK-only features.
|
||||
* Only text chat completions (with streaming SSE) are available via REST.
|
||||
*/
|
||||
export class PuterExecutor extends BaseExecutor {
|
||||
constructor() {
|
||||
super("puter", PROVIDERS["puter"] || { format: "openai" });
|
||||
}
|
||||
|
||||
buildUrl(_model: string, _stream: boolean, _urlIndex = 0, _credentials = null): string {
|
||||
return "https://api.puter.com/puterai/openai/v1/chat/completions";
|
||||
}
|
||||
|
||||
buildHeaders(credentials: any, stream = true): Record<string, string> {
|
||||
const headers: Record<string, string> = {
|
||||
"Content-Type": "application/json",
|
||||
};
|
||||
|
||||
const key = credentials?.apiKey || credentials?.accessToken;
|
||||
if (key) {
|
||||
headers["Authorization"] = `Bearer ${key}`;
|
||||
}
|
||||
|
||||
if (stream) {
|
||||
headers["Accept"] = "text/event-stream";
|
||||
}
|
||||
|
||||
return headers;
|
||||
}
|
||||
|
||||
transformRequest(model: string, body: any, _stream: boolean, _credentials: any): any {
|
||||
// Puter accepts model IDs directly from its catalog.
|
||||
// No transformation required — model string is passed as-is.
|
||||
return body;
|
||||
}
|
||||
}
|
||||
|
||||
export default PuterExecutor;
|
||||
150
open-sse/executors/vertex.ts
Normal file
150
open-sse/executors/vertex.ts
Normal file
@@ -0,0 +1,150 @@
|
||||
import { SignJWT, importPKCS8 } from "jose";
|
||||
import { BaseExecutor, ExecuteInput } from "./base.ts";
|
||||
import { PROVIDERS } from "../config/constants.ts";
|
||||
|
||||
interface ServiceAccount {
|
||||
type: string;
|
||||
project_id: string;
|
||||
private_key_id: string;
|
||||
private_key: string;
|
||||
client_email: string;
|
||||
[key: string]: unknown;
|
||||
}
|
||||
|
||||
const TOKEN_CACHE = new Map<string, { token: string; expiresAt: number }>();
|
||||
|
||||
function parseSAFromApiKey(apiKey: string): ServiceAccount {
|
||||
try {
|
||||
return JSON.parse(apiKey);
|
||||
} catch {
|
||||
throw new Error("Vertex AI requires a valid Service Account JSON as the API key");
|
||||
}
|
||||
}
|
||||
|
||||
async function getAccessToken(sa: ServiceAccount): Promise<string> {
|
||||
if (!sa.client_email || !sa.private_key) {
|
||||
throw new Error(
|
||||
"Service Account JSON is missing required fields (client_email or private_key)"
|
||||
);
|
||||
}
|
||||
|
||||
const cacheKey = sa.client_email;
|
||||
const cached = TOKEN_CACHE.get(cacheKey);
|
||||
|
||||
// Buffer of 60 seconds
|
||||
if (cached && Date.now() < cached.expiresAt - 60_000) {
|
||||
return cached.token;
|
||||
}
|
||||
|
||||
const privateKey = await importPKCS8(sa.private_key, "RS256");
|
||||
const now = Math.floor(Date.now() / 1000);
|
||||
|
||||
const jwt = await new SignJWT({
|
||||
iss: sa.client_email,
|
||||
sub: sa.client_email,
|
||||
aud: "https://oauth2.googleapis.com/token",
|
||||
iat: now,
|
||||
exp: now + 3600,
|
||||
scope: "https://www.googleapis.com/auth/cloud-platform",
|
||||
})
|
||||
.setProtectedHeader({ alg: "RS256", kid: sa.private_key_id })
|
||||
.sign(privateKey);
|
||||
|
||||
const tokenRes = await fetch("https://oauth2.googleapis.com/token", {
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/x-www-form-urlencoded" },
|
||||
body: new URLSearchParams({
|
||||
grant_type: "urn:ietf:params:oauth:grant-type:jwt-bearer",
|
||||
assertion: jwt,
|
||||
}),
|
||||
});
|
||||
|
||||
if (!tokenRes.ok) {
|
||||
const errorText = await tokenRes.text();
|
||||
throw new Error(
|
||||
`Failed to exchange JWT for Vertex access token: ${tokenRes.status} ${errorText}`
|
||||
);
|
||||
}
|
||||
|
||||
const tokenData = await tokenRes.json();
|
||||
const accessToken = tokenData.access_token;
|
||||
|
||||
if (!accessToken) {
|
||||
throw new Error("Vertex AI token exchange succeeded but no access_token found");
|
||||
}
|
||||
|
||||
TOKEN_CACHE.set(cacheKey, {
|
||||
token: accessToken,
|
||||
expiresAt: (now + 3600) * 1000,
|
||||
});
|
||||
|
||||
return accessToken;
|
||||
}
|
||||
|
||||
const PARTNER_MODELS = new Set([
|
||||
"claude-3-5-sonnet",
|
||||
"claude-3-opus",
|
||||
"claude-3-haiku",
|
||||
"deepseek-v3",
|
||||
"deepseek-v3.2",
|
||||
"deepseek-deepseek-r1",
|
||||
"qwen3-next-80b",
|
||||
"llama-3.1",
|
||||
"mistral-",
|
||||
"glm-5",
|
||||
"meta/llama",
|
||||
]);
|
||||
|
||||
function isPartnerModel(model: string) {
|
||||
return [...PARTNER_MODELS].some((prefix) => model.startsWith(prefix));
|
||||
}
|
||||
|
||||
export class VertexExecutor extends BaseExecutor {
|
||||
constructor() {
|
||||
super("vertex", PROVIDERS.vertex);
|
||||
}
|
||||
|
||||
async execute(input: ExecuteInput) {
|
||||
const { credentials, log } = input;
|
||||
if (credentials.apiKey && !credentials.accessToken) {
|
||||
try {
|
||||
const sa = parseSAFromApiKey(credentials.apiKey);
|
||||
credentials.accessToken = await getAccessToken(sa);
|
||||
} catch (err: any) {
|
||||
log?.error?.("VERTEX", `Failed to generate JWT token: ${err.message}`);
|
||||
throw err;
|
||||
}
|
||||
}
|
||||
return super.execute(input);
|
||||
}
|
||||
|
||||
buildUrl(model: string, stream: boolean, urlIndex = 0, credentials: any = null) {
|
||||
const region = credentials?.providerSpecificData?.region || "us-central1";
|
||||
let project = "unknown-project";
|
||||
|
||||
if (credentials?.apiKey) {
|
||||
try {
|
||||
const sa = parseSAFromApiKey(credentials.apiKey);
|
||||
if (sa.project_id) project = sa.project_id;
|
||||
} catch {
|
||||
// Ignored, handled in execute
|
||||
}
|
||||
}
|
||||
|
||||
if (isPartnerModel(model)) {
|
||||
return `https://aiplatform.googleapis.com/v1/projects/${project}/locations/global/endpoints/openapi/chat/completions`;
|
||||
}
|
||||
return `https://aiplatform.googleapis.com/v1/projects/${project}/locations/${region}/publishers/google/models/${model}:${stream ? "streamGenerateContent?alt=sse" : "generateContent"}`;
|
||||
}
|
||||
|
||||
buildHeaders(credentials: any, stream = true) {
|
||||
const headers: Record<string, string> = { "Content-Type": "application/json" };
|
||||
if (credentials.accessToken) {
|
||||
headers["Authorization"] = `Bearer ${credentials.accessToken}`;
|
||||
}
|
||||
if (stream) {
|
||||
headers["Accept"] = "text/event-stream";
|
||||
}
|
||||
return headers;
|
||||
}
|
||||
}
|
||||
@@ -16,6 +16,8 @@ import { resolveModelAlias } from "../services/modelDeprecation.ts";
|
||||
import { getUnsupportedParams } from "../config/providerRegistry.ts";
|
||||
import { createErrorResult, parseUpstreamError, formatProviderError } from "../utils/error.ts";
|
||||
import { HTTP_STATUS } from "../config/constants.ts";
|
||||
import { classifyProviderError, PROVIDER_ERROR_TYPES } from "../services/errorClassifier.ts";
|
||||
import { updateProviderConnection } from "@/lib/db/providers";
|
||||
import { handleBypassRequest } from "../utils/bypassHandler.ts";
|
||||
import {
|
||||
saveRequestUsage,
|
||||
@@ -25,6 +27,11 @@ import {
|
||||
} from "@/lib/usageDb";
|
||||
import { getModelNormalizeToolCallId, getModelPreserveOpenAIDeveloperRole } from "@/lib/localDb";
|
||||
import { getExecutor } from "../executors/index.ts";
|
||||
import {
|
||||
parseCodexQuotaHeaders,
|
||||
getCodexResetTime,
|
||||
getCodexModelScope,
|
||||
} from "../executors/codex.ts";
|
||||
import { translateNonStreamingResponse } from "./responseTranslator.ts";
|
||||
import { extractUsageFromResponse } from "./usageExtractor.ts";
|
||||
import { parseSSEToOpenAIResponse, parseSSEToResponsesOutput } from "./sseParser.ts";
|
||||
@@ -44,6 +51,11 @@ import { getIdempotencyKey, checkIdempotency, saveIdempotency } from "@/lib/idem
|
||||
import { createProgressTransform, wantsProgress } from "../utils/progressTracker.ts";
|
||||
import { isModelUnavailableError, getNextFamilyFallback } from "../services/modelFamilyFallback.ts";
|
||||
import { computeRequestHash, deduplicate, shouldDeduplicate } from "../services/requestDedup.ts";
|
||||
import {
|
||||
isBackgroundTask,
|
||||
getDegradedModel,
|
||||
getBackgroundDegradationConfig,
|
||||
} from "../services/backgroundTaskDetector.ts";
|
||||
import {
|
||||
shouldUseFallback,
|
||||
isFallbackDecision,
|
||||
@@ -93,7 +105,9 @@ export async function handleChatCore({
|
||||
userAgent,
|
||||
comboName,
|
||||
}) {
|
||||
const { provider, model, extendedContext } = modelInfo;
|
||||
let { provider, model, extendedContext } = modelInfo;
|
||||
const requestedModel =
|
||||
typeof body?.model === "string" && body.model.trim().length > 0 ? body.model : model;
|
||||
const startTime = Date.now();
|
||||
const persistFailureUsage = (statusCode: number, errorCode?: string | null) => {
|
||||
saveRequestUsage({
|
||||
@@ -112,6 +126,67 @@ export async function handleChatCore({
|
||||
}).catch(() => {});
|
||||
};
|
||||
|
||||
const persistCodexQuotaState = async (
|
||||
headers: Headers | Record<string, string> | null,
|
||||
status = 0
|
||||
) => {
|
||||
if (provider !== "codex" || !connectionId || !headers) return;
|
||||
|
||||
try {
|
||||
const quota = parseCodexQuotaHeaders(headers as Headers);
|
||||
if (!quota) return;
|
||||
|
||||
const existingProviderData =
|
||||
credentials?.providerSpecificData && typeof credentials.providerSpecificData === "object"
|
||||
? credentials.providerSpecificData
|
||||
: {};
|
||||
const scope = getCodexModelScope(model || requestedModel || "");
|
||||
const quotaState = {
|
||||
usage5h: quota.usage5h,
|
||||
limit5h: quota.limit5h,
|
||||
resetAt5h: quota.resetAt5h,
|
||||
usage7d: quota.usage7d,
|
||||
limit7d: quota.limit7d,
|
||||
resetAt7d: quota.resetAt7d,
|
||||
scope,
|
||||
updatedAt: new Date().toISOString(),
|
||||
};
|
||||
|
||||
const nextProviderData: Record<string, unknown> = {
|
||||
...existingProviderData,
|
||||
codexQuotaState: quotaState,
|
||||
};
|
||||
|
||||
// T03/T09: on 429, persist exact reset time per scope to avoid global over-blocking.
|
||||
if (status === 429) {
|
||||
const resetTimeMs = getCodexResetTime(quota);
|
||||
if (resetTimeMs && resetTimeMs > Date.now()) {
|
||||
const scopeUntil = new Date(resetTimeMs).toISOString();
|
||||
const scopeMapRaw =
|
||||
existingProviderData &&
|
||||
typeof existingProviderData === "object" &&
|
||||
existingProviderData.codexScopeRateLimitedUntil &&
|
||||
typeof existingProviderData.codexScopeRateLimitedUntil === "object"
|
||||
? existingProviderData.codexScopeRateLimitedUntil
|
||||
: {};
|
||||
|
||||
nextProviderData.codexScopeRateLimitedUntil = {
|
||||
...(scopeMapRaw as Record<string, unknown>),
|
||||
[scope]: scopeUntil,
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
await updateProviderConnection(connectionId, {
|
||||
providerSpecificData: nextProviderData,
|
||||
});
|
||||
|
||||
credentials.providerSpecificData = nextProviderData;
|
||||
} catch (err) {
|
||||
log?.debug?.("CODEX", `Failed to persist codex quota state: ${err?.message || err}`);
|
||||
}
|
||||
};
|
||||
|
||||
// ── Phase 9.2: Idempotency check ──
|
||||
const idempotencyKey = getIdempotencyKey(clientRawRequest?.headers);
|
||||
const cachedIdemp = checkIdempotency(idempotencyKey);
|
||||
@@ -157,6 +232,22 @@ export async function handleChatCore({
|
||||
// Detect source format and get target format
|
||||
// Model-specific targetFormat takes priority over provider default
|
||||
|
||||
// ── Background Task Redirection (T41) ──
|
||||
const bgConfig = getBackgroundDegradationConfig();
|
||||
if (bgConfig.enabled && isBackgroundTask(body, clientRawRequest?.headers)) {
|
||||
const degradedModel = getDegradedModel(model);
|
||||
if (degradedModel !== model) {
|
||||
log?.info?.(
|
||||
"BACKGROUND",
|
||||
`Background task detected: Redirecting ${model} → ${degradedModel}`
|
||||
);
|
||||
model = degradedModel;
|
||||
if (body && typeof body === "object") {
|
||||
body.model = model;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Apply custom model aliases (Settings → Model Aliases → Pattern→Target) before routing (#315, #472)
|
||||
// Custom aliases take priority over built-in and must be resolved here so the
|
||||
// downstream getModelTargetFormat() lookup AND the actual provider request use
|
||||
@@ -173,7 +264,17 @@ export async function handleChatCore({
|
||||
const targetFormat = modelTargetFormat || getTargetFormat(provider);
|
||||
|
||||
// Default to false unless client explicitly sets stream: true (OpenAI spec compliant)
|
||||
const stream = body.stream === true;
|
||||
const acceptHeader =
|
||||
clientRawRequest?.headers && typeof clientRawRequest.headers.get === "function"
|
||||
? clientRawRequest.headers.get("accept") || clientRawRequest.headers.get("Accept")
|
||||
: (clientRawRequest?.headers || {})["accept"] || (clientRawRequest?.headers || {})["Accept"];
|
||||
|
||||
const clientWantsJson =
|
||||
typeof acceptHeader === "string" &&
|
||||
acceptHeader.includes("application/json") &&
|
||||
!acceptHeader.includes("text/event-stream");
|
||||
|
||||
const stream = body.stream === true && !clientWantsJson;
|
||||
|
||||
// ── Phase 9.1: Semantic cache check (non-streaming, temp=0 only) ──
|
||||
if (isCacheable(body, clientRawRequest?.headers)) {
|
||||
@@ -457,7 +558,7 @@ export async function handleChatCore({
|
||||
// Non-stream responses need cloning for shared dedup consumers.
|
||||
const status = rawResult.response.status;
|
||||
const statusText = rawResult.response.statusText;
|
||||
const headers = Array.from(rawResult.response.headers.entries());
|
||||
const headers = Array.from(rawResult.response.headers.entries()) as [string, string][];
|
||||
const payload = await rawResult.response.text();
|
||||
|
||||
return {
|
||||
@@ -532,6 +633,7 @@ export async function handleChatCore({
|
||||
path: clientRawRequest?.endpoint || "/v1/chat/completions",
|
||||
status: error.name === "AbortError" ? 499 : HTTP_STATUS.BAD_GATEWAY,
|
||||
model,
|
||||
requestedModel,
|
||||
provider,
|
||||
connectionId,
|
||||
duration: Date.now() - startTime,
|
||||
@@ -603,6 +705,8 @@ export async function handleChatCore({
|
||||
}
|
||||
}
|
||||
|
||||
await persistCodexQuotaState(providerResponse.headers, providerResponse.status);
|
||||
|
||||
// Check provider response - return error info for fallback handling
|
||||
if (!providerResponse.ok) {
|
||||
trackPendingRequest(model, provider, connectionId, false);
|
||||
@@ -610,6 +714,54 @@ export async function handleChatCore({
|
||||
providerResponse,
|
||||
provider
|
||||
);
|
||||
|
||||
// T06/T10/T36: classify provider errors and persist terminal account states.
|
||||
const errorType = classifyProviderError(statusCode, message);
|
||||
if (connectionId && errorType) {
|
||||
try {
|
||||
if (errorType === PROVIDER_ERROR_TYPES.FORBIDDEN) {
|
||||
await updateProviderConnection(connectionId, {
|
||||
isActive: false,
|
||||
testStatus: "banned",
|
||||
lastErrorType: errorType,
|
||||
lastError: message,
|
||||
errorCode: statusCode,
|
||||
});
|
||||
console.warn(
|
||||
`[provider] Node ${connectionId} banned (${statusCode}) — disabling permanently`
|
||||
);
|
||||
} else if (errorType === PROVIDER_ERROR_TYPES.QUOTA_EXHAUSTED) {
|
||||
await updateProviderConnection(connectionId, {
|
||||
testStatus: "credits_exhausted",
|
||||
lastErrorType: errorType,
|
||||
lastError: message,
|
||||
errorCode: statusCode,
|
||||
});
|
||||
console.warn(`[provider] Node ${connectionId} exhausted quota (${statusCode})`);
|
||||
} else if (errorType === PROVIDER_ERROR_TYPES.ACCOUNT_DEACTIVATED) {
|
||||
await updateProviderConnection(connectionId, {
|
||||
isActive: false,
|
||||
testStatus: "expired",
|
||||
lastErrorType: errorType,
|
||||
lastError: message,
|
||||
errorCode: statusCode,
|
||||
});
|
||||
console.warn(
|
||||
`[provider] Node ${connectionId} account deactivated (${statusCode}) — marked expired`
|
||||
);
|
||||
} else if (errorType === PROVIDER_ERROR_TYPES.UNAUTHORIZED) {
|
||||
// Normal 401 (token/session auth issue): keep account active for refresh/re-auth.
|
||||
await updateProviderConnection(connectionId, {
|
||||
lastErrorType: errorType,
|
||||
lastError: message,
|
||||
errorCode: statusCode,
|
||||
});
|
||||
}
|
||||
} catch {
|
||||
// Best-effort state update; request flow should continue with fallback handling.
|
||||
}
|
||||
}
|
||||
|
||||
appendRequestLog({ model, provider, connectionId, status: `FAILED ${statusCode}` }).catch(
|
||||
() => {}
|
||||
);
|
||||
@@ -618,6 +770,7 @@ export async function handleChatCore({
|
||||
path: clientRawRequest?.endpoint || "/v1/chat/completions",
|
||||
status: statusCode,
|
||||
model,
|
||||
requestedModel,
|
||||
provider,
|
||||
connectionId,
|
||||
duration: Date.now() - startTime,
|
||||
@@ -808,6 +961,7 @@ export async function handleChatCore({
|
||||
path: clientRawRequest?.endpoint || "/v1/chat/completions",
|
||||
status: 200,
|
||||
model,
|
||||
requestedModel,
|
||||
provider,
|
||||
connectionId,
|
||||
duration: Date.now() - startTime,
|
||||
@@ -848,6 +1002,32 @@ export async function handleChatCore({
|
||||
? translateNonStreamingResponse(responseBody, targetFormat, sourceFormat)
|
||||
: responseBody;
|
||||
|
||||
// T26: Strip markdown code blocks if provider format is Claude
|
||||
if (sourceFormat === "claude" && !stream) {
|
||||
if (translatedResponse?.choices?.[0]?.message?.content) {
|
||||
const text = translatedResponse.choices[0].message.content;
|
||||
const codeBlockRegex =
|
||||
/^```(?:json|javascript|typescript|js|ts)?\s*\n?([\s\S]*?)\n?```\s*$/;
|
||||
const match = text.trim().match(codeBlockRegex);
|
||||
if (match) {
|
||||
translatedResponse.choices[0].message.content = match[1].trim();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// T18: Normalize finish_reason to 'tool_calls' if tool calls are present
|
||||
if (translatedResponse?.choices) {
|
||||
for (const choice of translatedResponse.choices) {
|
||||
if (
|
||||
choice.message?.tool_calls &&
|
||||
choice.message.tool_calls.length > 0 &&
|
||||
choice.finish_reason !== "tool_calls"
|
||||
) {
|
||||
choice.finish_reason = "tool_calls";
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Sanitize response for OpenAI SDK compatibility
|
||||
// Strips non-standard fields (x_groq, usage_breakdown, service_tier, etc.)
|
||||
// Extracts <think> tags into reasoning_content
|
||||
@@ -921,6 +1101,7 @@ export async function handleChatCore({
|
||||
path: clientRawRequest?.endpoint || "/v1/chat/completions",
|
||||
status: streamStatus || 200,
|
||||
model,
|
||||
requestedModel,
|
||||
provider,
|
||||
connectionId,
|
||||
duration: Date.now() - startTime,
|
||||
|
||||
@@ -16,6 +16,7 @@
|
||||
*/
|
||||
|
||||
import { getImageProvider, parseImageModel } from "../config/imageRegistry.ts";
|
||||
import { mapImageSize } from "../translator/image/sizeMapper.ts";
|
||||
import { saveCallLog } from "@/lib/usageDb";
|
||||
import {
|
||||
submitComfyWorkflow,
|
||||
@@ -95,11 +96,21 @@ export async function handleImageGeneration({ body, credentials, log, resolvedPr
|
||||
});
|
||||
}
|
||||
|
||||
// Route to format-specific handler
|
||||
if (providerConfig.format === "gemini-image") {
|
||||
return handleGeminiImageGeneration({ model, providerConfig, body, credentials, log });
|
||||
}
|
||||
|
||||
if (providerConfig.format === "imagen3") {
|
||||
return handleImagen3ImageGeneration({
|
||||
model,
|
||||
provider,
|
||||
providerConfig,
|
||||
body,
|
||||
credentials,
|
||||
log,
|
||||
});
|
||||
}
|
||||
|
||||
if (providerConfig.format === "hyperbolic") {
|
||||
return handleHyperbolicImageGeneration({
|
||||
model,
|
||||
@@ -539,7 +550,7 @@ async function handleNanoBananaImageGeneration({
|
||||
? body.aspectRatio
|
||||
: typeof body.aspect_ratio === "string"
|
||||
? body.aspect_ratio
|
||||
: inferAspectRatioFromSize(body.size) || "1:1";
|
||||
: mapImageSize(body.size);
|
||||
|
||||
let resolution =
|
||||
typeof body.resolution === "string"
|
||||
@@ -856,18 +867,6 @@ async function normalizeNanoBananaTaskResult(taskData, body, log) {
|
||||
return [];
|
||||
}
|
||||
|
||||
function inferAspectRatioFromSize(size) {
|
||||
if (typeof size !== "string") return null;
|
||||
const [wRaw, hRaw] = size.split("x");
|
||||
const width = Number(wRaw);
|
||||
const height = Number(hRaw);
|
||||
if (!Number.isFinite(width) || !Number.isFinite(height) || width <= 0 || height <= 0) return null;
|
||||
|
||||
const gcd = (a, b) => (b === 0 ? a : gcd(b, a % b));
|
||||
const div = gcd(Math.round(width), Math.round(height));
|
||||
return `${Math.round(width / div)}:${Math.round(height / div)}`;
|
||||
}
|
||||
|
||||
function inferResolutionFromSize(size) {
|
||||
if (typeof size !== "string") return null;
|
||||
const [wRaw, hRaw] = size.split("x");
|
||||
@@ -1081,3 +1080,113 @@ async function handleComfyUIImageGeneration({ model, provider, providerConfig, b
|
||||
return { success: false, status: 502, error: `Image provider error: ${err.message}` };
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Handle Imagen 3 image generation
|
||||
*/
|
||||
async function handleImagen3ImageGeneration({
|
||||
model,
|
||||
provider,
|
||||
providerConfig,
|
||||
body,
|
||||
credentials,
|
||||
log,
|
||||
}: any) {
|
||||
const startTime = Date.now();
|
||||
const token = credentials.apiKey || credentials.accessToken;
|
||||
const aspectRatio = mapImageSize(body.size);
|
||||
|
||||
const upstreamBody = {
|
||||
prompt: body.prompt,
|
||||
aspect_ratio: aspectRatio,
|
||||
number_of_images: body.n ?? 1,
|
||||
};
|
||||
|
||||
if (log) {
|
||||
const promptPreview = String(body.prompt ?? "").slice(0, 60);
|
||||
log.info(
|
||||
"IMAGE",
|
||||
`${provider}/${model} (imagen3) | prompt: "${promptPreview}..." | aspect_ratio: ${aspectRatio}`
|
||||
);
|
||||
}
|
||||
|
||||
try {
|
||||
const response = await fetch(providerConfig.baseUrl, {
|
||||
method: "POST",
|
||||
headers: {
|
||||
"Content-Type": "application/json",
|
||||
Authorization: `Bearer ${token}`,
|
||||
},
|
||||
body: JSON.stringify(upstreamBody),
|
||||
});
|
||||
|
||||
if (!response.ok) {
|
||||
const errorText = await response.text();
|
||||
if (log)
|
||||
log.error("IMAGE", `${provider} error ${response.status}: ${errorText.slice(0, 200)}`);
|
||||
|
||||
saveCallLog({
|
||||
method: "POST",
|
||||
path: "/v1/images/generations",
|
||||
status: response.status,
|
||||
model: `${provider}/${model}`,
|
||||
provider,
|
||||
duration: Date.now() - startTime,
|
||||
error: errorText.slice(0, 500),
|
||||
requestBody: upstreamBody,
|
||||
}).catch(() => {});
|
||||
|
||||
return { success: false, status: response.status, error: errorText };
|
||||
}
|
||||
|
||||
const data = await response.json();
|
||||
|
||||
// Normalize response to OpenAI format
|
||||
const images: any[] = [];
|
||||
if (Array.isArray(data.images)) {
|
||||
images.push(
|
||||
...data.images.map((img: any) => ({
|
||||
b64_json: img.image || img.b64_json || img.url || img,
|
||||
revised_prompt: body.prompt,
|
||||
}))
|
||||
);
|
||||
} else if (Array.isArray(data.data)) {
|
||||
images.push(...data.data);
|
||||
} else if (data.url || data.b64_json || data.image) {
|
||||
images.push({
|
||||
b64_json: data.image || data.b64_json || data.url,
|
||||
url: data.url,
|
||||
revised_prompt: body.prompt,
|
||||
});
|
||||
}
|
||||
|
||||
saveCallLog({
|
||||
method: "POST",
|
||||
path: "/v1/images/generations",
|
||||
status: 200,
|
||||
model: `${provider}/${model}`,
|
||||
provider,
|
||||
duration: Date.now() - startTime,
|
||||
responseBody: { images_count: images.length },
|
||||
}).catch(() => {});
|
||||
|
||||
return {
|
||||
success: true,
|
||||
data: { created: data.created || Math.floor(Date.now() / 1000), data: images },
|
||||
};
|
||||
} catch (err: any) {
|
||||
if (log) log.error("IMAGE", `${provider} fetch error: ${err.message}`);
|
||||
|
||||
saveCallLog({
|
||||
method: "POST",
|
||||
path: "/v1/images/generations",
|
||||
status: 502,
|
||||
model: `${provider}/${model}`,
|
||||
provider,
|
||||
duration: Date.now() - startTime,
|
||||
error: err.message,
|
||||
}).catch(() => {});
|
||||
|
||||
return { success: false, status: 502, error: `Image provider error: ${err.message}` };
|
||||
}
|
||||
}
|
||||
|
||||
@@ -103,6 +103,25 @@ export function translateNonStreamingResponse(
|
||||
message.content = "";
|
||||
}
|
||||
|
||||
if (process.env.DEBUG_RESPONSES_SSE_TO_JSON === "true") {
|
||||
const msgItems = output.filter((i) => toRecord(i).type === "message");
|
||||
console.log(`[ResponsesSSE] ${output.length} output items, ${msgItems.length} message items`);
|
||||
msgItems.forEach((item, idx) => {
|
||||
const itemObj = toRecord(item);
|
||||
let textLen = 0;
|
||||
if (Array.isArray(itemObj.content)) {
|
||||
for (const part of itemObj.content) {
|
||||
const partObj = toRecord(part);
|
||||
if (partObj.type === "output_text" && typeof partObj.text === "string") {
|
||||
textLen += partObj.text.length;
|
||||
}
|
||||
}
|
||||
}
|
||||
console.log(` [${idx}] text length: ${textLen}`);
|
||||
});
|
||||
console.log(` → Final text content length: ${textContent.length}`);
|
||||
}
|
||||
|
||||
const createdAt = toNumber(response.created_at, Math.floor(Date.now() / 1000));
|
||||
const model = toString(response.model || responseRoot.model, "openai-responses");
|
||||
const finishReason = toolCalls.length > 0 ? "tool_calls" : "stop";
|
||||
|
||||
@@ -23,9 +23,18 @@ export function parseSSEToOpenAIResponse(rawSSE, fallbackModel) {
|
||||
const first = chunks[0];
|
||||
const contentParts = [];
|
||||
const reasoningParts = [];
|
||||
const accumulatedToolCalls = new Map<string, any>();
|
||||
let unknownToolCallSeq = 0;
|
||||
let finishReason = "stop";
|
||||
let usage = null;
|
||||
|
||||
const getToolCallKey = (toolCall: any) => {
|
||||
if (toolCall?.id) return `id:${toolCall.id}`;
|
||||
if (Number.isInteger(toolCall?.index)) return `idx:${toolCall.index}`;
|
||||
unknownToolCallSeq += 1;
|
||||
return `seq:${unknownToolCallSeq}`;
|
||||
};
|
||||
|
||||
for (const chunk of chunks) {
|
||||
const choice = chunk?.choices?.[0];
|
||||
const delta = choice?.delta || {};
|
||||
@@ -36,6 +45,40 @@ export function parseSSEToOpenAIResponse(rawSSE, fallbackModel) {
|
||||
if (typeof delta.reasoning_content === "string" && delta.reasoning_content.length > 0) {
|
||||
reasoningParts.push(delta.reasoning_content);
|
||||
}
|
||||
|
||||
// T18: Accumulate tool calls correctly across streamed chunks
|
||||
if (delta.tool_calls) {
|
||||
for (const tc of delta.tool_calls) {
|
||||
const key = getToolCallKey(tc);
|
||||
const existing = accumulatedToolCalls.get(key);
|
||||
const deltaArgs = typeof tc?.function?.arguments === "string" ? tc.function.arguments : "";
|
||||
|
||||
if (!existing) {
|
||||
accumulatedToolCalls.set(key, {
|
||||
id: tc?.id ?? null,
|
||||
index: Number.isInteger(tc?.index) ? tc.index : accumulatedToolCalls.size,
|
||||
type: tc?.type || "function",
|
||||
function: {
|
||||
name: tc?.function?.name || "unknown",
|
||||
arguments: deltaArgs,
|
||||
},
|
||||
});
|
||||
} else {
|
||||
existing.id = existing.id || tc?.id || null;
|
||||
if (!Number.isInteger(existing.index) && Number.isInteger(tc?.index)) {
|
||||
existing.index = tc.index;
|
||||
}
|
||||
if (tc?.function?.name && !existing.function?.name) {
|
||||
existing.function = existing.function || {};
|
||||
existing.function.name = tc.function.name;
|
||||
}
|
||||
existing.function = existing.function || {};
|
||||
existing.function.arguments = `${existing.function.arguments || ""}${deltaArgs}`;
|
||||
accumulatedToolCalls.set(key, existing);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if (choice?.finish_reason) {
|
||||
finishReason = choice.finish_reason;
|
||||
}
|
||||
@@ -46,12 +89,22 @@ export function parseSSEToOpenAIResponse(rawSSE, fallbackModel) {
|
||||
|
||||
const message: Record<string, unknown> = {
|
||||
role: "assistant",
|
||||
content: contentParts.join(""),
|
||||
content: contentParts.length > 0 ? contentParts.join("") : null,
|
||||
};
|
||||
if (reasoningParts.length > 0) {
|
||||
message.reasoning_content = reasoningParts.join("");
|
||||
}
|
||||
|
||||
const finalToolCalls = [...accumulatedToolCalls.values()].filter(Boolean).sort((a, b) => {
|
||||
const ai = Number.isInteger(a?.index) ? a.index : 0;
|
||||
const bi = Number.isInteger(b?.index) ? b.index : 0;
|
||||
return ai - bi;
|
||||
});
|
||||
if (finalToolCalls.length > 0) {
|
||||
finishReason = "tool_calls"; // T18 normalization
|
||||
message.tool_calls = finalToolCalls;
|
||||
}
|
||||
|
||||
const result: Record<string, unknown> = {
|
||||
id: first.id || `chatcmpl-${Date.now()}`,
|
||||
object: "chat.completion",
|
||||
|
||||
@@ -433,23 +433,48 @@ async function handleListModelsCatalog(args: { provider?: string; capability?: s
|
||||
const start = Date.now();
|
||||
try {
|
||||
let path = "/v1/models";
|
||||
const params = new URLSearchParams();
|
||||
if (args.provider) params.set("provider", args.provider);
|
||||
if (args.capability) params.set("capability", args.capability);
|
||||
if (params.toString()) path += `?${params.toString()}`;
|
||||
let isProviderSpecific = false;
|
||||
let source = "local_catalog";
|
||||
let warning = undefined;
|
||||
|
||||
if (args.provider && !args.capability) {
|
||||
// Use direct provider fetch to get real-time API status
|
||||
path = `/api/providers/${encodeURIComponent(args.provider)}/models`;
|
||||
isProviderSpecific = true;
|
||||
} else {
|
||||
const params = new URLSearchParams();
|
||||
if (args.provider) params.set("provider", args.provider);
|
||||
if (args.capability) params.set("capability", args.capability);
|
||||
if (params.toString()) path += `?${params.toString()}`;
|
||||
}
|
||||
|
||||
const raw = toRecord(await omniRouteFetch(path));
|
||||
|
||||
// If we used the direct provider endpoint
|
||||
let rawModels = [];
|
||||
if (isProviderSpecific) {
|
||||
rawModels = Array.isArray(raw.models) ? raw.models : [];
|
||||
source = typeof raw.source === "string" ? raw.source : "api";
|
||||
if (raw.warning) warning = String(raw.warning);
|
||||
} else {
|
||||
rawModels = Array.isArray(raw.data) ? raw.data : [];
|
||||
source = "local_catalog";
|
||||
// OmniRoute's global /v1/models is always a cached/local catalog
|
||||
}
|
||||
|
||||
const result = {
|
||||
models: toArray(raw.data).map((rawModel) => {
|
||||
models: rawModels.map((rawModel) => {
|
||||
const model = toRecord(rawModel);
|
||||
return {
|
||||
id: toString(model.id, ""),
|
||||
provider: toString(model.owned_by, toString(model.provider, "unknown")),
|
||||
provider: toString(model.owned_by, toString(model.provider, args.provider || "unknown")),
|
||||
capabilities: toStringArray(model.capabilities, ["chat"]),
|
||||
status: toString(model.status, "available"),
|
||||
pricing: model.pricing,
|
||||
};
|
||||
}),
|
||||
source,
|
||||
...(warning ? { warning } : {}),
|
||||
};
|
||||
|
||||
await logToolCall(
|
||||
|
||||
@@ -8,6 +8,46 @@ import {
|
||||
} from "../config/constants.ts";
|
||||
import { getProviderCategory } from "../config/providerRegistry.ts";
|
||||
|
||||
// T06 (sub2api PR #1037): Signals that indicate permanent account deactivation.
|
||||
// When a 401 body contains these strings, the account is permanently dead
|
||||
// and should NOT be retried after token refresh.
|
||||
export const ACCOUNT_DEACTIVATED_SIGNALS = [
|
||||
"account_deactivated",
|
||||
"account has been deactivated",
|
||||
"account has been disabled",
|
||||
"your account has been suspended",
|
||||
"this account is deactivated",
|
||||
];
|
||||
|
||||
// T10 (sub2api PR #1169): Signals that indicate billing credits are exhausted.
|
||||
// Distinct from rate-limit 429 — the account won't recover until credits are added.
|
||||
export const CREDITS_EXHAUSTED_SIGNALS = [
|
||||
"insufficient_quota",
|
||||
"billing_hard_limit_reached",
|
||||
"exceeded your current quota",
|
||||
"credit_balance_too_low",
|
||||
"your credit balance is too low",
|
||||
"credits exhausted",
|
||||
"out of credits",
|
||||
"payment required",
|
||||
];
|
||||
|
||||
/**
|
||||
* T06: Returns true if response body indicates the account is permanently deactivated.
|
||||
*/
|
||||
export function isAccountDeactivated(errorText: string): boolean {
|
||||
const lower = String(errorText || "").toLowerCase();
|
||||
return ACCOUNT_DEACTIVATED_SIGNALS.some((sig) => lower.includes(sig));
|
||||
}
|
||||
|
||||
/**
|
||||
* T10: Returns true if response body indicates credits/quota are permanently exhausted.
|
||||
*/
|
||||
export function isCreditsExhausted(errorText: string): boolean {
|
||||
const lower = String(errorText || "").toLowerCase();
|
||||
return CREDITS_EXHAUSTED_SIGNALS.some((sig) => lower.includes(sig));
|
||||
}
|
||||
|
||||
// ─── Provider Profile Helper ────────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
@@ -201,6 +241,14 @@ export function classifyErrorText(errorText) {
|
||||
) {
|
||||
return RateLimitReason.QUOTA_EXHAUSTED;
|
||||
}
|
||||
// T10: credits_exhausted signals
|
||||
if (isCreditsExhausted(errorText)) {
|
||||
return RateLimitReason.QUOTA_EXHAUSTED;
|
||||
}
|
||||
// T06: account_deactivated signals
|
||||
if (isAccountDeactivated(errorText)) {
|
||||
return RateLimitReason.AUTH_ERROR;
|
||||
}
|
||||
if (
|
||||
lower.includes("rate limit") ||
|
||||
lower.includes("too many requests") ||
|
||||
@@ -294,13 +342,67 @@ export function checkFallbackError(
|
||||
errorText,
|
||||
backoffLevel = 0,
|
||||
model = null,
|
||||
provider = null
|
||||
provider = null,
|
||||
headers = null
|
||||
) {
|
||||
const errorStr = (errorText || "").toString();
|
||||
|
||||
function parseResetFromHeaders(headers, errorStr = "") {
|
||||
if (!headers) return null;
|
||||
|
||||
// Retry-After header
|
||||
const retryAfter =
|
||||
typeof headers.get === "function"
|
||||
? headers.get("retry-after")
|
||||
: headers["retry-after"] || headers["Retry-After"];
|
||||
|
||||
if (retryAfter) {
|
||||
const seconds = parseInt(retryAfter, 10);
|
||||
if (!isNaN(seconds) && String(seconds) === String(retryAfter).trim()) {
|
||||
return Date.now() + seconds * 1000;
|
||||
}
|
||||
const date = new Date(retryAfter);
|
||||
if (!isNaN(date.getTime())) return date.getTime();
|
||||
}
|
||||
|
||||
// X-RateLimit-Reset
|
||||
const rlReset =
|
||||
typeof headers.get === "function"
|
||||
? headers.get("x-ratelimit-reset")
|
||||
: headers["x-ratelimit-reset"] || headers["X-RateLimit-Reset"];
|
||||
|
||||
if (rlReset) {
|
||||
const ts = parseInt(rlReset, 10);
|
||||
if (!isNaN(ts)) {
|
||||
return ts > 10000000000 ? ts : ts * 1000;
|
||||
}
|
||||
}
|
||||
return null;
|
||||
}
|
||||
// Check error message FIRST - specific patterns take priority over status codes
|
||||
if (errorText) {
|
||||
const errorStr = typeof errorText === "string" ? errorText : JSON.stringify(errorText);
|
||||
const lowerError = errorStr.toLowerCase();
|
||||
|
||||
// T06 (sub2api #1037): Permanent account deactivation — do NOT retry, mark as permanent failure
|
||||
if (isAccountDeactivated(errorStr)) {
|
||||
return {
|
||||
shouldFallback: true,
|
||||
cooldownMs: 365 * 24 * 60 * 60 * 1000, // 1 year = effectively permanent
|
||||
reason: RateLimitReason.AUTH_ERROR,
|
||||
permanent: true,
|
||||
};
|
||||
}
|
||||
|
||||
// T10 (sub2api #1169): Credits/quota exhausted — long cooldown, distinct from rate limit
|
||||
if (isCreditsExhausted(errorStr)) {
|
||||
return {
|
||||
shouldFallback: true,
|
||||
cooldownMs: COOLDOWN_MS.paymentRequired ?? 3600 * 1000, // 1h cooldown
|
||||
reason: RateLimitReason.QUOTA_EXHAUSTED,
|
||||
creditsExhausted: true,
|
||||
};
|
||||
}
|
||||
|
||||
if (lowerError.includes("no credentials")) {
|
||||
return {
|
||||
shouldFallback: true,
|
||||
@@ -325,6 +427,18 @@ export function checkFallbackError(
|
||||
lowerError.includes("capacity") ||
|
||||
lowerError.includes("overloaded")
|
||||
) {
|
||||
const resetTime = parseResetFromHeaders(headers);
|
||||
if (resetTime) {
|
||||
const waitMs = resetTime - Date.now();
|
||||
if (waitMs > 60_000) {
|
||||
return {
|
||||
shouldFallback: true,
|
||||
cooldownMs: waitMs,
|
||||
newBackoffLevel: 0,
|
||||
reason: RateLimitReason.RATE_LIMIT_EXCEEDED,
|
||||
};
|
||||
}
|
||||
}
|
||||
const newLevel = Math.min(backoffLevel + 1, BACKOFF_CONFIG.maxLevel);
|
||||
const reason = classifyErrorText(errorStr);
|
||||
return {
|
||||
@@ -362,6 +476,19 @@ export function checkFallbackError(
|
||||
|
||||
// 429 - Rate limit with exponential backoff
|
||||
if (status === HTTP_STATUS.RATE_LIMITED) {
|
||||
const resetTime = parseResetFromHeaders(headers);
|
||||
if (resetTime) {
|
||||
const waitMs = resetTime - Date.now();
|
||||
if (waitMs > 60_000) {
|
||||
return {
|
||||
shouldFallback: true,
|
||||
cooldownMs: waitMs,
|
||||
newBackoffLevel: 0,
|
||||
reason: RateLimitReason.RATE_LIMIT_EXCEEDED,
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
const newLevel = Math.min(backoffLevel + 1, BACKOFF_CONFIG.maxLevel);
|
||||
return {
|
||||
shouldFallback: true,
|
||||
|
||||
@@ -841,7 +841,8 @@ export async function handleComboChat({
|
||||
errorText,
|
||||
0,
|
||||
null,
|
||||
provider
|
||||
provider,
|
||||
result.headers
|
||||
);
|
||||
|
||||
// Record failure in circuit breaker for transient errors
|
||||
@@ -865,6 +866,12 @@ export async function handleComboChat({
|
||||
if (!lastStatus) lastStatus = result.status;
|
||||
if (i > 0) fallbackCount++;
|
||||
log.warn("COMBO", `Model ${modelStr} failed, trying next`, { status: result.status });
|
||||
|
||||
if ([502, 503, 504].includes(result.status) && cooldownMs > 0 && cooldownMs <= 5000) {
|
||||
log.info("COMBO", `Waiting ${cooldownMs}ms before fallback to next model`);
|
||||
await new Promise((r) => setTimeout(r, cooldownMs));
|
||||
}
|
||||
|
||||
break; // Move to next model
|
||||
}
|
||||
}
|
||||
@@ -886,7 +893,20 @@ export async function handleComboChat({
|
||||
);
|
||||
}
|
||||
|
||||
const status = lastStatus || 406;
|
||||
if (!lastStatus) {
|
||||
return new Response(
|
||||
JSON.stringify({
|
||||
error: {
|
||||
message: "Service temporarily unavailable: all upstream accounts are inactive",
|
||||
type: "service_unavailable",
|
||||
code: "ALL_ACCOUNTS_INACTIVE",
|
||||
},
|
||||
}),
|
||||
{ status: 503, headers: { "Content-Type": "application/json" } }
|
||||
);
|
||||
}
|
||||
|
||||
const status = lastStatus;
|
||||
const msg = lastError || "All combo models unavailable";
|
||||
|
||||
if (earliestRetryAfter) {
|
||||
@@ -941,7 +961,7 @@ async function handleRoundRobinCombo({
|
||||
|
||||
const modelCount = orderedModels.length;
|
||||
if (modelCount === 0) {
|
||||
return unavailableResponse(406, "Round-robin combo has no models");
|
||||
return unavailableResponse(503, "Round-robin combo has no models");
|
||||
}
|
||||
|
||||
// Get and increment atomic counter
|
||||
@@ -1077,7 +1097,8 @@ async function handleRoundRobinCombo({
|
||||
errorText,
|
||||
0,
|
||||
null,
|
||||
provider
|
||||
provider,
|
||||
result.headers
|
||||
);
|
||||
|
||||
// Transient errors → mark in semaphore AND record circuit breaker failure
|
||||
@@ -1106,6 +1127,12 @@ async function handleRoundRobinCombo({
|
||||
if (!lastStatus) lastStatus = result.status;
|
||||
if (offset > 0) fallbackCount++;
|
||||
log.warn("COMBO-RR", `${modelStr} failed, trying next model`, { status: result.status });
|
||||
|
||||
if ([502, 503, 504].includes(result.status) && cooldownMs > 0 && cooldownMs <= 5000) {
|
||||
log.info("COMBO-RR", `Waiting ${cooldownMs}ms before fallback to next model`);
|
||||
await new Promise((r) => setTimeout(r, cooldownMs));
|
||||
}
|
||||
|
||||
break;
|
||||
}
|
||||
} finally {
|
||||
@@ -1136,7 +1163,20 @@ async function handleRoundRobinCombo({
|
||||
);
|
||||
}
|
||||
|
||||
const status = lastStatus || 406;
|
||||
if (!lastStatus) {
|
||||
return new Response(
|
||||
JSON.stringify({
|
||||
error: {
|
||||
message: "Service temporarily unavailable: all upstream accounts are inactive",
|
||||
type: "service_unavailable",
|
||||
code: "ALL_ACCOUNTS_INACTIVE",
|
||||
},
|
||||
}),
|
||||
{ status: 503, headers: { "Content-Type": "application/json" } }
|
||||
);
|
||||
}
|
||||
|
||||
const status = lastStatus;
|
||||
const msg = lastError || "All round-robin combo models unavailable";
|
||||
|
||||
if (earliestRetryAfter) {
|
||||
|
||||
53
open-sse/services/errorClassifier.ts
Normal file
53
open-sse/services/errorClassifier.ts
Normal file
@@ -0,0 +1,53 @@
|
||||
import { isAccountDeactivated, isCreditsExhausted } from "./accountFallback.ts";
|
||||
|
||||
export const PROVIDER_ERROR_TYPES = {
|
||||
RATE_LIMITED: "rate_limited", // 429 — transient, retry with backoff
|
||||
UNAUTHORIZED: "unauthorized", // 401 — token expired, refresh
|
||||
ACCOUNT_DEACTIVATED: "account_deactivated", // 401 + deactivation signal
|
||||
FORBIDDEN: "forbidden", // 403 — account banned/revoked, disable node
|
||||
SERVER_ERROR: "server_error", // 500/502/503 — retry limited
|
||||
QUOTA_EXHAUSTED: "quota_exhausted", // 402/429/400 + billing signals
|
||||
};
|
||||
|
||||
function responseBodyToString(responseBody: unknown): string {
|
||||
if (typeof responseBody === "string") return responseBody;
|
||||
if (responseBody !== null && typeof responseBody === "object") {
|
||||
try {
|
||||
return JSON.stringify(responseBody);
|
||||
} catch {
|
||||
return "";
|
||||
}
|
||||
}
|
||||
return "";
|
||||
}
|
||||
|
||||
export function classifyProviderError(statusCode: number, responseBody: unknown): string | null {
|
||||
const bodyStr = responseBodyToString(responseBody);
|
||||
const creditsExhausted = isCreditsExhausted(bodyStr);
|
||||
const accountDeactivated = isAccountDeactivated(bodyStr);
|
||||
|
||||
// T10: credits exhausted is terminal and can appear as 400/402/429 depending on provider.
|
||||
if (
|
||||
creditsExhausted &&
|
||||
(statusCode === 400 || statusCode === 402 || statusCode === 429 || statusCode === 403)
|
||||
) {
|
||||
return PROVIDER_ERROR_TYPES.QUOTA_EXHAUSTED;
|
||||
}
|
||||
|
||||
if (statusCode === 429) {
|
||||
return PROVIDER_ERROR_TYPES.RATE_LIMITED;
|
||||
}
|
||||
|
||||
// T06: only deactivation-like 401s should be treated as permanent account expiry.
|
||||
if (statusCode === 401) {
|
||||
return accountDeactivated
|
||||
? PROVIDER_ERROR_TYPES.ACCOUNT_DEACTIVATED
|
||||
: PROVIDER_ERROR_TYPES.UNAUTHORIZED;
|
||||
}
|
||||
|
||||
if (statusCode === 402) return PROVIDER_ERROR_TYPES.QUOTA_EXHAUSTED;
|
||||
if (statusCode === 403) return PROVIDER_ERROR_TYPES.FORBIDDEN;
|
||||
if (statusCode >= 500) return PROVIDER_ERROR_TYPES.SERVER_ERROR;
|
||||
|
||||
return null;
|
||||
}
|
||||
@@ -4,6 +4,8 @@
|
||||
* IP-based access control with blacklist, whitelist, priority modes, and temporary bans.
|
||||
*/
|
||||
|
||||
import { isIP } from "node:net";
|
||||
|
||||
// In-memory IP lists
|
||||
let _config = {
|
||||
enabled: false,
|
||||
@@ -161,10 +163,10 @@ export function createIPFilterMiddleware() {
|
||||
*/
|
||||
export function checkRequestIP(request) {
|
||||
const ip =
|
||||
request.headers?.get?.("x-forwarded-for")?.split(",")[0].trim() ||
|
||||
request.headers?.get?.("x-real-ip") ||
|
||||
request.headers?.get?.("cf-connecting-ip") ||
|
||||
request.ip ||
|
||||
pickFirstValidIp(request.headers?.get?.("cf-connecting-ip")) ||
|
||||
pickFirstValidIp(request.headers?.get?.("x-forwarded-for")) ||
|
||||
pickFirstValidIp(request.headers?.get?.("x-real-ip")) ||
|
||||
normalizeIP(request.ip || "") ||
|
||||
"unknown";
|
||||
return checkIP(ip);
|
||||
}
|
||||
@@ -177,6 +179,18 @@ function normalizeIP(ip) {
|
||||
return ip.replace(/^::ffff:/, "").trim();
|
||||
}
|
||||
|
||||
function pickFirstValidIp(rawValue) {
|
||||
if (typeof rawValue !== "string" || rawValue.trim().length === 0) return null;
|
||||
const candidates = rawValue.split(",");
|
||||
for (const candidate of candidates) {
|
||||
const normalized = normalizeIP(candidate);
|
||||
if (normalized && isIP(normalized) !== 0) {
|
||||
return normalized;
|
||||
}
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
function matchesAny(ip, ipSet) {
|
||||
// Direct match
|
||||
if (ipSet.has(ip)) return true;
|
||||
@@ -225,12 +239,13 @@ function matchesWildcard(ip, pattern) {
|
||||
}
|
||||
|
||||
function extractClientIP(req) {
|
||||
const headers = req.headers || {};
|
||||
return (
|
||||
req.headers?.["x-forwarded-for"]?.split(",")[0].trim() ||
|
||||
req.headers?.["x-real-ip"] ||
|
||||
req.headers?.["cf-connecting-ip"] ||
|
||||
req.socket?.remoteAddress ||
|
||||
req.ip ||
|
||||
pickFirstValidIp(headers["cf-connecting-ip"]) ||
|
||||
pickFirstValidIp(headers["x-forwarded-for"]) ||
|
||||
pickFirstValidIp(headers["x-real-ip"]) ||
|
||||
pickFirstValidIp(req.socket?.remoteAddress) ||
|
||||
pickFirstValidIp(req.ip) ||
|
||||
"unknown"
|
||||
);
|
||||
}
|
||||
|
||||
@@ -18,6 +18,8 @@ const BUILT_IN_ALIASES: Record<string, string> = {
|
||||
"gemini-1.5-flash": "gemini-2.5-flash",
|
||||
"gemini-1.0-pro": "gemini-2.5-pro",
|
||||
"gemini-2.0-flash": "gemini-2.5-flash",
|
||||
"gemini-3-pro-high": "gemini-3.1-pro-high",
|
||||
"gemini-3-pro-low": "gemini-3.1-pro-low",
|
||||
|
||||
// Claude legacy → current
|
||||
"claude-3-opus-20240229": "claude-opus-4-20250514",
|
||||
|
||||
@@ -101,6 +101,7 @@ const MODEL_UNAVAILABLE_FRAGMENTS = [
|
||||
"does not support",
|
||||
"not enabled for",
|
||||
"access to model",
|
||||
"improperly formed request", // Kiro 400 (model unavailable)
|
||||
];
|
||||
|
||||
/**
|
||||
|
||||
@@ -12,6 +12,7 @@ import Bottleneck from "bottleneck";
|
||||
import { parseRetryAfterFromBody, lockModel } from "./accountFallback.ts";
|
||||
import { getProviderCategory } from "../config/providerRegistry.ts";
|
||||
import { DEFAULT_API_LIMITS } from "../config/constants.ts";
|
||||
import { getCodexRateLimitKey } from "../executors/codex.ts";
|
||||
|
||||
interface LearnedLimitEntry {
|
||||
provider: string;
|
||||
@@ -195,8 +196,15 @@ export function isRateLimitEnabled(connectionId) {
|
||||
/**
|
||||
* Get or create a limiter for a given provider+connection combination
|
||||
*/
|
||||
function getLimiterKey(provider, connectionId, model = null) {
|
||||
if (provider === "codex" && model) {
|
||||
return `${provider}:${getCodexRateLimitKey(connectionId, model)}`;
|
||||
}
|
||||
return `${provider}:${connectionId}`;
|
||||
}
|
||||
|
||||
function getLimiter(provider, connectionId, model = null) {
|
||||
const key = model ? `${provider}:${connectionId}:${model}` : `${provider}:${connectionId}`;
|
||||
const key = getLimiterKey(provider, connectionId, model);
|
||||
|
||||
if (!limiters.has(key)) {
|
||||
const limiter = new Bottleneck({
|
||||
@@ -235,7 +243,7 @@ export async function withRateLimit(provider, connectionId, model, fn) {
|
||||
return fn();
|
||||
}
|
||||
|
||||
const limiter = getLimiter(provider, connectionId, null);
|
||||
const limiter = getLimiter(provider, connectionId, model);
|
||||
return limiter.schedule(fn);
|
||||
}
|
||||
|
||||
@@ -320,7 +328,7 @@ export function updateFromHeaders(provider, connectionId, headers, status, model
|
||||
if (!enabledConnections.has(connectionId)) return;
|
||||
if (!headers) return;
|
||||
|
||||
const limiter = getLimiter(provider, connectionId, null);
|
||||
const limiter = getLimiter(provider, connectionId, model);
|
||||
const headerMap =
|
||||
provider === "claude" || provider === "anthropic" ? ANTHROPIC_HEADERS : STANDARD_HEADERS;
|
||||
|
||||
@@ -340,7 +348,7 @@ export function updateFromHeaders(provider, connectionId, headers, status, model
|
||||
if (status === 429) {
|
||||
const retryAfterMs = parseResetTime(retryAfterStr) || 60000; // Default 60s
|
||||
const counts = limiter.counts();
|
||||
const limiterKey = `${provider}:${connectionId}`;
|
||||
const limiterKey = getLimiterKey(provider, connectionId, model);
|
||||
console.log(
|
||||
`🚫 [RATE-LIMIT] ${provider}:${connectionId.slice(0, 8)} — 429 received, pausing for ${Math.ceil(retryAfterMs / 1000)}s, dropping ${counts.QUEUED} queued request(s)`
|
||||
);
|
||||
@@ -397,7 +405,12 @@ export function updateFromHeaders(provider, connectionId, headers, status, model
|
||||
limiter.updateSettings(updates);
|
||||
|
||||
// Persist learned limits (debounced)
|
||||
recordLearnedLimit(provider, connectionId, { limit, remaining, minTime: updates.minTime });
|
||||
recordLearnedLimit(
|
||||
provider,
|
||||
connectionId,
|
||||
{ limit, remaining, minTime: updates.minTime },
|
||||
model
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -459,9 +472,10 @@ export function getLearnedLimits() {
|
||||
function recordLearnedLimit(
|
||||
provider: string,
|
||||
connectionId: string,
|
||||
limits: Partial<Omit<LearnedLimitEntry, "provider" | "connectionId" | "lastUpdated">>
|
||||
limits: Partial<Omit<LearnedLimitEntry, "provider" | "connectionId" | "lastUpdated">>,
|
||||
model: string | null = null
|
||||
) {
|
||||
const key = `${provider}:${connectionId}`;
|
||||
const key = getLimiterKey(provider, connectionId, model);
|
||||
learnedLimits[key] = {
|
||||
...limits,
|
||||
provider,
|
||||
|
||||
@@ -41,7 +41,13 @@ const SESSION_TTL_MS = 30 * 60 * 1000;
|
||||
const _cleanupTimer = setInterval(() => {
|
||||
const now = Date.now();
|
||||
for (const [key, entry] of sessions) {
|
||||
if (now - entry.lastActive > SESSION_TTL_MS) sessions.delete(key);
|
||||
if (now - entry.lastActive > SESSION_TTL_MS) {
|
||||
sessions.delete(key);
|
||||
for (const [apiKeyId, sessionSet] of activeSessionsByKey) {
|
||||
sessionSet.delete(key);
|
||||
if (sessionSet.size === 0) activeSessionsByKey.delete(apiKeyId);
|
||||
}
|
||||
}
|
||||
}
|
||||
}, 60_000);
|
||||
_cleanupTimer.unref();
|
||||
@@ -173,6 +179,103 @@ export function getActiveSessions(): Array<SessionEntry & { sessionId: string; a
|
||||
*/
|
||||
export function clearSessions(): void {
|
||||
sessions.clear();
|
||||
activeSessionsByKey.clear();
|
||||
}
|
||||
|
||||
// ─── T08: Per-API-Key Session Limit ─────────────────────────────────────────
|
||||
// Tracks concurrent sticky sessions per API key and enforces max_sessions limits.
|
||||
// Ref: sub2api PR #634 (fix: stabilize session hash + add user-level session limit)
|
||||
|
||||
// Map: apiKeyId → Set<sessionId>
|
||||
const activeSessionsByKey = new Map<string, Set<string>>();
|
||||
|
||||
/**
|
||||
* T08: Get the number of currently active sessions for an API key.
|
||||
* @param apiKeyId - The API key's UUID from the database
|
||||
*/
|
||||
export function getActiveSessionCountForKey(apiKeyId: string): number {
|
||||
return activeSessionsByKey.get(apiKeyId)?.size ?? 0;
|
||||
}
|
||||
|
||||
/**
|
||||
* T08: Register a session as belonging to an API key.
|
||||
* Call this after session creation is allowed (i.e., limit check passed).
|
||||
*/
|
||||
export function registerKeySession(apiKeyId: string, sessionId: string): void {
|
||||
if (!activeSessionsByKey.has(apiKeyId)) {
|
||||
activeSessionsByKey.set(apiKeyId, new Set());
|
||||
}
|
||||
activeSessionsByKey.get(apiKeyId)!.add(sessionId);
|
||||
}
|
||||
|
||||
/**
|
||||
* Check whether a given session is already registered for an API key.
|
||||
*/
|
||||
export function isSessionRegisteredForKey(apiKeyId: string, sessionId: string): boolean {
|
||||
return activeSessionsByKey.get(apiKeyId)?.has(sessionId) === true;
|
||||
}
|
||||
|
||||
/**
|
||||
* T08: Unregister a session from an API key's active set.
|
||||
* Call this when the request closes or the session TTL expires.
|
||||
*/
|
||||
export function unregisterKeySession(apiKeyId: string, sessionId: string): void {
|
||||
activeSessionsByKey.get(apiKeyId)?.delete(sessionId);
|
||||
// Clean up empty sets to avoid memory leaks
|
||||
if (activeSessionsByKey.get(apiKeyId)?.size === 0) {
|
||||
activeSessionsByKey.delete(apiKeyId);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* T08: Check whether adding a new session would exceed the key's max_sessions limit.
|
||||
* Returns null if allowed, or an error object to return as a 429 response.
|
||||
*
|
||||
* @param apiKeyId - The API key's UUID
|
||||
* @param maxSessions - The limit from the DB (0 = unlimited)
|
||||
*/
|
||||
export function checkSessionLimit(
|
||||
apiKeyId: string,
|
||||
maxSessions: number
|
||||
): { code: "SESSION_LIMIT_EXCEEDED"; message: string; limit: number; current: number } | null {
|
||||
if (!maxSessions || maxSessions <= 0) return null; // unlimited
|
||||
const current = getActiveSessionCountForKey(apiKeyId);
|
||||
if (current < maxSessions) return null;
|
||||
return {
|
||||
code: "SESSION_LIMIT_EXCEEDED",
|
||||
message:
|
||||
`You have reached the maximum number of active sessions (${maxSessions}). ` +
|
||||
`Please close unused sessions or wait for them to expire.`,
|
||||
limit: maxSessions,
|
||||
current,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* T04: Extract an external session ID from request headers.
|
||||
* Accepts both hyphenated and underscore forms for Nginx compatibility.
|
||||
* Nginx drops headers with underscores by default — use `underscores_in_headers on`
|
||||
* in nginx.conf, or use X-Session-Id (hyphenated) which passes cleanly.
|
||||
*
|
||||
* Ref: sub2api README + PR #634
|
||||
*
|
||||
* @param headers - Request headers (Headers object or plain object with .get())
|
||||
* @returns External session ID with "ext:" prefix, or null
|
||||
*/
|
||||
export function extractExternalSessionId(
|
||||
headers: Headers | { get?: (n: string) => string | null } | null | undefined
|
||||
): string | null {
|
||||
if (!headers || typeof (headers as Headers).get !== "function") return null;
|
||||
const h = headers as Headers;
|
||||
const raw =
|
||||
h.get("x-session-id") ?? // Preferred: hyphenated (passes through Nginx)
|
||||
h.get("x_session_id") ?? // Underscore variant (direct HTTP / custom clients)
|
||||
h.get("x-omniroute-session") ?? // OmniRoute-specific form
|
||||
h.get("session-id") ?? // Bare session-id
|
||||
null;
|
||||
if (!raw || !raw.trim()) return null;
|
||||
// Prefix "ext:" to ensure no collision with internal SHA-256 hash IDs
|
||||
return `ext:${raw.trim().slice(0, 64)}`; // max 64 chars to avoid abuse
|
||||
}
|
||||
|
||||
// ─── Internal Helpers ───────────────────────────────────────────────────────
|
||||
|
||||
@@ -13,12 +13,16 @@ export const ThinkingMode = {
|
||||
ADAPTIVE: "adaptive", // Scale based on request complexity
|
||||
};
|
||||
|
||||
import { capThinkingBudget, getDefaultThinkingBudget } from "@/shared/constants/modelSpecs";
|
||||
|
||||
// Effort → budget token mapping
|
||||
export const EFFORT_BUDGETS = {
|
||||
none: 0,
|
||||
low: 1024,
|
||||
medium: 10240,
|
||||
high: 131072,
|
||||
high: 131072, // Handled globally by capThinkingBudget later
|
||||
max: 131072, // T11: Claude "max" / "xhigh" — full budget
|
||||
xhigh: 131072, // T11: explicit alias used internally
|
||||
};
|
||||
|
||||
// thinkingLevel string → budget token mapping
|
||||
@@ -28,6 +32,8 @@ export const THINKING_LEVEL_MAP = {
|
||||
low: 1024,
|
||||
medium: 10240,
|
||||
high: 131072,
|
||||
max: 131072, // T11: max = full Claude budget (sub2api: xhigh)
|
||||
xhigh: 131072, // T11: explicit xhigh alias
|
||||
};
|
||||
|
||||
// Default config (passthrough = backward compatible)
|
||||
@@ -68,8 +74,8 @@ export function normalizeThinkingLevel(body) {
|
||||
|
||||
// Handle top-level thinkingLevel or thinking_level string fields
|
||||
const levelStr = result.thinkingLevel || result.thinking_level;
|
||||
if (typeof levelStr === "string" && THINKING_LEVEL_MAP[levelStr] !== undefined) {
|
||||
const budget = THINKING_LEVEL_MAP[levelStr];
|
||||
if (typeof levelStr === "string" && THINKING_LEVEL_MAP[levelStr.toLowerCase()] !== undefined) {
|
||||
const budget = THINKING_LEVEL_MAP[levelStr.toLowerCase()];
|
||||
// Convert to Claude thinking format as canonical representation
|
||||
result.thinking = {
|
||||
type: budget > 0 ? "enabled" : "disabled",
|
||||
@@ -83,15 +89,21 @@ export function normalizeThinkingLevel(body) {
|
||||
const geminiLevel =
|
||||
result.generationConfig?.thinkingConfig?.thinkingLevel ||
|
||||
result.generationConfig?.thinking_config?.thinkingLevel;
|
||||
if (typeof geminiLevel === "string" && THINKING_LEVEL_MAP[geminiLevel] !== undefined) {
|
||||
const budget = THINKING_LEVEL_MAP[geminiLevel];
|
||||
if (
|
||||
typeof geminiLevel === "string" &&
|
||||
THINKING_LEVEL_MAP[geminiLevel.toLowerCase()] !== undefined
|
||||
) {
|
||||
const budget = THINKING_LEVEL_MAP[geminiLevel.toLowerCase()];
|
||||
result.generationConfig = {
|
||||
...result.generationConfig,
|
||||
thinking_config: { thinking_budget: budget },
|
||||
thinkingConfig: { ...result.generationConfig.thinkingConfig, thinkingBudget: budget },
|
||||
};
|
||||
// Clean up camelCase variant if it was the source
|
||||
// Clean up string variants
|
||||
if (result.generationConfig.thinkingConfig) {
|
||||
delete result.generationConfig.thinkingConfig;
|
||||
delete result.generationConfig.thinkingConfig.thinkingLevel;
|
||||
}
|
||||
if (result.generationConfig.thinking_config) {
|
||||
delete result.generationConfig.thinking_config;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -118,7 +130,7 @@ export function ensureThinkingConfig(body) {
|
||||
const result = { ...body };
|
||||
result.thinking = {
|
||||
type: "enabled",
|
||||
budget_tokens: EFFORT_BUDGETS.medium, // 10240 default
|
||||
budget_tokens: getDefaultThinkingBudget(model) || EFFORT_BUDGETS.medium,
|
||||
};
|
||||
return result;
|
||||
}
|
||||
@@ -198,7 +210,7 @@ function setCustomBudget(body, budget) {
|
||||
};
|
||||
}
|
||||
|
||||
// OpenAI reasoning_effort mapping
|
||||
// OpenAI reasoning_effort mapping (T11: add 'max' tier for full budget)
|
||||
if (result.reasoning_effort !== undefined || result.reasoning !== undefined) {
|
||||
if (budget <= 0) {
|
||||
delete result.reasoning_effort;
|
||||
@@ -207,8 +219,10 @@ function setCustomBudget(body, budget) {
|
||||
result.reasoning_effort = "low";
|
||||
} else if (budget <= 10240) {
|
||||
result.reasoning_effort = "medium";
|
||||
} else {
|
||||
} else if (budget < 131072) {
|
||||
result.reasoning_effort = "high";
|
||||
} else {
|
||||
result.reasoning_effort = "max"; // T11: full budget → "max"
|
||||
}
|
||||
}
|
||||
|
||||
@@ -251,8 +265,11 @@ function applyAdaptiveBudget(body, cfg) {
|
||||
if (toolCount > 3) multiplier += 0.5;
|
||||
if (lastMsgLength > 2000) multiplier += 0.3;
|
||||
|
||||
const baseBudget = EFFORT_BUDGETS[cfg.effortLevel] || EFFORT_BUDGETS.medium;
|
||||
const budget = Math.min(Math.ceil(baseBudget * multiplier), 131072);
|
||||
const baseBudget =
|
||||
EFFORT_BUDGETS[cfg.effortLevel] ||
|
||||
getDefaultThinkingBudget(body.model || "") ||
|
||||
EFFORT_BUDGETS.medium;
|
||||
const budget = capThinkingBudget(body.model || "", Math.ceil(baseBudget * multiplier));
|
||||
|
||||
return setCustomBudget(body, budget);
|
||||
}
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
import * as fs from 'fs';
|
||||
import * as path from 'path';
|
||||
import * as fs from "fs";
|
||||
import * as path from "path";
|
||||
/**
|
||||
* Responses API Transformer
|
||||
* Converts OpenAI Chat Completions SSE to Codex Responses API SSE format
|
||||
@@ -402,6 +402,16 @@ export function createResponsesApiTransformStream(logger = null) {
|
||||
const newCallId = tc.id;
|
||||
const funcName = tc.function?.name;
|
||||
|
||||
// T37: Prevent merging if a new tool_call uses the same index
|
||||
if (state.funcCallIds[tcIdx] && newCallId && state.funcCallIds[tcIdx] !== newCallId) {
|
||||
closeToolCall(controller, tcIdx);
|
||||
delete state.funcCallIds[tcIdx];
|
||||
delete state.funcNames[tcIdx];
|
||||
delete state.funcArgsBuf[tcIdx];
|
||||
delete state.funcArgsDone[tcIdx];
|
||||
delete state.funcItemDone[tcIdx];
|
||||
}
|
||||
|
||||
if (funcName) state.funcNames[tcIdx] = funcName;
|
||||
|
||||
if (!state.funcCallIds[tcIdx] && newCallId) {
|
||||
|
||||
@@ -172,6 +172,9 @@ function convertEnumValuesToStrings(obj) {
|
||||
|
||||
if (obj.enum && Array.isArray(obj.enum)) {
|
||||
obj.enum = obj.enum.map((v) => String(v));
|
||||
if (!obj.type) {
|
||||
obj.type = "string";
|
||||
}
|
||||
}
|
||||
|
||||
for (const value of Object.values(obj)) {
|
||||
|
||||
22
open-sse/translator/image/sizeMapper.ts
Normal file
22
open-sse/translator/image/sizeMapper.ts
Normal file
@@ -0,0 +1,22 @@
|
||||
const OPENAI_SIZE_TO_ASPECT_RATIO: Record<string, string> = {
|
||||
"256x256": "1:1",
|
||||
"512x512": "1:1",
|
||||
"1024x1024": "1:1",
|
||||
"1792x1024": "16:9",
|
||||
"1024x1792": "9:16",
|
||||
"1536x1024": "3:2",
|
||||
"1024x1536": "2:3",
|
||||
};
|
||||
|
||||
// Supports direct aspect ratios (e.g. "16:9")
|
||||
const ASPECT_RATIO_PASSTHROUGH = /^\d+:\d+$/;
|
||||
|
||||
export function mapImageSize(sizeParam?: string | null): string {
|
||||
if (!sizeParam) return "1:1"; // default
|
||||
|
||||
// Native aspect ratio (e.g. "16:9") — pass-through
|
||||
if (ASPECT_RATIO_PASSTHROUGH.test(sizeParam)) return sizeParam;
|
||||
|
||||
// Map OpenAI sizes to aspect ratios
|
||||
return OPENAI_SIZE_TO_ASPECT_RATIO[sizeParam] ?? "1:1";
|
||||
}
|
||||
@@ -1,6 +1,10 @@
|
||||
import { register } from "../registry.ts";
|
||||
import { FORMATS } from "../formats.ts";
|
||||
import { DEFAULT_SAFETY_SETTINGS, tryParseJSON } from "../helpers/geminiHelper.ts";
|
||||
import {
|
||||
DEFAULT_SAFETY_SETTINGS,
|
||||
tryParseJSON,
|
||||
cleanJSONSchemaForAntigravity,
|
||||
} from "../helpers/geminiHelper.ts";
|
||||
import { DEFAULT_THINKING_GEMINI_SIGNATURE } from "../../config/defaultThinkingSignature.ts";
|
||||
|
||||
/**
|
||||
@@ -154,7 +158,9 @@ export function claudeToGeminiRequest(model, body, stream) {
|
||||
functionDeclarations.push({
|
||||
name: tool.name,
|
||||
description: tool.description || "",
|
||||
parameters: tool.input_schema || { type: "object", properties: {} },
|
||||
parameters: cleanJSONSchemaForAntigravity(
|
||||
tool.input_schema || { type: "object", properties: {} }
|
||||
),
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
@@ -27,6 +27,60 @@ type ClaudeTool = {
|
||||
defer_loading?: boolean;
|
||||
};
|
||||
|
||||
/**
|
||||
* T02: Recursively strips empty text blocks from content arrays.
|
||||
* Anthropic returns 400 "text content blocks must be non-empty" if any
|
||||
* text block has text: "". Must also recurse into nested tool_result.content.
|
||||
* Ref: sub2api PR #1212
|
||||
*/
|
||||
export function stripEmptyTextBlocks(content: unknown[] | undefined): unknown[] {
|
||||
if (!Array.isArray(content)) return content ?? [];
|
||||
return content
|
||||
.filter((block: unknown) => {
|
||||
if (
|
||||
block &&
|
||||
typeof block === "object" &&
|
||||
(block as Record<string, unknown>).type === "text"
|
||||
) {
|
||||
const text = (block as Record<string, unknown>).text;
|
||||
if (text === "" || text == null) return false;
|
||||
}
|
||||
return true;
|
||||
})
|
||||
.map((block: unknown) => {
|
||||
if (
|
||||
block &&
|
||||
typeof block === "object" &&
|
||||
(block as Record<string, unknown>).type === "tool_result" &&
|
||||
Array.isArray((block as Record<string, unknown>).content)
|
||||
) {
|
||||
// Recurse into nested tool_result.content
|
||||
return {
|
||||
...(block as Record<string, unknown>),
|
||||
content: stripEmptyTextBlocks((block as Record<string, unknown>).content as unknown[]),
|
||||
};
|
||||
}
|
||||
return block;
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* T15: Normalize content to string form.
|
||||
* Handles both string and array-of-blocks forms (Cursor, Codex 2.x, etc.).
|
||||
* Ref: sub2api PR #1197
|
||||
*/
|
||||
export function normalizeContentToString(content: string | unknown[] | null | undefined): string {
|
||||
if (!content) return "";
|
||||
if (typeof content === "string") return content;
|
||||
if (Array.isArray(content)) {
|
||||
return (content as Array<Record<string, unknown>>)
|
||||
.filter((b) => b.type === "text")
|
||||
.map((b) => String(b.text ?? ""))
|
||||
.join("\n");
|
||||
}
|
||||
return "";
|
||||
}
|
||||
|
||||
// Convert OpenAI request to Claude format
|
||||
export function openaiToClaudeRequest(model, body, stream) {
|
||||
// Check if tool prefix should be disabled (configured per-provider or global)
|
||||
@@ -61,11 +115,11 @@ export function openaiToClaudeRequest(model, body, stream) {
|
||||
const systemParts = [];
|
||||
|
||||
if (body.messages && Array.isArray(body.messages)) {
|
||||
// Extract system messages
|
||||
// Extract system messages (T15: handle both string and array content)
|
||||
for (const msg of body.messages) {
|
||||
if (msg.role === "system") {
|
||||
systemParts.push(
|
||||
typeof msg.content === "string" ? msg.content : extractTextContent(msg.content)
|
||||
typeof msg.content === "string" ? msg.content : normalizeContentToString(msg.content)
|
||||
);
|
||||
}
|
||||
}
|
||||
@@ -270,10 +324,14 @@ function getContentBlocksFromMessage(msg, toolNameMap = new Map(), disableToolPr
|
||||
const blocks = [];
|
||||
|
||||
if (msg.role === "tool") {
|
||||
// T02: Strip empty text blocks from nested tool_result content to avoid Anthropic 400
|
||||
const toolContent = Array.isArray(msg.content)
|
||||
? stripEmptyTextBlocks(msg.content)
|
||||
: msg.content;
|
||||
blocks.push({
|
||||
type: "tool_result",
|
||||
tool_use_id: msg.tool_call_id,
|
||||
content: msg.content,
|
||||
content: toolContent,
|
||||
});
|
||||
} else if (msg.role === "user") {
|
||||
if (typeof msg.content === "string") {
|
||||
@@ -287,10 +345,14 @@ function getContentBlocksFromMessage(msg, toolNameMap = new Map(), disableToolPr
|
||||
} else if (part.type === "tool_result") {
|
||||
// Skip tool_result with no tool_use_id (would be useless and may cause errors)
|
||||
if (!part.tool_use_id) continue;
|
||||
// T02: strip empty text blocks from nested content before passing to Anthropic
|
||||
const resultContent = Array.isArray(part.content)
|
||||
? stripEmptyTextBlocks(part.content)
|
||||
: part.content;
|
||||
blocks.push({
|
||||
type: "tool_result",
|
||||
tool_use_id: part.tool_use_id,
|
||||
content: part.content,
|
||||
content: resultContent,
|
||||
...(part.is_error && { is_error: part.is_error }),
|
||||
});
|
||||
} else if (part.type === "image_url") {
|
||||
|
||||
@@ -3,6 +3,11 @@ import { FORMATS } from "../formats.ts";
|
||||
import { DEFAULT_THINKING_GEMINI_SIGNATURE } from "../../config/defaultThinkingSignature.ts";
|
||||
import { ANTIGRAVITY_DEFAULT_SYSTEM } from "../../config/constants.ts";
|
||||
import { openaiToClaudeRequestForAntigravity } from "./openai-to-claude.ts";
|
||||
import {
|
||||
capMaxOutputTokens,
|
||||
capThinkingBudget,
|
||||
getDefaultThinkingBudget,
|
||||
} from "../../../src/shared/constants/modelSpecs.ts";
|
||||
|
||||
function generateUUID() {
|
||||
return crypto.randomUUID();
|
||||
@@ -88,7 +93,9 @@ function openaiToGeminiBase(model, body, stream) {
|
||||
result.generationConfig.topK = body.top_k;
|
||||
}
|
||||
if (body.max_tokens !== undefined) {
|
||||
result.generationConfig.maxOutputTokens = body.max_tokens;
|
||||
result.generationConfig.maxOutputTokens = capMaxOutputTokens(model, body.max_tokens);
|
||||
} else {
|
||||
result.generationConfig.maxOutputTokens = capMaxOutputTokens(model);
|
||||
}
|
||||
|
||||
// Build tool_call_id -> name map
|
||||
@@ -283,8 +290,12 @@ export function openaiToGeminiCLIRequest(model, body, stream) {
|
||||
|
||||
// Add thinking config for CLI
|
||||
if (body.reasoning_effort) {
|
||||
const budgetMap = { low: 1024, medium: 8192, high: 32768 };
|
||||
const budget = budgetMap[body.reasoning_effort] || 8192;
|
||||
const budgetMap = {
|
||||
low: 1024,
|
||||
medium: getDefaultThinkingBudget(model) || 8192,
|
||||
high: capThinkingBudget(model, 32768),
|
||||
};
|
||||
const budget = budgetMap[body.reasoning_effort] || getDefaultThinkingBudget(model) || 8192;
|
||||
gemini.generationConfig.thinkingConfig = {
|
||||
thinkingBudget: budget,
|
||||
include_thoughts: true,
|
||||
|
||||
@@ -257,6 +257,17 @@ function emitToolCall(state, emit, tc) {
|
||||
const newCallId = tc.id;
|
||||
const funcName = tc.function?.name;
|
||||
|
||||
// T37: If we already have a tool call at this index but the ID changed,
|
||||
// we must close the current one and start a new one to prevent merging.
|
||||
if (state.funcCallIds[tcIdx] && newCallId && state.funcCallIds[tcIdx] !== newCallId) {
|
||||
closeToolCall(state, emit, tcIdx);
|
||||
delete state.funcCallIds[tcIdx];
|
||||
delete state.funcNames[tcIdx];
|
||||
delete state.funcArgsBuf[tcIdx];
|
||||
delete state.funcArgsDone[tcIdx];
|
||||
delete state.funcItemDone[tcIdx];
|
||||
}
|
||||
|
||||
if (funcName) state.funcNames[tcIdx] = funcName;
|
||||
|
||||
if (!state.funcCallIds[tcIdx] && newCallId) {
|
||||
|
||||
@@ -12,6 +12,7 @@ type PendingToolCall = {
|
||||
export function transformToOllama(response, model) {
|
||||
let buffer = "";
|
||||
let pendingToolCalls: Record<number, PendingToolCall> = {};
|
||||
const completedToolCalls: PendingToolCall[] = [];
|
||||
|
||||
const transform = new TransformStream({
|
||||
transform(chunk, controller) {
|
||||
@@ -41,6 +42,13 @@ export function transformToOllama(response, model) {
|
||||
if (toolCalls) {
|
||||
for (const tc of toolCalls) {
|
||||
const idx = tc.index;
|
||||
|
||||
// T37: Prevent merging tool_calls on same index if ID changes
|
||||
if (pendingToolCalls[idx] && tc.id && pendingToolCalls[idx].id !== tc.id) {
|
||||
completedToolCalls.push(pendingToolCalls[idx]);
|
||||
delete pendingToolCalls[idx];
|
||||
}
|
||||
|
||||
if (!pendingToolCalls[idx]) {
|
||||
pendingToolCalls[idx] = { id: tc.id, function: { name: "", arguments: "" } };
|
||||
}
|
||||
@@ -59,7 +67,7 @@ export function transformToOllama(response, model) {
|
||||
|
||||
const finishReason = parsed.choices?.[0]?.finish_reason;
|
||||
if (finishReason === "tool_calls" || finishReason === "stop") {
|
||||
const toolCallsArr = Object.values(pendingToolCalls);
|
||||
const toolCallsArr = [...completedToolCalls, ...Object.values(pendingToolCalls)];
|
||||
if (toolCallsArr.length > 0) {
|
||||
const formattedCalls = toolCallsArr.map((tc) => ({
|
||||
function: {
|
||||
|
||||
@@ -222,16 +222,17 @@ export function createSSEStream(options: StreamOptions = {}) {
|
||||
const extracted = extractUsage(parsed);
|
||||
if (extracted) {
|
||||
// Non-destructive merge: never overwrite a positive value with 0
|
||||
// message_start carries input_tokens, message_delta carries output_tokens
|
||||
if (!usage) usage = {};
|
||||
if (extracted.prompt_tokens > 0) usage.prompt_tokens = extracted.prompt_tokens;
|
||||
if (extracted.completion_tokens > 0)
|
||||
usage.completion_tokens = extracted.completion_tokens;
|
||||
if (extracted.total_tokens > 0) usage.total_tokens = extracted.total_tokens;
|
||||
if (extracted.cache_read_input_tokens)
|
||||
usage.cache_read_input_tokens = extracted.cache_read_input_tokens;
|
||||
if (extracted.cache_creation_input_tokens)
|
||||
usage.cache_creation_input_tokens = extracted.cache_creation_input_tokens;
|
||||
// message_start carries input_tokens, message_delta carries output_tokens;
|
||||
if (!usage) usage = {} as any;
|
||||
const u = usage as Record<string, number>;
|
||||
const eu = extracted as Record<string, number>;
|
||||
if (eu.prompt_tokens > 0) u.prompt_tokens = eu.prompt_tokens;
|
||||
if (eu.completion_tokens > 0) u.completion_tokens = eu.completion_tokens;
|
||||
if (eu.total_tokens > 0) u.total_tokens = eu.total_tokens;
|
||||
if (eu.cache_read_input_tokens)
|
||||
u.cache_read_input_tokens = eu.cache_read_input_tokens;
|
||||
if (eu.cache_creation_input_tokens)
|
||||
u.cache_creation_input_tokens = eu.cache_creation_input_tokens;
|
||||
}
|
||||
// Track content length and accumulate from Claude format
|
||||
if (parsed.delta?.text) {
|
||||
@@ -263,6 +264,11 @@ export function createSSEStream(options: StreamOptions = {}) {
|
||||
}
|
||||
}
|
||||
|
||||
// T18: Track if we saw tool calls
|
||||
if (delta?.tool_calls && delta.tool_calls.length > 0) {
|
||||
(state as any).passthroughHasToolCalls = true;
|
||||
}
|
||||
|
||||
const content = delta?.content || delta?.reasoning_content;
|
||||
if (content && typeof content === "string") {
|
||||
totalContentLength += content.length;
|
||||
@@ -278,6 +284,20 @@ export function createSSEStream(options: StreamOptions = {}) {
|
||||
}
|
||||
|
||||
const isFinishChunk = parsed.choices?.[0]?.finish_reason;
|
||||
|
||||
// T18: Normalize finish_reason to 'tool_calls' if tool calls were used
|
||||
if (
|
||||
isFinishChunk &&
|
||||
(state as any).passthroughHasToolCalls &&
|
||||
parsed.choices[0].finish_reason !== "tool_calls"
|
||||
) {
|
||||
parsed.choices[0].finish_reason = "tool_calls";
|
||||
// If we modify it, we must output the modified object
|
||||
if (!injectedUsage && hasValidUsage(parsed.usage)) {
|
||||
output = `data: ${JSON.stringify(parsed)}\n`;
|
||||
injectedUsage = true;
|
||||
}
|
||||
}
|
||||
if (isFinishChunk && !hasValidUsage(parsed.usage)) {
|
||||
const estimated = estimateUsage(body, totalContentLength, FORMATS.OPENAI);
|
||||
parsed.usage = filterUsageForFormat(estimated, FORMATS.OPENAI);
|
||||
@@ -529,19 +549,17 @@ export function createSSEStream(options: StreamOptions = {}) {
|
||||
if (!state.usage) {
|
||||
state.usage = extracted;
|
||||
} else {
|
||||
if (extracted.prompt_tokens > 0)
|
||||
state.usage.prompt_tokens = extracted.prompt_tokens;
|
||||
if (extracted.completion_tokens > 0)
|
||||
state.usage.completion_tokens = extracted.completion_tokens;
|
||||
if (extracted.total_tokens > 0) state.usage.total_tokens = extracted.total_tokens;
|
||||
if (extracted.cache_read_input_tokens > 0)
|
||||
state.usage.cache_read_input_tokens = extracted.cache_read_input_tokens;
|
||||
if (extracted.cache_creation_input_tokens > 0)
|
||||
state.usage.cache_creation_input_tokens = extracted.cache_creation_input_tokens;
|
||||
if (extracted.cached_tokens > 0)
|
||||
state.usage.cached_tokens = extracted.cached_tokens;
|
||||
if (extracted.reasoning_tokens > 0)
|
||||
state.usage.reasoning_tokens = extracted.reasoning_tokens;
|
||||
const su = state.usage as Record<string, number>;
|
||||
const eu = extracted as Record<string, number>;
|
||||
if (eu.prompt_tokens > 0) su.prompt_tokens = eu.prompt_tokens;
|
||||
if (eu.completion_tokens > 0) su.completion_tokens = eu.completion_tokens;
|
||||
if (eu.total_tokens > 0) su.total_tokens = eu.total_tokens;
|
||||
if (eu.cache_read_input_tokens > 0)
|
||||
su.cache_read_input_tokens = eu.cache_read_input_tokens;
|
||||
if (eu.cache_creation_input_tokens > 0)
|
||||
su.cache_creation_input_tokens = eu.cache_creation_input_tokens;
|
||||
if (eu.cached_tokens > 0) su.cached_tokens = eu.cached_tokens;
|
||||
if (eu.reasoning_tokens > 0) su.reasoning_tokens = eu.reasoning_tokens;
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -134,7 +134,36 @@ export function createDisconnectAwareStream(transformStream, streamController) {
|
||||
controller.enqueue(value);
|
||||
} catch (error) {
|
||||
streamController.handleError(error);
|
||||
controller.error(error);
|
||||
|
||||
// T35: Encapsulate mid-stream errors as SSE events instead of abruptly aborting
|
||||
// This prevents TransferEncodingError on the client side
|
||||
const errorMsg = error instanceof Error ? error.message : "Upstream stream error";
|
||||
const statusCode =
|
||||
typeof error === "object" && error !== null && "statusCode" in error
|
||||
? (error as any).statusCode
|
||||
: 500;
|
||||
|
||||
const errorEvent = {
|
||||
object: "chat.completion.chunk",
|
||||
choices: [
|
||||
{
|
||||
index: 0,
|
||||
delta: {},
|
||||
finish_reason: "error",
|
||||
},
|
||||
],
|
||||
error: {
|
||||
message: errorMsg,
|
||||
type: "upstream_error",
|
||||
code: statusCode,
|
||||
},
|
||||
};
|
||||
|
||||
const encoder = new TextEncoder();
|
||||
controller.enqueue(encoder.encode(`data: ${JSON.stringify(errorEvent)}\n\n`));
|
||||
controller.enqueue(encoder.encode(`data: [DONE]\n\n`));
|
||||
|
||||
controller.close();
|
||||
}
|
||||
},
|
||||
|
||||
|
||||
114
package-lock.json
generated
114
package-lock.json
generated
@@ -1,12 +1,12 @@
|
||||
{
|
||||
"name": "omniroute",
|
||||
"version": "3.0.0-rc.5",
|
||||
"version": "3.0.0-rc.10",
|
||||
"lockfileVersion": 3,
|
||||
"requires": true,
|
||||
"packages": {
|
||||
"": {
|
||||
"name": "omniroute",
|
||||
"version": "3.0.0-rc.5",
|
||||
"version": "3.0.0-rc.10",
|
||||
"hasInstallScript": true,
|
||||
"license": "MIT",
|
||||
"workspaces": [
|
||||
@@ -28,7 +28,7 @@
|
||||
"jose": "^6.1.3",
|
||||
"lowdb": "^7.0.1",
|
||||
"monaco-editor": "^0.55.1",
|
||||
"next": "^16.1.6",
|
||||
"next": "^16.0.10",
|
||||
"next-intl": "^4.8.3",
|
||||
"node-machine-id": "^1.1.12",
|
||||
"open": "^11.0.0",
|
||||
@@ -61,7 +61,7 @@
|
||||
"concurrently": "^9.2.1",
|
||||
"cross-env": "^10.1.0",
|
||||
"eslint": "^9.39.2",
|
||||
"eslint-config-next": "16.1.6",
|
||||
"eslint-config-next": "^16.0.10",
|
||||
"husky": "^9.1.7",
|
||||
"lint-staged": "^16.2.7",
|
||||
"prettier": "^3.8.1",
|
||||
@@ -2567,15 +2567,15 @@
|
||||
}
|
||||
},
|
||||
"node_modules/@next/env": {
|
||||
"version": "16.1.7",
|
||||
"resolved": "https://registry.npmjs.org/@next/env/-/env-16.1.7.tgz",
|
||||
"integrity": "sha512-rJJbIdJB/RQr2F1nylZr/PJzamvNNhfr3brdKP6s/GW850jbtR70QlSfFselvIBbcPUOlQwBakexjFzqLzF6pg==",
|
||||
"version": "16.0.10",
|
||||
"resolved": "https://registry.npmjs.org/@next/env/-/env-16.0.10.tgz",
|
||||
"integrity": "sha512-8tuaQkyDVgeONQ1MeT9Mkk8pQmZapMKFh5B+OrFUlG3rVmYTXcXlBetBgTurKXGaIZvkoqRT9JL5K3phXcgang==",
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/@next/eslint-plugin-next": {
|
||||
"version": "16.1.6",
|
||||
"resolved": "https://registry.npmjs.org/@next/eslint-plugin-next/-/eslint-plugin-next-16.1.6.tgz",
|
||||
"integrity": "sha512-/Qq3PTagA6+nYVfryAtQ7/9FEr/6YVyvOtl6rZnGsbReGLf0jZU6gkpr1FuChAQpvV46a78p4cmHOVP8mbfSMQ==",
|
||||
"version": "16.0.10",
|
||||
"resolved": "https://registry.npmjs.org/@next/eslint-plugin-next/-/eslint-plugin-next-16.0.10.tgz",
|
||||
"integrity": "sha512-b2NlWN70bbPLmfyoLvvidPKWENBYYIe017ZGUpElvQjDytCWgxPJx7L9juxHt0xHvNVA08ZHJdOyhGzon/KJuw==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
@@ -2583,9 +2583,9 @@
|
||||
}
|
||||
},
|
||||
"node_modules/@next/swc-darwin-arm64": {
|
||||
"version": "16.1.7",
|
||||
"resolved": "https://registry.npmjs.org/@next/swc-darwin-arm64/-/swc-darwin-arm64-16.1.7.tgz",
|
||||
"integrity": "sha512-b2wWIE8sABdyafc4IM8r5Y/dS6kD80JRtOGrUiKTsACFQfWWgUQ2NwoUX1yjFMXVsAwcQeNpnucF2ZrujsBBPg==",
|
||||
"version": "16.0.10",
|
||||
"resolved": "https://registry.npmjs.org/@next/swc-darwin-arm64/-/swc-darwin-arm64-16.0.10.tgz",
|
||||
"integrity": "sha512-4XgdKtdVsaflErz+B5XeG0T5PeXKDdruDf3CRpnhN+8UebNa5N2H58+3GDgpn/9GBurrQ1uWW768FfscwYkJRg==",
|
||||
"cpu": [
|
||||
"arm64"
|
||||
],
|
||||
@@ -2599,9 +2599,9 @@
|
||||
}
|
||||
},
|
||||
"node_modules/@next/swc-darwin-x64": {
|
||||
"version": "16.1.7",
|
||||
"resolved": "https://registry.npmjs.org/@next/swc-darwin-x64/-/swc-darwin-x64-16.1.7.tgz",
|
||||
"integrity": "sha512-zcnVaaZulS1WL0Ss38R5Q6D2gz7MtBu8GZLPfK+73D/hp4GFMrC2sudLky1QibfV7h6RJBJs/gOFvYP0X7UVlQ==",
|
||||
"version": "16.0.10",
|
||||
"resolved": "https://registry.npmjs.org/@next/swc-darwin-x64/-/swc-darwin-x64-16.0.10.tgz",
|
||||
"integrity": "sha512-spbEObMvRKkQ3CkYVOME+ocPDFo5UqHb8EMTS78/0mQ+O1nqE8toHJVioZo4TvebATxgA8XMTHHrScPrn68OGw==",
|
||||
"cpu": [
|
||||
"x64"
|
||||
],
|
||||
@@ -2615,12 +2615,15 @@
|
||||
}
|
||||
},
|
||||
"node_modules/@next/swc-linux-arm64-gnu": {
|
||||
"version": "16.1.7",
|
||||
"resolved": "https://registry.npmjs.org/@next/swc-linux-arm64-gnu/-/swc-linux-arm64-gnu-16.1.7.tgz",
|
||||
"integrity": "sha512-2ant89Lux/Q3VyC8vNVg7uBaFVP9SwoK2jJOOR0L8TQnX8CAYnh4uctAScy2Hwj2dgjVHqHLORQZJ2wH6VxhSQ==",
|
||||
"version": "16.0.10",
|
||||
"resolved": "https://registry.npmjs.org/@next/swc-linux-arm64-gnu/-/swc-linux-arm64-gnu-16.0.10.tgz",
|
||||
"integrity": "sha512-uQtWE3X0iGB8apTIskOMi2w/MKONrPOUCi5yLO+v3O8Mb5c7K4Q5KD1jvTpTF5gJKa3VH/ijKjKUq9O9UhwOYw==",
|
||||
"cpu": [
|
||||
"arm64"
|
||||
],
|
||||
"libc": [
|
||||
"glibc"
|
||||
],
|
||||
"license": "MIT",
|
||||
"optional": true,
|
||||
"os": [
|
||||
@@ -2631,12 +2634,15 @@
|
||||
}
|
||||
},
|
||||
"node_modules/@next/swc-linux-arm64-musl": {
|
||||
"version": "16.1.7",
|
||||
"resolved": "https://registry.npmjs.org/@next/swc-linux-arm64-musl/-/swc-linux-arm64-musl-16.1.7.tgz",
|
||||
"integrity": "sha512-uufcze7LYv0FQg9GnNeZ3/whYfo+1Q3HnQpm16o6Uyi0OVzLlk2ZWoY7j07KADZFY8qwDbsmFnMQP3p3+Ftprw==",
|
||||
"version": "16.0.10",
|
||||
"resolved": "https://registry.npmjs.org/@next/swc-linux-arm64-musl/-/swc-linux-arm64-musl-16.0.10.tgz",
|
||||
"integrity": "sha512-llA+hiDTrYvyWI21Z0L1GiXwjQaanPVQQwru5peOgtooeJ8qx3tlqRV2P7uH2pKQaUfHxI/WVarvI5oYgGxaTw==",
|
||||
"cpu": [
|
||||
"arm64"
|
||||
],
|
||||
"libc": [
|
||||
"musl"
|
||||
],
|
||||
"license": "MIT",
|
||||
"optional": true,
|
||||
"os": [
|
||||
@@ -2647,12 +2653,15 @@
|
||||
}
|
||||
},
|
||||
"node_modules/@next/swc-linux-x64-gnu": {
|
||||
"version": "16.1.7",
|
||||
"resolved": "https://registry.npmjs.org/@next/swc-linux-x64-gnu/-/swc-linux-x64-gnu-16.1.7.tgz",
|
||||
"integrity": "sha512-KWVf2gxYvHtvuT+c4MBOGxuse5TD7DsMFYSxVxRBnOzok/xryNeQSjXgxSv9QpIVlaGzEn/pIuI6Koosx8CGWA==",
|
||||
"version": "16.0.10",
|
||||
"resolved": "https://registry.npmjs.org/@next/swc-linux-x64-gnu/-/swc-linux-x64-gnu-16.0.10.tgz",
|
||||
"integrity": "sha512-AK2q5H0+a9nsXbeZ3FZdMtbtu9jxW4R/NgzZ6+lrTm3d6Zb7jYrWcgjcpM1k8uuqlSy4xIyPR2YiuUr+wXsavA==",
|
||||
"cpu": [
|
||||
"x64"
|
||||
],
|
||||
"libc": [
|
||||
"glibc"
|
||||
],
|
||||
"license": "MIT",
|
||||
"optional": true,
|
||||
"os": [
|
||||
@@ -2663,12 +2672,15 @@
|
||||
}
|
||||
},
|
||||
"node_modules/@next/swc-linux-x64-musl": {
|
||||
"version": "16.1.7",
|
||||
"resolved": "https://registry.npmjs.org/@next/swc-linux-x64-musl/-/swc-linux-x64-musl-16.1.7.tgz",
|
||||
"integrity": "sha512-HguhaGwsGr1YAGs68uRKc4aGWxLET+NevJskOcCAwXbwj0fYX0RgZW2gsOCzr9S11CSQPIkxmoSbuVaBp4Z3dA==",
|
||||
"version": "16.0.10",
|
||||
"resolved": "https://registry.npmjs.org/@next/swc-linux-x64-musl/-/swc-linux-x64-musl-16.0.10.tgz",
|
||||
"integrity": "sha512-1TDG9PDKivNw5550S111gsO4RGennLVl9cipPhtkXIFVwo31YZ73nEbLjNC8qG3SgTz/QZyYyaFYMeY4BKZR/g==",
|
||||
"cpu": [
|
||||
"x64"
|
||||
],
|
||||
"libc": [
|
||||
"musl"
|
||||
],
|
||||
"license": "MIT",
|
||||
"optional": true,
|
||||
"os": [
|
||||
@@ -2679,9 +2691,9 @@
|
||||
}
|
||||
},
|
||||
"node_modules/@next/swc-win32-arm64-msvc": {
|
||||
"version": "16.1.7",
|
||||
"resolved": "https://registry.npmjs.org/@next/swc-win32-arm64-msvc/-/swc-win32-arm64-msvc-16.1.7.tgz",
|
||||
"integrity": "sha512-S0n3KrDJokKTeFyM/vGGGR8+pCmXYrjNTk2ZozOL1C/JFdfUIL9O1ATaJOl5r2POe56iRChbsszrjMAdWSv7kQ==",
|
||||
"version": "16.0.10",
|
||||
"resolved": "https://registry.npmjs.org/@next/swc-win32-arm64-msvc/-/swc-win32-arm64-msvc-16.0.10.tgz",
|
||||
"integrity": "sha512-aEZIS4Hh32xdJQbHz121pyuVZniSNoqDVx1yIr2hy+ZwJGipeqnMZBJHyMxv2tiuAXGx6/xpTcQJ6btIiBjgmg==",
|
||||
"cpu": [
|
||||
"arm64"
|
||||
],
|
||||
@@ -2695,9 +2707,9 @@
|
||||
}
|
||||
},
|
||||
"node_modules/@next/swc-win32-x64-msvc": {
|
||||
"version": "16.1.7",
|
||||
"resolved": "https://registry.npmjs.org/@next/swc-win32-x64-msvc/-/swc-win32-x64-msvc-16.1.7.tgz",
|
||||
"integrity": "sha512-mwgtg8CNZGYm06LeEd+bNnOUfwOyNem/rOiP14Lsz+AnUY92Zq/LXwtebtUiaeVkhbroRCQ0c8GlR4UT1U+0yg==",
|
||||
"version": "16.0.10",
|
||||
"resolved": "https://registry.npmjs.org/@next/swc-win32-x64-msvc/-/swc-win32-x64-msvc-16.0.10.tgz",
|
||||
"integrity": "sha512-E+njfCoFLb01RAFEnGZn6ERoOqhK1Gl3Lfz1Kjnj0Ulfu7oJbuMyvBKNj/bw8XZnenHDASlygTjZICQW+rYW1Q==",
|
||||
"cpu": [
|
||||
"x64"
|
||||
],
|
||||
@@ -7522,6 +7534,7 @@
|
||||
"version": "2.9.19",
|
||||
"resolved": "https://registry.npmjs.org/baseline-browser-mapping/-/baseline-browser-mapping-2.9.19.tgz",
|
||||
"integrity": "sha512-ipDqC8FrAl/76p2SSWKSI+H9tFwm7vYqXQrItCuiVPt26Km0jS+NzSsBWAaBusvSbQcfJG+JitdMm+wZAgTYqg==",
|
||||
"dev": true,
|
||||
"license": "Apache-2.0",
|
||||
"bin": {
|
||||
"baseline-browser-mapping": "dist/cli.js"
|
||||
@@ -9705,13 +9718,13 @@
|
||||
}
|
||||
},
|
||||
"node_modules/eslint-config-next": {
|
||||
"version": "16.1.6",
|
||||
"resolved": "https://registry.npmjs.org/eslint-config-next/-/eslint-config-next-16.1.6.tgz",
|
||||
"integrity": "sha512-vKq40io2B0XtkkNDYyleATwblNt8xuh3FWp8SpSz3pt7P01OkBFlKsJZ2mWt5WsCySlDQLckb1zMY9yE9Qy0LA==",
|
||||
"version": "16.0.10",
|
||||
"resolved": "https://registry.npmjs.org/eslint-config-next/-/eslint-config-next-16.0.10.tgz",
|
||||
"integrity": "sha512-BxouZUm0I45K4yjOOIzj24nTi0H2cGo0y7xUmk+Po/PYtJXFBYVDS1BguE7t28efXjKdcN0tmiLivxQy//SsZg==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"@next/eslint-plugin-next": "16.1.6",
|
||||
"@next/eslint-plugin-next": "16.0.10",
|
||||
"eslint-import-resolver-node": "^0.3.6",
|
||||
"eslint-import-resolver-typescript": "^3.5.2",
|
||||
"eslint-plugin-import": "^2.32.0",
|
||||
@@ -14658,14 +14671,13 @@
|
||||
}
|
||||
},
|
||||
"node_modules/next": {
|
||||
"version": "16.1.7",
|
||||
"resolved": "https://registry.npmjs.org/next/-/next-16.1.7.tgz",
|
||||
"integrity": "sha512-WM0L7WrSvKwoLegLYr6V+mz+RIofqQgVAfHhMp9a88ms0cFX8iX9ew+snpWlSBwpkURJOUdvCEt3uLl3NNzvWg==",
|
||||
"version": "16.0.10",
|
||||
"resolved": "https://registry.npmjs.org/next/-/next-16.0.10.tgz",
|
||||
"integrity": "sha512-RtWh5PUgI+vxlV3HdR+IfWA1UUHu0+Ram/JBO4vWB54cVPentCD0e+lxyAYEsDTqGGMg7qpjhKh6dc6aW7W/sA==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"@next/env": "16.1.7",
|
||||
"@next/env": "16.0.10",
|
||||
"@swc/helpers": "0.5.15",
|
||||
"baseline-browser-mapping": "^2.9.19",
|
||||
"caniuse-lite": "^1.0.30001579",
|
||||
"postcss": "8.4.31",
|
||||
"styled-jsx": "5.1.6"
|
||||
@@ -14677,14 +14689,14 @@
|
||||
"node": ">=20.9.0"
|
||||
},
|
||||
"optionalDependencies": {
|
||||
"@next/swc-darwin-arm64": "16.1.7",
|
||||
"@next/swc-darwin-x64": "16.1.7",
|
||||
"@next/swc-linux-arm64-gnu": "16.1.7",
|
||||
"@next/swc-linux-arm64-musl": "16.1.7",
|
||||
"@next/swc-linux-x64-gnu": "16.1.7",
|
||||
"@next/swc-linux-x64-musl": "16.1.7",
|
||||
"@next/swc-win32-arm64-msvc": "16.1.7",
|
||||
"@next/swc-win32-x64-msvc": "16.1.7",
|
||||
"@next/swc-darwin-arm64": "16.0.10",
|
||||
"@next/swc-darwin-x64": "16.0.10",
|
||||
"@next/swc-linux-arm64-gnu": "16.0.10",
|
||||
"@next/swc-linux-arm64-musl": "16.0.10",
|
||||
"@next/swc-linux-x64-gnu": "16.0.10",
|
||||
"@next/swc-linux-x64-musl": "16.0.10",
|
||||
"@next/swc-win32-arm64-msvc": "16.0.10",
|
||||
"@next/swc-win32-x64-msvc": "16.0.10",
|
||||
"sharp": "^0.34.4"
|
||||
},
|
||||
"peerDependencies": {
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "omniroute",
|
||||
"version": "3.0.0-rc.5",
|
||||
"version": "3.0.0-rc.10",
|
||||
"description": "Smart AI Router with auto fallback — route to FREE & cheap models, zero downtime. Works with Cursor, Cline, Claude Desktop, Codex, and any OpenAI-compatible tool.",
|
||||
"type": "module",
|
||||
"bin": {
|
||||
@@ -96,7 +96,7 @@
|
||||
"jose": "^6.1.3",
|
||||
"lowdb": "^7.0.1",
|
||||
"monaco-editor": "^0.55.1",
|
||||
"next": "^16.1.6",
|
||||
"next": "^16.0.10",
|
||||
"next-intl": "^4.8.3",
|
||||
"node-machine-id": "^1.1.12",
|
||||
"open": "^11.0.0",
|
||||
@@ -125,7 +125,7 @@
|
||||
"concurrently": "^9.2.1",
|
||||
"cross-env": "^10.1.0",
|
||||
"eslint": "^9.39.2",
|
||||
"eslint-config-next": "16.1.6",
|
||||
"eslint-config-next": "^16.0.10",
|
||||
"husky": "^9.1.7",
|
||||
"lint-staged": "^16.2.7",
|
||||
"prettier": "^3.8.1",
|
||||
|
||||
@@ -3017,6 +3017,7 @@ CooldownTimer.propTypes = {
|
||||
const ERROR_TYPE_LABELS = {
|
||||
runtime_error: { labelKey: "errorTypeRuntime", variant: "warning" },
|
||||
upstream_auth_error: { labelKey: "errorTypeUpstreamAuth", variant: "error" },
|
||||
account_deactivated: { labelKey: "Account Deactivated", variant: "error" },
|
||||
auth_missing: { labelKey: "errorTypeMissingCredential", variant: "warning" },
|
||||
token_refresh_failed: { labelKey: "errorTypeRefreshFailed", variant: "warning" },
|
||||
token_expired: { labelKey: "errorTypeTokenExpired", variant: "warning" },
|
||||
@@ -3025,10 +3026,14 @@ const ERROR_TYPE_LABELS = {
|
||||
network_error: { labelKey: "errorTypeNetworkError", variant: "warning" },
|
||||
unsupported: { labelKey: "errorTypeTestUnsupported", variant: "default" },
|
||||
upstream_error: { labelKey: "errorTypeUpstreamError", variant: "error" },
|
||||
banned: { labelKey: "403 Banned", variant: "error" },
|
||||
credits_exhausted: { labelKey: "No Credits", variant: "warning" },
|
||||
};
|
||||
|
||||
function inferErrorType(connection, isCooldown) {
|
||||
if (isCooldown) return "upstream_rate_limited";
|
||||
if (connection.testStatus === "banned") return "banned";
|
||||
if (connection.testStatus === "credits_exhausted") return "credits_exhausted";
|
||||
if (connection.lastErrorType) return connection.lastErrorType;
|
||||
|
||||
const code = Number(connection.errorCode);
|
||||
@@ -3108,6 +3113,16 @@ function getStatusPresentation(connection, effectiveStatus, isCooldown, t) {
|
||||
};
|
||||
}
|
||||
|
||||
if (errorType === "account_deactivated") {
|
||||
return {
|
||||
statusVariant: "error",
|
||||
statusLabel: t("statusDeactivated", "Deactivated"),
|
||||
errorType,
|
||||
errorBadge,
|
||||
errorTextClass: "text-red-600 font-bold",
|
||||
};
|
||||
}
|
||||
|
||||
if (
|
||||
errorType === "upstream_auth_error" ||
|
||||
errorType === "auth_missing" ||
|
||||
@@ -3153,6 +3168,26 @@ function getStatusPresentation(connection, effectiveStatus, isCooldown, t) {
|
||||
};
|
||||
}
|
||||
|
||||
if (errorType === "banned") {
|
||||
return {
|
||||
statusVariant: "error",
|
||||
statusLabel: t("statusBanned", "Banned (403)"),
|
||||
errorType,
|
||||
errorBadge,
|
||||
errorTextClass: "text-red-600 font-bold",
|
||||
};
|
||||
}
|
||||
|
||||
if (errorType === "credits_exhausted") {
|
||||
return {
|
||||
statusVariant: "warning",
|
||||
statusLabel: t("statusCreditsExhausted", "Out of Credits"),
|
||||
errorType,
|
||||
errorBadge,
|
||||
errorTextClass: "text-amber-500",
|
||||
};
|
||||
}
|
||||
|
||||
const fallbackStatusMap = {
|
||||
unavailable: t("statusUnavailable"),
|
||||
failed: t("statusFailed"),
|
||||
@@ -3520,12 +3555,16 @@ function AddApiKeyModal({
|
||||
const t = useTranslations("providers");
|
||||
const isBailian = provider === "bailian-coding-plan";
|
||||
const defaultBailianUrl = "https://coding-intl.dashscope.aliyuncs.com/apps/anthropic/v1";
|
||||
const isVertex = provider === "vertex";
|
||||
const defaultRegion = "us-central1";
|
||||
|
||||
const [formData, setFormData] = useState({
|
||||
name: "",
|
||||
apiKey: "",
|
||||
priority: 1,
|
||||
baseUrl: isBailian ? defaultBailianUrl : "",
|
||||
region: isVertex ? defaultRegion : "",
|
||||
validationModelId: "",
|
||||
});
|
||||
const [validating, setValidating] = useState(false);
|
||||
const [validationResult, setValidationResult] = useState(null);
|
||||
@@ -3539,7 +3578,11 @@ function AddApiKeyModal({
|
||||
const res = await fetch("/api/providers/validate", {
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify({ provider, apiKey: formData.apiKey }),
|
||||
body: JSON.stringify({
|
||||
provider,
|
||||
apiKey: formData.apiKey,
|
||||
validationModelId: formData.validationModelId || undefined,
|
||||
}),
|
||||
});
|
||||
const data = await res.json();
|
||||
setValidationResult(data.valid ? "success" : "failed");
|
||||
@@ -3573,7 +3616,11 @@ function AddApiKeyModal({
|
||||
const res = await fetch("/api/providers/validate", {
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify({ provider, apiKey: formData.apiKey }),
|
||||
body: JSON.stringify({
|
||||
provider,
|
||||
apiKey: formData.apiKey,
|
||||
validationModelId: formData.validationModelId || undefined,
|
||||
}),
|
||||
});
|
||||
const data = await res.json();
|
||||
isValid = !!data.valid;
|
||||
@@ -3602,6 +3649,10 @@ function AddApiKeyModal({
|
||||
payload.providerSpecificData = {
|
||||
baseUrl: validatedBailianBaseUrl,
|
||||
};
|
||||
} else if (isVertex) {
|
||||
payload.providerSpecificData = {
|
||||
region: formData.region,
|
||||
};
|
||||
}
|
||||
|
||||
const error = await onSave(payload);
|
||||
@@ -3635,6 +3686,7 @@ function AddApiKeyModal({
|
||||
value={formData.apiKey}
|
||||
onChange={(e) => setFormData({ ...formData, apiKey: e.target.value })}
|
||||
className="flex-1"
|
||||
placeholder={isVertex ? "Cole o Service Account JSON aqui" : undefined}
|
||||
/>
|
||||
<div className="pt-6">
|
||||
<Button
|
||||
@@ -3667,6 +3719,13 @@ function AddApiKeyModal({
|
||||
})}
|
||||
</p>
|
||||
)}
|
||||
<Input
|
||||
label="Model ID (opcional)"
|
||||
placeholder="ex: grok-3 ou meta-llama/Llama-3.1-8B-Instruct"
|
||||
value={formData.validationModelId}
|
||||
onChange={(e) => setFormData({ ...formData, validationModelId: e.target.value })}
|
||||
hint="Usado como fallback se a listagem de models não estiver disponível"
|
||||
/>
|
||||
<Input
|
||||
label={t("priorityLabel")}
|
||||
type="number"
|
||||
@@ -3684,6 +3743,15 @@ function AddApiKeyModal({
|
||||
hint="Optional: Custom base URL for bailian-coding-plan provider"
|
||||
/>
|
||||
)}
|
||||
{isVertex && (
|
||||
<Input
|
||||
label="Região (Region)"
|
||||
value={formData.region}
|
||||
onChange={(e) => setFormData({ ...formData, region: e.target.value })}
|
||||
placeholder={defaultRegion}
|
||||
hint="ex: us-central1 ou europe-west4. Partner models usam a região global automaticamente."
|
||||
/>
|
||||
)}
|
||||
<div className="flex gap-2">
|
||||
<Button
|
||||
onClick={handleSubmit}
|
||||
@@ -3732,6 +3800,8 @@ function EditConnectionModal({ isOpen, connection, onSave, onClose }: EditConnec
|
||||
apiKey: "",
|
||||
healthCheckInterval: 60,
|
||||
baseUrl: "",
|
||||
region: "",
|
||||
validationModelId: "",
|
||||
});
|
||||
const [testing, setTesting] = useState(false);
|
||||
const [testResult, setTestResult] = useState(null);
|
||||
@@ -3744,17 +3814,23 @@ function EditConnectionModal({ isOpen, connection, onSave, onClose }: EditConnec
|
||||
|
||||
const isBailian = connection?.provider === "bailian-coding-plan";
|
||||
const defaultBailianUrl = "https://coding-intl.dashscope.aliyuncs.com/apps/anthropic/v1";
|
||||
const isVertex = connection?.provider === "vertex";
|
||||
const defaultRegion = "us-central1";
|
||||
|
||||
useEffect(() => {
|
||||
if (connection) {
|
||||
const rawBaseUrl = connection.providerSpecificData?.baseUrl;
|
||||
const existingBaseUrl = typeof rawBaseUrl === "string" ? rawBaseUrl : "";
|
||||
const rawRegion = connection.providerSpecificData?.region;
|
||||
const existingRegion = typeof rawRegion === "string" ? rawRegion : "";
|
||||
setFormData({
|
||||
name: connection.name || "",
|
||||
priority: connection.priority || 1,
|
||||
apiKey: "",
|
||||
healthCheckInterval: connection.healthCheckInterval ?? 60,
|
||||
baseUrl: existingBaseUrl || (isBailian ? defaultBailianUrl : ""),
|
||||
region: existingRegion || (isVertex ? defaultRegion : ""),
|
||||
validationModelId: connection.providerSpecificData?.validationModelId || "",
|
||||
});
|
||||
// Load existing extra keys from providerSpecificData
|
||||
const existing = connection.providerSpecificData?.extraApiKeys;
|
||||
@@ -3771,7 +3847,13 @@ function EditConnectionModal({ isOpen, connection, onSave, onClose }: EditConnec
|
||||
setTesting(true);
|
||||
setTestResult(null);
|
||||
try {
|
||||
const res = await fetch(`/api/providers/${connection.id}/test`, { method: "POST" });
|
||||
const res = await fetch(`/api/providers/${connection.id}/test`, {
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify({
|
||||
validationModelId: formData.validationModelId || undefined,
|
||||
}),
|
||||
});
|
||||
const data = await res.json();
|
||||
setTestResult({
|
||||
valid: !!data.valid,
|
||||
@@ -3797,7 +3879,11 @@ function EditConnectionModal({ isOpen, connection, onSave, onClose }: EditConnec
|
||||
const res = await fetch("/api/providers/validate", {
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify({ provider: connection.provider, apiKey: formData.apiKey }),
|
||||
body: JSON.stringify({
|
||||
provider: connection.provider,
|
||||
apiKey: formData.apiKey,
|
||||
validationModelId: formData.validationModelId || undefined,
|
||||
}),
|
||||
});
|
||||
const data = await res.json();
|
||||
setValidationResult(data.valid ? "success" : "failed");
|
||||
@@ -3838,7 +3924,11 @@ function EditConnectionModal({ isOpen, connection, onSave, onClose }: EditConnec
|
||||
const res = await fetch("/api/providers/validate", {
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify({ provider: connection.provider, apiKey: formData.apiKey }),
|
||||
body: JSON.stringify({
|
||||
provider: connection.provider,
|
||||
apiKey: formData.apiKey,
|
||||
validationModelId: formData.validationModelId || undefined,
|
||||
}),
|
||||
});
|
||||
const data = await res.json();
|
||||
isValid = !!data.valid;
|
||||
@@ -3865,9 +3955,14 @@ function EditConnectionModal({ isOpen, connection, onSave, onClose }: EditConnec
|
||||
...(connection.providerSpecificData || {}),
|
||||
extraApiKeys: extraApiKeys.filter((k) => k.trim().length > 0),
|
||||
};
|
||||
if (formData.validationModelId) {
|
||||
updates.providerSpecificData.validationModelId = formData.validationModelId;
|
||||
}
|
||||
// Update baseUrl for bailian-coding-plan
|
||||
if (isBailian) {
|
||||
updates.providerSpecificData.baseUrl = validatedBailianBaseUrl;
|
||||
} else if (isVertex) {
|
||||
updates.providerSpecificData.region = formData.region;
|
||||
}
|
||||
}
|
||||
const error = (await onSave(updates)) as void | unknown;
|
||||
@@ -3935,7 +4030,7 @@ function EditConnectionModal({ isOpen, connection, onSave, onClose }: EditConnec
|
||||
type="password"
|
||||
value={formData.apiKey}
|
||||
onChange={(e) => setFormData({ ...formData, apiKey: e.target.value })}
|
||||
placeholder={t("enterNewApiKey")}
|
||||
placeholder={isVertex ? "Cole o Service Account JSON aqui" : t("enterNewApiKey")}
|
||||
hint={t("leaveBlankKeepCurrentApiKey")}
|
||||
className="flex-1"
|
||||
/>
|
||||
@@ -3959,6 +4054,13 @@ function EditConnectionModal({ isOpen, connection, onSave, onClose }: EditConnec
|
||||
{saveError}
|
||||
</div>
|
||||
)}
|
||||
<Input
|
||||
label="Model ID (opcional)"
|
||||
placeholder="ex: grok-3 ou meta-llama/Llama-3.1-8B-Instruct"
|
||||
value={formData.validationModelId}
|
||||
onChange={(e) => setFormData({ ...formData, validationModelId: e.target.value })}
|
||||
hint="Usado como fallback se a listagem de models não estiver disponível"
|
||||
/>
|
||||
</>
|
||||
)}
|
||||
|
||||
@@ -3972,6 +4074,16 @@ function EditConnectionModal({ isOpen, connection, onSave, onClose }: EditConnec
|
||||
/>
|
||||
)}
|
||||
|
||||
{isVertex && (
|
||||
<Input
|
||||
label="Região (Region)"
|
||||
value={formData.region}
|
||||
onChange={(e) => setFormData({ ...formData, region: e.target.value })}
|
||||
placeholder={defaultRegion}
|
||||
hint="ex: us-central1 ou europe-west4. Partner models usam a região global automaticamente."
|
||||
/>
|
||||
)}
|
||||
|
||||
{/* T07: Extra API Keys for round-robin rotation */}
|
||||
{!isOAuth && (
|
||||
<div className="flex flex-col gap-2">
|
||||
|
||||
@@ -4,6 +4,7 @@ import {
|
||||
isOpenAICompatibleProvider,
|
||||
isAnthropicCompatibleProvider,
|
||||
} from "@/shared/constants/providers";
|
||||
import { PROVIDER_MODELS } from "@/shared/constants/models";
|
||||
|
||||
type JsonRecord = Record<string, unknown>;
|
||||
|
||||
@@ -336,39 +337,85 @@ export async function GET(request, { params }) {
|
||||
);
|
||||
}
|
||||
|
||||
let modelsUrl = baseUrl.replace(/\/$/, "");
|
||||
if (modelsUrl.endsWith("/chat/completions")) {
|
||||
modelsUrl = modelsUrl.slice(0, -17) + "/models";
|
||||
} else if (modelsUrl.endsWith("/completions")) {
|
||||
modelsUrl = modelsUrl.slice(0, -12) + "/models";
|
||||
} else {
|
||||
modelsUrl = `${modelsUrl}/models`;
|
||||
let base = baseUrl.replace(/\/$/, "");
|
||||
if (base.endsWith("/chat/completions")) {
|
||||
base = base.slice(0, -17);
|
||||
} else if (base.endsWith("/completions")) {
|
||||
base = base.slice(0, -12);
|
||||
} else if (base.endsWith("/v1")) {
|
||||
base = base.slice(0, -3);
|
||||
}
|
||||
|
||||
const response = await fetch(modelsUrl, {
|
||||
method: "GET",
|
||||
headers: {
|
||||
"Content-Type": "application/json",
|
||||
Authorization: `Bearer ${apiKey}`,
|
||||
},
|
||||
});
|
||||
// T39: Try multiple endpoint formats
|
||||
const endpoints = [
|
||||
`${base}/v1/models`,
|
||||
`${base}/models`,
|
||||
`${baseUrl.replace(/\/$/, "")}/models`, // Original fallback
|
||||
];
|
||||
|
||||
if (!response.ok) {
|
||||
const errorText = await response.text();
|
||||
console.log(`Error fetching models from ${provider}:`, errorText);
|
||||
return NextResponse.json(
|
||||
{ error: `Failed to fetch models: ${response.status}` },
|
||||
{ status: response.status }
|
||||
);
|
||||
// Remove duplicates
|
||||
const uniqueEndpoints = [...new Set(endpoints)];
|
||||
let models = null;
|
||||
let lastErrorStatus = null;
|
||||
|
||||
for (const modelsUrl of uniqueEndpoints) {
|
||||
try {
|
||||
const response = await fetch(modelsUrl, {
|
||||
method: "GET",
|
||||
headers: {
|
||||
"Content-Type": "application/json",
|
||||
Authorization: `Bearer ${apiKey}`,
|
||||
},
|
||||
signal: AbortSignal.timeout(5000), // Quick timeout for fallbacks
|
||||
});
|
||||
|
||||
if (response.ok) {
|
||||
const data = await response.json();
|
||||
models = data.data || data.models || [];
|
||||
break; // Success!
|
||||
}
|
||||
|
||||
if (response.status === 401 || response.status === 403) {
|
||||
lastErrorStatus = response.status;
|
||||
throw new Error("auth_failed");
|
||||
}
|
||||
} catch (err: any) {
|
||||
if (err.message === "auth_failed") break; // Don't try other endpoints if auth failed
|
||||
}
|
||||
}
|
||||
|
||||
const data = await response.json();
|
||||
const models = data.data || data.models || [];
|
||||
// If all endpoints failed (but not because of auth), fallback to local catalog
|
||||
if (!models) {
|
||||
if (lastErrorStatus === 401 || lastErrorStatus === 403) {
|
||||
return NextResponse.json(
|
||||
{ error: `Auth failed: ${lastErrorStatus}` },
|
||||
{ status: lastErrorStatus }
|
||||
);
|
||||
}
|
||||
|
||||
console.warn(`[models] All endpoints failed for ${provider}, using local catalog`);
|
||||
const localModels = PROVIDER_MODELS[provider] || [];
|
||||
models = localModels.map((m: any) => ({
|
||||
id: m.id,
|
||||
name: m.name || m.id,
|
||||
owned_by: provider,
|
||||
}));
|
||||
}
|
||||
|
||||
// Track source for MCP tool T39 requirement
|
||||
const source =
|
||||
models === null || (models && models.length > 0 && models[0].owned_by === provider)
|
||||
? "local_catalog"
|
||||
: "api";
|
||||
|
||||
return NextResponse.json({
|
||||
provider,
|
||||
connectionId,
|
||||
models,
|
||||
source,
|
||||
...(source === "local_catalog"
|
||||
? { warning: "API unavailable — using cached catalog" }
|
||||
: {}),
|
||||
});
|
||||
}
|
||||
|
||||
|
||||
@@ -523,9 +523,10 @@ async function testApiKeyConnection(connection: any) {
|
||||
/**
|
||||
* Core test logic — reusable by test-batch without HTTP self-calls.
|
||||
* @param {string} connectionId
|
||||
* @param {string} validationModelId Optional custom model ID to test connection with
|
||||
* @returns {Promise<object>} Test result (same shape as the JSON response)
|
||||
*/
|
||||
export async function testSingleConnection(connectionId: string) {
|
||||
export async function testSingleConnection(connectionId: string, validationModelId?: string) {
|
||||
const connection = await getProviderConnectionById(connectionId);
|
||||
|
||||
if (!connection) {
|
||||
@@ -567,8 +568,17 @@ export async function testSingleConnection(connectionId: string) {
|
||||
diagnosis: (runtime as any).diagnosis,
|
||||
};
|
||||
} else if (connection.authType === "apikey") {
|
||||
const enrichedConnection = validationModelId
|
||||
? {
|
||||
...connection,
|
||||
providerSpecificData: {
|
||||
...((connection.providerSpecificData as any) || {}),
|
||||
validationModelId,
|
||||
},
|
||||
}
|
||||
: connection;
|
||||
result = await runWithProxyContext(proxyInfo?.proxy || null, () =>
|
||||
testApiKeyConnection(connection)
|
||||
testApiKeyConnection(enrichedConnection)
|
||||
);
|
||||
} else {
|
||||
result = await runWithProxyContext(proxyInfo?.proxy || null, () =>
|
||||
@@ -670,7 +680,17 @@ export async function testSingleConnection(connectionId: string) {
|
||||
export async function POST(request: Request, { params }: { params: Promise<{ id: string }> }) {
|
||||
try {
|
||||
const { id } = await params;
|
||||
const data = await testSingleConnection(id);
|
||||
|
||||
// Parse optional body for validationModelId
|
||||
let validationModelId;
|
||||
try {
|
||||
const body = await request.json();
|
||||
validationModelId = body?.validationModelId;
|
||||
} catch {
|
||||
// Body is optional
|
||||
}
|
||||
|
||||
const data = await testSingleConnection(id, validationModelId);
|
||||
|
||||
if (data.error === "Connection not found") {
|
||||
return NextResponse.json({ error: "Connection not found" }, { status: 404 });
|
||||
|
||||
@@ -30,9 +30,9 @@ export async function POST(request) {
|
||||
if (isValidationFailure(validation)) {
|
||||
return NextResponse.json({ error: validation.error }, { status: 400 });
|
||||
}
|
||||
const { provider, apiKey } = validation.data;
|
||||
const { provider, apiKey, validationModelId } = validation.data;
|
||||
|
||||
let providerSpecificData = {};
|
||||
let providerSpecificData: any = { validationModelId };
|
||||
|
||||
if (isOpenAICompatibleProvider(provider) || isAnthropicCompatibleProvider(provider)) {
|
||||
const node: any = await getProviderNodeById(provider);
|
||||
@@ -44,6 +44,7 @@ export async function POST(request) {
|
||||
);
|
||||
}
|
||||
providerSpecificData = {
|
||||
...providerSpecificData,
|
||||
baseUrl: node.baseUrl,
|
||||
apiType: node.apiType,
|
||||
};
|
||||
|
||||
@@ -1,6 +1,36 @@
|
||||
import { NextResponse } from "next/server";
|
||||
import { getUsageDb } from "@/lib/usageDb";
|
||||
import { computeAnalytics } from "@/lib/usageAnalytics";
|
||||
import { getDbInstance } from "@/lib/db/core";
|
||||
|
||||
function getRangeStartIso(range: string): string | null {
|
||||
const end = new Date();
|
||||
const start = new Date(end);
|
||||
|
||||
switch (range) {
|
||||
case "1d":
|
||||
start.setDate(start.getDate() - 1);
|
||||
break;
|
||||
case "7d":
|
||||
start.setDate(start.getDate() - 7);
|
||||
break;
|
||||
case "30d":
|
||||
start.setDate(start.getDate() - 30);
|
||||
break;
|
||||
case "90d":
|
||||
start.setDate(start.getDate() - 90);
|
||||
break;
|
||||
case "ytd":
|
||||
start.setMonth(0, 1);
|
||||
start.setHours(0, 0, 0, 0);
|
||||
break;
|
||||
case "all":
|
||||
default:
|
||||
return null;
|
||||
}
|
||||
|
||||
return start.toISOString();
|
||||
}
|
||||
|
||||
export async function GET(request) {
|
||||
try {
|
||||
@@ -36,6 +66,47 @@ export async function GET(request) {
|
||||
|
||||
const analytics = await computeAnalytics(history, range, connectionMap);
|
||||
|
||||
// T01: fallback transparency metrics from call_logs (requested_model vs routed model).
|
||||
try {
|
||||
const db = getDbInstance();
|
||||
const sinceIso = getRangeStartIso(range);
|
||||
const whereClause = sinceIso ? "WHERE timestamp >= @since" : "";
|
||||
const row = db
|
||||
.prepare(
|
||||
`
|
||||
SELECT
|
||||
COUNT(*) as total,
|
||||
SUM(CASE WHEN requested_model IS NOT NULL AND requested_model != '' THEN 1 ELSE 0 END) as with_requested,
|
||||
SUM(CASE
|
||||
WHEN requested_model IS NOT NULL
|
||||
AND requested_model != ''
|
||||
AND model IS NOT NULL
|
||||
AND requested_model != model
|
||||
THEN 1 ELSE 0 END
|
||||
) as fallbacks
|
||||
FROM call_logs
|
||||
${whereClause}
|
||||
`
|
||||
)
|
||||
.get(sinceIso ? { since: sinceIso } : {}) as
|
||||
| { total?: number; with_requested?: number; fallbacks?: number }
|
||||
| undefined;
|
||||
|
||||
const total = Number(row?.total || 0);
|
||||
const withRequested = Number(row?.with_requested || 0);
|
||||
const fallbackCount = Number(row?.fallbacks || 0);
|
||||
|
||||
analytics.summary.fallbackCount = fallbackCount;
|
||||
analytics.summary.fallbackRatePct =
|
||||
withRequested > 0 ? Number(((fallbackCount / withRequested) * 100).toFixed(2)) : 0;
|
||||
analytics.summary.requestedModelCoveragePct =
|
||||
total > 0 ? Number(((withRequested / total) * 100).toFixed(2)) : 0;
|
||||
} catch {
|
||||
analytics.summary.fallbackCount = 0;
|
||||
analytics.summary.fallbackRatePct = 0;
|
||||
analytics.summary.requestedModelCoveragePct = 0;
|
||||
}
|
||||
|
||||
return NextResponse.json(analytics);
|
||||
} catch (error) {
|
||||
console.error("Error computing analytics:", error);
|
||||
|
||||
@@ -40,6 +40,8 @@ interface ApiKeyMetadata {
|
||||
accessSchedule: AccessSchedule | null;
|
||||
maxRequestsPerDay: number | null;
|
||||
maxRequestsPerMinute: number | null;
|
||||
// T08: Per-key max concurrent sticky sessions (0 = unlimited)
|
||||
maxSessions: number;
|
||||
}
|
||||
|
||||
interface ApiKeyRow extends JsonRecord {
|
||||
@@ -197,6 +199,11 @@ function ensureApiKeysColumns(db: ApiKeysDbLike) {
|
||||
db.exec("ALTER TABLE api_keys ADD COLUMN max_requests_per_minute INTEGER");
|
||||
console.log("[DB] Added api_keys.max_requests_per_minute column");
|
||||
}
|
||||
// T08: max concurrent sticky sessions per key (0 = unlimited)
|
||||
if (!columnNames.has("max_sessions")) {
|
||||
db.exec("ALTER TABLE api_keys ADD COLUMN max_sessions INTEGER NOT NULL DEFAULT 0");
|
||||
console.log("[DB] Added api_keys.max_sessions column");
|
||||
}
|
||||
_schemaChecked = true;
|
||||
} catch (error) {
|
||||
const message = error instanceof Error ? error.message : String(error);
|
||||
@@ -222,7 +229,7 @@ function getPreparedStatements(db: ApiKeysDbLike): ApiKeysStatements {
|
||||
_stmtGetKeyById = db.prepare<ApiKeyRow>("SELECT * FROM api_keys WHERE id = ?");
|
||||
_stmtValidateKey = db.prepare<JsonRecord>("SELECT 1 FROM api_keys WHERE key = ?");
|
||||
_stmtGetKeyMetadata = db.prepare<ApiKeyRow>(
|
||||
"SELECT id, name, machine_id, allowed_models, allowed_connections, no_log, auto_resolve, is_active, access_schedule, max_requests_per_day, max_requests_per_minute FROM api_keys WHERE key = ?"
|
||||
"SELECT id, name, machine_id, allowed_models, allowed_connections, no_log, auto_resolve, is_active, access_schedule, max_requests_per_day, max_requests_per_minute, max_sessions FROM api_keys WHERE key = ?"
|
||||
);
|
||||
_stmtInsertKey = db.prepare(
|
||||
"INSERT INTO api_keys (id, name, key, machine_id, allowed_models, no_log, created_at) VALUES (?, ?, ?, ?, ?, ?, ?)"
|
||||
@@ -418,6 +425,8 @@ export async function updateApiKeyPermissions(
|
||||
accessSchedule?: AccessSchedule | null;
|
||||
maxRequestsPerDay?: number | null;
|
||||
maxRequestsPerMinute?: number | null;
|
||||
// T08: max concurrent sessions for this key (0 = unlimited)
|
||||
maxSessions?: number | null;
|
||||
}
|
||||
) {
|
||||
const db = getDbInstance() as ApiKeysDbLike;
|
||||
@@ -436,6 +445,7 @@ export async function updateApiKeyPermissions(
|
||||
accessSchedule: update.accessSchedule,
|
||||
maxRequestsPerDay: update.maxRequestsPerDay,
|
||||
maxRequestsPerMinute: update.maxRequestsPerMinute,
|
||||
maxSessions: (update as { maxSessions?: number | null }).maxSessions,
|
||||
};
|
||||
|
||||
if (
|
||||
@@ -447,7 +457,8 @@ export async function updateApiKeyPermissions(
|
||||
normalized.isActive === undefined &&
|
||||
normalized.accessSchedule === undefined &&
|
||||
normalized.maxRequestsPerDay === undefined &&
|
||||
normalized.maxRequestsPerMinute === undefined
|
||||
normalized.maxRequestsPerMinute === undefined &&
|
||||
(normalized as Record<string, unknown>).maxSessions === undefined
|
||||
) {
|
||||
return false;
|
||||
}
|
||||
@@ -464,6 +475,7 @@ export async function updateApiKeyPermissions(
|
||||
accessSchedule?: string | null;
|
||||
maxRequestsPerDay?: number | null;
|
||||
maxRequestsPerMinute?: number | null;
|
||||
maxSessions?: number;
|
||||
} = { id };
|
||||
|
||||
if (normalized.name !== undefined) {
|
||||
@@ -514,6 +526,12 @@ export async function updateApiKeyPermissions(
|
||||
params.maxRequestsPerMinute = normalized.maxRequestsPerMinute;
|
||||
}
|
||||
|
||||
const maxSessionsUpdate = (normalized as Record<string, unknown>).maxSessions;
|
||||
if (maxSessionsUpdate !== undefined) {
|
||||
updates.push("max_sessions = @maxSessions");
|
||||
params.maxSessions = typeof maxSessionsUpdate === "number" ? Math.max(0, maxSessionsUpdate) : 0;
|
||||
}
|
||||
|
||||
const result = db.prepare(`UPDATE api_keys SET ${updates.join(", ")} WHERE id = @id`).run(params);
|
||||
|
||||
if (result.changes === 0) return false;
|
||||
@@ -605,6 +623,8 @@ export async function getApiKeyMetadata(
|
||||
const rawMaxRPD = record.max_requests_per_day ?? record.maxRequestsPerDay;
|
||||
const rawMaxRPM = record.max_requests_per_minute ?? record.maxRequestsPerMinute;
|
||||
|
||||
const rawMaxSessions = record.max_sessions ?? record.maxSessions;
|
||||
|
||||
const metadata: ApiKeyMetadata = {
|
||||
id: metadataId,
|
||||
name: metadataName,
|
||||
@@ -619,6 +639,8 @@ export async function getApiKeyMetadata(
|
||||
accessSchedule: parseAccessSchedule(record.access_schedule ?? record.accessSchedule),
|
||||
maxRequestsPerDay: typeof rawMaxRPD === "number" && rawMaxRPD > 0 ? rawMaxRPD : null,
|
||||
maxRequestsPerMinute: typeof rawMaxRPM === "number" && rawMaxRPM > 0 ? rawMaxRPM : null,
|
||||
// T08: max concurrent sessions; 0 = unlimited (default & backward-compatible)
|
||||
maxSessions: typeof rawMaxSessions === "number" && rawMaxSessions > 0 ? rawMaxSessions : 0,
|
||||
};
|
||||
|
||||
if (!metadata.id) {
|
||||
|
||||
9
src/lib/db/migrations/009_requested_model.sql
Normal file
9
src/lib/db/migrations/009_requested_model.sql
Normal file
@@ -0,0 +1,9 @@
|
||||
-- Migration 009: Add requested_model to call_logs for billing transparency
|
||||
-- Tracks the model the client *asked* for vs the model that was *actually routed*.
|
||||
-- Needed when a combo falls back: requested_model ≠ model in call_logs.
|
||||
-- Ref: sub2api commits 0b845c25 + 4edcfe1f (T01 sub2api gap analysis)
|
||||
ALTER TABLE call_logs ADD COLUMN requested_model TEXT DEFAULT NULL;
|
||||
|
||||
-- Index for filtering/aggregating by requested_model in Analytics
|
||||
CREATE INDEX IF NOT EXISTS idx_call_logs_requested_model
|
||||
ON call_logs(requested_model);
|
||||
@@ -513,3 +513,98 @@ export async function deleteProviderNode(id: string) {
|
||||
backupDbFile("pre-write");
|
||||
return rowToCamel(existing);
|
||||
}
|
||||
|
||||
// ──────────────── T05: Rate-Limit DB Persistence ──────────────────────────
|
||||
// Allows rate-limit state to survive token refresh without being accidentally
|
||||
// cleared. DB column rate_limited_until already exists in schema.
|
||||
// Ref: sub2api PR #1218 (fix(openai): prevent rescheduling rate-limited accounts)
|
||||
|
||||
/**
|
||||
* T05: Persist when a connection is rate-limited, directly in DB.
|
||||
* This survives token refresh — OAuth flows must NOT override this field.
|
||||
*
|
||||
* @param connectionId - The provider_connections.id
|
||||
* @param until - Epoch ms when the rate limit expires (null to clear)
|
||||
*/
|
||||
export function setConnectionRateLimitUntil(connectionId: string, until: number | null): void {
|
||||
const db = getDbInstance() as unknown as DbLike;
|
||||
db.prepare(
|
||||
"UPDATE provider_connections SET rate_limited_until = ?, updated_at = ? WHERE id = ?"
|
||||
).run(until, new Date().toISOString(), connectionId);
|
||||
invalidateDbCache("connections");
|
||||
}
|
||||
|
||||
/**
|
||||
* T05: Check if a connection is currently rate-limited (DB-backed).
|
||||
* Use this before account selection to skip transiently rate-limited accounts.
|
||||
*
|
||||
* @returns true if rate_limited_until is set and in the future
|
||||
*/
|
||||
export function isConnectionRateLimited(connectionId: string): boolean {
|
||||
const db = getDbInstance() as unknown as DbLike;
|
||||
const row = db
|
||||
.prepare("SELECT rate_limited_until FROM provider_connections WHERE id = ?")
|
||||
.get(connectionId) as { rate_limited_until?: number | null } | undefined;
|
||||
if (!row?.rate_limited_until) return false;
|
||||
return Date.now() < row.rate_limited_until;
|
||||
}
|
||||
|
||||
/**
|
||||
* T05: Get all connections for a provider that are currently rate-limited.
|
||||
* Returns an array of { id, rateLimitedUntil } for dashboard display.
|
||||
*/
|
||||
export function getRateLimitedConnections(
|
||||
provider: string
|
||||
): Array<{ id: string; rateLimitedUntil: number }> {
|
||||
const db = getDbInstance() as unknown as DbLike;
|
||||
const now = Date.now();
|
||||
const rows = db
|
||||
.prepare(
|
||||
"SELECT id, rate_limited_until FROM provider_connections WHERE provider = ? AND rate_limited_until > ?"
|
||||
)
|
||||
.all(provider, now) as Array<{ id: string; rate_limited_until: number }>;
|
||||
return rows.map((r) => ({ id: r.id, rateLimitedUntil: r.rate_limited_until }));
|
||||
}
|
||||
|
||||
// ──────────────── T13: Stale Quota Display Fix ─────────────────────────────
|
||||
// Codex/Claude quotas display stale cumulative usage after the window resets.
|
||||
// By comparing resetAt timestamp to now(), we can show 0 when window has passed.
|
||||
// Ref: sub2api PR #1171 (fix: quota display shows stale cumulative usage after reset)
|
||||
|
||||
/**
|
||||
* T13: Get effective quota usage, zeroing it out if the window has already reset.
|
||||
*
|
||||
* @param used - Stored usage value (tokens used in the window)
|
||||
* @param resetAt - ISO-8601 string or epoch ms when the window resets, or null
|
||||
* @returns Effective usage: 0 if window expired, original value otherwise
|
||||
*/
|
||||
export function getEffectiveQuotaUsage(
|
||||
used: number,
|
||||
resetAt: string | number | null | undefined
|
||||
): number {
|
||||
if (!resetAt) return used;
|
||||
const resetTime = typeof resetAt === "number" ? resetAt : new Date(resetAt).getTime();
|
||||
if (isNaN(resetTime)) return used;
|
||||
// Window has passed — display should show 0 (pending next snapshot)
|
||||
if (Date.now() >= resetTime) return 0;
|
||||
return used;
|
||||
}
|
||||
|
||||
/**
|
||||
* T13: Format a reset countdown as a human-readable string: "2h 35m" or "4m 30s".
|
||||
* Returns null if resetAt is in the past or not set.
|
||||
*/
|
||||
export function formatResetCountdown(resetAt: string | number | null | undefined): string | null {
|
||||
if (!resetAt) return null;
|
||||
const resetTime = typeof resetAt === "number" ? resetAt : new Date(resetAt).getTime();
|
||||
if (isNaN(resetTime)) return null;
|
||||
const diffMs = resetTime - Date.now();
|
||||
if (diffMs <= 0) return null;
|
||||
const totalSeconds = Math.floor(diffMs / 1000);
|
||||
const hours = Math.floor(totalSeconds / 3600);
|
||||
const minutes = Math.floor((totalSeconds % 3600) / 60);
|
||||
const seconds = totalSeconds % 60;
|
||||
if (hours > 0) return `${hours}h ${minutes}m`;
|
||||
if (minutes > 0) return `${minutes}m ${seconds}s`;
|
||||
return `${seconds}s`;
|
||||
}
|
||||
|
||||
56
src/lib/ipUtils.ts
Normal file
56
src/lib/ipUtils.ts
Normal file
@@ -0,0 +1,56 @@
|
||||
import { isIP } from "node:net";
|
||||
|
||||
/**
|
||||
* T07: Extract the real client IP from X-Forwarded-For header.
|
||||
* Skips invalid entries like "unknown" or empty strings.
|
||||
* Falls back to remoteAddress if no valid IP found.
|
||||
* Ref: sub2api PR #1135
|
||||
*
|
||||
* @param xForwardedFor - Value of the X-Forwarded-For header (may be CSV)
|
||||
* @param remoteAddress - Fallback from the raw socket (req.socket.remoteAddress)
|
||||
* @returns The first valid IP address found, or "unknown"
|
||||
*/
|
||||
export function extractClientIp(
|
||||
xForwardedFor: string | null | undefined,
|
||||
remoteAddress: string | undefined
|
||||
): string {
|
||||
if (xForwardedFor) {
|
||||
const entries = xForwardedFor.split(",");
|
||||
for (const entry of entries) {
|
||||
const trimmed = entry.trim();
|
||||
if (trimmed && isIP(trimmed) !== 0) {
|
||||
return trimmed; // First valid IP wins
|
||||
}
|
||||
}
|
||||
}
|
||||
return remoteAddress?.trim() ?? "unknown";
|
||||
}
|
||||
|
||||
/**
|
||||
* Extract client IP from a Request or NextRequest object.
|
||||
* Checks X-Forwarded-For, X-Real-IP, CF-Connecting-IP, then socket.
|
||||
*/
|
||||
export function getClientIpFromRequest(req: {
|
||||
headers?: Headers | { get?: (n: string) => string | null };
|
||||
socket?: { remoteAddress?: string };
|
||||
ip?: string;
|
||||
}): string {
|
||||
// Helper to get header value from either Headers object or plain object
|
||||
const getHeader = (name: string): string | null => {
|
||||
if (!req.headers) return null;
|
||||
if (typeof (req.headers as Headers).get === "function") {
|
||||
return (req.headers as Headers).get(name);
|
||||
}
|
||||
return null;
|
||||
};
|
||||
|
||||
// Priority: CF-Connecting-IP (Cloudflare) > X-Forwarded-For > X-Real-IP > socket
|
||||
const cfIp = getHeader("cf-connecting-ip");
|
||||
if (cfIp && isIP(cfIp.trim()) !== 0) return cfIp.trim();
|
||||
|
||||
const xff = getHeader("x-forwarded-for");
|
||||
const realIp = getHeader("x-real-ip");
|
||||
const remoteAddress = req.ip ?? req.socket?.remoteAddress;
|
||||
|
||||
return extractClientIp(xff ?? realIp, remoteAddress);
|
||||
}
|
||||
@@ -23,6 +23,15 @@ export {
|
||||
createProviderNode,
|
||||
updateProviderNode,
|
||||
deleteProviderNode,
|
||||
|
||||
// T05: Rate-limit DB persistence (survives token refresh)
|
||||
setConnectionRateLimitUntil,
|
||||
isConnectionRateLimited,
|
||||
getRateLimitedConnections,
|
||||
|
||||
// T13: Stale quota display fix (zero out usage after window resets)
|
||||
getEffectiveQuotaUsage,
|
||||
formatResetCountdown,
|
||||
} from "./db/providers";
|
||||
|
||||
export {
|
||||
|
||||
140
src/lib/proxyHealth.ts
Normal file
140
src/lib/proxyHealth.ts
Normal file
@@ -0,0 +1,140 @@
|
||||
/**
|
||||
* T14: Proxy Fast-Fail — TCP health check with in-memory cache.
|
||||
*
|
||||
* When a configured HTTP/SOCKS5 proxy is unreachable, every request
|
||||
* through OmniRoute used to wait for the full PROXY_TIMEOUT_MS (30s)
|
||||
* before failing. This module detects dead proxies in <2s via a quick
|
||||
* TCP connection check, caching the result to avoid overhead per request.
|
||||
*
|
||||
* Ref: sub2api PR #1167 (fix: proxy-fast-fail)
|
||||
*/
|
||||
|
||||
import { createConnection } from "node:net";
|
||||
|
||||
// Configurable via env vars
|
||||
const FAST_FAIL_TIMEOUT_MS = parseInt(process.env.PROXY_FAST_FAIL_TIMEOUT_MS ?? "2000", 10);
|
||||
const HEALTH_CACHE_TTL_MS = parseInt(process.env.PROXY_HEALTH_CACHE_TTL_MS ?? "30000", 10);
|
||||
|
||||
interface ProxyHealthEntry {
|
||||
healthy: boolean;
|
||||
checkedAt: number;
|
||||
ttlMs: number;
|
||||
}
|
||||
|
||||
// In-memory cache: proxyUrl → health entry
|
||||
const proxyHealthCache = new Map<string, ProxyHealthEntry>();
|
||||
|
||||
/**
|
||||
* T14: Perform a fast TCP check to see if a proxy host:port is reachable.
|
||||
* Results are cached for `cacheTtlMs` (default 30s) to avoid checking every request.
|
||||
*
|
||||
* @param proxyUrl - Full proxy URL, e.g. http://user:pass@1.2.3.4:8080
|
||||
* @param timeoutMs - TCP connection timeout (default 2000ms)
|
||||
* @param cacheTtlMs - How long to cache the health result (default 30000ms)
|
||||
* @returns true if proxy TCP port is open, false otherwise
|
||||
*/
|
||||
export async function isProxyReachable(
|
||||
proxyUrl: string,
|
||||
timeoutMs = FAST_FAIL_TIMEOUT_MS,
|
||||
cacheTtlMs = HEALTH_CACHE_TTL_MS
|
||||
): Promise<boolean> {
|
||||
const cached = proxyHealthCache.get(proxyUrl);
|
||||
if (cached && Date.now() - cached.checkedAt < cached.ttlMs) {
|
||||
return cached.healthy;
|
||||
}
|
||||
|
||||
let url: URL;
|
||||
try {
|
||||
url = new URL(proxyUrl);
|
||||
} catch {
|
||||
// Malformed URL — treat as unreachable
|
||||
proxyHealthCache.set(proxyUrl, {
|
||||
healthy: false,
|
||||
checkedAt: Date.now(),
|
||||
ttlMs: cacheTtlMs,
|
||||
});
|
||||
return false;
|
||||
}
|
||||
|
||||
const host = url.hostname;
|
||||
const port = parseInt(url.port || defaultPortForScheme(url.protocol), 10);
|
||||
|
||||
if (!host || isNaN(port)) {
|
||||
proxyHealthCache.set(proxyUrl, {
|
||||
healthy: false,
|
||||
checkedAt: Date.now(),
|
||||
ttlMs: cacheTtlMs,
|
||||
});
|
||||
return false;
|
||||
}
|
||||
|
||||
const healthy = await tcpCheck(host, port, timeoutMs);
|
||||
proxyHealthCache.set(proxyUrl, { healthy, checkedAt: Date.now(), ttlMs: cacheTtlMs });
|
||||
return healthy;
|
||||
}
|
||||
|
||||
/**
|
||||
* Get the cached health status of a proxy without re-checking.
|
||||
* Returns null if there is no cached entry.
|
||||
*/
|
||||
export function getCachedProxyHealth(proxyUrl: string): boolean | null {
|
||||
const cached = proxyHealthCache.get(proxyUrl);
|
||||
if (!cached) return null;
|
||||
if (Date.now() - cached.checkedAt >= cached.ttlMs) return null; // stale
|
||||
return cached.healthy;
|
||||
}
|
||||
|
||||
/**
|
||||
* Invalidate the cached health for a proxy URL (force re-check on next call).
|
||||
*/
|
||||
export function invalidateProxyHealth(proxyUrl: string): void {
|
||||
proxyHealthCache.delete(proxyUrl);
|
||||
}
|
||||
|
||||
/**
|
||||
* Get all currently cached proxy health entries (for dashboard display).
|
||||
*/
|
||||
export function getAllProxyHealthStatuses(): Array<{
|
||||
proxyUrl: string;
|
||||
healthy: boolean;
|
||||
checkedAt: number;
|
||||
stale: boolean;
|
||||
}> {
|
||||
const now = Date.now();
|
||||
return [...proxyHealthCache.entries()].map(([proxyUrl, entry]) => ({
|
||||
proxyUrl,
|
||||
healthy: entry.healthy,
|
||||
checkedAt: entry.checkedAt,
|
||||
stale: now - entry.checkedAt >= entry.ttlMs,
|
||||
}));
|
||||
}
|
||||
|
||||
// ─── Internals ────────────────────────────────────────────────────────────────
|
||||
|
||||
function defaultPortForScheme(protocol: string): string {
|
||||
switch (protocol.replace(":", "").toLowerCase()) {
|
||||
case "https":
|
||||
return "443";
|
||||
case "socks5":
|
||||
case "socks5h":
|
||||
return "1080";
|
||||
case "http":
|
||||
default:
|
||||
return "8080";
|
||||
}
|
||||
}
|
||||
|
||||
function tcpCheck(host: string, port: number, timeoutMs: number): Promise<boolean> {
|
||||
return new Promise<boolean>((resolve) => {
|
||||
const socket = createConnection({ host, port }, () => {
|
||||
socket.destroy();
|
||||
resolve(true);
|
||||
});
|
||||
socket.setTimeout(timeoutMs);
|
||||
socket.on("error", () => resolve(false));
|
||||
socket.on("timeout", () => {
|
||||
socket.destroy();
|
||||
resolve(false);
|
||||
});
|
||||
});
|
||||
}
|
||||
@@ -180,6 +180,7 @@ export async function saveCallLog(entry: any) {
|
||||
path: entry.path || "/v1/chat/completions",
|
||||
status: entry.status || 0,
|
||||
model: entry.model || "-",
|
||||
requestedModel: entry.requestedModel || null, // T01: model the client asked for
|
||||
provider: entry.provider || "-",
|
||||
account,
|
||||
connectionId: entry.connectionId || null,
|
||||
@@ -205,10 +206,10 @@ export async function saveCallLog(entry: any) {
|
||||
const db = getDbInstance();
|
||||
db.prepare(
|
||||
`
|
||||
INSERT INTO call_logs (id, timestamp, method, path, status, model, provider,
|
||||
INSERT INTO call_logs (id, timestamp, method, path, status, model, requested_model, provider,
|
||||
account, connection_id, duration, tokens_in, tokens_out, request_type, source_format, target_format,
|
||||
api_key_id, api_key_name, combo_name, request_body, response_body, error)
|
||||
VALUES (@id, @timestamp, @method, @path, @status, @model, @provider,
|
||||
VALUES (@id, @timestamp, @method, @path, @status, @model, @requestedModel, @provider,
|
||||
@account, @connectionId, @duration, @tokensIn, @tokensOut, @requestType, @sourceFormat, @targetFormat,
|
||||
@apiKeyId, @apiKeyName, @comboName, @requestBody, @responseBody, @error)
|
||||
`
|
||||
@@ -374,6 +375,7 @@ export async function getCallLogs(filter: any = {}) {
|
||||
path: toStringOrNull(l.path),
|
||||
status: toNumber(l.status),
|
||||
model: toStringOrNull(l.model),
|
||||
requestedModel: toStringOrNull(l.requested_model), // T01: original model from client
|
||||
provider: toStringOrNull(l.provider),
|
||||
account: toStringOrNull(l.account),
|
||||
duration: toNumber(l.duration),
|
||||
|
||||
@@ -147,6 +147,21 @@ export default function RequestLoggerDetail({ log, detail, loading, onClose, onC
|
||||
<div className="text-[10px] text-text-muted uppercase tracking-wider mb-1">Model</div>
|
||||
<div className="text-sm font-medium text-primary font-mono">{log.model}</div>
|
||||
</div>
|
||||
<div>
|
||||
<div className="text-[10px] text-text-muted uppercase tracking-wider mb-1">
|
||||
Requested Model
|
||||
</div>
|
||||
<div
|
||||
className={`text-sm font-medium font-mono ${
|
||||
(detail?.requestedModel || log.requestedModel) &&
|
||||
(detail?.requestedModel || log.requestedModel) !== log.model
|
||||
? "text-amber-600 dark:text-amber-400"
|
||||
: "text-text-muted"
|
||||
}`}
|
||||
>
|
||||
{detail?.requestedModel || log.requestedModel || "—"}
|
||||
</div>
|
||||
</div>
|
||||
<div>
|
||||
<div className="text-[10px] text-text-muted uppercase tracking-wider mb-1">
|
||||
Provider
|
||||
|
||||
@@ -802,6 +802,30 @@ export const DEFAULT_PRICING = {
|
||||
reasoning: 1.8,
|
||||
cache_creation: 0.3,
|
||||
},
|
||||
// T12: MiniMax M2.7 — new default model (sub2api PR #1120)
|
||||
// Upgraded from M2.5, same API endpoint api.minimax.io
|
||||
// Pricing estimated, check https://platform.minimaxi.com/document/Price
|
||||
"minimax-m2.7": {
|
||||
input: 0.4,
|
||||
output: 1.6,
|
||||
cached: 0.2,
|
||||
reasoning: 2.4,
|
||||
cache_creation: 0.4,
|
||||
},
|
||||
"MiniMax-M2.7": {
|
||||
input: 0.4,
|
||||
output: 1.6,
|
||||
cached: 0.2,
|
||||
reasoning: 2.4,
|
||||
cache_creation: 0.4,
|
||||
},
|
||||
"minimax-m2.7-highspeed": {
|
||||
input: 0.4,
|
||||
output: 1.6,
|
||||
cached: 0.2,
|
||||
reasoning: 2.4,
|
||||
cache_creation: 0.4,
|
||||
},
|
||||
},
|
||||
|
||||
// ─── Free-tier API Key Providers (nominal $0 pricing) ───
|
||||
|
||||
@@ -546,6 +546,20 @@ export const APIKEY_PROVIDERS = {
|
||||
freeNote:
|
||||
"No API key needed — access GPT-5, Claude, Gemini, DeepSeek V3, Llama 4 free (1 req/15s)",
|
||||
},
|
||||
puter: {
|
||||
id: "puter",
|
||||
alias: "pu",
|
||||
name: "Puter AI",
|
||||
icon: "cloud_circle",
|
||||
color: "#6366F1",
|
||||
textIcon: "PU",
|
||||
website: "https://puter.com",
|
||||
hasFree: true,
|
||||
freeNote:
|
||||
"500+ models (GPT-5, Claude Opus 4, Gemini 3 Pro, Grok 4, DeepSeek V3...) — Users pay via free Puter account",
|
||||
passthroughModels: true,
|
||||
authHint: "Get token at puter.com/dashboard → Copy Auth Token",
|
||||
},
|
||||
"cloudflare-ai": {
|
||||
id: "cloudflare-ai",
|
||||
alias: "cf",
|
||||
|
||||
90
tests/unit/auth-terminal-status.test.mjs
Normal file
90
tests/unit/auth-terminal-status.test.mjs
Normal file
@@ -0,0 +1,90 @@
|
||||
import test from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import fs from "node:fs";
|
||||
import os from "node:os";
|
||||
import path from "node:path";
|
||||
|
||||
const TEST_DATA_DIR = fs.mkdtempSync(path.join(os.tmpdir(), "omniroute-auth-terminal-"));
|
||||
process.env.DATA_DIR = TEST_DATA_DIR;
|
||||
|
||||
const core = await import("../../src/lib/db/core.ts");
|
||||
const providersDb = await import("../../src/lib/db/providers.ts");
|
||||
const auth = await import("../../src/sse/services/auth.ts");
|
||||
|
||||
async function resetStorage() {
|
||||
core.resetDbInstance();
|
||||
fs.rmSync(TEST_DATA_DIR, { recursive: true, force: true });
|
||||
fs.mkdirSync(TEST_DATA_DIR, { recursive: true });
|
||||
}
|
||||
|
||||
test.after(() => {
|
||||
core.resetDbInstance();
|
||||
fs.rmSync(TEST_DATA_DIR, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
test("getProviderCredentials skips credits_exhausted connections", async () => {
|
||||
await resetStorage();
|
||||
|
||||
const exhausted = await providersDb.createProviderConnection({
|
||||
provider: "openai",
|
||||
authType: "apikey",
|
||||
apiKey: "sk-exhausted",
|
||||
isActive: true,
|
||||
testStatus: "credits_exhausted",
|
||||
});
|
||||
|
||||
const healthy = await providersDb.createProviderConnection({
|
||||
provider: "openai",
|
||||
authType: "apikey",
|
||||
apiKey: "sk-healthy",
|
||||
isActive: true,
|
||||
testStatus: "active",
|
||||
});
|
||||
|
||||
const selected = await auth.getProviderCredentials("openai");
|
||||
assert.ok(selected);
|
||||
assert.equal(selected.connectionId, healthy.id);
|
||||
assert.notEqual(selected.connectionId, exhausted.id);
|
||||
});
|
||||
|
||||
test("getProviderCredentials returns null when all active connections are terminal", async () => {
|
||||
await resetStorage();
|
||||
|
||||
await providersDb.createProviderConnection({
|
||||
provider: "openai",
|
||||
authType: "apikey",
|
||||
apiKey: "sk-only-exhausted",
|
||||
isActive: true,
|
||||
testStatus: "credits_exhausted",
|
||||
});
|
||||
|
||||
const selected = await auth.getProviderCredentials("openai");
|
||||
assert.equal(selected, null);
|
||||
});
|
||||
|
||||
test("markAccountUnavailable does not overwrite terminal status", async () => {
|
||||
await resetStorage();
|
||||
|
||||
const conn = await providersDb.createProviderConnection({
|
||||
provider: "openai",
|
||||
authType: "apikey",
|
||||
apiKey: "sk-terminal",
|
||||
isActive: true,
|
||||
testStatus: "credits_exhausted",
|
||||
lastError: "insufficient_quota",
|
||||
});
|
||||
|
||||
const result = await auth.markAccountUnavailable(
|
||||
conn.id,
|
||||
503,
|
||||
"temporary upstream error",
|
||||
"openai",
|
||||
"gpt-4.1"
|
||||
);
|
||||
|
||||
assert.equal(result.shouldFallback, true);
|
||||
assert.equal(result.cooldownMs, 0);
|
||||
|
||||
const after = await providersDb.getProviderConnectionById(conn.id);
|
||||
assert.equal(after.testStatus, "credits_exhausted");
|
||||
});
|
||||
35
tests/unit/error-classifier.test.mjs
Normal file
35
tests/unit/error-classifier.test.mjs
Normal file
@@ -0,0 +1,35 @@
|
||||
import test from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
|
||||
const { classifyProviderError, PROVIDER_ERROR_TYPES } =
|
||||
await import("../../open-sse/services/errorClassifier.ts");
|
||||
|
||||
test("classifyProviderError: 401 + account_deactivated => ACCOUNT_DEACTIVATED", () => {
|
||||
const body = JSON.stringify({
|
||||
error: { message: "account_deactivated: this account has been disabled" },
|
||||
});
|
||||
const result = classifyProviderError(401, body);
|
||||
assert.equal(result, PROVIDER_ERROR_TYPES.ACCOUNT_DEACTIVATED);
|
||||
});
|
||||
|
||||
test("classifyProviderError: plain 401 => UNAUTHORIZED", () => {
|
||||
const result = classifyProviderError(401, { error: { message: "token expired" } });
|
||||
assert.equal(result, PROVIDER_ERROR_TYPES.UNAUTHORIZED);
|
||||
});
|
||||
|
||||
test("classifyProviderError: 402 => QUOTA_EXHAUSTED", () => {
|
||||
const result = classifyProviderError(402, { error: { message: "payment required" } });
|
||||
assert.equal(result, PROVIDER_ERROR_TYPES.QUOTA_EXHAUSTED);
|
||||
});
|
||||
|
||||
test("classifyProviderError: 400 + billing signal => QUOTA_EXHAUSTED", () => {
|
||||
const result = classifyProviderError(400, {
|
||||
error: { message: "insufficient_quota: exceeded your current quota" },
|
||||
});
|
||||
assert.equal(result, PROVIDER_ERROR_TYPES.QUOTA_EXHAUSTED);
|
||||
});
|
||||
|
||||
test("classifyProviderError: 429 without billing signal => RATE_LIMITED", () => {
|
||||
const result = classifyProviderError(429, { error: { message: "too many requests" } });
|
||||
assert.equal(result, PROVIDER_ERROR_TYPES.RATE_LIMITED);
|
||||
});
|
||||
@@ -11,6 +11,7 @@ const {
|
||||
addToWhitelist,
|
||||
removeFromWhitelist,
|
||||
getIPFilterConfig,
|
||||
checkRequestIP,
|
||||
resetIPFilter,
|
||||
} = await import("../../open-sse/services/ipFilter.ts");
|
||||
|
||||
@@ -111,6 +112,48 @@ test("normalizes ::ffff: prefix", () => {
|
||||
assert.equal(checkIP("::ffff:1.2.3.4").allowed, false);
|
||||
});
|
||||
|
||||
// ─── T07: X-Forwarded-For validation ───────────────────────────────────────
|
||||
|
||||
test("checkRequestIP: skips invalid XFF entries and uses next valid IP", () => {
|
||||
configureIPFilter({ enabled: true, mode: "whitelist", whitelist: ["1.2.3.4"] });
|
||||
const req = {
|
||||
headers: {
|
||||
get(name) {
|
||||
if (name === "x-forwarded-for") return "unknown, 1.2.3.4";
|
||||
return null;
|
||||
},
|
||||
},
|
||||
};
|
||||
assert.equal(checkRequestIP(req).allowed, true);
|
||||
});
|
||||
|
||||
test("checkRequestIP: all-invalid XFF falls back to x-real-ip", () => {
|
||||
configureIPFilter({ enabled: true, mode: "whitelist", whitelist: ["9.9.9.9"] });
|
||||
const req = {
|
||||
headers: {
|
||||
get(name) {
|
||||
if (name === "x-forwarded-for") return "unknown, -, not_an_ip";
|
||||
if (name === "x-real-ip") return "9.9.9.9";
|
||||
return null;
|
||||
},
|
||||
},
|
||||
};
|
||||
assert.equal(checkRequestIP(req).allowed, true);
|
||||
});
|
||||
|
||||
test("checkRequestIP: empty headers fall back to request.ip", () => {
|
||||
configureIPFilter({ enabled: true, mode: "whitelist", whitelist: ["7.7.7.7"] });
|
||||
const req = {
|
||||
headers: {
|
||||
get() {
|
||||
return null;
|
||||
},
|
||||
},
|
||||
ip: "7.7.7.7",
|
||||
};
|
||||
assert.equal(checkRequestIP(req).allowed, true);
|
||||
});
|
||||
|
||||
// ─── Config API ─────────────────────────────────────────────────────────────
|
||||
|
||||
test("getIPFilterConfig: returns serializable config", () => {
|
||||
|
||||
@@ -13,7 +13,7 @@ test("handleImageGeneration(nanobanana): async submit+poll returns URL payload",
|
||||
if (u.includes("/generate-pro")) {
|
||||
const body = JSON.parse(options.body);
|
||||
assert.equal(body.prompt, "galaxy test");
|
||||
assert.equal(body.aspectRatio, "4:5");
|
||||
assert.equal(body.aspectRatio, "1:1");
|
||||
assert.equal(body.resolution, "2K");
|
||||
|
||||
return new Response(
|
||||
|
||||
76
tests/unit/openai-to-claude-strip-empty.test.mjs
Normal file
76
tests/unit/openai-to-claude-strip-empty.test.mjs
Normal file
@@ -0,0 +1,76 @@
|
||||
import test from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
|
||||
const { stripEmptyTextBlocks, openaiToClaudeRequest } =
|
||||
await import("../../open-sse/translator/request/openai-to-claude.ts");
|
||||
|
||||
test("stripEmptyTextBlocks removes empty text recursively inside tool_result content", () => {
|
||||
const input = [
|
||||
{ type: "text", text: "" },
|
||||
{ type: "text", text: "keep-top-level" },
|
||||
{
|
||||
type: "tool_result",
|
||||
content: [
|
||||
{ type: "text", text: "" },
|
||||
{ type: "text", text: "keep-nested" },
|
||||
{
|
||||
type: "tool_result",
|
||||
content: [
|
||||
{ type: "text", text: "" },
|
||||
{ type: "text", text: "keep-deep" },
|
||||
],
|
||||
},
|
||||
],
|
||||
},
|
||||
];
|
||||
|
||||
const out = stripEmptyTextBlocks(input);
|
||||
assert.deepEqual(out, [
|
||||
{ type: "text", text: "keep-top-level" },
|
||||
{
|
||||
type: "tool_result",
|
||||
content: [
|
||||
{ type: "text", text: "keep-nested" },
|
||||
{
|
||||
type: "tool_result",
|
||||
content: [{ type: "text", text: "keep-deep" }],
|
||||
},
|
||||
],
|
||||
},
|
||||
]);
|
||||
});
|
||||
|
||||
test("openaiToClaudeRequest applies strip to tool message array content", () => {
|
||||
const request = {
|
||||
messages: [
|
||||
{ role: "user", content: "run tool" },
|
||||
{
|
||||
role: "assistant",
|
||||
content: "",
|
||||
tool_calls: [
|
||||
{
|
||||
id: "call_1",
|
||||
type: "function",
|
||||
function: { name: "demo_tool", arguments: "{}" },
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
role: "tool",
|
||||
tool_call_id: "call_1",
|
||||
content: [
|
||||
{ type: "text", text: "" },
|
||||
{ type: "text", text: "tool ok" },
|
||||
],
|
||||
},
|
||||
],
|
||||
};
|
||||
|
||||
const translated = openaiToClaudeRequest("claude-sonnet-4", request, false);
|
||||
const toolMessage = translated.messages.find(
|
||||
(m) => Array.isArray(m.content) && m.content.some((b) => b.type === "tool_result")
|
||||
);
|
||||
assert.ok(toolMessage, "expected a translated tool_result user message");
|
||||
const toolResult = toolMessage.content.find((b) => b.type === "tool_result");
|
||||
assert.deepEqual(toolResult.content, [{ type: "text", text: "tool ok" }]);
|
||||
});
|
||||
@@ -11,7 +11,10 @@ import { DefaultExecutor } from "../../open-sse/executors/default.ts";
|
||||
import { CodexExecutor, setDefaultFastServiceTierEnabled } from "../../open-sse/executors/codex.ts";
|
||||
import { translateNonStreamingResponse } from "../../open-sse/handlers/responseTranslator.ts";
|
||||
import { extractUsageFromResponse } from "../../open-sse/handlers/usageExtractor.ts";
|
||||
import { parseSSEToResponsesOutput } from "../../open-sse/handlers/sseParser.ts";
|
||||
import {
|
||||
parseSSEToOpenAIResponse,
|
||||
parseSSEToResponsesOutput,
|
||||
} from "../../open-sse/handlers/sseParser.ts";
|
||||
|
||||
test("getModelInfoCore resolves unique non-openai unprefixed model", async () => {
|
||||
const info = await getModelInfoCore("claude-haiku-4-5-20251001", {});
|
||||
@@ -382,3 +385,56 @@ test("parseSSEToResponsesOutput returns null for invalid payload", () => {
|
||||
const parsed = parseSSEToResponsesOutput("data: not-json\n\ndata: [DONE]\n", "fallback-model");
|
||||
assert.equal(parsed, null);
|
||||
});
|
||||
|
||||
test("parseSSEToOpenAIResponse merges split tool call chunks by id without duplication", () => {
|
||||
const rawSSE = [
|
||||
`data: ${JSON.stringify({
|
||||
id: "chatcmpl_1",
|
||||
object: "chat.completion.chunk",
|
||||
choices: [
|
||||
{
|
||||
index: 0,
|
||||
delta: {
|
||||
tool_calls: [
|
||||
{
|
||||
id: "call_abc",
|
||||
index: 0,
|
||||
type: "function",
|
||||
function: { name: "sum", arguments: '{"a":' },
|
||||
},
|
||||
],
|
||||
},
|
||||
},
|
||||
],
|
||||
})}`,
|
||||
`data: ${JSON.stringify({
|
||||
id: "chatcmpl_1",
|
||||
object: "chat.completion.chunk",
|
||||
choices: [
|
||||
{
|
||||
index: 0,
|
||||
delta: {
|
||||
tool_calls: [
|
||||
{
|
||||
id: "call_abc",
|
||||
index: 0,
|
||||
type: "function",
|
||||
function: { arguments: "1}" },
|
||||
},
|
||||
],
|
||||
},
|
||||
finish_reason: "tool_calls",
|
||||
},
|
||||
],
|
||||
})}`,
|
||||
"data: [DONE]",
|
||||
].join("\n");
|
||||
|
||||
const parsed = parseSSEToOpenAIResponse(rawSSE, "gpt-5.1-codex");
|
||||
assert.ok(parsed);
|
||||
assert.equal(parsed.choices[0].finish_reason, "tool_calls");
|
||||
assert.equal(parsed.choices[0].message.tool_calls.length, 1);
|
||||
assert.equal(parsed.choices[0].message.tool_calls[0].id, "call_abc");
|
||||
assert.equal(parsed.choices[0].message.tool_calls[0].function.name, "sum");
|
||||
assert.equal(parsed.choices[0].message.tool_calls[0].function.arguments, '{"a":1}');
|
||||
});
|
||||
|
||||
@@ -8,6 +8,11 @@ const {
|
||||
getSessionConnection,
|
||||
getActiveSessionCount,
|
||||
getActiveSessions,
|
||||
extractExternalSessionId,
|
||||
checkSessionLimit,
|
||||
registerKeySession,
|
||||
unregisterKeySession,
|
||||
isSessionRegisteredForKey,
|
||||
clearSessions,
|
||||
} = await import("../../open-sse/services/sessionManager.ts");
|
||||
|
||||
@@ -39,11 +44,17 @@ test("generateSessionId: different model = different ID", () => {
|
||||
test("generateSessionId: different system prompt = different ID", () => {
|
||||
const body1 = {
|
||||
model: "claude-sonnet-4-20250514",
|
||||
messages: [{ role: "system", content: "A" }, { role: "user", content: "hi" }],
|
||||
messages: [
|
||||
{ role: "system", content: "A" },
|
||||
{ role: "user", content: "hi" },
|
||||
],
|
||||
};
|
||||
const body2 = {
|
||||
model: "claude-sonnet-4-20250514",
|
||||
messages: [{ role: "system", content: "B" }, { role: "user", content: "hi" }],
|
||||
messages: [
|
||||
{ role: "system", content: "B" },
|
||||
{ role: "user", content: "hi" },
|
||||
],
|
||||
};
|
||||
assert.notEqual(generateSessionId(body1), generateSessionId(body2));
|
||||
});
|
||||
@@ -147,3 +158,25 @@ test("touchSession with null sessionId: no-op", () => {
|
||||
touchSession(null);
|
||||
assert.equal(getActiveSessionCount(), 0);
|
||||
});
|
||||
|
||||
test("extractExternalSessionId accepts hyphen and underscore variants", () => {
|
||||
const h1 = new Headers({ "x-session-id": "abc-123" });
|
||||
const h2 = new Headers({ x_session_id: "def-456" });
|
||||
|
||||
assert.equal(extractExternalSessionId(h1), "ext:abc-123");
|
||||
assert.equal(extractExternalSessionId(h2), "ext:def-456");
|
||||
});
|
||||
|
||||
test("checkSessionLimit enforces max_sessions for new sessions only", () => {
|
||||
const keyId = "key-1";
|
||||
registerKeySession(keyId, "ext:sess-a");
|
||||
assert.equal(isSessionRegisteredForKey(keyId, "ext:sess-a"), true);
|
||||
|
||||
const violation = checkSessionLimit(keyId, 1);
|
||||
assert.ok(violation);
|
||||
assert.equal(violation.code, "SESSION_LIMIT_EXCEEDED");
|
||||
|
||||
unregisterKeySession(keyId, "ext:sess-a");
|
||||
assert.equal(isSessionRegisteredForKey(keyId, "ext:sess-a"), false);
|
||||
assert.equal(checkSessionLimit(keyId, 1), null);
|
||||
});
|
||||
|
||||
@@ -101,7 +101,8 @@ test("CUSTOM: sets OpenAI reasoning_effort from budget", () => {
|
||||
reasoning_effort: "low",
|
||||
};
|
||||
const result = applyThinkingBudget(body);
|
||||
assert.equal(result.reasoning_effort, "high");
|
||||
// T11 (sub2api gap): full budget (131072) now maps to "max" instead of "high"
|
||||
assert.equal(result.reasoning_effort, "max");
|
||||
setThinkingBudgetConfig(DEFAULT_THINKING_CONFIG);
|
||||
});
|
||||
|
||||
@@ -212,8 +213,8 @@ test("normalizeThinkingLevel: converts Gemini thinkingConfig.thinkingLevel", ()
|
||||
},
|
||||
};
|
||||
const result = normalizeThinkingLevel(body);
|
||||
assert.equal(result.generationConfig.thinking_config.thinking_budget, 131072);
|
||||
assert.equal(result.generationConfig.thinkingConfig, undefined);
|
||||
assert.equal(result.generationConfig.thinkingConfig.thinkingBudget, 131072);
|
||||
assert.equal(result.generationConfig.thinking_config, undefined);
|
||||
});
|
||||
|
||||
test("normalizeThinkingLevel: ignores unknown string values", () => {
|
||||
|
||||
Reference in New Issue
Block a user