mirror of
https://github.com/diegosouzapw/OmniRoute.git
synced 2026-07-30 20:05:40 +03:00
Compare commits
62 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
86030a0fab | ||
|
|
954c40067e | ||
|
|
6a36606bd5 | ||
|
|
20a72a0f45 | ||
|
|
bcc374eb31 | ||
|
|
f0e1f18c79 | ||
|
|
61b7203062 | ||
|
|
a7e95d00cf | ||
|
|
83c358deb1 | ||
|
|
7dd214f3db | ||
|
|
6698d33f04 | ||
|
|
6dcd5b77aa | ||
|
|
80f2c797c9 | ||
|
|
910471a26f | ||
|
|
ccab588948 | ||
|
|
50683e6600 | ||
|
|
75daf98112 | ||
|
|
9a681a27ad | ||
|
|
25c63c8b10 | ||
|
|
a069df41b8 | ||
|
|
b1183c2c9d | ||
|
|
b777b15ee8 | ||
|
|
35061dfc53 | ||
|
|
a65bca6e49 | ||
|
|
72203f2721 | ||
|
|
03ff03ed52 | ||
|
|
488d50b97e | ||
|
|
88cbd1bd83 | ||
|
|
27fe556bab | ||
|
|
3191b7a991 | ||
|
|
f8d045c275 | ||
|
|
0038fe5ff1 | ||
|
|
3ae810a18e | ||
|
|
afe72c8029 | ||
|
|
2341bba973 | ||
|
|
c0da968af2 | ||
|
|
df206d9792 | ||
|
|
49ad44dcaf | ||
|
|
7f785b8fa5 | ||
|
|
b4e674aeb0 | ||
|
|
8ed091703f | ||
|
|
464fd6d4d3 | ||
|
|
5b140d26c3 | ||
|
|
125fb81fa3 | ||
|
|
5c27e0f9ef | ||
|
|
7607cec7a2 | ||
|
|
9e4132fd3f | ||
|
|
668b235b07 | ||
|
|
ea6d0f573d | ||
|
|
d1b8afd3b8 | ||
|
|
0f8b9ca55b | ||
|
|
b5c84a91fb | ||
|
|
cc902db4ab | ||
|
|
faae82eae1 | ||
|
|
49ac0cadfb | ||
|
|
bd5f39e1c6 | ||
|
|
325d048340 | ||
|
|
f1805c8536 | ||
|
|
305545623a | ||
|
|
aac99f6ee2 | ||
|
|
4feea2e721 | ||
|
|
0e73b3b8e1 |
49
.agents/workflows/capture-release-evidences.md
Normal file
49
.agents/workflows/capture-release-evidences.md
Normal file
@@ -0,0 +1,49 @@
|
||||
---
|
||||
description: Automatically run the browser_subagent to visually validate all new UI features from the current release and capture evidence WebP recordings of the changes.
|
||||
---
|
||||
|
||||
# Capture Release Evidences Workflow
|
||||
|
||||
Use this workflow to automatically drive the `browser_subagent` to explore the newly deployed or locally running application and record evidence of the UI changes introduced in the latest release.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- OmniRoute must be actively running and accessible (e.g. locally at `http://localhost:20128` or on the Local VPS at `http://192.168.0.15:20128`).
|
||||
- The user must provide the target URL to be tested, or default to `http://192.168.0.15:20128`.
|
||||
|
||||
## Workflow Steps
|
||||
|
||||
### 1. Identify Target Features
|
||||
|
||||
Review the `CHANGELOG.md` for the latest version to map out the new UI elements. For example:
|
||||
|
||||
- **CLI Tools Settings**
|
||||
- **New Provider/Model Listings (e.g., Gemini 3.1, Qoder PAT)**
|
||||
- **New Feature Modals**
|
||||
|
||||
### 2. Run the Browser Subagent
|
||||
|
||||
For each identified feature, invoke the `browser_subagent` using the `default_api:browser_subagent` tool.
|
||||
**Important Task Guidelines for the Subagent:**
|
||||
|
||||
- `TaskName`: Give it a clear name like "Validate CLIProxyAPI Tool Tab".
|
||||
- `TaskSummary`: "Navigate to the CLI Tools tab and verify the new Integration settings."
|
||||
- `Task`: Provide unambiguous instructions for the subagent, such as: "Navigate to http://192.168.0.15:20128/dashboard. Click on the 'Settings' or 'CLI Tools' nav link. Scroll down to find the CLIProxyAPI integration card. Hover over it to trigger UI state. Verify the components render correctly and exit."
|
||||
- `RecordingName`: Ensure it describes the feature (e.g. `v3_4_5_cli_proxy_api`). This is required and strictly automatically saved as a WebP artifacts video by the system.
|
||||
|
||||
_(Note: The `browser_subagent` automatically creates a WebP recording named by the `RecordingName` parameter. No additional tools for screenshots are needed.)_
|
||||
|
||||
### 3. Generate Report Artifact
|
||||
|
||||
After the `browser_subagent` finishes its sessions, generate a final Markdown artifact (using `write_to_file` and `IsArtifact=true`) to present the recordings inline to the user using the `` syntax.
|
||||
|
||||
### Example Invocation
|
||||
|
||||
\```json
|
||||
{
|
||||
"TaskName": "Validating Qoder PAT Configuration UI",
|
||||
"TaskSummary": "Validates the Qoder provider configuration modal",
|
||||
"Task": "Go to http://192.168.0.15:20128/dashboard. Click on the 'Providers' tab. Find 'Qoder' in the list. Click 'Add Token' or 'Configure'. Type 'test_token' and submit. Return when done.",
|
||||
"RecordingName": "qoder_pat_ui_validation"
|
||||
}
|
||||
\```
|
||||
@@ -68,49 +68,78 @@ For each issue, determine its type:
|
||||
|
||||
Focus ONLY on **Bugs** for resolution. Feature requests and questions should be skipped with a note in the final report.
|
||||
|
||||
### 5. Analyze Each Bug — For each bug issue:
|
||||
### 5. Deep-Read Each Bug Issue (One-by-One Analysis)
|
||||
|
||||
#### 5a. Check Information Sufficiency
|
||||
**IMPORTANT**: Read each bug issue thoroughly, one at a time, before moving to the next. This is NOT a batch process — each issue needs focused attention.
|
||||
|
||||
Verify the issue contains enough information to reproduce and fix:
|
||||
#### 5a. Understand the Problem
|
||||
|
||||
For each bug issue, perform the full analysis:
|
||||
|
||||
1. **Read the entire body** — including Description, Steps to Reproduce, Expected/Actual Behavior, Error Logs, and Screenshots
|
||||
2. **Read ALL comments** — including bot triage comments (Kilo, etc.) and owner/community responses. Pay attention to:
|
||||
- Whether someone already responded with a fix
|
||||
- Whether a community member confirmed the issue is resolved
|
||||
- Whether the issue was marked as duplicate by a bot
|
||||
3. **Identify the claimed error** — extract the exact error message, status code, and provider/model involved
|
||||
|
||||
#### 5b. Check Information Sufficiency
|
||||
|
||||
Verify the issue contains enough to act on:
|
||||
|
||||
- [ ] Clear description of the problem
|
||||
- [ ] Steps to reproduce
|
||||
- [ ] Error messages or logs
|
||||
- [ ] Steps to reproduce OR error logs
|
||||
- [ ] Provider/model/version information
|
||||
- [ ] Expected vs actual behavior
|
||||
|
||||
#### 5b. If Information Is INSUFFICIENT
|
||||
#### 5c. Determine Issue Disposition
|
||||
|
||||
Call the `/issue-triage` workflow (located at `~/.gemini/antigravity/global_workflows/issue-triage.md`):
|
||||
// turbo
|
||||
For each bug, classify into one of 5 actions:
|
||||
|
||||
- Post a comment asking for more details using `gh issue comment`
|
||||
- Add `needs-info` label using `gh issue edit`
|
||||
- Mark this issue as **DEFERRED** and move to the next one
|
||||
| Disposition | When to Apply | Action |
|
||||
| ---------------------------- | ------------------------------------------------------------------------------------------- | --------------------------------------------------- |
|
||||
| **✅ CLOSE — Already Fixed** | Owner responded with fix + no user follow-up, OR community confirmed fix | Close with comment citing which version fixed it |
|
||||
| **✅ CLOSE — Duplicate** | Bot flagged >85% similarity + user provides no new info | Close referencing the original issue |
|
||||
| **📝 RESPOND — Needs Info** | Issue is real but missing critical reproduction details | Comment asking for specifics per `/issue-triage` |
|
||||
| **📝 RESPOND — User Config** | Error is caused by unsupported env (Node version, wrong model path, missing API enablement) | Comment explaining the user-side fix |
|
||||
| **🔧 FIX — Code Change** | Root cause is confirmed in the codebase | Research, implement, test, commit on release branch |
|
||||
|
||||
#### 5c. If Information Is SUFFICIENT
|
||||
#### 5d. For "FIX — Code Change" Issues
|
||||
|
||||
Proceed with resolution **on the release branch**:
|
||||
Before coding, perform deep source analysis:
|
||||
|
||||
1. **Research** — Search the codebase for files related to the issue
|
||||
2. **Root Cause** — Identify the root cause by reading the relevant source files
|
||||
3. **Implement Fix** — Apply the fix following existing code patterns and conventions
|
||||
4. **Test** — Build the project and run tests to verify the fix
|
||||
5. **Commit** — Commit with message format: `fix: <description> (#<issue_number>)`
|
||||
1. **Search the codebase** — `grep_search` for error strings, relevant function names, affected files
|
||||
2. **Search the web** — for upstream API changes, SDK updates, or breaking changes that explain the bug
|
||||
3. **Read the full source file** — don't rely on grep snippets; understand the surrounding logic
|
||||
4. **Verify the root cause** — confirm the bug is reproducible based on the code, not just a user misconfiguration
|
||||
5. **Implement the fix** — follow existing code patterns and conventions
|
||||
6. **Run tests** — `node --import tsx/esm --test tests/unit/*.test.mjs` (must pass 100%)
|
||||
7. **Commit** — `fix: <description> (#<issue_number>)`
|
||||
|
||||
> **⚠️ Do NOT create a separate branch.** All commits go directly on the release branch.
|
||||
#### 5e. For "RESPOND" Issues
|
||||
|
||||
Post a substantive comment that:
|
||||
|
||||
- Acknowledges the specific error they reported
|
||||
- Explains the likely root cause
|
||||
- Provides concrete steps to resolve (version upgrade, env var fix, model path correction)
|
||||
- Asks for follow-up info if needed
|
||||
|
||||
**Do NOT post generic template responses.** Every comment should reference the user's specific error messages and environment.
|
||||
|
||||
### 6. Generate Report & Wait for Validation
|
||||
|
||||
Present a summary report to the user via `notify_user` with `BlockedOnUser: true`:
|
||||
|
||||
| Issue | Title | Status | Action |
|
||||
| ----- | ----- | ------------- | ----------------------------- |
|
||||
| #N | Title | ✅ Ready | Files changed (not committed) |
|
||||
| #N | Title | ❓ Needs Info | Triage comment posted |
|
||||
| #N | Title | ⏭️ Skipped | Feature request / not a bug |
|
||||
| Issue | Title | Status | Action |
|
||||
| ----- | ----- | ------------- | --------------------------- |
|
||||
| #N | Title | ✅ Closed | Already fixed / duplicate |
|
||||
| #N | Title | 🔧 Fixed | Code fix applied |
|
||||
| #N | Title | 📝 Responded | Guidance comment posted |
|
||||
| #N | Title | ❓ Needs Info | Triage comment posted |
|
||||
| #N | Title | ⏭️ Skipped | Feature request / not a bug |
|
||||
|
||||
> **⚠️ IMPORTANT**: Do NOT commit, close issues, or generate releases at this step.
|
||||
> **⚠️ IMPORTANT**: Do NOT merge or generate releases at this step.
|
||||
> Wait for the user to review the changes and respond with **OK** before proceeding.
|
||||
|
||||
- If the user says **OK** or approves → Proceed to step 7
|
||||
|
||||
6
.gitignore
vendored
6
.gitignore
vendored
@@ -56,6 +56,7 @@ next-env.d.ts
|
||||
|
||||
# data and logs
|
||||
data/
|
||||
.data/
|
||||
logs/*
|
||||
|
||||
# analysis directories (generated, not tracked)
|
||||
@@ -153,4 +154,7 @@ vscode-extension/
|
||||
typescript
|
||||
|
||||
# Gemini Antigravity agent data
|
||||
.gemini/
|
||||
.gemini/
|
||||
|
||||
# Superpowers plans/specs (internal tooling, not project code)
|
||||
docs/superpowers/
|
||||
42
CHANGELOG.md
42
CHANGELOG.md
@@ -4,6 +4,48 @@
|
||||
|
||||
---
|
||||
|
||||
## [3.4.7] — 2026-04-03
|
||||
|
||||
### Features
|
||||
|
||||
- Added `Cryptography` node to Monitoring and MCP health checks (#798)
|
||||
- Hardened model-catalog route permissions mapping (`/models`) (#781)
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
- Fixed Claude OAuth token refreshes failing to preserve cache contexts (#937)
|
||||
- Fixed CC-Compatible provider errors rendering cached models unreachable (#937)
|
||||
- Fixed GitHub Executor errors related to invalid context arrays (#937)
|
||||
- Fixed NPM-installed CLI tools healthcheck failures on Windows (#935)
|
||||
- Fixed payload translation dropping valid content due to invalid API fields (#927)
|
||||
- Fixed runtime crash in Node 25 regarding API key execution (#867)
|
||||
- Fixed MCP standalone module-resolution (`ERR_MODULE_NOT_FOUND`) via `esbuild` (#936)
|
||||
- Fixed NVIDIA NIM routing credential resolution alias mismatch (#931)
|
||||
|
||||
### Security
|
||||
|
||||
- Added safe strict input boundary protection against raw `shell: true` remote-code execution injections.
|
||||
|
||||
---
|
||||
|
||||
## [3.4.6] - 2026-04-02
|
||||
|
||||
### ✨ New Features
|
||||
|
||||
- **Providers:** Registered new image, video, and audio generation providers from the community-requested list (#926).
|
||||
- **Dashboard UI:** Added standalone sidebar navigation for the new Memory and Skills modules (#926).
|
||||
- **i18n:** Added translation strings and layout mappings across 30 languages for the Memory and Skills namespaces.
|
||||
|
||||
### 🐛 Bug Fixes
|
||||
|
||||
- **Resilience:** Prevented the proxy Circuit Breaker from becoming stuck in an OPEN state indefinitely by handling direct transitions to CLOSED state inside fallback combo paths (#930).
|
||||
- **Protocol Translation:** Patched the streaming transformer to sanitize response blocks based on the expected _source_ protocol rather than the provider _target_ protocol, fixing Anthropics models wrapped in OpenAI payloads crashing Claude Code (#929).
|
||||
- **API Specs & Gemini:** Fixed `thought_signature` parsing in `openai-to-gemini` and `claude-to-gemini` translators, preventing HTTP 400 errors across all Gemini 3 API tool-calls.
|
||||
- **Providers:** Cleaned up non-OpenAI-compatible endpoints preventing valid upstream connections (#926).
|
||||
- **Cache Trends:** Fixed an invalid property mapping data mismatch causing Cache Trends UI charts to crash, and extracted redundant cache metric widgets (#926).
|
||||
|
||||
---
|
||||
|
||||
## [3.4.5] - 2026-04-02
|
||||
|
||||
### ✨ New Features
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
openapi: 3.1.0
|
||||
info:
|
||||
title: OmniRoute API
|
||||
version: 3.4.5
|
||||
version: 3.4.7
|
||||
description: |
|
||||
OmniRoute is a local-first AI API proxy router. It provides an OpenAI-compatible
|
||||
endpoint that routes requests to multiple AI providers with load balancing,
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "omniroute-desktop",
|
||||
"version": "3.4.5",
|
||||
"version": "3.4.7",
|
||||
"description": "OmniRoute Desktop Application",
|
||||
"main": "main.js",
|
||||
"author": {
|
||||
|
||||
4
llm.txt
4
llm.txt
@@ -8,7 +8,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
|
||||
**Key value:** One endpoint (`http://localhost:20128/v1`), unlimited models, zero downtime, minimal cost.
|
||||
|
||||
**Current version:** 3.4.5
|
||||
**Current version:** 3.4.6
|
||||
|
||||
## Tech Stack
|
||||
|
||||
@@ -277,7 +277,7 @@ OmniRoute solves the problem of managing multiple AI provider subscriptions, quo
|
||||
└── .env.example # Environment variable template
|
||||
```
|
||||
|
||||
## Key Features (v3.4.5)
|
||||
## Key Features (v3.4.6)
|
||||
|
||||
### Core Proxy
|
||||
- **60+ AI providers** with automatic format translation
|
||||
|
||||
@@ -192,22 +192,9 @@ export const REGISTRY: Record<string, RegistryEntry> = {
|
||||
clientSecretEnv: "GEMINI_OAUTH_CLIENT_SECRET",
|
||||
clientSecretDefault: "",
|
||||
},
|
||||
models: [
|
||||
{ id: "gemini-3.1-pro-high", name: "Gemini 3.1 Pro High" },
|
||||
{ id: "gemini-3.1-pro-low", name: "Gemini 3.1 Pro Low" },
|
||||
{ id: "gemini-3.1-pro", name: "Gemini 3.1 Pro" },
|
||||
{ id: "gemini-3-1-pro", name: "Gemini 3.1 Pro (Alt ID)" },
|
||||
{ id: "gemini-3.1-pro-preview", name: "Gemini 3.1 Pro Preview" },
|
||||
{ id: "gemini-3.1-flash-lite-preview", name: "Gemini 3.1 Flash Lite Preview" },
|
||||
{ id: "gemini-3-flash-preview", name: "Gemini 3 Flash Preview" },
|
||||
{ id: "gemini-2.5-pro", name: "Gemini 2.5 Pro" },
|
||||
{ id: "gemini-2.5-flash", name: "Gemini 2.5 Flash" },
|
||||
{ id: "gemini-2.5-flash-lite", name: "Gemini 2.5 Flash Lite" },
|
||||
{ id: "gemini-2.0-flash", name: "Gemini 2.0 Flash" },
|
||||
{ id: "gemini-2.0-flash-exp", name: "Gemini 2.0 Flash Exp" },
|
||||
{ id: "gemini-1.5-pro", name: "Gemini 1.5 Pro" },
|
||||
{ id: "gemini-1.5-flash", name: "Gemini 1.5 Flash" },
|
||||
],
|
||||
models: [],
|
||||
// Models are populated from Google's API via sync-models (per API key).
|
||||
// No hardcoded fallback — show nothing until a key is added.
|
||||
},
|
||||
|
||||
"gemini-cli": {
|
||||
|
||||
@@ -76,7 +76,12 @@ export class GithubExecutor extends BaseExecutor {
|
||||
if (!result || !result.response?.body) return result;
|
||||
|
||||
const isStreaming = input.stream === true;
|
||||
if (isStreaming) {
|
||||
const contentType = (result.response.headers.get("content-type") || "").toLowerCase();
|
||||
if (isStreaming && result.response.ok && contentType.includes("text/event-stream")) {
|
||||
// Preserve the original response body for downstream error handling.
|
||||
const sourceResponse = result.response.clone();
|
||||
if (!sourceResponse.body) return result;
|
||||
|
||||
const decoder = new TextDecoder();
|
||||
const transformStream = new TransformStream({
|
||||
transform(chunk, controller) {
|
||||
@@ -88,10 +93,10 @@ export class GithubExecutor extends BaseExecutor {
|
||||
},
|
||||
});
|
||||
|
||||
const newResponse = new Response(result.response.body.pipeThrough(transformStream), {
|
||||
status: result.response.status,
|
||||
statusText: result.response.statusText,
|
||||
headers: result.response.headers, // Headers class carries over correctly
|
||||
const newResponse = new Response(sourceResponse.body.pipeThrough(transformStream), {
|
||||
status: sourceResponse.status,
|
||||
statusText: sourceResponse.statusText,
|
||||
headers: new Headers(sourceResponse.headers),
|
||||
});
|
||||
result.response = newResponse;
|
||||
}
|
||||
|
||||
@@ -13,7 +13,9 @@ import { refreshWithRetry } from "../services/tokenRefresh.ts";
|
||||
import { createRequestLogger } from "../utils/requestLogger.ts";
|
||||
import { getModelTargetFormat, PROVIDER_ID_TO_ALIAS } from "../config/providerModels.ts";
|
||||
import { resolveModelAlias } from "../services/modelDeprecation.ts";
|
||||
import { getUnsupportedParams, getPassthroughProviders } from "../config/providerRegistry.ts";
|
||||
import { getUnsupportedParams } from "../config/providerRegistry.ts";
|
||||
import { hasPerModelQuota, lockModelIfPerModelQuota } from "../services/accountFallback.ts";
|
||||
import { COOLDOWN_MS } from "../config/constants.ts";
|
||||
import {
|
||||
buildErrorBody,
|
||||
createErrorResult,
|
||||
@@ -794,11 +796,13 @@ export async function handleChatCore({
|
||||
translatedBody = buildClaudeCodeCompatibleRequest({
|
||||
sourceBody: body,
|
||||
normalizedBody: normalizedForCc,
|
||||
claudeBody: sourceFormat === FORMATS.CLAUDE ? body : null,
|
||||
model,
|
||||
stream: upstreamStream,
|
||||
sessionId: ccSessionId,
|
||||
cwd: process.cwd(),
|
||||
now: new Date(),
|
||||
preserveCacheControl,
|
||||
});
|
||||
log?.debug?.("FORMAT", "claude-code-compatible bridge enabled");
|
||||
} else if (isClaudePassthrough && preserveCacheControl) {
|
||||
@@ -1375,16 +1379,20 @@ export async function handleChatCore({
|
||||
`[provider] Node ${connectionId} account deactivated (${statusCode}) — disabling permanently`
|
||||
);
|
||||
} else if (errorType === PROVIDER_ERROR_TYPES.RATE_LIMITED) {
|
||||
// For passthrough providers (e.g. Antigravity), each model has independent
|
||||
// quota. A 429 on one model must NOT lock out the entire connection — other
|
||||
// models may still have quota available. Use lockModel() instead.
|
||||
const isPassthrough = provider && getPassthroughProviders().has(provider);
|
||||
if (isPassthrough) {
|
||||
const { lockModel } = await import("../services/accountFallback.ts");
|
||||
const cooldown = retryAfterMs || 120_000; // 2 min default, same as COOLDOWN_MS.rateLimit
|
||||
lockModel(provider, connectionId, model, "rate_limited", cooldown);
|
||||
// For providers with per-model quotas (passthrough providers, Gemini),
|
||||
// each model has independent quota. A 429 on one model must NOT lock out
|
||||
// the entire connection — other models may still have quota available.
|
||||
if (
|
||||
lockModelIfPerModelQuota(
|
||||
provider,
|
||||
connectionId,
|
||||
model,
|
||||
"rate_limited",
|
||||
retryAfterMs || COOLDOWN_MS.rateLimit
|
||||
)
|
||||
) {
|
||||
console.warn(
|
||||
`[provider] Node ${connectionId} model-only rate limited (${statusCode}) for ${model} - ${Math.ceil(cooldown / 1000)}s (connection stays active)`
|
||||
`[provider] Node ${connectionId} model-only rate limited (${statusCode}) for ${model} - ${Math.ceil((retryAfterMs || COOLDOWN_MS.rateLimit) / 1000)}s (connection stays active)`
|
||||
);
|
||||
} else {
|
||||
const rateLimitedUntil = new Date(Date.now() + retryAfterMs).toISOString();
|
||||
@@ -1402,13 +1410,28 @@ export async function handleChatCore({
|
||||
);
|
||||
}
|
||||
} else if (errorType === PROVIDER_ERROR_TYPES.QUOTA_EXHAUSTED) {
|
||||
await updateProviderConnection(connectionId, {
|
||||
testStatus: "credits_exhausted",
|
||||
lastErrorType: errorType,
|
||||
lastError: message,
|
||||
errorCode: statusCode,
|
||||
});
|
||||
console.warn(`[provider] Node ${connectionId} exhausted quota (${statusCode})`);
|
||||
// Providers with per-model quotas — lock the model only, not the connection
|
||||
if (
|
||||
lockModelIfPerModelQuota(
|
||||
provider,
|
||||
connectionId,
|
||||
model,
|
||||
"quota_exhausted",
|
||||
retryAfterMs || COOLDOWN_MS.rateLimit
|
||||
)
|
||||
) {
|
||||
console.warn(
|
||||
`[provider] Node ${connectionId} model-only quota exhausted (${statusCode}) for ${model} - ${Math.ceil((retryAfterMs || COOLDOWN_MS.rateLimit) / 1000)}s (connection stays active)`
|
||||
);
|
||||
} else {
|
||||
await updateProviderConnection(connectionId, {
|
||||
testStatus: "credits_exhausted",
|
||||
lastErrorType: errorType,
|
||||
lastError: message,
|
||||
errorCode: statusCode,
|
||||
});
|
||||
console.warn(`[provider] Node ${connectionId} exhausted quota (${statusCode})`);
|
||||
}
|
||||
} else if (errorType === PROVIDER_ERROR_TYPES.ACCOUNT_DEACTIVATED) {
|
||||
await updateProviderConnection(connectionId, {
|
||||
isActive: false,
|
||||
@@ -1776,8 +1799,8 @@ export async function handleChatCore({
|
||||
// Sanitize response for OpenAI SDK compatibility
|
||||
// Strips non-standard fields (x_groq, usage_breakdown, service_tier, etc.)
|
||||
// Extracts <think> and <thinking> tags into reasoning_content
|
||||
// Target format determines output shape. If we are outputting OpenAI shape or pseudo-OpenAI shape, sanitize.
|
||||
if (targetFormat === FORMATS.OPENAI || targetFormat === FORMATS.OPENAI_RESPONSES) {
|
||||
// Source format determines output shape. If we are outputting OpenAI shape or pseudo-OpenAI shape, sanitize.
|
||||
if (sourceFormat === FORMATS.OPENAI || sourceFormat === FORMATS.OPENAI_RESPONSES) {
|
||||
translatedResponse = sanitizeOpenAIResponse(translatedResponse);
|
||||
}
|
||||
|
||||
|
||||
@@ -69,6 +69,12 @@ export const getHealthOutput = z.object({
|
||||
hitRate: z.number(),
|
||||
})
|
||||
.optional(),
|
||||
cryptography: z
|
||||
.object({
|
||||
status: z.enum(["healthy", "missing_or_invalid"]),
|
||||
provider: z.string(),
|
||||
})
|
||||
.optional(),
|
||||
});
|
||||
|
||||
export const getHealthTool: McpToolDefinition<typeof getHealthInput, typeof getHealthOutput> = {
|
||||
|
||||
@@ -212,6 +212,12 @@ async function handleGetHealth() {
|
||||
hitRate: toNumber(cacheStatsRaw.hitRate, 0),
|
||||
}
|
||||
: undefined,
|
||||
cryptography: health.cryptography
|
||||
? {
|
||||
status: toString(toRecord(health.cryptography).status, "missing_or_invalid"),
|
||||
provider: toString(toRecord(health.cryptography).provider, "unknown"),
|
||||
}
|
||||
: undefined,
|
||||
};
|
||||
|
||||
await logToolCall("omniroute_get_health", {}, result, Date.now() - start, true);
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "@omniroute/open-sse",
|
||||
"version": "3.4.5",
|
||||
"version": "3.4.7",
|
||||
"description": "Express SSE sidecar for OmniRoute — handles streaming, protocol translation, and provider orchestration",
|
||||
"type": "module",
|
||||
"main": "index.js",
|
||||
|
||||
@@ -100,13 +100,49 @@ export function lockModel(provider, connectionId, model, reason, cooldownMs) {
|
||||
if (!model) return; // No model → skip model-level locking
|
||||
ensureCleanupTimer();
|
||||
const key = `${provider}:${connectionId}:${model}`;
|
||||
const newUntil = Date.now() + cooldownMs;
|
||||
// Preserve the longer cooldown if an existing lock has more time remaining.
|
||||
// Safe without a mutex: no await between get/set, so this runs atomically
|
||||
// within Node.js's single-threaded event loop.
|
||||
const existing = modelLockouts.get(key);
|
||||
if (existing && existing.until > newUntil) return;
|
||||
modelLockouts.set(key, {
|
||||
reason,
|
||||
until: Date.now() + cooldownMs,
|
||||
until: newUntil,
|
||||
lockedAt: Date.now(),
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether a provider should use per-model lockouts instead of connection-wide cooldowns.
|
||||
* Gemini AI Studio has per-model quotas; passthrough providers have independent model limits.
|
||||
*/
|
||||
export function hasPerModelQuota(provider: string): boolean {
|
||||
if (provider === "gemini") return true;
|
||||
try {
|
||||
const { getPassthroughProviders } = require("../config/providerRegistry.ts");
|
||||
return getPassthroughProviders().has(provider);
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Lock a model (not connection) for a provider with per-model quotas.
|
||||
* No-ops for providers that don't use per-model lockouts.
|
||||
*/
|
||||
export function lockModelIfPerModelQuota(
|
||||
provider: string,
|
||||
connectionId: string,
|
||||
model: string | null,
|
||||
reason: string,
|
||||
cooldownMs: number
|
||||
): boolean {
|
||||
if (!hasPerModelQuota(provider) || !model) return false;
|
||||
lockModel(provider, connectionId, model, reason, cooldownMs);
|
||||
return true;
|
||||
}
|
||||
|
||||
/**
|
||||
* Check if a specific model on a specific account is locked
|
||||
* @returns {boolean}
|
||||
|
||||
@@ -16,6 +16,7 @@ export interface RoutingContext {
|
||||
requestHasTools?: boolean;
|
||||
requestHasVision?: boolean;
|
||||
estimatedInputTokens?: number;
|
||||
lastKnownGoodProvider?: string;
|
||||
}
|
||||
|
||||
export interface RoutingDecision {
|
||||
@@ -116,6 +117,34 @@ class LatencyStrategyImpl implements RouterStrategy {
|
||||
}
|
||||
}
|
||||
|
||||
// ── LKGPStrategy: tries last known good provider first ───────────────────────
|
||||
|
||||
class LKGPStrategyImpl implements RouterStrategy {
|
||||
readonly name = "lkgp";
|
||||
readonly description = "Tries last known good provider first, then falls back to rules";
|
||||
|
||||
select(pool: ProviderCandidate[], context: RoutingContext): RoutingDecision {
|
||||
if (context.lastKnownGoodProvider) {
|
||||
const best = pool.find(
|
||||
(c) => c.provider === context.lastKnownGoodProvider && c.circuitBreakerState !== "OPEN"
|
||||
);
|
||||
if (best) {
|
||||
return {
|
||||
provider: best.provider,
|
||||
model: best.model,
|
||||
strategy: this.name,
|
||||
reason: `LKGP: using last known good provider ${best.provider}`,
|
||||
candidatesConsidered: 1,
|
||||
finalScore: 1.0,
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
// Fallback to rules strategy
|
||||
return getStrategy("rules").select(pool, context);
|
||||
}
|
||||
}
|
||||
|
||||
// ── Registry ──────────────────────────────────────────────────────────────────
|
||||
|
||||
const strategyRegistry = new Map<string, RouterStrategy>();
|
||||
@@ -123,12 +152,14 @@ const strategyRegistry = new Map<string, RouterStrategy>();
|
||||
const rulesStrategy = new RulesStrategyImpl();
|
||||
const costStrategy = new CostStrategyImpl();
|
||||
const latencyStrategy = new LatencyStrategyImpl();
|
||||
const lkgpStrategy = new LKGPStrategyImpl();
|
||||
|
||||
strategyRegistry.set("rules", rulesStrategy);
|
||||
strategyRegistry.set("cost", costStrategy);
|
||||
strategyRegistry.set("eco", costStrategy); // alias
|
||||
strategyRegistry.set("latency", latencyStrategy);
|
||||
strategyRegistry.set("fast", latencyStrategy); // alias
|
||||
strategyRegistry.set("lkgp", lkgpStrategy);
|
||||
|
||||
export function getStrategy(name: string): RouterStrategy {
|
||||
const strategy = strategyRegistry.get(name);
|
||||
|
||||
@@ -1,5 +1,7 @@
|
||||
import { createHash, randomUUID } from "node:crypto";
|
||||
|
||||
import { prepareClaudeRequest } from "../translator/helpers/claudeHelper.ts";
|
||||
|
||||
export const CLAUDE_CODE_COMPATIBLE_PREFIX = "anthropic-compatible-cc-";
|
||||
export const CLAUDE_CODE_COMPATIBLE_DEFAULT_CHAT_PATH = "/v1/messages?beta=true";
|
||||
export const CLAUDE_CODE_COMPATIBLE_DEFAULT_MODELS_PATH = "/models";
|
||||
@@ -26,11 +28,13 @@ type MessageLike = {
|
||||
type BuildRequestOptions = {
|
||||
sourceBody?: Record<string, unknown> | null;
|
||||
normalizedBody?: Record<string, unknown> | null;
|
||||
claudeBody?: Record<string, unknown> | null;
|
||||
model: string;
|
||||
stream?: boolean;
|
||||
cwd?: string;
|
||||
now?: Date;
|
||||
sessionId?: string | null;
|
||||
preserveCacheControl?: boolean;
|
||||
};
|
||||
|
||||
export function isClaudeCodeCompatibleProvider(provider: string | null | undefined): boolean {
|
||||
@@ -137,25 +141,38 @@ export function resolveClaudeCodeCompatibleSessionId(headers?: HeaderLike): stri
|
||||
export function buildClaudeCodeCompatibleRequest({
|
||||
sourceBody,
|
||||
normalizedBody,
|
||||
claudeBody,
|
||||
model,
|
||||
stream = false,
|
||||
cwd = process.cwd(),
|
||||
now = new Date(),
|
||||
sessionId,
|
||||
preserveCacheControl = false,
|
||||
}: BuildRequestOptions) {
|
||||
const normalized = normalizedBody || {};
|
||||
const messages = Array.isArray(normalized.messages)
|
||||
? buildClaudeCodeCompatibleMessages(normalized.messages as MessageLike[])
|
||||
: [];
|
||||
const system = buildClaudeCodeCompatibleSystemBlocks(
|
||||
normalized.messages as MessageLike[],
|
||||
const preparedClaudeBody = claudeBody
|
||||
? prepareClaudeCodeCompatibleBody(claudeBody, preserveCacheControl)
|
||||
: null;
|
||||
const messages = preparedClaudeBody
|
||||
? buildClaudeCodeCompatibleMessagesFromClaude(
|
||||
preparedClaudeBody.messages as MessageLike[],
|
||||
preserveCacheControl
|
||||
)
|
||||
: Array.isArray(normalized.messages)
|
||||
? buildClaudeCodeCompatibleMessages(normalized.messages as MessageLike[])
|
||||
: [];
|
||||
const system = buildClaudeCodeCompatibleSystemBlocks({
|
||||
messages: normalized.messages as MessageLike[],
|
||||
systemBlocks: preparedClaudeBody?.system as Record<string, unknown>[] | undefined,
|
||||
cwd,
|
||||
now
|
||||
);
|
||||
now,
|
||||
});
|
||||
const resolvedSessionId = sessionId || randomUUID();
|
||||
const effort = resolveClaudeCodeCompatibleEffort(sourceBody, normalizedBody, model);
|
||||
const maxTokens = resolveClaudeCodeCompatibleMaxTokens(sourceBody, normalizedBody);
|
||||
const tools = buildClaudeCodeCompatibleTools(normalizedBody, sourceBody);
|
||||
const tools = preparedClaudeBody?.tools
|
||||
? (preparedClaudeBody.tools as Record<string, unknown>[])
|
||||
: buildClaudeCodeCompatibleTools(normalizedBody, sourceBody);
|
||||
const toolChoice =
|
||||
tools.length > 0
|
||||
? buildClaudeCodeCompatibleToolChoice(
|
||||
@@ -295,32 +312,83 @@ function buildClaudeCodeCompatibleMessages(messages: MessageLike[]) {
|
||||
}
|
||||
}
|
||||
|
||||
for (let i = merged.length - 1; i >= 0; i--) {
|
||||
if (merged[i].role !== "user") continue;
|
||||
const lastBlock = merged[i].content[merged[i].content.length - 1];
|
||||
if (lastBlock) {
|
||||
lastBlock.cache_control = { type: "ephemeral" };
|
||||
applyClaudeCodeCompatibleMessageCacheStrategy(merged);
|
||||
|
||||
return merged;
|
||||
}
|
||||
|
||||
function buildClaudeCodeCompatibleMessagesFromClaude(
|
||||
messages: MessageLike[] | undefined,
|
||||
preserveCacheControl: boolean
|
||||
) {
|
||||
const converted = Array.isArray(messages)
|
||||
? messages
|
||||
.map((message) => convertClaudeCodeCompatibleClaudeMessage(message, preserveCacheControl))
|
||||
.filter(
|
||||
(
|
||||
message
|
||||
): message is { role: "user" | "assistant"; content: Array<Record<string, unknown>> } =>
|
||||
!!message && message.content.length > 0
|
||||
)
|
||||
: [];
|
||||
|
||||
const merged: Array<{ role: "user" | "assistant"; content: Array<Record<string, unknown>> }> = [];
|
||||
|
||||
for (const message of converted) {
|
||||
const last = merged[merged.length - 1];
|
||||
if (last && last.role === message.role) {
|
||||
last.content.push(...message.content);
|
||||
continue;
|
||||
}
|
||||
merged.push({ role: message.role, content: [...message.content] });
|
||||
}
|
||||
|
||||
while (merged.length > 0 && merged[merged.length - 1].role === "assistant") {
|
||||
merged.pop();
|
||||
}
|
||||
|
||||
if (!preserveCacheControl) {
|
||||
for (const message of merged) {
|
||||
stripCacheControlFromContentBlocks(message.content);
|
||||
}
|
||||
applyClaudeCodeCompatibleMessageCacheStrategy(merged);
|
||||
}
|
||||
|
||||
if (merged.length === 0) {
|
||||
const fallbackText = converted
|
||||
.flatMap((message) => message.content)
|
||||
.map((block) => contentToText(block))
|
||||
.filter(Boolean)
|
||||
.join("\n")
|
||||
.trim();
|
||||
if (fallbackText) {
|
||||
return [
|
||||
{
|
||||
role: "user" as const,
|
||||
content: [{ type: "text", text: fallbackText, cache_control: { type: "ephemeral" } }],
|
||||
},
|
||||
];
|
||||
}
|
||||
break;
|
||||
}
|
||||
|
||||
return merged;
|
||||
}
|
||||
|
||||
function buildClaudeCodeCompatibleSystemBlocks(
|
||||
messages: MessageLike[] | undefined,
|
||||
cwd: string,
|
||||
now: Date
|
||||
) {
|
||||
const customSystemBlocks = Array.isArray(messages)
|
||||
? messages
|
||||
.filter((message) => {
|
||||
const role = String(message?.role || "").toLowerCase();
|
||||
return role === "system" || role === "developer";
|
||||
})
|
||||
.map((message) => contentToText(message?.content))
|
||||
.filter(Boolean)
|
||||
: [];
|
||||
function buildClaudeCodeCompatibleSystemBlocks({
|
||||
messages,
|
||||
systemBlocks,
|
||||
cwd,
|
||||
now,
|
||||
}: {
|
||||
messages: MessageLike[] | undefined;
|
||||
systemBlocks?: Array<Record<string, unknown>> | undefined;
|
||||
cwd: string;
|
||||
now: Date;
|
||||
}) {
|
||||
const customSystemBlocks =
|
||||
Array.isArray(systemBlocks) && systemBlocks.length > 0
|
||||
? systemBlocks
|
||||
: extractCustomSystemBlocks(messages);
|
||||
|
||||
const dateText = formatDate(now);
|
||||
const blocks: Array<Record<string, unknown>> = [
|
||||
@@ -340,12 +408,8 @@ function buildClaudeCodeCompatibleSystemBlocks(
|
||||
},
|
||||
];
|
||||
|
||||
for (const systemText of customSystemBlocks) {
|
||||
blocks.push({
|
||||
type: "text",
|
||||
text: systemText,
|
||||
cache_control: { type: "ephemeral" },
|
||||
});
|
||||
for (const systemBlock of customSystemBlocks) {
|
||||
blocks.push(systemBlock);
|
||||
}
|
||||
|
||||
return blocks;
|
||||
@@ -383,7 +447,14 @@ function buildClaudeCodeCompatibleTools(
|
||||
|
||||
return rawTools
|
||||
.map((tool) => convertClaudeCodeCompatibleTool(tool))
|
||||
.filter((tool): tool is Record<string, unknown> => !!tool);
|
||||
.filter((tool): tool is Record<string, unknown> => !!tool)
|
||||
.map((tool, index, tools) => {
|
||||
if (index !== findLastCacheableToolIndex(tools)) return tool;
|
||||
return {
|
||||
...tool,
|
||||
cache_control: { type: "ephemeral", ttl: "1h" },
|
||||
};
|
||||
});
|
||||
}
|
||||
|
||||
function convertClaudeCodeCompatibleTool(tool: unknown) {
|
||||
@@ -444,6 +515,190 @@ function buildClaudeCodeCompatibleToolChoice(choice: unknown) {
|
||||
return null;
|
||||
}
|
||||
|
||||
function prepareClaudeCodeCompatibleBody(
|
||||
claudeBody: Record<string, unknown>,
|
||||
preserveCacheControl: boolean
|
||||
) {
|
||||
const prepared = prepareClaudeRequest(
|
||||
{
|
||||
system: normalizeClaudeSystemInput(claudeBody.system),
|
||||
messages: normalizeClaudeMessageInput(claudeBody.messages),
|
||||
tools: normalizeClaudeToolInput(claudeBody.tools),
|
||||
thinking: readRecord(claudeBody.thinking) || claudeBody.thinking,
|
||||
},
|
||||
CLAUDE_CODE_COMPATIBLE_PREFIX,
|
||||
preserveCacheControl
|
||||
);
|
||||
|
||||
return readRecord(prepared);
|
||||
}
|
||||
|
||||
function normalizeClaudeSystemInput(system: unknown) {
|
||||
if (typeof system === "string") {
|
||||
const text = system.trim();
|
||||
return text ? [{ type: "text", text }] : [];
|
||||
}
|
||||
|
||||
if (!Array.isArray(system)) return [];
|
||||
return system
|
||||
.map((block) => normalizeClaudeContentBlock(block))
|
||||
.filter((block): block is Record<string, unknown> => !!block);
|
||||
}
|
||||
|
||||
function normalizeClaudeMessageInput(messages: unknown) {
|
||||
if (!Array.isArray(messages)) return [];
|
||||
return messages
|
||||
.map((message) => {
|
||||
const record = readRecord(message);
|
||||
if (!record) return null;
|
||||
|
||||
return {
|
||||
...record,
|
||||
content: normalizeClaudeContentInput(record.content),
|
||||
};
|
||||
})
|
||||
.filter((message): message is Record<string, unknown> => !!message);
|
||||
}
|
||||
|
||||
function normalizeClaudeToolInput(tools: unknown) {
|
||||
if (!Array.isArray(tools)) return [];
|
||||
return tools
|
||||
.map((tool) => readRecord(cloneValue(tool)))
|
||||
.filter((tool): tool is Record<string, unknown> => !!tool);
|
||||
}
|
||||
|
||||
function normalizeClaudeContentInput(content: unknown) {
|
||||
const blocks = normalizeClaudeContentBlocks(content);
|
||||
return blocks.length > 0 ? blocks : content;
|
||||
}
|
||||
|
||||
function normalizeClaudeContentBlocks(content: unknown) {
|
||||
if (typeof content === "string") {
|
||||
const text = content.trim();
|
||||
return text ? [{ type: "text", text }] : [];
|
||||
}
|
||||
|
||||
if (!Array.isArray(content)) {
|
||||
const block = normalizeClaudeContentBlock(content);
|
||||
return block ? [block] : [];
|
||||
}
|
||||
|
||||
return content
|
||||
.map((block) => normalizeClaudeContentBlock(block))
|
||||
.filter((block): block is Record<string, unknown> => !!block);
|
||||
}
|
||||
|
||||
function normalizeClaudeContentBlock(block: unknown) {
|
||||
const record = readRecord(cloneValue(block));
|
||||
if (!record) return null;
|
||||
|
||||
if (
|
||||
record.type === "text" ||
|
||||
(typeof record.type !== "string" && typeof record.text === "string")
|
||||
) {
|
||||
const text = toNonEmptyString(record.text);
|
||||
if (!text) return null;
|
||||
return {
|
||||
...record,
|
||||
type: "text",
|
||||
text,
|
||||
};
|
||||
}
|
||||
|
||||
return record;
|
||||
}
|
||||
|
||||
function convertClaudeCodeCompatibleClaudeMessage(
|
||||
message: MessageLike | null | undefined,
|
||||
preserveCacheControl: boolean
|
||||
) {
|
||||
const rawRole = String(message?.role || "").toLowerCase();
|
||||
const role = rawRole === "user" ? "user" : rawRole === "assistant" ? "assistant" : null;
|
||||
|
||||
if (!role) return null;
|
||||
|
||||
const content = normalizeClaudeContentBlocks(message?.content).map((block) => {
|
||||
if (preserveCacheControl) return block;
|
||||
const { cache_control, ...rest } = block;
|
||||
return rest;
|
||||
});
|
||||
if (content.length === 0) return null;
|
||||
|
||||
return {
|
||||
role,
|
||||
content,
|
||||
};
|
||||
}
|
||||
|
||||
function extractCustomSystemBlocks(messages: MessageLike[] | undefined) {
|
||||
if (!Array.isArray(messages)) return [];
|
||||
|
||||
return messages
|
||||
.filter((message) => {
|
||||
const role = String(message?.role || "").toLowerCase();
|
||||
return role === "system" || role === "developer";
|
||||
})
|
||||
.map((message) => contentToText(message?.content))
|
||||
.filter(Boolean)
|
||||
.map((text) => ({
|
||||
type: "text",
|
||||
text,
|
||||
cache_control: { type: "ephemeral" },
|
||||
}));
|
||||
}
|
||||
|
||||
function applyClaudeCodeCompatibleMessageCacheStrategy(
|
||||
messages: Array<{ role: "user" | "assistant"; content: Array<Record<string, unknown>> }>
|
||||
) {
|
||||
const userIndexes = messages.reduce((indexes, message, index) => {
|
||||
if (message.role === "user") indexes.push(index);
|
||||
return indexes;
|
||||
}, [] as number[]);
|
||||
const hasAssistant = messages.some((message) => message.role === "assistant");
|
||||
const secondToLastUserIndex = userIndexes.length >= 2 ? userIndexes[userIndexes.length - 2] : -1;
|
||||
|
||||
if (secondToLastUserIndex >= 0) {
|
||||
markLastContentCacheControl(messages[secondToLastUserIndex].content);
|
||||
} else if (!hasAssistant && userIndexes.length > 0) {
|
||||
markLastContentCacheControl(messages[userIndexes[userIndexes.length - 1]].content);
|
||||
}
|
||||
|
||||
for (let i = messages.length - 1; i >= 0; i--) {
|
||||
if (messages[i].role !== "assistant") continue;
|
||||
if (markLastContentCacheControl(messages[i].content)) break;
|
||||
}
|
||||
}
|
||||
|
||||
function stripCacheControlFromContentBlocks(content: Array<Record<string, unknown>>) {
|
||||
for (const block of content) {
|
||||
delete block.cache_control;
|
||||
}
|
||||
}
|
||||
|
||||
function markLastContentCacheControl(content: Array<Record<string, unknown>>, ttl?: string) {
|
||||
if (!Array.isArray(content) || content.length === 0) return false;
|
||||
const lastBlock = content[content.length - 1];
|
||||
if (!lastBlock) return false;
|
||||
lastBlock.cache_control = ttl ? { type: "ephemeral", ttl } : { type: "ephemeral" };
|
||||
return true;
|
||||
}
|
||||
|
||||
function findLastCacheableToolIndex(tools: Array<Record<string, unknown>>) {
|
||||
for (let i = tools.length - 1; i >= 0; i--) {
|
||||
if (!tools[i].defer_loading) {
|
||||
return i;
|
||||
}
|
||||
}
|
||||
return -1;
|
||||
}
|
||||
|
||||
function cloneValue<T>(value: T): T {
|
||||
if (typeof structuredClone === "function") {
|
||||
return structuredClone(value);
|
||||
}
|
||||
return JSON.parse(JSON.stringify(value)) as T;
|
||||
}
|
||||
|
||||
function contentToText(content: unknown): string {
|
||||
if (typeof content === "string") {
|
||||
return content.trim();
|
||||
|
||||
@@ -99,7 +99,10 @@ async function validateResponseQuality(
|
||||
if (json?.output || json?.result || json?.data || json?.response) return { valid: true };
|
||||
if (json?.error) {
|
||||
const err = json.error as Record<string, unknown>;
|
||||
return { valid: false, reason: `upstream error in 200 body: ${err?.message || JSON.stringify(json.error).substring(0, 200)}` };
|
||||
return {
|
||||
valid: false,
|
||||
reason: `upstream error in 200 body: ${err?.message || JSON.stringify(json.error).substring(0, 200)}`,
|
||||
};
|
||||
}
|
||||
return { valid: true };
|
||||
}
|
||||
@@ -809,6 +812,16 @@ export async function handleComboChat({
|
||||
const modePack =
|
||||
typeof autoConfigSource.modePack === "string" ? autoConfigSource.modePack : undefined;
|
||||
|
||||
// Retrieve last known good provider (LKGP) for this combo/model (#919)
|
||||
let lastKnownGoodProvider: string | undefined;
|
||||
try {
|
||||
const { getLKGP } = await import("../../src/lib/localDb");
|
||||
const lkgp = await getLKGP(combo.name, combo.id || combo.name);
|
||||
if (lkgp) lastKnownGoodProvider = lkgp;
|
||||
} catch {
|
||||
/* ignore db errors */
|
||||
}
|
||||
|
||||
const candidates = await buildAutoCandidates(eligibleModels, combo.name);
|
||||
if (candidates.length > 0) {
|
||||
let selectedProvider = null;
|
||||
@@ -819,7 +832,7 @@ export async function handleComboChat({
|
||||
try {
|
||||
const decision = selectWithStrategy(
|
||||
candidates,
|
||||
{ taskType, requestHasTools },
|
||||
{ taskType, requestHasTools, lastKnownGoodProvider },
|
||||
routingStrategy
|
||||
);
|
||||
selectedProvider = decision.provider;
|
||||
@@ -980,6 +993,14 @@ export async function handleComboChat({
|
||||
fallbackCount,
|
||||
strategy,
|
||||
});
|
||||
|
||||
// Record last known good provider (LKGP) for this combo/model (#919)
|
||||
if (provider) {
|
||||
import("../../src/lib/localDb")
|
||||
.then(({ setLKGP }) => setLKGP(combo.name, combo.id || combo.name, provider))
|
||||
.catch(() => {});
|
||||
}
|
||||
|
||||
return result;
|
||||
}
|
||||
|
||||
|
||||
@@ -200,6 +200,11 @@ function getLimiterKey(provider, connectionId, model = null) {
|
||||
if (provider === "codex" && model) {
|
||||
return `${provider}:${getCodexRateLimitKey(connectionId, model)}`;
|
||||
}
|
||||
// Gemini AI Studio has per-model quotas — use model-scoped limiter keys
|
||||
// so a 429 on one model doesn't pause requests for other models.
|
||||
if (provider === "gemini" && model) {
|
||||
return `${provider}:${connectionId}:${model}`;
|
||||
}
|
||||
return `${provider}:${connectionId}`;
|
||||
}
|
||||
|
||||
@@ -570,7 +575,7 @@ export function updateFromResponseBody(provider, connectionId, responseBody, sta
|
||||
const { retryAfterMs, reason } = parseRetryAfterFromBody(responseBody);
|
||||
|
||||
if (retryAfterMs && retryAfterMs > 0) {
|
||||
const limiter = getLimiter(provider, connectionId, null);
|
||||
const limiter = getLimiter(provider, connectionId, model);
|
||||
console.log(
|
||||
`🚫 [RATE-LIMIT] ${provider}:${connectionId.slice(0, 8)} — body-parsed retry: ${Math.ceil(retryAfterMs / 1000)}s (${reason})`
|
||||
);
|
||||
|
||||
@@ -84,7 +84,7 @@ export function convertOpenAIContentToParts(content) {
|
||||
const mimeType = mimePart.split(";")[0];
|
||||
|
||||
parts.push({
|
||||
inlineData: { mime_type: mimeType, data: data },
|
||||
inlineData: { mimeType, data },
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
@@ -147,6 +147,23 @@ export function claudeToGeminiRequest(model, body, stream) {
|
||||
if (parts.length > 0) {
|
||||
// Map Claude roles to Gemini roles
|
||||
const geminiRole = msg.role === "assistant" ? "model" : "user";
|
||||
|
||||
// Gemini 3+ requires thoughtSignature as a sibling part in model content
|
||||
// that contains functionCall parts. Inject if not already present from
|
||||
// a thinking block. (#927)
|
||||
if (geminiRole === "model") {
|
||||
const hasFunctionCall = parts.some((p) => p.functionCall);
|
||||
const hasSignature = parts.some((p) => p.thoughtSignature);
|
||||
if (hasFunctionCall && !hasSignature) {
|
||||
// Insert before the first functionCall part
|
||||
const fcIndex = parts.findIndex((p) => p.functionCall);
|
||||
parts.splice(fcIndex, 0, {
|
||||
thoughtSignature: DEFAULT_THINKING_GEMINI_SIGNATURE,
|
||||
text: "",
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
result.contents.push({ role: geminiRole, parts });
|
||||
}
|
||||
}
|
||||
@@ -175,7 +192,7 @@ export function claudeToGeminiRequest(model, body, stream) {
|
||||
if (body.thinking?.type === "enabled" && body.thinking.budget_tokens) {
|
||||
result.generationConfig.thinkingConfig = {
|
||||
thinkingBudget: body.thinking.budget_tokens,
|
||||
include_thoughts: true,
|
||||
includeThoughts: true,
|
||||
};
|
||||
}
|
||||
|
||||
|
||||
@@ -92,11 +92,13 @@ function convertGeminiContent(content) {
|
||||
parts.push({ type: "text", text: part.text });
|
||||
}
|
||||
|
||||
if (part.inlineData) {
|
||||
if (part.inlineData || part.inline_data) {
|
||||
const data = part.inlineData || part.inline_data;
|
||||
const mimeType = data.mimeType || data.mime_type || "image/png";
|
||||
parts.push({
|
||||
type: "image_url",
|
||||
image_url: {
|
||||
url: `data:${part.inlineData.mimeType};base64,${part.inlineData.data}`,
|
||||
url: `data:${mimeType};base64,${data.data}`,
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
@@ -33,7 +33,7 @@ type GeminiGenerationConfig = {
|
||||
maxOutputTokens?: unknown;
|
||||
thinkingConfig?: {
|
||||
thinkingBudget: number;
|
||||
include_thoughts: boolean;
|
||||
includeThoughts: boolean;
|
||||
};
|
||||
responseMimeType?: string;
|
||||
responseSchema?: unknown;
|
||||
@@ -156,7 +156,6 @@ function openaiToGeminiBase(model, body, stream) {
|
||||
});
|
||||
parts.push({
|
||||
thoughtSignature: DEFAULT_THINKING_GEMINI_SIGNATURE,
|
||||
text: "",
|
||||
});
|
||||
}
|
||||
|
||||
@@ -168,14 +167,24 @@ function openaiToGeminiBase(model, body, stream) {
|
||||
}
|
||||
|
||||
if (msg.tool_calls && Array.isArray(msg.tool_calls)) {
|
||||
// Gemini 3+ requires thoughtSignature as a sibling part in model content
|
||||
// that contains functionCall parts. If no reasoning_content was present
|
||||
// (which already injects the signature above), inject one now. (#927)
|
||||
const hasSignatureAlready = parts.some((p) => p.thoughtSignature);
|
||||
if (!hasSignatureAlready) {
|
||||
parts.push({
|
||||
thoughtSignature: DEFAULT_THINKING_GEMINI_SIGNATURE,
|
||||
});
|
||||
}
|
||||
|
||||
const toolCallIds = [];
|
||||
for (const tc of msg.tool_calls) {
|
||||
if (tc.type !== "function") continue;
|
||||
|
||||
const args = tryParseJSON(tc.function?.arguments || "{}");
|
||||
// Do NOT include thoughtSignature on functionCall parts — it is only valid
|
||||
// on thinking/reasoning parts and causes HTTP 400 "invalid argument" from the
|
||||
// Gemini API when present on a functionCall part (#725).
|
||||
// Do NOT include thoughtSignature ON the functionCall part itself — it is
|
||||
// only valid as a separate sibling part. Including it inside functionCall
|
||||
// causes HTTP 400 "invalid argument" (#725).
|
||||
parts.push({
|
||||
functionCall: {
|
||||
id: tc.id,
|
||||
@@ -306,7 +315,7 @@ export function openaiToGeminiCLIRequest(model, body, stream) {
|
||||
const budget = budgetMap[body.reasoning_effort] || getDefaultThinkingBudget(model) || 8192;
|
||||
gemini.generationConfig.thinkingConfig = {
|
||||
thinkingBudget: budget,
|
||||
include_thoughts: true,
|
||||
includeThoughts: true,
|
||||
};
|
||||
}
|
||||
|
||||
@@ -314,7 +323,7 @@ export function openaiToGeminiCLIRequest(model, body, stream) {
|
||||
if (body.thinking?.type === "enabled" && body.thinking.budget_tokens) {
|
||||
gemini.generationConfig.thinkingConfig = {
|
||||
thinkingBudget: body.thinking.budget_tokens,
|
||||
include_thoughts: true,
|
||||
includeThoughts: true,
|
||||
};
|
||||
}
|
||||
|
||||
@@ -435,7 +444,7 @@ function wrapInCloudCodeEnvelopeForClaude(model, claudeRequest, credentials = nu
|
||||
} else if (block.type === "image" && block.source) {
|
||||
parts.push({
|
||||
inlineData: {
|
||||
mime_type: block.source.media_type,
|
||||
mimeType: block.source.media_type,
|
||||
data: block.source.data,
|
||||
},
|
||||
});
|
||||
|
||||
@@ -171,6 +171,11 @@ export function geminiToClaudeResponse(chunk, state) {
|
||||
stopReason = "tool_use";
|
||||
} else if (reason === "max_tokens" || reason === "length") {
|
||||
stopReason = "max_tokens";
|
||||
} else if (reason === "safety" || reason === "recitation" || reason === "blocklist") {
|
||||
// Content blocked by Gemini safety. Any text streamed before this finish
|
||||
// reason has already been emitted to the client — this is unavoidable in
|
||||
// SSE streaming. Map to end_turn (Claude has no "content blocked" reason).
|
||||
stopReason = "end_turn";
|
||||
} else {
|
||||
stopReason = "end_turn";
|
||||
}
|
||||
|
||||
@@ -225,6 +225,11 @@ export function geminiToOpenAIResponse(chunk, state) {
|
||||
if (finishReason === "stop" && state.toolCalls.size > 0) {
|
||||
finishReason = "tool_calls";
|
||||
}
|
||||
// Content blocked by Gemini safety filters — pass through as "content_filter"
|
||||
// so downstream clients can distinguish from normal completion.
|
||||
if (finishReason === "safety" || finishReason === "recitation" || finishReason === "blocklist") {
|
||||
finishReason = "content_filter";
|
||||
}
|
||||
|
||||
const finalChunk: Record<string, unknown> = {
|
||||
id: `chatcmpl-${state.messageId}`,
|
||||
|
||||
@@ -585,9 +585,9 @@ export function createSSEStream(options: StreamOptions = {}) {
|
||||
// Content for call log is accumulated only from parsed (above) to avoid double-counting;
|
||||
// do not add again from item here.
|
||||
|
||||
// #723, #727: Sanitize intermediate stream chunks if target is OpenAI format loop
|
||||
// #723, #727: Sanitize intermediate stream chunks if source (client) is OpenAI format
|
||||
let itemSanitized: Record<string, unknown> = item;
|
||||
if (targetFormat === FORMATS.OPENAI || targetFormat === FORMATS.OPENAI_RESPONSES) {
|
||||
if (sourceFormat === FORMATS.OPENAI || sourceFormat === FORMATS.OPENAI_RESPONSES) {
|
||||
itemSanitized = sanitizeStreamingChunk(itemSanitized) as Record<string, unknown>;
|
||||
|
||||
// Extract reasoning tags from content if translation generated them
|
||||
|
||||
6
package-lock.json
generated
6
package-lock.json
generated
@@ -1,12 +1,12 @@
|
||||
{
|
||||
"name": "omniroute",
|
||||
"version": "3.4.5",
|
||||
"version": "3.4.7",
|
||||
"lockfileVersion": 3,
|
||||
"requires": true,
|
||||
"packages": {
|
||||
"": {
|
||||
"name": "omniroute",
|
||||
"version": "3.4.5",
|
||||
"version": "3.4.7",
|
||||
"hasInstallScript": true,
|
||||
"license": "MIT",
|
||||
"workspaces": [
|
||||
@@ -21047,7 +21047,7 @@
|
||||
},
|
||||
"open-sse": {
|
||||
"name": "@omniroute/open-sse",
|
||||
"version": "3.4.4"
|
||||
"version": "3.4.7"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "omniroute",
|
||||
"version": "3.4.5",
|
||||
"version": "3.4.7",
|
||||
"description": "Smart AI Router with auto fallback — route to FREE & cheap models, zero downtime. Works with Cursor, Cline, Claude Desktop, Codex, and any OpenAI-compatible tool.",
|
||||
"type": "module",
|
||||
"bin": {
|
||||
|
||||
@@ -254,6 +254,25 @@ if (existsSync(mitmSrc)) {
|
||||
}
|
||||
}
|
||||
|
||||
// ── Step 8.5: Bundle MCP server ────────────────────────────
|
||||
const mcpSrcFile = join(ROOT, "open-sse", "mcp-server", "server.ts");
|
||||
const mcpDestDir = join(APP_DIR, "open-sse", "mcp-server");
|
||||
const mcpDestFile = join(mcpDestDir, "server.js");
|
||||
|
||||
if (existsSync(mcpSrcFile)) {
|
||||
console.log(" 🔨 Bundling MCP Server (TypeScript → JavaScript)...");
|
||||
mkdirSync(mcpDestDir, { recursive: true });
|
||||
try {
|
||||
execSync(
|
||||
`npx esbuild ${JSON.stringify(mcpSrcFile)} --bundle --platform=node --packages=external --format=esm --outfile=${JSON.stringify(mcpDestFile)}`,
|
||||
{ cwd: ROOT, stdio: "inherit" }
|
||||
);
|
||||
console.log(" ✅ MCP Server bundled to app/open-sse/mcp-server/server.js");
|
||||
} catch (err) {
|
||||
console.warn(" ⚠️ MCP Server bundle error:", err.message);
|
||||
}
|
||||
}
|
||||
|
||||
// ── Step 9: Copy shared utilities needed at runtime ────────
|
||||
const sharedApiKey = join(ROOT, "src", "shared", "utils", "apiKey.js");
|
||||
const sharedApiKeyDest = join(APP_DIR, "src", "shared", "utils");
|
||||
|
||||
@@ -5,9 +5,10 @@ import { useTranslations } from "next-intl";
|
||||
export interface CacheTrendPoint {
|
||||
timestamp: string;
|
||||
requests: number;
|
||||
hits: number;
|
||||
misses: number;
|
||||
hitRate: number;
|
||||
cachedRequests: number;
|
||||
inputTokens: number;
|
||||
cachedTokens: number;
|
||||
cacheCreationTokens: number;
|
||||
}
|
||||
|
||||
interface CacheTrendsProps {
|
||||
@@ -27,7 +28,8 @@ export default function CacheTrends({
|
||||
|
||||
const trendData: CacheTrendPoint[] = data ?? [];
|
||||
const maxRequests = trendData.length > 0 ? Math.max(...trendData.map((p) => p.requests), 1) : 1;
|
||||
const peakHitRate = trendData.length > 0 ? Math.max(...trendData.map((p) => p.hitRate)) : null;
|
||||
const maxCachedRequests =
|
||||
trendData.length > 0 ? Math.max(...trendData.map((p) => p.cachedRequests), 0) : 0;
|
||||
|
||||
return (
|
||||
<div
|
||||
@@ -69,17 +71,21 @@ export default function CacheTrends({
|
||||
</div>
|
||||
) : (
|
||||
<>
|
||||
{peakHitRate !== null && (
|
||||
{maxCachedRequests > 0 && (
|
||||
<div className="text-xs text-text-muted">
|
||||
Peak hit rate:{" "}
|
||||
<span className="font-medium text-foreground">{peakHitRate.toFixed(1)}</span>
|
||||
{t("peakCached")}:{" "}
|
||||
<span className="font-medium text-foreground">
|
||||
{maxCachedRequests} / {maxRequests}
|
||||
</span>
|
||||
</div>
|
||||
)}
|
||||
<div className="flex items-end gap-1 h-32">
|
||||
{trendData.map((point) => {
|
||||
const height = Math.max(4, (point.requests / maxRequests) * 100);
|
||||
const hitHeight =
|
||||
point.requests > 0 ? Math.max(2, (point.hits / point.requests) * height) : 0;
|
||||
const cachedHeight =
|
||||
point.requests > 0
|
||||
? Math.max(2, (point.cachedRequests / point.requests) * height)
|
||||
: 0;
|
||||
const hour = new Date(point.timestamp).toLocaleTimeString([], {
|
||||
hour: "2-digit",
|
||||
minute: "2-digit",
|
||||
@@ -92,16 +98,17 @@ export default function CacheTrends({
|
||||
className="flex-1 flex flex-col items-center gap-1 group relative"
|
||||
>
|
||||
<div className="absolute bottom-full mb-1 hidden group-hover:block bg-surface-raised border border-border rounded px-2 py-1 text-xs whitespace-nowrap z-10">
|
||||
{hour}: {point.requests} requests, {point.hits} cached
|
||||
{hour}: {point.requests} {t("requests").toLowerCase()}, {point.cachedRequests}{" "}
|
||||
{t("cached").toLowerCase()}
|
||||
</div>
|
||||
<div className="w-full flex flex-col justify-end h-full gap-px">
|
||||
<div
|
||||
className="w-full bg-green-500/30 rounded-t"
|
||||
style={{ height: `${hitHeight}%` }}
|
||||
style={{ height: `${cachedHeight}%` }}
|
||||
/>
|
||||
<div
|
||||
className="w-full bg-text-muted/20 rounded-t"
|
||||
style={{ height: `${height - hitHeight}%` }}
|
||||
style={{ height: `${height - cachedHeight}%` }}
|
||||
/>
|
||||
</div>
|
||||
<span className="text-[10px] text-text-muted truncate w-full text-center">
|
||||
|
||||
3
src/app/(dashboard)/dashboard/cache/page.tsx
vendored
3
src/app/(dashboard)/dashboard/cache/page.tsx
vendored
@@ -435,9 +435,6 @@ export default function CachePage() {
|
||||
</Card>
|
||||
)}
|
||||
|
||||
{/* Prompt Cache Metrics (cumulative with reset) */}
|
||||
<CacheStatsCard />
|
||||
|
||||
{/* Cache Trend (24h) */}
|
||||
{trend.length > 0 && (
|
||||
<Card>
|
||||
|
||||
@@ -108,32 +108,32 @@ export default function MemoryPage() {
|
||||
return (
|
||||
<div className="space-y-6 p-6">
|
||||
<div className="flex items-center justify-between">
|
||||
<h1 className="text-2xl font-bold">Memory Management</h1>
|
||||
<h1 className="text-2xl font-bold">{t("title")}</h1>
|
||||
<div className="flex gap-2">
|
||||
<Button variant="outline" onClick={handleExport}>
|
||||
Export
|
||||
{t("export")}
|
||||
</Button>
|
||||
<Button variant="outline">Import</Button>
|
||||
<Button>Add Memory</Button>
|
||||
<Button variant="outline">{t("import")}</Button>
|
||||
<Button>{t("addMemory")}</Button>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div className="grid grid-cols-1 md:grid-cols-3 gap-4">
|
||||
<Card>
|
||||
<div className="p-4">
|
||||
<div className="text-sm text-gray-500">Total Entries</div>
|
||||
<div className="text-sm text-gray-500">{t("totalEntries")}</div>
|
||||
<div className="text-2xl font-bold">{stats.totalEntries}</div>
|
||||
</div>
|
||||
</Card>
|
||||
<Card>
|
||||
<div className="p-4">
|
||||
<div className="text-sm text-gray-500">Tokens Used</div>
|
||||
<div className="text-sm text-gray-500">{t("tokensUsed")}</div>
|
||||
<div className="text-2xl font-bold">{stats.tokensUsed.toLocaleString()}</div>
|
||||
</div>
|
||||
</Card>
|
||||
<Card>
|
||||
<div className="p-4">
|
||||
<div className="text-sm text-gray-500">Hit Rate</div>
|
||||
<div className="text-sm text-gray-500">{t("hitRate")}</div>
|
||||
<div className="text-2xl font-bold">{(stats.hitRate * 100).toFixed(1)}%</div>
|
||||
</div>
|
||||
</Card>
|
||||
@@ -142,20 +142,20 @@ export default function MemoryPage() {
|
||||
<Card>
|
||||
<div className="p-4">
|
||||
<div className="flex items-center justify-between mb-4">
|
||||
<h2 className="text-lg font-semibold">Memories</h2>
|
||||
<h2 className="text-lg font-semibold">{t("memories")}</h2>
|
||||
<div className="flex gap-2">
|
||||
<Input
|
||||
placeholder="Search memories..."
|
||||
placeholder={t("search")}
|
||||
value={searchQuery}
|
||||
onChange={(e) => setSearchQuery(e.target.value)}
|
||||
className="w-64"
|
||||
/>
|
||||
<Select value={filterType} onChange={(e) => setFilterType(e.target.value)}>
|
||||
<option value="all">All Types</option>
|
||||
<option value="factual">Factual</option>
|
||||
<option value="episodic">Episodic</option>
|
||||
<option value="procedural">Procedural</option>
|
||||
<option value="semantic">Semantic</option>
|
||||
<option value="all">{t("allTypes")}</option>
|
||||
<option value="factual">{t("factual")}</option>
|
||||
<option value="episodic">{t("episodic")}</option>
|
||||
<option value="procedural">{t("procedural")}</option>
|
||||
<option value="semantic">{t("semantic")}</option>
|
||||
</Select>
|
||||
</div>
|
||||
</div>
|
||||
@@ -164,11 +164,11 @@ export default function MemoryPage() {
|
||||
<table className="w-full">
|
||||
<thead>
|
||||
<tr className="border-b">
|
||||
<th className="text-left py-2 px-4">Type</th>
|
||||
<th className="text-left py-2 px-4">Key</th>
|
||||
<th className="text-left py-2 px-4">Content</th>
|
||||
<th className="text-left py-2 px-4">Created</th>
|
||||
<th className="text-left py-2 px-4">Actions</th>
|
||||
<th className="text-left py-2 px-4">{t("type")}</th>
|
||||
<th className="text-left py-2 px-4">{t("key")}</th>
|
||||
<th className="text-left py-2 px-4">{t("content")}</th>
|
||||
<th className="text-left py-2 px-4">{t("created")}</th>
|
||||
<th className="text-left py-2 px-4">{t("actions")}</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
@@ -182,7 +182,7 @@ export default function MemoryPage() {
|
||||
<td className="py-2 px-4">{new Date(memory.createdAt).toLocaleDateString()}</td>
|
||||
<td className="py-2 px-4">
|
||||
<Button variant="ghost" size="sm" onClick={() => handleDelete(memory.id)}>
|
||||
Delete
|
||||
{t("delete")}
|
||||
</Button>
|
||||
</td>
|
||||
</tr>
|
||||
|
||||
@@ -840,6 +840,7 @@ export default function ProviderDetailPage() {
|
||||
customModels: CompatModelRow[];
|
||||
modelCompatOverrides: Array<CompatModelRow & { id: string }>;
|
||||
}>({ customModels: [], modelCompatOverrides: [] });
|
||||
const [syncedAvailableModels, setSyncedAvailableModels] = useState<any[]>([]);
|
||||
const [compatSavingModelId, setCompatSavingModelId] = useState<string | null>(null);
|
||||
const [applyingCodexAuthId, setApplyingCodexAuthId] = useState<string | null>(null);
|
||||
const [exportingCodexAuthId, setExportingCodexAuthId] = useState<string | null>(null);
|
||||
@@ -881,7 +882,11 @@ export default function ProviderDetailPage() {
|
||||
!!(FREE_PROVIDERS as any)[providerId] || !!(OAUTH_PROVIDERS as any)[providerId];
|
||||
const providerSupportsPat = supportsApiKeyOnFreeProvider(providerId);
|
||||
const isOAuth = providerSupportsOAuth && !providerSupportsPat;
|
||||
const models = getModelsByProviderId(providerId);
|
||||
const registryModels = getModelsByProviderId(providerId);
|
||||
// For Gemini: always use synced API models (empty if no keys added yet)
|
||||
const models = providerId === "gemini"
|
||||
? syncedAvailableModels
|
||||
: registryModels;
|
||||
const providerAlias = getProviderAlias(providerId);
|
||||
const isManagedAvailableModelsProvider = isCompatible || providerId === "openrouter";
|
||||
const isSearchProvider = providerId.endsWith("-search");
|
||||
@@ -915,6 +920,20 @@ export default function ProviderDetailPage() {
|
||||
customModels: data.models || [],
|
||||
modelCompatOverrides: data.modelCompatOverrides || [],
|
||||
});
|
||||
// Fetch synced available models for Gemini
|
||||
if (providerId === "gemini") {
|
||||
try {
|
||||
const syncRes = await fetch("/api/synced-available-models?provider=gemini", {
|
||||
cache: "no-store",
|
||||
});
|
||||
if (syncRes.ok) {
|
||||
const syncData = await syncRes.json();
|
||||
setSyncedAvailableModels(syncData.models || []);
|
||||
}
|
||||
} catch {
|
||||
// Non-critical
|
||||
}
|
||||
}
|
||||
} catch (e) {
|
||||
console.error("fetchProviderModelMeta", e);
|
||||
}
|
||||
@@ -1060,6 +1079,10 @@ export default function ProviderDetailPage() {
|
||||
const res = await fetch(`/api/providers/${id}`, { method: "DELETE" });
|
||||
if (res.ok) {
|
||||
setConnections(connections.filter((c) => c.id !== id));
|
||||
// Refresh model list after connection deletion (synced models may change)
|
||||
if (providerId === "gemini") {
|
||||
await fetchProviderModelMeta();
|
||||
}
|
||||
}
|
||||
} catch (error) {
|
||||
console.log("Error deleting connection:", error);
|
||||
@@ -1087,8 +1110,72 @@ export default function ProviderDetailPage() {
|
||||
body: JSON.stringify({ provider: providerId, ...formData }),
|
||||
});
|
||||
if (res.ok) {
|
||||
const connectionData = await res.json();
|
||||
const newConnection = connectionData?.connection;
|
||||
await fetchConnections();
|
||||
setShowAddApiKeyModal(false);
|
||||
|
||||
// For Gemini: show progress dialog and sync models from endpoint
|
||||
if (providerId === "gemini" && newConnection?.id) {
|
||||
setShowImportModal(true);
|
||||
setImportProgress({
|
||||
current: 0,
|
||||
total: 0,
|
||||
phase: "fetching",
|
||||
status: t("fetchingModels"),
|
||||
logs: [],
|
||||
error: "",
|
||||
importedCount: 0,
|
||||
});
|
||||
|
||||
try {
|
||||
const syncRes = await fetch(`/api/providers/${newConnection.id}/sync-models`, {
|
||||
method: "POST",
|
||||
signal: AbortSignal.timeout(30_000), // 30s timeout — model sync shouldn't hang
|
||||
});
|
||||
const syncData = await syncRes.json();
|
||||
|
||||
if (!syncRes.ok || syncData.error) {
|
||||
setImportProgress((prev) => ({
|
||||
...prev,
|
||||
phase: "error",
|
||||
status: t("failedFetchModels"),
|
||||
error: syncData.error?.message || syncData.error || t("failedImportModels"),
|
||||
}));
|
||||
return null;
|
||||
}
|
||||
|
||||
const syncedCount = syncData.syncedModels || 0;
|
||||
const syncedModelList: Array<{ id: string; name?: string }> = syncData.models || [];
|
||||
const logs: string[] = [];
|
||||
if (syncedModelList.length > 0) {
|
||||
logs.push(`✓ ${syncedCount} models available`);
|
||||
logs.push("");
|
||||
for (const m of syncedModelList) {
|
||||
logs.push(` ${m.name || m.id}`);
|
||||
}
|
||||
}
|
||||
|
||||
setImportProgress((prev) => ({
|
||||
...prev,
|
||||
phase: "done",
|
||||
status: t("modelsImported", { count: syncedCount }),
|
||||
total: syncedCount,
|
||||
current: syncedCount,
|
||||
importedCount: syncedCount,
|
||||
logs,
|
||||
}));
|
||||
|
||||
await fetchProviderModelMeta();
|
||||
} catch (syncError) {
|
||||
setImportProgress((prev) => ({
|
||||
...prev,
|
||||
phase: "error",
|
||||
status: t("failedFetchModels"),
|
||||
error: String(syncError),
|
||||
}));
|
||||
}
|
||||
}
|
||||
return null;
|
||||
}
|
||||
const data = await res.json().catch(() => ({}));
|
||||
@@ -1963,7 +2050,7 @@ export default function ProviderDetailPage() {
|
||||
);
|
||||
}
|
||||
|
||||
const importButton = (
|
||||
const importButton = providerId === "gemini" ? null : (
|
||||
<div className="flex items-center gap-2 mb-4">
|
||||
<Button
|
||||
size="sm"
|
||||
@@ -2469,7 +2556,7 @@ export default function ProviderDetailPage() {
|
||||
{renderModelsSection()}
|
||||
|
||||
{/* Custom Models — available for providers without managed available-model metadata */}
|
||||
{!isManagedAvailableModelsProvider && (
|
||||
{!isManagedAvailableModelsProvider && providerId !== "gemini" && (
|
||||
<CustomModelsSection
|
||||
providerId={providerId}
|
||||
providerAlias={providerDisplayAlias}
|
||||
@@ -2758,11 +2845,16 @@ export default function ProviderDetailPage() {
|
||||
</div>
|
||||
)}
|
||||
|
||||
{/* Auto-reload notice */}
|
||||
{importProgress.phase === "done" && importProgress.importedCount > 0 && (
|
||||
<p className="text-xs text-text-muted text-center animate-pulse">
|
||||
{t("pageAutoRefresh")}
|
||||
</p>
|
||||
{/* Close button */}
|
||||
{importProgress.phase === "done" && (
|
||||
<div className="flex justify-center">
|
||||
<button
|
||||
onClick={() => setShowImportModal(false)}
|
||||
className="px-4 py-2 text-sm font-medium rounded-lg bg-primary text-white hover:opacity-90 transition-opacity"
|
||||
>
|
||||
{t("close") || "Close"}
|
||||
</button>
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
</Modal>
|
||||
|
||||
@@ -67,6 +67,13 @@ export async function GET() {
|
||||
dedup: {
|
||||
inflightRequests: getInflightCount(),
|
||||
},
|
||||
cryptography: {
|
||||
status:
|
||||
process.env.STORAGE_ENCRYPTION_KEY && process.env.STORAGE_ENCRYPTION_KEY.length >= 32
|
||||
? "healthy"
|
||||
: "missing_or_invalid",
|
||||
provider: "aes-256-gcm",
|
||||
},
|
||||
setupComplete: settings?.setupComplete || false,
|
||||
});
|
||||
} catch (error) {
|
||||
|
||||
@@ -130,16 +130,56 @@ const PROVIDER_MODELS_CONFIG: Record<string, ProviderModelsConfigEntry> = {
|
||||
parseResponse: (data) => data.data || [],
|
||||
},
|
||||
gemini: {
|
||||
url: "https://generativelanguage.googleapis.com/v1beta/models",
|
||||
url: "https://generativelanguage.googleapis.com/v1beta/models?pageSize=1000",
|
||||
method: "GET",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
authQuery: "key", // Use query param for API key
|
||||
parseResponse: (data) =>
|
||||
(data.models || []).map((m) => ({
|
||||
...m,
|
||||
id: (m.name || m.id || "").replace(/^models\//, ""),
|
||||
name: m.displayName || (m.name || "").replace(/^models\//, ""),
|
||||
})),
|
||||
parseResponse: (data) => {
|
||||
const METHOD_TO_ENDPOINT: Record<string, string> = {
|
||||
generateContent: "chat",
|
||||
embedContent: "embeddings",
|
||||
predict: "images",
|
||||
predictLongRunning: "images",
|
||||
bidiGenerateContent: "audio",
|
||||
generateAnswer: "chat",
|
||||
};
|
||||
const IGNORED_METHODS = new Set([
|
||||
"countTokens",
|
||||
"countTextTokens",
|
||||
"createCachedContent",
|
||||
"batchGenerateContent",
|
||||
"asyncBatchEmbedContent",
|
||||
]);
|
||||
|
||||
return (data.models || []).map((m: Record<string, unknown>) => {
|
||||
const methods: string[] = Array.isArray(m.supportedGenerationMethods)
|
||||
? m.supportedGenerationMethods
|
||||
: [];
|
||||
const endpoints = [
|
||||
...new Set(
|
||||
methods
|
||||
.filter((method) => !IGNORED_METHODS.has(method))
|
||||
.map((method) => METHOD_TO_ENDPOINT[method] || "chat")
|
||||
),
|
||||
];
|
||||
if (endpoints.length === 0) endpoints.push("chat");
|
||||
|
||||
return {
|
||||
...m,
|
||||
id: ((m.name as string) || (m.id as string) || "").replace(/^models\//, ""),
|
||||
name: (m.displayName as string) || ((m.name as string) || "").replace(/^models\//, ""),
|
||||
supportedEndpoints: endpoints,
|
||||
...(typeof m.inputTokenLimit === "number"
|
||||
? { inputTokenLimit: m.inputTokenLimit }
|
||||
: {}),
|
||||
...(typeof m.outputTokenLimit === "number"
|
||||
? { outputTokenLimit: m.outputTokenLimit }
|
||||
: {}),
|
||||
...(typeof m.description === "string" ? { description: m.description } : {}),
|
||||
...(m.thinking === true ? { supportsThinking: true } : {}),
|
||||
};
|
||||
});
|
||||
},
|
||||
},
|
||||
// gemini-cli handled via retrieveUserQuota (see GET handler)
|
||||
qwen: {
|
||||
@@ -648,7 +688,7 @@ export async function GET(
|
||||
// Build request URL
|
||||
let url = config.url;
|
||||
if (config.authQuery) {
|
||||
url += `?${config.authQuery}=${token}`;
|
||||
url += `${url.includes("?") ? "&" : "?"}${config.authQuery}=${token}`;
|
||||
}
|
||||
|
||||
// Build headers
|
||||
@@ -657,7 +697,7 @@ export async function GET(
|
||||
headers[config.authHeader] = (config.authPrefix || "") + token;
|
||||
}
|
||||
|
||||
// Make request
|
||||
// Make request (with pagination for providers that use nextPageToken, e.g. Gemini)
|
||||
const fetchOptions: any = {
|
||||
method: config.method,
|
||||
headers,
|
||||
@@ -667,24 +707,53 @@ export async function GET(
|
||||
fetchOptions.body = JSON.stringify(config.body);
|
||||
}
|
||||
|
||||
const response = await fetch(url, fetchOptions);
|
||||
let allModels: any[] = [];
|
||||
let pageUrl = url;
|
||||
let pageCount = 0;
|
||||
const MAX_PAGES = 20; // Safety limit
|
||||
const seenTokens = new Set<string>();
|
||||
|
||||
if (!response.ok) {
|
||||
const errorText = await response.text();
|
||||
console.log(`Error fetching models from ${provider}:`, errorText);
|
||||
return NextResponse.json(
|
||||
{ error: `Failed to fetch models: ${response.status}` },
|
||||
{ status: response.status }
|
||||
);
|
||||
while (pageUrl && pageCount < MAX_PAGES) {
|
||||
pageCount++;
|
||||
const response = await fetch(pageUrl, {
|
||||
...fetchOptions,
|
||||
signal: AbortSignal.timeout(15_000),
|
||||
});
|
||||
|
||||
if (!response.ok) {
|
||||
const errorText = await response.text();
|
||||
console.log(`Error fetching models from ${provider}:`, errorText);
|
||||
return NextResponse.json(
|
||||
{ error: `Failed to fetch models: ${response.status}` },
|
||||
{ status: response.status }
|
||||
);
|
||||
}
|
||||
|
||||
const data = await response.json();
|
||||
const pageModels = config.parseResponse(data);
|
||||
allModels = allModels.concat(pageModels);
|
||||
|
||||
const nextPageToken = data.nextPageToken;
|
||||
if (!nextPageToken) break;
|
||||
if (seenTokens.has(nextPageToken)) {
|
||||
console.warn(`[models] ${provider}: duplicate nextPageToken detected, stopping pagination`);
|
||||
break;
|
||||
}
|
||||
seenTokens.add(nextPageToken);
|
||||
pageUrl = `${config.url}${config.url.includes("?") ? "&" : "?"}pageToken=${encodeURIComponent(nextPageToken)}`;
|
||||
if (config.authQuery) {
|
||||
pageUrl += `&${config.authQuery}=${token}`;
|
||||
}
|
||||
}
|
||||
|
||||
const data = await response.json();
|
||||
const models = config.parseResponse(data);
|
||||
if (pageCount > 1) {
|
||||
console.log(`[models] ${provider}: fetched ${allModels.length} models across ${pageCount} pages`);
|
||||
}
|
||||
|
||||
return buildResponse({
|
||||
provider,
|
||||
connectionId,
|
||||
models,
|
||||
models: allModels,
|
||||
});
|
||||
} catch (error) {
|
||||
console.log("Error fetching provider models:", error);
|
||||
|
||||
@@ -169,11 +169,27 @@ export async function DELETE(request: Request, { params }: { params: Promise<{ i
|
||||
try {
|
||||
const { id } = await params;
|
||||
|
||||
// Fetch connection before deleting to check provider type
|
||||
const connection = await getProviderConnectionById(id);
|
||||
if (!connection) {
|
||||
return NextResponse.json({ error: "Connection not found" }, { status: 404 });
|
||||
}
|
||||
|
||||
const deleted = await deleteProviderConnection(id);
|
||||
if (!deleted) {
|
||||
return NextResponse.json({ error: "Connection not found" }, { status: 404 });
|
||||
}
|
||||
|
||||
// Clean up synced available models for this connection
|
||||
if (connection.provider === "gemini") {
|
||||
try {
|
||||
const { deleteSyncedAvailableModelsForConnection } = await import("@/lib/db/models");
|
||||
await deleteSyncedAvailableModelsForConnection("gemini", id);
|
||||
} catch (e) {
|
||||
console.error("Failed to clean up synced models for deleted gemini connection:", e);
|
||||
}
|
||||
}
|
||||
|
||||
// Auto sync to Cloud if enabled
|
||||
await syncToCloudIfEnabled();
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
import { NextResponse } from "next/server";
|
||||
import { getProviderConnectionById } from "@/models";
|
||||
import { getCustomModels, replaceCustomModels } from "@/lib/db/models";
|
||||
import { getCustomModels, replaceCustomModels, replaceSyncedAvailableModelsForConnection } from "@/lib/db/models";
|
||||
import {
|
||||
syncManagedAvailableModelAliases,
|
||||
usesManagedAvailableModels,
|
||||
@@ -188,11 +188,38 @@ export async function POST(request: Request, { params }: { params: Promise<{ id:
|
||||
id: m.id || m.name || m.model,
|
||||
name: m.name || m.displayName || m.id || m.model,
|
||||
source: "auto-sync",
|
||||
...(Array.isArray(m.supportedEndpoints) && m.supportedEndpoints.length > 0
|
||||
? { supportedEndpoints: m.supportedEndpoints }
|
||||
: {}),
|
||||
...(typeof m.inputTokenLimit === "number" ? { inputTokenLimit: m.inputTokenLimit } : {}),
|
||||
...(typeof m.outputTokenLimit === "number" ? { outputTokenLimit: m.outputTokenLimit } : {}),
|
||||
...(typeof m.description === "string" ? { description: m.description } : {}),
|
||||
...(m.supportsThinking === true ? { supportsThinking: true } : {}),
|
||||
}))
|
||||
.filter((m: any) => m.id && !registryIds.has(m.id));
|
||||
|
||||
const previousModels = await getCustomModels(logProvider);
|
||||
const replaced = await replaceCustomModels(logProvider, models);
|
||||
|
||||
// For Gemini: also write to syncedAvailableModels (unioned across API keys)
|
||||
if (logProvider === "gemini") {
|
||||
try {
|
||||
const syncedModels = models.map((m: any) => ({
|
||||
id: m.id,
|
||||
name: m.name || m.id,
|
||||
source: "api-sync" as const,
|
||||
...(m.supportedEndpoints ? { supportedEndpoints: m.supportedEndpoints } : {}),
|
||||
...(typeof m.inputTokenLimit === "number" ? { inputTokenLimit: m.inputTokenLimit } : {}),
|
||||
...(typeof m.outputTokenLimit === "number" ? { outputTokenLimit: m.outputTokenLimit } : {}),
|
||||
...(typeof m.description === "string" ? { description: m.description } : {}),
|
||||
...(m.supportsThinking === true ? { supportsThinking: true } : {}),
|
||||
}));
|
||||
await replaceSyncedAvailableModelsForConnection(logProvider, id, syncedModels);
|
||||
} catch (e) {
|
||||
console.error("Failed to union synced available models for gemini:", e);
|
||||
}
|
||||
}
|
||||
|
||||
const modelChanges = summarizeModelChanges(previousModels, replaced);
|
||||
|
||||
let syncedAliases = 0;
|
||||
|
||||
@@ -145,6 +145,8 @@ export async function POST(request: Request) {
|
||||
testStatus: testStatus || "unknown",
|
||||
});
|
||||
|
||||
// Note: Gemini model sync is now triggered client-side with progress dialog
|
||||
|
||||
// Hide sensitive fields
|
||||
const result: Record<string, any> = { ...newConnection };
|
||||
delete result.apiKey;
|
||||
|
||||
33
src/app/api/synced-available-models/route.ts
Normal file
33
src/app/api/synced-available-models/route.ts
Normal file
@@ -0,0 +1,33 @@
|
||||
import { getSyncedAvailableModels, getAllSyncedAvailableModels } from "@/lib/db/models";
|
||||
import { isAuthenticated } from "@/shared/utils/apiAuth";
|
||||
|
||||
/**
|
||||
* GET /api/synced-available-models?provider=<id>
|
||||
* List synced available models for a provider (or all providers).
|
||||
*/
|
||||
export async function GET(request: Request) {
|
||||
try {
|
||||
if (!(await isAuthenticated(request))) {
|
||||
return Response.json(
|
||||
{ error: { message: "Authentication required", type: "invalid_api_key" } },
|
||||
{ status: 401 }
|
||||
);
|
||||
}
|
||||
|
||||
const { searchParams } = new URL(request.url);
|
||||
const provider = searchParams.get("provider");
|
||||
|
||||
if (provider) {
|
||||
const models = await getSyncedAvailableModels(provider);
|
||||
return Response.json({ models });
|
||||
}
|
||||
|
||||
const allModels = await getAllSyncedAvailableModels();
|
||||
return Response.json(allModels);
|
||||
} catch {
|
||||
return Response.json(
|
||||
{ error: { message: "Failed to fetch synced available models", type: "server_error" } },
|
||||
{ status: 500 }
|
||||
);
|
||||
}
|
||||
}
|
||||
@@ -18,6 +18,8 @@ import { getAllModerationModels } from "@omniroute/open-sse/config/moderationReg
|
||||
import { getAllVideoModels } from "@omniroute/open-sse/config/videoRegistry.ts";
|
||||
import { getAllMusicModels } from "@omniroute/open-sse/config/musicRegistry.ts";
|
||||
import { REGISTRY } from "@omniroute/open-sse/config/providerRegistry.ts";
|
||||
import { getSyncedAvailableModels } from "@/lib/db/models";
|
||||
import { getCompatibleFallbackModels } from "@/lib/providers/managedAvailableModels";
|
||||
|
||||
const FALLBACK_ALIAS_TO_PROVIDER = {
|
||||
ag: "antigravity",
|
||||
@@ -179,7 +181,7 @@ export async function getUnifiedModelsResponse(
|
||||
connections = connections.filter((c) => c.isActive !== false);
|
||||
} catch (e) {
|
||||
// If database not available, show no provider models (safe default)
|
||||
console.log("Could not fetch providers, showing only combos/custom models");
|
||||
console.log("[catalog] Could not fetch providers:", e);
|
||||
}
|
||||
|
||||
// Get provider nodes (for compatible providers with custom prefixes)
|
||||
@@ -296,6 +298,61 @@ export async function getUnifiedModelsResponse(
|
||||
}
|
||||
}
|
||||
|
||||
// Gemini: synced API models exclusively (outside PROVIDER_MODELS loop since registry is empty)
|
||||
if (activeAliases.has("gemini") && !blockedProviders.has("gemini")) {
|
||||
try {
|
||||
const syncedModels = await getSyncedAvailableModels("gemini");
|
||||
for (const sm of syncedModels) {
|
||||
const aliasId = `gemini/${sm.id}`;
|
||||
if (getModelIsHidden("gemini", sm.id)) continue;
|
||||
|
||||
// Convert supportedEndpoints to type/subtype for endpoint categorization
|
||||
const endpoints = Array.isArray(sm.supportedEndpoints) ? sm.supportedEndpoints : ["chat"];
|
||||
let modelType: string | undefined;
|
||||
if (endpoints.includes("embeddings")) modelType = "embedding";
|
||||
else if (endpoints.includes("images")) modelType = "image";
|
||||
else if (endpoints.includes("audio")) modelType = "audio";
|
||||
|
||||
models.push({
|
||||
id: aliasId,
|
||||
object: "model",
|
||||
created: timestamp,
|
||||
owned_by: "gemini",
|
||||
permission: [],
|
||||
root: sm.id,
|
||||
parent: null,
|
||||
...(modelType ? { type: modelType } : {}),
|
||||
...(modelType === "audio" ? { subtype: "transcription" } : {}),
|
||||
...(sm.inputTokenLimit ? { context_length: sm.inputTokenLimit } : {}),
|
||||
...(endpoints.length > 1 || !endpoints.includes("chat")
|
||||
? { supported_endpoints: endpoints }
|
||||
: {}),
|
||||
});
|
||||
|
||||
// For audio models, also add a speech variant so they appear in both sections
|
||||
if (modelType === "audio") {
|
||||
models.push({
|
||||
id: aliasId,
|
||||
object: "model",
|
||||
created: timestamp,
|
||||
owned_by: "gemini",
|
||||
permission: [],
|
||||
root: sm.id,
|
||||
parent: null,
|
||||
type: "audio",
|
||||
subtype: "speech",
|
||||
...(sm.inputTokenLimit ? { context_length: sm.inputTokenLimit } : {}),
|
||||
...(endpoints.length > 1 || !endpoints.includes("chat")
|
||||
? { supported_endpoints: endpoints }
|
||||
: {}),
|
||||
});
|
||||
}
|
||||
}
|
||||
} catch (err) {
|
||||
console.error("[catalog] Error fetching synced Gemini models:", err);
|
||||
}
|
||||
}
|
||||
|
||||
// Helper: check if a provider is active (by provider id or alias)
|
||||
const isProviderActive = (provider: string) => {
|
||||
if (activeAliases.size === 0) return false; // No active connections = show nothing
|
||||
@@ -394,6 +451,8 @@ export async function getUnifiedModelsResponse(
|
||||
try {
|
||||
const customModelsMap = (await getAllCustomModels()) as Record<string, unknown>;
|
||||
for (const [providerId, rawProviderCustomModels] of Object.entries(customModelsMap)) {
|
||||
// Skip Gemini — handled by syncedAvailableModels above
|
||||
if (providerId === "gemini") continue;
|
||||
const providerCustomModels = Array.isArray(rawProviderCustomModels)
|
||||
? rawProviderCustomModels.filter(
|
||||
(model): model is Record<string, unknown> =>
|
||||
@@ -454,6 +513,9 @@ export async function getUnifiedModelsResponse(
|
||||
...(endpoints.length > 1 || !endpoints.includes("chat")
|
||||
? { supported_endpoints: endpoints }
|
||||
: {}),
|
||||
...(typeof (model as any).inputTokenLimit === "number"
|
||||
? { context_length: (model as any).inputTokenLimit }
|
||||
: {}),
|
||||
...(visionFields || {}),
|
||||
});
|
||||
|
||||
@@ -476,6 +538,9 @@ export async function getUnifiedModelsResponse(
|
||||
parent: aliasId,
|
||||
custom: true,
|
||||
...(modelType ? { type: modelType } : {}),
|
||||
...(typeof (model as any).inputTokenLimit === "number"
|
||||
? { context_length: (model as any).inputTokenLimit }
|
||||
: {}),
|
||||
...(providerVisionFields || {}),
|
||||
});
|
||||
}
|
||||
@@ -485,6 +550,47 @@ export async function getUnifiedModelsResponse(
|
||||
console.log("Could not fetch custom models");
|
||||
}
|
||||
|
||||
// Add managed fallback models for compatible providers that don't import a model list.
|
||||
for (const conn of connections) {
|
||||
const providerId = typeof conn.provider === "string" ? conn.provider : null;
|
||||
if (!providerId) continue;
|
||||
if (blockedProviders.has(providerId)) continue;
|
||||
|
||||
const fallbackModels = getCompatibleFallbackModels(providerId);
|
||||
if (!Array.isArray(fallbackModels) || fallbackModels.length === 0) continue;
|
||||
|
||||
const prefix = providerIdToPrefix[providerId];
|
||||
const alias = prefix || providerIdToAlias[providerId] || providerId;
|
||||
|
||||
for (const model of fallbackModels) {
|
||||
const modelId = typeof model.id === "string" ? model.id : null;
|
||||
if (!modelId) continue;
|
||||
if (getModelIsHidden(providerId, modelId)) continue;
|
||||
|
||||
const aliasId = `${alias}/${modelId}`;
|
||||
if (models.some((m) => m.id === aliasId)) continue;
|
||||
|
||||
const visionFields =
|
||||
getVisionCapabilityFields(aliasId) || getVisionCapabilityFields(modelId);
|
||||
const contextLength =
|
||||
typeof (model as any).contextLength === "number"
|
||||
? (model as any).contextLength
|
||||
: undefined;
|
||||
|
||||
models.push({
|
||||
id: aliasId,
|
||||
object: "model",
|
||||
created: timestamp,
|
||||
owned_by: providerId,
|
||||
permission: [],
|
||||
root: modelId,
|
||||
parent: null,
|
||||
...(contextLength ? { context_length: contextLength } : {}),
|
||||
...(visionFields || {}),
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
// Filter by API key permissions if requested
|
||||
const authHeader = request.headers.get("authorization");
|
||||
let finalModels = models;
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
import { CORS_ORIGIN } from "@/shared/utils/cors";
|
||||
import { PROVIDER_MODELS } from "@/shared/constants/models";
|
||||
import { getAllCustomModels, getSyncedAvailableModels } from "@/lib/db/models";
|
||||
|
||||
/**
|
||||
* Handle CORS preflight
|
||||
@@ -16,13 +17,13 @@ export async function OPTIONS() {
|
||||
|
||||
/**
|
||||
* GET /v1beta/models - Gemini compatible models list
|
||||
* Returns models in Gemini API format
|
||||
* Returns models in Gemini API format with real token limits when available.
|
||||
*/
|
||||
export async function GET() {
|
||||
try {
|
||||
// Collect all models from all providers
|
||||
const models = [];
|
||||
|
||||
// Built-in models (hardcoded defaults)
|
||||
for (const [provider, providerModels] of Object.entries(PROVIDER_MODELS)) {
|
||||
for (const model of providerModels) {
|
||||
models.push({
|
||||
@@ -36,8 +37,60 @@ export async function GET() {
|
||||
}
|
||||
}
|
||||
|
||||
// Gemini: always replace hardcoded entries with synced models (no fallback)
|
||||
// Always remove hardcoded gemini entries — even if sync returns empty
|
||||
for (let i = models.length - 1; i >= 0; i--) {
|
||||
if (typeof (models[i] as any).name === "string" && (models[i] as any).name.startsWith("models/gemini/")) {
|
||||
models.splice(i, 1);
|
||||
}
|
||||
}
|
||||
try {
|
||||
const syncedGeminiModels = await getSyncedAvailableModels("gemini");
|
||||
for (const m of syncedGeminiModels) {
|
||||
models.push({
|
||||
name: `models/gemini/${m.id}`,
|
||||
displayName: m.name || m.id,
|
||||
...(typeof m.description === "string" ? { description: m.description } : {}),
|
||||
supportedGenerationMethods: ["generateContent"],
|
||||
inputTokenLimit: typeof m.inputTokenLimit === "number" ? m.inputTokenLimit : 128000,
|
||||
outputTokenLimit: typeof m.outputTokenLimit === "number" ? m.outputTokenLimit : 8192,
|
||||
...(m.supportsThinking === true ? { thinking: true } : {}),
|
||||
});
|
||||
}
|
||||
} catch (err) {
|
||||
console.error("[v1beta/models] Error fetching synced Gemini models:", err);
|
||||
}
|
||||
|
||||
// Custom models (use stored metadata from provider APIs)
|
||||
try {
|
||||
const customModelsMap = (await getAllCustomModels()) as Record<string, unknown>;
|
||||
for (const [providerId, rawModels] of Object.entries(customModelsMap)) {
|
||||
if (!Array.isArray(rawModels)) continue;
|
||||
// Skip Gemini — handled by syncedAvailableModels above
|
||||
if (providerId === "gemini") continue;
|
||||
for (const model of rawModels) {
|
||||
if (!model || typeof model !== "object" || typeof (model as any).id !== "string") continue;
|
||||
const m = model as Record<string, unknown>;
|
||||
if (m.isHidden === true) continue;
|
||||
models.push({
|
||||
name: `models/${providerId}/${m.id}`,
|
||||
displayName: m.name || m.id,
|
||||
...(typeof m.description === "string" ? { description: m.description } : {}),
|
||||
supportedGenerationMethods: ["generateContent"],
|
||||
inputTokenLimit:
|
||||
typeof m.inputTokenLimit === "number" ? m.inputTokenLimit : 128000,
|
||||
outputTokenLimit:
|
||||
typeof m.outputTokenLimit === "number" ? m.outputTokenLimit : 8192,
|
||||
...(m.supportsThinking === true ? { thinking: true } : {}),
|
||||
});
|
||||
}
|
||||
}
|
||||
} catch {
|
||||
// Custom models are optional — skip on error
|
||||
}
|
||||
|
||||
return Response.json({ models });
|
||||
} catch (error) {
|
||||
} catch (error: any) {
|
||||
console.log("Error fetching models:", error);
|
||||
return Response.json({ error: { message: error.message } }, { status: 500 });
|
||||
}
|
||||
|
||||
@@ -156,6 +156,8 @@
|
||||
"playground": "Playground",
|
||||
"searchTools": "Search Tools",
|
||||
"agents": "Agents",
|
||||
"memory": "Memory",
|
||||
"skills": "Skills",
|
||||
"docs": "Docs",
|
||||
"issues": "Issues",
|
||||
"endpoints": "Endpoints",
|
||||
@@ -1266,6 +1268,57 @@
|
||||
"events": "Events",
|
||||
"artifacts": "Artifacts"
|
||||
},
|
||||
"memory": {
|
||||
"title": "Memory Management",
|
||||
"description": "View and manage stored memory entries",
|
||||
"memories": "Memories",
|
||||
"totalEntries": "Total Entries",
|
||||
"tokensUsed": "Tokens Used",
|
||||
"hitRate": "Hit Rate",
|
||||
"loading": "Loading memories...",
|
||||
"noMemories": "No memories found",
|
||||
"search": "Search memories...",
|
||||
"allTypes": "All Types",
|
||||
"export": "Export",
|
||||
"import": "Import",
|
||||
"addMemory": "Add Memory",
|
||||
"type": "Type",
|
||||
"key": "Key",
|
||||
"content": "Content",
|
||||
"created": "Created",
|
||||
"actions": "Actions",
|
||||
"factual": "Factual",
|
||||
"episodic": "Episodic",
|
||||
"procedural": "Procedural",
|
||||
"semantic": "Semantic"
|
||||
},
|
||||
"skills": {
|
||||
"title": "Skills",
|
||||
"description": "Manage and monitor AI skills",
|
||||
"skillsTab": "Skills",
|
||||
"executionsTab": "Executions",
|
||||
"sandboxTab": "Sandbox",
|
||||
"loading": "Loading skills...",
|
||||
"noSkills": "No skills found",
|
||||
"noExecutions": "No executions found",
|
||||
"enabled": "Enabled",
|
||||
"disabled": "Disabled",
|
||||
"version": "Version",
|
||||
"tableDescription": "Description",
|
||||
"skill": "Skill",
|
||||
"status": "Status",
|
||||
"duration": "Duration",
|
||||
"time": "Time",
|
||||
"sandboxConfig": "Sandbox Configuration",
|
||||
"cpuLimit": "CPU Limit",
|
||||
"cpuLimitDesc": "Maximum execution time per skill",
|
||||
"memoryLimit": "Memory Limit",
|
||||
"memoryLimitDesc": "Maximum memory allocation",
|
||||
"timeout": "Timeout",
|
||||
"timeoutDesc": "Maximum wait time for response",
|
||||
"networkAccess": "Network Access",
|
||||
"networkAccessDesc": "Allow outbound network requests"
|
||||
},
|
||||
"health": {
|
||||
"title": "System Health",
|
||||
"description": "Real-time monitoring of your OmniRoute instance",
|
||||
@@ -1542,6 +1595,7 @@
|
||||
"chatCompletions": "Chat Completions",
|
||||
"importingModels": "Importing...",
|
||||
"importFromModels": "Import from /models",
|
||||
"modelsImported": "{count} models imported",
|
||||
"allModelsAlreadyImported": "All models already imported",
|
||||
"noNewModelsToImport": "No new models to import — all models are already in the registry or custom models list",
|
||||
"skippingExistingModels": "Skipping {count} existing models",
|
||||
@@ -1563,6 +1617,7 @@
|
||||
"importFailed": "Import failed",
|
||||
"noNewModelsAdded": "No new models were added.",
|
||||
"adding": "Adding...",
|
||||
"close": "Close",
|
||||
"importingModelsTitle": "Importing Models",
|
||||
"copyModel": "Copy model",
|
||||
"removeModel": "Remove model",
|
||||
@@ -3015,6 +3070,7 @@
|
||||
"cachedTokensCol": "Cached",
|
||||
"cacheCreation": "Creation",
|
||||
"trend24h": "Cache Trend (24h)",
|
||||
"peakCached": "Peak cached",
|
||||
"cached": "Cached",
|
||||
"overview": "Overview",
|
||||
"entries": "Entries",
|
||||
@@ -3028,4 +3084,4 @@
|
||||
"expires": "Expires",
|
||||
"actions": "Actions"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -156,6 +156,8 @@
|
||||
"playground": "Playground",
|
||||
"searchTools": "Search Tools",
|
||||
"agents": "Agentes",
|
||||
"memory": "Memória",
|
||||
"skills": "Habilidades",
|
||||
"docs": "Documentação",
|
||||
"issues": "Problemas",
|
||||
"endpoints": "Endpoints",
|
||||
@@ -1266,6 +1268,57 @@
|
||||
"events": "Eventos",
|
||||
"artifacts": "Artefatos"
|
||||
},
|
||||
"memory": {
|
||||
"title": "Gerenciamento de Memória",
|
||||
"description": "Visualize e gerencie entradas de memória armazenadas",
|
||||
"memories": "Memórias",
|
||||
"totalEntries": "Total de Entradas",
|
||||
"tokensUsed": "Tokens Usados",
|
||||
"hitRate": "Taxa de Acerto",
|
||||
"loading": "Carregando memórias...",
|
||||
"noMemories": "Nenhuma memória encontrada",
|
||||
"search": "Buscar memórias...",
|
||||
"allTypes": "Todos os Tipos",
|
||||
"export": "Exportar",
|
||||
"import": "Importar",
|
||||
"addMemory": "Adicionar Memória",
|
||||
"type": "Tipo",
|
||||
"key": "Chave",
|
||||
"content": "Conteúdo",
|
||||
"created": "Criado",
|
||||
"actions": "Ações",
|
||||
"factual": "Factual",
|
||||
"episodic": "Episódica",
|
||||
"procedural": "Procedural",
|
||||
"semantic": "Semântica"
|
||||
},
|
||||
"skills": {
|
||||
"title": "Skills",
|
||||
"description": "Gerencie e monitore skills de IA",
|
||||
"skillsTab": "Skills",
|
||||
"executionsTab": "Execuções",
|
||||
"sandboxTab": "Sandbox",
|
||||
"loading": "Carregando skills...",
|
||||
"noSkills": "Nenhuma skill encontrada",
|
||||
"noExecutions": "Nenhuma execução encontrada",
|
||||
"enabled": "Ativado",
|
||||
"disabled": "Desativado",
|
||||
"version": "Versão",
|
||||
"tableDescription": "Descrição",
|
||||
"skill": "Skill",
|
||||
"status": "Status",
|
||||
"duration": "Duração",
|
||||
"time": "Tempo",
|
||||
"sandboxConfig": "Configuração da Sandbox",
|
||||
"cpuLimit": "Limite de CPU",
|
||||
"cpuLimitDesc": "Tempo máximo de execução por skill",
|
||||
"memoryLimit": "Limite de Memória",
|
||||
"memoryLimitDesc": "Alocação máxima de memória",
|
||||
"timeout": "Timeout",
|
||||
"timeoutDesc": "Tempo máximo de espera por resposta",
|
||||
"networkAccess": "Acesso à Rede",
|
||||
"networkAccessDesc": "Permitir requisições de rede de saída"
|
||||
},
|
||||
"health": {
|
||||
"title": "Saúde do Sistema",
|
||||
"description": "Monitoramento em tempo real da sua instância OmniRoute",
|
||||
|
||||
@@ -156,6 +156,8 @@
|
||||
"playground": "Playground",
|
||||
"searchTools": "Search Tools",
|
||||
"agents": "Agentes",
|
||||
"memory": "Memória",
|
||||
"skills": "Habilidades",
|
||||
"docs": "Documentos",
|
||||
"issues": "Problemas",
|
||||
"endpoints": "Endpoints",
|
||||
@@ -1264,7 +1266,58 @@
|
||||
"close": "Close",
|
||||
"metadata": "Metadata",
|
||||
"events": "Events",
|
||||
"artifacts": "Artifacts"
|
||||
"artifacts": "Artefatos"
|
||||
},
|
||||
"memory": {
|
||||
"title": "Gerenciamento de Memória",
|
||||
"description": "Visualize e gerencie entradas de memória armazenadas",
|
||||
"memories": "Memórias",
|
||||
"totalEntries": "Total de Entradas",
|
||||
"tokensUsed": "Tokens Usados",
|
||||
"hitRate": "Taxa de Acerto",
|
||||
"loading": "Carregando memórias...",
|
||||
"noMemories": "Nenhuma memória encontrada",
|
||||
"search": "Buscar memórias...",
|
||||
"allTypes": "Todos os Tipos",
|
||||
"export": "Exportar",
|
||||
"import": "Importar",
|
||||
"addMemory": "Adicionar Memória",
|
||||
"type": "Tipo",
|
||||
"key": "Chave",
|
||||
"content": "Conteúdo",
|
||||
"created": "Criado",
|
||||
"actions": "Ações",
|
||||
"factual": "Factual",
|
||||
"episodic": "Episódica",
|
||||
"procedural": "Procedural",
|
||||
"semantic": "Semântica"
|
||||
},
|
||||
"skills": {
|
||||
"title": "Skills",
|
||||
"description": "Gerencie e monitore skills de IA",
|
||||
"skillsTab": "Skills",
|
||||
"executionsTab": "Execuções",
|
||||
"sandboxTab": "Sandbox",
|
||||
"loading": "Carregando skills...",
|
||||
"noSkills": "Nenhuma skill encontrada",
|
||||
"noExecutions": "Nenhuma execução encontrada",
|
||||
"enabled": "Ativado",
|
||||
"disabled": "Desativado",
|
||||
"version": "Versão",
|
||||
"tableDescription": "Descrição",
|
||||
"skill": "Skill",
|
||||
"status": "Status",
|
||||
"duration": "Duração",
|
||||
"time": "Tempo",
|
||||
"sandboxConfig": "Configuração da Sandbox",
|
||||
"cpuLimit": "Limite de CPU",
|
||||
"cpuLimitDesc": "Tempo máximo de execução por skill",
|
||||
"memoryLimit": "Limite de Memória",
|
||||
"memoryLimitDesc": "Alocação máxima de memória",
|
||||
"timeout": "Timeout",
|
||||
"timeoutDesc": "Tempo máximo de espera por resposta",
|
||||
"networkAccess": "Acesso à Rede",
|
||||
"networkAccessDesc": "Permitir requisições de rede de saída"
|
||||
},
|
||||
"health": {
|
||||
"title": "Saúde do sistema",
|
||||
|
||||
@@ -383,6 +383,10 @@ export async function replaceCustomModels(
|
||||
source?: string;
|
||||
apiFormat?: string;
|
||||
supportedEndpoints?: string[];
|
||||
inputTokenLimit?: number;
|
||||
outputTokenLimit?: number;
|
||||
description?: string;
|
||||
supportsThinking?: boolean;
|
||||
}>,
|
||||
{ allowEmpty = false }: { allowEmpty?: boolean } = {}
|
||||
) {
|
||||
@@ -412,6 +416,27 @@ export async function replaceCustomModels(
|
||||
source: m.source || "auto-sync",
|
||||
apiFormat: m.apiFormat || (prev as any)?.apiFormat || "chat-completions",
|
||||
supportedEndpoints: m.supportedEndpoints || (prev as any)?.supportedEndpoints || ["chat"],
|
||||
// Preserve metadata from provider API (or previous sync)
|
||||
...(m.inputTokenLimit != null
|
||||
? { inputTokenLimit: m.inputTokenLimit }
|
||||
: (prev as any)?.inputTokenLimit != null
|
||||
? { inputTokenLimit: (prev as any).inputTokenLimit }
|
||||
: {}),
|
||||
...(m.outputTokenLimit != null
|
||||
? { outputTokenLimit: m.outputTokenLimit }
|
||||
: (prev as any)?.outputTokenLimit != null
|
||||
? { outputTokenLimit: (prev as any).outputTokenLimit }
|
||||
: {}),
|
||||
...(m.description != null
|
||||
? { description: m.description }
|
||||
: (prev as any)?.description != null
|
||||
? { description: (prev as any).description }
|
||||
: {}),
|
||||
...(m.supportsThinking != null
|
||||
? { supportsThinking: m.supportsThinking }
|
||||
: (prev as any)?.supportsThinking != null
|
||||
? { supportsThinking: (prev as any).supportsThinking }
|
||||
: {}),
|
||||
// Preserve existing compat flags
|
||||
...(prev && (prev as any).normalizeToolCallId !== undefined
|
||||
? { normalizeToolCallId: (prev as any).normalizeToolCallId }
|
||||
@@ -481,6 +506,108 @@ export async function removeCustomModel(providerId: string, modelId: string) {
|
||||
return true;
|
||||
}
|
||||
|
||||
// ──────────────── Synced Available Models ────────────────
|
||||
// Storage: namespace = 'syncedAvailableModels', key = '<providerId>:<connectionId>'
|
||||
// Each connection stores its own model list. Reads union across all connections
|
||||
// for a provider. Deleting a connection removes only its models.
|
||||
|
||||
export interface SyncedAvailableModel {
|
||||
id: string;
|
||||
name: string;
|
||||
source: "api-sync";
|
||||
supportedEndpoints?: string[];
|
||||
inputTokenLimit?: number;
|
||||
outputTokenLimit?: number;
|
||||
description?: string;
|
||||
supportsThinking?: boolean;
|
||||
}
|
||||
|
||||
/**
|
||||
* Get all synced available models for a provider, unioned across all connections.
|
||||
*/
|
||||
export async function getSyncedAvailableModels(providerId: string): Promise<SyncedAvailableModel[]> {
|
||||
const db = getDbInstance();
|
||||
const rows = db
|
||||
.prepare("SELECT key, value FROM key_value WHERE namespace = 'syncedAvailableModels' AND key LIKE ?")
|
||||
.all(`${providerId}:%`);
|
||||
const map = new Map<string, SyncedAvailableModel>();
|
||||
for (const row of rows) {
|
||||
const { key, value } = getKeyValue(row);
|
||||
if (!key || value === null) continue;
|
||||
const models: SyncedAvailableModel[] = JSON.parse(value);
|
||||
for (const m of models) {
|
||||
if (m.id) map.set(m.id, m);
|
||||
}
|
||||
}
|
||||
return Array.from(map.values());
|
||||
}
|
||||
|
||||
/**
|
||||
* Get all synced available models across all providers.
|
||||
*/
|
||||
export async function getAllSyncedAvailableModels(): Promise<Record<string, SyncedAvailableModel[]>> {
|
||||
const db = getDbInstance();
|
||||
const rows = db
|
||||
.prepare("SELECT key, value FROM key_value WHERE namespace = 'syncedAvailableModels'")
|
||||
.all();
|
||||
// Group by providerId (before the colon)
|
||||
const byProvider = new Map<string, Map<string, SyncedAvailableModel>>();
|
||||
for (const row of rows) {
|
||||
const { key, value } = getKeyValue(row);
|
||||
if (!key || value === null) continue;
|
||||
const providerId = key.split(":")[0];
|
||||
if (!byProvider.has(providerId)) byProvider.set(providerId, new Map());
|
||||
const models: SyncedAvailableModel[] = JSON.parse(value);
|
||||
const map = byProvider.get(providerId)!;
|
||||
for (const m of models) {
|
||||
if (m.id) map.set(m.id, m);
|
||||
}
|
||||
}
|
||||
const result: Record<string, SyncedAvailableModel[]> = {};
|
||||
for (const [providerId, map] of byProvider) {
|
||||
result[providerId] = Array.from(map.values());
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
/**
|
||||
* Replace the model list for a specific connection.
|
||||
* Key format: '<providerId>:<connectionId>'
|
||||
*/
|
||||
export async function replaceSyncedAvailableModelsForConnection(
|
||||
providerId: string,
|
||||
connectionId: string,
|
||||
models: SyncedAvailableModel[]
|
||||
): Promise<SyncedAvailableModel[]> {
|
||||
const db = getDbInstance();
|
||||
const key = `${providerId}:${connectionId}`;
|
||||
if (models.length === 0) {
|
||||
db.prepare("DELETE FROM key_value WHERE namespace = 'syncedAvailableModels' AND key = ?").run(key);
|
||||
} else {
|
||||
db.prepare(
|
||||
"INSERT OR REPLACE INTO key_value (namespace, key, value) VALUES ('syncedAvailableModels', ?, ?)"
|
||||
).run(key, JSON.stringify(models));
|
||||
}
|
||||
backupDbFile("pre-write");
|
||||
// Return the full unioned list for the provider
|
||||
return getSyncedAvailableModels(providerId);
|
||||
}
|
||||
|
||||
/**
|
||||
* Delete all synced models for a specific connection.
|
||||
* Returns the remaining unioned list for the provider.
|
||||
*/
|
||||
export async function deleteSyncedAvailableModelsForConnection(
|
||||
providerId: string,
|
||||
connectionId: string
|
||||
): Promise<SyncedAvailableModel[]> {
|
||||
const db = getDbInstance();
|
||||
const key = `${providerId}:${connectionId}`;
|
||||
db.prepare("DELETE FROM key_value WHERE namespace = 'syncedAvailableModels' AND key = ?").run(key);
|
||||
backupDbFile("pre-write");
|
||||
return getSyncedAvailableModels(providerId);
|
||||
}
|
||||
|
||||
export async function updateCustomModel(
|
||||
providerId: string,
|
||||
modelId: string,
|
||||
|
||||
@@ -248,6 +248,31 @@ export async function resetAllPricing() {
|
||||
return {};
|
||||
}
|
||||
|
||||
// ──────────────── LKGP (Last Known Good Provider) ────────────────
|
||||
|
||||
export async function getLKGP(comboName: string, modelId: string): Promise<string | null> {
|
||||
const db = getDbInstance();
|
||||
const key = `${comboName}:${modelId}`;
|
||||
const row = db
|
||||
.prepare("SELECT value FROM key_value WHERE namespace = 'lkgp' AND key = ?")
|
||||
.get(key) as { value?: string } | undefined;
|
||||
if (!row?.value) return null;
|
||||
try {
|
||||
return JSON.parse(row.value);
|
||||
} catch {
|
||||
return row.value;
|
||||
}
|
||||
}
|
||||
|
||||
export async function setLKGP(comboName: string, modelId: string, providerId: string) {
|
||||
const db = getDbInstance();
|
||||
const key = `${comboName}:${modelId}`;
|
||||
db.prepare("INSERT OR REPLACE INTO key_value (namespace, key, value) VALUES ('lkgp', ?, ?)").run(
|
||||
key,
|
||||
JSON.stringify(providerId)
|
||||
);
|
||||
}
|
||||
|
||||
// ──────────────── Proxy Config ────────────────
|
||||
|
||||
const DEFAULT_PROXY_CONFIG: ProxyConfig = { global: null, providers: {}, combos: {}, keys: {} };
|
||||
|
||||
@@ -58,9 +58,15 @@ export {
|
||||
getModelPreserveOpenAIDeveloperRole,
|
||||
getModelUpstreamExtraHeaders,
|
||||
getModelIsHidden,
|
||||
|
||||
// Synced Available Models
|
||||
getSyncedAvailableModels,
|
||||
getAllSyncedAvailableModels,
|
||||
replaceSyncedAvailableModelsForConnection,
|
||||
deleteSyncedAvailableModelsForConnection,
|
||||
} from "./db/models";
|
||||
|
||||
export type { ModelCompatPerProtocol, ModelCompatPatch } from "./db/models";
|
||||
export type { ModelCompatPerProtocol, ModelCompatPatch, SyncedAvailableModel } from "./db/models";
|
||||
|
||||
export {
|
||||
// Combos
|
||||
@@ -92,6 +98,10 @@ export {
|
||||
updateSettings,
|
||||
isCloudEnabled,
|
||||
|
||||
// LKGP (Last Known Good Provider) (#919)
|
||||
getLKGP,
|
||||
setLKGP,
|
||||
|
||||
// Pricing
|
||||
getPricing,
|
||||
getPricingForModel,
|
||||
|
||||
@@ -3,12 +3,12 @@ import { CLAUDE_CONFIG } from "../constants/oauth";
|
||||
export const claude = {
|
||||
config: CLAUDE_CONFIG,
|
||||
flowType: "authorization_code_pkce",
|
||||
buildAuthUrl: (config, _redirectUri, state, codeChallenge) => {
|
||||
buildAuthUrl: (config, redirectUri, state, codeChallenge) => {
|
||||
const params = new URLSearchParams({
|
||||
code: "true",
|
||||
client_id: config.clientId,
|
||||
response_type: "code",
|
||||
redirect_uri: config.redirectUri,
|
||||
redirect_uri: redirectUri,
|
||||
scope: config.scopes.join(" "),
|
||||
code_challenge: codeChallenge,
|
||||
code_challenge_method: config.codeChallengeMethod,
|
||||
@@ -16,7 +16,7 @@ export const claude = {
|
||||
});
|
||||
return `${config.authorizeUrl}?${params.toString()}`;
|
||||
},
|
||||
exchangeToken: async (config, code, _redirectUri, codeVerifier, state) => {
|
||||
exchangeToken: async (config, code, redirectUri, codeVerifier, state) => {
|
||||
let authCode = code;
|
||||
let codeState = "";
|
||||
if (authCode.includes("#")) {
|
||||
@@ -36,7 +36,7 @@ export const claude = {
|
||||
state: codeState || state,
|
||||
grant_type: "authorization_code",
|
||||
client_id: config.clientId,
|
||||
redirect_uri: config.redirectUri,
|
||||
redirect_uri: redirectUri,
|
||||
code_verifier: codeVerifier,
|
||||
}),
|
||||
});
|
||||
|
||||
@@ -4,6 +4,7 @@ import { useState, useMemo, useEffect } from "react";
|
||||
import PropTypes from "prop-types";
|
||||
import Modal from "./Modal";
|
||||
import { getModelsByProviderId, PROVIDER_ID_TO_ALIAS } from "@/shared/constants/models";
|
||||
import { getCompatibleFallbackModels } from "@/lib/providers/managedAvailableModels";
|
||||
import {
|
||||
OAUTH_PROVIDERS,
|
||||
FREE_PROVIDERS,
|
||||
@@ -160,9 +161,24 @@ export default function ModelSelectModal({
|
||||
value: `${nodePrefix}/${fullModel.replace(`${providerId}/`, "")}`,
|
||||
}));
|
||||
|
||||
const fallbackEntries = (
|
||||
getCompatibleFallbackModels(providerId, providerCustomModels) || []
|
||||
)
|
||||
.filter((fm) => !nodeModels.some((nm) => nm.id === fm.id))
|
||||
.map((fm) => ({
|
||||
id: fm.id,
|
||||
name: fm.name || fm.id,
|
||||
value: `${nodePrefix}/${fm.id}`,
|
||||
isFallback: true,
|
||||
}));
|
||||
|
||||
// Merge custom models for custom providers
|
||||
const customEntries = providerCustomModels
|
||||
.filter((cm) => !nodeModels.some((nm) => nm.id === cm.id))
|
||||
.filter(
|
||||
(cm) =>
|
||||
!nodeModels.some((nm) => nm.id === cm.id) &&
|
||||
!fallbackEntries.some((fm) => fm.id === cm.id)
|
||||
)
|
||||
.map((cm) => ({
|
||||
id: cm.id,
|
||||
name: cm.name || cm.id,
|
||||
@@ -170,7 +186,7 @@ export default function ModelSelectModal({
|
||||
isCustom: true,
|
||||
}));
|
||||
|
||||
const allModels = [...nodeModels, ...customEntries];
|
||||
const allModels = [...nodeModels, ...fallbackEntries, ...customEntries];
|
||||
|
||||
if (allModels.length > 0) {
|
||||
groups[providerId] = {
|
||||
|
||||
@@ -603,6 +603,46 @@ export const APIKEY_PROVIDERS = {
|
||||
"$0.025/day free credits — 200+ models (GPT-4o, Claude, Gemini, Llama) via single endpoint",
|
||||
passthroughModels: true,
|
||||
},
|
||||
novita: {
|
||||
id: "novita",
|
||||
alias: "novita",
|
||||
name: "Novita AI",
|
||||
icon: "auto_awesome",
|
||||
color: "#FF4081",
|
||||
textIcon: "NV",
|
||||
website: "https://novita.ai",
|
||||
passthroughModels: true,
|
||||
},
|
||||
piapi: {
|
||||
id: "piapi",
|
||||
alias: "pi",
|
||||
name: "PiAPI",
|
||||
icon: "api",
|
||||
color: "#7C4DFF",
|
||||
textIcon: "PI",
|
||||
website: "https://piapi.ai",
|
||||
passthroughModels: true,
|
||||
},
|
||||
getgoapi: {
|
||||
id: "getgoapi",
|
||||
alias: "ggo",
|
||||
name: "GoAPI",
|
||||
icon: "rocket_launch",
|
||||
color: "#FF6D00",
|
||||
textIcon: "GO",
|
||||
website: "https://api.getgoapi.com",
|
||||
passthroughModels: true,
|
||||
},
|
||||
laozhang: {
|
||||
id: "laozhang",
|
||||
alias: "lz",
|
||||
name: "LaoZhang AI",
|
||||
icon: "hub",
|
||||
color: "#FF1744",
|
||||
textIcon: "LZ",
|
||||
website: "https://api.laozhang.ai",
|
||||
passthroughModels: true,
|
||||
},
|
||||
};
|
||||
|
||||
export const OPENAI_COMPATIBLE_PREFIX = "openai-compatible-";
|
||||
|
||||
@@ -11,6 +11,8 @@ export const HIDEABLE_SIDEBAR_ITEM_IDS = [
|
||||
"cache",
|
||||
"cli-tools",
|
||||
"agents",
|
||||
"memory",
|
||||
"skills",
|
||||
"translator",
|
||||
"playground",
|
||||
"media",
|
||||
@@ -60,6 +62,8 @@ const PRIMARY_SIDEBAR_ITEMS: readonly SidebarItemDefinition[] = [
|
||||
const CLI_SIDEBAR_ITEMS: readonly SidebarItemDefinition[] = [
|
||||
{ id: "cli-tools", href: "/dashboard/cli-tools", i18nKey: "cliToolsShort", icon: "terminal" },
|
||||
{ id: "agents", href: "/dashboard/agents", i18nKey: "agents", icon: "smart_toy" },
|
||||
{ id: "memory", href: "/dashboard/memory", i18nKey: "memory", icon: "psychology" },
|
||||
{ id: "skills", href: "/dashboard/skills", i18nKey: "skills", icon: "auto_fix_high" },
|
||||
];
|
||||
|
||||
const DEBUG_SIDEBAR_ITEMS: readonly SidebarItemDefinition[] = [
|
||||
|
||||
@@ -660,7 +660,13 @@ const checkRunnable = async (
|
||||
const minimalEnv: Record<string, string | undefined> = {
|
||||
PATH: env.PATH,
|
||||
HOME: env.HOME || env.USERPROFILE,
|
||||
USERPROFILE: env.USERPROFILE, // Windows needs this for os.homedir()
|
||||
APPDATA: env.APPDATA, // Many npm CLI tools rely on APPDATA
|
||||
LOCALAPPDATA: env.LOCALAPPDATA,
|
||||
TEMP: env.TEMP,
|
||||
TMP: env.TMP,
|
||||
SystemRoot: env.SystemRoot, // Windows needs this
|
||||
ComSpec: env.ComSpec, // Windows shell
|
||||
PATHEXT: env.PATHEXT, // Windows cmd.exe needs this to resolve .cmd/.bat/.exe extensions
|
||||
};
|
||||
|
||||
|
||||
@@ -6,7 +6,7 @@ if (!process.env.API_KEY_SECRET) {
|
||||
}
|
||||
|
||||
function getApiKeySecret(): string {
|
||||
const secret = process.env.API_KEY_SECRET;
|
||||
const secret = process.env.API_KEY_SECRET || "omniroute-default-insecure-api-key-secret";
|
||||
if (!secret || secret.trim() === "") {
|
||||
throw new Error("API_KEY_SECRET is required for API key CRC operations");
|
||||
}
|
||||
|
||||
@@ -197,13 +197,21 @@ export class CircuitBreaker {
|
||||
// ─── Internal Methods ────────────────────────
|
||||
|
||||
_onSuccess() {
|
||||
if (this.state === STATE.HALF_OPEN) {
|
||||
if (this.state === STATE.OPEN) {
|
||||
// Direct call from combo path: timeout elapsed and request succeeded
|
||||
// without going through execute(), so transition OPEN → CLOSED directly
|
||||
this._transition(STATE.CLOSED);
|
||||
this.failureCount = 0;
|
||||
this.successCount = 0;
|
||||
this.lastFailureTime = null;
|
||||
} else if (this.state === STATE.HALF_OPEN) {
|
||||
this.successCount++;
|
||||
this._transition(STATE.CLOSED);
|
||||
this.failureCount = 0;
|
||||
} else {
|
||||
// In CLOSED state, just reset failure count
|
||||
this.failureCount = 0;
|
||||
}
|
||||
// In CLOSED state, just reset failure count
|
||||
this.failureCount = 0;
|
||||
this._persistToDb();
|
||||
}
|
||||
|
||||
@@ -211,7 +219,9 @@ export class CircuitBreaker {
|
||||
this.failureCount++;
|
||||
this.lastFailureTime = Date.now();
|
||||
|
||||
if (this.state === STATE.HALF_OPEN) {
|
||||
if (this.state === STATE.OPEN) {
|
||||
// Already OPEN — just update persistence (re-tripped by combo path)
|
||||
} else if (this.state === STATE.HALF_OPEN) {
|
||||
this._transition(STATE.OPEN);
|
||||
} else if (this.failureCount >= this.failureThreshold) {
|
||||
this._transition(STATE.OPEN);
|
||||
|
||||
@@ -567,7 +567,9 @@ async function handleSingleModelChat(
|
||||
}
|
||||
|
||||
// 6. Mark account as quota-exhausted on 429 response
|
||||
if (result.status === 429) {
|
||||
// For per-model quota providers (Gemini), a 429 on one model doesn't mean
|
||||
// the entire account is exhausted — skip connection-wide exhaustion marking.
|
||||
if (result.status === 429 && provider !== "gemini") {
|
||||
markAccountExhaustedFrom429(credentials.connectionId, provider);
|
||||
}
|
||||
|
||||
|
||||
@@ -14,8 +14,12 @@ import {
|
||||
checkFallbackError,
|
||||
isModelLocked,
|
||||
lockModel,
|
||||
hasPerModelQuota,
|
||||
} from "@omniroute/open-sse/services/accountFallback.ts";
|
||||
import { isLocalProvider, getPassthroughProviders } from "@omniroute/open-sse/config/providerRegistry.ts";
|
||||
import {
|
||||
isLocalProvider,
|
||||
getPassthroughProviders,
|
||||
} from "@omniroute/open-sse/config/providerRegistry.ts";
|
||||
import { COOLDOWN_MS } from "@omniroute/open-sse/config/constants.ts";
|
||||
import { getCodexModelScope } from "@omniroute/open-sse/executors/codex.ts";
|
||||
import * as log from "../utils/logger";
|
||||
@@ -307,6 +311,16 @@ const markMutexes = new Map<string, Promise<void>>();
|
||||
// Re-export for backwards compat with existing test imports.
|
||||
export { fisherYatesShuffle, getNextFromDeckSync as getNextFromDeck };
|
||||
|
||||
/**
|
||||
* Resolve provider aliases (e.g., nvidia -> nvidia_nim) for DB lookup
|
||||
*/
|
||||
function getProviderSearchPool(provider: string): string[] {
|
||||
const pool = [provider];
|
||||
if (provider === "nvidia") pool.push("nvidia_nim");
|
||||
if (provider === "nvidia_nim") pool.push("nvidia");
|
||||
return [...new Set(pool)];
|
||||
}
|
||||
|
||||
/**
|
||||
* Get provider credentials from localDb
|
||||
* Filters out unavailable accounts and returns the selected account based on strategy
|
||||
@@ -333,7 +347,14 @@ export async function getProviderCredentials(
|
||||
const allowSuppressedConnections = options.allowSuppressedConnections === true;
|
||||
const bypassQuotaPolicy = options.bypassQuotaPolicy === true;
|
||||
|
||||
const connectionsRaw = await getProviderConnections({ provider, isActive: true });
|
||||
// Fix #922: Check for aliases (nvidia/nvidia_nim) to ensure credentials are found
|
||||
const providersToSearch = getProviderSearchPool(provider);
|
||||
let connectionsRaw: any[] = [];
|
||||
for (const p of providersToSearch) {
|
||||
const results = await getProviderConnections({ provider: p, isActive: true });
|
||||
if (Array.isArray(results)) connectionsRaw.push(...results);
|
||||
}
|
||||
|
||||
let connections = (Array.isArray(connectionsRaw) ? connectionsRaw : [])
|
||||
.map(toProviderConnection)
|
||||
.filter((conn) => conn.id.length > 0);
|
||||
@@ -348,7 +369,12 @@ export async function getProviderCredentials(
|
||||
|
||||
if (connections.length === 0) {
|
||||
// Check all connections (including inactive) to see if rate limited
|
||||
const allConnectionsRaw = await getProviderConnections({ provider });
|
||||
// Fix #922: Also search aliases here
|
||||
let allConnectionsRaw: any[] = [];
|
||||
for (const p of providersToSearch) {
|
||||
const results = await getProviderConnections({ provider: p });
|
||||
if (Array.isArray(results)) allConnectionsRaw.push(...results);
|
||||
}
|
||||
const allConnections = (Array.isArray(allConnectionsRaw) ? allConnectionsRaw : [])
|
||||
.map(toProviderConnection)
|
||||
.filter((conn) => conn.id.length > 0);
|
||||
@@ -408,6 +434,8 @@ export async function getProviderCredentials(
|
||||
if (isAccountUnavailable(c.rateLimitedUntil)) return false;
|
||||
if (isTerminalConnectionStatus(c)) return false;
|
||||
if (provider === "codex" && isCodexScopeUnavailable(c, requestedModel)) return false;
|
||||
// Per-model lockout: if this specific model is locked on this connection, skip it
|
||||
if (requestedModel && isModelLocked(provider, c.id, requestedModel)) return false;
|
||||
}
|
||||
return true;
|
||||
});
|
||||
@@ -724,6 +752,27 @@ export async function markAccountUnavailable(
|
||||
try {
|
||||
await currentMutex;
|
||||
|
||||
// ── Per-model lockout for providers with independent model quotas ──
|
||||
// Providers like Gemini AI Studio have per-model quotas. A 429/404 on one
|
||||
// model must NOT lock out other models on the same API key.
|
||||
if (hasPerModelQuota(provider) && model && (status === 429 || status === 404)) {
|
||||
const reason = status === 404 ? "not_found" : "rate_limited";
|
||||
const cooldown = status === 404 ? COOLDOWN_MS.notFoundLocal : COOLDOWN_MS.rateLimit;
|
||||
lockModel(provider, connectionId, model, reason, cooldown);
|
||||
// Update last error for observability (without changing terminal status)
|
||||
updateProviderConnection(connectionId, {
|
||||
lastErrorType: reason,
|
||||
lastError: `Model ${model} ${reason}`,
|
||||
lastErrorAt: new Date().toISOString(),
|
||||
errorCode: status,
|
||||
}).catch(() => {});
|
||||
log.info(
|
||||
"AUTH",
|
||||
`Model-only lockout for ${provider}:${model} — ${status} ${reason} ${Math.ceil(cooldown / 1000)}s (connection stays active)`
|
||||
);
|
||||
return { shouldFallback: true, cooldownMs: cooldown };
|
||||
}
|
||||
|
||||
// Read current connection to get backoffLevel
|
||||
const connectionsRaw = await getProviderConnections({ provider });
|
||||
const connections = (Array.isArray(connectionsRaw) ? connectionsRaw : [])
|
||||
@@ -793,7 +842,13 @@ export async function markAccountUnavailable(
|
||||
| undefined;
|
||||
|
||||
const isPassthroughProvider = provider && getPassthroughProviders().has(provider);
|
||||
if ((isLocalProvider(connBaseUrl) || isPassthroughProvider) && status === 404 && provider && model) {
|
||||
const isPerModelQuotaProvider = hasPerModelQuota(provider);
|
||||
if (
|
||||
(isLocalProvider(connBaseUrl) || isPerModelQuotaProvider) &&
|
||||
status === 404 &&
|
||||
provider &&
|
||||
model
|
||||
) {
|
||||
const localCooldown = COOLDOWN_MS.notFoundLocal;
|
||||
lockModel(provider, connectionId, model, "not_found", localCooldown);
|
||||
log.info(
|
||||
@@ -803,12 +858,12 @@ export async function markAccountUnavailable(
|
||||
return { shouldFallback: true, cooldownMs: localCooldown };
|
||||
}
|
||||
|
||||
// ── 429 model-only lockout for passthrough providers ──
|
||||
// For passthrough providers like Antigravity, each model has independent quota.
|
||||
// A 429 on one model should NOT lock out the entire connection — other models
|
||||
// may still have quota available. Use lockModel() instead of connection-wide
|
||||
// rateLimitedUntil, same pattern as the 404 model-only lockout above.
|
||||
if (isPassthroughProvider && status === 429 && provider && model) {
|
||||
// ── 429 model-only lockout for per-model quota providers ──
|
||||
// For providers where each model has independent quota (passthrough providers,
|
||||
// Gemini AI Studio), a 429 on one model should NOT lock out the entire connection
|
||||
// — other models may still have quota available. Use lockModel() instead of
|
||||
// connection-wide rateLimitedUntil.
|
||||
if (isPerModelQuotaProvider && status === 429 && provider && model) {
|
||||
const modelCooldown = cooldownMs || COOLDOWN_MS.rateLimit;
|
||||
lockModel(provider, connectionId, model, reason || "rate_limited", modelCooldown);
|
||||
log.info(
|
||||
|
||||
989
src/sse/services/auth.ts.orig
Normal file
989
src/sse/services/auth.ts.orig
Normal file
@@ -0,0 +1,989 @@
|
||||
import {
|
||||
getProviderConnections,
|
||||
validateApiKey,
|
||||
updateProviderConnection,
|
||||
getSettings,
|
||||
getCachedSettings,
|
||||
} from "@/lib/localDb";
|
||||
import { getQuotaWindowStatus, isAccountQuotaExhausted } from "@/domain/quotaCache";
|
||||
import {
|
||||
isAccountUnavailable,
|
||||
getUnavailableUntil,
|
||||
getEarliestRateLimitedUntil,
|
||||
formatRetryAfter,
|
||||
checkFallbackError,
|
||||
isModelLocked,
|
||||
lockModel,
|
||||
hasPerModelQuota,
|
||||
} from "@omniroute/open-sse/services/accountFallback.ts";
|
||||
import { isLocalProvider, getPassthroughProviders } from "@omniroute/open-sse/config/providerRegistry.ts";
|
||||
import { COOLDOWN_MS } from "@omniroute/open-sse/config/constants.ts";
|
||||
import { getCodexModelScope } from "@omniroute/open-sse/executors/codex.ts";
|
||||
import * as log from "../utils/logger";
|
||||
import { fisherYatesShuffle, getNextFromDeckSync } from "@/shared/utils/shuffleDeck";
|
||||
|
||||
type JsonRecord = Record<string, unknown>;
|
||||
|
||||
interface ProviderConnectionView {
|
||||
id: string;
|
||||
isActive: boolean;
|
||||
rateLimitedUntil: string | null;
|
||||
testStatus: string | null;
|
||||
apiKey: string | null;
|
||||
accessToken: string | null;
|
||||
refreshToken: string | null;
|
||||
tokenExpiresAt: string | null;
|
||||
expiresAt: string | null;
|
||||
projectId: string | null;
|
||||
providerSpecificData: JsonRecord;
|
||||
lastUsedAt: string | null;
|
||||
consecutiveUseCount: number;
|
||||
priority: number;
|
||||
lastError: string | null;
|
||||
lastErrorType: string | null;
|
||||
lastErrorSource: string | null;
|
||||
errorCode: string | number | null;
|
||||
backoffLevel: number;
|
||||
}
|
||||
|
||||
interface RecoverableConnectionState {
|
||||
connectionId: string;
|
||||
testStatus?: string | null;
|
||||
lastError?: string | null;
|
||||
rateLimitedUntil?: string | null;
|
||||
errorCode?: string | number | null;
|
||||
lastErrorType?: string | null;
|
||||
lastErrorSource?: string | null;
|
||||
}
|
||||
|
||||
interface CredentialSelectionOptions {
|
||||
allowSuppressedConnections?: boolean;
|
||||
bypassQuotaPolicy?: boolean;
|
||||
}
|
||||
|
||||
const CODEX_QUOTA_THRESHOLD_PERCENT = 90;
|
||||
const MIN_QUOTA_THRESHOLD_PERCENT = 1;
|
||||
const MAX_QUOTA_THRESHOLD_PERCENT = 100;
|
||||
|
||||
function asRecord(value: unknown): JsonRecord {
|
||||
return value && typeof value === "object" && !Array.isArray(value) ? (value as JsonRecord) : {};
|
||||
}
|
||||
|
||||
function toStringOrNull(value: unknown): string | null {
|
||||
return typeof value === "string" && value.trim().length > 0 ? value : null;
|
||||
}
|
||||
|
||||
function toNumber(value: unknown, fallback = 0): number {
|
||||
if (typeof value === "number" && Number.isFinite(value)) return value;
|
||||
if (typeof value === "string" && value.trim().length > 0) {
|
||||
const parsed = Number(value);
|
||||
return Number.isFinite(parsed) ? parsed : fallback;
|
||||
}
|
||||
return fallback;
|
||||
}
|
||||
|
||||
function toProviderConnection(value: unknown): ProviderConnectionView {
|
||||
const row = asRecord(value);
|
||||
return {
|
||||
id: toStringOrNull(row.id) || "",
|
||||
isActive: row.isActive === true,
|
||||
rateLimitedUntil: toStringOrNull(row.rateLimitedUntil),
|
||||
testStatus: toStringOrNull(row.testStatus),
|
||||
apiKey: toStringOrNull(row.apiKey),
|
||||
accessToken: toStringOrNull(row.accessToken),
|
||||
refreshToken: toStringOrNull(row.refreshToken),
|
||||
tokenExpiresAt: toStringOrNull(row.tokenExpiresAt),
|
||||
expiresAt: toStringOrNull(row.expiresAt),
|
||||
projectId: toStringOrNull(row.projectId),
|
||||
providerSpecificData: asRecord(row.providerSpecificData),
|
||||
lastUsedAt: toStringOrNull(row.lastUsedAt),
|
||||
consecutiveUseCount: toNumber(row.consecutiveUseCount, 0),
|
||||
priority: toNumber(row.priority, 999),
|
||||
lastError: toStringOrNull(row.lastError),
|
||||
lastErrorType: toStringOrNull(row.lastErrorType),
|
||||
lastErrorSource: toStringOrNull(row.lastErrorSource),
|
||||
errorCode:
|
||||
typeof row.errorCode === "string" || typeof row.errorCode === "number" ? row.errorCode : null,
|
||||
backoffLevel: toNumber(row.backoffLevel, 0),
|
||||
};
|
||||
}
|
||||
|
||||
function toBooleanOrDefault(value: unknown, fallback: boolean): boolean {
|
||||
return typeof value === "boolean" ? value : fallback;
|
||||
}
|
||||
|
||||
function getCodexLimitPolicy(providerSpecificData: JsonRecord): {
|
||||
use5h: boolean;
|
||||
useWeekly: boolean;
|
||||
} {
|
||||
const policy = asRecord(providerSpecificData.codexLimitPolicy);
|
||||
return {
|
||||
use5h: toBooleanOrDefault(policy.use5h, true),
|
||||
useWeekly: toBooleanOrDefault(policy.useWeekly, true),
|
||||
};
|
||||
}
|
||||
|
||||
interface QuotaLimitPolicy {
|
||||
enabled: boolean;
|
||||
thresholdPercent: number;
|
||||
windows: string[];
|
||||
}
|
||||
|
||||
function normalizeQuotaThreshold(value: unknown, fallback = CODEX_QUOTA_THRESHOLD_PERCENT): number {
|
||||
const parsed = toNumber(value, fallback);
|
||||
return Math.min(MAX_QUOTA_THRESHOLD_PERCENT, Math.max(MIN_QUOTA_THRESHOLD_PERCENT, parsed));
|
||||
}
|
||||
|
||||
function normalizeWindowName(windowName: unknown): string | null {
|
||||
if (typeof windowName !== "string") return null;
|
||||
const normalized = windowName.trim().toLowerCase();
|
||||
return normalized.length > 0 ? normalized : null;
|
||||
}
|
||||
|
||||
function uniqueWindows(windows: string[]): string[] {
|
||||
return [...new Set(windows)];
|
||||
}
|
||||
|
||||
function normalizeCodexWindowName(windowName: unknown): string | null {
|
||||
if (typeof windowName !== "string") return null;
|
||||
const normalized = windowName.trim().toLowerCase();
|
||||
if (normalized === "session (5h)" || normalized === "5h" || normalized === "five_hour") {
|
||||
return "session";
|
||||
}
|
||||
if (normalized === "weekly (7d)" || normalized === "7d" || normalized === "seven_day") {
|
||||
return "weekly";
|
||||
}
|
||||
return normalized;
|
||||
}
|
||||
|
||||
function applyCodexWindowPolicy(rawWindows: string[], providerSpecificData: JsonRecord): string[] {
|
||||
const codexPolicy = getCodexLimitPolicy(providerSpecificData);
|
||||
const normalizedRaw = rawWindows.map(normalizeCodexWindowName).filter(Boolean) as string[];
|
||||
|
||||
// Preserve explicitly configured custom windows, but enforce canonical Codex windows
|
||||
// from toggles so weekly exhaustion is never skipped when useWeekly=true.
|
||||
let windows = [...normalizedRaw];
|
||||
windows = windows.filter((windowName) => {
|
||||
if (windowName === "session") return codexPolicy.use5h;
|
||||
if (windowName === "weekly") return codexPolicy.useWeekly;
|
||||
return true;
|
||||
});
|
||||
if (codexPolicy.use5h) windows.push("session");
|
||||
if (codexPolicy.useWeekly) windows.push("weekly");
|
||||
|
||||
return uniqueWindows(windows);
|
||||
}
|
||||
|
||||
function getCodexScopeRateLimitedUntil(
|
||||
providerSpecificData: JsonRecord,
|
||||
model: string | null
|
||||
): string | null {
|
||||
if (!model) return null;
|
||||
const scope = getCodexModelScope(model);
|
||||
const scopeMap = asRecord(providerSpecificData.codexScopeRateLimitedUntil);
|
||||
const value = scopeMap[scope];
|
||||
return typeof value === "string" && value.trim().length > 0 ? value : null;
|
||||
}
|
||||
|
||||
function isCodexScopeUnavailable(
|
||||
connection: ProviderConnectionView,
|
||||
model: string | null
|
||||
): boolean {
|
||||
const until = getCodexScopeRateLimitedUntil(connection.providerSpecificData, model);
|
||||
if (!until) return false;
|
||||
return new Date(until).getTime() > Date.now();
|
||||
}
|
||||
|
||||
function getEarliestCodexScopeRateLimitedUntil(
|
||||
connections: ProviderConnectionView[],
|
||||
model: string | null
|
||||
): string | null {
|
||||
let earliest: string | null = null;
|
||||
let earliestMs = Infinity;
|
||||
|
||||
for (const conn of connections) {
|
||||
const until = getCodexScopeRateLimitedUntil(conn.providerSpecificData, model);
|
||||
if (!until) continue;
|
||||
const ms = new Date(until).getTime();
|
||||
if (!Number.isFinite(ms) || ms <= Date.now()) continue;
|
||||
if (ms < earliestMs) {
|
||||
earliest = until;
|
||||
earliestMs = ms;
|
||||
}
|
||||
}
|
||||
|
||||
return earliest;
|
||||
}
|
||||
|
||||
function normalizeStatus(value: string | null): string {
|
||||
return (value || "").trim().toLowerCase();
|
||||
}
|
||||
|
||||
function isTerminalConnectionStatus(connection: ProviderConnectionView): boolean {
|
||||
const status = normalizeStatus(connection.testStatus);
|
||||
return status === "credits_exhausted" || status === "banned" || status === "expired";
|
||||
}
|
||||
|
||||
export function resolveQuotaLimitPolicy(
|
||||
provider: string,
|
||||
providerSpecificData: JsonRecord
|
||||
): QuotaLimitPolicy {
|
||||
const rawPolicy = asRecord(providerSpecificData.limitPolicy);
|
||||
const rawWindows = Array.isArray(rawPolicy.windows) ? rawPolicy.windows : [];
|
||||
const windows = rawWindows.map(normalizeWindowName).filter(Boolean) as string[];
|
||||
|
||||
if (provider === "codex") {
|
||||
const defaultWindows = applyCodexWindowPolicy(windows, providerSpecificData);
|
||||
const enabled = toBooleanOrDefault(rawPolicy.enabled, defaultWindows.length > 0);
|
||||
|
||||
return {
|
||||
enabled,
|
||||
thresholdPercent: normalizeQuotaThreshold(rawPolicy.thresholdPercent),
|
||||
windows: defaultWindows,
|
||||
};
|
||||
}
|
||||
|
||||
return {
|
||||
enabled: toBooleanOrDefault(rawPolicy.enabled, false),
|
||||
thresholdPercent: normalizeQuotaThreshold(rawPolicy.thresholdPercent),
|
||||
windows,
|
||||
};
|
||||
}
|
||||
|
||||
export function evaluateQuotaLimitPolicy(
|
||||
provider: string,
|
||||
connection: ProviderConnectionView
|
||||
): { blocked: boolean; reasons: string[]; resetAt: string | null } {
|
||||
const policy = resolveQuotaLimitPolicy(provider, connection.providerSpecificData);
|
||||
if (!policy.enabled || policy.windows.length === 0) {
|
||||
return { blocked: false, reasons: [], resetAt: null };
|
||||
}
|
||||
|
||||
const reasons: string[] = [];
|
||||
const resetCandidates: Array<string | null> = [];
|
||||
|
||||
for (const windowName of policy.windows) {
|
||||
const status = getQuotaWindowStatus(connection.id, windowName, policy.thresholdPercent);
|
||||
if (!status?.reachedThreshold) continue;
|
||||
reasons.push(`${windowName} usage ${Math.round(status.usedPercentage)}%`);
|
||||
resetCandidates.push(status.resetAt);
|
||||
}
|
||||
|
||||
return {
|
||||
blocked: reasons.length > 0,
|
||||
reasons,
|
||||
resetAt: getEarliestFutureDate(resetCandidates),
|
||||
};
|
||||
}
|
||||
|
||||
function parseFutureDateMs(value: string | null): number | null {
|
||||
if (!value) return null;
|
||||
const ms = new Date(value).getTime();
|
||||
if (!Number.isFinite(ms) || ms <= Date.now()) return null;
|
||||
return ms;
|
||||
}
|
||||
|
||||
function getEarliestFutureDate(candidates: Array<string | null>): string | null {
|
||||
return (
|
||||
candidates
|
||||
.map((candidate) => ({
|
||||
raw: candidate,
|
||||
ms: parseFutureDateMs(candidate),
|
||||
}))
|
||||
.filter((entry) => entry.ms !== null)
|
||||
.sort((a, b) => (a.ms as number) - (b.ms as number))[0]?.raw || null
|
||||
);
|
||||
}
|
||||
|
||||
// Mutex to prevent race conditions during account selection
|
||||
let selectionMutex = Promise.resolve();
|
||||
|
||||
// ─── Anti-Thundering Herd: per-connection mutex for markAccountUnavailable ───
|
||||
// Prevents multiple concurrent requests from marking the same connection
|
||||
// unavailable in parallel, which was the root cause of cascading 502 lockouts.
|
||||
const markMutexes = new Map<string, Promise<void>>();
|
||||
|
||||
// Strict-Random shuffle deck moved to src/shared/utils/shuffleDeck.ts
|
||||
// auth.ts uses getNextFromDeckSync (already inside selectionMutex).
|
||||
// Re-export for backwards compat with existing test imports.
|
||||
export { fisherYatesShuffle, getNextFromDeckSync as getNextFromDeck };
|
||||
|
||||
/**
|
||||
* Get provider credentials from localDb
|
||||
* Filters out unavailable accounts and returns the selected account based on strategy
|
||||
* @param {string} provider - Provider name
|
||||
* @param {string|null} excludeConnectionId - Connection ID to exclude (for retry with next account)
|
||||
*/
|
||||
export async function getProviderCredentials(
|
||||
provider: string,
|
||||
excludeConnectionId: string | null = null,
|
||||
allowedConnections: string[] | null = null,
|
||||
requestedModel: string | null = null,
|
||||
options: CredentialSelectionOptions = {}
|
||||
) {
|
||||
// Acquire mutex to prevent race conditions
|
||||
const currentMutex = selectionMutex;
|
||||
let resolveMutex: (() => void) | undefined;
|
||||
selectionMutex = new Promise((resolve) => {
|
||||
resolveMutex = resolve;
|
||||
});
|
||||
|
||||
try {
|
||||
await currentMutex;
|
||||
|
||||
const allowSuppressedConnections = options.allowSuppressedConnections === true;
|
||||
const bypassQuotaPolicy = options.bypassQuotaPolicy === true;
|
||||
|
||||
const connectionsRaw = await getProviderConnections({ provider, isActive: true });
|
||||
let connections = (Array.isArray(connectionsRaw) ? connectionsRaw : [])
|
||||
.map(toProviderConnection)
|
||||
.filter((conn) => conn.id.length > 0);
|
||||
// allowedConnections: restrict to specific connection IDs (from API key policy, #363)
|
||||
if (allowedConnections && allowedConnections.length > 0) {
|
||||
connections = connections.filter((conn) => allowedConnections.includes(conn.id));
|
||||
}
|
||||
log.debug(
|
||||
"AUTH",
|
||||
`${provider} | total connections: ${connections.length}, excludeId: ${excludeConnectionId || "none"}`
|
||||
);
|
||||
|
||||
if (connections.length === 0) {
|
||||
// Check all connections (including inactive) to see if rate limited
|
||||
const allConnectionsRaw = await getProviderConnections({ provider });
|
||||
const allConnections = (Array.isArray(allConnectionsRaw) ? allConnectionsRaw : [])
|
||||
.map(toProviderConnection)
|
||||
.filter((conn) => conn.id.length > 0);
|
||||
log.debug("AUTH", `${provider} | all connections (incl inactive): ${allConnections.length}`);
|
||||
if (allConnections.length > 0) {
|
||||
const earliest = getEarliestRateLimitedUntil(allConnections);
|
||||
if (earliest) {
|
||||
log.warn(
|
||||
"AUTH",
|
||||
`${provider} | all ${allConnections.length} accounts rate limited (${formatRetryAfter(earliest)})`
|
||||
);
|
||||
return {
|
||||
allRateLimited: true,
|
||||
retryAfter: earliest,
|
||||
retryAfterHuman: formatRetryAfter(earliest),
|
||||
};
|
||||
}
|
||||
log.warn("AUTH", `${provider} | ${allConnections.length} accounts found but none active`);
|
||||
allConnections.forEach((c) => {
|
||||
log.debug(
|
||||
"AUTH",
|
||||
` → ${c.id?.slice(0, 8)} | isActive=${c.isActive} | rateLimitedUntil=${c.rateLimitedUntil || "none"} | testStatus=${c.testStatus}`
|
||||
);
|
||||
});
|
||||
}
|
||||
log.warn("AUTH", `No credentials for ${provider}`);
|
||||
return null;
|
||||
}
|
||||
|
||||
// Auto-decay backoffLevel for accounts whose rateLimitedUntil has passed.
|
||||
// Without this, high backoffLevel permanently deprioritizes accounts even
|
||||
// after the rate limit window expires, creating a deadlock where the account
|
||||
// needs a successful request to reset but never gets selected.
|
||||
for (const c of connections) {
|
||||
if (
|
||||
c.backoffLevel > 0 &&
|
||||
!isTerminalConnectionStatus(c) &&
|
||||
!isAccountUnavailable(c.rateLimitedUntil)
|
||||
) {
|
||||
c.backoffLevel = 0;
|
||||
updateProviderConnection(c.id, {
|
||||
backoffLevel: 0,
|
||||
testStatus: "active",
|
||||
lastError: null,
|
||||
lastErrorAt: null,
|
||||
lastErrorType: null,
|
||||
lastErrorSource: null,
|
||||
errorCode: null,
|
||||
}).catch(() => {});
|
||||
}
|
||||
}
|
||||
|
||||
// Filter out unavailable accounts and excluded connection
|
||||
const availableConnections = connections.filter((c) => {
|
||||
if (excludeConnectionId && c.id === excludeConnectionId) return false;
|
||||
if (!allowSuppressedConnections) {
|
||||
if (isAccountUnavailable(c.rateLimitedUntil)) return false;
|
||||
if (isTerminalConnectionStatus(c)) return false;
|
||||
if (provider === "codex" && isCodexScopeUnavailable(c, requestedModel)) return false;
|
||||
// Per-model lockout: if this specific model is locked on this connection, skip it
|
||||
if (requestedModel && isModelLocked(provider, c.id, requestedModel)) return false;
|
||||
}
|
||||
return true;
|
||||
});
|
||||
|
||||
log.debug(
|
||||
"AUTH",
|
||||
`${provider} | available: ${availableConnections.length}/${connections.length}`
|
||||
);
|
||||
connections.forEach((c) => {
|
||||
const excluded = excludeConnectionId && c.id === excludeConnectionId;
|
||||
const rateLimited = isAccountUnavailable(c.rateLimitedUntil);
|
||||
const terminalStatus = isTerminalConnectionStatus(c);
|
||||
const codexScopeLimited = provider === "codex" && isCodexScopeUnavailable(c, requestedModel);
|
||||
if (excluded || rateLimited) {
|
||||
log.debug(
|
||||
"AUTH",
|
||||
` → ${c.id?.slice(0, 8)} | ${excluded ? "excluded" : ""} ${rateLimited ? `rateLimited until ${c.rateLimitedUntil}` : ""}${allowSuppressedConnections && rateLimited ? " (retained for combo live test)" : ""}`
|
||||
);
|
||||
} else if (terminalStatus) {
|
||||
log.debug(
|
||||
"AUTH",
|
||||
allowSuppressedConnections
|
||||
? ` → ${c.id?.slice(0, 8)} | retained terminal status=${c.testStatus} for combo live test`
|
||||
: ` → ${c.id?.slice(0, 8)} | skipped terminal status=${c.testStatus}`
|
||||
);
|
||||
} else if (codexScopeLimited) {
|
||||
const scopeUntil = getCodexScopeRateLimitedUntil(c.providerSpecificData, requestedModel);
|
||||
log.debug(
|
||||
"AUTH",
|
||||
allowSuppressedConnections
|
||||
? ` → ${c.id?.slice(0, 8)} | retained codex scope-limited account until ${scopeUntil} for combo live test`
|
||||
: ` → ${c.id?.slice(0, 8)} | codex scope-limited until ${scopeUntil}`
|
||||
);
|
||||
}
|
||||
});
|
||||
|
||||
if (availableConnections.length === 0) {
|
||||
const earliest =
|
||||
getEarliestRateLimitedUntil(connections) ||
|
||||
(provider === "codex"
|
||||
? getEarliestCodexScopeRateLimitedUntil(connections, requestedModel)
|
||||
: null);
|
||||
if (earliest) {
|
||||
// Find the connection with the earliest rateLimitedUntil to get its error info
|
||||
const rateLimitedConns = connections.filter(
|
||||
(c) => c.rateLimitedUntil && new Date(c.rateLimitedUntil).getTime() > Date.now()
|
||||
);
|
||||
const earliestConn = rateLimitedConns.sort(
|
||||
(a, b) =>
|
||||
new Date(a.rateLimitedUntil || 0).getTime() -
|
||||
new Date(b.rateLimitedUntil || 0).getTime()
|
||||
)[0];
|
||||
log.warn(
|
||||
"AUTH",
|
||||
`${provider} | all ${connections.length} active accounts rate limited (${formatRetryAfter(earliest)}) | lastErrorCode=${earliestConn?.errorCode}, lastError=${earliestConn?.lastError?.slice(0, 50)}`
|
||||
);
|
||||
return {
|
||||
allRateLimited: true,
|
||||
retryAfter: earliest,
|
||||
retryAfterHuman: formatRetryAfter(earliest),
|
||||
lastError: earliestConn?.lastError || null,
|
||||
lastErrorCode: earliestConn?.errorCode || null,
|
||||
};
|
||||
}
|
||||
log.warn("AUTH", `${provider} | all ${connections.length} accounts unavailable`);
|
||||
return null;
|
||||
}
|
||||
|
||||
let policyEligibleConnections = availableConnections;
|
||||
const blockedByPolicy: Array<{
|
||||
id: string;
|
||||
reasons: string[];
|
||||
resetAt: string | null;
|
||||
}> = [];
|
||||
|
||||
if (!bypassQuotaPolicy) {
|
||||
policyEligibleConnections = availableConnections.filter((connection) => {
|
||||
const evaluation = evaluateQuotaLimitPolicy(provider, connection);
|
||||
if (!evaluation.blocked) return true;
|
||||
|
||||
blockedByPolicy.push({
|
||||
id: connection.id,
|
||||
reasons: evaluation.reasons,
|
||||
resetAt: evaluation.resetAt,
|
||||
});
|
||||
return false;
|
||||
});
|
||||
} else if (availableConnections.length > 0) {
|
||||
log.debug("AUTH", `${provider} | bypassing quota policy for combo live test`);
|
||||
}
|
||||
|
||||
if (blockedByPolicy.length > 0) {
|
||||
log.info(
|
||||
"AUTH",
|
||||
`${provider} | quota policy filtered ${blockedByPolicy.length} account(s): ${blockedByPolicy
|
||||
.map((entry) => `${entry.id.slice(0, 8)}(${entry.reasons.join(", ")})`)
|
||||
.join("; ")}`
|
||||
);
|
||||
}
|
||||
|
||||
if (policyEligibleConnections.length === 0 && availableConnections.length > 0) {
|
||||
const earliestResetAt = getEarliestFutureDate(blockedByPolicy.map((entry) => entry.resetAt));
|
||||
const earliestResetMs = parseFutureDateMs(earliestResetAt);
|
||||
|
||||
const retryAfter = earliestResetMs
|
||||
? new Date(earliestResetMs).toISOString()
|
||||
: new Date(Date.now() + 5 * 60 * 1000).toISOString();
|
||||
|
||||
return {
|
||||
allRateLimited: true,
|
||||
retryAfter,
|
||||
retryAfterHuman: formatRetryAfter(retryAfter),
|
||||
lastError: `All ${provider} accounts reached configured quota threshold`,
|
||||
lastErrorCode: 429,
|
||||
};
|
||||
}
|
||||
|
||||
// Quota-aware: prioritize accounts with available quota
|
||||
const withQuota = policyEligibleConnections.filter((c) => !isAccountQuotaExhausted(c.id));
|
||||
const exhaustedQuota = policyEligibleConnections.filter((c) => isAccountQuotaExhausted(c.id));
|
||||
const orderedConnections =
|
||||
withQuota.length > 0 ? [...withQuota, ...exhaustedQuota] : policyEligibleConnections;
|
||||
|
||||
if (exhaustedQuota.length > 0) {
|
||||
log.debug(
|
||||
"AUTH",
|
||||
`${provider} | quota-aware: ${withQuota.length} with quota, ${exhaustedQuota.length} exhausted`
|
||||
);
|
||||
}
|
||||
|
||||
const settings = await getSettings();
|
||||
const strategy = settings.fallbackStrategy || "fill-first";
|
||||
|
||||
let connection;
|
||||
if (strategy === "round-robin") {
|
||||
const stickyLimit = toNumber((settings as Record<string, unknown>).stickyRoundRobinLimit, 3);
|
||||
|
||||
// If excluding an account (fallback scenario), skip sticky logic and go straight to LRU
|
||||
// This prevents the system from getting stuck on a failed account
|
||||
const isFallbackScenario = excludeConnectionId !== null;
|
||||
|
||||
if (!isFallbackScenario) {
|
||||
// Sort by lastUsed (most recent first) to find current candidate
|
||||
const byRecency = [...orderedConnections].sort((a: any, b: any) => {
|
||||
if (!a.lastUsedAt && !b.lastUsedAt) return (a.priority || 999) - (b.priority || 999);
|
||||
if (!a.lastUsedAt) return 1;
|
||||
if (!b.lastUsedAt) return -1;
|
||||
return new Date(b.lastUsedAt).getTime() - new Date(a.lastUsedAt).getTime();
|
||||
});
|
||||
|
||||
const current = byRecency[0];
|
||||
const currentCount = current?.consecutiveUseCount || 0;
|
||||
|
||||
if (current && current.lastUsedAt && currentCount < stickyLimit) {
|
||||
// Stay with current account
|
||||
connection = current;
|
||||
log.debug(
|
||||
"AUTH",
|
||||
`${provider} round-robin: staying with ${current.id?.slice(0, 8)}... (count=${currentCount}/${stickyLimit})`
|
||||
);
|
||||
// Update lastUsedAt and increment count (await to ensure persistence)
|
||||
await updateProviderConnection(connection.id, {
|
||||
lastUsedAt: new Date().toISOString(),
|
||||
consecutiveUseCount: (connection.consecutiveUseCount || 0) + 1,
|
||||
});
|
||||
} else {
|
||||
// Pick the least recently used (excluding current if possible)
|
||||
// Also penalize accounts with high backoffLevel (previously rate-limited)
|
||||
// so they don't get immediately re-selected after cooldown (#340)
|
||||
const sortedByOldest = [...orderedConnections].sort((a: any, b: any) => {
|
||||
// Penalize previously rate-limited accounts (backoffLevel > 0)
|
||||
const aBackoff = a.backoffLevel || 0;
|
||||
const bBackoff = b.backoffLevel || 0;
|
||||
if (aBackoff !== bBackoff) return aBackoff - bBackoff; // lower backoff first
|
||||
if (!a.lastUsedAt && !b.lastUsedAt) return (a.priority || 999) - (b.priority || 999);
|
||||
if (!a.lastUsedAt) return -1;
|
||||
if (!b.lastUsedAt) return 1;
|
||||
return new Date(a.lastUsedAt).getTime() - new Date(b.lastUsedAt).getTime();
|
||||
});
|
||||
|
||||
connection = sortedByOldest[0];
|
||||
log.debug(
|
||||
"AUTH",
|
||||
`${provider} round-robin: switching to LRU ${connection.id?.slice(0, 8)}... (current count=${currentCount} >= limit=${stickyLimit} or no lastUsedAt)`
|
||||
);
|
||||
|
||||
// Update lastUsedAt and reset count to 1 (await to ensure persistence)
|
||||
await updateProviderConnection(connection.id, {
|
||||
lastUsedAt: new Date().toISOString(),
|
||||
consecutiveUseCount: 1,
|
||||
});
|
||||
}
|
||||
} else {
|
||||
// Fallback scenario: excluded an account due to failure
|
||||
// Always pick the least recently used to ensure proper cycling
|
||||
// Also penalize accounts with high backoffLevel (#340)
|
||||
const sortedByOldest = [...orderedConnections].sort((a: any, b: any) => {
|
||||
const aBackoff = a.backoffLevel || 0;
|
||||
const bBackoff = b.backoffLevel || 0;
|
||||
if (aBackoff !== bBackoff) return aBackoff - bBackoff;
|
||||
if (!a.lastUsedAt && !b.lastUsedAt) return (a.priority || 999) - (b.priority || 999);
|
||||
if (!a.lastUsedAt) return -1;
|
||||
if (!b.lastUsedAt) return 1;
|
||||
return new Date(a.lastUsedAt).getTime() - new Date(b.lastUsedAt).getTime();
|
||||
});
|
||||
|
||||
connection = sortedByOldest[0];
|
||||
log.info(
|
||||
"AUTH",
|
||||
`${provider} round-robin: FALLBACK MODE - excluded ${excludeConnectionId?.slice(0, 8)}..., picked LRU ${connection.id?.slice(0, 8)}...`
|
||||
);
|
||||
|
||||
// Update lastUsedAt and reset count to 1 (await to ensure persistence)
|
||||
await updateProviderConnection(connection.id, {
|
||||
lastUsedAt: new Date().toISOString(),
|
||||
consecutiveUseCount: 1,
|
||||
});
|
||||
}
|
||||
} else if (strategy === "p2c") {
|
||||
// Power of Two Choices: pick 2 random, choose the one with fewer failures
|
||||
if (orderedConnections.length <= 2) {
|
||||
connection = orderedConnections[0];
|
||||
} else {
|
||||
const i = Math.floor(Math.random() * orderedConnections.length);
|
||||
let j = Math.floor(Math.random() * (orderedConnections.length - 1));
|
||||
if (j >= i) j++;
|
||||
const a = orderedConnections[i];
|
||||
const b = orderedConnections[j];
|
||||
// Prefer the one with fewer consecutive uses / better health
|
||||
const scoreA = (a.consecutiveUseCount || 0) + (a.lastError ? 10 : 0);
|
||||
const scoreB = (b.consecutiveUseCount || 0) + (b.lastError ? 10 : 0);
|
||||
connection = scoreA <= scoreB ? a : b;
|
||||
}
|
||||
} else if (strategy === "random") {
|
||||
// Random: Fisher-Yates-inspired random pick
|
||||
const idx = Math.floor(Math.random() * orderedConnections.length);
|
||||
connection = orderedConnections[idx];
|
||||
} else if (strategy === "least-used") {
|
||||
// Least Used: pick the one with oldest lastUsedAt
|
||||
const sorted = [...orderedConnections].sort((a, b) => {
|
||||
if (!a.lastUsedAt && !b.lastUsedAt) return (a.priority || 999) - (b.priority || 999);
|
||||
if (!a.lastUsedAt) return -1;
|
||||
if (!b.lastUsedAt) return 1;
|
||||
return new Date(a.lastUsedAt).getTime() - new Date(b.lastUsedAt).getTime();
|
||||
});
|
||||
connection = sorted[0];
|
||||
} else if (strategy === "cost-optimized") {
|
||||
// Cost Optimized: sort by priority ascending (lower = cheaper/preferred)
|
||||
// Future: can be enhanced with actual cost data per provider
|
||||
const sorted = [...orderedConnections].sort(
|
||||
(a, b) => (a.priority || 999) - (b.priority || 999)
|
||||
);
|
||||
connection = sorted[0];
|
||||
} else if (strategy === "strict-random") {
|
||||
// Strict Random: shuffle deck — uses each account once before reshuffling
|
||||
const ids = orderedConnections.map((c) => c.id);
|
||||
const selectedId = getNextFromDeckSync(`conn:${provider}`, ids);
|
||||
connection = orderedConnections.find((c) => c.id === selectedId) || orderedConnections[0];
|
||||
} else {
|
||||
// Default: fill-first (already sorted by priority in getProviderConnections)
|
||||
connection = orderedConnections[0];
|
||||
}
|
||||
|
||||
return {
|
||||
apiKey: connection.apiKey,
|
||||
accessToken: connection.accessToken,
|
||||
refreshToken: connection.refreshToken,
|
||||
expiresAt: connection.tokenExpiresAt || connection.expiresAt || null,
|
||||
projectId: connection.projectId,
|
||||
copilotToken:
|
||||
typeof connection.providerSpecificData.copilotToken === "string"
|
||||
? connection.providerSpecificData.copilotToken
|
||||
: null,
|
||||
providerSpecificData: connection.providerSpecificData,
|
||||
connectionId: connection.id,
|
||||
// Include current status for optimization check
|
||||
testStatus: connection.testStatus,
|
||||
lastError: connection.lastError,
|
||||
lastErrorType: connection.lastErrorType,
|
||||
lastErrorSource: connection.lastErrorSource,
|
||||
errorCode: connection.errorCode,
|
||||
rateLimitedUntil: connection.rateLimitedUntil,
|
||||
};
|
||||
} finally {
|
||||
if (resolveMutex) resolveMutex();
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Mark account as unavailable — reads backoffLevel from DB, calculates cooldown with exponential backoff, saves new level
|
||||
* @param {string} connectionId
|
||||
* @param {number} status - HTTP status code
|
||||
* @param {string} errorText - Error message
|
||||
* @param {string|null} provider
|
||||
* @param {string|null} model - Model name for per-model lockout
|
||||
* @returns {{ shouldFallback: boolean, cooldownMs: number }}
|
||||
*/
|
||||
export async function markAccountUnavailable(
|
||||
connectionId: string,
|
||||
status: number,
|
||||
errorText: string,
|
||||
provider: string | null = null,
|
||||
model: string | null = null
|
||||
) {
|
||||
const currentMutex = markMutexes.get(connectionId) || Promise.resolve();
|
||||
let resolveMutex: (() => void) | undefined;
|
||||
markMutexes.set(
|
||||
connectionId,
|
||||
new Promise((resolve) => {
|
||||
resolveMutex = resolve;
|
||||
})
|
||||
);
|
||||
|
||||
try {
|
||||
await currentMutex;
|
||||
|
||||
// ── Per-model lockout for providers with independent model quotas ──
|
||||
// Providers like Gemini AI Studio have per-model quotas. A 429/404 on one
|
||||
// model must NOT lock out other models on the same API key.
|
||||
if (hasPerModelQuota(provider) && model && (status === 429 || status === 404)) {
|
||||
const reason = status === 404 ? "not_found" : "rate_limited";
|
||||
const cooldown = status === 404
|
||||
? COOLDOWN_MS.notFoundLocal
|
||||
: COOLDOWN_MS.rateLimit;
|
||||
lockModel(provider, connectionId, model, reason, cooldown);
|
||||
// Update last error for observability (without changing terminal status)
|
||||
updateProviderConnection(connectionId, {
|
||||
lastErrorType: reason,
|
||||
lastError: `Model ${model} ${reason}`,
|
||||
lastErrorAt: new Date().toISOString(),
|
||||
errorCode: status,
|
||||
}).catch(() => {});
|
||||
log.info(
|
||||
"AUTH",
|
||||
`Model-only lockout for ${provider}:${model} — ${status} ${reason} ${Math.ceil(cooldown / 1000)}s (connection stays active)`
|
||||
);
|
||||
return { shouldFallback: true, cooldownMs: cooldown };
|
||||
}
|
||||
|
||||
// Read current connection to get backoffLevel
|
||||
const connectionsRaw = await getProviderConnections({ provider });
|
||||
const connections = (Array.isArray(connectionsRaw) ? connectionsRaw : [])
|
||||
.map(toProviderConnection)
|
||||
.filter((connection) => connection.id.length > 0);
|
||||
const conn = connections.find((connection) => connection.id === connectionId);
|
||||
const backoffLevel = conn?.backoffLevel || 0;
|
||||
|
||||
// T06/T10/T36: terminal statuses should not be overwritten by transient cooldown state.
|
||||
if (conn && isTerminalConnectionStatus(conn)) {
|
||||
log.info(
|
||||
"AUTH",
|
||||
`${connectionId.slice(0, 8)} terminal status=${conn.testStatus}, skipping cooldown overwrite`
|
||||
);
|
||||
return { shouldFallback: true, cooldownMs: 0 };
|
||||
}
|
||||
|
||||
// ─── Anti-Thundering Herd Guard ─────────────────────────────────
|
||||
// If this connection was ALREADY marked unavailable by a prior concurrent
|
||||
// request (within the mutex window), skip re-marking to avoid resetting
|
||||
// the cooldown timer or double-incrementing the backoff level.
|
||||
if (conn?.rateLimitedUntil && new Date(conn.rateLimitedUntil).getTime() > Date.now()) {
|
||||
log.info(
|
||||
"AUTH",
|
||||
`${connectionId.slice(0, 8)} already marked unavailable (until ${conn.rateLimitedUntil}), skipping duplicate mark`
|
||||
);
|
||||
return {
|
||||
shouldFallback: true,
|
||||
cooldownMs: new Date(conn.rateLimitedUntil).getTime() - Date.now(),
|
||||
};
|
||||
}
|
||||
|
||||
// T09: Codex scope-aware lockout guard (codex vs spark independent pools).
|
||||
if (provider === "codex" && model) {
|
||||
const scopeRateLimitedUntil = getCodexScopeRateLimitedUntil(
|
||||
conn?.providerSpecificData || {},
|
||||
model
|
||||
);
|
||||
if (scopeRateLimitedUntil && new Date(scopeRateLimitedUntil).getTime() > Date.now()) {
|
||||
log.info(
|
||||
"AUTH",
|
||||
`${connectionId.slice(0, 8)} already scope-limited for ${getCodexModelScope(model)} (until ${scopeRateLimitedUntil}), skipping duplicate mark`
|
||||
);
|
||||
return {
|
||||
shouldFallback: true,
|
||||
cooldownMs: new Date(scopeRateLimitedUntil).getTime() - Date.now(),
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
const result = checkFallbackError(
|
||||
status,
|
||||
errorText,
|
||||
backoffLevel,
|
||||
model,
|
||||
provider // ← Now passes provider for profile-aware cooldowns
|
||||
);
|
||||
const { shouldFallback, cooldownMs, newBackoffLevel, reason } = result;
|
||||
if (!shouldFallback) return { shouldFallback: false, cooldownMs: 0 };
|
||||
|
||||
// ── 404 model-only lockout: connection stays active ──
|
||||
// For local providers (detected by URL) and cloud providers with passthrough models
|
||||
// (like Antigravity), a 404 means the specific model doesn't exist or isn't available
|
||||
// for this account — it should NOT lock out the entire connection.
|
||||
const connBaseUrl = (conn?.providerSpecificData as Record<string, unknown>)?.baseUrl as
|
||||
| string
|
||||
| undefined;
|
||||
|
||||
const isPassthroughProvider = provider && getPassthroughProviders().has(provider);
|
||||
const isPerModelQuotaProvider = hasPerModelQuota(provider);
|
||||
if ((isLocalProvider(connBaseUrl) || isPerModelQuotaProvider) && status === 404 && provider && model) {
|
||||
const localCooldown = COOLDOWN_MS.notFoundLocal;
|
||||
lockModel(provider, connectionId, model, "not_found", localCooldown);
|
||||
log.info(
|
||||
"AUTH",
|
||||
`Model-only lockout for ${model} — 404 lockout ${localCooldown / 1000}s (connection stays active)`
|
||||
);
|
||||
return { shouldFallback: true, cooldownMs: localCooldown };
|
||||
}
|
||||
|
||||
// ── 429 model-only lockout for per-model quota providers ──
|
||||
// For providers where each model has independent quota (passthrough providers,
|
||||
// Gemini AI Studio), a 429 on one model should NOT lock out the entire connection
|
||||
// — other models may still have quota available. Use lockModel() instead of
|
||||
// connection-wide rateLimitedUntil.
|
||||
if (isPerModelQuotaProvider && status === 429 && provider && model) {
|
||||
const modelCooldown = cooldownMs || COOLDOWN_MS.rateLimit;
|
||||
lockModel(provider, connectionId, model, reason || "rate_limited", modelCooldown);
|
||||
log.info(
|
||||
"AUTH",
|
||||
`Model-only lockout for ${model} — 429 rate limit ${Math.ceil(modelCooldown / 1000)}s (connection stays active)`
|
||||
);
|
||||
return { shouldFallback: true, cooldownMs: modelCooldown };
|
||||
}
|
||||
|
||||
const rateLimitedUntil = getUnavailableUntil(cooldownMs);
|
||||
const errorMsg = typeof errorText === "string" ? errorText.slice(0, 100) : "Provider error";
|
||||
|
||||
// T09: Codex per-scope lockout (do not block the whole account globally).
|
||||
if (provider === "codex" && status === 429 && model && conn) {
|
||||
const scope = getCodexModelScope(model);
|
||||
const existingScopeMap = asRecord(conn.providerSpecificData.codexScopeRateLimitedUntil);
|
||||
const persistedScopeUntil = getCodexScopeRateLimitedUntil(conn.providerSpecificData, model);
|
||||
const scopeRateLimitedUntil = persistedScopeUntil || rateLimitedUntil;
|
||||
const scopeCooldownMs = Math.max(new Date(scopeRateLimitedUntil).getTime() - Date.now(), 0);
|
||||
|
||||
await updateProviderConnection(connectionId, {
|
||||
testStatus: "unavailable",
|
||||
lastError: errorMsg,
|
||||
errorCode: status,
|
||||
lastErrorAt: new Date().toISOString(),
|
||||
backoffLevel: newBackoffLevel ?? backoffLevel,
|
||||
providerSpecificData: {
|
||||
...conn.providerSpecificData,
|
||||
codexScopeRateLimitedUntil: {
|
||||
...existingScopeMap,
|
||||
[scope]: scopeRateLimitedUntil,
|
||||
},
|
||||
},
|
||||
});
|
||||
|
||||
if (scopeCooldownMs > 0) {
|
||||
lockModel(provider, connectionId, model, reason || "unknown", scopeCooldownMs);
|
||||
}
|
||||
|
||||
if (status && errorMsg) {
|
||||
console.error(`❌ ${provider} [${status}] (${scope}): ${errorMsg}`);
|
||||
}
|
||||
|
||||
return { shouldFallback: true, cooldownMs: scopeCooldownMs };
|
||||
}
|
||||
|
||||
await updateProviderConnection(connectionId, {
|
||||
rateLimitedUntil,
|
||||
testStatus: "unavailable",
|
||||
lastError: errorMsg,
|
||||
errorCode: status,
|
||||
lastErrorAt: new Date().toISOString(),
|
||||
backoffLevel: newBackoffLevel ?? backoffLevel,
|
||||
});
|
||||
|
||||
// T-AUTODISABLE: If auto-disable setting is enabled and error is permanent/terminal,
|
||||
// mark account as inactive so it is never retried again.
|
||||
// Uses getCachedSettings() to avoid DB overhead on hot error path.
|
||||
// NOTE: For permanent bans we disable immediately — no threshold needed,
|
||||
// because a permanent ban (403 "Verify your account" / ToS violation) will
|
||||
// NEVER recover, so retrying is pointless regardless of attempt count.
|
||||
if (result.permanent) {
|
||||
try {
|
||||
const settings = await getCachedSettings();
|
||||
const autoDisableEnabled = settings.autoDisableBannedAccounts ?? false;
|
||||
if (autoDisableEnabled) {
|
||||
await updateProviderConnection(connectionId, { isActive: false });
|
||||
log.info(
|
||||
"AUTH",
|
||||
`Auto-disabled ${connectionId.slice(0, 8)} — permanent ban detected (autoDisableBannedAccounts=true)`
|
||||
);
|
||||
}
|
||||
} catch (e) {
|
||||
log.info("AUTH", `Auto-disable check failed (non-fatal): ${e}`);
|
||||
}
|
||||
}
|
||||
|
||||
// Per-model lockout: lock the specific model if known
|
||||
if (provider && model && cooldownMs > 0) {
|
||||
lockModel(provider, connectionId, model, reason || "unknown", cooldownMs);
|
||||
}
|
||||
|
||||
if (provider && status && errorMsg) {
|
||||
console.error(`❌ ${provider} [${status}]: ${errorMsg}`);
|
||||
}
|
||||
|
||||
return { shouldFallback: true, cooldownMs };
|
||||
} finally {
|
||||
if (resolveMutex) resolveMutex();
|
||||
// Cleanup stale mutex entries (avoid memory leak)
|
||||
markMutexes.delete(connectionId);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Clear account error status (only if currently has error)
|
||||
* Optimized to avoid unnecessary DB updates
|
||||
*/
|
||||
export async function clearAccountError(
|
||||
connectionId: string,
|
||||
currentConnection: Partial<RecoverableConnectionState>
|
||||
) {
|
||||
// Only update if currently has error status
|
||||
const hasError =
|
||||
(currentConnection.testStatus && currentConnection.testStatus !== "active") ||
|
||||
currentConnection.lastError ||
|
||||
currentConnection.rateLimitedUntil ||
|
||||
currentConnection.errorCode ||
|
||||
currentConnection.lastErrorType ||
|
||||
currentConnection.lastErrorSource;
|
||||
|
||||
if (!hasError) return; // Skip if already clean
|
||||
|
||||
await updateProviderConnection(connectionId, {
|
||||
testStatus: "active",
|
||||
lastError: null,
|
||||
lastErrorAt: null,
|
||||
lastErrorType: null,
|
||||
lastErrorSource: null,
|
||||
errorCode: null,
|
||||
rateLimitedUntil: null,
|
||||
backoffLevel: 0,
|
||||
});
|
||||
log.info("AUTH", `Account ${connectionId.slice(0, 8)} error cleared`);
|
||||
}
|
||||
|
||||
export async function clearRecoveredProviderState(
|
||||
credentials: Partial<RecoverableConnectionState> | null
|
||||
) {
|
||||
if (!credentials?.connectionId) return;
|
||||
await clearAccountError(credentials.connectionId, credentials);
|
||||
}
|
||||
|
||||
/**
|
||||
* Extract API key from request headers
|
||||
*/
|
||||
export function extractApiKey(request: Request) {
|
||||
const authHeader = request.headers.get("Authorization");
|
||||
if (authHeader?.startsWith("Bearer ")) {
|
||||
return authHeader.slice(7);
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Validate API key (optional - for local use can skip)
|
||||
*/
|
||||
export async function isValidApiKey(apiKey: string) {
|
||||
if (!apiKey) return false;
|
||||
return await validateApiKey(apiKey);
|
||||
}
|
||||
3923
test-mcp-bundle.js
Normal file
3923
test-mcp-bundle.js
Normal file
File diff suppressed because it is too large
Load Diff
67
tests/unit/cc-compatible-model-catalog.test.mjs
Normal file
67
tests/unit/cc-compatible-model-catalog.test.mjs
Normal file
@@ -0,0 +1,67 @@
|
||||
import test from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import fs from "node:fs";
|
||||
import os from "node:os";
|
||||
import path from "node:path";
|
||||
|
||||
const TEST_DATA_DIR = fs.mkdtempSync(path.join(os.tmpdir(), "omniroute-cc-models-"));
|
||||
process.env.DATA_DIR = TEST_DATA_DIR;
|
||||
|
||||
const core = await import("../../src/lib/db/core.ts");
|
||||
const providersDb = await import("../../src/lib/db/providers.ts");
|
||||
const v1ModelsCatalog = await import("../../src/app/api/v1/models/catalog.ts");
|
||||
|
||||
async function resetStorage() {
|
||||
core.resetDbInstance();
|
||||
fs.rmSync(TEST_DATA_DIR, { recursive: true, force: true });
|
||||
fs.mkdirSync(TEST_DATA_DIR, { recursive: true });
|
||||
}
|
||||
|
||||
test.afterEach(async () => {
|
||||
await resetStorage();
|
||||
});
|
||||
|
||||
test.after(() => {
|
||||
core.resetDbInstance();
|
||||
fs.rmSync(TEST_DATA_DIR, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
test("v1 models exposes CC-compatible fallback models under the provider node prefix", async () => {
|
||||
await providersDb.createProviderNode({
|
||||
id: "anthropic-compatible-cc-cm",
|
||||
type: "anthropic-compatible",
|
||||
name: "Claude Max",
|
||||
prefix: "cm",
|
||||
baseUrl: "https://proxy.example.com",
|
||||
chatPath: "/v1/messages?beta=true",
|
||||
modelsPath: "/v1/models",
|
||||
});
|
||||
|
||||
await providersDb.createProviderConnection({
|
||||
provider: "anthropic-compatible-cc-cm",
|
||||
authType: "apikey",
|
||||
name: "cm-main",
|
||||
apiKey: "sk-test",
|
||||
isActive: true,
|
||||
providerSpecificData: {
|
||||
baseUrl: "https://proxy.example.com",
|
||||
chatPath: "/v1/messages?beta=true",
|
||||
modelsPath: "/v1/models",
|
||||
},
|
||||
});
|
||||
|
||||
const response = await v1ModelsCatalog.getUnifiedModelsResponse(
|
||||
new Request("http://localhost/api/v1/models", { method: "GET" })
|
||||
);
|
||||
|
||||
assert.equal(response.status, 200);
|
||||
const body = await response.json();
|
||||
const ids = new Set(body.data.map((item) => item.id));
|
||||
|
||||
assert.ok(ids.has("cm/claude-opus-4-6"));
|
||||
assert.ok(ids.has("cm/claude-sonnet-4-6"));
|
||||
assert.equal(
|
||||
[...ids].some((id) => id.startsWith("anthropic-compatible-cc-cm/")),
|
||||
false
|
||||
);
|
||||
});
|
||||
@@ -104,11 +104,14 @@ test("buildClaudeCodeCompatibleRequest keeps prior role history while dropping t
|
||||
{ role: "user", text: "u2" },
|
||||
]
|
||||
);
|
||||
assert.deepEqual(payload.messages.at(-1).content.at(-1).cache_control, { type: "ephemeral" });
|
||||
assert.deepEqual(payload.messages[0].content.at(-1).cache_control, { type: "ephemeral" });
|
||||
assert.deepEqual(payload.messages[1].content.at(-1).cache_control, { type: "ephemeral" });
|
||||
assert.equal(payload.messages[2].content.at(-1).cache_control, undefined);
|
||||
assert.equal(payload.system.length, 4);
|
||||
assert.equal(payload.system.at(-1).text, "sys");
|
||||
assert.equal(payload.tools.length, 1);
|
||||
assert.deepEqual(payload.tools[0], {
|
||||
const { cache_control, ...toolWithoutCacheControl } = payload.tools[0];
|
||||
assert.deepEqual(toolWithoutCacheControl, {
|
||||
name: "lookup_weather",
|
||||
description: "Fetch weather",
|
||||
input_schema: {
|
||||
@@ -119,11 +122,70 @@ test("buildClaudeCodeCompatibleRequest keeps prior role history while dropping t
|
||||
required: ["city"],
|
||||
},
|
||||
});
|
||||
assert.deepEqual(payload.tools[0].cache_control, { type: "ephemeral", ttl: "1h" });
|
||||
assert.deepEqual(payload.tool_choice, { type: "any" });
|
||||
assert.equal(payload.context_management.edits[0].type, "clear_thinking_20251015");
|
||||
assert.equal(JSON.parse(payload.metadata.user_id).session_id, "session-1");
|
||||
});
|
||||
|
||||
test("buildClaudeCodeCompatibleRequest preserves Claude cache markers when requested", () => {
|
||||
const payload = buildClaudeCodeCompatibleRequest({
|
||||
sourceBody: {
|
||||
max_tokens: 64,
|
||||
},
|
||||
normalizedBody: {
|
||||
max_tokens: 64,
|
||||
messages: [{ role: "user", content: "fallback" }],
|
||||
},
|
||||
claudeBody: {
|
||||
system: [{ type: "text", text: "sys", cache_control: { type: "ephemeral", ttl: "5m" } }],
|
||||
messages: [
|
||||
{
|
||||
role: "user",
|
||||
content: [{ type: "text", text: "u1", cache_control: { type: "ephemeral" } }],
|
||||
},
|
||||
{
|
||||
role: "assistant",
|
||||
content: [
|
||||
{
|
||||
type: "text",
|
||||
text: "a1",
|
||||
cache_control: { type: "ephemeral", ttl: "10m" },
|
||||
},
|
||||
],
|
||||
},
|
||||
{ role: "user", content: [{ type: "text", text: "u2" }] },
|
||||
],
|
||||
tools: [
|
||||
{
|
||||
name: "lookup_weather",
|
||||
description: "Fetch weather",
|
||||
input_schema: {
|
||||
type: "object",
|
||||
properties: {
|
||||
city: { type: "string" },
|
||||
},
|
||||
required: ["city"],
|
||||
},
|
||||
cache_control: { type: "ephemeral", ttl: "30m" },
|
||||
},
|
||||
],
|
||||
},
|
||||
model: "claude-sonnet-4-6",
|
||||
sessionId: "session-preserve",
|
||||
preserveCacheControl: true,
|
||||
});
|
||||
|
||||
assert.deepEqual(payload.system.at(-1).cache_control, { type: "ephemeral", ttl: "5m" });
|
||||
assert.deepEqual(payload.messages[0].content[0].cache_control, { type: "ephemeral" });
|
||||
assert.deepEqual(payload.messages[1].content[0].cache_control, {
|
||||
type: "ephemeral",
|
||||
ttl: "10m",
|
||||
});
|
||||
assert.equal(payload.messages[2].content[0].cache_control, undefined);
|
||||
assert.deepEqual(payload.tools[0].cache_control, { type: "ephemeral", ttl: "30m" });
|
||||
});
|
||||
|
||||
test("buildClaudeCodeCompatibleRequest falls back to a user turn when the source only has assistant/model text", () => {
|
||||
const payload = buildClaudeCodeCompatibleRequest({
|
||||
sourceBody: {
|
||||
@@ -181,6 +243,7 @@ test("buildClaudeCodeCompatibleRequest omits auto tool_choice while preserving t
|
||||
});
|
||||
|
||||
assert.equal(payload.tools.length, 1);
|
||||
assert.deepEqual(payload.tools[0].cache_control, { type: "ephemeral", ttl: "1h" });
|
||||
assert.equal(payload.tool_choice, undefined);
|
||||
});
|
||||
|
||||
@@ -356,6 +419,126 @@ test("handleChatCore forces upstream streaming for CC compatible while returning
|
||||
assert.equal(payload.usage.completion_tokens, 5);
|
||||
});
|
||||
|
||||
test("handleChatCore preserves Claude cache_control when CC-compatible requests originate from Claude", async () => {
|
||||
const calls = [];
|
||||
globalThis.fetch = async (url, init = {}) => {
|
||||
calls.push({
|
||||
url: String(url),
|
||||
method: init.method || "GET",
|
||||
headers: init.headers,
|
||||
body: init.body ? JSON.parse(String(init.body)) : null,
|
||||
});
|
||||
|
||||
return new Response(
|
||||
[
|
||||
"event: message_start",
|
||||
'data: {"type":"message_start","message":{"id":"msg_2","type":"message","role":"assistant","model":"claude-sonnet-4-6","usage":{"input_tokens":12,"output_tokens":0}}}',
|
||||
"",
|
||||
"event: content_block_start",
|
||||
'data: {"type":"content_block_start","index":0,"content_block":{"type":"text","text":""}}',
|
||||
"",
|
||||
"event: content_block_delta",
|
||||
'data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":"Preserved"}}',
|
||||
"",
|
||||
"event: message_delta",
|
||||
'data: {"type":"message_delta","delta":{"stop_reason":"end_turn"},"usage":{"output_tokens":3}}',
|
||||
"",
|
||||
"event: message_stop",
|
||||
'data: {"type":"message_stop"}',
|
||||
"",
|
||||
].join("\n"),
|
||||
{
|
||||
status: 200,
|
||||
headers: {
|
||||
"content-type": "text/event-stream",
|
||||
},
|
||||
}
|
||||
);
|
||||
};
|
||||
|
||||
const claudeBody = {
|
||||
model: "claude-sonnet-4-6",
|
||||
max_tokens: 64,
|
||||
system: [{ type: "text", text: "system", cache_control: { type: "ephemeral", ttl: "5m" } }],
|
||||
messages: [
|
||||
{
|
||||
role: "user",
|
||||
content: [{ type: "text", text: "u1", cache_control: { type: "ephemeral" } }],
|
||||
},
|
||||
{
|
||||
role: "assistant",
|
||||
content: [
|
||||
{
|
||||
type: "text",
|
||||
text: "a1",
|
||||
cache_control: { type: "ephemeral", ttl: "10m" },
|
||||
},
|
||||
],
|
||||
},
|
||||
{ role: "user", content: [{ type: "text", text: "u2" }] },
|
||||
],
|
||||
tools: [
|
||||
{
|
||||
name: "lookup_weather",
|
||||
description: "Fetch weather",
|
||||
input_schema: {
|
||||
type: "object",
|
||||
properties: {
|
||||
city: { type: "string" },
|
||||
},
|
||||
},
|
||||
cache_control: { type: "ephemeral", ttl: "30m" },
|
||||
},
|
||||
],
|
||||
};
|
||||
|
||||
const result = await handleChatCore({
|
||||
body: claudeBody,
|
||||
modelInfo: {
|
||||
provider: "anthropic-compatible-cc-test",
|
||||
model: "claude-sonnet-4-6",
|
||||
extendedContext: false,
|
||||
},
|
||||
credentials: {
|
||||
apiKey: "sk-test",
|
||||
providerSpecificData: {
|
||||
baseUrl: "https://proxy.example.com",
|
||||
chatPath: CLAUDE_CODE_COMPATIBLE_DEFAULT_CHAT_PATH,
|
||||
},
|
||||
},
|
||||
clientRawRequest: {
|
||||
endpoint: "/v1/messages",
|
||||
body: claudeBody,
|
||||
headers: new Headers({ accept: "application/json" }),
|
||||
},
|
||||
userAgent: "Claude-Code/1.0.0",
|
||||
log: {
|
||||
debug() {},
|
||||
info() {},
|
||||
warn() {},
|
||||
error() {},
|
||||
},
|
||||
});
|
||||
|
||||
assert.equal(result.success, true);
|
||||
assert.equal(calls.length, 1);
|
||||
assert.deepEqual(calls[0].body.system.at(-1).cache_control, {
|
||||
type: "ephemeral",
|
||||
ttl: "5m",
|
||||
});
|
||||
assert.deepEqual(calls[0].body.messages[0].content[0].cache_control, {
|
||||
type: "ephemeral",
|
||||
});
|
||||
assert.deepEqual(calls[0].body.messages[1].content[0].cache_control, {
|
||||
type: "ephemeral",
|
||||
ttl: "10m",
|
||||
});
|
||||
assert.deepEqual(calls[0].body.tools[0].cache_control, {
|
||||
type: "ephemeral",
|
||||
ttl: "30m",
|
||||
});
|
||||
});
|
||||
|
||||
test("provider-nodes create route rejects CC mode when feature flag is disabled", async () => {
|
||||
delete process.env.ENABLE_CC_COMPATIBLE_PROVIDER;
|
||||
|
||||
|
||||
55
tests/unit/claude-oauth-provider.test.mjs
Normal file
55
tests/unit/claude-oauth-provider.test.mjs
Normal file
@@ -0,0 +1,55 @@
|
||||
import test from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
|
||||
import { claude } from "../../src/lib/oauth/providers/claude.ts";
|
||||
import { CLAUDE_CONFIG } from "../../src/lib/oauth/constants/oauth.ts";
|
||||
|
||||
const originalFetch = globalThis.fetch;
|
||||
|
||||
test.afterEach(() => {
|
||||
globalThis.fetch = originalFetch;
|
||||
});
|
||||
|
||||
test("Claude OAuth provider uses the runtime redirectUri when building the auth URL", () => {
|
||||
const redirectUri = "http://localhost:43121/callback";
|
||||
const authUrl = claude.buildAuthUrl(CLAUDE_CONFIG, redirectUri, "state-123", "challenge-456");
|
||||
const parsed = new URL(authUrl);
|
||||
|
||||
assert.equal(parsed.searchParams.get("redirect_uri"), redirectUri);
|
||||
assert.equal(parsed.searchParams.get("state"), "state-123");
|
||||
assert.equal(parsed.searchParams.get("code_challenge"), "challenge-456");
|
||||
});
|
||||
|
||||
test("Claude OAuth provider uses the runtime redirectUri during token exchange", async () => {
|
||||
let captured = null;
|
||||
|
||||
globalThis.fetch = async (url, init = {}) => {
|
||||
captured = {
|
||||
url: String(url),
|
||||
method: init.method,
|
||||
headers: init.headers,
|
||||
body: JSON.parse(String(init.body)),
|
||||
};
|
||||
|
||||
return new Response(JSON.stringify({ access_token: "token-1" }), {
|
||||
status: 200,
|
||||
headers: { "Content-Type": "application/json" },
|
||||
});
|
||||
};
|
||||
|
||||
const redirectUri = "http://localhost:43121/callback";
|
||||
await claude.exchangeToken(
|
||||
CLAUDE_CONFIG,
|
||||
"auth-code#state-from-fragment",
|
||||
redirectUri,
|
||||
"verifier-123",
|
||||
"state-from-request"
|
||||
);
|
||||
|
||||
assert.equal(captured.url, CLAUDE_CONFIG.tokenUrl);
|
||||
assert.equal(captured.method, "POST");
|
||||
assert.equal(captured.body.redirect_uri, redirectUri);
|
||||
assert.equal(captured.body.code, "auth-code");
|
||||
assert.equal(captured.body.state, "state-from-fragment");
|
||||
assert.equal(captured.body.code_verifier, "verifier-123");
|
||||
});
|
||||
@@ -82,3 +82,30 @@ test("T27: SSE [DONE] guard applies only in streaming mode", async () => {
|
||||
BaseExecutor.prototype.execute = originalExecute;
|
||||
}
|
||||
});
|
||||
|
||||
test("T27: streaming error responses keep their original body readable", async () => {
|
||||
const executor = new GithubExecutor();
|
||||
const originalExecute = BaseExecutor.prototype.execute;
|
||||
|
||||
BaseExecutor.prototype.execute = async () => ({
|
||||
response: new Response("IDE token expired: unauthorized: token expired\n", {
|
||||
status: 401,
|
||||
headers: { "content-type": "text/plain; charset=utf-8" },
|
||||
}),
|
||||
url: "https://api.githubcopilot.com/chat/completions",
|
||||
});
|
||||
|
||||
try {
|
||||
const result = await executor.execute({
|
||||
model: "claude-sonnet-4.5",
|
||||
body: { messages: [] },
|
||||
stream: true,
|
||||
credentials: { accessToken: "token" },
|
||||
});
|
||||
|
||||
assert.equal(result.response.status, 401);
|
||||
assert.equal(await result.response.text(), "IDE token expired: unauthorized: token expired\n");
|
||||
} finally {
|
||||
BaseExecutor.prototype.execute = originalExecute;
|
||||
}
|
||||
});
|
||||
|
||||
@@ -5,12 +5,14 @@ import { getModelInfoCore } from "../../open-sse/services/model.ts";
|
||||
import { REGISTRY } from "../../open-sse/config/providerRegistry.ts";
|
||||
import { getStaticModelsForProvider } from "../../src/app/api/providers/[id]/models/route.ts";
|
||||
|
||||
test("T28: gemini catalog includes preview models from 9router", () => {
|
||||
test("T28: gemini-cli catalog includes preview models, gemini uses API sync", () => {
|
||||
// Gemini (AI Studio) no longer has a hardcoded registry — models come from
|
||||
// API sync via /api/providers/:id/models with pageSize=1000.
|
||||
const geminiIds = REGISTRY.gemini.models.map((m) => m.id);
|
||||
const geminiCliIds = REGISTRY["gemini-cli"].models.map((m) => m.id);
|
||||
assert.equal(geminiIds.length, 0, "gemini models should be empty (populated by API sync)");
|
||||
|
||||
assert.ok(geminiIds.includes("gemini-3.1-flash-lite-preview"));
|
||||
assert.ok(geminiIds.includes("gemini-3-flash-preview"));
|
||||
// gemini-cli still has hardcoded models (Cloud Code doesn't have a models API)
|
||||
const geminiCliIds = REGISTRY["gemini-cli"].models.map((m) => m.id);
|
||||
assert.ok(geminiCliIds.includes("gemini-3.1-flash-lite-preview"));
|
||||
assert.ok(geminiCliIds.includes("gemini-3-flash-preview"));
|
||||
});
|
||||
|
||||
@@ -2,6 +2,7 @@ import test from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
|
||||
const { REGISTRY } = await import("../../open-sse/config/providerRegistry.ts");
|
||||
const { getStaticModelsForProvider } = await import("../../src/app/api/providers/[id]/models/route.ts");
|
||||
const { resolveModelAlias: resolveDeprecatedAlias } =
|
||||
await import("../../open-sse/services/modelDeprecation.ts");
|
||||
const { normalizeThinkingLevel } = await import("../../open-sse/services/thinkingBudget.ts");
|
||||
@@ -14,10 +15,12 @@ const {
|
||||
capThinkingBudget,
|
||||
} = await import("../../src/shared/constants/modelSpecs.ts");
|
||||
|
||||
test("T31: registry exposes Gemini 3.1 Pro High/Low model IDs", () => {
|
||||
const geminiIds = REGISTRY.gemini.models.map((m) => m.id);
|
||||
assert.ok(geminiIds.includes("gemini-3.1-pro-high"));
|
||||
assert.ok(geminiIds.includes("gemini-3.1-pro-low"));
|
||||
test("T31: antigravity static catalog exposes Gemini 3.1 Pro High/Low model IDs", () => {
|
||||
// gemini-3.1-pro-high/low are Antigravity (Cloud Code sandbox) models,
|
||||
// not Gemini AI Studio models. They live in the static catalog, not the registry.
|
||||
const staticIds = (getStaticModelsForProvider("antigravity") || []).map((m) => m.id);
|
||||
assert.ok(staticIds.includes("gemini-3.1-pro-high"));
|
||||
assert.ok(staticIds.includes("gemini-3.1-pro-low"));
|
||||
});
|
||||
|
||||
test("T31: legacy Gemini aliases resolve to Gemini 3.1 IDs", () => {
|
||||
|
||||
Reference in New Issue
Block a user