feat: v2.0.0 - MCP server, A2A agent, proxy improvements and docs update

This commit is contained in:
diegosouzapw
2026-03-05 01:16:56 -03:00
parent 0d8f28a4a4
commit baa0208fa9
64 changed files with 2733 additions and 434 deletions

View File

@@ -9,26 +9,36 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]
> ### ✨ Major Feature Release — MCP Server, A2A Protocol, Auto-Combo Engine & VS Code Extension
_No unreleased changes._
---
## [2.0.0] — 2026-03-04
> ### 🚀 Major Release — MCP Server, A2A Protocol, Auto-Combo Engine & Full Type Safety Overhaul
>
> Full AI orchestration ecosystem: 16 MCP tools, A2A v0.3 server, self-healing Auto-Combo engine, and a VS Code extension with smart dispatch, budget tracking, and human checkpoints.
> **OmniRoute 2.0** transforms the AI gateway into a fully **agent-controllable platform**. AI agents can now discover, orchestrate, and optimize routing through 16 MCP tools or the A2A v0.3 protocol. Accompanied by a self-healing Auto-Combo engine, VS Code extension, 3 new dashboard pages, and a comprehensive type safety overhaul across the entire codebase.
### 🆕 MCP Server (16 Tools)
- **8 Essential Tools** — `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`
- **8 Advanced Tools** — `simulate_route`, `set_budget_guard`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`
- **Scoped Authorization** — 8 permission scopes (read:health, write:combo, etc.)
- **Audit Logging** — Every tool call logged with duration, arguments, and result
- **IDE Configs** — MCP configuration templates for Antigravity, Cursor, Copilot, Claude Desktop
- **Scoped Authorization** — 9 permission scopes (`read:health`, `read:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `write:combos`, `write:budget`, `write:resilience`) with wildcard support
- **Audit Logging** — Every tool call logged to SQLite with SHA-256 input hashing, output summarization, and duration tracking
- **IDE Configs** — MCP configuration templates for Claude Desktop, Cursor, VS Code Copilot, and stdio transport
- **Type-Safe Schemas** — All 16 tools defined with Zod input/output schemas, descriptions, and scope declarations
- 📖 **Documentation** — [`open-sse/mcp-server/README.md`](open-sse/mcp-server/README.md) with architecture, tool reference, and client examples in Python, TypeScript, and Go
### 🤖 A2A Server (Agent-to-Agent v0.3)
- **JSON-RPC 2.0** — Full router with `message/send`, `message/stream`, `tasks/get`, `tasks/cancel`
- **Agent Card** — Dynamic `/.well-known/agent.json` with 2 skills
- **Skills** — `smart-routing` (routing explanation, cost envelope, resilience trace, policy verdict) and `quota-management` (natural language quota queries)
- **SSE Streaming** — Real-time task streaming with 15s heartbeat
- **Task Manager** — State machine (submittedworkingcompleted/failed/canceled), TTL, cleanup
- **Routing Logger** — Decision audit trail with 7-day retention
- **Agent Card** — Dynamic `/.well-known/agent.json` with 2 skills and bearer auth
- **Skills** — `smart-routing` (routing explanation, cost envelope, resilience trace, policy verdict) and `quota-management` (natural language quota queries with ranking, free combo suggestions, and full summaries)
- **SSE Streaming** — Real-time task streaming with 15s heartbeat, chunk events, and completion metadata
- **Task Manager** — State machine (`submitted``working``completed`/`failed`/`cancelled`), TTL (5min default), auto-cleanup (2× TTL)
- **Routing Logger** — Decision audit trail with 7-day retention and routing statistics
- **Task Execution** — Generic executor with proper state transitions on success/failure
- 📖 **Documentation** — [`src/lib/a2a/README.md`](src/lib/a2a/README.md) with JSON-RPC methods, skill reference, client examples, and MCP vs A2A comparison
### ⚡ Auto-Combo Engine
@@ -56,26 +66,65 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- **MCP Dashboard** — Tool listing, usage stats, audit log with 30s auto-refresh
- **A2A Dashboard** — Agent Card display, skill listing, task history with routing metadata
- **Auto-Combo Dashboard** — Provider score bars, factor breakdown, mode pack selector, incident indicator, exclusion list
- **Error Pages** — Custom error and not-found pages for the dashboard
### 🔗 Integrations
- **OpenClaw** — Dynamic `provider.order` endpoint at `/api/cli-tools/openclaw/auto-order`
### 🔧 Code Quality & Type Safety
- **Eliminated `any` types** — Replaced `any` casts across `open-sse/` services, translators, and handlers with proper generics and explicit types
- **Zod Validation Schemas** — Added Zod-based validation for all MCP tool inputs/outputs and API validation layer
- **Shared Contracts** — Normalized quota and combos API responses with shared contracts (`src/shared/contracts/quota.ts`) for consistent data shapes across MCP, A2A, and REST APIs
- **TypeScript Translator Types** — Added strict types and modularized the translator registry with proper interfaces
- **DB Layer Hardening** — Improved database layer with proper error handling and type safety in the compliance module
- **A2A Lifecycle Safety** — Enhanced A2A task lifecycle with type-safe state transitions, preventing invalid state changes on completed tasks
- **Stream Handling** — Improved ComfyUI and stream handling with proper type safety
### 🧪 Tests
- **E2E Test Suite** — 6 scenarios (MCP, A2A, Auto-Combo, OpenClaw, Stress 100+50 parallel, Security)
- **Unit Tests** — Essential tools, advanced tools, extension services, Auto-Combo engine, extension advanced features
- **E2E Test Suite** — 6 scenarios covering MCP, A2A, Auto-Combo, OpenClaw, Stress (100+50 parallel), Security
- **Unit Tests** — Essential tools (139 tests), advanced tools (141 tests), Auto-Combo engine (162 tests), A2A lifecycle regression tests
- **Schema Hardening Tests** — `t06-schema-hardening.test.mjs` (132 tests) for input validation
- **Security Tests** — `t07-no-log-key-config.test.mjs` (138 tests), `t08-mcp-scope-enforcement.test.mjs` (72 tests)
- **Integration Tests** — `v1-contracts-behavior.test.mjs` (171 tests), `security-hardening.test.mjs` (103 tests)
- **Migrated Tests to TypeScript** — E2E ecosystem tests migrated from `.mjs` to `.ts` with proper typing
### 📁 New Files (35+)
### 📁 New Files (50+)
| Directory | Files |
| :------------------------------- | :--------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `open-sse/mcp-server/` | `server.ts`, `transport.ts`, `auth.ts`, `audit.ts`, `tools/advancedTools.ts` |
| `src/lib/a2a/` | `taskManager.ts`, `streaming.ts`, `routingLogger.ts`, `skills/smartRouting.ts`, `skills/quotaManagement.ts` |
| `open-sse/mcp-server/` | `server.ts`, `index.ts`, `audit.ts`, `scopeEnforcement.ts`, `tools/advancedTools.ts`, `README.md` |
| `open-sse/mcp-server/schemas/` | `tools.ts`, `a2a.ts`, `audit.ts`, `index.ts` |
| `src/lib/a2a/` | `taskManager.ts`, `taskExecution.ts`, `streaming.ts`, `routingLogger.ts`, `README.md` |
| `src/lib/a2a/skills/` | `smartRouting.ts`, `quotaManagement.ts` |
| `src/app/a2a/` | `route.ts` (JSON-RPC 2.0 dispatch handler) |
| `open-sse/services/autoCombo/` | `scoring.ts`, `taskFitness.ts`, `engine.ts`, `selfHealing.ts`, `modePacks.ts`, `persistence.ts`, `index.ts` |
| `vscode-extension/src/services/` | `mcpClient.ts`, `a2aClient.ts`, `policyEngine.ts`, `preflightDialog.ts`, `budgetGuard.ts`, `healthMonitor.ts`, `modePackSelector.ts`, `humanCheckpoint.ts` |
| `src/shared/contracts/` | `quota.ts` (shared API contracts) |
| `src/shared/constants/` | `mcpScopes.ts` |
| `src/lib/db/migrations/` | `002_mcp_a2a_tables.sql` |
| `src/app/(dashboard)/` | `dashboard/mcp/page.tsx`, `dashboard/a2a/page.tsx`, `dashboard/auto-combo/page.tsx` |
| `docs/` | `mcp-server.md`, `a2a-server.md`, `auto-combo.md`, `vscode-extension.md`, `integrations/ide-configs.md` |
| `vscode-extension/src/services/` | `mcpClient.ts`, `a2aClient.ts`, `policyEngine.ts`, `preflightDialog.ts`, `budgetGuard.ts`, `healthMonitor.ts`, `modePackSelector.ts`, `humanCheckpoint.ts` |
| `scripts/` | `check-cycles.mjs`, `check-docs-sync.mjs`, `check-route-validation.mjs`, `check-t11-any-budget.mjs`, `run-playwright-tests.mjs`, `runtime-env.mjs` |
| `tests/` | `t06-schema-hardening.test.mjs`, `t07-no-log-key-config.test.mjs`, `t08-mcp-scope-enforcement.test.mjs`, `ecosystem.test.ts` |
| `docs/` | `mcp-server.md`, `a2a-server.md`, `auto-combo.md`, `vscode-extension.md`, `integrations/ide-configs.md`, `RELEASE_CHECKLIST.md` |
### 📝 Commit History (`features-agente-mcp-a2a` branch)
| Commit | Date | Description |
| :-------- | :--------- | :--------------------------------------------------------------------------------------- |
| `e0ddb22` | 2026-03-03 | feat: add MCP server mode with `--mcp` flag for IDE integration |
| `09a1748` | 2026-03-03 | feat: add Phase 3 advanced MCP tools and A2A smart routing skill |
| `1e1a9c9` | 2026-03-04 | feat: migrate tests to TypeScript and add MCP advanced tools test suite |
| `ab77452` | 2026-03-04 | feat: normalize quota and combos API responses with shared contracts |
| `88ad4cc` | 2026-03-04 | feat: add MCP server, A2A protocol, auto-combo engine & VS Code extension |
| `cc429d4` | 2026-03-04 | feat: add TypeScript types and modularize translator registry |
| `adc8fdf` | 2026-03-04 | feat: add A2A protocol support and refactor API validation layer |
| `500cae3` | 2026-03-04 | refactor: replace `any` types with generics and add Zod validation schemas |
| `889e2ba` | 2026-03-04 | feat: add error pages, harden DB layer and compliance module |
| `cbd0b1c` | 2026-03-04 | refactor: harden open-sse services, eliminate any casts, add dashboard pages |
| `b33a853` | 2026-03-04 | feat: Introduce A2A lifecycle management, add type safety to ComfyUI and stream handling |
---

View File

@@ -5,9 +5,9 @@
### Never stop coding. Smart routing to **FREE & low-cost AI models** with automatic fallback.
_Your universal API proxy — one endpoint, 36+ providers, zero downtime._
_Your universal API proxy — one endpoint, 36+ providers, zero downtime. Now with **MCP & A2A** agent orchestration._
**Chat Completions • Embeddings • Image Generation • Video • Music • Audio • Reranking • 100% TypeScript**
**Chat Completions • Embeddings • Image Generation • Video • Music • Audio • Reranking • MCP Server • A2A Protocol • 100% TypeScript**
---
@@ -445,6 +445,7 @@ omniroute
| ----------------------- | ----------------------------------------------------------- |
| `omniroute` | Start server (`PORT=20128`, API and dashboard on same port) |
| `omniroute --port 3000` | Set canonical/API port to 3000 |
| `omniroute --mcp` | Start MCP server (stdio transport) for IDE integration |
| `omniroute --no-open` | Don't auto-open browser |
| `omniroute --help` | Show help |
@@ -594,6 +595,23 @@ When minimized, OmniRoute lives in your system tray with quick actions:
## 💡 Key Features
### 🤖 Agent Integration (NEW in v2.0)
| Feature | What It Does |
| ---------------------------- | ----------------------------------------------------------------------------------- |
| 🔧 **MCP Server (16 Tools)** | IDE agents control OmniRoute via Model Context Protocol — health, routing, budgets |
| 🤝 **A2A Server (v0.3)** | Multi-agent orchestration via JSON-RPC 2.0 with smart-routing & quota-management |
| ⚡ **Auto-Combo Engine** | Self-healing 6-factor scoring with task fitness, mode packs, and bandit exploration |
| 🎯 **Scope Enforcement** | 9 granular permission scopes for MCP tool access control |
| 📊 **Audit Logging** | SHA-256 hashed tool call audit trail in SQLite |
| 📡 **SSE Streaming** | Real-time A2A task streaming with heartbeat and completion events |
| 🛡️ **Budget Guard** | Session-level budget enforcement with degrade/block/alert actions |
| 📋 **Agent Card Discovery** | `/.well-known/agent.json` for automatic A2A agent discovery |
> 📖 **[MCP Server README](open-sse/mcp-server/README.md)** — Full tool reference, IDE configs, and client examples in Python/TypeScript/Go
>
> 📖 **[A2A Server README](src/lib/a2a/README.md)** — Skills reference, JSON-RPC methods, LangChain integration, and streaming examples
### 🧠 Core Routing & Intelligence
| Feature | What It Does |
@@ -1315,33 +1333,39 @@ Se não quiser criar credenciais próprias agora, ainda é possível usar o flux
## 🛠️ Tech Stack
- **Runtime**: Node.js 1822 LTS (⚠️ Node.js 24+ is **not supported**`better-sqlite3` native binaries are incompatible)
- **Language**: TypeScript 5.9 — **100% TypeScript** across `src/` and `open-sse/` (v1.0.6)
- **Language**: TypeScript 5.9 — **100% TypeScript** across `src/` and `open-sse/` (zero `any` in core modules since v2.0)
- **Framework**: Next.js 16 + React 19 + Tailwind CSS 4
- **Database**: LowDB (JSON) + SQLite (domain state + proxy logs)
- **Database**: LowDB (JSON) + SQLite (domain state + proxy logs + MCP audit + routing decisions)
- **Schemas**: Zod (MCP tool I/O validation, API contracts)
- **Protocols**: MCP (stdio/HTTP) + A2A v0.3 (JSON-RPC 2.0 + SSE)
- **Streaming**: Server-Sent Events (SSE)
- **Auth**: OAuth 2.0 (PKCE) + JWT + API Keys
- **Testing**: Node.js test runner (368+ unit tests)
- **Auth**: OAuth 2.0 (PKCE) + JWT + API Keys + MCP Scoped Authorization
- **Testing**: Node.js test runner + Vitest (900+ tests including unit, integration, E2E)
- **CI/CD**: GitHub Actions (auto npm publish + Docker Hub on release)
- **Website**: [omniroute.online](https://omniroute.online)
- **Package**: [npmjs.com/package/omniroute](https://www.npmjs.com/package/omniroute)
- **Docker**: [hub.docker.com/r/diegosouzapw/omniroute](https://hub.docker.com/r/diegosouzapw/omniroute)
- **Resilience**: Circuit breaker, exponential backoff, anti-thundering herd, TLS spoofing
- **Resilience**: Circuit breaker, exponential backoff, anti-thundering herd, TLS spoofing, auto-combo self-healing
---
## 📖 Documentation
| Document | Description |
| -------------------------------------------- | ---------------------------------------------- |
| [User Guide](docs/USER_GUIDE.md) | Providers, combos, CLI integration, deployment |
| [API Reference](docs/API_REFERENCE.md) | All endpoints with examples |
| [Troubleshooting](docs/TROUBLESHOOTING.md) | Common problems and solutions |
| [Architecture](docs/ARCHITECTURE.md) | System architecture and internals |
| [Contributing](CONTRIBUTING.md) | Development setup and guidelines |
| [OpenAPI Spec](docs/openapi.yaml) | OpenAPI 3.0 specification |
| [Security Policy](SECURITY.md) | Vulnerability reporting and security practices |
| [VM Deployment](docs/VM_DEPLOYMENT_GUIDE.md) | Complete guide: VM + nginx + Cloudflare setup |
| [Features Gallery](docs/FEATURES.md) | Visual dashboard tour with screenshots |
| Document | Description |
| ---------------------------------------------- | --------------------------------------------------- |
| [User Guide](docs/USER_GUIDE.md) | Providers, combos, CLI integration, deployment |
| [API Reference](docs/API_REFERENCE.md) | All endpoints with examples |
| [MCP Server](open-sse/mcp-server/README.md) | 16 MCP tools, IDE configs, Python/TS/Go clients |
| [A2A Server](src/lib/a2a/README.md) | JSON-RPC 2.0 protocol, skills, streaming, task mgmt |
| [Auto-Combo Engine](docs/auto-combo.md) | 6-factor scoring, mode packs, self-healing |
| [Troubleshooting](docs/TROUBLESHOOTING.md) | Common problems and solutions |
| [Architecture](docs/ARCHITECTURE.md) | System architecture and internals |
| [Contributing](CONTRIBUTING.md) | Development setup and guidelines |
| [OpenAPI Spec](docs/openapi.yaml) | OpenAPI 3.0 specification |
| [Security Policy](SECURITY.md) | Vulnerability reporting and security practices |
| [VM Deployment](docs/VM_DEPLOYMENT_GUIDE.md) | Complete guide: VM + nginx + Cloudflare setup |
| [Features Gallery](docs/FEATURES.md) | Visual dashboard tour with screenshots |
| [Release Checklist](docs/RELEASE_CHECKLIST.md) | Pre-release validation steps |
### 📸 Dashboard Preview
@@ -1407,7 +1431,7 @@ See [CONTRIBUTING.md](CONTRIBUTING.md) for detailed guidelines.
```bash
# Create a release — npm publish happens automatically
gh release create v1.0.6 --title "v1.0.6" --generate-notes
gh release create v2.0.0 --title "v2.0.0" --generate-notes
```
---

65
bin/mcp-server.mjs Normal file
View File

@@ -0,0 +1,65 @@
#!/usr/bin/env node
import { spawn } from "node:child_process";
import { existsSync } from "node:fs";
import { dirname, join } from "node:path";
import { fileURLToPath } from "node:url";
const __filename = fileURLToPath(import.meta.url);
const __dirname = dirname(__filename);
const ROOT = join(__dirname, "..");
function resolveMcpEntry(rootDir = ROOT) {
const candidates = [
// Preferred distributable JS entry (npm publish artifact)
join(rootDir, "app", "open-sse", "mcp-server", "server.js"),
// Local workspace TypeScript source fallback
join(rootDir, "open-sse", "mcp-server", "server.ts"),
];
for (const entry of candidates) {
if (existsSync(entry)) return entry;
}
return null;
}
function formatSpawnError(exitCode, signal) {
if (signal) return `MCP server exited by signal ${signal}`;
return `MCP server exited with code ${exitCode ?? 1}`;
}
export async function startMcpCli(rootDir = ROOT) {
const mcpEntry = resolveMcpEntry(rootDir);
if (!mcpEntry) {
throw new Error(
"MCP server entrypoint not found. Expected app/open-sse/mcp-server/server.js or open-sse/mcp-server/server.ts."
);
}
// `tsx` loader is only required for local `.ts` fallback; JS entry works without it.
const loaderArgs = mcpEntry.endsWith(".ts") ? ["--import", "tsx/esm"] : [];
await new Promise((resolve, reject) => {
const child = spawn(process.execPath, [...loaderArgs, mcpEntry], {
cwd: rootDir,
env: process.env,
stdio: "inherit",
});
child.once("error", reject);
child.once("exit", (code, signal) => {
if ((code ?? 0) === 0 && !signal) {
resolve(undefined);
return;
}
reject(new Error(formatSpawnError(code, signal)));
});
});
}
if (process.argv[1] && fileURLToPath(import.meta.url) === process.argv[1]) {
startMcpCli().catch((err) => {
console.error("\x1b[31m✖ Failed to start MCP server:\x1b[0m", err?.message || err);
process.exit(1);
});
}

View File

@@ -128,14 +128,13 @@ if (args.includes("--version") || args.includes("-v")) {
// ── MCP Server Mode ───────────────────────────────────────
if (args.includes("--mcp")) {
try {
const { startMcpStdio } = await import(join(ROOT, "open-sse", "mcp-server", "server.ts"));
await startMcpStdio();
const { startMcpCli } = await import(join(ROOT, "bin", "mcp-server.mjs"));
await startMcpCli(ROOT);
} catch (err) {
console.error("\x1b[31m✖ Failed to start MCP server:\x1b[0m", err.message || err);
process.exit(1);
}
// MCP server runs indefinitely via stdio — don't fall through to Next.js server
await new Promise(() => {}); // Keep process alive
process.exit(0);
}
function parsePort(value, fallback) {

View File

@@ -2,26 +2,37 @@ import { HTTP_STATUS, FETCH_TIMEOUT_MS } from "../config/constants.ts";
type JsonRecord = Record<string, unknown>;
type ProviderConfig = {
export type ProviderConfig = {
id?: string;
baseUrl?: string;
baseUrls?: string[];
responsesBaseUrl?: string;
chatPath?: string;
clientVersion?: string;
clientId?: string;
clientSecret?: string;
tokenUrl?: string;
refreshUrl?: string;
authUrl?: string;
headers?: Record<string, string>;
};
type ProviderCredentials = {
export type ProviderCredentials = {
accessToken?: string;
refreshToken?: string;
apiKey?: string;
expiresAt?: string;
providerSpecificData?: JsonRecord;
};
type ExecutorLog = {
export type ExecutorLog = {
debug?: (tag: string, message: string) => void;
info?: (tag: string, message: string) => void;
warn?: (tag: string, message: string) => void;
error?: (tag: string, message: string) => void;
};
type ExecuteInput = {
export type ExecuteInput = {
model: string;
body: unknown;
stream: boolean;

View File

@@ -232,7 +232,7 @@ export class CursorExecutor extends BaseExecutor {
const response = await fetch(url, {
method: "POST",
headers,
body,
body: body as unknown as BodyInit,
signal,
});

View File

@@ -1,4 +1,9 @@
import { BaseExecutor } from "./base.ts";
import {
BaseExecutor,
type ExecuteInput,
type ExecutorLog,
type ProviderCredentials,
} from "./base.ts";
import { PROVIDERS } from "../config/constants.ts";
import { v4 as uuidv4 } from "uuid";
import { refreshKiroToken } from "../services/tokenRefresh.ts";
@@ -56,7 +61,7 @@ export class KiroExecutor extends BaseExecutor {
super("kiro", PROVIDERS.kiro);
}
buildHeaders(credentials: { accessToken?: string }, stream = true) {
buildHeaders(credentials: ProviderCredentials, stream = true) {
void stream;
const headers = {
...this.config.headers,
@@ -81,21 +86,7 @@ export class KiroExecutor extends BaseExecutor {
/**
* Custom execute for Kiro - handles AWS EventStream binary response
*/
async execute({
model,
body,
stream,
credentials,
signal,
log,
}: {
model: string;
body: unknown;
stream: boolean;
credentials: { accessToken?: string; refreshToken?: string; providerSpecificData?: unknown };
signal?: AbortSignal;
log?: { error?: (tag: string, message: string) => void } | null;
}) {
async execute({ model, body, stream, credentials, signal, log }: ExecuteInput) {
const url = this.buildUrl(model, stream, 0);
const headers = this.buildHeaders(credentials, stream);
const transformedBody = this.transformRequest(model, body, stream, credentials);
@@ -166,8 +157,11 @@ export class KiroExecutor extends BaseExecutor {
if (!state.contextUsagePercentage) state.contextUsagePercentage = 0;
// Handle assistantResponseEvent
if (eventType === "assistantResponseEvent" && event.payload?.content) {
const content = event.payload.content;
if (eventType === "assistantResponseEvent") {
const content = typeof event.payload?.content === "string" ? event.payload.content : "";
if (!content) {
continue;
}
state.totalContentLength += content.length;
const chunk: JsonRecord = {
@@ -319,8 +313,15 @@ export class KiroExecutor extends BaseExecutor {
}
// Handle contextUsageEvent to extract contextUsagePercentage
if (eventType === "contextUsageEvent" && event.payload?.contextUsagePercentage) {
state.contextUsagePercentage = event.payload.contextUsagePercentage;
if (eventType === "contextUsageEvent") {
const contextUsage =
typeof event.payload?.contextUsagePercentage === "number"
? event.payload.contextUsagePercentage
: 0;
if (contextUsage <= 0) {
continue;
}
state.contextUsagePercentage = contextUsage;
// Mark that we received context usage event
state.hasContextUsage = true;
}
@@ -335,8 +336,14 @@ export class KiroExecutor extends BaseExecutor {
// Extract usage data from metricsEvent payload
const metrics = event.payload?.metricsEvent || event.payload;
if (metrics && typeof metrics === "object") {
const inputTokens = metrics.inputTokens || 0;
const outputTokens = metrics.outputTokens || 0;
const inputTokens =
typeof (metrics as JsonRecord).inputTokens === "number"
? ((metrics as JsonRecord).inputTokens as number)
: 0;
const outputTokens =
typeof (metrics as JsonRecord).outputTokens === "number"
? ((metrics as JsonRecord).outputTokens as number)
: 0;
if (inputTokens > 0 || outputTokens > 0) {
state.usage = {
@@ -443,10 +450,7 @@ export class KiroExecutor extends BaseExecutor {
});
}
async refreshCredentials(
credentials: { refreshToken?: string; providerSpecificData?: unknown },
log?: { error?: (tag: string, message: string) => void } | null
) {
async refreshCredentials(credentials: ProviderCredentials, log?: ExecutorLog | null) {
if (!credentials.refreshToken) return null;
try {

View File

@@ -220,7 +220,7 @@ export async function handleAudioTranscription({
credentials?: TranscriptionCredentials | null;
}): Promise<Response> {
const model = formData.get("model");
if (!model) {
if (typeof model !== "string" || !model) {
return errorResponse(400, "model is required");
}

View File

@@ -223,12 +223,15 @@ export function translateNonStreamingResponse(
finishReason = "tool_calls";
}
const createdMs = Date.parse(toString(response.createTime));
const created = Number.isFinite(createdMs)
? Math.floor(createdMs / 1000)
: Math.floor(Date.now() / 1000);
const result: JsonRecord = {
id: `chatcmpl-${toString(response.responseId, String(Date.now()))}`,
object: "chat.completion",
created: Math.floor(
new Date(toString(response.createTime, String(Date.now()))).getTime() / 1000
),
created,
model: toString(response.modelVersion, "gemini"),
choices: [
{

View File

@@ -0,0 +1,587 @@
# OmniRoute MCP Server
> **Model Context Protocol server** that exposes OmniRoute's gateway intelligence as **16 tools** for AI agents.
The MCP Server allows any AI agent (Claude Desktop, Cursor, VS Code Copilot, custom agents) to **monitor, control, and optimize** the OmniRoute AI gateway programmatically.
---
## Architecture
```
┌──────────────────────────────────────────────────────────────────┐
│ AI Agent / IDE │
│ (Claude Desktop, Cursor, VS Code, Custom) │
└──────────────────────┬───────────────────────────────────────────┘
│ MCP Protocol (stdio or HTTP)
┌──────────────────────────────────────────────────────────────────┐
│ OmniRoute MCP Server │
│ ┌──────────────┐ ┌─────────────────┐ ┌────────────────────┐ │
│ │ Scope │ │ 16 MCP Tools │ │ Audit Logger │ │
│ │ Enforcement │──│ (Phase 1 + 2) │──│ (SHA-256/SQLite) │ │
│ └──────────────┘ └────────┬────────┘ └────────────────────┘ │
└─────────────────────────────┼────────────────────────────────────┘
│ HTTP (internal)
┌──────────────────────────────────────────────────────────────────┐
│ OmniRoute Gateway (port 20128) │
│ /v1/chat/completions /api/combos /api/usage ... │
└──────────────────────────────────────────────────────────────────┘
```
---
## Quick Start
### 1. Environment Variables
```bash
# Required: OmniRoute base URL
export OMNIROUTE_BASE_URL="http://localhost:20128"
# Optional: API key for authenticated access
export OMNIROUTE_API_KEY="your-api-key"
# Optional: Scope enforcement (default: disabled)
export OMNIROUTE_MCP_ENFORCE_SCOPES="true"
export OMNIROUTE_MCP_SCOPES="read:health,read:combos,read:quota,read:usage,read:models,execute:completions,write:combos,write:budget,write:resilience"
```
### 2. stdio Transport (IDE Integration)
Add to your MCP client configuration:
**Claude Desktop** (`claude_desktop_config.json`):
```json
{
"mcpServers": {
"omniroute": {
"command": "node",
"args": ["path/to/9router/open-sse/mcp-server/server.ts"],
"env": {
"OMNIROUTE_BASE_URL": "http://localhost:20128",
"OMNIROUTE_API_KEY": "your-key"
}
}
}
}
```
**Cursor** (`.cursor/mcp.json`):
```json
{
"mcpServers": {
"omniroute": {
"command": "npx",
"args": ["tsx", "open-sse/mcp-server/server.ts"],
"env": {
"OMNIROUTE_BASE_URL": "http://localhost:20128"
}
}
}
}
```
**VS Code** (`.vscode/settings.json`):
```json
{
"mcp": {
"servers": {
"omniroute": {
"command": "npx",
"args": ["tsx", "open-sse/mcp-server/server.ts"],
"env": {
"OMNIROUTE_BASE_URL": "http://localhost:20128"
}
}
}
}
}
```
### 3. Start via CLI
```bash
# Direct start (stdio)
npx tsx open-sse/mcp-server/server.ts
# Or via OmniRoute CLI
omniroute --mcp
```
---
## Tool Reference
### Phase 1: Essential Tools (8)
| # | Tool | Scopes | Description |
| --- | ------------------------------- | --------------------- | -------------------------------------------------------------------------- |
| 1 | `omniroute_get_health` | `read:health` | Gateway health, uptime, memory, circuit breakers, rate limits, cache stats |
| 2 | `omniroute_list_combos` | `read:combos` | List all combos (model chains) with strategies and optional metrics |
| 3 | `omniroute_get_combo_metrics` | `read:combos` | Performance metrics for a specific combo |
| 4 | `omniroute_switch_combo` | `write:combos` | Activate or deactivate a combo for routing |
| 5 | `omniroute_check_quota` | `read:quota` | Remaining API quota per provider with token health status |
| 6 | `omniroute_route_request` | `execute:completions` | Send a chat completion through intelligent routing |
| 7 | `omniroute_cost_report` | `read:usage` | Cost report by period (session/day/week/month) with per-provider breakdown |
| 8 | `omniroute_list_models_catalog` | `read:models` | List all available models across providers with capabilities and pricing |
### Phase 2: Advanced Tools (8)
| # | Tool | Scopes | Description |
| --- | ---------------------------------- | ------------------------------------ | ---------------------------------------------------------------------------- |
| 9 | `omniroute_simulate_route` | `read:health`, `read:combos` | Dry-run routing simulation showing fallback tree and estimated costs |
| 10 | `omniroute_set_budget_guard` | `write:budget` | Set session budget with action on exceed: `degrade`, `block`, or `alert` |
| 11 | `omniroute_set_resilience_profile` | `write:resilience` | Apply resilience profile: `aggressive`, `balanced`, or `conservative` |
| 12 | `omniroute_test_combo` | `execute:completions`, `read:combos` | Test each provider in a combo with a real prompt, report latency/cost |
| 13 | `omniroute_get_provider_metrics` | `read:health` | Per-provider metrics with latency percentiles (p50/p95/p99), circuit breaker |
| 14 | `omniroute_best_combo_for_task` | `read:combos`, `read:health` | AI-powered combo recommendation by task type with budget/latency constraints |
| 15 | `omniroute_explain_route` | `read:health`, `read:usage` | Explain why a request was routed to a provider (scoring factors, fallbacks) |
| 16 | `omniroute_get_session_snapshot` | `read:usage` | Full session snapshot: cost, tokens, top models, errors, budget status |
---
## Client Examples
### Python — Full Agent Workflow
```python
"""
OmniRoute MCP Client — Python example using the mcp SDK.
Install: pip install mcp
"""
import asyncio
from mcp import ClientSession, StdioServerParameters
from mcp.client.stdio import stdio_client
async def main():
server = StdioServerParameters(
command="npx",
args=["tsx", "open-sse/mcp-server/server.ts"],
env={
"OMNIROUTE_BASE_URL": "http://localhost:20128",
"OMNIROUTE_API_KEY": "your-key",
},
)
async with stdio_client(server) as (read, write):
async with ClientSession(read, write) as session:
await session.initialize()
# 1. Check gateway health
health = await session.call_tool("omniroute_get_health", {})
print("Health:", health.content[0].text)
# 2. List available combos with metrics
combos = await session.call_tool("omniroute_list_combos", {
"includeMetrics": True
})
print("Combos:", combos.content[0].text)
# 3. Find the best combo for a coding task
best = await session.call_tool("omniroute_best_combo_for_task", {
"taskType": "coding",
"budgetConstraint": 0.50,
"latencyConstraint": 5000,
})
print("Best combo:", best.content[0].text)
# 4. Set a session budget guard
budget = await session.call_tool("omniroute_set_budget_guard", {
"maxCost": 1.00,
"action": "degrade",
"degradeToTier": "cheap",
})
print("Budget guard:", budget.content[0].text)
# 5. Route a request through intelligent pipeline
response = await session.call_tool("omniroute_route_request", {
"model": "claude-sonnet-4",
"messages": [
{"role": "user", "content": "Write a Python hello world"}
],
"role": "coding",
})
print("Response:", response.content[0].text)
# 6. Get the session snapshot
snapshot = await session.call_tool("omniroute_get_session_snapshot", {})
print("Session:", snapshot.content[0].text)
asyncio.run(main())
```
### TypeScript — Programmatic Agent
```typescript
import { Client } from "@modelcontextprotocol/sdk/client/index.js";
import { StdioClientTransport } from "@modelcontextprotocol/sdk/client/stdio.js";
async function main() {
const transport = new StdioClientTransport({
command: "npx",
args: ["tsx", "open-sse/mcp-server/server.ts"],
env: {
OMNIROUTE_BASE_URL: "http://localhost:20128",
OMNIROUTE_API_KEY: "your-key",
},
});
const client = new Client({ name: "my-agent", version: "1.0.0" });
await client.connect(transport);
// Check quota before deciding which model to use
const quota = await client.callTool({
name: "omniroute_check_quota",
arguments: { provider: "claude" },
});
console.log("Claude quota:", quota.content);
// Simulate the route before actually calling
const simulation = await client.callTool({
name: "omniroute_simulate_route",
arguments: {
model: "claude-sonnet-4",
promptTokenEstimate: 2000,
},
});
console.log("Route simulation:", simulation.content);
// Send the actual request
const result = await client.callTool({
name: "omniroute_route_request",
arguments: {
model: "claude-sonnet-4",
messages: [{ role: "user", content: "Explain async/await" }],
},
});
console.log("Result:", result.content);
// Cost report
const costs = await client.callTool({
name: "omniroute_cost_report",
arguments: { period: "session" },
});
console.log("Costs:", costs.content);
await client.close();
}
main();
```
### Go — HTTP Client
```go
package main
import (
"bytes"
"encoding/json"
"fmt"
"io"
"net/http"
)
// Simplified direct-API approach (bypass MCP, hit OmniRoute APIs directly)
// Useful if you don't need MCP protocol framing.
func callTool(baseURL, tool string, args map[string]any) (string, error) {
// MCP tools map to OmniRoute APIs:
endpoints := map[string]string{
"health": "/api/monitoring/health",
"combos": "/api/combos",
"quota": "/api/usage/quota",
"models": "/v1/models",
}
url := baseURL + endpoints[tool]
resp, err := http.Get(url)
if err != nil {
return "", err
}
defer resp.Body.Close()
body, _ := io.ReadAll(resp.Body)
return string(body), nil
}
func routeRequest(baseURL, model, prompt string) (string, error) {
payload := map[string]any{
"model": model,
"messages": []map[string]string{
{"role": "user", "content": prompt},
},
"stream": false,
}
data, _ := json.Marshal(payload)
resp, err := http.Post(
baseURL+"/v1/chat/completions",
"application/json",
bytes.NewReader(data),
)
if err != nil {
return "", err
}
defer resp.Body.Close()
body, _ := io.ReadAll(resp.Body)
return string(body), nil
}
func main() {
base := "http://localhost:20128"
health, _ := callTool(base, "health", nil)
fmt.Println("Health:", health)
result, _ := routeRequest(base, "auto", "Hello from Go!")
fmt.Println("Result:", result)
}
```
---
## Use Cases
### 🔄 Use Case 1: Auto-Healing Agent
An agent that monitors OmniRoute health and auto-switches combos when providers degrade.
```python
async def auto_healing_loop(session):
"""Monitor health and react to provider issues."""
while True:
# Check health
health = await session.call_tool("omniroute_get_health", {})
data = json.loads(health.content[0].text)
# Find providers with open circuit breakers
broken = [
cb for cb in data["circuitBreakers"]
if cb["state"] == "OPEN"
]
if broken:
# Switch to a different resilience profile
await session.call_tool("omniroute_set_resilience_profile", {
"profile": "conservative"
})
# Find best alternative combo
best = await session.call_tool("omniroute_best_combo_for_task", {
"taskType": "coding"
})
best_data = json.loads(best.content[0].text)
combo_id = best_data["recommendedCombo"]["id"]
# Activate it
await session.call_tool("omniroute_switch_combo", {
"comboId": combo_id, "active": True
})
print(f"⚠️ Auto-healed: switched to {combo_id}")
await asyncio.sleep(30) # Check every 30 seconds
```
### 💰 Use Case 2: Budget-Aware Coding Agent
An agent that monitors costs in real-time and degrades to cheaper models when nearing budget.
```python
async def budget_aware_coding(session, task: str, max_budget: float):
"""Complete a coding task within a budget."""
# Set budget guard
await session.call_tool("omniroute_set_budget_guard", {
"maxCost": max_budget,
"action": "degrade",
"degradeToTier": "cheap",
})
# Simulate first to estimate cost
sim = await session.call_tool("omniroute_simulate_route", {
"model": "claude-sonnet-4",
"promptTokenEstimate": len(task.split()) * 2,
})
sim_data = json.loads(sim.content[0].text)
estimated_cost = sim_data["fallbackTree"]["bestCaseCost"]
print(f"Estimated cost: ${estimated_cost:.4f}")
# Send request
result = await session.call_tool("omniroute_route_request", {
"model": "claude-sonnet-4",
"messages": [{"role": "user", "content": task}],
"role": "coding",
})
# Check remaining budget
snapshot = await session.call_tool("omniroute_get_session_snapshot", {})
snap_data = json.loads(snapshot.content[0].text)
print(f"Session cost: ${snap_data['costTotal']:.4f}")
if snap_data.get("budgetGuard"):
print(f"Budget remaining: ${snap_data['budgetGuard']['remaining']:.4f}")
return json.loads(result.content[0].text)["response"]["content"]
```
### 🧪 Use Case 3: Combo Benchmarking Agent
An agent that periodically benchmarks all combos and reports the fastest/cheapest.
```python
async def benchmark_combos(session):
"""Benchmark all enabled combos and rank them."""
combos = await session.call_tool("omniroute_list_combos", {
"includeMetrics": True,
})
combo_list = json.loads(combos.content[0].text)["combos"]
results = []
for combo in combo_list:
if not combo["enabled"]:
continue
test = await session.call_tool("omniroute_test_combo", {
"comboId": combo["id"],
"testPrompt": "Return the number 42.",
})
test_data = json.loads(test.content[0].text)
results.append({
"combo": combo["name"],
"fastest": test_data["summary"]["fastestProvider"],
"cheapest": test_data["summary"]["cheapestProvider"],
"success_rate": f'{test_data["summary"]["successful"]}/{test_data["summary"]["totalProviders"]}',
})
print("📊 Combo Benchmark Results:")
for r in results:
print(f" {r['combo']}: fastest={r['fastest']}, cheapest={r['cheapest']}, success={r['success_rate']}")
```
### 🔍 Use Case 4: Post-Mortem Debugging Agent
An agent that explains why a request was routed to a specific provider.
```typescript
async function debugRouting(client: Client, requestId: string) {
// Explain the routing decision
const explanation = await client.callTool({
name: "omniroute_explain_route",
arguments: { requestId },
});
const data = JSON.parse(explanation.content[0].text);
console.log(`Request ${requestId}:`);
console.log(` Provider: ${data.decision.providerSelected}`);
console.log(` Model: ${data.decision.modelUsed}`);
console.log(` Score: ${data.decision.score}`);
console.log(` Factors:`);
for (const factor of data.decision.factors) {
console.log(` ${factor.name}: ${factor.value} (weight: ${factor.weight})`);
}
if (data.decision.fallbacksTriggered.length > 0) {
console.log(` Fallbacks triggered:`);
for (const fb of data.decision.fallbacksTriggered) {
console.log(` ${fb.provider}: ${fb.reason}`);
}
}
}
```
### 📋 Use Case 5: Model Discovery Agent
An agent that discovers the cheapest models for a given capability.
```python
async def find_cheapest_models(session, capability="chat"):
"""Find the cheapest available models for a capability."""
catalog = await session.call_tool("omniroute_list_models_catalog", {
"capability": capability,
})
models = json.loads(catalog.content[0].text)["models"]
# Filter available models with pricing
priced = [
m for m in models
if m["status"] == "available" and m.get("pricing")
]
priced.sort(key=lambda m: m["pricing"]["inputPerMillion"] or float("inf"))
print(f"💡 Cheapest {capability} models:")
for m in priced[:5]:
input_cost = m["pricing"]["inputPerMillion"] or 0
output_cost = m["pricing"]["outputPerMillion"] or 0
print(f" {m['id']} ({m['provider']}): ${input_cost}/M in, ${output_cost}/M out")
```
---
## Security & Scope Enforcement
The MCP server supports **fine-grained scope enforcement** for multi-tenant environments:
| Scope | Tools |
| --------------------- | ---------------------------------------------------------------------------------------------- |
| `read:health` | `get_health`, `simulate_route`, `get_provider_metrics`, `best_combo_for_task`, `explain_route` |
| `read:combos` | `list_combos`, `get_combo_metrics`, `simulate_route`, `best_combo_for_task`, `test_combo` |
| `read:quota` | `check_quota` |
| `read:usage` | `cost_report`, `explain_route`, `get_session_snapshot` |
| `read:models` | `list_models_catalog` |
| `write:combos` | `switch_combo` |
| `write:budget` | `set_budget_guard` |
| `write:resilience` | `set_resilience_profile` |
| `execute:completions` | `route_request`, `test_combo` |
**Wildcard scopes:** Use `read:*` to grant all read scopes, or `*` for full access.
---
## Audit Logging
Every tool call is logged to the `mcp_tool_audit` SQLite table:
- **Input:** SHA-256 hashed (never stores raw prompts)
- **Output:** Truncated to 200 chars
- **Metadata:** Tool name, duration, success/error, API key ID
Access audit data via:
```typescript
import { getRecentAuditEntries, getAuditStats } from "./audit";
const entries = await getRecentAuditEntries(50);
const stats = await getAuditStats();
// stats: { totalCalls, successRate, avgDurationMs, topTools }
```
---
## File Structure
```
mcp-server/
├── server.ts # MCP server setup, essential tool handlers, entry point
├── index.ts # Barrel export
├── audit.ts # SQLite audit logger (SHA-256 input hashing)
├── scopeEnforcement.ts # Fine-grained scope enforcement
├── schemas/
│ ├── tools.ts # Zod schemas for all 16 tools (input/output/scopes)
│ ├── a2a.ts # A2A protocol types (Agent Card, Task, JSON-RPC)
│ ├── audit.ts # Audit & routing decision types + hash helpers
│ └── index.ts # Schema barrel export
├── tools/
│ └── advancedTools.ts # Phase 2 tool handlers (8 advanced tools)
└── __tests__/
├── essentialTools.test.ts
├── advancedTools.test.ts
└── a2aLifecycle.test.ts
```
---
## License
Part of [OmniRoute](https://github.com/diegosouzapw/OmniRoute) — MIT License.

View File

@@ -53,4 +53,33 @@ describe("A2A task lifecycle regressions", () => {
expect(loaded?.state).toBe("failed");
expect(loaded?.artifacts.at(-1)).toEqual({ type: "error", content: "upstream failure" });
});
it("transitions expired submitted tasks to failed without throwing", () => {
const tm = createManager();
const task = tm.createTask({
skill: "smart-routing",
messages: [{ role: "user", content: "hello" }],
});
task.expiresAt = new Date(Date.now() - 1_000).toISOString();
expect(() => tm.getTask(task.id)).not.toThrow();
const loaded = tm.getTask(task.id);
expect(loaded?.state).toBe("failed");
});
it("does not rewrite cancelled tasks to failed during cleanup", () => {
const tm = createManager();
const task = tm.createTask({
skill: "smart-routing",
messages: [{ role: "user", content: "cancel me" }],
});
tm.updateTask(task.id, "cancelled");
task.expiresAt = new Date(Date.now() - 1_000).toISOString();
// private in TS only; callable at runtime for regression test
(tm as any).cleanupExpired();
const loaded = tm.getTask(task.id);
expect(loaded?.state).toBe("cancelled");
});
});

View File

@@ -49,7 +49,7 @@ import {
handleExplainRoute,
handleGetSessionSnapshot,
} from "./tools/advancedTools.ts";
import { normalizeQuotaResponse } from "@/shared/contracts/quota";
import { normalizeQuotaResponse } from "../../src/shared/contracts/quota.ts";
// ============ Configuration ============
@@ -385,8 +385,15 @@ async function handleCostReport(args: { period?: string }) {
const start = Date.now();
try {
const period = args.period || "session";
const rangeMap: Record<string, string> = {
session: "1d",
day: "1d",
week: "7d",
month: "30d",
};
const range = rangeMap[period] || "30d";
const raw = toRecord(
await omniRouteFetch(`/api/usage/analytics?period=${encodeURIComponent(period)}`)
await omniRouteFetch(`/api/usage/analytics?range=${encodeURIComponent(range)}`)
);
const tokenCount = toRecord(raw.tokenCount);
const budget = toRecord(raw.budget);

View File

@@ -14,7 +14,7 @@
*/
import { logToolCall } from "../audit.ts";
import { normalizeQuotaResponse } from "@/shared/contracts/quota";
import { normalizeQuotaResponse } from "../../../src/shared/contracts/quota.ts";
const OMNIROUTE_BASE_URL = process.env.OMNIROUTE_BASE_URL || "http://localhost:20128";
const OMNIROUTE_API_KEY = process.env.OMNIROUTE_API_KEY || "";

View File

@@ -12,6 +12,7 @@ export function antigravityToOpenAIRequest(model, body, stream) {
model: string;
messages: JsonRecord[];
stream: unknown;
tools?: JsonRecord[];
[key: string]: unknown;
} = {
model: model,

View File

@@ -362,14 +362,16 @@ function openaiToClaudeRequestForAntigravity(model, body, stream) {
}
const updatedContent = msg.content.map((block) => {
const blockType = typeof block.type === "string" ? block.type : "";
const blockName = typeof block.name === "string" ? block.name : "";
if (
block.type === "tool_use" &&
block.name &&
block.name.startsWith(CLAUDE_OAUTH_TOOL_PREFIX)
blockType === "tool_use" &&
blockName &&
blockName.startsWith(CLAUDE_OAUTH_TOOL_PREFIX)
) {
return {
...block,
name: block.name.slice(CLAUDE_OAUTH_TOOL_PREFIX.length),
name: blockName.slice(CLAUDE_OAUTH_TOOL_PREFIX.length),
};
}
return block;

View File

@@ -150,7 +150,7 @@ export function claudeToOpenAIResponse(chunk, state) {
model: string;
choices: Array<{
index: number;
delta: { content: string };
delta: { content?: string };
finish_reason: string | null;
}>;
usage?: OpenAIUsage;

View File

@@ -15,6 +15,13 @@ type SocksDispatcherOptions = {
userId?: string;
password?: string;
};
type ProxyConfigObject = {
type?: string;
host?: string;
port?: string | number | null;
username?: string;
password?: string;
};
function getDispatcherCache(): DispatcherCache {
const globalWithCache = globalThis as GlobalWithDispatcherCache;
@@ -38,7 +45,7 @@ export function clearDispatcherCache() {
* `new URL("http://host:80")` strips port 80 since it's the HTTP default,
* but proxy servers commonly listen on port 80/443, so we need to preserve it.
*/
function extractExplicitPort(urlStr) {
function extractExplicitPort(urlStr: string): string | null {
try {
// Match port in the host portion: "scheme://[user:pass@]host:PORT[/...]"
const match = urlStr.match(/:\/\/(?:[^@]*@)?[^:/\s]+:(\d+)/);
@@ -50,13 +57,13 @@ function extractExplicitPort(urlStr) {
return null;
}
function defaultPortForProtocol(protocol) {
function defaultPortForProtocol(protocol: string): string {
if (protocol === "https:" || protocol === "wss:") return "443";
if (protocol === "socks5:") return "1080";
return "8080";
}
function normalizePort(port, protocol) {
function normalizePort(port: string | number | null | undefined, protocol: string): string {
if (!port) return defaultPortForProtocol(protocol);
const parsed = Number(port);
if (!Number.isInteger(parsed) || parsed < 1 || parsed > 65535) {
@@ -71,18 +78,18 @@ function normalizePort(port, protocol) {
* default ports (80 for http, 443 for https). Proxy servers commonly
* listen on these ports, so we must always include the port explicitly.
*/
function buildProxyUrlString(parsed, port) {
function buildProxyUrlString(parsed: URL, port: string): string {
const auth = parsed.username
? `${parsed.username}${parsed.password ? `:${parsed.password}` : ""}@`
: "";
return `${parsed.protocol}//${auth}${parsed.hostname}:${port}`;
}
export function isSocks5ProxyEnabled() {
export function isSocks5ProxyEnabled(): boolean {
return process.env.ENABLE_SOCKS5_PROXY === "true";
}
export function proxyUrlForLogs(proxyUrl) {
export function proxyUrlForLogs(proxyUrl: string): string {
const explicitPort = extractExplicitPort(proxyUrl);
const parsed = new URL(proxyUrl);
const port = explicitPort || parsed.port || defaultPortForProtocol(parsed.protocol);
@@ -90,10 +97,10 @@ export function proxyUrlForLogs(proxyUrl) {
}
export function normalizeProxyUrl(
proxyUrl,
proxyUrl: string,
source = "proxy",
{ allowSocks5 = isSocks5ProxyEnabled() } = {}
) {
): string {
// Extract the explicit port from the raw URL string BEFORE parsing,
// because `new URL()` silently strips default ports (80 for http,
// 443 for https), which are valid and common for proxy servers.
@@ -128,7 +135,10 @@ export function normalizeProxyUrl(
return buildProxyUrlString(parsed, port);
}
export function proxyConfigToUrl(proxyConfig, { allowSocks5 = isSocks5ProxyEnabled() } = {}) {
export function proxyConfigToUrl(
proxyConfig: unknown,
{ allowSocks5 = isSocks5ProxyEnabled() } = {}
): string | null {
if (!proxyConfig) return null;
if (typeof proxyConfig === "string") {
@@ -139,7 +149,8 @@ export function proxyConfigToUrl(proxyConfig, { allowSocks5 = isSocks5ProxyEnabl
throw new Error("[ProxyDispatcher] Invalid context proxy config");
}
const type = String(proxyConfig.type || "http").toLowerCase();
const config = proxyConfig as ProxyConfigObject;
const type = String(config.type || "http").toLowerCase();
const protocol = `${type}:`;
if (!SUPPORTED_PROTOCOLS.has(protocol)) {
@@ -150,23 +161,23 @@ export function proxyConfigToUrl(proxyConfig, { allowSocks5 = isSocks5ProxyEnabl
"[ProxyDispatcher] SOCKS5 proxy is disabled (set ENABLE_SOCKS5_PROXY=true to enable)"
);
}
if (!proxyConfig.host) {
if (!config.host) {
throw new Error("[ProxyDispatcher] Context proxy host is required");
}
const port = normalizePort(proxyConfig.port, protocol);
const port = normalizePort(config.port, protocol);
// Build the URL string manually to preserve the port through normalization.
const auth = proxyConfig.username
? `${encodeURIComponent(proxyConfig.username)}:${proxyConfig.password ? encodeURIComponent(proxyConfig.password) : ""}@`
const auth = config.username
? `${encodeURIComponent(config.username)}:${config.password ? encodeURIComponent(config.password) : ""}@`
: "";
const proxyUrlStr = `${type}://${auth}${proxyConfig.host}:${port}`;
const proxyUrlStr = `${type}://${auth}${config.host}:${port}`;
return normalizeProxyUrl(proxyUrlStr, "context proxy", { allowSocks5 });
}
export function createProxyDispatcher(proxyUrl) {
export function createProxyDispatcher(proxyUrl: string): Dispatcher {
const normalizedUrl = normalizeProxyUrl(proxyUrl, "proxy dispatcher");
const dispatcherCache = getDispatcherCache();

View File

@@ -16,6 +16,10 @@ type TlsFingerprintStore = { used: boolean };
const tlsFingerprintContext = new AsyncLocalStorage<TlsFingerprintStore>();
type FetchWithDispatcherOptions = RequestInit & { dispatcher?: unknown };
type FetchWithDispatcher = (
input: RequestInfo | URL,
init?: FetchWithDispatcherOptions
) => Promise<Response>;
type PatchState = {
originalFetch: typeof globalThis.fetch;
@@ -43,6 +47,7 @@ function getPatchState(): PatchState {
const patchState = getPatchState();
const originalFetch = patchState.originalFetch;
const originalFetchWithDispatcher = originalFetch as FetchWithDispatcher;
const proxyContext = patchState.proxyContext;
function noProxyMatch(targetUrl) {
@@ -141,7 +146,7 @@ export async function runWithProxyContext(proxyConfig, fn) {
async function patchedFetch(input: RequestInfo | URL, options: FetchWithDispatcherOptions = {}) {
if (options?.dispatcher) {
return originalFetch(input, options);
return originalFetchWithDispatcher(input, options);
}
const targetUrl = getTargetUrl(input);
@@ -161,7 +166,10 @@ async function patchedFetch(input: RequestInfo | URL, options: FetchWithDispatch
try {
const store = tlsFingerprintContext.getStore();
if (store) store.used = true;
return await tlsClient.fetch(targetUrl, options);
return await tlsClient.fetch(targetUrl, {
...options,
headers: options.headers,
});
} catch (error) {
const message = error instanceof Error ? error.message : String(error);
console.warn(
@@ -171,12 +179,12 @@ async function patchedFetch(input: RequestInfo | URL, options: FetchWithDispatch
if (store) store.used = false;
}
}
return originalFetch(input, options);
return originalFetchWithDispatcher(input, options);
}
try {
const dispatcher = createProxyDispatcher(proxyUrl);
return await originalFetch(input, { ...options, dispatcher });
return await originalFetchWithDispatcher(input, { ...options, dispatcher });
} catch (error) {
const message = error instanceof Error ? error.message : String(error);
console.error(`[ProxyFetch] Proxy request failed (${source}, fail-closed): ${message}`);

View File

@@ -35,12 +35,32 @@ function getProxyFromEnv(): string | undefined {
interface FetchOptions {
method?: string;
headers?: Record<string, string>;
headers?: HeadersInit;
body?: unknown;
redirect?: string;
signal?: AbortSignal;
}
function normalizeHeaders(headers: HeadersInit | undefined): Record<string, string> | undefined {
if (!headers) return undefined;
if (headers instanceof Headers) {
return Object.fromEntries(headers.entries());
}
if (Array.isArray(headers)) {
return Object.fromEntries(headers.map(([key, value]) => [key, String(value)]));
}
const normalized: Record<string, string> = {};
for (const [key, value] of Object.entries(headers)) {
if (typeof value === "string") {
normalized[key] = value;
}
}
return normalized;
}
/**
* TLS Client — Chrome 124 TLS fingerprint spoofing via wreq-js
* Singleton instance used to disguise Node.js TLS handshake as Chrome browser.
@@ -87,7 +107,7 @@ class TlsClient {
const wreqOptions: Record<string, unknown> = {
method,
headers: options.headers,
headers: normalizeHeaders(options.headers),
body: options.body,
redirect: options.redirect === "manual" ? "manual" : "follow",
};

6
package-lock.json generated
View File

@@ -36,6 +36,7 @@
"react-dom": "19.2.4",
"recharts": "^3.7.0",
"selfsigned": "^5.5.0",
"tsx": "^4.21.0",
"undici": "^7.19.2",
"uuid": "^13.0.0",
"wreq-js": "^2.0.1",
@@ -62,7 +63,6 @@
"lint-staged": "^16.2.7",
"prettier": "^3.8.1",
"tailwindcss": "^4",
"tsx": "^4.21.0",
"typescript": "^5.9.3",
"typescript-eslint": "^8.56.0",
"vitest": "^3.2.4",
@@ -6023,7 +6023,6 @@
"version": "0.27.3",
"resolved": "https://registry.npmjs.org/esbuild/-/esbuild-0.27.3.tgz",
"integrity": "sha512-8VwMnyGCONIs6cWue2IdpHxHnAjzxnw2Zr7MkVxB2vjmQ2ivqGFb4LEG3SMnv0Gb2F/G/2yA8zUaiL1gywDCCg==",
"dev": true,
"hasInstallScript": true,
"license": "MIT",
"bin": {
@@ -7093,7 +7092,6 @@
"version": "4.13.6",
"resolved": "https://registry.npmjs.org/get-tsconfig/-/get-tsconfig-4.13.6.tgz",
"integrity": "sha512-shZT/QMiSHc/YBLxxOkMtgSid5HFoauqCE3/exfsEcwg1WkeqjG+V40yBbBrsD+jW2HDXcs28xOfcbm2jI8Ddw==",
"dev": true,
"license": "MIT",
"dependencies": {
"resolve-pkg-maps": "^1.0.0"
@@ -11418,7 +11416,6 @@
"version": "4.21.0",
"resolved": "https://registry.npmjs.org/tsx/-/tsx-4.21.0.tgz",
"integrity": "sha512-5C1sg4USs1lfG0GFb2RLXsdpXqBSEhAaA/0kPL01wxzpMqLILNxIxIOKiILz+cdg/pLnOUxFYOR5yhHU666wbw==",
"dev": true,
"license": "MIT",
"dependencies": {
"esbuild": "~0.27.0",
@@ -11438,7 +11435,6 @@
"version": "2.3.3",
"resolved": "https://registry.npmjs.org/fsevents/-/fsevents-2.3.3.tgz",
"integrity": "sha512-5xoDfX+fL7faATnagmWPpbFtwh/R77WmMMqqHGS65C3vvB0YHrgF+B1YmZ3441tMj5n63k0212XNoJwzlhffQw==",
"dev": true,
"hasInstallScript": true,
"license": "MIT",
"optional": true,

View File

@@ -1,6 +1,6 @@
{
"name": "omniroute",
"version": "1.8.1",
"version": "2.0.0",
"description": "Smart AI Router with auto fallback — route to FREE & cheap models, zero downtime. Works with Cursor, Cline, Claude Desktop, Codex, and any OpenAI-compatible tool.",
"type": "module",
"bin": {
@@ -10,6 +10,8 @@
"files": [
"bin/",
"app/",
"open-sse/mcp-server/",
"src/shared/contracts/",
"scripts/postinstall.mjs",
"README.md",
"LICENSE"
@@ -99,6 +101,7 @@
"react-dom": "19.2.4",
"recharts": "^3.7.0",
"selfsigned": "^5.5.0",
"tsx": "^4.21.0",
"undici": "^7.19.2",
"uuid": "^13.0.0",
"wreq-js": "^2.0.1",
@@ -121,7 +124,6 @@
"lint-staged": "^16.2.7",
"prettier": "^3.8.1",
"tailwindcss": "^4",
"tsx": "^4.21.0",
"typescript": "^5.9.3",
"typescript-eslint": "^8.56.0",
"vitest": "^3.2.4",

View File

@@ -22,6 +22,21 @@ interface ExclusionEntry {
reason: string;
}
type AutoComboRecord = {
candidatePool?: unknown;
weights?: unknown;
};
type HealthRecord = {
providerHealth?: Record<string, { state?: string; lastFailure?: string | null }>;
circuitBreakers?: Array<{
provider?: string;
name?: string;
state?: string;
lastFailure?: string | null;
}>;
};
export default function AutoComboDashboard() {
const [scores, setScores] = useState<ProviderScore[]>([]);
const [exclusions, setExclusions] = useState<ExclusionEntry[]>([]);
@@ -35,11 +50,75 @@ export default function AutoComboDashboard() {
fetch("/api/monitoring/health"),
]);
if (combosRes.status === "fulfilled") {
const comboPayload = await combosRes.value.json();
const combos = Array.isArray(comboPayload?.combos)
? (comboPayload.combos as AutoComboRecord[])
: [];
const firstCombo = combos[0] || null;
const candidatePool = Array.isArray(firstCombo?.candidatePool)
? firstCombo.candidatePool.filter((entry): entry is string => typeof entry === "string")
: [];
const rawWeights =
firstCombo?.weights &&
typeof firstCombo.weights === "object" &&
!Array.isArray(firstCombo.weights)
? (firstCombo.weights as Record<string, unknown>)
: {};
const factors = Object.fromEntries(
Object.entries(rawWeights).map(([k, v]) => [k, typeof v === "number" ? v : 0])
);
const baseScore = candidatePool.length > 0 ? 1 / candidatePool.length : 0;
setScores(
candidatePool.map((provider) => ({
provider,
model: "auto",
score: baseScore,
factors,
}))
);
} else {
setScores([]);
}
if (healthRes.status === "fulfilled") {
const health = await healthRes.value.json();
const breakers = health?.circuitBreakers || [];
const openCount = breakers.filter((b: any) => b.state === "OPEN").length;
setIncidentMode(openCount / Math.max(breakers.length, 1) > 0.5);
const health = (await healthRes.value.json()) as HealthRecord;
const providerHealth =
health?.providerHealth && typeof health.providerHealth === "object"
? health.providerHealth
: {};
const breakersFromProviderHealth = Object.entries(providerHealth).map(
([provider, status]) => ({
provider,
state: status?.state || "CLOSED",
lastFailure: status?.lastFailure || null,
})
);
const breakersFromArray = Array.isArray(health?.circuitBreakers)
? health.circuitBreakers
: [];
const breakers =
breakersFromArray.length > 0
? breakersFromArray.map((breaker) => ({
provider: breaker.provider || breaker.name || "unknown",
state: breaker.state || "CLOSED",
lastFailure: breaker.lastFailure || null,
}))
: breakersFromProviderHealth;
const openBreakers = breakers.filter((breaker) => breaker.state === "OPEN");
setIncidentMode(openBreakers.length / Math.max(breakers.length, 1) > 0.5);
setExclusions(
openBreakers.map((breaker) => ({
provider: breaker.provider,
excludedAt: breaker.lastFailure || new Date().toISOString(),
cooldownMs: 5 * 60 * 1000,
reason: "Circuit breaker OPEN",
}))
);
} else {
setIncidentMode(false);
setExclusions([]);
}
} catch {
/* ignore */

View File

@@ -25,6 +25,51 @@ const SKILL_HANDLERS: Record<string, (task: any) => Promise<any>> = {
"quota-management": executeQuotaManagement,
};
type A2AMessage = { role: string; content: string };
function toMessageArray(raw: unknown): A2AMessage[] | null {
if (Array.isArray(raw)) {
const normalized = raw
.map((entry) => {
if (!entry || typeof entry !== "object" || Array.isArray(entry)) return null;
const msg = entry as Record<string, unknown>;
const role = typeof msg.role === "string" && msg.role.trim() ? msg.role : "user";
const content = typeof msg.content === "string" ? msg.content : null;
if (!content) return null;
return { role, content };
})
.filter((entry): entry is A2AMessage => !!entry);
return normalized.length > 0 ? normalized : null;
}
if (!raw || typeof raw !== "object" || Array.isArray(raw)) return null;
const message = raw as Record<string, unknown>;
const role = typeof message.role === "string" && message.role.trim() ? message.role : "user";
// Canonical A2A shape: { message: { role, content } }
if (typeof message.content === "string" && message.content.trim()) {
return [{ role, content: message.content }];
}
// Legacy compatibility: { message: { parts: [...] } }
if (Array.isArray(message.parts)) {
const text = message.parts
.map((part) => {
if (typeof part === "string") return part;
if (!part || typeof part !== "object" || Array.isArray(part)) return "";
const chunk = part as Record<string, unknown>;
if (typeof chunk.content === "string") return chunk.content;
if (typeof chunk.text === "string") return chunk.text;
return "";
})
.filter((chunk) => chunk.trim().length > 0)
.join("\n");
if (text) return [{ role, content: text }];
}
return null;
}
// ============ Auth ============
function authenticate(req: NextRequest): boolean {
@@ -77,9 +122,13 @@ export async function POST(req: NextRequest) {
// ── message/send ──────────────────────────────────────
case "message/send": {
const skill = params?.skill || "smart-routing";
const messages = params?.messages || params?.message?.parts;
if (!messages || !Array.isArray(messages)) {
return jsonRpcError(id, -32602, "Invalid params: messages array required");
const messages = toMessageArray(params?.messages) || toMessageArray(params?.message);
if (!messages) {
return jsonRpcError(
id,
-32602,
"Invalid params: provide `messages[]` or `message.content`"
);
}
const handler = SKILL_HANDLERS[skill];
@@ -125,9 +174,13 @@ export async function POST(req: NextRequest) {
// ── message/stream ────────────────────────────────────
case "message/stream": {
const skill = params?.skill || "smart-routing";
const messages = params?.messages || params?.message?.parts;
if (!messages || !Array.isArray(messages)) {
return jsonRpcError(id, -32602, "Invalid params: messages array required");
const messages = toMessageArray(params?.messages) || toMessageArray(params?.message);
if (!messages) {
return jsonRpcError(
id,
-32602,
"Invalid params: provide `messages[]` or `message.content`"
);
}
const handler = SKILL_HANDLERS[skill];

View File

@@ -28,10 +28,13 @@ export async function POST(request) {
if (isValidationFailure(validation)) {
return NextResponse.json({ error: validation.error }, { status: 400 });
}
const { password } = validation.data;
const password = typeof validation.data.password === "string" ? validation.data.password : "";
if (!password) {
return NextResponse.json({ error: "Invalid password payload" }, { status: 400 });
}
const settings = await getSettings();
const storedHash = settings.password;
const storedHash = typeof settings.password === "string" ? settings.password : "";
let isValid = false;
if (storedHash) {

View File

@@ -27,6 +27,10 @@ export async function POST(request) {
return value.slice(0, 4) + "****" + value.slice(-4);
}
function toOptionalString(value: unknown): string | null {
return typeof value === "string" ? value : null;
}
// Map connections — NEVER expose raw credentials
const mappedConnections = connections.map((conn) => ({
provider: conn.provider,
@@ -34,7 +38,7 @@ export async function POST(request) {
hasApiKey: !!conn.apiKey,
hasAccessToken: !!conn.accessToken,
hasRefreshToken: !!conn.refreshToken,
maskedApiKey: maskSecret(conn.apiKey),
maskedApiKey: maskSecret(toOptionalString(conn.apiKey)),
projectId: conn.projectId || null,
expiresAt: conn.expiresAt,
priority: conn.priority,

View File

@@ -45,7 +45,7 @@ export async function PUT(request: Request) {
}
// Update credentials
const updateData: Record<string, any> = {};
const updateData: Record<string, unknown> = {};
if (credentials.accessToken) {
updateData.accessToken = credentials.accessToken;
}
@@ -56,7 +56,11 @@ export async function PUT(request: Request) {
updateData.expiresAt = new Date(Date.now() + credentials.expiresIn * 1000).toISOString();
}
await updateProviderConnection(connection.id, updateData);
const connectionId = typeof connection.id === "string" ? connection.id : null;
if (!connectionId) {
return NextResponse.json({ error: "Invalid provider connection ID" }, { status: 500 });
}
await updateProviderConnection(connectionId, updateData);
return NextResponse.json({
success: true,

View File

@@ -35,7 +35,8 @@ export async function POST(request: Request) {
// Get model aliases
const modelAliases = await getModelAliases();
const resolved = modelAliases[alias];
const resolvedValue = modelAliases[alias];
const resolved = typeof resolvedValue === "string" ? resolvedValue : null;
if (resolved) {
// Parse provider/model

View File

@@ -24,9 +24,10 @@ export async function GET(request, { params }) {
}
// Mask the key value
const keyValue = typeof key.key === "string" ? key.key : null;
return NextResponse.json({
...key,
key: key.key ? key.key.slice(0, 8) + "****" + key.key.slice(-4) : null,
key: keyValue ? keyValue.slice(0, 8) + "****" + keyValue.slice(-4) : null,
});
} catch (error) {
console.log("Error fetching key:", error);

View File

@@ -11,7 +11,7 @@ export async function GET() {
// Mask key values — users should never see full keys after creation
const maskedKeys = keys.map((k) => ({
...k,
key: k.key ? k.key.slice(0, 8) + "****" + k.key.slice(-4) : null,
key: typeof k.key === "string" ? k.key.slice(0, 8) + "****" + k.key.slice(-4) : null,
}));
return NextResponse.json({ keys: maskedKeys });
} catch (error) {

View File

@@ -225,8 +225,9 @@ export async function POST(
const match = existing.find(
(c: any) => c.email === tokenData.email && c.authType === "oauth"
);
if (match) {
connection = await updateProviderConnection(match.id, {
const matchId = typeof match?.id === "string" ? match.id : null;
if (matchId) {
connection = await updateProviderConnection(matchId, {
...tokenData,
expiresAt,
testStatus: "active",
@@ -288,8 +289,9 @@ export async function POST(
const match = existing.find(
(c: any) => c.email === result.tokens.email && c.authType === "oauth"
);
if (match) {
connection = await updateProviderConnection(match.id, {
const matchId = typeof match?.id === "string" ? match.id : null;
if (matchId) {
connection = await updateProviderConnection(matchId, {
...result.tokens,
expiresAt,
testStatus: "active",
@@ -401,8 +403,9 @@ export async function POST(
const match = existing.find(
(c: any) => c.email === tokenData.email && c.authType === "oauth"
);
if (match) {
connection = await updateProviderConnection(match.id, {
const matchId = typeof match?.id === "string" ? match.id : null;
if (matchId) {
connection = await updateProviderConnection(matchId, {
...tokenData,
expiresAt,
testStatus: "active",

View File

@@ -2,6 +2,20 @@ import { NextResponse } from "next/server";
import { REGISTRY } from "@omniroute/open-sse/config/providerRegistry.ts";
import { getAllCustomModels, getPricing } from "@/lib/localDb";
function asRecord(value: unknown): Record<string, unknown> {
return value && typeof value === "object" && !Array.isArray(value)
? (value as Record<string, unknown>)
: {};
}
function asModelArray(value: unknown): Array<{ id?: string; name?: string }> {
if (!Array.isArray(value)) return [];
return value.filter((item) => item && typeof item === "object") as Array<{
id?: string;
name?: string;
}>;
}
/**
* GET /api/pricing/models
* Returns the full model catalog merged from three sources:
@@ -33,14 +47,15 @@ export async function GET() {
}
// ── 2. Custom models (DB) ───────────────────────────────────────
let customModelsMap: Record<string, any[]> = {};
let customModelsMap: Record<string, unknown> = {};
try {
customModelsMap = await getAllCustomModels();
customModelsMap = asRecord(await getAllCustomModels());
} catch {
/* DB may not be ready */
}
for (const [providerId, models] of Object.entries(customModelsMap)) {
for (const [providerId, rawModels] of Object.entries(customModelsMap)) {
const models = asModelArray(rawModels);
// Resolve alias — check if a registry entry maps this providerId
let alias = providerId;
for (const entry of Object.values(REGISTRY)) {
@@ -63,13 +78,17 @@ export async function GET() {
const existingIds = new Set(catalog[alias].models.map((m) => m.id));
for (const model of models) {
if (!existingIds.has(model.id)) {
const modelId = typeof model.id === "string" ? model.id : null;
if (!modelId || existingIds.has(modelId)) {
continue;
}
if (!existingIds.has(modelId)) {
catalog[alias].models.push({
id: model.id,
name: model.name || model.id,
id: modelId,
name: typeof model.name === "string" && model.name.trim() ? model.name : modelId,
custom: true,
});
existingIds.add(model.id);
existingIds.add(modelId);
}
}
}

View File

@@ -1,6 +1,17 @@
import { NextResponse } from "next/server";
import { getDbInstance } from "@/lib/db/core";
type JsonRecord = Record<string, unknown>;
function toNumber(value: unknown): number {
if (typeof value === "number" && Number.isFinite(value)) return value;
if (typeof value === "string" && value.trim().length > 0) {
const parsed = Number(value);
return Number.isFinite(parsed) ? parsed : 0;
}
return 0;
}
/**
* GET /api/providers/metrics — Aggregate per-provider stats from call_logs
* Returns: { metrics: { [provider]: { totalRequests, totalSuccesses, successRate, avgLatencyMs } } }
@@ -19,16 +30,30 @@ export async function GET() {
WHERE provider IS NOT NULL AND provider != '-'
GROUP BY provider`
)
.all();
.all() as JsonRecord[];
const metrics = {};
const metrics: Record<
string,
{
totalRequests: number;
totalSuccesses: number;
successRate: number;
avgLatencyMs: number;
}
> = {};
for (const row of rows) {
metrics[row.provider] = {
totalRequests: row.totalRequests,
totalSuccesses: row.totalSuccesses,
successRate:
row.totalRequests > 0 ? Math.round((row.totalSuccesses / row.totalRequests) * 100) : 0,
avgLatencyMs: row.avgLatencyMs || 0,
const provider =
typeof row.provider === "string" && row.provider.trim().length > 0
? row.provider
: "unknown";
const totalRequests = toNumber(row.totalRequests);
const totalSuccesses = toNumber(row.totalSuccesses);
const avgLatencyMs = toNumber(row.avgLatencyMs);
metrics[provider] = {
totalRequests,
totalSuccesses,
successRate: totalRequests > 0 ? Math.round((totalSuccesses / totalRequests) * 100) : 0,
avgLatencyMs,
};
}

View File

@@ -13,6 +13,12 @@ import {
validateBody,
} from "@/shared/validation/schemas";
type JsonRecord = Record<string, unknown>;
function asRecord(value: unknown): JsonRecord {
return value && typeof value === "object" && !Array.isArray(value) ? (value as JsonRecord) : {};
}
// PUT /api/provider-nodes/[id] - Update provider node
export async function PUT(request: Request, { params }: { params: Promise<{ id: string }> }) {
let rawBody;
@@ -61,7 +67,7 @@ export async function PUT(request: Request, { params }: { params: Promise<{ id:
}
}
const updates: Record<string, any> = {
const updates: Record<string, unknown> = {
name: name.trim(),
prefix: prefix.trim(),
baseUrl: sanitizedBaseUrl,
@@ -75,17 +81,27 @@ export async function PUT(request: Request, { params }: { params: Promise<{ id:
const connections = await getProviderConnections({ provider: id });
await Promise.all(
connections.map((connection) =>
updateProviderConnection(connection.id, {
providerSpecificData: {
...(connection.providerSpecificData || {}),
prefix: prefix.trim(),
apiType: node.type === "openai-compatible" ? apiType : undefined,
baseUrl: sanitizedBaseUrl,
nodeName: updated.name,
},
})
)
connections.flatMap((connectionRaw) => {
const connection = asRecord(connectionRaw);
const connectionId = typeof connection.id === "string" ? connection.id : "";
if (!connectionId) return [];
const providerSpecificData = {
...asRecord(connection.providerSpecificData),
prefix: prefix.trim(),
baseUrl: sanitizedBaseUrl,
nodeName: updated.name,
} as JsonRecord;
if (node.type === "openai-compatible") {
providerSpecificData.apiType = apiType;
}
return [
updateProviderConnection(connectionId, {
providerSpecificData,
}),
];
})
);
return NextResponse.json({ node: updated });

View File

@@ -5,6 +5,29 @@ import {
isAnthropicCompatibleProvider,
} from "@/shared/constants/providers";
type JsonRecord = Record<string, unknown>;
function asRecord(value: unknown): JsonRecord {
return value && typeof value === "object" && !Array.isArray(value) ? (value as JsonRecord) : {};
}
function getProviderBaseUrl(providerSpecificData: unknown): string | null {
const data = asRecord(providerSpecificData);
const baseUrl = data.baseUrl;
return typeof baseUrl === "string" && baseUrl.trim().length > 0 ? baseUrl : null;
}
type ProviderModelsConfigEntry = {
url: string;
method: "GET" | "POST";
headers: Record<string, string>;
authHeader?: string;
authPrefix?: string;
authQuery?: string;
body?: unknown;
parseResponse: (data: any) => any;
};
// Providers that return hardcoded models (no remote /models API)
const STATIC_MODEL_PROVIDERS = {
deepgram: () => [
@@ -33,7 +56,7 @@ const STATIC_MODEL_PROVIDERS = {
};
// Provider models endpoints configuration
const PROVIDER_MODELS_CONFIG = {
const PROVIDER_MODELS_CONFIG: Record<string, ProviderModelsConfigEntry> = {
claude: {
url: "https://api.anthropic.com/v1/models",
method: "GET",
@@ -238,8 +261,20 @@ export async function GET(request, { params }) {
return NextResponse.json({ error: "Connection not found" }, { status: 404 });
}
if (isOpenAICompatibleProvider(connection.provider)) {
const baseUrl = connection.providerSpecificData?.baseUrl;
const provider =
typeof connection.provider === "string" && connection.provider.trim().length > 0
? connection.provider
: null;
if (!provider) {
return NextResponse.json({ error: "Invalid connection provider" }, { status: 400 });
}
const connectionId = typeof connection.id === "string" ? connection.id : id;
const apiKey = typeof connection.apiKey === "string" ? connection.apiKey : "";
const accessToken = typeof connection.accessToken === "string" ? connection.accessToken : "";
if (isOpenAICompatibleProvider(provider)) {
const baseUrl = getProviderBaseUrl(connection.providerSpecificData);
if (!baseUrl) {
return NextResponse.json(
{ error: "No base URL configured for OpenAI compatible provider" },
@@ -260,13 +295,13 @@ export async function GET(request, { params }) {
method: "GET",
headers: {
"Content-Type": "application/json",
Authorization: `Bearer ${connection.apiKey}`,
Authorization: `Bearer ${apiKey}`,
},
});
if (!response.ok) {
const errorText = await response.text();
console.log(`Error fetching models from ${connection.provider}:`, errorText);
console.log(`Error fetching models from ${provider}:`, errorText);
return NextResponse.json(
{ error: `Failed to fetch models: ${response.status}` },
{ status: response.status }
@@ -277,14 +312,14 @@ export async function GET(request, { params }) {
const models = data.data || data.models || [];
return NextResponse.json({
provider: connection.provider,
connectionId: connection.id,
provider,
connectionId,
models,
});
}
if (isAnthropicCompatibleProvider(connection.provider)) {
let baseUrl = connection.providerSpecificData?.baseUrl;
if (isAnthropicCompatibleProvider(provider)) {
let baseUrl = getProviderBaseUrl(connection.providerSpecificData);
if (!baseUrl) {
return NextResponse.json(
{ error: "No base URL configured for Anthropic compatible provider" },
@@ -302,15 +337,15 @@ export async function GET(request, { params }) {
method: "GET",
headers: {
"Content-Type": "application/json",
"x-api-key": connection.apiKey,
"x-api-key": apiKey,
"anthropic-version": "2023-06-01",
Authorization: `Bearer ${connection.apiKey}`,
Authorization: `Bearer ${apiKey}`,
},
});
if (!response.ok) {
const errorText = await response.text();
console.log(`Error fetching models from ${connection.provider}:`, errorText);
console.log(`Error fetching models from ${provider}:`, errorText);
return NextResponse.json(
{ error: `Failed to fetch models: ${response.status}` },
{ status: response.status }
@@ -321,32 +356,38 @@ export async function GET(request, { params }) {
const models = data.data || data.models || [];
return NextResponse.json({
provider: connection.provider,
connectionId: connection.id,
provider,
connectionId,
models,
});
}
// Static model providers (no remote /models API)
const staticModelsFn = STATIC_MODEL_PROVIDERS[connection.provider];
const staticModelsFn =
provider in STATIC_MODEL_PROVIDERS
? STATIC_MODEL_PROVIDERS[provider as keyof typeof STATIC_MODEL_PROVIDERS]
: undefined;
if (staticModelsFn) {
return NextResponse.json({
provider: connection.provider,
connectionId: connection.id,
provider,
connectionId,
models: staticModelsFn(),
});
}
const config = PROVIDER_MODELS_CONFIG[connection.provider];
const config =
provider in PROVIDER_MODELS_CONFIG
? PROVIDER_MODELS_CONFIG[provider as keyof typeof PROVIDER_MODELS_CONFIG]
: undefined;
if (!config) {
return NextResponse.json(
{ error: `Provider ${connection.provider} does not support models listing` },
{ error: `Provider ${provider} does not support models listing` },
{ status: 400 }
);
}
// Get auth token
const token = connection.accessToken || connection.apiKey;
const token = accessToken || apiKey;
if (!token) {
return NextResponse.json({ error: "No valid token found" }, { status: 401 });
}
@@ -377,7 +418,7 @@ export async function GET(request, { params }) {
if (!response.ok) {
const errorText = await response.text();
console.log(`Error fetching models from ${connection.provider}:`, errorText);
console.log(`Error fetching models from ${provider}:`, errorText);
return NextResponse.json(
{ error: `Failed to fetch models: ${response.status}` },
{ status: response.status }
@@ -388,8 +429,8 @@ export async function GET(request, { params }) {
const models = config.parseResponse(data);
return NextResponse.json({
provider: connection.provider,
connectionId: connection.id,
provider,
connectionId,
models,
});
} catch (error) {

View File

@@ -531,6 +531,21 @@ export async function testSingleConnection(connectionId: string) {
return { valid: false, error: "Connection not found", diagnosis: null, latencyMs: 0 };
}
const provider = typeof connection.provider === "string" ? connection.provider : "";
if (!provider) {
return {
valid: false,
error: "Connection provider is invalid",
diagnosis: makeDiagnosis(
"validation_error",
"local",
"Connection provider is invalid",
"provider_invalid"
),
latencyMs: 0,
};
}
// Resolve proxy for this connection (key → combo → provider → global → direct)
let proxyInfo: any = null;
try {
@@ -541,7 +556,7 @@ export async function testSingleConnection(connectionId: string) {
let result;
const startTime = Date.now();
const runtime = await getProviderRuntimeStatus(connection.provider);
const runtime = await getProviderRuntimeStatus(provider);
if ((runtime as any)?.diagnosis) {
result = {
@@ -611,7 +626,7 @@ export async function testSingleConnection(connectionId: string) {
path: "/api/providers/test",
status: result.valid ? 200 : result.statusCode || 401,
model: "connection-test",
provider: connection.provider,
provider,
connectionId,
duration: latencyMs,
error: result.valid ? null : result.error || null,
@@ -627,8 +642,8 @@ export async function testSingleConnection(connectionId: string) {
proxy: proxyInfo?.proxy || null,
level: proxyInfo?.level || "provider-test",
levelId: proxyInfo?.levelId || null,
provider: connection.provider,
targetUrl: `${connection.provider}/connection-test`,
provider,
targetUrl: `${provider}/connection-test`,
latencyMs,
error: result.valid ? null : result.error || null,
connectionId,

View File

@@ -14,6 +14,12 @@ import {
validateBody,
} from "@/shared/validation/schemas";
type JsonRecord = Record<string, unknown>;
function asRecord(value: unknown): JsonRecord {
return value && typeof value === "object" && !Array.isArray(value) ? (value as JsonRecord) : {};
}
/**
* GET /api/rate-limits — Consolidated rate-limit status
*
@@ -26,13 +32,23 @@ import {
export async function GET() {
try {
const connections = await getProviderConnections();
const statuses = connections.map((conn) => ({
connectionId: conn.id,
provider: conn.provider,
name: conn.name || conn.email || conn.id.slice(0, 8),
rateLimitProtection: !!conn.rateLimitProtection,
...getRateLimitStatus(conn.provider, conn.id),
}));
const statuses = connections.map((connRaw) => {
const conn = asRecord(connRaw);
const connectionId = typeof conn.id === "string" ? conn.id : "";
const provider = typeof conn.provider === "string" ? conn.provider : "unknown";
const name =
(typeof conn.name === "string" && conn.name.trim()) ||
(typeof conn.email === "string" && conn.email.trim()) ||
(connectionId ? connectionId.slice(0, 8) : "unknown");
return {
connectionId,
provider,
name,
rateLimitProtection: conn.rateLimitProtection === true,
...getRateLimitStatus(provider, connectionId),
};
});
const lockouts = getAllModelLockouts();
const cacheStats = getCacheStats();

View File

@@ -6,6 +6,16 @@ import {
validateBody,
} from "@/shared/validation/schemas";
type JsonRecord = Record<string, unknown>;
function asRecord(value: unknown): JsonRecord {
return value && typeof value === "object" && !Array.isArray(value) ? (value as JsonRecord) : {};
}
function getErrorMessage(error: unknown, fallback: string): string {
return error instanceof Error && error.message ? error.message : fallback;
}
/**
* GET /api/resilience — Get current resilience configuration and status
*/
@@ -24,14 +34,17 @@ export async function GET() {
return NextResponse.json({
profiles: settings.providerProfiles || PROVIDER_PROFILES,
defaults: { ...DEFAULT_API_LIMITS, ...(settings.rateLimitDefaults || {}) },
defaults: {
...DEFAULT_API_LIMITS,
...asRecord(settings.rateLimitDefaults),
},
circuitBreakers,
rateLimitStatus,
});
} catch (err) {
} catch (err: unknown) {
console.error("[API] GET /api/resilience error:", err);
return NextResponse.json(
{ error: err.message || "Failed to load resilience status" },
{ error: getErrorMessage(err, "Failed to load resilience status") },
{ status: 500 }
);
}
@@ -74,10 +87,10 @@ export async function PATCH(request) {
...(profiles ? { profiles } : {}),
...(defaults ? { defaults } : {}),
});
} catch (err) {
} catch (err: unknown) {
console.error("[API] PATCH /api/resilience error:", err);
return NextResponse.json(
{ error: err.message || "Failed to save resilience settings" },
{ error: getErrorMessage(err, "Failed to save resilience settings") },
{ status: 500 }
);
}

View File

@@ -45,7 +45,7 @@ export async function PATCH(request) {
// If updating password, hash it
if (body.newPassword) {
const settings = await getSettings();
const currentHash = settings.password;
const currentHash = typeof settings.password === "string" ? settings.password : "";
// Verify current password if it exists
if (currentHash) {

View File

@@ -14,6 +14,12 @@ import {
validateBody,
} from "@/shared/validation/schemas";
function getProviderBaseUrl(providerSpecificData: unknown): string | undefined {
if (!providerSpecificData || typeof providerSpecificData !== "object") return undefined;
const baseUrl = (providerSpecificData as Record<string, unknown>).baseUrl;
return typeof baseUrl === "string" && baseUrl.trim().length > 0 ? baseUrl : undefined;
}
export async function POST(request) {
let rawBody;
try {
@@ -78,7 +84,7 @@ export async function POST(request) {
// Build URL and headers using provider service
const url = buildProviderUrl(provider, body.model || "test-model", true, {
baseUrlIndex: 0,
baseUrl: connection.providerSpecificData?.baseUrl,
baseUrl: getProviderBaseUrl(connection.providerSpecificData),
});
const headers = buildProviderHeaders(provider, credentials, true, body);

View File

@@ -33,6 +33,12 @@ function getModelId(value: JsonRecord): string {
return typeof model === "string" && model.trim().length > 0 ? model : "test-model";
}
function getProviderBaseUrl(providerSpecificData: unknown): string | undefined {
const data = asJsonRecord(providerSpecificData);
const baseUrl = data.baseUrl;
return typeof baseUrl === "string" && baseUrl.trim().length > 0 ? baseUrl : undefined;
}
export async function POST(request) {
let rawBody;
try {
@@ -177,7 +183,7 @@ export async function POST(request) {
// Build URL and headers
const url = buildProviderUrl(provider, model, true, {
baseUrlIndex: 0,
baseUrl: connection.providerSpecificData?.baseUrl,
baseUrl: getProviderBaseUrl(connection.providerSpecificData),
});
const headers = buildProviderHeaders(provider, credentials, true, actualBody);

View File

@@ -12,11 +12,23 @@ export async function GET(request) {
// Build connection map for account names
const { getProviderConnections } = await import("@/lib/localDb");
let connectionMap = {};
const connectionMap: Record<string, string> = {};
try {
const connections = await getProviderConnections();
for (const conn of connections) {
connectionMap[conn.id] = conn.name || conn.email || conn.id;
for (const connRaw of connections as unknown[]) {
const conn =
connRaw && typeof connRaw === "object" && !Array.isArray(connRaw)
? (connRaw as Record<string, unknown>)
: {};
const connectionId =
typeof conn.id === "string" && conn.id.trim().length > 0 ? conn.id : null;
if (!connectionId) continue;
const name =
(typeof conn.name === "string" && conn.name.trim()) ||
(typeof conn.email === "string" && conn.email.trim()) ||
connectionId;
connectionMap[connectionId] = name;
}
} catch {
/* ignore */

View File

@@ -42,7 +42,7 @@ function deriveTokenStatus(connection: ProviderConnectionRecord): QuotaTokenStat
function buildQuotaEntry(
connection: ProviderConnectionRecord,
learnedLimit: Record<string, unknown> | null,
learnedLimit: unknown,
rateStatus: Record<string, unknown>
): QuotaProviderEntry {
const provider =
@@ -64,16 +64,18 @@ function buildQuotaEntry(
let quotaTotal: number | null = null;
let quotaUsed = 0;
let percentRemaining = 100;
const learned =
learnedLimit && typeof learnedLimit === "object" && !Array.isArray(learnedLimit)
? (learnedLimit as Record<string, unknown>)
: null;
const learnedLimitValue =
learnedLimit && typeof learnedLimit.limit === "number" && Number.isFinite(learnedLimit.limit)
? learnedLimit.limit
learned && typeof learned.limit === "number" && Number.isFinite(learned.limit)
? learned.limit
: null;
const learnedRemainingValue =
learnedLimit &&
typeof learnedLimit.remaining === "number" &&
Number.isFinite(learnedLimit.remaining)
? learnedLimit.remaining
learned && typeof learned.remaining === "number" && Number.isFinite(learned.remaining)
? learned.remaining
: null;
if (learnedLimitValue !== null && learnedLimitValue > 0) {

View File

@@ -332,8 +332,14 @@ export async function getUnifiedModelsResponse(
// Add custom models (user-defined)
try {
const customModelsMap: Record<string, any[]> = await getAllCustomModels();
for (const [providerId, providerCustomModels] of Object.entries(customModelsMap)) {
const customModelsMap = (await getAllCustomModels()) as Record<string, unknown>;
for (const [providerId, rawProviderCustomModels] of Object.entries(customModelsMap)) {
const providerCustomModels = Array.isArray(rawProviderCustomModels)
? rawProviderCustomModels.filter(
(model): model is Record<string, unknown> =>
!!model && typeof model === "object" && !Array.isArray(model)
)
: [];
// For compatible providers, use the prefix from provider nodes
const prefix = providerIdToPrefix[providerId];
const alias = prefix || providerIdToAlias[providerId] || providerId;
@@ -351,8 +357,11 @@ export async function getUnifiedModelsResponse(
continue;
for (const model of providerCustomModels) {
const modelId = typeof model.id === "string" ? model.id : null;
if (!modelId) continue;
// Skip if already added as built-in
const aliasId = `${alias}/${model.id}`;
const aliasId = `${alias}/${modelId}`;
if (models.some((m) => m.id === aliasId)) continue;
models.push({
@@ -361,14 +370,14 @@ export async function getUnifiedModelsResponse(
created: timestamp,
owned_by: canonicalProviderId,
permission: [],
root: model.id,
root: modelId,
parent: null,
custom: true,
});
// Only add provider-prefixed version if different from alias
if (canonicalProviderId !== alias && !prefix) {
const providerPrefixedId = `${canonicalProviderId}/${model.id}`;
const providerPrefixedId = `${canonicalProviderId}/${modelId}`;
if (models.some((m) => m.id === providerPrefixedId)) continue;
models.push({
id: providerPrefixedId,
@@ -376,7 +385,7 @@ export async function getUnifiedModelsResponse(
created: timestamp,
owned_by: canonicalProviderId,
permission: [],
root: model.id,
root: modelId,
parent: aliasId,
custom: true,
});

View File

@@ -9,7 +9,6 @@
* @module domain/costRules
*/
import {
saveBudget,
loadBudget,
@@ -20,6 +19,32 @@ import {
deleteCostEntries,
} from "../lib/db/domainState";
interface BudgetConfig {
dailyLimitUsd: number;
monthlyLimitUsd?: number;
warningThreshold?: number;
}
interface CostEntry {
cost: number;
timestamp: number;
}
function toCostEntries(value: unknown): CostEntry[] {
if (!Array.isArray(value)) return [];
const entries: CostEntry[] = [];
for (const item of value) {
if (!item || typeof item !== "object" || Array.isArray(item)) continue;
const record = item as Record<string, unknown>;
const cost = typeof record.cost === "number" ? record.cost : Number(record.cost ?? 0);
const timestamp =
typeof record.timestamp === "number" ? record.timestamp : Number(record.timestamp ?? 0);
if (!Number.isFinite(cost) || !Number.isFinite(timestamp)) continue;
entries.push({ cost, timestamp });
}
return entries;
}
/**
* @typedef {Object} BudgetConfig
* @property {number} dailyLimitUsd - Max daily spend in USD
@@ -34,7 +59,7 @@ import {
*/
/** @type {Map<string, BudgetConfig>} In-memory cache for budgets */
const budgets = new Map();
const budgets = new Map<string, BudgetConfig>();
/** @type {boolean} */
let _budgetsLoaded = false;
@@ -45,7 +70,7 @@ let _budgetsLoaded = false;
* @param {string} apiKeyId
* @param {BudgetConfig} config
*/
export function setBudget(apiKeyId, config) {
export function setBudget(apiKeyId: string, config: BudgetConfig) {
const normalized = {
dailyLimitUsd: config.dailyLimitUsd,
monthlyLimitUsd: config.monthlyLimitUsd || 0,
@@ -65,14 +90,14 @@ export function setBudget(apiKeyId, config) {
* @param {string} apiKeyId
* @returns {BudgetConfig | null}
*/
export function getBudget(apiKeyId) {
export function getBudget(apiKeyId: string): BudgetConfig | null {
// Check in-memory cache first
if (budgets.has(apiKeyId)) {
return budgets.get(apiKeyId);
}
// Try loading from DB
try {
const fromDb = loadBudget(apiKeyId);
const fromDb = loadBudget(apiKeyId) as BudgetConfig | null;
if (fromDb) {
budgets.set(apiKeyId, fromDb);
return fromDb;
@@ -89,7 +114,7 @@ export function getBudget(apiKeyId) {
* @param {string} apiKeyId
* @param {number} cost - Cost in USD
*/
export function recordCost(apiKeyId, cost) {
export function recordCost(apiKeyId: string, cost: number): void {
const timestamp = Date.now();
try {
saveCostEntry(apiKeyId, cost, timestamp);
@@ -105,7 +130,7 @@ export function recordCost(apiKeyId, cost) {
* @param {number} [additionalCost=0] - Projected cost to check
* @returns {{ allowed: boolean, reason?: string, dailyUsed: number, dailyLimit: number, warningReached: boolean }}
*/
export function checkBudget(apiKeyId, additionalCost = 0) {
export function checkBudget(apiKeyId: string, additionalCost = 0) {
const budget = getBudget(apiKeyId);
if (!budget) {
return { allowed: true, dailyUsed: 0, dailyLimit: 0, warningReached: false };
@@ -139,13 +164,13 @@ export function checkBudget(apiKeyId, additionalCost = 0) {
* @param {string} apiKeyId
* @returns {number} Total cost today in USD
*/
export function getDailyTotal(apiKeyId) {
export function getDailyTotal(apiKeyId: string): number {
const todayStart = new Date();
todayStart.setHours(0, 0, 0, 0);
const startMs = todayStart.getTime();
try {
const entries = loadCostEntries(apiKeyId, startMs);
const entries = toCostEntries(loadCostEntries(apiKeyId, startMs));
return entries.reduce((sum, e) => sum + e.cost, 0);
} catch {
return 0;
@@ -158,7 +183,7 @@ export function getDailyTotal(apiKeyId) {
* @param {string} apiKeyId
* @returns {{ dailyTotal: number, monthlyTotal: number, totalEntries: number, budget: BudgetConfig | null }}
*/
export function getCostSummary(apiKeyId) {
export function getCostSummary(apiKeyId: string) {
const now = new Date();
const todayStart = new Date(now);
@@ -167,8 +192,8 @@ export function getCostSummary(apiKeyId) {
const monthStart = new Date(now.getFullYear(), now.getMonth(), 1);
try {
const dailyEntries = loadCostEntries(apiKeyId, todayStart.getTime());
const monthlyEntries = loadCostEntries(apiKeyId, monthStart.getTime());
const dailyEntries = toCostEntries(loadCostEntries(apiKeyId, todayStart.getTime()));
const monthlyEntries = toCostEntries(loadCostEntries(apiKeyId, monthStart.getTime()));
const dailyTotal = dailyEntries.reduce((sum, e) => sum + e.cost, 0);
const monthlyTotal = monthlyEntries.reduce((sum, e) => sum + e.cost, 0);

748
src/lib/a2a/README.md Normal file
View File

@@ -0,0 +1,748 @@
# OmniRoute A2A Server
> **Agent-to-Agent Protocol v0.3** — Enables any AI agent to use OmniRoute as an intelligent routing agent via JSON-RPC 2.0.
The A2A Server exposes OmniRoute as a **first-class agent** that other agents can discover, delegate tasks to, and collaborate with using the [A2A Protocol](https://google.github.io/A2A/).
---
## Architecture
```
┌──────────────────────────────────────────────────────────────────┐
│ Orchestrator Agent │
│ (LangChain, CrewAI, AutoGen, Custom Agent) │
└──────────────────────┬───────────────────────────────────────────┘
│ 1. GET /.well-known/agent.json (discover)
│ 2. POST /a2a (JSON-RPC 2.0)
┌──────────────────────────────────────────────────────────────────┐
│ OmniRoute A2A Server │
│ ┌────────────────┐ ┌────────────────┐ ┌───────────────────┐ │
│ │ Task Manager │ │ Skill Engine │ │ SSE Streaming │ │
│ │ (lifecycle) │──│ (registry) │──│ (real-time) │ │
│ └────────────────┘ └────────┬───────┘ └───────────────────┘ │
│ │ │
│ Skills: │ │
│ ├─ smart-routing ──────────┤ ┌────────────────────────────┐ │
│ └─ quota-management ───────┘ │ Routing Decision Logger │ │
│ └────────────────────────────┘ │
└──────────────────────────────────────────────────────────────────┘
▼ OmniRoute Gateway (internal)
/v1/chat/completions, /api/combos, /api/usage/quota
```
---
## Quick Start
### Agent Discovery
Every A2A-compatible agent exposes an **Agent Card** at `/.well-known/agent.json`:
```bash
curl http://localhost:20128/.well-known/agent.json
```
**Response:**
```json
{
"name": "OmniRoute",
"description": "Intelligent AI gateway with auto-routing across 50+ providers",
"url": "http://localhost:20128/a2a",
"version": "1.8.1",
"capabilities": {
"streaming": true,
"pushNotifications": false
},
"skills": [
{
"id": "smart-routing",
"name": "Smart Routing",
"description": "Routes prompts through OmniRoute intelligent pipeline",
"tags": ["routing", "llm", "multi-provider", "cost-optimization"],
"examples": [
"Write a hello world in Python",
"Explain quantum computing using the cheapest provider"
]
},
{
"id": "quota-management",
"name": "Quota Management",
"description": "Natural-language queries about provider quotas",
"tags": ["quota", "analytics", "cost"],
"examples": [
"Which provider has the most quota remaining?",
"Suggest a free combo for coding"
]
}
],
"authentication": {
"schemes": ["bearer"],
"apiKeyHeader": "Authorization"
}
}
```
---
## JSON-RPC 2.0 Methods
### `message/send` — Synchronous Execution
Send a message to a skill and receive the complete response.
```bash
curl -X POST http://localhost:20128/a2a \
-H "Content-Type: application/json" \
-H "Authorization: Bearer YOUR_KEY" \
-d '{
"jsonrpc": "2.0",
"id": "1",
"method": "message/send",
"params": {
"skill": "smart-routing",
"messages": [{"role": "user", "content": "Write a Python hello world"}],
"metadata": {"model": "auto", "combo": "fast-coding"}
}
}'
```
**Response:**
```json
{
"jsonrpc": "2.0",
"id": "1",
"result": {
"task": { "id": "a1b2c3d4-...", "state": "completed" },
"artifacts": [{ "type": "text", "content": "print('Hello, World!')" }],
"metadata": {
"routing_explanation": "Selected claude-sonnet via provider \"anthropic\" (latency: 1200ms, cost: $0.0030)",
"cost_envelope": { "estimated": 0.005, "actual": 0.003, "currency": "USD" },
"resilience_trace": [
{ "event": "primary_selected", "provider": "anthropic", "timestamp": "2026-03-04T..." }
],
"policy_verdict": { "allowed": true, "reason": "within budget and quota limits" }
}
}
}
```
### `message/stream` — SSE Streaming
Same as `message/send` but returns Server-Sent Events for real-time streaming.
```bash
curl -N -X POST http://localhost:20128/a2a \
-H "Content-Type: application/json" \
-H "Authorization: Bearer YOUR_KEY" \
-d '{
"jsonrpc": "2.0",
"id": "1",
"method": "message/stream",
"params": {
"skill": "smart-routing",
"messages": [{"role": "user", "content": "Explain quantum computing"}]
}
}'
```
**SSE Events:**
```
data: {"jsonrpc":"2.0","method":"message/stream","params":{"task":{"id":"...","state":"working"},"chunk":{"type":"text","content":"Quantum computing..."}}}
: heartbeat 2026-03-04T21:00:00Z
data: {"jsonrpc":"2.0","method":"message/stream","params":{"task":{"id":"...","state":"completed"},"metadata":{...}}}
```
### `tasks/get` — Query Task Status
```bash
curl -X POST http://localhost:20128/a2a \
-H "Content-Type: application/json" \
-H "Authorization: Bearer YOUR_KEY" \
-d '{"jsonrpc":"2.0","id":"2","method":"tasks/get","params":{"taskId":"TASK_UUID"}}'
```
### `tasks/cancel` — Cancel a Running Task
```bash
curl -X POST http://localhost:20128/a2a \
-H "Content-Type: application/json" \
-H "Authorization: Bearer YOUR_KEY" \
-d '{"jsonrpc":"2.0","id":"3","method":"tasks/cancel","params":{"taskId":"TASK_UUID"}}'
```
---
## Skills Reference
### `smart-routing`
Routes prompts through OmniRoute's intelligent pipeline with full observability.
**Parameters (in `metadata`):**
| Parameter | Type | Default | Description |
| --------- | -------- | ------------ | ---------------------------------------------------------------------------------------- |
| `model` | `string` | `"auto"` | Target model (e.g., `claude-sonnet-4`, `gpt-4o`, `auto`) |
| `combo` | `string` | active combo | Specific combo to route through |
| `budget` | `number` | none | Maximum cost in USD for this request |
| `role` | `string` | none | Task role hint: `coding`, `review`, `planning`, `analysis`, `debugging`, `documentation` |
**Returns:**
| Field | Description |
| ------------------------------ | --------------------------------------------------------- |
| `artifacts[].content` | The LLM response text |
| `metadata.routing_explanation` | Human-readable explanation of routing decision |
| `metadata.cost_envelope` | Estimated vs actual cost with currency |
| `metadata.resilience_trace` | Array of events (primary_selected, fallback_needed, etc.) |
| `metadata.policy_verdict` | Whether the request was allowed and why |
### `quota-management`
Answers natural-language queries about provider quotas.
**Query types (inferred from message content):**
| Query Pattern | Response Type |
| ---------------------------------------------- | -------------------------------------------------------- |
| Contains `"ranking"`, `"most quota"`, `"best"` | Providers ranked by remaining quota |
| Contains `"free"`, `"suggest"` | Lists free combos or suggests free-tier providers |
| Default | Full quota summary with warnings for low-quota providers |
---
## Task Lifecycle
```
submitted ──→ working ──→ completed
──→ failed
──────────→ cancelled
```
| State | Description |
| ----------- | ----------------------------------------------------- |
| `submitted` | Task created, queued for execution |
| `working` | Skill handler is executing |
| `completed` | Execution succeeded, artifacts available |
| `failed` | Execution failed or task expired (TTL: 5 min default) |
| `cancelled` | Cancelled by client via `tasks/cancel` |
- Terminal states: `completed`, `failed`, `cancelled` (no further transitions)
- Expired tasks in `submitted` or `working` are auto-marked as `failed`
- Tasks are garbage-collected after 2× TTL
---
## Client Examples
### Python — Orchestrator Agent
```python
"""
A2A Client — Python example.
Discovers OmniRoute agent, sends a task, and processes the result.
"""
import requests
import json
BASE_URL = "http://localhost:20128"
API_KEY = "your-api-key"
HEADERS = {
"Content-Type": "application/json",
"Authorization": f"Bearer {API_KEY}",
}
# 1. Discover agent capabilities
agent_card = requests.get(f"{BASE_URL}/.well-known/agent.json").json()
print(f"Agent: {agent_card['name']} v{agent_card['version']}")
print(f"Skills: {[s['id'] for s in agent_card['skills']]}")
# 2. Send a smart-routing task
response = requests.post(f"{BASE_URL}/a2a", headers=HEADERS, json={
"jsonrpc": "2.0",
"id": "task-1",
"method": "message/send",
"params": {
"skill": "smart-routing",
"messages": [{"role": "user", "content": "Write a Python quicksort implementation"}],
"metadata": {
"model": "auto",
"combo": "fast-coding",
"budget": 0.10,
}
}
})
result = response.json()["result"]
print(f"\n📝 Response: {result['artifacts'][0]['content'][:200]}...")
print(f"🔀 Routing: {result['metadata']['routing_explanation']}")
print(f"💰 Cost: ${result['metadata']['cost_envelope']['actual']}")
print(f"🛡️ Policy: {result['metadata']['policy_verdict']['reason']}")
# 3. Query quota status
quota_resp = requests.post(f"{BASE_URL}/a2a", headers=HEADERS, json={
"jsonrpc": "2.0",
"id": "task-2",
"method": "message/send",
"params": {
"skill": "quota-management",
"messages": [{"role": "user", "content": "Which provider has the most quota remaining?"}],
}
})
quota_result = quota_resp.json()["result"]
print(f"\n📊 Quota: {quota_result['artifacts'][0]['content']}")
```
### TypeScript — Multi-Agent Orchestrator
```typescript
/**
* A2A Client — TypeScript example.
* Shows agent discovery, task delegation, and streaming.
*/
const BASE_URL = "http://localhost:20128";
const API_KEY = "your-api-key";
interface JsonRpcResponse<T = any> {
jsonrpc: "2.0";
id: string | number;
result?: T;
error?: { code: number; message: string };
}
async function a2aCall<T>(method: string, params: Record<string, any>): Promise<T> {
const resp = await fetch(`${BASE_URL}/a2a`, {
method: "POST",
headers: {
"Content-Type": "application/json",
Authorization: `Bearer ${API_KEY}`,
},
body: JSON.stringify({
jsonrpc: "2.0",
id: `${method}-${Date.now()}`,
method,
params,
}),
});
const json: JsonRpcResponse<T> = await resp.json();
if (json.error) throw new Error(`[${json.error.code}] ${json.error.message}`);
return json.result!;
}
// ── Agent Discovery ──
const agentCard = await fetch(`${BASE_URL}/.well-known/agent.json`).then((r) => r.json());
console.log(`Connected to: ${agentCard.name} (${agentCard.skills.length} skills)`);
// ── Smart Routing: Send a coding task ──
const routingResult = await a2aCall("message/send", {
skill: "smart-routing",
messages: [{ role: "user", content: "Implement a Redis cache wrapper in TypeScript" }],
metadata: { model: "claude-sonnet-4", role: "coding" },
});
console.log("Response:", routingResult.artifacts[0].content);
console.log("Provider:", routingResult.metadata.routing_explanation);
// ── Quota Management: Find free alternatives ──
const quotaResult = await a2aCall("message/send", {
skill: "quota-management",
messages: [{ role: "user", content: "Suggest free combos for documentation" }],
});
console.log("Free combos:", quotaResult.artifacts[0].content);
// ── Streaming: Real-time response ──
const streamResp = await fetch(`${BASE_URL}/a2a`, {
method: "POST",
headers: {
"Content-Type": "application/json",
Authorization: `Bearer ${API_KEY}`,
},
body: JSON.stringify({
jsonrpc: "2.0",
id: "stream-1",
method: "message/stream",
params: {
skill: "smart-routing",
messages: [{ role: "user", content: "Explain microservices architecture" }],
},
}),
});
const reader = streamResp.body!.getReader();
const decoder = new TextDecoder();
while (true) {
const { done, value } = await reader.read();
if (done) break;
const chunk = decoder.decode(value);
for (const line of chunk.split("\n")) {
if (line.startsWith("data: ")) {
const event = JSON.parse(line.slice(6));
if (event.params.chunk) {
process.stdout.write(event.params.chunk.content);
}
if (event.params.task.state === "completed") {
console.log("\n✅ Stream completed");
}
}
}
}
```
### Python — LangChain A2A Integration
```python
"""
LangChain integration — Use OmniRoute A2A as a custom LLM.
"""
from langchain.llms.base import BaseLLM
from langchain.schema import LLMResult, Generation
import requests
from typing import List, Optional
class OmniRouteA2A(BaseLLM):
base_url: str = "http://localhost:20128"
api_key: str = ""
model: str = "auto"
combo: Optional[str] = None
@property
def _llm_type(self) -> str:
return "omniroute-a2a"
def _call(self, prompt: str, stop: Optional[List[str]] = None, **kwargs) -> str:
response = requests.post(
f"{self.base_url}/a2a",
headers={
"Content-Type": "application/json",
"Authorization": f"Bearer {self.api_key}",
},
json={
"jsonrpc": "2.0",
"id": "langchain-1",
"method": "message/send",
"params": {
"skill": "smart-routing",
"messages": [{"role": "user", "content": prompt}],
"metadata": {
"model": self.model,
**({"combo": self.combo} if self.combo else {}),
},
},
},
)
result = response.json()["result"]
return result["artifacts"][0]["content"]
def _generate(self, prompts: List[str], stop=None, **kwargs) -> LLMResult:
return LLMResult(
generations=[[Generation(text=self._call(p, stop))] for p in prompts]
)
# Usage
llm = OmniRouteA2A(
base_url="http://localhost:20128",
api_key="your-key",
model="auto",
combo="fast-coding",
)
result = llm("Write a Python function to merge two sorted lists")
print(result)
```
### Go — A2A Client
```go
package main
import (
"bytes"
"encoding/json"
"fmt"
"io"
"net/http"
)
const baseURL = "http://localhost:20128"
const apiKey = "your-api-key"
type JsonRpcRequest struct {
Jsonrpc string `json:"jsonrpc"`
ID string `json:"id"`
Method string `json:"method"`
Params interface{} `json:"params"`
}
type JsonRpcResponse struct {
Jsonrpc string `json:"jsonrpc"`
ID string `json:"id"`
Result interface{} `json:"result"`
Error *struct {
Code int `json:"code"`
Message string `json:"message"`
} `json:"error"`
}
func a2aCall(method string, params interface{}) (*JsonRpcResponse, error) {
body, _ := json.Marshal(JsonRpcRequest{
Jsonrpc: "2.0",
ID: "go-1",
Method: method,
Params: params,
})
req, _ := http.NewRequest("POST", baseURL+"/a2a", bytes.NewReader(body))
req.Header.Set("Content-Type", "application/json")
req.Header.Set("Authorization", "Bearer "+apiKey)
resp, err := http.DefaultClient.Do(req)
if err != nil {
return nil, err
}
defer resp.Body.Close()
data, _ := io.ReadAll(resp.Body)
var result JsonRpcResponse
json.Unmarshal(data, &result)
return &result, nil
}
func main() {
// Discover agent
resp, _ := http.Get(baseURL + "/.well-known/agent.json")
defer resp.Body.Close()
body, _ := io.ReadAll(resp.Body)
fmt.Println("Agent Card:", string(body))
// Send smart-routing task
result, _ := a2aCall("message/send", map[string]interface{}{
"skill": "smart-routing",
"messages": []map[string]string{{"role": "user", "content": "Hello from Go!"}},
"metadata": map[string]interface{}{"model": "auto"},
})
out, _ := json.MarshalIndent(result.Result, "", " ")
fmt.Println("Result:", string(out))
}
```
---
## Use Cases
### 🤖 Use Case 1: Multi-Agent Coding Pipeline
An orchestrator agent delegates code generation to OmniRoute, then passes the output to a review agent.
```python
def coding_pipeline(task: str):
# Step 1: Generate code via OmniRoute A2A
code_result = a2a_send("smart-routing", [
{"role": "user", "content": f"Write production-quality code: {task}"}
], metadata={"model": "auto", "role": "coding"})
code = code_result["artifacts"][0]["content"]
# Step 2: Review the code via OmniRoute A2A (different model)
review_result = a2a_send("smart-routing", [
{"role": "user", "content": f"Review this code for bugs and improvements:\n\n{code}"}
], metadata={"model": "auto", "role": "review"})
review = review_result["artifacts"][0]["content"]
# Step 3: Check costs
print(f"Code cost: ${code_result['metadata']['cost_envelope']['actual']}")
print(f"Review cost: ${review_result['metadata']['cost_envelope']['actual']}")
return {"code": code, "review": review}
```
### 💡 Use Case 2: Quota-Aware Agent Swarm
Multiple agents share quota through OmniRoute, using the quota skill to coordinate.
```python
async def quota_aware_agent(agent_name: str, task: str):
# Check quota before starting
quota = a2a_send("quota-management", [
{"role": "user", "content": "Which provider has the most quota remaining?"}
])
print(f"[{agent_name}] {quota['artifacts'][0]['content']}")
# Send request with budget constraint
result = a2a_send("smart-routing", [
{"role": "user", "content": task}
], metadata={"budget": 0.05})
policy = result["metadata"]["policy_verdict"]
if not policy["allowed"]:
print(f"[{agent_name}] ⚠️ Budget exceeded: {policy['reason']}")
# Fall back to free combo
quota = a2a_send("quota-management", [
{"role": "user", "content": "Suggest free combos"}
])
print(f"[{agent_name}] Free alternatives: {quota['artifacts'][0]['content']}")
return result
```
### 📊 Use Case 3: Real-Time Streaming Dashboard
A monitoring agent streams responses and displays progress in real-time.
```typescript
async function streamingDashboard(prompt: string) {
const response = await fetch(`${BASE_URL}/a2a`, {
method: "POST",
headers: { "Content-Type": "application/json", Authorization: `Bearer ${API_KEY}` },
body: JSON.stringify({
jsonrpc: "2.0",
id: "dash-1",
method: "message/stream",
params: { skill: "smart-routing", messages: [{ role: "user", content: prompt }] },
}),
});
let totalChunks = 0;
const reader = response.body!.getReader();
const decoder = new TextDecoder();
while (true) {
const { done, value } = await reader.read();
if (done) break;
for (const line of decoder.decode(value).split("\n")) {
if (line.startsWith("data: ")) {
const event = JSON.parse(line.slice(6));
const state = event.params.task.state;
if (state === "working" && event.params.chunk) {
totalChunks++;
process.stdout.write(
`\r[Chunk ${totalChunks}] ${event.params.chunk.content.slice(0, 50)}...`
);
}
if (state === "completed") {
const meta = event.params.metadata;
console.log(
`\n✅ Done | Cost: $${meta?.cost_envelope?.actual || 0} | Route: ${meta?.routing_explanation || "N/A"}`
);
}
if (state === "failed") {
console.error(`\n❌ Failed: ${event.params.metadata?.error}`);
}
}
}
}
}
```
### 🔁 Use Case 4: Task Polling Pattern
For long-running tasks, poll the task status instead of waiting synchronously.
```python
import time
def poll_task(task_id: str, timeout: int = 60):
"""Poll task status until completion or timeout."""
start = time.time()
while time.time() - start < timeout:
result = requests.post(f"{BASE_URL}/a2a", headers=HEADERS, json={
"jsonrpc": "2.0",
"id": "poll-1",
"method": "tasks/get",
"params": {"taskId": task_id},
}).json()
task = result["result"]["task"]
state = task["state"]
print(f" Task {task_id[:8]}... state={state}")
if state in ("completed", "failed", "cancelled"):
return task
time.sleep(2)
# Timeout — cancel the task
requests.post(f"{BASE_URL}/a2a", headers=HEADERS, json={
"jsonrpc": "2.0",
"id": "cancel-1",
"method": "tasks/cancel",
"params": {"taskId": task_id},
})
raise TimeoutError(f"Task {task_id} timed out after {timeout}s")
```
---
## Error Codes
| Code | Constant | Meaning |
| ------ | ------------------------ | ---------------------------------------- |
| -32700 | — | Parse error (invalid JSON) |
| -32600 | `INVALID_REQUEST` | Invalid JSON-RPC request or unauthorized |
| -32601 | `METHOD_NOT_FOUND` | Unknown method or skill |
| -32602 | `INVALID_PARAMS` | Missing or invalid parameters |
| -32603 | `INTERNAL_ERROR` | Skill execution failed |
| -32001 | `TASK_NOT_FOUND` | Task ID not found |
| -32002 | `TASK_ALREADY_COMPLETED` | Cannot modify a completed task |
| -32003 | `UNAUTHORIZED` | Invalid or missing API key |
| -32004 | `BUDGET_EXCEEDED` | Request exceeds configured budget |
| -32005 | `PROVIDER_UNAVAILABLE` | No available providers |
---
## Authentication
All `/a2a` requests require a Bearer token via the `Authorization` header:
```
Authorization: Bearer YOUR_OMNIROUTE_API_KEY
```
If no API key is configured on the server (`OMNIROUTE_API_KEY` is empty), authentication is bypassed.
---
## File Structure
```
src/lib/a2a/
├── taskManager.ts # Task lifecycle (create/update/cancel/list), TTL, cleanup
├── taskExecution.ts # Generic task executor with state management
├── streaming.ts # SSE stream formatting, heartbeat, chunk/completion events
├── routingLogger.ts # Routing decision logger (stats, history, retention)
└── skills/
├── smartRouting.ts # Smart routing skill (routes via /v1/chat/completions)
└── quotaManagement.ts # Quota management skill (natural-language quota queries)
src/app/a2a/
└── route.ts # Next.js API route handler (JSON-RPC 2.0 dispatch)
open-sse/mcp-server/
└── schemas/a2a.ts # Zod schemas (AgentCard, Task, JSON-RPC, SSE events)
```
---
## Comparison: MCP vs A2A
| Feature | MCP Server | A2A Server |
| ----------------- | ---------------------------- | ------------------------------------------------- |
| **Protocol** | Model Context Protocol | Agent-to-Agent Protocol v0.3 |
| **Transport** | stdio / HTTP | HTTP (JSON-RPC 2.0) |
| **Discovery** | Tool listing via MCP | `/.well-known/agent.json` |
| **Granularity** | 16 individual tools | 2 high-level skills |
| **Best for** | IDE agents (Cursor, VS Code) | Multi-agent systems (LangChain, CrewAI) |
| **Streaming** | Not supported | SSE via `message/stream` |
| **Task tracking** | No | Full lifecycle (submitted → completed) |
| **Observability** | Audit log per tool call | Cost envelope + resilience trace + policy verdict |
---
## License
Part of [OmniRoute](https://github.com/diegosouzapw/OmniRoute) — MIT License.

View File

@@ -124,6 +124,11 @@ export function createA2AStream(
controller.enqueue(encoder.encode(createChunkEvent(task.id, artifact.content)));
}
if (abortSignal?.aborted) {
controller.enqueue(encoder.encode(createFailureEvent(task.id, "Cancelled")));
return;
}
// Emit completion with metadata
controller.enqueue(encoder.encode(createCompletionEvent(task.id, result.metadata)));
} catch (err) {

View File

@@ -13,6 +13,7 @@ type StreamTaskLike = {
type StreamTaskResult = {
artifacts: Array<{ type: string; content: string }>;
metadata: Record<string, unknown>;
};
export async function executeA2ATaskWithState(

View File

@@ -50,7 +50,7 @@ export interface A2ATask {
// ============ Valid Transitions ============
const VALID_TRANSITIONS: Record<TaskState, TaskState[]> = {
submitted: ["working", "cancelled"],
submitted: ["working", "failed", "cancelled"],
working: ["completed", "failed", "cancelled"],
completed: [],
failed: [],
@@ -136,8 +136,14 @@ export class A2ATaskManager {
private cleanupExpired() {
const now = new Date();
for (const [id, task] of this.tasks) {
if (new Date(task.expiresAt) < now && task.state !== "completed" && task.state !== "failed") {
if (
new Date(task.expiresAt) < now &&
task.state !== "completed" &&
task.state !== "failed" &&
task.state !== "cancelled"
) {
task.state = "failed";
task.updatedAt = now.toISOString();
task.events.push({ timestamp: now.toISOString(), state: "failed", message: "TTL expired" });
}
// Remove terminal tasks older than 2x TTL

View File

@@ -9,6 +9,24 @@ import {
const CLOUD_URL = process.env.CLOUD_URL || process.env.NEXT_PUBLIC_CLOUD_URL;
const CLOUD_SYNC_TIMEOUT_MS = Number(process.env.CLOUD_SYNC_TIMEOUT_MS || 12000);
type JsonRecord = Record<string, unknown>;
function asRecord(value: unknown): JsonRecord {
return value && typeof value === "object" && !Array.isArray(value) ? (value as JsonRecord) : {};
}
function toStringOrNull(value: unknown): string | null {
return typeof value === "string" && value.trim().length > 0 ? value : null;
}
function toDateMs(value: unknown): number {
if (typeof value === "string" || typeof value === "number" || value instanceof Date) {
const parsed = new Date(value).getTime();
return Number.isFinite(parsed) ? parsed : 0;
}
return 0;
}
export async function fetchWithTimeout(url, options = {}, timeoutMs = CLOUD_SYNC_TIMEOUT_MS) {
const controller = new AbortController();
const timeoutId = setTimeout(() => controller.abort(), timeoutMs);
@@ -85,15 +103,20 @@ export async function syncToCloud(machineId, createdKey = null) {
* Simple logic: if Cloud is newer, sync entire provider
* cloudProviders is object keyed by provider ID
*/
async function updateLocalTokens(cloudProviders) {
async function updateLocalTokens(cloudProviders: unknown) {
const cloudProvidersMap = asRecord(cloudProviders);
const localProviders = await getProviderConnections();
for (const localProvider of localProviders) {
const cloudProvider = cloudProviders[localProvider.id];
if (!cloudProvider) continue;
for (const localProviderRaw of localProviders as unknown[]) {
const localProvider = asRecord(localProviderRaw);
const localProviderId = toStringOrNull(localProvider.id);
if (!localProviderId) continue;
const cloudUpdatedAt = new Date(cloudProvider.updatedAt || 0).getTime();
const localUpdatedAt = new Date(localProvider.updatedAt || 0).getTime();
const cloudProvider = asRecord(cloudProvidersMap[localProviderId]);
if (Object.keys(cloudProvider).length === 0) continue;
const cloudUpdatedAt = toDateMs(cloudProvider.updatedAt);
const localUpdatedAt = toDateMs(localProvider.updatedAt);
// Simple logic: if Cloud is newer, sync entire provider
if (cloudUpdatedAt > localUpdatedAt) {
@@ -119,7 +142,7 @@ async function updateLocalTokens(cloudProviders) {
updatedAt: cloudProvider.updatedAt,
};
await updateProviderConnection(localProvider.id, updates);
await updateProviderConnection(localProviderId, updates);
}
}
}

View File

@@ -6,24 +6,39 @@ import { v4 as uuidv4 } from "uuid";
import { getDbInstance } from "./core";
import { backupDbFile } from "./backup";
type JsonRecord = Record<string, unknown>;
function asRecord(value: unknown): JsonRecord {
return value && typeof value === "object" && !Array.isArray(value) ? (value as JsonRecord) : {};
}
function getSerializedData(value: unknown): string | null {
const row = asRecord(value);
return typeof row.data === "string" ? row.data : null;
}
export async function getCombos() {
const db = getDbInstance();
return db
.prepare("SELECT data FROM combos ORDER BY name")
.all()
.map((r) => JSON.parse(r.data));
.map((row) => getSerializedData(row))
.filter((row): row is string => row !== null)
.map((row) => JSON.parse(row));
}
export async function getComboById(id) {
const db = getDbInstance();
const row = db.prepare("SELECT data FROM combos WHERE id = ?").get(id);
return row ? JSON.parse(row.data) : null;
const payload = getSerializedData(row);
return payload ? JSON.parse(payload) : null;
}
export async function getComboByName(name) {
const db = getDbInstance();
const row = db.prepare("SELECT data FROM combos WHERE name = ?").get(name);
return row ? JSON.parse(row.data) : null;
const payload = getSerializedData(row);
return payload ? JSON.parse(payload) : null;
}
export async function createCombo(data) {
@@ -53,7 +68,9 @@ export async function updateCombo(id, data) {
const existing = db.prepare("SELECT data FROM combos WHERE id = ?").get(id);
if (!existing) return null;
const current = JSON.parse(existing.data);
const serializedCurrent = getSerializedData(existing);
if (!serializedCurrent) return null;
const current = JSON.parse(serializedCurrent);
const merged = { ...current, ...data, updatedAt: new Date().toISOString() };
db.prepare("UPDATE combos SET name = ?, data = ?, updated_at = ? WHERE id = ?").run(

View File

@@ -12,6 +12,21 @@
import { getDbInstance, isBuildPhase, isCloud } from "./core";
type JsonRecord = Record<string, unknown>;
function asRecord(value: unknown): JsonRecord {
return value && typeof value === "object" && !Array.isArray(value) ? (value as JsonRecord) : {};
}
function toNumber(value: unknown, fallback = 0): number {
if (typeof value === "number" && Number.isFinite(value)) return value;
if (typeof value === "string" && value.trim().length > 0) {
const parsed = Number(value);
return Number.isFinite(parsed) ? parsed : fallback;
}
return fallback;
}
// ──────────────── Fallback Chains ────────────────
/**
@@ -35,7 +50,8 @@ export function saveFallbackChain(model, chain) {
export function loadFallbackChain(model) {
const db = getDbInstance();
const row = db.prepare("SELECT chain FROM domain_fallback_chains WHERE model = ?").get(model);
return row ? JSON.parse(row.chain) : null;
const chain = asRecord(row).chain;
return typeof chain === "string" ? JSON.parse(chain) : null;
}
/**
@@ -45,9 +61,13 @@ export function loadFallbackChain(model) {
export function loadAllFallbackChains() {
const db = getDbInstance();
const rows = db.prepare("SELECT model, chain FROM domain_fallback_chains").all();
const result = {};
const result: Record<string, unknown> = {};
for (const row of rows) {
result[row.model] = JSON.parse(row.chain);
const record = asRecord(row);
const model = typeof record.model === "string" ? record.model : null;
const chain = typeof record.chain === "string" ? record.chain : null;
if (!model || !chain) continue;
result[model] = JSON.parse(chain);
}
return result;
}
@@ -99,11 +119,12 @@ export function saveBudget(apiKeyId, config) {
export function loadBudget(apiKeyId) {
const db = getDbInstance();
const row = db.prepare("SELECT * FROM domain_budgets WHERE api_key_id = ?").get(apiKeyId);
const record = asRecord(row);
if (!row) return null;
return {
dailyLimitUsd: row.daily_limit_usd,
monthlyLimitUsd: row.monthly_limit_usd,
warningThreshold: row.warning_threshold,
dailyLimitUsd: toNumber(record.daily_limit_usd),
monthlyLimitUsd: toNumber(record.monthly_limit_usd),
warningThreshold: toNumber(record.warning_threshold, 0.8),
};
}
@@ -203,9 +224,12 @@ export function loadLockoutState(identifier) {
const db = getDbInstance();
const row = db.prepare("SELECT * FROM domain_lockout_state WHERE identifier = ?").get(identifier);
if (!row) return null;
const record = asRecord(row);
const attemptsRaw = typeof record.attempts === "string" ? record.attempts : "[]";
const lockedUntilRaw = record.locked_until;
return {
attempts: JSON.parse(row.attempts),
lockedUntil: row.locked_until,
attempts: JSON.parse(attemptsRaw),
lockedUntil: typeof lockedUntilRaw === "number" ? lockedUntilRaw : null,
};
}
@@ -230,10 +254,14 @@ export function loadAllLockedIdentifiers() {
"SELECT identifier, locked_until FROM domain_lockout_state WHERE locked_until IS NOT NULL AND locked_until > ?"
)
.all(now)
.map((row) => ({
identifier: row.identifier,
lockedUntil: row.locked_until,
}));
.map((row) => {
const record = asRecord(row);
return {
identifier: typeof record.identifier === "string" ? record.identifier : "",
lockedUntil: toNumber(record.locked_until),
};
})
.filter((row) => row.identifier.length > 0);
}
// ──────────────── Circuit Breakers ────────────────
@@ -266,11 +294,13 @@ export function loadCircuitBreakerState(name) {
const db = getDbInstance();
const row = db.prepare("SELECT * FROM domain_circuit_breakers WHERE name = ?").get(name);
if (!row) return null;
const record = asRecord(row);
const options = typeof record.options === "string" ? JSON.parse(record.options) : null;
return {
state: row.state,
failureCount: row.failure_count,
lastFailureTime: row.last_failure_time,
options: row.options ? JSON.parse(row.options) : null,
state: typeof record.state === "string" ? record.state : "CLOSED",
failureCount: toNumber(record.failure_count),
lastFailureTime: toNumber(record.last_failure_time, 0) || null,
options,
};
}
@@ -283,12 +313,16 @@ export function loadAllCircuitBreakerStates() {
return db
.prepare("SELECT name, state, failure_count, last_failure_time FROM domain_circuit_breakers")
.all()
.map((row) => ({
name: row.name,
state: row.state,
failureCount: row.failure_count,
lastFailureTime: row.last_failure_time,
}));
.map((row) => {
const record = asRecord(row);
return {
name: typeof record.name === "string" ? record.name : "",
state: typeof record.state === "string" ? record.state : "CLOSED",
failureCount: toNumber(record.failure_count),
lastFailureTime: toNumber(record.last_failure_time, 0) || null,
};
})
.filter((row) => row.name.length > 0);
}
/**

View File

@@ -5,6 +5,20 @@
import { getDbInstance } from "./core";
import { backupDbFile } from "./backup";
type JsonRecord = Record<string, unknown>;
function asRecord(value: unknown): JsonRecord {
return value && typeof value === "object" && !Array.isArray(value) ? (value as JsonRecord) : {};
}
function getKeyValue(row: unknown): { key: string | null; value: string | null } {
const record = asRecord(row);
return {
key: typeof record.key === "string" ? record.key : null,
value: typeof record.value === "string" ? record.value : null,
};
}
// ──────────────── Model Aliases ────────────────
export async function getModelAliases() {
@@ -12,9 +26,11 @@ export async function getModelAliases() {
const rows = db
.prepare("SELECT key, value FROM key_value WHERE namespace = 'modelAliases'")
.all();
const result = {};
const result: Record<string, unknown> = {};
for (const row of rows) {
result[row.key] = JSON.parse(row.value);
const { key, value } = getKeyValue(row);
if (!key || value === null) continue;
result[key] = JSON.parse(value);
}
return result;
}
@@ -41,12 +57,15 @@ export async function getMitmAlias(toolName) {
const row = db
.prepare("SELECT value FROM key_value WHERE namespace = 'mitmAlias' AND key = ?")
.get(toolName);
return row ? JSON.parse(row.value) : {};
const value = getKeyValue(row).value;
return value ? JSON.parse(value) : {};
}
const rows = db.prepare("SELECT key, value FROM key_value WHERE namespace = 'mitmAlias'").all();
const result = {};
const result: Record<string, unknown> = {};
for (const row of rows) {
result[row.key] = JSON.parse(row.value);
const { key, value } = getKeyValue(row);
if (!key || value === null) continue;
result[key] = JSON.parse(value);
}
return result;
}
@@ -67,13 +86,18 @@ export async function getCustomModels(providerId) {
const row = db
.prepare("SELECT value FROM key_value WHERE namespace = 'customModels' AND key = ?")
.get(providerId);
return row ? JSON.parse(row.value) : [];
const value = getKeyValue(row).value;
return value ? JSON.parse(value) : [];
}
const rows = db
.prepare("SELECT key, value FROM key_value WHERE namespace = 'customModels'")
.all();
const result = {};
for (const row of rows) result[row.key] = JSON.parse(row.value);
const result: Record<string, unknown> = {};
for (const row of rows) {
const { key, value } = getKeyValue(row);
if (!key || value === null) continue;
result[key] = JSON.parse(value);
}
return result;
}
@@ -82,8 +106,12 @@ export async function getAllCustomModels() {
const rows = db
.prepare("SELECT key, value FROM key_value WHERE namespace = 'customModels'")
.all();
const result = {};
for (const row of rows) result[row.key] = JSON.parse(row.value);
const result: Record<string, unknown> = {};
for (const row of rows) {
const { key, value } = getKeyValue(row);
if (!key || value === null) continue;
result[key] = JSON.parse(value);
}
return result;
}
@@ -92,7 +120,8 @@ export async function addCustomModel(providerId, modelId, modelName, source = "m
const row = db
.prepare("SELECT value FROM key_value WHERE namespace = 'customModels' AND key = ?")
.get(providerId);
const models = row ? JSON.parse(row.value) : [];
const value = getKeyValue(row).value;
const models = value ? JSON.parse(value) : [];
const exists = models.find((m) => m.id === modelId);
if (exists) return exists;
@@ -113,7 +142,9 @@ export async function removeCustomModel(providerId, modelId) {
.get(providerId);
if (!row) return false;
const models = JSON.parse(row.value);
const value = getKeyValue(row).value;
if (!value) return false;
const models = JSON.parse(value);
const before = models.length;
const filtered = models.filter((m) => m.id !== modelId);

View File

@@ -29,7 +29,7 @@ function toProxyMap(value: unknown): ProxyMap {
}
function toProxyValue(value: unknown): ProxyValue {
if (value === null || typeof value === "string") return value;
if (value === null || typeof value === "string") return value as string | null;
if (value && typeof value === "object") return value as JsonRecord;
return null;
}

View File

@@ -14,6 +14,21 @@ import crypto from "node:crypto";
import { LRUCache } from "./cacheLayer";
import { getDbInstance } from "./db/core";
type JsonRecord = Record<string, unknown>;
function asRecord(value: unknown): JsonRecord {
return value && typeof value === "object" && !Array.isArray(value) ? (value as JsonRecord) : {};
}
function toNumber(value: unknown, fallback = 0): number {
if (typeof value === "number" && Number.isFinite(value)) return value;
if (typeof value === "string" && value.trim().length > 0) {
const parsed = Number(value);
return Number.isFinite(parsed) ? parsed : fallback;
}
return fallback;
}
// ─── Singleton ─────────────────
let memoryCache: LRUCache | null = null;
@@ -89,11 +104,18 @@ export function getCachedResponse(signature) {
.get(signature);
if (row) {
const parsed = JSON.parse(row.response);
const record = asRecord(row);
const responsePayload = typeof record.response === "string" ? record.response : null;
if (!responsePayload) {
stats.misses++;
return null;
}
const parsed = JSON.parse(responsePayload);
const tokensSaved = toNumber(record.tokens_saved, 0);
// Promote to memory cache
getMemoryCache().set(signature, {
response: parsed,
tokensSaved: row.tokens_saved,
tokensSaved,
});
// Update hit count in DB
db.prepare("UPDATE semantic_cache SET hit_count = hit_count + 1 WHERE signature = ?").run(
@@ -101,7 +123,7 @@ export function getCachedResponse(signature) {
);
stats.hits++;
stats.tokensSaved += row.tokens_saved || 0;
stats.tokensSaved += tokensSaved;
return parsed;
}
} catch {
@@ -171,9 +193,7 @@ export function invalidateByModel(model: string): number {
getMemoryCache().clear(); // Memory cache doesn't track model; full clear
try {
const db = getDbInstance();
const result = db
.prepare("DELETE FROM semantic_cache WHERE model = ?")
.run(model);
const result = db.prepare("DELETE FROM semantic_cache WHERE model = ?").run(model);
return result.changes || 0;
} catch {
return 0;
@@ -189,9 +209,7 @@ export function invalidateBySignature(signature: string): boolean {
getMemoryCache().delete(signature);
try {
const db = getDbInstance();
const result = db
.prepare("DELETE FROM semantic_cache WHERE signature = ?")
.run(signature);
const result = db.prepare("DELETE FROM semantic_cache WHERE signature = ?").run(signature);
return (result.changes || 0) > 0;
} catch {
return false;
@@ -208,9 +226,7 @@ export function invalidateStale(maxAgeMs: number): number {
try {
const db = getDbInstance();
const cutoff = new Date(Date.now() - maxAgeMs).toISOString();
const result = db
.prepare("DELETE FROM semantic_cache WHERE created_at < ?")
.run(cutoff);
const result = db.prepare("DELETE FROM semantic_cache WHERE created_at < ?").run(cutoff);
return result.changes || 0;
} catch {
return 0;
@@ -272,7 +288,7 @@ export function getCacheStats() {
const row = db
.prepare("SELECT COUNT(*) as count FROM semantic_cache WHERE expires_at > datetime('now')")
.get();
dbSize = row?.count || 0;
dbSize = toNumber(asRecord(row).count, 0);
} catch {
// DB not available
}

View File

@@ -14,6 +14,39 @@ import { shouldPersistToDisk, CALL_LOGS_DIR } from "./migrations";
import { isNoLog } from "../compliance";
import { sanitizePII } from "../piiSanitizer";
type JsonRecord = Record<string, unknown>;
function asRecord(value: unknown): JsonRecord {
return value && typeof value === "object" && !Array.isArray(value) ? (value as JsonRecord) : {};
}
function toNumber(value: unknown): number {
if (typeof value === "number" && Number.isFinite(value)) return value;
if (typeof value === "string" && value.trim().length > 0) {
const parsed = Number(value);
return Number.isFinite(parsed) ? parsed : 0;
}
return 0;
}
function toStringOrNull(value: unknown): string | null {
return typeof value === "string" ? value : null;
}
function parseJsonString(value: unknown): unknown | null {
if (typeof value !== "string" || value.trim().length === 0) return null;
try {
return JSON.parse(value);
} catch {
return null;
}
}
function hasTruncatedFlag(value: unknown): boolean {
if (!value || typeof value !== "object" || Array.isArray(value)) return false;
return (value as Record<string, unknown>)._truncated === true;
}
const CALL_LOGS_MAX = parseInt(process.env.CALL_LOGS_MAX || "200", 10);
const LOG_RETENTION_DAYS = parseInt(process.env.LOG_RETENTION_DAYS || "7", 10);
const CALL_LOG_PAYLOAD_MODE = (() => {
@@ -177,7 +210,8 @@ export async function saveCallLog(entry: any) {
).run(logEntry);
// 2. Trim old entries beyond CALL_LOGS_MAX
const count = db.prepare("SELECT COUNT(*) as cnt FROM call_logs").get()?.cnt || 0;
const countRow = asRecord(db.prepare("SELECT COUNT(*) as cnt FROM call_logs").get());
const count = toNumber(countRow.cnt);
if (count > CALL_LOGS_MAX) {
db.prepare(
`
@@ -326,26 +360,29 @@ export async function getCallLogs(filter: any = {}) {
const rows = db.prepare(sql).all(params);
return rows.map((l) => ({
id: l.id,
timestamp: l.timestamp,
method: l.method,
path: l.path,
status: l.status,
model: l.model,
provider: l.provider,
account: l.account,
duration: l.duration,
tokens: { in: l.tokens_in, out: l.tokens_out },
sourceFormat: l.source_format,
targetFormat: l.target_format,
error: l.error,
comboName: l.combo_name || null,
apiKeyId: l.api_key_id || null,
apiKeyName: l.api_key_name || null,
hasRequestBody: !!l.request_body,
hasResponseBody: !!l.response_body,
}));
return rows.map((row) => {
const l = asRecord(row);
return {
id: toStringOrNull(l.id),
timestamp: toStringOrNull(l.timestamp),
method: toStringOrNull(l.method),
path: toStringOrNull(l.path),
status: toNumber(l.status),
model: toStringOrNull(l.model),
provider: toStringOrNull(l.provider),
account: toStringOrNull(l.account),
duration: toNumber(l.duration),
tokens: { in: toNumber(l.tokens_in), out: toNumber(l.tokens_out) },
sourceFormat: toStringOrNull(l.source_format),
targetFormat: toStringOrNull(l.target_format),
error: toStringOrNull(l.error),
comboName: toStringOrNull(l.combo_name),
apiKeyId: toStringOrNull(l.api_key_id),
apiKeyName: toStringOrNull(l.api_key_name),
hasRequestBody: typeof l.request_body === "string" && l.request_body.length > 0,
hasResponseBody: typeof l.response_body === "string" && l.response_body.length > 0,
};
});
}
/**
@@ -355,31 +392,32 @@ export async function getCallLogById(id: string) {
const db = getDbInstance();
const row = db.prepare("SELECT * FROM call_logs WHERE id = ?").get(id);
if (!row) return null;
const entryRow = asRecord(row);
const entry = {
id: row.id,
timestamp: row.timestamp,
method: row.method,
path: row.path,
status: row.status,
model: row.model,
provider: row.provider,
account: row.account,
connectionId: row.connection_id,
duration: row.duration,
tokens: { in: row.tokens_in, out: row.tokens_out },
sourceFormat: row.source_format,
targetFormat: row.target_format,
apiKeyId: row.api_key_id,
apiKeyName: row.api_key_name,
comboName: row.combo_name,
requestBody: row.request_body ? JSON.parse(row.request_body) : null,
responseBody: row.response_body ? JSON.parse(row.response_body) : null,
error: row.error,
id: toStringOrNull(entryRow.id),
timestamp: toStringOrNull(entryRow.timestamp),
method: toStringOrNull(entryRow.method),
path: toStringOrNull(entryRow.path),
status: toNumber(entryRow.status),
model: toStringOrNull(entryRow.model),
provider: toStringOrNull(entryRow.provider),
account: toStringOrNull(entryRow.account),
connectionId: toStringOrNull(entryRow.connection_id),
duration: toNumber(entryRow.duration),
tokens: { in: toNumber(entryRow.tokens_in), out: toNumber(entryRow.tokens_out) },
sourceFormat: toStringOrNull(entryRow.source_format),
targetFormat: toStringOrNull(entryRow.target_format),
apiKeyId: toStringOrNull(entryRow.api_key_id),
apiKeyName: toStringOrNull(entryRow.api_key_name),
comboName: toStringOrNull(entryRow.combo_name),
requestBody: parseJsonString(entryRow.request_body),
responseBody: parseJsonString(entryRow.response_body),
error: toStringOrNull(entryRow.error),
};
// If payloads were truncated, try to read full version from disk
const needsDisk = entry.requestBody?._truncated || entry.responseBody?._truncated;
const needsDisk = hasTruncatedFlag(entry.requestBody) || hasTruncatedFlag(entry.responseBody);
if (needsDisk && CALL_LOGS_DIR) {
try {
const diskEntry = readFullLogFromDisk(entry);

View File

@@ -24,6 +24,15 @@ function normalizeModelName(model) {
return parts[parts.length - 1];
}
function toNumber(value: unknown, fallback = 0): number {
if (typeof value === "number" && Number.isFinite(value)) return value;
if (typeof value === "string" && value.trim().length > 0) {
const parsed = Number(value);
return Number.isFinite(parsed) ? parsed : fallback;
}
return fallback;
}
/**
* Calculate cost for a usage entry.
*
@@ -48,29 +57,39 @@ export async function calculateCost(provider, model, tokens) {
}
if (!pricing) return 0;
const pricingRecord =
pricing && typeof pricing === "object" && !Array.isArray(pricing)
? (pricing as Record<string, unknown>)
: {};
const inputPrice = toNumber(pricingRecord.input, 0);
const cachedPrice = toNumber(pricingRecord.cached, inputPrice);
const outputPrice = toNumber(pricingRecord.output, 0);
const reasoningPrice = toNumber(pricingRecord.reasoning, outputPrice);
const cacheCreationPrice = toNumber(pricingRecord.cache_creation, inputPrice);
let cost = 0;
const inputTokens = tokens.input ?? tokens.prompt_tokens ?? tokens.input_tokens ?? 0;
const cachedTokens =
tokens.cacheRead ?? tokens.cached_tokens ?? tokens.cache_read_input_tokens ?? 0;
const nonCachedInput = Math.max(0, inputTokens - cachedTokens);
cost += nonCachedInput * (pricing.input / 1000000);
cost += nonCachedInput * (inputPrice / 1000000);
if (cachedTokens > 0) {
cost += cachedTokens * ((pricing.cached || pricing.input) / 1000000);
cost += cachedTokens * (cachedPrice / 1000000);
}
const outputTokens = tokens.output ?? tokens.completion_tokens ?? tokens.output_tokens ?? 0;
cost += outputTokens * (pricing.output / 1000000);
cost += outputTokens * (outputPrice / 1000000);
const reasoningTokens = tokens.reasoning ?? tokens.reasoning_tokens ?? 0;
if (reasoningTokens > 0) {
cost += reasoningTokens * ((pricing.reasoning || pricing.output) / 1000000);
cost += reasoningTokens * (reasoningPrice / 1000000);
}
const cacheCreationTokens = tokens.cacheCreation ?? tokens.cache_creation_input_tokens ?? 0;
if (cacheCreationTokens > 0) {
cost += cacheCreationTokens * ((pricing.cache_creation || pricing.input) / 1000000);
cost += cacheCreationTokens * (cacheCreationPrice / 1000000);
}
return cost;

View File

@@ -10,6 +10,25 @@
import { getDbInstance } from "../db/core";
import { shouldPersistToDisk } from "./migrations";
type JsonRecord = Record<string, unknown>;
function asRecord(value: unknown): JsonRecord {
return value && typeof value === "object" && !Array.isArray(value) ? (value as JsonRecord) : {};
}
function toStringOrNull(value: unknown): string | null {
return typeof value === "string" && value.trim().length > 0 ? value : null;
}
function toNumber(value: unknown): number {
if (typeof value === "number" && Number.isFinite(value)) return value;
if (typeof value === "string" && value.trim().length > 0) {
const parsed = Number(value);
return Number.isFinite(parsed) ? parsed : 0;
}
return 0;
}
// ──────────────── Pending Requests (in-memory) ────────────────
const pendingRequests: {
@@ -23,7 +42,12 @@ const pendingRequests: {
/**
* Track a pending request.
*/
export function trackPendingRequest(model: string, provider: string, connectionId: string | null, started: boolean) {
export function trackPendingRequest(
model: string,
provider: string,
connectionId: string | null,
started: boolean
) {
const modelKey = provider ? `${model} (${provider})` : model;
if (!pendingRequests.byModel[modelKey]) pendingRequests.byModel[modelKey] = 0;
@@ -61,22 +85,25 @@ export async function getUsageDb() {
const db = getDbInstance();
const rows = db.prepare("SELECT * FROM usage_history ORDER BY timestamp ASC").all();
const history = rows.map((r) => ({
provider: r.provider,
model: r.model,
connectionId: r.connection_id,
apiKeyId: r.api_key_id,
apiKeyName: r.api_key_name,
tokens: {
input: r.tokens_input,
output: r.tokens_output,
cacheRead: r.tokens_cache_read,
cacheCreation: r.tokens_cache_creation,
reasoning: r.tokens_reasoning,
},
status: r.status,
timestamp: r.timestamp,
}));
const history = rows.map((row) => {
const r = asRecord(row);
return {
provider: toStringOrNull(r.provider),
model: toStringOrNull(r.model),
connectionId: toStringOrNull(r.connection_id),
apiKeyId: toStringOrNull(r.api_key_id),
apiKeyName: toStringOrNull(r.api_key_name),
tokens: {
input: toNumber(r.tokens_input),
output: toNumber(r.tokens_output),
cacheRead: toNumber(r.tokens_cache_read),
cacheCreation: toNumber(r.tokens_cache_creation),
reasoning: toNumber(r.tokens_reasoning),
},
status: toStringOrNull(r.status),
timestamp: toStringOrNull(r.timestamp),
};
});
return { data: { history } };
}
@@ -153,22 +180,25 @@ export async function getUsageHistory(filter: any = {}) {
sql += " ORDER BY timestamp ASC";
const rows = db.prepare(sql).all(params);
return rows.map((r) => ({
provider: r.provider,
model: r.model,
connectionId: r.connection_id,
apiKeyId: r.api_key_id,
apiKeyName: r.api_key_name,
tokens: {
input: r.tokens_input,
output: r.tokens_output,
cacheRead: r.tokens_cache_read,
cacheCreation: r.tokens_cache_creation,
reasoning: r.tokens_reasoning,
},
status: r.status,
timestamp: r.timestamp,
}));
return rows.map((row) => {
const r = asRecord(row);
return {
provider: toStringOrNull(r.provider),
model: toStringOrNull(r.model),
connectionId: toStringOrNull(r.connection_id),
apiKeyId: toStringOrNull(r.api_key_id),
apiKeyName: toStringOrNull(r.api_key_name),
tokens: {
input: toNumber(r.tokens_input),
output: toNumber(r.tokens_output),
cacheRead: toNumber(r.tokens_cache_read),
cacheCreation: toNumber(r.tokens_cache_creation),
reasoning: toNumber(r.tokens_reasoning),
},
status: toStringOrNull(r.status),
timestamp: toStringOrNull(r.timestamp),
};
});
}
// ──────────────── Request Log (log.txt) ────────────────
@@ -190,7 +220,19 @@ function formatLogDate(date = new Date()) {
/**
* Append to log.txt.
*/
export async function appendRequestLog({ model, provider, connectionId, tokens, status }: { model?: string; provider?: string; connectionId?: string; tokens?: any; status?: string | number }) {
export async function appendRequestLog({
model,
provider,
connectionId,
tokens,
status,
}: {
model?: string;
provider?: string;
connectionId?: string;
tokens?: any;
status?: string | number;
}) {
if (!shouldPersistToDisk) return;
try {
@@ -202,8 +244,11 @@ export async function appendRequestLog({ model, provider, connectionId, tokens,
try {
const { getProviderConnections } = await import("@/lib/localDb");
const connections = await getProviderConnections();
const conn = connections.find((c) => c.id === connectionId);
if (conn) account = conn.name || conn.email || account;
const connRaw = connections.find((c) => asRecord(c).id === connectionId);
if (connRaw) {
const conn = asRecord(connRaw);
account = toStringOrNull(conn.name) || toStringOrNull(conn.email) || account;
}
} catch {}
const sent =

View File

@@ -11,22 +11,46 @@ import { getDbInstance } from "../db/core";
import { getPendingRequests } from "./usageHistory";
import { calculateCost } from "./costCalculator";
type JsonRecord = Record<string, unknown>;
function asRecord(value: unknown): JsonRecord {
return value && typeof value === "object" && !Array.isArray(value) ? (value as JsonRecord) : {};
}
function toNumber(value: unknown): number {
if (typeof value === "number" && Number.isFinite(value)) return value;
if (typeof value === "string" && value.trim().length > 0) {
const parsed = Number(value);
return Number.isFinite(parsed) ? parsed : 0;
}
return 0;
}
function toStringOrEmpty(value: unknown): string {
return typeof value === "string" ? value : "";
}
/**
* Get aggregated usage stats.
*/
export async function getUsageStats() {
const db = getDbInstance();
const rows = db.prepare("SELECT * FROM usage_history ORDER BY timestamp ASC").all();
const rows = db.prepare("SELECT * FROM usage_history ORDER BY timestamp ASC").all() as unknown[];
const { getProviderConnections } = await import("@/lib/localDb");
let allConnections = [];
let allConnections: unknown[] = [];
try {
allConnections = await getProviderConnections();
const loadedConnections = await getProviderConnections();
allConnections = Array.isArray(loadedConnections) ? loadedConnections : [];
} catch {}
const connectionMap = {};
for (const conn of allConnections) {
connectionMap[conn.id] = conn.name || conn.email || conn.id;
const connectionMap: Record<string, string> = {};
for (const connRaw of allConnections) {
const conn = asRecord(connRaw);
const connectionId = toStringOrEmpty(conn.id);
if (!connectionId) continue;
connectionMap[connectionId] =
toStringOrEmpty(conn.name) || toStringOrEmpty(conn.email) || connectionId;
}
const pendingRequests = getPendingRequests();
@@ -75,19 +99,27 @@ export async function getUsageStats() {
const tenMinutesAgo = new Date(currentMinuteStart.getTime() - 9 * 60 * 1000);
for (const row of rows) {
const promptTokens = row.tokens_input || 0;
const completionTokens = row.tokens_output || 0;
const entryTime = new Date(row.timestamp);
for (const rowRaw of rows) {
const row = asRecord(rowRaw);
const provider = toStringOrEmpty(row.provider) || "unknown";
const model = toStringOrEmpty(row.model) || "unknown";
const timestamp = toStringOrEmpty(row.timestamp) || new Date(0).toISOString();
const connectionId = toStringOrEmpty(row.connection_id) || null;
const apiKeyId = toStringOrEmpty(row.api_key_id) || null;
const apiKeyName = toStringOrEmpty(row.api_key_name) || null;
const promptTokens = toNumber(row.tokens_input);
const completionTokens = toNumber(row.tokens_output);
const entryTime = new Date(timestamp);
const entryTokens = {
input: row.tokens_input,
output: row.tokens_output,
cacheRead: row.tokens_cache_read,
cacheCreation: row.tokens_cache_creation,
reasoning: row.tokens_reasoning,
input: toNumber(row.tokens_input),
output: toNumber(row.tokens_output),
cacheRead: toNumber(row.tokens_cache_read),
cacheCreation: toNumber(row.tokens_cache_creation),
reasoning: toNumber(row.tokens_reasoning),
};
const entryCost = await calculateCost(row.provider, row.model, entryTokens);
const entryCost = await calculateCost(provider, model, entryTokens);
stats.totalPromptTokens += promptTokens;
stats.totalCompletionTokens += completionTokens;
@@ -105,71 +137,70 @@ export async function getUsageStats() {
}
// By Provider
if (!stats.byProvider[row.provider]) {
stats.byProvider[row.provider] = {
if (!stats.byProvider[provider]) {
stats.byProvider[provider] = {
requests: 0,
promptTokens: 0,
completionTokens: 0,
cost: 0,
};
}
stats.byProvider[row.provider].requests++;
stats.byProvider[row.provider].promptTokens += promptTokens;
stats.byProvider[row.provider].completionTokens += completionTokens;
stats.byProvider[row.provider].cost += entryCost;
stats.byProvider[provider].requests++;
stats.byProvider[provider].promptTokens += promptTokens;
stats.byProvider[provider].completionTokens += completionTokens;
stats.byProvider[provider].cost += entryCost;
// By Model
const modelKey = row.provider ? `${row.model} (${row.provider})` : row.model;
const modelKey = provider ? `${model} (${provider})` : model;
if (!stats.byModel[modelKey]) {
stats.byModel[modelKey] = {
requests: 0,
promptTokens: 0,
completionTokens: 0,
cost: 0,
rawModel: row.model,
provider: row.provider,
lastUsed: row.timestamp,
rawModel: model,
provider,
lastUsed: timestamp,
};
}
stats.byModel[modelKey].requests++;
stats.byModel[modelKey].promptTokens += promptTokens;
stats.byModel[modelKey].completionTokens += completionTokens;
stats.byModel[modelKey].cost += entryCost;
if (new Date(row.timestamp) > new Date(stats.byModel[modelKey].lastUsed)) {
stats.byModel[modelKey].lastUsed = row.timestamp;
if (new Date(timestamp) > new Date(stats.byModel[modelKey].lastUsed)) {
stats.byModel[modelKey].lastUsed = timestamp;
}
// By Account
if (row.connection_id) {
const accountName =
connectionMap[row.connection_id] || `Account ${row.connection_id.slice(0, 8)}...`;
const accountKey = `${row.model} (${row.provider} - ${accountName})`;
if (connectionId) {
const accountName = connectionMap[connectionId] || `Account ${connectionId.slice(0, 8)}...`;
const accountKey = `${model} (${provider} - ${accountName})`;
if (!stats.byAccount[accountKey]) {
stats.byAccount[accountKey] = {
requests: 0,
promptTokens: 0,
completionTokens: 0,
cost: 0,
rawModel: row.model,
provider: row.provider,
connectionId: row.connection_id,
rawModel: model,
provider,
connectionId,
accountName,
lastUsed: row.timestamp,
lastUsed: timestamp,
};
}
stats.byAccount[accountKey].requests++;
stats.byAccount[accountKey].promptTokens += promptTokens;
stats.byAccount[accountKey].completionTokens += completionTokens;
stats.byAccount[accountKey].cost += entryCost;
if (new Date(row.timestamp) > new Date(stats.byAccount[accountKey].lastUsed)) {
stats.byAccount[accountKey].lastUsed = row.timestamp;
if (new Date(timestamp) > new Date(stats.byAccount[accountKey].lastUsed)) {
stats.byAccount[accountKey].lastUsed = timestamp;
}
}
// By API key
if (row.api_key_id || row.api_key_name) {
const keyName = row.api_key_name || row.api_key_id || "unknown";
const keyId = row.api_key_id || null;
if (apiKeyId || apiKeyName) {
const keyName = apiKeyName || apiKeyId || "unknown";
const keyId = apiKeyId || null;
const apiKey = keyId ? `${keyName} (${keyId})` : keyName;
if (!stats.byApiKey[apiKey]) {
stats.byApiKey[apiKey] = {
@@ -179,15 +210,15 @@ export async function getUsageStats() {
cost: 0,
apiKeyId: keyId,
apiKeyName: keyName,
lastUsed: row.timestamp,
lastUsed: timestamp,
};
}
stats.byApiKey[apiKey].requests++;
stats.byApiKey[apiKey].promptTokens += promptTokens;
stats.byApiKey[apiKey].completionTokens += completionTokens;
stats.byApiKey[apiKey].cost += entryCost;
if (new Date(row.timestamp) > new Date(stats.byApiKey[apiKey].lastUsed)) {
stats.byApiKey[apiKey].lastUsed = row.timestamp;
if (new Date(timestamp) > new Date(stats.byApiKey[apiKey].lastUsed)) {
stats.byApiKey[apiKey].lastUsed = timestamp;
}
}
}

View File

@@ -23,7 +23,6 @@ export interface ApiKeyMetadata {
noLog?: boolean;
budget?: number;
usedBudget?: number;
[key: string]: unknown;
}
export interface ApiKeyPolicyResult {

View File

@@ -75,7 +75,13 @@ export class CircuitBreaker {
try {
const saved = loadCircuitBreakerState(this.name);
if (saved) {
this.state = saved.state;
if (
saved.state === STATE.CLOSED ||
saved.state === STATE.OPEN ||
saved.state === STATE.HALF_OPEN
) {
this.state = saved.state;
}
this.failureCount = saved.failureCount;
this.lastFailureTime = saved.lastFailureTime;
if (this.state === STATE.HALF_OPEN) {

View File

@@ -28,7 +28,7 @@ export const ProvidersMapSchema = z.record(z.string(), ProviderSchema);
* @param {Record<string, object>} map - The providers map to validate
* @param {string} name - Name of the map for error messages
*/
export function validateProviders(map, name) {
export function validateProviders(map: Record<string, unknown>, name: string): void {
const result = ProvidersMapSchema.safeParse(map);
if (!result.success) {
const issues = result.error.issues.map((i) => ` ${i.path.join(".")}: ${i.message}`).join("\n");

View File

@@ -15,6 +15,69 @@ import {
} from "@omniroute/open-sse/services/accountFallback.ts";
import * as log from "../utils/logger";
type JsonRecord = Record<string, unknown>;
interface ProviderConnectionView {
id: string;
isActive: boolean;
rateLimitedUntil: string | null;
testStatus: string | null;
apiKey: string | null;
accessToken: string | null;
refreshToken: string | null;
tokenExpiresAt: string | null;
expiresAt: string | null;
projectId: string | null;
providerSpecificData: JsonRecord;
lastUsedAt: string | null;
consecutiveUseCount: number;
priority: number;
lastError: string | null;
errorCode: string | number | null;
backoffLevel: number;
}
function asRecord(value: unknown): JsonRecord {
return value && typeof value === "object" && !Array.isArray(value) ? (value as JsonRecord) : {};
}
function toStringOrNull(value: unknown): string | null {
return typeof value === "string" && value.trim().length > 0 ? value : null;
}
function toNumber(value: unknown, fallback = 0): number {
if (typeof value === "number" && Number.isFinite(value)) return value;
if (typeof value === "string" && value.trim().length > 0) {
const parsed = Number(value);
return Number.isFinite(parsed) ? parsed : fallback;
}
return fallback;
}
function toProviderConnection(value: unknown): ProviderConnectionView {
const row = asRecord(value);
return {
id: toStringOrNull(row.id) || "",
isActive: row.isActive === true,
rateLimitedUntil: toStringOrNull(row.rateLimitedUntil),
testStatus: toStringOrNull(row.testStatus),
apiKey: toStringOrNull(row.apiKey),
accessToken: toStringOrNull(row.accessToken),
refreshToken: toStringOrNull(row.refreshToken),
tokenExpiresAt: toStringOrNull(row.tokenExpiresAt),
expiresAt: toStringOrNull(row.expiresAt),
projectId: toStringOrNull(row.projectId),
providerSpecificData: asRecord(row.providerSpecificData),
lastUsedAt: toStringOrNull(row.lastUsedAt),
consecutiveUseCount: toNumber(row.consecutiveUseCount, 0),
priority: toNumber(row.priority, 999),
lastError: toStringOrNull(row.lastError),
errorCode:
typeof row.errorCode === "string" || typeof row.errorCode === "number" ? row.errorCode : null,
backoffLevel: toNumber(row.backoffLevel, 0),
};
}
// Mutex to prevent race conditions during account selection
let selectionMutex = Promise.resolve();
@@ -43,7 +106,10 @@ export async function getProviderCredentials(
try {
await currentMutex;
const connections = await getProviderConnections({ provider, isActive: true });
const connectionsRaw = await getProviderConnections({ provider, isActive: true });
const connections = (Array.isArray(connectionsRaw) ? connectionsRaw : [])
.map(toProviderConnection)
.filter((conn) => conn.id.length > 0);
log.debug(
"AUTH",
`${provider} | total connections: ${connections.length}, excludeId: ${excludeConnectionId || "none"}`
@@ -51,7 +117,10 @@ export async function getProviderCredentials(
if (connections.length === 0) {
// Check all connections (including inactive) to see if rate limited
const allConnections = await getProviderConnections({ provider });
const allConnectionsRaw = await getProviderConnections({ provider });
const allConnections = (Array.isArray(allConnectionsRaw) ? allConnectionsRaw : [])
.map(toProviderConnection)
.filter((conn) => conn.id.length > 0);
log.debug("AUTH", `${provider} | all connections (incl inactive): ${allConnections.length}`);
if (allConnections.length > 0) {
const earliest = getEarliestRateLimitedUntil(allConnections);
@@ -108,8 +177,9 @@ export async function getProviderCredentials(
(c) => c.rateLimitedUntil && new Date(c.rateLimitedUntil).getTime() > Date.now()
);
const earliestConn = rateLimitedConns.sort(
(a: any, b: any) =>
new Date(a.rateLimitedUntil).getTime() - new Date(b.rateLimitedUntil).getTime()
(a, b) =>
new Date(a.rateLimitedUntil || 0).getTime() -
new Date(b.rateLimitedUntil || 0).getTime()
)[0];
log.warn(
"AUTH",
@@ -132,7 +202,7 @@ export async function getProviderCredentials(
let connection;
if (strategy === "round-robin") {
const stickyLimit = settings.stickyRoundRobinLimit || 3;
const stickyLimit = toNumber((settings as Record<string, unknown>).stickyRoundRobinLimit, 3);
// Sort by lastUsed (most recent first) to find current candidate
const byRecency = [...availableConnections].sort((a: any, b: any) => {
@@ -191,7 +261,7 @@ export async function getProviderCredentials(
connection = availableConnections[idx];
} else if (strategy === "least-used") {
// Least Used: pick the one with oldest lastUsedAt
const sorted = [...availableConnections].sort((a: any, b: any) => {
const sorted = [...availableConnections].sort((a, b) => {
if (!a.lastUsedAt && !b.lastUsedAt) return (a.priority || 999) - (b.priority || 999);
if (!a.lastUsedAt) return -1;
if (!b.lastUsedAt) return 1;
@@ -202,7 +272,7 @@ export async function getProviderCredentials(
// Cost Optimized: sort by priority ascending (lower = cheaper/preferred)
// Future: can be enhanced with actual cost data per provider
const sorted = [...availableConnections].sort(
(a: any, b: any) => (a.priority || 999) - (b.priority || 999)
(a, b) => (a.priority || 999) - (b.priority || 999)
);
connection = sorted[0];
} else {
@@ -216,7 +286,10 @@ export async function getProviderCredentials(
refreshToken: connection.refreshToken,
expiresAt: connection.tokenExpiresAt || connection.expiresAt || null,
projectId: connection.projectId,
copilotToken: connection.providerSpecificData?.copilotToken,
copilotToken:
typeof connection.providerSpecificData.copilotToken === "string"
? connection.providerSpecificData.copilotToken
: null,
providerSpecificData: connection.providerSpecificData,
connectionId: connection.id,
// Include current status for optimization check
@@ -258,8 +331,11 @@ export async function markAccountUnavailable(
await currentMutex;
// Read current connection to get backoffLevel
const connections = await getProviderConnections({ provider });
const conn = connections.find((c) => c.id === connectionId);
const connectionsRaw = await getProviderConnections({ provider });
const connections = (Array.isArray(connectionsRaw) ? connectionsRaw : [])
.map(toProviderConnection)
.filter((connection) => connection.id.length > 0);
const conn = connections.find((connection) => connection.id === connectionId);
const backoffLevel = conn?.backoffLevel || 0;
// ─── Anti-Thundering Herd Guard ─────────────────────────────────

View File

@@ -55,3 +55,32 @@ test("stream execution marks task as failed when handler throws", async () => {
assert.equal(loaded?.state, "failed");
assert.deepEqual(loaded?.artifacts.at(-1), { type: "error", content: "upstream failure" });
});
test("expired submitted task transitions to failed without throwing", () => {
const tm = createManager();
const task = tm.createTask({
skill: "smart-routing",
messages: [{ role: "user", content: "hello" }],
});
task.expiresAt = new Date(Date.now() - 1_000).toISOString();
assert.doesNotThrow(() => tm.getTask(task.id));
const loaded = tm.getTask(task.id);
assert.equal(loaded?.state, "failed");
});
test("cleanup keeps cancelled tasks as cancelled", () => {
const tm = createManager();
const task = tm.createTask({
skill: "smart-routing",
messages: [{ role: "user", content: "cancel me" }],
});
tm.updateTask(task.id, "cancelled");
task.expiresAt = new Date(Date.now() - 1_000).toISOString();
// private in TS only; callable at runtime for regression test
tm.cleanupExpired();
const loaded = tm.getTask(task.id);
assert.equal(loaded?.state, "cancelled");
});

View File

@@ -9,6 +9,7 @@
},
"include": [],
"files": [
"src/app/api/settings/proxy/test/route.ts",
"src/lib/db/apiKeys.ts",
"src/lib/db/cliToolState.ts",
"src/lib/db/encryption.ts",
@@ -17,11 +18,14 @@
"src/lib/db/settings.ts",
"src/lib/db/stateReset.ts",
"open-sse/config/providerModels.ts",
"open-sse/config/providerRegistry.ts",
"open-sse/mcp-server/audit.ts",
"open-sse/mcp-server/server.ts",
"open-sse/mcp-server/tools/advancedTools.ts",
"open-sse/translator/registry.ts",
"open-sse/mcp-server/scopeEnforcement.ts",
"src/shared/validation/providerSchema.ts",
"src/shared/validation/schemas.ts",
"open-sse/handlers/responseSanitizer.ts",
"open-sse/handlers/responseTranslator.ts"
],