mirror of
https://github.com/diegosouzapw/OmniRoute.git
synced 2026-08-20 22:22:57 +03:00
feat: v2.0.0 - MCP server, A2A agent, proxy improvements and docs update
This commit is contained in:
83
CHANGELOG.md
83
CHANGELOG.md
@@ -9,26 +9,36 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
> ### ✨ Major Feature Release — MCP Server, A2A Protocol, Auto-Combo Engine & VS Code Extension
|
||||
_No unreleased changes._
|
||||
|
||||
---
|
||||
|
||||
## [2.0.0] — 2026-03-04
|
||||
|
||||
> ### 🚀 Major Release — MCP Server, A2A Protocol, Auto-Combo Engine & Full Type Safety Overhaul
|
||||
>
|
||||
> Full AI orchestration ecosystem: 16 MCP tools, A2A v0.3 server, self-healing Auto-Combo engine, and a VS Code extension with smart dispatch, budget tracking, and human checkpoints.
|
||||
> **OmniRoute 2.0** transforms the AI gateway into a fully **agent-controllable platform**. AI agents can now discover, orchestrate, and optimize routing through 16 MCP tools or the A2A v0.3 protocol. Accompanied by a self-healing Auto-Combo engine, VS Code extension, 3 new dashboard pages, and a comprehensive type safety overhaul across the entire codebase.
|
||||
|
||||
### 🆕 MCP Server (16 Tools)
|
||||
|
||||
- **8 Essential Tools** — `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`
|
||||
- **8 Advanced Tools** — `simulate_route`, `set_budget_guard`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`
|
||||
- **Scoped Authorization** — 8 permission scopes (read:health, write:combo, etc.)
|
||||
- **Audit Logging** — Every tool call logged with duration, arguments, and result
|
||||
- **IDE Configs** — MCP configuration templates for Antigravity, Cursor, Copilot, Claude Desktop
|
||||
- **Scoped Authorization** — 9 permission scopes (`read:health`, `read:combos`, `read:quota`, `read:usage`, `read:models`, `execute:completions`, `write:combos`, `write:budget`, `write:resilience`) with wildcard support
|
||||
- **Audit Logging** — Every tool call logged to SQLite with SHA-256 input hashing, output summarization, and duration tracking
|
||||
- **IDE Configs** — MCP configuration templates for Claude Desktop, Cursor, VS Code Copilot, and stdio transport
|
||||
- **Type-Safe Schemas** — All 16 tools defined with Zod input/output schemas, descriptions, and scope declarations
|
||||
- 📖 **Documentation** — [`open-sse/mcp-server/README.md`](open-sse/mcp-server/README.md) with architecture, tool reference, and client examples in Python, TypeScript, and Go
|
||||
|
||||
### 🤖 A2A Server (Agent-to-Agent v0.3)
|
||||
|
||||
- **JSON-RPC 2.0** — Full router with `message/send`, `message/stream`, `tasks/get`, `tasks/cancel`
|
||||
- **Agent Card** — Dynamic `/.well-known/agent.json` with 2 skills
|
||||
- **Skills** — `smart-routing` (routing explanation, cost envelope, resilience trace, policy verdict) and `quota-management` (natural language quota queries)
|
||||
- **SSE Streaming** — Real-time task streaming with 15s heartbeat
|
||||
- **Task Manager** — State machine (submitted→working→completed/failed/canceled), TTL, cleanup
|
||||
- **Routing Logger** — Decision audit trail with 7-day retention
|
||||
- **Agent Card** — Dynamic `/.well-known/agent.json` with 2 skills and bearer auth
|
||||
- **Skills** — `smart-routing` (routing explanation, cost envelope, resilience trace, policy verdict) and `quota-management` (natural language quota queries with ranking, free combo suggestions, and full summaries)
|
||||
- **SSE Streaming** — Real-time task streaming with 15s heartbeat, chunk events, and completion metadata
|
||||
- **Task Manager** — State machine (`submitted`→`working`→`completed`/`failed`/`cancelled`), TTL (5min default), auto-cleanup (2× TTL)
|
||||
- **Routing Logger** — Decision audit trail with 7-day retention and routing statistics
|
||||
- **Task Execution** — Generic executor with proper state transitions on success/failure
|
||||
- 📖 **Documentation** — [`src/lib/a2a/README.md`](src/lib/a2a/README.md) with JSON-RPC methods, skill reference, client examples, and MCP vs A2A comparison
|
||||
|
||||
### ⚡ Auto-Combo Engine
|
||||
|
||||
@@ -56,26 +66,65 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||
- **MCP Dashboard** — Tool listing, usage stats, audit log with 30s auto-refresh
|
||||
- **A2A Dashboard** — Agent Card display, skill listing, task history with routing metadata
|
||||
- **Auto-Combo Dashboard** — Provider score bars, factor breakdown, mode pack selector, incident indicator, exclusion list
|
||||
- **Error Pages** — Custom error and not-found pages for the dashboard
|
||||
|
||||
### 🔗 Integrations
|
||||
|
||||
- **OpenClaw** — Dynamic `provider.order` endpoint at `/api/cli-tools/openclaw/auto-order`
|
||||
|
||||
### 🔧 Code Quality & Type Safety
|
||||
|
||||
- **Eliminated `any` types** — Replaced `any` casts across `open-sse/` services, translators, and handlers with proper generics and explicit types
|
||||
- **Zod Validation Schemas** — Added Zod-based validation for all MCP tool inputs/outputs and API validation layer
|
||||
- **Shared Contracts** — Normalized quota and combos API responses with shared contracts (`src/shared/contracts/quota.ts`) for consistent data shapes across MCP, A2A, and REST APIs
|
||||
- **TypeScript Translator Types** — Added strict types and modularized the translator registry with proper interfaces
|
||||
- **DB Layer Hardening** — Improved database layer with proper error handling and type safety in the compliance module
|
||||
- **A2A Lifecycle Safety** — Enhanced A2A task lifecycle with type-safe state transitions, preventing invalid state changes on completed tasks
|
||||
- **Stream Handling** — Improved ComfyUI and stream handling with proper type safety
|
||||
|
||||
### 🧪 Tests
|
||||
|
||||
- **E2E Test Suite** — 6 scenarios (MCP, A2A, Auto-Combo, OpenClaw, Stress 100+50 parallel, Security)
|
||||
- **Unit Tests** — Essential tools, advanced tools, extension services, Auto-Combo engine, extension advanced features
|
||||
- **E2E Test Suite** — 6 scenarios covering MCP, A2A, Auto-Combo, OpenClaw, Stress (100+50 parallel), Security
|
||||
- **Unit Tests** — Essential tools (139 tests), advanced tools (141 tests), Auto-Combo engine (162 tests), A2A lifecycle regression tests
|
||||
- **Schema Hardening Tests** — `t06-schema-hardening.test.mjs` (132 tests) for input validation
|
||||
- **Security Tests** — `t07-no-log-key-config.test.mjs` (138 tests), `t08-mcp-scope-enforcement.test.mjs` (72 tests)
|
||||
- **Integration Tests** — `v1-contracts-behavior.test.mjs` (171 tests), `security-hardening.test.mjs` (103 tests)
|
||||
- **Migrated Tests to TypeScript** — E2E ecosystem tests migrated from `.mjs` to `.ts` with proper typing
|
||||
|
||||
### 📁 New Files (35+)
|
||||
### 📁 New Files (50+)
|
||||
|
||||
| Directory | Files |
|
||||
| :------------------------------- | :--------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `open-sse/mcp-server/` | `server.ts`, `transport.ts`, `auth.ts`, `audit.ts`, `tools/advancedTools.ts` |
|
||||
| `src/lib/a2a/` | `taskManager.ts`, `streaming.ts`, `routingLogger.ts`, `skills/smartRouting.ts`, `skills/quotaManagement.ts` |
|
||||
| `open-sse/mcp-server/` | `server.ts`, `index.ts`, `audit.ts`, `scopeEnforcement.ts`, `tools/advancedTools.ts`, `README.md` |
|
||||
| `open-sse/mcp-server/schemas/` | `tools.ts`, `a2a.ts`, `audit.ts`, `index.ts` |
|
||||
| `src/lib/a2a/` | `taskManager.ts`, `taskExecution.ts`, `streaming.ts`, `routingLogger.ts`, `README.md` |
|
||||
| `src/lib/a2a/skills/` | `smartRouting.ts`, `quotaManagement.ts` |
|
||||
| `src/app/a2a/` | `route.ts` (JSON-RPC 2.0 dispatch handler) |
|
||||
| `open-sse/services/autoCombo/` | `scoring.ts`, `taskFitness.ts`, `engine.ts`, `selfHealing.ts`, `modePacks.ts`, `persistence.ts`, `index.ts` |
|
||||
| `vscode-extension/src/services/` | `mcpClient.ts`, `a2aClient.ts`, `policyEngine.ts`, `preflightDialog.ts`, `budgetGuard.ts`, `healthMonitor.ts`, `modePackSelector.ts`, `humanCheckpoint.ts` |
|
||||
| `src/shared/contracts/` | `quota.ts` (shared API contracts) |
|
||||
| `src/shared/constants/` | `mcpScopes.ts` |
|
||||
| `src/lib/db/migrations/` | `002_mcp_a2a_tables.sql` |
|
||||
| `src/app/(dashboard)/` | `dashboard/mcp/page.tsx`, `dashboard/a2a/page.tsx`, `dashboard/auto-combo/page.tsx` |
|
||||
| `docs/` | `mcp-server.md`, `a2a-server.md`, `auto-combo.md`, `vscode-extension.md`, `integrations/ide-configs.md` |
|
||||
| `vscode-extension/src/services/` | `mcpClient.ts`, `a2aClient.ts`, `policyEngine.ts`, `preflightDialog.ts`, `budgetGuard.ts`, `healthMonitor.ts`, `modePackSelector.ts`, `humanCheckpoint.ts` |
|
||||
| `scripts/` | `check-cycles.mjs`, `check-docs-sync.mjs`, `check-route-validation.mjs`, `check-t11-any-budget.mjs`, `run-playwright-tests.mjs`, `runtime-env.mjs` |
|
||||
| `tests/` | `t06-schema-hardening.test.mjs`, `t07-no-log-key-config.test.mjs`, `t08-mcp-scope-enforcement.test.mjs`, `ecosystem.test.ts` |
|
||||
| `docs/` | `mcp-server.md`, `a2a-server.md`, `auto-combo.md`, `vscode-extension.md`, `integrations/ide-configs.md`, `RELEASE_CHECKLIST.md` |
|
||||
|
||||
### 📝 Commit History (`features-agente-mcp-a2a` branch)
|
||||
|
||||
| Commit | Date | Description |
|
||||
| :-------- | :--------- | :--------------------------------------------------------------------------------------- |
|
||||
| `e0ddb22` | 2026-03-03 | feat: add MCP server mode with `--mcp` flag for IDE integration |
|
||||
| `09a1748` | 2026-03-03 | feat: add Phase 3 advanced MCP tools and A2A smart routing skill |
|
||||
| `1e1a9c9` | 2026-03-04 | feat: migrate tests to TypeScript and add MCP advanced tools test suite |
|
||||
| `ab77452` | 2026-03-04 | feat: normalize quota and combos API responses with shared contracts |
|
||||
| `88ad4cc` | 2026-03-04 | feat: add MCP server, A2A protocol, auto-combo engine & VS Code extension |
|
||||
| `cc429d4` | 2026-03-04 | feat: add TypeScript types and modularize translator registry |
|
||||
| `adc8fdf` | 2026-03-04 | feat: add A2A protocol support and refactor API validation layer |
|
||||
| `500cae3` | 2026-03-04 | refactor: replace `any` types with generics and add Zod validation schemas |
|
||||
| `889e2ba` | 2026-03-04 | feat: add error pages, harden DB layer and compliance module |
|
||||
| `cbd0b1c` | 2026-03-04 | refactor: harden open-sse services, eliminate any casts, add dashboard pages |
|
||||
| `b33a853` | 2026-03-04 | feat: Introduce A2A lifecycle management, add type safety to ComfyUI and stream handling |
|
||||
|
||||
---
|
||||
|
||||
|
||||
62
README.md
62
README.md
@@ -5,9 +5,9 @@
|
||||
|
||||
### Never stop coding. Smart routing to **FREE & low-cost AI models** with automatic fallback.
|
||||
|
||||
_Your universal API proxy — one endpoint, 36+ providers, zero downtime._
|
||||
_Your universal API proxy — one endpoint, 36+ providers, zero downtime. Now with **MCP & A2A** agent orchestration._
|
||||
|
||||
**Chat Completions • Embeddings • Image Generation • Video • Music • Audio • Reranking • 100% TypeScript**
|
||||
**Chat Completions • Embeddings • Image Generation • Video • Music • Audio • Reranking • MCP Server • A2A Protocol • 100% TypeScript**
|
||||
|
||||
---
|
||||
|
||||
@@ -445,6 +445,7 @@ omniroute
|
||||
| ----------------------- | ----------------------------------------------------------- |
|
||||
| `omniroute` | Start server (`PORT=20128`, API and dashboard on same port) |
|
||||
| `omniroute --port 3000` | Set canonical/API port to 3000 |
|
||||
| `omniroute --mcp` | Start MCP server (stdio transport) for IDE integration |
|
||||
| `omniroute --no-open` | Don't auto-open browser |
|
||||
| `omniroute --help` | Show help |
|
||||
|
||||
@@ -594,6 +595,23 @@ When minimized, OmniRoute lives in your system tray with quick actions:
|
||||
|
||||
## 💡 Key Features
|
||||
|
||||
### 🤖 Agent Integration (NEW in v2.0)
|
||||
|
||||
| Feature | What It Does |
|
||||
| ---------------------------- | ----------------------------------------------------------------------------------- |
|
||||
| 🔧 **MCP Server (16 Tools)** | IDE agents control OmniRoute via Model Context Protocol — health, routing, budgets |
|
||||
| 🤝 **A2A Server (v0.3)** | Multi-agent orchestration via JSON-RPC 2.0 with smart-routing & quota-management |
|
||||
| ⚡ **Auto-Combo Engine** | Self-healing 6-factor scoring with task fitness, mode packs, and bandit exploration |
|
||||
| 🎯 **Scope Enforcement** | 9 granular permission scopes for MCP tool access control |
|
||||
| 📊 **Audit Logging** | SHA-256 hashed tool call audit trail in SQLite |
|
||||
| 📡 **SSE Streaming** | Real-time A2A task streaming with heartbeat and completion events |
|
||||
| 🛡️ **Budget Guard** | Session-level budget enforcement with degrade/block/alert actions |
|
||||
| 📋 **Agent Card Discovery** | `/.well-known/agent.json` for automatic A2A agent discovery |
|
||||
|
||||
> 📖 **[MCP Server README](open-sse/mcp-server/README.md)** — Full tool reference, IDE configs, and client examples in Python/TypeScript/Go
|
||||
>
|
||||
> 📖 **[A2A Server README](src/lib/a2a/README.md)** — Skills reference, JSON-RPC methods, LangChain integration, and streaming examples
|
||||
|
||||
### 🧠 Core Routing & Intelligence
|
||||
|
||||
| Feature | What It Does |
|
||||
@@ -1315,33 +1333,39 @@ Se não quiser criar credenciais próprias agora, ainda é possível usar o flux
|
||||
## 🛠️ Tech Stack
|
||||
|
||||
- **Runtime**: Node.js 18–22 LTS (⚠️ Node.js 24+ is **not supported** — `better-sqlite3` native binaries are incompatible)
|
||||
- **Language**: TypeScript 5.9 — **100% TypeScript** across `src/` and `open-sse/` (v1.0.6)
|
||||
- **Language**: TypeScript 5.9 — **100% TypeScript** across `src/` and `open-sse/` (zero `any` in core modules since v2.0)
|
||||
- **Framework**: Next.js 16 + React 19 + Tailwind CSS 4
|
||||
- **Database**: LowDB (JSON) + SQLite (domain state + proxy logs)
|
||||
- **Database**: LowDB (JSON) + SQLite (domain state + proxy logs + MCP audit + routing decisions)
|
||||
- **Schemas**: Zod (MCP tool I/O validation, API contracts)
|
||||
- **Protocols**: MCP (stdio/HTTP) + A2A v0.3 (JSON-RPC 2.0 + SSE)
|
||||
- **Streaming**: Server-Sent Events (SSE)
|
||||
- **Auth**: OAuth 2.0 (PKCE) + JWT + API Keys
|
||||
- **Testing**: Node.js test runner (368+ unit tests)
|
||||
- **Auth**: OAuth 2.0 (PKCE) + JWT + API Keys + MCP Scoped Authorization
|
||||
- **Testing**: Node.js test runner + Vitest (900+ tests including unit, integration, E2E)
|
||||
- **CI/CD**: GitHub Actions (auto npm publish + Docker Hub on release)
|
||||
- **Website**: [omniroute.online](https://omniroute.online)
|
||||
- **Package**: [npmjs.com/package/omniroute](https://www.npmjs.com/package/omniroute)
|
||||
- **Docker**: [hub.docker.com/r/diegosouzapw/omniroute](https://hub.docker.com/r/diegosouzapw/omniroute)
|
||||
- **Resilience**: Circuit breaker, exponential backoff, anti-thundering herd, TLS spoofing
|
||||
- **Resilience**: Circuit breaker, exponential backoff, anti-thundering herd, TLS spoofing, auto-combo self-healing
|
||||
|
||||
---
|
||||
|
||||
## 📖 Documentation
|
||||
|
||||
| Document | Description |
|
||||
| -------------------------------------------- | ---------------------------------------------- |
|
||||
| [User Guide](docs/USER_GUIDE.md) | Providers, combos, CLI integration, deployment |
|
||||
| [API Reference](docs/API_REFERENCE.md) | All endpoints with examples |
|
||||
| [Troubleshooting](docs/TROUBLESHOOTING.md) | Common problems and solutions |
|
||||
| [Architecture](docs/ARCHITECTURE.md) | System architecture and internals |
|
||||
| [Contributing](CONTRIBUTING.md) | Development setup and guidelines |
|
||||
| [OpenAPI Spec](docs/openapi.yaml) | OpenAPI 3.0 specification |
|
||||
| [Security Policy](SECURITY.md) | Vulnerability reporting and security practices |
|
||||
| [VM Deployment](docs/VM_DEPLOYMENT_GUIDE.md) | Complete guide: VM + nginx + Cloudflare setup |
|
||||
| [Features Gallery](docs/FEATURES.md) | Visual dashboard tour with screenshots |
|
||||
| Document | Description |
|
||||
| ---------------------------------------------- | --------------------------------------------------- |
|
||||
| [User Guide](docs/USER_GUIDE.md) | Providers, combos, CLI integration, deployment |
|
||||
| [API Reference](docs/API_REFERENCE.md) | All endpoints with examples |
|
||||
| [MCP Server](open-sse/mcp-server/README.md) | 16 MCP tools, IDE configs, Python/TS/Go clients |
|
||||
| [A2A Server](src/lib/a2a/README.md) | JSON-RPC 2.0 protocol, skills, streaming, task mgmt |
|
||||
| [Auto-Combo Engine](docs/auto-combo.md) | 6-factor scoring, mode packs, self-healing |
|
||||
| [Troubleshooting](docs/TROUBLESHOOTING.md) | Common problems and solutions |
|
||||
| [Architecture](docs/ARCHITECTURE.md) | System architecture and internals |
|
||||
| [Contributing](CONTRIBUTING.md) | Development setup and guidelines |
|
||||
| [OpenAPI Spec](docs/openapi.yaml) | OpenAPI 3.0 specification |
|
||||
| [Security Policy](SECURITY.md) | Vulnerability reporting and security practices |
|
||||
| [VM Deployment](docs/VM_DEPLOYMENT_GUIDE.md) | Complete guide: VM + nginx + Cloudflare setup |
|
||||
| [Features Gallery](docs/FEATURES.md) | Visual dashboard tour with screenshots |
|
||||
| [Release Checklist](docs/RELEASE_CHECKLIST.md) | Pre-release validation steps |
|
||||
|
||||
### 📸 Dashboard Preview
|
||||
|
||||
@@ -1407,7 +1431,7 @@ See [CONTRIBUTING.md](CONTRIBUTING.md) for detailed guidelines.
|
||||
|
||||
```bash
|
||||
# Create a release — npm publish happens automatically
|
||||
gh release create v1.0.6 --title "v1.0.6" --generate-notes
|
||||
gh release create v2.0.0 --title "v2.0.0" --generate-notes
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
65
bin/mcp-server.mjs
Normal file
65
bin/mcp-server.mjs
Normal file
@@ -0,0 +1,65 @@
|
||||
#!/usr/bin/env node
|
||||
|
||||
import { spawn } from "node:child_process";
|
||||
import { existsSync } from "node:fs";
|
||||
import { dirname, join } from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
|
||||
const __filename = fileURLToPath(import.meta.url);
|
||||
const __dirname = dirname(__filename);
|
||||
const ROOT = join(__dirname, "..");
|
||||
|
||||
function resolveMcpEntry(rootDir = ROOT) {
|
||||
const candidates = [
|
||||
// Preferred distributable JS entry (npm publish artifact)
|
||||
join(rootDir, "app", "open-sse", "mcp-server", "server.js"),
|
||||
// Local workspace TypeScript source fallback
|
||||
join(rootDir, "open-sse", "mcp-server", "server.ts"),
|
||||
];
|
||||
|
||||
for (const entry of candidates) {
|
||||
if (existsSync(entry)) return entry;
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
function formatSpawnError(exitCode, signal) {
|
||||
if (signal) return `MCP server exited by signal ${signal}`;
|
||||
return `MCP server exited with code ${exitCode ?? 1}`;
|
||||
}
|
||||
|
||||
export async function startMcpCli(rootDir = ROOT) {
|
||||
const mcpEntry = resolveMcpEntry(rootDir);
|
||||
if (!mcpEntry) {
|
||||
throw new Error(
|
||||
"MCP server entrypoint not found. Expected app/open-sse/mcp-server/server.js or open-sse/mcp-server/server.ts."
|
||||
);
|
||||
}
|
||||
|
||||
// `tsx` loader is only required for local `.ts` fallback; JS entry works without it.
|
||||
const loaderArgs = mcpEntry.endsWith(".ts") ? ["--import", "tsx/esm"] : [];
|
||||
|
||||
await new Promise((resolve, reject) => {
|
||||
const child = spawn(process.execPath, [...loaderArgs, mcpEntry], {
|
||||
cwd: rootDir,
|
||||
env: process.env,
|
||||
stdio: "inherit",
|
||||
});
|
||||
|
||||
child.once("error", reject);
|
||||
child.once("exit", (code, signal) => {
|
||||
if ((code ?? 0) === 0 && !signal) {
|
||||
resolve(undefined);
|
||||
return;
|
||||
}
|
||||
reject(new Error(formatSpawnError(code, signal)));
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
if (process.argv[1] && fileURLToPath(import.meta.url) === process.argv[1]) {
|
||||
startMcpCli().catch((err) => {
|
||||
console.error("\x1b[31m✖ Failed to start MCP server:\x1b[0m", err?.message || err);
|
||||
process.exit(1);
|
||||
});
|
||||
}
|
||||
@@ -128,14 +128,13 @@ if (args.includes("--version") || args.includes("-v")) {
|
||||
// ── MCP Server Mode ───────────────────────────────────────
|
||||
if (args.includes("--mcp")) {
|
||||
try {
|
||||
const { startMcpStdio } = await import(join(ROOT, "open-sse", "mcp-server", "server.ts"));
|
||||
await startMcpStdio();
|
||||
const { startMcpCli } = await import(join(ROOT, "bin", "mcp-server.mjs"));
|
||||
await startMcpCli(ROOT);
|
||||
} catch (err) {
|
||||
console.error("\x1b[31m✖ Failed to start MCP server:\x1b[0m", err.message || err);
|
||||
process.exit(1);
|
||||
}
|
||||
// MCP server runs indefinitely via stdio — don't fall through to Next.js server
|
||||
await new Promise(() => {}); // Keep process alive
|
||||
process.exit(0);
|
||||
}
|
||||
|
||||
function parsePort(value, fallback) {
|
||||
|
||||
@@ -2,26 +2,37 @@ import { HTTP_STATUS, FETCH_TIMEOUT_MS } from "../config/constants.ts";
|
||||
|
||||
type JsonRecord = Record<string, unknown>;
|
||||
|
||||
type ProviderConfig = {
|
||||
export type ProviderConfig = {
|
||||
id?: string;
|
||||
baseUrl?: string;
|
||||
baseUrls?: string[];
|
||||
responsesBaseUrl?: string;
|
||||
chatPath?: string;
|
||||
clientVersion?: string;
|
||||
clientId?: string;
|
||||
clientSecret?: string;
|
||||
tokenUrl?: string;
|
||||
refreshUrl?: string;
|
||||
authUrl?: string;
|
||||
headers?: Record<string, string>;
|
||||
};
|
||||
|
||||
type ProviderCredentials = {
|
||||
export type ProviderCredentials = {
|
||||
accessToken?: string;
|
||||
refreshToken?: string;
|
||||
apiKey?: string;
|
||||
expiresAt?: string;
|
||||
providerSpecificData?: JsonRecord;
|
||||
};
|
||||
|
||||
type ExecutorLog = {
|
||||
export type ExecutorLog = {
|
||||
debug?: (tag: string, message: string) => void;
|
||||
info?: (tag: string, message: string) => void;
|
||||
warn?: (tag: string, message: string) => void;
|
||||
error?: (tag: string, message: string) => void;
|
||||
};
|
||||
|
||||
type ExecuteInput = {
|
||||
export type ExecuteInput = {
|
||||
model: string;
|
||||
body: unknown;
|
||||
stream: boolean;
|
||||
|
||||
@@ -232,7 +232,7 @@ export class CursorExecutor extends BaseExecutor {
|
||||
const response = await fetch(url, {
|
||||
method: "POST",
|
||||
headers,
|
||||
body,
|
||||
body: body as unknown as BodyInit,
|
||||
signal,
|
||||
});
|
||||
|
||||
|
||||
@@ -1,4 +1,9 @@
|
||||
import { BaseExecutor } from "./base.ts";
|
||||
import {
|
||||
BaseExecutor,
|
||||
type ExecuteInput,
|
||||
type ExecutorLog,
|
||||
type ProviderCredentials,
|
||||
} from "./base.ts";
|
||||
import { PROVIDERS } from "../config/constants.ts";
|
||||
import { v4 as uuidv4 } from "uuid";
|
||||
import { refreshKiroToken } from "../services/tokenRefresh.ts";
|
||||
@@ -56,7 +61,7 @@ export class KiroExecutor extends BaseExecutor {
|
||||
super("kiro", PROVIDERS.kiro);
|
||||
}
|
||||
|
||||
buildHeaders(credentials: { accessToken?: string }, stream = true) {
|
||||
buildHeaders(credentials: ProviderCredentials, stream = true) {
|
||||
void stream;
|
||||
const headers = {
|
||||
...this.config.headers,
|
||||
@@ -81,21 +86,7 @@ export class KiroExecutor extends BaseExecutor {
|
||||
/**
|
||||
* Custom execute for Kiro - handles AWS EventStream binary response
|
||||
*/
|
||||
async execute({
|
||||
model,
|
||||
body,
|
||||
stream,
|
||||
credentials,
|
||||
signal,
|
||||
log,
|
||||
}: {
|
||||
model: string;
|
||||
body: unknown;
|
||||
stream: boolean;
|
||||
credentials: { accessToken?: string; refreshToken?: string; providerSpecificData?: unknown };
|
||||
signal?: AbortSignal;
|
||||
log?: { error?: (tag: string, message: string) => void } | null;
|
||||
}) {
|
||||
async execute({ model, body, stream, credentials, signal, log }: ExecuteInput) {
|
||||
const url = this.buildUrl(model, stream, 0);
|
||||
const headers = this.buildHeaders(credentials, stream);
|
||||
const transformedBody = this.transformRequest(model, body, stream, credentials);
|
||||
@@ -166,8 +157,11 @@ export class KiroExecutor extends BaseExecutor {
|
||||
if (!state.contextUsagePercentage) state.contextUsagePercentage = 0;
|
||||
|
||||
// Handle assistantResponseEvent
|
||||
if (eventType === "assistantResponseEvent" && event.payload?.content) {
|
||||
const content = event.payload.content;
|
||||
if (eventType === "assistantResponseEvent") {
|
||||
const content = typeof event.payload?.content === "string" ? event.payload.content : "";
|
||||
if (!content) {
|
||||
continue;
|
||||
}
|
||||
state.totalContentLength += content.length;
|
||||
|
||||
const chunk: JsonRecord = {
|
||||
@@ -319,8 +313,15 @@ export class KiroExecutor extends BaseExecutor {
|
||||
}
|
||||
|
||||
// Handle contextUsageEvent to extract contextUsagePercentage
|
||||
if (eventType === "contextUsageEvent" && event.payload?.contextUsagePercentage) {
|
||||
state.contextUsagePercentage = event.payload.contextUsagePercentage;
|
||||
if (eventType === "contextUsageEvent") {
|
||||
const contextUsage =
|
||||
typeof event.payload?.contextUsagePercentage === "number"
|
||||
? event.payload.contextUsagePercentage
|
||||
: 0;
|
||||
if (contextUsage <= 0) {
|
||||
continue;
|
||||
}
|
||||
state.contextUsagePercentage = contextUsage;
|
||||
// Mark that we received context usage event
|
||||
state.hasContextUsage = true;
|
||||
}
|
||||
@@ -335,8 +336,14 @@ export class KiroExecutor extends BaseExecutor {
|
||||
// Extract usage data from metricsEvent payload
|
||||
const metrics = event.payload?.metricsEvent || event.payload;
|
||||
if (metrics && typeof metrics === "object") {
|
||||
const inputTokens = metrics.inputTokens || 0;
|
||||
const outputTokens = metrics.outputTokens || 0;
|
||||
const inputTokens =
|
||||
typeof (metrics as JsonRecord).inputTokens === "number"
|
||||
? ((metrics as JsonRecord).inputTokens as number)
|
||||
: 0;
|
||||
const outputTokens =
|
||||
typeof (metrics as JsonRecord).outputTokens === "number"
|
||||
? ((metrics as JsonRecord).outputTokens as number)
|
||||
: 0;
|
||||
|
||||
if (inputTokens > 0 || outputTokens > 0) {
|
||||
state.usage = {
|
||||
@@ -443,10 +450,7 @@ export class KiroExecutor extends BaseExecutor {
|
||||
});
|
||||
}
|
||||
|
||||
async refreshCredentials(
|
||||
credentials: { refreshToken?: string; providerSpecificData?: unknown },
|
||||
log?: { error?: (tag: string, message: string) => void } | null
|
||||
) {
|
||||
async refreshCredentials(credentials: ProviderCredentials, log?: ExecutorLog | null) {
|
||||
if (!credentials.refreshToken) return null;
|
||||
|
||||
try {
|
||||
|
||||
@@ -220,7 +220,7 @@ export async function handleAudioTranscription({
|
||||
credentials?: TranscriptionCredentials | null;
|
||||
}): Promise<Response> {
|
||||
const model = formData.get("model");
|
||||
if (!model) {
|
||||
if (typeof model !== "string" || !model) {
|
||||
return errorResponse(400, "model is required");
|
||||
}
|
||||
|
||||
|
||||
@@ -223,12 +223,15 @@ export function translateNonStreamingResponse(
|
||||
finishReason = "tool_calls";
|
||||
}
|
||||
|
||||
const createdMs = Date.parse(toString(response.createTime));
|
||||
const created = Number.isFinite(createdMs)
|
||||
? Math.floor(createdMs / 1000)
|
||||
: Math.floor(Date.now() / 1000);
|
||||
|
||||
const result: JsonRecord = {
|
||||
id: `chatcmpl-${toString(response.responseId, String(Date.now()))}`,
|
||||
object: "chat.completion",
|
||||
created: Math.floor(
|
||||
new Date(toString(response.createTime, String(Date.now()))).getTime() / 1000
|
||||
),
|
||||
created,
|
||||
model: toString(response.modelVersion, "gemini"),
|
||||
choices: [
|
||||
{
|
||||
|
||||
587
open-sse/mcp-server/README.md
Normal file
587
open-sse/mcp-server/README.md
Normal file
@@ -0,0 +1,587 @@
|
||||
# OmniRoute MCP Server
|
||||
|
||||
> **Model Context Protocol server** that exposes OmniRoute's gateway intelligence as **16 tools** for AI agents.
|
||||
|
||||
The MCP Server allows any AI agent (Claude Desktop, Cursor, VS Code Copilot, custom agents) to **monitor, control, and optimize** the OmniRoute AI gateway programmatically.
|
||||
|
||||
---
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
┌──────────────────────────────────────────────────────────────────┐
|
||||
│ AI Agent / IDE │
|
||||
│ (Claude Desktop, Cursor, VS Code, Custom) │
|
||||
└──────────────────────┬───────────────────────────────────────────┘
|
||||
│ MCP Protocol (stdio or HTTP)
|
||||
▼
|
||||
┌──────────────────────────────────────────────────────────────────┐
|
||||
│ OmniRoute MCP Server │
|
||||
│ ┌──────────────┐ ┌─────────────────┐ ┌────────────────────┐ │
|
||||
│ │ Scope │ │ 16 MCP Tools │ │ Audit Logger │ │
|
||||
│ │ Enforcement │──│ (Phase 1 + 2) │──│ (SHA-256/SQLite) │ │
|
||||
│ └──────────────┘ └────────┬────────┘ └────────────────────┘ │
|
||||
└─────────────────────────────┼────────────────────────────────────┘
|
||||
│ HTTP (internal)
|
||||
▼
|
||||
┌──────────────────────────────────────────────────────────────────┐
|
||||
│ OmniRoute Gateway (port 20128) │
|
||||
│ /v1/chat/completions /api/combos /api/usage ... │
|
||||
└──────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Quick Start
|
||||
|
||||
### 1. Environment Variables
|
||||
|
||||
```bash
|
||||
# Required: OmniRoute base URL
|
||||
export OMNIROUTE_BASE_URL="http://localhost:20128"
|
||||
|
||||
# Optional: API key for authenticated access
|
||||
export OMNIROUTE_API_KEY="your-api-key"
|
||||
|
||||
# Optional: Scope enforcement (default: disabled)
|
||||
export OMNIROUTE_MCP_ENFORCE_SCOPES="true"
|
||||
export OMNIROUTE_MCP_SCOPES="read:health,read:combos,read:quota,read:usage,read:models,execute:completions,write:combos,write:budget,write:resilience"
|
||||
```
|
||||
|
||||
### 2. stdio Transport (IDE Integration)
|
||||
|
||||
Add to your MCP client configuration:
|
||||
|
||||
**Claude Desktop** (`claude_desktop_config.json`):
|
||||
|
||||
```json
|
||||
{
|
||||
"mcpServers": {
|
||||
"omniroute": {
|
||||
"command": "node",
|
||||
"args": ["path/to/9router/open-sse/mcp-server/server.ts"],
|
||||
"env": {
|
||||
"OMNIROUTE_BASE_URL": "http://localhost:20128",
|
||||
"OMNIROUTE_API_KEY": "your-key"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Cursor** (`.cursor/mcp.json`):
|
||||
|
||||
```json
|
||||
{
|
||||
"mcpServers": {
|
||||
"omniroute": {
|
||||
"command": "npx",
|
||||
"args": ["tsx", "open-sse/mcp-server/server.ts"],
|
||||
"env": {
|
||||
"OMNIROUTE_BASE_URL": "http://localhost:20128"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**VS Code** (`.vscode/settings.json`):
|
||||
|
||||
```json
|
||||
{
|
||||
"mcp": {
|
||||
"servers": {
|
||||
"omniroute": {
|
||||
"command": "npx",
|
||||
"args": ["tsx", "open-sse/mcp-server/server.ts"],
|
||||
"env": {
|
||||
"OMNIROUTE_BASE_URL": "http://localhost:20128"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 3. Start via CLI
|
||||
|
||||
```bash
|
||||
# Direct start (stdio)
|
||||
npx tsx open-sse/mcp-server/server.ts
|
||||
|
||||
# Or via OmniRoute CLI
|
||||
omniroute --mcp
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Tool Reference
|
||||
|
||||
### Phase 1: Essential Tools (8)
|
||||
|
||||
| # | Tool | Scopes | Description |
|
||||
| --- | ------------------------------- | --------------------- | -------------------------------------------------------------------------- |
|
||||
| 1 | `omniroute_get_health` | `read:health` | Gateway health, uptime, memory, circuit breakers, rate limits, cache stats |
|
||||
| 2 | `omniroute_list_combos` | `read:combos` | List all combos (model chains) with strategies and optional metrics |
|
||||
| 3 | `omniroute_get_combo_metrics` | `read:combos` | Performance metrics for a specific combo |
|
||||
| 4 | `omniroute_switch_combo` | `write:combos` | Activate or deactivate a combo for routing |
|
||||
| 5 | `omniroute_check_quota` | `read:quota` | Remaining API quota per provider with token health status |
|
||||
| 6 | `omniroute_route_request` | `execute:completions` | Send a chat completion through intelligent routing |
|
||||
| 7 | `omniroute_cost_report` | `read:usage` | Cost report by period (session/day/week/month) with per-provider breakdown |
|
||||
| 8 | `omniroute_list_models_catalog` | `read:models` | List all available models across providers with capabilities and pricing |
|
||||
|
||||
### Phase 2: Advanced Tools (8)
|
||||
|
||||
| # | Tool | Scopes | Description |
|
||||
| --- | ---------------------------------- | ------------------------------------ | ---------------------------------------------------------------------------- |
|
||||
| 9 | `omniroute_simulate_route` | `read:health`, `read:combos` | Dry-run routing simulation showing fallback tree and estimated costs |
|
||||
| 10 | `omniroute_set_budget_guard` | `write:budget` | Set session budget with action on exceed: `degrade`, `block`, or `alert` |
|
||||
| 11 | `omniroute_set_resilience_profile` | `write:resilience` | Apply resilience profile: `aggressive`, `balanced`, or `conservative` |
|
||||
| 12 | `omniroute_test_combo` | `execute:completions`, `read:combos` | Test each provider in a combo with a real prompt, report latency/cost |
|
||||
| 13 | `omniroute_get_provider_metrics` | `read:health` | Per-provider metrics with latency percentiles (p50/p95/p99), circuit breaker |
|
||||
| 14 | `omniroute_best_combo_for_task` | `read:combos`, `read:health` | AI-powered combo recommendation by task type with budget/latency constraints |
|
||||
| 15 | `omniroute_explain_route` | `read:health`, `read:usage` | Explain why a request was routed to a provider (scoring factors, fallbacks) |
|
||||
| 16 | `omniroute_get_session_snapshot` | `read:usage` | Full session snapshot: cost, tokens, top models, errors, budget status |
|
||||
|
||||
---
|
||||
|
||||
## Client Examples
|
||||
|
||||
### Python — Full Agent Workflow
|
||||
|
||||
```python
|
||||
"""
|
||||
OmniRoute MCP Client — Python example using the mcp SDK.
|
||||
Install: pip install mcp
|
||||
"""
|
||||
import asyncio
|
||||
from mcp import ClientSession, StdioServerParameters
|
||||
from mcp.client.stdio import stdio_client
|
||||
|
||||
async def main():
|
||||
server = StdioServerParameters(
|
||||
command="npx",
|
||||
args=["tsx", "open-sse/mcp-server/server.ts"],
|
||||
env={
|
||||
"OMNIROUTE_BASE_URL": "http://localhost:20128",
|
||||
"OMNIROUTE_API_KEY": "your-key",
|
||||
},
|
||||
)
|
||||
|
||||
async with stdio_client(server) as (read, write):
|
||||
async with ClientSession(read, write) as session:
|
||||
await session.initialize()
|
||||
|
||||
# 1. Check gateway health
|
||||
health = await session.call_tool("omniroute_get_health", {})
|
||||
print("Health:", health.content[0].text)
|
||||
|
||||
# 2. List available combos with metrics
|
||||
combos = await session.call_tool("omniroute_list_combos", {
|
||||
"includeMetrics": True
|
||||
})
|
||||
print("Combos:", combos.content[0].text)
|
||||
|
||||
# 3. Find the best combo for a coding task
|
||||
best = await session.call_tool("omniroute_best_combo_for_task", {
|
||||
"taskType": "coding",
|
||||
"budgetConstraint": 0.50,
|
||||
"latencyConstraint": 5000,
|
||||
})
|
||||
print("Best combo:", best.content[0].text)
|
||||
|
||||
# 4. Set a session budget guard
|
||||
budget = await session.call_tool("omniroute_set_budget_guard", {
|
||||
"maxCost": 1.00,
|
||||
"action": "degrade",
|
||||
"degradeToTier": "cheap",
|
||||
})
|
||||
print("Budget guard:", budget.content[0].text)
|
||||
|
||||
# 5. Route a request through intelligent pipeline
|
||||
response = await session.call_tool("omniroute_route_request", {
|
||||
"model": "claude-sonnet-4",
|
||||
"messages": [
|
||||
{"role": "user", "content": "Write a Python hello world"}
|
||||
],
|
||||
"role": "coding",
|
||||
})
|
||||
print("Response:", response.content[0].text)
|
||||
|
||||
# 6. Get the session snapshot
|
||||
snapshot = await session.call_tool("omniroute_get_session_snapshot", {})
|
||||
print("Session:", snapshot.content[0].text)
|
||||
|
||||
asyncio.run(main())
|
||||
```
|
||||
|
||||
### TypeScript — Programmatic Agent
|
||||
|
||||
```typescript
|
||||
import { Client } from "@modelcontextprotocol/sdk/client/index.js";
|
||||
import { StdioClientTransport } from "@modelcontextprotocol/sdk/client/stdio.js";
|
||||
|
||||
async function main() {
|
||||
const transport = new StdioClientTransport({
|
||||
command: "npx",
|
||||
args: ["tsx", "open-sse/mcp-server/server.ts"],
|
||||
env: {
|
||||
OMNIROUTE_BASE_URL: "http://localhost:20128",
|
||||
OMNIROUTE_API_KEY: "your-key",
|
||||
},
|
||||
});
|
||||
|
||||
const client = new Client({ name: "my-agent", version: "1.0.0" });
|
||||
await client.connect(transport);
|
||||
|
||||
// Check quota before deciding which model to use
|
||||
const quota = await client.callTool({
|
||||
name: "omniroute_check_quota",
|
||||
arguments: { provider: "claude" },
|
||||
});
|
||||
console.log("Claude quota:", quota.content);
|
||||
|
||||
// Simulate the route before actually calling
|
||||
const simulation = await client.callTool({
|
||||
name: "omniroute_simulate_route",
|
||||
arguments: {
|
||||
model: "claude-sonnet-4",
|
||||
promptTokenEstimate: 2000,
|
||||
},
|
||||
});
|
||||
console.log("Route simulation:", simulation.content);
|
||||
|
||||
// Send the actual request
|
||||
const result = await client.callTool({
|
||||
name: "omniroute_route_request",
|
||||
arguments: {
|
||||
model: "claude-sonnet-4",
|
||||
messages: [{ role: "user", content: "Explain async/await" }],
|
||||
},
|
||||
});
|
||||
console.log("Result:", result.content);
|
||||
|
||||
// Cost report
|
||||
const costs = await client.callTool({
|
||||
name: "omniroute_cost_report",
|
||||
arguments: { period: "session" },
|
||||
});
|
||||
console.log("Costs:", costs.content);
|
||||
|
||||
await client.close();
|
||||
}
|
||||
|
||||
main();
|
||||
```
|
||||
|
||||
### Go — HTTP Client
|
||||
|
||||
```go
|
||||
package main
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
)
|
||||
|
||||
// Simplified direct-API approach (bypass MCP, hit OmniRoute APIs directly)
|
||||
// Useful if you don't need MCP protocol framing.
|
||||
|
||||
func callTool(baseURL, tool string, args map[string]any) (string, error) {
|
||||
// MCP tools map to OmniRoute APIs:
|
||||
endpoints := map[string]string{
|
||||
"health": "/api/monitoring/health",
|
||||
"combos": "/api/combos",
|
||||
"quota": "/api/usage/quota",
|
||||
"models": "/v1/models",
|
||||
}
|
||||
|
||||
url := baseURL + endpoints[tool]
|
||||
resp, err := http.Get(url)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
body, _ := io.ReadAll(resp.Body)
|
||||
return string(body), nil
|
||||
}
|
||||
|
||||
func routeRequest(baseURL, model, prompt string) (string, error) {
|
||||
payload := map[string]any{
|
||||
"model": model,
|
||||
"messages": []map[string]string{
|
||||
{"role": "user", "content": prompt},
|
||||
},
|
||||
"stream": false,
|
||||
}
|
||||
data, _ := json.Marshal(payload)
|
||||
|
||||
resp, err := http.Post(
|
||||
baseURL+"/v1/chat/completions",
|
||||
"application/json",
|
||||
bytes.NewReader(data),
|
||||
)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
body, _ := io.ReadAll(resp.Body)
|
||||
return string(body), nil
|
||||
}
|
||||
|
||||
func main() {
|
||||
base := "http://localhost:20128"
|
||||
|
||||
health, _ := callTool(base, "health", nil)
|
||||
fmt.Println("Health:", health)
|
||||
|
||||
result, _ := routeRequest(base, "auto", "Hello from Go!")
|
||||
fmt.Println("Result:", result)
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Use Cases
|
||||
|
||||
### 🔄 Use Case 1: Auto-Healing Agent
|
||||
|
||||
An agent that monitors OmniRoute health and auto-switches combos when providers degrade.
|
||||
|
||||
```python
|
||||
async def auto_healing_loop(session):
|
||||
"""Monitor health and react to provider issues."""
|
||||
while True:
|
||||
# Check health
|
||||
health = await session.call_tool("omniroute_get_health", {})
|
||||
data = json.loads(health.content[0].text)
|
||||
|
||||
# Find providers with open circuit breakers
|
||||
broken = [
|
||||
cb for cb in data["circuitBreakers"]
|
||||
if cb["state"] == "OPEN"
|
||||
]
|
||||
|
||||
if broken:
|
||||
# Switch to a different resilience profile
|
||||
await session.call_tool("omniroute_set_resilience_profile", {
|
||||
"profile": "conservative"
|
||||
})
|
||||
|
||||
# Find best alternative combo
|
||||
best = await session.call_tool("omniroute_best_combo_for_task", {
|
||||
"taskType": "coding"
|
||||
})
|
||||
best_data = json.loads(best.content[0].text)
|
||||
combo_id = best_data["recommendedCombo"]["id"]
|
||||
|
||||
# Activate it
|
||||
await session.call_tool("omniroute_switch_combo", {
|
||||
"comboId": combo_id, "active": True
|
||||
})
|
||||
print(f"⚠️ Auto-healed: switched to {combo_id}")
|
||||
|
||||
await asyncio.sleep(30) # Check every 30 seconds
|
||||
```
|
||||
|
||||
### 💰 Use Case 2: Budget-Aware Coding Agent
|
||||
|
||||
An agent that monitors costs in real-time and degrades to cheaper models when nearing budget.
|
||||
|
||||
```python
|
||||
async def budget_aware_coding(session, task: str, max_budget: float):
|
||||
"""Complete a coding task within a budget."""
|
||||
# Set budget guard
|
||||
await session.call_tool("omniroute_set_budget_guard", {
|
||||
"maxCost": max_budget,
|
||||
"action": "degrade",
|
||||
"degradeToTier": "cheap",
|
||||
})
|
||||
|
||||
# Simulate first to estimate cost
|
||||
sim = await session.call_tool("omniroute_simulate_route", {
|
||||
"model": "claude-sonnet-4",
|
||||
"promptTokenEstimate": len(task.split()) * 2,
|
||||
})
|
||||
sim_data = json.loads(sim.content[0].text)
|
||||
estimated_cost = sim_data["fallbackTree"]["bestCaseCost"]
|
||||
print(f"Estimated cost: ${estimated_cost:.4f}")
|
||||
|
||||
# Send request
|
||||
result = await session.call_tool("omniroute_route_request", {
|
||||
"model": "claude-sonnet-4",
|
||||
"messages": [{"role": "user", "content": task}],
|
||||
"role": "coding",
|
||||
})
|
||||
|
||||
# Check remaining budget
|
||||
snapshot = await session.call_tool("omniroute_get_session_snapshot", {})
|
||||
snap_data = json.loads(snapshot.content[0].text)
|
||||
print(f"Session cost: ${snap_data['costTotal']:.4f}")
|
||||
if snap_data.get("budgetGuard"):
|
||||
print(f"Budget remaining: ${snap_data['budgetGuard']['remaining']:.4f}")
|
||||
|
||||
return json.loads(result.content[0].text)["response"]["content"]
|
||||
```
|
||||
|
||||
### 🧪 Use Case 3: Combo Benchmarking Agent
|
||||
|
||||
An agent that periodically benchmarks all combos and reports the fastest/cheapest.
|
||||
|
||||
```python
|
||||
async def benchmark_combos(session):
|
||||
"""Benchmark all enabled combos and rank them."""
|
||||
combos = await session.call_tool("omniroute_list_combos", {
|
||||
"includeMetrics": True,
|
||||
})
|
||||
combo_list = json.loads(combos.content[0].text)["combos"]
|
||||
|
||||
results = []
|
||||
for combo in combo_list:
|
||||
if not combo["enabled"]:
|
||||
continue
|
||||
|
||||
test = await session.call_tool("omniroute_test_combo", {
|
||||
"comboId": combo["id"],
|
||||
"testPrompt": "Return the number 42.",
|
||||
})
|
||||
test_data = json.loads(test.content[0].text)
|
||||
results.append({
|
||||
"combo": combo["name"],
|
||||
"fastest": test_data["summary"]["fastestProvider"],
|
||||
"cheapest": test_data["summary"]["cheapestProvider"],
|
||||
"success_rate": f'{test_data["summary"]["successful"]}/{test_data["summary"]["totalProviders"]}',
|
||||
})
|
||||
|
||||
print("📊 Combo Benchmark Results:")
|
||||
for r in results:
|
||||
print(f" {r['combo']}: fastest={r['fastest']}, cheapest={r['cheapest']}, success={r['success_rate']}")
|
||||
```
|
||||
|
||||
### 🔍 Use Case 4: Post-Mortem Debugging Agent
|
||||
|
||||
An agent that explains why a request was routed to a specific provider.
|
||||
|
||||
```typescript
|
||||
async function debugRouting(client: Client, requestId: string) {
|
||||
// Explain the routing decision
|
||||
const explanation = await client.callTool({
|
||||
name: "omniroute_explain_route",
|
||||
arguments: { requestId },
|
||||
});
|
||||
const data = JSON.parse(explanation.content[0].text);
|
||||
|
||||
console.log(`Request ${requestId}:`);
|
||||
console.log(` Provider: ${data.decision.providerSelected}`);
|
||||
console.log(` Model: ${data.decision.modelUsed}`);
|
||||
console.log(` Score: ${data.decision.score}`);
|
||||
console.log(` Factors:`);
|
||||
for (const factor of data.decision.factors) {
|
||||
console.log(` ${factor.name}: ${factor.value} (weight: ${factor.weight})`);
|
||||
}
|
||||
if (data.decision.fallbacksTriggered.length > 0) {
|
||||
console.log(` Fallbacks triggered:`);
|
||||
for (const fb of data.decision.fallbacksTriggered) {
|
||||
console.log(` ${fb.provider}: ${fb.reason}`);
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 📋 Use Case 5: Model Discovery Agent
|
||||
|
||||
An agent that discovers the cheapest models for a given capability.
|
||||
|
||||
```python
|
||||
async def find_cheapest_models(session, capability="chat"):
|
||||
"""Find the cheapest available models for a capability."""
|
||||
catalog = await session.call_tool("omniroute_list_models_catalog", {
|
||||
"capability": capability,
|
||||
})
|
||||
models = json.loads(catalog.content[0].text)["models"]
|
||||
|
||||
# Filter available models with pricing
|
||||
priced = [
|
||||
m for m in models
|
||||
if m["status"] == "available" and m.get("pricing")
|
||||
]
|
||||
priced.sort(key=lambda m: m["pricing"]["inputPerMillion"] or float("inf"))
|
||||
|
||||
print(f"💡 Cheapest {capability} models:")
|
||||
for m in priced[:5]:
|
||||
input_cost = m["pricing"]["inputPerMillion"] or 0
|
||||
output_cost = m["pricing"]["outputPerMillion"] or 0
|
||||
print(f" {m['id']} ({m['provider']}): ${input_cost}/M in, ${output_cost}/M out")
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Security & Scope Enforcement
|
||||
|
||||
The MCP server supports **fine-grained scope enforcement** for multi-tenant environments:
|
||||
|
||||
| Scope | Tools |
|
||||
| --------------------- | ---------------------------------------------------------------------------------------------- |
|
||||
| `read:health` | `get_health`, `simulate_route`, `get_provider_metrics`, `best_combo_for_task`, `explain_route` |
|
||||
| `read:combos` | `list_combos`, `get_combo_metrics`, `simulate_route`, `best_combo_for_task`, `test_combo` |
|
||||
| `read:quota` | `check_quota` |
|
||||
| `read:usage` | `cost_report`, `explain_route`, `get_session_snapshot` |
|
||||
| `read:models` | `list_models_catalog` |
|
||||
| `write:combos` | `switch_combo` |
|
||||
| `write:budget` | `set_budget_guard` |
|
||||
| `write:resilience` | `set_resilience_profile` |
|
||||
| `execute:completions` | `route_request`, `test_combo` |
|
||||
|
||||
**Wildcard scopes:** Use `read:*` to grant all read scopes, or `*` for full access.
|
||||
|
||||
---
|
||||
|
||||
## Audit Logging
|
||||
|
||||
Every tool call is logged to the `mcp_tool_audit` SQLite table:
|
||||
|
||||
- **Input:** SHA-256 hashed (never stores raw prompts)
|
||||
- **Output:** Truncated to 200 chars
|
||||
- **Metadata:** Tool name, duration, success/error, API key ID
|
||||
|
||||
Access audit data via:
|
||||
|
||||
```typescript
|
||||
import { getRecentAuditEntries, getAuditStats } from "./audit";
|
||||
|
||||
const entries = await getRecentAuditEntries(50);
|
||||
const stats = await getAuditStats();
|
||||
// stats: { totalCalls, successRate, avgDurationMs, topTools }
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## File Structure
|
||||
|
||||
```
|
||||
mcp-server/
|
||||
├── server.ts # MCP server setup, essential tool handlers, entry point
|
||||
├── index.ts # Barrel export
|
||||
├── audit.ts # SQLite audit logger (SHA-256 input hashing)
|
||||
├── scopeEnforcement.ts # Fine-grained scope enforcement
|
||||
├── schemas/
|
||||
│ ├── tools.ts # Zod schemas for all 16 tools (input/output/scopes)
|
||||
│ ├── a2a.ts # A2A protocol types (Agent Card, Task, JSON-RPC)
|
||||
│ ├── audit.ts # Audit & routing decision types + hash helpers
|
||||
│ └── index.ts # Schema barrel export
|
||||
├── tools/
|
||||
│ └── advancedTools.ts # Phase 2 tool handlers (8 advanced tools)
|
||||
└── __tests__/
|
||||
├── essentialTools.test.ts
|
||||
├── advancedTools.test.ts
|
||||
└── a2aLifecycle.test.ts
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## License
|
||||
|
||||
Part of [OmniRoute](https://github.com/diegosouzapw/OmniRoute) — MIT License.
|
||||
@@ -53,4 +53,33 @@ describe("A2A task lifecycle regressions", () => {
|
||||
expect(loaded?.state).toBe("failed");
|
||||
expect(loaded?.artifacts.at(-1)).toEqual({ type: "error", content: "upstream failure" });
|
||||
});
|
||||
|
||||
it("transitions expired submitted tasks to failed without throwing", () => {
|
||||
const tm = createManager();
|
||||
const task = tm.createTask({
|
||||
skill: "smart-routing",
|
||||
messages: [{ role: "user", content: "hello" }],
|
||||
});
|
||||
task.expiresAt = new Date(Date.now() - 1_000).toISOString();
|
||||
|
||||
expect(() => tm.getTask(task.id)).not.toThrow();
|
||||
const loaded = tm.getTask(task.id);
|
||||
expect(loaded?.state).toBe("failed");
|
||||
});
|
||||
|
||||
it("does not rewrite cancelled tasks to failed during cleanup", () => {
|
||||
const tm = createManager();
|
||||
const task = tm.createTask({
|
||||
skill: "smart-routing",
|
||||
messages: [{ role: "user", content: "cancel me" }],
|
||||
});
|
||||
tm.updateTask(task.id, "cancelled");
|
||||
task.expiresAt = new Date(Date.now() - 1_000).toISOString();
|
||||
|
||||
// private in TS only; callable at runtime for regression test
|
||||
(tm as any).cleanupExpired();
|
||||
|
||||
const loaded = tm.getTask(task.id);
|
||||
expect(loaded?.state).toBe("cancelled");
|
||||
});
|
||||
});
|
||||
|
||||
@@ -49,7 +49,7 @@ import {
|
||||
handleExplainRoute,
|
||||
handleGetSessionSnapshot,
|
||||
} from "./tools/advancedTools.ts";
|
||||
import { normalizeQuotaResponse } from "@/shared/contracts/quota";
|
||||
import { normalizeQuotaResponse } from "../../src/shared/contracts/quota.ts";
|
||||
|
||||
// ============ Configuration ============
|
||||
|
||||
@@ -385,8 +385,15 @@ async function handleCostReport(args: { period?: string }) {
|
||||
const start = Date.now();
|
||||
try {
|
||||
const period = args.period || "session";
|
||||
const rangeMap: Record<string, string> = {
|
||||
session: "1d",
|
||||
day: "1d",
|
||||
week: "7d",
|
||||
month: "30d",
|
||||
};
|
||||
const range = rangeMap[period] || "30d";
|
||||
const raw = toRecord(
|
||||
await omniRouteFetch(`/api/usage/analytics?period=${encodeURIComponent(period)}`)
|
||||
await omniRouteFetch(`/api/usage/analytics?range=${encodeURIComponent(range)}`)
|
||||
);
|
||||
const tokenCount = toRecord(raw.tokenCount);
|
||||
const budget = toRecord(raw.budget);
|
||||
|
||||
@@ -14,7 +14,7 @@
|
||||
*/
|
||||
|
||||
import { logToolCall } from "../audit.ts";
|
||||
import { normalizeQuotaResponse } from "@/shared/contracts/quota";
|
||||
import { normalizeQuotaResponse } from "../../../src/shared/contracts/quota.ts";
|
||||
|
||||
const OMNIROUTE_BASE_URL = process.env.OMNIROUTE_BASE_URL || "http://localhost:20128";
|
||||
const OMNIROUTE_API_KEY = process.env.OMNIROUTE_API_KEY || "";
|
||||
|
||||
@@ -12,6 +12,7 @@ export function antigravityToOpenAIRequest(model, body, stream) {
|
||||
model: string;
|
||||
messages: JsonRecord[];
|
||||
stream: unknown;
|
||||
tools?: JsonRecord[];
|
||||
[key: string]: unknown;
|
||||
} = {
|
||||
model: model,
|
||||
|
||||
@@ -362,14 +362,16 @@ function openaiToClaudeRequestForAntigravity(model, body, stream) {
|
||||
}
|
||||
|
||||
const updatedContent = msg.content.map((block) => {
|
||||
const blockType = typeof block.type === "string" ? block.type : "";
|
||||
const blockName = typeof block.name === "string" ? block.name : "";
|
||||
if (
|
||||
block.type === "tool_use" &&
|
||||
block.name &&
|
||||
block.name.startsWith(CLAUDE_OAUTH_TOOL_PREFIX)
|
||||
blockType === "tool_use" &&
|
||||
blockName &&
|
||||
blockName.startsWith(CLAUDE_OAUTH_TOOL_PREFIX)
|
||||
) {
|
||||
return {
|
||||
...block,
|
||||
name: block.name.slice(CLAUDE_OAUTH_TOOL_PREFIX.length),
|
||||
name: blockName.slice(CLAUDE_OAUTH_TOOL_PREFIX.length),
|
||||
};
|
||||
}
|
||||
return block;
|
||||
|
||||
@@ -150,7 +150,7 @@ export function claudeToOpenAIResponse(chunk, state) {
|
||||
model: string;
|
||||
choices: Array<{
|
||||
index: number;
|
||||
delta: { content: string };
|
||||
delta: { content?: string };
|
||||
finish_reason: string | null;
|
||||
}>;
|
||||
usage?: OpenAIUsage;
|
||||
|
||||
@@ -15,6 +15,13 @@ type SocksDispatcherOptions = {
|
||||
userId?: string;
|
||||
password?: string;
|
||||
};
|
||||
type ProxyConfigObject = {
|
||||
type?: string;
|
||||
host?: string;
|
||||
port?: string | number | null;
|
||||
username?: string;
|
||||
password?: string;
|
||||
};
|
||||
|
||||
function getDispatcherCache(): DispatcherCache {
|
||||
const globalWithCache = globalThis as GlobalWithDispatcherCache;
|
||||
@@ -38,7 +45,7 @@ export function clearDispatcherCache() {
|
||||
* `new URL("http://host:80")` strips port 80 since it's the HTTP default,
|
||||
* but proxy servers commonly listen on port 80/443, so we need to preserve it.
|
||||
*/
|
||||
function extractExplicitPort(urlStr) {
|
||||
function extractExplicitPort(urlStr: string): string | null {
|
||||
try {
|
||||
// Match port in the host portion: "scheme://[user:pass@]host:PORT[/...]"
|
||||
const match = urlStr.match(/:\/\/(?:[^@]*@)?[^:/\s]+:(\d+)/);
|
||||
@@ -50,13 +57,13 @@ function extractExplicitPort(urlStr) {
|
||||
return null;
|
||||
}
|
||||
|
||||
function defaultPortForProtocol(protocol) {
|
||||
function defaultPortForProtocol(protocol: string): string {
|
||||
if (protocol === "https:" || protocol === "wss:") return "443";
|
||||
if (protocol === "socks5:") return "1080";
|
||||
return "8080";
|
||||
}
|
||||
|
||||
function normalizePort(port, protocol) {
|
||||
function normalizePort(port: string | number | null | undefined, protocol: string): string {
|
||||
if (!port) return defaultPortForProtocol(protocol);
|
||||
const parsed = Number(port);
|
||||
if (!Number.isInteger(parsed) || parsed < 1 || parsed > 65535) {
|
||||
@@ -71,18 +78,18 @@ function normalizePort(port, protocol) {
|
||||
* default ports (80 for http, 443 for https). Proxy servers commonly
|
||||
* listen on these ports, so we must always include the port explicitly.
|
||||
*/
|
||||
function buildProxyUrlString(parsed, port) {
|
||||
function buildProxyUrlString(parsed: URL, port: string): string {
|
||||
const auth = parsed.username
|
||||
? `${parsed.username}${parsed.password ? `:${parsed.password}` : ""}@`
|
||||
: "";
|
||||
return `${parsed.protocol}//${auth}${parsed.hostname}:${port}`;
|
||||
}
|
||||
|
||||
export function isSocks5ProxyEnabled() {
|
||||
export function isSocks5ProxyEnabled(): boolean {
|
||||
return process.env.ENABLE_SOCKS5_PROXY === "true";
|
||||
}
|
||||
|
||||
export function proxyUrlForLogs(proxyUrl) {
|
||||
export function proxyUrlForLogs(proxyUrl: string): string {
|
||||
const explicitPort = extractExplicitPort(proxyUrl);
|
||||
const parsed = new URL(proxyUrl);
|
||||
const port = explicitPort || parsed.port || defaultPortForProtocol(parsed.protocol);
|
||||
@@ -90,10 +97,10 @@ export function proxyUrlForLogs(proxyUrl) {
|
||||
}
|
||||
|
||||
export function normalizeProxyUrl(
|
||||
proxyUrl,
|
||||
proxyUrl: string,
|
||||
source = "proxy",
|
||||
{ allowSocks5 = isSocks5ProxyEnabled() } = {}
|
||||
) {
|
||||
): string {
|
||||
// Extract the explicit port from the raw URL string BEFORE parsing,
|
||||
// because `new URL()` silently strips default ports (80 for http,
|
||||
// 443 for https), which are valid and common for proxy servers.
|
||||
@@ -128,7 +135,10 @@ export function normalizeProxyUrl(
|
||||
return buildProxyUrlString(parsed, port);
|
||||
}
|
||||
|
||||
export function proxyConfigToUrl(proxyConfig, { allowSocks5 = isSocks5ProxyEnabled() } = {}) {
|
||||
export function proxyConfigToUrl(
|
||||
proxyConfig: unknown,
|
||||
{ allowSocks5 = isSocks5ProxyEnabled() } = {}
|
||||
): string | null {
|
||||
if (!proxyConfig) return null;
|
||||
|
||||
if (typeof proxyConfig === "string") {
|
||||
@@ -139,7 +149,8 @@ export function proxyConfigToUrl(proxyConfig, { allowSocks5 = isSocks5ProxyEnabl
|
||||
throw new Error("[ProxyDispatcher] Invalid context proxy config");
|
||||
}
|
||||
|
||||
const type = String(proxyConfig.type || "http").toLowerCase();
|
||||
const config = proxyConfig as ProxyConfigObject;
|
||||
const type = String(config.type || "http").toLowerCase();
|
||||
const protocol = `${type}:`;
|
||||
|
||||
if (!SUPPORTED_PROTOCOLS.has(protocol)) {
|
||||
@@ -150,23 +161,23 @@ export function proxyConfigToUrl(proxyConfig, { allowSocks5 = isSocks5ProxyEnabl
|
||||
"[ProxyDispatcher] SOCKS5 proxy is disabled (set ENABLE_SOCKS5_PROXY=true to enable)"
|
||||
);
|
||||
}
|
||||
if (!proxyConfig.host) {
|
||||
if (!config.host) {
|
||||
throw new Error("[ProxyDispatcher] Context proxy host is required");
|
||||
}
|
||||
|
||||
const port = normalizePort(proxyConfig.port, protocol);
|
||||
const port = normalizePort(config.port, protocol);
|
||||
|
||||
// Build the URL string manually to preserve the port through normalization.
|
||||
const auth = proxyConfig.username
|
||||
? `${encodeURIComponent(proxyConfig.username)}:${proxyConfig.password ? encodeURIComponent(proxyConfig.password) : ""}@`
|
||||
const auth = config.username
|
||||
? `${encodeURIComponent(config.username)}:${config.password ? encodeURIComponent(config.password) : ""}@`
|
||||
: "";
|
||||
|
||||
const proxyUrlStr = `${type}://${auth}${proxyConfig.host}:${port}`;
|
||||
const proxyUrlStr = `${type}://${auth}${config.host}:${port}`;
|
||||
|
||||
return normalizeProxyUrl(proxyUrlStr, "context proxy", { allowSocks5 });
|
||||
}
|
||||
|
||||
export function createProxyDispatcher(proxyUrl) {
|
||||
export function createProxyDispatcher(proxyUrl: string): Dispatcher {
|
||||
const normalizedUrl = normalizeProxyUrl(proxyUrl, "proxy dispatcher");
|
||||
const dispatcherCache = getDispatcherCache();
|
||||
|
||||
|
||||
@@ -16,6 +16,10 @@ type TlsFingerprintStore = { used: boolean };
|
||||
const tlsFingerprintContext = new AsyncLocalStorage<TlsFingerprintStore>();
|
||||
|
||||
type FetchWithDispatcherOptions = RequestInit & { dispatcher?: unknown };
|
||||
type FetchWithDispatcher = (
|
||||
input: RequestInfo | URL,
|
||||
init?: FetchWithDispatcherOptions
|
||||
) => Promise<Response>;
|
||||
|
||||
type PatchState = {
|
||||
originalFetch: typeof globalThis.fetch;
|
||||
@@ -43,6 +47,7 @@ function getPatchState(): PatchState {
|
||||
|
||||
const patchState = getPatchState();
|
||||
const originalFetch = patchState.originalFetch;
|
||||
const originalFetchWithDispatcher = originalFetch as FetchWithDispatcher;
|
||||
const proxyContext = patchState.proxyContext;
|
||||
|
||||
function noProxyMatch(targetUrl) {
|
||||
@@ -141,7 +146,7 @@ export async function runWithProxyContext(proxyConfig, fn) {
|
||||
|
||||
async function patchedFetch(input: RequestInfo | URL, options: FetchWithDispatcherOptions = {}) {
|
||||
if (options?.dispatcher) {
|
||||
return originalFetch(input, options);
|
||||
return originalFetchWithDispatcher(input, options);
|
||||
}
|
||||
|
||||
const targetUrl = getTargetUrl(input);
|
||||
@@ -161,7 +166,10 @@ async function patchedFetch(input: RequestInfo | URL, options: FetchWithDispatch
|
||||
try {
|
||||
const store = tlsFingerprintContext.getStore();
|
||||
if (store) store.used = true;
|
||||
return await tlsClient.fetch(targetUrl, options);
|
||||
return await tlsClient.fetch(targetUrl, {
|
||||
...options,
|
||||
headers: options.headers,
|
||||
});
|
||||
} catch (error) {
|
||||
const message = error instanceof Error ? error.message : String(error);
|
||||
console.warn(
|
||||
@@ -171,12 +179,12 @@ async function patchedFetch(input: RequestInfo | URL, options: FetchWithDispatch
|
||||
if (store) store.used = false;
|
||||
}
|
||||
}
|
||||
return originalFetch(input, options);
|
||||
return originalFetchWithDispatcher(input, options);
|
||||
}
|
||||
|
||||
try {
|
||||
const dispatcher = createProxyDispatcher(proxyUrl);
|
||||
return await originalFetch(input, { ...options, dispatcher });
|
||||
return await originalFetchWithDispatcher(input, { ...options, dispatcher });
|
||||
} catch (error) {
|
||||
const message = error instanceof Error ? error.message : String(error);
|
||||
console.error(`[ProxyFetch] Proxy request failed (${source}, fail-closed): ${message}`);
|
||||
|
||||
@@ -35,12 +35,32 @@ function getProxyFromEnv(): string | undefined {
|
||||
|
||||
interface FetchOptions {
|
||||
method?: string;
|
||||
headers?: Record<string, string>;
|
||||
headers?: HeadersInit;
|
||||
body?: unknown;
|
||||
redirect?: string;
|
||||
signal?: AbortSignal;
|
||||
}
|
||||
|
||||
function normalizeHeaders(headers: HeadersInit | undefined): Record<string, string> | undefined {
|
||||
if (!headers) return undefined;
|
||||
|
||||
if (headers instanceof Headers) {
|
||||
return Object.fromEntries(headers.entries());
|
||||
}
|
||||
|
||||
if (Array.isArray(headers)) {
|
||||
return Object.fromEntries(headers.map(([key, value]) => [key, String(value)]));
|
||||
}
|
||||
|
||||
const normalized: Record<string, string> = {};
|
||||
for (const [key, value] of Object.entries(headers)) {
|
||||
if (typeof value === "string") {
|
||||
normalized[key] = value;
|
||||
}
|
||||
}
|
||||
return normalized;
|
||||
}
|
||||
|
||||
/**
|
||||
* TLS Client — Chrome 124 TLS fingerprint spoofing via wreq-js
|
||||
* Singleton instance used to disguise Node.js TLS handshake as Chrome browser.
|
||||
@@ -87,7 +107,7 @@ class TlsClient {
|
||||
|
||||
const wreqOptions: Record<string, unknown> = {
|
||||
method,
|
||||
headers: options.headers,
|
||||
headers: normalizeHeaders(options.headers),
|
||||
body: options.body,
|
||||
redirect: options.redirect === "manual" ? "manual" : "follow",
|
||||
};
|
||||
|
||||
6
package-lock.json
generated
6
package-lock.json
generated
@@ -36,6 +36,7 @@
|
||||
"react-dom": "19.2.4",
|
||||
"recharts": "^3.7.0",
|
||||
"selfsigned": "^5.5.0",
|
||||
"tsx": "^4.21.0",
|
||||
"undici": "^7.19.2",
|
||||
"uuid": "^13.0.0",
|
||||
"wreq-js": "^2.0.1",
|
||||
@@ -62,7 +63,6 @@
|
||||
"lint-staged": "^16.2.7",
|
||||
"prettier": "^3.8.1",
|
||||
"tailwindcss": "^4",
|
||||
"tsx": "^4.21.0",
|
||||
"typescript": "^5.9.3",
|
||||
"typescript-eslint": "^8.56.0",
|
||||
"vitest": "^3.2.4",
|
||||
@@ -6023,7 +6023,6 @@
|
||||
"version": "0.27.3",
|
||||
"resolved": "https://registry.npmjs.org/esbuild/-/esbuild-0.27.3.tgz",
|
||||
"integrity": "sha512-8VwMnyGCONIs6cWue2IdpHxHnAjzxnw2Zr7MkVxB2vjmQ2ivqGFb4LEG3SMnv0Gb2F/G/2yA8zUaiL1gywDCCg==",
|
||||
"dev": true,
|
||||
"hasInstallScript": true,
|
||||
"license": "MIT",
|
||||
"bin": {
|
||||
@@ -7093,7 +7092,6 @@
|
||||
"version": "4.13.6",
|
||||
"resolved": "https://registry.npmjs.org/get-tsconfig/-/get-tsconfig-4.13.6.tgz",
|
||||
"integrity": "sha512-shZT/QMiSHc/YBLxxOkMtgSid5HFoauqCE3/exfsEcwg1WkeqjG+V40yBbBrsD+jW2HDXcs28xOfcbm2jI8Ddw==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"resolve-pkg-maps": "^1.0.0"
|
||||
@@ -11418,7 +11416,6 @@
|
||||
"version": "4.21.0",
|
||||
"resolved": "https://registry.npmjs.org/tsx/-/tsx-4.21.0.tgz",
|
||||
"integrity": "sha512-5C1sg4USs1lfG0GFb2RLXsdpXqBSEhAaA/0kPL01wxzpMqLILNxIxIOKiILz+cdg/pLnOUxFYOR5yhHU666wbw==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"esbuild": "~0.27.0",
|
||||
@@ -11438,7 +11435,6 @@
|
||||
"version": "2.3.3",
|
||||
"resolved": "https://registry.npmjs.org/fsevents/-/fsevents-2.3.3.tgz",
|
||||
"integrity": "sha512-5xoDfX+fL7faATnagmWPpbFtwh/R77WmMMqqHGS65C3vvB0YHrgF+B1YmZ3441tMj5n63k0212XNoJwzlhffQw==",
|
||||
"dev": true,
|
||||
"hasInstallScript": true,
|
||||
"license": "MIT",
|
||||
"optional": true,
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "omniroute",
|
||||
"version": "1.8.1",
|
||||
"version": "2.0.0",
|
||||
"description": "Smart AI Router with auto fallback — route to FREE & cheap models, zero downtime. Works with Cursor, Cline, Claude Desktop, Codex, and any OpenAI-compatible tool.",
|
||||
"type": "module",
|
||||
"bin": {
|
||||
@@ -10,6 +10,8 @@
|
||||
"files": [
|
||||
"bin/",
|
||||
"app/",
|
||||
"open-sse/mcp-server/",
|
||||
"src/shared/contracts/",
|
||||
"scripts/postinstall.mjs",
|
||||
"README.md",
|
||||
"LICENSE"
|
||||
@@ -99,6 +101,7 @@
|
||||
"react-dom": "19.2.4",
|
||||
"recharts": "^3.7.0",
|
||||
"selfsigned": "^5.5.0",
|
||||
"tsx": "^4.21.0",
|
||||
"undici": "^7.19.2",
|
||||
"uuid": "^13.0.0",
|
||||
"wreq-js": "^2.0.1",
|
||||
@@ -121,7 +124,6 @@
|
||||
"lint-staged": "^16.2.7",
|
||||
"prettier": "^3.8.1",
|
||||
"tailwindcss": "^4",
|
||||
"tsx": "^4.21.0",
|
||||
"typescript": "^5.9.3",
|
||||
"typescript-eslint": "^8.56.0",
|
||||
"vitest": "^3.2.4",
|
||||
|
||||
@@ -22,6 +22,21 @@ interface ExclusionEntry {
|
||||
reason: string;
|
||||
}
|
||||
|
||||
type AutoComboRecord = {
|
||||
candidatePool?: unknown;
|
||||
weights?: unknown;
|
||||
};
|
||||
|
||||
type HealthRecord = {
|
||||
providerHealth?: Record<string, { state?: string; lastFailure?: string | null }>;
|
||||
circuitBreakers?: Array<{
|
||||
provider?: string;
|
||||
name?: string;
|
||||
state?: string;
|
||||
lastFailure?: string | null;
|
||||
}>;
|
||||
};
|
||||
|
||||
export default function AutoComboDashboard() {
|
||||
const [scores, setScores] = useState<ProviderScore[]>([]);
|
||||
const [exclusions, setExclusions] = useState<ExclusionEntry[]>([]);
|
||||
@@ -35,11 +50,75 @@ export default function AutoComboDashboard() {
|
||||
fetch("/api/monitoring/health"),
|
||||
]);
|
||||
|
||||
if (combosRes.status === "fulfilled") {
|
||||
const comboPayload = await combosRes.value.json();
|
||||
const combos = Array.isArray(comboPayload?.combos)
|
||||
? (comboPayload.combos as AutoComboRecord[])
|
||||
: [];
|
||||
const firstCombo = combos[0] || null;
|
||||
const candidatePool = Array.isArray(firstCombo?.candidatePool)
|
||||
? firstCombo.candidatePool.filter((entry): entry is string => typeof entry === "string")
|
||||
: [];
|
||||
const rawWeights =
|
||||
firstCombo?.weights &&
|
||||
typeof firstCombo.weights === "object" &&
|
||||
!Array.isArray(firstCombo.weights)
|
||||
? (firstCombo.weights as Record<string, unknown>)
|
||||
: {};
|
||||
const factors = Object.fromEntries(
|
||||
Object.entries(rawWeights).map(([k, v]) => [k, typeof v === "number" ? v : 0])
|
||||
);
|
||||
const baseScore = candidatePool.length > 0 ? 1 / candidatePool.length : 0;
|
||||
setScores(
|
||||
candidatePool.map((provider) => ({
|
||||
provider,
|
||||
model: "auto",
|
||||
score: baseScore,
|
||||
factors,
|
||||
}))
|
||||
);
|
||||
} else {
|
||||
setScores([]);
|
||||
}
|
||||
|
||||
if (healthRes.status === "fulfilled") {
|
||||
const health = await healthRes.value.json();
|
||||
const breakers = health?.circuitBreakers || [];
|
||||
const openCount = breakers.filter((b: any) => b.state === "OPEN").length;
|
||||
setIncidentMode(openCount / Math.max(breakers.length, 1) > 0.5);
|
||||
const health = (await healthRes.value.json()) as HealthRecord;
|
||||
const providerHealth =
|
||||
health?.providerHealth && typeof health.providerHealth === "object"
|
||||
? health.providerHealth
|
||||
: {};
|
||||
const breakersFromProviderHealth = Object.entries(providerHealth).map(
|
||||
([provider, status]) => ({
|
||||
provider,
|
||||
state: status?.state || "CLOSED",
|
||||
lastFailure: status?.lastFailure || null,
|
||||
})
|
||||
);
|
||||
const breakersFromArray = Array.isArray(health?.circuitBreakers)
|
||||
? health.circuitBreakers
|
||||
: [];
|
||||
const breakers =
|
||||
breakersFromArray.length > 0
|
||||
? breakersFromArray.map((breaker) => ({
|
||||
provider: breaker.provider || breaker.name || "unknown",
|
||||
state: breaker.state || "CLOSED",
|
||||
lastFailure: breaker.lastFailure || null,
|
||||
}))
|
||||
: breakersFromProviderHealth;
|
||||
|
||||
const openBreakers = breakers.filter((breaker) => breaker.state === "OPEN");
|
||||
setIncidentMode(openBreakers.length / Math.max(breakers.length, 1) > 0.5);
|
||||
setExclusions(
|
||||
openBreakers.map((breaker) => ({
|
||||
provider: breaker.provider,
|
||||
excludedAt: breaker.lastFailure || new Date().toISOString(),
|
||||
cooldownMs: 5 * 60 * 1000,
|
||||
reason: "Circuit breaker OPEN",
|
||||
}))
|
||||
);
|
||||
} else {
|
||||
setIncidentMode(false);
|
||||
setExclusions([]);
|
||||
}
|
||||
} catch {
|
||||
/* ignore */
|
||||
|
||||
@@ -25,6 +25,51 @@ const SKILL_HANDLERS: Record<string, (task: any) => Promise<any>> = {
|
||||
"quota-management": executeQuotaManagement,
|
||||
};
|
||||
|
||||
type A2AMessage = { role: string; content: string };
|
||||
|
||||
function toMessageArray(raw: unknown): A2AMessage[] | null {
|
||||
if (Array.isArray(raw)) {
|
||||
const normalized = raw
|
||||
.map((entry) => {
|
||||
if (!entry || typeof entry !== "object" || Array.isArray(entry)) return null;
|
||||
const msg = entry as Record<string, unknown>;
|
||||
const role = typeof msg.role === "string" && msg.role.trim() ? msg.role : "user";
|
||||
const content = typeof msg.content === "string" ? msg.content : null;
|
||||
if (!content) return null;
|
||||
return { role, content };
|
||||
})
|
||||
.filter((entry): entry is A2AMessage => !!entry);
|
||||
return normalized.length > 0 ? normalized : null;
|
||||
}
|
||||
|
||||
if (!raw || typeof raw !== "object" || Array.isArray(raw)) return null;
|
||||
const message = raw as Record<string, unknown>;
|
||||
const role = typeof message.role === "string" && message.role.trim() ? message.role : "user";
|
||||
|
||||
// Canonical A2A shape: { message: { role, content } }
|
||||
if (typeof message.content === "string" && message.content.trim()) {
|
||||
return [{ role, content: message.content }];
|
||||
}
|
||||
|
||||
// Legacy compatibility: { message: { parts: [...] } }
|
||||
if (Array.isArray(message.parts)) {
|
||||
const text = message.parts
|
||||
.map((part) => {
|
||||
if (typeof part === "string") return part;
|
||||
if (!part || typeof part !== "object" || Array.isArray(part)) return "";
|
||||
const chunk = part as Record<string, unknown>;
|
||||
if (typeof chunk.content === "string") return chunk.content;
|
||||
if (typeof chunk.text === "string") return chunk.text;
|
||||
return "";
|
||||
})
|
||||
.filter((chunk) => chunk.trim().length > 0)
|
||||
.join("\n");
|
||||
if (text) return [{ role, content: text }];
|
||||
}
|
||||
|
||||
return null;
|
||||
}
|
||||
|
||||
// ============ Auth ============
|
||||
|
||||
function authenticate(req: NextRequest): boolean {
|
||||
@@ -77,9 +122,13 @@ export async function POST(req: NextRequest) {
|
||||
// ── message/send ──────────────────────────────────────
|
||||
case "message/send": {
|
||||
const skill = params?.skill || "smart-routing";
|
||||
const messages = params?.messages || params?.message?.parts;
|
||||
if (!messages || !Array.isArray(messages)) {
|
||||
return jsonRpcError(id, -32602, "Invalid params: messages array required");
|
||||
const messages = toMessageArray(params?.messages) || toMessageArray(params?.message);
|
||||
if (!messages) {
|
||||
return jsonRpcError(
|
||||
id,
|
||||
-32602,
|
||||
"Invalid params: provide `messages[]` or `message.content`"
|
||||
);
|
||||
}
|
||||
|
||||
const handler = SKILL_HANDLERS[skill];
|
||||
@@ -125,9 +174,13 @@ export async function POST(req: NextRequest) {
|
||||
// ── message/stream ────────────────────────────────────
|
||||
case "message/stream": {
|
||||
const skill = params?.skill || "smart-routing";
|
||||
const messages = params?.messages || params?.message?.parts;
|
||||
if (!messages || !Array.isArray(messages)) {
|
||||
return jsonRpcError(id, -32602, "Invalid params: messages array required");
|
||||
const messages = toMessageArray(params?.messages) || toMessageArray(params?.message);
|
||||
if (!messages) {
|
||||
return jsonRpcError(
|
||||
id,
|
||||
-32602,
|
||||
"Invalid params: provide `messages[]` or `message.content`"
|
||||
);
|
||||
}
|
||||
|
||||
const handler = SKILL_HANDLERS[skill];
|
||||
|
||||
@@ -28,10 +28,13 @@ export async function POST(request) {
|
||||
if (isValidationFailure(validation)) {
|
||||
return NextResponse.json({ error: validation.error }, { status: 400 });
|
||||
}
|
||||
const { password } = validation.data;
|
||||
const password = typeof validation.data.password === "string" ? validation.data.password : "";
|
||||
if (!password) {
|
||||
return NextResponse.json({ error: "Invalid password payload" }, { status: 400 });
|
||||
}
|
||||
const settings = await getSettings();
|
||||
|
||||
const storedHash = settings.password;
|
||||
const storedHash = typeof settings.password === "string" ? settings.password : "";
|
||||
|
||||
let isValid = false;
|
||||
if (storedHash) {
|
||||
|
||||
@@ -27,6 +27,10 @@ export async function POST(request) {
|
||||
return value.slice(0, 4) + "****" + value.slice(-4);
|
||||
}
|
||||
|
||||
function toOptionalString(value: unknown): string | null {
|
||||
return typeof value === "string" ? value : null;
|
||||
}
|
||||
|
||||
// Map connections — NEVER expose raw credentials
|
||||
const mappedConnections = connections.map((conn) => ({
|
||||
provider: conn.provider,
|
||||
@@ -34,7 +38,7 @@ export async function POST(request) {
|
||||
hasApiKey: !!conn.apiKey,
|
||||
hasAccessToken: !!conn.accessToken,
|
||||
hasRefreshToken: !!conn.refreshToken,
|
||||
maskedApiKey: maskSecret(conn.apiKey),
|
||||
maskedApiKey: maskSecret(toOptionalString(conn.apiKey)),
|
||||
projectId: conn.projectId || null,
|
||||
expiresAt: conn.expiresAt,
|
||||
priority: conn.priority,
|
||||
|
||||
@@ -45,7 +45,7 @@ export async function PUT(request: Request) {
|
||||
}
|
||||
|
||||
// Update credentials
|
||||
const updateData: Record<string, any> = {};
|
||||
const updateData: Record<string, unknown> = {};
|
||||
if (credentials.accessToken) {
|
||||
updateData.accessToken = credentials.accessToken;
|
||||
}
|
||||
@@ -56,7 +56,11 @@ export async function PUT(request: Request) {
|
||||
updateData.expiresAt = new Date(Date.now() + credentials.expiresIn * 1000).toISOString();
|
||||
}
|
||||
|
||||
await updateProviderConnection(connection.id, updateData);
|
||||
const connectionId = typeof connection.id === "string" ? connection.id : null;
|
||||
if (!connectionId) {
|
||||
return NextResponse.json({ error: "Invalid provider connection ID" }, { status: 500 });
|
||||
}
|
||||
await updateProviderConnection(connectionId, updateData);
|
||||
|
||||
return NextResponse.json({
|
||||
success: true,
|
||||
|
||||
@@ -35,7 +35,8 @@ export async function POST(request: Request) {
|
||||
|
||||
// Get model aliases
|
||||
const modelAliases = await getModelAliases();
|
||||
const resolved = modelAliases[alias];
|
||||
const resolvedValue = modelAliases[alias];
|
||||
const resolved = typeof resolvedValue === "string" ? resolvedValue : null;
|
||||
|
||||
if (resolved) {
|
||||
// Parse provider/model
|
||||
|
||||
@@ -24,9 +24,10 @@ export async function GET(request, { params }) {
|
||||
}
|
||||
|
||||
// Mask the key value
|
||||
const keyValue = typeof key.key === "string" ? key.key : null;
|
||||
return NextResponse.json({
|
||||
...key,
|
||||
key: key.key ? key.key.slice(0, 8) + "****" + key.key.slice(-4) : null,
|
||||
key: keyValue ? keyValue.slice(0, 8) + "****" + keyValue.slice(-4) : null,
|
||||
});
|
||||
} catch (error) {
|
||||
console.log("Error fetching key:", error);
|
||||
|
||||
@@ -11,7 +11,7 @@ export async function GET() {
|
||||
// Mask key values — users should never see full keys after creation
|
||||
const maskedKeys = keys.map((k) => ({
|
||||
...k,
|
||||
key: k.key ? k.key.slice(0, 8) + "****" + k.key.slice(-4) : null,
|
||||
key: typeof k.key === "string" ? k.key.slice(0, 8) + "****" + k.key.slice(-4) : null,
|
||||
}));
|
||||
return NextResponse.json({ keys: maskedKeys });
|
||||
} catch (error) {
|
||||
|
||||
@@ -225,8 +225,9 @@ export async function POST(
|
||||
const match = existing.find(
|
||||
(c: any) => c.email === tokenData.email && c.authType === "oauth"
|
||||
);
|
||||
if (match) {
|
||||
connection = await updateProviderConnection(match.id, {
|
||||
const matchId = typeof match?.id === "string" ? match.id : null;
|
||||
if (matchId) {
|
||||
connection = await updateProviderConnection(matchId, {
|
||||
...tokenData,
|
||||
expiresAt,
|
||||
testStatus: "active",
|
||||
@@ -288,8 +289,9 @@ export async function POST(
|
||||
const match = existing.find(
|
||||
(c: any) => c.email === result.tokens.email && c.authType === "oauth"
|
||||
);
|
||||
if (match) {
|
||||
connection = await updateProviderConnection(match.id, {
|
||||
const matchId = typeof match?.id === "string" ? match.id : null;
|
||||
if (matchId) {
|
||||
connection = await updateProviderConnection(matchId, {
|
||||
...result.tokens,
|
||||
expiresAt,
|
||||
testStatus: "active",
|
||||
@@ -401,8 +403,9 @@ export async function POST(
|
||||
const match = existing.find(
|
||||
(c: any) => c.email === tokenData.email && c.authType === "oauth"
|
||||
);
|
||||
if (match) {
|
||||
connection = await updateProviderConnection(match.id, {
|
||||
const matchId = typeof match?.id === "string" ? match.id : null;
|
||||
if (matchId) {
|
||||
connection = await updateProviderConnection(matchId, {
|
||||
...tokenData,
|
||||
expiresAt,
|
||||
testStatus: "active",
|
||||
|
||||
@@ -2,6 +2,20 @@ import { NextResponse } from "next/server";
|
||||
import { REGISTRY } from "@omniroute/open-sse/config/providerRegistry.ts";
|
||||
import { getAllCustomModels, getPricing } from "@/lib/localDb";
|
||||
|
||||
function asRecord(value: unknown): Record<string, unknown> {
|
||||
return value && typeof value === "object" && !Array.isArray(value)
|
||||
? (value as Record<string, unknown>)
|
||||
: {};
|
||||
}
|
||||
|
||||
function asModelArray(value: unknown): Array<{ id?: string; name?: string }> {
|
||||
if (!Array.isArray(value)) return [];
|
||||
return value.filter((item) => item && typeof item === "object") as Array<{
|
||||
id?: string;
|
||||
name?: string;
|
||||
}>;
|
||||
}
|
||||
|
||||
/**
|
||||
* GET /api/pricing/models
|
||||
* Returns the full model catalog merged from three sources:
|
||||
@@ -33,14 +47,15 @@ export async function GET() {
|
||||
}
|
||||
|
||||
// ── 2. Custom models (DB) ───────────────────────────────────────
|
||||
let customModelsMap: Record<string, any[]> = {};
|
||||
let customModelsMap: Record<string, unknown> = {};
|
||||
try {
|
||||
customModelsMap = await getAllCustomModels();
|
||||
customModelsMap = asRecord(await getAllCustomModels());
|
||||
} catch {
|
||||
/* DB may not be ready */
|
||||
}
|
||||
|
||||
for (const [providerId, models] of Object.entries(customModelsMap)) {
|
||||
for (const [providerId, rawModels] of Object.entries(customModelsMap)) {
|
||||
const models = asModelArray(rawModels);
|
||||
// Resolve alias — check if a registry entry maps this providerId
|
||||
let alias = providerId;
|
||||
for (const entry of Object.values(REGISTRY)) {
|
||||
@@ -63,13 +78,17 @@ export async function GET() {
|
||||
|
||||
const existingIds = new Set(catalog[alias].models.map((m) => m.id));
|
||||
for (const model of models) {
|
||||
if (!existingIds.has(model.id)) {
|
||||
const modelId = typeof model.id === "string" ? model.id : null;
|
||||
if (!modelId || existingIds.has(modelId)) {
|
||||
continue;
|
||||
}
|
||||
if (!existingIds.has(modelId)) {
|
||||
catalog[alias].models.push({
|
||||
id: model.id,
|
||||
name: model.name || model.id,
|
||||
id: modelId,
|
||||
name: typeof model.name === "string" && model.name.trim() ? model.name : modelId,
|
||||
custom: true,
|
||||
});
|
||||
existingIds.add(model.id);
|
||||
existingIds.add(modelId);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,6 +1,17 @@
|
||||
import { NextResponse } from "next/server";
|
||||
import { getDbInstance } from "@/lib/db/core";
|
||||
|
||||
type JsonRecord = Record<string, unknown>;
|
||||
|
||||
function toNumber(value: unknown): number {
|
||||
if (typeof value === "number" && Number.isFinite(value)) return value;
|
||||
if (typeof value === "string" && value.trim().length > 0) {
|
||||
const parsed = Number(value);
|
||||
return Number.isFinite(parsed) ? parsed : 0;
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
|
||||
/**
|
||||
* GET /api/providers/metrics — Aggregate per-provider stats from call_logs
|
||||
* Returns: { metrics: { [provider]: { totalRequests, totalSuccesses, successRate, avgLatencyMs } } }
|
||||
@@ -19,16 +30,30 @@ export async function GET() {
|
||||
WHERE provider IS NOT NULL AND provider != '-'
|
||||
GROUP BY provider`
|
||||
)
|
||||
.all();
|
||||
.all() as JsonRecord[];
|
||||
|
||||
const metrics = {};
|
||||
const metrics: Record<
|
||||
string,
|
||||
{
|
||||
totalRequests: number;
|
||||
totalSuccesses: number;
|
||||
successRate: number;
|
||||
avgLatencyMs: number;
|
||||
}
|
||||
> = {};
|
||||
for (const row of rows) {
|
||||
metrics[row.provider] = {
|
||||
totalRequests: row.totalRequests,
|
||||
totalSuccesses: row.totalSuccesses,
|
||||
successRate:
|
||||
row.totalRequests > 0 ? Math.round((row.totalSuccesses / row.totalRequests) * 100) : 0,
|
||||
avgLatencyMs: row.avgLatencyMs || 0,
|
||||
const provider =
|
||||
typeof row.provider === "string" && row.provider.trim().length > 0
|
||||
? row.provider
|
||||
: "unknown";
|
||||
const totalRequests = toNumber(row.totalRequests);
|
||||
const totalSuccesses = toNumber(row.totalSuccesses);
|
||||
const avgLatencyMs = toNumber(row.avgLatencyMs);
|
||||
metrics[provider] = {
|
||||
totalRequests,
|
||||
totalSuccesses,
|
||||
successRate: totalRequests > 0 ? Math.round((totalSuccesses / totalRequests) * 100) : 0,
|
||||
avgLatencyMs,
|
||||
};
|
||||
}
|
||||
|
||||
|
||||
@@ -13,6 +13,12 @@ import {
|
||||
validateBody,
|
||||
} from "@/shared/validation/schemas";
|
||||
|
||||
type JsonRecord = Record<string, unknown>;
|
||||
|
||||
function asRecord(value: unknown): JsonRecord {
|
||||
return value && typeof value === "object" && !Array.isArray(value) ? (value as JsonRecord) : {};
|
||||
}
|
||||
|
||||
// PUT /api/provider-nodes/[id] - Update provider node
|
||||
export async function PUT(request: Request, { params }: { params: Promise<{ id: string }> }) {
|
||||
let rawBody;
|
||||
@@ -61,7 +67,7 @@ export async function PUT(request: Request, { params }: { params: Promise<{ id:
|
||||
}
|
||||
}
|
||||
|
||||
const updates: Record<string, any> = {
|
||||
const updates: Record<string, unknown> = {
|
||||
name: name.trim(),
|
||||
prefix: prefix.trim(),
|
||||
baseUrl: sanitizedBaseUrl,
|
||||
@@ -75,17 +81,27 @@ export async function PUT(request: Request, { params }: { params: Promise<{ id:
|
||||
|
||||
const connections = await getProviderConnections({ provider: id });
|
||||
await Promise.all(
|
||||
connections.map((connection) =>
|
||||
updateProviderConnection(connection.id, {
|
||||
providerSpecificData: {
|
||||
...(connection.providerSpecificData || {}),
|
||||
prefix: prefix.trim(),
|
||||
apiType: node.type === "openai-compatible" ? apiType : undefined,
|
||||
baseUrl: sanitizedBaseUrl,
|
||||
nodeName: updated.name,
|
||||
},
|
||||
})
|
||||
)
|
||||
connections.flatMap((connectionRaw) => {
|
||||
const connection = asRecord(connectionRaw);
|
||||
const connectionId = typeof connection.id === "string" ? connection.id : "";
|
||||
if (!connectionId) return [];
|
||||
|
||||
const providerSpecificData = {
|
||||
...asRecord(connection.providerSpecificData),
|
||||
prefix: prefix.trim(),
|
||||
baseUrl: sanitizedBaseUrl,
|
||||
nodeName: updated.name,
|
||||
} as JsonRecord;
|
||||
if (node.type === "openai-compatible") {
|
||||
providerSpecificData.apiType = apiType;
|
||||
}
|
||||
|
||||
return [
|
||||
updateProviderConnection(connectionId, {
|
||||
providerSpecificData,
|
||||
}),
|
||||
];
|
||||
})
|
||||
);
|
||||
|
||||
return NextResponse.json({ node: updated });
|
||||
|
||||
@@ -5,6 +5,29 @@ import {
|
||||
isAnthropicCompatibleProvider,
|
||||
} from "@/shared/constants/providers";
|
||||
|
||||
type JsonRecord = Record<string, unknown>;
|
||||
|
||||
function asRecord(value: unknown): JsonRecord {
|
||||
return value && typeof value === "object" && !Array.isArray(value) ? (value as JsonRecord) : {};
|
||||
}
|
||||
|
||||
function getProviderBaseUrl(providerSpecificData: unknown): string | null {
|
||||
const data = asRecord(providerSpecificData);
|
||||
const baseUrl = data.baseUrl;
|
||||
return typeof baseUrl === "string" && baseUrl.trim().length > 0 ? baseUrl : null;
|
||||
}
|
||||
|
||||
type ProviderModelsConfigEntry = {
|
||||
url: string;
|
||||
method: "GET" | "POST";
|
||||
headers: Record<string, string>;
|
||||
authHeader?: string;
|
||||
authPrefix?: string;
|
||||
authQuery?: string;
|
||||
body?: unknown;
|
||||
parseResponse: (data: any) => any;
|
||||
};
|
||||
|
||||
// Providers that return hardcoded models (no remote /models API)
|
||||
const STATIC_MODEL_PROVIDERS = {
|
||||
deepgram: () => [
|
||||
@@ -33,7 +56,7 @@ const STATIC_MODEL_PROVIDERS = {
|
||||
};
|
||||
|
||||
// Provider models endpoints configuration
|
||||
const PROVIDER_MODELS_CONFIG = {
|
||||
const PROVIDER_MODELS_CONFIG: Record<string, ProviderModelsConfigEntry> = {
|
||||
claude: {
|
||||
url: "https://api.anthropic.com/v1/models",
|
||||
method: "GET",
|
||||
@@ -238,8 +261,20 @@ export async function GET(request, { params }) {
|
||||
return NextResponse.json({ error: "Connection not found" }, { status: 404 });
|
||||
}
|
||||
|
||||
if (isOpenAICompatibleProvider(connection.provider)) {
|
||||
const baseUrl = connection.providerSpecificData?.baseUrl;
|
||||
const provider =
|
||||
typeof connection.provider === "string" && connection.provider.trim().length > 0
|
||||
? connection.provider
|
||||
: null;
|
||||
if (!provider) {
|
||||
return NextResponse.json({ error: "Invalid connection provider" }, { status: 400 });
|
||||
}
|
||||
|
||||
const connectionId = typeof connection.id === "string" ? connection.id : id;
|
||||
const apiKey = typeof connection.apiKey === "string" ? connection.apiKey : "";
|
||||
const accessToken = typeof connection.accessToken === "string" ? connection.accessToken : "";
|
||||
|
||||
if (isOpenAICompatibleProvider(provider)) {
|
||||
const baseUrl = getProviderBaseUrl(connection.providerSpecificData);
|
||||
if (!baseUrl) {
|
||||
return NextResponse.json(
|
||||
{ error: "No base URL configured for OpenAI compatible provider" },
|
||||
@@ -260,13 +295,13 @@ export async function GET(request, { params }) {
|
||||
method: "GET",
|
||||
headers: {
|
||||
"Content-Type": "application/json",
|
||||
Authorization: `Bearer ${connection.apiKey}`,
|
||||
Authorization: `Bearer ${apiKey}`,
|
||||
},
|
||||
});
|
||||
|
||||
if (!response.ok) {
|
||||
const errorText = await response.text();
|
||||
console.log(`Error fetching models from ${connection.provider}:`, errorText);
|
||||
console.log(`Error fetching models from ${provider}:`, errorText);
|
||||
return NextResponse.json(
|
||||
{ error: `Failed to fetch models: ${response.status}` },
|
||||
{ status: response.status }
|
||||
@@ -277,14 +312,14 @@ export async function GET(request, { params }) {
|
||||
const models = data.data || data.models || [];
|
||||
|
||||
return NextResponse.json({
|
||||
provider: connection.provider,
|
||||
connectionId: connection.id,
|
||||
provider,
|
||||
connectionId,
|
||||
models,
|
||||
});
|
||||
}
|
||||
|
||||
if (isAnthropicCompatibleProvider(connection.provider)) {
|
||||
let baseUrl = connection.providerSpecificData?.baseUrl;
|
||||
if (isAnthropicCompatibleProvider(provider)) {
|
||||
let baseUrl = getProviderBaseUrl(connection.providerSpecificData);
|
||||
if (!baseUrl) {
|
||||
return NextResponse.json(
|
||||
{ error: "No base URL configured for Anthropic compatible provider" },
|
||||
@@ -302,15 +337,15 @@ export async function GET(request, { params }) {
|
||||
method: "GET",
|
||||
headers: {
|
||||
"Content-Type": "application/json",
|
||||
"x-api-key": connection.apiKey,
|
||||
"x-api-key": apiKey,
|
||||
"anthropic-version": "2023-06-01",
|
||||
Authorization: `Bearer ${connection.apiKey}`,
|
||||
Authorization: `Bearer ${apiKey}`,
|
||||
},
|
||||
});
|
||||
|
||||
if (!response.ok) {
|
||||
const errorText = await response.text();
|
||||
console.log(`Error fetching models from ${connection.provider}:`, errorText);
|
||||
console.log(`Error fetching models from ${provider}:`, errorText);
|
||||
return NextResponse.json(
|
||||
{ error: `Failed to fetch models: ${response.status}` },
|
||||
{ status: response.status }
|
||||
@@ -321,32 +356,38 @@ export async function GET(request, { params }) {
|
||||
const models = data.data || data.models || [];
|
||||
|
||||
return NextResponse.json({
|
||||
provider: connection.provider,
|
||||
connectionId: connection.id,
|
||||
provider,
|
||||
connectionId,
|
||||
models,
|
||||
});
|
||||
}
|
||||
|
||||
// Static model providers (no remote /models API)
|
||||
const staticModelsFn = STATIC_MODEL_PROVIDERS[connection.provider];
|
||||
const staticModelsFn =
|
||||
provider in STATIC_MODEL_PROVIDERS
|
||||
? STATIC_MODEL_PROVIDERS[provider as keyof typeof STATIC_MODEL_PROVIDERS]
|
||||
: undefined;
|
||||
if (staticModelsFn) {
|
||||
return NextResponse.json({
|
||||
provider: connection.provider,
|
||||
connectionId: connection.id,
|
||||
provider,
|
||||
connectionId,
|
||||
models: staticModelsFn(),
|
||||
});
|
||||
}
|
||||
|
||||
const config = PROVIDER_MODELS_CONFIG[connection.provider];
|
||||
const config =
|
||||
provider in PROVIDER_MODELS_CONFIG
|
||||
? PROVIDER_MODELS_CONFIG[provider as keyof typeof PROVIDER_MODELS_CONFIG]
|
||||
: undefined;
|
||||
if (!config) {
|
||||
return NextResponse.json(
|
||||
{ error: `Provider ${connection.provider} does not support models listing` },
|
||||
{ error: `Provider ${provider} does not support models listing` },
|
||||
{ status: 400 }
|
||||
);
|
||||
}
|
||||
|
||||
// Get auth token
|
||||
const token = connection.accessToken || connection.apiKey;
|
||||
const token = accessToken || apiKey;
|
||||
if (!token) {
|
||||
return NextResponse.json({ error: "No valid token found" }, { status: 401 });
|
||||
}
|
||||
@@ -377,7 +418,7 @@ export async function GET(request, { params }) {
|
||||
|
||||
if (!response.ok) {
|
||||
const errorText = await response.text();
|
||||
console.log(`Error fetching models from ${connection.provider}:`, errorText);
|
||||
console.log(`Error fetching models from ${provider}:`, errorText);
|
||||
return NextResponse.json(
|
||||
{ error: `Failed to fetch models: ${response.status}` },
|
||||
{ status: response.status }
|
||||
@@ -388,8 +429,8 @@ export async function GET(request, { params }) {
|
||||
const models = config.parseResponse(data);
|
||||
|
||||
return NextResponse.json({
|
||||
provider: connection.provider,
|
||||
connectionId: connection.id,
|
||||
provider,
|
||||
connectionId,
|
||||
models,
|
||||
});
|
||||
} catch (error) {
|
||||
|
||||
@@ -531,6 +531,21 @@ export async function testSingleConnection(connectionId: string) {
|
||||
return { valid: false, error: "Connection not found", diagnosis: null, latencyMs: 0 };
|
||||
}
|
||||
|
||||
const provider = typeof connection.provider === "string" ? connection.provider : "";
|
||||
if (!provider) {
|
||||
return {
|
||||
valid: false,
|
||||
error: "Connection provider is invalid",
|
||||
diagnosis: makeDiagnosis(
|
||||
"validation_error",
|
||||
"local",
|
||||
"Connection provider is invalid",
|
||||
"provider_invalid"
|
||||
),
|
||||
latencyMs: 0,
|
||||
};
|
||||
}
|
||||
|
||||
// Resolve proxy for this connection (key → combo → provider → global → direct)
|
||||
let proxyInfo: any = null;
|
||||
try {
|
||||
@@ -541,7 +556,7 @@ export async function testSingleConnection(connectionId: string) {
|
||||
|
||||
let result;
|
||||
const startTime = Date.now();
|
||||
const runtime = await getProviderRuntimeStatus(connection.provider);
|
||||
const runtime = await getProviderRuntimeStatus(provider);
|
||||
|
||||
if ((runtime as any)?.diagnosis) {
|
||||
result = {
|
||||
@@ -611,7 +626,7 @@ export async function testSingleConnection(connectionId: string) {
|
||||
path: "/api/providers/test",
|
||||
status: result.valid ? 200 : result.statusCode || 401,
|
||||
model: "connection-test",
|
||||
provider: connection.provider,
|
||||
provider,
|
||||
connectionId,
|
||||
duration: latencyMs,
|
||||
error: result.valid ? null : result.error || null,
|
||||
@@ -627,8 +642,8 @@ export async function testSingleConnection(connectionId: string) {
|
||||
proxy: proxyInfo?.proxy || null,
|
||||
level: proxyInfo?.level || "provider-test",
|
||||
levelId: proxyInfo?.levelId || null,
|
||||
provider: connection.provider,
|
||||
targetUrl: `${connection.provider}/connection-test`,
|
||||
provider,
|
||||
targetUrl: `${provider}/connection-test`,
|
||||
latencyMs,
|
||||
error: result.valid ? null : result.error || null,
|
||||
connectionId,
|
||||
|
||||
@@ -14,6 +14,12 @@ import {
|
||||
validateBody,
|
||||
} from "@/shared/validation/schemas";
|
||||
|
||||
type JsonRecord = Record<string, unknown>;
|
||||
|
||||
function asRecord(value: unknown): JsonRecord {
|
||||
return value && typeof value === "object" && !Array.isArray(value) ? (value as JsonRecord) : {};
|
||||
}
|
||||
|
||||
/**
|
||||
* GET /api/rate-limits — Consolidated rate-limit status
|
||||
*
|
||||
@@ -26,13 +32,23 @@ import {
|
||||
export async function GET() {
|
||||
try {
|
||||
const connections = await getProviderConnections();
|
||||
const statuses = connections.map((conn) => ({
|
||||
connectionId: conn.id,
|
||||
provider: conn.provider,
|
||||
name: conn.name || conn.email || conn.id.slice(0, 8),
|
||||
rateLimitProtection: !!conn.rateLimitProtection,
|
||||
...getRateLimitStatus(conn.provider, conn.id),
|
||||
}));
|
||||
const statuses = connections.map((connRaw) => {
|
||||
const conn = asRecord(connRaw);
|
||||
const connectionId = typeof conn.id === "string" ? conn.id : "";
|
||||
const provider = typeof conn.provider === "string" ? conn.provider : "unknown";
|
||||
const name =
|
||||
(typeof conn.name === "string" && conn.name.trim()) ||
|
||||
(typeof conn.email === "string" && conn.email.trim()) ||
|
||||
(connectionId ? connectionId.slice(0, 8) : "unknown");
|
||||
|
||||
return {
|
||||
connectionId,
|
||||
provider,
|
||||
name,
|
||||
rateLimitProtection: conn.rateLimitProtection === true,
|
||||
...getRateLimitStatus(provider, connectionId),
|
||||
};
|
||||
});
|
||||
|
||||
const lockouts = getAllModelLockouts();
|
||||
const cacheStats = getCacheStats();
|
||||
|
||||
@@ -6,6 +6,16 @@ import {
|
||||
validateBody,
|
||||
} from "@/shared/validation/schemas";
|
||||
|
||||
type JsonRecord = Record<string, unknown>;
|
||||
|
||||
function asRecord(value: unknown): JsonRecord {
|
||||
return value && typeof value === "object" && !Array.isArray(value) ? (value as JsonRecord) : {};
|
||||
}
|
||||
|
||||
function getErrorMessage(error: unknown, fallback: string): string {
|
||||
return error instanceof Error && error.message ? error.message : fallback;
|
||||
}
|
||||
|
||||
/**
|
||||
* GET /api/resilience — Get current resilience configuration and status
|
||||
*/
|
||||
@@ -24,14 +34,17 @@ export async function GET() {
|
||||
|
||||
return NextResponse.json({
|
||||
profiles: settings.providerProfiles || PROVIDER_PROFILES,
|
||||
defaults: { ...DEFAULT_API_LIMITS, ...(settings.rateLimitDefaults || {}) },
|
||||
defaults: {
|
||||
...DEFAULT_API_LIMITS,
|
||||
...asRecord(settings.rateLimitDefaults),
|
||||
},
|
||||
circuitBreakers,
|
||||
rateLimitStatus,
|
||||
});
|
||||
} catch (err) {
|
||||
} catch (err: unknown) {
|
||||
console.error("[API] GET /api/resilience error:", err);
|
||||
return NextResponse.json(
|
||||
{ error: err.message || "Failed to load resilience status" },
|
||||
{ error: getErrorMessage(err, "Failed to load resilience status") },
|
||||
{ status: 500 }
|
||||
);
|
||||
}
|
||||
@@ -74,10 +87,10 @@ export async function PATCH(request) {
|
||||
...(profiles ? { profiles } : {}),
|
||||
...(defaults ? { defaults } : {}),
|
||||
});
|
||||
} catch (err) {
|
||||
} catch (err: unknown) {
|
||||
console.error("[API] PATCH /api/resilience error:", err);
|
||||
return NextResponse.json(
|
||||
{ error: err.message || "Failed to save resilience settings" },
|
||||
{ error: getErrorMessage(err, "Failed to save resilience settings") },
|
||||
{ status: 500 }
|
||||
);
|
||||
}
|
||||
|
||||
@@ -45,7 +45,7 @@ export async function PATCH(request) {
|
||||
// If updating password, hash it
|
||||
if (body.newPassword) {
|
||||
const settings = await getSettings();
|
||||
const currentHash = settings.password;
|
||||
const currentHash = typeof settings.password === "string" ? settings.password : "";
|
||||
|
||||
// Verify current password if it exists
|
||||
if (currentHash) {
|
||||
|
||||
@@ -14,6 +14,12 @@ import {
|
||||
validateBody,
|
||||
} from "@/shared/validation/schemas";
|
||||
|
||||
function getProviderBaseUrl(providerSpecificData: unknown): string | undefined {
|
||||
if (!providerSpecificData || typeof providerSpecificData !== "object") return undefined;
|
||||
const baseUrl = (providerSpecificData as Record<string, unknown>).baseUrl;
|
||||
return typeof baseUrl === "string" && baseUrl.trim().length > 0 ? baseUrl : undefined;
|
||||
}
|
||||
|
||||
export async function POST(request) {
|
||||
let rawBody;
|
||||
try {
|
||||
@@ -78,7 +84,7 @@ export async function POST(request) {
|
||||
// Build URL and headers using provider service
|
||||
const url = buildProviderUrl(provider, body.model || "test-model", true, {
|
||||
baseUrlIndex: 0,
|
||||
baseUrl: connection.providerSpecificData?.baseUrl,
|
||||
baseUrl: getProviderBaseUrl(connection.providerSpecificData),
|
||||
});
|
||||
const headers = buildProviderHeaders(provider, credentials, true, body);
|
||||
|
||||
|
||||
@@ -33,6 +33,12 @@ function getModelId(value: JsonRecord): string {
|
||||
return typeof model === "string" && model.trim().length > 0 ? model : "test-model";
|
||||
}
|
||||
|
||||
function getProviderBaseUrl(providerSpecificData: unknown): string | undefined {
|
||||
const data = asJsonRecord(providerSpecificData);
|
||||
const baseUrl = data.baseUrl;
|
||||
return typeof baseUrl === "string" && baseUrl.trim().length > 0 ? baseUrl : undefined;
|
||||
}
|
||||
|
||||
export async function POST(request) {
|
||||
let rawBody;
|
||||
try {
|
||||
@@ -177,7 +183,7 @@ export async function POST(request) {
|
||||
// Build URL and headers
|
||||
const url = buildProviderUrl(provider, model, true, {
|
||||
baseUrlIndex: 0,
|
||||
baseUrl: connection.providerSpecificData?.baseUrl,
|
||||
baseUrl: getProviderBaseUrl(connection.providerSpecificData),
|
||||
});
|
||||
const headers = buildProviderHeaders(provider, credentials, true, actualBody);
|
||||
|
||||
|
||||
@@ -12,11 +12,23 @@ export async function GET(request) {
|
||||
|
||||
// Build connection map for account names
|
||||
const { getProviderConnections } = await import("@/lib/localDb");
|
||||
let connectionMap = {};
|
||||
const connectionMap: Record<string, string> = {};
|
||||
try {
|
||||
const connections = await getProviderConnections();
|
||||
for (const conn of connections) {
|
||||
connectionMap[conn.id] = conn.name || conn.email || conn.id;
|
||||
for (const connRaw of connections as unknown[]) {
|
||||
const conn =
|
||||
connRaw && typeof connRaw === "object" && !Array.isArray(connRaw)
|
||||
? (connRaw as Record<string, unknown>)
|
||||
: {};
|
||||
const connectionId =
|
||||
typeof conn.id === "string" && conn.id.trim().length > 0 ? conn.id : null;
|
||||
if (!connectionId) continue;
|
||||
|
||||
const name =
|
||||
(typeof conn.name === "string" && conn.name.trim()) ||
|
||||
(typeof conn.email === "string" && conn.email.trim()) ||
|
||||
connectionId;
|
||||
connectionMap[connectionId] = name;
|
||||
}
|
||||
} catch {
|
||||
/* ignore */
|
||||
|
||||
@@ -42,7 +42,7 @@ function deriveTokenStatus(connection: ProviderConnectionRecord): QuotaTokenStat
|
||||
|
||||
function buildQuotaEntry(
|
||||
connection: ProviderConnectionRecord,
|
||||
learnedLimit: Record<string, unknown> | null,
|
||||
learnedLimit: unknown,
|
||||
rateStatus: Record<string, unknown>
|
||||
): QuotaProviderEntry {
|
||||
const provider =
|
||||
@@ -64,16 +64,18 @@ function buildQuotaEntry(
|
||||
let quotaTotal: number | null = null;
|
||||
let quotaUsed = 0;
|
||||
let percentRemaining = 100;
|
||||
const learned =
|
||||
learnedLimit && typeof learnedLimit === "object" && !Array.isArray(learnedLimit)
|
||||
? (learnedLimit as Record<string, unknown>)
|
||||
: null;
|
||||
|
||||
const learnedLimitValue =
|
||||
learnedLimit && typeof learnedLimit.limit === "number" && Number.isFinite(learnedLimit.limit)
|
||||
? learnedLimit.limit
|
||||
learned && typeof learned.limit === "number" && Number.isFinite(learned.limit)
|
||||
? learned.limit
|
||||
: null;
|
||||
const learnedRemainingValue =
|
||||
learnedLimit &&
|
||||
typeof learnedLimit.remaining === "number" &&
|
||||
Number.isFinite(learnedLimit.remaining)
|
||||
? learnedLimit.remaining
|
||||
learned && typeof learned.remaining === "number" && Number.isFinite(learned.remaining)
|
||||
? learned.remaining
|
||||
: null;
|
||||
|
||||
if (learnedLimitValue !== null && learnedLimitValue > 0) {
|
||||
|
||||
@@ -332,8 +332,14 @@ export async function getUnifiedModelsResponse(
|
||||
|
||||
// Add custom models (user-defined)
|
||||
try {
|
||||
const customModelsMap: Record<string, any[]> = await getAllCustomModels();
|
||||
for (const [providerId, providerCustomModels] of Object.entries(customModelsMap)) {
|
||||
const customModelsMap = (await getAllCustomModels()) as Record<string, unknown>;
|
||||
for (const [providerId, rawProviderCustomModels] of Object.entries(customModelsMap)) {
|
||||
const providerCustomModels = Array.isArray(rawProviderCustomModels)
|
||||
? rawProviderCustomModels.filter(
|
||||
(model): model is Record<string, unknown> =>
|
||||
!!model && typeof model === "object" && !Array.isArray(model)
|
||||
)
|
||||
: [];
|
||||
// For compatible providers, use the prefix from provider nodes
|
||||
const prefix = providerIdToPrefix[providerId];
|
||||
const alias = prefix || providerIdToAlias[providerId] || providerId;
|
||||
@@ -351,8 +357,11 @@ export async function getUnifiedModelsResponse(
|
||||
continue;
|
||||
|
||||
for (const model of providerCustomModels) {
|
||||
const modelId = typeof model.id === "string" ? model.id : null;
|
||||
if (!modelId) continue;
|
||||
|
||||
// Skip if already added as built-in
|
||||
const aliasId = `${alias}/${model.id}`;
|
||||
const aliasId = `${alias}/${modelId}`;
|
||||
if (models.some((m) => m.id === aliasId)) continue;
|
||||
|
||||
models.push({
|
||||
@@ -361,14 +370,14 @@ export async function getUnifiedModelsResponse(
|
||||
created: timestamp,
|
||||
owned_by: canonicalProviderId,
|
||||
permission: [],
|
||||
root: model.id,
|
||||
root: modelId,
|
||||
parent: null,
|
||||
custom: true,
|
||||
});
|
||||
|
||||
// Only add provider-prefixed version if different from alias
|
||||
if (canonicalProviderId !== alias && !prefix) {
|
||||
const providerPrefixedId = `${canonicalProviderId}/${model.id}`;
|
||||
const providerPrefixedId = `${canonicalProviderId}/${modelId}`;
|
||||
if (models.some((m) => m.id === providerPrefixedId)) continue;
|
||||
models.push({
|
||||
id: providerPrefixedId,
|
||||
@@ -376,7 +385,7 @@ export async function getUnifiedModelsResponse(
|
||||
created: timestamp,
|
||||
owned_by: canonicalProviderId,
|
||||
permission: [],
|
||||
root: model.id,
|
||||
root: modelId,
|
||||
parent: aliasId,
|
||||
custom: true,
|
||||
});
|
||||
|
||||
@@ -9,7 +9,6 @@
|
||||
* @module domain/costRules
|
||||
*/
|
||||
|
||||
|
||||
import {
|
||||
saveBudget,
|
||||
loadBudget,
|
||||
@@ -20,6 +19,32 @@ import {
|
||||
deleteCostEntries,
|
||||
} from "../lib/db/domainState";
|
||||
|
||||
interface BudgetConfig {
|
||||
dailyLimitUsd: number;
|
||||
monthlyLimitUsd?: number;
|
||||
warningThreshold?: number;
|
||||
}
|
||||
|
||||
interface CostEntry {
|
||||
cost: number;
|
||||
timestamp: number;
|
||||
}
|
||||
|
||||
function toCostEntries(value: unknown): CostEntry[] {
|
||||
if (!Array.isArray(value)) return [];
|
||||
const entries: CostEntry[] = [];
|
||||
for (const item of value) {
|
||||
if (!item || typeof item !== "object" || Array.isArray(item)) continue;
|
||||
const record = item as Record<string, unknown>;
|
||||
const cost = typeof record.cost === "number" ? record.cost : Number(record.cost ?? 0);
|
||||
const timestamp =
|
||||
typeof record.timestamp === "number" ? record.timestamp : Number(record.timestamp ?? 0);
|
||||
if (!Number.isFinite(cost) || !Number.isFinite(timestamp)) continue;
|
||||
entries.push({ cost, timestamp });
|
||||
}
|
||||
return entries;
|
||||
}
|
||||
|
||||
/**
|
||||
* @typedef {Object} BudgetConfig
|
||||
* @property {number} dailyLimitUsd - Max daily spend in USD
|
||||
@@ -34,7 +59,7 @@ import {
|
||||
*/
|
||||
|
||||
/** @type {Map<string, BudgetConfig>} In-memory cache for budgets */
|
||||
const budgets = new Map();
|
||||
const budgets = new Map<string, BudgetConfig>();
|
||||
|
||||
/** @type {boolean} */
|
||||
let _budgetsLoaded = false;
|
||||
@@ -45,7 +70,7 @@ let _budgetsLoaded = false;
|
||||
* @param {string} apiKeyId
|
||||
* @param {BudgetConfig} config
|
||||
*/
|
||||
export function setBudget(apiKeyId, config) {
|
||||
export function setBudget(apiKeyId: string, config: BudgetConfig) {
|
||||
const normalized = {
|
||||
dailyLimitUsd: config.dailyLimitUsd,
|
||||
monthlyLimitUsd: config.monthlyLimitUsd || 0,
|
||||
@@ -65,14 +90,14 @@ export function setBudget(apiKeyId, config) {
|
||||
* @param {string} apiKeyId
|
||||
* @returns {BudgetConfig | null}
|
||||
*/
|
||||
export function getBudget(apiKeyId) {
|
||||
export function getBudget(apiKeyId: string): BudgetConfig | null {
|
||||
// Check in-memory cache first
|
||||
if (budgets.has(apiKeyId)) {
|
||||
return budgets.get(apiKeyId);
|
||||
}
|
||||
// Try loading from DB
|
||||
try {
|
||||
const fromDb = loadBudget(apiKeyId);
|
||||
const fromDb = loadBudget(apiKeyId) as BudgetConfig | null;
|
||||
if (fromDb) {
|
||||
budgets.set(apiKeyId, fromDb);
|
||||
return fromDb;
|
||||
@@ -89,7 +114,7 @@ export function getBudget(apiKeyId) {
|
||||
* @param {string} apiKeyId
|
||||
* @param {number} cost - Cost in USD
|
||||
*/
|
||||
export function recordCost(apiKeyId, cost) {
|
||||
export function recordCost(apiKeyId: string, cost: number): void {
|
||||
const timestamp = Date.now();
|
||||
try {
|
||||
saveCostEntry(apiKeyId, cost, timestamp);
|
||||
@@ -105,7 +130,7 @@ export function recordCost(apiKeyId, cost) {
|
||||
* @param {number} [additionalCost=0] - Projected cost to check
|
||||
* @returns {{ allowed: boolean, reason?: string, dailyUsed: number, dailyLimit: number, warningReached: boolean }}
|
||||
*/
|
||||
export function checkBudget(apiKeyId, additionalCost = 0) {
|
||||
export function checkBudget(apiKeyId: string, additionalCost = 0) {
|
||||
const budget = getBudget(apiKeyId);
|
||||
if (!budget) {
|
||||
return { allowed: true, dailyUsed: 0, dailyLimit: 0, warningReached: false };
|
||||
@@ -139,13 +164,13 @@ export function checkBudget(apiKeyId, additionalCost = 0) {
|
||||
* @param {string} apiKeyId
|
||||
* @returns {number} Total cost today in USD
|
||||
*/
|
||||
export function getDailyTotal(apiKeyId) {
|
||||
export function getDailyTotal(apiKeyId: string): number {
|
||||
const todayStart = new Date();
|
||||
todayStart.setHours(0, 0, 0, 0);
|
||||
const startMs = todayStart.getTime();
|
||||
|
||||
try {
|
||||
const entries = loadCostEntries(apiKeyId, startMs);
|
||||
const entries = toCostEntries(loadCostEntries(apiKeyId, startMs));
|
||||
return entries.reduce((sum, e) => sum + e.cost, 0);
|
||||
} catch {
|
||||
return 0;
|
||||
@@ -158,7 +183,7 @@ export function getDailyTotal(apiKeyId) {
|
||||
* @param {string} apiKeyId
|
||||
* @returns {{ dailyTotal: number, monthlyTotal: number, totalEntries: number, budget: BudgetConfig | null }}
|
||||
*/
|
||||
export function getCostSummary(apiKeyId) {
|
||||
export function getCostSummary(apiKeyId: string) {
|
||||
const now = new Date();
|
||||
|
||||
const todayStart = new Date(now);
|
||||
@@ -167,8 +192,8 @@ export function getCostSummary(apiKeyId) {
|
||||
const monthStart = new Date(now.getFullYear(), now.getMonth(), 1);
|
||||
|
||||
try {
|
||||
const dailyEntries = loadCostEntries(apiKeyId, todayStart.getTime());
|
||||
const monthlyEntries = loadCostEntries(apiKeyId, monthStart.getTime());
|
||||
const dailyEntries = toCostEntries(loadCostEntries(apiKeyId, todayStart.getTime()));
|
||||
const monthlyEntries = toCostEntries(loadCostEntries(apiKeyId, monthStart.getTime()));
|
||||
|
||||
const dailyTotal = dailyEntries.reduce((sum, e) => sum + e.cost, 0);
|
||||
const monthlyTotal = monthlyEntries.reduce((sum, e) => sum + e.cost, 0);
|
||||
|
||||
748
src/lib/a2a/README.md
Normal file
748
src/lib/a2a/README.md
Normal file
@@ -0,0 +1,748 @@
|
||||
# OmniRoute A2A Server
|
||||
|
||||
> **Agent-to-Agent Protocol v0.3** — Enables any AI agent to use OmniRoute as an intelligent routing agent via JSON-RPC 2.0.
|
||||
|
||||
The A2A Server exposes OmniRoute as a **first-class agent** that other agents can discover, delegate tasks to, and collaborate with using the [A2A Protocol](https://google.github.io/A2A/).
|
||||
|
||||
---
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
┌──────────────────────────────────────────────────────────────────┐
|
||||
│ Orchestrator Agent │
|
||||
│ (LangChain, CrewAI, AutoGen, Custom Agent) │
|
||||
└──────────────────────┬───────────────────────────────────────────┘
|
||||
│ 1. GET /.well-known/agent.json (discover)
|
||||
│ 2. POST /a2a (JSON-RPC 2.0)
|
||||
▼
|
||||
┌──────────────────────────────────────────────────────────────────┐
|
||||
│ OmniRoute A2A Server │
|
||||
│ ┌────────────────┐ ┌────────────────┐ ┌───────────────────┐ │
|
||||
│ │ Task Manager │ │ Skill Engine │ │ SSE Streaming │ │
|
||||
│ │ (lifecycle) │──│ (registry) │──│ (real-time) │ │
|
||||
│ └────────────────┘ └────────┬───────┘ └───────────────────┘ │
|
||||
│ │ │
|
||||
│ Skills: │ │
|
||||
│ ├─ smart-routing ──────────┤ ┌────────────────────────────┐ │
|
||||
│ └─ quota-management ───────┘ │ Routing Decision Logger │ │
|
||||
│ └────────────────────────────┘ │
|
||||
└──────────────────────────────────────────────────────────────────┘
|
||||
│
|
||||
▼ OmniRoute Gateway (internal)
|
||||
/v1/chat/completions, /api/combos, /api/usage/quota
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Quick Start
|
||||
|
||||
### Agent Discovery
|
||||
|
||||
Every A2A-compatible agent exposes an **Agent Card** at `/.well-known/agent.json`:
|
||||
|
||||
```bash
|
||||
curl http://localhost:20128/.well-known/agent.json
|
||||
```
|
||||
|
||||
**Response:**
|
||||
|
||||
```json
|
||||
{
|
||||
"name": "OmniRoute",
|
||||
"description": "Intelligent AI gateway with auto-routing across 50+ providers",
|
||||
"url": "http://localhost:20128/a2a",
|
||||
"version": "1.8.1",
|
||||
"capabilities": {
|
||||
"streaming": true,
|
||||
"pushNotifications": false
|
||||
},
|
||||
"skills": [
|
||||
{
|
||||
"id": "smart-routing",
|
||||
"name": "Smart Routing",
|
||||
"description": "Routes prompts through OmniRoute intelligent pipeline",
|
||||
"tags": ["routing", "llm", "multi-provider", "cost-optimization"],
|
||||
"examples": [
|
||||
"Write a hello world in Python",
|
||||
"Explain quantum computing using the cheapest provider"
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "quota-management",
|
||||
"name": "Quota Management",
|
||||
"description": "Natural-language queries about provider quotas",
|
||||
"tags": ["quota", "analytics", "cost"],
|
||||
"examples": [
|
||||
"Which provider has the most quota remaining?",
|
||||
"Suggest a free combo for coding"
|
||||
]
|
||||
}
|
||||
],
|
||||
"authentication": {
|
||||
"schemes": ["bearer"],
|
||||
"apiKeyHeader": "Authorization"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## JSON-RPC 2.0 Methods
|
||||
|
||||
### `message/send` — Synchronous Execution
|
||||
|
||||
Send a message to a skill and receive the complete response.
|
||||
|
||||
```bash
|
||||
curl -X POST http://localhost:20128/a2a \
|
||||
-H "Content-Type: application/json" \
|
||||
-H "Authorization: Bearer YOUR_KEY" \
|
||||
-d '{
|
||||
"jsonrpc": "2.0",
|
||||
"id": "1",
|
||||
"method": "message/send",
|
||||
"params": {
|
||||
"skill": "smart-routing",
|
||||
"messages": [{"role": "user", "content": "Write a Python hello world"}],
|
||||
"metadata": {"model": "auto", "combo": "fast-coding"}
|
||||
}
|
||||
}'
|
||||
```
|
||||
|
||||
**Response:**
|
||||
|
||||
```json
|
||||
{
|
||||
"jsonrpc": "2.0",
|
||||
"id": "1",
|
||||
"result": {
|
||||
"task": { "id": "a1b2c3d4-...", "state": "completed" },
|
||||
"artifacts": [{ "type": "text", "content": "print('Hello, World!')" }],
|
||||
"metadata": {
|
||||
"routing_explanation": "Selected claude-sonnet via provider \"anthropic\" (latency: 1200ms, cost: $0.0030)",
|
||||
"cost_envelope": { "estimated": 0.005, "actual": 0.003, "currency": "USD" },
|
||||
"resilience_trace": [
|
||||
{ "event": "primary_selected", "provider": "anthropic", "timestamp": "2026-03-04T..." }
|
||||
],
|
||||
"policy_verdict": { "allowed": true, "reason": "within budget and quota limits" }
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### `message/stream` — SSE Streaming
|
||||
|
||||
Same as `message/send` but returns Server-Sent Events for real-time streaming.
|
||||
|
||||
```bash
|
||||
curl -N -X POST http://localhost:20128/a2a \
|
||||
-H "Content-Type: application/json" \
|
||||
-H "Authorization: Bearer YOUR_KEY" \
|
||||
-d '{
|
||||
"jsonrpc": "2.0",
|
||||
"id": "1",
|
||||
"method": "message/stream",
|
||||
"params": {
|
||||
"skill": "smart-routing",
|
||||
"messages": [{"role": "user", "content": "Explain quantum computing"}]
|
||||
}
|
||||
}'
|
||||
```
|
||||
|
||||
**SSE Events:**
|
||||
|
||||
```
|
||||
data: {"jsonrpc":"2.0","method":"message/stream","params":{"task":{"id":"...","state":"working"},"chunk":{"type":"text","content":"Quantum computing..."}}}
|
||||
|
||||
: heartbeat 2026-03-04T21:00:00Z
|
||||
|
||||
data: {"jsonrpc":"2.0","method":"message/stream","params":{"task":{"id":"...","state":"completed"},"metadata":{...}}}
|
||||
```
|
||||
|
||||
### `tasks/get` — Query Task Status
|
||||
|
||||
```bash
|
||||
curl -X POST http://localhost:20128/a2a \
|
||||
-H "Content-Type: application/json" \
|
||||
-H "Authorization: Bearer YOUR_KEY" \
|
||||
-d '{"jsonrpc":"2.0","id":"2","method":"tasks/get","params":{"taskId":"TASK_UUID"}}'
|
||||
```
|
||||
|
||||
### `tasks/cancel` — Cancel a Running Task
|
||||
|
||||
```bash
|
||||
curl -X POST http://localhost:20128/a2a \
|
||||
-H "Content-Type: application/json" \
|
||||
-H "Authorization: Bearer YOUR_KEY" \
|
||||
-d '{"jsonrpc":"2.0","id":"3","method":"tasks/cancel","params":{"taskId":"TASK_UUID"}}'
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Skills Reference
|
||||
|
||||
### `smart-routing`
|
||||
|
||||
Routes prompts through OmniRoute's intelligent pipeline with full observability.
|
||||
|
||||
**Parameters (in `metadata`):**
|
||||
|
||||
| Parameter | Type | Default | Description |
|
||||
| --------- | -------- | ------------ | ---------------------------------------------------------------------------------------- |
|
||||
| `model` | `string` | `"auto"` | Target model (e.g., `claude-sonnet-4`, `gpt-4o`, `auto`) |
|
||||
| `combo` | `string` | active combo | Specific combo to route through |
|
||||
| `budget` | `number` | none | Maximum cost in USD for this request |
|
||||
| `role` | `string` | none | Task role hint: `coding`, `review`, `planning`, `analysis`, `debugging`, `documentation` |
|
||||
|
||||
**Returns:**
|
||||
|
||||
| Field | Description |
|
||||
| ------------------------------ | --------------------------------------------------------- |
|
||||
| `artifacts[].content` | The LLM response text |
|
||||
| `metadata.routing_explanation` | Human-readable explanation of routing decision |
|
||||
| `metadata.cost_envelope` | Estimated vs actual cost with currency |
|
||||
| `metadata.resilience_trace` | Array of events (primary_selected, fallback_needed, etc.) |
|
||||
| `metadata.policy_verdict` | Whether the request was allowed and why |
|
||||
|
||||
### `quota-management`
|
||||
|
||||
Answers natural-language queries about provider quotas.
|
||||
|
||||
**Query types (inferred from message content):**
|
||||
|
||||
| Query Pattern | Response Type |
|
||||
| ---------------------------------------------- | -------------------------------------------------------- |
|
||||
| Contains `"ranking"`, `"most quota"`, `"best"` | Providers ranked by remaining quota |
|
||||
| Contains `"free"`, `"suggest"` | Lists free combos or suggests free-tier providers |
|
||||
| Default | Full quota summary with warnings for low-quota providers |
|
||||
|
||||
---
|
||||
|
||||
## Task Lifecycle
|
||||
|
||||
```
|
||||
submitted ──→ working ──→ completed
|
||||
──→ failed
|
||||
──────────→ cancelled
|
||||
```
|
||||
|
||||
| State | Description |
|
||||
| ----------- | ----------------------------------------------------- |
|
||||
| `submitted` | Task created, queued for execution |
|
||||
| `working` | Skill handler is executing |
|
||||
| `completed` | Execution succeeded, artifacts available |
|
||||
| `failed` | Execution failed or task expired (TTL: 5 min default) |
|
||||
| `cancelled` | Cancelled by client via `tasks/cancel` |
|
||||
|
||||
- Terminal states: `completed`, `failed`, `cancelled` (no further transitions)
|
||||
- Expired tasks in `submitted` or `working` are auto-marked as `failed`
|
||||
- Tasks are garbage-collected after 2× TTL
|
||||
|
||||
---
|
||||
|
||||
## Client Examples
|
||||
|
||||
### Python — Orchestrator Agent
|
||||
|
||||
```python
|
||||
"""
|
||||
A2A Client — Python example.
|
||||
Discovers OmniRoute agent, sends a task, and processes the result.
|
||||
"""
|
||||
import requests
|
||||
import json
|
||||
|
||||
BASE_URL = "http://localhost:20128"
|
||||
API_KEY = "your-api-key"
|
||||
HEADERS = {
|
||||
"Content-Type": "application/json",
|
||||
"Authorization": f"Bearer {API_KEY}",
|
||||
}
|
||||
|
||||
# 1. Discover agent capabilities
|
||||
agent_card = requests.get(f"{BASE_URL}/.well-known/agent.json").json()
|
||||
print(f"Agent: {agent_card['name']} v{agent_card['version']}")
|
||||
print(f"Skills: {[s['id'] for s in agent_card['skills']]}")
|
||||
|
||||
# 2. Send a smart-routing task
|
||||
response = requests.post(f"{BASE_URL}/a2a", headers=HEADERS, json={
|
||||
"jsonrpc": "2.0",
|
||||
"id": "task-1",
|
||||
"method": "message/send",
|
||||
"params": {
|
||||
"skill": "smart-routing",
|
||||
"messages": [{"role": "user", "content": "Write a Python quicksort implementation"}],
|
||||
"metadata": {
|
||||
"model": "auto",
|
||||
"combo": "fast-coding",
|
||||
"budget": 0.10,
|
||||
}
|
||||
}
|
||||
})
|
||||
result = response.json()["result"]
|
||||
print(f"\n📝 Response: {result['artifacts'][0]['content'][:200]}...")
|
||||
print(f"🔀 Routing: {result['metadata']['routing_explanation']}")
|
||||
print(f"💰 Cost: ${result['metadata']['cost_envelope']['actual']}")
|
||||
print(f"🛡️ Policy: {result['metadata']['policy_verdict']['reason']}")
|
||||
|
||||
# 3. Query quota status
|
||||
quota_resp = requests.post(f"{BASE_URL}/a2a", headers=HEADERS, json={
|
||||
"jsonrpc": "2.0",
|
||||
"id": "task-2",
|
||||
"method": "message/send",
|
||||
"params": {
|
||||
"skill": "quota-management",
|
||||
"messages": [{"role": "user", "content": "Which provider has the most quota remaining?"}],
|
||||
}
|
||||
})
|
||||
quota_result = quota_resp.json()["result"]
|
||||
print(f"\n📊 Quota: {quota_result['artifacts'][0]['content']}")
|
||||
```
|
||||
|
||||
### TypeScript — Multi-Agent Orchestrator
|
||||
|
||||
```typescript
|
||||
/**
|
||||
* A2A Client — TypeScript example.
|
||||
* Shows agent discovery, task delegation, and streaming.
|
||||
*/
|
||||
|
||||
const BASE_URL = "http://localhost:20128";
|
||||
const API_KEY = "your-api-key";
|
||||
|
||||
interface JsonRpcResponse<T = any> {
|
||||
jsonrpc: "2.0";
|
||||
id: string | number;
|
||||
result?: T;
|
||||
error?: { code: number; message: string };
|
||||
}
|
||||
|
||||
async function a2aCall<T>(method: string, params: Record<string, any>): Promise<T> {
|
||||
const resp = await fetch(`${BASE_URL}/a2a`, {
|
||||
method: "POST",
|
||||
headers: {
|
||||
"Content-Type": "application/json",
|
||||
Authorization: `Bearer ${API_KEY}`,
|
||||
},
|
||||
body: JSON.stringify({
|
||||
jsonrpc: "2.0",
|
||||
id: `${method}-${Date.now()}`,
|
||||
method,
|
||||
params,
|
||||
}),
|
||||
});
|
||||
const json: JsonRpcResponse<T> = await resp.json();
|
||||
if (json.error) throw new Error(`[${json.error.code}] ${json.error.message}`);
|
||||
return json.result!;
|
||||
}
|
||||
|
||||
// ── Agent Discovery ──
|
||||
const agentCard = await fetch(`${BASE_URL}/.well-known/agent.json`).then((r) => r.json());
|
||||
console.log(`Connected to: ${agentCard.name} (${agentCard.skills.length} skills)`);
|
||||
|
||||
// ── Smart Routing: Send a coding task ──
|
||||
const routingResult = await a2aCall("message/send", {
|
||||
skill: "smart-routing",
|
||||
messages: [{ role: "user", content: "Implement a Redis cache wrapper in TypeScript" }],
|
||||
metadata: { model: "claude-sonnet-4", role: "coding" },
|
||||
});
|
||||
console.log("Response:", routingResult.artifacts[0].content);
|
||||
console.log("Provider:", routingResult.metadata.routing_explanation);
|
||||
|
||||
// ── Quota Management: Find free alternatives ──
|
||||
const quotaResult = await a2aCall("message/send", {
|
||||
skill: "quota-management",
|
||||
messages: [{ role: "user", content: "Suggest free combos for documentation" }],
|
||||
});
|
||||
console.log("Free combos:", quotaResult.artifacts[0].content);
|
||||
|
||||
// ── Streaming: Real-time response ──
|
||||
const streamResp = await fetch(`${BASE_URL}/a2a`, {
|
||||
method: "POST",
|
||||
headers: {
|
||||
"Content-Type": "application/json",
|
||||
Authorization: `Bearer ${API_KEY}`,
|
||||
},
|
||||
body: JSON.stringify({
|
||||
jsonrpc: "2.0",
|
||||
id: "stream-1",
|
||||
method: "message/stream",
|
||||
params: {
|
||||
skill: "smart-routing",
|
||||
messages: [{ role: "user", content: "Explain microservices architecture" }],
|
||||
},
|
||||
}),
|
||||
});
|
||||
|
||||
const reader = streamResp.body!.getReader();
|
||||
const decoder = new TextDecoder();
|
||||
while (true) {
|
||||
const { done, value } = await reader.read();
|
||||
if (done) break;
|
||||
const chunk = decoder.decode(value);
|
||||
for (const line of chunk.split("\n")) {
|
||||
if (line.startsWith("data: ")) {
|
||||
const event = JSON.parse(line.slice(6));
|
||||
if (event.params.chunk) {
|
||||
process.stdout.write(event.params.chunk.content);
|
||||
}
|
||||
if (event.params.task.state === "completed") {
|
||||
console.log("\n✅ Stream completed");
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Python — LangChain A2A Integration
|
||||
|
||||
```python
|
||||
"""
|
||||
LangChain integration — Use OmniRoute A2A as a custom LLM.
|
||||
"""
|
||||
from langchain.llms.base import BaseLLM
|
||||
from langchain.schema import LLMResult, Generation
|
||||
import requests
|
||||
from typing import List, Optional
|
||||
|
||||
class OmniRouteA2A(BaseLLM):
|
||||
base_url: str = "http://localhost:20128"
|
||||
api_key: str = ""
|
||||
model: str = "auto"
|
||||
combo: Optional[str] = None
|
||||
|
||||
@property
|
||||
def _llm_type(self) -> str:
|
||||
return "omniroute-a2a"
|
||||
|
||||
def _call(self, prompt: str, stop: Optional[List[str]] = None, **kwargs) -> str:
|
||||
response = requests.post(
|
||||
f"{self.base_url}/a2a",
|
||||
headers={
|
||||
"Content-Type": "application/json",
|
||||
"Authorization": f"Bearer {self.api_key}",
|
||||
},
|
||||
json={
|
||||
"jsonrpc": "2.0",
|
||||
"id": "langchain-1",
|
||||
"method": "message/send",
|
||||
"params": {
|
||||
"skill": "smart-routing",
|
||||
"messages": [{"role": "user", "content": prompt}],
|
||||
"metadata": {
|
||||
"model": self.model,
|
||||
**({"combo": self.combo} if self.combo else {}),
|
||||
},
|
||||
},
|
||||
},
|
||||
)
|
||||
result = response.json()["result"]
|
||||
return result["artifacts"][0]["content"]
|
||||
|
||||
def _generate(self, prompts: List[str], stop=None, **kwargs) -> LLMResult:
|
||||
return LLMResult(
|
||||
generations=[[Generation(text=self._call(p, stop))] for p in prompts]
|
||||
)
|
||||
|
||||
# Usage
|
||||
llm = OmniRouteA2A(
|
||||
base_url="http://localhost:20128",
|
||||
api_key="your-key",
|
||||
model="auto",
|
||||
combo="fast-coding",
|
||||
)
|
||||
result = llm("Write a Python function to merge two sorted lists")
|
||||
print(result)
|
||||
```
|
||||
|
||||
### Go — A2A Client
|
||||
|
||||
```go
|
||||
package main
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
)
|
||||
|
||||
const baseURL = "http://localhost:20128"
|
||||
const apiKey = "your-api-key"
|
||||
|
||||
type JsonRpcRequest struct {
|
||||
Jsonrpc string `json:"jsonrpc"`
|
||||
ID string `json:"id"`
|
||||
Method string `json:"method"`
|
||||
Params interface{} `json:"params"`
|
||||
}
|
||||
|
||||
type JsonRpcResponse struct {
|
||||
Jsonrpc string `json:"jsonrpc"`
|
||||
ID string `json:"id"`
|
||||
Result interface{} `json:"result"`
|
||||
Error *struct {
|
||||
Code int `json:"code"`
|
||||
Message string `json:"message"`
|
||||
} `json:"error"`
|
||||
}
|
||||
|
||||
func a2aCall(method string, params interface{}) (*JsonRpcResponse, error) {
|
||||
body, _ := json.Marshal(JsonRpcRequest{
|
||||
Jsonrpc: "2.0",
|
||||
ID: "go-1",
|
||||
Method: method,
|
||||
Params: params,
|
||||
})
|
||||
|
||||
req, _ := http.NewRequest("POST", baseURL+"/a2a", bytes.NewReader(body))
|
||||
req.Header.Set("Content-Type", "application/json")
|
||||
req.Header.Set("Authorization", "Bearer "+apiKey)
|
||||
|
||||
resp, err := http.DefaultClient.Do(req)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
data, _ := io.ReadAll(resp.Body)
|
||||
|
||||
var result JsonRpcResponse
|
||||
json.Unmarshal(data, &result)
|
||||
return &result, nil
|
||||
}
|
||||
|
||||
func main() {
|
||||
// Discover agent
|
||||
resp, _ := http.Get(baseURL + "/.well-known/agent.json")
|
||||
defer resp.Body.Close()
|
||||
body, _ := io.ReadAll(resp.Body)
|
||||
fmt.Println("Agent Card:", string(body))
|
||||
|
||||
// Send smart-routing task
|
||||
result, _ := a2aCall("message/send", map[string]interface{}{
|
||||
"skill": "smart-routing",
|
||||
"messages": []map[string]string{{"role": "user", "content": "Hello from Go!"}},
|
||||
"metadata": map[string]interface{}{"model": "auto"},
|
||||
})
|
||||
out, _ := json.MarshalIndent(result.Result, "", " ")
|
||||
fmt.Println("Result:", string(out))
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Use Cases
|
||||
|
||||
### 🤖 Use Case 1: Multi-Agent Coding Pipeline
|
||||
|
||||
An orchestrator agent delegates code generation to OmniRoute, then passes the output to a review agent.
|
||||
|
||||
```python
|
||||
def coding_pipeline(task: str):
|
||||
# Step 1: Generate code via OmniRoute A2A
|
||||
code_result = a2a_send("smart-routing", [
|
||||
{"role": "user", "content": f"Write production-quality code: {task}"}
|
||||
], metadata={"model": "auto", "role": "coding"})
|
||||
code = code_result["artifacts"][0]["content"]
|
||||
|
||||
# Step 2: Review the code via OmniRoute A2A (different model)
|
||||
review_result = a2a_send("smart-routing", [
|
||||
{"role": "user", "content": f"Review this code for bugs and improvements:\n\n{code}"}
|
||||
], metadata={"model": "auto", "role": "review"})
|
||||
review = review_result["artifacts"][0]["content"]
|
||||
|
||||
# Step 3: Check costs
|
||||
print(f"Code cost: ${code_result['metadata']['cost_envelope']['actual']}")
|
||||
print(f"Review cost: ${review_result['metadata']['cost_envelope']['actual']}")
|
||||
|
||||
return {"code": code, "review": review}
|
||||
```
|
||||
|
||||
### 💡 Use Case 2: Quota-Aware Agent Swarm
|
||||
|
||||
Multiple agents share quota through OmniRoute, using the quota skill to coordinate.
|
||||
|
||||
```python
|
||||
async def quota_aware_agent(agent_name: str, task: str):
|
||||
# Check quota before starting
|
||||
quota = a2a_send("quota-management", [
|
||||
{"role": "user", "content": "Which provider has the most quota remaining?"}
|
||||
])
|
||||
print(f"[{agent_name}] {quota['artifacts'][0]['content']}")
|
||||
|
||||
# Send request with budget constraint
|
||||
result = a2a_send("smart-routing", [
|
||||
{"role": "user", "content": task}
|
||||
], metadata={"budget": 0.05})
|
||||
|
||||
policy = result["metadata"]["policy_verdict"]
|
||||
if not policy["allowed"]:
|
||||
print(f"[{agent_name}] ⚠️ Budget exceeded: {policy['reason']}")
|
||||
# Fall back to free combo
|
||||
quota = a2a_send("quota-management", [
|
||||
{"role": "user", "content": "Suggest free combos"}
|
||||
])
|
||||
print(f"[{agent_name}] Free alternatives: {quota['artifacts'][0]['content']}")
|
||||
|
||||
return result
|
||||
```
|
||||
|
||||
### 📊 Use Case 3: Real-Time Streaming Dashboard
|
||||
|
||||
A monitoring agent streams responses and displays progress in real-time.
|
||||
|
||||
```typescript
|
||||
async function streamingDashboard(prompt: string) {
|
||||
const response = await fetch(`${BASE_URL}/a2a`, {
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json", Authorization: `Bearer ${API_KEY}` },
|
||||
body: JSON.stringify({
|
||||
jsonrpc: "2.0",
|
||||
id: "dash-1",
|
||||
method: "message/stream",
|
||||
params: { skill: "smart-routing", messages: [{ role: "user", content: prompt }] },
|
||||
}),
|
||||
});
|
||||
|
||||
let totalChunks = 0;
|
||||
const reader = response.body!.getReader();
|
||||
const decoder = new TextDecoder();
|
||||
|
||||
while (true) {
|
||||
const { done, value } = await reader.read();
|
||||
if (done) break;
|
||||
|
||||
for (const line of decoder.decode(value).split("\n")) {
|
||||
if (line.startsWith("data: ")) {
|
||||
const event = JSON.parse(line.slice(6));
|
||||
const state = event.params.task.state;
|
||||
|
||||
if (state === "working" && event.params.chunk) {
|
||||
totalChunks++;
|
||||
process.stdout.write(
|
||||
`\r[Chunk ${totalChunks}] ${event.params.chunk.content.slice(0, 50)}...`
|
||||
);
|
||||
}
|
||||
if (state === "completed") {
|
||||
const meta = event.params.metadata;
|
||||
console.log(
|
||||
`\n✅ Done | Cost: $${meta?.cost_envelope?.actual || 0} | Route: ${meta?.routing_explanation || "N/A"}`
|
||||
);
|
||||
}
|
||||
if (state === "failed") {
|
||||
console.error(`\n❌ Failed: ${event.params.metadata?.error}`);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 🔁 Use Case 4: Task Polling Pattern
|
||||
|
||||
For long-running tasks, poll the task status instead of waiting synchronously.
|
||||
|
||||
```python
|
||||
import time
|
||||
|
||||
def poll_task(task_id: str, timeout: int = 60):
|
||||
"""Poll task status until completion or timeout."""
|
||||
start = time.time()
|
||||
while time.time() - start < timeout:
|
||||
result = requests.post(f"{BASE_URL}/a2a", headers=HEADERS, json={
|
||||
"jsonrpc": "2.0",
|
||||
"id": "poll-1",
|
||||
"method": "tasks/get",
|
||||
"params": {"taskId": task_id},
|
||||
}).json()
|
||||
|
||||
task = result["result"]["task"]
|
||||
state = task["state"]
|
||||
print(f" Task {task_id[:8]}... state={state}")
|
||||
|
||||
if state in ("completed", "failed", "cancelled"):
|
||||
return task
|
||||
time.sleep(2)
|
||||
|
||||
# Timeout — cancel the task
|
||||
requests.post(f"{BASE_URL}/a2a", headers=HEADERS, json={
|
||||
"jsonrpc": "2.0",
|
||||
"id": "cancel-1",
|
||||
"method": "tasks/cancel",
|
||||
"params": {"taskId": task_id},
|
||||
})
|
||||
raise TimeoutError(f"Task {task_id} timed out after {timeout}s")
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Error Codes
|
||||
|
||||
| Code | Constant | Meaning |
|
||||
| ------ | ------------------------ | ---------------------------------------- |
|
||||
| -32700 | — | Parse error (invalid JSON) |
|
||||
| -32600 | `INVALID_REQUEST` | Invalid JSON-RPC request or unauthorized |
|
||||
| -32601 | `METHOD_NOT_FOUND` | Unknown method or skill |
|
||||
| -32602 | `INVALID_PARAMS` | Missing or invalid parameters |
|
||||
| -32603 | `INTERNAL_ERROR` | Skill execution failed |
|
||||
| -32001 | `TASK_NOT_FOUND` | Task ID not found |
|
||||
| -32002 | `TASK_ALREADY_COMPLETED` | Cannot modify a completed task |
|
||||
| -32003 | `UNAUTHORIZED` | Invalid or missing API key |
|
||||
| -32004 | `BUDGET_EXCEEDED` | Request exceeds configured budget |
|
||||
| -32005 | `PROVIDER_UNAVAILABLE` | No available providers |
|
||||
|
||||
---
|
||||
|
||||
## Authentication
|
||||
|
||||
All `/a2a` requests require a Bearer token via the `Authorization` header:
|
||||
|
||||
```
|
||||
Authorization: Bearer YOUR_OMNIROUTE_API_KEY
|
||||
```
|
||||
|
||||
If no API key is configured on the server (`OMNIROUTE_API_KEY` is empty), authentication is bypassed.
|
||||
|
||||
---
|
||||
|
||||
## File Structure
|
||||
|
||||
```
|
||||
src/lib/a2a/
|
||||
├── taskManager.ts # Task lifecycle (create/update/cancel/list), TTL, cleanup
|
||||
├── taskExecution.ts # Generic task executor with state management
|
||||
├── streaming.ts # SSE stream formatting, heartbeat, chunk/completion events
|
||||
├── routingLogger.ts # Routing decision logger (stats, history, retention)
|
||||
└── skills/
|
||||
├── smartRouting.ts # Smart routing skill (routes via /v1/chat/completions)
|
||||
└── quotaManagement.ts # Quota management skill (natural-language quota queries)
|
||||
|
||||
src/app/a2a/
|
||||
└── route.ts # Next.js API route handler (JSON-RPC 2.0 dispatch)
|
||||
|
||||
open-sse/mcp-server/
|
||||
└── schemas/a2a.ts # Zod schemas (AgentCard, Task, JSON-RPC, SSE events)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Comparison: MCP vs A2A
|
||||
|
||||
| Feature | MCP Server | A2A Server |
|
||||
| ----------------- | ---------------------------- | ------------------------------------------------- |
|
||||
| **Protocol** | Model Context Protocol | Agent-to-Agent Protocol v0.3 |
|
||||
| **Transport** | stdio / HTTP | HTTP (JSON-RPC 2.0) |
|
||||
| **Discovery** | Tool listing via MCP | `/.well-known/agent.json` |
|
||||
| **Granularity** | 16 individual tools | 2 high-level skills |
|
||||
| **Best for** | IDE agents (Cursor, VS Code) | Multi-agent systems (LangChain, CrewAI) |
|
||||
| **Streaming** | Not supported | SSE via `message/stream` |
|
||||
| **Task tracking** | No | Full lifecycle (submitted → completed) |
|
||||
| **Observability** | Audit log per tool call | Cost envelope + resilience trace + policy verdict |
|
||||
|
||||
---
|
||||
|
||||
## License
|
||||
|
||||
Part of [OmniRoute](https://github.com/diegosouzapw/OmniRoute) — MIT License.
|
||||
@@ -124,6 +124,11 @@ export function createA2AStream(
|
||||
controller.enqueue(encoder.encode(createChunkEvent(task.id, artifact.content)));
|
||||
}
|
||||
|
||||
if (abortSignal?.aborted) {
|
||||
controller.enqueue(encoder.encode(createFailureEvent(task.id, "Cancelled")));
|
||||
return;
|
||||
}
|
||||
|
||||
// Emit completion with metadata
|
||||
controller.enqueue(encoder.encode(createCompletionEvent(task.id, result.metadata)));
|
||||
} catch (err) {
|
||||
|
||||
@@ -13,6 +13,7 @@ type StreamTaskLike = {
|
||||
|
||||
type StreamTaskResult = {
|
||||
artifacts: Array<{ type: string; content: string }>;
|
||||
metadata: Record<string, unknown>;
|
||||
};
|
||||
|
||||
export async function executeA2ATaskWithState(
|
||||
|
||||
@@ -50,7 +50,7 @@ export interface A2ATask {
|
||||
// ============ Valid Transitions ============
|
||||
|
||||
const VALID_TRANSITIONS: Record<TaskState, TaskState[]> = {
|
||||
submitted: ["working", "cancelled"],
|
||||
submitted: ["working", "failed", "cancelled"],
|
||||
working: ["completed", "failed", "cancelled"],
|
||||
completed: [],
|
||||
failed: [],
|
||||
@@ -136,8 +136,14 @@ export class A2ATaskManager {
|
||||
private cleanupExpired() {
|
||||
const now = new Date();
|
||||
for (const [id, task] of this.tasks) {
|
||||
if (new Date(task.expiresAt) < now && task.state !== "completed" && task.state !== "failed") {
|
||||
if (
|
||||
new Date(task.expiresAt) < now &&
|
||||
task.state !== "completed" &&
|
||||
task.state !== "failed" &&
|
||||
task.state !== "cancelled"
|
||||
) {
|
||||
task.state = "failed";
|
||||
task.updatedAt = now.toISOString();
|
||||
task.events.push({ timestamp: now.toISOString(), state: "failed", message: "TTL expired" });
|
||||
}
|
||||
// Remove terminal tasks older than 2x TTL
|
||||
|
||||
@@ -9,6 +9,24 @@ import {
|
||||
const CLOUD_URL = process.env.CLOUD_URL || process.env.NEXT_PUBLIC_CLOUD_URL;
|
||||
const CLOUD_SYNC_TIMEOUT_MS = Number(process.env.CLOUD_SYNC_TIMEOUT_MS || 12000);
|
||||
|
||||
type JsonRecord = Record<string, unknown>;
|
||||
|
||||
function asRecord(value: unknown): JsonRecord {
|
||||
return value && typeof value === "object" && !Array.isArray(value) ? (value as JsonRecord) : {};
|
||||
}
|
||||
|
||||
function toStringOrNull(value: unknown): string | null {
|
||||
return typeof value === "string" && value.trim().length > 0 ? value : null;
|
||||
}
|
||||
|
||||
function toDateMs(value: unknown): number {
|
||||
if (typeof value === "string" || typeof value === "number" || value instanceof Date) {
|
||||
const parsed = new Date(value).getTime();
|
||||
return Number.isFinite(parsed) ? parsed : 0;
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
|
||||
export async function fetchWithTimeout(url, options = {}, timeoutMs = CLOUD_SYNC_TIMEOUT_MS) {
|
||||
const controller = new AbortController();
|
||||
const timeoutId = setTimeout(() => controller.abort(), timeoutMs);
|
||||
@@ -85,15 +103,20 @@ export async function syncToCloud(machineId, createdKey = null) {
|
||||
* Simple logic: if Cloud is newer, sync entire provider
|
||||
* cloudProviders is object keyed by provider ID
|
||||
*/
|
||||
async function updateLocalTokens(cloudProviders) {
|
||||
async function updateLocalTokens(cloudProviders: unknown) {
|
||||
const cloudProvidersMap = asRecord(cloudProviders);
|
||||
const localProviders = await getProviderConnections();
|
||||
|
||||
for (const localProvider of localProviders) {
|
||||
const cloudProvider = cloudProviders[localProvider.id];
|
||||
if (!cloudProvider) continue;
|
||||
for (const localProviderRaw of localProviders as unknown[]) {
|
||||
const localProvider = asRecord(localProviderRaw);
|
||||
const localProviderId = toStringOrNull(localProvider.id);
|
||||
if (!localProviderId) continue;
|
||||
|
||||
const cloudUpdatedAt = new Date(cloudProvider.updatedAt || 0).getTime();
|
||||
const localUpdatedAt = new Date(localProvider.updatedAt || 0).getTime();
|
||||
const cloudProvider = asRecord(cloudProvidersMap[localProviderId]);
|
||||
if (Object.keys(cloudProvider).length === 0) continue;
|
||||
|
||||
const cloudUpdatedAt = toDateMs(cloudProvider.updatedAt);
|
||||
const localUpdatedAt = toDateMs(localProvider.updatedAt);
|
||||
|
||||
// Simple logic: if Cloud is newer, sync entire provider
|
||||
if (cloudUpdatedAt > localUpdatedAt) {
|
||||
@@ -119,7 +142,7 @@ async function updateLocalTokens(cloudProviders) {
|
||||
updatedAt: cloudProvider.updatedAt,
|
||||
};
|
||||
|
||||
await updateProviderConnection(localProvider.id, updates);
|
||||
await updateProviderConnection(localProviderId, updates);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -6,24 +6,39 @@ import { v4 as uuidv4 } from "uuid";
|
||||
import { getDbInstance } from "./core";
|
||||
import { backupDbFile } from "./backup";
|
||||
|
||||
type JsonRecord = Record<string, unknown>;
|
||||
|
||||
function asRecord(value: unknown): JsonRecord {
|
||||
return value && typeof value === "object" && !Array.isArray(value) ? (value as JsonRecord) : {};
|
||||
}
|
||||
|
||||
function getSerializedData(value: unknown): string | null {
|
||||
const row = asRecord(value);
|
||||
return typeof row.data === "string" ? row.data : null;
|
||||
}
|
||||
|
||||
export async function getCombos() {
|
||||
const db = getDbInstance();
|
||||
return db
|
||||
.prepare("SELECT data FROM combos ORDER BY name")
|
||||
.all()
|
||||
.map((r) => JSON.parse(r.data));
|
||||
.map((row) => getSerializedData(row))
|
||||
.filter((row): row is string => row !== null)
|
||||
.map((row) => JSON.parse(row));
|
||||
}
|
||||
|
||||
export async function getComboById(id) {
|
||||
const db = getDbInstance();
|
||||
const row = db.prepare("SELECT data FROM combos WHERE id = ?").get(id);
|
||||
return row ? JSON.parse(row.data) : null;
|
||||
const payload = getSerializedData(row);
|
||||
return payload ? JSON.parse(payload) : null;
|
||||
}
|
||||
|
||||
export async function getComboByName(name) {
|
||||
const db = getDbInstance();
|
||||
const row = db.prepare("SELECT data FROM combos WHERE name = ?").get(name);
|
||||
return row ? JSON.parse(row.data) : null;
|
||||
const payload = getSerializedData(row);
|
||||
return payload ? JSON.parse(payload) : null;
|
||||
}
|
||||
|
||||
export async function createCombo(data) {
|
||||
@@ -53,7 +68,9 @@ export async function updateCombo(id, data) {
|
||||
const existing = db.prepare("SELECT data FROM combos WHERE id = ?").get(id);
|
||||
if (!existing) return null;
|
||||
|
||||
const current = JSON.parse(existing.data);
|
||||
const serializedCurrent = getSerializedData(existing);
|
||||
if (!serializedCurrent) return null;
|
||||
const current = JSON.parse(serializedCurrent);
|
||||
const merged = { ...current, ...data, updatedAt: new Date().toISOString() };
|
||||
|
||||
db.prepare("UPDATE combos SET name = ?, data = ?, updated_at = ? WHERE id = ?").run(
|
||||
|
||||
@@ -12,6 +12,21 @@
|
||||
|
||||
import { getDbInstance, isBuildPhase, isCloud } from "./core";
|
||||
|
||||
type JsonRecord = Record<string, unknown>;
|
||||
|
||||
function asRecord(value: unknown): JsonRecord {
|
||||
return value && typeof value === "object" && !Array.isArray(value) ? (value as JsonRecord) : {};
|
||||
}
|
||||
|
||||
function toNumber(value: unknown, fallback = 0): number {
|
||||
if (typeof value === "number" && Number.isFinite(value)) return value;
|
||||
if (typeof value === "string" && value.trim().length > 0) {
|
||||
const parsed = Number(value);
|
||||
return Number.isFinite(parsed) ? parsed : fallback;
|
||||
}
|
||||
return fallback;
|
||||
}
|
||||
|
||||
// ──────────────── Fallback Chains ────────────────
|
||||
|
||||
/**
|
||||
@@ -35,7 +50,8 @@ export function saveFallbackChain(model, chain) {
|
||||
export function loadFallbackChain(model) {
|
||||
const db = getDbInstance();
|
||||
const row = db.prepare("SELECT chain FROM domain_fallback_chains WHERE model = ?").get(model);
|
||||
return row ? JSON.parse(row.chain) : null;
|
||||
const chain = asRecord(row).chain;
|
||||
return typeof chain === "string" ? JSON.parse(chain) : null;
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -45,9 +61,13 @@ export function loadFallbackChain(model) {
|
||||
export function loadAllFallbackChains() {
|
||||
const db = getDbInstance();
|
||||
const rows = db.prepare("SELECT model, chain FROM domain_fallback_chains").all();
|
||||
const result = {};
|
||||
const result: Record<string, unknown> = {};
|
||||
for (const row of rows) {
|
||||
result[row.model] = JSON.parse(row.chain);
|
||||
const record = asRecord(row);
|
||||
const model = typeof record.model === "string" ? record.model : null;
|
||||
const chain = typeof record.chain === "string" ? record.chain : null;
|
||||
if (!model || !chain) continue;
|
||||
result[model] = JSON.parse(chain);
|
||||
}
|
||||
return result;
|
||||
}
|
||||
@@ -99,11 +119,12 @@ export function saveBudget(apiKeyId, config) {
|
||||
export function loadBudget(apiKeyId) {
|
||||
const db = getDbInstance();
|
||||
const row = db.prepare("SELECT * FROM domain_budgets WHERE api_key_id = ?").get(apiKeyId);
|
||||
const record = asRecord(row);
|
||||
if (!row) return null;
|
||||
return {
|
||||
dailyLimitUsd: row.daily_limit_usd,
|
||||
monthlyLimitUsd: row.monthly_limit_usd,
|
||||
warningThreshold: row.warning_threshold,
|
||||
dailyLimitUsd: toNumber(record.daily_limit_usd),
|
||||
monthlyLimitUsd: toNumber(record.monthly_limit_usd),
|
||||
warningThreshold: toNumber(record.warning_threshold, 0.8),
|
||||
};
|
||||
}
|
||||
|
||||
@@ -203,9 +224,12 @@ export function loadLockoutState(identifier) {
|
||||
const db = getDbInstance();
|
||||
const row = db.prepare("SELECT * FROM domain_lockout_state WHERE identifier = ?").get(identifier);
|
||||
if (!row) return null;
|
||||
const record = asRecord(row);
|
||||
const attemptsRaw = typeof record.attempts === "string" ? record.attempts : "[]";
|
||||
const lockedUntilRaw = record.locked_until;
|
||||
return {
|
||||
attempts: JSON.parse(row.attempts),
|
||||
lockedUntil: row.locked_until,
|
||||
attempts: JSON.parse(attemptsRaw),
|
||||
lockedUntil: typeof lockedUntilRaw === "number" ? lockedUntilRaw : null,
|
||||
};
|
||||
}
|
||||
|
||||
@@ -230,10 +254,14 @@ export function loadAllLockedIdentifiers() {
|
||||
"SELECT identifier, locked_until FROM domain_lockout_state WHERE locked_until IS NOT NULL AND locked_until > ?"
|
||||
)
|
||||
.all(now)
|
||||
.map((row) => ({
|
||||
identifier: row.identifier,
|
||||
lockedUntil: row.locked_until,
|
||||
}));
|
||||
.map((row) => {
|
||||
const record = asRecord(row);
|
||||
return {
|
||||
identifier: typeof record.identifier === "string" ? record.identifier : "",
|
||||
lockedUntil: toNumber(record.locked_until),
|
||||
};
|
||||
})
|
||||
.filter((row) => row.identifier.length > 0);
|
||||
}
|
||||
|
||||
// ──────────────── Circuit Breakers ────────────────
|
||||
@@ -266,11 +294,13 @@ export function loadCircuitBreakerState(name) {
|
||||
const db = getDbInstance();
|
||||
const row = db.prepare("SELECT * FROM domain_circuit_breakers WHERE name = ?").get(name);
|
||||
if (!row) return null;
|
||||
const record = asRecord(row);
|
||||
const options = typeof record.options === "string" ? JSON.parse(record.options) : null;
|
||||
return {
|
||||
state: row.state,
|
||||
failureCount: row.failure_count,
|
||||
lastFailureTime: row.last_failure_time,
|
||||
options: row.options ? JSON.parse(row.options) : null,
|
||||
state: typeof record.state === "string" ? record.state : "CLOSED",
|
||||
failureCount: toNumber(record.failure_count),
|
||||
lastFailureTime: toNumber(record.last_failure_time, 0) || null,
|
||||
options,
|
||||
};
|
||||
}
|
||||
|
||||
@@ -283,12 +313,16 @@ export function loadAllCircuitBreakerStates() {
|
||||
return db
|
||||
.prepare("SELECT name, state, failure_count, last_failure_time FROM domain_circuit_breakers")
|
||||
.all()
|
||||
.map((row) => ({
|
||||
name: row.name,
|
||||
state: row.state,
|
||||
failureCount: row.failure_count,
|
||||
lastFailureTime: row.last_failure_time,
|
||||
}));
|
||||
.map((row) => {
|
||||
const record = asRecord(row);
|
||||
return {
|
||||
name: typeof record.name === "string" ? record.name : "",
|
||||
state: typeof record.state === "string" ? record.state : "CLOSED",
|
||||
failureCount: toNumber(record.failure_count),
|
||||
lastFailureTime: toNumber(record.last_failure_time, 0) || null,
|
||||
};
|
||||
})
|
||||
.filter((row) => row.name.length > 0);
|
||||
}
|
||||
|
||||
/**
|
||||
|
||||
@@ -5,6 +5,20 @@
|
||||
import { getDbInstance } from "./core";
|
||||
import { backupDbFile } from "./backup";
|
||||
|
||||
type JsonRecord = Record<string, unknown>;
|
||||
|
||||
function asRecord(value: unknown): JsonRecord {
|
||||
return value && typeof value === "object" && !Array.isArray(value) ? (value as JsonRecord) : {};
|
||||
}
|
||||
|
||||
function getKeyValue(row: unknown): { key: string | null; value: string | null } {
|
||||
const record = asRecord(row);
|
||||
return {
|
||||
key: typeof record.key === "string" ? record.key : null,
|
||||
value: typeof record.value === "string" ? record.value : null,
|
||||
};
|
||||
}
|
||||
|
||||
// ──────────────── Model Aliases ────────────────
|
||||
|
||||
export async function getModelAliases() {
|
||||
@@ -12,9 +26,11 @@ export async function getModelAliases() {
|
||||
const rows = db
|
||||
.prepare("SELECT key, value FROM key_value WHERE namespace = 'modelAliases'")
|
||||
.all();
|
||||
const result = {};
|
||||
const result: Record<string, unknown> = {};
|
||||
for (const row of rows) {
|
||||
result[row.key] = JSON.parse(row.value);
|
||||
const { key, value } = getKeyValue(row);
|
||||
if (!key || value === null) continue;
|
||||
result[key] = JSON.parse(value);
|
||||
}
|
||||
return result;
|
||||
}
|
||||
@@ -41,12 +57,15 @@ export async function getMitmAlias(toolName) {
|
||||
const row = db
|
||||
.prepare("SELECT value FROM key_value WHERE namespace = 'mitmAlias' AND key = ?")
|
||||
.get(toolName);
|
||||
return row ? JSON.parse(row.value) : {};
|
||||
const value = getKeyValue(row).value;
|
||||
return value ? JSON.parse(value) : {};
|
||||
}
|
||||
const rows = db.prepare("SELECT key, value FROM key_value WHERE namespace = 'mitmAlias'").all();
|
||||
const result = {};
|
||||
const result: Record<string, unknown> = {};
|
||||
for (const row of rows) {
|
||||
result[row.key] = JSON.parse(row.value);
|
||||
const { key, value } = getKeyValue(row);
|
||||
if (!key || value === null) continue;
|
||||
result[key] = JSON.parse(value);
|
||||
}
|
||||
return result;
|
||||
}
|
||||
@@ -67,13 +86,18 @@ export async function getCustomModels(providerId) {
|
||||
const row = db
|
||||
.prepare("SELECT value FROM key_value WHERE namespace = 'customModels' AND key = ?")
|
||||
.get(providerId);
|
||||
return row ? JSON.parse(row.value) : [];
|
||||
const value = getKeyValue(row).value;
|
||||
return value ? JSON.parse(value) : [];
|
||||
}
|
||||
const rows = db
|
||||
.prepare("SELECT key, value FROM key_value WHERE namespace = 'customModels'")
|
||||
.all();
|
||||
const result = {};
|
||||
for (const row of rows) result[row.key] = JSON.parse(row.value);
|
||||
const result: Record<string, unknown> = {};
|
||||
for (const row of rows) {
|
||||
const { key, value } = getKeyValue(row);
|
||||
if (!key || value === null) continue;
|
||||
result[key] = JSON.parse(value);
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
@@ -82,8 +106,12 @@ export async function getAllCustomModels() {
|
||||
const rows = db
|
||||
.prepare("SELECT key, value FROM key_value WHERE namespace = 'customModels'")
|
||||
.all();
|
||||
const result = {};
|
||||
for (const row of rows) result[row.key] = JSON.parse(row.value);
|
||||
const result: Record<string, unknown> = {};
|
||||
for (const row of rows) {
|
||||
const { key, value } = getKeyValue(row);
|
||||
if (!key || value === null) continue;
|
||||
result[key] = JSON.parse(value);
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
@@ -92,7 +120,8 @@ export async function addCustomModel(providerId, modelId, modelName, source = "m
|
||||
const row = db
|
||||
.prepare("SELECT value FROM key_value WHERE namespace = 'customModels' AND key = ?")
|
||||
.get(providerId);
|
||||
const models = row ? JSON.parse(row.value) : [];
|
||||
const value = getKeyValue(row).value;
|
||||
const models = value ? JSON.parse(value) : [];
|
||||
|
||||
const exists = models.find((m) => m.id === modelId);
|
||||
if (exists) return exists;
|
||||
@@ -113,7 +142,9 @@ export async function removeCustomModel(providerId, modelId) {
|
||||
.get(providerId);
|
||||
if (!row) return false;
|
||||
|
||||
const models = JSON.parse(row.value);
|
||||
const value = getKeyValue(row).value;
|
||||
if (!value) return false;
|
||||
const models = JSON.parse(value);
|
||||
const before = models.length;
|
||||
const filtered = models.filter((m) => m.id !== modelId);
|
||||
|
||||
|
||||
@@ -29,7 +29,7 @@ function toProxyMap(value: unknown): ProxyMap {
|
||||
}
|
||||
|
||||
function toProxyValue(value: unknown): ProxyValue {
|
||||
if (value === null || typeof value === "string") return value;
|
||||
if (value === null || typeof value === "string") return value as string | null;
|
||||
if (value && typeof value === "object") return value as JsonRecord;
|
||||
return null;
|
||||
}
|
||||
|
||||
@@ -14,6 +14,21 @@ import crypto from "node:crypto";
|
||||
import { LRUCache } from "./cacheLayer";
|
||||
import { getDbInstance } from "./db/core";
|
||||
|
||||
type JsonRecord = Record<string, unknown>;
|
||||
|
||||
function asRecord(value: unknown): JsonRecord {
|
||||
return value && typeof value === "object" && !Array.isArray(value) ? (value as JsonRecord) : {};
|
||||
}
|
||||
|
||||
function toNumber(value: unknown, fallback = 0): number {
|
||||
if (typeof value === "number" && Number.isFinite(value)) return value;
|
||||
if (typeof value === "string" && value.trim().length > 0) {
|
||||
const parsed = Number(value);
|
||||
return Number.isFinite(parsed) ? parsed : fallback;
|
||||
}
|
||||
return fallback;
|
||||
}
|
||||
|
||||
// ─── Singleton ─────────────────
|
||||
|
||||
let memoryCache: LRUCache | null = null;
|
||||
@@ -89,11 +104,18 @@ export function getCachedResponse(signature) {
|
||||
.get(signature);
|
||||
|
||||
if (row) {
|
||||
const parsed = JSON.parse(row.response);
|
||||
const record = asRecord(row);
|
||||
const responsePayload = typeof record.response === "string" ? record.response : null;
|
||||
if (!responsePayload) {
|
||||
stats.misses++;
|
||||
return null;
|
||||
}
|
||||
const parsed = JSON.parse(responsePayload);
|
||||
const tokensSaved = toNumber(record.tokens_saved, 0);
|
||||
// Promote to memory cache
|
||||
getMemoryCache().set(signature, {
|
||||
response: parsed,
|
||||
tokensSaved: row.tokens_saved,
|
||||
tokensSaved,
|
||||
});
|
||||
// Update hit count in DB
|
||||
db.prepare("UPDATE semantic_cache SET hit_count = hit_count + 1 WHERE signature = ?").run(
|
||||
@@ -101,7 +123,7 @@ export function getCachedResponse(signature) {
|
||||
);
|
||||
|
||||
stats.hits++;
|
||||
stats.tokensSaved += row.tokens_saved || 0;
|
||||
stats.tokensSaved += tokensSaved;
|
||||
return parsed;
|
||||
}
|
||||
} catch {
|
||||
@@ -171,9 +193,7 @@ export function invalidateByModel(model: string): number {
|
||||
getMemoryCache().clear(); // Memory cache doesn't track model; full clear
|
||||
try {
|
||||
const db = getDbInstance();
|
||||
const result = db
|
||||
.prepare("DELETE FROM semantic_cache WHERE model = ?")
|
||||
.run(model);
|
||||
const result = db.prepare("DELETE FROM semantic_cache WHERE model = ?").run(model);
|
||||
return result.changes || 0;
|
||||
} catch {
|
||||
return 0;
|
||||
@@ -189,9 +209,7 @@ export function invalidateBySignature(signature: string): boolean {
|
||||
getMemoryCache().delete(signature);
|
||||
try {
|
||||
const db = getDbInstance();
|
||||
const result = db
|
||||
.prepare("DELETE FROM semantic_cache WHERE signature = ?")
|
||||
.run(signature);
|
||||
const result = db.prepare("DELETE FROM semantic_cache WHERE signature = ?").run(signature);
|
||||
return (result.changes || 0) > 0;
|
||||
} catch {
|
||||
return false;
|
||||
@@ -208,9 +226,7 @@ export function invalidateStale(maxAgeMs: number): number {
|
||||
try {
|
||||
const db = getDbInstance();
|
||||
const cutoff = new Date(Date.now() - maxAgeMs).toISOString();
|
||||
const result = db
|
||||
.prepare("DELETE FROM semantic_cache WHERE created_at < ?")
|
||||
.run(cutoff);
|
||||
const result = db.prepare("DELETE FROM semantic_cache WHERE created_at < ?").run(cutoff);
|
||||
return result.changes || 0;
|
||||
} catch {
|
||||
return 0;
|
||||
@@ -272,7 +288,7 @@ export function getCacheStats() {
|
||||
const row = db
|
||||
.prepare("SELECT COUNT(*) as count FROM semantic_cache WHERE expires_at > datetime('now')")
|
||||
.get();
|
||||
dbSize = row?.count || 0;
|
||||
dbSize = toNumber(asRecord(row).count, 0);
|
||||
} catch {
|
||||
// DB not available
|
||||
}
|
||||
|
||||
@@ -14,6 +14,39 @@ import { shouldPersistToDisk, CALL_LOGS_DIR } from "./migrations";
|
||||
import { isNoLog } from "../compliance";
|
||||
import { sanitizePII } from "../piiSanitizer";
|
||||
|
||||
type JsonRecord = Record<string, unknown>;
|
||||
|
||||
function asRecord(value: unknown): JsonRecord {
|
||||
return value && typeof value === "object" && !Array.isArray(value) ? (value as JsonRecord) : {};
|
||||
}
|
||||
|
||||
function toNumber(value: unknown): number {
|
||||
if (typeof value === "number" && Number.isFinite(value)) return value;
|
||||
if (typeof value === "string" && value.trim().length > 0) {
|
||||
const parsed = Number(value);
|
||||
return Number.isFinite(parsed) ? parsed : 0;
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
|
||||
function toStringOrNull(value: unknown): string | null {
|
||||
return typeof value === "string" ? value : null;
|
||||
}
|
||||
|
||||
function parseJsonString(value: unknown): unknown | null {
|
||||
if (typeof value !== "string" || value.trim().length === 0) return null;
|
||||
try {
|
||||
return JSON.parse(value);
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
function hasTruncatedFlag(value: unknown): boolean {
|
||||
if (!value || typeof value !== "object" || Array.isArray(value)) return false;
|
||||
return (value as Record<string, unknown>)._truncated === true;
|
||||
}
|
||||
|
||||
const CALL_LOGS_MAX = parseInt(process.env.CALL_LOGS_MAX || "200", 10);
|
||||
const LOG_RETENTION_DAYS = parseInt(process.env.LOG_RETENTION_DAYS || "7", 10);
|
||||
const CALL_LOG_PAYLOAD_MODE = (() => {
|
||||
@@ -177,7 +210,8 @@ export async function saveCallLog(entry: any) {
|
||||
).run(logEntry);
|
||||
|
||||
// 2. Trim old entries beyond CALL_LOGS_MAX
|
||||
const count = db.prepare("SELECT COUNT(*) as cnt FROM call_logs").get()?.cnt || 0;
|
||||
const countRow = asRecord(db.prepare("SELECT COUNT(*) as cnt FROM call_logs").get());
|
||||
const count = toNumber(countRow.cnt);
|
||||
if (count > CALL_LOGS_MAX) {
|
||||
db.prepare(
|
||||
`
|
||||
@@ -326,26 +360,29 @@ export async function getCallLogs(filter: any = {}) {
|
||||
|
||||
const rows = db.prepare(sql).all(params);
|
||||
|
||||
return rows.map((l) => ({
|
||||
id: l.id,
|
||||
timestamp: l.timestamp,
|
||||
method: l.method,
|
||||
path: l.path,
|
||||
status: l.status,
|
||||
model: l.model,
|
||||
provider: l.provider,
|
||||
account: l.account,
|
||||
duration: l.duration,
|
||||
tokens: { in: l.tokens_in, out: l.tokens_out },
|
||||
sourceFormat: l.source_format,
|
||||
targetFormat: l.target_format,
|
||||
error: l.error,
|
||||
comboName: l.combo_name || null,
|
||||
apiKeyId: l.api_key_id || null,
|
||||
apiKeyName: l.api_key_name || null,
|
||||
hasRequestBody: !!l.request_body,
|
||||
hasResponseBody: !!l.response_body,
|
||||
}));
|
||||
return rows.map((row) => {
|
||||
const l = asRecord(row);
|
||||
return {
|
||||
id: toStringOrNull(l.id),
|
||||
timestamp: toStringOrNull(l.timestamp),
|
||||
method: toStringOrNull(l.method),
|
||||
path: toStringOrNull(l.path),
|
||||
status: toNumber(l.status),
|
||||
model: toStringOrNull(l.model),
|
||||
provider: toStringOrNull(l.provider),
|
||||
account: toStringOrNull(l.account),
|
||||
duration: toNumber(l.duration),
|
||||
tokens: { in: toNumber(l.tokens_in), out: toNumber(l.tokens_out) },
|
||||
sourceFormat: toStringOrNull(l.source_format),
|
||||
targetFormat: toStringOrNull(l.target_format),
|
||||
error: toStringOrNull(l.error),
|
||||
comboName: toStringOrNull(l.combo_name),
|
||||
apiKeyId: toStringOrNull(l.api_key_id),
|
||||
apiKeyName: toStringOrNull(l.api_key_name),
|
||||
hasRequestBody: typeof l.request_body === "string" && l.request_body.length > 0,
|
||||
hasResponseBody: typeof l.response_body === "string" && l.response_body.length > 0,
|
||||
};
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -355,31 +392,32 @@ export async function getCallLogById(id: string) {
|
||||
const db = getDbInstance();
|
||||
const row = db.prepare("SELECT * FROM call_logs WHERE id = ?").get(id);
|
||||
if (!row) return null;
|
||||
const entryRow = asRecord(row);
|
||||
|
||||
const entry = {
|
||||
id: row.id,
|
||||
timestamp: row.timestamp,
|
||||
method: row.method,
|
||||
path: row.path,
|
||||
status: row.status,
|
||||
model: row.model,
|
||||
provider: row.provider,
|
||||
account: row.account,
|
||||
connectionId: row.connection_id,
|
||||
duration: row.duration,
|
||||
tokens: { in: row.tokens_in, out: row.tokens_out },
|
||||
sourceFormat: row.source_format,
|
||||
targetFormat: row.target_format,
|
||||
apiKeyId: row.api_key_id,
|
||||
apiKeyName: row.api_key_name,
|
||||
comboName: row.combo_name,
|
||||
requestBody: row.request_body ? JSON.parse(row.request_body) : null,
|
||||
responseBody: row.response_body ? JSON.parse(row.response_body) : null,
|
||||
error: row.error,
|
||||
id: toStringOrNull(entryRow.id),
|
||||
timestamp: toStringOrNull(entryRow.timestamp),
|
||||
method: toStringOrNull(entryRow.method),
|
||||
path: toStringOrNull(entryRow.path),
|
||||
status: toNumber(entryRow.status),
|
||||
model: toStringOrNull(entryRow.model),
|
||||
provider: toStringOrNull(entryRow.provider),
|
||||
account: toStringOrNull(entryRow.account),
|
||||
connectionId: toStringOrNull(entryRow.connection_id),
|
||||
duration: toNumber(entryRow.duration),
|
||||
tokens: { in: toNumber(entryRow.tokens_in), out: toNumber(entryRow.tokens_out) },
|
||||
sourceFormat: toStringOrNull(entryRow.source_format),
|
||||
targetFormat: toStringOrNull(entryRow.target_format),
|
||||
apiKeyId: toStringOrNull(entryRow.api_key_id),
|
||||
apiKeyName: toStringOrNull(entryRow.api_key_name),
|
||||
comboName: toStringOrNull(entryRow.combo_name),
|
||||
requestBody: parseJsonString(entryRow.request_body),
|
||||
responseBody: parseJsonString(entryRow.response_body),
|
||||
error: toStringOrNull(entryRow.error),
|
||||
};
|
||||
|
||||
// If payloads were truncated, try to read full version from disk
|
||||
const needsDisk = entry.requestBody?._truncated || entry.responseBody?._truncated;
|
||||
const needsDisk = hasTruncatedFlag(entry.requestBody) || hasTruncatedFlag(entry.responseBody);
|
||||
if (needsDisk && CALL_LOGS_DIR) {
|
||||
try {
|
||||
const diskEntry = readFullLogFromDisk(entry);
|
||||
|
||||
@@ -24,6 +24,15 @@ function normalizeModelName(model) {
|
||||
return parts[parts.length - 1];
|
||||
}
|
||||
|
||||
function toNumber(value: unknown, fallback = 0): number {
|
||||
if (typeof value === "number" && Number.isFinite(value)) return value;
|
||||
if (typeof value === "string" && value.trim().length > 0) {
|
||||
const parsed = Number(value);
|
||||
return Number.isFinite(parsed) ? parsed : fallback;
|
||||
}
|
||||
return fallback;
|
||||
}
|
||||
|
||||
/**
|
||||
* Calculate cost for a usage entry.
|
||||
*
|
||||
@@ -48,29 +57,39 @@ export async function calculateCost(provider, model, tokens) {
|
||||
}
|
||||
if (!pricing) return 0;
|
||||
|
||||
const pricingRecord =
|
||||
pricing && typeof pricing === "object" && !Array.isArray(pricing)
|
||||
? (pricing as Record<string, unknown>)
|
||||
: {};
|
||||
const inputPrice = toNumber(pricingRecord.input, 0);
|
||||
const cachedPrice = toNumber(pricingRecord.cached, inputPrice);
|
||||
const outputPrice = toNumber(pricingRecord.output, 0);
|
||||
const reasoningPrice = toNumber(pricingRecord.reasoning, outputPrice);
|
||||
const cacheCreationPrice = toNumber(pricingRecord.cache_creation, inputPrice);
|
||||
|
||||
let cost = 0;
|
||||
|
||||
const inputTokens = tokens.input ?? tokens.prompt_tokens ?? tokens.input_tokens ?? 0;
|
||||
const cachedTokens =
|
||||
tokens.cacheRead ?? tokens.cached_tokens ?? tokens.cache_read_input_tokens ?? 0;
|
||||
const nonCachedInput = Math.max(0, inputTokens - cachedTokens);
|
||||
cost += nonCachedInput * (pricing.input / 1000000);
|
||||
cost += nonCachedInput * (inputPrice / 1000000);
|
||||
|
||||
if (cachedTokens > 0) {
|
||||
cost += cachedTokens * ((pricing.cached || pricing.input) / 1000000);
|
||||
cost += cachedTokens * (cachedPrice / 1000000);
|
||||
}
|
||||
|
||||
const outputTokens = tokens.output ?? tokens.completion_tokens ?? tokens.output_tokens ?? 0;
|
||||
cost += outputTokens * (pricing.output / 1000000);
|
||||
cost += outputTokens * (outputPrice / 1000000);
|
||||
|
||||
const reasoningTokens = tokens.reasoning ?? tokens.reasoning_tokens ?? 0;
|
||||
if (reasoningTokens > 0) {
|
||||
cost += reasoningTokens * ((pricing.reasoning || pricing.output) / 1000000);
|
||||
cost += reasoningTokens * (reasoningPrice / 1000000);
|
||||
}
|
||||
|
||||
const cacheCreationTokens = tokens.cacheCreation ?? tokens.cache_creation_input_tokens ?? 0;
|
||||
if (cacheCreationTokens > 0) {
|
||||
cost += cacheCreationTokens * ((pricing.cache_creation || pricing.input) / 1000000);
|
||||
cost += cacheCreationTokens * (cacheCreationPrice / 1000000);
|
||||
}
|
||||
|
||||
return cost;
|
||||
|
||||
@@ -10,6 +10,25 @@
|
||||
import { getDbInstance } from "../db/core";
|
||||
import { shouldPersistToDisk } from "./migrations";
|
||||
|
||||
type JsonRecord = Record<string, unknown>;
|
||||
|
||||
function asRecord(value: unknown): JsonRecord {
|
||||
return value && typeof value === "object" && !Array.isArray(value) ? (value as JsonRecord) : {};
|
||||
}
|
||||
|
||||
function toStringOrNull(value: unknown): string | null {
|
||||
return typeof value === "string" && value.trim().length > 0 ? value : null;
|
||||
}
|
||||
|
||||
function toNumber(value: unknown): number {
|
||||
if (typeof value === "number" && Number.isFinite(value)) return value;
|
||||
if (typeof value === "string" && value.trim().length > 0) {
|
||||
const parsed = Number(value);
|
||||
return Number.isFinite(parsed) ? parsed : 0;
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
|
||||
// ──────────────── Pending Requests (in-memory) ────────────────
|
||||
|
||||
const pendingRequests: {
|
||||
@@ -23,7 +42,12 @@ const pendingRequests: {
|
||||
/**
|
||||
* Track a pending request.
|
||||
*/
|
||||
export function trackPendingRequest(model: string, provider: string, connectionId: string | null, started: boolean) {
|
||||
export function trackPendingRequest(
|
||||
model: string,
|
||||
provider: string,
|
||||
connectionId: string | null,
|
||||
started: boolean
|
||||
) {
|
||||
const modelKey = provider ? `${model} (${provider})` : model;
|
||||
|
||||
if (!pendingRequests.byModel[modelKey]) pendingRequests.byModel[modelKey] = 0;
|
||||
@@ -61,22 +85,25 @@ export async function getUsageDb() {
|
||||
const db = getDbInstance();
|
||||
const rows = db.prepare("SELECT * FROM usage_history ORDER BY timestamp ASC").all();
|
||||
|
||||
const history = rows.map((r) => ({
|
||||
provider: r.provider,
|
||||
model: r.model,
|
||||
connectionId: r.connection_id,
|
||||
apiKeyId: r.api_key_id,
|
||||
apiKeyName: r.api_key_name,
|
||||
tokens: {
|
||||
input: r.tokens_input,
|
||||
output: r.tokens_output,
|
||||
cacheRead: r.tokens_cache_read,
|
||||
cacheCreation: r.tokens_cache_creation,
|
||||
reasoning: r.tokens_reasoning,
|
||||
},
|
||||
status: r.status,
|
||||
timestamp: r.timestamp,
|
||||
}));
|
||||
const history = rows.map((row) => {
|
||||
const r = asRecord(row);
|
||||
return {
|
||||
provider: toStringOrNull(r.provider),
|
||||
model: toStringOrNull(r.model),
|
||||
connectionId: toStringOrNull(r.connection_id),
|
||||
apiKeyId: toStringOrNull(r.api_key_id),
|
||||
apiKeyName: toStringOrNull(r.api_key_name),
|
||||
tokens: {
|
||||
input: toNumber(r.tokens_input),
|
||||
output: toNumber(r.tokens_output),
|
||||
cacheRead: toNumber(r.tokens_cache_read),
|
||||
cacheCreation: toNumber(r.tokens_cache_creation),
|
||||
reasoning: toNumber(r.tokens_reasoning),
|
||||
},
|
||||
status: toStringOrNull(r.status),
|
||||
timestamp: toStringOrNull(r.timestamp),
|
||||
};
|
||||
});
|
||||
|
||||
return { data: { history } };
|
||||
}
|
||||
@@ -153,22 +180,25 @@ export async function getUsageHistory(filter: any = {}) {
|
||||
sql += " ORDER BY timestamp ASC";
|
||||
|
||||
const rows = db.prepare(sql).all(params);
|
||||
return rows.map((r) => ({
|
||||
provider: r.provider,
|
||||
model: r.model,
|
||||
connectionId: r.connection_id,
|
||||
apiKeyId: r.api_key_id,
|
||||
apiKeyName: r.api_key_name,
|
||||
tokens: {
|
||||
input: r.tokens_input,
|
||||
output: r.tokens_output,
|
||||
cacheRead: r.tokens_cache_read,
|
||||
cacheCreation: r.tokens_cache_creation,
|
||||
reasoning: r.tokens_reasoning,
|
||||
},
|
||||
status: r.status,
|
||||
timestamp: r.timestamp,
|
||||
}));
|
||||
return rows.map((row) => {
|
||||
const r = asRecord(row);
|
||||
return {
|
||||
provider: toStringOrNull(r.provider),
|
||||
model: toStringOrNull(r.model),
|
||||
connectionId: toStringOrNull(r.connection_id),
|
||||
apiKeyId: toStringOrNull(r.api_key_id),
|
||||
apiKeyName: toStringOrNull(r.api_key_name),
|
||||
tokens: {
|
||||
input: toNumber(r.tokens_input),
|
||||
output: toNumber(r.tokens_output),
|
||||
cacheRead: toNumber(r.tokens_cache_read),
|
||||
cacheCreation: toNumber(r.tokens_cache_creation),
|
||||
reasoning: toNumber(r.tokens_reasoning),
|
||||
},
|
||||
status: toStringOrNull(r.status),
|
||||
timestamp: toStringOrNull(r.timestamp),
|
||||
};
|
||||
});
|
||||
}
|
||||
|
||||
// ──────────────── Request Log (log.txt) ────────────────
|
||||
@@ -190,7 +220,19 @@ function formatLogDate(date = new Date()) {
|
||||
/**
|
||||
* Append to log.txt.
|
||||
*/
|
||||
export async function appendRequestLog({ model, provider, connectionId, tokens, status }: { model?: string; provider?: string; connectionId?: string; tokens?: any; status?: string | number }) {
|
||||
export async function appendRequestLog({
|
||||
model,
|
||||
provider,
|
||||
connectionId,
|
||||
tokens,
|
||||
status,
|
||||
}: {
|
||||
model?: string;
|
||||
provider?: string;
|
||||
connectionId?: string;
|
||||
tokens?: any;
|
||||
status?: string | number;
|
||||
}) {
|
||||
if (!shouldPersistToDisk) return;
|
||||
|
||||
try {
|
||||
@@ -202,8 +244,11 @@ export async function appendRequestLog({ model, provider, connectionId, tokens,
|
||||
try {
|
||||
const { getProviderConnections } = await import("@/lib/localDb");
|
||||
const connections = await getProviderConnections();
|
||||
const conn = connections.find((c) => c.id === connectionId);
|
||||
if (conn) account = conn.name || conn.email || account;
|
||||
const connRaw = connections.find((c) => asRecord(c).id === connectionId);
|
||||
if (connRaw) {
|
||||
const conn = asRecord(connRaw);
|
||||
account = toStringOrNull(conn.name) || toStringOrNull(conn.email) || account;
|
||||
}
|
||||
} catch {}
|
||||
|
||||
const sent =
|
||||
|
||||
@@ -11,22 +11,46 @@ import { getDbInstance } from "../db/core";
|
||||
import { getPendingRequests } from "./usageHistory";
|
||||
import { calculateCost } from "./costCalculator";
|
||||
|
||||
type JsonRecord = Record<string, unknown>;
|
||||
|
||||
function asRecord(value: unknown): JsonRecord {
|
||||
return value && typeof value === "object" && !Array.isArray(value) ? (value as JsonRecord) : {};
|
||||
}
|
||||
|
||||
function toNumber(value: unknown): number {
|
||||
if (typeof value === "number" && Number.isFinite(value)) return value;
|
||||
if (typeof value === "string" && value.trim().length > 0) {
|
||||
const parsed = Number(value);
|
||||
return Number.isFinite(parsed) ? parsed : 0;
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
|
||||
function toStringOrEmpty(value: unknown): string {
|
||||
return typeof value === "string" ? value : "";
|
||||
}
|
||||
|
||||
/**
|
||||
* Get aggregated usage stats.
|
||||
*/
|
||||
export async function getUsageStats() {
|
||||
const db = getDbInstance();
|
||||
const rows = db.prepare("SELECT * FROM usage_history ORDER BY timestamp ASC").all();
|
||||
const rows = db.prepare("SELECT * FROM usage_history ORDER BY timestamp ASC").all() as unknown[];
|
||||
|
||||
const { getProviderConnections } = await import("@/lib/localDb");
|
||||
let allConnections = [];
|
||||
let allConnections: unknown[] = [];
|
||||
try {
|
||||
allConnections = await getProviderConnections();
|
||||
const loadedConnections = await getProviderConnections();
|
||||
allConnections = Array.isArray(loadedConnections) ? loadedConnections : [];
|
||||
} catch {}
|
||||
|
||||
const connectionMap = {};
|
||||
for (const conn of allConnections) {
|
||||
connectionMap[conn.id] = conn.name || conn.email || conn.id;
|
||||
const connectionMap: Record<string, string> = {};
|
||||
for (const connRaw of allConnections) {
|
||||
const conn = asRecord(connRaw);
|
||||
const connectionId = toStringOrEmpty(conn.id);
|
||||
if (!connectionId) continue;
|
||||
connectionMap[connectionId] =
|
||||
toStringOrEmpty(conn.name) || toStringOrEmpty(conn.email) || connectionId;
|
||||
}
|
||||
|
||||
const pendingRequests = getPendingRequests();
|
||||
@@ -75,19 +99,27 @@ export async function getUsageStats() {
|
||||
|
||||
const tenMinutesAgo = new Date(currentMinuteStart.getTime() - 9 * 60 * 1000);
|
||||
|
||||
for (const row of rows) {
|
||||
const promptTokens = row.tokens_input || 0;
|
||||
const completionTokens = row.tokens_output || 0;
|
||||
const entryTime = new Date(row.timestamp);
|
||||
for (const rowRaw of rows) {
|
||||
const row = asRecord(rowRaw);
|
||||
const provider = toStringOrEmpty(row.provider) || "unknown";
|
||||
const model = toStringOrEmpty(row.model) || "unknown";
|
||||
const timestamp = toStringOrEmpty(row.timestamp) || new Date(0).toISOString();
|
||||
const connectionId = toStringOrEmpty(row.connection_id) || null;
|
||||
const apiKeyId = toStringOrEmpty(row.api_key_id) || null;
|
||||
const apiKeyName = toStringOrEmpty(row.api_key_name) || null;
|
||||
|
||||
const promptTokens = toNumber(row.tokens_input);
|
||||
const completionTokens = toNumber(row.tokens_output);
|
||||
const entryTime = new Date(timestamp);
|
||||
|
||||
const entryTokens = {
|
||||
input: row.tokens_input,
|
||||
output: row.tokens_output,
|
||||
cacheRead: row.tokens_cache_read,
|
||||
cacheCreation: row.tokens_cache_creation,
|
||||
reasoning: row.tokens_reasoning,
|
||||
input: toNumber(row.tokens_input),
|
||||
output: toNumber(row.tokens_output),
|
||||
cacheRead: toNumber(row.tokens_cache_read),
|
||||
cacheCreation: toNumber(row.tokens_cache_creation),
|
||||
reasoning: toNumber(row.tokens_reasoning),
|
||||
};
|
||||
const entryCost = await calculateCost(row.provider, row.model, entryTokens);
|
||||
const entryCost = await calculateCost(provider, model, entryTokens);
|
||||
|
||||
stats.totalPromptTokens += promptTokens;
|
||||
stats.totalCompletionTokens += completionTokens;
|
||||
@@ -105,71 +137,70 @@ export async function getUsageStats() {
|
||||
}
|
||||
|
||||
// By Provider
|
||||
if (!stats.byProvider[row.provider]) {
|
||||
stats.byProvider[row.provider] = {
|
||||
if (!stats.byProvider[provider]) {
|
||||
stats.byProvider[provider] = {
|
||||
requests: 0,
|
||||
promptTokens: 0,
|
||||
completionTokens: 0,
|
||||
cost: 0,
|
||||
};
|
||||
}
|
||||
stats.byProvider[row.provider].requests++;
|
||||
stats.byProvider[row.provider].promptTokens += promptTokens;
|
||||
stats.byProvider[row.provider].completionTokens += completionTokens;
|
||||
stats.byProvider[row.provider].cost += entryCost;
|
||||
stats.byProvider[provider].requests++;
|
||||
stats.byProvider[provider].promptTokens += promptTokens;
|
||||
stats.byProvider[provider].completionTokens += completionTokens;
|
||||
stats.byProvider[provider].cost += entryCost;
|
||||
|
||||
// By Model
|
||||
const modelKey = row.provider ? `${row.model} (${row.provider})` : row.model;
|
||||
const modelKey = provider ? `${model} (${provider})` : model;
|
||||
if (!stats.byModel[modelKey]) {
|
||||
stats.byModel[modelKey] = {
|
||||
requests: 0,
|
||||
promptTokens: 0,
|
||||
completionTokens: 0,
|
||||
cost: 0,
|
||||
rawModel: row.model,
|
||||
provider: row.provider,
|
||||
lastUsed: row.timestamp,
|
||||
rawModel: model,
|
||||
provider,
|
||||
lastUsed: timestamp,
|
||||
};
|
||||
}
|
||||
stats.byModel[modelKey].requests++;
|
||||
stats.byModel[modelKey].promptTokens += promptTokens;
|
||||
stats.byModel[modelKey].completionTokens += completionTokens;
|
||||
stats.byModel[modelKey].cost += entryCost;
|
||||
if (new Date(row.timestamp) > new Date(stats.byModel[modelKey].lastUsed)) {
|
||||
stats.byModel[modelKey].lastUsed = row.timestamp;
|
||||
if (new Date(timestamp) > new Date(stats.byModel[modelKey].lastUsed)) {
|
||||
stats.byModel[modelKey].lastUsed = timestamp;
|
||||
}
|
||||
|
||||
// By Account
|
||||
if (row.connection_id) {
|
||||
const accountName =
|
||||
connectionMap[row.connection_id] || `Account ${row.connection_id.slice(0, 8)}...`;
|
||||
const accountKey = `${row.model} (${row.provider} - ${accountName})`;
|
||||
if (connectionId) {
|
||||
const accountName = connectionMap[connectionId] || `Account ${connectionId.slice(0, 8)}...`;
|
||||
const accountKey = `${model} (${provider} - ${accountName})`;
|
||||
if (!stats.byAccount[accountKey]) {
|
||||
stats.byAccount[accountKey] = {
|
||||
requests: 0,
|
||||
promptTokens: 0,
|
||||
completionTokens: 0,
|
||||
cost: 0,
|
||||
rawModel: row.model,
|
||||
provider: row.provider,
|
||||
connectionId: row.connection_id,
|
||||
rawModel: model,
|
||||
provider,
|
||||
connectionId,
|
||||
accountName,
|
||||
lastUsed: row.timestamp,
|
||||
lastUsed: timestamp,
|
||||
};
|
||||
}
|
||||
stats.byAccount[accountKey].requests++;
|
||||
stats.byAccount[accountKey].promptTokens += promptTokens;
|
||||
stats.byAccount[accountKey].completionTokens += completionTokens;
|
||||
stats.byAccount[accountKey].cost += entryCost;
|
||||
if (new Date(row.timestamp) > new Date(stats.byAccount[accountKey].lastUsed)) {
|
||||
stats.byAccount[accountKey].lastUsed = row.timestamp;
|
||||
if (new Date(timestamp) > new Date(stats.byAccount[accountKey].lastUsed)) {
|
||||
stats.byAccount[accountKey].lastUsed = timestamp;
|
||||
}
|
||||
}
|
||||
|
||||
// By API key
|
||||
if (row.api_key_id || row.api_key_name) {
|
||||
const keyName = row.api_key_name || row.api_key_id || "unknown";
|
||||
const keyId = row.api_key_id || null;
|
||||
if (apiKeyId || apiKeyName) {
|
||||
const keyName = apiKeyName || apiKeyId || "unknown";
|
||||
const keyId = apiKeyId || null;
|
||||
const apiKey = keyId ? `${keyName} (${keyId})` : keyName;
|
||||
if (!stats.byApiKey[apiKey]) {
|
||||
stats.byApiKey[apiKey] = {
|
||||
@@ -179,15 +210,15 @@ export async function getUsageStats() {
|
||||
cost: 0,
|
||||
apiKeyId: keyId,
|
||||
apiKeyName: keyName,
|
||||
lastUsed: row.timestamp,
|
||||
lastUsed: timestamp,
|
||||
};
|
||||
}
|
||||
stats.byApiKey[apiKey].requests++;
|
||||
stats.byApiKey[apiKey].promptTokens += promptTokens;
|
||||
stats.byApiKey[apiKey].completionTokens += completionTokens;
|
||||
stats.byApiKey[apiKey].cost += entryCost;
|
||||
if (new Date(row.timestamp) > new Date(stats.byApiKey[apiKey].lastUsed)) {
|
||||
stats.byApiKey[apiKey].lastUsed = row.timestamp;
|
||||
if (new Date(timestamp) > new Date(stats.byApiKey[apiKey].lastUsed)) {
|
||||
stats.byApiKey[apiKey].lastUsed = timestamp;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -23,7 +23,6 @@ export interface ApiKeyMetadata {
|
||||
noLog?: boolean;
|
||||
budget?: number;
|
||||
usedBudget?: number;
|
||||
[key: string]: unknown;
|
||||
}
|
||||
|
||||
export interface ApiKeyPolicyResult {
|
||||
|
||||
@@ -75,7 +75,13 @@ export class CircuitBreaker {
|
||||
try {
|
||||
const saved = loadCircuitBreakerState(this.name);
|
||||
if (saved) {
|
||||
this.state = saved.state;
|
||||
if (
|
||||
saved.state === STATE.CLOSED ||
|
||||
saved.state === STATE.OPEN ||
|
||||
saved.state === STATE.HALF_OPEN
|
||||
) {
|
||||
this.state = saved.state;
|
||||
}
|
||||
this.failureCount = saved.failureCount;
|
||||
this.lastFailureTime = saved.lastFailureTime;
|
||||
if (this.state === STATE.HALF_OPEN) {
|
||||
|
||||
@@ -28,7 +28,7 @@ export const ProvidersMapSchema = z.record(z.string(), ProviderSchema);
|
||||
* @param {Record<string, object>} map - The providers map to validate
|
||||
* @param {string} name - Name of the map for error messages
|
||||
*/
|
||||
export function validateProviders(map, name) {
|
||||
export function validateProviders(map: Record<string, unknown>, name: string): void {
|
||||
const result = ProvidersMapSchema.safeParse(map);
|
||||
if (!result.success) {
|
||||
const issues = result.error.issues.map((i) => ` ${i.path.join(".")}: ${i.message}`).join("\n");
|
||||
|
||||
@@ -15,6 +15,69 @@ import {
|
||||
} from "@omniroute/open-sse/services/accountFallback.ts";
|
||||
import * as log from "../utils/logger";
|
||||
|
||||
type JsonRecord = Record<string, unknown>;
|
||||
|
||||
interface ProviderConnectionView {
|
||||
id: string;
|
||||
isActive: boolean;
|
||||
rateLimitedUntil: string | null;
|
||||
testStatus: string | null;
|
||||
apiKey: string | null;
|
||||
accessToken: string | null;
|
||||
refreshToken: string | null;
|
||||
tokenExpiresAt: string | null;
|
||||
expiresAt: string | null;
|
||||
projectId: string | null;
|
||||
providerSpecificData: JsonRecord;
|
||||
lastUsedAt: string | null;
|
||||
consecutiveUseCount: number;
|
||||
priority: number;
|
||||
lastError: string | null;
|
||||
errorCode: string | number | null;
|
||||
backoffLevel: number;
|
||||
}
|
||||
|
||||
function asRecord(value: unknown): JsonRecord {
|
||||
return value && typeof value === "object" && !Array.isArray(value) ? (value as JsonRecord) : {};
|
||||
}
|
||||
|
||||
function toStringOrNull(value: unknown): string | null {
|
||||
return typeof value === "string" && value.trim().length > 0 ? value : null;
|
||||
}
|
||||
|
||||
function toNumber(value: unknown, fallback = 0): number {
|
||||
if (typeof value === "number" && Number.isFinite(value)) return value;
|
||||
if (typeof value === "string" && value.trim().length > 0) {
|
||||
const parsed = Number(value);
|
||||
return Number.isFinite(parsed) ? parsed : fallback;
|
||||
}
|
||||
return fallback;
|
||||
}
|
||||
|
||||
function toProviderConnection(value: unknown): ProviderConnectionView {
|
||||
const row = asRecord(value);
|
||||
return {
|
||||
id: toStringOrNull(row.id) || "",
|
||||
isActive: row.isActive === true,
|
||||
rateLimitedUntil: toStringOrNull(row.rateLimitedUntil),
|
||||
testStatus: toStringOrNull(row.testStatus),
|
||||
apiKey: toStringOrNull(row.apiKey),
|
||||
accessToken: toStringOrNull(row.accessToken),
|
||||
refreshToken: toStringOrNull(row.refreshToken),
|
||||
tokenExpiresAt: toStringOrNull(row.tokenExpiresAt),
|
||||
expiresAt: toStringOrNull(row.expiresAt),
|
||||
projectId: toStringOrNull(row.projectId),
|
||||
providerSpecificData: asRecord(row.providerSpecificData),
|
||||
lastUsedAt: toStringOrNull(row.lastUsedAt),
|
||||
consecutiveUseCount: toNumber(row.consecutiveUseCount, 0),
|
||||
priority: toNumber(row.priority, 999),
|
||||
lastError: toStringOrNull(row.lastError),
|
||||
errorCode:
|
||||
typeof row.errorCode === "string" || typeof row.errorCode === "number" ? row.errorCode : null,
|
||||
backoffLevel: toNumber(row.backoffLevel, 0),
|
||||
};
|
||||
}
|
||||
|
||||
// Mutex to prevent race conditions during account selection
|
||||
let selectionMutex = Promise.resolve();
|
||||
|
||||
@@ -43,7 +106,10 @@ export async function getProviderCredentials(
|
||||
try {
|
||||
await currentMutex;
|
||||
|
||||
const connections = await getProviderConnections({ provider, isActive: true });
|
||||
const connectionsRaw = await getProviderConnections({ provider, isActive: true });
|
||||
const connections = (Array.isArray(connectionsRaw) ? connectionsRaw : [])
|
||||
.map(toProviderConnection)
|
||||
.filter((conn) => conn.id.length > 0);
|
||||
log.debug(
|
||||
"AUTH",
|
||||
`${provider} | total connections: ${connections.length}, excludeId: ${excludeConnectionId || "none"}`
|
||||
@@ -51,7 +117,10 @@ export async function getProviderCredentials(
|
||||
|
||||
if (connections.length === 0) {
|
||||
// Check all connections (including inactive) to see if rate limited
|
||||
const allConnections = await getProviderConnections({ provider });
|
||||
const allConnectionsRaw = await getProviderConnections({ provider });
|
||||
const allConnections = (Array.isArray(allConnectionsRaw) ? allConnectionsRaw : [])
|
||||
.map(toProviderConnection)
|
||||
.filter((conn) => conn.id.length > 0);
|
||||
log.debug("AUTH", `${provider} | all connections (incl inactive): ${allConnections.length}`);
|
||||
if (allConnections.length > 0) {
|
||||
const earliest = getEarliestRateLimitedUntil(allConnections);
|
||||
@@ -108,8 +177,9 @@ export async function getProviderCredentials(
|
||||
(c) => c.rateLimitedUntil && new Date(c.rateLimitedUntil).getTime() > Date.now()
|
||||
);
|
||||
const earliestConn = rateLimitedConns.sort(
|
||||
(a: any, b: any) =>
|
||||
new Date(a.rateLimitedUntil).getTime() - new Date(b.rateLimitedUntil).getTime()
|
||||
(a, b) =>
|
||||
new Date(a.rateLimitedUntil || 0).getTime() -
|
||||
new Date(b.rateLimitedUntil || 0).getTime()
|
||||
)[0];
|
||||
log.warn(
|
||||
"AUTH",
|
||||
@@ -132,7 +202,7 @@ export async function getProviderCredentials(
|
||||
|
||||
let connection;
|
||||
if (strategy === "round-robin") {
|
||||
const stickyLimit = settings.stickyRoundRobinLimit || 3;
|
||||
const stickyLimit = toNumber((settings as Record<string, unknown>).stickyRoundRobinLimit, 3);
|
||||
|
||||
// Sort by lastUsed (most recent first) to find current candidate
|
||||
const byRecency = [...availableConnections].sort((a: any, b: any) => {
|
||||
@@ -191,7 +261,7 @@ export async function getProviderCredentials(
|
||||
connection = availableConnections[idx];
|
||||
} else if (strategy === "least-used") {
|
||||
// Least Used: pick the one with oldest lastUsedAt
|
||||
const sorted = [...availableConnections].sort((a: any, b: any) => {
|
||||
const sorted = [...availableConnections].sort((a, b) => {
|
||||
if (!a.lastUsedAt && !b.lastUsedAt) return (a.priority || 999) - (b.priority || 999);
|
||||
if (!a.lastUsedAt) return -1;
|
||||
if (!b.lastUsedAt) return 1;
|
||||
@@ -202,7 +272,7 @@ export async function getProviderCredentials(
|
||||
// Cost Optimized: sort by priority ascending (lower = cheaper/preferred)
|
||||
// Future: can be enhanced with actual cost data per provider
|
||||
const sorted = [...availableConnections].sort(
|
||||
(a: any, b: any) => (a.priority || 999) - (b.priority || 999)
|
||||
(a, b) => (a.priority || 999) - (b.priority || 999)
|
||||
);
|
||||
connection = sorted[0];
|
||||
} else {
|
||||
@@ -216,7 +286,10 @@ export async function getProviderCredentials(
|
||||
refreshToken: connection.refreshToken,
|
||||
expiresAt: connection.tokenExpiresAt || connection.expiresAt || null,
|
||||
projectId: connection.projectId,
|
||||
copilotToken: connection.providerSpecificData?.copilotToken,
|
||||
copilotToken:
|
||||
typeof connection.providerSpecificData.copilotToken === "string"
|
||||
? connection.providerSpecificData.copilotToken
|
||||
: null,
|
||||
providerSpecificData: connection.providerSpecificData,
|
||||
connectionId: connection.id,
|
||||
// Include current status for optimization check
|
||||
@@ -258,8 +331,11 @@ export async function markAccountUnavailable(
|
||||
await currentMutex;
|
||||
|
||||
// Read current connection to get backoffLevel
|
||||
const connections = await getProviderConnections({ provider });
|
||||
const conn = connections.find((c) => c.id === connectionId);
|
||||
const connectionsRaw = await getProviderConnections({ provider });
|
||||
const connections = (Array.isArray(connectionsRaw) ? connectionsRaw : [])
|
||||
.map(toProviderConnection)
|
||||
.filter((connection) => connection.id.length > 0);
|
||||
const conn = connections.find((connection) => connection.id === connectionId);
|
||||
const backoffLevel = conn?.backoffLevel || 0;
|
||||
|
||||
// ─── Anti-Thundering Herd Guard ─────────────────────────────────
|
||||
|
||||
@@ -55,3 +55,32 @@ test("stream execution marks task as failed when handler throws", async () => {
|
||||
assert.equal(loaded?.state, "failed");
|
||||
assert.deepEqual(loaded?.artifacts.at(-1), { type: "error", content: "upstream failure" });
|
||||
});
|
||||
|
||||
test("expired submitted task transitions to failed without throwing", () => {
|
||||
const tm = createManager();
|
||||
const task = tm.createTask({
|
||||
skill: "smart-routing",
|
||||
messages: [{ role: "user", content: "hello" }],
|
||||
});
|
||||
task.expiresAt = new Date(Date.now() - 1_000).toISOString();
|
||||
|
||||
assert.doesNotThrow(() => tm.getTask(task.id));
|
||||
const loaded = tm.getTask(task.id);
|
||||
assert.equal(loaded?.state, "failed");
|
||||
});
|
||||
|
||||
test("cleanup keeps cancelled tasks as cancelled", () => {
|
||||
const tm = createManager();
|
||||
const task = tm.createTask({
|
||||
skill: "smart-routing",
|
||||
messages: [{ role: "user", content: "cancel me" }],
|
||||
});
|
||||
tm.updateTask(task.id, "cancelled");
|
||||
task.expiresAt = new Date(Date.now() - 1_000).toISOString();
|
||||
|
||||
// private in TS only; callable at runtime for regression test
|
||||
tm.cleanupExpired();
|
||||
|
||||
const loaded = tm.getTask(task.id);
|
||||
assert.equal(loaded?.state, "cancelled");
|
||||
});
|
||||
|
||||
@@ -9,6 +9,7 @@
|
||||
},
|
||||
"include": [],
|
||||
"files": [
|
||||
"src/app/api/settings/proxy/test/route.ts",
|
||||
"src/lib/db/apiKeys.ts",
|
||||
"src/lib/db/cliToolState.ts",
|
||||
"src/lib/db/encryption.ts",
|
||||
@@ -17,11 +18,14 @@
|
||||
"src/lib/db/settings.ts",
|
||||
"src/lib/db/stateReset.ts",
|
||||
"open-sse/config/providerModels.ts",
|
||||
"open-sse/config/providerRegistry.ts",
|
||||
"open-sse/mcp-server/audit.ts",
|
||||
"open-sse/mcp-server/server.ts",
|
||||
"open-sse/mcp-server/tools/advancedTools.ts",
|
||||
"open-sse/translator/registry.ts",
|
||||
"open-sse/mcp-server/scopeEnforcement.ts",
|
||||
"src/shared/validation/providerSchema.ts",
|
||||
"src/shared/validation/schemas.ts",
|
||||
"open-sse/handlers/responseSanitizer.ts",
|
||||
"open-sse/handlers/responseTranslator.ts"
|
||||
],
|
||||
|
||||
Reference in New Issue
Block a user