Files
OmniRoute/open-sse/mcp-server/README.md
Brandon Bennett 6615a5445b feat: combo-lane awareness + activation UX + MCP visibility (Wave 2 of #9654) (#10039)
* feat(admission): per-target lane-aware probes for combo/fusion fan-out (#9654 Wave 2)

Combo and fusion fan out N targets without ever consulting the adaptive-admission
layer: the parent request holds one lease, but each fan-out target is dispatched
unconditionally. With virtual lanes enabled (OMNIROUTE_CHAT_VIRTUAL_LANES=1), a
connection whose lane queue is full now SKIPS additional fan-out targets instead
of piling more queued work onto an already-congested session.

Adds PerTargetAdmissionHook (admission/types.ts) + createPerTargetAdmissionHook
factory (chatAdmission.ts): strictly non-blocking (maxWaitMs 0 - skip, never
queue), a no-op when virtual lanes are off, keyed to the parent tenantKey, and
release-on-admit so the probe is a capacity gate, not a hold.

Threaded through every parallel fan-out path:
- priority/weighted executeTarget + round-robin skip chains (combo.ts)
- fusion panel before fan-out (fusion.ts), judge fallback prefers survivors
- chaos parallel panel (autoCombo/chaosEngine.ts)
- tryFusionDispatch / tryRuntimeUnitDispatch / buildBaseOptions (dispatchPrelude.ts)
- chat.ts primary + safety-net redirect call sites

Snapshot exposes virtualLanes so the no-op gate is cheap and honest.

Tests: tests/unit/combo-lane-awareness-9654.test.ts (10 tests) - factory
semantics, priority/RR skip, fusion panel drop + all-skipped 503, no-hook
backward-compat baseline.

* feat(flags): activation UX - env-wins adaptive virtual-lanes flag + env docs (#9654 Wave 2)

U7: make adaptive virtual admission lanes discoverable + activatable.
- New OMNIROUTE_CHAT_VIRTUAL_LANES feature flag (boolean/runtime/requiresRestart) in featureFlagDefinitions + en.json i18n key.
- lib/admissionVirtualLanes.ts: env-wins resolver (env > DB > default) + boot warm folding a DB-sourced override into the process-global runtime env via reloadAdaptiveAdmissionRuntime(options.env) - no process.env mutation, no open-sse changes. Env still wins; DB toggle gates at next boot.
- GET /api/settings/feature-flags special-cases the flag to report the gate true source (ccDiscoveryAliases precedent); flagPayload helper dedupes the payload shape.
- Wire the warm into instrumentation-node registerNodejs (non-fatal, DB-ready).
- Document the master switch in .env.example + ENVIRONMENT.md with the system-1/system-2 distinction; zero new env-doc-sync drift.
- 11 new tests (resolver precedence + warm); 60/60 across feature-flag suites; typecheck core clean; ESLint + doc gates green.

* feat(mcp): surface adaptive admission lane data in omniroute_get_health (#9654 Wave 2)

U8: make adaptive virtual-lane admission visible to agents via the MCP health tool. handleGetHealth now surfaces a curated adaptiveAdmission block from the health payload (which already carried the runtime snapshot but was dropping it): virtualLanes/pressure/utilization/laneCount/laneQueuedCount/laneQueuedCost, laneTenants capped at top-10 by queued cost, admitted/rejected/wouldReject counts, shutdown. Block omitted entirely when the health endpoint reports none.

isLaneFlagOn mirrors the runtime 1|true convention so a string serialization can never invert a boolean lane report. getHealthOutput schema extended with the matching optional shape; tool description updated.

4 new dispatch tests (full block, top-10 cap/order, omission, defensive coercion of string flags + malformed lane entries) - 22/22 in essentialTools.test.ts. README: Adaptive Admission Lane Data table + Skills & Tool Navigability audit (29/43 schema entries covered, 14 undocumented, tool_search keyword runtime discovery, full catalog in docs/frameworks/MCP-SERVER.md).

No new lint errors (4 pre-existing in server.ts), typecheck core clean, doc counts + fabricated-docs gates green.

* docs: add changelog entry for #9654 Wave 2 (#10039)

* fix(codeql): suppress js/insufficient-password-hash false positive in lane-key fingerprinting (#10039)

resolveSessionId sha256-hashes bearer/x-api-key/x-goog-api-key to derive a deterministic, non-reversible per-key lane-bucket ID for virtual admission lanes (#9654). This is not password storage or verification, so the rule is a false positive; suppress it inline (same house style as src/lib/sync/tokens.ts) to clear the codeqlAlerts ratchet (2 > baseline 1) that blocks #10039 and every PR against release/v3.8.50.

* docs(mcp): complete MCP server README tool reference (#10039)

The MCP server README covered only 29 of the 43 schema entries, listing the
remaining tools solely as a gap note with omniroute_tool_search as the runtime
fallback. Add tool-reference tables for the agent-skills trio, oneproxy trio,
web_fetch/web_search, tool_search, create_combo, set_routing_strategy,
pick_fastest_model, sync_pricing, and db_health_check so the README covers the
full schemas catalog, and fold the coverage note into the tool_search discovery
paragraph.

* fix(chat): drop unused correlationId from safety-net combo redirect (#10039)

handleComboChat's HandleComboChatOptions has no correlationId member and
the combo pipeline never consumes it; the property was copied from the
handleSingleModelChat options shape by accident and introduced a new
TS2353 under the open-sse workspace typecheck gate.

* fix(i18n): translate featureFlagChatVirtualLanesEnabledDescription into 42 locales (#10039)

en.json gained the flag description in this PR but the locale catalogs
were never mirrored, failing the pt-BR key-parity (#6695) and vi
completeness gates. Adds a real translation to every locale, keeping the
zh-CN/zh-TW glossary canonical terms (提供者/儀表板) and no ICU drift.

* chore(quality): ratchet open-sse-typecheck baseline down (#10039)

The Wave 2 admission refactor removed 66 baselined open-sse type errors;
re-freeze the baseline so the gate pins the new, tighter state.

* docs: resync provider reference to 341 and CLI tools to 34

The release branch gained an 11th no-auth provider (freeaiapikey registry
resync, #10233) and a 26th CLI Code tool without regenerating the
auto-generated docs, leaving every PR against release/v3.8.50 failing the
Docs Gates strict validator (code 341 vs doc 340, CLI 34 vs "33 tools").

Regenerate docs/reference/PROVIDER_REFERENCE.md and sync the provider/tool
counts across README.md, AGENTS.md, llm.txt plus 42 i18n mirrors,
package.json description, and the four diagram SVGs.

* fix(tests): align count expectations with live catalogs (pre-existing release drift)

Release/v3.8.50 currently fails five gates on its own tree; this PR inherits
them. Fix the stale expectations to match live code:

- feature-flags-settings: 48 -> 49 flags (Wave 2 adds OMNIROUTE_CHAT_VIRTUAL_LANES)
- cli-tools-schema / cli-catalog-counts: 33 -> 34 tools (zcode added; 26 code = 21 visible + 5 none)
- optional-transformers-dependency: onnxruntime-node ~1.24.3 -> ~1.27.0 (bump #10382)
- stryker.conf.json: register chatcore-header-drop-warn-dedupe-10315 test
- check-public-creds: freeze zcodeProtocol clientId false positive (client identifier, not a credential)

* fix(tests): follow release's onnxruntime-node revert to ~1.24.3

release/v3.8.50's #10543 pinned onnxruntime-node back to ~1.24.3 after
#10403's ~1.27.0 bump caused npm to nest a second native copy under
@huggingface/transformers and broke the Docker SONAME contract. This
PR's own drift-alignment commit (57b9c033) predates that revert and
still expected ~1.27.0; the 3-way merge did not flag it as a textual
conflict since only one side touched this exact line, but the merged
tree became internally inconsistent (package.json ~1.24.3 vs test
expecting ~1.27.0). Align the test with the now-canonical release
value.

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>

* fix(quality): dedupe stryker.conf.json chatcore-header-drop-warn-dedupe entry

The 3-way merge applied both sides' insertion of the same test-file entry
at different positions, producing a duplicate with broken indentation.
Adopted release's clean version of the file.

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>

---------

Co-authored-by: Brandon Bennett <brandonbennett@macbookair.myfiosgateway.com>
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
Co-authored-by: Brandon Bennett <branben@users.noreply.github.com>
2026-08-18 11:31:46 -03:00

690 lines
31 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# OmniRoute MCP Server
> **Model Context Protocol server** that exposes OmniRoute's gateway intelligence as **107 tools** for AI agents.
>
> **Source of truth for the full tool catalog and REST surface:** [`docs/frameworks/MCP-SERVER.md`](../../docs/frameworks/MCP-SERVER.md). This README focuses on architecture, configuration, and integration examples; the catalog below is a summary subset.
The MCP Server allows any AI agent (Claude Desktop, Cursor, VS Code Copilot, custom agents) to **monitor, control, and optimize** the OmniRoute AI gateway programmatically.
---
## Architecture
```
┌──────────────────────────────────────────────────────────────────┐
│ AI Agent / IDE │
│ (Claude Desktop, Cursor, VS Code, Custom) │
└──────────────────────┬───────────────────────────────────────────┘
│ MCP Protocol (stdio or HTTP)
┌──────────────────────────────────────────────────────────────────┐
│ OmniRoute MCP Server │
│ ┌──────────────┐ ┌─────────────────┐ ┌────────────────────┐ │
│ │ Scope │ │ 107 MCP Tools │ │ Audit Logger │ │
│ │ Enforcement │──│ (core + memory │──│ (SHA-256/SQLite) │ │
│ │ │ │ + skills + …) │ │ │ │
│ └──────────────┘ └────────┬────────┘ └────────────────────┘ │
└─────────────────────────────┼────────────────────────────────────┘
│ HTTP (internal)
┌──────────────────────────────────────────────────────────────────┐
│ OmniRoute Gateway (port 20128) │
│ /v1/chat/completions /api/combos /api/usage ... │
└──────────────────────────────────────────────────────────────────┘
```
---
## Quick Start
### 1. Environment Variables
```bash
# Required: OmniRoute base URL
export OMNIROUTE_BASE_URL="http://localhost:20128"
# Optional: API key for authenticated access
export OMNIROUTE_API_KEY="your-api-key"
# Optional: Scope enforcement (default: disabled)
export OMNIROUTE_MCP_ENFORCE_SCOPES="true"
export OMNIROUTE_MCP_SCOPES="read:health,read:combos,read:quota,read:usage,read:models,read:cache,read:compression,read:tools,execute:completions,write:combos,write:budget,write:resilience,write:cache,write:compression"
```
### 2. stdio Transport (IDE Integration)
Add to your MCP client configuration:
**Claude Desktop** (`claude_desktop_config.json`):
```json
{
"mcpServers": {
"omniroute": {
"command": "node",
"args": ["path/to/omniroute/open-sse/mcp-server/server.ts"],
"env": {
"OMNIROUTE_BASE_URL": "http://localhost:20128",
"OMNIROUTE_API_KEY": "your-key"
}
}
}
}
```
**Cursor** (`.cursor/mcp.json`):
```json
{
"mcpServers": {
"omniroute": {
"command": "npx",
"args": ["tsx", "open-sse/mcp-server/server.ts"],
"env": {
"OMNIROUTE_BASE_URL": "http://localhost:20128"
}
}
}
}
```
**VS Code** (`.vscode/settings.json`):
```json
{
"mcp": {
"servers": {
"omniroute": {
"command": "npx",
"args": ["tsx", "open-sse/mcp-server/server.ts"],
"env": {
"OMNIROUTE_BASE_URL": "http://localhost:20128"
}
}
}
}
}
```
### 3. Start via CLI
```bash
# Direct start (stdio)
npx tsx open-sse/mcp-server/server.ts
# Or via OmniRoute CLI
omniroute --mcp
```
---
## Tool Reference
### Phase 1: Essential Tools (8)
| # | Tool | Scopes | Description |
| --- | ------------------------------- | --------------------- | --------------------------------------------------------------------------------------------------- |
| 1 | `omniroute_get_health` | `read:health` | Gateway health, uptime, memory, circuit breakers, rate limits, cache stats + adaptive lane pressure |
| 2 | `omniroute_list_combos` | `read:combos` | List all combos (model chains) with strategies and optional metrics |
| 3 | `omniroute_get_combo_metrics` | `read:combos` | Performance metrics for a specific combo |
| 4 | `omniroute_switch_combo` | `write:combos` | Activate or deactivate a combo for routing |
| 5 | `omniroute_check_quota` | `read:quota` | Remaining API quota per provider with token health status |
| 6 | `omniroute_route_request` | `execute:completions` | Send a chat completion through intelligent routing |
| 7 | `omniroute_cost_report` | `read:usage` | Cost report by period (session/day/week/month) with per-provider breakdown |
| 8 | `omniroute_list_models_catalog` | `read:models` | List all available models across providers with capabilities and pricing |
### Phase 2: Advanced Tools (8)
| # | Tool | Scopes | Description |
| --- | ---------------------------------- | ------------------------------------ | ---------------------------------------------------------------------------------------------- |
| 9 | `omniroute_simulate_route` | `read:health`, `read:combos` | Dry-run routing simulation showing fallback tree and estimated costs |
| 10 | `omniroute_set_budget_guard` | `write:budget` | Set session budget with action on exceed: `degrade`, `block`, or `alert` |
| 11 | `omniroute_set_resilience_profile` | `write:resilience` | Apply resilience profile: `aggressive`, `balanced`, or `conservative` |
| 12 | `omniroute_test_combo` | `execute:completions`, `read:combos` | Test each provider in a combo with a real prompt and a real upstream call, report latency/cost |
| 13 | `omniroute_get_provider_metrics` | `read:health` | Per-provider metrics with latency percentiles (p50/p95/p99), circuit breaker |
| 14 | `omniroute_best_combo_for_task` | `read:combos`, `read:health` | AI-powered combo recommendation by task type with budget/latency constraints |
| 15 | `omniroute_explain_route` | `read:health`, `read:usage` | Explain why a request was routed to a provider (scoring factors, fallbacks) |
| 16 | `omniroute_get_session_snapshot` | `read:usage` | Full session snapshot: cost, tokens, top models, errors, budget status |
### Cache and Compression Tools
| # | Tool | Scopes | Description |
| --- | ----------------------------------- | ------------------- | ---------------------------------------------------------------------------- |
| 21 | `omniroute_cache_stats` | `read:cache` | Semantic cache, prompt-cache, and idempotency statistics |
| 22 | `omniroute_cache_flush` | `write:cache` | Flush cache entries globally or by signature/model |
| 23 | `omniroute_compression_status` | `read:compression` | Compression settings, analytics summary, and provider-aware cache statistics |
| 24 | `omniroute_compression_configure` | `write:compression` | Configure compression mode and trigger thresholds at runtime |
| 25 | `omniroute_set_compression_engine` | `write:compression` | Set Caveman, RTK, or stacked compression mode and pipeline |
| 26 | `omniroute_list_compression_combos` | `read:compression` | List named compression combos and routing assignments |
| 27 | `omniroute_compression_combo_stats` | `read:compression` | Read analytics grouped by compression combo and engine |
| 28 | `omniroute_ccr_store` | `write:compression` | Store content in the caller-isolated in-memory CCR store |
| 29 | `omniroute_ccr_retrieve` | `read:compression` | Retrieve full or ranged caller-owned CCR content |
| 30 | `omniroute_ccr_inspect` | `read:compression` | Inspect CCR metadata without returning content |
| 31 | `omniroute_ccr_list` | `read:compression` | List paginated caller-owned CCR metadata |
| 32 | `omniroute_ccr_delete` | `write:compression` | Delete a caller-owned CCR block |
| 33 | `omniroute_ccr_stats` | `read:compression` | Report caller usage, bounded-store limits, and lifecycle counters |
CCR storage is bounded and in-memory only: 2 MiB per block, 16 MiB per principal, 64 MiB global,
with a 24-hour default TTL. Full MCP retrieval is capped at 256 KiB; larger blocks use ranged or
grep retrieval. All lifecycle operations are isolated by the authenticated caller principal.
MCP listable metadata descriptions are compressed at registration/list time when description
compression is enabled. `omniroute_compression_status` exposes those savings separately as
`analytics.mcpDescriptionCompression` with `source: "mcp_metadata_estimate"`, so clients do not
mistake metadata shrink estimates for provider token receipts.
### Discovery & Web Tools
| Tool | Scopes | Description |
| ----------------------- | ---------------- | ------------------------------------------------------------------------------------------------------------------------------------ |
| `omniroute_tool_search` | `read:tools` | Keyword search across the registered MCP tools; returns compact one-line signatures for token-efficient discovery |
| `omniroute_web_fetch` | `execute:search` | Fetch and extract a URL's content through the web-fetch gateway (Firecrawl, Jina Reader, Tavily, TinyFish) with automatic failover |
| `omniroute_web_search` | `execute:search` | Web search through the search gateway (Serper, Brave, Perplexity, Exa, Tavily, Google PSE, Linkup, SearchAPI, SearXNG) with failover |
### Skills & Catalog Tools
| Tool | Scopes | Description |
| --------------------------------- | -------------- | -------------------------------------------------------------------------------------------------------- |
| `omniroute_agent_skills_list` | `read:catalog` | List all 42 agent skills with optional `category` (`api`\|`cli`) and `area` filters; metadata + coverage |
| `omniroute_agent_skills_get` | `read:catalog` | Full metadata + SKILL.md content for a single skill by canonical `id` |
| `omniroute_agent_skills_coverage` | `read:catalog` | Coverage stats: how many of the 22 API and 20 CLI skills have SKILL.md files on disk vs catalog totals |
### Proxy, Pricing & Data Tools
| Tool | Scopes | Description |
| --------------------------- | --------------------------------- | ------------------------------------------------------------------------------------------ |
| `omniroute_oneproxy_fetch` | `read:proxies` | Fetch free proxies from the 1proxy marketplace (protocol/country/quality/limit filters) |
| `omniroute_oneproxy_rotate` | `read:proxies` | Get the next available proxy by strategy (`random` / `quality` / `sequential`) |
| `omniroute_oneproxy_stats` | `read:proxies` | Pool stats, sync status, distribution by protocol and country |
| `omniroute_sync_pricing` | `pricing:write` | Sync pricing from external sources (LiteLLM) without overwriting user-set prices; `dryRun` |
| `omniroute_db_health_check` | `read:health`, `write:resilience` | Diagnose (and optionally auto-repair) database drift — broken combo refs, orphan rows |
### Combo & Routing Tools
| Tool | Scopes | Description |
| -------------------------------- | ------------------------------------------ | -------------------------------------------------------------------------------------------- |
| `omniroute_create_combo` | `write:combos` | Register a new combo (model chain) with name, ordered model list, and optional strategy |
| `omniroute_set_routing_strategy` | `write:combos` | Update combo routing strategy at runtime (`priority` / `weighted` / `auto` / etc.) |
| `omniroute_pick_fastest_model` | `read:combos`, `read:health`, `read:usage` | Pick the fastest reliable provider-model pair from live telemetry; can apply latency routing |
---
### Adaptive Admission Lane Data
`omniroute_get_health` includes an `adaptiveAdmission` block whenever the gateway's adaptive
virtual-lane admission is active. It is a curated subset of the live admission snapshot:
| Field | Meaning |
| ------------------ | ---------------------------------------------------------------------- |
| `virtualLanes` | Whether per-tenant virtual-lane admission is enabled |
| `pressure` | Current pressure state (e.g. `healthy`, `high`, `critical`) |
| `utilization` | Current capacity utilization (0.01.0) |
| `laneCount` | Number of live lanes |
| `laneQueuedCount` | Total requests queued across lanes |
| `laneQueuedCost` | Total estimated cost queued across lanes |
| `laneTenants` | Top 10 lanes by queued cost (`tenantKey`, `queuedCount`, `queuedCost`) |
| `admittedCount` | Requests admitted since boot |
| `rejectedCount` | Requests rejected since boot |
| `wouldRejectCount` | Requests that would be rejected under the current limit |
| `shutdown` | Whether the admission runtime is shutting down |
`tenantKey` is an opaque per-API-key derived identifier, never the raw key. The block is omitted
entirely when the health endpoint reports no adaptive-admission data.
### Skills & Tool Navigability
The tables above cover the full `schemas/` catalog (43 entries); the authoritative reference with
scope-enforcement and transport details lives in
[`docs/frameworks/MCP-SERVER.md`](../../docs/frameworks/MCP-SERVER.md).
Agents never need to read this file to find a capability: `omniroute_tool_search` performs keyword
search across the registered tool set and returns compact one-line signatures (token-efficient
discovery), so newly added capabilities stay discoverable at runtime.
---
## Client Examples
### Python — Full Agent Workflow
```python
"""
OmniRoute MCP Client — Python example using the mcp SDK.
Install: pip install mcp
"""
import asyncio
from mcp import ClientSession, StdioServerParameters
from mcp.client.stdio import stdio_client
async def main():
server = StdioServerParameters(
command="npx",
args=["tsx", "open-sse/mcp-server/server.ts"],
env={
"OMNIROUTE_BASE_URL": "http://localhost:20128",
"OMNIROUTE_API_KEY": "your-key",
},
)
async with stdio_client(server) as (read, write):
async with ClientSession(read, write) as session:
await session.initialize()
# 1. Check gateway health
health = await session.call_tool("omniroute_get_health", {})
print("Health:", health.content[0].text)
# 2. List available combos with metrics
combos = await session.call_tool("omniroute_list_combos", {
"includeMetrics": True
})
print("Combos:", combos.content[0].text)
# 3. Find the best combo for a coding task
best = await session.call_tool("omniroute_best_combo_for_task", {
"taskType": "coding",
"budgetConstraint": 0.50,
"latencyConstraint": 5000,
})
print("Best combo:", best.content[0].text)
# 4. Set a session budget guard
budget = await session.call_tool("omniroute_set_budget_guard", {
"maxCost": 1.00,
"action": "degrade",
"degradeToTier": "cheap",
})
print("Budget guard:", budget.content[0].text)
# 5. Route a request through intelligent pipeline
response = await session.call_tool("omniroute_route_request", {
"model": "claude-sonnet-4",
"messages": [
{"role": "user", "content": "Write a Python hello world"}
],
"role": "coding",
})
print("Response:", response.content[0].text)
# 6. Get the session snapshot
snapshot = await session.call_tool("omniroute_get_session_snapshot", {})
print("Session:", snapshot.content[0].text)
asyncio.run(main())
```
### TypeScript — Programmatic Agent
```typescript
import { Client } from "@modelcontextprotocol/sdk/client/index.js";
import { StdioClientTransport } from "@modelcontextprotocol/sdk/client/stdio.js";
async function main() {
const transport = new StdioClientTransport({
command: "npx",
args: ["tsx", "open-sse/mcp-server/server.ts"],
env: {
OMNIROUTE_BASE_URL: "http://localhost:20128",
OMNIROUTE_API_KEY: "your-key",
},
});
const client = new Client({ name: "my-agent", version: "1.0.0" });
await client.connect(transport);
// Check quota before deciding which model to use
const quota = await client.callTool({
name: "omniroute_check_quota",
arguments: { provider: "claude" },
});
console.log("Claude quota:", quota.content);
// Simulate the route before actually calling
const simulation = await client.callTool({
name: "omniroute_simulate_route",
arguments: {
model: "claude-sonnet-4",
promptTokenEstimate: 2000,
},
});
console.log("Route simulation:", simulation.content);
// Send the actual request
const result = await client.callTool({
name: "omniroute_route_request",
arguments: {
model: "claude-sonnet-4",
messages: [{ role: "user", content: "Explain async/await" }],
},
});
console.log("Result:", result.content);
// Cost report
const costs = await client.callTool({
name: "omniroute_cost_report",
arguments: { period: "session" },
});
console.log("Costs:", costs.content);
await client.close();
}
main();
```
### Go — HTTP Client
```go
package main
import (
"bytes"
"encoding/json"
"fmt"
"io"
"net/http"
)
// Simplified direct-API approach (bypass MCP, hit OmniRoute APIs directly)
// Useful if you don't need MCP protocol framing.
func callTool(baseURL, tool string, args map[string]any) (string, error) {
// MCP tools map to OmniRoute APIs:
endpoints := map[string]string{
"health": "/api/monitoring/health",
"combos": "/api/combos",
"quota": "/api/usage/quota",
"models": "/v1/models",
}
url := baseURL + endpoints[tool]
resp, err := http.Get(url)
if err != nil {
return "", err
}
defer resp.Body.Close()
body, _ := io.ReadAll(resp.Body)
return string(body), nil
}
func routeRequest(baseURL, model, prompt string) (string, error) {
payload := map[string]any{
"model": model,
"messages": []map[string]string{
{"role": "user", "content": prompt},
},
"stream": false,
}
data, _ := json.Marshal(payload)
resp, err := http.Post(
baseURL+"/v1/chat/completions",
"application/json",
bytes.NewReader(data),
)
if err != nil {
return "", err
}
defer resp.Body.Close()
body, _ := io.ReadAll(resp.Body)
return string(body), nil
}
func main() {
base := "http://localhost:20128"
health, _ := callTool(base, "health", nil)
fmt.Println("Health:", health)
result, _ := routeRequest(base, "auto", "Hello from Go!")
fmt.Println("Result:", result)
}
```
---
## Use Cases
### 🔄 Use Case 1: Auto-Healing Agent
An agent that monitors OmniRoute health and auto-switches combos when providers degrade.
```python
async def auto_healing_loop(session):
"""Monitor health and react to provider issues."""
while True:
# Check health
health = await session.call_tool("omniroute_get_health", {})
data = json.loads(health.content[0].text)
# Find providers with open circuit breakers
broken = [
cb for cb in data["circuitBreakers"]
if cb["state"] == "OPEN"
]
if broken:
# Switch to a different resilience profile
await session.call_tool("omniroute_set_resilience_profile", {
"profile": "conservative"
})
# Find best alternative combo
best = await session.call_tool("omniroute_best_combo_for_task", {
"taskType": "coding"
})
best_data = json.loads(best.content[0].text)
combo_id = best_data["recommendedCombo"]["id"]
# Activate it
await session.call_tool("omniroute_switch_combo", {
"comboId": combo_id, "active": True
})
print(f"⚠️ Auto-healed: switched to {combo_id}")
await asyncio.sleep(30) # Check every 30 seconds
```
### 💰 Use Case 2: Budget-Aware Coding Agent
An agent that monitors costs in real-time and degrades to cheaper models when nearing budget.
```python
async def budget_aware_coding(session, task: str, max_budget: float):
"""Complete a coding task within a budget."""
# Set budget guard
await session.call_tool("omniroute_set_budget_guard", {
"maxCost": max_budget,
"action": "degrade",
"degradeToTier": "cheap",
})
# Simulate first to estimate cost
sim = await session.call_tool("omniroute_simulate_route", {
"model": "claude-sonnet-4",
"promptTokenEstimate": len(task.split()) * 2,
})
sim_data = json.loads(sim.content[0].text)
estimated_cost = sim_data["fallbackTree"]["bestCaseCost"]
print(f"Estimated cost: ${estimated_cost:.4f}")
# Send request
result = await session.call_tool("omniroute_route_request", {
"model": "claude-sonnet-4",
"messages": [{"role": "user", "content": task}],
"role": "coding",
})
# Check remaining budget
snapshot = await session.call_tool("omniroute_get_session_snapshot", {})
snap_data = json.loads(snapshot.content[0].text)
print(f"Session cost: ${snap_data['costTotal']:.4f}")
if snap_data.get("budgetGuard"):
print(f"Budget remaining: ${snap_data['budgetGuard']['remaining']:.4f}")
return json.loads(result.content[0].text)["response"]["content"]
```
### 🧪 Use Case 3: Combo Benchmarking Agent
An agent that periodically benchmarks all combos and reports the fastest/cheapest.
```python
async def benchmark_combos(session):
"""Benchmark all enabled combos and rank them."""
combos = await session.call_tool("omniroute_list_combos", {
"includeMetrics": True,
})
combo_list = json.loads(combos.content[0].text)["combos"]
results = []
for combo in combo_list:
if not combo["enabled"]:
continue
test = await session.call_tool("omniroute_test_combo", {
"comboId": combo["id"],
"testPrompt": "Return the number 42.",
})
test_data = json.loads(test.content[0].text)
results.append({
"combo": combo["name"],
"fastest": test_data["summary"]["fastestProvider"],
"cheapest": test_data["summary"]["cheapestProvider"],
"success_rate": f'{test_data["summary"]["successful"]}/{test_data["summary"]["totalProviders"]}',
})
print("📊 Combo Benchmark Results:")
for r in results:
print(f" {r['combo']}: fastest={r['fastest']}, cheapest={r['cheapest']}, success={r['success_rate']}")
```
### 🔍 Use Case 4: Post-Mortem Debugging Agent
An agent that explains why a request was routed to a specific provider.
```typescript
async function debugRouting(client: Client, requestId: string) {
// Explain the routing decision
const explanation = await client.callTool({
name: "omniroute_explain_route",
arguments: { requestId },
});
const data = JSON.parse(explanation.content[0].text);
console.log(`Request ${requestId}:`);
console.log(` Provider: ${data.decision.providerSelected}`);
console.log(` Model: ${data.decision.modelUsed}`);
console.log(` Score: ${data.decision.score}`);
console.log(` Factors:`);
for (const factor of data.decision.factors) {
console.log(` ${factor.name}: ${factor.value} (weight: ${factor.weight})`);
}
if (data.decision.fallbacksTriggered.length > 0) {
console.log(` Fallbacks triggered:`);
for (const fb of data.decision.fallbacksTriggered) {
console.log(` ${fb.provider}: ${fb.reason}`);
}
}
}
```
### 📋 Use Case 5: Model Discovery Agent
An agent that discovers the cheapest models for a given capability.
```python
async def find_cheapest_models(session, capability="chat"):
"""Find the cheapest available models for a capability."""
catalog = await session.call_tool("omniroute_list_models_catalog", {
"capability": capability,
})
models = json.loads(catalog.content[0].text)["models"]
# Filter available models with pricing
priced = [
m for m in models
if m["status"] == "available" and m.get("pricing")
]
priced.sort(key=lambda m: m["pricing"]["inputPerMillion"] or float("inf"))
print(f"💡 Cheapest {capability} models:")
for m in priced[:5]:
input_cost = m["pricing"]["inputPerMillion"] or 0
output_cost = m["pricing"]["outputPerMillion"] or 0
print(f" {m['id']} ({m['provider']}): ${input_cost}/M in, ${output_cost}/M out")
```
---
## Security & Scope Enforcement
The MCP server supports **fine-grained scope enforcement** for multi-tenant environments:
| Scope | Tools |
| --------------------- | ---------------------------------------------------------------------------------------------- |
| `read:health` | `get_health`, `simulate_route`, `get_provider_metrics`, `best_combo_for_task`, `explain_route` |
| `read:combos` | `list_combos`, `get_combo_metrics`, `simulate_route`, `best_combo_for_task`, `test_combo` |
| `read:quota` | `check_quota` |
| `read:usage` | `cost_report`, `explain_route`, `get_session_snapshot` |
| `read:models` | `list_models_catalog` |
| `read:cache` | `cache_stats` |
| `read:compression` | `compression_status`, `list_compression_combos`, `compression_combo_stats` |
| `write:combos` | `switch_combo` |
| `write:budget` | `set_budget_guard` |
| `write:resilience` | `set_resilience_profile` |
| `write:cache` | `cache_flush` |
| `write:compression` | `compression_configure`, `set_compression_engine` |
| `execute:completions` | `route_request`, `test_combo` |
**Wildcard scopes:** Use `read:*` to grant all read scopes, or `*` for full access.
---
## Audit Logging
Every tool call is logged to the `mcp_tool_audit` SQLite table:
- **Input:** SHA-256 hashed (never stores raw prompts)
- **Output:** Truncated to 200 chars
- **Metadata:** Tool name, duration, success/error, API key ID
Access audit data via:
```typescript
import { getRecentAuditEntries, getAuditStats } from "./audit";
const entries = await getRecentAuditEntries(50);
const stats = await getAuditStats();
// stats: { totalCalls, successRate, avgDurationMs, topTools }
```
---
## File Structure
```
mcp-server/
├── server.ts # MCP server setup, essential tool handlers, entry point
├── index.ts # Barrel export
├── audit.ts # SQLite audit logger (SHA-256 input hashing)
├── scopeEnforcement.ts # Fine-grained scope enforcement
├── schemas/
│ ├── tools.ts # Zod schemas for core, cache, compression, and proxy tools
│ ├── a2a.ts # A2A protocol types (Agent Card, Task, JSON-RPC)
│ ├── audit.ts # Audit & routing decision types + hash helpers
│ └── index.ts # Schema barrel export
├── tools/
│ └── advancedTools.ts # Phase 2 tool handlers (8 advanced tools)
└── __tests__/
├── essentialTools.test.ts
├── advancedTools.test.ts
└── a2aLifecycle.test.ts
```
---
## License
Part of [OmniRoute](https://github.com/diegosouzapw/OmniRoute) — MIT License.