mirror of
https://github.com/diegosouzapw/OmniRoute.git
synced 2026-08-20 06:02:14 +03:00
feat(mcp): surface adaptive admission lane data in omniroute_get_health (#9654 Wave 2)
U8: make adaptive virtual-lane admission visible to agents via the MCP health tool. handleGetHealth now surfaces a curated adaptiveAdmission block from the health payload (which already carried the runtime snapshot but was dropping it): virtualLanes/pressure/utilization/laneCount/laneQueuedCount/laneQueuedCost, laneTenants capped at top-10 by queued cost, admitted/rejected/wouldReject counts, shutdown. Block omitted entirely when the health endpoint reports none. isLaneFlagOn mirrors the runtime 1|true convention so a string serialization can never invert a boolean lane report. getHealthOutput schema extended with the matching optional shape; tool description updated. 4 new dispatch tests (full block, top-10 cap/order, omission, defensive coercion of string flags + malformed lane entries) - 22/22 in essentialTools.test.ts. README: Adaptive Admission Lane Data table + Skills & Tool Navigability audit (29/43 schema entries covered, 14 undocumented, tool_search keyword runtime discovery, full catalog in docs/frameworks/MCP-SERVER.md). No new lint errors (4 pre-existing in server.ts), typecheck core clean, doc counts + fabricated-docs gates green.
This commit is contained in:
Brandon Bennett
@@ -1,6 +1,6 @@
|
||||
# OmniRoute MCP Server
|
||||
|
||||
> **Model Context Protocol server** that exposes OmniRoute's gateway intelligence as **109 tools** for AI agents.
|
||||
> **Model Context Protocol server** that exposes OmniRoute's gateway intelligence as **107 tools** for AI agents.
|
||||
>
|
||||
> **Source of truth for the full tool catalog and REST surface:** [`docs/frameworks/MCP-SERVER.md`](../../docs/frameworks/MCP-SERVER.md). This README focuses on architecture, configuration, and integration examples; the catalog below is a summary subset.
|
||||
|
||||
@@ -20,7 +20,7 @@ The MCP Server allows any AI agent (Claude Desktop, Cursor, VS Code Copilot, cus
|
||||
┌──────────────────────────────────────────────────────────────────┐
|
||||
│ OmniRoute MCP Server │
|
||||
│ ┌──────────────┐ ┌─────────────────┐ ┌────────────────────┐ │
|
||||
│ │ Scope │ │ 109 MCP Tools │ │ Audit Logger │ │
|
||||
│ │ Scope │ │ 107 MCP Tools │ │ Audit Logger │ │
|
||||
│ │ Enforcement │──│ (core + memory │──│ (SHA-256/SQLite) │ │
|
||||
│ │ │ │ + skills + …) │ │ │ │
|
||||
│ └──────────────┘ └────────┬────────┘ └────────────────────┘ │
|
||||
@@ -120,23 +120,18 @@ omniroute --mcp
|
||||
|
||||
## Tool Reference
|
||||
|
||||
### Phase 1: Essential Tools (13)
|
||||
### Phase 1: Essential Tools (8)
|
||||
|
||||
| # | Tool | Scopes | Description |
|
||||
| --- | ------------------------------- | --------------------- | -------------------------------------------------------------------------- |
|
||||
| 1 | `omniroute_tool_search` | `read:tools` | Discover tools from the registered MCP catalog |
|
||||
| 2 | `omniroute_get_health` | `read:health` | Gateway health, uptime, memory, circuit breakers, rate limits, cache stats |
|
||||
| 3 | `omniroute_list_combos` | `read:combos` | List all combos (model chains) with strategies and optional metrics |
|
||||
| 4 | `omniroute_get_combo_metrics` | `read:combos` | Performance metrics for a specific combo |
|
||||
| 5 | `omniroute_switch_combo` | `write:combos` | Activate or deactivate a combo for routing |
|
||||
| 6 | `omniroute_create_combo` | `write:combos` | Create a validated combo through the existing combo API |
|
||||
| 7 | `omniroute_check_quota` | `read:quota` | Remaining API quota per provider with token health status |
|
||||
| 8 | `omniroute_route_request` | `execute:completions` | Send a chat completion through intelligent routing |
|
||||
| 9 | `omniroute_cost_report` | `read:usage` | Cost report by period (session/day/week/month) with per-provider breakdown |
|
||||
| 10 | `omniroute_list_models_catalog` | `read:models` | List all available models across providers with capabilities and pricing |
|
||||
| 11 | `omniroute_radar_catalog` | `read:radar` | Read the local signed Radar catalog with provider/family filters |
|
||||
| 12 | `omniroute_web_search` | `execute:search` | Search the web through configured search providers |
|
||||
| 13 | `omniroute_web_fetch` | `execute:search` | Fetch web content through configured fetch providers |
|
||||
| # | Tool | Scopes | Description |
|
||||
| --- | ------------------------------- | --------------------- | --------------------------------------------------------------------------------------------------- |
|
||||
| 1 | `omniroute_get_health` | `read:health` | Gateway health, uptime, memory, circuit breakers, rate limits, cache stats + adaptive lane pressure |
|
||||
| 2 | `omniroute_list_combos` | `read:combos` | List all combos (model chains) with strategies and optional metrics |
|
||||
| 3 | `omniroute_get_combo_metrics` | `read:combos` | Performance metrics for a specific combo |
|
||||
| 4 | `omniroute_switch_combo` | `write:combos` | Activate or deactivate a combo for routing |
|
||||
| 5 | `omniroute_check_quota` | `read:quota` | Remaining API quota per provider with token health status |
|
||||
| 6 | `omniroute_route_request` | `execute:completions` | Send a chat completion through intelligent routing |
|
||||
| 7 | `omniroute_cost_report` | `read:usage` | Cost report by period (session/day/week/month) with per-provider breakdown |
|
||||
| 8 | `omniroute_list_models_catalog` | `read:models` | List all available models across providers with capabilities and pricing |
|
||||
|
||||
### Phase 2: Advanced Tools (8)
|
||||
|
||||
@@ -180,6 +175,45 @@ mistake metadata shrink estimates for provider token receipts.
|
||||
|
||||
---
|
||||
|
||||
### Adaptive Admission Lane Data
|
||||
|
||||
`omniroute_get_health` includes an `adaptiveAdmission` block whenever the gateway's adaptive
|
||||
virtual-lane admission is active. It is a curated subset of the live admission snapshot:
|
||||
|
||||
| Field | Meaning |
|
||||
| ------------------ | ---------------------------------------------------------------------- |
|
||||
| `virtualLanes` | Whether per-tenant virtual-lane admission is enabled |
|
||||
| `pressure` | Current pressure state (e.g. `healthy`, `high`, `critical`) |
|
||||
| `utilization` | Current capacity utilization (0.0–1.0) |
|
||||
| `laneCount` | Number of live lanes |
|
||||
| `laneQueuedCount` | Total requests queued across lanes |
|
||||
| `laneQueuedCost` | Total estimated cost queued across lanes |
|
||||
| `laneTenants` | Top 10 lanes by queued cost (`tenantKey`, `queuedCount`, `queuedCost`) |
|
||||
| `admittedCount` | Requests admitted since boot |
|
||||
| `rejectedCount` | Requests rejected since boot |
|
||||
| `wouldRejectCount` | Requests that would be rejected under the current limit |
|
||||
| `shutdown` | Whether the admission runtime is shutting down |
|
||||
|
||||
`tenantKey` is an opaque per-API-key derived identifier, never the raw key. The block is omitted
|
||||
entirely when the health endpoint reports no adaptive-admission data.
|
||||
|
||||
### Skills & Tool Navigability
|
||||
|
||||
The catalog rows above are a curated summary, not the full surface: they cover 29 of the 43 schema
|
||||
entries in `schemas/` (audit snapshot — expect drift as the catalog grows), and the authoritative
|
||||
full catalog lives in
|
||||
[`docs/frameworks/MCP-SERVER.md`](../../docs/frameworks/MCP-SERVER.md). The schema entries without a
|
||||
README row are the agent-skills group (`agent_skills_coverage` / `agent_skills_get` /
|
||||
`agent_skills_list`), the oneproxy group (`oneproxy_fetch` / `oneproxy_rotate` / `oneproxy_stats`),
|
||||
`web_fetch` / `web_search`, `tool_search`, `create_combo`, `set_routing_strategy`,
|
||||
`pick_fastest_model`, `sync_pricing`, and `db_health_check`.
|
||||
|
||||
Agents never need the README to find these: `omniroute_tool_search` performs keyword search
|
||||
across the registered tool set and returns compact signatures (token-efficient discovery), so
|
||||
undiscovered capabilities stay discoverable at runtime.
|
||||
|
||||
---
|
||||
|
||||
## Client Examples
|
||||
|
||||
### Python — Full Agent Workflow
|
||||
|
||||
@@ -400,4 +400,120 @@ describe("omniroute_get_health handler (via MCP dispatch)", () => {
|
||||
const data = JSON.parse(content[0].text);
|
||||
expect(data.degraded).toBeUndefined();
|
||||
});
|
||||
|
||||
it("should surface the curated adaptive-admission lane block when health carries it", async () => {
|
||||
mockHealthSources({
|
||||
health: {
|
||||
uptime: 100,
|
||||
version: "3.8.50",
|
||||
adaptiveAdmission: {
|
||||
virtualLanes: true,
|
||||
pressure: "high",
|
||||
utilization: 0.72,
|
||||
laneCount: 3,
|
||||
laneQueuedCount: 12,
|
||||
laneQueuedCost: 340,
|
||||
laneTenants: [
|
||||
{ tenantKey: "lane-a", queuedCount: 6, queuedCost: 200 },
|
||||
{ tenantKey: "lane-b", queuedCount: 4, queuedCost: 90 },
|
||||
{ tenantKey: "lane-c", queuedCount: 2, queuedCost: 50 },
|
||||
],
|
||||
admittedCount: 900,
|
||||
rejectedCount: 7,
|
||||
wouldRejectCount: 3,
|
||||
shutdown: false,
|
||||
},
|
||||
},
|
||||
resilience: { circuitBreakers: [] },
|
||||
rateLimits: { limits: [] },
|
||||
});
|
||||
|
||||
const result = await client.callTool({ name: "omniroute_get_health", arguments: {} });
|
||||
|
||||
expect(result.isError).toBeFalsy();
|
||||
const content = result.content as Array<{ type: string; text: string }>;
|
||||
const data = JSON.parse(content[0].text);
|
||||
expect(data.adaptiveAdmission.virtualLanes).toBe(true);
|
||||
expect(data.adaptiveAdmission.pressure).toBe("high");
|
||||
expect(data.adaptiveAdmission.utilization).toBe(0.72);
|
||||
expect(data.adaptiveAdmission.laneTenants).toHaveLength(3);
|
||||
expect(data.adaptiveAdmission.laneTenants[0]).toEqual({
|
||||
tenantKey: "lane-a",
|
||||
queuedCount: 6,
|
||||
queuedCost: 200,
|
||||
});
|
||||
expect(data.adaptiveAdmission.admittedCount).toBe(900);
|
||||
expect(data.adaptiveAdmission.rejectedCount).toBe(7);
|
||||
expect(data.adaptiveAdmission.wouldRejectCount).toBe(3);
|
||||
expect(data.adaptiveAdmission.shutdown).toBe(false);
|
||||
});
|
||||
|
||||
it("should coerce string lane flags and malformed lane entries defensively", async () => {
|
||||
mockHealthSources({
|
||||
health: {
|
||||
uptime: 1,
|
||||
version: "x",
|
||||
adaptiveAdmission: {
|
||||
virtualLanes: "true",
|
||||
shutdown: "false",
|
||||
laneTenants: ["garbage", { tenantKey: "ok", queuedCount: 2, queuedCost: 7 }],
|
||||
},
|
||||
},
|
||||
resilience: {},
|
||||
rateLimits: {},
|
||||
});
|
||||
|
||||
const result = await client.callTool({ name: "omniroute_get_health", arguments: {} });
|
||||
|
||||
const content = result.content as Array<{ type: string; text: string }>;
|
||||
const data = JSON.parse(content[0].text);
|
||||
// "true" string counts as on; "false" string must NOT invert to on.
|
||||
expect(data.adaptiveAdmission.virtualLanes).toBe(true);
|
||||
expect(data.adaptiveAdmission.shutdown).toBe(false);
|
||||
// Malformed entries degrade to zeroed records instead of throwing.
|
||||
expect(data.adaptiveAdmission.laneTenants).toEqual([
|
||||
{ tenantKey: "ok", queuedCount: 2, queuedCost: 7 },
|
||||
{ tenantKey: "", queuedCount: 0, queuedCost: 0 },
|
||||
]);
|
||||
});
|
||||
|
||||
it("should cap laneTenants at the top 10 by queued cost", async () => {
|
||||
const laneTenants = Array.from({ length: 12 }, (_, i) => ({
|
||||
tenantKey: `tenant-${i}`,
|
||||
queuedCount: i,
|
||||
queuedCost: i * 10,
|
||||
}));
|
||||
mockHealthSources({
|
||||
health: {
|
||||
uptime: 1,
|
||||
version: "x",
|
||||
adaptiveAdmission: { virtualLanes: true, laneTenants },
|
||||
},
|
||||
resilience: {},
|
||||
rateLimits: {},
|
||||
});
|
||||
|
||||
const result = await client.callTool({ name: "omniroute_get_health", arguments: {} });
|
||||
|
||||
const content = result.content as Array<{ type: string; text: string }>;
|
||||
const data = JSON.parse(content[0].text);
|
||||
expect(data.adaptiveAdmission.laneTenants).toHaveLength(10);
|
||||
// Highest queued cost first, lowest dropped from the cap.
|
||||
expect(data.adaptiveAdmission.laneTenants[0].tenantKey).toBe("tenant-11");
|
||||
expect(data.adaptiveAdmission.laneTenants[9].tenantKey).toBe("tenant-2");
|
||||
});
|
||||
|
||||
it("should omit adaptiveAdmission entirely when the health payload has none", async () => {
|
||||
mockHealthSources({
|
||||
health: { uptime: 1, version: "x" },
|
||||
resilience: { circuitBreakers: [] },
|
||||
rateLimits: { limits: [] },
|
||||
});
|
||||
|
||||
const result = await client.callTool({ name: "omniroute_get_health", arguments: {} });
|
||||
|
||||
const content = result.content as Array<{ type: string; text: string }>;
|
||||
const data = JSON.parse(content[0].text);
|
||||
expect(data).not.toHaveProperty("adaptiveAdmission");
|
||||
});
|
||||
});
|
||||
|
||||
@@ -67,6 +67,27 @@ export const getHealthOutput = z.object({
|
||||
provider: z.string(),
|
||||
})
|
||||
.optional(),
|
||||
adaptiveAdmission: z
|
||||
.object({
|
||||
virtualLanes: z.boolean(),
|
||||
pressure: z.string(),
|
||||
utilization: z.number(),
|
||||
laneCount: z.number(),
|
||||
laneQueuedCount: z.number(),
|
||||
laneQueuedCost: z.number(),
|
||||
laneTenants: z.array(
|
||||
z.object({
|
||||
tenantKey: z.string(),
|
||||
queuedCount: z.number(),
|
||||
queuedCost: z.number(),
|
||||
})
|
||||
),
|
||||
admittedCount: z.number(),
|
||||
rejectedCount: z.number(),
|
||||
wouldRejectCount: z.number(),
|
||||
shutdown: z.boolean(),
|
||||
})
|
||||
.optional(),
|
||||
degraded: z
|
||||
.array(
|
||||
z.object({
|
||||
@@ -80,7 +101,7 @@ export const getHealthOutput = z.object({
|
||||
export const getHealthTool: McpToolDefinition<typeof getHealthInput, typeof getHealthOutput> = {
|
||||
name: "omniroute_get_health",
|
||||
description:
|
||||
"Returns the current health status of OmniRoute including uptime, memory usage, circuit breaker states for all providers, rate limit status, and cache statistics. If an underlying source (health/resilience/rate-limits) could not be reached, it is listed in `degraded` instead of being silently reported as empty/zero.",
|
||||
"Returns the current health status of OmniRoute including uptime, memory usage, circuit breaker states for all providers, rate limit status, and cache statistics. When adaptive virtual-lane admission is active, a curated `adaptiveAdmission` block reports per-lane queue pressure (top tenants by queued cost). If an underlying source (health/resilience/rate-limits) could not be reached, it is listed in `degraded` instead of being silently reported as empty/zero.",
|
||||
inputSchema: getHealthInput,
|
||||
outputSchema: getHealthOutput,
|
||||
scopes: ["read:health"],
|
||||
|
||||
@@ -164,6 +164,12 @@ function toNumber(value: unknown, fallback = 0): number {
|
||||
return typeof value === "number" && Number.isFinite(value) ? value : fallback;
|
||||
}
|
||||
|
||||
// Mirrors the runtime's env convention for lane flags ("1" | "true" are on) so a
|
||||
// future string serialization can never silently invert a boolean lane report.
|
||||
function isLaneFlagOn(value: unknown): boolean {
|
||||
return value === true || value === "1" || value === "true";
|
||||
}
|
||||
|
||||
function toStringArray(value: unknown, fallback: string[] = []): string[] {
|
||||
const values = toArray(value).filter((entry): entry is string => typeof entry === "string");
|
||||
return values.length > 0 ? values : fallback;
|
||||
@@ -292,6 +298,20 @@ async function handleGetHealth() {
|
||||
const cacheStatsRaw = toRecord(health.cacheStats);
|
||||
const resilienceCircuitBreakers = toArray(resilience.circuitBreakers);
|
||||
const rateLimitEntries = toArray(rateLimits.limits);
|
||||
const adaptiveAdmissionRaw = toRecord(health.adaptiveAdmission);
|
||||
// Curated lane subset: top lanes by queued cost so a congested tenant is
|
||||
// visible first without shipping the whole admission snapshot to agents.
|
||||
const laneTenants = toArray(adaptiveAdmissionRaw.laneTenants)
|
||||
.map((tenant) => {
|
||||
const record = toRecord(tenant);
|
||||
return {
|
||||
tenantKey: toString(record.tenantKey),
|
||||
queuedCount: toNumber(record.queuedCount, 0),
|
||||
queuedCost: toNumber(record.queuedCost, 0),
|
||||
};
|
||||
})
|
||||
.sort((a, b) => b.queuedCost - a.queuedCost)
|
||||
.slice(0, 10);
|
||||
|
||||
// Surface fetch failures instead of letting Promise.allSettled's {} fallback
|
||||
// masquerade as genuine zero/empty data (indistinguishable "no data" vs.
|
||||
@@ -333,6 +353,22 @@ async function handleGetHealth() {
|
||||
provider: toString(toRecord(health.cryptography).provider, "unknown"),
|
||||
}
|
||||
: undefined,
|
||||
adaptiveAdmission:
|
||||
Object.keys(adaptiveAdmissionRaw).length > 0
|
||||
? {
|
||||
virtualLanes: isLaneFlagOn(adaptiveAdmissionRaw.virtualLanes),
|
||||
pressure: toString(adaptiveAdmissionRaw.pressure),
|
||||
utilization: toNumber(adaptiveAdmissionRaw.utilization, 0),
|
||||
laneCount: toNumber(adaptiveAdmissionRaw.laneCount, 0),
|
||||
laneQueuedCount: toNumber(adaptiveAdmissionRaw.laneQueuedCount, 0),
|
||||
laneQueuedCost: toNumber(adaptiveAdmissionRaw.laneQueuedCost, 0),
|
||||
laneTenants,
|
||||
admittedCount: toNumber(adaptiveAdmissionRaw.admittedCount, 0),
|
||||
rejectedCount: toNumber(adaptiveAdmissionRaw.rejectedCount, 0),
|
||||
wouldRejectCount: toNumber(adaptiveAdmissionRaw.wouldRejectCount, 0),
|
||||
shutdown: isLaneFlagOn(adaptiveAdmissionRaw.shutdown),
|
||||
}
|
||||
: undefined,
|
||||
degraded: degraded.length > 0 ? degraded : undefined,
|
||||
};
|
||||
|
||||
|
||||
Reference in New Issue
Block a user