mirror of
https://github.com/diegosouzapw/OmniRoute.git
synced 2026-07-26 09:52:11 +03:00
feat: add principal-scoped CCR MCP lifecycle (#7282)
* feat: add principal-scoped CCR MCP lifecycle * refactor: extract CCR MCP schemas * refactor: reduce CCR store complexity * fix: preserve CCR retrieval feedback * fix: keep CCR expiry/accounting scoped to accessed entries --------- Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
This commit is contained in:
@@ -6,13 +6,9 @@ lastUpdated: 2026-06-28
|
||||
|
||||
# OmniRoute MCP Server Documentation
|
||||
|
||||
> Model Context Protocol server with 94 tools across routing, cache, compression, memory, skills, proxy, pool, and context source operations.
|
||||
> Model Context Protocol server with 104 tools across routing, cache, compression, memory, skills, proxy, pool, and context source operations.
|
||||
>
|
||||
> Source of truth: `open-sse/mcp-server/schemas/tools.ts` (34 base) + `memoryTools.ts` (3) + `skillTools.ts` (4) + `agentSkillTools.ts` (3) + `poolTools.ts` (6) + `gamificationTools.ts` (8) + `pluginTools.ts` (8) + `notionTools.ts` (6) + `obsidianTools.ts` (22) = **94** (`TOTAL_MCP_TOOL_COUNT`). Tool registration and scope wiring lives in `open-sse/mcp-server/server.ts`.
|
||||
|
||||

|
||||
|
||||
> Source: [diagrams/mcp-tools-94.mmd](../diagrams/mcp-tools-94.mmd) (regenerate via `npm run docs:render-diagrams`).
|
||||
> Source of truth: `open-sse/mcp-server/server.ts` computes **104 unique tools** with `countUniqueMcpTools()`: 42 canonical definitions (including the six CCR lifecycle tools and the agent-skills trio), plus memory (3), skills (4), GitHub skills (3), pool (6), gamification (8), plugins (8), Notion (6), Obsidian (22), and two RTK-only compression tools.
|
||||
|
||||
## Installation
|
||||
|
||||
@@ -110,7 +106,7 @@ Cursor, Cline, and compatible MCP client setup.
|
||||
| `omniroute_cache_stats` | `read:cache` | Semantic cache, prompt-cache, and idempotency stats |
|
||||
| `omniroute_cache_flush` | `write:cache` | Flush cache globally or by signature/model |
|
||||
|
||||
## Compression Tools (5)
|
||||
## Compression Tools (13)
|
||||
|
||||
| Tool | Scopes | Description |
|
||||
| :---------------------------------- | :------------------ | :----------------------------------------------------------------------------------------------------------------------- |
|
||||
@@ -119,6 +115,20 @@ Cursor, Cline, and compatible MCP client setup.
|
||||
| `omniroute_set_compression_engine` | `write:compression` | Pick the active engine (off/caveman/rtk/stacked) and Caveman/RTK intensity |
|
||||
| `omniroute_list_compression_combos` | `read:compression` | List named compression combos and their engine pipelines |
|
||||
| `omniroute_compression_combo_stats` | `read:compression` | Analytics grouped by compression combo and engine |
|
||||
| `omniroute_ccr_store` | `write:compression` | Store caller-isolated content in the bounded in-memory CCR store and return a marker plus `ccr://` reference |
|
||||
| `omniroute_ccr_retrieve` | `read:compression` | Retrieve CCR content in full or with head, tail, lines, grep, and stats modes |
|
||||
| `omniroute_ccr_inspect` | `read:compression` | Inspect caller-owned CCR metadata without returning content |
|
||||
| `omniroute_ccr_list` | `read:compression` | List paginated metadata for caller-owned CCR blocks |
|
||||
| `omniroute_ccr_delete` | `write:compression` | Delete a caller-owned CCR block |
|
||||
| `omniroute_ccr_stats` | `read:compression` | Report caller-scoped memory usage, lifecycle counters, and store limits |
|
||||
| `omniroute_rtk_discover` | `read:compression` | Discover recurring noise in opt-in RTK output samples |
|
||||
| `omniroute_rtk_learn` | `read:compression` | Generate a reviewable RTK filter draft from opt-in samples |
|
||||
|
||||
CCR entries are in-memory only and disappear on restart. Each block is limited to 2 MiB, each
|
||||
principal to 16 MiB, and the global store to 64 MiB. Entries default to a 24-hour TTL (maximum
|
||||
seven days). Full MCP retrieval is limited to 256 KiB; larger blocks remain available through the
|
||||
ranged and grep modes. Storage, retrieval, listing, inspection, deletion, and stats are isolated by
|
||||
the authenticated API-key principal. Audit records contain hashes and size metadata, never content.
|
||||
|
||||
`omniroute_compression_status` reports MCP description compression separately under
|
||||
`analytics.mcpDescriptionCompression`. Those values are metadata-size estimates for MCP listable
|
||||
@@ -127,7 +137,7 @@ receipts and are marked with `source: "mcp_metadata_estimate"`.
|
||||
|
||||
### MCP Accessibility Tree Filter (v3.8.0)
|
||||
|
||||
Separate from the 5 compression tools above, OmniRoute includes a post-execution filter that
|
||||
Separate from the compression tools above, OmniRoute includes a post-execution filter that
|
||||
compresses the **tool results** of MCP browser/accessibility tools before they are returned to the
|
||||
agent. This filter is not itself a tool — it runs transparently on any tool result that contains
|
||||
verbose accessibility-tree or browser-snapshot text (≥2000 chars).
|
||||
@@ -217,7 +227,7 @@ See [AGENT-SKILLS.md](./AGENT-SKILLS.md) for the full catalog and how external a
|
||||
|
||||
## Related Frameworks (v3.8.0)
|
||||
|
||||
The MCP tool inventory above (94 tools = 34 core + 3 memory + 4 skills + 3 agent-skills + 6 pool + 8 gamification + 8 plugins + 6 notion + 22 obsidian) is intentionally
|
||||
The MCP tool inventory above (104 unique tools, computed by `countUniqueMcpTools()`) is intentionally
|
||||
scoped to runtime routing/cache/compression/memory/skills/proxy/context-source operations. Two adjacent
|
||||
frameworks ship alongside the MCP server in v3.8.0 and are documented separately:
|
||||
|
||||
@@ -331,7 +341,7 @@ MCP tool, prompt, and resource registries can compress descriptions at registrat
|
||||
|
||||
Description compression shrinks each tool's metadata; **tool-cardinality reduction** goes one step further by reducing _how many_ tools are announced at all. Advertising fewer tools in the `tools/list` manifest cuts the per-request token cost the client's model pays for the tool catalog ("layer 5" compression). The implementation is a pure, stateless filter in `open-sse/mcp-server/toolCardinality.ts` (`reduceToolManifest`), wired into the registration loop in `createMcpServer()` (`open-sse/mcp-server/server.ts`).
|
||||
|
||||
**Opt-in, off by default.** The filter only runs when at least one of two environment variables is set; with neither set, all 94 tools are announced unchanged.
|
||||
**Opt-in, off by default.** The filter only runs when at least one of two environment variables is set; with neither set, all 104 tools are announced unchanged.
|
||||
|
||||
| Variable | Mode |
|
||||
| :--------------- | :-------------------------------------------------------------------------------------- |
|
||||
|
||||
@@ -1,8 +1,8 @@
|
||||
# OmniRoute MCP Server
|
||||
|
||||
> **Model Context Protocol server** that exposes OmniRoute's gateway intelligence as **37 tools** for AI agents.
|
||||
> **Model Context Protocol server** that exposes OmniRoute's gateway intelligence as **104 tools** for AI agents.
|
||||
>
|
||||
> **Source of truth for the full tool catalog and REST surface:** [`docs/frameworks/MCP-SERVER.md`](../../docs/MCP-SERVER.md). This README focuses on architecture, configuration, and integration examples; the catalog below is a summary subset.
|
||||
> **Source of truth for the full tool catalog and REST surface:** [`docs/frameworks/MCP-SERVER.md`](../../docs/frameworks/MCP-SERVER.md). This README focuses on architecture, configuration, and integration examples; the catalog below is a summary subset.
|
||||
|
||||
The MCP Server allows any AI agent (Claude Desktop, Cursor, VS Code Copilot, custom agents) to **monitor, control, and optimize** the OmniRoute AI gateway programmatically.
|
||||
|
||||
@@ -20,7 +20,7 @@ The MCP Server allows any AI agent (Claude Desktop, Cursor, VS Code Copilot, cus
|
||||
┌──────────────────────────────────────────────────────────────────┐
|
||||
│ OmniRoute MCP Server │
|
||||
│ ┌──────────────┐ ┌─────────────────┐ ┌────────────────────┐ │
|
||||
│ │ Scope │ │ 37 MCP Tools │ │ Audit Logger │ │
|
||||
│ │ Scope │ │ 104 MCP Tools │ │ Audit Logger │ │
|
||||
│ │ Enforcement │──│ (core + memory │──│ (SHA-256/SQLite) │ │
|
||||
│ │ │ │ + skills + …) │ │ │ │
|
||||
│ └──────────────┘ └────────┬────────┘ └────────────────────┘ │
|
||||
@@ -157,6 +157,16 @@ omniroute --mcp
|
||||
| 25 | `omniroute_set_compression_engine` | `write:compression` | Set Caveman, RTK, or stacked compression mode and pipeline |
|
||||
| 26 | `omniroute_list_compression_combos` | `read:compression` | List named compression combos and routing assignments |
|
||||
| 27 | `omniroute_compression_combo_stats` | `read:compression` | Read analytics grouped by compression combo and engine |
|
||||
| 28 | `omniroute_ccr_store` | `write:compression` | Store content in the caller-isolated in-memory CCR store |
|
||||
| 29 | `omniroute_ccr_retrieve` | `read:compression` | Retrieve full or ranged caller-owned CCR content |
|
||||
| 30 | `omniroute_ccr_inspect` | `read:compression` | Inspect CCR metadata without returning content |
|
||||
| 31 | `omniroute_ccr_list` | `read:compression` | List paginated caller-owned CCR metadata |
|
||||
| 32 | `omniroute_ccr_delete` | `write:compression` | Delete a caller-owned CCR block |
|
||||
| 33 | `omniroute_ccr_stats` | `read:compression` | Report caller usage, bounded-store limits, and lifecycle counters |
|
||||
|
||||
CCR storage is bounded and in-memory only: 2 MiB per block, 16 MiB per principal, 64 MiB global,
|
||||
with a 24-hour default TTL. Full MCP retrieval is capped at 256 KiB; larger blocks use ranged or
|
||||
grep retrieval. All lifecycle operations are isolated by the authenticated caller principal.
|
||||
|
||||
MCP listable metadata descriptions are compressed at registration/list time when description
|
||||
compression is enabled. `omniroute_compression_status` exposes those savings separately as
|
||||
|
||||
@@ -18,10 +18,9 @@ describe("getAllToolDefinitions", () => {
|
||||
const names = all.map((t) => t.name);
|
||||
expect(new Set(names).size).toBe(names.length);
|
||||
});
|
||||
it("includes compressionTools-only entries (omniroute_ccr_retrieve, not in MCP_TOOLS)", () => {
|
||||
// Regression: compressionTools carries omniroute_ccr_retrieve, which is absent from
|
||||
// MCP_TOOLS — if the collection is dropped from the catalog, tool_search can never
|
||||
// surface it. Guards against the catalog omission caught in core review.
|
||||
expect(all.find((t) => t.name === "omniroute_ccr_retrieve")).toBeTruthy();
|
||||
it("includes every canonical CCR lifecycle tool", () => {
|
||||
for (const name of ["store", "retrieve", "inspect", "list", "delete", "stats"]) {
|
||||
expect(all.find((tool) => tool.name === `omniroute_ccr_${name}`)).toBeTruthy();
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
202
open-sse/mcp-server/schemas/ccrTools.ts
Normal file
202
open-sse/mcp-server/schemas/ccrTools.ts
Normal file
@@ -0,0 +1,202 @@
|
||||
import { z } from "zod";
|
||||
|
||||
import type { McpToolDefinition } from "./toolDefinition.ts";
|
||||
|
||||
const ccrHash = z
|
||||
.string()
|
||||
.regex(/^[a-f0-9]{24}$/i)
|
||||
.describe("24-hex content hash from a CCR marker or ccr:// URI");
|
||||
|
||||
export const ccrEntryMetadataOutput = z.object({
|
||||
hash: ccrHash,
|
||||
bytes: z.number().int().nonnegative(),
|
||||
chars: z.number().int().nonnegative(),
|
||||
lines: z.number().int().nonnegative(),
|
||||
contentType: z.string(),
|
||||
source: z.enum(["compression", "mcp", "ionizer", "session-dedup"]),
|
||||
createdAt: z.number().int().nonnegative(),
|
||||
lastAccessedAt: z.number().int().nonnegative(),
|
||||
expiresAt: z.number().int().nonnegative(),
|
||||
retrievalCount: z.number().int().nonnegative(),
|
||||
});
|
||||
|
||||
export const ccrReferenceOutput = z.object({
|
||||
hash: ccrHash,
|
||||
uri: z.string().startsWith("ccr://"),
|
||||
marker: z.string(),
|
||||
});
|
||||
|
||||
export const ccrStoreInput = z.object({
|
||||
content: z
|
||||
.string()
|
||||
.min(1)
|
||||
.refine((content) => Buffer.byteLength(content, "utf8") <= 2 * 1024 * 1024, {
|
||||
message: "Content exceeds the 2 MiB UTF-8 CCR block limit",
|
||||
})
|
||||
.describe("Verbatim content to keep in the in-memory CCR store (maximum 2 MiB UTF-8)"),
|
||||
contentType: z.string().trim().min(1).max(128).optional(),
|
||||
ttlSeconds: z
|
||||
.number()
|
||||
.int()
|
||||
.min(60)
|
||||
.max(7 * 24 * 60 * 60)
|
||||
.optional(),
|
||||
});
|
||||
|
||||
export const ccrStoreOutput = z.union([
|
||||
z.object({
|
||||
stored: z.literal(true),
|
||||
reference: ccrReferenceOutput,
|
||||
metadata: ccrEntryMetadataOutput,
|
||||
}),
|
||||
z.object({
|
||||
stored: z.literal(false),
|
||||
reason: z.enum(["block_too_large", "principal_budget_exceeded", "global_budget_exceeded"]),
|
||||
}),
|
||||
]);
|
||||
|
||||
export const ccrStoreTool: McpToolDefinition<typeof ccrStoreInput, typeof ccrStoreOutput> = {
|
||||
name: "omniroute_ccr_store",
|
||||
description:
|
||||
"Store verbatim content in the caller-isolated in-memory CCR store and return a ccr:// reference plus the compatible CCR marker. Entries expire automatically and are not persisted across restarts.",
|
||||
inputSchema: ccrStoreInput,
|
||||
outputSchema: ccrStoreOutput,
|
||||
scopes: ["write:compression"],
|
||||
auditLevel: "basic",
|
||||
phase: 2,
|
||||
sourceEndpoints: [],
|
||||
};
|
||||
|
||||
export const ccrRetrieveInput = z.object({
|
||||
hash: ccrHash,
|
||||
mode: z.enum(["full", "head", "tail", "lines", "grep", "stats"]).optional(),
|
||||
n: z.number().int().positive().max(10_000).optional(),
|
||||
start: z.number().int().positive().optional(),
|
||||
end: z.number().int().positive().optional(),
|
||||
pattern: z.string().max(512).optional(),
|
||||
unique: z.boolean().optional(),
|
||||
});
|
||||
|
||||
export const ccrRetrieveOutput = z.union([
|
||||
z.object({
|
||||
found: z.literal(false),
|
||||
error: z.string(),
|
||||
}),
|
||||
z.object({
|
||||
found: z.literal(true),
|
||||
metadata: ccrEntryMetadataOutput,
|
||||
content: z.string().optional(),
|
||||
tooLargeForFull: z.boolean().optional(),
|
||||
suggestedModes: z.array(z.enum(["head", "tail", "lines", "grep", "stats"])).optional(),
|
||||
error: z.string().optional(),
|
||||
}),
|
||||
]);
|
||||
|
||||
export const ccrRetrieveTool: McpToolDefinition<typeof ccrRetrieveInput, typeof ccrRetrieveOutput> =
|
||||
{
|
||||
name: "omniroute_ccr_retrieve",
|
||||
description:
|
||||
"Retrieve caller-owned CCR content by hash. Full MCP responses are capped at 256 KiB; use head, tail, lines, grep, or stats for larger blocks.",
|
||||
inputSchema: ccrRetrieveInput,
|
||||
outputSchema: ccrRetrieveOutput,
|
||||
scopes: ["read:compression"],
|
||||
auditLevel: "basic",
|
||||
phase: 2,
|
||||
sourceEndpoints: ["/api/compression/retrieve"],
|
||||
};
|
||||
|
||||
export const ccrInspectInput = z.object({ hash: ccrHash });
|
||||
export const ccrInspectOutput = z.union([
|
||||
z.object({ found: z.literal(false) }),
|
||||
z.object({
|
||||
found: z.literal(true),
|
||||
reference: ccrReferenceOutput,
|
||||
metadata: ccrEntryMetadataOutput,
|
||||
}),
|
||||
]);
|
||||
export const ccrInspectTool: McpToolDefinition<typeof ccrInspectInput, typeof ccrInspectOutput> = {
|
||||
name: "omniroute_ccr_inspect",
|
||||
description: "Inspect metadata for a caller-owned CCR block without returning its content.",
|
||||
inputSchema: ccrInspectInput,
|
||||
outputSchema: ccrInspectOutput,
|
||||
scopes: ["read:compression"],
|
||||
auditLevel: "basic",
|
||||
phase: 2,
|
||||
sourceEndpoints: [],
|
||||
};
|
||||
|
||||
export const ccrListInput = z.object({
|
||||
offset: z.number().int().nonnegative().optional(),
|
||||
limit: z.number().int().min(1).max(100).optional(),
|
||||
});
|
||||
export const ccrListOutput = z.object({
|
||||
entries: z.array(z.object({ reference: ccrReferenceOutput, metadata: ccrEntryMetadataOutput })),
|
||||
total: z.number().int().nonnegative(),
|
||||
offset: z.number().int().nonnegative(),
|
||||
limit: z.number().int().positive(),
|
||||
hasMore: z.boolean(),
|
||||
});
|
||||
export const ccrListTool: McpToolDefinition<typeof ccrListInput, typeof ccrListOutput> = {
|
||||
name: "omniroute_ccr_list",
|
||||
description: "List paginated metadata for CCR blocks owned by the current caller.",
|
||||
inputSchema: ccrListInput,
|
||||
outputSchema: ccrListOutput,
|
||||
scopes: ["read:compression"],
|
||||
auditLevel: "basic",
|
||||
phase: 2,
|
||||
sourceEndpoints: [],
|
||||
};
|
||||
|
||||
export const ccrDeleteInput = z.object({ hash: ccrHash });
|
||||
export const ccrDeleteOutput = z.object({ deleted: z.boolean() });
|
||||
export const ccrDeleteTool: McpToolDefinition<typeof ccrDeleteInput, typeof ccrDeleteOutput> = {
|
||||
name: "omniroute_ccr_delete",
|
||||
description: "Delete a caller-owned block from the in-memory CCR store.",
|
||||
inputSchema: ccrDeleteInput,
|
||||
outputSchema: ccrDeleteOutput,
|
||||
scopes: ["write:compression"],
|
||||
auditLevel: "basic",
|
||||
phase: 2,
|
||||
sourceEndpoints: [],
|
||||
};
|
||||
|
||||
export const ccrStatsInput = z.object({});
|
||||
export const ccrStatsOutput = z.object({
|
||||
storage: z.literal("memory"),
|
||||
entries: z.number().int().nonnegative(),
|
||||
bytes: z.number().int().nonnegative(),
|
||||
limits: z.object({
|
||||
maxEntries: z.number().int().positive(),
|
||||
maxBlockBytes: z.number().int().positive(),
|
||||
maxPrincipalBytes: z.number().int().positive(),
|
||||
maxGlobalBytes: z.number().int().positive(),
|
||||
defaultTtlSeconds: z.number().int().positive(),
|
||||
maxTtlSeconds: z.number().int().positive(),
|
||||
maxMcpFullBytes: z.number().int().positive(),
|
||||
}),
|
||||
lifecycle: z.object({
|
||||
expiredEvictions: z.number().int().nonnegative(),
|
||||
capacityEvictions: z.number().int().nonnegative(),
|
||||
rejectedStores: z.number().int().nonnegative(),
|
||||
}),
|
||||
});
|
||||
export const ccrStatsTool: McpToolDefinition<typeof ccrStatsInput, typeof ccrStatsOutput> = {
|
||||
name: "omniroute_ccr_stats",
|
||||
description:
|
||||
"Return caller-scoped CCR entry and byte usage, lifecycle counters, and in-memory store limits.",
|
||||
inputSchema: ccrStatsInput,
|
||||
outputSchema: ccrStatsOutput,
|
||||
scopes: ["read:compression"],
|
||||
auditLevel: "basic",
|
||||
phase: 2,
|
||||
sourceEndpoints: [],
|
||||
};
|
||||
|
||||
export const CCR_MCP_TOOLS = [
|
||||
ccrStoreTool,
|
||||
ccrRetrieveTool,
|
||||
ccrInspectTool,
|
||||
ccrListTool,
|
||||
ccrDeleteTool,
|
||||
ccrStatsTool,
|
||||
] as const;
|
||||
@@ -69,6 +69,26 @@ export {
|
||||
cacheFlushInput,
|
||||
cacheFlushOutput,
|
||||
cacheFlushTool,
|
||||
ccrEntryMetadataOutput,
|
||||
ccrReferenceOutput,
|
||||
ccrStoreInput,
|
||||
ccrStoreOutput,
|
||||
ccrStoreTool,
|
||||
ccrRetrieveInput,
|
||||
ccrRetrieveOutput,
|
||||
ccrRetrieveTool,
|
||||
ccrInspectInput,
|
||||
ccrInspectOutput,
|
||||
ccrInspectTool,
|
||||
ccrListInput,
|
||||
ccrListOutput,
|
||||
ccrListTool,
|
||||
ccrDeleteInput,
|
||||
ccrDeleteOutput,
|
||||
ccrDeleteTool,
|
||||
ccrStatsInput,
|
||||
ccrStatsOutput,
|
||||
ccrStatsTool,
|
||||
} from "./tools.ts";
|
||||
|
||||
// A2A schemas
|
||||
|
||||
@@ -12,6 +12,7 @@
|
||||
import { z } from "zod";
|
||||
import { toolSearchTool } from "./toolSearch.ts";
|
||||
import { pickFastestModelTool } from "./pickFastestModel.ts";
|
||||
import { CCR_MCP_TOOLS } from "./ccrTools.ts";
|
||||
import {
|
||||
AUTO_ROUTING_STRATEGY_VALUES,
|
||||
ROUTING_STRATEGY_VALUES,
|
||||
@@ -24,6 +25,7 @@ import {
|
||||
export type { AuditLevel, McpToolDefinition } from "./toolDefinition.ts";
|
||||
import type { McpToolDefinition } from "./toolDefinition.ts";
|
||||
export { pickFastestModelInput, pickFastestModelOutput } from "./pickFastestModel.ts";
|
||||
export * from "./ccrTools.ts";
|
||||
|
||||
// ============ Phase 1: Essential Tools (8) ============
|
||||
|
||||
@@ -1462,6 +1464,7 @@ export const MCP_TOOLS = [
|
||||
setCompressionEngineTool,
|
||||
listCompressionCombosTool,
|
||||
compressionComboStatsTool,
|
||||
...CCR_MCP_TOOLS,
|
||||
oneproxyFetchTool,
|
||||
oneproxyRotateTool,
|
||||
oneproxyStatsTool,
|
||||
|
||||
@@ -110,6 +110,7 @@ const TOTAL_MCP_TOOL_COUNT = countUniqueMcpTools({
|
||||
pluginTools,
|
||||
notionTools,
|
||||
obsidianTools,
|
||||
compressionTools,
|
||||
});
|
||||
|
||||
type JsonRecord = Record<string, unknown>;
|
||||
|
||||
@@ -76,9 +76,8 @@ export function getAllToolDefinitions(): ToolCatalogEntry[] {
|
||||
pluginTools,
|
||||
notionTools,
|
||||
obsidianTools,
|
||||
// compressionTools holds omniroute_ccr_retrieve, which is NOT in MCP_TOOLS — without it
|
||||
// a `tool_search("compression")` would miss that tool. The other 5 overlap MCP_TOOLS and
|
||||
// are resolved by the dedup-by-name below (first wins).
|
||||
// Keep the concrete handler collection in the catalog as a parity guard. Canonical CCR
|
||||
// definitions now live in MCP_TOOLS too; deduplication below keeps each name visible once.
|
||||
compressionTools,
|
||||
];
|
||||
|
||||
|
||||
@@ -4,6 +4,7 @@
|
||||
* Tools:
|
||||
* 1. omniroute_compression_status — Get compression config, analytics, and cache stats
|
||||
* 2. omniroute_compression_configure — Update compression settings
|
||||
* 3. CCR lifecycle tools — Store, retrieve, inspect, list, delete, and stats
|
||||
*/
|
||||
|
||||
import { logToolCall } from "../audit.ts";
|
||||
@@ -241,8 +242,23 @@ import {
|
||||
setCompressionEngineInput,
|
||||
listCompressionCombosInput,
|
||||
compressionComboStatsInput,
|
||||
ccrStoreInput,
|
||||
ccrRetrieveInput,
|
||||
ccrInspectInput,
|
||||
ccrListInput,
|
||||
ccrDeleteInput,
|
||||
ccrStatsInput,
|
||||
} from "../schemas/tools.ts";
|
||||
import { handleCcrRetrieve } from "../../services/compression/engines/ccr/index.ts";
|
||||
import {
|
||||
MAX_CCR_MCP_FULL_BYTES,
|
||||
buildCcrReference,
|
||||
deleteCcrBlock,
|
||||
getCcrStoreStats,
|
||||
handleCcrRetrieve,
|
||||
inspectCcrBlock,
|
||||
listCcrBlocks,
|
||||
tryStoreBlock,
|
||||
} from "../../services/compression/engines/ccr/index.ts";
|
||||
import {
|
||||
listRtkCommandSamples,
|
||||
discoverRepeatedNoise,
|
||||
@@ -252,22 +268,161 @@ import {
|
||||
import { resolveCallerScopeContext } from "../scopeEnforcement.ts";
|
||||
import { resolveMcpCallerApiKeyId } from "../mcpCallerIdentity.ts";
|
||||
|
||||
const ccrRetrieveInput = z.object({
|
||||
hash: z
|
||||
.string()
|
||||
.min(6)
|
||||
.max(64)
|
||||
.describe("24-hex content hash from a [CCR retrieve hash=<hash>] marker"),
|
||||
mode: z
|
||||
.enum(["full", "head", "tail", "lines", "grep", "stats"])
|
||||
.optional()
|
||||
.describe("Retrieval mode: full (default) | head | tail | lines | grep | stats"),
|
||||
n: z.number().int().positive().max(10000).optional().describe("head/tail: number of lines"),
|
||||
start: z.number().int().positive().optional().describe("lines: 1-indexed inclusive start"),
|
||||
end: z.number().int().positive().optional().describe("lines: 1-indexed inclusive end"),
|
||||
pattern: z.string().max(512).optional().describe("grep: regex (validated safe; ReDoS-rejected)"),
|
||||
unique: z.boolean().optional().describe("grep: dedupe matching lines"),
|
||||
});
|
||||
async function resolveCcrPrincipal(
|
||||
extra: McpToolExtraLike | undefined,
|
||||
scopes: readonly string[]
|
||||
): Promise<string | undefined> {
|
||||
const apiKeyPrincipal = await resolveMcpCallerApiKeyId();
|
||||
if (apiKeyPrincipal) return apiKeyPrincipal;
|
||||
const { callerId } = resolveCallerScopeContext(extra, scopes);
|
||||
return callerId === "anonymous" ? undefined : callerId;
|
||||
}
|
||||
|
||||
export function buildCcrStoreAuditInput(args: z.infer<typeof ccrStoreInput>) {
|
||||
return {
|
||||
bytes: Buffer.byteLength(args.content, "utf8"),
|
||||
contentType: args.contentType,
|
||||
ttlSeconds: args.ttlSeconds,
|
||||
};
|
||||
}
|
||||
|
||||
export async function handleCcrStoreTool(
|
||||
args: z.infer<typeof ccrStoreInput>,
|
||||
extra?: McpToolExtraLike
|
||||
) {
|
||||
const start = Date.now();
|
||||
const principal = await resolveCcrPrincipal(extra, ["write:compression"]);
|
||||
const result = tryStoreBlock(args.content, principal, {
|
||||
contentType: args.contentType,
|
||||
source: "mcp",
|
||||
ttlSeconds: args.ttlSeconds,
|
||||
});
|
||||
const auditInput = buildCcrStoreAuditInput(args);
|
||||
if (!result.stored) {
|
||||
const output = { stored: false as const, reason: result.reason };
|
||||
await logToolCall(
|
||||
"omniroute_ccr_store",
|
||||
auditInput,
|
||||
output,
|
||||
Date.now() - start,
|
||||
false,
|
||||
result.reason
|
||||
);
|
||||
return output;
|
||||
}
|
||||
const output = {
|
||||
stored: true as const,
|
||||
reference: buildCcrReference(result.hash, result.metadata.chars),
|
||||
metadata: result.metadata,
|
||||
};
|
||||
await logToolCall("omniroute_ccr_store", auditInput, output, Date.now() - start, true);
|
||||
return output;
|
||||
}
|
||||
|
||||
export async function handleCcrRetrieveTool(
|
||||
args: z.infer<typeof ccrRetrieveInput>,
|
||||
extra?: McpToolExtraLike
|
||||
) {
|
||||
const start = Date.now();
|
||||
const principal = await resolveCcrPrincipal(extra, ["read:compression"]);
|
||||
const metadata = inspectCcrBlock(args.hash, principal);
|
||||
if (!metadata) {
|
||||
const output = { found: false as const, error: "CCR block not found or expired" };
|
||||
await logToolCall(
|
||||
"omniroute_ccr_retrieve",
|
||||
args,
|
||||
output,
|
||||
Date.now() - start,
|
||||
false,
|
||||
"NOT_FOUND"
|
||||
);
|
||||
return output;
|
||||
}
|
||||
if ((!args.mode || args.mode === "full") && metadata.bytes > MAX_CCR_MCP_FULL_BYTES) {
|
||||
const output = {
|
||||
found: true as const,
|
||||
tooLargeForFull: true as const,
|
||||
metadata,
|
||||
suggestedModes: ["head", "tail", "lines", "grep", "stats"] as const,
|
||||
};
|
||||
await logToolCall("omniroute_ccr_retrieve", args, output, Date.now() - start, true);
|
||||
return output;
|
||||
}
|
||||
const queried = handleCcrRetrieve(args, principal);
|
||||
const refreshedMetadata = inspectCcrBlock(args.hash, principal) ?? metadata;
|
||||
const output =
|
||||
"content" in queried
|
||||
? { found: true as const, metadata: refreshedMetadata, content: queried.content }
|
||||
: { found: true as const, metadata: refreshedMetadata, error: queried.error };
|
||||
await logToolCall(
|
||||
"omniroute_ccr_retrieve",
|
||||
args,
|
||||
{
|
||||
...output,
|
||||
...(typeof output.content === "string"
|
||||
? { content: `[${Buffer.byteLength(output.content, "utf8")} bytes]` }
|
||||
: {}),
|
||||
},
|
||||
Date.now() - start,
|
||||
!("error" in output),
|
||||
"error" in output ? "INVALID_QUERY" : undefined
|
||||
);
|
||||
return output;
|
||||
}
|
||||
|
||||
export async function handleCcrInspectTool(
|
||||
args: z.infer<typeof ccrInspectInput>,
|
||||
extra?: McpToolExtraLike
|
||||
) {
|
||||
const start = Date.now();
|
||||
const principal = await resolveCcrPrincipal(extra, ["read:compression"]);
|
||||
const metadata = inspectCcrBlock(args.hash, principal);
|
||||
const output = metadata
|
||||
? { found: true as const, reference: buildCcrReference(args.hash, metadata.chars), metadata }
|
||||
: { found: false as const };
|
||||
await logToolCall("omniroute_ccr_inspect", args, output, Date.now() - start, Boolean(metadata));
|
||||
return output;
|
||||
}
|
||||
|
||||
export async function handleCcrListTool(
|
||||
args: z.infer<typeof ccrListInput>,
|
||||
extra?: McpToolExtraLike
|
||||
) {
|
||||
const start = Date.now();
|
||||
const principal = await resolveCcrPrincipal(extra, ["read:compression"]);
|
||||
const result = listCcrBlocks(principal, args);
|
||||
const output = {
|
||||
...result,
|
||||
entries: result.entries.map((metadata) => ({
|
||||
reference: buildCcrReference(metadata.hash, metadata.chars),
|
||||
metadata,
|
||||
})),
|
||||
};
|
||||
await logToolCall("omniroute_ccr_list", args, output, Date.now() - start, true);
|
||||
return output;
|
||||
}
|
||||
|
||||
export async function handleCcrDeleteTool(
|
||||
args: z.infer<typeof ccrDeleteInput>,
|
||||
extra?: McpToolExtraLike
|
||||
) {
|
||||
const start = Date.now();
|
||||
const principal = await resolveCcrPrincipal(extra, ["write:compression"]);
|
||||
const output = { deleted: deleteCcrBlock(args.hash, principal) };
|
||||
await logToolCall("omniroute_ccr_delete", args, output, Date.now() - start, true);
|
||||
return output;
|
||||
}
|
||||
|
||||
export async function handleCcrStatsTool(
|
||||
args: z.infer<typeof ccrStatsInput>,
|
||||
extra?: McpToolExtraLike
|
||||
) {
|
||||
const start = Date.now();
|
||||
const principal = await resolveCcrPrincipal(extra, ["read:compression"]);
|
||||
const output = getCcrStoreStats(principal);
|
||||
await logToolCall("omniroute_ccr_stats", args, output, Date.now() - start, true);
|
||||
return output;
|
||||
}
|
||||
|
||||
export async function handleSetCompressionEngine(
|
||||
args: z.infer<typeof setCompressionEngineInput>
|
||||
@@ -323,12 +478,24 @@ export async function handleCompressionComboStats(
|
||||
// T07 — RTK learn/discover exposed via MCP (read-only; suggestions only). Mines the opt-in
|
||||
// raw-output sample store, exactly like the /api/context/rtk/{discover,learn} routes.
|
||||
const rtkDiscoverInput = z.object({
|
||||
limit: z.number().int().positive().max(2000).optional().describe("Max samples to scan (default 500)"),
|
||||
limit: z
|
||||
.number()
|
||||
.int()
|
||||
.positive()
|
||||
.max(2000)
|
||||
.optional()
|
||||
.describe("Max samples to scan (default 500)"),
|
||||
});
|
||||
|
||||
const rtkLearnInput = z.object({
|
||||
command: z.string().min(1).max(500).describe("The command to learn an RTK filter draft for"),
|
||||
limit: z.number().int().positive().max(2000).optional().describe("Max samples to scan (default 500)"),
|
||||
limit: z
|
||||
.number()
|
||||
.int()
|
||||
.positive()
|
||||
.max(2000)
|
||||
.optional()
|
||||
.describe("Max samples to scan (default 500)"),
|
||||
});
|
||||
|
||||
function resolveSampleLimit(limit?: number): number {
|
||||
@@ -401,6 +568,14 @@ export const compressionTools = {
|
||||
handler: (args: z.infer<typeof compressionComboStatsInput>) =>
|
||||
handleCompressionComboStats(args),
|
||||
},
|
||||
omniroute_ccr_store: {
|
||||
name: "omniroute_ccr_store",
|
||||
description:
|
||||
"Store verbatim content in the caller-isolated in-memory CCR store and return a ccr:// reference plus the compatible CCR marker. Entries expire automatically and are not persisted across restarts.",
|
||||
scopes: ["write:compression"],
|
||||
inputSchema: ccrStoreInput,
|
||||
handler: handleCcrStoreTool,
|
||||
},
|
||||
omniroute_ccr_retrieve: {
|
||||
name: "omniroute_ccr_retrieve",
|
||||
description:
|
||||
@@ -411,22 +586,36 @@ export const compressionTools = {
|
||||
"Scope: read:compression. Always available (sticky-on).",
|
||||
scopes: ["read:compression"],
|
||||
inputSchema: ccrRetrieveInput,
|
||||
handler: async (args: z.infer<typeof ccrRetrieveInput>, extra?: McpToolExtraLike) => {
|
||||
// Retrieve must use the SAME principal the CCR store used at compression time:
|
||||
// `String(apiKeyInfo.id)` (chatCore → getApiKeyMetadata(rawKey)). On MCP HTTP
|
||||
// transports the raw key lives in httpAuthContext (not in extra.authInfo, since
|
||||
// OmniRoute auth is API-key not OAuth-clientId) — resolve it to the same key id
|
||||
// so the block is found. Without this the caller resolved to "anonymous" and the
|
||||
// store-key never matched (#5649). Cross-tenant IDOR stays closed: a different
|
||||
// key → different id → miss; no key → undefined → anonymous bucket only.
|
||||
const apiKeyPrincipal = await resolveMcpCallerApiKeyId();
|
||||
if (apiKeyPrincipal) {
|
||||
return handleCcrRetrieve(args, apiKeyPrincipal);
|
||||
}
|
||||
// Fallback (unchanged): OAuth clientId / session scope context, then anonymous.
|
||||
const { callerId } = resolveCallerScopeContext(extra, ["read:compression"]);
|
||||
return handleCcrRetrieve(args, callerId === "anonymous" ? undefined : callerId);
|
||||
},
|
||||
handler: handleCcrRetrieveTool,
|
||||
},
|
||||
omniroute_ccr_inspect: {
|
||||
name: "omniroute_ccr_inspect",
|
||||
description: "Inspect metadata for a caller-owned CCR block without returning its content.",
|
||||
scopes: ["read:compression"],
|
||||
inputSchema: ccrInspectInput,
|
||||
handler: handleCcrInspectTool,
|
||||
},
|
||||
omniroute_ccr_list: {
|
||||
name: "omniroute_ccr_list",
|
||||
description: "List paginated metadata for CCR blocks owned by the current caller.",
|
||||
scopes: ["read:compression"],
|
||||
inputSchema: ccrListInput,
|
||||
handler: handleCcrListTool,
|
||||
},
|
||||
omniroute_ccr_delete: {
|
||||
name: "omniroute_ccr_delete",
|
||||
description: "Delete a caller-owned block from the in-memory CCR store.",
|
||||
scopes: ["write:compression"],
|
||||
inputSchema: ccrDeleteInput,
|
||||
handler: handleCcrDeleteTool,
|
||||
},
|
||||
omniroute_ccr_stats: {
|
||||
name: "omniroute_ccr_stats",
|
||||
description:
|
||||
"Return caller-scoped CCR entry and byte usage, lifecycle counters, and in-memory store limits.",
|
||||
scopes: ["read:compression"],
|
||||
inputSchema: ccrStatsInput,
|
||||
handler: handleCcrStatsTool,
|
||||
},
|
||||
omniroute_rtk_discover: {
|
||||
name: "omniroute_rtk_discover",
|
||||
|
||||
@@ -26,8 +26,8 @@
|
||||
* does not affect another's (cross-tenant state drift protection).
|
||||
*
|
||||
* Memory bound:
|
||||
* - Both `ccrStore` and `retrievalCounts` are capped at MAX_CCR_ENTRIES
|
||||
* entries using FIFO eviction (Map insertion-order guarantees).
|
||||
* - Entries are capped by count, global bytes, per-principal bytes, block bytes and TTL.
|
||||
* - Expired and least-recently-used entries are removed before a store is rejected.
|
||||
*
|
||||
* Conservative guards:
|
||||
* - Never touch `role: "system"`.
|
||||
@@ -60,12 +60,69 @@ const RETRIEVAL_THRESHOLD = 3;
|
||||
* ramp (only the >= threshold cliff remains — the legacy binary behavior).
|
||||
*/
|
||||
const RETRIEVAL_RAMP_FACTOR_DEFAULT = 2;
|
||||
/**
|
||||
* Maximum number of entries in each bounded store.
|
||||
* When inserting beyond this cap, the oldest entry (Map insertion order) is evicted.
|
||||
* 5 000 entries × ~2 KB average ≈ 10 MB upper bound for each map.
|
||||
*/
|
||||
/** Maximum number of entries in the principal-scoped, LRU-ordered store. */
|
||||
export const MAX_CCR_ENTRIES = 5_000;
|
||||
export const MAX_CCR_BLOCK_BYTES = 2 * 1024 * 1024;
|
||||
export const MAX_CCR_PRINCIPAL_BYTES = 16 * 1024 * 1024;
|
||||
export const MAX_CCR_GLOBAL_BYTES = 64 * 1024 * 1024;
|
||||
export const DEFAULT_CCR_TTL_SECONDS = 24 * 60 * 60;
|
||||
export const MAX_CCR_TTL_SECONDS = 7 * 24 * 60 * 60;
|
||||
export const MAX_CCR_MCP_FULL_BYTES = 256 * 1024;
|
||||
|
||||
export type CcrEntrySource = "compression" | "mcp" | "ionizer" | "session-dedup";
|
||||
|
||||
export interface CcrEntryMetadata {
|
||||
hash: string;
|
||||
bytes: number;
|
||||
chars: number;
|
||||
lines: number;
|
||||
contentType: string;
|
||||
source: CcrEntrySource;
|
||||
createdAt: number;
|
||||
lastAccessedAt: number;
|
||||
expiresAt: number;
|
||||
retrievalCount: number;
|
||||
}
|
||||
|
||||
type CcrEntry = Omit<CcrEntryMetadata, "retrievalCount"> & {
|
||||
principalId: string;
|
||||
content: string;
|
||||
};
|
||||
|
||||
export interface StoreCcrBlockOptions {
|
||||
contentType?: string;
|
||||
source?: CcrEntrySource;
|
||||
ttlSeconds?: number;
|
||||
now?: number;
|
||||
}
|
||||
|
||||
export type StoreCcrBlockResult =
|
||||
| { stored: true; hash: string; metadata: CcrEntryMetadata }
|
||||
| {
|
||||
stored: false;
|
||||
hash: string;
|
||||
reason: "block_too_large" | "principal_budget_exceeded" | "global_budget_exceeded";
|
||||
};
|
||||
|
||||
export interface CcrStoreStats {
|
||||
storage: "memory";
|
||||
entries: number;
|
||||
bytes: number;
|
||||
limits: {
|
||||
maxEntries: number;
|
||||
maxBlockBytes: number;
|
||||
maxPrincipalBytes: number;
|
||||
maxGlobalBytes: number;
|
||||
defaultTtlSeconds: number;
|
||||
maxTtlSeconds: number;
|
||||
maxMcpFullBytes: number;
|
||||
};
|
||||
lifecycle: {
|
||||
expiredEvictions: number;
|
||||
capacityEvictions: number;
|
||||
rejectedStores: number;
|
||||
};
|
||||
}
|
||||
|
||||
// ─── principal-scoped, bounded content store ──────────────────────────────────
|
||||
|
||||
@@ -73,9 +130,12 @@ export const MAX_CCR_ENTRIES = 5_000;
|
||||
* Store key = `${principalId ?? "__anon__"} ${contentHash}`.
|
||||
* Using a compound key scopes data to the principal that stored it.
|
||||
*/
|
||||
const ccrStore = new Map<string, string>();
|
||||
/** Retrieval counter store — same scoping as ccrStore. */
|
||||
const ccrStore = new Map<string, CcrEntry>();
|
||||
const retrievalCounts = new Map<string, number>();
|
||||
const principalBytesMap = new Map<string, number>();
|
||||
let ccrTotalBytes = 0;
|
||||
type CcrLifecycleCounters = CcrStoreStats["lifecycle"];
|
||||
const lifecycleByPrincipal = new Map<string, CcrLifecycleCounters>();
|
||||
|
||||
/** Sentinel used when no principalId is provided. */
|
||||
const ANON = "__anon__";
|
||||
@@ -84,18 +144,88 @@ function buildStoreKey(hash: string, principalId?: string): string {
|
||||
return `${principalId ?? ANON} ${hash}`;
|
||||
}
|
||||
|
||||
/**
|
||||
* Insert a value into a bounded Map, evicting the oldest entry when over the cap.
|
||||
*/
|
||||
function boundedSet<V>(map: Map<string, V>, key: string, value: V): void {
|
||||
if (!map.has(key) && map.size >= MAX_CCR_ENTRIES) {
|
||||
// Map preserves insertion order — the first iterator result is the oldest entry.
|
||||
const firstKey = map.keys().next().value;
|
||||
if (firstKey !== undefined) {
|
||||
map.delete(firstKey);
|
||||
function readLifecycleCounters(principalId: string): CcrLifecycleCounters {
|
||||
return (
|
||||
lifecycleByPrincipal.get(principalId) ?? {
|
||||
expiredEvictions: 0,
|
||||
capacityEvictions: 0,
|
||||
rejectedStores: 0,
|
||||
}
|
||||
);
|
||||
}
|
||||
|
||||
function mutableLifecycleCounters(principalId: string): CcrLifecycleCounters {
|
||||
const existing = lifecycleByPrincipal.get(principalId);
|
||||
if (existing) return existing;
|
||||
const counters = { expiredEvictions: 0, capacityEvictions: 0, rejectedStores: 0 };
|
||||
lifecycleByPrincipal.set(principalId, counters);
|
||||
return counters;
|
||||
}
|
||||
|
||||
function publicMetadata(entry: CcrEntry): CcrEntryMetadata {
|
||||
const { principalId: _principalId, content: _content, ...metadata } = entry;
|
||||
return {
|
||||
...metadata,
|
||||
retrievalCount: retrievalCounts.get(buildStoreKey(entry.hash, entry.principalId)) ?? 0,
|
||||
};
|
||||
}
|
||||
|
||||
function setRetrievalCount(key: string, count: number): void {
|
||||
if (!retrievalCounts.has(key) && retrievalCounts.size >= MAX_CCR_ENTRIES) {
|
||||
const oldestKey = retrievalCounts.keys().next().value;
|
||||
if (oldestKey !== undefined) retrievalCounts.delete(oldestKey);
|
||||
}
|
||||
map.set(key, value);
|
||||
retrievalCounts.delete(key);
|
||||
retrievalCounts.set(key, count);
|
||||
}
|
||||
|
||||
function removeEntry(key: string, reason?: "expired" | "capacity"): boolean {
|
||||
const entry = ccrStore.get(key);
|
||||
if (!entry) return false;
|
||||
ccrStore.delete(key);
|
||||
ccrTotalBytes = Math.max(0, ccrTotalBytes - entry.bytes);
|
||||
const remainingPrincipalBytes = Math.max(
|
||||
0,
|
||||
(principalBytesMap.get(entry.principalId) ?? 0) - entry.bytes
|
||||
);
|
||||
if (remainingPrincipalBytes === 0) principalBytesMap.delete(entry.principalId);
|
||||
else principalBytesMap.set(entry.principalId, remainingPrincipalBytes);
|
||||
const counters = mutableLifecycleCounters(entry.principalId);
|
||||
if (reason === "expired") counters.expiredEvictions++;
|
||||
if (reason === "capacity") counters.capacityEvictions++;
|
||||
return true;
|
||||
}
|
||||
|
||||
function purgeExpired(now = Date.now()): void {
|
||||
for (const [key, entry] of ccrStore) {
|
||||
if (entry.expiresAt <= now) removeEntry(key, "expired");
|
||||
}
|
||||
}
|
||||
|
||||
function getActiveEntry(key: string, now = Date.now()): CcrEntry | null {
|
||||
const entry = ccrStore.get(key);
|
||||
if (!entry) return null;
|
||||
if (entry.expiresAt <= now) {
|
||||
removeEntry(key, "expired");
|
||||
return null;
|
||||
}
|
||||
return entry;
|
||||
}
|
||||
|
||||
function principalBytes(principalId: string): number {
|
||||
return principalBytesMap.get(principalId) ?? 0;
|
||||
}
|
||||
|
||||
function evictOldestMatching(predicate: (entry: CcrEntry) => boolean): boolean {
|
||||
for (const [key, entry] of ccrStore) {
|
||||
if (predicate(entry)) return removeEntry(key, "capacity");
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
function normalizeTtlSeconds(value: number | undefined): number {
|
||||
if (!Number.isFinite(value) || value === undefined) return DEFAULT_CCR_TTL_SECONDS;
|
||||
return Math.max(60, Math.min(MAX_CCR_TTL_SECONDS, Math.floor(value)));
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -107,26 +237,114 @@ function hashContent(text: string): string {
|
||||
return crypto.createHash("sha256").update(text).digest("hex").slice(0, 24);
|
||||
}
|
||||
|
||||
function rejectStore(
|
||||
hash: string,
|
||||
owner: string,
|
||||
reason: Exclude<StoreCcrBlockResult, { stored: true }>["reason"]
|
||||
): StoreCcrBlockResult {
|
||||
mutableLifecycleCounters(owner).rejectedStores++;
|
||||
return { stored: false, hash, reason };
|
||||
}
|
||||
|
||||
function enforcePrincipalBudget(owner: string, bytes: number): boolean {
|
||||
while (
|
||||
principalBytes(owner) + bytes > MAX_CCR_PRINCIPAL_BYTES &&
|
||||
evictOldestMatching((entry) => entry.principalId === owner)
|
||||
) {
|
||||
// Evict the owner's least-recently-used entries until this block fits.
|
||||
}
|
||||
return principalBytes(owner) + bytes <= MAX_CCR_PRINCIPAL_BYTES;
|
||||
}
|
||||
|
||||
function enforceGlobalBudget(bytes: number): boolean {
|
||||
while (
|
||||
(ccrStore.size >= MAX_CCR_ENTRIES || ccrTotalBytes + bytes > MAX_CCR_GLOBAL_BYTES) &&
|
||||
evictOldestMatching(() => true)
|
||||
) {
|
||||
// Enforce both entry and global byte caps with LRU eviction.
|
||||
}
|
||||
return ccrTotalBytes + bytes <= MAX_CCR_GLOBAL_BYTES;
|
||||
}
|
||||
|
||||
/**
|
||||
* Store a block in the CCR store under the given principal.
|
||||
* Returns the 24-hex content hash (for embedding in the marker).
|
||||
*/
|
||||
export function storeBlock(text: string, principalId?: string): string {
|
||||
export function tryStoreBlock(
|
||||
text: string,
|
||||
principalId?: string,
|
||||
options: StoreCcrBlockOptions = {}
|
||||
): StoreCcrBlockResult {
|
||||
const hash = hashContent(text);
|
||||
const owner = principalId ?? ANON;
|
||||
const key = buildStoreKey(hash, principalId);
|
||||
if (!ccrStore.has(key)) {
|
||||
boundedSet(ccrStore, key, text);
|
||||
const now = options.now ?? Date.now();
|
||||
purgeExpired(now);
|
||||
|
||||
const existing = ccrStore.get(key);
|
||||
if (existing) {
|
||||
existing.lastAccessedAt = now;
|
||||
existing.expiresAt = now + normalizeTtlSeconds(options.ttlSeconds) * 1000;
|
||||
ccrStore.delete(key);
|
||||
ccrStore.set(key, existing);
|
||||
return { stored: true, hash, metadata: publicMetadata(existing) };
|
||||
}
|
||||
return hash;
|
||||
|
||||
const bytes = Buffer.byteLength(text, "utf8");
|
||||
if (bytes > MAX_CCR_BLOCK_BYTES) {
|
||||
return rejectStore(hash, owner, "block_too_large");
|
||||
}
|
||||
|
||||
if (!enforcePrincipalBudget(owner, bytes)) {
|
||||
return rejectStore(hash, owner, "principal_budget_exceeded");
|
||||
}
|
||||
|
||||
if (!enforceGlobalBudget(bytes)) {
|
||||
return rejectStore(hash, owner, "global_budget_exceeded");
|
||||
}
|
||||
|
||||
const ttlSeconds = normalizeTtlSeconds(options.ttlSeconds);
|
||||
const entry: CcrEntry = {
|
||||
hash,
|
||||
principalId: owner,
|
||||
content: text,
|
||||
bytes,
|
||||
chars: text.length,
|
||||
lines: text.length === 0 ? 0 : text.split("\n").length,
|
||||
contentType: options.contentType?.trim().slice(0, 128) || "text/plain",
|
||||
source: options.source ?? "compression",
|
||||
createdAt: now,
|
||||
lastAccessedAt: now,
|
||||
expiresAt: now + ttlSeconds * 1000,
|
||||
};
|
||||
ccrStore.set(key, entry);
|
||||
ccrTotalBytes += bytes;
|
||||
principalBytesMap.set(owner, principalBytes(owner) + bytes);
|
||||
return { stored: true, hash, metadata: publicMetadata(entry) };
|
||||
}
|
||||
|
||||
export function storeBlock(
|
||||
text: string,
|
||||
principalId?: string,
|
||||
options: StoreCcrBlockOptions = {}
|
||||
): string {
|
||||
const result = tryStoreBlock(text, principalId, options);
|
||||
if (!result.stored) throw new RangeError(`CCR store rejected block: ${result.reason}`);
|
||||
return result.hash;
|
||||
}
|
||||
|
||||
/**
|
||||
* Retrieve the verbatim block for a given hash and principal.
|
||||
* Returns null if not found or if the principal does not match the stored key.
|
||||
*/
|
||||
export function retrieveBlock(hash: string, principalId?: string): string | null {
|
||||
export function retrieveBlock(hash: string, principalId?: string, now = Date.now()): string | null {
|
||||
const key = buildStoreKey(hash, principalId);
|
||||
return ccrStore.get(key) ?? null;
|
||||
const entry = getActiveEntry(key, now);
|
||||
if (!entry) return null;
|
||||
entry.lastAccessedAt = now;
|
||||
ccrStore.delete(key);
|
||||
ccrStore.set(key, entry);
|
||||
return entry.content;
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -134,7 +352,7 @@ export function retrieveBlock(hash: string, principalId?: string): string | null
|
||||
*/
|
||||
export function recordRetrieval(hash: string, principalId?: string): void {
|
||||
const key = buildStoreKey(hash, principalId);
|
||||
boundedSet(retrievalCounts, key, (retrievalCounts.get(key) ?? 0) + 1);
|
||||
setRetrievalCount(key, (retrievalCounts.get(key) ?? 0) + 1);
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -182,6 +400,65 @@ export function resolveRetrievalRampFactor(env: NodeJS.ProcessEnv = process.env)
|
||||
export function resetCcrStore(): void {
|
||||
ccrStore.clear();
|
||||
retrievalCounts.clear();
|
||||
principalBytesMap.clear();
|
||||
ccrTotalBytes = 0;
|
||||
lifecycleByPrincipal.clear();
|
||||
}
|
||||
|
||||
export function inspectCcrBlock(
|
||||
hash: string,
|
||||
principalId?: string,
|
||||
now = Date.now()
|
||||
): CcrEntryMetadata | null {
|
||||
const entry = getActiveEntry(buildStoreKey(hash, principalId), now);
|
||||
return entry ? publicMetadata(entry) : null;
|
||||
}
|
||||
|
||||
export function listCcrBlocks(
|
||||
principalId?: string,
|
||||
options: { offset?: number; limit?: number; now?: number } = {}
|
||||
): { entries: CcrEntryMetadata[]; total: number; offset: number; limit: number; hasMore: boolean } {
|
||||
purgeExpired(options.now);
|
||||
const owner = principalId ?? ANON;
|
||||
const offset = Math.max(0, Math.floor(options.offset ?? 0));
|
||||
const limit = Math.max(1, Math.min(100, Math.floor(options.limit ?? 25)));
|
||||
const all: CcrEntryMetadata[] = [];
|
||||
for (const entry of ccrStore.values()) {
|
||||
if (entry.principalId === owner) all.push(publicMetadata(entry));
|
||||
}
|
||||
all.reverse();
|
||||
return {
|
||||
entries: all.slice(offset, offset + limit),
|
||||
total: all.length,
|
||||
offset,
|
||||
limit,
|
||||
hasMore: offset + limit < all.length,
|
||||
};
|
||||
}
|
||||
|
||||
export function deleteCcrBlock(hash: string, principalId?: string, _now = Date.now()): boolean {
|
||||
return removeEntry(buildStoreKey(hash, principalId));
|
||||
}
|
||||
|
||||
export function getCcrStoreStats(principalId?: string, now = Date.now()): CcrStoreStats {
|
||||
purgeExpired(now);
|
||||
const owner = principalId ?? ANON;
|
||||
const entries = Array.from(ccrStore.values()).filter((entry) => entry.principalId === owner);
|
||||
return {
|
||||
storage: "memory",
|
||||
entries: entries.length,
|
||||
bytes: entries.reduce((sum, entry) => sum + entry.bytes, 0),
|
||||
limits: {
|
||||
maxEntries: MAX_CCR_ENTRIES,
|
||||
maxBlockBytes: MAX_CCR_BLOCK_BYTES,
|
||||
maxPrincipalBytes: MAX_CCR_PRINCIPAL_BYTES,
|
||||
maxGlobalBytes: MAX_CCR_GLOBAL_BYTES,
|
||||
defaultTtlSeconds: DEFAULT_CCR_TTL_SECONDS,
|
||||
maxTtlSeconds: MAX_CCR_TTL_SECONDS,
|
||||
maxMcpFullBytes: MAX_CCR_MCP_FULL_BYTES,
|
||||
},
|
||||
lifecycle: { ...readLifecycleCounters(owner) },
|
||||
};
|
||||
}
|
||||
|
||||
// ─── MCP tool handler (pure function) ────────────────────────────────────────
|
||||
@@ -226,10 +503,21 @@ type MessageLike = {
|
||||
/**
|
||||
* Build a CCR marker string for a block.
|
||||
*/
|
||||
function buildMarker(hash: string, charCount: number): string {
|
||||
export function buildCcrMarker(hash: string, charCount: number): string {
|
||||
return `[CCR retrieve hash=${hash} chars=${charCount}]`;
|
||||
}
|
||||
|
||||
export function buildCcrReference(
|
||||
hash: string,
|
||||
charCount: number
|
||||
): {
|
||||
hash: string;
|
||||
uri: string;
|
||||
marker: string;
|
||||
} {
|
||||
return { hash, uri: `ccr://${hash}`, marker: buildCcrMarker(hash, charCount) };
|
||||
}
|
||||
|
||||
/**
|
||||
* Replace a large text block with a CCR marker if it shrinks the content.
|
||||
* Returns the new text and a flag indicating whether replacement happened.
|
||||
@@ -254,14 +542,15 @@ function maybeCcrReplace(
|
||||
return { text, replaced: false, hash: null };
|
||||
}
|
||||
|
||||
const marker = buildMarker(hash, text.length);
|
||||
const marker = buildCcrMarker(hash, text.length);
|
||||
|
||||
// Only replace if it actually shrinks
|
||||
if (marker.length >= text.length) {
|
||||
return { text, replaced: false, hash: null };
|
||||
}
|
||||
|
||||
storeBlock(text, principalId);
|
||||
const stored = tryStoreBlock(text, principalId, { source: "compression" });
|
||||
if (!stored.stored) return { text, replaced: false, hash: null };
|
||||
return { text: marker, replaced: true, hash };
|
||||
}
|
||||
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
// open-sse/services/compression/engines/ionizer/sample.ts
|
||||
import { storeBlock } from "../ccr/index.ts";
|
||||
import { tryStoreBlock } from "../ccr/index.ts";
|
||||
|
||||
type MessageLike = { role?: string; content?: unknown; [key: string]: unknown };
|
||||
|
||||
@@ -131,8 +131,7 @@ export interface IonizerPassResult {
|
||||
|
||||
function isPlainObjectArray(v: unknown): v is Array<Record<string, unknown>> {
|
||||
return (
|
||||
Array.isArray(v) &&
|
||||
v.every((el) => el !== null && typeof el === "object" && !Array.isArray(el))
|
||||
Array.isArray(v) && v.every((el) => el !== null && typeof el === "object" && !Array.isArray(el))
|
||||
);
|
||||
}
|
||||
|
||||
@@ -169,8 +168,12 @@ export function applyIonizerPass(
|
||||
});
|
||||
if (res.keptCount >= res.totalCount) return m;
|
||||
|
||||
const hash = storeBlock(serialized, opts.principalId);
|
||||
const marker = `[ionizer: kept ${res.keptCount}/${res.totalCount} rows; full → CCR retrieve hash=${hash} chars=${serialized.length}]`;
|
||||
const stored = tryStoreBlock(serialized, opts.principalId, {
|
||||
contentType: "application/json",
|
||||
source: "ionizer",
|
||||
});
|
||||
if (!stored.stored) return m;
|
||||
const marker = `[ionizer: kept ${res.keptCount}/${res.totalCount} rows; full → CCR retrieve hash=${stored.hash} chars=${serialized.length}]`;
|
||||
const newContent = `${JSON.stringify(res.kept)}\n${marker}`;
|
||||
if (newContent.length >= serialized.length) return m;
|
||||
|
||||
@@ -194,7 +197,9 @@ export function runIonizerPass(
|
||||
principalId?: string
|
||||
): IonizerPassResult {
|
||||
if (stepConfig["enabled"] === false) return { messages, ionizedCount: 0 };
|
||||
const threshold = typeof stepConfig["threshold"] === "number" ? (stepConfig["threshold"] as number) : 200;
|
||||
const targetRows = typeof stepConfig["targetRows"] === "number" ? (stepConfig["targetRows"] as number) : 50;
|
||||
const threshold =
|
||||
typeof stepConfig["threshold"] === "number" ? (stepConfig["threshold"] as number) : 200;
|
||||
const targetRows =
|
||||
typeof stepConfig["targetRows"] === "number" ? (stepConfig["targetRows"] as number) : 50;
|
||||
return applyIonizerPass(messages, { threshold, targetRows, principalId });
|
||||
}
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
// open-sse/services/compression/engines/session-dedup/fuzzy.ts
|
||||
import { storeBlock } from "../ccr/index.ts";
|
||||
import { buildCcrMarker, tryStoreBlock } from "../ccr/index.ts";
|
||||
|
||||
type MessageLike = { role?: string; content?: unknown; [key: string]: unknown };
|
||||
|
||||
@@ -111,8 +111,9 @@ export function applyFuzzyPass(messages: MessageLike[], opts: FuzzyPassOptions):
|
||||
|
||||
const replacements = new Map<number, string>();
|
||||
for (const nd of nearDups) {
|
||||
const hash = storeBlock(nd.block.text, opts.principalId);
|
||||
const marker = `[CCR retrieve hash=${hash} chars=${nd.block.text.length}]`;
|
||||
const stored = tryStoreBlock(nd.block.text, opts.principalId, { source: "session-dedup" });
|
||||
if (!stored.stored) continue;
|
||||
const marker = buildCcrMarker(stored.hash, nd.block.text.length);
|
||||
if (marker.length < nd.block.text.length) replacements.set(nd.block.index, marker);
|
||||
}
|
||||
if (replacements.size === 0) return { messages, fuzzyCount: 0 };
|
||||
@@ -138,9 +139,7 @@ export function runFuzzyPass(
|
||||
principalId?: string
|
||||
): FuzzyPassResult {
|
||||
const raw = stepConfig["fuzzy"] as
|
||||
| boolean
|
||||
| { enabled?: boolean; minJaccard?: number; shingleSize?: number }
|
||||
| undefined;
|
||||
boolean | { enabled?: boolean; minJaccard?: number; shingleSize?: number } | undefined;
|
||||
const cfg = typeof raw === "boolean" ? { enabled: raw } : raw;
|
||||
if (!cfg?.enabled) return { messages, fuzzyCount: 0 };
|
||||
return applyFuzzyPass(messages, {
|
||||
|
||||
@@ -63,6 +63,12 @@ export const MCP_TOOL_SCOPES: Record<string, readonly McpScope[]> = {
|
||||
omniroute_set_compression_engine: ["write:compression"],
|
||||
omniroute_list_compression_combos: ["read:compression"],
|
||||
omniroute_compression_combo_stats: ["read:compression"],
|
||||
omniroute_ccr_store: ["write:compression"],
|
||||
omniroute_ccr_retrieve: ["read:compression"],
|
||||
omniroute_ccr_inspect: ["read:compression"],
|
||||
omniroute_ccr_list: ["read:compression"],
|
||||
omniroute_ccr_delete: ["write:compression"],
|
||||
omniroute_ccr_stats: ["read:compression"],
|
||||
omniroute_oneproxy_fetch: ["read:proxies"],
|
||||
omniroute_oneproxy_rotate: ["read:proxies"],
|
||||
omniroute_oneproxy_stats: ["read:proxies"],
|
||||
|
||||
@@ -184,9 +184,9 @@ describe("ccr security: [MEDIUM] scoped retrieval feedback (shouldSkipCompressio
|
||||
});
|
||||
});
|
||||
|
||||
// ─── bounded memory (FIFO eviction) ──────────────────────────────────────────
|
||||
// ─── bounded memory (LRU eviction) ───────────────────────────────────────────
|
||||
|
||||
describe("ccr security: [MEDIUM] bounded memory (FIFO eviction)", () => {
|
||||
describe("ccr security: [MEDIUM] bounded memory (LRU eviction)", () => {
|
||||
beforeEach(() => resetCcrStore());
|
||||
|
||||
it("MAX_CCR_ENTRIES is exported and positive", () => {
|
||||
@@ -194,7 +194,7 @@ describe("ccr security: [MEDIUM] bounded memory (FIFO eviction)", () => {
|
||||
assert.ok(MAX_CCR_ENTRIES > 0, "MAX_CCR_ENTRIES must be positive");
|
||||
});
|
||||
|
||||
it("FIFO eviction: oldest entry is evicted when cap is reached", () => {
|
||||
it("LRU eviction: least-recently-used entry is evicted when cap is reached", () => {
|
||||
const principal = "evictionTest";
|
||||
|
||||
// Fill to exactly cap with unique blocks
|
||||
@@ -205,7 +205,10 @@ describe("ccr security: [MEDIUM] bounded memory (FIFO eviction)", () => {
|
||||
storeBlock(`eviction block ${i} ${"y".repeat(30)}`, principal);
|
||||
}
|
||||
|
||||
// Confirm first block is present before overflow
|
||||
const secondText = `eviction block 1 ${"y".repeat(30)}`;
|
||||
const secondHash = contentHash(secondText);
|
||||
|
||||
// Touch the first block so the second one becomes least recently used.
|
||||
assert.equal(
|
||||
retrieveBlock(firstHash, principal),
|
||||
firstText,
|
||||
@@ -215,12 +218,13 @@ describe("ccr security: [MEDIUM] bounded memory (FIFO eviction)", () => {
|
||||
// Insert one more unique block — should evict the first (oldest) entry
|
||||
storeBlock(`eviction overflow block ${"z".repeat(50)}`, principal);
|
||||
|
||||
// First block should now be gone
|
||||
// The untouched second block should now be gone; the recently used first remains.
|
||||
assert.equal(
|
||||
retrieveBlock(firstHash, principal),
|
||||
retrieveBlock(secondHash, principal),
|
||||
null,
|
||||
"[MEDIUM] FIFO eviction: oldest block must be evicted when cap is reached"
|
||||
"[MEDIUM] LRU eviction: least-recently-used block must be evicted when cap is reached"
|
||||
);
|
||||
assert.equal(retrieveBlock(firstHash, principal), firstText);
|
||||
});
|
||||
|
||||
it("store does not grow unboundedly beyond MAX_CCR_ENTRIES distinct principals", () => {
|
||||
|
||||
220
tests/unit/compression/ccr-mcp-integration.test.ts
Normal file
220
tests/unit/compression/ccr-mcp-integration.test.ts
Normal file
@@ -0,0 +1,220 @@
|
||||
import { describe, it, beforeEach } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { mkdtempSync } from "node:fs";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
|
||||
process.env.DATA_DIR = mkdtempSync(join(tmpdir(), "omniroute-ccr-mcp-"));
|
||||
|
||||
const ccr = await import("../../../open-sse/services/compression/engines/ccr/index.ts");
|
||||
const tools = await import("../../../open-sse/mcp-server/tools/compressionTools.ts");
|
||||
const schemas = await import("../../../open-sse/mcp-server/schemas/tools.ts");
|
||||
const { MCP_TOOL_SCOPES } = await import("../../../src/shared/constants/mcpScopes.ts");
|
||||
|
||||
const readExtra = {
|
||||
authInfo: { clientId: "tenant-a", scopes: ["read:compression"] },
|
||||
};
|
||||
const writeExtra = {
|
||||
authInfo: { clientId: "tenant-a", scopes: ["write:compression"] },
|
||||
};
|
||||
|
||||
describe("CCR in-memory store contract", () => {
|
||||
beforeEach(() => ccr.resetCcrStore());
|
||||
|
||||
it("stores metadata, references, pagination, deletion, and caller-scoped stats", () => {
|
||||
const first = ccr.tryStoreBlock("first\nblock", "tenant-a", {
|
||||
contentType: "text/markdown",
|
||||
source: "mcp",
|
||||
ttlSeconds: 120,
|
||||
now: 1_000,
|
||||
});
|
||||
const second = ccr.tryStoreBlock("second block", "tenant-a", {
|
||||
source: "mcp",
|
||||
now: 2_000,
|
||||
});
|
||||
assert.equal(first.stored, true);
|
||||
assert.equal(second.stored, true);
|
||||
if (!first.stored || !second.stored) return;
|
||||
|
||||
assert.deepEqual(ccr.buildCcrReference(first.hash, first.metadata.chars), {
|
||||
hash: first.hash,
|
||||
uri: `ccr://${first.hash}`,
|
||||
marker: `[CCR retrieve hash=${first.hash} chars=11]`,
|
||||
});
|
||||
assert.equal(first.metadata.contentType, "text/markdown");
|
||||
assert.equal(first.metadata.lines, 2);
|
||||
assert.equal("content" in first.metadata, false);
|
||||
|
||||
const page = ccr.listCcrBlocks("tenant-a", { limit: 1, now: 2_000 });
|
||||
assert.equal(page.total, 2);
|
||||
assert.equal(page.entries[0].hash, second.hash);
|
||||
assert.equal(page.hasMore, true);
|
||||
assert.equal(ccr.getCcrStoreStats("tenant-a", 2_000).entries, 2);
|
||||
|
||||
assert.equal(ccr.deleteCcrBlock(first.hash, "tenant-b", 2_000), false);
|
||||
assert.equal(ccr.deleteCcrBlock(first.hash, "tenant-a", 2_000), true);
|
||||
assert.equal(ccr.inspectCcrBlock(first.hash, "tenant-a", 2_000), null);
|
||||
});
|
||||
|
||||
it("expires entries deterministically and counts expiration eviction", () => {
|
||||
const stored = ccr.tryStoreBlock("expires", "tenant-a", { ttlSeconds: 60, now: 1_000 });
|
||||
assert.equal(stored.stored, true);
|
||||
if (!stored.stored) return;
|
||||
assert.ok(ccr.inspectCcrBlock(stored.hash, "tenant-a", 60_999));
|
||||
assert.equal(ccr.inspectCcrBlock(stored.hash, "tenant-a", 61_000), null);
|
||||
assert.equal(ccr.getCcrStoreStats("tenant-a", 61_000).lifecycle.expiredEvictions, 1);
|
||||
});
|
||||
|
||||
it("expires only the requested entry on hot-path reads", () => {
|
||||
const expired = ccr.tryStoreBlock("expired", "tenant-a", { ttlSeconds: 60, now: 1_000 });
|
||||
const active = ccr.tryStoreBlock("active", "tenant-b", { ttlSeconds: 120, now: 1_000 });
|
||||
assert.equal(expired.stored, true);
|
||||
assert.equal(active.stored, true);
|
||||
if (!expired.stored || !active.stored) return;
|
||||
|
||||
assert.equal(ccr.retrieveBlock(active.hash, "tenant-b", 61_000), "active");
|
||||
assert.equal(ccr.getCcrStoreStats("tenant-a", 60_999).lifecycle.expiredEvictions, 0);
|
||||
assert.equal(ccr.inspectCcrBlock(expired.hash, "tenant-a", 61_000), null);
|
||||
assert.equal(ccr.getCcrStoreStats("tenant-a", 61_000).lifecycle.expiredEvictions, 1);
|
||||
});
|
||||
|
||||
it("rejects blocks above the UTF-8 byte limit", () => {
|
||||
const result = ccr.tryStoreBlock("ü".repeat(ccr.MAX_CCR_BLOCK_BYTES), "tenant-a");
|
||||
assert.deepEqual(result.stored, false);
|
||||
if (!result.stored) assert.equal(result.reason, "block_too_large");
|
||||
assert.equal(ccr.getCcrStoreStats("tenant-a").lifecycle.rejectedStores, 1);
|
||||
});
|
||||
|
||||
it("evicts least-recently-used owner entries to enforce the principal byte budget", () => {
|
||||
const hashes: string[] = [];
|
||||
for (let i = 0; i < 9; i++) {
|
||||
const prefix = `${i}:`;
|
||||
const content = prefix + "x".repeat(ccr.MAX_CCR_BLOCK_BYTES - prefix.length);
|
||||
const result = ccr.tryStoreBlock(content, "tenant-a", { now: i + 1 });
|
||||
assert.equal(result.stored, true);
|
||||
if (result.stored) hashes.push(result.hash);
|
||||
}
|
||||
const stats = ccr.getCcrStoreStats("tenant-a", 10);
|
||||
assert.ok(stats.bytes <= ccr.MAX_CCR_PRINCIPAL_BYTES);
|
||||
assert.equal(stats.entries, 8);
|
||||
assert.equal(ccr.inspectCcrBlock(hashes[0], "tenant-a", 10), null);
|
||||
assert.ok(stats.lifecycle.capacityEvictions >= 1);
|
||||
});
|
||||
|
||||
it("releases principal byte budget when an entry is deleted", () => {
|
||||
const content = "x".repeat(ccr.MAX_CCR_BLOCK_BYTES);
|
||||
const stored = ccr.tryStoreBlock(content, "tenant-a", { now: 1 });
|
||||
assert.equal(stored.stored, true);
|
||||
if (!stored.stored) return;
|
||||
assert.equal(ccr.getCcrStoreStats("tenant-a", 1).bytes, ccr.MAX_CCR_BLOCK_BYTES);
|
||||
assert.equal(ccr.deleteCcrBlock(stored.hash, "tenant-a", 1), true);
|
||||
assert.equal(ccr.getCcrStoreStats("tenant-a", 1).bytes, 0);
|
||||
});
|
||||
|
||||
it("keeps all read, list, inspect, delete, and stats operations principal-isolated", () => {
|
||||
const result = ccr.tryStoreBlock("tenant-a secret", "tenant-a", { source: "mcp" });
|
||||
assert.equal(result.stored, true);
|
||||
if (!result.stored) return;
|
||||
assert.equal(ccr.retrieveBlock(result.hash, "tenant-b"), null);
|
||||
assert.equal(ccr.inspectCcrBlock(result.hash, "tenant-b"), null);
|
||||
assert.equal(ccr.listCcrBlocks("tenant-b").total, 0);
|
||||
assert.equal(ccr.deleteCcrBlock(result.hash, "tenant-b"), false);
|
||||
assert.equal(ccr.getCcrStoreStats("tenant-b").entries, 0);
|
||||
assert.deepEqual(ccr.getCcrStoreStats("tenant-b").lifecycle, {
|
||||
expiredEvictions: 0,
|
||||
capacityEvictions: 0,
|
||||
rejectedStores: 0,
|
||||
});
|
||||
assert.equal(ccr.retrieveBlock(result.hash, "tenant-a"), "tenant-a secret");
|
||||
});
|
||||
});
|
||||
|
||||
describe("CCR MCP contracts and handlers", () => {
|
||||
beforeEach(() => ccr.resetCcrStore());
|
||||
|
||||
it("registers all six tools canonically with least-privilege scopes", () => {
|
||||
const expected = {
|
||||
omniroute_ccr_store: ["write:compression"],
|
||||
omniroute_ccr_retrieve: ["read:compression"],
|
||||
omniroute_ccr_inspect: ["read:compression"],
|
||||
omniroute_ccr_list: ["read:compression"],
|
||||
omniroute_ccr_delete: ["write:compression"],
|
||||
omniroute_ccr_stats: ["read:compression"],
|
||||
} as const;
|
||||
for (const [name, scopes] of Object.entries(expected)) {
|
||||
assert.ok(schemas.MCP_TOOL_MAP[name], `${name} must exist in MCP_TOOL_MAP`);
|
||||
assert.deepEqual(schemas.MCP_TOOL_MAP[name].scopes, scopes);
|
||||
assert.deepEqual(MCP_TOOL_SCOPES[name], scopes);
|
||||
}
|
||||
});
|
||||
|
||||
it("validates the MCP store limit in UTF-8 bytes instead of JavaScript characters", () => {
|
||||
const multibyte = "ü".repeat(ccr.MAX_CCR_BLOCK_BYTES / 2 + 1);
|
||||
assert.equal(schemas.ccrStoreInput.safeParse({ content: multibyte }).success, false);
|
||||
const result = ccr.tryStoreBlock(multibyte, "tenant-a", { source: "mcp" });
|
||||
assert.equal(result.stored, false);
|
||||
if (!result.stored) assert.equal(result.reason, "block_too_large");
|
||||
});
|
||||
|
||||
it("stores through MCP and exposes metadata without content in inspect/list", async () => {
|
||||
const stored = await tools.handleCcrStoreTool(
|
||||
{ content: "sensitive body", contentType: "text/plain", ttlSeconds: 120 },
|
||||
writeExtra
|
||||
);
|
||||
assert.equal(stored.stored, true);
|
||||
if (!stored.stored) return;
|
||||
|
||||
const inspected = await tools.handleCcrInspectTool({ hash: stored.reference.hash }, readExtra);
|
||||
assert.equal(inspected.found, true);
|
||||
assert.equal(JSON.stringify(inspected).includes("sensitive body"), false);
|
||||
const listed = await tools.handleCcrListTool({ limit: 10 }, readExtra);
|
||||
assert.equal(listed.total, 1);
|
||||
assert.equal(JSON.stringify(listed).includes("sensitive body"), false);
|
||||
assert.equal((await tools.handleCcrStatsTool({}, readExtra)).entries, 1);
|
||||
assert.deepEqual(await tools.handleCcrDeleteTool({ hash: stored.reference.hash }, writeExtra), {
|
||||
deleted: true,
|
||||
});
|
||||
});
|
||||
|
||||
it("does not include content in the CCR store audit input", () => {
|
||||
const secret = "never-persist-this-content";
|
||||
const auditInput = tools.buildCcrStoreAuditInput({
|
||||
content: secret,
|
||||
contentType: "text/plain",
|
||||
});
|
||||
assert.equal(JSON.stringify(auditInput).includes(secret), false);
|
||||
assert.deepEqual(auditInput, {
|
||||
bytes: Buffer.byteLength(secret, "utf8"),
|
||||
contentType: "text/plain",
|
||||
ttlSeconds: undefined,
|
||||
});
|
||||
});
|
||||
|
||||
it("refuses oversized full MCP responses and recommends ranged modes", async () => {
|
||||
const content = "large\n".repeat(50_000);
|
||||
const stored = ccr.tryStoreBlock(content, "tenant-a", { source: "mcp" });
|
||||
assert.equal(stored.stored, true);
|
||||
if (!stored.stored) return;
|
||||
const full = await tools.handleCcrRetrieveTool({ hash: stored.hash }, readExtra);
|
||||
assert.equal(full.found, true);
|
||||
assert.equal("content" in full, false);
|
||||
assert.equal("tooLargeForFull" in full && full.tooLargeForFull, true);
|
||||
|
||||
const head = await tools.handleCcrRetrieveTool(
|
||||
{ hash: stored.hash, mode: "head", n: 2 },
|
||||
readExtra
|
||||
);
|
||||
assert.equal(head.found, true);
|
||||
assert.equal("content" in head ? head.content : undefined, "large\nlarge");
|
||||
});
|
||||
|
||||
it("does not reveal another principal's block through MCP handlers", async () => {
|
||||
const stored = ccr.tryStoreBlock("only tenant a", "tenant-a", { source: "mcp" });
|
||||
assert.equal(stored.stored, true);
|
||||
if (!stored.stored) return;
|
||||
const other = { authInfo: { clientId: "tenant-b", scopes: ["read:compression"] } };
|
||||
assert.equal((await tools.handleCcrRetrieveTool({ hash: stored.hash }, other)).found, false);
|
||||
assert.equal((await tools.handleCcrInspectTool({ hash: stored.hash }, other)).found, false);
|
||||
assert.equal((await tools.handleCcrListTool({}, other)).total, 0);
|
||||
});
|
||||
});
|
||||
@@ -7,8 +7,8 @@ import assert from "node:assert/strict";
|
||||
// MCP_TOOLS (open-sse/mcp-server/schemas/tools.ts) and agentSkillTools
|
||||
// (open-sse/mcp-server/tools/agentSkillTools.ts), so the additive sum reported 99
|
||||
// while only 96 distinct tool names actually exist. countUniqueMcpTools
|
||||
// (open-sse/mcp-server/toolCount.ts) fixes this by unioning tool names into a Set
|
||||
// before counting, so a tool present in multiple collections is only counted once.
|
||||
// (open-sse/mcp-server/toolCount.ts) fixes this by unioning tool names from every
|
||||
// registered collection into a Set, so each user-visible tool is counted once.
|
||||
|
||||
const { countUniqueMcpTools } = await import("../../open-sse/mcp-server/toolCount.ts");
|
||||
const { MCP_TOOLS } = await import("../../open-sse/mcp-server/schemas/tools.ts");
|
||||
@@ -21,6 +21,7 @@ const { gamificationTools } = await import("../../open-sse/mcp-server/tools/gami
|
||||
const { pluginTools } = await import("../../open-sse/mcp-server/tools/pluginTools.ts");
|
||||
const { notionTools } = await import("../../open-sse/mcp-server/tools/notionTools.ts");
|
||||
const { obsidianTools } = await import("../../open-sse/mcp-server/tools/obsidianTools.ts");
|
||||
const { compressionTools } = await import("../../open-sse/mcp-server/tools/compressionTools.ts");
|
||||
|
||||
type NamedTool = { name: string };
|
||||
|
||||
@@ -53,6 +54,7 @@ test("#6854: countUniqueMcpTools de-duplicates tools registered in multiple coll
|
||||
pluginTools: pluginTools as unknown as NamedTool[],
|
||||
notionTools: notionTools as unknown as NamedTool[],
|
||||
obsidianTools: obsidianTools as unknown as NamedTool[],
|
||||
compressionTools: compressionTools as unknown as Record<string, NamedTool>,
|
||||
};
|
||||
|
||||
const total = countUniqueMcpTools(collections);
|
||||
@@ -71,11 +73,6 @@ test("#6854: countUniqueMcpTools de-duplicates tools registered in multiple coll
|
||||
);
|
||||
|
||||
assert.equal(total, uniqueNames.size, "countUniqueMcpTools must equal the unique-name count");
|
||||
assert.equal(
|
||||
total,
|
||||
naiveAdditiveSum - overlap.length,
|
||||
"unique count must be exactly the additive sum minus the double-counted overlap"
|
||||
);
|
||||
assert.ok(
|
||||
total < naiveAdditiveSum,
|
||||
"unique count must be strictly less than the naive additive sum given a known overlap"
|
||||
|
||||
Reference in New Issue
Block a user