OmniRoute/AGENTS.md

# omniroute — Agent Guidelines

## Project

Unified AI proxy/router — route any LLM through one endpoint. Multi-provider support
with **160+ providers** (OpenAI, Anthropic, Gemini, DeepSeek, Groq, xAI, Mistral, Fireworks,
Cohere, NVIDIA, Cerebras, Pollinations, Puter, Cloudflare AI, HuggingFace, DeepInfra,
SambaNova, Meta Llama API, Moonshot AI, AI21 Labs, Databricks, Snowflake, and many more)
with **MCP Server** (37 tools), **A2A v0.3 Protocol**, and **Electron desktop app**.

## Stack

- **Runtime**: Next.js 16 (App Router), Node.js `>=20.20.2 <21`, `>=22.22.2 <23`, or `>=24.0.0 <25`, ES Modules (`"type": "module"`)
- **Language**: TypeScript 5.9 (`src/`) + JavaScript (`open-sse/`, `electron/`)
- **Database**: better-sqlite3 (SQLite) — `DATA_DIR` configurable, default `~/.omniroute/`
- **Streaming**: SSE via `open-sse` internal workspace package
- **Styling**: Tailwind CSS v4
- **i18n**: next-intl with 40+ languages
- **Desktop**: Electron (cross-platform: Windows, macOS, Linux)
- **Schemas**: Zod v4 for all API / MCP input validation

---

## Build, Lint, and Test Commands

| Command                             | Description                       |
| ----------------------------------- | --------------------------------- |
| `npm run dev`                       | Start Next.js dev server          |
| `npm run build`                     | Production build (isolated)       |
| `npm run start`                     | Run production build              |
| `npm run build:cli`                 | Build CLI package                 |
| `npm run lint`                      | ESLint on all source files        |
| `npm run typecheck:core`            | TypeScript core type checking     |
| `npm run typecheck:noimplicit:core` | Strict checking (no implicit any) |
| `npm run check`                     | Run lint + test                   |
| `npm run check:cycles`              | Check for circular dependencies   |
| `npm run electron:dev`              | Run Electron app in dev mode      |
| `npm run electron:build`            | Build Electron app for current OS |

### Running Tests

```bash
# All tests (unit + vitest + ecosystem + e2e)
npm run test:all

# Single test file (Node.js native test runner — most tests use this)
node --import tsx/esm --test tests/unit/your-file.test.ts
node --import tsx/esm --test tests/unit/plan3-p0.test.ts
node --import tsx/esm --test tests/unit/fixes-p1.test.ts
node --import tsx/esm --test tests/unit/security-fase01.test.ts

# Integration tests
node --import tsx/esm --test tests/integration/*.test.ts

# Vitest (MCP server, autoCombo)
npm run test:vitest

# E2E with Playwright
npm run test:e2e

# Protocol clients E2E (MCP transports, A2A)
npm run test:protocols:e2e

# Ecosystem compatibility tests
npm run test:ecosystem

# Coverage (see CONTRIBUTING.md)
npm run test:coverage
```

**For authoritative coverage requirements, test execution, and PR gates, see [`CONTRIBUTING.md`](CONTRIBUTING.md#running-tests).**

---

## Code Style Guidelines

### Formatting (Prettier — enforced via lint-staged)

2 spaces · semicolons required · double quotes (`"`) · 100 char width · es5 trailing commas.
Always run `prettier --write` on changed files.

### TypeScript

- **Target**: ES2022 · **Module**: `esnext` · **Resolution**: `bundler`
- `strict: false` — prefer explicit types, don't rely on inference
- Path aliases: `@/*` → `src/`, `@omniroute/open-sse` → `open-sse/`, `@omniroute/open-sse/*` → `open-sse/*`

### ESLint Rules

- **Security (error, everywhere)**: `no-eval`, `no-implied-eval`, `no-new-func`
- **Relaxed in `open-sse/` and `tests/`**: `@typescript-eslint/no-explicit-any` = warn
- React hooks rules and `@next/next/no-assign-module-variable` disabled in `open-sse/` and `tests/`

### Naming

| Element             | Convention                       | Example                              |
| ------------------- | -------------------------------- | ------------------------------------ |
| Files               | camelCase / kebab-case           | `chatCore.ts`, `tokenHealthCheck.ts` |
| React components    | PascalCase                       | `Dashboard.tsx`, `ProviderCard.tsx`  |
| Functions/variables | camelCase                        | `getHealth()`, `switchCombo()`       |
| Constants           | UPPER_SNAKE                      | `MAX_RETRIES`, `DEFAULT_TIMEOUT`     |
| Interfaces          | PascalCase (`I` prefix optional) | `ProviderConfig`                     |
| Enums               | PascalCase (members too)         | `LogLevel.Error`                     |

### Imports

- **Order**: external → internal (`@/`, `@omniroute/open-sse`) → relative (`./`, `../`)
- **No barrel imports** from `localDb.ts` — import from the specific `db/` module instead

### Error Handling

- try/catch with specific error types; always log with context (pino logger)
- Never silently swallow errors in SSE streams — use abort signals for cleanup
- Return proper HTTP status codes (4xx client, 5xx server)

### Security

- **NEVER** commit API keys, secrets, or credentials
- Validate all user inputs with Zod schemas
- Auth middleware required on all API routes
- Never log SQLite encryption keys
- Sanitize user content (dompurify for HTML)

---

## Architecture

### Data Layer (`src/lib/db/`)

All persistence uses SQLite through domain-specific modules:
`core.ts`, `providers.ts`, `models.ts`, `combos.ts`, `apiKeys.ts`, `settings.ts`,
`backup.ts`, `proxies.ts`, `prompts.ts`, `webhooks.ts`, `detailedLogs.ts`,
`domainState.ts`, `registeredKeys.ts`, `quotaSnapshots.ts`, `modelComboMappings.ts`,
`cliToolState.ts`, `encryption.ts`, `readCache.ts`, `secrets.ts`, `stateReset.ts`,
`contextHandoffs.ts`, `compression.ts`.
Schema migrations live in `db/migrations/` and run via `migrationRunner.ts`.
`src/lib/localDb.ts` is a **re-export layer only** — never add logic there.

#### DB Internals

- **`core.ts`**: `getDbInstance()` returns a singleton `better-sqlite3` instance with WAL
  journaling. `SCHEMA_SQL` defines 15 base tables. Helpers: `rowToCamel`, `encryptConnectionFields`.
- **`migrationRunner.ts`**: Applies versioned SQL files from `db/migrations/` inside transactions.
  Tracks applied migrations in `_omniroute_migrations` table.
- **Migrations**: 22 files (`001_initial_schema.sql` → `022_compression_settings.sql`).
  Each migration is idempotent and runs in a transaction.
- **Domain modules** import `getDbInstance()` from `core.ts` for all CRUD operations.
  Each module owns a specific table/set of tables (e.g., `providers.ts` → `provider_connections`,
  `combos.ts` → `combos`). Encryption helpers protect sensitive fields at rest.
- **`localDb.ts`** re-exports all domain modules — consumers import from here for convenience.

### API Route Layer (`src/app/api/v1/`)

Next.js App Router routes — each follows a consistent pattern:

```
Route → CORS preflight → Body validation (Zod) → Optional auth (extractApiKey/isValidApiKey)
  → API key policy enforcement (enforceApiKeyPolicy) → Handler delegation (open-sse)
```

| Route                           | Handler                   | Notes                                     |
| ------------------------------- | ------------------------- | ----------------------------------------- |
| `chat/completions/route.ts`     | `handleChat()`            | + prompt injection guard (clones request) |
| `responses/route.ts`            | `handleChat()` (unified)  | Responses API format                      |
| `embeddings/route.ts`           | `handleEmbedding()`       | Model listing + creation                  |
| `images/generations/route.ts`   | `handleImageGeneration()` | Model listing + creation                  |
| `audio/transcriptions/route.ts` | audio handler             | Multipart form data                       |
| `audio/speech/route.ts`         | TTS handler               | Binary audio response                     |
| `videos/generations/route.ts`   | video handler             | ComfyUI/SD WebUI                          |
| `music/generations/route.ts`    | music handler             | ComfyUI workflows                         |
| `moderations/route.ts`          | moderation handler        | Content safety                            |
| `rerank/route.ts`               | rerank handler            | Document relevance                        |
| `search/route.ts`               | search handler            | Web search (5 providers)                  |

**No global Next.js middleware file** — interception is route-specific. Auth is optional
(controlled by `REQUIRE_API_KEY` env). Prompt injection guard is unique to chat completions.

### Request Pipeline (`open-sse/`)

The `open-sse/` workspace is the core streaming engine. Full request flow:

```
Client Request
  → src/app/api/v1/.../route.ts (Next.js route)
    → open-sse/handlers/chatCore.ts::handleChatCore()
      → Semantic/signature cache check
      → Rate limit check (rateLimitManager)
      → Combo routing? → open-sse/services/combo.ts::handleComboChat()
        → resolveComboTargets() → ordered ResolvedComboTarget[]
        → For each target: handleSingleModel() (wraps chatCore)
      → translateRequest() (open-sse/translator/)
        → Convert source format (e.g., OpenAI) → target format (e.g., Claude)
      → getExecutor() → provider-specific executor instance
        → executor.execute() (BaseExecutor → DefaultExecutor or provider-specific)
          → buildUrl() + buildHeaders() + transformRequest()
          → fetch() to upstream provider
          → Retry logic with exponential backoff
      → Response translation back to client format
      → If Responses API: responsesTransformer.ts TransformStream
  → SSE stream or JSON response to client
```

**Handlers** (`open-sse/handlers/`): `chatCore.ts`, `responsesHandler.ts`, `embeddings.ts`,
`imageGeneration.ts`, `videoGeneration.ts`, `musicGeneration.ts`, `audioSpeech.ts`,
`audioTranscription.ts`, `moderations.ts`, `rerank.ts`, `search.ts`.

**Upstream headers**: merged after default auth; same header name replaces executor value.
**T5 intra-family fallback** recomputes headers using only the fallback model id.
Forbidden header names: `src/shared/constants/upstreamHeaders.ts` — keep sanitize,
Zod schemas, and unit tests aligned when editing.

### Provider Categories

- **Free** (4): Qoder AI, Qwen Code, Gemini CLI (deprecated), Kiro AI
- **OAuth** (8): Claude Code, Antigravity, Codex, GitHub Copilot, Cursor, Kimi Coding, Kilo Code, Cline
- **API Key** (120+): OpenAI, Anthropic, Gemini, DeepSeek, Groq, xAI, Mistral, Perplexity,
  Together, Fireworks, Cerebras, Cohere, NVIDIA, Nebius, SiliconFlow, Hyperbolic,
  HuggingFace, OpenRouter, Vertex AI, Cloudflare AI, Scaleway, AI/ML API, Pollinations,
  Puter, Longcat, Alibaba, Kimi, Minimax, Blackbox, Synthetic, Kilo Gateway,
  Z.AI, GLM, Deepgram, AssemblyAI, ElevenLabs, Cartesia, PlayHT, Inworld,
  NanoBanana, SD WebUI, ComfyUI, Ollama Cloud, Perplexity Search, Serper, Brave, Exa,
  Tavily, OpenCode Zen/Go, Bailian Coding Plan, DeepInfra, Vercel AI Gateway,
  Lambda AI, SambaNova, nScale, OVHcloud AI, Baseten, PublicAI, Moonshot AI,
  Meta Llama API, v0 (Vercel), Morph, Featherless AI, FriendliAI, LlamaGate,
  Galadriel, Weights & Biases Inference, Volcengine, AI21 Labs, Venice.ai,
  Codestral, Upstage, Maritalk, Xiaomi MiMo, Inference.net, NanoGPT, Predibase,
  Bytez, Heroku AI, Databricks, Snowflake Cortex, GigaChat (Sber), CrofAI,
  AgentRouter, ChatGPT Web, Baidu Qianfan, AWS Polly, RunwayML, GitLab Duo,
  Amazon Q, Empower, Poe, and many more.
- **Self-Hosted** (8+): LM Studio, vLLM, Lemonade, Llamafile, Triton, Docker Model Runner, Xinference, Oobabooga
- **Custom**: OpenAI-compatible (`openai-compatible-*`) and Anthropic-compatible (`anthropic-compatible-*`) prefixes

Providers are registered in `src/shared/constants/providers.ts` with Zod validation at module load.

### Executors (`open-sse/executors/`)

Provider-specific request executors: `base.ts`, `default.ts`, `cursor.ts`, `codex.ts`,
`antigravity.ts`, `github.ts`, `gemini-cli.ts`, `kiro.ts`, `qoder.ts`, `vertex.ts`,
`cloudflare-ai.ts`, `opencode.ts`, `pollinations.ts`, `puter.ts`.

#### Executor Internals

- **`base.ts`** (`BaseExecutor`): Abstract base with `buildUrl()`, `buildHeaders()`,
  `transformRequest()`, retry logic (exponential backoff), and `execute()`. Subclasses
  override URL/header/transform methods for provider-specific behavior.
- **`default.ts`** (`DefaultExecutor extends BaseExecutor`): Handles most OpenAI-compatible
  providers. Reads provider config from `providerRegistry.ts` to resolve base URL, auth
  header format, and request transformations.
- **`getExecutor()`** (`executors/index.ts`): Factory that returns the correct executor
  instance based on provider ID. Provider-specific executors (Cursor, Codex, Vertex, etc.)
  override only what differs from the default.

### Translator (`open-sse/translator/`)

Translates between API formats (OpenAI-format ↔ Anthropic, Gemini, etc.).
Includes request/response translators with helpers for image handling.

#### Translator Internals

- **`translator/index.ts`**: Exports `translateRequest()` and format constants. Called by
  `chatCore.ts` before executor dispatch.
- **Flow**: `translateRequest(body, sourceFormat, targetFormat)` → detects source format
  (OpenAI, Anthropic, Gemini) → applies the matching translator module → returns
  transformed body ready for the target provider.
- **Response translation** runs in reverse after upstream response, converting back to
  the client's expected format.

### Transformer (`open-sse/transformer/`)

`responsesTransformer.ts` — transforms Responses API format to/from Chat Completions format.

#### Transformer Internals

- **`createResponsesApiTransformStream()`**: Returns a `TransformStream` that converts
  Chat Completions SSE chunks (`data: {"choices":[...]}`) into Responses API SSE events
  (`response.output_item.added`, `response.output_text.delta`, etc.).
- Used when the client sends a Responses API request: the request is internally converted
  to Chat Completions format, dispatched normally, and the response is piped through this
  transform stream before reaching the client.

### Services (`open-sse/services/`)

36+ service modules including: `combo.ts` (routing engine), `usage.ts`, `tokenRefresh.ts`,
`rateLimitManager.ts`, `accountFallback.ts`, `sessionManager.ts`, `wildcardRouter.ts`,
`autoCombo/`, `intentClassifier.ts`, `taskAwareRouter.ts`, `thinkingBudget.ts`,
`contextManager.ts`, `modelDeprecation.ts`, `modelFamilyFallback.ts`,
`emergencyFallback.ts`, `workflowFSM.ts`, `backgroundTaskDetector.ts`, `ipFilter.ts`,
`signatureCache.ts`, `volumeDetector.ts`, `contextHandoff.ts`, `compression/` (prompt
compression pipeline), and more.

#### Prompt Compression Pipeline (`compression/`)

Modular prompt compression that runs proactively before the existing reactive context manager.

- **`strategySelector.ts`**: Selects compression mode based on config, compression combo assignments,
  combo overrides, auto-trigger thresholds, and defaults. Priority: assigned compression combo >
  combo override > auto-trigger > default mode > off.
- **`lite.ts`**: 5 lite-mode techniques: `collapseWhitespace`, `dedupSystemPrompt`,
  `compressToolResults`, `removeRedundantContent`, `replaceImageUrls`. Target: 10-15% savings at
  <1ms latency.
- **`caveman.ts` / `cavemanRules.ts`**: Caveman-style semantic condensation backed by built-in
  rules plus file-loaded language packs under `compression/rules/`.
- **`engines/rtk/`**: Rule-based terminal/tool-output compression inspired by RTK patterns. Detects
  command output classes, applies JSON filter packs, deduplicates repeated lines, strips ANSI/code
  noise, and preserves errors/actionable context. The RTK JSON DSL supports replace,
  match-output short-circuit, strip/keep, per-line truncation, head/tail/max-line truncation,
  inline tests, trust-gated project/global custom filters, and optional redacted raw-output
  retention for authenticated recovery.
- **`engines/registry.ts`**: Registers engines (`caveman`, `rtk`) and powers stacked pipelines.
- **`stats.ts`**: Per-request compression stats tracking (original tokens, compressed tokens,
  savings %, techniques used, engine breakdown, compression combo id).
- **`types.ts`**: `CompressionMode` (off/lite/standard/aggressive/ultra/rtk/stacked),
  `CompressionConfig`, `CompressionStats`, `CompressionResult`.
- DB settings in `src/lib/db/compression.ts`, compression combos in
  `src/lib/db/compressionCombos.ts`, API routes under `src/app/api/settings/compression/`,
  `src/app/api/context/*`, and preview/language-pack routes under `src/app/api/compression/*`.

#### Combo Routing Engine (`combo.ts`)

- **`handleComboChat()`**: Entry point for combo-routed requests. Receives the combo config
  and iterates through targets in order until one succeeds or all fail.
- **`resolveComboTargets()`**: Expands a combo configuration into an ordered array of
  `ResolvedComboTarget[]`, each specifying provider + model + account + credentials.
- **Strategies** (13): priority, weighted, fill-first, round-robin, P2C, random, least-used,
  cost-optimized, strict-random, auto, lkgp, context-optimized, context-relay.
- Each target calls **`handleSingleModel()`** which wraps `handleChatCore()` with
  per-target error handling and circuit breaker checks.

### Domain Layer (`src/domain/`)

Policy engine modules: `policyEngine.ts`, `comboResolver.ts`, `costRules.ts`,
`degradation.ts`, `fallbackPolicy.ts`, `lockoutPolicy.ts`, `modelAvailability.ts`,
`providerExpiration.ts`, `quotaCache.ts`, `responses.ts`, `configAudit.ts`.

### MCP Server (`open-sse/mcp-server/`)

37 tools, 3 transports (stdio / SSE / Streamable HTTP). Scoped auth (10 scopes), Zod schemas.

**Core tools** (20): get_health, list_combos, get_combo_metrics, switch_combo, check_quota,
route_request, cost_report, list_models_catalog, web_search, simulate_route, set_budget_guard,
set_routing_strategy, set_resilience_profile, test_combo, get_provider_metrics,
best_combo_for_task, explain_route, get_session_snapshot, db_health_check, sync_pricing.

**Cache tools** (2): cache_stats, cache_flush.

**Compression tools** (5): compression_status, compression_configure, set_compression_engine,
list_compression_combos, compression_combo_stats.

**1proxy tools** (3): oneproxy_fetch, oneproxy_rotate, oneproxy_stats.

**Memory tools** (3): memory_search, memory_add, memory_clear.

**Skill tools** (4): skills_list, skills_enable, skills_execute, skills_executions.

#### MCP Internals

- **Tool registration**: Each tool is an object with `{ name, description, inputSchema: ZodSchema,
handler: async (args) => {...} }`. Zod validates inputs before the handler fires.
- **`createMcpServer()`** and **`startMcpStdio()`** exported from `mcp-server/index.ts`.
  `createMcpServer()` wires all tool sets; `startMcpStdio()` launches the stdio transport.
- **Transports**: stdio (CLI `omniroute --mcp`), SSE (`/api/mcp/sse`), Streamable HTTP
  (`/api/mcp/stream`). All share the same tool/scope engine.
- **Scopes** (10): Control which tool categories an API key can access. Enforcement happens
  before handler dispatch.
- **Audit**: Every tool invocation is logged to SQLite (`mcp_audit` table) with tool name,
  args, success/failure, API key attribution, and timestamp.

### A2A Server (`src/lib/a2a/`)

JSON-RPC 2.0, SSE streaming, Task Manager with TTL cleanup.
Agent Card at `/.well-known/agent.json`.
Skills: `quotaManagement.ts`, `smartRouting.ts`.

#### A2A Internals

- **`taskManager.ts`**: State machine lifecycle for tasks: `submitted → working →
completed | failed | canceled`. Tasks have TTL and are cleaned up automatically.
- **JSON-RPC methods**: `message/send` (sync), `message/stream` (SSE), `tasks/get`,
  `tasks/cancel`. Dispatched via `POST /a2a`.
- **Skills**: Registered in a DB-backed registry. Each skill receives task context
  (messages, metadata) and returns structured results. `quotaManagement.ts` summarizes
  quota; `smartRouting.ts` recommends routing decisions.
- **Agent Card**: `/.well-known/agent.json` exposes capabilities, skills, and metadata
  for client auto-discovery.

### ACP Module (`src/lib/acp/`)

Agent Communication Protocol registry and manager.

### Memory System (`src/lib/memory/`)

Extraction, injection, retrieval, summarization, and store modules for persistent
conversational memory across sessions.

### Skills System (`src/lib/skills/`)

Extensible skill framework: registry, executor, sandbox, built-in skills,
custom skill support, interception, and injection.

#### Skills Internals

- **`registry.ts`**: DB-backed skill registration and discovery. Skills have metadata
  (name, description, version, enabled status) stored in SQLite.
- **`executor.ts`**: Execution engine with configurable timeout and retry logic.
  Receives skill name + input, looks up the skill, runs it in the sandbox.
- **`sandbox.ts`**: Isolation layer for custom (user-provided) skills. Limits resource
  access and execution time.
- **Built-in skills**: Ship with OmniRoute (e.g., quota management, routing). Located
  alongside the registry.
- **Interception/Injection**: Skills can intercept requests in the pipeline (pre/post
  processing) or inject context into prompts.

### Compliance (`src/lib/compliance/`)

Policy index for compliance enforcement.

### MITM Proxy (`src/mitm/`)

MITM proxy capability with certificate management, DNS handling, and target routing.

### Middleware (`src/middleware/`)

Request middleware including `promptInjectionGuard.ts`.

### Adding a New Provider

1. Register in `src/shared/constants/providers.ts`
2. Add executor in `open-sse/executors/` (if custom logic needed)
3. Add translator in `open-sse/translator/` (if non-OpenAI format)
4. Add OAuth config in `src/lib/oauth/constants/oauth.ts` (if OAuth-based)
5. Add models in `open-sse/config/providerRegistry.ts`

---

## Subdirectory AGENTS.md Files

- **[`open-sse/AGENTS.md`](open-sse/AGENTS.md)** — Streaming engine, request pipeline, handlers, and executors
- **[`src/lib/db/AGENTS.md`](src/lib/db/AGENTS.md)** — SQLite persistence, domain modules, migrations
- **[`open-sse/services/AGENTS.md`](open-sse/services/AGENTS.md)** — Routing engine, combo resolution, strategy selection

---

## Review Focus

- **DB ops** go through `src/lib/db/` modules, never raw SQL in routes
- **Provider requests** flow through `open-sse/handlers/`
- **MCP/A2A pages** are tabs inside `/dashboard/endpoint`, not standalone routes
- **No memory leaks** in SSE streams (abort signals, cleanup)
- **Rate limit headers** must be parsed correctly
- All API inputs validated with **Zod schemas**
- **Provider constants** validated at module load via Zod (`src/shared/validation/providerSchema.ts`)
- **Pricing data** syncs from LiteLLM via `src/lib/pricingSync.ts`
- **Memory/Skills** are cross-cutting: affect MCP tools, request pipeline, and A2A skills
- **⛔ NEVER close a contributor's PR** after using their code — always merge via GitHub so they get credit. See `.agents/workflows/review-prs.md` for full policy.