mirror of
https://github.com/diegosouzapw/OmniRoute.git
synced 2026-08-04 14:22:09 +03:00
docs: comprehensive v0.8.5 documentation update
- README: expand features table from 16 to 40+ entries across 5 categories (Core Routing, Multi-Modal APIs, Resilience, Observability, Deployment) - README: update tech stack to reflect 100% TypeScript codebase - README: update Docker tag 0.6.0 → 0.8.5, release command to v0.8.5 - CHANGELOG: add detailed v0.8.5 section (40 commits categorized) covering TLS spoofing, SQLite logs, unified logging, full TS migration, Qwen fix, VPS compatibility, CI improvements, dependency bumps - ARCHITECTURE: update all open-sse file references .js → .ts - CODEBASE_DOCUMENTATION: update all open-sse file references .js → .ts
This commit is contained in:
diegosouzapw
49
CHANGELOG.md
49
CHANGELOG.md
@@ -7,6 +7,54 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||
|
||||
---
|
||||
|
||||
## [0.8.5] — 2026-02-17
|
||||
|
||||
### Added
|
||||
|
||||
- 🔒 **TLS fingerprint spoofing** — Implement browser-like TLS fingerprinting via `wreq-js` to bypass bot detection on providers that enforce TLS client fingerprint checks (`3dd0cc1`, PR #52)
|
||||
- 💾 **SQLite proxy log persistence** — Proxy request/response logs now persist to SQLite database, surviving server restarts. Previously, logs were lost on restart (`f1664fe`, PR #53)
|
||||
- 📋 **Unified test logging** — Shared `Logger` + Proxy logging infrastructure for all provider connection test flows. Consistent log formatting across batch and individual tests (`bce302e`, PR #55)
|
||||
|
||||
### Refactored
|
||||
|
||||
- 🔷 **Full TypeScript migration — `src/`** — Migrated the entire `src/` directory from JavaScript to TypeScript. All `.js`/`.jsx` files converted to `.ts`/`.tsx` with proper type annotations across API routes, lib modules, components, services, stores, domain layer, and shared utilities (`d0ca595`)
|
||||
- **Wave 1**: Shared component interfaces + EventTarget fixes (`dfdd2a2`)
|
||||
- **Wave 2**: Utils & services typed fields, Zustand stores, logger, sync scheduler (`89dd107`, `b2907cd`)
|
||||
- **Wave 3a**: Lib layer, DB, compliance, domain layer typed (`9e13fe2`)
|
||||
- **Wave 3b**: Usage, CLI runtime, SSE auth/logger typed (`a291abd`)
|
||||
- **Wave 3c**: OAuth services + server utils typed (`d62cf8d`)
|
||||
- **Wave 4a**: 7 API routes — providers, cli-tools, oauth (`7cdb923`)
|
||||
- **Wave 4b**: 7 more API routes — providers, test, usage, nodes (`5592c2e`)
|
||||
- **Wave 4c**: 8 files — components, SSE handlers, services (`d8ce9dc`)
|
||||
- **Dashboard hardening**: Resolve all TypeScript errors across dashboard pages (`7a463a3`, PR #61)
|
||||
- 🔷 **Full TypeScript migration — `open-sse/`** — Migrated all 94 `.js` files in the SSE routing engine to TypeScript (PR #62)
|
||||
- **Phase 1**: Rename all 94 `.js` → `.ts` files (`256e443`)
|
||||
- **Phase 6**: Reduce `@ts-ignore` from 231 → 186 with targeted fixes (`6a54b84`)
|
||||
- **Phase 7**: Eliminate ALL `@ts-ignore` annotations (186 → 0) and ALL TypeScript errors (237 → 0) — zero `@ts-ignore`, zero errors (`7b37a3c`)
|
||||
- Typing strategies: `Record<string, any>` for dynamic objects, optional function params, `as any` casts for custom Error/Array properties, `declare var EdgeRuntime` for edge compatibility, proper `fs`/`path` imports
|
||||
|
||||
### Fixed
|
||||
|
||||
- 🐛 **Qwen token refresh** — Detect `invalid_request` as unrecoverable error and switch broken test endpoints to `checkExpiry` method instead of failing silently (`1e0ffbc`, PR #60)
|
||||
- 🐛 **VPS batch test compatibility** — Eliminate HTTP self-calls in batch provider connection tests for VPS environments where localhost is unreachable (`a3bbbb5`, PR #54)
|
||||
- 🐛 **E2E test assertions** — Correct API endpoints and response format assertions in end-to-end tests (`92b5e66`)
|
||||
- 🐛 **CI coverage thresholds** — Lower coverage thresholds, use production server for E2E, block ESLint major upgrades from breaking CI (`3ca4b6b`, PR #51)
|
||||
|
||||
### Changed
|
||||
|
||||
- 📖 **Documentation update** — Updated all documentation to reflect JS → TS migration, corrected file extensions and import paths (`7ff8aa2`)
|
||||
- ⬆️ **CI/CD** — Bump `actions/checkout` v4 → v6, `actions/setup-node` v4 → v6, `peter-evans/dockerhub-description` v4 → v5
|
||||
|
||||
### Dependencies
|
||||
|
||||
- ⬆️ `undici` 7.21.0 → 7.22.0 (production)
|
||||
- ⬆️ `actions/checkout` 4 → 6
|
||||
- ⬆️ `actions/setup-node` 4 → 6
|
||||
- ⬆️ `peter-evans/dockerhub-description` 4 → 5
|
||||
- 🚫 `eslint` 10.0.0 blocked — major version incompatible with `eslint-config-next`
|
||||
|
||||
---
|
||||
|
||||
## [0.8.0] — 2026-02-16
|
||||
|
||||
### Added
|
||||
@@ -143,6 +191,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||
|
||||
---
|
||||
|
||||
[0.8.5]: https://github.com/diegosouzapw/OmniRoute/compare/v0.8.0...v0.8.5
|
||||
[0.8.0]: https://github.com/diegosouzapw/OmniRoute/compare/v0.7.0...v0.8.0
|
||||
[0.7.0]: https://github.com/diegosouzapw/OmniRoute/compare/v0.6.0...v0.7.0
|
||||
[0.6.0]: https://github.com/diegosouzapw/OmniRoute/compare/v0.5.0...v0.6.0
|
||||
|
||||
90
README.md
90
README.md
@@ -5,7 +5,7 @@
|
||||
|
||||
**Never stop coding. Auto-route to FREE & cheap AI models with smart fallback.**
|
||||
|
||||
**36+ Providers • Embeddings • Image Generation • Think Tag Parsing**
|
||||
**36+ Providers • Embeddings • Image Generation • Audio • Reranking • Full TypeScript**
|
||||
|
||||
**Free AI Provider for OpenClaw.**
|
||||
|
||||
@@ -158,31 +158,69 @@ docker compose --profile cli up -d
|
||||
| Image | Tag | Size | Description |
|
||||
| ------------------------ | -------- | ------ | --------------------- |
|
||||
| `diegosouzapw/omniroute` | `latest` | ~250MB | Latest stable release |
|
||||
| `diegosouzapw/omniroute` | `0.6.0` | ~250MB | Current version |
|
||||
| `diegosouzapw/omniroute` | `0.8.5` | ~250MB | Current version |
|
||||
|
||||
---
|
||||
|
||||
## 💡 Key Features
|
||||
|
||||
| Feature | What It Does |
|
||||
| ------------------------------- | --------------------------------------------- |
|
||||
| 🎯 **Smart 3-Tier Fallback** | Auto-route: Subscription → Cheap → Free |
|
||||
| 📊 **Real-Time Quota Tracking** | Live token count + reset countdown |
|
||||
| 🔄 **Format Translation** | OpenAI ↔ Claude ↔ Gemini seamless |
|
||||
| 👥 **Multi-Account Support** | Multiple accounts per provider |
|
||||
| 🔄 **Auto Token Refresh** | OAuth tokens refresh automatically |
|
||||
| 🎨 **Custom Combos** | Create unlimited model combinations |
|
||||
| 🧩 **Custom Models** | Add any model ID to any provider |
|
||||
| 📝 **Request Logging** | Debug mode with full request/response logs |
|
||||
| 💾 **Cloud Sync** | Sync config across devices |
|
||||
| 📊 **Usage Analytics** | Track tokens, cost, trends over time |
|
||||
| 🌐 **Deploy Anywhere** | Localhost, VPS, Docker, Cloudflare Workers |
|
||||
| 🔌 **Circuit Breaker** | Auto-open/close per-provider with cooldowns |
|
||||
| 🛡️ **Anti-Thundering Herd** | Mutex + auto rate-limit for API key providers |
|
||||
| 🧠 **Semantic Cache** | Two-tier cache reduces cost & latency |
|
||||
| ⚡ **Request Idempotency** | 5s dedup window for duplicate requests |
|
||||
| 📈 **Progress Tracking** | Opt-in SSE progress events for streaming |
|
||||
| 🧪 **LLM Evaluations** | Golden set testing with 4 match strategies |
|
||||
### 🧠 Core Routing & Intelligence
|
||||
|
||||
| Feature | What It Does |
|
||||
| ------------------------------- | --------------------------------------------------------------------------------- |
|
||||
| 🎯 **Smart 4-Tier Fallback** | Auto-route: Subscription → API Key → Cheap → Free |
|
||||
| 📊 **Real-Time Quota Tracking** | Live token count + reset countdown per provider |
|
||||
| 🔄 **Format Translation** | OpenAI ↔ Claude ↔ Gemini ↔ Cursor ↔ Kiro seamless |
|
||||
| 👥 **Multi-Account Support** | Multiple accounts per provider with P2C selection |
|
||||
| 🔄 **Auto Token Refresh** | OAuth tokens refresh automatically with retry |
|
||||
| 🎨 **Custom Combos** | 6 strategies: priority, weighted, round-robin, random, least-used, cost-optimized |
|
||||
| 🧩 **Custom Models** | Add any model ID to any provider |
|
||||
| 🌐 **Wildcard Router** | Route `provider/*` patterns to any provider dynamically |
|
||||
| 🧠 **Thinking Budget** | Passthrough, auto, custom, and adaptive modes for reasoning models |
|
||||
| 💬 **System Prompt Injection** | Global system prompt applied across all requests |
|
||||
| 📄 **Responses API** | Full OpenAI Responses API (`/v1/responses`) support for Codex |
|
||||
|
||||
### 🎵 Multi-Modal APIs
|
||||
|
||||
| Feature | What It Does |
|
||||
| -------------------------- | --------------------------------------------------- |
|
||||
| 🖼️ **Image Generation** | `/v1/images/generations` — 4 providers, 9+ models |
|
||||
| 📐 **Embeddings** | `/v1/embeddings` — 6 providers, 9+ models |
|
||||
| 🎤 **Audio Transcription** | `/v1/audio/transcriptions` — Whisper-compatible |
|
||||
| 🔊 **Text-to-Speech** | `/v1/audio/speech` — Multi-provider audio synthesis |
|
||||
| 🛡️ **Moderations** | `/v1/moderations` — Content safety checks |
|
||||
| 🔀 **Reranking** | `/v1/rerank` — Document relevance reranking |
|
||||
|
||||
### 🛡️ Resilience & Security
|
||||
|
||||
| Feature | What It Does |
|
||||
| ------------------------------- | ------------------------------------------------------------ |
|
||||
| 🔌 **Circuit Breaker** | Auto-open/close per-provider with configurable thresholds |
|
||||
| 🛡️ **Anti-Thundering Herd** | Mutex + semaphore rate-limit for API key providers |
|
||||
| 🧠 **Semantic Cache** | Two-tier cache (signature + semantic) reduces cost & latency |
|
||||
| ⚡ **Request Idempotency** | 5s dedup window for duplicate requests |
|
||||
| 🔒 **TLS Fingerprint Spoofing** | Bypass TLS-based bot detection via wreq-js |
|
||||
| 🌐 **IP Filtering** | Allowlist/blocklist for API access control |
|
||||
| 📋 **Compliance Audit Log** | Tamper-proof request logs with opt-out per API key |
|
||||
|
||||
### 📊 Observability & Analytics
|
||||
|
||||
| Feature | What It Does |
|
||||
| ------------------------ | ------------------------------------------------------ |
|
||||
| 📝 **Request Logging** | Debug mode with full request/response logs |
|
||||
| 💾 **SQLite Proxy Logs** | Persistent proxy logs survive server restarts |
|
||||
| 📊 **Usage Analytics** | Track tokens, cost, trends over time |
|
||||
| 📈 **Progress Tracking** | Opt-in SSE progress events for streaming |
|
||||
| 🧪 **LLM Evaluations** | Golden set testing with 4 match strategies |
|
||||
| 🔍 **Request Telemetry** | p50/p95/p99 latency aggregation + X-Request-Id tracing |
|
||||
|
||||
### ☁️ Deployment & Sync
|
||||
|
||||
| Feature | What It Does |
|
||||
| ------------------------- | ------------------------------------------------- |
|
||||
| 💾 **Cloud Sync** | Sync config across devices via Cloudflare Workers |
|
||||
| 🌐 **Deploy Anywhere** | Localhost, VPS, Docker, Cloudflare Workers |
|
||||
| 🔑 **API Key Management** | Generate, rotate, and scope API keys per provider |
|
||||
|
||||
---
|
||||
|
||||
@@ -254,17 +292,17 @@ registerSuite({
|
||||
## 🛠️ Tech Stack
|
||||
|
||||
- **Runtime**: Node.js 20+
|
||||
- **Language**: TypeScript 5.9 (src/) + JavaScript (open-sse/)
|
||||
- **Language**: TypeScript 5.9 — **100% TypeScript** across `src/` and `open-sse/` (v0.8.5)
|
||||
- **Framework**: Next.js 16 + React 19 + Tailwind CSS 4
|
||||
- **Database**: LowDB (JSON) + SQLite (domain state)
|
||||
- **Database**: LowDB (JSON) + SQLite (domain state + proxy logs)
|
||||
- **Streaming**: Server-Sent Events (SSE)
|
||||
- **Auth**: OAuth 2.0 (PKCE) + JWT + API Keys
|
||||
- **Testing**: Node.js test runner (368+ unit tests)
|
||||
- **CI/CD**: GitHub Actions (auto npm publish on release)
|
||||
- **CI/CD**: GitHub Actions (auto npm publish + Docker Hub on release)
|
||||
- **Website**: [omniroute.online](https://omniroute.online)
|
||||
- **Package**: [npmjs.com/package/omniroute](https://www.npmjs.com/package/omniroute)
|
||||
- **Docker**: [hub.docker.com/r/diegosouzapw/omniroute](https://hub.docker.com/r/diegosouzapw/omniroute)
|
||||
- **Resilience**: Circuit breaker, exponential backoff, anti-thundering herd
|
||||
- **Resilience**: Circuit breaker, exponential backoff, anti-thundering herd, TLS spoofing
|
||||
|
||||
---
|
||||
|
||||
@@ -309,7 +347,7 @@ See [CONTRIBUTING.md](CONTRIBUTING.md) for detailed guidelines.
|
||||
|
||||
```bash
|
||||
# Create a release — npm publish happens automatically
|
||||
gh release create v0.8.0 --title "v0.8.0" --generate-notes
|
||||
gh release create v0.8.5 --title "v0.8.5" --generate-notes
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
@@ -167,32 +167,32 @@ Management domains:
|
||||
Main flow modules:
|
||||
|
||||
- Entry: `src/sse/handlers/chat.ts`
|
||||
- Core orchestration: `open-sse/handlers/chatCore.js`
|
||||
- Core orchestration: `open-sse/handlers/chatCore.ts`
|
||||
- Provider execution adapters: `open-sse/executors/*`
|
||||
- Format detection/provider config: `open-sse/services/provider.js`
|
||||
- Model parse/resolve: `src/sse/services/model.ts`, `open-sse/services/model.js`
|
||||
- Account fallback logic: `open-sse/services/accountFallback.js`
|
||||
- Translation registry: `open-sse/translator/index.js`
|
||||
- Stream transformations: `open-sse/utils/stream.js`, `open-sse/utils/streamHandler.js`
|
||||
- Usage extraction/normalization: `open-sse/utils/usageTracking.js`
|
||||
- Think tag parser: `open-sse/utils/thinkTagParser.js`
|
||||
- Embedding handler: `open-sse/handlers/embeddings.js`
|
||||
- Embedding provider registry: `open-sse/config/embeddingRegistry.js`
|
||||
- Image generation handler: `open-sse/handlers/imageGeneration.js`
|
||||
- Image provider registry: `open-sse/config/imageRegistry.js`
|
||||
- Format detection/provider config: `open-sse/services/provider.ts`
|
||||
- Model parse/resolve: `src/sse/services/model.ts`, `open-sse/services/model.ts`
|
||||
- Account fallback logic: `open-sse/services/accountFallback.ts`
|
||||
- Translation registry: `open-sse/translator/index.ts`
|
||||
- Stream transformations: `open-sse/utils/stream.ts`, `open-sse/utils/streamHandler.ts`
|
||||
- Usage extraction/normalization: `open-sse/utils/usageTracking.ts`
|
||||
- Think tag parser: `open-sse/utils/thinkTagParser.ts`
|
||||
- Embedding handler: `open-sse/handlers/embeddings.ts`
|
||||
- Embedding provider registry: `open-sse/config/embeddingRegistry.ts`
|
||||
- Image generation handler: `open-sse/handlers/imageGeneration.ts`
|
||||
- Image provider registry: `open-sse/config/imageRegistry.ts`
|
||||
|
||||
Services (business logic):
|
||||
|
||||
- Account selection/scoring: `open-sse/services/accountSelector.js`
|
||||
- Context lifecycle management: `open-sse/services/contextManager.js`
|
||||
- IP filter enforcement: `open-sse/services/ipFilter.js`
|
||||
- Session tracking: `open-sse/services/sessionManager.js`
|
||||
- Request deduplication: `open-sse/services/signatureCache.js`
|
||||
- System prompt injection: `open-sse/services/systemPrompt.js`
|
||||
- Thinking budget management: `open-sse/services/thinkingBudget.js`
|
||||
- Wildcard model routing: `open-sse/services/wildcardRouter.js`
|
||||
- Rate limit management: `open-sse/services/rateLimitManager.js`
|
||||
- Circuit breaker: `open-sse/services/circuitBreaker.js`
|
||||
- Account selection/scoring: `open-sse/services/accountSelector.ts`
|
||||
- Context lifecycle management: `open-sse/services/contextManager.ts`
|
||||
- IP filter enforcement: `open-sse/services/ipFilter.ts`
|
||||
- Session tracking: `open-sse/services/sessionManager.ts`
|
||||
- Request deduplication: `open-sse/services/signatureCache.ts`
|
||||
- System prompt injection: `open-sse/services/systemPrompt.ts`
|
||||
- Thinking budget management: `open-sse/services/thinkingBudget.ts`
|
||||
- Wildcard model routing: `open-sse/services/wildcardRouter.ts`
|
||||
- Rate limit management: `open-sse/services/rateLimitManager.ts`
|
||||
- Circuit breaker: `open-sse/services/circuitBreaker.ts`
|
||||
|
||||
Domain layer modules:
|
||||
|
||||
@@ -242,7 +242,7 @@ Domain State DB (SQLite):
|
||||
- Dashboard cookie auth: `src/proxy.ts`, `src/app/api/auth/login/route.ts`
|
||||
- API key generation/verification: `src/shared/utils/apiKey.ts`
|
||||
- Provider secrets persisted in `providerConnections` entries
|
||||
- Outbound proxy support via `open-sse/utils/proxyFetch.js` (env vars) and `open-sse/utils/networkProxy.js` (configurable per-provider or global)
|
||||
- Outbound proxy support via `open-sse/utils/proxyFetch.ts` (env vars) and `open-sse/utils/networkProxy.ts` (configurable per-provider or global)
|
||||
|
||||
## 5) Cloud Sync
|
||||
|
||||
@@ -327,7 +327,7 @@ flowchart TD
|
||||
Q -- No --> R[Return all unavailable]
|
||||
```
|
||||
|
||||
Fallback decisions are driven by `open-sse/services/accountFallback.js` using status codes and error-message heuristics.
|
||||
Fallback decisions are driven by `open-sse/services/accountFallback.ts` using status codes and error-message heuristics.
|
||||
|
||||
## OAuth Onboarding and Token Refresh Lifecycle
|
||||
|
||||
@@ -359,7 +359,7 @@ sequenceDiagram
|
||||
Test-->>UI: validation result
|
||||
```
|
||||
|
||||
Refresh during live traffic is executed inside `open-sse/handlers/chatCore.js` via executor `refreshCredentials()`.
|
||||
Refresh during live traffic is executed inside `open-sse/handlers/chatCore.ts` via executor `refreshCredentials()`.
|
||||
|
||||
## Cloud Sync Lifecycle (Enable / Sync / Disable)
|
||||
|
||||
@@ -560,15 +560,15 @@ flowchart LR
|
||||
### Routing and Execution Core
|
||||
|
||||
- `src/sse/handlers/chat.ts`: request parse, combo handling, account selection loop
|
||||
- `open-sse/handlers/chatCore.js`: translation, executor dispatch, retry/refresh handling, stream setup
|
||||
- `open-sse/handlers/chatCore.ts`: translation, executor dispatch, retry/refresh handling, stream setup
|
||||
- `open-sse/executors/*`: provider-specific network and format behavior
|
||||
|
||||
### Translation Registry and Format Converters
|
||||
|
||||
- `open-sse/translator/index.js`: translator registry and orchestration
|
||||
- `open-sse/translator/index.ts`: translator registry and orchestration
|
||||
- Request translators: `open-sse/translator/request/*`
|
||||
- Response translators: `open-sse/translator/response/*`
|
||||
- Format constants: `open-sse/translator/formats.js`
|
||||
- Format constants: `open-sse/translator/formats.ts`
|
||||
|
||||
### Persistence
|
||||
|
||||
@@ -577,7 +577,7 @@ flowchart LR
|
||||
|
||||
## Provider Executor Coverage (Strategy Pattern)
|
||||
|
||||
Each provider has a specialized executor extending `BaseExecutor` (in `open-sse/executors/base.js`), which provides URL building, header construction, retry with exponential backoff, credential refresh hooks, and the `execute()` orchestration method.
|
||||
Each provider has a specialized executor extending `BaseExecutor` (in `open-sse/executors/base.ts`), which provides URL building, header construction, retry with exponential backoff, credential refresh hooks, and the `execute()` orchestration method.
|
||||
|
||||
| Executor | Provider(s) | Special Handling |
|
||||
| --------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------ | -------------------------------------------------------------------- |
|
||||
@@ -650,10 +650,10 @@ Translations are selected dynamically based on source payload shape and provider
|
||||
| -------------------------------------------------- | ------------------ | ---------------------------------------------------- |
|
||||
| `POST /v1/chat/completions` | OpenAI Chat | `src/sse/handlers/chat.ts` |
|
||||
| `POST /v1/messages` | Claude Messages | Same handler (auto-detected) |
|
||||
| `POST /v1/responses` | OpenAI Responses | `open-sse/handlers/responsesHandler.js` |
|
||||
| `POST /v1/embeddings` | OpenAI Embeddings | `open-sse/handlers/embeddings.js` |
|
||||
| `POST /v1/responses` | OpenAI Responses | `open-sse/handlers/responsesHandler.ts` |
|
||||
| `POST /v1/embeddings` | OpenAI Embeddings | `open-sse/handlers/embeddings.ts` |
|
||||
| `GET /v1/embeddings` | Model listing | API route |
|
||||
| `POST /v1/images/generations` | OpenAI Images | `open-sse/handlers/imageGeneration.js` |
|
||||
| `POST /v1/images/generations` | OpenAI Images | `open-sse/handlers/imageGeneration.ts` |
|
||||
| `GET /v1/images/generations` | Model listing | API route |
|
||||
| `POST /v1/providers/{provider}/chat/completions` | OpenAI Chat | Dedicated per-provider with model validation |
|
||||
| `POST /v1/providers/{provider}/embeddings` | OpenAI Embeddings | Dedicated per-provider with model validation |
|
||||
@@ -668,11 +668,11 @@ Translations are selected dynamically based on source payload shape and provider
|
||||
|
||||
## Bypass Handler
|
||||
|
||||
The bypass handler (`open-sse/utils/bypassHandler.js`) intercepts known "throwaway" requests from Claude CLI — warmup pings, title extractions, and token counts — and returns a **fake response** without consuming upstream provider tokens. This is triggered only when `User-Agent` contains `claude-cli`.
|
||||
The bypass handler (`open-sse/utils/bypassHandler.ts`) intercepts known "throwaway" requests from Claude CLI — warmup pings, title extractions, and token counts — and returns a **fake response** without consuming upstream provider tokens. This is triggered only when `User-Agent` contains `claude-cli`.
|
||||
|
||||
## Request Logger Pipeline
|
||||
|
||||
The request logger (`open-sse/utils/requestLogger.js`) provides a 7-stage debug logging pipeline, disabled by default, enabled via `ENABLE_REQUEST_LOGS=true`:
|
||||
The request logger (`open-sse/utils/requestLogger.ts`) provides a 7-stage debug logging pipeline, disabled by default, enabled via `ENABLE_REQUEST_LOGS=true`:
|
||||
|
||||
```
|
||||
1_req_client.json → 2_req_source.json → 3_req_openai.json → 4_req_target.json
|
||||
@@ -746,7 +746,7 @@ Environment variables actively used by code:
|
||||
## Known Architectural Notes
|
||||
|
||||
1. `usageDb` and `localDb` now share the same base directory policy (`DATA_DIR` -> `XDG_CONFIG_HOME/omniroute` -> `~/.omniroute`) with legacy file migration.
|
||||
2. `/api/v1/route.js` returns a static model list and is not the main models source used by `/v1/models`.
|
||||
2. `/api/v1/route.ts` returns a static model list and is not the main models source used by `/v1/models`.
|
||||
3. Request logger writes full headers/body when enabled; treat log directory as sensitive.
|
||||
4. Cloud behavior depends on correct `NEXT_PUBLIC_BASE_URL` and cloud endpoint reachability.
|
||||
5. The `open-sse/` directory is published as the `@omniroute/open-sse` **npm workspace package**. Source code imports it via `@omniroute/open-sse/...` (resolved by Next.js `transpilePackages`). File paths in this document still use the directory name `open-sse/` for consistency.
|
||||
|
||||
@@ -110,18 +110,18 @@ The **single source of truth** for all provider configuration.
|
||||
|
||||
| File | Purpose |
|
||||
| ----------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `constants.js` | `PROVIDERS` object with base URLs, OAuth credentials (defaults), headers, and default system prompts for every provider. Also defines `HTTP_STATUS`, `ERROR_TYPES`, `COOLDOWN_MS`, `BACKOFF_CONFIG`, and `SKIP_PATTERNS`. |
|
||||
| `credentialLoader.js` | Loads external credentials from `data/provider-credentials.json` and merges them over the hardcoded defaults in `PROVIDERS`. Keeps secrets out of source control while maintaining backwards compatibility. |
|
||||
| `providerModels.js` | Central model registry: maps provider aliases → model IDs. Functions like `getModels()`, `getProviderByAlias()`. |
|
||||
| `codexInstructions.js` | System instructions injected into Codex requests (editing constraints, sandbox rules, approval policies). |
|
||||
| `defaultThinkingSignature.js` | Default "thinking" signatures for Claude and Gemini models. |
|
||||
| `ollamaModels.js` | Schema definition for local Ollama models (name, size, family, quantization). |
|
||||
| `constants.ts` | `PROVIDERS` object with base URLs, OAuth credentials (defaults), headers, and default system prompts for every provider. Also defines `HTTP_STATUS`, `ERROR_TYPES`, `COOLDOWN_MS`, `BACKOFF_CONFIG`, and `SKIP_PATTERNS`. |
|
||||
| `credentialLoader.ts` | Loads external credentials from `data/provider-credentials.json` and merges them over the hardcoded defaults in `PROVIDERS`. Keeps secrets out of source control while maintaining backwards compatibility. |
|
||||
| `providerModels.ts` | Central model registry: maps provider aliases → model IDs. Functions like `getModels()`, `getProviderByAlias()`. |
|
||||
| `codexInstructions.ts` | System instructions injected into Codex requests (editing constraints, sandbox rules, approval policies). |
|
||||
| `defaultThinkingSignature.ts` | Default "thinking" signatures for Claude and Gemini models. |
|
||||
| `ollamaModels.ts` | Schema definition for local Ollama models (name, size, family, quantization). |
|
||||
|
||||
#### Credential Loading Flow
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
A["App starts"] --> B["constants.js defines PROVIDERS\nwith hardcoded defaults"]
|
||||
A["App starts"] --> B["constants.ts defines PROVIDERS\nwith hardcoded defaults"]
|
||||
B --> C{"data/provider-credentials.json\nexists?"}
|
||||
C -->|Yes| D["credentialLoader reads JSON"]
|
||||
C -->|No| E["Use hardcoded defaults"]
|
||||
@@ -194,15 +194,15 @@ classDiagram
|
||||
|
||||
| Executor | Provider | Key Specializations |
|
||||
| ---------------- | ------------------------------------------ | ------------------------------------------------------------------------------------------------------------------- |
|
||||
| `base.js` | — | Abstract base: URL building, headers, retry logic, credential refresh |
|
||||
| `default.js` | Claude, Gemini, OpenAI, GLM, Kimi, MiniMax | Generic OAuth token refresh for standard providers |
|
||||
| `antigravity.js` | Google Cloud Code | Project/session ID generation, multi-URL fallback, custom retry parsing from error messages ("reset after 2h7m23s") |
|
||||
| `cursor.js` | Cursor IDE | **Most complex**: SHA-256 checksum auth, Protobuf request encoding, binary EventStream → SSE response parsing |
|
||||
| `codex.js` | OpenAI Codex | Injects system instructions, manages thinking levels, removes unsupported parameters |
|
||||
| `gemini-cli.js` | Google Gemini CLI | Custom URL building (`streamGenerateContent`), Google OAuth token refresh |
|
||||
| `github.js` | GitHub Copilot | Dual token system (GitHub OAuth + Copilot token), VSCode header mimicking |
|
||||
| `kiro.js` | AWS CodeWhisperer | AWS EventStream binary parsing, AMZN event frames, token estimation |
|
||||
| `index.js` | — | Factory: maps provider name → executor class, with default fallback |
|
||||
| `base.ts` | — | Abstract base: URL building, headers, retry logic, credential refresh |
|
||||
| `default.ts` | Claude, Gemini, OpenAI, GLM, Kimi, MiniMax | Generic OAuth token refresh for standard providers |
|
||||
| `antigravity.ts` | Google Cloud Code | Project/session ID generation, multi-URL fallback, custom retry parsing from error messages ("reset after 2h7m23s") |
|
||||
| `cursor.ts` | Cursor IDE | **Most complex**: SHA-256 checksum auth, Protobuf request encoding, binary EventStream → SSE response parsing |
|
||||
| `codex.ts` | OpenAI Codex | Injects system instructions, manages thinking levels, removes unsupported parameters |
|
||||
| `gemini-cli.ts` | Google Gemini CLI | Custom URL building (`streamGenerateContent`), Google OAuth token refresh |
|
||||
| `github.ts` | GitHub Copilot | Dual token system (GitHub OAuth + Copilot token), VSCode header mimicking |
|
||||
| `kiro.ts` | AWS CodeWhisperer | AWS EventStream binary parsing, AMZN event frames, token estimation |
|
||||
| `index.ts` | — | Factory: maps provider name → executor class, with default fallback |
|
||||
|
||||
---
|
||||
|
||||
@@ -212,12 +212,12 @@ The **orchestration layer** — coordinates translation, execution, streaming, a
|
||||
|
||||
| File | Purpose |
|
||||
| --------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `chatCore.js` | **Central orchestrator** (~600 lines). Handles the complete request lifecycle: format detection → translation → executor dispatch → streaming/non-streaming response → token refresh → error handling → usage logging. |
|
||||
| `responsesHandler.js` | Adapter for OpenAI's Responses API: converts Responses format → Chat Completions → sends to `chatCore` → converts SSE back to Responses format. |
|
||||
| `embeddings.js` | Embedding generation handler: resolves embedding model → provider, dispatches to provider API, returns OpenAI-compatible embedding response. Supports 6+ providers. |
|
||||
| `imageGeneration.js` | Image generation handler: resolves image model → provider, supports OpenAI-compatible, Gemini-image (Antigravity), and fallback (Nebius) modes. Returns base64 or URL images. |
|
||||
| `chatCore.ts` | **Central orchestrator** (~600 lines). Handles the complete request lifecycle: format detection → translation → executor dispatch → streaming/non-streaming response → token refresh → error handling → usage logging. |
|
||||
| `responsesHandler.ts` | Adapter for OpenAI's Responses API: converts Responses format → Chat Completions → sends to `chatCore` → converts SSE back to Responses format. |
|
||||
| `embeddings.ts` | Embedding generation handler: resolves embedding model → provider, dispatches to provider API, returns OpenAI-compatible embedding response. Supports 6+ providers. |
|
||||
| `imageGeneration.ts` | Image generation handler: resolves image model → provider, supports OpenAI-compatible, Gemini-image (Antigravity), and fallback (Nebius) modes. Returns base64 or URL images. |
|
||||
|
||||
#### Request Lifecycle (chatCore.js)
|
||||
#### Request Lifecycle (chatCore.ts)
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
@@ -262,20 +262,20 @@ Business logic that supports the handlers and executors.
|
||||
|
||||
| File | Purpose |
|
||||
| -------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `provider.js` | **Format detection** (`detectFormat`): analyzes request body structure to identify Claude/OpenAI/Gemini/Antigravity/Responses formats (includes `max_tokens` heuristic for Claude). Also: URL building, header building, thinking config normalization. Supports `openai-compatible-*` and `anthropic-compatible-*` dynamic providers. |
|
||||
| `model.js` | Model string parsing (`claude/model-name` → `{provider: "claude", model: "model-name"}`), alias resolution with collision detection, input sanitization (rejects path traversal/control chars), and model info resolution with async alias getter support. |
|
||||
| `accountFallback.js` | Rate-limit handling: exponential backoff (1s → 2s → 4s → max 2min), account cooldown management, error classification (which errors trigger fallback vs. not). |
|
||||
| `tokenRefresh.js` | OAuth token refresh for **every provider**: Google (Gemini, Antigravity), Claude, Codex, Qwen, iFlow, GitHub (OAuth + Copilot dual-token), Kiro (AWS SSO OIDC + Social Auth). Includes in-flight promise deduplication cache and retry with exponential backoff. |
|
||||
| `combo.js` | **Combo models**: chains of fallback models. If model A fails with a fallback-eligible error, try model B, then C, etc. Returns actual upstream status codes. |
|
||||
| `usage.js` | Fetches quota/usage data from provider APIs (GitHub Copilot quotas, Antigravity model quotas, Codex rate limits, Kiro usage breakdowns, Claude settings). |
|
||||
| `accountSelector.js` | Smart account selection with scoring algorithm: considers priority, health status, round-robin position, and cooldown state to pick the optimal account for each request. |
|
||||
| `contextManager.js` | Request context lifecycle management: creates and tracks per-request context objects with metadata (request ID, timestamps, provider info) for debugging and logging. |
|
||||
| `ipFilter.js` | IP-based access control: supports allowlist and blocklist modes. Validates client IP against configured rules before processing API requests. |
|
||||
| `sessionManager.js` | Session tracking with client fingerprinting: tracks active sessions using hashed client identifiers, monitors request counts, and provides session metrics. |
|
||||
| `signatureCache.js` | Request signature-based deduplication cache: prevents duplicate requests by caching recent request signatures and returning cached responses for identical requests within a time window. |
|
||||
| `systemPrompt.js` | Global system prompt injection: prepends or appends a configurable system prompt to all requests, with per-provider compatibility handling. |
|
||||
| `thinkingBudget.js` | Reasoning token budget management: supports passthrough, auto (strip thinking config), custom (fixed budget), and adaptive (complexity-scaled) modes for controlling thinking/reasoning tokens. |
|
||||
| `wildcardRouter.js` | Wildcard model pattern routing: resolves wildcard patterns (e.g., `*/claude-*`) to concrete provider/model pairs based on availability and priority. |
|
||||
| `provider.ts` | **Format detection** (`detectFormat`): analyzes request body structure to identify Claude/OpenAI/Gemini/Antigravity/Responses formats (includes `max_tokens` heuristic for Claude). Also: URL building, header building, thinking config normalization. Supports `openai-compatible-*` and `anthropic-compatible-*` dynamic providers. |
|
||||
| `model.ts` | Model string parsing (`claude/model-name` → `{provider: "claude", model: "model-name"}`), alias resolution with collision detection, input sanitization (rejects path traversal/control chars), and model info resolution with async alias getter support. |
|
||||
| `accountFallback.ts` | Rate-limit handling: exponential backoff (1s → 2s → 4s → max 2min), account cooldown management, error classification (which errors trigger fallback vs. not). |
|
||||
| `tokenRefresh.ts` | OAuth token refresh for **every provider**: Google (Gemini, Antigravity), Claude, Codex, Qwen, iFlow, GitHub (OAuth + Copilot dual-token), Kiro (AWS SSO OIDC + Social Auth). Includes in-flight promise deduplication cache and retry with exponential backoff. |
|
||||
| `combo.ts` | **Combo models**: chains of fallback models. If model A fails with a fallback-eligible error, try model B, then C, etc. Returns actual upstream status codes. |
|
||||
| `usage.ts` | Fetches quota/usage data from provider APIs (GitHub Copilot quotas, Antigravity model quotas, Codex rate limits, Kiro usage breakdowns, Claude settings). |
|
||||
| `accountSelector.ts` | Smart account selection with scoring algorithm: considers priority, health status, round-robin position, and cooldown state to pick the optimal account for each request. |
|
||||
| `contextManager.ts` | Request context lifecycle management: creates and tracks per-request context objects with metadata (request ID, timestamps, provider info) for debugging and logging. |
|
||||
| `ipFilter.ts` | IP-based access control: supports allowlist and blocklist modes. Validates client IP against configured rules before processing API requests. |
|
||||
| `sessionManager.ts` | Session tracking with client fingerprinting: tracks active sessions using hashed client identifiers, monitors request counts, and provides session metrics. |
|
||||
| `signatureCache.ts` | Request signature-based deduplication cache: prevents duplicate requests by caching recent request signatures and returning cached responses for identical requests within a time window. |
|
||||
| `systemPrompt.ts` | Global system prompt injection: prepends or appends a configurable system prompt to all requests, with per-provider compatibility handling. |
|
||||
| `thinkingBudget.ts` | Reasoning token budget management: supports passthrough, auto (strip thinking config), custom (fixed budget), and adaptive (complexity-scaled) modes for controlling thinking/reasoning tokens. |
|
||||
| `wildcardRouter.ts` | Wildcard model pattern routing: resolves wildcard patterns (e.g., `*/claude-*`) to concrete provider/model pairs based on availability and priority. |
|
||||
|
||||
#### Token Refresh Deduplication
|
||||
|
||||
@@ -377,8 +377,8 @@ graph TD
|
||||
| `request/` | 8 translators | Convert request bodies between formats. Each file self-registers via `register(from, to, fn)` on import. |
|
||||
| `response/` | 7 translators | Convert streaming response chunks between formats. Handles SSE event types, thinking blocks, tool calls. |
|
||||
| `helpers/` | 6 helpers | Shared utilities: `claudeHelper` (system prompt extraction, thinking config), `geminiHelper` (parts/contents mapping), `openaiHelper` (format filtering), `toolCallHelper` (ID generation, missing response injection), `maxTokensHelper`, `responsesApiHelper`. |
|
||||
| `index.js` | — | Translation engine: `translateRequest()`, `translateResponse()`, state management, registry. |
|
||||
| `formats.js` | — | Format constants: `OPENAI`, `CLAUDE`, `GEMINI`, `ANTIGRAVITY`, `KIRO`, `CURSOR`, `OPENAI_RESPONSES`. |
|
||||
| `index.ts` | — | Translation engine: `translateRequest()`, `translateResponse()`, state management, registry. |
|
||||
| `formats.ts` | — | Format constants: `OPENAI`, `CLAUDE`, `GEMINI`, `ANTIGRAVITY`, `KIRO`, `CURSOR`, `OPENAI_RESPONSES`. |
|
||||
|
||||
#### Key Design: Self-Registering Plugins
|
||||
|
||||
@@ -397,13 +397,13 @@ import "./request/claude-to-openai.js"; // ← self-registers
|
||||
|
||||
| File | Purpose |
|
||||
| ------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
|
||||
| `error.js` | Error response building (OpenAI-compatible format), upstream error parsing, Antigravity retry-time extraction from error messages, SSE error streaming. |
|
||||
| `stream.js` | **SSE Transform Stream** — the core streaming pipeline. Two modes: `TRANSLATE` (full format translation) and `PASSTHROUGH` (normalize + extract usage). Handles chunk buffering, usage estimation, content length tracking. Per-stream encoder/decoder instances avoid shared state. |
|
||||
| `streamHelpers.js` | Low-level SSE utilities: `parseSSELine` (whitespace-tolerant), `hasValuableContent` (filters empty chunks for OpenAI/Claude/Gemini), `fixInvalidId`, `formatSSE` (format-aware SSE serialization with `perf_metrics` cleanup). |
|
||||
| `usageTracking.js` | Token usage extraction from any format (Claude/OpenAI/Gemini/Responses), estimation with separate tool/message char-per-token ratios, buffer addition (2000 tokens safety margin), format-specific field filtering, console logging with ANSI colors. |
|
||||
| `requestLogger.js` | File-based request logging (opt-in via `ENABLE_REQUEST_LOGS=true`). Creates session folders with numbered files: `1_req_client.json` → `7_res_client.txt`. All I/O is async (fire-and-forget). Masks sensitive headers. |
|
||||
| `bypassHandler.js` | Intercepts specific patterns from Claude CLI (title extraction, warmup, count) and returns fake responses without calling any provider. Supports both streaming and non-streaming. Intentionally limited to Claude CLI scope. |
|
||||
| `networkProxy.js` | Resolves outbound proxy URL for a given provider with precedence: provider-specific config → global config → environment variables (`HTTPS_PROXY`/`HTTP_PROXY`/`ALL_PROXY`). Supports `NO_PROXY` exclusions. Caches config for 30s. |
|
||||
| `error.ts` | Error response building (OpenAI-compatible format), upstream error parsing, Antigravity retry-time extraction from error messages, SSE error streaming. |
|
||||
| `stream.ts` | **SSE Transform Stream** — the core streaming pipeline. Two modes: `TRANSLATE` (full format translation) and `PASSTHROUGH` (normalize + extract usage). Handles chunk buffering, usage estimation, content length tracking. Per-stream encoder/decoder instances avoid shared state. |
|
||||
| `streamHelpers.ts` | Low-level SSE utilities: `parseSSELine` (whitespace-tolerant), `hasValuableContent` (filters empty chunks for OpenAI/Claude/Gemini), `fixInvalidId`, `formatSSE` (format-aware SSE serialization with `perf_metrics` cleanup). |
|
||||
| `usageTracking.ts` | Token usage extraction from any format (Claude/OpenAI/Gemini/Responses), estimation with separate tool/message char-per-token ratios, buffer addition (2000 tokens safety margin), format-specific field filtering, console logging with ANSI colors. |
|
||||
| `requestLogger.ts` | File-based request logging (opt-in via `ENABLE_REQUEST_LOGS=true`). Creates session folders with numbered files: `1_req_client.json` → `7_res_client.txt`. All I/O is async (fire-and-forget). Masks sensitive headers. |
|
||||
| `bypassHandler.ts` | Intercepts specific patterns from Claude CLI (title extraction, warmup, count) and returns fake responses without calling any provider. Supports both streaming and non-streaming. Intentionally limited to Claude CLI scope. |
|
||||
| `networkProxy.ts` | Resolves outbound proxy URL for a given provider with precedence: provider-specific config → global config → environment variables (`HTTPS_PROXY`/`HTTP_PROXY`/`ALL_PROXY`). Supports `NO_PROXY` exclusions. Caches config for 30s. |
|
||||
|
||||
#### SSE Streaming Pipeline
|
||||
|
||||
@@ -484,7 +484,7 @@ All formats translate through **OpenAI format as the hub**. Adding a new provide
|
||||
|
||||
### 5.2 Executor Strategy Pattern
|
||||
|
||||
Each provider has a dedicated executor class inheriting from `BaseExecutor`. The factory in `executors/index.js` selects the right one at runtime.
|
||||
Each provider has a dedicated executor class inheriting from `BaseExecutor`. The factory in `executors/index.ts` selects the right one at runtime.
|
||||
|
||||
### 5.3 Self-Registering Plugin System
|
||||
|
||||
|
||||
Reference in New Issue
Block a user