From 7296a6461ae06f1c8b5a22c08fbf7bc5e879d6f8 Mon Sep 17 00:00:00 2001 From: diegosouzapw Date: Tue, 17 Feb 2026 08:47:58 -0300 Subject: [PATCH] docs: comprehensive v0.8.5 documentation update MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - README: expand features table from 16 to 40+ entries across 5 categories (Core Routing, Multi-Modal APIs, Resilience, Observability, Deployment) - README: update tech stack to reflect 100% TypeScript codebase - README: update Docker tag 0.6.0 โ†’ 0.8.5, release command to v0.8.5 - CHANGELOG: add detailed v0.8.5 section (40 commits categorized) covering TLS spoofing, SQLite logs, unified logging, full TS migration, Qwen fix, VPS compatibility, CI improvements, dependency bumps - ARCHITECTURE: update all open-sse file references .js โ†’ .ts - CODEBASE_DOCUMENTATION: update all open-sse file references .js โ†’ .ts --- CHANGELOG.md | 49 ++++++++++++++++++ README.md | 90 ++++++++++++++++++++++++---------- docs/ARCHITECTURE.md | 70 +++++++++++++------------- docs/CODEBASE_DOCUMENTATION.md | 90 +++++++++++++++++----------------- 4 files changed, 193 insertions(+), 106 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 881ca5d01b..cbd0ea0da5 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,54 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 --- +## [0.8.5] โ€” 2026-02-17 + +### Added + +- ๐Ÿ”’ **TLS fingerprint spoofing** โ€” Implement browser-like TLS fingerprinting via `wreq-js` to bypass bot detection on providers that enforce TLS client fingerprint checks (`3dd0cc1`, PR #52) +- ๐Ÿ’พ **SQLite proxy log persistence** โ€” Proxy request/response logs now persist to SQLite database, surviving server restarts. Previously, logs were lost on restart (`f1664fe`, PR #53) +- ๐Ÿ“‹ **Unified test logging** โ€” Shared `Logger` + Proxy logging infrastructure for all provider connection test flows. Consistent log formatting across batch and individual tests (`bce302e`, PR #55) + +### Refactored + +- ๐Ÿ”ท **Full TypeScript migration โ€” `src/`** โ€” Migrated the entire `src/` directory from JavaScript to TypeScript. All `.js`/`.jsx` files converted to `.ts`/`.tsx` with proper type annotations across API routes, lib modules, components, services, stores, domain layer, and shared utilities (`d0ca595`) + - **Wave 1**: Shared component interfaces + EventTarget fixes (`dfdd2a2`) + - **Wave 2**: Utils & services typed fields, Zustand stores, logger, sync scheduler (`89dd107`, `b2907cd`) + - **Wave 3a**: Lib layer, DB, compliance, domain layer typed (`9e13fe2`) + - **Wave 3b**: Usage, CLI runtime, SSE auth/logger typed (`a291abd`) + - **Wave 3c**: OAuth services + server utils typed (`d62cf8d`) + - **Wave 4a**: 7 API routes โ€” providers, cli-tools, oauth (`7cdb923`) + - **Wave 4b**: 7 more API routes โ€” providers, test, usage, nodes (`5592c2e`) + - **Wave 4c**: 8 files โ€” components, SSE handlers, services (`d8ce9dc`) + - **Dashboard hardening**: Resolve all TypeScript errors across dashboard pages (`7a463a3`, PR #61) +- ๐Ÿ”ท **Full TypeScript migration โ€” `open-sse/`** โ€” Migrated all 94 `.js` files in the SSE routing engine to TypeScript (PR #62) + - **Phase 1**: Rename all 94 `.js` โ†’ `.ts` files (`256e443`) + - **Phase 6**: Reduce `@ts-ignore` from 231 โ†’ 186 with targeted fixes (`6a54b84`) + - **Phase 7**: Eliminate ALL `@ts-ignore` annotations (186 โ†’ 0) and ALL TypeScript errors (237 โ†’ 0) โ€” zero `@ts-ignore`, zero errors (`7b37a3c`) + - Typing strategies: `Record` for dynamic objects, optional function params, `as any` casts for custom Error/Array properties, `declare var EdgeRuntime` for edge compatibility, proper `fs`/`path` imports + +### Fixed + +- ๐Ÿ› **Qwen token refresh** โ€” Detect `invalid_request` as unrecoverable error and switch broken test endpoints to `checkExpiry` method instead of failing silently (`1e0ffbc`, PR #60) +- ๐Ÿ› **VPS batch test compatibility** โ€” Eliminate HTTP self-calls in batch provider connection tests for VPS environments where localhost is unreachable (`a3bbbb5`, PR #54) +- ๐Ÿ› **E2E test assertions** โ€” Correct API endpoints and response format assertions in end-to-end tests (`92b5e66`) +- ๐Ÿ› **CI coverage thresholds** โ€” Lower coverage thresholds, use production server for E2E, block ESLint major upgrades from breaking CI (`3ca4b6b`, PR #51) + +### Changed + +- ๐Ÿ“– **Documentation update** โ€” Updated all documentation to reflect JS โ†’ TS migration, corrected file extensions and import paths (`7ff8aa2`) +- โฌ†๏ธ **CI/CD** โ€” Bump `actions/checkout` v4 โ†’ v6, `actions/setup-node` v4 โ†’ v6, `peter-evans/dockerhub-description` v4 โ†’ v5 + +### Dependencies + +- โฌ†๏ธ `undici` 7.21.0 โ†’ 7.22.0 (production) +- โฌ†๏ธ `actions/checkout` 4 โ†’ 6 +- โฌ†๏ธ `actions/setup-node` 4 โ†’ 6 +- โฌ†๏ธ `peter-evans/dockerhub-description` 4 โ†’ 5 +- ๐Ÿšซ `eslint` 10.0.0 blocked โ€” major version incompatible with `eslint-config-next` + +--- + ## [0.8.0] โ€” 2026-02-16 ### Added @@ -143,6 +191,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 --- +[0.8.5]: https://github.com/diegosouzapw/OmniRoute/compare/v0.8.0...v0.8.5 [0.8.0]: https://github.com/diegosouzapw/OmniRoute/compare/v0.7.0...v0.8.0 [0.7.0]: https://github.com/diegosouzapw/OmniRoute/compare/v0.6.0...v0.7.0 [0.6.0]: https://github.com/diegosouzapw/OmniRoute/compare/v0.5.0...v0.6.0 diff --git a/README.md b/README.md index aa5f48e50c..5969487cef 100644 --- a/README.md +++ b/README.md @@ -5,7 +5,7 @@ **Never stop coding. Auto-route to FREE & cheap AI models with smart fallback.** - **36+ Providers โ€ข Embeddings โ€ข Image Generation โ€ข Think Tag Parsing** + **36+ Providers โ€ข Embeddings โ€ข Image Generation โ€ข Audio โ€ข Reranking โ€ข Full TypeScript** **Free AI Provider for OpenClaw.** @@ -158,31 +158,69 @@ docker compose --profile cli up -d | Image | Tag | Size | Description | | ------------------------ | -------- | ------ | --------------------- | | `diegosouzapw/omniroute` | `latest` | ~250MB | Latest stable release | -| `diegosouzapw/omniroute` | `0.6.0` | ~250MB | Current version | +| `diegosouzapw/omniroute` | `0.8.5` | ~250MB | Current version | --- ## ๐Ÿ’ก Key Features -| Feature | What It Does | -| ------------------------------- | --------------------------------------------- | -| ๐ŸŽฏ **Smart 3-Tier Fallback** | Auto-route: Subscription โ†’ Cheap โ†’ Free | -| ๐Ÿ“Š **Real-Time Quota Tracking** | Live token count + reset countdown | -| ๐Ÿ”„ **Format Translation** | OpenAI โ†” Claude โ†” Gemini seamless | -| ๐Ÿ‘ฅ **Multi-Account Support** | Multiple accounts per provider | -| ๐Ÿ”„ **Auto Token Refresh** | OAuth tokens refresh automatically | -| ๐ŸŽจ **Custom Combos** | Create unlimited model combinations | -| ๐Ÿงฉ **Custom Models** | Add any model ID to any provider | -| ๐Ÿ“ **Request Logging** | Debug mode with full request/response logs | -| ๐Ÿ’พ **Cloud Sync** | Sync config across devices | -| ๐Ÿ“Š **Usage Analytics** | Track tokens, cost, trends over time | -| ๐ŸŒ **Deploy Anywhere** | Localhost, VPS, Docker, Cloudflare Workers | -| ๐Ÿ”Œ **Circuit Breaker** | Auto-open/close per-provider with cooldowns | -| ๐Ÿ›ก๏ธ **Anti-Thundering Herd** | Mutex + auto rate-limit for API key providers | -| ๐Ÿง  **Semantic Cache** | Two-tier cache reduces cost & latency | -| โšก **Request Idempotency** | 5s dedup window for duplicate requests | -| ๐Ÿ“ˆ **Progress Tracking** | Opt-in SSE progress events for streaming | -| ๐Ÿงช **LLM Evaluations** | Golden set testing with 4 match strategies | +### ๐Ÿง  Core Routing & Intelligence + +| Feature | What It Does | +| ------------------------------- | --------------------------------------------------------------------------------- | +| ๐ŸŽฏ **Smart 4-Tier Fallback** | Auto-route: Subscription โ†’ API Key โ†’ Cheap โ†’ Free | +| ๐Ÿ“Š **Real-Time Quota Tracking** | Live token count + reset countdown per provider | +| ๐Ÿ”„ **Format Translation** | OpenAI โ†” Claude โ†” Gemini โ†” Cursor โ†” Kiro seamless | +| ๐Ÿ‘ฅ **Multi-Account Support** | Multiple accounts per provider with P2C selection | +| ๐Ÿ”„ **Auto Token Refresh** | OAuth tokens refresh automatically with retry | +| ๐ŸŽจ **Custom Combos** | 6 strategies: priority, weighted, round-robin, random, least-used, cost-optimized | +| ๐Ÿงฉ **Custom Models** | Add any model ID to any provider | +| ๐ŸŒ **Wildcard Router** | Route `provider/*` patterns to any provider dynamically | +| ๐Ÿง  **Thinking Budget** | Passthrough, auto, custom, and adaptive modes for reasoning models | +| ๐Ÿ’ฌ **System Prompt Injection** | Global system prompt applied across all requests | +| ๐Ÿ“„ **Responses API** | Full OpenAI Responses API (`/v1/responses`) support for Codex | + +### ๐ŸŽต Multi-Modal APIs + +| Feature | What It Does | +| -------------------------- | --------------------------------------------------- | +| ๐Ÿ–ผ๏ธ **Image Generation** | `/v1/images/generations` โ€” 4 providers, 9+ models | +| ๐Ÿ“ **Embeddings** | `/v1/embeddings` โ€” 6 providers, 9+ models | +| ๐ŸŽค **Audio Transcription** | `/v1/audio/transcriptions` โ€” Whisper-compatible | +| ๐Ÿ”Š **Text-to-Speech** | `/v1/audio/speech` โ€” Multi-provider audio synthesis | +| ๐Ÿ›ก๏ธ **Moderations** | `/v1/moderations` โ€” Content safety checks | +| ๐Ÿ”€ **Reranking** | `/v1/rerank` โ€” Document relevance reranking | + +### ๐Ÿ›ก๏ธ Resilience & Security + +| Feature | What It Does | +| ------------------------------- | ------------------------------------------------------------ | +| ๐Ÿ”Œ **Circuit Breaker** | Auto-open/close per-provider with configurable thresholds | +| ๐Ÿ›ก๏ธ **Anti-Thundering Herd** | Mutex + semaphore rate-limit for API key providers | +| ๐Ÿง  **Semantic Cache** | Two-tier cache (signature + semantic) reduces cost & latency | +| โšก **Request Idempotency** | 5s dedup window for duplicate requests | +| ๐Ÿ”’ **TLS Fingerprint Spoofing** | Bypass TLS-based bot detection via wreq-js | +| ๐ŸŒ **IP Filtering** | Allowlist/blocklist for API access control | +| ๐Ÿ“‹ **Compliance Audit Log** | Tamper-proof request logs with opt-out per API key | + +### ๐Ÿ“Š Observability & Analytics + +| Feature | What It Does | +| ------------------------ | ------------------------------------------------------ | +| ๐Ÿ“ **Request Logging** | Debug mode with full request/response logs | +| ๐Ÿ’พ **SQLite Proxy Logs** | Persistent proxy logs survive server restarts | +| ๐Ÿ“Š **Usage Analytics** | Track tokens, cost, trends over time | +| ๐Ÿ“ˆ **Progress Tracking** | Opt-in SSE progress events for streaming | +| ๐Ÿงช **LLM Evaluations** | Golden set testing with 4 match strategies | +| ๐Ÿ” **Request Telemetry** | p50/p95/p99 latency aggregation + X-Request-Id tracing | + +### โ˜๏ธ Deployment & Sync + +| Feature | What It Does | +| ------------------------- | ------------------------------------------------- | +| ๐Ÿ’พ **Cloud Sync** | Sync config across devices via Cloudflare Workers | +| ๐ŸŒ **Deploy Anywhere** | Localhost, VPS, Docker, Cloudflare Workers | +| ๐Ÿ”‘ **API Key Management** | Generate, rotate, and scope API keys per provider | --- @@ -254,17 +292,17 @@ registerSuite({ ## ๐Ÿ› ๏ธ Tech Stack - **Runtime**: Node.js 20+ -- **Language**: TypeScript 5.9 (src/) + JavaScript (open-sse/) +- **Language**: TypeScript 5.9 โ€” **100% TypeScript** across `src/` and `open-sse/` (v0.8.5) - **Framework**: Next.js 16 + React 19 + Tailwind CSS 4 -- **Database**: LowDB (JSON) + SQLite (domain state) +- **Database**: LowDB (JSON) + SQLite (domain state + proxy logs) - **Streaming**: Server-Sent Events (SSE) - **Auth**: OAuth 2.0 (PKCE) + JWT + API Keys - **Testing**: Node.js test runner (368+ unit tests) -- **CI/CD**: GitHub Actions (auto npm publish on release) +- **CI/CD**: GitHub Actions (auto npm publish + Docker Hub on release) - **Website**: [omniroute.online](https://omniroute.online) - **Package**: [npmjs.com/package/omniroute](https://www.npmjs.com/package/omniroute) - **Docker**: [hub.docker.com/r/diegosouzapw/omniroute](https://hub.docker.com/r/diegosouzapw/omniroute) -- **Resilience**: Circuit breaker, exponential backoff, anti-thundering herd +- **Resilience**: Circuit breaker, exponential backoff, anti-thundering herd, TLS spoofing --- @@ -309,7 +347,7 @@ See [CONTRIBUTING.md](CONTRIBUTING.md) for detailed guidelines. ```bash # Create a release โ€” npm publish happens automatically -gh release create v0.8.0 --title "v0.8.0" --generate-notes +gh release create v0.8.5 --title "v0.8.5" --generate-notes ``` --- diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index 8ff873e753..29f4c3dcfa 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -167,32 +167,32 @@ Management domains: Main flow modules: - Entry: `src/sse/handlers/chat.ts` -- Core orchestration: `open-sse/handlers/chatCore.js` +- Core orchestration: `open-sse/handlers/chatCore.ts` - Provider execution adapters: `open-sse/executors/*` -- Format detection/provider config: `open-sse/services/provider.js` -- Model parse/resolve: `src/sse/services/model.ts`, `open-sse/services/model.js` -- Account fallback logic: `open-sse/services/accountFallback.js` -- Translation registry: `open-sse/translator/index.js` -- Stream transformations: `open-sse/utils/stream.js`, `open-sse/utils/streamHandler.js` -- Usage extraction/normalization: `open-sse/utils/usageTracking.js` -- Think tag parser: `open-sse/utils/thinkTagParser.js` -- Embedding handler: `open-sse/handlers/embeddings.js` -- Embedding provider registry: `open-sse/config/embeddingRegistry.js` -- Image generation handler: `open-sse/handlers/imageGeneration.js` -- Image provider registry: `open-sse/config/imageRegistry.js` +- Format detection/provider config: `open-sse/services/provider.ts` +- Model parse/resolve: `src/sse/services/model.ts`, `open-sse/services/model.ts` +- Account fallback logic: `open-sse/services/accountFallback.ts` +- Translation registry: `open-sse/translator/index.ts` +- Stream transformations: `open-sse/utils/stream.ts`, `open-sse/utils/streamHandler.ts` +- Usage extraction/normalization: `open-sse/utils/usageTracking.ts` +- Think tag parser: `open-sse/utils/thinkTagParser.ts` +- Embedding handler: `open-sse/handlers/embeddings.ts` +- Embedding provider registry: `open-sse/config/embeddingRegistry.ts` +- Image generation handler: `open-sse/handlers/imageGeneration.ts` +- Image provider registry: `open-sse/config/imageRegistry.ts` Services (business logic): -- Account selection/scoring: `open-sse/services/accountSelector.js` -- Context lifecycle management: `open-sse/services/contextManager.js` -- IP filter enforcement: `open-sse/services/ipFilter.js` -- Session tracking: `open-sse/services/sessionManager.js` -- Request deduplication: `open-sse/services/signatureCache.js` -- System prompt injection: `open-sse/services/systemPrompt.js` -- Thinking budget management: `open-sse/services/thinkingBudget.js` -- Wildcard model routing: `open-sse/services/wildcardRouter.js` -- Rate limit management: `open-sse/services/rateLimitManager.js` -- Circuit breaker: `open-sse/services/circuitBreaker.js` +- Account selection/scoring: `open-sse/services/accountSelector.ts` +- Context lifecycle management: `open-sse/services/contextManager.ts` +- IP filter enforcement: `open-sse/services/ipFilter.ts` +- Session tracking: `open-sse/services/sessionManager.ts` +- Request deduplication: `open-sse/services/signatureCache.ts` +- System prompt injection: `open-sse/services/systemPrompt.ts` +- Thinking budget management: `open-sse/services/thinkingBudget.ts` +- Wildcard model routing: `open-sse/services/wildcardRouter.ts` +- Rate limit management: `open-sse/services/rateLimitManager.ts` +- Circuit breaker: `open-sse/services/circuitBreaker.ts` Domain layer modules: @@ -242,7 +242,7 @@ Domain State DB (SQLite): - Dashboard cookie auth: `src/proxy.ts`, `src/app/api/auth/login/route.ts` - API key generation/verification: `src/shared/utils/apiKey.ts` - Provider secrets persisted in `providerConnections` entries -- Outbound proxy support via `open-sse/utils/proxyFetch.js` (env vars) and `open-sse/utils/networkProxy.js` (configurable per-provider or global) +- Outbound proxy support via `open-sse/utils/proxyFetch.ts` (env vars) and `open-sse/utils/networkProxy.ts` (configurable per-provider or global) ## 5) Cloud Sync @@ -327,7 +327,7 @@ flowchart TD Q -- No --> R[Return all unavailable] ``` -Fallback decisions are driven by `open-sse/services/accountFallback.js` using status codes and error-message heuristics. +Fallback decisions are driven by `open-sse/services/accountFallback.ts` using status codes and error-message heuristics. ## OAuth Onboarding and Token Refresh Lifecycle @@ -359,7 +359,7 @@ sequenceDiagram Test-->>UI: validation result ``` -Refresh during live traffic is executed inside `open-sse/handlers/chatCore.js` via executor `refreshCredentials()`. +Refresh during live traffic is executed inside `open-sse/handlers/chatCore.ts` via executor `refreshCredentials()`. ## Cloud Sync Lifecycle (Enable / Sync / Disable) @@ -560,15 +560,15 @@ flowchart LR ### Routing and Execution Core - `src/sse/handlers/chat.ts`: request parse, combo handling, account selection loop -- `open-sse/handlers/chatCore.js`: translation, executor dispatch, retry/refresh handling, stream setup +- `open-sse/handlers/chatCore.ts`: translation, executor dispatch, retry/refresh handling, stream setup - `open-sse/executors/*`: provider-specific network and format behavior ### Translation Registry and Format Converters -- `open-sse/translator/index.js`: translator registry and orchestration +- `open-sse/translator/index.ts`: translator registry and orchestration - Request translators: `open-sse/translator/request/*` - Response translators: `open-sse/translator/response/*` -- Format constants: `open-sse/translator/formats.js` +- Format constants: `open-sse/translator/formats.ts` ### Persistence @@ -577,7 +577,7 @@ flowchart LR ## Provider Executor Coverage (Strategy Pattern) -Each provider has a specialized executor extending `BaseExecutor` (in `open-sse/executors/base.js`), which provides URL building, header construction, retry with exponential backoff, credential refresh hooks, and the `execute()` orchestration method. +Each provider has a specialized executor extending `BaseExecutor` (in `open-sse/executors/base.ts`), which provides URL building, header construction, retry with exponential backoff, credential refresh hooks, and the `execute()` orchestration method. | Executor | Provider(s) | Special Handling | | --------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------ | -------------------------------------------------------------------- | @@ -650,10 +650,10 @@ Translations are selected dynamically based on source payload shape and provider | -------------------------------------------------- | ------------------ | ---------------------------------------------------- | | `POST /v1/chat/completions` | OpenAI Chat | `src/sse/handlers/chat.ts` | | `POST /v1/messages` | Claude Messages | Same handler (auto-detected) | -| `POST /v1/responses` | OpenAI Responses | `open-sse/handlers/responsesHandler.js` | -| `POST /v1/embeddings` | OpenAI Embeddings | `open-sse/handlers/embeddings.js` | +| `POST /v1/responses` | OpenAI Responses | `open-sse/handlers/responsesHandler.ts` | +| `POST /v1/embeddings` | OpenAI Embeddings | `open-sse/handlers/embeddings.ts` | | `GET /v1/embeddings` | Model listing | API route | -| `POST /v1/images/generations` | OpenAI Images | `open-sse/handlers/imageGeneration.js` | +| `POST /v1/images/generations` | OpenAI Images | `open-sse/handlers/imageGeneration.ts` | | `GET /v1/images/generations` | Model listing | API route | | `POST /v1/providers/{provider}/chat/completions` | OpenAI Chat | Dedicated per-provider with model validation | | `POST /v1/providers/{provider}/embeddings` | OpenAI Embeddings | Dedicated per-provider with model validation | @@ -668,11 +668,11 @@ Translations are selected dynamically based on source payload shape and provider ## Bypass Handler -The bypass handler (`open-sse/utils/bypassHandler.js`) intercepts known "throwaway" requests from Claude CLI โ€” warmup pings, title extractions, and token counts โ€” and returns a **fake response** without consuming upstream provider tokens. This is triggered only when `User-Agent` contains `claude-cli`. +The bypass handler (`open-sse/utils/bypassHandler.ts`) intercepts known "throwaway" requests from Claude CLI โ€” warmup pings, title extractions, and token counts โ€” and returns a **fake response** without consuming upstream provider tokens. This is triggered only when `User-Agent` contains `claude-cli`. ## Request Logger Pipeline -The request logger (`open-sse/utils/requestLogger.js`) provides a 7-stage debug logging pipeline, disabled by default, enabled via `ENABLE_REQUEST_LOGS=true`: +The request logger (`open-sse/utils/requestLogger.ts`) provides a 7-stage debug logging pipeline, disabled by default, enabled via `ENABLE_REQUEST_LOGS=true`: ``` 1_req_client.json โ†’ 2_req_source.json โ†’ 3_req_openai.json โ†’ 4_req_target.json @@ -746,7 +746,7 @@ Environment variables actively used by code: ## Known Architectural Notes 1. `usageDb` and `localDb` now share the same base directory policy (`DATA_DIR` -> `XDG_CONFIG_HOME/omniroute` -> `~/.omniroute`) with legacy file migration. -2. `/api/v1/route.js` returns a static model list and is not the main models source used by `/v1/models`. +2. `/api/v1/route.ts` returns a static model list and is not the main models source used by `/v1/models`. 3. Request logger writes full headers/body when enabled; treat log directory as sensitive. 4. Cloud behavior depends on correct `NEXT_PUBLIC_BASE_URL` and cloud endpoint reachability. 5. The `open-sse/` directory is published as the `@omniroute/open-sse` **npm workspace package**. Source code imports it via `@omniroute/open-sse/...` (resolved by Next.js `transpilePackages`). File paths in this document still use the directory name `open-sse/` for consistency. diff --git a/docs/CODEBASE_DOCUMENTATION.md b/docs/CODEBASE_DOCUMENTATION.md index aac7f45767..42f8ff60f7 100644 --- a/docs/CODEBASE_DOCUMENTATION.md +++ b/docs/CODEBASE_DOCUMENTATION.md @@ -110,18 +110,18 @@ The **single source of truth** for all provider configuration. | File | Purpose | | ----------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `constants.js` | `PROVIDERS` object with base URLs, OAuth credentials (defaults), headers, and default system prompts for every provider. Also defines `HTTP_STATUS`, `ERROR_TYPES`, `COOLDOWN_MS`, `BACKOFF_CONFIG`, and `SKIP_PATTERNS`. | -| `credentialLoader.js` | Loads external credentials from `data/provider-credentials.json` and merges them over the hardcoded defaults in `PROVIDERS`. Keeps secrets out of source control while maintaining backwards compatibility. | -| `providerModels.js` | Central model registry: maps provider aliases โ†’ model IDs. Functions like `getModels()`, `getProviderByAlias()`. | -| `codexInstructions.js` | System instructions injected into Codex requests (editing constraints, sandbox rules, approval policies). | -| `defaultThinkingSignature.js` | Default "thinking" signatures for Claude and Gemini models. | -| `ollamaModels.js` | Schema definition for local Ollama models (name, size, family, quantization). | +| `constants.ts` | `PROVIDERS` object with base URLs, OAuth credentials (defaults), headers, and default system prompts for every provider. Also defines `HTTP_STATUS`, `ERROR_TYPES`, `COOLDOWN_MS`, `BACKOFF_CONFIG`, and `SKIP_PATTERNS`. | +| `credentialLoader.ts` | Loads external credentials from `data/provider-credentials.json` and merges them over the hardcoded defaults in `PROVIDERS`. Keeps secrets out of source control while maintaining backwards compatibility. | +| `providerModels.ts` | Central model registry: maps provider aliases โ†’ model IDs. Functions like `getModels()`, `getProviderByAlias()`. | +| `codexInstructions.ts` | System instructions injected into Codex requests (editing constraints, sandbox rules, approval policies). | +| `defaultThinkingSignature.ts` | Default "thinking" signatures for Claude and Gemini models. | +| `ollamaModels.ts` | Schema definition for local Ollama models (name, size, family, quantization). | #### Credential Loading Flow ```mermaid flowchart TD - A["App starts"] --> B["constants.js defines PROVIDERS\nwith hardcoded defaults"] + A["App starts"] --> B["constants.ts defines PROVIDERS\nwith hardcoded defaults"] B --> C{"data/provider-credentials.json\nexists?"} C -->|Yes| D["credentialLoader reads JSON"] C -->|No| E["Use hardcoded defaults"] @@ -194,15 +194,15 @@ classDiagram | Executor | Provider | Key Specializations | | ---------------- | ------------------------------------------ | ------------------------------------------------------------------------------------------------------------------- | -| `base.js` | โ€” | Abstract base: URL building, headers, retry logic, credential refresh | -| `default.js` | Claude, Gemini, OpenAI, GLM, Kimi, MiniMax | Generic OAuth token refresh for standard providers | -| `antigravity.js` | Google Cloud Code | Project/session ID generation, multi-URL fallback, custom retry parsing from error messages ("reset after 2h7m23s") | -| `cursor.js` | Cursor IDE | **Most complex**: SHA-256 checksum auth, Protobuf request encoding, binary EventStream โ†’ SSE response parsing | -| `codex.js` | OpenAI Codex | Injects system instructions, manages thinking levels, removes unsupported parameters | -| `gemini-cli.js` | Google Gemini CLI | Custom URL building (`streamGenerateContent`), Google OAuth token refresh | -| `github.js` | GitHub Copilot | Dual token system (GitHub OAuth + Copilot token), VSCode header mimicking | -| `kiro.js` | AWS CodeWhisperer | AWS EventStream binary parsing, AMZN event frames, token estimation | -| `index.js` | โ€” | Factory: maps provider name โ†’ executor class, with default fallback | +| `base.ts` | โ€” | Abstract base: URL building, headers, retry logic, credential refresh | +| `default.ts` | Claude, Gemini, OpenAI, GLM, Kimi, MiniMax | Generic OAuth token refresh for standard providers | +| `antigravity.ts` | Google Cloud Code | Project/session ID generation, multi-URL fallback, custom retry parsing from error messages ("reset after 2h7m23s") | +| `cursor.ts` | Cursor IDE | **Most complex**: SHA-256 checksum auth, Protobuf request encoding, binary EventStream โ†’ SSE response parsing | +| `codex.ts` | OpenAI Codex | Injects system instructions, manages thinking levels, removes unsupported parameters | +| `gemini-cli.ts` | Google Gemini CLI | Custom URL building (`streamGenerateContent`), Google OAuth token refresh | +| `github.ts` | GitHub Copilot | Dual token system (GitHub OAuth + Copilot token), VSCode header mimicking | +| `kiro.ts` | AWS CodeWhisperer | AWS EventStream binary parsing, AMZN event frames, token estimation | +| `index.ts` | โ€” | Factory: maps provider name โ†’ executor class, with default fallback | --- @@ -212,12 +212,12 @@ The **orchestration layer** โ€” coordinates translation, execution, streaming, a | File | Purpose | | --------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `chatCore.js` | **Central orchestrator** (~600 lines). Handles the complete request lifecycle: format detection โ†’ translation โ†’ executor dispatch โ†’ streaming/non-streaming response โ†’ token refresh โ†’ error handling โ†’ usage logging. | -| `responsesHandler.js` | Adapter for OpenAI's Responses API: converts Responses format โ†’ Chat Completions โ†’ sends to `chatCore` โ†’ converts SSE back to Responses format. | -| `embeddings.js` | Embedding generation handler: resolves embedding model โ†’ provider, dispatches to provider API, returns OpenAI-compatible embedding response. Supports 6+ providers. | -| `imageGeneration.js` | Image generation handler: resolves image model โ†’ provider, supports OpenAI-compatible, Gemini-image (Antigravity), and fallback (Nebius) modes. Returns base64 or URL images. | +| `chatCore.ts` | **Central orchestrator** (~600 lines). Handles the complete request lifecycle: format detection โ†’ translation โ†’ executor dispatch โ†’ streaming/non-streaming response โ†’ token refresh โ†’ error handling โ†’ usage logging. | +| `responsesHandler.ts` | Adapter for OpenAI's Responses API: converts Responses format โ†’ Chat Completions โ†’ sends to `chatCore` โ†’ converts SSE back to Responses format. | +| `embeddings.ts` | Embedding generation handler: resolves embedding model โ†’ provider, dispatches to provider API, returns OpenAI-compatible embedding response. Supports 6+ providers. | +| `imageGeneration.ts` | Image generation handler: resolves image model โ†’ provider, supports OpenAI-compatible, Gemini-image (Antigravity), and fallback (Nebius) modes. Returns base64 or URL images. | -#### Request Lifecycle (chatCore.js) +#### Request Lifecycle (chatCore.ts) ```mermaid sequenceDiagram @@ -262,20 +262,20 @@ Business logic that supports the handlers and executors. | File | Purpose | | -------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `provider.js` | **Format detection** (`detectFormat`): analyzes request body structure to identify Claude/OpenAI/Gemini/Antigravity/Responses formats (includes `max_tokens` heuristic for Claude). Also: URL building, header building, thinking config normalization. Supports `openai-compatible-*` and `anthropic-compatible-*` dynamic providers. | -| `model.js` | Model string parsing (`claude/model-name` โ†’ `{provider: "claude", model: "model-name"}`), alias resolution with collision detection, input sanitization (rejects path traversal/control chars), and model info resolution with async alias getter support. | -| `accountFallback.js` | Rate-limit handling: exponential backoff (1s โ†’ 2s โ†’ 4s โ†’ max 2min), account cooldown management, error classification (which errors trigger fallback vs. not). | -| `tokenRefresh.js` | OAuth token refresh for **every provider**: Google (Gemini, Antigravity), Claude, Codex, Qwen, iFlow, GitHub (OAuth + Copilot dual-token), Kiro (AWS SSO OIDC + Social Auth). Includes in-flight promise deduplication cache and retry with exponential backoff. | -| `combo.js` | **Combo models**: chains of fallback models. If model A fails with a fallback-eligible error, try model B, then C, etc. Returns actual upstream status codes. | -| `usage.js` | Fetches quota/usage data from provider APIs (GitHub Copilot quotas, Antigravity model quotas, Codex rate limits, Kiro usage breakdowns, Claude settings). | -| `accountSelector.js` | Smart account selection with scoring algorithm: considers priority, health status, round-robin position, and cooldown state to pick the optimal account for each request. | -| `contextManager.js` | Request context lifecycle management: creates and tracks per-request context objects with metadata (request ID, timestamps, provider info) for debugging and logging. | -| `ipFilter.js` | IP-based access control: supports allowlist and blocklist modes. Validates client IP against configured rules before processing API requests. | -| `sessionManager.js` | Session tracking with client fingerprinting: tracks active sessions using hashed client identifiers, monitors request counts, and provides session metrics. | -| `signatureCache.js` | Request signature-based deduplication cache: prevents duplicate requests by caching recent request signatures and returning cached responses for identical requests within a time window. | -| `systemPrompt.js` | Global system prompt injection: prepends or appends a configurable system prompt to all requests, with per-provider compatibility handling. | -| `thinkingBudget.js` | Reasoning token budget management: supports passthrough, auto (strip thinking config), custom (fixed budget), and adaptive (complexity-scaled) modes for controlling thinking/reasoning tokens. | -| `wildcardRouter.js` | Wildcard model pattern routing: resolves wildcard patterns (e.g., `*/claude-*`) to concrete provider/model pairs based on availability and priority. | +| `provider.ts` | **Format detection** (`detectFormat`): analyzes request body structure to identify Claude/OpenAI/Gemini/Antigravity/Responses formats (includes `max_tokens` heuristic for Claude). Also: URL building, header building, thinking config normalization. Supports `openai-compatible-*` and `anthropic-compatible-*` dynamic providers. | +| `model.ts` | Model string parsing (`claude/model-name` โ†’ `{provider: "claude", model: "model-name"}`), alias resolution with collision detection, input sanitization (rejects path traversal/control chars), and model info resolution with async alias getter support. | +| `accountFallback.ts` | Rate-limit handling: exponential backoff (1s โ†’ 2s โ†’ 4s โ†’ max 2min), account cooldown management, error classification (which errors trigger fallback vs. not). | +| `tokenRefresh.ts` | OAuth token refresh for **every provider**: Google (Gemini, Antigravity), Claude, Codex, Qwen, iFlow, GitHub (OAuth + Copilot dual-token), Kiro (AWS SSO OIDC + Social Auth). Includes in-flight promise deduplication cache and retry with exponential backoff. | +| `combo.ts` | **Combo models**: chains of fallback models. If model A fails with a fallback-eligible error, try model B, then C, etc. Returns actual upstream status codes. | +| `usage.ts` | Fetches quota/usage data from provider APIs (GitHub Copilot quotas, Antigravity model quotas, Codex rate limits, Kiro usage breakdowns, Claude settings). | +| `accountSelector.ts` | Smart account selection with scoring algorithm: considers priority, health status, round-robin position, and cooldown state to pick the optimal account for each request. | +| `contextManager.ts` | Request context lifecycle management: creates and tracks per-request context objects with metadata (request ID, timestamps, provider info) for debugging and logging. | +| `ipFilter.ts` | IP-based access control: supports allowlist and blocklist modes. Validates client IP against configured rules before processing API requests. | +| `sessionManager.ts` | Session tracking with client fingerprinting: tracks active sessions using hashed client identifiers, monitors request counts, and provides session metrics. | +| `signatureCache.ts` | Request signature-based deduplication cache: prevents duplicate requests by caching recent request signatures and returning cached responses for identical requests within a time window. | +| `systemPrompt.ts` | Global system prompt injection: prepends or appends a configurable system prompt to all requests, with per-provider compatibility handling. | +| `thinkingBudget.ts` | Reasoning token budget management: supports passthrough, auto (strip thinking config), custom (fixed budget), and adaptive (complexity-scaled) modes for controlling thinking/reasoning tokens. | +| `wildcardRouter.ts` | Wildcard model pattern routing: resolves wildcard patterns (e.g., `*/claude-*`) to concrete provider/model pairs based on availability and priority. | #### Token Refresh Deduplication @@ -377,8 +377,8 @@ graph TD | `request/` | 8 translators | Convert request bodies between formats. Each file self-registers via `register(from, to, fn)` on import. | | `response/` | 7 translators | Convert streaming response chunks between formats. Handles SSE event types, thinking blocks, tool calls. | | `helpers/` | 6 helpers | Shared utilities: `claudeHelper` (system prompt extraction, thinking config), `geminiHelper` (parts/contents mapping), `openaiHelper` (format filtering), `toolCallHelper` (ID generation, missing response injection), `maxTokensHelper`, `responsesApiHelper`. | -| `index.js` | โ€” | Translation engine: `translateRequest()`, `translateResponse()`, state management, registry. | -| `formats.js` | โ€” | Format constants: `OPENAI`, `CLAUDE`, `GEMINI`, `ANTIGRAVITY`, `KIRO`, `CURSOR`, `OPENAI_RESPONSES`. | +| `index.ts` | โ€” | Translation engine: `translateRequest()`, `translateResponse()`, state management, registry. | +| `formats.ts` | โ€” | Format constants: `OPENAI`, `CLAUDE`, `GEMINI`, `ANTIGRAVITY`, `KIRO`, `CURSOR`, `OPENAI_RESPONSES`. | #### Key Design: Self-Registering Plugins @@ -397,13 +397,13 @@ import "./request/claude-to-openai.js"; // โ† self-registers | File | Purpose | | ------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -| `error.js` | Error response building (OpenAI-compatible format), upstream error parsing, Antigravity retry-time extraction from error messages, SSE error streaming. | -| `stream.js` | **SSE Transform Stream** โ€” the core streaming pipeline. Two modes: `TRANSLATE` (full format translation) and `PASSTHROUGH` (normalize + extract usage). Handles chunk buffering, usage estimation, content length tracking. Per-stream encoder/decoder instances avoid shared state. | -| `streamHelpers.js` | Low-level SSE utilities: `parseSSELine` (whitespace-tolerant), `hasValuableContent` (filters empty chunks for OpenAI/Claude/Gemini), `fixInvalidId`, `formatSSE` (format-aware SSE serialization with `perf_metrics` cleanup). | -| `usageTracking.js` | Token usage extraction from any format (Claude/OpenAI/Gemini/Responses), estimation with separate tool/message char-per-token ratios, buffer addition (2000 tokens safety margin), format-specific field filtering, console logging with ANSI colors. | -| `requestLogger.js` | File-based request logging (opt-in via `ENABLE_REQUEST_LOGS=true`). Creates session folders with numbered files: `1_req_client.json` โ†’ `7_res_client.txt`. All I/O is async (fire-and-forget). Masks sensitive headers. | -| `bypassHandler.js` | Intercepts specific patterns from Claude CLI (title extraction, warmup, count) and returns fake responses without calling any provider. Supports both streaming and non-streaming. Intentionally limited to Claude CLI scope. | -| `networkProxy.js` | Resolves outbound proxy URL for a given provider with precedence: provider-specific config โ†’ global config โ†’ environment variables (`HTTPS_PROXY`/`HTTP_PROXY`/`ALL_PROXY`). Supports `NO_PROXY` exclusions. Caches config for 30s. | +| `error.ts` | Error response building (OpenAI-compatible format), upstream error parsing, Antigravity retry-time extraction from error messages, SSE error streaming. | +| `stream.ts` | **SSE Transform Stream** โ€” the core streaming pipeline. Two modes: `TRANSLATE` (full format translation) and `PASSTHROUGH` (normalize + extract usage). Handles chunk buffering, usage estimation, content length tracking. Per-stream encoder/decoder instances avoid shared state. | +| `streamHelpers.ts` | Low-level SSE utilities: `parseSSELine` (whitespace-tolerant), `hasValuableContent` (filters empty chunks for OpenAI/Claude/Gemini), `fixInvalidId`, `formatSSE` (format-aware SSE serialization with `perf_metrics` cleanup). | +| `usageTracking.ts` | Token usage extraction from any format (Claude/OpenAI/Gemini/Responses), estimation with separate tool/message char-per-token ratios, buffer addition (2000 tokens safety margin), format-specific field filtering, console logging with ANSI colors. | +| `requestLogger.ts` | File-based request logging (opt-in via `ENABLE_REQUEST_LOGS=true`). Creates session folders with numbered files: `1_req_client.json` โ†’ `7_res_client.txt`. All I/O is async (fire-and-forget). Masks sensitive headers. | +| `bypassHandler.ts` | Intercepts specific patterns from Claude CLI (title extraction, warmup, count) and returns fake responses without calling any provider. Supports both streaming and non-streaming. Intentionally limited to Claude CLI scope. | +| `networkProxy.ts` | Resolves outbound proxy URL for a given provider with precedence: provider-specific config โ†’ global config โ†’ environment variables (`HTTPS_PROXY`/`HTTP_PROXY`/`ALL_PROXY`). Supports `NO_PROXY` exclusions. Caches config for 30s. | #### SSE Streaming Pipeline @@ -484,7 +484,7 @@ All formats translate through **OpenAI format as the hub**. Adding a new provide ### 5.2 Executor Strategy Pattern -Each provider has a dedicated executor class inheriting from `BaseExecutor`. The factory in `executors/index.js` selects the right one at runtime. +Each provider has a dedicated executor class inheriting from `BaseExecutor`. The factory in `executors/index.ts` selects the right one at runtime. ### 5.3 Self-Registering Plugin System