Commit Graph

26 Commits

Author SHA1 Message Date
tombii
c6eadc504b fix(usage): include cache tokens in input token counts
- Fix getLoggedInputTokens to return full prompt_tokens (input + cache_read + cache_creation)
- Fix usageExtractor for non-streaming Claude responses to calculate total correctly
- Add formatUsageLog helper to show CR=<cache_read> in logs
- Add migration 012 to fix historical token counts in usage_history
- Move prompt cache metrics from Settings to /dashboard/cache page

Per Claude API docs:
Total input tokens = input_tokens + cache_creation_input_tokens + cache_read_input_tokens

Fixes issue where totalInputTokens (71k) was less than totalCacheCreationTokens (1.35M).

Tested:
- All 1134 unit tests pass
- Cache metrics API returns correct totals
- Migration is idempotent and tracked in _omniroute_migrations
- Logs show cache read tokens: 'in=6055 | out=211 | CR=22399'

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-30 19:24:26 +02:00
Diego Rodrigues de Sa e Souza
2fc4c7ea33 Merge pull request #728 from rdself/codex/normalize-provider-limits-labels
normalize provider limits labels
2026-03-28 23:06:16 -03:00
R.D.
538028c150 normalize provider limits quota labels 2026-03-28 21:17:07 -04:00
R.D.
04de492019 fix: add four-stage request log payloads 2026-03-28 22:09:27 -03:00
diegosouzapw
7abea2020c Merge feature-tests: schema coercion, tool sanitization, Codex auth export, enhanced test suite 2026-03-28 16:27:32 -03:00
diegosouzapw
8595964ab8 feat/fix: implement upstream sync tasks 1-7 2026-03-28 14:48:57 -03:00
diegosouzapw
55325773dc feat(open-sse): add schema coercion and tool sanitization
- introduce open-sse/translator/helpers/schemaCoercion.ts to coerce
  numeric JSON Schema fields encoded as strings
- wire coerceToolSchemas and sanitizeToolDescriptions into translator
  pipeline; ensure tool descriptions are sanitized
- inject empty reasoning content for tool calls when target is OpenAI
  format
- update qwen base URL to DashScope-compatible endpoint
- extend antigravity static catalog with Gemini 3.1 pro preview models and
  update Gemini model specs with preview aliases
- implement call log max cap caching with TTL; expose invalidateCallLogsMaxCache
  and invalidate on settings PATCH
- add tests: call-log-cap.test.mjs and tool-request-sanitization.test.mjs;
  extend tests for Windsurf integration and gemini previews
- update CLI runtime and tools to include Windsurf as a guide-only tool
- add maxCallLogs to validation schemas (settings and updateSettings)
- add Czech README (README.cs.md) to repository
2026-03-28 12:33:13 -03:00
cai kerui
ae8d2ac2e1 Merge branch 'main' into codex/claude-cache-log-accounting 2026-03-27 17:25:38 +09:00
cai kerui
93beb068a3 Add Claude prompt cache logging and exclude cache reads 2026-03-27 15:14:54 +09:00
diegosouzapw
c34b3f41bd feat: Add requested model to logs, enhance background task detection, and introduce AI SDK compatibility utilities. 2026-03-23 11:08:14 -03:00
diegosouzapw
7a7f3be0d2 feat(sub2api): implement T01-T15 gap analysis tasks (3.0.0-rc.6)
T01 (P1): requested_model column in call_logs
- Migration 009_requested_model.sql: ALTER TABLE call_logs ADD COLUMN requested_model
- callLogs.ts: INSERT + SELECT updated to include requestedModel field

T02 (P1): Strip empty text blocks from nested tool_result.content
- New stripEmptyTextBlocks() recursive helper in openai-to-claude.ts
- Applied on tool_result content before forwarding to Anthropic
- Prevents 400 'text content blocks must be non-empty' errors

T03 (P1): Parse x-codex-5h-*/x-codex-7d-* headers for precise quota reset
- parseCodexQuotaHeaders() in codex.ts extracts usage/limit/resetAt
- getCodexResetTime() returns furthest-out reset timestamp for safe unblocking

T04 (P1): X-Session-Id header for external sticky routing
- extractExternalSessionId() in sessionManager.ts reads x-session-id,
  x-omniroute-session, session-id headers with 'ext:' prefix to avoid collisions

T06 (P2): account_deactivated permanent expired status on 401
- ACCOUNT_DEACTIVATED_SIGNALS constant + isAccountDeactivated() in accountFallback.ts
- Returns 1-year cooldown (effectively permanent) to prevent retrying dead accounts

T07 (P2): X-Forwarded-For IP validation
- New src/lib/ipUtils.ts with extractClientIp() and getClientIpFromRequest()
- Skips 'unknown'/non-IP entries in X-Forwarded-For chain

T10 (P2): credits_exhausted distinct account status
- CREDITS_EXHAUSTED_SIGNALS + isCreditsExhausted() in accountFallback.ts
- Returns 1h cooldown with creditsExhausted flag, distinct from rate_limit 429

T11 (P1): max reasoning_effort -> budget_tokens: 131072
- EFFORT_BUDGETS and THINKING_LEVEL_MAP updated with max: 131072, xhigh: 131072
- Reverse mapping now returns 'max' for full-budget responses
- Unit test updated to expect 'max' (was 'high')

T12 (P3): Model pricing updates
- MiniMax M2.7 / MiniMax-M2.7 / minimax-m2.7-highspeed pricing added

T15 (P1): Array content normalization for system/tool messages
- normalizeContentToString() helper exported from openai-to-claude.ts
- System messages with array content now correctly collapsed to string
2026-03-22 20:55:35 -03:00
diegosouzapw
601cc21a44 feat: call log response content, per-model tool call ID, key PATCH & validation (#470) 2026-03-19 08:41:01 -03:00
Prakersh Maheshwari
4953727aa7 fix(callLogs): support Claude format usage and include cache tokens (#476)
saveCallLog only read prompt_tokens/completion_tokens (OpenAI format).
When sourceFormat=claude, the openai-to-claude translator writes
input_tokens/output_tokens instead, causing all cross-format requests
(Codex-via-Claude, Kiro-via-Claude, etc.) to show 0|0 tokens in
call_logs.

Also includes cache_read and cache_creation tokens in tokens_in total
so heavily-cached requests don't show misleadingly low input counts.

Changes:
- Read prompt_tokens || input_tokens (supports both formats)
- Read completion_tokens || output_tokens (supports both formats)
- Sum cache_read_input_tokens + cache_creation_input_tokens into total
2026-03-19 08:37:49 -03:00
diegosouzapw
a3deacd718 feat: Implement historical model latency and success rate tracking for auto-combo routing and update Claude and Deepseek pricing and model registrations. 2026-03-17 16:18:36 -03:00
Regis
564e983c68 feat(search): add unified web search routing with 5 providers
Add POST /v1/search — a unified search endpoint routing queries across
5 providers (Serper, Brave, Perplexity Search, Exa, Tavily) with
automatic failover, in-memory caching, and request coalescing.

No open-source AI gateway offers unified search routing. This chains
free tiers for 5,500+ searches/month with zero downtime.

Providers: Serper ($0.001/q, 2500/mo free), Brave ($0.005/q, 1000/mo),
Perplexity Search ($0.005/q), Exa ($0.007/q, 1000/mo), Tavily
($0.008/q, 1000/mo). Auto-select picks cheapest with credentials.

Architecture follows existing patterns:
- searchRegistry.ts (same as embeddingRegistry.ts)
- search.ts handler (same as embeddings.ts)
- route.ts (same as /v1/embeddings/route.ts)
- searchCache.ts (bounded TTL cache + request coalescing)

Schema finalized — all future fields defined as optional with safe
defaults. No breaking changes when implementing content extraction,
answer synthesis, or ranking.

Key features:
- Per-provider request builders and response normalizers
- Enriched response: display_url, score, favicon_url, content block,
  metadata, answer block, errors array, upstream_latency_ms metrics
- Cost-sorted auto-select with failover on 429/5xx/timeout
- Credential fallback (perplexity-search reuses perplexity chat key)
- Cache key includes all result-affecting parameters
- max_results clamped to provider limits, sanitized error responses
- Factored validators (validateSearchProvider factory)
- CORS headers on all responses
- Dashboard: Search & Discovery section, search provider template
- DB migration 007: request_type column in call_logs
- 28 unit tests (registry, cache, coalescing, validation)
2026-03-17 18:28:35 +01:00
diegosouzapw
095c84ac16 fix(providerRegistry): remove duplicate claude-haiku-4-5-20251001 from anthropic provider to prevent ambiguous model resolution 2026-03-17 13:10:23 -03:00
diegosouzapw
4643c19abc feat: add Kiro credit tracking in usage fetcher (#337) 2026-03-15 08:47:27 -03:00
Diego Rodrigues de Sa e Souza
ce560ebe9d fix: resolve issues #273, #276, #277 — image routing, models route, missing-key error (#282)
Squash merge PR #282: bug fixes for #273 (Gemini image routing), #276 (Ollama Cloud models), #277 (missing apiKey error), lint fix, and all security code-scanning patches.
2026-03-10 09:48:50 -03:00
Diego Rodrigues de Sa e Souza
44ee62e391 feat(#248,#260): TTS provider support + friendly display names (#262)
* feat(#248,#260): TTS provider support + friendly display names

## #248 — Text to Speech endpoint expansion
- audioRegistry: add Inworld, Cartesia, PlayHT providers with correct auth/format tags
- audioSpeech handler: add handleInworldSpeech (base64 decode), handleCartesiaSpeech (X-API-Key + transcript), handlePlayHtSpeech (X-USER-ID)
- /dashboard/media: add Speech tab with voice selector, format selector, and inline <audio> player (Blob URL)
- /dashboard/playground: add Text to Speech option; detect audio/* Content-Type and render inline audio player

## #260 — Friendly provider/account display names
- src/lib/display/names.ts: new centralized helper module with getAccountDisplayName() and getProviderDisplayName()
- src/lib/usage/usageStats.ts: replace raw connectionId.slice(0,8) fallbacks with getAccountDisplayName()
- src/app/api/rate-limits/route.ts: replace raw connectionId.slice(0,8) fallback with getAccountDisplayName()

* feat(#248,#261): ElevenLabs voice presets + mobile UX fixes

## #248 — ElevenLabs/Cartesia/Deepgram/Inworld voice presets
- MediaPageClient: replace static SPEECH_VOICES with provider-aware PROVIDER_VOICE_PRESETS
  - ElevenLabs: 9 premade voices with name+type (Rachel, Domi, Bella, Antoni, Elli, Josh, Arnold, Adam, Sam)
  - Cartesia: 3 built-in voices (Barbershop Man, Friendly Reading Man, California Girl)
  - Deepgram: 5 Aura voices (Asteria, Luna, Stella, Zeus, Orion)
  - Inworld: 2 voices (Eva, Marcus)
  - default: OpenAI 6 standard voices (alloy, echo, fable, onyx, nova, shimmer)
- Voice dropdown updates automatically when model input prefix changes

## #261 — Mobile UX improvements
- DashboardLayout: add h-dvh to mobile sidebar wrapper for proper scroll containment
- Sidebar: add h-full to aside element so nav can scroll on short screens
- ProvidersPage: change all 4 section headers from flex justify-between to flex flex-wrap so actions wrap on narrow screens instead of overflowing

---------

Co-authored-by: diegosouzapw <diegosouzapw@users.noreply.github.com>
2026-03-09 13:43:48 -03:00
diegosouzapw
baa0208fa9 feat: v2.0.0 - MCP server, A2A agent, proxy improvements and docs update 2026-03-05 01:16:56 -03:00
diegosouzapw
052eb8d330 refactor: replace any types with generics and add Zod validation schemas
Eliminate `any` usage across the codebase by introducing proper generics,
typed interfaces (StatementLike, DbLike, PromptRow, etc.), and helper
conversion functions (toNumber, toString, parseVariables). Add
comprehensive Zod validation schemas for API endpoint inputs to enforce
runtime type safety alongside compile-time checks.
2026-03-04 18:59:27 -03:00
benzntech
5a2fdacebe feat(memory): optimize RAM usage for low-memory deployments
- Add Node.js heap limit (256MB default) via OMNIROUTE_MEMORY_MB
- Convert LRU caches to byte-based limits (2-4MB)
- Reduce in-memory buffer sizes: 500→200 entries
- Add .env file loading in CLI for global npm installs
- Add PM2 ecosystem.config.js example for deployment
- Document memory tuning env vars in USER_GUIDE.md

Estimated RAM reduction: ~60-70% (from ~512MB+ to ~150-200MB)
2026-02-23 14:16:47 +05:30
nyatoru
0610909116 feat(codex): add workspace binding via chatgpt-account-id header 2026-02-23 01:11:53 +07:00
diegosouzapw
0279341221 feat(phase4): focus indicators, session management, k6 load tests, remove @ts-check
- U-3/U-4: Global focus-visible indicators + CSS focus-ring utility
- C-6: Removed @ts-check from 18 .ts files (redundant in TypeScript)
- T-5: k6 load test script (tests/load/proxy-load.js)
- P-3: SessionInfoCard component in Security settings tab
2026-02-18 14:51:15 -03:00
diegosouzapw
4c93f0618c feat: Add dashboard loading and error UIs, introduce standardized API response utilities, enhance call log redaction and configurable retention, and document architectural decisions. 2026-02-18 11:34:05 -03:00
diegosouzapw
71d14209a4 feat: OmniRoute v1.0.0 — Intelligent AI Gateway & Universal LLM Proxy
OmniRoute is an intelligent API gateway that unifies 20+ AI providers behind a single
OpenAI-compatible endpoint. Features include intelligent routing with 6 strategies,
multi-format translation (OpenAI/Claude/Gemini/Responses API), circuit breakers,
semantic caching, combo fallback chains, real-time health monitoring, and a full
dashboard with provider management, analytics, and CLI tool integration.

Key highlights:
- 20+ providers (Claude Code, Codex, Gemini CLI, GitHub Copilot, iFlow, Qwen, Kiro, etc.)
- 6 routing strategies (Fill First, Round Robin, P2C, Random, Least Used, Cost Optimized)
- Export/Import database backup with full archive support
- Translator Playground with 4 modes (Playground, Chat Tester, Test Bench, Live Monitor)
- 100% TypeScript across src/ and open-sse/
- Docker support with multi-stage builds
- Comprehensive documentation and 9 dashboard screenshots
2026-02-18 00:02:15 -03:00