mirror of
https://github.com/diegosouzapw/OmniRoute.git
synced 2026-07-26 09:52:11 +03:00
* perf: startup parallelization, stream TextEncoder lift, auth middleware bottlenecks
Startup (~100-300ms faster cold start):
- Parallelize 4 early imports via Promise.all() in registerNodejs()
- Parallelize 10 independent background services via Promise.allSettled()
- Each service has independent try/catch — no failure domino effect
Streaming pipeline (8 fewer TextEncoder GC allocations per SSE event):
- Lift new TextEncoder() from per-chunk inside buildClaudeStreamingResponse
to function scope alongside existing decoder singleton
Auth middleware bottlenecks (from PerfBottleneckAnalysis):
- Backoff decay loop: replace updateProviderConnection (full CRUD:
SELECT+encrypt+cache-invalidate+backup) with resetConnectionBackoff
(targeted UPDATE of backoff/error columns only)
- Dual .filter() for quota: replace two passes calling
isQuotaExhaustedForRequest per connection with a single for loop
partitioning into withQuota/exhaustedQuota
- Debug-log filter recomputation: capture connectionFilterStatus Map
during the filter pass; debug loop reads 6 string comparisons instead
of 6 function calls per connection
Supporting:
- Add resetConnectionBackoff to src/lib/db/providers.ts (patterned after
clearConnectionErrorIfUnchanged, no CAS check)
- Re-export resetConnectionBackoff from src/lib/localDb.ts
- Update integration-wiring.test.ts regex for parallelized dynamic import
* perf: P2C quota cache, lazy provider init, structuredClone elimination, getSettings→getCachedSettings
- **auth.ts: P2C quota re-evaluation cache** — quotaResults Map threaded
through selectPoolSubset → compareP2CConnections → getP2CConnectionScore.
Populated during filter + partition passes, eliminating redundant
evaluateQuotaLimitPolicy / isQuotaExhaustedForRequest calls when the
P2C comparator re-evaluates previously-scored connections.
- **constants.ts: lazy PROVIDERS via Proxy** — replaces eager
generateLegacyProviders() + loadProviderCredentials() at module load
with Proxy delegating to deferred init on first property access.
- **providerModels.ts: lazy PROVIDER_MODELS + PROVIDER_ID_TO_ALIAS** —
same Proxy pattern for both exports; generateModels()/generateAliasMap()
deferred until first read.
- **stream.ts: structuredClone → minimal object spread** — replaces
O(n) deep clone of SSE response chunks with targeted reconstruction
of only mutated fields (usage, delta.content, finish_reason).
- **progressTracker.ts: TextDecoder lift** — module-level decoder
instead of per-chunk new TextDecoder().
- **Route files: getSettings() → getCachedSettings()** — 13 API route
files converted from uncached per-request DB reads to TTL-cached
wrapper (5s default), eliminating redundant queries on every request.
- **settings.ts: re-export getCachedSettings** from readCache for
non-localDb consumers.
- **Remove settingsCache.ts** — dead file, no imports reference it.
TS compile: 0 errors. Auth tests: 225/225 pass. Services: 269/269 pass.
* perf: Phase 1 tangible wins — egressCache eviction, mmap_size PRAGMA, composite indexes, proxyFallback lazy import
- egressCache: lazy TTL eviction on getCachedEgressIp access (bounds memory
leak to distinct proxy URLs, typically <100)
- mmap_size: apply stored PRAGMA from key_value table (256MiB default) after
applyStoredDatabaseOptimizationSettings — setting was stored but never applied
- schemaColumns: add idx_uh_provider_model_timestamp (covers getModelLatencyStats)
and idx_pc_provider_auth_type (covers 6+ provider_connections queries)
- proxyFallback: convert static import to dynamic import() inside error handler
(defers 210ms module load from startup to first proxy-retry scenario)
* perf: add dedup expression index, unref() sweep timers
- Add COALESCE expression index idx_uh_dedup on usage_history
matching the exact dedup query pattern. Eliminates FULL TABLE
SCAN on every saveRequestUsage insert.
- Add composite idx_uh_provider_model_timestamp on usage_history.
- Add composite idx_pc_provider_auth_type on provider_connections.
- Add .unref() to setInterval in batchProcessor.ts (polling loop).
- Add .unref() to setInterval in runtimeHeartbeat.ts (heartbeat).
* perf: bump SQLite cache_size default from 16MB to 64MB
New installs now start with 64MB page cache (was 16MB). Existing
users' stored settings are unchanged. Reduces disk reads for the
typical ~250MB database by keeping ~25% of pages in memory.
Also resolved pre-existing merge conflict in webhooks.ts.
* docs: add Redis production config guide and proxy port clash investigation report
- docs/redis-production-config.md: comprehensive Redis tuning guide
covering client options, server config, Docker settings, scaling,
and monitoring for all three Redis workloads (rate limiting,
auth cache, quota store)
- docs/proxy-port-clash-report.md: investigation confirming proxy
subsystem has no port binding issues; real EADDRINUSE history
traced to process supervisor crash-loop restart race (#4425) and
live-dashboard port clash (#6324), both already fixed
* fix: address PR #7893 review — add Proxy traps, extract migrations to reduce providers.ts size
- Add set trap to PROVIDER_ID_TO_ALIAS Proxy (providerModels.ts)
- Add deleteProperty traps to all three lazy Proxies (PROVIDERS,
PROVIDER_MODELS, PROVIDER_ID_TO_ALIAS)
- Extract autoMigrateLegacyEncryptedConnections and getGheCopilotHosts
from providers.ts (1129→1036 lines, -93) into providers/migrations.ts
- Both functions re-exported via providers.ts for backward compat
File-size ratchet resolved: src/lib/db/providers.ts now 1036 lines.
* fix: resolve merge conflict markers in 3 route/test files
- model-combo-mappings/route.ts: kept upstream version (Zod pagination
via validateBody + isValidationFailure), restored missing return
statement for GET handler
- playground/presets/route.ts: kept stashed version details (satisfies
type-narrowing + inlined Response) — functionally identical
- error-sanitization.test.ts: matches upstream exactly (no diff)
Test verification: same 7 pre-existing failures confirmed on upstream
baseline (c1bdd91e7). Zero regressions from conflict resolution.
Closes remaining uncommitted work from PR #7046 rebase.
* test(db): add resetConnectionBackoff coverage + fix file-size ratchet regression
- Add tests/unit/reset-connection-backoff.test.ts: covers the new
resetConnectionBackoff lightweight-UPDATE helper (clears backoff/error
columns and re-activates a connection, unconditional-write behavior on
terminal statuses, no-op for empty/unknown ids). Zero prior coverage
per pre-merge review of PR #7893 (Hard Rule #8).
- open-sse/services/batchProcessor.ts: fold the new unref() call into the
existing setInterval(...).unref() chain instead of a separate statement,
keeping the file at the frozen 915-line ratchet (Timeout.unref() already
returns `this`, so no type cast is needed).
- src/lib/localDb.ts: drop one blank separator line so the new
resetConnectionBackoff re-export stays within the frozen 808-line ratchet.
Pre-merge fix for PR #7893 (perf/startup-stream-auth-optimizations) per
/green-prs plan-file _tasks/pipeline/prs/1-analyzed/7893-*.plan.md.
Co-authored-by: Diego Rodrigues de Sa e Souza <diegosouzapw@users.noreply.github.com>
---------
Co-authored-by: oyi77 <oyi77@users.noreply.github.com>
Co-authored-by: Diego Rodrigues de Sa e Souza <diegosouza.pw@gmail.com>
Co-authored-by: Diego Rodrigues de Sa e Souza <diegosouzapw@users.noreply.github.com>
282 lines
11 KiB
TypeScript
282 lines
11 KiB
TypeScript
import { generateModels, generateAliasMap, type RegistryModel } from "./providerRegistry.ts";
|
|
|
|
// Lazy PROVIDER_MODELS: deferred until first property access to speed up startup.
|
|
// The Proxy defers `generateModels()` from module-evaluation time to the first read.
|
|
let _models: Record<string, RegistryModel[]> | null = null;
|
|
function initModels(): Record<string, RegistryModel[]> {
|
|
if (!_models) _models = generateModels();
|
|
return _models;
|
|
}
|
|
|
|
export const PROVIDER_MODELS: Record<string, RegistryModel[]> = new Proxy(
|
|
{} as Record<string, RegistryModel[]>,
|
|
{
|
|
get(_, prop) {
|
|
if (typeof prop === 'symbol') return undefined;
|
|
return Reflect.get(initModels(), prop, _models);
|
|
},
|
|
has(_, prop) {
|
|
if (typeof prop === 'symbol') return false;
|
|
return Reflect.has(initModels(), prop);
|
|
},
|
|
ownKeys() {
|
|
return Reflect.ownKeys(initModels());
|
|
},
|
|
getOwnPropertyDescriptor(_, prop) {
|
|
if (typeof prop === 'symbol') return undefined;
|
|
return Object.getOwnPropertyDescriptor(initModels(), prop);
|
|
},
|
|
set(_, prop, value) {
|
|
if (typeof prop === 'symbol') return false;
|
|
(initModels() as Record<string, RegistryModel[]>)[prop] = value;
|
|
return true;
|
|
},
|
|
deleteProperty(_, prop) {
|
|
if (typeof prop === 'symbol') return false;
|
|
return Reflect.deleteProperty(initModels(), prop);
|
|
},
|
|
}
|
|
);
|
|
export const PROVIDER_ID_TO_ALIAS: Record<string, string> = new Proxy(
|
|
{} as Record<string, string>,
|
|
{
|
|
get(_, prop) {
|
|
if (typeof prop === 'symbol') return undefined;
|
|
return Reflect.get(initAliases(), prop, _aliases);
|
|
},
|
|
has(_, prop) {
|
|
if (typeof prop === 'symbol') return false;
|
|
return Reflect.has(initAliases(), prop);
|
|
},
|
|
ownKeys() {
|
|
return Reflect.ownKeys(initAliases());
|
|
},
|
|
getOwnPropertyDescriptor(_, prop) {
|
|
if (typeof prop === 'symbol') return undefined;
|
|
return Object.getOwnPropertyDescriptor(initAliases(), prop);
|
|
},
|
|
set(_, prop, value) {
|
|
if (typeof prop === 'symbol') return false;
|
|
(initAliases() as Record<string, string>)[prop] = value;
|
|
return true;
|
|
},
|
|
deleteProperty(_, prop) {
|
|
if (typeof prop === 'symbol') return false;
|
|
return Reflect.deleteProperty(initAliases(), prop);
|
|
},
|
|
}
|
|
);
|
|
|
|
let _aliases: Record<string, string> | null = null;
|
|
function initAliases(): Record<string, string> {
|
|
if (!_aliases) _aliases = generateAliasMap();
|
|
return _aliases;
|
|
}
|
|
|
|
// Helper functions
|
|
export function getProviderModels(aliasOrId: string): RegistryModel[] {
|
|
// Accept either the public alias (the /v1/models prefix, e.g. "gh") or the raw
|
|
// provider id (e.g. "github") and resolve id→alias before reading the namespace
|
|
// map — so callers don't need to know which form they hold. We resolve here rather
|
|
// than mirroring raw-id keys into PROVIDER_MODELS, whose keys ARE the public
|
|
// prefixes (a raw id like "opencode" would collide with the opencode-zen route —
|
|
// see #2798/#3870).
|
|
const alias = PROVIDER_ID_TO_ALIAS[aliasOrId] || aliasOrId;
|
|
return PROVIDER_MODELS[alias] || PROVIDER_MODELS[aliasOrId] || [];
|
|
}
|
|
|
|
export function getDefaultModel(aliasOrId: string): string | null {
|
|
const models = PROVIDER_MODELS[aliasOrId];
|
|
return models?.[0]?.id || null;
|
|
}
|
|
|
|
export function getProviderModel(aliasOrId: string, modelId: string): RegistryModel | undefined {
|
|
const models = PROVIDER_MODELS[aliasOrId];
|
|
if (!models) return undefined;
|
|
return models.find((model) => model.id === modelId);
|
|
}
|
|
|
|
export function isValidModel(
|
|
aliasOrId: string,
|
|
modelId: string,
|
|
passthroughProviders = new Set<string>()
|
|
): boolean {
|
|
if (passthroughProviders.has(aliasOrId)) return true;
|
|
const models = PROVIDER_MODELS[aliasOrId];
|
|
if (!models) return false;
|
|
return models.some((m) => m.id === modelId);
|
|
}
|
|
|
|
export function findModelName(aliasOrId: string, modelId: string): string {
|
|
const models = PROVIDER_MODELS[aliasOrId];
|
|
if (!models) return modelId;
|
|
const found = models.find((m) => m.id === modelId);
|
|
return found?.name || modelId;
|
|
}
|
|
|
|
export function getModelTargetFormat(aliasOrId: string, modelId: string): string | null {
|
|
const models = PROVIDER_MODELS[aliasOrId];
|
|
const found = models?.find((m) => m.id === modelId);
|
|
if (found?.targetFormat) return found.targetFormat;
|
|
// #5842: OpenAI "*-pro" reasoning models (o1-pro, gpt-5.x-pro) are only served by
|
|
// the native /v1/responses endpoint — /v1/chat/completions 404s ("only supported
|
|
// in v1/responses"). Curated catalog entries are tagged explicitly; this heuristic
|
|
// covers dynamically-synced ids that post-date the catalog (same spirit as the gh
|
|
// executor's /codex/i routing, 9router#102). Scoped to the openai alias so other
|
|
// providers shipping *-pro ids keep their own endpoint semantics.
|
|
if (aliasOrId === "openai" && /-pro$/i.test(modelId)) return "openai-responses";
|
|
return null;
|
|
}
|
|
|
|
export function getModelStripTypes(aliasOrId: string, modelId: string): string[] {
|
|
const models = PROVIDER_MODELS[aliasOrId];
|
|
if (!models) return [];
|
|
const found = models.find((m) => m.id === modelId);
|
|
return Array.isArray(found?.strip) ? [...found.strip] : [];
|
|
}
|
|
|
|
export function getModelsByProviderId(providerId: string): RegistryModel[] {
|
|
const alias = PROVIDER_ID_TO_ALIAS[providerId] || providerId;
|
|
return PROVIDER_MODELS[alias] || [];
|
|
}
|
|
|
|
/**
|
|
* Model-level upstream header-response timeout override, when the registry
|
|
* entry for `modelId` sets one (#6354). Returns `undefined` when the model
|
|
* isn't found or has no override, so callers can fall through to the
|
|
* provider-level/global defaults unchanged.
|
|
*/
|
|
export function getModelTimeoutMs(aliasOrId: string, modelId: string): number | undefined {
|
|
// Callers (e.g. chatCore's timeout resolution) pass the raw provider id
|
|
// ("codex"), not the public alias ("cx") that PROVIDER_MODELS is keyed by
|
|
// — resolve id→alias the same way getProviderModels()/getModelsByProviderId()
|
|
// do, so the override actually resolves (#6354).
|
|
const alias = PROVIDER_ID_TO_ALIAS[aliasOrId] || aliasOrId;
|
|
return getProviderModel(alias, modelId)?.timeoutMs;
|
|
}
|
|
|
|
const CLAUDE_MODEL_PATTERN = /(?:^|[\/._-])claude(?:[._-]|$)/;
|
|
const CLAUDE_MAX_EFFORT_UNSUPPORTED_FAMILY_PATTERNS = [/(?:^|[\/._-])haiku(?:[._-]|$)/] as const;
|
|
const ANTHROPIC_COMPATIBLE_PREFIX = "anthropic-compatible-";
|
|
|
|
export function supportsClaudeMaxEffort(modelId: string | null | undefined): boolean {
|
|
if (typeof modelId !== "string" || modelId.length === 0) return false;
|
|
const normalized = modelId.toLowerCase();
|
|
const claudeMatch = normalized.match(CLAUDE_MODEL_PATTERN);
|
|
if (!claudeMatch) return false;
|
|
const claudeScopedId = normalized.slice(claudeMatch.index ?? 0);
|
|
return !CLAUDE_MAX_EFFORT_UNSUPPORTED_FAMILY_PATTERNS.some((pattern) =>
|
|
pattern.test(claudeScopedId)
|
|
);
|
|
}
|
|
|
|
// Reasoning-effort suffixes the Claude/Claude-Code model picker appends to a base
|
|
// model id (an "Effort" slider: Low/Medium/High/Extra-High/Max). Longest/most
|
|
// specific token first so the `-${level}` match below picks "xhigh" before "high".
|
|
export const CLAUDE_EFFORT_SUFFIXES = ["xhigh", "max", "high", "medium", "low"] as const;
|
|
export type ClaudeEffortSuffix = (typeof CLAUDE_EFFORT_SUFFIXES)[number];
|
|
|
|
/**
|
|
* Split a trailing reasoning-effort suffix off a Claude model id, e.g.
|
|
* "claude-opus-4-8-high" -> { baseModel: "claude-opus-4-8", effort: "high" }.
|
|
*
|
|
* VS Code (and other clients) advertise claude-...-{low,medium,high,xhigh,max} via
|
|
* the model catalog; Anthropic has no such model id, so the suffixed string must be
|
|
* stripped before it is sent upstream (otherwise the relay returns HTTP 404) and
|
|
* surfaced as reasoning_effort so the translator / Claude-Code bridge convert it into
|
|
* Claude thinking/effort config. Mirrors codex's splitCodexReasoningSuffix but also
|
|
* covers "max" (codex's EFFORT_ORDER intentionally omits it). The `-${level}` anchor
|
|
* keeps "xhigh" from colliding with "high".
|
|
*/
|
|
export function splitClaudeEffortSuffix(model: unknown): {
|
|
baseModel: string;
|
|
effort: ClaudeEffortSuffix | null;
|
|
} {
|
|
const id = typeof model === "string" ? model : "";
|
|
const lower = id.toLowerCase();
|
|
for (const level of CLAUDE_EFFORT_SUFFIXES) {
|
|
if (lower.endsWith(`-${level}`)) {
|
|
return { baseModel: id.slice(0, -(level.length + 1)), effort: level };
|
|
}
|
|
}
|
|
return { baseModel: id, effort: null };
|
|
}
|
|
|
|
function getDatedClaudeAliasDate(candidate: string, modelId: string): number | null {
|
|
if (!modelId.startsWith(`${candidate}-`)) return null;
|
|
const suffix = modelId.slice(candidate.length + 1);
|
|
if (!/^\d{8}$/.test(suffix)) return null;
|
|
return Number(suffix);
|
|
}
|
|
|
|
function findCanonicalClaudeEffortModel(modelId: string): RegistryModel | undefined {
|
|
const id = splitClaudeEffortSuffix(modelId).baseModel.toLowerCase();
|
|
const claudeMatch = id.match(CLAUDE_MODEL_PATTERN);
|
|
if (!claudeMatch) return undefined;
|
|
|
|
const claudeOffset = claudeMatch[0]?.indexOf("claude") ?? 0;
|
|
const claudeStart = (claudeMatch.index ?? 0) + Math.max(claudeOffset, 0);
|
|
const claudeScopedId = id.slice(claudeStart).replace(/\.(?=\d)/g, "-");
|
|
const candidates = [claudeScopedId];
|
|
if (claudeScopedId.endsWith("-thinking")) {
|
|
candidates.push(claudeScopedId.slice(0, -"-thinking".length));
|
|
}
|
|
|
|
const claudeModels = getModelsByProviderId("claude");
|
|
for (const candidate of candidates) {
|
|
const exact = claudeModels.find((entry) => entry.id.toLowerCase() === candidate);
|
|
if (exact) return exact;
|
|
|
|
if (!/-\d+-\d+$/.test(candidate)) continue;
|
|
const datedAliases = claudeModels
|
|
.map((entry) => ({
|
|
entry,
|
|
date: getDatedClaudeAliasDate(candidate, entry.id.toLowerCase()),
|
|
}))
|
|
.filter(
|
|
(item): item is { entry: RegistryModel; date: number } =>
|
|
item.date !== null && item.entry.supportsXHighEffort !== undefined
|
|
)
|
|
.sort((a, b) => b.date - a.date || a.entry.id.localeCompare(b.entry.id));
|
|
if (datedAliases[0]) return datedAliases[0].entry;
|
|
}
|
|
|
|
return undefined;
|
|
}
|
|
|
|
function resolveProviderModelList(aliasOrId: string): {
|
|
alias: string;
|
|
models: RegistryModel[] | null;
|
|
} {
|
|
const resolvedId = aliasOrId.startsWith(ANTHROPIC_COMPATIBLE_PREFIX) ? "claude" : aliasOrId;
|
|
const alias = PROVIDER_ID_TO_ALIAS[resolvedId] || resolvedId;
|
|
const models = PROVIDER_MODELS[alias] || PROVIDER_MODELS[resolvedId] || null;
|
|
return { alias, models };
|
|
}
|
|
|
|
export function supportsXHighEffort(aliasOrId: string, modelId: string): boolean {
|
|
const { models: providerModels } = resolveProviderModelList(aliasOrId);
|
|
const model = providerModels?.find((entry) => entry.id === modelId);
|
|
if (model?.supportsXHighEffort !== undefined) {
|
|
return model.supportsXHighEffort !== false;
|
|
}
|
|
|
|
const canonicalClaudeModel = findCanonicalClaudeEffortModel(modelId);
|
|
if (canonicalClaudeModel?.supportsXHighEffort !== undefined) {
|
|
return canonicalClaudeModel.supportsXHighEffort !== false;
|
|
}
|
|
|
|
// Keep explicit false entries as the unsupported-model list. Unlisted models
|
|
// and models without an explicit flag pass through unchanged. Unknown
|
|
// providers follow the same rule except for canonical Claude aliases above.
|
|
return true;
|
|
}
|
|
|
|
/** @deprecated Use supportsXHighEffort(); max normalization now follows the same opt-out policy. */
|
|
export function supportsXHighEffortForMaxNormalization(
|
|
aliasOrId: string,
|
|
modelId: string
|
|
): boolean {
|
|
return supportsXHighEffort(aliasOrId, modelId);
|
|
}
|