mirror of
https://github.com/diegosouzapw/OmniRoute.git
synced 2026-07-26 09:52:11 +03:00
* perf: startup parallelization, stream TextEncoder lift, auth middleware bottlenecks
Startup (~100-300ms faster cold start):
- Parallelize 4 early imports via Promise.all() in registerNodejs()
- Parallelize 10 independent background services via Promise.allSettled()
- Each service has independent try/catch — no failure domino effect
Streaming pipeline (8 fewer TextEncoder GC allocations per SSE event):
- Lift new TextEncoder() from per-chunk inside buildClaudeStreamingResponse
to function scope alongside existing decoder singleton
Auth middleware bottlenecks (from PerfBottleneckAnalysis):
- Backoff decay loop: replace updateProviderConnection (full CRUD:
SELECT+encrypt+cache-invalidate+backup) with resetConnectionBackoff
(targeted UPDATE of backoff/error columns only)
- Dual .filter() for quota: replace two passes calling
isQuotaExhaustedForRequest per connection with a single for loop
partitioning into withQuota/exhaustedQuota
- Debug-log filter recomputation: capture connectionFilterStatus Map
during the filter pass; debug loop reads 6 string comparisons instead
of 6 function calls per connection
Supporting:
- Add resetConnectionBackoff to src/lib/db/providers.ts (patterned after
clearConnectionErrorIfUnchanged, no CAS check)
- Re-export resetConnectionBackoff from src/lib/localDb.ts
- Update integration-wiring.test.ts regex for parallelized dynamic import
* perf: P2C quota cache, lazy provider init, structuredClone elimination, getSettings→getCachedSettings
- **auth.ts: P2C quota re-evaluation cache** — quotaResults Map threaded
through selectPoolSubset → compareP2CConnections → getP2CConnectionScore.
Populated during filter + partition passes, eliminating redundant
evaluateQuotaLimitPolicy / isQuotaExhaustedForRequest calls when the
P2C comparator re-evaluates previously-scored connections.
- **constants.ts: lazy PROVIDERS via Proxy** — replaces eager
generateLegacyProviders() + loadProviderCredentials() at module load
with Proxy delegating to deferred init on first property access.
- **providerModels.ts: lazy PROVIDER_MODELS + PROVIDER_ID_TO_ALIAS** —
same Proxy pattern for both exports; generateModels()/generateAliasMap()
deferred until first read.
- **stream.ts: structuredClone → minimal object spread** — replaces
O(n) deep clone of SSE response chunks with targeted reconstruction
of only mutated fields (usage, delta.content, finish_reason).
- **progressTracker.ts: TextDecoder lift** — module-level decoder
instead of per-chunk new TextDecoder().
- **Route files: getSettings() → getCachedSettings()** — 13 API route
files converted from uncached per-request DB reads to TTL-cached
wrapper (5s default), eliminating redundant queries on every request.
- **settings.ts: re-export getCachedSettings** from readCache for
non-localDb consumers.
- **Remove settingsCache.ts** — dead file, no imports reference it.
TS compile: 0 errors. Auth tests: 225/225 pass. Services: 269/269 pass.
* perf: Phase 1 tangible wins — egressCache eviction, mmap_size PRAGMA, composite indexes, proxyFallback lazy import
- egressCache: lazy TTL eviction on getCachedEgressIp access (bounds memory
leak to distinct proxy URLs, typically <100)
- mmap_size: apply stored PRAGMA from key_value table (256MiB default) after
applyStoredDatabaseOptimizationSettings — setting was stored but never applied
- schemaColumns: add idx_uh_provider_model_timestamp (covers getModelLatencyStats)
and idx_pc_provider_auth_type (covers 6+ provider_connections queries)
- proxyFallback: convert static import to dynamic import() inside error handler
(defers 210ms module load from startup to first proxy-retry scenario)
* perf: add dedup expression index, unref() sweep timers
- Add COALESCE expression index idx_uh_dedup on usage_history
matching the exact dedup query pattern. Eliminates FULL TABLE
SCAN on every saveRequestUsage insert.
- Add composite idx_uh_provider_model_timestamp on usage_history.
- Add composite idx_pc_provider_auth_type on provider_connections.
- Add .unref() to setInterval in batchProcessor.ts (polling loop).
- Add .unref() to setInterval in runtimeHeartbeat.ts (heartbeat).
* perf: bump SQLite cache_size default from 16MB to 64MB
New installs now start with 64MB page cache (was 16MB). Existing
users' stored settings are unchanged. Reduces disk reads for the
typical ~250MB database by keeping ~25% of pages in memory.
Also resolved pre-existing merge conflict in webhooks.ts.
* docs: add Redis production config guide and proxy port clash investigation report
- docs/redis-production-config.md: comprehensive Redis tuning guide
covering client options, server config, Docker settings, scaling,
and monitoring for all three Redis workloads (rate limiting,
auth cache, quota store)
- docs/proxy-port-clash-report.md: investigation confirming proxy
subsystem has no port binding issues; real EADDRINUSE history
traced to process supervisor crash-loop restart race (#4425) and
live-dashboard port clash (#6324), both already fixed
* fix: address PR #7893 review — add Proxy traps, extract migrations to reduce providers.ts size
- Add set trap to PROVIDER_ID_TO_ALIAS Proxy (providerModels.ts)
- Add deleteProperty traps to all three lazy Proxies (PROVIDERS,
PROVIDER_MODELS, PROVIDER_ID_TO_ALIAS)
- Extract autoMigrateLegacyEncryptedConnections and getGheCopilotHosts
from providers.ts (1129→1036 lines, -93) into providers/migrations.ts
- Both functions re-exported via providers.ts for backward compat
File-size ratchet resolved: src/lib/db/providers.ts now 1036 lines.
* fix: resolve merge conflict markers in 3 route/test files
- model-combo-mappings/route.ts: kept upstream version (Zod pagination
via validateBody + isValidationFailure), restored missing return
statement for GET handler
- playground/presets/route.ts: kept stashed version details (satisfies
type-narrowing + inlined Response) — functionally identical
- error-sanitization.test.ts: matches upstream exactly (no diff)
Test verification: same 7 pre-existing failures confirmed on upstream
baseline (c1bdd91e7). Zero regressions from conflict resolution.
Closes remaining uncommitted work from PR #7046 rebase.
* test(db): add resetConnectionBackoff coverage + fix file-size ratchet regression
- Add tests/unit/reset-connection-backoff.test.ts: covers the new
resetConnectionBackoff lightweight-UPDATE helper (clears backoff/error
columns and re-activates a connection, unconditional-write behavior on
terminal statuses, no-op for empty/unknown ids). Zero prior coverage
per pre-merge review of PR #7893 (Hard Rule #8).
- open-sse/services/batchProcessor.ts: fold the new unref() call into the
existing setInterval(...).unref() chain instead of a separate statement,
keeping the file at the frozen 915-line ratchet (Timeout.unref() already
returns `this`, so no type cast is needed).
- src/lib/localDb.ts: drop one blank separator line so the new
resetConnectionBackoff re-export stays within the frozen 808-line ratchet.
Pre-merge fix for PR #7893 (perf/startup-stream-auth-optimizations) per
/green-prs plan-file _tasks/pipeline/prs/1-analyzed/7893-*.plan.md.
Co-authored-by: Diego Rodrigues de Sa e Souza <diegosouzapw@users.noreply.github.com>
---------
Co-authored-by: oyi77 <oyi77@users.noreply.github.com>
Co-authored-by: Diego Rodrigues de Sa e Souza <diegosouza.pw@gmail.com>
Co-authored-by: Diego Rodrigues de Sa e Souza <diegosouzapw@users.noreply.github.com>
143 lines
5.2 KiB
TypeScript
143 lines
5.2 KiB
TypeScript
/**
|
|
* GET /api/playground/presets — list presets
|
|
* POST /api/playground/presets — create a preset
|
|
*
|
|
* Auth: optional (extractApiKey + isValidApiKey; required when REQUIRE_API_KEY=true).
|
|
* Hard Rule #5: all DB access via src/lib/db/playgroundPresets (never raw SQL).
|
|
* Hard Rule #12: all error paths via buildErrorBody.
|
|
*/
|
|
|
|
import { buildErrorBody, sanitizeErrorMessage } from "@omniroute/open-sse/utils/error.ts";
|
|
import { HTTP_STATUS } from "@omniroute/open-sse/config/constants.ts";
|
|
import { extractApiKey, isValidApiKey } from "@/sse/services/auth";
|
|
import { requireManagementAuth } from "@/lib/api/requireManagementAuth";
|
|
import { listPlaygroundPresets, createPlaygroundPreset } from "@/lib/db/playgroundPresets";
|
|
import { PlaygroundPresetCreateSchema } from "@/shared/schemas/playground";
|
|
import { isRequireApiKeyEnabled } from "@/shared/utils/featureFlags";
|
|
import { paginationSchema } from "@/shared/validation/schemas";
|
|
import { isValidationFailure, validateBody } from "@/shared/validation/helpers";
|
|
|
|
const CORS_HEADERS = {
|
|
"Access-Control-Allow-Methods": "GET, POST, OPTIONS",
|
|
"Access-Control-Allow-Headers": "*",
|
|
};
|
|
|
|
function errorResp(status: number, message: string): Response {
|
|
return new Response(JSON.stringify(buildErrorBody(status, sanitizeErrorMessage(message))), {
|
|
status,
|
|
headers: { "Content-Type": "application/json", ...CORS_HEADERS },
|
|
});
|
|
}
|
|
|
|
async function checkAuth(request: Request): Promise<Response | null> {
|
|
const apiKeyRaw = extractApiKey(request);
|
|
// External-client path: if an API key is presented it MUST be valid.
|
|
if (apiKeyRaw) {
|
|
if (!(await isValidApiKey(apiKeyRaw))) {
|
|
return errorResp(HTTP_STATUS.UNAUTHORIZED, "Invalid API key");
|
|
}
|
|
return null;
|
|
}
|
|
// Dashboard path: the Playground page itself calls this route with a
|
|
// cookie/session (no API key). Under REQUIRE_API_KEY=true that previously
|
|
// 401'd the authenticated dashboard (QA P1: preset auth mismatch). Accept a
|
|
// valid management/dashboard session as an alternative to an API key.
|
|
const managementError = await requireManagementAuth(request);
|
|
if (!managementError) return null;
|
|
// Neither an API key nor a valid dashboard session: enforce only when
|
|
// API-key enforcement is on (preserves the legacy anonymous-allowed default).
|
|
if (isRequireApiKeyEnabled()) {
|
|
return errorResp(HTTP_STATUS.UNAUTHORIZED, "Authentication required");
|
|
}
|
|
return null;
|
|
}
|
|
|
|
export async function OPTIONS(): Promise<Response> {
|
|
return new Response(null, { headers: CORS_HEADERS });
|
|
}
|
|
|
|
/**
|
|
* GET /api/playground/presets
|
|
* Returns { presets: PlaygroundPresetListItem[] }
|
|
*/
|
|
export async function GET(request: Request): Promise<Response> {
|
|
const authError = await checkAuth(request);
|
|
if (authError) return authError;
|
|
|
|
try {
|
|
const { searchParams } = new URL(request.url);
|
|
const raw = {
|
|
offset: searchParams.get("offset") || undefined,
|
|
limit: searchParams.get("limit") || undefined,
|
|
} satisfies { offset?: string; limit?: string };
|
|
const validation = validateBody(paginationSchema, raw);
|
|
if (isValidationFailure(validation)) {
|
|
return new Response(JSON.stringify(validation.error), {
|
|
status: 400,
|
|
headers: { "Content-Type": "application/json", ...CORS_HEADERS },
|
|
});
|
|
}
|
|
const { limit, offset } = validation.data;
|
|
const result = listPlaygroundPresets(limit !== undefined ? { limit, offset } : undefined);
|
|
return new Response(JSON.stringify({ presets: result.items, total: result.total }), {
|
|
status: 200,
|
|
headers: { "Content-Type": "application/json", ...CORS_HEADERS },
|
|
});
|
|
} catch (err: unknown) {
|
|
const safeMsg = sanitizeErrorMessage(
|
|
err instanceof Error ? err.message : "Failed to list presets"
|
|
);
|
|
return errorResp(HTTP_STATUS.SERVER_ERROR, safeMsg);
|
|
}
|
|
}
|
|
|
|
/**
|
|
* POST /api/playground/presets
|
|
* Body: PlaygroundPresetCreateSchema
|
|
* Returns the created preset with status 201.
|
|
*/
|
|
export async function POST(request: Request): Promise<Response> {
|
|
const authError = await checkAuth(request);
|
|
if (authError) return authError;
|
|
|
|
// 1. Parse JSON body
|
|
let rawBody: unknown;
|
|
try {
|
|
rawBody = await request.json();
|
|
} catch {
|
|
return errorResp(HTTP_STATUS.BAD_REQUEST, "Invalid JSON body");
|
|
}
|
|
|
|
// 2. Validate (Hard Rule #7)
|
|
const parsed = PlaygroundPresetCreateSchema.safeParse(rawBody);
|
|
if (!parsed.success) {
|
|
const firstIssue = parsed.error.issues[0];
|
|
const message = firstIssue
|
|
? `${firstIssue.path.join(".") || "body"}: ${firstIssue.message}`
|
|
: "Invalid request body";
|
|
return errorResp(HTTP_STATUS.BAD_REQUEST, message);
|
|
}
|
|
const body = parsed.data;
|
|
|
|
// 3. Create preset (Hard Rule #5 — via DB module)
|
|
try {
|
|
const created = createPlaygroundPreset({
|
|
name: body.name,
|
|
endpoint: body.endpoint,
|
|
model: body.model,
|
|
system: body.system ?? null,
|
|
params: body.params,
|
|
});
|
|
|
|
return new Response(JSON.stringify(created), {
|
|
status: 201,
|
|
headers: { "Content-Type": "application/json", ...CORS_HEADERS },
|
|
});
|
|
} catch (err: unknown) {
|
|
const safeMsg = sanitizeErrorMessage(
|
|
err instanceof Error ? err.message : "Failed to create preset"
|
|
);
|
|
return errorResp(HTTP_STATUS.SERVER_ERROR, safeMsg);
|
|
}
|
|
}
|