mirror of
https://github.com/diegosouzapw/OmniRoute.git
synced 2026-07-31 04:12:10 +03:00
0ecc380928
Phase 1 of client-side quota tracking for NVIDIA NIM (no rate-limit headers, no usage API): - Register nvidia in PROVIDER_DEFAULT_RATE_LIMITS (40 RPM sliding window, matching the documented free-tier note), operator-overridable via a new ResilienceSettings.providerQuotaOverrides map. - Per-connection concurrency cap (default 6) via a new nvidiaConcurrencyGate leaf module wrapping rateLimitSemaphore, wired into DefaultExecutor.execute(). - Per-model 429 lockout: confirmed already satisfied by #6773's passthroughModels flag on the nvidia registry entry (no new code needed) — added as a regression-guard test instead. Phase 2 (AIMD adaptive ceiling learning) and Phase 3 (dashboard quota card + combo-routing headroom preference) are explicitly deferred to follow-up issues, per the plan's own scope note.