pjandro/OmniRoute
1
0
Fork 0
mirror of https://github.com/diegosouzapw/OmniRoute.git synced 2026-09-17 12:22:34 +03:00
Code Issues Packages Projects Releases Wiki Activity

fix(resilience): increase requestQueue.maxWaitMs default from 15s to 30s (#13553)

Browse Source 
Raises the default request-queue wait (`RATE_LIMIT_MAX_WAIT_MS`) from 15s to 30s, so bursts behind a rate-limited provider queue a bit longer before being rejected (#13504). The env var still overrides it, and `executionMaxWaitMs` (the 10-minute execution backstop) is unchanged. Approved by the maintainer as a policy change.

Maintainer additions: updated the documented default in `.env.example` and `docs/reference/ENVIRONMENT.md` to match.

Validated in one consolidated batch of this series (37 PRs boarded together on `release/v3.8.51`): `typecheck:core`, `check:open-sse-typecheck` and `check:dashboard-typecheck` clean; ESLint clean on every changed file; file-size, complexity, cognitive-complexity, changelog-integrity, docs-counts, docs-sync and migration-numbering gates green (only the pre-existing `open-sse/utils/stream.ts` file-size red remains, inherited from the base); 3,743 focused `node:test` cases plus 34 vitest cases green.

Thanks @KooshaPari!
This commit is contained in:
Koosha Paridehpour
2026-09-14 19:53:18 -07:00
committed by GitHub
parent d4a1470e40
commit 918546acf2
4 changed files with 8 additions and 8 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
.env.example
View File

@@ -2204,9 +2204,9 @@ APP_LOG_TO_FILE=true
# Also configurable from Dashboard > Settings > Feature Flags.
# OMNIROUTE_CONTROL_PLANE_PROXY_DIRECT_FALLBACK=false
# Rate limit maximum wait time before failing a request (ms). Default: 15000 (15s)
# Rate limit maximum wait time before failing a request (ms). Default: 30000 (30s)
# Used by: open-sse/services/rateLimitManager.ts
# RATE_LIMIT_MAX_WAIT_MS=15000
# RATE_LIMIT_MAX_WAIT_MS=30000
# Limiter-managed execution backstop (Bottleneck `expiration`): bounds a job's
# post-dispatch execution, never queue wait. Must stay ABOVE upstream

2
docs/reference/ENVIRONMENT.md
View File

@@ -1116,7 +1116,7 @@ Anthropic-compatible provider instead.
| `PROXY_AUTO_REMOVE_AFTER` | `3` | `src/lib/proxyHealth/scheduler.ts` | Consecutive failures before the scheduler auto-removes a proxy (when `PROXY_AUTO_REMOVE=true`). |
| `PROXY_AUTO_DISABLE` | `false` | `src/lib/proxyHealth/scheduler.ts` | Set `true` to let the scheduler soft-disable (status `dead`, never deleted) a proxy after repeated consecutive failures, instead of removing it. Non-destructive alternative to `PROXY_AUTO_REMOVE`: the proxy drops out of pool/rotation resolution immediately (the alive-status filter used by scope-pool resolution already excludes it) and is automatically re-activated once it starts passing probes again. Shares the `PROXY_AUTO_REMOVE_AFTER` threshold. If both flags are `true`, `PROXY_AUTO_REMOVE` wins. |
| `OMNIROUTE_CONTROL_PLANE_PROXY_DIRECT_FALLBACK` | `false` | `src/shared/constants/featureFlagDefinitions.ts` | Allow OAuth and provider validation flows to bypass a pinned proxy and connect directly when proxy reachability pre-checks fail. Effective precedence is Feature Flags DB override > env var > default. |
| `RATE_LIMIT_MAX_WAIT_MS` | `15000` (15s) | `open-sse/services/rateLimitManager.ts` | Max time to wait on a 429 before failing the request. |
| `RATE_LIMIT_MAX_WAIT_MS` | `30000` (30s) | `open-sse/services/rateLimitManager.ts` | Max time to wait on a 429 before failing the request. |
| `RATE_LIMIT_EXECUTION_MAX_WAIT_MS` | `600000` (10 min) | `open-sse/services/rateLimitManager.ts` | Ceiling for how long an admitted request may stay in execution before its rate-limit reservation expires — decoupled from the queue-wait budget so slow fetch-start on non-incremental gateways does not time out (#12027). |
| `RATE_LIMIT_MAX_QUEUE_DEPTH` | `0` (disabled) | `open-sse/services/rateLimitManager.ts` | Queue admission cap: reject with a 429 `queue_full` once this many requests are already queued. `0` = unbounded (default). |
| `RATE_LIMIT_AUTO_ENABLE` | _(unset)_ | `open-sse/services/rateLimitManager.ts` | Force the auto-enable rate limit safety net on/off regardless of the persisted Dashboard setting. Accepts `true`/`1`/`on` to force on, `false`/`0`/`off` to force off. |

4
src/lib/resilience/settings.ts
View File

@@ -43,8 +43,8 @@ export type {
} from "./settings/types";
export const DEFAULT_REQUEST_QUEUE_MAX_WAIT_MS = (() => {
const parsed = Number(process.env.RATE_LIMIT_MAX_WAIT_MS || "15000");
return Number.isFinite(parsed) && parsed > 0 ? Math.trunc(parsed) : 15000;
const parsed = Number(process.env.RATE_LIMIT_MAX_WAIT_MS || "30000");
return Number.isFinite(parsed) && parsed > 0 ? Math.trunc(parsed) : 30000;
})();
// Limiter-managed execution backstop (Bottleneck `expiration`). Deliberately

6
tests/unit/ratelimit-admission-control-6593.test.ts
View File

@@ -177,10 +177,10 @@ test("#6593 withRateLimit: default maxQueueDepth=0 preserves unbounded-queue beh
// --- Default maxWaitMs ----------------------------------------------------
test("#6593 DEFAULT_REQUEST_QUEUE_MAX_WAIT_MS is 15s absent RATE_LIMIT_MAX_WAIT_MS", () => {
test("#6593 DEFAULT_REQUEST_QUEUE_MAX_WAIT_MS is 30s absent RATE_LIMIT_MAX_WAIT_MS", () => {
assert.equal(process.env.RATE_LIMIT_MAX_WAIT_MS, undefined);
assert.equal(resilienceSettings.DEFAULT_REQUEST_QUEUE_MAX_WAIT_MS, 15000);
assert.equal(resilienceSettings.DEFAULT_RESILIENCE_SETTINGS.requestQueue.maxWaitMs, 15000);
assert.equal(resilienceSettings.DEFAULT_REQUEST_QUEUE_MAX_WAIT_MS, 30000);
assert.equal(resilienceSettings.DEFAULT_RESILIENCE_SETTINGS.requestQueue.maxWaitMs, 30000);
});
test("requestQueue.executionMaxWaitMs defaults to a 10-minute backstop, separate from maxWaitMs", () => {