Files
OmniRoute/tests/unit/lib/rate-limit-maxwaitms-disable-execution.test.ts
initguru 4c4d5c7fbe fix(resilience): allow maxWaitMs=0 as disable sentinel for execution expiration (#12902)
* fix(resilience): allow maxWaitMs=0 as disable sentinel for execution expiration

maxWaitMs normalization clamped the value to min:1, silently rewriting
an operator's 0 ("disable the limiter-managed execution deadline") into
1 — a 1ms expiration that killed every long-running job instantly. This
broke long-running reasoning models (GLM-5.2 with reasoning.effort=max
spends minutes before the first token, exceeding any practical
maxWaitMs; the TTB safety net is FETCH_TIMEOUT_MS, default 600s).

Fix: lower the floor to min:0 so 0 is preserved as the disable sentinel.
Issue #4165 follow-up.

Tests: 7/7 (resilience-normalize-maxwaitms-disable 5 + rate-limit-
maxwaitms-disable-execution 2). typecheck:core clean.

* fix(resilience): relax requestQueueSettingsSchema.maxWaitMs to allow 0

normalizeRequestQueueSettings already treats maxWaitMs=0 as an explicit
disable sentinel (queue-wait budget off), but the settings API schema
still rejected 0 with min(1), so an operator could never actually reach
the fix through PATCH /api/resilience. executionMaxWaitMs is untouched
(stays min(1) — separate field, separate decision, see #12902 item 4).

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>

* test(resilience): prove maxWaitMs=0 vs #12715's queue-wait gate behavior

Answers the open technical question from #12902's review: does a
GLOBAL maxWaitMs=0 reintroduce the unbounded-queue regression #12715
fixed (a request hanging ~6min until the client aborts)?

Evidence, exercising the real gate chatCore.ts actually calls
(accountSemaphore.acquireMany({ timeoutMs: requestQueue.maxWaitMs }),
not the Bottleneck reservoir the PR's own tests cover) under real
contention (maxConcurrency=1, two concurrent acquires):

  - No: it does not hang. setTimeout(reject, 0) fires on the next
    tick, so a second contending request is rejected with
    SEMAPHORE_TIMEOUT in low milliseconds, never minutes.
  - But it is also not a genuine 'no cap' — an operator setting 0
    expecting 'wait as long as it takes' instead gets near-zero
    tolerance for even momentary contention on any configured
    concurrency gate (global/provider/account). This is a real
    asymmetry vs. the Bottleneck reservoir path (where 0 truly means
    unbounded) left for the maintainer to decide how to resolve —
    not something this pass can decide unilaterally.

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>

---------

Co-authored-by: Jihyun Son <jihyun.son@sk.com>
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
2026-09-17 10:44:30 -03:00

89 lines
3.0 KiB
TypeScript

/**
* A-1 통합 검증: maxWaitMs=0 disable sentinel 이 withRateLimit 에서
* Bottleneck execution expiration 을 실제로 끄는지 확인.
*
* #4165 테스트는 maxWaitMs=40 + 400ms 작업이 expiration(40ms)에 504 로 죽는
* 것을 검증(positive). 이 테스트는 그 반대(negative):
* maxWaitMs=0 + 400ms 작업은 expiration 이 없으므로 504 없이 완료되어야 함.
* TTB 안전망은 FETCH_TIMEOUT_MS(기본 600s) 가 담당.
*/
import test from "node:test";
import assert from "node:assert/strict";
import fs from "node:fs";
import os from "node:os";
import path from "node:path";
const TEST_DATA_DIR = fs.mkdtempSync(path.join(os.tmpdir(), "omniroute-rl-disable-"));
process.env.DATA_DIR = TEST_DATA_DIR;
const core = await import("../../../src/lib/db/core.ts");
const resilienceSettings = await import("../../../src/lib/resilience/settings.ts");
const rateLimitManager = await import("../../../open-sse/services/rateLimitManager.ts");
function wait(ms: number) {
const { promise, resolve } = Promise.withResolvers<void>();
setTimeout(resolve, ms);
return promise;
}
test.afterEach(async () => {
await rateLimitManager.__resetRateLimitManagerForTests();
});
test.after(() => {
core.resetDbInstance();
fs.rmSync(TEST_DATA_DIR, { recursive: true, force: true });
});
test("maxWaitMs=0 은 Bottleneck execution expiration 을 끈다: 400ms 작업이 504 없이 완료", async () => {
await rateLimitManager.applyRequestQueueSettings({
...resilienceSettings.DEFAULT_RESILIENCE_SETTINGS.requestQueue,
autoEnableApiKeyProviders: false,
concurrentRequests: 1,
requestsPerMinute: 100000,
minTimeBetweenRequestsMs: 0,
maxWaitMs: 0, // disable sentinel — expiration off
});
rateLimitManager.enableRateLimitProtection("conn-disable");
// maxWaitMs=0 이면 expiration 없음 → 400ms 작업이 완료되어야 함
// (대조: #4165 에서 maxWaitMs=40 이면 400ms 작업이 504 로 죽음)
const result = await rateLimitManager.withRateLimit(
"openai",
"conn-disable",
"gpt-4o",
async () => {
await wait(400);
return "completed-despite-long-job";
}
);
assert.equal(result, "completed-despite-long-job");
});
test("maxWaitMs=0 일 때 RATE_LIMIT_EXECUTION_TIMEOUT 코드가 발생하지 않는다", async () => {
await rateLimitManager.applyRequestQueueSettings({
...resilienceSettings.DEFAULT_RESILIENCE_SETTINGS.requestQueue,
autoEnableApiKeyProviders: false,
concurrentRequests: 1,
requestsPerMinute: 100000,
minTimeBetweenRequestsMs: 0,
maxWaitMs: 0,
});
rateLimitManager.enableRateLimitProtection("conn-disable-code");
let code: string | undefined;
try {
await rateLimitManager.withRateLimit("openai", "conn-disable-code", "gpt-4o", async () => {
await wait(200);
return "ok";
});
} catch (err) {
code = (err as Error & { code?: string }).code;
}
assert.notEqual(
code,
"RATE_LIMIT_EXECUTION_TIMEOUT",
"maxWaitMs=0 일 때는 execution timeout 코드가 나면 안 됨"
);
});