mirror of
https://github.com/diegosouzapw/OmniRoute.git
synced 2026-09-19 13:23:50 +03:00
4d2abc313e
The fetch-start (headers-wait) phase inherited the flat, non-adaptive FETCH_TIMEOUT_MS (default 600_000ms) with no ceiling comparable to a real client's patience, unlike the body-phase readiness watchdog which already adapts per payload shape. Codex's own hard client-abort window for a stalled turn is ~120s, 5x shorter than the old default — so when an upstream never returned any response at all (not even headers, e.g. a stalled NVIDIA target behind a tool-heavy Responses->Chat translation), OmniRoute kept the connection open on keepalives only, guaranteeing the client gave up first with an opaque 499 instead of OmniRoute detecting and failing the stall fast. Adds resolveFetchStartTimeout() (open-sse/utils/fetchStartTimeoutPolicy.ts) that caps the headers-wait timeout to 110s for STREAMING requests only, leaving non-streaming requests on the existing flat default. Wired into BaseExecutor.execute(). The existing TimeoutError classification path (chatCore.ts) already maps this to a 504, so no change was needed there. Regression test: tests/unit/issue-11526-repro.test.ts.
53 lines
2.4 KiB
TypeScript
53 lines
2.4 KiB
TypeScript
// #11526: the fetch-start (headers-wait) phase had no ceiling comparable to a
|
|
// real client's patience for STREAMING requests — it inherited the flat,
|
|
// non-adaptive FETCH_TIMEOUT_MS (default 600_000ms / 10 minutes), five times
|
|
// longer than Codex's own ~120s hard client-abort window. When an upstream
|
|
// never returns a response at all (not even headers), OmniRoute kept the
|
|
// connection open with nothing but keepalives, guaranteeing the client gave
|
|
// up first with an opaque 499 instead of OmniRoute detecting the stall and
|
|
// failing fast/over within a client-realistic window.
|
|
//
|
|
// This mirrors the adaptive philosophy of streamReadinessPolicy.ts's
|
|
// resolveStreamReadinessTimeout (which already protects the BODY phase, after
|
|
// headers arrive) but inverted: instead of bumping a small base timeout up for
|
|
// heavy payloads, it caps an oversized base timeout down for the HEADERS
|
|
// phase of streaming requests specifically. Non-streaming requests are left
|
|
// on the existing flat default — providers that are legitimately slow to
|
|
// accept a connection (but not streaming SSE) are unaffected.
|
|
|
|
export type FetchStartTimeoutPolicyInput = {
|
|
baseTimeoutMs: number;
|
|
/** Only streaming requests are capped — non-streaming keeps the flat default. */
|
|
stream?: boolean | null;
|
|
capMs?: number;
|
|
};
|
|
|
|
export type FetchStartTimeoutPolicyResult = {
|
|
timeoutMs: number;
|
|
baseTimeoutMs: number;
|
|
/** True when the base timeout was reduced by the streaming cap. */
|
|
capped: boolean;
|
|
};
|
|
|
|
// Codex's documented hard client-abort window for a stalled turn (nothing but
|
|
// keepalives in flight) is ~120s. Keep the cap safely under that so OmniRoute's
|
|
// own headers-phase watchdog always fires before the client gives up on its own.
|
|
export const CODEX_CLIENT_ABORT_MS = 120_000;
|
|
export const DEFAULT_FETCH_START_TIMEOUT_CAP_MS = 110_000;
|
|
|
|
export function resolveFetchStartTimeout(
|
|
input: FetchStartTimeoutPolicyInput
|
|
): FetchStartTimeoutPolicyResult {
|
|
const baseTimeoutMs = Math.max(0, Math.floor(input.baseTimeoutMs || 0));
|
|
if (baseTimeoutMs <= 0 || !input.stream) {
|
|
return { timeoutMs: baseTimeoutMs, baseTimeoutMs, capped: false };
|
|
}
|
|
|
|
const capMs = Math.max(0, Math.floor(input.capMs ?? DEFAULT_FETCH_START_TIMEOUT_CAP_MS));
|
|
if (capMs <= 0 || baseTimeoutMs <= capMs) {
|
|
return { timeoutMs: baseTimeoutMs, baseTimeoutMs, capped: false };
|
|
}
|
|
|
|
return { timeoutMs: capMs, baseTimeoutMs, capped: true };
|
|
}
|