mirror of
https://github.com/diegosouzapw/OmniRoute.git
synced 2026-09-17 12:22:34 +03:00
d8c0448293
Behind the new `OPENCODE_RESPONSES_STALL_ROTATION` flag (default off): a streamed Responses reply with no first body byte within `RESPONSES_FIRST_BYTE_TIMEOUT_MS` (15s) cools the account and rotates once; a second stall fails fast instead of waiting the 80s readiness timeout.
Maintainer rework before merge (kept the idea, no default behavior change):
- The TLS first-byte watchdog from #12656 is restored byte for byte (the PR had changed its pump, timer and cancel); the stall guard lives in its own module.
- Proxy-less multi-account setups now rotate the same way as proxied ones (the original threw for them), a client abort during the wait rethrows instead of rotating, and the env var is documented as flag-only.
Validated first on the combined board of all 38 PRs of this batch (10 merged as-is, 28 after the maintainer rework) on top of release/v3.8.51 c0f92ec: typecheck:core, check:open-sse-typecheck and check:dashboard-typecheck clean; ESLint clean on every changed file; file-size (rebaselined for the combined growth), complexity, cognitive-complexity, changelog-integrity, docs-counts, docs-sync, migration-numbering and i18n new-key gates green; 735 focused node:test cases with the only batch-caused failure (a flag-count assertion) fixed. Then re-validated alone on the fresh release tip right before this merge: ESLint on the changed files, typecheck:core, check:open-sse-typecheck, the file-size/complexity/changelog gates and this PR's own tests.
Thanks @maxmad64bis!
49 lines
2.0 KiB
TypeScript
49 lines
2.0 KiB
TypeScript
/**
|
|
* opencodeResponsesStall.ts — opt-in first-byte stall guard for streamed
|
|
* Responses replies in the opencode executor (#13484).
|
|
*
|
|
* A streamed Responses reply opens with `response.created` before any
|
|
* generation, so a 2xx Responses stream that stays silent past the window is
|
|
* stalled, not thinking. Chat Completions streams are left alone: gateways may
|
|
* legitimately hold them until the answer is ready.
|
|
*
|
|
* Gated by OPENCODE_RESPONSES_STALL_ROTATION (default off). With the flag off
|
|
* the window is 0 and every guard call hands back the very same result object,
|
|
* so the stream readiness timeout stays the only bound, as before.
|
|
*/
|
|
|
|
import { isOpencodeResponsesStallRotationEnabled } from "@/shared/utils/featureFlags";
|
|
import { getResponsesFirstByteTimeoutMs } from "@/shared/utils/runtimeTimeouts";
|
|
import { guardResponsesStreamFirstByte } from "../utils/firstByteWatchdog.ts";
|
|
|
|
export { isResponsesFirstByteTimeout } from "../utils/firstByteWatchdog.ts";
|
|
|
|
/** First-byte window (ms) for this request, or 0 when the guard does not apply. */
|
|
export function resolveResponsesStallWindowMs(
|
|
stream: boolean | undefined,
|
|
requestFormat: string | null
|
|
): number {
|
|
if (!stream || requestFormat !== "openai-responses") return 0;
|
|
if (!isOpencodeResponsesStallRotationEnabled()) return 0;
|
|
return getResponsesFirstByteTimeoutMs();
|
|
}
|
|
|
|
/**
|
|
* Returns `result` itself when `windowMs` is 0 or it carries no 2xx body;
|
|
* otherwise resolves once the first body byte arrives, or throws
|
|
* RESPONSES_FIRST_BYTE_TIMEOUT (or the abort reason when `signal` fires).
|
|
*/
|
|
export async function guardResponsesStall<T>(
|
|
result: T,
|
|
windowMs: number,
|
|
signal?: AbortSignal | null
|
|
): Promise<T> {
|
|
if (windowMs <= 0 || !result || typeof result !== "object" || !("response" in result)) {
|
|
return result;
|
|
}
|
|
const response = (result as { response: Response }).response;
|
|
if (!response?.ok || !response.body) return result;
|
|
const guarded = await guardResponsesStreamFirstByte(response, windowMs, signal);
|
|
return { ...result, response: guarded };
|
|
}
|