fix(resilience): extend process crash guard to combo hedge cancels and upstream fetch failures (#13636) (#14064)

Closes a real process-killer: the direct-response start timeout could fire after the fetch promise had already settled, and aborting at that point delivered the abort reason to a promise nobody was awaiting — Node promotes that to an `unhandledRejection` → `uncaughtException` and the process dies (#12861). The timer is now a no-op once the attempt has settled, and the same guard is extended to combo hedge cancels and upstream fetch failures.

Validated as a combined board first (this PR merged with the 11 siblings of the same batch on the release tip): eslint on every changed file with the suppressions file, typecheck:core, check:open-sse-typecheck, complexity, cognitive-complexity, changelog-integrity, i18n new-key coverage, docs-sync, migration-numbering, provider-consistency and a duplicate-identifier audit all green, plus 275 passing / 0 failing focused node:test cases across the 28 test files the batch touches. Then re-validated alone on the fresh tip before this merge: conflicts re-resolved, file sizes rebaselined for this PR's own growth, eslint and this PR's focused tests re-run.

Thanks @HouMinXi!

Co-authored-by: Diego Rodrigues de Sa e Souza <diegosouza.pw@gmail.com>
This commit is contained in:
Bob.Hou
2026-09-18 09:21:32 -04:00
committed by GitHub
parent 9b92477b40
commit 815dedd3c9
10 changed files with 522 additions and 18 deletions

View File

@@ -11,6 +11,7 @@
export {
isClientAbortError,
isRecoverableUpstreamTimeoutError,
shouldSwallowUncaught,
attachRequestStreamGuards,
installProcessCrashGuard,

View File

@@ -9,6 +9,17 @@ import headResponseGuard from "./head-response-guard.cjs";
import { resolveTlsOptions, createServerListener } from "./tls-options.mjs";
import { getMainServerTimeoutConfig } from "./main-server-timeouts.mjs";
import { createSystemdNotifier } from "./systemd-notify.mjs";
import { installProcessCrashGuard } from "./httpClientAbortGuard.mjs";
// Safety net (#12861): this is the actual production entry point (see the
// keepAliveTimeout comment below for why `run-next.mjs`-only fixes don't
// reach real installs). Without this, a client abort OR a recoverable
// upstream-fetch timeout that a retry path already handles (see
// open-sse/utils/directResponseStartTimeout.ts) can surface as an
// unhandledRejection -> uncaughtException and take the whole server down —
// exactly the asymmetry `run-next.mjs` already closed for dev. Benign errors
// are swallowed and logged; genuine bugs still crash loudly.
installProcessCrashGuard();
// systemd sd_notify (Type=notify / WatchdogSec=): this process is the one
// whose event loop can freeze (cold /v1/models rebuild), so it must own the