Files
OmniRoute/tests
Nick Sullivan 7b8055c7f8 fix(resilience): count STREAM_EARLY_EOF as a provider failure in combo routing (#9251)
* fix(resilience): count STREAM_EARLY_EOF as a provider failure in combo routing

A STREAM_EARLY_EOF is an upstream that accepted the request (HTTP 200), opened
the SSE stream, then closed it without emitting a single non-ping event. The
combo path classified it together with STREAM_READINESS_TIMEOUT through
isStreamReadinessFailureErrorBody(), and the readiness exemption in
shouldRecordProviderBreakerFailure meant the whole-provider circuit breaker
never saw it.

During a provider-wide outage that makes the breaker blind. Over a 7-day window
on our router we recorded 311 of these events, 302 of them on one model, 265
inside the upstream's published incident window — and the provider breaker sat
at CLOSED / failure_count=0 the entire time. Every request kept being dispatched
to the failing provider instead of shedding to the next combo target.

The two codes are different signals. The readiness probe is a pre-flight
liveness check on a connection we have not committed to, so failing it means
"this connection looks stale". An early EOF means the provider took the request
and then failed to serve it. The single-model path already treats it that way:
shouldTripProviderBreakerForResult has no readiness exemption, so a 502 early
EOF trips the breaker there. This makes the combo path consistent.

isStreamReadinessFailureErrorBody keeps matching both codes, because the
transient-retry and round-robin semaphore-cooldown paths in combo.ts do want
identical treatment for both. Only the breaker needs to tell them apart, so the
distinction is added as a narrow predicate and an optional argument rather than
by changing the shared classifier. Omitting the new argument reproduces the
previous behaviour exactly.

Follows the additive-override pattern established by the isProxyUnreachable
work, and leaves the existing exclusions for client aborts and plain 429s
untouched.

* test: register stream-early-eof-breaker in stryker tap.testFiles

The mutation test-coverage gate (check:mutation-test-coverage --strict)
detects unit tests that cover a mutated module but are missing from
stryker.conf.json tap.testFiles, so their mutant kills would not count.

comboPredicates.ts is one of the mutated modules, and the new
stream-early-eof-breaker.test.ts covers it, so the gate correctly flagged
the omission. 8376-econnrefused-breaker.test.ts -- the test this one is
modeled on -- is already registered; this just brings the new file in line.

No production code change.

---------

Co-authored-by: Nick Sullivan <nick@technick.ai>
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
2026-08-04 18:07:09 -03:00
..
2026-07-29 15:18:55 -03:00
2026-07-29 15:18:55 -03:00
2026-05-23 01:46:59 -03:00
2026-07-13 09:12:40 -03:00
2026-07-29 15:18:55 -03:00
2026-07-29 15:18:55 -03:00
2026-07-29 15:18:55 -03:00
2026-07-06 02:25:17 -03:00
2026-07-29 15:18:55 -03:00
2026-07-29 15:18:55 -03:00
2026-07-29 15:18:55 -03:00
2026-07-29 15:18:55 -03:00
2026-06-13 17:27:40 -03:00