mirror of
https://github.com/diegosouzapw/OmniRoute.git
synced 2026-07-26 09:52:11 +03:00
2 lines
3.7 KiB
JSON
2 lines
3.7 KiB
JSON
{"author":{"id":"MDQ6VXNlcjE0NDAzMDM=","is_bot":false,"login":"matteoantoci","name":"Matteo Antoci"},"body":"## Enhancement: Shorter timeouts for zombie stream detection\n\n### Current behavior\n\nAll timeout-sensitive phases share `FETCH_TIMEOUT_MS` / `STREAM_IDLE_TIMEOUT_MS` (default 10 minutes). Additionally, there is **no request-level deadline** — if a request gets stuck in a pre-fetch phase (rate limiter queue, account semaphore, translation, etc.), it hangs indefinitely with no timeout to recover.\n\nObserved failure modes:\n- Upstream never returns HTTP headers → 10 min hang\n- Upstream returns HTTP 200 but never sends SSE data → 10 min hang before `ensureStreamReadiness` catches it\n- Upstream sends initial content then stalls mid-stream → 10 min hang before idle timeout\n- Request stuck in rate limiter queue or account fallback → **indefinite hang** (no timeout covers this phase)\n\n### Proposed behavior\n\nSplit timeouts into distinct phases, plus add a request-level hard deadline for pre-streaming phases:\n\n| Phase | Proposed default | Current | Env override |\n|-------|-----------------|---------|-------------|\n| **Request deadline** (pre-streaming phases) | 600s | none | `FETCH_TIMEOUT_MS` |\n| Fetch headers (time to HTTP response) | 60s | 600s | `FETCH_HEADERS_TIMEOUT_MS` |\n| First content (time to first useful SSE event) | 60s | 600s | `STREAM_FIRST_CONTENT_TIMEOUT_MS` |\n| Stream idle (mid-stream pauses) | 120s | 600s | `STREAM_IDLE_TIMEOUT_MS` |\n\n**Request deadline** covers the entire pre-streaming lifecycle (rate limiter wait, account fallback, translation, fetch setup, `ensureStreamReadiness`). Once the stream is confirmed active (useful content received), the deadline is cancelled — active streams are governed only by the idle timeout, which resets on every chunk. This means reasoning models that think for 10+ minutes are NOT cut, as long as they send tokens.\n\n### Implementation\n\n1. Add `STREAM_FIRST_CONTENT_TIMEOUT_MS` (default 60s) to `runtimeTimeouts.ts`\n2. Change `FETCH_HEADERS_TIMEOUT_MS` default from `fetchTimeoutMs` to 60s\n3. Lower `DEFAULT_STREAM_IDLE_TIMEOUT_MS` from 600s to 120s\n4. Use `STREAM_FIRST_CONTENT_TIMEOUT_MS` in `ensureStreamReadiness()` call (currently uses `STREAM_IDLE_TIMEOUT_MS`)\n5. Use `FETCH_HEADERS_TIMEOUT_MS` in `BaseExecutor.execute()` fetch-start timeout (currently uses `getTimeoutMs()`)\n6. Add `requestTimeoutMs` option to `createStreamController()` — starts a hard deadline timer when the request begins\n7. Cancel the deadline timer once `ensureStreamReadiness` passes (stream confirmed active)\n8. The deadline uses `FETCH_TIMEOUT_MS` as the default, configurable via env var\n\n### Impact\n\n- Zombie streams detected in ~60s instead of ~10min\n- Mid-stream stalls detected in ~2min instead of ~10min\n- **Pre-fetch hangs (rate limiter, account fallback) now covered** — 600s hard deadline prevents indefinite hangs\n- Combo fallback triggers much faster\n- No change to actively streaming behavior — deadline is cancelled once stream starts, and idle timer resets on every chunk\n- All timeouts are env-overridable for operators who need different values\n\n### Context\n\nObserved with mimo-v2.5-pro via opencode-go in three separate incidents:\n1. Provider returned HTTP 200 but never sent SSE data → 15+ min hang\n2. Provider sent initial SSE content then stalled completely → 13+ min hang\n3. Request stuck after account fallback (429 on first account, second account's request never reached fetch) → 24+ min hang with zero log output after fallback","comments":[],"createdAt":"2026-04-28T09:51:04Z","labels":[],"number":1716,"title":"[Feature] Separate fetch-start and first-content timeouts from stream idle timeout","url":"https://github.com/diegosouzapw/OmniRoute/issues/1716"}
|