OmniRoute/_ideia/_fetch/1814.json

{"author":{"id":"MDQ6VXNlcjQwODM4MTI=","is_bot":false,"login":"apoapostolov","name":"Apostol Apostolov"},"body":"Hermes requests routed through OmniRoute to `glm/glm-5-turbo` can appear stalled for 30-85s before the first useful completion finishes.\n\nObserved on 2026-04-30:\n- `POST /v1/chat/completions`\n- provider: `glm`\n- requested model: `glm/glm-5-turbo`\n- status: `200`\n- duration: `84609 ms` on one call, with many others in the 25-45s range\n- prompt tokens: `68906`\n- completion tokens: `5449`\n- stream: `true`\n- tools: `25`\n- `finish_reason: \"tool_calls\"`\n- request body contained `messages`, `model`, `stream`, `stream_options`, `tools`, and `_omniroute`, but no `max_tokens` or `max_completion_tokens`\n\nThe gateway logs look healthy, so this does not appear to be a crash. The user-visible problem is that requests with very large prompts and no explicit output cap can look like the endpoint is hanging even though they eventually complete.\n\nPossible improvements:\n- add a configurable default output cap when the client omits one\n- surface a warning/metric when streaming requests arrive with no cap and very large prompts\n- add clearer observability around time-to-first-token vs total completion time\n\nI can share the local call-log details if useful.","comments":[],"createdAt":"2026-04-30T10:49:11Z","labels":[],"number":1814,"title":"[Feature] Streamed GLM chat/completions requests can look stalled when no output cap is forwarded","url":"https://github.com/diegosouzapw/OmniRoute/issues/1814"}