From 4eeab0facc39c5b2af85d35b93d5cb0a641a6f11 Mon Sep 17 00:00:00 2001 From: adevwithpurpose Date: Sat, 15 Aug 2026 03:48:51 -0300 Subject: [PATCH] docs(env): document OMNIROUTE_CHAT_ADMISSION_HEAP_SHED_RATIO (#10183, #10268) --- .env.example | 5 +++++ docs/reference/ENVIRONMENT.md | 1 + 2 files changed, 6 insertions(+) diff --git a/.env.example b/.env.example index f984a0ea4d..2522ad468d 100644 --- a/.env.example +++ b/.env.example @@ -350,6 +350,11 @@ ALLOW_API_KEY_REVEAL=false # OMNIROUTE_CHAT_HARD_MAX_BODY_BYTES=52428800 # Maximum heavyweight requests simultaneously admitted in one process. Default 1. # OMNIROUTE_CHAT_MAX_HEAVY_IN_FLIGHT=1 +# Heap-pressure shed ratio (heapUsed/heap_size_limit) for the structural admission gate +# (#10183, #10268): a second concurrent heavyweight request past OMNIROUTE_CHAT_MAX_HEAVY_IN_FLIGHT +# is only shed with a retryable 503 when the heap is ALSO under this much pressure — on a +# healthy heap it is admitted instead. Range (0, 1]. Default 0.75. +# OMNIROUTE_CHAT_ADMISSION_HEAP_SHED_RATIO=0.75 # Message count that classifies an otherwise small body as heavyweight. Default 200. # OMNIROUTE_CHAT_HEAVY_MESSAGE_COUNT=200 # Tool count that classifies an otherwise small body as heavyweight. Default 64. diff --git a/docs/reference/ENVIRONMENT.md b/docs/reference/ENVIRONMENT.md index fbaa8357b8..33ddb2caa9 100644 --- a/docs/reference/ENVIRONMENT.md +++ b/docs/reference/ENVIRONMENT.md @@ -192,6 +192,7 @@ OmniRoute uses **SQLite** (via `better-sqlite3`) for all persistence. These vari | `OMNIROUTE_CHAT_LARGE_BODY_BYTES` | `262144` (256 KB) | `src/shared/middleware/chatBodyAdmission.ts` | Actual request bodies at or above this threshold require an atomic process-local heavyweight admission lease before JSON parsing. | | `OMNIROUTE_CHAT_HARD_MAX_BODY_BYTES` | `52428800` (50 MB) | `src/shared/middleware/chatBodyAdmission.ts` | Chat-route hard cap enforced against bytes read during bounded ingestion, including requests with missing, invalid, or dishonest `Content-Length`; excess receives `413`. | | `OMNIROUTE_CHAT_MAX_HEAVY_IN_FLIGHT` | `1` | `src/shared/middleware/chatBodyAdmission.ts` | Maximum heavyweight chat requests admitted concurrently in one process. When capacity is unavailable, OmniRoute returns retryable `503` with `Retry-After`. | +| `OMNIROUTE_CHAT_ADMISSION_HEAP_SHED_RATIO` | `0.75` | `src/shared/middleware/chatBodyAdmission.ts` | Heap-pressure shed ratio (`heapUsed / heap_size_limit`) for the structural admission gate (#10183, #10268). A second concurrent heavyweight request past `OMNIROUTE_CHAT_MAX_HEAVY_IN_FLIGHT` is only shed with the retryable `503` when the heap is ALSO at or above this ratio; on a healthy heap it is admitted instead. | | `OMNIROUTE_CHAT_HEAVY_MESSAGE_COUNT` | `200` | `src/shared/middleware/chatBodyAdmission.ts` | Message count that classifies a chat request as heavyweight even when its body is below the byte threshold. | | `OMNIROUTE_CHAT_HEAVY_TOOL_COUNT` | `64` | `src/shared/middleware/chatBodyAdmission.ts` | Tool count that classifies a chat request as heavyweight even when its body is below the byte threshold. | | `OMNIROUTE_CHAT_HEAVY_ESTIMATED_TOKENS` | `32000` | `src/shared/middleware/chatBodyAdmission.ts` | Conservative string-size token estimate that classifies a request as heavyweight; this is an admission-cost proxy, not provider billing tokenization. |