pjandro/OmniRoute
1
0
Fork 0
mirror of https://github.com/diegosouzapw/OmniRoute.git synced 2026-08-15 11:43:10 +03:00
Code Issues Packages Projects Releases Wiki Activity

docs(env): document OMNIROUTE_CHAT_ADMISSION_HEAP_SHED_RATIO (#10183, #10268)

Browse Source
This commit is contained in:
adevwithpurpose
2026-08-15 03:48:51 -03:00
parent 0a9df1e10d
commit 4eeab0facc
2 changed files with 6 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

5
.env.example
View File

@@ -350,6 +350,11 @@ ALLOW_API_KEY_REVEAL=false
# OMNIROUTE_CHAT_HARD_MAX_BODY_BYTES=52428800
# Maximum heavyweight requests simultaneously admitted in one process. Default 1.
# OMNIROUTE_CHAT_MAX_HEAVY_IN_FLIGHT=1
# Heap-pressure shed ratio (heapUsed/heap_size_limit) for the structural admission gate
# (#10183, #10268): a second concurrent heavyweight request past OMNIROUTE_CHAT_MAX_HEAVY_IN_FLIGHT
# is only shed with a retryable 503 when the heap is ALSO under this much pressure — on a
# healthy heap it is admitted instead. Range (0, 1]. Default 0.75.
# OMNIROUTE_CHAT_ADMISSION_HEAP_SHED_RATIO=0.75
# Message count that classifies an otherwise small body as heavyweight. Default 200.
# OMNIROUTE_CHAT_HEAVY_MESSAGE_COUNT=200
# Tool count that classifies an otherwise small body as heavyweight. Default 64.

1
docs/reference/ENVIRONMENT.md
View File

@@ -192,6 +192,7 @@ OmniRoute uses **SQLite** (via `better-sqlite3`) for all persistence. These vari
| `OMNIROUTE_CHAT_LARGE_BODY_BYTES` | `262144` (256 KB) | `src/shared/middleware/chatBodyAdmission.ts` | Actual request bodies at or above this threshold require an atomic process-local heavyweight admission lease before JSON parsing. |
| `OMNIROUTE_CHAT_HARD_MAX_BODY_BYTES` | `52428800` (50 MB) | `src/shared/middleware/chatBodyAdmission.ts` | Chat-route hard cap enforced against bytes read during bounded ingestion, including requests with missing, invalid, or dishonest `Content-Length`; excess receives `413`. |
| `OMNIROUTE_CHAT_MAX_HEAVY_IN_FLIGHT` | `1` | `src/shared/middleware/chatBodyAdmission.ts` | Maximum heavyweight chat requests admitted concurrently in one process. When capacity is unavailable, OmniRoute returns retryable `503` with `Retry-After`. |
| `OMNIROUTE_CHAT_ADMISSION_HEAP_SHED_RATIO` | `0.75` | `src/shared/middleware/chatBodyAdmission.ts` | Heap-pressure shed ratio (`heapUsed / heap_size_limit`) for the structural admission gate (#10183, #10268). A second concurrent heavyweight request past `OMNIROUTE_CHAT_MAX_HEAVY_IN_FLIGHT` is only shed with the retryable `503` when the heap is ALSO at or above this ratio; on a healthy heap it is admitted instead. |
| `OMNIROUTE_CHAT_HEAVY_MESSAGE_COUNT` | `200` | `src/shared/middleware/chatBodyAdmission.ts` | Message count that classifies a chat request as heavyweight even when its body is below the byte threshold. |
| `OMNIROUTE_CHAT_HEAVY_TOOL_COUNT` | `64` | `src/shared/middleware/chatBodyAdmission.ts` | Tool count that classifies a chat request as heavyweight even when its body is below the byte threshold. |
| `OMNIROUTE_CHAT_HEAVY_ESTIMATED_TOKENS` | `32000` | `src/shared/middleware/chatBodyAdmission.ts` | Conservative string-size token estimate that classifies a request as heavyweight; this is an admission-cost proxy, not provider billing tokenization. |