From 326d0e81cb385cdacb83ae014870234c15be99a7 Mon Sep 17 00:00:00 2001 From: Ravi Tharuma <25951435+RaviTharuma@users.noreply.github.com> Date: Sun, 16 Aug 2026 05:42:31 +0200 Subject: [PATCH] docs(ops): k8s probe recommendations (TCP liveness, HTTP /healthz readiness) (#10297) * docs(ops): recommend TCP liveness and HTTP /healthz readiness for k8s Stock Docker HEALTHCHECK hits /api/monitoring/health (deep). Orchestrators should not use that path for kubelet liveness. Document /healthz vs deep health, note same-process event-loop limits, and link related issues. * docs: add changelog fragment for #10297 --------- Co-authored-by: Ravi Tharuma Co-authored-by: diegosouzapw --- .../10297-k8s-probe-recommendations.md | 1 + docs/guides/DOCKER_GUIDE.md | 17 ++++- docs/ops/MONITORING_GUIDE.md | 70 +++++++++++++++++-- 3 files changed, 83 insertions(+), 5 deletions(-) create mode 100644 changelog.d/maintenance/10297-k8s-probe-recommendations.md diff --git a/changelog.d/maintenance/10297-k8s-probe-recommendations.md b/changelog.d/maintenance/10297-k8s-probe-recommendations.md new file mode 100644 index 0000000000..adc9d4491e --- /dev/null +++ b/changelog.d/maintenance/10297-k8s-probe-recommendations.md @@ -0,0 +1 @@ +- **docs(ops):** document Kubernetes probe recommendations — TCP (or soft HTTP) liveness, HTTP `/healthz` readiness, avoid `/api/monitoring/health` as kubelet liveness ([#10297](https://github.com/diegosouzapw/OmniRoute/pull/10297)) — thanks @RaviTharuma diff --git a/docs/guides/DOCKER_GUIDE.md b/docs/guides/DOCKER_GUIDE.md index 523d87bf77..2fb1b75f79 100644 --- a/docs/guides/DOCKER_GUIDE.md +++ b/docs/guides/DOCKER_GUIDE.md @@ -326,7 +326,22 @@ prefix). Traefik should route `PathPrefix(`/omniroute`)` to the container withou `/omniroute/_next/...`. The Docker healthcheck probes `/api/monitoring/health` prefixed with the active -`OMNIROUTE_BASE_PATH`. +`OMNIROUTE_BASE_PATH`. That path is a **deep** check (DB + monitoring summary). It is +appropriate for Docker’s infrequent `HEALTHCHECK`, but **not** for Kubernetes +`livenessProbe` intervals. + +For orchestrators (Kubernetes, Nomad, etc.): + +| Probe | Prefer | Avoid | +| --- | --- | --- | +| Liveness | TCP on the main port (`PORT`, default `20128`), or soft HTTP `/healthz` | `/api/monitoring/health` as liveness | +| Readiness | HTTP `GET /healthz` | Tight timeouts that treat event-loop busy as dead | +| Deep / blackbox | `/api/monitoring/health` | — | + +`/healthz` only reports process lifecycle (`ok` / `starting` / `stopping`). It still +runs on the same Node event loop as request handling, so CPU-bound catalog or +compression work can delay it — busy ≠ dead. Full probe guidance: +[Monitoring guide — Kubernetes probe recommendations](../ops/MONITORING_GUIDE.md#kubernetes-probe-recommendations). ## Docker Compose with Caddy (HTTPS Auto-TLS) diff --git a/docs/ops/MONITORING_GUIDE.md b/docs/ops/MONITORING_GUIDE.md index a9d1db9422..82a66eeb49 100644 --- a/docs/ops/MONITORING_GUIDE.md +++ b/docs/ops/MONITORING_GUIDE.md @@ -1,7 +1,7 @@ --- title: "Monitoring & Observability Guide" -version: 3.8.40 -lastUpdated: 2026-06-28 +version: 3.8.50 +lastUpdated: 2026-08-13 --- # Monitoring & Observability Guide @@ -103,9 +103,29 @@ Per-combo: ## Health Check API -> **Note:** Only `GET /api/monitoring/health` is exposed as a REST endpoint. All other monitoring data (provider health, autopilot issues, quota monitors, token health, latency) is accessed via the **MCP tool** `observability_snapshot` or the **dashboard** pages — there are no dedicated REST routes for these. +OmniRoute exposes **two** HTTP health surfaces. They are not interchangeable for orchestrators. -### System Health +| Path | Purpose | Weight | Use for | +| --- | --- | --- | --- | +| `GET /healthz` | Lifecycle liveness/readiness (`ok` / `starting` / `stopping`) | Trivial (phase flag only) | Kubernetes **readiness**; soft **liveness** if you must use HTTP | +| `GET /api/monitoring/health` | Deep system + provider summary (DB, heap, catalog counts, …) | Heavy (sync DB / monitoring work) | Dashboards, blackbox deep checks, Docker’s built-in healthcheck | + +> **Note:** Provider health matrices, autopilot issues, quota monitors, token health, and latency detail beyond `/api/monitoring/health` are available via the **MCP tool** `observability_snapshot` or the **dashboard** pages — there are no dedicated REST routes for those. + +Both routes run on the **same Node event loop** as request handling. A CPU-bound path (large `GET /v1/models` catalog work, long-context compression / token counting) can delay **all** HTTP handlers, including `/healthz`. Event-loop busy ≠ process dead. Prefer fixing the hog; probe tuning only reduces false kills. + +### Lightweight orchestrator probe + +```bash +GET /healthz +# or HEAD /healthz +``` + +- **200** + body `ok` when the server lifecycle phase is ready +- **503** + `starting` / `stopping` during boot or shutdown +- Implementation: `src/app/healthz/route.ts` (no DB ping) + +### System Health (deep) ```bash GET /api/monitoring/health @@ -135,6 +155,48 @@ Response: } ``` +### Kubernetes probe recommendations + +OmniRoute is a **single Node process** (one event loop). Stock Docker `HEALTHCHECK` targets `/api/monitoring/health` — that is **too heavy** for kubelet liveness intervals. + +| Probe | Recommended target | Notes | +| --- | --- | --- | +| **Startup** | HTTP `GET /healthz` with a long `failureThreshold` (or large `startPeriod`) | Cold start + SQLite migration can exceed a few seconds | +| **Readiness** | HTTP `GET /healthz` | Remove endpoints while starting/stopping; still flaps if the loop is CPU-blocked | +| **Liveness** | **TCP** on the main service port (`PORT`, default `20128`), **or** HTTP `/healthz` with soft thresholds | Do **not** kill the pod on short event-loop stalls; busy ≠ dead | +| **Deep health** | `GET /api/monitoring/health` from an external checker | Not for kubelet `livenessProbe` / tight `readinessProbe` | + +Example shape (adjust thresholds to your cold-start and compression load): + +```yaml +ports: + - name: http + containerPort: 20128 +startupProbe: + httpGet: + path: /healthz + port: http + failureThreshold: 30 + periodSeconds: 5 +readinessProbe: + httpGet: + path: /healthz + port: http + periodSeconds: 5 + timeoutSeconds: 2 + failureThreshold: 6 +livenessProbe: + tcpSocket: + port: http + periodSeconds: 10 + timeoutSeconds: 3 + failureThreshold: 6 +``` + +**Do not** point kubelet **liveness** at `/api/monitoring/health`. That path does real DB/monitoring work and will false-positive under load. + +Related: [#10052](https://github.com/diegosouzapw/OmniRoute/issues/10052) (probes while the event loop is busy), [#9685](https://github.com/diegosouzapw/OmniRoute/issues/9685) / [#10055](https://github.com/diegosouzapw/OmniRoute/pull/10055) (catalog pricing hog), [#10117](https://github.com/diegosouzapw/OmniRoute/issues/10117) (compression token-count hog). + ### Provider Health > **No REST endpoint.** Provider health data is available via the MCP tool `observability_snapshot` or the dashboard `/dashboard/providers` page.