mirror of
https://github.com/diegosouzapw/OmniRoute.git
synced 2026-08-11 01:32:22 +03:00
a448b146bf
* test(integration): add general live-test tool for the real "default" combo Temporary WIP commit on this deferred branch — lands in its own separate PR once the bug-fix extraction batch is done (never bundled into a bug-fix PR). Unlike liveGeminiShared.ts (provisions its own narrow 2-model Gemini-only combo), this reads the REAL "default" combo currently configured on the target instance directly from the DB and exercises every provider/model step in it directly, bypassing combo routing, so live-test coverage always matches whatever is actually configured instead of a hardcoded snapshot. Live-verified against omniroute-beta (seeded with the real 18-model, 5-provider default combo): 14/18 models pass consistently across non-streaming + streaming Chat Completions and streaming Responses API. The 4 consistent failures are real external state (cerebras credits_exhausted, one deprecated openrouter free-tier model), not code regressions. (cherry picked from commit c40b13a48fd897259c56f5122e9e57a3dc7654ba) * test(integration): add rootless wire-capture correlation to the live-test tool Temporary WIP commit on this deferred branch — lands in the same final live-test-tool PR as the general default-combo suite, never bundled into a bug-fix PR. liveContainerHarness.ts spins up a dedicated, throwaway podman container (same runner-base image target as the operator's local dev/beta containers) so wire-capture tests are fully self-contained: builds the image if missing, starts the container with a persistent data dir, waits for health, seeds the real "default" combo + provider connections from the operator's local omniroute-dev instance (idempotent — only runs once per data dir), and provisions API keys via the running instance's own auth flow. wireCapture.ts captures the container's actual network traffic via `podman unshare nsenter --net=<container netns> -- tcpdump` — no root needed, verified working live (this generalizes the root-requiring `sudo nsenter -t $PID` command scripts/sre/tcp-close-analyzer.py already documented for the same rootless-Podman netns problem; that script's docstring now documents both). Capture and analysis needed two real fixes found only by running the pipeline live: `-U` (unbuffered tcpdump writes) plus a `pkill -f <pcap path>` fallback, since `podman unshare -> nsenter -> tcpdump` is a 3-level subprocess chain and SIGTERM to the top-level process doesn't reach the tcpdump grandchild, leaving an orphaned process and a truncated/unreadable pcap; and filtering on the container's internal listening port (20128) rather than the dynamically-assigned host port, since capture happens inside the container's own network namespace where only the internal port is meaningful. live-default-combo-wire-capture.test.ts (gated on RUN_LIVE_WIRE_CAPTURE=1) ties it together: sends a small representative sample of requests through the real default combo, then cross-checks each one's app-level JSON status against the actual HTTP status line observed on the wire via scripts/sre/tcp-close-analyzer.py's stream reassembly — catching bugs where the app layer claims success but the wire shows a truncated/reset stream, not just what liveDefaultComboShared.ts's existing breadth suite already covers. Live-verified end-to-end: 4/4 sampled requests correlated correctly across 8 captured TCP streams, container + capture process fully torn down afterward (verified no orphaned podman container or tcpdump process left running). sendModelRequest/filterActiveModelTargets (liveDefaultComboShared.ts) gain optional baseUrl/apiKey overrides, defaulting to the existing module-level omniroute-beta target, so the wire-capture suite can point the same request-sending logic at its own dedicated container instead. (cherry picked from commit 914a7e42cbe914f257db9f72eedc902ee1532083) --------- Co-authored-by: Markus Hartung <mail@hartmark.se>