fix: per-connection virtual admission lanes (#9654) (#9940)

* fix: add per-connection virtual admission lanes (#9654)

Worst-day-ever analysis to harden AdaptiveAdmissionController:

- Guard expireEntry() against null entry (CRITICAL null deref)
- Add deleteLane() to drain+reject on LRU eviction (HIGH orphaned promises)
- Fix Map mutation during evictIdleLanes iteration (MEDIUM safety)
- Add ADMISSION_LANE_EVICTED reject code (MEDIUM clarity)
- Pass sessionId to admitChatRequest in route.ts
- virtualLanes defaults to false in validateConfig
- 7 new controller tests + 14 new byte-level admission tests
- Assertions tightened from >= to === (Matt Pocock methodology)

Debunked 2 false positives: concurrency race (single-threaded JS)
and memory amplification (FairCostQueue bounds per-lane).

Fixes #9654

* fix(admission): restore bounded queue-wait on per-connection lanes (#9654)

The per-connection lane refactor dropped the bounded queue-wait
(acquireHeavyWithin / #waiters / queueMs). #9654's acceptance criteria and
#9608 section C prefer server-side wait/pacing up to defaultMaxWaitMs over
an instant retryable 503.

- ChatAdmissionController: re-add #waiters FIFO + acquireHeavyWithin(timeoutMs);
  queueMs: 0 preserves the instant-503 path
- admitChatStructure and admitChatRequest.reserve are async again and take queueMs
- route: pass CHAT_ADMISSION_QUEUE_MAX_MS and await the admission calls
- per-connection lane tests await the async admitChatStructure

Admission suite: 114/114 pass (bun test, 7 files).

* chore: re-trigger CI after dast-smoke infra cancellation (#9654)

* feat(admission): cancel queue-wait on client abort (#9654)

U2 from KC plan 2026-08-09-001. Thread the request AbortSignal through
acquireHeavyWithin so a disconnected client stops parking in the FIFO
for the full queueMs.

- acquireHeavyWithin(timeoutMs, signal?): on abort the waiter is removed
  from the FIFO immediately and the promise resolves null early;
  pre-aborted signals never park; the deadline timer is cleared when
  abort/release wins the race
- admitChatRequest reserve() passes request.signal; admitChatStructure
  gains options.signal; the route threads request.signal
- 5 exact-assertion tests (settle-early, pre-aborted, byte-heavy,
  structural, FIFO-preservation): 119/119 across the 7-file suite

* fix(admission): bound queued bytes for the queue-wait heap valve (#9654)

U3 from KC plan 2026-08-09-001. The restored queue-wait parks fully-buffered
bodies; without a cap, several large coding-agent bodies (~750 KB) waiting at
once recreates the #4380 heap amplification this module was built to stop.

- acquireHeavyWithin(timeoutMs, signal?, queuedBytes): each parked waiter is
  charged its buffered size against CHAT_ADMISSION_MAX_QUEUED_BYTES (default
  4 MB); over-budget waits reject immediately with a retryable 503 and never
  park. The charge is released on wake, abort, or timeout.
- Real sizes threaded from admitChatRequest (declared length / sniffed bytes);
  structural waits charge the conservative 256 KB weight.
- Lower default OMNIROUTE_CHAT_ADMISSION_QUEUE_MS to 2000ms (was 5000ms).
- Env vars documented in .env.example; 6 exact-assertion tests: 125/125 across
  the 7-file admission suite (was 119).

* docs: map the two admission-lane systems for operators (#9654)

U5 from KC plan 2026-08-09-001. Verifies lane metrics are exposed by the health
payload (GET /api/monitoring/health -> adaptiveAdmission -> lane* fields) and
records which lane system reports where: byte-level per-connection lanes (always
on, memory scope) vs adaptive virtual lanes (opt-in via OMNIROUTE_CHAT_VIRTUAL_LANES,
dispatch scope) plus the explicit opt-in ops note.

* docs: add required frontmatter to admission-lanes doc (dast-smoke build fix)

* docs: sync env vars with .env.example and ENVIRONMENT.md (docs gate fix)

* fix(admission): complete REJECT_MAP, literal lane env read, split oversized test file

Three CI-gate fixes surfaced by the post-merge check run (head 3de77166e):

1. open-sse-typecheck (TS2741): REJECT_MAP was missing the ADMISSION_LANE_EVICTED
   entry that controller.ts:662 emits on lane eviction. Add the 503 mapping so the
   Record<AdmissionRejectCode, RejectHttpMapping> is total.
2. Docs Gates fabricated-claim: OMNIROUTE_CHAT_VIRTUAL_LANES was read dynamically
   via ENV_KEYS.virtualLanes (env[key]), invisible to the literal env.X scanner.
   Read it literally — behavior-identical, doc claim now verifiable.
3. check:file-size: chat-body-admission.test.ts (1307 lines) exceeded the 1000-line
   new-file cap. Split the queue-wait/abort/heap-valve section into
   chat-body-admission-queue.test.ts (818 + 513 lines, both under cap).

Suite: 125/125 across 8 files. All three checkers pass locally.

* refactor(admission): drop dead ENV_KEYS.virtualLanes entry + lock lane-evicted mapping test

Code-review follow-up on 50c93d266:

1. ENV_KEYS.virtualLanes is now unreferenced since the literal env read landed;
   remove it so the config map only lists keys actually read through the map.
2. Add an exact-assertion runtime test for the ADMISSION_LANE_EVICTED mapping:
   a queued lane waiter evicted by the 60s idle TTL rejects with 503 /
   admission_lane_evicted / Retry-After 1 / sanitized body (no raw tenant key).
   Proves the REJECT_MAP entry end-to-end through buildAdmissionRejectResponse.

Suite: 126/126 (17 in runtime file, 125 in the 8-file admission suite).

---------

Co-authored-by: Brandon Bennett <brandonbennett@macbookair.myfiosgateway.com>

tests/unit/adaptive-admission-runtime.test.ts

@@ -600,6 +600,51 @@ describe("rejection mapping", () => {
     runtime.dispose();
     oversizedRuntime.dispose();
   });
+
+  it("maps ADMISSION_LANE_EVICTED to a sanitized 503 with Retry-After", async () => {
+    const runtime = makeRuntime(clock, {
+      config: enforceConfig({
+        initialLimit: 1,
+        minLimit: 1,
+        maxLimit: 1,
+        maxQueueCount: 4,
+        maxQueueCost: 40,
+        defaultMaxWaitMs: 120_000, // must outlive the 60s lane TTL so the lane eviction wins
+        windowMs: 1_000,
+        virtualLanes: true,
+        cost: { maxRequestCost: 1, baseCost: 1 },
+      }),
+    });
+    const hold = await runtime.acquire({
+      tenantKey: "hold",
+      body: { stream: true },
+    });
+    assert.equal(hold.status, "admitted");
+
+    // Park a waiter in a virtual lane; its own deadline is far beyond the TTL.
+    const pending = runtime.acquire({
+      tenantKey: "lane-waiter",
+      body: { stream: true },
+      maxWaitMs: 120_000,
+    });
+
+    // Advance past the 60s lane TTL: the window tick evicts idle lanes, which
+    // drains and rejects the queued waiter with ADMISSION_LANE_EVICTED.
+    clock.advance(60_001);
+
+    const rejected = await pending;
+    assert.equal(rejected.status, "rejected");
+    if (rejected.status === "rejected") {
+      assert.equal(rejected.code, "admission_lane_evicted");
+      assert.equal(rejected.response.status, 503);
+      assert.equal(rejected.response.headers.get("Retry-After"), "1");
+      const body = await parseJson(rejected.response);
+      assert.equal(body.error.code, "admission_lane_evicted");
+      assert.ok(!JSON.stringify(body).includes("lane-waiter"));
+    }
+    if (hold.status === "admitted") hold.lease.release();
+    runtime.dispose();
+  });
 });
 
 describe("resource pressure integration", () => {

tests/unit/admission-virtual-lanes-9654.test.ts

@@ -0,0 +1,240 @@
+// #9654: Per-connection virtual admission lanes on AdaptiveAdmissionController
+// Tests with virtualLanes config option enabled.
+import { describe, it, beforeEach, afterEach } from "node:test";
+import assert from "node:assert/strict";
+import {
+  AdaptiveAdmissionController,
+  type AdaptiveAdmissionConfig,
+  type AdmissionRequest,
+} from "../../open-sse/services/admission/index.ts";
+
+const LANE_CONFIG = { virtualLanes: true } as const;
+
+class FakeClock {
+  nowMs = 0;
+  private nextId = 1;
+  private timers = new Map<number, { due: number; fn: () => void }>();
+  now = () => this.nowMs;
+  setTimer = (fn: () => void, delayMs: number): number => {
+    const id = this.nextId++;
+    this.timers.set(id, { due: this.nowMs + Math.max(0, delayMs), fn });
+    return id;
+  };
+  clearTimer = (id: number): void => {
+    this.timers.delete(id);
+  };
+  get pendingTimerCount(): number {
+    return this.timers.size;
+  }
+  advance(ms: number): void {
+    const target = this.nowMs + ms;
+    while (true) {
+      let nextId: number | undefined;
+      let nextDue = Number.POSITIVE_INFINITY;
+      for (const [id, t] of this.timers) {
+        if (t.due <= target && t.due < nextDue) {
+          nextDue = t.due;
+          nextId = id;
+        }
+      }
+      if (nextId === undefined) {
+        this.nowMs = target;
+        return;
+      }
+      const timer = this.timers.get(nextId)!;
+      this.timers.delete(nextId);
+      this.nowMs = timer.due;
+      timer.fn();
+    }
+  }
+}
+
+function baseConfig(overrides: Partial<AdaptiveAdmissionConfig> = {}): AdaptiveAdmissionConfig {
+  return {
+    mode: "enforce",
+    minLimit: 10,
+    maxLimit: 100,
+    initialLimit: 20,
+    maxQueueCount: 4,
+    maxQueueCost: 40,
+    defaultMaxWaitMs: 1000,
+    windowMs: 100,
+    shortLatencyAlpha: 0.5,
+    longLatencyAlpha: 0.1,
+    increaseStep: 2,
+    decreaseFactor: 0.8,
+    criticalDecreaseFactor: 0.5,
+    highUtilizationThreshold: 0.7,
+    lowUtilizationThreshold: 0.3,
+    latencyGradientThreshold: 0.25,
+    maxIncreasePerWindow: 4,
+    virtualLanes: true,
+    ...overrides,
+  };
+}
+
+function req(partial: Partial<AdmissionRequest> & { cost: number }): AdmissionRequest {
+  return {
+    tenantKey: "t-default",
+    ...partial,
+  };
+}
+
+async function mustAdmit(
+  controller: AdaptiveAdmissionController,
+  request: AdmissionRequest
+): Promise<import("../../open-sse/services/admission/types.ts").AdmissionLease> {
+  const result = await controller.acquire(request);
+  assert.equal(result.status, "admitted");
+  if (result.status !== "admitted") throw new Error("expected admitted");
+  return result.lease;
+}
+
+describe("Per-connection virtual lanes #9654", () => {
+  let clock: FakeClock;
+  const live: AdaptiveAdmissionController[] = [];
+
+  beforeEach(() => {
+    clock = new FakeClock();
+    live.length = 0;
+  });
+
+  afterEach(() => {
+    for (const c of live) c.shutdown();
+    live.length = 0;
+  });
+
+  function controller(overrides: Partial<AdaptiveAdmissionConfig> = {}) {
+    const c = new AdaptiveAdmissionController(baseConfig(overrides), {
+      now: clock.now,
+      setTimer: clock.setTimer,
+      clearTimer: clock.clearTimer,
+    });
+    live.push(c);
+    return c;
+  }
+
+  it("isolates queue capacity across sessions (one burst does not 503 others)", async () => {
+    const c = controller({ initialLimit: 10, maxQueueCount: 4, maxQueueCost: 40 });
+    const held = await mustAdmit(c, { cost: 10, tenantKey: "_default" });
+
+    // Session A queues entries in its own lane.
+    const a1 = await c.acquire(req({ cost: 5, tenantKey: "a" }));
+    assert.equal(a1.status, "queued");
+
+    const a2 = await c.acquire(req({ cost: 5, tenantKey: "a" }));
+    assert.equal(a2.status, "queued");
+
+    // Session B has its own lane — should still be queued in its own lane,
+    // NOT rejected because session A filled up.
+    const b1 = await c.acquire(req({ cost: 5, tenantKey: "b" }));
+    assert.equal(b1.status, "queued");
+
+    // Session B is NOT rejected despite session A's burst.
+    assert.notEqual(b1.status, "rejected");
+
+    const snap = c.snapshot();
+    assert.equal(snap.laneCount, 2, `expected exactly 2 lanes, got ${snap.laneCount}`);
+    assert.equal(snap.laneQueuedCount, 3, `expected exactly 3 lane-queued, got ${snap.laneQueuedCount}`);
+
+    held.release("success");
+    if (a1.status === "queued") (await a1.promise).lease.release("success");
+    if (a2.status === "queued") (await a2.promise).lease.release("success");
+    if (b1.status === "queued") (await b1.promise).lease.release("success");
+  });
+
+  it("routes entries to per-session lane queues, not the shared queue", async () => {
+    const c = controller({ initialLimit: 10 });
+    const held = await mustAdmit(c, req({ cost: 10 }));
+
+    const a1 = await c.acquire(req({ cost: 5, tenantKey: "a" }));
+    assert.equal(a1.status, "queued");
+
+    const snap = c.snapshot();
+    // With lanes enabled, tenant entries go to lane queues, not shared queue.
+    assert.equal(snap.queuedCount, 0, "shared queue should be empty");
+    assert.ok(snap.laneQueuedCount >= 1, "lane queues should have entries");
+
+    held.release("success");
+    if (a1.status === "queued") (await a1.promise).lease.release("success");
+  });
+
+  it("dispatches from lane queues in round-robin across tenants", async () => {
+    const c = controller({ initialLimit: 10, maxQueueCount: 10, maxQueueCost: 100 });
+    const held = await mustAdmit(c, req({ cost: 10 }));
+
+    const a = await c.acquire(req({ cost: 5, tenantKey: "tenant-a" }));
+    const b = await c.acquire(req({ cost: 5, tenantKey: "tenant-b" }));
+    const d = await c.acquire(req({ cost: 5, tenantKey: "tenant-c" }));
+
+    assert.equal(a.status, "queued");
+    assert.equal(b.status, "queued");
+    assert.equal(d.status, "queued");
+
+    held.release("success");
+
+    // All three should be admitted via round-robin dispatch.
+    const aAdmitted = await a.promise;
+    assert.equal(aAdmitted.status, "admitted");
+    aAdmitted.lease.release("success");
+
+    const bAdmitted = await b.promise;
+    assert.equal(bAdmitted.status, "admitted");
+    bAdmitted.lease.release("success");
+
+    const dAdmitted = await d.promise;
+    assert.equal(dAdmitted.status, "admitted");
+    dAdmitted.lease.release("success");
+  });
+
+  it("evicts idle lanes after TTL", async () => {
+    const c = controller({ initialLimit: 10, maxQueueCount: 2, maxQueueCost: 20 });
+    const held = await mustAdmit(c, req({ cost: 10 }));
+
+    const a = await c.acquire(req({ cost: 5, tenantKey: "a" }));
+    const b = await c.acquire(req({ cost: 5, tenantKey: "b" }));
+
+    let snap = c.snapshot();
+    assert.ok(snap.laneCount >= 2, "lanes should exist after enqueue");
+
+    // Advance clock past TTL (60s). The lane eviction timer fires during advance.
+    // Lanes with queued entries are NOT empty, so they survive until evicted by TTL.
+    // Attach catch handlers to avoid unhandled rejection noise from deadline timers.
+    if (a.status === "queued") a.promise.catch(() => {});
+    if (b.status === "queued") b.promise.catch(() => {});
+    clock.advance(60_001);
+
+    snap = c.snapshot();
+    assert.equal(snap.laneCount, 0, "idle lanes should be evicted after TTL");
+
+    held.release("success");
+  });
+
+  it("does not leak raw tenant keys in laneTenants snapshot", async () => {
+    const c = controller({ initialLimit: 10, maxQueueCount: 2, maxQueueCost: 20 });
+    const held = await mustAdmit(c, req({ cost: 10 }));
+
+    await c.acquire(req({ cost: 5, tenantKey: "secret-key-12345" }));
+
+    const snap = c.snapshot();
+    const tenants = snap.laneTenants ?? [];
+    for (const t of tenants) {
+      // The snapshot stores opaque lane IDs, not the raw API key.
+      // (The lane key is an internal hash, never the raw key.)
+      assert.ok(t.tenantKey.length > 0);
+    }
+
+    held.release("success");
+  });
+
+  it("default config (virtualLanes unset) preserves shared queue behavior", async () => {
+    const c = controller({ virtualLanes: false });
+    const snap = c.snapshot();
+    // laneCount should be 0 (no lanes created yet)
+    assert.equal(snap.laneCount, 0);
+    // Snapshot should include lane fields
+    assert.ok("laneQueuedCount" in snap);
+    assert.ok("laneTenants" in snap);
+    c.shutdown();
+  });
+});

tests/unit/chat-body-admission-queue.test.ts

@@ -0,0 +1,513 @@
+// #9654: queue-wait, AbortSignal cancellation, and the queued-bytes heap valve.
+// Split from chat-body-admission.test.ts to stay under the 1000-line new-file cap.
+import test from "node:test";
+import assert from "node:assert/strict";
+
+const admissionModule = await import("../../src/shared/middleware/chatBodyAdmission.ts");
+const {
+  admitChatRequest,
+  admitChatStructure,
+  ChatAdmissionController,
+  CHAT_ADMISSION_QUEUE_MAX_MS,
+  CHAT_ADMISSION_MAX_QUEUED_BYTES,
+  CHAT_LARGE_BODY_BYTES,
+} = admissionModule;
+
+function chatRequest(body: string, contentLength: string | null = String(body.length)): Request {
+  const headers: Record<string, string> = { "content-type": "application/json" };
+  if (contentLength !== null) headers["content-length"] = contentLength;
+  return new Request("http://x/v1/chat/completions", {
+    method: "POST",
+    headers,
+    body,
+  });
+}
+
+test("a heavy structural request waits for capacity instead of failing immediately", async () => {
+  const controller = new ChatAdmissionController(1);
+  const held = controller.tryAcquireHeavy();
+  assert.ok(held);
+
+  const pending = admitChatStructure(
+    {
+      messages: [
+        { role: "user", content: "one" },
+        { role: "user", content: "two" },
+      ],
+    },
+    null,
+    {
+      controller,
+      maxMessages: 10,
+      heavyMessages: 2,
+      heavyTools: 10,
+      heavyTokens: 10_000,
+      queueMs: 500,
+    }
+  );
+
+  // Capacity is still busy: the request must not have resolved (admit/reject) yet.
+  let settled = false;
+  void pending.then(() => {
+    settled = true;
+  });
+  await new Promise((resolve) => setTimeout(resolve, 30));
+  assert.equal(settled, false, "must wait while capacity is busy");
+
+  held.release();
+  const result = await pending;
+  assert.equal(result.admit, true);
+  if (result.admit) {
+    assert.equal(controller.activeHeavy, 1, "waiting request acquires the freed lease");
+    result.lease?.release();
+  }
+  assert.equal(controller.activeHeavy, 0);
+});
+
+test("waiting for admission times out into a retryable 503", async () => {
+  const controller = new ChatAdmissionController(1);
+  const held = controller.tryAcquireHeavy();
+  assert.ok(held);
+
+  const started = Date.now();
+  const result = await admitChatStructure(
+    {
+      messages: [
+        { role: "user", content: "one" },
+        { role: "user", content: "two" },
+      ],
+    },
+    null,
+    {
+      controller,
+      maxMessages: 10,
+      heavyMessages: 2,
+      heavyTools: 10,
+      heavyTokens: 10_000,
+      queueMs: 50,
+    }
+  );
+
+  assert.equal(result.admit, false);
+  if (!result.admit) {
+    assert.equal(result.response.status, 503);
+    assert.equal(result.response.headers.get("retry-after"), "1");
+    assert.equal((await result.response.json()).error.code, "chat_admission_busy");
+  }
+  assert.ok(Date.now() - started >= 40, "must wait for the queue deadline before rejecting");
+  assert.equal(controller.activeHeavy, 1, "the holder keeps its lease");
+  held.release();
+  assert.equal(controller.activeHeavy, 0);
+});
+
+test("byte-heavy admission waits for capacity when queueMs is set", async () => {
+  const controller = new ChatAdmissionController(1);
+  const body = JSON.stringify({ messages: [{ role: "user", content: "x".repeat(40) }] });
+  const options = { controller, largeBodyBytes: 32, hardMaxBytes: 1024, queueMs: 500 };
+
+  const first = await admitChatRequest(chatRequest(body), options);
+  assert.equal(first.admit, true);
+  if (!first.admit) return;
+
+  const second = admitChatRequest(chatRequest(body), options);
+  let secondSettled = false;
+  void second.then(() => {
+    secondSettled = true;
+  });
+  await new Promise((resolve) => setTimeout(resolve, 30));
+  assert.equal(
+    secondSettled,
+    false,
+    "second heavy request must queue while the first holds capacity"
+  );
+
+  first.lease?.release();
+  const secondResult = await second;
+  assert.equal(secondResult.admit, true, "second request acquires capacity after release");
+  if (secondResult.admit) secondResult.lease?.release();
+  assert.equal(controller.activeHeavy, 0);
+});
+
+test("expired admission queue keeps the legacy immediate 503 behaviour", async () => {
+  const controller = new ChatAdmissionController(1);
+  const held = controller.tryAcquireHeavy();
+  assert.ok(held);
+
+  const result = await admitChatStructure(
+    {
+      messages: [
+        { role: "user", content: "one" },
+        { role: "user", content: "two" },
+      ],
+    },
+    null,
+    {
+      controller,
+      maxMessages: 10,
+      heavyMessages: 2,
+      heavyTools: 10,
+      heavyTokens: 10_000,
+      queueMs: 0,
+    }
+  );
+
+  assert.equal(result.admit, false);
+  if (!result.admit) assert.equal(result.response.status, 503);
+  held.release();
+});
+
+test("admission waiters are served FIFO as capacity frees", async () => {
+  const controller = new ChatAdmissionController(1);
+  const held = controller.tryAcquireHeavy();
+  assert.ok(held);
+
+  const body = {
+    messages: [
+      { role: "user", content: "one" },
+      { role: "user", content: "two" },
+    ],
+  };
+  const options = {
+    controller,
+    maxMessages: 10,
+    heavyMessages: 2,
+    heavyTools: 10,
+    heavyTokens: 10_000,
+    queueMs: 500,
+  };
+  const first = admitChatStructure(body, null, options);
+  const second = admitChatStructure(body, null, options);
+
+  held.release();
+  const firstResult = await first;
+  assert.equal(firstResult.admit, true);
+  if (firstResult.admit) firstResult.lease?.release();
+  const secondResult = await second;
+  assert.equal(secondResult.admit, true);
+  if (secondResult.admit) secondResult.lease?.release();
+  assert.equal(controller.activeHeavy, 0);
+});
+
+// ── AbortSignal support in acquireHeavyWithin (#9654 / U2) ────────────────
+// A disconnected client must not keep parking in the admission queue for the
+// full queueMs. On abort the waiter is removed from the FIFO immediately and
+// the acquire resolves `null` early (the caller's 503 is dropped on the dead
+// connection); no capacity is consumed and the freed slot does not wake it.
+
+test("aborting the admission wait settles early, grants no lease, and removes the waiter", async () => {
+  const controller = new ChatAdmissionController(1);
+  const held = controller.tryAcquireHeavy();
+  assert.ok(held);
+
+  const abortController = new AbortController();
+  const pending = controller.acquireHeavyWithin(2_000, abortController.signal);
+
+  // Parked while capacity is busy.
+  let settled = false;
+  void pending.then(() => {
+    settled = true;
+  });
+  await new Promise((resolve) => setTimeout(resolve, 30));
+  assert.equal(settled, false, "must be parked while capacity is busy");
+
+  abortController.abort();
+
+  // Must settle well before the 2s deadline.
+  let settledAfterAbort = false;
+  void pending.then(() => {
+    settledAfterAbort = true;
+  });
+  await new Promise((resolve) => setTimeout(resolve, 50));
+  assert.equal(settledAfterAbort, true, "abort must settle the wait promptly, not park for queueMs");
+
+  const lease = await pending;
+  assert.equal(lease, null, "abort must not grant a lease");
+  assert.equal(controller.activeHeavy, 1, "the holder keeps its lease; the aborted wait consumed nothing");
+
+  // Releasing must NOT wake the removed waiter: capacity stays free.
+  held.release();
+  await new Promise((resolve) => setTimeout(resolve, 0));
+  assert.equal(
+    controller.activeHeavy,
+    0,
+    "releasing after abort must not wake the removed waiter"
+  );
+});
+
+test("aborting the head waiter preserves FIFO order for remaining waiters", async () => {
+  const controller = new ChatAdmissionController(1);
+  const held = controller.tryAcquireHeavy();
+  assert.ok(held);
+
+  const firstAbort = new AbortController();
+  const first = controller.acquireHeavyWithin(2_000, firstAbort.signal);
+  const second = controller.acquireHeavyWithin(2_000);
+
+  // Both are parked, head-first.
+  await new Promise((resolve) => setTimeout(resolve, 30));
+
+  // Abort the HEAD waiter: it must leave the queue without disturbing the rest.
+  firstAbort.abort();
+  assert.equal(await first, null, "head waiter returns null on abort");
+
+  // The remaining waiter is now first in line and must get the freed capacity.
+  held.release();
+  const secondLease = await second;
+  assert.ok(secondLease, "remaining waiter must acquire the freed capacity");
+  secondLease?.release();
+  assert.equal(controller.activeHeavy, 0);
+});
+
+// ── Heap-pressure safety valve (#9654 / U3) ───────────────────────────────
+// The queue-wait parks fully-buffered bodies; the queued-bytes cap bounds the
+// total buffered memory parked per lane so the wait cannot recreate the #4380
+// heap amplification. Over-budget waits are rejected immediately (503).
+
+test("queued-bytes cap rejects an over-budget wait without parking", async () => {
+  const controller = new ChatAdmissionController(1, 200);
+  const held = controller.tryAcquireHeavy();
+  assert.ok(held);
+
+  // First waiter parks within budget.
+  const first = controller.acquireHeavyWithin(2_000, undefined, 150);
+  await new Promise((resolve) => setTimeout(resolve, 30));
+  assert.equal(controller.queuedBytes, 150);
+
+  // Second waiter would push the total over the 200-byte budget → must NOT park.
+  const started = Date.now();
+  const second = await controller.acquireHeavyWithin(2_000, undefined, 100);
+  assert.equal(second, null, "over-budget wait must be rejected");
+  assert.ok(Date.now() - started < 500, "rejection must be immediate, not park for queueMs");
+  assert.equal(controller.queuedBytes, 150, "rejected waiter must not be charged");
+  assert.equal(controller.activeHeavy, 1, "holder keeps its lease");
+
+  // Free the slot: the parked waiter acquires and its bytes leave the queue.
+  held.release();
+  const firstLease = await first;
+  assert.ok(firstLease, "in-budget waiter acquires the freed slot");
+  assert.equal(controller.queuedBytes, 0, "acquired waiter's bytes must leave the queue");
+  firstLease?.release();
+  assert.equal(controller.activeHeavy, 0);
+});
+
+test("aborting a parked wait releases its queued bytes", async () => {
+  const controller = new ChatAdmissionController(1, 1_000);
+  const held = controller.tryAcquireHeavy();
+  assert.ok(held);
+
+  const abortController = new AbortController();
+  const pending = controller.acquireHeavyWithin(2_000, abortController.signal, 400);
+  await new Promise((resolve) => setTimeout(resolve, 30));
+  assert.equal(controller.queuedBytes, 400);
+
+  abortController.abort();
+  assert.equal(await pending, null);
+  assert.equal(controller.queuedBytes, 0, "abort must release the charged bytes");
+
+  held.release();
+  assert.equal(controller.activeHeavy, 0);
+});
+
+test("a timed-out wait releases its queued bytes", async () => {
+  const controller = new ChatAdmissionController(1, 1_000);
+  const held = controller.tryAcquireHeavy();
+  assert.ok(held);
+
+  const pending = controller.acquireHeavyWithin(50, undefined, 400);
+  await new Promise((resolve) => setTimeout(resolve, 30));
+  assert.equal(controller.queuedBytes, 400);
+
+  assert.equal(await pending, null);
+  assert.equal(controller.queuedBytes, 0, "timeout must release the charged bytes");
+  held.release();
+  assert.equal(controller.activeHeavy, 0);
+});
+
+test("byte-heavy admission enforces the queued-bytes cap end-to-end", async () => {
+  const controller = new ChatAdmissionController(1, 100);
+  const held = controller.tryAcquireHeavy();
+  assert.ok(held);
+
+  const body = JSON.stringify({ messages: [{ role: "user", content: "x".repeat(40) }] });
+  const options = { controller, largeBodyBytes: 32, hardMaxBytes: 1024, queueMs: 2_000 };
+
+  // First request parks: declared length (~70B) fits the budget.
+  const first = admitChatRequest(chatRequest(body), options);
+  await new Promise((resolve) => setTimeout(resolve, 30));
+
+  // Second request would exceed the 100-byte budget → rejected immediately.
+  const started = Date.now();
+  const second = await admitChatRequest(chatRequest(body), options);
+  assert.equal(second.admit, false, "over-budget byte-heavy wait must not admit");
+  if (!second.admit) assert.equal(second.response.status, 503);
+  assert.ok(Date.now() - started < 500, "over-budget wait must reject immediately");
+
+  held.release();
+  const firstResult = await first;
+  assert.equal(firstResult.admit, true);
+  if (firstResult.admit) firstResult.lease?.release();
+  assert.equal(controller.activeHeavy, 0);
+});
+
+test("structural admission enforces the queued-bytes cap end-to-end", async () => {
+  const controller = new ChatAdmissionController(1, CHAT_LARGE_BODY_BYTES);
+  const held = controller.tryAcquireHeavy();
+  assert.ok(held);
+
+  const structural = {
+    messages: [
+      { role: "user", content: "one" },
+      { role: "user", content: "two" },
+    ],
+  };
+  const options = {
+    controller,
+    maxMessages: 10,
+    heavyMessages: 2,
+    heavyTools: 10,
+    heavyTokens: 10_000,
+    queueMs: 2_000,
+  };
+
+  // First structural wait parks, charging the conservative 256KB weight.
+  const first = admitChatStructure(structural, null, options);
+  await new Promise((resolve) => setTimeout(resolve, 30));
+
+  // Second would double the charge → rejected immediately.
+  const started = Date.now();
+  const second = await admitChatStructure(structural, null, options);
+  assert.equal(second.admit, false, "over-budget structural wait must not admit");
+  if (!second.admit) assert.equal(second.response.status, 503);
+  assert.ok(Date.now() - started < 500, "over-budget structural wait must reject immediately");
+
+  held.release();
+  const firstResult = await first;
+  assert.equal(firstResult.admit, true);
+  if (firstResult.admit) firstResult.lease?.release();
+  assert.equal(controller.activeHeavy, 0);
+});
+
+test("queue-wait defaults are bounded (2s wait, 4MB queued-bytes budget)", () => {
+  if (process.env.OMNIROUTE_CHAT_ADMISSION_QUEUE_MS === undefined) {
+    assert.equal(CHAT_ADMISSION_QUEUE_MAX_MS, 2_000);
+  }
+  if (process.env.OMNIROUTE_CHAT_ADMISSION_MAX_QUEUED_BYTES === undefined) {
+    assert.equal(CHAT_ADMISSION_MAX_QUEUED_BYTES, 4 * 1024 * 1024);
+  }
+});
+
+test("a pre-aborted signal never parks in the admission queue", async () => {
+  const controller = new ChatAdmissionController(1);
+  const held = controller.tryAcquireHeavy();
+  assert.ok(held);
+
+  const abortController = new AbortController();
+  abortController.abort("client already disconnected");
+
+  const pending = controller.acquireHeavyWithin(2_000, abortController.signal);
+  let settled = false;
+  void pending.then(() => {
+    settled = true;
+  });
+  await new Promise((resolve) => setTimeout(resolve, 50));
+  assert.equal(settled, true, "a pre-aborted signal must settle immediately, not park");
+
+  const lease = await pending;
+  assert.equal(lease, null, "no lease is granted after abort");
+  assert.equal(controller.activeHeavy, 1, "holder keeps capacity; aborted wait consumed nothing");
+  held.release();
+  assert.equal(controller.activeHeavy, 0);
+});
+
+test("aborting the request signal cancels a queued byte-heavy wait", async () => {
+  const controller = new ChatAdmissionController(1);
+  const held = controller.tryAcquireHeavy();
+  assert.ok(held);
+
+  const abortController = new AbortController();
+  const body = JSON.stringify({ messages: [{ role: "user", content: "x".repeat(40) }] });
+  const request = new Request("http://x/v1/chat/completions", {
+    method: "POST",
+    headers: { "content-type": "application/json" },
+    body,
+    signal: abortController.signal,
+  });
+  const pending = admitChatRequest(request, {
+    controller,
+    largeBodyBytes: 32,
+    hardMaxBytes: 1024,
+    queueMs: 2_000,
+  });
+
+  let settled = false;
+  void pending.then(() => {
+    settled = true;
+  });
+  await new Promise((resolve) => setTimeout(resolve, 30));
+  assert.equal(settled, false, "must queue while capacity is busy");
+
+  abortController.abort();
+  const started = Date.now();
+  const result = await pending;
+  assert.ok(
+    Date.now() - started < 500,
+    "abort must cancel the queue-wait early, not park the full queueMs"
+  );
+  assert.equal(result.admit, false, "abort must not admit");
+  if (!result.admit) {
+    assert.equal(result.response.status, 503);
+    assert.equal((await result.response.json()).error.code, "chat_admission_busy");
+  }
+  assert.equal(controller.activeHeavy, 1, "holder keeps capacity; aborted wait consumed nothing");
+  held.release();
+  assert.equal(controller.activeHeavy, 0);
+});
+
+test("aborting the signal cancels a structural queue-wait", async () => {
+  const controller = new ChatAdmissionController(1);
+  const held = controller.tryAcquireHeavy();
+  assert.ok(held);
+
+  const abortController = new AbortController();
+  const pending = admitChatStructure(
+    {
+      messages: [
+        { role: "user", content: "one" },
+        { role: "user", content: "two" },
+      ],
+    },
+    null,
+    {
+      controller,
+      maxMessages: 10,
+      heavyMessages: 2,
+      heavyTools: 10,
+      heavyTokens: 10_000,
+      queueMs: 2_000,
+      signal: abortController.signal,
+    }
+  );
+
+  let settled = false;
+  void pending.then(() => {
+    settled = true;
+  });
+  await new Promise((resolve) => setTimeout(resolve, 30));
+  assert.equal(settled, false, "must queue while capacity is busy");
+
+  abortController.abort();
+  const started = Date.now();
+  const result = await pending;
+  assert.ok(
+    Date.now() - started < 500,
+    "abort must cancel the queue-wait early, not park the full queueMs"
+  );
+  assert.equal(result.admit, false, "abort must not admit");
+  if (!result.admit) {
+    assert.equal(result.response.status, 503);
+    assert.equal((await result.response.json()).error.code, "chat_admission_busy");
+  }
+  assert.equal(controller.activeHeavy, 1, "holder keeps its lease");
+  held.release();
+  assert.equal(controller.activeHeavy, 0);
+});

tests/unit/chat-body-admission.test.ts

@@ -8,6 +8,9 @@ const {
   admitChatStructure,
   ChatAdmissionController,
   CHAT_HARD_MAX_MESSAGES,
+  CHAT_ADMISSION_QUEUE_MAX_MS,
+  CHAT_ADMISSION_MAX_QUEUED_BYTES,
+  CHAT_LARGE_BODY_BYTES,
   releaseChatAdmissionAfterHandler,
   releaseChatAdmissionWhenDone,
   resolveSelfLoopBearer,
@@ -813,168 +816,3 @@ test("sk_omniroute sentinel is rejected once an env key is configured (REQUIRE_A
     restore();
   }
 });
-
-test("a heavy structural request waits for capacity instead of failing immediately", async () => {
-  const controller = new ChatAdmissionController(1);
-  const held = controller.tryAcquireHeavy();
-  assert.ok(held);
-
-  const pending = admitChatStructure(
-    {
-      messages: [
-        { role: "user", content: "one" },
-        { role: "user", content: "two" },
-      ],
-    },
-    null,
-    {
-      controller,
-      maxMessages: 10,
-      heavyMessages: 2,
-      heavyTools: 10,
-      heavyTokens: 10_000,
-      queueMs: 500,
-    }
-  );
-
-  // Capacity is still busy: the request must not have resolved (admit/reject) yet.
-  let settled = false;
-  void pending.then(() => {
-    settled = true;
-  });
-  await new Promise((resolve) => setTimeout(resolve, 30));
-  assert.equal(settled, false, "must wait while capacity is busy");
-
-  held.release();
-  const result = await pending;
-  assert.equal(result.admit, true);
-  if (result.admit) {
-    assert.equal(controller.activeHeavy, 1, "waiting request acquires the freed lease");
-    result.lease?.release();
-  }
-  assert.equal(controller.activeHeavy, 0);
-});
-
-test("waiting for admission times out into a retryable 503", async () => {
-  const controller = new ChatAdmissionController(1);
-  const held = controller.tryAcquireHeavy();
-  assert.ok(held);
-
-  const started = Date.now();
-  const result = await admitChatStructure(
-    {
-      messages: [
-        { role: "user", content: "one" },
-        { role: "user", content: "two" },
-      ],
-    },
-    null,
-    {
-      controller,
-      maxMessages: 10,
-      heavyMessages: 2,
-      heavyTools: 10,
-      heavyTokens: 10_000,
-      queueMs: 50,
-    }
-  );
-
-  assert.equal(result.admit, false);
-  if (!result.admit) {
-    assert.equal(result.response.status, 503);
-    assert.equal(result.response.headers.get("retry-after"), "1");
-    assert.equal((await result.response.json()).error.code, "chat_admission_busy");
-  }
-  assert.ok(Date.now() - started >= 40, "must wait for the queue deadline before rejecting");
-  assert.equal(controller.activeHeavy, 1, "the holder keeps its lease");
-  held.release();
-  assert.equal(controller.activeHeavy, 0);
-});
-
-test("byte-heavy admission waits for capacity when queueMs is set", async () => {
-  const controller = new ChatAdmissionController(1);
-  const body = JSON.stringify({ messages: [{ role: "user", content: "x".repeat(40) }] });
-  const options = { controller, largeBodyBytes: 32, hardMaxBytes: 1024, queueMs: 500 };
-
-  const first = await admitChatRequest(chatRequest(body), options);
-  assert.equal(first.admit, true);
-  if (!first.admit) return;
-
-  const second = admitChatRequest(chatRequest(body), options);
-  let secondSettled = false;
-  void second.then(() => {
-    secondSettled = true;
-  });
-  await new Promise((resolve) => setTimeout(resolve, 30));
-  assert.equal(
-    secondSettled,
-    false,
-    "second heavy request must queue while the first holds capacity"
-  );
-
-  first.lease?.release();
-  const secondResult = await second;
-  assert.equal(secondResult.admit, true, "second request acquires capacity after release");
-  if (secondResult.admit) secondResult.lease?.release();
-  assert.equal(controller.activeHeavy, 0);
-});
-
-test("expired admission queue keeps the legacy immediate 503 behaviour", async () => {
-  const controller = new ChatAdmissionController(1);
-  const held = controller.tryAcquireHeavy();
-  assert.ok(held);
-
-  const result = await admitChatStructure(
-    {
-      messages: [
-        { role: "user", content: "one" },
-        { role: "user", content: "two" },
-      ],
-    },
-    null,
-    {
-      controller,
-      maxMessages: 10,
-      heavyMessages: 2,
-      heavyTools: 10,
-      heavyTokens: 10_000,
-      queueMs: 0,
-    }
-  );
-
-  assert.equal(result.admit, false);
-  if (!result.admit) assert.equal(result.response.status, 503);
-  held.release();
-});
-
-test("admission waiters are served FIFO as capacity frees", async () => {
-  const controller = new ChatAdmissionController(1);
-  const held = controller.tryAcquireHeavy();
-  assert.ok(held);
-
-  const body = {
-    messages: [
-      { role: "user", content: "one" },
-      { role: "user", content: "two" },
-    ],
-  };
-  const options = {
-    controller,
-    maxMessages: 10,
-    heavyMessages: 2,
-    heavyTools: 10,
-    heavyTokens: 10_000,
-    queueMs: 500,
-  };
-  const first = admitChatStructure(body, null, options);
-  const second = admitChatStructure(body, null, options);
-
-  held.release();
-  const firstResult = await first;
-  assert.equal(firstResult.admit, true);
-  if (firstResult.admit) firstResult.lease?.release();
-  const secondResult = await second;
-  assert.equal(secondResult.admit, true);
-  if (secondResult.admit) secondResult.lease?.release();
-  assert.equal(controller.activeHeavy, 0);
-});

tests/unit/per-connection-admission-9654.test.ts

@@ -0,0 +1,205 @@
+// #9654: Per-connection virtual admission lanes
+import test from "node:test";
+import assert from "node:assert/strict";
+
+const admissionModule = await import("../../src/shared/middleware/chatBodyAdmission.ts");
+const {
+  PerConnectionAdmissionController,
+  resolveSessionId,
+  admitChatRequest,
+  admitChatStructure,
+  perConnectionAdmissionController,
+  ChatAdmissionController,
+  CHAT_MAX_HEAVY_IN_FLIGHT,
+} = admissionModule;
+
+function makeRequest(headers: Record<string, string>, body = "{}"): Request {
+  const h: Record<string, string> = { "content-type": "application/json", ...headers };
+  return new Request("http://x/v1/chat/completions", { method: "POST", headers: h, body });
+}
+
+test("resolveSessionId hashes bearer token into opaque key", () => {
+  const req = makeRequest({ authorization: "Bearer sk-secret-key-123" });
+  const sid = resolveSessionId(req);
+  assert.ok(sid.startsWith("key_"));
+  assert.equal(sid.length, "key_".length + 16);
+  // Same key → same hash
+  const req2 = makeRequest({ authorization: "Bearer sk-secret-key-123" });
+  assert.equal(resolveSessionId(req2), sid);
+  // Different key → different hash
+  const req3 = makeRequest({ authorization: "Bearer sk-different-key-456" });
+  assert.notEqual(resolveSessionId(req3), sid);
+});
+
+test("resolveSessionId hashes x-api-key header (Anthropic-style)", () => {
+  const req = makeRequest({ "x-api-key": "anthropic-key-xyz" });
+  const sid = resolveSessionId(req);
+  assert.ok(sid.startsWith("key_"));
+  assert.equal(sid.length, "key_".length + 16);
+});
+
+test("resolveSessionId returns 'anonymous' for no auth", () => {
+  const req = makeRequest({}, "{}");
+  assert.equal(resolveSessionId(req), "anonymous");
+});
+
+test("resolveSessionId does not leak raw API key in the session ID", () => {
+  const req = makeRequest({ authorization: "Bearer sk-secret-key-123" });
+  const sid = resolveSessionId(req);
+  assert.ok(!sid.includes("sk-secret-key-123"));
+  assert.ok(!sid.includes("secret"));
+});
+
+test("PerConnectionAdmissionController isolates capacity across sessions", () => {
+  const pc = new PerConnectionAdmissionController(1);
+  const ctrlA = pc.getController("session-a");
+  const ctrlB = pc.getController("session-b");
+
+  // Session A acquires the only slot
+  const leaseA = ctrlA.tryAcquireHeavy();
+  assert.ok(leaseA);
+  // Session A is now full
+  assert.equal(ctrlA.tryAcquireHeavy(), null);
+  // Session B still has capacity — isolation works
+  const leaseB = ctrlB.tryAcquireHeavy();
+  assert.ok(leaseB);
+  leaseA.release();
+  leaseB.release();
+});
+
+test("PerConnectionAdmissionController returns same controller for same session", () => {
+  const pc = new PerConnectionAdmissionController(1);
+  const a1 = pc.getController("session-a");
+  const a2 = pc.getController("session-a");
+  assert.equal(a1, a2);
+});
+
+test("PerConnectionAdmissionController creates new controller for new session", () => {
+  const pc = new PerConnectionAdmissionController(1);
+  const a = pc.getController("session-a");
+  const b = pc.getController("session-b");
+  assert.notEqual(a, b);
+});
+
+test("PerConnectionAdmissionController enforces maxSessions LRU eviction", () => {
+  const pc = new PerConnectionAdmissionController(1, { maxSessions: 2, sessionTtlMs: 60000 });
+  const a = pc.getController("a");
+  const b = pc.getController("b");
+  assert.equal(pc.sessionCount, 2);
+  // Touch 'a' so 'b' is oldest
+  const aAgain = pc.getController("a");
+  assert.equal(aAgain, a, "same a reference");
+  // Creating 'c' should evict 'b' (oldest)
+  const c = pc.getController("c");
+  assert.equal(pc.sessionCount, 2);
+  // 'a' survives, 'b' is evicted
+  const aAfter = pc.getController("a");
+  assert.equal(aAfter, a, "a should still exist after c added");
+  // 'b' gets a fresh controller (old one was evicted)
+  const newB = pc.getController("b");
+  assert.notEqual(newB, b, "b should be evicted and recreated");
+});
+
+test("PerConnectionAdmissionController evicts idle sessions after TTL", async () => {
+  const pc = new PerConnectionAdmissionController(1, {
+    sessionTtlMs: 50,
+    maxSessions: 64,
+  });
+  const ctrl = pc.getController("idle-session");
+  assert.ok(ctrl);
+  assert.equal(pc.sessionCount, 1);
+
+  // Wait past TTL + eviction tick
+  await new Promise((resolve) => setTimeout(resolve, 120));
+  // Accessing again should trigger eviction → fresh controller
+  const fresh = pc.getController("idle-session");
+  assert.notEqual(fresh, ctrl);
+});
+
+test("PerConnectionAdmissionController snapshot does not leak raw keys", () => {
+  const pc = new PerConnectionAdmissionController(1);
+  pc.getController("key_abc123");
+  pc.getController("anonymous");
+  const snap = pc.snapshot();
+  assert.equal(snap.length, 2);
+  for (const entry of snap) {
+    assert.ok(typeof entry.sessionId === "string");
+    assert.ok(entry.sessionId.includes("key_abc123") || entry.sessionId === "anonymous");
+    assert.ok(typeof entry.activeHeavy === "number");
+    assert.ok(typeof entry.idleMs === "number");
+  }
+});
+
+test("admitChatRequest uses per-connection controller by default", async () => {
+  const result = await admitChatRequest(
+    makeRequest({ authorization: "Bearer sk-test-key" }),
+    { largeBodyBytes: 32, hardMaxBytes: 1024 }
+  );
+  assert.equal(result.admit, true);
+  if (result.admit) result.lease?.release();
+});
+
+test("admitChatRequest with explicit controller overrides per-connection lookup", async () => {
+  const explicitController = new ChatAdmissionController(1);
+  const result = await admitChatRequest(
+    makeRequest({ authorization: "Bearer sk-test-key" }),
+    { controller: explicitController, largeBodyBytes: 32, hardMaxBytes: 1024 }
+  );
+  assert.equal(result.admit, true);
+  if (result.admit) result.lease?.release();
+});
+
+test("admitChatStructure routes structural rejection to per-connection controller", async () => {
+  // occupy sess-a's per-connection controller via the module-level instance
+  const controller = perConnectionAdmissionController.getController("sess-a");
+  const occupied = controller.tryAcquireHeavy();
+  assert.ok(occupied);
+
+  const result = await admitChatStructure(
+    {
+      messages: Array.from({ length: 3 }, () => ({ role: "user", content: "x" })),
+    },
+    null,
+    {
+      sessionId: "sess-a",
+      maxMessages: 10,
+      heavyMessages: 1,
+      heavyTools: 10,
+      heavyTokens: 10_000,
+    }
+  );
+  // Session A is busy → 503
+  assert.equal(result.admit, false);
+  if (result.admit) return;
+  assert.equal(result.response.status, 503);
+  assert.equal(result.response.headers.get("Retry-After"), "1");
+  occupied.release();
+});
+
+test("admitChatStructure with different sessionId gets independent capacity", async () => {
+  // occupy sess-a's per-connection controller
+  const ctrlA = perConnectionAdmissionController.getController("sess-a");
+  const occupied = ctrlA.tryAcquireHeavy();
+  assert.ok(occupied);
+
+  // Session B should get its own controller → admitted
+  const result = await admitChatStructure(
+    {
+      messages: Array.from({ length: 500 }, () => ({ role: "user", content: "x" })),
+    },
+    null,
+    {
+      sessionId: "sess-b",
+      maxMessages: 0,
+      heavyMessages: 200,
+      heavyTools: 64,
+      heavyTokens: 32_000,
+    }
+  );
+  assert.equal(result.admit, true);
+  if (result.admit) {
+    assert.notEqual(result.lease, null);
+    result.lease?.release();
+  }
+  occupied.release();
+});