Commit Graph

2 Commits

Author SHA1 Message Date
anhtahaylove
fde6241d41 test: close the database before removing temp DATA_DIR (#13290) (#13292)
* test: close the database before removing temp DATA_DIR (#13290)

Tests that set their own DATA_DIR and removed it in test.after() failed on
Windows with EPERM: nothing closed the SQLite connection, so the directory
still had an open handle and the -shm/-wal sidecars kept it locked. maxRetries
could not help because every retry hit the same open handle.

Adds tests/_setup/tempDataDir.ts with cleanupTempDataDir()/createTempDataDir(),
which close the DB singleton (lazily imported, so tests that never touch the
database do not pull in the DB layer) and then remove the directory
best-effort. Applies it to the five suites confirmed failing.

The helper's own test proves the ordering matters: skipping the close makes it
fail with 'cleanup must remove the directory'.

* test: close the database before removing temp DATA_DIR (15 more suites)

Converts the suites that measurably emitted EPERM during a full run to the
shared cleanupTempDataDir helper from #13292.

Measured on the same 15 files:
  base   -> 22 fail, 40 EPERM lines
  branch ->  7 fail, 10 EPERM lines

The 7 remaining failures are pre-existing and unrelated to teardown:
rtk-learn-discover-routes and executor-map-golden already fail on a clean
base (6 and 3 failures respectively).

* test: close the database before removing temp DATA_DIR (final 9 suites)

Completes the #13290 sweep. Two teardown shapes needed the helper:

- after()/t.after() hooks that removed DATA_DIR directly
- beforeEach() hooks that wiped DATA_DIR between tests while the previous
  test's connection was still open. These failed *before* the test body ran,
  so every test in the file reported the same EPERM path.

Three of them already called core.resetDbInstance() right before rmSync and
still leaked, which is the product-side connection leak tracked in #13303.

Measured per file, EPERM lines now 0 across all nine. Remaining failures are
pre-existing on a clean base (firefly 4->1, driverFactory 1, responses-* 1
each) and unrelated to teardown.

* test: add the missing cleanupTempDataDir import to two responses suites

The previous commit swapped rmSync for cleanupTempDataDir in these two files but
did not add the import, so both suites died with
ReferenceError: cleanupTempDataDir is not defined before running any test.

responses-parse-once-4041:            0 pass / 1 fail -> 4 pass / 0 fail
responses-route-early-keepalive-wiring: 0 pass / 1 fail -> 3 pass / 0 fail

Both now report 0 EPERM.

* test: close SQLite handles in three silently-leaking suites

These three suites requested DATA_DIR cleanup but the delete failed on
Windows because a SQLite connection was still open. They pass today, so
the leak is invisible: they carry state between tests and would surface
later as an unrelated-looking assertion, as #13303 already did in the
Firefly suite (a 500 instead of a 401).

agentbridge-mitm-router-key-6403 and agent-bridge-bypass-flow removed
their own temp dir in test.after() without closing the DB first; both now
use the shared cleanupTempDataDir helper, which closes the singleton
before removing the directory.

issue-agent-route-execution is a different case: it has no teardown at
all, so the connection stayed open until process exit and the
isolateDataDir cleanup hook then hit EPERM. It now closes the DB in
test.after().

Verified with a probe on fs.rmSync: all three reported a failed delete
before, and zero across three consecutive runs after, while the same
probe still reports four leaks in the Firefly suite.

* test: remove temp DATA_DIR in five suites that never cleaned up

These five suites create their own mkdtemp DATA_DIR, open the SQLite DB and
never remove the directory, so every run leaves a storage.sqlite behind in the
OS temp dir. Each dir is private to its suite, so this leaked disk space rather
than corrupting results - but the churn is pointless.

Each now closes the DB and removes its directory through the shared
cleanupTempDataDir helper.

Verified with an exit-time probe that lists storage.sqlite* still present in
DATA_DIR: it fired for these suites before the change and is silent after,
with the same test counts (22/14/5/3/3 passing).
2026-09-17 02:31:53 -03:00
Markus Hartung
beb6ec857b feat(dashboard): agentic conversation tracking — v4, decoupled + storage-architecture concern resolved (#10263)
* feat(responses): virtualize previous_response_id continuation regardless of upstream support

OmniRoute now exposes OpenAI-compatible previous_response_id/store
continuation to clients unconditionally, even when the selected upstream
provider has no native Responses-API state support. Reconstruction happens
server-side in handleChatImplementation, before any downstream validation
or provider translation: OmniRoute resolves the response id back to the
full input/output it previously produced, prepends it to the client's
delta, and forwards the full reconstructed history upstream exactly as it
does today. Client<->OmniRoute traffic shrinks to the new delta only;
OmniRoute<->provider traffic is unchanged.

Storage reuses the existing call-log pipeline artifact (already gated by
call_log_pipeline_enabled, already retained/cleaned up by the existing
call-log lifecycle) instead of duplicating conversation content into a
second store -- only a lightweight call_logs.response_id index is new.
Every lookup is scoped by api_key_id so one client can never resolve
another client's stored conversation, and any unresolvable/missing/
size-limit-omitted state fails closed with OpenAI's own
previous_response_not_found contract.

Stacked on feat/openai-responses-store-toggle (#10121).

* feat(dashboard): agentic conversation tracking with live transcript view

Every agentic chat request now gets a conversation id (X-ConversationId
response header). OmniRoute detects when a follow-up request continues the
same conversation via fingerprint + bounded prefix-hash matching, with a
strict-growth invariant to prevent false merges between independent
single-shot requests that happen to share identical opening content.
Continuation detection excludes the system message from the identity
anchor, since real coding-agent CLIs commonly regenerate it every request
with live context (timestamp, cwd, git status) — without this, that
volatility alone broke every continuation check against real traffic.

- `/dashboard/logs`: new toggleable Conversation column.
- `/dashboard/logs/timeline`: requests sharing a conversation id share a
  timeline lane, connected by an arrow, with a configurable lane-reuse
  window.
- Request detail panel: new Full Conversation transcript above the raw SSE
  event stream — Markdown rendering, per-turn timestamps, turn-relative
  view, click-any-turn navigation, live auto-refresh building the
  transcript in real time from the in-flight SSE chunk buffer while a
  request is still streaming, auto-scroll-to-bottom as the live turn grows.
- New `/dashboard/conversations` page listing conversations with 2+ turns,
  no-forking model (an edited/duplicated mid-history turn mints its own
  independent conversation instead of merging), pagination, duplicate-
  anchor fix.
- Configurable auto-refresh intervals on both the timeline and
  conversations list pages.
- Responses API tool-call gap fix: turnsFromOpenAiMessages only handled
  role-based Chat Completions messages, so bare {type:"function_call"} /
  {type:"function_call_output"} / {type:"reasoning"} items (real Responses
  API traffic) silently vanished from the Conversation Context panel.
- truncateForLog now counts input[] (Responses API), not just messages[]
  (Chat Completions), so a truncated /v1/responses request still shows a
  placeholder instead of nothing.
- RequestTimeline.tsx now reads the same debugEnabled/emailsVisible
  settings RequestLoggerV2.tsx already used, instead of hardcoding both
  false — the timeline view never showed SSE/stream-chunk events or
  respected email-masking, regardless of the actual setting.

Migrations 147/148 (agentic_conversations, conversation_turn_nodes) — 135
and 136 are now taken upstream; 143-145 are documented KNOWN_GAPS, so this
uses the next free slot past upstream's current highest.

Test plan:
- npm run typecheck:core — clean
- npm run lint — clean
- node --import tsx/esm scripts/check/check-migration-numbering.mjs — OK, 0 collisions
- 109 unit tests across the conversation-tracking, migration-renumber, and
  dashboard-wiring surface — 0 failures

* refactor(dashboard): reuse call-log artifacts for conversation transcript content

conversation_turn_nodes no longer stores turn text/tool-call content
(text_preview/block_kind/tool_name) -- it's identity-only now (id/parent/
content_hash), matching agentic_conversations' existing lightweight-index
shape. Every node's originating request is already fully captured by the
call-log pipeline artifact its last_correlation_id points at, so the
/dashboard/conversations tree view resolves each node's actual display
content on demand from there (open-sse/services/conversationTurnContent.ts),
re-running the same extractCanonicalTurns/hashTurnContent the write path
used and matching by content_hash, instead of duplicating conversation
content into a second store under a separate retention/gating policy. This
also drops the old 8000-char text_preview truncation entirely -- resolved
content is always full and untruncated.

The frontend contract is unchanged (tree API still returns
{textPreview, blockKind, toolName} per node), so the dashboard UI itself
(page.tsx, RequestLoggerDetail/RequestTimeline, sidebar, i18n) needed no
changes.

Renumbered the cherry-picked 147/148 migrations to 153/154 -- 147 now
collides with 147_api_keys_model_access_mode.sql, which landed on
release/v3.8.50 after this work was originally built.

Also includes a standalone, unrelated fix carried along from this rebase:
close isProviderModelHidden's missing function-body brace in
modelSelectModalHelpers.ts (separately landed as #10206).

Stacked on feat/responses-previous-response-id-virtualization (#3), which
is itself stacked on feat/openai-responses-store-toggle (#10121).

* fix(dashboard): resync conversation list on open so the live-text poll starts immediately

openConversation() seeded activeConversation (and therefore activeCallLogId,
which gates the live-partial-text poll effect) from whatever row snapshot the
list's own fixed-interval poll last produced. A conversation opened right
after a reply started streaming -- after that tick, before the next -- had
activeCallLogId still null, so the live-text poll never started; only a
subsequent background list-poll resync (already existed) picked it up,
which is why closing and reopening the same conversation "just worked".

loadConversations() is now a shared callback so openConversation can force
one immediately on open instead of waiting on pollSeconds.

Live-verified against omniroute-dev: opening a conversation mid-stream now
shows live reasoning on the first open.

* style: prettier formatting for conversationTurnContent.test.ts

* fix(db): close migration numbering gap left by decoupling from #3/#10262

153/154 (originally 154/155) were chosen back when this branch stacked on
top of the previous_response_id migration (153_call_logs_response_id.sql).
Decoupling removed that migration from this branch's history, leaving an
unused 153 slot that check-migration-numbering.test.ts correctly flags as
a gap.

* refactor(dashboard): split RequestTimeline/RequestLoggerDetail under the 1000-line file-size cap

Both files exceeded check-file-size's new-file cap after this PR's own
additions (RequestTimeline 1048, RequestLoggerDetail 1163). Extracted pure
non-component logic (types, constants, allocateLanes and its helpers) out
of RequestTimeline.tsx into RequestTimeline.utils.ts, and the two
self-contained presentational sub-components (PayloadSection,
ConversationContextSection + its private helper) out of
RequestLoggerDetail.tsx into RequestLoggerDetail.sections.tsx. No behavior
change; existing external imports (default exports, allocateLanes,
TimelineLog, CONVERSATION_LANE_REUSE_STORAGE_KEY) still resolve from the
original file paths.

* fix(db): renumber agentic-conversation migrations to clear 153 collision + sync migration-count docs

The refresh-merge of release/v3.8.50 exposed that the feature's three
migrations collided at slot 153 with the base's radar_local_model_state
(153) and its own call_logs_response_id. Migration runner enforces unique
numeric prefixes -> every DB init threw, red-ing Vitest, all Unit shards and
the DB-backed quality gates. Renumber the feature's pair to
155_agentic_conversations / 156_conversation_turn_nodes and move
call_logs_response_id to 154 (keeps 153_radar base-owned, preserves
agentic-before-turn_nodes ordering). Update SQL headers and the
154/156 references in feature code + tests.

Migration count is now 151 (was 148 stale in README/AGENTS/llm.txt) — sync
the doc counts to clear the docs-accuracy gate.

Co-authored-by: diegosouzapw <diegosouza.pw@gmail.com>

* fix(ui): drop unused CONVERSATION_LANE_REUSE_STORAGE_KEY re-export from RequestTimeline

Knip 6.32 (baseline 415) flags the public re-export of
CONVERSATION_LANE_REUSE_STORAGE_KEY from RequestTimeline.tsx as dead: no
external consumer imports it through that re-export (it is imported and
used directly from RequestTimeline.utils.ts inside the component). Removed
the unused re-export; the internal import stays. DEAD_TOTAL 416 -> 415,
back to the frozen baseline.

Co-authored-by: diegosouzapw <diegosouza.pw@gmail.com>

* fix(agentic-conversations): guard resolveConversationId, drop dead whole-chain export

- Wrap resolveConversationId() in try/catch in chat.ts, matching the
  defensive pattern used by every other best-effort side call nearby, so a
  DB hiccup in conversation tracking can't turn a working chat request into
  a hard failure.
- Remove getConversationTurnTree: knip's project scope excludes tests/**,
  so an export used only by tests can never register as used there. Swap
  its 8 test call sites to the paginated getConversationTurnPage (already
  the dashboard's canonical query) with a generous limit, collapsing to one
  query path instead of keeping a second whole-chain export alive solely
  for test convenience.
- Regenerate i18n llm.txt mirrors from root (pre-existing drift on this
  branch, unrelated to the above, caught by the docs-sync pre-commit gate).

Addresses PR review feedback.

* fix(i18n): close requestLogger conversation-column gap, fix domain-modules count drift

- fr.json, vi.json were missing requestLogger.columns.conversation (added
  in the conversation-tracking feature), failing i18n-vi-completeness.test.ts.
- docs/i18n/*/llm.txt mirrors still said 117 domain-specific files after an
  earlier rebase fixed the migration count but missed this companion number,
  failing check-docs-sync.mjs across all 42 locales.

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>

* fix(docs): restore PROXY_LOG_INCLUDE_IPS env/doc entries (env-doc-sync red)

.env.example and docs/reference/ENVIRONMENT.md were both missing the
PROXY_LOG_INCLUDE_IPS entry that src/lib/proxyLogger.ts already reads
(confirmed present at this branch's merge-base too, so this predates
the conversation-tracking work and is unrelated to it) -- the entry
was added on release/v3.8.50 after this branch's last sync and this
branch never picked it up. That gap red-lines
tests/unit/check-env-doc-sync.test.ts and
tests/unit/issue-7793-env-doc-sync-repro.test.ts (Unit Tests
fast-path 2/4 in CI). Restore both entries verbatim from the current
release/v3.8.50 tip -- no feature-code change.

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>

---------

Co-authored-by: hartmark <hartmark@users.noreply.github.com>
Co-authored-by: diegosouzapw <diegosouza.pw@gmail.com>
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
2026-08-18 11:32:33 -03:00