diff --git a/docs/architecture/REPOSITORY_MAP.md b/docs/architecture/REPOSITORY_MAP.md index 0fed1da379..85b7349f31 100644 --- a/docs/architecture/REPOSITORY_MAP.md +++ b/docs/architecture/REPOSITORY_MAP.md @@ -123,6 +123,7 @@ src/ | `app/a2a/` | A2A JSON-RPC 2.0 entry point (`POST /a2a`) | | `app/.well-known/agent.json/` | A2A Agent Card (discovery) | | `app/(dashboard)/dashboard/` | Dashboard UI pages (~30 pages: providers, combos, settings, memory, skills, webhooks, evals, audit, batch, cache, costs, health, system, etc.) | +| `app/(dashboard)/dashboard/memory/` | Memory Studio (plan 21): `page.tsx` (3-tab shell), `components/` (MemoryConceptCard, MemoryEngineStatus, EmbeddingSourceSelector, EditMemoryModal, RetrievePreview, QdrantConfigCard, RerankConfigCard), `components/tabs/` (MemoriesTab, PlaygroundTab, EngineTab), `hooks/` (useEngineStatus, useMemorySettings) | | `app/docs/` | Embedded documentation viewer (renders `docs/*.md`) | | `app/landing/` | Marketing landing page | | `app/login/`, `forgot-password/`, `forbidden/` | Auth-related pages | @@ -153,7 +154,10 @@ src/ | `evals/` | Eval framework (suites, runner, runtime) — see `docs/frameworks/EVALS.md` | | `guardrails/` | PII masker, prompt injection, vision bridge — see `docs/security/GUARDRAILS.md` | | `jobs/` | Background jobs (cron-like) | -| `memory/` | Conversational memory (SQLite FTS5 + Qdrant) — see `docs/frameworks/MEMORY.md` | +| `memory/` | Conversational memory (SQLite FTS5 + sqlite-vec hybrid RRF + Qdrant tier 2) — see `docs/frameworks/MEMORY.md` | +| `memory/embedding/` | Multi-source embedding layer: `index.ts` (resolver), `remote.ts`, `staticPotion.ts`, `transformersLocal.ts`, `cache.ts`, `types.ts` (plan 21) | +| `memory/vectorStore.ts` | sqlite-vec v0.1.9 wrapper — KNN brute-force + hybrid RRF (FTS5 + vector, k=60). Lazy-init, degrades gracefully when sqlite-vec unavailable. (plan 21) | +| `memory/reindex.ts` | `runReindexBatch()` — processes memories with `needs_reindex=1` in background; called by `POST /api/memory/reindex` and lazy-backfill path. (plan 21) | | `monitoring/` | Health checks, metrics emission | | `oauth/` | OAuth flows for 14 providers (claude, codex, antigravity, cursor, github, gemini, kimi-coding, kilocode, cline, qwen, kiro, qoder, gitlab-duo, windsurf) | | `plugins/` | Plugin registry | @@ -172,7 +176,8 @@ src/ | Subdir | Purpose | | ---------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `db/core.ts` | `getDbInstance()` singleton with WAL journaling | -| `db/migrations/` | 55 versioned SQL files (idempotent, transactional, numbered `001`..`055`) | +| `db/migrations/` | Versioned SQL files (idempotent, transactional). `073_memory_vec.sql` adds `memory_vec_meta` + `needs_reindex` column (plan 21). | +| `db/memoryVec.ts`| CRUD for `memory_vec_meta` (active_dim, embedding_signature, last_reset_at, vec_loaded) + `markMemoryNeedsReindex`, `getMemoryReindexQueue`, etc. (plan 21) | | `db/.ts` | One module per domain: providers, combos, apiKeys, users, sessions, usage, audit*log, webhooks, skills, memory_entries, cloud_agent_tasks, evals*\*, reasoning_cache, etc. | ### `src/domain/` diff --git a/docs/frameworks/MEMORY.md b/docs/frameworks/MEMORY.md index 07b226fcae..018a551ed5 100644 --- a/docs/frameworks/MEMORY.md +++ b/docs/frameworks/MEMORY.md @@ -1,13 +1,13 @@ --- title: "Memory System" -version: 3.8.2 -lastUpdated: 2026-05-13 +version: 3.8.6 +lastUpdated: 2026-05-28 --- # Memory System > **Source of truth:** `src/lib/memory/` and `src/app/api/memory/` -> **Last updated:** 2026-05-13 — v3.8.0 +> **Last updated:** 2026-05-28 — v3.8.6 (plan 21 — Memory Engine Redesign) OmniRoute provides persistent conversational memory keyed by API key (and optionally session id). Memories are extracted automatically from LLM responses @@ -27,17 +27,143 @@ Client → /v1/chat/completions (apiKeyInfo resolved upstream) → resolveMemoryOwnerId(apiKeyInfo) # extracts id → getMemorySettings() # cached settings → shouldInjectMemory(body, {enabled}) # gate - → retrieveMemories(apiKeyId, config) # SQL + optional FTS5 + → retrieveMemories(apiKeyId, config) # SQL + FTS5 + optional vector → injectMemory(body, memories, provider) # system or user message → upstream provider call → on response: extractFacts(text, apiKeyId, sessionId) # non-blocking → setImmediate → createMemory(fact) per match + → embed(content) + upsertVector(id, vec) ``` The injection and extraction call-sites are wired in `open-sse/handlers/chatCore.ts` (look for `retrieveMemories`, `injectMemory`, and `extractFacts`). +## Engine architecture (3-tier resolution) + +The Memory Engine resolves the retrieval path at runtime based on available +infrastructure and settings. Three tiers exist, applied in priority order: + +``` + ┌─────────────────────────────────────────────────────────────┐ + │ TIER 0 — Keyword (FTS5) │ + │ Always available. SQLite FTS5 full-text search over │ + │ content + key. Used when strategy = "exact" or as fallback. │ + └──────────────────────────────────┬──────────────────────────┘ + │ strategy = semantic|hybrid? + ▼ + ┌─────────────────────────────────────────────────────────────┐ + │ TIER 1 — Embedded Vector (sqlite-vec) │ + │ sqlite-vec v0.1.9 loaded via db.loadExtension(). │ + │ KNN brute-force over Float32 vectors. Active when: │ + │ • sqlite-vec loadExtension succeeds │ + │ • An embedding source is available (remote | static | │ + │ transformers) that can produce a Float32Array │ + │ • vec_memories table exists (created on first ready()) │ + └──────────────────────────────────┬──────────────────────────┘ + │ qdrant.enabled? + ▼ + ┌─────────────────────────────────────────────────────────────┐ + │ TIER 2 — Qdrant (opt-in external vector database) │ + │ When enabled, replaces sqlite-vec for semantic/hybrid. │ + │ Requires running Qdrant instance + configured host/port. │ + └─────────────────────────────────────────────────────────────┘ +``` + +Degradation is automatic and transparent: +- If sqlite-vec fails to load, tier 1 is unavailable → falls back to tier 0. +- If embedding source returns an error, tier 1 falls back to tier 0. +- If Qdrant is unhealthy, tier 2 falls back to tier 1 (or tier 0 if tier 1 + is also unavailable). + +## Embedding sources + +The embedding layer (`src/lib/memory/embedding/`) resolves which source to use +based on `MemorySettingsExtended.embeddingSource`: + +| Source | Description | Key required | Cold start | +| -------------- | ------------------------------------------------------------------------------- | ------------ | ---------- | +| `remote` | Uses a configured provider's embedding API (OpenAI, Cohere, etc.) | Yes | None | +| `static` | Local lookup-table embedding via `potion-base-8M` (WordPiece + mean pooling) | No | ~200ms | +| `transformers` | Local ONNX inference via `@huggingface/transformers` v4, `all-MiniLM-L6-v2` | No | ~3s + ~400MB RAM | +| `auto` | Runtime resolution: remote (if key exists) → static → transformers → null | Depends | Depends | + +**Resolution order for `auto`:** +1. Find first provider in `listEmbeddingProviders()` with `hasKey === true` → `remote`. +2. If `settings.staticEnabled === true` → `static`. +3. If `settings.transformersEnabled === true` → `transformers`. +4. Otherwise → `null` (degrades to FTS5 keyword search). + +The embedding cache (`src/lib/memory/embedding/cache.ts`) uses an in-memory +LRU map keyed by `${source}:${model}:${dim}:${sha256(text)}`, capped at +`MEMORY_EMBEDDING_CACHE_MAX` entries (default 1000) with a TTL of +`MEMORY_EMBEDDING_CACHE_TTL_MS` (default 5 min). Shared across all callers +per process lifecycle. + +## Hybrid RRF (k=60) + +When `strategy = "hybrid"` and the vector store is available, retrieval uses +Reciprocal Rank Fusion to merge FTS5 and vector results: + +``` +RRF(d) = Σ 1 / (k + rank_i(d)) where k = 60 (configurable via MEMORY_RRF_K) + i +``` + +Concretely: +1. Run FTS5 search → ranked list `R_fts` (position 1..N). +2. Run KNN vector search → ranked list `R_vec` (position 1..M). +3. For each unique `memoryId`: + `rrf_score = 1/(60 + fts_rank)` + `1/(60 + vec_rank)` (0 if not in list). +4. Sort by `rrf_score` DESC, apply token budget walk. + +RRF is well-known to be effective without needing score normalization across +heterogeneous retrieval systems. The default `k=60` is from the original +Cormack et al. paper and works well for small corpora (<10k memories). + +## Backfill (lazy + reindex) + +When the embedding model changes (detected via `embedding_signature`), the +vector store is rebuilt and all existing memories are marked +`needs_reindex = 1` in the `memories` table. + +**Lazy backfill**: On the next retrieval, any memory missing a vector entry is +embedded and inserted into `vec_memories` before the search runs. This +amortizes the backfill cost across real requests without blocking startup. + +**Explicit reindex**: The Engine tab in `/dashboard/memory` provides a +"Reindex Now" button that calls `POST /api/memory/reindex`. The handler calls +`runReindexBatch()` from `src/lib/memory/reindex.ts`, which processes up to +`limit` pending entries per request. Progress can be polled via +`GET /api/memory/engine-status` (`vectorStore.needsReindex`). + +The `memory_vec_meta` table (migration `073_memory_vec.sql`) stores: +- `active_dim` — current vector dimension (null = not yet calibrated). +- `embedding_signature` — `${source}:${model}:${dim}` used to detect changes. +- `last_reset_at` — timestamp of last full reset. +- `vec_loaded` — 0/1 flag whether sqlite-vec loaded successfully. + +## Settings extension + +Seven new fields were added to `MemorySettingsExtended` (plan 21, D9) in +`src/shared/schemas/memory.ts`, persisted via `src/lib/db/settings.ts`: + +| Field | Type | Default | Description | +| ---------------------- | --------------------------------------------- | ------------ | --------------------------------------------- | +| `embeddingSource` | `"remote" \| "static" \| "transformers" \| "auto"` | `"auto"` | Which embedding source to use | +| `embeddingProviderModel` | `string \| null` | `null` | Provider/model in `provider/model` format | +| `transformersEnabled` | `boolean` | `false` | Opt-in for Transformers.js (MiniLM, ~400MB) | +| `staticEnabled` | `boolean` | `false` | Opt-in for static potion-base-8M local model | +| `rerankEnabled` | `boolean` | `false` | Enable reranking step (adds +200-500ms/req) | +| `rerankProviderModel` | `string \| null` | `null` | Rerank provider/model in `provider/model` format | +| `vectorStore` | `"sqlite-vec" \| "qdrant" \| "auto"` | `"auto"` | Which vector backend to use | + +These are exposed via `GET /PUT /api/settings/memory` (schema `MemorySettingsExtendedSchema`). + +> **TODO (D20):** Scope `global` (sharing memories across all API keys) is not +> implemented in this release. It requires schema changes and a global retrieval +> path. Track separately. + ## Storage Layers ### Primary: SQLite (`memories` table) @@ -78,10 +204,10 @@ Used by `retrieval.ts` for the `semantic` and `hybrid` strategies (see below). The retrieval code guards with `hasTable("memory_fts")` and falls back to chronological order if the FTS table is missing or the FTS query throws. -### Optional: Qdrant (vector store) +### Optional: Qdrant (vector store tier 2) -`src/lib/memory/qdrant.ts` implements an optional Qdrant integration for true -semantic memory: +`src/lib/memory/qdrant.ts` implements an optional Qdrant integration as tier 2 +vector store. Enabled via `qdrantEnabled` in settings / toggle in Engine tab. - `upsertSemanticMemoryPoint()` — embed `key + content` with the configured embedding model, ensure the collection exists (creates cosine-distance @@ -90,24 +216,24 @@ apiKeyId, sessionId, key, content, metadata, createdAtUnix, expiresAtUnix}`. - `searchSemanticMemory(query, topK, scope)` — embed the query, search the collection filtered by `kind = "omniroute_memory"` and optionally by `apiKeyId` / `sessionId`. Caps `topK` to `[1, 20]`. -- `deleteSemanticMemoryPoint(id)` — single point delete. +- `deleteSemanticMemoryPoint(id)` — single point delete. Called by + `deleteMemory()` after the SQLite row is removed (D15). - `cleanupSemanticMemoryPoints({retentionDays})` — bulk delete points whose `expiresAtUnix` is in the past or whose `createdAtUnix` is older than the retention cutoff. Counts first so the dashboard can show actual numbers. - `checkQdrantHealth()` — `GET /readyz` health probe with latency. -> **TODO**: The chat pipeline (`chatCore.ts`) and the in-tree `retrieveMemories()` -> implementation do not currently call `upsertSemanticMemoryPoint` or -> `searchSemanticMemory`. The Qdrant integration is feature-flagged via -> `qdrantEnabled` in settings, but at the time of writing the -> `searchSemanticMemory` results are not fused into retrieval — the -> `semantic`/`hybrid` retrieval strategies use SQLite FTS5 only. The settings UI -> in `dashboard/settings → MemorySkillsTab` exposes Qdrant config, health, -> search test, and cleanup, but the corresponding `/api/settings/qdrant`, -> `/api/settings/qdrant/health`, `/api/settings/qdrant/search`, and -> `/api/settings/qdrant/cleanup` routes are referenced from the UI but **not -> present** under `src/app/api/settings/qdrant/` (only `embedding-models/` is -> wired). Treat Qdrant as preview/optional plumbing. +The settings UI exposes Qdrant config, health check, semantic search test, +and cleanup in the **Engine tab** of `/dashboard/memory`. The corresponding +routes under `src/app/api/settings/qdrant/` are all wired as of v3.8.6: + +| Route | Method | Description | +| ----- | ------ | ----------- | +| `/api/settings/qdrant` | `GET` / `PUT` | Read / update Qdrant settings | +| `/api/settings/qdrant/health` | `GET` | Liveness probe + latency | +| `/api/settings/qdrant/search` | `POST` | Semantic search test | +| `/api/settings/qdrant/cleanup` | `POST` | Remove expired / old points | +| `/api/settings/qdrant/embedding-models` | `GET` | List available embedding models | ## Memory Types @@ -202,6 +328,8 @@ Memory configuration is **stored in the DB settings table**, not in env vars. in-process; `invalidateMemorySettingsCache()` is called by the settings PUT route after writes. +### Legacy fields (all versions) + | DB key | Type | Default | UI control | | --------------------- | ------- | -------------------------------------------------- | ----------------------------------------------- | | `memoryEnabled` | boolean | `true` | Memory on/off | @@ -213,14 +341,38 @@ route after writes. Note: the UI strategy `"recent"` maps to the internal `"exact"` retrieval strategy via `toMemoryRetrievalConfig()` (chronological order). +### New fields (v3.8.6, plan 21 D9) + +See also the "Settings extension" section above for field descriptions. + +| DB key | API field | Default | +| ------------------------- | ---------------------- | ------------- | +| `memoryEmbeddingSource` | `embeddingSource` | `"auto"` | +| `memoryEmbeddingModel` | `embeddingProviderModel` | `null` | +| `memoryTransformersEnabled` | `transformersEnabled` | `false` | +| `memoryStaticEnabled` | `staticEnabled` | `false` | +| `memoryRerankEnabled` | `rerankEnabled` | `false` | +| `memoryRerankModel` | `rerankProviderModel` | `null` | +| `memoryVectorStore` | `vectorStore` | `"auto"` | + Qdrant-related DB keys (`qdrantEnabled`, `qdrantHost`, `qdrantPort`, `qdrantApiKey`, `qdrantCollection` default `"omniroute_memory"`, `qdrantEmbeddingModel` default `"openai/text-embedding-3-small"`) are read by `normalizeQdrantConfig()` in `qdrant.ts`. -No `MEMORY_*` or `QDRANT_*` env vars exist today — everything is per-instance -DB settings. `OMNIROUTE_MEMORY_MB` (commented out in `.env.example`) is -unrelated and refers to Node heap sizing. +### Environment variables (v3.8.6) + +Six optional env vars tune the engine's runtime behaviour (documented in `.env.example`): + +| Variable | Default | Description | +| ------------------------------- | ------- | -------------------------------------------------- | +| `MEMORY_EMBEDDING_CACHE_TTL_MS` | `300000` | Embedding cache TTL (5 min) | +| `MEMORY_EMBEDDING_CACHE_MAX` | `1000` | Max entries in embedding LRU cache | +| `MEMORY_TRANSFORMERS_MODEL` | `Xenova/all-MiniLM-L6-v2` | HF repo for Transformers.js model | +| `MEMORY_STATIC_MODEL` | `minishlab/potion-base-8M` | HF repo for static potion model | +| `MEMORY_STATIC_CACHE_DIR` | `/embeddings` | Where to store downloaded models | +| `MEMORY_VEC_TOP_K` | `20` | Default top-K for vector search | +| `MEMORY_RRF_K` | `60` | RRF k constant for hybrid search | ## Summarisation (`summarization.ts`) @@ -240,15 +392,39 @@ loss is one-way: original text is overwritten. All endpoints require management auth (`requireManagementAuth`). -| Method | Path | Description | -| -------- | ---------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `GET` | `/api/memory` | Paginated list with filters: `apiKeyId`, `type`, `sessionId`, `q`, `limit`, `page`, `offset`. Response includes `stats.total` and `stats.byType` | -| `POST` | `/api/memory` | Create entry (Zod-validated: `content`, `key`, optional `type`, `sessionId`, `apiKeyId`, `metadata`, `expiresAt`). Calls `createMemory()` which upserts on `(apiKeyId, key)` | -| `GET` | `/api/memory/[id]` | Fetch a single entry by UUID | -| `DELETE` | `/api/memory/[id]` | Delete an entry; returns 404 when missing | -| `GET` | `/api/memory/health` | Runs `verifyExtractionPipeline("health-check")` — round-trip create→list→delete to confirm the store is alive. Returns `{working, latencyMs, error?}` | -| `GET` | `/api/settings/memory` | Current normalised `MemorySettings` | -| `PUT` | `/api/settings/memory` | Update one or more of `enabled`, `maxTokens`, `retentionDays`, `strategy`, `skillsEnabled` | +### Core memory endpoints (existing + updated) + +| Method | Path | Description | +| ---------- | --------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `GET` | `/api/memory` | Paginated list with filters: `apiKeyId`, `type`, `sessionId`, `q`, `limit`, `page`, `offset`. Response includes `stats.total`, `stats.tokensUsed`, `stats.hitRate`, `cacheStats` | +| `POST` | `/api/memory` | Create entry (Zod-validated: `content`, `key`, optional `type`, `sessionId`, `apiKeyId`, `metadata`, `expiresAt`). Calls `createMemory()` which upserts on `(apiKeyId, key)` | +| `GET` | `/api/memory/[id]` | Fetch a single entry by UUID | +| `PUT` | `/api/memory/[id]` | Update entry fields (`type`, `key`, `content`, `metadata`). Body: `MemoryUpdatePutSchema`. Also syncs vector if embedding source available. | +| `DELETE` | `/api/memory/[id]` | Delete an entry; also deletes from `vec_memories` (D15) and Qdrant best-effort. Returns 404 when missing. | +| `GET` | `/api/memory/health` | Runs `verifyExtractionPipeline("health-check")` — round-trip create→list→delete. Returns `{working, latencyMs, error?}` | + +### New memory engine endpoints (plan 21) + +| Method | Path | Description | +| -------- | ---------------------------------- | --------------------------------------------------------------------------------------- | +| `POST` | `/api/memory/retrieve-preview` | Dry-run of `retrieveMemories` — returns ranked results with score, tier, tokens. Body: `RetrievePreviewSchema`. Does NOT inject or modify memories. | +| `GET` | `/api/memory/embedding-providers` | Lists providers with embedding models, indicating which have a configured API key. | +| `GET` | `/api/memory/engine-status` | Returns full engine status: keyword tier, embedding resolution, vector store stats, Qdrant health, rerank config. Shape: `MemoryEngineStatusSchema`. | +| `POST` | `/api/memory/summarize` | Manually trigger memory compaction. Body: `MemorySummarizeSchema` (`olderThanDays`, `apiKeyId?`, `dryRun`). Returns `{candidates, tokensSaved}`. | +| `POST` | `/api/memory/reindex` | Trigger vector reindex for memories with `needs_reindex=1`. Body: `MemoryReindexSchema` (`force`). Returns `{started, pending}`. | + +### Settings endpoints + +| Method | Path | Description | +| -------- | ---------------------------------- | ------------------------------------------------------------------------ | +| `GET` | `/api/settings/memory` | Current normalised `MemorySettingsExtended` (7 new fields + legacy) | +| `PUT` | `/api/settings/memory` | Update any field from `MemorySettingsExtendedSchema` (12 total fields) | +| `GET` | `/api/settings/qdrant` | Current Qdrant settings (`QdrantSettingsSchema`) | +| `PUT` | `/api/settings/qdrant` | Update Qdrant settings. Body: `QdrantSettingsUpdateSchema`. `apiKey` = empty string removes key. | +| `GET` | `/api/settings/qdrant/health` | Liveness probe against configured Qdrant instance. Returns `QdrantHealthResultSchema`. | +| `POST` | `/api/settings/qdrant/search` | Semantic search test against Qdrant. Body: `QdrantSearchSchema` (`query`, `topK`). | +| `POST` | `/api/settings/qdrant/cleanup` | Remove Qdrant points for expired / old memories. | +| `GET` | `/api/settings/qdrant/embedding-models` | List embedding models available for Qdrant. | The `/api/memory` list query supports either `page`-based pagination (`parsePaginationParams`) **or** raw `offset` — when `offset` is present it @@ -259,31 +435,55 @@ takes precedence and a derived `page` is computed for the response shape. When the MCP server is enabled, three memory tools are registered: - `omniroute_memory_search` — `{apiKeyId, query?, type?, maxTokens?, limit?}` - → wraps `retrieveMemories()` with `retrievalStrategy: "exact"`, optionally - filters by `type`, and reports `totalTokens`. + → wraps `retrieveMemories()`. As of v3.8.6 (D16), the `strategy` is read + from `getMemorySettings()` instead of being hardcoded to `"exact"`. If + `query` is provided and `strategy` is `semantic` or `hybrid`, the vector + store is used when available. - `omniroute_memory_add` — `{apiKeyId, sessionId?, type, key, content, -metadata?}` → wraps `createMemory()`. +metadata?}` → wraps `createMemory()`. Accepts only the 4 canonical types: + `factual`, `episodic`, `procedural`, `semantic` (D17). - `omniroute_memory_clear` — `{apiKeyId, type?, olderThan?}` → lists matching entries, optionally filters by created-before timestamp, then deletes each - via `deleteMemory()`. + via `deleteMemory()` (which also removes vectors from sqlite-vec + Qdrant). See [MCP-SERVER.md](./MCP-SERVER.md) for transport and scope details. -## Dashboard +## Dashboard (Memory Studio) -`src/app/(dashboard)/dashboard/memory/page.tsx` provides: +`src/app/(dashboard)/dashboard/memory/page.tsx` is now a **3-tab Studio**: +### Tab: Memórias / Memories +- Concept card (collapsible "How it works" explainer). - Real-time list, search, and pagination (debounced 300 ms). - Type filter (`factual` / `episodic` / `procedural` / `semantic` / all). - Add-memory modal (key, content, type). -- Delete per row. +- Inline edit (pencil button → `PUT /api/memory/[id]`). +- Delete per row (with confirmation dialog). - JSON export of the current page; JSON import via file picker. +- Stat cards: `totalEntries`, `tokensUsed`, `hitRate`. +- "Compact old" button → `POST /api/memory/summarize` (dry-run first shows + candidate count, then confirms). - A green/red health dot driven by `GET /api/memory/health`. -- Stat cards: `totalEntries`, `tokensUsed`, `hitRate` (the latter two come - from the API stats payload). -Memory and Qdrant settings live under -`/dashboard/settings → Memory & Skills` (`MemorySkillsTab.tsx`). +### Tab: Playground +- Query input + strategy selector (Exact / Semantic / Hybrid) + token budget. +- "Simulate" → `POST /api/memory/retrieve-preview` — shows ranked results with + `score`, `tier`, `tokens`, `vecScore`, `ftsScore`. +- Resolution panel showing which embedding source / vector store was used and + whether a fallback occurred. + +### Tab: Engine +- Engine status panel (keyword FTS5 chip, embedding chip, vector store chip, + Qdrant health chip, rerank chip). +- "Reindex Now" button → `POST /api/memory/reindex`. +- Embedding source selector (auto / remote / static / transformers + toggles). +- Qdrant config card (enable toggle, host/port/collection/key, test connection, + semantic search test, cleanup). +- Rerank config card (enable toggle, provider/model selector). + +Memory and Qdrant settings also live under +`/dashboard/settings → Memory & Skills` (`MemorySkillsTab.tsx`) for +the legacy/global settings surface. ## Caching @@ -316,12 +516,28 @@ default TTL 5 min). - [API_REFERENCE.md](../reference/API_REFERENCE.md) — broader API surface. - Source modules: - `src/lib/memory/types.ts`, `schemas.ts` - - `src/lib/memory/store.ts`, `retrieval.ts`, `injection.ts` + - `src/lib/memory/store.ts`, `retrieval.ts`, `injection.ts`, `reindex.ts` - `src/lib/memory/extraction.ts`, `summarization.ts`, `verify.ts` - `src/lib/memory/settings.ts`, `qdrant.ts`, `cache.ts` + - `src/lib/memory/vectorStore.ts` — sqlite-vec + hybrid RRF + - `src/lib/memory/embedding/index.ts` — multi-source embedding layer + - `src/lib/memory/embedding/types.ts`, `remote.ts`, `staticPotion.ts`, + `transformersLocal.ts`, `cache.ts` + - `src/shared/schemas/memory.ts` — Zod schemas for all memory API bodies + - `src/shared/schemas/qdrant.ts` — Zod schemas for Qdrant settings/ops + - `src/lib/db/memoryVec.ts` — CRUD for `memory_vec_meta` - `src/lib/db/migrations/015_create_memories.sql`, - `022_add_memory_fts5.sql`, `023_fix_memory_fts_uuid.sql` + `022_add_memory_fts5.sql`, `023_fix_memory_fts_uuid.sql`, + `073_memory_vec.sql` - `src/app/api/memory/route.ts`, `[id]/route.ts`, `health/route.ts` + - `src/app/api/memory/retrieve-preview/route.ts` + - `src/app/api/memory/engine-status/route.ts` + - `src/app/api/memory/embedding-providers/route.ts` + - `src/app/api/memory/summarize/route.ts` + - `src/app/api/memory/reindex/route.ts` - `src/app/api/settings/memory/route.ts` + - `src/app/api/settings/qdrant/route.ts` + sub-routes + - `src/app/(dashboard)/dashboard/memory/` — Studio UI (page + components + + tabs + hooks) - `open-sse/handlers/chatCore.ts` (injection / extraction wiring) - `open-sse/mcp-server/tools/memoryTools.ts` diff --git a/docs/reference/openapi.yaml b/docs/reference/openapi.yaml index faa3b50d07..7d6bff37b6 100644 --- a/docs/reference/openapi.yaml +++ b/docs/reference/openapi.yaml @@ -75,6 +75,11 @@ tags: description: Fallback chain management - name: Telemetry description: Telemetry and token health monitoring + - name: Memory + description: >- + Conversational memory management — CRUD, engine status, playground preview, + summarization, reindex, and Qdrant settings (plan 21 — v3.8.6). + All routes require management auth. paths: # ─── Proxy Endpoints ────────────────────────────────────────── @@ -2730,6 +2735,801 @@ paths: "500": description: Failed to parse OpenAPI spec + # ─── Memory Engine (plan 21 — v3.8.6) ───────────────────────── + + /api/memory: + get: + tags: [Memory] + summary: List memory entries + security: + - ManagementSessionAuth: [] + parameters: + - name: apiKeyId + in: query + schema: + type: string + - name: type + in: query + schema: + type: string + enum: [factual, episodic, procedural, semantic] + - name: sessionId + in: query + schema: + type: string + - name: q + in: query + schema: + type: string + - name: limit + in: query + schema: + type: integer + minimum: 1 + maximum: 200 + default: 50 + - name: page + in: query + schema: + type: integer + minimum: 1 + default: 1 + - name: offset + in: query + schema: + type: integer + minimum: 0 + responses: + "200": + description: Paginated list of memories with stats + content: + application/json: + schema: + type: object + properties: + data: + type: array + items: + $ref: "#/components/schemas/MemoryEntry" + total: + type: integer + totalPages: + type: integer + stats: + type: object + properties: + total: + type: integer + tokensUsed: + type: integer + hitRate: + type: number + cacheStats: + type: object + properties: + hits: + type: integer + misses: + type: integer + "401": + $ref: "#/components/responses/Unauthorized" + post: + tags: [Memory] + summary: Create a memory entry + security: + - ManagementSessionAuth: [] + requestBody: + required: true + content: + application/json: + schema: + type: object + required: [content, key] + properties: + content: + type: string + minLength: 1 + key: + type: string + minLength: 1 + type: + type: string + enum: [factual, episodic, procedural, semantic] + default: factual + sessionId: + type: string + nullable: true + apiKeyId: + type: string + metadata: + type: object + additionalProperties: true + expiresAt: + type: string + format: date-time + nullable: true + responses: + "201": + description: Created memory entry + content: + application/json: + schema: + $ref: "#/components/schemas/MemoryEntry" + "400": + description: Validation error + "401": + $ref: "#/components/responses/Unauthorized" + + /api/memory/{id}: + parameters: + - name: id + in: path + required: true + schema: + type: string + description: Memory UUID + get: + tags: [Memory] + summary: Get a single memory entry + security: + - ManagementSessionAuth: [] + responses: + "200": + description: Memory entry + content: + application/json: + schema: + $ref: "#/components/schemas/MemoryEntry" + "401": + $ref: "#/components/responses/Unauthorized" + "404": + description: Memory not found + put: + tags: [Memory] + summary: Update a memory entry + description: >- + Update `type`, `key`, `content`, and/or `metadata` of an existing memory. + If an embedding source is available, the vector in `vec_memories` is also + regenerated. Corresponds to `MemoryUpdatePutSchema`. + security: + - ManagementSessionAuth: [] + requestBody: + required: true + content: + application/json: + schema: + type: object + properties: + type: + type: string + enum: [factual, episodic, procedural, semantic] + key: + type: string + minLength: 1 + content: + type: string + minLength: 1 + metadata: + type: object + additionalProperties: true + additionalProperties: false + responses: + "200": + description: Updated memory entry + content: + application/json: + schema: + $ref: "#/components/schemas/MemoryEntry" + "400": + description: Validation error + "401": + $ref: "#/components/responses/Unauthorized" + "404": + description: Memory not found + delete: + tags: [Memory] + summary: Delete a memory entry + description: >- + Deletes the SQLite row, removes the vector from `vec_memories`, and + best-effort deletes the point from Qdrant. + security: + - ManagementSessionAuth: [] + responses: + "200": + description: Deleted + content: + application/json: + schema: + type: object + properties: + success: + type: boolean + "401": + $ref: "#/components/responses/Unauthorized" + "404": + description: Memory not found + + /api/memory/health: + get: + tags: [Memory] + summary: Memory store health check + description: >- + Round-trip create→list→delete to verify the store is alive. + security: + - ManagementSessionAuth: [] + responses: + "200": + description: Health result + content: + application/json: + schema: + type: object + properties: + working: + type: boolean + latencyMs: + type: number + error: + type: string + nullable: true + "401": + $ref: "#/components/responses/Unauthorized" + + /api/memory/retrieve-preview: + post: + tags: [Memory] + summary: Dry-run memory retrieval (Playground) + description: >- + Simulates `retrieveMemories()` for a given query and returns the ranked + results with score, tier, and token count. Does NOT modify any memory. + Corresponds to `RetrievePreviewSchema`. + security: + - ManagementSessionAuth: [] + requestBody: + required: true + content: + application/json: + schema: + type: object + required: [query] + properties: + query: + type: string + minLength: 1 + strategy: + type: string + enum: [exact, semantic, hybrid] + default: hybrid + maxTokens: + type: integer + minimum: 1 + maximum: 16000 + default: 2000 + apiKeyId: + type: string + description: Optional — tests global pool when omitted + limit: + type: integer + minimum: 1 + maximum: 100 + default: 20 + additionalProperties: false + responses: + "200": + description: Preview results + content: + application/json: + schema: + type: object + properties: + memories: + type: array + items: + type: object + properties: + id: + type: string + type: + type: string + enum: [factual, episodic, procedural, semantic] + key: + type: string + content: + type: string + score: + type: number + tokens: + type: integer + tier: + type: string + enum: [fts5, vector, hybrid-rrf, qdrant] + vecScore: + type: number + nullable: true + ftsScore: + type: number + nullable: true + resolution: + type: object + properties: + embeddingSource: + type: string + enum: [remote, static, transformers] + nullable: true + embeddingModel: + type: string + nullable: true + vectorStore: + type: string + enum: [sqlite-vec, qdrant, none] + strategyUsed: + type: string + enum: [exact, semantic, hybrid] + rerankApplied: + type: boolean + fallbackReason: + type: string + nullable: true + totalTokensUsed: + type: integer + budgetMaxTokens: + type: integer + "400": + description: Validation error + "401": + $ref: "#/components/responses/Unauthorized" + + /api/memory/embedding-providers: + get: + tags: [Memory] + summary: List embedding providers + description: >- + Returns all providers that have embedding-capable models, indicating + which have an active API key configured. + security: + - ManagementSessionAuth: [] + responses: + "200": + description: Provider list + content: + application/json: + schema: + type: object + properties: + providers: + type: array + items: + type: object + properties: + provider: + type: string + hasKey: + type: boolean + models: + type: array + items: + type: object + properties: + id: + type: string + description: "Format: provider/model" + name: + type: string + dimensions: + type: integer + nullable: true + "401": + $ref: "#/components/responses/Unauthorized" + + /api/memory/engine-status: + get: + tags: [Memory] + summary: Memory engine status + description: >- + Returns the full engine status including keyword tier availability, + embedding resolution, vector store statistics (sqlite-vec), Qdrant + health, and rerank configuration. Corresponds to `MemoryEngineStatusSchema`. + security: + - ManagementSessionAuth: [] + responses: + "200": + description: Engine status + content: + application/json: + schema: + type: object + properties: + keyword: + type: object + properties: + available: + type: boolean + backend: + type: string + enum: [FTS5] + embedding: + type: object + properties: + source: + type: string + enum: [remote, static, transformers] + nullable: true + model: + type: string + nullable: true + dimensions: + type: integer + nullable: true + available: + type: boolean + reason: + type: string + cacheStats: + type: object + properties: + hits: + type: integer + misses: + type: integer + size: + type: integer + vectorStore: + type: object + properties: + backend: + type: string + enum: [sqlite-vec, qdrant, none] + available: + type: boolean + rowCount: + type: integer + needsReindex: + type: integer + reason: + type: string + qdrant: + type: object + properties: + enabled: + type: boolean + healthy: + type: boolean + nullable: true + latencyMs: + type: number + nullable: true + error: + type: string + nullable: true + rerank: + type: object + properties: + enabled: + type: boolean + provider: + type: string + nullable: true + model: + type: string + nullable: true + available: + type: boolean + reason: + type: string + "401": + $ref: "#/components/responses/Unauthorized" + + /api/memory/summarize: + post: + tags: [Memory] + summary: Compact old memories + description: >- + Manually triggers memory compaction for memories older than + `olderThanDays`. Use `dryRun: true` to preview candidates. + Corresponds to `MemorySummarizeSchema`. + security: + - ManagementSessionAuth: [] + requestBody: + required: true + content: + application/json: + schema: + type: object + properties: + olderThanDays: + type: integer + minimum: 1 + maximum: 365 + default: 30 + apiKeyId: + type: string + description: Optional — compacts all keys when omitted + dryRun: + type: boolean + default: false + additionalProperties: false + responses: + "200": + description: Summarization result + content: + application/json: + schema: + type: object + properties: + candidates: + type: integer + tokensSaved: + type: integer + dryRun: + type: boolean + "400": + description: Validation error + "401": + $ref: "#/components/responses/Unauthorized" + + /api/memory/reindex: + post: + tags: [Memory] + summary: Trigger vector reindex + description: >- + Starts background reindexing of memories with `needs_reindex = 1`. + Use `force: true` to regenerate ALL vectors regardless of index status. + Corresponds to `MemoryReindexSchema`. + security: + - ManagementSessionAuth: [] + requestBody: + required: true + content: + application/json: + schema: + type: object + properties: + force: + type: boolean + default: false + description: >- + When true, marks all memories needs_reindex=1 before running. + additionalProperties: false + responses: + "200": + description: Reindex started + content: + application/json: + schema: + type: object + properties: + started: + type: boolean + pending: + type: integer + description: Memories still pending after this batch + "400": + description: Validation error + "401": + $ref: "#/components/responses/Unauthorized" + + /api/settings/memory: + get: + tags: [Memory, Settings] + summary: Get memory settings + description: >- + Returns the extended memory settings including 7 new fields added in + plan 21 (embeddingSource, embeddingProviderModel, transformersEnabled, + staticEnabled, rerankEnabled, rerankProviderModel, vectorStore). + security: + - ManagementSessionAuth: [] + responses: + "200": + description: Extended memory settings + content: + application/json: + schema: + $ref: "#/components/schemas/MemorySettingsExtended" + "401": + $ref: "#/components/responses/Unauthorized" + put: + tags: [Memory, Settings] + summary: Update memory settings + description: >- + Update any subset of the extended memory settings. + All fields are optional; only provided fields are updated. + Schema: `MemorySettingsExtendedSchema`. + security: + - ManagementSessionAuth: [] + requestBody: + required: true + content: + application/json: + schema: + $ref: "#/components/schemas/MemorySettingsExtended" + responses: + "200": + description: Updated memory settings + content: + application/json: + schema: + $ref: "#/components/schemas/MemorySettingsExtended" + "400": + description: Validation error + "401": + $ref: "#/components/responses/Unauthorized" + + /api/settings/qdrant: + get: + tags: [Memory, Settings] + summary: Get Qdrant settings + description: >- + Returns current Qdrant configuration. The `apiKey` field is never + returned raw — use `hasApiKey` / `apiKeyMasked` instead. + security: + - ManagementSessionAuth: [] + responses: + "200": + description: Qdrant settings + content: + application/json: + schema: + $ref: "#/components/schemas/QdrantSettings" + "401": + $ref: "#/components/responses/Unauthorized" + put: + tags: [Memory, Settings] + summary: Update Qdrant settings + description: >- + Update Qdrant configuration. Pass `apiKey: ""` to remove the stored + key. Schema: `QdrantSettingsUpdateSchema`. + security: + - ManagementSessionAuth: [] + requestBody: + required: true + content: + application/json: + schema: + type: object + properties: + enabled: + type: boolean + host: + type: string + port: + type: integer + minimum: 1 + maximum: 65535 + collection: + type: string + minLength: 1 + embeddingModel: + type: string + minLength: 1 + apiKey: + type: string + description: Empty string removes the key + additionalProperties: false + responses: + "200": + description: Updated Qdrant settings + content: + application/json: + schema: + $ref: "#/components/schemas/QdrantSettings" + "400": + description: Validation error + "401": + $ref: "#/components/responses/Unauthorized" + + /api/settings/qdrant/health: + get: + tags: [Memory] + summary: Qdrant health probe + description: >- + Performs a liveness check against the configured Qdrant instance. + Returns latency and any connection error (sanitized — no stack traces). + security: + - ManagementSessionAuth: [] + responses: + "200": + description: Health result + content: + application/json: + schema: + $ref: "#/components/schemas/QdrantHealthResult" + "401": + $ref: "#/components/responses/Unauthorized" + + /api/settings/qdrant/search: + post: + tags: [Memory] + summary: Qdrant semantic search test + description: >- + Performs a test semantic search against the Qdrant collection. + Useful for validating that the integration works end-to-end. + Schema: `QdrantSearchSchema`. + security: + - ManagementSessionAuth: [] + requestBody: + required: true + content: + application/json: + schema: + type: object + required: [query] + properties: + query: + type: string + minLength: 1 + topK: + type: integer + minimum: 1 + maximum: 50 + default: 5 + additionalProperties: false + responses: + "200": + description: Search results + content: + application/json: + schema: + type: object + properties: + results: + type: array + items: + type: object + "400": + description: Validation error + "401": + $ref: "#/components/responses/Unauthorized" + "503": + description: Qdrant unavailable (structured error, no stack trace) + + /api/settings/qdrant/cleanup: + post: + tags: [Memory] + summary: Clean up expired Qdrant points + description: >- + Removes Qdrant points for memories that have expired or exceeded + the configured retention window. + security: + - ManagementSessionAuth: [] + responses: + "200": + description: Cleanup result + content: + application/json: + schema: + type: object + properties: + deleted: + type: integer + checked: + type: integer + "401": + $ref: "#/components/responses/Unauthorized" + "503": + description: Qdrant unavailable (structured error, no stack trace) + + /api/settings/qdrant/embedding-models: + get: + tags: [Memory] + summary: List Qdrant embedding models + description: Returns the list of embedding models available for use with Qdrant. + security: + - ManagementSessionAuth: [] + responses: + "200": + description: Embedding models list + content: + application/json: + schema: + type: object + properties: + models: + type: array + items: + type: string + "401": + $ref: "#/components/responses/Unauthorized" + components: securitySchemes: BearerAuth: @@ -3262,3 +4062,127 @@ components: type: integer priority: type: integer + + # ─── Memory Engine schemas (plan 21 — v3.8.6) ───────────────── + + MemoryEntry: + type: object + description: A single persisted memory entry + properties: + id: + type: string + description: UUID + apiKeyId: + type: string + sessionId: + type: string + nullable: true + type: + type: string + enum: [factual, episodic, procedural, semantic] + key: + type: string + description: Stable upsert key (e.g. preference:i_prefer_python) + content: + type: string + metadata: + type: object + additionalProperties: true + createdAt: + type: string + format: date-time + updatedAt: + type: string + format: date-time + expiresAt: + type: string + format: date-time + nullable: true + needsReindex: + type: integer + description: 1 if the vector for this memory is stale or missing + + MemorySettingsExtended: + type: object + description: >- + Extended memory settings including 7 new fields from plan 21. + All fields are optional for PUT (patch semantics). + properties: + enabled: + type: boolean + maxTokens: + type: integer + minimum: 0 + maximum: 16000 + retentionDays: + type: integer + minimum: 1 + maximum: 365 + strategy: + type: string + enum: [recent, semantic, hybrid] + skillsEnabled: + type: boolean + embeddingSource: + type: string + enum: [remote, static, transformers, auto] + description: >- + Which embedding source to use. "auto" = remote > static > transformers. + embeddingProviderModel: + type: string + nullable: true + description: >- + Embedding provider/model in "provider/model" format (e.g. openai/text-embedding-3-small). + transformersEnabled: + type: boolean + description: Opt-in for Transformers.js local MiniLM model (~400MB RAM) + staticEnabled: + type: boolean + description: Opt-in for static potion-base-8M local model + rerankEnabled: + type: boolean + description: Enable reranking step (+200-500ms/req) + rerankProviderModel: + type: string + nullable: true + description: Rerank provider/model in "provider/model" format + vectorStore: + type: string + enum: [sqlite-vec, qdrant, auto] + description: Which vector backend to use + + QdrantSettings: + type: object + description: Qdrant vector database configuration (read shape — no raw apiKey) + properties: + enabled: + type: boolean + host: + type: string + port: + type: integer + minimum: 1 + maximum: 65535 + collection: + type: string + embeddingModel: + type: string + hasApiKey: + type: boolean + apiKeyMasked: + type: string + nullable: true + description: First 4 chars of the configured API key, or null + + QdrantHealthResult: + type: object + description: Result of a Qdrant liveness probe + properties: + ok: + type: boolean + latencyMs: + type: number + error: + type: string + nullable: true + description: Sanitized error message (no stack traces) diff --git a/src/i18n/messages/en.json b/src/i18n/messages/en.json index 5180d0ca69..5b6b6c2d7b 100644 --- a/src/i18n/messages/en.json +++ b/src/i18n/messages/en.json @@ -2978,84 +2978,91 @@ "a2aStep3": "Track and cancel tasks with {code1} and {code2}." }, "memory": { - "title": "Memory Management", - "description": "View and manage stored memory entries", - "memories": "Memories", - "totalEntries": "Total Entries", - "tokensUsed": "Tokens Used", - "hitRate": "Hit Rate", - "loading": "Loading...", - "noMemories": "No memories found", - "search": "Search memories...", - "allTypes": "All Types", - "export": "Export", - "import": "Import", - "addMemory": "Add Memory", - "type": "Type", - "key": "Key", - "content": "Content", - "created": "Created", - "actions": "Actions", - "delete": "Delete", - "factual": "Factual", - "episodic": "Episodic", - "procedural": "Procedural", - "semantic": "Semantic", "a": "A", - "pipelineOk": "Pipeline OK ({latencyMs}ms)", - "pipelineError": "Pipeline error", - "healthUnknown": "Health unknown", - "checkingHealth": "Checking…", - "checkHealth": "Check health", - "pageInfo": "Page {page} of {totalPages} ({total} total)", - "previous": "Previous", - "next": "Next", + "actions": "Actions", + "addMemory": "Add Memory", + "allTypes": "All Types", "cancel": "Cancel", - "save": "Save", - "keyPlaceholder": "e.g. user.preferences.theme", - "contentPlaceholder": "Value or JSON content to remember", - "tabs": { - "memories": "Memories", - "playground": "Playground", - "engine": "Engine" - }, + "checkHealth": "Check health", + "checkingHealth": "Checking…", + "compactOld": "Compact old", "concept": { "title": "Conversational Memory", "description": "OmniRoute learns from every conversation, remembering facts, episodes, procedures, and semantic concepts that make responses more accurate and context-aware.", "howWorksToggle": "How it works", "howWorksContent": "1. Automatic extraction: at the end of each response, facts and episodes are detected and saved automatically.\n2. Retrieval: before each response, the most relevant memories are searched via FTS5 (exact), vector (semantic), or hybrid RRF.\n3. Injection: relevant memories are injected into the assistant context to improve response quality.\n4. Management: use this page to view, edit, export, and compact old memories." }, - "tooltip": { - "totalEntries": "Total memories stored for this API key", - "tokensUsed": "Estimated total tokens occupied by active memories", - "hitRate": "Read-by-ID cache hit rate (not semantic recall accuracy)", - "factual": "Objective, permanent facts from user context", - "episodic": "Events and experiences from conversation history", - "procedural": "Procedures, workflows, and instructions the assistant should follow", - "semantic": "Concepts, preferences, and domain knowledge" - }, - "emptyState": { - "title": "No memories yet", - "description": "Memories are created automatically from conversations. You can also add them manually using the button above." - }, + "content": "Content", + "contentPlaceholder": "Value or JSON content to remember", + "created": "Created", + "delete": "Delete", + "deleteConfirmDesc": "This action cannot be undone. The memory will be permanently removed.", + "deleteConfirmTitle": "Delete memory?", + "description": "View and manage stored memory entries", + "editMemory": "Edit memory", "editModal": { "title": "Edit Memory", "metadataLabel": "Metadata (JSON)", "metadataInvalid": "Invalid JSON", "saveFailed": "Failed to save memory" }, - "editMemory": "Edit memory", - "deleteConfirmTitle": "Delete memory?", - "deleteConfirmDesc": "This action cannot be undone. The memory will be permanently removed.", - "importResult": "{imported} imported, {skipped} skipped", - "importError": "Failed to import file", - "compactOld": "Compact old", - "summarize": { - "title": "Compact old memories", - "noCandidates": "No memories eligible for compaction (criteria: >30 days old).", - "candidatesDesc": "{count} memories will be compacted into summaries. This action cannot be undone.", - "confirm": "Compact now" + "embedding": { + "autoLabel": "Automatic", + "autoDesc": "Uses best available: remote provider > static > transformers", + "remoteLabel": "Remote provider", + "remoteDesc": "Uses embedding via provider API (requires API key)", + "staticLabel": "Static local (potion)", + "staticDesc": "Local embedding without WASM or external dependencies", + "transformersLabel": "Transformers.js (MiniLM)", + "transformersDesc": "Local embedding via @huggingface/transformers (~400MB RAM)", + "providerModelLabel": "Provider / Model", + "noRemoteProviders": "No providers with configured API key", + "selectProviderModel": "Select a model", + "staticEnabledLabel": "Enable Static Potion", + "staticEnabledDesc": "Download and use potion-base-8M model locally", + "transformersEnabledLabel": "Enable Transformers.js", + "transformersEnabledDesc": "Opt-in for local MiniLM (~400MB RAM, ~3s cold start)", + "transformersWarning": "Requires ~400MB RAM and ~3s cold start on the first semantic query." }, + "emptyState": { + "title": "No memories yet", + "description": "Memories are created automatically from conversations. You can also add them manually using the button above." + }, + "engine": { + "statusTitle": "Engine Status", + "embeddingTitle": "Embedding Source", + "rerankTitle": "Rerank (optional)", + "reindexNow": "Reindex now", + "reindexStarted": "Reindexing started ({pending} pending)", + "reindexFailed": "Failed to start reindexing", + "keywordLabel": "Keyword (FTS5)", + "keywordReason": "Keyword search always available", + "embeddingLabel": "Embedding", + "vectorStoreLabel": "Vector Store", + "qdrantLabel": "Qdrant", + "rerankLabel": "Rerank", + "qdrantDisabled": "Disabled", + "qdrantOk": "Healthy ({latencyMs}ms)", + "qdrantError": "Connection error", + "needsReindex": "{count} memory(ies) need reindexing" + }, + "episodic": "Episodic", + "export": "Export", + "factual": "Factual", + "healthUnknown": "Health unknown", + "hitRate": "Hit Rate", + "import": "Import", + "importError": "Failed to import file", + "importResult": "{imported} imported, {skipped} skipped", + "key": "Key", + "keyPlaceholder": "e.g. user.preferences.theme", + "loading": "Loading...", + "memories": "Memories", + "next": "Next", + "noMemories": "No memories found", + "pageInfo": "Page {page} of {totalPages} ({total} total)", + "pipelineError": "Pipeline error", + "pipelineOk": "Pipeline OK ({latencyMs}ms)", "playground": { "infoTitle": "Memory Playground", "infoDesc": "Simulate what would be retrieved for a given query. No memories are modified — read-only preview.", @@ -3079,42 +3086,8 @@ "none": "none", "errorFetch": "Failed to fetch preview" }, - "engine": { - "statusTitle": "Engine Status", - "embeddingTitle": "Embedding Source", - "rerankTitle": "Rerank (optional)", - "reindexNow": "Reindex now", - "reindexStarted": "Reindexing started ({pending} pending)", - "reindexFailed": "Failed to start reindexing", - "keywordLabel": "Keyword (FTS5)", - "keywordReason": "Keyword search always available", - "embeddingLabel": "Embedding", - "vectorStoreLabel": "Vector Store", - "qdrantLabel": "Qdrant", - "rerankLabel": "Rerank", - "qdrantDisabled": "Disabled", - "qdrantOk": "Healthy ({latencyMs}ms)", - "qdrantError": "Connection error", - "needsReindex": "{count} memory(ies) need reindexing" - }, - "embedding": { - "autoLabel": "Automatic", - "autoDesc": "Uses best available: remote provider > static > transformers", - "remoteLabel": "Remote provider", - "remoteDesc": "Uses embedding via provider API (requires API key)", - "staticLabel": "Static local (potion)", - "staticDesc": "Local embedding without WASM or external dependencies", - "transformersLabel": "Transformers.js (MiniLM)", - "transformersDesc": "Local embedding via @huggingface/transformers (~400MB RAM)", - "providerModelLabel": "Provider / Model", - "noRemoteProviders": "No providers with configured API key", - "selectProviderModel": "Select a model", - "staticEnabledLabel": "Enable Static Potion", - "staticEnabledDesc": "Download and use potion-base-8M model locally", - "transformersEnabledLabel": "Enable Transformers.js", - "transformersEnabledDesc": "Opt-in for local MiniLM (~400MB RAM, ~3s cold start)", - "transformersWarning": "Requires ~400MB RAM and ~3s cold start on the first semantic query." - }, + "previous": "Previous", + "procedural": "Procedural", "qdrant": { "title": "Qdrant (Vector Store Tier 2)", "description": "Optional Qdrant integration for scalable semantic search", @@ -3155,7 +3128,34 @@ "noProviderWithKey": "No provider with configured API key. Configure a provider to use rerank.", "selectProviderModel": "Select a provider/model" }, - "saving": "Saving..." + "save": "Save", + "saving": "Saving...", + "search": "Search memories...", + "semantic": "Semantic", + "summarize": { + "title": "Compact old memories", + "noCandidates": "No memories eligible for compaction (criteria: >30 days old).", + "candidatesDesc": "{count} memories will be compacted into summaries. This action cannot be undone.", + "confirm": "Compact now" + }, + "tabs": { + "memories": "Memories", + "playground": "Playground", + "engine": "Engine" + }, + "title": "Memory Management", + "tokensUsed": "Tokens Used", + "tooltip": { + "totalEntries": "Total memories stored for this API key", + "tokensUsed": "Estimated total tokens occupied by active memories", + "hitRate": "Read-by-ID cache hit rate (not semantic recall accuracy)", + "factual": "Objective, permanent facts from user context", + "episodic": "Events and experiences from conversation history", + "procedural": "Procedures, workflows, and instructions the assistant should follow", + "semantic": "Concepts, preferences, and domain knowledge" + }, + "totalEntries": "Total Entries", + "type": "Type" }, "skills": { "title": "Skills", @@ -7432,4 +7432,4 @@ "resetIn": "reset in", "quotaTotal": "total" } -} \ No newline at end of file +} diff --git a/src/i18n/messages/pt-BR.json b/src/i18n/messages/pt-BR.json index 5173930db7..b0400b230e 100644 --- a/src/i18n/messages/pt-BR.json +++ b/src/i18n/messages/pt-BR.json @@ -2975,84 +2975,91 @@ "a2aStep3": "Rastreie e cancele tarefas com {code1} and {code2}." }, "memory": { - "title": "Gerenciamento de Memória", - "description": "Visualize e gerencie entradas de memória armazenadas", - "memories": "Memórias", - "totalEntries": "Total de Entradas", - "tokensUsed": "Tokens Usados", - "hitRate": "Taxa de Acerto", - "loading": "Carregando...", - "noMemories": "Nenhuma memória encontrada", - "search": "Buscar memórias...", - "allTypes": "Todos os Tipos", - "export": "Exportar", - "import": "Importar", - "addMemory": "Adicionar Memória", - "type": "Tipo", - "key": "Chave", - "content": "Conteúdo", - "created": "Criado", - "actions": "Ações", - "delete": "Delete", - "factual": "Factual", - "episodic": "Episódica", - "procedural": "Procedural", - "semantic": "Semântica", "a": "A", - "pipelineOk": "Pipeline OK ({latencyMs}ms)", - "pipelineError": "Erro de pipeline", - "healthUnknown": "Saúde desconhecida", - "checkingHealth": "Verificando…", - "checkHealth": "Verifique a saúde", - "pageInfo": "Página {page} de {totalPages} ({total} total)", - "previous": "Anterior", - "next": "Próximo", + "actions": "Ações", + "addMemory": "Adicionar Memória", + "allTypes": "Todos os Tipos", "cancel": "Cancelar", - "save": "Salvar", - "keyPlaceholder": "por exemplo usuário.preferências.tema", - "contentPlaceholder": "Valor ou conteúdo JSON para lembrar", - "tabs": { - "memories": "Memórias", - "playground": "Playground", - "engine": "Engine" - }, + "checkHealth": "Verifique a saúde", + "checkingHealth": "Verificando…", + "compactOld": "Compactar antigas", "concept": { "title": "Memória Conversacional", "description": "O OmniRoute aprende com cada conversa, lembrando de fatos, episódios, procedimentos e conceitos semânticos que tornam as respostas mais precisas e contextualizadas.", "howWorksToggle": "Como funciona", "howWorksContent": "1. Extração automática: ao final de cada resposta, fatos e episódios são detectados e salvos automaticamente.\n2. Recuperação: antes de cada resposta, as memórias mais relevantes são buscadas por FTS5 (exato), vetor (semântico) ou RRF híbrido.\n3. Injeção: as memórias relevantes são injetadas no contexto do assistente para melhorar a qualidade da resposta.\n4. Gestão: use esta página para visualizar, editar, exportar e compactar memórias antigas." }, - "tooltip": { - "totalEntries": "Total de memórias armazenadas para esta chave de API", - "tokensUsed": "Total de tokens estimados ocupados pelas memórias ativas", - "hitRate": "Taxa de cache de leitura por ID (não é precisão semântica)", - "factual": "Fatos objetivos e permanentes do contexto do usuário", - "episodic": "Eventos e experiências do histórico da conversa", - "procedural": "Procedimentos, fluxos e instruções que o assistente deve seguir", - "semantic": "Conceitos, preferências e conhecimento de domínio" - }, - "emptyState": { - "title": "Nenhuma memória ainda", - "description": "As memórias são criadas automaticamente das conversas. Você também pode adicionar manualmente usando o botão acima." - }, + "content": "Conteúdo", + "contentPlaceholder": "Valor ou conteúdo JSON para lembrar", + "created": "Criado", + "delete": "Delete", + "deleteConfirmDesc": "Esta ação não pode ser desfeita. A memória será removida permanentemente.", + "deleteConfirmTitle": "Excluir memória?", + "description": "Visualize e gerencie entradas de memória armazenadas", + "editMemory": "Editar memória", "editModal": { "title": "Editar Memória", "metadataLabel": "Metadados (JSON)", "metadataInvalid": "JSON inválido", "saveFailed": "Falha ao salvar memória" }, - "editMemory": "Editar memória", - "deleteConfirmTitle": "Excluir memória?", - "deleteConfirmDesc": "Esta ação não pode ser desfeita. A memória será removida permanentemente.", - "importResult": "{imported} importadas, {skipped} ignoradas", - "importError": "Erro ao importar arquivo", - "compactOld": "Compactar antigas", - "summarize": { - "title": "Compactar memórias antigas", - "noCandidates": "Nenhuma memória elegível para compactação (critério: >30 dias).", - "candidatesDesc": "{count} memórias serão compactadas em summaries. Esta ação não pode ser desfeita.", - "confirm": "Compactar agora" + "embedding": { + "autoLabel": "Automático", + "autoDesc": "Usa o melhor disponível: provider remoto > static > transformers", + "remoteLabel": "Provider remoto", + "remoteDesc": "Usa embedding via API de um provider configurado (requer chave)", + "staticLabel": "Static local (potion)", + "staticDesc": "Embedding local sem WASM, sem dependências externas", + "transformersLabel": "Transformers.js (MiniLM)", + "transformersDesc": "Embedding local via @huggingface/transformers (~400MB RAM)", + "providerModelLabel": "Provider / Modelo", + "noRemoteProviders": "Nenhum provider com chave configurada", + "selectProviderModel": "Selecione um modelo", + "staticEnabledLabel": "Habilitar Static Potion", + "staticEnabledDesc": "Baixa e usa o modelo potion-base-8M localmente", + "transformersEnabledLabel": "Habilitar Transformers.js", + "transformersEnabledDesc": "Opt-in para MiniLM local (~400MB RAM, cold start ~3s)", + "transformersWarning": "Requer ~400MB de RAM e ~3s de cold start na primeira query semântica." }, + "emptyState": { + "title": "Nenhuma memória ainda", + "description": "As memórias são criadas automaticamente das conversas. Você também pode adicionar manualmente usando o botão acima." + }, + "engine": { + "statusTitle": "Status do Engine", + "embeddingTitle": "Fonte de Embedding", + "rerankTitle": "Rerank (opcional)", + "reindexNow": "Reindexar agora", + "reindexStarted": "Reindexação iniciada ({pending} pendentes)", + "reindexFailed": "Falha ao iniciar reindexação", + "keywordLabel": "Keyword (FTS5)", + "keywordReason": "Busca por palavras-chave sempre disponível", + "embeddingLabel": "Embedding", + "vectorStoreLabel": "Vector Store", + "qdrantLabel": "Qdrant", + "rerankLabel": "Rerank", + "qdrantDisabled": "Desabilitado", + "qdrantOk": "Saudável ({latencyMs}ms)", + "qdrantError": "Erro de conexão", + "needsReindex": "{count} memória(s) precisam de reindexação" + }, + "episodic": "Episódica", + "export": "Exportar", + "factual": "Factual", + "healthUnknown": "Saúde desconhecida", + "hitRate": "Taxa de Acerto", + "import": "Importar", + "importError": "Erro ao importar arquivo", + "importResult": "{imported} importadas, {skipped} ignoradas", + "key": "Chave", + "keyPlaceholder": "por exemplo usuário.preferências.tema", + "loading": "Carregando...", + "memories": "Memórias", + "next": "Próximo", + "noMemories": "Nenhuma memória encontrada", + "pageInfo": "Página {page} de {totalPages} ({total} total)", + "pipelineError": "Erro de pipeline", + "pipelineOk": "Pipeline OK ({latencyMs}ms)", "playground": { "infoTitle": "Memory Playground", "infoDesc": "Simule o que seria recuperado para uma determinada query. Nenhuma memória é modificada — apenas visualização.", @@ -3076,42 +3083,8 @@ "none": "nenhum", "errorFetch": "Falha ao buscar preview" }, - "engine": { - "statusTitle": "Status do Engine", - "embeddingTitle": "Fonte de Embedding", - "rerankTitle": "Rerank (opcional)", - "reindexNow": "Reindexar agora", - "reindexStarted": "Reindexação iniciada ({pending} pendentes)", - "reindexFailed": "Falha ao iniciar reindexação", - "keywordLabel": "Keyword (FTS5)", - "keywordReason": "Busca por palavras-chave sempre disponível", - "embeddingLabel": "Embedding", - "vectorStoreLabel": "Vector Store", - "qdrantLabel": "Qdrant", - "rerankLabel": "Rerank", - "qdrantDisabled": "Desabilitado", - "qdrantOk": "Saudável ({latencyMs}ms)", - "qdrantError": "Erro de conexão", - "needsReindex": "{count} memória(s) precisam de reindexação" - }, - "embedding": { - "autoLabel": "Automático", - "autoDesc": "Usa o melhor disponível: provider remoto > static > transformers", - "remoteLabel": "Provider remoto", - "remoteDesc": "Usa embedding via API de um provider configurado (requer chave)", - "staticLabel": "Static local (potion)", - "staticDesc": "Embedding local sem WASM, sem dependências externas", - "transformersLabel": "Transformers.js (MiniLM)", - "transformersDesc": "Embedding local via @huggingface/transformers (~400MB RAM)", - "providerModelLabel": "Provider / Modelo", - "noRemoteProviders": "Nenhum provider com chave configurada", - "selectProviderModel": "Selecione um modelo", - "staticEnabledLabel": "Habilitar Static Potion", - "staticEnabledDesc": "Baixa e usa o modelo potion-base-8M localmente", - "transformersEnabledLabel": "Habilitar Transformers.js", - "transformersEnabledDesc": "Opt-in para MiniLM local (~400MB RAM, cold start ~3s)", - "transformersWarning": "Requer ~400MB de RAM e ~3s de cold start na primeira query semântica." - }, + "previous": "Anterior", + "procedural": "Procedural", "qdrant": { "title": "Qdrant (Vector Store Tier 2)", "description": "Integração opcional com Qdrant para busca semântica escalável", @@ -3152,7 +3125,34 @@ "noProviderWithKey": "Nenhum provider com chave configurada. Configure um provider para usar rerank.", "selectProviderModel": "Selecione um provider/modelo" }, - "saving": "Salvando..." + "save": "Salvar", + "saving": "Salvando...", + "search": "Buscar memórias...", + "semantic": "Semântica", + "summarize": { + "title": "Compactar memórias antigas", + "noCandidates": "Nenhuma memória elegível para compactação (critério: >30 dias).", + "candidatesDesc": "{count} memórias serão compactadas em summaries. Esta ação não pode ser desfeita.", + "confirm": "Compactar agora" + }, + "tabs": { + "memories": "Memórias", + "playground": "Playground", + "engine": "Engine" + }, + "title": "Gerenciamento de Memória", + "tokensUsed": "Tokens Usados", + "tooltip": { + "totalEntries": "Total de memórias armazenadas para esta chave de API", + "tokensUsed": "Total de tokens estimados ocupados pelas memórias ativas", + "hitRate": "Taxa de cache de leitura por ID (não é precisão semântica)", + "factual": "Fatos objetivos e permanentes do contexto do usuário", + "episodic": "Eventos e experiências do histórico da conversa", + "procedural": "Procedimentos, fluxos e instruções que o assistente deve seguir", + "semantic": "Conceitos, preferências e conhecimento de domínio" + }, + "totalEntries": "Total de Entradas", + "type": "Tipo" }, "skills": { "title": "Skills", @@ -7422,4 +7422,4 @@ "resetIn": "redefinir em", "quotaTotal": "total" } -} \ No newline at end of file +} diff --git a/tests/e2e/memory-engine.spec.ts b/tests/e2e/memory-engine.spec.ts new file mode 100644 index 0000000000..a0b2669a7a --- /dev/null +++ b/tests/e2e/memory-engine.spec.ts @@ -0,0 +1,607 @@ +import { expect, test, type Page, type Route } from "@playwright/test"; +import { gotoDashboardRoute } from "./helpers/dashboardAuth"; + +const NAVIGATION_TIMEOUT_MS = 300_000; + +// --------------------------------------------------------------------------- +// Shared types +// --------------------------------------------------------------------------- + +type MemoryEntry = { + id: string; + apiKeyId: string; + sessionId: string | null; + type: "factual" | "episodic" | "procedural" | "semantic"; + key: string; + content: string; + metadata: Record; + createdAt: string; + updatedAt: string; + expiresAt: string | null; +}; + +type MemoryStats = { + totalEntries: number; + tokensUsed: number; + hitRate: number; + cacheStats: { hits: number; misses: number }; +}; + +type MemorySettings = { + enabled: boolean; + maxTokens: number; + retentionDays: number; + strategy: "recent" | "semantic" | "hybrid"; + skillsEnabled: boolean; + embeddingSource: "remote" | "static" | "transformers" | "auto"; + embeddingProviderModel: string | null; + transformersEnabled: boolean; + staticEnabled: boolean; + rerankEnabled: boolean; + rerankProviderModel: string | null; + vectorStore: "sqlite-vec" | "qdrant" | "auto"; +}; + +type EngineStatus = { + keyword: { available: true; backend: "FTS5" }; + embedding: { + source: "remote" | "static" | "transformers" | null; + model: string | null; + dimensions: number | null; + available: boolean; + reason: string; + cacheStats: { hits: number; misses: number; size: number }; + }; + vectorStore: { + backend: "sqlite-vec" | "qdrant" | "none"; + available: boolean; + rowCount: number; + needsReindex: number; + reason: string; + }; + qdrant: { + enabled: boolean; + healthy: boolean | null; + latencyMs: number | null; + error: string | null; + }; + rerank: { + enabled: boolean; + provider: string | null; + model: string | null; + available: boolean; + reason: string; + }; +}; + +// --------------------------------------------------------------------------- +// Fixtures helpers +// --------------------------------------------------------------------------- + +async function fulfillJson(route: Route, body: unknown, status = 200) { + await route.fulfill({ + status, + contentType: "application/json", + body: JSON.stringify(body), + }); +} + +function makeMemory(overrides: Partial = {}): MemoryEntry { + return { + id: "mem-test-1", + apiKeyId: "key-1", + sessionId: null, + type: "factual", + key: "test.preference.language", + content: "The user prefers English responses.", + metadata: {}, + createdAt: new Date("2026-05-01T10:00:00.000Z").toISOString(), + updatedAt: new Date("2026-05-01T10:00:00.000Z").toISOString(), + expiresAt: null, + ...overrides, + }; +} + +function defaultSettings(): MemorySettings { + return { + enabled: true, + maxTokens: 2000, + retentionDays: 30, + strategy: "hybrid", + skillsEnabled: false, + embeddingSource: "auto", + embeddingProviderModel: null, + transformersEnabled: false, + staticEnabled: false, + rerankEnabled: false, + rerankProviderModel: null, + vectorStore: "auto", + }; +} + +function defaultEngineStatus(): EngineStatus { + return { + keyword: { available: true, backend: "FTS5" }, + embedding: { + source: null, + model: null, + dimensions: null, + available: false, + reason: "No embedding source configured", + cacheStats: { hits: 0, misses: 0, size: 0 }, + }, + vectorStore: { + backend: "none", + available: false, + rowCount: 0, + needsReindex: 0, + reason: "sqlite-vec unavailable in this environment", + }, + qdrant: { enabled: false, healthy: null, latencyMs: null, error: null }, + rerank: { + enabled: false, + provider: null, + model: null, + available: false, + reason: "Rerank disabled", + }, + }; +} + +// --------------------------------------------------------------------------- +// Route interceptors +// --------------------------------------------------------------------------- + +async function setupMemoryRoutes( + page: Page, + state: { + memories: MemoryEntry[]; + stats: MemoryStats; + settings: MemorySettings; + engineStatus: EngineStatus; + createCalls: number; + updateCalls: number; + deleteCalls: number; + settingsCalls: number; + reindexCalls: number; + previewCalls: number; + }, +) { + // GET/POST /api/memory + await page.route(/\/api\/memory(\?.*)?$/, async (route) => { + const method = route.request().method(); + + if (method === "GET") { + await fulfillJson(route, { + data: state.memories, + total: state.memories.length, + totalPages: 1, + stats: { + total: state.memories.length, + tokensUsed: state.stats.tokensUsed, + hitRate: state.stats.hitRate, + cacheStats: state.stats.cacheStats, + }, + }); + return; + } + + if (method === "POST") { + state.createCalls += 1; + const body = route.request().postDataJSON() as Partial; + const newMemory = makeMemory({ + id: `mem-new-${state.createCalls}`, + key: body.key ?? "new.key", + content: body.content ?? "New memory content.", + type: (body.type as MemoryEntry["type"]) ?? "factual", + sessionId: body.sessionId ?? null, + apiKeyId: body.apiKeyId ?? "key-1", + metadata: body.metadata ?? {}, + createdAt: new Date().toISOString(), + updatedAt: new Date().toISOString(), + }); + state.memories = [...state.memories, newMemory]; + state.stats.totalEntries = state.memories.length; + await fulfillJson(route, newMemory, 201); + return; + } + + await fulfillJson(route, { error: { message: "Method not allowed" } }, 405); + }); + + // GET/PUT /api/memory/[id] (must be after /api/memory$ route) + await page.route(/\/api\/memory\/[^/]+$/, async (route) => { + const method = route.request().method(); + const memoryId = route.request().url().split("/").pop()?.split("?")[0] ?? ""; + + if (method === "GET") { + const mem = state.memories.find((m) => m.id === memoryId); + if (!mem) { + await fulfillJson(route, { error: { message: "Not found" } }, 404); + return; + } + await fulfillJson(route, mem); + return; + } + + if (method === "PUT") { + state.updateCalls += 1; + const body = route.request().postDataJSON() as Partial; + state.memories = state.memories.map((m) => { + if (m.id !== memoryId) return m; + return { + ...m, + ...body, + updatedAt: new Date().toISOString(), + }; + }); + const updated = state.memories.find((m) => m.id === memoryId); + await fulfillJson(route, updated ?? { error: { message: "Not found" } }, updated ? 200 : 404); + return; + } + + if (method === "DELETE") { + state.deleteCalls += 1; + state.memories = state.memories.filter((m) => m.id !== memoryId); + state.stats.totalEntries = state.memories.length; + await fulfillJson(route, { success: true }); + return; + } + + await fulfillJson(route, { error: { message: "Method not allowed" } }, 405); + }); + + // GET /api/memory/engine-status + await page.route(/\/api\/memory\/engine-status$/, async (route) => { + await fulfillJson(route, state.engineStatus); + }); + + // GET /api/memory/embedding-providers + await page.route(/\/api\/memory\/embedding-providers$/, async (route) => { + await fulfillJson(route, { providers: [] }); + }); + + // POST /api/memory/retrieve-preview + await page.route(/\/api\/memory\/retrieve-preview$/, async (route) => { + state.previewCalls += 1; + await fulfillJson(route, { + memories: state.memories.slice(0, 3).map((m) => ({ + id: m.id, + type: m.type, + key: m.key, + content: m.content, + score: 0.9, + tokens: 24, + tier: "fts5", + vecScore: null, + ftsScore: 0.9, + })), + resolution: { + embeddingSource: null, + embeddingModel: null, + vectorStore: "none", + strategyUsed: "exact", + rerankApplied: false, + fallbackReason: "No embedding source available, fell back to FTS5.", + }, + totalTokensUsed: state.memories.length * 24, + budgetMaxTokens: 2000, + }); + }); + + // POST /api/memory/reindex + await page.route(/\/api\/memory\/reindex$/, async (route) => { + state.reindexCalls += 1; + await fulfillJson(route, { started: true, pending: 0, queued: 0 }); + }); + + // GET/PUT /api/settings/memory + await page.route(/\/api\/settings\/memory$/, async (route) => { + const method = route.request().method(); + if (method === "GET") { + await fulfillJson(route, state.settings); + return; + } + if (method === "PUT") { + state.settingsCalls += 1; + const body = route.request().postDataJSON() as Partial; + state.settings = { ...state.settings, ...body }; + await fulfillJson(route, state.settings); + return; + } + await fulfillJson(route, { error: { message: "Method not allowed" } }, 405); + }); + + // GET/PUT /api/settings/qdrant (Engine tab, QdrantConfigCard) + await page.route(/\/api\/settings\/qdrant$/, async (route) => { + const method = route.request().method(); + if (method === "GET") { + await fulfillJson(route, { + enabled: false, + host: "", + port: 6333, + collection: "omniroute_memory", + embeddingModel: "openai/text-embedding-3-small", + hasApiKey: false, + apiKeyMasked: null, + }); + return; + } + if (method === "PUT") { + await fulfillJson(route, { enabled: false, host: "", port: 6333 }); + return; + } + await fulfillJson(route, { error: { message: "Method not allowed" } }, 405); + }); + + // GET /api/settings/qdrant/health + await page.route(/\/api\/settings\/qdrant\/health$/, async (route) => { + await fulfillJson(route, { ok: false, latencyMs: 0, error: "Connection refused" }); + }); + + // GET /api/settings/qdrant/embedding-models + await page.route(/\/api\/settings\/qdrant\/embedding-models$/, async (route) => { + await fulfillJson(route, { models: [] }); + }); + + // GET /api/memory/health + await page.route(/\/api\/memory\/health$/, async (route) => { + await fulfillJson(route, { working: true, latencyMs: 5 }); + }); +} + +// --------------------------------------------------------------------------- +// Test suite +// --------------------------------------------------------------------------- + +test.describe("Memory Engine Studio — /dashboard/memory", () => { + test.setTimeout(600_000); + + test("1. /dashboard/memory renders 3 tabs and concept card", async ({ page }) => { + const state = { + memories: [makeMemory()], + stats: { totalEntries: 1, tokensUsed: 24, hitRate: 0.75, cacheStats: { hits: 3, misses: 1 } }, + settings: defaultSettings(), + engineStatus: defaultEngineStatus(), + createCalls: 0, + updateCalls: 0, + deleteCalls: 0, + settingsCalls: 0, + reindexCalls: 0, + previewCalls: 0, + }; + + await setupMemoryRoutes(page, state); + + await gotoDashboardRoute(page, "/dashboard/memory", { timeoutMs: NAVIGATION_TIMEOUT_MS }); + + // All 3 tabs should be visible + await expect(page.getByTestId("tab-memories")).toBeVisible({ timeout: 30_000 }); + await expect(page.getByTestId("tab-playground")).toBeVisible({ timeout: 10_000 }); + await expect(page.getByTestId("tab-engine")).toBeVisible({ timeout: 10_000 }); + }); + + test("2. Memories tab renders table and Total card", async ({ page }) => { + const state = { + memories: [makeMemory()], + stats: { totalEntries: 1, tokensUsed: 48, hitRate: 0.5, cacheStats: { hits: 1, misses: 1 } }, + settings: defaultSettings(), + engineStatus: defaultEngineStatus(), + createCalls: 0, + updateCalls: 0, + deleteCalls: 0, + settingsCalls: 0, + reindexCalls: 0, + previewCalls: 0, + }; + + await setupMemoryRoutes(page, state); + + await gotoDashboardRoute(page, "/dashboard/memory", { timeoutMs: NAVIGATION_TIMEOUT_MS }); + + // Wait for memories tab (active by default) + await expect(page.getByTestId("tab-memories")).toBeVisible({ timeout: 30_000 }); + + // The memory key should appear in the table + await expect(async () => { + await expect(page.getByText("test.preference.language")).toBeVisible({ timeout: 10_000 }); + }).toPass({ timeout: 30_000, intervals: [1000, 2000] }); + }); + + test("3. Add Memory modal → entry appears in table", async ({ page }) => { + const state = { + memories: [] as MemoryEntry[], + stats: { totalEntries: 0, tokensUsed: 0, hitRate: 0, cacheStats: { hits: 0, misses: 0 } }, + settings: defaultSettings(), + engineStatus: defaultEngineStatus(), + createCalls: 0, + updateCalls: 0, + deleteCalls: 0, + settingsCalls: 0, + reindexCalls: 0, + previewCalls: 0, + }; + + await setupMemoryRoutes(page, state); + + await gotoDashboardRoute(page, "/dashboard/memory", { timeoutMs: NAVIGATION_TIMEOUT_MS }); + + // Wait for the page to be ready (empty state is shown initially) + await expect(page.getByTestId("tab-memories")).toBeVisible({ timeout: 30_000 }); + + // Click Add Memory + await expect(async () => { + const addBtn = page.getByRole("button", { name: /add memory/i }).first(); + await expect(addBtn).toBeVisible({ timeout: 10_000 }); + await addBtn.click(); + }).toPass({ timeout: 30_000, intervals: [1000, 2000] }); + + // Fill in the add-memory form fields + const keyInput = page.getByPlaceholder(/e\.g.*preferences/i).or(page.getByLabel(/key/i)).first(); + await expect(keyInput).toBeVisible({ timeout: 10_000 }); + await keyInput.fill("test.new.memory"); + + const contentInput = page + .getByPlaceholder(/content|value/i) + .or(page.getByLabel(/content/i)) + .first(); + await expect(contentInput).toBeVisible({ timeout: 5_000 }); + await contentInput.fill("New memory added via modal."); + + // Submit the form + const saveBtn = page.getByRole("button", { name: /save|add|create/i }).last(); + await expect(saveBtn).toBeVisible({ timeout: 5_000 }); + await saveBtn.click(); + + // createCalls should increment + await expect.poll(() => state.createCalls).toBeGreaterThanOrEqual(1); + }); + + test("4. Edit memory — pencil → modal → save → change reflected", async ({ page }) => { + const mem = makeMemory({ id: "mem-edit-1", key: "edit.test.key", content: "Original content." }); + const state = { + memories: [mem], + stats: { totalEntries: 1, tokensUsed: 24, hitRate: 0.8, cacheStats: { hits: 4, misses: 1 } }, + settings: defaultSettings(), + engineStatus: defaultEngineStatus(), + createCalls: 0, + updateCalls: 0, + deleteCalls: 0, + settingsCalls: 0, + reindexCalls: 0, + previewCalls: 0, + }; + + await setupMemoryRoutes(page, state); + + await gotoDashboardRoute(page, "/dashboard/memory", { timeoutMs: NAVIGATION_TIMEOUT_MS }); + + // Wait for the memory to appear + await expect(async () => { + await expect(page.getByText("edit.test.key")).toBeVisible({ timeout: 10_000 }); + }).toPass({ timeout: 30_000, intervals: [1000, 2000] }); + + // Click the edit (pencil) button for our memory + const editBtn = page.getByTestId(`edit-memory-${mem.id}`); + await expect(editBtn).toBeVisible({ timeout: 10_000 }); + await editBtn.click(); + + // The edit modal should appear + const contentInput = page.getByLabel(/content/i).or(page.locator("textarea")).first(); + await expect(contentInput).toBeVisible({ timeout: 10_000 }); + await contentInput.fill("Updated content via modal."); + + // Save the changes + const saveBtn = page.getByRole("button", { name: /save/i }).last(); + await expect(saveBtn).toBeVisible({ timeout: 5_000 }); + await saveBtn.click(); + + // updateCalls should increment + await expect.poll(() => state.updateCalls).toBeGreaterThanOrEqual(1); + }); + + test("5. Playground tab — query and Simulate renders results", async ({ page }) => { + const state = { + memories: [makeMemory(), makeMemory({ id: "mem-2", key: "test.key.2", content: "Second fact." })], + stats: { totalEntries: 2, tokensUsed: 48, hitRate: 0.6, cacheStats: { hits: 3, misses: 2 } }, + settings: defaultSettings(), + engineStatus: defaultEngineStatus(), + createCalls: 0, + updateCalls: 0, + deleteCalls: 0, + settingsCalls: 0, + reindexCalls: 0, + previewCalls: 0, + }; + + await setupMemoryRoutes(page, state); + + await gotoDashboardRoute(page, "/dashboard/memory", { timeoutMs: NAVIGATION_TIMEOUT_MS }); + + // Navigate to Playground tab + await expect(page.getByTestId("tab-playground")).toBeVisible({ timeout: 30_000 }); + await page.getByTestId("tab-playground").click(); + + // Fill in query + const queryInput = page.getByTestId("playground-query-input"); + await expect(queryInput).toBeVisible({ timeout: 10_000 }); + await queryInput.fill("test"); + + // Click Simulate + const submitBtn = page.getByTestId("playground-submit"); + await expect(submitBtn).toBeVisible({ timeout: 5_000 }); + await expect(submitBtn).toBeEnabled({ timeout: 5_000 }); + await submitBtn.click(); + + // Wait for previewCalls to increment + await expect.poll(() => state.previewCalls).toBeGreaterThanOrEqual(1); + + // Results section should appear (result count heading) + await expect( + page.getByText(/result\(s\)|resultado\(s\)/, { exact: false }), + ).toBeVisible({ timeout: 15_000 }); + }); + + test("6. Engine tab — status chips render and toggle transformers", async ({ page }) => { + const state = { + memories: [], + stats: { totalEntries: 0, tokensUsed: 0, hitRate: 0, cacheStats: { hits: 0, misses: 0 } }, + settings: defaultSettings(), + engineStatus: defaultEngineStatus(), + createCalls: 0, + updateCalls: 0, + deleteCalls: 0, + settingsCalls: 0, + reindexCalls: 0, + previewCalls: 0, + }; + + await setupMemoryRoutes(page, state); + + await gotoDashboardRoute(page, "/dashboard/memory", { timeoutMs: NAVIGATION_TIMEOUT_MS }); + + // Navigate to Engine tab + await expect(page.getByTestId("tab-engine")).toBeVisible({ timeout: 30_000 }); + await page.getByTestId("tab-engine").click(); + + // Status section should be visible (Reindex Now button is a proxy for the engine panel) + const reindexBtn = page.getByTestId("reindex-now-button"); + await expect(reindexBtn).toBeVisible({ timeout: 20_000 }); + + // Engine status heading + await expect( + page.getByText(/engine status|status do engine/i, { exact: false }), + ).toBeVisible({ timeout: 10_000 }); + }); + + test("7. Reindex Now button triggers POST /api/memory/reindex", async ({ page }) => { + const state = { + memories: [], + stats: { totalEntries: 0, tokensUsed: 0, hitRate: 0, cacheStats: { hits: 0, misses: 0 } }, + settings: defaultSettings(), + engineStatus: { ...defaultEngineStatus(), vectorStore: { ...defaultEngineStatus().vectorStore, needsReindex: 5 } }, + createCalls: 0, + updateCalls: 0, + deleteCalls: 0, + settingsCalls: 0, + reindexCalls: 0, + previewCalls: 0, + }; + + await setupMemoryRoutes(page, state); + + await gotoDashboardRoute(page, "/dashboard/memory", { timeoutMs: NAVIGATION_TIMEOUT_MS }); + + // Navigate to Engine tab + await expect(page.getByTestId("tab-engine")).toBeVisible({ timeout: 30_000 }); + await page.getByTestId("tab-engine").click(); + + // Click Reindex Now + const reindexBtn = page.getByTestId("reindex-now-button"); + await expect(reindexBtn).toBeVisible({ timeout: 20_000 }); + await reindexBtn.click(); + + // reindexCalls should increment (request was made) + await expect.poll(() => state.reindexCalls).toBeGreaterThanOrEqual(1); + }); +}); diff --git a/tests/e2e/memory-qdrant-routes.spec.ts b/tests/e2e/memory-qdrant-routes.spec.ts new file mode 100644 index 0000000000..5af89e8854 --- /dev/null +++ b/tests/e2e/memory-qdrant-routes.spec.ts @@ -0,0 +1,358 @@ +import { expect, test, type Page, type Route } from "@playwright/test"; +import { gotoDashboardRoute } from "./helpers/dashboardAuth"; + +const NAVIGATION_TIMEOUT_MS = 300_000; + +// --------------------------------------------------------------------------- +// Helpers +// --------------------------------------------------------------------------- + +async function fulfillJson(route: Route, body: unknown, status = 200) { + await route.fulfill({ + status, + contentType: "application/json", + body: JSON.stringify(body), + }); +} + +type QdrantSettings = { + enabled: boolean; + host: string; + port: number; + collection: string; + embeddingModel: string; + hasApiKey: boolean; + apiKeyMasked: string | null; +}; + +type MemorySettings = { + enabled: boolean; + maxTokens: number; + retentionDays: number; + strategy: "recent" | "semantic" | "hybrid"; + skillsEnabled: boolean; + embeddingSource: "remote" | "static" | "transformers" | "auto"; + embeddingProviderModel: string | null; + transformersEnabled: boolean; + staticEnabled: boolean; + rerankEnabled: boolean; + rerankProviderModel: string | null; + vectorStore: "sqlite-vec" | "qdrant" | "auto"; +}; + +function defaultQdrantSettings(): QdrantSettings { + return { + enabled: false, + host: "", + port: 6333, + collection: "omniroute_memory", + embeddingModel: "openai/text-embedding-3-small", + hasApiKey: false, + apiKeyMasked: null, + }; +} + +function defaultMemorySettings(): MemorySettings { + return { + enabled: true, + maxTokens: 2000, + retentionDays: 30, + strategy: "hybrid", + skillsEnabled: false, + embeddingSource: "auto", + embeddingProviderModel: null, + transformersEnabled: false, + staticEnabled: false, + rerankEnabled: false, + rerankProviderModel: null, + vectorStore: "auto", + }; +} + +/** + * Set up all route mocks needed for the Engine tab + QdrantConfigCard. + * + * Key security assertion: health/search/cleanup endpoints return error payloads + * WITHOUT a stack trace (no "at /…" lines) — validates Hard Rule #12 compliance. + */ +async function setupQdrantRoutes( + page: Page, + state: { + qdrantSettings: QdrantSettings; + memorySettings: MemorySettings; + healthCalls: number; + settingsPutCalls: number; + searchCalls: number; + cleanupCalls: number; + }, +) { + // /api/memory (GET) — empty list, needed by MemoriesTab which is the default + await page.route(/\/api\/memory(\?.*)?$/, async (route) => { + if (route.request().method() === "GET") { + await fulfillJson(route, { + data: [], + total: 0, + totalPages: 1, + stats: { total: 0, tokensUsed: 0, hitRate: 0 }, + }); + return; + } + await fulfillJson(route, { error: { message: "Not allowed" } }, 405); + }); + + // /api/memory/engine-status + await page.route(/\/api\/memory\/engine-status$/, async (route) => { + await fulfillJson(route, { + keyword: { available: true, backend: "FTS5" }, + embedding: { + source: null, + model: null, + dimensions: null, + available: false, + reason: "No embedding source", + cacheStats: { hits: 0, misses: 0, size: 0 }, + }, + vectorStore: { + backend: "none", + available: false, + rowCount: 0, + needsReindex: 0, + reason: "sqlite-vec unavailable", + }, + qdrant: { enabled: false, healthy: null, latencyMs: null, error: null }, + rerank: { + enabled: false, + provider: null, + model: null, + available: false, + reason: "Rerank disabled", + }, + }); + }); + + // /api/memory/embedding-providers + await page.route(/\/api\/memory\/embedding-providers$/, async (route) => { + await fulfillJson(route, { providers: [] }); + }); + + // /api/memory/health + await page.route(/\/api\/memory\/health$/, async (route) => { + await fulfillJson(route, { working: true, latencyMs: 5 }); + }); + + // /api/memory/reindex + await page.route(/\/api\/memory\/reindex$/, async (route) => { + await fulfillJson(route, { started: true, pending: 0 }); + }); + + // GET/PUT /api/settings/memory + await page.route(/\/api\/settings\/memory$/, async (route) => { + const method = route.request().method(); + if (method === "GET") { + await fulfillJson(route, state.memorySettings); + return; + } + if (method === "PUT") { + const body = route.request().postDataJSON() as Partial; + state.memorySettings = { ...state.memorySettings, ...body }; + await fulfillJson(route, state.memorySettings); + return; + } + await fulfillJson(route, { error: { message: "Method not allowed" } }, 405); + }); + + // GET/PUT /api/settings/qdrant + await page.route(/\/api\/settings\/qdrant$/, async (route) => { + const method = route.request().method(); + if (method === "GET") { + await fulfillJson(route, state.qdrantSettings); + return; + } + if (method === "PUT") { + state.settingsPutCalls += 1; + const body = route.request().postDataJSON() as Partial; + state.qdrantSettings = { + ...state.qdrantSettings, + ...body, + // PUT returns the sanitized version (no raw apiKey field) + }; + await fulfillJson(route, { + ...state.qdrantSettings, + hasApiKey: false, + apiKeyMasked: null, + }); + return; + } + await fulfillJson(route, { error: { message: "Method not allowed" } }, 405); + }); + + // GET /api/settings/qdrant/health — simulates a refused connection. + // The error message MUST NOT contain a stack trace (Hard Rule #12). + await page.route(/\/api\/settings\/qdrant\/health$/, async (route) => { + state.healthCalls += 1; + // Return a structured error that is safe (no stack trace) + await fulfillJson(route, { + ok: false, + latencyMs: 0, + error: "connect ECONNREFUSED 127.0.0.1:6333", + }); + }); + + // GET /api/settings/qdrant/embedding-models + await page.route(/\/api\/settings\/qdrant\/embedding-models$/, async (route) => { + await fulfillJson(route, { models: ["openai/text-embedding-3-small"] }); + }); + + // POST /api/settings/qdrant/search — simulates failed search (Qdrant not running) + await page.route(/\/api\/settings\/qdrant\/search$/, async (route) => { + state.searchCalls += 1; + await fulfillJson( + route, + { + error: { + message: "Qdrant connection failed: ECONNREFUSED", + code: "QDRANT_UNAVAILABLE", + }, + }, + 503, + ); + }); + + // POST /api/settings/qdrant/cleanup — simulates failed cleanup (Qdrant not running) + await page.route(/\/api\/settings\/qdrant\/cleanup$/, async (route) => { + state.cleanupCalls += 1; + await fulfillJson( + route, + { + error: { + message: "Qdrant cleanup failed: ECONNREFUSED", + code: "QDRANT_UNAVAILABLE", + }, + }, + 503, + ); + }); +} + +// --------------------------------------------------------------------------- +// Tests +// --------------------------------------------------------------------------- + +test.describe("Memory Qdrant routes — Engine tab integration", () => { + test.setTimeout(600_000); + + test("Engine tab renders Qdrant config card", async ({ page }) => { + const state = { + qdrantSettings: defaultQdrantSettings(), + memorySettings: defaultMemorySettings(), + healthCalls: 0, + settingsPutCalls: 0, + searchCalls: 0, + cleanupCalls: 0, + }; + + await setupQdrantRoutes(page, state); + + await gotoDashboardRoute(page, "/dashboard/memory", { timeoutMs: NAVIGATION_TIMEOUT_MS }); + + // Navigate to Engine tab + await expect(page.getByTestId("tab-engine")).toBeVisible({ timeout: 30_000 }); + await page.getByTestId("tab-engine").click(); + + // Qdrant section should be visible + await expect( + page.getByText(/qdrant/i, { exact: false }), + ).toBeVisible({ timeout: 20_000 }); + + // Qdrant enabled switch should be visible + const qdrantSwitch = page.getByTestId("qdrant-enabled-switch"); + await expect(qdrantSwitch).toBeVisible({ timeout: 15_000 }); + }); + + test("Test Connection button triggers GET /api/settings/qdrant/health with sanitized error", async ({ + page, + }) => { + const state = { + qdrantSettings: { ...defaultQdrantSettings(), host: "localhost" }, + memorySettings: defaultMemorySettings(), + healthCalls: 0, + settingsPutCalls: 0, + searchCalls: 0, + cleanupCalls: 0, + }; + + await setupQdrantRoutes(page, state); + + await gotoDashboardRoute(page, "/dashboard/memory", { timeoutMs: NAVIGATION_TIMEOUT_MS }); + + // Navigate to Engine tab + await expect(page.getByTestId("tab-engine")).toBeVisible({ timeout: 30_000 }); + await page.getByTestId("tab-engine").click(); + + // Find and click the Test Connection button + const testConnBtn = page.getByTestId("qdrant-test-connection"); + await expect(testConnBtn).toBeVisible({ timeout: 20_000 }); + await testConnBtn.click(); + + // healthCalls should increment + await expect.poll(() => state.healthCalls).toBeGreaterThanOrEqual(1); + + // The error should surface in the UI — but without a stack trace + // "ECONNREFUSED" is acceptable; "at /" (stack trace marker) is not + await expect(async () => { + const bodyText = await page.locator("body").innerText(); + // Error is shown (connection refused, not just silent failure) + expect( + bodyText.toLowerCase().includes("error") || + bodyText.toLowerCase().includes("refused") || + bodyText.toLowerCase().includes("failed") || + bodyText.toLowerCase().includes("erro"), + ).toBe(true); + // Must NOT contain a stack trace + expect(bodyText).not.toMatch(/\sat\s\//); + }).toPass({ timeout: 15_000, intervals: [1000, 2000] }); + }); + + test("Cleanup button triggers POST /api/settings/qdrant/cleanup with sanitized error", async ({ + page, + }) => { + const state = { + qdrantSettings: { ...defaultQdrantSettings(), host: "localhost" }, + memorySettings: defaultMemorySettings(), + healthCalls: 0, + settingsPutCalls: 0, + searchCalls: 0, + cleanupCalls: 0, + }; + + await setupQdrantRoutes(page, state); + + await gotoDashboardRoute(page, "/dashboard/memory", { timeoutMs: NAVIGATION_TIMEOUT_MS }); + + // Navigate to Engine tab + await expect(page.getByTestId("tab-engine")).toBeVisible({ timeout: 30_000 }); + await page.getByTestId("tab-engine").click(); + + // Find and click the Cleanup button + const cleanupBtn = page.getByTestId("qdrant-cleanup"); + await expect(cleanupBtn).toBeVisible({ timeout: 20_000 }); + await cleanupBtn.click(); + + // cleanupCalls should increment + await expect.poll(() => state.cleanupCalls).toBeGreaterThanOrEqual(1); + + // The error should surface in the UI but without a stack trace + await expect(async () => { + const bodyText = await page.locator("body").innerText(); + // Error surfaces + expect( + bodyText.toLowerCase().includes("error") || + bodyText.toLowerCase().includes("failed") || + bodyText.toLowerCase().includes("falh") || + bodyText.toLowerCase().includes("cleanup"), + ).toBe(true); + // Must NOT contain a stack trace + expect(bodyText).not.toMatch(/\sat\s\//); + }).toPass({ timeout: 15_000, intervals: [1000, 2000] }); + }); +});