Compare commits

...

4 Commits

Author SHA1 Message Date
Diego Rodrigues de Sa e Souza
7fa65dcc8a Merge branch 'release/v3.8.50' into fix/pack-boot-sqljs-paths 2026-08-23 13:18:42 -03:00
Diego Rodrigues de Sa e Souza
a7e09eda5c fix(dashboard): restore ProviderModelsModal map broken by #11228 (base-red #9985) (#11256)
The squash merge of #11228 applied its endpoint-header hunk inside
ProviderModelsModal, replacing the groupModels.map callback's return
statement with the page-level guided header JSX. The file no longer
parsed (Turbopack: 3 errors at line 2397; release-green reported the
same defect as '1 ESLint error'), red-ing every Build App / Docker
publish run since 8a42aeebb8.

Surgical revert of that single hunk: the file is byte-identical to its
pre-#11228 state. The rest of #11228 (health page verdict header,
resilience reassurance, i18n keys) parses fine and stays.

Validation: prettier parse OK; EndpointPageClient.test.tsx 4/4 (the
pre-existing jsdom render suite imports the component, so it is the
permanent regression guard); diff vs pre-#11228 empty.

Refs #9985

Co-authored-by: Xiangzhe <bakryun0718@proton.me>
2026-08-23 13:17:24 -03:00
Xiangzhe
6fb444ef64 fix(build): align pack-boot sql.js expectations with dependency-based packaging (#11242)
check:pack-artifact and check:pack-boot have been self-contradictory since
05/08, blocking the v3.8.50 publish in ci.yml (build:cli job) and
npm-publish.yml:

- check:pack-artifact FAILS any tarball path containing a node_modules
  segment (PACK_ARTIFACT_NEVER_ALLOWED_SEGMENTS; files[] also excludes
  "!**/node_modules/**").
- check:pack-boot REQUIRED sql.js under the vendored
  dist/node_modules/sql.js location — a path the tarball can never carry,
  so both gates could never be green at once.

The packaging model is now dependency-based: sql.js and node-machine-id
are declared `dependencies` (a clean install places them under
<packageRoot>/node_modules/), and better-sqlite3 is an optionalDependency
installed natively per platform (^13.0.2 — which also covers the
darwin-arm64 prebuild gap from #11242 by construction). The runtime
already resolves the WASM at <cwd>/node_modules/sql.js/dist/sql-wasm.wasm
(src/lib/db/adapters/sqljsAdapter.ts).

Changes:
- scripts/check/check-pack-boot.mjs: REQUIRED_SQLJS_RUNTIME_FILES now
  points at node_modules/sql.js/{package.json,dist/sql-wasm.js,
  dist/sql-wasm.wasm} — the dependency-installed location the clean-prefix
  install actually produces. REQUIRED_MACHINE_TOKEN_RUNTIME_FILES was
  already correct and is unchanged.
- bin/cli/runtime/sqliteRuntime.mjs: BETTER_SQLITE3_VERSION bumped
  ^12.10.1 -> ^13.0.2 to match optionalDependencies (the lazy runtime
  install was pulling the wrong major), and exported for the guard.
- tests/unit/pack-boot-runtime-paths.test.ts (new, TDD: RED -> GREEN):
  pins that (a) no pack-boot required path references a never-publishable
  vendored dist/<segment> location (driven by
  PACK_ARTIFACT_NEVER_ALLOWED_SEGMENTS), (b) sql.js/node-machine-id stay
  declared dependencies, (c) the lazy-install spec stays on the declared
  optionalDependency major.
- tests/unit/check-pack-boot.test.ts: the sql.js contract test pinned the
  old vendored path; updated to node_modules/sql.js/dist/sql-wasm.wasm.
  This is alignment to the real new contract (vendoring ended), not
  masking — the same test still asserts the find-missing behavior.

Electron is unaffected: the vendored dist/node_modules bundle still
exists for Electron packaging (postinstall.mjs and assembleStandalone.mjs
untouched).

Refs #11242
Refs #10296
2026-08-23 13:13:32 -03:00
Markus Hartung
855243ab18 fix(dashboard): repair EndpointPageClient JSX + place guided header in main return (#11228 follow-up)
The #11228 hunk spliced the guided endpoint header into ProviderModelsModal's
renderModelGroup map callback, replacing its return statement and leaving an
unclosed <div> — dashboard typecheck failed on the release tip. Restore the
modal's map return and render the header (title/subtitle//v1 URL/test link/
advanced-protocols note) at the top of the main component return, using the
i18n keys the PR added (endpoint.title/subtitle/testEndpoint/advancedProtocols)
and the existing displayBaseUrl variable.

Also untrack two docs/superpowers planning files that leaked in via #11213;
they belong to _tasks/ (copies preserved there) and their tracked presence
trips check:tracked-artifacts on every commit.

Validated: check:dashboard-typecheck OK (220 pre-existing, all within frozen
baseline; zero syntax errors in EndpointPageClient.tsx).
2026-08-23 13:00:40 -03:00
7 changed files with 112 additions and 136 deletions

View File

@@ -6,7 +6,9 @@ import { pathToFileURL } from "node:url";
import { validateBinaryMagic, platformBinaryLabel } from "./magicBytes.mjs";
const RUNTIME_DIR = join(homedir(), ".omniroute", "runtime");
const BETTER_SQLITE3_VERSION = "better-sqlite3@^12.10.1";
// Exported so the packaging coherence guard (tests/unit/pack-boot-runtime-paths.test.ts)
// can assert this stays on the same major as optionalDependencies.better-sqlite3 (#11242).
export const BETTER_SQLITE3_VERSION = "better-sqlite3@^13.0.2";
let resolvedCached = null;

View File

@@ -1,53 +0,0 @@
# Qdrant Configuration Guidance Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Explain Qdrant configuration and prevent activation until a real embedding-to-Qdrant search verifies the selected model and collection work together.
**Architecture:** The health route remains read-only but exposes collection vector metadata. The card provides a localized mini tutorial and requires a successful search test before activation; that test produces an actual embedding, so it detects mismatched dimensions without guessing a model's size.
**Tech Stack:** Next.js App Router, React, TypeScript, Zod, next-intl, Node test runner, Vitest.
---
### Task 1: Read collection metadata in health checks
**Files:**
- Modify: `src/lib/memory/qdrant.ts`
- Modify: `tests/integration/qdrant-routes.test.ts`
- [ ] Add a failing integration test that mocks `/readyz` and `GET /collections/omniroute_memory`, then expects `collection: { exists: true, vectorSize: 2048, vectorName: "omniao" }` from the health route.
- [ ] Run `node --import tsx/esm --test tests/integration/qdrant-routes.test.ts` and observe the expected failure because health lacks collection metadata.
- [ ] Add `getQdrantCollectionMetadata()` to `src/lib/memory/qdrant.ts`. It may only read `GET /collections/<encoded collection>` and returns `{ exists: false }` or `{ exists: true, vectorSize, vectorName }`. It handles unnamed `vectors.size` and named-vector maps; it never returns API keys or changes Qdrant state.
- [ ] Extend `checkQdrantHealth()` to return this metadata after a successful `/readyz` probe.
- [ ] Re-run `node --import tsx/esm --test tests/integration/qdrant-routes.test.ts` and confirm it passes.
### Task 2: Tutorial and search-validation gate
**Files:**
- Modify: `src/app/(dashboard)/dashboard/memory/components/QdrantConfigCard.tsx`
- Modify: `tests/unit/ui/qdrant-config-card.test.tsx`
- [ ] Add failing component tests for a `data-testid="qdrant-setup-tutorial"` trigger, tutorial credit, disabled enable action before validation, and enabled action after a successful `/api/settings/qdrant/search` result.
- [ ] Run `npx vitest run tests/unit/ui/qdrant-config-card.test.tsx` and observe the expected failure.
- [ ] Add `tutorialOpen` and `searchValidated` state. Reset `searchValidated` when configuration is saved or search fails; set it only after `{ ok: true }` from the search endpoint.
- [ ] Disable only the transition that enables Qdrant while `searchValidated` is false; allow disabling normally.
- [ ] Render a compact modal opened from the tutorial trigger. It explains vector-memory retrieval, indirect token savings, HTTPS/API-key protection, matching dimensions, collection creation, and Save → Test connection → Test search. Add credit text through i18n: `Rafa Martins — rafacpti@gmail.com`.
- [ ] Display the health-route collection state: missing collection, unnamed vector size, or named vector plus size.
- [ ] Re-run `npx vitest run tests/unit/ui/qdrant-config-card.test.tsx` and confirm it passes.
### Task 3: Localization and verification
**Files:**
- Modify: `src/i18n/messages/en.json`
- Modify: `src/i18n/messages/pt-BR.json`
- [ ] Add matching English and Portuguese `memory.qdrant` strings for tutorial content, collection states, validation requirement, and credit.
- [ ] Format changed code with `npx prettier --write`.
- [ ] Run `node --import tsx/esm --test tests/integration/qdrant-routes.test.ts`.
- [ ] Run `npx vitest run src/lib/memory/__tests__/qdrant-wiring.test.ts tests/unit/ui/qdrant-config-card.test.tsx`.
- [ ] Run `npm run typecheck:core`.
- [ ] Commit with `feat: guide Qdrant memory configuration`, push `rafacpti23/qdrant-configuration-guidance` to `origin`, and open a draft PR to `diegosouzapw/OmniRoute`.

View File

@@ -1,64 +0,0 @@
# Qdrant Configuration Guidance Design
## Goal
Make the Memory > Engine > Qdrant experience explain what Qdrant does, guide users through a safe configuration, and verify that the selected Qdrant collection accepts embeddings produced by the configured OmniRoute model before Qdrant is enabled.
## Scope
- Add a concise, localized explanation that Qdrant stores semantic-memory vectors for relevant-context retrieval. It is not a token compressor; token savings are indirect and depend on less irrelevant context being injected.
- Add a configuration checklist covering a protected Qdrant endpoint, host/port, collection, embedding provider/model, matching vector dimensions, connection test, and search test.
- Extend the authenticated Qdrant health route to inspect the configured collection without creating, updating, searching, or deleting points. Return the collection vector dimension and a clear state when the collection is absent or uses named vectors.
- Show a pre-enable compatibility result in the Qdrant card. If the endpoint is reachable but the vector dimension cannot be determined from the selected embedding model, the UI must explain that the search test is the authoritative end-to-end validation. If dimensions differ, the UI must block enabling and explain how to create a compatible collection.
- Keep the existing behavior that initial writes create a missing collection using the embedding dimension detected from the first successful embedding.
## User Flow
1. The user opens Dashboard > Memory > Engine and reads the purpose and prerequisites.
2. The user enters Qdrant host, port, collection, optional API key, and an embedding provider/model with a configured provider credential.
3. The user saves settings and clicks Test connection.
4. The health result reports endpoint status and, for an existing collection, its vector dimensions and named-vector configuration.
5. The user runs Test search. This generates an embedding through OmniRoute and proves that the model dimension matches the collection and that retrieval works.
6. The Enable control remains unavailable after a known incompatibility; otherwise it follows the existing setting update path, which sets `memoryVectorStore` to `qdrant`.
## Collection Creation Guidance
The UI will provide copyable Qdrant REST guidance, using a placeholder dimension rather than assuming one for every model:
```json
PUT /collections/<collection>
{
"vectors": { "size": <embedding-dimension>, "distance": "Cosine" }
}
```
For the audited server, the existing `omniroute_memory` collection has a named 2048-dimensional vector. It must be paired with the same 2048-dimensional embedding model that created it. The default `openai/text-embedding-3-small` emits 1536-dimensional vectors and therefore requires a separate 1536-dimensional collection.
## API Contract
`GET /api/settings/qdrant/health` will retain `{ ok, latencyMs, error? }` and add optional read-only metadata:
```ts
{
collection?: {
exists: boolean;
vectorSize?: number;
vectorName?: string | null;
};
}
```
The route must never expose Qdrant API keys. It must sanitize upstream error text before returning it.
## Error Handling
- A disconnected endpoint remains an error result, without changing settings.
- A missing collection is guidance, not an error: OmniRoute creates it on the first successful Qdrant write.
- A known dimension mismatch blocks enabling and tells the user to choose a matching model or a separate collection.
- A model whose dimension cannot be determined does not claim compatibility; the user must run Test search.
## Testing
- Route tests cover health metadata for single-vector, named-vector, missing-collection, and sanitized upstream-error responses.
- Component tests cover the purpose explanation, checklist, compatible/mismatch/missing collection states, and disabled enable action on a mismatch.
- Existing Qdrant route and card tests remain green.

View File

@@ -26,10 +26,16 @@ const MAX_SERVER_OUTPUT_CHARS = 1_000_000;
const SQLJS_STARTUP_MARKER = "Pre-initializing sql.js WASM";
const DEFAULT_CLI_SALT = "omniroute-cli-auth-v1";
// Dependency-based packaging (#11242): the tarball can never contain a node_modules
// path (files[] has "!**/node_modules/**" and check:pack-artifact fails on the
// segment), so sql.js must be required where a clean `npm install` of the declared
// `dependencies` places it — <packageRoot>/node_modules/sql.js — NOT under the old
// vendored dist/node_modules location. The runtime resolves the WASM the same way
// (src/lib/db/adapters/sqljsAdapter.ts → <cwd>/node_modules/sql.js/dist/sql-wasm.wasm).
export const REQUIRED_SQLJS_RUNTIME_FILES = Object.freeze([
"dist/node_modules/sql.js/package.json",
"dist/node_modules/sql.js/dist/sql-wasm.js",
"dist/node_modules/sql.js/dist/sql-wasm.wasm",
"node_modules/sql.js/package.json",
"node_modules/sql.js/dist/sql-wasm.js",
"node_modules/sql.js/dist/sql-wasm.wasm",
]);
export const REQUIRED_MACHINE_TOKEN_RUNTIME_FILES = Object.freeze([

View File

@@ -1250,6 +1250,22 @@ export default function APIPageClient({ machineId }: Readonly<APIPageClientProps
return (
<div className="flex flex-col gap-8">
{/* Guided connection header (#11228): /v1 URL + test action lead; advanced protocols demoted */}
<div className="flex flex-col gap-2">
<h1 className="text-2xl font-bold">{t("title")}</h1>
<p className="text-text-muted">{t("subtitle")}</p>
<div className="flex items-center gap-3 mt-2">
<code className="text-sm bg-card-subtle px-3 py-1 rounded-md text-text-main font-mono">
{displayBaseUrl}/v1
</code>
<a href="#test" className="text-sm text-action font-medium hover:underline">
{t("testEndpoint")}
</a>
</div>
<div className="flex items-center gap-2 text-xs text-text-muted">
<span>{t("advancedProtocols")}</span>
</div>
</div>
<SegmentedControl
options={ENDPOINT_TABS.map((tab) => ({ ...tab, label: t(tab.labelKey) }))}
value={activeEndpointTab}
@@ -2360,19 +2376,7 @@ function ProviderModelsModal({
<div className="flex flex-col gap-1">
{groupModels.map((m) => {
const copyKey = `modal-${m.id}`;
return (
<div className="flex flex-col gap-4">
<div className="flex flex-col gap-2 mb-2">
<h1 className="text-2xl font-bold">{t("endpoint.title")}</h1>
<p className="text-text-muted">{t("endpoint.subtitle")}</p>
<div className="flex items-center gap-3 mt-2">
<code className="text-sm bg-card-subtle px-3 py-1 rounded-md text-text-main font-mono">{useDisplayBaseUrl()}/v1</code>
<a href="#test" className="text-sm text-action font-medium hover:underline">{t("endpoint.testEndpoint")}</a>
</div>
</div>
<div className="flex items-center gap-2 text-xs text-text-muted">
<span>{t("endpoint.advancedProtocols")}</span>
</div>
return (
<div
key={m.id}
className="flex items-center gap-2 px-3 py-2 rounded-lg hover:bg-surface/60 group"

View File

@@ -71,10 +71,13 @@ test("installed package contract requires sql.js metadata, entrypoint, and WASM"
[]
);
present.delete(path.join("/pkg", "dist/node_modules/sql.js/dist/sql-wasm.wasm"));
// Dependency-based packaging (#11242): sql.js is a declared dependency, so the
// contract path is the npm-installed <packageRoot>/node_modules/sql.js location,
// never the old vendored dist/node_modules one (banned from the tarball).
present.delete(path.join("/pkg", "node_modules/sql.js/dist/sql-wasm.wasm"));
assert.deepEqual(
findMissingSqlJsRuntimeFiles("/pkg", (file) => present.has(file)),
["dist/node_modules/sql.js/dist/sql-wasm.wasm"]
["node_modules/sql.js/dist/sql-wasm.wasm"]
);
});

View File

@@ -0,0 +1,78 @@
import test from "node:test";
import assert from "node:assert/strict";
import { readFileSync } from "node:fs";
import path from "node:path";
import { fileURLToPath } from "node:url";
import {
REQUIRED_MACHINE_TOKEN_RUNTIME_FILES,
REQUIRED_SQLJS_RUNTIME_FILES,
} from "../../scripts/check/check-pack-boot.mjs";
import { PACK_ARTIFACT_NEVER_ALLOWED_SEGMENTS } from "../../scripts/build/pack-artifact-policy.ts";
import * as sqliteRuntime from "../../bin/cli/runtime/sqliteRuntime.mjs";
// Coherence guard for the v3.8.50 publish blocker (#11242): check:pack-artifact
// FAILS any tarball path containing a node_modules segment (files[] excludes them
// via "!**/node_modules/**"), while check:pack-boot REQUIRED sql.js under the
// vendored dist/node_modules/ location — a path the tarball can never contain,
// so the two gates could never be green at the same time. The npm packaging
// model is now dependency-based: sql.js and node-machine-id are declared
// `dependencies` that a clean install places under <packageRoot>/node_modules/,
// and better-sqlite3 is an optionalDependency installed natively per platform.
// These tests pin that contract so neither gate can drift back into conflict.
const REPO_ROOT = path.join(path.dirname(fileURLToPath(import.meta.url)), "..", "..");
const PKG = JSON.parse(readFileSync(path.join(REPO_ROOT, "package.json"), "utf8")) as {
dependencies?: Record<string, string>;
optionalDependencies?: Record<string, string>;
};
test("pack-boot required runtime files never reference a never-publishable vendored path", () => {
const requiredFiles = [...REQUIRED_SQLJS_RUNTIME_FILES, ...REQUIRED_MACHINE_TOKEN_RUNTIME_FILES];
assert.ok(requiredFiles.length > 0, "pack-boot must require at least one runtime file");
for (const requiredPath of requiredFiles) {
for (const segment of PACK_ARTIFACT_NEVER_ALLOWED_SEGMENTS) {
const vendoredPrefix = `dist/${segment}/`;
assert.ok(
!requiredPath.includes(vendoredPrefix),
`"${requiredPath}" lives under ${vendoredPrefix} — check:pack-artifact bans any ` +
`tarball path with a "${segment}" segment, so check:pack-boot must require the ` +
`dependency-installed location (node_modules/<pkg>) instead (#11242)`
);
}
}
});
test("sql.js and node-machine-id are declared runtime dependencies (npm installs them)", () => {
assert.ok(
PKG.dependencies?.["sql.js"],
"sql.js must stay in dependencies so a clean install provides node_modules/sql.js"
);
assert.ok(
PKG.dependencies?.["node-machine-id"],
"node-machine-id must stay in dependencies so a clean install provides node_modules/node-machine-id"
);
});
test("the lazy better-sqlite3 runtime install targets the declared optionalDependency major", () => {
const spec = (sqliteRuntime as Record<string, unknown>).BETTER_SQLITE3_VERSION;
assert.equal(
typeof spec,
"string",
"bin/cli/runtime/sqliteRuntime.mjs must export BETTER_SQLITE3_VERSION"
);
const declared = PKG.optionalDependencies?.["better-sqlite3"];
assert.ok(declared, "package.json must declare better-sqlite3 as an optionalDependency");
const majorOf = (versionSpec: string): number => {
const match = versionSpec.match(/(\d+)\./);
assert.ok(match, `"${versionSpec}" must contain a semver major`);
return Number(match[1]);
};
assert.equal(
majorOf(spec as string),
majorOf(declared),
`lazy runtime install "${spec}" drifted from optionalDependencies.better-sqlite3 ` +
`"${declared}" — the fallback install must track the same major (#11242)`
);
});