mirror of
https://github.com/diegosouzapw/OmniRoute.git
synced 2026-08-23 23:52:18 +03:00
Compare commits
4 Commits
fix/11244-
...
fix/11233-
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
6b83df1a75 | ||
|
|
a7e09eda5c | ||
|
|
855243ab18 | ||
|
|
592a7efc18 |
@@ -1,53 +0,0 @@
|
||||
# Qdrant Configuration Guidance Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Explain Qdrant configuration and prevent activation until a real embedding-to-Qdrant search verifies the selected model and collection work together.
|
||||
|
||||
**Architecture:** The health route remains read-only but exposes collection vector metadata. The card provides a localized mini tutorial and requires a successful search test before activation; that test produces an actual embedding, so it detects mismatched dimensions without guessing a model's size.
|
||||
|
||||
**Tech Stack:** Next.js App Router, React, TypeScript, Zod, next-intl, Node test runner, Vitest.
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Read collection metadata in health checks
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/lib/memory/qdrant.ts`
|
||||
- Modify: `tests/integration/qdrant-routes.test.ts`
|
||||
|
||||
- [ ] Add a failing integration test that mocks `/readyz` and `GET /collections/omniroute_memory`, then expects `collection: { exists: true, vectorSize: 2048, vectorName: "omniao" }` from the health route.
|
||||
- [ ] Run `node --import tsx/esm --test tests/integration/qdrant-routes.test.ts` and observe the expected failure because health lacks collection metadata.
|
||||
- [ ] Add `getQdrantCollectionMetadata()` to `src/lib/memory/qdrant.ts`. It may only read `GET /collections/<encoded collection>` and returns `{ exists: false }` or `{ exists: true, vectorSize, vectorName }`. It handles unnamed `vectors.size` and named-vector maps; it never returns API keys or changes Qdrant state.
|
||||
- [ ] Extend `checkQdrantHealth()` to return this metadata after a successful `/readyz` probe.
|
||||
- [ ] Re-run `node --import tsx/esm --test tests/integration/qdrant-routes.test.ts` and confirm it passes.
|
||||
|
||||
### Task 2: Tutorial and search-validation gate
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/app/(dashboard)/dashboard/memory/components/QdrantConfigCard.tsx`
|
||||
- Modify: `tests/unit/ui/qdrant-config-card.test.tsx`
|
||||
|
||||
- [ ] Add failing component tests for a `data-testid="qdrant-setup-tutorial"` trigger, tutorial credit, disabled enable action before validation, and enabled action after a successful `/api/settings/qdrant/search` result.
|
||||
- [ ] Run `npx vitest run tests/unit/ui/qdrant-config-card.test.tsx` and observe the expected failure.
|
||||
- [ ] Add `tutorialOpen` and `searchValidated` state. Reset `searchValidated` when configuration is saved or search fails; set it only after `{ ok: true }` from the search endpoint.
|
||||
- [ ] Disable only the transition that enables Qdrant while `searchValidated` is false; allow disabling normally.
|
||||
- [ ] Render a compact modal opened from the tutorial trigger. It explains vector-memory retrieval, indirect token savings, HTTPS/API-key protection, matching dimensions, collection creation, and Save → Test connection → Test search. Add credit text through i18n: `Rafa Martins — rafacpti@gmail.com`.
|
||||
- [ ] Display the health-route collection state: missing collection, unnamed vector size, or named vector plus size.
|
||||
- [ ] Re-run `npx vitest run tests/unit/ui/qdrant-config-card.test.tsx` and confirm it passes.
|
||||
|
||||
### Task 3: Localization and verification
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/i18n/messages/en.json`
|
||||
- Modify: `src/i18n/messages/pt-BR.json`
|
||||
|
||||
- [ ] Add matching English and Portuguese `memory.qdrant` strings for tutorial content, collection states, validation requirement, and credit.
|
||||
- [ ] Format changed code with `npx prettier --write`.
|
||||
- [ ] Run `node --import tsx/esm --test tests/integration/qdrant-routes.test.ts`.
|
||||
- [ ] Run `npx vitest run src/lib/memory/__tests__/qdrant-wiring.test.ts tests/unit/ui/qdrant-config-card.test.tsx`.
|
||||
- [ ] Run `npm run typecheck:core`.
|
||||
- [ ] Commit with `feat: guide Qdrant memory configuration`, push `rafacpti23/qdrant-configuration-guidance` to `origin`, and open a draft PR to `diegosouzapw/OmniRoute`.
|
||||
@@ -1,64 +0,0 @@
|
||||
# Qdrant Configuration Guidance Design
|
||||
|
||||
## Goal
|
||||
|
||||
Make the Memory > Engine > Qdrant experience explain what Qdrant does, guide users through a safe configuration, and verify that the selected Qdrant collection accepts embeddings produced by the configured OmniRoute model before Qdrant is enabled.
|
||||
|
||||
## Scope
|
||||
|
||||
- Add a concise, localized explanation that Qdrant stores semantic-memory vectors for relevant-context retrieval. It is not a token compressor; token savings are indirect and depend on less irrelevant context being injected.
|
||||
- Add a configuration checklist covering a protected Qdrant endpoint, host/port, collection, embedding provider/model, matching vector dimensions, connection test, and search test.
|
||||
- Extend the authenticated Qdrant health route to inspect the configured collection without creating, updating, searching, or deleting points. Return the collection vector dimension and a clear state when the collection is absent or uses named vectors.
|
||||
- Show a pre-enable compatibility result in the Qdrant card. If the endpoint is reachable but the vector dimension cannot be determined from the selected embedding model, the UI must explain that the search test is the authoritative end-to-end validation. If dimensions differ, the UI must block enabling and explain how to create a compatible collection.
|
||||
- Keep the existing behavior that initial writes create a missing collection using the embedding dimension detected from the first successful embedding.
|
||||
|
||||
## User Flow
|
||||
|
||||
1. The user opens Dashboard > Memory > Engine and reads the purpose and prerequisites.
|
||||
2. The user enters Qdrant host, port, collection, optional API key, and an embedding provider/model with a configured provider credential.
|
||||
3. The user saves settings and clicks Test connection.
|
||||
4. The health result reports endpoint status and, for an existing collection, its vector dimensions and named-vector configuration.
|
||||
5. The user runs Test search. This generates an embedding through OmniRoute and proves that the model dimension matches the collection and that retrieval works.
|
||||
6. The Enable control remains unavailable after a known incompatibility; otherwise it follows the existing setting update path, which sets `memoryVectorStore` to `qdrant`.
|
||||
|
||||
## Collection Creation Guidance
|
||||
|
||||
The UI will provide copyable Qdrant REST guidance, using a placeholder dimension rather than assuming one for every model:
|
||||
|
||||
```json
|
||||
PUT /collections/<collection>
|
||||
{
|
||||
"vectors": { "size": <embedding-dimension>, "distance": "Cosine" }
|
||||
}
|
||||
```
|
||||
|
||||
For the audited server, the existing `omniroute_memory` collection has a named 2048-dimensional vector. It must be paired with the same 2048-dimensional embedding model that created it. The default `openai/text-embedding-3-small` emits 1536-dimensional vectors and therefore requires a separate 1536-dimensional collection.
|
||||
|
||||
## API Contract
|
||||
|
||||
`GET /api/settings/qdrant/health` will retain `{ ok, latencyMs, error? }` and add optional read-only metadata:
|
||||
|
||||
```ts
|
||||
{
|
||||
collection?: {
|
||||
exists: boolean;
|
||||
vectorSize?: number;
|
||||
vectorName?: string | null;
|
||||
};
|
||||
}
|
||||
```
|
||||
|
||||
The route must never expose Qdrant API keys. It must sanitize upstream error text before returning it.
|
||||
|
||||
## Error Handling
|
||||
|
||||
- A disconnected endpoint remains an error result, without changing settings.
|
||||
- A missing collection is guidance, not an error: OmniRoute creates it on the first successful Qdrant write.
|
||||
- A known dimension mismatch blocks enabling and tells the user to choose a matching model or a separate collection.
|
||||
- A model whose dimension cannot be determined does not claim compatibility; the user must run Test search.
|
||||
|
||||
## Testing
|
||||
|
||||
- Route tests cover health metadata for single-vector, named-vector, missing-collection, and sanitized upstream-error responses.
|
||||
- Component tests cover the purpose explanation, checklist, compatible/mismatch/missing collection states, and disabled enable action on a mismatch.
|
||||
- Existing Qdrant route and card tests remain green.
|
||||
@@ -413,6 +413,11 @@ export const EMBEDDING_PROVIDERS: Record<string, EmbeddingProvider> = {
|
||||
const EMBEDDING_PROVIDER_ALIASES: Record<string, string> = {
|
||||
jina: "jina-ai",
|
||||
voyage: "voyage-ai",
|
||||
// The dashboard stores LM Studio connections under the hyphenated provider
|
||||
// id "lm-studio" while the embedding registry keys the provider "lmstudio"
|
||||
// (#11233). Alias the dashboard id so "lm-studio/<model>" resolves instead
|
||||
// of failing with an unknown-provider 400.
|
||||
"lm-studio": "lmstudio",
|
||||
};
|
||||
|
||||
/** Family name used by clients; Jina's public SKU is omni-small. */
|
||||
|
||||
@@ -182,12 +182,8 @@ export async function handleEmbedding({
|
||||
)
|
||||
: [];
|
||||
const nativeModalities = [
|
||||
...(isJinaNativeEmbeddingInput(body.input)
|
||||
? collectJinaNativeModalities(body.input)
|
||||
: []),
|
||||
...(isGeminiNativeEmbeddingInput(body.input)
|
||||
? collectGeminiNativeModalities(body.input)
|
||||
: []),
|
||||
...(isJinaNativeEmbeddingInput(body.input) ? collectJinaNativeModalities(body.input) : []),
|
||||
...(isGeminiNativeEmbeddingInput(body.input) ? collectGeminiNativeModalities(body.input) : []),
|
||||
].filter((modality) => modality !== "text");
|
||||
if (structuredItems.length > 0 || nativeModalities.length > 0) {
|
||||
const supportedModalities = getEmbeddingModelModalities(providerConfig, model);
|
||||
@@ -266,7 +262,10 @@ export async function handleEmbedding({
|
||||
}
|
||||
|
||||
let upstreamUrl = providerConfig.baseUrl;
|
||||
if (provider === "ollama-local") {
|
||||
if (provider === "ollama-local" || provider === "lmstudio") {
|
||||
// Keyless local servers (#2824 ollama-local, #11233 lmstudio): honor the
|
||||
// configured connection's baseUrl when one was hydrated, and fall back to
|
||||
// the static localhost registry default otherwise.
|
||||
const configuredBaseUrl = credentials?.providerSpecificData?.baseUrl;
|
||||
const rawBaseUrl =
|
||||
typeof configuredBaseUrl === "string" && configuredBaseUrl.trim().length > 0
|
||||
@@ -277,11 +276,11 @@ export async function handleEmbedding({
|
||||
// (CodeQL js/polynomial-redos) since baseUrl is operator-configured
|
||||
// per-connection data. See open-sse/utils/urlSanitize.ts.
|
||||
const normalizedBaseUrl = stripTrailingSlashes(rawBaseUrl.trim());
|
||||
const ollamaHost = normalizedBaseUrl
|
||||
const localServerHost = normalizedBaseUrl
|
||||
.replace(/\/v1\/(?:chat\/completions|embeddings)$/i, "")
|
||||
.replace(/\/api\/chat$/i, "")
|
||||
.replace(/\/v1$/i, "");
|
||||
upstreamUrl = `${ollamaHost}/v1/embeddings`;
|
||||
upstreamUrl = `${localServerHost}/v1/embeddings`;
|
||||
}
|
||||
let normalizeProviderResponse:
|
||||
((data: Record<string, unknown>) => Record<string, unknown>) | null = null;
|
||||
@@ -321,10 +320,7 @@ export async function handleEmbedding({
|
||||
// become N embeddings. Native multimodal parts take the same path.
|
||||
const useGeminiNativeTransport =
|
||||
providerConfig.structuredInputProtocol === "gemini-embed-content" &&
|
||||
(isGeminiEmbedding2Family(model) ||
|
||||
canonicalStructured ||
|
||||
geminiNative ||
|
||||
jinaNative);
|
||||
(isGeminiEmbedding2Family(model) || canonicalStructured || geminiNative || jinaNative);
|
||||
|
||||
if (providerConfig.structuredInputProtocol === "jina-v1" && jinaNative && canonicalStructured) {
|
||||
try {
|
||||
@@ -462,13 +458,7 @@ export async function handleEmbedding({
|
||||
// best-effort.
|
||||
if (connectionId) {
|
||||
try {
|
||||
await markAccountUnavailable(
|
||||
connectionId,
|
||||
response.status,
|
||||
errorText,
|
||||
provider,
|
||||
model
|
||||
);
|
||||
await markAccountUnavailable(connectionId, response.status, errorText, provider, model);
|
||||
} catch {
|
||||
// swallow — the upstream error response takes priority
|
||||
}
|
||||
|
||||
@@ -1250,6 +1250,22 @@ export default function APIPageClient({ machineId }: Readonly<APIPageClientProps
|
||||
|
||||
return (
|
||||
<div className="flex flex-col gap-8">
|
||||
{/* Guided connection header (#11228): /v1 URL + test action lead; advanced protocols demoted */}
|
||||
<div className="flex flex-col gap-2">
|
||||
<h1 className="text-2xl font-bold">{t("title")}</h1>
|
||||
<p className="text-text-muted">{t("subtitle")}</p>
|
||||
<div className="flex items-center gap-3 mt-2">
|
||||
<code className="text-sm bg-card-subtle px-3 py-1 rounded-md text-text-main font-mono">
|
||||
{displayBaseUrl}/v1
|
||||
</code>
|
||||
<a href="#test" className="text-sm text-action font-medium hover:underline">
|
||||
{t("testEndpoint")}
|
||||
</a>
|
||||
</div>
|
||||
<div className="flex items-center gap-2 text-xs text-text-muted">
|
||||
<span>{t("advancedProtocols")}</span>
|
||||
</div>
|
||||
</div>
|
||||
<SegmentedControl
|
||||
options={ENDPOINT_TABS.map((tab) => ({ ...tab, label: t(tab.labelKey) }))}
|
||||
value={activeEndpointTab}
|
||||
@@ -2360,19 +2376,7 @@ function ProviderModelsModal({
|
||||
<div className="flex flex-col gap-1">
|
||||
{groupModels.map((m) => {
|
||||
const copyKey = `modal-${m.id}`;
|
||||
return (
|
||||
<div className="flex flex-col gap-4">
|
||||
<div className="flex flex-col gap-2 mb-2">
|
||||
<h1 className="text-2xl font-bold">{t("endpoint.title")}</h1>
|
||||
<p className="text-text-muted">{t("endpoint.subtitle")}</p>
|
||||
<div className="flex items-center gap-3 mt-2">
|
||||
<code className="text-sm bg-card-subtle px-3 py-1 rounded-md text-text-main font-mono">{useDisplayBaseUrl()}/v1</code>
|
||||
<a href="#test" className="text-sm text-action font-medium hover:underline">{t("endpoint.testEndpoint")}</a>
|
||||
</div>
|
||||
</div>
|
||||
<div className="flex items-center gap-2 text-xs text-text-muted">
|
||||
<span>{t("endpoint.advancedProtocols")}</span>
|
||||
</div>
|
||||
return (
|
||||
<div
|
||||
key={m.id}
|
||||
className="flex items-center gap-2 px-3 py-2 rounded-lg hover:bg-surface/60 group"
|
||||
|
||||
@@ -249,11 +249,14 @@ export async function createEmbeddingResponse(
|
||||
`[${provider}] All ${credentials.expiredCount || 1} connection(s) authentication expired — please reconnect in the dashboard`
|
||||
);
|
||||
}
|
||||
} else if (provider === "ollama-local") {
|
||||
// Ollama is keyless, but a configured connection can still provide a
|
||||
// custom local host. Hydrate that optional connection without imposing an
|
||||
// authentication requirement, then keep the static localhost default when
|
||||
// no connection exists.
|
||||
} else if (provider === "ollama-local" || provider === "lmstudio") {
|
||||
// Ollama and LM Studio are keyless, but a configured connection can still
|
||||
// provide a custom local host. Hydrate that optional connection without
|
||||
// imposing an authentication requirement, then keep the static localhost
|
||||
// default when no connection exists. getProviderCredentials("lmstudio")
|
||||
// resolves the dashboard's hyphenated "lm-studio" connection via the
|
||||
// provider search pool/alias (#11233); a selection or rate-limit failure
|
||||
// must not break the flow — proceed without credentials.
|
||||
const localCredentials = await getProviderCredentials(credentialsProviderId);
|
||||
if (
|
||||
localCredentials &&
|
||||
|
||||
144
tests/unit/lmstudio-connection-baseurl-11233.test.ts
Normal file
144
tests/unit/lmstudio-connection-baseurl-11233.test.ts
Normal file
@@ -0,0 +1,144 @@
|
||||
import test from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { mkdtempSync, rmSync } from "node:fs";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
|
||||
const TEST_DATA_DIR = mkdtempSync(join(tmpdir(), "omniroute-lmstudio-embedding-11233-"));
|
||||
process.env.DATA_DIR = TEST_DATA_DIR;
|
||||
|
||||
const { parseEmbeddingModel } = await import("../../open-sse/config/embeddingRegistry.ts");
|
||||
const { handleEmbedding } = await import("../../open-sse/handlers/embeddings.ts");
|
||||
const core = await import("../../src/lib/db/core.ts");
|
||||
const { createProviderConnection } = await import("../../src/lib/db/providers.ts");
|
||||
const { createEmbeddingResponse } = await import("../../src/lib/embeddings/service.ts");
|
||||
|
||||
test.after(() => {
|
||||
core.resetDbInstance();
|
||||
rmSync(TEST_DATA_DIR, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
// Issue #11233: the dashboard stores LM Studio connections under the provider
|
||||
// id "lm-studio" (hyphenated), but the embedding registry keys the provider as
|
||||
// "lmstudio" with no alias. Two symptoms resulted:
|
||||
// 1. "lm-studio/<model>" embedding requests failed with 400 unknown provider.
|
||||
// 2. "lmstudio/<model>" requests always hit the hardcoded localhost:1234
|
||||
// endpoint, ignoring the baseUrl of the configured connection.
|
||||
// The fix mirrors the ollama-local pattern from #2824/#9225: an embedding
|
||||
// provider alias plus optional (non-auth) connection hydration and the same
|
||||
// baseUrl normalization in the handler.
|
||||
|
||||
test("lm-studio model strings resolve to the lmstudio embedding provider", () => {
|
||||
assert.deepEqual(parseEmbeddingModel("lm-studio/nomic-embed-text"), {
|
||||
provider: "lmstudio",
|
||||
model: "nomic-embed-text",
|
||||
});
|
||||
});
|
||||
|
||||
test("lmstudio routes to the configured connection baseUrl", async () => {
|
||||
const originalFetch = globalThis.fetch;
|
||||
let capturedUrl: string | null = null;
|
||||
globalThis.fetch = async (url) => {
|
||||
capturedUrl = String(url);
|
||||
return new Response(
|
||||
JSON.stringify({
|
||||
data: [{ object: "embedding", embedding: [0.1, 0.2], index: 0 }],
|
||||
usage: { prompt_tokens: 2, total_tokens: 2 },
|
||||
}),
|
||||
{ status: 200, headers: { "content-type": "application/json" } }
|
||||
);
|
||||
};
|
||||
|
||||
try {
|
||||
const result = await handleEmbedding({
|
||||
body: { model: "lmstudio/nomic-embed-text", input: "hello" },
|
||||
resolvedProvider: {
|
||||
id: "lmstudio",
|
||||
baseUrl: "http://localhost:1234/v1/embeddings",
|
||||
authType: "none",
|
||||
authHeader: "none",
|
||||
models: [],
|
||||
},
|
||||
resolvedModel: "nomic-embed-text",
|
||||
credentials: {
|
||||
providerSpecificData: { baseUrl: "http://192.168.1.50:1234/v1" },
|
||||
},
|
||||
log: null,
|
||||
});
|
||||
|
||||
assert.equal(result.success, true);
|
||||
} finally {
|
||||
globalThis.fetch = originalFetch;
|
||||
}
|
||||
|
||||
assert.equal(capturedUrl, "http://192.168.1.50:1234/v1/embeddings");
|
||||
});
|
||||
|
||||
test("lmstudio keeps the static localhost default without credentials", async () => {
|
||||
const originalFetch = globalThis.fetch;
|
||||
let capturedUrl: string | null = null;
|
||||
globalThis.fetch = async (url) => {
|
||||
capturedUrl = String(url);
|
||||
return new Response(
|
||||
JSON.stringify({
|
||||
data: [{ object: "embedding", embedding: [0.3, 0.4], index: 0 }],
|
||||
usage: { prompt_tokens: 2, total_tokens: 2 },
|
||||
}),
|
||||
{ status: 200, headers: { "content-type": "application/json" } }
|
||||
);
|
||||
};
|
||||
|
||||
try {
|
||||
const result = await handleEmbedding({
|
||||
body: { model: "lmstudio/nomic-embed-text", input: "hello" },
|
||||
credentials: null,
|
||||
log: null,
|
||||
});
|
||||
|
||||
assert.equal(result.success, true);
|
||||
} finally {
|
||||
globalThis.fetch = originalFetch;
|
||||
}
|
||||
|
||||
assert.equal(capturedUrl, "http://localhost:1234/v1/embeddings");
|
||||
});
|
||||
|
||||
test("lmstudio service hydrates the lm-studio connection host without requiring a key", async () => {
|
||||
await createProviderConnection({
|
||||
provider: "lm-studio",
|
||||
authType: "none",
|
||||
name: "LAN LM Studio",
|
||||
isActive: true,
|
||||
providerSpecificData: { baseUrl: "http://10.20.0.60:1234/v1/" },
|
||||
});
|
||||
|
||||
const originalFetch = globalThis.fetch;
|
||||
let captured: { url: string; headers: Record<string, string> } | null = null;
|
||||
globalThis.fetch = async (url, options = {}) => {
|
||||
captured = {
|
||||
url: String(url),
|
||||
headers: (options.headers as Record<string, string>) || {},
|
||||
};
|
||||
return new Response(
|
||||
JSON.stringify({
|
||||
data: [{ object: "embedding", embedding: [0.5, 0.6], index: 0 }],
|
||||
usage: { prompt_tokens: 2, total_tokens: 2 },
|
||||
}),
|
||||
{ status: 200, headers: { "content-type": "application/json" } }
|
||||
);
|
||||
};
|
||||
|
||||
try {
|
||||
const response = await createEmbeddingResponse({
|
||||
model: "lm-studio/nomic-embed-text",
|
||||
input: "hello",
|
||||
});
|
||||
assert.equal(response.status, 200);
|
||||
} finally {
|
||||
globalThis.fetch = originalFetch;
|
||||
}
|
||||
|
||||
assert.ok(captured);
|
||||
assert.equal(captured.url, "http://10.20.0.60:1234/v1/embeddings");
|
||||
assert.equal(captured.headers.Authorization, undefined);
|
||||
});
|
||||
Reference in New Issue
Block a user