mirror of
https://github.com/diegosouzapw/OmniRoute.git
synced 2026-08-05 23:02:10 +03:00
models.dev catalogs Pixtral 12B under the short id `pixtral-12b` (attachment: true, image modality), while requests use the Mistral API alias `pixtral-12b-latest`. getSyncedCapabilityForResolved tried only exact / raw / static-spec-canonical ids, all of which miss the short form, so vision fell through to the #4071 model-id heuristic and `attachment` stayed null. Add a last-resort fallback that retries the synced lookup with a trailing `-latest` stripped, so synced metadata wins for these aliases. Models whose `-latest` id is stored verbatim (e.g. pixtral-large-latest) keep resolving directly. TDD: tests/unit/model-capabilities-mistral-vision-sync-4073.test.ts (RED 2/4 -> GREEN 4/4) seeds a synced fixture and asserts the verdict comes via the synced path (attachment === true/false), not the heuristic. The models.dev sync remains manual-only (no scheduled refresh) — documented as a separate follow-up.
This commit is contained in:
committed by
GitHub
parent
46c76103da
commit
a0e7e92e64
@@ -13,6 +13,7 @@ _In development — bullets added per PR; finalized at release._
|
||||
- **fix(sse): restore MCP / third-party tool names on the native Claude path (MCP dispatch broken in Claude Code)** — since 3.8.27, every MCP tool call routed through OmniRoute to a native Claude OAuth provider failed client-side with `Error: No such tool available: <PascalCaseName>`: tool schemas arrived fine but the streamed `tool_use.name` reached Claude Code in its cloaked form (e.g. `McpN8nMcpSearchWorkflows` instead of the registered `mcp__n8n-mcp__search_workflows`). The native-Claude tool-name cloak stashes its per-request alias→original map as a **non-enumerable** `_toolNameMap` on the request body; the request-inspector capture added in 3.8.27 rebuilds the captured body from its serialized form (`JSON.parse(JSON.stringify(...))`), which drops non-enumerable properties, so `finalBody._toolNameMap` was empty and the response-side un-cloak silently fell back to the static built-in map — never restoring dynamic MCP / snake_case names. Built-in tools (Bash/Read/…) were unaffected (static map); cross-format paths were unaffected (they attach the map enumerably). The provider-request capture now re-attaches the per-request map (kept non-enumerable, so it still never re-serializes upstream) when the captured copy lost it, restoring MCP tool dispatch. ([#4091](https://github.com/diegosouzapw/OmniRoute/issues/4091) — thanks @pedrotecinf, @NakHalal)
|
||||
- **fix(dashboard): Logs auto-refresh self-heals in embedded/proxied hosts that pin or mis-fire visibility** — a follow-up to #4054: the Request Logger still froze auto-refresh on some hosts (reported on 3.8.28 Docker, works on 3.8.24). #4054 made the initial visibility fail-open, but the pause is event-driven — a host that fires a one-shot `visibilitychange` → hidden and then keeps reporting `"hidden"` (or recovers without firing the event again) left the cached visibility flag stuck `false`, so the interval ticked but never polled (only the manual Refresh button worked). The poll tick now also re-checks the **live** `document.visibilityState`, and a **window `focus`** listener re-arms polling (a focused window is a reliable signal the page is actively viewed). A genuinely backgrounded browser tab still pauses (it reports `"hidden"` and never receives focus), preserving the #3109 network-saturation optimization. ([#4133](https://github.com/diegosouzapw/OmniRoute/issues/4133) — thanks @tjengbudi)
|
||||
- **fix(capabilities): unify vision model-id detection into one shared source** — three code paths kept independent, drifting vision-model lists, so the same model id could get up to three different verdicts. Two concrete bugs: lite compression's gate was missing pixtral / llava / qwen-vl / glm-4v / kimi-vl / mistral-medium-3, so it **stripped images for those real vision models and blinded them** (same class as #4071 / #4012); and the `/v1/models` list was too broad, flagging text models (`gemma`, bare `kimi` like `kimi-k2`) as vision. All three (`modelCapabilities` routing fallback, `/v1/models` listing, lite image-strip gate) now delegate to a single conservative source `src/shared/constants/visionModels.ts`, which also restores `glm-4v` / `gemini-3` coverage and keeps the #3328 MiniMax M3 carve-out. ([#4072](https://github.com/diegosouzapw/OmniRoute/issues/4072) — thanks @diego-anselmo)
|
||||
- **fix(capabilities): resolve models.dev-synced vision metadata for Mistral `-latest` aliases** — root cause behind the #4071 heuristic: `getResolvedModelCapabilities("mistral/pixtral-12b-latest").supportsVision` resolved `null` (vision came only from the #4071 model-id heuristic, with `attachment` still `null`) even though models.dev exposes the model as multimodal. Confirmed against the live models.dev API: it catalogs Pixtral 12B under the **short** id `pixtral-12b` (with `attachment: true`, `modalities.input: ["text","image"]`), while requests use the Mistral API alias `pixtral-12b-latest`. The synced lookup tried the exact / raw / static-spec-canonical ids — all of which miss the short form — so it fell through to the heuristic. `getSyncedCapabilityForResolved` now adds a last-resort fallback that retries with a trailing `-latest` stripped, so synced metadata (`attachment` / image modalities) wins for these aliases; models whose `-latest` id is stored verbatim (e.g. `pixtral-large-latest`) keep resolving directly. Note: the models.dev sync is currently manual-only (Settings → models.dev) with no scheduled refresh, so a fresh instance still relies on the #4071 heuristic until that sync runs — a periodic-refresh cadence is left as a separate follow-up. ([#4073](https://github.com/diegosouzapw/OmniRoute/issues/4073) — thanks @diego-anselmo)
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -192,6 +192,17 @@ function getStaticSpecCanonicalModelId(modelId: string | null, rawModel: string
|
||||
return null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Strip a trailing `-latest` alias suffix from a model id (#4073). Returns the
|
||||
* short id (`pixtral-12b-latest` → `pixtral-12b`) or `null` when there is no
|
||||
* `-latest` suffix to drop. Used only as a last-resort synced-lookup fallback.
|
||||
*/
|
||||
function stripLatestAlias(modelId: string | null): string | null {
|
||||
if (!modelId) return null;
|
||||
const stripped = modelId.replace(/-latest$/i, "");
|
||||
return stripped && stripped !== modelId ? stripped : null;
|
||||
}
|
||||
|
||||
function getSyncedCapabilityForResolved(
|
||||
provider: string | null,
|
||||
model: string | null,
|
||||
@@ -208,7 +219,27 @@ function getSyncedCapabilityForResolved(
|
||||
}
|
||||
|
||||
const canonical = getStaticSpecCanonicalModelId(model, rawModel);
|
||||
return canonical && canonical !== model ? getSyncedCapability(provider, canonical) : null;
|
||||
if (canonical && canonical !== model) {
|
||||
const byCanonical = getSyncedCapability(provider, canonical);
|
||||
if (byCanonical) return byCanonical;
|
||||
}
|
||||
|
||||
// #4073: models.dev catalogs some `-latest` aliases under their short id
|
||||
// (e.g. Mistral `pixtral-12b-latest` is stored as `pixtral-12b`). When every
|
||||
// exact lookup above misses, retry once with a trailing `-latest` stripped so
|
||||
// the synced metadata (`attachment` / image modalities) still wins over the
|
||||
// last-resort #4071 model-id heuristic. Only fires as a fallback, so models
|
||||
// whose `-latest` id IS stored verbatim (e.g. `pixtral-large-latest`) keep
|
||||
// resolving directly above.
|
||||
for (const candidate of [model, rawModel]) {
|
||||
const base = stripLatestAlias(candidate);
|
||||
if (base && base !== model && base !== rawModel) {
|
||||
const byAlias = getSyncedCapability(provider, base);
|
||||
if (byAlias) return byAlias;
|
||||
}
|
||||
}
|
||||
|
||||
return null;
|
||||
}
|
||||
|
||||
/**
|
||||
|
||||
148
tests/unit/model-capabilities-mistral-vision-sync-4073.test.ts
Normal file
148
tests/unit/model-capabilities-mistral-vision-sync-4073.test.ts
Normal file
@@ -0,0 +1,148 @@
|
||||
/**
|
||||
* #4073 — models.dev synced metadata must resolve for Mistral `-latest` aliases.
|
||||
*
|
||||
* Root cause (confirmed against the live models.dev API): models.dev catalogs
|
||||
* Mistral Pixtral 12B under the SHORT id `pixtral-12b` (with `attachment: true`,
|
||||
* `modalities.input: ["text","image"]`), while requests use the Mistral API
|
||||
* alias `pixtral-12b-latest`. The synced lookup in `getSyncedCapabilityForResolved`
|
||||
* tried the exact id, the raw id and the static-spec canonical id — all of which
|
||||
* miss for `pixtral-12b-latest` — so vision fell through to the #4071 model-id
|
||||
* heuristic and `attachment` stayed `null` (the symptom reported in #4073).
|
||||
*
|
||||
* The discriminator between "resolved via synced metadata" and "guessed via the
|
||||
* #4071 heuristic" is `attachment`: the synced path sets `attachment` from
|
||||
* `synced.attachment`; the heuristic only flips `supportsVision` and leaves
|
||||
* `attachment` null. So these tests assert on `attachment` to prove the synced
|
||||
* path — not the heuristic — produced the verdict.
|
||||
*
|
||||
* Other Mistral vision models already worked because models.dev keeps their
|
||||
* `-latest` id verbatim (e.g. `pixtral-large-latest`, `mistral-medium-latest`);
|
||||
* `pixtral-12b` is the one short-formed alias, hence the keying fix.
|
||||
*/
|
||||
import test from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import fs from "node:fs";
|
||||
import os from "node:os";
|
||||
import path from "node:path";
|
||||
|
||||
const TEST_DATA_DIR = fs.mkdtempSync(path.join(os.tmpdir(), "omniroute-mistral-vision-"));
|
||||
process.env.DATA_DIR = TEST_DATA_DIR;
|
||||
|
||||
const core = await import("../../src/lib/db/core.ts");
|
||||
const modelsDevSync = await import("../../src/lib/modelsDevSync.ts");
|
||||
const modelCapabilities = await import("../../src/lib/modelCapabilities.ts");
|
||||
|
||||
function buildCapability(overrides = {}) {
|
||||
return {
|
||||
tool_call: null,
|
||||
reasoning: null,
|
||||
attachment: null,
|
||||
structured_output: null,
|
||||
temperature: null,
|
||||
modalities_input: "[]",
|
||||
modalities_output: "[]",
|
||||
knowledge_cutoff: null,
|
||||
release_date: null,
|
||||
last_updated: null,
|
||||
status: null,
|
||||
family: null,
|
||||
open_weights: null,
|
||||
limit_context: null,
|
||||
limit_input: null,
|
||||
limit_output: null,
|
||||
interleaved_field: null,
|
||||
...overrides,
|
||||
};
|
||||
}
|
||||
|
||||
function resetStorage() {
|
||||
core.resetDbInstance();
|
||||
fs.rmSync(TEST_DATA_DIR, { recursive: true, force: true });
|
||||
fs.mkdirSync(TEST_DATA_DIR, { recursive: true });
|
||||
}
|
||||
|
||||
// Mirrors the real models.dev `mistral` provider keying observed on the live API:
|
||||
// short-form `pixtral-12b` (vision), verbatim `pixtral-large-latest` (vision),
|
||||
// short-form `ministral-8b` (text-only).
|
||||
function seedMistralCapabilities() {
|
||||
modelsDevSync.saveModelsDevCapabilities({
|
||||
mistral: {
|
||||
"pixtral-12b": buildCapability({
|
||||
attachment: true,
|
||||
modalities_input: JSON.stringify(["text", "image"]),
|
||||
modalities_output: JSON.stringify(["text"]),
|
||||
family: "pixtral",
|
||||
status: "stable",
|
||||
}),
|
||||
"pixtral-large-latest": buildCapability({
|
||||
attachment: true,
|
||||
modalities_input: JSON.stringify(["text", "image"]),
|
||||
modalities_output: JSON.stringify(["text"]),
|
||||
family: "pixtral",
|
||||
status: "stable",
|
||||
}),
|
||||
"ministral-8b": buildCapability({
|
||||
attachment: false,
|
||||
modalities_input: JSON.stringify(["text"]),
|
||||
modalities_output: JSON.stringify(["text"]),
|
||||
family: "ministral",
|
||||
status: "stable",
|
||||
}),
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
test.beforeEach(() => {
|
||||
resetStorage();
|
||||
});
|
||||
|
||||
test.after(() => {
|
||||
core.resetDbInstance();
|
||||
fs.rmSync(TEST_DATA_DIR, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
test("#4073 mistral/pixtral-12b-latest resolves vision via the synced `-latest` alias (not the heuristic)", () => {
|
||||
seedMistralCapabilities();
|
||||
|
||||
const latest = modelCapabilities.getResolvedModelCapabilities("mistral/pixtral-12b-latest");
|
||||
// attachment === true can ONLY come from the synced row keyed `pixtral-12b`.
|
||||
assert.equal(latest.attachment, true, "synced attachment must resolve via the stripped `-latest` alias");
|
||||
assert.equal(latest.supportsVision, true);
|
||||
});
|
||||
|
||||
test("#4073 exact-keyed `-latest` models still resolve directly (no regression)", () => {
|
||||
seedMistralCapabilities();
|
||||
|
||||
// pixtral-large-latest is stored verbatim — the direct lookup must keep working.
|
||||
const large = modelCapabilities.getResolvedModelCapabilities("mistral/pixtral-large-latest");
|
||||
assert.equal(large.attachment, true);
|
||||
assert.equal(large.supportsVision, true);
|
||||
|
||||
// And the bare short id resolves directly too.
|
||||
const bare = modelCapabilities.getResolvedModelCapabilities("mistral/pixtral-12b");
|
||||
assert.equal(bare.attachment, true);
|
||||
assert.equal(bare.supportsVision, true);
|
||||
});
|
||||
|
||||
test("#4073 the `-latest` strip carries the synced verdict for text-only models too", () => {
|
||||
seedMistralCapabilities();
|
||||
|
||||
// ministral-8b is text-only; the heuristic does not recognise it, so the only
|
||||
// way attachment is a concrete `false` (not null) is the synced row resolving
|
||||
// through the stripped alias. This proves the strip returns the row's real
|
||||
// verdict rather than fabricating a positive.
|
||||
const ministral = modelCapabilities.getResolvedModelCapabilities("mistral/ministral-8b-latest");
|
||||
assert.equal(ministral.attachment, false, "synced false must win, resolved via stripped alias");
|
||||
assert.equal(ministral.supportsVision, false);
|
||||
});
|
||||
|
||||
test("#4073 the `-latest` strip never fabricates a match for an unknown id", () => {
|
||||
seedMistralCapabilities();
|
||||
|
||||
// No synced row for `unknown-text-model` (stripped) nor its `-latest` form, and
|
||||
// the heuristic doesn't recognise it → attachment null, vision null. The strip
|
||||
// must not invent a capability out of nothing.
|
||||
const unknown = modelCapabilities.getResolvedModelCapabilities("mistral/unknown-text-model-latest");
|
||||
assert.equal(unknown.attachment, null);
|
||||
assert.equal(unknown.supportsVision, null);
|
||||
});
|
||||
Reference in New Issue
Block a user