fix(capabilities): resolve synced vision metadata for Mistral -latest aliases (#4073) (#4161)

models.dev catalogs Pixtral 12B under the short id `pixtral-12b` (attachment:
true, image modality), while requests use the Mistral API alias
`pixtral-12b-latest`. getSyncedCapabilityForResolved tried only exact / raw /
static-spec-canonical ids, all of which miss the short form, so vision fell
through to the #4071 model-id heuristic and `attachment` stayed null.

Add a last-resort fallback that retries the synced lookup with a trailing
`-latest` stripped, so synced metadata wins for these aliases. Models whose
`-latest` id is stored verbatim (e.g. pixtral-large-latest) keep resolving
directly. TDD: tests/unit/model-capabilities-mistral-vision-sync-4073.test.ts
(RED 2/4 -> GREEN 4/4) seeds a synced fixture and asserts the verdict comes via
the synced path (attachment === true/false), not the heuristic.

The models.dev sync remains manual-only (no scheduled refresh) — documented as a
separate follow-up.
This commit is contained in:
Diego Rodrigues de Sa e Souza
2026-06-18 05:00:52 -03:00
committed by GitHub
parent 46c76103da
commit a0e7e92e64
3 changed files with 181 additions and 1 deletions

View File

@@ -13,6 +13,7 @@ _In development — bullets added per PR; finalized at release._
- **fix(sse): restore MCP / third-party tool names on the native Claude path (MCP dispatch broken in Claude Code)** — since 3.8.27, every MCP tool call routed through OmniRoute to a native Claude OAuth provider failed client-side with `Error: No such tool available: <PascalCaseName>`: tool schemas arrived fine but the streamed `tool_use.name` reached Claude Code in its cloaked form (e.g. `McpN8nMcpSearchWorkflows` instead of the registered `mcp__n8n-mcp__search_workflows`). The native-Claude tool-name cloak stashes its per-request alias→original map as a **non-enumerable** `_toolNameMap` on the request body; the request-inspector capture added in 3.8.27 rebuilds the captured body from its serialized form (`JSON.parse(JSON.stringify(...))`), which drops non-enumerable properties, so `finalBody._toolNameMap` was empty and the response-side un-cloak silently fell back to the static built-in map — never restoring dynamic MCP / snake_case names. Built-in tools (Bash/Read/…) were unaffected (static map); cross-format paths were unaffected (they attach the map enumerably). The provider-request capture now re-attaches the per-request map (kept non-enumerable, so it still never re-serializes upstream) when the captured copy lost it, restoring MCP tool dispatch. ([#4091](https://github.com/diegosouzapw/OmniRoute/issues/4091) — thanks @pedrotecinf, @NakHalal)
- **fix(dashboard): Logs auto-refresh self-heals in embedded/proxied hosts that pin or mis-fire visibility** — a follow-up to #4054: the Request Logger still froze auto-refresh on some hosts (reported on 3.8.28 Docker, works on 3.8.24). #4054 made the initial visibility fail-open, but the pause is event-driven — a host that fires a one-shot `visibilitychange` → hidden and then keeps reporting `"hidden"` (or recovers without firing the event again) left the cached visibility flag stuck `false`, so the interval ticked but never polled (only the manual Refresh button worked). The poll tick now also re-checks the **live** `document.visibilityState`, and a **window `focus`** listener re-arms polling (a focused window is a reliable signal the page is actively viewed). A genuinely backgrounded browser tab still pauses (it reports `"hidden"` and never receives focus), preserving the #3109 network-saturation optimization. ([#4133](https://github.com/diegosouzapw/OmniRoute/issues/4133) — thanks @tjengbudi)
- **fix(capabilities): unify vision model-id detection into one shared source** — three code paths kept independent, drifting vision-model lists, so the same model id could get up to three different verdicts. Two concrete bugs: lite compression's gate was missing pixtral / llava / qwen-vl / glm-4v / kimi-vl / mistral-medium-3, so it **stripped images for those real vision models and blinded them** (same class as #4071 / #4012); and the `/v1/models` list was too broad, flagging text models (`gemma`, bare `kimi` like `kimi-k2`) as vision. All three (`modelCapabilities` routing fallback, `/v1/models` listing, lite image-strip gate) now delegate to a single conservative source `src/shared/constants/visionModels.ts`, which also restores `glm-4v` / `gemini-3` coverage and keeps the #3328 MiniMax M3 carve-out. ([#4072](https://github.com/diegosouzapw/OmniRoute/issues/4072) — thanks @diego-anselmo)
- **fix(capabilities): resolve models.dev-synced vision metadata for Mistral `-latest` aliases** — root cause behind the #4071 heuristic: `getResolvedModelCapabilities("mistral/pixtral-12b-latest").supportsVision` resolved `null` (vision came only from the #4071 model-id heuristic, with `attachment` still `null`) even though models.dev exposes the model as multimodal. Confirmed against the live models.dev API: it catalogs Pixtral 12B under the **short** id `pixtral-12b` (with `attachment: true`, `modalities.input: ["text","image"]`), while requests use the Mistral API alias `pixtral-12b-latest`. The synced lookup tried the exact / raw / static-spec-canonical ids — all of which miss the short form — so it fell through to the heuristic. `getSyncedCapabilityForResolved` now adds a last-resort fallback that retries with a trailing `-latest` stripped, so synced metadata (`attachment` / image modalities) wins for these aliases; models whose `-latest` id is stored verbatim (e.g. `pixtral-large-latest`) keep resolving directly. Note: the models.dev sync is currently manual-only (Settings → models.dev) with no scheduled refresh, so a fresh instance still relies on the #4071 heuristic until that sync runs — a periodic-refresh cadence is left as a separate follow-up. ([#4073](https://github.com/diegosouzapw/OmniRoute/issues/4073) — thanks @diego-anselmo)
---

View File

@@ -192,6 +192,17 @@ function getStaticSpecCanonicalModelId(modelId: string | null, rawModel: string
return null;
}
/**
* Strip a trailing `-latest` alias suffix from a model id (#4073). Returns the
* short id (`pixtral-12b-latest` → `pixtral-12b`) or `null` when there is no
* `-latest` suffix to drop. Used only as a last-resort synced-lookup fallback.
*/
function stripLatestAlias(modelId: string | null): string | null {
if (!modelId) return null;
const stripped = modelId.replace(/-latest$/i, "");
return stripped && stripped !== modelId ? stripped : null;
}
function getSyncedCapabilityForResolved(
provider: string | null,
model: string | null,
@@ -208,7 +219,27 @@ function getSyncedCapabilityForResolved(
}
const canonical = getStaticSpecCanonicalModelId(model, rawModel);
return canonical && canonical !== model ? getSyncedCapability(provider, canonical) : null;
if (canonical && canonical !== model) {
const byCanonical = getSyncedCapability(provider, canonical);
if (byCanonical) return byCanonical;
}
// #4073: models.dev catalogs some `-latest` aliases under their short id
// (e.g. Mistral `pixtral-12b-latest` is stored as `pixtral-12b`). When every
// exact lookup above misses, retry once with a trailing `-latest` stripped so
// the synced metadata (`attachment` / image modalities) still wins over the
// last-resort #4071 model-id heuristic. Only fires as a fallback, so models
// whose `-latest` id IS stored verbatim (e.g. `pixtral-large-latest`) keep
// resolving directly above.
for (const candidate of [model, rawModel]) {
const base = stripLatestAlias(candidate);
if (base && base !== model && base !== rawModel) {
const byAlias = getSyncedCapability(provider, base);
if (byAlias) return byAlias;
}
}
return null;
}
/**

View File

@@ -0,0 +1,148 @@
/**
* #4073 — models.dev synced metadata must resolve for Mistral `-latest` aliases.
*
* Root cause (confirmed against the live models.dev API): models.dev catalogs
* Mistral Pixtral 12B under the SHORT id `pixtral-12b` (with `attachment: true`,
* `modalities.input: ["text","image"]`), while requests use the Mistral API
* alias `pixtral-12b-latest`. The synced lookup in `getSyncedCapabilityForResolved`
* tried the exact id, the raw id and the static-spec canonical id — all of which
* miss for `pixtral-12b-latest` — so vision fell through to the #4071 model-id
* heuristic and `attachment` stayed `null` (the symptom reported in #4073).
*
* The discriminator between "resolved via synced metadata" and "guessed via the
* #4071 heuristic" is `attachment`: the synced path sets `attachment` from
* `synced.attachment`; the heuristic only flips `supportsVision` and leaves
* `attachment` null. So these tests assert on `attachment` to prove the synced
* path — not the heuristic — produced the verdict.
*
* Other Mistral vision models already worked because models.dev keeps their
* `-latest` id verbatim (e.g. `pixtral-large-latest`, `mistral-medium-latest`);
* `pixtral-12b` is the one short-formed alias, hence the keying fix.
*/
import test from "node:test";
import assert from "node:assert/strict";
import fs from "node:fs";
import os from "node:os";
import path from "node:path";
const TEST_DATA_DIR = fs.mkdtempSync(path.join(os.tmpdir(), "omniroute-mistral-vision-"));
process.env.DATA_DIR = TEST_DATA_DIR;
const core = await import("../../src/lib/db/core.ts");
const modelsDevSync = await import("../../src/lib/modelsDevSync.ts");
const modelCapabilities = await import("../../src/lib/modelCapabilities.ts");
function buildCapability(overrides = {}) {
return {
tool_call: null,
reasoning: null,
attachment: null,
structured_output: null,
temperature: null,
modalities_input: "[]",
modalities_output: "[]",
knowledge_cutoff: null,
release_date: null,
last_updated: null,
status: null,
family: null,
open_weights: null,
limit_context: null,
limit_input: null,
limit_output: null,
interleaved_field: null,
...overrides,
};
}
function resetStorage() {
core.resetDbInstance();
fs.rmSync(TEST_DATA_DIR, { recursive: true, force: true });
fs.mkdirSync(TEST_DATA_DIR, { recursive: true });
}
// Mirrors the real models.dev `mistral` provider keying observed on the live API:
// short-form `pixtral-12b` (vision), verbatim `pixtral-large-latest` (vision),
// short-form `ministral-8b` (text-only).
function seedMistralCapabilities() {
modelsDevSync.saveModelsDevCapabilities({
mistral: {
"pixtral-12b": buildCapability({
attachment: true,
modalities_input: JSON.stringify(["text", "image"]),
modalities_output: JSON.stringify(["text"]),
family: "pixtral",
status: "stable",
}),
"pixtral-large-latest": buildCapability({
attachment: true,
modalities_input: JSON.stringify(["text", "image"]),
modalities_output: JSON.stringify(["text"]),
family: "pixtral",
status: "stable",
}),
"ministral-8b": buildCapability({
attachment: false,
modalities_input: JSON.stringify(["text"]),
modalities_output: JSON.stringify(["text"]),
family: "ministral",
status: "stable",
}),
},
});
}
test.beforeEach(() => {
resetStorage();
});
test.after(() => {
core.resetDbInstance();
fs.rmSync(TEST_DATA_DIR, { recursive: true, force: true });
});
test("#4073 mistral/pixtral-12b-latest resolves vision via the synced `-latest` alias (not the heuristic)", () => {
seedMistralCapabilities();
const latest = modelCapabilities.getResolvedModelCapabilities("mistral/pixtral-12b-latest");
// attachment === true can ONLY come from the synced row keyed `pixtral-12b`.
assert.equal(latest.attachment, true, "synced attachment must resolve via the stripped `-latest` alias");
assert.equal(latest.supportsVision, true);
});
test("#4073 exact-keyed `-latest` models still resolve directly (no regression)", () => {
seedMistralCapabilities();
// pixtral-large-latest is stored verbatim — the direct lookup must keep working.
const large = modelCapabilities.getResolvedModelCapabilities("mistral/pixtral-large-latest");
assert.equal(large.attachment, true);
assert.equal(large.supportsVision, true);
// And the bare short id resolves directly too.
const bare = modelCapabilities.getResolvedModelCapabilities("mistral/pixtral-12b");
assert.equal(bare.attachment, true);
assert.equal(bare.supportsVision, true);
});
test("#4073 the `-latest` strip carries the synced verdict for text-only models too", () => {
seedMistralCapabilities();
// ministral-8b is text-only; the heuristic does not recognise it, so the only
// way attachment is a concrete `false` (not null) is the synced row resolving
// through the stripped alias. This proves the strip returns the row's real
// verdict rather than fabricating a positive.
const ministral = modelCapabilities.getResolvedModelCapabilities("mistral/ministral-8b-latest");
assert.equal(ministral.attachment, false, "synced false must win, resolved via stripped alias");
assert.equal(ministral.supportsVision, false);
});
test("#4073 the `-latest` strip never fabricates a match for an unknown id", () => {
seedMistralCapabilities();
// No synced row for `unknown-text-model` (stripped) nor its `-latest` form, and
// the heuristic doesn't recognise it → attachment null, vision null. The strip
// must not invent a capability out of nothing.
const unknown = modelCapabilities.getResolvedModelCapabilities("mistral/unknown-text-model-latest");
assert.equal(unknown.attachment, null);
assert.equal(unknown.supportsVision, null);
});