fix(models): honor MODELS_DEV_SYNC_ENABLED=0 over dashboard settings (#10299)

* fix(deps): bump nanoid, dompurify for 2 new Dependabot alerts (#189, #190)

Bumps: nanoid ^3.3.17 (was transitive, now overridden), dompurify ^3.4.13
(with monaco-editor scoped override). Closes Dependabot #189, #190.

Remaining #182-#188 (js-yaml + mermaid) already closed by #9651 merge —
awaiting Dependabot re-scan.

npm audit → 0 vulnerabilities.

* fix(repo): harden .gitignore to also ignore a _tasks symlink (/_tasks)

_tasks is a SEPARATE nested git repo (gitignored). The pattern _tasks/ (trailing
slash) ignores only a directory, not a SYMLINK named _tasks. A self-referential
_tasks symlink can slip in via git add -A and, once pulled, checkout materializes
it over the real _tasks repo (destroying plans/specs/hands-off). Anchored /_tasks
ignores the symlink too, preventing re-capture.

* Hide health-check excluded models from /v1/models catalog (#10026)

Mirror the request-time exclusion rule (provider_specific_data.excludedModels)
in the unified catalog builder: a model is hidden when its provider has
connections but none of them is eligible for it. Applied across the
PROVIDER_MODELS, synced, custom, alias-backed, and managed-fallback loops
so ghost models no longer appear as available.

Co-authored-by: ritheshcn25 <ritheshcn25@users.noreply.github.com>

* fix(models): memoize getModelsDevPricing (event loop / healthz) (#10055)

* fix(models): memoize getModelsDevPricing for /v1/models catalog

resolveCatalogPricing called getModelsDevPricing once per model while
building GET /v1/models. Each call re-scanned models_dev_pricing and
JSON.parsed every row (~10k SQL scans + multi-GB parse work), pegging
the event loop so even /healthz timed out (#9685, #10052).

Memoize the parsed map until saveModelsDevPricing / clearModelsDevPricing
and add a unit test for invalidation.

Signed-off-by: Ravi Tharuma <RaviTharuma@users.noreply.github.com>

* fix(db): invalidate modelsDevPricing cache on DB reset (#10055)

Copilot review fixes:
1. Register invalidateModelsDevPricingCache() with DB state reset system
   so resetDbInstance() clears the process-local memo, preventing stale
   pricing data from surviving across DB reset/restore operations.
2. Add test assertion verifying DB reset bypasses the memo (Copilot #10055).

The process-local memo at modelsDevSync.ts:204 caches getModelsDevPricing()
results until saveModelsDevPricing()/clearModelsDevPricing() to avoid
re-scanning all pricing rows on every /v1/models request. Without this hook,
backup restore and test DB resets would serve stale cached data from the
previous connection.

Tests: npm run test:unit:serial -- tests/unit/modelsDevSync-extended.test.ts

---------

Signed-off-by: Ravi Tharuma <RaviTharuma@users.noreply.github.com>
Co-authored-by: Ravi Tharuma <RaviTharuma@users.noreply.github.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>

* fix(models): honor MODELS_DEV_SYNC_ENABLED=0 over dashboard settings

The file header already advertised this env var but nothing read it.
When catalog/compression pin the event loop, the dashboard (same process)
cannot turn models.dev sync off. Let 0/false/off win over sqlite so an
operator can recover with env + restart. Skip getModelsDevPricing SQL
scans while the kill switch is set.

* fix(models): restore prettier formatting after base merge

Co-authored-by: diegosouzapw <diegosouza.pw@gmail.com>

* test(models): cover env kill switch during live settings updates

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>

---------

Signed-off-by: Ravi Tharuma <RaviTharuma@users.noreply.github.com>
Co-authored-by: diegosouzapw <diegosouzapw@users.noreply.github.com>
Co-authored-by: Diego Rodrigues de Sa e Souza <diegosouza.pw@outlook.com>
Co-authored-by: diegosouzapw <diegosouza.pw@gmail.com>
Co-authored-by: ritheshcn25 <rithesh.chandran@snb.ca>
Co-authored-by: ritheshcn25 <ritheshcn25@users.noreply.github.com>
Co-authored-by: Ravi Tharuma <RaviTharuma@users.noreply.github.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: adevwithpurpose <adevwithpurpose@users.noreply.github.com>
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
This commit is contained in:
Ravi Tharuma
2026-08-17 13:02:47 +02:00
committed by GitHub
parent b1a2ff6887
commit fbc67f1338
7 changed files with 159 additions and 38 deletions

View File

@@ -0,0 +1 @@
- **fix(models):** honor `MODELS_DEV_SYNC_ENABLED=0` as a hard kill switch over the dashboard setting so a wedged `/healthz` / UI can be recovered without HTTP (`src/lib/modelsDevSync.ts`)

View File

@@ -934,7 +934,7 @@ Chrome-driven session refresh (ARP) for the Adobe Firefly web provider (`open-ss
| Variable | Default | Source File | Description |
| ----------------------------------- | ------------- | ---------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `MODELS_DEV_SYNC_ENABLED` | `false` | `src/lib/modelsDevSync.ts` | Opt-in switch for the models.dev capability sync. Set to anything non-empty it wins over the `modelsDevSyncEnabled` setting (Dashboard > Settings > AI) in either direction, so a deployment can pin the sync on or off without depending on database state surviving a rebuild; unset, it defers to that setting. On for `1`, `true`, `yes` or `on` in any casing; any other value is off. |
| `MODELS_DEV_SYNC_ENABLED` | _(unset)_ | `src/lib/modelsDevSync.ts` | Hard override for models.dev pricing sync. Unset = honor Settings > AI (`modelsDevSyncEnabled`). `0`/`false`/`off`/`no` **wins over the DB** and skips both periodic sync and `getModelsDevPricing()` SQL/JSON scans (recovery when the dashboard is wedged on the same event loop). `1`/`true`/`on`/`yes` forces sync on. |
| `MODELS_DEV_SYNC_INTERVAL` | `86400` (24h) | `src/lib/modelsDevSync.ts` | Development-time model catalog sync interval in seconds. |
| `CONTEXT_WINDOW_RECONCILE_INTERVAL` | `86400` (24h) | `src/lib/contextWindowResolver.ts` | Interval (seconds) for the self-correcting context-window reconciler (5004): pins provider-declared windows from `/models` discovery as `auto:discovery` overrides when they diverge from the catalog. Set to `0` to disable. Reuses already-synced data (no new fetch); never overwrites `manual` overrides. |

View File

@@ -368,6 +368,18 @@ async function buildUnifiedModelsResponseCore(
return collected;
};
// Health-check exclusions (provider_specific_data.excludedModels) are enforced
// at request time in getProviderCredentials(); mirror the same rule in the
// catalog so ghost models do not appear as available. A model is hidden when
// the provider HAS connections but NONE of them is eligible for it.
const isExcludedByProviderConnections = (providerKey: string, modelId: string) => {
const providerId = aliasToProviderId[providerKey] || providerKey;
const alias = providerIdToAlias[providerId] || providerKey;
const providerConnections = getConnectionsForProvider(providerId, alias, providerKey);
if (providerConnections.length === 0) return false; // noAuth / no DB row: keep
return !hasEligibleConnectionForModel(providerConnections, modelId);
};
const providerSupportsModel = (providerKey: string, modelId: string) => {
const providerId = aliasToProviderId[providerKey] || providerKey;
const alias = providerIdToAlias[providerId] || providerKey;
@@ -793,6 +805,7 @@ async function buildUnifiedModelsResponseCore(
if (!providerSupportsModel(canonicalProviderId, model.id)) continue;
const aliasId = `${alias}/${model.id}`;
if (getModelIsHidden(canonicalProviderId, model.id)) continue;
if (isExcludedByProviderConnections(canonicalProviderId, model.id)) continue;
if (shouldHidePaid(canonicalProviderId, model.id, (model as { pricing?: unknown }).pricing))
continue;
@@ -913,6 +926,7 @@ async function buildUnifiedModelsResponseCore(
continue;
}
if (getModelIsHidden(providerId, sm.id)) continue;
if (isExcludedByProviderConnections(canonicalProviderId, sm.id)) continue;
// #6457: some upstream discovery catalogs (e.g. HuggingFace's live
// `/v1/models`) return image/diffusion models with no modality info,
// so `endpoints` below would default to ["chat"] and misrepresent
@@ -1307,6 +1321,7 @@ async function buildUnifiedModelsResponseCore(
continue;
if (model.isHidden === true) continue;
if (getModelIsHidden(canonicalProviderId, modelId)) continue;
if (isExcludedByProviderConnections(canonicalProviderId, modelId)) continue;
// #6328: apply hidePaidModels to user-defined custom rows too.
// Custom entries do not carry pricing, so shouldHidePaid() decides
// via FREE_MODEL_IDS_BY_PROVIDER — matches synced/PROVIDER_MODELS.
@@ -1487,6 +1502,7 @@ async function buildUnifiedModelsResponseCore(
}
if (getModelIsHidden(canonicalProviderId, modelId)) continue;
if (isExcludedByProviderConnections(canonicalProviderId, modelId)) continue;
// #6328: apply hidePaidModels to alias-backed rows too. Alias mappings
// point at providerKey/modelId with no pricing, so shouldHidePaid()
// decides via the FREE_MODEL_IDS_BY_PROVIDER catalog tier.
@@ -1560,6 +1576,7 @@ async function buildUnifiedModelsResponseCore(
const modelId = typeof model.id === "string" ? model.id : null;
if (!modelId) continue;
if (getModelIsHidden(canonicalProviderId, modelId)) continue;
if (isExcludedByProviderConnections(canonicalProviderId, modelId)) continue;
// #6328: apply hidePaidModels to managed-fallback rows too. Compatible
// provider fallbacks lack pricing; shouldHidePaid() decides via the
// FREE_MODEL_IDS_BY_PROVIDER catalog tier.

View File

@@ -374,18 +374,24 @@ async function applyModelsDevSyncSection(
currentSnapshot: RuntimeSettingsSnapshot,
force: boolean
) {
const { startPeriodicSync, stopPeriodicSync } = await import("@/lib/modelsDevSync");
const {
startPeriodicSync,
stopPeriodicSync,
isModelsDevSyncEnvDisabled,
isModelsDevSyncEnvForcedOn,
} = await import("@/lib/modelsDevSync");
const skipBackgroundSyncInTests =
(isAutomatedTestProcess() && process.env.OMNIROUTE_ENABLE_RUNTIME_BACKGROUND_TASKS !== "1") ||
isTruthyEnvFlag(process.env.OMNIROUTE_DISABLE_BACKGROUND_SERVICES);
if (skipBackgroundSyncInTests) {
if (skipBackgroundSyncInTests || isModelsDevSyncEnvDisabled()) {
stopPeriodicSync();
return;
}
const wasEnabled = previousSnapshot.modelsDevSyncEnabled === true;
const isEnabled = currentSnapshot.modelsDevSyncEnabled === true;
const isEnabled =
isModelsDevSyncEnvForcedOn() || currentSnapshot.modelsDevSyncEnabled === true;
const intervalChanged =
previousSnapshot.modelsDevSyncInterval !== currentSnapshot.modelsDevSyncInterval;

View File

@@ -14,12 +14,11 @@
* 3. LiteLLM sync (`pricing_synced` namespace)
* 4. Hardcoded defaults (`pricing.ts`)
*
* Opt-in, default off. Enabled either from Dashboard > Settings > AI or with
* MODELS_DEV_SYNC_ENABLED, which wins over that setting whenever it is set to
* anything non-empty, in either direction, so a deployment can pin the sync on
* or off regardless of what is stored. Unset or empty, it defers to the
* setting. On for "1", "true", "yes" or "on" in any casing; every other value
* is off.
* Settings UI (`modelsDevSyncEnabled`) controls the periodic sync by default.
* `MODELS_DEV_SYNC_ENABLED=0|false|off|no` is a hard kill switch: it wins over
* the DB setting so an operator can recover a wedged process (dashboard /
* /healthz frozen on the same event loop — #10052) without the UI. Unset =
* honor settings. `1|true|on|yes` forces sync on even if the setting is off.
*/
import { getDbInstance } from "./db/core";
@@ -76,12 +75,34 @@ interface SyncResult {
const MODELS_DEV_API_URL = "https://models.dev/api.json";
const TRUE_ENV_VALUES = new Set(["1", "true", "yes", "on"]);
const parsedInterval = parseInt(process.env.MODELS_DEV_SYNC_INTERVAL || "86400", 10);
const SYNC_INTERVAL_MS =
Number.isFinite(parsedInterval) && parsedInterval > 0 ? parsedInterval * 1000 : 86400 * 1000;
/** Parse MODELS_DEV_SYNC_ENABLED. Invalid / empty → unset (honor DB settings). */
export function readModelsDevSyncEnvFlag(
value: string | undefined = process.env.MODELS_DEV_SYNC_ENABLED
): "true" | "false" | "unset" {
if (value == null) return "unset";
const normalized = value.trim().toLowerCase();
if (normalized === "") return "unset";
if (normalized === "1" || normalized === "true" || normalized === "yes" || normalized === "on") {
return "true";
}
if (normalized === "0" || normalized === "false" || normalized === "no" || normalized === "off") {
return "false";
}
return "unset";
}
export function isModelsDevSyncEnvDisabled(): boolean {
return readModelsDevSyncEnvFlag() === "false";
}
export function isModelsDevSyncEnvForcedOn(): boolean {
return readModelsDevSyncEnvFlag() === "true";
}
// ─── Periodic sync state ─────────────────────────────────
let syncTimer: ReturnType<typeof setInterval> | null = null;
@@ -212,8 +233,15 @@ let pricingMemoVersion = -1; // -1: never equals a real cacheVersion (starts at
/**
* Read synced pricing from `models_dev_pricing` namespace.
* Results are memoized until `saveModelsDevPricing` / `clearModelsDevPricing`.
*/
export function getModelsDevPricing(): PricingByProvider {
// Kill switch: skip the SQL + JSON.parse scan entirely so a leftover
// models_dev_pricing namespace cannot pin the event loop (#9685 / #10052).
if (isModelsDevSyncEnvDisabled()) {
return {};
}
const currentVersion = getModelCatalogCacheVersion();
if (pricingMemo !== null && pricingMemoVersion === currentVersion) {
return pricingMemo;
@@ -737,35 +765,16 @@ export function getSyncStatus(): SyncStatus {
* Initialize models.dev sync if enabled.
*/
export async function initModelsDevSync(): Promise<void> {
if (isModelsDevSyncEnvDisabled()) {
console.log("[MODELS_DEV] Disabled (MODELS_DEV_SYNC_ENABLED=0)");
return;
}
const { getSettings } = await import("./localDb");
const settings = await getSettings();
// Until now the docblock above advertised MODELS_DEV_SYNC_ENABLED and nothing
// read it: the only control was the stored setting, so an operator following
// that line got silence whichever value they set. This makes the variable real.
//
// An explicit env value decides, in either direction, and only an unset or
// empty one defers to the setting. That means a deployment can pin the sync
// off from its compose file or unit even when a previous operator left the
// dashboard toggle on, which is the case a force-on-only variable cannot
// express and the reason for choosing this shape.
//
// It is worth being plain that this is a third resolution pattern rather than
// a reuse of an existing one, because the two in the tree solve different
// problems: shared/utils/featureFlags.ts::resolveFeatureFlag puts the DB
// override ABOVE the env var, so a deployment cannot override an operator's
// stored choice at all; db/ccDiscoveryAliases.ts::getCcAliasGlobalState reads
// only "1" and "true" and can force a flag ON, letting every other value
// including "false" fall through to the DB. Neither can turn a
// dashboard-enabled switch off from the environment. Following either one
// here would leave the variable unable to do the thing it is being added for.
const envValue = process.env.MODELS_DEV_SYNC_ENABLED?.trim();
const enabled = envValue
? TRUE_ENV_VALUES.has(envValue.toLowerCase())
: settings.modelsDevSyncEnabled === true;
if (!enabled) {
console.log("[MODELS_DEV] Disabled (enable via Settings > AI or MODELS_DEV_SYNC_ENABLED=true)");
if (!isModelsDevSyncEnvForcedOn() && settings.modelsDevSyncEnabled !== true) {
console.log("[MODELS_DEV] Disabled (enable via Settings > AI or MODELS_DEV_SYNC_ENABLED=1)");
return;
}

View File

@@ -30,6 +30,7 @@ const { getPayloadRulesConfig, resetPayloadRulesConfigForTests } =
await import("../../open-sse/services/payloadRules.ts");
const { getCacheControlSettings, invalidateCacheControlSettingsCache } =
await import("../../src/lib/cacheControlSettings.ts");
const { getSyncStatus, stopPeriodicSync } = await import("../../src/lib/modelsDevSync.ts");
async function resetStorage() {
stopRuntimeConfigHotReloadForTests();
@@ -128,6 +129,50 @@ test("updateSettings applies runtime settings incrementally without restart", as
assert.equal(getGeminiThoughtSignatureMode(), "enabled");
});
test("MODELS_DEV_SYNC_ENABLED=0 blocks a live settings update from starting the sync timer", async () => {
const previousEnvFlag = process.env.MODELS_DEV_SYNC_ENABLED;
const previousBackgroundTasks = process.env.OMNIROUTE_ENABLE_RUNTIME_BACKGROUND_TASKS;
process.env.MODELS_DEV_SYNC_ENABLED = "0";
process.env.OMNIROUTE_ENABLE_RUNTIME_BACKGROUND_TASKS = "1";
stopPeriodicSync();
try {
await applyRuntimeSettings(
{
...(await settingsDb.getSettings()),
modelsDevSyncEnabled: false,
modelsDevSyncInterval: 3_600_000,
},
{ force: true, source: "test:startup" }
);
const persistedSettings = await settingsDb.updateSettings({
modelsDevSyncEnabled: true,
modelsDevSyncInterval: 3_600_000,
});
assert.equal(
persistedSettings.modelsDevSyncEnabled,
true,
"the live settings update must persist the dashboard toggle"
);
assert.equal(
getSyncStatus().enabled,
false,
"MODELS_DEV_SYNC_ENABLED=0 must prevent a live settings update from starting the timer"
);
} finally {
stopPeriodicSync();
if (previousEnvFlag === undefined) delete process.env.MODELS_DEV_SYNC_ENABLED;
else process.env.MODELS_DEV_SYNC_ENABLED = previousEnvFlag;
if (previousBackgroundTasks === undefined) {
delete process.env.OMNIROUTE_ENABLE_RUNTIME_BACKGROUND_TASKS;
} else {
process.env.OMNIROUTE_ENABLE_RUNTIME_BACKGROUND_TASKS = previousBackgroundTasks;
}
}
});
test("hot-reload watcher picks up external sqlite changes via polling fallback", async () => {
await applyRuntimeSettings(await settingsDb.getSettings(), {
force: true,

View File

@@ -746,3 +746,46 @@ test("an unset MODELS_DEV_SYNC_ENABLED still defers to a stored setting of true"
else process.env.MODELS_DEV_SYNC_ENABLED = previous;
}
});
test("MODELS_DEV_SYNC_ENABLED=0 is a hard kill switch that wins over the setting", async () => {
const previous = process.env.MODELS_DEV_SYNC_ENABLED;
process.env.MODELS_DEV_SYNC_ENABLED = "0";
try {
const modelsDev = await importFresh("env-kill-switch-0");
mockFetchWith(MOCK_MODELS_DEV_DATA);
const pricing = modelsDev.transformModelsDevToPricing(MOCK_MODELS_DEV_DATA);
modelsDev.saveModelsDevPricing(pricing);
assert.deepEqual(
modelsDev.getModelsDevPricing(),
{},
"kill switch skips the SQL/JSON pricing scan entirely"
);
await settingsDb.updateSettings({
modelsDevSyncEnabled: true,
modelsDevSyncInterval: 15,
});
await modelsDev.initModelsDevSync();
assert.equal(
modelsDev.getSyncStatus().enabled,
false,
"kill switch must prevent the periodic sync even when the setting is on"
);
assert.equal(modelsDev.readModelsDevSyncEnvFlag("0"), "false");
assert.equal(modelsDev.readModelsDevSyncEnvFlag("false"), "false");
assert.equal(modelsDev.readModelsDevSyncEnvFlag("off"), "false");
assert.equal(modelsDev.readModelsDevSyncEnvFlag("no"), "false");
assert.equal(modelsDev.readModelsDevSyncEnvFlag(""), "unset");
assert.equal(modelsDev.readModelsDevSyncEnvFlag("maybe"), "unset");
assert.equal(modelsDev.readModelsDevSyncEnvFlag("1"), "true");
} finally {
if (previous === undefined) delete process.env.MODELS_DEV_SYNC_ENABLED;
else process.env.MODELS_DEV_SYNC_ENABLED = previous;
await settingsDb.updateSettings({
modelsDevSyncEnabled: false,
modelsDevSyncInterval: 15,
});
}
});