mirror of
https://github.com/diegosouzapw/OmniRoute.git
synced 2026-09-20 05:42:19 +03:00
chore(lifecycle): gate DEFAULT_DEGRADATION_MAP against retired ids (#12535)
Third hand-maintained table naming model ids and the only one outside the retired-model gate — extending `check-model-lifecycle.mjs` to cover it is the durable fix, and the three retired rows it flushed out were already dead code behind the 410 `model_shutdown` answer. --- Validated in one consolidated worktree cut from `release/v3.8.51`, boarded with the rest of this batch — zero conflicts between the 19 PRs. - `typecheck:core` clean; `check:dashboard-typecheck` OK (206 pre-existing, all within the frozen baseline); `check:changelog-integrity` OK - complexity 2802 / baseline 3218 and cognitive-complexity 1267 / baseline 1437 — both under baseline - 226 of 228 focused assertions green across the batch's 23 test files. The 2 remaining belong to #12551, which is held separately. Two batch-owned defects were found and fixed in flight, both pure base drift: `173_xp_action_counts.sql` collided with `173_call_logs_video_content_removed.sql` (renumbered to 176 on #12651 — it aborted every DB open, which is what 53 of the first run's failures were), and the feature-flag catalog was missing the `SERVER_OWNED_TOOL_LOOP_ENABLED` row the base gained after #12552 was written. ⚠️ base-red inherited: #12732 — `Docs Gates`, `Merge integrity`, `No new ESLint warnings`, `Unit Tests fast-path` and `Fast Quality Gates` reproduce on the pure tip (provider count 356 vs the 358 the modules define, SKILL.md drift, and `open-sse/utils/stream.ts` at 3115 > frozen 3098, which this batch does not touch). Thanks @pacocartones — the `file:line` citations and the explicit out-of-scope notes on every one of these made a 19-PR batch reviewable in one pass.
This commit is contained in:
Paco Cartones
@@ -0,0 +1,7 @@
|
||||
- **chore(lifecycle):** `check:model-lifecycle` now also diffs `DEFAULT_DEGRADATION_MAP`
|
||||
(the background-task redirect table) against the vendor lifecycle snapshot, refusing a
|
||||
retired id as source or target, with a table-driven unit test beside it. Three rows
|
||||
whose source the vendor had retired — `claude-sonnet-4-20250514`, `gemini-3-pro-preview`
|
||||
and `gpt-5.1-codex` (whose target `gpt-5.1-codex-mini` is retired too) — were dead code,
|
||||
since `checkLifecycle` answers 410 before the redirect runs; they are dropped
|
||||
(#12535 — thanks @pacocartones)
|
||||
@@ -57,34 +57,34 @@ assertion weakening and other masking remain owned by the independently blocking
|
||||
|
||||
Runs on every PR to `main`. Blocks merge on failure.
|
||||
|
||||
| Script (`npm run ...`) | Validates | Blocking |
|
||||
| --------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------- |
|
||||
| `check:node-runtime` | Node.js version is within the supported range | Yes |
|
||||
| `check:cycles` | Circular imports — all `src/` + `open-sse/` modules | Yes |
|
||||
| `check:route-validation:t06` | Zod schemas present on all routes (Tier 6 policy) | Yes |
|
||||
| `check:any-budget:t11` | `@ts-expect-error // any` count does not exceed budget (Tier 11 catraca) | Yes |
|
||||
| `check:provider-consistency` | Every provider in `providers.ts` has a matching entry in `providerRegistry.ts` (and vice-versa, within the allowlist) | Yes |
|
||||
| `check:model-lifecycle` | The two hand-maintained routing tables do not point at retired models (#11503): `FITNESS_TABLE` (`taskFitness.ts`) scores no routable retired id, every `BUILT_IN_ALIASES` target is a live catalog model, and every retired id the catalog still routes is either forwarded or listed in `allowedRetiredInCatalog`. Offline — compares against the vendor snapshot `config/quality/model-lifecycle.json`, refreshed by hand with `npm run quality:refresh-model-lifecycle` (network; not wired into CI). `allowedRetiredInCatalog` is a burn-down ratchet: add an entry only with a tracking issue. | Yes |
|
||||
| `check:fetch-targets` | Every `fetch("/api/...")` in client-side `src/` resolves to a real `route.ts` | Yes |
|
||||
| `check:deps` | All `npm install`-able deps across every `package.json` in the repo are in `dependency-allowlist.json`; new unpinned or slopsquatted packages flagged | Yes |
|
||||
| `audit:deps` | `npm audit` (root + electron) — no high/critical advisories (overlaps osv `check:vuln-ratchet`; see Rationalization Backlog) | Yes |
|
||||
| `check:lockfile` | `package-lock.json` integrity — https registry, integrity hashes, no host overrides | Yes |
|
||||
| `check:licenses` | SPDX license allowlist for production dependencies | Yes |
|
||||
| `check:tracked-artifacts` | No build artifacts / committed `node_modules` symlinks (also runs in husky pre-commit; pre-push is intentionally light — #6716) | Yes |
|
||||
| `check:file-size` | No source file exceeds the per-extension cap (ratchet: frozen large files in `frozen` list) | Yes |
|
||||
| `check:error-helper` | Error responses in executors/handlers use `buildErrorBody()` / `sanitizeErrorMessage()` (Hard Rule #12) | Yes |
|
||||
| `check:migration-numbering` | Migration SQL files are sequentially numbered, no gaps or duplicates | Yes |
|
||||
| `check:public-creds` | No literal OAuth `client_id`/`client_secret` or Firebase Web keys outside `publicCreds.ts` (Hard Rule #11) | Yes |
|
||||
| `check:db-rules` | No raw SQL outside `src/lib/db/` modules; no barrel-imports from `localDb.ts` (Hard Rules #2/#5) | Yes |
|
||||
| `check:known-symbols` | Provider executors, routing strategies, and translators registered in their dispatch tables match the files on disk — no orphaned or undeclared symbols | Yes |
|
||||
| `check:route-guard-membership` | Every route that spawns a child process is classified by `isLocalOnlyPath()` (Hard Rules #15/#17) | Yes |
|
||||
| `check:test-discovery` | Every `*.test.ts` / `*.spec.ts` file in the repo is collected by at least one test runner (ratchet: orphan list in `test-discovery-baseline.json` can only shrink) | Yes |
|
||||
| `check:agent-skills-sync` | Generated agent-skills artifacts match their source catalog (no drift) |
|
||||
| `check:provider-asset-provenance` | Provider logos/assets carry a recorded provenance entry |
|
||||
| `lint:json` | JSON config files parse and satisfy the repo lint rules |
|
||||
| `typecheck:core` | TypeScript compilation without errors (advisory warnings only) | Yes |
|
||||
| `typecheck:noimplicit:core` | Strict `noImplicitAny` — forward-looking; many pre-existing call sites still need annotations | **Advisory** (`continue-on-error: true`) |
|
||||
| `check:dashboard-typecheck` | `tsc` scoped to `src/app/(dashboard)/**` (#7033) — `typecheck:core`'s curated 27-file allowlist does not include any dashboard TSX, and `next build` never type-checks it either (`next.config.mjs` sets `ignoreBuildErrors: true`), so orphaned-identifier regressions there (#6625/#6909) were invisible to CI. Diffs against a frozen per-file/per-TS-code count baseline (`config/quality/dashboard-typecheck-baseline.json`, same stale-enforcement pattern as `check:known-symbols`) — only NEW errors beyond the baselined count fail the gate; ratchet down with `--update` when a pre-existing error is fixed. | Yes |
|
||||
| Script (`npm run ...`) | Validates | Blocking |
|
||||
| --------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------- |
|
||||
| `check:node-runtime` | Node.js version is within the supported range | Yes |
|
||||
| `check:cycles` | Circular imports — all `src/` + `open-sse/` modules | Yes |
|
||||
| `check:route-validation:t06` | Zod schemas present on all routes (Tier 6 policy) | Yes |
|
||||
| `check:any-budget:t11` | `@ts-expect-error // any` count does not exceed budget (Tier 11 catraca) | Yes |
|
||||
| `check:provider-consistency` | Every provider in `providers.ts` has a matching entry in `providerRegistry.ts` (and vice-versa, within the allowlist) | Yes |
|
||||
| `check:model-lifecycle` | The three hand-maintained routing tables stay consistent with the checked-in lifecycle snapshot (#11503): `FITNESS_TABLE` (`taskFitness.ts`) scores no retired id that `REGISTRY` can route; every `BUILT_IN_ALIASES` target is present in `REGISTRY` and absent from the retired-id snapshot; every retired id still in `REGISTRY` is forwarded or listed in `allowedRetiredInCatalog`; and no `DEFAULT_DEGRADATION_MAP` source or target appears retired in that snapshot. This does not prove that a model is currently served by a live upstream. Offline — compares against `config/quality/model-lifecycle.json`, refreshed by hand with `npm run quality:refresh-model-lifecycle` (network; not wired into CI). `allowedRetiredInCatalog` is a burn-down ratchet: add an entry only with a tracking issue. | Yes |
|
||||
| `check:fetch-targets` | Every `fetch("/api/...")` in client-side `src/` resolves to a real `route.ts` | Yes |
|
||||
| `check:deps` | All `npm install`-able deps across every `package.json` in the repo are in `dependency-allowlist.json`; new unpinned or slopsquatted packages flagged | Yes |
|
||||
| `audit:deps` | `npm audit` (root + electron) — no high/critical advisories (overlaps osv `check:vuln-ratchet`; see Rationalization Backlog) | Yes |
|
||||
| `check:lockfile` | `package-lock.json` integrity — https registry, integrity hashes, no host overrides | Yes |
|
||||
| `check:licenses` | SPDX license allowlist for production dependencies | Yes |
|
||||
| `check:tracked-artifacts` | No build artifacts / committed `node_modules` symlinks (also runs in husky pre-commit; pre-push is intentionally light — #6716) | Yes |
|
||||
| `check:file-size` | No source file exceeds the per-extension cap (ratchet: frozen large files in `frozen` list) | Yes |
|
||||
| `check:error-helper` | Error responses in executors/handlers use `buildErrorBody()` / `sanitizeErrorMessage()` (Hard Rule #12) | Yes |
|
||||
| `check:migration-numbering` | Migration SQL files are sequentially numbered, no gaps or duplicates | Yes |
|
||||
| `check:public-creds` | No literal OAuth `client_id`/`client_secret` or Firebase Web keys outside `publicCreds.ts` (Hard Rule #11) | Yes |
|
||||
| `check:db-rules` | No raw SQL outside `src/lib/db/` modules; no barrel-imports from `localDb.ts` (Hard Rules #2/#5) | Yes |
|
||||
| `check:known-symbols` | Provider executors, routing strategies, and translators registered in their dispatch tables match the files on disk — no orphaned or undeclared symbols | Yes |
|
||||
| `check:route-guard-membership` | Every route that spawns a child process is classified by `isLocalOnlyPath()` (Hard Rules #15/#17) | Yes |
|
||||
| `check:test-discovery` | Every `*.test.ts` / `*.spec.ts` file in the repo is collected by at least one test runner (ratchet: orphan list in `test-discovery-baseline.json` can only shrink) | Yes |
|
||||
| `check:agent-skills-sync` | Generated agent-skills artifacts match their source catalog (no drift) |
|
||||
| `check:provider-asset-provenance` | Provider logos/assets carry a recorded provenance entry |
|
||||
| `lint:json` | JSON config files parse and satisfy the repo lint rules |
|
||||
| `typecheck:core` | TypeScript compilation without errors (advisory warnings only) | Yes |
|
||||
| `typecheck:noimplicit:core` | Strict `noImplicitAny` — forward-looking; many pre-existing call sites still need annotations | **Advisory** (`continue-on-error: true`) |
|
||||
| `check:dashboard-typecheck` | `tsc` scoped to `src/app/(dashboard)/**` (#7033) — `typecheck:core`'s curated 27-file allowlist does not include any dashboard TSX, and `next build` never type-checks it either (`next.config.mjs` sets `ignoreBuildErrors: true`), so orphaned-identifier regressions there (#6625/#6909) were invisible to CI. Diffs against a frozen per-file/per-TS-code count baseline (`config/quality/dashboard-typecheck-baseline.json`, same stale-enforcement pattern as `check:known-symbols`) — only NEW errors beyond the baselined count fail the gate; ratchet down with `--update` when a pre-existing error is fixed. | Yes |
|
||||
|
||||
### Job: `quality-gate`
|
||||
|
||||
|
||||
@@ -45,22 +45,24 @@ const DEFAULT_DETECTION_PATTERNS = [
|
||||
"label this",
|
||||
];
|
||||
|
||||
// Every source and target must be absent from the retired-id snapshot: a retired source is
|
||||
// a dead row (checkLifecycle answers 410 before the redirect runs), while a retired target
|
||||
// is normally rejected with 410 when lifecycle validation runs again after the redirect
|
||||
// (unless alias resolution maps it to an accepted id). `npm run check:model-lifecycle`
|
||||
// diffs this map against config/quality/model-lifecycle.json.
|
||||
const DEFAULT_DEGRADATION_MAP: Record<string, string> = {
|
||||
// Premium → Cheap alternatives
|
||||
"claude-opus-4-6": "gemini-3-flash",
|
||||
"claude-opus-4-6-thinking": "gemini-3-flash",
|
||||
"claude-opus-4-5-20251101": "gemini-3-flash",
|
||||
"claude-sonnet-4-5-20250929": "gemini-3-flash",
|
||||
"claude-sonnet-4-20250514": "gemini-3-flash",
|
||||
"claude-sonnet-4": "gemini-3-flash",
|
||||
"gemini-3.1-pro": "gemini-3-flash",
|
||||
"gemini-3.1-pro-high": "gemini-3-flash",
|
||||
"gemini-3-pro-preview": "gemini-3-flash-preview",
|
||||
"gemini-2.5-pro": "gemini-3-flash",
|
||||
"gpt-4o": "gpt-4o-mini",
|
||||
"gpt-5": "gpt-5-mini",
|
||||
"gpt-5.1": "gpt-5-mini",
|
||||
"gpt-5.1-codex": "gpt-5.1-codex-mini",
|
||||
};
|
||||
|
||||
// ── State ───────────────────────────────────────────────────────────────────
|
||||
|
||||
@@ -1,19 +1,25 @@
|
||||
#!/usr/bin/env node
|
||||
// scripts/check/check-model-lifecycle.mjs
|
||||
// Gate anti-drift (#11503): as duas tabelas mantidas à mão que decidem roteamento —
|
||||
// Gate anti-drift (#11503): as três tabelas mantidas à mão que decidem roteamento —
|
||||
// FITNESS_TABLE (open-sse/services/autoCombo/taskFitness.ts, camada 4 do task fitness) e
|
||||
// BUILT_IN_ALIASES (open-sse/services/modelDeprecation.ts, reescreve `body.model` em toda
|
||||
// request) — apodrecem em silêncio quando o fornecedor aposenta um modelo. Em
|
||||
// request), além de DEFAULT_DEGRADATION_MAP (backgroundTaskDetector.ts) — apodrecem em
|
||||
// silêncio quando o fornecedor aposenta um modelo. Em
|
||||
// release/v3.8.51 o resultado foi uma inversão de ranking (modelo morto 0.98 vs flagship
|
||||
// vivo 0.50) e aliases que garantiam 404. Este gate compara as duas contra o snapshot de
|
||||
// vivo 0.50) e aliases apontando para ids obsoletos. Este gate compara as três contra o snapshot de
|
||||
// ciclo de vida em config/quality/model-lifecycle.json (sem rede; regenerar com
|
||||
// `npm run quality:refresh-model-lifecycle`).
|
||||
//
|
||||
// Três checagens, todas somadas antes do exit — nenhuma aborta as outras:
|
||||
// Quatro checagens, todas somadas antes do exit — nenhuma aborta as outras:
|
||||
// (a) nenhum padrão do FITNESS_TABLE pontua um id aposentado que o catálogo roteia;
|
||||
// (b) nenhum alvo de BUILT_IN_ALIASES está aposentado ou ausente do catálogo;
|
||||
// (c) todo id aposentado ainda presente no REGISTRY tem encaminhamento em
|
||||
// BUILT_IN_ALIASES ou consta em `allowedRetiredInCatalog` (a catraca a queimar).
|
||||
// BUILT_IN_ALIASES ou consta em `allowedRetiredInCatalog` (a catraca a queimar);
|
||||
// (d) nenhuma linha de DEFAULT_DEGRADATION_MAP (open-sse/services/backgroundTaskDetector.ts)
|
||||
// tem origem ou destino aposentado. A origem aposentada é linha morta: checkLifecycle
|
||||
// devolve 410 antes de resolveBackgroundTaskRedirect rodar. O destino aposentado é o
|
||||
// normalmente rejeitado com 410 quando o ciclo de vida é validado novamente após o
|
||||
// redirecionamento; a resolução de alias ainda pode convertê-lo em um id aceito.
|
||||
//
|
||||
// (a) é deliberadamente restrita aos ids ROTEÁVEIS: linhas versionadas legítimas como
|
||||
// `gpt-4o` também casam com ids aposentados que o catálogo nunca serviu
|
||||
@@ -88,6 +94,22 @@ export function findUnforwardedRetiredIds(routableRetiredIds, aliases, allowlist
|
||||
.map((id) => `${id} is retired but still routable with no BUILT_IN_ALIASES forward`);
|
||||
}
|
||||
|
||||
/** (d) Linhas de DEFAULT_DEGRADATION_MAP com origem ou destino aposentado. */
|
||||
export function findRetiredDegradationRows(degradationMap, retiredIds) {
|
||||
const violations = [];
|
||||
for (const [source, target] of Object.entries(degradationMap ?? {})) {
|
||||
if (isRetiredId(source, retiredIds)) {
|
||||
violations.push(
|
||||
`${source} → ${target} (the vendor has retired the source id; checkLifecycle rejects it before the redirect runs)`
|
||||
);
|
||||
}
|
||||
if (isRetiredId(target, retiredIds)) {
|
||||
violations.push(`${source} → ${target} (the vendor has retired the target id)`);
|
||||
}
|
||||
}
|
||||
return violations;
|
||||
}
|
||||
|
||||
export function readSnapshot(snapshotPath = SNAPSHOT_PATH) {
|
||||
const snapshot = JSON.parse(fs.readFileSync(snapshotPath, "utf8"));
|
||||
const retiredIds = new Set(
|
||||
@@ -102,12 +124,18 @@ async function loadProductionTables() {
|
||||
// Nenhum gate pode migrar o banco do operador: taskFitness.ts importa src/lib/db/core.ts,
|
||||
// então DATA_DIR aponta para um diretório descartável ANTES do import dinâmico.
|
||||
process.env.DATA_DIR = fs.mkdtempSync(path.join(os.tmpdir(), "omniroute-lifecycle-gate-"));
|
||||
const [{ REGISTRY }, { getStaticFitnessTableScore }, { getBuiltInAliases }] = await Promise.all([
|
||||
const [
|
||||
{ REGISTRY },
|
||||
{ getStaticFitnessTableScore },
|
||||
{ getBuiltInAliases },
|
||||
{ getDefaultDegradationMap },
|
||||
] = await Promise.all([
|
||||
import(pathToFileURL(path.join(ROOT, "open-sse/config/providers/index.ts")).href),
|
||||
import(pathToFileURL(path.join(ROOT, "open-sse/services/autoCombo/taskFitness.ts")).href),
|
||||
import(pathToFileURL(path.join(ROOT, "open-sse/services/modelDeprecation.ts")).href),
|
||||
import(pathToFileURL(path.join(ROOT, "open-sse/services/backgroundTaskDetector.ts")).href),
|
||||
]);
|
||||
return { REGISTRY, getStaticFitnessTableScore, getBuiltInAliases };
|
||||
return { REGISTRY, getStaticFitnessTableScore, getBuiltInAliases, getDefaultDegradationMap };
|
||||
}
|
||||
|
||||
function report(label, violations, hint) {
|
||||
@@ -125,11 +153,13 @@ function report(label, violations, hint) {
|
||||
|
||||
async function main() {
|
||||
const { snapshot, retiredIds } = readSnapshot();
|
||||
const { REGISTRY, getStaticFitnessTableScore, getBuiltInAliases } = await loadProductionTables();
|
||||
const { REGISTRY, getStaticFitnessTableScore, getBuiltInAliases, getDefaultDegradationMap } =
|
||||
await loadProductionTables();
|
||||
|
||||
const catalogIds = collectCatalogIds(REGISTRY);
|
||||
const routableRetired = catalogIds.filter((id) => isRetiredId(id, retiredIds)).sort();
|
||||
const aliases = getBuiltInAliases();
|
||||
const degradationMap = getDefaultDegradationMap();
|
||||
|
||||
let failures = 0;
|
||||
failures += report(
|
||||
@@ -138,7 +168,7 @@ async function main() {
|
||||
"drop the row from FITNESS_TABLE in open-sse/services/autoCombo/taskFitness.ts, or replace it with the versioned id of the live successor."
|
||||
);
|
||||
failures += report(
|
||||
`all ${Object.keys(aliases).length} BUILT_IN_ALIASES targets are live catalog models`,
|
||||
`all ${Object.keys(aliases).length} BUILT_IN_ALIASES targets are present in REGISTRY and absent from the retired-id snapshot`,
|
||||
findBadAliasTargets(aliases, catalogIds, retiredIds),
|
||||
"point the alias at the replacement the vendor publishes (see `sources` in config/quality/model-lifecycle.json). Never invent a target."
|
||||
);
|
||||
@@ -148,8 +178,14 @@ async function main() {
|
||||
"add a BUILT_IN_ALIASES forward to the vendor's replacement, remove the model from the provider catalog, or (last resort) add the id to `allowedRetiredInCatalog` in config/quality/model-lifecycle.json with a tracking issue."
|
||||
);
|
||||
|
||||
failures += report(
|
||||
`none of the ${Object.keys(degradationMap).length} DEFAULT_DEGRADATION_MAP rows names a retired id`,
|
||||
findRetiredDegradationRows(degradationMap, retiredIds),
|
||||
"drop the row from DEFAULT_DEGRADATION_MAP in open-sse/services/backgroundTaskDetector.ts (a retired source can never reach the redirect), or point a retired target at the replacement the vendor publishes (see `sources` in config/quality/model-lifecycle.json)."
|
||||
);
|
||||
|
||||
if (failures) {
|
||||
console.error(`[model-lifecycle] FAIL — ${failures} violation(s) across 3 check(s).`);
|
||||
console.error(`[model-lifecycle] FAIL — ${failures} violation(s) across 4 check(s).`);
|
||||
process.exit(1);
|
||||
}
|
||||
console.log(
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
* Unit coverage for the #11503 drift gate (`scripts/check/check-model-lifecycle.mjs`).
|
||||
*
|
||||
* The gate's value is that it goes red when a hand-maintained routing table starts
|
||||
* pointing at a model the vendor retired, so each of its three checks is exercised here
|
||||
* pointing at a model the vendor retired, so each of its four checks is exercised here
|
||||
* against small fixtures rather than against the live catalog (which would make the test
|
||||
* a duplicate of the gate run itself, and red for reasons unrelated to the logic).
|
||||
*/
|
||||
@@ -14,6 +14,7 @@ import {
|
||||
findRetiredFitnessRows,
|
||||
findBadAliasTargets,
|
||||
findUnforwardedRetiredIds,
|
||||
findRetiredDegradationRows,
|
||||
} from "../../scripts/check/check-model-lifecycle.mjs";
|
||||
|
||||
const RETIRED = new Set(["dead-model-1", "dead-model-2", "gpt-5.2-codex"]);
|
||||
@@ -93,3 +94,32 @@ describe("check-model-lifecycle: (c) routable retired ids", () => {
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
describe("check-model-lifecycle: (d) DEFAULT_DEGRADATION_MAP rows", () => {
|
||||
it("flags a retired source id as a dead row", () => {
|
||||
const violations = findRetiredDegradationRows({ "dead-model-1": "live-1" }, RETIRED);
|
||||
assert.equal(violations.length, 1);
|
||||
assert.match(violations[0], /retired the source id; checkLifecycle rejects it/);
|
||||
});
|
||||
|
||||
it("flags a retired target id", () => {
|
||||
const violations = findRetiredDegradationRows({ "live-1": "dead-model-1" }, RETIRED);
|
||||
assert.equal(violations.length, 1);
|
||||
assert.match(violations[0], /retired the target id/);
|
||||
});
|
||||
|
||||
it("reports both ends when source and target are retired", () => {
|
||||
const violations = findRetiredDegradationRows({ "dead-model-1": "dead-model-2" }, RETIRED);
|
||||
assert.equal(violations.length, 2);
|
||||
});
|
||||
|
||||
it("treats a vendor-prefixed source as retired when its bare form is", () => {
|
||||
const violations = findRetiredDegradationRows({ "openai/gpt-5.2-codex": "live-1" }, RETIRED);
|
||||
assert.equal(violations.length, 1);
|
||||
});
|
||||
|
||||
it("passes for a map of live ids", () => {
|
||||
assert.deepEqual(findRetiredDegradationRows({ "live-1": "live-2" }, RETIRED), []);
|
||||
assert.deepEqual(findRetiredDegradationRows({}, RETIRED), []);
|
||||
});
|
||||
});
|
||||
|
||||
56
tests/unit/model-lifecycle-degradation-map.test.ts
Normal file
56
tests/unit/model-lifecycle-degradation-map.test.ts
Normal file
@@ -0,0 +1,56 @@
|
||||
/**
|
||||
* Follow-up to #11503 / #11507: `DEFAULT_DEGRADATION_MAP` (backgroundTaskDetector.ts) is the
|
||||
* third hand-maintained routing table that names model ids, and it was outside the
|
||||
* retired-model gate. A retired *source* is a dead row — `checkLifecycle` answers 410
|
||||
* `model_shutdown` before `resolveBackgroundTaskRedirect` runs — and a retired *target*
|
||||
* is normally rejected with 410 when lifecycle validation runs again after the redirect,
|
||||
* unless alias resolution maps it to an accepted id.
|
||||
*
|
||||
* Table-driven over the production default map and the checked-in lifecycle snapshot, mirroring
|
||||
* `model-deprecation-aliases-11503.test.ts`, so a new dead row fails by name.
|
||||
*/
|
||||
import { describe, it } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { readFileSync } from "node:fs";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import { getDefaultDegradationMap } from "../../open-sse/services/backgroundTaskDetector.ts";
|
||||
import { isVendorRetiredId } from "../../open-sse/services/modelLifecycle.ts";
|
||||
|
||||
const lifecycle = JSON.parse(
|
||||
readFileSync(
|
||||
fileURLToPath(new URL("../../config/quality/model-lifecycle.json", import.meta.url)),
|
||||
"utf8"
|
||||
)
|
||||
) as { retired: Record<string, { status: string }> };
|
||||
|
||||
const retiredIds = new Set(
|
||||
Object.entries(lifecycle.retired)
|
||||
.filter(([, entry]) => entry.status === "retired")
|
||||
.map(([id]) => id.toLowerCase())
|
||||
);
|
||||
|
||||
describe("DEFAULT_DEGRADATION_MAP names no retired model id", () => {
|
||||
const rows = Object.entries(getDefaultDegradationMap());
|
||||
|
||||
it("has rows to check", () => {
|
||||
assert.ok(rows.length > 0);
|
||||
});
|
||||
|
||||
for (const [source, target] of rows) {
|
||||
it(`degrades from ${source}, an id the vendor has not retired`, () => {
|
||||
assert.ok(
|
||||
!retiredIds.has(source.toLowerCase()),
|
||||
`"${source}" → "${target}" is dead: the vendor has retired "${source}", so checkLifecycle rejects the request before the background redirect runs`
|
||||
);
|
||||
assert.equal(isVendorRetiredId(source), false);
|
||||
});
|
||||
|
||||
it(`degrades ${source} to ${target}, an id the vendor has not retired`, () => {
|
||||
assert.ok(
|
||||
!retiredIds.has(target.toLowerCase()),
|
||||
`"${source}" → "${target}" forwards background tasks to "${target}", which the vendor has retired`
|
||||
);
|
||||
assert.equal(isVendorRetiredId(target), false);
|
||||
});
|
||||
}
|
||||
});
|
||||
Reference in New Issue
Block a user