mirror of
https://github.com/diegosouzapw/OmniRoute.git
synced 2026-09-15 11:22:15 +03:00
* fix(sse): restore task-aware routing config on restart (#8601)
The T05 Task-Aware Smart Routing config was persisted to settings.taskRouting
by PUT /api/settings/task-routing but never read back, so it silently reverted
to enabled:false + the hardcoded default model map on every restart.
Two root causes, both fixed:
- No boot hydration existed. Adds hydrateTaskRoutingConfig(settings), wired into
src/instrumentation-node.ts next to the Thinking-Budget restore (#5312). It
accepts either the JSON string the route persists or an already-parsed object,
and fails open on malformed values. applyRuntimeSettings does not cover this
key, same as the Global System Prompt (#2470).
- The config lived in a plain module-level `let`, which is duplicated per module
graph — a boot hydration would have landed on the instrumentation graph's copy
and never reached the one src/sse/handlers/chat.ts reads. This is the exact
break #5312 fix-A hit on the VPS. Moves the store to the globalThis pattern
already used by thinkingBudget.ts and systemPrompt.ts.
Runtime stats are never restored from the persisted blob.
Note the hydration is wired into instrumentation-node.ts, not the unused
src/server-init.ts.
* docs(changelog): add fragment for #8604 task-routing boot restore
* fix(sse): route task-aware defaults by intent, guard fitness pattern order (#8602, #8603)
Two related defects in the hand-maintained model-quality tables.
#8602 — DEFAULT_TASK_MODEL_MAP hardcoded literal provider/model ids
(openai/gpt-4o, gemini/gemini-2.5-flash-lite, deepseek/deepseek-chat, ...).
Wrong twice over: the ids rotted by a generation or two, and applyTaskAwareRouting
overwrites body.model directly, so a literal target skipped auto-combo's 13-factor
scoring (quota, circuit-breaker health, cost, latency, stability), connection
cooldown and model lockout — hard-failing for any operator with no connection for
that provider. Refreshing the strings would only reset the rot clock, so the
defaults now name auto/* INTENTS that resolve against the operator's actually
connected backends:
coding -> auto/coding
analysis -> auto/reasoning
vision -> auto/vision
summarization -> auto/chat:fast
background -> auto/chat:cheap
creative and chat stay pass-through. Operators can still pin a specific model via
PUT /api/settings/task-routing; only the shipped defaults change. No provider/model
literal remains in the module.
#8603 — the pattern-shadowing fix LANDED UPSTREAM while this PR was open
(9f5be229b, Train 1D). lookupStaticFitnessTable now ranks patterns longest-first,
so gpt-4o-mini no longer inherits gpt-4o's 0.9 and deepseek-v3.2 no longer inherits
deepseek-v3's 0.85. This PR therefore no longer changes that behaviour — the
upstream scan is kept verbatim.
What remains for #8603 is the regression guard. The resolution chain hits the DB
(user_override / arena_elo / models.dev tier) before reaching layer 4, so asserting
the ordering through getTaskFitness would depend on DB fixture state. The layer is
exposed as getStaticFitnessTableScore and pinned directly by
taskFitness-pattern-order-8603.test.ts (7 cases), so the guarantee survives future
edits to FITNESS_TABLE. Those 7 cases were written against this PR's original
implementation and pass unchanged against the upstream one — independent
confirmation that the two are behaviourally equivalent.
* docs(changelog): add fragment for #8605 task-routing intent + fitness order
133 lines
4.4 KiB
TypeScript
133 lines
4.4 KiB
TypeScript
/**
|
|
* TDD regression for #8601: the Task-Aware Smart Routing (T05) config is persisted
|
|
* to `settings.taskRouting` by PUT /api/settings/task-routing but never read back,
|
|
* so it silently reverts to `enabled: false` + the hardcoded default model map on
|
|
* every restart.
|
|
*
|
|
* Two independent root causes, one test each:
|
|
*
|
|
* A. No hydration entry point existed at all. `hydrateTaskRoutingConfig(settings)`
|
|
* must parse the persisted value (a JSON *string*, as the route writes it) and
|
|
* apply it, mirroring `hydrateThinkingBudgetConfig` (#5312) and
|
|
* `setSystemPromptConfig` (#2470).
|
|
*
|
|
* B. The config lived in a plain module-level `let`, which is DUPLICATED per module
|
|
* graph — so a boot hydration on the instrumentation graph would never reach the
|
|
* copy that `src/sse/handlers/chat.ts` reads. This is the exact break #5312 fix-A
|
|
* hit on the VPS. The store must live on `globalThis`, like `thinkingBudget.ts`
|
|
* and `systemPrompt.ts`. Importing the module under two distinct specifiers gives
|
|
* two real module instances, which reproduces the duplication directly.
|
|
*/
|
|
import { test } from "node:test";
|
|
import assert from "node:assert/strict";
|
|
import {
|
|
hydrateTaskRoutingConfig,
|
|
setTaskRoutingConfig,
|
|
getTaskRoutingConfig,
|
|
getDefaultTaskModelMap,
|
|
} from "../../open-sse/services/taskAwareRouter.ts";
|
|
|
|
const MODULE_PATH = "../../open-sse/services/taskAwareRouter.ts";
|
|
|
|
function resetConfig(): void {
|
|
setTaskRoutingConfig({
|
|
enabled: false,
|
|
taskModelMap: getDefaultTaskModelMap(),
|
|
detectionEnabled: true,
|
|
});
|
|
}
|
|
|
|
// ── A. hydration from persisted settings ─────────────────────────────────────
|
|
|
|
test("#8601 hydrateTaskRoutingConfig restores a JSON-string persisted config", () => {
|
|
resetConfig();
|
|
|
|
// Shape written by src/app/api/settings/task-routing/route.ts:66
|
|
const persisted = JSON.stringify({
|
|
enabled: true,
|
|
detectionEnabled: true,
|
|
taskModelMap: { ...getDefaultTaskModelMap(), coding: "auto/coding" },
|
|
});
|
|
|
|
assert.equal(hydrateTaskRoutingConfig({ taskRouting: persisted }), true);
|
|
|
|
const config = getTaskRoutingConfig();
|
|
assert.equal(config.enabled, true);
|
|
assert.equal(config.taskModelMap.coding, "auto/coding");
|
|
|
|
resetConfig();
|
|
});
|
|
|
|
test("#8601 hydrateTaskRoutingConfig accepts an already-parsed object", () => {
|
|
resetConfig();
|
|
|
|
assert.equal(
|
|
hydrateTaskRoutingConfig({ taskRouting: { enabled: true, detectionEnabled: false } }),
|
|
true
|
|
);
|
|
assert.equal(getTaskRoutingConfig().enabled, true);
|
|
assert.equal(getTaskRoutingConfig().detectionEnabled, false);
|
|
|
|
resetConfig();
|
|
});
|
|
|
|
test("#8601 hydrateTaskRoutingConfig is a no-op for missing/invalid settings", () => {
|
|
resetConfig();
|
|
|
|
for (const settings of [
|
|
undefined,
|
|
null,
|
|
{},
|
|
{ taskRouting: "" },
|
|
{ taskRouting: "not json" },
|
|
{ taskRouting: "[]" },
|
|
{ taskRouting: 42 },
|
|
]) {
|
|
assert.equal(
|
|
hydrateTaskRoutingConfig(settings),
|
|
false,
|
|
`should ignore ${JSON.stringify(settings)}`
|
|
);
|
|
assert.equal(getTaskRoutingConfig().enabled, false);
|
|
}
|
|
|
|
resetConfig();
|
|
});
|
|
|
|
test("#8601 hydration never resurrects stats from the persisted blob", () => {
|
|
resetConfig();
|
|
|
|
hydrateTaskRoutingConfig({
|
|
taskRouting: JSON.stringify({ enabled: true, stats: { detected: 999, routed: 999 } }),
|
|
});
|
|
|
|
const { stats } = getTaskRoutingConfig();
|
|
assert.equal(stats.detected, 0);
|
|
assert.equal(stats.routed, 0);
|
|
|
|
resetConfig();
|
|
});
|
|
|
|
// ── B. cross-module-graph sharing (the #5312 fix-A trap) ─────────────────────
|
|
|
|
test("#8601 config store is shared across duplicated module instances", async () => {
|
|
resetConfig();
|
|
|
|
// Distinct specifiers → two genuinely separate module records, the same way the
|
|
// Next.js instrumentation graph and the app-route/open-sse graph each get their own.
|
|
const graphA = await import(`${MODULE_PATH}?graph=a`);
|
|
const graphB = await import(`${MODULE_PATH}?graph=b`);
|
|
|
|
graphA.setTaskRoutingConfig({ enabled: true, detectionEnabled: true });
|
|
|
|
assert.equal(
|
|
graphB.getTaskRoutingConfig().enabled,
|
|
true,
|
|
"a config set on one module instance must be visible from another — otherwise boot " +
|
|
"hydration lands on the instrumentation copy and the chat handler never sees it"
|
|
);
|
|
|
|
graphA.setTaskRoutingConfig({ enabled: false });
|
|
resetConfig();
|
|
});
|