pjandro/OmniRoute
1
0
Fork 0
mirror of https://github.com/diegosouzapw/OmniRoute.git synced 2026-08-03 05:45:04 +03:00
Code Issues Packages Projects Releases Wiki Activity

refactor(sse): extract combo target resolution into combo/targetResolution.ts (#8592)

Browse Source 
* refactor(sse): extract combo dispatch prelude into combo/dispatchPrelude.ts

Pure move, no behaviour change. First of ~7 PRs decomposing the combo.ts
god-file (#3501).

handleComboChat evaluates a series of dispatch branches before it ever
reaches target resolution or the sequential attempt loop. None of them
iterate targets in priority order or need the failover/retry/credential
gate machinery that follows, so they move to a leaf:

  - context-cache pin routing (Fix #679), including the
    pinIsDurablyUnhealthy / isPinnedModelDurablyUnhealthy health gate
  - fusion panel dispatch + the #6455 misconfiguration warn
  - pipeline chaining
  - nested combo-ref execute-mode runtime-unit dispatch

Only the chaos and round-robin hand-offs stay inline (11 and 13 lines);
extracting those would be pure indirection.

open-sse/services/combo.ts 3642 -> 3341 (-301)
open-sse/services/combo/dispatchPrelude.ts: 619 (under the 800 cap)

Each helper keeps the fall-through protocol the inline blocks had: return
a Response to OWN the request, return null to fall through. A flipped
null/Response would silently bypass the whole combo strategy, so the new
tests pin both directions for every branch.

combo.ts re-exports pinIsDurablyUnhealthy so combo-pin-health-gate.test.ts
keeps resolving. The leaf takes handleComboChat as a `runCombo` parameter
instead of importing it, so combo/ keeps zero back-edges into combo.ts.

Complexity-neutral: the first cut added +3 violations (two
max-lines-per-function, one complexity) inside the new leaf, so
evaluatePinnedResponse, orderRuntimeUnits, recordRuntimeUnitStickySuccess
and buildBaseOptions were split out. check:complexity now measures 2169
and check:cognitive-complexity 956 — identical to the pristine base.

* test(sse): close the dispatch-prelude coverage holes found by mutation testing

An adversarial mutation audit of the suite added in the previous commit
found it guarded the fall-through protocol well but asserted almost
nothing about what the helpers do once they OWN the request. 5 of 12
seeded mutations survived. Worst case: deleting the pinned-model
dispatch call outright left all 12 tests green.

Three holes, now closed (8 tests -> 20):

Hole A — the honored-pin path had zero coverage. Both existing pin tests
DROP the pin, so the dispatch, the 200-but-empty quality gate, the
[408, 429, 500, 502, 503, 504] failover list and the catch(pinErr)
branch were unguarded — exactly the logic the 2026-06-21 / 2026-06-22
incident comments call load-bearing. Adds five tests over a seeded
healthy provider connection so the pin is actually honored.

Hole B — orderRuntimeUnits was only ever driven with `priority`, which
is a no-op through it. Four of five strategy branches could be deleted
with nothing failing. Adds round-robin rotation and weighted sticky
ordering tests.

Hole C — recordRuntimeUnitStickySuccess never did anything under test:
both its guards need weighted/round-robin, so an early return changed
nothing. Covered by the new sticky-batch test.

Verified by re-running the mutations rather than assuming: all 7 that
previously survived (delete-pin-dispatch, serve-despite-failed-quality,
never-fail-over-on-transient, rr-counter-not-advanced, rotation-removed,
weighted-sticky-skipped, sticky-recording-no-op) are now killed.

The first sticky-batch test I wrote was itself vacuous — asserting "same
unit twice" holds equally when the recording helper is stubbed out, since
nothing advances the counter either. It now asserts the batch runs out
and rotation resumes on the third dispatch, which is what actually
distinguishes the two.

Also restores API_KEY_SECRET in test.after; it was set at module load and
never put back, inconsistent with the DATA_DIR handling beside it.

* fix(ci): teach known-symbols gate the relocated fusion/pipeline dispatch

The combo sub-check of check:known-symbols asserts every canonical routing
strategy has a real dispatch branch. It scanned a hardcoded file list and
matched only `strategy === "..."`, so the prelude extraction tripped it twice:

  [combo] 2 estratégia(s) canônica(s) sem branch de despacho em combo.ts:
      ✗ fusion
      ✗ pipeline

Both branches are still wired — they just moved to combo/dispatchPrelude.ts and
took the early-return guard form `if (strategy !== "fusion") return null;` that
extracting a branch into a `tryXDispatch()` leaf naturally produces.

Two changes, both extending existing precedent (the list already carries the
Block J leaves for the same reason):

- register combo/dispatchPrelude.ts in comboDispatchFiles
- widen the extractor to `strategy [!=]== "..."` so the inverted guard counts

Loose `==`/`!=` stay rejected, and no `handledNotCanonical` fallout: the gate
now reports 20 canonical strategies, all 20 via despacho.

* chore(ci): register combo-dispatch-prelude test in stryker tap.testFiles

check:mutation-test-coverage --strict failed once the known-symbols fix let
Fast Quality Gates advance to it:

  ✗ 2 covering unit test(s) across 2 module(s) are missing from
    stryker.conf.json tap.testFiles
      open-sse/services/combo/comboStructure.ts
      open-sse/services/combo/rrState.ts

The new tests/unit/combo-dispatch-prelude.test.ts exercises both modules, and
both are already in stryker's mutate list, so without the registration its
mutant kills would not have counted toward the nightly mutation gate.

Note (unchanged, still out of scope): combo/dispatchPrelude.ts itself is not in
stryker's `mutate` list. Adding it would widen the nightly mutation surface,
which is a separate call from fixing this drift.

* docs(changelog): add fragment for #8582 combo dispatch prelude

* refactor(sse): extract combo target resolution into combo/targetResolution.ts

Pure move, no behaviour change. Lifts the target-resolution stage of
handleComboChat — everything between the dispatch prelude and the attempt
loop — into a new leaf, open-sse/services/combo/targetResolution.ts.

Moved verbatim: provider-wildcard expansion, weighted step-group resolution
+ sticky-weighted eligibility, request-tag routing, the known-context-overflow
early return, the smart/pipeline-enabled auto dispatch, auto-strategy
ordering, per-strategy ordering, cache-strategy affinity, session stickiness,
eval scores, request-compatibility + context-requirement filters, task-aware
reordering, prompt-cache affinity, and the priority-strategy pre-screen.

The three early exits become an { earlyResponse } result so the host decides
to return them (same pattern as resolveAutoStrategyOrder). The values the
attempt loop still reads — orderedTargets, stickyWeightedLimit,
getWeightedStepKeyForTarget, the session-stickiness result and preScreenMap —
are returned instead of closed over. Loop config (maxRetries, retryDelayMs,
fallbackDelayMs, maxSetRetries, setRetryDelayMs) stays in combo.ts.

buildAutoCandidates is dependency-injected because it lives in combo.ts, so
the leaf keeps zero back-edges into its host.

combo.ts 3640 -> 3321 lines; new leaf 484 lines (under the 800 cap).
Part of the #3501 god-file decomposition campaign.

* refactor(sse): split targetResolution into stage helpers, ratchet combo.ts file-size baseline

Follow-up to the target-resolution extraction: the moved region landed as one
311-line function, which converted inline code inside the (already-violating)
handleComboChat into a NEW separately-counted violating function — check:complexity
2169 -> 2171 and check:cognitive-complexity 956 -> 957.

Split resolveComboTargetPipeline along its natural stage boundaries into 14
helpers (wildcard expansion, weighted eviction/eligibility/sticky-key/selection,
step-key mapper, context-overflow response, pool-size log, smart-pipeline dispatch
and its fall-through logger, strategy ordering, continuity filters, task-aware
ordering, prompt-cache enablement/first-target protection/affinity stage). Each
stage takes the previous stage's output and returns the next; still a pure move.

The leaf now contributes ZERO complexity, max-lines-per-function and
cognitive-complexity violations. Both ratchets are back at base 4053e2314 values:
check:complexity 2169, check:cognitive-complexity 956. (Both still print RED
against their frozen ceilings 2130/951 — pre-existing base-red per #8580.)

Also ratchets ONLY the open-sse/services/combo.ts entry in
config/quality/file-size-baseline.json from 3642 to 3322, with a justification
note in the file's existing style. No sweep of unrelated entries.

* chore: stack targetResolution on dispatchPrelude tip, rebank + skills

Rebased onto refactor/combo-dispatch-prelude. Keep both leaves in
check-known-symbols. Regenerate file-size baseline; sync agent skills.

* fix(sse): restore #8494 capability fail-closed after targetResolution extract

Stacking targetResolution onto the dispatchPrelude tip dropped the #8488/#8494
compatFilterFailOpen wiring: hard capability filters emptied the pool into a
generic 404 no_executable_targets, and fail-open never re-admitted the pool.

Restore describeCapabilityFilterExhaustion earlyResponse in
applyContinuityFilters and the matching round-robin path, then rebank the
file-size baseline for tip growth the incomplete prior rebank missed.

* fix(sse): realign model-lockout cooldown options with the post-#8254 type

This branch predates #8254, which renamed the recordModelLockoutFailure option
`exactCooldownVerified` -> `exactCooldownIsUpstreamReset` and changed combo.ts's
predicate from `lockoutHintVerified` (#8393's `lockoutHintMs > 0`) to
`lockoutHintMs > mlSettings.baseCooldownMs`. Rebasing onto the current tip brought
the renamed type without updating these two call sites, so typecheck:core failed
with TS2353 at both.

Restores the base expression verbatim rather than re-wiring `lockoutHintVerified`
under the new name. The base predicate is the correct one: selectLockoutCooldownMs
returns the parsed hint ONLY when `lockoutHintMs > baseCooldownMs`, and otherwise
returns 0 or a synthetic baseCooldownMs — so `lockoutHintMs > 0` would mark a
synthetic cooldown as an upstream reset and let it bypass the #7940 maxCooldownMs
cap, which is the bug #8254 fixed.

---------

Co-authored-by: MumuTW <johnsxn.us@gmail.com>
Co-authored-by: diegosouzapw <diegosouzapw@users.noreply.github.com>
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
This commit is contained in:
MumuTW
2026-07-28 06:13:31 +08:00
committed by GitHub
parent cd7a492984
commit 0eeb8f45c0
5 changed files with 886 additions and 371 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
changelog.d/maintenance/8592-combo-target-resolution.md Normal file
View File

@@ -0,0 +1 @@
- **refactor(sse):** extract combo target resolution into `combo/targetResolution.ts` (`resolveComboTargetPipeline`) — pure move, no behaviour change; #3501 PR 2 of ~7 ([#8592](https://github.com/diegosouzapw/OmniRoute/pull/8592)) — thanks @MumuTW

393
open-sse/services/combo.ts
View File

@@ -65,10 +65,6 @@ import { phaseComboSetup } from "./combo/comboSetup.ts";
import { checkCredentialGate, logCredentialSkip } from "./credentialGate.ts";
import { emit } from "../../src/lib/events/eventBus";
import { notifyWebhookEvent } from "../../src/lib/webhookDispatcher";
import { parseAutoPrefix } from "./autoCombo/autoPrefix.ts";
import { resolveAutoStrategyOrder } from "./combo/resolveAutoStrategy.ts";
import { applyStrategyOrdering } from "./combo/applyStrategyOrdering.ts";
import { handlePipelineCombo, buildPipelineResponse } from "./autoCombo/pipelineRouter.ts";
import { type ProviderCandidate } from "./autoCombo/scoring.ts";
import { estimateTokens } from "./contextManager.ts";
import { getSessionConnection } from "./sessionManager.ts";
@@ -89,7 +85,6 @@ import {
expandPromptCacheAffinityTargets,
expandPromptCacheAffinityTargetsFromConnections,
resolvePromptCacheAffinityKey,
shouldProtectOriginalFirst,
} from "./combo/promptCacheAffinity.ts";
import type { CompressionMode } from "./compression/types.ts";
import { getCachedProviderConnections } from "../../src/lib/db/readCache";
@@ -113,7 +108,6 @@ import type {
HandleRoundRobinOptions,
ResolvedComboTarget,
AutoProviderCandidate,
ComboRuntimeStep,
HistoricalLatencyStatsEntry,
} from "./combo/types.ts";
@@ -121,7 +115,6 @@ import {
MAX_RR_COUNTERS,
rrCounters,
rrStickyTargets,
weightedStickyTargets,
clampStickyRoundRobinTargetLimit,
clampStickyWeightedTargetLimit,
getStickyRoundRobinStartIndex,
@@ -193,15 +186,12 @@ import {
} from "./combo/providerWildcard.ts";
import { resolveShadowTargets, scheduleShadowRouting } from "./combo/shadowRouting.ts";
import { attemptCompatRejectedFallback } from "./combo/comboCompatFallback.ts";
import { applyContextRequirements } from "./combo/contextRequirements.ts";
import {
computeCompatRejectedTargets,
describeCapabilityFilterExhaustion,
filterTargetsByRequestCompatibility,
resolveComboRuntimeUnits,
resolveComboTargets,
resolveWeightedTargets,
resolveWeightedStepGroups,
} from "./combo/comboStructure.ts";
import { getKnownContextOverflow } from "./combo/knownContextOverflow.ts";
import {
@@ -219,22 +209,13 @@ import {
calculateResetWindowAffinity,
type ResetWindowConfig,
} from "./combo/quotaScoring.ts";
import {
fetchResetAwareQuotaWithCache,
preScreenTargets,
type PreScreenResult,
} from "./combo/quotaStrategies.ts";
import { fetchResetAwareQuotaWithCache, preScreenTargets } from "./combo/quotaStrategies.ts";
import {
buildAutoQuotaThresholds,
resolveQuotaExhaustionCutoffForTarget,
} from "./combo/quotaExhaustionCutoff.ts";
import {
classifyTask,
getConversationCacheKey,
isTaskRoutingStrategy,
reorderByTaskWeight,
} from "./taskAwareRouting.ts";
import { expandTargetsByFingerprints } from "./combo/fingerprintExpansion.ts";
import { resolveComboTargetPipeline } from "./combo/targetResolution.ts";
export { RESET_WINDOW_NAMES };
export { QUOTA_SOFT_DEPRIORITIZE_FACTOR, setCandidateQuotaSoftPenalty };
@@ -711,356 +692,26 @@ export async function handleComboChat({
const maxSetRetries = config.maxSetRetries ?? 0;
const setRetryDelayMs = resolveDelayMs(config.setRetryDelayMs, 2000);
const isTargetSelectableForWeighted = async (target: ResolvedComboTarget): Promise<boolean> => {
const rawModel = parseModel(target.modelStr).model || target.modelStr;
if (target.provider && getCircuitBreaker(target.provider).getStatus().state === "OPEN")
return false;
if (
resilienceSettings.providerCooldown.enabled &&
Boolean(target.provider && target.provider !== "unknown") &&
isProviderInCooldown(target.provider, target.connectionId ?? undefined, resilienceSettings)
) {
return false;
}
if (
target.provider &&
rawModel &&
isModelLocked(target.provider, target.connectionId || "", rawModel)
) {
return false;
}
return isModelAvailable ? await isModelAvailable(target.modelStr, target) : true;
};
// #2562: Expand provider-wildcard steps (e.g. `fta/*`, `openai/gpt-4*`) into
// concrete model entries sourced from the live synced-models catalog + registry.
// Must run before any step-group / target resolution so that wildcard-originated
// steps are treated identically to hand-authored entries by all downstream logic
// (including the sticky-weighted eligibility pass below).
const expandedCombo = await expandProviderWildcardsInCombo(combo);
const expandedAllCombos = allCombos
? Array.isArray(allCombos)
? await expandProviderWildcardsInCollection(allCombos as ComboLike[])
: {
...allCombos,
combos: await expandProviderWildcardsInCollection(
((allCombos as { combos?: ComboLike[] }).combos || []) as ComboLike[]
),
}
: allCombos;
const stickyWeightedLimit = clampStickyWeightedTargetLimit(
(config as Record<string, unknown>).stickyWeightedLimit
);
if (
strategy === "weighted" &&
!weightedStickyTargets.has(combo.name) &&
weightedStickyTargets.size >= MAX_RR_COUNTERS
) {
const oldest = weightedStickyTargets.keys().next().value;
if (oldest !== undefined) weightedStickyTargets.delete(oldest);
}
let stepGroups: Array<{ step: ComboRuntimeStep; targets: ResolvedComboTarget[] }> | undefined;
const weightedEligibleKeys = new Set<string>();
if (strategy === "weighted") {
stepGroups = resolveWeightedStepGroups(expandedCombo, expandedAllCombos);
for (const group of stepGroups) {
const availability = await Promise.all(group.targets.map(isTargetSelectableForWeighted));
if (availability.some(Boolean)) weightedEligibleKeys.add(group.step.executionKey);
}
}
const rawStickyWeightedKey =
strategy === "weighted" ? getStickyWeightedExecutionKey(combo.name, stickyWeightedLimit) : null;
const stickyWeightedKey =
rawStickyWeightedKey && weightedEligibleKeys.has(rawStickyWeightedKey)
? rawStickyWeightedKey
: null;
if (strategy !== "weighted" || stickyWeightedLimit <= 1) {
weightedStickyTargets.delete(combo.name);
} else if (rawStickyWeightedKey && !stickyWeightedKey) {
weightedStickyTargets.delete(combo.name);
}
const weightedResolution =
strategy === "weighted"
? resolveWeightedTargets(
expandedCombo,
expandedAllCombos,
stickyWeightedKey,
weightedEligibleKeys,
stepGroups
)
: null;
const getWeightedStepKeyForTarget = (target: ResolvedComboTarget): string | null => {
if (!weightedResolution?.orderedSteps) return null;
const step = weightedResolution.orderedSteps.find(
(entry) =>
target.executionKey === entry.executionKey ||
target.executionKey.startsWith(entry.executionKey + ">")
);
return step?.executionKey || null;
};
let orderedTargets =
strategy === "weighted"
? weightedResolution?.orderedTargets || []
: resolveComboTargets(
expandedCombo,
expandedAllCombos,
clampComboDepth(config.maxComboDepth)
);
orderedTargets = await applyRequestTagRouting(orderedTargets, body, log);
const knownContextOverflow = getKnownContextOverflow(orderedTargets, body);
if (knownContextOverflow) {
const { requiredContextTokens, maxKnownContextTokens } = knownContextOverflow;
log.warn(
"COMBO",
`Request context exceeds every known target limit (${requiredContextTokens} > ${maxKnownContextTokens} tokens)`
);
return errorResponseWithComboDiagnostics(
400,
`Request requires approximately ${requiredContextTokens} tokens, but the largest known context limit in this combo is ${maxKnownContextTokens} tokens. Reduce or compact the request context.`,
{
poolSize: orderedTargets.length,
attempted: 0,
excluded: orderedTargets.map((target) => ({
provider: target.provider,
model: target.modelStr,
reason: "context_window",
})),
attemptOrder: [],
terminalReason: "context_length_exceeded",
},
{ code: "context_length_exceeded", type: "invalid_request_error" }
);
}
if (strategy === "weighted") {
log.info(
"COMBO",
`Weighted selection${stickyWeightedKey ? " (sticky)" : ""}${allCombos ? " with nested resolution" : ""}: ${orderedTargets.length} total targets`
);
} else if (allCombos) {
log.info("COMBO", `${strategy} with nested resolution: ${orderedTargets.length} total targets`);
}
// Pipeline dispatch: route smart/pipeline-enabled combos through the multi-stage pipeline
if (strategy === "auto") {
const autoParsed = parseAutoPrefix(combo.name);
const autoVariant = autoParsed.valid ? autoParsed.variant : undefined;
if (autoVariant === "smart" || config.pipeline_enabled) {
try {
const pipelineRaw = await handlePipelineCombo({
body,
combo,
handleChatCore: handleSingleModelWithTimeout,
log: {
info: log.info,
warn: log.warn,
error: log.error ?? log.warn,
},
settings: settings ?? {},
signal: signal ?? undefined,
});
// handlePipelineCombo resolves to a PipelineResult (buffered text) or,
// in the streaming-final-stage case, a Response. Callers downstream
// (chat.ts → withSessionHeader) require a Response, so adapt the
// PipelineResult here instead of leaking the raw object.
return pipelineRaw instanceof Response
? pipelineRaw
: buildPipelineResponse(pipelineRaw, body);
} catch (pipelineErr) {
const pipelineMsg = pipelineErr instanceof Error ? pipelineErr.message : "";
if (pipelineMsg === "PIPELINE_DISABLED") {
log.info("COMBO", "Pipeline disabled, falling through to standard auto routing");
} else if (pipelineMsg === "PIPELINE_TOKEN_THRESHOLD") {
log.info(
"COMBO",
"Pipeline skipped (prompt below token threshold), falling through to standard auto routing"
);
} else {
log.warn("COMBO", "Pipeline dispatch failed, falling through to standard auto routing", {
err: pipelineErr,
});
}
}
}
}
// #4945 regression guard: when an "auto" combo uses an EXPLICIT router
// (routingStrategy lkgp/cost/etc, not the default "rules" scorer), that router
// pins orderedTargets[0]. The task-aware reordering below must then refine only
// the fallback order, never override the router's primary choice.
let autoUsedExplicitRouter = false;
if (strategy === "auto") {
const autoResult = await resolveAutoStrategyOrder({
orderedTargets,
body,
combo,
settings,
config,
relayOptions,
resilienceSettings,
log,
buildAutoCandidates,
});
if ("earlyResponse" in autoResult) return autoResult.earlyResponse;
orderedTargets = autoResult.orderedTargets;
autoUsedExplicitRouter = autoResult.autoUsedExplicitRouter;
} else {
orderedTargets = await applyStrategyOrdering(strategy, orderedTargets, {
combo,
config,
body,
log,
apiKeyAllowedConnections,
});
}
// An explicit cache-optimized combo outranks the global cache-affinity default,
// but only protects its ordering when this request actually produced a reusable
// cache key. Cache misses retain the normal session/eval routing behavior.
const cacheStrategyAffinityApplied =
strategy === "cache-optimized" && applyPromptCacheAffinity(orderedTargets, body).applied;
// #6168: session stickiness opt-out. Per-combo `config.disableSessionStickiness`
// overrides the global `settings.disableSessionStickiness` fallback (default false,
// preserving the #3825 prompt-cache/504 fix). When disabled, skip the reorder and
// treat the result as a no-op so the recordStickyBinding write-back below is skipped.
const disableSessionStickiness =
cacheStrategyAffinityApplied ||
resolveDisableSessionStickiness(
config as Record<string, unknown> | null | undefined,
settings as Record<string, unknown> | null | undefined
);
const _sticky = disableSessionStickiness
? ({ targets: orderedTargets, messageHash: null, stuck: false } as const)
: await applySessionStickiness(
orderedTargets,
// #7270: normalize both wire shapes (.messages / Responses-API .input) so the
// stickiness key is derivable on the /v1/responses surface, not just Chat Completions.
normalizeStickinessMessages(body as { messages?: unknown; input?: unknown })
);
orderedTargets = _sticky.targets;
if (!cacheStrategyAffinityApplied) {
orderedTargets = orderTargetsByEvalScores(orderedTargets, config.evalRouting, log);
}
const compatFilterFailOpen =
(config as { compatFilterFailOpen?: unknown }).compatFilterFailOpen === true ||
(settings as { compatFilterFailOpen?: unknown } | null | undefined)?.compatFilterFailOpen ===
true;
const preCompatTargets = orderedTargets;
orderedTargets = filterTargetsByRequestCompatibility(orderedTargets, body, log, undefined, {
failOpen: compatFilterFailOpen,
});
if (orderedTargets.length === 0 && preCompatTargets.length > 0) {
const exhaustion = describeCapabilityFilterExhaustion(preCompatTargets, body, combo.name);
if (exhaustion) {
recordComboFailure(effectiveSessionId, combo.name);
return errorResponseWithComboDiagnostics(
400,
exhaustion.message,
{
poolSize: preCompatTargets.length,
attempted: 0,
excluded: exhaustion.excluded,
attemptOrder: [],
terminalReason: exhaustion.terminalReason,
recovery: buildRecoveryHint("no_executable_targets"),
},
{ code: "capability_mismatch", type: "invalid_request_error" }
);
}
}
orderedTargets = applyContextRequirements(orderedTargets, config.contextRequirements, log);
// Task-aware reordering: only active for strategies ["smart","task","task-aware","task_aware","auto"].
// Additive — does not affect any of the other 15 strategies.
if (isTaskRoutingStrategy(strategy)) {
const task = classifyTask(body);
const conversationCacheKey = getConversationCacheKey(body);
const taskReordered = reorderByTaskWeight(orderedTargets, task);
// #4945 regression guard: when an explicit auto router (lkgp/cost/…) pinned
// orderedTargets[0], keep that primary choice and let task-aware refine only
// the fallback tail — otherwise task weighting silently defeats the operator's
// chosen LKGP/cost selection. reorderByTaskWeight returns the same target
// objects (no clone), so identity filtering is safe.
const pinnedFirst = autoUsedExplicitRouter ? orderedTargets[0] : undefined;
const nextOrder = pinnedFirst
? [pinnedFirst, ...taskReordered.filter((t) => t !== pinnedFirst)]
: taskReordered;
if (nextOrder[0]?.modelStr !== orderedTargets[0]?.modelStr) {
const reasons =
Array.isArray(task.reasons) && task.reasons.length > 0
? ` (${task.reasons.join(",")})`
: "";
log.info(
"COMBO",
`task-route task=${task.level}${reasons} cacheKey=${conversationCacheKey ?? "none"} → ${nextOrder[0]?.modelStr}`
);
}
orderedTargets = nextOrder;
}
// Prompt-cache locality is applied after request eligibility and task routing.
// Session stickiness and explicit auto-router pins remain stronger continuity
// decisions; quota, health, and circuit-breaker gates still run per attempt.
const autoConfigForCacheWeight =
strategy === "auto"
? ((combo.autoConfig ||
((config as Record<string, unknown>).auto &&
typeof (config as Record<string, unknown>).auto === "object"
? (config as Record<string, unknown>).auto
: null) ||
config) as Record<string, unknown>)
: null;
const autoWeightsForCache =
autoConfigForCacheWeight?.weights && typeof autoConfigForCacheWeight.weights === "object"
? (autoConfigForCacheWeight.weights as Record<string, unknown>)
: null;
const autoUsesCacheScore = Number(autoWeightsForCache?.cacheAffinity) > 0;
const promptCacheAffinityEnabled =
settings?.promptCacheAffinityEnabled !== false && !autoUsesCacheScore;
const promptCacheAffinityTargets =
promptCacheAffinityEnabled && resolvePromptCacheAffinityKey(body)
? await expandPromptCacheAffinityTargets(orderedTargets)
: orderedTargets;
const promptCacheAffinity = applyPromptCacheAffinity(
promptCacheAffinityTargets,
const targetResolution = await resolveComboTargetPipeline({
body,
promptCacheAffinityEnabled
);
if (promptCacheAffinity.applied) {
const protectedOriginal =
shouldProtectOriginalFirst(_sticky.stuck, autoUsedExplicitRouter, strategy) &&
orderedTargets[0];
const protectedFirst = protectedOriginal
? (promptCacheAffinity.targets.find(
(target) =>
target === protectedOriginal ||
target.executionKey === protectedOriginal.executionKey ||
target.executionKey.startsWith(`${protectedOriginal.executionKey}@`)
) ?? protectedOriginal)
: null;
orderedTargets = protectedFirst
? [
protectedFirst,
...promptCacheAffinity.targets.filter((target) => target !== protectedFirst),
]
: promptCacheAffinity.targets;
log.debug?.("COMBO", "Prompt-cache affinity applied", {
source: promptCacheAffinity.source,
fingerprint: promptCacheAffinity.fingerprint,
targetCount: orderedTargets.length,
});
}
// Parallel pre-screen: check provider profiles and model availability for all targets
// Only runs for priority strategy where sequential checking causes latency
const preScreenMap =
strategy === "priority"
? await preScreenTargets(orderedTargets, isModelAvailable).catch(
() => new Map<string, PreScreenResult>()
)
: new Map<string, PreScreenResult>();
combo,
strategy,
config,
settings,
allCombos,
relayOptions,
signal,
apiKeyAllowedConnections,
log,
resilienceSettings,
isModelAvailable,
handleSingleModelWithTimeout,
buildAutoCandidates,
});
if ("earlyResponse" in targetResolution) return targetResolution.earlyResponse;
const { stickyWeightedLimit, getWeightedStepKeyForTarget, preScreenMap } = targetResolution;
const _sticky = targetResolution.sticky;
let orderedTargets = targetResolution.orderedTargets;
// #5923 (Finding #4) — reset-window config for the shared per-target quota-
// exhaustion cutoff below. The "auto" strategy already applies its own cutoff
@@ -2587,7 +2238,7 @@ async function handleRoundRobinCombo({
{ code: "context_length_exceeded", type: "invalid_request_error" }
);
}
// Align with the main/auto paths: combo config OR top-level settings.
// Align with the main/auto paths: combo config OR top-level settings (#8488 / #8494).
const rrCompatFailOpen =
(config as { compatFilterFailOpen?: unknown }).compatFilterFailOpen === true ||
(settings as { compatFilterFailOpen?: unknown } | null | undefined)?.compatFilterFailOpen ===

714
open-sse/services/combo/targetResolution.ts Normal file
View File

@@ -0,0 +1,714 @@
/**
* resolveComboTargetPipeline — the target-resolution phase of handleComboChat (combo.ts).
*
* Sits between the dispatch prelude (pinned model / fusion / chaos / pipeline / nested
* execute-mode / round-robin) and the attempt loop. It turns the raw combo definition
* into the final `orderedTargets` array the attempt loop iterates, in this order:
*
* 1. provider-wildcard expansion of the combo + the combos collection (#2562)
* 2. weighted step-group resolution + sticky-weighted eligibility
* 3. request-tag routing
* 4. known-context-overflow early return
* 5. smart/pipeline-enabled dispatch (auto strategy)
* 6. auto-strategy candidate build / scoring / ordering, or per-strategy ordering
* 7. prompt-cache strategy affinity, session stickiness, eval scores,
* request compatibility, context requirements
* 8. task-aware reordering
* 9. prompt-cache affinity application
* 10. the parallel pre-screen (priority strategy only)
*
* Behaviour is byte-identical to the inline block it replaces — the two early exits
* (context overflow, pipeline dispatch, auto-strategy `earlyResponse`) become an
* `{ earlyResponse }` result so the host decides to return them, and the values the
* attempt loop still consumes (`orderedTargets`, `stickyWeightedLimit`,
* `getWeightedStepKeyForTarget`, `sticky`, `preScreenMap`) are returned instead of
* closed over.
*
* See _tasks/quality/2026-06-19-DESIGN-godfiles-decomposition.md §4.
*/
import { isModelLocked } from "../accountFallback.ts";
import { parseAutoPrefix } from "../autoCombo/autoPrefix.ts";
import { handlePipelineCombo, buildPipelineResponse } from "../autoCombo/pipelineRouter.ts";
import type { resolveComboSetupConfig } from "../comboConfig.ts";
import { orderTargetsByEvalScores } from "../evalRouting.ts";
import { parseModel } from "../model.ts";
import { isProviderInCooldown } from "../providerCooldownTracker.ts";
import {
classifyTask,
getConversationCacheKey,
isTaskRoutingStrategy,
reorderByTaskWeight,
} from "../taskAwareRouting.ts";
import { errorResponseWithComboDiagnostics } from "../../utils/error.ts";
import { getCircuitBreaker } from "../../../src/shared/utils/circuitBreaker";
import type { ResilienceSettings } from "../../../src/lib/resilience/settings";
import { applyStrategyOrdering } from "./applyStrategyOrdering.ts";
import { clampComboDepth } from "./comboPredicates.ts";
import {
describeCapabilityFilterExhaustion,
filterTargetsByRequestCompatibility,
resolveComboTargets,
resolveWeightedStepGroups,
resolveWeightedTargets,
} from "./comboStructure.ts";
import { applyContextRequirements } from "./contextRequirements.ts";
import { recordComboFailure } from "./failureTracker.ts";
import { getKnownContextOverflow } from "./knownContextOverflow.ts";
import { buildRecoveryHint } from "./pinRecovery.ts";
import {
applyPromptCacheAffinity,
expandPromptCacheAffinityTargets,
resolvePromptCacheAffinityKey,
shouldProtectOriginalFirst,
} from "./promptCacheAffinity.ts";
import {
expandProviderWildcardsInCombo,
expandProviderWildcardsInCollection,
} from "./providerWildcard.ts";
import { preScreenTargets, type PreScreenResult } from "./quotaStrategies.ts";
import { resolveAutoStrategyOrder, type ResolveAutoStrategyDeps } from "./resolveAutoStrategy.ts";
import {
MAX_RR_COUNTERS,
clampStickyWeightedTargetLimit,
getStickyWeightedExecutionKey,
weightedStickyTargets,
} from "./rrState.ts";
import {
applySessionStickiness,
normalizeStickinessMessages,
resolveDisableSessionStickiness,
type ApplyStickinessResult,
} from "./sessionStickiness.ts";
import { applyRequestTagRouting } from "./autoStrategy.ts";
import type {
ComboCollectionLike,
ComboLike,
ComboLogger,
ComboRelayOptions,
ComboRuntimeStep,
HandleSingleModel,
IsModelAvailable,
ResolvedComboTarget,
} from "./types.ts";
export interface ResolveComboTargetPipelineDeps {
body: Record<string, unknown>;
combo: ComboLike;
strategy: string;
config: ReturnType<typeof resolveComboSetupConfig>;
settings?: Record<string, unknown> | null;
allCombos?: ComboCollectionLike;
relayOptions?: ComboRelayOptions | null;
signal?: AbortSignal | null;
apiKeyAllowedConnections: string[] | null;
log: ComboLogger;
resilienceSettings: ResilienceSettings;
isModelAvailable?: IsModelAvailable;
/** handleSingleModel already wrapped by buildTargetTimeoutRunner. */
handleSingleModelWithTimeout: HandleSingleModel;
/**
* Dependency-injected `buildAutoCandidates` — it lives in `combo.ts` (the host of
* this leaf), so importing it directly would create an import cycle.
*/
buildAutoCandidates: ResolveAutoStrategyDeps["buildAutoCandidates"];
}
export interface ResolvedComboTargetPipeline {
orderedTargets: ResolvedComboTarget[];
/** Sticky-weighted target limit — the attempt loop records sticky success with it. */
stickyWeightedLimit: number;
/** Maps an attempted target back to its weighted step key (sticky-weighted write-back). */
getWeightedStepKeyForTarget: (target: ResolvedComboTarget) => string | null;
/** Session-stickiness result — the attempt loop reads `.messageHash` on success/failure. */
sticky: ApplyStickinessResult;
preScreenMap: Map<string, PreScreenResult>;
}
export type ResolveComboTargetPipelineResult =
{ earlyResponse: Response } | ResolvedComboTargetPipeline;
type WeightedResolution = ReturnType<typeof resolveWeightedTargets> | null;
type WeightedStepGroups =
Array<{ step: ComboRuntimeStep; targets: ResolvedComboTarget[] }> | undefined;
/**
* Weighted-strategy eligibility predicate: a step counts as selectable only when at
* least one of its targets clears the provider breaker, the connection cooldown, the
* per-model lockout and the caller's availability probe.
*/
async function isTargetSelectableForWeighted(
target: ResolvedComboTarget,
resilienceSettings: ResilienceSettings,
isModelAvailable?: IsModelAvailable
): Promise<boolean> {
const rawModel = parseModel(target.modelStr).model || target.modelStr;
if (target.provider && getCircuitBreaker(target.provider).getStatus().state === "OPEN")
return false;
if (
resilienceSettings.providerCooldown.enabled &&
Boolean(target.provider && target.provider !== "unknown") &&
isProviderInCooldown(target.provider, target.connectionId ?? undefined, resilienceSettings)
) {
return false;
}
if (
target.provider &&
rawModel &&
isModelLocked(target.provider, target.connectionId || "", rawModel)
) {
return false;
}
return isModelAvailable ? await isModelAvailable(target.modelStr, target) : true;
}
/**
* #2562: Expand provider-wildcard steps (e.g. `fta/*`, `openai/gpt-4*`) into
* concrete model entries sourced from the live synced-models catalog + registry.
* Must run before any step-group / target resolution so that wildcard-originated
* steps are treated identically to hand-authored entries by all downstream logic
* (including the sticky-weighted eligibility pass below).
*/
async function expandComboWildcards(
combo: ComboLike,
allCombos: ComboCollectionLike
): Promise<{ expandedCombo: ComboLike; expandedAllCombos: ComboCollectionLike }> {
const expandedCombo = await expandProviderWildcardsInCombo(combo);
const expandedAllCombos = allCombos
? Array.isArray(allCombos)
? await expandProviderWildcardsInCollection(allCombos as ComboLike[])
: {
...allCombos,
combos: await expandProviderWildcardsInCollection(
((allCombos as { combos?: ComboLike[] }).combos || []) as ComboLike[]
),
}
: allCombos;
return { expandedCombo, expandedAllCombos };
}
/** LRU-evict the oldest sticky-weighted entry once the counter map is at capacity. */
function evictOldestWeightedSticky(strategy: string, comboName: string): void {
if (
strategy === "weighted" &&
!weightedStickyTargets.has(comboName) &&
weightedStickyTargets.size >= MAX_RR_COUNTERS
) {
const oldest = weightedStickyTargets.keys().next().value;
if (oldest !== undefined) weightedStickyTargets.delete(oldest);
}
}
/** Resolve the weighted step groups and the subset whose targets are still selectable. */
async function collectWeightedEligibility(
expandedCombo: ComboLike,
expandedAllCombos: ComboCollectionLike,
resilienceSettings: ResilienceSettings,
isModelAvailable?: IsModelAvailable
): Promise<{ stepGroups: WeightedStepGroups; weightedEligibleKeys: Set<string> }> {
const weightedEligibleKeys = new Set<string>();
const stepGroups = resolveWeightedStepGroups(expandedCombo, expandedAllCombos);
for (const group of stepGroups) {
const availability = await Promise.all(
group.targets.map((target) =>
isTargetSelectableForWeighted(target, resilienceSettings, isModelAvailable)
)
);
if (availability.some(Boolean)) weightedEligibleKeys.add(group.step.executionKey);
}
return { stepGroups, weightedEligibleKeys };
}
/**
* Honor the persisted sticky-weighted pin only while its step is still eligible;
* drop the stored pin otherwise (and whenever stickiness is off for this combo).
*/
function resolveStickyWeightedKey(
strategy: string,
comboName: string,
stickyWeightedLimit: number,
weightedEligibleKeys: Set<string>
): string | null {
const rawStickyWeightedKey =
strategy === "weighted" ? getStickyWeightedExecutionKey(comboName, stickyWeightedLimit) : null;
const stickyWeightedKey =
rawStickyWeightedKey && weightedEligibleKeys.has(rawStickyWeightedKey)
? rawStickyWeightedKey
: null;
if (strategy !== "weighted" || stickyWeightedLimit <= 1) {
weightedStickyTargets.delete(comboName);
} else if (rawStickyWeightedKey && !stickyWeightedKey) {
weightedStickyTargets.delete(comboName);
}
return stickyWeightedKey;
}
/** Full weighted-strategy resolution: eviction → eligibility → sticky pin → ordering. */
async function resolveWeightedSelection(
deps: ResolveComboTargetPipelineDeps,
expandedCombo: ComboLike,
expandedAllCombos: ComboCollectionLike,
stickyWeightedLimit: number
): Promise<{ weightedResolution: WeightedResolution; stickyWeightedKey: string | null }> {
const { strategy } = deps;
const comboName = deps.combo.name;
evictOldestWeightedSticky(strategy, comboName);
let stepGroups: WeightedStepGroups;
let weightedEligibleKeys = new Set<string>();
if (strategy === "weighted") {
const eligibility = await collectWeightedEligibility(
expandedCombo,
expandedAllCombos,
deps.resilienceSettings,
deps.isModelAvailable
);
stepGroups = eligibility.stepGroups;
weightedEligibleKeys = eligibility.weightedEligibleKeys;
}
const stickyWeightedKey = resolveStickyWeightedKey(
strategy,
comboName,
stickyWeightedLimit,
weightedEligibleKeys
);
const weightedResolution =
strategy === "weighted"
? resolveWeightedTargets(
expandedCombo,
expandedAllCombos,
stickyWeightedKey,
weightedEligibleKeys,
stepGroups
)
: null;
return { weightedResolution, stickyWeightedKey };
}
/** Maps an attempted target back to the weighted step it came from (sticky write-back). */
function buildWeightedStepKeyMapper(
weightedResolution: WeightedResolution
): (target: ResolvedComboTarget) => string | null {
return (target: ResolvedComboTarget): string | null => {
if (!weightedResolution?.orderedSteps) return null;
const step = weightedResolution.orderedSteps.find(
(entry) =>
target.executionKey === entry.executionKey ||
target.executionKey.startsWith(entry.executionKey + ">")
);
return step?.executionKey || null;
};
}
/** 400 rejection for a request no target in the pool can physically accept. */
function buildContextOverflowResponse(
overflow: { requiredContextTokens: number; maxKnownContextTokens: number },
orderedTargets: ResolvedComboTarget[],
log: ComboLogger
): Response {
const { requiredContextTokens, maxKnownContextTokens } = overflow;
log.warn(
"COMBO",
`Request context exceeds every known target limit (${requiredContextTokens} > ${maxKnownContextTokens} tokens)`
);
return errorResponseWithComboDiagnostics(
400,
`Request requires approximately ${requiredContextTokens} tokens, but the largest known context limit in this combo is ${maxKnownContextTokens} tokens. Reduce or compact the request context.`,
{
poolSize: orderedTargets.length,
attempted: 0,
excluded: orderedTargets.map((target) => ({
provider: target.provider,
model: target.modelStr,
reason: "context_window",
})),
attemptOrder: [],
terminalReason: "context_length_exceeded",
},
{ code: "context_length_exceeded", type: "invalid_request_error" }
);
}
function logTargetPoolSize(
strategy: string,
allCombos: ComboCollectionLike,
orderedTargets: ResolvedComboTarget[],
stickyWeightedKey: string | null,
log: ComboLogger
): void {
if (strategy === "weighted") {
log.info(
"COMBO",
`Weighted selection${stickyWeightedKey ? " (sticky)" : ""}${allCombos ? " with nested resolution" : ""}: ${orderedTargets.length} total targets`
);
} else if (allCombos) {
log.info("COMBO", `${strategy} with nested resolution: ${orderedTargets.length} total targets`);
}
}
/**
* Pipeline dispatch: route smart/pipeline-enabled combos through the multi-stage
* pipeline. Returns the finished Response, or null to fall through to standard
* auto routing (pipeline disabled, below token threshold, or dispatch failure).
*/
async function dispatchSmartPipeline(
deps: ResolveComboTargetPipelineDeps
): Promise<Response | null> {
const { body, combo, strategy, config, settings, signal, log } = deps;
if (strategy !== "auto") return null;
const autoParsed = parseAutoPrefix(combo.name);
const autoVariant = autoParsed.valid ? autoParsed.variant : undefined;
if (autoVariant !== "smart" && !config.pipeline_enabled) return null;
try {
const pipelineRaw = await handlePipelineCombo({
body,
combo,
handleChatCore: deps.handleSingleModelWithTimeout,
log: {
info: log.info,
warn: log.warn,
error: log.error ?? log.warn,
},
settings: settings ?? {},
signal: signal ?? undefined,
});
// handlePipelineCombo resolves to a PipelineResult (buffered text) or,
// in the streaming-final-stage case, a Response. Callers downstream
// (chat.ts → withSessionHeader) require a Response, so adapt the
// PipelineResult here instead of leaking the raw object.
return pipelineRaw instanceof Response ? pipelineRaw : buildPipelineResponse(pipelineRaw, body);
} catch (pipelineErr) {
logPipelineFallthrough(pipelineErr, log);
return null;
}
}
function logPipelineFallthrough(pipelineErr: unknown, log: ComboLogger): void {
const pipelineMsg = pipelineErr instanceof Error ? pipelineErr.message : "";
if (pipelineMsg === "PIPELINE_DISABLED") {
log.info("COMBO", "Pipeline disabled, falling through to standard auto routing");
} else if (pipelineMsg === "PIPELINE_TOKEN_THRESHOLD") {
log.info(
"COMBO",
"Pipeline skipped (prompt below token threshold), falling through to standard auto routing"
);
} else {
log.warn("COMBO", "Pipeline dispatch failed, falling through to standard auto routing", {
err: pipelineErr,
});
}
}
/**
* Strategy ordering: the `auto` router for auto combos, the per-strategy chain for
* everything else. `autoUsedExplicitRouter` is the #4945 guard — when an explicit
* router (lkgp/cost/…) pinned orderedTargets[0], task-aware reordering below must
* refine only the fallback order, never override the router's primary choice.
*/
async function orderByStrategy(
deps: ResolveComboTargetPipelineDeps,
initialOrderedTargets: ResolvedComboTarget[]
): Promise<
| { earlyResponse: Response }
| { orderedTargets: ResolvedComboTarget[]; autoUsedExplicitRouter: boolean }
> {
const { strategy, body, combo, settings, config, log } = deps;
if (strategy === "auto") {
const autoResult = await resolveAutoStrategyOrder({
orderedTargets: initialOrderedTargets,
body,
combo,
settings,
config,
relayOptions: deps.relayOptions,
resilienceSettings: deps.resilienceSettings,
log,
buildAutoCandidates: deps.buildAutoCandidates,
});
if ("earlyResponse" in autoResult) return { earlyResponse: autoResult.earlyResponse };
return {
orderedTargets: autoResult.orderedTargets,
autoUsedExplicitRouter: autoResult.autoUsedExplicitRouter,
};
}
const orderedTargets = await applyStrategyOrdering(strategy, initialOrderedTargets, {
combo,
config,
body,
log,
apiKeyAllowedConnections: deps.apiKeyAllowedConnections,
});
return { orderedTargets, autoUsedExplicitRouter: false };
}
/**
* Continuity + eligibility filters: cache-strategy affinity, session stickiness,
* eval-score ordering, request compatibility and per-combo context requirements.
*
* May return `{ earlyResponse }` when hard capability filters (#8488 / #8494) empty
* the pool — tools / vision / structured_output fail closed as 400 capability_mismatch
* unless `compatFilterFailOpen` is set on the combo config or settings.
*/
async function applyContinuityFilters(
deps: ResolveComboTargetPipelineDeps,
initialOrderedTargets: ResolvedComboTarget[]
): Promise<
| { orderedTargets: ResolvedComboTarget[]; sticky: ApplyStickinessResult }
| { earlyResponse: Response }
> {
const { strategy, body, combo, config, settings, log, relayOptions } = deps;
// An explicit cache-optimized combo outranks the global cache-affinity default,
// but only protects its ordering when this request actually produced a reusable
// cache key. Cache misses retain the normal session/eval routing behavior.
const cacheStrategyAffinityApplied =
strategy === "cache-optimized" && applyPromptCacheAffinity(initialOrderedTargets, body).applied;
// #6168: session stickiness opt-out. Per-combo `config.disableSessionStickiness`
// overrides the global `settings.disableSessionStickiness` fallback (default false,
// preserving the #3825 prompt-cache/504 fix). When disabled, skip the reorder and
// treat the result as a no-op so the recordStickyBinding write-back below is skipped.
const disableSessionStickiness =
cacheStrategyAffinityApplied ||
resolveDisableSessionStickiness(
config as Record<string, unknown> | null | undefined,
settings as Record<string, unknown> | null | undefined
);
const sticky: ApplyStickinessResult = disableSessionStickiness
? { targets: initialOrderedTargets, messageHash: null, stuck: false }
: await applySessionStickiness(
initialOrderedTargets,
// #7270: normalize both wire shapes (.messages / Responses-API .input) so the
// stickiness key is derivable on the /v1/responses surface, not just Chat Completions.
normalizeStickinessMessages(body as { messages?: unknown; input?: unknown })
);
let orderedTargets = sticky.targets;
if (!cacheStrategyAffinityApplied) {
orderedTargets = orderTargetsByEvalScores(orderedTargets, config.evalRouting, log);
}
// #8488 / #8494: fail closed when hard capability filters empty the pool.
// Opt-in escape hatch: combo.config.compatFilterFailOpen OR settings.compatFilterFailOpen.
const compatFilterFailOpen =
(config as { compatFilterFailOpen?: unknown }).compatFilterFailOpen === true ||
(settings as { compatFilterFailOpen?: unknown } | null | undefined)?.compatFilterFailOpen ===
true;
const preCompatTargets = orderedTargets;
orderedTargets = filterTargetsByRequestCompatibility(orderedTargets, body, log, undefined, {
failOpen: compatFilterFailOpen,
});
if (orderedTargets.length === 0 && preCompatTargets.length > 0) {
const exhaustion = describeCapabilityFilterExhaustion(preCompatTargets, body, combo.name);
if (exhaustion) {
// Match handleComboChat: only track failures under context-cache protection pins.
const effectiveSessionId: string | null = combo.context_cache_protection
? (relayOptions?.sessionId ?? null)
: null;
recordComboFailure(effectiveSessionId, combo.name);
return {
earlyResponse: errorResponseWithComboDiagnostics(
400,
exhaustion.message,
{
poolSize: preCompatTargets.length,
attempted: 0,
excluded: exhaustion.excluded,
attemptOrder: [],
terminalReason: exhaustion.terminalReason,
recovery: buildRecoveryHint("no_executable_targets"),
},
{ code: "capability_mismatch", type: "invalid_request_error" }
),
};
}
}
orderedTargets = applyContextRequirements(orderedTargets, config.contextRequirements, log);
return { orderedTargets, sticky };
}
/**
* Task-aware reordering: only active for strategies
* ["smart","task","task-aware","task_aware","auto"]. Additive — does not affect any
* of the other 15 strategies.
*/
function applyTaskAwareOrdering(
deps: ResolveComboTargetPipelineDeps,
orderedTargets: ResolvedComboTarget[],
autoUsedExplicitRouter: boolean
): ResolvedComboTarget[] {
const { strategy, body, log } = deps;
if (!isTaskRoutingStrategy(strategy)) return orderedTargets;
const task = classifyTask(body);
const conversationCacheKey = getConversationCacheKey(body);
const taskReordered = reorderByTaskWeight(orderedTargets, task);
// #4945 regression guard: when an explicit auto router (lkgp/cost/…) pinned
// orderedTargets[0], keep that primary choice and let task-aware refine only
// the fallback tail — otherwise task weighting silently defeats the operator's
// chosen LKGP/cost selection. reorderByTaskWeight returns the same target
// objects (no clone), so identity filtering is safe.
const pinnedFirst = autoUsedExplicitRouter ? orderedTargets[0] : undefined;
const nextOrder = pinnedFirst
? [pinnedFirst, ...taskReordered.filter((t) => t !== pinnedFirst)]
: taskReordered;
if (nextOrder[0]?.modelStr !== orderedTargets[0]?.modelStr) {
const reasons =
Array.isArray(task.reasons) && task.reasons.length > 0 ? ` (${task.reasons.join(",")})` : "";
log.info(
"COMBO",
`task-route task=${task.level}${reasons} cacheKey=${conversationCacheKey ?? "none"} → ${nextOrder[0]?.modelStr}`
);
}
return nextOrder;
}
/**
* Prompt-cache affinity is skipped when the auto scorer already weights cacheAffinity
* itself — otherwise the same signal would be applied twice.
*/
function isPromptCacheAffinityEnabled(
strategy: string,
combo: ComboLike,
config: ReturnType<typeof resolveComboSetupConfig>,
settings?: Record<string, unknown> | null
): boolean {
const autoConfigForCacheWeight =
strategy === "auto"
? ((combo.autoConfig ||
((config as Record<string, unknown>).auto &&
typeof (config as Record<string, unknown>).auto === "object"
? (config as Record<string, unknown>).auto
: null) ||
config) as Record<string, unknown>)
: null;
const autoWeightsForCache =
autoConfigForCacheWeight?.weights && typeof autoConfigForCacheWeight.weights === "object"
? (autoConfigForCacheWeight.weights as Record<string, unknown>)
: null;
const autoUsesCacheScore = Number(autoWeightsForCache?.cacheAffinity) > 0;
return settings?.promptCacheAffinityEnabled !== false && !autoUsesCacheScore;
}
/**
* Keep the stronger continuity decision (session pin / explicit auto-router pin) at
* the head of the cache-affinity ordering rather than letting affinity override it.
*/
function protectFirstTarget(
affinityTargets: ResolvedComboTarget[],
protectedOriginal: ResolvedComboTarget | false | undefined
): ResolvedComboTarget[] {
const protectedFirst = protectedOriginal
? (affinityTargets.find(
(target) =>
target === protectedOriginal ||
target.executionKey === protectedOriginal.executionKey ||
target.executionKey.startsWith(`${protectedOriginal.executionKey}@`)
) ?? protectedOriginal)
: null;
return protectedFirst
? [protectedFirst, ...affinityTargets.filter((target) => target !== protectedFirst)]
: affinityTargets;
}
/**
* Prompt-cache locality is applied after request eligibility and task routing.
* Session stickiness and explicit auto-router pins remain stronger continuity
* decisions; quota, health, and circuit-breaker gates still run per attempt.
*/
async function applyPromptCacheStage(
deps: ResolveComboTargetPipelineDeps,
orderedTargets: ResolvedComboTarget[],
stickyStuck: boolean,
autoUsedExplicitRouter: boolean
): Promise<ResolvedComboTarget[]> {
const { strategy, body, combo, config, settings, log } = deps;
const promptCacheAffinityEnabled = isPromptCacheAffinityEnabled(
strategy,
combo,
config,
settings
);
const promptCacheAffinityTargets =
promptCacheAffinityEnabled && resolvePromptCacheAffinityKey(body)
? await expandPromptCacheAffinityTargets(orderedTargets)
: orderedTargets;
const promptCacheAffinity = applyPromptCacheAffinity(
promptCacheAffinityTargets,
body,
promptCacheAffinityEnabled
);
if (!promptCacheAffinity.applied) return orderedTargets;
const protectedOriginal =
shouldProtectOriginalFirst(stickyStuck, autoUsedExplicitRouter, strategy) && orderedTargets[0];
const nextTargets = protectFirstTarget(promptCacheAffinity.targets, protectedOriginal);
log.debug?.("COMBO", "Prompt-cache affinity applied", {
source: promptCacheAffinity.source,
fingerprint: promptCacheAffinity.fingerprint,
targetCount: nextTargets.length,
});
return nextTargets;
}
export async function resolveComboTargetPipeline(
deps: ResolveComboTargetPipelineDeps
): Promise<ResolveComboTargetPipelineResult> {
const { body, combo, strategy, config, allCombos, log, isModelAvailable } = deps;
const { expandedCombo, expandedAllCombos } = await expandComboWildcards(combo, allCombos);
const stickyWeightedLimit = clampStickyWeightedTargetLimit(
(config as Record<string, unknown>).stickyWeightedLimit
);
const { weightedResolution, stickyWeightedKey } = await resolveWeightedSelection(
deps,
expandedCombo,
expandedAllCombos,
stickyWeightedLimit
);
const getWeightedStepKeyForTarget = buildWeightedStepKeyMapper(weightedResolution);
let orderedTargets =
strategy === "weighted"
? weightedResolution?.orderedTargets || []
: resolveComboTargets(
expandedCombo,
expandedAllCombos,
clampComboDepth(config.maxComboDepth)
);
orderedTargets = await applyRequestTagRouting(orderedTargets, body, log);
const overflow = getKnownContextOverflow(orderedTargets, body);
if (overflow) {
return { earlyResponse: buildContextOverflowResponse(overflow, orderedTargets, log) };
}
logTargetPoolSize(strategy, allCombos, orderedTargets, stickyWeightedKey, log);
const pipelineResponse = await dispatchSmartPipeline(deps);
if (pipelineResponse) return { earlyResponse: pipelineResponse };
const ordering = await orderByStrategy(deps, orderedTargets);
if ("earlyResponse" in ordering) return ordering;
const { autoUsedExplicitRouter } = ordering;
const continuity = await applyContinuityFilters(deps, ordering.orderedTargets);
if ("earlyResponse" in continuity) return continuity;
orderedTargets = applyTaskAwareOrdering(deps, continuity.orderedTargets, autoUsedExplicitRouter);
orderedTargets = await applyPromptCacheStage(
deps,
orderedTargets,
continuity.sticky.stuck,
autoUsedExplicitRouter
);
// Parallel pre-screen: check provider profiles and model availability for all targets
// Only runs for priority strategy where sequential checking causes latency
const preScreenMap =
strategy === "priority"
? await preScreenTargets(orderedTargets, isModelAvailable).catch(
() => new Map<string, PreScreenResult>()
)
: new Map<string, PreScreenResult>();
return {
orderedTargets,
stickyWeightedLimit,
getWeightedStepKeyForTarget,
sticky: continuity.sticky,
preScreenMap,
};
}

1
scripts/check/check-known-symbols.ts
View File

@@ -492,6 +492,7 @@ async function main(): Promise<void> {
// #3501: the fusion/pipeline dispatch branches moved here with the prelude
// extraction; the `strategy === "..."` checks are unchanged, just relocated.
"open-sse/services/combo/dispatchPrelude.ts",
"open-sse/services/combo/targetResolution.ts",
];
const comboSource = comboDispatchFiles
.map((rel) => readFileSync(resolvePath(REPO_ROOT, rel), "utf8"))

148
tests/unit/combo-target-resolution-split.test.ts Normal file
View File

@@ -0,0 +1,148 @@
import test from "node:test";
import assert from "node:assert/strict";
import fs from "node:fs";
import os from "node:os";
import path from "node:path";
// Split guard for the #3501 god-file decomposition (PR 2): the target-resolution
// stage of handleComboChat (wildcard expansion → weighted step groups → known
// context overflow → strategy ordering → stickiness/eval/compat/context filters →
// task-aware reorder → prompt-cache affinity → pre-screen) was extracted verbatim
// into resolveComboTargetPipeline. These tests pin the leaf's own contract: the
// shape it hands back to the attempt loop, the pass-through ordering for the plain
// `priority` path, and the `earlyResponse` exit for a request that exceeds every
// target's known context window. The strategy-specific branches stay covered
// end-to-end by the combo-* consumer suites through combo.ts.
const TEST_DATA_DIR = fs.mkdtempSync(path.join(os.tmpdir(), "omniroute-combo-target-resolution-"));
const ORIGINAL_DATA_DIR = process.env.DATA_DIR;
process.env.DATA_DIR = TEST_DATA_DIR;
const core = await import("../../src/lib/db/core.ts");
const { saveModelsDevCapabilities, clearModelsDevCapabilities } =
await import("../../src/lib/modelsDevSync.ts");
const { resolveComboTargetPipeline } =
await import("../../open-sse/services/combo/targetResolution.ts");
test.after(() => {
core.resetDbInstance();
if (ORIGINAL_DATA_DIR === undefined) {
delete process.env.DATA_DIR;
} else {
process.env.DATA_DIR = ORIGINAL_DATA_DIR;
}
fs.rmSync(TEST_DATA_DIR, { recursive: true, force: true });
});
test.beforeEach(() => {
clearModelsDevCapabilities();
});
const noopLog = { info() {}, warn() {}, error() {}, debug() {} } as never;
function capabilityEntry(limitContext: number) {
return {
tool_call: true,
reasoning: false,
attachment: false,
structured_output: true,
temperature: true,
modalities_input: JSON.stringify(["text"]),
modalities_output: JSON.stringify(["text"]),
knowledge_cutoff: null,
release_date: null,
last_updated: null,
status: null,
family: null,
open_weights: false,
limit_context: limitContext,
limit_input: limitContext,
limit_output: 4096,
interleaved_field: null,
};
}
const deps = (overrides: Record<string, unknown> = {}): never =>
({
body: { messages: [{ role: "user", content: "hi" }] },
combo: { id: "c1", name: "c1", models: ["openai/gpt-4o", "anthropic/claude-3"], config: {} },
strategy: "priority",
config: {},
settings: null,
allCombos: null,
relayOptions: null,
signal: null,
apiKeyAllowedConnections: null,
log: noopLog,
resilienceSettings: { providerCooldown: { enabled: false } },
isModelAvailable: undefined,
handleSingleModelWithTimeout: async () => new Response("{}"),
buildAutoCandidates: async () => [],
...overrides,
}) as never;
test("exports resolveComboTargetPipeline", () => {
assert.equal(typeof resolveComboTargetPipeline, "function");
});
test("priority strategy resolves combo models into orderedTargets in declared order", async () => {
const result = await resolveComboTargetPipeline(deps());
assert.ok(!("earlyResponse" in result), "expected a resolved pipeline, not an early response");
if ("earlyResponse" in result) return;
assert.deepEqual(
result.orderedTargets.map((t) => t.modelStr),
["openai/gpt-4o", "anthropic/claude-3"]
);
});
test("returns the derived values the attempt loop consumes", async () => {
const result = await resolveComboTargetPipeline(deps());
assert.ok(!("earlyResponse" in result));
if ("earlyResponse" in result) return;
assert.equal(typeof result.stickyWeightedLimit, "number");
assert.equal(typeof result.getWeightedStepKeyForTarget, "function");
assert.ok(result.preScreenMap instanceof Map);
assert.equal(result.sticky.messageHash === null || typeof result.sticky.messageHash, "string");
// Non-weighted strategies have no weighted step resolution, so the mapper is a
// constant null — the sticky-weighted write-back in combo.ts is then skipped.
assert.equal(result.getWeightedStepKeyForTarget(result.orderedTargets[0]), null);
});
test("an empty combo yields an empty target pool (combo.ts turns it into a 404)", async () => {
const result = await resolveComboTargetPipeline(deps({ combo: { name: "empty", models: [] } }));
assert.ok(!("earlyResponse" in result));
if ("earlyResponse" in result) return;
assert.deepEqual(result.orderedTargets, []);
});
test("request exceeding every known context window returns a 400 earlyResponse", async () => {
saveModelsDevCapabilities({
"unit-target-resolution": {
tiny: capabilityEntry(8_000),
small: capabilityEntry(16_000),
},
});
const result = await resolveComboTargetPipeline(
deps({
combo: {
id: "c2",
name: "known-context-overflow",
models: ["unit-target-resolution/tiny", "unit-target-resolution/small"],
config: {},
},
body: { messages: [{ role: "user", content: "word ".repeat(200_000) }] },
})
);
assert.ok("earlyResponse" in result, "expected a context-overflow early response");
if (!("earlyResponse" in result)) return;
assert.equal(result.earlyResponse.status, 400);
const body = (await result.earlyResponse.json()) as {
error?: { code?: string };
diagnostics?: { terminalReason?: string; attempted?: number };
};
assert.equal(body.error?.code, "context_length_exceeded");
assert.equal(body.diagnostics?.terminalReason, "context_length_exceeded");
assert.equal(body.diagnostics?.attempted, 0);
});