From 0eeb8f45c03100641f796de48dddaf09b1987680 Mon Sep 17 00:00:00 2001 From: MumuTW <42820974+MumuTW@users.noreply.github.com> Date: Tue, 28 Jul 2026 06:13:31 +0800 Subject: [PATCH] refactor(sse): extract combo target resolution into combo/targetResolution.ts (#8592) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * refactor(sse): extract combo dispatch prelude into combo/dispatchPrelude.ts Pure move, no behaviour change. First of ~7 PRs decomposing the combo.ts god-file (#3501). handleComboChat evaluates a series of dispatch branches before it ever reaches target resolution or the sequential attempt loop. None of them iterate targets in priority order or need the failover/retry/credential gate machinery that follows, so they move to a leaf: - context-cache pin routing (Fix #679), including the pinIsDurablyUnhealthy / isPinnedModelDurablyUnhealthy health gate - fusion panel dispatch + the #6455 misconfiguration warn - pipeline chaining - nested combo-ref execute-mode runtime-unit dispatch Only the chaos and round-robin hand-offs stay inline (11 and 13 lines); extracting those would be pure indirection. open-sse/services/combo.ts 3642 -> 3341 (-301) open-sse/services/combo/dispatchPrelude.ts: 619 (under the 800 cap) Each helper keeps the fall-through protocol the inline blocks had: return a Response to OWN the request, return null to fall through. A flipped null/Response would silently bypass the whole combo strategy, so the new tests pin both directions for every branch. combo.ts re-exports pinIsDurablyUnhealthy so combo-pin-health-gate.test.ts keeps resolving. The leaf takes handleComboChat as a `runCombo` parameter instead of importing it, so combo/ keeps zero back-edges into combo.ts. Complexity-neutral: the first cut added +3 violations (two max-lines-per-function, one complexity) inside the new leaf, so evaluatePinnedResponse, orderRuntimeUnits, recordRuntimeUnitStickySuccess and buildBaseOptions were split out. check:complexity now measures 2169 and check:cognitive-complexity 956 — identical to the pristine base. * test(sse): close the dispatch-prelude coverage holes found by mutation testing An adversarial mutation audit of the suite added in the previous commit found it guarded the fall-through protocol well but asserted almost nothing about what the helpers do once they OWN the request. 5 of 12 seeded mutations survived. Worst case: deleting the pinned-model dispatch call outright left all 12 tests green. Three holes, now closed (8 tests -> 20): Hole A — the honored-pin path had zero coverage. Both existing pin tests DROP the pin, so the dispatch, the 200-but-empty quality gate, the [408, 429, 500, 502, 503, 504] failover list and the catch(pinErr) branch were unguarded — exactly the logic the 2026-06-21 / 2026-06-22 incident comments call load-bearing. Adds five tests over a seeded healthy provider connection so the pin is actually honored. Hole B — orderRuntimeUnits was only ever driven with `priority`, which is a no-op through it. Four of five strategy branches could be deleted with nothing failing. Adds round-robin rotation and weighted sticky ordering tests. Hole C — recordRuntimeUnitStickySuccess never did anything under test: both its guards need weighted/round-robin, so an early return changed nothing. Covered by the new sticky-batch test. Verified by re-running the mutations rather than assuming: all 7 that previously survived (delete-pin-dispatch, serve-despite-failed-quality, never-fail-over-on-transient, rr-counter-not-advanced, rotation-removed, weighted-sticky-skipped, sticky-recording-no-op) are now killed. The first sticky-batch test I wrote was itself vacuous — asserting "same unit twice" holds equally when the recording helper is stubbed out, since nothing advances the counter either. It now asserts the batch runs out and rotation resumes on the third dispatch, which is what actually distinguishes the two. Also restores API_KEY_SECRET in test.after; it was set at module load and never put back, inconsistent with the DATA_DIR handling beside it. * fix(ci): teach known-symbols gate the relocated fusion/pipeline dispatch The combo sub-check of check:known-symbols asserts every canonical routing strategy has a real dispatch branch. It scanned a hardcoded file list and matched only `strategy === "..."`, so the prelude extraction tripped it twice: [combo] 2 estratégia(s) canônica(s) sem branch de despacho em combo.ts: ✗ fusion ✗ pipeline Both branches are still wired — they just moved to combo/dispatchPrelude.ts and took the early-return guard form `if (strategy !== "fusion") return null;` that extracting a branch into a `tryXDispatch()` leaf naturally produces. Two changes, both extending existing precedent (the list already carries the Block J leaves for the same reason): - register combo/dispatchPrelude.ts in comboDispatchFiles - widen the extractor to `strategy [!=]== "..."` so the inverted guard counts Loose `==`/`!=` stay rejected, and no `handledNotCanonical` fallout: the gate now reports 20 canonical strategies, all 20 via despacho. * chore(ci): register combo-dispatch-prelude test in stryker tap.testFiles check:mutation-test-coverage --strict failed once the known-symbols fix let Fast Quality Gates advance to it: ✗ 2 covering unit test(s) across 2 module(s) are missing from stryker.conf.json tap.testFiles open-sse/services/combo/comboStructure.ts open-sse/services/combo/rrState.ts The new tests/unit/combo-dispatch-prelude.test.ts exercises both modules, and both are already in stryker's mutate list, so without the registration its mutant kills would not have counted toward the nightly mutation gate. Note (unchanged, still out of scope): combo/dispatchPrelude.ts itself is not in stryker's `mutate` list. Adding it would widen the nightly mutation surface, which is a separate call from fixing this drift. * docs(changelog): add fragment for #8582 combo dispatch prelude * refactor(sse): extract combo target resolution into combo/targetResolution.ts Pure move, no behaviour change. Lifts the target-resolution stage of handleComboChat — everything between the dispatch prelude and the attempt loop — into a new leaf, open-sse/services/combo/targetResolution.ts. Moved verbatim: provider-wildcard expansion, weighted step-group resolution + sticky-weighted eligibility, request-tag routing, the known-context-overflow early return, the smart/pipeline-enabled auto dispatch, auto-strategy ordering, per-strategy ordering, cache-strategy affinity, session stickiness, eval scores, request-compatibility + context-requirement filters, task-aware reordering, prompt-cache affinity, and the priority-strategy pre-screen. The three early exits become an { earlyResponse } result so the host decides to return them (same pattern as resolveAutoStrategyOrder). The values the attempt loop still reads — orderedTargets, stickyWeightedLimit, getWeightedStepKeyForTarget, the session-stickiness result and preScreenMap — are returned instead of closed over. Loop config (maxRetries, retryDelayMs, fallbackDelayMs, maxSetRetries, setRetryDelayMs) stays in combo.ts. buildAutoCandidates is dependency-injected because it lives in combo.ts, so the leaf keeps zero back-edges into its host. combo.ts 3640 -> 3321 lines; new leaf 484 lines (under the 800 cap). Part of the #3501 god-file decomposition campaign. * refactor(sse): split targetResolution into stage helpers, ratchet combo.ts file-size baseline Follow-up to the target-resolution extraction: the moved region landed as one 311-line function, which converted inline code inside the (already-violating) handleComboChat into a NEW separately-counted violating function — check:complexity 2169 -> 2171 and check:cognitive-complexity 956 -> 957. Split resolveComboTargetPipeline along its natural stage boundaries into 14 helpers (wildcard expansion, weighted eviction/eligibility/sticky-key/selection, step-key mapper, context-overflow response, pool-size log, smart-pipeline dispatch and its fall-through logger, strategy ordering, continuity filters, task-aware ordering, prompt-cache enablement/first-target protection/affinity stage). Each stage takes the previous stage's output and returns the next; still a pure move. The leaf now contributes ZERO complexity, max-lines-per-function and cognitive-complexity violations. Both ratchets are back at base 4053e2314 values: check:complexity 2169, check:cognitive-complexity 956. (Both still print RED against their frozen ceilings 2130/951 — pre-existing base-red per #8580.) Also ratchets ONLY the open-sse/services/combo.ts entry in config/quality/file-size-baseline.json from 3642 to 3322, with a justification note in the file's existing style. No sweep of unrelated entries. * chore: stack targetResolution on dispatchPrelude tip, rebank + skills Rebased onto refactor/combo-dispatch-prelude. Keep both leaves in check-known-symbols. Regenerate file-size baseline; sync agent skills. * fix(sse): restore #8494 capability fail-closed after targetResolution extract Stacking targetResolution onto the dispatchPrelude tip dropped the #8488/#8494 compatFilterFailOpen wiring: hard capability filters emptied the pool into a generic 404 no_executable_targets, and fail-open never re-admitted the pool. Restore describeCapabilityFilterExhaustion earlyResponse in applyContinuityFilters and the matching round-robin path, then rebank the file-size baseline for tip growth the incomplete prior rebank missed. * fix(sse): realign model-lockout cooldown options with the post-#8254 type This branch predates #8254, which renamed the recordModelLockoutFailure option `exactCooldownVerified` -> `exactCooldownIsUpstreamReset` and changed combo.ts's predicate from `lockoutHintVerified` (#8393's `lockoutHintMs > 0`) to `lockoutHintMs > mlSettings.baseCooldownMs`. Rebasing onto the current tip brought the renamed type without updating these two call sites, so typecheck:core failed with TS2353 at both. Restores the base expression verbatim rather than re-wiring `lockoutHintVerified` under the new name. The base predicate is the correct one: selectLockoutCooldownMs returns the parsed hint ONLY when `lockoutHintMs > baseCooldownMs`, and otherwise returns 0 or a synthetic baseCooldownMs — so `lockoutHintMs > 0` would mark a synthetic cooldown as an upstream reset and let it bypass the #7940 maxCooldownMs cap, which is the bug #8254 fixed. --------- Co-authored-by: MumuTW Co-authored-by: diegosouzapw Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com> --- .../8592-combo-target-resolution.md | 1 + open-sse/services/combo.ts | 393 +--------- open-sse/services/combo/targetResolution.ts | 714 ++++++++++++++++++ scripts/check/check-known-symbols.ts | 1 + .../combo-target-resolution-split.test.ts | 148 ++++ 5 files changed, 886 insertions(+), 371 deletions(-) create mode 100644 changelog.d/maintenance/8592-combo-target-resolution.md create mode 100644 open-sse/services/combo/targetResolution.ts create mode 100644 tests/unit/combo-target-resolution-split.test.ts diff --git a/changelog.d/maintenance/8592-combo-target-resolution.md b/changelog.d/maintenance/8592-combo-target-resolution.md new file mode 100644 index 0000000000..dfb8efd2b2 --- /dev/null +++ b/changelog.d/maintenance/8592-combo-target-resolution.md @@ -0,0 +1 @@ +- **refactor(sse):** extract combo target resolution into `combo/targetResolution.ts` (`resolveComboTargetPipeline`) — pure move, no behaviour change; #3501 PR 2 of ~7 ([#8592](https://github.com/diegosouzapw/OmniRoute/pull/8592)) — thanks @MumuTW diff --git a/open-sse/services/combo.ts b/open-sse/services/combo.ts index a9c24d2ec8..5ae2518d34 100644 --- a/open-sse/services/combo.ts +++ b/open-sse/services/combo.ts @@ -65,10 +65,6 @@ import { phaseComboSetup } from "./combo/comboSetup.ts"; import { checkCredentialGate, logCredentialSkip } from "./credentialGate.ts"; import { emit } from "../../src/lib/events/eventBus"; import { notifyWebhookEvent } from "../../src/lib/webhookDispatcher"; -import { parseAutoPrefix } from "./autoCombo/autoPrefix.ts"; -import { resolveAutoStrategyOrder } from "./combo/resolveAutoStrategy.ts"; -import { applyStrategyOrdering } from "./combo/applyStrategyOrdering.ts"; -import { handlePipelineCombo, buildPipelineResponse } from "./autoCombo/pipelineRouter.ts"; import { type ProviderCandidate } from "./autoCombo/scoring.ts"; import { estimateTokens } from "./contextManager.ts"; import { getSessionConnection } from "./sessionManager.ts"; @@ -89,7 +85,6 @@ import { expandPromptCacheAffinityTargets, expandPromptCacheAffinityTargetsFromConnections, resolvePromptCacheAffinityKey, - shouldProtectOriginalFirst, } from "./combo/promptCacheAffinity.ts"; import type { CompressionMode } from "./compression/types.ts"; import { getCachedProviderConnections } from "../../src/lib/db/readCache"; @@ -113,7 +108,6 @@ import type { HandleRoundRobinOptions, ResolvedComboTarget, AutoProviderCandidate, - ComboRuntimeStep, HistoricalLatencyStatsEntry, } from "./combo/types.ts"; @@ -121,7 +115,6 @@ import { MAX_RR_COUNTERS, rrCounters, rrStickyTargets, - weightedStickyTargets, clampStickyRoundRobinTargetLimit, clampStickyWeightedTargetLimit, getStickyRoundRobinStartIndex, @@ -193,15 +186,12 @@ import { } from "./combo/providerWildcard.ts"; import { resolveShadowTargets, scheduleShadowRouting } from "./combo/shadowRouting.ts"; import { attemptCompatRejectedFallback } from "./combo/comboCompatFallback.ts"; -import { applyContextRequirements } from "./combo/contextRequirements.ts"; import { computeCompatRejectedTargets, describeCapabilityFilterExhaustion, filterTargetsByRequestCompatibility, resolveComboRuntimeUnits, resolveComboTargets, - resolveWeightedTargets, - resolveWeightedStepGroups, } from "./combo/comboStructure.ts"; import { getKnownContextOverflow } from "./combo/knownContextOverflow.ts"; import { @@ -219,22 +209,13 @@ import { calculateResetWindowAffinity, type ResetWindowConfig, } from "./combo/quotaScoring.ts"; -import { - fetchResetAwareQuotaWithCache, - preScreenTargets, - type PreScreenResult, -} from "./combo/quotaStrategies.ts"; +import { fetchResetAwareQuotaWithCache, preScreenTargets } from "./combo/quotaStrategies.ts"; import { buildAutoQuotaThresholds, resolveQuotaExhaustionCutoffForTarget, } from "./combo/quotaExhaustionCutoff.ts"; -import { - classifyTask, - getConversationCacheKey, - isTaskRoutingStrategy, - reorderByTaskWeight, -} from "./taskAwareRouting.ts"; import { expandTargetsByFingerprints } from "./combo/fingerprintExpansion.ts"; +import { resolveComboTargetPipeline } from "./combo/targetResolution.ts"; export { RESET_WINDOW_NAMES }; export { QUOTA_SOFT_DEPRIORITIZE_FACTOR, setCandidateQuotaSoftPenalty }; @@ -711,356 +692,26 @@ export async function handleComboChat({ const maxSetRetries = config.maxSetRetries ?? 0; const setRetryDelayMs = resolveDelayMs(config.setRetryDelayMs, 2000); - const isTargetSelectableForWeighted = async (target: ResolvedComboTarget): Promise => { - const rawModel = parseModel(target.modelStr).model || target.modelStr; - if (target.provider && getCircuitBreaker(target.provider).getStatus().state === "OPEN") - return false; - if ( - resilienceSettings.providerCooldown.enabled && - Boolean(target.provider && target.provider !== "unknown") && - isProviderInCooldown(target.provider, target.connectionId ?? undefined, resilienceSettings) - ) { - return false; - } - if ( - target.provider && - rawModel && - isModelLocked(target.provider, target.connectionId || "", rawModel) - ) { - return false; - } - return isModelAvailable ? await isModelAvailable(target.modelStr, target) : true; - }; - - // #2562: Expand provider-wildcard steps (e.g. `fta/*`, `openai/gpt-4*`) into - // concrete model entries sourced from the live synced-models catalog + registry. - // Must run before any step-group / target resolution so that wildcard-originated - // steps are treated identically to hand-authored entries by all downstream logic - // (including the sticky-weighted eligibility pass below). - const expandedCombo = await expandProviderWildcardsInCombo(combo); - const expandedAllCombos = allCombos - ? Array.isArray(allCombos) - ? await expandProviderWildcardsInCollection(allCombos as ComboLike[]) - : { - ...allCombos, - combos: await expandProviderWildcardsInCollection( - ((allCombos as { combos?: ComboLike[] }).combos || []) as ComboLike[] - ), - } - : allCombos; - - const stickyWeightedLimit = clampStickyWeightedTargetLimit( - (config as Record).stickyWeightedLimit - ); - if ( - strategy === "weighted" && - !weightedStickyTargets.has(combo.name) && - weightedStickyTargets.size >= MAX_RR_COUNTERS - ) { - const oldest = weightedStickyTargets.keys().next().value; - if (oldest !== undefined) weightedStickyTargets.delete(oldest); - } - let stepGroups: Array<{ step: ComboRuntimeStep; targets: ResolvedComboTarget[] }> | undefined; - const weightedEligibleKeys = new Set(); - if (strategy === "weighted") { - stepGroups = resolveWeightedStepGroups(expandedCombo, expandedAllCombos); - for (const group of stepGroups) { - const availability = await Promise.all(group.targets.map(isTargetSelectableForWeighted)); - if (availability.some(Boolean)) weightedEligibleKeys.add(group.step.executionKey); - } - } - const rawStickyWeightedKey = - strategy === "weighted" ? getStickyWeightedExecutionKey(combo.name, stickyWeightedLimit) : null; - const stickyWeightedKey = - rawStickyWeightedKey && weightedEligibleKeys.has(rawStickyWeightedKey) - ? rawStickyWeightedKey - : null; - if (strategy !== "weighted" || stickyWeightedLimit <= 1) { - weightedStickyTargets.delete(combo.name); - } else if (rawStickyWeightedKey && !stickyWeightedKey) { - weightedStickyTargets.delete(combo.name); - } - const weightedResolution = - strategy === "weighted" - ? resolveWeightedTargets( - expandedCombo, - expandedAllCombos, - stickyWeightedKey, - weightedEligibleKeys, - stepGroups - ) - : null; - const getWeightedStepKeyForTarget = (target: ResolvedComboTarget): string | null => { - if (!weightedResolution?.orderedSteps) return null; - const step = weightedResolution.orderedSteps.find( - (entry) => - target.executionKey === entry.executionKey || - target.executionKey.startsWith(entry.executionKey + ">") - ); - return step?.executionKey || null; - }; - let orderedTargets = - strategy === "weighted" - ? weightedResolution?.orderedTargets || [] - : resolveComboTargets( - expandedCombo, - expandedAllCombos, - clampComboDepth(config.maxComboDepth) - ); - - orderedTargets = await applyRequestTagRouting(orderedTargets, body, log); - - const knownContextOverflow = getKnownContextOverflow(orderedTargets, body); - if (knownContextOverflow) { - const { requiredContextTokens, maxKnownContextTokens } = knownContextOverflow; - log.warn( - "COMBO", - `Request context exceeds every known target limit (${requiredContextTokens} > ${maxKnownContextTokens} tokens)` - ); - return errorResponseWithComboDiagnostics( - 400, - `Request requires approximately ${requiredContextTokens} tokens, but the largest known context limit in this combo is ${maxKnownContextTokens} tokens. Reduce or compact the request context.`, - { - poolSize: orderedTargets.length, - attempted: 0, - excluded: orderedTargets.map((target) => ({ - provider: target.provider, - model: target.modelStr, - reason: "context_window", - })), - attemptOrder: [], - terminalReason: "context_length_exceeded", - }, - { code: "context_length_exceeded", type: "invalid_request_error" } - ); - } - - if (strategy === "weighted") { - log.info( - "COMBO", - `Weighted selection${stickyWeightedKey ? " (sticky)" : ""}${allCombos ? " with nested resolution" : ""}: ${orderedTargets.length} total targets` - ); - } else if (allCombos) { - log.info("COMBO", `${strategy} with nested resolution: ${orderedTargets.length} total targets`); - } - - // Pipeline dispatch: route smart/pipeline-enabled combos through the multi-stage pipeline - if (strategy === "auto") { - const autoParsed = parseAutoPrefix(combo.name); - const autoVariant = autoParsed.valid ? autoParsed.variant : undefined; - if (autoVariant === "smart" || config.pipeline_enabled) { - try { - const pipelineRaw = await handlePipelineCombo({ - body, - combo, - handleChatCore: handleSingleModelWithTimeout, - log: { - info: log.info, - warn: log.warn, - error: log.error ?? log.warn, - }, - settings: settings ?? {}, - signal: signal ?? undefined, - }); - // handlePipelineCombo resolves to a PipelineResult (buffered text) or, - // in the streaming-final-stage case, a Response. Callers downstream - // (chat.ts → withSessionHeader) require a Response, so adapt the - // PipelineResult here instead of leaking the raw object. - return pipelineRaw instanceof Response - ? pipelineRaw - : buildPipelineResponse(pipelineRaw, body); - } catch (pipelineErr) { - const pipelineMsg = pipelineErr instanceof Error ? pipelineErr.message : ""; - if (pipelineMsg === "PIPELINE_DISABLED") { - log.info("COMBO", "Pipeline disabled, falling through to standard auto routing"); - } else if (pipelineMsg === "PIPELINE_TOKEN_THRESHOLD") { - log.info( - "COMBO", - "Pipeline skipped (prompt below token threshold), falling through to standard auto routing" - ); - } else { - log.warn("COMBO", "Pipeline dispatch failed, falling through to standard auto routing", { - err: pipelineErr, - }); - } - } - } - } - - // #4945 regression guard: when an "auto" combo uses an EXPLICIT router - // (routingStrategy lkgp/cost/etc, not the default "rules" scorer), that router - // pins orderedTargets[0]. The task-aware reordering below must then refine only - // the fallback order, never override the router's primary choice. - let autoUsedExplicitRouter = false; - if (strategy === "auto") { - const autoResult = await resolveAutoStrategyOrder({ - orderedTargets, - body, - combo, - settings, - config, - relayOptions, - resilienceSettings, - log, - buildAutoCandidates, - }); - if ("earlyResponse" in autoResult) return autoResult.earlyResponse; - orderedTargets = autoResult.orderedTargets; - autoUsedExplicitRouter = autoResult.autoUsedExplicitRouter; - } else { - orderedTargets = await applyStrategyOrdering(strategy, orderedTargets, { - combo, - config, - body, - log, - apiKeyAllowedConnections, - }); - } - // An explicit cache-optimized combo outranks the global cache-affinity default, - // but only protects its ordering when this request actually produced a reusable - // cache key. Cache misses retain the normal session/eval routing behavior. - const cacheStrategyAffinityApplied = - strategy === "cache-optimized" && applyPromptCacheAffinity(orderedTargets, body).applied; - // #6168: session stickiness opt-out. Per-combo `config.disableSessionStickiness` - // overrides the global `settings.disableSessionStickiness` fallback (default false, - // preserving the #3825 prompt-cache/504 fix). When disabled, skip the reorder and - // treat the result as a no-op so the recordStickyBinding write-back below is skipped. - const disableSessionStickiness = - cacheStrategyAffinityApplied || - resolveDisableSessionStickiness( - config as Record | null | undefined, - settings as Record | null | undefined - ); - const _sticky = disableSessionStickiness - ? ({ targets: orderedTargets, messageHash: null, stuck: false } as const) - : await applySessionStickiness( - orderedTargets, - // #7270: normalize both wire shapes (.messages / Responses-API .input) so the - // stickiness key is derivable on the /v1/responses surface, not just Chat Completions. - normalizeStickinessMessages(body as { messages?: unknown; input?: unknown }) - ); - orderedTargets = _sticky.targets; - if (!cacheStrategyAffinityApplied) { - orderedTargets = orderTargetsByEvalScores(orderedTargets, config.evalRouting, log); - } - const compatFilterFailOpen = - (config as { compatFilterFailOpen?: unknown }).compatFilterFailOpen === true || - (settings as { compatFilterFailOpen?: unknown } | null | undefined)?.compatFilterFailOpen === - true; - const preCompatTargets = orderedTargets; - orderedTargets = filterTargetsByRequestCompatibility(orderedTargets, body, log, undefined, { - failOpen: compatFilterFailOpen, - }); - if (orderedTargets.length === 0 && preCompatTargets.length > 0) { - const exhaustion = describeCapabilityFilterExhaustion(preCompatTargets, body, combo.name); - if (exhaustion) { - recordComboFailure(effectiveSessionId, combo.name); - return errorResponseWithComboDiagnostics( - 400, - exhaustion.message, - { - poolSize: preCompatTargets.length, - attempted: 0, - excluded: exhaustion.excluded, - attemptOrder: [], - terminalReason: exhaustion.terminalReason, - recovery: buildRecoveryHint("no_executable_targets"), - }, - { code: "capability_mismatch", type: "invalid_request_error" } - ); - } - } - orderedTargets = applyContextRequirements(orderedTargets, config.contextRequirements, log); - - // Task-aware reordering: only active for strategies ["smart","task","task-aware","task_aware","auto"]. - // Additive — does not affect any of the other 15 strategies. - if (isTaskRoutingStrategy(strategy)) { - const task = classifyTask(body); - const conversationCacheKey = getConversationCacheKey(body); - const taskReordered = reorderByTaskWeight(orderedTargets, task); - // #4945 regression guard: when an explicit auto router (lkgp/cost/…) pinned - // orderedTargets[0], keep that primary choice and let task-aware refine only - // the fallback tail — otherwise task weighting silently defeats the operator's - // chosen LKGP/cost selection. reorderByTaskWeight returns the same target - // objects (no clone), so identity filtering is safe. - const pinnedFirst = autoUsedExplicitRouter ? orderedTargets[0] : undefined; - const nextOrder = pinnedFirst - ? [pinnedFirst, ...taskReordered.filter((t) => t !== pinnedFirst)] - : taskReordered; - if (nextOrder[0]?.modelStr !== orderedTargets[0]?.modelStr) { - const reasons = - Array.isArray(task.reasons) && task.reasons.length > 0 - ? ` (${task.reasons.join(",")})` - : ""; - log.info( - "COMBO", - `task-route task=${task.level}${reasons} cacheKey=${conversationCacheKey ?? "none"} → ${nextOrder[0]?.modelStr}` - ); - } - orderedTargets = nextOrder; - } - - // Prompt-cache locality is applied after request eligibility and task routing. - // Session stickiness and explicit auto-router pins remain stronger continuity - // decisions; quota, health, and circuit-breaker gates still run per attempt. - const autoConfigForCacheWeight = - strategy === "auto" - ? ((combo.autoConfig || - ((config as Record).auto && - typeof (config as Record).auto === "object" - ? (config as Record).auto - : null) || - config) as Record) - : null; - const autoWeightsForCache = - autoConfigForCacheWeight?.weights && typeof autoConfigForCacheWeight.weights === "object" - ? (autoConfigForCacheWeight.weights as Record) - : null; - const autoUsesCacheScore = Number(autoWeightsForCache?.cacheAffinity) > 0; - const promptCacheAffinityEnabled = - settings?.promptCacheAffinityEnabled !== false && !autoUsesCacheScore; - const promptCacheAffinityTargets = - promptCacheAffinityEnabled && resolvePromptCacheAffinityKey(body) - ? await expandPromptCacheAffinityTargets(orderedTargets) - : orderedTargets; - const promptCacheAffinity = applyPromptCacheAffinity( - promptCacheAffinityTargets, + const targetResolution = await resolveComboTargetPipeline({ body, - promptCacheAffinityEnabled - ); - if (promptCacheAffinity.applied) { - const protectedOriginal = - shouldProtectOriginalFirst(_sticky.stuck, autoUsedExplicitRouter, strategy) && - orderedTargets[0]; - const protectedFirst = protectedOriginal - ? (promptCacheAffinity.targets.find( - (target) => - target === protectedOriginal || - target.executionKey === protectedOriginal.executionKey || - target.executionKey.startsWith(`${protectedOriginal.executionKey}@`) - ) ?? protectedOriginal) - : null; - orderedTargets = protectedFirst - ? [ - protectedFirst, - ...promptCacheAffinity.targets.filter((target) => target !== protectedFirst), - ] - : promptCacheAffinity.targets; - log.debug?.("COMBO", "Prompt-cache affinity applied", { - source: promptCacheAffinity.source, - fingerprint: promptCacheAffinity.fingerprint, - targetCount: orderedTargets.length, - }); - } - - // Parallel pre-screen: check provider profiles and model availability for all targets - // Only runs for priority strategy where sequential checking causes latency - const preScreenMap = - strategy === "priority" - ? await preScreenTargets(orderedTargets, isModelAvailable).catch( - () => new Map() - ) - : new Map(); + combo, + strategy, + config, + settings, + allCombos, + relayOptions, + signal, + apiKeyAllowedConnections, + log, + resilienceSettings, + isModelAvailable, + handleSingleModelWithTimeout, + buildAutoCandidates, + }); + if ("earlyResponse" in targetResolution) return targetResolution.earlyResponse; + const { stickyWeightedLimit, getWeightedStepKeyForTarget, preScreenMap } = targetResolution; + const _sticky = targetResolution.sticky; + let orderedTargets = targetResolution.orderedTargets; // #5923 (Finding #4) — reset-window config for the shared per-target quota- // exhaustion cutoff below. The "auto" strategy already applies its own cutoff @@ -2587,7 +2238,7 @@ async function handleRoundRobinCombo({ { code: "context_length_exceeded", type: "invalid_request_error" } ); } - // Align with the main/auto paths: combo config OR top-level settings. + // Align with the main/auto paths: combo config OR top-level settings (#8488 / #8494). const rrCompatFailOpen = (config as { compatFilterFailOpen?: unknown }).compatFilterFailOpen === true || (settings as { compatFilterFailOpen?: unknown } | null | undefined)?.compatFilterFailOpen === diff --git a/open-sse/services/combo/targetResolution.ts b/open-sse/services/combo/targetResolution.ts new file mode 100644 index 0000000000..c71f48bccd --- /dev/null +++ b/open-sse/services/combo/targetResolution.ts @@ -0,0 +1,714 @@ +/** + * resolveComboTargetPipeline — the target-resolution phase of handleComboChat (combo.ts). + * + * Sits between the dispatch prelude (pinned model / fusion / chaos / pipeline / nested + * execute-mode / round-robin) and the attempt loop. It turns the raw combo definition + * into the final `orderedTargets` array the attempt loop iterates, in this order: + * + * 1. provider-wildcard expansion of the combo + the combos collection (#2562) + * 2. weighted step-group resolution + sticky-weighted eligibility + * 3. request-tag routing + * 4. known-context-overflow early return + * 5. smart/pipeline-enabled dispatch (auto strategy) + * 6. auto-strategy candidate build / scoring / ordering, or per-strategy ordering + * 7. prompt-cache strategy affinity, session stickiness, eval scores, + * request compatibility, context requirements + * 8. task-aware reordering + * 9. prompt-cache affinity application + * 10. the parallel pre-screen (priority strategy only) + * + * Behaviour is byte-identical to the inline block it replaces — the two early exits + * (context overflow, pipeline dispatch, auto-strategy `earlyResponse`) become an + * `{ earlyResponse }` result so the host decides to return them, and the values the + * attempt loop still consumes (`orderedTargets`, `stickyWeightedLimit`, + * `getWeightedStepKeyForTarget`, `sticky`, `preScreenMap`) are returned instead of + * closed over. + * + * See _tasks/quality/2026-06-19-DESIGN-godfiles-decomposition.md §4. + */ +import { isModelLocked } from "../accountFallback.ts"; +import { parseAutoPrefix } from "../autoCombo/autoPrefix.ts"; +import { handlePipelineCombo, buildPipelineResponse } from "../autoCombo/pipelineRouter.ts"; +import type { resolveComboSetupConfig } from "../comboConfig.ts"; +import { orderTargetsByEvalScores } from "../evalRouting.ts"; +import { parseModel } from "../model.ts"; +import { isProviderInCooldown } from "../providerCooldownTracker.ts"; +import { + classifyTask, + getConversationCacheKey, + isTaskRoutingStrategy, + reorderByTaskWeight, +} from "../taskAwareRouting.ts"; +import { errorResponseWithComboDiagnostics } from "../../utils/error.ts"; +import { getCircuitBreaker } from "../../../src/shared/utils/circuitBreaker"; +import type { ResilienceSettings } from "../../../src/lib/resilience/settings"; +import { applyStrategyOrdering } from "./applyStrategyOrdering.ts"; +import { clampComboDepth } from "./comboPredicates.ts"; +import { + describeCapabilityFilterExhaustion, + filterTargetsByRequestCompatibility, + resolveComboTargets, + resolveWeightedStepGroups, + resolveWeightedTargets, +} from "./comboStructure.ts"; +import { applyContextRequirements } from "./contextRequirements.ts"; +import { recordComboFailure } from "./failureTracker.ts"; +import { getKnownContextOverflow } from "./knownContextOverflow.ts"; +import { buildRecoveryHint } from "./pinRecovery.ts"; +import { + applyPromptCacheAffinity, + expandPromptCacheAffinityTargets, + resolvePromptCacheAffinityKey, + shouldProtectOriginalFirst, +} from "./promptCacheAffinity.ts"; +import { + expandProviderWildcardsInCombo, + expandProviderWildcardsInCollection, +} from "./providerWildcard.ts"; +import { preScreenTargets, type PreScreenResult } from "./quotaStrategies.ts"; +import { resolveAutoStrategyOrder, type ResolveAutoStrategyDeps } from "./resolveAutoStrategy.ts"; +import { + MAX_RR_COUNTERS, + clampStickyWeightedTargetLimit, + getStickyWeightedExecutionKey, + weightedStickyTargets, +} from "./rrState.ts"; +import { + applySessionStickiness, + normalizeStickinessMessages, + resolveDisableSessionStickiness, + type ApplyStickinessResult, +} from "./sessionStickiness.ts"; +import { applyRequestTagRouting } from "./autoStrategy.ts"; +import type { + ComboCollectionLike, + ComboLike, + ComboLogger, + ComboRelayOptions, + ComboRuntimeStep, + HandleSingleModel, + IsModelAvailable, + ResolvedComboTarget, +} from "./types.ts"; + +export interface ResolveComboTargetPipelineDeps { + body: Record; + combo: ComboLike; + strategy: string; + config: ReturnType; + settings?: Record | null; + allCombos?: ComboCollectionLike; + relayOptions?: ComboRelayOptions | null; + signal?: AbortSignal | null; + apiKeyAllowedConnections: string[] | null; + log: ComboLogger; + resilienceSettings: ResilienceSettings; + isModelAvailable?: IsModelAvailable; + /** handleSingleModel already wrapped by buildTargetTimeoutRunner. */ + handleSingleModelWithTimeout: HandleSingleModel; + /** + * Dependency-injected `buildAutoCandidates` — it lives in `combo.ts` (the host of + * this leaf), so importing it directly would create an import cycle. + */ + buildAutoCandidates: ResolveAutoStrategyDeps["buildAutoCandidates"]; +} + +export interface ResolvedComboTargetPipeline { + orderedTargets: ResolvedComboTarget[]; + /** Sticky-weighted target limit — the attempt loop records sticky success with it. */ + stickyWeightedLimit: number; + /** Maps an attempted target back to its weighted step key (sticky-weighted write-back). */ + getWeightedStepKeyForTarget: (target: ResolvedComboTarget) => string | null; + /** Session-stickiness result — the attempt loop reads `.messageHash` on success/failure. */ + sticky: ApplyStickinessResult; + preScreenMap: Map; +} + +export type ResolveComboTargetPipelineResult = + { earlyResponse: Response } | ResolvedComboTargetPipeline; + +type WeightedResolution = ReturnType | null; + +type WeightedStepGroups = + Array<{ step: ComboRuntimeStep; targets: ResolvedComboTarget[] }> | undefined; + +/** + * Weighted-strategy eligibility predicate: a step counts as selectable only when at + * least one of its targets clears the provider breaker, the connection cooldown, the + * per-model lockout and the caller's availability probe. + */ +async function isTargetSelectableForWeighted( + target: ResolvedComboTarget, + resilienceSettings: ResilienceSettings, + isModelAvailable?: IsModelAvailable +): Promise { + const rawModel = parseModel(target.modelStr).model || target.modelStr; + if (target.provider && getCircuitBreaker(target.provider).getStatus().state === "OPEN") + return false; + if ( + resilienceSettings.providerCooldown.enabled && + Boolean(target.provider && target.provider !== "unknown") && + isProviderInCooldown(target.provider, target.connectionId ?? undefined, resilienceSettings) + ) { + return false; + } + if ( + target.provider && + rawModel && + isModelLocked(target.provider, target.connectionId || "", rawModel) + ) { + return false; + } + return isModelAvailable ? await isModelAvailable(target.modelStr, target) : true; +} + +/** + * #2562: Expand provider-wildcard steps (e.g. `fta/*`, `openai/gpt-4*`) into + * concrete model entries sourced from the live synced-models catalog + registry. + * Must run before any step-group / target resolution so that wildcard-originated + * steps are treated identically to hand-authored entries by all downstream logic + * (including the sticky-weighted eligibility pass below). + */ +async function expandComboWildcards( + combo: ComboLike, + allCombos: ComboCollectionLike +): Promise<{ expandedCombo: ComboLike; expandedAllCombos: ComboCollectionLike }> { + const expandedCombo = await expandProviderWildcardsInCombo(combo); + const expandedAllCombos = allCombos + ? Array.isArray(allCombos) + ? await expandProviderWildcardsInCollection(allCombos as ComboLike[]) + : { + ...allCombos, + combos: await expandProviderWildcardsInCollection( + ((allCombos as { combos?: ComboLike[] }).combos || []) as ComboLike[] + ), + } + : allCombos; + return { expandedCombo, expandedAllCombos }; +} + +/** LRU-evict the oldest sticky-weighted entry once the counter map is at capacity. */ +function evictOldestWeightedSticky(strategy: string, comboName: string): void { + if ( + strategy === "weighted" && + !weightedStickyTargets.has(comboName) && + weightedStickyTargets.size >= MAX_RR_COUNTERS + ) { + const oldest = weightedStickyTargets.keys().next().value; + if (oldest !== undefined) weightedStickyTargets.delete(oldest); + } +} + +/** Resolve the weighted step groups and the subset whose targets are still selectable. */ +async function collectWeightedEligibility( + expandedCombo: ComboLike, + expandedAllCombos: ComboCollectionLike, + resilienceSettings: ResilienceSettings, + isModelAvailable?: IsModelAvailable +): Promise<{ stepGroups: WeightedStepGroups; weightedEligibleKeys: Set }> { + const weightedEligibleKeys = new Set(); + const stepGroups = resolveWeightedStepGroups(expandedCombo, expandedAllCombos); + for (const group of stepGroups) { + const availability = await Promise.all( + group.targets.map((target) => + isTargetSelectableForWeighted(target, resilienceSettings, isModelAvailable) + ) + ); + if (availability.some(Boolean)) weightedEligibleKeys.add(group.step.executionKey); + } + return { stepGroups, weightedEligibleKeys }; +} + +/** + * Honor the persisted sticky-weighted pin only while its step is still eligible; + * drop the stored pin otherwise (and whenever stickiness is off for this combo). + */ +function resolveStickyWeightedKey( + strategy: string, + comboName: string, + stickyWeightedLimit: number, + weightedEligibleKeys: Set +): string | null { + const rawStickyWeightedKey = + strategy === "weighted" ? getStickyWeightedExecutionKey(comboName, stickyWeightedLimit) : null; + const stickyWeightedKey = + rawStickyWeightedKey && weightedEligibleKeys.has(rawStickyWeightedKey) + ? rawStickyWeightedKey + : null; + if (strategy !== "weighted" || stickyWeightedLimit <= 1) { + weightedStickyTargets.delete(comboName); + } else if (rawStickyWeightedKey && !stickyWeightedKey) { + weightedStickyTargets.delete(comboName); + } + return stickyWeightedKey; +} + +/** Full weighted-strategy resolution: eviction → eligibility → sticky pin → ordering. */ +async function resolveWeightedSelection( + deps: ResolveComboTargetPipelineDeps, + expandedCombo: ComboLike, + expandedAllCombos: ComboCollectionLike, + stickyWeightedLimit: number +): Promise<{ weightedResolution: WeightedResolution; stickyWeightedKey: string | null }> { + const { strategy } = deps; + const comboName = deps.combo.name; + evictOldestWeightedSticky(strategy, comboName); + let stepGroups: WeightedStepGroups; + let weightedEligibleKeys = new Set(); + if (strategy === "weighted") { + const eligibility = await collectWeightedEligibility( + expandedCombo, + expandedAllCombos, + deps.resilienceSettings, + deps.isModelAvailable + ); + stepGroups = eligibility.stepGroups; + weightedEligibleKeys = eligibility.weightedEligibleKeys; + } + const stickyWeightedKey = resolveStickyWeightedKey( + strategy, + comboName, + stickyWeightedLimit, + weightedEligibleKeys + ); + const weightedResolution = + strategy === "weighted" + ? resolveWeightedTargets( + expandedCombo, + expandedAllCombos, + stickyWeightedKey, + weightedEligibleKeys, + stepGroups + ) + : null; + return { weightedResolution, stickyWeightedKey }; +} + +/** Maps an attempted target back to the weighted step it came from (sticky write-back). */ +function buildWeightedStepKeyMapper( + weightedResolution: WeightedResolution +): (target: ResolvedComboTarget) => string | null { + return (target: ResolvedComboTarget): string | null => { + if (!weightedResolution?.orderedSteps) return null; + const step = weightedResolution.orderedSteps.find( + (entry) => + target.executionKey === entry.executionKey || + target.executionKey.startsWith(entry.executionKey + ">") + ); + return step?.executionKey || null; + }; +} + +/** 400 rejection for a request no target in the pool can physically accept. */ +function buildContextOverflowResponse( + overflow: { requiredContextTokens: number; maxKnownContextTokens: number }, + orderedTargets: ResolvedComboTarget[], + log: ComboLogger +): Response { + const { requiredContextTokens, maxKnownContextTokens } = overflow; + log.warn( + "COMBO", + `Request context exceeds every known target limit (${requiredContextTokens} > ${maxKnownContextTokens} tokens)` + ); + return errorResponseWithComboDiagnostics( + 400, + `Request requires approximately ${requiredContextTokens} tokens, but the largest known context limit in this combo is ${maxKnownContextTokens} tokens. Reduce or compact the request context.`, + { + poolSize: orderedTargets.length, + attempted: 0, + excluded: orderedTargets.map((target) => ({ + provider: target.provider, + model: target.modelStr, + reason: "context_window", + })), + attemptOrder: [], + terminalReason: "context_length_exceeded", + }, + { code: "context_length_exceeded", type: "invalid_request_error" } + ); +} + +function logTargetPoolSize( + strategy: string, + allCombos: ComboCollectionLike, + orderedTargets: ResolvedComboTarget[], + stickyWeightedKey: string | null, + log: ComboLogger +): void { + if (strategy === "weighted") { + log.info( + "COMBO", + `Weighted selection${stickyWeightedKey ? " (sticky)" : ""}${allCombos ? " with nested resolution" : ""}: ${orderedTargets.length} total targets` + ); + } else if (allCombos) { + log.info("COMBO", `${strategy} with nested resolution: ${orderedTargets.length} total targets`); + } +} + +/** + * Pipeline dispatch: route smart/pipeline-enabled combos through the multi-stage + * pipeline. Returns the finished Response, or null to fall through to standard + * auto routing (pipeline disabled, below token threshold, or dispatch failure). + */ +async function dispatchSmartPipeline( + deps: ResolveComboTargetPipelineDeps +): Promise { + const { body, combo, strategy, config, settings, signal, log } = deps; + if (strategy !== "auto") return null; + const autoParsed = parseAutoPrefix(combo.name); + const autoVariant = autoParsed.valid ? autoParsed.variant : undefined; + if (autoVariant !== "smart" && !config.pipeline_enabled) return null; + try { + const pipelineRaw = await handlePipelineCombo({ + body, + combo, + handleChatCore: deps.handleSingleModelWithTimeout, + log: { + info: log.info, + warn: log.warn, + error: log.error ?? log.warn, + }, + settings: settings ?? {}, + signal: signal ?? undefined, + }); + // handlePipelineCombo resolves to a PipelineResult (buffered text) or, + // in the streaming-final-stage case, a Response. Callers downstream + // (chat.ts → withSessionHeader) require a Response, so adapt the + // PipelineResult here instead of leaking the raw object. + return pipelineRaw instanceof Response ? pipelineRaw : buildPipelineResponse(pipelineRaw, body); + } catch (pipelineErr) { + logPipelineFallthrough(pipelineErr, log); + return null; + } +} + +function logPipelineFallthrough(pipelineErr: unknown, log: ComboLogger): void { + const pipelineMsg = pipelineErr instanceof Error ? pipelineErr.message : ""; + if (pipelineMsg === "PIPELINE_DISABLED") { + log.info("COMBO", "Pipeline disabled, falling through to standard auto routing"); + } else if (pipelineMsg === "PIPELINE_TOKEN_THRESHOLD") { + log.info( + "COMBO", + "Pipeline skipped (prompt below token threshold), falling through to standard auto routing" + ); + } else { + log.warn("COMBO", "Pipeline dispatch failed, falling through to standard auto routing", { + err: pipelineErr, + }); + } +} + +/** + * Strategy ordering: the `auto` router for auto combos, the per-strategy chain for + * everything else. `autoUsedExplicitRouter` is the #4945 guard — when an explicit + * router (lkgp/cost/…) pinned orderedTargets[0], task-aware reordering below must + * refine only the fallback order, never override the router's primary choice. + */ +async function orderByStrategy( + deps: ResolveComboTargetPipelineDeps, + initialOrderedTargets: ResolvedComboTarget[] +): Promise< + | { earlyResponse: Response } + | { orderedTargets: ResolvedComboTarget[]; autoUsedExplicitRouter: boolean } +> { + const { strategy, body, combo, settings, config, log } = deps; + if (strategy === "auto") { + const autoResult = await resolveAutoStrategyOrder({ + orderedTargets: initialOrderedTargets, + body, + combo, + settings, + config, + relayOptions: deps.relayOptions, + resilienceSettings: deps.resilienceSettings, + log, + buildAutoCandidates: deps.buildAutoCandidates, + }); + if ("earlyResponse" in autoResult) return { earlyResponse: autoResult.earlyResponse }; + return { + orderedTargets: autoResult.orderedTargets, + autoUsedExplicitRouter: autoResult.autoUsedExplicitRouter, + }; + } + const orderedTargets = await applyStrategyOrdering(strategy, initialOrderedTargets, { + combo, + config, + body, + log, + apiKeyAllowedConnections: deps.apiKeyAllowedConnections, + }); + return { orderedTargets, autoUsedExplicitRouter: false }; +} + +/** + * Continuity + eligibility filters: cache-strategy affinity, session stickiness, + * eval-score ordering, request compatibility and per-combo context requirements. + * + * May return `{ earlyResponse }` when hard capability filters (#8488 / #8494) empty + * the pool — tools / vision / structured_output fail closed as 400 capability_mismatch + * unless `compatFilterFailOpen` is set on the combo config or settings. + */ +async function applyContinuityFilters( + deps: ResolveComboTargetPipelineDeps, + initialOrderedTargets: ResolvedComboTarget[] +): Promise< + | { orderedTargets: ResolvedComboTarget[]; sticky: ApplyStickinessResult } + | { earlyResponse: Response } +> { + const { strategy, body, combo, config, settings, log, relayOptions } = deps; + // An explicit cache-optimized combo outranks the global cache-affinity default, + // but only protects its ordering when this request actually produced a reusable + // cache key. Cache misses retain the normal session/eval routing behavior. + const cacheStrategyAffinityApplied = + strategy === "cache-optimized" && applyPromptCacheAffinity(initialOrderedTargets, body).applied; + // #6168: session stickiness opt-out. Per-combo `config.disableSessionStickiness` + // overrides the global `settings.disableSessionStickiness` fallback (default false, + // preserving the #3825 prompt-cache/504 fix). When disabled, skip the reorder and + // treat the result as a no-op so the recordStickyBinding write-back below is skipped. + const disableSessionStickiness = + cacheStrategyAffinityApplied || + resolveDisableSessionStickiness( + config as Record | null | undefined, + settings as Record | null | undefined + ); + const sticky: ApplyStickinessResult = disableSessionStickiness + ? { targets: initialOrderedTargets, messageHash: null, stuck: false } + : await applySessionStickiness( + initialOrderedTargets, + // #7270: normalize both wire shapes (.messages / Responses-API .input) so the + // stickiness key is derivable on the /v1/responses surface, not just Chat Completions. + normalizeStickinessMessages(body as { messages?: unknown; input?: unknown }) + ); + let orderedTargets = sticky.targets; + if (!cacheStrategyAffinityApplied) { + orderedTargets = orderTargetsByEvalScores(orderedTargets, config.evalRouting, log); + } + // #8488 / #8494: fail closed when hard capability filters empty the pool. + // Opt-in escape hatch: combo.config.compatFilterFailOpen OR settings.compatFilterFailOpen. + const compatFilterFailOpen = + (config as { compatFilterFailOpen?: unknown }).compatFilterFailOpen === true || + (settings as { compatFilterFailOpen?: unknown } | null | undefined)?.compatFilterFailOpen === + true; + const preCompatTargets = orderedTargets; + orderedTargets = filterTargetsByRequestCompatibility(orderedTargets, body, log, undefined, { + failOpen: compatFilterFailOpen, + }); + if (orderedTargets.length === 0 && preCompatTargets.length > 0) { + const exhaustion = describeCapabilityFilterExhaustion(preCompatTargets, body, combo.name); + if (exhaustion) { + // Match handleComboChat: only track failures under context-cache protection pins. + const effectiveSessionId: string | null = combo.context_cache_protection + ? (relayOptions?.sessionId ?? null) + : null; + recordComboFailure(effectiveSessionId, combo.name); + return { + earlyResponse: errorResponseWithComboDiagnostics( + 400, + exhaustion.message, + { + poolSize: preCompatTargets.length, + attempted: 0, + excluded: exhaustion.excluded, + attemptOrder: [], + terminalReason: exhaustion.terminalReason, + recovery: buildRecoveryHint("no_executable_targets"), + }, + { code: "capability_mismatch", type: "invalid_request_error" } + ), + }; + } + } + orderedTargets = applyContextRequirements(orderedTargets, config.contextRequirements, log); + return { orderedTargets, sticky }; +} + +/** + * Task-aware reordering: only active for strategies + * ["smart","task","task-aware","task_aware","auto"]. Additive — does not affect any + * of the other 15 strategies. + */ +function applyTaskAwareOrdering( + deps: ResolveComboTargetPipelineDeps, + orderedTargets: ResolvedComboTarget[], + autoUsedExplicitRouter: boolean +): ResolvedComboTarget[] { + const { strategy, body, log } = deps; + if (!isTaskRoutingStrategy(strategy)) return orderedTargets; + const task = classifyTask(body); + const conversationCacheKey = getConversationCacheKey(body); + const taskReordered = reorderByTaskWeight(orderedTargets, task); + // #4945 regression guard: when an explicit auto router (lkgp/cost/…) pinned + // orderedTargets[0], keep that primary choice and let task-aware refine only + // the fallback tail — otherwise task weighting silently defeats the operator's + // chosen LKGP/cost selection. reorderByTaskWeight returns the same target + // objects (no clone), so identity filtering is safe. + const pinnedFirst = autoUsedExplicitRouter ? orderedTargets[0] : undefined; + const nextOrder = pinnedFirst + ? [pinnedFirst, ...taskReordered.filter((t) => t !== pinnedFirst)] + : taskReordered; + if (nextOrder[0]?.modelStr !== orderedTargets[0]?.modelStr) { + const reasons = + Array.isArray(task.reasons) && task.reasons.length > 0 ? ` (${task.reasons.join(",")})` : ""; + log.info( + "COMBO", + `task-route task=${task.level}${reasons} cacheKey=${conversationCacheKey ?? "none"} → ${nextOrder[0]?.modelStr}` + ); + } + return nextOrder; +} + +/** + * Prompt-cache affinity is skipped when the auto scorer already weights cacheAffinity + * itself — otherwise the same signal would be applied twice. + */ +function isPromptCacheAffinityEnabled( + strategy: string, + combo: ComboLike, + config: ReturnType, + settings?: Record | null +): boolean { + const autoConfigForCacheWeight = + strategy === "auto" + ? ((combo.autoConfig || + ((config as Record).auto && + typeof (config as Record).auto === "object" + ? (config as Record).auto + : null) || + config) as Record) + : null; + const autoWeightsForCache = + autoConfigForCacheWeight?.weights && typeof autoConfigForCacheWeight.weights === "object" + ? (autoConfigForCacheWeight.weights as Record) + : null; + const autoUsesCacheScore = Number(autoWeightsForCache?.cacheAffinity) > 0; + return settings?.promptCacheAffinityEnabled !== false && !autoUsesCacheScore; +} + +/** + * Keep the stronger continuity decision (session pin / explicit auto-router pin) at + * the head of the cache-affinity ordering rather than letting affinity override it. + */ +function protectFirstTarget( + affinityTargets: ResolvedComboTarget[], + protectedOriginal: ResolvedComboTarget | false | undefined +): ResolvedComboTarget[] { + const protectedFirst = protectedOriginal + ? (affinityTargets.find( + (target) => + target === protectedOriginal || + target.executionKey === protectedOriginal.executionKey || + target.executionKey.startsWith(`${protectedOriginal.executionKey}@`) + ) ?? protectedOriginal) + : null; + return protectedFirst + ? [protectedFirst, ...affinityTargets.filter((target) => target !== protectedFirst)] + : affinityTargets; +} + +/** + * Prompt-cache locality is applied after request eligibility and task routing. + * Session stickiness and explicit auto-router pins remain stronger continuity + * decisions; quota, health, and circuit-breaker gates still run per attempt. + */ +async function applyPromptCacheStage( + deps: ResolveComboTargetPipelineDeps, + orderedTargets: ResolvedComboTarget[], + stickyStuck: boolean, + autoUsedExplicitRouter: boolean +): Promise { + const { strategy, body, combo, config, settings, log } = deps; + const promptCacheAffinityEnabled = isPromptCacheAffinityEnabled( + strategy, + combo, + config, + settings + ); + const promptCacheAffinityTargets = + promptCacheAffinityEnabled && resolvePromptCacheAffinityKey(body) + ? await expandPromptCacheAffinityTargets(orderedTargets) + : orderedTargets; + const promptCacheAffinity = applyPromptCacheAffinity( + promptCacheAffinityTargets, + body, + promptCacheAffinityEnabled + ); + if (!promptCacheAffinity.applied) return orderedTargets; + const protectedOriginal = + shouldProtectOriginalFirst(stickyStuck, autoUsedExplicitRouter, strategy) && orderedTargets[0]; + const nextTargets = protectFirstTarget(promptCacheAffinity.targets, protectedOriginal); + log.debug?.("COMBO", "Prompt-cache affinity applied", { + source: promptCacheAffinity.source, + fingerprint: promptCacheAffinity.fingerprint, + targetCount: nextTargets.length, + }); + return nextTargets; +} + +export async function resolveComboTargetPipeline( + deps: ResolveComboTargetPipelineDeps +): Promise { + const { body, combo, strategy, config, allCombos, log, isModelAvailable } = deps; + + const { expandedCombo, expandedAllCombos } = await expandComboWildcards(combo, allCombos); + const stickyWeightedLimit = clampStickyWeightedTargetLimit( + (config as Record).stickyWeightedLimit + ); + const { weightedResolution, stickyWeightedKey } = await resolveWeightedSelection( + deps, + expandedCombo, + expandedAllCombos, + stickyWeightedLimit + ); + const getWeightedStepKeyForTarget = buildWeightedStepKeyMapper(weightedResolution); + let orderedTargets = + strategy === "weighted" + ? weightedResolution?.orderedTargets || [] + : resolveComboTargets( + expandedCombo, + expandedAllCombos, + clampComboDepth(config.maxComboDepth) + ); + + orderedTargets = await applyRequestTagRouting(orderedTargets, body, log); + + const overflow = getKnownContextOverflow(orderedTargets, body); + if (overflow) { + return { earlyResponse: buildContextOverflowResponse(overflow, orderedTargets, log) }; + } + + logTargetPoolSize(strategy, allCombos, orderedTargets, stickyWeightedKey, log); + + const pipelineResponse = await dispatchSmartPipeline(deps); + if (pipelineResponse) return { earlyResponse: pipelineResponse }; + + const ordering = await orderByStrategy(deps, orderedTargets); + if ("earlyResponse" in ordering) return ordering; + const { autoUsedExplicitRouter } = ordering; + + const continuity = await applyContinuityFilters(deps, ordering.orderedTargets); + if ("earlyResponse" in continuity) return continuity; + orderedTargets = applyTaskAwareOrdering(deps, continuity.orderedTargets, autoUsedExplicitRouter); + orderedTargets = await applyPromptCacheStage( + deps, + orderedTargets, + continuity.sticky.stuck, + autoUsedExplicitRouter + ); + + // Parallel pre-screen: check provider profiles and model availability for all targets + // Only runs for priority strategy where sequential checking causes latency + const preScreenMap = + strategy === "priority" + ? await preScreenTargets(orderedTargets, isModelAvailable).catch( + () => new Map() + ) + : new Map(); + + return { + orderedTargets, + stickyWeightedLimit, + getWeightedStepKeyForTarget, + sticky: continuity.sticky, + preScreenMap, + }; +} diff --git a/scripts/check/check-known-symbols.ts b/scripts/check/check-known-symbols.ts index c1b7eae6aa..5824bd5d0e 100644 --- a/scripts/check/check-known-symbols.ts +++ b/scripts/check/check-known-symbols.ts @@ -492,6 +492,7 @@ async function main(): Promise { // #3501: the fusion/pipeline dispatch branches moved here with the prelude // extraction; the `strategy === "..."` checks are unchanged, just relocated. "open-sse/services/combo/dispatchPrelude.ts", + "open-sse/services/combo/targetResolution.ts", ]; const comboSource = comboDispatchFiles .map((rel) => readFileSync(resolvePath(REPO_ROOT, rel), "utf8")) diff --git a/tests/unit/combo-target-resolution-split.test.ts b/tests/unit/combo-target-resolution-split.test.ts new file mode 100644 index 0000000000..df5538f776 --- /dev/null +++ b/tests/unit/combo-target-resolution-split.test.ts @@ -0,0 +1,148 @@ +import test from "node:test"; +import assert from "node:assert/strict"; +import fs from "node:fs"; +import os from "node:os"; +import path from "node:path"; + +// Split guard for the #3501 god-file decomposition (PR 2): the target-resolution +// stage of handleComboChat (wildcard expansion → weighted step groups → known +// context overflow → strategy ordering → stickiness/eval/compat/context filters → +// task-aware reorder → prompt-cache affinity → pre-screen) was extracted verbatim +// into resolveComboTargetPipeline. These tests pin the leaf's own contract: the +// shape it hands back to the attempt loop, the pass-through ordering for the plain +// `priority` path, and the `earlyResponse` exit for a request that exceeds every +// target's known context window. The strategy-specific branches stay covered +// end-to-end by the combo-* consumer suites through combo.ts. + +const TEST_DATA_DIR = fs.mkdtempSync(path.join(os.tmpdir(), "omniroute-combo-target-resolution-")); +const ORIGINAL_DATA_DIR = process.env.DATA_DIR; +process.env.DATA_DIR = TEST_DATA_DIR; + +const core = await import("../../src/lib/db/core.ts"); +const { saveModelsDevCapabilities, clearModelsDevCapabilities } = + await import("../../src/lib/modelsDevSync.ts"); +const { resolveComboTargetPipeline } = + await import("../../open-sse/services/combo/targetResolution.ts"); + +test.after(() => { + core.resetDbInstance(); + if (ORIGINAL_DATA_DIR === undefined) { + delete process.env.DATA_DIR; + } else { + process.env.DATA_DIR = ORIGINAL_DATA_DIR; + } + fs.rmSync(TEST_DATA_DIR, { recursive: true, force: true }); +}); + +test.beforeEach(() => { + clearModelsDevCapabilities(); +}); + +const noopLog = { info() {}, warn() {}, error() {}, debug() {} } as never; + +function capabilityEntry(limitContext: number) { + return { + tool_call: true, + reasoning: false, + attachment: false, + structured_output: true, + temperature: true, + modalities_input: JSON.stringify(["text"]), + modalities_output: JSON.stringify(["text"]), + knowledge_cutoff: null, + release_date: null, + last_updated: null, + status: null, + family: null, + open_weights: false, + limit_context: limitContext, + limit_input: limitContext, + limit_output: 4096, + interleaved_field: null, + }; +} + +const deps = (overrides: Record = {}): never => + ({ + body: { messages: [{ role: "user", content: "hi" }] }, + combo: { id: "c1", name: "c1", models: ["openai/gpt-4o", "anthropic/claude-3"], config: {} }, + strategy: "priority", + config: {}, + settings: null, + allCombos: null, + relayOptions: null, + signal: null, + apiKeyAllowedConnections: null, + log: noopLog, + resilienceSettings: { providerCooldown: { enabled: false } }, + isModelAvailable: undefined, + handleSingleModelWithTimeout: async () => new Response("{}"), + buildAutoCandidates: async () => [], + ...overrides, + }) as never; + +test("exports resolveComboTargetPipeline", () => { + assert.equal(typeof resolveComboTargetPipeline, "function"); +}); + +test("priority strategy resolves combo models into orderedTargets in declared order", async () => { + const result = await resolveComboTargetPipeline(deps()); + assert.ok(!("earlyResponse" in result), "expected a resolved pipeline, not an early response"); + if ("earlyResponse" in result) return; + assert.deepEqual( + result.orderedTargets.map((t) => t.modelStr), + ["openai/gpt-4o", "anthropic/claude-3"] + ); +}); + +test("returns the derived values the attempt loop consumes", async () => { + const result = await resolveComboTargetPipeline(deps()); + assert.ok(!("earlyResponse" in result)); + if ("earlyResponse" in result) return; + assert.equal(typeof result.stickyWeightedLimit, "number"); + assert.equal(typeof result.getWeightedStepKeyForTarget, "function"); + assert.ok(result.preScreenMap instanceof Map); + assert.equal(result.sticky.messageHash === null || typeof result.sticky.messageHash, "string"); + // Non-weighted strategies have no weighted step resolution, so the mapper is a + // constant null — the sticky-weighted write-back in combo.ts is then skipped. + assert.equal(result.getWeightedStepKeyForTarget(result.orderedTargets[0]), null); +}); + +test("an empty combo yields an empty target pool (combo.ts turns it into a 404)", async () => { + const result = await resolveComboTargetPipeline(deps({ combo: { name: "empty", models: [] } })); + assert.ok(!("earlyResponse" in result)); + if ("earlyResponse" in result) return; + assert.deepEqual(result.orderedTargets, []); +}); + +test("request exceeding every known context window returns a 400 earlyResponse", async () => { + saveModelsDevCapabilities({ + "unit-target-resolution": { + tiny: capabilityEntry(8_000), + small: capabilityEntry(16_000), + }, + }); + + const result = await resolveComboTargetPipeline( + deps({ + combo: { + id: "c2", + name: "known-context-overflow", + models: ["unit-target-resolution/tiny", "unit-target-resolution/small"], + config: {}, + }, + body: { messages: [{ role: "user", content: "word ".repeat(200_000) }] }, + }) + ); + + assert.ok("earlyResponse" in result, "expected a context-overflow early response"); + if (!("earlyResponse" in result)) return; + assert.equal(result.earlyResponse.status, 400); + const body = (await result.earlyResponse.json()) as { + error?: { code?: string }; + diagnostics?: { terminalReason?: string; attempted?: number }; + }; + assert.equal(body.error?.code, "context_length_exceeded"); + assert.equal(body.diagnostics?.terminalReason, "context_length_exceeded"); + assert.equal(body.diagnostics?.attempted, 0); +});