mirror of
https://github.com/diegosouzapw/OmniRoute.git
synced 2026-09-21 14:22:14 +03:00
Right call not to reuse the combo loop's `fallbackCount`: it only increments after a leg fails or is skipped, so the second target would still report 0 at dispatch. Stamping the ordered index (or round-robin offset) at the gate is the only place the number is actually known. This PR also carries the batch's file-size rebaseline, since it merges first and the ceiling has to cover every intermediate state. --- Validated in one consolidated worktree cut from `release/v3.8.51`, boarded with the other 19 PRs of this batch. Two in-batch conflicts, both additive and resolved by keeping each side: the `ENVIRONMENT.md` table (#13035 + #13011) and the `chatHelpers.ts` import block (#12975 on the tip + #13017). - `typecheck:core` clean; `check:dashboard-typecheck` OK (206 pre-existing, within baseline); `check:changelog-integrity` OK; `check:docs-counts` migrations ✓ - complexity 2816 / baseline 3218 and cognitive-complexity 1271 / baseline 1437 — both under baseline - 531 of 532 focused assertions green across the batch's 46 test files - `check-file-size` rebaselined for the batch's real growth (annotation `_rebaseline_2026_09_11_mergebatch_v3851_houminxi`, landed on #13038), attributed per PR The single red is **not this batch**: `tests/unit/combo/quota-weighted-strategy.test.ts` → "A/B isolation: 7 hard-empty + 2 at 0.5% + 1 at 40%, floor=1" asserts an order between two connections of identical weight and flakes on the pure tip too — 2 failures in 4 runs at `origin/release/v3.8.51` with nothing from this batch applied. ⚠️ base-red inherited: #12732 — `Docs Gates`, `Merge integrity`, `No new ESLint warnings`, `Unit Tests fast-path` and `Fast Quality Gates` reproduce on the pure tip (provider count 356 vs the 358 the modules define, SKILL.md drift, and `open-sse/utils/stream.ts` at 3115 > frozen 3098, untouched here). Thanks @HouMinXi — the live evidence on these (X500 logs, `storage.sqlite` state, real `/v1/models` probes, the 36-minute outage write-up) is what let a 20-PR batch be reviewed as a unit.
This commit is contained in:
1
changelog.d/fixes/13038-fallback-attempts-chat.md
Normal file
1
changelog.d/fixes/13038-fallback-attempts-chat.md
Normal file
@@ -0,0 +1 @@
|
||||
- **fix(api):** thread `X-OmniRoute-Fallback-Attempts` through combo chat completions so streaming and non-streaming responses report how many prior legs were attempted ([#13038](https://github.com/diegosouzapw/OmniRoute/pull/13038))
|
||||
@@ -1,4 +1,5 @@
|
||||
{
|
||||
"_rebaseline_2026_09_11_mergebatch_v3851_houminxi": "/merge-batch 2026-09-11 (v3.8.51), batch by HouMinXi. Final combined values, set on the first PR merged so every intermediate state is covered. open-sse/handlers/chatCore.ts 6036->6144: #13069 routes the non-streaming leg through the same provider-failure classification, model lockout and credential-refresh path the streaming leg already used (+443/-340 = +103 net; it extracts applyProviderFailureClassification and wires both legs to it, which is what #13043 reported missing), plus #13050 stamping that the client asked for SSE before the web_search fallback flips stream off (+6) and #13038 threading the dispatched target index (+3). src/sse/services/auth.ts 3488->3542: #13017 adds the explicit-pin one-shot probe for a recoverable inactive row with its 60s storm gate (+42 net) and #13061 makes a grok-cli 402 a connection-wide shared-wallet signal instead of a per-model billing miss (+12 net). src/sse/handlers/chat.ts 2458->2462: #13038 (+5). open-sse/services/combo/executeTargetAttempt.ts 1205->1212: #13006 feeds the 402 it already classified into the quota cache instead of dropping it (+7). open-sse/services/accountFallback.ts 2468->2469: #13060 adds the Cline re-auth phrase to OAUTH_INVALID_TOKEN_SIGNALS (+1). open-sse/utils/stream.ts is deliberately NOT rebaselined: already 3115 > 3098 on the pure tip with zero contribution from this batch (base-red #12732, owned by /sweep-reds). The file also carried \"open-sse/handlers/chatCore.ts\" twice (6026 and 6036); JSON keeps the last, so the first was dead weight any writer could have picked instead. Collapsed to one entry at the live value. Covered by 531 focused assertions across the batch's 46 test files.",
|
||||
"_rebaseline_2026_09_11_12358_chat_pipeline_custom_node": "PR #12358 own test growth: tests/integration/chat-pipeline.test.ts 1648->1736 (+88). One new integration case, \"#11884 chat pipeline sends a custom node's edited Chat API type upstream\": it seeds a custom OpenAI-compatible node with an edited Chat/Responses API type, stubs fetch, drives handleChatCore and asserts the upstream request carries the live connection setting rather than the format baked into the node id at creation. Irreducible at this layer — the point of the test is the full route-to-upstream path, which is what #11884 regressed. Nothing else in the file changed. Covered by the case itself plus tests/unit/chat-helpers.test.ts (28/28).",
|
||||
"_rebaseline_2026_09_10_12975_rotation_correlation_id": "PR #12975 own growth: open-sse/executors/base.ts 1751->1753 (+2) and open-sse/handlers/chatCore.ts 6021->6024 (+3). The opencode rotation lines carry the request correlationId: one optional ExecuteInput field and one correlationId argument at each of the three executor.execute call sites in handleChatCore. Irreducible plumbing at existing call sites; the rotation logic itself lives in open-sse/executors/opencode.ts and the new leaf predicates (under cap). Covered by tests/unit/opencode-transient-rotation.test.ts and tests/unit/chat-correlation-id-exhaustion.test.ts.",
|
||||
"_rebaseline_2026_09_11_mergebatch_v3851_maxmad_opencode": "/merge-batch 2026-09-11 (v3.8.51), PRs #13141, #13146 and #12975 by maxmad64bis. src/sse/services/auth.ts 3450->3488 (+38): #13146 adds the narrow ruleScope===model branch to markAccountUnavailable (gated on status 400; every other status keeps its path) plus the HONORS_RULE_LOCK_SCOPE_PROVIDERS opencode entry, taking it to 3464; #12975 then adds buildExhaustionOptions so the exhaustion log lines carry the request correlationId (+24). open-sse/services/accountFallback.ts 2467->2468 (+1): #13141 routes hasFutureRateLimitUntil through the tolerant epoch normalizer; #13146 is net zero there (+16/-16). open-sse/executors/base.ts 1751->1753 (+2): #12975 adds the optional ExecuteInput.correlationId field with its doc comment. src/sse/handlers/chat.ts is NOT rebaselined: #12975 threads correlationId through the three executor call sites (+2) but the file lands at 2452, still under its existing 2458 freeze. open-sse/utils/stream.ts is deliberately NOT rebaselined either: it is already 3115 > 3098 on the pure tip with zero contribution from this batch (base-red #12732, owned by /sweep-reds). No new branching beyond the two guarded branches named above. Covered by tests/unit/combo-predicates-epoch-cooldown.test.ts, opencode-400-model-unavailable.test.ts, agentrouter-error-rules.test.ts, opencode-transient-rotation.test.ts and chat-correlation-id-exhaustion.test.ts.",
|
||||
@@ -430,16 +431,15 @@
|
||||
"open-sse/executors/codex.ts": 1505,
|
||||
"open-sse/executors/cursor.ts": 1759,
|
||||
"open-sse/executors/muse-spark-web.ts": 1405,
|
||||
"open-sse/handlers/chatCore.ts": 6026,
|
||||
"open-sse/handlers/chatCore.ts": 6036,
|
||||
"open-sse/handlers/chatCore.ts": 6144,
|
||||
"open-sse/handlers/imageGeneration.ts": 3259,
|
||||
"open-sse/handlers/search.ts": 1789,
|
||||
"open-sse/mcp-server/schemas/tools.ts": 1621,
|
||||
"open-sse/mcp-server/server.ts": 1572,
|
||||
"open-sse/services/accountFallback.ts": 2468,
|
||||
"open-sse/services/accountFallback.ts": 2469,
|
||||
"open-sse/services/adobeFireflyBrowserLogin.ts": 1401,
|
||||
"open-sse/services/combo.ts": 4080,
|
||||
"open-sse/services/combo/executeTargetAttempt.ts": 1205,
|
||||
"open-sse/services/combo/executeTargetAttempt.ts": 1212,
|
||||
"open-sse/translator/response/openai-responses.ts": 1466,
|
||||
"open-sse/utils/cursorAgentProtobuf.ts": 1547,
|
||||
"open-sse/utils/proxyFetch.ts": 1271,
|
||||
@@ -471,8 +471,8 @@
|
||||
"src/shared/components/RequestLoggerV2.tsx": 1718,
|
||||
"src/shared/constants/providers/apikey/gateways.ts": 1502,
|
||||
"src/shared/services/cliRuntime.ts": 1296,
|
||||
"src/sse/handlers/chat.ts": 2458,
|
||||
"src/sse/services/auth.ts": 3488,
|
||||
"src/sse/handlers/chat.ts": 2462,
|
||||
"src/sse/services/auth.ts": 3542,
|
||||
"tests/unit/account-fallback-service.test.ts": 2453,
|
||||
"tests/unit/provider-validation-specialty.test.ts": 4656,
|
||||
"open-sse/services/autoCombo/virtualFactory.ts": 1219,
|
||||
|
||||
@@ -495,6 +495,7 @@ export async function handleChatCore({
|
||||
// applied to a CLONE of `body` at the persistAttemptLogs sink (surface 1) —
|
||||
// the model-bound `body` itself is never touched.
|
||||
videoBridgeLog = undefined,
|
||||
fallbackAttempts = undefined,
|
||||
}) {
|
||||
let { provider, model, extendedContext } = modelInfo;
|
||||
// #12150 P1b: true iff the video-bridge guardrail rendered >=1 transcript
|
||||
@@ -5433,6 +5434,7 @@ export async function handleChatCore({
|
||||
requestId: skillRequestId,
|
||||
compressionResponseMeta,
|
||||
comboStrategy,
|
||||
fallbackAttempts,
|
||||
});
|
||||
// #6426: align response body `model` with the `X-OmniRoute-Model` header
|
||||
// (both must be the resolved backend model). Some upstreams (notably legacy
|
||||
@@ -5612,6 +5614,7 @@ export async function handleChatCore({
|
||||
pendingRequestId,
|
||||
compressionResponseMeta,
|
||||
comboStrategy,
|
||||
fallbackAttempts,
|
||||
});
|
||||
|
||||
// The streaming headers (turn-state included, when present) are committed to
|
||||
|
||||
@@ -21,6 +21,7 @@ export function buildNonStreamingResponseHeaders(
|
||||
requestId: string | null | undefined;
|
||||
compressionResponseMeta?: string | null | undefined;
|
||||
comboStrategy?: string | null | undefined;
|
||||
fallbackAttempts?: number;
|
||||
},
|
||||
deps: { attachOmniRouteMetaHeaders: typeof defaultAttachMeta; now: () => number } = {
|
||||
attachOmniRouteMetaHeaders: defaultAttachMeta,
|
||||
@@ -40,6 +41,7 @@ export function buildNonStreamingResponseHeaders(
|
||||
costUsd: args.estimatedCost,
|
||||
requestId: args.requestId,
|
||||
strategy: args.comboStrategy ?? "single",
|
||||
...(args.fallbackAttempts !== undefined ? { fallbackAttempts: args.fallbackAttempts } : {}),
|
||||
});
|
||||
if (args.compressionResponseMeta) {
|
||||
responseHeaders[OMNIROUTE_RESPONSE_HEADERS.compression] = args.compressionResponseMeta;
|
||||
|
||||
@@ -19,6 +19,7 @@ export function assembleStreamingResponseHeaders(
|
||||
pendingRequestId: string;
|
||||
compressionResponseMeta?: string | null | undefined;
|
||||
comboStrategy?: string | null | undefined;
|
||||
fallbackAttempts?: number;
|
||||
},
|
||||
buildStreamingResponseHeaders: typeof defaultBuildStreaming = defaultBuildStreaming
|
||||
): Record<string, string> {
|
||||
@@ -31,6 +32,7 @@ export function assembleStreamingResponseHeaders(
|
||||
usage: null,
|
||||
costUsd: 0,
|
||||
strategy: args.comboStrategy ?? "single",
|
||||
...(args.fallbackAttempts !== undefined ? { fallbackAttempts: args.fallbackAttempts } : {}),
|
||||
}),
|
||||
"x-omniroute-request-id": args.pendingRequestId,
|
||||
};
|
||||
|
||||
@@ -1,4 +1,9 @@
|
||||
import type { ComboLogger, HandleSingleModel, IsModelAvailable, ResolvedComboTarget } from "./types";
|
||||
import type {
|
||||
ComboLogger,
|
||||
HandleSingleModel,
|
||||
IsModelAvailable,
|
||||
ResolvedComboTarget,
|
||||
} from "./types";
|
||||
|
||||
/**
|
||||
* Last-resort fallback tier for combo routing (#6238).
|
||||
@@ -40,7 +45,8 @@ export async function attemptCompatRejectedFallback(
|
||||
): Promise<Response | null> {
|
||||
if (rejectedTargets.length === 0) return null;
|
||||
|
||||
for (const target of rejectedTargets) {
|
||||
for (let i = 0; i < rejectedTargets.length; i++) {
|
||||
const target = rejectedTargets[i];
|
||||
if (ctx.isModelAvailable) {
|
||||
const available = await ctx.isModelAvailable(target.modelStr, target);
|
||||
if (!available) {
|
||||
@@ -67,6 +73,7 @@ export async function attemptCompatRejectedFallback(
|
||||
const result = await ctx.handleSingleModel(body, target.modelStr, {
|
||||
...target,
|
||||
effectiveComboStrategy: ctx.strategy,
|
||||
fallbackAttempts: i,
|
||||
});
|
||||
if (result.ok) {
|
||||
ctx.log.info("COMBO", `Last-resort compat fallback succeeded via ${target.modelStr}`);
|
||||
|
||||
@@ -129,8 +129,9 @@ export async function evaluateExecuteTargetGates(opts: {
|
||||
...target,
|
||||
allowRateLimitedConnection: true,
|
||||
modelAbortSignal: abortSignal,
|
||||
fallbackAttempts: i,
|
||||
}
|
||||
: { ...target, modelAbortSignal: abortSignal };
|
||||
: { ...target, modelAbortSignal: abortSignal, fallbackAttempts: i };
|
||||
|
||||
if (target.connectionId && !allowRateLimitedConnection) {
|
||||
const persistedSkip = await resolvePersistedConnectionCooldownSkipReason(
|
||||
|
||||
@@ -487,8 +487,8 @@ export async function handleRoundRobinCombo({
|
||||
const allowRateLimitedConnection =
|
||||
Boolean(provider && provider !== "unknown") && transientRateLimitedProviders.has(provider);
|
||||
const targetForAttempt = allowRateLimitedConnection
|
||||
? { ...target, allowRateLimitedConnection: true }
|
||||
: target;
|
||||
? { ...target, allowRateLimitedConnection: true, fallbackAttempts: offset }
|
||||
: { ...target, fallbackAttempts: offset };
|
||||
|
||||
// Pre-check availability
|
||||
if (isModelAvailable) {
|
||||
|
||||
@@ -79,6 +79,7 @@ async function executeModelUnit(args: {
|
||||
isModelAvailable?: IsModelAvailable;
|
||||
failoverBeforeRetry: unknown;
|
||||
effectiveComboStrategy: string;
|
||||
fallbackAttempts: number;
|
||||
}): Promise<Response> {
|
||||
if (args.isModelAvailable) {
|
||||
const available = await args.isModelAvailable(args.unit.modelStr, args.unit);
|
||||
@@ -88,6 +89,7 @@ async function executeModelUnit(args: {
|
||||
...args.unit,
|
||||
effectiveComboStrategy: args.effectiveComboStrategy,
|
||||
failoverBeforeRetry: args.failoverBeforeRetry,
|
||||
fallbackAttempts: args.fallbackAttempts,
|
||||
});
|
||||
}
|
||||
|
||||
@@ -142,6 +144,7 @@ async function executeRuntimeUnit(args: {
|
||||
nesting: ComboNestingContext;
|
||||
failoverBeforeRetry: unknown;
|
||||
effectiveComboStrategy: string;
|
||||
fallbackAttempts: number;
|
||||
}): Promise<Response> {
|
||||
if (args.unit.kind === "model") {
|
||||
return executeModelUnit({
|
||||
@@ -151,6 +154,7 @@ async function executeRuntimeUnit(args: {
|
||||
isModelAvailable: args.isModelAvailable,
|
||||
failoverBeforeRetry: args.failoverBeforeRetry,
|
||||
effectiveComboStrategy: args.effectiveComboStrategy,
|
||||
fallbackAttempts: args.fallbackAttempts,
|
||||
});
|
||||
}
|
||||
return executeComboRefUnit({
|
||||
@@ -289,6 +293,7 @@ export async function executeRuntimeUnitCombo(args: {
|
||||
nesting: args.nesting,
|
||||
failoverBeforeRetry: args.config.failoverBeforeRetry,
|
||||
effectiveComboStrategy: effectiveStrategy,
|
||||
fallbackAttempts: fallbackCount,
|
||||
});
|
||||
lastResponse = response;
|
||||
if (response.ok) {
|
||||
|
||||
@@ -60,8 +60,10 @@ export type SingleModelTarget =
|
||||
modelAbortSignal?: AbortSignal | null;
|
||||
/** True when this target was selected via context-cache session pinning. */
|
||||
modelPinned?: boolean;
|
||||
/** Prior combo legs already attempted before this dispatch (#12339). */
|
||||
fallbackAttempts?: number;
|
||||
})
|
||||
| { modelAbortSignal: AbortSignal };
|
||||
| { modelAbortSignal: AbortSignal; fallbackAttempts?: number };
|
||||
|
||||
export type HandleSingleModel = (
|
||||
body: Record<string, unknown>,
|
||||
|
||||
@@ -1133,6 +1133,7 @@ async function handleChatImplementation(
|
||||
providerId?: string | null;
|
||||
effectiveComboStrategy?: string | null;
|
||||
modelAbortSignal?: AbortSignal | null;
|
||||
fallbackAttempts?: number;
|
||||
}
|
||||
) =>
|
||||
handleSingleModelChat(
|
||||
@@ -1180,6 +1181,7 @@ async function handleChatImplementation(
|
||||
// entry (trackPendingRequest(false) never runs) — live incident,
|
||||
// log id 1784418258231-14961a.
|
||||
modelAbortSignal: target?.modelAbortSignal ?? null,
|
||||
fallbackAttempts: target?.fallbackAttempts,
|
||||
},
|
||||
target?.effectiveComboStrategy ?? combo.strategy,
|
||||
true
|
||||
@@ -1392,6 +1394,7 @@ async function handleSingleModelChat(
|
||||
* the signal used for the actual dispatch, not left unused.
|
||||
*/
|
||||
modelAbortSignal?: AbortSignal | null;
|
||||
fallbackAttempts?: number;
|
||||
} = {},
|
||||
comboStrategy: string | null = null,
|
||||
isCombo: boolean = false
|
||||
@@ -1465,6 +1468,7 @@ async function handleSingleModelChat(
|
||||
videoBridgeLog: runtimeOptions.videoBridgeLog,
|
||||
// #7360 follow-up — see the primary handleSingleModel closure above.
|
||||
modelAbortSignal: target?.modelAbortSignal ?? null,
|
||||
fallbackAttempts: target?.fallbackAttempts,
|
||||
},
|
||||
resolvedTarget?.effectiveComboStrategy ?? redirectCombo.strategy ?? "priority",
|
||||
false
|
||||
@@ -1957,6 +1961,7 @@ async function handleSingleModelChat(
|
||||
reasoningTransportFallback: runtimeOptions.reasoningTransportFallback ?? "drop",
|
||||
managedLease: runtimeOptions.managedLease ?? null,
|
||||
videoBridgeLog: runtimeOptions.videoBridgeLog,
|
||||
fallbackAttempts: runtimeOptions.fallbackAttempts,
|
||||
},
|
||||
runtimeOptions
|
||||
);
|
||||
|
||||
@@ -455,6 +455,7 @@ export async function executeChatWithBreaker({
|
||||
// for every non-video request. Passed straight through to handleChatCore;
|
||||
// see its own destructure default for the shape and consumers.
|
||||
videoBridgeLog = undefined,
|
||||
fallbackAttempts = undefined,
|
||||
}: ExecuteChatWithBreakerOptions): Promise<ExecuteChatWithBreakerResult> {
|
||||
let tlsFingerprintUsed = false;
|
||||
const normalizedTrafficType: TrafficType =
|
||||
@@ -515,6 +516,7 @@ export async function executeChatWithBreaker({
|
||||
reasoningTransportFallback,
|
||||
managedLease,
|
||||
videoBridgeLog,
|
||||
fallbackAttempts,
|
||||
skipResourcePressureGuard: true,
|
||||
onCredentialsRefreshed: async (newCreds: any) => {
|
||||
await updateProviderCredentials(credentials.connectionId, {
|
||||
|
||||
@@ -87,3 +87,16 @@ test("compression meta present → compression header set to that value", () =>
|
||||
);
|
||||
assert.ok(Object.values(h).includes("engine:x; source=header"));
|
||||
});
|
||||
|
||||
test("forwards fallbackAttempts into the non-streaming meta payload", () => {
|
||||
const { deps, metaCalls } = makeDeps();
|
||||
buildNonStreamingResponseHeaders(baseArgs({ fallbackAttempts: 3 }), deps);
|
||||
assert.equal(metaCalls.length, 1);
|
||||
assert.equal(metaCalls[0].meta.fallbackAttempts, 3);
|
||||
});
|
||||
|
||||
test("omitted fallbackAttempts does not invent a count", () => {
|
||||
const { deps, metaCalls } = makeDeps();
|
||||
buildNonStreamingResponseHeaders(baseArgs(), deps);
|
||||
assert.equal("fallbackAttempts" in metaCalls[0].meta, false);
|
||||
});
|
||||
|
||||
@@ -6,9 +6,8 @@
|
||||
import { test } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
|
||||
const { assembleStreamingResponseHeaders } = await import(
|
||||
"../../open-sse/handlers/chatCore/streamingResponseHeaders.ts"
|
||||
);
|
||||
const { assembleStreamingResponseHeaders } =
|
||||
await import("../../open-sse/handlers/chatCore/streamingResponseHeaders.ts");
|
||||
|
||||
function makeBuild() {
|
||||
const calls: Array<{ headers: unknown; meta: Record<string, unknown> }> = [];
|
||||
@@ -51,7 +50,10 @@ test("buildStreamingResponseHeaders receives zeroed latency/usage/cost and cache
|
||||
|
||||
test("no compression meta → no compression header", () => {
|
||||
const { build } = makeBuild();
|
||||
const h = assembleStreamingResponseHeaders(baseArgs({ compressionResponseMeta: undefined }), build);
|
||||
const h = assembleStreamingResponseHeaders(
|
||||
baseArgs({ compressionResponseMeta: undefined }),
|
||||
build
|
||||
);
|
||||
assert.ok(!Object.values(h).includes("engine:z"));
|
||||
});
|
||||
|
||||
@@ -63,3 +65,16 @@ test("compression meta present → compression header set", () => {
|
||||
);
|
||||
assert.ok(Object.values(h).includes("engine:z; source=routing"));
|
||||
});
|
||||
|
||||
test("forwards fallbackAttempts into the streaming meta payload", () => {
|
||||
const { build, calls } = makeBuild();
|
||||
assembleStreamingResponseHeaders(baseArgs({ fallbackAttempts: 2 }), build);
|
||||
assert.equal(calls.length, 1);
|
||||
assert.equal(calls[0].meta.fallbackAttempts, 2);
|
||||
});
|
||||
|
||||
test("omitted fallbackAttempts does not invent a count", () => {
|
||||
const { build, calls } = makeBuild();
|
||||
assembleStreamingResponseHeaders(baseArgs(), build);
|
||||
assert.equal("fallbackAttempts" in calls[0].meta, false);
|
||||
});
|
||||
|
||||
54
tests/unit/combo-compat-fallback-attempts-12339.test.ts
Normal file
54
tests/unit/combo-compat-fallback-attempts-12339.test.ts
Normal file
@@ -0,0 +1,54 @@
|
||||
import test from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { attemptCompatRejectedFallback } from "../../open-sse/services/combo/comboCompatFallback.ts";
|
||||
import type { ResolvedComboTarget } from "../../open-sse/services/combo/types.ts";
|
||||
|
||||
function modelTarget(overrides: Partial<ResolvedComboTarget> = {}): ResolvedComboTarget {
|
||||
return {
|
||||
kind: "model",
|
||||
stepId: "s1",
|
||||
executionKey: "ek-1",
|
||||
modelStr: "openai/compat-b",
|
||||
provider: "openai",
|
||||
providerId: null,
|
||||
connectionId: "c1",
|
||||
weight: 1,
|
||||
label: null,
|
||||
...overrides,
|
||||
};
|
||||
}
|
||||
|
||||
test("compat fallback stamps fallbackAttempts from the rejected-target index", async () => {
|
||||
const seen: Array<{ model: string; fallbackAttempts?: number }> = [];
|
||||
const targets = [
|
||||
modelTarget({ executionKey: "ek-0", stepId: "s0", modelStr: "openai/compat-a" }),
|
||||
modelTarget({ executionKey: "ek-1", stepId: "s1", modelStr: "openai/compat-b" }),
|
||||
];
|
||||
const ok = () =>
|
||||
new Response(JSON.stringify({ choices: [{ message: { content: "ok" } }] }), {
|
||||
status: 200,
|
||||
headers: { "content-type": "application/json" },
|
||||
});
|
||||
const result = await attemptCompatRejectedFallback(
|
||||
targets,
|
||||
{ messages: [] },
|
||||
{
|
||||
handleSingleModel: async (_body, modelStr, target) => {
|
||||
seen.push({
|
||||
model: modelStr,
|
||||
fallbackAttempts: (target as { fallbackAttempts?: number } | undefined)?.fallbackAttempts,
|
||||
});
|
||||
if (modelStr === "openai/compat-a") {
|
||||
return new Response("fail", { status: 500 });
|
||||
}
|
||||
return ok();
|
||||
},
|
||||
log: { info() {}, warn() {}, debug() {}, error() {} },
|
||||
strategy: "round-robin",
|
||||
}
|
||||
);
|
||||
assert.equal(result?.ok, true);
|
||||
assert.equal(seen.length, 2);
|
||||
assert.equal(seen[0].fallbackAttempts, 0);
|
||||
assert.equal(seen[1].fallbackAttempts, 1);
|
||||
});
|
||||
@@ -8,7 +8,10 @@ import test from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
|
||||
import { executeRuntimeUnitCombo } from "../../open-sse/services/combo/runtimeUnits.ts";
|
||||
import type { ResolvedComboUnit, ComboNestingContext } from "../../open-sse/services/combo/types.ts";
|
||||
import type {
|
||||
ResolvedComboUnit,
|
||||
ComboNestingContext,
|
||||
} from "../../open-sse/services/combo/types.ts";
|
||||
|
||||
function noopLog() {
|
||||
return { info: () => {}, warn: () => {}, error: () => {}, debug: () => {} };
|
||||
@@ -86,3 +89,71 @@ test(
|
||||
assert.equal(body.diagnostics?.terminalReason, "max_attempts_exceeded");
|
||||
}
|
||||
);
|
||||
|
||||
test("nested runtime-unit dispatch stamps fallbackAttempts from the unit index", async () => {
|
||||
const units: ResolvedComboUnit[] = [
|
||||
{
|
||||
kind: "model",
|
||||
stepId: "step-a",
|
||||
executionKey: "a",
|
||||
modelStr: "openai/ru-a",
|
||||
provider: "openai",
|
||||
providerId: null,
|
||||
connectionId: null,
|
||||
weight: 1,
|
||||
label: null,
|
||||
},
|
||||
{
|
||||
kind: "model",
|
||||
stepId: "step-b",
|
||||
executionKey: "b",
|
||||
modelStr: "anthropic/ru-b",
|
||||
provider: "anthropic",
|
||||
providerId: null,
|
||||
connectionId: null,
|
||||
weight: 1,
|
||||
label: null,
|
||||
},
|
||||
];
|
||||
const nesting: ComboNestingContext = {
|
||||
depth: 0,
|
||||
maxDepth: 5,
|
||||
visitedComboNames: [],
|
||||
rootComboName: "ru-fallback-12339",
|
||||
attemptBudget: { count: 0, limit: 8 },
|
||||
};
|
||||
const seen: Array<{ model: string; fallbackAttempts?: number }> = [];
|
||||
const ok = () =>
|
||||
new Response(JSON.stringify({ choices: [{ message: { content: "ok" } }] }), {
|
||||
status: 200,
|
||||
headers: { "content-type": "application/json" },
|
||||
});
|
||||
await executeRuntimeUnitCombo({
|
||||
body: { messages: [{ role: "user", content: "hi" }] },
|
||||
combo: { name: "ru-fallback-12339", strategy: "pipeline" },
|
||||
strategy: "pipeline",
|
||||
units,
|
||||
handleSingleModel: async (_body, modelStr, target) => {
|
||||
seen.push({
|
||||
model: modelStr,
|
||||
fallbackAttempts: (target as { fallbackAttempts?: number } | undefined)?.fallbackAttempts,
|
||||
});
|
||||
if (modelStr === "openai/ru-a") {
|
||||
return new Response(JSON.stringify({ error: { message: "upstream 500" } }), {
|
||||
status: 500,
|
||||
headers: { "content-type": "application/json" },
|
||||
});
|
||||
}
|
||||
return ok();
|
||||
},
|
||||
log: noopLog() as never,
|
||||
config: { maxRetries: 0, retryDelayMs: 0 },
|
||||
allCombos: [],
|
||||
nesting,
|
||||
baseOptions: {} as never,
|
||||
runCombo: async () => failResponse(),
|
||||
});
|
||||
assert.equal(seen.length, 2);
|
||||
assert.equal(seen[0].fallbackAttempts, 0);
|
||||
assert.equal(seen[1].fallbackAttempts, 1);
|
||||
});
|
||||
|
||||
@@ -286,3 +286,70 @@ test("body-specific 400 surfaces via {ok,response} not null", async () => {
|
||||
assert.equal(result?.ok, false);
|
||||
assert.equal(result?.response?.status, 400);
|
||||
});
|
||||
|
||||
test("spreads stamped fallbackAttempts onto the handleSingleModel target", async () => {
|
||||
const { executeTargetAttempt } =
|
||||
await import("../../../open-sse/services/combo/executeTargetAttempt.ts");
|
||||
let seen: unknown;
|
||||
const target = {
|
||||
...modelTarget({ connectionId: "c1" }),
|
||||
fallbackAttempts: 2,
|
||||
} as ResolvedComboTarget & { fallbackAttempts: number };
|
||||
const deps = baseDeps({
|
||||
maxRetries: 0,
|
||||
handleSingleModelWithTimeout: async (_body, _model, dispatched) => {
|
||||
seen = dispatched;
|
||||
return new Response(JSON.stringify({ choices: [{ message: { content: "ok" } }] }), {
|
||||
status: 200,
|
||||
headers: { "content-type": "application/json" },
|
||||
});
|
||||
},
|
||||
});
|
||||
const state = emptyState({
|
||||
orderedTargets: [target],
|
||||
abortControllers: new Map([[0, new AbortController()]]),
|
||||
});
|
||||
const result = await executeTargetAttempt({
|
||||
index: 0,
|
||||
state,
|
||||
deps,
|
||||
targetForAttempt: target,
|
||||
profile: {},
|
||||
protectedPriorityTarget: false,
|
||||
});
|
||||
assert.equal(result?.ok, true);
|
||||
assert.equal((seen as { fallbackAttempts?: number } | undefined)?.fallbackAttempts, 2);
|
||||
});
|
||||
|
||||
test("injection: dropping fallbackAttempts from the dispatch target goes red", async () => {
|
||||
const { executeTargetAttempt } =
|
||||
await import("../../../open-sse/services/combo/executeTargetAttempt.ts");
|
||||
let seen: unknown;
|
||||
const target = {
|
||||
...modelTarget({ connectionId: "c1" }),
|
||||
fallbackAttempts: 2,
|
||||
} as ResolvedComboTarget & { fallbackAttempts: number };
|
||||
const deps = baseDeps({
|
||||
maxRetries: 0,
|
||||
handleSingleModelWithTimeout: async (_body, _model, dispatched) => {
|
||||
seen = dispatched;
|
||||
return new Response(JSON.stringify({ choices: [{ message: { content: "ok" } }] }), {
|
||||
status: 200,
|
||||
headers: { "content-type": "application/json" },
|
||||
});
|
||||
},
|
||||
});
|
||||
const state = emptyState({
|
||||
orderedTargets: [target],
|
||||
abortControllers: new Map([[0, new AbortController()]]),
|
||||
});
|
||||
await executeTargetAttempt({
|
||||
index: 0,
|
||||
state,
|
||||
deps,
|
||||
targetForAttempt: target,
|
||||
profile: {},
|
||||
protectedPriorityTarget: false,
|
||||
});
|
||||
assert.equal(Object.prototype.hasOwnProperty.call(seen as object, "fallbackAttempts"), true);
|
||||
});
|
||||
|
||||
@@ -156,3 +156,69 @@ test("protected priority non-quota skip returns 503 response not null", async ()
|
||||
assert.equal(decision.result?.response?.status, 503);
|
||||
}
|
||||
});
|
||||
|
||||
test("proceed stamps fallbackAttempts from the ordered-target index", async () => {
|
||||
const { evaluateExecuteTargetGates } =
|
||||
await import("../../../open-sse/services/combo/executeTargetGates.ts");
|
||||
const first = modelTarget({ executionKey: "ek-0", stepId: "s0" });
|
||||
const second = modelTarget({ executionKey: "ek-1", stepId: "s1" });
|
||||
const state = emptyState({
|
||||
orderedTargets: [first, second],
|
||||
abortControllers: new Map([
|
||||
[0, new AbortController()],
|
||||
[1, new AbortController()],
|
||||
]),
|
||||
});
|
||||
const firstDecision = await evaluateExecuteTargetGates({
|
||||
index: 0,
|
||||
state,
|
||||
deps: baseDeps(),
|
||||
});
|
||||
const secondDecision = await evaluateExecuteTargetGates({
|
||||
index: 1,
|
||||
state,
|
||||
deps: baseDeps(),
|
||||
});
|
||||
assert.equal(firstDecision.kind, "proceed");
|
||||
assert.equal(secondDecision.kind, "proceed");
|
||||
if (firstDecision.kind === "proceed") {
|
||||
assert.equal(
|
||||
(firstDecision.targetForAttempt as ResolvedComboTarget & { fallbackAttempts?: number })
|
||||
.fallbackAttempts,
|
||||
0
|
||||
);
|
||||
}
|
||||
if (secondDecision.kind === "proceed") {
|
||||
assert.equal(
|
||||
(secondDecision.targetForAttempt as ResolvedComboTarget & { fallbackAttempts?: number })
|
||||
.fallbackAttempts,
|
||||
1
|
||||
);
|
||||
}
|
||||
});
|
||||
|
||||
test("injection: dropping fallbackAttempts from targetForAttempt goes red", async () => {
|
||||
const { evaluateExecuteTargetGates } =
|
||||
await import("../../../open-sse/services/combo/executeTargetGates.ts");
|
||||
const first = modelTarget({ executionKey: "ek-0", stepId: "s0" });
|
||||
const second = modelTarget({ executionKey: "ek-1", stepId: "s1" });
|
||||
const state = emptyState({
|
||||
orderedTargets: [first, second],
|
||||
abortControllers: new Map([
|
||||
[0, new AbortController()],
|
||||
[1, new AbortController()],
|
||||
]),
|
||||
});
|
||||
const decision = await evaluateExecuteTargetGates({
|
||||
index: 1,
|
||||
state,
|
||||
deps: baseDeps(),
|
||||
});
|
||||
assert.equal(decision.kind, "proceed");
|
||||
if (decision.kind === "proceed") {
|
||||
assert.equal(
|
||||
Object.prototype.hasOwnProperty.call(decision.targetForAttempt, "fallbackAttempts"),
|
||||
true
|
||||
);
|
||||
}
|
||||
});
|
||||
|
||||
@@ -19,7 +19,10 @@ test("buildOmniRouteResponseMetaHeaders emits X-OmniRoute-Decision for a combo s
|
||||
model: "gpt-4o",
|
||||
latencyMs: 42,
|
||||
});
|
||||
assert.equal(headers["X-OmniRoute-Decision"], "strategy=priority; provider=openai; latency_ms=42");
|
||||
assert.equal(
|
||||
headers["X-OmniRoute-Decision"],
|
||||
"strategy=priority; provider=openai; latency_ms=42"
|
||||
);
|
||||
});
|
||||
|
||||
test("strategy: single (non-combo request) still emits the header", () => {
|
||||
@@ -28,7 +31,10 @@ test("strategy: single (non-combo request) still emits the header", () => {
|
||||
provider: "anthropic",
|
||||
latencyMs: 10,
|
||||
});
|
||||
assert.equal(headers["X-OmniRoute-Decision"], "strategy=single; provider=anthropic; latency_ms=10");
|
||||
assert.equal(
|
||||
headers["X-OmniRoute-Decision"],
|
||||
"strategy=single; provider=anthropic; latency_ms=10"
|
||||
);
|
||||
});
|
||||
|
||||
test("omitted strategy AND provider -> header absent entirely", () => {
|
||||
@@ -70,5 +76,55 @@ test("buildNonStreamingResponseHeaders falls back to strategy=single when comboS
|
||||
requestId: "req-2",
|
||||
comboStrategy: null,
|
||||
});
|
||||
assert.match(headers["X-OmniRoute-Decision"], /^strategy=single; provider=openai; latency_ms=\d+$/);
|
||||
assert.match(
|
||||
headers["X-OmniRoute-Decision"],
|
||||
/^strategy=single; provider=openai; latency_ms=\d+$/
|
||||
);
|
||||
});
|
||||
|
||||
test("assembleStreamingResponseHeaders emits X-OmniRoute-Fallback-Attempts when count > 0", () => {
|
||||
const headers = assembleStreamingResponseHeaders({
|
||||
providerHeaders: new Headers(),
|
||||
provider: "openai",
|
||||
model: "gpt-4o",
|
||||
pendingRequestId: "req-3",
|
||||
comboStrategy: "priority",
|
||||
fallbackAttempts: 2,
|
||||
});
|
||||
assert.equal(headers["X-OmniRoute-Fallback-Attempts"], "2");
|
||||
});
|
||||
|
||||
test("buildNonStreamingResponseHeaders emits X-OmniRoute-Fallback-Attempts when count > 0", () => {
|
||||
const headers = buildNonStreamingResponseHeaders({
|
||||
provider: "openai",
|
||||
model: "gpt-4o",
|
||||
startTime: Date.now(),
|
||||
responseUsage: null,
|
||||
estimatedCost: 0,
|
||||
requestId: "req-4",
|
||||
comboStrategy: "priority",
|
||||
fallbackAttempts: 1,
|
||||
});
|
||||
assert.equal(headers["X-OmniRoute-Fallback-Attempts"], "1");
|
||||
});
|
||||
|
||||
test("builders omit X-OmniRoute-Fallback-Attempts when count is 0", () => {
|
||||
const streaming = assembleStreamingResponseHeaders({
|
||||
providerHeaders: new Headers(),
|
||||
provider: "openai",
|
||||
model: "gpt-4o",
|
||||
pendingRequestId: "req-5",
|
||||
fallbackAttempts: 0,
|
||||
});
|
||||
const nonStreaming = buildNonStreamingResponseHeaders({
|
||||
provider: "openai",
|
||||
model: "gpt-4o",
|
||||
startTime: Date.now(),
|
||||
responseUsage: null,
|
||||
estimatedCost: 0,
|
||||
requestId: "req-6",
|
||||
fallbackAttempts: 0,
|
||||
});
|
||||
assert.equal(streaming["X-OmniRoute-Fallback-Attempts"], undefined);
|
||||
assert.equal(nonStreaming["X-OmniRoute-Fallback-Attempts"], undefined);
|
||||
});
|
||||
|
||||
Reference in New Issue
Block a user