mirror of
https://github.com/diegosouzapw/OmniRoute.git
synced 2026-08-03 13:52:09 +03:00
* chore(release): open v3.8.35 development cycle
* fix db vacuum scheduler settings (#4726)
Scheduled VACUUM now follows Storage page settings (scheduledVacuum/vacuumHour) as single source of truth; env-flag control path removed. 11/11 vacuum-scheduler tests pass against release/v3.8.35 tip; no orphaned env refs. Integrated into release/v3.8.35.
* fix(tier): noAuth providers count as free; free filter returns empty … (#4753)
noAuth providers now classified free (union of legacy list + NOAUTH_PROVIDERS chat-tier derivation), -free arena_elo alias, and auto/<cat>:free returns an empty pool when no free candidate matches (opt-in legacy fallback via OMNIROUTE_AUTO_FREE_FALLBACK_TO_FULL_POOL). New env var documented in .env.example + ENVIRONMENT.md; CHANGELOG bullet added (maintainer co-author). 46/46 node + 56/56 vitest tests pass on release tip; env-doc-sync, docs-sync, typecheck:core, lint, file-size all green. Integrated into release/v3.8.35.
* refactor(chatCore): extrai 11 helpers de nível superior para 6 leaves puros (#3501) (#4571)
chatCore god-file decomposition (#3501): extract 6 pure leaves (cacheUsageMeta, executorClientHeaders, nonStreamingResponseBody, skillsFormat, streamErrorResult, streamFinalize) from chatCore.ts. Rebased onto release/v3.8.35 tip (resolved single chatCore.ts conflict — removed now-extracted inline buildExecutorClientHeaders). 265/265 chatcore tests, 26/26 new leaf tests, typecheck:core, cycles, file-size all green. Integrated into release/v3.8.35.
* refactor(chatCore): extrai resolveExecutorWithProxy + getExecutionCredentials para leaves (#3501) (#4646)
chatCore #3501: extract resolveExecutorWithProxy + getExecutionCredentials to leaves (executorProxy.ts, executionCredentials.ts). Clean cherry-pick onto release tip post-#4571. 12/12 new leaf tests, typecheck:core, cycles, file-size green. Integrated into release/v3.8.35.
* refactor(chatCore): extrai transforms de mensagens Claude p/ leaf (#3501) (#4708)
chatCore #3501: extract Claude upstream-message transforms to leaf (claudeUpstreamMessages.ts + claudeMessageTypes.ts). Clean cherry-pick post-#4646. 8/8 new leaf tests, typecheck/cycles/file-size green. Integrated into release/v3.8.35.
* refactor(chatCore): extrai persistAttemptLogs para leaf (#3501) (#4717)
chatCore #3501: extract persistAttemptLogs to leaf (attemptLogging.ts). Rebased onto release tip post-#4708 (resolved imports conflict: kept tip's resolveCompressionHeader from compression Phase 3, dropped now-unused logTruncation import moved into the leaf). 288/288 chatcore tests, typecheck/cycles/file-size green. Integrated into release/v3.8.35.
* refactor(chatCore): extrai stageTrace + compressionUsageReceipt para leaves (#3501) (#4721)
chatCore #3501: extract stageTrace + compressionUsageReceipt to leaves. Clean cherry-pick post-#4717. 6/6 new leaf tests, typecheck/cycles/file-size green. Integrated into release/v3.8.35.
* refactor(chatCore): extrai prepareUpstreamBody (1ª sub-fatia do executeProviderRequest, #3501) (#4730)
chatCore #3501: extract prepareUpstreamBody (first sub-slice of executeProviderRequest) to leaf (upstreamBody.ts). Clean cherry-pick post-#4721. 7/7 new leaf tests, full 301/301 chatcore suite, typecheck/cycles/file-size green. Completes the 6-PR chatCore decomposition stack into release/v3.8.35.
* fix(db): make db-backup import size cap configurable (#4719) (#4757)
Co-authored-by: Diego Rodrigues de Sa e Souza <souzamiriamrodrigues790@gmail.com>
* chore(quality): expand check:release-green to the FULL release-PR gate set (#4758)
The release-green pre-flight (Solution C) previously covered only a subset of the
gates that run exclusively on the release PR (PR→main), so reds still accrued
silently on release/** and surfaced in ~40-min layers at release time (v3.8.34:
3 CI rounds — CodeQL sanitization, then the fail-fast Quality Ratchet revealing
openapi then cyclomatic-complexity one push at a time, plus zizmor/integration).
Now check:release-green reproduces the COMPLETE release-PR gate set and reports
EVERY red in one pass (collected, not fail-fast):
- New DRIFT ratchets (report-only, rebaselined at release, never block):
cyclomatic complexity, dead-code, type-coverage, compression-budget,
openapi-coverage, workflow-lint (zizmor), codeql-ratchet.
- New HARD gates (real defects): docs-all (fabricated-docs strict + i18n mirror
sync) and the integration test suite (gated behind !--quick).
The only release-PR gates it still cannot reproduce locally are GitHub-side CodeQL
semantic analysis and SonarQube/SonarCloud (external services).
The nightly-release-green workflow and /green-prs inherit the expanded coverage
automatically (they invoke this script), so cycle drift is now surfaced
continuously and the release PR is green on its first CI run.
Co-authored-by: Diego Rodrigues de Sa e Souza <souzamiriamrodrigues790@gmail.com>
* fix(dashboard): add missing onboarding.tiers step title (#4698) (#4755)
Co-authored-by: Diego Rodrigues de Sa e Souza <souzamiriamrodrigues790@gmail.com>
* feat(compression): Output Styles registry + D0 telemetry (Phase 4A) (#4694)
Phase 4A: Output Styles registry + D0 telemetry. Integrated into release/v3.8.35.
* feat(compression): SLM tier for ultra (Phase 4B) [stacked on #4694] (#4707)
Phase 4B: SLM tier for ultra. Integrated into release/v3.8.35.
* feat(compression): context-budget adaptive compression (Phase 4C) [stacked on #4707] (#4716)
Phase 4C: adaptive context-budget compression. Integrated into release/v3.8.35.
* feat(compression): offline evaluation harness (Phase 4 D1) [stacked on #4716] (#4720)
Phase 4 D1: offline evaluation harness. Integrated into release/v3.8.35.
* fix(sse): deepseek-web folds role:tool results into prompt transcript (#4712) (#4756)
Co-authored-by: Diego Rodrigues de Sa e Souza <souzamiriamrodrigues790@gmail.com>
* fix(dashboard): remove dead unconditional useLiveRequests call in HomePageClient (#4759, #4745, #4596) (#4761)
Co-authored-by: Diego Rodrigues de Sa e Souza <souzamiriamrodrigues790@gmail.com>
* fix(dashboard): dedupe provider nodes by id on compatible-provider add (#4746) (#4768)
Co-authored-by: Diego Rodrigues de Sa e Souza <souzamiriamrodrigues790@gmail.com>
* chore(db): re-export compressionRunTelemetry from localDb to satisfy db-rules (#4775)
Co-authored-by: Diego Rodrigues de Sa e Souza <souzamiriamrodrigues790@gmail.com>
* docs(security): add canonical STRIDE-based threat model (#4783)
Canonical STRIDE threat model. Integrated into release/v3.8.35.
* test(dashboard): add smoke test for home client dashboard (#4793)
Smoke test guarding the dashboard home client render (regression #4745/#4759). Code fix already landed via #4761; this PR's jsdom smoke test is the net-new regression guard. Integrated into release/v3.8.35.
* fix(combos): auto-promote zeroLatencyOptimizationsEnabled so legacy configs (pre-3.8.33 fallbackCompressionMode="lite") round-trip on the first GUI edit (#4774)
Auto-promote zeroLatencyOptimizationsEnabled + strip v3.8.31-era removed keys so legacy combo configs round-trip through PUT /api/combos/{id} on first GUI edit (closes #4382 followup). Pre-merge: rewrote the now-stale reject test to assert auto-promotion + added passthrough/round-trip regression guards; reconciled combos/page.tsx file-size baseline. Integrated into release/v3.8.35.
* refactor(chatCore): extrai parse + usage-stats não-streaming do executeProviderRequest (#3501) (#4762)
chatCore #3501: extract parseNonStreamingResponseBody + recordNonStreamingUsageStats. Integrated into release/v3.8.35.
* refactor(chatCore): extrai recordContextEditingTelemetryHook (#3501) (#4779)
chatCore #3501: extract recordContextEditingTelemetryHook. Integrated into release/v3.8.35.
* refactor(chatCore): extrai recordCompressionCacheStats (#3501) (#4792)
chatCore #3501: extract recordCompressionCacheStats. Integrated into release/v3.8.35.
* refactor(chatCore): extrai writeCavemanOutputAnalytics (#3501) (#4794)
chatCore #3501: extract writeCavemanOutputAnalytics. Integrated into release/v3.8.35.
* refactor(chatCore): extrai scheduleQuotaShareConsumption (POST-hook não-streaming, #3501) (#4780)
chatCore #3501: extract scheduleQuotaShareConsumption (non-streaming POST-hook). Integrated into release/v3.8.35.
* refactor(chatCore): extrai emitRequestGamificationEvent (helper compartilhado DRY, #3501) (#4776)
chatCore #3501: extract emitRequestGamificationEvent (DRY streaming/non-streaming). Integrated into release/v3.8.35.
* refactor(chatCore): extrai runPluginOnResponseHook (#3501) (#4782)
chatCore #3501: extract runPluginOnResponseHook. Integrated into release/v3.8.35.
* refactor(chatCore): extrai scheduleStreamingQuotaShareConsumption (POST-hook streaming, #3501) (#4784)
chatCore #3501: extract scheduleStreamingQuotaShareConsumption (streaming POST-hook). Integrated into release/v3.8.35.
* refactor(chatCore): extrai recordStreamingUsageStats (analytics de usage streaming, #3501) (#4791)
chatCore #3501: extract recordStreamingUsageStats. Integrated into release/v3.8.35.
* refactor(chatCore): extrai recordStreamingCost (custo por-request streaming, #3501) (#4790)
chatCore #3501: extract recordStreamingCost (per-request streaming cost). Integrated into release/v3.8.35.
* docs(readme): credit ponytail + OmniCompress; restore env-doc-sync release-green (#4799)
README compression credits (ponytail/OmniCompress) + env-doc-sync ignore for eval-only OMNIROUTE_EVAL_CREDENTIALS (restores release-green after #4720). Integrated into release/v3.8.35.
* chore(quality): trim combo-config.test.ts comments under file-size cap (#4774 follow-up) (#4800)
Restore file-size release-green. Integrated into release/v3.8.35.
* feat(api-docs): Redoc-rendered /api/docs + consolidate OpenAPI spec to docs/openapi.yaml (#4781)
Redoc /api/docs + OpenAPI spec consolidated to docs/openapi.yaml (canonical 201-path complete spec; old path → legacy fallback). All refs/gates/tests/CI updated. Integrated into release/v3.8.35.
* docs(compression): declare Phase 4 layers — Output Styles, adaptive dial, per-request control (#4801)
The README compression section listed the 9 input engines but not the Phase 4
layers now in production:
- Output Styles (output-axis steering: terse-prose / less-code / terse-cjk, lite/full/ultra)
- adaptive context-budget dial (reserve-output|percentage|absolute · floor|replace-autotrigger|off)
- per-request x-omniroute-compression precedence + the offline eval harness
Also bumped the highlights range to v3.8.35, expanded the compression feature bullet,
and marked the GUIDE's Phase 4 row Shipped (was 'Planned' — it's merged on v3.8.35).
Co-authored-by: Diego Rodrigues de Sa e Souza <souzamiriamrodrigues790@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore(release): finalize v3.8.35 CHANGELOG + docs reconciliation
- CHANGELOG: complete 3.8.35 section (all 35 commits since v3.8.34,
contributor attribution: @rdself @megamen32 @KooshaPari @JxnLexn)
- docs(security): align THREAT_MODEL.md refs with real code
(routeGuard.ts, tokenLimits.ts, /api/monitoring/health) — fabricated-docs gate
- check:fabricated-docs: skip docs/superpowers/specs (dated research reports)
- i18n: sync 3.8.35 section into 41 CHANGELOG mirrors (docs-sync size gate)
- ratchet rebaseline: cyclomatic 1916->1920, eslintWarnings 3907->3912
(inherited cycle drift; release-finalize diff is docs-only)
* fix(release): resolve inherited base-reds surfaced by v3.8.35 release CI
Cycle base-reds that only run on PR→main (not the PR→release fast-path):
- test(autoCombo): suffixComposition-4517 used node:test in a vitest-only dir
(#4753) → vitest found no suite. Switch to the vitest API. (Vitest job)
- test(agentSkills): openapiParser fixture wrote docs/reference/openapi.yaml;
parser reads docs/openapi.yaml since #4781 → point fixture at the new path.
(Unit/Coverage/Node24/Node26 shard 4)
- test(integration): proxy-pipeline source-scan expected inline streaming-cost
code that #4790/#3501 extracted to the recordStreamingCost leaf → assert the
delegation instead. (Integration 1/2)
- fix(chatCore): derive the log trace id from crypto, not Math.random
(CodeQL js/insecure-randomness — log-correlation id, not a secret).
- test(resilience): circuit-breaker invalid-cooldown fallback asserted t>29000,
flaking on slow CI where ~1.6s elapsed gave t=28401 → tolerate wall-clock
drift (t>25000). (Unit 6/8)
* fix(usage): derive pending-request id from crypto, not Math.random
CodeQL js/insecure-randomness (#669): the pending-request id generated in
trackPendingRequest (usageHistory.ts) flows into attempt logging and was flagged
as insecure randomness in a security context. It's a log-correlation id, not a
secret — switch to crypto RNG to clear the alert. Pairs with the chatCore traceId
fix in 37c49781a (same sink).
---------
Co-authored-by: Diego Rodrigues de Sa e Souza <souzamiriamrodrigues790@gmail.com>
Co-authored-by: Randi <55005611+rdself@users.noreply.github.com>
Co-authored-by: Demiurge The Single <megamen932@gmail.com>
Co-authored-by: KooshaPari <42529354+KooshaPari@users.noreply.github.com>
Co-authored-by: Jan Leon <Jan.gaschler@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
799 lines
25 KiB
TypeScript
799 lines
25 KiB
TypeScript
import test from "node:test";
|
|
import assert from "node:assert/strict";
|
|
|
|
const {
|
|
resolveComboConfig,
|
|
getDefaultComboConfig,
|
|
resolveComboTargetTimeoutMs,
|
|
DEFAULT_COMBO_TARGET_TIMEOUT_MS,
|
|
resolveComboQueueDepth,
|
|
} = await import("../../open-sse/services/comboConfig.ts");
|
|
const { createComboSchema, updateComboDefaultsSchema } =
|
|
await import("../../src/shared/validation/schemas.ts");
|
|
const { MAX_TIMER_TIMEOUT_MS } = await import("../../src/shared/utils/runtimeTimeouts.ts");
|
|
|
|
test("getDefaultComboConfig returns a fresh copy of the defaults", () => {
|
|
const first = getDefaultComboConfig();
|
|
const second = getDefaultComboConfig();
|
|
|
|
assert.notEqual(first, second);
|
|
assert.equal(first.strategy, "priority");
|
|
assert.equal(first.maxRetries, 1);
|
|
assert.equal(first.retryDelayMs, 2000);
|
|
assert.equal(first.fallbackDelayMs, 0);
|
|
assert.ok(!("timeoutMs" in first));
|
|
assert.ok(!("healthCheckEnabled" in first));
|
|
assert.equal(first.handoffThreshold, 0.85);
|
|
assert.equal(first.maxMessagesForSummary, 30);
|
|
assert.deepEqual(first.handoffProviders, ["codex"]);
|
|
assert.equal(first.nestedComboMode, "flatten");
|
|
assert.equal(first.failoverBeforeRetry, true);
|
|
assert.equal(first.maxSetRetries, 0);
|
|
assert.equal(first.setRetryDelayMs, 2000);
|
|
assert.equal(first.reasoningTokenBufferEnabled, true);
|
|
assert.equal(first.zeroLatencyOptimizationsEnabled, false);
|
|
assert.equal(first.hedging, false);
|
|
assert.equal(first.fallbackCompressionMode, "lite");
|
|
assert.equal(first.fallbackCompressionThreshold, 1000);
|
|
assert.equal(first.predictiveTtftMs, 0);
|
|
assert.equal(first.evalRouting.enabled, false);
|
|
assert.equal(first.evalRouting.maxAgeHours, 720);
|
|
|
|
first.strategy = "weighted";
|
|
assert.equal(second.strategy, "priority");
|
|
});
|
|
|
|
test("resolveComboConfig applies the full cascade from defaults to combo overrides", () => {
|
|
const result = resolveComboConfig(
|
|
{
|
|
config: {
|
|
maxRetries: 4,
|
|
},
|
|
},
|
|
{
|
|
comboDefaults: {
|
|
strategy: "round-robin",
|
|
timeoutMs: 120000,
|
|
targetTimeoutMs: 90000,
|
|
},
|
|
providerOverrides: {
|
|
openai: {
|
|
timeoutMs: 60000,
|
|
targetTimeoutMs: 45000,
|
|
retryDelayMs: 500,
|
|
fallbackDelayMs: 100,
|
|
},
|
|
},
|
|
},
|
|
"openai"
|
|
);
|
|
|
|
assert.equal(result.strategy, "round-robin");
|
|
assert.equal(result.retryDelayMs, 500);
|
|
assert.equal(result.fallbackDelayMs, 100);
|
|
assert.equal(result.maxRetries, 4);
|
|
assert.equal(result.targetTimeoutMs, 45000);
|
|
assert.ok(!("timeoutMs" in result));
|
|
assert.ok(!("healthCheckEnabled" in result));
|
|
});
|
|
|
|
test("resolveComboConfig cascades reasoning token buffer feature flag", () => {
|
|
const providerDisabled = resolveComboConfig(
|
|
{},
|
|
{
|
|
comboDefaults: {
|
|
reasoningTokenBufferEnabled: true,
|
|
},
|
|
providerOverrides: {
|
|
openai: {
|
|
reasoningTokenBufferEnabled: false,
|
|
},
|
|
},
|
|
},
|
|
"openai"
|
|
);
|
|
|
|
const comboEnabled = resolveComboConfig(
|
|
{
|
|
config: {
|
|
reasoningTokenBufferEnabled: true,
|
|
},
|
|
},
|
|
{
|
|
comboDefaults: {
|
|
reasoningTokenBufferEnabled: false,
|
|
},
|
|
}
|
|
);
|
|
|
|
assert.equal(providerDisabled.reasoningTokenBufferEnabled, false);
|
|
assert.equal(comboEnabled.reasoningTokenBufferEnabled, true);
|
|
});
|
|
|
|
test("resolveComboConfig preserves nested routing defaults for partial overrides", () => {
|
|
const result = resolveComboConfig(
|
|
{
|
|
config: {
|
|
shadowRouting: { enabled: true },
|
|
evalRouting: { enabled: true, suiteIds: ["coding-proficiency"] },
|
|
},
|
|
},
|
|
{
|
|
comboDefaults: {
|
|
shadowRouting: { sampleRate: 0.5 },
|
|
evalRouting: { maxAgeHours: 168 },
|
|
},
|
|
}
|
|
);
|
|
|
|
assert.equal(result.shadowRouting.enabled, true);
|
|
assert.equal(result.shadowRouting.sampleRate, 0.5);
|
|
assert.equal(result.shadowRouting.maxTargets, 2);
|
|
assert.equal(result.shadowRouting.timeoutMs, 30000);
|
|
assert.equal(result.evalRouting.enabled, true);
|
|
assert.deepEqual(result.evalRouting.suiteIds, ["coding-proficiency"]);
|
|
assert.equal(result.evalRouting.maxAgeHours, 168);
|
|
assert.equal(result.evalRouting.minCases, 1);
|
|
assert.equal(result.evalRouting.cacheTtlMs, 60000);
|
|
});
|
|
|
|
test("resolveComboConfig ignores null, undefined, and legacy resilience overrides", () => {
|
|
const result = resolveComboConfig(
|
|
{
|
|
config: {
|
|
timeoutMs: null,
|
|
trackMetrics: false,
|
|
},
|
|
},
|
|
{
|
|
comboDefaults: {
|
|
timeoutMs: undefined,
|
|
queueTimeoutMs: 15000,
|
|
},
|
|
providerOverrides: {
|
|
openai: {
|
|
strategy: null,
|
|
concurrencyPerModel: 9,
|
|
},
|
|
},
|
|
},
|
|
"openai"
|
|
);
|
|
|
|
assert.ok(!("timeoutMs" in result));
|
|
assert.equal(result.queueTimeoutMs, 15000);
|
|
assert.equal(result.concurrencyPerModel, 9);
|
|
assert.equal(result.trackMetrics, false);
|
|
assert.equal(result.strategy, "priority");
|
|
});
|
|
|
|
test("updateComboDefaultsSchema accepts arbitrarily large timeout defaults and provider overrides", () => {
|
|
const parsed = updateComboDefaultsSchema.parse({
|
|
comboDefaults: {
|
|
timeoutMs: 3600000,
|
|
targetTimeoutMs: 30000,
|
|
reasoningTokenBufferEnabled: false,
|
|
},
|
|
providerOverrides: {
|
|
anthropic: {
|
|
timeoutMs: 5400000,
|
|
targetTimeoutMs: 45000,
|
|
reasoningTokenBufferEnabled: false,
|
|
},
|
|
},
|
|
});
|
|
|
|
assert.equal(parsed.comboDefaults.timeoutMs, 3600000);
|
|
assert.equal(parsed.comboDefaults.targetTimeoutMs, 30000);
|
|
assert.equal(parsed.comboDefaults.reasoningTokenBufferEnabled, false);
|
|
assert.equal(parsed.providerOverrides.anthropic.timeoutMs, 5400000);
|
|
assert.equal(parsed.providerOverrides.anthropic.targetTimeoutMs, 45000);
|
|
assert.equal(parsed.providerOverrides.anthropic.reasoningTokenBufferEnabled, false);
|
|
});
|
|
|
|
test("combo config schema accepts explicit zero-latency opt-in controls", () => {
|
|
const parsed = createComboSchema.parse({
|
|
name: "zero-latency-opt-in",
|
|
models: ["openai/gpt-4o-mini", "anthropic/claude-3-haiku"],
|
|
config: {
|
|
zeroLatencyOptimizationsEnabled: true,
|
|
hedging: true,
|
|
hedgeDelayMs: 250,
|
|
fallbackCompressionMode: "lite",
|
|
fallbackCompressionThreshold: 2500,
|
|
predictiveTtftMs: 1800,
|
|
},
|
|
});
|
|
|
|
assert.equal(parsed.config.zeroLatencyOptimizationsEnabled, true);
|
|
assert.equal(parsed.config.hedging, true);
|
|
assert.equal(parsed.config.hedgeDelayMs, 250);
|
|
assert.equal(parsed.config.fallbackCompressionMode, "lite");
|
|
assert.equal(parsed.config.fallbackCompressionThreshold, 2500);
|
|
assert.equal(parsed.config.predictiveTtftMs, 1800);
|
|
});
|
|
|
|
test("combo config schema auto-promotes the zero-latency gate for legacy configs without opt-in", () => {
|
|
// Pre-3.8.33 combos carry zero-latency subfeatures without the
|
|
// zeroLatencyOptimizationsEnabled gate. The schema now auto-promotes the gate
|
|
// (instead of 400-ing on the first GUI edit) so they round-trip. See #4774/#4382.
|
|
const result = createComboSchema.safeParse({
|
|
name: "zero-latency-legacy",
|
|
models: ["openai/gpt-4o-mini", "anthropic/claude-3-haiku"],
|
|
config: {
|
|
hedging: true,
|
|
fallbackCompressionMode: "lite",
|
|
predictiveTtftMs: 1800,
|
|
},
|
|
});
|
|
|
|
assert.equal(result.success, true);
|
|
assert.equal(result.data.config.zeroLatencyOptimizationsEnabled, true);
|
|
// The enabled subfeatures are preserved verbatim.
|
|
assert.equal(result.data.config.hedging, true);
|
|
assert.equal(result.data.config.fallbackCompressionMode, "lite");
|
|
assert.equal(result.data.config.predictiveTtftMs, 1800);
|
|
});
|
|
|
|
test("combo config schema leaves the zero-latency gate untouched when no subfeature is enabled", () => {
|
|
// A plain config with no zero-latency subfeature must NOT be auto-promoted —
|
|
// the gate stays at its default (false) so we don't silently flip optimizations on.
|
|
const result = createComboSchema.safeParse({
|
|
name: "no-zero-latency",
|
|
models: ["openai/gpt-4o-mini", "anthropic/claude-3-haiku"],
|
|
config: {
|
|
fallbackCompressionMode: "off",
|
|
},
|
|
});
|
|
|
|
assert.equal(result.success, true);
|
|
assert.notEqual(result.data.config.zeroLatencyOptimizationsEnabled, true);
|
|
});
|
|
|
|
test("combo config schema no longer rejects v3.8.31-era removed config keys (#4382 round-trip)", () => {
|
|
// Keys dropped after v3.8.31 still live in stored JSON. The schema switched
|
|
// .strict() (which 400'd) → .passthrough(); the route + migration 103 scrub them.
|
|
const result = createComboSchema.safeParse({
|
|
name: "legacy-removed-keys",
|
|
models: ["openai/gpt-4o-mini", "anthropic/claude-3-haiku"],
|
|
config: {
|
|
queueDepth: 4,
|
|
fallbackDelayMs: 200,
|
|
maxComboDepth: 3,
|
|
shadowRouting: { enabled: false },
|
|
resetAwareEnabled: true,
|
|
},
|
|
});
|
|
|
|
assert.equal(result.success, true);
|
|
});
|
|
|
|
test("combo config schema allows zero-latency tuning fields when subfeatures stay disabled", () => {
|
|
const parsed = createComboSchema.parse({
|
|
name: "zero-latency-disabled-tuning",
|
|
models: ["openai/gpt-4o-mini", "anthropic/claude-3-haiku"],
|
|
config: {
|
|
hedgeDelayMs: 250,
|
|
fallbackCompressionMode: "off",
|
|
fallbackCompressionThreshold: 2500,
|
|
predictiveTtftMs: 0,
|
|
},
|
|
});
|
|
|
|
assert.equal(parsed.config.hedgeDelayMs, 250);
|
|
assert.equal(parsed.config.fallbackCompressionMode, "off");
|
|
assert.equal(parsed.config.fallbackCompressionThreshold, 2500);
|
|
assert.equal(parsed.config.predictiveTtftMs, 0);
|
|
});
|
|
|
|
test("resolveComboTargetTimeoutMs inherits the upstream timeout and only shortens it", () => {
|
|
assert.equal(resolveComboTargetTimeoutMs({}, 600000), 600000);
|
|
assert.equal(resolveComboTargetTimeoutMs({ targetTimeoutMs: 30000 }, 600000), 30000);
|
|
assert.equal(resolveComboTargetTimeoutMs({ targetTimeoutMs: 900000 }, 600000), 600000);
|
|
assert.equal(resolveComboTargetTimeoutMs({ targetTimeoutMs: 0 }, 600000), 600000);
|
|
assert.equal(resolveComboTargetTimeoutMs({ targetTimeoutMs: 30000 }, 0), 30000);
|
|
assert.equal(resolveComboTargetTimeoutMs({}, 0), 0);
|
|
assert.equal(
|
|
resolveComboTargetTimeoutMs({ targetTimeoutMs: 999999999999 }, 0),
|
|
MAX_TIMER_TIMEOUT_MS
|
|
);
|
|
assert.equal(resolveComboTargetTimeoutMs({}, 999999999999), MAX_TIMER_TIMEOUT_MS);
|
|
});
|
|
|
|
test("resolveComboTargetTimeoutMs falls back to the saner combo default when unset", () => {
|
|
// The combo default is the documented 120s fallback-latency cap.
|
|
assert.equal(DEFAULT_COMBO_TARGET_TIMEOUT_MS, 120000);
|
|
// Unset config → use the default (capped at the ceiling), NOT the full upstream ceiling.
|
|
// This is what shortens a hung-target failover from 600s to 120s (escalated cmqlrhd7c).
|
|
assert.equal(resolveComboTargetTimeoutMs({}, 600000, 120000), 120000);
|
|
// Operators can still extend beyond the default, up to the ceiling.
|
|
assert.equal(resolveComboTargetTimeoutMs({ targetTimeoutMs: 300000 }, 600000, 120000), 300000);
|
|
// Explicit config above the ceiling is still capped at the ceiling.
|
|
assert.equal(resolveComboTargetTimeoutMs({ targetTimeoutMs: 900000 }, 600000, 120000), 600000);
|
|
// A default larger than the ceiling is clamped to the ceiling.
|
|
assert.equal(resolveComboTargetTimeoutMs({}, 100000, 120000), 100000);
|
|
// Backward-compat: omitting the default arg keeps the legacy inherit-the-ceiling behavior.
|
|
assert.equal(resolveComboTargetTimeoutMs({}, 600000), 600000);
|
|
// Disabled upstream timeout (0 = unbounded) stays unbounded even with a default present.
|
|
assert.equal(resolveComboTargetTimeoutMs({}, 0, 120000), 0);
|
|
});
|
|
|
|
test("combo timeout schema rejects values beyond the safe timer limit", () => {
|
|
const result = createComboSchema.safeParse({
|
|
name: "unsafe-timeout",
|
|
models: ["openai/gpt-4"],
|
|
config: {
|
|
targetTimeoutMs: MAX_TIMER_TIMEOUT_MS + 1,
|
|
},
|
|
});
|
|
|
|
assert.equal(result.success, false);
|
|
});
|
|
|
|
test("resolveComboConfig preserves explicit empty handoffProviders overrides", () => {
|
|
const result = resolveComboConfig(
|
|
{
|
|
config: {
|
|
handoffProviders: [],
|
|
},
|
|
},
|
|
{
|
|
comboDefaults: {
|
|
handoffProviders: ["codex"],
|
|
},
|
|
}
|
|
);
|
|
|
|
assert.deepEqual(result.handoffProviders, []);
|
|
});
|
|
|
|
test("resolveComboConfig skips provider overrides when provider is absent", () => {
|
|
const result = resolveComboConfig(
|
|
{ config: {} },
|
|
{
|
|
comboDefaults: { strategy: "random" },
|
|
providerOverrides: {
|
|
openai: { strategy: "weighted" },
|
|
},
|
|
}
|
|
);
|
|
|
|
assert.equal(result.strategy, "random");
|
|
});
|
|
|
|
test("resolveComboConfig tolerates invalid or missing inputs and falls back to defaults", () => {
|
|
assert.deepEqual(resolveComboConfig(null, null, "openai"), getDefaultComboConfig());
|
|
assert.deepEqual(resolveComboConfig({}, { comboDefaults: null }, null), getDefaultComboConfig());
|
|
});
|
|
|
|
test("createComboSchema accepts context-relay strategy with handoff config", () => {
|
|
const parsed = createComboSchema.parse({
|
|
name: "codex-relay",
|
|
models: ["codex/gpt-5.4"],
|
|
strategy: "context-relay",
|
|
config: {
|
|
handoffThreshold: 0.85,
|
|
maxMessagesForSummary: 24,
|
|
handoffModel: "",
|
|
},
|
|
});
|
|
|
|
assert.equal(parsed.strategy, "context-relay");
|
|
assert.equal(parsed.config.handoffThreshold, 0.85);
|
|
assert.equal(parsed.config.maxMessagesForSummary, 24);
|
|
});
|
|
|
|
test("createComboSchema accepts eval-driven routing config", () => {
|
|
const parsed = createComboSchema.parse({
|
|
name: "eval-ranked",
|
|
models: ["openai/gpt-4o-mini", "anthropic/claude-3-haiku"],
|
|
strategy: "priority",
|
|
config: {
|
|
evalRouting: {
|
|
enabled: true,
|
|
suiteIds: ["golden-set", "coding-proficiency"],
|
|
maxAgeHours: 168,
|
|
minCases: 5,
|
|
qualityWeight: 0.9,
|
|
latencyWeight: 0.1,
|
|
cacheTtlMs: 30000,
|
|
},
|
|
},
|
|
});
|
|
|
|
assert.equal(parsed.config.evalRouting.enabled, true);
|
|
assert.deepEqual(parsed.config.evalRouting.suiteIds, ["golden-set", "coding-proficiency"]);
|
|
});
|
|
|
|
test("createComboSchema accepts SLA-aware auto routing config", () => {
|
|
const parsed = createComboSchema.parse({
|
|
name: "sla-auto",
|
|
models: ["openai/gpt-4o-mini", "gemini/gemini-2.5-flash"],
|
|
strategy: "auto",
|
|
config: {
|
|
routerStrategy: "sla-aware",
|
|
slaTargetP95Ms: "1500",
|
|
slaMaxErrorRate: "0.05",
|
|
slaMaxCostPer1MTokens: "4.5",
|
|
slaHardConstraints: true,
|
|
sla: {
|
|
targetP95Ms: "2000",
|
|
maxErrorRate: "0.1",
|
|
hardConstraints: false,
|
|
},
|
|
},
|
|
});
|
|
|
|
assert.equal(parsed.strategy, "auto");
|
|
assert.equal(parsed.config.routerStrategy, "sla-aware");
|
|
assert.equal(parsed.config.slaTargetP95Ms, 1500);
|
|
assert.equal(parsed.config.slaMaxErrorRate, 0.05);
|
|
assert.equal(parsed.config.slaMaxCostPer1MTokens, 4.5);
|
|
assert.equal(parsed.config.slaHardConstraints, true);
|
|
assert.equal(parsed.config.sla.targetP95Ms, 2000);
|
|
assert.equal(parsed.config.sla.maxErrorRate, 0.1);
|
|
});
|
|
|
|
test("createComboSchema accepts structured combo steps with pinned connection and combo refs", () => {
|
|
const parsed = createComboSchema.parse({
|
|
name: "codex-pinned",
|
|
strategy: "priority",
|
|
models: [
|
|
{
|
|
kind: "model",
|
|
id: "step-codex-a",
|
|
providerId: "codex",
|
|
model: "gpt-5.4",
|
|
connectionId: "conn-codex-a",
|
|
weight: 10,
|
|
},
|
|
{
|
|
kind: "combo-ref",
|
|
id: "step-fallback",
|
|
comboName: "backup-codex",
|
|
weight: 5,
|
|
},
|
|
],
|
|
});
|
|
|
|
assert.equal(parsed.models[0].kind, "model");
|
|
assert.equal(parsed.models[0].providerId, "codex");
|
|
assert.equal(parsed.models[0].connectionId, "conn-codex-a");
|
|
assert.equal(parsed.models[1].kind, "combo-ref");
|
|
assert.equal(parsed.models[1].comboName, "backup-codex");
|
|
});
|
|
|
|
test("createComboSchema accepts composite tiers that reference normalized combo steps", () => {
|
|
const parsed = createComboSchema.parse({
|
|
name: "tiered-codex",
|
|
strategy: "priority",
|
|
models: [
|
|
{
|
|
kind: "model",
|
|
id: "step-primary",
|
|
providerId: "codex",
|
|
model: "gpt-5.4",
|
|
connectionId: "conn-codex-a",
|
|
},
|
|
{
|
|
kind: "model",
|
|
id: "step-backup",
|
|
providerId: "codex",
|
|
model: "gpt-5.4",
|
|
connectionId: "conn-codex-b",
|
|
},
|
|
],
|
|
config: {
|
|
compositeTiers: {
|
|
defaultTier: "primary",
|
|
tiers: {
|
|
primary: {
|
|
stepId: "step-primary",
|
|
fallbackTier: "backup",
|
|
label: "Codex A",
|
|
},
|
|
backup: {
|
|
stepId: "step-backup",
|
|
description: "Fallback account",
|
|
},
|
|
},
|
|
},
|
|
},
|
|
});
|
|
|
|
assert.equal(parsed.config.compositeTiers.defaultTier, "primary");
|
|
assert.equal(parsed.config.compositeTiers.tiers.primary.stepId, "step-primary");
|
|
assert.equal(parsed.config.compositeTiers.tiers.primary.fallbackTier, "backup");
|
|
assert.equal(parsed.config.compositeTiers.tiers.backup.stepId, "step-backup");
|
|
});
|
|
|
|
test("updateComboDefaultsSchema rejects composite tiers in global defaults and provider overrides", () => {
|
|
const result = updateComboDefaultsSchema.safeParse({
|
|
comboDefaults: {
|
|
compositeTiers: {
|
|
defaultTier: "primary",
|
|
tiers: {
|
|
primary: {
|
|
stepId: "step-primary",
|
|
},
|
|
},
|
|
},
|
|
},
|
|
providerOverrides: {
|
|
codex: {
|
|
compositeTiers: {
|
|
defaultTier: "backup",
|
|
tiers: {
|
|
backup: {
|
|
stepId: "step-backup",
|
|
},
|
|
},
|
|
},
|
|
},
|
|
},
|
|
});
|
|
|
|
assert.equal(result.success, false);
|
|
assert.deepEqual(
|
|
result.error.issues.map((issue) => ({
|
|
path: issue.path.join("."),
|
|
message: issue.message,
|
|
})),
|
|
[
|
|
{
|
|
path: "comboDefaults.compositeTiers",
|
|
message: "compositeTiers is only supported on concrete combos",
|
|
},
|
|
{
|
|
path: "providerOverrides.codex.compositeTiers",
|
|
message: "compositeTiers is only supported on concrete combos",
|
|
},
|
|
]
|
|
);
|
|
});
|
|
|
|
test("createComboSchema accepts failoverBeforeRetry, maxSetRetries and setRetryDelayMs", () => {
|
|
const parsed = createComboSchema.parse({
|
|
name: "failover-test",
|
|
models: ["openai/gpt-4"],
|
|
strategy: "priority",
|
|
config: {
|
|
failoverBeforeRetry: true,
|
|
maxSetRetries: 3,
|
|
setRetryDelayMs: 1500,
|
|
},
|
|
});
|
|
|
|
assert.equal(parsed.config.failoverBeforeRetry, true);
|
|
assert.equal(parsed.config.maxSetRetries, 3);
|
|
assert.equal(parsed.config.setRetryDelayMs, 1500);
|
|
});
|
|
|
|
test("createComboSchema accepts nestedComboMode and rejects invalid values", () => {
|
|
const parsed = createComboSchema.parse({
|
|
name: "nested-execute",
|
|
models: [{ kind: "combo-ref", comboName: "child" }],
|
|
strategy: "priority",
|
|
config: { nestedComboMode: "execute" },
|
|
});
|
|
assert.equal(parsed.config.nestedComboMode, "execute");
|
|
|
|
const flatten = createComboSchema.parse({
|
|
name: "nested-flatten",
|
|
models: ["openai/gpt-4o-mini"],
|
|
config: { nestedComboMode: "flatten" },
|
|
});
|
|
assert.equal(flatten.config.nestedComboMode, "flatten");
|
|
|
|
const invalid = createComboSchema.safeParse({
|
|
name: "nested-invalid",
|
|
models: ["openai/gpt-4o-mini"],
|
|
config: { nestedComboMode: "redirect" },
|
|
});
|
|
assert.equal(invalid.success, false);
|
|
});
|
|
|
|
test("createComboSchema accepts per-combo stickyRoundRobinLimit and rejects out-of-range", () => {
|
|
const parsed = createComboSchema.parse({
|
|
name: "sticky-override",
|
|
models: ["openai/gpt-4o-mini"],
|
|
strategy: "round-robin",
|
|
config: { stickyRoundRobinLimit: 2 },
|
|
});
|
|
assert.equal(parsed.config.stickyRoundRobinLimit, 2);
|
|
|
|
const tooHigh = createComboSchema.safeParse({
|
|
name: "sticky-too-high",
|
|
models: ["openai/gpt-4o-mini"],
|
|
strategy: "round-robin",
|
|
config: { stickyRoundRobinLimit: 1001 },
|
|
});
|
|
assert.equal(tooHigh.success, false);
|
|
});
|
|
|
|
test("createComboSchema accepts per-combo stickyWeightedLimit and rejects out-of-range", () => {
|
|
const parsed = createComboSchema.parse({
|
|
name: "sticky-weighted",
|
|
models: [{ model: "openai/gpt-4o-mini", weight: 100 }],
|
|
strategy: "weighted",
|
|
config: { stickyWeightedLimit: 2 },
|
|
});
|
|
assert.equal(parsed.config.stickyWeightedLimit, 2);
|
|
|
|
const tooHigh = createComboSchema.safeParse({
|
|
name: "sticky-weighted-too-high",
|
|
models: [{ model: "openai/gpt-4o-mini", weight: 100 }],
|
|
strategy: "weighted",
|
|
config: { stickyWeightedLimit: 1001 },
|
|
});
|
|
assert.equal(tooHigh.success, false);
|
|
});
|
|
|
|
test("createComboSchema coerces string numbers for maxSetRetries and setRetryDelayMs", () => {
|
|
const parsed = createComboSchema.parse({
|
|
name: "coerce-test",
|
|
models: ["openai/gpt-4"],
|
|
strategy: "priority",
|
|
config: {
|
|
maxSetRetries: "2",
|
|
setRetryDelayMs: "500",
|
|
},
|
|
});
|
|
|
|
assert.equal(parsed.config.maxSetRetries, 2);
|
|
assert.equal(parsed.config.setRetryDelayMs, 500);
|
|
});
|
|
|
|
test("createComboSchema rejects maxSetRetries out of range", () => {
|
|
const tooHigh = createComboSchema.safeParse({
|
|
name: "bad-max",
|
|
models: ["openai/gpt-4"],
|
|
strategy: "priority",
|
|
config: { maxSetRetries: 11 },
|
|
});
|
|
assert.equal(tooHigh.success, false);
|
|
|
|
const negative = createComboSchema.safeParse({
|
|
name: "bad-max",
|
|
models: ["openai/gpt-4"],
|
|
strategy: "priority",
|
|
config: { maxSetRetries: -1 },
|
|
});
|
|
assert.equal(negative.success, false);
|
|
});
|
|
|
|
test("createComboSchema rejects setRetryDelayMs out of range", () => {
|
|
const tooHigh = createComboSchema.safeParse({
|
|
name: "bad-delay",
|
|
models: ["openai/gpt-4"],
|
|
strategy: "priority",
|
|
config: { setRetryDelayMs: 60001 },
|
|
});
|
|
assert.equal(tooHigh.success, false);
|
|
|
|
const negative = createComboSchema.safeParse({
|
|
name: "bad-delay",
|
|
models: ["openai/gpt-4"],
|
|
strategy: "priority",
|
|
config: { setRetryDelayMs: -1 },
|
|
});
|
|
assert.equal(negative.success, false);
|
|
});
|
|
|
|
test("resolveComboConfig cascades nestedComboMode", () => {
|
|
const result = resolveComboConfig(
|
|
{ config: { nestedComboMode: "execute" } },
|
|
{ comboDefaults: { nestedComboMode: "flatten" } }
|
|
);
|
|
assert.equal(result.nestedComboMode, "execute");
|
|
|
|
const defaulted = resolveComboConfig({ config: {} }, { comboDefaults: {} });
|
|
assert.equal(defaulted.nestedComboMode, "flatten");
|
|
});
|
|
|
|
test("resolveComboConfig cascades failoverBeforeRetry, maxSetRetries and setRetryDelayMs", () => {
|
|
const result = resolveComboConfig(
|
|
{
|
|
config: {
|
|
failoverBeforeRetry: true,
|
|
maxSetRetries: 2,
|
|
setRetryDelayMs: 3000,
|
|
},
|
|
},
|
|
{
|
|
comboDefaults: {
|
|
failoverBeforeRetry: false,
|
|
maxSetRetries: 0,
|
|
setRetryDelayMs: 2000,
|
|
},
|
|
}
|
|
);
|
|
|
|
assert.equal(result.failoverBeforeRetry, true);
|
|
assert.equal(result.maxSetRetries, 2);
|
|
assert.equal(result.setRetryDelayMs, 3000);
|
|
});
|
|
|
|
// Issue #3872: combo round-robin always queued ~20 deep before cascading on
|
|
// semaphore saturation because handleRoundRobinCombo never threaded a queue depth
|
|
// into accountSemaphore.acquire (it fell back to the hardcoded DEFAULT_MAX_QUEUE_SIZE
|
|
// of 20). Expose `queueDepth` as combo config so operators can shrink the pre-cascade
|
|
// queue (0 = fail over immediately) for faster failover, while keeping 20 as the
|
|
// backward-compatible default.
|
|
test("getDefaultComboConfig exposes the backward-compatible queueDepth default of 20", () => {
|
|
assert.equal(getDefaultComboConfig().queueDepth, 20);
|
|
});
|
|
|
|
test("resolveComboConfig cascades queueDepth from defaults through provider and combo overrides", () => {
|
|
const fromDefault = resolveComboConfig({ config: {} }, { comboDefaults: {} });
|
|
assert.equal(fromDefault.queueDepth, 20);
|
|
|
|
const cascaded = resolveComboConfig(
|
|
{
|
|
config: {
|
|
queueDepth: 1,
|
|
},
|
|
},
|
|
{
|
|
comboDefaults: {
|
|
queueDepth: 10,
|
|
},
|
|
providerOverrides: {
|
|
openai: {
|
|
queueDepth: 5,
|
|
},
|
|
},
|
|
},
|
|
"openai"
|
|
);
|
|
|
|
// Most specific (combo.config) wins over provider override and global default.
|
|
assert.equal(cascaded.queueDepth, 1);
|
|
});
|
|
|
|
test("resolveComboQueueDepth defaults to 20, honors configured values, and clamps the range", () => {
|
|
assert.equal(resolveComboQueueDepth(null), 20);
|
|
assert.equal(resolveComboQueueDepth({}), 20);
|
|
assert.equal(resolveComboQueueDepth({ queueDepth: 5 }), 5);
|
|
// 0 is a valid, meaningful value: queue nothing → fail over to the next member immediately.
|
|
assert.equal(resolveComboQueueDepth({ queueDepth: 0 }), 0);
|
|
// Invalid / negative inputs fall back to the safe default.
|
|
assert.equal(resolveComboQueueDepth({ queueDepth: -3 }), 20);
|
|
assert.equal(resolveComboQueueDepth({ queueDepth: Number.NaN }), 20);
|
|
// Out-of-range high values are clamped, not trusted.
|
|
assert.equal(resolveComboQueueDepth({ queueDepth: 99999 }), 100);
|
|
// Fractional values floor to an integer queue slot count.
|
|
assert.equal(resolveComboQueueDepth({ queueDepth: 3.9 }), 3);
|
|
});
|
|
|
|
test("createComboSchema accepts queueDepth, coerces strings, and allows 0 for immediate failover", () => {
|
|
const parsed = createComboSchema.parse({
|
|
name: "fast-failover",
|
|
models: ["openai/gpt-4o-mini", "anthropic/claude-3-haiku"],
|
|
strategy: "round-robin",
|
|
config: {
|
|
queueDepth: "0",
|
|
},
|
|
});
|
|
|
|
assert.equal(parsed.config.queueDepth, 0);
|
|
});
|
|
|
|
test("createComboSchema rejects queueDepth outside the supported range", () => {
|
|
const tooHigh = createComboSchema.safeParse({
|
|
name: "bad-queue-depth-high",
|
|
models: ["openai/gpt-4"],
|
|
strategy: "round-robin",
|
|
config: { queueDepth: 101 },
|
|
});
|
|
assert.equal(tooHigh.success, false);
|
|
|
|
const negative = createComboSchema.safeParse({
|
|
name: "bad-queue-depth-negative",
|
|
models: ["openai/gpt-4"],
|
|
strategy: "round-robin",
|
|
config: { queueDepth: -1 },
|
|
});
|
|
assert.equal(negative.success, false);
|
|
});
|