mirror of
https://github.com/diegosouzapw/OmniRoute.git
synced 2026-07-26 09:52:11 +03:00
Release v3.8.43 (#5609)
* chore(release): open v3.8.43 development cycle * docs(relay): clarify backend routing contract (#5621) Integrated into release/v3.8.43 (drift-shed: cherry-picked the real change onto the release tip; stale-base drift dropped). * fix(security): avoid rendering error stacks (#5624) Integrated into release/v3.8.43 (drift-shed: cherry-picked the real change onto the release tip; stale-base drift dropped). * fix(chatgpt-web): restore dot-form Pro model ids (#5549) Integrated into release/v3.8.43 (drift-shed: cherry-picked the real change onto the release tip; stale-base drift dropped). * feat(commandCode): add multimodal image support for CC vision models (#5557) Integrated into release/v3.8.43 (drift-shed: cherry-picked the real change onto the release tip; stale-base drift dropped). * fix(providers): validate M365 Copilot web credentials (#5432) Integrated into release/v3.8.43 (drift-shed: cherry-picked the real change onto the release tip; stale-base drift dropped). * fix(sse): bound chat hot-path heap — pressure-aware admission + response cap + clone reductions (#5152) (#5425) Integrated into release/v3.8.43 (drift-shed: cherry-picked the real change onto the release tip; stale-base drift dropped). * fix: model lockout not recording for 429 rate_limit_exceeded from Antigravity ## Problem When Antigravity returns HTTP 429 with `rate_limit_exceeded` error code, the model lockout system never records the failure, so the model is not cooled down despite being rate-limited. ### Root Cause Antigravity's 429 error text is: `"Resource has been exhausted (e.g. check quota)."` The QUOTA_PATTERNS in `classify429.ts` contained overly broad regexes: - `/resource.*exhaust/i` — matches "Resource has been exhausted" - `/check.*quota/i` — matches "check quota" This caused `classifyErrorText()` to return `QUOTA_EXHAUSTED` (wrong), which set `providerExhausted = true` in the combo target exhaustion logic. With `providerExhausted`, the retry path was skipped entirely, and while the "done retrying" path should still record lockout, the misclassification cascaded into incorrect provider-level exhaustion state. Additionally, `targetExhaustion.ts` used the raw error text string instead of the structured error code (`rate_limit_exceeded`) that was already parsed from the response body. ## Fix 1. **classify429.ts** — Removed overly broad `/resource.*exhaust/i` and `/check.*quota/i` from QUOTA_PATTERNS. Antigravity's rate-limit wording is not a true quota exhaustion signal. 2. **targetExhaustion.ts** — Added optional `structuredError` to `ApplyComboTargetExhaustionOptions`. When available, the structured error code (e.g. `rate_limit_exceeded`) takes precedence over raw error text for exhaustion classification. 3. **combo.ts** — Passes `structuredError` to both `applyComboTargetExhaustion` call sites (dispatch path + retry-or-rotate path). ## Effect `structuredError.code = "rate_limit_exceeded"` → classified as rate-limit (not quota) → `providerExhausted = false` → retry proceeds → `recordModelLockoutFailure` called → model enters lockout with proper cooldown (120s base, exponential backoff). ## Tests Added 2 new tests for `structuredError.code` precedence in exhaustion classification. All 28 related tests pass. * fix(checks): normalize route paths on windows (#5613) Integrated into release/v3.8.43. Windows path-normalization fix for the route-guard membership gate + regression test (Rule #18). Co-authored test added by maintainer. * fix: truncate tool list when provider limit exceeds MAX_TOOLS_LIMIT (grok-cli 200) - Add proactive PROVIDER_TOOL_LIMITS map with grok-cli: 200 - Fix regex to capture 'maximum is 200' (not '427 tools provided') - Remove broken truncation gate that skipped limits >= MAX_TOOLS_LIMIT (128) - Add tests for Grok regex, proactive limits, and limits above threshold Refs #5563 * test(chatcore): cover grok-cli tool-list truncation via prepareUpstreamBody (#5563) Co-authored-by: diegosouzapw <diegosouza.pw@gmail.com> * fix(security): v3.8.15 hardening follow-ups (Seg2/Seg3/Seg4/Bug3) (#5512) Security v3.8.15 hardening follow-ups: Seg2 (CHANGEME boot warn), Seg3 (auth_token cookie maxAge 30d), Seg4 (VS Code path-token once-per-process warning), Bug3 (real global install path resolution), Bug1 (segment-match node_modules in auto-update detection). All 5 carry TDD regression guards. * Fix HuggingChat web session routing (#5592) (#5592) Integrated into release/v3.8.43. HuggingChat web session-routing fix (root parent-message fetch + cookie propagation + encrypted-credential guard) + 24-model catalog refresh. Maintainer adjustments (co-authored): reverted the freeModelCatalog.data.ts whole-file reformat down to the surgical 24-record huggingchat change (preserving the auto-generated compact format), and added a 502 regression test for the null parent-message-id path (Rule #18). * fix: preserve system role for GLM 5.1/5.2 (#5610) (#5663) * fix: restore Codex Responses WS TLS profile + apply proxy (#5591, #5611) (#5668) * fix: allow saving providers without a live validator (#5565, #5567) (#5669) * fix: static model catalog for jules/linkup/ollama/searchapi search providers (#5569, #5571, #5573, #5575) (#5672) * fix: live AI/ML API catalog + deprecate dead CablyAI (#5570, #5568) (#5673) * fix: correct 404 provider setup links for ollama/searchapi/you.com (#5572, #5574, #5576) (#5674) * fix: page call_logs cleanup queries to avoid startup OOM on large DBs (#5618) (#5675) * fix: use PowerShell Expand-Archive on Windows for embedded-service install (#5590) (#5678) * fix: treat array content blocks as valid output in detectMalformedNonStream (#5559) (#5680) * fix: render memory engine status detail strings in English (#5596) (#5685) * fix: free proxy pool silent sync failure — iplocate txt + per-source isolation + surface errors (#5595) (#5686) * chore(quality): close QG v2 tail — drop orphan semcheck.yaml + Fase 9 maturity re-eval (#5681) - Remove semcheck.yaml: orphan config (zero workflow/script wiring) with stale rule counts; deterministic doc-accuracy coverage already exists (check:fabricated-docs --strict + docs-counts-sync + docs-symbols). Drop the REPOSITORY_MAP row referencing it. - Add docs/ops/MATURITY_REEVAL.md (Fase 9): re-measures maturity post-Ondas 0-3. The two biggest structural weaknesses from QUALITY_GATE_PLAYBOOK (2026-06-16) are now closed: fast-gates hole (quality.yml runs typecheck:core + impacted TIA unit tests + vitest + shards) and mutation-score-as-ratchet (check-mutation-ratchet.mjs + seeded baseline + nightly blocking job). Residual gap is owner/infra-gated (branch-protection main, SLSA L3, CodeQL advanced). - Record agent-lsp as deferred/opt-in (doc-only scaffold, no wiring). * fix(ci): stabilize nightly-mutation — guard tap.testFiles drift + anti-flake eps (#5682) Root cause (NOT a timeout): the nightly-mutation run fails on cold-cache nights because the blocking mutation-ratchet job measures modules below baseline, while warm-cache nights pass — the verdict tracked GitHub Actions cache state, not code quality. Proven via a local Stryker probe on headers.ts: covering unit tests (no-memory-header, strip-reasoning) had drifted OUT of stryker.conf.json tap.testFiles, so their mutants went covered-but-unkilled = Survived on a cold full run (COVERED score 61.73 vs 94.29 baseline); adding them restores the kills. - Add scripts/check/check-mutation-test-coverage.mjs: guards that every UNIT test importing a Stryker-mutated module is listed in tap.testFiles. Advisory by default, --strict in CI (wired in quality.yml fast-gates). Prevents recurrence. - Add the 38 drifted covering unit tests to stryker.conf.json tap.testFiles (138 -> 176). Monotonically safe: more covering tests only raise/hold the score. - Add MUTATION_RATCHET_EPS (1.0pt) anti-flake tolerance to check-mutation-ratchet so sub-point tap-runner jitter no longer false-fails the gate. Lowers no baseline. - Tests: check-mutation-test-coverage (3) + eps cases in check-mutation-ratchet. Residual: a clean post-merge nightly confirms scores return to/above baseline; any marginal residual gets a baseline re-seed (operator). * refactor(dashboard): split sidebarVisibility god-file into types + sections leaves (#5683) Behavior-preserving decomposition: src/shared/constants/sidebarVisibility.ts 1197 -> 291 LOC by extracting two leaves under sidebarVisibility/: - types.ts (160): HIDEABLE_SIDEBAR_ITEM_IDS + all sidebar types (self-contained). - sections.ts (762): section building-block consts + SIDEBAR_SECTIONS (imports types only — cycle-safe). COMPRESSION_CONTEXT_GROUP + SIDEBAR_SECTIONS stay exported; host re-exports both + 'export *' of types, so every consumer import path is unchanged. Byte-identical data verified via JSON.stringify of HIDEABLE_SIDEBAR_ITEM_IDS / SIDEBAR_ICON_ACCENTS / COMPRESSION_CONTEXT_GROUP / SIDEBAR_SECTIONS / SIDEBAR_PRESETS + getSectionItems output (identical before/after). typecheck:core, check:cycles (no cycles), check:file-size (3 files <800), and the 3 sidebar suites (20/20) pass. No logic changed. Note: file-size frozen baseline for sidebarVisibility.ts (1198) can ratchet to 291 to lock the shrink (left for the release ratchet / operator). * fix: surface fusion-specific config on the Global Routing tab (#5598) (#5688) * fix(executor): route OpenAI-compatible MCP Responses requests to /responses (#5483) Closes #5483. OpenAI-compatible providers receiving a Responses-shaped request carrying MCP / tool_search tools now route to the upstream /responses endpoint instead of downgrading to /chat/completions, preserving Codex deferred tool discovery. Detection helpers extracted to open-sse/executors/forceResponsesUpstream.ts. Thanks to @KooshaPari. * fix(ci): make release-green pre-flight gates visible + bounded so unit reds are not missed (#5644) Integrated into release/v3.8.43. * fix(body-size): raise LLM API payload limit for responses routes (#5652) Integrated into release/v3.8.43. Thanks @JxnLexn! * fix(test): use lightweight health probe for batch e2e (#5651) Integrated into release/v3.8.43. Thanks @KooshaPari! * feat(compression): T05/C5 — preserveSystemPrompt mode enum + legacy back-compat (#5653) Integrated into release/v3.8.43. Includes the legacy-boolean back-compat derivation so existing preserveSystemPrompt=false installs keep whenNoCache behavior. * routing: optimize latency strategy with perf metrics (#5629) Integrated into release/v3.8.43. Thanks @KooshaPari! * feat(db): models/5004 — self-correcting model context-window overrides (#5667) Integrated into release/v3.8.43. * feat(providers): complete SenseNova free Token Plan — chat + Text-to-Image (port from 9router#2233) (#5679) Integrated into release/v3.8.43. * feat(api): routing/4985 — configurable response-body validation + failover (#5684) Integrated into release/v3.8.43. * fix(chatcore): default Claude tool type to "custom" when missing (#5662) Integrated into release/v3.8.43. Port from 9router#2196. Co-authored-by: warelik <warelik@users.noreply.github.com> * fix(translator): merge consecutive same-role contents for Gemini (port from 9router#2191) (#5661) Integrated into release/v3.8.43. Port from 9router#2191. * chore(bun): add locked bun runtime dependency (#5615) Integrated into release/v3.8.43. Bun 1.3.10 pinned via npm lockfile (adopt-partial decision). Thanks @KooshaPari! * chore(bun): run validated ts scripts with bun (#5612) Integrated into release/v3.8.43. Thanks @KooshaPari! * chore(bun): run CI script checks with bun (#5617) Integrated into release/v3.8.43. Validated bun==node output for all 3 gates (provider-consistency, compression-budget, known-symbols). Thanks @KooshaPari! * fix(build): make pack validator bun safe (#5643) Integrated into release/v3.8.43. Forward-compat guard; node/npm path unchanged. Thanks @KooshaPari! * docs: document Bun as the allow-listed build/dev script runner (Node stays the published runtime) (#5703) Integrated into release/v3.8.43. * feat(analytics): show $0 cost for flat-rate subscription/cookie providers (#5552) (#5704) * refactor(api): extract unified-catalog helpers into cohesive leaf modules (#5699) BLOCO E2 of the god-files campaign. The module-level pure/standalone helpers in src/app/api/v1/models/catalog.ts (1611 LOC) were lifted out verbatim into five cohesive leaf modules so the catalog host shrinks toward the 800-LOC file-size cap without any behavior change (host now 1345 LOC; the heavy getUnifiedModelsResponse orchestrator is untouched — its in-function closures stay put): - catalogHelpers.ts — pure numeric/array/shape helpers + shared catalog types - catalogOpenrouter.ts — OpenRouter id/modality/free-model/display-name helpers - catalogVision.ts — vision-capability field derivation (+ isVisionModelId re-export) - catalogProviderMaps.ts — alias<->providerId resolution maps (buildAliasMaps) - catalogRequest.ts — /v1/models API-key auth gating + Codex CLI client detection The host re-exports getCustomVisionCapabilityFields and isVisionModelId so the public API consumed by other tests (llm-selector-custom-vision-models, vision-detection- consistency) is unchanged; all 9 catalog/vision suites stay green. Adds tests/unit/catalog-helpers-extraction.test.ts: characterization tests for every extracted helper + a guard asserting the host preserves its public exports. Validated: typecheck:core, 50 catalog characterization tests, 12 new leaf tests, integration-wiring, check:cycles, check:file-size (no new violations), ESLint, Prettier. * feat(mcp): T07 — expose RTK learn/discover as MCP tools (#5691) Adds two read-only MCP tools wrapping the existing RTK discovery primitives: omniroute_rtk_discover (discoverRepeatedNoise/suggestFilter over recently captured raw tool output → candidate noise patterns + suggested filter) and omniroute_rtk_learn (listRtkCommandSamples + commandToId). Scope read:compression, MCP audit-logged, no new engine logic. Regression guard: tests/unit/compression/rtk-mcp-tools.test.ts. gaps v3.8.42 — T07. * feat(compression): T05/C3 — opt-in LLM-tier compression engine (#5702) Adds an opt-in, default-off LLM-tier compression engine ('llm') that condenses non-system message prose via a pluggable chat-completion backend, mirroring the llmlingua contract. Safe by construction: no-op default backend (pass-through out of the box), not in the default stacked pipeline, enabled defaults false, fenced code blocks + system messages never sent to the model, fail-open everywhere, minTokens floor. Real production backend is a VPS-validated follow-up (Hard Rule #18). Regression guard: tests/unit/compression/llm-compressor-engine.test.ts (8). gaps v3.8.42 — T05/C3. * refactor(db): extract compat/aliases/mitm helpers from db/models.ts into leaf modules (#5705) BLOCO E3 of the god-files campaign. db/models.ts (1250 LOC) mixed six concerns; the three cleanly-separable ones plus the shared key_value helpers were lifted out verbatim into a new src/lib/db/models/ subdirectory, leaving the tightly-coupled custom/synced/ flags trio in the host (host now 936 LOC). The host re-exports every moved public symbol so the module's public API (consumed by ~29 test files + localDb) is unchanged. - models/shared.ts — asRecord / toNonEmptyString / getKeyValue + JsonRecord (19 LOC) - models/compat.ts — model-compat overrides + sanitizeUpstreamHeadersMap (249 LOC) - models/aliases.ts — model-alias CRUD + cascade delete (61 LOC) - models/mitmAlias.ts — MITM alias get/set (32 LOC) The custom/synced/flags trio stays in the host because it is genuinely coupled (flags->getCustomModelRow, flags->readCompatList, custom->removeModelCompatOverride, synced->getModelIsDeleted, setModelIsHidden->updateCustomModel) — splitting it cleanly is a follow-up. Dependency DAG is acyclic (verified by check:cycles). Adds tests/unit/db-models-split.test.ts: characterization of the pure extracted helpers + a guard asserting the host preserves its full public export surface. Validated: typecheck:core, check:cycles (no cycles), 77 existing db/models consumer tests (db-models-crud/extended/aliases-cascade + 7 more) green, 7 new tests, ESLint, Prettier, check:file-size (host 936 < frozen 1259; no new violations). * refactor(db): extract pricing/lkgp/cache-metrics from db/settings.ts into leaf modules (#5709) BLOCO E3 of the god-files campaign. db/settings.ts (1154 LOC) mixed five concerns; the three cleanly-separable ones plus the shared toRecord/JsonRecord helper were lifted out verbatim into a new src/lib/db/settings/ subdirectory, leaving the Settings-core + Proxy config concerns in the host (host now 646 LOC). The host re-exports every moved public symbol so the module's public API (consumed by ~93 test files + localDb) is unchanged. - settings/shared.ts — toRecord + JsonRecord (9 LOC) - settings/pricing.ts — pricing layers/sources/per-model + update/reset (254 LOC) - settings/lkgp.ts — Last-Known-Good-Provider get/set/clear (49 LOC) - settings/cacheMetrics.ts — cache metrics + trend (235 LOC) Settings-core + the Proxy-config concern stay in the host: proxy is the most tangled (245-line resolveProxyForConnection, resolution cache, imports from ./proxies) and getSettings is the most central function — leaving them is the correct coupled-core stop. Pricing/LKGP/Cache have NO dependency on Settings/Proxy helpers (verified); the dependency DAG is acyclic (check:cycles). Adds tests/unit/db-settings-split.test.ts: characterization of the shared toRecord helper + a guard asserting the host preserves its full public export surface. Validated: typecheck:core, check:cycles (no cycles), 149 existing+new db/settings consumer tests green (db-settings-crud/extended, 8 pricing suites, cache-metrics, 2 proxy-resolution suites + 29 new), ESLint, Prettier, check:file-size (host 646 < frozen 1155). * fix(translator): re-apply lost defensive hardening for Gemini merge + Claude tool defaults (#5706) Re-applies two dropped gemini-code-assist hardening fixes (defaultClaudeToolType non-object passthrough; mergeConsecutiveSameRoleContents shallow-copy) with regression tests. Follow-up to #5661/#5662. Integrated into release/v3.8.43. * feat(codex): generate fallback profiles for compatible models (#5701) setup-codex now generates Codex profiles for compatible text models from the live /v1/models catalog when the model id doesn't match a hand-tuned pattern, skipping media/embedding models. Integrated into release/v3.8.43. * docs(changelog): credit @Chewji9875 for #5563 + #5579 Add CHANGELOG credit bullets for grok-cli tool-limit (#5563) and Antigravity 429 lockout (#5579). Documentation-only. * test(dashboard): repoint sidebar quota-share placement scan to sections.ts (#5711) The D1 god-file split (#5683) moved the nav-item id definitions out of src/shared/constants/sidebarVisibility.ts into the extracted leaf src/shared/constants/sidebarVisibility/sections.ts. This source-scan test still read the old monolith path, so it found 0 occurrences of id: "costs-quota-share" and failed (base-red on release/v3.8.43). Repoint SIDEBAR_PATH to sections.ts where the ids now live. All four placement assertions (quota-share after quota, same array, far from costs-budget, exactly one occurrence) hold against the new source. * refactor(db): extract columns/nodes/rate-limit leaves from db/providers.ts (#5714) db/providers.ts was a 1106-line god-file mixing four concerns. Extract the three acyclic, cohesive slices into sibling leaf modules under src/lib/db/providers/, leaving the tightly-coupled connection-CRUD core in the host: - providers/columns.ts (116) 10 pure column-normalizer helpers (DB-free) - providers/nodes.ts (163) 6 provider-node CRUD functions - providers/rateLimit.ts (177) 6 rate-limit/quota runtime helpers + formatResetCountdown Host providers.ts: 1106 -> 719 lines. The connection-CRUD core does not call any node or rate-limit function (verified), so the host re-exports the 12 moved public symbols via `export { ... } from './providers/<leaf>'` — the module's public API stays IDENTICAL (23 symbols). Bodies moved verbatim (byte-identical); the only edit to a moved line is the added `export` on the 10 previously-private normalizers. Behavior-preserving: 122 existing provider/quota/rate-limit consumer tests stay green; new tests/unit/db-providers-split.test.ts guards the re-export barrel + characterizes the pure column helpers (38 assertions). Refs #3501 (god-file structural shrink). * refactor(db): extract types + pure mappers from db/proxies.ts (#5717) db/proxies.ts was a 1059-line god-file. Extract the two acyclic, DB-free slices into sibling leaf modules under src/lib/db/proxies/, leaving the tightly-coupled CRUD + assignment + resolution core in the host: - proxies/types.ts (65) 10 proxy type/interface declarations - proxies/mappers.ts (180) pure row mappers / scope normalizers / payload coercers (toRecord, mapProxyRow, mapAssignmentRow, isRelayProxyType, extractRelayAuth, toRegistryProxyResolution, normalizeScope, normalizeAssignmentScopeId, toLegacyProxyLevel, coerceProxyPayload, redactProxySecrets) Host proxies.ts: 1059 -> 847 lines. The resolution functions call createProxy/assignProxyToScope, so the CRUD+resolution core CANNOT be extracted without an import cycle and stays in the host. The host re-exports the 2 moved public functions (extractRelayAuth, redactProxySecrets) via `export { ... } from './proxies/mappers'` — the public API stays IDENTICAL (20 functions; no types were ever publicly exported). Bodies moved verbatim; the only host edits are the new leaf imports, the re-export, dropping the now unused `import { decrypt }`, and two prettier line-wrap reflows of retained ternary/union lines (token-identical). Behavior-preserving: 69 existing proxy/registry/relay/family consumer tests stay green; new tests/unit/db-proxies-split.test.ts guards the re-export barrel + characterizes the pure mappers (35 assertions). Refs #3501. * refactor(db): extract static migration data tables from migrationRunner.ts (#5721) migrationRunner.ts (1124 lines, frozen-baselined) is the startup migration orchestrator. As a conservative, zero-behaviour-risk first slice, extract the six static migration-compatibility DATA tables (verbatim) into a pure-data leaf, leaving the entire orchestrator + all SQL-running helpers in the host: - migrationRunner/constants.ts (118) RENAMED_MIGRATION_COMPATIBILITY, LEGACY_VERSION_SLOT_MIGRATIONS, SUPERSEDED_DUPLICATE_MIGRATIONS, PHYSICAL_SCHEMA_SENTINELS, INITIAL_SCHEMA_SENTINELS, OPTIONAL_FTS5_MIGRATION_VERSIONS Host migrationRunner.ts: 1124 -> 1023. The runtime fts5SupportCache (a WeakMap, mutable state) stays in the host. No public API change (these consts were module-internal). Data moved byte-identical (sed-extracted, verbatim verified); the only host edits are the leaf import + one prettier collapse of a pre-existing 2-line union type annotation to 1 line (token-identical, typecheck-confirmed). Characterize-first (operator-chosen): the existing db-migration-runner.test.ts (26 tests) + no-migration-collisions/weak-rng-fixes/check-db-rules (11) prove the reconciliation/dedup/already-applied BEHAVIOUR is unchanged; the new tests/unit/db-migrationrunner-constants-split.test.ts (7 tests) PINS THE DATA (counts + shape + spot-checks of every table) so a dropped/transposed row is caught immediately. Refs #3501. * refactor(db): extract pure SQL-source builders from usageAnalytics.ts (#5722) usageAnalytics.ts (924 lines, frozen-baselined) mixes two pure SQL-source builders with ~20 getXxxRows() query functions. Extract the contiguous, DB-free builder block verbatim into a leaf, leaving every query function in the host: - usageAnalytics/sources.ts (208) AnalyticsParams, BuildUnifiedSourceOptions, UnifiedSourceResult + buildUnifiedSource + buildPresetUnifiedSource (pure string builders; no DB, no imports) Host usageAnalytics.ts: 924 -> 723. The query functions do not call the builders (callers build the unified source then pass the string in), so the host re-exports the 5 moved public symbols (2 fns + 3 types) and imports AnalyticsParams as a type for its query signatures — the public API stays IDENTICAL (39 symbols). Builder bodies moved byte-identical; the two orphaned section-header banners that described the moved block were removed with it; the retained query-function suffix is byte-identical to the original. Behavior-preserving: 37 existing analytics consumer tests stay green (usage-analytics 12, usage-endpoint-dimension 3, db-usage-analytics-3500 22); new tests/unit/db-usageanalytics-split.test.ts (25 assertions) characterizes buildUnifiedSource's needsAggregated branching (raw-only vs raw+daily_usage_summary) + guards the 39-symbol re-export barrel. Refs #3501. * docs(readme): refresh metrics, list 17 strategies, add Quota-Share + real provider logos - Unify provider count to 236; MCP tools 87->94; cloud agents 3->4 (+Cursor); compression 9->10 engines (+relevance) - Tests -> 21,000+ across 2,586 files; footer -> v3.8.43 - Raise lower bounds to real values: 90+ free, 80+ commands, 24+ CLIs - Language flag grid 33->43 (15/14/14, all locales) - List all 17 routing strategies; new Quota-Share section before Resilience - Real provider logos (lobe-icons + local agentrouter) in providers grid and Free Forever - Top Contributors: refreshed stats + add herjarsa; 280+ title; half-size avatars; contrib.rocks 100->200 - Acknowledgments: refreshed star counts; fix headroom repo rename * docs(readme): update provider counts and add new badges * feat(memory): T10/TV6 — opt-in typed memory decay (#5723) Opt-in typed memory decay so the conversational memory store self-prunes stale episodic noise. access_count + last_accessed_at telemetry (migration 111) is always-on/non-destructive; the sweep is opt-in (MEMORY_TYPED_DECAY_ENABLED, default false). Only episodic decays by default (30d); factual/procedural/semantic immune; access_count>=3 earns immunity; deletions reuse deleteMemory (SQLite+vec+Qdrant in sync), fail-open. Regression guard: tests/unit/memory/typed-decay.test.ts (15). gaps v3.8.42 — T10/TV6. * feat(dashboard): T06/T03 — drag-reorder compression pipeline editor + studio e2e (#5727) T06: named-combos editor gains a @dnd-kit/sortable drag-to-reorder stacked pipeline backed by a pure model (compressionPipelineModel.ts: add/remove/move/update, engine->intensity invariant, never-empty). CompressionPipelineEditor.tsx replaces the inline fixed list in CompressionCombosPageClient; order persists via the existing combos endpoint (no API change). T03: adds tests/e2e/compression-studio.spec.ts (Tela A render + Play/Compare tab switch), the dedicated compression-studio e2e combo-live-studio.spec.ts did not cover. TDD: compression-pipeline-model.test.ts (11) + compression-pipeline-editor.test.tsx (4). gaps v3.8.42 — T06 + T03. * fix(thinking): wire Thinking-Budget boot hydration into live instrumentation path (#5312) (#5729) hydrateThinkingBudgetConfig was only called from the unused src/server-init.ts, which never runs in production, so the dashboard Thinking-Budget mode silently reverted to passthrough on every restart. Wire it into the real boot path (src/instrumentation-node.ts), next to the Global System Prompt restore. Surfaced by live Anthropic-OAuth validation on the VPS (fix A of #5312 was non-functional even though its direct unit test passed). New guard tests/unit/thinking-budget-boot-wiring-5312.test.ts asserts the production boot module calls the hydration, closing the test gap that let this ship. * refactor(usage): extract pure formatting helpers from callLogs.ts (#5725) callLogs.ts (996 lines, frozen-baselined) mixes pure log-formatting / sanitization helpers with DB CRUD, disk-artifact, and rotation logic. Extract the ten pure, DB-free helpers verbatim into a leaf, leaving all stateful code in the host: - callLogs/format.ts (129) asRecord, toNumber, toStringOrNull, truncateText, parseInlineError, normalizeDetailState, sanitizeErrorForLog, toStoredErrorSummary, protectPipelinePayloads, buildRequestSummary Host callLogs.ts: 996 -> 885. The stateful generateLogId (mutates logIdCounter) stays in the host. These helpers were all module-internal, so the public API is unchanged (10 exported functions). Bodies moved byte-identical; the host's now unused 'sanitizePII' import (only referenced inside the moved bodies) moved to the leaf; prettier wrapped buildRequestSummary's signature across lines once the 'export' prefix pushed it past 100 cols (token-identical). Behavior-preserving: 46 existing call-log consumer tests stay green (call-log-cap 14, pagination 4, file-rotation 5, log-retention 5, startup 1, oom 2, trim-sql 2, db-settings-maintenance 13); new tests/unit/calllogs-format-split.test.ts (26 assertions) characterizes the pure helpers + guards the 10-function public API. Refs #3501. * refactor(usage): extract pure stat/coercer helpers from usageHistory.ts (#5728) usageHistory.ts (987 lines, frozen-baselined) mixes pure DB-free helpers with an in-memory pending-request state machine and DB CRUD. Extract the contiguous pure block verbatim into a leaf, leaving all stateful code in the host: - usageHistory/helpers.ts (85) asRecord, toStringOrNull, normalizeServiceTier, toNumber, percentile, stdDev, truncatePendingPreview (+ its MAX_PREVIEW_* bounds, co-located) Host usageHistory.ts: 987 -> 916. The pending-request state machine (module Maps + track/update/finalize/sweep) and DB CRUD stay in the host. These helpers were all module-internal, so the public API is unchanged (21 direct exports + the pre-existing getCompletedDetails re-export = 22). Bodies moved byte-identical (leaf 0 non-verbatim lines); the host's local 'type JsonRecord' moved with the bodies that used it (host no longer references it — typecheck-confirmed). Behavior-preserving: 38 existing usage-history consumer tests stay green (usage-history-db 5, api-key-usage-limits 6, log-retention 5, usage-endpoint-dimension 3, provider-request-failure-pipeline 6, database-settings-maintenance 13); new tests/unit/usagehistory-helpers-split.test.ts (30 assertions) pins the percentile/stdDev formulas + normalizeServiceTier + guards the public API. Refs #3501. * refactor(usage): extract pure quota-normalize helpers from providerLimits.ts (#5730) providerLimits.ts (954 lines, frozen-baselined) is the heavily DB/network-coupled provider quota sync module. Extract a small, fully SELF-CONTAINED leaf of pure quota-key/quota-value normalization helpers (+ the isRecord type guard they share), leaving all sync/DB/network code in the host: - providerLimits/quotaNormalize.ts (72) isRecord, isUsageQuotaKeyAllowed, normalizeUsageQuotaKey, normalizeUsageQuotasForProvider, sanitizeUsageQuotasForProvider Host providerLimits.ts: 954 -> 890. The leaf imports only the external antigravity/agy model-alias helpers the moved bodies reference (moved from the host's import block) — it does NOT import the host, so check:cycles stays clean (no cycle). isRecord (used ~9x in the host) is co-extracted and imported back. These five were all module-internal, so the public API is unchanged (13 exported functions). Bodies moved byte-identical. Behavior-preserving: 18 existing provider-limits consumer tests stay green (sanitize-scope 3, db-provider-limits 3, proxy-fail-closed 3, rotating-expired-guard 7, codex-quota-sync 2); new tests/unit/providerlimits-quotanormalize-split.test.ts (19 assertions) pins isRecord + isUsageQuotaKeyAllowed + guards the 13-function public API. Refs #3501. * refactor(memory): extract pure scoring/conversion helpers from retrieval.ts (#5733) retrieval.ts (1192 lines — ABOVE its 1171 frozen baseline) is the memory retrieval engine (DB + vector + rerank network). Extract the pure, DB-free scoring/conversion helpers (+ the MemoryRow row shape they share) verbatim into a self-contained leaf, leaving all DB/vector/network code in the host: - retrieval/scoring.ts (104) interface MemoryRow + estimateTokens, parseMetadata, rowToMemory, getRelevanceScore Host retrieval.ts: 1192 -> 1072 — back UNDER the 1171 frozen baseline (the split also repairs the pre-existing file-size drift). The leaf imports only ../types, never the host, so check:cycles stays clean (no cycle). MemoryRow moved to the leaf and imported back as a type by the host's DB row functions. The public estimateTokens is re-exported from the leaf; the host also imports it for its internal token-budget loops. The other three helpers were module-internal, so the public API is unchanged (7 exports). Bodies moved byte-identical. Behavior-preserving: 38 existing memory-retrieval consumer tests stay green (rerank 5, hybrid 6, semantic 6, engine-status 9, stats-api 12); new tests/unit/retrieval-scoring-split.test.ts (11 assertions) pins estimateTokens (ceil(len/4)) + parseMetadata + rowToMemory mapping + getRelevanceScore (+20 phrase / +3 token) and guards the public API. Refs #3501. * refactor(sse): extract reasoning-tag detection/extraction from responseSanitizer.ts (#5734) responseSanitizer.ts (1133 lines, frozen-baselined) mixes reasoning-tag detection/extraction with response/usage/streaming sanitization. Extract the cohesive, ZERO-IMPORT reasoning block verbatim into a self-contained leaf: - responseSanitizer/reasoning.ts (143) the reasoning regex consts + collapseExcessiveNewlines, cleanReasoningFragment, splitClosingOnlyReasoningPrefix, movePrefixBeforeContentTagToThinking, extractThinkingFromContent, normalizeReasoningRouteId, isAntigravityReasoningRoute, isTextualReasoningTagNativeRoute, shouldParseTextualReasoningTags Host responseSanitizer.ts: 1133 -> 1003. The block's helpers only call each other, so the leaf has ZERO imports — it cannot import the host (check:cycles clean). The host imports back collapseExcessiveNewlines (6 call sites) + extractThinkingFromContent, and re-exports the two public symbols (extractThinkingFromContent, shouldParseTextualReasoningTags) — the public API stays IDENTICAL (7 exports). Bodies moved byte-identical; two long declarations (REASONING_TAG_FRAGMENT_REGEX, movePrefixBeforeContentTagToThinking signature) were line-wrapped by prettier once the 'export' prefix pushed them past 100 cols (token-identical). Behavior-preserving: 47 existing consumer tests stay green (response-sanitizer 36, strip-reasoning-header 8, textual-toolcall-false-positive 3); new tests/unit/responsesanitizer-reasoning-split.test.ts (11 assertions) characterizes extractThinkingFromContent + shouldParseTextualReasoningTags and guards the public API. Refs #3501. * refactor(sse): extract rate-limit header parsing from rateLimitManager.ts (#5736) rateLimitManager.ts (1034 lines, frozen-baselined) is the stateful rate-limiter (Bottleneck limiters, watchdog timers, learned-limits Map). Extract the pure, ZERO-IMPORT header-parsing block verbatim into a self-contained leaf, leaving all stateful machinery in the host: - rateLimitManager/headers.ts (94) STANDARD_HEADERS, ANTHROPIC_HEADERS, parseResetTime, toPlainHeaders Host rateLimitManager.ts: 1034 -> 945. The four items are pure (no limiter state, no external deps), so the leaf has ZERO imports — it cannot import the host (check:cycles clean). The host imports all four back (used by updateFromHeaders). They were module-internal, so the public API is unchanged (17 exports). Bodies moved byte-identical. Behavior-preserving: 21 existing rate-limit consumer tests stay green (rate-limit-manager 7, limiter-lifecycle 4, queue-timeout-msg 2, idle-eviction 6, body-lock 2); new tests/unit/ratelimitmanager-headers-split.test.ts (7 assertions) pins parseResetTime (durations / bare-number / nullish) + toPlainHeaders + guards the 17-function public API (with a watchdog-timer teardown hook so the runner exits cleanly). Refs #3501. * fix(config): back boot-hydrated proxy config singletons with globalThis (#5312) (#5742) Next.js compiles instrumentation.ts as a separate webpack module graph from the app-route/open-sse executors, so a module-local `let _config` is duplicated: the boot-time hydration (applyRuntimeSettings / restore hooks) lands on the instrumentation graph's copy, but the request path (base.ts) reads a different, un-hydrated copy. Live VPS validation proved the Thinking-Budget hydrate ran to completion at boot yet base.ts still read the passthrough default — why #5312 fix A stayed broken after the boot-wiring fix. Back the singletons with globalThis (the pattern systemPrompt.ts already uses for #2470) so all graph copies share one instance: - thinkingBudget.ts — dashboard Thinking-Budget mode reaches the executor - backgroundTaskDetector.ts — opt-in background degradation actually fires - systemTransforms.ts — operator pipeline overrides reach the request path payloadRules.ts was already safe (lazy per-request DB self-load, #2986). Guards: thinking-budget-globalthis-5312 + runtime-config-globalthis-5312 (assert globalThis sharing; a module-local let fails them, RED->GREEN). * refactor(evals): extract built-in golden-set suites from evalRunner.ts (#5740) Move the 7 static built-in eval suites (golden-set, coding-proficiency, reasoning-logic, multilingual, safety-guardrails, instruction-following, codex-comparison) plus the builtInSuites aggregate into the pure-data leaf src/lib/evals/evalRunner/builtinSuites.ts (zero imports, no side effects). evalRunner.ts keeps all logic (register/get/list/evaluate/run/scorecard/reset) and registers the leaf suites at module load, mirroring the original inline calls. Public API is unchanged (7 exported functions; the suite consts were already module-private). Host 960->301 LOC; leaf 676 LOC (< 800 cap); host was frozen-satisfied (961), so this is debt reduction. Suite data moved verbatim (652 data lines byte-identical). New split-guard test characterizes the suite ids/case counts/key cases and proves the host registers every leaf suite at load. * refactor(models): extract pure transform layer from modelsDevSync.ts (#5743) Move the models.dev data-model types, the provider-id mapping table (MODELS_DEV_PROVIDER_MAP + mapProviderId), and the raw->OmniRoute transforms (transformModelsDevToPricing, transformModelsDevToCapabilities) into the pure leaf src/lib/modelsDevSync/transform.ts (zero imports, no DB, no module state). modelsDevSync.ts keeps all sync orchestration, DB access, caches and the periodic-sync timer; it imports the transforms for internal use and re-exports mapProviderId/transformModelsDevToPricing/transformModelsDevToCapabilities plus the ModelCapabilityEntry/CapabilitiesByProvider types, so the public API is unchanged. Host 924->677 LOC; leaf 279 LOC (< 800 cap); host was frozen-satisfied (934), so this is debt reduction. 238 moved lines are byte-identical. New split-guard test characterizes the provider map + both transforms and proves the host re-exports them. * refactor(resilience): split settings.ts into types + normalize leaves (#5745) Decompose the (fully pure) resilience settings module into two sibling leaves: - src/lib/resilience/settings/types.ts: the settings shape (11 public interfaces + JsonRecord/AuthCategory), zero imports. - src/lib/resilience/settings/normalize.ts: the coercers (asRecord/toInteger/ toBoolean/feature-flag resolvers) + the 11 per-section normalize* functions. settings.ts keeps DEFAULT_RESILIENCE_SETTINGS, DEFAULT_REQUEST_QUEUE_MAX_WAIT_MS, buildLegacyFallback, and the public orchestrators (resolveResilienceSettings, mergeResilienceSettings, buildLegacyResilienceCompat); it imports the coercers/normalizers for internal use and re-exports the 11 settings interfaces, so the public API is unchanged. Host 840->363 LOC; leaves 182 + 359 LOC (< 800 cap); host was frozen-satisfied (841), so this is debt reduction. 472 moved lines are byte-identical; no cycles (leaves never import the host). New split-guard test characterizes the coercers/normalizers and the host resolve/merge/compat orchestration. * docs(readme): document faster/leaner install — skip native build, sql.js fallback (#5713) Documents the optional better-sqlite3 + pure-JS fallback chain and OMNIROUTE_SKIP_POSTINSTALL/CI skip flags. Docs-only, claims verified. (#5550) * feat(compression): T02 opt-in per-engine pipeline circuit-breaker (#5735) Opt-in, default-off per-engine circuit-breaker for the stacked compression pipeline. Byte-identical to legacy when off. 9 regression tests. * docs: sync MCP tool count to 95 + routing-strategy count (#5732) Sync CLAUDE.md/README.md to canonical MCP tool count (95, 35 base) and routing strategies (17). Numbers fact-checked against getAllToolDefinitions()/ROUTING_STRATEGY_VALUES. * feat(api): add first-class Ollama local provider card (#5712) First-class ollama-local provider card (localhost:11434/v1, keyless, passthrough models) in LOCAL_PROVIDERS + SELF_HOSTED + default.ts executor case. Docs count 236→237, Local 11→12 (full README sweep). 4 tests. (#5578) * feat(api): add opt-in API-key provider quota-policy bypass scope (#5731) Adds an opt-in per-API-key scope (policy:bypass-provider-quota) that lets a key skip provider/account-side quota cutoffs during routing. Operator USD budgets/usage limits still enforced unconditionally (fail-closed, before the bypass). Default-off; UI toggle + badge in API Manager. Integrated into release/v3.8.43. * feat(codex): opt-in auto-sync of Codex profiles after model discovery (#5737) Auto-sync ~/.codex/*.config.toml profiles after a provider model sync, reusing the setup-codex generator. Opt-in, default OFF (OMNIROUTE_AUTO_SYNC_CODEX_PROFILES=true; also honors CLI_ALLOW_CONFIG_WRITES). Never touches the active Codex config. Gating test added. Co-authored-by: diegosouzapw <diegosouza.pw@gmail.com> * feat(providers): opt-in CLI profile auto-sync toggles + Claude Code auto-sync (#5755) Providers-dashboard 'CLI profile auto-sync' card (Codex + Claude Code toggles), feature-flag backed (default off), + Claude Code auto-sync mirroring the Codex path. Follow-up to #5737. * feat(compression): T08/H8 (2.3) — graduated CCR retrieval-feedback ramp (#5739) Turns CCR retrieval feedback from a binary cliff into a graduated ramp: each prior retrieval raises a block's effective minChars linearly (effectiveMinChars); >= 3 retrievals still excluded (Infinity). retrievalRampFactor default 2 (config/env COMPRESSION_CCR_RETRIEVAL_RAMP_FACTOR); 1 = legacy binary. Regression guard: tests/unit/compression/ccr-retrieval-ramp.test.ts (12); 51 existing CCR tests green. gaps v3.8.42 — T08/H8 (2.3). * feat(compression): T08/H5 (2.4) — usage-observed prefix freeze (opt-in) (#5744) Evolves the cache-aware guard to also learn which system prompts recur: observed >= threshold → treated as a stable cacheable prefix and preserved even for providers the static check misses. Content-addressed by a hash of the system prompt (OpenAI/Claude/Gemini), in-memory, freeze=preserve (never mutates). Opt-in/default-off (COMPRESSION_PREFIX_FREEZE_ENABLED); respects the never preserve-mode. New prefixFreeze.ts wired into resolveCacheAwareConfig. Regression guard: prefix-freeze.test.ts (10); 44 cache-aware tests green. gaps v3.8.42 — T08/H5 (2.4). * feat(compression): T08/H7 (2.5) — read-lifecycle engine (collapse superseded reads) (#5754) New opt-in, default-off read-lifecycle engine: collapses stale/superseded file-Read tool results (same path re-read OR modified later) to a stub, keeping the current Read intact. Anthropic + OpenAI tool shapes; conservative (known tool names, exact path, strictly-later); fail-open. Lossy → opt-in. Regression guard: read-lifecycle.test.ts (10); 41 registry/pipeline suites green. gaps v3.8.42 — T08/H7 (2.5). Completes Onda 2. * fix(sse): anti-thundering-herd guard tolerates numeric-epoch cooldowns (#5747) markAccountUnavailable's dedupe guard used a raw `new Date()` on rateLimitedUntil, which can hold a numeric-epoch string (e.g. the Antigravity full-quota path via setConnectionRateLimitUntil). That produced Invalid Date/NaN, so the guard never detected an already cooling connection — a second concurrent failure on the same connection overwrote a long quota-exhaustion cooldown with a much shorter fresh backoff cooldown, making the account selectable again far sooner than intended. Reuses the existing cooldownUntilMs normalizer (#3954) instead of a raw Date parse. * fix(chat): harden non-streaming SSE aggregation (#5746) * fix: repoint DashScope/Alibaba setup links to consoles (#5665) (#5762) * fix: point Quick Start step 1 to API Keys page, not Endpoint (#5695) (#5763) * fix: onboarding wizard saves providers with unsupported validation (#5692) (#5764) * docs(security): document full LOCAL_ONLY route set + GHSA-fhh6-4qxv-rpqj + audit path (#5599) (#5748) Expand ROUTE_GUARD_TIERS.md Tier 1 (LOCAL_ONLY): - link the GHSA advisory and explain the attack class (RCE via a subprocess spawn reachable from non-loopback traffic) - replace the 3-example prefix table with the full LOCAL_ONLY set, mirroring LOCAL_ONLY_API_PREFIXES / LOCAL_ONLY_API_PATTERNS in routeGuard.ts (the authoritative source; check-route-guard-membership enforces the code side) - add an "Operator guidance & auditing" section for users behind nginx/Cloudflare/Tailscale: don't forge X-Forwarded-For loopback, keep the manage-scope bypass minimal, and how to audit non-loopback access Docs-only; SECURITY.md already links here. Closes #5599 * docs(security): document banned-keyword / account-ban detection (#5600) (#5756) * docs(security): add BAN_DETECTION.md — banned-keyword / account-ban detection (#5600) New docs/security/BAN_DETECTION.md documenting the previously-undocumented system: - the 8 built-in ACCOUNT_DEACTIVATED_SIGNALS + custom keywords are additive - detection flow (body substring match -> terminal `banned` state, skipped in account selection; `deactivated` on 401/403; autoDisableBannedAccounts) - scope: global (all providers); the signal strings target OAuth/subscription scrapers - custom keywords: add path, 200-char cap, hot-reload, and the false-positive warning (raw substring match -> prefer full ban sentences, not "quota"/"limit") - recovery: terminal states never auto-recover -> re-test / re-auth / re-enable Registered in security meta.json; cross-linked from RESILIENCE_GUIDE (terminal states). Docs-only. Closes #5600 * docs(security): clarify deactivated vs expired terminal-status split (#5600) The same ACCOUNT_DEACTIVATED signal surfaces as two different terminal statuses depending on the code path: chatCore.ts inline writes 'deactivated' (401/403 via classifyProviderError), while markAccountUnavailable() -> resolveTerminalConnectionStatus() writes 'expired'. Document both. * fix: surface relay proxy-test errors instead of silent failure (#5716) (#5765) * refactor(api): extract pure discovery leaves from provider-models route (#5758) Split src/app/api/providers/[id]/models/route.ts (2511 -> 1818 LOC) by moving the cohesive, DB-free discovery building blocks into four leaves under discovery/: - helpers.ts record/string coercion, Azure + base-url helpers, bearer/named-openai header builders - normalizers.ts Antigravity / DataRobot / OpenAI-like / SAP models response normalizers - providerModelsConfig.ts PROVIDER_MODELS_CONFIG + ProviderModelsConfigEntry - providerSets.ts NAMED_OPENAI_STYLE_PROVIDERS + isNamedOpenAIStyleProvider The host keeps all request orchestration and imports the leaves back. The moved symbols were module-private, so the route's public export set (GET) is unchanged and no external importer needs updating. Bodies are byte-identical: the code-line multiset of host + leaves equals the original route verbatim. Tests: - repoint the qwen-web source-guard in catalog-updates-v3829-kimi-qwen to the new config leaf (assertions unchanged) - add provider-models-discovery-split as the split regression guard (leaf public surface + host wiring + the #5570 cablyai->aimlapi entry swap) * fix(memory): enabling Qdrant activates it as the engine + inline guidance (#5597) (#5741) * fix(memory): enabling Qdrant now activates it as the engine + inline guidance (#5597) Enabling Qdrant in the Engine tab was inert: retrieval only routes to Qdrant when memoryVectorStore === "qdrant" (the default "auto" never selects it), and the card only wrote qdrantEnabled — nothing set the engine selector, and there is no UI for it. So users configured Qdrant, saw "enabled", but it was never actually used. - PUT /api/settings/qdrant now sets memoryVectorStore alongside the toggle: enable -> "qdrant", disable -> "auto". Editing other fields leaves it untouched. - Add inline guidance to QdrantConfigCard: a Tier-1-vs-Tier-2 banner + per-field help (host, collection, embedding model). Note there is no "vector dimension" or "distance metric" field: dimension is auto-detected from the embedder, distance is always Cosine. - Document the real behavior in MEMORY.md: engine gate, no back-fill of existing memories, dimension auto-detect, Cosine-only, API-key-only auth. Tests: tests/integration/qdrant-routes.test.ts — enable->qdrant, disable->auto, and field-edit-without-enabled leaves the engine untouched (TDD: red -> green). Closes #5597 * fix(memory): invalidate memory-settings cache on Qdrant toggle (#5597) The PUT handler wrote memoryVectorStore to the DB but retrieval reads through getMemorySettings(), a module-level cache. Without busting it, the engine switch did not take effect until a process restart (the DB said qdrant, retrieval kept routing to sqlite-vec). Now calls invalidateMemorySettingsCache() after the write, mirroring src/app/api/settings/memory/route.ts. Regression test warms the cache, toggles via the route, and asserts getMemorySettings().vectorStore flips to qdrant (fails without the invalidate call). * fix(compression): record Context Editing telemetry on the streaming path (#5761) Streaming SSE responses now preserve context_management from the final message_delta snapshot and fire the telemetry hook in onStreamComplete, so context-clear savings surface in compression analytics for streaming (not just non-streaming). Additive telemetry, Claude-only, opt-in-neutral. gaps v3.8.42 — T01 (5.1). Test: context-editing-streaming-telemetry.test.ts (3, failing->passing). * Persist batch item checkpoints during recovery (#5753) * fix(sse): checkpoint batch item recovery * fix(db): renumber batch checkpoints migration 110→112 (collision with #5667) 110 was taken by 110_model_context_overrides.sql (#5667), which landed on the release branch after this PR branched. migrationRunner throws a hard version- collision error on startup when two files share a numeric prefix. 112 is the next free slot (110/111 taken on the release tip). Co-authored-by: diegosouzapw <diegosouza.pw@gmail.com> --------- Co-authored-by: diegosouzapw <diegosouza.pw@gmail.com> * fix: resolve CCR MCP retrieve principal from api-key auth context (#5649) (#5768) * feat(cli): show version in startup banner (integrates #5752) (#5769) * feat(cli): show version in startup banner Print dim 'v<version>' line below ASCII art logo in omniroute serve. Uses readFileSync (same pattern as program.mjs) to read package.json. Closes #5749. * test(cli): guard startup-banner version line (#5752) Source-inspection test (same pattern as cli-serve-port.test.ts) asserting serve.mjs parses the version from package.json and prints v${_pkg.version} in the startup banner — satisfies Hard Rule #8 for the bin/ change. Co-authored-by: diegosouzapw <diegosouza.pw@gmail.com> * docs(changelog): credit #5752 startup-banner version line (thanks @chirag127) --------- Co-authored-by: Chirag Singhal <76880977+chirag127@users.noreply.github.com> * fix(proxyfetch): skip fallback for non-replayable bodies (#5770) * chore(release): open v3.8.42 cycle Bump version to 3.8.42, add CHANGELOG placeholder, sync openapi/electron/open-sse + 42 i18n CHANGELOG mirrors. * chore: remove unused qdrant schema aliases (#5404) Integrated into release/v3.8.42 * chore: remove unused memory schema aliases (#5403) Integrated into release/v3.8.42 * chore: remove unused quota schema types (#5402) Integrated into release/v3.8.42 * chore: remove unused playground row type (#5401) Integrated into release/v3.8.42 * chore: remove unused codegraph exports (#5400) Integrated into release/v3.8.42 * chore: remove unused notion client type (#5399) Integrated into release/v3.8.42 * chore: remove unused settings types (#5398) Integrated into release/v3.8.42 * chore: remove unused combo types (#5396) Integrated into release/v3.8.42 * chore: remove unused provider types (#5393) Integrated into release/v3.8.42 * chore: remove unused skillssh skill type (#5392) Integrated into release/v3.8.42 * chore: remove unused status hex key type (#5391) Integrated into release/v3.8.42 * chore: remove unused batch provider type (#5390) Integrated into release/v3.8.42 * chore: remove unused skills schema types (#5389) Integrated into release/v3.8.42 * chore: remove unused codex auth input type (#5388) Integrated into release/v3.8.42 * chore: remove unused memory schema types (#5387) Integrated into release/v3.8.42 * chore: remove unused playground row type (#5386) Integrated into release/v3.8.42 * chore: remove unused qdrant schema types (#5385) Integrated into release/v3.8.42 * chore: remove unused kiro social schema (#5384) Integrated into release/v3.8.42 * chore: remove unused memory schema types (#5383) Integrated into release/v3.8.42 * chore: remove unused audit action type (#5382) Integrated into release/v3.8.42 * chore: remove unused agent skills schema types (#5381) Integrated into release/v3.8.42 * chore: remove unused shared logger default export (#5380) Integrated into release/v3.8.42 * chore: remove unused sse logger helpers (#5378) Integrated into release/v3.8.42 * chore: remove unused sse model legacy helpers (#5377) Integrated into release/v3.8.42 * chore: remove unused v1 search response schema (#5376) Integrated into release/v3.8.42 * chore: remove unused cloud agent result schemas (#5375) Integrated into release/v3.8.42 * chore: remove unused a2a routing logger readers (#5374) Integrated into release/v3.8.42 * chore: remove unused webhook delivery detail export (#5372) Integrated into release/v3.8.42 * chore: remove unused api key type (#5395) Integrated into release/v3.8.42 * chore: remove unused usage types (#5397) Integrated into release/v3.8.42 * chore: remove unused cloud agent input types (#5373) Integrated into release/v3.8.42 * deps: bump electron from 42.4.1 to 42.5.1 in /electron (#5413) Integrated into release/v3.8.42 * deps: bump the production group with 11 updates (#5414) Integrated into release/v3.8.42 * fix: frame non-streaming JSON responses (#5416) Integrated into release/v3.8.42 * fix(services): runNpm shell on win32 + prefix via env for Node 24 EINVAL (#5379) (#5474) Node 24 refuses execFile of npm.cmd without a shell (nodejs/node#52554), so embedded-service install (9Router/CLIProxy) failed with spawn EINVAL on Windows. runNpm now enables shell on win32 only; to stay Hard-Rule-#13 safe under a shell, the install --prefix is passed via npm_config_prefix (env) instead of an argv path (survives spaces), and the user-supplied version is constrained by SERVICE_VERSION_PATTERN at the route boundary. * fix(cli): restore dist/tls-options.mjs to npm tarball (#5452) (#5503) Closes #5452 * fix(dashboard): render onboarding wizard on /providers/new (#5427) (#5505) Closes #5427 * fix(db): EBUSY-safe database import on Windows (#5406) (#5507) Closes #5406 * chore: remove unused gamification streak exports (#5463) * chore: remove unused headroom log tail export (#5464) * chore(dead-code): remove unused prompt cache control helper (#5466) * chore(duplication): share vscode metadata helpers (#5471) * chore(duplication): share auth zip extractors (#5475) * chore(duplication): share vscode tokenized request helper (#5479) * chore(duplication): share quota strategy ranking helpers (#5482) * chore(duplication): share recharts donut card (#5484) * chore(duplication): share provider specific validation (#5485) * chore(duplication): share batch response formatter (#5488) * chore(duplication): share redis runtime helpers (#5490) * chore(duplication): share version manager request parsing (#5492) * chore(duplication): share media generation route helpers (#5493) * chore(duplication): share settings transform schemas (#5496) * chore(duplication): share relay stream finalizer (#5497) * chore(duplication): share machine id fallback (#5498) * chore(duplication): share node sqlite adapter (#5500) * fix: treat terminal stream cancels as complete (#5491) * fix post-merge ci regressions (#5467) * fix: gate claude adaptive thinking defaults (#5480) Co-authored-by: KooshaPari <koosha@example.com> * fix(fallback): normalize provider error rule headers (#5473) Co-authored-by: KooshaPari <koosha@example.com> * fix(rate-limit): normalize queue refresh settings (#5499) Co-authored-by: KooshaPari <koosha@example.com> * chore(ci): add npm fetch-retry + release-freeze protocol (Hard Rule #21) (#5506) - .npmrc: bump fetch-retries 2->5 with backoff so transient registry ECONNRESET during npm ci (electron-release, v3.8.41) retries instead of failing the job; applies repo-wide. - CLAUDE.md Hard Rule #21: release-freeze coordination marker (label release-freeze) that campaign workflows honor before merging into the active release branch, preventing the mid-release commit races that forced CHANGELOG re-reconciliation in v3.8.40/v3.8.41. * chore(duplication): share service install helpers (#5495) Share service install helpers; re-add SERVICE_VERSION_PATTERN regex to the shared schema (dropped in extraction, #5474) + tests rejecting malformed versions. Co-authored-by: diegosouzapw <diegosouza.pw@gmail.com> * chore(duplication): share proxy route handlers (#5472) Share proxy route handlers; add resolveProxyLookupResponse regression test (3 branches + custom whereUsed param name). Co-authored-by: diegosouzapw <diegosouza.pw@gmail.com> * chore(duplication): share combo builder model options (#5477) Share combo builder model options; add regression test locking custom-model source classification (manual->custom, api-sync->imported). Co-authored-by: diegosouzapw <diegosouza.pw@gmail.com> * chore(dead-code): ratchet dead code baseline (#5468) Ratchet dead-code baseline to the true measured value (310 -> 225) after the v3.8.42 dead-code + duplication wave. Measured by check-dead-code.mjs on the tip. Co-authored-by: diegosouzapw <diegosouza.pw@gmail.com> * fix(dashboard): provider-add UX — i18n labels, surface import warning, default key name (#5511) * fix(dashboard): provider-add UX — real i18n labels, surface import warning, default key name (#5421 #5428 #5429 #5431 #5435) Three rough edges in the Add-API-Key / model-import flow, all from the provider-catalog audit: 1. Validation Model + Account ID form fields shipped untranslated i18n stub copy ('Validation Model Id Label', etc.) that rendered verbatim. Replaced with real copy in en.json. 2. Model import silently fell back to the cached/local catalog — the route returns a 'warning' field the import hook never read. New pure helper extractImportWarning surfaces it as a log line. 3. Required connection-name field defaulted to '' (let browser autofill inject garbage like 'wiw'); now defaults to 'main'. Regression guard: tests/unit/provider-add-ux-i18n-import-warning.test.ts. * fix(dashboard): compress AddApiKeyModal comment to keep file under frozen size cap * fix(providers): align Muse Spark (Meta AI) cookie copy to ecto_1_sess (#5449) (#5513) * fix(providers): align Muse Spark (Meta AI) cookie copy to ecto_1_sess (#5449) The default Meta AI session cookie migrated from the retired abra_sess to ecto_1_sess (META_AI_DEFAULT_COOKIE), but the provider form hint and one 401 auth-failure message still named abra_sess, telling users to paste a cookie that no longer exists. Both strings now name ecto_1_sess. Regression guard: tests/unit/muse-spark-cookie-copy-5449.test.ts. * chore: reconcile CHANGELOG with release (keep #5449 + #5511 bullets) * fix(providers): correct FriendliAI (serverless) + Novita (/openai/v1) endpoints (#5430 #5455) (#5515) * fix(providers): correct FriendliAI (serverless) and Novita (/openai/v1) endpoints (#5430 #5455) Both rejected valid keys, verified live with real provider keys: - FriendliAI baseUrl was /dedicated/v1/... which 403s a serverless flp_* token; switched to /serverless/v1/... + serverless modelsUrl. - Novita baseUrl was the legacy /v3/... with a typo'd model id ai-ai/... (both 404); switched to OpenAI-compat /openai/v1/... + meta-llama/llama-3.1-8b-instruct. Regression guard: tests/unit/provider-endpoints-friendliai-novita.test.ts. * chore: reconcile CHANGELOG with release (keep #5430/#5455 + prior bullets) * fix(providers): gate import for tool-only providers + sanitize Coze validation error (#5420 #5426) (#5522) #5420: the 'Import Models' button now hides for tool-only providers (web search / web fetch) via a capability check over resolved serviceKinds, not just the -search suffix — firecrawl/jina-reader (webFetch) no longer show an Import button that 400s. No LLM/media provider is affected. #5426: Coze key validation no longer leaks the raw upstream envelope ({code,msg,logId,from}) into the UI; the Coze error becomes a friendly message, scoped to provider === 'coze' so no other provider is affected. Regression guards: tests/unit/model-listing-capability-5420.test.ts, tests/unit/coze-validation-error-5426.test.ts. * fix(providers): correct LongCat free tier — GA LongCat-2.0, one-time 10M (KYC) (#5508) LongCat's preview ended and the Flash-* line was retired (2026-05-29); the API now exposes only the GA LongCat-2.0 (1M context, 128K output). The free tier is a ONE-TIME 10M-token grant unlocked after account signup + KYC verification — NOT a recurring daily/monthly allowance. The catalog still described the retired preview/Flash models and a recurring 150M / 5M-per-day budget; this corrects every reference. Config / code: - registry/longcat: model LongCat-2.0-Preview -> LongCat-2.0, name + comment reflect one-time 10M (KYC) and pay-as-you-go beyond it. - freeModelCatalog: longcat-2.0-preview (150M, recurring-daily) -> LongCat-2.0 (10M, freeType one-time-initial via creditTokens). - freeTierCatalog: drop longcat from the recurring-monthly budget map (one-time credits are excluded by that catalog's own rule). - regional.ts freeNote: one-time 10M after signup + KYC, not recurring. - providerCostData: longcat-flash-lite -> longcat-2.0 (pay-as-you-go 0.75/2.95 per 1M, 10M free quota). - validation probe model longcat -> LongCat-2.0. Tests: - free-tier-catalog: longcat now absent from FREE_TIER_BUDGETS; providerCount 22->21 (clean 21->20); documented total ~1.39B. - tierResolver: sample model flash-lite -> LongCat-2.0. Docs: - README, PROVIDERS-GUIDE, FREE-TIERS-GUIDE, FREE_TIERS: 50M/day Flash-Lite -> one-time 10M LongCat-2.0 (KYC); 'No auth' -> API key + KYC. - Regenerated PROVIDER_REFERENCE.md (picks up the new freeNote). typecheck:core clean; changed-file lint 0 errors; docs-sync PASS. * fix(providers): Bytez OpenAI-compat base URL + auth-only key validation (#5422) (#5528) Bytez IS OpenAI-compatible at .../models/v2/openai/v1, but the registry stored the bare .../models/v2 base, so validation's chat-probe hit .../models/v2/chat/completions -> 404 -> 'endpoint not supported'. Part A: registry baseUrl -> full OpenAI-compat chat path. Part B: a Bytez account only serves catalog-provisioned models, so chat-probe validation 404s even for valid keys. validateBytezProvider instead probes the auth-only GET .../models/v2/list/tasks (200=valid, 401/403=invalid). Verified live with a real key: list/tasks -> 200 (valid) / 401 (invalid). Regression guard: tests/unit/bytez-validation-5422.test.ts. * fix(providers): remove dead Phind provider + dedupe HuggingChat catalog listing (#5530) Integrated into release/v3.8.42 (round 3). Dead Phind removal + HuggingChat dedupe, verified complete. * fix: protect dynamic dashboard tests with CSRF (#5405) Integrated into release/v3.8.42 (round 3). Reworked CSRF (HMAC-signed synchronized token). * docs: clarify bifrost relay backend envs (#5520) Integrated into release/v3.8.42 (round 3). Doc-only: bifrost relay envs. * test(quota): guard Claude-Code identity version lockstep (Phase 2) (#5514) Integrated into release/v3.8.42 (round 3). Claude-Code identity version lockstep guard. * feat(compression): T02 — honest default-on pipeline inflation guard (H1) (#5527) Integrated into release/v3.8.42 (round 3). T02 pipeline inflation guard * feat(compression): T05/C2 — caveman dedup + ultra packs for de, fr, ja (#5529) Integrated into release/v3.8.42 (round 3). T05/C2 caveman packs de/fr/ja * feat(compression): T05/C6 — Chinese (zh / wenyan) caveman pack + detection (#5532) Integrated into release/v3.8.42 (round 3). T05/C6 zh/wenyan pack + detection * feat(compression): T07/R9 — gradle + dotnet RTK catalog filters (#5537) Integrated into release/v3.8.42 (round 3). T07/R9 RTK gradle+dotnet filters * refactor(dashboard): T11 — drop duplicate caveman on/off toggle from the compression settings tab (#5524) Integrated into release/v3.8.42 (round 3). T11 consolidate duplicate caveman controls; i18n'd the panel hint string (source key). * test relay routing fallback headers (#5526) Integrated into release/v3.8.42 (round 3). Relay fallback header extraction + tests (drift-shed: dependabot #5415 commit dropped). * fix(opencode-plugin): bump to 0.2.0 + auto-publish on release (#5363) - Bump @omniroute/opencode-plugin from 0.1.0 to 0.2.0 so CI publishes the accumulated fixes (auto combos, schema fields, debug logging) that were merged after the initial 0.1.0 publish on May 24. - Add auto-bump step in npm-publish.yml: detects if the plugin dir changed since the last release tag and auto-increments patch version, so the plugin never falls behind again on future releases. Co-authored-by: herjarsa <herjarsa@users.noreply.github.com> * [codex] add bifrost auto fallback cooldown (#5519) Integrated into release/v3.8.42 (round 3). Bifrost auto fallback cooldown; header reconciled with #5526 helper + env-doc. Co-authored-by: diegosouzapw <diegosouza.pw@gmail.com> * fix onboarding schema client import (#5525) Integrated into release/v3.8.42 (round 3). Browser-safe onboarding schema import (drift-shed: dependabot #5415 dropped). * docs: add relay backend strategy guide (#5547) Port #5533 relay strategy guide to release/v3.8.42 (doc-only). * fix(chatgpt-web): support GPT-5.5 Pro handoff (#5536) Integrated into release/v3.8.42 (round 3). GPT-5.5 Pro async stream_handoff support (drift-shed: dependabot #5415 dropped). * fix(providers): persist Configured filter across page reloads (#5510) Integrated into release/v3.8.42 (round 3). Persist Configured filter across reloads; extracted shouldSyncProviderDisplayMode race guard + TDD test (Closes #4059). Co-authored-by: diegosouzapw <diegosouza.pw@gmail.com> * fix(mimocode): route per-account traffic through SOCKS5 proxy dispatchers (#5521) Integrated into release/v3.8.42 (round 3). Per-account SOCKS5 dispatcher routing — completes #3837's stored proxy config with the actual undici dispatcher layer. Rebased onto .42 (dropped the CI-workflow-deletion commits; merged proxyUrlMap dispatch with #3837's acct.proxy storage). Co-authored-by: diegosouzapw <diegosouza.pw@gmail.com> * fix(chatgpt-web): portable SHA3-512 for sentinel PoW under Electron/BoringSSL (#5531) (#5540) * fix(build): keep ioredis out of the client/CLI bundle via SPAWN_CAPABLE_PREFIXES leaf (#5546) Fix the dast-smoke ioredis client-bundle regression (proven: dast-smoke green). Remaining reds are pre-existing base-reds/flakes (base.ts file-size, GOLDEN provider drift, shard-1 compression flakes) inherited by all PRs — not from this change. * chore(release): finalize v3.8.42 CHANGELOG + cycle-close reconciliation - Reconcile CHANGELOG.md for v3.8.42: 40 bullets covering all 89 commits since v3.8.41 (4 features, 26 fixes, 10 maintenance incl. 2 rollups for the 35-PR dead-code sweep + 17-PR DRY consolidation), dedup the merge- artifact duplicate New Features headers, set release date 2026-06-30. - Sync 42 docs/i18n/*/CHANGELOG.md mirrors. - Document 3 new chatgpt-web/TLS env vars in .env.example + ENVIRONMENT.md (OMNIROUTE_CGPT_WEB_PRO_TIMEOUT_MS, _PRO_POLL_INTERVAL_MS, OMNIROUTE_CHATGPT_STREAM_FIRST_BYTE_TIMEOUT_MS). - Cycle-close ratchet rebaselines: eslintWarnings 4116->4121, file-size base.ts/chatgpt-web.ts/strategySelector.ts/chatgpt-web.test.ts (all inherited drift, justified inline). - Regenerate provider translate-path golden snapshot for the merged bytez/friendliai/novita endpoint fixes. * chore(changelog): cover #5415 dev-deps bump merged from main The release/v3.8.42 ↔ main merge (c4c1b56ba) brought #5415 (development dependency group, 9 updates) and #5533 (relay backend guide) from main. #5533's content is already covered by the #5547 port bullet; add a Maintenance bullet for #5415 and re-sync the 42 i18n CHANGELOG mirrors. * test: relocate 2 orphaned test files to collected runner paths check:test-discovery flagged two cycle-merged tests that no runner collects (they never ran → false coverage confidence): - compression-settings-tab-consolidation.test.tsx (#5524) → tests/unit/ui/ (vitest UI runner collects tests/unit/ui/**/*.test.tsx); 3/3 pass. - providers/providerPageStorage.test.ts (#5510) → tests/unit/dashboard/ ('providers' is not a collected subdir; 'dashboard' is, same ../../../ import depth); 30/30 pass under the node runner. Both confirmed green when actually executed; no assertions weakened. * fix(release): repair inherited base-red tests from #5480/#5527/#5427/#5521 The fast-path (PR->release/**) does not run the full unit+integration suites, so four merged feature PRs shipped with stale/incorrect tests that only surface on the release PR (PR->main). Repairs (features are correct; align tests to the new behavior — no assertions weakened): - #5480 (gate claude adaptive thinking): adaptive thinking is now injected only for a real Claude Code client (x-app:cli / claude-code UA), not for any bare Claude OAuth token. claude-thinking-tool-choice-guard + base-thinking-budget-5312 now identify as a Claude Code client to exercise the adaptive path (3 tests). - #5527 (T02 inflation guard): the guard reverts a stacked body that did not shrink in tokens. The bail-out/advancement fixtures used growth-appending mock engines; they now carry a droppable padding message the engines empty, so the body realistically shrinks and the marker assertions survive. bailout (5), stacked-async (3), engine-enabled-toggle (2). - #5427 (render onboarding wizard at /providers/new): integration-wiring asserted the old redirect stub; now asserts the route renders ProviderOnboardingWizard. - #5521 (mimocode SOCKS5 per-account proxy): the constructor's default account omitted the proxy field (undefined), breaking the 'all proxies null' backward compat guard. Default it to null, mirroring syncAccountsFromCredentials(). * fix(proxyfetch): skip fallback for non-replayable bodies --------- Co-authored-by: Diego Rodrigues de Sa e Souza <diegosouza.pw@gmail.com> Co-authored-by: Jan Leon <Jan.gaschler@gmail.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Randi <55005611+rdself@users.noreply.github.com> Co-authored-by: Diego Rodrigues de Sa e Souza <8016841+diegosouzapw@users.noreply.github.com> Co-authored-by: KooshaPari <42529354+KooshaPari@users.noreply.github.com> Co-authored-by: KooshaPari <koosha@example.com> Co-authored-by: backryun <bakryun0718@proton.me> Co-authored-by: Hernan Javier Ardila Sanchez <hjasgr@gmail.com> Co-authored-by: herjarsa <herjarsa@users.noreply.github.com> Co-authored-by: Arthur Bodera <abodera@gmail.com> Co-authored-by: PizzaV <103120356+pizzav-xyz@users.noreply.github.com> Co-authored-by: OpenClaw Auto <openclaw-auto@example.invalid> * Move CLI profile sync toggles to CLI Code (#5778) * move CLI profile sync toggles to CLI Code * test CLI profile auto-sync toggles * Document CLI profile auto-sync flags * docs(changelog): note CLI profile auto-sync card moved to CLI Code (#5778) --------- Co-authored-by: Diego Rodrigues de Sa e Souza <diegosouza.pw@gmail.com> * fix(grok-cli): parse expires_at from auth.json and exp from JWT to fix auto-refresh (#5775) * fix(grok-cli): parse expires_at from auth.json and exp from JWT to fix auto-refresh * docs(changelog): note grok-cli token auto-refresh fix (#5775) --------- Co-authored-by: Diego Rodrigues de Sa e Souza <diegosouza.pw@gmail.com> * fix(providers): import intentional local-catalog-only providers instead of 502 (#5460, #5465) (#5787) The model-sync route returned a hard 502 ('Remote model discovery failed; local catalog fallback not synced') for every provider whose local catalog is its ONLY discovery source (Reka #5460, t3.chat #5465, embedding/rerank like voyage-ai/jina-ai, Qwen-OAuth, and web-cookie providers). The /models route now flags catalogs that are the provider's intended source (no remote /models endpoint) with intentional:true; model-sync imports those instead of 502-ing, while a genuinely degraded remote fallback still surfaces. New dependency-free leaf degradedLocalCatalog.ts. Also fixes t3.chat's confusing add-credential hint: it no longer renders the circular 'Required cookie: convex-session-id + Cookie header...' copy and wires the step-by-step DevTools hint (t3ChatWebCookieHint) already translated in every locale. Regression guards: tests/unit/sync-models-degraded-local-catalog-5460-5465.test.ts, tests/unit/t3chat-web-cookie-hint-5465.test.ts, + intentional-flag assertions in tests/unit/provider-models-route.test.ts. * fix(api): self-hydrate model aliases from DB on GET after restart (#5777) * Fix grammatical errors in readme (#5738) * fix(api): self-hydrate model aliases from DB on GET when in-memory state is empty In the standalone production build, webpack creates two separate copies of modelDeprecation.ts — one hydrated by the startup path (used for request routing) and one used by the /api/settings/model-aliases API route. The API route's copy starts with an empty _customAliases after each server restart, causing the Settings → Routing UI to show 'No exact-match aliases configured' even though the aliases are persisted in the DB. The GET handler now detects an empty _customAliases state and reads the modelAliases key from the settings blob in the DB, calling setCustomAliases() to hydrate this module instance. This is a best-effort fallback — when _customAliases is already populated (e.g. by the startup path in dev mode), no DB read occurs. Regression test: tests/unit/model-aliases-settings-route-selfheal.test.ts - Verifies hydration from DB when in-memory state is empty - Verifies no hydration when in-memory state is already populated - Verifies graceful handling when no modelAliases exist in DB --------- Co-authored-by: Chirag Singhal <76880977+chirag127@users.noreply.github.com> Co-authored-by: marcelpeterson <marcelpeterson@users.noreply.github.com> Co-authored-by: Diego Rodrigues de Sa e Souza <diegosouza.pw@gmail.com> * refactor(usage): extract 5 provider usage families into leaves (#5782) Split open-sse/services/usage.ts (1723 -> 901 LOC) by moving the Cursor, Kimi, Codex, Claude and Kiro usage-fetcher families into cohesive leaves under open-sse/services/usage/ (mirroring the existing glm/minimax/antigravity/quota/ scalars leaves): - usage/cursor.ts getCursorUsage (+ CURSOR_USAGE_CONFIG, decodeCursorJwtSub) - usage/kimi.ts getKimiUsage (+ KIMI_CONFIG, getKimiPlanName) - usage/codex.ts getCodexUsage (+ CODEX_CONFIG) - usage/claude.ts getClaudeUsage / getClaudePlanLabel (+ CLAUDE_CONFIG, legacy) - usage/kiro.ts getKiroUsage / buildKiroUsageResult / discoverKiroProfileArn (+ helpers) The host keeps the getUsageForProvider dispatcher and imports the fetchers back; the public export set is unchanged — buildKiroUsageResult + discoverKiroProfileArn are re-exported from the kiro leaf (the kiro-* tests import them from services/usage) and __testing stays wired to the moved claude/kiro internals. Bodies are verbatim: the code-line multiset of host + leaves equals the original. Adds tests/unit/usage-families-split.test.ts pinning the leaf surface, the kiro re-export identity, the __testing wiring, and getClaudePlanLabel's pure logic. * chore(docs): sync i18n CHANGELOG mirrors with root [3.8.43] section (#5789) Regenerate the docs/i18n/<locale>/CHANGELOG.md [3.8.43] blocks from the root CHANGELOG so the mirror body size returns within the 25% docs-sync tolerance. Clears a pre-existing release-time drift (mirrors were ~26% smaller than root) that was failing check-docs-sync and blocking every local commit on the release branch. * fix(providers): correct stale/broken provider metadata (#5487, #5461, #5534, #5470) (#5790) - #5487 Qoder: replace the untranslated i18n stubs (personalAccessTokenLabel, qoderPatHint, qoderPatPlaceholder) with real copy; extend the STUB_KEYS guard. - #5461 Scaleway: website pointed at scaleway.com/en/ai/generative-apis (HTTP 404); repoint at the live docs URL /en/docs/ai-data/generative-apis/. - #5534 Microsoft 365 Copilot: rewrite the vague authHint with concrete DevTools WebSocket steps (the token lives on the Chathub WS URL, not an Authorization header). - #5470 Together AI: retired the $25 signup credit and is now fully prepaid (min $5); hasFree false + a prepaid notice instead of the stale free-tier freeNote (verified live). Regression guards: tests/unit/provider-metadata-5461-5470-5534.test.ts + Qoder keys added to tests/unit/provider-add-ux-i18n-import-warning.test.ts. * fix(dashboard): neutral badge for unsupported validation + clickable OAuth error links (#5442, #5486) (#5795) - #5442 LMArena (and any provider with no live validator) returns { unsupported: true } from /api/providers/validate and Save succeeds, but the Add-API-Key modal only had success/failed states so it rendered a red 'Invalid' badge. Add an 'unsupported' result → neutral info 'N/A' badge via the pure leaf validationBadgeProps(); both validate handlers now map data.unsupported to it. - #5486 GitLab Duo's OAuth setup error embeds a registration URL (gitlab.com/-/profile/applications) but the OAuth error step rendered it as dead red text. New LinkifiedText component (+ pure ReDoS-safe linkify util) makes any http(s) URL in an OAuth error clickable; the GitLab Duo backend message already carries the full setup steps. Regression guards: tests/unit/validation-badge-unsupported-5442.test.ts, tests/unit/oauth-error-linkify-5486.test.ts. Frozen god-files kept within cap (AddApiKeyModal 868/868, OAuthModal 968/969). * fix(system): route in-app auto-update npm calls through the win32 shell helper (#5542) (#5797) The in-app auto-update flow called execFileAsync("npm", ...) directly for the version lookup (versionCheck.getLatestVersionFromNpmCli), dependency install, global install, and native rebuild. On Windows npm is npm.cmd and Node >=24 refuses to execFile a .cmd without a shell (nodejs/node#52554), so those calls threw 'spawn npm ENOENT'. Route them through buildNpmExecOptions (the same win32-shell helper the embedded-services installer uses, fix #5379). The global install spec is validated with SERVICE_VERSION_PATTERN before it is shell-joined (Hard Rule #13). Not the pnpm/npx swap the issue proposed — that is the wrong direction for an 'npm install -g' flow already solved elsewhere in-repo. Regression guard: tests/unit/autoupdate-npm-win32-5542.test.ts. * refactor(sse): extract cursor protobuf wire primitives into a leaf (#5794) Split open-sse/utils/cursorAgentProtobuf.ts (1520 -> 1400 LOC) by moving the low-level protobuf wire-format primitives — varint/tag/length-delimited encode+ decode + the generic field walker (encodeVarint, encodeTag, encodeBytes, encodeString, encodeMessage, encode{UInt32,Bool,Double}Field, decodeVarint, checkedLen, decodeFields, findField, decode{String,Varint}Field, the Field type and the WT_VARINT/WT_LEN wire-type constants) — into cursorAgentProtobuf/wire.ts. These primitives were module-private, so the host's public API is unchanged; the host imports them back internally. Bodies are verbatim: the code-line multiset of host + wire.ts equals the original. First layer of the codec decomposition — the value/framing codec and the message encoders/decoders build on this and stay in the host (they share host-retained helpers; splitting them is a separate step). Adds tests/unit/cursor-protobuf-wire-split.test.ts pinning the leaf surface, the encode/decode round-trip invariants, the buffer-overrun guard, and the host wiring. * test(runtime): guard tsx/esm→esbuild transform path on boot (#5757) (#5773) #5757 reported that a fresh `npm install omniroute` pulls `esbuild@0.28.1` transitively via `tsx` (a runtime dependency the CLI registers at boot in `bin/omniroute.mjs`), and proposed forcing `esbuild@0.27.4`. That override is unsafe: `tsx@4.22.4` requires `esbuild@~0.28.0` and `fumadocs-mdx@15` (also a runtime dep) requires `esbuild@^0.28.0`; forcing 0.27.x pushes esbuild below both, and 0.28.1 is currently the latest release. The reported transform failure also does not reproduce — OmniRoute targets ES2022, its minimum supported Node is 22.2 (destructuring is native), and tsx targets the running Node, so esbuild never lowers to an unsupported target. Instead of an unsafe version pin, add two regression guards: - functional: spawn the real `node --import tsx/esm` loader on a fixture packed with modern syntax (destructuring/spread, class+private fields, optional chaining, nullish, logical assignment, async + top-level await) and assert it transforms + runs correctly. Fails if a future esbuild regresses the boot path. - dependency-shape: assert the resolved esbuild stays within tsx's declared range, so nobody reintroduces the out-of-range override this issue proposed. No production code changed; no esbuild version pinned. * fix(deps): add missing runtime deps @toon-format/toon and safe-regex (#5771) Both packages are imported at runtime but were only declared for their type shims (safe-regex was via @types/safe-regex; @toon-format/toon had no declaration at all). Missing runtime deps mean: - open-sse/services/compression/engines/headroom/toon.ts imports @toon-format/toon → MODULE_NOT_FOUND on cold pnpm/npm install - open-sse/services/compression/engines/ccr/ccrQuery.ts imports safe-regex → MODULE_NOT_FOUND Both engines are wired into the stacked compression pipeline (default enabled), so a fresh clone that does not have a stale node_modules from a previous version crashes as soon as the pipeline runs. Verified with pnpm ls / grep before/after. * fix(oauth): clamp grok-cli expired-token expiresIn to a positive value (#5775 follow-up) (#5820) An already-expired grok-cli token (real expires_at/exp in the past) produced a negative expiresIn, which is truthy in the import-token route and maps to a PAST expiresAt — AutoCombo then reads that as 'already expired' and excludes the connection instead of refreshing it. Clamp with Math.max(1, expiresIn) so an expired token is treated as due-for-refresh. Extends #5775 (thanks @Chewji9875). Regression: 2 new cases in tests/unit/grok-cli-oauth.test.ts (expired JWT exp + expired JSON expires_at), both failing-then-passing. * fix(model-aliases): back custom-alias store with globalThis (#5777 follow-up) (#5821) #5777 self-healed the GET /api/settings/model-aliases symptom at the route layer, but the root cause remained: modelDeprecation.ts held _customAliases in a plain module-level let, which webpack duplicates across the startup and app-route module graphs (same class as #5312). Startup hydration landed on one copy; the API route read the other (empty) one. Back the store with globalThis (__omniroute_customAliases__) so both instances share one store — the exact pattern already used by thinkingBudget.ts/backgroundTaskDetector.ts (#5312). The route-layer DB self-heal from #5777 stays as a harmless fallback. Extends #5777 (thanks @jleonar2). Regression: tests/unit/model-aliases-globalthis-5777.test.ts (fails on the plain-let store: never populates globalThis, never reads a sibling instance's write). * chore(release): rebaseline file-size + test-masking ratchets for v3.8.43 (#5609) DRIFT acumulado dos 109 commits do ciclo v3.8.43 (fast-gate PR->release nao roda check:file-size/test-masking; base-reds so afloram na release-PR): - file-size: 8 god-files existentes cresceram + 2 arquivos novos acima do cap + 4 test files cresceram -> frozen ajustado ao estado atual. - test-masking: chatgpt-web.test.ts 281->280 asserts allowlisted (#5549 consolidou 2 assert.equal num unico map-driven; refactor legitimo, nao masking). Modularizacao dos god-files deferida (#3501). * refactor(sse): extract openai-to-gemini pure helpers into a leaf (#5824) Split open-sse/translator/request/openai-to-gemini.ts (873 -> 756 LOC, back under the 800-line cap) by moving the module-private pure helpers — the historical-tool- context string builders (stringifyHistoricalToolArguments, buildInertHistorical*, escapeHistoricalContext*, buildHistoricalToolResultContext), deepCleanUndefined, extractClientThoughtSignature, buildChangedToolNameMap, isVertexGeminiProvider, and applyAntigravityGenerationDefaults (with its GeminiGenerationConfig shape) — into openai-to-gemini/helpers.ts. These were module-private, so the translator's public API is unchanged; the host imports them back internally. Bodies are verbatim: the code-line multiset of host + leaf equals the original. Adds tests/unit/openai-to-gemini-helpers-split.test.ts pinning the leaf's pure behaviour (escaping, undefined-pruning, signature extraction, antigravity generation-config defaults) and the host wiring. * fix(db): re-export modelContextOverrides from localDb (check:db-rules #5609) * test(discovery): wire tests/unit/memory into node runner glob (#5609) typed-decay.test.ts (TV6 typed memory decay, 15 asserts) sat in tests/unit/memory/ which no runner glob collected -> orphan (never ran). Adds 'memory' to the subdir brace-glob in all runner sources (package.json scripts + ci.yml shards) and the COLLECTORS mirror in check-test-discovery.mjs (drift-check keeps them in sync). Passes standalone (15/15); DATA_DIR isolation handled per-file by tests/_setup/isolateDataDir.ts. * test: align 3 stale release tests to landed behavior (#5609) Base-reds surfaced on the release PR (fast-gate PR->release skips these shards): - api-manager-page-static: Self-service Visibility now has 5 switches (added the API-key provider quota-policy bypass toggle, #5731); bump inventory 4->5 while keeping the invariant that every switch declares type=button (verified 5/5 typed). - security-hardening (callLogs PII): #5725 extracted sanitizeErrorForLog into callLogs/format.ts; assert the new wiring (callLogs imports it + format.ts imports piiSanitizer) instead of the removed direct import — PII sanitization still intact. - memory-glm-injection: #5610 made GLM 5.1+ ACCEPT the system role (z.ai docs), so glm-5.1 must PRESERVE system, not fold it. Flip the stale #1701-era assertion. * test(shared): align t3-web web-session expected metadata with hintKey (#5835) The t3-web provider metadata intentionally carries `hintKey: "t3ChatWebCookieHint"` (#5465 — the generic cookie hint reads circular for t3.chat), but the metadata assertion in web-session-credentials was never updated, so it deep-equals against an object missing the field. This is a stale-test base-red on release/v3.8.43 that turns the whole PR queue's "Unit Tests fast-path (1/2)" red. Align the expected object to the shipped source of truth. * test(compression): de-flake rtk_discover sample seeding seedSamples() persisted two byte-identical raw outputs. The raw-output filename is keyed on Date.now() (ms) + a content hash (rawOutput.ts), so two identical captures landing in the same millisecond collapse to one file (the 2nd write overwrites the 1st) -> sampleCount 1 instead of 2. Reproduced at ~25% (501/2000 trials), matching the intermittent Coverage Shard (5/8) failure on fast CI runners. Seed two DISTINCT captures so the store deterministically holds 2 samples regardless of timing (0/2000 collisions after the change). * test(e2e): anchor compression-studio smoke on play-input, not async play-lane The T03 smoke asserted `play-lane` visible on mount, but those per-lane buttons only render after a preview-compression run populates `batch.lanes` (usePreviewCompression keeps batch null until run(); there is no mount auto-run). The smoke intentionally does not drive a compression cascade, so `play-lane` can never appear -> the E2E added in #5727 failed all 3 retries (E2E Tests 4/9). Anchor on the always-present `play-input` panel, which proves the studio body mounted without needing async lane data. * fix(security): explicit http(s) scheme allowlist in linkifyText href CodeQL flagged the <a href> in LinkifiedText (#5486) with js/xss (high) and js/client-side-unvalidated-url-redirection (medium) because href traces back to user-provided text. URL_RE already requires an http(s):// prefix, so a javascript:/data: scheme can never reach href — but that guarantee was only implied by the regex. Validate the scheme explicitly via new URL().protocol before exposing href (non-http(s) degrades to plain text): defense-in-depth that also makes the sink provably safe to static analysis. Regression test added. * fix(ci): register mark-account-unavailable test in stryker tap.testFiles check:mutation-test-coverage --strict (Fast Quality Gates) flagged tests/unit/mark-account-unavailable-numeric-epoch-guard.test.ts as a covering unit test missing from stryker.conf.json tap.testFiles, so its mutant kills would not count (--strict). Add it. Pre-existing tap.testFiles drift on the release tip that fails Fast Quality Gates on every PR into release/v3.8.43, not just this branch. * chore(release): rebaseline eslintWarnings ratchet 4121->4158 (v3.8.43 cycle drift) * chore(release): rebaseline complexity 1981->1982 + cognitive-complexity 842->845 (v3.8.43 cycle drift) * chore(release): rebaseline deadExports 225->227 (v3.8.43 cycle drift) * fix(dashboard): add error boundaries for Combos and MITM Proxy pages (#5788) Integrated into release/v3.8.43 * fix(cli): rename process title to omniroute (#5791) Integrated into release/v3.8.43 * fix(providers): add claude-sonnet-5 to Kiro model catalog (#5796) Integrated into release/v3.8.43 * fix(kiro): bound Claude id dash->dot minor group to protect date-suffixed ids (#5825) Integrated into release/v3.8.43 * fix(db): allowlist modelContextOverrides as intentionally-internal to green release DB-rules gate (#5798) (#5827) Integrated into release/v3.8.43 * fix(sse): stop reasoning-summary drop + duplicated deltas on claude→codex streaming (#5786) (#5832) Integrated into release/v3.8.43 * fix(dashboard): guard null modelAliases values in model picker (#5792) Integrated into release/v3.8.43 * fix(github): drop trailing assistant prefill for Copilot chat (#5802) Integrated into release/v3.8.43 * fix(oauth): disambiguate OAuth connections on username to prevent cross-IdP overwrites (#5803) Integrated into release/v3.8.43 * fix(translator): strip orphaned tool results across request formats (#5805) Integrated into release/v3.8.43 * fix(kiro): stop injecting placeholder user turn on tool-result turns (#5807) Integrated into release/v3.8.43 * fix(mitm): clean up privileged hosts entries on exit when possible (#5808) Integrated into release/v3.8.43 * fix(translator): prevent doubled tool args in OpenAI-to-Claude (#5828) Integrated into release/v3.8.43 * fix(usage): keep tool definitions visible when request log is truncated (#5829) Integrated into release/v3.8.43 * fix(db): preserve healthCheckInterval=0 across create/update (#5822) Integrated into release/v3.8.43 * fix: unify dashboard csrf origin fallback (#5856) Integrated into release/v3.8.43 * fix(kimi-web): migrate to www.kimi.com Connect-RPC API (kimi.moonshot.cn retired) (#5858) Integrated into release/v3.8.43 * fix(qwen-web): unblock validator + chat completion (retired endpoint + missing SPA version header) (#5855) Integrated into release/v3.8.43 * fix(antigravity): 429 hang on credit exhaustion and precise reset time lockout (Cleaned) (#5846) Integrated into release/v3.8.43 * fix(cli): correct rootDir resolution in doctor.mjs on Windows (#5844) (#5845) Integrated into release/v3.8.43 * Show startup time in ready banner (#5799) Integrated into release/v3.8.43 * extracted CorrelationId observability changes from #5275 (#5834) Integrated into release/v3.8.43 * refactor(executors): deduplicate shared utilities and add comprehensive tests (#5720) Integrated into release/v3.8.43 * Harden provider node URL validation (#5760) Integrated into release/v3.8.43 * [codex] Tune adaptive stream readiness timeouts (#5767) Integrated into release/v3.8.43 * fix: restore om-usage HTTP endpoint (#5859) Integrated into release/v3.8.43 * fix(sse): strip zero-width markers from streamed responses (parity with non-streaming) (#5857) Integrated into release/v3.8.43 * [codex] Protect long-running agent goal streams (#5772) Integrated into release/v3.8.43 * refactor(oauth): remove dead legacy OAuth service classes (#5838) The src/lib/oauth/services/ service-class hierarchy is superseded — the live OAuth flow runs through src/lib/oauth/providers.ts + providers/. The old per-provider 'class *Service extends OAuthService' implementations and their barrel had zero production or test references. Removed oauth/openai/github/claude/codex/antigravity/ qwen/qoder + the index barrel (-1559 LOC). Kept kiro.ts, cursor.ts, codexImport.ts (routes import them directly by path, never via the deleted barrel). Proven safe by typecheck:core staying green (a live reference would fail the build) + a filesystem guard test pinning the removal. Salvage of closed PR #5039. gaps v3.8.42 - T10 (5.7). * chore(docs): scope release-freeze to /generate-release only (Hard Rule #21) (#5839) A freeze is authorized ONLY inside /generate-release (raised Phase 0a, lifted Phase 12c). No campaign/session/agent may open a release-freeze mid-development; if one is ever unavoidable outside the release flow it must be requested from the operator in chat first with an explicit "estou criando um freeze" alert. Also codifies: never lift an active captain freeze to unblock campaign merges (it auto-lifts at 12c). * fix(chat): preserve JSON default when stream is omitted (#5866) * fix(chat): preserve JSON default when stream is omitted * chore(chat): type route record guard * fix(api): gate early SSE keepalive on explicit stream intent, keep body untouched Remove the stream:false body normalization so the legacy streaming default (resolveStreamFlag) and the per-key streamDefaultMode json opt-in keep deciding the response framing; the keepalive wrapper is only applied when stream:true is explicit or Accept forces SSE. Co-authored-by: diegosouzapw <diegosouza.pw@gmail.com> --------- Co-authored-by: diegosouzapw <diegosouza.pw@gmail.com> * feat(usage): report usage command quotas as percentages + honor observed provider quota resets (#5874) * feat: report usage command quotas as percentages Convert @@om-usage and the HTTP usage endpoint to report personal API key quotas as remaining percentages while keeping USD amounts out of the command output. Scale provider quota remaining percentages by the configured quota cutoff so the protected reserve reads as 0% left. Restore provider USD cost drilldown in the quota dashboard.\n\nAlso sync the 3.8.43 i18n changelog mirrors so the docs-sync pre-commit gate remains green.\n\nTests: DISABLE_SQLITE_AUTO_BACKUP=true node --import tsx/esm --test tests/unit/internal-usage-command.test.ts; DISABLE_SQLITE_AUTO_BACKUP=true node --import tsx/esm --test tests/unit/api-key-usage-limits.test.ts; DISABLE_SQLITE_AUTO_BACKUP=true node --import tsx/esm --test tests/unit/provider-window-costs.test.ts; DISABLE_SQLITE_AUTO_BACKUP=true node --import tsx/esm --test tests/unit/api-manager-usage-command.test.ts tests/unit/apikeys-usage-command.test.ts; npx eslint <changed files>; npm run typecheck:core; npm run build; npm run check:migration-numbering; npm run check:docs-sync; docker build --target runner-base (cherry picked from commitf66abd2028) * fix: honor observed provider quota resets Detect same-resetAt quota resets when provider usage drops back to the reset floor, and prefer that observed snapshot over stale recorded weekly events for provider USD windows and API-key USD quotas.\n\nTests: npx eslint changed files\nTests: npm run typecheck:core\nTests: DISABLE_SQLITE_AUTO_BACKUP=true node --import tsx/esm --test tests/unit/lib/quota-reset-events.test.ts tests/unit/provider-window-costs.test.ts tests/unit/api-key-usage-limits.test.ts\nTests: npm run build\nTests: docker build --target runner-base --build-arg OMNIROUTE_BUILD_MEMORY_MB=4096 -t omniroute:quota-reset-window-20260702002300 . (cherry picked from commit39c12a6f17) * docs(changelog): credit usage quota percentages extraction from #5863 Co-authored-by: diegosouzapw <diegosouza.pw@gmail.com> --------- Co-authored-by: Wital <wital@example.com> * fix(github): keep Copilot access-token sessions active (#5875) * fix(github): keep Copilot access-token sessions active GitHub Copilot device-flow accounts may have a GitHub access token and short-lived Copilot token without a refresh token. The proactive health check was treating that as terminal no_refresh_token and marking the connection expired minutes after login. Keep those sessions active, clear stale no_refresh_token state, and refresh the Copilot sub-token when needed.\n\nTests:\n- npx eslint src/lib/tokenHealthCheck.ts tests/unit/token-health-no-refresh-token-expired-5326.test.ts\n- DISABLE_SQLITE_AUTO_BACKUP=true node --import tsx/esm --test tests/unit/token-health-no-refresh-token-expired-5326.test.ts tests/unit/token-health-check.test.ts tests/unit/token-health-check-circuit-breaker.test.ts tests/unit/token-refresh-service.test.ts tests/unit/token-refresh-route-service.test.ts tests/unit/executor-github.test.ts\n- npm run typecheck:core\n- npm run build (cherry picked from commit68095d4796) * docs(changelog): credit Copilot token-health fix extraction from #5863 Co-authored-by: diegosouzapw <diegosouza.pw@gmail.com> --------- Co-authored-by: Wital <wital@example.com> * feat: add NEXT_PUBLIC_LIVE_WS_PUBLIC_URL for custom domain WebSocket support (#5878) * docs: add ai_features scope to GitLab Duo OAuth env setup instructions * docs: add LIVE_WS_ALLOWED_HOSTS env var to example config for LAN/Tailscale setups * feat: add web socket public URL for reverse proxy/Cloudflare Tunnel WebSocket setups * fix(dashboard): resolve live WS public URL at runtime via handshake with scheme validation - Read NEXT_PUBLIC_LIVE_WS_PUBLIC_URL lazily in /api/v1/ws (function, not module-level const) so runtime env changes are honored in prebuilt images. - Only echo/consume publicUrl when it is a ws:// or wss:// URL (server and client guards); anything else is rejected to null. - useLiveDashboard now fetches /api/v1/ws?handshake=1 before connecting and prefers: explicit wsUrl > build-time env > handshake publicUrl > default. - Align GitLab Duo scopes line in .env.example with GITLAB_DUO_CONFIG.scope. - Extend tests: lazy env read + scheme validation cases. - CHANGELOG entry for 3.8.43. Co-authored-by: diegosouzapw <diegosouza.pw@gmail.com> --------- Co-authored-by: Septianata Rizky Pratama <ian.rizkypratama@gmail.com> * Add .editorconfig to improve repository standards (#5879) * chore(ci): pass sonar.projectVersion to SonarQube scan so the new-code baseline advances per release (#5880) * fix(dashboard): Modal — two-field auth (Token ID + Token Secret) (#5446) (#5881) * fix(dashboard): add Modal Token ID + Token Secret fields (#5446) Modal authenticates with a Token ID (ak-…) + Token Secret (as-…) pair sent as `Authorization: Bearer <TOKEN_ID>:<TOKEN_SECRET>`. The add-connection form only exposed a single API-key field, so users could not enter both credentials. Add a dedicated two-field form for the `modal` provider: the existing field is relabeled "Token ID" and a new "Token Secret" field is rendered below it. Both are combined into the single encrypted `apiKey` value via a new pure helper `combineModalCredential(id, secret)` → `id:secret`, so the generic bearer executor path emits `Bearer <id:secret>` with no registry/executor/DB changes. An empty secret returns the id verbatim, preserving the ability to paste a pre-combined `id:secret` into the single field. The field hint points to https://modal.com/settings → API Tokens. Registry (baseUrl/executor), DB schema, and the request-time header path are untouched — Modal remains bring-your-own-deploy. Tests: tests/unit/modal-credential-combine.test.ts (5, TDD). * docs(changelog): add v3.8.43 bullet for Modal two-field auth (#5446) * fix(mcp): forwarded caller auth wins over OMNIROUTE_API_KEY env fallback (#5819) (#5882) * fix(middleware): run operator hook code in hardened vm sandbox instead of new Function (#5872) (#5885) * fix(providers): include custom compatible providers in auto/ routing (#5873) (#5886) * fix(db): honor autoBackupEnabled setting for pre-write backups (#5871) (#5888) * fix(dashboard): gate Token Expired badge on terminal testStatus, not raw token expiry (#5836) (#5883) * docs: use pnpm --allow-build flag instead of unsupported approve-builds -g (#5554) (#5884) * fix(dashboard): pre-fill Modal Validation Model Id with the server probe model (#5446) (#5892) * fix(api): strip upstream x-middleware-* headers from proxied responses (#5849) (#5893) * fix(providers): restore codex inference for unprefixed gpt-5.5 on codex-only setups (#5887) (#5895) * test(autoCombo): stabilize model fitness source expectation (#5890) * test(autoCombo): make fitness source test stable against model caps * chore(ci): retrigger checks for PR 5890 * docs(changelog): add 3.8.43 bullet for the autoCombo fitness-source test stabilization (#5890) --------- Co-authored-by: kooshapari <kooshapari@users.noreply.github.com> Co-authored-by: Diego Rodrigues de Sa e Souza <diegosouza.pw@gmail.com> * docs(architecture): Router Backends & Embedded Services ADR (#5603) (#5891) * routing: add router backend registry * docs(architecture): add Router Backends & Embedded Services ADR (#5603) Document the two orthogonal axes that #5603 asked to clarify: an engine's lifecycle (in-process / supervised / external / disabled) vs the relay routing backend selection (ts / bifrost / auto). Anchors the ADR on the typed `src/domain/routing/routerBackends.ts` registry as the single source of truth, and captures the /api/services/* status-code contract (409/200/404/403/500 + the LOCAL_ONLY loopback guard) so dashboard errors are interpretable. Stacked on the router-backend-registry work so it documents a real contract. * docs(architecture): reduce ADR PR to docs-only — registry lands via #5868; describe adoption as tracked, not current * docs(changelog): add 3.8.43 bullet for the Router Backends ADR (#5891) --------- Co-authored-by: KooshaPari <kooshapari@gmail.com> * fix(ci): re-green release/v3.8.43 fast-gates — db-rules stale allowlist + 4 more base-reds (#5798) (#5896) * fix(db): remove stale modelContextOverrides allowlist entry from check:db-rules (#5798) * fix(ci): clear release/v3.8.43 fast-gates base-reds (env-docs, ADR refs, mutation-cov, ratchets) (#5798) * fix(sse): type-safe resolveBaseUrl/resolveEffectiveKey coercions in BaseExecutor (typecheck:core base-red, #5798) * chore(quality): freeze base.ts at post-typecheck-fix size (#5798) * fix(docs): add required MDX frontmatter to ROUTER_BACKENDS ADR (build base-red, #5798) * fix(image): keep bare gpt-5.5 codex mapping in image resolver (#5902) * fix: preserve codex bare image model over combo shadowing * docs(changelog): credit #5902 codex bare image alias fix * docs(changelog): restore #5902 bullet after merge auto-resolve --------- Co-authored-by: Diego Rodrigues de Sa e Souza <diegosouza.pw@gmail.com> * fix(providers): route OpenAI responses-only models to /v1/responses (#5842) (#5901) * fix(providers): route OpenAI responses-only models to /v1/responses (#5842) * docs(changelog): restore #5842 bullet after merge auto-resolve ate it * docs(changelog): keep #5842 bullet additive over release tip * chore(release): v3.8.43 — 2026-07-02 * chore(release): allowlist 3 verified-legitimate test-assert reductions (#5805/#5856/#5855) * chore(release): rebaseline file-size caps for base.ts + 2 aligned test files (v3.8.43 release-close) * docs(changelog): add v3.8.43 Contributors section + sync i18n mirrors --------- Co-authored-by: KooshaPari <42529354+KooshaPari@users.noreply.github.com> Co-authored-by: Arthur Bodera <abodera@gmail.com> Co-authored-by: Wahyu Hidayatulloh Pamungkas <87377496+Stazyu@users.noreply.github.com> Co-authored-by: skyzea1 <161649495+skyzea1@users.noreply.github.com> Co-authored-by: José Victor Ferreira <root@josevictor.me> Co-authored-by: Choti Wongbussakorn <126886556+Chewji9875@users.noreply.github.com> Co-authored-by: backryun <bakryun0718@proton.me> Co-authored-by: Jan Leon <Jan.gaschler@gmail.com> Co-authored-by: warelik <warelik@users.noreply.github.com> Co-authored-by: WITALO ROCHA <witalo_rocha@hotmail.com> Co-authored-by: Randi <55005611+rdself@users.noreply.github.com> Co-authored-by: Alex <alexgild@gmail.com> Co-authored-by: Chirag Singhal <76880977+chirag127@users.noreply.github.com> Co-authored-by: Ardem2025 <ardemb22@gmail.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: KooshaPari <koosha@example.com> Co-authored-by: Hernan Javier Ardila Sanchez <hjasgr@gmail.com> Co-authored-by: herjarsa <herjarsa@users.noreply.github.com> Co-authored-by: PizzaV <103120356+pizzav-xyz@users.noreply.github.com> Co-authored-by: OpenClaw Auto <openclaw-auto@example.invalid> Co-authored-by: jleonar2 <92810914+jleonar2@users.noreply.github.com> Co-authored-by: marcelpeterson <marcelpeterson@users.noreply.github.com> Co-authored-by: Yuan Li <atom.long@outlook.com> Co-authored-by: janeza2 <49841619+janeza2@users.noreply.github.com> Co-authored-by: Aris <arissunandar399@gmail.com> Co-authored-by: Isha Tiwari <156085572+ishatiwari21@users.noreply.github.com> Co-authored-by: Markus Hartung <mail@hartmark.se> Co-authored-by: Nguyen Minh <lop123thcs@gmail.com> Co-authored-by: Denis Kotsyuba <kocubads96@gmail.com> Co-authored-by: Wital <wital@example.com> Co-authored-by: Septianata Rizky Pratama <ian.rizkypratama@gmail.com> Co-authored-by: Shiva Vinodkumar <127319648+shiva24082@users.noreply.github.com> Co-authored-by: kooshapari <kooshapari@users.noreply.github.com> Co-authored-by: KooshaPari <kooshapari@gmail.com>
This commit is contained in:
Diego Rodrigues de Sa e Souza
committed by
GitHub
GitHub
parent
1085514c56
commit
b729a8f273
55
tests/e2e/compression-studio.spec.ts
Normal file
55
tests/e2e/compression-studio.spec.ts
Normal file
@@ -0,0 +1,55 @@
|
||||
import { expect, test } from "@playwright/test";
|
||||
import { gotoDashboardRoute } from "./helpers/dashboardAuth";
|
||||
|
||||
/**
|
||||
* T03 — Compression Studio (Tela A) smoke e2e (gaps v3.8.42).
|
||||
*
|
||||
* The studio's reducers/renderers (compressionFlowModel, WaterfallInspector,
|
||||
* CompressionCockpit, PlayView/CompareView) are unit/vitest-covered. What unit tests
|
||||
* CANNOT catch is a client-side render / hydration crash on the real page — the same
|
||||
* "no `useTranslations` trap" risk the compression-UI plan flagged. The existing
|
||||
* combo-live-studio spec guards `/dashboard/combos/live`; this is its missing
|
||||
* counterpart for the dedicated compression studio (Tela A), which that spec does not
|
||||
* touch.
|
||||
*
|
||||
* It loads /dashboard/compression/studio and asserts the studio mounts (Play tab + its
|
||||
* lane), then flips to the Compare tab and asserts the switch took effect. The
|
||||
* visibility assertions ARE the hydration-trap guard: a render-time crash would mean
|
||||
* none of these testids ever appear.
|
||||
*
|
||||
* Out of scope (kept unit-covered, to stay non-flaky): driving a live WS compression
|
||||
* cascade (needs `compression.step` events an e2e cannot inject) and asserting console
|
||||
* output (dev-mode on-demand compilation emits transient fast-refresh noise).
|
||||
*/
|
||||
test.describe("Compression Studio (Tela A)", () => {
|
||||
test("loads /dashboard/compression/studio and renders the Play lane without crashing", async ({
|
||||
page,
|
||||
}) => {
|
||||
await gotoDashboardRoute(page, "/dashboard/compression/studio");
|
||||
|
||||
const playTab = page.locator('[data-testid="tab-play"]');
|
||||
await expect(playTab).toBeVisible({ timeout: 30_000 });
|
||||
// Play view is the default tab → its playground input proves the studio body mounted.
|
||||
// NOTE: the per-lane `play-lane` buttons only render AFTER a preview-compression run
|
||||
// populates `batch.lanes` (usePreviewCompression keeps `batch` null until `run()` is
|
||||
// called — there is no mount auto-run). This smoke test intentionally does not drive a
|
||||
// compression cascade (see the "Out of scope" note above), so asserting `play-lane`
|
||||
// here can never become visible. Anchor on the always-present input panel instead.
|
||||
await expect(page.locator('[data-testid="play-input"]')).toBeVisible({
|
||||
timeout: 30_000,
|
||||
});
|
||||
});
|
||||
|
||||
test("switches from Play to Compare", async ({ page }) => {
|
||||
await gotoDashboardRoute(page, "/dashboard/compression/studio");
|
||||
|
||||
const compareTab = page.locator('[data-testid="tab-compare"]');
|
||||
await expect(compareTab).toBeVisible({ timeout: 30_000 });
|
||||
await compareTab.click();
|
||||
await expect(compareTab).toHaveAttribute("aria-pressed", "true");
|
||||
// Compare view mounted → its load control is present.
|
||||
await expect(page.locator('[data-testid="compare-load"]').first()).toBeVisible({
|
||||
timeout: 30_000,
|
||||
});
|
||||
});
|
||||
});
|
||||
@@ -3,7 +3,7 @@ import assert from "node:assert/strict";
|
||||
|
||||
const API_KEY = process.env.OMNIROUTE_API_KEY;
|
||||
const BASE_URL = process.env.OMNIROUTE_URL || "http://localhost:20128";
|
||||
const MODEL = process.env.TEST_GEMINI_MODEL || "default";
|
||||
const MODEL = "default";
|
||||
|
||||
const skip = !API_KEY ? "OMNIROUTE_API_KEY not set — skipping live test" : undefined;
|
||||
|
||||
@@ -54,7 +54,14 @@ async function readSSEStream(response: Response, onChunk?: (chunk: string) => vo
|
||||
}
|
||||
|
||||
test("live request returns streamChunks", { skip }, async () => {
|
||||
console.log("[TEST] BASE_URL=", BASE_URL, "OMNIROUTE_URL=", process.env.OMNIROUTE_URL, "API_KEY set=", !!API_KEY);
|
||||
console.log(
|
||||
"[TEST] BASE_URL=",
|
||||
BASE_URL,
|
||||
"OMNIROUTE_URL=",
|
||||
process.env.OMNIROUTE_URL,
|
||||
"API_KEY set=",
|
||||
!!API_KEY
|
||||
);
|
||||
|
||||
const messages = [
|
||||
{ role: "system", content: "Execute the user prompt and provide a detailed explanation." },
|
||||
@@ -106,9 +113,10 @@ test("live request returns streamChunks", { skip }, async () => {
|
||||
);
|
||||
if (activeRequestResponse.ok) {
|
||||
let activeRequest = await activeRequestResponse.json();
|
||||
if (activeRequest.active &&
|
||||
Array.isArray(activeRequest.pipelinePayloads.streamChunks.provider) &&
|
||||
activeRequest.pipelinePayloads.streamChunks.provider.length > 0
|
||||
if (
|
||||
activeRequest.active &&
|
||||
Array.isArray(activeRequest.pipelinePayloads.streamChunks.provider) &&
|
||||
activeRequest.pipelinePayloads.streamChunks.provider.length > 0
|
||||
) {
|
||||
console.log("Stream chunks:", activeRequest.pipelinePayloads.streamChunks.provider);
|
||||
sawLogChunksWhileStreaming = true;
|
||||
@@ -140,9 +148,13 @@ test("live request returns streamChunks", { skip }, async () => {
|
||||
let finishedRequest = await logDetailResponse.json();
|
||||
|
||||
assert.equal(finishedRequest.id, requestId, "log detail id should match request id");
|
||||
assert.equal(finishedRequest.active, false, "request should be marked as inactive after completion");
|
||||
assert.equal(
|
||||
finishedRequest.active,
|
||||
false,
|
||||
"request should be marked as inactive after completion"
|
||||
);
|
||||
|
||||
assert.ok(Array.isArray(finishedRequest.pipelinePayloads.streamChunks.provider) );
|
||||
assert.ok(Array.isArray(finishedRequest.pipelinePayloads.streamChunks.provider));
|
||||
assert.ok(Array.isArray(finishedRequest.pipelinePayloads.streamChunks.client));
|
||||
|
||||
assert.ok(finishedRequest.pipelinePayloads.streamChunks.provider.length > 0);
|
||||
|
||||
@@ -13,6 +13,19 @@ const REPO_ROOT = fileURLToPath(new URL("../..", import.meta.url));
|
||||
const RELAY_PORT = await getFreePort();
|
||||
const SERVER_PORT = await getFreePort();
|
||||
|
||||
type FileUploadResponse = {
|
||||
id?: string;
|
||||
};
|
||||
|
||||
type BatchResponse = {
|
||||
id?: string;
|
||||
status?: string;
|
||||
request_counts?: {
|
||||
completed?: number;
|
||||
failed?: number;
|
||||
};
|
||||
};
|
||||
|
||||
function getFreePort() {
|
||||
return new Promise<number>((resolve, reject) => {
|
||||
const server = net.createServer();
|
||||
@@ -34,6 +47,47 @@ function sleep(ms: number) {
|
||||
return new Promise((r) => setTimeout(r, ms));
|
||||
}
|
||||
|
||||
function summarizeText(text: string, maxLength = 800) {
|
||||
const compact = text.replace(/\s+/g, " ").trim();
|
||||
return compact.length > maxLength ? `${compact.slice(0, maxLength)}...` : compact;
|
||||
}
|
||||
|
||||
function formatServerTail(proc: ReturnType<typeof createServerProcess>) {
|
||||
return [
|
||||
"--- stdout ---",
|
||||
...proc.stdoutLines.slice(-40),
|
||||
"--- stderr ---",
|
||||
...proc.stderrLines.slice(-40),
|
||||
].join("\n");
|
||||
}
|
||||
|
||||
async function readJsonForTest<T>(
|
||||
response: Response,
|
||||
label: string,
|
||||
proc: ReturnType<typeof createServerProcess>
|
||||
): Promise<T> {
|
||||
const text = await response.text();
|
||||
let body: T;
|
||||
try {
|
||||
body = JSON.parse(text) as T;
|
||||
} catch {
|
||||
throw new Error(
|
||||
[
|
||||
`${label} returned invalid JSON (${response.status} ${response.statusText}, content-type=${response.headers.get("content-type") || "unknown"})`,
|
||||
summarizeText(text),
|
||||
formatServerTail(proc),
|
||||
].join("\n")
|
||||
);
|
||||
}
|
||||
|
||||
assert.equal(
|
||||
response.status,
|
||||
200,
|
||||
`${label} failed (${response.status}): ${JSON.stringify(body)}`
|
||||
);
|
||||
return body;
|
||||
}
|
||||
|
||||
/* ---------- Fake embedding relay ---------- */
|
||||
function createFakeEmbeddingRelay() {
|
||||
let requestCount = 0;
|
||||
@@ -162,24 +216,29 @@ function createServerProcess() {
|
||||
|
||||
async function waitForServer(baseUrl: string, proc: ReturnType<typeof createServerProcess>) {
|
||||
const startedAt = Date.now();
|
||||
while (Date.now() - startedAt < 120_000) {
|
||||
const readinessTimeoutMs = 240_000;
|
||||
const probeTimeoutMs = 15_000;
|
||||
let lastReadiness = "";
|
||||
while (Date.now() - startedAt < readinessTimeoutMs) {
|
||||
if (proc.exitInfo) {
|
||||
throw new Error(
|
||||
[
|
||||
`Server exited early (code=${proc.exitInfo.code}, signal=${proc.exitInfo.signal})`,
|
||||
"--- stdout ---",
|
||||
...proc.stdoutLines.slice(-40),
|
||||
"--- stderr ---",
|
||||
...proc.stderrLines.slice(-40),
|
||||
formatServerTail(proc),
|
||||
].join("\n")
|
||||
);
|
||||
}
|
||||
try {
|
||||
const resp = await fetch(`${baseUrl}/api/monitoring/health`, {
|
||||
signal: AbortSignal.timeout(5_000),
|
||||
});
|
||||
if (resp.ok) return;
|
||||
} catch {
|
||||
for (const readinessPath of ["/api/health/ping", "/api/monitoring/health"]) {
|
||||
const resp = await fetch(`${baseUrl}${readinessPath}`, {
|
||||
signal: AbortSignal.timeout(probeTimeoutMs),
|
||||
});
|
||||
if (resp.ok) return;
|
||||
const body = await resp.text().catch(() => "");
|
||||
lastReadiness = `${readinessPath} -> ${resp.status}: ${summarizeText(body, 200)}`;
|
||||
}
|
||||
} catch (error) {
|
||||
lastReadiness = error instanceof Error ? error.message : String(error);
|
||||
// not ready yet
|
||||
}
|
||||
await sleep(500);
|
||||
@@ -187,10 +246,8 @@ async function waitForServer(baseUrl: string, proc: ReturnType<typeof createServ
|
||||
throw new Error(
|
||||
[
|
||||
"Timed out waiting for server",
|
||||
"--- stdout ---",
|
||||
...proc.stdoutLines.slice(-40),
|
||||
"--- stderr ---",
|
||||
...proc.stderrLines.slice(-40),
|
||||
`Last readiness probe: ${lastReadiness}`,
|
||||
formatServerTail(proc),
|
||||
].join("\n")
|
||||
);
|
||||
}
|
||||
@@ -208,6 +265,18 @@ async function stopProcess(child: ReturnType<typeof spawn>) {
|
||||
}
|
||||
}
|
||||
|
||||
async function removeDirWithRetry(dir: string) {
|
||||
for (let attempt = 0; attempt < 5; attempt++) {
|
||||
try {
|
||||
fs.rmSync(dir, { recursive: true, force: true });
|
||||
return;
|
||||
} catch (error) {
|
||||
if (attempt === 4) throw error;
|
||||
await sleep(250);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/* ---------- Test ---------- */
|
||||
const relay = createFakeEmbeddingRelay();
|
||||
let app: ReturnType<typeof createServerProcess>;
|
||||
@@ -248,7 +317,7 @@ test.after(async () => {
|
||||
await relay.stop();
|
||||
} catch {}
|
||||
if (fs.existsSync(TEST_DATA_DIR)) {
|
||||
fs.rmSync(TEST_DATA_DIR, { recursive: true, force: true });
|
||||
await removeDirWithRetry(TEST_DATA_DIR);
|
||||
}
|
||||
});
|
||||
|
||||
@@ -281,9 +350,12 @@ test("batch E2E: upload file, create batch, verify rate-limit logs appear", asyn
|
||||
method: "POST",
|
||||
body: formData,
|
||||
});
|
||||
const uploadText = await uploadResp.text();
|
||||
assert.equal(uploadResp.status, 200, `File upload failed (${uploadResp.status}): ${uploadText}`);
|
||||
const uploadBody = JSON.parse(uploadText);
|
||||
assert.match(
|
||||
uploadResp.headers.get("content-type") || "",
|
||||
/json/i,
|
||||
"File upload should return JSON"
|
||||
);
|
||||
const uploadBody = await readJsonForTest<FileUploadResponse>(uploadResp, "File upload", app);
|
||||
const fileId = uploadBody.id;
|
||||
assert.ok(fileId, "file id missing from upload response");
|
||||
|
||||
@@ -297,22 +369,35 @@ test("batch E2E: upload file, create batch, verify rate-limit logs appear", asyn
|
||||
completion_window: "24h",
|
||||
}),
|
||||
});
|
||||
const batchText = await batchResp.text();
|
||||
assert.equal(batchResp.status, 200, `Batch creation failed (${batchResp.status}): ${batchText}`);
|
||||
const batchBody = JSON.parse(batchText);
|
||||
const batchBody = await readJsonForTest<BatchResponse>(batchResp, "Batch creation", app);
|
||||
const batchId = batchBody.id;
|
||||
assert.ok(batchId, "batch id missing from create response");
|
||||
|
||||
// 3. Poll for batch completion
|
||||
let batchStatus = "";
|
||||
let attempts = 0;
|
||||
let lastPollSummary = "";
|
||||
const maxAttempts = 120;
|
||||
while (attempts < maxAttempts) {
|
||||
await sleep(2_000);
|
||||
attempts++;
|
||||
const sr = await fetch(`${app.baseUrl}/api/v1/batches/${batchId}`);
|
||||
const sb = (await sr.json()) as any;
|
||||
batchStatus = sb.status;
|
||||
const text = await sr.text();
|
||||
let sb: BatchResponse;
|
||||
try {
|
||||
sb = JSON.parse(text);
|
||||
} catch {
|
||||
lastPollSummary = `poll ${attempts} returned invalid JSON (${sr.status} ${sr.statusText}, content-type=${sr.headers.get("content-type") || "unknown"}): ${summarizeText(text, 300)}`;
|
||||
console.warn(`[poll ${attempts}] ${lastPollSummary}`);
|
||||
continue;
|
||||
}
|
||||
if (!sr.ok) {
|
||||
lastPollSummary = `poll ${attempts} failed (${sr.status} ${sr.statusText}): ${JSON.stringify(sb)}`;
|
||||
console.warn(`[poll ${attempts}] ${lastPollSummary}`);
|
||||
continue;
|
||||
}
|
||||
batchStatus = sb.status || "";
|
||||
lastPollSummary = `poll ${attempts} status=${batchStatus}`;
|
||||
console.log(
|
||||
`[poll ${attempts}] batch ${batchId} status=${batchStatus} completed=${sb.request_counts?.completed} failed=${sb.request_counts?.failed}`
|
||||
);
|
||||
@@ -322,6 +407,7 @@ test("batch E2E: upload file, create batch, verify rate-limit logs appear", asyn
|
||||
batchStatus,
|
||||
"completed",
|
||||
`Batch did not complete; final status: ${batchStatus}. ` +
|
||||
`Last poll: ${lastPollSummary}\n` +
|
||||
`Server [BATCH] logs:\n${[...app.stdoutLines, ...app.stderrLines].filter((l) => l.includes("[BATCH]")).join("\n")}`
|
||||
);
|
||||
|
||||
@@ -348,7 +434,7 @@ test("batch E2E: upload file, create batch, verify rate-limit logs appear", asyn
|
||||
|
||||
// 5. Verify batch results
|
||||
const finalResp = await fetch(`${app.baseUrl}/api/v1/batches/${batchId}`);
|
||||
const finalBody = (await finalResp.json()) as any;
|
||||
const finalBody = await readJsonForTest<BatchResponse>(finalResp, "Final batch fetch", app);
|
||||
assert.equal(
|
||||
finalBody.request_counts?.completed,
|
||||
2,
|
||||
|
||||
@@ -8,7 +8,6 @@
|
||||
* Uses same env vars as tests/manual/gemini.http:
|
||||
* OMNIROUTE_URL — base URL (default http://localhost:20128)
|
||||
* OMNIROUTE_API_KEY — API key for auth
|
||||
* TEST_GEMINI_MODEL — model override (default gemini/gemma-4-31b-it)
|
||||
* TEST_THINKING_GEMINI_MODEL — thinking model override, skipped if unset
|
||||
*/
|
||||
|
||||
@@ -17,7 +16,7 @@ import assert from "node:assert/strict";
|
||||
|
||||
const API_KEY = process.env.OMNIROUTE_API_KEY;
|
||||
const BASE_URL = process.env.OMNIROUTE_URL || "http://localhost:20128";
|
||||
const MODEL = process.env.TEST_GEMINI_MODEL || "gemini/gemma-4-31b-it";
|
||||
const MODEL = "default";
|
||||
const THINKING_MODEL = process.env.TEST_THINKING_GEMINI_MODEL || "gemini/gemini-2.5-flash";
|
||||
const NUM_HISTORICAL_ROUNDS = 15;
|
||||
|
||||
|
||||
246
tests/integration/live-gemini-nonstream.test.ts
Normal file
246
tests/integration/live-gemini-nonstream.test.ts
Normal file
@@ -0,0 +1,246 @@
|
||||
/**
|
||||
* tests/integration/live-gemini-nonstream.test.ts
|
||||
*
|
||||
* Non-streaming variant of live-gemini-workload.test.ts.
|
||||
* Reuses the same CASE_BUILDERS payload generators but sends stream: false.
|
||||
* Validates that non-streaming responses return content and complete without errors.
|
||||
*/
|
||||
import test from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
|
||||
import {
|
||||
skip,
|
||||
randomInt,
|
||||
sendAndValidate,
|
||||
CASE_BUILDERS,
|
||||
ensureTestEnvironment,
|
||||
DELAY_BETWEEN_REQUESTS_MS,
|
||||
type Message,
|
||||
} from "./liveGeminiShared.ts";
|
||||
|
||||
test.before(async () => {
|
||||
await ensureTestEnvironment();
|
||||
});
|
||||
|
||||
// ── Non-streaming concurrent load — 5 parallel threads × 5 iterations ──
|
||||
|
||||
test("[00] non-streaming: concurrent load — 5 threads × 2 iterations", { skip }, async () => {
|
||||
const THREAD_COUNT = 5;
|
||||
const SET_COUNT = 2;
|
||||
const TOTAL_REQUESTS = THREAD_COUNT * SET_COUNT;
|
||||
|
||||
console.log(
|
||||
`\n Non-streaming concurrent: ${THREAD_COUNT} threads × ${SET_COUNT} iterations = ${TOTAL_REQUESTS} requests`
|
||||
);
|
||||
|
||||
const start = performance.now();
|
||||
|
||||
const requestWindows: { cid: string; start: number; end: number }[] = [];
|
||||
let parallelViolation: string | null = null;
|
||||
|
||||
const threadResults = await Promise.allSettled(
|
||||
Array.from({ length: THREAD_COUNT }, (_, threadIdx) =>
|
||||
(async () => {
|
||||
const results: {
|
||||
status: number;
|
||||
duration: number;
|
||||
tokens: number;
|
||||
contentLength: number;
|
||||
correlationId: string;
|
||||
}[] = [];
|
||||
for (let set = 1; set <= SET_COUNT; set++) {
|
||||
if (parallelViolation) break;
|
||||
|
||||
const idx = randomInt(0, CASE_BUILDERS.length - 1);
|
||||
const tc = CASE_BUILDERS[idx];
|
||||
const label = `ns-t${threadIdx + 1}-i${set}: ${tc.name}`;
|
||||
const requestStart = Date.now();
|
||||
const r = await sendAndValidate(label, tc.build, false);
|
||||
const requestEnd = Date.now();
|
||||
|
||||
const cid = r.correlationId;
|
||||
const myWindow = { cid, start: requestStart, end: requestEnd };
|
||||
const siblings = requestWindows.filter((w) => w.cid === cid);
|
||||
for (const s of siblings) {
|
||||
if (myWindow.start < s.end && s.start < myWindow.end) {
|
||||
parallelViolation =
|
||||
`PARALLEL REQUEST DETECTED: cid=${cid.slice(0, 12)}… ` +
|
||||
`window [${new Date(myWindow.start).toISOString().slice(11, 23)}–${new Date(myWindow.end).toISOString().slice(11, 23)}] ` +
|
||||
`overlaps with [${new Date(s.start).toISOString().slice(11, 23)}–${new Date(s.end).toISOString().slice(11, 23)}]`;
|
||||
break;
|
||||
}
|
||||
}
|
||||
requestWindows.push(myWindow);
|
||||
results.push({ ...r } as any);
|
||||
}
|
||||
return results;
|
||||
})()
|
||||
)
|
||||
);
|
||||
|
||||
const totalDuration = performance.now() - start;
|
||||
|
||||
const fulfilled = threadResults.filter((r) => r.status === "fulfilled") as PromiseFulfilledResult<
|
||||
{
|
||||
status: number;
|
||||
duration: number;
|
||||
tokens: number;
|
||||
contentLength: number;
|
||||
correlationId: string;
|
||||
}[]
|
||||
>[];
|
||||
const rejected = threadResults.filter((r) => r.status === "rejected") as PromiseRejectedResult[];
|
||||
|
||||
const allResults = fulfilled.flatMap((r) => r.value);
|
||||
const totalTokens = allResults.reduce((sum, r) => sum + r.tokens, 0);
|
||||
const avgDuration =
|
||||
allResults.length > 0
|
||||
? Math.round(allResults.reduce((s, r) => s + r.duration, 0) / allResults.length)
|
||||
: 0;
|
||||
|
||||
console.log(
|
||||
`\n Non-streaming concurrent summary: ${fulfilled.length}/${THREAD_COUNT} threads completed | ` +
|
||||
`${allResults.length}/${TOTAL_REQUESTS} requests succeeded | ` +
|
||||
`${Math.round(totalDuration)}ms wall clock | ` +
|
||||
`${avgDuration}ms avg per request | ` +
|
||||
`${totalTokens} total tokens`
|
||||
);
|
||||
|
||||
if (rejected.length > 0) {
|
||||
for (const r of rejected) {
|
||||
const msg = r.reason instanceof Error ? r.message : String(r.reason);
|
||||
console.log(` THREAD FAILED: ${msg}`);
|
||||
}
|
||||
}
|
||||
|
||||
if (parallelViolation) {
|
||||
console.log(`\n !! ${parallelViolation}`);
|
||||
}
|
||||
|
||||
assert.ok(
|
||||
fulfilled.length === THREAD_COUNT,
|
||||
`expected all ${THREAD_COUNT} threads to complete, ${rejected.length} failed`
|
||||
);
|
||||
assert.ok(
|
||||
allResults.length === TOTAL_REQUESTS,
|
||||
`expected ${TOTAL_REQUESTS} total requests, got ${allResults.length}`
|
||||
);
|
||||
assert.ok(!parallelViolation, parallelViolation);
|
||||
|
||||
// Verify all correlation IDs are unique
|
||||
const cids = allResults.map((r) => r.correlationId);
|
||||
const uniqueCids = new Set(cids);
|
||||
assert.equal(
|
||||
uniqueCids.size,
|
||||
cids.length,
|
||||
`expected ${cids.length} unique CIDs, got ${uniqueCids.size}`
|
||||
);
|
||||
});
|
||||
|
||||
// ── Non-streaming sequential test ───────────────────────────────────────
|
||||
|
||||
test("[01] non-streaming: sequential — 1 thread × 5 iterations", { skip }, async () => {
|
||||
const SET_COUNT = 5;
|
||||
|
||||
console.log(`\n Non-streaming sequential: 1 thread × ${SET_COUNT} iterations`);
|
||||
|
||||
const start = performance.now();
|
||||
const results: {
|
||||
status: number;
|
||||
duration: number;
|
||||
tokens: number;
|
||||
contentLength: number;
|
||||
correlationId: string;
|
||||
}[] = [];
|
||||
|
||||
for (let i = 1; i <= SET_COUNT; i++) {
|
||||
const idx = randomInt(0, CASE_BUILDERS.length - 1);
|
||||
const tc = CASE_BUILDERS[idx];
|
||||
const label = `ns-i${i}: ${tc.name}`;
|
||||
try {
|
||||
if (i > 1) await new Promise((r) => setTimeout(r, DELAY_BETWEEN_REQUESTS_MS));
|
||||
const r = await sendAndValidate(label, tc.build, false);
|
||||
results.push(r);
|
||||
} catch (err) {
|
||||
const msg = err instanceof Error ? err.message : String(err);
|
||||
console.log(` ${label.padEnd(45)} FAILED: ${msg}`);
|
||||
assert.fail(`iteration ${i} failed: ${msg}`);
|
||||
}
|
||||
}
|
||||
|
||||
const totalDuration = performance.now() - start;
|
||||
const totalTokens = results.reduce((sum, r) => sum + r.tokens, 0);
|
||||
const avgDuration =
|
||||
results.length > 0
|
||||
? Math.round(results.reduce((s, r) => s + r.duration, 0) / results.length)
|
||||
: 0;
|
||||
|
||||
console.log(
|
||||
`\n Non-streaming summary: ${results.length}/${SET_COUNT} succeeded | ` +
|
||||
`${Math.round(totalDuration)}ms wall clock | ` +
|
||||
`${avgDuration}ms avg per request | ` +
|
||||
`${totalTokens} total tokens`
|
||||
);
|
||||
|
||||
const cids = results.map((r) => r.correlationId);
|
||||
const uniqueCids = new Set(cids);
|
||||
assert.equal(
|
||||
uniqueCids.size,
|
||||
cids.length,
|
||||
`expected ${cids.length} unique CIDs, got ${uniqueCids.size}`
|
||||
);
|
||||
assert.equal(results.length, SET_COUNT, `expected ${SET_COUNT} results, got ${results.length}`);
|
||||
});
|
||||
|
||||
// ── Non-streaming: all payloads return content ──────────────────────────
|
||||
|
||||
test("[02] non-streaming: all payloads return content", { skip }, async () => {
|
||||
const failures: string[] = [];
|
||||
|
||||
for (let i = 0; i < CASE_BUILDERS.length; i++) {
|
||||
const tc = CASE_BUILDERS[i];
|
||||
const label = `ns-${String(i + 1).padStart(2, "0")}: ${tc.name}`;
|
||||
try {
|
||||
const r = await sendAndValidate(label, tc.build, false);
|
||||
if (r.contentLength === 0) {
|
||||
failures.push(`${tc.name}: 0 bytes content`);
|
||||
}
|
||||
if (r.status !== 200) {
|
||||
failures.push(`${tc.name}: HTTP ${r.status}`);
|
||||
}
|
||||
} catch (err) {
|
||||
failures.push(`${tc.name}: ${err instanceof Error ? err.message : String(err)}`);
|
||||
}
|
||||
}
|
||||
|
||||
if (failures.length > 0) {
|
||||
console.log(`\n Non-streaming failures (${failures.length}):`);
|
||||
for (const f of failures) console.log(` ${f}`);
|
||||
}
|
||||
|
||||
assert.equal(
|
||||
failures.length,
|
||||
0,
|
||||
`${failures.length}/${CASE_BUILDERS.length} non-streaming payloads failed`
|
||||
);
|
||||
});
|
||||
|
||||
// ── Non-streaming: correlation IDs are unique ───────────────────────────
|
||||
|
||||
test("[03] non-streaming: correlation IDs are unique per request", { skip }, async () => {
|
||||
const cids: string[] = [];
|
||||
const count = 5;
|
||||
|
||||
for (let i = 0; i < count; i++) {
|
||||
const tc = CASE_BUILDERS[i % CASE_BUILDERS.length];
|
||||
const r = await sendAndValidate(`ns-cid-${i + 1}: ${tc.name}`, tc.build, false);
|
||||
cids.push(r.correlationId);
|
||||
}
|
||||
|
||||
const unique = new Set(cids);
|
||||
assert.equal(
|
||||
unique.size,
|
||||
count,
|
||||
`expected ${count} unique CIDs, got ${unique.size}: ${cids.join(", ")}`
|
||||
);
|
||||
});
|
||||
File diff suppressed because it is too large
Load Diff
188
tests/integration/live-gemini.test.ts
Normal file
188
tests/integration/live-gemini.test.ts
Normal file
@@ -0,0 +1,188 @@
|
||||
import test from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
|
||||
import { ensureTestEnvironment, MODEL, BASE_URL, API_KEY } from "./liveGeminiShared.ts";
|
||||
|
||||
const DIRECT_MODEL = process.env.TEST_GEMINI_DIRECT_MODEL || "gemini/gemini-2.0-flash";
|
||||
|
||||
const skip = !API_KEY ? "OMNIROUTE_API_KEY not set — skipping live test" : undefined;
|
||||
|
||||
test.before(async () => {
|
||||
await ensureTestEnvironment();
|
||||
});
|
||||
|
||||
async function readSSEStream(response: Response): Promise<{
|
||||
fullContent: string;
|
||||
finishReason: string;
|
||||
model: string;
|
||||
totalTokens: number;
|
||||
}> {
|
||||
const reader = response.body!.getReader();
|
||||
const decoder = new TextDecoder();
|
||||
let buffer = "";
|
||||
let fullContent = "";
|
||||
let finishReason = "unknown";
|
||||
let model = "";
|
||||
let totalTokens = 0;
|
||||
let chunkCount = 0;
|
||||
let sampleLines: string[] = [];
|
||||
|
||||
while (true) {
|
||||
const { done, value } = await reader.read();
|
||||
if (done) break;
|
||||
|
||||
buffer += decoder.decode(value, { stream: true });
|
||||
const lines = buffer.split("\n");
|
||||
buffer = lines.pop() || "";
|
||||
|
||||
for (const line of lines) {
|
||||
if (!line.startsWith("data: ")) continue;
|
||||
chunkCount++;
|
||||
const data = line.slice(6).trim();
|
||||
if (data === "[DONE]") continue;
|
||||
|
||||
try {
|
||||
const parsed = JSON.parse(data) as Record<string, unknown>;
|
||||
const choice = ((parsed.choices ?? []) as Array<Record<string, unknown>>)[0];
|
||||
if (choice) {
|
||||
const delta = choice.delta as Record<string, unknown> | undefined;
|
||||
if (delta?.content) {
|
||||
fullContent += delta.content as string;
|
||||
} else if (delta?.reasoning_content) {
|
||||
fullContent += delta.reasoning_content as string;
|
||||
}
|
||||
if (choice.finish_reason) finishReason = choice.finish_reason as string;
|
||||
}
|
||||
if (!model) {
|
||||
if (parsed.model) {
|
||||
model = parsed.model as string;
|
||||
} else if (choice?.model) {
|
||||
model = choice.model as string;
|
||||
}
|
||||
}
|
||||
const usage = parsed.usage as Record<string, number> | undefined;
|
||||
if (usage) {
|
||||
totalTokens =
|
||||
usage.total_tokens ?? (usage.prompt_tokens ?? 0) + (usage.completion_tokens ?? 0);
|
||||
}
|
||||
} catch {
|
||||
// skip malformed chunks
|
||||
}
|
||||
|
||||
if (sampleLines.length < 3) {
|
||||
sampleLines.push(data.slice(0, 200));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
console.log(
|
||||
`[SSE] total raw chunks: ${chunkCount}, content length: ${fullContent.length}, finish: ${finishReason}, model: ${model}, tokens: ${totalTokens}`
|
||||
);
|
||||
if (sampleLines.length > 0) {
|
||||
console.log(`[SSE] sample lines: ${JSON.stringify(sampleLines)}`);
|
||||
}
|
||||
|
||||
return { fullContent, finishReason, model, totalTokens };
|
||||
}
|
||||
|
||||
test("live Gemini — single hello-world request via combo 'default'", { skip }, async (t) => {
|
||||
const controller = new AbortController();
|
||||
const timeout = setTimeout(() => controller.abort(), 120_000);
|
||||
|
||||
try {
|
||||
console.log(`[TEST] Sending request with model=${MODEL} to ${BASE_URL}/v1/chat/completions`);
|
||||
const response = await fetch(`${BASE_URL}/v1/chat/completions`, {
|
||||
method: "POST",
|
||||
headers: {
|
||||
"Content-Type": "application/json",
|
||||
Authorization: `Bearer ${API_KEY}`,
|
||||
},
|
||||
body: JSON.stringify({
|
||||
model: MODEL,
|
||||
messages: [{ role: "user", content: "Say 'Hello world' and nothing else." }],
|
||||
stream: true,
|
||||
max_tokens: 50,
|
||||
temperature: 0,
|
||||
}),
|
||||
signal: controller.signal,
|
||||
});
|
||||
clearTimeout(timeout);
|
||||
|
||||
console.log(`[TEST] Response status: ${response.status}`);
|
||||
if (!response.ok) {
|
||||
const body = await response.text().catch(() => "unknown");
|
||||
console.log(`[TEST] Response body: ${body}`);
|
||||
}
|
||||
|
||||
assert.equal(response.status, 200, "Expected HTTP 200 from combo request");
|
||||
|
||||
const { fullContent, finishReason, model, totalTokens } = await readSSEStream(response);
|
||||
|
||||
assert.ok(fullContent.length > 0, "response should have content");
|
||||
assert.ok(
|
||||
finishReason === "stop" || finishReason === "length",
|
||||
`expected stop/length finish, got ${finishReason}`
|
||||
);
|
||||
assert.ok(totalTokens > 0, `should have non-zero token count, got ${totalTokens}`);
|
||||
assert.ok(
|
||||
model.toLowerCase().includes("gemini") || model.toLowerCase().includes("gemma"),
|
||||
`response model "${model}" should be a Gemini/Gemma model`
|
||||
);
|
||||
console.log(
|
||||
`[TEST] OK: model=${model}, finish=${finishReason}, tokens=${totalTokens}, content=${fullContent.length} chars`
|
||||
);
|
||||
} catch (err) {
|
||||
clearTimeout(timeout);
|
||||
throw err;
|
||||
}
|
||||
});
|
||||
|
||||
test("live Gemini — direct model request (skip combo)", { skip }, async (t) => {
|
||||
const controller = new AbortController();
|
||||
const timeout = setTimeout(() => controller.abort(), 120_000);
|
||||
|
||||
try {
|
||||
console.log(`[TEST] Direct model request with model=${DIRECT_MODEL}`);
|
||||
const response = await fetch(`${BASE_URL}/v1/chat/completions`, {
|
||||
method: "POST",
|
||||
headers: {
|
||||
"Content-Type": "application/json",
|
||||
Authorization: `Bearer ${API_KEY}`,
|
||||
},
|
||||
body: JSON.stringify({
|
||||
model: DIRECT_MODEL,
|
||||
messages: [{ role: "user", content: "Say 'Hello world' and nothing else." }],
|
||||
stream: true,
|
||||
max_tokens: 50,
|
||||
temperature: 0,
|
||||
}),
|
||||
signal: controller.signal,
|
||||
});
|
||||
clearTimeout(timeout);
|
||||
|
||||
console.log(`[TEST] Direct response status: ${response.status}`);
|
||||
if (!response.ok) {
|
||||
const body = await response.text().catch(() => "unknown");
|
||||
console.log(`[TEST] Direct response body: ${body}`);
|
||||
}
|
||||
|
||||
assert.equal(response.status, 200, "Expected HTTP 200 for direct model");
|
||||
|
||||
const { fullContent, finishReason, model, totalTokens } = await readSSEStream(response);
|
||||
|
||||
assert.ok(fullContent.length > 0, "response should have content");
|
||||
assert.ok(
|
||||
finishReason === "stop" || finishReason === "length",
|
||||
`expected stop/length finish, got ${finishReason}`
|
||||
);
|
||||
assert.ok(totalTokens > 0, `should have non-zero token count, got ${totalTokens}`);
|
||||
assert.ok(
|
||||
model.toLowerCase().includes("gemini") || model.toLowerCase().includes("gemma"),
|
||||
`response model "${model}" should be a Gemini/Gemma model`
|
||||
);
|
||||
console.log(`[TEST] Direct OK: model=${model}, finish=${finishReason}, tokens=${totalTokens}`);
|
||||
} catch (err) {
|
||||
clearTimeout(timeout);
|
||||
throw err;
|
||||
}
|
||||
});
|
||||
1108
tests/integration/liveGeminiShared.ts
Normal file
1108
tests/integration/liveGeminiShared.ts
Normal file
File diff suppressed because it is too large
Load Diff
@@ -27,6 +27,7 @@ process.env.API_KEY_SECRET = "test-secret-qdrant-routes";
|
||||
|
||||
const core = await import("../../src/lib/db/core.ts");
|
||||
const localDb = await import("../../src/lib/localDb.ts");
|
||||
const memorySettings = await import("../../src/lib/memory/settings.ts");
|
||||
|
||||
// ── Route imports ──
|
||||
const qdrantSettingsRoute = await import("../../src/app/api/settings/qdrant/route.ts");
|
||||
@@ -49,6 +50,9 @@ async function resetStorage() {
|
||||
core.resetDbInstance();
|
||||
fs.rmSync(TEST_DATA_DIR, { recursive: true, force: true });
|
||||
fs.mkdirSync(TEST_DATA_DIR, { recursive: true });
|
||||
// #5597 follow-up: the memory-settings cache is a module-level singleton that
|
||||
// survives per-test DB resets — bust it so each test starts from a clean read.
|
||||
memorySettings.invalidateMemorySettingsCache();
|
||||
}
|
||||
|
||||
async function makeAuthRequest(
|
||||
@@ -143,6 +147,99 @@ test("PUT /api/settings/qdrant — updates settings and returns new masked shape
|
||||
assert.strictEqual(body.apiKey, undefined, "raw apiKey must not be in response");
|
||||
});
|
||||
|
||||
// ── #5597: enabling Qdrant must also activate it as the engine ──
|
||||
// Regression: retrieval only routes to Qdrant when memoryVectorStore === "qdrant"
|
||||
// (retrieval.ts:342/470/694). The card only wrote `qdrantEnabled` and never the
|
||||
// engine selector, so enabling Qdrant was inert — it stayed on the default "auto"
|
||||
// (which never selects Qdrant). Enabling now also sets memoryVectorStore=qdrant.
|
||||
|
||||
test("PUT enabled=true also activates Qdrant as the engine (memoryVectorStore=qdrant)", async () => {
|
||||
const req = await makeAuthRequest("PUT", "http://localhost/api/settings/qdrant", {
|
||||
enabled: true,
|
||||
host: "qdrant-server",
|
||||
collection: "c",
|
||||
});
|
||||
const res = await qdrantSettingsRoute.PUT(req as any);
|
||||
assert.strictEqual(res.status, 200);
|
||||
|
||||
const s = (await localDb.getSettings()) as Record<string, unknown>;
|
||||
assert.strictEqual(
|
||||
s.memoryVectorStore,
|
||||
"qdrant",
|
||||
"enabling Qdrant must select it as the active vector store, else it stays inert"
|
||||
);
|
||||
});
|
||||
|
||||
test("PUT enabled=false resets the engine back to auto (sqlite-vec)", async () => {
|
||||
await qdrantSettingsRoute.PUT(
|
||||
(await makeAuthRequest("PUT", "http://localhost/api/settings/qdrant", {
|
||||
enabled: true,
|
||||
host: "qdrant-server",
|
||||
collection: "c",
|
||||
})) as any
|
||||
);
|
||||
await qdrantSettingsRoute.PUT(
|
||||
(await makeAuthRequest("PUT", "http://localhost/api/settings/qdrant", {
|
||||
enabled: false,
|
||||
})) as any
|
||||
);
|
||||
|
||||
const s = (await localDb.getSettings()) as Record<string, unknown>;
|
||||
assert.strictEqual(
|
||||
s.memoryVectorStore,
|
||||
"auto",
|
||||
"disabling Qdrant must fall back to auto (sqlite-vec), not stay on qdrant"
|
||||
);
|
||||
});
|
||||
|
||||
test("PUT without the enabled field must not change memoryVectorStore", async () => {
|
||||
// User already on qdrant; editing only the collection must not reset the engine.
|
||||
await localDb.updateSettings({ memoryVectorStore: "qdrant", qdrantEnabled: true });
|
||||
await qdrantSettingsRoute.PUT(
|
||||
(await makeAuthRequest("PUT", "http://localhost/api/settings/qdrant", {
|
||||
collection: "renamed",
|
||||
})) as any
|
||||
);
|
||||
|
||||
const s = (await localDb.getSettings()) as Record<string, unknown>;
|
||||
assert.strictEqual(
|
||||
s.memoryVectorStore,
|
||||
"qdrant",
|
||||
"editing other fields must leave the engine selection untouched"
|
||||
);
|
||||
});
|
||||
|
||||
// #5597 follow-up: writing memoryVectorStore to the DB is not enough — retrieval reads
|
||||
// through getMemorySettings(), a module-level cache. The PUT handler must invalidate it
|
||||
// so the engine switch takes effect without a process restart.
|
||||
test("PUT enabled=true invalidates the memory-settings cache (retrieval sees qdrant, no restart)", async () => {
|
||||
// Warm the cache with the pre-toggle value (default auto → not qdrant).
|
||||
const before = await memorySettings.getMemorySettings();
|
||||
assert.notStrictEqual(
|
||||
before.vectorStore,
|
||||
"qdrant",
|
||||
"precondition: cache warmed with a non-qdrant vectorStore"
|
||||
);
|
||||
|
||||
const res = await qdrantSettingsRoute.PUT(
|
||||
(await makeAuthRequest("PUT", "http://localhost/api/settings/qdrant", {
|
||||
enabled: true,
|
||||
host: "qdrant-server",
|
||||
collection: "c",
|
||||
})) as any
|
||||
);
|
||||
assert.strictEqual(res.status, 200);
|
||||
|
||||
// Without the cache invalidation, getMemorySettings() would still return the stale
|
||||
// "auto" value and retrieval would keep routing to sqlite-vec until a restart.
|
||||
const after = await memorySettings.getMemorySettings();
|
||||
assert.strictEqual(
|
||||
after.vectorStore,
|
||||
"qdrant",
|
||||
"PUT must invalidate the memory-settings cache so retrieval routes to Qdrant without a restart"
|
||||
);
|
||||
});
|
||||
|
||||
test("PUT /api/settings/qdrant — 400 invalid settings (invalid port type in strict schema)", async () => {
|
||||
const req = await makeAuthRequest("PUT", "http://localhost/api/settings/qdrant", {
|
||||
port: "not-a-number",
|
||||
|
||||
@@ -168,7 +168,20 @@ test("callLogs.ts wires no-log and PII sanitization before persistence", () => {
|
||||
content.includes('from "../compliance"') || content.includes('from "../compliance/noLog"'),
|
||||
"callLogs.ts should import compliance module"
|
||||
);
|
||||
assert.ok(content.includes('from "../piiSanitizer"'), "callLogs.ts should import piiSanitizer");
|
||||
// PII sanitization for error strings was extracted to callLogs/format.ts by #5725
|
||||
// (sanitizeErrorForLog); callLogs.ts still wires it in before persistence, and the
|
||||
// extracted helper keeps the piiSanitizer dependency — so the "sanitize before
|
||||
// persist" invariant holds post-refactor (verified on both the helper and the file).
|
||||
assert.ok(
|
||||
content.includes("sanitizeErrorForLog") && content.includes('from "./callLogs/format"'),
|
||||
"callLogs.ts should wire the extracted PII-sanitizing error helper (sanitizeErrorForLog)"
|
||||
);
|
||||
const formatHelperContent = readIfExists("src/lib/usage/callLogs/format.ts");
|
||||
assert.ok(formatHelperContent, "src/lib/usage/callLogs/format.ts should exist");
|
||||
assert.ok(
|
||||
formatHelperContent.includes('from "../../piiSanitizer"'),
|
||||
"callLogs/format.ts should import piiSanitizer (PII sanitization still wired post-#5725)"
|
||||
);
|
||||
assert.ok(content.includes("isNoLog("), "callLogs.ts should check no-log policy");
|
||||
|
||||
const payloadHelperContent = readIfExists("src/lib/logPayloads.ts");
|
||||
|
||||
@@ -113,3 +113,67 @@ test("stream recovery OFF (default): a truncated stream is NOT retried", async (
|
||||
await readSSE(response);
|
||||
assert.equal(calls, 1, "default path calls upstream exactly once — zero behavior change");
|
||||
});
|
||||
|
||||
test("/goal requests enable early stream recovery even when the global default is OFF", async () => {
|
||||
await h.seedConnection("openai", { apiKey: "sk-openai-primary" });
|
||||
const apiKey = await h.seedApiKey();
|
||||
// No setting, no env → global recovery stays OFF. The /goal detector should opt in.
|
||||
|
||||
let calls = 0;
|
||||
globalThis.fetch = (async () => {
|
||||
calls++;
|
||||
return calls === 1 ? truncatedOpenAIStream() : completeOpenAIStream();
|
||||
}) as typeof fetch;
|
||||
|
||||
const response = await h.handleChat(
|
||||
h.buildRequest({
|
||||
authKey: apiKey.key,
|
||||
body: {
|
||||
model: "openai/gpt-4o-mini",
|
||||
stream: true,
|
||||
messages: [{ role: "user", content: "/goal finish this long-running task" }],
|
||||
},
|
||||
})
|
||||
);
|
||||
|
||||
assert.equal(response.status, 200);
|
||||
const sse = await readSSE(response);
|
||||
assert.equal(calls, 2, "goal mode should re-open an early-truncated stream");
|
||||
assert.match(sse, /RECOVERED/, "client receives the recovered attempt");
|
||||
assert.doesNotMatch(sse, /PARTIAL/, "the discarded first attempt must not leak to the client");
|
||||
});
|
||||
|
||||
test("/goal requests do NOT re-enable stream recovery when the operator explicitly disabled it", async () => {
|
||||
await h.seedConnection("openai", { apiKey: "sk-openai-primary" });
|
||||
const apiKey = await h.seedApiKey();
|
||||
// Operator explicitly turned recovery OFF via a DB/settings override — the
|
||||
// agentGoalPolicy heuristic must be fail-closed and never override this.
|
||||
await h.settingsDb.updateSettings({
|
||||
resilienceSettings: { streamRecovery: { enabled: false } },
|
||||
});
|
||||
|
||||
let calls = 0;
|
||||
globalThis.fetch = (async () => {
|
||||
calls++;
|
||||
return truncatedOpenAIStream();
|
||||
}) as typeof fetch;
|
||||
|
||||
const response = await h.handleChat(
|
||||
h.buildRequest({
|
||||
authKey: apiKey.key,
|
||||
body: {
|
||||
model: "openai/gpt-4o-mini",
|
||||
stream: true,
|
||||
messages: [{ role: "user", content: "/goal finish this long-running task" }],
|
||||
},
|
||||
})
|
||||
);
|
||||
|
||||
assert.equal(response.status, 200);
|
||||
await readSSE(response);
|
||||
assert.equal(
|
||||
calls,
|
||||
1,
|
||||
"explicit operator opt-out must win over the goal-policy heuristic — zero re-open"
|
||||
);
|
||||
});
|
||||
|
||||
@@ -2292,8 +2292,8 @@
|
||||
}
|
||||
},
|
||||
"url": {
|
||||
"nonStream": "https://kimi.moonshot.cn/api/chat",
|
||||
"stream": "https://kimi.moonshot.cn/api/chat"
|
||||
"nonStream": "https://www.kimi.com",
|
||||
"stream": "https://www.kimi.com"
|
||||
}
|
||||
},
|
||||
"kiro": {
|
||||
|
||||
39
tests/unit/_fixtures/tsx-runtime-modern-syntax.ts
Normal file
39
tests/unit/_fixtures/tsx-runtime-modern-syntax.ts
Normal file
@@ -0,0 +1,39 @@
|
||||
/**
|
||||
* #5757 guard fixture — NOT a test file (no `.test.` in the name, so the runner
|
||||
* skips it). Exercised by `tests/unit/tsx-runtime-transform-5757.test.ts`.
|
||||
*
|
||||
* It concentrates the modern JS/TS syntax that the published CLI's runtime
|
||||
* `tsx/esm` loader (`bin/omniroute.mjs` → `await import("tsx/esm")`) must
|
||||
* transform through esbuild at startup: object/array destructuring + rest,
|
||||
* object/array spread, class + private fields, optional chaining, nullish
|
||||
* coalescing, logical assignment, async/await and top-level await.
|
||||
*
|
||||
* If a future esbuild (pulled transitively via `tsx`) cannot transform this on a
|
||||
* supported Node runtime, running this file fails — which is the whole point.
|
||||
*/
|
||||
class Box {
|
||||
value = 41; // public class field
|
||||
#secret = 1; // private field
|
||||
|
||||
bump(): number {
|
||||
this.value += this.#secret;
|
||||
return this.value;
|
||||
}
|
||||
}
|
||||
|
||||
async function main() {
|
||||
const { a, b, ...rest } = { a: 1, b: 2, c: 3, d: 4 }; // object destructuring + rest
|
||||
const [first, ...tail] = [10, 20, 30]; // array destructuring + rest
|
||||
const merged = { ...rest, first }; // object spread
|
||||
const arr = [...tail, first]; // array spread
|
||||
const bumped = new Box().bump(); // class + private field
|
||||
const maybe: { x?: { y?: number } } = {};
|
||||
const opt = maybe?.x?.y ?? 99; // optional chaining + nullish coalescing
|
||||
let acc = 0;
|
||||
acc ||= bumped; // logical assignment
|
||||
const total = await Promise.resolve(a + b + first + opt + acc); // async/await
|
||||
return { a, b, rest, first, tail, merged, arr, bumped, opt, total };
|
||||
}
|
||||
|
||||
const result = await main(); // top-level await
|
||||
console.log("TSX_TRANSFORM_OK " + JSON.stringify(result));
|
||||
@@ -11,6 +11,11 @@ const core = await import("../../src/lib/db/core.ts");
|
||||
const usageHistory = await import("../../src/lib/usage/usageHistory.ts");
|
||||
const callLogs = await import("../../src/lib/usage/callLogs.ts");
|
||||
|
||||
// Captured stream chunks carry a per-chunk arrival-time prefix ("[HH:MM:SS.mmm] ")
|
||||
// added by the request logger for streaming-latency observability (#5834). Strip it
|
||||
// before comparing the raw chunk payload.
|
||||
const stripChunkTs = (chunk: string): string => chunk.replace(/^\[\d{2}:\d{2}:\d{2}\.\d{3}\] /, "");
|
||||
|
||||
test.after(() => {
|
||||
core.resetDbInstance();
|
||||
fs.rmSync(TEST_DATA_DIR, { recursive: true, force: true });
|
||||
@@ -544,7 +549,7 @@ test("createRequestLogger and trackPendingRequest with matching model propagate
|
||||
string,
|
||||
string[]
|
||||
>;
|
||||
assert.equal(apiChunks?.provider[0], 'data: {"chunk":"hello"}');
|
||||
assert.equal(stripChunkTs(apiChunks?.provider[0]), 'data: {"chunk":"hello"}');
|
||||
});
|
||||
|
||||
test("finalizePendingRequestById completes the exact stream when same model requests overlap", () => {
|
||||
|
||||
80
tests/unit/agent-goal-policy.test.ts
Normal file
80
tests/unit/agent-goal-policy.test.ts
Normal file
@@ -0,0 +1,80 @@
|
||||
import test from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
|
||||
import {
|
||||
DEFAULT_AGENT_GOAL_READINESS_MAX_TIMEOUT_MS,
|
||||
isAgentGoalRequestBody,
|
||||
resolveAgentGoalPolicy,
|
||||
} from "../../open-sse/utils/agentGoalPolicy.ts";
|
||||
|
||||
test("detects Claude /goal slash command in nested message content", () => {
|
||||
assert.equal(
|
||||
isAgentGoalRequestBody({
|
||||
model: "claude-sonnet-4-5",
|
||||
messages: [
|
||||
{
|
||||
role: "user",
|
||||
content: [
|
||||
{ type: "text", text: "please continue" },
|
||||
{ type: "text", text: "/goal finish the migration without stopping" },
|
||||
],
|
||||
},
|
||||
],
|
||||
}),
|
||||
true
|
||||
);
|
||||
});
|
||||
|
||||
test("does not detect ordinary goal text or longer slash commands", () => {
|
||||
assert.equal(isAgentGoalRequestBody({ messages: [{ content: "goal: keep going" }] }), false);
|
||||
assert.equal(isAgentGoalRequestBody({ messages: [{ content: "/goals list" }] }), false);
|
||||
});
|
||||
|
||||
test("header can force goal policy and env can tune or disable recovery", () => {
|
||||
const forced = resolveAgentGoalPolicy(
|
||||
{ messages: [{ content: "normal prompt" }] },
|
||||
{ "x-omniroute-agent-goal": "true" },
|
||||
{
|
||||
OMNIROUTE_AGENT_GOAL_READINESS_MAX_TIMEOUT_MS: "900000",
|
||||
OMNIROUTE_AGENT_GOAL_STREAM_RECOVERY: "false",
|
||||
}
|
||||
);
|
||||
|
||||
assert.equal(forced.detected, true);
|
||||
assert.equal(forced.readinessMaxTimeoutMs, 900_000);
|
||||
assert.equal(forced.streamRecoveryEnabled, false);
|
||||
});
|
||||
|
||||
test("goal policy defaults to 10 minute readiness cap with recovery enabled", () => {
|
||||
const policy = resolveAgentGoalPolicy({ messages: [{ content: "/goal ship it" }] }, null, {});
|
||||
|
||||
assert.equal(policy.detected, true);
|
||||
assert.equal(policy.readinessMaxTimeoutMs, DEFAULT_AGENT_GOAL_READINESS_MAX_TIMEOUT_MS);
|
||||
assert.equal(policy.streamRecoveryEnabled, true);
|
||||
});
|
||||
|
||||
test("OMNIROUTE_AGENT_GOAL_POLICY_ENABLED defaults to true — heuristic stays active", () => {
|
||||
const policy = resolveAgentGoalPolicy({ messages: [{ content: "/goal ship it" }] }, null, {});
|
||||
|
||||
assert.equal(policy.detected, true);
|
||||
assert.equal(policy.streamRecoveryEnabled, true);
|
||||
});
|
||||
|
||||
test("OMNIROUTE_AGENT_GOAL_POLICY_ENABLED=false disables the heuristic entirely (no-op)", () => {
|
||||
const policy = resolveAgentGoalPolicy(
|
||||
{ messages: [{ content: "/goal ship it" }] },
|
||||
{ "x-omniroute-agent-goal": "true" },
|
||||
{
|
||||
OMNIROUTE_AGENT_GOAL_POLICY_ENABLED: "false",
|
||||
OMNIROUTE_AGENT_GOAL_READINESS_MAX_TIMEOUT_MS: "900000",
|
||||
}
|
||||
);
|
||||
|
||||
assert.equal(policy.detected, false, "detection must be a no-op even when header forces it");
|
||||
assert.equal(
|
||||
policy.readinessMaxTimeoutMs,
|
||||
DEFAULT_AGENT_GOAL_READINESS_MAX_TIMEOUT_MS,
|
||||
"readiness timeout must never be elevated when the policy is disabled"
|
||||
);
|
||||
assert.equal(policy.streamRecoveryEnabled, false);
|
||||
});
|
||||
@@ -55,6 +55,10 @@ test("classify429: AG 'Individual quota reached' message → quota_exhausted", (
|
||||
assert.equal(classify429(msg), "quota_exhausted");
|
||||
});
|
||||
|
||||
test("classify429: AG G1 Credits Exhausted message → quota_exhausted", () => {
|
||||
assert.equal(classify429("insufficient_g1_credits_balance"), "quota_exhausted");
|
||||
});
|
||||
|
||||
test("classify429: standard Gemini rate limit 'resource has been exhausted' -> rate_limited or unknown, not quota_exhausted", () => {
|
||||
const msg =
|
||||
"RESOURCE_EXHAUSTED: Resource has been exhausted (e.g. queries per minute limit was reached).";
|
||||
|
||||
88
tests/unit/antigravity-429-quota-tdd.test.ts
Normal file
88
tests/unit/antigravity-429-quota-tdd.test.ts
Normal file
@@ -0,0 +1,88 @@
|
||||
import { test } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
|
||||
import { classify429 as classify429AG } from "../../open-sse/services/antigravity429Engine.ts";
|
||||
import { classify429 as classify429Shared } from "../../src/shared/utils/classify429.ts";
|
||||
import {
|
||||
parseRetryFromErrorText,
|
||||
checkFallbackError,
|
||||
} from "../../open-sse/services/accountFallback.ts";
|
||||
|
||||
test("TDD S1: classify429 (Antigravity engine) detects INSUFFICIENT_G1_CREDITS_BALANCE", () => {
|
||||
const msg = JSON.stringify({
|
||||
error: {
|
||||
code: 429,
|
||||
message: "Resource has been exhausted (e.g. check quota).",
|
||||
status: "RESOURCE_EXHAUSTED",
|
||||
details: [
|
||||
{
|
||||
"@type": "type.googleapis.com/google.rpc.ErrorInfo",
|
||||
reason: "INSUFFICIENT_G1_CREDITS_BALANCE",
|
||||
},
|
||||
],
|
||||
},
|
||||
});
|
||||
|
||||
const category = classify429AG(msg);
|
||||
assert.equal(category, "quota_exhausted");
|
||||
});
|
||||
|
||||
test("TDD S1: classify429 (Shared utility) detects INSUFFICIENT_G1_CREDITS_BALANCE", () => {
|
||||
const body = {
|
||||
error: {
|
||||
code: 429,
|
||||
message: "Resource has been exhausted (e.g. check quota).",
|
||||
status: "RESOURCE_EXHAUSTED",
|
||||
details: [
|
||||
{
|
||||
"@type": "type.googleapis.com/google.rpc.ErrorInfo",
|
||||
reason: "INSUFFICIENT_G1_CREDITS_BALANCE",
|
||||
},
|
||||
],
|
||||
},
|
||||
};
|
||||
|
||||
const kind = classify429Shared({ status: 429, body });
|
||||
assert.equal(kind, "quota_exhausted");
|
||||
});
|
||||
|
||||
test("TDD S2: Regression: standard Gemini rate limit 'queries per minute limit was reached' -> rate_limit (shared) and rate_limited (AG)", () => {
|
||||
const msg =
|
||||
"RESOURCE_EXHAUSTED: Resource has been exhausted (e.g. queries per minute limit was reached).";
|
||||
assert.notEqual(classify429AG(msg), "quota_exhausted");
|
||||
assert.equal(classify429Shared({ status: 429, body: msg }), "rate_limit");
|
||||
});
|
||||
|
||||
test("TDD S3: parseRetryFromErrorText parses resets in 5h and resets in 164h27m24s", () => {
|
||||
// Antigravity returns: "Individual quota reached. Contact your administrator to enable overages. Resets in 5h."
|
||||
const msg5h =
|
||||
"Individual quota reached. Contact your administrator to enable overages. Resets in 5h.";
|
||||
const msgWeekly =
|
||||
"Individual quota reached. Contact your administrator to enable overages. Resets in 164h27m24s.";
|
||||
|
||||
const val5h = parseRetryFromErrorText(msg5h);
|
||||
assert.equal(val5h, 5 * 3600 * 1000);
|
||||
|
||||
const valWeekly = parseRetryFromErrorText(msgWeekly);
|
||||
assert.equal(valWeekly, 164 * 3600 * 1000 + 27 * 60 * 1000 + 24 * 1000);
|
||||
});
|
||||
|
||||
test("TDD S3: checkFallbackError extracts retry hint for oauth providers even if useUpstreamRetryHints is false", () => {
|
||||
const errorText =
|
||||
"Individual quota reached. Contact your administrator to enable overages. Resets in 5h.";
|
||||
const res = checkFallbackError(
|
||||
429,
|
||||
errorText,
|
||||
0,
|
||||
"gemini-3.5-flash",
|
||||
"antigravity", // which uses oauth provider profile (useUpstreamRetryHints: false)
|
||||
null
|
||||
);
|
||||
|
||||
assert.equal(res.shouldFallback, true);
|
||||
assert.equal(res.usedUpstreamRetryHint, false);
|
||||
// Connection cooldown should be the default/scaled backoff cooldown (e.g. ~5000 ms) because useUpstreamRetryHints is false
|
||||
assert.notEqual(res.cooldownMs, 5 * 3600 * 1000);
|
||||
// But quotaResetHintMs MUST be the precise parsed reset time (5h = 18,000,000 ms)
|
||||
assert.equal(res.quotaResetHintMs, 5 * 3600 * 1000);
|
||||
});
|
||||
34
tests/unit/api-key-provider-quota-bypass-scope.test.ts
Normal file
34
tests/unit/api-key-provider-quota-bypass-scope.test.ts
Normal file
@@ -0,0 +1,34 @@
|
||||
import test from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import fs from "node:fs";
|
||||
import path from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
|
||||
import {
|
||||
API_KEY_BYPASS_PROVIDER_QUOTA_SCOPE,
|
||||
hasProviderQuotaBypassScope,
|
||||
} from "../../src/shared/constants/apiKeyPolicyScopes.ts";
|
||||
|
||||
const repoRoot = path.resolve(path.dirname(fileURLToPath(import.meta.url)), "../..");
|
||||
|
||||
test("provider quota bypass scope helper is explicit and strict", () => {
|
||||
assert.equal(API_KEY_BYPASS_PROVIDER_QUOTA_SCOPE, "policy:bypass-provider-quota");
|
||||
assert.equal(hasProviderQuotaBypassScope([API_KEY_BYPASS_PROVIDER_QUOTA_SCOPE]), true);
|
||||
assert.equal(hasProviderQuotaBypassScope(["policy:other"]), false);
|
||||
assert.equal(hasProviderQuotaBypassScope(null), false);
|
||||
});
|
||||
|
||||
test("chat handler maps API key provider quota bypass scope to auth bypass option", () => {
|
||||
const source = fs.readFileSync(path.join(repoRoot, "src/sse/handlers/chat.ts"), "utf8");
|
||||
|
||||
assert.match(source, /hasProviderQuotaBypassScope\(apiKeyInfo\?\.scopes\)/);
|
||||
assert.match(source, /bypassProviderQuotaPolicy[\s\S]*bypassQuotaPolicy: true/);
|
||||
assert.match(source, /relayOptions[\s\S]*bypassProviderQuotaPolicy: true/);
|
||||
});
|
||||
|
||||
test("auto combo disables hard provider quota cutoffs when relay requests bypass", () => {
|
||||
const source = fs.readFileSync(path.join(repoRoot, "open-sse/services/combo.ts"), "utf8");
|
||||
|
||||
assert.match(source, /relayOptions\?\.bypassProviderQuotaPolicy === true/);
|
||||
assert.match(source, /quotaPreflight:[\s\S]*enabled: false/);
|
||||
});
|
||||
@@ -231,6 +231,27 @@ test("getApiKeyUsageLimitStatus cuts weekly USD spend at observed provider quota
|
||||
null,
|
||||
"2026-06-20T02:10:00.000Z"
|
||||
);
|
||||
db.prepare(
|
||||
`
|
||||
INSERT INTO provider_quota_reset_events
|
||||
(provider, connection_id, window_key, window_started_at, window_resets_at,
|
||||
observed_at, previous_remaining_percentage, new_remaining_percentage,
|
||||
previous_used_percentage, new_used_percentage, raw_data)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
`
|
||||
).run(
|
||||
"claude",
|
||||
"conn-claude",
|
||||
"weekly (7d)",
|
||||
"2026-06-18T23:00:00.000Z",
|
||||
"2026-06-25T23:00:00.000Z",
|
||||
"2026-06-18T23:04:00.000Z",
|
||||
0,
|
||||
100,
|
||||
100,
|
||||
0,
|
||||
null
|
||||
);
|
||||
|
||||
const metadata = await apiKeysDb.getApiKeyMetadata(created.key);
|
||||
assert.ok(metadata);
|
||||
@@ -287,25 +308,51 @@ test("buildApiKeyUsageLimitText returns API-key quota spend percentage and reset
|
||||
assert.equal(
|
||||
text,
|
||||
[
|
||||
"Cota diaria",
|
||||
"Daily quota",
|
||||
"$10.00",
|
||||
"Gasto diario",
|
||||
"Daily spent",
|
||||
"$2.00",
|
||||
"Uso diario",
|
||||
"Daily used",
|
||||
"20%",
|
||||
"Resets in 7h",
|
||||
"Resets in 7h 0m",
|
||||
"",
|
||||
"Cota semanal",
|
||||
"Weekly quota",
|
||||
"$50.00",
|
||||
"Gasto semanal",
|
||||
"Weekly spent",
|
||||
"$5.25",
|
||||
"Uso semanal",
|
||||
"Weekly used",
|
||||
"11%",
|
||||
"Resets in 6d",
|
||||
"Resets in 6d 0h 0m",
|
||||
].join("\n")
|
||||
);
|
||||
});
|
||||
|
||||
test("buildApiKeyUsageLimitPercentText returns remaining percentages only", () => {
|
||||
const text = usageLimits.buildApiKeyUsageLimitPercentText(
|
||||
{
|
||||
enabled: true,
|
||||
dailyLimitUsd: 10,
|
||||
weeklyLimitUsd: 50,
|
||||
dailySpentUsd: 2,
|
||||
weeklySpentUsd: 5.25,
|
||||
dailyWindowStartIso: "2026-06-19T03:00:00.000Z",
|
||||
dailyResetAtIso: "2026-06-20T03:00:00.000Z",
|
||||
weeklyWindowStartIso: "2026-06-12T20:00:00.000Z",
|
||||
weeklyResetAtIso: "2026-06-25T20:00:00.000Z",
|
||||
dailyExceeded: false,
|
||||
weeklyExceeded: false,
|
||||
},
|
||||
Date.parse("2026-06-19T20:00:00.000Z")
|
||||
);
|
||||
|
||||
assert.equal(
|
||||
text,
|
||||
["Daily", "80% left", "⏱ reset in 7h 0m", "", "Weekly", "90% left", "⏱ reset in 6d 0h 0m"].join(
|
||||
"\n"
|
||||
)
|
||||
);
|
||||
});
|
||||
|
||||
test("buildApiKeyUsageLimitRejection includes over-quota percentage and reset hint", async () => {
|
||||
const response = usageLimits.buildApiKeyUsageLimitRejection(
|
||||
new Request("http://localhost/v1/messages", {
|
||||
@@ -331,7 +378,37 @@ test("buildApiKeyUsageLimitRejection includes over-quota percentage and reset hi
|
||||
const body = (await response.json()) as { error: { message: string } };
|
||||
assert.equal(
|
||||
body.error.message,
|
||||
"This API key reached its weekly USD usage quota ($1.09 of $1.00, 109%). Resets in 6d. Choose another allowed model after reset."
|
||||
"This API key reached its weekly USD usage quota ($1.09 of $1.00, 109%). Resets in 6d 0h 0m. Choose another allowed model after reset."
|
||||
);
|
||||
});
|
||||
|
||||
test("buildApiKeyUsageLimitRejection can hide USD amounts for client-facing policy errors", async () => {
|
||||
const response = usageLimits.buildApiKeyUsageLimitRejection(
|
||||
new Request("http://localhost/v1/messages", {
|
||||
headers: { "anthropic-version": "2023-06-01" },
|
||||
}),
|
||||
{
|
||||
enabled: true,
|
||||
dailyLimitUsd: 10,
|
||||
weeklyLimitUsd: 1,
|
||||
dailySpentUsd: 0.25,
|
||||
weeklySpentUsd: 1.09,
|
||||
dailyWindowStartIso: "2026-06-19T03:00:00.000Z",
|
||||
dailyResetAtIso: "2026-06-20T03:00:00.000Z",
|
||||
weeklyWindowStartIso: "2026-06-12T20:00:00.000Z",
|
||||
weeklyResetAtIso: "2026-06-25T20:00:00.000Z",
|
||||
dailyExceeded: false,
|
||||
weeklyExceeded: true,
|
||||
},
|
||||
Date.parse("2026-06-19T20:00:00.000Z"),
|
||||
{ showUsd: false }
|
||||
);
|
||||
|
||||
assert.equal(response.status, 400);
|
||||
const body = (await response.json()) as { error: { message: string } };
|
||||
assert.equal(
|
||||
body.error.message,
|
||||
"This API key reached its weekly usage quota (109%). Resets in 6d 0h 0m. Choose another allowed model after reset."
|
||||
);
|
||||
});
|
||||
|
||||
|
||||
@@ -61,13 +61,14 @@ test("permissions modal switch buttons declare button type", () => {
|
||||
selfServiceBlock.match(/<button\s+type="button"\s+role="switch"/g) ?? []
|
||||
).length;
|
||||
|
||||
// Self-service Visibility block has 4 switches: own-usage visibility,
|
||||
// shared-account quota visibility, disable-non-public-models (#3041), and the
|
||||
// per-key local usage command allowance (#4034).
|
||||
// Self-service Visibility block has 5 switches: own-usage visibility,
|
||||
// shared-account quota visibility, disable-non-public-models (#3041), the
|
||||
// per-key local usage command allowance (#4034), and the API-key provider
|
||||
// quota-policy bypass scope (#5731).
|
||||
// The invariant is that every switch declares type="button"
|
||||
// (typedSwitchButtonCount === switchButtonCount) to avoid implicit submit.
|
||||
assert.equal(switchButtonCount, 4);
|
||||
assert.equal(typedSwitchButtonCount, 4);
|
||||
assert.equal(switchButtonCount, 5);
|
||||
assert.equal(typedSwitchButtonCount, 5);
|
||||
});
|
||||
|
||||
test("permissions modal exposes Claude Code default wildcard model", () => {
|
||||
|
||||
@@ -9,10 +9,18 @@ import {
|
||||
SELF_ACCOUNT_QUOTA_SCOPE,
|
||||
SELF_USAGE_SCOPE,
|
||||
} from "../../src/shared/constants/selfServiceScopes.ts";
|
||||
import { API_KEY_BYPASS_PROVIDER_QUOTA_SCOPE } from "../../src/shared/constants/apiKeyPolicyScopes.ts";
|
||||
|
||||
test("create scopes enable own usage by default without shared account quota", () => {
|
||||
assert.deepEqual(buildApiKeyCreateScopes({ manageEnabled: false }), [SELF_USAGE_SCOPE]);
|
||||
assert.deepEqual(buildApiKeyCreateScopes({ manageEnabled: true }), ["manage", SELF_USAGE_SCOPE]);
|
||||
assert.deepEqual(
|
||||
buildApiKeyCreateScopes({
|
||||
manageEnabled: false,
|
||||
bypassProviderQuotaPolicyEnabled: true,
|
||||
}),
|
||||
[SELF_USAGE_SCOPE, API_KEY_BYPASS_PROVIDER_QUOTA_SCOPE]
|
||||
);
|
||||
assert.deepEqual(
|
||||
buildApiKeyCreateScopes({
|
||||
manageEnabled: false,
|
||||
@@ -28,6 +36,7 @@ test("permission scope merge preserves unrelated scopes while toggling managed s
|
||||
manageEnabled: true,
|
||||
selfUsageEnabled: true,
|
||||
selfAccountQuotaEnabled: true,
|
||||
bypassProviderQuotaPolicyEnabled: true,
|
||||
});
|
||||
|
||||
assert.deepEqual(scopes, [
|
||||
@@ -35,6 +44,7 @@ test("permission scope merge preserves unrelated scopes while toggling managed s
|
||||
SELF_USAGE_SCOPE,
|
||||
"manage",
|
||||
SELF_ACCOUNT_QUOTA_SCOPE,
|
||||
API_KEY_BYPASS_PROVIDER_QUOTA_SCOPE,
|
||||
]);
|
||||
});
|
||||
|
||||
@@ -45,8 +55,29 @@ test("permission scope merge removes shared quota visibility when own usage is d
|
||||
manageEnabled: false,
|
||||
selfUsageEnabled: false,
|
||||
selfAccountQuotaEnabled: true,
|
||||
bypassProviderQuotaPolicyEnabled: false,
|
||||
}
|
||||
);
|
||||
|
||||
assert.deepEqual(scopes, ["custom:scope"]);
|
||||
});
|
||||
|
||||
test("permission scope merge toggles provider quota policy bypass without dropping custom scopes", () => {
|
||||
const enabled = mergeApiKeyPermissionScopes(["custom:scope"], {
|
||||
manageEnabled: false,
|
||||
selfUsageEnabled: false,
|
||||
selfAccountQuotaEnabled: false,
|
||||
bypassProviderQuotaPolicyEnabled: true,
|
||||
});
|
||||
|
||||
assert.deepEqual(enabled, ["custom:scope", API_KEY_BYPASS_PROVIDER_QUOTA_SCOPE]);
|
||||
|
||||
const disabled = mergeApiKeyPermissionScopes(enabled, {
|
||||
manageEnabled: false,
|
||||
selfUsageEnabled: false,
|
||||
selfAccountQuotaEnabled: false,
|
||||
bypassProviderQuotaPolicyEnabled: false,
|
||||
});
|
||||
|
||||
assert.deepEqual(disabled, ["custom:scope"]);
|
||||
});
|
||||
|
||||
173
tests/unit/apikey-connection-health-check.test.ts
Normal file
173
tests/unit/apikey-connection-health-check.test.ts
Normal file
@@ -0,0 +1,173 @@
|
||||
/**
|
||||
* Regression test: API-key-only connections must not be falsely expired.
|
||||
*
|
||||
* Bug: tokenHealthCheck.checkConnection() marked API-key-only connections
|
||||
* (e.g. gemini with just an API key, no OAuth refresh token) as
|
||||
* testStatus="expired" because it expected OAuth refresh tokens for any
|
||||
* provider in the supportsTokenRefresh set.
|
||||
*
|
||||
* Fix: connections that have an apiKey configured are skipped during OAuth
|
||||
* token validation, since they don't require refresh tokens.
|
||||
*/
|
||||
import test from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import fs from "node:fs";
|
||||
import os from "node:os";
|
||||
import path from "node:path";
|
||||
|
||||
process.env.NODE_ENV = "test";
|
||||
|
||||
const TEST_DATA_DIR = fs.mkdtempSync(path.join(os.tmpdir(), "omniroute-apikey-health-"));
|
||||
process.env.DATA_DIR = TEST_DATA_DIR;
|
||||
|
||||
const core = await import("../../src/lib/db/core.ts");
|
||||
const providersDb = await import("../../src/lib/db/providers.ts");
|
||||
const { checkConnection } = await import("../../src/lib/tokenHealthCheck.ts");
|
||||
|
||||
async function resetStorage() {
|
||||
core.resetDbInstance();
|
||||
for (let attempt = 0; attempt < 10; attempt++) {
|
||||
try {
|
||||
if (fs.existsSync(TEST_DATA_DIR)) {
|
||||
fs.rmSync(TEST_DATA_DIR, { recursive: true, force: true });
|
||||
}
|
||||
break;
|
||||
} catch (error: any) {
|
||||
if ((error?.code === "EBUSY" || error?.code === "EPERM") && attempt < 9) {
|
||||
await new Promise((resolve) => setTimeout(resolve, 50 * (attempt + 1)));
|
||||
} else {
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
}
|
||||
fs.mkdirSync(TEST_DATA_DIR, { recursive: true });
|
||||
}
|
||||
|
||||
test.after(async () => {
|
||||
core.resetDbInstance();
|
||||
fs.rmSync(TEST_DATA_DIR, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
test("API-key-only gemini connection is NOT marked expired by health check", async () => {
|
||||
await resetStorage();
|
||||
|
||||
// Create a gemini connection with an API key but no refresh token
|
||||
const conn = await providersDb.createProviderConnection({
|
||||
provider: "gemini",
|
||||
name: "gemini-apikey-test",
|
||||
apiKey: "AIzaSyTest1234567890abcdefghijklmnop",
|
||||
isActive: true,
|
||||
testStatus: "active",
|
||||
healthCheckInterval: 60,
|
||||
// No refreshToken — this is an API-key-only connection
|
||||
});
|
||||
|
||||
assert.equal(conn.testStatus, "active", "precondition: connection starts as active");
|
||||
|
||||
// Run the health check
|
||||
await checkConnection(conn);
|
||||
|
||||
// Re-read from DB
|
||||
const updated = await providersDb.getProviderConnectionById(conn.id);
|
||||
|
||||
assert.equal(
|
||||
updated?.testStatus,
|
||||
"active",
|
||||
"API-key-only connection should remain active — not be marked expired"
|
||||
);
|
||||
assert.notEqual(
|
||||
updated?.errorCode,
|
||||
"no_refresh_token",
|
||||
"API-key-only connection should not get no_refresh_token error"
|
||||
);
|
||||
});
|
||||
|
||||
test("gemini connection WITHOUT apiKey AND WITHOUT refreshToken IS marked expired", async () => {
|
||||
await resetStorage();
|
||||
|
||||
// Create a gemini OAuth connection that lost its refresh token
|
||||
const conn = await providersDb.createProviderConnection({
|
||||
provider: "gemini",
|
||||
name: "gemini-oauth-no-refresh",
|
||||
accessToken: "ya29.expired-token",
|
||||
isActive: true,
|
||||
testStatus: "active",
|
||||
healthCheckInterval: 60,
|
||||
// No apiKey, no refreshToken — this is a broken OAuth connection
|
||||
});
|
||||
|
||||
assert.equal(conn.testStatus, "active", "precondition: connection starts as active");
|
||||
|
||||
// Run the health check
|
||||
await checkConnection(conn);
|
||||
|
||||
// Re-read from DB
|
||||
const updated = await providersDb.getProviderConnectionById(conn.id);
|
||||
|
||||
assert.equal(
|
||||
updated?.testStatus,
|
||||
"expired",
|
||||
"OAuth connection without refresh token should be marked expired"
|
||||
);
|
||||
assert.equal(
|
||||
updated?.errorCode,
|
||||
"no_refresh_token",
|
||||
"OAuth connection should get no_refresh_token error code"
|
||||
);
|
||||
});
|
||||
|
||||
test("API-key-only antigravity connection is NOT marked expired by health check", async () => {
|
||||
await resetStorage();
|
||||
|
||||
// antigravity also supports token refresh — verify the fix applies to all providers
|
||||
const conn = await providersDb.createProviderConnection({
|
||||
provider: "antigravity",
|
||||
name: "agy-apikey-test",
|
||||
apiKey: "sk-ant-test1234567890",
|
||||
isActive: true,
|
||||
testStatus: "active",
|
||||
healthCheckInterval: 60,
|
||||
});
|
||||
|
||||
await checkConnection(conn);
|
||||
|
||||
const updated = await providersDb.getProviderConnectionById(conn.id);
|
||||
|
||||
assert.equal(
|
||||
updated?.testStatus,
|
||||
"active",
|
||||
"antigravity API-key-only connection should remain active"
|
||||
);
|
||||
});
|
||||
|
||||
test("connection with both apiKey and refreshToken: refresh path is tried", async () => {
|
||||
await resetStorage();
|
||||
|
||||
// Edge case: connection has both an API key and a refresh token
|
||||
// The health check tries the refresh token path first.
|
||||
// With a stale/invalid refresh token, the connection gets marked expired
|
||||
// even though an API key exists — the refresh path takes precedence.
|
||||
const conn = await providersDb.createProviderConnection({
|
||||
provider: "gemini",
|
||||
name: "gemini-dual-auth",
|
||||
apiKey: "AIzaSyTest1234567890abcdefghijklmnop",
|
||||
refreshToken: "1//old-refresh-token",
|
||||
accessToken: "ya29.expired-token",
|
||||
isActive: true,
|
||||
testStatus: "active",
|
||||
healthCheckInterval: 60,
|
||||
});
|
||||
|
||||
await checkConnection(conn);
|
||||
|
||||
const updated = await providersDb.getProviderConnectionById(conn.id);
|
||||
|
||||
// The refresh token path is tried first. Since the refresh token is invalid,
|
||||
// the connection gets marked expired. This is expected — the operator should
|
||||
// either remove the stale refresh token or re-authenticate.
|
||||
assert.equal(
|
||||
updated?.testStatus,
|
||||
"expired",
|
||||
"dual-auth connection with stale refresh token should be expired (refresh path takes precedence)"
|
||||
);
|
||||
});
|
||||
@@ -107,3 +107,29 @@ test("auth login route lazily migrates INITIAL_PASSWORD to a persisted hash befo
|
||||
true
|
||||
);
|
||||
});
|
||||
|
||||
test("auth login route sets a bounded maxAge on the auth_token cookie (Seg3)", async () => {
|
||||
process.env.INITIAL_PASSWORD = "bootstrap-secret";
|
||||
const setCalls: unknown[][] = [];
|
||||
loginRoute.authRouteInternals.getCookieStore = async () => ({
|
||||
set: (...args: unknown[]) => setCalls.push(args),
|
||||
});
|
||||
|
||||
const response = await loginRoute.POST(
|
||||
new Request("http://localhost/api/auth/login", {
|
||||
method: "POST",
|
||||
headers: { "content-type": "application/json" },
|
||||
body: JSON.stringify({ password: "bootstrap-secret" }),
|
||||
})
|
||||
);
|
||||
|
||||
assert.equal(response.status, 200);
|
||||
assert.equal(setCalls.length, 1);
|
||||
const [cookieName, , options] = setCalls[0] as [string, string, Record<string, unknown>];
|
||||
assert.equal(cookieName, "auth_token");
|
||||
// 30 days in seconds — must match the JWT 30d expiry so the cookie is not an open-ended
|
||||
// session cookie outliving its token.
|
||||
assert.equal(options.maxAge, 60 * 60 * 24 * 30);
|
||||
assert.equal(options.httpOnly, true);
|
||||
assert.equal(options.path, "/");
|
||||
});
|
||||
|
||||
@@ -176,6 +176,13 @@ const cases: Case[] = [
|
||||
{ name: "/api/settings MANAGEMENT", path: "/api/settings", expectedClass: "MANAGEMENT" },
|
||||
{ name: "/api/audit MANAGEMENT", path: "/api/audit", expectedClass: "MANAGEMENT" },
|
||||
|
||||
{
|
||||
name: "/api/usage/om-usage is PUBLIC (handler enforces its own API key auth)",
|
||||
path: "/api/usage/om-usage",
|
||||
method: "GET",
|
||||
expectedClass: "PUBLIC",
|
||||
},
|
||||
|
||||
{
|
||||
name: "Unknown top-level path defaults MANAGEMENT (fail-closed)",
|
||||
path: "/totally-unknown",
|
||||
|
||||
@@ -26,7 +26,7 @@ function request(path: string, cookie = "auth_token=session-a", token?: string):
|
||||
}
|
||||
|
||||
describe("dashboard CSRF tokens", () => {
|
||||
it("accepts a valid token for dashboard test mutation paths", () => {
|
||||
it("accepts a valid token for dashboard management mutation paths", () => {
|
||||
const issued = issueDashboardCsrfToken(request("/api/auth/csrf"), 1_000);
|
||||
|
||||
assert.ok(issued);
|
||||
@@ -42,15 +42,13 @@ describe("dashboard CSRF tokens", () => {
|
||||
validateDashboardCsrfToken(request("/api/combos/test", undefined, issued.token), 1_000),
|
||||
true
|
||||
);
|
||||
});
|
||||
|
||||
it("rejects tokens on non-test management paths", () => {
|
||||
const issued = issueDashboardCsrfToken(request("/api/auth/csrf"), 1_000);
|
||||
|
||||
assert.ok(issued);
|
||||
assert.equal(
|
||||
validateDashboardCsrfToken(request("/api/keys", undefined, issued.token), 1_000),
|
||||
false
|
||||
true
|
||||
);
|
||||
assert.equal(
|
||||
validateDashboardCsrfToken(request("/api/settings", undefined, issued.token), 1_000),
|
||||
true
|
||||
);
|
||||
});
|
||||
|
||||
|
||||
@@ -325,12 +325,12 @@ test("runAuthzPipeline accepts dashboard mutations from configured public origin
|
||||
assert.equal(response.headers.get("x-omniroute-route-class"), "MANAGEMENT");
|
||||
});
|
||||
|
||||
test("runAuthzPipeline rejects dashboard test mutations from dynamic public origins without CSRF", async () => {
|
||||
test("runAuthzPipeline rejects dashboard mutations from dynamic public origins without CSRF", async () => {
|
||||
await forceAuthRequired();
|
||||
|
||||
const response = await pipeline.runAuthzPipeline(
|
||||
request("http://127.0.0.1:20128/api/models/test", {
|
||||
method: "POST",
|
||||
request("http://127.0.0.1:20128/api/settings", {
|
||||
method: "PATCH",
|
||||
headers: {
|
||||
cookie: await dashboardCookie(),
|
||||
host: "127.0.0.1:20128",
|
||||
@@ -348,7 +348,7 @@ test("runAuthzPipeline rejects dashboard test mutations from dynamic public orig
|
||||
assert.equal(body.error.code, "INVALID_ORIGIN");
|
||||
});
|
||||
|
||||
test("runAuthzPipeline accepts dashboard test mutations from dynamic public origins with CSRF", async () => {
|
||||
test("runAuthzPipeline accepts dashboard mutations from dynamic public origins with CSRF", async () => {
|
||||
await forceAuthRequired();
|
||||
|
||||
const cookie = await dashboardCookie();
|
||||
@@ -359,10 +359,16 @@ test("runAuthzPipeline accepts dashboard test mutations from dynamic public orig
|
||||
);
|
||||
assert.ok(issued);
|
||||
|
||||
for (const path of ["/api/combos/test", "/api/models/test", "/api/models/test-all"]) {
|
||||
for (const [method, path] of [
|
||||
["POST", "/api/models/test"],
|
||||
["POST", "/api/keys"],
|
||||
["PATCH", "/api/settings"],
|
||||
["PUT", "/api/combos/combo-1"],
|
||||
["DELETE", "/api/webhooks/webhook-1"],
|
||||
] as const) {
|
||||
const response = await pipeline.runAuthzPipeline(
|
||||
request(`http://127.0.0.1:20128${path}`, {
|
||||
method: "POST",
|
||||
method,
|
||||
headers: {
|
||||
cookie,
|
||||
host: "127.0.0.1:20128",
|
||||
@@ -381,7 +387,7 @@ test("runAuthzPipeline accepts dashboard test mutations from dynamic public orig
|
||||
}
|
||||
});
|
||||
|
||||
test("runAuthzPipeline keeps non-test management mutations pinned to known origins with CSRF", async () => {
|
||||
test("runAuthzPipeline does not let CSRF bypass cross-site fetch metadata", async () => {
|
||||
await forceAuthRequired();
|
||||
|
||||
const cookie = await dashboardCookie();
|
||||
@@ -393,15 +399,15 @@ test("runAuthzPipeline keeps non-test management mutations pinned to known origi
|
||||
assert.ok(issued);
|
||||
|
||||
const response = await pipeline.runAuthzPipeline(
|
||||
request("http://127.0.0.1:20128/api/keys", {
|
||||
method: "POST",
|
||||
request("http://127.0.0.1:20128/api/settings", {
|
||||
method: "PATCH",
|
||||
headers: {
|
||||
cookie,
|
||||
host: "127.0.0.1:20128",
|
||||
origin: "https://random-tunnel.example.test",
|
||||
"content-type": "application/json",
|
||||
[dashboardCsrfConstants.DASHBOARD_CSRF_HEADER]: issued.token,
|
||||
"sec-fetch-site": "same-origin",
|
||||
"sec-fetch-site": "cross-site",
|
||||
},
|
||||
body: "{}",
|
||||
}),
|
||||
|
||||
66
tests/unit/auto-custom-provider-5873.test.ts
Normal file
66
tests/unit/auto-custom-provider-5873.test.ts
Normal file
@@ -0,0 +1,66 @@
|
||||
import test from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import fs from "node:fs";
|
||||
import os from "node:os";
|
||||
import path from "node:path";
|
||||
|
||||
// #5873 regression guard: custom OpenAI-/Anthropic-compatible providers have
|
||||
// dynamic connection IDs (`*-compatible-*`) that are never keys of the static
|
||||
// provider registry. The Auto-Combo virtual factory previously skipped any
|
||||
// connection whose provider was absent from the registry, silently excluding
|
||||
// every custom provider from `auto/` routing. It must now fall back to the
|
||||
// connection's defaultModel instead.
|
||||
|
||||
const TEST_DATA_DIR = fs.mkdtempSync(path.join(os.tmpdir(), "omniroute-auto-custom-5873-"));
|
||||
const ORIGINAL_DATA_DIR = process.env.DATA_DIR;
|
||||
|
||||
process.env.DATA_DIR = TEST_DATA_DIR;
|
||||
|
||||
const core = await import("../../src/lib/db/core.ts");
|
||||
const providersDb = await import("../../src/lib/db/providers.ts");
|
||||
const virtualFactory = await import("../../open-sse/services/autoCombo/virtualFactory.ts");
|
||||
|
||||
type VirtualComboResult = Awaited<ReturnType<typeof virtualFactory.createVirtualAutoCombo>>;
|
||||
|
||||
async function resetStorage() {
|
||||
core.resetDbInstance();
|
||||
fs.rmSync(TEST_DATA_DIR, { recursive: true, force: true });
|
||||
fs.mkdirSync(TEST_DATA_DIR, { recursive: true });
|
||||
}
|
||||
|
||||
test.beforeEach(async () => {
|
||||
await resetStorage();
|
||||
});
|
||||
|
||||
test.after(async () => {
|
||||
await resetStorage();
|
||||
fs.rmSync(TEST_DATA_DIR, { recursive: true, force: true });
|
||||
|
||||
if (ORIGINAL_DATA_DIR === undefined) {
|
||||
delete process.env.DATA_DIR;
|
||||
} else {
|
||||
process.env.DATA_DIR = ORIGINAL_DATA_DIR;
|
||||
}
|
||||
});
|
||||
|
||||
test("createVirtualAutoCombo includes custom openai-compatible providers via defaultModel (#5873)", async () => {
|
||||
const customProvider = "openai-compatible-chat-02669115-2545-4896-b003-cb4dac09d441";
|
||||
await providersDb.createProviderConnection({
|
||||
provider: customProvider,
|
||||
authType: "apikey",
|
||||
name: "My Custom LLM",
|
||||
apiKey: "sk-custom-key",
|
||||
defaultModel: "my-custom-model",
|
||||
});
|
||||
|
||||
const combo: VirtualComboResult = await virtualFactory.createVirtualAutoCombo("fast");
|
||||
|
||||
assert.equal(combo.strategy, "auto");
|
||||
const candidate = combo.models.find((model) => model.providerId === customProvider);
|
||||
assert.ok(
|
||||
candidate,
|
||||
"custom openai-compatible providers must not be excluded from auto/ routing"
|
||||
);
|
||||
assert.equal(candidate.model, `${customProvider}/my-custom-model`);
|
||||
assert.ok(combo.autoConfig.candidatePool.includes(customProvider));
|
||||
});
|
||||
71
tests/unit/auto-update-mode-detection.test.ts
Normal file
71
tests/unit/auto-update-mode-detection.test.ts
Normal file
@@ -0,0 +1,71 @@
|
||||
import test from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
|
||||
const { resolveAutoUpdateMode, isUnderNodeModules } = await import(
|
||||
"../../src/lib/system/autoUpdate.ts"
|
||||
);
|
||||
|
||||
test("non-npm modes pass through untouched (operator choice wins)", () => {
|
||||
assert.equal(
|
||||
resolveAutoUpdateMode("source", { isGitRepo: false, currentDir: "/x" }),
|
||||
"source"
|
||||
);
|
||||
assert.equal(
|
||||
resolveAutoUpdateMode("docker-compose", {
|
||||
isGitRepo: true,
|
||||
currentDir: "/x/node_modules/y",
|
||||
}),
|
||||
"docker-compose"
|
||||
);
|
||||
});
|
||||
|
||||
test("npm + git repo → source (a source checkout self-updates via git)", () => {
|
||||
assert.equal(
|
||||
resolveAutoUpdateMode("npm", {
|
||||
isGitRepo: true,
|
||||
currentDir: "/home/me/omniroute/dist/lib/system",
|
||||
}),
|
||||
"source"
|
||||
);
|
||||
});
|
||||
|
||||
test("npm + global install under node_modules → npm", () => {
|
||||
assert.equal(
|
||||
resolveAutoUpdateMode("npm", {
|
||||
isGitRepo: false,
|
||||
currentDir: "/usr/lib/node_modules/omniroute/dist/lib/system",
|
||||
}),
|
||||
"npm"
|
||||
);
|
||||
});
|
||||
|
||||
test("npm + no git + not under node_modules → source (downloaded build/zip)", () => {
|
||||
assert.equal(
|
||||
resolveAutoUpdateMode("npm", {
|
||||
isGitRepo: false,
|
||||
currentDir: "/opt/omniroute/dist/lib/system",
|
||||
}),
|
||||
"source"
|
||||
);
|
||||
});
|
||||
|
||||
test("Bug1: a substring-only node_modules path is not treated as an install", () => {
|
||||
// The old heuristic (`currentDir.includes("node_modules")`) returned "npm" for this path,
|
||||
// misclassifying it as a global install. The segment match treats it as source.
|
||||
assert.equal(isUnderNodeModules("/opt/my-node_modules-backup/dist"), false);
|
||||
assert.equal(
|
||||
resolveAutoUpdateMode("npm", {
|
||||
isGitRepo: false,
|
||||
currentDir: "/opt/my-node_modules-backup/dist",
|
||||
}),
|
||||
"source"
|
||||
);
|
||||
});
|
||||
|
||||
test("isUnderNodeModules matches real segments on both path separators", () => {
|
||||
assert.equal(isUnderNodeModules("/usr/lib/node_modules/omniroute"), true);
|
||||
assert.equal(isUnderNodeModules("C:\\Users\\me\\node_modules\\omniroute"), true);
|
||||
assert.equal(isUnderNodeModules("/usr/lib/node_modules"), true); // trailing segment
|
||||
assert.equal(isUnderNodeModules("/opt/app/dist"), false);
|
||||
assert.equal(isUnderNodeModules("/opt/mynode_modulesbar/dist"), false);
|
||||
});
|
||||
57
tests/unit/autoupdate-npm-win32-5542.test.ts
Normal file
57
tests/unit/autoupdate-npm-win32-5542.test.ts
Normal file
@@ -0,0 +1,57 @@
|
||||
import test from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import fs from "node:fs";
|
||||
|
||||
// #5542 — On Windows, npm is `npm.cmd`; Node ≥24 refuses to execFile a `.cmd`
|
||||
// without a shell (nodejs/node#52554), so the in-app auto-update flow threw
|
||||
// "spawn npm ENOENT" for the version lookup, dependency install, global install,
|
||||
// and native rebuild. Those npm calls now go through buildNpmExecOptions (the
|
||||
// same win32-shell helper the embedded-services installer uses, fix #5379).
|
||||
const { buildNpmExecOptions, SERVICE_VERSION_PATTERN } = await import(
|
||||
"../../src/lib/services/installers/utils.ts"
|
||||
);
|
||||
|
||||
test("#5542 npm exec options enable the shell on win32 (resolves npm.cmd → no ENOENT)", () => {
|
||||
const win = buildNpmExecOptions("win32", { cwd: "/x", timeoutMs: 1000 });
|
||||
assert.equal(win.shell, true, "win32 must enable the shell so npm.cmd resolves");
|
||||
assert.equal(win.timeout, 1000);
|
||||
assert.equal(win.cwd, "/x");
|
||||
|
||||
const linux = buildNpmExecOptions("linux", { cwd: "/x", timeoutMs: 1000 });
|
||||
assert.notEqual(linux.shell, true, "non-win32 must not enable the shell");
|
||||
});
|
||||
|
||||
test("#5542 the update version spec is validated before it is shell-joined (Hard Rule #13)", () => {
|
||||
assert.ok(SERVICE_VERSION_PATTERN.test("3.8.43"));
|
||||
assert.ok(SERVICE_VERSION_PATTERN.test("3.8.43-beta.1"));
|
||||
assert.ok(!SERVICE_VERSION_PATTERN.test("1.0.0; rm -rf /"));
|
||||
assert.ok(!SERVICE_VERSION_PATTERN.test("$(whoami)"));
|
||||
assert.ok(!SERVICE_VERSION_PATTERN.test("1.0 && curl evil"));
|
||||
});
|
||||
|
||||
test("#5542 the auto-update npm call sites route through buildNpmExecOptions", () => {
|
||||
const routeSrc = fs.readFileSync(
|
||||
new URL("../../src/app/api/system/version/route.ts", import.meta.url),
|
||||
"utf8"
|
||||
);
|
||||
const checkSrc = fs.readFileSync(
|
||||
new URL("../../src/lib/system/versionCheck.ts", import.meta.url),
|
||||
"utf8"
|
||||
);
|
||||
assert.ok(routeSrc.includes("buildNpmExecOptions"), "version route must use the win32-shell helper");
|
||||
assert.ok(checkSrc.includes("buildNpmExecOptions"), "versionCheck must use the win32-shell helper");
|
||||
// The global install spec must be guarded before it reaches the shell.
|
||||
assert.ok(
|
||||
routeSrc.includes("SERVICE_VERSION_PATTERN.test(latest)"),
|
||||
"version route must validate the version spec before shell-joining it"
|
||||
);
|
||||
// Every npm invocation in the route must pass buildNpmExecOptions (not a bare
|
||||
// inline options object that would lack the win32 shell).
|
||||
const npmCalls = routeSrc.match(/execFileAsync\(\s*\n?\s*"npm",/g) || [];
|
||||
const npmViaHelper = routeSrc.match(/"npm",[\s\S]{0,120}?buildNpmExecOptions\(/g) || [];
|
||||
assert.equal(
|
||||
npmViaHelper.length,
|
||||
npmCalls.length,
|
||||
`all ${npmCalls.length} npm calls must route through buildNpmExecOptions`
|
||||
);
|
||||
});
|
||||
@@ -319,10 +319,147 @@ test("processPendingBatches caches rate-limit headers across sequential batches"
|
||||
assert.strictEqual(afterReset.timestamp, 0, "reset should clear cached timestamp");
|
||||
});
|
||||
|
||||
test("processPendingBatches should recover stale in_progress batches", async () => {
|
||||
test("processPendingBatches should recover checkpointed stale batches without replaying completed items", async () => {
|
||||
const lines = [
|
||||
JSON.stringify({
|
||||
custom_id: "already-done",
|
||||
method: "POST",
|
||||
url: "/v1/chat/completions",
|
||||
body: { model: "gpt-4", messages: [{ role: "user", content: "first" }] },
|
||||
}),
|
||||
JSON.stringify({
|
||||
custom_id: "needs-dispatch",
|
||||
method: "POST",
|
||||
url: "/v1/chat/completions",
|
||||
body: { model: "gpt-4", messages: [{ role: "user", content: "second" }] },
|
||||
}),
|
||||
];
|
||||
const file = await localDb.createFile({
|
||||
bytes: Buffer.byteLength(lines.join("\n") + "\n"),
|
||||
filename: "checkpointed_stale_test.jsonl",
|
||||
purpose: "batch_input",
|
||||
content: Buffer.from(lines.join("\n") + "\n"),
|
||||
});
|
||||
|
||||
const batch = await localDb.createBatch({
|
||||
endpoint: "/v1/chat/completions",
|
||||
status: "in_progress",
|
||||
inputFileId: file.id,
|
||||
completionWindow: "24h",
|
||||
inProgressAt: Math.floor(Date.now() / 1000),
|
||||
});
|
||||
await localDb.updateBatch(batch.id, {
|
||||
requestCountsTotal: 2,
|
||||
requestCountsCompleted: 1,
|
||||
});
|
||||
await localDb.ensureBatchItemCheckpoints(batch.id, [
|
||||
{ lineNumber: 1, customId: "already-done" },
|
||||
{ lineNumber: 2, customId: "needs-dispatch" },
|
||||
]);
|
||||
await localDb.markBatchItemResult(
|
||||
batch.id,
|
||||
{ lineNumber: 1, customId: "already-done" },
|
||||
{
|
||||
id: "req_checkpointed",
|
||||
custom_id: "already-done",
|
||||
response: {
|
||||
status_code: 200,
|
||||
body: {
|
||||
id: "chatcmpl-checkpointed",
|
||||
choices: [{ message: { content: "from checkpoint" } }],
|
||||
usage: { prompt_tokens: 10, completion_tokens: 5 },
|
||||
},
|
||||
},
|
||||
}
|
||||
);
|
||||
|
||||
let callCount = 0;
|
||||
mock.method(dispatch, "dispatchBatchApiRequest", async () => {
|
||||
callCount++;
|
||||
return new Response(
|
||||
JSON.stringify({
|
||||
id: "chatcmpl-live",
|
||||
choices: [{ message: { content: "processed once" } }],
|
||||
usage: { prompt_tokens: 8, completion_tokens: 4 },
|
||||
}),
|
||||
{ status: 200, headers: { "Content-Type": "application/json" } }
|
||||
);
|
||||
});
|
||||
|
||||
await batchProcessor.processPendingBatches();
|
||||
|
||||
await waitForAllBatches();
|
||||
|
||||
const updated = await localDb.getBatch(batch.id);
|
||||
assert.strictEqual(updated?.status, "completed", "Stale batch should be recovered and completed");
|
||||
assert.strictEqual(callCount, 1, "only the unchecked item should be dispatched");
|
||||
assert.strictEqual(updated?.requestCountsCompleted, 2);
|
||||
assert.ok(updated?.outputFileId, "recovered batch should emit an output file");
|
||||
|
||||
const output = localDb.getFileContent(updated!.outputFileId!);
|
||||
assert.ok(output, "output file content should exist");
|
||||
const outputRows = output
|
||||
.toString()
|
||||
.trim()
|
||||
.split("\n")
|
||||
.map((line) => JSON.parse(line));
|
||||
assert.deepStrictEqual(
|
||||
outputRows.map((row) => row.custom_id),
|
||||
["already-done", "needs-dispatch"]
|
||||
);
|
||||
});
|
||||
|
||||
test("processPendingBatches should not replay interrupted checkpoint items", async () => {
|
||||
const file = await localDb.createFile({
|
||||
bytes: 10,
|
||||
filename: "stale_test.jsonl",
|
||||
filename: "stale_processing_checkpoint.jsonl",
|
||||
purpose: "batch_input",
|
||||
content: Buffer.from(
|
||||
JSON.stringify({
|
||||
custom_id: "in-flight",
|
||||
method: "POST",
|
||||
url: "/v1/chat/completions",
|
||||
body: { model: "gpt-4", messages: [{ role: "user", content: "hi" }] },
|
||||
}) + "\n"
|
||||
),
|
||||
});
|
||||
|
||||
const batch = await localDb.createBatch({
|
||||
endpoint: "/v1/chat/completions",
|
||||
status: "in_progress",
|
||||
inputFileId: file.id,
|
||||
completionWindow: "24h",
|
||||
inProgressAt: Math.floor(Date.now() / 1000),
|
||||
});
|
||||
await localDb.updateBatch(batch.id, { requestCountsTotal: 1 });
|
||||
await localDb.ensureBatchItemCheckpoints(batch.id, [{ lineNumber: 1, customId: "in-flight" }]);
|
||||
await localDb.markBatchItemProcessing(batch.id, { lineNumber: 1, customId: "in-flight" });
|
||||
|
||||
let callCount = 0;
|
||||
mock.method(dispatch, "dispatchBatchApiRequest", async () => {
|
||||
callCount++;
|
||||
throw new Error("interrupted checkpoint should not dispatch");
|
||||
});
|
||||
|
||||
await batchProcessor.processPendingBatches();
|
||||
|
||||
await waitForAllBatches();
|
||||
|
||||
const updated = await localDb.getBatch(batch.id);
|
||||
assert.strictEqual(updated?.status, "completed");
|
||||
assert.strictEqual(callCount, 0, "interrupted checkpoint should not be replayed");
|
||||
assert.strictEqual(updated?.requestCountsFailed, 1);
|
||||
assert.ok(updated?.errorFileId, "interrupted item should be emitted as an error row");
|
||||
|
||||
const errorOutput = localDb.getFileContent(updated!.errorFileId!);
|
||||
assert.ok(errorOutput, "error file content should exist");
|
||||
assert.match(errorOutput.toString(), /not replayed to avoid duplicate provider work/);
|
||||
});
|
||||
|
||||
test("processPendingBatches should fail stale batches without checkpoints instead of replaying", async () => {
|
||||
const file = await localDb.createFile({
|
||||
bytes: 10,
|
||||
filename: "legacy_stale_no_checkpoints.jsonl",
|
||||
purpose: "batch_input",
|
||||
content: Buffer.from(
|
||||
JSON.stringify({
|
||||
@@ -340,77 +477,20 @@ test("processPendingBatches should recover stale in_progress batches", async ()
|
||||
completionWindow: "24h",
|
||||
inProgressAt: Math.floor(Date.now() / 1000),
|
||||
});
|
||||
await localDb.updateBatch(batch.id, { requestCountsTotal: 1 });
|
||||
|
||||
let callCount = 0;
|
||||
mock.method(dispatch, "dispatchBatchApiRequest", async () => {
|
||||
return new Response(
|
||||
JSON.stringify({
|
||||
id: "chatcmpl-stale",
|
||||
choices: [{ message: { content: "recovered" } }],
|
||||
usage: { prompt_tokens: 10, completion_tokens: 5 },
|
||||
}),
|
||||
{ status: 200, headers: { "Content-Type": "application/json" } }
|
||||
);
|
||||
callCount++;
|
||||
throw new Error("legacy stale batch should not dispatch");
|
||||
});
|
||||
|
||||
await batchProcessor.processPendingBatches();
|
||||
|
||||
await waitForAllBatches();
|
||||
|
||||
const updated = await localDb.getBatch(batch.id);
|
||||
assert.strictEqual(updated?.status, "completed", "Stale batch should be recovered and completed");
|
||||
assert.ok(
|
||||
updated?.inProgressAt != null,
|
||||
"inProgressAt should be set (from fresh start, not stale)"
|
||||
);
|
||||
});
|
||||
|
||||
test("processPendingBatches should recover stale finalizing batches", async () => {
|
||||
const file = await localDb.createFile({
|
||||
bytes: 10,
|
||||
filename: "stale_finalizing.jsonl",
|
||||
purpose: "batch_input",
|
||||
content: Buffer.from(
|
||||
JSON.stringify({
|
||||
method: "POST",
|
||||
url: "/v1/chat/completions",
|
||||
body: { model: "gpt-4", messages: [{ role: "user", content: "hi" }] },
|
||||
}) + "\n"
|
||||
),
|
||||
});
|
||||
|
||||
const batch = await localDb.createBatch({
|
||||
endpoint: "/v1/chat/completions",
|
||||
status: "finalizing",
|
||||
inputFileId: file.id,
|
||||
completionWindow: "24h",
|
||||
finalizingAt: Math.floor(Date.now() / 1000),
|
||||
});
|
||||
|
||||
mock.method(dispatch, "dispatchBatchApiRequest", async () => {
|
||||
return new Response(
|
||||
JSON.stringify({
|
||||
id: "chatcmpl-stale-finalizing",
|
||||
choices: [{ message: { content: "recovered" } }],
|
||||
usage: { prompt_tokens: 10, completion_tokens: 5 },
|
||||
}),
|
||||
{ status: 200, headers: { "Content-Type": "application/json" } }
|
||||
);
|
||||
});
|
||||
|
||||
await batchProcessor.processPendingBatches();
|
||||
|
||||
await waitForAllBatches();
|
||||
|
||||
const updated = await localDb.getBatch(batch.id);
|
||||
assert.strictEqual(
|
||||
updated?.status,
|
||||
"completed",
|
||||
"Stale finalizing batch should be recovered and completed"
|
||||
);
|
||||
assert.ok(
|
||||
updated?.inProgressAt != null,
|
||||
"inProgressAt should be set by fresh start after recovery"
|
||||
);
|
||||
assert.strictEqual(updated?.status, "failed");
|
||||
assert.strictEqual(callCount, 0, "legacy stale batch should not be replayed");
|
||||
assert.match(updated?.errors?.[0]?.message ?? "", /Cannot safely recover stale batch/);
|
||||
});
|
||||
|
||||
test("processPendingBatches should respect BATCH_MAX_CONCURRENT (default 1)", async () => {
|
||||
|
||||
@@ -21,6 +21,8 @@ const {
|
||||
formatFileResponse,
|
||||
deleteFile,
|
||||
getTerminalBatches,
|
||||
ensureBatchItemCheckpoints,
|
||||
markBatchItemResult,
|
||||
} = await import("../../src/lib/localDb.ts");
|
||||
const { getDbInstance } = await import("../../src/lib/db/core.ts");
|
||||
const {
|
||||
@@ -45,6 +47,7 @@ test.afterEach(async () => {
|
||||
}
|
||||
try {
|
||||
const db = getDbInstance();
|
||||
db.prepare("DELETE FROM batch_item_checkpoints").run();
|
||||
db.prepare("DELETE FROM batches").run();
|
||||
db.prepare("DELETE FROM files").run();
|
||||
db.prepare("DELETE FROM provider_connections").run();
|
||||
@@ -695,28 +698,45 @@ test("Batch processor recovers orphaned finalizing batches during startup recove
|
||||
updateBatch(batch.id, {
|
||||
status: "finalizing",
|
||||
finalizingAt: Math.floor(Date.now() / 1000),
|
||||
requestCountsTotal: 1,
|
||||
requestCountsCompleted: 1,
|
||||
});
|
||||
ensureBatchItemCheckpoints(batch.id, [{ lineNumber: 1, customId: "req-recovery" }]);
|
||||
markBatchItemResult(
|
||||
batch.id,
|
||||
{ lineNumber: 1, customId: "req-recovery" },
|
||||
{
|
||||
id: "req_checkpointed_recovery",
|
||||
custom_id: "req-recovery",
|
||||
response: {
|
||||
status_code: 200,
|
||||
body: {
|
||||
id: "chatcmpl-mock-recovery",
|
||||
object: "chat.completion",
|
||||
model: "gpt-4o-mini",
|
||||
choices: [
|
||||
{
|
||||
index: 0,
|
||||
message: { role: "assistant", content: "recovered ok" },
|
||||
finish_reason: "stop",
|
||||
},
|
||||
],
|
||||
usage: {
|
||||
prompt_tokens: 1,
|
||||
completion_tokens: 1,
|
||||
total_tokens: 2,
|
||||
},
|
||||
},
|
||||
},
|
||||
}
|
||||
);
|
||||
|
||||
const originalFetch = globalThis.fetch;
|
||||
(globalThis as any).fetch = async () => {
|
||||
return Response.json({
|
||||
id: "chatcmpl-mock-recovery",
|
||||
object: "chat.completion",
|
||||
model: "gpt-4o-mini",
|
||||
choices: [
|
||||
{
|
||||
index: 0,
|
||||
message: { role: "assistant", content: "recovered ok" },
|
||||
finish_reason: "stop",
|
||||
},
|
||||
],
|
||||
usage: {
|
||||
prompt_tokens: 1,
|
||||
completion_tokens: 1,
|
||||
total_tokens: 2,
|
||||
},
|
||||
});
|
||||
};
|
||||
let fetchCount = 0;
|
||||
globalThis.fetch = (async () => {
|
||||
fetchCount++;
|
||||
throw new Error("checkpointed finalizing batch should not dispatch");
|
||||
}) as typeof fetch;
|
||||
|
||||
try {
|
||||
await processPendingBatches();
|
||||
@@ -724,6 +744,7 @@ test("Batch processor recovers orphaned finalizing batches during startup recove
|
||||
|
||||
const recoveredBatch = getBatch(batch.id);
|
||||
assert.strictEqual(recoveredBatch?.status, "completed");
|
||||
assert.strictEqual(fetchCount, 0);
|
||||
} finally {
|
||||
globalThis.fetch = originalFetch;
|
||||
}
|
||||
|
||||
36
tests/unit/binary-manager-extract-zip-5590.test.ts
Normal file
36
tests/unit/binary-manager-extract-zip-5590.test.ts
Normal file
@@ -0,0 +1,36 @@
|
||||
import test from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
|
||||
// #5590 — On Windows, `unzip` is not a system command (it only ships inside Git
|
||||
// for Windows' usr/bin, which Node's spawn PATH doesn't see), so installing an
|
||||
// embedded service (CLIProxyAPI) failed instantly with `spawn unzip ENOENT`.
|
||||
// Extraction must use PowerShell's built-in Expand-Archive on win32.
|
||||
const { buildExtractZipCommand } = await import("../../src/lib/versionManager/binaryManager.ts");
|
||||
|
||||
test("#5590 extractZip uses PowerShell Expand-Archive on Windows, not `unzip`", () => {
|
||||
const { command, args } = buildExtractZipCommand("win32", "C:\\tmp\\cli.zip", "C:\\tmp\\dest");
|
||||
assert.equal(command, "powershell");
|
||||
assert.notEqual(command, "unzip");
|
||||
assert.ok(args.includes("-NoProfile") && args.includes("-NonInteractive"));
|
||||
const script = args[args.length - 1];
|
||||
assert.match(
|
||||
script,
|
||||
/Expand-Archive -LiteralPath 'C:\\tmp\\cli\.zip' -DestinationPath 'C:\\tmp\\dest' -Force/
|
||||
);
|
||||
});
|
||||
|
||||
test("#5590 extractZip keeps using `unzip` on non-Windows platforms", () => {
|
||||
for (const platform of ["linux", "darwin"] as NodeJS.Platform[]) {
|
||||
const { command, args } = buildExtractZipCommand(platform, "/tmp/cli.zip", "/tmp/dest");
|
||||
assert.equal(command, "unzip");
|
||||
assert.deepEqual(args, ["-o", "/tmp/cli.zip", "-d", "/tmp/dest"]);
|
||||
}
|
||||
});
|
||||
|
||||
test("#5590 single quotes in a Windows path are escaped for the PowerShell -Command string", () => {
|
||||
const { args } = buildExtractZipCommand("win32", "C:\\o'brien\\cli.zip", "C:\\o'brien\\dest");
|
||||
const script = args[args.length - 1];
|
||||
// Each literal ' must be doubled ('') inside the single-quoted PowerShell string.
|
||||
assert.match(script, /'C:\\o''brien\\cli\.zip'/);
|
||||
assert.match(script, /'C:\\o''brien\\dest'/);
|
||||
});
|
||||
@@ -4,6 +4,7 @@ import * as bodySizeGuard from "../../src/shared/middleware/bodySizeGuard.ts";
|
||||
import {
|
||||
MAX_BODY_BYTES_AUDIO,
|
||||
MAX_BODY_BYTES_FILE,
|
||||
MAX_BODY_BYTES_LLM_API,
|
||||
getBodySizeLimit,
|
||||
checkBodySize,
|
||||
} from "../../src/shared/middleware/bodySizeGuard.ts";
|
||||
@@ -23,6 +24,14 @@ test("body size guard uses maxBodySizeMb from settings for regular API routes",
|
||||
});
|
||||
|
||||
test("body size guard keeps dedicated upload limits as lower bounds", () => {
|
||||
assert.equal(
|
||||
getBodySizeLimit("/api/v1/responses", { maxBodySizeMb: 10 }),
|
||||
MAX_BODY_BYTES_LLM_API
|
||||
);
|
||||
assert.equal(
|
||||
getBodySizeLimit("/api/v1/chat/completions", { maxBodySizeMb: 10 }),
|
||||
MAX_BODY_BYTES_LLM_API
|
||||
);
|
||||
assert.equal(
|
||||
getBodySizeLimit("/api/v1/audio/transcriptions", { maxBodySizeMb: 1 }),
|
||||
MAX_BODY_BYTES_AUDIO
|
||||
@@ -49,6 +58,32 @@ test("checkBodySize reports the configured request limit in 413 responses", asyn
|
||||
assert.match(body.error.message, /100 MB/);
|
||||
});
|
||||
|
||||
test("/api/v1/responses route guard allows 15 MB agent payloads by default", () => {
|
||||
const fifteenMb = 15 * 1024 * 1024;
|
||||
const request = new Request("http://localhost/api/v1/responses", {
|
||||
method: "POST",
|
||||
headers: { "content-length": String(fifteenMb) },
|
||||
});
|
||||
|
||||
assert.equal(checkBodySize(request, getBodySizeLimit("/api/v1/responses")), null);
|
||||
});
|
||||
|
||||
test("/api/v1/responses route guard rejects payloads above the LLM API floor", async () => {
|
||||
const tooBig = MAX_BODY_BYTES_LLM_API + 1;
|
||||
const request = new Request("http://localhost/api/v1/responses", {
|
||||
method: "POST",
|
||||
headers: { "content-length": String(tooBig) },
|
||||
});
|
||||
|
||||
const response = checkBodySize(request, getBodySizeLimit("/api/v1/responses"));
|
||||
|
||||
assert.ok(response);
|
||||
assert.equal(response.status, 413);
|
||||
const body = await response.json();
|
||||
assert.equal(body.error.code, "PAYLOAD_TOO_LARGE");
|
||||
assert.match(body.error.message, /50 MB/);
|
||||
});
|
||||
|
||||
test("/api/v1/files route has 512 MB dedicated limit floor", () => {
|
||||
const limit = getBodySizeLimit("/api/v1/files", { maxBodySizeMb: 1 });
|
||||
assert.equal(limit, MAX_BODY_BYTES_FILE);
|
||||
|
||||
@@ -8,6 +8,7 @@ import {
|
||||
mutationScoreForFile,
|
||||
measureMutationScores,
|
||||
readBaselineMutationScores,
|
||||
MUTATION_RATCHET_EPS,
|
||||
} from "../../../scripts/check/check-mutation-ratchet.mjs";
|
||||
|
||||
// ── evaluateMutationRatchet: direction UP (score can only improve) ───────────
|
||||
@@ -19,6 +20,17 @@ test("mutation ratchet flags a drop (direction up)", () => {
|
||||
assert.equal(evaluateMutationRatchet(75.0, 75.0).improved, false);
|
||||
});
|
||||
|
||||
// ── eps anti-flake: a drop WITHIN eps holds; a drop BEYOND eps regresses ──────
|
||||
test("mutation ratchet tolerates sub-eps jitter but catches real drops", () => {
|
||||
assert.ok(MUTATION_RATCHET_EPS > 0);
|
||||
// drop of 0.5pt with default eps 1.0 -> within tolerance, not a regression
|
||||
assert.equal(evaluateMutationRatchet(74.5, 75.0).regressed, false);
|
||||
// drop just past eps -> regression
|
||||
assert.equal(evaluateMutationRatchet(75.0 - MUTATION_RATCHET_EPS - 0.01, 75.0).regressed, true);
|
||||
// explicit eps argument is honored (0 eps = strict)
|
||||
assert.equal(evaluateMutationRatchet(74.99, 75.0, 0).regressed, true);
|
||||
});
|
||||
|
||||
// ── mutationScoreForFile: covered score = detected/(detected+survived),
|
||||
// NoCoverage EXCLUDED (it is a coverage gap, not a test-quality signal). ─────
|
||||
test("mutationScoreForFile computes the covered score and excludes NoCoverage", () => {
|
||||
|
||||
74
tests/unit/build/check-mutation-test-coverage.test.ts
Normal file
74
tests/unit/build/check-mutation-test-coverage.test.ts
Normal file
@@ -0,0 +1,74 @@
|
||||
import { test } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import {
|
||||
moduleFragment,
|
||||
testImportsModule,
|
||||
findCoverageDrift,
|
||||
} from "../../../scripts/check/check-mutation-test-coverage.mjs";
|
||||
|
||||
test("moduleFragment returns the 3-segment suffix without extension", () => {
|
||||
assert.equal(
|
||||
moduleFragment("open-sse/handlers/chatCore/headers.ts"),
|
||||
"handlers/chatCore/headers"
|
||||
);
|
||||
assert.equal(moduleFragment("src/sse/services/auth.ts"), "sse/services/auth");
|
||||
// shallow paths just use what is available
|
||||
assert.equal(moduleFragment("a/b.ts"), "a/b");
|
||||
});
|
||||
|
||||
test("testImportsModule matches static, dynamic and require imports of the module path", () => {
|
||||
const frag = "handlers/chatCore/headers";
|
||||
// static import-from
|
||||
assert.equal(
|
||||
testImportsModule(`import { x } from "@omniroute/open-sse/handlers/chatCore/headers";`, frag),
|
||||
true
|
||||
);
|
||||
// dynamic await import, even split across lines
|
||||
assert.equal(
|
||||
testImportsModule(`const { y } = await import(\n "../../open-sse/handlers/chatCore/headers.ts"\n);`, frag),
|
||||
true
|
||||
);
|
||||
// require()
|
||||
assert.equal(
|
||||
testImportsModule(`const z = require("../../open-sse/handlers/chatCore/headers.ts");`, frag),
|
||||
true
|
||||
);
|
||||
// unrelated module is not matched
|
||||
assert.equal(
|
||||
testImportsModule(`import { a } from "@omniroute/open-sse/handlers/chatCore/idempotency";`, frag),
|
||||
false
|
||||
);
|
||||
// the fragment appearing only in a comment (not an import string) is NOT a match
|
||||
assert.equal(
|
||||
testImportsModule(`// see handlers/chatCore/headers for details\nconst a = 1;`, frag),
|
||||
false
|
||||
);
|
||||
});
|
||||
|
||||
test("findCoverageDrift flags covering unit tests absent from tap.testFiles", () => {
|
||||
const mutate = [
|
||||
"open-sse/handlers/chatCore/headers.ts",
|
||||
"open-sse/handlers/chatCore/idempotency.ts",
|
||||
"_a_comment_entry",
|
||||
];
|
||||
const tapTestFiles = ["tests/unit/chatcore-headers.test.ts"];
|
||||
const unitTests = [
|
||||
// covers headers, already in tap -> not drift
|
||||
{ path: "tests/unit/chatcore-headers.test.ts", content: `await import("../../open-sse/handlers/chatCore/headers.ts");` },
|
||||
// covers headers, NOT in tap -> drift
|
||||
{ path: "tests/unit/no-memory-header.test.ts", content: `const { isNoMemoryRequested } = await import("../../open-sse/handlers/chatCore/headers.ts");` },
|
||||
// covers idempotency, NOT in tap -> drift
|
||||
{ path: "tests/unit/idempo.test.ts", content: `import { x } from "@omniroute/open-sse/handlers/chatCore/idempotency";` },
|
||||
// covers nothing mutated -> ignored
|
||||
{ path: "tests/unit/unrelated.test.ts", content: `import { z } from "@/lib/foo";` },
|
||||
];
|
||||
const drift = findCoverageDrift({ mutate, tapTestFiles, unitTests });
|
||||
assert.deepEqual(drift["open-sse/handlers/chatCore/headers.ts"], [
|
||||
"tests/unit/no-memory-header.test.ts",
|
||||
]);
|
||||
assert.deepEqual(drift["open-sse/handlers/chatCore/idempotency.ts"], [
|
||||
"tests/unit/idempo.test.ts",
|
||||
]);
|
||||
// comment-only mutate entries are skipped
|
||||
assert.equal("_a_comment_entry" in drift, false);
|
||||
});
|
||||
111
tests/unit/call-log-oom-unbounded-5618.test.ts
Normal file
111
tests/unit/call-log-oom-unbounded-5618.test.ts
Normal file
@@ -0,0 +1,111 @@
|
||||
import test from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import fs from "node:fs";
|
||||
import os from "node:os";
|
||||
import path from "node:path";
|
||||
|
||||
// #5618 — `cleanupExpiredLogs` → `rotateCallLogs` ran at daemon startup and used
|
||||
// unbounded `SELECT … FROM call_logs … .all()` calls. node:sqlite's
|
||||
// StatementSync.all() materializes the whole result set, so on a large
|
||||
// storage.sqlite (~170 MB+) the JS heap blew up and the process crashed before
|
||||
// binding. The two startup queries must page with LIMIT instead of loading the
|
||||
// whole table at once.
|
||||
|
||||
const TEST_DATA_DIR = fs.mkdtempSync(path.join(os.tmpdir(), "omniroute-calllogs-oom-"));
|
||||
process.env.DATA_DIR = TEST_DATA_DIR;
|
||||
process.env.CALL_LOG_RETENTION_DAYS = "3650";
|
||||
|
||||
const core = await import("../../src/lib/db/core.ts");
|
||||
const callLogs = await import("../../src/lib/usage/callLogs.ts");
|
||||
const bounded = await import("../../src/lib/usage/callLogsBoundedQueries.ts");
|
||||
|
||||
function insertCallLog(id: string, timestamp: string, artifact: string | null = null) {
|
||||
const db = core.getDbInstance();
|
||||
db.prepare(
|
||||
`INSERT INTO call_logs (id, timestamp, method, path, status, model, provider, detail_state, artifact_relpath)
|
||||
VALUES (@id, @timestamp, 'POST', '/v1/chat/completions', 200, 'openai/gpt-4.1', 'openai', 'none', @artifact)`
|
||||
).run({ id, timestamp, artifact });
|
||||
}
|
||||
|
||||
function seed(total: number, withArtifact: boolean) {
|
||||
const db = core.getDbInstance();
|
||||
const base = Date.parse("2026-01-01T00:00:00.000Z");
|
||||
db.transaction(() => {
|
||||
for (let i = 0; i < total; i++) {
|
||||
insertCallLog(
|
||||
`r-${String(i).padStart(6, "0")}`,
|
||||
new Date(base + i * 1000).toISOString(),
|
||||
withArtifact ? `2026-01/${i}.json` : null
|
||||
);
|
||||
}
|
||||
})();
|
||||
}
|
||||
|
||||
function captureSql(run: () => void): string[] {
|
||||
const db = core.getDbInstance();
|
||||
const orig = db.prepare.bind(db);
|
||||
const sqls: string[] = [];
|
||||
(db as unknown as { prepare: unknown }).prepare = (sql: string) => {
|
||||
sqls.push(sql);
|
||||
return orig(sql);
|
||||
};
|
||||
try {
|
||||
run();
|
||||
} finally {
|
||||
(db as unknown as { prepare: unknown }).prepare = orig;
|
||||
}
|
||||
return sqls;
|
||||
}
|
||||
|
||||
const unboundedSelectsOnCallLogs = (sqls: string[]) =>
|
||||
sqls.filter(
|
||||
(s) => /SELECT/i.test(s) && /\bFROM\s+call_logs\b/i.test(s) && !/LIMIT/i.test(s)
|
||||
);
|
||||
|
||||
test.beforeEach(() => {
|
||||
core.resetDbInstance();
|
||||
fs.rmSync(TEST_DATA_DIR, { recursive: true, force: true });
|
||||
fs.mkdirSync(TEST_DATA_DIR, { recursive: true });
|
||||
});
|
||||
|
||||
test.after(() => {
|
||||
core.resetDbInstance();
|
||||
fs.rmSync(TEST_DATA_DIR, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
test("#5618 collectReferencedArtifacts pages with LIMIT and collects across pages — no unbounded .all()", () => {
|
||||
const total = 6000; // > one 5000-row page → exercises the pagination loop
|
||||
seed(total, true);
|
||||
|
||||
let referenced: Set<string> | undefined;
|
||||
const sqls = captureSql(() => {
|
||||
referenced = bounded.collectReferencedArtifacts();
|
||||
});
|
||||
|
||||
assert.equal(referenced!.size, total, "every artifact path is collected across all pages");
|
||||
assert.deepEqual(
|
||||
unboundedSelectsOnCallLogs(sqls),
|
||||
[],
|
||||
`unbounded SELECT on call_logs (OOM risk): ${unboundedSelectsOnCallLogs(sqls).join("; ")}`
|
||||
);
|
||||
});
|
||||
|
||||
test("#5618 deleteCallLogsBefore selects ids with LIMIT (bounded) instead of all at once", () => {
|
||||
const total = 1200;
|
||||
seed(total, false);
|
||||
|
||||
let result: { deletedRows: number } | undefined;
|
||||
const sqls = captureSql(() => {
|
||||
result = callLogs.deleteCallLogsBefore("2030-01-01T00:00:00.000Z");
|
||||
});
|
||||
|
||||
assert.equal(result!.deletedRows, total, "all rows before the cutoff are deleted (across pages)");
|
||||
const unboundedIdSelects = sqls.filter(
|
||||
(s) => /SELECT\s+id\s+FROM\s+call_logs/i.test(s) && !/LIMIT/i.test(s)
|
||||
);
|
||||
assert.deepEqual(
|
||||
unboundedIdSelects,
|
||||
[],
|
||||
`unbounded id SELECT on call_logs (OOM risk): ${unboundedIdSelects.join("; ")}`
|
||||
);
|
||||
});
|
||||
130
tests/unit/call-logs-correlation-sort.test.ts
Normal file
130
tests/unit/call-logs-correlation-sort.test.ts
Normal file
@@ -0,0 +1,130 @@
|
||||
import test from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
|
||||
// Pure-function coverage for buildCallLogListRows (src/app/api/usage/call-logs/route.ts):
|
||||
// - merges active/completed in-memory entries with persisted DB rows
|
||||
// - sorts by priority (active > completed > persisted) then newest-first
|
||||
// - carries correlationId through so the GET handler can filter on it
|
||||
import { buildCallLogListRows } from "../../src/app/api/usage/call-logs/route.ts";
|
||||
|
||||
test("buildCallLogListRows: active requests sort before completed and persisted rows", () => {
|
||||
const now = 1_000_000;
|
||||
const rows = buildCallLogListRows({
|
||||
logs: [
|
||||
{
|
||||
id: "persisted-1",
|
||||
timestamp: new Date(now - 5_000).toISOString(),
|
||||
correlationId: "corr-a",
|
||||
},
|
||||
],
|
||||
connections: [],
|
||||
pendingDetails: [
|
||||
{
|
||||
id: "active-1",
|
||||
startedAt: now - 1_000,
|
||||
provider: "openai",
|
||||
model: "gpt-4o",
|
||||
connectionId: "conn-1",
|
||||
correlationId: "corr-a",
|
||||
},
|
||||
],
|
||||
completedDetails: [
|
||||
{
|
||||
id: "completed-1",
|
||||
startedAt: now - 3_000,
|
||||
completedAt: now - 2_000,
|
||||
provider: "anthropic",
|
||||
model: "claude",
|
||||
connectionId: "conn-2",
|
||||
correlationId: "corr-b",
|
||||
},
|
||||
],
|
||||
now,
|
||||
});
|
||||
|
||||
assert.equal(rows.length, 3);
|
||||
// active (priority 0) first, then completed (priority 1), then persisted (priority 2)
|
||||
assert.equal(rows[0].id, "active-1");
|
||||
assert.equal(rows[0].active, true);
|
||||
assert.equal(rows[1].id, "completed-1");
|
||||
assert.equal(rows[1].completed, true);
|
||||
assert.equal(rows[2].id, "persisted-1");
|
||||
});
|
||||
|
||||
test("buildCallLogListRows: within the same priority, newest timestamp sorts first", () => {
|
||||
const now = 2_000_000;
|
||||
const rows = buildCallLogListRows({
|
||||
logs: [
|
||||
{ id: "old", timestamp: new Date(now - 10_000).toISOString() },
|
||||
{ id: "new", timestamp: new Date(now - 1_000).toISOString() },
|
||||
],
|
||||
connections: [],
|
||||
pendingDetails: [],
|
||||
completedDetails: [],
|
||||
now,
|
||||
});
|
||||
|
||||
assert.deepEqual(
|
||||
rows.map((r: any) => r.id),
|
||||
["new", "old"]
|
||||
);
|
||||
});
|
||||
|
||||
test("buildCallLogListRows: in-memory entries carry correlationId for downstream filtering", () => {
|
||||
const now = 3_000_000;
|
||||
const rows = buildCallLogListRows({
|
||||
logs: [],
|
||||
connections: [],
|
||||
pendingDetails: [
|
||||
{
|
||||
id: "active-cid",
|
||||
startedAt: now - 500,
|
||||
provider: "openai",
|
||||
model: "gpt-4o",
|
||||
connectionId: "conn-1",
|
||||
correlationId: "corr-xyz",
|
||||
},
|
||||
],
|
||||
completedDetails: [
|
||||
{
|
||||
id: "completed-no-cid",
|
||||
startedAt: now - 4_000,
|
||||
completedAt: now - 3_000,
|
||||
provider: "anthropic",
|
||||
model: "claude",
|
||||
connectionId: "conn-2",
|
||||
},
|
||||
],
|
||||
now,
|
||||
});
|
||||
|
||||
const active = rows.find((r: any) => r.id === "active-cid");
|
||||
const completed = rows.find((r: any) => r.id === "completed-no-cid");
|
||||
assert.equal(active?.correlationId, "corr-xyz");
|
||||
assert.equal(completed?.correlationId, null);
|
||||
});
|
||||
|
||||
test("buildCallLogListRows: dedupes completed in-memory entries already persisted to the DB", () => {
|
||||
const now = 4_000_000;
|
||||
const rows = buildCallLogListRows({
|
||||
logs: [{ id: "dup-1", timestamp: new Date(now - 1_000).toISOString() }],
|
||||
connections: [],
|
||||
pendingDetails: [],
|
||||
completedDetails: [
|
||||
{
|
||||
id: "dup-1",
|
||||
startedAt: now - 3_000,
|
||||
completedAt: now - 2_000,
|
||||
provider: "openai",
|
||||
model: "gpt-4o",
|
||||
connectionId: "conn-1",
|
||||
},
|
||||
],
|
||||
now,
|
||||
});
|
||||
|
||||
assert.equal(rows.length, 1);
|
||||
assert.equal(rows[0].id, "dup-1");
|
||||
// persisted row wins (no `completed` flag)
|
||||
assert.equal(rows[0].completed, undefined);
|
||||
});
|
||||
161
tests/unit/calllogs-format-split.test.ts
Normal file
161
tests/unit/calllogs-format-split.test.ts
Normal file
@@ -0,0 +1,161 @@
|
||||
/**
|
||||
* Characterization + API-surface test: callLogs.ts god-file decomposition.
|
||||
*
|
||||
* The 10 pure formatting/sanitization helpers were extracted verbatim from
|
||||
* src/lib/usage/callLogs.ts into the pure leaf src/lib/usage/callLogs/format.ts
|
||||
* (no DB, no fs). The DB CRUD / disk-artifact / rotation code stays in the host.
|
||||
*
|
||||
* Verifies that:
|
||||
* 1. The pure helpers behave correctly (DB-free).
|
||||
* 2. The host callLogs.ts still exposes the FULL public API (10 functions).
|
||||
* 3. The format leaf exports its helpers directly.
|
||||
*
|
||||
* NOTE: helpers that call sanitizePII are exercised with non-PII text, since PII
|
||||
* redaction is opt-in / off by default (Hard Rule #20), so sanitizePII is a
|
||||
* pass-through here — the test pins the formatting logic, not PII behaviour.
|
||||
*/
|
||||
|
||||
import { describe, it } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
|
||||
import {
|
||||
asRecord,
|
||||
toNumber,
|
||||
toStringOrNull,
|
||||
truncateText,
|
||||
parseInlineError,
|
||||
normalizeDetailState,
|
||||
toStoredErrorSummary,
|
||||
buildRequestSummary,
|
||||
} from "../../src/lib/usage/callLogs/format.ts";
|
||||
|
||||
describe("callLogs/format — coercers", () => {
|
||||
it("asRecord keeps plain objects, rejects arrays/primitives/null", () => {
|
||||
const o = { a: 1 };
|
||||
assert.equal(asRecord(o), o);
|
||||
assert.deepEqual(asRecord([1, 2]), {});
|
||||
assert.deepEqual(asRecord("x"), {});
|
||||
assert.deepEqual(asRecord(null), {});
|
||||
});
|
||||
|
||||
it("toNumber parses finite numbers and numeric strings, else 0", () => {
|
||||
assert.equal(toNumber(5), 5);
|
||||
assert.equal(toNumber("42"), 42);
|
||||
assert.equal(toNumber(" 7 "), 7);
|
||||
assert.equal(toNumber("abc"), 0);
|
||||
assert.equal(toNumber(Infinity), 0);
|
||||
assert.equal(toNumber(null), 0);
|
||||
});
|
||||
|
||||
it("toStringOrNull keeps non-blank strings, else null", () => {
|
||||
assert.equal(toStringOrNull("hi"), "hi");
|
||||
assert.equal(toStringOrNull(" "), null);
|
||||
assert.equal(toStringOrNull(5), null);
|
||||
});
|
||||
|
||||
it("truncateText slices only when over the limit", () => {
|
||||
assert.equal(truncateText("hello", 10), "hello");
|
||||
assert.equal(truncateText("hello", 3), "hel");
|
||||
});
|
||||
});
|
||||
|
||||
describe("callLogs/format — parseInlineError", () => {
|
||||
it("returns null for non-string / blank", () => {
|
||||
assert.equal(parseInlineError(null), null);
|
||||
assert.equal(parseInlineError(" "), null);
|
||||
});
|
||||
it("parses valid JSON, falls back to the raw string on invalid JSON", () => {
|
||||
assert.deepEqual(parseInlineError('{"code":429}'), { code: 429 });
|
||||
assert.equal(parseInlineError("not json"), "not json");
|
||||
});
|
||||
});
|
||||
|
||||
describe("callLogs/format — normalizeDetailState", () => {
|
||||
it("passes through the four known states", () => {
|
||||
for (const s of ["ready", "missing", "corrupt", "legacy-inline"]) {
|
||||
assert.equal(normalizeDetailState(s), s);
|
||||
}
|
||||
});
|
||||
it("maps anything else to 'none'", () => {
|
||||
assert.equal(normalizeDetailState("bogus"), "none");
|
||||
assert.equal(normalizeDetailState(undefined), "none");
|
||||
});
|
||||
});
|
||||
|
||||
describe("callLogs/format — toStoredErrorSummary", () => {
|
||||
it("returns null for nullish error", () => {
|
||||
assert.equal(toStoredErrorSummary(null), null);
|
||||
assert.equal(toStoredErrorSummary(undefined), null);
|
||||
});
|
||||
it("stringifies a plain-text error (PII pass-through by default)", () => {
|
||||
assert.equal(toStoredErrorSummary("boom"), "boom");
|
||||
});
|
||||
it("JSON-stringifies an Error object's sanitized shape", () => {
|
||||
const out = toStoredErrorSummary(new Error("kaboom"));
|
||||
assert.equal(typeof out, "string");
|
||||
assert.ok(out.includes("kaboom"));
|
||||
assert.ok(out.includes("message"));
|
||||
});
|
||||
});
|
||||
|
||||
describe("callLogs/format — buildRequestSummary", () => {
|
||||
it("returns null for non-search request types", () => {
|
||||
assert.equal(buildRequestSummary("chat", { query: "x" }), null);
|
||||
assert.equal(buildRequestSummary(null, { query: "x" }), null);
|
||||
});
|
||||
it("summarizes a search request's query + non-query/provider filters", () => {
|
||||
const out = buildRequestSummary("search", { query: "cats", provider: "p", topK: 5 });
|
||||
assert.equal(typeof out, "string");
|
||||
const parsed = JSON.parse(out);
|
||||
assert.equal(parsed.query, "cats");
|
||||
assert.deepEqual(parsed.filters, { topK: 5 });
|
||||
assert.equal("provider" in (parsed.filters ?? {}), false);
|
||||
});
|
||||
it("returns null when a search request carries no summarizable fields", () => {
|
||||
assert.equal(buildRequestSummary("search", {}), null);
|
||||
assert.equal(buildRequestSummary("search", { provider: "only" }), null);
|
||||
});
|
||||
});
|
||||
|
||||
// ── host public API surface ──────────────────────────────────────────────────
|
||||
|
||||
const host = await import("../../src/lib/usage/callLogs.ts");
|
||||
|
||||
describe("callLogs.ts public API surface (10 functions)", () => {
|
||||
const expected = [
|
||||
"cleanupOrphanCallLogFiles",
|
||||
"cleanupOverflowCallLogFiles",
|
||||
"deleteCallLogsBefore",
|
||||
"exportCallLogsSince",
|
||||
"getCallLogById",
|
||||
"getCallLogs",
|
||||
"rotateCallLogs",
|
||||
"saveCallLog",
|
||||
"scheduleCallLogRotation",
|
||||
"trimCallLogsToMaxRows",
|
||||
];
|
||||
for (const name of expected) {
|
||||
it(`exposes ${name} as a function`, () => {
|
||||
assert.equal(typeof host[name], "function", `${name} must be a function on the host`);
|
||||
});
|
||||
}
|
||||
it("loses no public function in the split", () => {
|
||||
const missing = expected.filter((n) => typeof host[n] !== "function");
|
||||
assert.deepEqual(missing, [], `missing: ${missing.join(", ")}`);
|
||||
});
|
||||
});
|
||||
|
||||
describe("format.ts exports its helpers directly", () => {
|
||||
it("the moved helpers are functions on the leaf", async () => {
|
||||
const fmt = await import("../../src/lib/usage/callLogs/format.ts");
|
||||
for (const fn of [
|
||||
"asRecord",
|
||||
"toNumber",
|
||||
"sanitizeErrorForLog",
|
||||
"protectPipelinePayloads",
|
||||
"buildRequestSummary",
|
||||
]) {
|
||||
assert.equal(typeof fmt[fn], "function", fn);
|
||||
}
|
||||
});
|
||||
});
|
||||
190
tests/unit/catalog-helpers-extraction.test.ts
Normal file
190
tests/unit/catalog-helpers-extraction.test.ts
Normal file
@@ -0,0 +1,190 @@
|
||||
/**
|
||||
* Characterization tests for the catalog helper extraction (BLOCO E2 of the
|
||||
* god-files campaign). The pure, dependency-free helpers and the request/vision/
|
||||
* provider-map helpers were lifted out of `src/app/api/v1/models/catalog.ts`
|
||||
* verbatim into cohesive leaf modules so the catalog host shrinks toward the
|
||||
* file-size cap. These tests pin the behavior of each extracted helper AND guard
|
||||
* the host module's preserved public API (re-exports relied on by other tests).
|
||||
*/
|
||||
import test from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
|
||||
import {
|
||||
isPositiveFiniteNumber,
|
||||
parseJsonStringArray,
|
||||
intersectStringArrays,
|
||||
minKnownNumber,
|
||||
maybeOmitCatalogModelName,
|
||||
} from "../../src/app/api/v1/models/catalogHelpers.ts";
|
||||
import {
|
||||
qualifyOpenRouterModelId,
|
||||
normalizeOpenRouterModalities,
|
||||
getOpenRouterModelType,
|
||||
isZeroPrice,
|
||||
isOpenRouterFreeModel,
|
||||
getOpenRouterDisplayName,
|
||||
} from "../../src/app/api/v1/models/catalogOpenrouter.ts";
|
||||
import {
|
||||
isVisionModelId,
|
||||
getVisionCapabilityFields,
|
||||
getCustomVisionCapabilityFields,
|
||||
} from "../../src/app/api/v1/models/catalogVision.ts";
|
||||
import {
|
||||
FALLBACK_ALIAS_TO_PROVIDER,
|
||||
buildAliasMaps,
|
||||
} from "../../src/app/api/v1/models/catalogProviderMaps.ts";
|
||||
import { isCodexModelCatalogClient } from "../../src/app/api/v1/models/catalogRequest.ts";
|
||||
|
||||
test("catalogHelpers: isPositiveFiniteNumber", () => {
|
||||
assert.equal(isPositiveFiniteNumber(1), true);
|
||||
assert.equal(isPositiveFiniteNumber(0), false);
|
||||
assert.equal(isPositiveFiniteNumber(-1), false);
|
||||
assert.equal(isPositiveFiniteNumber(Number.NaN), false);
|
||||
assert.equal(isPositiveFiniteNumber(Infinity), false);
|
||||
assert.equal(isPositiveFiniteNumber("5"), false);
|
||||
assert.equal(isPositiveFiniteNumber(undefined), false);
|
||||
});
|
||||
|
||||
test("catalogHelpers: parseJsonStringArray keeps only non-empty strings", () => {
|
||||
assert.deepEqual(parseJsonStringArray('["a","b"]'), ["a", "b"]);
|
||||
assert.deepEqual(parseJsonStringArray('["a", 1, "", "c"]'), ["a", "c"]);
|
||||
assert.deepEqual(parseJsonStringArray("not json"), []);
|
||||
assert.deepEqual(parseJsonStringArray(""), []);
|
||||
assert.deepEqual(parseJsonStringArray('{"x":1}'), []);
|
||||
assert.deepEqual(parseJsonStringArray(null), []);
|
||||
});
|
||||
|
||||
test("catalogHelpers: intersectStringArrays (dedup + common)", () => {
|
||||
assert.deepEqual(
|
||||
intersectStringArrays([
|
||||
["a", "b", "c"],
|
||||
["b", "c", "d"],
|
||||
]),
|
||||
["b", "c"]
|
||||
);
|
||||
assert.deepEqual(
|
||||
intersectStringArrays([
|
||||
["a", "a", "b"],
|
||||
["a", "b"],
|
||||
]),
|
||||
["a", "b"]
|
||||
);
|
||||
assert.deepEqual(intersectStringArrays([]), []);
|
||||
assert.deepEqual(intersectStringArrays([["a"], []]), []);
|
||||
});
|
||||
|
||||
test("catalogHelpers: minKnownNumber ignores non-positive/unknown", () => {
|
||||
assert.equal(minKnownNumber([3, 1, 2]), 1);
|
||||
assert.equal(minKnownNumber([undefined, 0, -5, 7]), 7);
|
||||
assert.equal(minKnownNumber([undefined, undefined]), undefined);
|
||||
assert.equal(minKnownNumber([]), undefined);
|
||||
});
|
||||
|
||||
test("catalogHelpers: maybeOmitCatalogModelName drops name only when excluding", () => {
|
||||
const model = { id: "x", name: "X" };
|
||||
assert.deepEqual(maybeOmitCatalogModelName(model, true), { id: "x", name: "X" });
|
||||
assert.deepEqual(maybeOmitCatalogModelName(model, false), { id: "x" });
|
||||
// No `name` key -> returned unchanged regardless of flag.
|
||||
assert.deepEqual(maybeOmitCatalogModelName({ id: "y" }, false), { id: "y" });
|
||||
});
|
||||
|
||||
test("catalogOpenrouter: qualify + modality normalization + type", () => {
|
||||
assert.equal(qualifyOpenRouterModelId("foo/bar"), "openrouter/foo/bar");
|
||||
assert.equal(qualifyOpenRouterModelId("openrouter/foo"), "openrouter/foo");
|
||||
assert.deepEqual(normalizeOpenRouterModalities(["text", 1, "", "image"]), ["text", "image"]);
|
||||
assert.deepEqual(normalizeOpenRouterModalities("nope"), []);
|
||||
assert.equal(getOpenRouterModelType(["text"], ["image"]), "image");
|
||||
assert.equal(getOpenRouterModelType(["text"], ["audio"]), "audio");
|
||||
assert.equal(getOpenRouterModelType(["text"], ["embedding"]), "embedding");
|
||||
assert.equal(getOpenRouterModelType(["text"], ["text"]), "chat");
|
||||
});
|
||||
|
||||
test("catalogOpenrouter: free-model detection + display name", () => {
|
||||
assert.equal(isZeroPrice(0), true);
|
||||
assert.equal(isZeroPrice("0"), true);
|
||||
assert.equal(isZeroPrice("0.5"), false);
|
||||
assert.equal(isZeroPrice("x"), false);
|
||||
assert.equal(isOpenRouterFreeModel({ id: "z/model:free" }), true);
|
||||
assert.equal(
|
||||
isOpenRouterFreeModel({ id: "z/model", pricing: { prompt: "0", completion: "0" } }),
|
||||
true
|
||||
);
|
||||
assert.equal(
|
||||
isOpenRouterFreeModel({ id: "z/model", pricing: { prompt: "0.1", completion: "0" } }),
|
||||
false
|
||||
);
|
||||
assert.equal(getOpenRouterDisplayName({ id: "z/m", name: "Some Model" }), "Some Model");
|
||||
assert.equal(
|
||||
getOpenRouterDisplayName({ id: "z/m:free", name: "Free Model" }),
|
||||
"Free Model (Grátis)"
|
||||
);
|
||||
// Already labelled "grátis" is not double-tagged.
|
||||
assert.equal(
|
||||
getOpenRouterDisplayName({ id: "z/m:free", name: "Modelo Grátis" }),
|
||||
"Modelo Grátis"
|
||||
);
|
||||
});
|
||||
|
||||
test("catalogVision: re-exports isVisionModelId and derives capability fields", () => {
|
||||
assert.equal(typeof isVisionModelId, "function");
|
||||
// id-based heuristic
|
||||
const visionFields = getVisionCapabilityFields("gpt-4o");
|
||||
assert.ok(visionFields, "gpt-4o must be detected as vision-capable");
|
||||
assert.equal(visionFields?.capabilities.vision, true);
|
||||
assert.equal(getVisionCapabilityFields("kimi-k2"), null);
|
||||
});
|
||||
|
||||
test("catalogVision: getCustomVisionCapabilityFields honours explicit flag", () => {
|
||||
// explicit true wins
|
||||
assert.equal(
|
||||
getCustomVisionCapabilityFields({ supportsVision: true }, "x")?.capabilities.vision,
|
||||
true
|
||||
);
|
||||
// explicit false wins even for a vision-looking id
|
||||
assert.equal(getCustomVisionCapabilityFields({ supportsVision: false }, "gpt-4o"), null);
|
||||
// no flag -> falls back to id heuristic
|
||||
assert.equal(getCustomVisionCapabilityFields(null, "gpt-4o")?.capabilities.vision, true);
|
||||
assert.equal(getCustomVisionCapabilityFields(null, "kimi-k2"), null);
|
||||
});
|
||||
|
||||
test("catalogProviderMaps: buildAliasMaps seeds the fallback aliases", () => {
|
||||
const { aliasToProviderId, providerIdToAlias } = buildAliasMaps();
|
||||
assert.equal(typeof aliasToProviderId, "object");
|
||||
assert.equal(typeof providerIdToAlias, "object");
|
||||
// Fallback entries are always present even if upstream maps loaded partially.
|
||||
for (const [alias, providerId] of Object.entries(FALLBACK_ALIAS_TO_PROVIDER)) {
|
||||
assert.equal(aliasToProviderId[alias], providerId, `alias ${alias} -> ${providerId}`);
|
||||
}
|
||||
assert.equal(aliasToProviderId["cx"], "codex");
|
||||
assert.equal(aliasToProviderId["kr"], "kiro");
|
||||
});
|
||||
|
||||
test("catalogRequest: isCodexModelCatalogClient detects codex originator/user-agent", () => {
|
||||
const byOriginator = new Request("https://x/v1/models", {
|
||||
headers: { originator: "codex_cli_rs" },
|
||||
});
|
||||
assert.equal(isCodexModelCatalogClient(byOriginator), true);
|
||||
const byUserAgent = new Request("https://x/v1/models", {
|
||||
headers: { "user-agent": "codex_exec/0.137" },
|
||||
});
|
||||
assert.equal(isCodexModelCatalogClient(byUserAgent), true);
|
||||
const other = new Request("https://x/v1/models", {
|
||||
headers: { "user-agent": "curl/8.0" },
|
||||
});
|
||||
assert.equal(isCodexModelCatalogClient(other), false);
|
||||
});
|
||||
|
||||
test("host catalog.ts preserves its public API after the extraction", async () => {
|
||||
const host = (await import("../../src/app/api/v1/models/catalog.ts")) as Record<string, unknown>;
|
||||
assert.equal(typeof host.getUnifiedModelsResponse, "function", "getUnifiedModelsResponse export");
|
||||
assert.equal(
|
||||
typeof host.getCustomVisionCapabilityFields,
|
||||
"function",
|
||||
"getCustomVisionCapabilityFields re-export (llm-selector-custom-vision-models.test.ts)"
|
||||
);
|
||||
assert.equal(
|
||||
typeof host.isVisionModelId,
|
||||
"function",
|
||||
"isVisionModelId re-export (vision-detection-consistency.test.ts)"
|
||||
);
|
||||
});
|
||||
@@ -43,19 +43,31 @@ test("kmca still exposes kimi-k2.6 and kimi-k2.6-thinking alongside the new mode
|
||||
|
||||
// ── Bug #3 / issue #3931: qwen-web in PROVIDER_MODELS_CONFIG ──────────────────
|
||||
|
||||
const ROUTE_FILE = path.join("src", "app", "api", "providers", "[id]", "models", "route.ts");
|
||||
// PROVIDER_MODELS_CONFIG was extracted from the discovery route into the
|
||||
// discovery/ leaf (refactor: split provider-models discovery route). The
|
||||
// source-guard follows the config to its new home.
|
||||
const CONFIG_FILE = path.join(
|
||||
"src",
|
||||
"app",
|
||||
"api",
|
||||
"providers",
|
||||
"[id]",
|
||||
"models",
|
||||
"discovery",
|
||||
"providerModelsConfig.ts"
|
||||
);
|
||||
|
||||
test("PROVIDER_MODELS_CONFIG contains a qwen-web entry (issue #3931 bug #3)", () => {
|
||||
const src = fs.readFileSync(ROUTE_FILE, "utf-8");
|
||||
const src = fs.readFileSync(CONFIG_FILE, "utf-8");
|
||||
assert.match(
|
||||
src,
|
||||
/"qwen-web"\s*:/,
|
||||
'"qwen-web" key missing from PROVIDER_MODELS_CONFIG in models/route.ts'
|
||||
'"qwen-web" key missing from PROVIDER_MODELS_CONFIG in discovery/providerModelsConfig.ts'
|
||||
);
|
||||
});
|
||||
|
||||
test("qwen-web PROVIDER_MODELS_CONFIG entry targets chat.qwen.ai/api/v2/models", () => {
|
||||
const src = fs.readFileSync(ROUTE_FILE, "utf-8");
|
||||
const src = fs.readFileSync(CONFIG_FILE, "utf-8");
|
||||
assert.match(
|
||||
src,
|
||||
/chat\.qwen\.ai\/api\/v2\/models/,
|
||||
|
||||
@@ -30,6 +30,7 @@ const providerModelsRoute = await import("../../src/app/api/providers/[id]/model
|
||||
const originalFetch = globalThis.fetch;
|
||||
const originalFlag = process.env.ENABLE_CC_COMPATIBLE_PROVIDER;
|
||||
const originalAllowPrivateProviderUrls = process.env.OMNIROUTE_ALLOW_PRIVATE_PROVIDER_URLS;
|
||||
const originalAllowLocalProviderUrls = process.env.OMNIROUTE_ALLOW_LOCAL_PROVIDER_URLS;
|
||||
|
||||
async function resetStorage() {
|
||||
core.resetDbInstance();
|
||||
@@ -49,6 +50,11 @@ test.afterEach(async () => {
|
||||
} else {
|
||||
process.env.OMNIROUTE_ALLOW_PRIVATE_PROVIDER_URLS = originalAllowPrivateProviderUrls;
|
||||
}
|
||||
if (originalAllowLocalProviderUrls === undefined) {
|
||||
delete process.env.OMNIROUTE_ALLOW_LOCAL_PROVIDER_URLS;
|
||||
} else {
|
||||
process.env.OMNIROUTE_ALLOW_LOCAL_PROVIDER_URLS = originalAllowLocalProviderUrls;
|
||||
}
|
||||
await resetStorage();
|
||||
});
|
||||
|
||||
@@ -64,6 +70,11 @@ test.after(() => {
|
||||
} else {
|
||||
process.env.OMNIROUTE_ALLOW_PRIVATE_PROVIDER_URLS = originalAllowPrivateProviderUrls;
|
||||
}
|
||||
if (originalAllowLocalProviderUrls === undefined) {
|
||||
delete process.env.OMNIROUTE_ALLOW_LOCAL_PROVIDER_URLS;
|
||||
} else {
|
||||
process.env.OMNIROUTE_ALLOW_LOCAL_PROVIDER_URLS = originalAllowLocalProviderUrls;
|
||||
}
|
||||
core.resetDbInstance();
|
||||
fs.rmSync(TEST_DATA_DIR, { recursive: true, force: true });
|
||||
});
|
||||
@@ -880,8 +891,9 @@ test("provider-nodes validate route rejects invalid JSON and schema errors", asy
|
||||
assert.equal(invalidBodyPayload.error.details.length >= 1, true);
|
||||
});
|
||||
|
||||
test("provider-nodes validate route blocks private provider hosts before fetch", async () => {
|
||||
test("provider-nodes validate route allows local provider hosts by default", async () => {
|
||||
delete process.env.OMNIROUTE_ALLOW_PRIVATE_PROVIDER_URLS;
|
||||
delete process.env.OMNIROUTE_ALLOW_LOCAL_PROVIDER_URLS;
|
||||
|
||||
let called = false;
|
||||
globalThis.fetch = async () => {
|
||||
@@ -900,9 +912,35 @@ test("provider-nodes validate route blocks private provider hosts before fetch",
|
||||
})
|
||||
);
|
||||
|
||||
assert.equal(response.status, 200);
|
||||
assert.deepEqual(await response.json(), { valid: true, error: null });
|
||||
assert.equal(called, true);
|
||||
});
|
||||
|
||||
test("provider-nodes validate route blocks cloud metadata provider hosts before fetch", async () => {
|
||||
delete process.env.OMNIROUTE_ALLOW_PRIVATE_PROVIDER_URLS;
|
||||
delete process.env.OMNIROUTE_ALLOW_LOCAL_PROVIDER_URLS;
|
||||
|
||||
let called = false;
|
||||
globalThis.fetch = async () => {
|
||||
called = true;
|
||||
return Response.json({ data: [] });
|
||||
};
|
||||
|
||||
const response = await providerNodesValidateRoute.POST(
|
||||
new Request("http://localhost/api/provider-nodes/validate", {
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify({
|
||||
baseUrl: "http://169.254.169.254/latest/meta-data",
|
||||
apiKey: "sk-metadata-test",
|
||||
}),
|
||||
})
|
||||
);
|
||||
|
||||
assert.equal(response.status, 503);
|
||||
assert.deepEqual(await response.json(), {
|
||||
error: "Blocked private or local provider URL",
|
||||
error: "Blocked cloud-metadata endpoint",
|
||||
});
|
||||
assert.equal(called, false);
|
||||
const auditEntries = compliance.getAuditLog({
|
||||
@@ -914,8 +952,8 @@ test("provider-nodes validate route blocks private provider hosts before fetch",
|
||||
assert.equal(auditEntries[0].status, "blocked");
|
||||
assert.deepEqual(auditEntries[0].metadata, {
|
||||
route: "/api/provider-nodes/validate",
|
||||
reason: "Blocked private or local provider URL",
|
||||
baseUrl: "http://127.0.0.1:11434/v1",
|
||||
reason: "Blocked cloud-metadata endpoint",
|
||||
baseUrl: "http://169.254.169.254/latest/meta-data",
|
||||
});
|
||||
});
|
||||
|
||||
|
||||
136
tests/unit/chat-body-admission.test.ts
Normal file
136
tests/unit/chat-body-admission.test.ts
Normal file
@@ -0,0 +1,136 @@
|
||||
// #5152: heap-pressure-aware admission for POST /v1/chat/completions.
|
||||
//
|
||||
// The homelab OOM crash-loop was a per-request transient explosion — a large coding-agent
|
||||
// compact body cloned/parsed/fanned-out across a combo allocates hundreds of MB of JS
|
||||
// objects, and concurrent compacts stack past the V8 heap ceiling, OOM-crashing the whole
|
||||
// process. A fixed size cap is wrong (those bodies are legitimate); instead we shed a large
|
||||
// body with 503 only when the heap is ALREADY under pressure, and 413 only for pathological
|
||||
// bodies. These tests pin that policy and the cheap fast-path for ordinary traffic.
|
||||
import test from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
|
||||
const {
|
||||
evaluateChatBodyAdmission,
|
||||
checkChatAdmission,
|
||||
CHAT_LARGE_BODY_BYTES,
|
||||
} = await import("../../src/shared/middleware/chatBodyAdmission.ts");
|
||||
|
||||
const MB = 1024 * 1024;
|
||||
const HEAP_LIMIT = 3072 * MB; // mirror the homelab --max-old-space-size=3072
|
||||
|
||||
test("small body is always admitted, even under heap pressure", () => {
|
||||
const decision = evaluateChatBodyAdmission({
|
||||
contentLength: 10 * 1024,
|
||||
heapUsedBytes: 0.95 * HEAP_LIMIT,
|
||||
heapLimitBytes: HEAP_LIMIT,
|
||||
});
|
||||
assert.equal(decision.admit, true);
|
||||
});
|
||||
|
||||
test("unknown content-length is admitted", () => {
|
||||
const decision = evaluateChatBodyAdmission({
|
||||
contentLength: null,
|
||||
heapUsedBytes: 0.95 * HEAP_LIMIT,
|
||||
heapLimitBytes: HEAP_LIMIT,
|
||||
});
|
||||
assert.equal(decision.admit, true);
|
||||
});
|
||||
|
||||
test("large body on a HEALTHY heap is admitted (normal case — guard is invisible)", () => {
|
||||
const decision = evaluateChatBodyAdmission({
|
||||
contentLength: 746578, // the production compact body
|
||||
heapUsedBytes: 0.11 * HEAP_LIMIT, // ~350 MB live baseline
|
||||
heapLimitBytes: HEAP_LIMIT,
|
||||
});
|
||||
assert.equal(decision.admit, true);
|
||||
});
|
||||
|
||||
test("large body under heap PRESSURE is shed with 503 + retry", () => {
|
||||
const decision = evaluateChatBodyAdmission({
|
||||
contentLength: 746578,
|
||||
heapUsedBytes: 0.8 * HEAP_LIMIT,
|
||||
heapLimitBytes: HEAP_LIMIT,
|
||||
});
|
||||
assert.equal(decision.admit, false);
|
||||
assert.equal(decision.status, 503);
|
||||
assert.equal(decision.code, "heap_pressure");
|
||||
});
|
||||
|
||||
test("pathological body is rejected with 413 regardless of heap state", () => {
|
||||
const decision = evaluateChatBodyAdmission({
|
||||
contentLength: 200 * MB,
|
||||
heapUsedBytes: 0.05 * HEAP_LIMIT, // heap totally healthy
|
||||
heapLimitBytes: HEAP_LIMIT,
|
||||
});
|
||||
assert.equal(decision.admit, false);
|
||||
assert.equal(decision.status, 413);
|
||||
});
|
||||
|
||||
test("shed threshold is exactly at the ratio boundary", () => {
|
||||
const atBoundary = evaluateChatBodyAdmission({
|
||||
contentLength: CHAT_LARGE_BODY_BYTES,
|
||||
heapUsedBytes: 0.75 * HEAP_LIMIT,
|
||||
heapLimitBytes: HEAP_LIMIT,
|
||||
shedRatio: 0.75,
|
||||
});
|
||||
assert.equal(atBoundary.admit, false, "at the ratio it sheds");
|
||||
|
||||
const justBelow = evaluateChatBodyAdmission({
|
||||
contentLength: CHAT_LARGE_BODY_BYTES,
|
||||
heapUsedBytes: 0.7499 * HEAP_LIMIT,
|
||||
heapLimitBytes: HEAP_LIMIT,
|
||||
shedRatio: 0.75,
|
||||
});
|
||||
assert.equal(justBelow.admit, true, "just below the ratio it admits");
|
||||
});
|
||||
|
||||
test("checkChatAdmission returns null for a small-body request (fast path, no heap sample)", () => {
|
||||
const request = new Request("http://x/v1/chat/completions", {
|
||||
method: "POST",
|
||||
headers: { "content-length": "1024" },
|
||||
});
|
||||
assert.equal(checkChatAdmission(request), null);
|
||||
});
|
||||
|
||||
test("checkChatAdmission sheds a large body with 503 + Retry-After under injected heap pressure", async () => {
|
||||
const request = new Request("http://x/v1/chat/completions", {
|
||||
method: "POST",
|
||||
headers: { "content-length": "746578" },
|
||||
});
|
||||
const res = checkChatAdmission(request, {
|
||||
heapUsedBytes: 0.9 * HEAP_LIMIT,
|
||||
heapLimitBytes: HEAP_LIMIT,
|
||||
});
|
||||
assert.ok(res, "expected a 503 rejection");
|
||||
assert.equal(res.status, 503);
|
||||
assert.equal(res.headers.get("Retry-After"), "2");
|
||||
const body = await res.json();
|
||||
assert.equal(body.error.code, "heap_pressure");
|
||||
assert.ok(!String(body.error.message).includes("at /"), "must not leak a stack trace");
|
||||
});
|
||||
|
||||
test("checkChatAdmission admits a large body when injected heap is healthy", () => {
|
||||
const request = new Request("http://x/v1/chat/completions", {
|
||||
method: "POST",
|
||||
headers: { "content-length": "746578" },
|
||||
});
|
||||
const res = checkChatAdmission(request, {
|
||||
heapUsedBytes: 0.1 * HEAP_LIMIT,
|
||||
heapLimitBytes: HEAP_LIMIT,
|
||||
});
|
||||
assert.equal(res, null);
|
||||
});
|
||||
|
||||
test("checkChatAdmission 413 response does not leak a stack trace", async () => {
|
||||
// Force a pathological size via the hard-cap env so we exercise the real wrapper.
|
||||
const request = new Request("http://x/v1/chat/completions", {
|
||||
method: "POST",
|
||||
headers: { "content-length": String(500 * MB) },
|
||||
});
|
||||
const res = checkChatAdmission(request);
|
||||
assert.ok(res, "expected a rejection Response");
|
||||
assert.equal(res.status, 413);
|
||||
const body = await res.json();
|
||||
assert.equal(body.error.code, "PAYLOAD_TOO_LARGE");
|
||||
assert.ok(!String(body.error.message).includes("at /"), "must not leak a stack trace");
|
||||
});
|
||||
40
tests/unit/chat-build-client-raw-request.test.ts
Normal file
40
tests/unit/chat-build-client-raw-request.test.ts
Normal file
@@ -0,0 +1,40 @@
|
||||
// #5152: handleChat used to clone the body twice for logging — once into a local
|
||||
// `rawClientBody` and again inside buildClientRawRequest — doubling per-request heap
|
||||
// residency on the hot path (and cloning even when clientRawRequest was already provided).
|
||||
// The outer clone was removed; buildClientRawRequest still owns the (single) deep clone.
|
||||
// These tests pin that the logging snapshot remains an ISOLATED copy so dropping the outer
|
||||
// clone cannot leak a shared reference that downstream mutation would corrupt.
|
||||
import test from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { buildClientRawRequest } from "../../src/sse/handlers/chat.ts";
|
||||
|
||||
function req(body: unknown) {
|
||||
return new Request("http://x/v1/chat/completions", {
|
||||
method: "POST",
|
||||
headers: { "content-type": "application/json" },
|
||||
body: JSON.stringify(body),
|
||||
});
|
||||
}
|
||||
|
||||
test("buildClientRawRequest deep-clones the body (not the same reference)", () => {
|
||||
const body = { model: "m", messages: [{ role: "user", content: "hi" }] };
|
||||
const out = buildClientRawRequest(req(body), body);
|
||||
assert.deepEqual(out.body, body);
|
||||
assert.notEqual(out.body, body, "must be a distinct object");
|
||||
assert.notEqual(out.body.messages, body.messages, "nested arrays must be cloned too");
|
||||
});
|
||||
|
||||
test("mutating the original body after capture does not corrupt the snapshot", () => {
|
||||
const body = { model: "m", messages: [{ role: "user", content: "original" }] };
|
||||
const out = buildClientRawRequest(req(body), body);
|
||||
body.messages[0].content = "MUTATED";
|
||||
body.messages.push({ role: "user", content: "added" });
|
||||
assert.equal(out.body.messages.length, 1, "snapshot length is frozen at capture time");
|
||||
assert.equal(out.body.messages[0].content, "original", "snapshot content is isolated");
|
||||
});
|
||||
|
||||
test("endpoint and headers are captured from the request", () => {
|
||||
const out = buildClientRawRequest(req({ model: "m" }), { model: "m" });
|
||||
assert.equal(out.endpoint, "/v1/chat/completions");
|
||||
assert.equal(out.headers["content-type"], "application/json");
|
||||
});
|
||||
@@ -70,6 +70,52 @@ function makeRequest(extraHeaders = {}) {
|
||||
});
|
||||
}
|
||||
|
||||
function makeStreamingRequest(extraHeaders = {}) {
|
||||
return new Request("http://localhost/v1/chat/completions", {
|
||||
method: "POST",
|
||||
headers: {
|
||||
"Content-Type": "application/json",
|
||||
Accept: "text/event-stream",
|
||||
...extraHeaders,
|
||||
},
|
||||
body: JSON.stringify({
|
||||
model: "openai/gpt-4.1",
|
||||
messages: [{ role: "user", content: "Stream OK only." }],
|
||||
max_tokens: 16,
|
||||
stream: true,
|
||||
temperature: 0,
|
||||
}),
|
||||
});
|
||||
}
|
||||
|
||||
function makeRequestWithoutStreamFlag(extraHeaders = {}) {
|
||||
return new Request("http://localhost/v1/chat/completions", {
|
||||
method: "POST",
|
||||
headers: {
|
||||
"Content-Type": "application/json",
|
||||
...extraHeaders,
|
||||
},
|
||||
body: JSON.stringify({
|
||||
model: "openai/gpt-4.1",
|
||||
messages: [{ role: "user", content: "Reply with OMITTED STREAM OK only." }],
|
||||
max_tokens: 16,
|
||||
temperature: 0,
|
||||
}),
|
||||
});
|
||||
}
|
||||
|
||||
async function readAll(response: Response): Promise<string> {
|
||||
const reader = response.body!.getReader();
|
||||
const decoder = new TextDecoder();
|
||||
let out = "";
|
||||
while (true) {
|
||||
const { done, value } = await reader.read();
|
||||
if (done) break;
|
||||
if (value) out += decoder.decode(value, { stream: true });
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
test.beforeEach(async () => {
|
||||
globalThis.fetch = originalFetch;
|
||||
await resetStorage();
|
||||
@@ -200,6 +246,71 @@ test("combo live test bypasses semantic cache and forces a fresh upstream reques
|
||||
}
|
||||
});
|
||||
|
||||
test("chat completions route emits early keepalive while waiting for stream readiness", async () => {
|
||||
await seedHealthyConnection();
|
||||
|
||||
globalThis.fetch = async () => {
|
||||
await new Promise((resolve) => setTimeout(resolve, 2200));
|
||||
return new Response(
|
||||
[
|
||||
`data: ${JSON.stringify({
|
||||
id: "chatcmpl-slow-stream",
|
||||
object: "chat.completion.chunk",
|
||||
choices: [{ index: 0, delta: { role: "assistant", content: "OK" } }],
|
||||
})}`,
|
||||
"",
|
||||
"data: [DONE]",
|
||||
"",
|
||||
].join("\n"),
|
||||
{
|
||||
status: 200,
|
||||
headers: { "Content-Type": "text/event-stream" },
|
||||
}
|
||||
);
|
||||
};
|
||||
|
||||
const response = await chatRoute.POST(makeStreamingRequest());
|
||||
assert.equal(response.status, 200);
|
||||
assert.match(response.headers.get("content-type") || "", /text\/event-stream/);
|
||||
|
||||
const body = await readAll(response);
|
||||
assert.match(body, /: omniroute-keepalive/);
|
||||
assert.match(body, /OK/);
|
||||
assert.match(body, /\[DONE\]/);
|
||||
});
|
||||
|
||||
test("chat completions route returns JSON without early SSE framing when stream is omitted and Accept is application/json", async () => {
|
||||
await seedHealthyConnection();
|
||||
|
||||
globalThis.fetch = async () => {
|
||||
await new Promise((resolve) => setTimeout(resolve, 2200));
|
||||
return Response.json({
|
||||
id: "chatcmpl-slow-json",
|
||||
choices: [
|
||||
{
|
||||
message: {
|
||||
role: "assistant",
|
||||
content: "OK",
|
||||
},
|
||||
},
|
||||
],
|
||||
});
|
||||
};
|
||||
|
||||
const response = await chatRoute.POST(
|
||||
makeRequestWithoutStreamFlag({
|
||||
Accept: "application/json",
|
||||
"X-OmniRoute-No-Cache": "true",
|
||||
"X-Request-Id": "chat-route-omitted-stream-json",
|
||||
})
|
||||
);
|
||||
assert.equal(response.status, 200);
|
||||
assert.doesNotMatch(response.headers.get("content-type") || "", /text\/event-stream/);
|
||||
|
||||
const body = (await response.json()) as any;
|
||||
assert.equal(body.choices[0].message.content, "OK");
|
||||
});
|
||||
|
||||
test("combo live test does not use cooldown-aware request retry on upstream failures", async () => {
|
||||
await seedHealthyConnection();
|
||||
await settingsDb.updateSettings({
|
||||
|
||||
@@ -149,3 +149,96 @@ test("truncateForLog summarizes oversized payloads instead of cloning", () => {
|
||||
// the original (huge) is NOT returned — it is a fresh summary object
|
||||
assert.notEqual(summary, huge);
|
||||
});
|
||||
|
||||
test("truncateForLog keeps a bounded `tools` field alive when the request is summarized", () => {
|
||||
// A request whose message history alone blows well past the 8KB summary
|
||||
// threshold, but which also carries `tools` — a field that used to be
|
||||
// silently dropped once the payload got summarized.
|
||||
const tools = [
|
||||
{
|
||||
type: "function",
|
||||
function: {
|
||||
name: "get_weather",
|
||||
description: "Get the current weather for a location",
|
||||
parameters: {
|
||||
type: "object",
|
||||
properties: { location: { type: "string" } },
|
||||
required: ["location"],
|
||||
},
|
||||
},
|
||||
},
|
||||
{
|
||||
type: "function",
|
||||
function: {
|
||||
name: "search_web",
|
||||
description: "Search the web for a query",
|
||||
parameters: {
|
||||
type: "object",
|
||||
properties: { query: { type: "string" } },
|
||||
required: ["query"],
|
||||
},
|
||||
},
|
||||
},
|
||||
];
|
||||
const huge = {
|
||||
model: "gpt-4o",
|
||||
provider: "openai",
|
||||
stream: true,
|
||||
messages: Array.from({ length: 50000 }, () => ({ role: "user", content: "x".repeat(64) })),
|
||||
tools,
|
||||
};
|
||||
|
||||
const summary = truncateForLog(huge) as Record<string, unknown>;
|
||||
|
||||
// it is still truncated/summarized — the fix must not change this
|
||||
assert.equal(summary._truncated, true);
|
||||
assert.equal(summary.messageCount, 50000);
|
||||
|
||||
// ...but `tools` now survives, bounded via cloneBoundedChatLogPayload
|
||||
assert.ok(summary.tools, "expected the summary to retain a `tools` field");
|
||||
const clonedTools = summary.tools as Array<Record<string, unknown>>;
|
||||
assert.equal(clonedTools.length, tools.length);
|
||||
assert.equal(
|
||||
(clonedTools[0].function as Record<string, unknown>).name,
|
||||
"get_weather"
|
||||
);
|
||||
assert.equal(
|
||||
(clonedTools[1].function as Record<string, unknown>).name,
|
||||
"search_web"
|
||||
);
|
||||
});
|
||||
|
||||
test("truncateForLog bounds an oversized `tools` array to the configured tail-item cap", () => {
|
||||
const maxTailItems = getChatLogArrayTailItems();
|
||||
const manyTools = Array.from({ length: maxTailItems + 50 }, (_, i) => ({
|
||||
type: "function",
|
||||
function: { name: `tool_${i}`, description: "d", parameters: {} },
|
||||
}));
|
||||
const huge = {
|
||||
model: "gpt-4o",
|
||||
messages: Array.from({ length: 50000 }, () => ({ role: "user", content: "x".repeat(64) })),
|
||||
tools: manyTools,
|
||||
};
|
||||
|
||||
const summary = truncateForLog(huge) as Record<string, unknown>;
|
||||
assert.equal(summary._truncated, true);
|
||||
|
||||
const clonedTools = summary.tools as unknown[];
|
||||
// marker entry + retained tail items — never the full (unbounded) list
|
||||
assert.equal(clonedTools.length, maxTailItems + 1);
|
||||
const marker = clonedTools[0] as Record<string, unknown>;
|
||||
assert.equal(marker._omniroute_truncated_array, true);
|
||||
assert.equal(marker.originalLength, manyTools.length);
|
||||
assert.equal(marker.retainedTailItems, maxTailItems);
|
||||
});
|
||||
|
||||
test("truncateForLog leaves small requests with `tools` unchanged (no regression)", () => {
|
||||
const small = {
|
||||
model: "gpt-4o",
|
||||
messages: [{ role: "user", content: "hi" }],
|
||||
tools: [{ type: "function", function: { name: "get_weather", parameters: {} } }],
|
||||
};
|
||||
const result = truncateForLog(small);
|
||||
// untouched — same reference, not a summary or a clone
|
||||
assert.equal(result, small);
|
||||
});
|
||||
|
||||
@@ -5,7 +5,10 @@
|
||||
// stream closes.
|
||||
import { test } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { readNonStreamingResponseBody } from "../../open-sse/handlers/chatCore/nonStreamingResponseBody.ts";
|
||||
import {
|
||||
readNonStreamingResponseBody,
|
||||
NonStreamingResponseTooLargeError,
|
||||
} from "../../open-sse/handlers/chatCore/nonStreamingResponseBody.ts";
|
||||
|
||||
test("falls back to response.text() when upstream is not streaming", async () => {
|
||||
const out = await readNonStreamingResponseBody(new Response("hello"), "application/json", false);
|
||||
@@ -21,8 +24,8 @@ test("drains an SSE stream chunk-by-chunk and concatenates until close", async (
|
||||
const enc = new TextEncoder();
|
||||
const body = new ReadableStream({
|
||||
start(controller) {
|
||||
controller.enqueue(enc.encode("data: {\"a\":1}\n\n"));
|
||||
controller.enqueue(enc.encode("data: {\"b\":2}\n\n"));
|
||||
controller.enqueue(enc.encode('data: {"a":1}\n\n'));
|
||||
controller.enqueue(enc.encode('data: {"b":2}\n\n'));
|
||||
controller.close();
|
||||
},
|
||||
});
|
||||
@@ -31,3 +34,76 @@ test("drains an SSE stream chunk-by-chunk and concatenates until close", async (
|
||||
assert.ok(out.includes('"a":1'));
|
||||
assert.ok(out.includes('"b":2'));
|
||||
});
|
||||
|
||||
test("returns after terminal SSE even when underlying cancel never resolves", async () => {
|
||||
const enc = new TextEncoder();
|
||||
let cancelled = false;
|
||||
const body = new ReadableStream({
|
||||
start(controller) {
|
||||
controller.enqueue(enc.encode('data: {"id":"x","choices":[{"delta":{"content":"ok"}}]}\n\n'));
|
||||
controller.enqueue(enc.encode("data: [DONE]\n\n"));
|
||||
},
|
||||
cancel() {
|
||||
cancelled = true;
|
||||
return new Promise(() => {});
|
||||
},
|
||||
});
|
||||
const response = new Response(body, { headers: { "Content-Type": "text/event-stream" } });
|
||||
|
||||
const out = await Promise.race([
|
||||
readNonStreamingResponseBody(response, "text/event-stream", true),
|
||||
new Promise<string>((_, reject) =>
|
||||
setTimeout(() => reject(new Error("read timed out after terminal SSE")), 1000)
|
||||
),
|
||||
]);
|
||||
|
||||
assert.ok(out.includes('"content":"ok"'));
|
||||
assert.ok(out.includes("[DONE]"));
|
||||
assert.equal(cancelled, true);
|
||||
});
|
||||
|
||||
// #5152: bound the non-streaming buffer so a runaway upstream body cannot fill the V8 heap.
|
||||
|
||||
test("aborts and throws when an SSE stream exceeds the byte cap (no unbounded string)", async () => {
|
||||
const enc = new TextEncoder();
|
||||
let cancelled = false;
|
||||
const body = new ReadableStream({
|
||||
pull(controller) {
|
||||
// 1 KB chunks with no terminal signal — would grow forever without the cap.
|
||||
controller.enqueue(enc.encode("data: " + "x".repeat(1024) + "\n"));
|
||||
},
|
||||
cancel() {
|
||||
cancelled = true;
|
||||
},
|
||||
});
|
||||
const response = new Response(body, { headers: { "Content-Type": "text/event-stream" } });
|
||||
|
||||
await assert.rejects(
|
||||
() => readNonStreamingResponseBody(response, "text/event-stream", true, 8 * 1024),
|
||||
(err) => err instanceof NonStreamingResponseTooLargeError && err.maxBytes === 8 * 1024
|
||||
);
|
||||
assert.equal(cancelled, true, "upstream reader must be cancelled on cap exceed");
|
||||
});
|
||||
|
||||
test("an SSE stream within the cap still drains normally", async () => {
|
||||
const enc = new TextEncoder();
|
||||
const body = new ReadableStream({
|
||||
start(controller) {
|
||||
controller.enqueue(enc.encode('data: {"ok":1}\n\n'));
|
||||
controller.close();
|
||||
},
|
||||
});
|
||||
const response = new Response(body, { headers: { "Content-Type": "text/event-stream" } });
|
||||
const out = await readNonStreamingResponseBody(response, "text/event-stream", true, 1024 * 1024);
|
||||
assert.ok(out.includes('"ok":1'));
|
||||
});
|
||||
|
||||
test("rejects a non-SSE response whose declared Content-Length exceeds the cap (no buffering)", async () => {
|
||||
const response = new Response("body-not-read", {
|
||||
headers: { "Content-Type": "application/json", "content-length": String(100 * 1024 * 1024) },
|
||||
});
|
||||
await assert.rejects(
|
||||
() => readNonStreamingResponseBody(response, "application/json", false, 1024 * 1024),
|
||||
(err) => err instanceof NonStreamingResponseTooLargeError
|
||||
);
|
||||
});
|
||||
|
||||
@@ -55,6 +55,61 @@ test("appendNonStreamingSseTerminalSignal detects [DONE] and terminal event type
|
||||
const stop: NonStreamingSseTerminalState = { currentEvent: "", pendingLine: "" };
|
||||
assert.equal(appendNonStreamingSseTerminalSignal(stop, "event: message_stop\ndata: {}\n"), true);
|
||||
|
||||
const responseTerminalEvents = [
|
||||
"response.completed",
|
||||
"response.done",
|
||||
"response.cancelled",
|
||||
"response.canceled",
|
||||
"response.failed",
|
||||
"response.incomplete",
|
||||
];
|
||||
for (const eventType of responseTerminalEvents) {
|
||||
const typed: NonStreamingSseTerminalState = { currentEvent: "", pendingLine: "" };
|
||||
assert.equal(
|
||||
appendNonStreamingSseTerminalSignal(typed, `data: {"type":"${eventType}"}\n`),
|
||||
true,
|
||||
eventType
|
||||
);
|
||||
|
||||
const eventOnly: NonStreamingSseTerminalState = { currentEvent: "", pendingLine: "" };
|
||||
assert.equal(
|
||||
appendNonStreamingSseTerminalSignal(eventOnly, `event: ${eventType}\n\n`),
|
||||
true,
|
||||
eventType
|
||||
);
|
||||
}
|
||||
|
||||
const splitTerminalData: NonStreamingSseTerminalState = { currentEvent: "", pendingLine: "" };
|
||||
assert.equal(
|
||||
appendNonStreamingSseTerminalSignal(splitTerminalData, "event: response.completed\n"),
|
||||
false
|
||||
);
|
||||
assert.equal(
|
||||
appendNonStreamingSseTerminalSignal(
|
||||
splitTerminalData,
|
||||
'data: {"response":{"status":"completed"}}\n'
|
||||
),
|
||||
true
|
||||
);
|
||||
|
||||
const messageDeltaWithType: NonStreamingSseTerminalState = { currentEvent: "", pendingLine: "" };
|
||||
assert.equal(
|
||||
appendNonStreamingSseTerminalSignal(
|
||||
messageDeltaWithType,
|
||||
'data: {"type":"message_delta","delta":{"stop_reason":"end_turn"}}\n'
|
||||
),
|
||||
true
|
||||
);
|
||||
|
||||
const messageDeltaFromEvent: NonStreamingSseTerminalState = { currentEvent: "", pendingLine: "" };
|
||||
assert.equal(
|
||||
appendNonStreamingSseTerminalSignal(
|
||||
messageDeltaFromEvent,
|
||||
'event: message_delta\ndata: {"delta":{"stop_reason":"end_turn"}}\n'
|
||||
),
|
||||
true
|
||||
);
|
||||
|
||||
const delta: NonStreamingSseTerminalState = { currentEvent: "", pendingLine: "" };
|
||||
assert.equal(
|
||||
appendNonStreamingSseTerminalSignal(delta, 'data: {"type":"content_block_delta"}\n'),
|
||||
@@ -62,6 +117,20 @@ test("appendNonStreamingSseTerminalSignal detects [DONE] and terminal event type
|
||||
);
|
||||
});
|
||||
|
||||
test("parseNonStreamingSSEPayload still parses Claude event/data buffers", () => {
|
||||
const raw =
|
||||
'event: message_start\ndata: {"type":"message_start","message":{"id":"msg_1","model":"claude","role":"assistant","usage":{"input_tokens":1}}}\n\n' +
|
||||
'event: content_block_start\ndata: {"type":"content_block_start","index":0,"content_block":{"type":"text","text":""}}\n\n' +
|
||||
'event: content_block_delta\ndata: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":"hi"}}\n\n' +
|
||||
'event: message_delta\ndata: {"type":"message_delta","delta":{"stop_reason":"end_turn"},"usage":{"output_tokens":1}}\n\n' +
|
||||
'event: message_stop\ndata: {"type":"message_stop"}\n\n';
|
||||
const result = parseNonStreamingSSEPayload(raw, FORMATS.CLAUDE, "claude");
|
||||
assert.ok(result !== null);
|
||||
assert.equal(result?.format, FORMATS.CLAUDE);
|
||||
assert.deepEqual(result?.body.content, [{ type: "text", text: "hi" }]);
|
||||
assert.equal(result?.body.stop_reason, "end_turn");
|
||||
});
|
||||
|
||||
test("parseNonStreamingSSEPayload parses an OpenAI-format SSE buffer", () => {
|
||||
const raw =
|
||||
'data: {"id":"x","choices":[{"delta":{"content":"hi"},"finish_reason":"stop"}]}\n\ndata: [DONE]\n';
|
||||
|
||||
@@ -437,17 +437,19 @@ test("chatCore sanitization normalizes mixed content blocks and removes unsuppor
|
||||
textBlocks.some((block) => block.text === "[draft.txt]\nDraft text"),
|
||||
true
|
||||
);
|
||||
// Orphaned tool_result blocks (no matching tool_use anywhere in the request) are
|
||||
// stripped by `stripOrphanedToolResults` (#5805) before reaching content normalization,
|
||||
// to avoid strict-upstream 400s. They are therefore removed entirely — neither preserved
|
||||
// as tool_result blocks nor inlined as "[Tool Result: …]" text.
|
||||
assert.equal(
|
||||
textBlocks.some((block) => block.text === "[Tool Result: tool-1]\ndone"),
|
||||
true
|
||||
content.some((block) => block.type === "tool_result"),
|
||||
false
|
||||
);
|
||||
assert.equal(
|
||||
textBlocks.some((block) => block.text === "[Tool Result: tool-2]\nstructured result"),
|
||||
true
|
||||
);
|
||||
assert.equal(
|
||||
textBlocks.some((block) => block.text === '[Tool Result: tool-3]\n{"status":"ok","count":2}'),
|
||||
true
|
||||
textBlocks.some(
|
||||
(block) => typeof block.text === "string" && block.text.includes("[Tool Result:")
|
||||
),
|
||||
false
|
||||
);
|
||||
});
|
||||
|
||||
|
||||
@@ -46,6 +46,41 @@ test("leaves the model untouched when it already matches", async () => {
|
||||
assert.equal(out.model, "model-a");
|
||||
});
|
||||
|
||||
// PR #5563: the `effectiveToolLimit < MAX_TOOLS_LIMIT` gate was removed from
|
||||
// truncateToolList, so providers whose proactive limit is >= the 128 default
|
||||
// (e.g. grok-cli at 200) are actually truncated. Without the gate removal these
|
||||
// two assertions fail (250 tools would pass through untruncated).
|
||||
test("truncates the tool list to the grok-cli proactive limit (200) when exceeded", async () => {
|
||||
const tools = Array.from({ length: 250 }, (_, i) => ({
|
||||
type: "function",
|
||||
function: { name: `tool_${i}`, parameters: {} },
|
||||
}));
|
||||
const out = await prepareUpstreamBody({
|
||||
translatedBody: { model: "grok-cli-model", messages: [], tools },
|
||||
modelToCall: "grok-cli-model",
|
||||
provider: "grok-cli",
|
||||
targetFormat: "claude",
|
||||
credentials: null,
|
||||
});
|
||||
assert.ok(Array.isArray(out.tools));
|
||||
assert.equal(out.tools.length, 200);
|
||||
});
|
||||
|
||||
test("preserves the full tool list when within the grok-cli limit", async () => {
|
||||
const tools = Array.from({ length: 150 }, (_, i) => ({
|
||||
type: "function",
|
||||
function: { name: `tool_${i}`, parameters: {} },
|
||||
}));
|
||||
const out = await prepareUpstreamBody({
|
||||
translatedBody: { model: "grok-cli-model", messages: [], tools },
|
||||
modelToCall: "grok-cli-model",
|
||||
provider: "grok-cli",
|
||||
targetFormat: "claude",
|
||||
credentials: null,
|
||||
});
|
||||
assert.equal(out.tools.length, 150);
|
||||
});
|
||||
|
||||
test("backfills the Qwen OAuth user when missing", async () => {
|
||||
const out = await prepareUpstreamBody({
|
||||
translatedBody: { model: "qwen-max", messages: [] },
|
||||
|
||||
@@ -842,7 +842,7 @@ test("GPT-5.5 Pro non-streaming: stream_handoff polls conversation detail for fi
|
||||
try {
|
||||
const executor = new ChatGptWebExecutor();
|
||||
const result = await executor.execute({
|
||||
model: "gpt-5-5-pro-extended",
|
||||
model: "gpt-5.5-pro-extended",
|
||||
body: { messages: [{ role: "user", content: "hard problem" }] },
|
||||
stream: false,
|
||||
credentials: { apiKey: "cookie-pro-poll" },
|
||||
@@ -904,7 +904,7 @@ test("GPT-5.5 Pro streaming: preserves interim reasoning and appends final polle
|
||||
try {
|
||||
const executor = new ChatGptWebExecutor();
|
||||
const result = await executor.execute({
|
||||
model: "gpt-5-5-pro-extended",
|
||||
model: "gpt-5.5-pro-extended",
|
||||
body: { messages: [{ role: "user", content: "hard problem" }], stream: true },
|
||||
stream: true,
|
||||
credentials: { apiKey: "cookie-pro-stream" },
|
||||
@@ -1271,18 +1271,19 @@ test("Provider registry: chatgpt-web exposes the current ChatGPT Web model catal
|
||||
assert.equal(entry.authHeader, "cookie");
|
||||
|
||||
const ids = (entry.models || []).map((m) => m.id);
|
||||
// Mirrors /backend-api/models for ChatGPT Web. Retired GPT-5/GPT-5.1
|
||||
// entries should stay out of this list.
|
||||
// Public OmniRoute ids stay in historical dot form even though ChatGPT's
|
||||
// backend routes use dash-form slugs. Retired GPT-5/GPT-5.1 entries should
|
||||
// stay out of this list.
|
||||
assert.deepEqual(ids, [
|
||||
"gpt-5-5-pro",
|
||||
"gpt-5-5-pro-extended",
|
||||
"gpt-5-5-thinking",
|
||||
"gpt-5-5",
|
||||
"gpt-5-4-pro",
|
||||
"gpt-5-4-thinking",
|
||||
"gpt-5-4-t-mini",
|
||||
"gpt-5-3",
|
||||
"gpt-5-3-mini",
|
||||
"gpt-5.5-pro",
|
||||
"gpt-5.5-pro-extended",
|
||||
"gpt-5.5-thinking",
|
||||
"gpt-5.5",
|
||||
"gpt-5.4-pro",
|
||||
"gpt-5.4-thinking",
|
||||
"gpt-5.4-thinking-mini",
|
||||
"gpt-5.3",
|
||||
"gpt-5.3-mini",
|
||||
"o3",
|
||||
]);
|
||||
});
|
||||
@@ -1292,19 +1293,21 @@ test("Executor MODEL_MAP: OmniRoute IDs translate to ChatGPT backend slugs", asy
|
||||
const m = installMockFetch();
|
||||
try {
|
||||
const cases: Array<[string, string]> = [
|
||||
["gpt-5-3", "gpt-5-3"],
|
||||
["gpt-5-5-thinking", "gpt-5-5-thinking"],
|
||||
["gpt-5-4-t-mini", "gpt-5-4-t-mini"],
|
||||
["gpt-5-5-pro", "gpt-5-5-pro"],
|
||||
["gpt-5-5-pro-extended", "gpt-5-5-pro"],
|
||||
["o3", "o3"],
|
||||
// Legacy dot-form ids are still accepted for direct provider/model callers.
|
||||
// Public catalog ids.
|
||||
["gpt-5.3", "gpt-5-3"],
|
||||
["gpt-5.5-thinking", "gpt-5-5-thinking"],
|
||||
["gpt-5.4-thinking-mini", "gpt-5-4-t-mini"],
|
||||
["gpt-5.5", "gpt-5-5"],
|
||||
["gpt-5.5-pro", "gpt-5-5-pro"],
|
||||
["gpt-5.5-pro-extended", "gpt-5-5-pro"],
|
||||
["gpt-5.4-pro", "gpt-5-4-pro"],
|
||||
["o3", "o3"],
|
||||
// Backend dash-form slugs are still accepted for direct provider/model callers.
|
||||
["gpt-5-3", "gpt-5-3"],
|
||||
["gpt-5-5-thinking", "gpt-5-5-thinking"],
|
||||
["gpt-5-4-t-mini", "gpt-5-4-t-mini"],
|
||||
["gpt-5-5-pro", "gpt-5-5-pro"],
|
||||
["gpt-5-5-pro-extended", "gpt-5-5-pro"],
|
||||
];
|
||||
for (const [omniId, expectedSlug] of cases) {
|
||||
m.calls.urls.length = 0;
|
||||
@@ -1331,6 +1334,18 @@ test("MODEL_MAP drift guard: every advertised catalog id reaches ChatGPT as a ba
|
||||
reset();
|
||||
const { getRegistryEntry } = await import("../../open-sse/config/providerRegistry.ts");
|
||||
const ids = (getRegistryEntry("chatgpt-web")?.models || []).map((m) => m.id);
|
||||
const expectedSlugById: Record<string, string> = {
|
||||
"gpt-5.5-pro": "gpt-5-5-pro",
|
||||
"gpt-5.5-pro-extended": "gpt-5-5-pro",
|
||||
"gpt-5.5-thinking": "gpt-5-5-thinking",
|
||||
"gpt-5.5": "gpt-5-5",
|
||||
"gpt-5.4-pro": "gpt-5-4-pro",
|
||||
"gpt-5.4-thinking": "gpt-5-4-thinking",
|
||||
"gpt-5.4-thinking-mini": "gpt-5-4-t-mini",
|
||||
"gpt-5.3": "gpt-5-3",
|
||||
"gpt-5.3-mini": "gpt-5-3-mini",
|
||||
o3: "o3",
|
||||
};
|
||||
const m = installMockFetch();
|
||||
try {
|
||||
for (const omniId of ids) {
|
||||
@@ -1351,11 +1366,7 @@ test("MODEL_MAP drift guard: every advertised catalog id reaches ChatGPT as a ba
|
||||
!body.model.includes("."),
|
||||
`${omniId} reached the backend as "${body.model}" (still dot-form)`
|
||||
);
|
||||
if (omniId.endsWith("-extended")) {
|
||||
assert.equal(body.model, "gpt-5-5-pro", `${omniId} should select the base Pro slug`);
|
||||
} else {
|
||||
assert.equal(body.model, omniId, `${omniId} should pass through as its backend slug`);
|
||||
}
|
||||
assert.equal(body.model, expectedSlugById[omniId], `${omniId} should map to backend slug`);
|
||||
}
|
||||
} finally {
|
||||
m.restore();
|
||||
@@ -1370,7 +1381,7 @@ test("GPT-5.5 Pro Extended sends base slug with extended effort and Temporary Ch
|
||||
try {
|
||||
const executor = new ChatGptWebExecutor();
|
||||
const result = await executor.execute({
|
||||
model: "gpt-5-5-pro-extended",
|
||||
model: "gpt-5.5-pro-extended",
|
||||
body: { messages: [{ role: "user", content: "hi" }] },
|
||||
stream: false,
|
||||
credentials: { apiKey: "cookie-pro-extended" },
|
||||
@@ -1399,7 +1410,7 @@ test("GPT-5.5 Pro standard sends standard effort", async () => {
|
||||
try {
|
||||
const executor = new ChatGptWebExecutor();
|
||||
await executor.execute({
|
||||
model: "gpt-5-5-pro",
|
||||
model: "gpt-5.5-pro",
|
||||
body: { messages: [{ role: "user", content: "hi" }] },
|
||||
stream: false,
|
||||
credentials: { apiKey: "cookie-pro-standard" },
|
||||
@@ -1422,7 +1433,7 @@ test("GPT-5.5 Pro store:false keeps Temporary Chat enabled for background utilit
|
||||
try {
|
||||
const executor = new ChatGptWebExecutor();
|
||||
const result = await executor.execute({
|
||||
model: "gpt-5-5-pro-extended",
|
||||
model: "gpt-5.5-pro-extended",
|
||||
body: {
|
||||
store: false,
|
||||
messages: [
|
||||
|
||||
@@ -39,6 +39,17 @@ test("routeFileToApiPath resolves dynamic [param] segments to a concrete placeho
|
||||
);
|
||||
});
|
||||
|
||||
test("routeFileToApiPath normalizes Windows backslash separators before stripping prefixes", () => {
|
||||
// On Windows, node:path join() yields backslash-separated paths; without the
|
||||
// leading `.replace(/\\/g, "/")` the `^src\/app` strip never matches and the
|
||||
// route-guard gate produces wrong API paths (false negatives that miss
|
||||
// RCE-capable spawn routes). See PR #5613.
|
||||
assert.equal(
|
||||
routeFileToApiPath("src\\app\\api\\services\\9router\\install\\route.ts"),
|
||||
"/api/services/9router/install"
|
||||
);
|
||||
});
|
||||
|
||||
test("no unclassified routes when every spawn-capable route is local-only", () => {
|
||||
const routes = [
|
||||
"/api/mcp/tools",
|
||||
|
||||
@@ -41,10 +41,25 @@ test("classify429: Antigravity 'Individual quota reached' body returns 'quota_ex
|
||||
"Resets in 164h27m24s.";
|
||||
assert.equal(looksLikeQuotaExhausted(body), true);
|
||||
assert.equal(classify429({ status: 429, body }), "quota_exhausted");
|
||||
assert.equal(
|
||||
classify429({ status: 429, body: { error: { message: body } } }),
|
||||
"quota_exhausted"
|
||||
);
|
||||
assert.equal(classify429({ status: 429, body: { error: { message: body } } }), "quota_exhausted");
|
||||
});
|
||||
|
||||
test("classify429: Antigravity INSUFFICIENT_G1_CREDITS_BALANCE body returns 'quota_exhausted'", () => {
|
||||
const body = {
|
||||
error: {
|
||||
code: 429,
|
||||
message: "Resource has been exhausted (e.g. check quota).",
|
||||
status: "RESOURCE_EXHAUSTED",
|
||||
details: [
|
||||
{
|
||||
"@type": "type.googleapis.com/google.rpc.ErrorInfo",
|
||||
reason: "INSUFFICIENT_G1_CREDITS_BALANCE",
|
||||
},
|
||||
],
|
||||
},
|
||||
};
|
||||
assert.equal(looksLikeQuotaExhausted(body), true);
|
||||
assert.equal(classify429({ status: 429, body }), "quota_exhausted");
|
||||
});
|
||||
|
||||
test("classify429: Antigravity quota patterns do not over-match plain rate limits", () => {
|
||||
|
||||
89
tests/unit/claude-profile-auto-sync-gate.test.ts
Normal file
89
tests/unit/claude-profile-auto-sync-gate.test.ts
Normal file
@@ -0,0 +1,89 @@
|
||||
import test from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
|
||||
import { autoSyncClaudeProfilesFromLiveCatalog } from "@/lib/cli-helper/claudeProfileAutoSync";
|
||||
|
||||
// Claude Code profile auto-sync writes files into the operator's ~/.claude/profiles, so it
|
||||
// MUST be opt-in (default OFF) and short-circuit before any catalog fetch / file write when
|
||||
// either gate is closed. These tests pin the two gates (the OMNIROUTE_AUTO_SYNC_CLAUDE_PROFILES
|
||||
// feature flag, read env-first here with no DB override, + the CLI_ALLOW_CONFIG_WRITES
|
||||
// write-guard) so a future change can't silently turn it back on.
|
||||
|
||||
const GATE_ENV = ["OMNIROUTE_AUTO_SYNC_CLAUDE_PROFILES", "CLI_ALLOW_CONFIG_WRITES"] as const;
|
||||
|
||||
function snapshotEnv(): Record<string, string | undefined> {
|
||||
const snap: Record<string, string | undefined> = {};
|
||||
for (const k of GATE_ENV) snap[k] = process.env[k];
|
||||
return snap;
|
||||
}
|
||||
|
||||
function restoreEnv(snap: Record<string, string | undefined>): void {
|
||||
for (const k of GATE_ENV) {
|
||||
if (snap[k] === undefined) delete process.env[k];
|
||||
else process.env[k] = snap[k];
|
||||
}
|
||||
}
|
||||
|
||||
function mockSyncRequest(): Request {
|
||||
return new Request("http://127.0.0.1:20128/api/providers/openai/sync-models", { method: "POST" });
|
||||
}
|
||||
|
||||
test("Claude auto-sync is OFF by default (flag unset) — returns disabled, never fetches or writes", async () => {
|
||||
const snap = snapshotEnv();
|
||||
try {
|
||||
delete process.env.OMNIROUTE_AUTO_SYNC_CLAUDE_PROFILES;
|
||||
const result = await autoSyncClaudeProfilesFromLiveCatalog(
|
||||
mockSyncRequest(),
|
||||
"test:default-off"
|
||||
);
|
||||
assert.equal(result.ok, false);
|
||||
assert.equal(result.reason, "disabled");
|
||||
} finally {
|
||||
restoreEnv(snap);
|
||||
}
|
||||
});
|
||||
|
||||
test("explicit OMNIROUTE_AUTO_SYNC_CLAUDE_PROFILES=false stays disabled", async () => {
|
||||
const snap = snapshotEnv();
|
||||
try {
|
||||
process.env.OMNIROUTE_AUTO_SYNC_CLAUDE_PROFILES = "false";
|
||||
const result = await autoSyncClaudeProfilesFromLiveCatalog(
|
||||
mockSyncRequest(),
|
||||
"test:explicit-false"
|
||||
);
|
||||
assert.equal(result.ok, false);
|
||||
assert.equal(result.reason, "disabled");
|
||||
} finally {
|
||||
restoreEnv(snap);
|
||||
}
|
||||
});
|
||||
|
||||
test("non-truthy flag values stay disabled", async () => {
|
||||
const snap = snapshotEnv();
|
||||
try {
|
||||
for (const v of ["0", "no", "off", "maybe", ""]) {
|
||||
process.env.OMNIROUTE_AUTO_SYNC_CLAUDE_PROFILES = v;
|
||||
const result = await autoSyncClaudeProfilesFromLiveCatalog(mockSyncRequest(), `test:${v}`);
|
||||
assert.equal(result.ok, false, `value '${v}' must not enable auto-sync`);
|
||||
assert.equal(result.reason, "disabled", `value '${v}' must report disabled`);
|
||||
}
|
||||
} finally {
|
||||
restoreEnv(snap);
|
||||
}
|
||||
});
|
||||
|
||||
test("enabled flag but CLI_ALLOW_CONFIG_WRITES=false is blocked by the write-guard (no write)", async () => {
|
||||
const snap = snapshotEnv();
|
||||
try {
|
||||
process.env.OMNIROUTE_AUTO_SYNC_CLAUDE_PROFILES = "true";
|
||||
process.env.CLI_ALLOW_CONFIG_WRITES = "false";
|
||||
const result = await autoSyncClaudeProfilesFromLiveCatalog(
|
||||
mockSyncRequest(),
|
||||
"test:write-guard"
|
||||
);
|
||||
assert.equal(result.ok, false);
|
||||
assert.match(result.reason, /CLI config writes are disabled/);
|
||||
} finally {
|
||||
restoreEnv(snap);
|
||||
}
|
||||
});
|
||||
80
tests/unit/claude-tool-type-default-2195.test.ts
Normal file
80
tests/unit/claude-tool-type-default-2195.test.ts
Normal file
@@ -0,0 +1,80 @@
|
||||
import test from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
|
||||
// Port of 9router#2196 (fixes #2195): Claude's tool schema requires each tool to
|
||||
// carry an explicit `type` discriminator. Anthropic's first-party API infers
|
||||
// "custom" when omitted, but strict Anthropic-compatible gateways (e.g. MiniMax)
|
||||
// reject the payload with HTTP 400. defaultClaudeToolType() backfills the missing
|
||||
// `type` so legacy Claude-format tool definitions survive strict gateways.
|
||||
|
||||
const { defaultClaudeToolType } = await import(
|
||||
"../../open-sse/handlers/chatCore/claudeToolDefaults.ts"
|
||||
);
|
||||
|
||||
test("backfills type:'custom' on a Claude tool missing the type field", () => {
|
||||
const tools = [
|
||||
{ name: "get_weather", description: "Get weather", input_schema: { type: "object" } },
|
||||
];
|
||||
const out = defaultClaudeToolType(tools) as Array<Record<string, unknown>>;
|
||||
assert.equal(out[0].type, "custom");
|
||||
// Other fields are preserved untouched.
|
||||
assert.equal(out[0].name, "get_weather");
|
||||
assert.equal(out[0].description, "Get weather");
|
||||
assert.deepEqual(out[0].input_schema, { type: "object" });
|
||||
});
|
||||
|
||||
test("leaves tools that already declare a type untouched", () => {
|
||||
const tools = [
|
||||
{ type: "custom", name: "a", input_schema: {} },
|
||||
{ type: "computer_20241022", name: "computer" },
|
||||
{ type: "bash_20241022", name: "bash" },
|
||||
];
|
||||
const out = defaultClaudeToolType(tools) as Array<Record<string, unknown>>;
|
||||
assert.deepEqual(
|
||||
out.map((t) => t.type),
|
||||
["custom", "computer_20241022", "bash_20241022"]
|
||||
);
|
||||
// Non-custom built-in tool types must be preserved, not overwritten.
|
||||
assert.equal(out[1].type, "computer_20241022");
|
||||
});
|
||||
|
||||
test("normalizes a mixed array — only the type-less entries get defaulted", () => {
|
||||
const tools = [
|
||||
{ type: "computer_20241022", name: "computer" },
|
||||
{ name: "get_weather", input_schema: {} },
|
||||
];
|
||||
const out = defaultClaudeToolType(tools) as Array<Record<string, unknown>>;
|
||||
assert.equal(out[0].type, "computer_20241022");
|
||||
assert.equal(out[1].type, "custom");
|
||||
});
|
||||
|
||||
test("returns non-array input unchanged (no tools / undefined)", () => {
|
||||
assert.equal(defaultClaudeToolType(undefined), undefined);
|
||||
assert.equal(defaultClaudeToolType(null), null);
|
||||
const obj = { not: "an array" };
|
||||
assert.equal(defaultClaudeToolType(obj), obj);
|
||||
});
|
||||
|
||||
test("does not mutate the original tool objects (returns new entries for defaulted tools)", () => {
|
||||
const original = { name: "x", input_schema: {} };
|
||||
const tools = [original];
|
||||
const out = defaultClaudeToolType(tools) as Array<Record<string, unknown>>;
|
||||
assert.equal(original.type, undefined, "original tool must stay untouched");
|
||||
assert.equal(out[0].type, "custom");
|
||||
});
|
||||
|
||||
test("passes non-object array entries through unchanged (no garbage wrapping)", () => {
|
||||
// A null/primitive entry must NOT be spread into a fabricated tool like
|
||||
// { type: "custom", '0': 'h' } — pass it through as-is.
|
||||
const tools = [
|
||||
{ name: "real_tool", input_schema: {} }, // object → defaulted
|
||||
null,
|
||||
"weird",
|
||||
42,
|
||||
];
|
||||
const out = defaultClaudeToolType(tools) as unknown[];
|
||||
assert.equal((out[0] as Record<string, unknown>).type, "custom", "real object gets defaulted");
|
||||
assert.equal(out[1], null, "null passes through unchanged");
|
||||
assert.equal(out[2], "weird", "string passes through unchanged");
|
||||
assert.equal(out[3], 42, "number passes through unchanged");
|
||||
});
|
||||
64
tests/unit/cli-doctor-rooturl-windows-5844.test.ts
Normal file
64
tests/unit/cli-doctor-rooturl-windows-5844.test.ts
Normal file
@@ -0,0 +1,64 @@
|
||||
import test from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import fs from "node:fs";
|
||||
import path from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
|
||||
// Regression test for #5844: on Windows, `bin/cli/commands/doctor.mjs` used to
|
||||
// resolve its `rootDir` via `new URL(import.meta.url).pathname`, which keeps the
|
||||
// leading drive-letter slash from a `file:///C:/...` URL (e.g. `/C:/Users/...`).
|
||||
// Feeding that string into `path.resolve()` on win32 does NOT strip the leading
|
||||
// slash the way `fileURLToPath` does, so the resolved rootDir ends up malformed
|
||||
// (effectively doubling/mismatching the drive segment), breaking every doctor
|
||||
// check that depends on rootDir (DB migrations, native binary probe, etc).
|
||||
//
|
||||
// `fileURLToPath` handles the platform-specific URL -> path conversion correctly
|
||||
// (stripping the extra leading slash on win32), which is why the fix in PR #5845
|
||||
// switched `doctor.mjs` to use it instead of `new URL(...).pathname`.
|
||||
//
|
||||
// fileURLToPath() throws when parsing a Windows-style file URL on a POSIX host,
|
||||
// so we can't call it directly here to prove the "after" behavior cross-platform.
|
||||
// Instead we assert two things that together lock the regression:
|
||||
// 1. The OLD pattern (`new URL(u).pathname`) demonstrably produces the buggy
|
||||
// `/C:/...` prefix for a synthetic Windows file URL — proving why it was wrong.
|
||||
// 2. The current source of doctor.mjs uses `fileURLToPath(import.meta.url)` and
|
||||
// does NOT use the old `new URL(import.meta.url).pathname` pattern anymore.
|
||||
|
||||
test("old `new URL(...).pathname` pattern produces the buggy /C: prefix on a Windows file URL", () => {
|
||||
const windowsFileUrl = "file:///C:/Users/x/node_modules/omniroute/bin/cli/commands/doctor.mjs";
|
||||
const buggyPathname = new URL(windowsFileUrl).pathname;
|
||||
|
||||
// This is the defect: the pathname keeps the leading slash before the drive
|
||||
// letter, so `path.resolve(path.dirname(buggyPathname), ...)` on win32 would
|
||||
// treat `/C:/Users/...` as a POSIX-relative-looking segment instead of the
|
||||
// real Windows path `C:\Users\...`, producing a malformed rootDir.
|
||||
assert.equal(buggyPathname.startsWith("/C:"), true);
|
||||
assert.equal(buggyPathname, "/C:/Users/x/node_modules/omniroute/bin/cli/commands/doctor.mjs");
|
||||
});
|
||||
|
||||
test("fileURLToPath does not keep the leading-slash drive-letter defect on the current platform", () => {
|
||||
// On any platform, fileURLToPath never yields a path starting with "/C:" for a
|
||||
// same-platform URL — it fully normalizes drive letters (win32) or leaves POSIX
|
||||
// paths untouched (posix), unlike the raw `.pathname` accessor used by the bug.
|
||||
const here = fileURLToPath(import.meta.url);
|
||||
assert.equal(here.startsWith("/C:"), false);
|
||||
});
|
||||
|
||||
test("doctor.mjs source uses fileURLToPath(import.meta.url) and not the buggy new URL(...).pathname pattern", () => {
|
||||
const doctorSource = fs.readFileSync(
|
||||
path.resolve("bin/cli/commands/doctor.mjs"),
|
||||
"utf8"
|
||||
);
|
||||
|
||||
assert.match(
|
||||
doctorSource,
|
||||
/fileURLToPath\(import\.meta\.url\)/,
|
||||
"doctor.mjs must resolve rootDir via fileURLToPath(import.meta.url)"
|
||||
);
|
||||
|
||||
assert.doesNotMatch(
|
||||
doctorSource,
|
||||
/new URL\(import\.meta\.url\)\.pathname/,
|
||||
"doctor.mjs must not regress to the buggy new URL(import.meta.url).pathname pattern"
|
||||
);
|
||||
});
|
||||
46
tests/unit/cli-serve-startup-time.test.ts
Normal file
46
tests/unit/cli-serve-startup-time.test.ts
Normal file
@@ -0,0 +1,46 @@
|
||||
import test from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
|
||||
const fs = await import("node:fs");
|
||||
const path = await import("node:path");
|
||||
|
||||
const servePath = path.resolve(import.meta.dirname, "../../bin/cli/commands/serve.mjs");
|
||||
const serveSource = fs.readFileSync(servePath, "utf-8");
|
||||
|
||||
test("serve startup time uses monotonic performance.now()", () => {
|
||||
assert.match(serveSource, /performance\.now\(\)/);
|
||||
assert.match(serveSource, /typeof startedAt === "number"/);
|
||||
});
|
||||
|
||||
test("serve startup banner includes started in elapsed time", () => {
|
||||
assert.match(serveSource, /started in/);
|
||||
});
|
||||
|
||||
test("serve daemon mode does not accept startedAt", () => {
|
||||
assert.match(
|
||||
serveSource,
|
||||
/function\s+runDaemon\s*\(\s*serverJs\s*,\s*env\s*,\s*memoryLimit\s*,\s*dashboardPort\s*,\s*apiPort\s*\)/
|
||||
);
|
||||
assert.ok(
|
||||
!/function\s+runDaemon\s*\(\s*serverJs\s*,\s*env\s*,\s*memoryLimit\s*,\s*dashboardPort\s*,\s*apiPort\s*,\s*startedAt\s*\)/.test(
|
||||
serveSource
|
||||
)
|
||||
);
|
||||
});
|
||||
|
||||
test("serve runWithSupervisor uses startedAt before defaulted useTray", () => {
|
||||
const signatureRegex =
|
||||
/async\s+function\s+runWithSupervisor\s*\([\s\S]*?startedAt\s*,\s*useTray\s*=\s*false\s*\)/;
|
||||
assert.match(
|
||||
serveSource,
|
||||
signatureRegex,
|
||||
"runWithSupervisor should declare startedAt before the defaulted useTray parameter"
|
||||
);
|
||||
|
||||
const callRegex = /runWithSupervisor\s*\([\s\S]*?startedAt\s*,\s*useTray\s*\)/;
|
||||
assert.match(
|
||||
serveSource,
|
||||
callRegex,
|
||||
"runWithSupervisor should be called with startedAt before useTray"
|
||||
);
|
||||
});
|
||||
38
tests/unit/cli-serve-version-banner.test.ts
Normal file
38
tests/unit/cli-serve-version-banner.test.ts
Normal file
@@ -0,0 +1,38 @@
|
||||
import test from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import fs from "node:fs";
|
||||
import path from "node:path";
|
||||
|
||||
/**
|
||||
* Regression guard for the startup-banner version line (#5752).
|
||||
*
|
||||
* `runServe` prints `v<version>` under the ASCII banner. The version is parsed
|
||||
* once at module load from the repo-root package.json. These source-inspection
|
||||
* assertions (same technique as cli-serve-port.test.ts) ensure the banner never
|
||||
* silently loses the version again — mirroring how the CLI is exercised without
|
||||
* spawning a real server.
|
||||
*/
|
||||
const serveSource = fs.readFileSync(
|
||||
path.resolve(import.meta.dirname, "../../bin/cli/commands/serve.mjs"),
|
||||
"utf-8",
|
||||
);
|
||||
|
||||
test("serve banner: version is parsed from package.json at module load", () => {
|
||||
assert.match(
|
||||
serveSource,
|
||||
/_pkg\s*=\s*JSON\.parse\(\s*readFileSync\(/,
|
||||
"serve.mjs should parse the version from package.json into _pkg",
|
||||
);
|
||||
assert.ok(
|
||||
serveSource.includes("package.json"),
|
||||
"serve.mjs should reference package.json for the version source",
|
||||
);
|
||||
});
|
||||
|
||||
test("serve banner: startup banner prints v<version>", () => {
|
||||
assert.match(
|
||||
serveSource,
|
||||
/v\$\{_pkg\.version\}/,
|
||||
"serve.mjs should print v${_pkg.version} in the startup banner",
|
||||
);
|
||||
});
|
||||
@@ -1,6 +1,12 @@
|
||||
import { test } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { buildProfileSettings } from "../../../bin/cli/commands/setup-claude.mjs";
|
||||
import fs from "node:fs/promises";
|
||||
import os from "node:os";
|
||||
import path from "node:path";
|
||||
import {
|
||||
buildProfileSettings,
|
||||
syncClaudeProfilesFromModels,
|
||||
} from "../../../bin/cli/commands/setup-claude.mjs";
|
||||
import { buildClaudeEnv, resolveLaunchTarget } from "../../../bin/cli/commands/launch.mjs";
|
||||
import { categoriseModel } from "../../../bin/cli/commands/setup-codex.mjs";
|
||||
|
||||
@@ -34,6 +40,52 @@ test("profile names match setup-codex (cross-CLI consistency)", () => {
|
||||
assert.equal(categoriseModel("kmc/kimi-k2.7").name, "kimi-k27");
|
||||
});
|
||||
|
||||
test("syncClaudeProfilesFromModels writes directory-per-profile settings + threads baseUrl, skips non-ids", async () => {
|
||||
const claudeHome = await fs.mkdtemp(path.join(os.tmpdir(), "omniroute-claude-profiles-"));
|
||||
try {
|
||||
const result = await syncClaudeProfilesFromModels([{ id: "glm/glm-5.2" }, { id: "" }], {
|
||||
claudeHome,
|
||||
baseUrl: "http://vps:20128",
|
||||
});
|
||||
|
||||
assert.equal(result.written, 1);
|
||||
assert.equal(result.skipped, 1);
|
||||
assert.deepEqual(
|
||||
result.profiles.map((p) => p.name),
|
||||
["glm52"]
|
||||
);
|
||||
|
||||
// Directory-per-profile: <claudeHome>/profiles/<name>/settings.json
|
||||
const settingsPath = path.join(claudeHome, "profiles", "glm52", "settings.json");
|
||||
const json = JSON.parse(await fs.readFile(settingsPath, "utf8"));
|
||||
assert.equal(json.model, "glm/glm-5.2");
|
||||
assert.equal(json.env.ANTHROPIC_BASE_URL, "http://vps:20128");
|
||||
assert.equal(json.env.CLAUDE_CODE_ENABLE_GATEWAY_MODEL_DISCOVERY, "1");
|
||||
// The auth token must never be written to disk.
|
||||
assert.equal(JSON.stringify(json).includes("ANTHROPIC_AUTH_TOKEN"), false);
|
||||
} finally {
|
||||
await fs.rm(claudeHome, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
test("syncClaudeProfilesFromModels dry-run writes nothing", async () => {
|
||||
const claudeHome = await fs.mkdtemp(path.join(os.tmpdir(), "omniroute-claude-dry-"));
|
||||
try {
|
||||
const result = await syncClaudeProfilesFromModels([{ id: "glm/glm-5.2" }], {
|
||||
claudeHome,
|
||||
baseUrl: "http://vps:20128",
|
||||
dryRun: true,
|
||||
});
|
||||
assert.equal(result.written, 1);
|
||||
await assert.rejects(
|
||||
fs.stat(path.join(claudeHome, "profiles", "glm52", "settings.json")),
|
||||
/ENOENT/
|
||||
);
|
||||
} finally {
|
||||
await fs.rm(claudeHome, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
// ── launch env (Claude Code) ─────────────────────────────────────────────────
|
||||
|
||||
test("buildClaudeEnv still accepts a bare port (backward compatible)", () => {
|
||||
|
||||
81
tests/unit/cli/setup-codex.test.ts
Normal file
81
tests/unit/cli/setup-codex.test.ts
Normal file
@@ -0,0 +1,81 @@
|
||||
import { test } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import fs from "node:fs/promises";
|
||||
import os from "node:os";
|
||||
import path from "node:path";
|
||||
import {
|
||||
fallbackCodexProfile,
|
||||
isCodexCompatibleTextModel,
|
||||
syncCodexProfilesFromModels,
|
||||
} from "../../../bin/cli/commands/setup-codex.mjs";
|
||||
|
||||
test("fallbackCodexProfile creates profiles for compatible live catalog models", () => {
|
||||
const cfg = fallbackCodexProfile("new-provider/future-chat-1", {
|
||||
id: "new-provider/future-chat-1",
|
||||
context_length: 250000,
|
||||
max_output_tokens: 65536,
|
||||
output_modalities: ["text"],
|
||||
});
|
||||
|
||||
assert.deepEqual(cfg, {
|
||||
name: "new-provider-future-chat-1",
|
||||
ctx: 250000,
|
||||
compact: 212500,
|
||||
summary: false,
|
||||
toolLimit: 32768,
|
||||
});
|
||||
});
|
||||
|
||||
test("fallbackCodexProfile skips media and non-text models", () => {
|
||||
assert.equal(
|
||||
isCodexCompatibleTextModel({
|
||||
id: "antigravity/gemini-3.1-flash-image",
|
||||
type: "image",
|
||||
output_modalities: ["image"],
|
||||
}),
|
||||
false
|
||||
);
|
||||
assert.equal(
|
||||
fallbackCodexProfile("veo-free/seedance", {
|
||||
id: "veo-free/seedance",
|
||||
name: "Seedance",
|
||||
context_length: 128000,
|
||||
}),
|
||||
null
|
||||
);
|
||||
});
|
||||
|
||||
test("syncCodexProfilesFromModels writes compatible profiles and skips media", async () => {
|
||||
const codexHome = await fs.mkdtemp(path.join(os.tmpdir(), "omniroute-codex-profiles-"));
|
||||
try {
|
||||
const result = await syncCodexProfilesFromModels(
|
||||
[
|
||||
{
|
||||
id: "new-provider/future-chat-1",
|
||||
context_length: 250000,
|
||||
output_modalities: ["text"],
|
||||
},
|
||||
{
|
||||
id: "video-provider/seedance",
|
||||
type: "video",
|
||||
output_modalities: ["video"],
|
||||
},
|
||||
],
|
||||
{ codexHome }
|
||||
);
|
||||
|
||||
assert.equal(result.written, 1);
|
||||
assert.equal(result.skipped, 1);
|
||||
const content = await fs.readFile(
|
||||
path.join(codexHome, "new-provider-future-chat-1.config.toml"),
|
||||
"utf8"
|
||||
);
|
||||
assert.match(content, /model\s+= "new-provider\/future-chat-1"/);
|
||||
await assert.rejects(
|
||||
fs.stat(path.join(codexHome, "video-provider-seedance.config.toml")),
|
||||
/ENOENT/
|
||||
);
|
||||
} finally {
|
||||
await fs.rm(codexHome, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
72
tests/unit/codex-gpt55-routing-5887.test.ts
Normal file
72
tests/unit/codex-gpt55-routing-5887.test.ts
Normal file
@@ -0,0 +1,72 @@
|
||||
/**
|
||||
* Issue #5887 — regression: an unprefixed `gpt-5.5` request from a codex-only
|
||||
* setup (no OpenAI connection) stopped auto-routing to the `codex` provider.
|
||||
*
|
||||
* Root cause: `gpt-5.5` was added to the OpenAI static catalog
|
||||
* (`open-sse/config/providers/registry/openai/index.ts`), so the
|
||||
* `if (providers.includes("openai"))` short-circuit in
|
||||
* `resolveModelByProviderInference` (open-sse/services/model.ts) started
|
||||
* firing BEFORE the codex-preference block — making that block unreachable for
|
||||
* `gpt-5.5`. Result: a codex-only user (no OpenAI connection) had `gpt-5.5`
|
||||
* routed to `openai`, and Codex-only hosted image generation failed.
|
||||
*
|
||||
* Fix: move the codex-preference block ahead of the OpenAI short-circuit. It is
|
||||
* guarded by `!activeProviders.has("openai")`, so users WITH an active OpenAI
|
||||
* connection keep OpenAI as the default (behavior preserved).
|
||||
*/
|
||||
import test from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import fs from "node:fs";
|
||||
import os from "node:os";
|
||||
import path from "node:path";
|
||||
|
||||
const TEST_DATA_DIR = fs.mkdtempSync(path.join(os.tmpdir(), "omniroute-gpt55-5887-"));
|
||||
process.env.DATA_DIR = TEST_DATA_DIR;
|
||||
|
||||
const core = await import("../../src/lib/db/core.ts");
|
||||
const providersDb = await import("../../src/lib/db/providers.ts");
|
||||
const { getModelInfoCore } = await import("../../open-sse/services/model.ts");
|
||||
|
||||
let openaiConnectionId: number | string | undefined;
|
||||
|
||||
test.after(() => {
|
||||
core.resetDbInstance();
|
||||
fs.rmSync(TEST_DATA_DIR, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
// (a) Codex active, OpenAI NOT active → bare gpt-5.5 must infer codex.
|
||||
// FAILS before the fix (OpenAI static-catalog short-circuit wins).
|
||||
test("#5887(a) codex-only setup infers codex for unprefixed gpt-5.5", async () => {
|
||||
await providersDb.createProviderConnection({
|
||||
provider: "codex",
|
||||
authType: "oauth",
|
||||
email: "codex@example.com",
|
||||
providerSpecificData: { workspaceId: "ws-1" },
|
||||
});
|
||||
|
||||
const info = await getModelInfoCore("gpt-5.5", null);
|
||||
assert.equal(info.provider, "codex", "gpt-5.5 must infer codex when only codex is active");
|
||||
// #2877: the bare id must be preserved — no `-medium` effort suffix baked in.
|
||||
assert.equal(info.model, "gpt-5.5", "codex inference keeps the bare gpt-5.5 id");
|
||||
});
|
||||
|
||||
// (b) OpenAI active → OpenAI stays the default for gpt-5.5 (behavior preserved).
|
||||
test("#5887(b) active OpenAI connection keeps gpt-5.5 on openai", async () => {
|
||||
const conn = await providersDb.createProviderConnection({
|
||||
provider: "openai",
|
||||
authType: "apikey",
|
||||
apiKey: "sk-test",
|
||||
});
|
||||
openaiConnectionId = (conn as { id?: number | string })?.id;
|
||||
|
||||
const info = await getModelInfoCore("gpt-5.5", null);
|
||||
assert.equal(info.provider, "openai", "OpenAI stays default when openai is active");
|
||||
assert.equal(info.model, "gpt-5.5");
|
||||
});
|
||||
|
||||
// (c) Non-regression: a normal OpenAI model still routes to openai.
|
||||
test("#5887(c) gpt-4o routes to openai with openai active", async () => {
|
||||
assert.ok(openaiConnectionId !== undefined, "openai connection created in (b)");
|
||||
const info = await getModelInfoCore("gpt-4o", null);
|
||||
assert.equal(info.provider, "openai");
|
||||
});
|
||||
89
tests/unit/codex-profile-auto-sync-gate.test.ts
Normal file
89
tests/unit/codex-profile-auto-sync-gate.test.ts
Normal file
@@ -0,0 +1,89 @@
|
||||
import test from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
|
||||
import { autoSyncCodexProfilesFromLiveCatalog } from "@/lib/cli-helper/codexProfileAutoSync";
|
||||
|
||||
// #5737 hardening: the Codex profile auto-sync writes files into the operator's ~/.codex,
|
||||
// so it MUST be opt-in (default OFF) and MUST short-circuit before any catalog fetch / file
|
||||
// write when either gate is closed. These tests pin the two gates (env opt-in + the existing
|
||||
// CLI_ALLOW_CONFIG_WRITES write-guard) so a future change can't silently turn it back on.
|
||||
|
||||
const GATE_ENV = ["OMNIROUTE_AUTO_SYNC_CODEX_PROFILES", "CLI_ALLOW_CONFIG_WRITES"] as const;
|
||||
|
||||
function snapshotEnv(): Record<string, string | undefined> {
|
||||
const snap: Record<string, string | undefined> = {};
|
||||
for (const k of GATE_ENV) snap[k] = process.env[k];
|
||||
return snap;
|
||||
}
|
||||
|
||||
function restoreEnv(snap: Record<string, string | undefined>): void {
|
||||
for (const k of GATE_ENV) {
|
||||
if (snap[k] === undefined) delete process.env[k];
|
||||
else process.env[k] = snap[k];
|
||||
}
|
||||
}
|
||||
|
||||
function mockSyncRequest(): Request {
|
||||
// A local model-sync request; the gate returns before any header is actually forwarded.
|
||||
return new Request("http://127.0.0.1:20128/api/providers/openai/sync-models", { method: "POST" });
|
||||
}
|
||||
|
||||
test("auto-sync is OFF by default (flag unset) — returns disabled, never fetches or writes", async () => {
|
||||
const snap = snapshotEnv();
|
||||
try {
|
||||
delete process.env.OMNIROUTE_AUTO_SYNC_CODEX_PROFILES;
|
||||
const result = await autoSyncCodexProfilesFromLiveCatalog(
|
||||
mockSyncRequest(),
|
||||
"test:default-off"
|
||||
);
|
||||
assert.equal(result.ok, false);
|
||||
assert.equal(result.reason, "disabled");
|
||||
} finally {
|
||||
restoreEnv(snap);
|
||||
}
|
||||
});
|
||||
|
||||
test("explicit OMNIROUTE_AUTO_SYNC_CODEX_PROFILES=false stays disabled", async () => {
|
||||
const snap = snapshotEnv();
|
||||
try {
|
||||
process.env.OMNIROUTE_AUTO_SYNC_CODEX_PROFILES = "false";
|
||||
const result = await autoSyncCodexProfilesFromLiveCatalog(
|
||||
mockSyncRequest(),
|
||||
"test:explicit-false"
|
||||
);
|
||||
assert.equal(result.ok, false);
|
||||
assert.equal(result.reason, "disabled");
|
||||
} finally {
|
||||
restoreEnv(snap);
|
||||
}
|
||||
});
|
||||
|
||||
test("non-truthy flag values ('0', 'no', 'off', 'maybe') stay disabled", async () => {
|
||||
const snap = snapshotEnv();
|
||||
try {
|
||||
for (const v of ["0", "no", "off", "maybe", ""]) {
|
||||
process.env.OMNIROUTE_AUTO_SYNC_CODEX_PROFILES = v;
|
||||
const result = await autoSyncCodexProfilesFromLiveCatalog(mockSyncRequest(), `test:${v}`);
|
||||
assert.equal(result.ok, false, `value '${v}' must not enable auto-sync`);
|
||||
assert.equal(result.reason, "disabled", `value '${v}' must report disabled`);
|
||||
}
|
||||
} finally {
|
||||
restoreEnv(snap);
|
||||
}
|
||||
});
|
||||
|
||||
test("enabled flag but CLI_ALLOW_CONFIG_WRITES=false is blocked by the write-guard (no write)", async () => {
|
||||
const snap = snapshotEnv();
|
||||
try {
|
||||
process.env.OMNIROUTE_AUTO_SYNC_CODEX_PROFILES = "true";
|
||||
process.env.CLI_ALLOW_CONFIG_WRITES = "false";
|
||||
const result = await autoSyncCodexProfilesFromLiveCatalog(
|
||||
mockSyncRequest(),
|
||||
"test:write-guard"
|
||||
);
|
||||
assert.equal(result.ok, false);
|
||||
assert.match(result.reason, /CLI config writes are disabled/);
|
||||
} finally {
|
||||
restoreEnv(snap);
|
||||
}
|
||||
});
|
||||
96
tests/unit/combo-response-validation-failover.test.ts
Normal file
96
tests/unit/combo-response-validation-failover.test.ts
Normal file
@@ -0,0 +1,96 @@
|
||||
/**
|
||||
* Feature 4985 — a combo with a configured `responseValidation` predicate must fail over
|
||||
* when a leg returns a 200 OK whose body fails the predicate, exactly like an HTTP error.
|
||||
* Mirrors the #5085 empty-content failover harness (2 legs, different providers).
|
||||
*/
|
||||
import test from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import fs from "node:fs";
|
||||
import os from "node:os";
|
||||
import path from "node:path";
|
||||
|
||||
const TEST_DATA_DIR = fs.mkdtempSync(path.join(os.tmpdir(), "omniroute-combo-4985-"));
|
||||
process.env.DATA_DIR = TEST_DATA_DIR;
|
||||
process.env.API_KEY_SECRET = process.env.API_KEY_SECRET || "combo-4985-test-secret";
|
||||
|
||||
const { handleComboChat } = await import("../../open-sse/services/combo.ts");
|
||||
|
||||
const noop = () => {};
|
||||
const log = { info: noop, warn: noop, debug: noop, error: noop };
|
||||
|
||||
function content200(model: string, text: string) {
|
||||
return new Response(
|
||||
JSON.stringify({
|
||||
id: "ok",
|
||||
object: "chat.completion",
|
||||
model,
|
||||
choices: [{ index: 0, message: { role: "assistant", content: text }, finish_reason: "stop" }],
|
||||
}),
|
||||
{ status: 200, headers: { "Content-Type": "application/json" } }
|
||||
);
|
||||
}
|
||||
|
||||
function makeCombo(models: string[], responseValidation: Record<string, unknown>) {
|
||||
return {
|
||||
name: "test-combo-4985",
|
||||
strategy: "priority",
|
||||
models: models.map((m) => ({ model: m })),
|
||||
config: { responseValidation },
|
||||
};
|
||||
}
|
||||
|
||||
test("4985 fails over when leg 1's 200 body trips a forbidden substring", async () => {
|
||||
const modelsCalled: string[] = [];
|
||||
const handleSingleModel = async (_body: unknown, modelStr: string) => {
|
||||
modelsCalled.push(modelStr);
|
||||
if (modelsCalled.length === 1) return content200(modelStr, "Sorry, I cannot help with that.");
|
||||
return content200(modelStr, "Here is the answer you asked for.");
|
||||
};
|
||||
|
||||
const result = await handleComboChat({
|
||||
body: { model: "test", messages: [{ role: "user", content: "hi" }] },
|
||||
combo: makeCombo(["nvidia/minimaxai/minimax-m3", "openai/gpt-4o-mini"], {
|
||||
forbiddenSubstrings: ["I cannot help"],
|
||||
}),
|
||||
handleSingleModel,
|
||||
log,
|
||||
settings: {},
|
||||
allCombos: [],
|
||||
});
|
||||
|
||||
assert.equal(
|
||||
modelsCalled.length,
|
||||
2,
|
||||
`predicate failure on leg 1 must advance to leg 2, tried: ${modelsCalled.join(", ")}`
|
||||
);
|
||||
assert.equal(result.status, 200, "the combo must surface the healthy second leg's 200");
|
||||
const body = JSON.parse(await result.text());
|
||||
assert.match(
|
||||
String(body?.choices?.[0]?.message?.content ?? ""),
|
||||
/answer you asked for/,
|
||||
"surfaced content must come from the leg that passed the predicate"
|
||||
);
|
||||
});
|
||||
|
||||
test("4985 does NOT fail over when the configured predicate passes", async () => {
|
||||
const modelsCalled: string[] = [];
|
||||
const handleSingleModel = async (_body: unknown, modelStr: string) => {
|
||||
modelsCalled.push(modelStr);
|
||||
return content200(modelStr, "A perfectly good answer.");
|
||||
};
|
||||
|
||||
const result = await handleComboChat({
|
||||
body: { model: "test", messages: [{ role: "user", content: "hi" }] },
|
||||
combo: makeCombo(["nvidia/minimaxai/minimax-m3", "openai/gpt-4o-mini"], {
|
||||
forbiddenSubstrings: ["I cannot help"],
|
||||
minContentLength: 5,
|
||||
}),
|
||||
handleSingleModel,
|
||||
log,
|
||||
settings: {},
|
||||
allCombos: [],
|
||||
});
|
||||
|
||||
assert.equal(modelsCalled.length, 1, "a passing predicate must not trigger a needless failover");
|
||||
assert.equal(result.status, 200);
|
||||
});
|
||||
117
tests/unit/combo-response-validation.test.ts
Normal file
117
tests/unit/combo-response-validation.test.ts
Normal file
@@ -0,0 +1,117 @@
|
||||
import test from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import {
|
||||
evaluateResponseValidation,
|
||||
resolveJsonPath,
|
||||
parseJsonPath,
|
||||
extractContentText,
|
||||
type ResponseValidationConfig,
|
||||
} from "../../open-sse/services/combo/responseValidation.ts";
|
||||
|
||||
// Feature 4985 — configurable response-body validation predicate.
|
||||
|
||||
const chat = (content: string | null, extra: Record<string, unknown> = {}) => ({
|
||||
choices: [{ message: { content, ...extra } }],
|
||||
});
|
||||
|
||||
test("no config → always valid", () => {
|
||||
assert.deepEqual(evaluateResponseValidation(chat("anything"), undefined), { valid: true });
|
||||
assert.deepEqual(evaluateResponseValidation(chat("anything"), null), { valid: true });
|
||||
assert.deepEqual(evaluateResponseValidation(chat("anything"), {}), { valid: true });
|
||||
});
|
||||
|
||||
test("forbiddenSubstrings: fails when the content contains one", () => {
|
||||
const cfg: ResponseValidationConfig = { forbiddenSubstrings: ["I cannot help", "as an AI"] };
|
||||
assert.equal(evaluateResponseValidation(chat("Sure, here you go"), cfg).valid, true);
|
||||
const bad = evaluateResponseValidation(chat("Sorry, I cannot help with that"), cfg);
|
||||
assert.equal(bad.valid, false);
|
||||
assert.match(bad.reason ?? "", /forbidden substring/);
|
||||
});
|
||||
|
||||
test("requiredSubstrings: fails when a required substring is missing", () => {
|
||||
const cfg: ResponseValidationConfig = { requiredSubstrings: ["```"] };
|
||||
assert.equal(evaluateResponseValidation(chat("```js\ncode\n```"), cfg).valid, true);
|
||||
assert.equal(evaluateResponseValidation(chat("no fence here"), cfg).valid, false);
|
||||
});
|
||||
|
||||
test("minContentLength: fails on near-empty content", () => {
|
||||
const cfg: ResponseValidationConfig = { minContentLength: 10 };
|
||||
assert.equal(evaluateResponseValidation(chat("plenty of characters here"), cfg).valid, true);
|
||||
assert.equal(evaluateResponseValidation(chat(" hi "), cfg).valid, false);
|
||||
});
|
||||
|
||||
test("jsonPathPredicates: exists / nonEmpty / equals / notEquals", () => {
|
||||
const body = chat("hello", { tool_calls: [] });
|
||||
(body as Record<string, unknown>).usage = { total_tokens: 0 };
|
||||
(body.choices[0] as Record<string, unknown>).finish_reason = "stop";
|
||||
|
||||
assert.equal(
|
||||
evaluateResponseValidation(body, {
|
||||
jsonPathPredicates: [{ path: "choices[0].message.content", condition: "nonEmpty" }],
|
||||
}).valid,
|
||||
true
|
||||
);
|
||||
assert.equal(
|
||||
evaluateResponseValidation(body, {
|
||||
jsonPathPredicates: [{ path: "choices[0].message.refusal", condition: "exists" }],
|
||||
}).valid,
|
||||
false
|
||||
);
|
||||
assert.equal(
|
||||
evaluateResponseValidation(body, {
|
||||
jsonPathPredicates: [{ path: "choices[0].finish_reason", condition: "equals", value: "stop" }],
|
||||
}).valid,
|
||||
true
|
||||
);
|
||||
assert.equal(
|
||||
evaluateResponseValidation(body, {
|
||||
jsonPathPredicates: [
|
||||
{ path: "choices[0].finish_reason", condition: "notEquals", value: "content_filter" },
|
||||
],
|
||||
}).valid,
|
||||
true
|
||||
);
|
||||
assert.equal(
|
||||
evaluateResponseValidation(body, {
|
||||
jsonPathPredicates: [{ path: "choices[0].finish_reason", condition: "equals", value: "length" }],
|
||||
}).valid,
|
||||
false
|
||||
);
|
||||
});
|
||||
|
||||
test("the first failing check wins; otherwise valid", () => {
|
||||
const cfg: ResponseValidationConfig = {
|
||||
forbiddenSubstrings: ["BAD"],
|
||||
requiredSubstrings: ["GOOD"],
|
||||
minContentLength: 3,
|
||||
};
|
||||
assert.equal(evaluateResponseValidation(chat("this is GOOD enough"), cfg).valid, true);
|
||||
assert.equal(evaluateResponseValidation(chat("this is BAD and GOOD"), cfg).valid, false);
|
||||
});
|
||||
|
||||
test("parseJsonPath tokenizes dot + bracket paths without regex", () => {
|
||||
assert.deepEqual(parseJsonPath("choices[0].message.content"), ["choices", 0, "message", "content"]);
|
||||
assert.deepEqual(parseJsonPath("a[1][2].b"), ["a", 1, 2, "b"]);
|
||||
assert.deepEqual(parseJsonPath("plain"), ["plain"]);
|
||||
});
|
||||
|
||||
test("resolveJsonPath returns undefined for missing hops (no throw)", () => {
|
||||
const obj = { choices: [{ message: { content: "x" } }] };
|
||||
assert.equal(resolveJsonPath(obj, "choices[0].message.content"), "x");
|
||||
assert.equal(resolveJsonPath(obj, "choices[5].message.content"), undefined);
|
||||
assert.equal(resolveJsonPath(obj, "a.b.c"), undefined);
|
||||
assert.equal(resolveJsonPath(null, "a.b"), undefined);
|
||||
});
|
||||
|
||||
test("extractContentText handles string, array parts, and Responses API output", () => {
|
||||
assert.equal(extractContentText(chat("hello")), "hello");
|
||||
assert.equal(
|
||||
extractContentText({ choices: [{ message: { content: [{ text: "a" }, { text: "b" }] } }] }),
|
||||
"ab"
|
||||
);
|
||||
assert.equal(
|
||||
extractContentText({ output: [{ content: [{ text: "resp" }] }] }),
|
||||
"resp"
|
||||
);
|
||||
assert.equal(extractContentText({}), "");
|
||||
});
|
||||
@@ -118,6 +118,40 @@ test("an unknown provider is never marked (guard)", () => {
|
||||
assert.equal(s.exhaustedConnections.size, 0);
|
||||
});
|
||||
|
||||
test("structuredError.code takes precedence over raw errorText for exhaustion classification", () => {
|
||||
const s = sets();
|
||||
const exhausted = applyComboTargetExhaustion(target(), {
|
||||
...baseOpts,
|
||||
errorText: "Resource has been exhausted (e.g. check quota).",
|
||||
result: { status: 429 },
|
||||
fallbackResult: {},
|
||||
sets: s,
|
||||
structuredError: { code: "rate_limit_exceeded" },
|
||||
});
|
||||
assert.equal(exhausted, false, "rate_limit_exceeded should NOT mark provider exhausted");
|
||||
assert.ok(
|
||||
s.transientRateLimitedProviders.has("test-dedup-provider"),
|
||||
"should be in transientRateLimitedProviders"
|
||||
);
|
||||
});
|
||||
|
||||
test("structuredError.code with non-matching value falls back to classifyErrorText behavior", () => {
|
||||
const s = sets();
|
||||
const exhausted = applyComboTargetExhaustion(target(), {
|
||||
...baseOpts,
|
||||
errorText: "Rate limit reached",
|
||||
result: { status: 429 },
|
||||
fallbackResult: {},
|
||||
sets: s,
|
||||
structuredError: { code: "some_unknown_code" },
|
||||
});
|
||||
assert.equal(exhausted, false, "unknown code + non-quota errorText → not exhausted");
|
||||
assert.ok(
|
||||
s.transientRateLimitedProviders.has("test-dedup-provider"),
|
||||
"should be in transientRateLimitedProviders"
|
||||
);
|
||||
});
|
||||
|
||||
test("a 200/benign status with no exhaustion mutates nothing and returns false", () => {
|
||||
const s = sets();
|
||||
const exhausted = applyComboTargetExhaustion(target(), {
|
||||
@@ -127,5 +161,8 @@ test("a 200/benign status with no exhaustion mutates nothing and returns false",
|
||||
sets: s,
|
||||
});
|
||||
assert.equal(exhausted, false);
|
||||
assert.equal(s.exhaustedProviders.size + s.exhaustedConnections.size + s.transientRateLimitedProviders.size, 0);
|
||||
assert.equal(
|
||||
s.exhaustedProviders.size + s.exhaustedConnections.size + s.transientRateLimitedProviders.size,
|
||||
0
|
||||
);
|
||||
});
|
||||
|
||||
508
tests/unit/command-code-vision.test.ts
Normal file
508
tests/unit/command-code-vision.test.ts
Normal file
@@ -0,0 +1,508 @@
|
||||
/**
|
||||
* Vision / multimodal support tests for the Command Code executor.
|
||||
*
|
||||
* Verifies that vision-capable models (MiniMax M3, MiMo V2.5, Kimi K2, Qwen 3.x, GPT-5, Claude 3/4, Fable 5, Gemini 3.x, Stepfun, Fugu, etc.)
|
||||
* receive image parts in Command Code CLI format, while text-only
|
||||
* models strip images as before (no regression).
|
||||
*/
|
||||
import test from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import fs from "node:fs";
|
||||
import os from "node:os";
|
||||
import path from "node:path";
|
||||
|
||||
const TEST_DATA_DIR = fs.mkdtempSync(path.join(os.tmpdir(), "omniroute-cmd-code-vision-"));
|
||||
process.env.DATA_DIR = TEST_DATA_DIR;
|
||||
|
||||
const { getExecutor } = await import("../../open-sse/executors/index.ts");
|
||||
const core = await import("../../src/lib/db/core.ts");
|
||||
|
||||
const originalFetch = globalThis.fetch;
|
||||
|
||||
function commandCodeStream(lines: unknown[]) {
|
||||
const text = lines.map((l) => JSON.stringify(l)).join("\n") + "\n";
|
||||
return new Response(text, { status: 200, headers: { "Content-Type": "application/x-ndjson" } });
|
||||
}
|
||||
|
||||
test.after(() => {
|
||||
globalThis.fetch = originalFetch;
|
||||
core.resetDbInstance();
|
||||
fs.rmSync(TEST_DATA_DIR, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
test.afterEach(() => {
|
||||
globalThis.fetch = originalFetch;
|
||||
});
|
||||
|
||||
// ── helpers ────────────────────────────────────────────────────────────
|
||||
|
||||
type FetchCall = { url: string; init: Record<string, unknown>; body: Record<string, unknown> };
|
||||
|
||||
function captureFetch(response: Response) {
|
||||
const calls: FetchCall[] = [];
|
||||
globalThis.fetch = async (url, init: RequestInit = {}) => {
|
||||
calls.push({
|
||||
url: String(url),
|
||||
init: init as Record<string, unknown>,
|
||||
body: JSON.parse(String(init.body)),
|
||||
});
|
||||
return response;
|
||||
};
|
||||
return calls;
|
||||
}
|
||||
|
||||
function userContent(calls: FetchCall[]): unknown {
|
||||
return (
|
||||
(calls[0].body.params as Record<string, unknown[]>).messages as Record<string, unknown>[]
|
||||
)[0].content;
|
||||
}
|
||||
|
||||
// ── vision models: image parts preserved in CC CLI format ─────────────
|
||||
|
||||
test("vision model minimax-m3 preserves image_url part as CC CLI {type:image}", async () => {
|
||||
const calls = captureFetch(
|
||||
commandCodeStream([{ type: "text-delta", text: "ok" }, { type: "finish" }])
|
||||
);
|
||||
|
||||
await getExecutor("command-code").execute({
|
||||
model: "MiniMaxAI/MiniMax-M3",
|
||||
stream: false,
|
||||
credentials: { apiKey: "cc_test_key" },
|
||||
body: {
|
||||
messages: [
|
||||
{
|
||||
role: "user",
|
||||
content: [
|
||||
{ type: "text", text: "What's in this?" },
|
||||
{
|
||||
type: "image_url",
|
||||
image_url: { url: "data:image/png;base64,iVBORw0KGgo=" },
|
||||
},
|
||||
],
|
||||
},
|
||||
],
|
||||
},
|
||||
});
|
||||
|
||||
const content = userContent(calls);
|
||||
assert.ok(Array.isArray(content), "vision model user content must be an array");
|
||||
const parts = content as Record<string, unknown>[];
|
||||
assert.equal(parts.length, 2);
|
||||
|
||||
// Text part preserved
|
||||
assert.equal(parts[0].type, "text");
|
||||
assert.equal(parts[0].text, "What's in this?");
|
||||
|
||||
// Image part converted to CC CLI format
|
||||
assert.equal(parts[1].type, "image");
|
||||
assert.equal(parts[1].image, "data:image/png;base64,iVBORw0KGgo=");
|
||||
});
|
||||
|
||||
test("vision model minimax-m3 preserves image_url with HTTP URL", async () => {
|
||||
const calls = captureFetch(
|
||||
commandCodeStream([{ type: "text-delta", text: "ok" }, { type: "finish" }])
|
||||
);
|
||||
|
||||
await getExecutor("command-code").execute({
|
||||
model: "minimax-m3",
|
||||
stream: false,
|
||||
credentials: { apiKey: "cc_test_key" },
|
||||
body: {
|
||||
messages: [
|
||||
{
|
||||
role: "user",
|
||||
content: [
|
||||
{ type: "text", text: "Describe" },
|
||||
{
|
||||
type: "image_url",
|
||||
image_url: { url: "https://example.com/photo.jpg" },
|
||||
},
|
||||
],
|
||||
},
|
||||
],
|
||||
},
|
||||
});
|
||||
|
||||
const content = userContent(calls);
|
||||
assert.ok(Array.isArray(content));
|
||||
const parts = content as Record<string, unknown>[];
|
||||
assert.equal(parts.length, 2);
|
||||
assert.equal(parts[1].type, "image");
|
||||
assert.equal(parts[1].image, "https://example.com/photo.jpg");
|
||||
});
|
||||
|
||||
test("vision model mimo-v2.5 preserves image parts", async () => {
|
||||
const calls = captureFetch(
|
||||
commandCodeStream([{ type: "text-delta", text: "ok" }, { type: "finish" }])
|
||||
);
|
||||
|
||||
await getExecutor("command-code").execute({
|
||||
model: "mimo-v2.5",
|
||||
stream: false,
|
||||
credentials: { apiKey: "cc_test_key" },
|
||||
body: {
|
||||
messages: [
|
||||
{
|
||||
role: "user",
|
||||
content: [
|
||||
{ type: "text", text: "Analyze" },
|
||||
{
|
||||
type: "image_url",
|
||||
image_url: { url: "https://example.com/img.png" },
|
||||
},
|
||||
],
|
||||
},
|
||||
],
|
||||
},
|
||||
});
|
||||
|
||||
const content = userContent(calls);
|
||||
assert.ok(Array.isArray(content));
|
||||
const parts = content as Record<string, unknown>[];
|
||||
assert.equal(parts.length, 2);
|
||||
assert.equal(parts[1].type, "image");
|
||||
});
|
||||
|
||||
test("vision model mimo-v2.5-pro is text-only (no image parts)", async () => {
|
||||
const calls = captureFetch(
|
||||
commandCodeStream([{ type: "text-delta", text: "ok" }, { type: "finish" }])
|
||||
);
|
||||
|
||||
await getExecutor("command-code").execute({
|
||||
model: "mimo-v2.5-pro",
|
||||
stream: false,
|
||||
credentials: { apiKey: "cc_test_key" },
|
||||
body: {
|
||||
messages: [
|
||||
{
|
||||
role: "user",
|
||||
content: [
|
||||
{ type: "text", text: "Hi" },
|
||||
{
|
||||
type: "image_url",
|
||||
image_url: { url: "https://example.com/img.png" },
|
||||
},
|
||||
],
|
||||
},
|
||||
],
|
||||
},
|
||||
});
|
||||
|
||||
const content = userContent(calls);
|
||||
// mimo-v2.5-pro is text-only — content must be flattened to a plain string
|
||||
assert.equal(typeof content, "string");
|
||||
assert.equal(content, "Hi");
|
||||
});
|
||||
|
||||
test("vision model mimo-v2-omni preserves image parts", async () => {
|
||||
const calls = captureFetch(
|
||||
commandCodeStream([{ type: "text-delta", text: "ok" }, { type: "finish" }])
|
||||
);
|
||||
|
||||
await getExecutor("command-code").execute({
|
||||
model: "mimo-v2-omni",
|
||||
stream: false,
|
||||
credentials: { apiKey: "cc_test_key" },
|
||||
body: {
|
||||
messages: [
|
||||
{
|
||||
role: "user",
|
||||
content: [
|
||||
{ type: "text", text: "Check" },
|
||||
{
|
||||
type: "image_url",
|
||||
image_url: { url: "data:image/jpeg;base64,/9j/4AAQ=" },
|
||||
},
|
||||
],
|
||||
},
|
||||
],
|
||||
},
|
||||
});
|
||||
|
||||
const content = userContent(calls);
|
||||
assert.ok(Array.isArray(content));
|
||||
const parts = content as Record<string, unknown>[];
|
||||
assert.equal(parts.length, 2);
|
||||
assert.equal(parts[1].type, "image");
|
||||
assert.equal(parts[1].image, "data:image/jpeg;base64,/9j/4AAQ=");
|
||||
});
|
||||
|
||||
// ── non-vision models: images still stripped (no regression) ──────────
|
||||
|
||||
test("text-only model deepseek-v4-pro strips image_url parts", async () => {
|
||||
const calls = captureFetch(
|
||||
commandCodeStream([{ type: "text-delta", text: "ok" }, { type: "finish" }])
|
||||
);
|
||||
|
||||
await getExecutor("command-code").execute({
|
||||
model: "deepseek/deepseek-v4-pro",
|
||||
stream: false,
|
||||
credentials: { apiKey: "cc_test_key" },
|
||||
body: {
|
||||
messages: [
|
||||
{
|
||||
role: "user",
|
||||
content: [
|
||||
{ type: "text", text: "Hello" },
|
||||
{
|
||||
type: "image_url",
|
||||
image_url: { url: "https://example.com/img.png" },
|
||||
},
|
||||
],
|
||||
},
|
||||
],
|
||||
},
|
||||
});
|
||||
|
||||
const content = userContent(calls);
|
||||
// Non-vision model: content is a plain string, images stripped
|
||||
assert.equal(typeof content, "string");
|
||||
assert.equal(content, "Hello");
|
||||
});
|
||||
|
||||
test("text-only model deepseek-v4-flash strips image_url parts (no regression)", async () => {
|
||||
const calls = captureFetch(
|
||||
commandCodeStream([{ type: "text-delta", text: "ok" }, { type: "finish" }])
|
||||
);
|
||||
|
||||
await getExecutor("command-code").execute({
|
||||
model: "deepseek/deepseek-v4-flash",
|
||||
stream: false,
|
||||
credentials: { apiKey: "cc_test_key" },
|
||||
body: {
|
||||
messages: [
|
||||
{
|
||||
role: "user",
|
||||
content: [
|
||||
{ type: "text", text: "Text only" },
|
||||
{
|
||||
type: "image_url",
|
||||
image_url: { url: "data:image/png;base64,AAA=" },
|
||||
},
|
||||
],
|
||||
},
|
||||
],
|
||||
},
|
||||
});
|
||||
|
||||
const content = userContent(calls);
|
||||
assert.equal(typeof content, "string");
|
||||
assert.equal(content, "Text only");
|
||||
});
|
||||
|
||||
// ── edge cases ────────────────────────────────────────────────────────
|
||||
|
||||
test("vision model with only image content emits empty text fallback", async () => {
|
||||
const calls = captureFetch(
|
||||
commandCodeStream([{ type: "text-delta", text: "ok" }, { type: "finish" }])
|
||||
);
|
||||
|
||||
await getExecutor("command-code").execute({
|
||||
model: "minimax-m3",
|
||||
stream: false,
|
||||
credentials: { apiKey: "cc_test_key" },
|
||||
body: {
|
||||
messages: [
|
||||
{
|
||||
role: "user",
|
||||
content: [
|
||||
{
|
||||
type: "image_url",
|
||||
image_url: { url: "data:image/png;base64,iVBOR=" },
|
||||
},
|
||||
],
|
||||
},
|
||||
],
|
||||
},
|
||||
});
|
||||
|
||||
const content = userContent(calls);
|
||||
assert.ok(Array.isArray(content));
|
||||
const parts = content as Record<string, unknown>[];
|
||||
// Single image part preserved — no empty text injected because
|
||||
// the image itself keeps content non-empty.
|
||||
assert.equal(parts.length, 1);
|
||||
assert.equal(parts[0].type, "image");
|
||||
assert.equal(parts[0].image, "data:image/png;base64,iVBOR=");
|
||||
});
|
||||
|
||||
test("vision model passes plain string content through unchanged", async () => {
|
||||
const calls = captureFetch(
|
||||
commandCodeStream([{ type: "text-delta", text: "ok" }, { type: "finish" }])
|
||||
);
|
||||
|
||||
await getExecutor("command-code").execute({
|
||||
model: "minimax-m3",
|
||||
stream: false,
|
||||
credentials: { apiKey: "cc_test_key" },
|
||||
body: {
|
||||
messages: [{ role: "user", content: "Plain string message" }],
|
||||
},
|
||||
});
|
||||
|
||||
const content = userContent(calls);
|
||||
assert.equal(typeof content, "string");
|
||||
assert.equal(content, "Plain string message");
|
||||
});
|
||||
|
||||
test("vision model honors body.model rewrite for vision detection", async () => {
|
||||
// #5166 scenario: body.model overwrites the execute model arg.
|
||||
// Vision detection must use the rewritten model id.
|
||||
const calls = captureFetch(
|
||||
commandCodeStream([{ type: "text-delta", text: "ok" }, { type: "finish" }])
|
||||
);
|
||||
|
||||
// execute() gets a non-vision combo model, body.model rewrites to a vision model
|
||||
await getExecutor("command-code").execute({
|
||||
model: "gpt-5.4-mini",
|
||||
stream: false,
|
||||
credentials: { apiKey: "cc_test_key" },
|
||||
body: {
|
||||
model: "MiniMaxAI/MiniMax-M3",
|
||||
messages: [
|
||||
{
|
||||
role: "user",
|
||||
content: [
|
||||
{ type: "text", text: "Describe" },
|
||||
{
|
||||
type: "image_url",
|
||||
image_url: { url: "https://example.com/img.png" },
|
||||
},
|
||||
],
|
||||
},
|
||||
],
|
||||
},
|
||||
});
|
||||
|
||||
const content = userContent(calls);
|
||||
// body.model = MiniMax-M3 (vision) → images preserved
|
||||
assert.ok(Array.isArray(content));
|
||||
const parts = content as Record<string, unknown>[];
|
||||
assert.equal(parts.length, 2);
|
||||
assert.equal(parts[1].type, "image");
|
||||
});
|
||||
|
||||
test("vision model with multiple image parts preserves all of them", async () => {
|
||||
const calls = captureFetch(
|
||||
commandCodeStream([{ type: "text-delta", text: "ok" }, { type: "finish" }])
|
||||
);
|
||||
|
||||
await getExecutor("command-code").execute({
|
||||
model: "minimax-m3",
|
||||
stream: false,
|
||||
credentials: { apiKey: "cc_test_key" },
|
||||
body: {
|
||||
messages: [
|
||||
{
|
||||
role: "user",
|
||||
content: [
|
||||
{ type: "text", text: "Compare" },
|
||||
{
|
||||
type: "image_url",
|
||||
image_url: { url: "https://example.com/a.jpg" },
|
||||
},
|
||||
{
|
||||
type: "image_url",
|
||||
image_url: { url: "https://example.com/b.jpg" },
|
||||
},
|
||||
],
|
||||
},
|
||||
],
|
||||
},
|
||||
});
|
||||
|
||||
const content = userContent(calls);
|
||||
assert.ok(Array.isArray(content));
|
||||
const parts = content as Record<string, unknown>[];
|
||||
assert.equal(parts.length, 3);
|
||||
assert.equal(parts[0].type, "text");
|
||||
assert.equal(parts[1].type, "image");
|
||||
assert.equal(parts[1].image, "https://example.com/a.jpg");
|
||||
assert.equal(parts[2].type, "image");
|
||||
assert.equal(parts[2].image, "https://example.com/b.jpg");
|
||||
});
|
||||
|
||||
test("vision model with image_url as plain string (no object wrapper) still works", async () => {
|
||||
const calls = captureFetch(
|
||||
commandCodeStream([{ type: "text-delta", text: "ok" }, { type: "finish" }])
|
||||
);
|
||||
|
||||
await getExecutor("command-code").execute({
|
||||
model: "minimax-m3",
|
||||
stream: false,
|
||||
credentials: { apiKey: "cc_test_key" },
|
||||
body: {
|
||||
messages: [
|
||||
{
|
||||
role: "user",
|
||||
content: [
|
||||
{ type: "text", text: "Look" },
|
||||
{
|
||||
type: "image_url",
|
||||
image_url: "https://example.com/img.png",
|
||||
},
|
||||
],
|
||||
},
|
||||
],
|
||||
},
|
||||
});
|
||||
|
||||
const content = userContent(calls);
|
||||
assert.ok(Array.isArray(content));
|
||||
const parts = content as Record<string, unknown>[];
|
||||
assert.equal(parts.length, 2);
|
||||
assert.equal(parts[1].type, "image");
|
||||
assert.equal(parts[1].image, "https://example.com/img.png");
|
||||
});
|
||||
|
||||
// ── CC vision models (Command Code docs registry) ──────────────────
|
||||
|
||||
const VISION_CASES = [
|
||||
["Kimi K2.6", "moonshotai/Kimi-K2.6"],
|
||||
["Kimi K2.7 Code", "moonshotai/Kimi-K2.7-Code"],
|
||||
["Kimi K2.5", "moonshotai/Kimi-K2.5"],
|
||||
["Qwen 3.6 Plus", "Qwen/Qwen3.6-Plus"],
|
||||
["Qwen 3.7 Plus", "Qwen/Qwen3.7-Plus"],
|
||||
["Step 3.7 Flash", "stepfun/Step-3.7-Flash"],
|
||||
["GPT-5.5", "gpt-5.5"],
|
||||
["GPT-5.4", "gpt-5.4"],
|
||||
["GPT-5.3 Codex", "gpt-5.3-codex"],
|
||||
["GPT-5.4 Mini", "gpt-5.4-mini"],
|
||||
["Claude Fable 5", "claude-fable-5"],
|
||||
["Sakana Fugu Ultra", "sakana/fugu-ultra"],
|
||||
["Claude Opus 4.7 (isVisionModelId)", "claude-opus-4-7"],
|
||||
["Claude Sonnet 4.6 (isVisionModelId)", "claude-sonnet-4-6"],
|
||||
["Gemini 3.5 Flash (isVisionModelId)", "google/gemini-3.5-flash"],
|
||||
];
|
||||
|
||||
for (const [name, model] of VISION_CASES) {
|
||||
test(`vision model ${name} preserves image parts`, async () => {
|
||||
const calls = captureFetch(
|
||||
commandCodeStream([{ type: "text-delta", text: "ok" }, { type: "finish" }])
|
||||
);
|
||||
await getExecutor("command-code").execute({
|
||||
model,
|
||||
stream: false,
|
||||
credentials: { apiKey: "cc_test_key" },
|
||||
body: {
|
||||
messages: [
|
||||
{
|
||||
role: "user",
|
||||
content: [
|
||||
{ type: "text", text: "Check" },
|
||||
{
|
||||
type: "image_url",
|
||||
image_url: { url: "https://example.com/img.png" },
|
||||
},
|
||||
],
|
||||
},
|
||||
],
|
||||
},
|
||||
});
|
||||
const content = userContent(calls);
|
||||
assert.ok(Array.isArray(content), `${name} user content must be an array`);
|
||||
const parts = content;
|
||||
assert.equal(parts.length, 2);
|
||||
assert.equal(parts[1].type, "image");
|
||||
});
|
||||
}
|
||||
111
tests/unit/compression-pipeline-model.test.ts
Normal file
111
tests/unit/compression-pipeline-model.test.ts
Normal file
@@ -0,0 +1,111 @@
|
||||
import { describe, it } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import {
|
||||
allowedIntensities,
|
||||
normalizeStep,
|
||||
moveLayer,
|
||||
addLayer,
|
||||
removeLayer,
|
||||
updateLayer,
|
||||
type EngineIntensities,
|
||||
type PipelineStep,
|
||||
} from "../../src/shared/components/compression/compressionPipelineModel.ts";
|
||||
|
||||
// T06 — pure pipeline model. Reorder/add/remove/update preserve invariants.
|
||||
|
||||
const TABLE: EngineIntensities = {
|
||||
rtk: ["standard", "aggressive"],
|
||||
caveman: ["lite", "full", "ultra"],
|
||||
llmlingua: ["standard"],
|
||||
};
|
||||
|
||||
const BASE: PipelineStep[] = [
|
||||
{ engine: "rtk", intensity: "standard" },
|
||||
{ engine: "caveman", intensity: "full" },
|
||||
{ engine: "llmlingua", intensity: "standard" },
|
||||
];
|
||||
|
||||
describe("compressionPipelineModel — normalize", () => {
|
||||
it("keeps a valid intensity and coerces an invalid one to the first allowed", () => {
|
||||
assert.deepEqual(normalizeStep({ engine: "caveman", intensity: "ultra" }, TABLE), {
|
||||
engine: "caveman",
|
||||
intensity: "ultra",
|
||||
});
|
||||
assert.deepEqual(normalizeStep({ engine: "caveman", intensity: "bogus" }, TABLE), {
|
||||
engine: "caveman",
|
||||
intensity: "lite",
|
||||
});
|
||||
});
|
||||
|
||||
it("falls back to standard for an unknown engine", () => {
|
||||
assert.deepEqual(allowedIntensities("nope", TABLE), ["standard"]);
|
||||
assert.deepEqual(normalizeStep({ engine: "nope" }, TABLE), {
|
||||
engine: "nope",
|
||||
intensity: "standard",
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe("compressionPipelineModel — moveLayer", () => {
|
||||
it("moves a step and is always a permutation (same length + members)", () => {
|
||||
const moved = moveLayer(BASE, 0, 2);
|
||||
assert.deepEqual(
|
||||
moved.map((s) => s.engine),
|
||||
["caveman", "llmlingua", "rtk"]
|
||||
);
|
||||
assert.equal(moved.length, BASE.length);
|
||||
});
|
||||
|
||||
it("does not mutate the input", () => {
|
||||
const copy = BASE.map((s) => ({ ...s }));
|
||||
moveLayer(BASE, 0, 2);
|
||||
assert.deepEqual(BASE, copy);
|
||||
});
|
||||
|
||||
it("returns an unchanged copy for out-of-range or no-op moves", () => {
|
||||
assert.deepEqual(moveLayer(BASE, 1, 1), BASE);
|
||||
assert.deepEqual(moveLayer(BASE, -1, 2), BASE);
|
||||
assert.deepEqual(moveLayer(BASE, 0, 9), BASE);
|
||||
});
|
||||
});
|
||||
|
||||
describe("compressionPipelineModel — add/remove", () => {
|
||||
it("addLayer appends a normalized step", () => {
|
||||
const next = addLayer(BASE, { engine: "caveman", intensity: "bogus" }, TABLE);
|
||||
assert.equal(next.length, 4);
|
||||
assert.deepEqual(next[3], { engine: "caveman", intensity: "lite" });
|
||||
});
|
||||
|
||||
it("removeLayer drops the indexed step but never goes below minLength", () => {
|
||||
const next = removeLayer(BASE, 1);
|
||||
assert.deepEqual(
|
||||
next.map((s) => s.engine),
|
||||
["rtk", "llmlingua"]
|
||||
);
|
||||
const single: PipelineStep[] = [{ engine: "rtk", intensity: "standard" }];
|
||||
assert.deepEqual(removeLayer(single, 0), single, "must not remove the last step");
|
||||
});
|
||||
|
||||
it("removeLayer ignores out-of-range index", () => {
|
||||
assert.deepEqual(removeLayer(BASE, 9), BASE);
|
||||
});
|
||||
});
|
||||
|
||||
describe("compressionPipelineModel — updateLayer", () => {
|
||||
it("patches a step and re-normalizes intensity for a new engine", () => {
|
||||
const next = updateLayer(BASE, 0, { engine: "caveman" }, TABLE);
|
||||
// rtk(standard) → caveman: 'standard' is not a caveman intensity → first allowed (lite)
|
||||
assert.deepEqual(next[0], { engine: "caveman", intensity: "lite" });
|
||||
// other steps untouched
|
||||
assert.deepEqual(next[1], BASE[1]);
|
||||
});
|
||||
|
||||
it("keeps a still-valid intensity when only the engine changes to a compatible one", () => {
|
||||
const next = updateLayer(BASE, 1, { intensity: "ultra" }, TABLE);
|
||||
assert.deepEqual(next[1], { engine: "caveman", intensity: "ultra" });
|
||||
});
|
||||
|
||||
it("ignores out-of-range index", () => {
|
||||
assert.deepEqual(updateLayer(BASE, 9, { engine: "rtk" }, TABLE), BASE);
|
||||
});
|
||||
});
|
||||
81
tests/unit/compression/cache-aware-preserve-mode.test.ts
Normal file
81
tests/unit/compression/cache-aware-preserve-mode.test.ts
Normal file
@@ -0,0 +1,81 @@
|
||||
/**
|
||||
* T05/C5 — resolveCacheAwareConfig materializes the `preserveSystemPromptMode` enum
|
||||
* into the engine-facing `preserveSystemPrompt` boolean using the cache signal.
|
||||
* The legacy boolean behaviour is covered by strategySelector-cache-aware.test.ts /
|
||||
* compression-cache-guard-3955.test.ts (unchanged); this file covers the new enum.
|
||||
*/
|
||||
import { describe, it } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { resolveCacheAwareConfig } from "../../../open-sse/services/compression/cacheAwareConfig.ts";
|
||||
import type {
|
||||
CompressionConfig,
|
||||
PreserveSystemPromptMode,
|
||||
} from "../../../open-sse/services/compression/types.ts";
|
||||
|
||||
function cfg(overrides: Partial<CompressionConfig> = {}): CompressionConfig {
|
||||
return {
|
||||
enabled: true,
|
||||
defaultMode: "standard",
|
||||
autoTriggerTokens: 0,
|
||||
cacheMinutes: 5,
|
||||
preserveSystemPrompt: true,
|
||||
comboOverrides: {},
|
||||
...overrides,
|
||||
} as CompressionConfig;
|
||||
}
|
||||
|
||||
// A request whose prefix the upstream caches (anthropic + explicit cache_control).
|
||||
const CACHING_BODY = {
|
||||
messages: [{ role: "system", content: "x", cache_control: { type: "ephemeral" } }],
|
||||
};
|
||||
const CACHING_CTX = { provider: "anthropic", targetFormat: "claude" } as const;
|
||||
// google has no prompt caching → no cacheable prefix to protect.
|
||||
const NON_CACHING_BODY = { messages: [{ role: "system", content: "x" }] };
|
||||
const NON_CACHING_CTX = { provider: "google" } as const;
|
||||
|
||||
function resolved(mode: PreserveSystemPromptMode, caching: boolean): boolean {
|
||||
const out = resolveCacheAwareConfig(
|
||||
cfg({ preserveSystemPromptMode: mode, preserveSystemPrompt: true }),
|
||||
caching ? CACHING_BODY : NON_CACHING_BODY,
|
||||
caching ? CACHING_CTX : NON_CACHING_CTX
|
||||
);
|
||||
return out.preserveSystemPrompt;
|
||||
}
|
||||
|
||||
describe("T05/C5 preserveSystemPromptMode -> effective boolean", () => {
|
||||
it("always: preserves regardless of cache", () => {
|
||||
assert.equal(resolved("always", true), true);
|
||||
assert.equal(resolved("always", false), true);
|
||||
});
|
||||
|
||||
it("whenNoCache: preserves only when a cache is present", () => {
|
||||
assert.equal(resolved("whenNoCache", true), true);
|
||||
assert.equal(resolved("whenNoCache", false), false);
|
||||
});
|
||||
|
||||
it("never: compresses the system prompt even when it would break the prompt cache", () => {
|
||||
assert.equal(resolved("never", true), false);
|
||||
assert.equal(resolved("never", false), false);
|
||||
});
|
||||
|
||||
it("the explicit mode overrides a contradicting legacy boolean", () => {
|
||||
// mode=never wins even though the legacy boolean asks to preserve.
|
||||
const out = resolveCacheAwareConfig(
|
||||
cfg({ preserveSystemPromptMode: "never", preserveSystemPrompt: true }),
|
||||
CACHING_BODY,
|
||||
CACHING_CTX
|
||||
);
|
||||
assert.equal(out.preserveSystemPrompt, false);
|
||||
});
|
||||
|
||||
it("no body: honors the mode at its no-cache baseline", () => {
|
||||
assert.equal(
|
||||
resolveCacheAwareConfig(cfg({ preserveSystemPromptMode: "whenNoCache" })).preserveSystemPrompt,
|
||||
false
|
||||
);
|
||||
assert.equal(
|
||||
resolveCacheAwareConfig(cfg({ preserveSystemPromptMode: "always" })).preserveSystemPrompt,
|
||||
true
|
||||
);
|
||||
});
|
||||
});
|
||||
90
tests/unit/compression/ccr-mcp-principal-5649.test.ts
Normal file
90
tests/unit/compression/ccr-mcp-principal-5649.test.ts
Normal file
@@ -0,0 +1,90 @@
|
||||
/**
|
||||
* #5649 — CCR MCP retrieve principal resolution.
|
||||
*
|
||||
* CCR stores blocks keyed by `String(apiKeyInfo.id)` at compression time. The MCP
|
||||
* `omniroute_ccr_retrieve` tool used to resolve the caller via `extra.authInfo.clientId`
|
||||
* (never populated for API-key auth) → "anonymous" → a store-key miss ("block not
|
||||
* found"). The fix resolves the caller's API-key id from the auth headers via the SAME
|
||||
* `getApiKeyMetadata` lookup, so retrieval matches storage — without weakening
|
||||
* cross-tenant IDOR isolation.
|
||||
*/
|
||||
import { test } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
|
||||
import { resolvePrincipalFromHeaders } from "../../../open-sse/mcp-server/mcpCallerIdentity.ts";
|
||||
import {
|
||||
storeBlock,
|
||||
retrieveBlock,
|
||||
handleCcrRetrieve,
|
||||
resetCcrStore,
|
||||
} from "../../../open-sse/services/compression/engines/ccr/index.ts";
|
||||
|
||||
// A fake key→metadata lookup: maps a raw key to a DB-row id, exactly like getApiKeyMetadata.
|
||||
const fakeLookup = (map: Record<string, string>) => async (rawKey: string) =>
|
||||
map[rawKey] ? { id: map[rawKey] } : null;
|
||||
|
||||
test("#5649 resolves a Bearer API key to its principal id (not anonymous)", async () => {
|
||||
const id = await resolvePrincipalFromHeaders(
|
||||
{ Authorization: "Bearer sk-tenant-A" },
|
||||
fakeLookup({ "sk-tenant-A": "42" })
|
||||
);
|
||||
assert.equal(id, "42", "a valid Bearer key must resolve to its api-key id, not undefined/anonymous");
|
||||
});
|
||||
|
||||
test("#5649 resolves x-api-key (with anthropic-version gate) to its principal id", async () => {
|
||||
const id = await resolvePrincipalFromHeaders(
|
||||
{ "x-api-key": "sk-anthropic", "anthropic-version": "2023-06-01" },
|
||||
fakeLookup({ "sk-anthropic": "99" })
|
||||
);
|
||||
assert.equal(id, "99");
|
||||
});
|
||||
|
||||
test("#5649 distinct keys resolve to distinct principals (IDOR isolation preserved)", async () => {
|
||||
const lookup = fakeLookup({ "sk-A": "42", "sk-B": "77" });
|
||||
const a = await resolvePrincipalFromHeaders({ Authorization: "Bearer sk-A" }, lookup);
|
||||
const b = await resolvePrincipalFromHeaders({ Authorization: "Bearer sk-B" }, lookup);
|
||||
assert.equal(a, "42");
|
||||
assert.equal(b, "77");
|
||||
assert.notEqual(a, b);
|
||||
});
|
||||
|
||||
test("#5649 no auth headers → undefined (never calls the lookup)", async () => {
|
||||
let called = false;
|
||||
const id = await resolvePrincipalFromHeaders({}, async () => {
|
||||
called = true;
|
||||
return { id: "x" };
|
||||
});
|
||||
assert.equal(id, undefined);
|
||||
assert.equal(called, false, "must not hit the DB when there is no key");
|
||||
});
|
||||
|
||||
test("#5649 unknown / unresolvable key → undefined (fail closed to anonymous bucket)", async () => {
|
||||
const id = await resolvePrincipalFromHeaders(
|
||||
{ Authorization: "Bearer sk-unknown" },
|
||||
fakeLookup({ "sk-A": "42" })
|
||||
);
|
||||
assert.equal(id, undefined);
|
||||
});
|
||||
|
||||
test("#5649 end-to-end: a block stored under the api-key id is retrievable by the resolved principal, not by another tenant", async () => {
|
||||
resetCcrStore();
|
||||
const lookup = fakeLookup({ "sk-A": "42", "sk-B": "77" });
|
||||
const bigText = "confidential block for tenant 42 ".repeat(40);
|
||||
|
||||
// Storage side (mirrors chatCore: principal = String(apiKeyInfo.id)).
|
||||
const hash = storeBlock(bigText, "42");
|
||||
|
||||
// Retrieval side: resolve the SAME key's headers → "42" → block found.
|
||||
const owner = await resolvePrincipalFromHeaders({ Authorization: "Bearer sk-A" }, lookup);
|
||||
assert.equal(owner, "42");
|
||||
assert.equal(retrieveBlock(hash, owner), bigText, "owner key must retrieve its own block");
|
||||
const ownerResult = handleCcrRetrieve({ hash }, owner);
|
||||
assert.ok("content" in ownerResult, "owner retrieve returns content");
|
||||
|
||||
// A different tenant's key resolves to a different principal → blocked.
|
||||
const other = await resolvePrincipalFromHeaders({ Authorization: "Bearer sk-B" }, lookup);
|
||||
assert.equal(other, "77");
|
||||
assert.equal(retrieveBlock(hash, other), null, "[HIGH IDOR] other tenant must not retrieve the block");
|
||||
const otherResult = handleCcrRetrieve({ hash }, other);
|
||||
assert.ok("error" in otherResult, "[HIGH IDOR] cross-tenant retrieve returns error");
|
||||
});
|
||||
116
tests/unit/compression/ccr-retrieval-ramp.test.ts
Normal file
116
tests/unit/compression/ccr-retrieval-ramp.test.ts
Normal file
@@ -0,0 +1,116 @@
|
||||
import { describe, it, beforeEach } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import {
|
||||
ccrEngine,
|
||||
effectiveMinChars,
|
||||
resolveRetrievalRampFactor,
|
||||
recordRetrieval,
|
||||
storeBlock,
|
||||
resetCcrStore,
|
||||
} from "../../../open-sse/services/compression/engines/ccr/index.ts";
|
||||
|
||||
// T08/H8 — graduated retrieval feedback. Frequently-retrieved CCR blocks resist compression
|
||||
// progressively (effective minChars ramps up), and at/above the threshold are never compressed.
|
||||
|
||||
const BASE = 600;
|
||||
const P = "p1";
|
||||
|
||||
beforeEach(() => resetCcrStore());
|
||||
|
||||
function retrieve(hash: string, n: number) {
|
||||
for (let i = 0; i < n; i++) recordRetrieval(hash, P);
|
||||
}
|
||||
|
||||
describe("effectiveMinChars (H8 ramp)", () => {
|
||||
it("never-retrieved block keeps the base minimum", () => {
|
||||
assert.equal(effectiveMinChars(BASE, "h", P, 2), BASE);
|
||||
});
|
||||
|
||||
it("ramps linearly with retrieval count below the threshold (factor 2)", () => {
|
||||
retrieve("h", 1);
|
||||
assert.equal(effectiveMinChars(BASE, "h", P, 2), BASE * 2); // 1200
|
||||
retrieve("h", 1); // count = 2
|
||||
assert.equal(effectiveMinChars(BASE, "h", P, 2), BASE * 3); // 1800
|
||||
});
|
||||
|
||||
it("returns Infinity at/above the retrieval threshold (subsumes the binary cliff)", () => {
|
||||
retrieve("h", 3);
|
||||
assert.equal(effectiveMinChars(BASE, "h", P, 2), Number.POSITIVE_INFINITY);
|
||||
});
|
||||
|
||||
it("rampFactor <= 1 disables the ramp (binary: base until the cliff)", () => {
|
||||
retrieve("h", 2);
|
||||
assert.equal(effectiveMinChars(BASE, "h", P, 1), BASE);
|
||||
retrieve("h", 1); // count = 3 → cliff still applies
|
||||
assert.equal(effectiveMinChars(BASE, "h", P, 1), Number.POSITIVE_INFINITY);
|
||||
});
|
||||
|
||||
it("isolates retrieval feedback per (principal, hash)", () => {
|
||||
retrieve("h", 2);
|
||||
// different hash → unaffected
|
||||
assert.equal(effectiveMinChars(BASE, "other", P, 2), BASE);
|
||||
});
|
||||
});
|
||||
|
||||
describe("resolveRetrievalRampFactor (env)", () => {
|
||||
it("defaults to 2 with no env", () => {
|
||||
assert.equal(resolveRetrievalRampFactor({} as NodeJS.ProcessEnv), 2);
|
||||
});
|
||||
it("reads a valid env value", () => {
|
||||
assert.equal(
|
||||
resolveRetrievalRampFactor({
|
||||
COMPRESSION_CCR_RETRIEVAL_RAMP_FACTOR: "3",
|
||||
} as NodeJS.ProcessEnv),
|
||||
3
|
||||
);
|
||||
});
|
||||
it("falls back to the default on an invalid value", () => {
|
||||
assert.equal(
|
||||
resolveRetrievalRampFactor({
|
||||
COMPRESSION_CCR_RETRIEVAL_RAMP_FACTOR: "0",
|
||||
} as NodeJS.ProcessEnv),
|
||||
2
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
describe("ccrEngine.apply — retrieval-aware compression (H8)", () => {
|
||||
const block = (len: number) => "x".repeat(len);
|
||||
const run = (content: string, retrievalRampFactor = 2) =>
|
||||
ccrEngine.apply(
|
||||
{ messages: [{ role: "user", content }] },
|
||||
{ stepConfig: { minChars: BASE, retrievalRampFactor }, principalId: P }
|
||||
);
|
||||
|
||||
it("compresses a large never-retrieved block normally", () => {
|
||||
const out = run(block(1000));
|
||||
assert.equal(out.compressed, true);
|
||||
});
|
||||
|
||||
it("a block retrieved twice (effMin 1800) is skipped at 1000 chars but compressed at 2000", () => {
|
||||
const hash = storeBlock(block(1000), P); // hash of the 1000-char block
|
||||
retrieve(hash, 2); // count 2 → effMin = base*3 = 1800
|
||||
assert.equal(run(block(1000)).compressed, false, "1000 < 1800 → skipped by the ramp");
|
||||
|
||||
// a distinct, larger block that has NOT been retrieved still compresses
|
||||
assert.equal(run(block(2000)).compressed, true);
|
||||
});
|
||||
|
||||
it("a block retrieved >= threshold is never compressed", () => {
|
||||
const hash = storeBlock(block(5000), P);
|
||||
retrieve(hash, 3);
|
||||
assert.equal(run(block(5000)).compressed, false, "retrieved 3x → excluded (Infinity)");
|
||||
});
|
||||
|
||||
it("rampFactor 1 keeps the legacy binary behavior (retrieved-twice still compresses)", () => {
|
||||
const hash = storeBlock(block(1000), P);
|
||||
retrieve(hash, 2);
|
||||
assert.equal(
|
||||
run(block(1000), 1).compressed,
|
||||
true,
|
||||
"no ramp → still compressed below threshold"
|
||||
);
|
||||
retrieve(hash, 1); // count 3
|
||||
assert.equal(run(block(1000), 1).compressed, false, "cliff still applies at the threshold");
|
||||
});
|
||||
});
|
||||
@@ -51,6 +51,7 @@ describe("getCompressionSettings", () => {
|
||||
assert.equal(settings.autoTriggerTokens, 0);
|
||||
assert.equal(settings.cacheMinutes, 5);
|
||||
assert.equal(settings.preserveSystemPrompt, true);
|
||||
assert.equal(settings.preserveSystemPromptMode, "always");
|
||||
assert.deepEqual(settings.comboOverrides, {});
|
||||
assert.equal(settings.ultra?.enabled, false);
|
||||
assert.equal(settings.ultra?.compressionRate, 0.5);
|
||||
@@ -77,6 +78,25 @@ describe("updateCompressionSettings", () => {
|
||||
await updateCompressionSettings({ defaultMode: "off" } as any);
|
||||
});
|
||||
|
||||
it("round-trips preserveSystemPromptMode and ignores unknown tokens (T05/C5)", async () => {
|
||||
await updateCompressionSettings({ preserveSystemPromptMode: "never" } as any);
|
||||
let settings = await getCompressionSettings();
|
||||
assert.equal(settings.preserveSystemPromptMode, "never");
|
||||
|
||||
await updateCompressionSettings({ preserveSystemPromptMode: "whenNoCache" } as any);
|
||||
settings = await getCompressionSettings();
|
||||
assert.equal(settings.preserveSystemPromptMode, "whenNoCache");
|
||||
|
||||
// An unknown persisted token is rejected on read and falls back to the safe
|
||||
// default mode ("always"), never crashing the settings load.
|
||||
await updateCompressionSettings({ preserveSystemPromptMode: "garbage" } as any);
|
||||
settings = await getCompressionSettings();
|
||||
assert.equal(settings.preserveSystemPromptMode, "always");
|
||||
|
||||
// Reset
|
||||
await updateCompressionSettings({ preserveSystemPromptMode: "always" } as any);
|
||||
});
|
||||
|
||||
it("updates autoTriggerTokens", async () => {
|
||||
await updateCompressionSettings({ autoTriggerTokens: 5000 } as any);
|
||||
const settings = await getCompressionSettings();
|
||||
|
||||
110
tests/unit/compression/llm-compressor-engine.test.ts
Normal file
110
tests/unit/compression/llm-compressor-engine.test.ts
Normal file
@@ -0,0 +1,110 @@
|
||||
import { describe, it, afterEach } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import {
|
||||
llmCompressorEngine,
|
||||
setLlmCompressorBackend,
|
||||
type LlmCompressorBackend,
|
||||
} from "../../../open-sse/services/compression/engines/llm/index.ts";
|
||||
|
||||
// T05/C3 — opt-in LLM-tier compressor. Default-off / no-op; fail-open; code & system protected.
|
||||
|
||||
// A fake backend that halves prose length (always shorter for len >= 2).
|
||||
const halveBackend: LlmCompressorBackend = async (text) =>
|
||||
text.slice(0, Math.max(1, Math.floor(text.length / 2)));
|
||||
|
||||
const throwingBackend: LlmCompressorBackend = async () => {
|
||||
throw new Error("backend boom");
|
||||
};
|
||||
|
||||
function body(messages: unknown[]) {
|
||||
return { messages } as Record<string, unknown>;
|
||||
}
|
||||
|
||||
afterEach(() => setLlmCompressorBackend(null));
|
||||
|
||||
describe("llmCompressorEngine (T05/C3)", () => {
|
||||
it("is a pass-through with the default (no-op) backend even when enabled", async () => {
|
||||
const input = body([{ role: "user", content: "some prose that could be compressed" }]);
|
||||
const out = await llmCompressorEngine.applyAsync!(input, {
|
||||
stepConfig: { enabled: true, minTokens: 0 },
|
||||
});
|
||||
assert.equal(out.compressed, false);
|
||||
assert.equal(out.body, input, "default no-op backend must not mutate the body");
|
||||
});
|
||||
|
||||
it("does nothing when not enabled, even with a compressing backend", async () => {
|
||||
setLlmCompressorBackend(halveBackend);
|
||||
const input = body([{ role: "user", content: "lots of prose here to compress" }]);
|
||||
const off = await llmCompressorEngine.applyAsync!(input, { stepConfig: { minTokens: 0 } });
|
||||
assert.equal(off.compressed, false);
|
||||
assert.equal(off.body, input);
|
||||
});
|
||||
|
||||
it("compresses prose when enabled with a real backend, protecting code + system messages", async () => {
|
||||
setLlmCompressorBackend(halveBackend);
|
||||
const code = "```js\nconst secret = 42;\n```";
|
||||
const input = body([
|
||||
{ role: "system", content: "You are a careful assistant. Do not change me." },
|
||||
{
|
||||
role: "user",
|
||||
content: `Here is a long explanation in prose.\n\n${code}\n\nAnd more prose after.`,
|
||||
},
|
||||
]);
|
||||
const out = await llmCompressorEngine.applyAsync!(input, {
|
||||
stepConfig: { enabled: true, minTokens: 0 },
|
||||
});
|
||||
assert.equal(out.compressed, true);
|
||||
const msgs = (out.body as { messages: Array<{ role: string; content: string }> }).messages;
|
||||
// System message untouched.
|
||||
assert.equal(msgs[0].content, "You are a careful assistant. Do not change me.");
|
||||
// The fenced code block survives verbatim (never sent to the backend).
|
||||
assert.ok(msgs[1].content.includes(code), "code block must be preserved verbatim");
|
||||
// The user message got shorter overall.
|
||||
assert.ok(msgs[1].content.length < (input.messages as { content: string }[])[1].content.length);
|
||||
});
|
||||
|
||||
it("fail-opens: a throwing backend leaves the body unchanged", async () => {
|
||||
setLlmCompressorBackend(throwingBackend);
|
||||
const input = body([{ role: "user", content: "prose that the backend will choke on" }]);
|
||||
const out = await llmCompressorEngine.applyAsync!(input, {
|
||||
stepConfig: { enabled: true, minTokens: 0 },
|
||||
});
|
||||
assert.equal(out.compressed, false);
|
||||
assert.equal(
|
||||
(out.body as { messages: { content: string }[] }).messages[0].content,
|
||||
"prose that the backend will choke on"
|
||||
);
|
||||
});
|
||||
|
||||
it("respects the minTokens floor (skips small prompts)", async () => {
|
||||
setLlmCompressorBackend(halveBackend);
|
||||
const input = body([{ role: "user", content: "tiny" }]);
|
||||
const out = await llmCompressorEngine.applyAsync!(input, {
|
||||
stepConfig: { enabled: true, minTokens: 2000 },
|
||||
});
|
||||
assert.equal(out.compressed, false);
|
||||
});
|
||||
|
||||
it("sync apply is always a no-op pass-through", () => {
|
||||
const input = body([{ role: "user", content: "anything" }]);
|
||||
const out = llmCompressorEngine.apply(input);
|
||||
assert.equal(out.compressed, false);
|
||||
assert.equal(out.body, input);
|
||||
});
|
||||
|
||||
it("validateConfig accepts valid config and rejects bad fields", () => {
|
||||
assert.equal(llmCompressorEngine.validateConfig({ enabled: false }).valid, true);
|
||||
assert.equal(
|
||||
llmCompressorEngine.validateConfig({ enabled: true, compressionRate: 0.5, minTokens: 1000 }).valid,
|
||||
true
|
||||
);
|
||||
assert.equal(llmCompressorEngine.validateConfig({ enabled: "yes" }).valid, false);
|
||||
assert.equal(llmCompressorEngine.validateConfig({ compressionRate: 2 }).valid, false);
|
||||
assert.equal(llmCompressorEngine.validateConfig({ minTokens: -1 }).valid, false);
|
||||
});
|
||||
|
||||
it("defaults to disabled in its config schema (opt-in)", () => {
|
||||
const enabledField = llmCompressorEngine.getConfigSchema!().find((f) => f.key === "enabled");
|
||||
assert.equal(enabledField?.defaultValue, false);
|
||||
});
|
||||
});
|
||||
156
tests/unit/compression/pipeline-circuit-breaker.test.ts
Normal file
156
tests/unit/compression/pipeline-circuit-breaker.test.ts
Normal file
@@ -0,0 +1,156 @@
|
||||
import { describe, it, beforeEach, afterEach } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import {
|
||||
resolvePipelineBreakerConfig,
|
||||
canRunEngine,
|
||||
recordEngineFailure,
|
||||
recordEngineSuccess,
|
||||
getEngineBreakerState,
|
||||
resetPipelineEngineBreakers,
|
||||
DEFAULT_PIPELINE_BREAKER,
|
||||
type PipelineCircuitBreakerConfig,
|
||||
} from "../../../open-sse/services/compression/pipelineEngineBreaker.ts";
|
||||
import {
|
||||
registerCompressionEngine,
|
||||
unregisterCompressionEngine,
|
||||
} from "../../../open-sse/services/compression/engines/registry.ts";
|
||||
import { applyStackedCompression } from "../../../open-sse/services/compression/strategySelector.ts";
|
||||
|
||||
// T02 — pipeline engine circuit-breaker. Pure breaker semantics + an integration through the
|
||||
// stacked loop with a deliberately-throwing engine. Opt-in / default-off.
|
||||
|
||||
const ON: PipelineCircuitBreakerConfig = { enabled: true, failureThreshold: 2, cooldownMs: 1000 };
|
||||
|
||||
beforeEach(() => resetPipelineEngineBreakers());
|
||||
|
||||
describe("pipelineEngineBreaker — config resolution", () => {
|
||||
it("defaults to disabled with no partial and no env", () => {
|
||||
const cfg = resolvePipelineBreakerConfig(undefined, {} as NodeJS.ProcessEnv);
|
||||
assert.equal(cfg.enabled, false);
|
||||
assert.equal(cfg.failureThreshold, DEFAULT_PIPELINE_BREAKER.failureThreshold);
|
||||
assert.equal(cfg.cooldownMs, DEFAULT_PIPELINE_BREAKER.cooldownMs);
|
||||
});
|
||||
|
||||
it("reads env when no partial is given", () => {
|
||||
const cfg = resolvePipelineBreakerConfig(undefined, {
|
||||
COMPRESSION_PIPELINE_BREAKER_ENABLED: "true",
|
||||
COMPRESSION_PIPELINE_BREAKER_THRESHOLD: "5",
|
||||
COMPRESSION_PIPELINE_BREAKER_COOLDOWN_MS: "7000",
|
||||
} as NodeJS.ProcessEnv);
|
||||
assert.equal(cfg.enabled, true);
|
||||
assert.equal(cfg.failureThreshold, 5);
|
||||
assert.equal(cfg.cooldownMs, 7000);
|
||||
});
|
||||
|
||||
it("partial wins over env", () => {
|
||||
const cfg = resolvePipelineBreakerConfig({ enabled: false }, {
|
||||
COMPRESSION_PIPELINE_BREAKER_ENABLED: "true",
|
||||
} as NodeJS.ProcessEnv);
|
||||
assert.equal(cfg.enabled, false);
|
||||
});
|
||||
});
|
||||
|
||||
describe("pipelineEngineBreaker — state machine", () => {
|
||||
it("is a pass-through when disabled (never opens, never records)", () => {
|
||||
const off = resolvePipelineBreakerConfig({ enabled: false });
|
||||
recordEngineFailure("rtk", off);
|
||||
recordEngineFailure("rtk", off);
|
||||
recordEngineFailure("rtk", off);
|
||||
assert.equal(canRunEngine("rtk", off), true);
|
||||
assert.equal(getEngineBreakerState("rtk").open, false);
|
||||
});
|
||||
|
||||
it("opens after the failure threshold and short-circuits within the cooldown", () => {
|
||||
assert.equal(canRunEngine("rtk", ON, 0), true);
|
||||
recordEngineFailure("rtk", ON, 0); // 1 < 2 → still closed
|
||||
assert.equal(canRunEngine("rtk", ON, 0), true);
|
||||
recordEngineFailure("rtk", ON, 0); // 2 >= 2 → OPEN until 1000
|
||||
assert.equal(getEngineBreakerState("rtk").open, true);
|
||||
assert.equal(canRunEngine("rtk", ON, 500), false, "OPEN within cooldown");
|
||||
});
|
||||
|
||||
it("half-opens after the cooldown; a failed probe re-opens immediately", () => {
|
||||
recordEngineFailure("rtk", ON, 0);
|
||||
recordEngineFailure("rtk", ON, 0); // OPEN until 1000
|
||||
assert.equal(canRunEngine("rtk", ON, 1500), true, "half-open probe allowed past cooldown");
|
||||
// a single probe failure re-opens (failures was left at threshold-1)
|
||||
recordEngineFailure("rtk", ON, 1500);
|
||||
assert.equal(canRunEngine("rtk", ON, 1600), false, "re-opened after failed probe");
|
||||
});
|
||||
|
||||
it("a successful probe fully closes the breaker", () => {
|
||||
recordEngineFailure("rtk", ON, 0);
|
||||
recordEngineFailure("rtk", ON, 0);
|
||||
assert.equal(canRunEngine("rtk", ON, 1500), true); // half-open
|
||||
recordEngineSuccess("rtk", ON);
|
||||
assert.equal(getEngineBreakerState("rtk").open, false);
|
||||
assert.equal(getEngineBreakerState("rtk").failures, 0);
|
||||
assert.equal(canRunEngine("rtk", ON, 1600), true);
|
||||
});
|
||||
});
|
||||
|
||||
describe("pipelineEngineBreaker — pipeline integration", () => {
|
||||
const ENGINE_ID = "test-cb-throw";
|
||||
let calls = 0;
|
||||
|
||||
beforeEach(() => {
|
||||
calls = 0;
|
||||
registerCompressionEngine({
|
||||
id: ENGINE_ID,
|
||||
name: "throwing test engine",
|
||||
targets: ["messages"],
|
||||
stackable: true,
|
||||
apply() {
|
||||
calls += 1;
|
||||
throw new Error("boom");
|
||||
},
|
||||
compress() {
|
||||
throw new Error("boom");
|
||||
},
|
||||
getConfigSchema() {
|
||||
return [];
|
||||
},
|
||||
validateConfig() {
|
||||
return { valid: true, errors: [] };
|
||||
},
|
||||
});
|
||||
});
|
||||
|
||||
afterEach(() => unregisterCompressionEngine(ENGINE_ID));
|
||||
|
||||
function run() {
|
||||
// Object step: a bare-string step that is not a known alias normalizes to caveman, so the
|
||||
// pipeline must reference the test engine via an explicit `{ engine }` object.
|
||||
return applyStackedCompression(
|
||||
{ messages: [{ role: "user", content: "hello world" }] },
|
||||
[{ engine: ENGINE_ID, intensity: "standard" }],
|
||||
{ circuitBreaker: { enabled: true, failureThreshold: 2, cooldownMs: 60_000 } }
|
||||
);
|
||||
}
|
||||
|
||||
it("a throwing engine fails open (no throw) and opens the breaker after the threshold", () => {
|
||||
// First two runs: engine throws, caught + recorded (fail-open → body unchanged).
|
||||
const r1 = run();
|
||||
assert.equal(r1.compressed, false);
|
||||
assert.deepEqual(r1.body, { messages: [{ role: "user", content: "hello world" }] });
|
||||
run();
|
||||
// After 2 failures the breaker is OPEN.
|
||||
assert.equal(getEngineBreakerState(ENGINE_ID).open, true);
|
||||
|
||||
const callsBefore = calls;
|
||||
const r3 = run();
|
||||
// Third run: breaker OPEN → engine is skipped entirely (never invoked again).
|
||||
assert.equal(calls, callsBefore, "engine must not be invoked while the breaker is open");
|
||||
assert.equal(r3.compressed, false);
|
||||
});
|
||||
|
||||
it("with the breaker disabled (default), a throwing engine propagates (legacy behavior)", () => {
|
||||
assert.throws(() =>
|
||||
applyStackedCompression(
|
||||
{ messages: [{ role: "user", content: "x" }] },
|
||||
[{ engine: ENGINE_ID, intensity: "standard" }],
|
||||
{}
|
||||
)
|
||||
);
|
||||
});
|
||||
});
|
||||
121
tests/unit/compression/prefix-freeze.test.ts
Normal file
121
tests/unit/compression/prefix-freeze.test.ts
Normal file
@@ -0,0 +1,121 @@
|
||||
import { describe, it, beforeEach, afterEach } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import {
|
||||
resolvePrefixFreezeConfig,
|
||||
extractStablePrefixHash,
|
||||
observePrefix,
|
||||
isPrefixFrozen,
|
||||
getPrefixObservations,
|
||||
resetPrefixFreeze,
|
||||
DEFAULT_PREFIX_FREEZE,
|
||||
} from "../../../open-sse/services/compression/prefixFreeze.ts";
|
||||
import { resolveCacheAwareConfig } from "../../../open-sse/services/compression/cacheAwareConfig.ts";
|
||||
import type { CompressionConfig } from "../../../open-sse/services/compression/types.ts";
|
||||
|
||||
// T08/H5 — usage-observed prefix freeze. Pure observer + integration via resolveCacheAwareConfig.
|
||||
|
||||
beforeEach(() => resetPrefixFreeze());
|
||||
|
||||
describe("prefixFreeze — config", () => {
|
||||
it("defaults to disabled / threshold 3", () => {
|
||||
const cfg = resolvePrefixFreezeConfig({} as NodeJS.ProcessEnv);
|
||||
assert.equal(cfg.enabled, false);
|
||||
assert.equal(cfg.threshold, DEFAULT_PREFIX_FREEZE.threshold);
|
||||
});
|
||||
it("reads env", () => {
|
||||
const cfg = resolvePrefixFreezeConfig({
|
||||
COMPRESSION_PREFIX_FREEZE_ENABLED: "true",
|
||||
COMPRESSION_PREFIX_FREEZE_THRESHOLD: "5",
|
||||
} as NodeJS.ProcessEnv);
|
||||
assert.equal(cfg.enabled, true);
|
||||
assert.equal(cfg.threshold, 5);
|
||||
});
|
||||
});
|
||||
|
||||
describe("prefixFreeze — extractStablePrefixHash", () => {
|
||||
it("hashes an OpenAI system message", () => {
|
||||
const h = extractStablePrefixHash({
|
||||
messages: [
|
||||
{ role: "system", content: "You are helpful" },
|
||||
{ role: "user", content: "hi" },
|
||||
],
|
||||
});
|
||||
assert.equal(typeof h, "string");
|
||||
assert.equal(h!.length, 24);
|
||||
});
|
||||
it("hashes a Claude `system` field and a Gemini `systemInstruction`", () => {
|
||||
const claude = extractStablePrefixHash({ system: "SP", messages: [] });
|
||||
const gemini = extractStablePrefixHash({ systemInstruction: { parts: [{ text: "SP" }] } });
|
||||
assert.ok(claude && gemini);
|
||||
// same underlying text → same hash regardless of shape
|
||||
assert.equal(claude, gemini);
|
||||
});
|
||||
it("returns null when there is no system prompt", () => {
|
||||
assert.equal(extractStablePrefixHash({ messages: [{ role: "user", content: "hi" }] }), null);
|
||||
assert.equal(extractStablePrefixHash("not an object"), null);
|
||||
});
|
||||
it("different system prompts hash differently", () => {
|
||||
const a = extractStablePrefixHash({ system: "A" });
|
||||
const b = extractStablePrefixHash({ system: "B" });
|
||||
assert.notEqual(a, b);
|
||||
});
|
||||
});
|
||||
|
||||
describe("prefixFreeze — observe/isFrozen", () => {
|
||||
it("freezes only at/above the threshold, isolated per hash", () => {
|
||||
observePrefix("h");
|
||||
observePrefix("h");
|
||||
assert.equal(isPrefixFrozen("h", 3), false);
|
||||
assert.equal(getPrefixObservations("h"), 2);
|
||||
observePrefix("h");
|
||||
assert.equal(isPrefixFrozen("h", 3), true);
|
||||
// a different prefix is unaffected
|
||||
assert.equal(isPrefixFrozen("other", 3), false);
|
||||
});
|
||||
});
|
||||
|
||||
describe("resolveCacheAwareConfig — H5 integration", () => {
|
||||
const baseConfig = {
|
||||
defaultMode: "standard",
|
||||
preserveSystemPromptMode: "whenNoCache",
|
||||
preserveSystemPrompt: false,
|
||||
} as unknown as CompressionConfig;
|
||||
const body = {
|
||||
messages: [
|
||||
{ role: "system", content: "You are a careful assistant." },
|
||||
{ role: "user", content: "hi" },
|
||||
],
|
||||
};
|
||||
const ctx = { provider: "test-noncaching-provider" };
|
||||
|
||||
afterEach(() => {
|
||||
delete process.env.COMPRESSION_PREFIX_FREEZE_ENABLED;
|
||||
delete process.env.COMPRESSION_PREFIX_FREEZE_THRESHOLD;
|
||||
});
|
||||
|
||||
it("does nothing when disabled (default): non-caching provider never freezes the prefix", () => {
|
||||
for (let i = 0; i < 5; i++) {
|
||||
const out = resolveCacheAwareConfig(baseConfig, body, ctx);
|
||||
assert.equal(out.preserveSystemPrompt, false);
|
||||
}
|
||||
});
|
||||
|
||||
it("when enabled, an observed-stable prefix flips preserveSystemPrompt at the threshold", () => {
|
||||
process.env.COMPRESSION_PREFIX_FREEZE_ENABLED = "true";
|
||||
process.env.COMPRESSION_PREFIX_FREEZE_THRESHOLD = "3";
|
||||
assert.equal(resolveCacheAwareConfig(baseConfig, body, ctx).preserveSystemPrompt, false); // 1
|
||||
assert.equal(resolveCacheAwareConfig(baseConfig, body, ctx).preserveSystemPrompt, false); // 2
|
||||
assert.equal(resolveCacheAwareConfig(baseConfig, body, ctx).preserveSystemPrompt, true); // 3 → frozen
|
||||
});
|
||||
|
||||
it("respects mode `never`: a frozen prefix is still not preserved", () => {
|
||||
process.env.COMPRESSION_PREFIX_FREEZE_ENABLED = "true";
|
||||
process.env.COMPRESSION_PREFIX_FREEZE_THRESHOLD = "1";
|
||||
const neverConfig = {
|
||||
...baseConfig,
|
||||
preserveSystemPromptMode: "never",
|
||||
} as unknown as CompressionConfig;
|
||||
const out = resolveCacheAwareConfig(neverConfig, body, ctx);
|
||||
assert.equal(out.preserveSystemPrompt, false);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,93 @@
|
||||
import test from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import fs from "node:fs";
|
||||
import os from "node:os";
|
||||
import path from "node:path";
|
||||
|
||||
// T05/C5 back-compat regression (#5653) at the DB read path. The new DEFAULT_COMPRESSION_CONFIG
|
||||
// seeds preserveSystemPromptMode='always'. getCompressionSettings() spreads that default, so a
|
||||
// legacy install that persisted ONLY the `preserveSystemPrompt` boolean (no mode row) would have
|
||||
// the default 'always' shadow the boolean — silently flipping `preserveSystemPrompt=false` installs
|
||||
// from "compress unless cached" (whenNoCache) to "always preserve". The read path must instead
|
||||
// derive the mode from the boolean when no mode row exists.
|
||||
//
|
||||
// core.ts freezes DATA_DIR/SQLITE_FILE into module consts at first import, so the temp DATA_DIR is
|
||||
// set BEFORE any import and the db file is shared across tests. Each test isolates by wiping the
|
||||
// `compression` namespace and busting the module-level TTL cache via resetDbInstance() (which hands
|
||||
// out a NEW db object, so the cache — keyed by db ref — misses).
|
||||
const TEMP_DIR = fs.mkdtempSync(path.join(os.tmpdir(), "omniroute-preserve-mode-"));
|
||||
process.env.DATA_DIR = TEMP_DIR;
|
||||
|
||||
async function freshCompressionDb() {
|
||||
const { getDbInstance, resetDbInstance } = await import("../../../src/lib/db/core.ts");
|
||||
const db = getDbInstance(); // runs migrations on first call
|
||||
db.prepare("DELETE FROM key_value WHERE namespace = 'compression'").run();
|
||||
return { db, resetDbInstance };
|
||||
}
|
||||
|
||||
async function readSettings() {
|
||||
const { getCompressionSettings } = await import("../../../src/lib/db/compression.ts");
|
||||
return getCompressionSettings();
|
||||
}
|
||||
|
||||
test.after(async () => {
|
||||
try {
|
||||
const { resetDbInstance } = await import("../../../src/lib/db/core.ts");
|
||||
resetDbInstance();
|
||||
} catch {
|
||||
/* core never loaded */
|
||||
}
|
||||
try {
|
||||
fs.rmSync(TEMP_DIR, { recursive: true, force: true });
|
||||
} catch {
|
||||
/* best-effort */
|
||||
}
|
||||
});
|
||||
|
||||
test("legacy preserveSystemPrompt=false (no mode row) derives whenNoCache", async () => {
|
||||
const { db, resetDbInstance } = await freshCompressionDb();
|
||||
db.prepare(
|
||||
"INSERT OR REPLACE INTO key_value (namespace, key, value) VALUES ('compression','preserveSystemPrompt','false')"
|
||||
).run();
|
||||
resetDbInstance(); // new db object => TTL cache miss on the next read
|
||||
|
||||
const cfg = await readSettings();
|
||||
assert.equal(
|
||||
cfg.preserveSystemPromptMode,
|
||||
"whenNoCache",
|
||||
"legacy preserveSystemPrompt=false must derive whenNoCache, not inherit the 'always' default"
|
||||
);
|
||||
|
||||
// End-to-end: without a cacheable prefix, a legacy-off install must still compress the prompt.
|
||||
const { resolveCacheAwareConfig } = await import(
|
||||
"../../../open-sse/services/compression/cacheAwareConfig.ts"
|
||||
);
|
||||
assert.equal(
|
||||
resolveCacheAwareConfig(cfg).preserveSystemPrompt,
|
||||
false,
|
||||
"legacy-off install must compress the system prompt when there is no cache"
|
||||
);
|
||||
});
|
||||
|
||||
test("fresh install (no override rows) defaults to always", async () => {
|
||||
const { resetDbInstance } = await freshCompressionDb(); // wipes all compression rows
|
||||
resetDbInstance();
|
||||
|
||||
const cfg = await readSettings();
|
||||
assert.equal(cfg.preserveSystemPromptMode, "always", "fresh default mode is always");
|
||||
assert.equal(cfg.preserveSystemPrompt, true, "fresh default boolean is true");
|
||||
});
|
||||
|
||||
test("an explicit mode row wins over the legacy boolean", async () => {
|
||||
const { db, resetDbInstance } = await freshCompressionDb();
|
||||
db.prepare(
|
||||
"INSERT OR REPLACE INTO key_value (namespace, key, value) VALUES ('compression','preserveSystemPrompt','false')"
|
||||
).run();
|
||||
db.prepare(
|
||||
"INSERT OR REPLACE INTO key_value (namespace, key, value) VALUES ('compression','preserveSystemPromptMode','\"never\"')"
|
||||
).run();
|
||||
resetDbInstance();
|
||||
|
||||
const cfg = await readSettings();
|
||||
assert.equal(cfg.preserveSystemPromptMode, "never", "an explicit stored mode row wins");
|
||||
});
|
||||
78
tests/unit/compression/preserve-system-prompt-mode.test.ts
Normal file
78
tests/unit/compression/preserve-system-prompt-mode.test.ts
Normal file
@@ -0,0 +1,78 @@
|
||||
import test from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import {
|
||||
PRESERVE_SYSTEM_PROMPT_MODES,
|
||||
isPreserveSystemPromptMode,
|
||||
normalizePreserveSystemPromptMode,
|
||||
resolvePreserveSystemPrompt,
|
||||
modeToBaselineBoolean,
|
||||
} from "../../../open-sse/services/compression/preserveSystemPromptMode.ts";
|
||||
|
||||
// T05/C5 — preserveSystemPrompt boolean -> enum (always | whenNoCache | never) + back-compat shim.
|
||||
|
||||
test("isPreserveSystemPromptMode only accepts the three known tokens", () => {
|
||||
for (const m of PRESERVE_SYSTEM_PROMPT_MODES) assert.equal(isPreserveSystemPromptMode(m), true);
|
||||
for (const bad of ["", "Always", "no-cache", true, false, 1, null, undefined, {}]) {
|
||||
assert.equal(isPreserveSystemPromptMode(bad), false, `${String(bad)} must be rejected`);
|
||||
}
|
||||
});
|
||||
|
||||
test("normalize: an explicit mode wins over the legacy boolean", () => {
|
||||
assert.equal(
|
||||
normalizePreserveSystemPromptMode({ preserveSystemPrompt: true, preserveSystemPromptMode: "never" }),
|
||||
"never"
|
||||
);
|
||||
assert.equal(
|
||||
normalizePreserveSystemPromptMode({ preserveSystemPrompt: false, preserveSystemPromptMode: "always" }),
|
||||
"always"
|
||||
);
|
||||
});
|
||||
|
||||
test("normalize: the shim maps the legacy boolean 1:1 (true->always, false->whenNoCache)", () => {
|
||||
// The shim is a behaviour identity: legacy `false` already meant "compress unless
|
||||
// there is a cache" via the #3890/#3955 guard, i.e. exactly `whenNoCache`.
|
||||
assert.equal(normalizePreserveSystemPromptMode({ preserveSystemPrompt: true }), "always");
|
||||
assert.equal(normalizePreserveSystemPromptMode({ preserveSystemPrompt: false }), "whenNoCache");
|
||||
});
|
||||
|
||||
test("normalize: an invalid mode string falls back to the legacy boolean", () => {
|
||||
assert.equal(
|
||||
normalizePreserveSystemPromptMode({
|
||||
preserveSystemPrompt: false,
|
||||
preserveSystemPromptMode: "garbage" as unknown as "always",
|
||||
}),
|
||||
"whenNoCache"
|
||||
);
|
||||
});
|
||||
|
||||
test("resolve: always preserves and never compresses regardless of cache", () => {
|
||||
assert.equal(resolvePreserveSystemPrompt("always", { hasCache: true }), true);
|
||||
assert.equal(resolvePreserveSystemPrompt("always", { hasCache: false }), true);
|
||||
assert.equal(resolvePreserveSystemPrompt("never", { hasCache: true }), false);
|
||||
assert.equal(resolvePreserveSystemPrompt("never", { hasCache: false }), false);
|
||||
});
|
||||
|
||||
test("resolve: whenNoCache preserves only when a cache is present", () => {
|
||||
assert.equal(resolvePreserveSystemPrompt("whenNoCache", { hasCache: true }), true);
|
||||
assert.equal(resolvePreserveSystemPrompt("whenNoCache", { hasCache: false }), false);
|
||||
});
|
||||
|
||||
test("modeToBaselineBoolean is the no-cache projection", () => {
|
||||
assert.equal(modeToBaselineBoolean("always"), true);
|
||||
assert.equal(modeToBaselineBoolean("whenNoCache"), false);
|
||||
assert.equal(modeToBaselineBoolean("never"), false);
|
||||
});
|
||||
|
||||
test("back-compat identity: legacy config resolves to the old cache-guard outcome", () => {
|
||||
// Old behaviour: preserveSystemPrompt=false stays false without cache, forced true with cache.
|
||||
const legacyFalse = { preserveSystemPrompt: false };
|
||||
const mode = normalizePreserveSystemPromptMode(legacyFalse);
|
||||
assert.equal(resolvePreserveSystemPrompt(mode, { hasCache: false }), false);
|
||||
assert.equal(resolvePreserveSystemPrompt(mode, { hasCache: true }), true);
|
||||
|
||||
// Old behaviour: preserveSystemPrompt=true is always preserved.
|
||||
const legacyTrue = { preserveSystemPrompt: true };
|
||||
const modeT = normalizePreserveSystemPromptMode(legacyTrue);
|
||||
assert.equal(resolvePreserveSystemPrompt(modeT, { hasCache: false }), true);
|
||||
assert.equal(resolvePreserveSystemPrompt(modeT, { hasCache: true }), true);
|
||||
});
|
||||
163
tests/unit/compression/read-lifecycle.test.ts
Normal file
163
tests/unit/compression/read-lifecycle.test.ts
Normal file
@@ -0,0 +1,163 @@
|
||||
import { describe, it } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import {
|
||||
readLifecycleEngine,
|
||||
extractInvocations,
|
||||
findSupersededReadCallIds,
|
||||
} from "../../../open-sse/services/compression/engines/readLifecycle/index.ts";
|
||||
|
||||
// T08/H7 — read-lifecycle: collapse superseded file-Read tool results (opt-in, lossy).
|
||||
|
||||
const STUB_MARK = "read superseded";
|
||||
|
||||
function run(messages: unknown[], enabled = true) {
|
||||
return readLifecycleEngine.apply({ messages }, { stepConfig: { enabled } });
|
||||
}
|
||||
|
||||
// ── Anthropic fixtures ──
|
||||
const anthropicRead = (id: string, path: string) => ({
|
||||
role: "assistant",
|
||||
content: [{ type: "tool_use", id, name: "Read", input: { file_path: path } }],
|
||||
});
|
||||
const anthropicWrite = (id: string, path: string) => ({
|
||||
role: "assistant",
|
||||
content: [{ type: "tool_use", id, name: "Edit", input: { file_path: path } }],
|
||||
});
|
||||
const anthropicResult = (id: string, text: string) => ({
|
||||
role: "user",
|
||||
content: [{ type: "tool_result", tool_use_id: id, content: text }],
|
||||
});
|
||||
function anthropicResultText(msg: unknown, id: string): string | undefined {
|
||||
const blocks = (msg as { content?: unknown[] }).content;
|
||||
const b = Array.isArray(blocks)
|
||||
? (blocks.find((x) => (x as { tool_use_id?: string }).tool_use_id === id) as
|
||||
{ content?: string } | undefined)
|
||||
: undefined;
|
||||
return b?.content;
|
||||
}
|
||||
|
||||
// ── OpenAI fixtures ──
|
||||
const openaiRead = (id: string, path: string) => ({
|
||||
role: "assistant",
|
||||
tool_calls: [
|
||||
{
|
||||
id,
|
||||
type: "function",
|
||||
function: { name: "read_file", arguments: JSON.stringify({ file_path: path }) },
|
||||
},
|
||||
],
|
||||
});
|
||||
const openaiResult = (id: string, text: string) => ({
|
||||
role: "tool",
|
||||
tool_call_id: id,
|
||||
content: text,
|
||||
});
|
||||
|
||||
describe("read-lifecycle — pure analysis", () => {
|
||||
it("extracts invocations from both shapes", () => {
|
||||
const { invocations, readPathByCallId } = extractInvocations([
|
||||
anthropicRead("c1", "/a.ts"),
|
||||
openaiRead("c2", "/b.ts"),
|
||||
]);
|
||||
assert.equal(invocations.length, 2);
|
||||
assert.equal(invocations[0].kind, "read");
|
||||
assert.equal(invocations[0].path, "/a.ts");
|
||||
assert.equal(readPathByCallId.get("c1"), "/a.ts");
|
||||
assert.equal(readPathByCallId.get("c2"), "/b.ts");
|
||||
});
|
||||
|
||||
it("marks a read superseded by a later read of the same path", () => {
|
||||
const { invocations } = extractInvocations([
|
||||
anthropicRead("c1", "/a.ts"),
|
||||
anthropicRead("c2", "/a.ts"),
|
||||
]);
|
||||
const s = findSupersededReadCallIds(invocations);
|
||||
assert.equal(s.has("c1"), true);
|
||||
assert.equal(s.has("c2"), false);
|
||||
});
|
||||
|
||||
it("marks a read superseded by a later write of the same path", () => {
|
||||
const { invocations } = extractInvocations([
|
||||
anthropicRead("c1", "/a.ts"),
|
||||
anthropicWrite("w1", "/a.ts"),
|
||||
]);
|
||||
const s = findSupersededReadCallIds(invocations);
|
||||
assert.equal(s.has("c1"), true);
|
||||
assert.equal(s.has("w1"), false); // writes are never collapsed
|
||||
});
|
||||
|
||||
it("does not supersede reads of different paths", () => {
|
||||
const { invocations } = extractInvocations([
|
||||
anthropicRead("c1", "/a.ts"),
|
||||
anthropicRead("c2", "/b.ts"),
|
||||
]);
|
||||
const s = findSupersededReadCallIds(invocations);
|
||||
assert.equal(s.size, 0);
|
||||
});
|
||||
});
|
||||
|
||||
describe("read-lifecycle — engine integration", () => {
|
||||
it("collapses an earlier Anthropic read, keeping the latest intact", () => {
|
||||
const out = run([
|
||||
anthropicRead("c1", "/a.ts"),
|
||||
anthropicResult("c1", "OLD content of a.ts"),
|
||||
anthropicRead("c2", "/a.ts"),
|
||||
anthropicResult("c2", "NEW content of a.ts"),
|
||||
]);
|
||||
assert.equal(out.compressed, true);
|
||||
const msgs = (out.body as { messages: unknown[] }).messages;
|
||||
assert.match(anthropicResultText(msgs[1], "c1") ?? "", new RegExp(STUB_MARK));
|
||||
assert.equal(anthropicResultText(msgs[3], "c2"), "NEW content of a.ts");
|
||||
});
|
||||
|
||||
it("collapses a read superseded by a later write", () => {
|
||||
const out = run([
|
||||
anthropicRead("c1", "/a.ts"),
|
||||
anthropicResult("c1", "OLD"),
|
||||
anthropicWrite("w1", "/a.ts"),
|
||||
anthropicResult("w1", "edit applied"),
|
||||
]);
|
||||
assert.equal(out.compressed, true);
|
||||
const msgs = (out.body as { messages: unknown[] }).messages;
|
||||
assert.match(anthropicResultText(msgs[1], "c1") ?? "", new RegExp(STUB_MARK));
|
||||
assert.equal(anthropicResultText(msgs[3], "w1"), "edit applied"); // write result untouched
|
||||
});
|
||||
|
||||
it("collapses an earlier OpenAI read (role:tool message)", () => {
|
||||
const out = run([
|
||||
openaiRead("c1", "/a.ts"),
|
||||
openaiResult("c1", "OLD"),
|
||||
openaiRead("c2", "/a.ts"),
|
||||
openaiResult("c2", "NEW"),
|
||||
]);
|
||||
assert.equal(out.compressed, true);
|
||||
const msgs = (out.body as { messages: Array<{ content?: string }> }).messages;
|
||||
assert.match(msgs[1].content ?? "", new RegExp(STUB_MARK));
|
||||
assert.equal(msgs[3].content, "NEW");
|
||||
});
|
||||
|
||||
it("no-ops for a single read of a path", () => {
|
||||
const out = run([anthropicRead("c1", "/a.ts"), anthropicResult("c1", "only read")]);
|
||||
assert.equal(out.compressed, false);
|
||||
});
|
||||
|
||||
it("is a pass-through when disabled (default)", () => {
|
||||
const messages = [
|
||||
anthropicRead("c1", "/a.ts"),
|
||||
anthropicResult("c1", "OLD"),
|
||||
anthropicRead("c2", "/a.ts"),
|
||||
anthropicResult("c2", "NEW"),
|
||||
];
|
||||
const out = run(messages, false);
|
||||
assert.equal(out.compressed, false);
|
||||
assert.equal(
|
||||
anthropicResultText((out.body as { messages: unknown[] }).messages[1], "c1"),
|
||||
"OLD"
|
||||
);
|
||||
});
|
||||
|
||||
it("schema defaults enabled to false (opt-in)", () => {
|
||||
const field = readLifecycleEngine.getConfigSchema!().find((f) => f.key === "enabled");
|
||||
assert.equal(field?.defaultValue, false);
|
||||
});
|
||||
});
|
||||
88
tests/unit/compression/rtk-mcp-tools.test.ts
Normal file
88
tests/unit/compression/rtk-mcp-tools.test.ts
Normal file
@@ -0,0 +1,88 @@
|
||||
import { describe, it, beforeEach, after } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import fs from "node:fs";
|
||||
import os from "node:os";
|
||||
import path from "node:path";
|
||||
|
||||
// T07 — omniroute_rtk_discover / omniroute_rtk_learn MCP tools (read-only; audited).
|
||||
|
||||
const TEST_DATA_DIR = fs.mkdtempSync(path.join(os.tmpdir(), "omni-rtk-mcp-"));
|
||||
process.env.DATA_DIR = TEST_DATA_DIR;
|
||||
|
||||
const core = await import("../../../src/lib/db/core.ts");
|
||||
const { handleRtkDiscover, handleRtkLearn } = await import(
|
||||
"../../../open-sse/mcp-server/tools/compressionTools.ts"
|
||||
);
|
||||
const { maybePersistRtkRawOutput } = await import(
|
||||
"../../../open-sse/services/compression/engines/rtk/rawOutput.ts"
|
||||
);
|
||||
const { getRecentAuditEntries } = await import("../../../open-sse/mcp-server/audit.ts");
|
||||
|
||||
const NOISE = [
|
||||
"Welcome to Gradle 8.5!",
|
||||
"> Task :app:compileJava UP-TO-DATE",
|
||||
"Downloading https://repo1.maven.org/foo.jar",
|
||||
"BUILD SUCCESSFUL in 12s",
|
||||
].join("\n");
|
||||
|
||||
function seedSamples() {
|
||||
// retention "always" forces a capture even on a successful (non-failure) output.
|
||||
// The two captures MUST differ in content: the raw-output filename is keyed on
|
||||
// Date.now() (ms) + a hash of the content (see rawOutput.ts), so two BYTE-IDENTICAL
|
||||
// captures that land in the same millisecond collapse to the same filename — the 2nd
|
||||
// write overwrites the 1st, leaving only 1 sample. That made this test ~25% flaky on
|
||||
// fast CI runners ("expected 2, got 1"). Distinct content = 2 files regardless of
|
||||
// timing; two real build runs never emit byte-identical output anyway.
|
||||
maybePersistRtkRawOutput(NOISE, { retention: "always", command: "gradle build" });
|
||||
maybePersistRtkRawOutput(`${NOISE}\n> Task :app:test UP-TO-DATE`, {
|
||||
retention: "always",
|
||||
command: "gradle build",
|
||||
});
|
||||
}
|
||||
|
||||
beforeEach(() => {
|
||||
core.resetDbInstance();
|
||||
fs.rmSync(TEST_DATA_DIR, { recursive: true, force: true });
|
||||
fs.mkdirSync(TEST_DATA_DIR, { recursive: true });
|
||||
core.getDbInstance(); // run migrations → mcp_tool_audit table exists
|
||||
});
|
||||
|
||||
after(() => {
|
||||
core.resetDbInstance();
|
||||
fs.rmSync(TEST_DATA_DIR, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
describe("RTK MCP tools (T07)", () => {
|
||||
it("omniroute_rtk_discover returns ranked noise candidates from the sample store", async () => {
|
||||
seedSamples();
|
||||
const result = await handleRtkDiscover({ limit: 100 });
|
||||
assert.equal(typeof result.sampleCount, "number");
|
||||
assert.ok(result.sampleCount >= 2, `expected the 2 seeded samples, got ${result.sampleCount}`);
|
||||
assert.ok(Array.isArray(result.candidates));
|
||||
});
|
||||
|
||||
it("omniroute_rtk_learn returns a filter draft for a specific command", async () => {
|
||||
seedSamples();
|
||||
const result = await handleRtkLearn({ command: "gradle build", limit: 100 });
|
||||
assert.equal(result.command, "gradle build");
|
||||
assert.ok(result.sampleCount >= 1, "expected at least one matching sample");
|
||||
assert.ok(result.filter && typeof result.filter === "object", "expected a suggested filter draft");
|
||||
});
|
||||
|
||||
it("returns an empty/baseline result with no samples (no throw)", async () => {
|
||||
const discover = await handleRtkDiscover({});
|
||||
assert.equal(discover.sampleCount, 0);
|
||||
assert.deepEqual(discover.candidates, []);
|
||||
const learn = await handleRtkLearn({ command: "npm test" });
|
||||
assert.equal(learn.sampleCount, 0);
|
||||
});
|
||||
|
||||
it("both tools are audited in mcp_tool_audit", async () => {
|
||||
await handleRtkDiscover({});
|
||||
await handleRtkLearn({ command: "gradle build" });
|
||||
const entries = await getRecentAuditEntries(20);
|
||||
const names = entries.map((e) => e.toolName);
|
||||
assert.ok(names.includes("omniroute_rtk_discover"), "discover must be audited");
|
||||
assert.ok(names.includes("omniroute_rtk_learn"), "learn must be audited");
|
||||
});
|
||||
});
|
||||
126
tests/unit/context-editing-streaming-telemetry.test.ts
Normal file
126
tests/unit/context-editing-streaming-telemetry.test.ts
Normal file
@@ -0,0 +1,126 @@
|
||||
/**
|
||||
* T01 (Onda 5, item 5.1) — Context Editing `applied_edits` telemetry on the STREAMING path.
|
||||
*
|
||||
* Anthropic surfaces `context_management.applied_edits[]` on the final `message_delta`
|
||||
* snapshot of an SSE stream (spec `2026-06-16-anthropic-context-editing-spec.md` §3). Before
|
||||
* this change the streaming reconstruction (`buildStreamSummaryFromEvents` → Claude branch)
|
||||
* dropped `context_management` entirely, so streaming context-clear savings never reached
|
||||
* `recordContextEditingTelemetry` — only the non-streaming JSON path was covered.
|
||||
*
|
||||
* These tests pin the two pure units the fix touches:
|
||||
* 1. the collector must preserve `context_management` from the message_delta snapshot;
|
||||
* 2. the existing `extractContextEditingTelemetry` must then read it end-to-end.
|
||||
*/
|
||||
import test from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
|
||||
const collector = await import("../../open-sse/utils/streamPayloadCollector.ts");
|
||||
const { extractContextEditingTelemetry } = await import("../../open-sse/config/contextEditing.ts");
|
||||
|
||||
type SSEEvent = { event?: string; data: unknown };
|
||||
type AppliedEdit = { cleared_input_tokens?: number; cleared_tool_uses?: number };
|
||||
type ClaudeSummary = {
|
||||
type?: string;
|
||||
stop_reason?: string;
|
||||
content?: unknown[];
|
||||
context_management?: { applied_edits?: AppliedEdit[] };
|
||||
};
|
||||
|
||||
function claudeStreamWithAppliedEdits(): SSEEvent[] {
|
||||
return [
|
||||
{
|
||||
data: {
|
||||
type: "message_start",
|
||||
message: {
|
||||
id: "msg_ce_stream_1",
|
||||
model: "claude-sonnet-4-6",
|
||||
role: "assistant",
|
||||
usage: { input_tokens: 120, output_tokens: 0 },
|
||||
},
|
||||
},
|
||||
},
|
||||
{ data: { type: "content_block_start", index: 0, content_block: { type: "text", text: "" } } },
|
||||
{
|
||||
data: { type: "content_block_delta", index: 0, delta: { type: "text_delta", text: "Done." } },
|
||||
},
|
||||
{
|
||||
data: {
|
||||
type: "message_delta",
|
||||
delta: { stop_reason: "end_turn" },
|
||||
context_management: {
|
||||
applied_edits: [
|
||||
{
|
||||
type: "clear_tool_uses_20250919",
|
||||
cleared_input_tokens: 30000,
|
||||
cleared_tool_uses: 5,
|
||||
},
|
||||
],
|
||||
},
|
||||
usage: { output_tokens: 42 },
|
||||
},
|
||||
},
|
||||
{ data: { type: "message_stop" } },
|
||||
];
|
||||
}
|
||||
|
||||
function claudeStreamWithoutContextManagement(): SSEEvent[] {
|
||||
return [
|
||||
{
|
||||
data: {
|
||||
type: "message_start",
|
||||
message: { id: "msg_plain", model: "claude-sonnet-4-6", role: "assistant", usage: {} },
|
||||
},
|
||||
},
|
||||
{ data: { type: "content_block_start", index: 0, content_block: { type: "text", text: "" } } },
|
||||
{ data: { type: "content_block_delta", index: 0, delta: { type: "text_delta", text: "Hi" } } },
|
||||
{
|
||||
data: {
|
||||
type: "message_delta",
|
||||
delta: { stop_reason: "end_turn" },
|
||||
usage: { output_tokens: 3 },
|
||||
},
|
||||
},
|
||||
{ data: { type: "message_stop" } },
|
||||
];
|
||||
}
|
||||
|
||||
test("streaming collector preserves context_management from the final message_delta snapshot", () => {
|
||||
const summary = collector.buildStreamSummaryFromEvents(
|
||||
claudeStreamWithAppliedEdits()
|
||||
) as ClaudeSummary;
|
||||
|
||||
assert.ok(summary && typeof summary === "object", "summary should be an object");
|
||||
assert.ok(summary.context_management, "context_management must survive reconstruction");
|
||||
const edits = summary.context_management?.applied_edits;
|
||||
assert.ok(Array.isArray(edits) && edits.length === 1, "applied_edits[] must be preserved");
|
||||
assert.equal(edits[0].cleared_input_tokens, 30000);
|
||||
assert.equal(edits[0].cleared_tool_uses, 5);
|
||||
// The rest of the Claude summary must remain intact.
|
||||
assert.equal(summary.type, "message");
|
||||
assert.equal(summary.stop_reason, "end_turn");
|
||||
assert.ok(Array.isArray(summary.content));
|
||||
});
|
||||
|
||||
test("extractContextEditingTelemetry reads applied_edits from the reconstructed streaming body", () => {
|
||||
const summary = collector.buildStreamSummaryFromEvents(claudeStreamWithAppliedEdits());
|
||||
const tele = extractContextEditingTelemetry(summary);
|
||||
|
||||
assert.ok(tele, "telemetry must be extracted from the streaming summary");
|
||||
assert.equal(tele.editCount, 1);
|
||||
assert.equal(tele.clearedInputTokens, 30000);
|
||||
assert.equal(tele.clearedToolUses, 5);
|
||||
});
|
||||
|
||||
test("streaming collector adds no context_management key when the stream carries none", () => {
|
||||
const summary = collector.buildStreamSummaryFromEvents(
|
||||
claudeStreamWithoutContextManagement()
|
||||
) as ClaudeSummary;
|
||||
|
||||
assert.ok(summary && typeof summary === "object");
|
||||
assert.equal(
|
||||
summary.context_management,
|
||||
undefined,
|
||||
"must not fabricate an empty context_management key"
|
||||
);
|
||||
assert.equal(extractContextEditingTelemetry(summary), null, "no edits → no telemetry");
|
||||
});
|
||||
86
tests/unit/context-window-reconcile.test.ts
Normal file
86
tests/unit/context-window-reconcile.test.ts
Normal file
@@ -0,0 +1,86 @@
|
||||
import { describe, it } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import {
|
||||
reconcileContextWindows,
|
||||
type DiscoveredWindow,
|
||||
type ReconcileDeps,
|
||||
} from "../../src/lib/contextWindowResolver.ts";
|
||||
|
||||
// Feature 5004 — pure reconcile logic (auto:discovery overrides), deps injected.
|
||||
|
||||
function makeDeps(
|
||||
catalog: Record<string, number | null>,
|
||||
existing: Record<string, string> = {}
|
||||
) {
|
||||
const writes: Array<[string, string, number]> = [];
|
||||
const removes: Array<[string, string]> = [];
|
||||
const deps: ReconcileDeps = {
|
||||
getCatalogWindow: (p, m) => (`${p}/${m}` in catalog ? catalog[`${p}/${m}`] : null),
|
||||
getExistingSource: (p, m) => existing[`${p}/${m}`] ?? null,
|
||||
writeAuto: (p, m, w) => writes.push([p, m, w]),
|
||||
removeOverride: (p, m) => removes.push([p, m]),
|
||||
};
|
||||
return { deps, writes, removes };
|
||||
}
|
||||
|
||||
describe("reconcileContextWindows (5004)", () => {
|
||||
it("writes an auto override when the discovered window diverges from the catalog", () => {
|
||||
const discovered: DiscoveredWindow[] = [{ provider: "openai", modelId: "gpt-x", window: 400000 }];
|
||||
const { deps, writes } = makeDeps({ "openai/gpt-x": 128000 });
|
||||
const r = reconcileContextWindows(discovered, deps);
|
||||
assert.deepEqual(writes, [["openai", "gpt-x", 400000]]);
|
||||
assert.equal(r.written, 1);
|
||||
assert.equal(r.scanned, 1);
|
||||
});
|
||||
|
||||
it("does nothing when the discovered window already matches the catalog", () => {
|
||||
const discovered: DiscoveredWindow[] = [{ provider: "openai", modelId: "gpt-x", window: 128000 }];
|
||||
const { deps, writes, removes } = makeDeps({ "openai/gpt-x": 128000 });
|
||||
const r = reconcileContextWindows(discovered, deps);
|
||||
assert.deepEqual(writes, []);
|
||||
assert.deepEqual(removes, []);
|
||||
assert.equal(r.written, 0);
|
||||
});
|
||||
|
||||
it("self-heals: removes a stale auto override once the catalog catches up", () => {
|
||||
const discovered: DiscoveredWindow[] = [{ provider: "openai", modelId: "gpt-x", window: 128000 }];
|
||||
const { deps, removes } = makeDeps({ "openai/gpt-x": 128000 }, { "openai/gpt-x": "auto:discovery" });
|
||||
const r = reconcileContextWindows(discovered, deps);
|
||||
assert.deepEqual(removes, [["openai", "gpt-x"]]);
|
||||
assert.equal(r.removed, 1);
|
||||
});
|
||||
|
||||
it("never overwrites or removes a manual override", () => {
|
||||
const discovered: DiscoveredWindow[] = [{ provider: "openai", modelId: "gpt-x", window: 999999 }];
|
||||
const { deps, writes, removes } = makeDeps({ "openai/gpt-x": 128000 }, { "openai/gpt-x": "manual" });
|
||||
const r = reconcileContextWindows(discovered, deps);
|
||||
assert.deepEqual(writes, []);
|
||||
assert.deepEqual(removes, []);
|
||||
assert.equal(r.skippedManual, 1);
|
||||
assert.equal(r.written, 0);
|
||||
});
|
||||
|
||||
it("skips invalid windows and empty keys", () => {
|
||||
const discovered: DiscoveredWindow[] = [
|
||||
{ provider: "openai", modelId: "a", window: 0 },
|
||||
{ provider: "openai", modelId: "b", window: -5 },
|
||||
{ provider: "openai", modelId: "c", window: 1.5 },
|
||||
{ provider: "openai", modelId: "d", window: null },
|
||||
{ provider: "", modelId: "e", window: 100000 },
|
||||
{ provider: "openai", modelId: "", window: 100000 },
|
||||
];
|
||||
const { deps, writes } = makeDeps({});
|
||||
const r = reconcileContextWindows(discovered, deps);
|
||||
assert.deepEqual(writes, []);
|
||||
assert.equal(r.scanned, 6);
|
||||
assert.equal(r.written, 0);
|
||||
});
|
||||
|
||||
it("writes an override when the catalog does not know the model (catalog null)", () => {
|
||||
const discovered: DiscoveredWindow[] = [{ provider: "local", modelId: "my-7b", window: 131072 }];
|
||||
const { deps, writes } = makeDeps({}); // catalog null for everything
|
||||
const r = reconcileContextWindows(discovered, deps);
|
||||
assert.deepEqual(writes, [["local", "my-7b", 131072]]);
|
||||
assert.equal(r.written, 1);
|
||||
});
|
||||
});
|
||||
83
tests/unit/cursor-protobuf-wire-split.test.ts
Normal file
83
tests/unit/cursor-protobuf-wire-split.test.ts
Normal file
@@ -0,0 +1,83 @@
|
||||
// Split-guard for the cursorAgentProtobuf wire-primitive extraction (god-file
|
||||
// decomposition): the low-level protobuf wire codec (varint/tag/length-delimited
|
||||
// encode+decode + the generic field walker) moved verbatim from cursorAgentProtobuf.ts
|
||||
// into cursorAgentProtobuf/wire.ts. These primitives were module-private, so the host's
|
||||
// public API is unchanged; the host imports them back internally. The locks pin the
|
||||
// leaf's surface, the encode↔decode round-trip invariants, the overrun guard, and that
|
||||
// the host now imports the wire leaf instead of defining the primitives inline.
|
||||
import { test } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import fs from "node:fs";
|
||||
import path from "node:path";
|
||||
|
||||
import * as wire from "../../open-sse/utils/cursorAgentProtobuf/wire.ts";
|
||||
|
||||
test("wire leaf exposes the primitives and wire-type constants", () => {
|
||||
assert.equal(wire.WT_VARINT, 0);
|
||||
assert.equal(wire.WT_LEN, 2);
|
||||
for (const fn of [
|
||||
"encodeVarint",
|
||||
"encodeTag",
|
||||
"encodeBytes",
|
||||
"encodeString",
|
||||
"encodeMessage",
|
||||
"encodeUInt32Field",
|
||||
"encodeBoolField",
|
||||
"encodeDoubleField",
|
||||
"decodeVarint",
|
||||
"checkedLen",
|
||||
"decodeFields",
|
||||
"findField",
|
||||
"decodeStringField",
|
||||
"decodeVarintField",
|
||||
]) {
|
||||
assert.equal(
|
||||
typeof (wire as Record<string, unknown>)[fn],
|
||||
"function",
|
||||
`${fn} must be exported`
|
||||
);
|
||||
}
|
||||
});
|
||||
|
||||
test("varint round-trips across byte-boundaries and bigints", () => {
|
||||
for (const n of [0, 1, 127, 128, 300, 16384, 2 ** 31]) {
|
||||
const [decoded, next] = wire.decodeVarint(wire.encodeVarint(n), 0);
|
||||
assert.equal(decoded, BigInt(n), `varint ${n}`);
|
||||
assert.equal(next, wire.encodeVarint(n).length);
|
||||
}
|
||||
const big = 9007199254740993n; // > Number.MAX_SAFE_INTEGER
|
||||
assert.equal(wire.decodeVarint(wire.encodeVarint(big), 0)[0], big);
|
||||
});
|
||||
|
||||
test("string / uint32 / bool fields round-trip through decodeFields", () => {
|
||||
assert.equal(wire.decodeStringField(wire.encodeString(1, "héllo"), 1), "héllo");
|
||||
assert.equal(wire.decodeStringField(wire.encodeString(3, ""), 3), "");
|
||||
assert.equal(wire.decodeVarintField(wire.encodeUInt32Field(2, 42), 2), 42);
|
||||
assert.equal(wire.decodeVarintField(wire.encodeBoolField(4, true), 4), 1);
|
||||
assert.equal(wire.decodeVarintField(wire.encodeBoolField(4, false), 4), 0);
|
||||
});
|
||||
|
||||
test("decodeFields tags length-delimited vs varint fields and encodeMessage nests", () => {
|
||||
const nested = wire.encodeMessage(5, [wire.encodeString(1, "x"), wire.encodeUInt32Field(2, 7)]);
|
||||
const [outer] = wire.decodeFields(nested);
|
||||
assert.equal(outer.fieldNumber, 5);
|
||||
assert.equal(outer.wireType, 2);
|
||||
if (outer.wireType === 2) {
|
||||
const inner = wire.decodeFields(outer.bytes);
|
||||
assert.equal(inner.length, 2);
|
||||
assert.equal(wire.decodeStringField(outer.bytes, 1), "x");
|
||||
assert.equal(wire.decodeVarintField(outer.bytes, 2), 7);
|
||||
}
|
||||
});
|
||||
|
||||
test("checkedLen rejects a length that overruns the buffer", () => {
|
||||
assert.throws(() => wire.checkedLen(5n, 0, Buffer.alloc(3)), /overruns buffer/);
|
||||
assert.equal(wire.checkedLen(3n, 0, Buffer.alloc(3)), 3);
|
||||
});
|
||||
|
||||
test("host imports the wire leaf and no longer defines the primitives inline", () => {
|
||||
const host = fs.readFileSync(path.join("open-sse", "utils", "cursorAgentProtobuf.ts"), "utf-8");
|
||||
assert.match(host, /from "\.\/cursorAgentProtobuf\/wire\.ts"/);
|
||||
assert.doesNotMatch(host, /^function encodeVarint\(/m, "encodeVarint must live in the wire leaf");
|
||||
assert.doesNotMatch(host, /^function decodeFields\(/m, "decodeFields must live in the wire leaf");
|
||||
});
|
||||
85
tests/unit/db-backup-autobackup-setting-5871.test.ts
Normal file
85
tests/unit/db-backup-autobackup-setting-5871.test.ts
Normal file
@@ -0,0 +1,85 @@
|
||||
/**
|
||||
* Regression guard for #5871.
|
||||
*
|
||||
* Full pre-write SQLite backups (~70MB) must honor the persisted
|
||||
* `backup.autoBackupEnabled` dashboard setting, not only the
|
||||
* `DISABLE_SQLITE_AUTO_BACKUP` env var. Before the fix, disabling auto-backup in
|
||||
* the UI had no effect and pre-write snapshots kept firing (bounded only by the
|
||||
* 60-minute throttle).
|
||||
*
|
||||
* NOTE: `isSqliteAutoBackupDisabled()` short-circuits to `true` under the test
|
||||
* runner, so `backupDbFile()` always returns null in tests regardless of the new
|
||||
* gate. We therefore exercise the setting-gate logic directly via the exported
|
||||
* `isAutoBackupDisabledBySetting()` helper, which is exactly what `backupDbFile()`
|
||||
* consults for non-manual / non-pre-restore reasons.
|
||||
*/
|
||||
import test from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import fs from "node:fs";
|
||||
import os from "node:os";
|
||||
import path from "node:path";
|
||||
|
||||
const TEST_DATA_DIR = fs.mkdtempSync(path.join(os.tmpdir(), "omniroute-backup-5871-"));
|
||||
process.env.DATA_DIR = TEST_DATA_DIR;
|
||||
process.env.DISABLE_SQLITE_AUTO_BACKUP = "true";
|
||||
|
||||
const core = await import("../../src/lib/db/core.ts");
|
||||
const backup = await import("../../src/lib/db/backup.ts");
|
||||
const databaseSettings = await import("../../src/lib/db/databaseSettings.ts");
|
||||
|
||||
function resetStorage() {
|
||||
core.resetDbInstance();
|
||||
fs.rmSync(TEST_DATA_DIR, { recursive: true, force: true });
|
||||
fs.mkdirSync(TEST_DATA_DIR, { recursive: true });
|
||||
}
|
||||
|
||||
test.beforeEach(() => {
|
||||
resetStorage();
|
||||
core.getDbInstance();
|
||||
});
|
||||
|
||||
test.after(() => {
|
||||
core.resetDbInstance();
|
||||
fs.rmSync(TEST_DATA_DIR, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
test("fresh install (seeded default autoBackupEnabled=false) → auto backups disabled", () => {
|
||||
// The DB seed persists databaseSettings.autoBackupEnabled=false by default, so the
|
||||
// dashboard toggle reads "off" out of the box — the gate must honor that.
|
||||
assert.equal(databaseSettings.getUserDatabaseSettings().backup.autoBackupEnabled, false);
|
||||
assert.equal(backup.isAutoBackupDisabledBySetting(), true);
|
||||
});
|
||||
|
||||
test("no persisted value at all → auto backups are NOT disabled (backups allowed)", () => {
|
||||
// Remove every persisted autoBackupEnabled row to exercise the "absent" branch.
|
||||
const db = core.getDbInstance();
|
||||
db.prepare("DELETE FROM key_value WHERE key IN (?, ?)").run(
|
||||
"autoBackupEnabled",
|
||||
"backup.autoBackupEnabled"
|
||||
);
|
||||
db.prepare("DELETE FROM key_value WHERE namespace='settings' AND key IN (?, ?)").run(
|
||||
"backup",
|
||||
"databaseSettings"
|
||||
);
|
||||
assert.equal(backup.isAutoBackupDisabledBySetting(), false);
|
||||
});
|
||||
|
||||
test("autoBackupEnabled=false → auto backups are disabled (gate trips)", () => {
|
||||
databaseSettings.updateDatabaseSettings({ backup: { autoBackupEnabled: false } });
|
||||
assert.equal(backup.isAutoBackupDisabledBySetting(), true);
|
||||
});
|
||||
|
||||
test("autoBackupEnabled=true → auto backups are NOT disabled", () => {
|
||||
databaseSettings.updateDatabaseSettings({ backup: { autoBackupEnabled: true } });
|
||||
assert.equal(backup.isAutoBackupDisabledBySetting(), false);
|
||||
});
|
||||
|
||||
test("persisted value survives a getUserDatabaseSettings round-trip", () => {
|
||||
databaseSettings.updateDatabaseSettings({ backup: { autoBackupEnabled: false } });
|
||||
assert.equal(databaseSettings.getUserDatabaseSettings().backup.autoBackupEnabled, false);
|
||||
assert.equal(backup.isAutoBackupDisabledBySetting(), true);
|
||||
|
||||
databaseSettings.updateDatabaseSettings({ backup: { autoBackupEnabled: true } });
|
||||
assert.equal(databaseSettings.getUserDatabaseSettings().backup.autoBackupEnabled, true);
|
||||
assert.equal(backup.isAutoBackupDisabledBySetting(), false);
|
||||
});
|
||||
110
tests/unit/db-migrationrunner-constants-split.test.ts
Normal file
110
tests/unit/db-migrationrunner-constants-split.test.ts
Normal file
@@ -0,0 +1,110 @@
|
||||
/**
|
||||
* Characterization / snapshot test: migrationRunner.ts god-file decomposition.
|
||||
*
|
||||
* The static migration-compatibility data tables were extracted verbatim from
|
||||
* src/lib/db/migrationRunner.ts into the pure-data leaf
|
||||
* src/lib/db/migrationRunner/constants.ts (no imports, no DB, no behaviour).
|
||||
*
|
||||
* These tables drive the reconciliation / dedup / already-applied detection
|
||||
* paths in runMigrations(). The existing db-migration-runner.test.ts proves the
|
||||
* BEHAVIOUR is unchanged; this test PINS THE DATA so a bad move (a dropped row,
|
||||
* a transposed version, a corrupted sentinel) is caught immediately — the data
|
||||
* is the thing the move could silently break.
|
||||
*
|
||||
* Pure value assertions — no DB handle is opened.
|
||||
*/
|
||||
|
||||
import { describe, it } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
|
||||
import {
|
||||
RENAMED_MIGRATION_COMPATIBILITY,
|
||||
LEGACY_VERSION_SLOT_MIGRATIONS,
|
||||
SUPERSEDED_DUPLICATE_MIGRATIONS,
|
||||
PHYSICAL_SCHEMA_SENTINELS,
|
||||
INITIAL_SCHEMA_SENTINELS,
|
||||
OPTIONAL_FTS5_MIGRATION_VERSIONS,
|
||||
} from "../../src/lib/db/migrationRunner/constants.ts";
|
||||
|
||||
// ── small tables — full snapshot ─────────────────────────────────────────────
|
||||
|
||||
describe("migrationRunner/constants — exact small-table snapshots", () => {
|
||||
it("LEGACY_VERSION_SLOT_MIGRATIONS is the 9-entry list, in order", () => {
|
||||
assert.deepEqual(
|
||||
LEGACY_VERSION_SLOT_MIGRATIONS.map((m) => `${m.version}:${m.name}`),
|
||||
[
|
||||
"028:evals_tables",
|
||||
"029:webhooks_templates",
|
||||
"030:mcp_scopes_api_keys",
|
||||
"031:api_keys_expires",
|
||||
"032:detailed_logs_warnings",
|
||||
"033:provider_connections_block_extra_usage",
|
||||
"033:add_batch_id_to_call_logs",
|
||||
"046:remove_status_from_files",
|
||||
"051:remove_status_from_files",
|
||||
]
|
||||
);
|
||||
});
|
||||
|
||||
it("SUPERSEDED_DUPLICATE_MIGRATIONS is the single 041→050 session_account_affinity entry", () => {
|
||||
assert.deepEqual(SUPERSEDED_DUPLICATE_MIGRATIONS, [
|
||||
{
|
||||
version: "041",
|
||||
name: "session_account_affinity",
|
||||
supersededByVersion: "050",
|
||||
supersededByName: "session_account_affinity",
|
||||
},
|
||||
]);
|
||||
});
|
||||
|
||||
it("INITIAL_SCHEMA_SENTINELS pins the three baseline tables", () => {
|
||||
assert.deepEqual(INITIAL_SCHEMA_SENTINELS, ["provider_connections", "combos", "call_logs"]);
|
||||
});
|
||||
|
||||
it("OPTIONAL_FTS5_MIGRATION_VERSIONS is exactly {022, 023}", () => {
|
||||
assert.ok(OPTIONAL_FTS5_MIGRATION_VERSIONS instanceof Set);
|
||||
assert.deepEqual([...OPTIONAL_FTS5_MIGRATION_VERSIONS].sort(), ["022", "023"]);
|
||||
});
|
||||
});
|
||||
|
||||
// ── large tables — count + shape + spot-checks (corruption guard) ─────────────
|
||||
|
||||
describe("migrationRunner/constants — large-table integrity", () => {
|
||||
it("RENAMED_MIGRATION_COMPATIBILITY has 10 well-formed entries", () => {
|
||||
assert.equal(RENAMED_MIGRATION_COMPATIBILITY.length, 10);
|
||||
for (const e of RENAMED_MIGRATION_COMPATIBILITY) {
|
||||
assert.equal(typeof e.fromVersion, "string");
|
||||
assert.equal(typeof e.fromName, "string");
|
||||
assert.equal(typeof e.toVersion, "string");
|
||||
assert.equal(typeof e.toName, "string");
|
||||
}
|
||||
});
|
||||
|
||||
it("RENAMED_MIGRATION_COMPATIBILITY spot-checks the boundary renames", () => {
|
||||
const first = RENAMED_MIGRATION_COMPATIBILITY[0];
|
||||
assert.deepEqual(first, {
|
||||
fromVersion: "022",
|
||||
fromName: "call_logs_summary_storage",
|
||||
toVersion: "025",
|
||||
toName: "call_logs_summary_storage",
|
||||
});
|
||||
// both manifest_routing collisions (052→059 and 056→059) must survive
|
||||
const manifest = RENAMED_MIGRATION_COMPATIBILITY.filter((e) => e.toName === "manifest_routing");
|
||||
assert.deepEqual(manifest.map((e) => e.fromVersion).sort(), ["052", "056"]);
|
||||
});
|
||||
|
||||
it("PHYSICAL_SCHEMA_SENTINELS has 15 well-formed entries incl. the newest 064", () => {
|
||||
assert.equal(PHYSICAL_SCHEMA_SENTINELS.length, 15);
|
||||
for (const e of PHYSICAL_SCHEMA_SENTINELS) {
|
||||
assert.equal(typeof e.version, "string");
|
||||
assert.equal(typeof e.tableName, "string");
|
||||
assert.equal(typeof e.description, "string");
|
||||
}
|
||||
const byVersion = Object.fromEntries(
|
||||
PHYSICAL_SCHEMA_SENTINELS.map((e) => [e.version, e.tableName])
|
||||
);
|
||||
assert.equal(byVersion["064"], "session_model_history");
|
||||
assert.equal(byVersion["002"], "mcp_tool_audit");
|
||||
assert.equal(byVersion["028"], "batches");
|
||||
});
|
||||
});
|
||||
90
tests/unit/db-model-context-overrides.test.ts
Normal file
90
tests/unit/db-model-context-overrides.test.ts
Normal file
@@ -0,0 +1,90 @@
|
||||
import { describe, it, beforeEach, after } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import fs from "node:fs";
|
||||
import os from "node:os";
|
||||
import path from "node:path";
|
||||
|
||||
// Feature 5004 — model_context_overrides DB module round-trip.
|
||||
|
||||
const moduleDataDir = fs.mkdtempSync(path.join(os.tmpdir(), "omni-mco-module-"));
|
||||
process.env.DATA_DIR = moduleDataDir;
|
||||
|
||||
const coreDb = await import("../../src/lib/db/core.ts");
|
||||
const mco = await import("../../src/lib/db/modelContextOverrides.ts");
|
||||
|
||||
function resetStorage() {
|
||||
coreDb.resetDbInstance();
|
||||
fs.rmSync(moduleDataDir, { recursive: true, force: true });
|
||||
fs.mkdirSync(moduleDataDir, { recursive: true });
|
||||
}
|
||||
|
||||
beforeEach(() => {
|
||||
resetStorage();
|
||||
// Touch the DB so migration 110 creates the table.
|
||||
coreDb.getDbInstance();
|
||||
});
|
||||
|
||||
after(() => {
|
||||
coreDb.resetDbInstance();
|
||||
fs.rmSync(moduleDataDir, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
describe("modelContextOverrides", () => {
|
||||
it("returns null when there is no override", () => {
|
||||
assert.equal(mco.getModelContextOverride("openai", "gpt-5"), null);
|
||||
assert.equal(mco.getModelContextOverrideRecord("openai", "gpt-5"), null);
|
||||
});
|
||||
|
||||
it("round-trips a manual override (set -> get -> record)", () => {
|
||||
assert.equal(mco.setModelContextOverride("openai", "gpt-5", 400000), true);
|
||||
assert.equal(mco.getModelContextOverride("openai", "gpt-5"), 400000);
|
||||
const rec = mco.getModelContextOverrideRecord("openai", "gpt-5");
|
||||
assert.equal(rec?.realContext, 400000);
|
||||
assert.equal(rec?.source, "manual");
|
||||
assert.equal(rec?.provider, "openai");
|
||||
assert.equal(rec?.modelId, "gpt-5");
|
||||
});
|
||||
|
||||
it("upserts on the same (provider, model) key and records the source", () => {
|
||||
mco.setModelContextOverride("anthropic", "claude-sonnet-4-5", 200000, "auto:discovery");
|
||||
assert.equal(mco.getModelContextOverrideRecord("anthropic", "claude-sonnet-4-5")?.source, "auto:discovery");
|
||||
// Re-set as manual overwrites the same row.
|
||||
mco.setModelContextOverride("anthropic", "claude-sonnet-4-5", 1000000, "manual");
|
||||
const rec = mco.getModelContextOverrideRecord("anthropic", "claude-sonnet-4-5");
|
||||
assert.equal(rec?.realContext, 1000000);
|
||||
assert.equal(rec?.source, "manual");
|
||||
assert.equal(mco.listModelContextOverrides().length, 1);
|
||||
});
|
||||
|
||||
it("rejects non-positive / non-integer windows and empty keys (no write)", () => {
|
||||
assert.equal(mco.setModelContextOverride("openai", "gpt-5", 0), false);
|
||||
assert.equal(mco.setModelContextOverride("openai", "gpt-5", -1), false);
|
||||
assert.equal(mco.setModelContextOverride("openai", "gpt-5", 1.5), false);
|
||||
assert.equal(mco.setModelContextOverride("", "gpt-5", 1000), false);
|
||||
assert.equal(mco.setModelContextOverride("openai", " ", 1000), false);
|
||||
assert.equal(mco.getModelContextOverride("openai", "gpt-5"), null);
|
||||
});
|
||||
|
||||
it("trims keys so lookups match writes", () => {
|
||||
mco.setModelContextOverride(" openai ", " gpt-5 ", 333000);
|
||||
assert.equal(mco.getModelContextOverride("openai", "gpt-5"), 333000);
|
||||
});
|
||||
|
||||
it("removes an override", () => {
|
||||
mco.setModelContextOverride("groq", "llama-3.3-70b", 128000);
|
||||
assert.equal(mco.removeModelContextOverride("groq", "llama-3.3-70b"), true);
|
||||
assert.equal(mco.getModelContextOverride("groq", "llama-3.3-70b"), null);
|
||||
assert.equal(mco.removeModelContextOverride("groq", "llama-3.3-70b"), false);
|
||||
});
|
||||
|
||||
it("lists all overrides", () => {
|
||||
mco.setModelContextOverride("openai", "gpt-5", 400000);
|
||||
mco.setModelContextOverride("anthropic", "claude-sonnet-4-5", 200000, "auto:discovery");
|
||||
const all = mco.listModelContextOverrides();
|
||||
assert.equal(all.length, 2);
|
||||
assert.deepEqual(
|
||||
all.map((o) => `${o.provider}/${o.modelId}`).sort(),
|
||||
["anthropic/claude-sonnet-4-5", "openai/gpt-5"]
|
||||
);
|
||||
});
|
||||
});
|
||||
117
tests/unit/db-models-split.test.ts
Normal file
117
tests/unit/db-models-split.test.ts
Normal file
@@ -0,0 +1,117 @@
|
||||
/**
|
||||
* Characterization tests for the db/models god-file decomposition (BLOCO E3).
|
||||
* The compat-overrides, model-alias, MITM-alias and shared helpers were lifted out
|
||||
* of src/lib/db/models.ts verbatim into cohesive leaf modules under db/models/.
|
||||
* These tests pin the behavior of the pure extracted helpers AND guard that the
|
||||
* host module still re-exports the full public API consumed across the codebase.
|
||||
*
|
||||
* (Behavioral DB-backed coverage of the moved CRUD/alias paths lives in the
|
||||
* existing db-models-crud / db-models-extended / db-model-aliases-cascade suites.)
|
||||
*/
|
||||
import test from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
|
||||
import { asRecord, toNonEmptyString, getKeyValue } from "../../src/lib/db/models/shared.ts";
|
||||
import {
|
||||
sanitizeUpstreamHeadersMap,
|
||||
isCompatProtocolKey,
|
||||
deepMergeCompatByProtocol,
|
||||
compatByProtocolHasEntries,
|
||||
} from "../../src/lib/db/models/compat.ts";
|
||||
|
||||
test("shared: asRecord only unwraps plain objects", () => {
|
||||
assert.deepEqual(asRecord({ a: 1 }), { a: 1 });
|
||||
assert.deepEqual(asRecord(null), {});
|
||||
assert.deepEqual(asRecord([1, 2]), {});
|
||||
assert.deepEqual(asRecord("x"), {});
|
||||
});
|
||||
|
||||
test("shared: toNonEmptyString trims and rejects blanks", () => {
|
||||
assert.equal(toNonEmptyString(" hi "), "hi");
|
||||
assert.equal(toNonEmptyString(" "), null);
|
||||
assert.equal(toNonEmptyString(""), null);
|
||||
assert.equal(toNonEmptyString(5), null);
|
||||
});
|
||||
|
||||
test("shared: getKeyValue reads key/value strings only", () => {
|
||||
assert.deepEqual(getKeyValue({ key: "k", value: "v" }), { key: "k", value: "v" });
|
||||
assert.deepEqual(getKeyValue({ key: 1, value: null }), { key: null, value: null });
|
||||
assert.deepEqual(getKeyValue(undefined), { key: null, value: null });
|
||||
});
|
||||
|
||||
test("compat: sanitizeUpstreamHeadersMap enforces structural validity", () => {
|
||||
assert.deepEqual(sanitizeUpstreamHeadersMap({ "X-Api-Key": "abc" }), { "X-Api-Key": "abc" });
|
||||
// names with whitespace or colon are dropped
|
||||
assert.deepEqual(sanitizeUpstreamHeadersMap({ "Bad Name": "v" }), {});
|
||||
assert.deepEqual(sanitizeUpstreamHeadersMap({ "bad:name": "v" }), {});
|
||||
// CRLF in value is rejected
|
||||
assert.deepEqual(sanitizeUpstreamHeadersMap({ "X-H": "a\r\nb" }), {});
|
||||
// null / non-object => empty
|
||||
assert.deepEqual(sanitizeUpstreamHeadersMap(null), {});
|
||||
// cap at 16 entries
|
||||
const many: Record<string, string> = {};
|
||||
for (let i = 0; i < 40; i++) many[`H-${i}`] = "v";
|
||||
assert.equal(Object.keys(sanitizeUpstreamHeadersMap(many)).length, 16);
|
||||
});
|
||||
|
||||
test("compat: isCompatProtocolKey recognizes known protocol keys only", () => {
|
||||
assert.equal(isCompatProtocolKey("openai"), true);
|
||||
assert.equal(isCompatProtocolKey("definitely-not-a-protocol"), false);
|
||||
});
|
||||
|
||||
test("compat: deepMergeCompatByProtocol + compatByProtocolHasEntries", () => {
|
||||
const merged = deepMergeCompatByProtocol(undefined, {
|
||||
openai: { normalizeToolCallId: true },
|
||||
});
|
||||
assert.equal(merged.openai?.normalizeToolCallId, true);
|
||||
assert.equal(compatByProtocolHasEntries(merged), true);
|
||||
assert.equal(compatByProtocolHasEntries(undefined), false);
|
||||
assert.equal(compatByProtocolHasEntries({}), false);
|
||||
// a later patch deep-merges without clobbering prior fields
|
||||
const merged2 = deepMergeCompatByProtocol(merged, {
|
||||
openai: { preserveOpenAIDeveloperRole: false },
|
||||
});
|
||||
assert.equal(merged2.openai?.normalizeToolCallId, true);
|
||||
assert.equal(merged2.openai?.preserveOpenAIDeveloperRole, false);
|
||||
});
|
||||
|
||||
test("host db/models.ts preserves its full public API after the split", async () => {
|
||||
const host = (await import("../../src/lib/db/models.ts")) as Record<string, unknown>;
|
||||
// re-exported from leaves
|
||||
for (const name of [
|
||||
"sanitizeUpstreamHeadersMap",
|
||||
"getModelCompatOverrides",
|
||||
"mergeModelCompatOverride",
|
||||
"removeModelCompatOverride",
|
||||
"MODEL_COMPAT_PROTOCOL_KEYS",
|
||||
"getModelAliases",
|
||||
"setModelAlias",
|
||||
"deleteModelAlias",
|
||||
"deleteModelAliasesForProvider",
|
||||
"getMitmAlias",
|
||||
"setMitmAliasAll",
|
||||
]) {
|
||||
assert.ok(host[name] !== undefined, `db/models must still export ${name}`);
|
||||
}
|
||||
// kept in host (custom / synced / flags)
|
||||
for (const name of [
|
||||
"getCustomModels",
|
||||
"getAllCustomModels",
|
||||
"addCustomModel",
|
||||
"replaceCustomModels",
|
||||
"removeCustomModel",
|
||||
"updateCustomModel",
|
||||
"getSyncedAvailableModels",
|
||||
"getAllSyncedAvailableModels",
|
||||
"replaceSyncedAvailableModelsForConnection",
|
||||
"getModelNormalizeToolCallId",
|
||||
"getModelPreserveOpenAIDeveloperRole",
|
||||
"getModelIsHidden",
|
||||
"getHiddenModelsByProvider",
|
||||
"getModelIsDeleted",
|
||||
"setModelIsHidden",
|
||||
"getModelUpstreamExtraHeaders",
|
||||
]) {
|
||||
assert.equal(typeof host[name], "function", `db/models must still export ${name}`);
|
||||
}
|
||||
});
|
||||
156
tests/unit/db-providers-cross-idp-dedup-2244.test.ts
Normal file
156
tests/unit/db-providers-cross-idp-dedup-2244.test.ts
Normal file
@@ -0,0 +1,156 @@
|
||||
// Cross-IdP OAuth account dedup — createProviderConnection matched OAuth
|
||||
// connections by email only, so two different IdPs that happen to share an
|
||||
// email address (e.g. a Google account and a HuggingFace account) would
|
||||
// silently overwrite each other on the second login. Disambiguate on
|
||||
// providerSpecificData.username when BOTH the incoming and an existing
|
||||
// connection carry one; fall back to the legacy email-only match when
|
||||
// neither side has a username (backward compat for rows created before
|
||||
// this fix). This test fails before the fix (case b overwrites instead of
|
||||
// inserting a new row).
|
||||
import test from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import fs from "node:fs";
|
||||
import os from "node:os";
|
||||
import path from "node:path";
|
||||
|
||||
const TEST_DATA_DIR = fs.mkdtempSync(path.join(os.tmpdir(), "omniroute-cross-idp-dedup-"));
|
||||
process.env.DATA_DIR = TEST_DATA_DIR;
|
||||
|
||||
const core = await import("../../src/lib/db/core.ts");
|
||||
const providersDb = await import("../../src/lib/db/providers.ts");
|
||||
|
||||
async function resetStorage() {
|
||||
core.resetDbInstance();
|
||||
for (let attempt = 0; attempt < 10; attempt++) {
|
||||
try {
|
||||
if (fs.existsSync(TEST_DATA_DIR)) {
|
||||
fs.rmSync(TEST_DATA_DIR, { recursive: true, force: true });
|
||||
}
|
||||
break;
|
||||
} catch (error: unknown) {
|
||||
const code = (error as { code?: string } | null)?.code;
|
||||
if ((code === "EBUSY" || code === "EPERM") && attempt < 9) {
|
||||
await new Promise((resolve) => setTimeout(resolve, 50 * (attempt + 1)));
|
||||
} else {
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
}
|
||||
fs.mkdirSync(TEST_DATA_DIR, { recursive: true });
|
||||
}
|
||||
|
||||
test.beforeEach(async () => {
|
||||
await resetStorage();
|
||||
});
|
||||
|
||||
test.after(async () => {
|
||||
core.resetDbInstance();
|
||||
fs.rmSync(TEST_DATA_DIR, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
test("#2244 cross-IdP dedup: same email + same username updates the existing connection", async () => {
|
||||
const first = await providersDb.createProviderConnection({
|
||||
provider: "glm",
|
||||
authType: "oauth",
|
||||
email: "shared@example.com",
|
||||
providerSpecificData: { username: "alice-google" },
|
||||
isActive: true,
|
||||
});
|
||||
const second = await providersDb.createProviderConnection({
|
||||
provider: "glm",
|
||||
authType: "oauth",
|
||||
email: "shared@example.com",
|
||||
providerSpecificData: { username: "alice-google" },
|
||||
isActive: true,
|
||||
});
|
||||
|
||||
const conns = await providersDb.getProviderConnections({ provider: "glm" });
|
||||
assert.equal(conns.length, 1, "same email + same username must dedupe to a single connection");
|
||||
assert.equal((first as { id: string }).id, (second as { id: string }).id);
|
||||
});
|
||||
|
||||
test("#2244 cross-IdP dedup: same email + DIFFERENT username creates a separate connection", async () => {
|
||||
await providersDb.createProviderConnection({
|
||||
provider: "glm",
|
||||
authType: "oauth",
|
||||
email: "shared@example.com",
|
||||
providerSpecificData: { username: "alice-google" },
|
||||
isActive: true,
|
||||
});
|
||||
await providersDb.createProviderConnection({
|
||||
provider: "glm",
|
||||
authType: "oauth",
|
||||
email: "shared@example.com",
|
||||
providerSpecificData: { username: "alice-huggingface" },
|
||||
isActive: true,
|
||||
});
|
||||
|
||||
const conns = await providersDb.getProviderConnections({ provider: "glm" });
|
||||
assert.equal(
|
||||
conns.length,
|
||||
2,
|
||||
"two different IdP identities sharing an email must NOT be collapsed into one connection"
|
||||
);
|
||||
const usernames = conns
|
||||
.map((c) => (c as { providerSpecificData?: { username?: string } }).providerSpecificData?.username)
|
||||
.sort();
|
||||
assert.deepEqual(usernames, ["alice-google", "alice-huggingface"]);
|
||||
});
|
||||
|
||||
test("#2244 cross-IdP dedup: legacy rows without username still dedupe against incoming without username", async () => {
|
||||
const first = await providersDb.createProviderConnection({
|
||||
provider: "glm",
|
||||
authType: "oauth",
|
||||
email: "legacy@example.com",
|
||||
providerSpecificData: {},
|
||||
isActive: true,
|
||||
});
|
||||
const second = await providersDb.createProviderConnection({
|
||||
provider: "glm",
|
||||
authType: "oauth",
|
||||
email: "legacy@example.com",
|
||||
providerSpecificData: {},
|
||||
isActive: true,
|
||||
});
|
||||
|
||||
const conns = await providersDb.getProviderConnections({ provider: "glm" });
|
||||
assert.equal(
|
||||
conns.length,
|
||||
1,
|
||||
"legacy email-only rows without a username must keep deduping on email alone"
|
||||
);
|
||||
assert.equal((first as { id: string }).id, (second as { id: string }).id);
|
||||
});
|
||||
|
||||
test("#2244 cross-IdP dedup: Codex workspaceId matching path is unaffected", async () => {
|
||||
const first = await providersDb.createProviderConnection({
|
||||
provider: "codex",
|
||||
authType: "oauth",
|
||||
email: "team@example.com",
|
||||
providerSpecificData: { workspaceId: "ws-1", username: "team-user" },
|
||||
isActive: true,
|
||||
});
|
||||
const second = await providersDb.createProviderConnection({
|
||||
provider: "codex",
|
||||
authType: "oauth",
|
||||
email: "team@example.com",
|
||||
providerSpecificData: { workspaceId: "ws-1", username: "team-user-renamed" },
|
||||
isActive: true,
|
||||
});
|
||||
const third = await providersDb.createProviderConnection({
|
||||
provider: "codex",
|
||||
authType: "oauth",
|
||||
email: "team@example.com",
|
||||
providerSpecificData: { workspaceId: "ws-2", username: "team-user" },
|
||||
isActive: true,
|
||||
});
|
||||
|
||||
const conns = await providersDb.getProviderConnections({ provider: "codex" });
|
||||
assert.equal(
|
||||
conns.length,
|
||||
2,
|
||||
"Codex must keep matching on workspaceId + email regardless of username, and a different workspace must stay a separate connection"
|
||||
);
|
||||
assert.equal((first as { id: string }).id, (second as { id: string }).id);
|
||||
assert.notEqual((first as { id: string }).id, (third as { id: string }).id);
|
||||
});
|
||||
174
tests/unit/db-providers-split.test.ts
Normal file
174
tests/unit/db-providers-split.test.ts
Normal file
@@ -0,0 +1,174 @@
|
||||
/**
|
||||
* Characterization + API-surface test: providers.ts god-file decomposition.
|
||||
*
|
||||
* The host src/lib/db/providers.ts was split into three sibling leaf modules
|
||||
* under src/lib/db/providers/:
|
||||
* - columns.ts — 10 pure column-normalizer helpers (DB-free)
|
||||
* - nodes.ts — 6 provider-node CRUD functions
|
||||
* - rateLimit.ts — 6 rate-limit/quota runtime helpers + formatResetCountdown
|
||||
*
|
||||
* This test verifies that:
|
||||
* 1. The pure column helpers in columns.ts behave correctly (DB-free).
|
||||
* 2. The host providers.ts still re-exports the FULL public API surface (23
|
||||
* symbols) so every existing consumer keeps importing from the same path.
|
||||
* 3. The leaf modules export their own pieces directly.
|
||||
*
|
||||
* Pure typeof/behaviour checks only — no function is invoked against the DB,
|
||||
* so no SQLite handle is opened (no resetDbInstance teardown required).
|
||||
*/
|
||||
|
||||
import { describe, it } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
|
||||
// ── 1. columns.ts — pure helpers behaviour ───────────────────────────────────
|
||||
|
||||
import {
|
||||
normalizeBooleanColumn,
|
||||
sanitizeRateLimitOverrides,
|
||||
sanitizeQuotaWindowThresholds,
|
||||
serializeJsonField,
|
||||
toRecord,
|
||||
toStringOrNull,
|
||||
toNumberOrZero,
|
||||
} from "../../src/lib/db/providers/columns.ts";
|
||||
|
||||
describe("providers/columns — normalizeBooleanColumn", () => {
|
||||
it("passes through real booleans", () => {
|
||||
assert.equal(normalizeBooleanColumn(true, false), true);
|
||||
assert.equal(normalizeBooleanColumn(false, true), false);
|
||||
});
|
||||
it("maps SQLite integer flags (1/0)", () => {
|
||||
assert.equal(normalizeBooleanColumn(1, false), true);
|
||||
assert.equal(normalizeBooleanColumn(0, true), false);
|
||||
});
|
||||
it("parses string flags case-insensitively", () => {
|
||||
assert.equal(normalizeBooleanColumn("True", false), true);
|
||||
assert.equal(normalizeBooleanColumn("0", true), false);
|
||||
});
|
||||
it("returns the fallback for unrecognized values", () => {
|
||||
assert.equal(normalizeBooleanColumn("maybe", true), true);
|
||||
assert.equal(normalizeBooleanColumn(undefined, false), false);
|
||||
});
|
||||
});
|
||||
|
||||
describe("providers/columns — sanitizeRateLimitOverrides", () => {
|
||||
it("returns null for nullish / non-object / array input", () => {
|
||||
assert.equal(sanitizeRateLimitOverrides(null), null);
|
||||
assert.equal(sanitizeRateLimitOverrides(undefined), null);
|
||||
assert.equal(sanitizeRateLimitOverrides("x"), null);
|
||||
assert.equal(sanitizeRateLimitOverrides([1, 2]), null);
|
||||
});
|
||||
it("keeps only allowed keys with non-negative integers", () => {
|
||||
assert.deepEqual(sanitizeRateLimitOverrides({ rpm: 10, bogus: 5, tpm: -1 }), { rpm: 10 });
|
||||
});
|
||||
it("returns null when nothing valid remains", () => {
|
||||
assert.equal(sanitizeRateLimitOverrides({ rpm: 1.5, nope: 3 }), null);
|
||||
});
|
||||
});
|
||||
|
||||
describe("providers/columns — sanitizeQuotaWindowThresholds", () => {
|
||||
it("keeps only 0-100 integers", () => {
|
||||
assert.deepEqual(sanitizeQuotaWindowThresholds({ a: 50, b: 120, c: 0 }), { a: 50, c: 0 });
|
||||
});
|
||||
it("returns null when empty", () => {
|
||||
assert.equal(sanitizeQuotaWindowThresholds({ a: 200 }), null);
|
||||
});
|
||||
});
|
||||
|
||||
describe("providers/columns — small coercers", () => {
|
||||
it("serializeJsonField stringifies objects, null otherwise", () => {
|
||||
assert.equal(serializeJsonField({ a: 1 }), '{"a":1}');
|
||||
assert.equal(serializeJsonField(null), null);
|
||||
assert.equal(serializeJsonField("str"), null);
|
||||
});
|
||||
it("toRecord returns objects as-is, {} otherwise", () => {
|
||||
const o = { x: 1 };
|
||||
assert.equal(toRecord(o), o);
|
||||
assert.deepEqual(toRecord(null), {});
|
||||
assert.deepEqual(toRecord(42), {});
|
||||
});
|
||||
it("toStringOrNull / toNumberOrZero coerce by type", () => {
|
||||
assert.equal(toStringOrNull("hi"), "hi");
|
||||
assert.equal(toStringOrNull(5), null);
|
||||
assert.equal(toNumberOrZero(7), 7);
|
||||
assert.equal(toNumberOrZero("7"), 0);
|
||||
});
|
||||
});
|
||||
|
||||
// ── 2. providers.ts — full public API surface preserved ──────────────────────
|
||||
|
||||
const host = await import("../../src/lib/db/providers.ts");
|
||||
|
||||
describe("providers.ts public API surface (23 symbols)", () => {
|
||||
const expected = [
|
||||
// Connection CRUD (kept in host)
|
||||
"getProviderConnections",
|
||||
"getProviderConnectionById",
|
||||
"createProviderConnection",
|
||||
"updateProviderConnection",
|
||||
"deleteProviderConnection",
|
||||
"deleteProviderConnections",
|
||||
"deleteProviderConnectionsByProvider",
|
||||
"reorderProviderConnections",
|
||||
"cleanupProviderConnections",
|
||||
"getDistinctGroups",
|
||||
"autoMigrateLegacyEncryptedConnections",
|
||||
// Provider nodes (re-exported from ./providers/nodes)
|
||||
"getProviderNodes",
|
||||
"getProviderNodeById",
|
||||
"resolveProviderNodeForConnection",
|
||||
"createProviderNode",
|
||||
"updateProviderNode",
|
||||
"deleteProviderNode",
|
||||
// Rate-limit / quota runtime (re-exported from ./providers/rateLimit)
|
||||
"setConnectionRateLimitUntil",
|
||||
"isConnectionRateLimited",
|
||||
"getRateLimitedConnections",
|
||||
"getEffectiveQuotaUsage",
|
||||
"clearStaleCrashCooldowns",
|
||||
"formatResetCountdown",
|
||||
];
|
||||
|
||||
for (const name of expected) {
|
||||
it(`re-exports ${name} as a function`, () => {
|
||||
assert.equal(typeof host[name], "function", `${name} must be a function on the host module`);
|
||||
});
|
||||
}
|
||||
|
||||
it("exposes exactly the 23 expected callables (no public symbol lost)", () => {
|
||||
const missing = expected.filter((n) => typeof host[n] !== "function");
|
||||
assert.deepEqual(missing, [], `missing public exports: ${missing.join(", ")}`);
|
||||
});
|
||||
});
|
||||
|
||||
// ── 3. leaf modules export their own pieces ──────────────────────────────────
|
||||
|
||||
describe("leaf modules export their slices directly", () => {
|
||||
it("nodes.ts exports the 6 node CRUD functions", async () => {
|
||||
const nodes = await import("../../src/lib/db/providers/nodes.ts");
|
||||
for (const fn of [
|
||||
"getProviderNodes",
|
||||
"getProviderNodeById",
|
||||
"resolveProviderNodeForConnection",
|
||||
"createProviderNode",
|
||||
"updateProviderNode",
|
||||
"deleteProviderNode",
|
||||
]) {
|
||||
assert.equal(typeof nodes[fn], "function", `nodes.${fn}`);
|
||||
}
|
||||
});
|
||||
|
||||
it("rateLimit.ts exports the 6 runtime helpers", async () => {
|
||||
const rl = await import("../../src/lib/db/providers/rateLimit.ts");
|
||||
for (const fn of [
|
||||
"setConnectionRateLimitUntil",
|
||||
"isConnectionRateLimited",
|
||||
"getRateLimitedConnections",
|
||||
"getEffectiveQuotaUsage",
|
||||
"clearStaleCrashCooldowns",
|
||||
"formatResetCountdown",
|
||||
]) {
|
||||
assert.equal(typeof rl[fn], "function", `rateLimit.${fn}`);
|
||||
}
|
||||
});
|
||||
});
|
||||
153
tests/unit/db-proxies-split.test.ts
Normal file
153
tests/unit/db-proxies-split.test.ts
Normal file
@@ -0,0 +1,153 @@
|
||||
/**
|
||||
* Characterization + API-surface test: proxies.ts god-file decomposition.
|
||||
*
|
||||
* The host src/lib/db/proxies.ts was split into two sibling leaf modules under
|
||||
* src/lib/db/proxies/:
|
||||
* - types.ts — the 10 proxy type/interface declarations
|
||||
* - mappers.ts — the pure row mappers / scope normalizers / payload coercers
|
||||
* (toRecord, mapProxyRow, isRelayProxyType, extractRelayAuth,
|
||||
* normalizeScope, toLegacyProxyLevel, coerceProxyPayload,
|
||||
* redactProxySecrets, …)
|
||||
*
|
||||
* The tightly-coupled CRUD + assignment + resolution core stays in the host
|
||||
* (resolution calls createProxy/assignProxyToScope, so extracting it would
|
||||
* create an import cycle).
|
||||
*
|
||||
* Verifies that:
|
||||
* 1. The pure mappers behave correctly (DB-free).
|
||||
* 2. The host proxies.ts still re-exports the FULL public API (20 functions).
|
||||
* 3. The mappers leaf exports its pieces directly.
|
||||
*
|
||||
* Pure typeof/behaviour checks only — no DB handle is opened.
|
||||
*/
|
||||
|
||||
import { describe, it } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
|
||||
// ── 1. mappers.ts — pure helper behaviour ────────────────────────────────────
|
||||
|
||||
import {
|
||||
isRelayProxyType,
|
||||
extractRelayAuth,
|
||||
normalizeScope,
|
||||
normalizeAssignmentScopeId,
|
||||
toLegacyProxyLevel,
|
||||
} from "../../src/lib/db/proxies/mappers.ts";
|
||||
|
||||
describe("proxies/mappers — isRelayProxyType", () => {
|
||||
it("is true only for relay proxy types", () => {
|
||||
for (const t of ["vercel", "deno", "cloudflare"]) {
|
||||
assert.equal(isRelayProxyType(t), true, t);
|
||||
}
|
||||
});
|
||||
it("is false for non-relay or non-string input", () => {
|
||||
assert.equal(isRelayProxyType("http"), false);
|
||||
assert.equal(isRelayProxyType("socks5"), false);
|
||||
assert.equal(isRelayProxyType(123), false);
|
||||
assert.equal(isRelayProxyType(null), false);
|
||||
});
|
||||
});
|
||||
|
||||
describe("proxies/mappers — normalizeScope", () => {
|
||||
it("maps the legacy 'key' alias to 'account'", () => {
|
||||
assert.equal(normalizeScope("key"), "account");
|
||||
});
|
||||
it("passes through known scopes (case-insensitive)", () => {
|
||||
assert.equal(normalizeScope("Provider"), "provider");
|
||||
assert.equal(normalizeScope("ACCOUNT"), "account");
|
||||
assert.equal(normalizeScope("combo"), "combo");
|
||||
});
|
||||
it("defaults to 'global' for unknown / empty", () => {
|
||||
assert.equal(normalizeScope("bogus"), "global");
|
||||
assert.equal(normalizeScope(""), "global");
|
||||
});
|
||||
});
|
||||
|
||||
describe("proxies/mappers — normalizeAssignmentScopeId", () => {
|
||||
it("returns the sentinel for the global scope", () => {
|
||||
assert.equal(normalizeAssignmentScopeId("global", "ignored"), "__global__");
|
||||
});
|
||||
it("returns the scopeId (or null) for other scopes", () => {
|
||||
assert.equal(normalizeAssignmentScopeId("provider", "openai"), "openai");
|
||||
assert.equal(normalizeAssignmentScopeId("account", null), null);
|
||||
assert.equal(normalizeAssignmentScopeId("account", undefined), null);
|
||||
});
|
||||
});
|
||||
|
||||
describe("proxies/mappers — toLegacyProxyLevel", () => {
|
||||
it("maps 'account' back to the legacy 'key' level", () => {
|
||||
assert.equal(toLegacyProxyLevel("account"), "key");
|
||||
});
|
||||
it("passes through the other scopes unchanged", () => {
|
||||
assert.equal(toLegacyProxyLevel("global"), "global");
|
||||
assert.equal(toLegacyProxyLevel("provider"), "provider");
|
||||
assert.equal(toLegacyProxyLevel("combo"), "combo");
|
||||
});
|
||||
});
|
||||
|
||||
describe("proxies/mappers — extractRelayAuth", () => {
|
||||
it("returns undefined for non-string input", () => {
|
||||
assert.equal(extractRelayAuth(null), undefined);
|
||||
assert.equal(extractRelayAuth(42), undefined);
|
||||
});
|
||||
it("returns undefined for invalid JSON (try/catch)", () => {
|
||||
assert.equal(extractRelayAuth("not json"), undefined);
|
||||
});
|
||||
it("returns the plaintext relayAuth field when present", () => {
|
||||
assert.equal(extractRelayAuth(JSON.stringify({ relayAuth: "tok-123" })), "tok-123");
|
||||
});
|
||||
it("returns undefined when no relay auth is stored", () => {
|
||||
assert.equal(extractRelayAuth(JSON.stringify({ other: "x" })), undefined);
|
||||
});
|
||||
});
|
||||
|
||||
// ── 2. proxies.ts — full public API surface preserved ────────────────────────
|
||||
|
||||
const host = await import("../../src/lib/db/proxies.ts");
|
||||
|
||||
describe("proxies.ts public API surface (20 symbols)", () => {
|
||||
const expected = [
|
||||
"assignProxyToScope",
|
||||
"bulkAssignProxyToScope",
|
||||
"createProxy",
|
||||
"createProxyAndAssign",
|
||||
"deleteProxyById",
|
||||
"extractRelayAuth", // re-export from mappers
|
||||
"getProxyAssignments",
|
||||
"getProxyById",
|
||||
"getProxyHealthStats",
|
||||
"getProxyRegistryGeneration",
|
||||
"getProxyWhereUsed",
|
||||
"listProxies",
|
||||
"migrateLegacyProxyConfigToRegistry",
|
||||
"redactProxySecrets", // re-export from mappers
|
||||
"resolveProxyForConnectionFromRegistry",
|
||||
"resolveProxyForProvider",
|
||||
"resolveProxyForScopeFromRegistry",
|
||||
"updateProxy",
|
||||
"updateProxyAndAssign",
|
||||
"upsertProxy",
|
||||
];
|
||||
|
||||
for (const name of expected) {
|
||||
it(`re-exports ${name} as a function`, () => {
|
||||
assert.equal(typeof host[name], "function", `${name} must be a function on the host module`);
|
||||
});
|
||||
}
|
||||
|
||||
it("exposes exactly the 20 expected callables (no public symbol lost)", () => {
|
||||
const missing = expected.filter((n) => typeof host[n] !== "function");
|
||||
assert.deepEqual(missing, [], `missing public exports: ${missing.join(", ")}`);
|
||||
});
|
||||
});
|
||||
|
||||
// ── 3. mappers leaf exports its slice directly ───────────────────────────────
|
||||
|
||||
describe("mappers.ts exports its public helpers directly", () => {
|
||||
it("re-exported public mappers are functions on the leaf", async () => {
|
||||
const mappers = await import("../../src/lib/db/proxies/mappers.ts");
|
||||
assert.equal(typeof mappers.extractRelayAuth, "function");
|
||||
assert.equal(typeof mappers.redactProxySecrets, "function");
|
||||
assert.equal(typeof mappers.coerceProxyPayload, "function");
|
||||
});
|
||||
});
|
||||
90
tests/unit/db-settings-split.test.ts
Normal file
90
tests/unit/db-settings-split.test.ts
Normal file
@@ -0,0 +1,90 @@
|
||||
/**
|
||||
* Characterization test: settings.ts god-file decomposition.
|
||||
* Verifies that:
|
||||
* 1. toRecord in shared.ts has correct behavior (DB-free, pure function).
|
||||
* 2. The host settings.ts still re-exports the full public API surface.
|
||||
*/
|
||||
|
||||
import { describe, it } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
|
||||
// ── 1. shared.ts — toRecord ──────────────────────────────────────────────────
|
||||
|
||||
import { toRecord } from "../../src/lib/db/settings/shared.ts";
|
||||
|
||||
describe("toRecord", () => {
|
||||
it("returns the object as-is when given a plain object", () => {
|
||||
const obj = { a: 1, b: "two" };
|
||||
assert.deepStrictEqual(toRecord(obj), obj);
|
||||
});
|
||||
|
||||
it("returns {} for null", () => {
|
||||
assert.deepStrictEqual(toRecord(null), {});
|
||||
});
|
||||
|
||||
it("returns {} for undefined", () => {
|
||||
assert.deepStrictEqual(toRecord(undefined), {});
|
||||
});
|
||||
|
||||
it("returns {} for a string", () => {
|
||||
assert.deepStrictEqual(toRecord("hello"), {});
|
||||
});
|
||||
|
||||
it("returns {} for a number", () => {
|
||||
assert.deepStrictEqual(toRecord(42), {});
|
||||
});
|
||||
|
||||
it("returns {} for an array (arrays are objects but toRecord returns the array cast)", () => {
|
||||
// toRecord casts arrays as JsonRecord — they ARE objects, so the cast succeeds.
|
||||
const arr = [1, 2, 3];
|
||||
assert.strictEqual(toRecord(arr), arr);
|
||||
});
|
||||
});
|
||||
|
||||
// ── 2. settings.ts — public API surface ─────────────────────────────────────
|
||||
|
||||
const settingsModule = await import("../../src/lib/db/settings.ts");
|
||||
|
||||
describe("settings.ts public API surface", () => {
|
||||
const expectedFunctions = [
|
||||
// Settings core
|
||||
"getSettings",
|
||||
"updateSettings",
|
||||
"isCloudEnabled",
|
||||
// Proxy helpers (exported)
|
||||
"bumpProxyConfigGeneration",
|
||||
// Proxy config
|
||||
"getProxyConfig",
|
||||
"getProxyForLevel",
|
||||
"setProxyForLevel",
|
||||
"deleteProxyForLevel",
|
||||
"resolveProxyForConnection",
|
||||
"setProxyConfig",
|
||||
// Pricing (re-exported from ./settings/pricing)
|
||||
"getPricing",
|
||||
"getPricingWithSources",
|
||||
"getPricingForModel",
|
||||
"updatePricing",
|
||||
"resetPricing",
|
||||
"resetAllPricing",
|
||||
// LKGP (re-exported from ./settings/lkgp)
|
||||
"getLKGP",
|
||||
"setLKGP",
|
||||
"clearAllLKGP",
|
||||
// Cache metrics (re-exported from ./settings/cacheMetrics)
|
||||
"getCacheMetrics",
|
||||
"updateCacheMetrics",
|
||||
"getCacheTrend",
|
||||
"resetCacheMetrics",
|
||||
] as const;
|
||||
|
||||
for (const name of expectedFunctions) {
|
||||
it(`exports "${name}" as a function`, () => {
|
||||
assert.strictEqual(
|
||||
typeof (settingsModule as Record<string, unknown>)[name],
|
||||
"function",
|
||||
`Expected "${name}" to be exported as a function`
|
||||
);
|
||||
});
|
||||
}
|
||||
});
|
||||
130
tests/unit/db-usageanalytics-split.test.ts
Normal file
130
tests/unit/db-usageanalytics-split.test.ts
Normal file
@@ -0,0 +1,130 @@
|
||||
/**
|
||||
* Characterization + API-surface test: usageAnalytics.ts god-file decomposition.
|
||||
*
|
||||
* The two pure SQL-source-string builders (buildUnifiedSource,
|
||||
* buildPresetUnifiedSource) + their types were extracted verbatim from
|
||||
* src/lib/db/usageAnalytics.ts into the pure leaf
|
||||
* src/lib/db/usageAnalytics/sources.ts (no DB, no imports). The ~20 query
|
||||
* functions stay in the host.
|
||||
*
|
||||
* Verifies that:
|
||||
* 1. buildUnifiedSource's needsAggregated branching is preserved (pure logic).
|
||||
* 2. The host usageAnalytics.ts still exposes the FULL public API (39 names).
|
||||
* 3. The sources leaf exports the builders directly.
|
||||
*
|
||||
* Pure value assertions — no DB handle is opened.
|
||||
*/
|
||||
|
||||
import { describe, it } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
|
||||
import {
|
||||
buildUnifiedSource,
|
||||
buildPresetUnifiedSource,
|
||||
} from "../../src/lib/db/usageAnalytics/sources.ts";
|
||||
|
||||
// ── 1. buildUnifiedSource — pure branching ───────────────────────────────────
|
||||
|
||||
describe("usageAnalytics/sources — buildUnifiedSource", () => {
|
||||
it("includes the daily_usage_summary leg for a wide window with no api-key filter", () => {
|
||||
const { unifiedSource, unifiedParams } = buildUnifiedSource({
|
||||
sinceIso: "2024-06-01T00:00:00.000Z",
|
||||
untilIso: null,
|
||||
rawCutoffDate: "2024-06-15",
|
||||
apiKeyWhere: "",
|
||||
apiKeyParams: {},
|
||||
});
|
||||
assert.ok(typeof unifiedSource === "string" && unifiedSource.length > 0);
|
||||
// needsAggregated = (sinceDate < rawCutoffDate) && !apiKeyWhere => true
|
||||
assert.ok(
|
||||
unifiedSource.includes("daily_usage_summary"),
|
||||
"aggregated leg must be present when the window predates the raw cutoff"
|
||||
);
|
||||
assert.equal(unifiedParams.rawCutoff, "2024-06-15");
|
||||
assert.equal(unifiedParams.rawCutoffDate, "2024-06-15");
|
||||
});
|
||||
|
||||
it("drops the aggregated leg when an api-key filter is active (raw-only)", () => {
|
||||
const { unifiedSource, unifiedParams } = buildUnifiedSource({
|
||||
sinceIso: "2024-06-01T00:00:00.000Z",
|
||||
untilIso: null,
|
||||
rawCutoffDate: "2024-06-15",
|
||||
apiKeyWhere: "(api_key_id IN (@apiKey0))",
|
||||
apiKeyParams: { apiKey0: "k1" },
|
||||
});
|
||||
// needsAggregated = ... && !apiKeyWhere => false
|
||||
assert.ok(
|
||||
!unifiedSource.includes("daily_usage_summary"),
|
||||
"aggregated leg must be absent once an api-key filter scopes the query to raw rows"
|
||||
);
|
||||
// raw leg floors at @since (not @rawCutoff) and carries the api-key params
|
||||
assert.equal(unifiedParams.since, "2024-06-01T00:00:00.000Z");
|
||||
assert.equal(unifiedParams.apiKey0, "k1");
|
||||
assert.equal(unifiedParams.rawCutoff, undefined);
|
||||
});
|
||||
|
||||
it("buildPresetUnifiedSource returns the unifiedSource/unifiedParams shape", () => {
|
||||
const result = buildPresetUnifiedSource({
|
||||
sinceIso: null,
|
||||
untilIso: null,
|
||||
rawCutoffDate: "2024-06-15",
|
||||
apiKeyWhere: "",
|
||||
apiKeyParams: {},
|
||||
});
|
||||
assert.equal(typeof result.unifiedSource, "string");
|
||||
assert.equal(typeof result.unifiedParams, "object");
|
||||
assert.ok(result.unifiedParams !== null);
|
||||
});
|
||||
});
|
||||
|
||||
// ── 2. usageAnalytics.ts — full public API surface preserved ─────────────────
|
||||
|
||||
const host = await import("../../src/lib/db/usageAnalytics.ts");
|
||||
|
||||
describe("usageAnalytics.ts public API surface", () => {
|
||||
// the 22 runtime functions (the 17 row/param interfaces are type-only and
|
||||
// erased at runtime, so they are asserted via typecheck, not here)
|
||||
const expectedFns = [
|
||||
"buildUnifiedSource", // re-exported from sources
|
||||
"buildPresetUnifiedSource", // re-exported from sources
|
||||
"getUsageSummary",
|
||||
"getDailyUsage",
|
||||
"getDailyCostRows",
|
||||
"getHeatmapRows",
|
||||
"getModelUsageRows",
|
||||
"getProviderCostRows",
|
||||
"getProviderUsageRows",
|
||||
"getAccountCostRows",
|
||||
"getAccountUsageRows",
|
||||
"getApiKeyUsageRows",
|
||||
"getServiceTierUsageRows",
|
||||
"getApiKeyMetadataRows",
|
||||
"getWeeklyPatternRows",
|
||||
"getPresetCostModelRows",
|
||||
"getEndpointUsageRows",
|
||||
"getAllUsageHistory",
|
||||
"getAllDomainCostHistory",
|
||||
"getAllDomainBudgets",
|
||||
];
|
||||
|
||||
for (const name of expectedFns) {
|
||||
it(`exposes ${name} as a function`, () => {
|
||||
assert.equal(typeof host[name], "function", `${name} must be a function on the host module`);
|
||||
});
|
||||
}
|
||||
|
||||
it("loses no public runtime function in the split", () => {
|
||||
const missing = expectedFns.filter((n) => typeof host[n] !== "function");
|
||||
assert.deepEqual(missing, [], `missing: ${missing.join(", ")}`);
|
||||
});
|
||||
});
|
||||
|
||||
// ── 3. sources leaf exports the builders directly ────────────────────────────
|
||||
|
||||
describe("sources.ts exports the builders directly", () => {
|
||||
it("buildUnifiedSource / buildPresetUnifiedSource are functions on the leaf", async () => {
|
||||
const sources = await import("../../src/lib/db/usageAnalytics/sources.ts");
|
||||
assert.equal(typeof sources.buildUnifiedSource, "function");
|
||||
assert.equal(typeof sources.buildPresetUnifiedSource, "function");
|
||||
});
|
||||
});
|
||||
@@ -102,6 +102,31 @@ test("detectMalformedNonStream returns null for valid chat completion", () => {
|
||||
assert.equal(detectMalformedNonStream(body), null);
|
||||
});
|
||||
|
||||
test("detectMalformedNonStream returns null for OpenAI choices whose content is a text-block array (#5559)", () => {
|
||||
// Cline (OAuth) returns choices[].message.content as an array of Anthropic-style
|
||||
// blocks inside an OpenAI envelope; this must count as real output, not empty_choices.
|
||||
const body = {
|
||||
id: "chatcmpl-kimi",
|
||||
object: "chat.completion",
|
||||
model: "moonshotai/kimi-k2.6",
|
||||
choices: [
|
||||
{
|
||||
index: 0,
|
||||
message: { role: "assistant", content: [{ type: "text", text: "Here is my analysis." }] },
|
||||
finish_reason: "stop",
|
||||
},
|
||||
],
|
||||
};
|
||||
assert.equal(detectMalformedNonStream(body), null);
|
||||
});
|
||||
|
||||
test("detectMalformedNonStream returns 'empty_choices' for an OpenAI choice with an empty text-block array (#5559 guard)", () => {
|
||||
const body = {
|
||||
choices: [{ message: { role: "assistant", content: [{ type: "text", text: "" }] }, finish_reason: "stop" }],
|
||||
};
|
||||
assert.equal(detectMalformedNonStream(body), "empty_choices");
|
||||
});
|
||||
|
||||
test("detectMalformedNonStream returns null for a valid Claude-native message (#4942 regression)", () => {
|
||||
const body = {
|
||||
type: "message",
|
||||
|
||||
12
tests/unit/error-boundary-stack.test.ts
Normal file
12
tests/unit/error-boundary-stack.test.ts
Normal file
@@ -0,0 +1,12 @@
|
||||
import test from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { readFile } from "node:fs/promises";
|
||||
import { join } from "node:path";
|
||||
|
||||
const repoRoot = join(import.meta.dirname, "../..");
|
||||
|
||||
test("app error boundary does not render stack traces", async () => {
|
||||
const source = await readFile(join(repoRoot, "src/app/error.tsx"), "utf8");
|
||||
|
||||
assert.doesNotMatch(source, /error\.stack/);
|
||||
});
|
||||
105
tests/unit/evalrunner-builtinsuites-split.test.ts
Normal file
105
tests/unit/evalrunner-builtinsuites-split.test.ts
Normal file
@@ -0,0 +1,105 @@
|
||||
/**
|
||||
* Split-guard — evalRunner ↔ evalRunner/builtinSuites
|
||||
*
|
||||
* Guards the extraction of the 7 built-in golden-set suites (pure data) into
|
||||
* the leaf `src/lib/evals/evalRunner/builtinSuites.ts`. Characterizes the suite
|
||||
* data (ids, case counts, key cases) and proves the host still registers every
|
||||
* leaf suite at module load. DB-free by design: only getSuite(builtInId) is
|
||||
* exercised, which short-circuits on the in-memory Map before any DB fallback.
|
||||
*/
|
||||
|
||||
import { describe, it, after } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
|
||||
import {
|
||||
goldenSet,
|
||||
codingSuite,
|
||||
reasoningSuite,
|
||||
multilingualSuite,
|
||||
safetySuite,
|
||||
instructionSuite,
|
||||
codexComparisonSuite,
|
||||
builtInSuites,
|
||||
} from "../../src/lib/evals/evalRunner/builtinSuites.ts";
|
||||
|
||||
import { getSuite, resetSuites } from "../../src/lib/evals/evalRunner.ts";
|
||||
|
||||
describe("evalRunner/builtinSuites split-guard", () => {
|
||||
after(() => {
|
||||
// Restore built-ins in case a sibling test mutated the registry.
|
||||
resetSuites();
|
||||
});
|
||||
|
||||
it("leaf exports the 7 built-in suites plus the aggregate array", () => {
|
||||
const named = [
|
||||
goldenSet,
|
||||
codingSuite,
|
||||
reasoningSuite,
|
||||
multilingualSuite,
|
||||
safetySuite,
|
||||
instructionSuite,
|
||||
codexComparisonSuite,
|
||||
];
|
||||
for (const s of named) {
|
||||
assert.ok(s, "suite must be defined");
|
||||
assert.equal(typeof s.id, "string");
|
||||
assert.equal(typeof s.name, "string");
|
||||
assert.ok(Array.isArray(s.cases));
|
||||
}
|
||||
// Same references, same order — the host relies on this ordering for load-time registration.
|
||||
assert.equal(builtInSuites.length, 7);
|
||||
assert.deepEqual(builtInSuites, named);
|
||||
});
|
||||
|
||||
it("built-in suite ids and case counts are stable (data pin)", () => {
|
||||
assert.deepEqual(
|
||||
builtInSuites.map((s) => [s.id, s.cases.length]),
|
||||
[
|
||||
["golden-set", 10],
|
||||
["coding-proficiency", 5],
|
||||
["reasoning-logic", 5],
|
||||
["multilingual", 5],
|
||||
["safety-guardrails", 6],
|
||||
["instruction-following", 5],
|
||||
["codex-comparison", 8],
|
||||
]
|
||||
);
|
||||
});
|
||||
|
||||
it("golden-set first case is the contains-hello greeting", () => {
|
||||
const gs01 = goldenSet.cases[0];
|
||||
assert.equal(gs01.id, "gs-01");
|
||||
assert.equal(gs01.name, "Simple greeting");
|
||||
assert.equal(gs01.model, "gpt-4o");
|
||||
assert.deepEqual(gs01.expected, { strategy: "contains", value: "hello" });
|
||||
});
|
||||
|
||||
it("every codex-comparison case targets the codex model", () => {
|
||||
assert.equal(codexComparisonSuite.id, "codex-comparison");
|
||||
assert.equal(codexComparisonSuite.cases.length, 8);
|
||||
for (const c of codexComparisonSuite.cases) {
|
||||
assert.equal(c.model, "codex");
|
||||
}
|
||||
});
|
||||
|
||||
it("every safety-guardrails case carries the safety tag", () => {
|
||||
assert.equal(safetySuite.cases.length, 6);
|
||||
for (const c of safetySuite.cases) {
|
||||
assert.ok(
|
||||
Array.isArray(c.tags) && c.tags.includes("safety"),
|
||||
`case ${c.id} needs safety tag`
|
||||
);
|
||||
}
|
||||
});
|
||||
|
||||
it("host registers every leaf suite at module load", () => {
|
||||
resetSuites();
|
||||
for (const s of builtInSuites) {
|
||||
const registered = getSuite(s.id);
|
||||
assert.ok(registered, `suite ${s.id} should be registered on the host`);
|
||||
assert.equal(registered.id, s.id);
|
||||
assert.equal(registered.name, s.name);
|
||||
assert.equal(registered.cases.length, s.cases.length);
|
||||
}
|
||||
});
|
||||
});
|
||||
@@ -112,10 +112,7 @@ test("DefaultExecutor.buildUrl honors a custom providerSpecificData.baseUrl for
|
||||
const openai = new DefaultExecutor("openai");
|
||||
|
||||
// No override → hardcoded OpenAI endpoint (unchanged behavior).
|
||||
assert.equal(
|
||||
openai.buildUrl("gpt-4o", true),
|
||||
"https://api.openai.com/v1/chat/completions"
|
||||
);
|
||||
assert.equal(openai.buildUrl("gpt-4o", true), "https://api.openai.com/v1/chat/completions");
|
||||
|
||||
// Custom base URL (e.g. a proxy/gateway) must be used instead of api.openai.com.
|
||||
assert.equal(
|
||||
@@ -170,6 +167,15 @@ test("DefaultExecutor.buildUrl handles openai-compatible and anthropic-compatibl
|
||||
}),
|
||||
"https://proxy.example/v1/responses"
|
||||
);
|
||||
assert.equal(
|
||||
openAICompat.buildUrl("gpt-4.1", true, 0, {
|
||||
providerSpecificData: {
|
||||
baseUrl: "https://proxy.example/v1/",
|
||||
_omnirouteForceResponsesUpstream: true,
|
||||
},
|
||||
}),
|
||||
"https://proxy.example/v1/responses"
|
||||
);
|
||||
assert.equal(
|
||||
openAILegacyResponsesCompat.buildUrl("gpt-5.4", true, 0, {
|
||||
providerSpecificData: {
|
||||
@@ -871,6 +877,46 @@ test("DefaultExecutor.transformRequest only injects stream usage for OpenAI chat
|
||||
assert.equal((responsesResult as any).stream_options, undefined);
|
||||
});
|
||||
|
||||
test("DefaultExecutor.execute routes Responses-shaped MCP requests to /responses for OpenAI-compatible providers", async () => {
|
||||
const originalFetch = globalThis.fetch;
|
||||
const calls: Array<{ url: string; body: any }> = [];
|
||||
|
||||
globalThis.fetch = async (url, init = {}) => {
|
||||
calls.push({
|
||||
url: String(url),
|
||||
body: JSON.parse(String(init.body)),
|
||||
});
|
||||
return new Response(JSON.stringify({ ok: true }), {
|
||||
status: 200,
|
||||
headers: { "Content-Type": "application/json" },
|
||||
});
|
||||
};
|
||||
|
||||
try {
|
||||
const executor = new DefaultExecutor("openai-compatible-test");
|
||||
await executor.execute({
|
||||
model: "gpt-4.1",
|
||||
body: {
|
||||
model: "gpt-4.1",
|
||||
input: "find tools",
|
||||
tools: [{ type: "tool_search" }],
|
||||
},
|
||||
stream: false,
|
||||
credentials: {
|
||||
apiKey: "test-key",
|
||||
providerSpecificData: { baseUrl: "https://proxy.example/v1/" },
|
||||
},
|
||||
});
|
||||
} finally {
|
||||
globalThis.fetch = originalFetch;
|
||||
}
|
||||
|
||||
assert.equal(calls.length, 1);
|
||||
assert.equal(calls[0].url, "https://proxy.example/v1/responses");
|
||||
assert.equal(calls[0].body.stream_options, undefined);
|
||||
assert.deepEqual(calls[0].body.tools, [{ type: "tool_search" }]);
|
||||
});
|
||||
|
||||
test("DefaultExecutor.transformRequest respects disableStreamOptions for OpenAI chat targets", () => {
|
||||
const openAICompat = new DefaultExecutor("openai-compatible-test");
|
||||
const chatBody = { model: "gpt-4.1", messages: [{ role: "user", content: "hi" }] };
|
||||
|
||||
119
tests/unit/executor-github-prefill-sanitize.test.ts
Normal file
119
tests/unit/executor-github-prefill-sanitize.test.ts
Normal file
@@ -0,0 +1,119 @@
|
||||
import test from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
|
||||
import { GithubExecutor } from "../../open-sse/executors/github.ts";
|
||||
|
||||
// GitHub Copilot's /chat/completions endpoint rejects a conversation that ends with an
|
||||
// assistant message: "This model does not support assistant message prefill. The
|
||||
// conversation must end with a user message." Anthropic clients (e.g. newest Claude
|
||||
// Desktop) send a trailing assistant turn as a prefill seed — the Anthropic API honors
|
||||
// it, but Copilot 400s. GithubExecutor.dropTrailingAssistantPrefill() strips the
|
||||
// trailing assistant message(s) before dispatch, scoped to the github executor only.
|
||||
// Port of 9router#2143 (author: Manuel <baslr@users.noreply.github.com>).
|
||||
|
||||
test("dropTrailingAssistantPrefill drops a single trailing assistant message", () => {
|
||||
const executor = new GithubExecutor();
|
||||
const messages = [
|
||||
{ role: "user", content: "Hi" },
|
||||
{ role: "assistant", content: "Here is the answer:" },
|
||||
];
|
||||
|
||||
const out = executor.dropTrailingAssistantPrefill(messages);
|
||||
|
||||
assert.equal(out.length, 1);
|
||||
assert.equal(out[0].role, "user");
|
||||
});
|
||||
|
||||
test("dropTrailingAssistantPrefill drops multiple consecutive trailing assistant messages", () => {
|
||||
const executor = new GithubExecutor();
|
||||
const messages = [
|
||||
{ role: "user", content: "Hi" },
|
||||
{ role: "assistant", content: "one" },
|
||||
{ role: "assistant", content: "two" },
|
||||
];
|
||||
|
||||
const out = executor.dropTrailingAssistantPrefill(messages);
|
||||
|
||||
assert.equal(out.length, 1);
|
||||
assert.equal(out[0].role, "user");
|
||||
});
|
||||
|
||||
test("dropTrailingAssistantPrefill is a no-op (same reference) when the conversation ends with a user message", () => {
|
||||
const executor = new GithubExecutor();
|
||||
const messages = [
|
||||
{ role: "user", content: "Hi" },
|
||||
{ role: "assistant", content: "Hello" },
|
||||
{ role: "user", content: "More" },
|
||||
];
|
||||
|
||||
const out = executor.dropTrailingAssistantPrefill(messages);
|
||||
|
||||
assert.equal(out, messages, "must return the same array reference when nothing changes");
|
||||
assert.equal(out.length, 3);
|
||||
});
|
||||
|
||||
test("dropTrailingAssistantPrefill is a no-op when the conversation ends with a tool message", () => {
|
||||
const executor = new GithubExecutor();
|
||||
const messages = [
|
||||
{ role: "user", content: "Hi" },
|
||||
{ role: "assistant", content: null, tool_calls: [{ id: "x" }] },
|
||||
{ role: "tool", tool_call_id: "x", content: "result" },
|
||||
];
|
||||
|
||||
const out = executor.dropTrailingAssistantPrefill(messages);
|
||||
|
||||
assert.equal(out, messages, "must return the same array reference when nothing changes");
|
||||
assert.equal(out.length, 3);
|
||||
assert.equal(out[2].role, "tool");
|
||||
});
|
||||
|
||||
test("dropTrailingAssistantPrefill never empties an assistant-only conversation", () => {
|
||||
const executor = new GithubExecutor();
|
||||
const messages = [{ role: "assistant", content: "only" }];
|
||||
|
||||
const out = executor.dropTrailingAssistantPrefill(messages);
|
||||
|
||||
assert.equal(out.length, 1, "must keep at least one message");
|
||||
assert.equal(out[0].role, "assistant");
|
||||
});
|
||||
|
||||
test("dropTrailingAssistantPrefill is null/empty safe", () => {
|
||||
const executor = new GithubExecutor();
|
||||
|
||||
assert.deepEqual(executor.dropTrailingAssistantPrefill([]), []);
|
||||
assert.equal(executor.dropTrailingAssistantPrefill(undefined), undefined);
|
||||
assert.equal(executor.dropTrailingAssistantPrefill(null), null);
|
||||
});
|
||||
|
||||
test("GithubExecutor.transformRequest drops the trailing assistant prefill end-to-end", () => {
|
||||
const executor = new GithubExecutor();
|
||||
const body = {
|
||||
model: "claude-sonnet-4.6",
|
||||
messages: [
|
||||
{ role: "user", content: "Hi" },
|
||||
{ role: "assistant", content: "Here is the answer:" },
|
||||
],
|
||||
};
|
||||
|
||||
const out = executor.transformRequest("claude-sonnet-4.6", body, false, {});
|
||||
|
||||
assert.equal(out.messages.length, 1);
|
||||
assert.equal(out.messages[0].role, "user");
|
||||
});
|
||||
|
||||
test("GithubExecutor.transformRequest leaves a user-terminated conversation untouched end-to-end", () => {
|
||||
const executor = new GithubExecutor();
|
||||
const body = {
|
||||
model: "claude-sonnet-4.6",
|
||||
messages: [
|
||||
{ role: "user", content: "Hi" },
|
||||
{ role: "assistant", content: "Hello" },
|
||||
{ role: "user", content: "More" },
|
||||
],
|
||||
};
|
||||
|
||||
const out = executor.transformRequest("claude-sonnet-4.6", body, false, {});
|
||||
|
||||
assert.equal(out.messages.length, 3);
|
||||
assert.equal(out.messages[2].role, "user");
|
||||
});
|
||||
@@ -112,6 +112,11 @@ test("GithubExecutor.transformRequest injects JSON response instructions for Cla
|
||||
reasoning_text: "internal",
|
||||
reasoning_content: "internal",
|
||||
},
|
||||
// Trailing user turn: dropTrailingAssistantPrefill (9router#2143) strips a
|
||||
// conversation that ends in "assistant", which would otherwise remove the very
|
||||
// message this test inspects below. Keep the array ending in "user" so this test
|
||||
// stays focused on response_format injection + reasoning-field stripping.
|
||||
{ role: "user", content: "thanks" },
|
||||
],
|
||||
};
|
||||
|
||||
@@ -225,6 +230,11 @@ test("GithubExecutor.transformRequest leaves string content and missing content
|
||||
role: "assistant",
|
||||
tool_calls: [{ id: "c1", type: "function", function: { name: "f", arguments: "{}" } }],
|
||||
},
|
||||
// Trailing tool response: dropTrailingAssistantPrefill (9router#2143) strips a
|
||||
// conversation that ends in "assistant", which would otherwise remove the very
|
||||
// tool_calls message this test inspects below. A real tool round-trip ends in
|
||||
// "tool", not "assistant" — model that shape instead.
|
||||
{ role: "tool", tool_call_id: "c1", content: "result" },
|
||||
],
|
||||
};
|
||||
const result = executor.transformRequest("claude-sonnet-4.6", body, true, {});
|
||||
|
||||
@@ -62,4 +62,46 @@ describe("HuggingChatExecutor", () => {
|
||||
assert.ok(typeof result.url === "string");
|
||||
assert.ok(typeof result.headers === "object");
|
||||
});
|
||||
|
||||
// PR #5592: after a conversation is created, the executor GETs
|
||||
// /chat/api/v2/conversations/{id} to obtain the root parent message id.
|
||||
// When that GET fails or returns malformed data, fetchInitialParentMessageId
|
||||
// returns null and the executor must surface a 502 instead of proceeding with
|
||||
// an undefined parent id. This defensive path was previously untested.
|
||||
it("returns 502 when the initial parent message id cannot be fetched", async () => {
|
||||
const realFetch = globalThis.fetch;
|
||||
globalThis.fetch = (async (_input: RequestInfo | URL, init?: RequestInit) => {
|
||||
const method = (init?.method || "GET").toUpperCase();
|
||||
if (method === "POST") {
|
||||
// Step 1: conversation creation succeeds.
|
||||
return new Response(JSON.stringify({ conversationId: "conv-test-123" }), {
|
||||
status: 200,
|
||||
headers: { "Content-Type": "application/json" },
|
||||
});
|
||||
}
|
||||
// Step 2: the parent-message GET fails -> fetchInitialParentMessageId -> null.
|
||||
return new Response("", { status: 500 });
|
||||
}) as typeof globalThis.fetch;
|
||||
|
||||
try {
|
||||
const executor = new mod.HuggingChatExecutor();
|
||||
const result = await executor.execute({
|
||||
model: "meta-llama/Llama-4-Scout-17B-16E-Instruct",
|
||||
body: { messages: [{ role: "user", content: "hi" }] },
|
||||
stream: false,
|
||||
credentials: { apiKey: "hf-chat=fake-cookie" },
|
||||
signal: null,
|
||||
});
|
||||
assert.equal(result.response.status, 502);
|
||||
const json = await result.response.json();
|
||||
assert.ok(
|
||||
json.error.message.includes("initial parent message id"),
|
||||
`expected the parent-message 502 message, got: ${json.error.message}`
|
||||
);
|
||||
// Rule #12 sanity: the error body carries a static message, never a stack frame.
|
||||
assert.ok(!json.error.message.includes("at /"));
|
||||
} finally {
|
||||
globalThis.fetch = realFetch;
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
239
tests/unit/executor-kimi-web-decoder.test.ts
Normal file
239
tests/unit/executor-kimi-web-decoder.test.ts
Normal file
@@ -0,0 +1,239 @@
|
||||
// Tests for the Connect frame decoder and event-delta extractor that back
|
||||
// the international Kimi web executor (www.kimi.com Connect-RPC API).
|
||||
//
|
||||
// These tests pin the wire-format parsing that the executor relies on —
|
||||
// the riskiest piece of the migration per code review (PR #5858, I3).
|
||||
|
||||
import { describe, it } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
|
||||
const {
|
||||
frameConnectMessage,
|
||||
decodeConnectFrame,
|
||||
extractDelta,
|
||||
isEndOfStream,
|
||||
foldMessages,
|
||||
} = await import("../../open-sse/executors/kimi-web.ts");
|
||||
|
||||
describe("frameConnectMessage + decodeConnectMessage round-trip", () => {
|
||||
it("round-trips a JSON payload through frame and decode", () => {
|
||||
const json = '{"hello":"world"}';
|
||||
const framed = frameConnectMessage(json);
|
||||
assert.equal(framed.length, 5 + json.length);
|
||||
// First byte is flags = 0 (uncompressed).
|
||||
assert.equal(framed[0], 0);
|
||||
// Bytes 1-4 are big-endian length.
|
||||
const len = (framed[1] << 24) | (framed[2] << 16) | (framed[3] << 8) | framed[4];
|
||||
assert.equal(len, json.length);
|
||||
|
||||
const { consumed, frame } = decodeConnectFrame(framed, 0);
|
||||
assert.equal(consumed, framed.length);
|
||||
assert.equal(frame?.flags, 0);
|
||||
assert.deepEqual(frame?.message, { hello: "world" });
|
||||
});
|
||||
|
||||
it("returns consumed=0 when the buffer has fewer than 5 bytes (need more)", () => {
|
||||
const short = new Uint8Array([0x00, 0x00, 0x00]);
|
||||
const { consumed, frame } = decodeConnectFrame(short, 0);
|
||||
assert.equal(consumed, 0);
|
||||
assert.equal(frame, null);
|
||||
});
|
||||
|
||||
it("returns consumed=0 when the buffer has header but not enough payload yet", () => {
|
||||
// Header claims 100 bytes of payload, but we only have 5 header + 10 payload.
|
||||
const partial = new Uint8Array(15);
|
||||
partial[0] = 0;
|
||||
partial[1] = 0;
|
||||
partial[2] = 0;
|
||||
partial[3] = 0;
|
||||
partial[4] = 100;
|
||||
const { consumed, frame } = decodeConnectFrame(partial, 0);
|
||||
assert.equal(consumed, 0);
|
||||
assert.equal(frame, null);
|
||||
});
|
||||
|
||||
it("consumes the first frame and leaves the rest in the buffer for the next call", () => {
|
||||
const a = frameConnectMessage('{"a":1}');
|
||||
const b = frameConnectMessage('{"b":2}');
|
||||
const merged = new Uint8Array(a.length + b.length);
|
||||
merged.set(a, 0);
|
||||
merged.set(b, a.length);
|
||||
|
||||
const first = decodeConnectFrame(merged, 0);
|
||||
assert.equal(first.consumed, a.length);
|
||||
assert.deepEqual(first.frame?.message, { a: 1 });
|
||||
|
||||
const second = decodeConnectFrame(merged, first.consumed);
|
||||
assert.equal(second.consumed, b.length);
|
||||
assert.deepEqual(second.frame?.message, { b: 2 });
|
||||
});
|
||||
|
||||
it("decodes a frame whose length has the high bit (bit 31) set without sign issues", () => {
|
||||
// Construct a header claiming length 2,147,483,648 (0x80000000) — the
|
||||
// signed-shift bug would read this as -2147483648. With the decoder's
|
||||
// correction it should be treated as MAX_FRAME_LEN+1 and consumed=-1.
|
||||
const oversized = new Uint8Array(5);
|
||||
oversized[0] = 0;
|
||||
oversized[1] = 0x80;
|
||||
oversized[2] = 0x00;
|
||||
oversized[3] = 0x00;
|
||||
oversized[4] = 0x00;
|
||||
const { consumed } = decodeConnectFrame(oversized, 0);
|
||||
assert.equal(consumed, -1, "frames above MAX_FRAME_LEN must signal -1");
|
||||
});
|
||||
|
||||
it("returns a null message (still consumed) when payload is not valid JSON", () => {
|
||||
const bad = new Uint8Array(5 + 3);
|
||||
bad[0] = 0;
|
||||
bad[4] = 3;
|
||||
bad[5] = 0x7b; // {
|
||||
bad[6] = 0x7d; // }
|
||||
bad[7] = 0x2c; // , (trailing — invalid JSON)
|
||||
const { consumed, frame } = decodeConnectFrame(bad, 0);
|
||||
assert.equal(consumed, 8);
|
||||
assert.equal(frame?.message, null);
|
||||
assert.equal(frame?.flags, 0);
|
||||
});
|
||||
});
|
||||
|
||||
describe("extractDelta", () => {
|
||||
it("returns null on null/empty input", () => {
|
||||
assert.equal(extractDelta(null), null);
|
||||
});
|
||||
|
||||
it("returns null on heartbeats and unrelated events", () => {
|
||||
assert.equal(extractDelta({ heartbeat: {} }), null);
|
||||
assert.equal(extractDelta({ op: "set", mask: "chat.name" }), null);
|
||||
assert.equal(extractDelta({ op: "set", mask: "block.stage" }), null);
|
||||
});
|
||||
|
||||
it("extracts initial answer text from op=set, mask=block.text", () => {
|
||||
const delta = extractDelta({
|
||||
op: "set",
|
||||
mask: "block.text",
|
||||
block: { text: { content: "Hello" } },
|
||||
});
|
||||
assert.deepEqual(delta, { kind: "text", text: "Hello" });
|
||||
});
|
||||
|
||||
it("extracts answer delta from op=append, mask=block.text.content", () => {
|
||||
const delta = extractDelta({
|
||||
op: "append",
|
||||
mask: "block.text.content",
|
||||
block: { text: { content: " world" } },
|
||||
});
|
||||
assert.deepEqual(delta, { kind: "text", text: " world" });
|
||||
});
|
||||
|
||||
it("extracts initial reasoning from op=set, mask=block.think", () => {
|
||||
const delta = extractDelta({
|
||||
op: "set",
|
||||
mask: "block.think",
|
||||
block: { think: { content: "Reasoning..." } },
|
||||
});
|
||||
assert.deepEqual(delta, { kind: "think", text: "Reasoning..." });
|
||||
});
|
||||
|
||||
it("extracts reasoning delta from op=append, mask=block.think.content", () => {
|
||||
const delta = extractDelta({
|
||||
op: "append",
|
||||
mask: "block.think.content",
|
||||
block: { think: { content: " continued" } },
|
||||
});
|
||||
assert.deepEqual(delta, { kind: "think", text: " continued" });
|
||||
});
|
||||
|
||||
it("returns null when content is empty (no useful delta)", () => {
|
||||
assert.equal(
|
||||
extractDelta({ op: "set", mask: "block.text", block: { text: { content: "" } } }),
|
||||
null
|
||||
);
|
||||
assert.equal(
|
||||
extractDelta({ op: "append", mask: "block.text.content", block: { text: {} } }),
|
||||
null
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
describe("isEndOfStream", () => {
|
||||
it("returns true when assistant message flips to MESSAGE_STATUS_COMPLETED", () => {
|
||||
assert.equal(
|
||||
isEndOfStream({
|
||||
op: "set",
|
||||
mask: "message",
|
||||
message: { role: "assistant", status: "MESSAGE_STATUS_COMPLETED" },
|
||||
}),
|
||||
true
|
||||
);
|
||||
});
|
||||
|
||||
it("returns false for non-assistant completed messages (system/user)", () => {
|
||||
assert.equal(
|
||||
isEndOfStream({
|
||||
op: "set",
|
||||
mask: "message",
|
||||
message: { role: "user", status: "MESSAGE_STATUS_COMPLETED" },
|
||||
}),
|
||||
false
|
||||
);
|
||||
});
|
||||
|
||||
it("returns false for assistant messages that are still generating", () => {
|
||||
assert.equal(
|
||||
isEndOfStream({
|
||||
op: "set",
|
||||
mask: "message",
|
||||
message: { role: "assistant", status: "MESSAGE_STATUS_GENERATING" },
|
||||
}),
|
||||
false
|
||||
);
|
||||
});
|
||||
|
||||
it("returns false for non-message events", () => {
|
||||
assert.equal(isEndOfStream({ heartbeat: {} }), false);
|
||||
assert.equal(isEndOfStream(null), false);
|
||||
});
|
||||
});
|
||||
|
||||
describe("foldMessages", () => {
|
||||
it("returns empty string for empty input", () => {
|
||||
assert.equal(foldMessages([]), "");
|
||||
});
|
||||
|
||||
it("returns user content as-is when only a user message is present", () => {
|
||||
assert.equal(foldMessages([{ role: "user", content: "hi" }]), "hi");
|
||||
});
|
||||
|
||||
it("prepends system content to user content", () => {
|
||||
const out = foldMessages([
|
||||
{ role: "system", content: "Be terse." },
|
||||
{ role: "user", content: "hi" },
|
||||
]);
|
||||
assert.equal(out, "Be terse.\n\nhi");
|
||||
});
|
||||
|
||||
it("labels assistant turns and concatenates with prior user content", () => {
|
||||
const out = foldMessages([
|
||||
{ role: "user", content: "q1" },
|
||||
{ role: "assistant", content: "a1" },
|
||||
{ role: "user", content: "q2" },
|
||||
]);
|
||||
assert.equal(out, "q1\n\nAssistant: a1\n\nq2");
|
||||
});
|
||||
|
||||
it("stringifies non-string content (arrays/objects) instead of dropping it", () => {
|
||||
const out = foldMessages([{ role: "user", content: [{ type: "text", text: "x" }] }]);
|
||||
assert.ok(out.includes("text"));
|
||||
assert.ok(out.includes("x"));
|
||||
});
|
||||
|
||||
it("silently drops tool/function messages (limitation: kimi-web is single-turn)", () => {
|
||||
const out = foldMessages([
|
||||
{ role: "user", content: "hi" },
|
||||
{ role: "tool", content: "result" },
|
||||
{ role: "function", content: "fn-result" },
|
||||
]);
|
||||
// Tool/function messages contribute nothing; user content survives.
|
||||
assert.equal(out, "hi");
|
||||
});
|
||||
});
|
||||
@@ -1,3 +1,10 @@
|
||||
// Tests for the international Kimi web executor (www.kimi.com Connect-RPC API).
|
||||
//
|
||||
// Previously this provider targeted kimi.moonshot.cn; that domain now redirects
|
||||
// every non-CN visitor to www.kimi.com, which uses a Connect-RPC streaming API.
|
||||
// These tests pin the parser behavior of the Connect envelope framing and the
|
||||
// JSON event-delta extractor.
|
||||
|
||||
import { describe, it } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
|
||||
@@ -9,20 +16,77 @@ describe("KimiWebExecutor", () => {
|
||||
assert.ok(executor);
|
||||
});
|
||||
|
||||
it("execute returns error on fetch failure", async () => {
|
||||
it("execute returns a 400 error when no JWT is provided", async () => {
|
||||
const executor = new mod.KimiWebExecutor();
|
||||
const result = await executor.execute({
|
||||
model: "kimi-default",
|
||||
body: { messages: [{ role: "user", content: "hi" }] },
|
||||
stream: false,
|
||||
credentials: { apiKey: "" },
|
||||
signal: null,
|
||||
} as never);
|
||||
assert.equal(result.response.status, 400);
|
||||
const body = (await result.response.json()) as { error: { code: string } };
|
||||
assert.match(body.error.code, /HTTP_400|400/);
|
||||
});
|
||||
|
||||
it("execute targets www.kimi.com (not kimi.moonshot.cn)", async () => {
|
||||
const executor = new mod.KimiWebExecutor();
|
||||
let capturedUrl = "";
|
||||
const originalFetch = globalThis.fetch;
|
||||
try {
|
||||
const result = await executor.execute({
|
||||
globalThis.fetch = (async (url: any) => {
|
||||
capturedUrl = String(url);
|
||||
return new Response(new ReadableStream({ start: (c) => c.close() }), {
|
||||
status: 200,
|
||||
headers: { "content-type": "application/connect+json" },
|
||||
});
|
||||
}) as typeof fetch;
|
||||
await executor.execute({
|
||||
model: "kimi-default",
|
||||
body: { messages: [{ role: "user", content: "hi" }] },
|
||||
stream: false,
|
||||
credentials: { apiKey: "" },
|
||||
credentials: { apiKey: "kimi-auth=fake.jwt.token" },
|
||||
signal: null,
|
||||
});
|
||||
assert.ok(result.response instanceof Response);
|
||||
assert.ok(result.url.includes("kimi.moonshot.cn"));
|
||||
} catch {
|
||||
// Network error expected
|
||||
} as never);
|
||||
assert.ok(capturedUrl.startsWith("https://www.kimi.com/"), `got ${capturedUrl}`);
|
||||
assert.ok(!capturedUrl.includes("moonshot.cn"));
|
||||
} finally {
|
||||
globalThis.fetch = originalFetch;
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe("extractKimiJwt", () => {
|
||||
const { extractKimiJwt } = mod;
|
||||
|
||||
it("returns empty string for empty input", () => {
|
||||
assert.equal(extractKimiJwt(""), "");
|
||||
assert.equal(extractKimiJwt(" "), "");
|
||||
});
|
||||
|
||||
it("extracts a bare JWT", () => {
|
||||
const jwt = "eyJhbGci.eyJzdWIi.c2ln";
|
||||
assert.equal(extractKimiJwt(jwt), jwt);
|
||||
});
|
||||
|
||||
it("extracts kimi-auth from a full Cookie header", () => {
|
||||
const jwt = "eyJhbGciOiJIUzUxMiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiJ4In0.signature";
|
||||
const pasted = `_ga=GA1.1.x; theme=dark; kimi-auth=${jwt}; _gcl_au=1.1.x; lang=en-US`;
|
||||
assert.equal(extractKimiJwt(pasted), jwt);
|
||||
});
|
||||
|
||||
it("strips a leading Cookie: header label", () => {
|
||||
const jwt = "eyJhbGci.eyJzdWIi.c2ln";
|
||||
assert.equal(extractKimiJwt(`Cookie: kimi-auth=${jwt}`), jwt);
|
||||
});
|
||||
|
||||
it("strips a leading Authorization: Bearer label", () => {
|
||||
const jwt = "eyJhbGci.eyJzdWIi.c2ln";
|
||||
assert.equal(extractKimiJwt(`Authorization: Bearer ${jwt}`), jwt);
|
||||
});
|
||||
|
||||
it("returns empty when no JWT is present", () => {
|
||||
assert.equal(extractKimiJwt("foo=bar; baz=qux"), "");
|
||||
});
|
||||
});
|
||||
|
||||
@@ -154,6 +154,33 @@ describe("QwenWebExecutor (v2 migration)", () => {
|
||||
assert.equal(headers.source || headers.Source, "web", "source: web header present");
|
||||
});
|
||||
|
||||
it("sends the Qwen SPA build 'version' header on the v2 chat completion request", async () => {
|
||||
globalThis.fetch = (async (url: any, init: any = {}) => {
|
||||
calls.push({ url: String(url), init });
|
||||
if (String(url).includes("/api/v2/chats/new")) return chatCreatedResponse();
|
||||
return sseResponse([
|
||||
{ choices: [{ delta: { phase: "answer", content: "ok", status: "finished" } }] },
|
||||
]);
|
||||
}) as any;
|
||||
|
||||
const executor = new mod.QwenWebExecutor();
|
||||
await executor.execute({
|
||||
model: "qwen3.7-plus",
|
||||
body: { messages: [{ role: "user", content: "hi" }] },
|
||||
stream: false,
|
||||
credentials: { apiKey: "token=t; cna=c" },
|
||||
signal: null,
|
||||
} as any);
|
||||
|
||||
// Without the `version` header the v2 endpoint short-circuits with a
|
||||
// Bad_Request envelope before ever reaching the model router — see
|
||||
// open-sse/executors/qwen-web.ts::QWEN_SPA_VERSION.
|
||||
const completionCall = calls.find((call) => call.url.includes("/api/v2/chat/completions"));
|
||||
assert.ok(completionCall, "chat/completions call must have been made");
|
||||
const headers = completionCall!.init.headers as Record<string, string>;
|
||||
assert.equal(headers.version, "0.2.66", "SPA build version header present");
|
||||
});
|
||||
|
||||
it("maps the thinking phase to reasoning_content, not the answer content", async () => {
|
||||
globalThis.fetch = (async (url: any) => {
|
||||
if (String(url).includes("/api/v2/chats/new")) return chatCreatedResponse();
|
||||
|
||||
@@ -30,17 +30,19 @@ const {
|
||||
isControlPlaneProxyDirectFallbackEnabled,
|
||||
} = await import("../../src/shared/utils/featureFlags.ts");
|
||||
|
||||
const EXPECTED_FEATURE_FLAG_COUNT = 40;
|
||||
|
||||
// ──────────────────────────────────────────────────────
|
||||
// Test group 1 — Flag definitions registry
|
||||
// ──────────────────────────────────────────────────────
|
||||
describe("featureFlagDefinitions", () => {
|
||||
it("has exactly 38 flag definitions", () => {
|
||||
assert.strictEqual(FEATURE_FLAG_DEFINITIONS.length, 38);
|
||||
it("has exactly 40 flag definitions", () => {
|
||||
assert.strictEqual(FEATURE_FLAG_DEFINITIONS.length, EXPECTED_FEATURE_FLAG_COUNT);
|
||||
});
|
||||
|
||||
it("has unique keys for all flags", () => {
|
||||
const keys = FEATURE_FLAG_DEFINITIONS.map((d) => d.key);
|
||||
assert.strictEqual(new Set(keys).size, 38);
|
||||
assert.strictEqual(new Set(keys).size, EXPECTED_FEATURE_FLAG_COUNT);
|
||||
});
|
||||
|
||||
it("has valid categories for all flags", () => {
|
||||
@@ -158,6 +160,21 @@ describe("featureFlagDefinitions", () => {
|
||||
assert.strictEqual(def.requiresRestart, false);
|
||||
assert.strictEqual(def.warningLevel, "danger");
|
||||
});
|
||||
|
||||
it("defines CLI profile auto-sync flags as CLI booleans disabled by default", () => {
|
||||
for (const key of [
|
||||
"OMNIROUTE_AUTO_SYNC_CODEX_PROFILES",
|
||||
"OMNIROUTE_AUTO_SYNC_CLAUDE_PROFILES",
|
||||
]) {
|
||||
const def = FEATURE_FLAG_DEFINITIONS.find((d) => d.key === key);
|
||||
assert.ok(def, `${key} should exist`);
|
||||
assert.strictEqual(def.category, "cli");
|
||||
assert.strictEqual(def.type, "boolean");
|
||||
assert.strictEqual(def.defaultValue, "false");
|
||||
assert.strictEqual(def.requiresRestart, false);
|
||||
assert.strictEqual(def.warningLevel, "caution");
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
// ──────────────────────────────────────────────────────
|
||||
@@ -295,9 +312,9 @@ describe("resolveFeatureFlag", () => {
|
||||
});
|
||||
|
||||
describe("resolveAllFeatureFlags", () => {
|
||||
it("returns all 38 flags", () => {
|
||||
it("returns all 40 flags", () => {
|
||||
const all = resolveAllFeatureFlags();
|
||||
assert.strictEqual(all.length, 38);
|
||||
assert.strictEqual(all.length, EXPECTED_FEATURE_FLAG_COUNT);
|
||||
});
|
||||
|
||||
it("marks DB-overridden flags with source 'db'", () => {
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user