Files
OmniRoute/tests/unit/chaos-config.test.ts
Moseyuh333 eeec4d9e87 feat(chaos): big update - optimize, fix bugs, add features, enhance UX (#6728)
* feat(chaos): add Chaos Mode — multi-model parallel/collaborative execution

- New DB column chaos_mode_enabled on api_keys table
- API key create/PATCH routes support chaosModeEnabled toggle
- Core library src/lib/chaos/chaosConfig.ts for persistent config
- API routes: GET/PUT/DELETE /api/chaos/config
- Chaos execution POST /api/skills/collect/chaos with key auth
- Dashboard page at /dashboard/chaos with full config UI
- Sidebar entry in Agentic Features section
- Chaos mode toggle in API Key editor permissions panel
- i18n keys for chaos config (en.json)

* feat(chaos): big update — optimize, fix bugs, add features

=== Changes ===

1. NEW: src/lib/chaos/chaosExecutor.ts — shared execution engine
   - Removed ~150 lines of duplicate dispatch logic between two API routes
   - Single executeChaosRun() function used by both endpoints
   - Added concurrency limit (max 10 parallel requests)
   - Added proper TypeScript interfaces (ChaosRunInput, ChaosRunResult)
   - Added error logging throughout

2. FIX: src/app/api/skills/collect/chaos/route.ts
   - Was MISSING logger import (log.error was undefined at runtime)
   - Reduced from 388 lines → 142 lines by delegating to shared executor
   - Added maxTokens support in schema validation

3. REFACTOR: src/app/api/chaos/run/route.ts
   - Simplified to thin wrapper: auth + validate + delegate to executor
   - Added maxTokens support

4. ENHANCE: src/lib/chaos/chaosConfig.ts
   - Added maxTokens config field (256-128k, default 4096)
   - Persisted per-instance via settings table

5. ENHANCE: UI — ChaosConfigPageClient.tsx
   - Loads available providers from /api/models for dropdown autocomplete
   - Added datalist-based provider selector in overrides section
   - Added Max Tokens configuration input
   - Added expandable provider list showing all detected providers
   - Fixed duplicate override detection

* fix(chaos): fetch providers from /api/providers instead of /api/keys

* fix(chaos): remove dead code isOverrideDuplicate, fix maxTokens fallback to include global config

* fix(chaos): resetConfig now shows error on HTTP failure (was silent)

* feat(dashboard): Chaos Mode — multi-model parallel/collaborative execution

Splits the PR down to only the genuinely new Chaos Mode feature (drops the
duplicate Skill Collector/GitHub-discovery portion already shipped via
#6186). Replaces the loopback fetch() dispatch (hardcoded to the wrong port)
with the established in-process synthetic-Request/route-handler pattern used
by src/lib/batches/dispatch.ts, moves settings persistence off raw SQL, and
adds unit test coverage for chaosConfig, chaosExecutor and the 3 chaos API
routes.

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>

* fix(chaos): fix external Bearer-auth bypass and stale config cache in tests

validateApiKey() returns a plain boolean for both the deployment-time env key
and a DB-backed key, so branching on `keyInfo === true` in
verifyChaosKey() (src/app/api/skills/collect/chaos/route.ts) treated every
valid API key as having full env-key access, silently skipping the
chaosModeEnabled permission check entirely. Now always resolves through
getApiKeyMetadata() and only bypasses the per-key check for the synthesized
env-key record (id: "env-key").

Also exports invalidateChaosConfigCache() from chaosConfig.ts and wires it
into the route tests' resetStorage() — the in-process config cache was
surviving DB resets between tests, causing state to leak across cases.

Fixes CHANGELOG-eat from the release merge (re-inserted the Chaos Mode
bullet against the base CHANGELOG.md, verified additive via
check-changelog-integrity.mjs) and re-syncs against release/v3.8.47 tip.

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>

* docs(changelog): Chaos Mode overhaul bullet referencing #6728 after release sync

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>

* fix(merge): restore #6126 clinepass files reverted by release auto-resolve + baseline re-merge

The release sync's auto-resolve reverted sibling PR #6126's clinepass work
(registry, catalog, oauth constants, clineAuth.ts, token-refresh case, tests)
and the file-size baseline — all outside this PR's scope. Restored to the
release versions, re-applied only this PR's own baseline entries, restored the
#6126 CHANGELOG bullet (re-inserting only this PR's own).

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>

* fix(dashboard): chaos client hook must not import the server Pino logger

useChaosConfigData ("use client") pulled @/sse/utils/logger → shared Pino →
logRotation/dataPaths → node:fs into the browser bundle, breaking next build
(Turbopack: Can't resolve 'fs') — caught by the DAST smoke's isolated build.
console.error matches every other dashboard client component.

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>

* test(api-manager): align switch-count invariant with the extracted toggle components

The Self-service block now renders 4 inline switches; the #5731 quota-bypass
and #6728 chaos-access toggles were extracted into dedicated components. The
type="button" invariant is preserved AND extended: the test now also asserts
each extracted component's switches declare type="button".

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>

* chore(sync): merge release tip + restore own CHANGELOG bullet

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>

---------

Co-authored-by: Moseyuh333 <Moseyuh333@users.noreply.github.com>
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
Co-authored-by: Diego Rodrigues de Sa e Souza <diegosouza.pw@gmail.com>
2026-07-09 23:49:48 -03:00

127 lines
4.4 KiB
TypeScript

/**
* Unit tests for src/lib/chaos/chaosConfig.ts
*
* Regression coverage for #6679: the original PR queried `SELECT/INSERT/DELETE
* ... FROM settings ...` against a table that was never created in this schema
* (only `key_value` exists) — every read silently fell back to
* DEFAULT_CHAOS_CONFIG and every write/reset threw at runtime. This module now
* routes persistence through src/lib/db/settings.ts::getSettings/updateSettings
* (the `key_value` table, namespace 'settings'), matching the repo convention
* (CLAUDE.md → Database: never write raw SQL outside src/lib/db/).
*/
import test from "node:test";
import assert from "node:assert/strict";
import fs from "node:fs";
import os from "node:os";
import path from "node:path";
const TEST_DATA_DIR = fs.mkdtempSync(path.join(os.tmpdir(), "omniroute-chaos-config-"));
const ORIGINAL_DATA_DIR = process.env.DATA_DIR;
process.env.DATA_DIR = TEST_DATA_DIR;
const core = await import("../../src/lib/db/core.ts");
const chaosConfig = await import("../../src/lib/chaos/chaosConfig.ts");
async function resetStorage() {
core.resetDbInstance();
fs.rmSync(TEST_DATA_DIR, { recursive: true, force: true });
fs.mkdirSync(TEST_DATA_DIR, { recursive: true });
}
test.beforeEach(async () => {
await resetStorage();
});
test.after(() => {
core.resetDbInstance();
fs.rmSync(TEST_DATA_DIR, { recursive: true, force: true });
if (ORIGINAL_DATA_DIR === undefined) {
delete process.env.DATA_DIR;
} else {
process.env.DATA_DIR = ORIGINAL_DATA_DIR;
}
});
test("getChaosConfig returns DEFAULT_CHAOS_CONFIG when nothing is persisted yet", async () => {
const config = await chaosConfig.getChaosConfig();
assert.deepEqual(config, chaosConfig.DEFAULT_CHAOS_CONFIG);
});
test("setChaosConfig persists via the settings store and getChaosConfig reads it back", async () => {
const settingsDb = await import("../../src/lib/db/settings.ts");
const written = await chaosConfig.setChaosConfig({
enabled: true,
defaultMode: "collaborative",
providerOverrides: [{ providerId: "openai", modelId: "gpt-4o", enabled: true }],
systemPrompt: "Custom prompt",
timeoutMs: 60_000,
maxTokens: 8192,
});
assert.equal(written.enabled, true);
assert.equal(written.defaultMode, "collaborative");
const readBack = await chaosConfig.getChaosConfig();
assert.deepEqual(readBack, written);
// Confirms the write actually lands in key_value (namespace 'settings'), not a
// nonexistent `settings` table (the bug this fix replaces).
const rawSettings = await settingsDb.getSettings();
assert.equal((rawSettings as Record<string, unknown>).chaosModeConfig !== undefined, true);
});
test("setChaosConfig rejects invalid config (Zod parse, not raw SQL failure)", async () => {
const invalidConfig = {
enabled: true,
defaultMode: "not-a-real-mode",
providerOverrides: [],
timeoutMs: 60_000,
maxTokens: 4096,
} as unknown as Awaited<ReturnType<typeof chaosConfig.getChaosConfig>>;
await assert.rejects(() => chaosConfig.setChaosConfig(invalidConfig));
});
test("resetChaosConfig clears the stored config back to defaults", async () => {
await chaosConfig.setChaosConfig({
enabled: true,
defaultMode: "parallel",
providerOverrides: [],
timeoutMs: 90_000,
maxTokens: 2048,
});
assert.equal((await chaosConfig.getChaosConfig()).enabled, true);
const reset = await chaosConfig.resetChaosConfig();
assert.deepEqual(reset, chaosConfig.DEFAULT_CHAOS_CONFIG);
assert.deepEqual(await chaosConfig.getChaosConfig(), chaosConfig.DEFAULT_CHAOS_CONFIG);
});
test("getChaosConfig falls back to defaults when the stored payload fails schema validation", async () => {
const settingsDb = await import("../../src/lib/db/settings.ts");
await settingsDb.updateSettings({
chaosModeConfig: { enabled: "not-a-boolean", defaultMode: "parallel" },
});
const config = await chaosConfig.getChaosConfig();
assert.deepEqual(config, chaosConfig.DEFAULT_CHAOS_CONFIG);
});
test("in-memory cache is invalidated on write so config updates are visible immediately", async () => {
const first = await chaosConfig.getChaosConfig();
assert.equal(first.enabled, false);
await chaosConfig.setChaosConfig({
enabled: true,
defaultMode: "parallel",
providerOverrides: [],
timeoutMs: 120_000,
maxTokens: 4096,
});
const second = await chaosConfig.getChaosConfig();
assert.equal(second.enabled, true);
});