feat(chaos): big update - optimize, fix bugs, add features, enhance UX (#6728)

* feat(chaos): add Chaos Mode — multi-model parallel/collaborative execution

- New DB column chaos_mode_enabled on api_keys table
- API key create/PATCH routes support chaosModeEnabled toggle
- Core library src/lib/chaos/chaosConfig.ts for persistent config
- API routes: GET/PUT/DELETE /api/chaos/config
- Chaos execution POST /api/skills/collect/chaos with key auth
- Dashboard page at /dashboard/chaos with full config UI
- Sidebar entry in Agentic Features section
- Chaos mode toggle in API Key editor permissions panel
- i18n keys for chaos config (en.json)

* feat(chaos): big update — optimize, fix bugs, add features

=== Changes ===

1. NEW: src/lib/chaos/chaosExecutor.ts — shared execution engine
   - Removed ~150 lines of duplicate dispatch logic between two API routes
   - Single executeChaosRun() function used by both endpoints
   - Added concurrency limit (max 10 parallel requests)
   - Added proper TypeScript interfaces (ChaosRunInput, ChaosRunResult)
   - Added error logging throughout

2. FIX: src/app/api/skills/collect/chaos/route.ts
   - Was MISSING logger import (log.error was undefined at runtime)
   - Reduced from 388 lines → 142 lines by delegating to shared executor
   - Added maxTokens support in schema validation

3. REFACTOR: src/app/api/chaos/run/route.ts
   - Simplified to thin wrapper: auth + validate + delegate to executor
   - Added maxTokens support

4. ENHANCE: src/lib/chaos/chaosConfig.ts
   - Added maxTokens config field (256-128k, default 4096)
   - Persisted per-instance via settings table

5. ENHANCE: UI — ChaosConfigPageClient.tsx
   - Loads available providers from /api/models for dropdown autocomplete
   - Added datalist-based provider selector in overrides section
   - Added Max Tokens configuration input
   - Added expandable provider list showing all detected providers
   - Fixed duplicate override detection

* fix(chaos): fetch providers from /api/providers instead of /api/keys

* fix(chaos): remove dead code isOverrideDuplicate, fix maxTokens fallback to include global config

* fix(chaos): resetConfig now shows error on HTTP failure (was silent)

* feat(dashboard): Chaos Mode — multi-model parallel/collaborative execution

Splits the PR down to only the genuinely new Chaos Mode feature (drops the
duplicate Skill Collector/GitHub-discovery portion already shipped via
#6186). Replaces the loopback fetch() dispatch (hardcoded to the wrong port)
with the established in-process synthetic-Request/route-handler pattern used
by src/lib/batches/dispatch.ts, moves settings persistence off raw SQL, and
adds unit test coverage for chaosConfig, chaosExecutor and the 3 chaos API
routes.

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>

* fix(chaos): fix external Bearer-auth bypass and stale config cache in tests

validateApiKey() returns a plain boolean for both the deployment-time env key
and a DB-backed key, so branching on `keyInfo === true` in
verifyChaosKey() (src/app/api/skills/collect/chaos/route.ts) treated every
valid API key as having full env-key access, silently skipping the
chaosModeEnabled permission check entirely. Now always resolves through
getApiKeyMetadata() and only bypasses the per-key check for the synthesized
env-key record (id: "env-key").

Also exports invalidateChaosConfigCache() from chaosConfig.ts and wires it
into the route tests' resetStorage() — the in-process config cache was
surviving DB resets between tests, causing state to leak across cases.

Fixes CHANGELOG-eat from the release merge (re-inserted the Chaos Mode
bullet against the base CHANGELOG.md, verified additive via
check-changelog-integrity.mjs) and re-syncs against release/v3.8.47 tip.

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>

* docs(changelog): Chaos Mode overhaul bullet referencing #6728 after release sync

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>

* fix(merge): restore #6126 clinepass files reverted by release auto-resolve + baseline re-merge

The release sync's auto-resolve reverted sibling PR #6126's clinepass work
(registry, catalog, oauth constants, clineAuth.ts, token-refresh case, tests)
and the file-size baseline — all outside this PR's scope. Restored to the
release versions, re-applied only this PR's own baseline entries, restored the
#6126 CHANGELOG bullet (re-inserting only this PR's own).

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>

* fix(dashboard): chaos client hook must not import the server Pino logger

useChaosConfigData ("use client") pulled @/sse/utils/logger → shared Pino →
logRotation/dataPaths → node:fs into the browser bundle, breaking next build
(Turbopack: Can't resolve 'fs') — caught by the DAST smoke's isolated build.
console.error matches every other dashboard client component.

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>

* test(api-manager): align switch-count invariant with the extracted toggle components

The Self-service block now renders 4 inline switches; the #5731 quota-bypass
and #6728 chaos-access toggles were extracted into dedicated components. The
type="button" invariant is preserved AND extended: the test now also asserts
each extracted component's switches declare type="button".

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>

* chore(sync): merge release tip + restore own CHANGELOG bullet

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>

---------

Co-authored-by: Moseyuh333 <Moseyuh333@users.noreply.github.com>
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
Co-authored-by: Diego Rodrigues de Sa e Souza <diegosouza.pw@gmail.com>
This commit is contained in:
Moseyuh333
2026-07-10 09:49:48 +07:00
committed by GitHub
parent a5c555b0de
commit eeec4d9e87
36 changed files with 2703 additions and 36 deletions

View File

@@ -18,6 +18,7 @@ _Living section — bullets land here as PRs merge into `release/v3.8.47` (paral
- **ClinePass dual-auth**: ClinePass now offers both sign-in methods on its dashboard page — OAuth (reusing the Cline WorkOS flow) as the primary "Connect" path, or a pasted BYOK API key via "Manual API key", instead of only the API-key-only provider shipped in #5942. The registry alias was aligned to `cp` (matching the `OAUTH_PROVIDERS` catalog alias) so `<alias>/<modelId>` routing resolves correctly, the OAuth refresh dispatch now routes `clinepass` to the shared Cline refresh flow, and the duplicate API-key-only catalog entry was removed to keep ClinePass listed once. Regression guard: `tests/unit/clinepass-provider.test.ts`. (#6126 — thanks @hajilok)
- **feat(oauth):** Kiro/Amazon Q auto-import now supports enterprise **External IdP** ("Your organization") logins via Microsoft Entra/Okta/Auth0/OneLogin/Ping/Google/Cognito — these org-issued tokens are not AWS SSO tokens (no `aorAAAAAG`-prefixed refresh token) and can't refresh through the AWS OIDC/Kiro-social path, so `tryAwsSsoCache()` now detects them (`authMethod`/`provider === "externalidp"`) and refreshes via the org IdP's own `tokenEndpoint` (public-client OAuth2 refresh grant, no client secret), persisting `TokenType: EXTERNAL_IDP` gating so the runtime executor sends the header the AWS CodeWhisperer API requires for these accounts; `tokenEndpoint` is SSRF-guarded against an HTTPS + known-IdP-host-suffix allowlist. (#6363 — thanks @artickc)
- **Kiro long-lived API key auth**: new `/api/oauth/kiro/api-key` route + `KiroService.validateApiKey` let a Kiro account be linked with a long-lived AWS CodeWhisperer/Kiro API key instead of the interactive OAuth device flow, with live per-account model discovery (`ListAvailableModels`, 5-minute cache) layered over the existing static registry fallback (#6587 — thanks @strangersp)
- **Chaos Mode**: multi-model parallel/collaborative task execution — dispatches a task to every active provider connection at once (parallel) or chains outputs sequentially so each model builds on the previous one's answer (collaborative), configurable via Dashboard → Chaos Mode (`GET`/`PUT`/`DELETE /api/chaos/config`) and gated per-API-key via a new `chaosModeEnabled` permission (opt-in — disabled by default globally and per key). `POST /api/chaos/run` (dashboard session) and `POST /api/skills/collect/chaos` (external Bearer-token) delegate to a shared `executeChaosRun()` engine (`src/lib/chaos/chaosExecutor.ts`) that dispatches in-process via the established synthetic-Request/route-handler pattern (no network hop, no hardcoded port), with a concurrency cap (max 10 parallel), configurable `max_tokens` (256128k), a clear error when `stream` is requested, and collaborative-chain info (provider order + input size). Fixes external Bearer-auth bypass and stale config-cache leakage. Regression guard: `tests/unit/chaos-config.test.ts`, `tests/unit/chaos-executor.test.ts`, `tests/unit/chaos-api-routes.test.ts`. ([#6728](https://github.com/diegosouzapw/OmniRoute/pull/6728) — thanks @Moseyuh333)
### 🐛 Bug Fixes

View File

@@ -25,6 +25,8 @@ import { SELF_ACCOUNT_QUOTA_SCOPE, SELF_USAGE_SCOPE } from "@/shared/constants/s
import { extractApiErrorMessage } from "@/shared/http/apiErrorMessage";
import { hasProviderQuotaBypassScope } from "@/shared/constants/apiKeyPolicyScopes";
import { UsageLimitSettings } from "./components/UsageLimitSettings";
import { ChaosModeAccessToggle } from "./components/ChaosModeAccessToggle";
import { BypassProviderQuotaToggle } from "./components/BypassProviderQuotaToggle";
// Constants for validation
const MAX_KEY_NAME_LENGTH = 200;
@@ -125,6 +127,7 @@ interface ApiKey {
streamDefaultMode?: StreamDefaultMode;
disableNonPublicModels?: boolean;
allowUsageCommand?: boolean;
chaosModeEnabled?: boolean;
usageLimitEnabled?: boolean;
dailyUsageLimitUsd?: number | null;
weeklyUsageLimitUsd?: number | null;
@@ -518,7 +521,8 @@ export default function ApiManagerPageClient() {
const res = await fetch(`/api/keys/${encodeURIComponent(key.id)}/devices`);
if (!res.ok) return [key.id, 0] as const;
const data = await res.json();
const count = typeof data?.count === "number" && Number.isFinite(data.count) ? data.count : 0;
const count =
typeof data?.count === "number" && Number.isFinite(data.count) ? data.count : 0;
return [key.id, count] as const;
} catch {
return [key.id, 0] as const;
@@ -791,7 +795,8 @@ export default function ApiManagerPageClient() {
usageLimitEnabled: boolean,
dailyUsageLimitUsd: number | null,
weeklyUsageLimitUsd: number | null,
blockedModels: string[]
blockedModels: string[],
chaosModeEnabled: boolean
) => {
if (!editingKey || !editingKey.id) return;
@@ -862,6 +867,7 @@ export default function ApiManagerPageClient() {
usageLimitEnabled,
dailyUsageLimitUsd,
weeklyUsageLimitUsd,
chaosModeEnabled,
}),
});
@@ -1645,7 +1651,8 @@ const PermissionsModal = memo(function PermissionsModal({
usageLimitEnabled: boolean,
dailyUsageLimitUsd: number | null,
weeklyUsageLimitUsd: number | null,
blockedModels: string[]
blockedModels: string[],
chaosModeEnabled: boolean
) => void;
}) {
const t = useTranslations("apiManager");
@@ -1731,6 +1738,7 @@ const PermissionsModal = memo(function PermissionsModal({
const [usageCommandEnabled, setUsageCommandEnabled] = useState(
apiKey?.allowUsageCommand === true
);
const [chaosModeEnabled, setChaosModeEnabled] = useState(apiKey?.chaosModeEnabled === true);
const [usageLimitEnabled, setUsageLimitEnabled] = useState(apiKey?.usageLimitEnabled === true);
const [dailyUsageLimitUsd, setDailyUsageLimitUsd] = useState(
typeof apiKey?.dailyUsageLimitUsd === "number" && apiKey.dailyUsageLimitUsd > 0
@@ -1935,7 +1943,8 @@ const PermissionsModal = memo(function PermissionsModal({
usageLimitEnabled,
parseUsdLimitInput(dailyUsageLimitUsd),
parseUsdLimitInput(weeklyUsageLimitUsd),
blockedModels
blockedModels,
chaosModeEnabled
);
}, [
onSave,
@@ -1974,6 +1983,7 @@ const PermissionsModal = memo(function PermissionsModal({
parseUsdLimitInput,
blockedClaudeCodeFamilies,
initialBlockedModels,
chaosModeEnabled,
apiKey?.scopes,
t,
]);
@@ -2563,30 +2573,17 @@ const PermissionsModal = memo(function PermissionsModal({
/>
</div>
{/* Chaos Mode Access Toggle */}
<ChaosModeAccessToggle
enabled={chaosModeEnabled}
onToggle={() => setChaosModeEnabled((prev) => !prev)}
/>
{/* Advanced Provider Quota Policy Override */}
<div className="flex items-start justify-between gap-3 p-3 rounded-lg border border-amber-500/20 bg-amber-500/5">
<div className="flex flex-col gap-1 pr-2">
<p className="text-sm font-medium text-text-main">Bypass provider quota cutoffs</p>
<p className="text-xs text-text-muted">
Allows this key to ignore upstream provider/account cutoff policy during routing. API
key USD quotas still apply.
</p>
</div>
<button
type="button"
role="switch"
aria-checked={bypassProviderQuotaPolicyEnabled}
onClick={() => setBypassProviderQuotaPolicyEnabled((prev) => !prev)}
className={`inline-flex shrink-0 items-center gap-1.5 px-2.5 py-1.5 rounded-md text-xs font-semibold transition-colors ${
bypassProviderQuotaPolicyEnabled
? "bg-amber-500/15 text-amber-700 dark:text-amber-300 border border-amber-500/30"
: "bg-black/5 dark:bg-white/5 text-text-muted border border-border"
}`}
>
<span className="material-symbols-outlined text-[14px]">alt_route</span>
{bypassProviderQuotaPolicyEnabled ? tc("enabled") : tc("disabled")}
</button>
</div>
<BypassProviderQuotaToggle
enabled={bypassProviderQuotaPolicyEnabled}
onToggle={() => setBypassProviderQuotaPolicyEnabled((prev) => !prev)}
/>
{/* Disable Non-Public Models Toggle */}
<div className="flex items-start justify-between gap-3 p-3 rounded-lg border border-border bg-surface/40">

View File

@@ -0,0 +1,45 @@
"use client";
import { useTranslations } from "next-intl";
/**
* "Bypass provider quota cutoffs" toggle for the API Key permissions modal.
* Extracted out of ApiManagerPageClient.tsx (frozen god-file — see
* config/quality/file-size-baseline.json) following the same pattern as
* UsageLimitSettings.tsx — pure UI move, no behavior change.
*/
export function BypassProviderQuotaToggle({
enabled,
onToggle,
}: {
enabled: boolean;
onToggle: () => void;
}) {
const tc = useTranslations("common");
return (
<div className="flex items-start justify-between gap-3 p-3 rounded-lg border border-amber-500/20 bg-amber-500/5">
<div className="flex flex-col gap-1 pr-2">
<p className="text-sm font-medium text-text-main">Bypass provider quota cutoffs</p>
<p className="text-xs text-text-muted">
Allows this key to ignore upstream provider/account cutoff policy during routing. API key
USD quotas still apply.
</p>
</div>
<button
type="button"
role="switch"
aria-checked={enabled}
onClick={onToggle}
className={`inline-flex shrink-0 items-center gap-1.5 px-2.5 py-1.5 rounded-md text-xs font-semibold transition-colors ${
enabled
? "bg-amber-500/15 text-amber-700 dark:text-amber-300 border border-amber-500/30"
: "bg-black/5 dark:bg-white/5 text-text-muted border border-border"
}`}
>
<span className="material-symbols-outlined text-[14px]">alt_route</span>
{enabled ? tc("enabled") : tc("disabled")}
</button>
</div>
);
}

View File

@@ -0,0 +1,45 @@
"use client";
import { useTranslations } from "next-intl";
/**
* Chaos Mode access toggle for the API Key permissions modal — gates a single
* API key's ability to call the Chaos Mode dispatch endpoints
* (`POST /api/chaos/run`, `POST /api/skills/collect/chaos`) via the
* `chaosModeEnabled` permission. Extracted out of ApiManagerPageClient.tsx
* (frozen god-file — see config/quality/file-size-baseline.json) following the
* same pattern as UsageLimitSettings.tsx.
*/
export function ChaosModeAccessToggle({
enabled,
onToggle,
}: {
enabled: boolean;
onToggle: () => void;
}) {
const tChaos = useTranslations("chaosConfig");
const tc = useTranslations("common");
return (
<div className="flex items-start justify-between gap-3 p-3 rounded-lg border border-amber-500/20 bg-amber-500/5">
<div className="flex flex-col gap-1">
<p className="text-sm font-medium text-text-main">{tChaos("keyPermission")}</p>
<p className="text-xs text-text-muted">{tChaos("keyPermissionDesc")}</p>
</div>
<button
type="button"
role="switch"
aria-checked={enabled}
onClick={onToggle}
className={`inline-flex shrink-0 items-center gap-1.5 px-2.5 py-1.5 rounded-md text-xs font-semibold transition-colors ${
enabled
? "bg-amber-500/15 text-amber-700 dark:text-amber-300 border border-amber-500/30"
: "bg-black/5 dark:bg-white/5 text-text-muted border border-border"
}`}
>
<span className="material-symbols-outlined text-[14px]">blender</span>
{tChaos("pageTitle")} - {enabled ? tc("enabled") : tc("disabled")}
</button>
</div>
);
}

View File

@@ -0,0 +1,107 @@
/**
* /dashboard/chaos — Chaos Mode Configuration Page
*
* Allows users to:
* - Enable/disable chaos mode globally
* - Set default mode (parallel/collaborative)
* - Override provider models for chaos mode
* - Set custom system prompt and max tokens
* - Configure timeout
* - Test chaos mode with a simple task
*
* State + handlers live in useChaosConfigPage.ts and the JSX sections are
* split into ./components/* — this file stays a thin composition/render
* function under the complexity/size ratchet
* (config/quality/complexity-baseline.json).
*/
"use client";
import { useChaosConfigPage } from "./useChaosConfigPage";
import { ChaosModeSelector } from "./components/ChaosModeSelector";
import { ChaosTestResultsPanel } from "./components/ChaosTestResultsPanel";
import { ChaosProviderOverridesPanel } from "./components/ChaosProviderOverridesPanel";
import { ChaosBasicSettingsFields } from "./components/ChaosBasicSettingsFields";
import { ChaosConfigActionsBar } from "./components/ChaosConfigActionsBar";
import { ChaosStatusMessage } from "./components/ChaosStatusMessage";
export default function ChaosConfigPage() {
const {
t,
config,
setConfig,
availableProviders,
loading,
saving,
testing,
testResult,
message,
saveConfig,
resetConfig,
testChaos,
addOverride,
updateOverride,
removeOverride,
} = useChaosConfigPage();
if (loading) {
return (
<div className="flex items-center justify-center h-64">
<div className="text-text-muted animate-pulse">{t("loadingProviderModels")}</div>
</div>
);
}
return (
<div className="max-w-3xl mx-auto p-4 sm:p-6 space-y-6">
{/* Header */}
<div>
<h1 className="text-2xl font-bold text-text-main">{t("pageTitle")}</h1>
<p className="text-sm text-text-muted mt-1">{t("pageSubtitle")}</p>
</div>
{/* Status Message */}
<ChaosStatusMessage message={message} />
{/* Enable toggle + timeout + max tokens + system prompt */}
<ChaosBasicSettingsFields
settings={config}
onChange={(patch) => setConfig((prev) => ({ ...prev, ...patch }))}
/>
{/* Default Mode Selector */}
<ChaosModeSelector
mode={config.defaultMode}
onChange={(defaultMode) => setConfig((prev) => ({ ...prev, defaultMode }))}
label={t("mode")}
parallelLabel={t("modeParallel")}
parallelDesc={t("modeParallelDesc")}
collaborativeLabel={t("modeCollaborative")}
collaborativeDesc={t("modeCollaborativeDesc")}
/>
<ChaosConfigActionsBar
saving={saving}
testing={testing}
testDisabled={!config.enabled}
onSave={saveConfig}
onReset={resetConfig}
onTest={testChaos}
/>
{/* Test Results */}
{testResult && <ChaosTestResultsPanel result={testResult} />}
{/* Provider Overrides */}
<ChaosProviderOverridesPanel
overrides={config.providerOverrides}
availableProviders={availableProviders}
title={t("providerOverrides")}
description={t("providerOverridesDesc")}
addLabel={t("addProvider")}
onAdd={addOverride}
onUpdate={updateOverride}
onRemove={removeOverride}
/>
</div>
);
}

View File

@@ -0,0 +1,28 @@
import type { ChaosProviderOverride } from "./components/ChaosProviderOverridesPanel";
export interface ChaosProviderInfo {
id: string;
name: string;
provider: string;
defaultModel: string | null;
}
export interface ChaosPageConfig {
enabled: boolean;
defaultMode: "parallel" | "collaborative";
providerOverrides: ChaosProviderOverride[];
systemPrompt?: string;
timeoutMs: number;
maxTokens: number;
}
export const DEFAULT_CHAOS_PAGE_CONFIG: ChaosPageConfig = {
enabled: false,
defaultMode: "parallel",
providerOverrides: [],
systemPrompt: "",
timeoutMs: 120_000,
maxTokens: 4096,
};
export type ChaosPageMessage = { type: "success" | "error"; text: string } | null;

View File

@@ -0,0 +1,122 @@
"use client";
import { useTranslations } from "next-intl";
export interface ChaosBasicSettings {
enabled: boolean;
timeoutMs: number;
maxTokens: number;
systemPrompt?: string;
}
function ChaosSystemPromptField({
value,
onChange,
}: {
value: string | undefined;
onChange: (value: string) => void;
}) {
const t = useTranslations("chaosConfig");
return (
<div className="p-3 rounded-lg border border-border bg-surface/40">
<p className="text-sm font-medium text-text-main">{t("systemPrompt")}</p>
<p className="text-xs text-text-muted mb-2">{t("systemPromptDesc")}</p>
<textarea
value={value || ""}
onChange={(e) => onChange(e.target.value)}
rows={3}
className="w-full px-3 py-1.5 rounded-md border border-border bg-surface text-sm text-text-main resize-y"
placeholder="Optional: override the default chaos mode system prompt..."
/>
</div>
);
}
/**
* Enable toggle + timeout + max tokens + system prompt fields for the Chaos
* Mode config page. Extracted out of ChaosConfigPageClient.tsx to keep the
* page component under the complexity/size ratchet
* (config/quality/complexity-baseline.json).
*/
export function ChaosBasicSettingsFields({
settings,
onChange,
}: {
settings: ChaosBasicSettings;
onChange: (patch: Partial<ChaosBasicSettings>) => void;
}) {
const t = useTranslations("chaosConfig");
return (
<>
{/* Enable/Disable Toggle */}
<div className="flex items-start justify-between gap-3 p-3 rounded-lg border border-border bg-surface/40">
<div className="flex flex-col gap-1">
<p className="text-sm font-medium text-text-main">{t("enableChaos")}</p>
<p className="text-xs text-text-muted">{t("enableChaosDesc")}</p>
</div>
<button
type="button"
role="switch"
aria-checked={settings.enabled}
onClick={() => onChange({ enabled: !settings.enabled })}
className={`inline-flex shrink-0 items-center gap-1.5 px-2.5 py-1.5 rounded-md text-xs font-semibold transition-colors ${
settings.enabled
? "bg-amber-500/15 text-amber-700 dark:text-amber-300 border border-amber-500/30"
: "bg-black/5 dark:bg-white/5 text-text-muted border border-border"
}`}
>
<span className="material-symbols-outlined text-[14px]">
{settings.enabled ? "toggle_on" : "toggle_off"}
</span>
{settings.enabled ? "Enabled" : "Disabled"}
</button>
</div>
{/* Timeout */}
<div className="p-3 rounded-lg border border-border bg-surface/40">
<p className="text-sm font-medium text-text-main">{t("timeout")}</p>
<p className="text-xs text-text-muted mb-2">{t("timeoutDesc")}</p>
<input
type="number"
min={5000}
max={600000}
step={5000}
value={settings.timeoutMs}
onChange={(e) =>
onChange({
timeoutMs: Math.max(5000, Math.min(600000, Number(e.target.value) || 120000)),
})
}
className="w-full px-3 py-1.5 rounded-md border border-border bg-surface text-sm text-text-main"
/>
</div>
{/* Max Tokens */}
<div className="p-3 rounded-lg border border-border bg-surface/40">
<p className="text-sm font-medium text-text-main">Max Tokens</p>
<p className="text-xs text-text-muted mb-2">
Maximum tokens per model response. Higher values cost more and take longer.
</p>
<input
type="number"
min={256}
max={128000}
step={256}
value={settings.maxTokens}
onChange={(e) =>
onChange({
maxTokens: Math.max(256, Math.min(128000, Number(e.target.value) || 4096)),
})
}
className="w-full px-3 py-1.5 rounded-md border border-border bg-surface text-sm text-text-main"
/>
</div>
<ChaosSystemPromptField
value={settings.systemPrompt}
onChange={(systemPrompt) => onChange({ systemPrompt })}
/>
</>
);
}

View File

@@ -0,0 +1,74 @@
"use client";
import { useTranslations } from "next-intl";
/**
* Save/Reset + Test-run action buttons for the Chaos Mode config page.
* Extracted out of ChaosConfigPageClient.tsx to keep the page component under
* the complexity/size ratchet (config/quality/complexity-baseline.json).
*/
export function ChaosConfigActionsBar({
saving,
testing,
testDisabled,
onSave,
onReset,
onTest,
}: {
saving: boolean;
testing: boolean;
testDisabled: boolean;
onSave: () => void;
onReset: () => void;
onTest: () => void;
}) {
const t = useTranslations("chaosConfig");
return (
<>
{/* Actions */}
<div className="flex gap-3">
<button
type="button"
onClick={onSave}
disabled={saving}
className="flex items-center gap-2 px-4 py-2 rounded-lg bg-primary text-white text-sm font-semibold hover:opacity-90 disabled:opacity-50"
>
{saving ? (
<span className="material-symbols-outlined text-[16px] animate-spin">sync</span>
) : (
<span className="material-symbols-outlined text-[16px]">save</span>
)}
{t("saveConfig")}
</button>
<button
type="button"
onClick={onReset}
disabled={saving}
className="flex items-center gap-2 px-4 py-2 rounded-lg border border-border text-text-muted text-sm hover:bg-black/5 dark:hover:bg-white/5 disabled:opacity-50"
>
<span className="material-symbols-outlined text-[16px]">restart_alt</span>
{t("configReset")}
</button>
</div>
{/* Test Button */}
<div className="p-3 rounded-lg border border-border bg-surface/40">
<p className="text-sm font-medium text-text-main mb-2">{t("testButton")}</p>
<button
type="button"
onClick={onTest}
disabled={testing || testDisabled}
className="flex items-center gap-2 px-4 py-2 rounded-lg bg-amber-500/15 text-amber-700 dark:text-amber-300 border border-amber-500/30 text-sm font-semibold hover:bg-amber-500/25 disabled:opacity-50"
>
{testing ? (
<span className="material-symbols-outlined text-[16px] animate-spin">sync</span>
) : (
<span className="material-symbols-outlined text-[16px]">play_arrow</span>
)}
{testing ? "Running..." : t("testButton")}
</button>
</div>
</>
);
}

View File

@@ -0,0 +1,60 @@
"use client";
/**
* Parallel / Collaborative mode selector for the Chaos Mode config page.
* Extracted out of ChaosConfigPageClient.tsx to keep the page component under
* the complexity/size ratchet (config/quality/complexity-baseline.json).
*/
export function ChaosModeSelector({
mode,
onChange,
label,
parallelLabel,
parallelDesc,
collaborativeLabel,
collaborativeDesc,
}: {
mode: "parallel" | "collaborative";
onChange: (mode: "parallel" | "collaborative") => void;
label: string;
parallelLabel: string;
parallelDesc: string;
collaborativeLabel: string;
collaborativeDesc: string;
}) {
return (
<div className="p-3 rounded-lg border border-border bg-surface/40">
<p className="text-sm font-medium text-text-main mb-2">{label}</p>
<div className="flex gap-2">
<button
type="button"
onClick={() => onChange("parallel")}
className={`flex-1 px-3 py-2 rounded-md text-xs font-semibold transition-all ${
mode === "parallel"
? "bg-primary text-white"
: "bg-black/5 dark:bg-white/5 text-text-muted hover:bg-black/10 dark:hover:bg-white/10"
}`}
>
<span className="material-symbols-outlined text-[16px] align-middle mr-1">
call_split
</span>
{parallelLabel}
<p className="text-[10px] opacity-70 mt-0.5">{parallelDesc}</p>
</button>
<button
type="button"
onClick={() => onChange("collaborative")}
className={`flex-1 px-3 py-2 rounded-md text-xs font-semibold transition-all ${
mode === "collaborative"
? "bg-primary text-white"
: "bg-black/5 dark:bg-white/5 text-text-muted hover:bg-black/10 dark:hover:bg-white/10"
}`}
>
<span className="material-symbols-outlined text-[16px] align-middle mr-1">merge</span>
{collaborativeLabel}
<p className="text-[10px] opacity-70 mt-0.5">{collaborativeDesc}</p>
</button>
</div>
</div>
);
}

View File

@@ -0,0 +1,156 @@
"use client";
export interface ChaosProviderInfo {
id: string;
name: string;
provider: string;
defaultModel: string | null;
}
export interface ChaosProviderOverride {
providerId: string;
modelId?: string;
enabled: boolean;
}
type UpdateOverride = (index: number, field: keyof ChaosProviderOverride, value: any) => void;
function ChaosProviderOverrideRow({
override,
index,
availableProviders,
onUpdate,
onRemove,
}: {
override: ChaosProviderOverride;
index: number;
availableProviders: ChaosProviderInfo[];
onUpdate: UpdateOverride;
onRemove: (index: number) => void;
}) {
return (
<div className="flex items-center gap-2 p-2 rounded-md bg-black/5 dark:bg-white/5">
{/* Provider dropdown with available options */}
<div className="flex-1 relative">
<input
type="text"
list={`provider-list-${index}`}
placeholder="Provider ID (type or select)"
value={override.providerId}
onChange={(e) => onUpdate(index, "providerId", e.target.value)}
className="w-full px-2 py-1 rounded border border-border bg-surface text-xs text-text-main"
/>
<datalist id={`provider-list-${index}`}>
{availableProviders.map((p) => (
<option key={p.id} value={p.provider} />
))}
</datalist>
</div>
<input
type="text"
placeholder="Model ID (optional)"
value={override.modelId || ""}
onChange={(e) => onUpdate(index, "modelId", e.target.value)}
className="flex-1 px-2 py-1 rounded border border-border bg-surface text-xs text-text-main"
/>
<button
type="button"
onClick={() => onUpdate(index, "enabled", !override.enabled)}
className={`px-2 py-1 rounded text-xs ${
override.enabled ? "bg-green-500/10 text-green-600" : "bg-red-500/10 text-red-600"
}`}
>
{override.enabled ? "ON" : "OFF"}
</button>
<button
type="button"
onClick={() => onRemove(index)}
className="px-2 py-1 rounded text-xs text-red-500 hover:bg-red-500/10"
>
<span className="material-symbols-outlined text-[14px]">close</span>
</button>
</div>
);
}
function ChaosAvailableProvidersHint({ availableProviders }: { availableProviders: ChaosProviderInfo[] }) {
if (availableProviders.length === 0) return null;
return (
<details className="text-xs text-text-muted">
<summary className="cursor-pointer hover:text-text-main">
Available providers ({availableProviders.length})
</summary>
<div className="mt-1 flex flex-wrap gap-1">
{availableProviders.map((p) => (
<span key={p.id} className="px-1.5 py-0.5 rounded bg-black/5 dark:bg-white/5">
{p.provider}
{p.defaultModel && <span className="opacity-60 ml-1">({p.defaultModel})</span>}
</span>
))}
</div>
</details>
);
}
/**
* Per-provider model override editor for the Chaos Mode config page.
* Extracted out of ChaosConfigPageClient.tsx to keep the page component under
* the complexity/size ratchet (config/quality/complexity-baseline.json).
*/
export function ChaosProviderOverridesPanel({
overrides,
availableProviders,
title,
description,
addLabel,
onAdd,
onUpdate,
onRemove,
}: {
overrides: ChaosProviderOverride[];
availableProviders: ChaosProviderInfo[];
title: string;
description: string;
addLabel: string;
onAdd: () => void;
onUpdate: UpdateOverride;
onRemove: (index: number) => void;
}) {
return (
<div className="p-3 rounded-lg border border-border bg-surface/40 space-y-3">
<div className="flex items-center justify-between">
<div>
<p className="text-sm font-medium text-text-main">{title}</p>
<p className="text-xs text-text-muted">{description}</p>
</div>
<button
type="button"
onClick={onAdd}
className="flex items-center gap-1 px-2 py-1 rounded-md text-xs font-semibold bg-primary/10 text-primary hover:bg-primary/20"
>
<span className="material-symbols-outlined text-[14px]">add</span>
{addLabel}
</button>
</div>
{overrides.length === 0 && (
<p className="text-xs text-text-muted italic">
No overrides all active providers will participate with their default models
</p>
)}
{overrides.map((override, idx) => (
<ChaosProviderOverrideRow
key={idx}
override={override}
index={idx}
availableProviders={availableProviders}
onUpdate={onUpdate}
onRemove={onRemove}
/>
))}
<ChaosAvailableProvidersHint availableProviders={availableProviders} />
</div>
);
}

View File

@@ -0,0 +1,23 @@
"use client";
import type { ChaosPageMessage } from "../chaosPageTypes";
/**
* Success/error status banner for the Chaos Mode config page. Extracted out
* of ChaosConfigPageClient.tsx to keep the page component under the
* complexity/size ratchet (config/quality/complexity-baseline.json).
*/
export function ChaosStatusMessage({ message }: { message: ChaosPageMessage }) {
if (!message) return null;
return (
<div
className={`p-3 rounded-lg text-sm font-medium ${
message.type === "success"
? "bg-green-500/10 text-green-700 dark:text-green-300 border border-green-500/20"
: "bg-red-500/10 text-red-700 dark:text-red-300 border border-red-500/20"
}`}
>
{message.text}
</div>
);
}

View File

@@ -0,0 +1,66 @@
"use client";
export interface ChaosModelResult {
providerId: string;
providerName: string;
modelId: string;
status: "success" | "error" | "skipped";
content: string | null;
error?: string;
durationMs: number;
}
export interface ChaosTestResult {
task: string;
mode: string;
startedAt: string;
totalProviders: number;
totalResults: number;
models: ChaosModelResult[];
summary?: string;
}
/**
* Test-run results panel for the Chaos Mode config page. Extracted out of
* ChaosConfigPageClient.tsx to keep the page component under the
* complexity/size ratchet (config/quality/complexity-baseline.json).
*/
export function ChaosTestResultsPanel({ result }: { result: ChaosTestResult }) {
return (
<div className="p-3 rounded-lg border border-border bg-surface/40 space-y-3">
<h3 className="text-sm font-bold text-text-main">
Test Results {result.mode} mode ({result.totalProviders} providers)
</h3>
<div className="text-xs text-text-muted">
Started: {new Date(result.startedAt).toLocaleTimeString()}
</div>
{result.models.map((model, idx) => (
<div
key={idx}
className={`p-2 rounded-md text-xs ${
model.status === "success"
? "bg-green-500/5 border border-green-500/20"
: "bg-red-500/5 border border-red-500/20"
}`}
>
<div className="font-medium text-text-main">
[{idx + 1}] {model.providerName} / {model.modelId}
<span className="ml-2 text-text-muted">({model.durationMs}ms)</span>
<span
className={`ml-2 ${model.status === "success" ? "text-green-500" : "text-red-500"}`}
>
{model.status}
</span>
</div>
{model.status === "success" && model.content && (
<div className="mt-1 text-text-muted line-clamp-3 whitespace-pre-wrap">
{model.content.slice(0, 300)}
{model.content.length > 300 ? "..." : ""}
</div>
)}
{model.error && <div className="mt-1 text-red-500">{model.error}</div>}
</div>
))}
</div>
);
}

View File

@@ -0,0 +1,12 @@
/**
* /dashboard/chaos/page.tsx — Chaos Mode Configuration
*/
import ChaosConfigPageClient from "./ChaosConfigPageClient";
export const metadata = {
title: "Chaos Mode — OmniRoute",
};
export default function Page() {
return <ChaosConfigPageClient />;
}

View File

@@ -0,0 +1,96 @@
"use client";
import { useEffect, useMemo, useState } from "react";
import {
DEFAULT_CHAOS_PAGE_CONFIG,
type ChaosPageConfig,
type ChaosProviderInfo,
} from "./chaosPageTypes";
function pickProviderName(conn: any): string {
return conn.provider || conn.name || conn.id || "";
}
function toProviderInfo(conn: any, providerName: string): ChaosProviderInfo {
return {
id: conn.id,
name: conn.name || conn.provider || providerName,
provider: conn.provider || providerName,
defaultModel: conn.defaultModel || null,
};
}
/** Extract a deduplicated ChaosProviderInfo[] from the /api/providers connections payload. */
function extractProvidersFromPayload(payload: unknown): ChaosProviderInfo[] {
const allConnections = (payload as any)?.connections || (payload as any)?.data || payload;
if (!Array.isArray(allConnections)) return [];
const extracted: ChaosProviderInfo[] = [];
const seenProvider = new Set<string>();
for (const conn of allConnections) {
const providerName = pickProviderName(conn);
if (!providerName || seenProvider.has(providerName.toLowerCase())) continue;
seenProvider.add(providerName.toLowerCase());
extracted.push(toProviderInfo(conn, providerName));
}
return extracted;
}
/** Fetch the persisted chaos config, or null on failure (caller keeps the default). */
async function fetchChaosConfig(): Promise<ChaosPageConfig | null> {
const res = await fetch("/api/chaos/config");
if (!res.ok) return null;
const data = await res.json();
return data.config || DEFAULT_CHAOS_PAGE_CONFIG;
}
/** Fetch active provider connections, or [] on failure/empty. */
async function fetchChaosProviders(): Promise<ChaosProviderInfo[]> {
const res = await fetch("/api/providers");
if (!res.ok) return [];
const data = await res.json();
return extractProvidersFromPayload(data);
}
/**
* Chaos config + provider-connection data loading for the Chaos Mode config
* page. Extracted out of the page component to keep it under the
* complexity/size ratchet (config/quality/complexity-baseline.json).
*/
export function useChaosConfigData() {
const [config, setConfig] = useState<ChaosPageConfig>(DEFAULT_CHAOS_PAGE_CONFIG);
const [providers, setProviders] = useState<ChaosProviderInfo[]>([]);
const [loading, setLoading] = useState(true);
// Build a unique list of provider names from active connections for the dropdown
const availableProviders = useMemo(() => {
const seen = new Set<string>();
return providers.filter((p) => {
const key = p.provider || p.name || p.id;
if (seen.has(key)) return false;
seen.add(key);
return true;
});
}, [providers]);
// Fetch current config + providers on mount
useEffect(() => {
async function load() {
try {
const [nextConfig, nextProviders] = await Promise.all([
fetchChaosConfig(),
fetchChaosProviders(),
]);
if (nextConfig) setConfig(nextConfig);
if (nextProviders.length > 0) setProviders(nextProviders);
} catch (err) {
console.error("[chaos] Failed to load config", err);
} finally {
setLoading(false);
}
}
load();
}, []);
return { config, setConfig, availableProviders, loading };
}

View File

@@ -0,0 +1,72 @@
"use client";
import { useCallback } from "react";
import type { ChaosProviderOverride } from "./components/ChaosProviderOverridesPanel";
import { useChaosConfigData } from "./useChaosConfigData";
import { useChaosConfigPersistence } from "./useChaosConfigPersistence";
import { useChaosTestRun } from "./useChaosTestRun";
export type { ChaosProviderInfo, ChaosPageConfig } from "./chaosPageTypes";
export { DEFAULT_CHAOS_PAGE_CONFIG } from "./chaosPageTypes";
/**
* All state + handlers for the Chaos Mode config page
* (src/app/(dashboard)/dashboard/chaos/ChaosConfigPageClient.tsx). Composes
* useChaosConfigData/useChaosConfigPersistence/useChaosTestRun so the page
* component itself stays a thin render function under the complexity/size
* ratchet (config/quality/complexity-baseline.json).
*/
export function useChaosConfigPage() {
const { config, setConfig, availableProviders, loading } = useChaosConfigData();
const { t, saving, message, setMessage, saveConfig, resetConfig } = useChaosConfigPersistence(
config,
setConfig
);
const { testing, testResult, testChaos } = useChaosTestRun(config, setMessage);
const addOverride = useCallback(() => {
setConfig((prev) => ({
...prev,
providerOverrides: [...prev.providerOverrides, { providerId: "", modelId: "", enabled: true }],
}));
}, [setConfig]);
const updateOverride = useCallback(
(index: number, field: keyof ChaosProviderOverride, value: any) => {
setConfig((prev) => {
const overrides = [...prev.providerOverrides];
overrides[index] = { ...overrides[index], [field]: value };
return { ...prev, providerOverrides: overrides };
});
},
[setConfig]
);
const removeOverride = useCallback(
(index: number) => {
setConfig((prev) => ({
...prev,
providerOverrides: prev.providerOverrides.filter((_, i) => i !== index),
}));
},
[setConfig]
);
return {
t,
config,
setConfig,
availableProviders,
loading,
saving,
testing,
testResult,
message,
saveConfig,
resetConfig,
testChaos,
addOverride,
updateOverride,
removeOverride,
};
}

View File

@@ -0,0 +1,65 @@
"use client";
import { useCallback, useState, type Dispatch, type SetStateAction } from "react";
import { useTranslations } from "next-intl";
import type { ChaosPageConfig, ChaosPageMessage } from "./chaosPageTypes";
/**
* Save / reset persistence for the Chaos Mode config page. Extracted out of
* the page component to keep it under the complexity/size ratchet
* (config/quality/complexity-baseline.json).
*/
export function useChaosConfigPersistence(
config: ChaosPageConfig,
setConfig: Dispatch<SetStateAction<ChaosPageConfig>>
) {
const t = useTranslations("chaosConfig");
const [saving, setSaving] = useState(false);
const [message, setMessage] = useState<ChaosPageMessage>(null);
const saveConfig = useCallback(async () => {
setSaving(true);
setMessage(null);
try {
const res = await fetch("/api/chaos/config", {
method: "PUT",
headers: { "Content-Type": "application/json" },
body: JSON.stringify(config),
});
if (res.ok) {
const data = await res.json();
setConfig(data.config);
setMessage({ type: "success", text: t("configSaved") });
} else {
setMessage({ type: "error", text: t("configError") });
}
} catch {
setMessage({ type: "error", text: t("configError") });
} finally {
setSaving(false);
}
}, [config, setConfig, t]);
const resetConfig = useCallback(async () => {
setSaving(true);
setMessage(null);
try {
const res = await fetch("/api/chaos/config", { method: "DELETE" });
if (res.ok) {
const data = await res.json();
setConfig(data.config);
setMessage({ type: "success", text: "Config reset to defaults" });
} else {
const err = await res.json().catch(() => ({ error: "Reset failed" }));
setMessage({ type: "error", text: err.error || "Reset failed" });
}
} catch {
setMessage({ type: "error", text: "Failed to reset config" });
} finally {
setSaving(false);
}
}, [setConfig]);
return { t, saving, message, setMessage, saveConfig, resetConfig };
}

View File

@@ -0,0 +1,48 @@
"use client";
import { useCallback, useState } from "react";
import { useTranslations } from "next-intl";
import type { ChaosTestResult } from "./components/ChaosTestResultsPanel";
import type { ChaosPageConfig, ChaosPageMessage } from "./chaosPageTypes";
/**
* "Test Chaos Mode" run trigger for the Chaos Mode config page. Extracted out
* of the page component to keep it under the complexity/size ratchet
* (config/quality/complexity-baseline.json).
*/
export function useChaosTestRun(config: ChaosPageConfig, setMessage: (message: ChaosPageMessage) => void) {
const t = useTranslations("chaosConfig");
const [testing, setTesting] = useState(false);
const [testResult, setTestResult] = useState<ChaosTestResult | null>(null);
const testChaos = useCallback(async () => {
setTesting(true);
setTestResult(null);
setMessage(null);
try {
const res = await fetch("/api/chaos/run", {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({
task: t("testTask"),
mode: config.defaultMode,
maxTokens: config.maxTokens,
}),
});
if (res.ok) {
const data: ChaosTestResult = await res.json();
setTestResult(data);
} else {
const err = await res.json().catch(() => ({ error: "Unknown error" }));
setMessage({ type: "error", text: err.error || "Test failed" });
}
} catch (err: any) {
setMessage({ type: "error", text: err.message || "Test failed" });
} finally {
setTesting(false);
}
}, [config.defaultMode, config.maxTokens, setMessage, t]);
return { testing, testResult, testChaos };
}

View File

@@ -0,0 +1,84 @@
/**
* GET /api/chaos/config — Get chaos mode configuration
* PUT /api/chaos/config — Update chaos mode configuration
* DELETE /api/chaos/config — Reset to defaults
*
* Chaos Mode global settings: which providers/models participate,
* default mode (parallel/collaborative), system prompt, timeout.
*/
import { NextResponse } from "next/server";
import { requireManagementAuth } from "@/lib/api/requireManagementAuth";
import { buildErrorBody, sanitizeErrorMessage } from "@omniroute/open-sse/utils/error";
import {
getChaosConfig,
setChaosConfig,
resetChaosConfig,
chaosConfigSchema,
} from "@/lib/chaos/chaosConfig";
import { validateBody, isValidationFailure } from "@/shared/validation/helpers";
import * as log from "@/sse/utils/logger";
export const dynamic = "force-dynamic";
/**
* GET /api/chaos/config
* Returns the current chaos mode configuration.
*/
export async function GET(request: Request) {
const authError = await requireManagementAuth(request);
if (authError) return authError;
try {
const config = await getChaosConfig();
return NextResponse.json({ config });
} catch (err) {
const msg = sanitizeErrorMessage(err);
log.error("chaos", "Error fetching chaos config", err);
return NextResponse.json(buildErrorBody(500, msg), { status: 500 });
}
}
/**
* PUT /api/chaos/config
* Update chaos mode configuration.
*/
export async function PUT(request: Request) {
const authError = await requireManagementAuth(request);
if (authError) return authError;
try {
const rawBody = await request.json();
const validation = validateBody(chaosConfigSchema, rawBody);
if (isValidationFailure(validation)) {
return NextResponse.json(buildErrorBody(400, validation.error.message), {
status: 400,
});
}
const config = await setChaosConfig(validation.data);
return NextResponse.json({ config, message: "Chaos config updated" });
} catch (err) {
const msg = sanitizeErrorMessage(err);
log.error("chaos", "Error updating chaos config", err);
return NextResponse.json(buildErrorBody(500, msg), { status: 500 });
}
}
/**
* DELETE /api/chaos/config
* Reset chaos config to defaults.
*/
export async function DELETE(request: Request) {
const authError = await requireManagementAuth(request);
if (authError) return authError;
try {
const config = await resetChaosConfig();
return NextResponse.json({ config, message: "Chaos config reset to defaults" });
} catch (err) {
const msg = sanitizeErrorMessage(err);
log.error("chaos", "Error resetting chaos config", err);
return NextResponse.json(buildErrorBody(500, msg), { status: 500 });
}
}

View File

@@ -0,0 +1,73 @@
/**
* POST /api/chaos/run — Unified Chaos Mode execution endpoint.
*
* Dashboard-friendly: uses the current management session (cookie) for auth.
* Delegates all execution logic to the shared chaosExecutor library.
*
* Body (JSON):
* task: string // REQUIRED — the task/goal
* providers?: string[] // Optional — filter specific providers
* mode?: "parallel" | "collaborative" // Optional — override global default
* systemPrompt?: string // Optional — override global system prompt
* maxTokens?: number // Optional — override max_tokens per model call
*
* Returns the same shape as POST /api/skills/collect/chaos.
*/
import { NextResponse } from "next/server";
import { z } from "zod";
import { requireManagementAuth } from "@/lib/api/requireManagementAuth";
import { validateBody, isValidationFailure } from "@/shared/validation/helpers";
import { buildErrorBody, sanitizeErrorMessage } from "@omniroute/open-sse/utils/error";
import { getChaosConfig } from "@/lib/chaos/chaosConfig";
import { executeChaosRun, type ChaosRunResult } from "@/lib/chaos/chaosExecutor";
import * as log from "@/sse/utils/logger";
export const dynamic = "force-dynamic";
const runSchema = z.object({
task: z.string().min(1, "Task is required").max(100_000, "task too long"),
providers: z.array(z.string().min(1)).max(50).optional(),
mode: z.enum(["parallel", "collaborative"]).optional(),
systemPrompt: z.string().max(10_000).optional(),
maxTokens: z.number().int().min(256).max(128_000).optional(),
});
export async function POST(request: Request) {
// Require dashboard management auth (cookie-based)
const authError = await requireManagementAuth(request);
if (authError) return authError;
try {
// Load global chaos config
const globalConfig = await getChaosConfig();
if (!globalConfig.enabled) {
return NextResponse.json(
buildErrorBody(400, "Chaos Mode is not enabled. Enable it in Dashboard → Chaos Mode."),
{ status: 400 }
);
}
const rawBody = await request.json();
const validation = validateBody(runSchema, rawBody);
if (isValidationFailure(validation)) {
return NextResponse.json(buildErrorBody(400, validation.error.message), { status: 400 });
}
const { task, providers, mode, systemPrompt, maxTokens } = validation.data;
const result: ChaosRunResult = await executeChaosRun({
task,
providers,
mode,
systemPrompt,
timeoutMs: globalConfig.timeoutMs,
maxTokens: maxTokens || globalConfig.maxTokens,
});
return NextResponse.json(result);
} catch (err) {
const msg = sanitizeErrorMessage(err);
log.error("chaos", "Chaos run error", err);
return NextResponse.json(buildErrorBody(500, msg), { status: 500 });
}
}

View File

@@ -86,6 +86,7 @@ export async function PATCH(request, { params }) {
usageLimitEnabled,
dailyUsageLimitUsd,
weeklyUsageLimitUsd,
chaosModeEnabled,
} = validation.data;
const payload: Parameters<typeof updateApiKeyPermissions>[1] = {};
@@ -112,6 +113,7 @@ export async function PATCH(request, { params }) {
if (usageLimitEnabled !== undefined) payload.usageLimitEnabled = usageLimitEnabled;
if (dailyUsageLimitUsd !== undefined) payload.dailyUsageLimitUsd = dailyUsageLimitUsd;
if (weeklyUsageLimitUsd !== undefined) payload.weeklyUsageLimitUsd = weeklyUsageLimitUsd;
if (chaosModeEnabled !== undefined) payload.chaosModeEnabled = chaosModeEnabled;
const updated = await updateApiKeyPermissions(id, payload);
if (!updated) {
@@ -145,6 +147,7 @@ export async function PATCH(request, { params }) {
...(usageLimitEnabled !== undefined && { usageLimitEnabled }),
...(dailyUsageLimitUsd !== undefined && { dailyUsageLimitUsd }),
...(weeklyUsageLimitUsd !== undefined && { weeklyUsageLimitUsd }),
...(chaosModeEnabled !== undefined && { chaosModeEnabled }),
});
} catch (error) {
log.error("keys", "Error updating key permissions", error);

View File

@@ -71,6 +71,7 @@ export async function POST(request) {
usageLimitEnabled,
dailyUsageLimitUsd,
weeklyUsageLimitUsd,
chaosModeEnabled,
} = validation.data;
// Always get machineId from server
@@ -82,7 +83,8 @@ export async function POST(request) {
allowUsageCommand === true ||
usageLimitEnabled === true ||
dailyUsageLimitUsd !== undefined ||
weeklyUsageLimitUsd !== undefined
weeklyUsageLimitUsd !== undefined ||
chaosModeEnabled === true
) {
await updateApiKeyPermissions(apiKey.id, {
...(noLog === true && { noLog: true }),
@@ -90,6 +92,7 @@ export async function POST(request) {
...(usageLimitEnabled === true && { usageLimitEnabled: true }),
...(dailyUsageLimitUsd !== undefined && { dailyUsageLimitUsd }),
...(weeklyUsageLimitUsd !== undefined && { weeklyUsageLimitUsd }),
...(chaosModeEnabled === true && { chaosModeEnabled: true }),
});
}
@@ -114,6 +117,7 @@ export async function POST(request) {
usageLimitEnabled: usageLimitEnabled === true,
dailyUsageLimitUsd: dailyUsageLimitUsd ?? null,
weeklyUsageLimitUsd: weeklyUsageLimitUsd ?? null,
chaosModeEnabled: chaosModeEnabled === true,
streamDefaultMode: "legacy",
},
{ status: 201 }

View File

@@ -0,0 +1,147 @@
/**
* POST /api/skills/collect/chaos
*
* Chaos Mode — spawn multiple models across providers for parallel or collaborative
* task execution. Each active provider contributes one model instance; all models
* work on the same task simultaneously (parallel) or in a chain where each sees
* the previous model's output (collaborative).
*
* External API: uses Bearer token auth (API key with chaos_mode_enabled).
*
* Body (JSON):
* task: string // REQUIRED — the task/goal for all models
* providers?: string[] // Optional filter — only these provider IDs
* mode?: "parallel" | "collaborative" // Default: from global config
* systemPrompt?: string // Optional custom system prompt override
* maxTokens?: number // Optional — max_tokens per model call
*
* Returns:
* {
* task, mode, startedAt,
* totalProviders, totalResults,
* models: [{ providerId, providerName, modelId, status, content, error?, durationMs }],
* summary?: string
* }
*/
import { NextResponse } from "next/server";
import { z } from "zod";
import { validateBody, isValidationFailure } from "@/shared/validation/helpers";
import { buildErrorBody, sanitizeErrorMessage } from "@omniroute/open-sse/utils/error";
import { validateApiKey, getApiKeyMetadata } from "@/lib/localDb";
import { getChaosConfig } from "@/lib/chaos/chaosConfig";
import { executeChaosRun, type ChaosRunResult } from "@/lib/chaos/chaosExecutor";
import * as log from "@/sse/utils/logger";
export const dynamic = "force-dynamic";
// ── Schema ───────────────────────────────────────────────────────────────────
const chaosSchema = z.object({
task: z.string().min(1, "task is required").max(100_000, "task too long"),
providers: z.array(z.string().min(1)).max(50).optional(),
mode: z.enum(["parallel", "collaborative"]).optional(),
systemPrompt: z.string().max(10_000).optional(),
maxTokens: z.number().int().min(256).max(128_000).optional(),
});
// ── Auth helpers ─────────────────────────────────────────────────────────────
/**
* Extract Bearer token from Authorization header.
*/
function extractBearerToken(request: Request): string | null {
const auth = request.headers.get("Authorization");
if (!auth || !auth.startsWith("Bearer ")) return null;
return auth.slice(7).trim();
}
/**
* Verify API key has chaos mode enabled.
* validateApiKey returns a plain boolean for BOTH the deployment-time env key and a
* DB-backed key (see src/lib/db/apiKeys.ts::validateApiKey) — it never returns the
* key record, so the env-key/DB-key distinction has to be made via getApiKeyMetadata,
* whose synthesized env-key record is tagged `id: "env-key"` (src/lib/db/apiKeys.ts::
* getApiKeyMetadata) and always carries "manage" scope.
*/
async function verifyChaosKey(bearerToken: string): Promise<{ ok: boolean; error?: string }> {
const isValid = await validateApiKey(bearerToken);
if (!isValid) {
return { ok: false, error: "Invalid API key" };
}
const metadata = await getApiKeyMetadata(bearerToken);
if (!metadata) {
return { ok: false, error: "Invalid API key" };
}
// Env key has full access (see getApiKeyMetadata's synthesized "env-key" record).
if (metadata.id === "env-key") {
return { ok: true };
}
if (!metadata.chaosModeEnabled) {
return {
ok: false,
error: "Chaos Mode is not enabled for this API key. Enable it in API Key settings.",
};
}
return { ok: true };
}
// ── Main handler ─────────────────────────────────────────────────────────────
export async function POST(request: Request) {
try {
// ── API Key auth check ─────────────────────────────────────────────
const bearerToken = extractBearerToken(request);
if (!bearerToken) {
return NextResponse.json(
buildErrorBody(401, "Missing or invalid Authorization header — Bearer token required"),
{ status: 401 }
);
}
const auth = await verifyChaosKey(bearerToken);
if (!auth.ok) {
return NextResponse.json(buildErrorBody(403, auth.error!), { status: 403 });
}
// ── Parse request body ─────────────────────────────────────────────
const rawBody = await request.json();
const validation = validateBody(chaosSchema, rawBody);
if (isValidationFailure(validation)) {
return NextResponse.json(buildErrorBody(400, validation.error.message), {
status: 400,
});
}
const { task, providers, mode, systemPrompt, maxTokens } = validation.data;
// ── Load global chaos config ───────────────────────────────────────
const globalConfig = await getChaosConfig();
if (!globalConfig.enabled) {
return NextResponse.json(
buildErrorBody(400, "Chaos Mode is not enabled globally. Enable it in Dashboard → Chaos Mode."),
{ status: 400 }
);
}
// ── Execute via the shared executor ────────────────────────────────
const result: ChaosRunResult = await executeChaosRun({
task,
providers,
mode,
systemPrompt,
timeoutMs: globalConfig.timeoutMs,
maxTokens: maxTokens || globalConfig.maxTokens,
apiKey: bearerToken,
});
return NextResponse.json(result);
} catch (err) {
const msg = sanitizeErrorMessage(err);
log.error("chaos", "Chaos external API error", err);
return NextResponse.json(buildErrorBody(500, msg), { status: 500 });
}
}

View File

@@ -871,6 +871,7 @@
"skills": "Skills",
"omniSkills": "OmniSkills",
"agentSkills": "AgentSkills",
"chaosConfig": "Chaos Mode",
"docs": "Docs",
"issues": "Issues",
"endpoints": "Endpoints",
@@ -9149,5 +9150,35 @@
"auth": "Auth",
"feasibility": "Feasibility",
"models": "Models"
},
"chaosConfig": {
"pageTitle": "Chaos Mode",
"pageSubtitle": "Run multiple AI models in parallel or collaboratively on the same task",
"enableChaos": "Enable Chaos Mode",
"enableChaosDesc": "Allow API keys with chaos mode enabled to use this feature",
"mode": "Default Mode",
"modeParallel": "Parallel",
"modeCollaborative": "Collaborative",
"modeParallelDesc": "All models run simultaneously — fastest results",
"modeCollaborativeDesc": "Models chain outputs — each sees the previous result",
"timeout": "Timeout (ms)",
"timeoutDesc": "Max time per model call (5000-600000ms)",
"systemPrompt": "System Prompt (optional)",
"systemPromptDesc": "Custom instructions for all chaos mode model instances",
"providerOverrides": "Provider Overrides",
"providerOverridesDesc": "Select specific models per provider for chaos mode",
"providerId": "Provider",
"modelId": "Model",
"addProvider": "Add Provider",
"removeProvider": "Remove",
"saveConfig": "Save Configuration",
"configSaved": "Chaos configuration saved successfully",
"configError": "Failed to save chaos configuration",
"configReset": "Reset to Defaults",
"keyPermission": "Chaos Mode Access",
"keyPermissionDesc": "Allow this API key to use Chaos Mode (multi-model parallel execution)",
"testButton": "Test Chaos Mode",
"testTask": "Write a short poem about artificial intelligence",
"loadingProviderModels": "Loading providers..."
}
}

View File

@@ -0,0 +1,119 @@
/**
* lib/chaos/chaosConfig.ts
*
* Chaos Mode configuration — persisted per-instance settings for:
* - Which providers/models participate
* - Default mode (parallel vs collaborative)
* - System prompt overrides
* - Max timeout per model call
*/
import { z } from "zod";
import { getSettings, updateSettings } from "@/lib/db/settings";
// ── Schema ───────────────────────────────────────────────────────────────────
export const chaosConfigSchema = z.object({
enabled: z.boolean().default(false),
defaultMode: z.enum(["parallel", "collaborative"]).default("parallel"),
providerOverrides: z
.array(
z.object({
providerId: z.string().min(1),
modelId: z.string().optional(),
enabled: z.boolean().default(true),
})
)
.max(200)
.default([]),
systemPrompt: z.string().max(10_000).optional(),
timeoutMs: z.number().int().min(5_000).max(600_000).default(120_000),
maxTokens: z.number().int().min(256).max(128_000).default(4096),
});
export type ChaosConfig = z.infer<typeof chaosConfigSchema>;
export const DEFAULT_CHAOS_CONFIG: ChaosConfig = {
enabled: false,
defaultMode: "parallel",
providerOverrides: [],
systemPrompt: undefined,
timeoutMs: 120_000,
maxTokens: 4096,
};
// ── Persistence ──────────────────────────────────────────────────────────────
//
// Persisted via the shared settings store (src/lib/db/settings.ts::getSettings/
// updateSettings — the `key_value` table, namespace 'settings') rather than
// hand-rolled SQL against a nonexistent `settings` table (the original PR queried
// a table that was never created — every read silently fell back to defaults and
// every write/reset threw). Follows the repo convention of routing all settings
// reads/writes through src/lib/db/settings.ts (see CLAUDE.md → Database).
const CONFIG_KEY = "chaosModeConfig";
let _configCache: ChaosConfig | null = null;
/**
* Get the current Chaos Mode configuration.
*/
export async function getChaosConfig(): Promise<ChaosConfig> {
if (_configCache) return _configCache;
try {
const settings = await getSettings();
const raw = settings[CONFIG_KEY];
if (raw === undefined || raw === null) {
_configCache = DEFAULT_CHAOS_CONFIG;
return _configCache;
}
const result = chaosConfigSchema.safeParse(raw);
if (result.success) {
_configCache = result.data;
return result.data;
}
// Fall back to default if stored config is invalid
_configCache = DEFAULT_CHAOS_CONFIG;
return _configCache;
} catch {
_configCache = DEFAULT_CHAOS_CONFIG;
return _configCache;
}
}
/**
* Update the Chaos Mode configuration.
*/
export async function setChaosConfig(config: ChaosConfig): Promise<ChaosConfig> {
const validated = chaosConfigSchema.parse(config);
await updateSettings({ [CONFIG_KEY]: validated });
// Invalidate cache
_configCache = null;
return validated;
}
/**
* Reset chaos config to defaults.
*/
export async function resetChaosConfig(): Promise<ChaosConfig> {
await updateSettings({ [CONFIG_KEY]: null });
_configCache = null;
return DEFAULT_CHAOS_CONFIG;
}
/**
* Invalidate the in-memory config cache without touching persisted settings.
* Needed whenever the underlying DB/settings store is reset out-of-band (e.g.
* test teardown calling resetDbInstance()) — otherwise getChaosConfig() keeps
* serving a stale in-memory value after the store it was read from is gone.
*/
export function invalidateChaosConfigCache(): void {
_configCache = null;
}

View File

@@ -0,0 +1,439 @@
/**
* lib/chaos/chaosExecutor.ts
*
* Shared Chaos Mode execution engine — used by BOTH:
* - POST /api/chaos/run (dashboard, management-session auth)
* - POST /api/skills/collect/chaos (external, Bearer-token auth)
*
* Eliminates the ~150 lines of duplicate dispatch logic that previously existed
* in both route files.
*/
import { getProviderConnections } from "@/models";
import { getChaosConfig, type ChaosConfig } from "@/lib/chaos/chaosConfig";
import { POST as postChatCompletion } from "@/app/api/v1/chat/completions/route";
// Wrapped in an object (rather than called as a bare imported function) so unit
// tests can swap it out via `mock.method(chatDispatch, "postChatCompletion", ...)`
// without hitting real upstream providers — the same pattern src/lib/batches/
// dispatch.ts uses for its `dispatch` export (ES module named bindings are
// read-only and cannot be mocked directly).
export const chatDispatch = {
postChatCompletion,
};
// ── Exported types ───────────────────────────────────────────────────────────
export interface ProviderInfo {
id: string;
name: string;
provider: string;
defaultModel: string | null;
}
export interface ModelResult {
providerId: string;
providerName: string;
modelId: string;
status: "success" | "error" | "skipped";
content: string | null;
error?: string;
durationMs: number;
}
export type ChaosMode = "parallel" | "collaborative";
export interface ChaosRunInput {
task: string;
providers?: string[];
mode?: ChaosMode;
systemPrompt?: string;
/** Override the global timeout for this single run */
timeoutMs?: number;
/** Override max_tokens sent to each model (default 4096) */
maxTokens?: number;
/**
* API key to attribute the in-process dispatch calls to (usage accounting,
* per-key policy). Optional — omitted for dashboard-initiated runs, which fall
* back to the same "local mode" (no Authorization header) path used by
* src/lib/evals/runtime.ts and src/lib/batches/dispatch.ts.
*/
apiKey?: string | null;
}
export interface ChaosRunResult {
task: string;
mode: ChaosMode;
startedAt: string;
totalProviders: number;
totalResults: number;
models: ModelResult[];
summary?: string;
}
interface EffectiveRunParams {
mode: ChaosMode;
timeoutMs: number;
maxTokens: number;
effectiveSystemPrompt: string;
}
// ── Constants ─────────────────────────────────────────────────────────────────
export const DEFAULT_SYSTEM_PROMPT = `You are one of several AI models working in CHAOS MODE.
Your job:
1. Analyze the user's task thoroughly
2. Produce the best possible response using your unique strengths
3. Be concise but complete — your output will be combined with other models' outputs
4. Do NOT refer to "other models" or "CHAOS MODE" in your response — just answer the task directly`;
export const COLLABORATIVE_SYSTEM_PROMPT = `You are one of several AI models working in CHAOS MODE (collaborative).
Your job:
1. You will see the task AND the previous model's output
2. Build upon, refine, critique, or extend the previous work
3. Add new insights, fix issues, or provide an alternative perspective
4. Do NOT refer to "CHAOS MODE" or other models explicitly — just contribute your part naturally`;
const DEFAULT_MAX_TOKENS = 4096;
/** Maximum concurrent fetch requests in parallel mode */
const MAX_CONCURRENCY = 10;
// ── Internal helpers — config/param resolution ──────────────────────────────
function resolveEffectiveRunParams(
input: ChaosRunInput,
globalConfig: ChaosConfig
): EffectiveRunParams {
const mode: ChaosMode = input.mode || globalConfig.defaultMode || "parallel";
const timeoutMs = input.timeoutMs || globalConfig.timeoutMs || 120_000;
const maxTokens = input.maxTokens || globalConfig.maxTokens || DEFAULT_MAX_TOKENS;
const effectiveSystemPrompt =
input.systemPrompt ||
globalConfig.systemPrompt ||
(mode === "collaborative" ? COLLABORATIVE_SYSTEM_PROMPT : DEFAULT_SYSTEM_PROMPT);
return { mode, timeoutMs, maxTokens, effectiveSystemPrompt };
}
/**
* Resolve the best model ID for a provider connection.
* Applies provider overrides from global chaos config if present.
*/
function resolveModelId(
conn: { provider?: string; id?: string; defaultModel?: string | null },
overrides: ChaosConfig["providerOverrides"]
): string {
const override = overrides.find(
(o) =>
o.enabled &&
(o.providerId.toLowerCase() === (conn.provider || "").toLowerCase() ||
o.providerId.toLowerCase() === (conn.id || "").toLowerCase())
);
if (override?.modelId) return override.modelId;
return conn.defaultModel || conn.provider || conn.id || "unknown";
}
// ── Internal helpers — provider selection ───────────────────────────────────
/** Narrow `active` connections down to the caller-requested `providers` filter. */
function filterByRequestedProviders(active: any[], requested: string[]): any[] {
const filterSet = new Set(requested.map((p: string) => p.toLowerCase()));
const selected = active.filter((c: any) => filterSet.has((c.provider ?? "").toLowerCase()));
if (selected.length === 0) {
throw new Error(`None of the specified providers are active: ${requested.join(", ")}`);
}
return selected;
}
/** Narrow `active` connections down to the enabled global-config overrides (soft filter). */
function filterByEnabledOverrides(
active: any[],
enabledOverrides: ChaosConfig["providerOverrides"]
): any[] {
const overrideIds = new Set(enabledOverrides.map((o) => o.providerId.toLowerCase()));
const selected = active.filter(
(c: any) =>
overrideIds.has((c.provider ?? "").toLowerCase()) || overrideIds.has((c.id ?? "").toLowerCase())
);
return selected.length > 0 ? selected : active; // fallback to all active
}
function toProviderInfoList(selected: any[]): ProviderInfo[] {
const providerMap = new Map<string, ProviderInfo>();
for (const conn of selected) {
const providerKey = conn.provider || conn.id;
if (!providerMap.has(providerKey)) {
providerMap.set(providerKey, {
id: conn.id,
name: conn.name || conn.provider || providerKey,
provider: conn.provider || providerKey,
defaultModel: conn.defaultModel || null,
});
}
}
return Array.from(providerMap.values());
}
/**
* Fetch active provider connections and narrow them down to the requested
* (explicit `providers` filter, else enabled global-config overrides, else all
* active) set, deduplicated by provider id.
*/
async function selectChaosProviders(
input: ChaosRunInput,
globalConfig: ChaosConfig
): Promise<{ providers: ProviderInfo[]; enabledOverrides: ChaosConfig["providerOverrides"] }> {
const allConnections = await getProviderConnections().catch(() => [] as any[]);
const active = (Array.isArray(allConnections) ? allConnections : []).filter(
(c: any) => c.isActive !== false
);
if (active.length === 0) {
throw new Error("No active provider connections found");
}
const enabledOverrides = globalConfig.providerOverrides.filter((o) => o.enabled);
let selected = active;
if (input.providers && input.providers.length > 0) {
selected = filterByRequestedProviders(active, input.providers);
} else if (enabledOverrides.length > 0) {
selected = filterByEnabledOverrides(active, enabledOverrides);
}
return { providers: toProviderInfoList(selected), enabledOverrides };
}
// ── Internal helpers — single-model dispatch ────────────────────────────────
function buildDispatchRequest(
model: string,
messages: { role: string; content: string }[],
maxTokens: number,
timeoutMs: number,
apiKey?: string | null
): Request {
const headers = new Headers({ "Content-Type": "application/json" });
if (apiKey) {
headers.set("Authorization", `Bearer ${apiKey}`);
}
return new Request("http://localhost/api/v1/chat/completions", {
method: "POST",
headers,
body: JSON.stringify({ model, messages, stream: false, max_tokens: maxTokens }),
signal: AbortSignal.timeout(timeoutMs),
});
}
async function parseDispatchResponse(
res: Response,
ctx: { providerId: string; providerName: string; modelId: string },
start: number
): Promise<ModelResult> {
if (!res.ok) {
const errText = await res.text().catch(() => "unknown error");
return {
...ctx,
status: "error",
content: null,
error: `API ${res.status}: ${errText.slice(0, 500)}`,
durationMs: Math.round(performance.now() - start),
};
}
const data = await res.json();
const content =
data.choices?.[0]?.message?.content ?? data.choices?.[0]?.text ?? JSON.stringify(data);
return {
...ctx,
status: "success",
content,
durationMs: Math.round(performance.now() - start),
};
}
function buildDispatchErrorResult(
err: unknown,
ctx: { providerId: string; providerName: string; modelId: string },
start: number,
timeoutMs: number
): ModelResult {
const errObj = err as { name?: string; type?: string; message?: string };
const isAbort = errObj?.name === "AbortError" || errObj?.type === "aborted";
return {
...ctx,
status: "error",
content: null,
error: isAbort ? `timeout (${timeoutMs}ms)` : (errObj?.message ?? String(err)),
durationMs: Math.round(performance.now() - start),
};
}
/**
* Dispatch to OmniRoute's own /v1/chat/completions handler for a given
* provider+model — in-process, via a synthetic Request handed directly to the
* route's POST handler. No network hop, no port dependency. Mirrors the
* established pattern in src/lib/batches/dispatch.ts and src/lib/evals/runtime.ts
* (which the codebase's outbound-self-call convention requires — see #6679 review).
*/
async function dispatchToModel(
providerId: string,
providerName: string,
modelId: string,
messages: { role: string; content: string }[],
timeoutMs: number,
maxTokens: number,
apiKey?: string | null
): Promise<ModelResult> {
const start = performance.now();
const ctx = { providerId, providerName, modelId };
try {
const model = modelId || providerId;
const request = buildDispatchRequest(model, messages, maxTokens, timeoutMs, apiKey);
const res = await chatDispatch.postChatCompletion(request);
return await parseDispatchResponse(res, ctx, start);
} catch (err: unknown) {
return buildDispatchErrorResult(err, ctx, start, timeoutMs);
}
}
/**
* Run an array of async functions with a concurrency limit.
* Uses a simple pooling approach: start up to `limit` tasks at once,
* and as each completes, start the next one.
*/
async function runWithConcurrencyLimit<T>(tasks: (() => Promise<T>)[], limit: number): Promise<T[]> {
const results: T[] = new Array(tasks.length);
let nextIndex = 0;
async function worker(): Promise<void> {
while (nextIndex < tasks.length) {
const idx = nextIndex++;
results[idx] = await tasks[idx]();
}
}
const workers = Array.from({ length: Math.min(limit, tasks.length) }, () => worker());
await Promise.all(workers);
return results;
}
// ── Internal helpers — mode dispatch ────────────────────────────────────────
function buildMessages(
systemPrompt: string,
userContent: string
): { role: "system" | "user"; content: string }[] {
return [
{ role: "system", content: systemPrompt },
{ role: "user", content: userContent },
];
}
async function dispatchParallel(
providers: ProviderInfo[],
input: ChaosRunInput,
overrides: ChaosConfig["providerOverrides"],
params: EffectiveRunParams
): Promise<ModelResult[]> {
const tasks = providers.map((p) => () => {
const modelId = resolveModelId(p, overrides);
const messages = buildMessages(params.effectiveSystemPrompt, input.task);
return dispatchToModel(
p.id,
p.name,
modelId,
messages,
params.timeoutMs,
params.maxTokens,
input.apiKey
);
});
const results = await runWithConcurrencyLimit(tasks, MAX_CONCURRENCY);
// Sort: successes first, then errors, then by duration
results.sort((a, b) => {
if (a.status === "success" && b.status !== "success") return -1;
if (a.status !== "success" && b.status === "success") return 1;
return a.durationMs - b.durationMs;
});
return results;
}
async function dispatchCollaborative(
providers: ProviderInfo[],
input: ChaosRunInput,
overrides: ChaosConfig["providerOverrides"],
params: EffectiveRunParams
): Promise<ModelResult[]> {
const results: ModelResult[] = [];
let context = input.task;
for (const p of providers) {
const modelId = resolveModelId(p, overrides);
const messages = buildMessages(params.effectiveSystemPrompt, context);
const result = await dispatchToModel(
p.id,
p.name,
modelId,
messages,
params.timeoutMs,
params.maxTokens,
input.apiKey
);
results.push(result);
if (result.status === "success" && result.content) {
context = `Task: ${input.task}\n\nPrevious model's output:\n${result.content}\n\n---\n\nPlease refine, extend, critique, or provide an alternative perspective on the above.`;
}
}
return results;
}
function buildCollaborativeSummary(mode: ChaosMode, results: ModelResult[]): string | undefined {
if (mode !== "collaborative") return undefined;
return (
results
.filter((r) => r.status === "success" && r.content)
.map((r) => r.content!)
.join("\n\n---\n\n") || undefined
);
}
// ── Main execution function ──────────────────────────────────────────────────
/**
* Execute a Chaos Mode run.
*
* This is the single shared implementation used by both API routes.
*/
export async function executeChaosRun(input: ChaosRunInput): Promise<ChaosRunResult> {
const globalConfig = await getChaosConfig();
const params = resolveEffectiveRunParams(input, globalConfig);
const { providers, enabledOverrides } = await selectChaosProviders(input, globalConfig);
const startedAt = new Date().toISOString();
const results =
params.mode === "parallel"
? await dispatchParallel(providers, input, enabledOverrides, params)
: await dispatchCollaborative(providers, input, enabledOverrides, params);
const summary = buildCollaborativeSummary(params.mode, results);
return {
task: input.task,
mode: params.mode,
startedAt,
totalProviders: providers.length,
totalResults: results.length,
models: results,
...(summary ? { summary } : {}),
};
}

View File

@@ -44,4 +44,8 @@ export const API_KEY_COLUMN_FALLBACKS = [
name: "weekly_usage_limit_usd",
definition: "weekly_usage_limit_usd REAL",
},
{
name: "chaos_mode_enabled",
definition: "chaos_mode_enabled INTEGER NOT NULL DEFAULT 0",
},
] as const;

View File

@@ -43,6 +43,7 @@ import {
parseNullableTimestamp,
parseIsBanned,
parseStreamDefaultMode,
parseChaosModeEnabled,
} from "./apiKeys/rowParsers";
import type { AccessSchedule, RateLimitRule } from "./apiKeys/types";
@@ -95,6 +96,7 @@ interface ApiKeyMetadata {
usageLimitEnabled: boolean;
dailyUsageLimitUsd: number | null;
weeklyUsageLimitUsd: number | null;
chaosModeEnabled: boolean;
}
interface ApiKeyRow extends JsonRecord {
@@ -134,6 +136,8 @@ interface ApiKeyRow extends JsonRecord {
dailyUsageLimitUsd?: unknown;
weekly_usage_limit_usd?: unknown;
weeklyUsageLimitUsd?: unknown;
chaos_mode_enabled?: unknown;
chaosModeEnabled?: unknown;
}
interface StatementLike<TRow = unknown> {
@@ -180,6 +184,7 @@ interface ApiKeyView extends JsonRecord {
usageLimitEnabled?: boolean;
dailyUsageLimitUsd?: number | null;
weeklyUsageLimitUsd?: number | null;
chaosModeEnabled?: boolean;
}
// LRU cache for API key validation (valid keys only)
@@ -393,7 +398,7 @@ function getPreparedStatements(db: ApiKeysDbLike): ApiKeysStatements {
"SELECT id, expires_at, revoked_at, is_active, is_banned FROM api_keys WHERE key = ? OR key_hash = ?"
);
_stmtGetKeyMetadata = db.prepare<ApiKeyRow>(
"SELECT id, name, machine_id, allowed_models, blocked_models, allowed_combos, allowed_connections, allowed_quotas, no_log, auto_resolve, is_active, access_schedule, max_requests_per_day, max_requests_per_minute, throttle_delay_ms, max_sessions, revoked_at, expires_at, ip_allowlist, scopes, rate_limits, is_banned, key_hash, allowed_endpoints, stream_default_mode, disable_non_public_models, allow_usage_command, usage_limit_enabled, daily_usage_limit_usd, weekly_usage_limit_usd, proxy_id FROM api_keys WHERE key = ? OR key_hash = ?"
"SELECT id, name, machine_id, allowed_models, blocked_models, allowed_combos, allowed_connections, allowed_quotas, no_log, auto_resolve, is_active, access_schedule, max_requests_per_day, max_requests_per_minute, throttle_delay_ms, max_sessions, revoked_at, expires_at, ip_allowlist, scopes, rate_limits, is_banned, key_hash, allowed_endpoints, stream_default_mode, disable_non_public_models, allow_usage_command, usage_limit_enabled, daily_usage_limit_usd, weekly_usage_limit_usd, chaos_mode_enabled, proxy_id FROM api_keys WHERE key = ? OR key_hash = ?"
);
_stmtInsertKey = db.prepare(
"INSERT INTO api_keys (id, name, key, machine_id, allowed_models, no_log, created_at, key_prefix, key_hash, scopes) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)"
@@ -446,6 +451,7 @@ export async function getApiKeys() {
(camelRow as JsonRecord).disableNonPublicModels
);
camelRow.allowUsageCommand = parseAllowUsageCommand((camelRow as JsonRecord).allowUsageCommand);
camelRow.chaosModeEnabled = parseChaosModeEnabled((camelRow as JsonRecord).chaosModeEnabled);
Object.assign(camelRow, parseApiKeyUsageLimitFields(camelRow));
if (typeof camelRow.id === "string" && camelRow.id.length > 0) {
setNoLog(camelRow.id, camelRow.noLog === true);
@@ -558,6 +564,7 @@ export async function getApiKeyById(id: string) {
(camelRow as JsonRecord).disableNonPublicModels
);
camelRow.allowUsageCommand = parseAllowUsageCommand((camelRow as JsonRecord).allowUsageCommand);
camelRow.chaosModeEnabled = parseChaosModeEnabled((camelRow as JsonRecord).chaosModeEnabled);
Object.assign(camelRow, parseApiKeyUsageLimitFields(camelRow));
if (typeof camelRow.id === "string" && camelRow.id.length > 0) {
setNoLog(camelRow.id, camelRow.noLog === true);
@@ -684,6 +691,7 @@ export async function updateApiKeyPermissions(
usageLimitEnabled?: boolean;
dailyUsageLimitUsd?: number | null;
weeklyUsageLimitUsd?: number | null;
chaosModeEnabled?: boolean;
}
) {
const db = getDbInstance() as ApiKeysDbLike;
@@ -722,6 +730,7 @@ export async function updateApiKeyPermissions(
dailyUsageLimitUsd: (update as { dailyUsageLimitUsd?: number | null }).dailyUsageLimitUsd,
weeklyUsageLimitUsd: (update as { weeklyUsageLimitUsd?: number | null })
.weeklyUsageLimitUsd,
chaosModeEnabled: (update as { chaosModeEnabled?: boolean }).chaosModeEnabled,
};
if (
@@ -748,6 +757,7 @@ export async function updateApiKeyPermissions(
(normalized as Record<string, unknown>).streamDefaultMode === undefined &&
normalized.disableNonPublicModels === undefined &&
normalized.allowUsageCommand === undefined &&
normalized.chaosModeEnabled === undefined &&
!hasUsageLimitUpdate(normalized as Record<string, unknown>)
) {
return false;
@@ -781,6 +791,7 @@ export async function updateApiKeyPermissions(
usageLimitEnabled?: number;
dailyUsageLimitUsd?: number | null;
weeklyUsageLimitUsd?: number | null;
chaosModeEnabled?: number;
} = { id };
if (normalized.name !== undefined) {
@@ -884,6 +895,11 @@ export async function updateApiKeyPermissions(
params.allowUsageCommand = normalized.allowUsageCommand ? 1 : 0;
}
if (normalized.chaosModeEnabled !== undefined) {
updates.push("chaos_mode_enabled = @chaosModeEnabled");
params.chaosModeEnabled = normalized.chaosModeEnabled ? 1 : 0;
}
appendUsageLimitUpdates(normalized as Record<string, unknown>, updates, params);
const maxSessionsUpdate = (normalized as Record<string, unknown>).maxSessions;
@@ -1270,6 +1286,7 @@ export async function getApiKeyMetadata(
usageLimitEnabled: false,
dailyUsageLimitUsd: null,
weeklyUsageLimitUsd: null,
chaosModeEnabled: false,
};
}
@@ -1343,6 +1360,9 @@ export async function getApiKeyMetadata(
allowUsageCommand: parseAllowUsageCommand(
(record as JsonRecord).allow_usage_command ?? (record as JsonRecord).allowUsageCommand
),
chaosModeEnabled: parseChaosModeEnabled(
(record as JsonRecord).chaos_mode_enabled ?? (record as JsonRecord).chaosModeEnabled
),
...parseApiKeyUsageLimitFields(record as JsonRecord),
};

View File

@@ -47,6 +47,10 @@ export function parseAllowUsageCommand(value: unknown): boolean {
return value === true || value === 1 || value === "1";
}
export function parseChaosModeEnabled(value: unknown): boolean {
return value === true || value === 1 || value === "1";
}
export function parseIsActive(value: unknown): boolean {
// DEFAULT 1 — active unless explicitly set to 0
if (value === 0 || value === "0" || value === false) return false;

View File

@@ -17,6 +17,13 @@ const PUBLIC_API_ROUTE_PREFIXES = [
// The handler enforces its own auth via extractUsageCommandApiKey/isValidApiKey
// and the allowUsageCommand flag — it must not be gated by management auth.
"/api/usage/om-usage",
// Chaos Mode external dispatch endpoint (POST /api/skills/collect/chaos).
// This entry only bypasses the dashboard requireLogin (cookie) gate — the
// handler enforces its own Bearer-token auth (validateApiKey +
// chaosModeEnabled check) before doing any work. See src/app/api/skills/
// collect/chaos/route.ts. Do not widen this prefix to cover other
// /api/skills/collect/* routes without the same per-handler auth.
"/api/skills/collect/chaos",
];
const PUBLIC_READONLY_API_ROUTE_PREFIXES = [

View File

@@ -515,6 +515,14 @@ const AGENTIC_FEATURES_ITEMS: readonly SidebarSectionChild[] = [
subtitleKey: "agentSkillsSubtitle",
icon: "share",
},
{
id: "chaos-config",
href: "/dashboard/chaos",
i18nKey: "chaosConfig",
labelFallback: "Chaos Mode",
subtitleFallback: "Multi-model parallel execution",
icon: "blender",
},
{
id: "skills",
href: "/dashboard/omni-skills",

View File

@@ -74,6 +74,7 @@ export const HIDEABLE_SIDEBAR_ITEM_IDS = [
"memory",
"skills",
"agent-skills",
"chaos-config",
"mcp",
"a2a",
"plugins",

View File

@@ -25,6 +25,7 @@ export const createKeySchema = z.object({
usageLimitEnabled: z.boolean().optional(),
dailyUsageLimitUsd: z.coerce.number().min(0).optional().nullable(),
weeklyUsageLimitUsd: z.coerce.number().min(0).optional().nullable(),
chaosModeEnabled: z.boolean().optional(),
scopes: z.array(z.string().trim().min(1).max(64)).max(32).optional(),
});
@@ -110,6 +111,7 @@ export const updateKeyPermissionsSchema = z
usageLimitEnabled: z.boolean().optional(),
dailyUsageLimitUsd: z.coerce.number().min(0).optional().nullable(),
weeklyUsageLimitUsd: z.coerce.number().min(0).optional().nullable(),
chaosModeEnabled: z.boolean().optional(),
})
.superRefine((value, ctx) => {
if (
@@ -133,7 +135,8 @@ export const updateKeyPermissionsSchema = z
value.allowUsageCommand === undefined &&
value.usageLimitEnabled === undefined &&
value.dailyUsageLimitUsd === undefined &&
value.weeklyUsageLimitUsd === undefined
value.weeklyUsageLimitUsd === undefined &&
value.chaosModeEnabled === undefined
) {
ctx.addIssue({
code: z.ZodIssueCode.custom,

View File

@@ -61,14 +61,28 @@ test("permissions modal switch buttons declare button type", () => {
selfServiceBlock.match(/<button\s+type="button"\s+role="switch"/g) ?? []
).length;
// Self-service Visibility block has 5 switches: own-usage visibility,
// shared-account quota visibility, disable-non-public-models (#3041), the
// per-key local usage command allowance (#4034), and the API-key provider
// quota-policy bypass scope (#5731).
// Self-service Visibility block has 4 inline switches: own-usage visibility,
// shared-account quota visibility, disable-non-public-models (#3041), and the
// per-key local usage command allowance (#4034). The API-key provider
// quota-policy bypass scope (#5731) and the Chaos Mode access scope (#6728)
// were extracted into dedicated toggle components (asserted below).
// The invariant is that every switch declares type="button"
// (typedSwitchButtonCount === switchButtonCount) to avoid implicit submit.
assert.equal(switchButtonCount, 5);
assert.equal(typedSwitchButtonCount, 5);
assert.equal(switchButtonCount, 4);
assert.equal(typedSwitchButtonCount, 4);
// The extracted toggle components keep the same invariant.
for (const rel of [
"src/app/(dashboard)/dashboard/api-manager/components/BypassProviderQuotaToggle.tsx",
"src/app/(dashboard)/dashboard/api-manager/components/ChaosModeAccessToggle.tsx",
]) {
const componentSource = fs.readFileSync(path.join(repoRoot, rel), "utf8");
const compSwitches = (componentSource.match(/role="switch"/g) ?? []).length;
const compTyped = (componentSource.match(/<button\s+type="button"\s+role="switch"/g) ?? [])
.length;
assert.ok(compSwitches >= 1, `${rel} must render a switch`);
assert.equal(compTyped, compSwitches, `${rel}: every switch declares type="button"`);
}
});
test("permissions modal exposes Claude Code default wildcard model", () => {

View File

@@ -0,0 +1,320 @@
/**
* Unit tests for the 3 Chaos Mode API routes:
* - GET/PUT/DELETE /api/chaos/config (management-session auth)
* - POST /api/chaos/run (management-session auth)
* - POST /api/skills/collect/chaos (external Bearer-token auth)
*
* Covers auth, Zod validation, and the "Chaos Mode disabled" 400 short-circuit
* per Hard Rule #18 / the #6679 review checklist. Upstream model dispatch is
* stubbed via chaosExecutor's `chatDispatch` mock point (see chaos-executor.test.ts)
* so these tests never hit a real provider.
*/
import test, { mock } from "node:test";
import assert from "node:assert/strict";
import fs from "node:fs";
import os from "node:os";
import path from "node:path";
const TEST_DATA_DIR = fs.mkdtempSync(path.join(os.tmpdir(), "omniroute-chaos-routes-"));
const ORIGINAL_DATA_DIR = process.env.DATA_DIR;
const ORIGINAL_API_KEY_SECRET = process.env.API_KEY_SECRET;
const ORIGINAL_INITIAL_PASSWORD = process.env.INITIAL_PASSWORD;
process.env.DATA_DIR = TEST_DATA_DIR;
process.env.API_KEY_SECRET = process.env.API_KEY_SECRET ?? "chaos-routes-test-secret";
const core = await import("../../src/lib/db/core.ts");
const apiKeysDb = await import("../../src/lib/db/apiKeys.ts");
const providersDb = await import("../../src/lib/db/providers.ts");
const chaosConfig = await import("../../src/lib/chaos/chaosConfig.ts");
const chaosExecutor = await import("../../src/lib/chaos/chaosExecutor.ts");
const configRoute = await import("../../src/app/api/chaos/config/route.ts");
const runRoute = await import("../../src/app/api/chaos/run/route.ts");
const skillsChaosRoute = await import("../../src/app/api/skills/collect/chaos/route.ts");
async function resetStorage() {
core.resetDbInstance();
apiKeysDb.resetApiKeyState();
// The DB is about to be wiped out from under it — drop the in-memory chaos
// config cache too, or getChaosConfig() keeps serving a stale value (e.g. a
// prior test's `enabled: true`) after resetDbInstance() below.
chaosConfig.invalidateChaosConfigCache();
fs.rmSync(TEST_DATA_DIR, { recursive: true, force: true });
fs.mkdirSync(TEST_DATA_DIR, { recursive: true });
delete process.env.INITIAL_PASSWORD;
}
function makeRequest(method: string, url: string, body?: unknown, headers: Record<string, string> = {}) {
return new Request(url, {
method,
headers: {
...(body !== undefined ? { "content-type": "application/json" } : {}),
...headers,
},
body: body !== undefined ? JSON.stringify(body) : undefined,
});
}
function jsonResponse(body: unknown, status = 200): Response {
return new Response(JSON.stringify(body), {
status,
headers: { "Content-Type": "application/json" },
});
}
test.beforeEach(async () => {
await resetStorage();
});
test.afterEach(() => {
mock.restoreAll();
});
test.after(() => {
core.resetDbInstance();
apiKeysDb.resetApiKeyState();
fs.rmSync(TEST_DATA_DIR, { recursive: true, force: true });
if (ORIGINAL_DATA_DIR === undefined) {
delete process.env.DATA_DIR;
} else {
process.env.DATA_DIR = ORIGINAL_DATA_DIR;
}
if (ORIGINAL_API_KEY_SECRET === undefined) {
delete process.env.API_KEY_SECRET;
} else {
process.env.API_KEY_SECRET = ORIGINAL_API_KEY_SECRET;
}
if (ORIGINAL_INITIAL_PASSWORD === undefined) {
delete process.env.INITIAL_PASSWORD;
} else {
process.env.INITIAL_PASSWORD = ORIGINAL_INITIAL_PASSWORD;
}
});
// ═════════════════════════════════════════════════════════════════════════════
// GET/PUT/DELETE /api/chaos/config
// ═════════════════════════════════════════════════════════════════════════════
test("GET /api/chaos/config — requires management auth when a password is configured", async () => {
process.env.INITIAL_PASSWORD = "chaos-config-requires-login";
const res = await configRoute.GET(makeRequest("GET", "http://localhost/api/chaos/config"));
assert.ok(res.status === 401 || res.status === 403, `expected 401/403, got ${res.status}`);
const body = (await res.json()) as { error: { message: string } | string };
const errorMsg = typeof body.error === "string" ? body.error : body.error.message;
assert.ok(!errorMsg.includes("at /"), "Hard Rule #12: no stack trace exposure");
});
test("GET /api/chaos/config — returns defaults, PUT updates, DELETE resets", async () => {
const getRes = await configRoute.GET(makeRequest("GET", "http://localhost/api/chaos/config"));
assert.equal(getRes.status, 200);
const getBody = (await getRes.json()) as { config: typeof chaosConfig.DEFAULT_CHAOS_CONFIG };
// JSON.stringify drops keys whose value is `undefined` (systemPrompt), so compare
// against the JSON round-tripped shape rather than the raw in-memory default.
assert.deepEqual(
getBody.config,
JSON.parse(JSON.stringify(chaosConfig.DEFAULT_CHAOS_CONFIG))
);
const putRes = await configRoute.PUT(
makeRequest("PUT", "http://localhost/api/chaos/config", {
enabled: true,
defaultMode: "collaborative",
providerOverrides: [],
timeoutMs: 60_000,
maxTokens: 8192,
})
);
assert.equal(putRes.status, 200);
const putBody = (await putRes.json()) as { config: { enabled: boolean; defaultMode: string } };
assert.equal(putBody.config.enabled, true);
assert.equal(putBody.config.defaultMode, "collaborative");
const deleteRes = await configRoute.DELETE(
makeRequest("DELETE", "http://localhost/api/chaos/config")
);
assert.equal(deleteRes.status, 200);
const deleteBody = (await deleteRes.json()) as { config: typeof chaosConfig.DEFAULT_CHAOS_CONFIG };
// Same JSON.stringify undefined-key drop as the GET assertion above.
assert.deepEqual(
deleteBody.config,
JSON.parse(JSON.stringify(chaosConfig.DEFAULT_CHAOS_CONFIG))
);
});
test("PUT /api/chaos/config — 400 on schema validation failure", async () => {
const res = await configRoute.PUT(
makeRequest("PUT", "http://localhost/api/chaos/config", {
enabled: true,
defaultMode: "not-a-real-mode",
})
);
assert.equal(res.status, 400);
});
// ═════════════════════════════════════════════════════════════════════════════
// POST /api/chaos/run
// ═════════════════════════════════════════════════════════════════════════════
test("POST /api/chaos/run — requires management auth when a password is configured", async () => {
process.env.INITIAL_PASSWORD = "chaos-run-requires-login";
const res = await runRoute.POST(
makeRequest("POST", "http://localhost/api/chaos/run", { task: "hello" })
);
assert.ok(res.status === 401 || res.status === 403, `expected 401/403, got ${res.status}`);
});
test("POST /api/chaos/run — 400 when Chaos Mode is not enabled globally", async () => {
const res = await runRoute.POST(
makeRequest("POST", "http://localhost/api/chaos/run", { task: "hello" })
);
assert.equal(res.status, 400);
const body = (await res.json()) as { error: { message: string } };
assert.match(body.error.message, /not enabled/i);
});
test("POST /api/chaos/run — 400 on invalid body (missing task)", async () => {
await chaosConfig.setChaosConfig({
enabled: true,
defaultMode: "parallel",
providerOverrides: [],
timeoutMs: 120_000,
maxTokens: 4096,
});
const res = await runRoute.POST(makeRequest("POST", "http://localhost/api/chaos/run", {}));
assert.equal(res.status, 400);
});
test("POST /api/chaos/run — 200 happy path dispatches without an Authorization header (dashboard/local mode)", async () => {
await chaosConfig.setChaosConfig({
enabled: true,
defaultMode: "parallel",
providerOverrides: [],
timeoutMs: 120_000,
maxTokens: 4096,
});
await providersDb.createProviderConnection({
provider: "openai",
authType: "apikey",
name: "Run Route Provider",
apiKey: "sk-run-route",
defaultModel: "gpt-4o-mini",
});
let capturedAuth: string | null | undefined;
mock.method(chaosExecutor.chatDispatch, "postChatCompletion", async (req: Request) => {
capturedAuth = req.headers.get("Authorization");
return jsonResponse({ choices: [{ message: { content: "dashboard result" } }] });
});
const res = await runRoute.POST(
makeRequest("POST", "http://localhost/api/chaos/run", { task: "Summarize" })
);
assert.equal(res.status, 200);
const body = (await res.json()) as { models: { status: string; content: string }[] };
assert.equal(body.models[0].status, "success");
assert.equal(body.models[0].content, "dashboard result");
assert.equal(capturedAuth, null);
});
// ═════════════════════════════════════════════════════════════════════════════
// POST /api/skills/collect/chaos
// ═════════════════════════════════════════════════════════════════════════════
test("POST /api/skills/collect/chaos — 401 without an Authorization header", async () => {
const res = await skillsChaosRoute.POST(
makeRequest("POST", "http://localhost/api/skills/collect/chaos", { task: "hello" })
);
assert.equal(res.status, 401);
});
test("POST /api/skills/collect/chaos — 403 with an invalid API key", async () => {
const res = await skillsChaosRoute.POST(
makeRequest(
"POST",
"http://localhost/api/skills/collect/chaos",
{ task: "hello" },
{ Authorization: "Bearer sk-does-not-exist" }
)
);
assert.equal(res.status, 403);
});
test("POST /api/skills/collect/chaos — 403 when the API key does not have chaosModeEnabled", async () => {
const created = await apiKeysDb.createApiKey("No Chaos Key", "machine-chaos-01");
const res = await skillsChaosRoute.POST(
makeRequest(
"POST",
"http://localhost/api/skills/collect/chaos",
{ task: "hello" },
{ Authorization: `Bearer ${created.key}` }
)
);
assert.equal(res.status, 403);
const body = (await res.json()) as { error: { message: string } };
assert.match(body.error.message, /Chaos Mode is not enabled for this API key/);
});
test("POST /api/skills/collect/chaos — 400 when Chaos Mode is disabled globally even with a valid key", async () => {
const created = await apiKeysDb.createApiKey("Chaos Key", "machine-chaos-02");
await apiKeysDb.updateApiKeyPermissions(created.id, { chaosModeEnabled: true });
apiKeysDb.clearApiKeyCaches();
const res = await skillsChaosRoute.POST(
makeRequest(
"POST",
"http://localhost/api/skills/collect/chaos",
{ task: "hello" },
{ Authorization: `Bearer ${created.key}` }
)
);
assert.equal(res.status, 400);
});
test("POST /api/skills/collect/chaos — 200 happy path forwards the caller's key to the in-process dispatch", async () => {
await chaosConfig.setChaosConfig({
enabled: true,
defaultMode: "parallel",
providerOverrides: [],
timeoutMs: 120_000,
maxTokens: 4096,
});
await providersDb.createProviderConnection({
provider: "openai",
authType: "apikey",
name: "Skills Route Provider",
apiKey: "sk-skills-route",
defaultModel: "gpt-4o-mini",
});
const created = await apiKeysDb.createApiKey("Chaos Key External", "machine-chaos-03");
await apiKeysDb.updateApiKeyPermissions(created.id, { chaosModeEnabled: true });
apiKeysDb.clearApiKeyCaches();
let capturedAuth: string | null | undefined;
mock.method(chaosExecutor.chatDispatch, "postChatCompletion", async (req: Request) => {
capturedAuth = req.headers.get("Authorization");
return jsonResponse({ choices: [{ message: { content: "external result" } }] });
});
const res = await skillsChaosRoute.POST(
makeRequest(
"POST",
"http://localhost/api/skills/collect/chaos",
{ task: "Summarize" },
{ Authorization: `Bearer ${created.key}` }
)
);
assert.equal(res.status, 200);
const body = (await res.json()) as { models: { status: string; content: string }[] };
assert.equal(body.models[0].status, "success");
assert.equal(body.models[0].content, "external result");
assert.equal(capturedAuth, `Bearer ${created.key}`);
});

View File

@@ -0,0 +1,126 @@
/**
* Unit tests for src/lib/chaos/chaosConfig.ts
*
* Regression coverage for #6679: the original PR queried `SELECT/INSERT/DELETE
* ... FROM settings ...` against a table that was never created in this schema
* (only `key_value` exists) — every read silently fell back to
* DEFAULT_CHAOS_CONFIG and every write/reset threw at runtime. This module now
* routes persistence through src/lib/db/settings.ts::getSettings/updateSettings
* (the `key_value` table, namespace 'settings'), matching the repo convention
* (CLAUDE.md → Database: never write raw SQL outside src/lib/db/).
*/
import test from "node:test";
import assert from "node:assert/strict";
import fs from "node:fs";
import os from "node:os";
import path from "node:path";
const TEST_DATA_DIR = fs.mkdtempSync(path.join(os.tmpdir(), "omniroute-chaos-config-"));
const ORIGINAL_DATA_DIR = process.env.DATA_DIR;
process.env.DATA_DIR = TEST_DATA_DIR;
const core = await import("../../src/lib/db/core.ts");
const chaosConfig = await import("../../src/lib/chaos/chaosConfig.ts");
async function resetStorage() {
core.resetDbInstance();
fs.rmSync(TEST_DATA_DIR, { recursive: true, force: true });
fs.mkdirSync(TEST_DATA_DIR, { recursive: true });
}
test.beforeEach(async () => {
await resetStorage();
});
test.after(() => {
core.resetDbInstance();
fs.rmSync(TEST_DATA_DIR, { recursive: true, force: true });
if (ORIGINAL_DATA_DIR === undefined) {
delete process.env.DATA_DIR;
} else {
process.env.DATA_DIR = ORIGINAL_DATA_DIR;
}
});
test("getChaosConfig returns DEFAULT_CHAOS_CONFIG when nothing is persisted yet", async () => {
const config = await chaosConfig.getChaosConfig();
assert.deepEqual(config, chaosConfig.DEFAULT_CHAOS_CONFIG);
});
test("setChaosConfig persists via the settings store and getChaosConfig reads it back", async () => {
const settingsDb = await import("../../src/lib/db/settings.ts");
const written = await chaosConfig.setChaosConfig({
enabled: true,
defaultMode: "collaborative",
providerOverrides: [{ providerId: "openai", modelId: "gpt-4o", enabled: true }],
systemPrompt: "Custom prompt",
timeoutMs: 60_000,
maxTokens: 8192,
});
assert.equal(written.enabled, true);
assert.equal(written.defaultMode, "collaborative");
const readBack = await chaosConfig.getChaosConfig();
assert.deepEqual(readBack, written);
// Confirms the write actually lands in key_value (namespace 'settings'), not a
// nonexistent `settings` table (the bug this fix replaces).
const rawSettings = await settingsDb.getSettings();
assert.equal((rawSettings as Record<string, unknown>).chaosModeConfig !== undefined, true);
});
test("setChaosConfig rejects invalid config (Zod parse, not raw SQL failure)", async () => {
const invalidConfig = {
enabled: true,
defaultMode: "not-a-real-mode",
providerOverrides: [],
timeoutMs: 60_000,
maxTokens: 4096,
} as unknown as Awaited<ReturnType<typeof chaosConfig.getChaosConfig>>;
await assert.rejects(() => chaosConfig.setChaosConfig(invalidConfig));
});
test("resetChaosConfig clears the stored config back to defaults", async () => {
await chaosConfig.setChaosConfig({
enabled: true,
defaultMode: "parallel",
providerOverrides: [],
timeoutMs: 90_000,
maxTokens: 2048,
});
assert.equal((await chaosConfig.getChaosConfig()).enabled, true);
const reset = await chaosConfig.resetChaosConfig();
assert.deepEqual(reset, chaosConfig.DEFAULT_CHAOS_CONFIG);
assert.deepEqual(await chaosConfig.getChaosConfig(), chaosConfig.DEFAULT_CHAOS_CONFIG);
});
test("getChaosConfig falls back to defaults when the stored payload fails schema validation", async () => {
const settingsDb = await import("../../src/lib/db/settings.ts");
await settingsDb.updateSettings({
chaosModeConfig: { enabled: "not-a-boolean", defaultMode: "parallel" },
});
const config = await chaosConfig.getChaosConfig();
assert.deepEqual(config, chaosConfig.DEFAULT_CHAOS_CONFIG);
});
test("in-memory cache is invalidated on write so config updates are visible immediately", async () => {
const first = await chaosConfig.getChaosConfig();
assert.equal(first.enabled, false);
await chaosConfig.setChaosConfig({
enabled: true,
defaultMode: "parallel",
providerOverrides: [],
timeoutMs: 120_000,
maxTokens: 4096,
});
const second = await chaosConfig.getChaosConfig();
assert.equal(second.enabled, true);
});

View File

@@ -0,0 +1,243 @@
/**
* Unit tests for src/lib/chaos/chaosExecutor.ts
*
* Regression coverage for #6679: the original PR dispatched to models via a real
* network fetch() to `${OMNIROUTE_INTERNAL_URL || "http://localhost:30129"}` —
* 30129 is the author's personal dev port, not the project's default port 20128,
* so Chaos Mode silently failed on any standard install. This now dispatches
* in-process via the established synthetic-Request/route-handler pattern (see
* src/lib/batches/dispatch.ts, src/lib/evals/runtime.ts) — no network hop, no
* port dependency. `chatDispatch.postChatCompletion` is mocked here (same
* technique tests/unit/batch-processor.test.ts uses for dispatch.ts) so these
* tests never hit a real upstream provider.
*/
import test, { mock } from "node:test";
import assert from "node:assert/strict";
import fs from "node:fs";
import os from "node:os";
import path from "node:path";
const TEST_DATA_DIR = fs.mkdtempSync(path.join(os.tmpdir(), "omniroute-chaos-executor-"));
const ORIGINAL_DATA_DIR = process.env.DATA_DIR;
process.env.DATA_DIR = TEST_DATA_DIR;
const core = await import("../../src/lib/db/core.ts");
const providersDb = await import("../../src/lib/db/providers.ts");
const chaosConfig = await import("../../src/lib/chaos/chaosConfig.ts");
const chaosExecutor = await import("../../src/lib/chaos/chaosExecutor.ts");
async function resetStorage() {
core.resetDbInstance();
fs.rmSync(TEST_DATA_DIR, { recursive: true, force: true });
fs.mkdirSync(TEST_DATA_DIR, { recursive: true });
}
test.beforeEach(async () => {
await resetStorage();
});
test.afterEach(() => {
mock.restoreAll();
});
test.after(() => {
core.resetDbInstance();
fs.rmSync(TEST_DATA_DIR, { recursive: true, force: true });
if (ORIGINAL_DATA_DIR === undefined) {
delete process.env.DATA_DIR;
} else {
process.env.DATA_DIR = ORIGINAL_DATA_DIR;
}
});
function jsonResponse(body: unknown, status = 200): Response {
return new Response(JSON.stringify(body), {
status,
headers: { "Content-Type": "application/json" },
});
}
test("executeChaosRun throws when no active provider connections exist", async () => {
await assert.rejects(
() => chaosExecutor.executeChaosRun({ task: "hello" }),
/No active provider connections found/
);
});
test("executeChaosRun dispatches in-process (no fetch/network call) and returns per-provider results", async () => {
await providersDb.createProviderConnection({
provider: "openai",
authType: "apikey",
name: "Test OpenAI",
apiKey: "sk-test-openai",
defaultModel: "gpt-4o-mini",
});
let capturedRequest: Request | null = null;
mock.method(chaosExecutor.chatDispatch, "postChatCompletion", async (req: Request) => {
capturedRequest = req;
return jsonResponse({ choices: [{ message: { content: "hello from openai" } }] });
});
const result = await chaosExecutor.executeChaosRun({ task: "Summarize this repo" });
assert.equal(result.mode, "parallel");
assert.equal(result.totalProviders, 1);
assert.equal(result.models.length, 1);
assert.equal(result.models[0].status, "success");
assert.equal(result.models[0].content, "hello from openai");
assert.ok(capturedRequest, "postChatCompletion should have been invoked in-process");
assert.equal(
(capturedRequest as unknown as Request).url,
"http://localhost/api/v1/chat/completions"
);
});
test("executeChaosRun forwards the caller's apiKey as a Bearer Authorization header", async () => {
await providersDb.createProviderConnection({
provider: "anthropic",
authType: "apikey",
name: "Test Anthropic",
apiKey: "sk-test-anthropic",
defaultModel: "claude-3-5-sonnet",
});
let capturedAuth: string | null = null;
mock.method(chaosExecutor.chatDispatch, "postChatCompletion", async (req: Request) => {
capturedAuth = req.headers.get("Authorization");
return jsonResponse({ choices: [{ message: { content: "ok" } }] });
});
await chaosExecutor.executeChaosRun({ task: "task", apiKey: "sk-caller-key" });
assert.equal(capturedAuth, "Bearer sk-caller-key");
});
test("executeChaosRun omits Authorization header when no apiKey is provided (dashboard/local mode)", async () => {
await providersDb.createProviderConnection({
provider: "openai",
authType: "apikey",
name: "Test OpenAI",
apiKey: "sk-test-openai-2",
defaultModel: "gpt-4o-mini",
});
let capturedAuth: string | null | undefined;
mock.method(chaosExecutor.chatDispatch, "postChatCompletion", async (req: Request) => {
capturedAuth = req.headers.get("Authorization");
return jsonResponse({ choices: [{ message: { content: "ok" } }] });
});
await chaosExecutor.executeChaosRun({ task: "task" });
assert.equal(capturedAuth, null);
});
test("executeChaosRun surfaces upstream errors per-model instead of throwing", async () => {
await providersDb.createProviderConnection({
provider: "openai",
authType: "apikey",
name: "Failing OpenAI",
apiKey: "sk-test-failing",
defaultModel: "gpt-4o-mini",
});
mock.method(chaosExecutor.chatDispatch, "postChatCompletion", async () =>
jsonResponse({ error: "upstream exploded" }, 502)
);
const result = await chaosExecutor.executeChaosRun({ task: "task" });
assert.equal(result.models[0].status, "error");
assert.match(result.models[0].error ?? "", /API 502/);
});
test("executeChaosRun collaborative mode chains context between sequential model calls", async () => {
await providersDb.createProviderConnection({
provider: "openai",
authType: "apikey",
name: "Model A",
apiKey: "sk-test-a",
defaultModel: "gpt-4o-mini",
});
await providersDb.createProviderConnection({
provider: "anthropic",
authType: "apikey",
name: "Model B",
apiKey: "sk-test-b",
defaultModel: "claude-3-5-sonnet",
});
const seenUserMessages: string[] = [];
mock.method(chaosExecutor.chatDispatch, "postChatCompletion", async (req: Request) => {
const body = (await req.json()) as { messages: { role: string; content: string }[] };
const userMessage = body.messages.find((m) => m.role === "user");
seenUserMessages.push(userMessage?.content ?? "");
return jsonResponse({
choices: [{ message: { content: `reply-${seenUserMessages.length}` } }],
});
});
const result = await chaosExecutor.executeChaosRun({
task: "Original task",
mode: "collaborative",
});
assert.equal(result.mode, "collaborative");
assert.equal(result.models.length, 2);
assert.equal(seenUserMessages[0], "Original task");
assert.match(seenUserMessages[1], /Previous model's output:\nreply-1/);
assert.ok(result.summary && result.summary.includes("reply-2"));
});
test("executeChaosRun respects an explicit providers filter and errors when none match", async () => {
await providersDb.createProviderConnection({
provider: "openai",
authType: "apikey",
name: "Only OpenAI",
apiKey: "sk-test-only-openai",
defaultModel: "gpt-4o-mini",
});
mock.method(chaosExecutor.chatDispatch, "postChatCompletion", async () =>
jsonResponse({ choices: [{ message: { content: "ok" } }] })
);
await assert.rejects(
() => chaosExecutor.executeChaosRun({ task: "task", providers: ["anthropic"] }),
/None of the specified providers are active/
);
const result = await chaosExecutor.executeChaosRun({ task: "task", providers: ["openai"] });
assert.equal(result.models.length, 1);
});
test("executeChaosRun applies global config provider overrides for model resolution", async () => {
await providersDb.createProviderConnection({
provider: "openai",
authType: "apikey",
name: "Override target",
apiKey: "sk-test-override",
defaultModel: "gpt-4o-mini",
});
await chaosConfig.setChaosConfig({
enabled: true,
defaultMode: "parallel",
providerOverrides: [{ providerId: "openai", modelId: "gpt-4-turbo", enabled: true }],
timeoutMs: 120_000,
maxTokens: 4096,
});
let capturedModel: string | undefined;
mock.method(chaosExecutor.chatDispatch, "postChatCompletion", async (req: Request) => {
const body = (await req.json()) as { model: string };
capturedModel = body.model;
return jsonResponse({ choices: [{ message: { content: "ok" } }] });
});
await chaosExecutor.executeChaosRun({ task: "task" });
assert.equal(capturedModel, "gpt-4-turbo");
});