fix(mimocode): handle 400 with cooldown + account rotation (#6648)

* fix(electron): bump electron 42→43 + build better-sqlite3 from source (ABI 148) (#6605)

fix(electron): bump electron 42→43 + rebuild better-sqlite3 from source against the Electron ABI (148).

Electron 43 raises NODE_MODULE_VERSION to 148; better-sqlite3@12.11.1 has no electron-v148 prebuild, so the packaged app died with 'Nenhum driver SQLite disponível'. prepare-electron-standalone now compiles better-sqlite3 from source against the electron headers into build/Release (where 'bindings' resolves it). Validated by Electron Package Smoke (green) + local (node_register_module_v148).

Supersedes #6378. (--admin: the only reds are SonarQube/SonarCloud failing on a coverage-report artifact digest-mismatch — a GitHub Actions infra flake, not this diff; Sonar is green on main and the diff touches only the electron build.)

* deps: bump the development group across 1 directory with 6 updates (#6588)

deps: bump the development group (6 updates). Rebased onto current main; all checks green after the electron-smoke fix (#6605).

* fix(proxy): force CONNECT tunnel for HTTP proxied requests (undici 8.7) + production deps bump (#6620)

fix(proxy): force CONNECT tunnel for HTTP proxied requests (undici 8.7) + production deps bump.

undici 8.6+ changed ProxyAgent to forward plain-HTTP via request-proxy instead of CONNECT, breaking OAuth refresh through a connection proxy (501). proxyDispatcher now passes proxyTunnel:true. Validated: Unit Tests 3/8 (the OAuth-proxy test) green, new regression test green (fails without the fix on undici 8.7), SonarQube green.

Supersedes #6380. (--admin: the only red is Electron Package Smoke failing on a next-build artifact 'digest-mismatch' — a GitHub Actions infra flake corrupting the asar ('file data stream has unexpected number of bytes'); the better-sqlite3 rebuild itself succeeded (gyp ok) and the electron path is unchanged from #6605 which passed the smoke. Not this diff.)

* fix(mimocode): handle 400 with cooldown + account rotation

Treat HTTP 400 responses the same as 429: mark the account on cooldown
and continue to the next fingerprint/proxy. Previously, 400 fell through
to markSuccess and returned immediately, so only 1 of N accounts was ever
tried per request.

Refs: #5925

* chore(mimocode): drop unrelated dependency/electron drift from PR #6648's stale fork

package.json/package-lock.json (bun/eslint-config-next/cyclonedx bumps), electron/package.json,
electron/package-lock.json, open-sse/utils/proxyDispatcher.ts, prepare-electron-standalone.mjs
and tests/unit/proxy-dispatcher-family.test.ts were already present in the contributor's single
commit but are unrelated to the mimocode 400-handling fix — restored to release/v3.8.47's
versions so the PR stays scoped to open-sse/executors/mimocode.ts.

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>

* fix(mimocode): classify 400 body before rotating — rate-limit-text 400s rotate, malformed 400s fail fast (#2101/#4976 guard)

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>

* refactor(mimocode): extract auth-retry + 429/400 gating helpers — keep execute() under the cognitive-complexity gate

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>

---------

Co-authored-by: Diego Rodrigues de Sa e Souza <8016841+diegosouzapw@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: pizzav-xyz <pizzav-xyz@users.noreply.github.com>
Co-authored-by: Diego Rodrigues de Sa e Souza <diegosouza.pw@gmail.com>
This commit is contained in:
PizzaV
2026-07-08 19:08:48 +02:00
committed by GitHub
parent d0c775564a
commit 2d9f85d587
4 changed files with 282 additions and 45 deletions

View File

@@ -43,6 +43,7 @@ _Living section — bullets land here as PRs merge into `release/v3.8.47` (paral
- **fix(startup):** an update/restart could crash the whole server at boot with `TypeError: Cannot create property 'message' on string 'Database closed'`, masking the real failure and 500-ing every request until manually restarted ([#6560](https://github.com/diegosouzapw/OmniRoute/issues/6560), plausibly the root cause of #6594's post-upgrade 500) — `driverFactory.ts::preInitSqlJs()` cached its sql.js WASM adapter per file path in a `globalThis`-backed map for idempotency, but never checked whether the cached adapter had since been closed (e.g. by `gracefulShutdown`/`resetDbInstance` racing a reload); reusing that dead handle made the very next query throw sql.js's own bare string `"Database closed"` (not an `Error`) straight out of `instrumentation-node.ts`'s previously-unguarded `ensureDbInitialized()` call, and Next.js's internal `registerInstrumentation()` wrapper unconditionally does `err.message = ...` on whatever `register()` rejects with — assigning `.message` on a primitive string throws in strict mode, so the secondary `TypeError` is what actually crashed the process. Fixed in two parts: `preInitSqlJs()` now evicts a closed cached adapter and creates a fresh one instead of returning it; a new `ensureDbReadyForBoot()` wraps the DB-init call, normalizes any non-Error throw via `normalizeBootError()`, and retries once specifically for a transient "database closed" message (now succeeding against the fresh adapter) before re-throwing anything else as a real `Error`. Regression guard: `tests/unit/instrumentation-database-closed-6560.test.ts`.
- **fix(api):** `POST /api/keys` no longer hangs 2090+ seconds on a fresh install, even with valid auth ([#6570](https://github.com/diegosouzapw/OmniRoute/issues/6570)) — `cloudEnabled` defaults to `true` in `src/lib/db/settings.ts::getSettings()` on any install with no persisted settings row (i.e. every fresh install), so the create-key handler's unconditional `await syncKeysToCloudIfEnabled()` always attempted a real outbound `fetch()` to `CLOUD_URL` via `syncToCloud()`; when that endpoint is unset/unreachable/slow, the HTTP response blocked until the request settled or timed out — unlike sibling routes (`POST /api/keys/:id/regenerate`, `GET /api/combos`), which never touch this side effect at all. `src/app/api/keys/route.ts` now dispatches `syncKeysToCloudIfEnabled()` fire-and-forget (`void`) instead of awaiting it; its internal try/catch already logs failures, so cloud sync still runs, it just no longer blocks the response. Regression guard: `tests/unit/api-keys-create-no-hang-6570.test.ts` (asserts the route resolves in well under 2s even when the Cloud-sync `fetch()` is stubbed to never settle) + updated `tests/integration/api-keys.test.ts` (the pre-existing "triggers cloud sync"/"still succeeds when cloud sync fails" tests, which previously hung indefinitely on this exact path, now await the fire-and-forget sync tick before asserting).
- **fix(api):** editing any existing OpenAI Codex provider connection in the dashboard returned "Invalid request" and the edit could never be saved ([#6562](https://github.com/diegosouzapw/OmniRoute/issues/6562)) — `createProviderConnection()` (`src/lib/db/providers.ts`) auto-increments a new connection's `priority` to `MAX(priority)+1` per provider with no upper bound, and OAuth-imported connections (Codex `codex-auth/import` / `import-bulk`, up to 50 accounts per call, callable repeatedly — the standard Codex bulk-account-rotation workflow) never pass through `createProviderSchema`'s Zod validation at all, so nothing ever capped that value; `EditConnectionModal`'s `handleSubmit` always resends the connection's current `priority` unchanged on every save, and `updateProviderConnectionSchema` capped `priority`/`globalPriority` at `max(100)` — a UI-only ceiling the create path never enforced — so the first edit of any connection whose priority had already grown past 100 (routine once a Codex account count exceeds 100) failed validation regardless of which field the user actually changed. Raised the ceiling to `max(100_000)` on both fields — still bounded (a genuinely out-of-range value is still rejected), just wide enough to accept priorities the app itself already produces. Regression guard: `tests/unit/codex-connection-edit-6562.test.ts` (a Codex OAuth connection whose priority already exceeds the old 100 cap now validates + persists on edit; a control payload with a still-genuinely-invalid priority is still rejected with "Invalid request").
- **mimocode**: rotate accounts on MiMoCode's rate-limit-style 400s (body-classified) instead of failing on the first account; malformed 400s still fail fast with the real upstream error (#6648 — thanks @pizzav-xyz)
### 📝 Maintenance

View File

@@ -12,13 +12,20 @@
*
* Only the "mimo-auto" model is supported (1M context, 128K output).
* Supports multiple accounts: N fingerprints → N JWTs → round-robin with cooldown.
* On 429, account enters cooldown (exponential backoff). On 401/403, JWT is re-bootstrapped.
* On 429 — or a 400 carrying MiMoCode's rate-limit text — account enters cooldown
* (exponential backoff) and the next account is tried. On 401/403, JWT is
* re-bootstrapped. Any other 400 is a genuinely malformed request (#2101): it fails
* fast on the current account instead of being retried identically on every
* account, which would waste N round-trips, cooldown every account, and hide the
* real upstream diagnostic behind a generic "all accounts exhausted" error (#4976).
*/
import * as crypto from "node:crypto";
import * as os from "node:os";
import { BaseExecutor, type ExecuteInput, type ProviderCredentials } from "./base.ts";
import { createProxyDispatcher } from "../utils/proxyDispatcher.ts";
import { RATE_LIMIT_TEXT_PATTERNS } from "../services/accountFallback.ts";
import { buildErrorBody, sanitizeErrorMessage } from "../utils/error.ts";
import { fetch as undiciFetch, type Dispatcher } from "undici";
const BOOTSTRAP_PATH = "/api/free-ai/bootstrap";
@@ -350,6 +357,127 @@ export class MimocodeExecutor extends BaseExecutor {
account.consecutiveFails = 0;
}
/**
* POST the request with the account's JWT; on auth failure (401/403), re-bootstrap
* the account's JWT and retry once. Mutates `headers`' Authorization in place.
*/
private async fetchWithAuthRetry(
url: string,
headers: Record<string, string>,
reqBody: unknown,
signal: AbortSignal | null | undefined,
account: AccountState,
log: ExecuteInput["log"]
): Promise<Response> {
const jwt = await this.getJwtForAccount(account, signal);
headers["Authorization"] = `Bearer ${jwt}`;
const resp = await this.fetchWithProxy(
url,
{
method: "POST",
headers,
body: JSON.stringify(reqBody),
signal: signal ?? undefined,
},
account.fingerprint
);
if (resp.status !== 401 && resp.status !== 403) return resp;
// On auth failure, re-bootstrap this account and retry once
log?.warn?.(
"MIMOCODE",
`Auth failed (${resp.status}) on account ${account.fingerprint.slice(0, 8)}`
);
account.jwt = "";
account.expiresAt = 0;
account.consecutiveFails = 0;
const freshJwt = await this.getJwtForAccount(account, signal);
headers["Authorization"] = `Bearer ${freshJwt}`;
return this.fetchWithProxy(
url,
{
method: "POST",
headers,
body: JSON.stringify(reqBody),
signal: signal ?? undefined,
},
account.fingerprint
);
}
/**
* Gate 429/400 statuses before the success path: a 429 — or a 400 carrying
* MiMoCode's rate-limit text — puts the account on cooldown and rotates; any other
* 400 fails fast with the sanitized upstream error (#2101/#4976, see
* handleBadRequest). Returns "rotate", a fail-fast Response, or null to proceed.
*/
private async gateRetryableStatus(
resp: Response,
account: AccountState,
log: ExecuteInput["log"]
): Promise<"rotate" | Response | null> {
if (resp.status === 429) {
this.markCooldown(account);
log?.warn?.(
"MIMOCODE",
`Rate limited on account ${account.fingerprint.slice(0, 8)}, trying next…`
);
return "rotate";
}
if (resp.status !== 400) return null;
return (await this.handleBadRequest(resp, account, log)) ?? "rotate";
}
/**
* Classify a 400 response body (#2101/#4976).
*
* #4976: MiMoCode signals throttling via a non-standard 400 whose body carries
* rate-limit semantics (e.g. "Detected high-frequency non-compliant requests from
* you.") instead of a 429 — same RATE_LIMIT_TEXT_PATTERNS as accountFallback.ts's
* checkFallbackError(), so the two call sites never disagree on what counts as
* throttling. That case puts the account on cooldown and returns `null` (rotate).
*
* #2101: any other 400 is a genuinely malformed request that fails identically on
* every account — rotating would waste N round-trips, cooldown every account (a
* provider-wide outage for parallel requests), and hide the real diagnostic behind
* a generic exhaustion error. That case returns a fail-fast 400 Response carrying
* the sanitized upstream message, without touching cooldown/success state.
*/
private async handleBadRequest(
resp: Response,
account: AccountState,
log: ExecuteInput["log"]
): Promise<Response | null> {
const bodyText = await resp.text().catch(() => "");
if (RATE_LIMIT_TEXT_PATTERNS.some((p) => p.test(bodyText))) {
this.markCooldown(account);
log?.warn?.(
"MIMOCODE",
`Rate-limit-style 400 on account ${account.fingerprint.slice(0, 8)}, trying next…`
);
return null;
}
log?.warn?.(
"MIMOCODE",
`Malformed request (400) on account ${account.fingerprint.slice(0, 8)}, not retrying`
);
let upstreamMessage = bodyText;
try {
const parsed = JSON.parse(bodyText) as { error?: { message?: string } };
if (parsed?.error?.message) upstreamMessage = parsed.error.message;
} catch {
/* body wasn't JSON — use raw text */
}
const errorBody = buildErrorBody(400, sanitizeErrorMessage(upstreamMessage || "Bad request"));
return new Response(MimocodeExecutor.encoder.encode(JSON.stringify(errorBody)), {
status: 400,
headers: { "Content-Type": "application/json" },
});
}
buildUrl(
_model: string,
_stream: boolean,
@@ -457,51 +585,19 @@ export class MimocodeExecutor extends BaseExecutor {
for (let attempt = 0; attempt < this.accounts.length; attempt++) {
const account = this.pickAccount();
try {
const jwt = await this.getJwtForAccount(account, signal);
const headers = this.buildHeaders(input.credentials, stream);
headers["Authorization"] = `Bearer ${jwt}`;
const resp = await this.fetchWithAuthRetry(url, headers, reqBody, signal, account, log);
let resp = await this.fetchWithProxy(
url,
{
method: "POST",
headers,
body: JSON.stringify(reqBody),
signal: signal ?? undefined,
},
account.fingerprint
);
// On auth failure, re-bootstrap this account and retry once
if (resp.status === 401 || resp.status === 403) {
log?.warn?.(
"MIMOCODE",
`Auth failed (${resp.status}) on account ${account.fingerprint.slice(0, 8)}`
);
account.jwt = "";
account.expiresAt = 0;
account.consecutiveFails = 0;
const freshJwt = await this.getJwtForAccount(account, signal);
headers["Authorization"] = `Bearer ${freshJwt}`;
resp = await this.fetchWithProxy(
// 429/400 gating (#2101/#4976): cooldown+rotate, fail fast, or proceed.
const gate = await this.gateRetryableStatus(resp, account, log);
if (gate === "rotate") continue;
if (gate) {
return {
response: gate,
url,
{
method: "POST",
headers,
body: JSON.stringify(reqBody),
signal: signal ?? undefined,
},
account.fingerprint
);
}
if (resp.status === 429) {
this.markCooldown(account);
log?.warn?.(
"MIMOCODE",
`Rate limited on account ${account.fingerprint.slice(0, 8)}, trying next…`
);
continue;
headers: this.buildHeaders(input.credentials, stream),
transformedBody: reqBody,
};
}
this.markSuccess(account);

View File

@@ -265,8 +265,8 @@ const MALFORMED_REQUEST_PATTERNS = [
// non-standard 400 status whose body carries rate-limit semantics instead of a 429
// (#4976). When detected, the request is fallback-worthy at connection-cooldown scope
// (NOT a whole-provider breaker) so combo routing can fail over to another free target.
// Bounded, non-overlapping patterns only (ReDoS-safe — no nested quantifiers).
const RATE_LIMIT_TEXT_PATTERNS = [
// Exported: mimocode.ts's executor reuses this list directly (single source of truth).
export const RATE_LIMIT_TEXT_PATTERNS = [
/high.?frequency/i,
/non-compliant/i,
/too many requests/i,

View File

@@ -485,3 +485,143 @@ describe("mimocode per-account proxy", () => {
assert.strictEqual((testExec as any).proxyUrlMap.get(fp), "socks5://second.proxy:1080");
});
});
// #2101/#4976 regression guard: a 400 from MiMoCode must be classified by body text
// before deciding whether to rotate accounts. A rate-limit-style 400 (throttling
// disguised as a 400, #4976) is rotation-worthy; a genuinely malformed 400 (#2101)
// must fail fast on the FIRST account instead of being retried identically on every
// account (which would waste N round-trips, cooldown every account, and hide the
// real upstream diagnostic behind a generic "all accounts exhausted" error).
interface TestAccountState {
fingerprint: string;
jwt: string;
expiresAt: number;
cooldownUntil: number;
consecutiveFails: number;
}
interface ExecutorAccountAccess {
accounts: TestAccountState[];
nextAccountIdx: number;
}
function accountAccess(exec: MimocodeExecutor): ExecutorAccountAccess {
return exec as unknown as ExecutorAccountAccess;
}
describe("mimocode 400 classification (#2101/#4976)", () => {
function makeJwt(): string {
const header = Buffer.from(JSON.stringify({ alg: "none" })).toString("base64url");
const payload = Buffer.from(
JSON.stringify({ exp: Math.floor(Date.now() / 1000) + 3600 })
).toString("base64url");
return `${header}.${payload}.sig`;
}
function twoAccountExecutor(): MimocodeExecutor {
const exec = new MimocodeExecutor();
const access = accountAccess(exec);
access.accounts = [
{ fingerprint: "acct-a", jwt: "", expiresAt: 0, cooldownUntil: 0, consecutiveFails: 0 },
{ fingerprint: "acct-b", jwt: "", expiresAt: 0, cooldownUntil: 0, consecutiveFails: 0 },
];
access.nextAccountIdx = 0;
return exec;
}
it("rotates to the next account on a rate-limit-text 400 (#4976)", async () => {
const testExec = twoAccountExecutor();
let chatCalls = 0;
const originalFetch = globalThis.fetch;
globalThis.fetch = (async (url: unknown) => {
const urlStr = String(url);
if (urlStr.includes("/api/free-ai/bootstrap")) {
return new Response(JSON.stringify({ jwt: makeJwt() }), { status: 200 });
}
if (urlStr.includes("/api/free-ai/openai/chat")) {
chatCalls++;
if (chatCalls === 1) {
// MiMoCode's non-standard rate-limit signal on a 400 status (#4976).
return new Response(
JSON.stringify({
error: { message: "Detected high-frequency non-compliant requests from you." },
}),
{ status: 400 }
);
}
return new Response(JSON.stringify({ id: "ok", choices: [] }), { status: 200 });
}
throw new Error(`unexpected fetch: ${urlStr}`);
}) as typeof fetch;
try {
const result = await testExec.execute({
model: "mimo-auto",
body: { messages: [{ role: "user", content: "hi" }], stream: false },
stream: false,
signal: null,
credentials: {},
log: { debug: () => {}, info: () => {}, warn: () => {}, error: () => {} },
});
assert.strictEqual(chatCalls, 2, "should retry on the next account after the 400");
assert.strictEqual(result.response.status, 200);
const acctA = accountAccess(testExec).accounts[0];
assert.ok(acctA.cooldownUntil > Date.now(), "first account should be in cooldown");
assert.strictEqual(acctA.consecutiveFails, 1);
} finally {
globalThis.fetch = originalFetch;
}
});
it("fails fast without rotating on a malformed/generic 400 (#2101)", async () => {
const testExec = twoAccountExecutor();
let chatCalls = 0;
const originalFetch = globalThis.fetch;
globalThis.fetch = (async (url: unknown) => {
const urlStr = String(url);
if (urlStr.includes("/api/free-ai/bootstrap")) {
return new Response(JSON.stringify({ jwt: makeJwt() }), { status: 200 });
}
if (urlStr.includes("/api/free-ai/openai/chat")) {
chatCalls++;
return new Response(
JSON.stringify({ error: { message: "Invalid field: foo is not a recognized field" } }),
{ status: 400 }
);
}
throw new Error(`unexpected fetch: ${urlStr}`);
}) as typeof fetch;
try {
const result = await testExec.execute({
model: "mimo-auto",
body: { messages: [{ role: "user", content: "hi" }], stream: false },
stream: false,
signal: null,
credentials: {},
log: { debug: () => {}, info: () => {}, warn: () => {}, error: () => {} },
});
assert.strictEqual(chatCalls, 1, "must NOT rotate to another account on a malformed 400");
const acctA = accountAccess(testExec).accounts[0];
assert.strictEqual(acctA.cooldownUntil, 0, "malformed 400 must not trigger cooldown");
assert.strictEqual(acctA.consecutiveFails, 0);
const response = result.response;
assert.strictEqual(response.status, 400);
const parsed = (await response.json()) as { error: { message: string; code?: string } };
assert.notStrictEqual(
parsed.error.code,
"NO_ACCOUNTS",
"must surface the real upstream 400, not a generic exhaustion error"
);
assert.ok(
parsed.error.message.toLowerCase().includes("invalid field"),
`expected the real upstream diagnostic in the error message, got: ${parsed.error.message}`
);
} finally {
globalThis.fetch = originalFetch;
}
});
});