Files
OmniRoute/open-sse/executors/opencode.ts
Dizzle 94cf4c402a fix(executors): rotate to the next account on network throws when the account has a dedicated proxy (#10402)
OpencodeExecutor and MimocodeExecutor rotated to the next account only on
HTTP 429. A network exception (timeout, connection refused/reset) on one
account instead propagated out of execute() and failed the whole request,
even when other accounts remained available.

Both executors now rotate on a network exception only when the failed
account has its own dedicated proxy (account.proxy !== null) — a dead
proxy is genuinely account-scoped, so rotating away from it is safe.
Accounts sharing the default egress (no proxy configured) trigger the
same cooldown and are skipped for the rest of the request once the shared
egress is known down, but a later account with its own dedicated proxy is
still tried normally — a throw on a proxy-less account no longer strands
a proxied account further in the rotation. This behavior is gated behind
NETWORK_ROTATION_SHARED_EGRESS_GUARD (Feature Flag, default on); disabled,
it reproduces the immediate-propagation behavior this fix started from.

The shared rotation mechanics (pickAccount/markCooldown/markSuccess) are
extracted into executors/accountRotation.ts, used by both executors —
they had independently implemented the same round-robin+cooldown
skeleton. This also fixes an identical, pre-existing bug in
MimocodeExecutor that predates this PR: its catch block called
markCooldown unconditionally on any throw, with no proxy check and no
warn log (a silent exception swallow on a path that influences the
result).

The cooldown formula for both the proxy and shared-egress cases reuses
the repo's already-established "transient, not clearly attributable"
constants (errorConfig.ts TRANSIENT_COOLDOWN_MS/COOLDOWN_MS.transientMax,
already used by accountFallback.ts for network-error classification)
instead of introducing a separate value.

MimocodeExecutor's network-error 502 body also now goes through
buildErrorBody()/sanitizeErrorMessage() instead of embedding the raw
caught error message directly (Hard Rule #12), matching the sanitization
already used on its #2101 malformed-request path.

Validated by TDD (Hard Rule #18): tests/unit/account-rotation.test.ts
covers the shared module directly; opencode-proxy-rotation-4954.test.ts
and mimocode-executor.test.ts cover the proxy-configured rotation path,
the mixed-fleet case, the shared-egress single-network-call case, and the
NETWORK_ROTATION_SHARED_EGRESS_GUARD-disabled legacy path, for each
executor. tsc, lint, and the provider golden-path gates
(check:provider-consistency, check:provider-assets,
provider-translate-path-golden.test.ts) are clean on all touched files.

Co-authored-by: Max <maxmad64@gmail.com>
2026-08-16 00:16:04 -03:00

564 lines
22 KiB
TypeScript

import { BaseExecutor, type ExecuteInput, type ProviderCredentials } from "./base.ts";
import { PROVIDERS } from "../config/constants.ts";
import { getModelTargetFormat } from "../config/providerModels.ts";
import {
injectReasoningContentForThinkingModel,
isThinkingMessageModel,
} from "../utils/reasoningContentInjector.ts";
import { runWithProxyContext } from "../utils/proxyFetch.ts";
import { forwardOpencodeClientHeaders } from "../utils/opencodeHeaders.ts";
import {
type AccountProxyConfig,
type RotatableAccount,
pickAccount as pickRotatableAccount,
markCooldown as markAccountCooldown,
markSuccess as markAccountSuccess,
maskAccountId,
isNetworkErrorRotatable,
} from "./accountRotation.ts";
import { isNetworkRotationSharedEgressGuardEnabled } from "@/shared/utils/featureFlags";
/**
* Per-account proxy configuration, persisted by NoAuthAccountCard under
* `providerSpecificData.accountProxies` (keyed by the account id, which the UI
* stores in `providerSpecificData.fingerprints`). Same shape mimocode uses.
*/
export type OpencodeAccountProxyConfig = AccountProxyConfig;
/** Runtime rotation/cooldown state for one "OpenCode Free" account. */
interface OpencodeAccountState extends RotatableAccount {
/** Account id (UI: providerSpecificData.fingerprints[i]); "" for the default direct account. */
fingerprint: string;
}
const EFFORT_LEVELS = ["low", "medium", "high", "max"] as const;
/**
* Models that work WITHOUT any API key on the free/noauth opencode tier.
*
* The upstream free tier rotates frequently — when a `-free` suffix model is
* delisted upstream, the upstream returns "Model X is not supported" (a separate
* issue from this gate). The set is defined by two data sources:
*
* 1. **Known free models** — models explicitly listed in the noauth
* `opencode` provider registry (`open-sse/config/providers/registry/opencode/index.ts`).
* These are the canonical free models. `deepseek-v4-flash-free` appears in both
* the noauth AND the zen registry (it is free on both tiers).
* 2. **`-free` suffix** — any model whose id ends in `-free`. This automatically
* covers upstream free-tier additions without a code deploy.
*
* For `opencode-go`, there is no free tier — ALL models require an API key.
*/
const OPENCODE_FREE_MODELS = new Set([
"big-pickle",
"deepseek-v4-flash-free",
"mimo-v2.5-free",
"hy3-free",
"nemotron-3-ultra-free",
"north-mini-code-free",
]);
/**
* Models on opencode-go that support effort-tier aliases. Each entry maps the
* canonical base id to the set of effort suffixes the upstream supports.
*
* - deepseek-v4-pro: all four tiers (low/medium/high/max)
* - glm-5.2: high/max only (Z.AI maps these through the reasoning plane;
* low/medium are not supported on the OpenAI transport)
* - mimo-v2.5: high/max only (same reasoning; Xiaomi MiMo does not document
* low/medium effort tiers)
* - #8353 OpenCode Go registry effort variants (exact suffix sets from
* `opencode models opencode-go --verbose`; MiniMax M3 excluded — different
* thinking-mode mapping):
* deepseek-v4-flash high/max; grok-4.5 low/medium/high; hy3 none/low/high;
* kimi-k3 max; qwen3.6-plus / qwen3.7-max / qwen3.7-plus high/max
*/
const EFFORT_TIERS: Record<string, readonly string[]> = {
"deepseek-v4-pro": EFFORT_LEVELS,
"deepseek-v4-flash": ["high", "max"],
"glm-5.2": ["high", "max"],
"mimo-v2.5": ["high", "max"],
"grok-4.5": ["low", "medium", "high"],
hy3: ["none", "low", "high"],
"kimi-k3": ["max"],
"qwen3.6-plus": ["high", "max"],
"qwen3.7-max": ["high", "max"],
"qwen3.7-plus": ["high", "max"],
};
/**
* Parse a model string with an effort-level suffix.
* e.g. "deepseek-v4-pro-low" → { baseModel: "deepseek-v4-pro", effort: "low" }
* "glm-5.2-high" → { baseModel: "glm-5.2", effort: "high" }
* Returns null if the model doesn't match any known effort-tier pattern.
*/
export function parseEffortLevel(model: string): { baseModel: string; effort: string } | null {
const m = String(model || "");
for (const [baseModel, levels] of Object.entries(EFFORT_TIERS)) {
for (const level of levels) {
if (m === `${baseModel}-${level}`) {
return { baseModel, effort: level };
}
}
}
return null;
}
/**
* Determine whether a model requires an API key on the given opencode provider.
*
* - `opencode-go`: ALL models require a key (no free tier).
* - `opencode` / `opencode-zen`: premium = any model NOT in the free set (known
* free models OR ending in `-free`).
* - Unknown models are assumed premium (fail-safe).
*/
export function isPremiumOpencodeModel(model: string, provider: string): boolean {
// opencode-go has no free tier — every model requires a key.
if (provider === "opencode-go") return true;
// Models ending in `-free` are always free on the noauth/zen tier.
if (model.endsWith("-free")) return false;
// Check the known free model catalog.
return !OPENCODE_FREE_MODELS.has(model);
}
export class OpencodeExecutor extends BaseExecutor {
/** Delegates to `isPremiumOpencodeModel`. Exported for testability. */
static isPremiumModel(model: string, provider: string): boolean {
return isPremiumOpencodeModel(model, provider);
}
_requestFormat: string | null = null;
/**
* Per-account rotation state, rebuilt from credentials on each request. The
* default entry (fingerprint "") represents the single anonymous account with
* no configured proxy — preserves the historical direct pass-through when the
* user has not configured any per-account proxy.
*/
private accounts: OpencodeAccountState[] = [
{ fingerprint: "", cooldownUntil: 0, consecutiveFails: 0, proxy: null },
];
// Not `private`: passed as the mutable rotation cursor to the shared
// pickAccount() helper, which needs a plain `{ nextAccountIdx }` shape —
// TS's private-member nominal check rejects `this` there otherwise.
nextAccountIdx = 0;
constructor(provider: string) {
super(provider, PROVIDERS[provider] || PROVIDERS.openai);
}
/**
* Rebuild `accounts` from `providerSpecificData.fingerprints` +
* `providerSpecificData.accountProxies`. Each configured account id becomes a
* rotation slot carrying its own proxy. When the user configured no accounts
* at all, the single default direct account is kept (backward compatible).
*/
private syncAccountsFromCredentials(credentials: ProviderCredentials): void {
const psd = credentials?.providerSpecificData;
const fingerprints = Array.isArray(psd?.fingerprints)
? (psd!.fingerprints as unknown[]).filter((f): f is string => typeof f === "string")
: [];
const accountProxies = psd?.accountProxies as OpencodeAccountProxyConfig[] | undefined;
const proxyMap = Array.isArray(accountProxies)
? new Map(accountProxies.map((ap) => [ap.fingerprint, ap.proxy ?? null] as const))
: null;
if (fingerprints.length === 0) {
// No configured accounts — keep a single direct account.
this.accounts = [{ fingerprint: "", cooldownUntil: 0, consecutiveFails: 0, proxy: null }];
this.nextAccountIdx = 0;
return;
}
const previous = new Map(this.accounts.map((a) => [a.fingerprint, a] as const));
this.accounts = fingerprints.map((fp) => {
const prior = previous.get(fp);
return {
fingerprint: fp,
cooldownUntil: prior?.cooldownUntil ?? 0,
consecutiveFails: prior?.consecutiveFails ?? 0,
proxy: proxyMap ? (proxyMap.get(fp) ?? null) : null,
};
});
if (this.nextAccountIdx >= this.accounts.length) this.nextAccountIdx = 0;
}
/** Round-robin pick, skipping accounts in cooldown; falls back to the next index. */
private pickAccount(): OpencodeAccountState {
return pickRotatableAccount(this.accounts, this);
}
private markCooldown(account: OpencodeAccountState): void {
markAccountCooldown(account);
}
private markSuccess(account: OpencodeAccountState): void {
markAccountSuccess(account);
}
async execute(input: ExecuteInput) {
this._requestFormat = getModelTargetFormat(this.provider, input.model) || "openai";
// #8681: Gate premium opencode models behind a usable API key.
// When the connection is keyless (no apiKey, no accessToken) and the model
// is a premium model (not on the free tier), return a clear 402 error
// instead of proxying the raw upstream 401 "Missing API key" response.
const creds = input.credentials;
const isKeyless =
!creds?.apiKey && !creds?.accessToken && !creds?.providerSpecificData?.extraApiKeys;
if (isKeyless && isPremiumOpencodeModel(input.model, this.provider)) {
const bodyJson = JSON.stringify({
error: {
message: "This model requires an opencode API key — add one in Settings → Providers.",
type: "invalid_request_error",
code: "premium_model_requires_key",
},
});
return {
response: new Response(bodyJson, {
status: 402,
headers: { "Content-Type": "application/json" },
}),
url: "",
headers: {} as Record<string, string>,
transformedBody: null,
};
}
try {
this.syncAccountsFromCredentials(input.credentials);
const hasProxies = this.accounts.some((a) => a.proxy !== null);
// Fast path: no multi-account proxy wiring configured → original behavior.
if (this.accounts.length === 1 && !hasProxies) {
return await super.execute(input);
}
const { log } = input;
// This loop only ever dispatches through super.execute() (the HTTP request
// path), which always resolves the object-shaped arm of ExecutorExecuteResult
// — the bare-Response arm belongs to web/scraping executors only (base.ts:290).
type HttpExecuteResult = Extract<
Awaited<ReturnType<BaseExecutor["execute"]>>,
{ response: Response }
>;
let lastResult: HttpExecuteResult | null = null;
let lastSharedEgressError: unknown = null;
const sharedEgressGuardEnabled = isNetworkRotationSharedEgressGuardEnabled();
// Set once a proxy-less account's network throw reveals the shared
// egress is down (see NETWORK_ROTATION_SHARED_EGRESS_GUARD below) —
// subsequent proxy-less accounts this request are skipped without a
// network call, but proxied accounts (independent egress) are still
// tried normally.
let sharedEgressDown = false;
for (let attempt = 0; attempt < this.accounts.length; attempt++) {
const account = this.pickAccount();
const masked = maskAccountId(account.fingerprint);
if (sharedEgressGuardEnabled && sharedEgressDown && !account.proxy) {
log?.warn?.(
"OPENCODE",
`skipping account ${masked} (no dedicated proxy, shared egress already down this request)`
);
continue;
}
// #5217 (Gap 2): promoted debug→info so the per-request account/proxy
// rotation selection is visible in the Console log view at the default
// APP_LOG_LEVEL=info (users could not see which account/proxy was used).
// Token stays masked — never log the full account id.
log?.info?.(
"OPENCODE",
`dispatch via account ${masked} (idx ${attempt + 1}/${this.accounts.length})` +
(account.proxy
? ` through proxy ${account.proxy.host}:${account.proxy.port}`
: " direct")
);
// Pin egress to this account's proxy for the whole BaseExecutor dispatch
// (incl. its intra-URL 429 retries). skipUpstreamRetry lets THIS loop own
// the cross-account 429 fallback instead of BaseExecutor's same-key retry.
let result: HttpExecuteResult;
try {
// super.execute() here always dispatches the HTTP path (opencode is an
// OpenAI-compatible API, never the web/scraping bare-Response arm) —
// see base.ts:290-294.
result = (await runWithProxyContext(account.proxy, () =>
super.execute({ ...input, skipUpstreamRetry: true })
)) as HttpExecuteResult;
} catch (err) {
const reason = err instanceof Error ? err.message : String(err);
// A network exception (timeout, connection refused/reset) is only
// account-scoped when this account has its OWN egress (a configured
// proxy) — that's the case a dead/unreachable proxy justifies rotating
// away from. Without a proxy, accounts share the same network egress:
// the failure isn't attributable to this account. Never swallowed
// silently either way: logged before rotating, skipping, or rethrowing.
if (!isNetworkErrorRotatable(account)) {
if (sharedEgressGuardEnabled) {
this.markCooldown(account);
sharedEgressDown = true;
lastSharedEgressError = err;
log?.warn?.(
"OPENCODE",
`network error on account ${masked} (no dedicated proxy, shared egress), cooldown applied — trying next available account… (${reason})`
);
continue;
}
log?.warn?.(
"OPENCODE",
`network error on account ${masked} (no dedicated proxy, shared egress) — not rotating (${reason})`
);
throw err;
}
this.markCooldown(account);
log?.warn?.(
"OPENCODE",
`network error on account ${masked}, rotating to next… (${reason})`
);
continue;
}
lastResult = result;
const status = result.response.status;
if (status === 429) {
this.markCooldown(account);
log?.warn?.("OPENCODE", `Rate limited (429) on account ${masked}, rotating to next…`);
continue;
}
this.markSuccess(account);
return result;
}
// The loop exhausted without a result. If it's because every remaining
// proxy-less account was skipped once the shared egress was known down
// (rather than actually tried), propagate that original throw — an
// extra direct call here would just be a second doomed attempt against
// the same dead path, which is exactly the latency this guard exists
// to avoid (see NETWORK_ROTATION_SHARED_EGRESS_GUARD).
if (sharedEgressDown && !lastResult && lastSharedEgressError !== null) {
throw lastSharedEgressError;
}
// All accounts returned 429 (or errored) — surface the last response.
return lastResult ?? (await super.execute(input));
} finally {
this._requestFormat = null;
}
}
buildUrl(
model: string,
stream: boolean,
urlIndex = 0,
credentials: ProviderCredentials | null = null
) {
void urlIndex;
void credentials;
const base = this.config.baseUrl;
switch (this._requestFormat) {
case "claude":
return `${base}/messages`;
case "openai-responses":
return `${base}/responses`;
case "gemini":
return `${base}/models/${model}:${stream ? "streamGenerateContent?alt=sse" : "generateContent"}`;
default:
return `${base}/chat/completions`;
}
}
buildHeaders(
credentials: ProviderCredentials | null,
stream = true,
clientHeaders?: Record<string, string> | null,
model?: string
) {
const headers: Record<string, string> = { "Content-Type": "application/json" };
// #8467: honor Extra API Keys rotation via BaseExecutor.resolveEffectiveKey.
// Fall back to accessToken only when no apiKey/extras resolve to a key.
const key = credentials
? this.resolveEffectiveKey(credentials) || credentials.accessToken
: undefined;
if (key) {
if (this._requestFormat === "claude") {
headers["x-api-key"] = key;
} else {
headers["Authorization"] = `Bearer ${key}`;
}
}
if (this._requestFormat === "claude") {
headers["anthropic-version"] = "2023-06-01";
}
if (stream) {
headers["Accept"] = "text/event-stream";
}
// Opt-in (#5997): synthesize OpenCode CLI identity headers the client did not send.
// Cloudflare in front of opencode.ai/zen/go 403s server-side (VPS) requests lacking
// CLI identity, but the forward-only default is deliberate — fabricating a WRONG
// value risks upstream rejection (#5720 regressed with "opencode/local"), and this
// is deployment-specific. So it stays OFF by default and the VPS operator enables it
// with OPENCODE_SYNTHESIZE_CLI_HEADERS=true (values env-overridable). Client-supplied
// headers take precedence, EXCEPT User-Agent: a non-CLI client UA (curl/SDK) is
// replaced with the synthesized CLI UA because opencode.ai's free tier rejects
// generic client UAs from datacenter IPs (FreeUsageLimitError 429).
const synthesizeCli = /^(1|true|yes|on)$/i.test(
process.env.OPENCODE_SYNTHESIZE_CLI_HEADERS?.trim() ?? ""
);
const cliDefaults = synthesizeCli
? (() => {
const providerId = this.config?.id || this.provider || "opencode";
const envUAKey = `${providerId.toUpperCase().replace(/[^A-Z0-9]/g, "_")}_USER_AGENT`;
return {
userAgent:
process.env[envUAKey]?.trim() ||
process.env.OPENCODE_USER_AGENT?.trim() ||
"opencode-cli/1.0.0",
client: process.env.OPENCODE_CLIENT?.trim() || "cli",
project: process.env.OPENCODE_PROJECT?.trim() || "default",
};
})()
: undefined;
if (clientHeaders || cliDefaults) {
forwardOpencodeClientHeaders(headers, clientHeaders ?? {}, {
synthesizeRequestId: true,
cliDefaults,
});
}
void model;
return headers;
}
/**
* OpenCode's free DeepSeek V4 Flash endpoint accepts json_object but
* rejects json_schema response_format with HTTP 400. Preserve the schema
* as an instruction and downgrade only this proven-incompatible route to
* json_object so callers still receive structured JSON.
*/
private applyDeepSeekJsonSchemaFallback<T>(model: string, body: T): T {
if (
model !== "deepseek-v4-flash-free" ||
(this.provider !== "opencode" && this.provider !== "opencode-zen")
) {
return body;
}
if (!body || typeof body !== "object" || Array.isArray(body)) {
return body;
}
const record = body as Record<string, unknown>;
const responseFormat = record.response_format as
| {
type?: string;
json_schema?: {
schema?: unknown;
};
}
| undefined;
if (responseFormat?.type !== "json_schema" || !responseFormat.json_schema?.schema) {
return body;
}
const schemaJson = JSON.stringify(responseFormat.json_schema.schema, null, 2);
const prompt =
"You must respond with valid JSON that strictly follows " +
"this JSON schema:\\n```json\\n" +
schemaJson +
"\\n```\\nRespond ONLY with the JSON object, no other text.";
const messages: Array<Record<string, unknown>> = Array.isArray(record.messages)
? (record.messages as Array<Record<string, unknown>>).map((message) => ({ ...message }))
: [];
const systemMessage = messages.find((message) => message.role === "system");
if (systemMessage) {
if (typeof systemMessage.content === "string") {
systemMessage.content = `${systemMessage.content}\\n\\n${prompt}`;
} else if (Array.isArray(systemMessage.content)) {
systemMessage.content.push({
type: "text",
text: `\\n\\n${prompt}`,
});
}
} else {
messages.unshift({
role: "system",
content: prompt,
});
}
return {
...record,
messages,
response_format: {
type: "json_object",
},
} as T;
}
transformRequest(
model: string,
body: any,
stream: boolean,
credentials: ProviderCredentials
): any {
let modifiedBody = super.transformRequest(model, body, stream, credentials);
modifiedBody = this.applyDeepSeekJsonSchemaFallback(model, modifiedBody);
// 9router#1442: OpenCode upstreams (e.g. kimi-k2.6 via opencode-go) return
// 400 "Extra inputs are not permitted, field: 'client_metadata'" — an
// OpenAI-Codex/Claude-CLI passthrough field with no equivalent here. The
// DefaultExecutor strip only covers cerebras/mistral, and OpencodeExecutor
// extends BaseExecutor directly, so nothing removed it on this path.
if (
modifiedBody &&
typeof modifiedBody === "object" &&
!Array.isArray(modifiedBody) &&
Object.prototype.hasOwnProperty.call(modifiedBody, "client_metadata")
) {
delete (modifiedBody as Record<string, unknown>).client_metadata;
}
if (modifiedBody && typeof modifiedBody === "object" && !Array.isArray(modifiedBody)) {
const mb = modifiedBody as Record<string, unknown>;
if (Array.isArray(mb.tools) && mb.tools.length > 128) {
mb.tools = mb.tools.slice(0, 128);
}
}
if (modifiedBody && typeof modifiedBody === "object" && !Array.isArray(modifiedBody)) {
const mb = modifiedBody as Record<string, unknown>;
const parsed = parseEffortLevel(model);
if (parsed) {
mb.model = parsed.baseModel;
if (mb.reasoning_effort === undefined) {
mb.reasoning_effort = parsed.effort;
}
}
}
// #1543 / upstream PR #1099: thinking-mode upstreams routed through OpenCode
// (DeepSeek V4 Flash, Kimi, MiniMax, ...) require reasoning_content echoed
// back on assistant messages, or they 400 with "reasoning_content must be
// passed back". OpenAI clients drop it across turns, so we inject a
// placeholder for the affected model families.
if (isThinkingMessageModel(model)) {
modifiedBody = injectReasoningContentForThinkingModel(modifiedBody);
}
return modifiedBody;
}
}