Compare commits

...

4 Commits

Author SHA1 Message Date
Markus Hartung
e6cb82b27b fix(providers): scope Antigravity mitmAlias tier ids to the safe static alias (#11824) 2026-08-29 05:22:39 -03:00
Tobias Andersen
d8879371ea fix(combo): lock GitHub models rejected as "not supported" for future requests (#11781)
Follow-up to #11762/#11774, same bug class in combo's own model-lockout wiring: GitHub rejects several models (gpt-5.4, gpt-5.3-codex, etc.) with a 400 that's permanently unavailable for this account's Copilot integration, but nothing recorded a cross-request lockout — combo's #5249 in-request advance guard is correct but doesn't persist, so the same doomed model gets retried from scratch on every new request, indefinitely.

Fix: on a model-scoped 400 (`isModelScoped400`), call `lockModelIfPerModelQuota(provider, connectionId, rawModel, "model_capacity", 1h)`. GitHub already has per-model-quota enabled, so only the rejected model locks — siblings keep working. `isModelLocked()` is already checked pre-dispatch, so no other wiring needed.

Validated: 3/3 new tests + fixed a pre-existing test-isolation gap in combo-model-scoped-400-advance.test.ts (shared model name across sub-tests without clearing lockout state). Thanks!
2026-08-29 05:21:50 -03:00
Tobias Andersen
91f9a01fda fix(resilience): stop hammering permanently-moved endpoints and billing-suspended accounts (#11774)
Follow-up to #11762, same bug class hitting freeaiapikey (410 permanently-moved endpoint) and fireworks (412 billing-suspension) — both fell through checkFallbackError's generic transient-cooldown branch and got retried every ~1 minute for a full day.

Fix: `ENDPOINT_PERMANENTLY_MOVED_PATTERNS`/`isEndpointPermanentlyMoved()` → 24h lockout; `ACCOUNT_SUSPENDED_BILLING_PATTERNS`/`isAccountSuspendedForBilling()` → treated as credits-exhausted (1h cooldown), independent of status code so it also catches Fireworks' 412.

#11762 landed first and touched the same file — rebased/re-merged onto the updated tip (additive, no logic changes) and re-validated: 13/13 tests pass. Thanks for tracing this with real production logs again!
2026-08-29 05:19:15 -03:00
Tobias Andersen
87b3bdf85e fix(resilience): lock permanently retired models instead of short backoff (Gemini ban prevention) (#11762)
Root-caused via a real Gemini-ban incident log: deprecated-model 404/410s (e.g. gemini-2.5-flash "no longer available to new users") fell through checkFallbackError's generic transient-cooldown branch, so combo/auto-routing kept re-selecting a permanently dead model every cooldown window forever — the hammering that got the account flagged as abusive.

Fix: `MODEL_PERMANENTLY_UNAVAILABLE_PATTERNS` + `isModelPermanentlyUnavailable()` classify these as a 24h lockout instead, surfaced via `quotaResetHintMs` so combo's per-request model-lockout honors it in full.

Validated: 6/6 new tests + 133/133 existing accountFallback/error-classification tests, no regressions. Thanks for tracing this end-to-end with real production logs!
2026-08-29 05:09:37 -03:00
10 changed files with 484 additions and 22 deletions

View File

@@ -0,0 +1 @@
- **fix(providers):** Antigravity's dynamic mitmAlias table no longer routes `gemini-3.7-flash-{high,medium,low}` to a literal tier-suffixed upstream id just because one connected account's own discovery listed it directly — those display ids always resolve through the safe `gemini-3.7-flash-tiered` static alias, so one account's Google-provisioned access no longer 404s every sibling account of the provider ([#11824](https://github.com/diegosouzapw/OmniRoute/issues/11824), [#11651](https://github.com/diegosouzapw/OmniRoute/issues/11651))

View File

@@ -249,6 +249,59 @@ export const OAUTH_INVALID_TOKEN_SIGNALS = [
"invalid credentials",
];
// A model that upstream has permanently retired — Gemini's deprecated-model 404
// ("This model models/gemini-2.5-flash is no longer available to new users...")
// and Fireworks/OpenAI-compatible "end of life" 410s ("has reached its end of
// life ... and is no longer available") — will 404/410 on EVERY future request;
// no cooldown short enough to retry soon is ever correct. Without this check
// these fall through to the generic "all other errors" branch at the bottom of
// checkFallbackError, which only applies a short (seconds-to-minutes) transient
// cooldown, so combo/auto-routing keeps re-selecting the dead model roughly
// every cooldown window, forever — wasted upstream calls that, at volume, look
// like abusive traffic to the provider (observed: a Gemini free-tier key
// retried `gemini-2.5-flash`/`gemini-2.5-flash-lite` every ~15-45 minutes for a
// full day). Matched independent of MODEL_ACCESS_DENIED_PATTERNS below because
// those only fire for status 400; this needs to catch the far more common
// 404/410 status a retired model actually returns.
export const MODEL_PERMANENTLY_UNAVAILABLE_PATTERNS = [
/\bno longer available\b/i,
/\bno longer supported\b/i,
/\bhas reached (?:its |the )?end.?of.?life\b/i,
/\bmodel[\s\S]{0,40}?\b(?:deprecated|retired|discontinued|decommissioned)\b/i,
/\b(?:deprecated|retired|discontinued|decommissioned)[\s\S]{0,40}?\bmodel\b/i,
];
// A provider that has permanently retired its API base_url — the old endpoint
// keeps returning 410/404 on EVERY future request until the connection's
// base_url is updated by an operator; no cooldown short enough to retry soon
// is ever correct. Without this check these fall through to the generic
// "all other errors" branch, which only applies a short (seconds-to-minutes)
// transient cooldown, so combo/auto-routing keeps re-selecting the dead
// endpoint roughly every cooldown window, forever — wasted upstream calls
// that, at volume, look like abusive traffic (observed: freeaiapikey's moved
// endpoint retried every ~1 minute for a full day: "This API endpoint has
// moved. Please update your base_url to https://api.freeaiapikey.com/v1 —
// the old endpoint on freeaiapikey.com no longer works.").
export const ENDPOINT_PERMANENTLY_MOVED_PATTERNS = [
/\bendpoint has moved\b/i,
/\bno longer works\b/i,
/\bupdate your base.?url\b/i,
];
// A billing/account suspension that requires manual operator action (unpaid
// invoice, spending limit) — text varies per provider/account name, e.g.
// Fireworks: "Account hummern is suspended, possibly due to reaching the
// monthly spending limit or failure to pay past invoices." This does not
// match ACCOUNT_DEACTIVATED_SIGNALS' fixed "your account has been suspended"
// substring, and several providers surface it on a status (412) that
// checkFallbackError does not otherwise classify — so it fell through to the
// generic transient-error branch and got retried every few minutes, all day,
// against an account that cannot succeed until billing is fixed.
export const ACCOUNT_SUSPENDED_BILLING_PATTERNS = [
/\bsuspended\b[\s\S]{0,120}?\b(?:spending limit|billing|invoice|payment)\b/i,
/\b(?:spending limit|billing|invoice|payment)\b[\s\S]{0,120}?\bsuspended\b/i,
];
// Context overflow patterns — the prompt exceeds the model's maximum context length.
// Different providers phrase this differently. Used to decide whether a 400 error
// should trigger combo fallback (a different model may have a larger context window).
@@ -421,6 +474,34 @@ export function isCreditsExhausted(errorText: string): boolean {
return CREDITS_EXHAUSTED_SIGNALS.some((sig) => lower.includes(sig));
}
/**
* Returns true if the response body indicates the requested model has been
* permanently retired by the provider (see MODEL_PERMANENTLY_UNAVAILABLE_PATTERNS).
*/
export function isModelPermanentlyUnavailable(errorText: string): boolean {
const text = String(errorText || "");
return MODEL_PERMANENTLY_UNAVAILABLE_PATTERNS.some((p) => p.test(text));
}
/**
* Returns true if response body indicates the provider's API endpoint/base_url
* has permanently moved (see ENDPOINT_PERMANENTLY_MOVED_PATTERNS).
*/
export function isEndpointPermanentlyMoved(errorText: string): boolean {
const text = String(errorText || "");
return ENDPOINT_PERMANENTLY_MOVED_PATTERNS.some((p) => p.test(text));
}
/**
* Returns true if response body indicates the account is suspended for a
* billing reason (unpaid invoice, spending limit) — see
* ACCOUNT_SUSPENDED_BILLING_PATTERNS.
*/
export function isAccountSuspendedForBilling(errorText: string): boolean {
const text = String(errorText || "");
return ACCOUNT_SUSPENDED_BILLING_PATTERNS.some((p) => p.test(text));
}
/**
* T11: Returns true if response body indicates OAuth token is invalid/expired.
* This is different from permanent account deactivation - token refresh can recover.
@@ -1755,6 +1836,56 @@ export function checkFallbackError(
};
}
// A retired model (Gemini deprecated-model 404, Fireworks/etc. end-of-life 410)
// will fail identically on every future request — lock it for a long, fixed
// window instead of falling through to the generic transient-error branch's
// short backoff, which would otherwise keep re-selecting a permanently dead
// model roughly every cooldown window, all day, hammering the provider with
// guaranteed-to-fail requests (see MODEL_PERMANENTLY_UNAVAILABLE_PATTERNS).
// `quotaResetHintMs` flows into combo.ts's per-request model-lockout as an
// upstream-verified reset, so it is honored in full and not clamped to the
// normal ~20min model-lockout ceiling.
if (
(status === HTTP_STATUS.NOT_FOUND || status === HTTP_STATUS.GONE) &&
isModelPermanentlyUnavailable(errorStr)
) {
const cooldownMs = 24 * 60 * 60 * 1000; // 24h
return {
shouldFallback: true,
cooldownMs,
reason: "not_found",
quotaResetHintMs: cooldownMs,
};
}
// The provider's API endpoint/base_url has permanently moved — every future
// request against the stale base_url fails identically, so lock it for a
// long, fixed window instead of the generic transient-error branch's short
// backoff (see ENDPOINT_PERMANENTLY_MOVED_PATTERNS).
if (isEndpointPermanentlyMoved(errorStr)) {
const cooldownMs = 24 * 60 * 60 * 1000; // 24h
return {
shouldFallback: true,
cooldownMs,
reason: "not_found",
quotaResetHintMs: cooldownMs,
};
}
// The account is suspended for a billing reason (unpaid invoice, spending
// limit) that varies per provider/account name and can arrive on a status
// checkFallbackError does not otherwise classify (e.g. Fireworks 412) —
// treat it like a credits-exhausted account so it stops being retried
// every few minutes until billing is fixed (see ACCOUNT_SUSPENDED_BILLING_PATTERNS).
if (isAccountSuspendedForBilling(errorStr)) {
return {
shouldFallback: true,
cooldownMs: COOLDOWN_MS.paymentRequired ?? 3600 * 1000, // 1h cooldown
reason: RateLimitReason.QUOTA_EXHAUSTED,
creditsExhausted: true,
};
}
// Gemini-specific check — MUST run before isCreditsExhausted/
// isDailyQuotaExhausted/the generic text classifier below: Gemini's free-
// tier 429 boilerplate literally says "You exceeded your current quota,

View File

@@ -16,6 +16,7 @@ import {
hasPerModelQuota,
isAccountSemaphoreFull,
isModelLocked,
lockModelIfPerModelQuota,
MODEL_ACCESS_DENIED_PATTERNS,
recordModelLockoutFailure,
recordProviderFailure,
@@ -2236,6 +2237,26 @@ async function handleComboChatInner({
return { ok: false, response: result };
}
// A model-scoped 400 ("The requested model is not supported" / "not
// available for integrator") is permanent for THIS connection — the
// account/integration will not gain support for the model mid-session.
// Combo still advances to the next target immediately (unchanged,
// preserves #5249's cross-provider fallback), but without a lockout
// here the SAME dead model gets retried on every future, separate
// request forever (observed: every auto-combo request wasted several
// upstream 400s on the same GitHub models, all day). isModelLocked()
// is checked before dispatch (see the pre-check above this loop), so
// this lockout is honored on the next request.
if (result.status === 400 && isModelScoped400(errorText) && provider && rawModel) {
lockModelIfPerModelQuota(
provider,
targetWithConnection.connectionId || "",
rawModel,
"model_capacity",
60 * 60 * 1000 // 1h
);
}
// Trigger shared provider circuit breaker for 5xx errors and connection failures. If the
// next target is on the same provider, don't mark it failed (a different model may still
// succeed) — #8376: EXCEPT a proxy-unreachable failure, which poisons every model alike.
@@ -3302,24 +3323,20 @@ async function handleRoundRobinCombo({
"COMBO-RR",
`Maximum combo attempts (${maxGlobalAttempts}) exceeded. Terminating loop to prevent runaway requests.`
);
return errorResponseWithComboDiagnostics(
503,
"Maximum combo retry limit reached",
{
poolSize: modelCount,
attempted: globalAttempts,
excluded: [
...[...exhaustedProviders].map((p) => ({ provider: p, reason: "exhausted" })),
...[...exhaustedConnections].map((c) => formatExhaustedConnectionKey(String(c))),
],
attemptOrder: rrOutcomes.map((o) => ({
provider: o.model.split("/")[0] || "unknown",
model: o.model,
})),
terminalReason: "max_attempts_exceeded",
recovery: buildRecoveryHint("max_attempts_exceeded"),
}
);
return errorResponseWithComboDiagnostics(503, "Maximum combo retry limit reached", {
poolSize: modelCount,
attempted: globalAttempts,
excluded: [
...[...exhaustedProviders].map((p) => ({ provider: p, reason: "exhausted" })),
...[...exhaustedConnections].map((c) => formatExhaustedConnectionKey(String(c))),
],
attemptOrder: rrOutcomes.map((o) => ({
provider: o.model.split("/")[0] || "unknown",
model: o.model,
})),
terminalReason: "max_attempts_exceeded",
recovery: buildRecoveryHint("max_attempts_exceeded"),
});
}
if (retry > 0) {
log.info(

View File

@@ -387,6 +387,24 @@ export async function importManagedModels({
}
}
// #11824/#11651: `syncedIds` is a UNION across every connection of this provider
// (getSyncedAvailableModels), so an identity mapping derived above can route a
// display id to the literal tier-suffixed upstream id (e.g. "gemini-3.7-flash-high")
// just because ONE connected account's own discovery happens to list it directly.
// Google's Cloud Code Assist backend only allows those tier-suffixed ids on
// accounts/projects it specifically provisioned for them — every other account can
// only call the shared "-tiered" endpoint id. Since this mitmAlias table is global
// (not scoped per connection) and consulted first/authoritatively by
// cleanModelName(), letting one account's discovery win here silently 404s every
// sibling account. Force every display id that the static ANTIGRAVITY_MODEL_ALIASES
// table already knows only has a safe "-tiered" target to always resolve there,
// regardless of what any single connection's discovery reported.
for (const [displayId, safeTarget] of Object.entries(ANTIGRAVITY_MODEL_ALIASES)) {
if (safeTarget === "gemini-3.7-flash-tiered") {
mappings[displayId] = `antigravity/${safeTarget}`;
}
}
await setMitmAliasAll("antigravity", mappings);
}

View File

@@ -183,6 +183,7 @@
"tests/unit/combo-lockout-quota-reset-6863.test.ts",
"tests/unit/combo-max-depth-config.test.ts",
"tests/unit/combo-model-lockout-honors-reset-1308.test.ts",
"tests/unit/combo-model-scoped-400-advance.test.ts",
"tests/unit/combo-omnimodel-tag-stripping.test.ts",
"tests/unit/combo-param-validation-fallback-4519.test.ts",
"tests/unit/combo-prescreen.test.ts",
@@ -247,8 +248,10 @@
"tests/unit/format-provider-error-cause.test.ts",
"tests/unit/forwarded-header-budget.test.ts",
"tests/unit/fusion-vision-panel-3378.test.ts",
"tests/unit/gemini-deprecated-model-lockout.test.ts",
"tests/unit/gemini-web-capabilities-9356.test.ts",
"tests/unit/gemini-web-missing-browser-3516.test.ts",
"tests/unit/github-model-not-supported-lockout.test.ts",
"tests/unit/grok-cli-oauth.test.ts",
"tests/unit/guardrails-api-3496.test.ts",
"tests/unit/guardrails/visionBridge-responses-9597.test.ts",
@@ -297,6 +300,7 @@
"tests/unit/openrouter-free-model-credits-exhausted.test.ts",
"tests/unit/openrouter-passthrough-models.test.ts",
"tests/unit/openrouter-quota-6842.test.ts",
"tests/unit/permanent-failure-hammering-other-providers.test.ts",
"tests/unit/persist-429-cooldown-account-fallback.test.ts",
"tests/unit/plan3-p0.test.ts",
"tests/unit/plugin-sandbox-permissions.test.ts",

View File

@@ -26,6 +26,16 @@ process.env.DATA_DIR = TEST_DATA_DIR;
process.env.API_KEY_SECRET = process.env.API_KEY_SECRET || "combo-model-400-test-secret";
const { handleComboChat, isModelScoped400 } = await import("../../open-sse/services/combo.ts");
const { clearAllModelLockouts } = await import("../../open-sse/services/accountFallback.ts");
// Reused "github/claude-fable-5" across sub-tests below now persists a
// cross-request model lockout on a 400 "model not supported" (matches
// production combo.ts behavior). Reset it between tests so each sub-test
// still exercises a fresh dispatch instead of being skipped by a lockout
// left over from a previous sub-test in this file.
test.beforeEach(() => {
clearAllModelLockouts();
});
const noop = () => {};
const log = { info: noop, warn: noop, debug: noop, error: noop };
@@ -53,7 +63,10 @@ test("isModelScoped400 recognizes model-not-supported shapes (incl. invalid/Bad
assert.equal(isModelScoped400("Bad Request: The model is not supported"), true);
assert.equal(isModelScoped400("model claude-fable-5 does not support Responses API."), true);
assert.equal(isModelScoped400("unsupported_api_for_model"), true);
assert.equal(isModelScoped400("The model `x` does not exist or you do not have access to it."), true);
assert.equal(
isModelScoped400("The model `x` does not exist or you do not have access to it."),
true
);
// Genuinely body-specific — must NOT be treated as model-scoped
assert.equal(isModelScoped400("Invalid message format: the request body is malformed."), false);
assert.equal(isModelScoped400("malformed JSON in request body"), false);
@@ -68,7 +81,10 @@ async function assertAdvancesOn(errorMessage: string, label: string) {
handleSingleModel: async (_body: unknown, modelStr: string) => {
modelsCalled.push(modelStr);
if (modelStr === "github/claude-fable-5") {
return Response.json({ error: { message: errorMessage, type: "invalid_request_error" } }, { status: 400 });
return Response.json(
{ error: { message: errorMessage, type: "invalid_request_error" } },
{ status: 400 }
);
}
return okResponse(modelStr);
},
@@ -128,6 +144,10 @@ test("combo still STOPS on genuinely body-specific invalid message format", asyn
allCombos: [],
});
assert.equal(modelsCalled.length, 1, `body-specific 400 must stop at target 1; tried: ${modelsCalled.join(", ")}`);
assert.equal(
modelsCalled.length,
1,
`body-specific 400 must stop at target 1; tried: ${modelsCalled.join(", ")}`
);
assert.equal(response.status, 400, "body-specific 400 must surface to the client");
});

View File

@@ -0,0 +1,72 @@
/**
* Regression guard: a permanently retired model (Gemini's deprecated-model 404,
* "This model models/gemini-2.5-flash is no longer available to new users...",
* or a Fireworks/etc. end-of-life 410) must get a long, fixed lockout instead of
* falling through to the generic transient-error branch's short backoff.
*
* Without this classification, combo/auto-routing kept re-selecting the dead
* model roughly every cooldown window (a few minutes, escalating to ~20min max)
* for as long as the model stayed in the registry — all day, every day — sending
* guaranteed-to-fail requests to the provider. At volume this looks like abusive
* traffic and was implicated in a Gemini free-tier API key getting banned.
*/
import test from "node:test";
import assert from "node:assert/strict";
const { checkFallbackError, isModelPermanentlyUnavailable } =
await import("../../open-sse/services/accountFallback.ts");
const GEMINI_DEPRECATED_404 =
"[404]: This model models/gemini-2.5-flash is no longer available to new users. " +
"Please update your code to use models/gemini-3.6-flash for the latest features and improvements.";
const END_OF_LIFE_410 =
'[410]: {"type":"about:blank","title":"Gone","status":410,"detail":"The model ' +
"'minimaxai/minimax-m2.7' has reached its end of life on 2026-07-27T00:00:00Z and is no longer available.\"}\n";
test("isModelPermanentlyUnavailable matches Gemini's deprecated-model phrasing", () => {
assert.equal(isModelPermanentlyUnavailable(GEMINI_DEPRECATED_404), true);
});
test("isModelPermanentlyUnavailable matches end-of-life phrasing", () => {
assert.equal(isModelPermanentlyUnavailable(END_OF_LIFE_410), true);
});
test("isModelPermanentlyUnavailable does not match an ordinary 404", () => {
assert.equal(
isModelPermanentlyUnavailable("[404]: Model not found, inaccessible, and/or not deployed"),
false
);
});
test("checkFallbackError locks a deprecated Gemini model for 24h, not a short backoff", () => {
const result = checkFallbackError(404, GEMINI_DEPRECATED_404, 0, "gemini-2.5-flash", "gemini");
assert.equal(result.shouldFallback, true);
assert.equal(result.reason, "not_found");
assert.equal(result.cooldownMs, 24 * 60 * 60 * 1000);
// Feeds combo.ts's per-request model-lockout as an upstream-verified reset,
// so it bypasses the normal ~20min model-lockout ceiling.
assert.equal(result.quotaResetHintMs, 24 * 60 * 60 * 1000);
});
test("checkFallbackError locks an end-of-life model (410) for 24h too", () => {
const result = checkFallbackError(410, END_OF_LIFE_410, 0, "minimaxai/minimax-m2.7", "nvidia");
assert.equal(result.shouldFallback, true);
assert.equal(result.reason, "not_found");
assert.equal(result.cooldownMs, 24 * 60 * 60 * 1000);
});
test("a generic 404 (not a deprecation message) still falls through to the short transient cooldown", () => {
const result = checkFallbackError(
404,
"[404]: Model not found, inaccessible, and/or not deployed",
0,
"some-model",
"openrouter"
);
assert.equal(result.reason, "unknown");
assert.notEqual(result.cooldownMs, 24 * 60 * 60 * 1000);
});

View File

@@ -0,0 +1,50 @@
/**
* Regression guard: a GitHub Copilot 400 "The requested model is not
* supported" / "not available for integrator ..." — permanent for THIS
* account/integration — must get locked out via lockModelIfPerModelQuota so
* future, separate requests skip the same dead model instead of retrying it
* on every single auto-combo request forever (observed: every request in
* production logs wasted several upstream 400 calls on the same GitHub
* models — gpt-5.4, gpt-5.5, gpt-5.6-luna, etc. — all day).
*
* Combo's existing #5249/#2101 guard already lets the current request keep
* rotating to the next target — that behavior is unchanged and untested
* here. This guards the NEW cross-request lockout side effect only.
*/
import test from "node:test";
import assert from "node:assert/strict";
const { isModelScoped400 } = await import("../../open-sse/services/combo/comboPredicates.ts");
const { lockModelIfPerModelQuota, isModelLocked, hasPerModelQuota } =
await import("../../open-sse/services/accountFallback.ts");
const GITHUB_NOT_SUPPORTED_400 = "[400]: The requested model is not supported.";
const GITHUB_INTEGRATOR_400 =
'[400]: The requested model is not available for integrator "vscode-chat". ' +
"Available models: [gpt-4.1 claude-fable-5]. Verify the correct Copilot-Integration-Id header is being sent.";
test("isModelScoped400 matches GitHub's two 'model not supported' phrasings", () => {
assert.equal(isModelScoped400(GITHUB_NOT_SUPPORTED_400), true);
assert.equal(isModelScoped400(GITHUB_INTEGRATOR_400), true);
});
test("github has per-model quota (locks the model, not the whole connection)", () => {
assert.equal(hasPerModelQuota("github"), true);
});
test("lockModelIfPerModelQuota locks a model-not-supported GitHub model for future requests", () => {
const connectionId = `github-${Date.now()}`;
const locked = lockModelIfPerModelQuota(
"github",
connectionId,
"gpt-5.4",
"model_capacity",
60 * 60 * 1000
);
assert.equal(locked, true);
assert.equal(isModelLocked("github", connectionId, "gpt-5.4"), true);
// A sibling model on the same connection must stay eligible.
assert.equal(isModelLocked("github", connectionId, "gpt-5.5"), false);
});

View File

@@ -296,10 +296,70 @@ test("antigravity sync dynamically builds and saves mitmAlias mappings", async (
models.some((model) => model.id === "gemini-3.5-flash"),
false
);
assert.equal(mitmMappings["gemini-3.7-flash-high"], "antigravity/gemini-3.7-flash-high");
// #11824: gemini-3.7-flash-high is a known cross-account-unsafe display id (the
// tier-suffixed upstream id is only callable on Google projects specifically
// provisioned for it); it must always resolve through the static alias to the
// safe shared "-tiered" endpoint id, never as an identity mapping to the literal
// tier id — even when it appears directly in this connection's own discovery.
assert.equal(mitmMappings["gemini-3.7-flash-high"], "antigravity/gemini-3.7-flash-tiered");
assert.equal(mitmMappings["custom-antigravity-model"], "antigravity/custom-antigravity-model");
// Removed Antigravity 2.0 preview/agent aliases must not be reintroduced.
assert.equal(mitmMappings["gemini-3.5-flash-preview"], undefined);
assert.equal(mitmMappings["gemini-3-flash-agent"], undefined);
});
test("#11824: syncing account A's model catalog must not route account B's gemini-3.7-flash-high to an id B's own discovery never advertised", async () => {
const db = core.getDbInstance();
const now = "2026-08-27";
db.prepare(
"INSERT INTO provider_connections (id, provider, auth_type, name, is_active, created_at, updated_at) VALUES (?, ?, ?, ?, ?, ?, ?)"
).run("acct-a", "antigravity", "oauth", "Antigravity A", 1, now, now);
db.prepare(
"INSERT INTO provider_connections (id, provider, auth_type, name, is_active, created_at, updated_at) VALUES (?, ?, ?, ?, ?, ?, ?)"
).run("acct-b", "antigravity", "oauth", "Antigravity B", 1, now, now);
// Account A: Google has provisioned direct tier-suffixed callability for this
// project -- its own :fetchAvailableModels lists "gemini-3.7-flash-high" verbatim.
await importManagedModels({
providerId: "antigravity",
connectionId: "acct-a",
mode: "sync",
fetchedModels: [{ id: "gemini-3.7-flash-high", name: "Gemini 3.7 Flash High" }],
});
// Account B: Google has NOT provisioned the direct tier id for this project --
// its own :fetchAvailableModels only ever lists the tiered endpoint id.
await importManagedModels({
providerId: "antigravity",
connectionId: "acct-b",
mode: "sync",
fetchedModels: [{ id: "gemini-3.7-flash-tiered", name: "Gemini 3.7 Flash (Tiered)" }],
});
// Account B's OWN synced catalog never included the direct tier id.
const acctBOwnCatalog = await modelsDb.getSyncedAvailableModelsForConnection(
"antigravity",
"acct-b"
);
const acctBOwnIds = acctBOwnCatalog.map((m) => m.id);
assert.ok(
!acctBOwnIds.includes("gemini-3.7-flash-high"),
"precondition: account B's own discovery must not include the direct tier id"
);
// But the GLOBAL mitmAlias table used by cleanModelName() for EVERY connection
// (including B's requests) was rebuilt from the union of A + B, so it must not
// still route "gemini-3.7-flash-high" to the id only A supports.
const mitmMappings = await modelsDb.getMitmAlias("antigravity");
assert.notEqual(
mitmMappings["gemini-3.7-flash-high"],
"antigravity/gemini-3.7-flash-high",
"BUG: the shared mitmAlias table routes gemini-3.7-flash-high to the literal " +
"upstream id (only account A's project supports it) even for account B, " +
"whose own discovery never advertised that id -- this is what produces the " +
"per-account 404 reported in #11824/#11651."
);
assert.equal(mitmMappings["gemini-3.7-flash-high"], "antigravity/gemini-3.7-flash-tiered");
});

View File

@@ -0,0 +1,89 @@
/**
* Regression guard: a provider that permanently moved its API base_url
* (e.g. freeaiapikey's "This API endpoint has moved... the old endpoint on
* freeaiapikey.com no longer works.") or an account suspended for a billing
* reason that doesn't match ACCOUNT_DEACTIVATED_SIGNALS's fixed wording (e.g.
* Fireworks 412 "Account hummern is suspended, possibly due to reaching the
* monthly spending limit or failure to pay past invoices.") must get a long,
* fixed lockout instead of falling through to the generic transient-error
* branch's short backoff.
*
* Without this classification, combo/auto-routing kept re-selecting the dead
* endpoint/suspended account roughly every cooldown window (a few minutes) —
* observed retried every ~1 minute for a full hour in production logs —
* sending guaranteed-to-fail requests to the provider. Same bug class as the
* Gemini deprecated-model lockout gap, on different providers/status codes.
*/
import test from "node:test";
import assert from "node:assert/strict";
const { checkFallbackError, isEndpointPermanentlyMoved, isAccountSuspendedForBilling } =
await import("../../open-sse/services/accountFallback.ts");
const FREEAIAPIKEY_ENDPOINT_MOVED_410 =
"[410]: This API endpoint has moved. Please update your base_url to " +
"https://api.freeaiapikey.com/v1 — the old endpoint on freeaiapikey.com no longer works.";
const FIREWORKS_ACCOUNT_SUSPENDED_412 =
"[412]: Account hummern is suspended, possibly due to reaching the monthly spending limit " +
"or failure to pay past invoices. Please go to https://fireworks.ai/account/billing for more information.";
test("isEndpointPermanentlyMoved matches freeaiapikey's moved-endpoint phrasing", () => {
assert.equal(isEndpointPermanentlyMoved(FREEAIAPIKEY_ENDPOINT_MOVED_410), true);
});
test("isEndpointPermanentlyMoved does not match an ordinary 410", () => {
assert.equal(isEndpointPermanentlyMoved("[410]: Gone"), false);
});
test("isAccountSuspendedForBilling matches Fireworks's billing-suspension phrasing", () => {
assert.equal(isAccountSuspendedForBilling(FIREWORKS_ACCOUNT_SUSPENDED_412), true);
});
test("isAccountSuspendedForBilling does not match an unrelated suspension message", () => {
assert.equal(isAccountSuspendedForBilling("Your access has been suspended for review."), false);
});
test("checkFallbackError locks a permanently moved endpoint for 24h, not a short backoff", () => {
const result = checkFallbackError(
410,
FREEAIAPIKEY_ENDPOINT_MOVED_410,
0,
"anthropic/claude-sonnet-4.6",
"freeaiapikey"
);
assert.equal(result.shouldFallback, true);
assert.equal(result.reason, "not_found");
assert.equal(result.cooldownMs, 24 * 60 * 60 * 1000);
// Feeds combo.ts's per-request model-lockout as an upstream-verified reset,
// so it bypasses the normal ~20min model-lockout ceiling.
assert.equal(result.quotaResetHintMs, 24 * 60 * 60 * 1000);
});
test("checkFallbackError treats a billing-suspended account (412) as credits-exhausted", () => {
const result = checkFallbackError(
412,
FIREWORKS_ACCOUNT_SUSPENDED_412,
0,
"deepseek-v4-pro",
"fireworks"
);
assert.equal(result.shouldFallback, true);
assert.equal(result.creditsExhausted, true);
assert.ok(result.cooldownMs > 0, "cooldownMs should be positive");
});
test("a generic 412 (no billing-suspension phrasing) still falls through to the short transient cooldown", () => {
const result = checkFallbackError(
412,
"[412]: Precondition Failed",
0,
"some-model",
"some-provider"
);
assert.notEqual(result.creditsExhausted, true);
assert.notEqual(result.cooldownMs, 24 * 60 * 60 * 1000);
});