Compare commits

...

3 Commits

Author SHA1 Message Date
mikhailsal
370070f489 fix(stream): normalize delta.reasoning alias and separate reasoning in client response (#771)
* fix(stream): normalize delta.reasoning to reasoning_content in SSE streaming

NVIDIA kimi-k2.5 (and potentially other providers) send reasoning
tokens as `delta.reasoning` in SSE streaming chunks instead of the
standard OpenAI `delta.reasoning_content` field. This caused reasoning
content to be silently dropped during stream passthrough — clients
received only the final answer with no reasoning separation.

The non-streaming sanitizer (responseSanitizer.ts) already handled this
alias, but the streaming pipeline did not.

Fix applied in 4 locations:
- stream.ts passthrough: normalize + force re-serialize sanitized chunk
- stream.ts translate: accumulate reasoning from delta.reasoning
- sseParser.ts: collect delta.reasoning in parseSSEToOpenAIResponse
- streamPayloadCollector.ts: collect delta.reasoning in buildOpenAISummary

* fix: eliminate injectedUsage reuse bug and add reasoning alias tests

- Detect delta.reasoning alias before sanitizeStreamingChunk() which
  already normalizes it, removing dead post-sanitization normalization
- Replace injectedUsage reuse with separate needsReserialization flag
  so reasoning re-serialization cannot block finish_reason/usage
  mutations on the same SSE chunk (fixes CRITICAL review finding)
- Add unit test for parseSSEToOpenAIResponse reasoning alias
- Add unit test for buildStreamSummaryFromEvents reasoning alias

* fix(stream): separate reasoning from content in passthrough response body

The passthroughAccumulatedContent variable was mixing delta.content and
delta.reasoning_content into one string, causing the client_response
log and responseBody to lose reasoning separation.

- Add passthroughAccumulatedReasoning accumulator for reasoning deltas
- Set message.reasoning_content in responseBody when reasoning exists
- Only accumulate delta.content into passthroughAccumulatedContent

* fix: trim leading whitespace from assembled content in log summaries

NVIDIA and other providers emit token deltas with leading spaces
(e.g. ' The', ' user'). When joined, these produce a leading space in
the provider_response and parsed non-streaming response logs. Trim
the joined content and reasoning_content in both buildOpenAISummary
and parseSSEToOpenAIResponse for consistent log output.

* fix(stream): split combined reasoning+content deltas into separate SSE events

Some providers (e.g. NVIDIA NIM) send transition chunks with both
`delta.reasoning` and `delta.content` in the same SSE event.
After sanitization this becomes `reasoning_content` + `content`,
which violates the standard OpenAI streaming contract where these
fields are never mixed. Clients using if/else logic (LobeChat, etc.)
skip content when reasoning_content is present, losing the first
content token.

Split such combined chunks into two separate SSE events:
1. Reasoning-only event (finish_reason=null, no usage)
2. Content-only event (carries finish_reason and usage)
2026-03-29 16:12:22 -03:00
Paijo
7168f4014d fix: strip reasoning/thinking params for models that don't support them (#766)
Models like antigravity/claude-sonnet-4-6 route through Google's internal
Cloud Code API which returns HTTP 400 when thinking/reasoning parameters
are included in the request body.

Changes:
- open-sse/services/modelCapabilities.ts: add supportsReasoning() function
  with a denylist of known-unsupported patterns (antigravity/claude-sonnet-*)
  and a registry-based lookup hook (supportsReasoning flag per model)
- open-sse/services/thinkingBudget.ts: in applyThinkingBudget(), add early
  exit before the mode switch — if model string is present and
  supportsReasoning() returns false, call stripThinkingConfig() immediately
  regardless of the configured ThinkingMode

This is fully backward-compatible: models not in the denylist are unaffected,
and the supportsReasoning registry flag defaults to null (pass-through).

Fixes: HTTP 400 errors on antigravity provider when client sends requests
with thinking/reasoning budget parameters (e.g. claude-sonnet-4-6 via AG).

Co-authored-by: oyi77 <oyi77@github.com>
Co-authored-by: oyi77 <oyi77@users.noreply.github.com>
2026-03-29 16:12:19 -03:00
Paijo
f0912feefb feat: auto-disable permanently banned provider accounts (with Settings toggle) (#765)
* feat: auto-disable banned accounts setting with UI toggle

Add a configurable setting to automatically disable provider accounts
that return permanent/terminal errors (403 banned, ToS violation, etc.)

Changes:
- open-sse/services/accountFallback.ts: extend ACCOUNT_DEACTIVATED_SIGNALS
  with AG-specific ban messages ('verify your account', 'service disabled
  for violation')
- src/app/api/settings/auto-disable-accounts/route.ts: new GET/PUT endpoint
  for the setting (enabled bool + threshold int)
- src/shared/validation/schemas.ts: updateAutoDisableAccountsSchema
- src/sse/services/auth.ts: in markAccountUnavailable(), capture result.permanent
  from checkFallbackError() and — when autoDisableBannedAccounts is enabled and
  backoffLevel >= threshold — set isActive=false on the connection

Default: disabled (backward-compatible). Enable via Settings UI or PUT
/api/settings/auto-disable-accounts { "enabled": true, "threshold": 3 }

Fixes: antigravity accounts with 403/Verify-your-account errors being
retried indefinitely in the rotation pool.

Co-authored-by: oyi77 <oyi77@users.noreply.github.com>

* fix: address reviewer comments for auto-disable (use getCachedSettings, immediate disable on permanent bans)

---------

Co-authored-by: oyi77 <oyi77@github.com>
Co-authored-by: oyi77 <oyi77@users.noreply.github.com>
2026-03-29 16:12:17 -03:00
11 changed files with 299 additions and 10 deletions

View File

@@ -52,6 +52,10 @@ export function parseSSEToOpenAIResponse(rawSSE, fallbackModel) {
if (typeof delta.reasoning_content === "string" && delta.reasoning_content.length > 0) {
reasoningParts.push(delta.reasoning_content);
}
// Normalize `reasoning` alias (NVIDIA kimi-k2.5 etc.)
if (typeof delta.reasoning === "string" && delta.reasoning.length > 0 && !delta.reasoning_content) {
reasoningParts.push(delta.reasoning);
}
// T18: Accumulate tool calls correctly across streamed chunks
if (delta.tool_calls) {
@@ -94,12 +98,14 @@ export function parseSSEToOpenAIResponse(rawSSE, fallbackModel) {
}
}
const joinedContent = contentParts.length > 0 ? contentParts.join("").trim() : null;
const joinedReasoning = reasoningParts.length > 0 ? reasoningParts.join("").trim() : null;
const message: Record<string, unknown> = {
role: "assistant",
content: contentParts.length > 0 ? contentParts.join("") : null,
content: joinedContent || null,
};
if (reasoningParts.length > 0) {
message.reasoning_content = reasoningParts.join("");
if (joinedReasoning) {
message.reasoning_content = joinedReasoning;
}
const finalToolCalls = [...accumulatedToolCalls.values()].filter(Boolean).sort((a, b) => {

View File

@@ -17,6 +17,10 @@ export const ACCOUNT_DEACTIVATED_SIGNALS = [
"account has been disabled",
"your account has been suspended",
"this account is deactivated",
// AG (Antigravity/Google Cloud Code) permanent ban signals
"verify your account to continue",
"this service has been disabled in this account for violation",
"this service has been disabled in this account",
];
// T10 (sub2api PR #1169): Signals that indicate billing credits are exhausted.

View File

@@ -48,3 +48,54 @@ export function supportsToolCalling(modelStr: string): boolean {
return !blocked;
}
// Models that do NOT support reasoning/thinking parameters.
// AG (Antigravity) claude-sonnet-4-6 routes through a Google internal API
// that returns 400 if thinking params are included.
const REASONING_UNSUPPORTED_PATTERNS = [
"antigravity/claude-sonnet-4-6",
"antigravity/claude-sonnet-4-5",
"antigravity/claude-sonnet-4",
"ag/claude-sonnet-4-6",
"ag/claude-sonnet-4-5",
"ag/claude-sonnet-4",
];
function getRegistryReasoningFlag(providerIdOrAlias: string, modelId: string): boolean | null {
const providerAlias = PROVIDER_ID_TO_ALIAS[providerIdOrAlias] || providerIdOrAlias;
const models = PROVIDER_MODELS[providerAlias];
if (!Array.isArray(models)) return null;
const found = models.find((m) => m?.id === modelId);
if (!found) return null;
return typeof found.supportsReasoning === "boolean" ? found.supportsReasoning : null;
}
/**
* Returns whether a model supports reasoning/thinking parameters.
*
* Decision order:
* 1) Provider registry metadata (supportsReasoning flag) when available.
* 2) Explicit denylist for known unsupported models (e.g. AG Claude Sonnet).
* 3) Default true (pass through — safe, provider will ignore if unsupported).
*/
export function supportsReasoning(modelStr: string): boolean {
const parsed = parseModel(modelStr);
const provider = parsed.provider || parsed.providerAlias || "";
const model = parsed.model || modelStr;
if (provider) {
const fromRegistry = getRegistryReasoningFlag(provider, model);
if (fromRegistry !== null) return fromRegistry;
}
const normalized = String(modelStr || "").toLowerCase();
if (!normalized) return true;
const blocked = REASONING_UNSUPPORTED_PATTERNS.some((pattern) =>
normalized === pattern ||
normalized.endsWith(`/${pattern}`) ||
normalized.includes(pattern)
);
return !blocked;
}

View File

@@ -14,6 +14,7 @@ export const ThinkingMode = {
};
import { capThinkingBudget, getDefaultThinkingBudget } from "@/shared/constants/modelSpecs";
import { supportsReasoning } from "./modelCapabilities.ts";
// Effort → budget token mapping
export const EFFORT_BUDGETS = {
@@ -151,6 +152,13 @@ export function applyThinkingBudget(body, config = null) {
const cfg = config || _config;
if (!body || typeof body !== "object") return body;
// Early exit: strip ALL reasoning/thinking params for models that don't support them.
// Sending thinking params to unsupported models (e.g. AG claude-sonnet-4-6) causes 400 errors.
const modelStr = typeof body.model === "string" ? body.model : "";
if (modelStr && !supportsReasoning(modelStr)) {
return stripThinkingConfig(body);
}
// Pre-processing: convert string thinkingLevel to numeric budget
let processed = normalizeThinkingLevel(body);

View File

@@ -159,8 +159,9 @@ export function createSSEStream(options: StreamOptions = {}) {
// Track content length for usage estimation (both modes)
let totalContentLength = 0;
// Passthrough: accumulate content for call log response body
// Passthrough: accumulate content and reasoning separately for call log response body
let passthroughAccumulatedContent = "";
let passthroughAccumulatedReasoning = "";
// Guard against duplicate [DONE] events — ensures exactly one per stream
let doneSent = false;
@@ -304,6 +305,14 @@ export function createSSEStream(options: StreamOptions = {}) {
}
} else {
// Chat Completions: full sanitization pipeline
// Detect reasoning alias before sanitization strips it
const hadReasoningAlias = !!(
parsed.choices?.[0]?.delta?.reasoning &&
typeof parsed.choices[0].delta.reasoning === "string" &&
!parsed.choices[0].delta.reasoning_content
);
parsed = sanitizeStreamingChunk(parsed);
const idFixed = fixInvalidId(parsed);
@@ -323,6 +332,31 @@ export function createSSEStream(options: StreamOptions = {}) {
}
}
// Split combined reasoning+content deltas into separate SSE events.
// Standard OpenAI streaming never mixes both fields in one delta;
// clients (e.g. LobeChat) may skip content when reasoning_content
// is present, causing the first content token to be lost.
if (delta?.reasoning_content && delta?.content) {
const reasoningChunk = JSON.parse(JSON.stringify(parsed));
const rDelta = reasoningChunk.choices[0].delta;
delete rDelta.content;
reasoningChunk.choices[0].finish_reason = null;
delete reasoningChunk.usage;
const rOutput = `data: ${JSON.stringify(reasoningChunk)}\n`;
passthroughAccumulatedReasoning += delta.reasoning_content;
totalContentLength += delta.reasoning_content.length;
clientPayloadCollector.push(reasoningChunk);
reqLogger?.appendConvertedChunk?.(rOutput);
controller.enqueue(encoder.encode(rOutput));
controller.enqueue(encoder.encode("\n"));
delete delta.reasoning_content;
}
// Track whether we need to re-serialize (separate from injectedUsage
// to avoid blocking subsequent finish_reason / usage mutations)
const needsReserialization =
hadReasoningAlias || (delta?.content === "" && delta?.reasoning_content);
// T18: Track if we saw tool calls & accumulate for call log
if (delta?.tool_calls && delta.tool_calls.length > 0) {
passthroughHasToolCalls = true;
@@ -365,7 +399,7 @@ export function createSSEStream(options: StreamOptions = {}) {
if (typeof delta?.content === "string")
passthroughAccumulatedContent += delta.content;
if (typeof delta?.reasoning_content === "string")
passthroughAccumulatedContent += delta.reasoning_content;
passthroughAccumulatedReasoning += delta.reasoning_content;
const extracted = extractUsage(parsed);
if (extracted) {
@@ -398,7 +432,7 @@ export function createSSEStream(options: StreamOptions = {}) {
parsed.usage = filterUsageForFormat(buffered, FORMATS.OPENAI);
output = `data: ${JSON.stringify(parsed)}\n`;
injectedUsage = true;
} else if (idFixed) {
} else if (idFixed || needsReserialization) {
output = `data: ${JSON.stringify(parsed)}\n`;
injectedUsage = true;
}
@@ -483,6 +517,19 @@ export function createSSEStream(options: StreamOptions = {}) {
if (state?.accumulatedContent !== undefined) state.accumulatedContent += r;
}
}
// Normalize `reasoning` alias → `reasoning_content` (NVIDIA kimi-k2.5 etc.)
if (
parsed.choices?.[0]?.delta?.reasoning &&
!parsed.choices?.[0]?.delta?.reasoning_content
) {
const r = parsed.choices[0].delta.reasoning;
if (typeof r === "string") {
parsed.choices[0].delta.reasoning_content = r;
delete parsed.choices[0].delta.reasoning;
totalContentLength += r.length;
if (state?.accumulatedContent !== undefined) state.accumulatedContent += r;
}
}
// Gemini format - may have multiple parts
if (parsed.candidates?.[0]?.content?.parts) {
@@ -635,6 +682,10 @@ export function createSSEStream(options: StreamOptions = {}) {
role: "assistant",
content: content || null,
};
const reasoning = passthroughAccumulatedReasoning.trim();
if (reasoning) {
message.reasoning_content = reasoning;
}
if (passthroughToolCalls.size > 0) {
message.tool_calls = [...passthroughToolCalls.values()].sort(
(a, b) => a.index - b.index

View File

@@ -157,6 +157,10 @@ function buildOpenAISummary(events: StructuredSSEEvent[], fallbackModel?: string
if (typeof delta.reasoning_content === "string" && delta.reasoning_content.length > 0) {
reasoningParts.push(delta.reasoning_content);
}
// Normalize `reasoning` alias (NVIDIA kimi-k2.5 etc.)
if (typeof delta.reasoning === "string" && delta.reasoning.length > 0 && !delta.reasoning_content) {
reasoningParts.push(delta.reasoning);
}
if (Array.isArray(delta.tool_calls)) {
for (const item of delta.tool_calls) {
@@ -203,12 +207,14 @@ function buildOpenAISummary(events: StructuredSSEEvent[], fallbackModel?: string
}
}
const joinedContent = contentParts.length > 0 ? contentParts.join("").trim() : null;
const joinedReasoning = reasoningParts.length > 0 ? reasoningParts.join("").trim() : null;
const message: JsonRecord = {
role: "assistant",
content: contentParts.length > 0 ? contentParts.join("") : null,
content: joinedContent || null,
};
if (reasoningParts.length > 0) {
message.reasoning_content = reasoningParts.join("");
if (joinedReasoning) {
message.reasoning_content = joinedReasoning;
}
const finalToolCalls = [...toolCalls.values()].sort((a, b) => a.index - b.index);

View File

@@ -0,0 +1,57 @@
import { NextResponse } from "next/server";
import { getSettings, updateSettings } from "@/lib/localDb";
import { updateAutoDisableAccountsSchema } from "@/shared/validation/schemas";
import { isValidationFailure, validateBody } from "@/shared/validation/helpers";
export async function GET() {
try {
const settings = await getSettings();
return NextResponse.json({
enabled: settings.autoDisableBannedAccounts ?? false,
threshold: settings.autoDisableBannedThreshold ?? 3,
});
} catch (error) {
console.error("Error reading auto-disable accounts config:", error);
return NextResponse.json(
{ error: "Failed to read auto-disable accounts config" },
{ status: 500 }
);
}
}
export async function PUT(request: Request) {
let rawBody: unknown;
try {
rawBody = await request.json();
} catch {
return NextResponse.json(
{ error: { message: "Invalid request", details: [{ field: "body", message: "Invalid JSON body" }] } },
{ status: 400 }
);
}
try {
const validation = validateBody(updateAutoDisableAccountsSchema, rawBody);
if (isValidationFailure(validation)) {
return NextResponse.json({ error: validation.error }, { status: 400 });
}
const body = validation.data;
await updateSettings({
autoDisableBannedAccounts: body.enabled,
...(body.threshold !== undefined && { autoDisableBannedThreshold: body.threshold }),
});
const settings = await getSettings();
return NextResponse.json({
enabled: settings.autoDisableBannedAccounts ?? false,
threshold: settings.autoDisableBannedThreshold ?? 3,
});
} catch (error) {
console.error("Error updating auto-disable accounts config:", error);
return NextResponse.json(
{ error: "Failed to update auto-disable accounts config" },
{ status: 500 }
);
}
}

View File

@@ -1313,3 +1313,11 @@ export const v1SearchResponseSchema = z.object({
)
.optional(),
});
// ─── Auto-disable banned/error accounts ───────────────────────────────────
export const updateAutoDisableAccountsSchema = z
.object({
enabled: z.boolean(),
threshold: z.number().int().min(1).max(10).optional(),
})
.strict();

View File

@@ -3,6 +3,7 @@ import {
validateApiKey,
updateProviderConnection,
getSettings,
getCachedSettings,
} from "@/lib/localDb";
import { getQuotaWindowStatus, isAccountQuotaExhausted } from "@/domain/quotaCache";
import {
@@ -773,13 +774,14 @@ export async function markAccountUnavailable(
}
}
const { shouldFallback, cooldownMs, newBackoffLevel, reason } = checkFallbackError(
const result = checkFallbackError(
status,
errorText,
backoffLevel,
model,
provider // ← Now passes provider for profile-aware cooldowns
);
const { shouldFallback, cooldownMs, newBackoffLevel, reason } = result;
if (!shouldFallback) return { shouldFallback: false, cooldownMs: 0 };
// ── Local provider 404: model-only lockout, connection stays active ──
@@ -845,6 +847,28 @@ export async function markAccountUnavailable(
backoffLevel: newBackoffLevel ?? backoffLevel,
});
// T-AUTODISABLE: If auto-disable setting is enabled and error is permanent/terminal,
// mark account as inactive so it is never retried again.
// Uses getCachedSettings() to avoid DB overhead on hot error path.
// NOTE: For permanent bans we disable immediately — no threshold needed,
// because a permanent ban (403 "Verify your account" / ToS violation) will
// NEVER recover, so retrying is pointless regardless of attempt count.
if (result.permanent) {
try {
const settings = await getCachedSettings();
const autoDisableEnabled = settings.autoDisableBannedAccounts ?? false;
if (autoDisableEnabled) {
await updateProviderConnection(connectionId, { isActive: false });
log.info(
"AUTH",
`Auto-disabled ${connectionId.slice(0, 8)} — permanent ban detected (autoDisableBannedAccounts=true)`
);
}
} catch (e) {
log.info("AUTH", `Auto-disable check failed (non-fatal): ${e}`);
}
}
// Per-model lockout: lock the specific model if known
if (provider && model && cooldownMs > 0) {
lockModel(provider, connectionId, model, reason || "unknown", cooldownMs);

View File

@@ -503,3 +503,29 @@ test("parseSSEToOpenAIResponse merges split tool call chunks by id without dupli
assert.equal(parsed.choices[0].message.tool_calls[0].function.name, "sum");
assert.equal(parsed.choices[0].message.tool_calls[0].function.arguments, '{"a":1}');
});
test("parseSSEToOpenAIResponse normalizes delta.reasoning alias to reasoning_content", () => {
const rawSSE = [
`data: ${JSON.stringify({
id: "chatcmpl_2",
object: "chat.completion.chunk",
choices: [{ index: 0, delta: { reasoning: "Let me think..." } }],
})}`,
`data: ${JSON.stringify({
id: "chatcmpl_2",
object: "chat.completion.chunk",
choices: [{ index: 0, delta: { reasoning: " The answer is 4." } }],
})}`,
`data: ${JSON.stringify({
id: "chatcmpl_2",
object: "chat.completion.chunk",
choices: [{ index: 0, delta: { content: "2+2=4" }, finish_reason: "stop" }],
})}`,
"data: [DONE]",
].join("\n");
const parsed = parseSSEToOpenAIResponse(rawSSE, "moonshotai/kimi-k2.5");
assert.ok(parsed);
assert.equal(parsed.choices[0].message.reasoning_content, "Let me think... The answer is 4.");
assert.equal(parsed.choices[0].message.content, "2+2=4");
});

View File

@@ -155,3 +155,51 @@ test("builds compact Claude stream summary for detailed logs", () => {
assert.equal(compact.usage.output_tokens, 7);
assert.equal(compact._omniroute_stream.eventCount, 4);
});
test("builds compact OpenAI summary with reasoning alias (delta.reasoning)", () => {
const collector = createStructuredSSECollector({ stage: "provider_response" });
collector.push({
id: "chatcmpl_r1",
object: "chat.completion.chunk",
created: 100,
model: "moonshotai/kimi-k2.5",
choices: [{ index: 0, delta: { role: "assistant" } }],
});
collector.push({
id: "chatcmpl_r1",
object: "chat.completion.chunk",
created: 100,
model: "moonshotai/kimi-k2.5",
choices: [{ index: 0, delta: { reasoning: "Let me think..." } }],
});
collector.push({
id: "chatcmpl_r1",
object: "chat.completion.chunk",
created: 100,
model: "moonshotai/kimi-k2.5",
choices: [{ index: 0, delta: { content: "The answer is 4." } }],
});
collector.push({
id: "chatcmpl_r1",
object: "chat.completion.chunk",
created: 100,
model: "moonshotai/kimi-k2.5",
choices: [{ index: 0, delta: {}, finish_reason: "stop" }],
usage: { prompt_tokens: 10, completion_tokens: 20, total_tokens: 30 },
});
const summary = buildStreamSummaryFromEvents(
collector.getEvents(),
FORMATS.OPENAI,
"moonshotai/kimi-k2.5"
);
const compact = compactStructuredStreamPayload(
collector.build(summary, { includeEvents: false })
);
assert.equal(compact.object, "chat.completion");
assert.equal(compact.choices[0].message.content, "The answer is 4.");
assert.equal(compact.choices[0].message.reasoning_content, "Let me think...");
assert.equal(compact.choices[0].finish_reason, "stop");
});