Add a standardized degradation pattern for services depending on external
systems. withDegradation() tries primary → fallback → safe default,
tracking status in a global registry for dashboard visibility.
Features:
- Async and sync variants
- Global registry with per-feature status tracking
- Degradation levels: full → reduced → minimal → default
- Summary and report APIs for dashboard integration
- Reason tracking for debugging
Example: Rate limiting degrades from Redis → in-memory → permissive
instead of crashing when Redis is unavailable.
Closes#799
Add configAudit module that records every change to provider connections,
combos, and routing policies with:
- Before/after state snapshots
- Structured diff (added, removed, changed keys)
- Source tracking (dashboard, API, sync, auto-healing)
- Filtered retrieval with pagination
- Rollback state extraction
- Configuration snapshot export for backup
Enables traceability and quick rollback when config changes cause issues.
Closes#791
Add providerExpiration module to track OAuth token, subscription, and
API credit expiration dates per provider connection. Provides:
- setExpiration() / getExpiration() for CRUD operations
- getExpiringSoon() for proactive alerts
- getExpirationSummary() for dashboard health display
- detectExpirationFromResponse() for auto-detection from HTTP headers
- Status classification: active → expiring_soon → expired
Prevents silent failures from expired credentials by alerting operators
before tokens/subscriptions expire.
Closes#790
Add a providerDiversity module that tracks provider usage distribution
using a rolling time window and calculates Shannon entropy normalized
to [0..1]. This enables the auto-combo scoring engine to factor in
provider diversity — boosting underrepresented providers to reduce
single-point-of-failure risk.
Key features:
- Rolling window with configurable size and TTL
- Shannon entropy calculation normalized to [0..1]
- Per-provider diversity boost for auto-combo integration
- Diversity report for dashboard display
- Full test coverage
Closes#788
- Add windsurf and copilot entries to toolDescriptions in all 33 locale files
to fix MISSING_MESSAGE errors on the CLI Tools page (#748)
- Apply FETCH_TIMEOUT_MS to streaming requests' initial fetch() call to prevent
300s TCP default timeout causing silent failures on long-running requests (#769)
- Previously only non-streaming requests had timeout protection; streaming requests
relied solely on stream idle detection which doesn't cover initial connection hangs
Pre-existing any-budget violations in chatCore.ts (6), combo.ts (2), and
embeddings.ts (1 false positive in comment) — none introduced by GLM work.
Replace `as any` with `Record<string, unknown>` casts and reword comment.
Also removes docs/superpowers audit worksheet from git tracking (not part
of GLM Coding provider changes).
The model sync scheduler and sync-models endpoint were blindly
replacing custom models with all fetched models, including ones
already in the built-in registry. Now filters out registry models
before saving to custom models.
Compare fetched models against existing custom models AND built-in
registry models before posting. Only new models trigger
POST /api/provider-models calls. Shows skip count in import progress
when some models already exist. Adds i18n keys for all locales.
Live-tested all GLM Coding models against the /api/coding/paas/v4
endpoint. glm-4.7-flashx returns 429 "Insufficient balance or no
resource package" and is not listed on the /models endpoint.
All other models (glm-5.1, glm-5, glm-5-turbo, glm-4.7, glm-4.7-flash,
glm-4.6v, glm-4.6, glm-4.5v, glm-4.5, glm-4.5-air) return 200.
The "Import from /models" button was using the wrong Z.AI API surface
(Anthropic-compatible /api/anthropic/v1/models with x-api-key auth).
Switched to the correct Coding API endpoints with Authorization: Bearer
auth, matching the pattern used by the quota/usage tracking code.
- International: https://api.z.ai/api/coding/paas/v4/models
- China: https://open.bigmodel.cn/api/coding/paas/v4/models
- Auth: Authorization: Bearer <token> (not x-api-key)
- Region sourced from providerSpecificData.apiRegion
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Record the family-by-family GLM Coding audit, add regression coverage, and fix the documented GLM-5.1 context window override.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add missing glm-4.7-flashx variant to provider registry (confirmed in
Z.AI official GLM-4.7 overview docs as one of three variants)
- Remove glm-4.7/glm4.7 from tool calling denylist — official docs
explicitly show GLM-4.7 supporting function calling with tools param
- Add estimated pricing for glm-4.7-flashx ($0.3/$1.1) between free
Flash and standard 4.7 tiers
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(stream): normalize delta.reasoning to reasoning_content in SSE streaming
NVIDIA kimi-k2.5 (and potentially other providers) send reasoning
tokens as `delta.reasoning` in SSE streaming chunks instead of the
standard OpenAI `delta.reasoning_content` field. This caused reasoning
content to be silently dropped during stream passthrough — clients
received only the final answer with no reasoning separation.
The non-streaming sanitizer (responseSanitizer.ts) already handled this
alias, but the streaming pipeline did not.
Fix applied in 4 locations:
- stream.ts passthrough: normalize + force re-serialize sanitized chunk
- stream.ts translate: accumulate reasoning from delta.reasoning
- sseParser.ts: collect delta.reasoning in parseSSEToOpenAIResponse
- streamPayloadCollector.ts: collect delta.reasoning in buildOpenAISummary
* fix: eliminate injectedUsage reuse bug and add reasoning alias tests
- Detect delta.reasoning alias before sanitizeStreamingChunk() which
already normalizes it, removing dead post-sanitization normalization
- Replace injectedUsage reuse with separate needsReserialization flag
so reasoning re-serialization cannot block finish_reason/usage
mutations on the same SSE chunk (fixes CRITICAL review finding)
- Add unit test for parseSSEToOpenAIResponse reasoning alias
- Add unit test for buildStreamSummaryFromEvents reasoning alias
* fix(stream): separate reasoning from content in passthrough response body
The passthroughAccumulatedContent variable was mixing delta.content and
delta.reasoning_content into one string, causing the client_response
log and responseBody to lose reasoning separation.
- Add passthroughAccumulatedReasoning accumulator for reasoning deltas
- Set message.reasoning_content in responseBody when reasoning exists
- Only accumulate delta.content into passthroughAccumulatedContent
* fix: trim leading whitespace from assembled content in log summaries
NVIDIA and other providers emit token deltas with leading spaces
(e.g. ' The', ' user'). When joined, these produce a leading space in
the provider_response and parsed non-streaming response logs. Trim
the joined content and reasoning_content in both buildOpenAISummary
and parseSSEToOpenAIResponse for consistent log output.
* fix(stream): split combined reasoning+content deltas into separate SSE events
Some providers (e.g. NVIDIA NIM) send transition chunks with both
`delta.reasoning` and `delta.content` in the same SSE event.
After sanitization this becomes `reasoning_content` + `content`,
which violates the standard OpenAI streaming contract where these
fields are never mixed. Clients using if/else logic (LobeChat, etc.)
skip content when reasoning_content is present, losing the first
content token.
Split such combined chunks into two separate SSE events:
1. Reasoning-only event (finish_reason=null, no usage)
2. Content-only event (carries finish_reason and usage)
Models like antigravity/claude-sonnet-4-6 route through Google's internal
Cloud Code API which returns HTTP 400 when thinking/reasoning parameters
are included in the request body.
Changes:
- open-sse/services/modelCapabilities.ts: add supportsReasoning() function
with a denylist of known-unsupported patterns (antigravity/claude-sonnet-*)
and a registry-based lookup hook (supportsReasoning flag per model)
- open-sse/services/thinkingBudget.ts: in applyThinkingBudget(), add early
exit before the mode switch — if model string is present and
supportsReasoning() returns false, call stripThinkingConfig() immediately
regardless of the configured ThinkingMode
This is fully backward-compatible: models not in the denylist are unaffected,
and the supportsReasoning registry flag defaults to null (pass-through).
Fixes: HTTP 400 errors on antigravity provider when client sends requests
with thinking/reasoning budget parameters (e.g. claude-sonnet-4-6 via AG).
Co-authored-by: oyi77 <oyi77@github.com>
Co-authored-by: oyi77 <oyi77@users.noreply.github.com>
* feat: auto-disable banned accounts setting with UI toggle
Add a configurable setting to automatically disable provider accounts
that return permanent/terminal errors (403 banned, ToS violation, etc.)
Changes:
- open-sse/services/accountFallback.ts: extend ACCOUNT_DEACTIVATED_SIGNALS
with AG-specific ban messages ('verify your account', 'service disabled
for violation')
- src/app/api/settings/auto-disable-accounts/route.ts: new GET/PUT endpoint
for the setting (enabled bool + threshold int)
- src/shared/validation/schemas.ts: updateAutoDisableAccountsSchema
- src/sse/services/auth.ts: in markAccountUnavailable(), capture result.permanent
from checkFallbackError() and — when autoDisableBannedAccounts is enabled and
backoffLevel >= threshold — set isActive=false on the connection
Default: disabled (backward-compatible). Enable via Settings UI or PUT
/api/settings/auto-disable-accounts { "enabled": true, "threshold": 3 }
Fixes: antigravity accounts with 403/Verify-your-account errors being
retried indefinitely in the rotation pool.
Co-authored-by: oyi77 <oyi77@users.noreply.github.com>
* fix: address reviewer comments for auto-disable (use getCachedSettings, immediate disable on permanent bans)
---------
Co-authored-by: oyi77 <oyi77@github.com>
Co-authored-by: oyi77 <oyi77@users.noreply.github.com>
NVIDIA and other providers emit token deltas with leading spaces
(e.g. ' The', ' user'). When joined, these produce a leading space in
the provider_response and parsed non-streaming response logs. Trim
the joined content and reasoning_content in both buildOpenAISummary
and parseSSEToOpenAIResponse for consistent log output.
The passthroughAccumulatedContent variable was mixing delta.content and
delta.reasoning_content into one string, causing the client_response
log and responseBody to lose reasoning separation.
- Add passthroughAccumulatedReasoning accumulator for reasoning deltas
- Set message.reasoning_content in responseBody when reasoning exists
- Only accumulate delta.content into passthroughAccumulatedContent
- Detect delta.reasoning alias before sanitizeStreamingChunk() which
already normalizes it, removing dead post-sanitization normalization
- Replace injectedUsage reuse with separate needsReserialization flag
so reasoning re-serialization cannot block finish_reason/usage
mutations on the same SSE chunk (fixes CRITICAL review finding)
- Add unit test for parseSSEToOpenAIResponse reasoning alias
- Add unit test for buildStreamSummaryFromEvents reasoning alias
NVIDIA kimi-k2.5 (and potentially other providers) send reasoning
tokens as `delta.reasoning` in SSE streaming chunks instead of the
standard OpenAI `delta.reasoning_content` field. This caused reasoning
content to be silently dropped during stream passthrough — clients
received only the final answer with no reasoning separation.
The non-streaming sanitizer (responseSanitizer.ts) already handled this
alias, but the streaming pipeline did not.
Fix applied in 4 locations:
- stream.ts passthrough: normalize + force re-serialize sanitized chunk
- stream.ts translate: accumulate reasoning from delta.reasoning
- sseParser.ts: collect delta.reasoning in parseSSEToOpenAIResponse
- streamPayloadCollector.ts: collect delta.reasoning in buildOpenAISummary
Add a configurable setting to automatically disable provider accounts
that return permanent/terminal errors (403 banned, ToS violation, etc.)
Changes:
- open-sse/services/accountFallback.ts: extend ACCOUNT_DEACTIVATED_SIGNALS
with AG-specific ban messages ('verify your account', 'service disabled
for violation')
- src/app/api/settings/auto-disable-accounts/route.ts: new GET/PUT endpoint
for the setting (enabled bool + threshold int)
- src/shared/validation/schemas.ts: updateAutoDisableAccountsSchema
- src/sse/services/auth.ts: in markAccountUnavailable(), capture result.permanent
from checkFallbackError() and — when autoDisableBannedAccounts is enabled and
backoffLevel >= threshold — set isActive=false on the connection
Default: disabled (backward-compatible). Enable via Settings UI or PUT
/api/settings/auto-disable-accounts { "enabled": true, "threshold": 3 }
Fixes: antigravity accounts with 403/Verify-your-account errors being
retried indefinitely in the rotation pool.
Co-authored-by: oyi77 <oyi77@users.noreply.github.com>
Gemini API returns 400 error when tools have enum constraints on integer/number types:
"enum: only allowed for STRING type"
This fix removes enum constraints for integer and number types in JSON schemas
before sending to Gemini API, while keeping enum for string types.
Fixes tools like mcp__pointer__get-pointed-element that use integer enums
for cssLevel and textDetail parameters.
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
- Add `validateResponseQuality()` to detect empty/invalid 200 responses from
upstream providers in combo routing. Non-streaming responses with empty body,
invalid JSON, or missing content/tool_calls now trigger circuit breaker
failure and fallback to the next model instead of being returned to the client.
- Add missing `breaker._onSuccess()` calls in both priority and round-robin
combo paths. Previously failures accumulated without reset, causing premature
circuit breaker trips on healthy models.
- Update Cursor provider registry with Claude 4.6 model IDs (opus-high,
sonnet-high, haiku, opus + thinking variants). Keep 4.5 IDs for backward
compatibility.
- Update free-stack preset: replace duplicate qw/qwen3-coder-plus with
if/deepseek-v3.2 for better model diversity.
- Add paid-premium combo template for round-robin load distribution across
paid subscription providers (Cursor, Antigravity).
Made-with: Cursor
* chore(release): v3.2.8 — Docker auto-update UI and cache analytics fixes
* fix(sse): remove race condition in cache metrics tracking (#758)
- Remove in-memory metrics tracking (currentMetrics, trackCacheMetrics, updateCacheMetrics)
- Cache metrics now computed on-the-fly from usage_history table (single source of truth)
- Fixes CRITICAL issue from code review: concurrent requests overwriting metrics
- Fixes WARNING: duplicate metric tracking logic in streaming/non-streaming paths
Ref: PR #752 (merged before this fix was included)
* fix: handle allRateLimited credentials & forward extra body keys in embeddings/images routes (#757)
* fix: handle allRateLimited credentials in embeddings and images routes
When getProviderCredentials() returns an allRateLimited object (truthy,
but without apiKey/accessToken), the embeddings and images routes
incorrectly passed it to handlers as valid credentials. The handlers
then sent upstream requests without Authorization headers, causing
401 errors from providers (e.g. NVIDIA NIM).
This only manifested under concurrent requests: a chat/completions
call could trigger rate limiting on a provider account, and a
simultaneous embeddings request would receive the allRateLimited
sentinel — but treat it as valid credentials.
The chat pipeline already handled this case correctly. This commit
adds the same allRateLimited guard to all affected routes:
- POST /v1/embeddings
- POST /v1/providers/{provider}/embeddings
- POST /v1/images/generations
- POST /v1/providers/{provider}/images/generations
Also adds a defense-in-depth guard in the embeddings handler itself:
if no auth token is available for a non-local provider, return 401
immediately instead of sending an unauthenticated request upstream.
Made-with: Cursor
* fix(embeddings): forward extra body keys to upstream providers
The embeddings handler only forwarded model, input, dimensions, and
encoding_format to upstream providers, silently dropping any additional
fields. This broke asymmetric embedding APIs (e.g. NVIDIA NIM
nv-embedqa-e5-v5) that require input_type, and other providers
expecting user or truncate parameters.
Add a KNOWN_FIELDS exclusion set and forward all unrecognized body
keys to the upstream request, matching the passthrough pattern used
by the chat pipeline's DefaultExecutor.transformRequest().
Made-with: Cursor
* fix(auth): redirect and unconditional 401 on disabled requireLogin + fix test cases
* fix(build): remove legacy proxy.ts causing Next.js build collision
* fix(build): revert middleware.ts rename to proxy.ts because of Next.js Edge constraints
---------
Co-authored-by: diegosouzapw <diegosouzapw@users.noreply.github.com>
Co-authored-by: tombii <tombii@users.noreply.github.com>
Co-authored-by: Gorchakov-Pressure <117600961+Gorchakov-Pressure@users.noreply.github.com>
