Acquire admission once after API-key policy, preserve lazy raw-request snapshots, and bind lease settlement to JSON, SSE, abort, deadline, and failure lifecycles. Expose a low-cardinality health summary and preserve non-SSE Ollama errors unchanged.
Add bounded weighted admission with fair queuing, deadline and cancellation handling, exact lease accounting, and a default-shadow runtime. Keep asynchronous resource-pressure shedding as an independent safety fuse and bound request feature estimation.
* fix(reasoning): sanitize streamed K3 think tags
* refactor(stream): move think-tag helpers into thinkTagParser leaf module
open-sse/utils/stream.ts is frozen at 2887 lines with zero headroom, and the
kimi-coding-apikey think-tag handling added here pushed it to 2951. The parsing
half of that work has no SSE dependency, so it moves to thinkTagParser.ts (an
unfrozen leaf): the open-tag lookahead predicate and the end-of-stream flush
delta assembly. stream.ts keeps only the SSE envelope - enqueue, payload
collection, logging.
Getting back under the frozen ceiling needed more than just the PR's own
added lines, since even the fully self-gating helpers still cost a handful of
call-site lines stream.ts has zero room for. Along the way this also
deduplicates a synthetic chat-completion-chunk literal that was copy-pasted
three times in stream.ts (textual tool-call flush, think-tag flush, terminal
finish_reason synthesis) into one buildSyntheticChatChunk() in
streamHelpers.ts - pure DRY, no behavior change.
Behavior is unchanged: same tests, same counts.
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
---------
Co-authored-by: rinseaid <rinseaid@rinseaid.net>
Co-authored-by: diegosouzapw <diegosouzapw@users.noreply.github.com>
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
Fixes#8788.
- Added 'UND_ERR_SOCKET' to PROXY_UNREACHABLE_ERROR_CODES so socket disconnects and mid-stream closes are properly tagged with code PROXY_UNREACHABLE and errorCode proxy_unreachable.
- Ensured tagProxyUnreachable normalizes error code to PROXY_UNREACHABLE and errorCode to proxy_unreachable when catching socket-level disconnects.
- Added test in proxyfetch-undici-retry.test.ts verifying UND_ERR_SOCKET classification.
Wire NextAuth session-token merge/persist on successful perplexity-web
responses so rotated cookies survive in provider_connections, matching
chatgpt-web behavior.
synthOpenAIErrorChunk() sets error.type to upstream_empty_response but omits
error.code. The isRequestScopedUpstreamFailure guard only checks type for
context_length_exceeded, so an upstream_empty_response error slips past it.
- Added code: upstream_empty_response to the error object in synthOpenAIErrorChunk()
- Added test assertion verifying error.code matches error.type
This ensures the guard correctly identifies upstream empty responses as
request-scoped failures.
Closes#8469
An `auto/*` combo whose first step lands on an uncredentialed backend returns
HTTP 200 with `finish_reason: "stop"`, `content: null` and `error: null`. The
agent sees a clean empty assistant turn, has no error to stop on, and retries to
its cap.
The non-streaming path already refuses this: `isEmptyContentResponse` rewrites a
200-with-no-content into a 502 "Provider returned empty content", which the combo
layer classifies as a model-level transient and fails over on (#5085). The
streaming path had no equivalent, and neither existing guard covers it:
- `ensureStreamReadiness` is a LIVENESS probe, not a content one. Its failure
message says so — "Stream ended before producing a non-ping SSE event" — and
`hasStreamReadinessSignal` returns true for a bare `delta:{"role":"assistant"}`.
- `createDisconnectAwareStream`'s #7699 branch fires on a MISSING terminal marker
and is scoped to the Claude client format, because for other formats a
marker-less close is genuinely ambiguous.
The reported stream trips neither: OpenAI format, terminates with
`finish_reason: "stop"` and `[DONE]`, contains nothing.
"Completed normally but emitted zero content" is not ambiguous the way a missing
marker is, so this guard is format-agnostic. It reuses `hasUsefulStreamContent`,
which already existed in streamReadiness.ts — exported, correct, and wired to
nothing — and which already counts tool-call-only and reasoning-only output as
real (#2520). A watcher wraps it to handle frames split across network chunks and
to spot the terminal states where emptiness is legitimate, kept in step with
errorClassifier.ts's `LEGIT_EMPTY_OPENAI_FINISH` / `LEGIT_EMPTY_CLAUDE_STOP`:
length, tool_calls, content_filter, max_tokens, tool_use.
Two guards keep it from over-firing, one of which caught a real regression while
building this: the check applies only when bytes were forwarded, and only when
the body actually looked like SSE. A plain JSON completion travels through the
same wrapper and has no `data:` frames, so "no content seen" says nothing about
it — without the SSE gate, four existing stream tests failed.
The `if (done)` branch's reasoning moved into `resolveSilentCloseReason()`, which
also drops `pull` back under the function-length ceiling; cyclomatic lands at
2187 against a baseline of 2188.
This surfaces the error rather than failing over. Failing over would mean holding
every stream until its first content token, since the combo has already returned
leg 1's response by then — a much larger change. Surfacing the error satisfies
the issue's stated expectation ("surface the upstream error OR fail over") and
stops the retry loop, which is the reported harm.
Closes#8649
* fix(sse): stop thrashing the provider prompt cache for caching-aware clients
- shouldPreserveCacheControl: preserve client cache_control markers for every
combo strategy. The deterministic-strategy gate forced marker rewrites whose
per-request breakpoint positions are not stable turn-over-turn, thrashing the
upstream prompt cache (observed in production as ~200k cache_write tokens per
turn on quota-share combos). Preserving is never worse: on a stable target
the client's breakpoints advance deterministically; on a target switch both
approaches miss equally.
- prepareClaudeRequest: translator-path opt-in fallback — when preserve-mode
has nothing to preserve (client sent no cache_control anywhere), apply the
standard heuristic so requests never ship with zero cache breakpoints. The
claude-code-compatible relay path keeps its no-supplement contract.
- anthropic-beta: forward the client-negotiated context-1m-2025-08-07 through
the allowlist merge so a /model <id>[1m] client keeps its long-context
negotiation behind the proxy (never forced when the client did not send it).
* fix(sse): surface cache tokens in non-streaming OpenAI usage + finalize pending on quota-share block
- translateNonStreamingResponse (claude→openai): fold cache_read into
prompt_tokens and expose prompt_tokens_details.cached_tokens /
cache_creation_tokens, mirroring the streaming contract (#1426/#2215).
Non-streaming OpenAI clients behind a cached Claude upstream previously saw
prompt_tokens=<uncached remainder> (e.g. 23 for a ~9k request) with no cache
visibility.
- chatCore quota-share block: finalize the pending-request slot before
returning the policy 429 — the path never reaches upstream and the orphaned
pending lingered as a status-0 call-log row until the reaper swept it.
Validated live on the staging box (repro: blocked key → orphan row; after:
clean).
---------
Co-authored-by: diegosouzapw <diegosouzapw24@gmail.com>
`kieExecutor.createTask()` returns `JsonObject` (`Record<string, unknown>`), so
`createData.data` is `unknown` and the `createData?.data?.taskId` read that
image, video and music generation each duplicated could not compile. The same
three-line expression appeared verbatim in all three handlers.
`open-sse/utils/kieTask.ts` already holds two helpers with exactly this shape —
`normalizeKieTaskState()` and `parseKieResultJson()` both take `unknown`, guard
with `isJsonObject()` and return a declared type. `getKieTaskId()` follows them,
so the three handlers now share one guarded read instead of three unguarded ones.
`getKieCallbackUrl()` took `KieCallbackBody`, a weak type (all properties
optional). Passing a request body whose declared keys are `prompt` /
`timeout_ms` / `poll_interval_ms` tripped TS2559 "no properties in common" at
both music call sites. It receives arbitrary upstream request bodies, so it now
takes `unknown` and guards the same way its neighbours do; `KieCallbackBody`
had no other reference and is gone.
Behaviour is unchanged. `isJsonObject()` rejects arrays and null exactly where
optional chaining already yielded `undefined`, and the callers' `String(taskId)`
coercion moved inside the helper, so a numeric id still reaches `pollTask()` as
a string and a falsy id still takes the 502 branch.
Fixes 5 of the 208 `tsc -p open-sse/tsconfig.json` diagnostics with no new ones:
3 x TS2339 `taskId` on `unknown`, 2 x TS2559 on `KieCallbackBody`.
Refs #8484
Squash merge from local merge-train (Hard Rule owner-approved). Tip 029cdf4215cf465f0e1716ac9f84a84692b1e881 validated on 192.168.0.113: 26631/26653 pass.
* feat(db): cc discovery alias gate storage + EXPOSE_CC_DISCOVERY_ALIASES flag
Adds the gate for claude/<provider>/<model> discovery-alias mirror ids on
the /v1/models catalog: a new runtime feature flag (env forces on and wins
over the dashboard DB override), per-provider and per-model "on"/"off"/null
overrides stored in key_value under the ccDiscoveryAliases namespace, and a
pure precedence resolver (model > provider > global). Catalog wiring is a
separate follow-up task; this only lands the gate + storage.
* feat(sse): synthesize claude/ discovery aliases for the model catalog
* feat(api): advertise cc discovery aliases on /v1/models behind the 3-level gate
* feat(sse): resolve claude/ discovery aliases on the request path
* fix(sse): import getComboByName from db/combos, not the localDb barrel
* fix(sse): cover custom-node prefixes and the Codex WS bridge in cc discovery alias resolution
* feat(dashboard): cc discovery alias toggles + flag-screen env warning
Adds the operator-facing UI/API layer for the Claude Code discovery-alias
gate (claude/<provider>/<model> mirror ids on /v1/models): REST endpoint
for provider/model overrides, a provider-detail card with 3-state
(inherit/on/off) toggles, an info button on the Claude Code tool card
linking to Feature Flags, and an env-source warning on the
EXPOSE_CC_DISCOVERY_ALIASES flag card when it's forced on via env.
* feat(api): cc discovery usage metrics
* fix(api): record cc alias metric in the production wrapper + atomic counter upsert
* docs: document cc discovery aliases (Claude Code guide + feature flag catalog)
* fix(sse): don't mirror built-in auto/* combos as discovery aliases (advertised-but-unroutable)
* i18n(vi): translate the discovery-alias strings instead of shipping placeholders
vi is the one locale with a strict "no internal missing markers" test, so the 17
__MISSING__ entries this branch added (the provider ccAlias panel, the info
button, the feature-flag description and the env warning) would have turned that
test red the moment the base itself was repaired. Translated, keeping every ICU
placeholder ({modelId}, {error}) and the literal claude/<provider>/<model> id
shape intact.
* chore(quality): raise the frozen caps this feature legitimately grows
catalog.ts 1615 -> 1639: the alias synthesis is wired into the catalog builder,
which is where the per-key-filtered list is assembled — the only place the mirror
entries can be appended after model hiding has been applied.
localDb.ts 808 -> 810: two re-export lines for the new ccDiscoveryAliases db
module, which is exactly what the "Adding a New DB Module" recipe prescribes.
* refactor(dashboard,api): keep the complexity ratchets flat
The feature added four cyclomatic violations and one cognitive one, which the
ratchets reject — the baseline only moves when a metric improves. Split the new
code instead:
- appendCcDiscoveryAliases: the four skip-guards become isMirrorableId().
- resolveCcDiscoveryAliasStripWith: alias parsing and gate resolution become
parseCcAliasTarget() and resolveGateFor(), replacing a chain of ternaries that
each re-tested isComboAlias.
- FeatureFlagCard: the env-precedence warning becomes its own component instead
of a conditional branch inside an already-large render.
- ProviderCcAliasSection: the loader moves to useCcAliasData(), and the override
list and add-row become ModelOverrideList / AddOverrideRow, bringing both
oversized functions back under the 80-line rule.
Behavior unchanged — the 74 discovery-alias tests pass untouched. Both ratchets
now sit exactly at baseline (2188 / 971).
* feat(sse): selective upstream 4xx error passthrough util
* feat(sse): relay upstream 4xx error bodies verbatim on the Anthropic request path
createErrorResult() gains an opt-in 7th param opts.passthrough; when set and
the upstream body is eligible (per shouldPassthroughUpstreamError), the
returned response body is the upstream 4xx body verbatim instead of the
sanitized wrapper. Internal classification fields (error/rawMessage/
errorType/errorCode) are never affected.
Wired into chatCore.ts's 7 upstream-error createErrorResult call sites
(model_unavailable / context_overflow / generic upstream-error branches),
gated on sourceFormat === FORMATS.CLAUDE so only /v1/messages requests get
the verbatim body — OpenAI-format paths are unchanged.
* test(quality): register upstream-error-passthrough in stryker tap.testFiles
`tests/unit/upstream-error-passthrough.test.ts` covers `open-sse/utils/error.ts`,
an instrumented module, so `check:mutation-test-coverage --strict` requires it in
`tap.testFiles` — the gate flagged the drift on this PR.
* fix(sse): label HTTP 499 disconnects as client_disconnected
Preserve caller-supplied error type/code in buildErrorBody so stream
abort classification is not overwritten by the status-code table.
* docs(security): document buildErrorBody classification arg
---------
Co-authored-by: Diego Rodrigues de Sa e Souza <diegosouza.pw@gmail.com>
buildClientRawRequest deep-cloned the ENTIRE request body on every chat request, unbounded.
On the #7847 incident payload (3.05 MiB, 729 messages, 86 tools) that retains 3.19 MiB per
request, and it is pure waste: every consumer of clientRawRequest.body is observability and
none of them keeps the full payload.
chatCore.ts -> reqLogger.logClientRawRequest no-op when the logger is disabled, otherwise
re-clones via cloneBoundedForLog (0.08 MiB)
chatCore.ts -> trackPendingRequest clientRequest, surfaced by /api/logs/[id]
chat.ts -> recordRejectedRequestUsage requestBody
None feeds dispatch, translation or the upstream request, so the snapshot is now taken with
cloneBoundedForLog: 3.19 MiB -> 0.08 MiB, a 41x reduction, and retention no longer scales with
history length. It stays a clone rather than an alias because body is rewritten downstream
(plugin onRequest hook, compression) and the log must show what the client actually sent.
Bounding at the entry means the logger re-bounds an already-bounded value, which exposed that
cloneBoundedForLog was NOT idempotent -- each container exceeded its own bound once the marker
was added, so a second pass truncated again:
arrays [marker, ...24 items] is 25 entries > 24, so the marker and one real item were
dropped and originalLength was rewritten as 25 instead of the true 729
objects 80 keys + _omniroute_truncated_keys is 81 > 80, so a real key was evicted to make
room for the marker and the dropped count was reported as 1 instead of 20
strings the marker was appended AFTER slicing to maxLength, so the bounded string was
longer than the bound
Without this the persisted log payload would have changed shape versus before the fix. All
three now keep the marker inside the budget and treat an already-bounded value as final;
verified end to end -- the marker still reports originalLength 729.
TDD: tests/unit/repro-7847-bound-client-raw-request.test.ts was written first and failed on
three assertions (unbounded retention, retention scaling with history, and the idempotence
precondition) before either change.
Two changes with one root cause: several hot paths built a full JSON string only to read
its .length, and one of them silently changed the answer.
1. CORRECTNESS -- combo's fallback-compression trigger
estimateTokens(JSON.stringify(attemptBody)) took the STRING branch of estimateTokens,
which is ceil(length / CHARS_PER_TOKEN) over the raw JSON. An inline base64 image is
then charged as if every character of the data URL were prose. Measured on a 200 KB
inline image:
via string (before) 50,039 tokens
via object (after) 1,231 tokens
a 40x over-count, tripping fallback compression on requests nowhere near the context
window. This is the same class #8368/#8401 fixed on the request path; the combo call
site was missed. Passing the object routes through extractImageTokens, which charges
images structurally. Text-only bodies are unaffected -- verified identical, and pinned
by a test.
2. ALLOCATION -- jsonLength()
Adds an exact serialized-length walker: same O(n) scan, no string. Used by
estimateTokens' object branch and by streamReadinessPolicy (which runs on every
streaming request and only ever used .length).
Exactness matters because every consumer feeds a threshold, so this is property-tested
against JSON.stringify over 4000 generated structures covering escaping, lone
surrogates, omitted values, non-finite numbers, toJSON, Date, Map, cycles and BigInt.
Anything outside the plain-JSON subset falls back to JSON.stringify for THAT SUBTREE
only, so an exotic leaf never forces the message history back onto the allocating path.
Honest scoping of the memory win: the string was always transient, and V8 collects it
efficiently, so this is not 3 MiB of retained heap. Measured allocation churn over 20 calls
on a 3.06 MiB body: 3.1 MiB -> 0.5 MiB, about 6x less. The #8549 benchmark row for this
mechanism measures a HELD string and therefore overstates it; the correctness fix above is
the larger deliverable here.
Eight TS2339s in stream.ts read properties off `never`. `never` here did not
mean unreachable code — it meant a type predicate was lying.
function isClaudeEventPayload(payload: unknown): payload is JsonRecord
...
const flushedParsed = bufferedPayload as JsonRecord;
const isClaude = isClaudeEventPayload(flushedParsed);
} else if (!isClaude) { // JsonRecord minus JsonRecord = never
The predicate answers "does this payload carry a Claude event type?" — a
question about contents, not type. A Claude event and a non-Claude one are both
JsonRecord, so `payload is JsonRecord` narrows nothing; applied to an argument
already typed JsonRecord it collapsed the negative branch to `never`, taking
`.id` and `.choices` with it.
Changed the return type to `boolean`. No call site depended on the narrowing:
both other callers pass the value to updateClaudeEmptyResponseLifecycle(), whose
parameter is `unknown`, and passthroughTailProcessor.ts already declares this
function as `(payload: unknown) => boolean` — so this aligns the definition with
how the codebase already consumes it.
280 -> 272, zero new, on a line-number-agnostic diff of the full tsc error set.
The diff is one line, and a type predicate has no runtime representation, so the
emitted JavaScript is unchanged.
No test added, and no gap to fill: the `never`-typed branch is real, reachable
and already covered from both sides — stream-numeric-ids.test.ts exercises
"normalizes numeric id in final chunk without trailing newline" (the flush path
with a non-Claude payload, i.e. the branch TS believed impossible) and "Claude
passthrough does not normalize numeric ids" (the other arm). 338/338 across the
48 SSE/passthrough suites.
No explanatory comment either: stream.ts sits exactly at its frozen file-size
cap (2887), so a single added line fails check:file-size. The reasoning lives
here and in the PR rather than costing a baseline bump on a frozen file.
Co-authored-by: backryun <busan011@ormbiz.co.kr>
First slice of the TypeScript 7 migration split requested on #7697: resolve the
type diagnostics under `open-sse/tsconfig.json` in the lowest-risk modules, with
no toolchain change. 12 diagnostics across 8 files, all outside the hot path —
`chatCore.ts` and `stream.ts` are deliberately left for a later, standalone slice.
Fixes, by cause:
* `Transformer.cancel` (progressTracker, sseHeartbeat, and stream.ts's existing
handler) — the WHATWG Streams standard defines `transformer.cancel(reason)` and
Node implements it (verified on v24: cancelling the readable side invokes it),
but `lib.dom.d.ts` still omits it from `Transformer`, so every such handler was
TS2353. These handlers clear the heartbeat/progress intervals when an SSE client
disconnects, so deleting them to satisfy the checker would leak a timer per
abandoned stream. The interface is patched in `open-sse/types.d.ts` instead.
* `earlyStreamKeepalive` — `SettledHandler` was discriminated by `ok: true | false`.
This workspace compiles with `strictNullChecks: false`, where a boolean-literal
discriminant narrows the positive branch but not the negative one, so reading
`.error` off the rejected arm did not type-check (the two `.response` reads
elsewhere in the file did, which is why only one site errored). Retagged with a
string discriminant, which narrows both branches under the same settings.
* `toolCallShim` / `openai-responses` — assigning back to a property declared
`unknown` resets the `typeof` narrowing, so the following comparison no longer
saw a number/array. Both now read through a local. The `Read` limit clamp is
behavior-identical: its two branches are mutually exclusive at READ_MAX_LIMIT 2000.
* `sanitizeToolResultId` — takes `unknown` but forwards to a `string` parameter; a
non-string id previously reached `.replace()` and threw. Coerced instead.
* `openaiHelper` — `opts = {}` inferred `{}`; typed as `FilterToOpenAIFormatOptions`.
* `cursorAgentProtobuf` — `Buffer.alloc(0)` infers `Buffer<ArrayBuffer>` under
@types/node 26 while the decoded field is `Buffer<ArrayBufferLike>`; the locals
now use bare `Buffer`, matching `requestMetadata` a few lines above.
Validation: 335 -> 321 diagnostics with zero new errors (full tsc error-set diff
against the base config). typecheck:core clean, lint clean, check:type-coverage
92.17% -> 94.17%. All 114 existing test files that import a touched module pass;
`plan3-p0.test.ts` fails identically with and without this change (it reads the
developer's real ~/.omniroute DB instead of a test-scoped DATA_DIR).
The new test covers the three behavioral surfaces rather than the refactors the
existing keepalive/heartbeat suites already hold: that `transformer.cancel()`
really fires and can clear an interval, the id coercion, and the limit-clamp bounds.
* fix(api): stop the 2000-token safety buffer from inflating usage.prompt_tokens in the client response (#8331)
* fix(sse): scope #8331's usage-buffer fix around Claude-Code-compatible providers
The #8331 fix correctly stopped folding the 2000-token context-window safety
margin into client-visible prompt_tokens/input_tokens/total_tokens for normal
API metering clients. But it also silently changed the response shape for
Claude-Code-compatible providers, whose own context accounting reads the
buffered number straight out of usage — regressing
tests/unit/cc-compatible-provider.test.ts (expected 2007, got 7).
Fold the computed context_budget_* fields back into the visible usage fields
for that one path only (applyClientUsageBuffer's new
preserveContextBudgetInVisibleUsage option, gated on the existing
isClaudeCodeCompatible flag in chatCore.ts). Every other caller keeps the
real, unbuffered #8331 numbers.
Live incident: streamed assistant text was losing the spaces BETWEEN words
(e.g. "Bilden är en riktig JPEG nu" -> "Bildenärenriktig JPEG nu") on the
Responses-API and Claude streaming paths.
stripInternalReasoningPlaceholder() (#8081/#8162) is called on every
individual delta.content chunk, and unconditionally called .trim() even when
its sentinel ("(prior reasoning summary unavailable)") was never present in
that chunk. Tokenizers commonly emit sub-word tokens with a leading space as
part of the token (e.g. " en", " riktig") -- each such chunk got its only
whitespace character (the inter-word space) silently trimmed away before
being appended to the accumulated message, while the words themselves stayed
intact. Punctuation-only chunks were largely unaffected, matching what was
observed live.
Only trims when the sentinel is actually present -- preserves the original
#8081 intent (collapse a placeholder-only chunk to "") without touching the
overwhelming majority of chunks that never contain it.
Co-authored-by: Markus Hartung <markus.hartream@gmail.com>
Regression: #8162 (port of #8081) added an unconditional `.trim()` to
stripInternalReasoningPlaceholder(), applied to every streaming
delta.content chunk across 3 call-sites (openai-to-claude.ts,
openai-responses.ts, responsesTransformer.ts). Leading/trailing
whitespace at a chunk boundary is a real word boundary between
streaming fragments; trimming it glues adjacent chunks together on
the client ("Hello, " + "world." + " Bye." -> "Hello,world.Bye.").
Fix: early-return via .includes() before the replaceAll+trim, so the
function is a true no-op when the sentinel is absent from the chunk.
Behavior when the sentinel IS present is unchanged.
Validation:
- tests/unit/streaming-reasoning-dedup-5786.test.ts: the "(A-guard)"
test was RED on the base branch ('Hello,world.Bye.' vs
'Hello, world. Bye.'); GREEN after the fix (4/4 passing).
- tests/unit/translator-resp-openai-to-claude.test.ts: added a new
multi-chunk boundary-whitespace regression test, proven RED against
the pre-fix code (12/13), GREEN after (13/13).
- No regressions in responses-transformer.test.ts (17/17),
responses-transformer-dense-output.test.ts (3/3), or the other
suites exercising the shared placeholder utility (160/160 total
across all consumers).
Refs #8162
Refs #8081
Claude→OpenAI translation was emitting a literal </think> into
delta.content for ordinary Chat Completions clients. Reasoning already
ships as reasoning_content, so default to suppress and keep
x-omniroute-thinking-marker: on as the #4633 opt-in.
* fix(ci): resolve upstream-inherited check failures
* fix(streamHandler): log trackPendingRequest decrement failures instead of swallowing
The clearPendingRequest function had an empty catch block around the
trackPendingRequest decrement call. If it threw, the pending request
counter stayed incremented — causing drift, false-positive rate limiting,
and masked overload conditions.
Now logs the error with context for observability.
Fixes#8141
---------
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
* fix(sse): Gemini TPM classification, combo-cooldown-wait for auto/quota-share, and target-timeout floor
Gemini TPM/RPM 429s were misclassified as QUOTA_EXHAUSTED because
sanitizeErrorMessage() truncates to the first line, hiding Google's
metric name and retry hint on lines 2-3. Added a rawMessage field
(internal-only, never reaches the client) and classifyGeminiQuotaMetricFromText()
to classify from the untruncated text, reordered ahead of the generic
credits/daily-quota checks.
Widened comboCooldownWaitEnabled (wait out a short transient cooldown
instead of crystallizing a 429/503) from quota-share-only to also cover
auto-strategy combos, and raised the wait ceiling to 65s/130s-budget/90s-cap
to match Gemini's ~60s TPM/RPM windows.
The per-target timeout (DEFAULT_COMBO_TARGET_TIMEOUT_MS, 120s) was shorter
than the new 130s cooldown-wait budget, so a target could get cut off
mid-wait with a synthetic 524 instead of completing the retry. Added
resolveComboTargetTimeoutMsForCombo()/isComboCooldownWaitEligible() in
comboConfig.ts to raise the per-target floor to budgetMs+buffer only for
wait-eligible strategies (auto/quota-share), verified live: a 12-request
concurrent burst against a TPM-exhausted combo went from 2/12 succeeding
(10 x 524) to 12/12 succeeding with zero 503/524.
Also: liveGeminiShared.ts's sendAndValidate now fails fast on a 503
instead of retrying past it, and the health dashboard + request logger
surface TPM stats alongside RPM/RPD.
Co-authored-by: Markus Hartung <markus.hartung@gmail.com>
* fix(sse): combo-exhausted rejection logs now capture request body + attempted models
recordRejectedRequestUsage() (the fast path for combo requests that never
reach handleChatCore, e.g. all targets locked by resilience cooldown)
hardcoded provider: "-" and never passed a request body to saveCallLog(),
so /dashboard/logs entries for these failures were nearly useless for
debugging: no way to see the client's request or which models were tried.
- recordRejectedRequestUsage() now accepts requestBody and persists it
through the existing saveCallLog() artifact mechanism (same path
handleChatCore's own logging uses).
- Added summarizeComboAttemptedModels(), which reads the combo's own model
list (always available, unlike the response's combo-diagnostics headers —
a model-level resilience-lockout skip never touches the
exhaustedProviders/exhaustedConnections sets those headers are built
from) to populate a real "provider" value instead of "-".
- Wired both into the call site in src/sse/handlers/chat.ts.
NOTE: unrelated to the Gemini TPM/combo-cooldown-wait fix on this branch —
landed here per operator request, to be split into its own branch/PR.
Co-authored-by: Markus Hartung <markus.hartung@gmail.com>
* feat(sse): synthetic streaming keep-alive event + 5-minute Gemini cooldown-wait ceiling
Many clients enforce a first-SSE-byte timeout, which made it unsafe to wait
out a longer upstream rate-limit cooldown on a streaming request — the
client would abandon the connection before any bytes arrived. This landed
in two parts:
1. Synthetic startup "thinking" event (OpenAI chat/completions format):
the already-existing withEarlyStreamKeepalive wrapper (open-sse/utils/
earlyStreamKeepalive.ts, wired into /v1/chat/completions, /v1/messages,
/v1/responses since #2544) opens the SSE stream immediately once a
request runs past its threshold, but only ever sent empty/no-op
keepalive frames. Added a `startupFrame` option (defaults to
`keepaliveFrame` — zero behavior change unless a route opts in) so the
very first frame can carry real content instead. Wired
OPENAI_STARTUP_THINKING_FRAME (a reasoning_content delta: "OmniRoute:
got request, sending to provider") into /v1/chat/completions only —
Claude Messages and Responses API formats both require a preceding
envelope event (message_start / response.created) that a synthetic
pre-dispatch frame can't safely fabricate without risking a duplicate
envelope once the real stream arrives, so those two routes keep their
existing (safe, proven) keepalive frames unchanged.
2. Raised the "wait out a known cooldown, then retry" ceiling to 5 minutes
for both retry mechanisms, now that a client-side first-byte timeout is
no longer a risk on the (opted-in) route:
- comboCooldownWait (auto/quota-share combos, open-sse/services/combo.ts):
maxWaitMs hard clamp raised 90s -> 300s (src/lib/resilience/settings/
normalize.ts); defaults raised to maxWaitMs:90s/maxAttempts:5/
budgetMs:300s. comboConfig.ts's resolveComboTargetTimeoutMsForCombo
already derives the per-target timeout floor from budgetMs, so it
tracks the new ceiling with no further changes.
- waitForCooldown (direct, non-combo model requests, src/sse/handlers/
chat.ts): this mechanism had NO cumulative cap before — only a
per-wait cap (maxRetryWaitMs) and a retry count (maxRetries), so
maxRetries x maxRetryWaitMs could exceed 5 minutes with no ceiling.
Added a budgetMs field (mirrors comboCooldownWait) to
WaitForCooldownSettings/CooldownAwareRetrySettings, threaded a
requestRetryBudgetLeftMs tracker through chat.ts's requestAttemptLoop
(mirrors combo.ts's comboCooldownBudgetLeftMs), and made
getCooldownAwareRetryDecision refuse to wait once the cumulative
budget is exhausted even if the single wait is under maxRetryWaitMs.
Co-authored-by: Markus Hartung <markus.hartung@gmail.com>
* fix(sse): extend the synthetic keep-alive thinking event to /v1/responses
Live incident (OpenClaw, log id 1784407081908-cbc24f): a /v1/responses
request to gemini/gemma-4-31b-it took 56s to produce a first byte and the
client disconnected (499 request_signal_aborted) — the same client-first-byte-
timeout problem the previous commit fixed for /v1/chat/completions, but
/v1/responses only had the generic bare-comment keepalive (no content), so it
wasn't covered.
Added RESPONSES_STARTUP_THINKING_FRAME: a self-contained synthetic reasoning
item (response.output_item.added -> reasoning_summary_part.added ->
reasoning_summary_text.delta -> reasoning_summary_part.done), opened AND
closed within this one frame rather than left dangling — it never carries a
response_id, so it can't collide with the real upstream response's own
independent response.created lifecycle that follows. Mirrors the abbreviated
delta+part.done close pattern open-sse/utils/stream.ts's own
emitSyntheticResponsesReasoningSummary already uses for real mid-stream
reasoning content.
Wired into src/app/api/v1/responses/route.ts via the startupFrame option
added in the previous commit.
Co-authored-by: Markus Hartung <markus.hartung@gmail.com>
* fix(sse): combo cooldown-wait vars reset every setTry, crystallizing a bogus 503 instead of waiting
Live incident (log id 1784416706646-51): a request to the "default" combo
(strategy=auto, maxSetRetries=3) hit a real Gemini TPM 429 on both gemma-4
targets, correctly classified as a short 40s rate_limit lockout — then
crystallized a 503 "all upstream accounts are inactive" in 6.9s instead of
ever reaching the cooldown-aware wait.
Root cause: `lastError`/`earliestRetryAfter`/`lastStatus` were declared with
`let` INSIDE the `for (setTry...)` loop body, so they reset to null at the
start of every set-try. When both targets lock out on setTry 0, every
subsequent setTry (1..maxSetRetries) pre-skips both targets via the
isModelLocked check with no real dispatch — so on the FINAL setTry (the only
one whose values the post-loop decision reads, since it's gated behind
`if (setTry < maxSetRetries) continue`), lastStatus was null, hitting the
"!lastStatus" branch (ALL_ACCOUNTS_INACTIVE 503) and completely bypassing the
comboCooldownWaitEnabled / earliestRetryAfter wait logic — even though a
real 429 with a known ~40s retry-after WAS observed on setTry 0.
This bug predates today's Gemini TPM work (any combo with maxSetRetries > 0
whose targets all lock out on the first pass was affected) but was masked in
existing tests: the "auto strategy (2 models...)" regression test uses
maxSetRetries: 0, so it only ever runs ONE setTry iteration and never
exercises the reset-on-retry path. It also explains why the dedicated
12-concurrent-request burst test passed cleanly — with concurrent requests,
timing variance meant some request's FINAL setTry iteration still had a live
target to dispatch to, giving lastStatus/earliestRetryAfter fresh data. A
single isolated request has no such luck.
Fix: hoist lastError/earliestRetryAfter/lastStatus to just inside
dispatchWithCooldownRetry, before the setTry loop, so they persist across
set-tries (still reset fresh on each recursive dispatchWithCooldownRetry()
call after a wait, which is correct). recordedAttempts/fallbackCount/
exhaustedProviders etc. are intentionally left per-iteration (unrelated to
this bug).
New regression test in tests/unit/combo-quota-share-cooldown-wait.test.ts
reproduces the exact live scenario (2 targets, both lock out on setTry 0,
maxSetRetries: 3) — confirmed red (503) against the pre-fix code, green
(200, waits and retries) against the fix.
Co-authored-by: Markus Hartung <markus.hartung@gmail.com>
* test(sse): extend live Gemini workload to Responses API + add large-context TPM test
Two additions to the live Gemini test suite, both live-verified against the
dev instance:
1. sendAndValidate() (tests/integration/liveGeminiShared.ts) now accepts an
apiFormat: "chat" | "responses" parameter, building the Responses-API
request shape (input array, max_output_tokens) and parsing its SSE events
(response.output_text.delta / response.reasoning_summary_text.delta /
response.completed) via the new readResponsesSSEStream(). Wired into two
new tests in live-gemini-workload.test.ts ([30]/[31]), mirroring the
existing Chat Completions streaming coverage. Verified live: 24/25 + 5/5
payloads succeeded end-to-end through the new code path (the one failure
was a ~300s test-client fetch timeout unrelated to the Responses API code
itself — a separate, not-yet-addressed test-harness limitation).
2. genHugeContextMessage() builds a single message large enough (~4
chars/token estimate) to approach or exceed Gemini's free-tier TPM ceiling
(16000 input tokens/min for gemma-4) by itself. Every other prompt
generator in this file tops out around 1-2k tokens — nowhere near that
ceiling — so none of the existing workload tests ever exercised a REAL TPM
429, only RPM-style rate limiting. tests/integration/gemini-large-context-tpm.test.ts
sends two ~12-13k-token requests back-to-back (comfortably exceeding
16000/min together) to exercise the full path against production Gemini:
TPM classification, the comboCooldownWait retry, and the synthetic
keep-alive frame on a genuinely slow request.
Co-authored-by: Markus Hartung <markus.hartung@gmail.com>
* fix(sse): abandoned combo target dispatch now observes its own per-target timeout, fixing a permanent "pending" dashboard leak
Live incident (dashboard log id 1784418258231-14961a, reported as "an ongoing
request even though there's already a 200"): a combo target dispatch
abandoned by comboTargetTimeoutMs (open-sse/services/combo/targetTimeoutRunner.ts)
left a permanent phantom "pending" entry in the dashboard, even after the
overall combo request had already succeeded via a different retry.
Root cause: chatCore.ts's createStreamController — and everything downstream
that depends on it (withRateLimit's Promise.race against Bottleneck,
acquireAccountSemaphore) — only ever watches clientRawRequest.signal, which
is the ORIGINAL client's request signal (set once via buildClientRawRequest
and reused unchanged across every target dispatch in a combo). It has no
connection to targetTimeoutRunner.ts's OWN AbortController
(target.modelAbortSignal), which is what actually fires when
comboTargetTimeoutMs (300s) elapses. src/sse/handlers/chat.ts's
handleSingleModel bridge between combo.ts and handleSingleModelChat received
`target.modelAbortSignal` but silently dropped it — never forwarded it
anywhere. So when a target got abandoned (e.g. stuck inside a wedged
Bottleneck rate-limiter queue, see the WEDGED force-reset log line from the
same incident), its per-target timeout fired and let the COMBO move on and
retry successfully elsewhere — but the abandoned dispatch's own promise
chain never learned it had been superseded, so it hung forever waiting on a
signal that was never going to fire, and trackPendingRequest(false) (the
finalize call) never ran.
Fix: thread target.modelAbortSignal through as a new modelAbortSignal
runtimeOption, and merge it into clientRawRequest.signal (via the existing
mergeAbortSignals helper from open-sse/executors/base.ts) right before
dispatch, so an abandoned target's own promise chain now observes its abort
and can reach its cleanup path — new resolveDispatchClientRawRequest() makes
this mechanically testable in isolation.
Co-authored-by: Markus Hartung <markus.hartung@gmail.com>
* fix(sse): combo cooldown-wait state recording, rate-limit wedge recovery, OpenAI-format SSE error frames
Five related fixes surfaced by live incidents (dashboard log ids 1784457764961-73,
1784465227489-a2cbc0, 1784504040241-6f8b9a) while validating the Gemini TPM/cooldown-wait
work on this branch against real OpenClaw traffic:
- combo.ts: the model-lockout bail-out branches in dispatchWithCooldownRetry never
recorded lastStatus, so once every target in a set hit an existing lockout the final
check crystallized a bogus ALL_ACCOUNTS_INACTIVE 503 instead of reaching the
cooldown-wait decision, even with a real 429 + short retry-after observed.
- combo.ts/combo/types.ts: the "all credentials cooling down" pre-dispatch rejection
(buildModelCooldownBody) nests its retry hint as error.retry_after/reset_seconds, not
the top-level retryAfter every other 429 shape uses — combo's extraction only read the
latter, so earliestRetryAfter stayed null for this shape even after lastStatus was fixed.
- rateLimitManager.ts: the wedge-recovery watchdog used disconnect(), which releases the
heartbeat timer but never rejects jobs already QUEUED on that instance — orphaned
dispatches hung until the outer ~300s per-target timeout, well past real clients'
patience. Switched to stop({ dropWaitingJobs: true }), safe because the wedge condition
already requires RUNNING===0 && EXECUTING===0.
- earlyStreamKeepalive.ts: the in-band error frame emitted after committing to a 200 SSE
stream was hardcoded to Anthropic's `event: error` convention for every route, including
the OpenAI-format ones (/v1/chat/completions, /v1/responses) where that framing is
either invisible or malformed to a plain data-line parser. Added per-route
OPENAI_CHAT_ERROR_FRAME / OPENAI_RESPONSES_ERROR_FRAME and wired them in.
- chatCore.ts: persisted a synthetic clientResponse error body even when the client had
already disconnected (AbortError) before that body was ever computed — misleading the
dashboard into showing "what the client received" for a response that was never sent.
Also: RequestLoggerDetail.tsx — Provider/Client Event Stream panes lost their collapse
toggle when StreamSection replaced the collapsible PayloadSection (692d6be80, unifying
active/finished request views) without carrying the toggle over.
Each fix has a TDD regression test with a confirmed red-before-green cycle.
Co-authored-by: Markus Hartung <markus.hartung@gmail.com>
* test(sse): free-tier model + gemma-4 TPM-ceiling benchmark harness
Adds a live benchmark comparing free models OmniRoute exposes across
configured providers plus previously-unexercised no-auth providers
(felo-web, aihorde, opencode, duckduckgo-web — none need a connection
row, they were just never tried). Reuses liveGeminiShared.ts's SSE
parsers and CASE_BUILDERS instead of duplicating them.
Also adds a targeted TPM-stress test firing back-to-back large-context
prompts at the gemma-4-31b model across its 3 free hosts (Gemini,
NVIDIA, AI Horde) to isolate whether the documented 16k-tokens/minute
free-tier ceiling is Gemini-specific enforcement or an inherent
model property.
FORCE_TOOL_CHOICE_REQUIRED is a test-only, default-off env flag added
to liveGeminiShared.ts and live-gemini-agentic-loop.test.ts for an
earlier live A/B comparison of tool_choice: required vs unset — kept
as a reusable knob for future runs.
Co-Authored-By: Markus Hartung <markus.hartung@gmail.com>
* test(sse): benchmark for the 2026-07-22 newly-enabled provider batch
Adds NEWLY_ENABLED_MODELS to freeModelBenchmarkShared.ts (Mistral
Leanstral, OpenRouter's live "free"-tagged roster, OpenCode Zen's
current free models — refetched live from
https://opencode.ai/zen/v1/models since the static catalog had
drifted) and a dedicated workload benchmark test for them.
Co-Authored-By: Markus Hartung <markus.hartung@gmail.com>
* test(sse): sync geminiRateLimitTracker tests with e74a1722b's corrected Gemma 4 limits
e74a1722b updated geminiRateLimits.json's gemma-4-* entries from the stale
15/1500/-1 (rpm/rpd/tpm) to the real published free-tier values
16000/14400/16000, but never updated the tests asserting the old numbers.
Surfaced by running the full test:unit suite as a post-rebase sanity check.
Co-Authored-By: Markus Hartung <markus.hartream@gmail.com>
* chore(quality): file-size baseline for own-growth (#8213)
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
---------
Co-authored-by: Markus Hartung <markus.hartung@gmail.com>
Co-authored-by: Markus Hartung <markus.hartream@gmail.com>
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
* fix(sse): synthesize tool_calls for Gemini's malformed function-call abort reasons
Live incident (dashboard log id 1784489701456-d8c0e9): Gemini terminates a
stream with finishReason MALFORMED_FUNCTION_CALL/UNEXPECTED_TOOL_CALL when its
own parser rejects an attempted tool call — there's no real functionCall part,
only a human-readable finishMessage. gemini-to-openai.ts passed this through
raw as finish_reason (9router#2462's fix, correctly keeping it off a clean
"stop"/Claude end_turn), but a raw "malformed_function_call" isn't one of
OpenAI's 5 documented finish_reason values, so a real OpenAI-format client
(OpenClaw) has no handling for it at all and silently never notices the turn
failed — confirmed live via tests/integration/live-gemini-workload.test.ts's
[28] streaming case after the Gemini TPM/rebase work on this branch.
Fix: synthesize a tool_calls entry (arguments carry the error code + Gemini's
finishMessage, valid JSON) and finish_reason: "tool_calls" instead, routing
the failure into the ordinary "tool call arguments didn't parse" path every
OpenAI-compatible agent loop already handles. Defers to a real tool call if
one already completed earlier in the same turn — the real call wins, no
synthetic entry piles on top of it.
Tests (TDD, each confirmed red-before-green):
- 5 new unit tests in the existing 9router#2462 regression file, covering the
synthesis itself, UNEXPECTED_TOOL_CALL, the real-tool-call-wins edge case,
and no-regression on a clean STOP.
- New fixture (tests/fixtures/translation/gemini-malformed-function-call-stream.json):
the real 6-chunk event series from the live incident, sanitized (personal
paths/URLs replaced with generic placeholders, structure preserved exactly).
- New integration test chains the real translator into the real Responses API
transformer using that same fixture, proving correct behavior on BOTH
/v1/chat/completions and /v1/responses from one shared ground-truth event
series.
Co-authored-by: Markus Hartung <markus.hartung@gmail.com>
* fix(sse): don't drop a malformed tool-call failure when it lands beside a real one
Live incident (dashboard log id 1784589106014-2a42f8), analyzing why the
prior malformed-function-call fix (3568c7259) still wasn't reaching the
client in this case: Gemini can emit a REAL, valid functionCall AND finish
the SAME candidate with MALFORMED_FUNCTION_CALL — the model attempted
multiple tool calls in one turn (here: a real status-check call plus a
malformed "exec"+"cron" multi-call attempt), one parsed cleanly and the
other didn't.
The first fix version skipped synthesizing a failure signal whenever a real
tool call already existed (state.toolCalls.size > 0), on the assumption
that meant the model was retrying a LATER, separate attempt after an
earlier one already succeeded. That's indistinguishable, from the
translator's state, from this same-turn case — so it silently discarded
the malformed attempt's information entirely: the client saw the real call
succeed and never learned the other tool calls were attempted and rejected.
Fix: always synthesize the failure entry when a malformed abort reason is
seen, appending it alongside any real tool call rather than skipping it.
Multiple tool_calls in one response is normal, well-supported OpenAI
behavior (parallel tool calls), so this adds the failure as an additional
entry instead of replacing or hiding the real one.
Tests (TDD, confirmed red-before-green):
- Rewrote the unit test that encoded the old (wrong) assumption to assert
both the real and synthesized calls are present.
- New fixture (gemini-malformed-function-call-parallel-real-call-stream.json):
the real event series from this incident, sanitized.
- New integration tests (same file as 3568c7259's) prove both
/v1/chat/completions and /v1/responses surface both tool calls correctly
from this fixture.
Co-authored-by: Markus Hartung <markus.hartung@gmail.com>
* feat(sse): honor tool_choice when translating OpenAI requests to Gemini
Investigating a live report that gemini-3.1-flash-lite frequently narrates
an intended tool call in plain text instead of actually emitting one
(dashboard log id 1784591483850-49c408 — 9 raw provider chunks, all plain
text, zero functionCall parts, clean finishReason STOP): body.tool_choice
was never read anywhere in the OpenAI->Gemini request translator.
result.toolConfig was unconditionally hardcoded to
{ functionCallingConfig: { mode: "VALIDATED" } } whenever tools were
present, regardless of what the caller sent. VALIDATED lets the model
respond with plain text OR a schema-validated function call at its own
discretion — it never forces a call the way OpenAI's tool_choice:
"required" (Gemini's ANY mode) does, so a caller had no way to compel a
tool call even when explicitly requesting one.
Added convertOpenAIToolChoiceToGemini(), mirroring the existing
convertOpenAIToolChoice() in openai-to-claude.ts for the same OpenAI
tool_choice shapes (string "auto"/"none"/"required", or
{type:"function",function:{name}} to force one specific tool):
- unset/"auto" -> VALIDATED (unchanged default, no regression)
- "required"/"any" -> ANY (forces a call)
- "none" -> NONE (disables function calling)
- {type:"function",...} -> ANY + allowedFunctionNames: [name]
Wired into both Gemini request paths: the direct/base translator
(openaiToGeminiBase) and the Antigravity/Cloud Code envelope
(wrapInCloudCodeEnvelope), which now reuses the base translator's already-
computed toolConfig instead of re-deriving its own hardcoded VALIDATED.
This unblocks (but does not itself resolve) the live question — a
tool_choice: "required" A/B test against gemini-3.1-flash-lite follows to
confirm ANY mode actually changes the narrate-vs-act behavior in practice.
Also updates the T11 any-budget allowlist for this file: the "any" string
comparisons (tool_choice value "any", not a TypeScript type) are the same
documented false-positive pattern already carved out for executors/base.ts.
Tests (TDD, confirmed red-before-green): 9 new unit tests covering all
tool_choice shapes on both the direct and Antigravity/Cloud Code paths,
plus the no-tools and unset-default no-regression cases.
Co-authored-by: Markus Hartung <markus.hartung@gmail.com>
* chore(quality): file-size baseline for own-growth (#8211)
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
---------
Co-authored-by: Markus Hartung <markus.hartung@gmail.com>
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
* fix: strip internal reasoning placeholder from user-visible content (#8081)
The internal reasoning replay sentinel '(prior reasoning summary unavailable)'
can leak into user-visible assistant content when a model echoes it through
ordinary message.content / delta.content. Existing suppression only checked
reasoning_content fields and reasoning-specific events.
Changes:
- Add stripInternalReasoningPlaceholder() to reasoningPlaceholder.ts —
removes all occurrences of the sentinel and trims; returns '' when
nothing meaningful remains
- Streaming: strip in responsesTransformer.ts, openai-responses.ts, and
openai-to-claude.ts at the delta.content entry point; skip emission
entirely when only the placeholder was present
- Non-streaming: strip in responseSanitizer.ts sanitizeMessageContent()
and sanitizeResponsesMessageContent() (all three text paths)
translateText is unaffected (uses mode='translate' via plain newsClient).
The per-provider reasoning_content check remains as defense-in-depth.
* fix: skip only empty content block on reasoning-placeholder, keep finish_reason/tool_calls (#8081)
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
* chore(quality): rebaseline openai-responses.ts own-growth (#8081 guard)
---------
Co-authored-by: Austin Liu <austinliu@Austins-MacBook-Air-3.local>
Co-authored-by: Probe Test <probe@example.com>
Co-authored-by: Dingding-leo <Dingding-leo@users.noreply.github.com>
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
