Commit Graph

3 Commits

Author SHA1 Message Date
3g0r1ch
d87b97a786 feat(routing): adaptive feedback loop v2 — operational/semantic quality, confidence, TTFT/ITL, end-to-end test (#10881)
Obrigado — feature substancial e bem estruturada: separa qualidade operacional (comportamento de wire: 4xx/5xx, 429, respostas malformadas, stream interrompido) de qualidade semântica (só setada por avaliadores externos, nunca inferida do sucesso HTTP), com confidence/sample-awareness para não deixar poucos sucessos de sorte dominarem o ranking. Instrumentação de streaming (TTFT/ITL) threaded até RoutingEvent, endpoint de explicabilidade, e teste E2E determinístico cobrindo degradação→recuperação→blip.

Validação (worktree própria a partir de origin/release/v3.8.50, merge limpo, 0 conflitos):
- typecheck:core limpo, complexity/cognitive-complexity dentro do baseline
- 59/59 testes passando (mlx-provider, routing-adaptive-e2e, routing-events(-concurrency), routing-otel, routing-quality, routing-scoring-quality, stream-timing, auto-combo-scoring-clamp)
2026-08-20 17:28:30 -03:00
Xiangzhe
7c05c9b46b perf(video): add sampler and contact-sheet benchmark script 2026-08-18 08:25:16 -03:00
MumuTW
4bf47c9b80 chore(perf): add deterministic request-body heap benchmark (#7847) (#8549)
#7847 reports a 3.05 MiB request (729 messages / 86 tools) reaching ~12,282 MiB of V8
heap, and asks for "a regression benchmark that records peak heap for representative
500-800-message, tool-rich requests" before any fix lands. There is currently no memory
baseline in the repo at all (bench:compression is the only benchmark), so a clone-reduction
change could neither be justified nor regression-guarded.

npm run bench:heap-body attributes retained heap to each copy the chat path makes:

  | mechanism                          | call site                          | retained | x wire |
  | cloneLogPayload (unbounded)        | chat.ts buildClientRawRequest      | 3.18 MiB |  1.04x |
  | cloneBoundedForLog (bounded)       | requestLogger.logClientRawRequest  | 0.04 MiB |  0.01x |
  | structuredClone x3 (combo targets) | combo.ts attemptBody               | 9.53 MiB |  3.12x |
  | JSON.stringify (token estimate)    | combo.ts estimateTokens            | 3.06 MiB |  1.00x |
  | per request (sum)                  |                                    |15.81 MiB |  5.17x |

It measures the real production helpers rather than reimplementations, so a change to the
log bounds or the clone strategy is reflected directly.

Design notes:
- Deterministic: fixed-seed LCG, no Math.random(). Verified byte-identical across three
  consecutive runs — without that, a before/after delta measures noise, not the change.
- Corpus lives in its own side-effect-free module so the unit test can import it without
  booting SQLite (requestLogger transitively opens the DB at import time).
- Hermetic: DATA_DIR is redirected to a temp dir before importing, so the benchmark never
  touches the operator's real ~/.omniroute store.
- Node, not bun: --expose-gc and V8 heap accounting are the measurement; another engine's
  heap number would not describe the production runtime.
- --max-retained-mib exits non-zero, so this can become a CI gate once a target is agreed.

Reports only; wires nothing into CI and changes no production code.
2026-07-26 03:52:42 -03:00