Files
OmniRoute/tests
Wu Shuwen 10bb627576 fix(evals): mark eval-runner requests as self-managed so cases measure the model (#13139) (#13206)
* fix(evals): mark eval-runner requests as self-managed so cases measure the model

executeEvalCase() built its request with only Content-Type and Authorization, so
every graded case picked up the chat path's contextual injections: a selected
output style was prepended as a system message (gated on
`x-omniroute-compression`) and, once the request carried an API key, retrieved
memory plus the built-in `memory_*` tools were appended (gated on
`x-omniroute-no-memory`). An evaluation therefore measured the operator's
injected context as much as the model, and passing an API key to a run made its
score worse, because the key is what gives the request a memory owner (Refs #13139).

Both are documented request-header opt-outs, so the runner now sets them on every
case. Request construction moves to an exported buildEvalCaseRequest() so the
header contract is testable without invoking the chat route.

* docs(changelog): add the eval-runner self-managed-context fragment (#13206)

---------

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
2026-09-18 11:58:30 -03:00
..