* feat(eval): add router-eval harness (AIQ scoring, regression gate, Pareto search)
Extracts a standalone router-eval evaluation tool that replays routing
decisions (from NDJSON corpora or the usage_history/call_logs SQLite
tables) into an AIQ (success/latency/cost) score, compares baseline vs.
candidate router configs with a retained-run regression gate, and ranks
Pareto-optimal candidates across a search space — a sibling to the
existing eval:compression harness.
New scripts: scripts/router-eval/{index,compare,patch-compare,search,
trends}.ts, scripts/check/check-router-eval-regression.ts, and
src/lib/routerEval/index.ts, wired via 6 new package.json entries
(eval:router, eval:router:compare, eval:router:patch-compare,
eval:router:search, eval:router:trends, check:router-eval).
Reconstructed onto current release/v3.8.49 from the original ~142-commit
stale PR branch: only the genuinely new router-eval payload (17 files)
was extracted — the other ~560 changed files in the original diff were
base-drift already present on release in newer form. The new package.json
scripts now invoke `node --import tsx` instead of `bun`, matching the
`eval:compression` precedent (Bun is reserved for a closed 5-script
allowlist). The runtime-detection shim (`"Bun" in globalThis`) already
present in the harness gracefully falls back to better-sqlite3 under
Node, so no logic changes were needed there; the retained-run manifest's
previously-hardcoded `runtime: "bun"` field and matching CLI help text
were corrected to reflect the actual invocation.
All 27 existing router-eval unit tests pass unchanged.
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
* refactor(router-eval): decompose toRouterObservation below complexity gate
toRouterObservation had cyclomatic complexity 18 (gate max is 15). Extract
the per-field parsing/normalization into pure helpers (sampleId, model
fields, latency, cost derivation, success) so the entry point is a plain
sequential assembly of a RouterObservation. Behavior is unchanged — same
tests pass, same fallbacks, same precedence between input aliases.
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
---------
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
OmniRoute v3.8.29 — 115 commits since v3.8.28. Full CHANGELOG + 41 i18n mirrors. All content quality gates green (build, unit 8/8, vitest 188/188, PR test policy, quality gates extended, docs sync, quality ratchet). Remaining red CI checks are pre-existing release flakes (coverage-shard/integration/node-compat teardown), a new transitive undici advisory in electron devDeps, and a workflow-level CodeQL fail (0 open alerts). VPS-validated by the operator.
Follow-up to the plugins framework (#3041): wires the plugin hooks end-to-end and adds full test coverage.
- Wires `onRequest` / `onResponse` / `onError` hooks into the chat pipeline (`chatCore` now imports from the unified `hooks` registry).
- Loads active plugins on server startup (`pluginManager.loadAll()` in `server-init`) so they survive restarts.
- Ships a `welcome-banner` example plugin (`examples/plugins/`) + test fixture.
- Adds a comprehensive plugin test suite (manifest, db, config, hooks, manager lifecycle, loader IPC, permissions, scanner, welcome-banner e2e).
Integration fixes applied during review:
- `manager.install` now removes an orphaned `destDir` (DB row gone but files left on disk) before the atomic rename, guarded by path containment — it previously failed with `ENOTEMPTY` (a regression surfaced by the new lifecycle test).
- `plugins-db` / `plugins-manager-lifecycle` tests now initialize the DB via the real migration `076` (`getDbInstance`) rather than relying on ambient state, so a missing/renumbered migration fails loudly instead of being masked.
348/348 plugin tests pass; typecheck / cycles clean.
Co-authored-by: oyi77 <14921983+oyi77@users.noreply.github.com>
