mirror of
https://github.com/diegosouzapw/OmniRoute.git
synced 2026-09-21 22:32:22 +03:00
9c5d60027e
* fix(api): stream /api/logs/export with row cap to prevent V8 heap OOM (#13123) Fixes #13123 GET /api/logs/export buffered every matching row into a single JSON.stringify call with pretty-printing (null,2), roughly doubling the string size. On tables with tens of thousands of rows this crashed the Node process with a V8 heap OOM, taking the gateway down for minutes. Changes: - Stream the response via ReadableStream, serializing one row at a time so peak memory stays bounded regardless of table size. - Add a configurable row cap (limit query param, default 10000, max 50000) so callers cannot accidentally request unbounded exports. - Remove pretty-printing (callers can pretty-print client-side). - Include cap metadata (capped, limit, totalAvailable) when the cap fires so callers know they received a truncated result. - Preserve backward-compatible response envelope: { count, hours, type, logs, ... }. * fix(api): push the /api/logs/export row cap down into the DB layer (#13123) The route-layer streaming + cap from the previous pass still called exportCallLogsSince()/exportProxyLogsSince(), which hydrated and buffered EVERY matching row (including rows beyond the limit) before the cap was ever applied — peak V8 heap was essentially unchanged. Adds countCallLogsSince()/countProxyLogsSince() (cheap COUNT(*), no row hydration, used for totalAvailable) and iterateCallLogsSince()/ iterateProxyLogsSince() that bound the query with SQL LIMIT and yield/hydrate one row at a time: a generator over a LIMIT-bounded id list for call_logs, and fixed-size LIMIT/OFFSET pages for proxy_logs (the shared SqliteAdapter only exposes run/get/all, not a `.iterate()` cursor, so LIMIT/OFFSET pagination is the available cursor-equivalent without widening that interface across all 4 driver adapters). The route now streams from these instead, so the full matching row set is never buffered. Also moves capped/limit/totalAvailable into the response header instead of only the trailer, so a client consuming the stream incrementally learns about truncation before processing every row. Rewrote the test to call the real route.GET handler against a seeded test database instead of a local reimplementation of the stream builder, so a regression in the route or its DB-layer delegates is actually caught. Documents the pre-existing (now more clearly load-bearing) breaking change in a changelog fragment: `limit` defaults to 10,000 rows, so exports that previously returned everything are silently truncated. Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com> --------- Co-authored-by: Koosha Pari <koosha@phenotype.ai> Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
changelog.d/ — changelog fragments
A PR never edits CHANGELOG.md directly during the cycle. Instead it adds ONE new
file here — its changelog entry as a fragment. Two PRs never touch the same file, so
changelog merge conflicts (the "CHANGELOG-eat" cascade that forced a re-sync push + full
CI re-run after every sibling merge) are structurally impossible.
Convention
| Directory | Aggregates under |
|---|---|
features/ |
### ✨ New Features |
fixes/ |
### 🐛 Bug Fixes |
maintenance/ |
### 📝 Maintenance |
- Filename:
<PR-number>-<short-slug>.md(e.g.fixes/6700-dockerfile-better-sqlite3.md). The PR number prefix keeps aggregation order deterministic. - Content: the exact bullet line(s) that should land in
CHANGELOG.md, starting with-. Multi-line (continuation) bullets are fine. Keep the repo's credit format:(#PR — thanks @user). - One fragment per PR (rarely more, e.g. a PR that both fixes and adds).
Example
changelog.d/fixes/6496-cloudflare-relay-worker-syntax.md:
- **fix(providers):** Cloudflare relay Worker deploys use Service Worker syntax with `body_part` metadata ([#6496](https://github.com/diegosouzapw/OmniRoute/pull/6496)) — thanks @SeaXen
Aggregation
The release captain (or /generate-release) folds all fragments into CHANGELOG.md and
deletes them:
node scripts/release/aggregate-changelog.mjs # write + delete fragments
node scripts/release/aggregate-changelog.mjs --dry-run # preview only
Fragment well-formedness is enforced by npm run check:changelog-integrity (the same
gate that guards against CHANGELOG-eat for legacy direct edits).