Commit Graph

5 Commits

Author SHA1 Message Date
growab
e9f297021e fix(usage): correct token/request counting for 30D/90D/YTD/ALL ranges (#7300)
* chore(ci): add .mergify.yml to main — Mergify only reads config from the default branch (#7168)

* fix(ci): add the auto-enqueue pull_request_rule to the Mergify config (queue_conditions alone are eligibility-only) (#7179)

* fix(ci): migrate Mergify auto-enqueue to merge_protections_settings.auto_merge_conditions (rules-based path is EOL 2026-07-16) (#7216)

* fix(ci): drop Mergify batch settings (batching is a paid-tier feature; free plan queue is serial) (#7220)

* fix(ci): merge queue tolerates the advisory dast-smoke failure (its GH-hosted build hang dequeued every attempt) (#7225)

* fix(usage): correct token/request counting for 30D/90D/YTD/ALL ranges

Two bugs caused incorrect usage statistics for date ranges beyond the
raw data retention window:

1. Cutoff mismatch: the analytics route computed rawCutoffDate from
   aggregation.rawDataRetentionDays (migration 046 seeds =7) while
   cleanupUsageHistory rolls up and deletes at retention.usageHistory
   (=30). The window [day-30, day-7) existed in usage_history but was
   excluded from BOTH UNION legs — raw leg floored at day-7, aggregated
   leg ended at day-7 — producing undercounted token sums for 30D,
   90D, YTD, and ALL ranges.

   Fix: use dbSettings.retention.usageHistory for the raw cutoff in
   both route.ts and getRawDataCutoffDate() (aggregateHistory.ts),
   matching the actual cleanup boundary.

2. Request undercount: COUNT(*) on the unified source counted each
   daily_usage_summary row as 1, not total_requests. A day with 50
   rolled-up requests counted as 1.

   Fix: add a 'requests' column to both UNION legs (raw: 1, aggregated:
   total_requests), change COUNT(*) to SUM(requests) in 6 query
   functions, and change successfulRequests from
   SUM(CASE WHEN success=1 THEN 1 ELSE 0 END) to
   SUM(CASE WHEN success=1 THEN requests ELSE 0 END). Also set
   agg leg latency_ms to NULL so AVG(latency_ms) is not skewed.

Tests: 33/33 source-level tests pass (db-usageanalytics-split.test.ts),
verifying 'requests' column presence and SUM(requests) usage in all
affected queries. DB-level integration test added to
usage-analytics.test.ts (requires node + better-sqlite3).

* fix(ci): green CI reds on #7300 — file-size ratchet, stale test fixture, shallow-checkout selfref test

- src/app/api/usage/analytics/route.ts: trim the new comment to keep the file
  at the frozen file-size baseline (942 lines) after the retention.usageHistory
  cutoff fix — no logic change.
- tests/unit/usage-analytics-route.test.ts: the pre-existing "does not
  double-count raw and aggregated rows" test hardcoded a 30-day cutoff that
  matched the OLD (buggy) aggregation.rawDataRetentionDays default. Now that
  the raw/aggregated boundary correctly uses retention.usageHistory (365 days
  by default, matching cleanupUsageHistory's actual rollup/delete boundary),
  the fixture's synthetic "old" row was within the raw window and got
  excluded from the aggregated leg. Read the real retention setting instead
  of hardcoding 30 so the fixture reflects the corrected boundary. Same
  assertions (still expects no double-counting, totalRequests=2,
  totalTokens=185) — only the fixture dates change.
- tests/unit/check-test-masking-selfref-6634.test.ts: tolerate the shallow/
  single-ref checkout used by GitHub-hosted Unit Tests runners (no local
  origin/main ref) by fetching it on demand and skipping (never failing) when
  unreachable offline. Matches the fix already applied on another branch
  (2e42b8efc/#7174) for the same root cause, not yet on main.

Co-authored-by: diegosouzapw <diegosouza.pw@gmail.com>

* test(ci): make the #6634 selfref test checkout-independent (read the real file, no git ref)

The previous on-demand `git fetch origin main` + t.skip() fallback cleared the
shallow-checkout failure but tripped the PR Test Policy's test-masking gate
(a new .skip counts as a silenced assert — correctly so).

Drop the git dependency entirely instead: read the REAL current source of
tests/unit/check-test-masking.test.ts from disk (so the actual #6404 fixture
literals stay under test) and model the pre-#6404 state with an empty base,
which maximizes headTaut - baseTaut — the strictest input for the exclusion
this test asserts. No skip, no weakened assertion, same deepEqual guarantee.

Verified non-vacuous: neutralizing SELF_TEST_FIXTURE_RE in
scripts/check/check-test-masking.mjs makes this test fail; restoring it makes
it pass.

Co-authored-by: diegosouzapw <diegosouza.pw@gmail.com>

* test(ci): make the #6634 selfref guard hermetic — main's copy hard-fails every PR (#7341)

main's copy of this test still does git I/O inside a unit test:

    const baseSrc = git(['show', 'origin/main:' + FILE]);

Runners check out a shallow single ref, so origin/main does not resolve and the
test dies with 'fatal: invalid object name origin/main'. Every PR into main
fails Unit Tests (7/8) on it — today that is #7313, #7315, #7316, #7334, #7336
and #7337, six PRs red on a defect none of them introduced. #7313 has no other
red at all.

release/v3.8.49 already carries a fix (2e42b8efc, #7174: try/catch, fetch
origin/main on demand, t.skip() when unreachable), but it only reaches main at
release time — so main stays broken for the whole cycle. Cherry-picking it would
also import a new problem: PR Test Policy classifies t.skip() as a silenced
assertion, which we watched it correctly catch on #7300 today.

This is the hermetic version instead (ported from #7327, which does the same for
the release branch): read the file straight off disk, compare against an empty
base so baseTaut/baseExtTaut are 0 — the strictest possible comparison point —
and call evaluateMasking() directly. No git ref, no fetch, no skip, nothing the
runner's checkout depth can break.

The #6634 regression stays covered: the guard's logic lives in
SELF_TEST_FIXTURE_RE (check-test-masking.mjs:337), not in the test. Proven both
ways on main before committing — neutralise SELF_TEST_FIXTURE_RE to /$^/ and
the test FAILS; restore it and it passes 2/2, with check-test-masking.mjs left
byte-identical.

Co-authored-by: growab <nekron@icloud.com>

* chore(quality): tighten main's coverage baseline to the CI's real numbers (#7347)

main's ratchet had been failing --require-tighten on every PR: 11 metrics
improved but the baseline was never tightened. Same class as the #6634
selfref guard — an infra fix that lands only on the release branch leaves
main red for the whole cycle, and every PR into main pays for it.

Values are the merged-coverage numbers from a run on main itself (a local
run measures ~68% vs CI's ~80%; the baseline's own note warns about that
gap). Only the 11 coverage values change — gitleaks and semgrepFindings
keep main's own state.

No changelog fragment: #7326 carries it on release/v3.8.49, and a second
one here would double the entry at release time.

---------

Co-authored-by: Diego Rodrigues de Sa e Souza <8016841+diegosouzapw@users.noreply.github.com>
Co-authored-by: diegosouzapw <diegosouza.pw@gmail.com>
2026-07-23 11:15:18 -03:00
Diego Rodrigues de Sa e Souza
191009dd23 Release v3.8.7 (#2919)
* feat(plugins): WordPress-style plugin system backend

* fix(plugins): address code review feedback

- Path traversal guard: validate entryPoint stays within plugin dir
- install() now handles direct plugin directories (not just parent dirs)
- Non-null assertion replaced with explicit null check
- require efficiency: allowedModules map moved outside function
- Source wrapper: add newlines to prevent trailing comment issues
- Config validation: validate values against configSchema on save
- Dynamic import comment: clarify Node.js caching behavior

Co-Authored-By: OpenClaude (mimo-v2.5-pro) <openclaude@gitlawb.com>

* fix(plugins): replace vm with child_process, add auth to all routes

Addresses all remaining code review feedback:

1. **Loader rewrite**: Replaced Node.js vm module with child_process.fork()
   for proper process-level isolation. Complies with Rule 3 (no eval).
   Each plugin runs in a separate Node.js process with IPC communication.

2. **Auth on all routes**: Added requireManagementAuth to all 6 plugin
   API route files (list, install, scan, details, activate, deactivate, config).

3. **Env filtering**: Only safe env vars passed to plugin processes unless
   "env" permission is granted.

Co-Authored-By: OpenClaude (mimo-v2.5-pro) <openclaude@gitlawb.com>

* fix(plugins): security + ESM fixes for loader and manager

loader.ts:
- Fix IPC: use process.send()/process.on("message") instead of worker_threads.parentPort
- Fix ESM: write host script as .mjs (not .js) to force ESM execution
- Add timeout: 10s default on callHook() with Promise.race
- Add SIGKILL escalation: SIGTERM first, then SIGKILL after 3s grace
- Fix env filtering: use allowlist (safeKeys) instead of passing all env vars
- Clear timeout on successful IPC response (no timer leak)

manager.ts:
- Fix path traversal: use fs.realpath() instead of startsWith()
- Fix imports: use registerHook/unregisterHooks from hooks.ts
- Register hooks individually via registerHook(event, name, handler)

hooks.ts:
- Copied from feat/plugin-custom-hooks (canonical registry)

* feat(discovery): add discovery tool stub service

Phase 1 scaffold for automated provider discovery:
- DiscoveryConfig, DiscoveryResult types
- probeEndpoint() for URL availability checking
- scanProvider() stub (Phase 2 will implement real scanning)
- getDiscoveryResults() stub
- Default config: disabled (opt-in)

* chore(plugins): slop cleanup — pino logger, remove redundant sorts

- index.ts: replace console.log/error with pino structured logging
- hooks.ts: remove redundant .sort() in emitHookBlocking/runOnResponse (already sorted on registration)
- manager.ts: add readFile import

* test(plugins): add scanner, loader, manager unit tests

- scanner: 9 tests (discovery, hidden dirs, validation, entry point, multiple)
- loader: 5 tests (type contracts, Plugin/PluginContext/PluginResult interfaces)
- manager: 6 tests (singleton, lifecycle methods, error on unknown)
- Total: 20 tests, all passing

* fix(settings): add missing home page pin keys to updateSettingsSchema

* feat(plugins): add i18n keys to all 42 locales

* fix(settings): add missing security keys to updateSettingsSchema and add tests

* fix(usage): analytics route reads combo_name/requested_model from call_logs only

The 3.8.6 variant of #2904 added SELECTs of combo_name/requested_model
against usage_history, but those columns only exist in call_logs (no
migration adds them to usage_history). This returned HTTP 500 on
/api/usage/analytics. Restore the working query shape from the 3.8.7
variant. Fixes 18 failing usage-analytics-route tests.

* fix(types,test): resolve noImplicitAny in progressiveAging + align semaphore test to #2903 gate pruning

- progressiveAging: type compression results so messages[0].content is
  indexable (was TS7053 against {}); restores typecheck:noimplicit:core gate.
- services-branch-hardening: #2903 (perf-ram) prunes idle rate-limit gates
  on zero; assert no-running/empty-queue without assuming the entry persists.

* fix(analytics): address merged review regressions

* fix(executor): normalize max effort for openai shape providers

* Make zero-latency combo optimizations opt-in

* Address zero-latency combo review feedback

* chore(release): sync v3.8.7 touchpoints + credit contributors

- llm.txt → 3.8.7 (Current version + Key Features header)
- CHANGELOG: add Dmitry Kuznetsov & Nikolay Alafuzov to 3.8.6 Hall of Contributors
- version already 3.8.7 across package.json/open-sse/electron/openapi (from #2909)

* fix(cleanup): restore usage history cutoff boundary

* docs(changelog): rank 3.8.6 contributors in a commits table with their PRs

* fix(dashboard): theme ReactFlow Controls +/- buttons for dark mode

* fix(settings): add missing home page pin keys to updateSettingsSchema

* fix(settings): add missing security keys to updateSettingsSchema and add tests

* fix(executor): normalize max effort for openai shape providers

* Make zero-latency combo optimizations opt-in

* Address zero-latency combo review feedback

* fix(analytics): address merged review regressions

* fix(cleanup): restore usage history cutoff boundary

* feat(plugins): WordPress-style plugin system backend

* fix(plugins): address code review feedback

- Path traversal guard: validate entryPoint stays within plugin dir
- install() now handles direct plugin directories (not just parent dirs)
- Non-null assertion replaced with explicit null check
- require efficiency: allowedModules map moved outside function
- Source wrapper: add newlines to prevent trailing comment issues
- Config validation: validate values against configSchema on save
- Dynamic import comment: clarify Node.js caching behavior

Co-Authored-By: OpenClaude (mimo-v2.5-pro) <openclaude@gitlawb.com>

* fix(plugins): replace vm with child_process, add auth to all routes

Addresses all remaining code review feedback:

1. **Loader rewrite**: Replaced Node.js vm module with child_process.fork()
   for proper process-level isolation. Complies with Rule 3 (no eval).
   Each plugin runs in a separate Node.js process with IPC communication.

2. **Auth on all routes**: Added requireManagementAuth to all 6 plugin
   API route files (list, install, scan, details, activate, deactivate, config).

3. **Env filtering**: Only safe env vars passed to plugin processes unless
   "env" permission is granted.

Co-Authored-By: OpenClaude (mimo-v2.5-pro) <openclaude@gitlawb.com>

* fix(plugins): security + ESM fixes for loader and manager

loader.ts:
- Fix IPC: use process.send()/process.on("message") instead of worker_threads.parentPort
- Fix ESM: write host script as .mjs (not .js) to force ESM execution
- Add timeout: 10s default on callHook() with Promise.race
- Add SIGKILL escalation: SIGTERM first, then SIGKILL after 3s grace
- Fix env filtering: use allowlist (safeKeys) instead of passing all env vars
- Clear timeout on successful IPC response (no timer leak)

manager.ts:
- Fix path traversal: use fs.realpath() instead of startsWith()
- Fix imports: use registerHook/unregisterHooks from hooks.ts
- Register hooks individually via registerHook(event, name, handler)

hooks.ts:
- Copied from feat/plugin-custom-hooks (canonical registry)

* feat(discovery): add discovery tool stub service

Phase 1 scaffold for automated provider discovery:
- DiscoveryConfig, DiscoveryResult types
- probeEndpoint() for URL availability checking
- scanProvider() stub (Phase 2 will implement real scanning)
- getDiscoveryResults() stub
- Default config: disabled (opt-in)

* chore(plugins): slop cleanup — pino logger, remove redundant sorts

- index.ts: replace console.log/error with pino structured logging
- hooks.ts: remove redundant .sort() in emitHookBlocking/runOnResponse (already sorted on registration)
- manager.ts: add readFile import

* test(plugins): add scanner, loader, manager unit tests

- scanner: 9 tests (discovery, hidden dirs, validation, entry point, multiple)
- loader: 5 tests (type contracts, Plugin/PluginContext/PluginResult interfaces)
- manager: 6 tests (singleton, lifecycle methods, error on unknown)
- Total: 20 tests, all passing

* feat(plugins): add i18n keys to all 42 locales

* chore(plugins): remove duplicate migration 059_create_plugins.sql

* chore(plugins): remove duplicate migration 059_create_plugins.sql (post-merge)

* fix(sse): guard non-string error.code in proxyFetch + harden model parsing (#2463) (#2923)

Integrated into release/v3.8.7

* fix(docker): add runner-web stage with Playwright Chromium (#2832) (#2846)

Integrated into release/v3.8.7

* docs(changelog): document NVIDIA NIM and error code type-crash fix (#2463)

* test: ignore NVIDIA_BASE_URL and NVIDIA_MODEL in env contract check

---------

Co-authored-by: oyi77 <oyi77@users.noreply.github.com>
Co-authored-by: OpenClaude (mimo-v2.5-pro) <openclaude@gitlawb.com>
Co-authored-by: Apostol Apostolov <theapoapostolov@gmail.com>
Co-authored-by: Halil Tezcan KARABULUT <info@hlltzcnkb.com>
Co-authored-by: R.D. <rogerproself@gmail.com>
2026-05-29 19:54:00 -03:00
Halil Tezcan KARABULUT
efcd062002 Fix analytics history and log token accounting (#2904)
Integrated into release/v3.8.6
2026-05-29 13:18:14 -03:00
diegosouzapw
c63c5d5008 fix: resolve MCP endpoint auth bypass (#1970) and finalize DB settings 2026-05-05 09:59:43 -03:00
Paijo
6a45ea2c97 feat(db): consolidate all database settings into SystemStorageTab (closes #1935) (#1947)
Integrated into release/v3.7.9
2026-05-04 19:25:41 -03:00