Files
OmniRoute/tests/unit/db-usageanalytics-split.test.ts
growab e9f297021e fix(usage): correct token/request counting for 30D/90D/YTD/ALL ranges (#7300)
* chore(ci): add .mergify.yml to main — Mergify only reads config from the default branch (#7168)

* fix(ci): add the auto-enqueue pull_request_rule to the Mergify config (queue_conditions alone are eligibility-only) (#7179)

* fix(ci): migrate Mergify auto-enqueue to merge_protections_settings.auto_merge_conditions (rules-based path is EOL 2026-07-16) (#7216)

* fix(ci): drop Mergify batch settings (batching is a paid-tier feature; free plan queue is serial) (#7220)

* fix(ci): merge queue tolerates the advisory dast-smoke failure (its GH-hosted build hang dequeued every attempt) (#7225)

* fix(usage): correct token/request counting for 30D/90D/YTD/ALL ranges

Two bugs caused incorrect usage statistics for date ranges beyond the
raw data retention window:

1. Cutoff mismatch: the analytics route computed rawCutoffDate from
   aggregation.rawDataRetentionDays (migration 046 seeds =7) while
   cleanupUsageHistory rolls up and deletes at retention.usageHistory
   (=30). The window [day-30, day-7) existed in usage_history but was
   excluded from BOTH UNION legs — raw leg floored at day-7, aggregated
   leg ended at day-7 — producing undercounted token sums for 30D,
   90D, YTD, and ALL ranges.

   Fix: use dbSettings.retention.usageHistory for the raw cutoff in
   both route.ts and getRawDataCutoffDate() (aggregateHistory.ts),
   matching the actual cleanup boundary.

2. Request undercount: COUNT(*) on the unified source counted each
   daily_usage_summary row as 1, not total_requests. A day with 50
   rolled-up requests counted as 1.

   Fix: add a 'requests' column to both UNION legs (raw: 1, aggregated:
   total_requests), change COUNT(*) to SUM(requests) in 6 query
   functions, and change successfulRequests from
   SUM(CASE WHEN success=1 THEN 1 ELSE 0 END) to
   SUM(CASE WHEN success=1 THEN requests ELSE 0 END). Also set
   agg leg latency_ms to NULL so AVG(latency_ms) is not skewed.

Tests: 33/33 source-level tests pass (db-usageanalytics-split.test.ts),
verifying 'requests' column presence and SUM(requests) usage in all
affected queries. DB-level integration test added to
usage-analytics.test.ts (requires node + better-sqlite3).

* fix(ci): green CI reds on #7300 — file-size ratchet, stale test fixture, shallow-checkout selfref test

- src/app/api/usage/analytics/route.ts: trim the new comment to keep the file
  at the frozen file-size baseline (942 lines) after the retention.usageHistory
  cutoff fix — no logic change.
- tests/unit/usage-analytics-route.test.ts: the pre-existing "does not
  double-count raw and aggregated rows" test hardcoded a 30-day cutoff that
  matched the OLD (buggy) aggregation.rawDataRetentionDays default. Now that
  the raw/aggregated boundary correctly uses retention.usageHistory (365 days
  by default, matching cleanupUsageHistory's actual rollup/delete boundary),
  the fixture's synthetic "old" row was within the raw window and got
  excluded from the aggregated leg. Read the real retention setting instead
  of hardcoding 30 so the fixture reflects the corrected boundary. Same
  assertions (still expects no double-counting, totalRequests=2,
  totalTokens=185) — only the fixture dates change.
- tests/unit/check-test-masking-selfref-6634.test.ts: tolerate the shallow/
  single-ref checkout used by GitHub-hosted Unit Tests runners (no local
  origin/main ref) by fetching it on demand and skipping (never failing) when
  unreachable offline. Matches the fix already applied on another branch
  (2e42b8efc/#7174) for the same root cause, not yet on main.

Co-authored-by: diegosouzapw <diegosouza.pw@gmail.com>

* test(ci): make the #6634 selfref test checkout-independent (read the real file, no git ref)

The previous on-demand `git fetch origin main` + t.skip() fallback cleared the
shallow-checkout failure but tripped the PR Test Policy's test-masking gate
(a new .skip counts as a silenced assert — correctly so).

Drop the git dependency entirely instead: read the REAL current source of
tests/unit/check-test-masking.test.ts from disk (so the actual #6404 fixture
literals stay under test) and model the pre-#6404 state with an empty base,
which maximizes headTaut - baseTaut — the strictest input for the exclusion
this test asserts. No skip, no weakened assertion, same deepEqual guarantee.

Verified non-vacuous: neutralizing SELF_TEST_FIXTURE_RE in
scripts/check/check-test-masking.mjs makes this test fail; restoring it makes
it pass.

Co-authored-by: diegosouzapw <diegosouza.pw@gmail.com>

* test(ci): make the #6634 selfref guard hermetic — main's copy hard-fails every PR (#7341)

main's copy of this test still does git I/O inside a unit test:

    const baseSrc = git(['show', 'origin/main:' + FILE]);

Runners check out a shallow single ref, so origin/main does not resolve and the
test dies with 'fatal: invalid object name origin/main'. Every PR into main
fails Unit Tests (7/8) on it — today that is #7313, #7315, #7316, #7334, #7336
and #7337, six PRs red on a defect none of them introduced. #7313 has no other
red at all.

release/v3.8.49 already carries a fix (2e42b8efc, #7174: try/catch, fetch
origin/main on demand, t.skip() when unreachable), but it only reaches main at
release time — so main stays broken for the whole cycle. Cherry-picking it would
also import a new problem: PR Test Policy classifies t.skip() as a silenced
assertion, which we watched it correctly catch on #7300 today.

This is the hermetic version instead (ported from #7327, which does the same for
the release branch): read the file straight off disk, compare against an empty
base so baseTaut/baseExtTaut are 0 — the strictest possible comparison point —
and call evaluateMasking() directly. No git ref, no fetch, no skip, nothing the
runner's checkout depth can break.

The #6634 regression stays covered: the guard's logic lives in
SELF_TEST_FIXTURE_RE (check-test-masking.mjs:337), not in the test. Proven both
ways on main before committing — neutralise SELF_TEST_FIXTURE_RE to /$^/ and
the test FAILS; restore it and it passes 2/2, with check-test-masking.mjs left
byte-identical.

Co-authored-by: growab <nekron@icloud.com>

* chore(quality): tighten main's coverage baseline to the CI's real numbers (#7347)

main's ratchet had been failing --require-tighten on every PR: 11 metrics
improved but the baseline was never tightened. Same class as the #6634
selfref guard — an infra fix that lands only on the release branch leaves
main red for the whole cycle, and every PR into main pays for it.

Values are the merged-coverage numbers from a run on main itself (a local
run measures ~68% vs CI's ~80%; the baseline's own note warns about that
gap). Only the 11 coverage values change — gitleaks and semgrepFindings
keep main's own state.

No changelog fragment: #7326 carries it on release/v3.8.49, and a second
one here would double the entry at release time.

---------

Co-authored-by: Diego Rodrigues de Sa e Souza <8016841+diegosouzapw@users.noreply.github.com>
Co-authored-by: diegosouzapw <diegosouza.pw@gmail.com>
2026-07-23 11:15:18 -03:00

180 lines
7.2 KiB
TypeScript

/**
* Characterization + API-surface test: usageAnalytics.ts god-file decomposition.
*
* The two pure SQL-source-string builders (buildUnifiedSource,
* buildPresetUnifiedSource) + their types were extracted verbatim from
* src/lib/db/usageAnalytics.ts into the pure leaf
* src/lib/db/usageAnalytics/sources.ts (no DB, no imports). The ~20 query
* functions stay in the host.
*
* Verifies that:
* 1. buildUnifiedSource's needsAggregated branching is preserved (pure logic).
* 2. The host usageAnalytics.ts still exposes the FULL public API (39 names).
* 3. The sources leaf exports the builders directly.
*
* Pure value assertions — no DB handle is opened.
*/
import { describe, it } from "node:test";
import assert from "node:assert/strict";
import {
buildUnifiedSource,
buildPresetUnifiedSource,
} from "../../src/lib/db/usageAnalytics/sources.ts";
// ── 1. buildUnifiedSource — pure branching ───────────────────────────────────
describe("usageAnalytics/sources — buildUnifiedSource", () => {
it("includes the daily_usage_summary leg for a wide window with no api-key filter", () => {
const { unifiedSource, unifiedParams } = buildUnifiedSource({
sinceIso: "2024-06-01T00:00:00.000Z",
untilIso: null,
rawCutoffDate: "2024-06-15",
apiKeyWhere: "",
apiKeyParams: {},
});
assert.ok(typeof unifiedSource === "string" && unifiedSource.length > 0);
// needsAggregated = (sinceDate < rawCutoffDate) && !apiKeyWhere => true
assert.ok(
unifiedSource.includes("daily_usage_summary"),
"aggregated leg must be present when the window predates the raw cutoff"
);
assert.equal(unifiedParams.rawCutoff, "2024-06-15");
assert.equal(unifiedParams.rawCutoffDate, "2024-06-15");
// Regression: unified source must surface a `requests` column in both legs
// so COUNT(*)→SUM(requests) counts actual requests, not row count.
assert.ok(unifiedSource.includes("1 as requests"), "raw leg must surface requests column");
assert.ok(unifiedSource.includes("total_requests as requests"), "aggregated leg must surface total_requests as requests");
});
it("drops the aggregated leg when an api-key filter is active (raw-only)", () => {
const { unifiedSource, unifiedParams } = buildUnifiedSource({
sinceIso: "2024-06-01T00:00:00.000Z",
untilIso: null,
rawCutoffDate: "2024-06-15",
apiKeyWhere: "(api_key_id IN (@apiKey0))",
apiKeyParams: { apiKey0: "k1" },
});
// needsAggregated = ... && !apiKeyWhere => false
assert.ok(
!unifiedSource.includes("daily_usage_summary"),
"aggregated leg must be absent once an api-key filter scopes the query to raw rows"
);
// Regression: raw-only fallback must also surface the requests column.
assert.ok(unifiedSource.includes("1 as requests"), "raw-only fallback must surface requests column");
assert.equal(unifiedParams.rawCutoff, undefined);
assert.equal(unifiedParams.since, "2024-06-01T00:00:00.000Z");
assert.equal(unifiedParams.apiKey0, "k1");
});
it("buildPresetUnifiedSource returns the unifiedSource/unifiedParams shape", () => {
const result = buildPresetUnifiedSource({
sinceIso: null,
untilIso: null,
rawCutoffDate: "2024-06-15",
apiKeyWhere: "",
apiKeyParams: {},
});
assert.equal(typeof result.unifiedSource, "string");
assert.equal(typeof result.unifiedParams, "object");
assert.ok(result.unifiedParams !== null);
});
});
// ── 2. usageAnalytics.ts — full public API surface preserved ─────────────────
const host = await import("../../src/lib/db/usageAnalytics.ts");
describe("usageAnalytics.ts public API surface", () => {
// the 22 runtime functions (the 17 row/param interfaces are type-only and
// erased at runtime, so they are asserted via typecheck, not here)
const expectedFns = [
"buildUnifiedSource", // re-exported from sources
"buildPresetUnifiedSource", // re-exported from sources
"getUsageSummary",
"getDailyUsage",
"getDailyCostRows",
"getHeatmapRows",
"getModelUsageRows",
"getProviderCostRows",
"getProviderUsageRows",
"getAccountCostRows",
"getAccountUsageRows",
"getApiKeyUsageRows",
"getServiceTierUsageRows",
"getApiKeyMetadataRows",
"getWeeklyPatternRows",
"getPresetCostModelRows",
"getEndpointUsageRows",
"getAllUsageHistory",
"getAllDomainCostHistory",
"getAllDomainBudgets",
];
for (const name of expectedFns) {
it(`exposes ${name} as a function`, () => {
assert.equal(typeof host[name], "function", `${name} must be a function on the host module`);
});
}
it("loses no public runtime function in the split", () => {
const missing = expectedFns.filter((n) => typeof host[n] !== "function");
assert.deepEqual(missing, [], `missing: ${missing.join(", ")}`);
});
});
// ── 4. Regression: COUNT(*) → SUM(requests) in all query functions ──────────
// Verifies the fix without needing a DB driver — the function source must
// contain SUM(requests), not COUNT(*), so aggregated rows with total_requests>1
// are counted correctly instead of as 1-per-row.
describe("usageAnalytics queries use SUM(requests) not COUNT(*)", () => {
const queriesUsingCount = [
"getUsageSummary",
"getDailyUsage",
"getModelUsageRows",
"getProviderUsageRows",
"getServiceTierUsageRows",
];
for (const fn of queriesUsingCount) {
it(`${fn} uses SUM(requests) not COUNT(*)`, () => {
const src = host[fn].toString();
assert.ok(src.includes("SUM(requests)"), `${fn} must use SUM(requests) for correct aggregated row counting`);
assert.ok(!src.match(/COUNT\(\*\)/), `${fn} must not use COUNT(*) which undercounts aggregated rows`);
});
}
it("getWeeklyPatternRows inner subquery uses SUM(requests)", () => {
const src = host.getWeeklyPatternRows.toString();
assert.ok(src.includes("SUM(requests)"), "getWeeklyPatternRows inner subquery must use SUM(requests)");
});
it("getUsageSummary uses SUM(CASE WHEN success=1 THEN requests ELSE 0 END) for successfulRequests", () => {
const src = host.getUsageSummary.toString();
assert.ok(
src.includes("SUM(CASE WHEN success = 1 THEN requests ELSE 0 END)"),
"successfulRequests must weight by requests, not count 1-per-row"
);
});
it("getModelUsageRows uses SUM(CASE WHEN success=1 THEN requests ELSE 0 END) for successfulRequests", () => {
const src = host.getModelUsageRows.toString();
assert.ok(
src.includes("SUM(CASE WHEN success = 1 THEN requests ELSE 0 END)"),
"successfulRequests must weight by requests, not count 1-per-row"
);
});
});
// ── 3. sources leaf exports the builders directly ────────────────────────────
describe("sources.ts exports the builders directly", () => {
it("buildUnifiedSource / buildPresetUnifiedSource are functions on the leaf", async () => {
const sources = await import("../../src/lib/db/usageAnalytics/sources.ts");
assert.equal(typeof sources.buildUnifiedSource, "function");
assert.equal(typeof sources.buildPresetUnifiedSource, "function");
});
});