Files
OmniRoute/src/app/api/usage/analytics/route.ts
growab e9f297021e fix(usage): correct token/request counting for 30D/90D/YTD/ALL ranges (#7300)
* chore(ci): add .mergify.yml to main — Mergify only reads config from the default branch (#7168)

* fix(ci): add the auto-enqueue pull_request_rule to the Mergify config (queue_conditions alone are eligibility-only) (#7179)

* fix(ci): migrate Mergify auto-enqueue to merge_protections_settings.auto_merge_conditions (rules-based path is EOL 2026-07-16) (#7216)

* fix(ci): drop Mergify batch settings (batching is a paid-tier feature; free plan queue is serial) (#7220)

* fix(ci): merge queue tolerates the advisory dast-smoke failure (its GH-hosted build hang dequeued every attempt) (#7225)

* fix(usage): correct token/request counting for 30D/90D/YTD/ALL ranges

Two bugs caused incorrect usage statistics for date ranges beyond the
raw data retention window:

1. Cutoff mismatch: the analytics route computed rawCutoffDate from
   aggregation.rawDataRetentionDays (migration 046 seeds =7) while
   cleanupUsageHistory rolls up and deletes at retention.usageHistory
   (=30). The window [day-30, day-7) existed in usage_history but was
   excluded from BOTH UNION legs — raw leg floored at day-7, aggregated
   leg ended at day-7 — producing undercounted token sums for 30D,
   90D, YTD, and ALL ranges.

   Fix: use dbSettings.retention.usageHistory for the raw cutoff in
   both route.ts and getRawDataCutoffDate() (aggregateHistory.ts),
   matching the actual cleanup boundary.

2. Request undercount: COUNT(*) on the unified source counted each
   daily_usage_summary row as 1, not total_requests. A day with 50
   rolled-up requests counted as 1.

   Fix: add a 'requests' column to both UNION legs (raw: 1, aggregated:
   total_requests), change COUNT(*) to SUM(requests) in 6 query
   functions, and change successfulRequests from
   SUM(CASE WHEN success=1 THEN 1 ELSE 0 END) to
   SUM(CASE WHEN success=1 THEN requests ELSE 0 END). Also set
   agg leg latency_ms to NULL so AVG(latency_ms) is not skewed.

Tests: 33/33 source-level tests pass (db-usageanalytics-split.test.ts),
verifying 'requests' column presence and SUM(requests) usage in all
affected queries. DB-level integration test added to
usage-analytics.test.ts (requires node + better-sqlite3).

* fix(ci): green CI reds on #7300 — file-size ratchet, stale test fixture, shallow-checkout selfref test

- src/app/api/usage/analytics/route.ts: trim the new comment to keep the file
  at the frozen file-size baseline (942 lines) after the retention.usageHistory
  cutoff fix — no logic change.
- tests/unit/usage-analytics-route.test.ts: the pre-existing "does not
  double-count raw and aggregated rows" test hardcoded a 30-day cutoff that
  matched the OLD (buggy) aggregation.rawDataRetentionDays default. Now that
  the raw/aggregated boundary correctly uses retention.usageHistory (365 days
  by default, matching cleanupUsageHistory's actual rollup/delete boundary),
  the fixture's synthetic "old" row was within the raw window and got
  excluded from the aggregated leg. Read the real retention setting instead
  of hardcoding 30 so the fixture reflects the corrected boundary. Same
  assertions (still expects no double-counting, totalRequests=2,
  totalTokens=185) — only the fixture dates change.
- tests/unit/check-test-masking-selfref-6634.test.ts: tolerate the shallow/
  single-ref checkout used by GitHub-hosted Unit Tests runners (no local
  origin/main ref) by fetching it on demand and skipping (never failing) when
  unreachable offline. Matches the fix already applied on another branch
  (2e42b8efc/#7174) for the same root cause, not yet on main.

Co-authored-by: diegosouzapw <diegosouza.pw@gmail.com>

* test(ci): make the #6634 selfref test checkout-independent (read the real file, no git ref)

The previous on-demand `git fetch origin main` + t.skip() fallback cleared the
shallow-checkout failure but tripped the PR Test Policy's test-masking gate
(a new .skip counts as a silenced assert — correctly so).

Drop the git dependency entirely instead: read the REAL current source of
tests/unit/check-test-masking.test.ts from disk (so the actual #6404 fixture
literals stay under test) and model the pre-#6404 state with an empty base,
which maximizes headTaut - baseTaut — the strictest input for the exclusion
this test asserts. No skip, no weakened assertion, same deepEqual guarantee.

Verified non-vacuous: neutralizing SELF_TEST_FIXTURE_RE in
scripts/check/check-test-masking.mjs makes this test fail; restoring it makes
it pass.

Co-authored-by: diegosouzapw <diegosouza.pw@gmail.com>

* test(ci): make the #6634 selfref guard hermetic — main's copy hard-fails every PR (#7341)

main's copy of this test still does git I/O inside a unit test:

    const baseSrc = git(['show', 'origin/main:' + FILE]);

Runners check out a shallow single ref, so origin/main does not resolve and the
test dies with 'fatal: invalid object name origin/main'. Every PR into main
fails Unit Tests (7/8) on it — today that is #7313, #7315, #7316, #7334, #7336
and #7337, six PRs red on a defect none of them introduced. #7313 has no other
red at all.

release/v3.8.49 already carries a fix (2e42b8efc, #7174: try/catch, fetch
origin/main on demand, t.skip() when unreachable), but it only reaches main at
release time — so main stays broken for the whole cycle. Cherry-picking it would
also import a new problem: PR Test Policy classifies t.skip() as a silenced
assertion, which we watched it correctly catch on #7300 today.

This is the hermetic version instead (ported from #7327, which does the same for
the release branch): read the file straight off disk, compare against an empty
base so baseTaut/baseExtTaut are 0 — the strictest possible comparison point —
and call evaluateMasking() directly. No git ref, no fetch, no skip, nothing the
runner's checkout depth can break.

The #6634 regression stays covered: the guard's logic lives in
SELF_TEST_FIXTURE_RE (check-test-masking.mjs:337), not in the test. Proven both
ways on main before committing — neutralise SELF_TEST_FIXTURE_RE to /$^/ and
the test FAILS; restore it and it passes 2/2, with check-test-masking.mjs left
byte-identical.

Co-authored-by: growab <nekron@icloud.com>

* chore(quality): tighten main's coverage baseline to the CI's real numbers (#7347)

main's ratchet had been failing --require-tighten on every PR: 11 metrics
improved but the baseline was never tightened. Same class as the #6634
selfref guard — an infra fix that lands only on the release branch leaves
main red for the whole cycle, and every PR into main pays for it.

Values are the merged-coverage numbers from a run on main itself (a local
run measures ~68% vs CI's ~80%; the baseline's own note warns about that
gap). Only the 11 coverage values change — gitleaks and semgrepFindings
keep main's own state.

No changelog fragment: #7326 carries it on release/v3.8.49, and a second
one here would double the entry at release time.

---------

Co-authored-by: Diego Rodrigues de Sa e Souza <8016841+diegosouzapw@users.noreply.github.com>
Co-authored-by: diegosouzapw <diegosouza.pw@gmail.com>
2026-07-23 11:15:18 -03:00

929 lines
32 KiB
TypeScript

import { NextResponse } from "next/server";
import { getProviderById } from "@/shared/constants/providers";
import { requireManagementAuth } from "@/lib/api/requireManagementAuth";
import { getApiKeys } from "@/lib/db/apiKeys";
import { getUserDatabaseSettings } from "@/lib/db/databaseSettings";
import {
buildUnifiedSource,
buildPresetUnifiedSource,
getUsageSummary,
getDailyUsage,
getDailyCostRows,
getHeatmapRows,
getModelUsageRows,
getProviderCostRows,
getProviderUsageRows,
getAccountCostRows,
getAccountUsageRows,
getApiKeyUsageRows,
getServiceTierUsageRows,
getApiKeyMetadataRows,
getWeeklyPatternRows,
getPresetCostModelRows,
} from "@/lib/db/usageAnalytics";
import { getFallbackStats } from "@/lib/db/callLogStats";
import { buildByProviderRows } from "@/lib/usage/providerDisplayNames";
import { toNumber } from "@/shared/utils/numeric";
function getRangeStartIso(range: string): string | null {
const end = new Date();
const start = new Date(end);
switch (range) {
case "1d":
start.setDate(start.getDate() - 1);
break;
case "7d":
start.setDate(start.getDate() - 7);
break;
case "30d":
start.setDate(start.getDate() - 30);
break;
case "90d":
start.setDate(start.getDate() - 90);
break;
case "180d":
start.setDate(start.getDate() - 180);
break;
case "365d":
start.setDate(start.getDate() - 365);
break;
case "ytd":
start.setMonth(0, 1);
start.setHours(0, 0, 0, 0);
break;
case "all":
default:
return null;
}
return start.toISOString();
}
const WEEKDAY_LABELS = ["Sun", "Mon", "Tue", "Wed", "Thu", "Fri", "Sat"];
type PricingByProvider = Record<string, Record<string, Record<string, unknown>>>;
type UsageRows = Array<Record<string, unknown>>;
type ComputeCostFromPricing = (
pricing: Record<string, unknown> | null | undefined,
tokens: Record<string, number | undefined> | null | undefined,
options?: Record<string, unknown>
) => number;
type GetCodexFastCostMultiplier = (
provider: string | null | undefined,
model: string | null | undefined,
serviceTier: string | null | undefined
) => number;
function toStringValue(value: unknown, fallback = ""): string {
return typeof value === "string" && value.trim().length > 0 ? value : fallback;
}
function roundCost(value: number): number {
return Math.round(value * 1_000_000) / 1_000_000;
}
function normalizeServiceTier(value: unknown): "standard" | "priority" | "flex" {
const tier = typeof value === "string" ? value.trim().toLowerCase() : "";
if (tier === "priority" || tier === "fast") return "priority";
if (tier === "flex") return "flex";
return "standard";
}
function getServiceTierLabelId(serviceTier: string): string {
return normalizeServiceTier(serviceTier);
}
function appendWhereCondition(whereClause: string, condition: string): string {
return whereClause ? `${whereClause} AND (${condition})` : `WHERE (${condition})`;
}
function findKeyInsensitive(obj: Record<string, any> | undefined | null, key: string): any {
if (!obj || !key) return undefined;
return obj[key.toLowerCase()];
}
function uniqueValues(values: Array<string | null | undefined>): string[] {
const seen = new Set<string>();
const result: string[] = [];
for (const value of values) {
const normalized = typeof value === "string" ? value.trim().toLowerCase() : "";
if (!normalized || seen.has(normalized)) continue;
seen.add(normalized);
result.push(normalized);
}
return result;
}
function makeApiKeyUsageGroup(apiKeyId: string, fallbackName: string): string {
return apiKeyId ? `id:${apiKeyId}` : `name:${fallbackName}`;
}
function addApiKeyAlias(target: Set<string>, value: unknown): void {
if (typeof value !== "string") return;
const trimmed = value.trim();
if (trimmed) target.add(trimmed);
}
function stripCodexEffortSuffix(model: string): string {
return model.replace(/-(?:xhigh|high|medium|low|none)$/i, "");
}
function getPricingModelCandidates(
model: string,
normalizeModelName: (model: string) => string
): string[] {
const normalizedModel = normalizeModelName(model);
const lowerModel = model.toLowerCase();
const lowerNormalized = normalizedModel.toLowerCase();
const hyphenModel = lowerModel.replace(/\./g, "-");
const hyphenNormalized = lowerNormalized.replace(/\./g, "-");
const effortBaseModel = stripCodexEffortSuffix(lowerNormalized);
return uniqueValues([
lowerModel,
lowerNormalized,
hyphenModel,
hyphenNormalized,
effortBaseModel,
effortBaseModel.replace(/\./g, "-"),
lowerNormalized === "codex-auto-review" ? "gpt-5.5" : null,
]);
}
function resolveModelPricing(
pricingByProvider: PricingByProvider,
providerAliasMap: Record<string, string>,
providerRaw: string,
model: string,
normalizeModelName: (model: string) => string
): Record<string, unknown> | null {
const pLower = (providerRaw || "").toLowerCase();
let providerPricing = findKeyInsensitive(pricingByProvider, pLower);
if (!providerPricing) {
// providerAliasMap maps ID -> ALIAS. So if pLower is "codex", alias is "cx".
const alias = providerAliasMap[pLower];
if (alias) {
providerPricing = findKeyInsensitive(pricingByProvider, alias);
}
}
if (!providerPricing) {
// In case pLower was ALIAS and we want to try the ID (reverse search values)
for (const [id, alias] of Object.entries(providerAliasMap)) {
if (alias.toLowerCase() === pLower) {
providerPricing = findKeyInsensitive(pricingByProvider, id);
if (providerPricing) break;
}
}
}
if (!providerPricing) {
const np = pLower.replace(/-cn$/, "");
if (np && np !== pLower) {
providerPricing = findKeyInsensitive(pricingByProvider, np);
}
}
// Hardcoded known fallbacks
if (!providerPricing) {
if (pLower === "antigravity") providerPricing = findKeyInsensitive(pricingByProvider, "ag");
}
const modelCandidates = getPricingModelCandidates(model, normalizeModelName);
const tryFind = (prov: Record<string, unknown> | null | undefined) => {
if (!prov || typeof prov !== "object") return null;
for (const candidate of modelCandidates) {
const pricing = findKeyInsensitive(prov as Record<string, unknown>, candidate);
if (pricing) return pricing;
}
return null;
};
let pricing = providerPricing ? tryFind(providerPricing) : null;
if (!pricing) {
// Global fallback: search all providers for this exact model (helps with aliases)
for (const prov of Object.values(pricingByProvider)) {
const found = tryFind(prov as Record<string, unknown>);
if (found) {
pricing = found;
break;
}
}
}
// Last resort fallback for historical usage (e.g. "gpt-4" missing, matches "gpt-4.1" or first available)
if (!pricing && providerPricing && typeof providerPricing === "object") {
for (const [key, val] of Object.entries(providerPricing as Record<string, unknown>)) {
const lm = model.toLowerCase();
if (key.includes(lm) || lm.includes(key)) {
pricing = val;
break;
}
}
if (!pricing) {
const keys = Object.keys(providerPricing as Record<string, unknown>);
if (keys.length > 0) pricing = (providerPricing as Record<string, unknown>)[keys[0]];
}
}
return pricing as Record<string, unknown> | null;
}
function computeUsageRowCost(
row: Record<string, unknown>,
pricingByProvider: PricingByProvider,
providerAliasMap: Record<string, string>,
normalizeModelName: (model: string) => string,
computeCostFromPricing: ComputeCostFromPricing
): number {
const provider = toStringValue(row.provider);
const model = toStringValue(row.model);
if (!provider || !model) return 0;
const serviceTier = normalizeServiceTier(row.serviceTier ?? row.service_tier);
const pricing = resolveModelPricing(
pricingByProvider,
providerAliasMap,
provider,
model,
normalizeModelName
);
if (!pricing) return 0;
return computeCostFromPricing(
pricing,
{
input: toNumber(row.promptTokens),
output: toNumber(row.completionTokens),
cacheRead: toNumber(row.cacheReadTokens),
cacheCreation: toNumber(row.cacheCreationTokens),
reasoning: toNumber(row.reasoningTokens),
},
{
provider,
model,
serviceTier,
flatRateAsZero: true,
}
);
}
function computeUsageRowStandardCost(
row: Record<string, unknown>,
pricingByProvider: PricingByProvider,
providerAliasMap: Record<string, string>,
normalizeModelName: (model: string) => string,
computeCostFromPricing: ComputeCostFromPricing
): number {
return computeUsageRowCost(
{ ...row, serviceTier: "standard", service_tier: "standard" },
pricingByProvider,
providerAliasMap,
normalizeModelName,
computeCostFromPricing
);
}
function computeUsageSavingsTokens(
row: Record<string, unknown>,
serviceTier: string,
getCodexFastCostMultiplier: GetCodexFastCostMultiplier
): number {
const provider = toStringValue(row.provider);
const model = toStringValue(row.model);
const totalTokens = toNumber(row.totalTokens);
if (!provider || !model || totalTokens <= 0) return 0;
const standardMultiplier = getCodexFastCostMultiplier(provider, model, "standard");
if (standardMultiplier <= 0) return 0;
const actualMultiplier = getCodexFastCostMultiplier(provider, model, serviceTier);
const savingsRatio = Math.max(0, (standardMultiplier - actualMultiplier) / standardMultiplier);
return totalTokens * savingsRatio;
}
function formatUtcDate(date: Date): string {
return date.toISOString().slice(0, 10);
}
function computeActivityStreak(activityMap: Record<string, number>): number {
const cursor = new Date();
let streak = 0;
while ((activityMap[formatUtcDate(cursor)] || 0) > 0) {
streak += 1;
cursor.setUTCDate(cursor.getUTCDate() - 1);
}
return streak;
}
export async function GET(request: Request) {
const authError = await requireManagementAuth(request);
if (authError) return authError;
try {
const { searchParams } = new URL(request.url);
const range = searchParams.get("range") || "30d";
const startDate = searchParams.get("startDate") || undefined;
const endDate = searchParams.get("endDate") || undefined;
const apiKeyIdsParam = searchParams.get("apiKeyIds") || "";
const apiKeyIds = apiKeyIdsParam ? apiKeyIdsParam.split(",").filter(Boolean) : [];
const sinceIso = startDate || getRangeStartIso(range);
const untilIso = endDate || null;
const presetsParam = searchParams.get("presets");
const apiKeys = await getApiKeys();
const currentApiKeyNames = new Map<string, string>();
for (const apiKey of apiKeys) {
if (typeof apiKey.id === "string" && typeof apiKey.name === "string") {
currentApiKeyNames.set(apiKey.id, apiKey.name);
}
}
// Raw-data cutoff: must match cleanupUsageHistory's rollup/delete boundary —
// retention.usageHistory (src/lib/db/cleanup.ts), NOT aggregation.rawDataRetentionDays.
const dbSettings = getUserDatabaseSettings();
const rawRetentionDays = dbSettings.retention?.usageHistory ?? 30;
const rawCutoff = new Date();
rawCutoff.setDate(rawCutoff.getDate() - rawRetentionDays);
const rawCutoffIso = rawCutoff.toISOString();
const conditions = [];
const params: Record<string, string> = {};
if (sinceIso) {
conditions.push("timestamp >= @since");
params.since = sinceIso;
}
if (untilIso) {
conditions.push("timestamp <= @until");
params.until = untilIso;
}
let apiKeyWhere = "";
if (apiKeyIds.length > 0) {
const placeholders = apiKeyIds.map((_, i) => `@apiKey${i}`);
apiKeyIds.forEach((key, i) => {
params[`apiKey${i}`] = key;
});
apiKeyWhere = `(api_key_name IN (${placeholders.join(",")}) OR api_key_id IN (${placeholders.join(",")}))`;
conditions.push(apiKeyWhere);
}
const whereClause = conditions.length > 0 ? `WHERE ${conditions.join(" AND ")}` : "";
// Build the UNION data source that merges recent raw rows with older aggregated history.
// SQL is encapsulated in usageAnalytics.ts — the route only supplies filter parameters.
const rawCutoffDate = rawCutoffIso.split("T")[0];
const apiKeyParamEntries: Record<string, string> = {};
apiKeyIds.forEach((key, i) => {
apiKeyParamEntries[`apiKey${i}`] = key;
});
const { unifiedSource, unifiedParams } = buildUnifiedSource({
sinceIso: sinceIso ?? null,
untilIso: untilIso ?? null,
rawCutoffDate,
apiKeyWhere,
apiKeyParams: apiKeyParamEntries,
});
// Fetch pricing data for cost calculation (no rows loaded)
const { getPricing } = await import("@/lib/db/settings");
const rawPricingByProvider = (await getPricing()) as PricingByProvider;
// Pre-process pricing data to lowercase keys for O(1) lookups
const pricingByProvider: PricingByProvider = {};
for (const [providerKey, providerVal] of Object.entries(rawPricingByProvider || {})) {
const lowerProvider = {};
for (const [modelKey, modelVal] of Object.entries(providerVal || {})) {
(lowerProvider as any)[modelKey.toLowerCase()] = modelVal;
}
pricingByProvider[providerKey.toLowerCase()] = lowerProvider;
}
const { computeCostFromPricing, getCodexFastCostMultiplier, normalizeModelName } =
await import("@/lib/usage/costCalculator");
const { PROVIDER_ID_TO_ALIAS } = await import("@omniroute/open-sse/config/providerModels");
const summaryRow = getUsageSummary(unifiedSource, unifiedParams) as Record<string, unknown>;
const dailyRows = getDailyUsage(unifiedSource, unifiedParams) as UsageRows;
const dailyCostRows = getDailyCostRows(unifiedSource, unifiedParams) as UsageRows;
const heatmapStart = new Date();
heatmapStart.setUTCDate(heatmapStart.getUTCDate() - 364);
// Custom date range might need a wider heatmap window
if (startDate) {
const customStart = new Date(startDate);
if (customStart.getTime() < heatmapStart.getTime()) {
heatmapStart.setTime(customStart.getTime());
}
}
// Heatmap needs its own whereClause if api keys are filtered
const heatmapConditions = ["timestamp >= @heatmapStart"];
if (apiKeyWhere) heatmapConditions.push(apiKeyWhere);
const heatmapParams: Record<string, string> = { heatmapStart: heatmapStart.toISOString() };
if (apiKeyIds.length > 0) {
apiKeyIds.forEach((key, i) => {
heatmapParams[`apiKey${i}`] = key;
});
}
const heatmapRows = getHeatmapRows(heatmapConditions, heatmapParams) as UsageRows;
const modelRows = getModelUsageRows(unifiedSource, unifiedParams) as UsageRows;
const providerCostRows = getProviderCostRows(unifiedSource, unifiedParams) as UsageRows;
const providerRows = getProviderUsageRows(unifiedSource, unifiedParams) as UsageRows;
const accountCostWhereClause = whereClause
.replace(/timestamp/g, "usage_history.timestamp")
.replace(/api_key_/g, "usage_history.api_key_");
const accountCostRows = getAccountCostRows(accountCostWhereClause, params) as UsageRows;
const accountRows = getAccountUsageRows(accountCostWhereClause, params) as UsageRows;
const apiKeyWhereClause = appendWhereCondition(
whereClause,
"(api_key_id IS NOT NULL AND api_key_id != '') OR (api_key_name IS NOT NULL AND api_key_name != '')"
);
const apiKeyRows = getApiKeyUsageRows(apiKeyWhereClause, params) as UsageRows;
const serviceTierRows = getServiceTierUsageRows(unifiedSource, unifiedParams) as UsageRows;
const apiKeyMetadataRows = getApiKeyMetadataRows(apiKeyWhereClause, params) as UsageRows;
const apiKeyMetadata = new Map<string, { latestName: string; aliases: Set<string> }>();
for (const row of apiKeyMetadataRows) {
const apiKeyId = toStringValue(row.apiKeyId);
const apiKeyGroupKey = toStringValue(row.apiKeyGroupKey, "unknown");
const groupKey = makeApiKeyUsageGroup(apiKeyId, apiKeyGroupKey);
const existing = apiKeyMetadata.get(groupKey) || {
latestName: "",
aliases: new Set<string>(),
};
const apiKeyName = toStringValue(row.apiKeyName);
if (!existing.latestName && apiKeyName) existing.latestName = apiKeyName;
addApiKeyAlias(existing.aliases, apiKeyName);
apiKeyMetadata.set(groupKey, existing);
}
const weeklyRows = getWeeklyPatternRows(unifiedSource, unifiedParams) as UsageRows;
const fallbackRow = getFallbackStats(whereClause, params) as Record<string, unknown>;
const summary = {
totalRequests: Number(summaryRow?.totalRequests || 0),
promptTokens: Number(summaryRow?.promptTokens || 0),
completionTokens: Number(summaryRow?.completionTokens || 0),
totalTokens: Number(summaryRow?.totalTokens || 0),
uniqueModels: Number(summaryRow?.uniqueModels || 0),
uniqueAccounts: Number(summaryRow?.uniqueAccounts || 0),
uniqueApiKeys: Number(summaryRow?.uniqueApiKeys || 0),
successfulRequests: Number(summaryRow?.successfulRequests || 0),
successRatePct:
Number(summaryRow?.totalRequests || 0) > 0
? Number(
(
(Number(summaryRow?.successfulRequests || 0) /
Number(summaryRow?.totalRequests || 1)) *
100
).toFixed(2)
)
: 0,
avgLatencyMs: Math.round(Number(summaryRow?.avgLatencyMs || 0)),
totalCost: 0,
firstRequest: summaryRow?.firstRequest || "",
lastRequest: summaryRow?.lastRequest || "",
fallbackCount: Number(fallbackRow?.fallbacks || 0),
fastRequests: 0,
standardRequests: 0,
flexRequests: 0,
fastCost: 0,
standardCost: 0,
flexCost: 0,
flexSavings: 0,
flexUsageSavingsTokens: 0,
fastRequestSharePct: 0,
fallbackRatePct:
Number(fallbackRow?.fallback_eligible || 0) > 0
? Number(
(
(Number(fallbackRow?.fallbacks || 0) /
Number(fallbackRow?.fallback_eligible || 1)) *
100
).toFixed(2)
)
: 0,
requestedModelCoveragePct:
Number(fallbackRow?.total || 0) > 0
? Number(
(
(Number(fallbackRow?.with_requested || 0) / Number(fallbackRow?.total || 1)) *
100
).toFixed(2)
)
: 0,
streak: 0,
};
const dailyByModelMap: Record<string, Record<string, number>> = {};
const allModels = new Set<string>();
const dailyCostByDate = new Map<string, number>();
for (const row of dailyCostRows) {
const date = toStringValue(row.date);
if (!date) continue;
// Calculate costs
const cost = computeUsageRowCost(
row,
pricingByProvider,
PROVIDER_ID_TO_ALIAS,
normalizeModelName,
computeCostFromPricing
);
dailyCostByDate.set(date, (dailyCostByDate.get(date) || 0) + cost);
// Group tokens by model for the day
const model = normalizeModelName(row.model as string);
const tokens = Number(row.promptTokens) + Number(row.completionTokens);
if (!dailyByModelMap[date]) dailyByModelMap[date] = {};
dailyByModelMap[date][model] = (dailyByModelMap[date][model] || 0) + tokens;
allModels.add(model);
}
const dailyTrend = dailyRows.map((row) => ({
date: row.date,
requests: Number(row.requests),
promptTokens: Number(row.promptTokens),
completionTokens: Number(row.completionTokens),
totalTokens: Number(row.totalTokens),
cost: roundCost(dailyCostByDate.get(toStringValue(row.date)) || 0),
}));
const activityMap: Record<string, number> = {};
for (const row of heatmapRows) {
activityMap[row.date as string] = Number(row.totalTokens);
}
summary.streak = computeActivityStreak(activityMap);
const modelMap = new Map<string, Record<string, unknown>>();
for (const row of modelRows) {
const model = row.model as string;
const provider = row.provider as string;
const short = normalizeModelName(model);
const cost = computeUsageRowCost(
row,
pricingByProvider,
PROVIDER_ID_TO_ALIAS,
normalizeModelName,
computeCostFromPricing
);
const key = `${provider}::${short}`;
const existing = modelMap.get(key) || {
model: short,
provider,
rawModel: model,
requests: 0,
promptTokens: 0,
completionTokens: 0,
totalTokens: 0,
latencyWeightedTotal: 0,
successfulRequests: 0,
lastUsed: "",
cost: 0,
};
const requests = Number(row.requests) || 0;
existing.requests = Number(existing.requests || 0) + requests;
existing.promptTokens = Number(existing.promptTokens || 0) + Number(row.promptTokens || 0);
existing.completionTokens =
Number(existing.completionTokens || 0) + Number(row.completionTokens || 0);
existing.totalTokens = Number(existing.totalTokens || 0) + Number(row.totalTokens || 0);
existing.latencyWeightedTotal =
Number(existing.latencyWeightedTotal || 0) + Number(row.avgLatencyMs || 0) * requests;
existing.successfulRequests =
Number(existing.successfulRequests || 0) + Number(row.successfulRequests || 0);
if (!existing.lastUsed || String(row.lastUsed || "") > String(existing.lastUsed || "")) {
existing.lastUsed = row.lastUsed;
}
existing.cost = Number(existing.cost || 0) + cost;
modelMap.set(key, existing);
}
const byModel = Array.from(modelMap.values())
.map((row) => ({
model: row.model,
provider: row.provider,
rawModel: row.rawModel,
requests: Number(row.requests),
promptTokens: Number(row.promptTokens),
completionTokens: Number(row.completionTokens),
totalTokens: Number(row.totalTokens),
avgLatencyMs:
Number(row.requests) > 0
? Math.round(Number(row.latencyWeightedTotal || 0) / Number(row.requests))
: 0,
successRatePct:
Number(row.requests) > 0
? Number((Number(row.successfulRequests || 0) / Number(row.requests)) * 100).toFixed(2)
: 0,
lastUsed: row.lastUsed,
cost: roundCost(Number(row.cost || 0)),
}))
.sort((left, right) => Number(right.requests) - Number(left.requests))
.slice(0, 50);
const totalCost = Array.from(dailyCostByDate.values()).reduce((sum, cost) => sum + cost, 0);
summary.totalCost = roundCost(totalCost);
const providerCostByProvider = new Map<string, number>();
for (const row of providerCostRows) {
const provider = toStringValue(row.provider);
if (!provider) continue;
const cost = computeUsageRowCost(
row,
pricingByProvider,
PROVIDER_ID_TO_ALIAS,
normalizeModelName,
computeCostFromPricing
);
providerCostByProvider.set(provider, (providerCostByProvider.get(provider) || 0) + cost);
}
const byProvider = await buildByProviderRows(providerRows, providerCostByProvider);
const accountCostByAccount = new Map<string, number>();
for (const row of accountCostRows) {
const accountKey = toStringValue(row.accountKey, "unknown");
const cost = computeUsageRowCost(
row,
pricingByProvider,
PROVIDER_ID_TO_ALIAS,
normalizeModelName,
computeCostFromPricing
);
accountCostByAccount.set(accountKey, (accountCostByAccount.get(accountKey) || 0) + cost);
}
const byAccount = accountRows.map((row) => ({
account: toStringValue(row.account, "unknown"),
requests: Number(row.requests),
promptTokens: Number(row.promptTokens),
completionTokens: Number(row.completionTokens),
totalTokens: Number(row.totalTokens),
avgLatencyMs: Math.round(Number(row.avgLatencyMs)),
lastUsed: row.lastUsed,
cost: roundCost(accountCostByAccount.get(toStringValue(row.accountKey, "unknown")) || 0),
}));
const apiKeyMap = new Map<
string,
{
apiKey: string;
apiKeyId: string | null;
apiKeyName: string;
historicalApiKeyNames: string[];
requests: number;
promptTokens: number;
completionTokens: number;
totalTokens: number;
cost: number;
}
>();
for (const row of apiKeyRows) {
const apiKeyId = toStringValue(row.apiKeyId);
const apiKeyGroupKey = toStringValue(row.apiKeyGroupKey, "unknown");
const key = makeApiKeyUsageGroup(apiKeyId, apiKeyGroupKey);
const metadata = apiKeyMetadata.get(key);
const apiKeyName =
(apiKeyId ? currentApiKeyNames.get(apiKeyId) : undefined) ||
metadata?.latestName ||
apiKeyId ||
apiKeyGroupKey ||
"Unknown API key";
const existing = apiKeyMap.get(key) || {
apiKey: apiKeyId && apiKeyName !== apiKeyId ? `${apiKeyName} (${apiKeyId})` : apiKeyName,
apiKeyId: apiKeyId || null,
apiKeyName,
historicalApiKeyNames: Array.from(metadata?.aliases || []),
requests: 0,
promptTokens: 0,
completionTokens: 0,
totalTokens: 0,
cost: 0,
};
existing.requests += Number(row.requests);
existing.promptTokens += Number(row.promptTokens);
existing.completionTokens += Number(row.completionTokens);
existing.totalTokens += Number(row.totalTokens);
existing.cost += computeUsageRowCost(
row,
pricingByProvider,
PROVIDER_ID_TO_ALIAS,
normalizeModelName,
computeCostFromPricing
);
apiKeyMap.set(key, existing);
}
const byApiKey = Array.from(apiKeyMap.values())
.map((row) => ({ ...row, cost: roundCost(row.cost) }))
.sort((left, right) => right.cost - left.cost);
const serviceTierMap = new Map<
string,
{
serviceTier: "standard" | "priority" | "flex";
label: string;
requests: number;
promptTokens: number;
completionTokens: number;
totalTokens: number;
cost: number;
savings: number;
usageSavingsTokens: number;
}
>();
for (const row of serviceTierRows) {
const serviceTier = normalizeServiceTier(row.serviceTier);
const existing = serviceTierMap.get(serviceTier) || {
serviceTier,
label: getServiceTierLabelId(serviceTier),
requests: 0,
promptTokens: 0,
completionTokens: 0,
totalTokens: 0,
cost: 0,
savings: 0,
usageSavingsTokens: 0,
};
existing.requests += Number(row.requests || 0);
existing.promptTokens += Number(row.promptTokens || 0);
existing.completionTokens += Number(row.completionTokens || 0);
existing.totalTokens += Number(row.totalTokens || 0);
const actualCost = computeUsageRowCost(
row,
pricingByProvider,
PROVIDER_ID_TO_ALIAS,
normalizeModelName,
computeCostFromPricing
);
existing.cost += actualCost;
if (serviceTier === "flex") {
const standardCost = computeUsageRowStandardCost(
row,
pricingByProvider,
PROVIDER_ID_TO_ALIAS,
normalizeModelName,
computeCostFromPricing
);
existing.savings += Math.max(0, standardCost - actualCost);
existing.usageSavingsTokens += computeUsageSavingsTokens(
row,
serviceTier,
getCodexFastCostMultiplier
);
}
serviceTierMap.set(serviceTier, existing);
}
const byServiceTier = Array.from(serviceTierMap.values())
.map((row) => ({
...row,
cost: roundCost(row.cost),
savings: roundCost(row.savings),
usageSavingsTokens: Math.round(row.usageSavingsTokens),
}))
.sort((left, right) => {
const order = { priority: 0, flex: 1, standard: 2 } as const;
return order[left.serviceTier] - order[right.serviceTier];
});
const fastTier = serviceTierMap.get("priority");
const flexTier = serviceTierMap.get("flex");
const standardTier = serviceTierMap.get("standard");
summary.fastRequests = fastTier?.requests || 0;
summary.fastCost = roundCost(fastTier?.cost || 0);
summary.flexRequests = flexTier?.requests || 0;
summary.flexCost = roundCost(flexTier?.cost || 0);
summary.flexSavings = roundCost(flexTier?.savings || 0);
summary.flexUsageSavingsTokens = Math.round(flexTier?.usageSavingsTokens || 0);
summary.standardRequests = standardTier?.requests || 0;
summary.standardCost = roundCost(standardTier?.cost || 0);
summary.fastRequestSharePct =
summary.totalRequests > 0
? Number(((Number(summary.fastRequests) / Number(summary.totalRequests)) * 100).toFixed(2))
: 0;
const weeklyTokens = [0, 0, 0, 0, 0, 0, 0];
const weeklyCounts = [0, 0, 0, 0, 0, 0, 0];
const weeklyPattern = WEEKDAY_LABELS.map((day) => ({
day,
avgTokens: 0,
totalTokens: 0,
}));
for (const row of weeklyRows) {
const dayIdx = Number(row.dayOfWeek);
if (dayIdx >= 0 && dayIdx <= 6) {
const totalTokens = Number(row.totalTokens);
const days = Number(row.days);
weeklyTokens[dayIdx] = totalTokens;
weeklyCounts[dayIdx] = Number(row.requests);
weeklyPattern[dayIdx] = {
day: WEEKDAY_LABELS[dayIdx],
avgTokens: days > 0 ? Math.round(totalTokens / days) : 0,
totalTokens,
};
}
}
const dailyByModel = Object.keys(dailyByModelMap)
.sort()
.map((date) => ({ date, ...dailyByModelMap[date] }));
const modelNames = Array.from(allModels);
const analytics = {
summary,
dailyTrend,
activityMap,
byModel,
byProvider,
byApiKey,
byAccount,
byServiceTier,
weeklyPattern,
weeklyTokens,
weeklyCounts,
dailyByModel,
modelNames,
range,
} as any;
if (presetsParam) {
const allowedRanges = new Set(["1d", "7d", "30d", "90d", "ytd", "all"]);
const presetRanges = presetsParam
.split(",")
.map((preset) => preset.trim())
.filter((preset) => allowedRanges.has(preset));
const presetSummaries: Record<string, { totalCost: number }> = {};
for (const presetRange of presetRanges) {
if (presetRange === range) {
presetSummaries[presetRange] = {
totalCost: Number(analytics.summary?.totalCost || 0),
};
continue;
}
const presetSinceIso = getRangeStartIso(presetRange);
const { unifiedSource: pSrc, unifiedParams: pParams } = buildPresetUnifiedSource({
sinceIso: presetSinceIso ?? null,
untilIso: null,
rawCutoffDate,
apiKeyWhere,
apiKeyParams: apiKeyParamEntries,
});
const presetModelRows = getPresetCostModelRows(pSrc, pParams) as UsageRows;
let presetTotalCost = 0;
for (const row of presetModelRows) {
presetTotalCost += computeUsageRowCost(
row,
pricingByProvider,
PROVIDER_ID_TO_ALIAS,
normalizeModelName,
computeCostFromPricing
);
}
presetSummaries[presetRange] = {
totalCost: roundCost(presetTotalCost),
};
}
analytics.presetSummaries = presetSummaries;
}
return NextResponse.json(analytics);
} catch (error) {
console.error("Error computing analytics:", error);
// Surface the real (sanitized) reason so the dashboard can show it instead of a
// generic placeholder (#3356). buildErrorBody strips stacks/absolute paths.
const { buildErrorBody } = await import("@omniroute/open-sse/utils/error");
const message = error instanceof Error ? error.message : String(error);
return NextResponse.json(buildErrorBody(500, message || "Failed to compute analytics"), {
status: 500,
});
}
}