mirror of
https://github.com/diegosouzapw/OmniRoute.git
synced 2026-07-26 09:52:11 +03:00
test: hermetic notion thread-session mocks + drop duplicated usage-analytics suite (#8392)
* test: hermetic TLS mock for notion thread-session suite + drop duplicated usage-analytics file Root cause A (#8159): sendNotionInferenceRequest() in open-sse/executors/notion-web.ts was migrated from fetch() to tlsFetchNotion() (open-sse/services/notionTlsClient.ts, native tls-client-node binary) to get past Notion's Cloudflare TLS fingerprinting. #8159 updated the mock in the sibling tests/unit/executor-notion-web.test.ts (installNotionTlsMock, wired through __setTlsFetchOverrideForTesting) but never touched tests/unit/executor-notion-web-thread-sessions.test.ts (split out earlier by #7900) — its 3 execute()-driven tests still mocked globalThis.fetch, which tlsFetchNotion() never calls once the native TLS client loads successfully. Confirmed live: all 3 tests hit real https://app.notion.com with a fake cookie and got a real 401 (~8.1-8.5s each here; on a network with blocked/slow egress this would instead hang up to the client's ~190s timeout+grace per test — a CI-hang risk). Fix: replicate installNotionTlsMock verbatim from the sibling file into executor-notion-web-thread-sessions.test.ts so the 3 tests mock the TLS override point instead of global fetch. Suite is now fully hermetic — 8/8 pass, no network I/O, total runtime 31.0s -> 8.0s. Root cause B (#7700): tests/unit/usage-analytics-route-extra.test.ts was created as a byte-for-byte duplicate of 10 of the 22 tests in tests/unit/usage-analytics-route.test.ts. #7300 later fixed a fixture bug in the retention-window boundary test ("does not double-count raw and aggregated rows") in the main file only — reading getUserDatabaseSettings().retention.usageHistory live instead of a hardcoded 30-day cutoff (default retention is 365 days) — leaving the duplicate copy on the stale hardcoded value, which now fails (1 !== 2). Fix: delete the duplicate file. All 10 of its test names exist verbatim in the main file (verified with comm -12) and that file passes 22/22: - does not double-count raw and aggregated rows - does not persist guessed API key attribution - does not throw Unknown named parameter on short range (needsAggregated=false) - does not throw Unknown named parameter with apiKey filter on long range - groups renamed API key usage by stable ID - includes activityMap for heatmap - includes cost by API key - omits global aggregates when filtering by API key - returns 500 on database errors - returns weeklyPattern for the costs dashboard No coverage loss — same production code, same assertions, one fewer redundant file. Validation: - RED executor-notion-web-thread-sessions.test.ts: 5 pass / 3 fail (401 !== 200, real network hit), 31.0s - RED usage-analytics-route-extra.test.ts: 9 pass / 1 fail (1 !== 2), 18.7s - GREEN executor-notion-web-thread-sessions.test.ts: 8/8 pass, 8.0s, hermetic (no network) - GREEN executor-notion-web.test.ts (sibling, untouched): 37/37 pass, byte-identical diff - GREEN usage-analytics-route.test.ts (untouched): 22/22 pass, byte-identical diff - npx eslint on the changed file: clean - npm run typecheck:core: clean (exit 0) Refs #8159 Refs #7300 Refs #7700 * chore(quality): register usage-analytics-route-extra deletion in test-masking allowlist check:test-masking hard-flags any deleted test file without a _deletedWithReplacement entry. The deletion is legitimate (100% duplicate suite, coverage retained verbatim in tests/unit/usage-analytics-route.test.ts) -- same registration pattern as the video-dashscope entry. Refs #7700 Refs #7300
This commit is contained in:
committed by
GitHub
parent
fbea867d11
commit
1e0886cd09
@@ -40,6 +40,10 @@
|
||||
"tests/unit/video-dashscope.test.ts": {
|
||||
"replacement": "tests/unit/alibaba-video-media.test.ts",
|
||||
"reason": "v3.8.49 #8266 reorganized the Alibaba video catalog: the flat wan2.7-t2v id moved out of the plain \"alibaba\" provider (which now only exposes the dated wan2.7-t2v-2026-06-12) and lives only under \"qwen-cloud\" today. All 6 tests in the deleted file built requests against alibaba/wan2.7-t2v, which the new allowlist now rejects with HTTP 400 (verified directly against VIDEO_PROVIDERS in open-sse/config/videoRegistry.ts before deletion). Coverage for this exact id already exists and was confirmed green pre-deletion in tests/unit/alibaba-video-media.test.ts (including an explicit \"Alibaba rejects video models outside its own allowlist\" case for alibaba/wan2.7-t2v) and tests/unit/qwen-cloud-video-media.test.ts (covers the same flat id under qwen-cloud/wan2.7-t2v). Verified legitimate, not masking."
|
||||
},
|
||||
"tests/unit/usage-analytics-route-extra.test.ts": {
|
||||
"replacement": "tests/unit/usage-analytics-route.test.ts",
|
||||
"reason": "v3.8.49 base-red reconcile: the file was a 100% duplicate — all 10 of its test names exist verbatim (comm -12 verified) among the 22 in tests/unit/usage-analytics-route.test.ts, created by #7700 before #7300 fixed the retention-cutoff fixture only in the main file (reads getUserDatabaseSettings().retention.usageHistory live instead of hardcoding 30d). The stale copy red-failed 'does not double-count raw and aggregated rows' (1 !== 2) against correct production behavior. Zero unique coverage lost; the maintained main file passes 22/22. Verified legitimate, not masking."
|
||||
}
|
||||
},
|
||||
"tests/unit/catalog-updates-v3x.test.ts": "v3.8.45 #6248: fix(providers) remove deprecated MiMo V2 entries — os 5 asserts removidos pinavam specs de modelos mimo-v2-* que deixaram de existir no catálogo (54→49). Asserts seguem a remoção dos modelos, não enfraquecimento. Verificado legítimo. Prune após v3.8.45 mergear para main.",
|
||||
|
||||
@@ -6,9 +6,34 @@ import { describe, it } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
|
||||
const mod = await import("../../open-sse/executors/notion-web.ts");
|
||||
const { __setTlsFetchOverrideForTesting } = await import(
|
||||
"../../open-sse/services/notionTlsClient.ts"
|
||||
);
|
||||
|
||||
const COOKIE_WITH_SPACE = "token_v2=xyz; space_id=space-1; notion_user_id=user-1";
|
||||
|
||||
/** Mock the Chrome-JA3 path used by sendNotionInferenceRequest (not global fetch). */
|
||||
function installNotionTlsMock(
|
||||
handler: (url: string, opts: { headers?: Record<string, string>; body?: string }) => Promise<{
|
||||
status: number;
|
||||
text: string;
|
||||
}>
|
||||
): () => void {
|
||||
__setTlsFetchOverrideForTesting(async (url, options) => {
|
||||
const r = await handler(url, {
|
||||
headers: options.headers as Record<string, string> | undefined,
|
||||
body: options.body,
|
||||
});
|
||||
return {
|
||||
status: r.status,
|
||||
headers: new Headers(),
|
||||
text: r.text,
|
||||
body: null,
|
||||
};
|
||||
});
|
||||
return () => __setTlsFetchOverrideForTesting(null);
|
||||
}
|
||||
|
||||
describe("Notion thread session continuity", () => {
|
||||
const {
|
||||
__resetNotionThreadSessionsForTests,
|
||||
@@ -119,50 +144,48 @@ describe("Notion thread session continuity", () => {
|
||||
const executor = new NotionWebExecutor();
|
||||
const captured: Array<{ createThread?: boolean; threadId?: string }> = [];
|
||||
let n = 0;
|
||||
const originalFetch = globalThis.fetch;
|
||||
try {
|
||||
globalThis.fetch = (async (_url: string | URL, opts: RequestInit) => {
|
||||
const body = JSON.parse(String(opts.body)) as {
|
||||
createThread?: boolean;
|
||||
threadId?: string;
|
||||
const restore = installNotionTlsMock(async (_url, opts) => {
|
||||
const body = JSON.parse(String(opts.body)) as {
|
||||
createThread?: boolean;
|
||||
threadId?: string;
|
||||
};
|
||||
captured.push(body);
|
||||
n++;
|
||||
if (n === 1) {
|
||||
return {
|
||||
status: 200,
|
||||
text: JSON.stringify({
|
||||
id: "e1",
|
||||
type: "error",
|
||||
message: "Something went wrong. Please try again later.",
|
||||
subType: "temporarily-unavailable",
|
||||
isRetryable: false,
|
||||
}),
|
||||
};
|
||||
captured.push(body);
|
||||
n++;
|
||||
if (n === 1) {
|
||||
return new Response(
|
||||
JSON.stringify({
|
||||
id: "e1",
|
||||
type: "error",
|
||||
message: "Something went wrong. Please try again later.",
|
||||
subType: "temporarily-unavailable",
|
||||
isRetryable: false,
|
||||
}),
|
||||
{ status: 200 }
|
||||
);
|
||||
}
|
||||
const ndjson = [
|
||||
JSON.stringify({ type: "patch-start", data: { s: [] } }),
|
||||
JSON.stringify({
|
||||
type: "record-map",
|
||||
recordMap: {
|
||||
thread_message: {
|
||||
m1: {
|
||||
}
|
||||
const ndjson = [
|
||||
JSON.stringify({ type: "patch-start", data: { s: [] } }),
|
||||
JSON.stringify({
|
||||
type: "record-map",
|
||||
recordMap: {
|
||||
thread_message: {
|
||||
m1: {
|
||||
value: {
|
||||
value: {
|
||||
value: {
|
||||
step: {
|
||||
type: "agent-inference",
|
||||
value: [{ type: "text", content: "recovered" }],
|
||||
},
|
||||
step: {
|
||||
type: "agent-inference",
|
||||
value: [{ type: "text", content: "recovered" }],
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
}),
|
||||
].join("\n");
|
||||
return new Response(ndjson, { status: 200 });
|
||||
}) as typeof fetch;
|
||||
|
||||
},
|
||||
}),
|
||||
].join("\n");
|
||||
return { status: 200, text: ndjson };
|
||||
});
|
||||
try {
|
||||
const result = await executor.execute({
|
||||
model: "fable-5",
|
||||
body: { messages: turn1 },
|
||||
@@ -178,7 +201,7 @@ describe("Notion thread session continuity", () => {
|
||||
const json = (await result.response.json()) as { choices?: { message?: { content?: string } }[] };
|
||||
assert.match(String(json.choices?.[0]?.message?.content || ""), /recovered/);
|
||||
} finally {
|
||||
globalThis.fetch = originalFetch;
|
||||
restore();
|
||||
__resetNotionThreadSessionsForTests();
|
||||
}
|
||||
});
|
||||
@@ -206,33 +229,31 @@ describe("Notion thread session continuity", () => {
|
||||
__resetNotionThreadSessionsForTests();
|
||||
const executor = new mod.NotionWebExecutor();
|
||||
const captured: Array<{ createThread?: boolean; threadId?: string }> = [];
|
||||
const originalFetch = globalThis.fetch;
|
||||
try {
|
||||
globalThis.fetch = (async (_url: string | URL, opts: RequestInit) => {
|
||||
captured.push(JSON.parse(String(opts.body)));
|
||||
const ndjson = [
|
||||
JSON.stringify({ type: "patch-start", data: { s: [] } }),
|
||||
JSON.stringify({
|
||||
type: "record-map",
|
||||
recordMap: {
|
||||
thread_message: {
|
||||
m1: {
|
||||
const restore = installNotionTlsMock(async (_url, opts) => {
|
||||
captured.push(JSON.parse(String(opts.body)));
|
||||
const ndjson = [
|
||||
JSON.stringify({ type: "patch-start", data: { s: [] } }),
|
||||
JSON.stringify({
|
||||
type: "record-map",
|
||||
recordMap: {
|
||||
thread_message: {
|
||||
m1: {
|
||||
value: {
|
||||
value: {
|
||||
value: {
|
||||
step: {
|
||||
type: "agent-inference",
|
||||
value: [{ type: "text", content: "ok" }],
|
||||
},
|
||||
step: {
|
||||
type: "agent-inference",
|
||||
value: [{ type: "text", content: "ok" }],
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
}),
|
||||
].join("\n");
|
||||
return new Response(ndjson, { status: 200 });
|
||||
}) as typeof fetch;
|
||||
|
||||
},
|
||||
}),
|
||||
].join("\n");
|
||||
return { status: 200, text: ndjson };
|
||||
});
|
||||
try {
|
||||
const r1 = await executor.execute({
|
||||
model: "fable-5",
|
||||
body: { messages: [{ role: "user", content: "hello continuity" }] },
|
||||
@@ -265,7 +286,7 @@ describe("Notion thread session continuity", () => {
|
||||
assert.equal(captured[1].createThread, false);
|
||||
assert.equal(captured[1].threadId, t1);
|
||||
} finally {
|
||||
globalThis.fetch = originalFetch;
|
||||
restore();
|
||||
__resetNotionThreadSessionsForTests();
|
||||
}
|
||||
});
|
||||
@@ -276,38 +297,36 @@ describe("Notion thread session continuity", () => {
|
||||
const pinned = "aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee";
|
||||
let capturedCreateThread: boolean | undefined;
|
||||
let capturedThreadId: string | undefined;
|
||||
const originalFetch = globalThis.fetch;
|
||||
try {
|
||||
globalThis.fetch = (async (_url: string | URL, opts: RequestInit) => {
|
||||
const body = JSON.parse(String(opts.body)) as {
|
||||
createThread?: boolean;
|
||||
threadId?: string;
|
||||
};
|
||||
capturedCreateThread = body.createThread;
|
||||
capturedThreadId = body.threadId;
|
||||
const ndjson = [
|
||||
JSON.stringify({ type: "patch-start", data: { s: [] } }),
|
||||
JSON.stringify({
|
||||
type: "record-map",
|
||||
recordMap: {
|
||||
thread_message: {
|
||||
m1: {
|
||||
const restore = installNotionTlsMock(async (_url, opts) => {
|
||||
const body = JSON.parse(String(opts.body)) as {
|
||||
createThread?: boolean;
|
||||
threadId?: string;
|
||||
};
|
||||
capturedCreateThread = body.createThread;
|
||||
capturedThreadId = body.threadId;
|
||||
const ndjson = [
|
||||
JSON.stringify({ type: "patch-start", data: { s: [] } }),
|
||||
JSON.stringify({
|
||||
type: "record-map",
|
||||
recordMap: {
|
||||
thread_message: {
|
||||
m1: {
|
||||
value: {
|
||||
value: {
|
||||
value: {
|
||||
step: {
|
||||
type: "agent-inference",
|
||||
value: [{ type: "text", content: "ok" }],
|
||||
},
|
||||
step: {
|
||||
type: "agent-inference",
|
||||
value: [{ type: "text", content: "ok" }],
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
}),
|
||||
].join("\n");
|
||||
return new Response(ndjson, { status: 200 });
|
||||
}) as typeof fetch;
|
||||
|
||||
},
|
||||
}),
|
||||
].join("\n");
|
||||
return { status: 200, text: ndjson };
|
||||
});
|
||||
try {
|
||||
// Real ExecuteInput shape: clientHeaders only (headers is undefined).
|
||||
const result = await executor.execute({
|
||||
model: "fable-5",
|
||||
@@ -323,7 +342,7 @@ describe("Notion thread session continuity", () => {
|
||||
// Client-supplied thread id must force follow-up mode (createThread=false).
|
||||
assert.equal(capturedCreateThread, false);
|
||||
} finally {
|
||||
globalThis.fetch = originalFetch;
|
||||
restore();
|
||||
__resetNotionThreadSessionsForTests();
|
||||
}
|
||||
});
|
||||
|
||||
@@ -1,339 +0,0 @@
|
||||
import test from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import fs from "node:fs";
|
||||
import os from "node:os";
|
||||
import path from "node:path";
|
||||
|
||||
const TEST_DATA_DIR = fs.mkdtempSync(
|
||||
path.join(os.tmpdir(), "omniroute-usage-analytics-route-extra-")
|
||||
);
|
||||
process.env.DATA_DIR = TEST_DATA_DIR;
|
||||
const ORIGINAL_API_KEY_SECRET = process.env.API_KEY_SECRET;
|
||||
process.env.API_KEY_SECRET = "test-usage-analytics-secret";
|
||||
|
||||
const core = await import("../../src/lib/db/core.ts");
|
||||
const localDb = await import("../../src/lib/localDb.ts");
|
||||
const apiKeysDb = await import("../../src/lib/db/apiKeys.ts");
|
||||
const providersDb = await import("../../src/lib/db/providers.ts");
|
||||
const usageHistory = await import("../../src/lib/usage/usageHistory.ts");
|
||||
const analyticsRoute = await import("../../src/app/api/usage/analytics/route.ts");
|
||||
|
||||
const clearPendingRequests = usageHistory.clearPendingRequests;
|
||||
const EXPECTED_TOTAL_COST = 0.020925;
|
||||
|
||||
async function resetStorage() {
|
||||
core.resetDbInstance();
|
||||
apiKeysDb.resetApiKeyState();
|
||||
fs.rmSync(TEST_DATA_DIR, { recursive: true, force: true });
|
||||
fs.mkdirSync(TEST_DATA_DIR, { recursive: true });
|
||||
clearPendingRequests();
|
||||
}
|
||||
|
||||
async function seedAnalyticsData() {
|
||||
const db = core.getDbInstance();
|
||||
const now = new Date();
|
||||
for (let i = 0; i < 20; i++) {
|
||||
const timestamp = new Date(now.getTime() - i * 60 * 60 * 1000).toISOString();
|
||||
db.prepare(
|
||||
`INSERT INTO usage_history (provider, model, connection_id, api_key_id, api_key_name, tokens_input, tokens_output, success, latency_ms, timestamp)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`
|
||||
).run(
|
||||
i % 2 === 0 ? "openai" : "anthropic",
|
||||
i % 2 === 0 ? "gpt-4o" : "claude-sonnet",
|
||||
"test-conn",
|
||||
"test-key",
|
||||
"Primary Key",
|
||||
100 + i,
|
||||
50 + i,
|
||||
1,
|
||||
200 + i * 10,
|
||||
timestamp
|
||||
);
|
||||
}
|
||||
db.prepare(
|
||||
`INSERT INTO call_logs (provider, model, requested_model, connection_id, timestamp)
|
||||
VALUES (?, ?, ?, ?, ?)`
|
||||
).run("openai", "gpt-4o", "gpt-4o-mini", "test-conn", new Date().toISOString());
|
||||
}
|
||||
|
||||
function makeRequest(url: string) {
|
||||
return new Request(url, { method: "GET" });
|
||||
}
|
||||
|
||||
function assertClose(actual: number, expected: number, epsilon = 0.000001) {
|
||||
assert.ok(
|
||||
Math.abs(actual - expected) <= epsilon,
|
||||
`expected ${actual} to be within ${epsilon} of ${expected}`
|
||||
);
|
||||
}
|
||||
|
||||
test.beforeEach(async () => {
|
||||
await resetStorage();
|
||||
await localDb.updatePricing({
|
||||
openai: { "gpt-4o": { input: 2.5, output: 10 } },
|
||||
anthropic: { "claude-sonnet": { input: 3, output: 15 } },
|
||||
});
|
||||
});
|
||||
|
||||
test.after(() => {
|
||||
core.resetDbInstance();
|
||||
apiKeysDb.resetApiKeyState();
|
||||
fs.rmSync(TEST_DATA_DIR, { recursive: true, force: true });
|
||||
|
||||
if (ORIGINAL_API_KEY_SECRET === undefined) {
|
||||
delete process.env.API_KEY_SECRET;
|
||||
} else {
|
||||
process.env.API_KEY_SECRET = ORIGINAL_API_KEY_SECRET;
|
||||
}
|
||||
});
|
||||
test("GET /api/usage/analytics includes cost by API key", async () => {
|
||||
await seedAnalyticsData();
|
||||
|
||||
const response = await analyticsRoute.GET(makeRequest("http://localhost/api/usage/analytics"));
|
||||
const body = await response.json();
|
||||
|
||||
assert.equal(response.status, 200);
|
||||
assert.ok(Array.isArray(body.byApiKey));
|
||||
assert.equal(body.byApiKey.length, 1);
|
||||
assert.equal(body.byApiKey[0].apiKeyId, "test-key");
|
||||
assert.equal(body.byApiKey[0].apiKeyName, "Primary Key");
|
||||
assertClose(body.byApiKey[0].cost, body.summary.totalCost);
|
||||
});
|
||||
|
||||
test("GET /api/usage/analytics does not double-count raw and aggregated rows", async () => {
|
||||
const db = core.getDbInstance();
|
||||
const today = new Date();
|
||||
const todayStr = today.toISOString().split("T")[0];
|
||||
const cutoffDate = new Date();
|
||||
cutoffDate.setDate(cutoffDate.getDate() - 30);
|
||||
const olderDate = new Date(cutoffDate);
|
||||
olderDate.setDate(olderDate.getDate() - 1);
|
||||
const olderDateStr = olderDate.toISOString().split("T")[0];
|
||||
|
||||
db.prepare(
|
||||
`INSERT INTO usage_history (provider, model, connection_id, tokens_input, tokens_output, success, latency_ms, timestamp)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?)`
|
||||
).run("openai", "gpt-4o", "raw-current", 100, 50, 1, 200, today.toISOString());
|
||||
|
||||
const insertSummary = db.prepare(
|
||||
`INSERT INTO daily_usage_summary (provider, model, date, total_requests, total_input_tokens, total_output_tokens, total_cost)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?)`
|
||||
);
|
||||
insertSummary.run("openai", "gpt-4o", todayStr, 99, 9900, 9900, 0);
|
||||
insertSummary.run("openai", "gpt-4o", olderDateStr, 1, 25, 10, 0);
|
||||
|
||||
const response = await analyticsRoute.GET(
|
||||
makeRequest("http://localhost/api/usage/analytics?range=all")
|
||||
);
|
||||
const body = await response.json();
|
||||
|
||||
assert.equal(response.status, 200);
|
||||
assert.equal(body.summary.totalRequests, 2);
|
||||
assert.equal(body.summary.totalTokens, 185);
|
||||
assert.equal(body.summary.uniqueAccounts, 1);
|
||||
});
|
||||
|
||||
test("GET /api/usage/analytics omits global aggregates when filtering by API key", async () => {
|
||||
const apiKey = await apiKeysDb.createApiKey("Scoped Key", "machine1234567890");
|
||||
const db = core.getDbInstance();
|
||||
|
||||
db.prepare(
|
||||
`INSERT INTO usage_history (provider, model, connection_id, api_key_id, api_key_name, tokens_input, tokens_output, success, latency_ms, timestamp)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`
|
||||
).run(
|
||||
"openai",
|
||||
"gpt-4o",
|
||||
"scoped-conn",
|
||||
apiKey.id,
|
||||
"Scoped Key",
|
||||
100,
|
||||
50,
|
||||
1,
|
||||
200,
|
||||
new Date().toISOString()
|
||||
);
|
||||
|
||||
db.prepare(
|
||||
`INSERT INTO daily_usage_summary (provider, model, date, total_requests, total_input_tokens, total_output_tokens, total_cost)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?)`
|
||||
).run("openai", "gpt-4o", "2024-01-01", 99, 9900, 9900, 0);
|
||||
|
||||
const response = await analyticsRoute.GET(
|
||||
makeRequest(`http://localhost/api/usage/analytics?range=all&apiKeyIds=${apiKey.id}`)
|
||||
);
|
||||
const body = await response.json();
|
||||
|
||||
assert.equal(response.status, 200);
|
||||
assert.equal(body.summary.totalRequests, 1);
|
||||
assert.equal(body.summary.totalTokens, 150);
|
||||
assert.equal(body.byApiKey.length, 1);
|
||||
assert.equal(body.byApiKey[0].apiKeyId, apiKey.id);
|
||||
});
|
||||
|
||||
test("GET /api/usage/analytics groups renamed API key usage by stable ID", async () => {
|
||||
const apiKey = await apiKeysDb.createApiKey("Averyanov", "machine1234567890");
|
||||
await apiKeysDb.updateApiKeyPermissions(apiKey.id, { name: "Alexander Averyanov" });
|
||||
|
||||
const db = core.getDbInstance();
|
||||
const now = Date.now();
|
||||
const insertUsage = db.prepare(
|
||||
`INSERT INTO usage_history (provider, model, connection_id, api_key_id, api_key_name, tokens_input, tokens_output, success, latency_ms, timestamp)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`
|
||||
);
|
||||
insertUsage.run(
|
||||
"openai",
|
||||
"gpt-4o",
|
||||
"test-conn",
|
||||
apiKey.id,
|
||||
"Averyanov",
|
||||
100,
|
||||
50,
|
||||
1,
|
||||
200,
|
||||
new Date(now - 60_000).toISOString()
|
||||
);
|
||||
insertUsage.run(
|
||||
"openai",
|
||||
"gpt-4o",
|
||||
"test-conn",
|
||||
apiKey.id,
|
||||
"Desktop",
|
||||
200,
|
||||
100,
|
||||
1,
|
||||
250,
|
||||
new Date(now).toISOString()
|
||||
);
|
||||
|
||||
const response = await analyticsRoute.GET(makeRequest("http://localhost/api/usage/analytics"));
|
||||
const body = await response.json();
|
||||
|
||||
assert.equal(response.status, 200);
|
||||
assert.equal(body.summary.uniqueApiKeys, 1);
|
||||
assert.equal(body.byApiKey.length, 1);
|
||||
assert.equal(body.byApiKey[0].apiKeyId, apiKey.id);
|
||||
assert.equal(body.byApiKey[0].apiKeyName, "Alexander Averyanov");
|
||||
assert.deepEqual(body.byApiKey[0].historicalApiKeyNames.sort(), ["Averyanov", "Desktop"]);
|
||||
assert.equal(body.byApiKey[0].requests, 2);
|
||||
assert.equal(body.byApiKey[0].promptTokens, 300);
|
||||
assert.equal(body.byApiKey[0].completionTokens, 150);
|
||||
|
||||
const filteredResponse = await analyticsRoute.GET(
|
||||
makeRequest(`http://localhost/api/usage/analytics?apiKeyIds=${apiKey.id}`)
|
||||
);
|
||||
const filteredBody = await filteredResponse.json();
|
||||
|
||||
assert.equal(filteredResponse.status, 200);
|
||||
assert.equal(filteredBody.summary.totalRequests, 2);
|
||||
assert.equal(filteredBody.byApiKey.length, 1);
|
||||
assert.equal(filteredBody.byApiKey[0].apiKeyId, apiKey.id);
|
||||
});
|
||||
|
||||
test("GET /api/usage/analytics does not persist guessed API key attribution", async () => {
|
||||
await localDb.updatePricing({
|
||||
openai: { "gpt-4o": { input: 2.5, output: 10 } },
|
||||
});
|
||||
await apiKeysDb.createApiKey("Unrestricted Key", "machine1234567890");
|
||||
|
||||
const db = core.getDbInstance();
|
||||
db.prepare(
|
||||
`INSERT INTO usage_history (provider, model, connection_id, api_key_id, api_key_name, tokens_input, tokens_output, success, latency_ms, timestamp)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`
|
||||
).run("openai", "gpt-4o", "legacy-conn", null, null, 100, 50, 1, 200, new Date().toISOString());
|
||||
|
||||
const response = await analyticsRoute.GET(makeRequest("http://localhost/api/usage/analytics"));
|
||||
const body = await response.json();
|
||||
|
||||
assert.equal(response.status, 200);
|
||||
assert.equal(body.byApiKey.length, 0);
|
||||
|
||||
const row = db
|
||||
.prepare("SELECT api_key_id, api_key_name FROM usage_history WHERE connection_id = ?")
|
||||
.get("legacy-conn") as { api_key_id: string | null; api_key_name: string | null };
|
||||
assert.equal(row.api_key_id, null);
|
||||
assert.equal(row.api_key_name, null);
|
||||
});
|
||||
|
||||
test("GET /api/usage/analytics returns weeklyPattern for the costs dashboard", async () => {
|
||||
await seedAnalyticsData();
|
||||
|
||||
const response = await analyticsRoute.GET(makeRequest("http://localhost/api/usage/analytics"));
|
||||
const body = await response.json();
|
||||
|
||||
assert.equal(response.status, 200);
|
||||
assert.ok(Array.isArray(body.weeklyPattern));
|
||||
assert.equal(body.weeklyPattern.length, 7);
|
||||
assert.deepEqual(
|
||||
body.weeklyPattern.map((row) => row.day),
|
||||
["Sun", "Mon", "Tue", "Wed", "Thu", "Fri", "Sat"]
|
||||
);
|
||||
assert.ok(body.weeklyPattern.some((row) => row.totalTokens > 0 && row.avgTokens > 0));
|
||||
});
|
||||
|
||||
test("GET /api/usage/analytics includes activityMap for heatmap", async () => {
|
||||
await seedAnalyticsData();
|
||||
|
||||
const response = await analyticsRoute.GET(makeRequest("http://localhost/api/usage/analytics"));
|
||||
const body = await response.json();
|
||||
|
||||
assert.equal(response.status, 200);
|
||||
assert.ok(typeof body.activityMap === "object");
|
||||
assert.ok(Object.keys(body.activityMap).length > 0);
|
||||
});
|
||||
|
||||
test("GET /api/usage/analytics returns 500 on database errors", async () => {
|
||||
const response = await analyticsRoute.GET(makeRequest("http://localhost/api/usage/analytics"));
|
||||
const body = await response.json();
|
||||
|
||||
assert.equal(response.status, 200);
|
||||
assert.ok(body.summary.totalRequests === 0);
|
||||
});
|
||||
|
||||
test("GET /api/usage/analytics does not throw Unknown named parameter on short range (needsAggregated=false)", async () => {
|
||||
// Regression: shared params object leaked agg-only bindings (@sinceDate, @rawCutoffDate)
|
||||
// into queries that don't reference them, causing better-sqlite3 to throw.
|
||||
// A short range (1h) triggers needsAggregated=false because the entire window
|
||||
// falls within the raw-data-only period.
|
||||
const db = core.getDbInstance();
|
||||
const now = new Date();
|
||||
db.prepare(
|
||||
`INSERT INTO usage_history (provider, model, connection_id, tokens_input, tokens_output, success, latency_ms, timestamp)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?)`
|
||||
).run("openai", "gpt-4o", "test-conn", 100, 50, 1, 200, now.toISOString());
|
||||
|
||||
const response = await analyticsRoute.GET(
|
||||
makeRequest("http://localhost/api/usage/analytics?range=1h")
|
||||
);
|
||||
const body = await response.json();
|
||||
|
||||
assert.equal(response.status, 200);
|
||||
assert.equal(body.summary.totalRequests, 1);
|
||||
});
|
||||
|
||||
test("GET /api/usage/analytics does not throw Unknown named parameter with apiKey filter on long range", async () => {
|
||||
// Regression: Object.assign(presetParams, params) leaked all main-query bindings
|
||||
// into preset queries that only reference preset-prefixed placeholders.
|
||||
const apiKey = await apiKeysDb.createApiKey("Preset Key", "machine-preset1234");
|
||||
const db = core.getDbInstance();
|
||||
const now = new Date();
|
||||
|
||||
// Seed data old enough to trigger aggregated + preset path
|
||||
for (let i = 0; i < 5; i++) {
|
||||
const ts = new Date(now.getTime() - (35 + i) * 24 * 60 * 60 * 1000).toISOString();
|
||||
db.prepare(
|
||||
`INSERT INTO usage_history (provider, model, connection_id, api_key_id, api_key_name, tokens_input, tokens_output, success, latency_ms, timestamp)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`
|
||||
).run("openai", "gpt-4o", "test-conn", apiKey.id, apiKey.name, 100, 50, 1, 200, ts);
|
||||
}
|
||||
|
||||
const response = await analyticsRoute.GET(
|
||||
makeRequest(`http://localhost/api/usage/analytics?range=60d&apiKeyId=${apiKey.id}`)
|
||||
);
|
||||
const body = await response.json();
|
||||
|
||||
assert.equal(response.status, 200);
|
||||
// Core regression check: no "Unknown named parameter" error.
|
||||
// The exact count depends on raw-vs-aggregated boundary; we only need to
|
||||
// confirm the endpoint returns 200 without throwing.
|
||||
assert.ok(typeof body.summary.totalRequests === "number");
|
||||
});
|
||||
Reference in New Issue
Block a user