fix(compression): record no-op compression runs so Stacked is visible in analytics (#4268) (#5277)

A compression_analytics row was written only on a net-positive saving (chatCore: result.compressed || fallbackApplied || cavemanOutputModeApplied). A Stacked RTK->Caveman pipeline that ran on already-compact context saved nothing -> no row -> indistinguishable from 'never dispatched' (byMode.stacked.count stayed flat while Ultra climbed). Attempted-but-no-op runs are now recorded via writeCompressionSkip with a skip_reason. A new skip_reason column (added through the existing ensureCompressionAnalyticsColumns ALTER path) marks these rows; the analytics summary excludes them from every saving aggregate (via a single successWhere = 'skip_reason IS NULL' filter, so historical totals/averages are byte-identical) and surfaces them as per-mode 'skipped' plus totalSkipped / bySkipReason. The Mode Breakdown shows 'N skipped (no-op)'. TDD: compressionAnalytics.test.ts gains coverage that skip rows are counted separately without polluting saving aggregates, and that a skip-only mode still appears (count 0, skipped > 0). Fail-before proven by reverting the successWhere exclusion (both new tests fail). typecheck:core + eslint clean; 17/17 suite green.
2026-08-02 21:32:10 +03:00 · 2026-06-28 21:53:48 -03:00
parent ba38e31ff4
commit bc00e32b15
6 changed files with 219 additions and 17 deletions
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -15,6 +15,7 @@ _In development — bullets added per PR; finalized at release._

 ### 🔧 Bug Fixes

+- **compression (analytics):** record attempted-but-no-op compression runs so Stacked is no longer invisible when it saves nothing. Previously a `compression_analytics` row was written only on a net-positive saving, so a Stacked (RTK→Caveman) pipeline that ran on already-compact context produced no row — indistinguishable from "never dispatched" (`byMode.stacked.count` stayed flat while Ultra climbed). Such runs are now recorded with `skip_reason` and surfaced as a per-mode `skipped` count plus `totalSkipped`/`bySkipReason` in the analytics summary and the Mode Breakdown; the existing net-saving totals/averages are unchanged (skip rows are excluded from them) (#4268 — thanks @abdulkadirozyurt, @androw)
 - **cli (tray):** fix `omniroute server --tray` showing no tray on macOS/Linux with no error printed. The wired Unix tray path loaded `systray2` through an inline loader that called `require("module")` inside an ESM `.mjs` file (`"type":"module"`) → `ReferenceError: require is not defined`, silently swallowed (regressed in v3.8.34); even if it had loaded, `systray2` isn't in `node_modules` (it's lazily installed into `~/.omniroute/runtime`). The loader now delegates to the runtime loader, the icon path (`icon.png`) is corrected, `isTemplateIcon` is `false` (the full-color icon rendered as a white square under macOS template mode), and tray start failures are surfaced to stderr instead of being swallowed (#4605 — thanks @ProgMEM-CC)
 - **agent-bridge (antigravity):** unwrap the cloudcode-pa `.request` envelope when converting Antigravity IDE requests. The real IDE sends `cloudcode-pa.googleapis.com/v1internal:generateContent` with the Gemini request nested under `.request` (`{ project, model, request: { contents, systemInstruction, generationConfig } }`), but the bridge read those fields at the top level — yielding an empty conversation, so prompts hung mid-execution. The legacy `/v1beta/models/<model>:generateContent` top-level shape still works (#4294 — thanks @shabeer)
 - **dashboard:** add a GitHub releases fallback to the "Update Available" lookup. After the v3.8.28 fix added an npm-registry HTTP fallback, the banner could still stay hidden on networks that reach GitHub (where the news feed already loads) but not `registry.npmjs.org`. `resolveLatestVersion()` now tries npm CLI → npm registry → GitHub releases (`/repos/diegosouzapw/OmniRoute/releases/latest`) before giving up, and logs a warning only when all three fail (#4100)
--- a/open-sse/handlers/chatCore.ts
+++ b/open-sse/handlers/chatCore.ts
@@ -183,7 +183,10 @@ import {
  type RuntimeCompressionCombo,
 } from "./chatCore/compressionComboPredicates.ts";
 import { emitOutputStyleTelemetry } from "./chatCore/outputStyleTelemetry.ts";
-import { writeCompressionAnalytics } from "./chatCore/compressionAnalyticsWrite.ts";
+import {
+  writeCompressionAnalytics,
+  writeCompressionSkip,
+} from "./chatCore/compressionAnalyticsWrite.ts";
 import { runPluginOnRequestHook } from "./chatCore/pluginOnRequest.ts";
 import { recordContextEditingTelemetryHook } from "./chatCore/contextEditingTelemetry.ts";
 import { recordCompressionCacheStats } from "./chatCore/compressionCacheStats.ts";
@@ -1314,6 +1317,28 @@ export async function handleChatCore({
              cavemanOutputModeIntensity,
              log,
            });
+          } else {
+            // Compression was attempted (mode active, engines ran) but produced no
+            // recordable saving — e.g. a Stacked RTK→Caveman pipeline on already-compact
+            // context. Record a skip row so analytics can distinguish "ran but saved
+            // nothing" from "never ran" instead of dropping it silently (#4268).
+            compressionAnalyticsRecorded = true;
+            compressionAnalyticsWritePromise = writeCompressionSkip(
+              {
+                stats: result.stats,
+                provider,
+                effectiveModel,
+                effectiveServiceTier,
+                comboName,
+                mode,
+                compressionComboId: config.compressionComboId,
+                skillRequestId,
+                cavemanOutputModeApplied,
+                cavemanOutputModeIntensity,
+                log,
+              },
+              "no_savings"
+            );
          }

          if (result.compressed) {
--- a/open-sse/handlers/chatCore/compressionAnalyticsWrite.ts
+++ b/open-sse/handlers/chatCore/compressionAnalyticsWrite.ts
@@ -82,6 +82,42 @@ function buildEngineBreakdownRows(stats: CompressionStats, requestId: string) {
  }));
 }

+/**
+ * Record an attempted-but-no-op compression run (#4268). The pipeline ran (mode
+ * active, engines executed) but produced no recordable saving — without this, the
+ * row is dropped and "ran but saved nothing" is indistinguishable from "never ran".
+ * Writes a single skip row (tokens_saved = 0, skip_reason set); no engine breakdown,
+ * to keep skips out of the saving aggregates.
+ */
+export function writeCompressionSkip(opts: WriteOpts, skipReason: string): Promise<void> {
+  return (async () => {
+    try {
+      const { insertCompressionAnalyticsRow } = await import("@/lib/db/compressionAnalytics");
+      const { stats } = opts;
+      insertCompressionAnalyticsRow({
+        timestamp: new Date().toISOString(),
+        combo_id: opts.comboName ?? null,
+        provider: opts.provider ?? null,
+        mode: opts.mode,
+        engine: stats.engine ?? opts.mode,
+        compression_combo_id: stats.compressionComboId ?? opts.compressionComboId ?? null,
+        original_tokens: stats.originalTokens,
+        compressed_tokens: stats.compressedTokens,
+        tokens_saved: 0,
+        duration_ms: stats.durationMs ?? null,
+        request_id: opts.skillRequestId,
+        skip_reason: skipReason,
+      });
+    } catch (err) {
+      opts.log?.debug?.(
+        "COMPRESSION",
+        "Compression skip-analytics write skipped: " +
+          (err instanceof Error ? err.message : String(err))
+      );
+    }
+  })();
+}
+
 export function writeCompressionAnalytics(opts: WriteOpts): Promise<void> {
  return (async () => {
    try {
--- a/src/app/(dashboard)/dashboard/analytics/CompressionAnalyticsTab.tsx
+++ b/src/app/(dashboard)/dashboard/analytics/CompressionAnalyticsTab.tsx
@@ -16,9 +16,14 @@ interface CompressionAnalyticsSummary {
  totalTokensSaved: number;
  avgSavingsPct: number;
  avgDurationMs: number;
-  byMode: Record<string, { count: number; tokensSaved: number; avgSavingsPct: number }>;
+  byMode: Record<
+    string,
+    { count: number; tokensSaved: number; avgSavingsPct: number; skipped?: number }
+  >;
  byProvider: Record<string, { count: number; tokensSaved: number }>;
  last24h: Array<{ hour: string; count: number; tokensSaved: number }>;
+  totalSkipped?: number;
+  bySkipReason?: Record<string, number>;
  validationFallbacks: number;
  realUsage: {
    requestsWithReceipts: number;
@@ -60,11 +65,13 @@ function ModeBar({
  count,
  total,
  tokensSaved,
+  skipped = 0,
 }: {
  mode: string;
  count: number;
  total: number;
  tokensSaved: number;
+  skipped?: number;
 }) {
  const pct = total > 0 ? Math.round((count / total) * 100) : 0;
  return (
@@ -73,6 +80,11 @@ function ModeBar({
        <span className="font-medium text-text capitalize">{mode}</span>
        <span className="text-text-muted">
          {count} requests · {tokensSaved.toLocaleString()} tokens saved
+          {skipped > 0 && (
+            // #4268: attempted-but-no-op runs (e.g. Stacked saved nothing) are
+            // recorded now, so this mode is visible even when count is 0.
+            <span className="text-text-muted/70"> · {skipped.toLocaleString()} skipped (no-op)</span>
+          )}
        </span>
      </div>
      <div className="h-2 rounded-full bg-bg-muted overflow-hidden">
@@ -288,6 +300,7 @@ export default function CompressionAnalyticsTab() {
                count={data.count}
                total={stats.totalRequests}
                tokensSaved={data.tokensSaved}
+                skipped={data.skipped ?? 0}
              />
            ))}
          </div>
--- a/src/lib/db/compressionAnalytics.ts
+++ b/src/lib/db/compressionAnalytics.ts
@@ -28,6 +28,10 @@ export interface CompressionAnalyticsRow {
  rtk_raw_output_bytes?: number | null;
  rtk_raw_output_pointers?: string | null;
  rtk_raw_output_total_bytes?: number | null;
+  // Set on a no-op/skipped row: compression was attempted (mode active, engines
+  // ran) but produced no recordable saving. NULL on a normal saving row. Lets
+  // analytics distinguish "ran but saved nothing" from "never ran" (#4268).
+  skip_reason?: string | null;
 }

 /**
@@ -50,11 +54,21 @@ export interface CompressionAnalyticsSummary {
  totalTokensSaved: number;
  avgSavingsPct: number;
  avgDurationMs: number;
-  byMode: Record<string, { count: number; tokensSaved: number; avgSavingsPct: number }>;
+  // `count`/`tokensSaved`/`avgSavingsPct` cover net-saving runs only (skip rows
+  // excluded), preserving historical semantics. `skipped` = attempted-but-no-op
+  // runs for that mode, so Stacked is no longer invisible when it saves nothing (#4268).
+  byMode: Record<
+    string,
+    { count: number; tokensSaved: number; avgSavingsPct: number; skipped: number }
+  >;
  byEngine: Record<string, { count: number; tokensSaved: number; avgSavingsPct: number }>;
  byCompressionCombo: Record<string, { count: number; tokensSaved: number }>;
  byProvider: Record<string, { count: number; tokensSaved: number }>;
  last24h: Array<{ hour: string; count: number; tokensSaved: number }>;
+  // Total attempted-but-no-op compression runs (skip_reason set), and a breakdown
+  // by reason (e.g. "no_savings"). Recorded but excluded from the saving aggregates (#4268).
+  totalSkipped: number;
+  bySkipReason: Record<string, number>;
  validationFallbacks: number;
  realUsage: {
    requestsWithReceipts: number;
@@ -92,6 +106,7 @@ const COMPRESSION_ANALYTICS_COLUMNS = [
  ["rtk_raw_output_bytes", "INTEGER"],
  ["rtk_raw_output_pointers", "TEXT"],
  ["rtk_raw_output_total_bytes", "INTEGER"],
+  ["skip_reason", "TEXT"],
 ] as const;

 function ensureCompressionAnalyticsColumns(): void {
@@ -120,9 +135,9 @@ export function insertCompressionAnalyticsRow(row: CompressionAnalyticsRow): voi
      actual_total_tokens, actual_cache_read_tokens, actual_cache_write_tokens,
      estimated_usd_saved, mcp_description_tokens_saved, multimodal_skip_count,
      receipt_source, validation_fallback, output_mode, rtk_raw_output_pointer, rtk_raw_output_bytes,
-      rtk_raw_output_pointers, rtk_raw_output_total_bytes
+      rtk_raw_output_pointers, rtk_raw_output_total_bytes, skip_reason
    )
-    VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
+    VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
  `
  ).run(
    row.timestamp,
@@ -150,7 +165,8 @@ export function insertCompressionAnalyticsRow(row: CompressionAnalyticsRow): voi
    row.rtk_raw_output_pointer ?? null,
    row.rtk_raw_output_bytes ?? null,
    row.rtk_raw_output_pointers ?? null,
-    row.rtk_raw_output_total_bytes ?? null
+    row.rtk_raw_output_total_bytes ?? null,
+    row.skip_reason ?? null
  );
 }

@@ -366,6 +382,10 @@ export function getCompressionAnalyticsSummary(since?: string): CompressionAnaly

  const whereClause = cutoff ? "WHERE timestamp >= ?" : "";
  const params = cutoff ? [cutoff] : [];
+  // Saving aggregates count net-saving runs only: no-op/skip rows (skip_reason set)
+  // are excluded so historical totals/avgs are unchanged, while skips are surfaced
+  // separately below. (#4268)
+  const successWhere = appendCondition(whereClause, "skip_reason IS NULL");

  type ScalarRow = { total: number; totalSaved: number; avgPct: number; avgDur: number };
  const scalar = db
@@ -376,7 +396,7 @@ export function getCompressionAnalyticsSummary(since?: string): CompressionAnaly
      COALESCE(SUM(tokens_saved), 0) as totalSaved,
      COALESCE(AVG(CASE WHEN original_tokens > 0 THEN CAST(tokens_saved AS REAL) / original_tokens * 100 ELSE 0 END), 0) as avgPct,
      COALESCE(AVG(duration_ms), 0) as avgDur
-    FROM compression_analytics ${whereClause}
+    FROM compression_analytics ${successWhere}
  `
    )
    .get(...params) as ScalarRow | undefined;
@@ -386,15 +406,39 @@ export function getCompressionAnalyticsSummary(since?: string): CompressionAnaly
      `
    SELECT mode, COUNT(*) as cnt, COALESCE(SUM(tokens_saved), 0) as saved,
      COALESCE(AVG(CASE WHEN original_tokens > 0 THEN CAST(tokens_saved AS REAL) / original_tokens * 100 ELSE 0 END), 0) as avgPct
-    FROM compression_analytics ${whereClause}
+    FROM compression_analytics ${successWhere}
    GROUP BY mode
  `
    )
    .all(...params) as Array<{ mode: string; cnt: number; saved: number; avgPct: number }>;

-  const byMode: Record<string, { count: number; tokensSaved: number; avgSavingsPct: number }> = {};
+  // Attempted-but-no-op runs per mode (skip_reason set) — recorded since #4268 so
+  // Stacked is visible even when it saves nothing.
+  const skipModeRows = db
+    .prepare(
+      `
+    SELECT mode, COUNT(*) as cnt
+    FROM compression_analytics ${appendCondition(whereClause, "skip_reason IS NOT NULL")}
+    GROUP BY mode
+  `
+    )
+    .all(...params) as Array<{ mode: string; cnt: number }>;
+
+  const byMode: Record<
+    string,
+    { count: number; tokensSaved: number; avgSavingsPct: number; skipped: number }
+  > = {};
  for (const r of modeRows) {
-    byMode[r.mode] = { count: r.cnt, tokensSaved: r.saved, avgSavingsPct: Math.round(r.avgPct) };
+    byMode[r.mode] = {
+      count: r.cnt,
+      tokensSaved: r.saved,
+      avgSavingsPct: Math.round(r.avgPct),
+      skipped: 0,
+    };
+  }
+  for (const r of skipModeRows) {
+    if (byMode[r.mode]) byMode[r.mode].skipped = r.cnt;
+    else byMode[r.mode] = { count: 0, tokensSaved: 0, avgSavingsPct: 0, skipped: r.cnt };
  }

  const engineRows = db
@@ -402,7 +446,7 @@ export function getCompressionAnalyticsSummary(since?: string): CompressionAnaly
      `
    SELECT COALESCE(engine, mode) as engine, COUNT(*) as cnt, COALESCE(SUM(tokens_saved), 0) as saved,
      COALESCE(AVG(CASE WHEN original_tokens > 0 THEN CAST(tokens_saved AS REAL) / original_tokens * 100 ELSE 0 END), 0) as avgPct
-    FROM compression_analytics ${whereClause}
+    FROM compression_analytics ${successWhere}
    GROUP BY COALESCE(engine, mode)
  `
    )
@@ -423,7 +467,7 @@ export function getCompressionAnalyticsSummary(since?: string): CompressionAnaly
      `
    SELECT compression_combo_id as compressionComboId, COUNT(*) as cnt,
      COALESCE(SUM(tokens_saved), 0) as saved
-    FROM compression_analytics ${appendCondition(whereClause, "compression_combo_id IS NOT NULL")}
+    FROM compression_analytics ${appendCondition(successWhere, "compression_combo_id IS NOT NULL")}
    GROUP BY compression_combo_id ORDER BY cnt DESC
  `
    )
@@ -439,7 +483,7 @@ export function getCompressionAnalyticsSummary(since?: string): CompressionAnaly
    .prepare(
      `
    SELECT provider, COUNT(*) as cnt, COALESCE(SUM(tokens_saved), 0) as saved
-    FROM compression_analytics ${whereClause}
+    FROM compression_analytics ${successWhere}
    GROUP BY provider ORDER BY cnt DESC
  `
    )
@@ -465,7 +509,7 @@ export function getCompressionAnalyticsSummary(since?: string): CompressionAnaly
    SELECT strftime('%Y-%m-%dT%H:00:00Z', timestamp) as hour,
      COUNT(*) as cnt, COALESCE(SUM(tokens_saved), 0) as saved
    FROM compression_analytics
-    WHERE timestamp >= ?
+    WHERE timestamp >= ? AND skip_reason IS NULL
    GROUP BY hour ORDER BY hour ASC
  `
    )
@@ -493,7 +537,7 @@ export function getCompressionAnalyticsSummary(since?: string): CompressionAnaly
      COALESCE(SUM(actual_cache_read_tokens), 0) as cacheRead,
      COALESCE(SUM(actual_cache_write_tokens), 0) as cacheWrite,
      COALESCE(SUM(estimated_usd_saved), 0) as usdSaved
-    FROM compression_analytics ${appendCondition(whereClause, "receipt_source IS NOT NULL")}
+    FROM compression_analytics ${appendCondition(successWhere, "receipt_source IS NOT NULL")}
    GROUP BY receipt_source
  `
    )
@@ -534,7 +578,7 @@ export function getCompressionAnalyticsSummary(since?: string): CompressionAnaly
    .prepare(
      `
    SELECT COUNT(*) as cnt
-    FROM compression_analytics ${appendCondition(whereClause, "validation_fallback = 1")}
+    FROM compression_analytics ${appendCondition(successWhere, "validation_fallback = 1")}
  `
    )
    .get(...params) as { cnt: number } | undefined;
@@ -543,11 +587,29 @@ export function getCompressionAnalyticsSummary(since?: string): CompressionAnaly
    .prepare(
      `
    SELECT COUNT(*) as cnt, COALESCE(SUM(mcp_description_tokens_saved), 0) as saved
-    FROM compression_analytics ${appendCondition(whereClause, "mcp_description_tokens_saved > 0")}
+    FROM compression_analytics ${appendCondition(successWhere, "mcp_description_tokens_saved > 0")}
  `
    )
    .get(...params) as { cnt: number; saved: number } | undefined;

+  const skipReasonRows = db
+    .prepare(
+      `
+    SELECT skip_reason as reason, COUNT(*) as cnt
+    FROM compression_analytics ${appendCondition(whereClause, "skip_reason IS NOT NULL")}
+    GROUP BY skip_reason
+  `
+    )
+    .all(...params) as Array<{ reason: string | null; cnt: number }>;
+
+  const bySkipReason: Record<string, number> = {};
+  let totalSkipped = 0;
+  for (const r of skipReasonRows) {
+    const key = r.reason ?? "unknown";
+    bySkipReason[key] = r.cnt;
+    totalSkipped += r.cnt;
+  }
+
  return {
    totalRequests: scalar?.total ?? 0,
    totalTokensSaved: scalar?.totalSaved ?? 0,
@@ -558,6 +620,8 @@ export function getCompressionAnalyticsSummary(since?: string): CompressionAnaly
    byCompressionCombo,
    byProvider,
    last24h,
+    totalSkipped,
+    bySkipReason,
    validationFallbacks: fallbackRow?.cnt ?? 0,
    realUsage,
    mcpDescriptionCompression: {
--- a/tests/unit/compression/compressionAnalytics.test.ts
+++ b/tests/unit/compression/compressionAnalytics.test.ts
@@ -57,6 +57,8 @@ describe("compressionAnalytics", () => {
      byCompressionCombo: {},
      byProvider: {},
      last24h: summary.last24h,
+      totalSkipped: 0,
+      bySkipReason: {},
      validationFallbacks: 0,
      realUsage: {
        requestsWithReceipts: 0,
@@ -388,4 +390,65 @@ describe("compressionAnalytics", () => {
    assert.equal(summary.realUsage.requestsWithReceipts, 0);
    assert.equal(summary.realUsage.bySource.mcp_metadata_estimate, undefined);
  });
+
+  // #4268: attempted-but-no-op runs are recorded with skip_reason so Stacked is
+  // visible even when it saves nothing, while saving aggregates stay net-saving-only.
+  it("records skipped (no-op) runs separately without polluting saving aggregates", () => {
+    // One real saving run...
+    insertCompressionAnalyticsRow({
+      timestamp: new Date().toISOString(),
+      mode: "stacked",
+      original_tokens: 1000,
+      compressed_tokens: 600,
+      tokens_saved: 400,
+    });
+    // ...and two no-op stacked attempts (saved nothing).
+    insertCompressionAnalyticsRow({
+      timestamp: new Date().toISOString(),
+      mode: "stacked",
+      original_tokens: 500,
+      compressed_tokens: 500,
+      tokens_saved: 0,
+      skip_reason: "no_savings",
+    });
+    insertCompressionAnalyticsRow({
+      timestamp: new Date().toISOString(),
+      mode: "stacked",
+      original_tokens: 800,
+      compressed_tokens: 800,
+      tokens_saved: 0,
+      skip_reason: "no_savings",
+    });
+
+    const summary = getCompressionAnalyticsSummary();
+
+    // Saving aggregates count the net-saving run ONLY (skip rows excluded).
+    assert.equal(summary.totalRequests, 1, "totalRequests must exclude skip rows");
+    assert.equal(summary.totalTokensSaved, 400);
+    assert.equal(summary.byMode.stacked.count, 1, "byMode count excludes skip rows");
+    assert.equal(summary.byMode.stacked.tokensSaved, 400);
+
+    // ...but the skips are now visible instead of dropped.
+    assert.equal(summary.byMode.stacked.skipped, 2, "skipped attempts surfaced per mode");
+    assert.equal(summary.totalSkipped, 2);
+    assert.equal(summary.bySkipReason.no_savings, 2);
+  });
+
+  it("a mode with only no-op runs still appears (count 0, skipped > 0)", () => {
+    insertCompressionAnalyticsRow({
+      timestamp: new Date().toISOString(),
+      mode: "stacked",
+      original_tokens: 300,
+      compressed_tokens: 300,
+      tokens_saved: 0,
+      skip_reason: "no_savings",
+    });
+
+    const summary = getCompressionAnalyticsSummary();
+    assert.equal(summary.totalRequests, 0, "no net-saving runs");
+    assert.ok(summary.byMode.stacked, "stacked must appear even with only skip rows");
+    assert.equal(summary.byMode.stacked.count, 0);
+    assert.equal(summary.byMode.stacked.skipped, 1);
+    assert.equal(summary.totalSkipped, 1);
+  });
 });