perf(electron): verify better-sqlite3 v13 Node-API prebuilds instead of source rebuild (#10367)

better-sqlite3 v13 ships Node-API prebuilds for every packaged platform
(darwin/linux/linuxmusl/win32 x x64/arm64) inside the npm tarball, so the
Electron-ABI node-gyp source rebuild in prepare-electron-standalone.mjs is
obsolete. Replace it with a fail-fast prebuild verification that mirrors
better-sqlite3 lib/binding.js selection, and strip build/deps/src so the
packaged loader can only resolve the prebuild.

Verified locally on darwin-arm64: the same darwin-arm64.node prebuild loads
under both Node 24 (NODE_MODULE_VERSION 137) and Electron 43.3.0 under
ELECTRON_RUN_AS_NODE (148); DB create/migrate/read/write/close/reopen pass
in both runtimes and cross-runtime on each other's database files.

Issue #10321 Stage 6.
This commit is contained in:
backryun
2026-08-16 14:20:53 +09:00
committed by GitHub
parent 6d9336088c
commit 2162289f0a
7 changed files with 216 additions and 132 deletions

View File

@@ -697,11 +697,12 @@ jobs:
runs-on: ${{ matrix.os }}
timeout-minutes: 30
needs: build
# WS1.5 (v3.8.49 plan): the Electron rebuild/spawn path previously executed for
# WS1.5 (v3.8.49 plan): the Electron native-module path previously executed for
# the FIRST time on the release tag — the v3.8.48 Windows bug (npx.cmd spawned
# without shell, CVE-2024-27980 behavior change) could only surface at release.
# windows-latest runs prepare:bundle (the ABI rebuild + spawn plan) per release
# PR; ubuntu keeps the full pack + headless smoke.
# windows-latest runs prepare:bundle (better-sqlite3 prebuild verification since
# v13 — the node-gyp rebuild is gone) per release PR; ubuntu keeps the full
# pack + headless smoke.
strategy:
fail-fast: false
matrix:
@@ -738,7 +739,7 @@ jobs:
# precedent): its first-ever real run (2026-07-15, run 29457533565) died in
# 0.7s with the error swallowed by pwsh — bash shell captures stderr and
# continue-on-error keeps the heavy gate green while we harden it (#7336).
- name: Prepare Electron standalone (Windows ABI rebuild + spawn path)
- name: Prepare Electron standalone (Windows prebuild verification)
if: runner.os == 'Windows'
working-directory: electron
continue-on-error: true

View File

@@ -39,15 +39,15 @@ system tray, auto-updater, IPC bridge, and zero-config secret bootstrap.
Confirmed from `electron/package.json`:
| Package | Version |
| ------------------ | -------------------------- |
| `electron` | `^41.5.1` |
| `electron-builder` | `^26.10.0` |
| `electron-updater` | `^6.8.5` |
| `better-sqlite3` | `^12.9.0` |
| App version | `3.8.0` |
| App id | `online.omniroute.desktop` |
| Product name | `OmniRoute` |
| Package | Version |
| ------------------ | --------------------------------------------------------- |
| `electron` | `^41.5.1` |
| `electron-builder` | `^26.10.0` |
| `electron-updater` | `^6.8.5` |
| `better-sqlite3` | root `^13.0.2` (Node-API prebuilds — no Electron rebuild) |
| App version | `3.8.0` |
| App id | `online.omniroute.desktop` |
| Product name | `OmniRoute` |
## Scripts (root `package.json`)
@@ -260,14 +260,14 @@ Releases are published to GitHub Releases (`diegosouzapw/OmniRoute`), which is a
## Troubleshooting
| Symptom | Fix |
| --------------------------------------------------------------- | --------------------------------------------------------------------------- |
| `Cannot find module 'better-sqlite3'` after Electron major bump | `cd electron && npm rebuild` |
| `ERR_DLOPEN_FAILED` for native module | Re-run `prepare:bundle` and verify ABI matches Electron's Node |
| Window appears blank on Linux | Confirm Next.js server actually bound to PORT (check `[Server]` logs) |
| macOS notarization stalls | Ensure `APPLE_*` vars are exported, not just in `.env` |
| Windows SmartScreen warning | Sign with EV cert, or users right-click → "Run anyway" |
| Smoke test fails with port-in-use | Stop any local dev server on 20128 before running `electron:smoke:packaged` |
| Symptom | Fix |
| --------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `Cannot find module 'better-sqlite3'` after Electron major bump | better-sqlite3 v13 ships Node-API prebuilds — re-run `npm install` at the root and `prepare:bundle` (it verifies the prebuild for the current platform) |
| `ERR_DLOPEN_FAILED` for native module | Re-run `prepare:bundle` — it fails fast when the Node-API prebuild for the current platform is missing |
| Window appears blank on Linux | Confirm Next.js server actually bound to PORT (check `[Server]` logs) |
| macOS notarization stalls | Ensure `APPLE_*` vars are exported, not just in `.env` |
| Windows SmartScreen warning | Sign with EV cert, or users right-click → "Run anyway" |
| Smoke test fails with port-in-use | Stop any local dev server on 20128 before running `electron:smoke:packaged` |
## See Also

View File

@@ -39,7 +39,7 @@
* prune + validate (pack-artifact-policy) - Y - UNIQUE (prepublish)
* data/ dir creation - Y - UNIQUE (prepublish)
* --- electron-UNIQUE ---
* better-sqlite3 native strip + Electron-ABI rebuild - - Y UNIQUE (electron)
* better-sqlite3 prebuild verify + compile-input strip - - Y UNIQUE (electron)
* Turbopack hashed-module symlink materialize (node_modules) - - Y SHARED (opt-in: materializeSymlinks)
* symlink guard (assertBundleIsPackagable) - - Y UNIQUE (electron)
* removeGeneratedElectronArtifacts - - Y UNIQUE (electron)

View File

@@ -1,17 +1,73 @@
/**
* Spawn plan for the better-sqlite3 Electron-ABI rebuild (pure — import-safe for tests).
* better-sqlite3 Node-API prebuild planning (pure — import-safe for tests).
*
* On Windows, `npx.cmd` MUST be spawned through a shell: since Node's
* CVE-2024-27980 hardening, spawning `.cmd`/`.bat` shims without `shell: true`
* fails outright (spawnSync returns `status: null`), which broke the v3.8.47
* tag build ("better-sqlite3 rebuild against electron 43.1.0 failed (exit null)").
* The args are a fixed literal list — no untrusted input reaches the shell.
* Since better-sqlite3 v13 the packaged app no longer compiles the addon from
* source against the Electron headers: v13 ships Node-API (NAPI_VERSION=10)
* prebuilds for every platform we package, and Node-API addons are
* ABI-independent, so the same prebuild runs under plain Node and under the
* packaged app's ELECTRON_RUN_AS_NODE server (verified against electron 43 /
* NODE_MODULE_VERSION 148 — issue #10321 Stage 6). The historical
* `npx node-gyp rebuild` spawn plan existed because better-sqlite3@12 only
* shipped prebuilds up to electron-v146; v13 makes it obsolete.
*
* This module mirrors better-sqlite3's own `lib/binding.js` selection logic so
* the build fails fast when the prebuild the runtime loader would pick is
* missing, instead of shipping an app that falls back to sql.js and OOMs on a
* user machine.
*/
export function buildRebuildSpawnPlan(platform) {
const win = platform === "win32";
return {
command: win ? "npx.cmd" : "npx",
args: ["--yes", "node-gyp", "rebuild"],
shell: win,
};
import { existsSync } from "node:fs";
import { join } from "node:path";
export const SQLITE_PREBUILD_PLATFORMS = ["darwin", "linux", "linuxmusl", "win32"];
export const SQLITE_PREBUILD_ARCHS = ["x64", "arm64"];
/**
* Resolve the prebuild file name better-sqlite3's loader would pick for the
* given platform/arch. Mirrors lib/binding.js: linux without a glibc runtime
* version resolves to the linuxmusl prebuild.
*
* @param {string} platform - process.platform ("linux", "darwin", "win32")
* @param {string} arch - process.arch ("x64", "arm64")
* @param {{ glibcVersionRuntime?: string | null }} [reportHeader] - parsed
* process.report.getReport().header (injectable for tests)
*/
export function sqlitePrebuildFileName(platform, arch, reportHeader) {
const isMusl = platform === "linux" && !reportHeader?.glibcVersionRuntime;
const target = `${isMusl ? "linuxmusl" : platform}-${arch}`;
return `${target}.node`;
}
/**
* Whether a prebuild check applies for this platform/arch combination.
* Unsupported combos (e.g. freebsd-ia32) are skipped rather than failed: the
* runtime loader falls back to node-gyp build/ locations for those, which we
* do not package.
*/
export function isSqlitePrebuildSupported(platform, arch) {
return SQLITE_PREBUILD_PLATFORMS.includes(platform) && SQLITE_PREBUILD_ARCHS.includes(arch);
}
/**
* Assert that the runtime-selected prebuild exists in a staged module.
* Unsupported platform/arch combinations retain the historical fallback path.
*
* @returns {string | null} selected prebuild path, or null when unsupported
*/
export function assertSqlitePrebuildExists(moduleDir, platform, arch, reportHeader) {
if (!isSqlitePrebuildSupported(platform, arch)) return null;
const expected = join(
moduleDir,
"prebuilds",
sqlitePrebuildFileName(platform, arch, reportHeader)
);
if (!existsSync(expected)) {
throw new Error(
`[electron] better-sqlite3 prebuild missing for ${platform}-${arch} ` +
`(${expected}). The packaged app would fall back to sql.js and OOM. ` +
`Restore the prebuilds/ directory (npm cache / registry tarball) before packaging.`
);
}
return expected;
}

View File

@@ -1,11 +1,10 @@
#!/usr/bin/env node
import { cpSync, existsSync, lstatSync, readFileSync, readdirSync, rmSync } from "node:fs";
import { existsSync, lstatSync, readdirSync, rmSync } from "node:fs";
import { basename, dirname, join, relative } from "node:path";
import { fileURLToPath } from "node:url";
import { spawnSync } from "node:child_process";
import { assembleStandalone } from "./assembleStandalone.mjs";
import { buildRebuildSpawnPlan } from "./electronRebuildPlan.mjs";
import { assertSqlitePrebuildExists } from "./electronRebuildPlan.mjs";
import { pruneElectronRuntimeDocs } from "./electronRuntimeDocs.mjs";
const __filename = fileURLToPath(import.meta.url);
@@ -90,9 +89,7 @@ function removeNativeModules(baseDir, prefixes = ["keytar"]) {
// user machine as "Internal Server Error" on every route.
function assertNoStaleHashedNatives(baseDir, prefixes) {
if (!existsSync(baseDir)) return;
const leftovers = readdirSync(baseDir).filter((dir) =>
prefixes.some((p) => dir.startsWith(p))
);
const leftovers = readdirSync(baseDir).filter((dir) => prefixes.some((p) => dir.startsWith(p)));
if (leftovers.length > 0) {
throw new Error(
`[electron] stale native module copies survived cleanup in ${baseDir}: ` +
@@ -102,77 +99,43 @@ function assertNoStaleHashedNatives(baseDir, prefixes) {
}
}
// --- Electron-UNIQUE: rebuild better-sqlite3 against the Electron ABI --------
// --- Electron-UNIQUE: verify better-sqlite3 Node-API prebuilds ----------------
//
// The `npm ci` at the repo root compiles better-sqlite3 for the CI *Node* ABI
// (e.g. 137 for Node 24). The packaged app runs its Next.js server via
// ELECTRON_RUN_AS_NODE, so it needs the *Electron* ABI (146 for electron 42,
// 148 for electron 43). We cannot rely on electron-builder's @electron/rebuild
// here: it searches `electron/node_modules` (where better-sqlite3 does not live)
// and, with the default prebuild path, tries to fetch a prebuilt binary — but
// better-sqlite3@12.11.1 only ships prebuilds up to electron-v146, so electron
// 43 (v148) silently gets no rebuild and the app dies with "Nenhum driver
// SQLite disponível — better-sqlite3 (falhou)".
// better-sqlite3 >= 13 ships Node-API (NAPI_VERSION=10) prebuilds for every
// platform we package (darwin/linux/linuxmusl/win32 × x64/arm64) inside the
// npm tarball. Node-API addons are ABI-independent, so the same prebuild runs
// under plain Node (CI, CLI) and under the packaged app's ELECTRON_RUN_AS_NODE
// server (verified against electron 43 / NODE_MODULE_VERSION 148 — issue
// #10321 Stage 6). The historical source rebuild below existed because
// better-sqlite3@12 only shipped prebuilds up to electron-v146 and electron 43
// (v148) silently got no binary; v13 makes that obsolete.
//
// Instead we copy the *full* module (source + binding.gyp) from the root into
// the standalone and compile it from source against the Electron headers, so
// `bindings` finds a correct build/Release/better_sqlite3.node regardless of
// prebuild availability. Robust to any current/future electron version.
// Instead of compiling from source on every build (tens of seconds to minutes
// per platform), we fail fast when the prebuild for the CURRENT build platform
// is missing — a missing prebuild must kill the build here, not the app on a
// user machine with "Nenhum driver SQLite disponível — better-sqlite3 (falhou)".
function readElectronVersion() {
const pkg = JSON.parse(readFileSync(join(ROOT, "electron", "package.json"), "utf8"));
const raw = pkg.devDependencies?.electron || pkg.dependencies?.electron || "";
return String(raw).replace(/^[\^~]/, "");
}
function rebuildBetterSqlite3ForElectron(standaloneNodeModules) {
const srcMod = join(ROOT, "node_modules", "better-sqlite3");
if (!existsSync(srcMod)) {
console.warn("[electron] better-sqlite3 not found at repo root — skipping ABI rebuild.");
function verifyBetterSqlite3Prebuilds(standaloneNodeModules) {
const destMod = join(standaloneNodeModules, "better-sqlite3");
if (!existsSync(destMod)) {
console.warn("[electron] better-sqlite3 not found in standalone — skipping prebuild check.");
return;
}
const electronVersion = readElectronVersion();
if (!electronVersion) {
throw new Error("[electron] could not resolve electron version for better-sqlite3 rebuild.");
}
const destMod = join(standaloneNodeModules, "better-sqlite3");
// copyNatives only copies build/; we need the full module (src + binding.gyp)
// to compile from source. Overwrite the copied Node-ABI build in the process.
cpSync(srcMod, destMod, { recursive: true, force: true });
rmSync(join(destMod, "build"), { recursive: true, force: true });
console.log(`[electron] rebuilding better-sqlite3 against electron ${electronVersion} ABI…`);
const plan = buildRebuildSpawnPlan(process.platform);
const result = spawnSync(
plan.command,
plan.args,
{
cwd: destMod,
stdio: "inherit",
// .cmd shims must go through a shell on Windows (CVE-2024-27980 hardening
// makes a shell-less spawn fail with status null); args are fixed literals.
shell: plan.shell,
// Compile against the Electron headers (not Node's) so the .node lands in
// build/Release with the Electron NODE_MODULE_VERSION. No shell interpolation.
env: {
...process.env,
npm_config_runtime: "electron",
npm_config_target: electronVersion,
npm_config_disturl: "https://electronjs.org/headers",
npm_config_arch: process.arch,
npm_config_build_from_source: "true",
},
}
);
if (result.status !== 0) {
throw new Error(
`[electron] better-sqlite3 rebuild against electron ${electronVersion} failed (exit ${result.status}).`
);
}
// Drop the now-unneeded compile inputs to keep the packaged app lean.
for (const dir of ["deps", "src", "build/Debug", "build/obj.target"]) {
// Fail fast when the loader would find no prebuild for THIS build platform.
// Mirrors better-sqlite3's own lib/binding.js selection logic.
const reportHeader = process.report?.getReport?.().header;
assertSqlitePrebuildExists(destMod, process.platform, process.arch, reportHeader);
// Drop compile inputs and stale Node-ABI build outputs to keep the packaged
// app lean and to guarantee the loader resolves the prebuild, not a leftover
// build/Release/better_sqlite3.node compiled for a different ABI.
for (const dir of ["build", "deps", "src"]) {
rmSync(join(destMod, dir), { recursive: true, force: true });
}
console.log(
`[electron] better-sqlite3 Node-API prebuilds verified for ${process.platform}-${process.arch}.`
);
}
function logContextualError(error) {
@@ -217,12 +180,12 @@ if (docsPrune.removedFiles > 0) {
// Electron-UNIQUE post-assembly steps
removeGeneratedElectronArtifacts();
// Rebuild better-sqlite3 from source against the Electron ABI in the primary
// node_modules (where the standalone server resolves it). keytar is still
// stripped so electron-builder's @electron/rebuild handles it (it has electron
// prebuilds); also drop any stray Node-ABI better-sqlite3 under .next/node_modules
// so it cannot shadow the rebuilt one.
rebuildBetterSqlite3ForElectron(join(ELECTRON_STANDALONE_DIR, "node_modules"));
// Verify better-sqlite3 Node-API prebuilds in the primary node_modules (where
// the standalone server resolves it). keytar is still stripped so
// electron-builder's @electron/rebuild handles it (it has electron prebuilds);
// also drop any stray better-sqlite3 under .next/node_modules so it cannot
// shadow the prebuild-backed one.
verifyBetterSqlite3Prebuilds(join(ELECTRON_STANDALONE_DIR, "node_modules"));
removeNativeModules(join(ELECTRON_STANDALONE_DIR, "node_modules"), ["keytar"]);
removeNativeModules(join(ELECTRON_STANDALONE_DIR, NEXT_DIST_DIR, "node_modules"), [
"better-sqlite3",

View File

@@ -1,22 +0,0 @@
import { test } from "node:test";
import assert from "node:assert/strict";
import { buildRebuildSpawnPlan } from "../../scripts/build/electronRebuildPlan.mjs";
// Regression: v3.8.47 tag build — spawnSync("npx.cmd", ...) WITHOUT shell:true fails with
// status null on Windows runners (Node's CVE-2024-27980 hardening blocks spawning .cmd/.bat
// without a shell), killing the better-sqlite3 Electron-ABI rebuild:
// "[electron] better-sqlite3 rebuild against electron 43.1.0 failed (exit null)".
test("win32 rebuild plan spawns through a shell (cmd shims need it since CVE-2024-27980)", () => {
const plan = buildRebuildSpawnPlan("win32");
assert.equal(plan.command, "npx.cmd");
assert.equal(plan.shell, true);
assert.deepEqual(plan.args, ["--yes", "node-gyp", "rebuild"]);
});
test("posix rebuild plan spawns npx directly, no shell", () => {
const plan = buildRebuildSpawnPlan("linux");
assert.equal(plan.command, "npx");
assert.equal(plan.shell, false);
assert.deepEqual(plan.args, ["--yes", "node-gyp", "rebuild"]);
});

View File

@@ -0,0 +1,86 @@
import { test } from "node:test";
import assert from "node:assert/strict";
import fs from "node:fs";
import os from "node:os";
import path from "node:path";
import * as prebuildPlan from "../../scripts/build/electronRebuildPlan.mjs";
const {
SQLITE_PREBUILD_ARCHS,
SQLITE_PREBUILD_PLATFORMS,
isSqlitePrebuildSupported,
sqlitePrebuildFileName,
} = prebuildPlan;
// Since better-sqlite3 v13 (issue #10321 Stage 6) the Electron packaging no
// longer compiles the addon from source against the Electron headers: v13
// ships Node-API prebuilds for every packaged platform, and Node-API addons
// are ABI-independent (verified under electron 43 / NODE_MODULE_VERSION 148).
// These tests pin the prebuild selection logic that replaced the historical
// `npx node-gyp rebuild` spawn plan (whose win32 .cmd/shell quirk broke the
// v3.8.47 tag build — that entire code path is now gone).
test("prebuild file name mirrors better-sqlite3 lib/binding.js selection", () => {
assert.equal(sqlitePrebuildFileName("darwin", "arm64"), "darwin-arm64.node");
assert.equal(sqlitePrebuildFileName("darwin", "x64"), "darwin-x64.node");
assert.equal(sqlitePrebuildFileName("win32", "x64"), "win32-x64.node");
assert.equal(sqlitePrebuildFileName("win32", "arm64"), "win32-arm64.node");
});
test("linux resolves to the musl prebuild when glibcVersionRuntime is absent", () => {
// glibc build (GitHub ubuntu runner): header carries the runtime glibc version
assert.equal(
sqlitePrebuildFileName("linux", "x64", { glibcVersionRuntime: "2.39" }),
"linux-x64.node"
);
// musl build (Alpine): no glibcVersionRuntime -> linuxmusl prebuild
assert.equal(sqlitePrebuildFileName("linux", "x64", {}), "linuxmusl-x64.node");
assert.equal(sqlitePrebuildFileName("linux", "arm64", undefined), "linuxmusl-arm64.node");
});
test("prebuild support covers exactly the packaged platform/arch matrix", () => {
for (const platform of ["darwin", "linux", "win32"]) {
for (const arch of ["x64", "arm64"]) {
assert.equal(isSqlitePrebuildSupported(platform, arch), true);
}
}
assert.equal(isSqlitePrebuildSupported("freebsd", "x64"), false);
assert.equal(isSqlitePrebuildSupported("darwin", "ia32"), false);
});
test("packaged platform matrix matches the shipped prebuild inventory", () => {
// better-sqlite3 v13 prebuilds/: darwin/linux/linuxmusl/win32 × x64/arm64.
// The build fails fast when the prebuild for the CURRENT platform is missing,
// so this matrix must stay in sync with the npm tarball contents.
assert.deepEqual(SQLITE_PREBUILD_PLATFORMS, ["darwin", "linux", "linuxmusl", "win32"]);
assert.deepEqual(SQLITE_PREBUILD_ARCHS, ["x64", "arm64"]);
});
test("prebuild verification fails fast when the selected binary is missing", () => {
const assertSqlitePrebuildExists = (
prebuildPlan as typeof prebuildPlan & {
assertSqlitePrebuildExists?: (
moduleDir: string,
platform: string,
arch: string,
reportHeader?: { glibcVersionRuntime?: string | null }
) => string | null;
}
).assertSqlitePrebuildExists;
assert.equal(typeof assertSqlitePrebuildExists, "function");
const moduleDir = fs.mkdtempSync(path.join(os.tmpdir(), "sqlite-prebuild-"));
try {
assert.throws(
() => assertSqlitePrebuildExists?.(moduleDir, "darwin", "arm64"),
/better-sqlite3 prebuild missing for darwin-arm64/
);
const expected = path.join(moduleDir, "prebuilds", "darwin-arm64.node");
fs.mkdirSync(path.dirname(expected), { recursive: true });
fs.writeFileSync(expected, "napi");
assert.equal(assertSqlitePrebuildExists?.(moduleDir, "darwin", "arm64"), expected);
} finally {
fs.rmSync(moduleDir, { recursive: true, force: true });
}
});