fix(db): publish the sql.js database atomically instead of rewriting it in place (#10278)

sql.js has no incremental write path, so persist() rewrites the whole image on
every save. Going through fs.writeFileSync(filePath, ...) opened the destination
with O_TRUNC, leaving the on-disk database 0 bytes and then partial for the whole
write -- a window that scales with database size and recurs on every save.

Unlike better-sqlite3 / node:sqlite, that window is not covered by SQLite's
locking protocol, so it is visible to every other process reading the same file:
a backup job, a metrics exporter, an operator running sqlite3. Those readers get
SQLITE_CORRUPT ("database disk image is malformed") while PRAGMA
integrity_check passes moments later, which makes the failure look random and
blames the reader.

Now: temp file in the same directory, fsync, rename() over the destination.
rename is atomic on POSIX and on Windows for a same-volume replace, so a reader
sees either the previous image or the new one, never a truncated one. It also
closes a total-loss window: a crash mid-write used to leave the real database
truncated, and now only leaves a stale temp file behind.

The regression guard asserts the property that separates the two implementations
without racing a timer: a reader that opened the file before a save still reads a
complete, valid image afterwards, and the published file sits on a new inode.
It fails on the previous implementation and passes on this one.

Co-authored-by: Max <maxmad64@gmail.com>
Co-authored-by: adevwithpurpose <adevwithpurpose@users.noreply.github.com>
This commit is contained in:
Dizzle
2026-08-16 05:14:30 +02:00
committed by GitHub
parent 462f4fc9da
commit b67d9ef353
3 changed files with 170 additions and 1 deletions

View File

@@ -127,10 +127,62 @@ export async function createSqlJsAdapter(filePath: string): Promise<SqliteAdapte
let saveTimer: ReturnType<typeof setTimeout> | null = null;
let _isOpen = true;
/**
* Writes the whole database image out atomically: temp file in the SAME
* directory, fsync, then `rename()` over the destination.
*
* WHY NOT `writeFileSync(filePath, …)` DIRECTLY
* ---------------------------------------------
* sql.js has no incremental write path — every save rewrites the entire image.
* `writeFileSync` opens the destination with `O_TRUNC`, so for the whole
* duration of the write the on-disk database is 0 bytes and then partial. The
* window scales with the database size and recurs on every save, so on a busy
* instance it is open a significant fraction of the time.
*
* Unlike better-sqlite3 / node:sqlite, that window is not protected by SQLite's
* locking protocol, so it is visible to every OTHER process that reads the same
* file — a backup job, a metrics exporter, an operator running `sqlite3`. Those
* readers get `SQLITE_CORRUPT` ("database disk image is malformed") even though
* `PRAGMA integrity_check` passes moments later, which makes the failure look
* random and points the blame at the reader.
*
* `rename()` within a directory is atomic on POSIX and on Windows for a
* same-volume replace, so a reader now sees either the previous image or the
* new one — never a truncated one. It also removes the total-loss window: a
* crash mid-write used to leave the real database truncated, while it now only
* leaves a stale temp file behind.
*/
function persist(): void {
if (filePath === ":memory:") return;
const data = db.export();
fs.writeFileSync(filePath, Buffer.from(data));
// Same directory, so `rename` stays within one filesystem — a temp file in
// os.tmpdir() would make it a cross-device copy, which is not atomic.
const tmpPath = `${filePath}.tmp-${process.pid}-${Date.now()}`;
let fd: number | null = null;
try {
fd = fs.openSync(tmpPath, "w");
fs.writeFileSync(fd, Buffer.from(data));
// The rename is atomic, but only orders against data that already reached
// the disk; without this an unclean shutdown can publish an empty file.
fs.fsyncSync(fd);
fs.closeSync(fd);
fd = null;
fs.renameSync(tmpPath, filePath);
} catch (err) {
if (fd !== null) {
try {
fs.closeSync(fd);
} catch {
/* already closed */
}
}
try {
fs.unlinkSync(tmpPath);
} catch {
/* never created, or already gone */
}
throw err;
}
dirty = false;
}