feat(db): let the migration runner scan extra namespaced directories (#8770)

The runner reads exactly one directory and records the bare numeric prefix as the
version, so the numeric slots are a single global namespace. Any distribution that
ships its own migrations next to the upstream set has to draw from that same range
while upstream keeps appending to it — and when both sides claim a number, the
runner records one name for it and treats the other as already applied. That
migration then never runs, silently, on every already-provisioned database.

OMNIROUTE_EXTRA_MIGRATIONS_DIRS registers additional directories as
`namespace=dir` entries separated by the platform path delimiter. Files found
there are recorded as `<namespace>-<number>` (e.g. `ee-134`), a version space that
cannot collide with the upstream numeric one, and they are applied after the core
set. Unset — the default, and the only case for a plain install — nothing changes:
no filesystem access, identical behaviour.

Misconfiguration throws instead of being skipped. A malformed entry, a namespace
outside [a-z][a-z0-9]*, a duplicate namespace, or a directory that does not exist
aborts startup, because silently missing schema is the exact failure this exists to
prevent. Two files sharing a number inside the SAME namespace still collide and
throw, mirroring the runner's own guard.

Also fixes an inconsistency the tests surfaced: a missing core directory returned
early and took the extra directories down with it. They are an independent set.

The version-namespaced strings need no further plumbing — the applied set, the gap
reconciliation and the name-mismatch check all key on the version string, and the
numeric-only paths (`Number.parseInt`, the "032"/"041"/"042" special cases,
`isSchemaAlreadyApplied`) ignore them by construction.

11 new tests; the 7 neighbouring migration suites stay green (62 tests).
This commit is contained in:
Diego Rodrigues de Sa e Souza
2026-07-27 12:56:06 -03:00
committed by GitHub
parent b46bb6d6f1
commit 5f365bae7c
5 changed files with 501 additions and 2 deletions

View File

@@ -93,6 +93,7 @@ OmniRoute uses **SQLite** (via `better-sqlite3`) for all persistence. These vari
| `OMNIROUTE_FORCE_DB_HEALTHCHECK` | `0` | `src/lib/db/core.ts` | Set to `1` to force the DB healthcheck loop on, even when it would normally be skipped (e.g., short-lived tasks). |
| `OMNIROUTE_SKIP_POSTINSTALL` | `0` | `scripts/postinstall.mjs` | Set to `1` to skip the native-runtime warm-up during `npm install`. Useful in CI/headless installs where sqlite is already built. |
| `OMNIROUTE_MIGRATIONS_DIR` | _(auto-detect)_ | `src/lib/db/migrationRunner.ts` | Override the directory that the migration runner scans. Useful when shipping bundled migrations in custom builds. |
| `OMNIROUTE_EXTRA_MIGRATIONS_DIRS` | _(unset)_ | `src/lib/db/migrationRunner/extraDirs.ts` | Additional migration directories as `namespace=dir` entries separated by the platform path delimiter (e.g. `ee=/opt/app/enterprise/db/migrations`). Files found there are recorded as `<namespace>-<number>`, so a distribution shipping its own migrations never collides with the upstream numeric slots. A malformed entry, an invalid namespace or a missing directory throws at startup instead of silently skipping the schema. |
| `OMNIROUTE_MAX_PENDING_MIGRATIONS` | `50` | `src/lib/db/migrationRunner.ts` | Mass-pending-migrations safety threshold (#3416). Startup aborts if more than this many migrations are pending on an existing DB (guards against a wiped tracking table). Raise it to restore an older backup; set to `0` to disable the check. |
| `OMNIROUTE_SPEND_FLUSH_INTERVAL_MS` | _(default in code)_ | `src/lib/spend/batchWriter.ts` | Flush interval (ms) for the batched spend/cost writer. Lower values reduce write coalescing; higher values reduce DB contention. |
| `OMNIROUTE_SPEND_MAX_BUFFER_SIZE` | _(default in code)_ | `src/lib/spend/batchWriter.ts` | Max buffered spend entries before a forced flush. Raise on high-QPS deployments; lower when bounded memory matters more. |