Files
OmniRoute/scripts/build/postinstall.mjs
Diego Rodrigues de Sa e Souza e7f6b1d130 feat(radar): flag-gated signed free-model catalog overlay (#9515)
* feat(dashboard): add RADAR_ENABLED flag (default off)

* feat(db): radar feed cache + settings with encrypted supporter key

* feat(radar): signed feed sync with pinned key and version floor

- feedSchema.ts: Zod v4 schema mirroring the server feed format
  (discriminated union on budget.kind, enum constraints, etc.)
- pinnedKeys.ts: Ed25519 SPKI-DER pinned key + env override for forks
- verify.ts: signature verification over exact wire bytes, never throws
- sync.ts: full download/verify/validate/cache pipeline with injectable
  deps, feature-flag gate, opt-in gate, version floor (numeric compare),
  and sanitized error reasons (no stack traces)
- 40 tests covering: contract hash, key handling, sig verification,
  schema validation, version compare, all sync paths (disabled, opt_out,
  invalid_signature, invalid_schema, stale, updated, error), auth header
  injection, and cache-untouched assertions for every failure mode

* feat(radar): read-time overlay merge rules over the free catalog

Pure function applyFeed() merges the cached Radar feed over the static
baseline catalog at read time, honoring 4 rules:

1. Feed never overwrites a local override field.
2. enabled:false disables the entry with disabledBy:"radar" provenance.
3. User-added entry NOT in the feed survives untouched.
4. User deletion tombstone prevents feed resurrection.

getRadarCatalog() accessor in index.ts: flag off / no cache / corrupt
payload all fall back to baseline. Valid cache applies the overlay and
returns feed metadata (version, tier, fetchedAt).

TDD: 19 tests (4 rules + dedup + origin + accessor flag/cache/corrupt/
valid/bad-feed + baselineToMergedEntries converter).

* feat(dashboard): radar catalog and guided setup screens

- API routes: GET /api/radar/catalog, POST /api/radar/sync, POST /api/radar/settings
  - All gated on RADAR_ENABLED flag (404 when off)
  - Error responses via buildErrorBody(), never raw stack/message
  - Settings never echoes clear supporter key (masked omr_****<last4>)
  - Sync delegates to syncRadar() server-side, never proxies feed URL
- Dashboard pages:
  - /dashboard/radar: 4 states (flag off, opt-in pending, empty, populated)
  - /dashboard/radar/setup?provider=X: guided setup with steps, key URL, test connection
  - Uses existing Card component and next-intl patterns
- Sidebar: radar entry in costs group with icon
- i18n: pt-BR and en keys for radarPage and radarSetupPage namespaces
- Tests:
  - radar-api-routes.test.ts: 11 tests (flag-off 404, flag-on shape, error sanitization)
  - radar-page-state.test.ts: 5 tests (pure state logic)
  - All 90 radar tests pass (including prior 74)

* docs(radar): module doc and flag-off inertia test

Add docs/frameworks/RADAR.md covering the flag gate, the separate data-sync
opt-in and privacy promise, the Ed25519 signature/pinned-key security model,
tiers, the read-time overlay merge rules, and the self-hosting env vars —
plus index entries in CLAUDE.md/AGENTS.md/docs/README.md/REPOSITORY_MAP.md.

Document RADAR_FEED_URL and RADAR_FEED_PUBKEY in .env.example and
docs/reference/ENVIRONMENT.md to satisfy check:env-doc-sync, which was
failing on this branch since the sync.ts commit added the reads.

Add tests/unit/radar-inertia.test.ts as the single canonical place asserting
the "RADAR_ENABLED off => zero behavioral delta" claim end to end: the three
/api/radar/* routes 404, the flag resolves to the definition default with no
override, getRadarCatalog() returns exactly the baseline without touching the
cache, and computeFreeModelTotals() keeps its pinned values with the Radar
module imported alongside it.

* fix(db): renumber radar migration to 135 after collision with 134

The base branch introduced 134_proxy_logs_egress_ip while this branch carried
134_radar_cache_settings; the migration runner rejects duplicate numeric prefixes.
This migration has never been applied to a real database (the PR is unmerged), so
no retroactive isSchemaAlreadyApplied guard is needed.

* i18n(radar): translate radar catalog and setup strings to all locales

The UI-coverage ratchet measures (present - placeholder) / total_en, so the
__MISSING__ sentinels that i18n:sync-ui writes do not count as covered — only
real translations restore the metric. Scoped to this PR's namespaces
(radarPage, radarSetupPage, sidebar.radar*) instead of a bulk sync, which would
have pulled ~978 unrelated pending keys into this diff.

Placeholders and code identifiers verified preserved across all 1682 strings.

* fix(radar): trust the served-tier header instead of the signed body field

The signed feed body always carries tier:"live" by design (one signed
artifact per version — rewriting the field server-side per request
would break the exact-bytes Ed25519 signature). The server now returns
the tier ACTUALLY served via the x-omniroute-feed-tier response
header, so free users on a delayed community snapshot no longer see
"Ao vivo (tempo real)" in the UI.

sync.ts now reads and validates that header (falling back to the
body's tier only when the header is absent or holds an unrecognized
value) and stores the served tier in the cache; index.ts already
surfaces cache.tier to the UI unchanged.

* test(combo): shorten an assert message that exceeded the line limit

The assertion added by #9507 was 104 chars, so prettier reformatted it into
five lines on the next commit that touched the file, pushing it past its
frozen size (3449) and failing check:file-size. The message is shortened
(the issue reference stays in the comment directly above); the assertion
itself is unchanged, and the file is back to 3448 lines and prettier-clean.

* i18n(radar): use the canonical zh-TW glossary terms

The machine translation produced retired renderings the glossary gate blocks:
供應商 for provider (canonical 提供者) and 文檔 for documentation (canonical 文件).
Fixed across the 11 affected radar strings; tests/unit/i18n-glossary-consistency-check.test.ts
is back to 17/17.

* fix(radar): point the default feed URL at the domain that exists

radar.omniroute.dev was a placeholder for a domain that was never registered,
so an out-of-the-box sync would fail DNS resolution for every user. The live
feed is served from radar.omniroute.online (the subdomain the design always
specified), now behind Cloudflare TLS. Forks still override it via
RADAR_FEED_URL.

---------

Co-authored-by: diegosouzapw <diegosouzapw@users.noreply.github.com>
2026-08-06 05:58:58 -03:00

456 lines
16 KiB
JavaScript

#!/usr/bin/env node
/**
* OmniRoute — Postinstall Native Module Fix
*
* The npm package ships with a Next.js standalone build that includes
* native modules compiled for the build platform (Linux x64) inside
* dist/node_modules/. However, npm also installs these as top-level
* dependencies (in the root node_modules/), correctly compiled for
* the user's platform.
*
* This script copies the correctly-built native binaries from the root
* into the standalone dist directory — no rebuild or build tools needed.
*
* Modules repaired:
* - better-sqlite3 (SQLite bindings)
* - wreq-js (TLS client for OAuth providers)
* - tls-client-node (TLS client for chatgpt-web/claude-web/grok-web/lmarena/perplexity-web)
*
* Fixes: https://github.com/diegosouzapw/OmniRoute/issues/129
* Fixes: https://github.com/diegosouzapw/OmniRoute/issues/321
* Fixes: https://github.com/diegosouzapw/OmniRoute/issues/426
* Fixes: https://github.com/diegosouzapw/OmniRoute/issues/1634
* Fixes: https://github.com/diegosouzapw/OmniRoute/issues/7802
*/
import {
copyFileSync,
cpSync,
existsSync,
mkdirSync,
readFileSync,
readdirSync,
writeFileSync,
} from "node:fs";
import { dirname, join } from "node:path";
import { fileURLToPath } from "node:url";
import { PUBLISHED_BUILD_ARCH, PUBLISHED_BUILD_PLATFORM } from "./native-binary-compat.mjs";
import { hasStandaloneAppBundle, isTermux } from "./postinstallSupport.mjs";
import { colocateLlmlinguaOptionals } from "./colocateOptionals.mjs";
import { fixTlsClientNodeBinary } from "./fixTlsClientNodeBinary.mjs";
import { fixPlaywrightAndroid } from "./fixPlaywrightAndroid.mjs";
const __filename = fileURLToPath(import.meta.url);
const __dirname = dirname(__filename);
const ROOT = join(__dirname, "..", "..");
/**
* Patch node-gyp's common.gypi to include the android_ndk_path variable.
*
* On Termux/Android, node-gyp's bundled common.gypi (in ~/.cache/node-gyp/<version>/)
* does not define the `android_ndk_path` variable that the build system expects.
* Setting GYP_DEFINES="android_ndk_path=''" is not enough because common.gypi
* is parsed separately and the variable must be declared in the 'variables' section.
*
* This function finds and patches the common.gypi for the current Node.js version,
* adding `'android_ndk_path%': ''` to the variables block. The patch is idempotent.
*/
function patchNodeGypCommonGypi() {
try {
const nodeVersion = process.version; // e.g. "v26.4.0"
const gypDir = join(
process.env.HOME || process.env.USERPROFILE || "/root",
".cache",
"node-gyp",
nodeVersion.replace(/^v/, "")
);
const commonGypi = join(gypDir, "include", "node", "common.gypi");
if (!existsSync(commonGypi)) {
console.warn(` ⚠️ common.gypi not found at ${commonGypi}, skipping patch`);
return;
}
let content = readFileSync(commonGypi, "utf8");
// Check if already patched
if (content.includes("android_ndk_path")) {
return;
}
// Find the variables section and add android_ndk_path
// The pattern is: 'variables': { 'node_use_openssl%': ... }
// We insert our variable right after the opening of the variables block
const variablesMatch = content.match(/('variables'\s*:\s*\{)/);
if (variablesMatch) {
const insertPos = content.indexOf(variablesMatch[0]) + variablesMatch[0].length;
content =
content.slice(0, insertPos) + "\n 'android_ndk_path%': ''," + content.slice(insertPos);
writeFileSync(commonGypi, content, "utf8");
console.log(` ✅ Patched common.gypi for Android at ${commonGypi}`);
}
} catch (err) {
console.warn(` ⚠️ Could not patch common.gypi: ${err.message}`);
}
}
const appBinary = join(
ROOT,
"dist",
"node_modules",
"better-sqlite3",
"build",
"Release",
"better_sqlite3.node"
);
const rootBinary = join(
ROOT,
"node_modules",
"better-sqlite3",
"build",
"Release",
"better_sqlite3.node"
);
async function fixBetterSqliteBinary() {
if (!existsSync(join(ROOT, "dist", "node_modules", "better-sqlite3"))) {
return;
}
const platformMatch =
process.platform === PUBLISHED_BUILD_PLATFORM && process.arch === PUBLISHED_BUILD_ARCH;
if (platformMatch) {
try {
process.dlopen({ exports: {} }, appBinary);
return;
} catch (err) {
console.warn(` ⚠️ Bundled binary incompatible despite platform match: ${err.message}`);
}
}
console.log(`\n 🔧 Fixing better-sqlite3 binary for ${process.platform}-${process.arch}...`);
if (existsSync(rootBinary)) {
try {
mkdirSync(dirname(appBinary), { recursive: true });
copyFileSync(rootBinary, appBinary);
} catch (err) {
console.warn(` ⚠️ Failed to copy binary: ${err.message}`);
}
try {
process.dlopen({ exports: {} }, appBinary);
console.log(" ✅ Native module fixed successfully!\n");
return;
} catch (err) {
console.warn(` ⚠️ Copied binary failed to load: ${err.message}`);
}
}
console.log(" 📥 Attempting to download prebuilt binary via node-pre-gyp...");
try {
const { execSync } = await import("node:child_process");
const preGypBin = join(
ROOT,
"dist",
"node_modules",
".bin",
process.platform === "win32" ? "node-pre-gyp.cmd" : "node-pre-gyp"
);
const preGypFallback = join(
ROOT,
"dist",
"node_modules",
"@mapbox",
"node-pre-gyp",
"bin",
"node-pre-gyp"
);
const preGypCmd = existsSync(preGypBin) ? preGypBin : preGypFallback;
if (existsSync(preGypCmd)) {
execSync(`"${process.execPath}" "${preGypCmd}" install --fallback-to-build=false`, {
cwd: join(ROOT, "dist", "node_modules", "better-sqlite3"),
stdio: "inherit",
timeout: 60_000,
});
mkdirSync(dirname(appBinary), { recursive: true });
try {
process.dlopen({ exports: {} }, appBinary);
console.log(" ✅ Prebuilt binary downloaded and loaded successfully!\n");
return;
} catch (loadErr) {
console.warn(` ⚠️ Downloaded binary failed to load: ${loadErr.message}`);
}
} else {
console.warn(" ⚠️ node-pre-gyp not found, skipping prebuilt download.");
}
} catch (err) {
console.warn(` ⚠️ node-pre-gyp download failed: ${err.message.split("\n")[0]}`);
}
console.log(" ⚠️ Attempting npm rebuild (requires build tools)...");
try {
const { execSync } = await import("node:child_process");
// On Android/Termux, rebuild from source with --build-from-source flag
const isAndroid = process.platform === "android" || isTermux();
const rebuildCmd = isAndroid
? "npm install better-sqlite3 --build-from-source --force"
: "npm rebuild better-sqlite3";
const env = { ...process.env };
if (isAndroid) {
env.GYP_DEFINES = "android_ndk_path=''";
// Patch node-gyp's common.gypi to include android_ndk_path variable
// so the gyp build system doesn't fail with "Unknown variable"
patchNodeGypCommonGypi();
}
execSync(rebuildCmd, {
cwd: join(ROOT, "dist"),
stdio: "inherit",
timeout: isAndroid ? 600_000 : 300_000, // ARM compilation is slower
env,
});
process.dlopen({ exports: {} }, appBinary);
console.log(" ✅ Native module rebuilt successfully!\n");
return;
} catch (err) {
const isTimeout = err.killed || err.signal === "SIGTERM";
if (isTimeout) {
const secs = isAndroid ? 600 : 300;
console.warn(` ⚠️ npm rebuild timed out after ${secs}s.`);
} else {
console.warn(` ⚠️ npm rebuild failed: ${err.message}`);
}
}
console.warn("\n ⚠️ Could not fix better-sqlite3 native module automatically.");
console.warn(" The server may not start correctly.");
console.warn(" Manual fix options:");
if (process.platform === "win32") {
console.warn(" Option A (easiest — no build tools needed):");
console.warn(` cd "${join(ROOT, "dist", "node_modules", "better-sqlite3")}"`);
console.warn(" npx @mapbox/node-pre-gyp install --fallback-to-build=false");
console.warn(" Option B (requires Build Tools for Visual Studio):");
console.warn(` cd "${join(ROOT, "dist")}" && npm rebuild better-sqlite3`);
console.warn(" Install from: https://visualstudio.microsoft.com/visual-cpp-build-tools/");
console.warn(" Also ensure Python is installed: https://python.org");
} else if (process.platform === "darwin") {
console.warn(` cd ${join(ROOT, "dist")} && npm rebuild better-sqlite3`);
console.warn(" If build tools are missing: xcode-select --install");
} else {
console.warn(` cd ${join(ROOT, "dist")} && npm rebuild better-sqlite3`);
}
console.warn("");
}
/**
* Fix wreq-js native binary for the standalone dist directory.
*
* wreq-js ships platform-specific .node binaries under rust/.
* The standalone build may only contain Linux binaries from the CI.
* This copies the correct platform binary from the root install.
*
* Fixes: https://github.com/diegosouzapw/OmniRoute/issues/1634
*/
async function fixWreqJsBinary() {
// wreq-js native module is not loadable in Termux (libgcc path mismatch).
// The runtime already falls back gracefully when wreq-js is unavailable.
if (process.platform === "android" || isTermux()) {
console.log(
" [postinstall] wreq-js: skipped on Termux/Android " +
"(libgcc not available — OAuth TLS fingerprinting will use the fallback path)"
);
return;
}
const appWreqDir = join(ROOT, "dist", "node_modules", "wreq-js", "rust");
const rootWreqDir = join(ROOT, "node_modules", "wreq-js", "rust");
if (!existsSync(join(ROOT, "dist", "node_modules", "wreq-js"))) {
return;
}
const binaryName = `wreq-js.${process.platform}-${process.arch}.node`;
const appBinaryPath = join(appWreqDir, binaryName);
const rootBinaryPath = join(rootWreqDir, binaryName);
// Check if the platform binary already exists and loads
if (existsSync(appBinaryPath)) {
try {
process.dlopen({ exports: {} }, appBinaryPath);
return; // Already working
} catch (err) {
console.warn(` ⚠️ wreq-js binary exists but failed to load: ${err.message}`);
}
}
console.log(`\n 🔧 Fixing wreq-js binary for ${process.platform}-${process.arch}...`);
// Strategy 1: Copy from root node_modules
if (existsSync(rootBinaryPath)) {
try {
mkdirSync(appWreqDir, { recursive: true });
copyFileSync(rootBinaryPath, appBinaryPath);
process.dlopen({ exports: {} }, appBinaryPath);
console.log(" ✅ wreq-js native module fixed successfully!\n");
return;
} catch (err) {
console.warn(` ⚠️ Copied wreq-js binary failed to load: ${err.message}`);
}
}
// Strategy 2: Copy entire rust/ directory from root (gets all platform binaries)
if (existsSync(rootWreqDir)) {
try {
mkdirSync(appWreqDir, { recursive: true });
const files = readdirSync(rootWreqDir);
for (const file of files) {
if (file.endsWith(".node")) {
copyFileSync(join(rootWreqDir, file), join(appWreqDir, file));
}
}
if (existsSync(appBinaryPath)) {
process.dlopen({ exports: {} }, appBinaryPath);
console.log(" ✅ wreq-js native module fixed (full copy) successfully!\n");
return;
}
} catch (err) {
console.warn(` ⚠️ wreq-js full copy failed: ${err.message}`);
}
}
// Strategy 3: Rebuild wreq-js inside dist/
console.log(" 📥 Attempting npm rebuild wreq-js...");
try {
const { execSync } = await import("node:child_process");
execSync("npm rebuild wreq-js", {
cwd: join(ROOT, "dist"),
stdio: "inherit",
timeout: 120_000,
});
if (existsSync(appBinaryPath)) {
process.dlopen({ exports: {} }, appBinaryPath);
console.log(" ✅ wreq-js native module rebuilt successfully!\n");
return;
}
} catch (err) {
console.warn(` ⚠️ wreq-js rebuild failed: ${err.message}`);
}
console.warn(
`\n ⚠️ Could not fix wreq-js native module for ${process.platform}-${process.arch}.`
);
console.warn(" OAuth-based providers (Codex, Cursor, etc.) may not work.");
console.warn(` Manual fix: cd ${join(ROOT, "dist")} && npm install wreq-js --no-save\n`);
}
async function ensureSwcHelpers() {
if (!hasStandaloneAppBundle(ROOT)) {
return;
}
const swcHelpersApp = join(ROOT, "dist", "node_modules", "@swc", "helpers");
const swcHelpersRoot = join(ROOT, "node_modules", "@swc", "helpers");
if (existsSync(swcHelpersApp)) {
return;
}
if (existsSync(swcHelpersRoot)) {
try {
const { cpSync } = await import("node:fs");
mkdirSync(join(ROOT, "dist", "node_modules", "@swc"), { recursive: true });
cpSync(swcHelpersRoot, swcHelpersApp, { recursive: true });
console.log(" ✅ @swc/helpers copied to standalone dist/node_modules.\n");
} catch (err) {
console.warn(` ⚠️ Could not copy @swc/helpers: ${err.message}`);
console.warn(
" Try manually: cp -r node_modules/@swc/helpers dist/node_modules/@swc/helpers\n"
);
}
return;
}
console.warn(" ⚠️ @swc/helpers not found in root node_modules either.");
console.warn(" Try: npm install --save-exact @swc/helpers@0.5.19\n");
}
async function syncProjectEnv() {
try {
const { syncEnv } = await import("./sync-env.mjs");
syncEnv({ rootDir: ROOT });
} catch (err) {
console.warn(` ⚠️ .env sync skipped: ${err.message}`);
}
}
/**
* Co-locate the LLMLingua-2 SLM optional dependency closure into dist/node_modules so the
* compression "ultra" SLM tier (PR #4257) resolves a single @huggingface/transformers instance at
* runtime. No-op unless the optionals were installed (`--include=optional`). See colocateOptionals.mjs.
*/
async function ensureLlmlinguaOptionals() {
try {
colocateLlmlinguaOptionals({ rootDir: ROOT, log: (m) => console.log(m) });
} catch (err) {
// Best-effort: the SLM tier is itself fail-open, so a co-location hiccup never fails the install.
console.warn(` ⚠️ LLMLingua optional co-location skipped: ${err.message}`);
}
}
/**
* Preflight check for development installs (when standalone dist/ bundle is not present).
* Warns or errors if critical native dependencies like better-sqlite3 were skipped by npm >= 11
* allowScripts restrictions.
*/
async function verifyDevNativeModules() {
if (hasStandaloneAppBundle(ROOT)) {
return;
}
const criticalModules = [
{ name: "better-sqlite3", fatal: true },
{ name: "esbuild", fatal: true },
];
for (const { name, fatal } of criticalModules) {
if (!existsSync(join(ROOT, "node_modules", name))) {
const level = fatal ? "🔴 CRITICAL" : "⚠️ WARNING";
console.error(`\n ${level}: '${name}' is missing from node_modules/`);
console.error(` This usually happens with npm ≥ 11, which blocks install`);
console.error(` scripts for optional dependencies by default.`);
console.error(`\n Fix options:`);
console.error(` 1. npm approve-scripts ${name} && npm install`);
console.error(` 2. npm pack ${name} && tar -xzf ${name}-*.tgz -C node_modules`);
console.error(` && mv node_modules/package node_modules/${name}`);
console.error(` 3. Downgrade to npm 10: npm install -g npm@10\n`);
}
}
}
await verifyDevNativeModules();
await fixBetterSqliteBinary();
await fixWreqJsBinary();
await fixTlsClientNodeBinary({ rootDir: ROOT });
await fixPlaywrightAndroid({ rootDir: ROOT });
await ensureSwcHelpers();
await ensureLlmlinguaOptionals();
await syncProjectEnv();
// Warm up native runtimes (better-sqlite3 in ~/.omniroute/runtime/).
// Non-fatal: errors are caught inside postinstall.mjs.
try {
await import("../postinstall.mjs");
} catch {
// Silently skip — runtime warm-up is best-effort.
}