Files
OmniRoute/tests/unit/proxySubscription.service.test.ts
Adam f31f3c081e feat(proxy): operator-level proxy subscriptions (Karing-style) — hardened, ready for review (#7299)
* feat(proxy-subscriptions): src/lib/proxySubscription/parse.ts

* feat(proxy-subscriptions): src/lib/proxySubscription/subscriptionService.ts

* feat(proxy-subscriptions): src/lib/proxySubscription/index.ts

* feat(proxy-subscriptions): src/lib/db/migrations/123_proxy_subscriptions.sql

* feat(proxy-subscriptions): src/app/api/v1/management/proxy-subscriptions/route.ts

* feat(proxy-subscriptions): src/app/api/v1/management/proxy-subscriptions/[id]/route.ts

* feat(proxy-subscriptions): src/app/api/v1/management/proxy-subscriptions/[id]/refresh/route.ts

* feat(proxy-subscriptions): src/app/api/v1/management/proxy-subscriptions/[id]/nodes/route.ts

* feat(proxy-subscriptions): src/app/(dashboard)/dashboard/settings/components/proxy/SubscriptionTab.tsx

* feat(proxy-subscriptions): tests/unit/proxySubscription.parse.test.ts

* feat(proxy-subscriptions): tests/unit/proxySubscription.service.test.ts

* feat(proxy-subscriptions): docs/proxy-subscriptions.md

* feat(proxy-subscriptions): src/lib/db/proxies/types.ts

* feat(proxy-subscriptions): src/lib/db/proxies/mappers.ts

* feat(proxy-subscriptions): src/lib/db/proxies.ts

* feat(proxy-subscriptions): src/app/(dashboard)/dashboard/settings/components/ProxyTab.tsx

* i18n(proxy-subscriptions): add proxySubscriptionsTab key + use in ProxyTab

* i18n(proxy-subscriptions): add proxySubscriptionsTab key + use in ProxyTab

* i18n(proxy-subscriptions): add proxySubscriptionsTab key + use in ProxyTab

* i18n(proxy-subscriptions): add proxySubscriptionsTab key + use in ProxyTab

* test(proxy-subscriptions): extract isSubscriptionDue + unit tests

* test(proxy-subscriptions): extract isSubscriptionDue + unit tests

* test(proxy-subscriptions): extract isSubscriptionDue + unit tests

* test(proxy-subscriptions): add global->rule switch re-bind integration test

* feat(proxy-subscriptions): inline needs-local-core guidance in SubscriptionTab

* i18n: add proxySubscriptionsTab to en (rebased on current main)

* i18n: add proxySubscriptionsTab to zh-CN (rebased on current main)

* i18n: add proxySubscriptionsTab to pt-BR (rebased on current main)

* test(proxy-subscriptions): extract needsCore detection into pure module + unit tests

* test(proxy-subscriptions): extract needsCore detection into pure module + unit tests

* test(proxy-subscriptions): extract needsCore detection into pure module + unit tests

* refactor(proxy-subscriptions): extract scopes.ts into pure module + unit tests (#65)

* refactor(proxy-subscriptions): extract coreEndpoint.ts into pure module + unit tests (#65)

* refactor(proxy-subscriptions): extract subscriptionService.ts into pure module + unit tests (#65)

* refactor(proxy-subscriptions): extract proxySubscription.scopes.test.ts into pure module + unit tests (#65)

* refactor(proxy-subscriptions): extract proxySubscription.coreEndpoint.test.ts into pure module + unit tests (#65)

* security(proxy-subscriptions): fetchGuard.ts — SSRF guard + core scheme (#P0)

* security(proxy-subscriptions): coreEndpoint.ts — SSRF guard + core scheme (#P0)

* security(proxy-subscriptions): subscriptionService.ts — SSRF guard + core scheme (#P0)

* security(proxy-subscriptions): proxySubscription.fetchGuard.test.ts — SSRF guard + core scheme (#P0)

* security(proxy-subscriptions): proxySubscription.coreEndpoint.test.ts — SSRF guard + core scheme (#P0)

* refactor(proxy-subscriptions): subscriptionService.ts — concurrency lock / resilience / url redaction (#P1)

* refactor(proxy-subscriptions): url.ts — concurrency lock / resilience / url redaction (#P1)

* refactor(proxy-subscriptions): index.ts — concurrency lock / resilience / url redaction (#P1)

* refactor(proxy-subscriptions): route.ts — concurrency lock / resilience / url redaction (#P1)

* refactor(proxy-subscriptions): route.ts — concurrency lock / resilience / url redaction (#P1)

* refactor(proxy-subscriptions): proxySubscription.url.test.ts — concurrency lock / resilience / url redaction (#P1)

* i18n(proxy-subscriptions): subscriptionService.ts — stable error codes + locale keys (#P2-6)

* i18n(proxy-subscriptions): SubscriptionTab.tsx — stable error codes + locale keys (#P2-6)

* i18n(proxy-subscriptions): en.json — stable error codes + locale keys (#P2-6)

* i18n(proxy-subscriptions): zh-CN.json — stable error codes + locale keys (#P2-6)

* i18n(proxy-subscriptions): pt-BR.json — stable error codes + locale keys (#P2-6)

* i18n(proxy-subscriptions): en.json — normalize to LF line endings (#P2-6)

* i18n(proxy-subscriptions): zh-CN.json — normalize to LF line endings (#P2-6)

* i18n(proxy-subscriptions): pt-BR.json — normalize to LF line endings (#P2-6)

* enhance(proxy-subscriptions): reject subscription fetch if ANY resolved DNS address is blocked (P3-1)

* enhance(proxy-subscriptions): add withRetry() exponential-backoff helper (P3-2)

* enhance(proxy-subscriptions): DNS multi-record guard + retry/backoff fetch + batch scope writes + observability (P3-1..P3-4)

* enhance(proxy-subscriptions): batch addProxiesToScopePool() to drop N+1 writes (P3-3)

* enhance(proxy-subscriptions): add last_error_at + consecutive_failures observability columns (P3-4)

* enhance(proxy-subscriptions): surface consecutive failures + last error time in subscription cards (P3-4)

* test(proxy-subscriptions): cover multi-record DNS SSRF (block if ANY address internal) (P3-1)

* test(proxy-subscriptions): cover withRetry() first-success / retries / backoff / non-retryable stop (P3-2)

* fix(proxy-subscriptions): resolve js-yaml import + missing backup/generation-bump imports

Two bugs made the feature non-functional and its own test suite false:

1. parse.ts used `import yaml from "js-yaml"` (default import), but
   js-yaml@^5 is ESM-only with no default export — this threw a
   SyntaxError at module load, crashing every caller (index.ts
   re-exports parse.ts, so every API route hit this too). Switch to
   `import * as yaml`, matching how the rest of the codebase already
   imports js-yaml (hermes-agent.ts, openapiParser.ts, openapi/spec
   route.ts, guide-settings route.ts).

2. subscriptionService.ts's unapplySubscription() called backupDbFile()
   and bumpProxyRegistryGeneration() without importing either —
   backupDbFile exists but wasn't imported; bumpProxyRegistryGeneration
   was a private, non-exported function in db/proxies.ts. This threw a
   ReferenceError whenever a subscription with bound proxies was
   disabled/deleted (the normal path). Import backupDbFile from
   ../db/backup and export+import bumpProxyRegistryGeneration from
   ../db/proxies, matching the identical backup+bump pattern already
   used by deleteProxyById for the same proxy_assignments/proxy_registry
   mutation shape.

Fixing both unmasked a third, previously-unreachable bug (both crashes
happened before any test assertion could run): recomputeProxyEnabled()
checked `proxy_subscriptions.enabled = 1` alone, which stays true across
an unapply/disable cycle since unapplySubscription() never touches that
column — the proxyEnabled flag would get stuck on `true` even after the
subscription's proxies were fully detached. Changed the check to require
an actually-bound proxy_assignments row for an enabled subscription,
matching hasNonSubscriptionGlobalProxy()'s existing bound-check pattern.
Traced all 3 production call sites (mode/rule switch, disable, delete)
to confirm this doesn't change their outcome — only the previously-wrong
"unapply in isolation" case.

Also fixed the test file's own pre-existing bug: its provider_connections
inserts omitted created_at/updated_at (NOT NULL, no default in the
schema since 001_initial_schema.sql), which 0 assertions had ever
reached before because the SyntaxError always crashed the file first.

All 9 proxySubscription test files now run clean: 49/49 pass (previously
2 files crashed outright at import time, 0 assertions ever ran).

Separately confirmed via testing against the pristine PR head: this PR
has 3 more pre-existing gate failures unrelated to the above (file-size
on db/proxies.ts, cognitive-complexity, complexity, changelog-integrity
vs the current release tip) plus 3 pre-existing TS2345 errors in
parse.ts (lines 228/233/263, unrelated to the yaml import). All are the
PR's own scope/base-drift, out of scope for this fix — the PR has never
had a real CI run (base=main), so no gate has ever surfaced them; they
need the base retarget + a full CI pass called out in the plan file's
own remaining mandatory items.

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>

* fix(db): renumber proxy-subscriptions migrations to avoid version collision

release/v3.8.49 already ships 123_quota_auto_ping.sql and
124_generic_session_affinity_ttl.sql; this PR's 123/124 files collided,
tripping migrationRunner's version-collision guard and failing all
proxySubscription.service tests. Renumber to 127/128 (next free slots
after 126_reasoning_routing_rules.sql).

Co-authored-by: Diego Rodrigues de Sa e Souza <diegosouzapw@users.noreply.github.com>

* refactor(db): extract proxySubscriptions + registryGeneration to keep proxies.ts under file-size cap

Moves addProxiesToScopePool to ./proxySubscriptions.ts and the registry-generation
helpers to ./proxies/registryGeneration.ts (re-exported from proxies.ts), keeping the
module under its frozen 1177-line cap after the operator-proxy-subscriptions feature.

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>

* fix(db): renumber proxy-subscriptions migrations past release collision

Rebasing onto release/v3.8.49 surfaced a migration version collision: this
branch's 127_proxy_subscriptions.sql and 128_proxy_subscriptions_meta.sql
now collide with 127_usage_history_account_identity.sql and
128_auto_candidate_overrides.sql that landed on release since this branch
last synced. Renumbered to 131/132 (next free prefixes after the current
130_remove_unregistered_qwen_data.sql) and updated the in-file header
comments to match. No schema/behavior change.

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>

---------

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
Co-authored-by: Diego Rodrigues de Sa e Souza <diegosouza.pw@gmail.com>
Co-authored-by: Diego Rodrigues de Sa e Souza <diegosouzapw@users.noreply.github.com>
Co-authored-by: xier2012 <xier2012@users.noreply.github.com>
2026-07-21 13:16:41 -03:00

232 lines
7.8 KiB
TypeScript

import test from "node:test";
import assert from "node:assert/strict";
import fs from "node:fs";
import os from "node:os";
import path from "node:path";
const TEST_DATA_DIR = fs.mkdtempSync(path.join(os.tmpdir(), "omniroute-sub-svc-"));
process.env.DATA_DIR = TEST_DATA_DIR;
const core = await import("../../src/lib/db/core.ts");
const proxies = await import("../../src/lib/db/proxies.ts");
const sub = await import("../../src/lib/proxySubscription/index.ts");
function reset() {
core.resetDbInstance();
fs.rmSync(TEST_DATA_DIR, { recursive: true, force: true });
fs.mkdirSync(TEST_DATA_DIR, { recursive: true });
}
function nowIso() {
return new Date().toISOString();
}
function insertSubscription(
db: ReturnType<typeof core.getDbInstance>,
id: string,
mode: "global" | "rule",
opts: { enabled?: boolean; ruleProviders?: string[] } = {}
) {
db.prepare(
`INSERT INTO proxy_subscriptions
(id, name, url, enabled, mode, rule_providers, update_interval_minutes, status, created_at, updated_at)
VALUES (?, ?, ?, ?, ?, ?, ?, 'empty', ?, ?)`
).run(
id,
`sub-${id}`,
`https://example.com/${id}`,
opts.enabled === false ? 0 : 1,
mode,
opts.ruleProviders ? JSON.stringify(opts.ruleProviders) : null,
60,
nowIso(),
nowIso()
);
}
test.after(() => {
core.resetDbInstance();
fs.rmSync(TEST_DATA_DIR, { recursive: true, force: true });
});
test("global subscription binds its pool to the global scope and is resolvable", async () => {
await reset();
const db = core.getDbInstance();
insertSubscription(db, "s1", "global", { enabled: true });
const created = await proxies.createProxy({
name: "node1",
type: "http",
host: "10.0.0.1",
port: 8080,
source: "subscription",
subscriptionId: "s1",
status: "active",
});
await sub.applySubscription("s1");
const resolved = await proxies.resolveProxyForConnectionFromRegistry("conn-xyz");
assert.ok(resolved, "expected a global proxy to be resolved");
assert.equal(resolved?.proxy.host, "10.0.0.1");
const flag = db
.prepare("SELECT value FROM key_value WHERE namespace='settings' AND key='proxyEnabled'")
.get() as { value: string };
assert.equal(JSON.parse(flag.value), true);
// Unapply -> proxy removed from global scope, proxyEnabled recomputed to false.
await sub.unapplySubscription("s1");
const resolved2 = await proxies.resolveProxyForConnectionFromRegistry("conn-xyz");
assert.equal(resolved2, null, "expected direct (no proxy) after unapply");
const flag2 = db
.prepare("SELECT value FROM key_value WHERE namespace='settings' AND key='proxyEnabled'")
.get() as { value: string };
assert.equal(JSON.parse(flag2.value), false);
});
test("rule subscription binds only the selected provider scope", async () => {
await reset();
const db = core.getDbInstance();
insertSubscription(db, "s2", "rule", { enabled: true, ruleProviders: ["provA"] });
const created = await proxies.createProxy({
name: "node2",
type: "http",
host: "10.0.0.2",
port: 8080,
source: "subscription",
subscriptionId: "s2",
status: "active",
});
await sub.applySubscription("s2");
db.prepare(
"INSERT INTO provider_connections (id, provider, created_at, updated_at) VALUES (?,?,?,?)"
).run("connA", "provA", nowIso(), nowIso());
db.prepare(
"INSERT INTO provider_connections (id, provider, created_at, updated_at) VALUES (?,?,?,?)"
).run("connB", "provB", nowIso(), nowIso());
const rA = await proxies.resolveProxyForConnectionFromRegistry("connA");
assert.ok(rA, "provider A should resolve the rule proxy");
assert.equal(rA?.proxy.host, "10.0.0.2");
const rB = await proxies.resolveProxyForConnectionFromRegistry("connB");
assert.equal(rB, null, "provider B is not in the rule set -> direct");
});
test("fail-closed: a dead subscription proxy blocks the connection instead of leaking", async () => {
await reset();
const db = core.getDbInstance();
insertSubscription(db, "s3", "global", { enabled: true });
await proxies.createProxy({
name: "deadnode",
type: "http",
host: "10.0.0.9",
port: 1,
source: "subscription",
subscriptionId: "s3",
status: "dead",
});
await sub.applySubscription("s3");
const flag = db
.prepare("SELECT value FROM key_value WHERE namespace='settings' AND key='proxyEnabled'")
.get() as { value: string };
assert.equal(JSON.parse(flag.value), true);
// Dead proxy is excluded from resolution -> request would go direct.
const resolved = await proxies.resolveProxyForConnectionFromRegistry("connZ");
assert.equal(resolved, null);
// But the operator assigned a (now dead) proxy, so this must be blocked.
const blocked = proxies.hasBlockingProxyAssignment("connZ");
assert.equal(blocked, true);
});
test("deleteSubscription unbinds and removes its proxy rows", async () => {
await reset();
const db = core.getDbInstance();
insertSubscription(db, "s4", "global", { enabled: true });
await proxies.createProxy({
name: "node4",
type: "http",
host: "10.0.0.4",
port: 8080,
source: "subscription",
subscriptionId: "s4",
status: "active",
});
await sub.applySubscription("s4");
const ok = await sub.deleteSubscription("s4");
assert.equal(ok, true);
const rows = db
.prepare("SELECT id FROM proxy_registry WHERE subscription_id = ?")
.all("s4") as Array<{ id: string }>;
assert.equal(rows.length, 0, "subscription proxy rows should be removed");
const assignments = db
.prepare("SELECT 1 FROM proxy_assignments a JOIN proxy_registry p ON p.id=a.proxy_id WHERE p.source='subscription' LIMIT 1")
.get();
assert.equal(assignments, undefined, "no subscription proxy should remain assigned");
const subRow = db.prepare("SELECT 1 FROM proxy_subscriptions WHERE id='s4'").get();
assert.equal(subRow, undefined);
});
test("global→rule switch re-evaluates binding: drops global, binds the selected provider scope", async () => {
await reset();
const db = core.getDbInstance();
// Seed a proxy node directly (avoids a network fetch for this part).
await proxies.createProxy({
name: "node5",
type: "http",
host: "10.0.0.5",
port: 8080,
source: "subscription",
subscriptionId: "s5",
status: "active",
});
// Start in GLOBAL mode and bind the pool to the global scope.
insertSubscription(db, "s5", "global", { enabled: true });
await sub.applySubscription("s5");
const before = await proxies.resolveProxyForConnectionFromRegistry("connAny");
assert.ok(before, "global mode should resolve the node for any connection");
assert.equal(before?.proxy.host, "10.0.0.5");
// Switch to RULE mode targeting provider provA. updateSubscription must
// detach the previous global binding (unapplySubscription) and re-bind the
// pool to the selected provider scope. Stub fetch so syncSubscription's
// re-fetch returns a body that keeps node5 around.
const realFetch = globalThis.fetch;
globalThis.fetch = (async () => ({
ok: true,
text: async () =>
"proxies:\n - name: node5\n type: http\n server: 10.0.0.5\n port: 8080\n",
})) as unknown as typeof fetch;
try {
await sub.updateSubscription("s5", { mode: "rule", ruleProviders: ["provA"] });
} finally {
globalThis.fetch = realFetch;
}
// The global binding must be gone.
const afterGlobal = await proxies.resolveProxyForConnectionFromRegistry("connAny");
assert.equal(afterGlobal, null, "global binding should be dropped after switching to rule mode");
// The provider-scope binding must now resolve the node.
db.prepare(
"INSERT INTO provider_connections (id, provider, created_at, updated_at) VALUES (?,?,?,?)"
).run("connA", "provA", nowIso(), nowIso());
const afterRule = await proxies.resolveProxyForConnectionFromRegistry("connA");
assert.ok(afterRule, "rule mode should bind the node to provider provA");
assert.equal(afterRule?.proxy.host, "10.0.0.5");
});