feat(proxy): implement latency-optimized proxy rotation strategy (#6798)

* feat(proxy): implement latency-optimized proxy rotation strategy

Reconstructed onto release/v3.8.47 to drop unrelated main-drift (deps/electron/proxy
files belong to #6620, not this PR) and the direct CHANGELOG.md edit (fragments-first);
the author's env/docs/i18n deltas were re-applied cleanly onto the release tip.

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>

* fix(proxy): add latency-rotation env var to .env.example

PROXY_LATENCY_WINDOW_HOURS was referenced in src/lib/db/proxies.ts and
documented in docs/reference/ENVIRONMENT.md, but missing from
.env.example, tripping the env/docs sync gate.

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>

* chore(changelog): re-sync CHANGELOG.md to release tip

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>

* refactor(proxy): extract latency-strategy helpers to keep frozen files under cap

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>

* test(db-rules): expect 35 audited modules (proxyLatency joins INTENTIONALLY_INTERNAL)

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>

---------

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>

.env.example

@@ -1527,6 +1527,11 @@ APP_LOG_TO_FILE=true
 # Timeout for fast-fail health checks (ms).  Default: 2000
 # PROXY_FAST_FAIL_TIMEOUT_MS=2000
 
+# Time window (hours) for calculating the average latency of candidate proxies
+# in the latency-optimized pool strategy.  Default: 3
+# Used by: src/lib/db/proxies.ts
+# PROXY_LATENCY_WINDOW_HOURS=3
+
 # Health check result cache TTL (ms).  Default: 30000 (30s)
 # PROXY_HEALTH_CACHE_TTL_MS=30000
 

changelog.d/features/6798-latency-optimized-proxy-rotation.md

@@ -0,0 +1 @@
+- **feat(proxy):** add a latency-optimized proxy rotation strategy that ranks pool entries by measured round-trip latency, extending the existing round-robin/random/sticky proxy-pool selection (#6798 — thanks @iamraydoan).

docs/reference/ENVIRONMENT.md

@@ -835,6 +835,7 @@ Anthropic-compatible provider instead.
 | Variable                                        | Default                 | Source File                                                                             | Description                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               |
 | ----------------------------------------------- | ----------------------- | --------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
 | `PROXY_FAST_FAIL_TIMEOUT_MS`                    | `2000`                  | `src/lib/proxyHealth.ts`                                                                | Fast-fail health check timeout.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           |
+| `PROXY_LATENCY_WINDOW_HOURS`                    | `3`                     | `src/lib/db/proxies.ts`                                                                 | Time window (hours) for calculating the average latency of candidate proxies in the latency-optimized pool strategy.                                                                                                                                                                                                                                                                                                                                                                                                                                                      |
 | `PROXY_HEALTH_CACHE_TTL_MS`                     | `30000`                 | `src/lib/proxyHealth.ts`                                                                | Health check result cache TTL.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
 | `PROXY_HEALTH_UNHEALTHY_CACHE_TTL_MS`           | `2000`                  | `src/lib/proxyHealth.ts`                                                                | Cache TTL for failed proxy health probes. Keep this shorter than `PROXY_HEALTH_CACHE_TTL_MS` so transient proxy timeouts under high concurrency retry quickly without disabling fast-fail for truly dead proxies.                                                                                                                                                                                                                                                                                                                                                          |
 | `PROXY_HEALTH_ENABLED`                          | `true`                  | `src/lib/proxyHealth/scheduler.ts`                                                      | Set `false` to disable the background proxy health scheduler that periodically probes registered proxies.                                                                                                                                                                                                                                                                                                                                                                                                                                                                 |

scripts/check/check-db-rules.mjs

@@ -64,6 +64,7 @@ export const INTENTIONALLY_INTERNAL = new Set([
   "prompts", // DEAD? (production): zero callers de produção encontrados; domínio domain/prompts.ts é independente; testado por tests/integration/proxy-pipeline.test.ts
   "providerNodeSelect", // db-internal: importado só por db/providers.ts (selectProviderNodeForConnection — lógica pura de seleção de provider node split do providers.ts, #4421)
   "providerStats", // intentionally-internal: src/app/api/provider-stats/route.ts
+  "proxyLatency", // intentionally-internal: imported directly by src/lib/db/proxies.ts (anti-barrel, #6798)
   "recovery", // intentionally-internal: bin/cli/runtime.mjs (import() dinâmico) + tests
   "schemaColumns", // db-internal: importado só por db/core.ts (ensureProviderConnections/UsageHistory/CallLogsColumns + hasColumn/hasTable/getTableColumns — schema-column reconciliation split do core.ts, #4948)
   "secrets", // intentionally-internal: src/instrumentation-node.ts (import() dinâmico na inicialização)

src/app/(dashboard)/dashboard/settings/components/ProxyRegistryManager.tsx

@@ -8,6 +8,7 @@ import { ProxyStatusBadge } from "./ProxyStatusBadge";
 import { ProxyHealthCell } from "./ProxyHealthCell";
 import { ProxyBatchActions } from "./ProxyBatchActions";
 import { ProxyCheckboxCell } from "./ProxyCheckboxCell";
+import { POOL_STRATEGY_OPTIONS, isPoolStrategy, type PoolStrategy } from "./proxyStrategyOptions";
 
 type ProxyItem = {
   id: string;
@@ -180,9 +181,7 @@ export default function ProxyRegistryManager() {
   const [poolOpen, setPoolOpen] = useState(false);
   const [poolScope, setPoolScope] = useState("provider");
   const [poolScopeId, setPoolScopeId] = useState("");
-  const [poolStrategy, setPoolStrategy] = useState<"round-robin" | "random" | "sticky">(
-    "round-robin"
-  );
+  const [poolStrategy, setPoolStrategy] = useState<PoolStrategy>("round-robin");
   const [poolMembers, setPoolMembers] = useState<string[]>([]);
   const [poolAddProxyId, setPoolAddProxyId] = useState("");
   const [poolLoading, setPoolLoading] = useState(false);
@@ -569,11 +568,7 @@ export default function ProxyRegistryManager() {
         ? payload.members
         : [];
       setPoolMembers(members.map((m) => m.proxyId));
-      setPoolStrategy(
-        ["round-robin", "random", "sticky"].includes(payload?.strategy)
-          ? payload.strategy
-          : "round-robin"
-      );
+      setPoolStrategy(isPoolStrategy(payload?.strategy) ? payload.strategy : "round-robin");
       setPoolLoaded(true);
     } catch (e: any) {
       setError(e?.message || t("poolLoadFailed"));
@@ -638,7 +633,7 @@ export default function ProxyRegistryManager() {
     }
   };
 
-  const handlePoolStrategyChange = async (strategy: "round-robin" | "random" | "sticky") => {
+  const handlePoolStrategyChange = async (strategy: PoolStrategy) => {
     const previous = poolStrategy;
     setPoolStrategy(strategy);
     setError(null);
@@ -1151,7 +1146,9 @@ export default function ProxyRegistryManager() {
             </div>
             {poolScope !== "global" && (
               <div>
-                <label className="text-xs text-text-muted mb-1 block">{t("poolScopeIdLabel")}</label>
+                <label className="text-xs text-text-muted mb-1 block">
+                  {t("poolScopeIdLabel")}
+                </label>
                 <input
                   className="w-full px-3 py-2 rounded bg-bg-subtle border border-border"
                   value={poolScopeId}
@@ -1183,20 +1180,20 @@ export default function ProxyRegistryManager() {
           {poolLoaded && (
             <>
               <div>
-                <label className="text-xs text-text-muted mb-1 block">{t("poolStrategyLabel")}</label>
+                <label className="text-xs text-text-muted mb-1 block">
+                  {t("poolStrategyLabel")}
+                </label>
                 <select
                   className="w-full px-3 py-2 rounded bg-bg-subtle border border-border"
                   value={poolStrategy}
-                  onChange={(e) =>
-                    handlePoolStrategyChange(
-                      e.target.value as "round-robin" | "random" | "sticky"
-                    )
-                  }
+                  onChange={(e) => handlePoolStrategyChange(e.target.value as PoolStrategy)}
                   data-testid="proxy-registry-pool-strategy"
                 >
-                  <option value="round-robin">{t("strategyRoundRobin")}</option>
-                  <option value="random">{t("strategyRandom")}</option>
-                  <option value="sticky">{t("strategySticky")}</option>
+                  {POOL_STRATEGY_OPTIONS.map((opt) => (
+                    <option key={opt.value} value={opt.value}>
+                      {t(opt.labelKey)}
+                    </option>
+                  ))}
                 </select>
                 <p className="text-xs text-text-muted mt-1">{t("poolStrategyHint")}</p>
               </div>

src/app/(dashboard)/dashboard/settings/components/proxyStrategyOptions.ts

@@ -0,0 +1,17 @@
+// Pool rotation strategy options shared by ProxyRegistryManager's pool strategy
+// selector. Extracted so the union type has a single source of truth and the
+// <option> list can be rendered from data instead of literal JSX (#6798).
+export type PoolStrategy = "round-robin" | "random" | "sticky" | "latency";
+
+export const POOL_STRATEGY_VALUES: PoolStrategy[] = ["round-robin", "random", "sticky", "latency"];
+
+export const POOL_STRATEGY_OPTIONS: Array<{ value: PoolStrategy; labelKey: string }> = [
+  { value: "round-robin", labelKey: "strategyRoundRobin" },
+  { value: "random", labelKey: "strategyRandom" },
+  { value: "sticky", labelKey: "strategySticky" },
+  { value: "latency", labelKey: "strategyLatency" },
+];
+
+export function isPoolStrategy(value: unknown): value is PoolStrategy {
+  return POOL_STRATEGY_VALUES.includes(value as PoolStrategy);
+}

src/i18n/messages/en.json

@@ -8019,11 +8019,12 @@
     "poolLoad": "Load Pool",
     "poolLoadFailed": "Failed to load the proxy pool",
     "poolStrategyLabel": "Rotation strategy",
-    "poolStrategyHint": "round-robin cycles members in order; random picks uniformly; sticky holds one member for a window before advancing.",
+    "poolStrategyHint": "round-robin cycles members in order; random picks uniformly; sticky holds one member for a window; latency-optimized picks the fastest based on logs.",
     "poolStrategyFailed": "Failed to update the rotation strategy",
     "strategyRoundRobin": "Round-robin",
     "strategyRandom": "Random",
     "strategySticky": "Sticky",
+    "strategyLatency": "Latency-optimized",
     "poolMembersLabel": "Pool members ({count})",
     "poolNoMembers": "No proxies in this pool yet.",
     "poolRemove": "Remove",

src/lib/db/proxies.ts

@@ -6,6 +6,7 @@
 import { randomUUID, randomInt } from "crypto";
 import { getDbInstance } from "./core";
 import { backupDbFile } from "./backup";
+import { pickByLatency } from "./proxyLatency";
 import type {
   JsonRecord,
   ProxyScope,
@@ -19,10 +20,7 @@ import type {
   LegacyProxyConfig,
   ProxyRotationStrategy,
 } from "./proxies/types";
-import {
-  PROXY_ROTATION_STRATEGIES,
-  DEFAULT_PROXY_ROTATION_STRATEGY,
-} from "./proxies/types";
+import { PROXY_ROTATION_STRATEGIES, DEFAULT_PROXY_ROTATION_STRATEGY } from "./proxies/types";
 import {
   mapProxyRow,
   mapAssignmentRow,
@@ -743,6 +741,8 @@ function pickFromCandidates<T>(
     return candidates[randomInt(candidates.length)];
   }
 
+  if (state.strategy === "latency") return pickByLatency(db, candidates);
+
   if (state.strategy === "sticky") {
     const windowMs = state.stickyWindowMinutes * 60_000;
     const lastRotated = state.rotatedAt ? Date.parse(state.rotatedAt) : NaN;

src/lib/db/proxies/types.ts

@@ -5,11 +5,12 @@ export type ProxyScope = "global" | "provider" | "account" | "combo";
 // to `round-robin` (monotonic persisted cursor — never Math.random). `random`
 // picks uniformly from the alive set; `sticky` holds the same member for a
 // configurable window before advancing the cursor.
-export type ProxyRotationStrategy = "round-robin" | "random" | "sticky";
+export type ProxyRotationStrategy = "round-robin" | "random" | "sticky" | "latency";
 export const PROXY_ROTATION_STRATEGIES: readonly ProxyRotationStrategy[] = [
   "round-robin",
   "random",
   "sticky",
+  "latency",
 ];
 export const DEFAULT_PROXY_ROTATION_STRATEGY: ProxyRotationStrategy = "round-robin";
 

src/lib/db/proxyLatency.ts

@@ -0,0 +1,55 @@
+// Latency-based proxy rotation strategy: picks the candidate with the lowest
+// average latency observed in `proxy_logs` over a trailing window. Extracted
+// from proxies.ts to keep that frozen god-file under its line-count cap
+// (imported directly by src/lib/db/proxies.ts, anti-barrel, #6798).
+import { getDbInstance } from "./core";
+
+const PROXY_LATENCY_WINDOW_HOURS = parseInt(process.env.PROXY_LATENCY_WINDOW_HOURS ?? "3", 10);
+
+type LatencyLogRow = {
+  proxy_host: string;
+  proxy_port: number;
+  avg_latency: number | null;
+};
+
+// Builds a `"host:port" -> avg_latency_ms` map from proxy_logs rows recorded
+// within the trailing PROXY_LATENCY_WINDOW_HOURS window.
+function buildLatencyMap(db: ReturnType<typeof getDbInstance>): Map<string, number> {
+  const sinceIso = new Date(Date.now() - PROXY_LATENCY_WINDOW_HOURS * 60 * 60 * 1000).toISOString();
+
+  const latencyRows = db
+    .prepare(
+      `SELECT proxy_host, proxy_port, AVG(latency_ms) as avg_latency
+       FROM proxy_logs
+       WHERE timestamp >= ?
+       GROUP BY proxy_host, proxy_port`
+    )
+    .all(sinceIso) as LatencyLogRow[];
+
+  const latencyMap = new Map<string, number>();
+  for (const r of latencyRows) {
+    if (r.avg_latency !== null && r.avg_latency !== undefined) {
+      latencyMap.set(`${r.proxy_host}:${r.proxy_port}`, r.avg_latency);
+    }
+  }
+  return latencyMap;
+}
+
+// Picks the candidate with the lowest recorded average latency; candidates
+// with no logged latency are treated as -1 (best/first) so untested proxies
+// still get a chance to be selected and gather data.
+export function pickByLatency<T>(db: ReturnType<typeof getDbInstance>, candidates: T[]): T {
+  const latencyMap = buildLatencyMap(db);
+
+  const sorted = [...candidates].sort((a, b) => {
+    const pA = a as { host: string; port: number };
+    const pB = b as { host: string; port: number };
+    const keyA = `${pA.host}:${pA.port}`;
+    const keyB = `${pB.host}:${pB.port}`;
+    const latA = latencyMap.has(keyA) ? latencyMap.get(keyA)! : -1;
+    const latB = latencyMap.has(keyB) ? latencyMap.get(keyB)! : -1;
+    return latA - latB;
+  });
+
+  return sorted[0];
+}

src/shared/validation/schemas/proxy.ts

@@ -14,7 +14,6 @@ import {
 } from "@/shared/constants/upstreamHeaders";
 import { MAX_TIMER_TIMEOUT_MS } from "@/shared/utils/runtimeTimeouts";
 
-
 export const proxyConfigSchema = z
   .object({
     type: z
@@ -201,6 +200,7 @@ export const PROXY_POOL_ROTATION_STRATEGY_VALUES = [
   "round-robin",
   "random",
   "sticky",
+  "latency",
 ] as const;
 
 // Add/remove one proxy to/from a scope's pool. proxyId is REQUIRED (unlike the
@@ -240,4 +240,4 @@ export const proxyRotationStrategySchema = z
         path: ["scopeId"],
       });
     }
-  });
+  });

tests/unit/check-db-rules-classification.test.ts

@@ -121,7 +121,7 @@ test("INTENTIONALLY_INTERNAL is exported from check-db-rules.mjs", () => {
   assert.ok(INTENTIONALLY_INTERNAL.size > 0, "INTENTIONALLY_INTERNAL must not be empty");
 });
 
-test("INTENTIONALLY_INTERNAL contains the expected 34 audited modules", () => {
+test("INTENTIONALLY_INTERNAL contains the expected 35 audited modules", () => {
   const expected = [
     "_rowTypes",
     "accessTokens",
@@ -148,6 +148,7 @@ test("INTENTIONALLY_INTERNAL contains the expected 34 audited modules", () => {
     "prompts",
     "providerNodeSelect",
     "providerStats",
+    "proxyLatency",
     "recovery",
     "schemaColumns",
     "secrets",

tests/unit/proxy-rotation-latency.test.ts

@@ -0,0 +1,157 @@
+/**
+ * Unit tests for latency-optimized proxy rotation strategy.
+ */
+import test from "node:test";
+import assert from "node:assert/strict";
+import fs from "node:fs";
+import os from "node:os";
+import path from "node:path";
+
+const TEST_DATA_DIR = fs.mkdtempSync(path.join(os.tmpdir(), "omniroute-proxy-latency-"));
+process.env.DATA_DIR = TEST_DATA_DIR;
+process.env.API_KEY_SECRET = "test-secret";
+process.env.PROXY_LATENCY_WINDOW_HOURS = "6"; // Set custom 6-hour window at startup
+
+const core = await import("../../src/lib/db/core.ts");
+const proxiesDb = await import("../../src/lib/db/proxies.ts");
+
+async function resetStorage() {
+  core.resetDbInstance();
+  fs.rmSync(TEST_DATA_DIR, { recursive: true, force: true });
+  fs.mkdirSync(TEST_DATA_DIR, { recursive: true });
+}
+
+let proxySeq = 0;
+async function makeProxy(host: string, port: number) {
+  proxySeq++;
+  const proxy = await proxiesDb.createProxy({
+    name: `Latency proxy ${proxySeq}`,
+    type: "http",
+    host,
+    port,
+    status: "active",
+  });
+  return proxy!;
+}
+
+function insertLog(
+  db: ReturnType<typeof core.getDbInstance>,
+  host: string,
+  port: number,
+  latencyMs: number,
+  timestampIso: string
+) {
+  db.prepare(
+    "INSERT INTO proxy_logs (id, timestamp, proxy_host, proxy_port, latency_ms) VALUES (?, ?, ?, ?, ?)"
+  ).run(`log-${Date.now()}-${Math.random()}`, timestampIso, host, port, latencyMs);
+}
+
+test.after(async () => {
+  core.resetDbInstance();
+  fs.rmSync(TEST_DATA_DIR, { recursive: true, force: true });
+});
+
+test("latency strategy chooses proxy with lowest average latency within the window", async () => {
+  await resetStorage();
+  const db = core.getDbInstance();
+
+  const p1 = await makeProxy("10.0.0.1", 8081);
+  const p2 = await makeProxy("10.0.0.2", 8082);
+
+  await proxiesDb.addProxyToScopePool("provider", "openai", p1.id);
+  await proxiesDb.addProxyToScopePool("provider", "openai", p2.id);
+  await proxiesDb.setScopeRotationStrategy("provider", "openai", "latency");
+
+  const now = Date.now();
+  // Insert logs inside the window (e.g. 2 hours ago)
+  const insideWindow = new Date(now - 2 * 60 * 60 * 1000).toISOString();
+
+  // p1 average latency: (100 + 150) / 2 = 125ms
+  insertLog(db, "10.0.0.1", 8081, 100, insideWindow);
+  insertLog(db, "10.0.0.1", 8081, 150, insideWindow);
+
+  // p2 average latency: (200 + 300) / 2 = 250ms
+  insertLog(db, "10.0.0.2", 8082, 200, insideWindow);
+  insertLog(db, "10.0.0.2", 8082, 300, insideWindow);
+
+  const resolved = await proxiesDb.resolveProxyForScopeFromRegistry("provider", "openai");
+  assert.ok(resolved);
+  assert.equal(
+    (resolved as { proxy: { host: string } }).proxy.host,
+    "10.0.0.1",
+    "Should pick the one with lower average latency"
+  );
+});
+
+test("latency strategy prioritizes untested proxies (no logs)", async () => {
+  await resetStorage();
+  const db = core.getDbInstance();
+
+  const p1 = await makeProxy("10.0.0.1", 8081);
+  const p2 = await makeProxy("10.0.0.2", 8082); // untested
+
+  await proxiesDb.addProxyToScopePool("provider", "openai", p1.id);
+  await proxiesDb.addProxyToScopePool("provider", "openai", p2.id);
+  await proxiesDb.setScopeRotationStrategy("provider", "openai", "latency");
+
+  // Insert log for p1 only
+  const insideWindow = new Date(Date.now() - 2 * 60 * 60 * 1000).toISOString();
+  insertLog(db, "10.0.0.1", 8081, 50, insideWindow);
+
+  const resolved = await proxiesDb.resolveProxyForScopeFromRegistry("provider", "openai");
+  assert.ok(resolved);
+  assert.equal(
+    (resolved as { proxy: { host: string } }).proxy.host,
+    "10.0.0.2",
+    "Should prioritize untested proxy over tested one"
+  );
+});
+
+test("latency strategy ignores logs outside the configured time window", async () => {
+  await resetStorage();
+  const db = core.getDbInstance();
+
+  const p1 = await makeProxy("10.0.0.1", 8081);
+  const p2 = await makeProxy("10.0.0.2", 8082);
+
+  await proxiesDb.addProxyToScopePool("provider", "openai", p1.id);
+  await proxiesDb.addProxyToScopePool("provider", "openai", p2.id);
+  await proxiesDb.setScopeRotationStrategy("provider", "openai", "latency");
+
+  const now = Date.now();
+
+  // p1 has 300ms latency log inside 6h window (e.g. 2 hours ago)
+  const insideWindow = new Date(now - 2 * 60 * 60 * 1000).toISOString();
+  insertLog(db, "10.0.0.1", 8081, 300, insideWindow);
+
+  // p2 has 50ms latency log but OUTSIDE 6h window (e.g. 8 hours ago)
+  const outsideWindow = new Date(now - 8 * 60 * 60 * 1000).toISOString();
+  insertLog(db, "10.0.0.2", 8082, 50, outsideWindow);
+
+  // Since p2's log is outside the 6h window, p2 is considered untested within the window.
+  // Untested proxies are prioritized (score -1) over tested ones (score 300).
+  const resolved = await proxiesDb.resolveProxyForScopeFromRegistry("provider", "openai");
+  assert.ok(resolved);
+  assert.equal(
+    (resolved as { proxy: { host: string } }).proxy.host,
+    "10.0.0.2",
+    "p2 should be prioritized as untested within the 6h window"
+  );
+});
+
+test("latency strategy works normally with empty proxy_logs table", async () => {
+  await resetStorage();
+  const p1 = await makeProxy("10.0.0.1", 8081);
+  const p2 = await makeProxy("10.0.0.2", 8082);
+
+  await proxiesDb.addProxyToScopePool("provider", "openai", p1.id);
+  await proxiesDb.addProxyToScopePool("provider", "openai", p2.id);
+  await proxiesDb.setScopeRotationStrategy("provider", "openai", "latency");
+
+  // No logs inserted, both are untested. Should return one of them without crash.
+  const resolved = await proxiesDb.resolveProxyForScopeFromRegistry("provider", "openai");
+  assert.ok(resolved);
+  assert.ok(
+    ["10.0.0.1", "10.0.0.2"].includes((resolved as { proxy: { host: string } }).proxy.host)
+  );
+});