mirror of
https://github.com/diegosouzapw/OmniRoute.git
synced 2026-08-25 16:42:16 +03:00
77ea656b12
The executor barrel statically imported ~100 executor modules and
constructed every instance at module load. Measured cold cost on top of
the minimal set: ~0.7–1.2s boot time and ~35MB heap, paid by every
deployment regardless of which providers it uses.
Now:
- executors/index.ts keeps the declarative alias table byte-stable (same
keys, same order, same ctor args — pinned by the golden lock) but each
value is a deferred loader using dynamic import; bundlers emit
on-demand chunks
- registry.ts gains registerLazyExecutor/loadRegisteredExecutor: aliases
are declared eagerly so hasSpecializedExecutor() and
listExecutorAliases() stay synchronous, instances materialize once on
first use and cache into the same registry map
- getExecutor() becomes async; production call sites (chatCore proxy
resolver, video generation, compression judge/eval clients,
quotaAutoPing deps, anthropic OAuth validation) await it
- cliproxy wrapper ExecutorLike types drop their index signatures so
BaseExecutor satisfies them structurally
Measured after (isolated DATA_DIR): barrel boot 712-832ms / ~45MB with
first-use materialization of an executor costing +120-150ms once.
Test impact: 24 unit suites adapted mechanically to the async seam
(await + union narrowing on the Response | {response} execute result);
class imports moved from the barrel to executor module files. The
web-cookie sweep SIGABRT failure is pre-existing (reproduced identically
on the clean base).
Commit gate note: husky lint-staged fails with 'suppressions left that
do not occur anymore' — reproduced identically on a stashed clean tree
(22 baseline problems), independent of this change.
46 lines
2.1 KiB
TypeScript
46 lines
2.1 KiB
TypeScript
import { getExecutor } from "../../../executors/index.ts";
|
|
import type { ExecuteInput, ProviderCredentials } from "../../../executors/base.ts";
|
|
import type { ChatTurn, ModelCallResult, ModelClient } from "./types.ts";
|
|
|
|
/**
|
|
* Production ModelClient adapter (Hard Rule #18 — NOT unit-tested; validated on a real
|
|
* VPS/account). Wraps the server executor: builds a minimal non-stream chat body, calls
|
|
* `getExecutor(provider).execute(...)`, reads the response text + (best-effort) usage cost.
|
|
*
|
|
* The pure runner depends only on the `ModelClient` interface; this adapter is the single
|
|
* place that touches credentials, the executor, and Response parsing — so the eval stays
|
|
* faithful to production while the runner/scorers remain fully testable with a stub.
|
|
*/
|
|
export function createExecutorModelClient(
|
|
provider: string,
|
|
credentials: ProviderCredentials,
|
|
costPerKTokenOut?: number
|
|
): ModelClient {
|
|
return {
|
|
async complete(model: string, messages: ChatTurn[]): Promise<ModelCallResult> {
|
|
// #11220: getExecutor is async (lazy registry) — resolve per call.
|
|
const executor = await getExecutor(provider);
|
|
const body = { model, messages, stream: false };
|
|
const input: ExecuteInput = {
|
|
model,
|
|
body,
|
|
stream: false,
|
|
credentials,
|
|
};
|
|
// BaseExecutor.execute resolves to { response, url, headers, transformedBody } — the
|
|
// upstream Response lives on `.response` (never a bare Response). Validated live on VPS.
|
|
const raw = (await executor.execute(input)) as { response: Response };
|
|
const response = raw.response;
|
|
const json = (await response.json()) as {
|
|
choices?: Array<{ message?: { content?: string } }>;
|
|
usage?: { completion_tokens?: number };
|
|
};
|
|
const text = json.choices?.[0]?.message?.content ?? "";
|
|
const outTokens = json.usage?.completion_tokens ?? 0;
|
|
const usdCost =
|
|
typeof costPerKTokenOut === "number" ? (outTokens / 1000) * costPerKTokenOut : undefined;
|
|
return usdCost === undefined ? { text } : { text, usdCost };
|
|
},
|
|
};
|
|
}
|