From 4150167a630c61e4f064f419d503b0ef5b1628f5 Mon Sep 17 00:00:00 2001 From: Hermes Developer Date: Sat, 22 Aug 2026 11:55:05 -0400 Subject: [PATCH 1/2] feat(routing): subscription-first auto groupings (auto/subscription, auto/thrifty) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit OmniRoute answers "is this model free?" (hidePaidModels) and "can this connection ever bill me?" (STRICT_ZERO_COST), but both fail closed and every paid-side mechanism (cost-optimized, budgetCap, the cost-saver mode pack) is tier-agnostic. Nothing answers "use the plan quota I already pay for; when it runs out either stop, or step up one rung at a time; and come back when it resets." The blocker was that billing is a property of the CONNECTION, not the model: classifyTier() keys on (provider, model), while the same model is plan-included through an OAuth connection and metered through an API-key one. auth_type is not a safe proxy in either direction. So this adds a curated per-connection billing catalog, hand-set from published terms, following the same pattern FreeModelBudget.hardStopGuaranteed already established. Uncurated resolves to unknown and is consumed as metered, so new providers start outside the subscription rung. Two ids, sharing one rung model (subscription > keyless > free > cheap > premium): - auto/subscription: rung 0 only, hard-stop overage only, live quota verified per connection. Fails CLOSED — an empty pool is the intended answer. - auto/thrifty: all rungs ordered, exhausted ones gated out, scoring still runs within the survivors. Fails OPEN one rung at a time. Both reuse STRICT_ZERO_COST's connection-safety invariant: each connection in allowedConnectionIds is verified individually and the array is rewritten to the surviving subset, so autoStrategy.ts can only dispatch to a verified account. Reset re-entry: a cached quota reading whose own resetAt has passed is now stale regardless of TTL, and rung eligibility is recomputed per pool build with no persisted demotion that could outlive a reset. clampCooldownToReset() is implemented and tested but not yet wired — the quota cache is invalidated in auth.ts before any cooldown is written, so resetAt must be captured earlier there; that hot-path change belongs in its own PR. Both ids are opt-in by being requested; no existing pool, strategy or default changes. Settings are tuning-only on purpose (no enabled flag that could leave auto/subscription silently serving paid capacity). Also corrects docs/guides/TIERS.md, which advertised a combo strategy named "subscription" that has never existed in ROUTING_STRATEGY_VALUES. --- docs/guides/TIERS.md | 7 +- docs/routing/SUBSCRIPTION_LADDER.md | 181 ++++++++ open-sse/config/connectionBillingCatalog.ts | 143 ++++++ open-sse/services/autoCombo/builtinCatalog.ts | 30 +- .../services/autoCombo/connectionBilling.ts | 100 +++++ .../services/autoCombo/freeAccessQuota.ts | 9 +- .../services/autoCombo/subscriptionLadder.ts | 411 ++++++++++++++++++ .../services/autoCombo/suffixComposition.ts | 23 +- open-sse/services/autoCombo/virtualFactory.ts | 134 +++++- src/shared/validation/settingsSchemas.ts | 18 + .../autoCombo/subscription-ladder.test.ts | 407 +++++++++++++++++ 11 files changed, 1452 insertions(+), 11 deletions(-) create mode 100644 docs/routing/SUBSCRIPTION_LADDER.md create mode 100644 open-sse/config/connectionBillingCatalog.ts create mode 100644 open-sse/services/autoCombo/connectionBilling.ts create mode 100644 open-sse/services/autoCombo/subscriptionLadder.ts create mode 100644 tests/unit/autoCombo/subscription-ladder.test.ts diff --git a/docs/guides/TIERS.md b/docs/guides/TIERS.md index bb46d7f4ed..c79b012b12 100644 --- a/docs/guides/TIERS.md +++ b/docs/guides/TIERS.md @@ -24,8 +24,11 @@ it expires. | Antigravity / Devin Desktop | Built-in quotas | **Strategy**: route here first for every request that fits the model's -strengths. Quota tracker monitors approaching reset; combo strategies -`reset-aware` and `subscription` prioritize accordingly. +strengths. The quota tracker monitors approaching resets, and the `reset-aware` +combo strategy prioritizes accordingly. To route Tier 1 first and only step out +to paid tiers as quota runs out, use the `auto/thrifty` id — or `auto/subscription` +to stay on plan-included capacity and fail closed instead. See +[Subscription-first routing](../routing/SUBSCRIPTION_LADDER.md). ## Tier 2 — Cheap diff --git a/docs/routing/SUBSCRIPTION_LADDER.md b/docs/routing/SUBSCRIPTION_LADDER.md new file mode 100644 index 0000000000..2018557023 --- /dev/null +++ b/docs/routing/SUBSCRIPTION_LADDER.md @@ -0,0 +1,181 @@ +--- +title: "Subscription-first routing" +version: 3.8.50 +lastUpdated: 2026-08-22 +--- + +# Subscription-first routing + +> Two new `auto/*` ids — `auto/subscription` and `auto/thrifty`. Both are opt-in by being +> requested: nothing routes through them unless a caller asks for the id by name, and no +> existing pool, strategy, or default changes. + +## Why this exists + +OmniRoute already answers two cost questions, and neither is the one most operators ask. + +| Existing mechanism | Answers | +| -------------------------------------------------------- | ----------------------------------- | +| `hidePaidModels` (`autoCombo/paidModelFilter.ts`) | "is this model catalogued free?" | +| `freeAccessPolicy: "strict"` (`strictZeroCostFilter.ts`) | "can this connection ever bill me?" | +| `quotaPreflight` (`combo/quotaExhaustionCutoff.ts`) | "is this connection near its wall?" | +| `budgetCap` / `budgetFallback` (`autoCombo/engine.ts`) | "cap spend, degrade to cheapest" | + +Every free-only mechanism **fails closed** — an exhausted free pool is an empty pool, never a +step up to a paid option — and every paid-side mechanism is tier-agnostic. Neither answers: + +> "Use the quota I already pay for. When it runs out, either stop, or step up one rung at a +> time through the cheapest paid options — and come back the moment it resets." + +## Billing is a connection fact, not a model fact + +`classifyTier()` (`open-sse/services/tierResolver.ts`) keys on `(provider, model)` and returns +`free | cheap | premium` from catalog pricing. But whether a request costs incremental money +depends on **which connection serves it**: the same model is plan-included through a Claude Code +OAuth connection and billed per token through an API-key connection. + +`provider_connections.auth_type` is not a safe proxy in either direction — metered OAuth +connections exist, and plan-included API-key connections exist (a Copilot seat token is not a +metered API key). So billing class comes from a **curated catalog**, +`open-sse/config/connectionBillingCatalog.ts`, hand-set from each provider's published terms — +the same pattern `FreeModelBudget.hardStopGuaranteed` already established for free models. + +```ts +type ConnectionBillingClass = "subscription" | "metered" | "keyless" | "unknown"; +type ConnectionOverageBehavior = "hard-stop" | "meters-to-paid" | "unknown"; +``` + +Resolution order (`autoCombo/connectionBilling.ts`): the synthetic no-auth sentinel → +`keyless`; a catalog entry matching provider **and** `authType`; a provider-wide entry; +otherwise `unknown`. **Uncurated is not free** — `unknown` is consumed as `metered` +everywhere, so a provider added tomorrow starts outside the subscription rung and has to be +curated in deliberately. + +## The rung model + +Five rungs in escalation order. They differ in more than price — each has its **own** +exhaustion signal, which is why this is not merely a sort. + +| # | Rung | Membership | Exhausted when | +| --- | -------------- | -------------------------------------------------- | ----------------------------- | +| 0 | `subscription` | curated `billing: "subscription"` | quota window at/below cutoff | +| 1 | `keyless` | the synthetic no-auth path | connection cooldown / breaker | +| 2 | `free` | metered connection, `classifyTier() === "free"` | free allowance exhausted | +| 3 | `cheap` | metered connection, `classifyTier() === "cheap"` | per-rung budget consumed | +| 4 | `premium` | metered connection, `classifyTier() === "premium"` | per-rung budget consumed | + +Rungs 0-2 exhaust on **quota**, which is observable and already tracked. Rungs 3-4 have no +quota — a paid connection serves forever — so their only sane exhaustion signal is a per-rung +**budget**. Without one, "escalate when cheap is exhausted" has no trigger. + +## `auto/subscription` — fail closed + +Pool = rung 0 only, restricted to connections whose overage is a documented `hard-stop`, each +verified live to have quota headroom. Everything ambiguous is excluded: an uncurated provider, +an unverifiable quota reading, a stale reading, or an overage that meters to paid. + +An empty pool is the **intended** answer, not a defect — the caller's existing empty-pool path +turns it into a clear error rather than a silent, billable fallback. That is the whole promise +of the id. + +`keyless` deliberately does **not** qualify: this grouping means "the plan I pay for", so a +no-auth backend does not belong in it. Use `auto/thrifty` (or `auto/best-free`) for that. + +### Connection safety + +A candidate is not always tied to one connection — a logical candidate carries an +`allowedConnectionIds` allowlist, and the account actually used is chosen later, at dispatch, +by `open-sse/services/combo/autoStrategy.ts`. Both groupings therefore verify **each connection +individually** and rewrite `allowedConnectionIds` down to exactly the surviving subset — never +the full original list, never one arbitrarily-chosen member. Because `autoStrategy.ts` already +enforces that array as a hard allowlist, rewriting it here makes "verified" and "actually used" +the same set by construction. This is the same invariant, and the same reasoning, as +[STRICT_ZERO_COST](./STRICT_ZERO_COST.md). + +## `auto/thrifty` — escalate one rung at a time + +Pool = all rungs, ordered by rung index, with exhausted candidates gated out. The `auto` engine +still scores **within** the surviving pool: the ladder decides which rungs are in play, scoring +decides which candidate wins inside them. Ordering is stable within a rung, so the scorer's own +ranking is never reshuffled by this overlay. + +This is an ordering + gating overlay, **not** a new dispatcher: `combo.ts`'s speculative loop +already walks targets in order and falls through on failure, so a runtime exhaustion the +preflight did not catch still escalates to the next rung inside the same request. + +Where `auto/subscription` fails closed, `auto/thrifty` fails **open**: a plan-included +connection with no usable quota reading is still tried first. Trying it costs nothing, and if +it turns out to be exhausted the fall-through reaches the next rung anyway — whereas refusing +to try it would send the request to a paid rung on missing telemetry, the exact outcome the +grouping exists to avoid. + +## Returning to the plan after a reset + +Three independent things must expire before routing returns to rung 0. Fixing only one leaves +the ladder stuck on paid rungs long after the plan refilled. + +1. **The quota-state cache** — `freeAccessQuota.ts` caches per `(provider, connection)` with a + 180s TTL. A cached entry whose own `resetAt` has already passed describes a window that no + longer exists, so it is now treated as stale **regardless of age** and forces a refresh. + Without this, a plan that refilled at midnight keeps reading exhausted until the TTL happens + to lapse. +2. **The ladder's own state** — there is none, by design. Rung eligibility is recomputed from + live quota state on every pool build; no persisted "currently on rung 3" record exists that + could outlive a reset and wedge routing. +3. **The connection cooldown** — the exhausting 429 sets `rateLimitedUntil` from exponential + backoff, which for a plan connection can overshoot the real reset. `clampCooldownToReset()` + (`subscriptionLadder.ts`) narrows a cooldown to the upstream's own reset instant and can + never extend one. **It is implemented and tested but not yet wired**: the quota cache is + invalidated in `src/sse/services/auth.ts` _before_ any cooldown is written, so `resetAt` + must be captured earlier in that function — a change to the resilience hot path that + belongs in its own reviewed PR. Until then, re-entry waits out the connection cooldown + (which already prefers upstream `Retry-After` hints when the provider sends them). + +### Anti-flap + +A rung that just reset is re-admitted only above `reentryMinRemainingPercent` (default 5), +while a connection already in play only has to stay above `exitCutoffPercent` (default 2, +matching `quotaPreflight.defaultThresholdPercent`). The gap is the hysteresis band — without +it, a connection hovering at the cutoff oscillates between rungs on consecutive requests. + +## Configuration + +Tuning only. There is deliberately **no** `enabled` flag: a toggle able to switch these off +would leave `auto/subscription` quietly serving the full pool — paid models included — under a +name that promises the opposite. + +```jsonc +{ + "subscriptionLadder": { + "exitCutoffPercent": 2, + "reentryMinRemainingPercent": 5, + "rungBudgetUsd": { "cheap": 5.0, "premium": 0 }, // 0 disables a rung outright + }, +} +``` + +Budget gating is inert until a spend resolver is wired: with no accounting available a paid +rung is ordered but never gated. Rung ordering, quota-based exhaustion, and reset re-entry all +work without it. + +## Composition + +`subscription` and `thrifty` are `AutoTier` values, so they compose with every category: +`auto/coding:thrifty`, `auto/reasoning:subscription`, and so on. The two flat ids +(`auto/subscription`, `auto/thrifty`) are advertised in `/v1/models` and the dashboard. + +Neither id is paid-tier, so `isPaidTierAutoId()` returns `false` for both and +`auto/subscription` survives `hidePaidModels`. + +## Where the code lives + +| Concern | File | +| ------------------------------- | --------------------------------------------------- | +| Curated billing facts | `open-sse/config/connectionBillingCatalog.ts` | +| Classifier | `open-sse/services/autoCombo/connectionBilling.ts` | +| Rungs, both groupings, re-entry | `open-sse/services/autoCombo/subscriptionLadder.ts` | +| Wiring into the candidate pool | `open-sse/services/autoCombo/virtualFactory.ts` | +| Reset-aware cache staleness | `open-sse/services/autoCombo/freeAccessQuota.ts` | +| Tier surface | `open-sse/services/autoCombo/suffixComposition.ts` | +| Advertised ids | `open-sse/services/autoCombo/builtinCatalog.ts` | +| Tests | `tests/unit/autoCombo/subscription-ladder.test.ts` | diff --git a/open-sse/config/connectionBillingCatalog.ts b/open-sse/config/connectionBillingCatalog.ts new file mode 100644 index 0000000000..7bab61f6db --- /dev/null +++ b/open-sse/config/connectionBillingCatalog.ts @@ -0,0 +1,143 @@ +/** + * Curated billing classification for provider CONNECTIONS. + * + * The economic tier resolver (`open-sse/services/tierResolver.ts`) answers + * "how much does this MODEL cost per token?" — a fact about the catalog. It + * cannot answer the question subscription-first routing actually needs: + * + * "does serving this request through THIS connection cost incremental money, + * or is it already covered by a flat-rate plan the operator pays anyway?" + * + * That is a property of the credential, not the model: `claude/claude-*` is + * plan-included through a Claude Code OAuth connection and billed per token + * through an API-key connection — same provider, same model, opposite + * economics. + * + * `provider_connections.auth_type` alone is NOT a safe proxy in either + * direction: metered OAuth connections exist (cloud-billed OAuth), and + * plan-included API-key connections exist (Copilot seat tokens). So this is a + * CURATED table, hand-set from each provider's published terms — deliberately + * the same pattern `FreeModelBudget.hardStopGuaranteed` + * (`open-sse/config/freeModelCatalog.ts`) already established: a fact about + * the upstream's commercial terms, never derived from `authType` and never + * inferred from a live API response. + * + * Uncurated is not "free": anything absent here resolves to `unknown`, which + * every consumer treats as `metered`. New providers therefore start OUTSIDE + * the subscription rung and have to be curated in deliberately — the same + * fail-safe direction STRICT_ZERO_COST uses for uncatalogued models. + */ + +/** + * How a connection's upstream charges for the requests it serves. + * + * - `subscription` — covered by a flat-rate plan the operator already pays. + * Consuming its quota costs nothing incremental; the plan is a sunk cost. + * - `metered` — pay-per-token / pay-per-credit. Every request adds spend. + * - `keyless` — no credential exists at all, so no request can be billed by + * construction (the synthetic no-auth path). + * - `unknown` — not curated. Consumed as `metered` everywhere. + */ +export type ConnectionBillingClass = "subscription" | "metered" | "keyless" | "unknown"; + +/** + * What happens when a subscription plan's allowance runs out. + * + * - `hard-stop` — the upstream refuses further requests until the window + * resets. Exhaustion cannot cost money, so such a connection is admissible + * to the strictest "never spend a cent extra" grouping. + * - `meters-to-paid` — the upstream keeps serving and bills the overage. + * Perfectly usable while quota remains, never admissible to the strict + * grouping. + * - `unknown` — not established. Treated exactly like `meters-to-paid` by + * every consumer; it is the conservative default for a provider whose terms + * allow an operator to opt into usage-based billing past the plan. + */ +export type ConnectionOverageBehavior = "hard-stop" | "meters-to-paid" | "unknown"; + +export interface ConnectionBillingEntry { + /** Provider id as registered in `open-sse/config/providers/registry/`. */ + provider: string; + /** + * Restricts the entry to connections whose `authType` matches. Omit for a + * provider-wide entry. A matching auth-typed entry always wins over the + * provider-wide one, so a provider offering both a plan-included OAuth login + * and a metered API key can declare both. + */ + authType?: string; + billing: ConnectionBillingClass; + overage: ConnectionOverageBehavior; + /** Operator-visible justification for the classification. */ + reason: string; +} + +/** + * Curated entries. Conservative by design — a provider whose terms let the + * operator enable usage-based billing past the plan is recorded as `unknown` + * overage, not `hard-stop`, because the strict grouping's entire promise is + * that it cannot surprise you. + * + * Providers already classified free by the economic tier resolver (`kiro`, + * `qoder`, and the rest of `LEGACY_FREE_PROVIDERS` / + * `deriveNoAuthFreeProviders()` in `open-sse/services/tierConfig.ts`) are + * deliberately NOT listed here: they land on the ladder's `free` rung through + * `classifyTier()` and would only be double-claimed by an entry here. + */ +export const CONNECTION_BILLING_CATALOG: readonly ConnectionBillingEntry[] = [ + { + provider: "claude", + authType: "oauth", + billing: "subscription", + overage: "hard-stop", + reason: + "Claude Code OAuth serves the operator's Anthropic Pro/Max plan windows. " + + "Exceeding a window is refused until it resets; no per-token charge accrues.", + }, + { + provider: "codex", + authType: "oauth", + billing: "subscription", + overage: "hard-stop", + reason: + "Codex OAuth serves the ChatGPT plan's included Codex quota. Exhaustion is " + + "refused until the plan window resets rather than billed.", + }, + { + provider: "antigravity", + authType: "oauth", + billing: "subscription", + overage: "hard-stop", + reason: + "Antigravity OAuth serves built-in plan quotas that stop serving once consumed; " + + "OmniRoute already tracks their reset windows (see antigravityCredits.ts).", + }, + { + provider: "cursor", + authType: "oauth", + billing: "subscription", + overage: "unknown", + reason: + "Cursor Pro includes a request allowance, but usage-based pricing past the plan " + + "can be enabled per account and OmniRoute cannot observe that setting. Recorded " + + "as unknown overage so the strict grouping excludes it.", + }, + { + provider: "copilot-web", + authType: "apikey", + billing: "subscription", + overage: "unknown", + reason: + "GitHub Copilot is a per-seat subscription (the credential is a seat token, not a " + + "metered API key), but additional premium requests can be billed when the account " + + "opts in. Recorded as unknown overage.", + }, + { + provider: "devin-desktop", + authType: "oauth", + billing: "subscription", + overage: "meters-to-paid", + reason: + "Devin Desktop draws on the plan's included ACUs and continues billing past them, " + + "so it is plan-included while quota remains but never overage-safe.", + }, +]; diff --git a/open-sse/services/autoCombo/builtinCatalog.ts b/open-sse/services/autoCombo/builtinCatalog.ts index 1f759d5c28..8ec09d9c03 100644 --- a/open-sse/services/autoCombo/builtinCatalog.ts +++ b/open-sse/services/autoCombo/builtinCatalog.ts @@ -42,6 +42,12 @@ export const AUTO_TEMPLATE_VARIANTS: Record = { "auto/claude-opus": "smart", "auto/claude-sonnet": "coding", "auto/best-free": "cheap", + // Subscription-first routing (see `subscriptionLadder.ts`). `auto/subscription` + // maps to no weight variant on purpose: its pool is already restricted to + // plan-included connections, so the scorer should rank them on merit rather + // than bias toward cheap/fast within an allowance the operator already paid for. + "auto/subscription": undefined, + "auto/thrifty": "cheap", // Chaos mode — parallel dispatch to top-N stable models "auto/best-chaos": "chaos", "auto/chaos": "chaos", @@ -65,6 +71,18 @@ export const AUTO_SUFFIX_VARIANTS: string[] = [ "auto/multimodal", ]; +/** + * Flat `auto/*` ids that carry a tier overlay even though they are not written + * in `:` form. `auto/best-free` established the pattern; the + * two subscription-first ids reuse it so a caller can ask for the behavior + * without also having to pick a category. + */ +export const FLAT_TIER_OVERLAY_IDS: Record = { + "auto/best-free": "free", + "auto/subscription": "subscription", + "auto/thrifty": "thrifty", +}; + type ResolvedAutoVariant = { recognized: true; variant: AutoVariant | undefined } | { recognized: false }; @@ -119,8 +137,7 @@ export function isPaidTierAutoId(autoId: string): boolean { * a candidate filter so the virtual combo only scores vision-capable models. */ export type BuiltinAutoSpec = - | { variant: AutoVariant | undefined } - | { category: AutoCategory; tier?: AutoTier }; + { variant: AutoVariant | undefined } | { category: AutoCategory; tier?: AutoTier }; /** * Vision-flavored flat ids that MUST resolve to the `vision` category (candidate @@ -193,8 +210,9 @@ export async function createBuiltinAutoCombo( } if ("variant" in spec && spec.variant !== undefined) { + const overlayTier = FLAT_TIER_OVERLAY_IDS[modelStr]; const virtualCombo = await materialize(spec.variant, { - ...(modelStr === "auto/best-free" ? { tier: "free" as const } : {}), + ...(overlayTier ? { tier: overlayTier } : {}), }); virtualCombo.name = modelStr; virtualCombo.id = modelStr; @@ -205,7 +223,11 @@ export async function createBuiltinAutoCombo( // auto/best-chat, auto/pro-chat) still materialize via the default // (unconstrained) virtual combo rather than throwing "Unknown built-in". if (Object.prototype.hasOwnProperty.call(AUTO_TEMPLATE_VARIANTS, modelStr)) { - const virtualCombo = await materialize(undefined); + const overlayTier = FLAT_TIER_OVERLAY_IDS[modelStr]; + const virtualCombo = await materialize( + undefined, + overlayTier ? { tier: overlayTier } : undefined + ); virtualCombo.name = modelStr; virtualCombo.id = modelStr; return virtualCombo; diff --git a/open-sse/services/autoCombo/connectionBilling.ts b/open-sse/services/autoCombo/connectionBilling.ts new file mode 100644 index 0000000000..f005c057e1 --- /dev/null +++ b/open-sse/services/autoCombo/connectionBilling.ts @@ -0,0 +1,100 @@ +/** + * Pure classifier over the curated connection-billing catalog + * (`open-sse/config/connectionBillingCatalog.ts`). + * + * Kept dependency-light on purpose — the same constraint `paidModelFilter.ts` + * and `strictZeroCostFilter.ts` state in their own headers — so subscription + * routing is unit-testable without seeding the DB or the virtual factory. No + * provider name appears in this file: a connection is classified purely from + * the catalog plus the two facts the caller already has (`provider`, + * `authType`), so curating a new provider needs no code change here. + */ +import { + CONNECTION_BILLING_CATALOG, + type ConnectionBillingClass, + type ConnectionBillingEntry, + type ConnectionOverageBehavior, +} from "@omniroute/open-sse/config/connectionBillingCatalog.ts"; +import { SYNTHETIC_NOAUTH_CONNECTION_ID } from "./resilienceCandidateFilter"; + +/** The minimum a caller must know about a connection to classify it. */ +export interface BillableConnection { + provider: string; + /** `provider_connections.auth_type` — `oauth` / `apikey` / `cookie` / … */ + authType?: string | null; + /** Connection id; the synthetic no-auth sentinel classifies as `keyless`. */ + connectionId?: string | null; +} + +export interface ConnectionBillingVerdict { + billing: ConnectionBillingClass; + overage: ConnectionOverageBehavior; + reason: string; +} + +const UNKNOWN_VERDICT: ConnectionBillingVerdict = { + billing: "unknown", + overage: "unknown", + reason: "No curated billing entry for this provider/authType — assumed metered.", +}; + +const KEYLESS_VERDICT: ConnectionBillingVerdict = { + billing: "keyless", + overage: "hard-stop", + reason: + "Synthetic no-auth connection: no credential exists, so no request against it can be billed.", +}; + +/** + * Classify one connection. + * + * Resolution order, first match wins: + * 1. the synthetic no-auth sentinel → `keyless` (no credential can be billed); + * 2. a catalog entry matching BOTH provider and `authType`; + * 3. a provider-wide catalog entry (no `authType` declared); + * 4. otherwise `unknown`. + * + * `unknown` is never treated as free by any caller — `isPlanIncluded()` below + * returns false for it, so an uncurated provider stays outside the + * subscription rung until someone curates it deliberately. + */ +export function classifyConnectionBilling( + connection: BillableConnection, + catalog: readonly ConnectionBillingEntry[] = CONNECTION_BILLING_CATALOG +): ConnectionBillingVerdict { + if (connection.connectionId === SYNTHETIC_NOAUTH_CONNECTION_ID) return KEYLESS_VERDICT; + + const provider = connection.provider; + if (!provider) return UNKNOWN_VERDICT; + + const providerEntries = catalog.filter((entry) => entry.provider === provider); + if (providerEntries.length === 0) return UNKNOWN_VERDICT; + + const authType = typeof connection.authType === "string" ? connection.authType : null; + const authMatch = authType + ? providerEntries.find((entry) => entry.authType === authType) + : undefined; + const entry = authMatch ?? providerEntries.find((entry) => entry.authType === undefined); + if (!entry) return UNKNOWN_VERDICT; + + return { billing: entry.billing, overage: entry.overage, reason: entry.reason }; +} + +/** + * True when serving a request through this connection consumes an allowance + * the operator already pays for, rather than adding incremental spend. + * `keyless` qualifies: it costs nothing by construction. + */ +export function isPlanIncluded(verdict: ConnectionBillingVerdict): boolean { + return verdict.billing === "subscription" || verdict.billing === "keyless"; +} + +/** + * True when exhausting this connection's allowance cannot start costing money. + * The strict `auto/subscription` grouping admits nothing else: an operator who + * asked never to spend extra must not be surprised by a provider that meters + * past the plan, nor by one whose terms simply are not established. + */ +export function isOverageSafe(verdict: ConnectionBillingVerdict): boolean { + return verdict.overage === "hard-stop"; +} diff --git a/open-sse/services/autoCombo/freeAccessQuota.ts b/open-sse/services/autoCombo/freeAccessQuota.ts index 515ec57a87..2409c278f1 100644 --- a/open-sse/services/autoCombo/freeAccessQuota.ts +++ b/open-sse/services/autoCombo/freeAccessQuota.ts @@ -22,6 +22,7 @@ import { import { getCachedProviderConnections } from "@/lib/db/readCache"; import { defaultLogger as log } from "@omniroute/open-sse/utils/logger"; import type { FreeAccessState } from "./strictZeroCostFilter"; +import { isStateStaleForReset } from "./subscriptionLadder"; const USAGE_FETCHER_PROVIDER_SET = new Set(USAGE_FETCHER_PROVIDERS); @@ -191,7 +192,13 @@ export function resolveFreeAccessState( const key = cacheKey(provider, connectionId); const entry = cache.get(key); - const fresh = entry && Date.now() - entry.fetchedAtMs <= ttlMs(); + // Subscription-first routing (decision 3): an entry whose own `resetAt` has + // already passed describes a quota window that no longer exists, so it is + // stale REGARDLESS of its age. Without this, a plan that refilled at + // midnight keeps reading EXHAUSTED until the TTL happens to lapse, and + // routing stays on paid rungs for no reason. See `subscriptionLadder.ts`. + const resetElapsed = entry !== undefined && isStateStaleForReset(entry.state); + const fresh = entry && !resetElapsed && Date.now() - entry.fetchedAtMs <= ttlMs(); if (!fresh) { void refresh(provider, connectionId); } diff --git a/open-sse/services/autoCombo/subscriptionLadder.ts b/open-sse/services/autoCombo/subscriptionLadder.ts new file mode 100644 index 0000000000..3941201e77 --- /dev/null +++ b/open-sse/services/autoCombo/subscriptionLadder.ts @@ -0,0 +1,411 @@ +/** + * Subscription-first routing: the rung model, its two groupings, and the + * reset re-entry rules. + * + * OmniRoute already answers "is this model free?" (`paidModelFilter.ts`) and + * "can this connection ever bill me?" (`strictZeroCostFilter.ts`). Both fail + * CLOSED — an exhausted free pool is an empty pool, never a step up to a paid + * option. And every paid-side mechanism (`cost-optimized`, `budgetCap`, + * the `cost-saver` mode pack) is tier-agnostic. Neither side answers: + * + * "use the quota I already pay for; when it runs out either stop, or step up + * one rung at a time; and come back the moment it resets." + * + * This module supplies both halves of that, sharing one rung model: + * + * - `filterSubscriptionOnlyCandidates` — the strict grouping (`auto/subscription`). + * Rung 0 only, overage-safe connections only, verified live. Fails CLOSED. + * - `orderPoolByRung` — the escalating grouping (`auto/thrifty`). All rungs, + * ordered, with exhausted rungs gated out. Fails OPEN, one rung at a time. + * + * Design mirrors `strictZeroCostFilter.ts` deliberately: pure functions, the + * live quota lookup injected as a synchronous resolver, no DB or network + * import, and the SAME connection-safety invariant — every connection in a + * candidate's `allowedConnectionIds` is verified INDIVIDUALLY and the array is + * rewritten to exactly the surviving subset, never the full original list. + * `autoStrategy.ts` enforces `allowedConnectionIds` as a hard allowlist before + * selecting a connection at dispatch, so rewriting it here is sufficient to + * make "verified" and "actually used" the same set by construction. + */ +import { + classifyConnectionBilling, + isOverageSafe, + type BillableConnection, +} from "./connectionBilling"; +import type { ConnectionBillingEntry } from "@omniroute/open-sse/config/connectionBillingCatalog.ts"; +import type { FreeAccessState } from "./strictZeroCostFilter"; + +/** + * Rungs in escalation order. Index is the ordering key; membership is decided + * by `assignRung` below. + * + * The rungs differ in more than price — each has its OWN exhaustion signal, + * which is why this is not just a sort: + * + * subscription / keyless / free → exhausted on QUOTA (observable, tracked) + * cheap / premium → exhausted on BUDGET (no quota exists; a paid + * connection serves forever) + */ +export const RUNG_ORDER = ["subscription", "keyless", "free", "cheap", "premium"] as const; + +export type LadderRung = (typeof RUNG_ORDER)[number]; + +/** Rungs whose exhaustion is observable from provider quota state. */ +const QUOTA_BEARING_RUNGS: ReadonlySet = new Set([ + "subscription", + "keyless", + "free", +]); + +export function rungIndex(rung: LadderRung): number { + return RUNG_ORDER.indexOf(rung); +} + +/** A candidate as this module needs to see it — a structural subset of + * `VirtualAutoComboCandidate` (`virtualFactory.ts`), so this file has no + * dependency on that module's full type. */ +export interface LadderCandidate { + provider: string; + model: string; + connectionId: string | null; + allowedConnectionIds?: string[]; +} + +export interface LadderOptions { + /** Master switch. When false every exported filter is the identity function + * — the same off-by-default contract `filterPaidOnlyCandidates` holds. */ + enabled: boolean; + /** + * Live allowance/quota state for ONE (provider, connection) pair, resolved + * from the cache in `freeAccessQuota.ts`. Synchronous by design: nothing in + * a candidate-pool build may await a network call. + * + * `undefined` means "no usage adapter for this provider, or nothing fresh + * cached". The two groupings interpret that OPPOSITELY on purpose — see + * `admitUnknownQuota` below. + */ + resolveFreeAccessState: (provider: string, connectionId: string) => FreeAccessState | undefined; + /** + * `authType` for a connection id (`provider_connections.auth_type`), needed + * to classify billing. Unknown ids resolve to `null` → the provider-wide + * catalog entry, or `unknown` billing. + */ + resolveAuthType: (connectionId: string) => string | null; + /** Economic tier of a (provider, model) pair — `classifyTier()` in + * production, injected so this module needs no registry/pricing import. */ + resolveEconomicTier: (provider: string, model: string) => "free" | "cheap" | "premium"; + /** + * Remaining-percent at or below which a quota-bearing connection counts as + * exhausted. Default 2, matching `quotaPreflight.defaultThresholdPercent` + * (`src/lib/resilience/settings/types.ts`) so the two agree. + */ + exitCutoffPercent?: number; + /** + * Remaining-percent a quota-bearing connection must EXCEED to be re-admitted + * after having been exhausted. Strictly greater than `exitCutoffPercent`; + * the gap is the hysteresis band that stops a connection hovering at the + * cutoff from oscillating between rungs on consecutive requests. Default 5. + */ + reentryMinRemainingPercent?: number; + /** Max age of a `FreeAccessState.checkedAt` before it is treated as stale. */ + maxStateAgeMs: number; + /** + * Whether a connection with no usable quota reading is admitted. + * + * - `auto/thrifty` passes TRUE: trying a plan-included connection costs + * nothing, and if it turns out to be exhausted the dispatcher's + * fall-through reaches the next rung anyway. Refusing to try it would + * send a request to a PAID rung on missing telemetry — the exact + * outcome the grouping exists to avoid. + * - `auto/subscription` passes FALSE: its promise is that no request can + * cost extra, and an unverifiable connection cannot support that promise. + */ + admitUnknownQuota: boolean; + /** + * Budget consumed so far on a paid rung, in USD, for the operator's current + * budget window. `null`/`undefined` means no spend accounting is available, + * in which case paid rungs are NOT budget-gated (they still order after + * every plan-included rung). See the spec's open question on the ledger. + */ + resolveRungSpendUsd?: (rung: LadderRung) => number | null; + /** Per-rung budget in USD. A rung mapped to 0 is disabled outright. */ + rungBudgetUsd?: Partial>; + /** `now` injection for deterministic tests. */ + now?: () => number; + /** Catalog override for tests; production callers never pass this. */ + catalog?: readonly ConnectionBillingEntry[]; +} + +const DEFAULT_EXIT_CUTOFF_PERCENT = 2; +const DEFAULT_REENTRY_MIN_REMAINING_PERCENT = 5; + +/** + * Which rung a specific (candidate, connection) pair belongs to. + * + * Billing class decides first because it is the fact that actually determines + * whether money moves; only a genuinely metered connection falls through to + * the model's economic tier. `unknown` billing is metered by definition + * (`connectionBilling.ts`), so an uncurated provider lands on a paid rung + * rather than silently joining the subscription rung. + */ +export function assignRung( + candidate: Pick, + connection: BillableConnection, + options: Pick +): LadderRung { + const verdict = classifyConnectionBilling(connection, options.catalog); + if (verdict.billing === "subscription") return "subscription"; + if (verdict.billing === "keyless") return "keyless"; + return options.resolveEconomicTier(candidate.provider, candidate.model); +} + +/** + * Is this connection's plan allowance usable right now? + * + * `hasBeenExhausted` selects which side of the hysteresis band applies: a + * connection that is currently in play only has to stay above the exit cutoff, + * while one that already dropped out has to climb back above the (higher) + * re-entry threshold before it is admitted again. + */ +export function isQuotaUsable( + state: FreeAccessState | undefined, + options: Pick< + LadderOptions, + | "exitCutoffPercent" + | "reentryMinRemainingPercent" + | "maxStateAgeMs" + | "admitUnknownQuota" + | "now" + >, + hasBeenExhausted = false +): boolean { + if (!state) return options.admitUnknownQuota; + if (state.status === "EXHAUSTED") return false; + if (state.status === "UNKNOWN") return options.admitUnknownQuota; + + const now = (options.now ?? Date.now)(); + const checkedAtMs = Date.parse(state.checkedAt); + if (!Number.isFinite(checkedAtMs) || now - checkedAtMs > options.maxStateAgeMs) { + return options.admitUnknownQuota; + } + + if (state.remainingFreeAllowance === null) return options.admitUnknownQuota; + + const exitCutoff = options.exitCutoffPercent ?? DEFAULT_EXIT_CUTOFF_PERCENT; + const reentryFloor = Math.max( + options.reentryMinRemainingPercent ?? DEFAULT_REENTRY_MIN_REMAINING_PERCENT, + exitCutoff + ); + const threshold = hasBeenExhausted ? reentryFloor : exitCutoff; + return state.remainingFreeAllowance > threshold; +} + +/** + * Decision 3 — re-entry after a plan quota resets. + * + * A cached state whose own `resetAt` has already passed describes a window + * that no longer exists. Waiting out the cache TTL before re-reading it is + * pure lag on the single transition subscription-first routing cares most + * about, so such an entry is stale REGARDLESS of its age. + * + * Consumed by `freeAccessQuota.ts`, which owns the cache; kept here so the + * rule sits with the rest of the ladder's semantics and is testable without + * touching the cache. + */ +export function isStateStaleForReset( + state: Pick | undefined, + now: number = Date.now() +): boolean { + if (!state?.resetAt) return false; + const resetAtMs = Date.parse(state.resetAt); + if (!Number.isFinite(resetAtMs)) return false; + return resetAtMs <= now; +} + +/** + * Decision 3, second half — never hold a plan-included connection in cooldown + * past the moment its own upstream says the quota is back. + * + * The exhausting 429 sets `rateLimitedUntil` from exponential backoff + * (`baseCooldownMs * 2 ** failureIndex`, `src/sse/services/auth.ts`), which for + * a subscription connection routinely overshoots the real reset — leaving + * routing stuck on paid rungs long after the plan refilled. + * + * This only ever NARROWS a cooldown, and only when the upstream itself + * supplied the reset instant. An absent, unparseable, or already-past + * `resetAt` returns the original cooldown untouched. + */ +export function clampCooldownToReset( + cooldownMs: number, + resetAt: string | null | undefined, + now: number = Date.now() +): number { + if (!resetAt) return cooldownMs; + const resetAtMs = Date.parse(resetAt); + if (!Number.isFinite(resetAtMs)) return cooldownMs; + const untilResetMs = resetAtMs - now; + if (untilResetMs <= 0) return cooldownMs; + return Math.min(cooldownMs, untilResetMs); +} + +/** True when a paid rung has consumed its configured budget for the window. */ +export function isRungBudgetExhausted(rung: LadderRung, options: LadderOptions): boolean { + const budget = options.rungBudgetUsd?.[rung]; + if (budget === undefined) return false; + if (budget <= 0) return true; // explicitly disabled + const spent = options.resolveRungSpendUsd?.(rung); + if (spent === null || spent === undefined) return false; // no accounting → not gated + return spent >= budget; +} + +/** + * Connections on a candidate that are usable right now, paired with the rung + * each one sits on. Quota-bearing rungs are verified per connection; paid + * rungs have nothing per-connection to verify (they are gated per rung). + */ +function evaluateConnections( + candidate: LadderCandidate, + options: LadderOptions, + accept: (rung: LadderRung) => boolean +): { rung: LadderRung; connectionIds: string[] } | null { + const connectionIds = candidate.connectionId + ? [candidate.connectionId] + : (candidate.allowedConnectionIds ?? []); + if (connectionIds.length === 0) return null; + + let bestRung: LadderRung | null = null; + const usable: string[] = []; + + for (const connectionId of connectionIds) { + const rung = assignRung( + candidate, + { + provider: candidate.provider, + authType: options.resolveAuthType(connectionId), + connectionId, + }, + options + ); + if (!accept(rung)) continue; + + if (QUOTA_BEARING_RUNGS.has(rung)) { + const state = options.resolveFreeAccessState(candidate.provider, connectionId); + if (!isQuotaUsable(state, options)) continue; + } else if (isRungBudgetExhausted(rung, options)) { + continue; + } + + usable.push(connectionId); + // A candidate reachable through several accounts is represented by its + // CHEAPEST usable rung: that is the rung a request through it would + // actually land on once dispatch picks from the surviving allowlist. + if (bestRung === null || rungIndex(rung) < rungIndex(bestRung)) bestRung = rung; + } + + if (usable.length === 0 || bestRung === null) return null; + return { rung: bestRung, connectionIds: usable }; +} + +/** Rewrite a candidate's connection allowlist to the verified subset, keeping + * the identity-when-nothing-changed contract the sibling filters hold. */ +function withVerifiedConnections( + candidate: T, + connectionIds: string[] +): { candidate: T; changed: boolean } { + if (candidate.connectionId !== null) return { candidate, changed: false }; + const original = candidate.allowedConnectionIds ?? []; + const isSameSet = + original.length === connectionIds.length && connectionIds.every((id) => original.includes(id)); + if (isSameSet) return { candidate, changed: false }; + return { candidate: { ...candidate, allowedConnectionIds: connectionIds }, changed: true }; +} + +/** + * `auto/subscription` — the strict grouping. Keeps only candidates servable by + * a plan-included connection whose overage is a documented hard stop, with + * live quota headroom verified per connection. + * + * Fails CLOSED in every ambiguous case: uncurated provider, unverifiable + * quota, or an overage that meters to paid. An empty result is the correct, + * intended answer for an operator who asked never to spend extra — the + * caller's existing empty-pool path handles it, exactly as `hidePaidModels` + * already does. + */ +export function filterSubscriptionOnlyCandidates( + pool: T[], + options: LadderOptions +): T[] { + if (!options.enabled) return pool; + + const strictOptions: LadderOptions = { ...options, admitUnknownQuota: false }; + const kept: T[] = []; + let changed = false; + + for (const candidate of pool) { + const connectionIds = candidate.connectionId + ? [candidate.connectionId] + : (candidate.allowedConnectionIds ?? []); + + const safe = connectionIds.filter((connectionId) => { + const connection: BillableConnection = { + provider: candidate.provider, + authType: options.resolveAuthType(connectionId), + connectionId, + }; + const verdict = classifyConnectionBilling(connection, options.catalog); + // `keyless` is plan-included in the ladder's sense but is NOT a + // subscription: this grouping is "the plan I pay for", so a no-auth + // backend does not belong in it. + if (verdict.billing !== "subscription") return false; + if (!isOverageSafe(verdict)) return false; + const state = strictOptions.resolveFreeAccessState(candidate.provider, connectionId); + return isQuotaUsable(state, strictOptions); + }); + + if (safe.length === 0) { + changed = true; + continue; + } + const result = withVerifiedConnections(candidate, safe); + if (result.changed) changed = true; + kept.push(result.candidate); + } + + return changed ? kept : pool; +} + +/** + * `auto/thrifty` — the escalating grouping. Returns the pool ordered by rung, + * with candidates whose every connection is exhausted (quota) or whose rung is + * budget-exhausted removed. + * + * Ordering only — the `auto` engine still scores WITHIN the surviving pool, so + * this decides which rungs are in play, not which candidate wins on one. The + * combo dispatcher already walks targets in order and falls through on + * failure, so a runtime exhaustion the preflight did not catch still escalates + * to the next rung inside the same request. + * + * Rung eligibility is recomputed from live state on every pool build and + * nothing is persisted: there is deliberately no sticky "currently on rung 3" + * record that could outlive a quota reset and wedge routing on paid rungs. + */ +export function orderPoolByRung(pool: T[], options: LadderOptions): T[] { + if (!options.enabled) return pool; + + const ranked: Array<{ candidate: T; rung: LadderRung; order: number }> = []; + for (const [order, candidate] of pool.entries()) { + const evaluated = evaluateConnections(candidate, options, () => true); + if (!evaluated) continue; + const result = withVerifiedConnections(candidate, evaluated.connectionIds); + ranked.push({ candidate: result.candidate, rung: evaluated.rung, order }); + } + + ranked.sort((a, b) => { + const byRung = rungIndex(a.rung) - rungIndex(b.rung); + // Stable within a rung: preserve the pool's incoming order so the auto + // scorer's own ranking is not reshuffled by this overlay. + return byRung !== 0 ? byRung : a.order - b.order; + }); + + return ranked.map((entry) => entry.candidate); +} diff --git a/open-sse/services/autoCombo/suffixComposition.ts b/open-sse/services/autoCombo/suffixComposition.ts index 2299de1d8b..9ab041c90a 100644 --- a/open-sse/services/autoCombo/suffixComposition.ts +++ b/open-sse/services/autoCombo/suffixComposition.ts @@ -23,7 +23,20 @@ import { isVisionModelId } from "@/shared/constants/visionModels"; import { isVisionBridgeForcedModel } from "@/shared/constants/visionBridgeDefaults"; export type AutoCategory = "coding" | "reasoning" | "vision" | "chat" | "multimodal"; -export type AutoTier = "fast" | "cheap" | "floor" | "free" | "reliable" | "pro"; +export type AutoTier = + | "fast" + | "cheap" + | "floor" + | "free" + | "reliable" + | "pro" + // Subscription-first routing. Unlike every tier above, these two narrow by + // the CONNECTION's billing class, not the model's price — so they are + // applied in `virtualFactory.ts` against live connection state rather than + // by `buildAutoCandidateFilter` below, which only sees (provider, model). + // See `subscriptionLadder.ts` and `docs/routing/SUBSCRIPTION_LADDER.md`. + | "subscription" + | "thrifty"; export const AUTO_CATEGORIES: readonly AutoCategory[] = [ "coding", @@ -39,6 +52,8 @@ export const AUTO_TIERS: readonly AutoTier[] = [ "free", "reliable", "pro", + "subscription", + "thrifty", ]; const CATEGORY_SET = new Set(AUTO_CATEGORIES); @@ -84,6 +99,9 @@ export function tierToWeightVariant(tier?: AutoTier): AutoVariant | "reliability return "fast"; case "cheap": case "floor": + // The ladder already orders plan-included rungs first; within a rung it + // should still lean cheap rather than reach for the most expensive model. + case "thrifty": return "cheap"; case "reliable": return "reliability"; @@ -118,8 +136,7 @@ export function buildAutoCandidateFilter( } try { const caps = getResolvedModelCapabilities({ provider: c.provider, model: c.model }); - const capable = - caps.supportsVision === true || isVisionModelId(c.model); + const capable = caps.supportsVision === true || isVisionModelId(c.model); if (!capable) return false; // #vison-pool: registry entries whose catalog OVERSTATES vision support // (opencode-go/opencode-zen/tokenrouter — the backend models are text-only) diff --git a/open-sse/services/autoCombo/virtualFactory.ts b/open-sse/services/autoCombo/virtualFactory.ts index 28071144fb..cb646784d1 100644 --- a/open-sse/services/autoCombo/virtualFactory.ts +++ b/open-sse/services/autoCombo/virtualFactory.ts @@ -26,6 +26,11 @@ import { buildFamilyCandidateFilter, type ModelFamily } from "./modelFamily"; import { getHiddenModelsByProvider } from "@/models"; import { getSyncedAvailableModelsByConnection, getCustomModels } from "@/lib/db/models"; import { filterPaidOnlyCandidates } from "./paidModelFilter"; +import { + filterSubscriptionOnlyCandidates, + orderPoolByRung, + type LadderOptions, +} from "./subscriptionLadder"; import { filterStrictZeroCostCandidates, filterTosAvoidCandidates } from "./strictZeroCostFilter"; import { resolveFreeAccessState } from "./freeAccessQuota"; import { isModelExcludedByConnection } from "@/domain/connectionModelRules"; @@ -143,6 +148,98 @@ type VirtualAutoCombo = AutoComboConfig & { export interface PreparedVirtualAutoComboInputs { readonly regularCandidates: readonly VirtualAutoComboCandidate[]; readonly familyCandidates: readonly VirtualAutoComboCandidate[]; + /** + * `provider_connections.auth_type` per connection id. Subscription-first + * routing classifies billing per CONNECTION (`connectionBilling.ts`), and + * the candidate pool only carries connection ids — so the auth types are + * captured here, during the one bulk connection read this function already + * does, instead of re-reading the DB per pool narrowing. + */ + readonly authTypeByConnectionId?: ReadonlyMap; + /** Operator settings for the subscription ladder; absent = feature off. */ + readonly subscriptionLadder?: SubscriptionLadderSettings; +} + +/** + * Operator-facing knobs for subscription-first routing. Mirrors the Zod shape + * in `src/shared/validation/settingsSchemas.ts`. + * + * Deliberately TUNING ONLY — there is no `enabled` flag here. `auto/subscription` + * and `auto/thrifty` are new ids that nothing routes through unless a caller + * asks for them by name, so requesting the id IS the opt-in. A settings toggle + * that could switch them off would be actively dangerous: it would leave an id + * whose whole promise is "plan-included only" quietly serving the full pool, + * paid models included. + */ +export interface SubscriptionLadderSettings { + exitCutoffPercent?: number; + reentryMinRemainingPercent?: number; + rungBudgetUsd?: Record; + /** Staleness bound for a cached quota reading, derived from the existing + * `autoRefreshProviderQuotaInterval` exactly as STRICT_ZERO_COST does. */ + maxStateAgeMs: number; +} + +function readSubscriptionLadderSettings( + settings: Record +): SubscriptionLadderSettings { + const maxStateAgeMs = (Number(settings.autoRefreshProviderQuotaInterval) || 180) * 1000; + const raw = settings.subscriptionLadder; + if (!raw || typeof raw !== "object") return { maxStateAgeMs }; + const value = raw as Record; + const numeric = (key: string): number | undefined => + typeof value[key] === "number" && Number.isFinite(value[key] as number) + ? (value[key] as number) + : undefined; + const exitCutoffPercent = numeric("exitCutoffPercent"); + const reentryMinRemainingPercent = numeric("reentryMinRemainingPercent"); + return { + maxStateAgeMs, + ...(exitCutoffPercent === undefined ? {} : { exitCutoffPercent }), + ...(reentryMinRemainingPercent === undefined ? {} : { reentryMinRemainingPercent }), + ...(value.rungBudgetUsd && typeof value.rungBudgetUsd === "object" + ? { rungBudgetUsd: value.rungBudgetUsd as Record } + : {}), + }; +} + +/** + * Build the injected dependencies the pure ladder module needs. Everything it + * touches is resolved here — the live quota cache, connection auth types, and + * the economic tier resolver — so `subscriptionLadder.ts` itself stays free of + * DB, network, and registry imports. + */ +function buildLadderOptions( + prepared: PreparedVirtualAutoComboInputs, + tier: "subscription" | "thrifty" +): LadderOptions { + const tuning = prepared.subscriptionLadder; + const authTypes = prepared.authTypeByConnectionId; + return { + enabled: true, + resolveFreeAccessState, + resolveAuthType: (connectionId: string) => authTypes?.get(connectionId) ?? null, + resolveEconomicTier: (provider: string, model: string) => { + try { + const resolved = classifyTier(provider, model).tier; + return resolved === "free" || resolved === "premium" ? resolved : "cheap"; + } catch { + // Same conservative default `safeClassifyTier` uses in suffixComposition. + return "cheap"; + } + }, + maxStateAgeMs: tuning?.maxStateAgeMs ?? 180_000, + // The two groupings read a missing quota reading OPPOSITELY on purpose — + // see `LadderOptions.admitUnknownQuota`. + admitUnknownQuota: tier === "thrifty", + ...(tuning?.exitCutoffPercent === undefined + ? {} + : { exitCutoffPercent: tuning.exitCutoffPercent }), + ...(tuning?.reentryMinRemainingPercent === undefined + ? {} + : { reentryMinRemainingPercent: tuning.reentryMinRemainingPercent }), + ...(tuning?.rungBudgetUsd ? { rungBudgetUsd: tuning.rungBudgetUsd } : {}), + }; } function toExpiryMs(value: unknown): number | null { @@ -621,8 +718,15 @@ export async function prepareVirtualAutoComboInputs( const regularCandidates = buildPreparedPool(false); // #6453/#8183: family selectors bypass the reliability-curated no-auth allowlist. const familyCandidates = buildPreparedPool(true); + // Subscription-first routing inputs, captured from the connection read above + // so no later stage has to touch the DB again. + const authTypeByConnectionId = new Map(); + for (const conn of connections) { + authTypeByConnectionId.set(conn.id, typeof conn.authType === "string" ? conn.authType : null); + } + const subscriptionLadder = readSubscriptionLadderSettings(settings); if (!options.includeResolvedCapabilities) { - return { regularCandidates, familyCandidates }; + return { regularCandidates, familyCandidates, authTypeByConnectionId, subscriptionLadder }; } // One uninterrupted bulk read of all three capability tables for this prepare only. @@ -636,6 +740,8 @@ export async function prepareVirtualAutoComboInputs( return { regularCandidates: await attachPreparedCapabilityValues(regularCandidates, capabilityState), familyCandidates: await attachPreparedCapabilityValues(familyCandidates, capabilityState), + authTypeByConnectionId, + subscriptionLadder, }; } @@ -807,6 +913,32 @@ export async function createVirtualAutoComboFromPrepared( } } + // Subscription-first routing (`auto/subscription`, `auto/thrifty`). Applied + // AFTER the category/tier narrowing above because, unlike every other tier, + // these two select on the connection's billing class and its live quota + // state rather than on the model's catalog price — see + // `subscriptionLadder.ts` and `docs/routing/SUBSCRIPTION_LADDER.md`. + if (spec?.tier === "subscription" || spec?.tier === "thrifty") { + const ladderOptions = buildLadderOptions(prepared, spec.tier); + const beforeCount = effectivePool.length; + effectivePool = + spec.tier === "subscription" + ? filterSubscriptionOnlyCandidates(effectivePool, ladderOptions) + : orderPoolByRung(effectivePool, ladderOptions); + if (spec.tier === "subscription" && effectivePool.length === 0 && beforeCount > 0) { + // Intended, not a defect: the operator asked for plan-included capacity + // only, and right now there is none with verified headroom. Failing + // closed here is the entire promise of the id — the caller's existing + // empty-pool path turns it into a clear error rather than a silent, + // billable fallback. + warnEmptyAutoPoolOnce( + "auto/subscription", + "auto/subscription: no plan-included connection has verified quota headroom; " + + "returning an empty pool rather than falling back to paid capacity." + ); + } + } + let weights: ScoringWeights = { ...DEFAULT_WEIGHTS }; let explorationRate = 0.05; // Default exploration rate let routerStrategy = "lkgp"; // All auto variants use LKGP diff --git a/src/shared/validation/settingsSchemas.ts b/src/shared/validation/settingsSchemas.ts index 388e3f73c7..085c36c4af 100644 --- a/src/shared/validation/settingsSchemas.ts +++ b/src/shared/validation/settingsSchemas.ts @@ -138,6 +138,24 @@ export const updateSettingsSchema = z.object({ // curated `tos` verdict is "avoid" (proxy/self-hosted use conflicts with the // provider's own terms) — a contractual concern, not an economic one. excludeTosAvoid: z.boolean().optional(), + // Subscription-first routing tuning (`auto/subscription`, `auto/thrifty`). + // TUNING ONLY — there is deliberately no `enabled` flag: both ids are opt-in + // by being requested, and a toggle able to switch them off would leave + // `auto/subscription` silently serving paid capacity under a name that + // promises the opposite. See open-sse/services/autoCombo/subscriptionLadder.ts. + subscriptionLadder: z + .object({ + // Remaining-% at or below which a plan-included connection counts as + // exhausted. Matches quotaPreflight.defaultThresholdPercent's default. + exitCutoffPercent: z.number().min(0).max(100).optional(), + // Remaining-% a connection must EXCEED to be re-admitted after having + // been exhausted. The gap above exitCutoffPercent is the hysteresis band + // that stops a connection hovering at the cutoff from oscillating. + reentryMinRemainingPercent: z.number().min(0).max(100).optional(), + // Per-rung spend ceiling in USD. 0 disables a rung outright. + rungBudgetUsd: z.record(z.string().max(32), z.number().min(0)).optional(), + }) + .optional(), hideHealthCheckLogs: z.boolean().optional(), hideEndpointCloudflaredTunnel: z.boolean().optional(), hideEndpointTailscaleFunnel: z.boolean().optional(), diff --git a/tests/unit/autoCombo/subscription-ladder.test.ts b/tests/unit/autoCombo/subscription-ladder.test.ts new file mode 100644 index 0000000000..4c49c1821b --- /dev/null +++ b/tests/unit/autoCombo/subscription-ladder.test.ts @@ -0,0 +1,407 @@ +/** + * Subscription-first routing — regression guard for `connectionBilling.ts` and + * `subscriptionLadder.ts`, wired into + * `open-sse/services/autoCombo/virtualFactory.ts::createVirtualAutoComboFromPrepared` + * for the `auto/subscription` and `auto/thrifty` ids. + * + * Pure and dependency-light by design, mirroring + * `strict-zero-cost-filter.test.ts`: every side-effecting dependency (live + * quota state, connection auth types, the economic tier resolver, the billing + * catalog) is injected, so nothing here touches the DB, the network, or global + * state. + */ +import { test } from "vitest"; +import assert from "node:assert/strict"; + +import type { ConnectionBillingEntry } from "@omniroute/open-sse/config/connectionBillingCatalog.ts"; +import { + classifyConnectionBilling, + isOverageSafe, + isPlanIncluded, +} from "@omniroute/open-sse/services/autoCombo/connectionBilling.ts"; +import { SYNTHETIC_NOAUTH_CONNECTION_ID } from "@omniroute/open-sse/services/autoCombo/resilienceCandidateFilter.ts"; +import { + RUNG_ORDER, + assignRung, + clampCooldownToReset, + filterSubscriptionOnlyCandidates, + isQuotaUsable, + isStateStaleForReset, + orderPoolByRung, + type LadderCandidate, + type LadderOptions, +} from "@omniroute/open-sse/services/autoCombo/subscriptionLadder.ts"; +import type { FreeAccessState } from "@omniroute/open-sse/services/autoCombo/strictZeroCostFilter.ts"; + +const NOW = Date.parse("2026-08-22T12:00:00.000Z"); + +/** Synthetic catalog — never the real one, so these tests keep passing when + * the curated entries are edited (the autodiscovery contract). */ +const CATALOG: readonly ConnectionBillingEntry[] = [ + { + provider: "planned", + authType: "oauth", + billing: "subscription", + overage: "hard-stop", + reason: "test fixture: plan-included, refuses past the allowance", + }, + { + provider: "planned", + authType: "apikey", + billing: "metered", + overage: "meters-to-paid", + reason: "test fixture: same provider, metered credential", + }, + { + provider: "overflowing", + billing: "subscription", + overage: "meters-to-paid", + reason: "test fixture: plan-included but bills past the allowance", + }, + { + provider: "metered-co", + billing: "metered", + overage: "meters-to-paid", + reason: "test fixture: pay per token", + }, +]; + +function state(overrides: Partial = {}): FreeAccessState { + return { + status: "SAFE", + remainingFreeAllowance: 50, + resetAt: null, + checkedAt: new Date(NOW - 1000).toISOString(), + ...overrides, + }; +} + +function options(overrides: Partial = {}): LadderOptions { + return { + enabled: true, + resolveFreeAccessState: () => state(), + resolveAuthType: () => "oauth", + resolveEconomicTier: () => "cheap", + maxStateAgeMs: 180_000, + admitUnknownQuota: false, + now: () => NOW, + catalog: CATALOG, + ...overrides, + }; +} + +function candidate(overrides: Partial = {}): LadderCandidate { + return { + provider: "planned", + model: "m1", + connectionId: "c1", + ...overrides, + }; +} + +// ── classification ────────────────────────────────────────────────────────── + +test("the synthetic no-auth connection classifies as keyless without consulting the catalog", () => { + const verdict = classifyConnectionBilling( + { provider: "metered-co", connectionId: SYNTHETIC_NOAUTH_CONNECTION_ID }, + CATALOG + ); + assert.equal(verdict.billing, "keyless"); + assert.equal(isPlanIncluded(verdict), true); +}); + +test("an authType-specific entry wins over the provider-wide one", () => { + const oauth = classifyConnectionBilling( + { provider: "planned", authType: "oauth", connectionId: "c1" }, + CATALOG + ); + const apikey = classifyConnectionBilling( + { provider: "planned", authType: "apikey", connectionId: "c2" }, + CATALOG + ); + assert.equal(oauth.billing, "subscription"); + assert.equal(apikey.billing, "metered"); +}); + +test("a provider-wide entry applies when no authType entry matches", () => { + const verdict = classifyConnectionBilling( + { provider: "overflowing", authType: "cookie", connectionId: "c1" }, + CATALOG + ); + assert.equal(verdict.billing, "subscription"); + assert.equal(isOverageSafe(verdict), false); +}); + +test("an uncurated provider is unknown — never silently plan-included", () => { + const verdict = classifyConnectionBilling( + { provider: "brand-new", authType: "oauth", connectionId: "c1" }, + CATALOG + ); + assert.equal(verdict.billing, "unknown"); + assert.equal(isPlanIncluded(verdict), false); + assert.equal(isOverageSafe(verdict), false); +}); + +test("rung assignment prefers billing class, falling back to the economic tier", () => { + const opts = options({ resolveEconomicTier: () => "premium" }); + assert.equal( + assignRung( + { provider: "planned", model: "m1" }, + { provider: "planned", authType: "oauth" }, + opts + ), + "subscription" + ); + assert.equal( + assignRung( + { provider: "metered-co", model: "m1" }, + { provider: "metered-co", authType: "apikey" }, + opts + ), + "premium" + ); + assert.equal( + assignRung( + { provider: "whatever", model: "m1" }, + { provider: "whatever", connectionId: SYNTHETIC_NOAUTH_CONNECTION_ID }, + opts + ), + "keyless" + ); +}); + +// ── auto/subscription — fails closed ──────────────────────────────────────── + +test("disabled leaves the pool byte-identical (the opt-in contract)", () => { + const pool = [candidate()]; + assert.equal(filterSubscriptionOnlyCandidates(pool, options({ enabled: false })), pool); + assert.equal(orderPoolByRung(pool, options({ enabled: false })), pool); +}); + +test("a plan-included, hard-stop connection with headroom is kept", () => { + const pool = [candidate()]; + assert.deepEqual(filterSubscriptionOnlyCandidates(pool, options()), pool); +}); + +test("a subscription that meters past the plan is excluded", () => { + const pool = [candidate({ provider: "overflowing" })]; + assert.deepEqual(filterSubscriptionOnlyCandidates(pool, options()), []); +}); + +test("a metered connection is excluded even on a provider that also sells a plan", () => { + const pool = [candidate({ connectionId: "c2" })]; + const result = filterSubscriptionOnlyCandidates( + pool, + options({ resolveAuthType: () => "apikey" }) + ); + assert.deepEqual(result, []); +}); + +test("keyless is not a subscription — auto/subscription means the plan you pay for", () => { + const pool = [candidate({ provider: "freebie", connectionId: SYNTHETIC_NOAUTH_CONNECTION_ID })]; + assert.deepEqual(filterSubscriptionOnlyCandidates(pool, options()), []); +}); + +test("an unverifiable quota reading fails closed", () => { + const pool = [candidate()]; + assert.deepEqual( + filterSubscriptionOnlyCandidates(pool, options({ resolveFreeAccessState: () => undefined })), + [] + ); + assert.deepEqual( + filterSubscriptionOnlyCandidates( + pool, + options({ resolveFreeAccessState: () => state({ status: "UNKNOWN" }) }) + ), + [] + ); +}); + +test("a stale quota reading fails closed even when it says SAFE", () => { + const stale = state({ checkedAt: new Date(NOW - 10 * 60_000).toISOString() }); + assert.deepEqual( + filterSubscriptionOnlyCandidates( + [candidate()], + options({ resolveFreeAccessState: () => stale }) + ), + [] + ); +}); + +test("a multi-account candidate keeps only the connections proven safe", () => { + const pool = [candidate({ connectionId: null, allowedConnectionIds: ["a", "b", "c"] })]; + const result = filterSubscriptionOnlyCandidates( + pool, + options({ + resolveFreeAccessState: (_provider, connectionId) => + connectionId === "b" ? state({ status: "EXHAUSTED", remainingFreeAllowance: 0 }) : state(), + }) + ); + assert.equal(result.length, 1); + assert.deepEqual(result[0].allowedConnectionIds, ["a", "c"]); +}); + +test("a multi-account candidate with no safe connection is dropped, not emptied", () => { + const pool = [candidate({ connectionId: null, allowedConnectionIds: ["a", "b"] })]; + const result = filterSubscriptionOnlyCandidates( + pool, + options({ + resolveFreeAccessState: () => state({ status: "EXHAUSTED", remainingFreeAllowance: 0 }), + }) + ); + assert.deepEqual(result, []); +}); + +// ── auto/thrifty — escalates ─────────────────────────────────────────────── + +test("rungs order plan-included capacity ahead of every paid rung", () => { + const pool: LadderCandidate[] = [ + { provider: "metered-co", model: "premium-model", connectionId: "p1" }, + { provider: "metered-co", model: "cheap-model", connectionId: "c1" }, + { provider: "planned", model: "plan-model", connectionId: "s1" }, + { provider: "anything", model: "keyless-model", connectionId: SYNTHETIC_NOAUTH_CONNECTION_ID }, + ]; + const result = orderPoolByRung( + pool, + options({ + admitUnknownQuota: true, + resolveAuthType: (id) => (id === "s1" ? "oauth" : "apikey"), + resolveEconomicTier: (_provider, model) => (model === "premium-model" ? "premium" : "cheap"), + }) + ); + assert.deepEqual( + result.map((c) => c.model), + ["plan-model", "keyless-model", "cheap-model", "premium-model"] + ); +}); + +test("an exhausted plan connection steps aside so a paid rung can serve", () => { + const pool: LadderCandidate[] = [ + { provider: "planned", model: "plan-model", connectionId: "s1" }, + { provider: "metered-co", model: "cheap-model", connectionId: "c1" }, + ]; + const result = orderPoolByRung( + pool, + options({ + admitUnknownQuota: true, + resolveAuthType: (id) => (id === "s1" ? "oauth" : "apikey"), + resolveFreeAccessState: (_provider, connectionId) => + connectionId === "s1" + ? state({ status: "EXHAUSTED", remainingFreeAllowance: 0 }) + : undefined, + }) + ); + assert.deepEqual( + result.map((c) => c.model), + ["cheap-model"] + ); +}); + +test("the ladder admits an unverifiable plan connection rather than paying on missing telemetry", () => { + const pool = [candidate({ connectionId: "s1", model: "plan-model" })]; + const result = orderPoolByRung( + pool, + options({ admitUnknownQuota: true, resolveFreeAccessState: () => undefined }) + ); + assert.equal(result.length, 1); +}); + +test("ordering is stable within a rung so the auto scorer is not reshuffled", () => { + const pool: LadderCandidate[] = [ + { provider: "metered-co", model: "first", connectionId: "a" }, + { provider: "metered-co", model: "second", connectionId: "b" }, + { provider: "metered-co", model: "third", connectionId: "c" }, + ]; + const result = orderPoolByRung( + pool, + options({ admitUnknownQuota: true, resolveAuthType: () => "apikey" }) + ); + assert.deepEqual( + result.map((c) => c.model), + ["first", "second", "third"] + ); +}); + +test("a rung budgeted at zero is disabled outright", () => { + const pool: LadderCandidate[] = [ + { provider: "metered-co", model: "cheap-model", connectionId: "c1" }, + { provider: "metered-co", model: "premium-model", connectionId: "p1" }, + ]; + const result = orderPoolByRung( + pool, + options({ + admitUnknownQuota: true, + resolveAuthType: () => "apikey", + resolveEconomicTier: (_p, model) => (model === "premium-model" ? "premium" : "cheap"), + rungBudgetUsd: { premium: 0 }, + }) + ); + assert.deepEqual( + result.map((c) => c.model), + ["cheap-model"] + ); +}); + +test("a paid rung drops out once its budget is spent, and is ungated without accounting", () => { + const pool: LadderCandidate[] = [ + { provider: "metered-co", model: "cheap-model", connectionId: "c1" }, + ]; + const base = { + admitUnknownQuota: true, + resolveAuthType: () => "apikey", + rungBudgetUsd: { cheap: 5 }, + }; + assert.deepEqual(orderPoolByRung(pool, options({ ...base, resolveRungSpendUsd: () => 5 })), []); + assert.equal(orderPoolByRung(pool, options({ ...base, resolveRungSpendUsd: () => 1 })).length, 1); + // No spend accounting available → the rung is ordered, never gated. + assert.equal( + orderPoolByRung(pool, options({ ...base, resolveRungSpendUsd: () => null })).length, + 1 + ); +}); + +// ── decision 3: returning to the plan after a reset ───────────────────────── + +test("a cached reading whose own resetAt has passed is stale regardless of age", () => { + assert.equal( + isStateStaleForReset(state({ resetAt: new Date(NOW - 1).toISOString() }), NOW), + true + ); + assert.equal( + isStateStaleForReset(state({ resetAt: new Date(NOW + 60_000).toISOString() }), NOW), + false + ); + assert.equal(isStateStaleForReset(state({ resetAt: null }), NOW), false); + assert.equal(isStateStaleForReset(state({ resetAt: "not-a-date" }), NOW), false); + assert.equal(isStateStaleForReset(undefined, NOW), false); +}); + +test("hysteresis: re-entry needs more headroom than staying in did", () => { + const opts = options({ exitCutoffPercent: 2, reentryMinRemainingPercent: 5 }); + const hovering = state({ remainingFreeAllowance: 3 }); + // Still in play at 3% remaining… + assert.equal(isQuotaUsable(hovering, opts, false), true); + // …but not enough to climb back after having dropped out. + assert.equal(isQuotaUsable(hovering, opts, true), false); + assert.equal(isQuotaUsable(state({ remainingFreeAllowance: 6 }), opts, true), true); +}); + +test("a re-entry floor below the exit cutoff cannot create a re-entry gap", () => { + const opts = options({ exitCutoffPercent: 10, reentryMinRemainingPercent: 1 }); + assert.equal(isQuotaUsable(state({ remainingFreeAllowance: 5 }), opts, true), false); +}); + +test("cooldown is clamped to the upstream's own reset instant, never extended", () => { + const resetIn60s = new Date(NOW + 60_000).toISOString(); + assert.equal(clampCooldownToReset(600_000, resetIn60s, NOW), 60_000); + // Already shorter than the reset → untouched. + assert.equal(clampCooldownToReset(10_000, resetIn60s, NOW), 10_000); + // Absent / unparseable / already elapsed → untouched, never widened. + assert.equal(clampCooldownToReset(600_000, null, NOW), 600_000); + assert.equal(clampCooldownToReset(600_000, "nonsense", NOW), 600_000); + assert.equal(clampCooldownToReset(600_000, new Date(NOW - 1).toISOString(), NOW), 600_000); +}); + +test("rung order is the documented escalation order", () => { + assert.deepEqual([...RUNG_ORDER], ["subscription", "keyless", "free", "cheap", "premium"]); +}); From 7c4d44fe18f736c71e3347f29c3860e9a2c58b3e Mon Sep 17 00:00:00 2001 From: Hermes Developer Date: Sat, 22 Aug 2026 11:56:35 -0400 Subject: [PATCH 2/2] docs(changelog): fragment for #11146 --- changelog.d/features/11146-subscription-first-routing.md | 1 + 1 file changed, 1 insertion(+) create mode 100644 changelog.d/features/11146-subscription-first-routing.md diff --git a/changelog.d/features/11146-subscription-first-routing.md b/changelog.d/features/11146-subscription-first-routing.md new file mode 100644 index 0000000000..d95ea9c74a --- /dev/null +++ b/changelog.d/features/11146-subscription-first-routing.md @@ -0,0 +1 @@ +- **feat(routing):** subscription-first auto groupings — `auto/subscription` routes only through plan-included connections with a documented hard-stop overage and fails closed on exhaustion, while `auto/thrifty` orders the pool `subscription → keyless → free → cheap → premium` and steps up one rung at a time as each is exhausted. Billing class comes from a curated per-connection catalog (uncurated is treated as metered, never plan-included), both reuse STRICT_ZERO_COST's per-connection verification, and a quota reading whose `resetAt` has passed is now refreshed regardless of TTL so routing returns to plan capacity as soon as it resets ([#11146](https://github.com/diegosouzapw/OmniRoute/pull/11146))