mirror of
https://github.com/diegosouzapw/OmniRoute.git
synced 2026-08-25 08:32:11 +03:00
feat(routing): subscription-first auto groupings (auto/subscription, auto/thrifty) [defer to 3.8.51] (#11146)
Merged into release/v3.8.51 via batch validation: subscription-ladder + free-regime vitest suites green (32/32) on the combined tree, check:provider-consistency OK (353 canonical providers), static gates green (virtualFactory.ts frozen at merge size with dated rebaseline). Also pushed a docs commit marking rungBudgetUsd as not-yet-enforced per review, and synced the branch onto the updated release tip. Strong opt-in design failing closed where money is involved — thanks @yourspraveen!
This commit is contained in:
Praveen K Palaniswamy
committed by
GitHub
GitHub
parent
a0ceccc6f0
commit
b39e5ecb20
1
changelog.d/features/11146-subscription-first-routing.md
Normal file
1
changelog.d/features/11146-subscription-first-routing.md
Normal file
@@ -0,0 +1 @@
|
||||
- **feat(routing):** subscription-first auto groupings — `auto/subscription` routes only through plan-included connections with a documented hard-stop overage and fails closed on exhaustion, while `auto/thrifty` orders the pool `subscription → keyless → free → cheap → premium` and steps up one rung at a time as each is exhausted. Billing class comes from a curated per-connection catalog (uncurated is treated as metered, never plan-included), both reuse STRICT_ZERO_COST's per-connection verification, and a quota reading whose `resetAt` has passed is now refreshed regardless of TTL so routing returns to plan capacity as soon as it resets ([#11146](https://github.com/diegosouzapw/OmniRoute/pull/11146))
|
||||
@@ -478,7 +478,8 @@
|
||||
"_rebaseline_2026_08_23_tip_drift_post_batch0823": "Tip drift after the 2026-08-23 merge wave: chatBodyAdmission.ts 1009->1118 (+109, gate count incl. +1) and auth.ts 3337->3344 (+7), both grown by merges already on origin/release/v3.8.50 (verified identical on the pristine tip) — not by the codex-appserver-hardening PR that carries this bump. Owner pre-authorized baseline bumps 2026-08-22.",
|
||||
"_rebaseline_2026_08_24_11355_cooldown_recovery_guards": "PR #11355 own growth: test/route.ts 1215->1237, +22 (startup crash-recovery guard: clearStaleCrashCooldowns() now parses the persisted rate_limited_until deadline and skips clearing rows still genuinely in the future, instead of clearing every non-terminal cooldown unconditionally). Cohesive fix at the existing test-route dispatch chokepoint alongside the #11141 probe builder. Covered by tests/unit/startup-stale-cooldown-recovery.test.ts + tests/unit/repro-zai-cooldown-cleared-by-connection-test.test.ts.",
|
||||
"src/lib/guardrails/videoBridgeRuntime.ts": 1009,
|
||||
"_rebaseline_2026_08_24_video_bridge_fu02_fu07_sampler": "PRs #11344 (FU-02 one-frame scene-aware determinism) + #11381 (FU-07 opt-in segment_aware structural sampling) own growth: videoBridgeRuntime.ts <1000->1009, +9 (sum of both boarded together in the same merge-batch). #11344 adds the deterministic one-frame midpoint fallback + policyEffective=uniform report at the existing scene_aware seam; #11381 adds the bounded local-only FFmpeg structural pre-analysis pass (scene/freeze/blur/exposure/SI-TI) and its budget-reallocation logic. Covered by tests/unit/guardrails/videoBridgeSampler.test.ts, tests/unit/guardrails/videoBridgeFu07StructuralSampling.test.ts, tests/integration/video-bridge-sampler-ffmpeg.test.ts. Owner pre-authorized rebaseline for legitimate PR growth (2026-08-19 directive)."
|
||||
"_rebaseline_2026_08_24_video_bridge_fu02_fu07_sampler": "PRs #11344 (FU-02 one-frame scene-aware determinism) + #11381 (FU-07 opt-in segment_aware structural sampling) own growth: videoBridgeRuntime.ts <1000->1009, +9 (sum of both boarded together in the same merge-batch). #11344 adds the deterministic one-frame midpoint fallback + policyEffective=uniform report at the existing scene_aware seam; #11381 adds the bounded local-only FFmpeg structural pre-analysis pass (scene/freeze/blur/exposure/SI-TI) and its budget-reallocation logic. Covered by tests/unit/guardrails/videoBridgeSampler.test.ts, tests/unit/guardrails/videoBridgeFu07StructuralSampling.test.ts, tests/integration/video-bridge-sampler-ffmpeg.test.ts. Owner pre-authorized rebaseline for legitimate PR growth (2026-08-19 directive).",
|
||||
"open-sse/services/autoCombo/virtualFactory.ts": 1130
|
||||
},
|
||||
"_rebaseline_base_2026_08_10_proxyfetch": "Base-red fix (green-prs sweep, issue #9985): open-sse/utils/proxyFetch.ts 1207 > cap 1000 — new proxied-TLS fetch helper introduced by the Fal reference-image work. Owner-authorized quick rebaseline to green; structural slim tracked for v3.9.0.",
|
||||
"_rebaseline_2026_07_27_v3849_train2": "Merge-train 2 (7 PRs) — owner-approved 2026-07-27. Single entry: chatCore.ts 4955->5006 (#8595, Responses multi-turn image compaction before the context hard-reject). Genuine irreducible growth at the existing compaction chokepoint in handleChatCore — the PR adds a last-resort retry against the concrete budget plus the estimateFinalInputTokens helper, both wired at the pre-existing call site rather than a new branch. Covered by tests/unit/8560-responses-image-compaction.test.ts (4 tests).",
|
||||
@@ -646,5 +647,6 @@
|
||||
"_rebaseline_2026_08_18_imageregistry_merge_train": "merge-train 2026-08-18 (owner-authorized, /merge-prs batch of 84): open-sse/config/imageRegistry.ts crossed the 1000-line new-file cap for the first time purely from combining three independent, already-legitimate provider registrations boarded in the same local merge-train — #10542 (aihorde optional-key image catalog), #10494 (gemini-web image generation), #10594 (freepik/magnific provider rename + validation). 996 on release tip -> 1019 on the train tip. Each PR individually adds a small, additive IMAGE_PROVIDERS registry entry at the existing chokepoint; none crosses the cap alone. Not modularized as part of this train's gate fix (out of scope for a merge reconciliation, not a feature change). Covered by each PR's own focused tests (aihorde-image-catalog/generation, gemini-web image tests, freepik/magnific provider tests).",
|
||||
"_rebaseline_2026_08_20_v3850_merge_train_batch1": "Merge-train batch1 (2026-08-19/20, 30 PRs boarded onto release/v3.8.50): gateways.ts 1255->1268 = PR #10722 (Token Kiosk OpenAI-compatible provider gateway catalog entry, +13 declarative lines, same god-file no-split rationale as prior gateways.ts rebaselines); chatHelpers.ts (uncapped, not previously frozen) new 1017 = PR #10797 (relay/bifrost error normalization, +23/-2, own-PR growth, existing file already near cap from accumulated chokepoint wiring per its own rebaseline history above); chatBodyAdmission.ts (uncapped) new 1005 = pre-existing base-red on the pure release tip (1004>1000 before this train boarded anything, no PR in this batch touches this file) — frozen here at its current size, not authorizing further growth. Owner-authorized rebaseline (2026-08-19 merge-prs session).",
|
||||
"_rebaseline_2026_08_20_8338_cursor_image_provider": "PR (reimplementation of #8338, @valvesss): imageRegistry.ts 1019->1033 = new cursor IMAGE_PROVIDERS entry (Cursor plan image generation via Agent CLI), +14 lines of declarative provider metadata. Same god-registry no-split rationale as prior imageRegistry/gateways rebaselines.",
|
||||
"_rebaseline_2026_08_20_imageregistry_1034": "imageRegistry.ts 1033->1034: +1 line drift between #10842 (cursor image provider, froze at 1033) and its actual merged state on release (measured 1034) — trivial rebaseline, not a new feature."
|
||||
"_rebaseline_2026_08_20_imageregistry_1034": "imageRegistry.ts 1033->1034: +1 line drift between #10842 (cursor image provider, froze at 1033) and its actual merged state on release (measured 1034) — trivial rebaseline, not a new feature.",
|
||||
"_rebaseline_2026_08_25_11146_subscription_first_auto": "PR #11146 (@yourspraveen, subscription-first auto groupings auto/subscription+auto/thrifty): open-sse/services/autoCombo/virtualFactory.ts is a NEW file in this PR landing at 1128 lines (+2 margin) — two opt-in flat auto ids built on the established auto/best-free pattern (connectionBillingCatalog + subscriptionLadder pure functions). Frozen at merge size per owner-authorized rebaseline directive (2026-08-19, merge-batch Step 4); no further growth without split rationale."
|
||||
}
|
||||
|
||||
@@ -24,8 +24,11 @@ it expires.
|
||||
| Antigravity / Devin Desktop | Built-in quotas |
|
||||
|
||||
**Strategy**: route here first for every request that fits the model's
|
||||
strengths. Quota tracker monitors approaching reset; combo strategies
|
||||
`reset-aware` and `subscription` prioritize accordingly.
|
||||
strengths. The quota tracker monitors approaching resets, and the `reset-aware`
|
||||
combo strategy prioritizes accordingly. To route Tier 1 first and only step out
|
||||
to paid tiers as quota runs out, use the `auto/thrifty` id — or `auto/subscription`
|
||||
to stay on plan-included capacity and fail closed instead. See
|
||||
[Subscription-first routing](../routing/SUBSCRIPTION_LADDER.md).
|
||||
|
||||
## Tier 2 — Cheap
|
||||
|
||||
|
||||
182
docs/routing/SUBSCRIPTION_LADDER.md
Normal file
182
docs/routing/SUBSCRIPTION_LADDER.md
Normal file
@@ -0,0 +1,182 @@
|
||||
---
|
||||
title: "Subscription-first routing"
|
||||
version: 3.8.50
|
||||
lastUpdated: 2026-08-22
|
||||
---
|
||||
|
||||
# Subscription-first routing
|
||||
|
||||
> Two new `auto/*` ids — `auto/subscription` and `auto/thrifty`. Both are opt-in by being
|
||||
> requested: nothing routes through them unless a caller asks for the id by name, and no
|
||||
> existing pool, strategy, or default changes.
|
||||
|
||||
## Why this exists
|
||||
|
||||
OmniRoute already answers two cost questions, and neither is the one most operators ask.
|
||||
|
||||
| Existing mechanism | Answers |
|
||||
| -------------------------------------------------------- | ----------------------------------- |
|
||||
| `hidePaidModels` (`autoCombo/paidModelFilter.ts`) | "is this model catalogued free?" |
|
||||
| `freeAccessPolicy: "strict"` (`strictZeroCostFilter.ts`) | "can this connection ever bill me?" |
|
||||
| `quotaPreflight` (`combo/quotaExhaustionCutoff.ts`) | "is this connection near its wall?" |
|
||||
| `budgetCap` / `budgetFallback` (`autoCombo/engine.ts`) | "cap spend, degrade to cheapest" |
|
||||
|
||||
Every free-only mechanism **fails closed** — an exhausted free pool is an empty pool, never a
|
||||
step up to a paid option — and every paid-side mechanism is tier-agnostic. Neither answers:
|
||||
|
||||
> "Use the quota I already pay for. When it runs out, either stop, or step up one rung at a
|
||||
> time through the cheapest paid options — and come back the moment it resets."
|
||||
|
||||
## Billing is a connection fact, not a model fact
|
||||
|
||||
`classifyTier()` (`open-sse/services/tierResolver.ts`) keys on `(provider, model)` and returns
|
||||
`free | cheap | premium` from catalog pricing. But whether a request costs incremental money
|
||||
depends on **which connection serves it**: the same model is plan-included through a Claude Code
|
||||
OAuth connection and billed per token through an API-key connection.
|
||||
|
||||
`provider_connections.auth_type` is not a safe proxy in either direction — metered OAuth
|
||||
connections exist, and plan-included API-key connections exist (a Copilot seat token is not a
|
||||
metered API key). So billing class comes from a **curated catalog**,
|
||||
`open-sse/config/connectionBillingCatalog.ts`, hand-set from each provider's published terms —
|
||||
the same pattern `FreeModelBudget.hardStopGuaranteed` already established for free models.
|
||||
|
||||
```ts
|
||||
type ConnectionBillingClass = "subscription" | "metered" | "keyless" | "unknown";
|
||||
type ConnectionOverageBehavior = "hard-stop" | "meters-to-paid" | "unknown";
|
||||
```
|
||||
|
||||
Resolution order (`autoCombo/connectionBilling.ts`): the synthetic no-auth sentinel →
|
||||
`keyless`; a catalog entry matching provider **and** `authType`; a provider-wide entry;
|
||||
otherwise `unknown`. **Uncurated is not free** — `unknown` is consumed as `metered`
|
||||
everywhere, so a provider added tomorrow starts outside the subscription rung and has to be
|
||||
curated in deliberately.
|
||||
|
||||
## The rung model
|
||||
|
||||
Five rungs in escalation order. They differ in more than price — each has its **own**
|
||||
exhaustion signal, which is why this is not merely a sort.
|
||||
|
||||
| # | Rung | Membership | Exhausted when |
|
||||
| --- | -------------- | -------------------------------------------------- | ----------------------------- |
|
||||
| 0 | `subscription` | curated `billing: "subscription"` | quota window at/below cutoff |
|
||||
| 1 | `keyless` | the synthetic no-auth path | connection cooldown / breaker |
|
||||
| 2 | `free` | metered connection, `classifyTier() === "free"` | free allowance exhausted |
|
||||
| 3 | `cheap` | metered connection, `classifyTier() === "cheap"` | per-rung budget consumed |
|
||||
| 4 | `premium` | metered connection, `classifyTier() === "premium"` | per-rung budget consumed |
|
||||
|
||||
Rungs 0-2 exhaust on **quota**, which is observable and already tracked. Rungs 3-4 have no
|
||||
quota — a paid connection serves forever — so their only sane exhaustion signal is a per-rung
|
||||
**budget**. Without one, "escalate when cheap is exhausted" has no trigger.
|
||||
|
||||
## `auto/subscription` — fail closed
|
||||
|
||||
Pool = rung 0 only, restricted to connections whose overage is a documented `hard-stop`, each
|
||||
verified live to have quota headroom. Everything ambiguous is excluded: an uncurated provider,
|
||||
an unverifiable quota reading, a stale reading, or an overage that meters to paid.
|
||||
|
||||
An empty pool is the **intended** answer, not a defect — the caller's existing empty-pool path
|
||||
turns it into a clear error rather than a silent, billable fallback. That is the whole promise
|
||||
of the id.
|
||||
|
||||
`keyless` deliberately does **not** qualify: this grouping means "the plan I pay for", so a
|
||||
no-auth backend does not belong in it. Use `auto/thrifty` (or `auto/best-free`) for that.
|
||||
|
||||
### Connection safety
|
||||
|
||||
A candidate is not always tied to one connection — a logical candidate carries an
|
||||
`allowedConnectionIds` allowlist, and the account actually used is chosen later, at dispatch,
|
||||
by `open-sse/services/combo/autoStrategy.ts`. Both groupings therefore verify **each connection
|
||||
individually** and rewrite `allowedConnectionIds` down to exactly the surviving subset — never
|
||||
the full original list, never one arbitrarily-chosen member. Because `autoStrategy.ts` already
|
||||
enforces that array as a hard allowlist, rewriting it here makes "verified" and "actually used"
|
||||
the same set by construction. This is the same invariant, and the same reasoning, as
|
||||
[STRICT_ZERO_COST](./STRICT_ZERO_COST.md).
|
||||
|
||||
## `auto/thrifty` — escalate one rung at a time
|
||||
|
||||
Pool = all rungs, ordered by rung index, with exhausted candidates gated out. The `auto` engine
|
||||
still scores **within** the surviving pool: the ladder decides which rungs are in play, scoring
|
||||
decides which candidate wins inside them. Ordering is stable within a rung, so the scorer's own
|
||||
ranking is never reshuffled by this overlay.
|
||||
|
||||
This is an ordering + gating overlay, **not** a new dispatcher: `combo.ts`'s speculative loop
|
||||
already walks targets in order and falls through on failure, so a runtime exhaustion the
|
||||
preflight did not catch still escalates to the next rung inside the same request.
|
||||
|
||||
Where `auto/subscription` fails closed, `auto/thrifty` fails **open**: a plan-included
|
||||
connection with no usable quota reading is still tried first. Trying it costs nothing, and if
|
||||
it turns out to be exhausted the fall-through reaches the next rung anyway — whereas refusing
|
||||
to try it would send the request to a paid rung on missing telemetry, the exact outcome the
|
||||
grouping exists to avoid.
|
||||
|
||||
## Returning to the plan after a reset
|
||||
|
||||
Three independent things must expire before routing returns to rung 0. Fixing only one leaves
|
||||
the ladder stuck on paid rungs long after the plan refilled.
|
||||
|
||||
1. **The quota-state cache** — `freeAccessQuota.ts` caches per `(provider, connection)` with a
|
||||
180s TTL. A cached entry whose own `resetAt` has already passed describes a window that no
|
||||
longer exists, so it is now treated as stale **regardless of age** and forces a refresh.
|
||||
Without this, a plan that refilled at midnight keeps reading exhausted until the TTL happens
|
||||
to lapse.
|
||||
2. **The ladder's own state** — there is none, by design. Rung eligibility is recomputed from
|
||||
live quota state on every pool build; no persisted "currently on rung 3" record exists that
|
||||
could outlive a reset and wedge routing.
|
||||
3. **The connection cooldown** — the exhausting 429 sets `rateLimitedUntil` from exponential
|
||||
backoff, which for a plan connection can overshoot the real reset. `clampCooldownToReset()`
|
||||
(`subscriptionLadder.ts`) narrows a cooldown to the upstream's own reset instant and can
|
||||
never extend one. **It is implemented and tested but not yet wired**: the quota cache is
|
||||
invalidated in `src/sse/services/auth.ts` _before_ any cooldown is written, so `resetAt`
|
||||
must be captured earlier in that function — a change to the resilience hot path that
|
||||
belongs in its own reviewed PR. Until then, re-entry waits out the connection cooldown
|
||||
(which already prefers upstream `Retry-After` hints when the provider sends them).
|
||||
|
||||
### Anti-flap
|
||||
|
||||
A rung that just reset is re-admitted only above `reentryMinRemainingPercent` (default 5),
|
||||
while a connection already in play only has to stay above `exitCutoffPercent` (default 2,
|
||||
matching `quotaPreflight.defaultThresholdPercent`). The gap is the hysteresis band — without
|
||||
it, a connection hovering at the cutoff oscillates between rungs on consecutive requests.
|
||||
|
||||
## Configuration
|
||||
|
||||
Tuning only. There is deliberately **no** `enabled` flag: a toggle able to switch these off
|
||||
would leave `auto/subscription` quietly serving the full pool — paid models included — under a
|
||||
name that promises the opposite.
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"subscriptionLadder": {
|
||||
"exitCutoffPercent": 2,
|
||||
"reentryMinRemainingPercent": 5,
|
||||
"rungBudgetUsd": { "cheap": 5.0, "premium": 0 }, // 0 disables a rung outright
|
||||
},
|
||||
}
|
||||
```
|
||||
|
||||
Budget gating is inert until a spend resolver is wired: with no accounting available a paid
|
||||
rung is ordered but never gated. As of v3.8.51 the `rungBudgetUsd` setting is accepted by the
|
||||
schema but NOT yet enforced — treat it as reserved configuration, not an active spend cap. Rung ordering, quota-based exhaustion, and reset re-entry all
|
||||
work without it.
|
||||
|
||||
## Composition
|
||||
|
||||
`subscription` and `thrifty` are `AutoTier` values, so they compose with every category:
|
||||
`auto/coding:thrifty`, `auto/reasoning:subscription`, and so on. The two flat ids
|
||||
(`auto/subscription`, `auto/thrifty`) are advertised in `/v1/models` and the dashboard.
|
||||
|
||||
Neither id is paid-tier, so `isPaidTierAutoId()` returns `false` for both and
|
||||
`auto/subscription` survives `hidePaidModels`.
|
||||
|
||||
## Where the code lives
|
||||
|
||||
| Concern | File |
|
||||
| ------------------------------- | --------------------------------------------------- |
|
||||
| Curated billing facts | `open-sse/config/connectionBillingCatalog.ts` |
|
||||
| Classifier | `open-sse/services/autoCombo/connectionBilling.ts` |
|
||||
| Rungs, both groupings, re-entry | `open-sse/services/autoCombo/subscriptionLadder.ts` |
|
||||
| Wiring into the candidate pool | `open-sse/services/autoCombo/virtualFactory.ts` |
|
||||
| Reset-aware cache staleness | `open-sse/services/autoCombo/freeAccessQuota.ts` |
|
||||
| Tier surface | `open-sse/services/autoCombo/suffixComposition.ts` |
|
||||
| Advertised ids | `open-sse/services/autoCombo/builtinCatalog.ts` |
|
||||
| Tests | `tests/unit/autoCombo/subscription-ladder.test.ts` |
|
||||
143
open-sse/config/connectionBillingCatalog.ts
Normal file
143
open-sse/config/connectionBillingCatalog.ts
Normal file
@@ -0,0 +1,143 @@
|
||||
/**
|
||||
* Curated billing classification for provider CONNECTIONS.
|
||||
*
|
||||
* The economic tier resolver (`open-sse/services/tierResolver.ts`) answers
|
||||
* "how much does this MODEL cost per token?" — a fact about the catalog. It
|
||||
* cannot answer the question subscription-first routing actually needs:
|
||||
*
|
||||
* "does serving this request through THIS connection cost incremental money,
|
||||
* or is it already covered by a flat-rate plan the operator pays anyway?"
|
||||
*
|
||||
* That is a property of the credential, not the model: `claude/claude-*` is
|
||||
* plan-included through a Claude Code OAuth connection and billed per token
|
||||
* through an API-key connection — same provider, same model, opposite
|
||||
* economics.
|
||||
*
|
||||
* `provider_connections.auth_type` alone is NOT a safe proxy in either
|
||||
* direction: metered OAuth connections exist (cloud-billed OAuth), and
|
||||
* plan-included API-key connections exist (Copilot seat tokens). So this is a
|
||||
* CURATED table, hand-set from each provider's published terms — deliberately
|
||||
* the same pattern `FreeModelBudget.hardStopGuaranteed`
|
||||
* (`open-sse/config/freeModelCatalog.ts`) already established: a fact about
|
||||
* the upstream's commercial terms, never derived from `authType` and never
|
||||
* inferred from a live API response.
|
||||
*
|
||||
* Uncurated is not "free": anything absent here resolves to `unknown`, which
|
||||
* every consumer treats as `metered`. New providers therefore start OUTSIDE
|
||||
* the subscription rung and have to be curated in deliberately — the same
|
||||
* fail-safe direction STRICT_ZERO_COST uses for uncatalogued models.
|
||||
*/
|
||||
|
||||
/**
|
||||
* How a connection's upstream charges for the requests it serves.
|
||||
*
|
||||
* - `subscription` — covered by a flat-rate plan the operator already pays.
|
||||
* Consuming its quota costs nothing incremental; the plan is a sunk cost.
|
||||
* - `metered` — pay-per-token / pay-per-credit. Every request adds spend.
|
||||
* - `keyless` — no credential exists at all, so no request can be billed by
|
||||
* construction (the synthetic no-auth path).
|
||||
* - `unknown` — not curated. Consumed as `metered` everywhere.
|
||||
*/
|
||||
export type ConnectionBillingClass = "subscription" | "metered" | "keyless" | "unknown";
|
||||
|
||||
/**
|
||||
* What happens when a subscription plan's allowance runs out.
|
||||
*
|
||||
* - `hard-stop` — the upstream refuses further requests until the window
|
||||
* resets. Exhaustion cannot cost money, so such a connection is admissible
|
||||
* to the strictest "never spend a cent extra" grouping.
|
||||
* - `meters-to-paid` — the upstream keeps serving and bills the overage.
|
||||
* Perfectly usable while quota remains, never admissible to the strict
|
||||
* grouping.
|
||||
* - `unknown` — not established. Treated exactly like `meters-to-paid` by
|
||||
* every consumer; it is the conservative default for a provider whose terms
|
||||
* allow an operator to opt into usage-based billing past the plan.
|
||||
*/
|
||||
export type ConnectionOverageBehavior = "hard-stop" | "meters-to-paid" | "unknown";
|
||||
|
||||
export interface ConnectionBillingEntry {
|
||||
/** Provider id as registered in `open-sse/config/providers/registry/`. */
|
||||
provider: string;
|
||||
/**
|
||||
* Restricts the entry to connections whose `authType` matches. Omit for a
|
||||
* provider-wide entry. A matching auth-typed entry always wins over the
|
||||
* provider-wide one, so a provider offering both a plan-included OAuth login
|
||||
* and a metered API key can declare both.
|
||||
*/
|
||||
authType?: string;
|
||||
billing: ConnectionBillingClass;
|
||||
overage: ConnectionOverageBehavior;
|
||||
/** Operator-visible justification for the classification. */
|
||||
reason: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* Curated entries. Conservative by design — a provider whose terms let the
|
||||
* operator enable usage-based billing past the plan is recorded as `unknown`
|
||||
* overage, not `hard-stop`, because the strict grouping's entire promise is
|
||||
* that it cannot surprise you.
|
||||
*
|
||||
* Providers already classified free by the economic tier resolver (`kiro`,
|
||||
* `qoder`, and the rest of `LEGACY_FREE_PROVIDERS` /
|
||||
* `deriveNoAuthFreeProviders()` in `open-sse/services/tierConfig.ts`) are
|
||||
* deliberately NOT listed here: they land on the ladder's `free` rung through
|
||||
* `classifyTier()` and would only be double-claimed by an entry here.
|
||||
*/
|
||||
export const CONNECTION_BILLING_CATALOG: readonly ConnectionBillingEntry[] = [
|
||||
{
|
||||
provider: "claude",
|
||||
authType: "oauth",
|
||||
billing: "subscription",
|
||||
overage: "hard-stop",
|
||||
reason:
|
||||
"Claude Code OAuth serves the operator's Anthropic Pro/Max plan windows. " +
|
||||
"Exceeding a window is refused until it resets; no per-token charge accrues.",
|
||||
},
|
||||
{
|
||||
provider: "codex",
|
||||
authType: "oauth",
|
||||
billing: "subscription",
|
||||
overage: "hard-stop",
|
||||
reason:
|
||||
"Codex OAuth serves the ChatGPT plan's included Codex quota. Exhaustion is " +
|
||||
"refused until the plan window resets rather than billed.",
|
||||
},
|
||||
{
|
||||
provider: "antigravity",
|
||||
authType: "oauth",
|
||||
billing: "subscription",
|
||||
overage: "hard-stop",
|
||||
reason:
|
||||
"Antigravity OAuth serves built-in plan quotas that stop serving once consumed; " +
|
||||
"OmniRoute already tracks their reset windows (see antigravityCredits.ts).",
|
||||
},
|
||||
{
|
||||
provider: "cursor",
|
||||
authType: "oauth",
|
||||
billing: "subscription",
|
||||
overage: "unknown",
|
||||
reason:
|
||||
"Cursor Pro includes a request allowance, but usage-based pricing past the plan " +
|
||||
"can be enabled per account and OmniRoute cannot observe that setting. Recorded " +
|
||||
"as unknown overage so the strict grouping excludes it.",
|
||||
},
|
||||
{
|
||||
provider: "copilot-web",
|
||||
authType: "apikey",
|
||||
billing: "subscription",
|
||||
overage: "unknown",
|
||||
reason:
|
||||
"GitHub Copilot is a per-seat subscription (the credential is a seat token, not a " +
|
||||
"metered API key), but additional premium requests can be billed when the account " +
|
||||
"opts in. Recorded as unknown overage.",
|
||||
},
|
||||
{
|
||||
provider: "devin-desktop",
|
||||
authType: "oauth",
|
||||
billing: "subscription",
|
||||
overage: "meters-to-paid",
|
||||
reason:
|
||||
"Devin Desktop draws on the plan's included ACUs and continues billing past them, " +
|
||||
"so it is plan-included while quota remains but never overage-safe.",
|
||||
},
|
||||
];
|
||||
@@ -44,6 +44,12 @@ export const AUTO_TEMPLATE_VARIANTS: Record<string, AutoVariant | undefined> = {
|
||||
"auto/claude-opus": "smart",
|
||||
"auto/claude-sonnet": "coding",
|
||||
"auto/best-free": "cheap",
|
||||
// Subscription-first routing (see `subscriptionLadder.ts`). `auto/subscription`
|
||||
// maps to no weight variant on purpose: its pool is already restricted to
|
||||
// plan-included connections, so the scorer should rank them on merit rather
|
||||
// than bias toward cheap/fast within an allowance the operator already paid for.
|
||||
"auto/subscription": undefined,
|
||||
"auto/thrifty": "cheap",
|
||||
// Chaos mode — parallel dispatch to top-N stable models
|
||||
"auto/best-chaos": "chaos",
|
||||
"auto/chaos": "chaos",
|
||||
@@ -67,6 +73,18 @@ export const AUTO_SUFFIX_VARIANTS: string[] = [
|
||||
"auto/multimodal",
|
||||
];
|
||||
|
||||
/**
|
||||
* Flat `auto/*` ids that carry a tier overlay even though they are not written
|
||||
* in `<category>:<tier>` form. `auto/best-free` established the pattern; the
|
||||
* two subscription-first ids reuse it so a caller can ask for the behavior
|
||||
* without also having to pick a category.
|
||||
*/
|
||||
export const FLAT_TIER_OVERLAY_IDS: Record<string, AutoTier> = {
|
||||
"auto/best-free": "free",
|
||||
"auto/subscription": "subscription",
|
||||
"auto/thrifty": "thrifty",
|
||||
};
|
||||
|
||||
type ResolvedAutoVariant =
|
||||
{ recognized: true; variant: AutoVariant | undefined } | { recognized: false };
|
||||
|
||||
@@ -199,8 +217,9 @@ export async function createBuiltinAutoCombo(
|
||||
}
|
||||
|
||||
if ("variant" in spec && spec.variant !== undefined) {
|
||||
const overlayTier = FLAT_TIER_OVERLAY_IDS[modelStr];
|
||||
const virtualCombo = await materialize(spec.variant, {
|
||||
...(modelStr === "auto/best-free" ? { tier: "free" as const } : {}),
|
||||
...(overlayTier ? { tier: overlayTier } : {}),
|
||||
});
|
||||
virtualCombo.name = modelStr;
|
||||
virtualCombo.id = modelStr;
|
||||
@@ -211,7 +230,11 @@ export async function createBuiltinAutoCombo(
|
||||
// auto/best-chat, auto/pro-chat) still materialize via the default
|
||||
// (unconstrained) virtual combo rather than throwing "Unknown built-in".
|
||||
if (Object.prototype.hasOwnProperty.call(AUTO_TEMPLATE_VARIANTS, modelStr)) {
|
||||
const virtualCombo = await materialize(undefined);
|
||||
const overlayTier = FLAT_TIER_OVERLAY_IDS[modelStr];
|
||||
const virtualCombo = await materialize(
|
||||
undefined,
|
||||
overlayTier ? { tier: overlayTier } : undefined
|
||||
);
|
||||
virtualCombo.name = modelStr;
|
||||
virtualCombo.id = modelStr;
|
||||
return virtualCombo;
|
||||
|
||||
100
open-sse/services/autoCombo/connectionBilling.ts
Normal file
100
open-sse/services/autoCombo/connectionBilling.ts
Normal file
@@ -0,0 +1,100 @@
|
||||
/**
|
||||
* Pure classifier over the curated connection-billing catalog
|
||||
* (`open-sse/config/connectionBillingCatalog.ts`).
|
||||
*
|
||||
* Kept dependency-light on purpose — the same constraint `paidModelFilter.ts`
|
||||
* and `strictZeroCostFilter.ts` state in their own headers — so subscription
|
||||
* routing is unit-testable without seeding the DB or the virtual factory. No
|
||||
* provider name appears in this file: a connection is classified purely from
|
||||
* the catalog plus the two facts the caller already has (`provider`,
|
||||
* `authType`), so curating a new provider needs no code change here.
|
||||
*/
|
||||
import {
|
||||
CONNECTION_BILLING_CATALOG,
|
||||
type ConnectionBillingClass,
|
||||
type ConnectionBillingEntry,
|
||||
type ConnectionOverageBehavior,
|
||||
} from "@omniroute/open-sse/config/connectionBillingCatalog.ts";
|
||||
import { SYNTHETIC_NOAUTH_CONNECTION_ID } from "./resilienceCandidateFilter";
|
||||
|
||||
/** The minimum a caller must know about a connection to classify it. */
|
||||
export interface BillableConnection {
|
||||
provider: string;
|
||||
/** `provider_connections.auth_type` — `oauth` / `apikey` / `cookie` / … */
|
||||
authType?: string | null;
|
||||
/** Connection id; the synthetic no-auth sentinel classifies as `keyless`. */
|
||||
connectionId?: string | null;
|
||||
}
|
||||
|
||||
export interface ConnectionBillingVerdict {
|
||||
billing: ConnectionBillingClass;
|
||||
overage: ConnectionOverageBehavior;
|
||||
reason: string;
|
||||
}
|
||||
|
||||
const UNKNOWN_VERDICT: ConnectionBillingVerdict = {
|
||||
billing: "unknown",
|
||||
overage: "unknown",
|
||||
reason: "No curated billing entry for this provider/authType — assumed metered.",
|
||||
};
|
||||
|
||||
const KEYLESS_VERDICT: ConnectionBillingVerdict = {
|
||||
billing: "keyless",
|
||||
overage: "hard-stop",
|
||||
reason:
|
||||
"Synthetic no-auth connection: no credential exists, so no request against it can be billed.",
|
||||
};
|
||||
|
||||
/**
|
||||
* Classify one connection.
|
||||
*
|
||||
* Resolution order, first match wins:
|
||||
* 1. the synthetic no-auth sentinel → `keyless` (no credential can be billed);
|
||||
* 2. a catalog entry matching BOTH provider and `authType`;
|
||||
* 3. a provider-wide catalog entry (no `authType` declared);
|
||||
* 4. otherwise `unknown`.
|
||||
*
|
||||
* `unknown` is never treated as free by any caller — `isPlanIncluded()` below
|
||||
* returns false for it, so an uncurated provider stays outside the
|
||||
* subscription rung until someone curates it deliberately.
|
||||
*/
|
||||
export function classifyConnectionBilling(
|
||||
connection: BillableConnection,
|
||||
catalog: readonly ConnectionBillingEntry[] = CONNECTION_BILLING_CATALOG
|
||||
): ConnectionBillingVerdict {
|
||||
if (connection.connectionId === SYNTHETIC_NOAUTH_CONNECTION_ID) return KEYLESS_VERDICT;
|
||||
|
||||
const provider = connection.provider;
|
||||
if (!provider) return UNKNOWN_VERDICT;
|
||||
|
||||
const providerEntries = catalog.filter((entry) => entry.provider === provider);
|
||||
if (providerEntries.length === 0) return UNKNOWN_VERDICT;
|
||||
|
||||
const authType = typeof connection.authType === "string" ? connection.authType : null;
|
||||
const authMatch = authType
|
||||
? providerEntries.find((entry) => entry.authType === authType)
|
||||
: undefined;
|
||||
const entry = authMatch ?? providerEntries.find((entry) => entry.authType === undefined);
|
||||
if (!entry) return UNKNOWN_VERDICT;
|
||||
|
||||
return { billing: entry.billing, overage: entry.overage, reason: entry.reason };
|
||||
}
|
||||
|
||||
/**
|
||||
* True when serving a request through this connection consumes an allowance
|
||||
* the operator already pays for, rather than adding incremental spend.
|
||||
* `keyless` qualifies: it costs nothing by construction.
|
||||
*/
|
||||
export function isPlanIncluded(verdict: ConnectionBillingVerdict): boolean {
|
||||
return verdict.billing === "subscription" || verdict.billing === "keyless";
|
||||
}
|
||||
|
||||
/**
|
||||
* True when exhausting this connection's allowance cannot start costing money.
|
||||
* The strict `auto/subscription` grouping admits nothing else: an operator who
|
||||
* asked never to spend extra must not be surprised by a provider that meters
|
||||
* past the plan, nor by one whose terms simply are not established.
|
||||
*/
|
||||
export function isOverageSafe(verdict: ConnectionBillingVerdict): boolean {
|
||||
return verdict.overage === "hard-stop";
|
||||
}
|
||||
@@ -22,6 +22,7 @@ import {
|
||||
import { getCachedProviderConnections } from "@/lib/db/readCache";
|
||||
import { defaultLogger as log } from "@omniroute/open-sse/utils/logger";
|
||||
import type { FreeAccessState } from "./strictZeroCostFilter";
|
||||
import { isStateStaleForReset } from "./subscriptionLadder";
|
||||
|
||||
const USAGE_FETCHER_PROVIDER_SET = new Set<string>(USAGE_FETCHER_PROVIDERS);
|
||||
|
||||
@@ -191,7 +192,13 @@ export function resolveFreeAccessState(
|
||||
|
||||
const key = cacheKey(provider, connectionId);
|
||||
const entry = cache.get(key);
|
||||
const fresh = entry && Date.now() - entry.fetchedAtMs <= ttlMs();
|
||||
// Subscription-first routing (decision 3): an entry whose own `resetAt` has
|
||||
// already passed describes a quota window that no longer exists, so it is
|
||||
// stale REGARDLESS of its age. Without this, a plan that refilled at
|
||||
// midnight keeps reading EXHAUSTED until the TTL happens to lapse, and
|
||||
// routing stays on paid rungs for no reason. See `subscriptionLadder.ts`.
|
||||
const resetElapsed = entry !== undefined && isStateStaleForReset(entry.state);
|
||||
const fresh = entry && !resetElapsed && Date.now() - entry.fetchedAtMs <= ttlMs();
|
||||
if (!fresh) {
|
||||
void refresh(provider, connectionId);
|
||||
}
|
||||
|
||||
411
open-sse/services/autoCombo/subscriptionLadder.ts
Normal file
411
open-sse/services/autoCombo/subscriptionLadder.ts
Normal file
@@ -0,0 +1,411 @@
|
||||
/**
|
||||
* Subscription-first routing: the rung model, its two groupings, and the
|
||||
* reset re-entry rules.
|
||||
*
|
||||
* OmniRoute already answers "is this model free?" (`paidModelFilter.ts`) and
|
||||
* "can this connection ever bill me?" (`strictZeroCostFilter.ts`). Both fail
|
||||
* CLOSED — an exhausted free pool is an empty pool, never a step up to a paid
|
||||
* option. And every paid-side mechanism (`cost-optimized`, `budgetCap`,
|
||||
* the `cost-saver` mode pack) is tier-agnostic. Neither side answers:
|
||||
*
|
||||
* "use the quota I already pay for; when it runs out either stop, or step up
|
||||
* one rung at a time; and come back the moment it resets."
|
||||
*
|
||||
* This module supplies both halves of that, sharing one rung model:
|
||||
*
|
||||
* - `filterSubscriptionOnlyCandidates` — the strict grouping (`auto/subscription`).
|
||||
* Rung 0 only, overage-safe connections only, verified live. Fails CLOSED.
|
||||
* - `orderPoolByRung` — the escalating grouping (`auto/thrifty`). All rungs,
|
||||
* ordered, with exhausted rungs gated out. Fails OPEN, one rung at a time.
|
||||
*
|
||||
* Design mirrors `strictZeroCostFilter.ts` deliberately: pure functions, the
|
||||
* live quota lookup injected as a synchronous resolver, no DB or network
|
||||
* import, and the SAME connection-safety invariant — every connection in a
|
||||
* candidate's `allowedConnectionIds` is verified INDIVIDUALLY and the array is
|
||||
* rewritten to exactly the surviving subset, never the full original list.
|
||||
* `autoStrategy.ts` enforces `allowedConnectionIds` as a hard allowlist before
|
||||
* selecting a connection at dispatch, so rewriting it here is sufficient to
|
||||
* make "verified" and "actually used" the same set by construction.
|
||||
*/
|
||||
import {
|
||||
classifyConnectionBilling,
|
||||
isOverageSafe,
|
||||
type BillableConnection,
|
||||
} from "./connectionBilling";
|
||||
import type { ConnectionBillingEntry } from "@omniroute/open-sse/config/connectionBillingCatalog.ts";
|
||||
import type { FreeAccessState } from "./strictZeroCostFilter";
|
||||
|
||||
/**
|
||||
* Rungs in escalation order. Index is the ordering key; membership is decided
|
||||
* by `assignRung` below.
|
||||
*
|
||||
* The rungs differ in more than price — each has its OWN exhaustion signal,
|
||||
* which is why this is not just a sort:
|
||||
*
|
||||
* subscription / keyless / free → exhausted on QUOTA (observable, tracked)
|
||||
* cheap / premium → exhausted on BUDGET (no quota exists; a paid
|
||||
* connection serves forever)
|
||||
*/
|
||||
export const RUNG_ORDER = ["subscription", "keyless", "free", "cheap", "premium"] as const;
|
||||
|
||||
export type LadderRung = (typeof RUNG_ORDER)[number];
|
||||
|
||||
/** Rungs whose exhaustion is observable from provider quota state. */
|
||||
const QUOTA_BEARING_RUNGS: ReadonlySet<LadderRung> = new Set<LadderRung>([
|
||||
"subscription",
|
||||
"keyless",
|
||||
"free",
|
||||
]);
|
||||
|
||||
export function rungIndex(rung: LadderRung): number {
|
||||
return RUNG_ORDER.indexOf(rung);
|
||||
}
|
||||
|
||||
/** A candidate as this module needs to see it — a structural subset of
|
||||
* `VirtualAutoComboCandidate` (`virtualFactory.ts`), so this file has no
|
||||
* dependency on that module's full type. */
|
||||
export interface LadderCandidate {
|
||||
provider: string;
|
||||
model: string;
|
||||
connectionId: string | null;
|
||||
allowedConnectionIds?: string[];
|
||||
}
|
||||
|
||||
export interface LadderOptions {
|
||||
/** Master switch. When false every exported filter is the identity function
|
||||
* — the same off-by-default contract `filterPaidOnlyCandidates` holds. */
|
||||
enabled: boolean;
|
||||
/**
|
||||
* Live allowance/quota state for ONE (provider, connection) pair, resolved
|
||||
* from the cache in `freeAccessQuota.ts`. Synchronous by design: nothing in
|
||||
* a candidate-pool build may await a network call.
|
||||
*
|
||||
* `undefined` means "no usage adapter for this provider, or nothing fresh
|
||||
* cached". The two groupings interpret that OPPOSITELY on purpose — see
|
||||
* `admitUnknownQuota` below.
|
||||
*/
|
||||
resolveFreeAccessState: (provider: string, connectionId: string) => FreeAccessState | undefined;
|
||||
/**
|
||||
* `authType` for a connection id (`provider_connections.auth_type`), needed
|
||||
* to classify billing. Unknown ids resolve to `null` → the provider-wide
|
||||
* catalog entry, or `unknown` billing.
|
||||
*/
|
||||
resolveAuthType: (connectionId: string) => string | null;
|
||||
/** Economic tier of a (provider, model) pair — `classifyTier()` in
|
||||
* production, injected so this module needs no registry/pricing import. */
|
||||
resolveEconomicTier: (provider: string, model: string) => "free" | "cheap" | "premium";
|
||||
/**
|
||||
* Remaining-percent at or below which a quota-bearing connection counts as
|
||||
* exhausted. Default 2, matching `quotaPreflight.defaultThresholdPercent`
|
||||
* (`src/lib/resilience/settings/types.ts`) so the two agree.
|
||||
*/
|
||||
exitCutoffPercent?: number;
|
||||
/**
|
||||
* Remaining-percent a quota-bearing connection must EXCEED to be re-admitted
|
||||
* after having been exhausted. Strictly greater than `exitCutoffPercent`;
|
||||
* the gap is the hysteresis band that stops a connection hovering at the
|
||||
* cutoff from oscillating between rungs on consecutive requests. Default 5.
|
||||
*/
|
||||
reentryMinRemainingPercent?: number;
|
||||
/** Max age of a `FreeAccessState.checkedAt` before it is treated as stale. */
|
||||
maxStateAgeMs: number;
|
||||
/**
|
||||
* Whether a connection with no usable quota reading is admitted.
|
||||
*
|
||||
* - `auto/thrifty` passes TRUE: trying a plan-included connection costs
|
||||
* nothing, and if it turns out to be exhausted the dispatcher's
|
||||
* fall-through reaches the next rung anyway. Refusing to try it would
|
||||
* send a request to a PAID rung on missing telemetry — the exact
|
||||
* outcome the grouping exists to avoid.
|
||||
* - `auto/subscription` passes FALSE: its promise is that no request can
|
||||
* cost extra, and an unverifiable connection cannot support that promise.
|
||||
*/
|
||||
admitUnknownQuota: boolean;
|
||||
/**
|
||||
* Budget consumed so far on a paid rung, in USD, for the operator's current
|
||||
* budget window. `null`/`undefined` means no spend accounting is available,
|
||||
* in which case paid rungs are NOT budget-gated (they still order after
|
||||
* every plan-included rung). See the spec's open question on the ledger.
|
||||
*/
|
||||
resolveRungSpendUsd?: (rung: LadderRung) => number | null;
|
||||
/** Per-rung budget in USD. A rung mapped to 0 is disabled outright. */
|
||||
rungBudgetUsd?: Partial<Record<LadderRung, number>>;
|
||||
/** `now` injection for deterministic tests. */
|
||||
now?: () => number;
|
||||
/** Catalog override for tests; production callers never pass this. */
|
||||
catalog?: readonly ConnectionBillingEntry[];
|
||||
}
|
||||
|
||||
const DEFAULT_EXIT_CUTOFF_PERCENT = 2;
|
||||
const DEFAULT_REENTRY_MIN_REMAINING_PERCENT = 5;
|
||||
|
||||
/**
|
||||
* Which rung a specific (candidate, connection) pair belongs to.
|
||||
*
|
||||
* Billing class decides first because it is the fact that actually determines
|
||||
* whether money moves; only a genuinely metered connection falls through to
|
||||
* the model's economic tier. `unknown` billing is metered by definition
|
||||
* (`connectionBilling.ts`), so an uncurated provider lands on a paid rung
|
||||
* rather than silently joining the subscription rung.
|
||||
*/
|
||||
export function assignRung(
|
||||
candidate: Pick<LadderCandidate, "provider" | "model">,
|
||||
connection: BillableConnection,
|
||||
options: Pick<LadderOptions, "resolveEconomicTier" | "catalog">
|
||||
): LadderRung {
|
||||
const verdict = classifyConnectionBilling(connection, options.catalog);
|
||||
if (verdict.billing === "subscription") return "subscription";
|
||||
if (verdict.billing === "keyless") return "keyless";
|
||||
return options.resolveEconomicTier(candidate.provider, candidate.model);
|
||||
}
|
||||
|
||||
/**
|
||||
* Is this connection's plan allowance usable right now?
|
||||
*
|
||||
* `hasBeenExhausted` selects which side of the hysteresis band applies: a
|
||||
* connection that is currently in play only has to stay above the exit cutoff,
|
||||
* while one that already dropped out has to climb back above the (higher)
|
||||
* re-entry threshold before it is admitted again.
|
||||
*/
|
||||
export function isQuotaUsable(
|
||||
state: FreeAccessState | undefined,
|
||||
options: Pick<
|
||||
LadderOptions,
|
||||
| "exitCutoffPercent"
|
||||
| "reentryMinRemainingPercent"
|
||||
| "maxStateAgeMs"
|
||||
| "admitUnknownQuota"
|
||||
| "now"
|
||||
>,
|
||||
hasBeenExhausted = false
|
||||
): boolean {
|
||||
if (!state) return options.admitUnknownQuota;
|
||||
if (state.status === "EXHAUSTED") return false;
|
||||
if (state.status === "UNKNOWN") return options.admitUnknownQuota;
|
||||
|
||||
const now = (options.now ?? Date.now)();
|
||||
const checkedAtMs = Date.parse(state.checkedAt);
|
||||
if (!Number.isFinite(checkedAtMs) || now - checkedAtMs > options.maxStateAgeMs) {
|
||||
return options.admitUnknownQuota;
|
||||
}
|
||||
|
||||
if (state.remainingFreeAllowance === null) return options.admitUnknownQuota;
|
||||
|
||||
const exitCutoff = options.exitCutoffPercent ?? DEFAULT_EXIT_CUTOFF_PERCENT;
|
||||
const reentryFloor = Math.max(
|
||||
options.reentryMinRemainingPercent ?? DEFAULT_REENTRY_MIN_REMAINING_PERCENT,
|
||||
exitCutoff
|
||||
);
|
||||
const threshold = hasBeenExhausted ? reentryFloor : exitCutoff;
|
||||
return state.remainingFreeAllowance > threshold;
|
||||
}
|
||||
|
||||
/**
|
||||
* Decision 3 — re-entry after a plan quota resets.
|
||||
*
|
||||
* A cached state whose own `resetAt` has already passed describes a window
|
||||
* that no longer exists. Waiting out the cache TTL before re-reading it is
|
||||
* pure lag on the single transition subscription-first routing cares most
|
||||
* about, so such an entry is stale REGARDLESS of its age.
|
||||
*
|
||||
* Consumed by `freeAccessQuota.ts`, which owns the cache; kept here so the
|
||||
* rule sits with the rest of the ladder's semantics and is testable without
|
||||
* touching the cache.
|
||||
*/
|
||||
export function isStateStaleForReset(
|
||||
state: Pick<FreeAccessState, "resetAt"> | undefined,
|
||||
now: number = Date.now()
|
||||
): boolean {
|
||||
if (!state?.resetAt) return false;
|
||||
const resetAtMs = Date.parse(state.resetAt);
|
||||
if (!Number.isFinite(resetAtMs)) return false;
|
||||
return resetAtMs <= now;
|
||||
}
|
||||
|
||||
/**
|
||||
* Decision 3, second half — never hold a plan-included connection in cooldown
|
||||
* past the moment its own upstream says the quota is back.
|
||||
*
|
||||
* The exhausting 429 sets `rateLimitedUntil` from exponential backoff
|
||||
* (`baseCooldownMs * 2 ** failureIndex`, `src/sse/services/auth.ts`), which for
|
||||
* a subscription connection routinely overshoots the real reset — leaving
|
||||
* routing stuck on paid rungs long after the plan refilled.
|
||||
*
|
||||
* This only ever NARROWS a cooldown, and only when the upstream itself
|
||||
* supplied the reset instant. An absent, unparseable, or already-past
|
||||
* `resetAt` returns the original cooldown untouched.
|
||||
*/
|
||||
export function clampCooldownToReset(
|
||||
cooldownMs: number,
|
||||
resetAt: string | null | undefined,
|
||||
now: number = Date.now()
|
||||
): number {
|
||||
if (!resetAt) return cooldownMs;
|
||||
const resetAtMs = Date.parse(resetAt);
|
||||
if (!Number.isFinite(resetAtMs)) return cooldownMs;
|
||||
const untilResetMs = resetAtMs - now;
|
||||
if (untilResetMs <= 0) return cooldownMs;
|
||||
return Math.min(cooldownMs, untilResetMs);
|
||||
}
|
||||
|
||||
/** True when a paid rung has consumed its configured budget for the window. */
|
||||
export function isRungBudgetExhausted(rung: LadderRung, options: LadderOptions): boolean {
|
||||
const budget = options.rungBudgetUsd?.[rung];
|
||||
if (budget === undefined) return false;
|
||||
if (budget <= 0) return true; // explicitly disabled
|
||||
const spent = options.resolveRungSpendUsd?.(rung);
|
||||
if (spent === null || spent === undefined) return false; // no accounting → not gated
|
||||
return spent >= budget;
|
||||
}
|
||||
|
||||
/**
|
||||
* Connections on a candidate that are usable right now, paired with the rung
|
||||
* each one sits on. Quota-bearing rungs are verified per connection; paid
|
||||
* rungs have nothing per-connection to verify (they are gated per rung).
|
||||
*/
|
||||
function evaluateConnections(
|
||||
candidate: LadderCandidate,
|
||||
options: LadderOptions,
|
||||
accept: (rung: LadderRung) => boolean
|
||||
): { rung: LadderRung; connectionIds: string[] } | null {
|
||||
const connectionIds = candidate.connectionId
|
||||
? [candidate.connectionId]
|
||||
: (candidate.allowedConnectionIds ?? []);
|
||||
if (connectionIds.length === 0) return null;
|
||||
|
||||
let bestRung: LadderRung | null = null;
|
||||
const usable: string[] = [];
|
||||
|
||||
for (const connectionId of connectionIds) {
|
||||
const rung = assignRung(
|
||||
candidate,
|
||||
{
|
||||
provider: candidate.provider,
|
||||
authType: options.resolveAuthType(connectionId),
|
||||
connectionId,
|
||||
},
|
||||
options
|
||||
);
|
||||
if (!accept(rung)) continue;
|
||||
|
||||
if (QUOTA_BEARING_RUNGS.has(rung)) {
|
||||
const state = options.resolveFreeAccessState(candidate.provider, connectionId);
|
||||
if (!isQuotaUsable(state, options)) continue;
|
||||
} else if (isRungBudgetExhausted(rung, options)) {
|
||||
continue;
|
||||
}
|
||||
|
||||
usable.push(connectionId);
|
||||
// A candidate reachable through several accounts is represented by its
|
||||
// CHEAPEST usable rung: that is the rung a request through it would
|
||||
// actually land on once dispatch picks from the surviving allowlist.
|
||||
if (bestRung === null || rungIndex(rung) < rungIndex(bestRung)) bestRung = rung;
|
||||
}
|
||||
|
||||
if (usable.length === 0 || bestRung === null) return null;
|
||||
return { rung: bestRung, connectionIds: usable };
|
||||
}
|
||||
|
||||
/** Rewrite a candidate's connection allowlist to the verified subset, keeping
|
||||
* the identity-when-nothing-changed contract the sibling filters hold. */
|
||||
function withVerifiedConnections<T extends LadderCandidate>(
|
||||
candidate: T,
|
||||
connectionIds: string[]
|
||||
): { candidate: T; changed: boolean } {
|
||||
if (candidate.connectionId !== null) return { candidate, changed: false };
|
||||
const original = candidate.allowedConnectionIds ?? [];
|
||||
const isSameSet =
|
||||
original.length === connectionIds.length && connectionIds.every((id) => original.includes(id));
|
||||
if (isSameSet) return { candidate, changed: false };
|
||||
return { candidate: { ...candidate, allowedConnectionIds: connectionIds }, changed: true };
|
||||
}
|
||||
|
||||
/**
|
||||
* `auto/subscription` — the strict grouping. Keeps only candidates servable by
|
||||
* a plan-included connection whose overage is a documented hard stop, with
|
||||
* live quota headroom verified per connection.
|
||||
*
|
||||
* Fails CLOSED in every ambiguous case: uncurated provider, unverifiable
|
||||
* quota, or an overage that meters to paid. An empty result is the correct,
|
||||
* intended answer for an operator who asked never to spend extra — the
|
||||
* caller's existing empty-pool path handles it, exactly as `hidePaidModels`
|
||||
* already does.
|
||||
*/
|
||||
export function filterSubscriptionOnlyCandidates<T extends LadderCandidate>(
|
||||
pool: T[],
|
||||
options: LadderOptions
|
||||
): T[] {
|
||||
if (!options.enabled) return pool;
|
||||
|
||||
const strictOptions: LadderOptions = { ...options, admitUnknownQuota: false };
|
||||
const kept: T[] = [];
|
||||
let changed = false;
|
||||
|
||||
for (const candidate of pool) {
|
||||
const connectionIds = candidate.connectionId
|
||||
? [candidate.connectionId]
|
||||
: (candidate.allowedConnectionIds ?? []);
|
||||
|
||||
const safe = connectionIds.filter((connectionId) => {
|
||||
const connection: BillableConnection = {
|
||||
provider: candidate.provider,
|
||||
authType: options.resolveAuthType(connectionId),
|
||||
connectionId,
|
||||
};
|
||||
const verdict = classifyConnectionBilling(connection, options.catalog);
|
||||
// `keyless` is plan-included in the ladder's sense but is NOT a
|
||||
// subscription: this grouping is "the plan I pay for", so a no-auth
|
||||
// backend does not belong in it.
|
||||
if (verdict.billing !== "subscription") return false;
|
||||
if (!isOverageSafe(verdict)) return false;
|
||||
const state = strictOptions.resolveFreeAccessState(candidate.provider, connectionId);
|
||||
return isQuotaUsable(state, strictOptions);
|
||||
});
|
||||
|
||||
if (safe.length === 0) {
|
||||
changed = true;
|
||||
continue;
|
||||
}
|
||||
const result = withVerifiedConnections(candidate, safe);
|
||||
if (result.changed) changed = true;
|
||||
kept.push(result.candidate);
|
||||
}
|
||||
|
||||
return changed ? kept : pool;
|
||||
}
|
||||
|
||||
/**
|
||||
* `auto/thrifty` — the escalating grouping. Returns the pool ordered by rung,
|
||||
* with candidates whose every connection is exhausted (quota) or whose rung is
|
||||
* budget-exhausted removed.
|
||||
*
|
||||
* Ordering only — the `auto` engine still scores WITHIN the surviving pool, so
|
||||
* this decides which rungs are in play, not which candidate wins on one. The
|
||||
* combo dispatcher already walks targets in order and falls through on
|
||||
* failure, so a runtime exhaustion the preflight did not catch still escalates
|
||||
* to the next rung inside the same request.
|
||||
*
|
||||
* Rung eligibility is recomputed from live state on every pool build and
|
||||
* nothing is persisted: there is deliberately no sticky "currently on rung 3"
|
||||
* record that could outlive a quota reset and wedge routing on paid rungs.
|
||||
*/
|
||||
export function orderPoolByRung<T extends LadderCandidate>(pool: T[], options: LadderOptions): T[] {
|
||||
if (!options.enabled) return pool;
|
||||
|
||||
const ranked: Array<{ candidate: T; rung: LadderRung; order: number }> = [];
|
||||
for (const [order, candidate] of pool.entries()) {
|
||||
const evaluated = evaluateConnections(candidate, options, () => true);
|
||||
if (!evaluated) continue;
|
||||
const result = withVerifiedConnections(candidate, evaluated.connectionIds);
|
||||
ranked.push({ candidate: result.candidate, rung: evaluated.rung, order });
|
||||
}
|
||||
|
||||
ranked.sort((a, b) => {
|
||||
const byRung = rungIndex(a.rung) - rungIndex(b.rung);
|
||||
// Stable within a rung: preserve the pool's incoming order so the auto
|
||||
// scorer's own ranking is not reshuffled by this overlay.
|
||||
return byRung !== 0 ? byRung : a.order - b.order;
|
||||
});
|
||||
|
||||
return ranked.map((entry) => entry.candidate);
|
||||
}
|
||||
@@ -23,7 +23,20 @@ import { isVisionModelId } from "@/shared/constants/visionModels";
|
||||
import { isVisionBridgeForcedModel } from "@/shared/constants/visionBridgeDefaults";
|
||||
|
||||
export type AutoCategory = "coding" | "reasoning" | "vision" | "chat" | "multimodal";
|
||||
export type AutoTier = "fast" | "cheap" | "floor" | "free" | "reliable" | "pro";
|
||||
export type AutoTier =
|
||||
| "fast"
|
||||
| "cheap"
|
||||
| "floor"
|
||||
| "free"
|
||||
| "reliable"
|
||||
| "pro"
|
||||
// Subscription-first routing. Unlike every tier above, these two narrow by
|
||||
// the CONNECTION's billing class, not the model's price — so they are
|
||||
// applied in `virtualFactory.ts` against live connection state rather than
|
||||
// by `buildAutoCandidateFilter` below, which only sees (provider, model).
|
||||
// See `subscriptionLadder.ts` and `docs/routing/SUBSCRIPTION_LADDER.md`.
|
||||
| "subscription"
|
||||
| "thrifty";
|
||||
|
||||
export const AUTO_CATEGORIES: readonly AutoCategory[] = [
|
||||
"coding",
|
||||
@@ -39,6 +52,8 @@ export const AUTO_TIERS: readonly AutoTier[] = [
|
||||
"free",
|
||||
"reliable",
|
||||
"pro",
|
||||
"subscription",
|
||||
"thrifty",
|
||||
];
|
||||
|
||||
const CATEGORY_SET = new Set<string>(AUTO_CATEGORIES);
|
||||
@@ -84,6 +99,9 @@ export function tierToWeightVariant(tier?: AutoTier): AutoVariant | "reliability
|
||||
return "fast";
|
||||
case "cheap":
|
||||
case "floor":
|
||||
// The ladder already orders plan-included rungs first; within a rung it
|
||||
// should still lean cheap rather than reach for the most expensive model.
|
||||
case "thrifty":
|
||||
return "cheap";
|
||||
case "reliable":
|
||||
return "reliability";
|
||||
@@ -118,8 +136,7 @@ export function buildAutoCandidateFilter(
|
||||
}
|
||||
try {
|
||||
const caps = getResolvedModelCapabilities({ provider: c.provider, model: c.model });
|
||||
const capable =
|
||||
caps.supportsVision === true || isVisionModelId(c.model);
|
||||
const capable = caps.supportsVision === true || isVisionModelId(c.model);
|
||||
if (!capable) return false;
|
||||
// #vison-pool: registry entries whose catalog OVERSTATES vision support
|
||||
// (opencode-go/opencode-zen/tokenrouter — the backend models are text-only)
|
||||
|
||||
@@ -28,6 +28,11 @@ import { buildFamilyCandidateFilter, type ModelFamily } from "./modelFamily";
|
||||
import { getHiddenModelsByProvider } from "@/models";
|
||||
import { getSyncedAvailableModelsByConnection, getCustomModels } from "@/lib/db/models";
|
||||
import { filterPaidOnlyCandidates } from "./paidModelFilter";
|
||||
import {
|
||||
filterSubscriptionOnlyCandidates,
|
||||
orderPoolByRung,
|
||||
type LadderOptions,
|
||||
} from "./subscriptionLadder";
|
||||
import { filterStrictZeroCostCandidates, filterTosAvoidCandidates } from "./strictZeroCostFilter";
|
||||
import { resolveFreeAccessState } from "./freeAccessQuota";
|
||||
import { isModelExcludedByConnection } from "@/domain/connectionModelRules";
|
||||
@@ -145,6 +150,98 @@ type VirtualAutoCombo = AutoComboConfig & {
|
||||
export interface PreparedVirtualAutoComboInputs {
|
||||
readonly regularCandidates: readonly VirtualAutoComboCandidate[];
|
||||
readonly familyCandidates: readonly VirtualAutoComboCandidate[];
|
||||
/**
|
||||
* `provider_connections.auth_type` per connection id. Subscription-first
|
||||
* routing classifies billing per CONNECTION (`connectionBilling.ts`), and
|
||||
* the candidate pool only carries connection ids — so the auth types are
|
||||
* captured here, during the one bulk connection read this function already
|
||||
* does, instead of re-reading the DB per pool narrowing.
|
||||
*/
|
||||
readonly authTypeByConnectionId?: ReadonlyMap<string, string | null>;
|
||||
/** Operator settings for the subscription ladder; absent = feature off. */
|
||||
readonly subscriptionLadder?: SubscriptionLadderSettings;
|
||||
}
|
||||
|
||||
/**
|
||||
* Operator-facing knobs for subscription-first routing. Mirrors the Zod shape
|
||||
* in `src/shared/validation/settingsSchemas.ts`.
|
||||
*
|
||||
* Deliberately TUNING ONLY — there is no `enabled` flag here. `auto/subscription`
|
||||
* and `auto/thrifty` are new ids that nothing routes through unless a caller
|
||||
* asks for them by name, so requesting the id IS the opt-in. A settings toggle
|
||||
* that could switch them off would be actively dangerous: it would leave an id
|
||||
* whose whole promise is "plan-included only" quietly serving the full pool,
|
||||
* paid models included.
|
||||
*/
|
||||
export interface SubscriptionLadderSettings {
|
||||
exitCutoffPercent?: number;
|
||||
reentryMinRemainingPercent?: number;
|
||||
rungBudgetUsd?: Record<string, number>;
|
||||
/** Staleness bound for a cached quota reading, derived from the existing
|
||||
* `autoRefreshProviderQuotaInterval` exactly as STRICT_ZERO_COST does. */
|
||||
maxStateAgeMs: number;
|
||||
}
|
||||
|
||||
function readSubscriptionLadderSettings(
|
||||
settings: Record<string, unknown>
|
||||
): SubscriptionLadderSettings {
|
||||
const maxStateAgeMs = (Number(settings.autoRefreshProviderQuotaInterval) || 180) * 1000;
|
||||
const raw = settings.subscriptionLadder;
|
||||
if (!raw || typeof raw !== "object") return { maxStateAgeMs };
|
||||
const value = raw as Record<string, unknown>;
|
||||
const numeric = (key: string): number | undefined =>
|
||||
typeof value[key] === "number" && Number.isFinite(value[key] as number)
|
||||
? (value[key] as number)
|
||||
: undefined;
|
||||
const exitCutoffPercent = numeric("exitCutoffPercent");
|
||||
const reentryMinRemainingPercent = numeric("reentryMinRemainingPercent");
|
||||
return {
|
||||
maxStateAgeMs,
|
||||
...(exitCutoffPercent === undefined ? {} : { exitCutoffPercent }),
|
||||
...(reentryMinRemainingPercent === undefined ? {} : { reentryMinRemainingPercent }),
|
||||
...(value.rungBudgetUsd && typeof value.rungBudgetUsd === "object"
|
||||
? { rungBudgetUsd: value.rungBudgetUsd as Record<string, number> }
|
||||
: {}),
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Build the injected dependencies the pure ladder module needs. Everything it
|
||||
* touches is resolved here — the live quota cache, connection auth types, and
|
||||
* the economic tier resolver — so `subscriptionLadder.ts` itself stays free of
|
||||
* DB, network, and registry imports.
|
||||
*/
|
||||
function buildLadderOptions(
|
||||
prepared: PreparedVirtualAutoComboInputs,
|
||||
tier: "subscription" | "thrifty"
|
||||
): LadderOptions {
|
||||
const tuning = prepared.subscriptionLadder;
|
||||
const authTypes = prepared.authTypeByConnectionId;
|
||||
return {
|
||||
enabled: true,
|
||||
resolveFreeAccessState,
|
||||
resolveAuthType: (connectionId: string) => authTypes?.get(connectionId) ?? null,
|
||||
resolveEconomicTier: (provider: string, model: string) => {
|
||||
try {
|
||||
const resolved = classifyTier(provider, model).tier;
|
||||
return resolved === "free" || resolved === "premium" ? resolved : "cheap";
|
||||
} catch {
|
||||
// Same conservative default `safeClassifyTier` uses in suffixComposition.
|
||||
return "cheap";
|
||||
}
|
||||
},
|
||||
maxStateAgeMs: tuning?.maxStateAgeMs ?? 180_000,
|
||||
// The two groupings read a missing quota reading OPPOSITELY on purpose —
|
||||
// see `LadderOptions.admitUnknownQuota`.
|
||||
admitUnknownQuota: tier === "thrifty",
|
||||
...(tuning?.exitCutoffPercent === undefined
|
||||
? {}
|
||||
: { exitCutoffPercent: tuning.exitCutoffPercent }),
|
||||
...(tuning?.reentryMinRemainingPercent === undefined
|
||||
? {}
|
||||
: { reentryMinRemainingPercent: tuning.reentryMinRemainingPercent }),
|
||||
...(tuning?.rungBudgetUsd ? { rungBudgetUsd: tuning.rungBudgetUsd } : {}),
|
||||
};
|
||||
}
|
||||
|
||||
function toExpiryMs(value: unknown): number | null {
|
||||
@@ -656,8 +753,15 @@ export async function prepareVirtualAutoComboInputs(
|
||||
const regularCandidates = buildPreparedPool(false);
|
||||
// #6453/#8183: family selectors bypass the reliability-curated no-auth allowlist.
|
||||
const familyCandidates = buildPreparedPool(true);
|
||||
// Subscription-first routing inputs, captured from the connection read above
|
||||
// so no later stage has to touch the DB again.
|
||||
const authTypeByConnectionId = new Map<string, string | null>();
|
||||
for (const conn of connections) {
|
||||
authTypeByConnectionId.set(conn.id, typeof conn.authType === "string" ? conn.authType : null);
|
||||
}
|
||||
const subscriptionLadder = readSubscriptionLadderSettings(settings);
|
||||
if (!options.includeResolvedCapabilities) {
|
||||
return { regularCandidates, familyCandidates };
|
||||
return { regularCandidates, familyCandidates, authTypeByConnectionId, subscriptionLadder };
|
||||
}
|
||||
|
||||
// One uninterrupted bulk read of all three capability tables for this prepare only.
|
||||
@@ -671,6 +775,8 @@ export async function prepareVirtualAutoComboInputs(
|
||||
return {
|
||||
regularCandidates: await attachPreparedCapabilityValues(regularCandidates, capabilityState),
|
||||
familyCandidates: await attachPreparedCapabilityValues(familyCandidates, capabilityState),
|
||||
authTypeByConnectionId,
|
||||
subscriptionLadder,
|
||||
};
|
||||
}
|
||||
|
||||
@@ -842,6 +948,32 @@ export async function createVirtualAutoComboFromPrepared(
|
||||
}
|
||||
}
|
||||
|
||||
// Subscription-first routing (`auto/subscription`, `auto/thrifty`). Applied
|
||||
// AFTER the category/tier narrowing above because, unlike every other tier,
|
||||
// these two select on the connection's billing class and its live quota
|
||||
// state rather than on the model's catalog price — see
|
||||
// `subscriptionLadder.ts` and `docs/routing/SUBSCRIPTION_LADDER.md`.
|
||||
if (spec?.tier === "subscription" || spec?.tier === "thrifty") {
|
||||
const ladderOptions = buildLadderOptions(prepared, spec.tier);
|
||||
const beforeCount = effectivePool.length;
|
||||
effectivePool =
|
||||
spec.tier === "subscription"
|
||||
? filterSubscriptionOnlyCandidates(effectivePool, ladderOptions)
|
||||
: orderPoolByRung(effectivePool, ladderOptions);
|
||||
if (spec.tier === "subscription" && effectivePool.length === 0 && beforeCount > 0) {
|
||||
// Intended, not a defect: the operator asked for plan-included capacity
|
||||
// only, and right now there is none with verified headroom. Failing
|
||||
// closed here is the entire promise of the id — the caller's existing
|
||||
// empty-pool path turns it into a clear error rather than a silent,
|
||||
// billable fallback.
|
||||
warnEmptyAutoPoolOnce(
|
||||
"auto/subscription",
|
||||
"auto/subscription: no plan-included connection has verified quota headroom; " +
|
||||
"returning an empty pool rather than falling back to paid capacity."
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
let weights: ScoringWeights = { ...DEFAULT_WEIGHTS };
|
||||
let explorationRate = 0.05; // Default exploration rate
|
||||
let routerStrategy = "lkgp"; // All auto variants use LKGP
|
||||
|
||||
@@ -138,6 +138,24 @@ export const updateSettingsSchema = z.object({
|
||||
// curated `tos` verdict is "avoid" (proxy/self-hosted use conflicts with the
|
||||
// provider's own terms) — a contractual concern, not an economic one.
|
||||
excludeTosAvoid: z.boolean().optional(),
|
||||
// Subscription-first routing tuning (`auto/subscription`, `auto/thrifty`).
|
||||
// TUNING ONLY — there is deliberately no `enabled` flag: both ids are opt-in
|
||||
// by being requested, and a toggle able to switch them off would leave
|
||||
// `auto/subscription` silently serving paid capacity under a name that
|
||||
// promises the opposite. See open-sse/services/autoCombo/subscriptionLadder.ts.
|
||||
subscriptionLadder: z
|
||||
.object({
|
||||
// Remaining-% at or below which a plan-included connection counts as
|
||||
// exhausted. Matches quotaPreflight.defaultThresholdPercent's default.
|
||||
exitCutoffPercent: z.number().min(0).max(100).optional(),
|
||||
// Remaining-% a connection must EXCEED to be re-admitted after having
|
||||
// been exhausted. The gap above exitCutoffPercent is the hysteresis band
|
||||
// that stops a connection hovering at the cutoff from oscillating.
|
||||
reentryMinRemainingPercent: z.number().min(0).max(100).optional(),
|
||||
// Per-rung spend ceiling in USD. 0 disables a rung outright.
|
||||
rungBudgetUsd: z.record(z.string().max(32), z.number().min(0)).optional(),
|
||||
})
|
||||
.optional(),
|
||||
hideHealthCheckLogs: z.boolean().optional(),
|
||||
hideEndpointCloudflaredTunnel: z.boolean().optional(),
|
||||
hideEndpointTailscaleFunnel: z.boolean().optional(),
|
||||
|
||||
407
tests/unit/autoCombo/subscription-ladder.test.ts
Normal file
407
tests/unit/autoCombo/subscription-ladder.test.ts
Normal file
@@ -0,0 +1,407 @@
|
||||
/**
|
||||
* Subscription-first routing — regression guard for `connectionBilling.ts` and
|
||||
* `subscriptionLadder.ts`, wired into
|
||||
* `open-sse/services/autoCombo/virtualFactory.ts::createVirtualAutoComboFromPrepared`
|
||||
* for the `auto/subscription` and `auto/thrifty` ids.
|
||||
*
|
||||
* Pure and dependency-light by design, mirroring
|
||||
* `strict-zero-cost-filter.test.ts`: every side-effecting dependency (live
|
||||
* quota state, connection auth types, the economic tier resolver, the billing
|
||||
* catalog) is injected, so nothing here touches the DB, the network, or global
|
||||
* state.
|
||||
*/
|
||||
import { test } from "vitest";
|
||||
import assert from "node:assert/strict";
|
||||
|
||||
import type { ConnectionBillingEntry } from "@omniroute/open-sse/config/connectionBillingCatalog.ts";
|
||||
import {
|
||||
classifyConnectionBilling,
|
||||
isOverageSafe,
|
||||
isPlanIncluded,
|
||||
} from "@omniroute/open-sse/services/autoCombo/connectionBilling.ts";
|
||||
import { SYNTHETIC_NOAUTH_CONNECTION_ID } from "@omniroute/open-sse/services/autoCombo/resilienceCandidateFilter.ts";
|
||||
import {
|
||||
RUNG_ORDER,
|
||||
assignRung,
|
||||
clampCooldownToReset,
|
||||
filterSubscriptionOnlyCandidates,
|
||||
isQuotaUsable,
|
||||
isStateStaleForReset,
|
||||
orderPoolByRung,
|
||||
type LadderCandidate,
|
||||
type LadderOptions,
|
||||
} from "@omniroute/open-sse/services/autoCombo/subscriptionLadder.ts";
|
||||
import type { FreeAccessState } from "@omniroute/open-sse/services/autoCombo/strictZeroCostFilter.ts";
|
||||
|
||||
const NOW = Date.parse("2026-08-22T12:00:00.000Z");
|
||||
|
||||
/** Synthetic catalog — never the real one, so these tests keep passing when
|
||||
* the curated entries are edited (the autodiscovery contract). */
|
||||
const CATALOG: readonly ConnectionBillingEntry[] = [
|
||||
{
|
||||
provider: "planned",
|
||||
authType: "oauth",
|
||||
billing: "subscription",
|
||||
overage: "hard-stop",
|
||||
reason: "test fixture: plan-included, refuses past the allowance",
|
||||
},
|
||||
{
|
||||
provider: "planned",
|
||||
authType: "apikey",
|
||||
billing: "metered",
|
||||
overage: "meters-to-paid",
|
||||
reason: "test fixture: same provider, metered credential",
|
||||
},
|
||||
{
|
||||
provider: "overflowing",
|
||||
billing: "subscription",
|
||||
overage: "meters-to-paid",
|
||||
reason: "test fixture: plan-included but bills past the allowance",
|
||||
},
|
||||
{
|
||||
provider: "metered-co",
|
||||
billing: "metered",
|
||||
overage: "meters-to-paid",
|
||||
reason: "test fixture: pay per token",
|
||||
},
|
||||
];
|
||||
|
||||
function state(overrides: Partial<FreeAccessState> = {}): FreeAccessState {
|
||||
return {
|
||||
status: "SAFE",
|
||||
remainingFreeAllowance: 50,
|
||||
resetAt: null,
|
||||
checkedAt: new Date(NOW - 1000).toISOString(),
|
||||
...overrides,
|
||||
};
|
||||
}
|
||||
|
||||
function options(overrides: Partial<LadderOptions> = {}): LadderOptions {
|
||||
return {
|
||||
enabled: true,
|
||||
resolveFreeAccessState: () => state(),
|
||||
resolveAuthType: () => "oauth",
|
||||
resolveEconomicTier: () => "cheap",
|
||||
maxStateAgeMs: 180_000,
|
||||
admitUnknownQuota: false,
|
||||
now: () => NOW,
|
||||
catalog: CATALOG,
|
||||
...overrides,
|
||||
};
|
||||
}
|
||||
|
||||
function candidate(overrides: Partial<LadderCandidate> = {}): LadderCandidate {
|
||||
return {
|
||||
provider: "planned",
|
||||
model: "m1",
|
||||
connectionId: "c1",
|
||||
...overrides,
|
||||
};
|
||||
}
|
||||
|
||||
// ── classification ──────────────────────────────────────────────────────────
|
||||
|
||||
test("the synthetic no-auth connection classifies as keyless without consulting the catalog", () => {
|
||||
const verdict = classifyConnectionBilling(
|
||||
{ provider: "metered-co", connectionId: SYNTHETIC_NOAUTH_CONNECTION_ID },
|
||||
CATALOG
|
||||
);
|
||||
assert.equal(verdict.billing, "keyless");
|
||||
assert.equal(isPlanIncluded(verdict), true);
|
||||
});
|
||||
|
||||
test("an authType-specific entry wins over the provider-wide one", () => {
|
||||
const oauth = classifyConnectionBilling(
|
||||
{ provider: "planned", authType: "oauth", connectionId: "c1" },
|
||||
CATALOG
|
||||
);
|
||||
const apikey = classifyConnectionBilling(
|
||||
{ provider: "planned", authType: "apikey", connectionId: "c2" },
|
||||
CATALOG
|
||||
);
|
||||
assert.equal(oauth.billing, "subscription");
|
||||
assert.equal(apikey.billing, "metered");
|
||||
});
|
||||
|
||||
test("a provider-wide entry applies when no authType entry matches", () => {
|
||||
const verdict = classifyConnectionBilling(
|
||||
{ provider: "overflowing", authType: "cookie", connectionId: "c1" },
|
||||
CATALOG
|
||||
);
|
||||
assert.equal(verdict.billing, "subscription");
|
||||
assert.equal(isOverageSafe(verdict), false);
|
||||
});
|
||||
|
||||
test("an uncurated provider is unknown — never silently plan-included", () => {
|
||||
const verdict = classifyConnectionBilling(
|
||||
{ provider: "brand-new", authType: "oauth", connectionId: "c1" },
|
||||
CATALOG
|
||||
);
|
||||
assert.equal(verdict.billing, "unknown");
|
||||
assert.equal(isPlanIncluded(verdict), false);
|
||||
assert.equal(isOverageSafe(verdict), false);
|
||||
});
|
||||
|
||||
test("rung assignment prefers billing class, falling back to the economic tier", () => {
|
||||
const opts = options({ resolveEconomicTier: () => "premium" });
|
||||
assert.equal(
|
||||
assignRung(
|
||||
{ provider: "planned", model: "m1" },
|
||||
{ provider: "planned", authType: "oauth" },
|
||||
opts
|
||||
),
|
||||
"subscription"
|
||||
);
|
||||
assert.equal(
|
||||
assignRung(
|
||||
{ provider: "metered-co", model: "m1" },
|
||||
{ provider: "metered-co", authType: "apikey" },
|
||||
opts
|
||||
),
|
||||
"premium"
|
||||
);
|
||||
assert.equal(
|
||||
assignRung(
|
||||
{ provider: "whatever", model: "m1" },
|
||||
{ provider: "whatever", connectionId: SYNTHETIC_NOAUTH_CONNECTION_ID },
|
||||
opts
|
||||
),
|
||||
"keyless"
|
||||
);
|
||||
});
|
||||
|
||||
// ── auto/subscription — fails closed ────────────────────────────────────────
|
||||
|
||||
test("disabled leaves the pool byte-identical (the opt-in contract)", () => {
|
||||
const pool = [candidate()];
|
||||
assert.equal(filterSubscriptionOnlyCandidates(pool, options({ enabled: false })), pool);
|
||||
assert.equal(orderPoolByRung(pool, options({ enabled: false })), pool);
|
||||
});
|
||||
|
||||
test("a plan-included, hard-stop connection with headroom is kept", () => {
|
||||
const pool = [candidate()];
|
||||
assert.deepEqual(filterSubscriptionOnlyCandidates(pool, options()), pool);
|
||||
});
|
||||
|
||||
test("a subscription that meters past the plan is excluded", () => {
|
||||
const pool = [candidate({ provider: "overflowing" })];
|
||||
assert.deepEqual(filterSubscriptionOnlyCandidates(pool, options()), []);
|
||||
});
|
||||
|
||||
test("a metered connection is excluded even on a provider that also sells a plan", () => {
|
||||
const pool = [candidate({ connectionId: "c2" })];
|
||||
const result = filterSubscriptionOnlyCandidates(
|
||||
pool,
|
||||
options({ resolveAuthType: () => "apikey" })
|
||||
);
|
||||
assert.deepEqual(result, []);
|
||||
});
|
||||
|
||||
test("keyless is not a subscription — auto/subscription means the plan you pay for", () => {
|
||||
const pool = [candidate({ provider: "freebie", connectionId: SYNTHETIC_NOAUTH_CONNECTION_ID })];
|
||||
assert.deepEqual(filterSubscriptionOnlyCandidates(pool, options()), []);
|
||||
});
|
||||
|
||||
test("an unverifiable quota reading fails closed", () => {
|
||||
const pool = [candidate()];
|
||||
assert.deepEqual(
|
||||
filterSubscriptionOnlyCandidates(pool, options({ resolveFreeAccessState: () => undefined })),
|
||||
[]
|
||||
);
|
||||
assert.deepEqual(
|
||||
filterSubscriptionOnlyCandidates(
|
||||
pool,
|
||||
options({ resolveFreeAccessState: () => state({ status: "UNKNOWN" }) })
|
||||
),
|
||||
[]
|
||||
);
|
||||
});
|
||||
|
||||
test("a stale quota reading fails closed even when it says SAFE", () => {
|
||||
const stale = state({ checkedAt: new Date(NOW - 10 * 60_000).toISOString() });
|
||||
assert.deepEqual(
|
||||
filterSubscriptionOnlyCandidates(
|
||||
[candidate()],
|
||||
options({ resolveFreeAccessState: () => stale })
|
||||
),
|
||||
[]
|
||||
);
|
||||
});
|
||||
|
||||
test("a multi-account candidate keeps only the connections proven safe", () => {
|
||||
const pool = [candidate({ connectionId: null, allowedConnectionIds: ["a", "b", "c"] })];
|
||||
const result = filterSubscriptionOnlyCandidates(
|
||||
pool,
|
||||
options({
|
||||
resolveFreeAccessState: (_provider, connectionId) =>
|
||||
connectionId === "b" ? state({ status: "EXHAUSTED", remainingFreeAllowance: 0 }) : state(),
|
||||
})
|
||||
);
|
||||
assert.equal(result.length, 1);
|
||||
assert.deepEqual(result[0].allowedConnectionIds, ["a", "c"]);
|
||||
});
|
||||
|
||||
test("a multi-account candidate with no safe connection is dropped, not emptied", () => {
|
||||
const pool = [candidate({ connectionId: null, allowedConnectionIds: ["a", "b"] })];
|
||||
const result = filterSubscriptionOnlyCandidates(
|
||||
pool,
|
||||
options({
|
||||
resolveFreeAccessState: () => state({ status: "EXHAUSTED", remainingFreeAllowance: 0 }),
|
||||
})
|
||||
);
|
||||
assert.deepEqual(result, []);
|
||||
});
|
||||
|
||||
// ── auto/thrifty — escalates ───────────────────────────────────────────────
|
||||
|
||||
test("rungs order plan-included capacity ahead of every paid rung", () => {
|
||||
const pool: LadderCandidate[] = [
|
||||
{ provider: "metered-co", model: "premium-model", connectionId: "p1" },
|
||||
{ provider: "metered-co", model: "cheap-model", connectionId: "c1" },
|
||||
{ provider: "planned", model: "plan-model", connectionId: "s1" },
|
||||
{ provider: "anything", model: "keyless-model", connectionId: SYNTHETIC_NOAUTH_CONNECTION_ID },
|
||||
];
|
||||
const result = orderPoolByRung(
|
||||
pool,
|
||||
options({
|
||||
admitUnknownQuota: true,
|
||||
resolveAuthType: (id) => (id === "s1" ? "oauth" : "apikey"),
|
||||
resolveEconomicTier: (_provider, model) => (model === "premium-model" ? "premium" : "cheap"),
|
||||
})
|
||||
);
|
||||
assert.deepEqual(
|
||||
result.map((c) => c.model),
|
||||
["plan-model", "keyless-model", "cheap-model", "premium-model"]
|
||||
);
|
||||
});
|
||||
|
||||
test("an exhausted plan connection steps aside so a paid rung can serve", () => {
|
||||
const pool: LadderCandidate[] = [
|
||||
{ provider: "planned", model: "plan-model", connectionId: "s1" },
|
||||
{ provider: "metered-co", model: "cheap-model", connectionId: "c1" },
|
||||
];
|
||||
const result = orderPoolByRung(
|
||||
pool,
|
||||
options({
|
||||
admitUnknownQuota: true,
|
||||
resolveAuthType: (id) => (id === "s1" ? "oauth" : "apikey"),
|
||||
resolveFreeAccessState: (_provider, connectionId) =>
|
||||
connectionId === "s1"
|
||||
? state({ status: "EXHAUSTED", remainingFreeAllowance: 0 })
|
||||
: undefined,
|
||||
})
|
||||
);
|
||||
assert.deepEqual(
|
||||
result.map((c) => c.model),
|
||||
["cheap-model"]
|
||||
);
|
||||
});
|
||||
|
||||
test("the ladder admits an unverifiable plan connection rather than paying on missing telemetry", () => {
|
||||
const pool = [candidate({ connectionId: "s1", model: "plan-model" })];
|
||||
const result = orderPoolByRung(
|
||||
pool,
|
||||
options({ admitUnknownQuota: true, resolveFreeAccessState: () => undefined })
|
||||
);
|
||||
assert.equal(result.length, 1);
|
||||
});
|
||||
|
||||
test("ordering is stable within a rung so the auto scorer is not reshuffled", () => {
|
||||
const pool: LadderCandidate[] = [
|
||||
{ provider: "metered-co", model: "first", connectionId: "a" },
|
||||
{ provider: "metered-co", model: "second", connectionId: "b" },
|
||||
{ provider: "metered-co", model: "third", connectionId: "c" },
|
||||
];
|
||||
const result = orderPoolByRung(
|
||||
pool,
|
||||
options({ admitUnknownQuota: true, resolveAuthType: () => "apikey" })
|
||||
);
|
||||
assert.deepEqual(
|
||||
result.map((c) => c.model),
|
||||
["first", "second", "third"]
|
||||
);
|
||||
});
|
||||
|
||||
test("a rung budgeted at zero is disabled outright", () => {
|
||||
const pool: LadderCandidate[] = [
|
||||
{ provider: "metered-co", model: "cheap-model", connectionId: "c1" },
|
||||
{ provider: "metered-co", model: "premium-model", connectionId: "p1" },
|
||||
];
|
||||
const result = orderPoolByRung(
|
||||
pool,
|
||||
options({
|
||||
admitUnknownQuota: true,
|
||||
resolveAuthType: () => "apikey",
|
||||
resolveEconomicTier: (_p, model) => (model === "premium-model" ? "premium" : "cheap"),
|
||||
rungBudgetUsd: { premium: 0 },
|
||||
})
|
||||
);
|
||||
assert.deepEqual(
|
||||
result.map((c) => c.model),
|
||||
["cheap-model"]
|
||||
);
|
||||
});
|
||||
|
||||
test("a paid rung drops out once its budget is spent, and is ungated without accounting", () => {
|
||||
const pool: LadderCandidate[] = [
|
||||
{ provider: "metered-co", model: "cheap-model", connectionId: "c1" },
|
||||
];
|
||||
const base = {
|
||||
admitUnknownQuota: true,
|
||||
resolveAuthType: () => "apikey",
|
||||
rungBudgetUsd: { cheap: 5 },
|
||||
};
|
||||
assert.deepEqual(orderPoolByRung(pool, options({ ...base, resolveRungSpendUsd: () => 5 })), []);
|
||||
assert.equal(orderPoolByRung(pool, options({ ...base, resolveRungSpendUsd: () => 1 })).length, 1);
|
||||
// No spend accounting available → the rung is ordered, never gated.
|
||||
assert.equal(
|
||||
orderPoolByRung(pool, options({ ...base, resolveRungSpendUsd: () => null })).length,
|
||||
1
|
||||
);
|
||||
});
|
||||
|
||||
// ── decision 3: returning to the plan after a reset ─────────────────────────
|
||||
|
||||
test("a cached reading whose own resetAt has passed is stale regardless of age", () => {
|
||||
assert.equal(
|
||||
isStateStaleForReset(state({ resetAt: new Date(NOW - 1).toISOString() }), NOW),
|
||||
true
|
||||
);
|
||||
assert.equal(
|
||||
isStateStaleForReset(state({ resetAt: new Date(NOW + 60_000).toISOString() }), NOW),
|
||||
false
|
||||
);
|
||||
assert.equal(isStateStaleForReset(state({ resetAt: null }), NOW), false);
|
||||
assert.equal(isStateStaleForReset(state({ resetAt: "not-a-date" }), NOW), false);
|
||||
assert.equal(isStateStaleForReset(undefined, NOW), false);
|
||||
});
|
||||
|
||||
test("hysteresis: re-entry needs more headroom than staying in did", () => {
|
||||
const opts = options({ exitCutoffPercent: 2, reentryMinRemainingPercent: 5 });
|
||||
const hovering = state({ remainingFreeAllowance: 3 });
|
||||
// Still in play at 3% remaining…
|
||||
assert.equal(isQuotaUsable(hovering, opts, false), true);
|
||||
// …but not enough to climb back after having dropped out.
|
||||
assert.equal(isQuotaUsable(hovering, opts, true), false);
|
||||
assert.equal(isQuotaUsable(state({ remainingFreeAllowance: 6 }), opts, true), true);
|
||||
});
|
||||
|
||||
test("a re-entry floor below the exit cutoff cannot create a re-entry gap", () => {
|
||||
const opts = options({ exitCutoffPercent: 10, reentryMinRemainingPercent: 1 });
|
||||
assert.equal(isQuotaUsable(state({ remainingFreeAllowance: 5 }), opts, true), false);
|
||||
});
|
||||
|
||||
test("cooldown is clamped to the upstream's own reset instant, never extended", () => {
|
||||
const resetIn60s = new Date(NOW + 60_000).toISOString();
|
||||
assert.equal(clampCooldownToReset(600_000, resetIn60s, NOW), 60_000);
|
||||
// Already shorter than the reset → untouched.
|
||||
assert.equal(clampCooldownToReset(10_000, resetIn60s, NOW), 10_000);
|
||||
// Absent / unparseable / already elapsed → untouched, never widened.
|
||||
assert.equal(clampCooldownToReset(600_000, null, NOW), 600_000);
|
||||
assert.equal(clampCooldownToReset(600_000, "nonsense", NOW), 600_000);
|
||||
assert.equal(clampCooldownToReset(600_000, new Date(NOW - 1).toISOString(), NOW), 600_000);
|
||||
});
|
||||
|
||||
test("rung order is the documented escalation order", () => {
|
||||
assert.deepEqual([...RUNG_ORDER], ["subscription", "keyless", "free", "cheap", "premium"]);
|
||||
});
|
||||
Reference in New Issue
Block a user