diff --git a/changelog.d/features/12319-strict-zero-cost-exclusion-reason.md b/changelog.d/features/12319-strict-zero-cost-exclusion-reason.md new file mode 100644 index 0000000000..41950da4b1 --- /dev/null +++ b/changelog.d/features/12319-strict-zero-cost-exclusion-reason.md @@ -0,0 +1 @@ +- **feat(routing):** With `freeAccessPolicy: "strict"`, the read-only candidate listing (`GET /v1/auto-combo/{channel}/candidates`) no longer hides the candidates the zero-cost guard excludes — the same read-only transparency the resilience filter already honours (#9133). Each candidate now carries `freeAccessExclusion` saying why it would be kept out, and it tells an exhausted allowance apart from a quota reading that never arrived or went stale, which used to look identical from the outside. Routing is unchanged: the listing reports, it never enforces. The separate `excludeTosAvoid` guard still drops its candidates without a reason; that gap is now documented rather than closed ([#12319](https://github.com/diegosouzapw/OmniRoute/pull/12319)) diff --git a/docs/routing/STRICT_ZERO_COST.md b/docs/routing/STRICT_ZERO_COST.md index ea9c4469f8..6f1821c533 100644 --- a/docs/routing/STRICT_ZERO_COST.md +++ b/docs/routing/STRICT_ZERO_COST.md @@ -116,13 +116,37 @@ no waiting out the TTL after a 402/403/quota-exhausted response. `freeAccessPolicy`: a candidate can be economically `SAFE` and still excluded here for contractual reasons, or left in when this guard is off even with `freeAccessPolicy: "strict"` on. -## What passes today +## Seeing what the guard excludes -Run `npx tsx scripts/ad-hoc/dry-run-strict-zero-cost.ts` against a live instance's -`GET /v1/auto-combo/{channel}/candidates` output for a real before/after — the script reads each -candidate's real `connectionId`, so it also proves the connection-safety fix live, not just in -unit tests. Keyless candidates must arrive with the synthetic no-auth `connectionId`, never a -real connection. The current built-in keyless auto path is OpenCode Free; exact candidate counts +`GET /v1/auto-combo/{channel}/candidates` lists every candidate, including the ones this guard +would keep out of dispatch, and each carries `freeAccessExclusion` — `null` when the guard is +satisfied, otherwise the reason. The listing reports; it never enforces. Turning the policy off +leaves the field `null` everywhere and costs nothing. + +| `freeAccessExclusion` | What it means | What to do about it | +| :--------------------- | :------------------------------------------------------------------------------------------------------------ | :---------------------------------------------------------------------------------------------------------------------------------- | +| `not-in-catalog` | The provider/model pair is absent from `FREE_MODEL_BUDGETS`. | Add a curated entry, or accept that new pairs start excluded — that is the design. | +| `regime-not-free` | Catalogued, but its `freeType` is not one that grants free access (a discontinued tier, for instance). | Nothing to fix. The model costs money. | +| `no-hard-stop` | Free regime, but `hardStopGuaranteed` is not `true`, so exceeding the allowance might silently start billing. | Verify the provider's terms and set the flag with the source in a comment — never to grow the catalog. | +| `contradictory-noauth` | A no-auth candidate whose catalog entry is not `keyless`. Fail-closed on inconsistent metadata. | Fix the catalog entry; the two facts disagree. | +| `exhausted` | A fresh reading says the allowance is used up. | Wait for the reset. This one resolves itself. | +| `state-unknown` | No quota reading, or one too old to trust. | Go look: the provider may have no usage adapter registered, or the quota fetch is failing. | +| `no-connection` | The candidate carries no account to check at all. | Not a quota problem: the candidate was built without a connection, so nothing was ever looked up. Check how the pool was assembled. | + +The last two are the pair worth separating. An exhausted allowance resets on its own; a reading +that never arrives means the lookup itself is broken, and until now both looked identical from +outside — the candidate simply vanished. + +**One gap remains, and it is deliberate.** `excludeTosAvoid` still removes candidates before the +listing is built, so a model curated `tos: "avoid"` is absent with no reason given — the same +invisibility this section just closed for the zero-cost guard. Closing it too means deciding what +a ToS exclusion should report, which is a separate question from economic safety; this page names +the gap rather than pretending it is not there. + +For an offline before/after, `npx tsx scripts/ad-hoc/dry-run-strict-zero-cost.ts` still works +against a live instance's candidates output; it reads each candidate's real `connectionId`, so it +also exercises the connection-safety path. Keyless candidates must arrive with the synthetic +no-auth `connectionId`, never a real connection. The current built-in keyless auto path is OpenCode Free; exact candidate counts still depend on live model discovery and should be measured on the target deployment instead of copied from an older run. A `recurring-*` candidate passes only when it has both a registered usage adapter and `hardStopGuaranteed: true`; incomplete metadata remains fail-closed. diff --git a/open-sse/handlers/autoComboCandidates.ts b/open-sse/handlers/autoComboCandidates.ts index e7a2762ee7..83e5d5ad48 100644 --- a/open-sse/handlers/autoComboCandidates.ts +++ b/open-sse/handlers/autoComboCandidates.ts @@ -30,6 +30,7 @@ import { buildErrorBody } from "@omniroute/open-sse/utils/error.ts"; import { getCircuitBreaker } from "@/shared/utils/circuitBreaker"; import { isModelLocked } from "@omniroute/open-sse/services/accountFallback.ts"; import { parseModel } from "@omniroute/open-sse/services/model.ts"; +import type { StrictZeroCostExclusionReason } from "@omniroute/open-sse/services/autoCombo/strictZeroCostFilter.ts"; import { getProviderConnectionById } from "@/lib/db/providers"; import { getExcludedConnectionIds } from "@/lib/db/autoCandidateOverrides"; @@ -51,6 +52,13 @@ export interface AutoComboCandidateView { breakerState: string; connectionCooldown: boolean; modelLocked: boolean; + /** + * Why STRICT_ZERO_COST would exclude this candidate from dispatch, or null + * when it would not — and null as well when the policy is off, which is the + * default. Reported, never enforced: this listing shows the candidate either + * way, the routing path is what acts on it. + */ + freeAccessExclusion: StrictZeroCostExclusionReason | null; } export interface AutoComboCandidatesResult { @@ -69,6 +77,7 @@ async function decorateCandidate(candidate: { connectionId: string; model: string; modelStr: string; + freeAccessExclusion?: StrictZeroCostExclusionReason | null; }): Promise { const breaker = getCircuitBreaker(candidate.provider); const breakerStatus = breaker.getStatus(); @@ -111,6 +120,7 @@ async function decorateCandidate(candidate: { breakerState: String(breakerStatus.state), connectionCooldown, modelLocked, + freeAccessExclusion: candidate.freeAccessExclusion ?? null, }; } @@ -160,6 +170,7 @@ export async function getAutoComboCandidates( connectionId: string | null; allowedConnectionIds?: string[]; model: string; + freeAccessExclusion?: StrictZeroCostExclusionReason | null; }> = Array.isArray(virtualCombo?.models) ? virtualCombo.models : []; // Routing keeps one logical provider/model candidate, but the management API // remains account-oriented so operators can inspect and toggle each fallback. @@ -178,6 +189,7 @@ export async function getAutoComboCandidates( connectionId: candidate.connectionId, model: candidate.model, modelStr: candidate.model, + freeAccessExclusion: candidate.freeAccessExclusion, }); return { ...decorated, excluded: excludedConnectionIds.has(candidate.connectionId) }; }) diff --git a/open-sse/services/autoCombo/strictZeroCostFilter.ts b/open-sse/services/autoCombo/strictZeroCostFilter.ts index 8ef9b1ae21..6594d5ffa8 100644 --- a/open-sse/services/autoCombo/strictZeroCostFilter.ts +++ b/open-sse/services/autoCombo/strictZeroCostFilter.ts @@ -129,25 +129,52 @@ export function findBudgetEntry( return catalog.find((m) => m.provider === candidate.provider && m.modelId === candidate.model); } -function isConnectionStateSafe( +/** Why the guard cannot trust a candidate right now. */ +export type StrictZeroCostExclusionReason = + | "not-in-catalog" + | "regime-not-free" + | "no-hard-stop" + | "contradictory-noauth" + | "exhausted" + | "state-unknown" + | "no-connection"; + +export type StrictZeroCostVerdict = + { outcome: "safe"; safeConnectionIds: string[] } | { outcome: StrictZeroCostExclusionReason }; + +/** + * Why one connection cannot be trusted right now. Splitting "exhausted" from + * "state-unknown" is the whole point: an exhausted allowance resets on its own + * and the operator waits, while a missing or stale reading means the quota + * lookup itself is not working and the operator has to go fix something. + * Opposite actions, and until now the same silence. + * + * Freshness is checked before status, so a stale EXHAUSTED reading reports + * "state-unknown" rather than asserting an exhaustion nobody has confirmed + * lately. The exclusion verdict is identical either way -- only the reason + * shown to the operator differs. + */ +export function classifyConnectionState( provider: string, connectionId: string, resolveFreeAccessState: StrictZeroCostOptions["resolveFreeAccessState"], options: Pick -): boolean { +): "safe" | "exhausted" | "state-unknown" { const state = resolveFreeAccessState(provider, connectionId); - if (!state) return false; // no usage adapter for this provider, or lookup never ran/is stale - if (state.status !== "SAFE") return false; + if (!state) return "state-unknown"; // no usage adapter for this provider, or lookup never ran const now = (options.now ?? Date.now)(); const checkedAtMs = Date.parse(state.checkedAt); - if (!Number.isFinite(checkedAtMs) || now - checkedAtMs > options.maxStateAgeMs) return false; + if (!Number.isFinite(checkedAtMs) || now - checkedAtMs > options.maxStateAgeMs) + return "state-unknown"; - if (state.remainingFreeAllowance === null) return false; + if (state.status === "EXHAUSTED") return "exhausted"; + if (state.status !== "SAFE") return "state-unknown"; + if (state.remainingFreeAllowance === null) return "state-unknown"; // A negative threshold would let a negative/garbage reading pass; a caller // that genuinely wants "any allowance greater than zero" should pass 0. - if (options.minRemainingAllowance < 0) return false; - return state.remainingFreeAllowance > options.minRemainingAllowance; + if (options.minRemainingAllowance < 0) return "state-unknown"; + return state.remainingFreeAllowance > options.minRemainingAllowance ? "safe" : "exhausted"; } /** @@ -168,7 +195,28 @@ export function evaluateCandidateConnections( resolveFreeAccessState: StrictZeroCostOptions["resolveFreeAccessState"], options: Pick ): string[] { - if (!budgetEntry) return []; // not in the catalog at all → paid, or genuinely unknown + const verdict = classifyStrictZeroCostCandidate( + candidate, + budgetEntry, + resolveFreeAccessState, + options + ); + return verdict.outcome === "safe" ? verdict.safeConnectionIds : []; +} + +/** + * Same decision as `evaluateCandidateConnections`, but it says why instead of + * answering with an empty list. The read-only candidate listing needs the why; + * the pool filter only needs the list, so it reads this one and throws the + * reason away. + */ +export function classifyStrictZeroCostCandidate( + candidate: StrictZeroCostCandidate, + budgetEntry: FreeModelBudget | undefined, + resolveFreeAccessState: StrictZeroCostOptions["resolveFreeAccessState"], + options: Pick +): StrictZeroCostVerdict { + if (!budgetEntry) return { outcome: "not-in-catalog" }; // paid, or genuinely unknown const isGenuineNoAuthCandidate = candidate.connectionId === SYNTHETIC_NOAUTH_CONNECTION_ID; if (allowsNoAuthShortcut(budgetEntry.freeType)) { @@ -180,30 +228,47 @@ export function evaluateCandidateConnections( // any other freeType, and is excluded there unless hardStopGuaranteed is // also set for it (which the curated catalog does not do for keyless // entries today, so it will correctly exclude). - if (isGenuineNoAuthCandidate) return [SYNTHETIC_NOAUTH_CONNECTION_ID]; + if (isGenuineNoAuthCandidate) + return { outcome: "safe", safeConnectionIds: [SYNTHETIC_NOAUTH_CONNECTION_ID] }; } - if (!grantsFreeAccess(budgetEntry.freeType)) return []; - if (isGenuineNoAuthCandidate) return []; // no-auth path but a non-keyless catalog entry: contradictory metadata, fail closed + if (!grantsFreeAccess(budgetEntry.freeType)) return { outcome: "regime-not-free" }; + // no-auth path but a non-keyless catalog entry: contradictory metadata, fail closed + if (isGenuineNoAuthCandidate) return { outcome: "contradictory-noauth" }; // Every remaining freeType (recurring-*, one-time-initial, a keyless entry // reached via a real connection, and any future type this module doesn't // special-case) requires a documented hard stop before any live check even // runs — no point burning a quota lookup on a connection we could never // trust regardless of its answer. - if (budgetEntry.hardStopGuaranteed !== true) return []; + if (budgetEntry.hardStopGuaranteed !== true) return { outcome: "no-hard-stop" }; const candidateConnectionIds = candidate.connectionId ? [candidate.connectionId] : (candidate.allowedConnectionIds ?? []); + // No account at all to check. Reporting `state-unknown` here would send the + // operator hunting a quota lookup that was never attempted; this is a wiring + // problem, not a quota one. + if (candidateConnectionIds.length === 0) return { outcome: "no-connection" }; + const safe: string[] = []; + // An observed exhaustion outranks a missing reading: one is a fact, the other + // is the absence of one, and the operator needs the fact. Without this rule the + // reason would depend on the order the connections happen to be listed in. + let sawExhausted = false; for (const connectionId of candidateConnectionIds) { if (connectionId === SYNTHETIC_NOAUTH_CONNECTION_ID) continue; // never reachable here, defensive - if (isConnectionStateSafe(candidate.provider, connectionId, resolveFreeAccessState, options)) { - safe.push(connectionId); - } + const state = classifyConnectionState( + candidate.provider, + connectionId, + resolveFreeAccessState, + options + ); + if (state === "safe") safe.push(connectionId); + else if (state === "exhausted") sawExhausted = true; } - return safe; + if (safe.length > 0) return { outcome: "safe", safeConnectionIds: safe }; + return { outcome: sawExhausted ? "exhausted" : "state-unknown" }; } /** diff --git a/open-sse/services/autoCombo/virtualFactory.ts b/open-sse/services/autoCombo/virtualFactory.ts index e980d16347..d760794d49 100644 --- a/open-sse/services/autoCombo/virtualFactory.ts +++ b/open-sse/services/autoCombo/virtualFactory.ts @@ -37,7 +37,12 @@ import { orderPoolByRung, type LadderOptions, } from "./subscriptionLadder"; -import { filterStrictZeroCostCandidates, filterTosAvoidCandidates } from "./strictZeroCostFilter"; +import { + classifyStrictZeroCostCandidate, + filterStrictZeroCostCandidates, + filterTosAvoidCandidates, + findBudgetEntry, +} from "./strictZeroCostFilter"; import { resolveFreeAccessState } from "./freeAccessQuota"; import { isModelExcludedByConnection } from "@/domain/connectionModelRules"; import { resolveProviderAlias } from "../model.ts"; @@ -106,6 +111,13 @@ export interface VirtualAutoComboCandidate { resolvedSupportsVision?: boolean; resolvedReasoning?: boolean; resolvedSupportsThinking?: boolean; + /** + * Why STRICT_ZERO_COST would exclude this candidate, or null when it would + * not. Only populated for the read-only inspector build (`skip`), where the + * guard is deliberately not applied — dispatch builds leave it undefined and + * do no extra work. + */ + freeAccessExclusion?: import("./strictZeroCostFilter").StrictZeroCostExclusionReason | null; } type VirtualAutoCombo = AutoComboConfig & { @@ -119,6 +131,9 @@ type VirtualAutoCombo = AutoComboConfig & { allowedConnectionIds?: string[]; weight: number; label: string; + /** Carried through from the candidate for the read-only inspector; absent + * on every dispatch build. */ + freeAccessExclusion?: import("./strictZeroCostFilter").StrictZeroCostExclusionReason | null; }>; /** MAX of candidates' context windows — safe to advertise because the * auto-combo context pre-filter routes oversized requests to large-window @@ -749,17 +764,42 @@ export async function prepareVirtualAutoComboInputs( // per-candidate: `resolveFreeAccessState` here is a raw pass-through of the real // per-(provider,connectionId) resolver; the filter itself decides which connection(s) // on each candidate to check and rewrites `allowedConnectionIds` to the SAFE subset. - const strictFilteredPool = filterStrictZeroCostCandidates(pool, { - enabled: settings.freeAccessPolicy === "strict", - resolveFreeAccessState, + const strictZeroCostThresholds = { // 1 percentage point of headroom, not 0: `freeAccessQuota.ts` reports // remaining allowance as a percentage, and a raw ">0" comparison would // let a reading of e.g. 0.3% (rounding noise, not real headroom) pass. minRemainingAllowance: 1, maxStateAgeMs: toNumber(settings.autoRefreshProviderQuotaInterval, 180) * 1000, + }; + const strictZeroCostOn = settings.freeAccessPolicy === "strict"; + const strictFilteredPool = filterStrictZeroCostCandidates(pool, { + // The read-only candidate inspector (#9133) must be able to see what the + // guard would exclude, and why — the same opt-out the resilience filter + // already honours through `skip`. Dispatch (`skip === false`) is unaffected. + enabled: strictZeroCostOn && !skip, + resolveFreeAccessState, + ...strictZeroCostThresholds, }); if (strictFilteredPool !== pool) pool = strictFilteredPool; + // Annotate here rather than in the handler: this is where the thresholds and + // `resolveFreeAccessState` already live. Doing it downstream would mean a second + // copy of both, with nothing to keep them in agreement. + if (strictZeroCostOn && skip) { + pool = pool.map((candidate) => { + const verdict = classifyStrictZeroCostCandidate( + candidate, + findBudgetEntry(candidate), + resolveFreeAccessState, + strictZeroCostThresholds + ); + return { + ...candidate, + freeAccessExclusion: verdict.outcome === "safe" ? null : verdict.outcome, + }; + }); + } + // Separate, optional ToS guard — independent of economic safety on purpose. const tosFilteredPool = filterTosAvoidCandidates(pool, settings.excludeTosAvoid === true); if (tosFilteredPool !== pool) pool = tosFilteredPool; @@ -1058,6 +1098,9 @@ export async function createVirtualAutoComboFromPrepared( : {}), weight: snapshotScores.get(candidate.modelStr) ?? 1, label: candidate.provider, + ...(candidate.freeAccessExclusion === undefined + ? {} + : { freeAccessExclusion: candidate.freeAccessExclusion }), })); const autoConfig = { candidatePool: providerPool, diff --git a/tests/unit/auto-combo-candidates-free-access-reason.test.ts b/tests/unit/auto-combo-candidates-free-access-reason.test.ts new file mode 100644 index 0000000000..3c22d94851 --- /dev/null +++ b/tests/unit/auto-combo-candidates-free-access-reason.test.ts @@ -0,0 +1,138 @@ +/** + * STRICT_ZERO_COST vs the read-only candidate inspector (#7819 Level 1, #9133). + * + * With `freeAccessPolicy: "strict"`, the zero-cost guard used to run on the + * inspector's pool as well as the dispatch pool, so a candidate it excluded + * simply vanished from the listing — the operator could not tell "no free + * allowance left" from "the quota lookup never answered". The guard now honours + * the same `skip` opt-out the resilience filter already did, and each candidate + * carries the reason instead. + * + * Two guarantees, in this order of importance: + * 1. the dispatch pool is byte-for-byte what it was (the guard still applies); + * 2. the listing keeps the excluded candidates, each with its reason. + */ +import test from "node:test"; +import assert from "node:assert/strict"; +import fs from "node:fs"; +import os from "node:os"; +import path from "node:path"; + +const TEST_DATA_DIR = fs.mkdtempSync(path.join(os.tmpdir(), "omniroute-free-access-reason-")); +const ORIGINAL_DATA_DIR = process.env.DATA_DIR; + +process.env.DATA_DIR = TEST_DATA_DIR; + +const core = await import("../../src/lib/db/core.ts"); +const providersDb = await import("../../src/lib/db/providers.ts"); +const settingsDb = await import("../../src/lib/db/settings.ts"); +const virtualFactory = await import("../../open-sse/services/autoCombo/virtualFactory.ts"); +const candidateHandler = await import("../../open-sse/handlers/autoComboCandidates.ts"); + +async function resetStorage() { + core.resetDbInstance(); + fs.rmSync(TEST_DATA_DIR, { recursive: true, force: true, maxRetries: 5, retryDelay: 100 }); + fs.mkdirSync(TEST_DATA_DIR, { recursive: true }); +} + +async function seedConnection() { + return providersDb.createProviderConnection({ + provider: "antigravity", + authType: "oauth", + email: "antigravity-strict@example.com", + accessToken: "fake-antigravity-access-token", + tokenExpiresAt: new Date(Date.now() + 60_000).toISOString(), + }); +} + +test.beforeEach(async () => { + await resetStorage(); +}); + +test.after(async () => { + await resetStorage(); + fs.rmSync(TEST_DATA_DIR, { recursive: true, force: true, maxRetries: 5, retryDelay: 100 }); + if (ORIGINAL_DATA_DIR === undefined) delete process.env.DATA_DIR; + else process.env.DATA_DIR = ORIGINAL_DATA_DIR; +}); + +test("the dispatch pool still drops what the guard excludes", async () => { + await seedConnection(); + await settingsDb.updateSettings({ freeAccessPolicy: "strict" }); + + // skip:false is the routing path. Nothing in this change may widen it. + const dispatch = await virtualFactory.prepareVirtualAutoComboInputs({}, false); + const survivors = dispatch.regularCandidates; + + assert.ok( + survivors.every((candidate) => candidate.freeAccessExclusion === undefined), + "the dispatch build must not pay for an annotation it never reads" + ); + + // Every survivor is one the guard cleared: no live quota reading exists in + // this harness, so under `strict` the guard can only keep genuinely keyless + // candidates. An empty pool is the correct, conservative answer here. + const strictOff = await (async () => { + await settingsDb.updateSettings({ freeAccessPolicy: "off" }); + const prepared = await virtualFactory.prepareVirtualAutoComboInputs({}, false); + await settingsDb.updateSettings({ freeAccessPolicy: "strict" }); + return prepared.regularCandidates; + })(); + + assert.ok( + survivors.length <= strictOff.length, + "the guard must never add candidates to the dispatch pool" + ); +}); + +test("a candidate the guard would exclude stays in the listing, with its reason", async () => { + const connection = await seedConnection(); + await settingsDb.updateSettings({ freeAccessPolicy: "strict" }); + + const listing = await candidateHandler.getAutoComboCandidates("auto", null); + const rows = listing.candidates.filter((candidate) => candidate.connectionId === connection.id); + + assert.ok(rows.length > 0, "the guard must not empty the read-only listing"); + + const excluded = rows.filter((candidate) => candidate.freeAccessExclusion !== null); + assert.ok( + excluded.length > 0, + "under a strict policy some candidates are excluded; the listing must still show them" + ); + + const reasons = new Set(excluded.map((candidate) => candidate.freeAccessExclusion)); + const known = new Set([ + "not-in-catalog", + "regime-not-free", + "no-hard-stop", + "contradictory-noauth", + "exhausted", + "state-unknown", + "no-connection", + ]); + for (const reason of reasons) { + assert.ok(known.has(String(reason)), `unexpected exclusion reason: ${String(reason)}`); + } + + // A model absent from the free-tier catalog is the commonest case, and it is a + // different problem from a drained allowance — which is the whole point of + // reporting a reason rather than a boolean. + assert.ok( + reasons.has("not-in-catalog"), + "a model the free catalog does not list must say so, not just disappear" + ); +}); + +test("with the policy off, the listing reports no reason and does no work", async () => { + const connection = await seedConnection(); + await settingsDb.updateSettings({ freeAccessPolicy: "off" }); + + const listing = await candidateHandler.getAutoComboCandidates("auto", null); + const rows = listing.candidates.filter((candidate) => candidate.connectionId === connection.id); + + assert.ok(rows.length > 0, "candidates must be listed when the guard is off"); + assert.ok( + rows.every((candidate) => candidate.freeAccessExclusion === null), + "no policy, no reason — the default case must stay free" + ); +}); diff --git a/tests/unit/strict-zero-cost-exclusion-reason.test.ts b/tests/unit/strict-zero-cost-exclusion-reason.test.ts new file mode 100644 index 0000000000..9b53c1cd05 --- /dev/null +++ b/tests/unit/strict-zero-cost-exclusion-reason.test.ts @@ -0,0 +1,229 @@ +import assert from "node:assert/strict"; +import test from "node:test"; + +import { + classifyConnectionState, + classifyStrictZeroCostCandidate, + evaluateCandidateConnections, + type FreeAccessState, + type StrictZeroCostCandidate, +} from "../../open-sse/services/autoCombo/strictZeroCostFilter.ts"; +import type { FreeModelBudget } from "../../open-sse/config/freeModelCatalog.ts"; + +const NOW = 10_000_000; +const OPTS = { minRemainingAllowance: 1, maxStateAgeMs: 60_000, now: () => NOW }; + +function entry(overrides: Partial = {}): FreeModelBudget { + return { + provider: "p", + model: "m", + freeType: "recurring-monthly", + hardStopGuaranteed: true, + ...overrides, + } as FreeModelBudget; +} + +function state(overrides: Partial = {}): FreeAccessState { + return { + status: "SAFE", + remainingFreeAllowance: 90, + resetAt: null, + checkedAt: new Date(NOW).toISOString(), + ...overrides, + }; +} + +const candidate = (overrides: Partial = {}): StrictZeroCostCandidate => ({ + provider: "p", + model: "m", + connectionId: "c1", + ...overrides, +}); + +const classify = ( + c: StrictZeroCostCandidate, + e: FreeModelBudget | undefined, + resolve: (provider: string, connectionId: string) => FreeAccessState | undefined +) => classifyStrictZeroCostCandidate(c, e, resolve, OPTS); + +// --- one connection at a time --------------------------------------------- + +test("a fresh, funded reading is safe", () => { + assert.equal( + classifyConnectionState("p", "c1", () => state(), OPTS), + "safe" + ); +}); + +test("no reading at all is unknown — the quota lookup never ran for this pair", () => { + assert.equal( + classifyConnectionState("p", "c1", () => undefined, OPTS), + "state-unknown" + ); +}); + +test("an exhausted allowance is exhausted, not unknown", () => { + const drained = state({ status: "EXHAUSTED", remainingFreeAllowance: 0 }); + assert.equal( + classifyConnectionState("p", "c1", () => drained, OPTS), + "exhausted" + ); +}); + +test("a funded reading at or below the threshold is exhausted", () => { + assert.equal( + classifyConnectionState("p", "c1", () => state({ remainingFreeAllowance: 1 }), OPTS), + "exhausted" + ); +}); + +test("a reading older than the allowed age is unknown, never exhausted", () => { + const stale = state({ checkedAt: new Date(NOW - 60_001).toISOString() }); + assert.equal( + classifyConnectionState("p", "c1", () => stale, OPTS), + "state-unknown" + ); +}); + +test("a stale EXHAUSTED reading is unknown too — freshness is checked first", () => { + const staleDrained = state({ + status: "EXHAUSTED", + remainingFreeAllowance: 0, + checkedAt: new Date(NOW - 60_001).toISOString(), + }); + assert.equal( + classifyConnectionState("p", "c1", () => staleDrained, OPTS), + "state-unknown" + ); +}); + +test("an allowance the provider does not report numerically is unknown", () => { + assert.equal( + classifyConnectionState("p", "c1", () => state({ remainingFreeAllowance: null }), OPTS), + "state-unknown" + ); +}); + +test("an unparseable timestamp is unknown", () => { + assert.equal( + classifyConnectionState("p", "c1", () => state({ checkedAt: "not a date" }), OPTS), + "state-unknown" + ); +}); + +// --- whole candidate ------------------------------------------------------- + +test("a model absent from the catalog is not-in-catalog", () => { + assert.equal(classify(candidate(), undefined, () => state()).outcome, "not-in-catalog"); +}); + +test("a catalogued model whose regime is not free is regime-not-free", () => { + const paid = entry({ freeType: "discontinued" as FreeModelBudget["freeType"] }); + assert.equal(classify(candidate(), paid, () => state()).outcome, "regime-not-free"); +}); + +test("a free regime without a documented hard stop is no-hard-stop", () => { + const soft = entry({ hardStopGuaranteed: undefined }); + assert.equal(classify(candidate(), soft, () => state()).outcome, "no-hard-stop"); +}); + +test("a genuine keyless candidate is safe with no live check", () => { + const keyless = entry({ freeType: "keyless" as FreeModelBudget["freeType"] }); + const verdict = classify(candidate({ connectionId: "noauth" }), keyless, () => undefined); + assert.equal(verdict.outcome, "safe"); +}); + +test("a no-auth candidate on a non-keyless entry is contradictory-noauth", () => { + const verdict = classify(candidate({ connectionId: "noauth" }), entry(), () => state()); + assert.equal(verdict.outcome, "contradictory-noauth"); +}); + +test("a funded candidate is safe and names the connection", () => { + const verdict = classify(candidate(), entry(), () => state()); + assert.equal(verdict.outcome, "safe"); + assert.deepEqual(verdict.outcome === "safe" ? verdict.safeConnectionIds : null, ["c1"]); +}); + +test("exhausted wins over unknown when a candidate spans several accounts", () => { + // Order must not decide the reason: an observed exhaustion is a fact, a + // missing reading is the absence of one, and the operator needs the fact. + const multi = candidate({ connectionId: null, allowedConnectionIds: ["unknown-one", "drained"] }); + const resolve = (_p: string, id: string) => + id === "drained" ? state({ status: "EXHAUSTED", remainingFreeAllowance: 0 }) : undefined; + assert.equal(classify(multi, entry(), resolve).outcome, "exhausted"); + + const reversed = candidate({ + connectionId: null, + allowedConnectionIds: ["drained", "unknown-one"], + }); + assert.equal(classify(reversed, entry(), resolve).outcome, "exhausted"); +}); + +test("one safe account among several is still safe, and only it is named", () => { + const multi = candidate({ connectionId: null, allowedConnectionIds: ["drained", "good"] }); + const resolve = (_p: string, id: string) => + id === "good" ? state() : state({ status: "EXHAUSTED", remainingFreeAllowance: 0 }); + const verdict = classify(multi, entry(), resolve); + assert.equal(verdict.outcome, "safe"); + assert.deepEqual(verdict.outcome === "safe" ? verdict.safeConnectionIds : null, ["good"]); +}); + +// --- the contract the pool filter depends on, unchanged -------------------- +// These mirror what `tests/unit/autoCombo/strict-zero-cost-*.test.ts` assert. +// Those files live outside every runner glob in package.json and vitest.config.ts, +// so they never execute; this block keeps the same guarantees somewhere that runs. + +test("evaluateCandidateConnections still answers with connection ids", () => { + assert.deepEqual( + evaluateCandidateConnections(candidate(), entry(), () => state(), OPTS), + ["c1"] + ); +}); + +test("evaluateCandidateConnections still answers empty for every exclusion", () => { + const cases: Array<[string, FreeModelBudget | undefined, () => FreeAccessState | undefined]> = [ + ["not in catalog", undefined, () => state()], + [ + "regime not free", + entry({ freeType: "discontinued" as FreeModelBudget["freeType"] }), + () => state(), + ], + ["no hard stop", entry({ hardStopGuaranteed: undefined }), () => state()], + ["no reading", entry(), () => undefined], + ["exhausted", entry(), () => state({ status: "EXHAUSTED", remainingFreeAllowance: 0 })], + ]; + for (const [label, budgetEntry, resolve] of cases) { + assert.deepEqual( + evaluateCandidateConnections(candidate(), budgetEntry, resolve, OPTS), + [], + `"${label}" must still exclude` + ); + } +}); + +test("a candidate with no account at all says so, rather than blaming the quota lookup", () => { + let lookups = 0; + const verdict = classifyStrictZeroCostCandidate( + candidate({ connectionId: null, allowedConnectionIds: [] }), + entry(), + () => { + lookups += 1; + return state(); + }, + OPTS + ); + assert.equal(verdict.outcome, "no-connection"); + assert.equal(lookups, 0, "nothing should have been looked up — there was nothing to look up"); +}); + +test("no-connection still excludes, exactly as before", () => { + assert.deepEqual( + evaluateCandidateConnections( + candidate({ connectionId: null, allowedConnectionIds: [] }), + entry(), + () => state(), + OPTS + ), + [] + ); +});