Compare commits

...

3 Commits

Author SHA1 Message Date
diegosouzapw
852384f683 feat(quality): complete test:scoped — --full map rebuild, stdin selection, CI loader parity (#8084 D1)
- test:scoped:full (documented in the script header since #9143 but never wired) rebuilds
  config/quality/test-impact-map.json and then selects.
- select-impacted-tests.mjs gains --stdin so --staged selects from the index; the git-diff
  path only ever saw commits, so staged-only runs silently fell back to the heuristic.
- Loader parity with npm run test:unit / quality.yml TIA step (#6787): tests/unit/dashboard/**
  under --import tsx (CJS transform), tests/unit/serial/** at --test-concurrency=1, the rest
  under tsx/esm. The single tsx/esm invocation false-redded every dashboard test the map
  selected ("Unexpected token 'export'").
- CONTRIBUTING.md → Running Tests documents the three modes and the fail-safe exit 1.

Refs #8084
2026-09-01 16:55:07 -03:00
Diego Rodrigues de Sa e Souza
1586476183 fix(release): drain the 2026-09-01 base-red window — passthrough usage regression + radarPage i18n keys (#12327)
* fix(sse): forward the upstream's real trailing usage in passthrough; estimate only at flush

#12151 injected estimated usage into the finish chunk and dropped the real
trailing usage block that genuine OpenAI upstreams send afterwards — metered
clients got estimates instead of real token counts. The estimate now leaves
via a canonical usage-only chunk at flush, only when the upstream stayed
silent; a real trailing block is forwarded verbatim and wins. The tool_calls
finish_reason normalization now materializes its own rewrite (it piggybacked
on the removed finish-time rewrite), and the dead collectSSE helper goes with
it (subsumes #12324).

* fix(i18n): seed the radarPage limits/training keys the #12320 UI already consumes

RadarCatalogTable.tsx references radarPage.colLimits / trainsOnPrompts /
trainsOnPromptsHelp but #12320 never added them to en.json, so the EN
fallback could not resolve the __MISSING__ markers across 42 locales.
Real translations for pt-BR and vi; the rest resolve via the EN fallback.

* fix(sse): carry the chat stream id into flush-time synthetic chunks

The estimated usage-only chunk emitted at flush used passthroughResponsesId,
which is only ever set on the Responses path — on the chat path the synthetic
chunk shipped id: null, breaking the string-id invariant pinned by
stream-numeric-ids.test.ts. Track the upstream chat-completion id in the
passthrough loop and reuse it (falling back to the Responses id, then a
generated one). Sibling sweep: 74 files importing utils/stream — 603/603.
2026-09-01 16:17:48 -03:00
Dizzle
c9f9b6274e feat(providers): expose usage-supported in provider plugin manifest (#12214)
The provider plugin manifest already exposed usage-fetch (40 providers, #11903); this publishes the second capability, usage-supported, so integrators can tell without reading TypeScript whether the server usage routes accept a provider. #11903 closed #11722 after shipping only half of it and said so at the time — this is the follow-up it promised.

The two scopes genuinely differ and the docs now say how: usage-fetch resolves on id or alias (the dispatcher accepts both), usage-supported on id alone, because the runtime guard does a plain USAGE_SUPPORTED_PROVIDERS.includes(providerId) with no alias resolution. 42 providers carry both tags, 4 carry only usage-fetch (opencode, opencode-zen, openrouter, xai) and 3 only usage-supported (adobe-firefly, firefly, xiaomi-mimo-token-plan) — 7 measured differences, so neither tag implies the other. No list mutation, no new route, schemaVersion stays 1.

USAGE_SUPPORTED_PROVIDERS moved out of src/shared/constants/providers.ts into an import-free leaf at open-sse/services/usage/supportedProviders.ts, keeping the manifest's import graph light — the same move fetcherProviders.ts got in #11903, landed on the correct side of the workspace boundary.

Base note: the branch forked 46 commits before kilocode joined the list, so a wholesale take of its providers.ts would have silently dropped that id. Verified against the current release tip before merging — both sides hold the same 46 ids, nothing lost.

Verified on the current tip: typecheck:core clean, check:cycles OK across 417 files (the import-free-leaf claim holds), and 63/63 focused tests across provider-plugin-manifest, usage-fetcher-registration-coverage, adobe-firefly and agentrouter-quota-dashboard-rendering.

Thanks @maxmad64bis — and for finishing the half of #11722 that was left open rather than letting it sit.
2026-09-01 16:02:46 -03:00
16 changed files with 396 additions and 191 deletions

View File

@@ -177,6 +177,13 @@ npm run test:all
# Single test file (Node.js native test runner — most tests use this)
node --import tsx/esm --test tests/unit/your-file.test.ts
# Only the unit tests impacted by your change (same TIA selector as the CI gate, #8084)
npm run test:scoped # changes in the last commit (or the working tree)
npm run test:scoped:staged # staged changes only — pairs well with a pre-commit run
npm run test:scoped:full # rebuild the import-graph map first (after adding/moving files)
# Exit 1 + "run the full suite" means a hub file (tsconfig, package.json, …) or an
# unmapped source changed — the selector fails safe, it never silently skips.
# Vitest (MCP server, autoCombo, cache)
npm run test:vitest

View File

@@ -0,0 +1 @@
- **feat(providers):** the provider plugin manifest now also advertises a `usage-supported` capability for the 46 providers whose usage API is accepted by the server and Dashboard routes, so integrators can distinguish "the server will serve quota for this provider" from "a fetcher is wired" without reading TypeScript. Discovery only — no fetcher or quota change. `usage-fetch` resolves on id or alias (the usage dispatcher accepts both); `usage-supported` resolves on id alone, matching the runtime guard `USAGE_SUPPORTED_PROVIDERS.includes(providerId)`. `USAGE_SUPPORTED_PROVIDERS` moved to a zero-dependency leaf (`open-sse/services/usage/supportedProviders.ts`) and is re-exported from `providers.ts`, mirroring the `fetcherProviders` leaf from #11903 and keeping the manifest a light module. ([#12214](https://github.com/diegosouzapw/OmniRoute/pull/12214)) — thanks @maxmad64bis

View File

@@ -48,7 +48,7 @@ The manifest contains:
- JSON-safe model metadata such as context length, vision/reasoning flags, and
unsupported params
- capability tags including `apikey`, `oauth`, `custom-executor`,
`passthrough-models`, `responses`, `sidecar-candidate`, and `usage-fetch`
`passthrough-models`, `responses`, `sidecar-candidate`, `usage-fetch`, and `usage-supported`
The manifest intentionally excludes:
@@ -74,6 +74,7 @@ re-reading the TypeScript sources.
| `custom-executor` | Runs a non-default executor, so it stays on the TypeScript path. |
| `sidecar-candidate` | Mirrors `sidecar.eligible` — safe to consider for sidecar import. |
| `usage-fetch` | Has a wired usage or quota fetcher (`getUsageForProvider`). |
| `usage-supported` | The usage API accepts this provider (`isSupportedUsageConnection`). |
`usage-fetch` is discovery only. It reports that OmniRoute knows how to read usage for the
provider; it does not activate fetching, change quota semantics, or imply that the
@@ -86,6 +87,16 @@ with aliases and is slightly longer than the number of tagged providers: entries
not chat providers in the manifest registry (for example the `firecrawl` search provider
and the `amazon-q` ACP provider) have no manifest entry to tag.
`usage-supported` answers whether the server and Dashboard usage routes accept a connection
for the provider. It mirrors `isSupportedUsageConnection()` (`src/lib/usage/providerLimits.ts`)
and `supportsProviderQuota()` (`src/shared/utils/providerQuotaVisibility.ts`), both gated by
`USAGE_SUPPORTED_PROVIDERS` (`open-sse/services/usage/supportedProviders.ts`). Unlike
`usage-fetch`, it is emitted on the provider id alone — the runtime guard does
`USAGE_SUPPORTED_PROVIDERS.includes(providerId)` with no alias resolution, so the manifest
keeps the same rule. The two tags have different perimeters: 4 providers carry only
`usage-fetch` (`opencode`, `opencode-zen`, `openrouter`, `xai`) and 1 carries only
`usage-supported` (`xiaomi-mimo-token-plan`), so one does not imply the other.
## Sidecar Use
Sidecars should treat `sidecar.eligible` as a conservative candidate signal, not

View File

@@ -1,5 +1,6 @@
import type { RegistryEntry, RegistryModel } from "./providers/shared.ts";
import { USAGE_FETCHER_PROVIDERS } from "../services/usage/fetcherProviders.ts";
import { USAGE_SUPPORTED_PROVIDERS } from "../services/usage/supportedProviders.ts";
export type ProviderPluginCapability =
| "apikey"
@@ -8,7 +9,8 @@ export type ProviderPluginCapability =
| "passthrough-models"
| "responses"
| "sidecar-candidate"
| "usage-fetch";
| "usage-fetch"
| "usage-supported";
export interface ProviderPluginModel {
id: string;
@@ -66,6 +68,15 @@ const SIDECAR_COMPATIBLE_EXECUTORS = new Set(["default"]);
*/
const USAGE_FETCHER_PROVIDER_SET = new Set<string>(USAGE_FETCHER_PROVIDERS);
/**
* Providers whose usage API is accepted by dashboard/server routes (#10078).
* Unlike USAGE_FETCHER_PROVIDERS this gate is checked with a plain
* `USAGE_SUPPORTED_PROVIDERS.includes(providerId)` — no alias resolution —
* so the manifest must emit on the identifier alone to stay faithful to the
* runtime guard.
*/
const USAGE_SUPPORTED_PROVIDER_SET = new Set<string>(USAGE_SUPPORTED_PROVIDERS);
function compactObject<T extends Record<string, unknown>>(value: T): Partial<T> {
return Object.fromEntries(
Object.entries(value).filter(([, entryValue]) => entryValue !== undefined)
@@ -142,6 +153,9 @@ function capabilitiesFor(entry: RegistryEntry, eligible: boolean): ProviderPlugi
) {
capabilities.add("usage-fetch");
}
if (USAGE_SUPPORTED_PROVIDER_SET.has(entry.id)) {
capabilities.add("usage-supported");
}
return [...capabilities].sort();
}

View File

@@ -0,0 +1,79 @@
/**
* usage/supportedProviders.ts — registration list of providers whose usage/quota
* API is accepted by the dashboard and server routes.
*
* Extracted from `src/shared/constants/providers.ts` so that light consumers —
* the provider-plugin manifest (`config/providerPluginManifest.ts`) above all —
* can read the list without pulling the ~12-module provider registry, and
* without an open-sse module reaching across the workspace boundary into
* `src/` (the open-sse typecheck gate forbids open-sse → src imports). Same
* pattern as `fetcherProviders.ts` (#11903): pure data — no imports, no module
* state — so it cannot introduce a cycle. `src/shared/constants/providers.ts`
* re-exports the value, so every existing `@/shared/constants/providers`
* import path keeps working unchanged.
*
* Typed `readonly string[]` (not `as const`): the dashboard/server gates call
* `USAGE_SUPPORTED_PROVIDERS.includes(providerId)` with a plain `string`, which
* a literal-tuple type would reject (TS2345).
*/
// Providers that support usage/quota API
export const USAGE_SUPPORTED_PROVIDERS: readonly string[] = [
"antigravity",
"agy",
"kiro",
"amazon-q",
"github",
"codex",
"claude",
"cursor",
"qoder",
"kimi-coding",
"kimi-coding-apikey",
"glm",
"glm-cn",
"zai",
"glmt",
"opencode-go",
"ollama-cloud",
"minimax",
"minimax-cn",
"crof",
"nanogpt",
"deepseek",
"xiaomi-mimo",
"xiaomi-mimo-token-plan",
"vertex",
"vertex-partner",
"codebuddy-cn",
// PromptQL playground credits (getCreditSummary → USD micros)
"promptql",
"pql",
// Adobe Firefly web (cookie/JWT as apikey) — GET firefly.adobe.io/v1/credits/balance
"adobe-firefly",
"firefly",
"hyperagent",
"ha",
// xAI OAuth (Grok) weekly quota (id + public alias, same pattern as ha/agy)
"xai-oauth",
"xao",
// Grok Build subscription, billing credits, and auto top-up status
"grok-cli",
// Firecrawl team credits (GET /v2/team/credit-usage)
"firecrawl",
// Volcano Ark Plan subscriptions (agent-plan / coding-plan)
"volcengine-agent-plan",
"volcengine-coding-plan",
// Command Code credits + 5h/weekly rolling windows
"command-code",
"conol-web",
"cnl",
// Alibaba Coding Plan triple-window quota (#9603 UI gap — fetcher existed, list entry missing)
"bailian-coding-plan",
// Qwen Cloud / Model Studio personal Token Plan (cookie-authenticated console gateway)
"qwen-cloud-token-plan",
// AgentRouter (New-API) console balance quota (consoleApiKey + newApiUserId)
"agentrouter",
// Kilo Code personal USD balance (GET /api/profile/balance, existing OAuth token)
"kilocode",
];

View File

@@ -771,6 +771,7 @@ export function createSSEStream(options: StreamOptions = {}) {
const passthroughResponsesOutputItems: unknown[] = [];
const passthroughResponsesPendingFunctionCalls = new Map<string, JsonRecord>();
let passthroughResponsesId: string | null = null;
let passthroughLastChatId: string | null = null;
let passthroughResponsesCurrentFunctionCallKey: string | null = null;
const passthroughResponsesReasoningSummarySeen = new Set<string>();
// #6199 — commentary-phase items announced via `response.output_item.added` are
@@ -1955,6 +1956,16 @@ export function createSSEStream(options: StreamOptions = {}) {
const isFinishChunk = parsed.choices?.[0]?.finish_reason;
// Remember the upstream's chat-completion id so synthetic chunks
// emitted at flush (e.g. the estimated usage-only chunk) carry the
// stream's real string id instead of null on the chat path
// (passthroughResponsesId is only ever set on the Responses path).
if (typeof parsed.id === "string" && parsed.id) {
passthroughLastChatId = parsed.id;
} else if (typeof parsed.id === "number") {
passthroughLastChatId = String(parsed.id);
}
if (isFinishChunk) {
passthroughSawFinishReason = true;
}
@@ -1973,28 +1984,21 @@ export function createSSEStream(options: StreamOptions = {}) {
parsed.choices[0].finish_reason !== "tool_calls"
) {
parsed.choices[0].finish_reason = "tool_calls";
// If we modify it, we must output the modified object
if (!injectedUsage && hasValidUsage(parsed.usage)) {
output = `data: ${JSON.stringify(parsed)}\n\n`;
injectedUsage = true;
}
// If we modify it, we must output the modified object. This used to
// piggyback on the estimated-usage rewrite below; with the estimate
// moved to flush() (#12151 follow-up) the rewrite must happen here.
// injectedUsage doubles as the "output already rewritten" latch —
// without it the raw line overwrites this rewrite further down.
output = `data: ${JSON.stringify(parsed)}\n\n`;
injectedUsage = true;
}
if (
isFinishChunk &&
!passthroughForwardedUsage &&
!hasValidUsage(parsed.usage) &&
!hasValidUsage(usage) &&
totalContentLength > 0
) {
const estimated = estimateUsage(body, totalContentLength, sourceFormat || FORMATS.OPENAI);
if (hasValidUsage(estimated)) {
parsed.usage = filterUsageForFormat(estimated, sourceFormat || FORMATS.OPENAI);
output = `data: ${JSON.stringify(parsed)}\n\n`;
usage = estimated;
passthroughForwardedUsage = true;
injectedUsage = true;
}
} else if (isFinishChunk && hasValidUsage(usage) && !passthroughForwardedUsage) {
// #12151 follow-up: do NOT inject estimated usage into the finish chunk.
// A genuine OpenAI upstream sends its usage in a trailing empty-choices
// chunk AFTER the finish; estimating here marked passthroughForwardedUsage
// and made the real trailing block get dropped in favor of the estimate
// (billing regression pinned by tests/unit/stream-utils.test.ts). The
// estimate is now emitted in flush(), only when the upstream stayed silent.
if (isFinishChunk && hasValidUsage(usage) && !passthroughForwardedUsage) {
const buffered = addBufferToUsage(usage);
parsed.usage = filterUsageForFormat(buffered, sourceFormat || FORMATS.OPENAI);
output = `data: ${JSON.stringify(parsed)}\n\n`;
@@ -2510,6 +2514,30 @@ export function createSSEStream(options: StreamOptions = {}) {
forward(controller, encoder.encode(finishOutput));
clientPayloadCollector.push(syntheticFinishChunk);
}
// #12151: upstream never reported usage — emit the estimate as a
// canonical OpenAI trailing usage-only chunk (empty choices) before
// [DONE], so metered clients still see token counts. When the
// upstream DID send usage (trailing or in-band), it was forwarded
// already and passthroughForwardedUsage guards this off.
if (
shouldEmitDoneTerminator &&
!passthroughForwardedUsage &&
hasValidUsage(usage)
) {
const usageOnlyChunk = {
id: passthroughLastChatId ?? passthroughResponsesId ?? `chatcmpl-${Date.now()}`,
object: "chat.completion.chunk",
created: Math.floor(Date.now() / 1000),
model,
choices: [],
usage: filterUsageForFormat(usage, sourceFormat || FORMATS.OPENAI),
};
const usageOutput = `data: ${JSON.stringify(usageOnlyChunk)}\n\n`;
reqLogger?.appendConvertedChunk?.(usageOutput);
forward(controller, encoder.encode(usageOutput));
clientPayloadCollector.push(usageOnlyChunk);
passthroughForwardedUsage = true;
}
await emitFinalSseMetadata(controller, usage);
doneSent = true;
if (shouldEmitDoneTerminator) {

View File

@@ -126,6 +126,7 @@
"test:unit:fast": "cross-env DISABLE_SQLITE_AUTO_BACKUP=true node --max-old-space-size=8192 --import tsx/esm --import ./open-sse/utils/setupPolyfill.ts --import ./tests/_setup/isolateDataDir.ts --test --test-force-exit --test-isolation=none tests/unit/*.test.ts \"tests/unit/{api,auth,authz,build,cli,cli-helper,combo,compression,correctness,cors,db,db-adapters,docs,gamification,guardrails,lib,mcp,memory,runtime,security,services,settings,shared,translator,ui,usage}/**/*.test.ts\" \"tests/unit/**/*.test.mjs\" && cross-env DISABLE_SQLITE_AUTO_BACKUP=true node --max-old-space-size=8192 --import tsx --import ./open-sse/utils/setupPolyfill.ts --import ./tests/_setup/isolateDataDir.ts --test --test-force-exit --test-isolation=none \"tests/unit/dashboard/**/*.test.ts\" && npm run test:unit:serial",
"test:scoped": "bash scripts/quality/test-scoped.sh",
"test:scoped:staged": "bash scripts/quality/test-scoped.sh --staged",
"test:scoped:full": "bash scripts/quality/test-scoped.sh --full",
"test:unit:shard": "concurrently --kill-others-on-fail -n s1,s2 \"npm:test:unit:shard:1\" \"npm:test:unit:shard:2\"",
"test:unit:shard:1": "cross-env DISABLE_SQLITE_AUTO_BACKUP=true node --max-old-space-size=8192 --import tsx/esm --import ./open-sse/utils/setupPolyfill.ts --import ./tests/_setup/isolateDataDir.ts --test --test-force-exit --test-concurrency=10 --test-shard=1/2 tests/unit/*.test.ts \"tests/unit/{api,auth,authz,build,cli,cli-helper,combo,compression,correctness,cors,db,db-adapters,docs,gamification,guardrails,lib,mcp,memory,runtime,security,services,settings,shared,translator,ui,usage}/**/*.test.ts\" \"tests/unit/**/*.test.mjs\" && cross-env DISABLE_SQLITE_AUTO_BACKUP=true node --max-old-space-size=8192 --import tsx --import ./open-sse/utils/setupPolyfill.ts --import ./tests/_setup/isolateDataDir.ts --test --test-force-exit --test-concurrency=10 --test-shard=1/2 \"tests/unit/dashboard/**/*.test.ts\" && cross-env DISABLE_SQLITE_AUTO_BACKUP=true node --max-old-space-size=8192 --import tsx/esm --import ./open-sse/utils/setupPolyfill.ts --import ./tests/_setup/isolateDataDir.ts --test --test-force-exit --test-concurrency=1 --test-shard=1/2 \"tests/unit/serial/**/*.test.ts\"",
"test:unit:shard:2": "cross-env DISABLE_SQLITE_AUTO_BACKUP=true node --max-old-space-size=8192 --import tsx/esm --import ./open-sse/utils/setupPolyfill.ts --import ./tests/_setup/isolateDataDir.ts --test --test-force-exit --test-concurrency=10 --test-shard=2/2 tests/unit/*.test.ts \"tests/unit/{api,auth,authz,build,cli,cli-helper,combo,compression,correctness,cors,db,db-adapters,docs,gamification,guardrails,lib,mcp,memory,runtime,security,services,settings,shared,translator,ui,usage}/**/*.test.ts\" \"tests/unit/**/*.test.mjs\" && cross-env DISABLE_SQLITE_AUTO_BACKUP=true node --max-old-space-size=8192 --import tsx --import ./open-sse/utils/setupPolyfill.ts --import ./tests/_setup/isolateDataDir.ts --test --test-force-exit --test-concurrency=10 --test-shard=2/2 \"tests/unit/dashboard/**/*.test.ts\" && cross-env DISABLE_SQLITE_AUTO_BACKUP=true node --max-old-space-size=8192 --import tsx/esm --import ./open-sse/utils/setupPolyfill.ts --import ./tests/_setup/isolateDataDir.ts --test --test-force-exit --test-concurrency=1 --test-shard=2/2 \"tests/unit/serial/**/*.test.ts\"",

View File

@@ -39,7 +39,18 @@ export function selectImpacted({ changed, map }) {
return [...out].sort();
}
// `--stdin`: read the changed-file list from stdin (one path per line) instead of
// diffing git. Used by scripts/quality/test-scoped.sh so `--staged` selects from the
// index — the git-diff path here only knows about commits, never the working tree.
export function changedFilesFromStdin(text) {
return String(text || "")
.split(/\r?\n/)
.map((s) => s.trim())
.filter(Boolean);
}
function changedFiles() {
if (process.argv.includes("--stdin")) return changedFilesFromStdin(fs.readFileSync(0, "utf8"));
const baseRef = process.env.GITHUB_BASE_REF;
const baseTarget = process.env.GITHUB_BASE_SHA || (baseRef ? `origin/${baseRef}` : "HEAD~1");
const stdout = execFileSync(

View File

@@ -2,25 +2,46 @@
# test-scoped — run only unit tests impacted by your changes.
#
# Usage:
# npm run test:scoped # tests for changes vs HEAD~1
# npm run test:scoped -- --staged # tests for staged changes only
# npm run test:scoped # tests for changes vs HEAD~1 (working tree if no commit)
# npm run test:scoped:staged # tests for staged changes only
# npm run test:scoped:full # rebuild the import-graph impact map first, then select
#
# This is the local DX companion to the CI TIA gate (#8084 D1). The CI version
# builds a full import-graph impact map; for local dev we use a fast heuristic:
# This is the local DX companion to the CI TIA gate (#8084 D1). It uses the SAME
# selector as CI (scripts/quality/select-impacted-tests.mjs) against the import-graph
# impact map (config/quality/test-impact-map.json, gitignored):
# - Changed test files → run those directly
# - Changed source files → run tests that share the file's directory/name prefix
# - Hub files (tsconfig, package.json, etc.) → suggest full suite
# - Changed source files → run every unit test whose import graph reaches them
# - Hub files (tsconfig, package.json, …) or unmapped sources → full suite (fail-safe)
#
# For the full TIA (import-graph based), use: npm run test:scoped:full
# (requires a pre-built impact map via: node scripts/quality/build-test-impact-map.mjs)
# The map is a snapshot of the import graph: rebuild it (`--full`) after adding tests,
# moving files, or pulling a big base update — a stale map falls back to __RUN_ALL__
# for unknown sources, never to a silent skip.
#
# Loader parity with `npm run test:unit` / CI (#6787): tests/unit/dashboard/** runs
# under `--import tsx` (CJS transform — required for ESM-only deep imports such as
# @lobehub/icons/es/*), tests/unit/serial/** at --test-concurrency=1, everything else
# under `--import tsx/esm`. A single tsx/esm invocation false-reds every dashboard
# test the map selects ("Unexpected token 'export'").
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
REPO_ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)"
MAP_FILE="$REPO_ROOT/config/quality/test-impact-map.json"
STAGED=false
FULL=false
for arg in "$@"; do
case "$arg" in
--staged) STAGED=true ;;
--full) FULL=true ;;
-h|--help) sed -n '2,25p' "${BASH_SOURCE[0]}"; exit 0 ;;
*) echo "[test:scoped] unknown argument: $arg (use --staged, --full)"; exit 2 ;;
esac
done
# ── 1. Determine changed files ───────────────────────────────────────────────
if [[ "${1:-}" == "--staged" ]]; then
if [ "$STAGED" = true ]; then
CHANGED=$(git -C "$REPO_ROOT" diff --name-only --diff-filter=ACMR --cached)
else
CHANGED=$(git -C "$REPO_ROOT" diff --name-only --diff-filter=ACMR HEAD~1...HEAD 2>/dev/null || \
@@ -32,80 +53,51 @@ if [ -z "$CHANGED" ]; then
exit 0
fi
# ── 2. Classify changes ──────────────────────────────────────────────────────
HUB_RE="(setupPolyfill|tsconfig|package\\.json|package-lock\\.json|\\.env|vitest\\.config|stryker\\.conf)"
TEST_FILES=()
SRC_FILES=()
HIT_HUB=false
# ── 2. Impact map (build on --full or when missing) ──────────────────────────
if [ "$FULL" = true ] || [ ! -f "$MAP_FILE" ]; then
echo "[test:scoped] Building the import-graph impact map (config/quality/test-impact-map.json)…"
(cd "$REPO_ROOT" && node scripts/quality/build-test-impact-map.mjs)
fi
while IFS= read -r f; do
[ -z "$f" ] && continue
if echo "$f" | grep -qE "$HUB_RE"; then
HIT_HUB=true
elif echo "$f" | grep -qE '^tests/unit/.*\.test\.(ts|mjs)$'; then
TEST_FILES+=("$f")
elif echo "$f" | grep -qE '^(src|open-sse)/'; then
SRC_FILES+=("$f")
fi
done <<< "$CHANGED"
# ── 3. Select impacted tests (same selector as the CI TIA gate) ──────────────
SEL=$(printf '%s\n' "$CHANGED" | node "$REPO_ROOT/scripts/quality/select-impacted-tests.mjs" --stdin)
# ── 3. Hub file changed → full suite ─────────────────────────────────────────
if [ "$HIT_HUB" = true ]; then
echo "[test:scoped] Hub file changed — run full suite: npm run test:unit"
if echo "$SEL" | grep -q "__RUN_ALL__"; then
echo "[test:scoped] Hub file or unmapped source changed — run the full suite: npm run test:unit"
echo "[test:scoped] (if you just added a source file, rebuild the map: npm run test:scoped:full)"
exit 1
fi
# ── 4. Collect tests to run ──────────────────────────────────────────────────
RUN_TESTS=()
mapfile -t RUN_TESTS < <(printf '%s\n' "$SEL" | grep -v '^$' | sort -u)
# Direct test file changes always run
for tf in "${TEST_FILES[@]}"; do
RUN_TESTS+=("$tf")
done
# For source files, try the impact map first; fall back to heuristic
MAP_FILE="$REPO_ROOT/config/quality/test-impact-map.json"
if [ ${#SRC_FILES[@]} -gt 0 ] && [ -f "$MAP_FILE" ]; then
# Use the TIA selection with the impact map
SEL=$(printf '%s\n' "${SRC_FILES[@]}" | node "$REPO_ROOT/scripts/quality/select-impacted-tests.mjs" 2>/dev/null || echo "__RUN_ALL__")
if echo "$SEL" | grep -q "__RUN_ALL__"; then
echo "[test:scoped] Unmapped source change — run full suite: npm run test:unit"
exit 1
fi
while IFS= read -r t; do
[ -n "$t" ] && RUN_TESTS+=("$t")
done <<< "$SEL"
elif [ ${#SRC_FILES[@]} -gt 0 ]; then
# No impact map — heuristic: suggest building it
echo "[test:scoped] No impact map found. Build it with: node scripts/quality/build-test-impact-map.mjs"
echo "[test:scoped] Or run the full suite: npm run test:unit"
echo ""
echo "[test:scoped] Changed source files:"
printf ' %s\n' "${SRC_FILES[@]}"
if [ ${#TEST_FILES[@]} -gt 0 ]; then
echo "[test:scoped] Running changed test files only..."
else
exit 1
fi
fi
# Deduplicate
IFS=$'\n' SORTED=($(printf '%s\n' "${RUN_TESTS[@]}" | sort -u)); unset IFS
if [ ${#SORTED[@]} -eq 0 ]; then
echo "[test:scoped] No impacted tests — source changes don't map to any unit test."
if [ ${#RUN_TESTS[@]} -eq 0 ]; then
echo "[test:scoped] No impacted unit tests — the change does not reach any node:test file."
exit 0
fi
echo "[test:scoped] Running ${#SORTED[@]} impacted test(s)..."
echo "[test:scoped] Running ${#RUN_TESTS[@]} impacted test(s)..."
# ── 4. Split by loader (mirror package.json test:unit / quality.yml TIA step) ──
DASH=(); SERIAL=(); REST=()
for f in "${RUN_TESTS[@]}"; do
case "$f" in
tests/unit/dashboard/*) DASH+=("$f") ;;
tests/unit/serial/*) SERIAL+=("$f") ;;
*) REST+=("$f") ;;
esac
done
# ── 5. Run selected tests ────────────────────────────────────────────────────
cd "$REPO_ROOT"
exec cross-env \
DISABLE_SQLITE_AUTO_BACKUP=true \
node --max-old-space-size=8192 \
--import tsx/esm \
--import ./open-sse/utils/setupPolyfill.ts \
--import ./tests/_setup/isolateDataDir.ts \
--test --test-force-exit --test-concurrency=4 \
"${SORTED[@]}"
NODE_COMMON=(--max-old-space-size=8192 --import ./open-sse/utils/setupPolyfill.ts --import ./tests/_setup/isolateDataDir.ts --test --test-force-exit)
export DISABLE_SQLITE_AUTO_BACKUP=true
RC=0
if [ ${#REST[@]} -gt 0 ]; then
node --import tsx/esm "${NODE_COMMON[@]}" --test-concurrency=4 "${REST[@]}" || RC=$?
fi
if [ ${#DASH[@]} -gt 0 ]; then
node --import tsx "${NODE_COMMON[@]}" --test-concurrency=4 "${DASH[@]}" || RC=$?
fi
if [ ${#SERIAL[@]} -gt 0 ]; then
node --import tsx/esm "${NODE_COMMON[@]}" --test-concurrency=1 "${SERIAL[@]}" || RC=$?
fi
exit $RC

View File

@@ -13381,6 +13381,9 @@
"colProvider": "Provider",
"colModel": "Model",
"colQuota": "Quota",
"colLimits": "Limits",
"trainsOnPrompts": "Trains on prompts",
"trainsOnPromptsHelp": "This provider discloses that it may use your prompts to train models",
"colContext": "Context",
"colCapabilities": "Capabilities",
"colTos": "ToS Risk",

View File

@@ -13382,12 +13382,12 @@
"colProvider": "Provedor",
"colModel": "Modelo",
"colQuota": "Cota",
"colLimits": "Limites",
"trainsOnPrompts": "Treina com prompts",
"trainsOnPromptsHelp": "Este provedor declara que pode usar seus prompts para treinar modelos",
"colContext": "Contexto",
"colCapabilities": "Capacidades",
"colTos": "Risco ToS",
"colLimits": "__MISSING__:Rate limits",
"trainsOnPrompts": "__MISSING__:Trains on prompts",
"trainsOnPromptsHelp": "__MISSING__:This provider's terms state it may train on the prompts you send. Models without this badge either state they do not, or do not document it — an absent statement is not a guarantee.",
"newBadge": "novo",
"setupGuide": "Guia de configuração",
"disabledByFeed": "Desativado pelo feed Radar",

View File

@@ -13382,12 +13382,12 @@
"colProvider": "Nhà cung cấp",
"colModel": "Mô hình",
"colQuota": "Hạn ngạch",
"colLimits": "Giới hạn",
"trainsOnPrompts": "Huấn luyện bằng prompt",
"trainsOnPromptsHelp": "Nhà cung cấp này công bố có thể dùng prompt của bạn để huấn luyện mô hình",
"colContext": "Ngữ cảnh",
"colCapabilities": "Khả năng",
"colTos": "Rủi ro ToS",
"colLimits": "__MISSING__:Rate limits",
"trainsOnPrompts": "__MISSING__:Trains on prompts",
"trainsOnPromptsHelp": "__MISSING__:This provider's terms state it may train on the prompts you send. Models without this badge either state they do not, or do not document it — an absent statement is not a guarantee.",
"newBadge": "mới",
"setupGuide": "Hướng dẫn thiết lập",
"disabledByFeed": "Bị vô hiệu hóa bởi nguồn cấp dữ liệu Radar",

View File

@@ -482,66 +482,7 @@ export const ID_TO_ALIAS = new Proxy({} as Record<string, string>, {
},
});
// Providers that support usage/quota API
export const USAGE_SUPPORTED_PROVIDERS = [
"antigravity",
"agy",
"kiro",
"amazon-q",
"github",
"codex",
"claude",
"cursor",
"qoder",
"kimi-coding",
"kimi-coding-apikey",
"glm",
"glm-cn",
"zai",
"glmt",
"opencode-go",
"ollama-cloud",
"minimax",
"minimax-cn",
"crof",
"nanogpt",
"deepseek",
"xiaomi-mimo",
"xiaomi-mimo-token-plan",
"vertex",
"vertex-partner",
"codebuddy-cn",
// PromptQL playground credits (getCreditSummary → USD micros)
"promptql",
"pql",
// Adobe Firefly web (cookie/JWT as apikey) — GET firefly.adobe.io/v1/credits/balance
"adobe-firefly",
"firefly",
"hyperagent",
"ha",
// xAI OAuth (Grok) weekly quota (id + public alias, same pattern as ha/agy)
"xai-oauth",
"xao",
// Grok Build subscription, billing credits, and auto top-up status
"grok-cli",
// Firecrawl team credits (GET /v2/team/credit-usage)
"firecrawl",
// Volcano Ark Plan subscriptions (agent-plan / coding-plan)
"volcengine-agent-plan",
"volcengine-coding-plan",
// Command Code credits + 5h/weekly rolling windows
"command-code",
"conol-web",
"cnl",
// Alibaba Coding Plan triple-window quota (#9603 UI gap — fetcher existed, list entry missing)
"bailian-coding-plan",
// Qwen Cloud / Model Studio personal Token Plan (cookie-authenticated console gateway)
"qwen-cloud-token-plan",
// AgentRouter (New-API) console balance quota (consoleApiKey + newApiUserId)
"agentrouter",
// Kilo Code personal USD balance (GET /api/profile/balance, existing OAuth token)
"kilocode",
];
export { USAGE_SUPPORTED_PROVIDERS } from "@omniroute/open-sse/services/usage/supportedProviders.ts";
// ── Zod validation, lazily on first AI_PROVIDERS access (perf: skips the walk
// for processes that never touch AI_PROVIDERS, e.g. short-lived CLI commands) ──

View File

@@ -7,6 +7,7 @@ import {
} from "../../open-sse/config/providerPluginManifest.ts";
import type { RegistryEntry } from "../../open-sse/config/providers/shared.ts";
import { USAGE_FETCHER_PROVIDERS } from "../../open-sse/services/usage/fetcherProviders.ts";
import { USAGE_SUPPORTED_PROVIDERS } from "../../open-sse/services/usage/supportedProviders.ts";
const registryFixture: Record<string, RegistryEntry> = {
openai: {
@@ -182,3 +183,81 @@ test("usage-fetch matches the fetcher list by alias too (#11722)", () => {
assert.ok(entry);
assert.ok(entry.capabilities.includes("usage-fetch"));
});
test("manifest advertises usage-supported for providers whose usage API is accepted (#10078)", () => {
// claude is in USAGE_SUPPORTED_PROVIDERS, openai is not — assert against the real
// list so the test cannot drift silently if the list moves.
const claude = getProviderPluginManifestEntryFromRegistry(registryFixture, "claude");
assert.ok(claude);
assert.ok(
(USAGE_SUPPORTED_PROVIDERS as readonly string[]).includes("claude"),
"fixture guard: claude must stay in USAGE_SUPPORTED_PROVIDERS for this test to mean anything"
);
assert.ok(
claude.capabilities.includes("usage-supported"),
"claude is in USAGE_SUPPORTED_PROVIDERS, so the manifest must advertise usage-supported"
);
});
test("manifest omits usage-supported for providers outside USAGE_SUPPORTED_PROVIDERS (#10078)", () => {
for (const providerId of ["openai", "anthropic", "claude-web"]) {
const entry = getProviderPluginManifestEntryFromRegistry(registryFixture, providerId);
assert.ok(entry, `fixture guard: ${providerId} must resolve`);
assert.equal(
(USAGE_SUPPORTED_PROVIDERS as readonly string[]).includes(entry.id),
false,
`fixture guard: ${entry.id} must stay out of USAGE_SUPPORTED_PROVIDERS`
);
assert.equal(
entry.capabilities.includes("usage-supported"),
false,
`${entry.id} is not in USAGE_SUPPORTED_PROVIDERS, so usage-supported must not be advertised`
);
}
});
test("usage-supported matches only on id, not alias (#10078)", () => {
// USAGE_SUPPORTED_PROVIDERS is checked with a plain .includes(providerId) — no alias
// resolution (providerQuotaVisibility.ts:12, providerLimits.ts:178). The manifest must
// keep the same rule: an alias-only hit must NOT emit the tag.
const aliasOnlyFixture: Record<string, RegistryEntry> = {
"some-provider": {
id: "some-provider",
alias: "claude",
format: "openai",
executor: "default",
baseUrl: "https://some.example/v1/chat/completions",
authType: "apikey",
authHeader: "bearer",
models: [{ id: "m1", name: "M1" }],
},
};
assert.equal(
(USAGE_SUPPORTED_PROVIDERS as readonly string[]).includes("some-provider"),
false,
"fixture guard: the id must NOT be in the list"
);
assert.ok(
(USAGE_SUPPORTED_PROVIDERS as readonly string[]).includes("claude"),
"fixture guard: the alias must be in the list, otherwise this test proves nothing"
);
const entry = getProviderPluginManifestEntryFromRegistry(aliasOnlyFixture, "some-provider");
assert.ok(entry);
assert.equal(
entry.capabilities.includes("usage-supported"),
false,
"usage-supported is id-only — an alias hit must not advertise it"
);
// Sanity: the same entry MUST still carry usage-fetch via its alias, proving the
// two tags deliberately diverge on alias handling.
assert.ok(
(USAGE_FETCHER_PROVIDERS as readonly string[]).includes("claude"),
"fixture guard: claude must also be in USAGE_FETCHER_PROVIDERS for the divergence check"
);
assert.ok(entry.capabilities.includes("usage-fetch"));
});

View File

@@ -34,23 +34,6 @@ test("passthrough no fake: tool_only contentLength==0 -> no estimate (tool_calls
import { createSSEStream } from "../../open-sse/utils/stream.ts";
function collectSSE(stream: TransformStream<Uint8Array, Uint8Array>) {
return async (writable: WritableStream<Uint8Array>, readable: ReadableStream<Uint8Array>) => {
const chunks: string[] = [];
const decoder = new TextDecoder();
const reader = readable.getReader();
try {
while (true) {
const { done, value } = await reader.read();
if (done) break;
chunks.push(decoder.decode(value, { stream: true }));
}
} finally {
reader.releaseLock();
}
return chunks.join("");
};
}
function parseSSEUsage(sseText: string): unknown[] {
return sseText
@@ -107,7 +90,7 @@ test("passthrough SSE: finish stop without usage + include_usage:true -> emits u
assert.ok(typeof usage.completion_tokens === "number" && usage.completion_tokens > 0);
});
test("passthrough SSE: trailing choices:[] valid after estimated finish -> trailing is dropped (estimated wins)", async () => {
test("passthrough SSE: real trailing choices:[] usage is forwarded; no estimate is emitted (real wins)", async () => {
const body = { model: "m", messages: [{ role: "user", content: "hi" }], stream: true, stream_options: { include_usage: true } };
const stream = createSSEStream({
mode: "passthrough" as const,
@@ -129,17 +112,23 @@ test("passthrough SSE: trailing choices:[] valid after estimated finish -> trail
})();
const enc = new TextEncoder();
await writer.write(enc.encode(`data: ${JSON.stringify({ id: "chatcmpl-1", object: "chat.completion.chunk", choices: [{ index: 0, delta: { content: "hello world" }, finish_reason: null }] })}\n\n`));
// finish without usage -> should estimate (injectedUsage=false at that point)
// finish without usage -> passes through untouched (estimate only happens at flush, and only if no usage ever arrives)
await writer.write(enc.encode(`data: ${JSON.stringify({ id: "chatcmpl-1", object: "chat.completion.chunk", choices: [{ index: 0, delta: {}, finish_reason: "stop" }] })}\n\n`));
// trailing choices:[] with valid usage 50ms after -> inside empty-choices block hasValid(emptyChoicesUsage)&&!injectedUsage is now false, so chunk is dropped (warn path)
// trailing choices:[] with valid usage -> forwarded verbatim (marks passthroughForwardedUsage, so flush skips the estimate)
await writer.write(enc.encode(`data: ${JSON.stringify({ id: "chatcmpl-1", object: "chat.completion.chunk", choices: [], usage: { prompt_tokens: 8, completion_tokens: 6, total_tokens: 14 } })}\n\n`));
await writer.write(enc.encode("data: [DONE]\n\n"));
await writer.close();
const text = await readAll;
const parsed = parseSSEUsage(text);
const withUsage = parsed.filter((p: unknown) => (p as Record<string, unknown>).usage);
// With the guard, the trailing valid is dropped (estimated was already sent on finish). Without guard we would see 2 (double). We assert drop.
// If upstream ever sends real include_usage trailing, this documents the v1 tradeoff: estimated wins, valid is dropped.
assert.equal(withUsage.length, 1, `expected 1 usage (estimated, trailing dropped), got ${withUsage.length} — usages: ${JSON.stringify(withUsage.map((p) => (p as Record<string, unknown>).usage))}`);
assert.equal((withUsage[0] as Record<string, unknown> & { usage: Record<string, unknown> }).usage.estimated, true);
// v2 contract (#12151 follow-up): the upstream's REAL trailing usage block is forwarded
// and wins; the estimate exists only for upstreams that never report usage (emitted at
// flush). Exactly one usage block ever reaches the client — never two, never estimated
// when a real one arrived (the v1 "estimated wins" tradeoff was a billing regression).
assert.equal(withUsage.length, 1, `expected 1 usage (the real trailing block), got ${withUsage.length} — usages: ${JSON.stringify(withUsage.map((p) => (p as Record<string, unknown>).usage))}`);
const forwarded = (withUsage[0] as Record<string, unknown> & { usage: Record<string, unknown> }).usage;
assert.equal(forwarded.estimated, undefined);
assert.equal(forwarded.prompt_tokens, 8);
assert.equal(forwarded.completion_tokens, 6);
assert.equal(forwarded.total_tokens, 14);
});

View File

@@ -21,9 +21,7 @@ const MAP = {
"tests/unit/api/chat-route.test.ts",
"tests/unit/combo/combo-strategy.test.ts",
],
"src/shared/constants/routingStrategies.ts": [
"tests/unit/combo/combo-strategy.test.ts",
],
"src/shared/constants/routingStrategies.ts": ["tests/unit/combo/combo-strategy.test.ts"],
},
};
@@ -88,3 +86,54 @@ test("selectImpacted: non-source files are ignored (no __RUN_ALL__)", () => {
});
assert.deepEqual(sel, []);
});
// ── #8084 D1 completion: stdin mode + loader parity ─────────────────────────
import fs from "node:fs";
import path from "node:path";
import { changedFilesFromStdin } from "../../scripts/quality/select-impacted-tests.mjs";
test("changedFilesFromStdin: one path per line, trimmed, blanks dropped", () => {
assert.deepEqual(changedFilesFromStdin(" src/a.ts \n\nopen-sse/b.ts\r\n\n"), [
"src/a.ts",
"open-sse/b.ts",
]);
assert.deepEqual(changedFilesFromStdin(""), []);
assert.deepEqual(changedFilesFromStdin(undefined), []);
});
const SCRIPT = fs.readFileSync(
path.resolve(import.meta.dirname, "../../scripts/quality/test-scoped.sh"),
"utf8"
);
test("test-scoped.sh feeds the selector via --stdin (staged mode must not read git commits)", () => {
assert.match(SCRIPT, /select-impacted-tests\.mjs" --stdin/);
});
test("test-scoped.sh mirrors the CI loader split (#6787): dashboard→tsx, serial→concurrency=1, rest→tsx/esm", () => {
assert.match(SCRIPT, /tests\/unit\/dashboard\/\*\) DASH\+=/);
assert.match(SCRIPT, /tests\/unit\/serial\/\*\) SERIAL\+=/);
assert.match(
SCRIPT,
/node --import tsx "\$\{NODE_COMMON\[@\]\}" --test-concurrency=4 "\$\{DASH\[@\]\}"/
);
assert.match(
SCRIPT,
/node --import tsx\/esm "\$\{NODE_COMMON\[@\]\}" --test-concurrency=1 "\$\{SERIAL\[@\]\}"/
);
assert.match(
SCRIPT,
/node --import tsx\/esm "\$\{NODE_COMMON\[@\]\}" --test-concurrency=4 "\$\{REST\[@\]\}"/
);
});
test("package.json exposes every mode the script header documents", () => {
const pkg = JSON.parse(
fs.readFileSync(path.resolve(import.meta.dirname, "../../package.json"), "utf8")
);
for (const name of ["test:scoped", "test:scoped:staged", "test:scoped:full"]) {
assert.ok(pkg.scripts[name], `missing script ${name}`);
assert.match(pkg.scripts[name], /scripts\/quality\/test-scoped\.sh/);
}
assert.match(pkg.scripts["test:scoped:full"], /--full/);
});