mirror of
https://github.com/diegosouzapw/OmniRoute.git
synced 2026-09-14 19:02:17 +03:00
5a0a131bc7
* feat(usage): devin-cli agentic quota + openrouter credits in Provider Limits Two provider families with live quota APIs were missing from the Provider Limits dashboard because their list entries were absent: - devin-cli: new usage leaf querying the Codeium seat-management Connect API (exa.seat_management_pb.SeatManagementService/GetUserStatus, protobuf over POST with the raw `Basic <token>-<token>` auth header the CLI itself uses). Surfaces the plan name plus daily/weekly agentic quota percentages with reset timestamps from the GetUserStatus plan_status payload, via a minimal hand-rolled protobuf encoder/reader (no proto dependency warranted for two fixed messages). - openrouter: the /key + /credits quota fetcher (#6842) was already wired into the dispatcher but gated out of the bulk sync — add it to USAGE_SUPPORTED_PROVIDERS and PROVIDER_LIMITS_APIKEY_PROVIDERS so key limits and account credits actually surface. * fix(build): externalize tiktoken so tiktoken_bg.wasm resolves at runtime The vendored ChatGPT Web connector v4.0.7 (#12181) imports tiktoken (get_encoding) at module level. tiktoken's node build reads tiktoken_bg.wasm via a __dirname-relative fs.readFileSync during import; when Next bundles the package the wasm asset is not traced into the server chunk, and page-data collection for every route reaching the tokenizer (e.g. /api/providers/[id]/chatgpt-web-codex-doctor) aborts with "Missing tiktoken_bg.wasm" — breaking the whole standalone build. Externalize it like the other runtime-resolved native/wasm packages (sql.js, sqlite-vec, better-sqlite3): the require stays at runtime, where node_modules/tiktoken/tiktoken_bg.wasm resolves normally. * fix(openrouter): /credits balance survives a /key failure OpenRouter is credit-based, not subscription-based: the authoritative remaining-credits signal is GET /api/v1/credits (total_credits - total_usage, the documented "get remaining credits" endpoint), while the /key limit fields are optional per-key caps that most accounts never set. fetchOpenrouterQuota previously treated /key as mandatory — any /key failure (429 rate limit, transient error, unexpected shape) discarded the whole payload and the Usage dashboard showed "OpenRouter (usage endpoint unreachable)" even though /credits was reachable. Now: - /key unavailable + /credits OK → credits-only quota (creditBalance = total_credits - total_usage) instead of null - /key 401/403 alone no longer means an invalid token; only a double auth-rejection (both endpoints) does - null is returned only when both endpoints fail, and the dashboard label reflects that ("credits endpoint unreachable") * fix(openrouter): render AI Credits as a USD credit count in Provider Limits The Provider Limits card's dollar renderer only activates on isCredits/creditCount rows (QuotaCardExpanded), but openrouter went through parseGeneric — which drops `currency` and never sets those flags — so the credits balance rendered as a meaningless "100% left" (the unlimited-credits row is always 100%) instead of the actual credit count. Route openrouter's `credits` quota through buildCreditsQuota() like the DeepSeek/AgentRouter credits rows: label "AI Credits", dollar-formatted balance. Free-tier request windows keep the generic percentage treatment. * fix(usage): document DEVIN_SEAT_API_URL and split quota parsers Keep fetchOpenrouterQuota and decodeProtoFields under the complexity ratchets, and add the seat-management URL to the env/docs contract. Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com> * test(usage): drop duplicated GLM quota-ordering test in provider-limits-ui * test(usage): drop stale openrouter ACCEPTED_DIVERGENCE OpenRouter is now in both USAGE_FETCHER_PROVIDERS and USAGE_SUPPORTED_PROVIDERS, so the recorded aggregator divergence is no longer real. Add the changelog fragment. Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com> --------- Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com> Co-authored-by: Diego Rodrigues de Sa e Souza <diegosouza.pw@gmail.com>
128 lines
6.2 KiB
Markdown
128 lines
6.2 KiB
Markdown
---
|
|
title: "Provider Plugin Manifest"
|
|
version: 3.8.42
|
|
lastUpdated: 2026-07-01
|
|
---
|
|
|
|
# Provider Plugin Manifest
|
|
|
|
`open-sse/config/providerPluginManifest.ts` defines the JSON-safe provider
|
|
plugin contract. `open-sse/config/providerPluginManifestRegistry.ts` binds that
|
|
contract to the current provider registry for sidecars such as Bifrost,
|
|
CLIProxyAPI, or a future Go/Rust router. The TypeScript registry remains the
|
|
source of truth, but sidecars can consume the manifest without importing
|
|
executor code, OAuth defaults, headers, or process environment state.
|
|
|
|
The same manifest is available over HTTP at
|
|
`GET /api/v1/provider-plugin-manifest` for sidecars that run out-of-process.
|
|
|
|
OmniRoute advertises that URL to Bifrost and CLIProxyAPI via the
|
|
`X-OmniRoute-Provider-Manifest-Url` request header. Set
|
|
`OMNIROUTE_PROVIDER_MANIFEST_URL` when the sidecar needs a public or container
|
|
network URL instead of the local request origin.
|
|
|
|
## Refreshing the Manifest
|
|
|
|
The HTTP endpoint returns `Cache-Control: public, max-age=60` and a strong
|
|
`ETag`. A sidecar should retain the last validated manifest and send its ETag
|
|
in `If-None-Match` when refreshing. A `304 Not Modified` response has no body;
|
|
the sidecar keeps its cached manifest. If no validated cached manifest exists,
|
|
the sidecar must issue an unconditional request instead of accepting a `304`.
|
|
|
|
## Goal
|
|
|
|
Move provider metadata toward a plugin contract so the hot request path can
|
|
eventually be owned by a lower-latency sidecar while OmniRoute keeps the
|
|
TypeScript route as the policy gate and fallback. The manifest is additive: it
|
|
does not change request routing by itself.
|
|
|
|
## Contract
|
|
|
|
The manifest contains:
|
|
|
|
- provider id and alias
|
|
- upstream format and executor name
|
|
- auth type, auth header, and optional auth prefix
|
|
- static endpoint metadata
|
|
- sidecar eligibility and explicit reasons when a provider should stay on TS
|
|
- JSON-safe model metadata such as context length, vision/reasoning flags, and
|
|
unsupported params
|
|
- capability tags including `apikey`, `oauth`, `custom-executor`,
|
|
`passthrough-models`, `responses`, `sidecar-candidate`, `usage-fetch`, and `usage-supported`
|
|
|
|
The manifest intentionally excludes:
|
|
|
|
- OAuth client secrets and default secret values
|
|
- runtime environment resolution
|
|
- request headers and public credential helpers
|
|
- dynamic URL builders
|
|
- executor functions
|
|
- session pool internals
|
|
|
|
## Capability Tags
|
|
|
|
`capabilities` is a sorted array of tags derived from the registry entry. Integrators
|
|
should treat it as the machine-readable answer to "what can this provider do", instead of
|
|
re-reading the TypeScript sources.
|
|
|
|
| Tag | Meaning |
|
|
| -------------------- | ----------------------------------------------------------------- |
|
|
| `apikey` | Accepts an API key (`authType` is `apikey` or `optional`). |
|
|
| `oauth` | Uses an OAuth or session flow. |
|
|
| `responses` | Exposes an OpenAI Responses-API base URL. |
|
|
| `passthrough-models` | Serves models straight from upstream instead of a static catalog. |
|
|
| `custom-executor` | Runs a non-default executor, so it stays on the TypeScript path. |
|
|
| `sidecar-candidate` | Mirrors `sidecar.eligible` — safe to consider for sidecar import. |
|
|
| `usage-fetch` | Has a wired usage or quota fetcher (`getUsageForProvider`). |
|
|
| `usage-supported` | The usage API accepts this provider (`isSupportedUsageConnection`). |
|
|
|
|
`usage-fetch` is discovery only. It reports that OmniRoute knows how to read usage for the
|
|
provider; it does not activate fetching, change quota semantics, or imply that the
|
|
Dashboard quota widget is enabled for the provider — that widget is gated separately by
|
|
`USAGE_SUPPORTED_PROVIDERS`. The source of truth is `USAGE_FETCHER_PROVIDERS` in
|
|
`open-sse/services/usage/fetcherProviders.ts`.
|
|
|
|
That list is keyed by the strings the usage dispatcher accepts, so it mixes canonical ids
|
|
with aliases and is slightly longer than the number of tagged providers: entries that are
|
|
not chat providers in the manifest registry (for example the `firecrawl` search provider
|
|
and the `amazon-q` ACP provider) have no manifest entry to tag.
|
|
|
|
`usage-supported` answers whether the server and Dashboard usage routes accept a connection
|
|
for the provider. It mirrors `isSupportedUsageConnection()` (`src/lib/usage/providerLimits.ts`)
|
|
and `supportsProviderQuota()` (`src/shared/utils/providerQuotaVisibility.ts`), both gated by
|
|
`USAGE_SUPPORTED_PROVIDERS` (`open-sse/services/usage/supportedProviders.ts`). Unlike
|
|
`usage-fetch`, it is emitted on the provider id alone — the runtime guard does
|
|
`USAGE_SUPPORTED_PROVIDERS.includes(providerId)` with no alias resolution, so the manifest
|
|
keeps the same rule. The two tags have different perimeters: 3 providers carry only
|
|
`usage-fetch` (`opencode`, `opencode-zen`, `xai`) and 1 carries only
|
|
`usage-supported` (`xiaomi-mimo-token-plan`), so one does not imply the other.
|
|
|
|
## Sidecar Use
|
|
|
|
Sidecars should treat `sidecar.eligible` as a conservative candidate signal, not
|
|
as an unconditional routing decision. The first import target should be
|
|
API-key, static-endpoint providers using the default executor. Providers with
|
|
custom web executors, OAuth/session flows, dynamic URL builders, or pool config
|
|
stay on the TypeScript fallback path until a sidecar implements equivalent
|
|
behavior and telemetry proves parity.
|
|
|
|
Suggested migration phases:
|
|
|
|
1. Generate and validate the provider plugin manifest from the TS registry.
|
|
2. Teach Bifrost or CLIProxyAPI to import the manifest for API-key/static
|
|
providers.
|
|
3. Route eligible providers through the sidecar behind `OMNIROUTE_RELAY_BACKEND`
|
|
while keeping TS fallback enabled.
|
|
4. Promote providers only when success rate, p99 latency, streaming behavior,
|
|
and unsupported-param handling match the TS path.
|
|
5. Add sidecar-native plugins for custom executors one provider family at a
|
|
time.
|
|
|
|
## Why Not Embed Providers Directly In Next
|
|
|
|
The Next frontend should not own provider execution. It should call the API
|
|
boundary. The backend can then decide whether to use the TypeScript executor,
|
|
Bifrost, CLIProxyAPI, or a future native sidecar. This keeps request signing,
|
|
allowlist checks, DB policy, and fallback behavior centralized before any
|
|
sidecar handoff.
|