mirror of
https://github.com/diegosouzapw/OmniRoute.git
synced 2026-08-11 09:42:15 +03:00
docs(guardrails): document Audio Bridge runtime
This commit is contained in:
diegosouzapw
@@ -7,7 +7,7 @@ lastUpdated: 2026-08-08
|
||||
# Guardrails
|
||||
|
||||
> **Source of truth:** `src/lib/guardrails/`
|
||||
> **Last updated:** 2026-08-08 — v3.8.50 (Modality Bridge PR-2: dashboard settings page, live stats, test action, and media-provider shortcuts)
|
||||
> **Last updated:** 2026-08-08 — v3.8.50 (Modality Bridge PR-3: Audio Bridge runtime and functional Audio settings tab)
|
||||
|
||||
Guardrails enforce safety, policy, and content transformations at the boundary
|
||||
between OmniRoute and upstream providers. Each guardrail can inspect (and
|
||||
@@ -20,12 +20,13 @@ request. Blocking is an explicit decision (`block: true`), never an accident.
|
||||
|
||||
## Built-in Guardrails
|
||||
|
||||
The registry auto-loads four guardrails in priority order on import
|
||||
The registry auto-loads five guardrails in priority order on import
|
||||
(see `registry.ts` → `registerDefaultGuardrails()`):
|
||||
|
||||
| Priority | Name | Stage(s) | File |
|
||||
| -------- | ------------------- | -------------- | --------------------- |
|
||||
| `5` | `vision-bridge` | `preCall` | `visionBridge.ts` |
|
||||
| `6` | `audio-bridge` | `preCall` | `audioBridge.ts` |
|
||||
| `10` | `pii-masker` | `pre` + `post` | `piiMasker.ts` |
|
||||
| `20` | `prompt-injection` | `preCall` | `promptInjection.ts` |
|
||||
| `95` | `credential-masker` | `pre` + `post` | `credentialMasker.ts` |
|
||||
@@ -113,8 +114,8 @@ The new `modalityBridge*` keys are Zod-validated in `updateSettingsSchema`
|
||||
`modalityBridgeVisionMode`, `modalityBridgeVisionModel`,
|
||||
`modalityBridgeVisionTaskAware`, `modalityBridgeVisionPrompt`,
|
||||
`modalityBridgeVisionTimeout`, `modalityBridgeVisionMaxImages`, the
|
||||
`modalityBridgeCache*` trio, and the PR-3-reserved `modalityBridgeAudio*`
|
||||
group. Migration `141_modality_bridge_settings.sql` copies existing legacy
|
||||
`modalityBridgeCache*` trio, and the `modalityBridgeAudio*` group used by the
|
||||
Audio Bridge. Migration `141_modality_bridge_settings.sql` copies existing legacy
|
||||
`visionBridge*` values to the matching new keys (idempotent, never overwrites
|
||||
an operator-set `modalityBridge*` value); the legacy keys stay accepted as a
|
||||
read fallback for one release cycle.
|
||||
@@ -130,8 +131,8 @@ swap is already visible in the response body's `model` field.
|
||||
|
||||
`GET /api/modality-bridge/stats` (management auth, same tier as
|
||||
`GET /api/settings`) returns the in-memory per-modality counters
|
||||
`{ bridged, cacheHits, failures, lastUsedAt }` for `vision` (and the
|
||||
PR-3-reserved `audio`). Counters reset on process restart by design
|
||||
`{ bridged, cacheHits, failures, lastUsedAt }` for `vision` and `audio`.
|
||||
Counters reset on process restart by design
|
||||
(telemetry, not accounting).
|
||||
|
||||
#### Dashboard configuration
|
||||
@@ -139,11 +140,12 @@ PR-3-reserved `audio`). Counters reset on process restart by design
|
||||
The dedicated dashboard page is
|
||||
`/dashboard/settings/modality-bridge`. Its URL-addressable `Vision`, `Audio`,
|
||||
and `Video` tabs preserve query parameters while switching the `tab` value.
|
||||
The Vision tab is live: it exposes enablement, mode, model selection (including
|
||||
the automatic default), task-aware prompting, advanced timeout/image/cache
|
||||
limits, runtime counters, and a guarded sample request. Audio and Video are
|
||||
explicit placeholders: Audio is reserved for PR-3, while Video remains tracked
|
||||
in issue `#9760`.
|
||||
The Vision tab exposes enablement, mode, model selection (including the automatic
|
||||
default), task-aware prompting, advanced timeout/image/cache limits, runtime
|
||||
counters, and a guarded sample request. The Audio tab is also live: it exposes
|
||||
enablement, an STT-only model picker with Auto, timeout/max-clip limits, audio
|
||||
counters, and an `input_audio` sample test. Video remains the explicit placeholder
|
||||
tracked in issue `#9760`.
|
||||
|
||||
The former Vision Bridge card under AI settings is a compatibility link to the
|
||||
new page; it no longer owns a second copy of the form. Media Providers also
|
||||
@@ -165,6 +167,64 @@ new mode/task-aware/cache defaults and the settings resolver live in
|
||||
`deps` constructor option so tests can inject fake `getSettings` and
|
||||
`callVisionModel` implementations.
|
||||
|
||||
### Audio Bridge (`audioBridge.ts`) — Modality Bridge PR-3
|
||||
|
||||
Intercepts audio-bearing chat requests before they reach a target that is not
|
||||
known to accept audio input. It never reroutes the chat request: audio parts are
|
||||
transcribed through the existing OpenAI-compatible multipart endpoint and the
|
||||
chosen chat model continues with text transcripts.
|
||||
|
||||
Flow:
|
||||
|
||||
1. Resolve `supportsAudio` through `getResolvedModelCapabilities()`. Explicit
|
||||
provider-registry metadata wins, then static model metadata, then synced
|
||||
`modalities_input`. A declared input list without `audio` is `false`; no
|
||||
capability evidence remains `null`. Both `false` and `null` activate the
|
||||
conservative bridge, while `true` bypasses it.
|
||||
2. Resolve `modalityBridgeAudio*` settings and extract spliceable top-level
|
||||
audio parts from every message through the shared `detectMediaParts()`
|
||||
detector. Supported wire shapes are OpenAI `input_audio`, `audio_url`, and
|
||||
`source.media_type: "audio/*"`. Nested audio is detected for routing but not
|
||||
removed by the splice path. Work is capped by `modalityBridgeAudioMaxClips`;
|
||||
later parts stay untouched.
|
||||
3. Honor a configured `provider/model`, or let `selectAudioBridgeModel()` walk
|
||||
`AUDIO_TRANSCRIPTION_PROVIDERS` in stable catalog order and select the first
|
||||
model with a usable active provider credential.
|
||||
4. `callAudioTranscription()` converts base64/data-URI audio to a multipart
|
||||
`file`, or downloads a remote `audio_url` through the public-only outbound
|
||||
guard with DNS pinning and a 25 MB bound. It then POSTs the file and selected
|
||||
model to the local `/v1/audio/transcriptions` self-loop, authenticated with
|
||||
`resolveSelfLoopBearer()`. The existing transcription route performs normal
|
||||
credential lookup, cooldown/rate-limit handling, and provider dispatch.
|
||||
5. Successful calls replace their parts with `[Audio N]: <transcript>`. Calls
|
||||
run with `Promise.allSettled`: an individual failure preserves that original
|
||||
audio part (#4012 contract). If every call fails and the target is proven
|
||||
`supportsAudio === false`, the parts become
|
||||
`[Audio N]: (unavailable — no STT provider connected)` (#8430 contract). For
|
||||
an unknown target (`null`), an all-failure result stays untouched. A proven
|
||||
text-only target with no usable STT credential receives the same explicit
|
||||
stub without issuing a network call.
|
||||
|
||||
Successful transcripts use the process-wide Modality Bridge LRU/TTL cache. The
|
||||
key combines the audio reference, the stable `audio-transcription` operation
|
||||
label, and selected STT model; failures are never cached. Audio attempts update
|
||||
the shared `bridged`, `cacheHits`, `failures`, and `lastUsedAt` counters.
|
||||
Transformed responses carry
|
||||
`x-omniroute-modality-bridge: audio->text;model=<sttModel>;parts=<n>`; untouched
|
||||
requests do not receive an Audio Bridge segment.
|
||||
|
||||
Runtime settings are DB-backed and Zod-validated:
|
||||
|
||||
| Key | Default | Range |
|
||||
| ----------------------------- | ------- | -------------- |
|
||||
| `modalityBridgeAudioEnabled` | `true` | — |
|
||||
| `modalityBridgeAudioModel` | `""` | Auto or STT ID |
|
||||
| `modalityBridgeAudioTimeout` | `60000` | 1000–300000 |
|
||||
| `modalityBridgeAudioMaxClips` | `3` | 1–10 |
|
||||
|
||||
The shared cache remains controlled by `modalityBridgeCacheEnabled`,
|
||||
`modalityBridgeCacheTtlMinutes`, and `modalityBridgeCacheMaxEntries`.
|
||||
|
||||
### PII Masker (`piiMasker.ts`)
|
||||
|
||||
Runs on **both** stages.
|
||||
@@ -370,8 +430,8 @@ Environment variables read by the built-in guardrails:
|
||||
| `PII_REDACTION_ENABLED` | `pii-masker` | When `true`, request PII is redacted (independent of injection mode). |
|
||||
| `PII_RESPONSE_SANITIZATION` / `_MODE` | `pii-masker` (downstream) | Controls response-side masker behavior. |
|
||||
|
||||
The Vision Bridge reads runtime config from the DB-backed settings store
|
||||
(`getSettings()`), not env vars. The primary keys are
|
||||
The Modality Bridge guardrails read runtime config from the DB-backed settings
|
||||
store (`getSettings()`), not env vars. Vision's primary keys are
|
||||
`modalityBridgeVisionEnabled`, `modalityBridgeVisionMode`,
|
||||
`modalityBridgeVisionModel`, `modalityBridgeVisionTaskAware`,
|
||||
`modalityBridgeVisionPrompt`, `modalityBridgeVisionTimeout`,
|
||||
@@ -382,6 +442,11 @@ fallback; dashboard writes use the primary keys. Defaults and the fallback
|
||||
resolver live in `src/shared/constants/modalityBridgeDefaults.ts`, with legacy
|
||||
constants retained in `src/shared/constants/visionBridgeDefaults.ts`.
|
||||
|
||||
Audio uses `modalityBridgeAudioEnabled`, `modalityBridgeAudioModel`,
|
||||
`modalityBridgeAudioTimeout`, and `modalityBridgeAudioMaxClips`, plus the shared
|
||||
`modalityBridgeCache*` settings. Audio has no legacy-key fallback because these
|
||||
keys were introduced with the Modality Bridge schema.
|
||||
|
||||
## Custom Guardrails
|
||||
|
||||
```typescript
|
||||
@@ -419,9 +484,11 @@ Steps:
|
||||
|
||||
Use `resetGuardrailsForTests()` between tests to start from a known state.
|
||||
Pass `{ registerDefaults: false }` to start with an empty registry and
|
||||
register only the guardrails under test. The Vision Bridge guardrail accepts
|
||||
dependency injection (`deps.getSettings`, `deps.callVisionModel`) so tests can
|
||||
exercise the full flow without DB or network access.
|
||||
register only the guardrails under test. Vision Bridge accepts dependency
|
||||
injection (`deps.getSettings`, `deps.callVisionModel`); Audio Bridge exposes the
|
||||
equivalent seams for settings, capabilities, STT model selection, credential
|
||||
checks, and transcription. Tests can therefore exercise both flows without DB
|
||||
or network access.
|
||||
|
||||
## See Also
|
||||
|
||||
@@ -430,6 +497,7 @@ exercise the full flow without DB or network access.
|
||||
prompt-injection and PII masking
|
||||
- `src/shared/constants/visionBridgeDefaults.ts` — Vision Bridge defaults and
|
||||
forced-bridge model list
|
||||
- `src/shared/constants/modalityBridgeDefaults.ts` — shared Vision/Audio runtime defaults
|
||||
- `docs/architecture/RESILIENCE_GUIDE.md` — orthogonal layer (circuit breaker, cooldowns)
|
||||
- `docs/reference/ENVIRONMENT.md` — full env var reference
|
||||
|
||||
|
||||
Reference in New Issue
Block a user