mirror of
https://github.com/diegosouzapw/OmniRoute.git
synced 2026-08-13 18:52:18 +03:00
chore(repo): remove one-shot reports and stale docs approved by owner
This commit is contained in:
backryun
4
.gitignore
vendored
4
.gitignore
vendored
@@ -57,10 +57,6 @@ node_modules/
|
||||
*.map
|
||||
.DS_Store
|
||||
|
||||
# Obsidian sync plugin — committed for community distribution
|
||||
!obsidian-plugin/
|
||||
obsidian-plugin/node_modules/
|
||||
|
||||
# Serena AI assistant config (local-only tool, not project code)
|
||||
.serena/
|
||||
|
||||
|
||||
@@ -1,115 +0,0 @@
|
||||
# Devin Claude Bridge Progress
|
||||
|
||||
Updated: 2026-07-28
|
||||
|
||||
## Baseline
|
||||
|
||||
- Fork version: `3.8.49`.
|
||||
- Starting branch: `release/v3.8.49`.
|
||||
- Starting commit: `ed7db3ee5f89a144b2d931d8605534522f83de30`.
|
||||
- Fixed runtime artifacts: Node `26.0.0`, Claude Code `2.1.220`, Devin CLI `3000.2.17`.
|
||||
- Existing `devin-cli` remains unchanged; the new path is the separate
|
||||
`devin-cli-agentic` provider.
|
||||
|
||||
## Implemented architecture
|
||||
|
||||
- Claude Code runs only inside the non-root bridge container with its own empty config
|
||||
volume and local OmniRoute base URL.
|
||||
- `devin-cli-agentic` preserves Anthropic messages, tool schemas, `tool_use`, and
|
||||
`tool_result`, then calls the official Devin CLI over ACP stdio.
|
||||
- The executor starts `devin acp --agent-type summarizer`. This is the only fixed official
|
||||
ACP role in the pinned CLI that has no Devin-owned tools.
|
||||
- The request is framed as an execution trace. Devin can return one strict client tool
|
||||
envelope; Claude Code executes that tool locally.
|
||||
- Internal ACP `tool_call` events, unsupported blocks, invalid schemas, narrative actions,
|
||||
timeouts, cancellation, and process failure all fail closed.
|
||||
- Provider and network policy prevent combo/auto/Anthropic fallback.
|
||||
|
||||
## Offline proof
|
||||
|
||||
- Focused serializer, parser, executor, ACP lifecycle, wire-format, environment, and audit
|
||||
tests pass (39/39).
|
||||
- The contract suite covers Anthropic JSON/SSE, `tool_use`, `tool_result` continuation,
|
||||
fragmented ACP frames, stderr, early exit, timeout, cancellation, and fail-closed provider
|
||||
loss.
|
||||
- The production bridge image builds with the pinned CLIs.
|
||||
- Real Claude Code offline E2E loads `CLAUDE.md`, the project skill and slash command, fires
|
||||
hooks, executes local tools over multiple turns, observes a failed test, repairs the file,
|
||||
reruns the test, and completes.
|
||||
- The isolation verifier proves non-root/read-only execution, isolated mounts and config,
|
||||
blocked Anthropic/Claude access, no host credential mounts, local-only inference, and no
|
||||
fallback.
|
||||
|
||||
Evidence is generated under `.sandbox/evidence` and ignored by Git.
|
||||
|
||||
## Regression status
|
||||
|
||||
- `typecheck:core`, focused ESLint, Prettier, shell/Node syntax, and the complete documentation
|
||||
accuracy suite pass.
|
||||
- The broad `npm run check` is not reported as passed: after its lint phase, the repository
|
||||
test runner remained alive while an existing `ioredis` client repeatedly retried an
|
||||
unavailable local Redis endpoint after `quota-redis-store.test.ts`. The bridge-focused
|
||||
suites, production image build, offline E2E, isolation proof, and live gate do not use that
|
||||
Redis service and all pass.
|
||||
|
||||
## Live Devin proof
|
||||
|
||||
Passed with the official in-container login and discovered model
|
||||
`swe-1-7-lightning`. The terminal live run completed all three scenarios:
|
||||
|
||||
1. Claude Code loaded the fixture instructions, issued client-owned `Read` calls, and
|
||||
returned a correct defect analysis.
|
||||
2. Claude Code issued a real `Edit` changing subtraction to addition, then a client-owned
|
||||
`Bash` call running `npm test`; the test reported one pass and zero failures.
|
||||
3. Claude Code initialization listed `bridge-check` and `bridge-proof`, read the corrected
|
||||
source and test, executed another client-owned `npm test`, and completed successfully.
|
||||
|
||||
The live evidence validator parses stream JSON and requires successful tool results. It does
|
||||
not accept a textual claim that a tool ran. It also rejects terminal summaries that report a
|
||||
blocker, incomplete work, or required next steps.
|
||||
|
||||
The final live gate reported:
|
||||
|
||||
```text
|
||||
PASS: validated Claude evidence for LIVE_ANALYSIS_COMPLETE
|
||||
PASS: validated Claude evidence for LIVE_FIX_COMPLETE
|
||||
PASS: validated Claude evidence for BRIDGE_E2E_COMPLETE
|
||||
PASS: three live Devin-backed Claude Code scenarios completed
|
||||
PASS: live model swe-1-7-lightning was discovered and validated by three scenarios
|
||||
```
|
||||
|
||||
The same gate validated the network audit: only the Devin guard path was used, no internal
|
||||
Devin tool event was accepted, and the Claude egress audit remained empty.
|
||||
|
||||
## Investigation conclusion
|
||||
|
||||
The initial default-agent hypothesis failed because ACP permission modes do not turn the
|
||||
default Devin agent into a raw inference backend. Even `ask` mode can emit Devin-owned
|
||||
`tool_call` events. A discovered `allowed-tools: []` agent configuration was not consumed by
|
||||
`devin acp` in CLI `3000.2.17`.
|
||||
|
||||
The working adaptation uses the official `summarizer` agent because it is structurally
|
||||
no-tools. Its fixed summarization behavior can produce intermediate prose, so the bridge
|
||||
frames requests as execution traces, detects future-action narration, performs at most one
|
||||
strict repair, and otherwise fails. Live validation also exposed transient ACP timeouts;
|
||||
the harness now spaces independent scenarios rather than weakening routing or retrying into
|
||||
another provider.
|
||||
|
||||
## Safety record
|
||||
|
||||
No host Claude executable, configuration, login, OAuth token, Keychain, or Anthropic API was
|
||||
used. The dedicated Docker volumes remain role-separated. No credential value is written to
|
||||
the repository or evidence output.
|
||||
|
||||
During the early baseline, a focused test without isolated `DATA_DIR` initialized the
|
||||
repository's normal OmniRoute database at `/Users/lucasisrael/.omniroute/storage.sqlite`.
|
||||
It was not rolled back or touched again. Every bridge command now pins database and temporary
|
||||
paths under the worktree's `.sandbox` directory.
|
||||
|
||||
## Remaining limits
|
||||
|
||||
- The no-tools backend has a summarizer system role rather than a neutral generation role.
|
||||
- One client tool call per response is supported; parallel tool calls are rejected.
|
||||
- ACP processes are per-turn and stateless.
|
||||
- Live Devin availability can still produce explicit `502`/`504` failures.
|
||||
- Images and unadvertised vision/effort/large-context capabilities remain unsupported.
|
||||
@@ -1,200 +0,0 @@
|
||||
---
|
||||
title: "Incident Response Runbook — OmniRoute (2026-06-18)"
|
||||
version: 3.8.50
|
||||
lastUpdated: 2026-08-06
|
||||
---
|
||||
|
||||
# Incident Response Runbook — OmniRoute (2026-06-18)
|
||||
|
||||
**Status**: Authoritative. The 71-pillar audit (L61) references this doc
|
||||
for the `Obs > 2.00` gate.
|
||||
**Owner**: observability-circle (lead: security-circle lead).
|
||||
**SLOs**: see `docs/PERF_BUDGETS.md` § 1 (top-level SLOs) and
|
||||
`ops/slos.yaml` (machine-readable form, generated by the Bifrost team).
|
||||
**Disclosure policy**: see `SECURITY.md` (vulnerability disclosure only,
|
||||
separate flow).
|
||||
|
||||
This runbook is the operational playbook for **non-security** incidents:
|
||||
outages, latency regressions, error-budget burn, and provider-side
|
||||
failures. Vulnerability disclosure stays on `SECURITY.md`; do not route
|
||||
those through this runbook.
|
||||
|
||||
---
|
||||
|
||||
## 1. Severity ladder
|
||||
|
||||
| Sev | Definition | Examples | Page on | Resolve by |
|
||||
|---|---|---|---|---|
|
||||
| **SEV-1** | User-visible outage; > 50 % of requests failing or > 2x SLO breach for 5 min. | Cluster down; auth layer broken; 5xx flood. | On-call P0 (immediate) | 4 h |
|
||||
| **SEV-2** | Significant degradation; 1.5–2x SLO breach for 15 min, or single-tenant impact. | Single provider down; p95 > 1.5x budget; rate-limit runaway. | On-call P1 (15 min) | 24 h |
|
||||
| **SEV-3** | Latent bug or near-miss; no current user impact but error budget at risk. | Memory leak trending up; circuit breaker tripping on one provider. | Slack `#omniroute-ops` (next standup) | 7 d |
|
||||
| **SEV-4** | Cosmetic / informational. | Log line noise; non-binding UI glitch. | Next weekly review | Next refactor cycle |
|
||||
|
||||
**Burn-rate escalation** (per `docs/PERF_BUDGETS.md` § 1): 6x for 5 min
|
||||
is SEV-1; 2x for 1 h is SEV-2; sustained < 1x for 7 d demotes to SEV-3.
|
||||
|
||||
---
|
||||
|
||||
## 2. Detection sources
|
||||
|
||||
| Source | Signal | Routing |
|
||||
|---|---|---|
|
||||
| Prometheus (`/metrics`) | Counter deltas (5xx, latency) | Alertmanager → PagerDuty |
|
||||
| Grafana SLO dashboards | SLO burn-rate panels | Slack `#omniroute-ops` |
|
||||
| Uptime probe (`/api/health/ping`) | 3 consecutive failures from 3 regions | Alertmanager → PagerDuty |
|
||||
| Dependabot | New CVE in dependency | GitHub issue + Slack `#security` |
|
||||
| User report (support@) | Manual triage | Slack `#omniroute-triage` |
|
||||
| Error budget burn alert | `slo_burn_rate > threshold` | Alertmanager |
|
||||
|
||||
Prometheus and Alertmanager are configured in the deploy repo (see
|
||||
`docs/operations/DEPLOY.md` once published; currently inline in
|
||||
`docker-compose.prod.yml`).
|
||||
|
||||
---
|
||||
|
||||
## 3. First-15-minutes checklist
|
||||
|
||||
When paged, the on-call engineer runs this checklist verbatim. **Do
|
||||
not** skip steps; each is timed.
|
||||
|
||||
1. **0:00** — Acknowledge the page in PagerDuty. Stops the escalation
|
||||
timer and notifies the secondary.
|
||||
2. **0:02** — Open the [SLO dashboard][dash] and the [incident
|
||||
channel][chan] (`#inc-YYYY-MM-DD-slug`). Post a single-line ack
|
||||
with the alert name and the time.
|
||||
3. **0:05** — Classify severity per § 1. If SEV-1 or SEV-2, declare
|
||||
the incident in the channel and tag `@incident-commander`.
|
||||
4. **0:08** — Capture the alert payload, the most recent deploy SHA,
|
||||
and the top 5 slow / erroring endpoints. Post to the channel.
|
||||
5. **0:12** — Decide: **mitigate first, root-cause later**. Choose
|
||||
one of:
|
||||
- **Roll back** to the last green deploy (`bin/rollback.sh vX.Y.Z`).
|
||||
- **Failover** to the healthy replicas (Caddy LB removes the bad
|
||||
replica automatically; verify with `curl /api/health/ping`).
|
||||
- **Disable** the broken connection(s) via `PUT /api/providers/{connectionId}`
|
||||
with body `{ "isActive": false }` (per-connection toggle, safe by
|
||||
default; repeat per key/account — see § 4.1).
|
||||
6. **0:15** — Post the chosen mitigation in the channel. If the page
|
||||
is still firing after 5 more minutes, escalate to the secondary.
|
||||
|
||||
[chan]: TBD — set to your team's incident-chat channel (e.g. a Discord/Slack `#inc-*` channel); not provisioned by this repo.
|
||||
[dash]: TBD — set to your Grafana/observability dashboard URL; not provisioned by this repo.
|
||||
|
||||
---
|
||||
|
||||
## 4. Mitigation runbooks (per failure mode)
|
||||
|
||||
### 4.1 Provider outage (single provider down)
|
||||
|
||||
1. `PUT /api/providers/{connectionId}` with body `{ "isActive": false }` —
|
||||
deactivates that connection; combo routing and account selection skip it
|
||||
on the next request (`src/app/api/providers/[id]/route.ts`). There is no
|
||||
single whole-provider kill switch — if the provider has more than one
|
||||
key/account, repeat per connection, or let the automatic provider circuit
|
||||
breaker trip on its own (`src/shared/utils/circuitBreaker.ts`,
|
||||
`domain_circuit_breakers` table; see `docs/architecture/RESILIENCE_GUIDE.md`).
|
||||
2. Verify p95 returns to budget within 5 min.
|
||||
3. If all connections for a model are down, apply the same `isActive: false`
|
||||
toggle to every connection offering that model — there is no separate
|
||||
per-model disable endpoint. Combo routing's automatic Model Lockout
|
||||
(`open-sse/services/accountFallback.ts`; see
|
||||
`docs/architecture/RESILIENCE_GUIDE.md`) also skips a model that keeps
|
||||
erroring, without manual action.
|
||||
4. Update the status page (if one is configured — see § 5) with a banner if
|
||||
the outage exceeds 15 min.
|
||||
|
||||
### 4.2 Cluster-wide latency regression
|
||||
|
||||
1. Check the most recent deploy (`/api/monitoring/health` returns `appVersion`).
|
||||
2. If p95 doubled vs the 7-day baseline, **roll back** to the prior
|
||||
SHA via `bin/rollback.sh`.
|
||||
3. If the regression is provider-side, see § 4.1.
|
||||
|
||||
### 4.3 Auth layer broken (5xx on /v1/responses for all keys)
|
||||
|
||||
1. Check the authz-inventory endpoint:
|
||||
`curl https://api.omniroute.dev/api/settings/authz-inventory | jq`.
|
||||
It returns a route-tier inventory (`tiers`, `bypassEnabled`,
|
||||
`bypassPrefixes`, `spawnCapablePrefixes`, `cors` — see
|
||||
`src/app/api/settings/authz-inventory/route.ts`); there is no
|
||||
`policies_active` field. A non-200 response, or a `tiers` array that
|
||||
fails to populate, means the settings/DB layer the auth pipeline reads
|
||||
from is down — not just a single bad key.
|
||||
2. If the endpoint itself errors or returns malformed data, restore the
|
||||
settings store from the last good backup (`bin/restore-policies.sh <sha>`).
|
||||
3. If the endpoint is healthy but requests still 5xx for every key, verify
|
||||
`JWT_SECRET` / `API_KEY_SECRET` are set and unchanged for this deploy,
|
||||
and that `isValidApiKey` (`src/sse/services/auth.ts`) can reach the DB.
|
||||
4. Roll back if the cause is unclear.
|
||||
|
||||
### 4.4 Data-layer incident (sqlite corruption, audit log gap)
|
||||
|
||||
1. **Stop the cluster** (`docker compose -f docker-compose.prod.yml
|
||||
stop`) — preventing further writes is more important than uptime.
|
||||
2. Snapshot the data volume (`bin/snapshot-data.sh`).
|
||||
3. Open a SEV-1; this is data-loss territory. Page the data-team.
|
||||
4. Restore from the last verified backup (see `docs/BACKUP.md` once
|
||||
published; currently the runbook is `bin/restore-data.sh <sha>`).
|
||||
|
||||
### 4.5 Security incident (vulnerability disclosure)
|
||||
|
||||
**Stop.** This is the `SECURITY.md` path, not this runbook. Page the
|
||||
security on-call (`@security-team`); do not post details to
|
||||
`#omniroute-ops`.
|
||||
|
||||
---
|
||||
|
||||
## 5. Communication
|
||||
|
||||
| Audience | Channel | Cadence | Owner |
|
||||
|---|---|---|---|
|
||||
| Engineering | `#inc-YYYY-MM-DD-slug` | Real-time | Incident commander |
|
||||
| Status page | TBD — not provisioned by this repo | Every 30 min during SEV-1/2 | On-call |
|
||||
| Customers (email) | TBD — set your announcement list/address | At SEV-1 start + resolution | Comms lead |
|
||||
| Upstream providers | Direct contact | At SEV-1 start | Vendor mgmt |
|
||||
| Postmortem | `docs/postmortem/YYYY-MM-DD-slug.md` | Within 5 business days | Incident commander |
|
||||
|
||||
Postmortem template is at `docs/postmortem/TEMPLATE.md` (forthcoming; no
|
||||
dedicated ADR covers it yet — once written, register it in
|
||||
`docs/architecture/cluster-decisions.md` following this repo's 71-pillar/ADR
|
||||
numbering convention, e.g. ADR-041 there).
|
||||
|
||||
---
|
||||
|
||||
## 6. On-call rotation
|
||||
|
||||
| Role | Primary | Secondary | Rotation |
|
||||
|---|---|---|---|
|
||||
| Engineering on-call | security-circle lead | @open-sse | Weekly, Mon 09:00 PDT |
|
||||
| Security on-call | @security-team | — | Weekly |
|
||||
| Data on-call | @db-team | — | Weekly |
|
||||
| Comms lead | @comms | — | As needed |
|
||||
|
||||
**Handoff**: every Monday 09:00 PDT, the outgoing on-call posts a
|
||||
written handoff to the incoming in `#omniroute-ops-handoff` covering:
|
||||
open SEV-3/4 items, scheduled maintenance windows, and any
|
||||
in-flight mitigations.
|
||||
|
||||
---
|
||||
|
||||
## 7. Postmortem expectations
|
||||
|
||||
- **Blameless**. People did the best they could with the information
|
||||
they had. Focus on systems, signals, and decision points.
|
||||
- **Within 5 business days** of resolution. File via
|
||||
`gh issue create --label postmortem --label SEV-1` (or `--label SEV-2`).
|
||||
- **Action items** must be assigned, dated, and tracked in
|
||||
`docs/TECH_DEBT.md` (P0 < 30 d, P1 < 90 d per that doc's SLA).
|
||||
- **Mandatory attendees**: incident commander, on-call, any engineer
|
||||
who touched the mitigation, and one person who was *not* involved
|
||||
(fresh-eyes review).
|
||||
|
||||
---
|
||||
|
||||
## 8. Review log
|
||||
|
||||
| Date | Reviewer | Change |
|
||||
|---|---|---|
|
||||
| 2026-06-18 | security-circle lead | Initial runbook; severity ladder + 15-min checklist + 4.1–4.5 mitigation runbooks. Closes 71-pillar audit L61 (1/3 → 2/3). |
|
||||
| 2026-07-18 | observability-circle | Corrected § 4.1/4.3 to the real provider-disable (`PUT /api/providers/{connectionId}`) and authz-inventory (`tiers`/`bypassEnabled`/`cors`, no `policies_active`) mechanisms; removed foreign branding and the nonexistent ADR-024/029 references. |
|
||||
| 2026-07-18 (planned) | observability-circle | Wire on-call rotation into PagerDuty schedule; add the postmortem template. |
|
||||
@@ -1,233 +0,0 @@
|
||||
---
|
||||
title: "Performance Budgets — OmniRoute (2026-06-18)"
|
||||
version: 3.8.50
|
||||
lastUpdated: 2026-08-06
|
||||
---
|
||||
|
||||
# Performance Budgets — OmniRoute (2026-06-18)
|
||||
|
||||
**Status**: Authoritative. SLO targets that the 71-pillar audit (L13)
|
||||
references for the `Perf > 2.00` gate.
|
||||
**Methodology**: per-endpoint p50/p95/p99 latency budgets, plus a
|
||||
top-level availability SLO. Budgets are derived from the 3-replica
|
||||
Caddy + Redis topology (commit `038439fa7`); adjust on infra change.
|
||||
**Enforcement**: none yet. § 6 sketches a `benches/perf-gate.k6.js` k6
|
||||
script that would assert the SLOs below, but it is a design reference,
|
||||
not a committed file — no `bench/` or `benches/` directory exists in
|
||||
this repo today. This doc is a target-setting reference only until a
|
||||
CI gate is built as follow-up work.
|
||||
**Re-evaluation cadence**: quarterly, or on any major infra change.
|
||||
|
||||
---
|
||||
|
||||
## 1. Top-level SLOs
|
||||
|
||||
| SLO | Target | Window | Page on breach |
|
||||
|---|---|---|---|
|
||||
| **Availability** (2xx or 4xx for /v1/* and /api/settings/*) | 99.9% | rolling 30 days | on-call P2 |
|
||||
| **Error budget burn rate** (1xx normalized rate) | < 2x for 1h, < 6x for 5m | 1h / 5m windows | on-call P1 |
|
||||
| **Aggregate p95 latency** (all /v1/*) | ≤ 1.5 s | rolling 5 min | on-call P2 |
|
||||
| **Aggregate p99 latency** (all /v1/*) | ≤ 4.0 s | rolling 5 min | on-call P2 |
|
||||
|
||||
**Error budget**: 30-day window = 43.2 minutes of unavailability at
|
||||
99.9%. Burn rate > 2x is P2; > 6x is P1.
|
||||
|
||||
---
|
||||
|
||||
## 2. Per-endpoint latency budgets
|
||||
|
||||
All budgets measured **server-side** (Next.js Route Handler entry to
|
||||
response start, or last byte for streaming). Stream endpoints are
|
||||
measured to time-of-first-byte (TTFB) since the body is incremental.
|
||||
|
||||
### 2.1 Inference endpoints (the hot path)
|
||||
|
||||
| Endpoint | Method | p50 | p95 | p99 | Notes |
|
||||
|---|---|---|---|---|---|
|
||||
| `/v1/responses` (non-stream) | POST | 800 ms | 1.8 s | 3.5 s | Includes translator + provider roundtrip |
|
||||
| `/v1/responses` (stream) | POST (TTFB) | 350 ms | 900 ms | 1.8 s | TTFB only; total duration unbounded |
|
||||
| `/v1/relay/chat/completions` (non-stream) | POST | 1.0 s | 2.2 s | 4.0 s | Includes per-(token,IP) rate-limit check |
|
||||
| `/v1/relay/chat/completions` (stream) | POST (TTFB) | 400 ms | 1.0 s | 2.0 s | |
|
||||
| `/v1/embeddings` | POST | 300 ms | 700 ms | 1.4 s | Pure provider roundtrip; cheap |
|
||||
| `/v1/rerank` | POST | 600 ms | 1.4 s | 2.8 s | |
|
||||
| `/v1/moderations` | POST | 250 ms | 600 ms | 1.2 s | Lightweight classification |
|
||||
| `/v1/audio/speech` | POST | 1.2 s | 3.0 s | 6.0 s | Audio synthesis is slow; budget reflects that |
|
||||
| `/v1/audio/transcriptions` | POST | 2.0 s | 5.0 s | 10.0 s | STT is bounded by audio duration + model size |
|
||||
| `/v1/images/generations` | POST | 4.0 s | 8.0 s | 15.0 s | Image gen is async-bound by provider |
|
||||
| `/v1/videos/generations` | POST (TTFB) | 600 ms | 1.5 s | 3.0 s | Async; client polls `/v1/videos/{id}` |
|
||||
| `/v1/music/generations` | POST | 3.0 s | 6.0 s | 12.0 s | |
|
||||
|
||||
### 2.2 Files + batches
|
||||
|
||||
| Endpoint | Method | p50 | p95 | p99 | Notes |
|
||||
|---|---|---|---|---|---|
|
||||
| `/v1/files` (GET) | GET | 80 ms | 200 ms | 400 ms | Cached list |
|
||||
| `/v1/files` (POST upload) | POST | 500 ms | 1.2 s | 2.5 s | 25 MB cap; multipart parse |
|
||||
| `/v1/files/{id}` (GET) | GET | 60 ms | 150 ms | 300 ms | |
|
||||
| `/v1/files/{id}` (DELETE) | DELETE | 80 ms | 200 ms | 400 ms | |
|
||||
| `/v1/files/{id}/content` (download) | GET | 100 ms | 300 ms | 600 ms | + per-MB throughput |
|
||||
| `/v1/batches` (GET) | GET | 150 ms | 400 ms | 800 ms | |
|
||||
| `/v1/batches` (POST create) | POST | 200 ms | 500 ms | 1.0 s | Validates input file then enqueues |
|
||||
| `/v1/batches/{id}` (GET) | GET | 100 ms | 300 ms | 600 ms | |
|
||||
| `/v1/batches/{id}` (DELETE) | DELETE | 100 ms | 300 ms | 600 ms | |
|
||||
| `/v1/batches/delete-completed` (POST) | POST | 400 ms | 1.0 s | 2.0 s | Mass delete; n rows |
|
||||
|
||||
### 2.3 Agents
|
||||
|
||||
| Endpoint | Method | p50 | p95 | p99 | Notes |
|
||||
|---|---|---|---|---|---|
|
||||
| `/v1/agents/health` | GET | 1.5 s | 4.5 s | 5.0 s | 5s per-provider timeout cap; expect 3-provider total |
|
||||
| `/v1/agents/credentials` | GET | 100 ms | 250 ms | 500 ms | Metadata only; values never returned |
|
||||
| `/v1/agents/tasks` (GET list) | GET | 150 ms | 400 ms | 800 ms | |
|
||||
| `/v1/agents/tasks` (POST create) | POST | 250 ms | 600 ms | 1.2 s | Just enqueues; doesn't run agent |
|
||||
| `/v1/agents/tasks/{id}` (GET) | GET | 100 ms | 300 ms | 600 ms | |
|
||||
| `/v1/agents/tasks/{id}` (DELETE) | DELETE | 150 ms | 400 ms | 800 ms | |
|
||||
|
||||
### 2.4 Combos / me / providers
|
||||
|
||||
| Endpoint | Method | p50 | p95 | p99 |
|
||||
|---|---|---|---|---|
|
||||
| `/v1/combos` | GET | 80 ms | 200 ms | 400 ms |
|
||||
| `/v1/me/status` | GET | 60 ms | 150 ms | 300 ms |
|
||||
| `/v1/providers/{provider}/models` | GET | 100 ms | 250 ms | 500 ms |
|
||||
|
||||
### 2.5 Web / search
|
||||
|
||||
| Endpoint | Method | p50 | p95 | p99 | Notes |
|
||||
|---|---|---|---|---|---|
|
||||
| `/v1/web/fetch` | POST | 1.5 s | 4.0 s | 8.0 s | 10s timeout cap; recurse depth 3 |
|
||||
| `/v1/search` | POST | 800 ms | 2.0 s | 4.0 s | Provider search latency varies |
|
||||
|
||||
### 2.6 VSCode-CLI shim (token-scoped)
|
||||
|
||||
These are the legacy passthrough paths. Budgets are tighter because
|
||||
they're called frequently by the VSCode-CLI extension in tight loops.
|
||||
|
||||
| Endpoint | Method | p50 | p95 | p99 |
|
||||
|---|---|---|---|---|
|
||||
| `/v1/vscode/{token}/v1/chat/completions` | POST | 700 ms | 1.6 s | 3.0 s |
|
||||
| `/v1/vscode/{token}/v1/models` | GET | 60 ms | 150 ms | 300 ms |
|
||||
| `/v1/vscode/{token}/combos` | GET | 80 ms | 200 ms | 400 ms |
|
||||
| `/v1/vscode/{token}/chat/completions` (legacy) | POST | 700 ms | 1.6 s | 3.0 s |
|
||||
| `/v1/vscode/{token}/models` (legacy) | GET | 60 ms | 150 ms | 300 ms |
|
||||
| `/v1/vscode/{token}/responses` | POST | 800 ms | 1.8 s | 3.5 s |
|
||||
|
||||
### 2.7 Management / settings
|
||||
|
||||
Management endpoints are operator-only and not part of the hot path.
|
||||
Budgets are set conservatively; breaches don't page on-call but do
|
||||
flag in the weekly perf review.
|
||||
|
||||
| Endpoint group | p50 | p95 | p99 |
|
||||
|---|---|---|---|
|
||||
| `/api/settings/*` (GET) | 100 ms | 300 ms | 600 ms |
|
||||
| `/api/settings/*` (POST/PATCH/DELETE) | 200 ms | 500 ms | 1.0 s |
|
||||
| `/api/keys/*` (CRUD) | 150 ms | 400 ms | 800 ms |
|
||||
| `/api/quota/*` (CRUD) | 150 ms | 400 ms | 800 ms |
|
||||
| `/api/monitoring/health` (heavy) | 500 ms | 1.5 s | 3.0 s |
|
||||
|
||||
### 2.8 Public probes
|
||||
|
||||
| Endpoint | Method | p50 | p95 | p99 |
|
||||
|---|---|---|---|---|
|
||||
| `/api/health/ping` | GET | 5 ms | 20 ms | 50 ms |
|
||||
| `/api/monitoring/health` | GET | 5 ms | 20 ms | 50 ms |
|
||||
| `/api/docs` | GET | 20 ms | 80 ms | 200 ms (HTML shell, no provider call) |
|
||||
|
||||
---
|
||||
|
||||
## 3. Throughput targets
|
||||
|
||||
| Tier | Per-replica RPS | Cluster RPS (3 replicas) | Notes |
|
||||
|---|---|---|---|
|
||||
| Inference (non-stream) | 50 RPS | 150 RPS | Bounded by provider quota + translator CPU |
|
||||
| Inference (stream) | 25 concurrent streams | 75 streams | Bounded by Node event-loop + memory |
|
||||
| Embeddings | 200 RPS | 600 RPS | Cheap |
|
||||
| Files (upload) | 10 RPS | 30 RPS | Multipart parse + DB write |
|
||||
| Files (download) | 100 RPS | 300 RPS | Static-content via Next.js |
|
||||
| Combos / me / providers | 500 RPS | 1,500 RPS | Cached |
|
||||
| WebSocket | 100 concurrent connections | 300 | Per-IP cap 5 |
|
||||
|
||||
**Cluster ceiling** (all endpoints combined, sustained): ~1,000 RPS
|
||||
before p95 latency begins to climb. Scale horizontally beyond that
|
||||
by adding replicas; the Caddy LB is stateless.
|
||||
|
||||
---
|
||||
|
||||
## 4. Resource budgets
|
||||
|
||||
| Resource | Per-replica cap | Notes |
|
||||
|---|---|---|
|
||||
| RSS memory | 1.5 GB | Spikes during audio/video gen; expect brief 2 GB |
|
||||
| Event-loop lag (p99) | 50 ms | Alert via `clinic doctor` regression |
|
||||
| Heap retained | 800 MB | Old-gen GC tuning in `node --max-old-space-size` |
|
||||
| File descriptors | 2,000 | `ulimit -n 4096` recommended at host |
|
||||
| DB connections (sql.js) | 1 per replica | sql.js is in-process; no pool needed |
|
||||
| Redis connections | 20 per replica | Pooled; idle reaped at 5 min |
|
||||
|
||||
---
|
||||
|
||||
## 5. Cold-start budget
|
||||
|
||||
Next.js App Router cold-start on a fresh container:
|
||||
|
||||
| Phase | Budget |
|
||||
|---|---|
|
||||
| Container start → HTTP listening | ≤ 800 ms |
|
||||
| First request TTFB (warm) | ≤ 200 ms |
|
||||
| Translator registry bootstrap | ≤ 500 ms (one-time, first /v1/responses) |
|
||||
|
||||
**Measurement script**: `bin/cold-start-bench.sh` (already in the repo
|
||||
since v3.8.36; `bin/` is the canonical scripts dir).
|
||||
|
||||
---
|
||||
|
||||
## 6. Regression gate (k6 reference, not yet implemented)
|
||||
|
||||
The sketch below shows how a future `benches/perf-gate.k6.js` script
|
||||
would assert the SLOs above. Nothing in this section is committed or
|
||||
wired into CI today — it is a design reference for follow-up work, not
|
||||
a running gate.
|
||||
|
||||
```javascript
|
||||
// benches/perf-gate.k6.js — pseudo-code; not yet committed
|
||||
import http from 'k6/http';
|
||||
import { check, Trend } from 'k6';
|
||||
|
||||
const responsesTTFB = new Trend('v1_responses_ttfb', true);
|
||||
|
||||
export const options = {
|
||||
scenarios: {
|
||||
smoke: {
|
||||
executor: 'constant-vus',
|
||||
vus: 10,
|
||||
duration: '1m',
|
||||
},
|
||||
},
|
||||
thresholds: {
|
||||
'http_req_duration{endpoint:v1_responses}': ['p(95)<1800', 'p(99)<3500'],
|
||||
'http_req_failed': ['rate<0.01'],
|
||||
'v1_responses_ttfb': ['p(95)<900'],
|
||||
},
|
||||
};
|
||||
|
||||
export default function () {
|
||||
const res = http.post(`${__ENV.BASE_URL}/api/v1/responses`, JSON.stringify({
|
||||
model: 'gpt-4o-mini',
|
||||
input: 'ping',
|
||||
}), { headers: { 'Authorization': `Bearer ${__ENV.API_KEY}` }});
|
||||
check(res, { 'status is 200': (r) => r.status === 200 });
|
||||
responsesTTFB.add(res.timings.waiting);
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 7. Review log
|
||||
|
||||
| Date | Reviewer | Change |
|
||||
|---|---|---|
|
||||
| 2026-06-18 | security-circle lead | Initial per-endpoint budgets derived from 3-replica Caddy + Redis topology |
|
||||
| 2026-07-18 | observability-circle | Clarified this doc ships zero enforcement today (no `bench/`/`benches/` dir, no CI gate) and fixed the stale "not yet committed" claim about `bin/cold-start-bench.sh` (present since v3.8.36). |
|
||||
| 2026-07-18 (planned) | observability-circle | Wire `benches/perf-gate.k6.js` into CI; gate on p95 + p99 breach |
|
||||
| 2026-09-18 (planned) | observability-circle | Quarterly review; adjust after real-traffic baseline data |
|
||||
@@ -1,273 +0,0 @@
|
||||
---
|
||||
title: "Combo Context Requirements Feature"
|
||||
version: 3.8.50
|
||||
lastUpdated: 2026-08-06
|
||||
---
|
||||
|
||||
# Combo Context Requirements Feature
|
||||
|
||||
## Overview
|
||||
|
||||
The Context Requirements feature allows combo configurations to filter and sort targets based on their context window size. This is useful for use cases requiring large context windows like:
|
||||
|
||||
- Long document processing (100k+ tokens)
|
||||
- Large codebase analysis
|
||||
- Extensive conversation histories
|
||||
- Multi-file code reviews
|
||||
|
||||
## Configuration
|
||||
|
||||
### Schema
|
||||
|
||||
Add `contextRequirements` to your combo's runtime config:
|
||||
|
||||
```json
|
||||
{
|
||||
"contextRequirements": {
|
||||
"minContextWindow": 128000,
|
||||
"preferLargeContext": true,
|
||||
"contextFilterMode": "strict"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Fields
|
||||
|
||||
#### `minContextWindow` (optional)
|
||||
|
||||
- **Type**: `number` (0 to 10,000,000)
|
||||
- **Default**: `undefined` (no filtering)
|
||||
- **Description**: Filters out models with context windows below this threshold
|
||||
|
||||
**Examples**:
|
||||
|
||||
- `32000` - Filter out models with <32K context
|
||||
- `128000` - Require 128K+ context (GPT-4 Turbo, Claude 3)
|
||||
- `200000` - Require 200K+ context (Claude 3 Opus)
|
||||
- `1000000` - Require 1M+ context (Gemini 1.5 Pro)
|
||||
|
||||
#### `preferLargeContext` (optional)
|
||||
|
||||
- **Type**: `boolean`
|
||||
- **Default**: `false`
|
||||
- **Description**: When `true`, sorts remaining targets by context size (descending). Large context models are tried first.
|
||||
|
||||
#### `contextFilterMode` (optional)
|
||||
|
||||
- **Type**: `"strict"` | `"lenient"`
|
||||
- **Default**: `"lenient"`
|
||||
- **Description**: How to handle models with unknown context window limits
|
||||
- `"strict"`: Excludes models with unknown context limits when a known-good target remains; fail-opens to unknowns if the pool would otherwise be empty (#8786)
|
||||
- `"lenient"`: Includes models with unknown context limits
|
||||
|
||||
## Behavior
|
||||
|
||||
### Filtering Pipeline
|
||||
|
||||
Context requirements are applied after `filterTargetsByRequestCompatibility()`:
|
||||
|
||||
1. **Request compatibility filtering** - Removes models incompatible with request (tools, vision, structured output)
|
||||
2. **Context requirements filtering** - Applies `minContextWindow` and `contextFilterMode`
|
||||
3. **Context-based sorting** - If `preferLargeContext` is true, sorts by context size descending
|
||||
|
||||
### Filter Mode Logic
|
||||
|
||||
When `minContextWindow` is set:
|
||||
|
||||
**Lenient mode** (default):
|
||||
|
||||
- ✅ Includes models with context >= minContextWindow
|
||||
- ✅ Includes models with unknown context limits
|
||||
- ❌ Excludes models with context < minContextWindow
|
||||
|
||||
**Strict mode**:
|
||||
|
||||
- ✅ Includes models with context >= minContextWindow
|
||||
- ❌ Excludes models with unknown context limits (when at least one known-good target remains)
|
||||
- ❌ Excludes models with context < minContextWindow
|
||||
- ⚠️ **Fail-open (#8786)**: if strict filtering would empty the pool and at least one
|
||||
unknown-context target exists, those unknowns are restored instead of returning
|
||||
`404 Combo has no executable targets`. Known-too-small targets are never resurrected.
|
||||
When the pool is still empty (every known target is below `minContextWindow`), the
|
||||
API returns `terminalReason: "context_requirements_exhausted"` with a recovery hint.
|
||||
|
||||
### Sorting Logic
|
||||
|
||||
When `preferLargeContext` is true:
|
||||
|
||||
- Models are sorted by context window size (descending)
|
||||
- Unknown context models sort to the end
|
||||
- Original strategy order is used as a tiebreaker
|
||||
|
||||
## Use Cases
|
||||
|
||||
### Example 1: Long Document Processing
|
||||
|
||||
```json
|
||||
{
|
||||
"name": "Document Analysis",
|
||||
"strategy": "fusion",
|
||||
"config": {
|
||||
"contextRequirements": {
|
||||
"minContextWindow": 128000,
|
||||
"preferLargeContext": true,
|
||||
"contextFilterMode": "strict"
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
This configuration:
|
||||
|
||||
- Requires 128K+ context window
|
||||
- Prefers larger context models (Gemini 1.5 Pro > Claude 3 Opus > GPT-4 Turbo)
|
||||
- Excludes models with unknown context limits
|
||||
|
||||
### Example 2: Large Codebase Analysis
|
||||
|
||||
```json
|
||||
{
|
||||
"name": "Code Review",
|
||||
"strategy": "auto",
|
||||
"config": {
|
||||
"contextRequirements": {
|
||||
"minContextWindow": 200000,
|
||||
"preferLargeContext": true,
|
||||
"contextFilterMode": "lenient"
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
This configuration:
|
||||
|
||||
- Requires 200K+ context window
|
||||
- Prefers larger context models
|
||||
- Includes models with unknown limits (lenient)
|
||||
|
||||
### Example 3: Prefer Large Context Without Strict Requirements
|
||||
|
||||
```json
|
||||
{
|
||||
"name": "Flexible Chat",
|
||||
"strategy": "weighted",
|
||||
"config": {
|
||||
"contextRequirements": {
|
||||
"preferLargeContext": true
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
This configuration:
|
||||
|
||||
- No minimum requirement (all models eligible)
|
||||
- Sorts by context size (largest first)
|
||||
- Useful when large context is preferred but not required
|
||||
|
||||
## API Response
|
||||
|
||||
When context requirements filter targets, the combo logger outputs:
|
||||
|
||||
```
|
||||
[COMBO] Context requirements: filtered 10 → 3 targets (minContextWindow: 128000, mode: strict)
|
||||
[COMBO] Context requirements: kept models gemini-1.5-pro, claude-3-opus-20240229, gpt-4-turbo
|
||||
[COMBO] Context requirements: sorted by context size (descending): gemini-1.5-pro(1000000), claude-3-opus-20240229(200000), gpt-4-turbo(128000)
|
||||
```
|
||||
|
||||
## Implementation Details
|
||||
|
||||
### Backend Module
|
||||
|
||||
`open-sse/services/combo/contextRequirements.ts`:
|
||||
|
||||
- `applyContextRequirements()` - Main filtering function
|
||||
- `getTargetContextWindow()` - Context lookup helper
|
||||
- Uses `getModelContextLimit()` from `modelCapabilities.ts`
|
||||
|
||||
### Integration Point
|
||||
|
||||
`open-sse/services/combo.ts` line 1187:
|
||||
|
||||
```typescript
|
||||
orderedTargets = filterTargetsByRequestCompatibility(orderedTargets, body, log);
|
||||
orderedTargets = applyContextRequirements(orderedTargets, config.contextRequirements, log);
|
||||
```
|
||||
|
||||
### Schema Definition
|
||||
|
||||
`src/shared/validation/schemas/combo.ts`:
|
||||
|
||||
```typescript
|
||||
contextRequirements: z
|
||||
.object({
|
||||
minContextWindow: z.coerce.number().int().min(0).max(10_000_000).optional(),
|
||||
preferLargeContext: z.boolean().optional(),
|
||||
contextFilterMode: z.enum(["strict", "lenient"]).optional(),
|
||||
})
|
||||
.strict()
|
||||
.optional(),
|
||||
```
|
||||
|
||||
## Testing
|
||||
|
||||
### Run Tests
|
||||
|
||||
```bash
|
||||
# Unit tests (schema + logic)
|
||||
npm test tests/unit/combo-context-requirements.test.ts
|
||||
|
||||
# Integration tests (end-to-end)
|
||||
npm test tests/unit/combo/context-requirements-integration.test.ts
|
||||
```
|
||||
|
||||
### Test Coverage
|
||||
|
||||
- Schema validation: 6 tests
|
||||
- Filtering logic: 6 tests
|
||||
- Integration: 5 tests
|
||||
- **Total**: 17/17 passing ✅
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### All targets filtered out
|
||||
|
||||
**Problem**: All targets removed, combo returns "no compatible models"
|
||||
|
||||
**Solutions**:
|
||||
|
||||
1. Lower `minContextWindow` threshold
|
||||
2. Switch to `"lenient"` mode to include unknown context models
|
||||
3. Remove `minContextWindow` and use only `preferLargeContext`
|
||||
|
||||
### Unknown context models excluded
|
||||
|
||||
**Problem**: Custom/new models excluded even though they have large context
|
||||
|
||||
**Solutions**:
|
||||
|
||||
1. Switch to `"lenient"` mode (default)
|
||||
2. Add model context limit to `modelCapabilities.ts`
|
||||
3. Remove context filtering and rely on strategy order
|
||||
|
||||
### Sorting not applied
|
||||
|
||||
**Problem**: `preferLargeContext` doesn't change order
|
||||
|
||||
**Check**:
|
||||
|
||||
1. Verify `preferLargeContext: true` in config
|
||||
2. Check if all targets have unknown context (all sort equal)
|
||||
3. Verify multiple targets remain after filtering
|
||||
|
||||
## Related
|
||||
|
||||
- [Auto-Combo Routing Strategies](./routing/AUTO-COMBO.md)
|
||||
- [Resilience Guide](./architecture/RESILIENCE_GUIDE.md)
|
||||
|
||||
## Version History
|
||||
|
||||
- **v3.8.47**: Initial implementation
|
||||
- Added `contextRequirements` config
|
||||
- Created backend filtering module
|
||||
- Full test coverage (no dedicated dashboard UI yet — configure via combo JSON)
|
||||
@@ -1,232 +0,0 @@
|
||||
# Runbook reagowania na incydenty — OmniRoute (2026-06-18)
|
||||
|
||||
**Status**: Dokument autorytatywny. Audyt 71 filarów (L61) odwołuje się do tego
|
||||
dokumentu w bramce `Obs > 2.00`.
|
||||
**Właściciel**: observability-circle (lead: security-circle lead).
|
||||
**SLO**: zob. `docs/PERF_BUDGETS.md` § 1 (SLO najwyższego poziomu) oraz
|
||||
`ops/slos.yaml` (forma maszynowo czytelna, generowana przez zespół Bifrost).
|
||||
**Polityka ujawniania**: zob. `SECURITY.md` (wyłącznie ujawnianie podatności,
|
||||
osobny przepływ).
|
||||
|
||||
Ten runbook to operacyjny playbook dla incydentów **niezwiązanych z bezpieczeństwem**:
|
||||
awarie, regresje opóźnień, spalanie budżetu błędów oraz awarie po stronie
|
||||
dostawców. Ujawnianie podatności pozostaje w `SECURITY.md`; nie kieruj
|
||||
tych spraw przez ten runbook.
|
||||
|
||||
---
|
||||
|
||||
## 1. Skala ważności
|
||||
|
||||
| Sev | Definicja | Przykłady | Powiadomienie | Rozwiązanie do |
|
||||
| --------- | ------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------- | ----------------------------------------- | ---------------------------- |
|
||||
| **SEV-1** | Awaria widoczna dla użytkownika; > 50 % żądań kończy się niepowodzeniem lub naruszenie SLO > 2x przez 5 min. | Klaster niedostępny; warstwa auth uszkodzona; powódź 5xx. | On-call P0 (natychmiast) | 4 h |
|
||||
| **SEV-2** | Znacząca degradacja; naruszenie SLO 1,5–2x przez 15 min lub wpływ na jednego najemcę. | Jeden dostawca niedostępny; p95 > 1,5x budżetu; niekontrolowane rate-limity. | On-call P1 (15 min) | 24 h |
|
||||
| **SEV-3** | Uśpiony błąd lub near-miss; brak bieżącego wpływu na użytkownika, ale budżet błędów zagrożony. | Wyciek pamięci w trendzie wzrostowym; circuit breaker wyłącza jednego dostawcę. | Slack `#omniroute-ops` (następny standup) | 7 d |
|
||||
| **SEV-4** | Kosmetyczny / informacyjny. | Szum w logach; nieblokujący glitch UI. | Następny przegląd tygodniowy | Następny cykl refaktoryzacji |
|
||||
|
||||
**Eskalacja burn-rate** (zgodnie z `docs/PERF_BUDGETS.md` § 1): 6x przez 5 min
|
||||
to SEV-1; 2x przez 1 h to SEV-2; utrzymanie < 1x przez 7 d obniża do SEV-3.
|
||||
|
||||
---
|
||||
|
||||
## 2. Źródła detekcji
|
||||
|
||||
| Źródło | Sygnał | Routing |
|
||||
| --------------------------------- | ------------------------------------ | ------------------------------------- |
|
||||
| Prometheus (`/metrics`) | Delty liczników (5xx, latency) | Alertmanager → PagerDuty |
|
||||
| Panele SLO w Grafana | Panele burn-rate SLO | Slack `#omniroute-ops` |
|
||||
| Sonda uptime (`/api/health/ping`) | 3 kolejne niepowodzenia z 3 regionów | Alertmanager → PagerDuty |
|
||||
| Dependabot | Nowe CVE w zależności (CVSS ≥ 7) | GitHub Security → security-circle |
|
||||
| Ręczny raport użytkownika | Zgłoszenie w Discord / GitHub issue | Triage przez dyżurnego (on-call) |
|
||||
| Chaos-drill (kwartalny) | Wstrzyknięte awarie | Planowany drill; wyniki w `docs/ops/` |
|
||||
|
||||
Alerty **nie** idą na prywatne DM. Domyślny kanał to `#omniroute-ops`; PagerDuty
|
||||
stronicuje rotację on-call. Pełna matryca alertów: `ops/alertmanager/rules.yml`
|
||||
(gdy jest wdrożona; do tego czasu reguły są w konfiguracji Prometheus w
|
||||
`deploy/observability/`).
|
||||
|
||||
---
|
||||
|
||||
## 3. Pierwsze 15 minut (SEV-1 / SEV-2)
|
||||
|
||||
1. **Potwierdź**. Otwórz panel SLO i sprawdź, czy alert jest prawdziwy, a nie
|
||||
flapping. Jeśli flapping — wycisz na 15 min i zbadaj.
|
||||
2. **Zadeklaruj**. Opublikuj w `#omniroute-ops`:
|
||||
```
|
||||
INCIDENT <sev> — <jedna linia objawu>
|
||||
IC: @you
|
||||
Status: investigating
|
||||
Następna aktualizacja: <teraz + 15 min>
|
||||
```
|
||||
3. **Stabilizuj** przed diagnozą główną przyczyny. Preferowana kolejność:
|
||||
- Odetnij zły deploy: `kubectl rollout undo deploy/omniroute` (lub
|
||||
równoważne dla Twojego środowiska; zob. `docs/ops/DEPLOYMENT.md`).
|
||||
- Przełącz combo / dostawcę: `POST /api/combos/:id/switch` lub MCP
|
||||
`switch_combo`.
|
||||
- Włącz tryb degradacji: ustaw
|
||||
`OMNIROUTE_DEGRADATION_MODE=lite` (pomija niekrytyczne middleware).
|
||||
- Rate-limit ruch wejściowy na edge, jeśli to flood.
|
||||
4. **Aktualizuj** co 15 min do złagodzenia lub rozwiązania.
|
||||
|
||||
Nie debuguj w produkcji przy SEV-1. Przywróć ostatni znany dobry stan, potem
|
||||
rób post-mortem offline.
|
||||
|
||||
---
|
||||
|
||||
## 4. Macierz runbooków
|
||||
|
||||
| Klasa awarii | Pierwszy ruch | Runbook |
|
||||
| ------------------------------------ | ------------------------------------------------------------- | ---------------------------------------------------------- |
|
||||
| Całkowity outage (wszystkie regiony) | Rollback ostatniego deployu; sprawdź status edge / DNS | `docs/ops/DEPLOYMENT.md` § rollback |
|
||||
| Pojedynczy dostawca 5xx / timeout | Wyłącz dostawcę w combo; włącz fallback | `docs/architecture/RESILIENCE_GUIDE.md` |
|
||||
| Spalanie budżetu błędów (latency) | Sprawdź p95 per-route; włącz compression / cache | `docs/PERF_BUDGETS.md` § 1–3 |
|
||||
| Wyczerpanie połączeń SQLite | Zrestartuj z większym pool; sprawdź długotrwałe transakcje | `docs/architecture/CODEBASE_DOCUMENTATION.md` (warstwa DB) |
|
||||
| Wyciek pamięci / OOM | Heap snapshot; rolling restart; oznacz SEV-3 na follow-up | wewnętrzny runbook profilowania |
|
||||
| Wygaśnięcie certu / TLS | Wdróż odnowiony cert; sprawdź automatyzację renew | `docs/ops/TLS.md` (gdy jest; w przeciwnym razie ręcznie) |
|
||||
| Awaria odświeżania tokena OAuth | Wymuś re-auth na dotkniętych kontach; sprawdź status dostawcy | `docs/security/OAUTH.md` |
|
||||
| Powódź rate-limit (wejście) | Zaciśnij limity na kluczu API; zbanuj obrażający klucz | `docs/architecture/AUTHZ_GUIDE.md` |
|
||||
| Awaria zależności (npm / CVE) | Pin / patch; w razie potrzeby wyłącz funkcję | `SECURITY.md` + Dependabot |
|
||||
|
||||
Każdy runbook musi kończyć się kryteriami **done** i właścicielem follow-upu.
|
||||
|
||||
---
|
||||
|
||||
## 5. Role w czasie incydentu
|
||||
|
||||
| Rola | Kto | Odpowiedzialności |
|
||||
| --------------------------- | ------------------------------------ | ---------------------------------------------------------------- |
|
||||
| **Incident Commander (IC)** | Dyżurny on-call (lub delegat) | Deklaruje sev, prowadzi mostek, zatwierdza mitigacje, zamyka |
|
||||
| **Tech lead** | Inżynier znający dotknięty podsystem | Diagnozuje, proponuje mitigacje, wdraża poprawki |
|
||||
| **Comms** | IC lub wolontariusz | Aktualizacje na Slacku, status page, odpowiedzi dla użytkowników |
|
||||
| **Scribe** | Wolontariusz | Notatki z linii czasu na potrzeby post-mortem |
|
||||
| **Executive sponsor** | Tylko SEV-1 | Escalation path; decyzje o zasobach |
|
||||
|
||||
Jedna osoba = jedna rola, gdy to możliwe. IC **nie** debuguje.
|
||||
|
||||
---
|
||||
|
||||
## 6. Komunikacja
|
||||
|
||||
- **Wewnętrzna**: `#omniroute-ops` jest źródłem prawdy. Wątek na incydent na
|
||||
deklarację; wszystkie aktualizacje w wątku.
|
||||
- **Zewnętrzna** (gdy dotyczy użytkowników zewnętrznych): status page
|
||||
(status.omniroute.example — zastąp prawdziwym URL, gdy będzie live). SEV-1
|
||||
dostaje publiczny post w ≤ 30 min; SEV-2 w ≤ 2 h, jeśli wpływ jest
|
||||
zewnętrzny.
|
||||
- **Nie** spekuluj o root cause publicznie. Podawaj objawy i ETA mitigacji.
|
||||
- Po złagodzeniu: jedna wiadomość „mitigated, monitoring for 30 min”, potem
|
||||
„resolved” z linkiem do post-mortem (gdy będzie gotowy).
|
||||
|
||||
---
|
||||
|
||||
## 7. Łagodzenie vs rozwiązanie
|
||||
|
||||
| Stan | Znaczenie | Kiedy używać |
|
||||
| --------------- | -------------------------------------------------------- | ------------------------------------------ |
|
||||
| `investigating` | Alert potwierdzony, przyczyna nieznana | Pierwsze 15 min |
|
||||
| `mitigating` | Stosowana poprawka; wpływ powinien spadać | Podczas rollbacku / failover |
|
||||
| `mitigated` | Wpływ na użytkownika ustał; root cause może być otwarty | Po udanej stabilizacji |
|
||||
| `resolved` | Root cause znany i trwale naprawiony (lub zaakceptowany) | Po merge poprawki lub decyzji o akceptacji |
|
||||
| `wontfix` | Zaakceptowane ryzyko; udokumentowane | Tylko SEV-3/4 za zgodą IC |
|
||||
|
||||
SEV-1/2 nie mogą pozostać w `mitigated` dłużej niż 7 dni bez eskalacji do
|
||||
executive sponsora.
|
||||
|
||||
---
|
||||
|
||||
## 8. Post-mortem (obowiązkowy dla SEV-1/2)
|
||||
|
||||
Szablon (skopiuj do `docs/postmortems/YYYY-MM-DD-<slug>.md`):
|
||||
|
||||
```markdown
|
||||
# Post-mortem: <tytuł>
|
||||
|
||||
- Data: YYYY-MM-DD
|
||||
- Sev: SEV-N
|
||||
- IC: @handle
|
||||
- Czas trwania: wykrycie → mitigacja → rozwiązanie
|
||||
- Dotknięci użytkownicy / budżet błędów spalony: <liczby>
|
||||
|
||||
## Streszczenie
|
||||
|
||||
<5 zdań, bez winy>
|
||||
|
||||
## Linia czasu
|
||||
|
||||
| Czas (UTC) | Event |
|
||||
| ---------- | ----- |
|
||||
| HH:MM | ... |
|
||||
|
||||
## Root cause
|
||||
|
||||
<co faktycznie się zepsuło; 5× dlaczego jeśli pomocne>
|
||||
|
||||
## Co poszło dobrze
|
||||
|
||||
- ...
|
||||
|
||||
## Co poszło źle
|
||||
|
||||
- ...
|
||||
|
||||
## Action items
|
||||
|
||||
| AI | Właściciel | Termin | Status |
|
||||
| --- | ---------- | ---------- | ------ |
|
||||
| ... | @handle | YYYY-MM-DD | open |
|
||||
|
||||
## Lekcje
|
||||
|
||||
<1–3 trwałe zmiany procesu lub kodu>
|
||||
```
|
||||
|
||||
Zasady:
|
||||
|
||||
- **Bez obwiniania.** System zawiódł, nie osoba.
|
||||
- Action items mają właściciela i termin; otwarte AI są przeglądane na
|
||||
cotygodniowym standupie ops.
|
||||
- Opublikuj w ciągu **5 dni roboczych** od rozwiązania.
|
||||
- SEV-3 dostaje post-mortem tylko gdy IC uzna to za wartościowe; SEV-4 nigdy.
|
||||
|
||||
---
|
||||
|
||||
## 9. Kwartalne chaos-drille
|
||||
|
||||
Harmonogram (własność: observability-circle):
|
||||
|
||||
| Kwartał | Scenariusz | Sukces = |
|
||||
| ------- | ------------------------------------------- | --------------------------------------------- |
|
||||
| Q1 | Kill pod główny podczas peak load | Failover < 30 s; zero utraty danych |
|
||||
| Q2 | Wstrzyknij 5xx u top-1 dostawcy | Combo przełącza się; budżet błędów trzyma się |
|
||||
| Q3 | Partycja sieci do SQLite (gdy sklastrowany) | Degradacja read-only; brak korupcji |
|
||||
| Q4 | Wygaśnięcie certu TLS (staging) | Alert odpala; renew w SLO |
|
||||
|
||||
Wyniki lądują w `docs/ops/chaos/YYYY-QN.md`. Niezaliczony drill otwiera SEV-3
|
||||
z AI na lukę.
|
||||
|
||||
---
|
||||
|
||||
## 10. Powiązane dokumenty
|
||||
|
||||
| Dokument | Rola |
|
||||
| --------------------------------------- | ------------------------------------- |
|
||||
| `docs/PERF_BUDGETS.md` | SLO, budżety błędów, progi burn-rate |
|
||||
| `ops/slos.yaml` | Maszynowa forma SLO (Bifrost) |
|
||||
| `SECURITY.md` | Ujawnianie podatności (osobny flow) |
|
||||
| `docs/architecture/RESILIENCE_GUIDE.md` | Fallback, circuit breaker, degradacja |
|
||||
| `docs/ops/DEPLOYMENT.md` | Deploy / rollback |
|
||||
| `docs/architecture/AUTHZ_GUIDE.md` | Nadużycia kluczy API, rate-limity |
|
||||
| `docs/postmortems/` | Archiwum wpisów post-mortem |
|
||||
|
||||
---
|
||||
|
||||
## 11. Historia zmian
|
||||
|
||||
| Data | Zmiana |
|
||||
| ---------- | ---------------------------------------------- |
|
||||
| 2026-06-18 | Wstępna wersja autorytatywna (L61 / gate Obs). |
|
||||
|
||||
---
|
||||
|
||||
_Ten dokument jest autorytatywny dla operacyjnego reagowania na incydenty.
|
||||
Poprawki: PR do `docs/INCIDENT_RESPONSE.md` z recenzją observability-circle._
|
||||
@@ -1,231 +0,0 @@
|
||||
# Budżety wydajności — OmniRoute (2026-06-18)
|
||||
|
||||
**Status**: Miarodajny. Cele SLO, do których odwołuje się audyt 71 filarów (L13)
|
||||
przy bramce `Perf > 2.00`.
|
||||
**Metodyka**: budżety opóźnień p50/p95/p99 per endpoint oraz
|
||||
nadrzędne SLO dostępności. Budżety wyprowadzono z topologii 3 replik
|
||||
Caddy + Redis (commit `038439fa7`); korygować przy zmianie infrastruktury.
|
||||
**Egzekwowanie**: na razie brak. § 6 szkicuje skrypt k6 `benches/perf-gate.k6.js`,
|
||||
który asertowałby poniższe SLO, lecz jest to odniesienie projektowe,
|
||||
a nie zacommitowany plik — w repozytorium nie ma dziś katalogu `bench/` ani `benches/`.
|
||||
Ten dokument służy wyłącznie do ustalania celów, dopóki bramka CI
|
||||
nie powstanie jako praca następcza.
|
||||
**Częstotliwość ponownej oceny**: kwartalnie lub przy każdej istotnej zmianie infrastruktury.
|
||||
|
||||
---
|
||||
|
||||
## 1. Nadrzędne SLO
|
||||
|
||||
| SLO | Cel | Okno | Page przy naruszeniu |
|
||||
| ------------------------------------------------------------- | ------------------------ | --------------- | -------------------- |
|
||||
| **Dostępność** (2xx lub 4xx dla /v1/* i /api/settings/*) | 99.9% | rolling 30 days | on-call P2 |
|
||||
| **Tempo spalania error budget** (znormalizowany wskaźnik 1xx) | < 2x for 1h, < 6x for 5m | 1h / 5m windows | on-call P1 |
|
||||
| **Zagregowane opóźnienie p95** (wszystkie /v1/*) | ≤ 1.5 s | rolling 5 min | on-call P2 |
|
||||
| **Zagregowane opóźnienie p99** (wszystkie /v1/*) | ≤ 4.0 s | rolling 5 min | on-call P2 |
|
||||
|
||||
**Error budget**: okno 30-dniowe = 43,2 minuty niedostępności przy
|
||||
99.9%. Tempo spalania > 2x to P2; > 6x to P1.
|
||||
|
||||
---
|
||||
|
||||
## 2. Budżety opóźnień per endpoint
|
||||
|
||||
Wszystkie budżety mierzone **po stronie serwera** (od wejścia do Next.js Route Handler
|
||||
do startu odpowiedzi albo do ostatniego bajtu przy streamingu). Endpointy streamowe
|
||||
mierzone do time-of-first-byte (TTFB), ponieważ body jest przyrostowe.
|
||||
|
||||
### 2.1 Endpointy inferencji (ścieżka krytyczna)
|
||||
|
||||
| Endpoint | Method | p50 | p95 | p99 | Notes |
|
||||
| ----------------------------------------- | ----------- | ------ | ------ | ------ | ------------------------------------------------------------- |
|
||||
| `/v1/responses` (non-stream) | POST | 800 ms | 1.8 s | 3.5 s | Obejmuje translator + roundtrip do providera |
|
||||
| `/v1/responses` (stream) | POST (TTFB) | 350 ms | 900 ms | 1.8 s | Tylko TTFB; całkowity czas nieograniczony |
|
||||
| `/v1/relay/chat/completions` (non-stream) | POST | 1.0 s | 2.2 s | 4.0 s | Obejmuje sprawdzenie rate-limit per-(token,IP) |
|
||||
| `/v1/relay/chat/completions` (stream) | POST (TTFB) | 400 ms | 1.0 s | 2.0 s | |
|
||||
| `/v1/embeddings` | POST | 300 ms | 700 ms | 1.4 s | Czysty roundtrip do providera; tani |
|
||||
| `/v1/rerank` | POST | 600 ms | 1.4 s | 2.8 s | |
|
||||
| `/v1/moderations` | POST | 250 ms | 600 ms | 1.2 s | Lekka klasyfikacja |
|
||||
| `/v1/audio/speech` | POST | 1.2 s | 3.0 s | 6.0 s | Synteza audio jest wolna; budżet to odzwierciedla |
|
||||
| `/v1/audio/transcriptions` | POST | 2.0 s | 5.0 s | 10.0 s | STT ograniczony czasem audio + rozmiarem modelu |
|
||||
| `/v1/images/generations` | POST | 4.0 s | 8.0 s | 15.0 s | Generacja obrazów ograniczona asynchronicznie przez providera |
|
||||
| `/v1/videos/generations` | POST (TTFB) | 600 ms | 1.5 s | 3.0 s | Async; klient odpytuje `/v1/videos/{id}` |
|
||||
| `/v1/music/generations` | POST | 3.0 s | 6.0 s | 12.0 s | |
|
||||
|
||||
### 2.2 Pliki + batche
|
||||
|
||||
| Endpoint | Method | p50 | p95 | p99 | Notes |
|
||||
| ------------------------------------- | ------ | ------ | ------ | ------ | ---------------------------------------- |
|
||||
| `/v1/files` (GET) | GET | 80 ms | 200 ms | 400 ms | Lista z cache |
|
||||
| `/v1/files` (POST upload) | POST | 500 ms | 1.2 s | 2.5 s | Limit 25 MB; parsowanie multipart |
|
||||
| `/v1/files/{id}` (GET) | GET | 60 ms | 150 ms | 300 ms | |
|
||||
| `/v1/files/{id}` (DELETE) | DELETE | 80 ms | 200 ms | 400 ms | |
|
||||
| `/v1/files/{id}/content` (download) | GET | 100 ms | 300 ms | 600 ms | + przepustowość per-MB |
|
||||
| `/v1/batches` (GET) | GET | 150 ms | 400 ms | 800 ms | |
|
||||
| `/v1/batches` (POST create) | POST | 200 ms | 500 ms | 1.0 s | Waliduje plik wejściowy, potem enqueuuje |
|
||||
| `/v1/batches/{id}` (GET) | GET | 100 ms | 300 ms | 600 ms | |
|
||||
| `/v1/batches/{id}` (DELETE) | DELETE | 100 ms | 300 ms | 600 ms | |
|
||||
| `/v1/batches/delete-completed` (POST) | POST | 400 ms | 1.0 s | 2.0 s | Masowe usuwanie; n wierszy |
|
||||
|
||||
### 2.3 Agenci
|
||||
|
||||
| Endpoint | Method | p50 | p95 | p99 | Notes |
|
||||
| -------------------------------- | ------ | ------ | ------ | ------ | ------------------------------------------------------------- |
|
||||
| `/v1/agents/health` | GET | 1.5 s | 4.5 s | 5.0 s | Limit timeout 5s per provider; oczekiwane łącznie 3 providery |
|
||||
| `/v1/agents/credentials` | GET | 100 ms | 250 ms | 500 ms | Tylko metadane; wartości nigdy nie są zwracane |
|
||||
| `/v1/agents/tasks` (GET list) | GET | 150 ms | 400 ms | 800 ms | |
|
||||
| `/v1/agents/tasks` (POST create) | POST | 250 ms | 600 ms | 1.2 s | Tylko enqueuuje; nie uruchamia agenta |
|
||||
| `/v1/agents/tasks/{id}` (GET) | GET | 100 ms | 300 ms | 600 ms | |
|
||||
| `/v1/agents/tasks/{id}` (DELETE) | DELETE | 150 ms | 400 ms | 800 ms | |
|
||||
|
||||
### 2.4 Combos / me / providers
|
||||
|
||||
| Endpoint | Method | p50 | p95 | p99 |
|
||||
| --------------------------------- | ------ | ------ | ------ | ------ |
|
||||
| `/v1/combos` | GET | 80 ms | 200 ms | 400 ms |
|
||||
| `/v1/me/status` | GET | 60 ms | 150 ms | 300 ms |
|
||||
| `/v1/providers/{provider}/models` | GET | 100 ms | 250 ms | 500 ms |
|
||||
|
||||
### 2.5 Web / search
|
||||
|
||||
| Endpoint | Method | p50 | p95 | p99 | Notes |
|
||||
| --------------- | ------ | ------ | ----- | ----- | ---------------------------------------------- |
|
||||
| `/v1/web/fetch` | POST | 1.5 s | 4.0 s | 8.0 s | Limit timeout 10s; głębokość rekurencji 3 |
|
||||
| `/v1/search` | POST | 800 ms | 2.0 s | 4.0 s | Opóźnienie wyszukiwania u providera bywa różne |
|
||||
|
||||
### 2.6 Shim VSCode-CLI (scoped tokenem)
|
||||
|
||||
To ścieżki legacy passthrough. Budżety są ciaśniejsze, bo
|
||||
rozszerzenie VSCode-CLI wywołuje je często w ciasnych pętlach.
|
||||
|
||||
| Endpoint | Method | p50 | p95 | p99 |
|
||||
| ---------------------------------------------- | ------ | ------ | ------ | ------ |
|
||||
| `/v1/vscode/{token}/v1/chat/completions` | POST | 700 ms | 1.6 s | 3.0 s |
|
||||
| `/v1/vscode/{token}/v1/models` | GET | 60 ms | 150 ms | 300 ms |
|
||||
| `/v1/vscode/{token}/combos` | GET | 80 ms | 200 ms | 400 ms |
|
||||
| `/v1/vscode/{token}/chat/completions` (legacy) | POST | 700 ms | 1.6 s | 3.0 s |
|
||||
| `/v1/vscode/{token}/models` (legacy) | GET | 60 ms | 150 ms | 300 ms |
|
||||
| `/v1/vscode/{token}/responses` | POST | 800 ms | 1.8 s | 3.5 s |
|
||||
|
||||
### 2.7 Zarządzanie / settings
|
||||
|
||||
Endpointy zarządzania są wyłącznie operatorskie i nie należą do ścieżki krytycznej.
|
||||
Budżety ustawiono konserwatywnie; naruszenia nie page'ują on-call, ale
|
||||
są flagowane w tygodniowym przeglądzie wydajności.
|
||||
|
||||
| Endpoint group | p50 | p95 | p99 |
|
||||
| ------------------------------------- | ------ | ------ | ------ |
|
||||
| `/api/settings/*` (GET) | 100 ms | 300 ms | 600 ms |
|
||||
| `/api/settings/*` (POST/PATCH/DELETE) | 200 ms | 500 ms | 1.0 s |
|
||||
| `/api/keys/*` (CRUD) | 150 ms | 400 ms | 800 ms |
|
||||
| `/api/quota/*` (CRUD) | 150 ms | 400 ms | 800 ms |
|
||||
| `/api/monitoring/health` (heavy) | 500 ms | 1.5 s | 3.0 s |
|
||||
|
||||
### 2.8 Publiczne sondy
|
||||
|
||||
| Endpoint | Method | p50 | p95 | p99 |
|
||||
| ------------------------ | ------ | ----- | ----- | ------------------------------------- |
|
||||
| `/api/health/ping` | GET | 5 ms | 20 ms | 50 ms |
|
||||
| `/api/monitoring/health` | GET | 5 ms | 20 ms | 50 ms |
|
||||
| `/api/docs` | GET | 20 ms | 80 ms | 200 ms (HTML shell, no provider call) |
|
||||
|
||||
---
|
||||
|
||||
## 3. Cele przepustowości
|
||||
|
||||
| Tier | Per-replica RPS | Cluster RPS (3 replicas) | Notes |
|
||||
| ----------------------- | -------------------------- | ------------------------ | --------------------------------------------------- |
|
||||
| Inference (non-stream) | 50 RPS | 150 RPS | Ograniczone przez quota providera + CPU translatora |
|
||||
| Inference (stream) | 25 concurrent streams | 75 streams | Ograniczone przez event-loop Node + pamięć |
|
||||
| Embeddings | 200 RPS | 600 RPS | Tanie |
|
||||
| Files (upload) | 10 RPS | 30 RPS | Parsowanie multipart + zapis do DB |
|
||||
| Files (download) | 100 RPS | 300 RPS | Treść statyczna przez Next.js |
|
||||
| Combos / me / providers | 500 RPS | 1,500 RPS | Z cache |
|
||||
| WebSocket | 100 concurrent connections | 300 | Limit per-IP: 5 |
|
||||
|
||||
**Sufit klastra** (wszystkie endpointy łącznie, obciążenie ciągłe): ~1 000 RPS,
|
||||
zanim p95 latency zacznie rosnąć. Powyżej tego skalować horyzontalnie
|
||||
przez dodawanie replik; Caddy LB jest bezstanowy.
|
||||
|
||||
---
|
||||
|
||||
## 4. Budżety zasobów
|
||||
|
||||
| Resource | Per-replica cap | Notes |
|
||||
| ----------------------- | --------------- | ------------------------------------------------------ |
|
||||
| RSS memory | 1.5 GB | Skoki przy gen. audio/wideo; spodziewane chwilowe 2 GB |
|
||||
| Event-loop lag (p99) | 50 ms | Alert przez regresję `clinic doctor` |
|
||||
| Heap retained | 800 MB | Strojenie old-gen GC w `node --max-old-space-size` |
|
||||
| File descriptors | 2,000 | Na hoście zalecane `ulimit -n 4096` |
|
||||
| DB connections (sql.js) | 1 per replica | sql.js działa in-process; pool nie jest potrzebny |
|
||||
| Redis connections | 20 per replica | Z poola; idle usuwane po 5 min |
|
||||
|
||||
---
|
||||
|
||||
## 5. Budżet cold-start
|
||||
|
||||
Cold-start Next.js App Router na świeżym kontenerze:
|
||||
|
||||
| Phase | Budget |
|
||||
| -------------------------------- | ---------------------------------------- |
|
||||
| Container start → HTTP listening | ≤ 800 ms |
|
||||
| First request TTFB (warm) | ≤ 200 ms |
|
||||
| Translator registry bootstrap | ≤ 500 ms (one-time, first /v1/responses) |
|
||||
|
||||
**Skrypt pomiarowy**: `bin/cold-start-bench.sh` (już w repozytorium
|
||||
od v3.8.36; `bin/` to kanoniczny katalog skryptów).
|
||||
|
||||
---
|
||||
|
||||
## 6. Bramka regresji (odniesienie k6, jeszcze niezaimplementowane)
|
||||
|
||||
Poniższy szkic pokazuje, jak przyszły skrypt `benches/perf-gate.k6.js`
|
||||
asertowałby powyższe SLO. Nic z tej sekcji nie jest dziś zacommitowane ani
|
||||
podpięte do CI — to odniesienie projektowe do pracy następczej, a nie
|
||||
działająca bramka.
|
||||
|
||||
```javascript
|
||||
// benches/perf-gate.k6.js — pseudo-code; not yet committed
|
||||
import http from "k6/http";
|
||||
import { check, Trend } from "k6";
|
||||
|
||||
const responsesTTFB = new Trend("v1_responses_ttfb", true);
|
||||
|
||||
export const options = {
|
||||
scenarios: {
|
||||
smoke: {
|
||||
executor: "constant-vus",
|
||||
vus: 10,
|
||||
duration: "1m",
|
||||
},
|
||||
},
|
||||
thresholds: {
|
||||
"http_req_duration{endpoint:v1_responses}": ["p(95)<1800", "p(99)<3500"],
|
||||
http_req_failed: ["rate<0.01"],
|
||||
v1_responses_ttfb: ["p(95)<900"],
|
||||
},
|
||||
};
|
||||
|
||||
export default function () {
|
||||
const res = http.post(
|
||||
`${__ENV.BASE_URL}/api/v1/responses`,
|
||||
JSON.stringify({
|
||||
model: "gpt-4o-mini",
|
||||
input: "ping",
|
||||
}),
|
||||
{ headers: { Authorization: `Bearer ${__ENV.API_KEY}` } }
|
||||
);
|
||||
check(res, { "status is 200": (r) => r.status === 200 });
|
||||
responsesTTFB.add(res.timings.waiting);
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 7. Dziennik przeglądów
|
||||
|
||||
| Date | Reviewer | Change |
|
||||
| -------------------- | -------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
|
||||
| 2026-06-18 | security-circle lead | Wstępne budżety per endpoint wyprowadzone z topologii 3 replik Caddy + Redis |
|
||||
| 2026-07-18 | observability-circle | Doprecyzowano, że dokument dziś nie egzekwuje nic (brak katalogu `bench/`/`benches/`, brak bramki CI) oraz poprawiono nieaktualne twierdzenie „not yet committed” o `bin/cold-start-bench.sh` (obecny od v3.8.36). |
|
||||
| 2026-07-18 (planned) | observability-circle | Podpięcie `benches/perf-gate.k6.js` do CI; bramka przy naruszeniu p95 + p99 |
|
||||
| 2026-09-18 (planned) | observability-circle | Przegląd kwartalny; korekta po danych bazowych z ruchu produkcyjnego |
|
||||
@@ -1,262 +0,0 @@
|
||||
# Funkcja wymagań kontekstu combo (Context Requirements)
|
||||
|
||||
## Przegląd
|
||||
|
||||
Funkcja Context Requirements pozwala konfiguracjom combo filtrować i sortować cele (targets) na podstawie rozmiaru okna kontekstu. Jest to przydatne w przypadkach użycia wymagających dużych okien kontekstu, takich jak:
|
||||
|
||||
- Przetwarzanie długich dokumentów (100k+ tokenów)
|
||||
- Analiza dużych baz kodu
|
||||
- Rozbudowane historie rozmów
|
||||
- Przeglądy kodu obejmujące wiele plików
|
||||
|
||||
## Konfiguracja
|
||||
|
||||
### Schemat
|
||||
|
||||
Dodaj `contextRequirements` do runtime config swojego combo:
|
||||
|
||||
```json
|
||||
{
|
||||
"contextRequirements": {
|
||||
"minContextWindow": 128000,
|
||||
"preferLargeContext": true,
|
||||
"contextFilterMode": "strict"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Pola
|
||||
|
||||
#### `minContextWindow` (opcjonalne)
|
||||
|
||||
- **Typ**: `number` (0 do 10,000,000)
|
||||
- **Domyślnie**: `undefined` (bez filtrowania)
|
||||
- **Opis**: Odfiltrowuje modele z oknami kontekstu poniżej tego progu
|
||||
|
||||
**Przykłady**:
|
||||
|
||||
- `32000` - Odfiltruj modele z kontekstem <32K
|
||||
- `128000` - Wymagaj kontekstu 128K+ (GPT-4 Turbo, Claude 3)
|
||||
- `200000` - Wymagaj kontekstu 200K+ (Claude 3 Opus)
|
||||
- `1000000` - Wymagaj kontekstu 1M+ (Gemini 1.5 Pro)
|
||||
|
||||
#### `preferLargeContext` (opcjonalne)
|
||||
|
||||
- **Typ**: `boolean`
|
||||
- **Domyślnie**: `false`
|
||||
- **Opis**: Gdy `true`, sortuje pozostałe cele według rozmiaru kontekstu (malejąco). Modele z dużym kontekstem są próbowane jako pierwsze.
|
||||
|
||||
#### `contextFilterMode` (opcjonalne)
|
||||
|
||||
- **Typ**: `"strict"` | `"lenient"`
|
||||
- **Domyślnie**: `"lenient"`
|
||||
- **Opis**: Sposób obsługi modeli z nieznanymi limitami okna kontekstu
|
||||
- `"strict"`: Wyklucza modele z nieznanymi limitami kontekstu
|
||||
- `"lenient"`: Uwzględnia modele z nieznanymi limitami kontekstu
|
||||
|
||||
## Zachowanie
|
||||
|
||||
### Potok filtrowania
|
||||
|
||||
Wymagania kontekstu są stosowane po `filterTargetsByRequestCompatibility()`:
|
||||
|
||||
1. **Filtrowanie zgodności żądania** - Usuwa modele niekompatybilne z żądaniem (tools, vision, structured output)
|
||||
2. **Filtrowanie wymagań kontekstu** - Stosuje `minContextWindow` i `contextFilterMode`
|
||||
3. **Sortowanie według kontekstu** - Jeśli `preferLargeContext` jest true, sortuje malejąco według rozmiaru kontekstu
|
||||
|
||||
### Logika trybu filtrowania
|
||||
|
||||
Gdy ustawiono `minContextWindow`:
|
||||
|
||||
**Tryb lenient** (domyślny):
|
||||
|
||||
- ✅ Uwzględnia modele z kontekstem >= minContextWindow
|
||||
- ✅ Uwzględnia modele z nieznanymi limitami kontekstu
|
||||
- ❌ Wyklucza modele z kontekstem < minContextWindow
|
||||
|
||||
**Tryb strict**:
|
||||
|
||||
- ✅ Uwzględnia modele z kontekstem >= minContextWindow
|
||||
- ❌ Wyklucza modele z nieznanymi limitami kontekstu
|
||||
- ❌ Wyklucza modele z kontekstem < minContextWindow
|
||||
|
||||
### Logika sortowania
|
||||
|
||||
Gdy `preferLargeContext` jest true:
|
||||
|
||||
- Modele są sortowane według rozmiaru okna kontekstu (malejąco)
|
||||
- Modele z nieznanym kontekstem trafiają na koniec
|
||||
- Oryginalna kolejność strategii służy jako rozstrzygnięcie remisów
|
||||
|
||||
## Przypadki użycia
|
||||
|
||||
### Przykład 1: Przetwarzanie długich dokumentów
|
||||
|
||||
```json
|
||||
{
|
||||
"name": "Document Analysis",
|
||||
"strategy": "fusion",
|
||||
"config": {
|
||||
"contextRequirements": {
|
||||
"minContextWindow": 128000,
|
||||
"preferLargeContext": true,
|
||||
"contextFilterMode": "strict"
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Ta konfiguracja:
|
||||
|
||||
- Wymaga okna kontekstu 128K+
|
||||
- Preferuje modele z większym kontekstem (Gemini 1.5 Pro > Claude 3 Opus > GPT-4 Turbo)
|
||||
- Wyklucza modele z nieznanymi limitami kontekstu
|
||||
|
||||
### Przykład 2: Analiza dużej bazy kodu
|
||||
|
||||
```json
|
||||
{
|
||||
"name": "Code Review",
|
||||
"strategy": "auto",
|
||||
"config": {
|
||||
"contextRequirements": {
|
||||
"minContextWindow": 200000,
|
||||
"preferLargeContext": true,
|
||||
"contextFilterMode": "lenient"
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Ta konfiguracja:
|
||||
|
||||
- Wymaga okna kontekstu 200K+
|
||||
- Preferuje modele z większym kontekstem
|
||||
- Uwzględnia modele z nieznanymi limitami (lenient)
|
||||
|
||||
### Przykład 3: Preferencja dużego kontekstu bez ścisłych wymagań
|
||||
|
||||
```json
|
||||
{
|
||||
"name": "Flexible Chat",
|
||||
"strategy": "weighted",
|
||||
"config": {
|
||||
"contextRequirements": {
|
||||
"preferLargeContext": true
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Ta konfiguracja:
|
||||
|
||||
- Brak minimalnego wymagania (wszystkie modele kwalifikują się)
|
||||
- Sortuje według rozmiaru kontekstu (największy najpierw)
|
||||
- Przydatne, gdy duży kontekst jest preferowany, ale nie wymagany
|
||||
|
||||
## Odpowiedź API
|
||||
|
||||
Gdy wymagania kontekstu filtrują cele, logger combo wypisuje:
|
||||
|
||||
```
|
||||
[COMBO] Context requirements: filtered 10 → 3 targets (minContextWindow: 128000, mode: strict)
|
||||
[COMBO] Context requirements: kept models gemini-1.5-pro, claude-3-opus-20240229, gpt-4-turbo
|
||||
[COMBO] Context requirements: sorted by context size (descending): gemini-1.5-pro(1000000), claude-3-opus-20240229(200000), gpt-4-turbo(128000)
|
||||
```
|
||||
|
||||
## Szczegóły implementacji
|
||||
|
||||
### Moduł backendu
|
||||
|
||||
`open-sse/services/combo/contextRequirements.ts`:
|
||||
|
||||
- `applyContextRequirements()` - Główna funkcja filtrowania
|
||||
- `getTargetContextWindow()` - Pomocnicza funkcja wyszukiwania kontekstu
|
||||
- Używa `getModelContextLimit()` z `modelCapabilities.ts`
|
||||
|
||||
### Punkt integracji
|
||||
|
||||
`open-sse/services/combo.ts` linia 1187:
|
||||
|
||||
```typescript
|
||||
orderedTargets = filterTargetsByRequestCompatibility(orderedTargets, body, log);
|
||||
orderedTargets = applyContextRequirements(orderedTargets, config.contextRequirements, log);
|
||||
```
|
||||
|
||||
### Definicja schematu
|
||||
|
||||
`src/shared/validation/schemas/combo.ts`:
|
||||
|
||||
```typescript
|
||||
contextRequirements: z
|
||||
.object({
|
||||
minContextWindow: z.coerce.number().int().min(0).max(10_000_000).optional(),
|
||||
preferLargeContext: z.boolean().optional(),
|
||||
contextFilterMode: z.enum(["strict", "lenient"]).optional(),
|
||||
})
|
||||
.strict()
|
||||
.optional(),
|
||||
```
|
||||
|
||||
## Testowanie
|
||||
|
||||
### Uruchamianie testów
|
||||
|
||||
```bash
|
||||
# Unit tests (schema + logic)
|
||||
npm test tests/unit/combo-context-requirements.test.ts
|
||||
|
||||
# Integration tests (end-to-end)
|
||||
npm test tests/unit/combo/context-requirements-integration.test.ts
|
||||
```
|
||||
|
||||
### Pokrycie testami
|
||||
|
||||
- Walidacja schematu: 6 testów
|
||||
- Logika filtrowania: 6 testów
|
||||
- Integracja: 5 testów
|
||||
- **Razem**: 17/17 przechodzi ✅
|
||||
|
||||
## Rozwiązywanie problemów
|
||||
|
||||
### Wszystkie cele odfiltrowane
|
||||
|
||||
**Problem**: Wszystkie cele usunięte, combo zwraca „no compatible models”
|
||||
|
||||
**Rozwiązania**:
|
||||
|
||||
1. Obniż próg `minContextWindow`
|
||||
2. Przełącz na tryb `"lenient"`, aby uwzględnić modele z nieznanym kontekstem
|
||||
3. Usuń `minContextWindow` i używaj wyłącznie `preferLargeContext`
|
||||
|
||||
### Modele z nieznanym kontekstem wykluczone
|
||||
|
||||
**Problem**: Niestandardowe/nowe modele wykluczone, mimo że mają duży kontekst
|
||||
|
||||
**Rozwiązania**:
|
||||
|
||||
1. Przełącz na tryb `"lenient"` (domyślny)
|
||||
2. Dodaj limit kontekstu modelu w `modelCapabilities.ts`
|
||||
3. Usuń filtrowanie kontekstu i polegaj na kolejności strategii
|
||||
|
||||
### Sortowanie nie jest stosowane
|
||||
|
||||
**Problem**: `preferLargeContext` nie zmienia kolejności
|
||||
|
||||
**Sprawdź**:
|
||||
|
||||
1. Zweryfikuj `preferLargeContext: true` w config
|
||||
2. Sprawdź, czy wszystkie cele mają nieznany kontekst (wszystkie sortują się równo)
|
||||
3. Upewnij się, że po filtrowaniu pozostało wiele celów
|
||||
|
||||
## Powiązane
|
||||
|
||||
- [Strategie routingu Auto-Combo](./routing/AUTO-COMBO.md)
|
||||
- [Przewodnik po odporności (Resilience)](./architecture/RESILIENCE_GUIDE.md)
|
||||
|
||||
## Historia wersji
|
||||
|
||||
- **v3.8.47**: Pierwsza implementacja
|
||||
- Dodano config `contextRequirements`
|
||||
- Utworzono backendowy moduł filtrowania
|
||||
- Pełne pokrycie testami (brak jeszcze dedykowanego UI w dashboardzie — konfiguracja przez combo JSON)
|
||||
@@ -1,83 +0,0 @@
|
||||
# Analiza konfliktu portów proxy
|
||||
|
||||
## Podsumowanie
|
||||
|
||||
W systemie proxy auto-select / proxyFallback / proxyEgress **nie ma konfliktu portów**.
|
||||
Podsystem proxy używa **wcześniej przypisanych portów z rejestru** — nigdy nie bindowuje
|
||||
się bezpośrednio do portów TCP. Prawdziwa historia EADDRINUSE leży w warstwie
|
||||
**process supervisor**, gdzie główny port nasłuchu serwera może kolidować podczas
|
||||
restartów w pętli awarii (crash-loop).
|
||||
|
||||
---
|
||||
|
||||
## Podsystem proxy: brak bindowania portów
|
||||
|
||||
| Moduł | Co robi |
|
||||
| ---------------------- | ----------------------------------------------------------------------------------------------------- |
|
||||
| `proxyAutoSelector.ts` | Wybiera konfigurację proxy z DB, stosując health scores i grupy rotacji |
|
||||
| `proxyFallback.ts` | Implementuje strategie retry/fallback, gdy wybrane proxy zawodzi (spróbuj innego proxy, potem direct) |
|
||||
| `proxyEgress.ts` | Sondowanie/propagacja informacji o egress IP do logowania — używa HTTP echo, nie bindowania portów |
|
||||
| `proxyDispatcher.ts` | Tworzy dispatchery `undici.ProxyAgent` — to poziom HTTP (forward proxy), nie gniazda nasłuchu TCP |
|
||||
| `proxyFetch.ts` | Spatchowany globalny fetch, który stosuje dispatchery proxy na poziomie undici |
|
||||
|
||||
Żaden z tych modułów nie wywołuje `net.createServer()`, `http.createServer()` ani `app.listen()`.
|
||||
Zarządzanie portami odbywa się wyłącznie w cyklu życia żądania — undici zarządza pulą
|
||||
połączeń TCP wewnętrznie.
|
||||
|
||||
**Przepływ fallback** (z `proxyFetch.ts` `runWithProxyContext`):
|
||||
|
||||
1. Spróbuj przypisanego proxy → proxy dispatcher
|
||||
2. Jeśli nieosiągalne → direct fallback (bez dispatchera)
|
||||
3. Jeśli nadal zawodzi → błąd propagowany w górę
|
||||
|
||||
W tym przepływie nie następuje alokacja ani zwalnianie portów.
|
||||
|
||||
---
|
||||
|
||||
## Prawdziwa przyczyna EADDRINUSE: wyścig restartu w crash-loop
|
||||
|
||||
Rzeczywisty konflikt portów był w **process supervisor** (`bin/cli/runtime/`):
|
||||
|
||||
| Plik | Rola |
|
||||
| ----------------------- | --------------------------------------------------------------------------------- |
|
||||
| `processSupervisor.mjs` | `ServerSupervisor` — uruchamia proces potomny, monitoruje kod wyjścia, restartuje |
|
||||
| `supervisorPolicy.mjs` | `waitUntilPortFree()`, `isPortFree()`, stałe polityki restartu |
|
||||
|
||||
**Przyczyna główna:** Gdy proces potomny serwera ulegał awarii i był natychmiast restartowany,
|
||||
OS nie zdążył jeszcze zwolnić gniazda nasłuchu (TIME_WAIT / TCP lingering). Próba restartu
|
||||
bindowała ten sam port i natychmiast kończyła się `EADDRINUSE`, powodując
|
||||
kolejną awarię → kolejny restart → wyczerpany budżet restartów → gateway martwy.
|
||||
|
||||
**Poprawka (#4425, w `supervisorPolicy.mjs`):**
|
||||
|
||||
1. Dodano `isPortFree(port)` — próbuje `net.createServer().listen()` na docelowym
|
||||
porcie; zwraca `false` przy EADDRINUSE.
|
||||
2. Dodano `waitUntilPortFree(port, timeoutMs=10000, intervalMs=250)` — odpytuje co 250ms
|
||||
przez maks. 10s, aż port będzie wolny, dopiero potem pozwala na restart.
|
||||
3. Podniesiono `RESTART_RESET_MS` z 30s → 60s — okno awarii było zbyt krótkie, co powodowało
|
||||
szybkie kaskadowe restarty w obrębie okna.
|
||||
4. Podniesiono `DEFAULT_MAX_RESTARTS` z 2 → 3 — większy zapas na przejściowe awarie.
|
||||
|
||||
Narzędzia `writePidFile()` / `killAllSubprocesses()` / `cleanupPidFile()` w
|
||||
`bin/cli/utils/pid.mjs` zapewniają czysty cykl życia pliku PID.
|
||||
|
||||
## Powiązane: Live-Dashboard EADDRINUSE (#6324)
|
||||
|
||||
Równoległa poprawka (`live-ws-eaddrinuse-6324.test.ts`) gwarantuje, że `startLiveDashboardServer()`
|
||||
odrzuca z właściwym błędem `EADDRINUSE` (zamiast nieobsłużonego zdarzenia socket 'error',
|
||||
które crashowałoby proces). Serwer dashboardu używa osobnego portu względem głównego
|
||||
serwera API, więc gdy oba są skonfigurowane na ten sam port, drugie bindowanie kończy się
|
||||
niepowodzeniem w sposób kontrolowany (gracefully).
|
||||
|
||||
---
|
||||
|
||||
## Stan obecny
|
||||
|
||||
| Ryzyko | Status | Pozostało |
|
||||
| -------------------------------------------- | ----------------------- | --------- |
|
||||
| Supervisor restart EADDRINUSE | **Naprawione** (#4425) | Brak |
|
||||
| LiveWS port clash | **Naprawione** (#6324) | Brak |
|
||||
| Proxy selection port clash | **Nigdy nie dotyczyło** | Brak |
|
||||
| Two Redis CLIENT factories bind no TCP ports | **Nigdy nie dotyczyło** | Brak |
|
||||
|
||||
Nie są potrzebne dalsze działania w sprawie konfliktu portów.
|
||||
@@ -1,370 +0,0 @@
|
||||
# Subskrypcje proxy operatora (styl Karing)
|
||||
|
||||
> Notatki projektowe i implementacyjne dla operatorowego przepływu subskrypcji
|
||||
> proxy w OmniRoute. To jest cięcie v1: pojedynczy operator wkleja linki
|
||||
> subskrypcji, wybiera tryb (global lub rule), a OmniRoute wiąże wynikową pulę
|
||||
> proxy z istniejącą rezolucją scope. Multi-tenant per-API-key, zaawansowane
|
||||
> reguły ruchu, wagi per-rule sterowane latencją itd. są jawnie poza zakresem
|
||||
> i wymienione w §7.
|
||||
|
||||
---
|
||||
|
||||
## 1. Motywacja
|
||||
|
||||
Dziś pula proxy OmniRoute jest ręcznie kuratorowana: każdy węzeł żyje w
|
||||
`proxy_registry` z ręcznie wpisanym host/port/credentials, a każde powiązanie z
|
||||
upstreamowymi dispatcherami (account → provider → combo → global → direct) to
|
||||
ręczny wiersz `proxy_assignments`. Operatorzy, którzy już utrzymują subskrypcję
|
||||
Clash/V2Ray/sing-box (np. z usługi airport), muszą przepisywać każdy węzeł do
|
||||
OmniRoute i ponownie je wiązać przy każdej zmianie listy upstream.
|
||||
|
||||
Celem v1 jest uczynienie OmniRoute first-class dla subskrypcji
|
||||
**dostarczanych przez operatora**, podobnie jak Karing / Clash / sing-box
|
||||
pozwalają wkleić URL `https://...` i zostawić zarządzanie cyklem życia klientowi.
|
||||
|
||||
## 2. Historie użytkownika
|
||||
|
||||
| # | Jako | Chcę | Aby |
|
||||
| --- | -------- | --------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------ |
|
||||
| U1 | Operator | wkleić URL subskrypcji raz | nie przepisywać węzłów przy każdym odświeżeniu airport |
|
||||
| U2 | Operator | włączać/wyłączać subskrypcję | móc wrócić do direct bez usuwania URL |
|
||||
| U3 | Operator | wybrać tryb **global** | cały ruch każdego providera wychodził przez subskrypcję |
|
||||
| U4 | Operator | wybrać tryb **rule** i wskazać konkretnych providerów | tylko wybrani providerzy szli przez proxy; pozostali zostawali direct |
|
||||
| U5 | Operator | podać lokalny endpoint SOCKS5 sing-box/clash | węzły SS/VMess/Trojan/VLESS (których dispatcher OmniRoute nie mówi natywnie) stawały się używalne przez lokalny mostek kernela |
|
||||
| U6 | Operator | widzieć status pobrania i niedawną zredagowaną (redacted) sumę węzłów | debugować „dlaczego pusto / błąd” bez wycieku credentials |
|
||||
|
||||
## 3. Poza zakresem (v1)
|
||||
|
||||
- Nadpisania subskrypcji per-API-key (multi-tenant). v1 jest wyłącznie operatorskie.
|
||||
- Reguły ruchu per-provider poza `global` / `rule-on-selected-providers`.
|
||||
- Inteligentny routing oparty o latencję między węzłami subskrypcji a innymi pulami
|
||||
(istniejące `resolveProxyForConnectionFromRegistry` już to robi dla puli
|
||||
globalnej; v1 tylko dokłada do niej węzły subskrypcji).
|
||||
- Auto-import URL/hasła z nagłówków lub query params.
|
||||
- Mitygacja SSRF poza endpointami local-core wyłącznie na loopback (sam URL
|
||||
subskrypcji jest kontrolowany przez operatora, więc ufamy mu tak samo jak
|
||||
dzisiejszym URL-om providerów upstream).
|
||||
|
||||
## 4. Architektura
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────┐
|
||||
│ dashboard / settings / 代理 / 订阅代理 │
|
||||
│ (client component, SubscriptionTab) │
|
||||
└──────────────────┬──────────────────────┘
|
||||
│ fetch
|
||||
▼
|
||||
┌────────────────────────────────────────────────────────┐
|
||||
│ /api/v1/management/proxy-subscriptions │
|
||||
│ ├ GET list │
|
||||
│ ├ POST create │
|
||||
│ ├ GET /:id │
|
||||
│ ├ PATCH /:id │
|
||||
│ ├ DELETE /:id │
|
||||
│ ├ POST /:id/refresh │
|
||||
│ └ GET /:id/nodes │
|
||||
└────────────────────────┬───────────────────────────────┘
|
||||
│ uses
|
||||
▼
|
||||
┌────────────────────────────────────────────────────────┐
|
||||
│ src/lib/proxySubscription/ │
|
||||
│ ├ parse.ts (Clash YAML / V2Ray JSON / URIs) │
|
||||
│ ├ subscriptionService.ts │
|
||||
│ │ CRUD, sync, apply, unapply, scheduler │
|
||||
│ └ index.ts (barrel) │
|
||||
└──────────┬─────────────────────────────┬───────────────┘
|
||||
│ upsert/scope-bind │ DB
|
||||
▼ ▼
|
||||
┌─────────────────────────┐ ┌──────────────────────────┐
|
||||
│ proxy_registry │ │ proxy_subscriptions │
|
||||
│ (existing) + │ │ (NEW — subscription │
|
||||
│ subscription_id column │ │ metadata + scheduler │
|
||||
│ + status/health checks │ │ state) │
|
||||
└─────────────────────────┘ └──────────────────────────┘
|
||||
│
|
||||
▼ (existing)
|
||||
resolveProxyForConnectionFromRegistry
|
||||
hasBlockingProxyAssignment (fail-closed)
|
||||
proxyDispatcher (open-sse/utils/proxyDispatcher)
|
||||
```
|
||||
|
||||
Kluczowa decyzja projektowa: **nie wymyślamy nowego scope ani pipeline’u routingu**.
|
||||
Upsertujemy węzły pochodzące z subskrypcji do `proxy_registry` z `source =
|
||||
'subscription'` + `subscription_id`, a następnie `applySubscription()` przechodzi
|
||||
istniejące API `addProxyToScopePool(scope, scopeId, proxyId)`. Dzięki temu:
|
||||
|
||||
- Istniejąca rotacja, health checki i strażniki fail-closed działają „za darmo”.
|
||||
- Istniejące dashboardy (ProxyPoolTab, SourceToggleBar, GlobalConfigTab) działają
|
||||
bez zmian — węzły subskrypcji po prostu pojawiają się w puli z odznaką `source`.
|
||||
- Usunięcie/wyłączenie subskrypcji czysto usuwa jej powiązania bez
|
||||
ruszania ręcznych proxy.
|
||||
|
||||
## 5. Model danych
|
||||
|
||||
### 5.1 Nowa tabela `proxy_subscriptions`
|
||||
|
||||
| Column | Type | Notes |
|
||||
| ------------------------- | -------------------------------- | ---------------------------------------------------------------------- |
|
||||
| `id` | TEXT PK | UUID |
|
||||
| `name` | TEXT NOT NULL | nazwa wyświetlana |
|
||||
| `url` | TEXT NOT NULL | URL subskrypcji |
|
||||
| `enabled` | INTEGER NOT NULL DEFAULT 0 | 1 = aktywna |
|
||||
| `mode` | TEXT NOT NULL DEFAULT `'global'` | `'global'` lub `'rule'` |
|
||||
| `rule_providers` | TEXT NULL | tablica JSON ID providerów (tylko mode='rule') |
|
||||
| `local_core_endpoint` | TEXT NULL | loopback SOCKS5/HTTP dla SS/VMess/itd. (np. `socks5://127.0.0.1:2080`) |
|
||||
| `update_interval_minutes` | INTEGER NOT NULL DEFAULT 60 | kadencja odświeżania w tle |
|
||||
| `last_fetched_at` | TEXT NULL | znacznik czasu ISO ostatniego udanego pobrania |
|
||||
| `status` | TEXT NOT NULL DEFAULT `'empty'` | `'ok'` / `'error'` / `'empty'` |
|
||||
| `error` | TEXT NULL | ostatni tekst błędu / ostrzeżenia (zredagowany) |
|
||||
| `last_nodes` | TEXT NULL | tablica JSON, zredagowane podsumowania węzłów |
|
||||
| `created_at` | TEXT NOT NULL | ISO |
|
||||
| `updated_at` | TEXT NOT NULL | ISO |
|
||||
|
||||
Indeks: `idx_proxy_subscriptions_enabled (enabled)` na tick schedulera.
|
||||
|
||||
### 5.2 Rozszerzone `proxy_registry`
|
||||
|
||||
Dodana jedna kolumna:
|
||||
|
||||
| Column | Type | Notes |
|
||||
| ----------------- | --------- | ------------------------------------------------------------------------------------ |
|
||||
| `subscription_id` | TEXT NULL | FK z konwencji (bez wymuszanego FK; wiersz subskrypcji żyje w `proxy_subscriptions`) |
|
||||
|
||||
Istniejące wiersze po upgrade: `subscription_id = NULL`, zachowanie bez zmian.
|
||||
Migracja: `ALTER TABLE proxy_registry ADD COLUMN subscription_id TEXT;`
|
||||
(stosowana jako `131_proxy_subscriptions.sql`, idempotentna dzięki semantyce
|
||||
`ALTER` w migration runnerze).
|
||||
|
||||
### 5.3 Izolacja testów rozszerzonego `proxy_subscriptions`
|
||||
|
||||
Migration runner stosuje nowe migracje automatycznie; jedyne miejsca, które
|
||||
muszą znać nową kolumnę, to `types.ts` i `mappers.ts` (po jednym dodatkowym
|
||||
polu) oraz `proxies.ts` (3 instrukcje SQL: INSERT/UPDATE/SELECT).
|
||||
|
||||
## 6. Tryby
|
||||
|
||||
### 6.1 Tryb global
|
||||
|
||||
- Pula powiązana z `scope='global', scope_id=NULL`.
|
||||
- Ustawienie `proxyEnabled` wymuszane na `true`, gdy aktywna jest jakakolwiek
|
||||
subskrypcja (lub jakiekolwiek globalne proxy spoza subskrypcji).
|
||||
- Cały ruch providerów wychodzi przez pulę subskrypcji, z rotacją/health
|
||||
stosowanymi przez istniejące `resolveProxyForConnectionFromRegistry`.
|
||||
|
||||
### 6.2 Tryb rule
|
||||
|
||||
- Pula powiązana z `scope='provider', scope_id=<selected provider id>` dla każdego
|
||||
wybranego providera.
|
||||
- Providerzy spoza listy przechodzą na direct (własne proxy na poziomie
|
||||
providera albo brak proxy).
|
||||
- Przełączenie subskrypcji z global → rule najpierw wywołuje `unapplySubscription`,
|
||||
by odłączyć poprzednie powiązania globalne, a potem ponownie synchronizuje.
|
||||
|
||||
## 7. Wsparcie protokołów
|
||||
|
||||
Istniejący `proxyDispatcher` mówi tylko **http / https / socks5 / vercel /
|
||||
deno / cloudflare**. v1 idzie za tym:
|
||||
|
||||
| Parser-detected type | Goes into pool directly? | Needs `localCoreEndpoint`? |
|
||||
| --------------------------------- | ------------------------ | -------------------------------------- |
|
||||
| `http` / `https` | yes | no |
|
||||
| `socks5` | yes | no |
|
||||
| `ss` / `ssr` | no | yes (sing-box/clash → loopback SOCKS5) |
|
||||
| `vmess` / `vless` | no | yes |
|
||||
| `trojan` | no | yes |
|
||||
| `hysteria` / `tuic` / `wireguard` | no | yes |
|
||||
| `relay` (vercel/deno/cloudflare) | yes | no |
|
||||
|
||||
Bez `localCoreEndpoint` węzły klasy SS pojawiają się w statusie jako
|
||||
ostrzeżenie, ale **nie są routowane**. To odpowiada polityce „fail-closed, ale
|
||||
nie kłam o możliwościach”: nigdy cicho nie gubimy ruchu; raportujemy
|
||||
nieroutowalne węzły i zostawiamy decyzję operatorowi.
|
||||
|
||||
## 8. Parser (`src/lib/proxySubscription/parse.ts`)
|
||||
|
||||
Ręcznie napisany, bez zewnętrznej zależności. Akceptowane wejścia:
|
||||
|
||||
1. **Clash / Clash.Meta YAML** — tablica `proxies:`, z dispatch po `type`.
|
||||
2. **Lista URI owinięta Base64** — `parseSubscription` wykrywa base64 po długości
|
||||
i zestawie znaków, dekoduje, potem parsuje URI.
|
||||
3. **JSON-array-of-URI w stylu V2RayN** — używa URI `vmess://` / `vless://`.
|
||||
4. **Zwykła lista URI** — `ss://`, `vmess://`, `vless://`, `trojan://`,
|
||||
`hysteria://`, `tuic://`, `wireguard://`, `socks5://`, `http(s)://`.
|
||||
|
||||
Wyjście:
|
||||
|
||||
```ts
|
||||
type ParsedSubscription = {
|
||||
nodes: DirectlyUsableNode[]; // http/https/socks5/relay
|
||||
needsCore: NeedsCoreNode[]; // ss/vmess/... — redacted summary
|
||||
rawProtocols: string[]; // for diagnostics
|
||||
parserWarnings: string[]; // per-line parse errors, redacted
|
||||
};
|
||||
|
||||
type DirectlyUsableNode = {
|
||||
name: string;
|
||||
type: "http" | "https" | "socks5" | "vercel" | "deno" | "cloudflare";
|
||||
host: string;
|
||||
port: number;
|
||||
username?: string;
|
||||
password?: string;
|
||||
};
|
||||
```
|
||||
|
||||
`redactedNodeSummary` zwraca serializowalną do JSON tablicę `{name, type,
|
||||
host, port, hasCredentials}` z pominiętymi credentials. To trafia do
|
||||
`last_nodes` na potrzeby UI operatora.
|
||||
|
||||
## 9. Bezpieczeństwo
|
||||
|
||||
- **SSRF na `localCoreEndpoint`**: jedyna powierzchnia SSRF to lokalny
|
||||
endpoint core (sam URL subskrypcji dostarcza operator). Dozwolone
|
||||
hosty: `127.0.0.1`, `::1`, `localhost`. Każdy inny host jest odrzucany przy
|
||||
parsowaniu ze statusem `subscription_needs_core_endpoint_invalid`.
|
||||
- **Brak outboundu do hostów wewnętrznych operatora** z URL subskrypcji. Pobranie
|
||||
URL idzie przez `fetch` Node (ten sam model zaufania co istniejące
|
||||
health checki `proxyLatency` i taski ping providerów). Operator
|
||||
już ufa URL, bo go wkleił.
|
||||
- **Fail-closed**: jeśli proxy subskrypcji jest martwe, ale nadal powiązane ze
|
||||
scope, `hasBlockingProxyAssignment` zwraca true i ruch kończy się fail-closed —
|
||||
zgodnie z istniejącą polityką dla dowolnego proxy z puli. Operator zawsze może
|
||||
wyłączyć subskrypcję lub usunąć powiązanie.
|
||||
- **Brak echa sekretów**: `last_nodes` jest zredagowane; UI nigdy nie odsyła
|
||||
sekretów. `password` / `username` są przechowywane zaszyfrowane at rest przez
|
||||
istniejący tor szyfrowania `proxy_registry`.
|
||||
- **Brak zapisu cross-tenant**: trasy API są strzeżone przez `requireManagementAuth`
|
||||
(sesja dashboardu LUB klucz API ze scope manage). Nadpisania per-API-key są
|
||||
jawnie poza zakresem.
|
||||
|
||||
## 10. UI
|
||||
|
||||
Nowa podzakładka **"订阅代理"** w `dashboard / settings / 代理`, umieszczona po
|
||||
„documentation”. Widok listy pokazuje:
|
||||
|
||||
- Name + URL (obcięty, pełny URL w atrybucie `title`)
|
||||
- Odznaka statusu: `ok` / `error` / `empty`
|
||||
- Przełącznik Enabled (optimistic toggle)
|
||||
- Przyciski akcji: edit / refresh / delete
|
||||
|
||||
Formularz edycji ma:
|
||||
|
||||
- Name (tekst, wymagane)
|
||||
- URL (tekst, wymagane, walidowane jako URL)
|
||||
- Przełącznik Mode (global / rule)
|
||||
- Multi-select providerów (widoczny tylko w trybie rule; zasilany z
|
||||
`/api/providers`)
|
||||
- Local core endpoint (tekst, opcjonalny; placeholder `socks5://127.0.0.1:2080`)
|
||||
- Update interval (liczba, domyślnie 60 minut)
|
||||
- Przełącznik Enabled
|
||||
|
||||
Gdy `status === 'error'`, baner ostrzeżenia inline pokazuje `subscription.error`.
|
||||
Gdy `status === 'ok'` i są węzły wymagające local core, miękki
|
||||
baner ostrzeżenia pokazuje, które protokoły pominięto.
|
||||
|
||||
## 11. Migracja i rollout
|
||||
|
||||
1. Nowa migracja `131_proxy_subscriptions.sql` uruchamia się przy pierwszym otwarciu DB po
|
||||
upgrade (auto-wykrywana przez istniejący migration runner).
|
||||
2. Migracja jest **idempotentna**: `ALTER TABLE … ADD COLUMN …` na już
|
||||
zmigrowanej DB to no-op w SQLite, gdy owinięte w ścieżkę runnera
|
||||
„ignore duplicate column”. Zob. istniejące precedensy
|
||||
`040_oneproxy_proxy_fields.sql` i `093_proxy_enable_toggles.sql`.
|
||||
3. Bez backfill: istniejące wiersze dostają `subscription_id = NULL`, co serwis
|
||||
traktuje jako „manual, not subscription-managed”.
|
||||
4. UI ukrywa zakładkę przy zerze subskrypcji, ale API jest zawsze
|
||||
dostępne — to celowe, by operatorzy headless mogli zarządzać
|
||||
subskrypcjami wyłącznie przez API.
|
||||
|
||||
## 12. Auto-odświeżanie
|
||||
|
||||
`startSubscriptionScheduler()` jest idempotentny i:
|
||||
|
||||
- Pomija przeglądarkę (`typeof window !== "undefined"`).
|
||||
- Pomija przy `NODE_ENV=test`.
|
||||
- W przeciwnym razie startuje 60s `setInterval`, który:
|
||||
- Listuje włączone subskrypcje.
|
||||
- Dla każdej liczy `due = now - lastFetchedAt >= updateIntervalMinutes * 60_000`.
|
||||
- Wywołuje `syncSubscription` dla zaległych, połykając błędy (logowane).
|
||||
- Timer interwału ma `.unref()`, więc nigdy nie blokuje wyjścia procesu.
|
||||
|
||||
Scheduler startuje przy:
|
||||
|
||||
- Pierwszym `GET /api/v1/management/proxy-subscriptions` (otwarcie dashboardu).
|
||||
- Dowolnym wywołaniu `syncSubscription` (defensywnie — dla ścieżek CLI / automacji,
|
||||
które omijają GET).
|
||||
|
||||
## 13. Strategia testów
|
||||
|
||||
`tests/unit/proxySubscription.parse.test.ts` — 7 czystych przypadków parsera, bez DB,
|
||||
uruchamialne w <1s:
|
||||
|
||||
1. Clash YAML z węzłami `direct` (http) i `needsCore` (ss).
|
||||
2. Lista URI owinięta Base64 (poprawnie zdekodowana).
|
||||
3. V2Ray JSON-array-of-URI (vmess / vless).
|
||||
4. Zwykła lista URI (mieszane protokoły).
|
||||
5. Outboundy Clash.Meta (socks5).
|
||||
6. Puste / nieznane wejście → `nodes=[]`, `needsCore=[]`, parserWarnings wypełnione.
|
||||
7. `redactedNodeSummary` usuwa credentials.
|
||||
|
||||
`tests/unit/proxySubscription.service.test.ts` — 4 testy integracyjne używające
|
||||
`process.env.DATA_DIR` + `core.resetDbInstance()`:
|
||||
|
||||
1. **Global**: utwórz włączoną subskrypcję global → `syncSubscription` →
|
||||
zweryfikuj wiersze puli w `proxy_registry` z ustawionym `subscription_id` →
|
||||
`resolveProxyForConnectionFromRegistry` zwraca jeden z tych wierszy →
|
||||
`proxyEnabled` jest true.
|
||||
2. **Rule**: utwórz włączoną subskrypcję rule na providerze P1 → zweryfikuj, że tylko
|
||||
scope P1 jest powiązany, scope P2 nietknięty.
|
||||
3. **Fail-closed**: URL pobrania subskrypcji jest nieosiągalny → `status='error'`,
|
||||
pula pusta, a jeśli kiedykolwiek miała wiersze, są wyczyszczone;
|
||||
`hasBlockingProxyAssignment` zwraca false (brak martwych proxy w żadnym scope).
|
||||
4. **Delete**: usuń subskrypcję → wiersze rejestru dla tej subskrypcji są
|
||||
usuwane z `force: true` (ręczne usunięcia nie mogą zablokować kaskady) →
|
||||
`proxyEnabled` przeliczone.
|
||||
|
||||
Komenda uruchomienia testów:
|
||||
|
||||
```bash
|
||||
node --import tsx/esm \
|
||||
--import ./open-sse/utils/setupPolyfill.ts \
|
||||
--import ./tests/_setup/isolateDataDir.ts \
|
||||
--test \
|
||||
tests/unit/proxySubscription.parse.test.ts \
|
||||
tests/unit/proxySubscription.service.test.ts
|
||||
```
|
||||
|
||||
## 14. Prace przyszłe (NIE w v1)
|
||||
|
||||
- Nadpisania subskrypcji per-API-key (multi-tenant; wymaga tabeli `key_subscription_overrides`).
|
||||
- Reguły ruchu per-provider z matcherami domen (weszłyby w istniejącą tabelę `interceptionRules`).
|
||||
- Rotacja ważona latencją między pulami subskrypcji (mamy już `ProxyRotationStrategy = "latency"`; wystarczy wystawić w UI).
|
||||
- Proxyowanie samego pobrania subskrypcji przez osobny egress (by operatorzy mogli pobierać zza firmowego firewalla).
|
||||
- Podgląd sparsowanej subskrypcji po stronie przeglądarki przed zapisem (dziś trzeba save → wait → see nodes).
|
||||
|
||||
## 15. Pliki dodane / zmienione
|
||||
|
||||
**Dodane (nowe):**
|
||||
|
||||
- `src/lib/proxySubscription/parse.ts`
|
||||
- `src/lib/proxySubscription/subscriptionService.ts`
|
||||
- `src/lib/proxySubscription/index.ts`
|
||||
- `src/lib/db/migrations/131_proxy_subscriptions.sql`
|
||||
- `src/app/api/v1/management/proxy-subscriptions/route.ts`
|
||||
- `src/app/api/v1/management/proxy-subscriptions/[id]/route.ts`
|
||||
- `src/app/api/v1/management/proxy-subscriptions/[id]/refresh/route.ts`
|
||||
- `src/app/api/v1/management/proxy-subscriptions/[id]/nodes/route.ts`
|
||||
- `src/app/(dashboard)/dashboard/settings/components/proxy/SubscriptionTab.tsx`
|
||||
- `tests/unit/proxySubscription.parse.test.ts`
|
||||
- `tests/unit/proxySubscription.service.test.ts`
|
||||
- `docs/proxy-subscriptions.md` (ten plik)
|
||||
|
||||
**Zmodyfikowane (minimalnie):**
|
||||
|
||||
- `src/lib/db/proxies/types.ts` — `+ subscriptionId: string | null` na
|
||||
`ProxyRegistryRecord`; `+ subscriptionId?: string | null` na `ProxyPayload`.
|
||||
- `src/lib/db/proxies/mappers.ts` — `mapProxyRow` czyta
|
||||
`subscription_id` z wiersza.
|
||||
- `src/lib/db/proxies.ts` — INSERT / UPDATE / SELECT dodają `subscription_id`.
|
||||
- `src/app/(dashboard)/dashboard/settings/components/ProxyTab.tsx` — dodaje
|
||||
jedną nową podzakładkę ("订阅代理") + fallback `literal` dla etykiet, których
|
||||
jeszcze nie ma w katalogu i18n.
|
||||
@@ -1,86 +0,0 @@
|
||||
---
|
||||
title: "Proxy Port Clash Investigation"
|
||||
version: 3.8.50
|
||||
lastUpdated: 2026-08-06
|
||||
---
|
||||
|
||||
# Proxy Port Clash Investigation
|
||||
|
||||
## Summary
|
||||
|
||||
There is **no port clash** in the proxy auto-select / proxyFallback / proxyEgress system.
|
||||
The proxy subsystem uses **pre-assigned registry ports** — it never binds to TCP ports
|
||||
directly. The real EADDRINUSE history is in the **process supervisor** layer, where
|
||||
the server's main listen port can clash during crash-loop restarts.
|
||||
|
||||
---
|
||||
|
||||
## Proxy Subsystem: No Port Binding
|
||||
|
||||
| Module | What It Does |
|
||||
|---|---|
|
||||
| `proxyAutoSelector.ts` | Selects a proxy config from the DB by applying health scores and rotation groups |
|
||||
| `proxyFallback.ts` | Implements retry/fallback strategies when a selected proxy fails (try another proxy, then direct) |
|
||||
| `proxyEgress.ts` | Probes/propagates egress IP info for logging — uses HTTP echo, not port binding |
|
||||
| `proxyDispatcher.ts` | Creates `undici.ProxyAgent` dispatchers — these are HTTP-level (forward proxy), not TCP listen sockets |
|
||||
| `proxyFetch.ts` | Patched global fetch that applies proxy dispatchers at the undici level |
|
||||
|
||||
None of these modules call `net.createServer()`, `http.createServer()`, or `app.listen()`.
|
||||
Port management is entirely within the request life cycle — undici manages the TCP
|
||||
connection pool internally.
|
||||
|
||||
**Fallback flow** (from `proxyFetch.ts` `runWithProxyContext`):
|
||||
1. Try assigned proxy → proxy dispatcher
|
||||
2. If unreachable → direct fallback (no dispatcher)
|
||||
3. If still failing → error propagated up
|
||||
|
||||
No port allocation or release happens in this flow.
|
||||
|
||||
---
|
||||
|
||||
## Real EADDRINUSE Root Cause: Crash-Loop Restart Race
|
||||
|
||||
The actual port clash was in the **process supervisor** (`bin/cli/runtime/`):
|
||||
|
||||
| File | Role |
|
||||
|---|---|
|
||||
| `processSupervisor.mjs` | `ServerSupervisor` — spawns a child process, monitors exit code, restarts |
|
||||
| `supervisorPolicy.mjs` | `waitUntilPortFree()`, `isPortFree()`, restart policy constants |
|
||||
|
||||
**Root cause:** When the server child process crashed and was immediately restarted, the
|
||||
OS had not yet released the listen socket (TIME_WAIT / TCP lingering). The restart
|
||||
attempt would bind to the same port and immediately fail with `EADDRINUSE`, causing
|
||||
another crash → another restart → exhausted restart budget → gateway dead.
|
||||
|
||||
**Fix (#4425, in `supervisorPolicy.mjs`):**
|
||||
1. Added `isPortFree(port)` — attempts a `net.createServer().listen()` on the target
|
||||
port; resolves `false` if EADDRINUSE.
|
||||
2. Added `waitUntilPortFree(port, timeoutMs=10000, intervalMs=250)` — polls every 250ms
|
||||
for up to 10s until the port is free, then allows the restart.
|
||||
3. Bumped `RESTART_RESET_MS` from 30s → 60s — the crash window was too short, causing
|
||||
rapid cascading restarts inside the window.
|
||||
4. Bumped `DEFAULT_MAX_RESTARTS` from 2 → 3 — more headroom for transient failures.
|
||||
|
||||
The `writePidFile()` / `killAllSubprocesses()` / `cleanupPidFile()` utilities in
|
||||
`bin/cli/utils/pid.mjs` ensure clean PID file lifecycle.
|
||||
|
||||
## Related: Live-Dashboard EADDRINUSE (#6324)
|
||||
|
||||
A parallel fix (`live-ws-eaddrinuse-6324.test.ts`) ensures `startLiveDashboardServer()`
|
||||
rejects with a proper `EADDRINUSE` error (instead of an unhandled socket 'error' event
|
||||
that would crash the process). The dashboard server uses a separate port from the main
|
||||
API server, so when both are configured on the same port, the second bind fails
|
||||
gracefully.
|
||||
|
||||
---
|
||||
|
||||
## Current State
|
||||
|
||||
| Risk | Status | Remaining |
|
||||
|---|---|---|
|
||||
| Supervisor restart EADDRINUSE | **Fixed** (#4425) | None |
|
||||
| LiveWS port clash | **Fixed** (#6324) | None |
|
||||
| Proxy selection port clash | **Never applicable** | None |
|
||||
| Two Redis CLIENT factories bind no TCP ports | **Never applicable** | None |
|
||||
|
||||
No further action needed on port clash.
|
||||
@@ -1,377 +0,0 @@
|
||||
---
|
||||
title: "Operator Proxy Subscriptions (Karing-style)"
|
||||
version: 3.8.50
|
||||
lastUpdated: 2026-08-06
|
||||
---
|
||||
|
||||
# Operator Proxy Subscriptions (Karing-style)
|
||||
|
||||
> Feature design + implementation notes for OmniRoute's operator-level proxy
|
||||
> subscription flow. This is the v1 cut: a single operator pastes subscription
|
||||
> links, picks a mode (global or rule), and OmniRoute binds the resulting proxy
|
||||
> pool into the existing scope resolution. Multi-tenant per-API-key, advanced
|
||||
> traffic rules, latency-driven per-rule weights, and so on are explicitly
|
||||
> out-of-scope and listed in §7.
|
||||
|
||||
---
|
||||
|
||||
## 1. Motivation
|
||||
|
||||
Today, OmniRoute's proxy pool is hand-curated: every node lives in
|
||||
`proxy_registry` with hand-written host/port/credentials, and every binding to
|
||||
the upstream dispatchers (account → provider → combo → global → direct) is a
|
||||
manual `proxy_assignments` row. Operators who already maintain a Clash/V2Ray/
|
||||
sing-box subscription (e.g. from an airport service) have to retype every node
|
||||
into OmniRoute and re-bind them whenever the upstream list changes.
|
||||
|
||||
The goal of v1 is to make OmniRoute first-class for **operator-supplied**
|
||||
subscriptions, similar to how Karing / Clash / sing-box let users paste a
|
||||
`https://...` URL and have the client manage the lifecycle.
|
||||
|
||||
## 2. User stories
|
||||
|
||||
| # | As a(n) | I want to | So that |
|
||||
|---|---------|-----------|---------|
|
||||
| U1 | Operator | paste a subscription URL once | I don't retype nodes every time the airport refreshes |
|
||||
| U2 | Operator | toggle the subscription on/off | I can fall back to direct without deleting the URL |
|
||||
| U3 | Operator | pick **global** mode | every provider's traffic exits via the subscription |
|
||||
| U4 | Operator | pick **rule** mode and select specific providers | only selected providers route through the proxy; others stay direct |
|
||||
| U5 | Operator | supply a local sing-box/clash SOCKS5 endpoint | SS/VMess/Trojan/VLESS nodes (which OmniRoute's dispatcher can't speak natively) become usable through a local kernel bridge |
|
||||
| U6 | Operator | see fetch status and a recent redacted node summary | I can debug "why is this empty / erroring" without leaking credentials |
|
||||
|
||||
## 3. Non-goals (v1)
|
||||
|
||||
- Per-API-key subscription overrides (multi-tenant). v1 is operator-only.
|
||||
- Per-provider traffic rules beyond `global` / `rule-on-selected-providers`.
|
||||
- Latency-based smart routing between subscription nodes and other pools
|
||||
(existing `resolveProxyForConnectionFromRegistry` already does this for the
|
||||
global pool; v1 just feeds subscription nodes into it).
|
||||
- Auto-importing URL/password from headers or query params.
|
||||
- SSRF mitigation beyond loopback-only local-core endpoints (the subscription
|
||||
URL itself is operator-controlled, so we trust it the same way we trust
|
||||
upstream provider URLs today).
|
||||
|
||||
## 4. Architecture
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────┐
|
||||
│ dashboard / settings / 代理 / 订阅代理 │
|
||||
│ (client component, SubscriptionTab) │
|
||||
└──────────────────┬──────────────────────┘
|
||||
│ fetch
|
||||
▼
|
||||
┌────────────────────────────────────────────────────────┐
|
||||
│ /api/v1/management/proxy-subscriptions │
|
||||
│ ├ GET list │
|
||||
│ ├ POST create │
|
||||
│ ├ GET /:id │
|
||||
│ ├ PATCH /:id │
|
||||
│ ├ DELETE /:id │
|
||||
│ ├ POST /:id/refresh │
|
||||
│ └ GET /:id/nodes │
|
||||
└────────────────────────┬───────────────────────────────┘
|
||||
│ uses
|
||||
▼
|
||||
┌────────────────────────────────────────────────────────┐
|
||||
│ src/lib/proxySubscription/ │
|
||||
│ ├ parse.ts (Clash YAML / V2Ray JSON / URIs) │
|
||||
│ ├ subscriptionService.ts │
|
||||
│ │ CRUD, sync, apply, unapply, scheduler │
|
||||
│ └ index.ts (barrel) │
|
||||
└──────────┬─────────────────────────────┬───────────────┘
|
||||
│ upsert/scope-bind │ DB
|
||||
▼ ▼
|
||||
┌─────────────────────────┐ ┌──────────────────────────┐
|
||||
│ proxy_registry │ │ proxy_subscriptions │
|
||||
│ (existing) + │ │ (NEW — subscription │
|
||||
│ subscription_id column │ │ metadata + scheduler │
|
||||
│ + status/health checks │ │ state) │
|
||||
└─────────────────────────┘ └──────────────────────────┘
|
||||
│
|
||||
▼ (existing)
|
||||
resolveProxyForConnectionFromRegistry
|
||||
hasBlockingProxyAssignment (fail-closed)
|
||||
proxyDispatcher (open-sse/utils/proxyDispatcher)
|
||||
```
|
||||
|
||||
Key design decision: **we do not invent a new scope or routing pipeline**. We
|
||||
upsert subscription-derived nodes into `proxy_registry` with `source =
|
||||
'subscription'` + `subscription_id`, and then `applySubscription()` walks the
|
||||
existing `addProxyToScopePool(scope, scopeId, proxyId)` API. This means:
|
||||
|
||||
- Existing rotation, health checks, and fail-closed guards apply for free.
|
||||
- Existing dashboards (ProxyPoolTab, SourceToggleBar, GlobalConfigTab) work
|
||||
unchanged — subscription nodes just appear in the pool with a `source`
|
||||
badge.
|
||||
- Deleting/disabling a subscription cleanly removes its bindings without
|
||||
touching manual proxies.
|
||||
|
||||
## 5. Data model
|
||||
|
||||
### 5.1 New table `proxy_subscriptions`
|
||||
|
||||
| Column | Type | Notes |
|
||||
|---|---|---|
|
||||
| `id` | TEXT PK | UUID |
|
||||
| `name` | TEXT NOT NULL | display name |
|
||||
| `url` | TEXT NOT NULL | subscription URL |
|
||||
| `enabled` | INTEGER NOT NULL DEFAULT 0 | 1 = active |
|
||||
| `mode` | TEXT NOT NULL DEFAULT `'global'` | `'global'` or `'rule'` |
|
||||
| `rule_providers` | TEXT NULL | JSON array of provider IDs (mode='rule' only) |
|
||||
| `local_core_endpoint` | TEXT NULL | loopback SOCKS5/HTTP for SS/VMess/etc. (e.g. `socks5://127.0.0.1:2080`) |
|
||||
| `update_interval_minutes` | INTEGER NOT NULL DEFAULT 60 | background refresh cadence |
|
||||
| `last_fetched_at` | TEXT NULL | ISO timestamp of last successful fetch |
|
||||
| `status` | TEXT NOT NULL DEFAULT `'empty'` | `'ok'` / `'error'` / `'empty'` |
|
||||
| `error` | TEXT NULL | last error / warning text (redacted) |
|
||||
| `last_nodes` | TEXT NULL | JSON array, redacted node summaries |
|
||||
| `created_at` | TEXT NOT NULL | ISO |
|
||||
| `updated_at` | TEXT NOT NULL | ISO |
|
||||
|
||||
Index: `idx_proxy_subscriptions_enabled (enabled)` for the scheduler tick.
|
||||
|
||||
### 5.2 Extended `proxy_registry`
|
||||
|
||||
Added one column:
|
||||
|
||||
| Column | Type | Notes |
|
||||
|---|---|---|
|
||||
| `subscription_id` | TEXT NULL | FK by convention (no enforced FK; subscription row lives in `proxy_subscriptions`) |
|
||||
|
||||
Existing rows on upgrade: `subscription_id = NULL`, behavior unchanged.
|
||||
Migration: `ALTER TABLE proxy_registry ADD COLUMN subscription_id TEXT;`
|
||||
(applied as `131_proxy_subscriptions.sql`, idempotent via the migration
|
||||
runner's `ALTER` semantics).
|
||||
|
||||
### 5.3 Extended `proxy_subscriptions` test isolation
|
||||
|
||||
The migration runner applies new migrations automatically; the only places
|
||||
that need to know about the new column are `types.ts` and `mappers.ts` (one
|
||||
extra field each) and `proxies.ts` (3 SQL statements: INSERT/UPDATE/SELECT).
|
||||
|
||||
## 6. Modes
|
||||
|
||||
### 6.1 Global mode
|
||||
|
||||
- Pool bound to `scope='global', scope_id=NULL`.
|
||||
- `proxyEnabled` setting forced to `true` whenever any subscription (or any
|
||||
non-subscription global proxy) is active.
|
||||
- All provider traffic exits via the subscription pool, with rotation/health
|
||||
applied by the existing `resolveProxyForConnectionFromRegistry`.
|
||||
|
||||
### 6.2 Rule mode
|
||||
|
||||
- Pool bound to `scope='provider', scope_id=<selected provider id>` for each
|
||||
selected provider.
|
||||
- Providers NOT in the list fall through to direct (their own provider-level
|
||||
proxy or no proxy).
|
||||
- Toggling a subscription from global → rule first calls `unapplySubscription`
|
||||
to detach the previous global bindings, then re-syncs.
|
||||
|
||||
## 7. Protocol support
|
||||
|
||||
The existing `proxyDispatcher` only speaks **http / https / socks5 / vercel /
|
||||
deno / cloudflare**. v1 follows that:
|
||||
|
||||
| Parser-detected type | Goes into pool directly? | Needs `localCoreEndpoint`? |
|
||||
|---|---|---|
|
||||
| `http` / `https` | yes | no |
|
||||
| `socks5` | yes | no |
|
||||
| `ss` / `ssr` | no | yes (sing-box/clash → loopback SOCKS5) |
|
||||
| `vmess` / `vless` | no | yes |
|
||||
| `trojan` | no | yes |
|
||||
| `hysteria` / `tuic` / `wireguard` | no | yes |
|
||||
| `relay` (vercel/deno/cloudflare) | yes | no |
|
||||
|
||||
Without `localCoreEndpoint`, SS-class nodes are surfaced in the status as a
|
||||
warning but **not routed**. This matches the "fail-closed, but don't lie about
|
||||
capability" policy: we never silently drop traffic; we report unrouteable
|
||||
nodes and let the operator decide.
|
||||
|
||||
## 8. Parser (`src/lib/proxySubscription/parse.ts`)
|
||||
|
||||
Hand-rolled, no external dependency. Inputs accepted:
|
||||
|
||||
1. **Clash / Clash.Meta YAML** — `proxies:` array, with `type` dispatch.
|
||||
2. **Base64-wrapped URI list** — `parseSubscription` detects base64 by length
|
||||
and charset, decodes, then URI-parses.
|
||||
3. **V2RayN-style JSON-array-of-URI** — uses `vmess://` / `vless://` URIs.
|
||||
4. **Plain URI list** — `ss://`, `vmess://`, `vless://`, `trojan://`,
|
||||
`hysteria://`, `tuic://`, `wireguard://`, `socks5://`, `http(s)://`.
|
||||
|
||||
Output:
|
||||
|
||||
```ts
|
||||
type ParsedSubscription = {
|
||||
nodes: DirectlyUsableNode[]; // http/https/socks5/relay
|
||||
needsCore: NeedsCoreNode[]; // ss/vmess/... — redacted summary
|
||||
rawProtocols: string[]; // for diagnostics
|
||||
parserWarnings: string[]; // per-line parse errors, redacted
|
||||
};
|
||||
|
||||
type DirectlyUsableNode = {
|
||||
name: string;
|
||||
type: "http" | "https" | "socks5" | "vercel" | "deno" | "cloudflare";
|
||||
host: string;
|
||||
port: number;
|
||||
username?: string;
|
||||
password?: string;
|
||||
};
|
||||
```
|
||||
|
||||
`redactedNodeSummary` returns a JSON-serializable array of `{name, type,
|
||||
host, port, hasCredentials}` with credentials omitted. This is what gets
|
||||
persisted in `last_nodes` for the operator UI.
|
||||
|
||||
## 9. Security
|
||||
|
||||
- **SSRF on `localCoreEndpoint`**: the only SSRF surface here is the local
|
||||
core endpoint (the subscription URL itself is operator-supplied). Allowed
|
||||
hosts: `127.0.0.1`, `::1`, `localhost`. Any other host is rejected at parse
|
||||
time with a `subscription_needs_core_endpoint_invalid` status.
|
||||
- **No outbound to operator-internal hosts** from a subscription URL. The URL
|
||||
fetch goes through Node's `fetch` (same trust model as the existing
|
||||
`proxyLatency` health checks and the provider ping tasks). The operator
|
||||
already trusts the URL by pasting it.
|
||||
- **Fail-closed**: if a subscription's proxy is dead but still bound to a
|
||||
scope, `hasBlockingProxyAssignment` returns true and traffic fails closed —
|
||||
matches existing policy for any pool proxy. The operator can always disable
|
||||
the subscription or remove the binding.
|
||||
- **No secret echo**: `last_nodes` is redacted; the UI never sends secrets
|
||||
back. `password` / `username` are stored encrypted at rest by the existing
|
||||
`proxy_registry` encryption path.
|
||||
- **No cross-tenant write**: the API routes are gated by `requireManagementAuth`
|
||||
(dashboard session OR a manage-scope API key). Per-API-key overrides are
|
||||
explicitly out-of-scope.
|
||||
|
||||
## 10. UI
|
||||
|
||||
A new sub-tab **"订阅代理"** in `dashboard / settings / 代理`, placed after
|
||||
"documentation". List view shows:
|
||||
|
||||
- Name + URL (truncated, with full URL in `title` attribute)
|
||||
- Status badge: `ok` / `error` / `empty`
|
||||
- Enabled switch (optimistic toggle)
|
||||
- Action buttons: edit / refresh / delete
|
||||
|
||||
The edit form has:
|
||||
|
||||
- Name (text, required)
|
||||
- URL (text, required, validated as URL)
|
||||
- Mode toggle (global / rule)
|
||||
- Provider multi-select (visible only in rule mode; populated from
|
||||
`/api/providers`)
|
||||
- Local core endpoint (text, optional; placeholder `socks5://127.0.0.1:2080`)
|
||||
- Update interval (number, default 60 minutes)
|
||||
- Enabled toggle
|
||||
|
||||
When `status === 'error'`, an inline warning banner shows `subscription.error`.
|
||||
When `status === 'ok'` and there are nodes that needed a local core, a soft
|
||||
warning banner shows which protocols were skipped.
|
||||
|
||||
## 11. Migration & rollout
|
||||
|
||||
1. New migration `131_proxy_subscriptions.sql` runs on first DB open after
|
||||
upgrade (auto-discovered by the existing migration runner).
|
||||
2. The migration is **idempotent**: `ALTER TABLE … ADD COLUMN …` against an
|
||||
already-migrated DB is a no-op in SQLite when wrapped in the runner's
|
||||
"ignore duplicate column" path. See the existing
|
||||
`040_oneproxy_proxy_fields.sql` and `093_proxy_enable_toggles.sql`
|
||||
precedents.
|
||||
3. No backfill: existing rows get `subscription_id = NULL`, which the
|
||||
service treats as "manual, not subscription-managed".
|
||||
4. UI hides the tab when there are zero subscriptions, but the API is always
|
||||
available — that's intentional, so headless operators can manage
|
||||
subscriptions via API only.
|
||||
|
||||
## 12. Auto-refresh
|
||||
|
||||
`startSubscriptionScheduler()` is idempotent and:
|
||||
|
||||
- Skips in the browser (`typeof window !== "undefined"`).
|
||||
- Skips under `NODE_ENV=test`.
|
||||
- Otherwise starts a 60s `setInterval` that:
|
||||
- Lists enabled subscriptions.
|
||||
- For each, computes `due = now - lastFetchedAt >= updateIntervalMinutes * 60_000`.
|
||||
- Calls `syncSubscription` for due ones, swallowing errors (logged).
|
||||
- The interval timer is `.unref()`'d so it never blocks process exit.
|
||||
|
||||
The scheduler is started on:
|
||||
- First `GET /api/v1/management/proxy-subscriptions` (dashboard open).
|
||||
- Any `syncSubscription` call (defensive — for CLI / automation paths that
|
||||
bypass the GET).
|
||||
|
||||
## 13. Testing strategy
|
||||
|
||||
`tests/unit/proxySubscription.parse.test.ts` — 7 pure-parser cases, no DB,
|
||||
runnable in <1s:
|
||||
|
||||
1. Clash YAML with `direct` (http) and `needsCore` (ss) nodes.
|
||||
2. Base64-wrapped URI list (decoded correctly).
|
||||
3. V2Ray JSON-array-of-URI (vmess / vless).
|
||||
4. Plain URI list (mixed protocols).
|
||||
5. Clash.Meta outbounds (socks5).
|
||||
6. Empty / unknown input → `nodes=[]`, `needsCore=[]`, parserWarnings filled.
|
||||
7. `redactedNodeSummary` strips credentials.
|
||||
|
||||
`tests/unit/proxySubscription.service.test.ts` — 4 integration tests using
|
||||
`process.env.DATA_DIR` + `core.resetDbInstance()`:
|
||||
|
||||
1. **Global**: create enabled global subscription → `syncSubscription` →
|
||||
verify pool rows in `proxy_registry` with `subscription_id` set →
|
||||
`resolveProxyForConnectionFromRegistry` returns one of those rows →
|
||||
`proxyEnabled` is true.
|
||||
2. **Rule**: create enabled rule subscription on provider P1 → verify only
|
||||
P1's scope is bound, P2's scope is untouched.
|
||||
3. **Fail-closed**: subscription fetch URL is unreachable → `status='error'`,
|
||||
pool is empty, but if pool ever had rows they are cleaned up;
|
||||
`hasBlockingProxyAssignment` returns false (no dead proxies in any scope).
|
||||
4. **Delete**: delete subscription → registry rows for that subscription are
|
||||
removed with `force: true` (manual deletions can't cascade-block it) →
|
||||
`proxyEnabled` recomputed.
|
||||
|
||||
Test runner command:
|
||||
|
||||
```bash
|
||||
node --import tsx/esm \
|
||||
--import ./open-sse/utils/setupPolyfill.ts \
|
||||
--import ./tests/_setup/isolateDataDir.ts \
|
||||
--test \
|
||||
tests/unit/proxySubscription.parse.test.ts \
|
||||
tests/unit/proxySubscription.service.test.ts
|
||||
```
|
||||
|
||||
## 14. Future work (NOT in v1)
|
||||
|
||||
- Per-API-key subscription overrides (multi-tenant; needs a `key_subscription_overrides` table).
|
||||
- Per-provider traffic rules with domain matchers (would slot into the existing `interceptionRules` table).
|
||||
- Latency-weighted rotation across subscription pools (we already have `ProxyRotationStrategy = "latency"`; just expose it in the UI).
|
||||
- Proxying the subscription fetch itself through a separate egress (so operators can fetch behind a corporate firewall).
|
||||
- Browser-side preview of a parsed subscription before saving (currently must save → wait → see nodes).
|
||||
|
||||
## 15. Files touched / added
|
||||
|
||||
**Added (new):**
|
||||
|
||||
- `src/lib/proxySubscription/parse.ts`
|
||||
- `src/lib/proxySubscription/subscriptionService.ts`
|
||||
- `src/lib/proxySubscription/index.ts`
|
||||
- `src/lib/db/migrations/131_proxy_subscriptions.sql`
|
||||
- `src/app/api/v1/management/proxy-subscriptions/route.ts`
|
||||
- `src/app/api/v1/management/proxy-subscriptions/[id]/route.ts`
|
||||
- `src/app/api/v1/management/proxy-subscriptions/[id]/refresh/route.ts`
|
||||
- `src/app/api/v1/management/proxy-subscriptions/[id]/nodes/route.ts`
|
||||
- `src/app/(dashboard)/dashboard/settings/components/proxy/SubscriptionTab.tsx`
|
||||
- `tests/unit/proxySubscription.parse.test.ts`
|
||||
- `tests/unit/proxySubscription.service.test.ts`
|
||||
- `docs/proxy-subscriptions.md` (this file)
|
||||
|
||||
**Modified (minimal):**
|
||||
|
||||
- `src/lib/db/proxies/types.ts` — `+ subscriptionId: string | null` on
|
||||
`ProxyRegistryRecord`; `+ subscriptionId?: string | null` on `ProxyPayload`.
|
||||
- `src/lib/db/proxies/mappers.ts` — `mapProxyRow` reads
|
||||
`subscription_id` from the row.
|
||||
- `src/lib/db/proxies.ts` — INSERT / UPDATE / SELECT add `subscription_id`.
|
||||
- `src/app/(dashboard)/dashboard/settings/components/ProxyTab.tsx` — adds
|
||||
one new sub-tab ("订阅代理") + the `literal` fallback for labels that
|
||||
aren't in the i18n catalog yet.
|
||||
@@ -1,113 +0,0 @@
|
||||
# Video Generation Through Preset Jobs
|
||||
|
||||
Custom provider nodes whose `/videos` surface is an **async submit → poll → fetch-result API** (instead of a synchronous generation endpoint) can be wired into the `/api/v1/videos/generations` route without any new provider code. The model row carries a `generationConfig.preset`, and the dispatcher routes the request through a single job executor that is configured entirely by declarative preset data.
|
||||
|
||||
## How dispatch works
|
||||
|
||||
1. The route parses `model` as `provider/model` and resolves the provider node's credentials (`POST /api/v1/videos/generations`).
|
||||
2. `handleVideoGeneration` (in `open-sse/handlers/videoGeneration.ts`) checks whether the provider is a **custom provider node** (no entry in the static video registry).
|
||||
3. For custom nodes it reads the custom model row via `getCustomModelVideoPreset(provider, model)`:
|
||||
- The model row has `generationConfig.preset` set (e.g. `"agnes-video-job"`) → dispatch through the **job executor** (`open-sse/handlers/videoGeneration/job.ts`).
|
||||
- The preset name does not match any known preset → **502** `Unknown video job preset: <preset>` (server-side misconfiguration).
|
||||
- No preset configured → fall back to the generic OpenAI-compatible sync handler, mirroring the images route.
|
||||
4. The job executor runs the preset pipeline: **submit** the job, **poll** for terminal status, **read** the finished video URL, and return the standard OpenAI-compatible response shape.
|
||||
|
||||
The executor is one handler family; every provider-specific detail (paths, auth, body shape, status/result fields, poll cadence) is data in the preset definition.
|
||||
|
||||
## Response contract
|
||||
|
||||
Both the sync and job paths return the same shape:
|
||||
|
||||
```json
|
||||
{
|
||||
"created": 1234567890,
|
||||
"data": [{ "url": "https://…", "format": "mp4" }]
|
||||
}
|
||||
```
|
||||
|
||||
This is the shape the media-generation consumer reads (`data.data[0].url`), so preset-job providers are drop-in replacements for sync providers.
|
||||
|
||||
## Presets
|
||||
|
||||
Presets live in `open-sse/handlers/videoGeneration/job.ts` (`VIDEO_JOB_PRESETS`). Each preset declares:
|
||||
|
||||
| Field | Meaning |
|
||||
| -------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
|
||||
| `authHeaderName` / `authScheme` | `x-api-key` with `raw` value (Agnes, muapi) or `Authorization` with `Bearer` prefix (Sora). Missing credentials → request goes out without an auth header. |
|
||||
| `baseUrlFallback` | Default base URL. Overridden by the provider connection's `providerSpecificData.baseUrl` (or top-level `baseUrl`), which wins when set. |
|
||||
| `submit.path` / `submit.buildBody` | Where and how the job is submitted. `{model}` in the path is substituted with the encoded model id; the body is built from `model`/`prompt`/`duration` plus pass-through of every other request field. |
|
||||
| `taskIdPath` | Dot path into the submit response identifying the job (e.g. `task_id`, `request_id`, `id`). Missing job id → **502**. |
|
||||
| `poll.pathTemplate` | Poll URL template; `{taskId}` is substituted. |
|
||||
| `statusPath` / `statusDone` / `statusFailed` | Where the job status lives and which values are terminal. |
|
||||
| `resultPath` | Dot path into the poll response holding the finished video URL: a string, a string array, or an array of `{ url }` objects are all accepted. Completed job with no readable URL → **502**. |
|
||||
| `maxPolls` / `pollIntervalMs` | Poll budget (default 60 polls × 2000 ms). Exhausted → **504** `Video job timed out`. |
|
||||
|
||||
### `agnes-video-job` — Agnes Video V2.0
|
||||
|
||||
- Auth: `x-api-key: <key>` (raw).
|
||||
- Base URL fallback: `https://apihub.agnes-ai.com`.
|
||||
- Submit: `POST /v1/videos` with `{ model, prompt, ...extras }` — image, mode, `num_frames`, `frame_rate` and other provider knobs pass through untouched.
|
||||
- Job id: `task_id` from the submit response.
|
||||
- Poll: `GET /v1/videos/{taskId}`; status at `status` (`completed` / `failed`).
|
||||
- Result: `metadata.url` — the completed video URL is returned as JSON metadata, not a binary body.
|
||||
|
||||
### `muapi-video-job` — muapi.ai
|
||||
|
||||
- Auth: `x-api-key: <key>` (raw).
|
||||
- Base URL fallback: `https://api.muapi.ai`.
|
||||
- Submit: `POST /api/v1/{model}` with `{ prompt, duration?, ...extras }`.
|
||||
- Job id: `request_id` from the submit response.
|
||||
- Poll: `GET /api/v1/predictions/{taskId}/result`; status at `status` (`completed` / `failed`).
|
||||
- Result: `outputs` — an array of video URLs.
|
||||
|
||||
### `sora-job` — OpenAI Sora
|
||||
|
||||
- Auth: `Authorization: Bearer <key>`.
|
||||
- Base URL fallback: `https://api.openai.com`.
|
||||
- Submit: `POST /v1/videos` with `{ model, prompt, seconds?, ...extras }`. `seconds` is a **string** enum (`"4" | "8" | "12"`) in the Sora API, so a numeric `duration` is stringified; size mapping is intentionally not forced.
|
||||
- Job id: `id` from the submit response.
|
||||
- Poll: `GET /v1/videos/{taskId}`; status at `status` (`completed` / `failed`).
|
||||
- Result: `data` — an array whose entries are either a URL string or `{ url: "…" }`.
|
||||
|
||||
## Setup
|
||||
|
||||
1. **Register the provider node** as an OpenAI-compatible custom provider (`providerSpecificData.baseUrl` optional — the preset's `baseUrlFallback` is used when absent).
|
||||
2. **Register a custom model** tagged with the `videos` endpoint and a `generationConfig`:
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "super-video-v1",
|
||||
"name": "Super Video v1",
|
||||
"source": "manual",
|
||||
"apiFormat": "chat-completions",
|
||||
"supportedEndpoints": ["videos"],
|
||||
"generationConfig": { "preset": "agnes-video-job" }
|
||||
}
|
||||
```
|
||||
|
||||
`addCustomModel` (in `src/lib/db/models.ts`) accepts `generationConfig?: { preset: string }` as its final parameter and persists it on the model row; `updateCustomModel` forwards it the same way. The provider-models API accepts `generationConfig` on create and update.
|
||||
|
||||
3. **Call the route** as usual:
|
||||
|
||||
```bash
|
||||
curl -X POST http://localhost:8787/api/v1/videos/generations \
|
||||
-H "Content-Type: application/json" \
|
||||
-H "Authorization: Bearer $API_KEY" \
|
||||
-d '{
|
||||
"model": "my-custom-provider/super-video-v1",
|
||||
"prompt": "a cat playing piano",
|
||||
"duration": 5
|
||||
}'
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
| Symptom | Cause |
|
||||
| ------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------ |
|
||||
| `400 Unknown video provider: …` | Non-custom provider not in the static registry; preset jobs only apply to custom provider nodes. |
|
||||
| `502 Unknown video job preset: …` | `generationConfig.preset` does not match any preset in `VIDEO_JOB_PRESETS`. Fix the model row. |
|
||||
| `502 Video provider did not return a job id (…)` | Submit succeeded but the response had no readable value at `taskIdPath`. |
|
||||
| `502 Video job failed (…)` / `Video job completed but no result URL found (…)` | Poll reached a terminal `statusFailed` state, or `resultPath` held no readable URL. |
|
||||
| `504 Video job timed out after 60 polls (…)` | Job never reached a terminal status within the poll budget. |
|
||||
| Upstream 4xx/5xx passthrough | `fetchJson` returns the upstream status when the submit/poll request itself is not OK. |
|
||||
| Requests go out without auth | No `apiKey`/`accessToken` on the provider connection; the executor sends `Content-Type` only. |
|
||||
@@ -1,89 +0,0 @@
|
||||
# OmniRoute Performance Audit — Phase 3 Report
|
||||
|
||||
## Measured data
|
||||
|
||||
| Metric | Value |
|
||||
|--------|-------|
|
||||
| Cold-start open-sse module load | **2,317ms** (first import) |
|
||||
| proxyFallback.ts import cost | **210ms** (SQLite init + undici re-import) |
|
||||
| proxyDispatcher.ts import cost | **69ms** |
|
||||
| Handlers/streaming code | **686ms** |
|
||||
| Services (token refresh, etc.) | **172ms** |
|
||||
| Provider registry (211 files, 1.7MB) | **<5ms** (per-file lazy) |
|
||||
| Provider constants lazy Proxy | **0.24ms** (first access) |
|
||||
| Provider models lazy Proxy | **0.17ms** (first access) |
|
||||
| Static provider imports (eager) | **~201 files** (module eval, ~200–500ms I/O) |
|
||||
| Executor singletons at module level | **42** |
|
||||
| Module-level `setInterval` timers | **24** (many NOT `unref()`-ed) |
|
||||
| Polyfill/global-patch operations | **5+** |
|
||||
| DB size | 1.4GB+, usage_history 250K+ rows |
|
||||
| SQLite cache_size | 16MB (conservative) |
|
||||
| mmap_size in settings | 256MB (never applied as PRAGMA — **now fixed**) |
|
||||
| Per-chunk transform layers | 2–5 `pipeThrough()` calls |
|
||||
| Chunk transform GC pressure | Moderate (structuredClone removed, TextDecoder lifted) |
|
||||
| Upstream HTTP | undici 3‑tier dispatcher (well‑pooled) |
|
||||
| Sync DB writes post-streaming | #1 bottleneck: saveRequestUsage + saveCallLog block event loop |
|
||||
|
||||
## Ranked findings (effort × impact)
|
||||
|
||||
### Implemented in this PR
|
||||
|
||||
| # | Finding | Impact | Effort | Fix |
|
||||
|---|---------|--------|--------|-----|
|
||||
| 1 | 🔴 **Proxy fallback loaded eagerly at startup** | **210ms** on first import | Low | Dynamic `import()` in proxyFetch.ts error handler |
|
||||
| 2 | 🔴 **egressCache memory leak** (never evicts) | HIGH — unbounded growth | Very Low | Lazy TTL cleanup on `getCachedEgressIp` |
|
||||
| 3 | 🔴 **Missing composite index: usage_history(provider, model, timestamp)** | HIGH — full scan on `getModelLatencyStats` | Very Low | `CREATE INDEX IF NOT EXISTS …` in schemaColumns.ts |
|
||||
| 4 | 🔴 **Missing composite index: provider_connections(provider, auth_type)** | HIGH — full scan on 6+ queries | Very Low | `CREATE INDEX IF NOT EXISTS …` in schemaColumns.ts |
|
||||
| 5 | 🔴 **mmap_size PRAGMA never applied** | HIGH — 256MB setting stored but unused | Very Low | PRAGMA applied after `applyStoredDatabaseOptimizationSettings` |
|
||||
|
||||
### Already in PR #7893 (pre-Phase 1)
|
||||
|
||||
| # | Finding | Impact | Effort |
|
||||
|---|---------|--------|--------|
|
||||
| 6 | 🔴 **Startup serialization** | 500+ms serial blocking (early imports + background services) | Low → wrapped in Promise.all / Promise.allSettled |
|
||||
| 7 | 🟡 **Per-chunk structuredClone in createSSEStream** | GC pressure on every chunk | Low → replaced with minimal object spread |
|
||||
| 8 | 🟡 **Per-chunk `new TextDecoder()` in progressTracker** | Minor GC churn | Very Low → module-level const |
|
||||
| 9 | 🟡 **P2C quota re-evaluated per comparison (exponential blowup)** | N² work on each pool filter | Medium → Map cache threaded through pipeline |
|
||||
| 10 | 🟡 **Dual `.filter()` passes in selectPoolSubset** | Double iteration on active set | Very Low → single `for` loop |
|
||||
| 11 | 🟢 **Debug-loop re-filters 6 function calls** | No-op in production | Very Low → Map-based string comparisons |
|
||||
| 12 | 🟢 **Backoff decay loop uses full CRUD update** | SELECT+encrypt+invalidate per unused connection | Low → targeted `resetConnectionBackoff` |
|
||||
| 13 | 🟢 **Lazy PROVIDERS/PROVIDER_MODELS** | Startup saving per lazy Proxy 0.2ms | Low → Proxy on constants.ts + providerModels.ts |
|
||||
| 14 | 🟢 **TextEncoder lift (claude-web.ts)** | Eliminates per-chunk instances | Low → module-level encoder |
|
||||
| 15 | 🟢 **13 route files `getSettings()` → `getCachedSettings()`** | Avoids redundant decrypts | Low → import swap |
|
||||
| 16 | 🟢 **settingsCache.ts dead file deletion** | Cleanup | Very Low → removed |
|
||||
|
||||
### Future opportunities (not yet implemented)
|
||||
|
||||
| # | Finding | Impact | Effort | Priority |
|
||||
|---|---------|--------|--------|----------|
|
||||
| 17 | 🔴 **`saveRequestUsage` dedup guard uses COALESCE on indexed columns** | FULL TABLE SCAN on every request completion | Medium | **NEXT** |
|
||||
| 18 | 🔴 **24 module-level `setInterval` timers (many NOT `unref()`-ed)** | Prevent process exit + 2μs/call overhead | Low | Soon |
|
||||
| 19 | 🔴 **`providerFallback.ts` (2nd path via proxyAutoSelector→transport→validation)** | 210ms but already lazy (route handlers only) | Low | Bonded |
|
||||
| 20 | 🟡 **Sync DB writes block event loop after every stream** | saveRequestUsage + saveCallLog serialize through single-writer lock | High | Candidate for worker_thread |
|
||||
| 21 | 🟡 **DB cache_size conservate (16MB)** | For 1.4GB DB, increases page reads | Very Low | PRAGMA change |
|
||||
| 22 | 🟡 **Enable Redis for auth cache + quota store** | Offloads SQLite read/write pressure | Low | Config change + doc |
|
||||
| 23 | 🟡 **DashboardLayout is `"use client"` with 7+ heavy children** | Entire dashboard forced to client render | High | Structural layout split |
|
||||
| 24 | 🟢 **mermaid (84MB unused in src/) in dependencies** | Install bloat, not server-side cost | Very Low | Move to devDeps |
|
||||
| 25 | 🟢 **3 duplicated deps in root + open-sse** | Redundant install | Very Low | Deduplicate |
|
||||
| 26 | 🟡 **`SELECT *` unbounded in `getUsageHistory` (admin API)** | Risks scan of 250K+ rows | Low | Add LIMIT |
|
||||
| 27 | 🟢 **Sync `readFileSync` at module eval in config loading** | Blocks event-loop-startup once | Very Low | Could defer |
|
||||
| 28 | 🟡 **SetInterval timers: confirm all `unref()`-ed for remaining** | ~12 without `unref()` prevent clean exit | Low | Audit + fix |
|
||||
|
||||
## Status summary
|
||||
|
||||
| Category | Status |
|
||||
|----------|--------|
|
||||
| PR #7893 (original 16 optimizations) | **OPEN** — all core changes verified |
|
||||
| Phase 1 tangible wins (5 items) | **Implemented** — uncommitted |
|
||||
| Phase 2 EventLoopHealth | **Completed** — hot path is clean, timers need `unref()` |
|
||||
| Phase 2 RequestTrace | **Not completed** (agent lost on session boundary) |
|
||||
| Phase 2 TransitiveDeps | **Not completed** (agent lost on session boundary) |
|
||||
| Phase 3 Report | **This document** |
|
||||
|
||||
## Recommended next actions
|
||||
|
||||
1. **Commit Phase 1 wins** (egressCache, mmap_size, indexes, proxyFallback lazy) → push to PR #7893
|
||||
2. **Complete #17** — fix `COALESCE` defeating index in `saveRequestUsage` dedup guard
|
||||
3. **Complete #18** — add `unref()` to all 24 module-level `setInterval` timers
|
||||
4. **Complete #21** — bump `cache_size` PRAGMA to 64-128MB
|
||||
5. **Document Redis configuration** for auth cache + quota store offload
|
||||
@@ -1,62 +0,0 @@
|
||||
# Quality Ratchet
|
||||
|
||||
| Métrica | Baseline | Atual | Status |
|
||||
| ----------------------------------------------------------------- | -------- | ----- | --------------------- |
|
||||
| eslintWarnings | 0 | 0 | ok |
|
||||
| eslintErrors | 0 | 0 | ok |
|
||||
| coverage.statements | 80.8 | — | SKIP (ausente) |
|
||||
| coverage.lines | 80.8 | — | SKIP (ausente) |
|
||||
| coverage.functions | 86.42 | — | SKIP (ausente) |
|
||||
| coverage.branches | 78.1 | — | SKIP (ausente) |
|
||||
| coverage.chatCore.lines | 72.45 | — | SKIP (ausente) |
|
||||
| coverage.combo.lines | 85.42 | — | SKIP (ausente) |
|
||||
| coverage.accountFallback.lines | 96.78 | — | SKIP (ausente) |
|
||||
| coverage.auth.lines | 92.55 | — | SKIP (ausente) |
|
||||
| coverage.routeGuard.lines | 98.73 | — | SKIP (ausente) |
|
||||
| coverage.error.lines | 92.13 | — | SKIP (ausente) |
|
||||
| coverage.publicCreds.lines | 99.07 | — | SKIP (ausente) |
|
||||
| coverage.circuitBreaker.lines | 95.09 | — | SKIP (ausente) |
|
||||
| openapiCoverage.pct | 38 | 38 | ok |
|
||||
| i18nUiCoverage.pct | 99 | 99 | ok |
|
||||
| deadExports | 227 | — | SKIP (dedicated gate) |
|
||||
| cognitiveComplexity | 1223 | — | SKIP (dedicated gate) |
|
||||
| typeCoveragePct | 92.17 | — | SKIP (dedicated gate) |
|
||||
| codeqlAlerts | 0 | — | SKIP (dedicated gate) |
|
||||
| secretFindings | 0 | — | SKIP (dedicated gate) |
|
||||
| zizmorFindings | 190 | — | SKIP (dedicated gate) |
|
||||
| vulnCount | 10 | — | SKIP (dedicated gate) |
|
||||
| bundleSize | 7666 | — | SKIP (dedicated gate) |
|
||||
| openapiBreaking | 0 | — | SKIP (dedicated gate) |
|
||||
| mutationScore.src/sse/services/auth.ts | 52.57 | — | SKIP (dedicated gate) |
|
||||
| mutationScore.open-sse/services/accountFallback.ts | 68.38 | — | SKIP (dedicated gate) |
|
||||
| mutationScore.src/server/authz/routeGuard.ts | 76.08 | — | SKIP (dedicated gate) |
|
||||
| mutationScore.src/shared/utils/circuitBreaker.ts | 56.94 | — | SKIP (dedicated gate) |
|
||||
| mutationScore.open-sse/utils/error.ts | 43.83 | — | SKIP (dedicated gate) |
|
||||
| mutationScore.open-sse/utils/publicCreds.ts | 59.76 | — | SKIP (dedicated gate) |
|
||||
| mutationScore.open-sse/services/combo/autoStrategy.ts | 41.33 | — | SKIP (dedicated gate) |
|
||||
| mutationScore.open-sse/services/combo/comboStructure.ts | 57.82 | — | SKIP (dedicated gate) |
|
||||
| mutationScore.open-sse/services/combo/validateQuality.ts | 61.33 | — | SKIP (dedicated gate) |
|
||||
| mutationScore.open-sse/services/combo/comboPredicates.ts | 56.62 | — | SKIP (dedicated gate) |
|
||||
| mutationScore.open-sse/services/combo/rrState.ts | 70.88 | — | SKIP (dedicated gate) |
|
||||
| mutationScore.open-sse/services/combo/shadowRouting.ts | 48 | — | SKIP (dedicated gate) |
|
||||
| mutationScore.open-sse/services/combo/targetSorters.ts | 68.3 | — | SKIP (dedicated gate) |
|
||||
| mutationScore.open-sse/services/combo/comboData.ts | 76.94 | — | SKIP (dedicated gate) |
|
||||
| mutationScore.open-sse/services/combo/quotaScoring.ts | 39.73 | — | SKIP (dedicated gate) |
|
||||
| mutationScore.open-sse/services/combo/quotaStrategies.ts | 50.3 | — | SKIP (dedicated gate) |
|
||||
| mutationScore.open-sse/handlers/chatCore/passthroughHelpers.ts | 80.89 | — | SKIP (dedicated gate) |
|
||||
| mutationScore.open-sse/handlers/chatCore/sanitization.ts | 70.15 | — | SKIP (dedicated gate) |
|
||||
| mutationScore.open-sse/handlers/chatCore/upstreamTimeouts.ts | 33 | — | SKIP (dedicated gate) |
|
||||
| mutationScore.open-sse/handlers/chatCore/comboContextCache.ts | 13.62 | — | SKIP (dedicated gate) |
|
||||
| mutationScore.open-sse/handlers/chatCore/idempotency.ts | 42.82 | — | SKIP (dedicated gate) |
|
||||
| mutationScore.open-sse/handlers/chatCore/responseHeaders.ts | 62.7 | — | SKIP (dedicated gate) |
|
||||
| mutationScore.open-sse/handlers/chatCore/executorHelpers.ts | 70.39 | — | SKIP (dedicated gate) |
|
||||
| mutationScore.open-sse/handlers/chatCore/memoryExtraction.ts | 62.06 | — | SKIP (dedicated gate) |
|
||||
| mutationScore.open-sse/handlers/chatCore/nonStreamingSse.ts | 72.82 | — | SKIP (dedicated gate) |
|
||||
| mutationScore.open-sse/handlers/chatCore/passthroughToolNames.ts | 66.42 | — | SKIP (dedicated gate) |
|
||||
| mutationScore.open-sse/handlers/chatCore/headers.ts | 94.29 | — | SKIP (dedicated gate) |
|
||||
| mutationScore.open-sse/handlers/chatCore/logTruncation.ts | 77.64 | — | SKIP (dedicated gate) |
|
||||
| mutationScore.open-sse/handlers/chatCore/memorySkillsInjection.ts | 13.49 | — | SKIP (dedicated gate) |
|
||||
| mutationScore.open-sse/handlers/chatCore/semanticCache.ts | 60.16 | — | SKIP (dedicated gate) |
|
||||
| mutationScore.open-sse/handlers/chatCore/telemetryHelpers.ts | 83.18 | — | SKIP (dedicated gate) |
|
||||
|
||||
**Sem regressões — gate OK.**
|
||||
@@ -1,29 +0,0 @@
|
||||
#!/bin/bash
|
||||
set -e
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
|
||||
PLUGIN_SRC="$(dirname "$SCRIPT_DIR")/obsidian-plugin"
|
||||
DESKTOP_VAULT="${1:-$HOME/Documents/Vault/Omniroute-Test}"
|
||||
MOBILE_VAULT="${2:-$HOME/Documents/Vault/Test}"
|
||||
|
||||
echo "Building plugin..."
|
||||
cd "$PLUGIN_SRC"
|
||||
npm run build 2>&1 | tail -3
|
||||
|
||||
echo "Installing to desktop vault: $DESKTOP_VAULT"
|
||||
mkdir -p "$DESKTOP_VAULT/.obsidian/plugins/omniroute-sync"
|
||||
cp "$PLUGIN_SRC/dist/main.js" "$DESKTOP_VAULT/.obsidian/plugins/omniroute-sync/"
|
||||
cp "$PLUGIN_SRC/manifest.json" "$DESKTOP_VAULT/.obsidian/plugins/omniroute-sync/"
|
||||
cp "$PLUGIN_SRC/styles.css" "$DESKTOP_VAULT/.obsidian/plugins/omniroute-sync/"
|
||||
echo " ✓ Desktop plugin installed"
|
||||
|
||||
if [ -d "$MOBILE_VAULT" ]; then
|
||||
echo "Installing to mobile vault: $MOBILE_VAULT"
|
||||
mkdir -p "$MOBILE_VAULT/.obsidian/plugins/omniroute-sync"
|
||||
cp "$PLUGIN_SRC/dist/main.js" "$MOBILE_VAULT/.obsidian/plugins/omniroute-sync/"
|
||||
cp "$PLUGIN_SRC/manifest.json" "$MOBILE_VAULT/.obsidian/plugins/omniroute-sync/"
|
||||
cp "$PLUGIN_SRC/styles.css" "$MOBILE_VAULT/.obsidian/plugins/omniroute-sync/"
|
||||
echo " ✓ Mobile plugin installed"
|
||||
fi
|
||||
|
||||
echo "Done! Restart Obsidian on both devices to load the plugin."
|
||||
Reference in New Issue
Block a user