mirror of
https://github.com/diegosouzapw/OmniRoute.git
synced 2026-07-26 09:52:11 +03:00
feat(issue-agent): surface RecordedTriageTimeoutError as 504 (#7315)
* chore(ci): add .mergify.yml to main — Mergify only reads config from the default branch (#7168)
* feat: scaffold issue agent and router eval provenance
* feat: wire recorded issue triage runner
* feat: ingest recorded issue context
* feat: persist issue agent audit log
* feat: import recorded github issue exports
* docs: document issue agent env toggle
* fix(issue-agent): validate run requests
* fix(ci): add the auto-enqueue pull_request_rule to the Mergify config (queue_conditions alone are eligibility-only) (#7179)
* fix: validate issue agent run requests
* docs: add issue agent execution traceability
* feat(issue-agent): route recorded triage through chat
* test(issue-agent): verify recorded triage through chat route
* docs(issue-agent): add executable triage session artifacts
* fix(ci): migrate Mergify auto-enqueue to merge_protections_settings.auto_merge_conditions (rules-based path is EOL 2026-07-16) (#7216)
* fix(ci): drop Mergify batch settings (batching is a paid-tier feature; free plan queue is serial) (#7220)
* feat(issue-agent): surface RecordedTriageTimeoutError as 504
When the recorded-triage chat completion times out, the AbortController
fires an AbortError that previously surfaced as a generic 400 to the
caller. This change:
* Adds a `RecordedTriageTimeoutError` that wraps the AbortError
with the timeoutMs context.
* Re-throws it from `executeRecordedTriageChatCompletion` so the
caller can distinguish timeouts from other failures.
* In the runs route, catches it and returns a 504 with code
`ISSUE_AGENT_TIMEOUT` so clients can render a useful error.
Tests:
* issue-agent-execution.test.ts — verifies the typed error
* issue-agent-route-execution.test.ts — covers timeout path
* issue-agent-runs-route.test.ts — verifies 504 mapping
* fix(ci): merge queue tolerates the advisory dast-smoke failure (its GH-hosted build hang dequeued every attempt) (#7225)
* test(ci): make the #6634 selfref guard hermetic — main's copy hard-fails every PR (#7341)
main's copy of this test still does git I/O inside a unit test:
const baseSrc = git(['show', 'origin/main:' + FILE]);
Runners check out a shallow single ref, so origin/main does not resolve and the
test dies with 'fatal: invalid object name origin/main'. Every PR into main
fails Unit Tests (7/8) on it — today that is #7313, #7315, #7316, #7334, #7336
and #7337, six PRs red on a defect none of them introduced. #7313 has no other
red at all.
release/v3.8.49 already carries a fix (2e42b8efc, #7174: try/catch, fetch
origin/main on demand, t.skip() when unreachable), but it only reaches main at
release time — so main stays broken for the whole cycle. Cherry-picking it would
also import a new problem: PR Test Policy classifies t.skip() as a silenced
assertion, which we watched it correctly catch on #7300 today.
This is the hermetic version instead (ported from #7327, which does the same for
the release branch): read the file straight off disk, compare against an empty
base so baseTaut/baseExtTaut are 0 — the strictest possible comparison point —
and call evaluateMasking() directly. No git ref, no fetch, no skip, nothing the
runner's checkout depth can break.
The #6634 regression stays covered: the guard's logic lives in
SELF_TEST_FIXTURE_RE (check-test-masking.mjs:337), not in the test. Proven both
ways on main before committing — neutralise SELF_TEST_FIXTURE_RE to /$^/ and
the test FAILS; restore it and it passes 2/2, with check-test-masking.mjs left
byte-identical.
Co-authored-by: growab <nekron@icloud.com>
* chore(quality): tighten main's coverage baseline to the CI's real numbers (#7347)
main's ratchet had been failing --require-tighten on every PR: 11 metrics
improved but the baseline was never tightened. Same class as the #6634
selfref guard — an infra fix that lands only on the release branch leaves
main red for the whole cycle, and every PR into main pays for it.
Values are the merged-coverage numbers from a run on main itself (a local
run measures ~68% vs CI's ~80%; the baseline's own note warns about that
gap). Only the 11 coverage values change — gitleaks and semgrepFindings
keep main's own state.
No changelog fragment: #7326 carries it on release/v3.8.49, and a second
one here would double the entry at release time.
* fix(issue-agent): dryRun-explicit test fixture + sanitize the generic error catch
Two independent Hard Rule #12/#18 fixes on the recorded-triage runs route:
1. tests/unit/issue-agent-route-execution.test.ts's "preserves the normal
chat route provider failure response" test omitted dryRun from its
request body. createRecordedTriageRun() computes `dryRun: input.dryRun
!== false`, so an omitted dryRun defaults to true (dry-run mode) and the
route returns the deterministic dry-run summary WITHOUT ever calling
executeRecordedTriageChatCompletion() -- the mocked 429 fetch was never
invoked, so the test always observed the 200 dry-run response instead.
Add the missing `dryRun: false`, matching the sibling test above it. The
normal chat-completions route also enriches upstream errors with a
"[provider/model] [status]:" prefix and a connection-cooldown hint
(RESILIENCE_GUIDE.md) rather than passing them through byte-for-byte, so
the assertion now checks the original message survives (status +
substring) instead of exact-matching the mocked JSON shape.
2. src/app/api/issue-agent/runs/route.ts's generic (non-timeout) catch
returned raw `error.message` straight to the client. Validation failures
thrown by this module (bad issue URL, malformed GitHub export) are safe,
curated messages -- but appendIssueAgentAuditRecord()'s mkdir/appendFile
under DATA_DIR can throw a real Node fs error (ENOENT/EACCES/EEXIST, ...)
whose raw `.message` embeds the server's absolute filesystem path. Add
isNodeSystemError() (keys off NodeJS.ErrnoException's `.code`, which only
Node's own fs/system errors set) to replace that class of error with a
generic message, and route everything else through sanitizeErrorMessage()
per Hard Rule #12. Add a regression test that forces a real audit-write
failure (pre-creating a file where audit.ts expects to mkdir) and asserts
the response contains neither the errno code nor the DATA_DIR path.
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
---------
Co-authored-by: Diego Rodrigues de Sa e Souza <8016841+diegosouzapw@users.noreply.github.com>
Co-authored-by: KooshaPari <koosha@example.com>
Co-authored-by: growab <nekron@icloud.com>
This commit is contained in:
@@ -649,6 +649,10 @@ NEXT_PUBLIC_ENABLE_SOCKS5_PROXY=true
|
||||
# Legacy alias for OMNIROUTE_API_KEY.
|
||||
# ROUTER_API_KEY=
|
||||
|
||||
# Enable the offline/local Issue Agent recorded-triage endpoint.
|
||||
# Used by: src/app/api/issue-agent/runs/route.ts. Default: disabled.
|
||||
# OMNIROUTE_ISSUE_AGENT_ENABLED=false
|
||||
|
||||
# CLI remote-mode context/profile for `omniroute` commands (overrides the active
|
||||
# context in the local contexts store). Equivalent to the `--context <name>` flag.
|
||||
# Used by: bin/cli/program.mjs, bin/cli/api.mjs (remote mode).
|
||||
|
||||
@@ -411,6 +411,7 @@ detection above).
|
||||
| `OMNIROUTE_API_KEY` | _(unset)_ | MCP/A2A modules | API key for internal MCP tool and A2A skill calls. |
|
||||
| `OMNIROUTE_API_KEY_ID` | _(unset)_ | `open-sse/mcp-server/audit.ts` | Key ID for MCP audit log attribution. |
|
||||
| `ROUTER_API_KEY` | _(unset)_ | Legacy | Legacy alias for `OMNIROUTE_API_KEY`. |
|
||||
| `OMNIROUTE_ISSUE_AGENT_ENABLED` | `false` | `src/app/api/issue-agent/runs/route.ts` | Enables the offline/local Issue Agent recorded-triage endpoint. Leave disabled unless explicitly running local recorded-triage workflows. |
|
||||
| `OMNIROUTE_CONTEXT` | _(active context)_ | `bin/cli/program.mjs`, `bin/cli/api.mjs` | CLI remote-mode context/profile for `omniroute` commands; overrides the active context in the local contexts store. Equivalent to `--context <name>`. |
|
||||
| `OMNIROUTE_MCP_ENFORCE_SCOPES` | `true` | `open-sse/mcp-server/server.ts` | Enforce scope-based access control on MCP tool calls. |
|
||||
| `OMNIROUTE_MCP_SCOPES` | _(all)_ | `open-sse/mcp-server/server.ts` | Comma-separated scopes: `admin`, `combos`, `health`, `models`, `routing`, `budget`, `metrics`, `pricing`, `memory`, `skills`. |
|
||||
|
||||
@@ -0,0 +1,36 @@
|
||||
# Issue-Agent Executable Triage: Session Overview
|
||||
|
||||
Machine status: `in_progress`
|
||||
Updated at: `2026-07-14`
|
||||
Issue: `https://github.com/diegosouzapw/OmniRoute/issues/5980`
|
||||
PR: `https://github.com/diegosouzapw/OmniRoute/pull/7002`
|
||||
|
||||
## Goal
|
||||
|
||||
Deliver GitHub issue #5980 as a production issue-agent workflow. The workflow
|
||||
must execute recorded GitHub triage through OmniRoute routing, persist a complete
|
||||
audit trail, return an actionable result, and cover all terminal outcomes.
|
||||
|
||||
## Current State
|
||||
|
||||
| artifact_id | requirement | status | current evidence | next proof |
|
||||
| ----------- | ---------------------------------------------------------------------- | -------------------------------- | ---------------------------------------------------------------------------------------------------- | ------------------------------------------------------------ |
|
||||
| AC1 | configured provider/model/policy use normal chat routing | `implemented_pending_acceptance` | `623e0d541`, `fa2c1d7c6`; real-route test invokes the issue-agent route and mocks only provider HTTP | prove routing-policy semantics and terminal failure handling |
|
||||
| AC2 | persist lifecycle, request, output, usage/cost/runtime, terminal error | `not_started` | audit JSONL currently records only pre-execution run context | lifecycle persistence tests |
|
||||
| AC3 | return actionable triage result | `not_started` | route forwards raw completion body | result contract and integration test |
|
||||
| AC4 | success, provider failure, timeout, budget stop | `not_started` | only success-route coverage exists | terminal-outcome test matrix |
|
||||
| release | CI/review evidence | `in_progress` | route-validation and focused tests have prior passing evidence | rerun final gates on PR head |
|
||||
|
||||
## Decisions
|
||||
|
||||
| decision_id | decision | rationale | status |
|
||||
| ----------- | -------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------- | ------------- |
|
||||
| DEC-001 | Use the in-process `POST` export from `/api/v1/chat/completions` | preserves existing admission, initialization, guardrails, and provider routing | `implemented` |
|
||||
| DEC-002 | Keep issue-agent execution opt-in with `OMNIROUTE_ISSUE_AGENT_ENABLED=true` | prevents unrequested autonomous execution | `implemented` |
|
||||
| DEC-003 | Treat AC1 as incomplete until policy and error semantics are verified end-to-end | request construction alone does not prove the chat route consumes the policy or returns correct terminal state | `active` |
|
||||
|
||||
## Traceability
|
||||
|
||||
The canonical WBS is `03_DAG_WBS.md`; the canonical QA matrix is
|
||||
`06_TESTING_STRATEGY.md`. Every status change must identify its commit SHA,
|
||||
exact command, observed result, and PR head.
|
||||
@@ -0,0 +1,28 @@
|
||||
# Issue-Agent Executable Triage: Research
|
||||
|
||||
Machine status: `complete_for_current_phase`
|
||||
|
||||
## In-Repository Findings
|
||||
|
||||
| research_id | source | finding | consequence |
|
||||
| ----------- | ------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------- |
|
||||
| RES-001 | `src/app/api/issue-agent/runs/route.ts` | the endpoint validates body, rejects unsupported mode/disabled execution, builds recorded context, writes audit JSONL, then delegates non-dry runs | execution behavior is centralized at the issue-agent route |
|
||||
| RES-002 | `src/app/api/v1/chat/completions/route.ts` | standard chat entrypoint exports `POST` and owns the normal chat request path | AC1 must exercise this export rather than a fake internal seam |
|
||||
| RES-003 | `src/lib/issueAgent/execution.ts` | provider and model are resolved into the chat request; policy is only encoded as `X-OmniRoute-Mode` | an implementation review must establish that this header is a consumed routing-policy contract |
|
||||
| RES-004 | `src/lib/issueAgent/audit.ts` | audit persistence occurs before execution and writes run context/steps only | AC2 is unsatisfied: no transition, completion, usage/cost/runtime, or terminal-error record exists |
|
||||
| RES-005 | `tests/unit/issue-agent-route-execution.test.ts` | live route test initializes isolated DB, calls the actual issue-agent `POST`, and mocks only `globalThis.fetch` at provider boundary | strong AC1 path evidence, but it verifies success only and does not prove policy consumption |
|
||||
|
||||
## Validation Evidence
|
||||
|
||||
| evidence_id | command | observed | scope | evidence_sha |
|
||||
| ----------- | -------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------- | ----------------------------------------------------- | ------------ |
|
||||
| EVD-001 | `bun test tests/unit/issue-agent-execution.test.ts tests/unit/issue-agent-route-execution.test.ts tests/unit/issue-agent-runs-route.test.ts` | prior focused run reported green | AC1 focused path | `fa2c1d7c6` |
|
||||
| EVD-002 | `npm run check:route-validation:t06` | prior run reported pass | request route validation | `e6a63eb33` |
|
||||
| EVD-003 | `npm run typecheck:core` | unresolved `omniglyph` declarations outside issue-agent paths | release gate blocked by pre-existing unrelated errors | pre-existing |
|
||||
|
||||
## Research Conclusions
|
||||
|
||||
The normal chat route is correctly selected as the AC1 integration seam. The
|
||||
remaining design work must use a persisted run-lifecycle model rather than
|
||||
extending the pre-execution JSONL row. No external API research was needed:
|
||||
the implementation uses existing in-repository routes and provider adapters.
|
||||
@@ -0,0 +1,43 @@
|
||||
# Issue-Agent Executable Triage: Specifications
|
||||
|
||||
Machine status: `in_progress`
|
||||
|
||||
## Acceptance Contract
|
||||
|
||||
| ac_id | requirement | acceptance evidence | status |
|
||||
| ----- | --------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------- | -------------------------------- |
|
||||
| AC1 | Non-dry recorded triage executes through normal chat routing with selected provider, model, and policy | actual issue-agent route reaches chat `POST`; provider-boundary mock observes selected target; policy is proven consumed by routing | `implemented_pending_acceptance` |
|
||||
| AC2 | Persist `accepted`, `running`, and terminal state plus sanitized request/prompt, model output, usage, cost, runtime, and terminal error | durable queryable record contains each field for success and failures | `pending` |
|
||||
| AC3 | API returns a useful, structured triage result derived from model output | response has stable triage schema and is not a raw opaque provider payload | `pending` |
|
||||
| AC4 | Tests cover success, provider/model failure, timeout, and budget stop | each outcome asserts HTTP response and persisted terminal record | `pending` |
|
||||
|
||||
## API Contract (Target)
|
||||
|
||||
| field | rule |
|
||||
| ------------------- | -------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `mode` | must be `recorded-triage` |
|
||||
| execution selection | accepts configured `provider`, `model`, `routingPolicy`, and bounded `timeoutMs` |
|
||||
| `runId` | stable execution identifier returned for every accepted run |
|
||||
| result | includes structured triage decision/summary/actions and execution metadata |
|
||||
| errors | return sanitized terminal error with explicit terminal status; never leak provider credentials or unredacted issue content |
|
||||
|
||||
## Persistence Contract (Target)
|
||||
|
||||
| field group | required values |
|
||||
| -------------- | --------------------------------------------------------------------------------------------------------- |
|
||||
| identity | run ID, issue URL/repository/number, mode, timestamps |
|
||||
| lifecycle | `accepted`, `running`, `succeeded`, `failed`, `timed_out`, or `budget_stopped` with transition timestamps |
|
||||
| input | redacted recorded context and rendered prompt fingerprint/content according to retention policy |
|
||||
| routing | requested provider/model/policy and resolved execution target |
|
||||
| output | sanitized model output and structured triage result |
|
||||
| accounting | input/output/total tokens, cost, and runtime when available |
|
||||
| terminal error | normalized code/message for failure, timeout, and budget stop |
|
||||
|
||||
## Assumptions, Risks, Uncertainties
|
||||
|
||||
| aru_id | type | statement | mitigation | status |
|
||||
| ------- | ----------- | ---------------------------------------------------------------------------------------- | ------------------------------------------------------------------------- | ------ |
|
||||
| ARU-001 | risk | `X-OmniRoute-Mode` may not be a consumed routing-policy input in the chat route | trace the policy contract and test an observable policy effect | `open` |
|
||||
| ARU-002 | risk | current catch maps all thrown execution errors to HTTP 400 and does not persist them | introduce typed terminal outcomes and persistence before response mapping | `open` |
|
||||
| ARU-003 | risk | current audit row is emitted before execution and cannot represent final execution state | replace/extend with append-only lifecycle records or durable run storage | `open` |
|
||||
| ARU-004 | uncertainty | provider response metadata may differ by adapter | normalize accounting fields and preserve unknowns explicitly | `open` |
|
||||
@@ -0,0 +1,25 @@
|
||||
# Issue-Agent Executable Triage: DAG and WBS
|
||||
|
||||
Machine status: `in_progress`
|
||||
|
||||
| id | phase | acceptance criterion | status | source paths | test paths | evidence_sha | depends_on |
|
||||
| ------- | ----------- | ---------------------------------------------------------------- | -------- | ----------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------ | --------------------------- | ------------------------- |
|
||||
| WBS-001 | contract | AC1-AC4 | complete | `src/app/api/issue-agent/runs/route.ts` | `tests/unit/issue-agent-runs-route.test.ts` | `a4378a26d` | - |
|
||||
| WBS-002 | execution | AC1: execute through normal chat-completions routing/policy seam | pending | `src/app/api/issue-agent/runs/route.ts`; `src/app/api/v1/chat/completions/route.ts` | `tests/unit/issue-agent-runs-route.test.ts` | `e6a` (reconciled baseline) | WBS-001 |
|
||||
| WBS-003 | persistence | AC2: persist lifecycle, input, output, usage, and terminal error | pending | `src/lib/issueAgent/*`; `src/app/api/issue-agent/runs/route.ts` | `tests/unit/issue-agent-audit.test.ts`; `tests/unit/issue-agent-runner.test.ts` | `e6a` (reconciled baseline) | WBS-002 |
|
||||
| WBS-004 | result | AC3: return an actionable triage result from execution | pending | `src/lib/issueAgent/*`; `src/app/api/issue-agent/runs/route.ts` | `tests/unit/issue-agent-runner.test.ts`; `tests/unit/issue-agent-runs-route.test.ts` | `e6a` (reconciled baseline) | WBS-002, WBS-003 |
|
||||
| WBS-005 | acceptance | AC4: cover success, provider failure, timeout, and budget stop | pending | `src/lib/issueAgent/*` | `tests/unit/issue-agent-*.test.ts` | `e6a` (reconciled baseline) | WBS-002, WBS-003, WBS-004 |
|
||||
| WBS-006 | release | PR validation and maintainer review | pending | `.github/workflows/*` | CI checks | `a4378a26d` | WBS-005 |
|
||||
|
||||
## Dependency Graph
|
||||
|
||||
`WBS-001 -> WBS-002 -> WBS-003 -> WBS-004 -> WBS-005 -> WBS-006`
|
||||
|
||||
`a4378a26d` is a prerequisite validation repair: it validates the issue-agent request body through the shared route validator and passes `npm run check:route-validation:t06` (535 routes). It does not satisfy AC1-AC4.
|
||||
|
||||
## Machine Evidence Contract
|
||||
|
||||
Every WBS item must maintain: `id`, `acceptance_criterion`, `status`, `source_paths`, `test_paths`, `command`, `expected`, `observed`, `evidence_sha`, `updated_at`, and `pr_url`.
|
||||
|
||||
PR: `https://github.com/diegosouzapw/OmniRoute/pull/7002`
|
||||
Issue: `https://github.com/diegosouzapw/OmniRoute/issues/5980`
|
||||
@@ -0,0 +1,37 @@
|
||||
# Issue-Agent Executable Triage: Implementation Strategy
|
||||
|
||||
Machine status: `in_progress`
|
||||
|
||||
## Phase Plan
|
||||
|
||||
| phase | work package | dependency | exit evidence | status |
|
||||
| ----- | ----------------------------------------------------------- | ----------------- | ----------------------------------------------------------------------- | ------------- |
|
||||
| P1 | verify/finish routing-policy contract and failure semantics | existing AC1 seam | actual chat route test proves policy consumption and non-2xx mapping | `in_progress` |
|
||||
| P2 | introduce durable execution lifecycle persistence | P1 | records transitions, request/prompt, output, accounting, terminal error | `pending` |
|
||||
| P3 | normalize actionable triage result | P2 | stable API result schema derived from completion | `pending` |
|
||||
| P4 | implement terminal outcome controls | P2 | provider failure, timeout, budget stop transition tests | `pending` |
|
||||
| P5 | release validation and PR review | P1-P4 | focused tests, route gate, relevant typecheck/CI evidence | `pending` |
|
||||
|
||||
## Architecture
|
||||
|
||||
1. Keep `src/app/api/issue-agent/runs/route.ts` as the API adapter: validation,
|
||||
feature gate, and response formatting only.
|
||||
2. Keep the standard chat `POST` as the routing boundary; do not add a parallel
|
||||
provider invocation path.
|
||||
3. Extract lifecycle persistence and result normalization into focused
|
||||
`src/lib/issueAgent/` modules. Do not overload the existing pre-execution audit
|
||||
writer with unrelated transport behavior.
|
||||
4. Use typed execution outcomes so provider failure, abort/timeout, and budget
|
||||
termination are distinguishable before HTTP mapping and persistence.
|
||||
5. Add tests from the actual route down to a mocked external provider boundary;
|
||||
use unit tests for pure normalization and lifecycle state transitions.
|
||||
|
||||
## Quality Controls
|
||||
|
||||
| control | command or review | threshold |
|
||||
| ------------------ | ----------------------------------------------------- | ---------------------------------------------------------- |
|
||||
| route contract | `npm run check:route-validation:t06` | pass |
|
||||
| AC1 route behavior | focused `bun test` issue-agent route/execution suites | policy and provider/model assertions pass |
|
||||
| AC2-AC4 | lifecycle/result/terminal-outcome suites | all required states persist and API matches |
|
||||
| static safety | `npm run typecheck:core` | distinguish new failures from existing `omniglyph` blocker |
|
||||
| patch integrity | `git diff --check origin/main...HEAD` | pass |
|
||||
@@ -0,0 +1,21 @@
|
||||
# Issue-Agent Executable Triage: Known Issues
|
||||
|
||||
Machine status: `open`
|
||||
|
||||
| issue_id | severity | status | evidence | impact | resolution owner |
|
||||
| -------- | -------- | ------ | ------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------- | ------------------------- |
|
||||
| KI-001 | P1 | `open` | `execution.ts` places `routingPolicy` in `X-OmniRoute-Mode`; the researched chat route has no observed consumer in the AC1 path | AC1 does not yet prove configured routing policy affects routing | AC1 implementation/review |
|
||||
| KI-002 | P1 | `open` | issue-agent route catches execution errors and returns `{ error }` with HTTP 400 after writing only pre-execution audit | provider failure, timeout, and budget stop lack correct terminal semantics and persistence | AC2/AC4 implementation |
|
||||
| KI-003 | P1 | `open` | `audit.ts` serializes only run context/steps before execution | AC2 fields for lifecycle, prompt, output, token/cost/runtime, and error are missing | AC2 implementation |
|
||||
| KI-004 | P1 | `open` | API returns raw `completion.body` | AC3 has no stable actionable triage result contract | AC3 implementation |
|
||||
| KI-005 | P2 | `open` | `npm run typecheck:core` has unresolved `omniglyph` declarations in `open-sse/services/compression/*` | full typecheck cannot be used as issue-agent completion evidence until separately resolved or excluded with provenance | release validation |
|
||||
|
||||
## Resolved/Verified
|
||||
|
||||
| issue_id | status | evidence |
|
||||
| -------- | ---------- | ----------------------------------------------------------------------------------------------------------------------------- |
|
||||
| KI-R001 | `verified` | `fa2c1d7c6` adds an isolated test that invokes the actual issue-agent route and mocks only provider HTTP for the success path |
|
||||
| KI-R002 | `verified` | `e6a63eb33` applies shared request-body validation to the issue-agent route; prior route-validation gate passed |
|
||||
|
||||
No workaround in this document changes the acceptance contract. Open P1 items
|
||||
block declaring AC1-AC4 complete.
|
||||
@@ -0,0 +1,25 @@
|
||||
# Issue-Agent Executable Triage: Testing Strategy
|
||||
|
||||
Machine status: `in_progress`
|
||||
|
||||
## QA Matrix
|
||||
|
||||
| qa_id | AC | scenario | command | expected | observed | status | evidence_sha |
|
||||
| ------ | ------------ | ------------------------------------------------------------------- | ------------------------------------------------------------------------------------------ | ----------------------------------------------------- | --------------------------------------- | ------------- | ------------ |
|
||||
| QA-001 | prerequisite | request schema validation | `npm run check:route-validation:t06` | all routes pass | 535 routes scanned; pass | pass | `a4378a26d` |
|
||||
| QA-002 | AC1 | selected provider/model/policy reaches normal chat-completions seam | `bun test tests/unit/issue-agent-runs-route.test.ts` | captured request uses configured routing inputs | not implemented | pending | `e6a` |
|
||||
| QA-003 | AC2 | run lifecycle persists input, output, usage, terminal error | `bun test tests/unit/issue-agent-audit.test.ts tests/unit/issue-agent-runner.test.ts` | durable records for every terminal state | not implemented | pending | `e6a` |
|
||||
| QA-004 | AC3 | successful execution returns actionable triage output | `bun test tests/unit/issue-agent-runner.test.ts tests/unit/issue-agent-runs-route.test.ts` | output derives from routed execution, not placeholder | not implemented | pending | `e6a` |
|
||||
| QA-005 | AC4 | provider/model failure | `bun test tests/unit/issue-agent-runner.test.ts` | failed lifecycle and sanitized error persisted | missing coverage | pending | `e6a` |
|
||||
| QA-006 | AC4 | timeout | `bun test tests/unit/issue-agent-runner.test.ts` | timed-out lifecycle and terminal error persisted | missing coverage | pending | `e6a` |
|
||||
| QA-007 | AC4 | budget stop | `bun test tests/unit/issue-agent-runner.test.ts` | budget stop is explicit and persisted | missing coverage | pending | `e6a` |
|
||||
| QA-008 | release | core type safety | `npm run typecheck:core` | pass | pending rerun after dependency recovery | pending | `e6a` |
|
||||
| QA-009 | release | whitespace integrity | `git diff --check origin/main...HEAD` | no errors | passed before remote rewrite | pass/reverify | `a4378a26d` |
|
||||
|
||||
## Test Rules
|
||||
|
||||
Tests must mock only the external provider boundary. AC1 must exercise the in-process `POST` export from `src/app/api/v1/chat/completions/route.ts` so admission, policy, translator initialization, and routing remain in the execution path. Each terminal outcome asserts both API behavior and persisted audit state.
|
||||
|
||||
## Evidence Requirements
|
||||
|
||||
Before a WBS item is marked complete, record the exact command output, commit SHA, test identifiers, and whether the test environment had a lockfile-compatible dependency set. The current recovered environment has incomplete dependencies due to `npm ci` disk exhaustion; no pending test may be reported as passing until rerun.
|
||||
141
src/app/api/issue-agent/runs/route.ts
Normal file
141
src/app/api/issue-agent/runs/route.ts
Normal file
@@ -0,0 +1,141 @@
|
||||
import { NextResponse } from "next/server";
|
||||
import { z } from "zod";
|
||||
import { appendIssueAgentAuditRecord } from "@/lib/issueAgent/audit";
|
||||
import {
|
||||
executeRecordedTriageChatCompletion,
|
||||
RecordedTriageTimeoutError,
|
||||
} from "@/lib/issueAgent/execution";
|
||||
import { normalizeGitHubIssueExport } from "@/lib/issueAgent/githubExport";
|
||||
import { createRecordedTriageRun } from "@/lib/issueAgent/recordedTriage";
|
||||
import { POST as postChatCompletion } from "@/app/api/v1/chat/completions/route";
|
||||
import { isValidationFailure, validateBody } from "@/shared/validation/helpers";
|
||||
import { sanitizeErrorMessage } from "@omniroute/open-sse/utils/error";
|
||||
|
||||
const issueAgentRunRequestSchema = z.object({
|
||||
mode: z.string().optional(),
|
||||
issueUrl: z.string().optional(),
|
||||
dryRun: z.boolean().optional(),
|
||||
model: z.string().min(1).max(256).optional(),
|
||||
provider: z.string().min(1).max(128).optional(),
|
||||
routingPolicy: z.string().min(1).max(128).optional(),
|
||||
timeoutMs: z.number().int().min(1).max(120_000).optional(),
|
||||
recordedContext: z.unknown().optional(),
|
||||
githubExport: z.unknown().optional(),
|
||||
});
|
||||
|
||||
const ENABLED_VALUES = new Set(["1", "true", "yes", "on"]);
|
||||
|
||||
function isIssueAgentEnabled(): boolean {
|
||||
return ENABLED_VALUES.has((process.env.OMNIROUTE_ISSUE_AGENT_ENABLED ?? "").toLowerCase());
|
||||
}
|
||||
|
||||
/**
|
||||
* Node's own fs/system errors (ENOENT, EACCES, EEXIST, ...) always set `.code`
|
||||
* to an uppercase errno identifier and embed the absolute path in `.message`
|
||||
* (e.g. `appendIssueAgentAuditRecord`'s mkdir/appendFile on DATA_DIR). Hand-thrown
|
||||
* validation errors in this module (createRecordedTriageRun,
|
||||
* normalizeGitHubIssueExport) are plain `new Error("...")` with a curated,
|
||||
* path-free message and never set `.code` — so this check safely distinguishes
|
||||
* "safe to show the client" validation failures from opaque internal/system
|
||||
* errors that could otherwise leak the server's filesystem layout.
|
||||
*/
|
||||
function isNodeSystemError(error: unknown): error is NodeJS.ErrnoException {
|
||||
return (
|
||||
error instanceof Error &&
|
||||
typeof (error as NodeJS.ErrnoException).code === "string" &&
|
||||
/^[A-Z]/.test((error as NodeJS.ErrnoException).code as string)
|
||||
);
|
||||
}
|
||||
|
||||
export async function GET() {
|
||||
return NextResponse.json({
|
||||
ok: true,
|
||||
enabled: isIssueAgentEnabled(),
|
||||
supportedModes: ["recorded-triage"],
|
||||
execution: "disabled-by-default",
|
||||
});
|
||||
}
|
||||
|
||||
export async function POST(request: Request) {
|
||||
let body: unknown;
|
||||
try {
|
||||
body = await request.json();
|
||||
} catch {
|
||||
return NextResponse.json({ error: "Invalid JSON body" }, { status: 400 });
|
||||
}
|
||||
|
||||
const validation = validateBody(issueAgentRunRequestSchema, body);
|
||||
if (isValidationFailure(validation)) {
|
||||
return NextResponse.json({ error: validation.error }, { status: 400 });
|
||||
}
|
||||
|
||||
const parsed = validation.data;
|
||||
if (parsed.mode !== "recorded-triage") {
|
||||
return NextResponse.json(
|
||||
{ error: "Unsupported issue-agent mode", supportedModes: ["recorded-triage"] },
|
||||
{ status: 400 }
|
||||
);
|
||||
}
|
||||
|
||||
if (!isIssueAgentEnabled()) {
|
||||
return NextResponse.json(
|
||||
{
|
||||
error: "Issue Agent execution is disabled",
|
||||
enabled: false,
|
||||
requiredEnv: "OMNIROUTE_ISSUE_AGENT_ENABLED=true",
|
||||
},
|
||||
{ status: 403 }
|
||||
);
|
||||
}
|
||||
|
||||
try {
|
||||
const normalized = parsed.githubExport ? normalizeGitHubIssueExport(parsed.githubExport) : null;
|
||||
const run = createRecordedTriageRun({
|
||||
...parsed,
|
||||
issueUrl: parsed.issueUrl ?? normalized?.issueUrl,
|
||||
recordedContext: parsed.recordedContext ?? normalized?.recordedContext,
|
||||
});
|
||||
const audit = await appendIssueAgentAuditRecord(run);
|
||||
if (run.dryRun) {
|
||||
return NextResponse.json({ ...run, auditPath: audit.path });
|
||||
}
|
||||
|
||||
const completion = await executeRecordedTriageChatCompletion(
|
||||
{
|
||||
run,
|
||||
model: parsed.model,
|
||||
provider: parsed.provider,
|
||||
routingPolicy: parsed.routingPolicy,
|
||||
timeoutMs: parsed.timeoutMs,
|
||||
},
|
||||
postChatCompletion
|
||||
);
|
||||
return NextResponse.json(
|
||||
{
|
||||
...run,
|
||||
runner: "omniroute-chat-completions",
|
||||
auditPath: audit.path,
|
||||
completion: completion.body,
|
||||
},
|
||||
{ status: completion.status }
|
||||
);
|
||||
} catch (error) {
|
||||
if (error instanceof RecordedTriageTimeoutError) {
|
||||
return NextResponse.json(
|
||||
{ error: sanitizeErrorMessage(error.message), code: "ISSUE_AGENT_TIMEOUT" },
|
||||
{ status: 504 }
|
||||
);
|
||||
}
|
||||
// Hard Rule #12: never put a raw err.message in a response. Validation
|
||||
// failures thrown by this module (bad issue URL, malformed GitHub export)
|
||||
// are safe, curated messages meant for the client — sanitizeErrorMessage()
|
||||
// is a no-op for them. An opaque Node system/fs error (e.g. audit.ts
|
||||
// failing to mkdir/appendFile under DATA_DIR) is replaced with a generic
|
||||
// message instead, since its raw text would otherwise embed the server's
|
||||
// absolute filesystem path.
|
||||
const message = isNodeSystemError(error)
|
||||
? "Issue Agent request failed due to an internal error"
|
||||
: sanitizeErrorMessage(error instanceof Error ? error.message : "Invalid issue-agent request");
|
||||
return NextResponse.json({ error: message }, { status: 400 });
|
||||
}
|
||||
}
|
||||
43
src/lib/issueAgent/audit.ts
Normal file
43
src/lib/issueAgent/audit.ts
Normal file
@@ -0,0 +1,43 @@
|
||||
import { mkdir, appendFile } from "node:fs/promises";
|
||||
import { join } from "node:path";
|
||||
import { homedir } from "node:os";
|
||||
|
||||
import type { RecordedTriageRun } from "./recordedTriage";
|
||||
|
||||
export interface IssueAgentAuditOptions {
|
||||
dataDir?: string;
|
||||
now?: Date;
|
||||
}
|
||||
|
||||
export interface IssueAgentAuditResult {
|
||||
path: string;
|
||||
}
|
||||
|
||||
function defaultDataDir(): string {
|
||||
return process.env.DATA_DIR || join(homedir(), ".omniroute");
|
||||
}
|
||||
|
||||
export async function appendIssueAgentAuditRecord(
|
||||
run: RecordedTriageRun,
|
||||
options: IssueAgentAuditOptions = {}
|
||||
): Promise<IssueAgentAuditResult> {
|
||||
const dataDir = options.dataDir || defaultDataDir();
|
||||
const auditDir = join(dataDir, "issue-agent");
|
||||
const auditPath = join(auditDir, "audit.jsonl");
|
||||
const row = {
|
||||
ts: (options.now ?? new Date()).toISOString(),
|
||||
runId: run.runId,
|
||||
mode: run.mode,
|
||||
repository: run.repository,
|
||||
issueNumber: run.issueNumber,
|
||||
issueUrl: run.issueUrl,
|
||||
dryRun: run.dryRun,
|
||||
runner: run.runner,
|
||||
context: run.context,
|
||||
steps: run.steps,
|
||||
};
|
||||
|
||||
await mkdir(auditDir, { recursive: true });
|
||||
await appendFile(auditPath, `${JSON.stringify(row)}\n`, "utf8");
|
||||
return { path: auditPath };
|
||||
}
|
||||
114
src/lib/issueAgent/execution.ts
Normal file
114
src/lib/issueAgent/execution.ts
Normal file
@@ -0,0 +1,114 @@
|
||||
import type { RecordedTriageRun } from "@/lib/issueAgent/recordedTriage";
|
||||
|
||||
export interface RecordedTriageExecutionInput {
|
||||
run: RecordedTriageRun;
|
||||
model?: string;
|
||||
provider?: string;
|
||||
routingPolicy?: string;
|
||||
timeoutMs?: number;
|
||||
}
|
||||
|
||||
export type ChatCompletionsPost = (request: Request) => Promise<Response>;
|
||||
|
||||
export interface RecordedTriageChatCompletion {
|
||||
status: number;
|
||||
body: unknown;
|
||||
}
|
||||
|
||||
export class RecordedTriageTimeoutError extends Error {
|
||||
constructor(timeoutMs: number) {
|
||||
super(`Issue Agent triage timed out after ${timeoutMs}ms`);
|
||||
this.name = "RecordedTriageTimeoutError";
|
||||
}
|
||||
}
|
||||
|
||||
const DEFAULT_MODEL = "auto/quality";
|
||||
const DEFAULT_TIMEOUT_MS = 60_000;
|
||||
const MAX_TIMEOUT_MS = 120_000;
|
||||
|
||||
function configuredString(value: string | undefined, envName: string): string | undefined {
|
||||
const configured = value ?? process.env[envName];
|
||||
const normalized = configured?.trim();
|
||||
return normalized || undefined;
|
||||
}
|
||||
|
||||
function resolveModel(input: RecordedTriageExecutionInput): string {
|
||||
const model = configuredString(input.model, "OMNIROUTE_ISSUE_AGENT_MODEL") ?? DEFAULT_MODEL;
|
||||
const provider = configuredString(input.provider, "OMNIROUTE_ISSUE_AGENT_PROVIDER");
|
||||
|
||||
if (!provider || model.includes("/")) return model;
|
||||
return `${provider}/${model}`;
|
||||
}
|
||||
|
||||
function resolveTimeoutMs(input: RecordedTriageExecutionInput): number {
|
||||
const configured = input.timeoutMs ?? Number(process.env.OMNIROUTE_ISSUE_AGENT_TIMEOUT_MS);
|
||||
if (!Number.isFinite(configured) || configured <= 0) return DEFAULT_TIMEOUT_MS;
|
||||
return Math.min(Math.floor(configured), MAX_TIMEOUT_MS);
|
||||
}
|
||||
|
||||
function buildMessages(run: RecordedTriageRun) {
|
||||
return [
|
||||
{
|
||||
role: "system",
|
||||
content:
|
||||
"You are the OmniRoute Issue Agent. Analyze only the recorded GitHub context and produce a concise, actionable triage response. Do not claim to have accessed external state.",
|
||||
},
|
||||
{
|
||||
role: "user",
|
||||
content: [
|
||||
`Issue: ${run.repository}#${run.issueNumber}`,
|
||||
`URL: ${run.issueUrl}`,
|
||||
`Intent: ${run.context.intent}`,
|
||||
`Title: ${run.context.issueTitle ?? "(untitled)"}`,
|
||||
"Recorded context:",
|
||||
run.context.redactedDigestSource || "(none)",
|
||||
].join("\n"),
|
||||
},
|
||||
];
|
||||
}
|
||||
|
||||
/**
|
||||
* Route recorded-triage work through the same in-process chat endpoint used by
|
||||
* clients, retaining its initialization, admission, guardrails, and policy path.
|
||||
*/
|
||||
export async function executeRecordedTriageChatCompletion(
|
||||
input: RecordedTriageExecutionInput,
|
||||
post: ChatCompletionsPost
|
||||
): Promise<RecordedTriageChatCompletion> {
|
||||
const controller = new AbortController();
|
||||
const timeoutMs = resolveTimeoutMs(input);
|
||||
const timeout = setTimeout(() => controller.abort(), timeoutMs);
|
||||
const routingPolicy = configuredString(
|
||||
input.routingPolicy,
|
||||
"OMNIROUTE_ISSUE_AGENT_ROUTING_POLICY"
|
||||
);
|
||||
|
||||
try {
|
||||
const response = await post(
|
||||
new Request("http://localhost/api/v1/chat/completions", {
|
||||
method: "POST",
|
||||
signal: controller.signal,
|
||||
headers: {
|
||||
"Content-Type": "application/json",
|
||||
...(routingPolicy ? { "X-OmniRoute-Mode": routingPolicy } : {}),
|
||||
},
|
||||
body: JSON.stringify({
|
||||
model: resolveModel(input),
|
||||
messages: buildMessages(input.run),
|
||||
max_tokens: 1200,
|
||||
temperature: 0,
|
||||
stream: false,
|
||||
}),
|
||||
})
|
||||
);
|
||||
|
||||
return { status: response.status, body: await response.json() };
|
||||
} catch (error) {
|
||||
if (error instanceof Error && error.name === "AbortError") {
|
||||
throw new RecordedTriageTimeoutError(timeoutMs);
|
||||
}
|
||||
throw error;
|
||||
} finally {
|
||||
clearTimeout(timeout);
|
||||
}
|
||||
}
|
||||
57
src/lib/issueAgent/githubExport.ts
Normal file
57
src/lib/issueAgent/githubExport.ts
Normal file
@@ -0,0 +1,57 @@
|
||||
import type { RecordedTriageComment, RecordedTriageContextInput } from "./recordedTriage";
|
||||
|
||||
export interface NormalizedGitHubIssueExport {
|
||||
issueUrl: string;
|
||||
recordedContext: RecordedTriageContextInput;
|
||||
}
|
||||
|
||||
function asRecord(value: unknown): Record<string, unknown> {
|
||||
return value && typeof value === "object" ? (value as Record<string, unknown>) : {};
|
||||
}
|
||||
|
||||
function readString(value: unknown): string | undefined {
|
||||
return typeof value === "string" && value.trim().length > 0 ? value : undefined;
|
||||
}
|
||||
|
||||
function isBotUser(user: Record<string, unknown>, fallbackAuthor: string | undefined): boolean {
|
||||
const type = readString(user.type);
|
||||
if (type?.toLowerCase() === "bot") return true;
|
||||
return /\[bot\]$/i.test(fallbackAuthor ?? "");
|
||||
}
|
||||
|
||||
function normalizeComment(value: unknown): RecordedTriageComment {
|
||||
const row = asRecord(value);
|
||||
const user = asRecord(row.user);
|
||||
const author = readString(row.author) ?? readString(row.user_login) ?? readString(user.login);
|
||||
const body = readString(row.body) ?? "";
|
||||
const explicitBot = typeof row.isBot === "boolean" ? row.isBot : undefined;
|
||||
return {
|
||||
author,
|
||||
body,
|
||||
isBot: explicitBot ?? isBotUser(user, author),
|
||||
};
|
||||
}
|
||||
|
||||
export function normalizeGitHubIssueExport(value: unknown): NormalizedGitHubIssueExport {
|
||||
const root = asRecord(value);
|
||||
const issue = asRecord(root.issue ?? root.pull_request ?? root.pr ?? root);
|
||||
const issueUrl = readString(issue.html_url) ?? readString(issue.url) ?? readString(root.html_url) ?? readString(root.url);
|
||||
if (!issueUrl) {
|
||||
throw new Error("Recorded GitHub export must include html_url or url");
|
||||
}
|
||||
|
||||
const rawComments = Array.isArray(root.comments)
|
||||
? root.comments
|
||||
: Array.isArray(issue.comments)
|
||||
? issue.comments
|
||||
: [];
|
||||
|
||||
return {
|
||||
issueUrl,
|
||||
recordedContext: {
|
||||
title: readString(issue.title),
|
||||
body: readString(issue.body),
|
||||
comments: rawComments.slice(0, 100).map(normalizeComment),
|
||||
},
|
||||
};
|
||||
}
|
||||
124
src/lib/issueAgent/recordedTriage.ts
Normal file
124
src/lib/issueAgent/recordedTriage.ts
Normal file
@@ -0,0 +1,124 @@
|
||||
import { createHash } from "node:crypto";
|
||||
import { redactSecrets } from "@/shared/utils/logRedaction";
|
||||
|
||||
export interface RecordedTriageComment {
|
||||
author?: string;
|
||||
body?: string;
|
||||
isBot?: boolean;
|
||||
}
|
||||
|
||||
export interface RecordedTriageContextInput {
|
||||
title?: string;
|
||||
body?: string;
|
||||
comments?: RecordedTriageComment[];
|
||||
}
|
||||
|
||||
export interface RecordedTriageInput {
|
||||
issueUrl?: string;
|
||||
dryRun?: boolean;
|
||||
recordedContext?: RecordedTriageContextInput;
|
||||
}
|
||||
|
||||
export interface RecordedTriageContextSummary {
|
||||
issueTitle: string | null;
|
||||
commentCount: number;
|
||||
humanCommentCount: number;
|
||||
botCommentCount: number;
|
||||
intent: "bugfix" | "review" | "question" | "unknown";
|
||||
redactedDigestSource: string;
|
||||
}
|
||||
|
||||
export interface RecordedTriageRun {
|
||||
accepted: true;
|
||||
mode: "recorded-triage";
|
||||
runner: "deterministic-recorded-triage" | "omniroute-chat-completions";
|
||||
runId: string;
|
||||
issueUrl: string;
|
||||
repository: string;
|
||||
issueNumber: number;
|
||||
dryRun: boolean;
|
||||
context: RecordedTriageContextSummary;
|
||||
steps: string[];
|
||||
}
|
||||
|
||||
const GITHUB_ISSUE_URL =
|
||||
/^https:\/\/(?:[^@/\s]+@)?github\.com\/([^/\s]+)\/([^/\s]+)\/(?:issues|pull)\/(\d+)(?:[/?#].*)?$/i;
|
||||
|
||||
const RECORDED_TRIAGE_STEPS = [
|
||||
"load-recorded-github-context",
|
||||
"classify-mention-intent",
|
||||
"draft-safe-response-plan",
|
||||
"emit-audit-record",
|
||||
];
|
||||
|
||||
function redactUrlCredentials(issueUrl: string): string {
|
||||
return issueUrl.replace(/^https:\/\/[^@/\s]+@github\.com\//i, "https://[REDACTED]@github.com/");
|
||||
}
|
||||
|
||||
function buildRunId(repository: string, issueNumber: number): string {
|
||||
const digest = createHash("sha256")
|
||||
.update(`${repository}#${issueNumber}`)
|
||||
.digest("hex")
|
||||
.slice(0, 16);
|
||||
return `issue-agent-recorded-triage-${digest}`;
|
||||
}
|
||||
|
||||
function normalizeText(value: unknown, maxLength = 400): string {
|
||||
if (typeof value !== "string") return "";
|
||||
return value.trim().replace(/\s+/g, " ").slice(0, maxLength);
|
||||
}
|
||||
|
||||
function classifyIntent(text: string): RecordedTriageContextSummary["intent"] {
|
||||
const lower = text.toLowerCase();
|
||||
if (/\b(bug|fix|failing|regression|broken|patch)\b/.test(lower)) return "bugfix";
|
||||
if (/\b(review|pr|pull request|approve|merge)\b/.test(lower)) return "review";
|
||||
if (/\b(question|why|how|what|help)\b/.test(lower)) return "question";
|
||||
return "unknown";
|
||||
}
|
||||
|
||||
function summarizeContext(
|
||||
input: RecordedTriageContextInput | undefined
|
||||
): RecordedTriageContextSummary {
|
||||
const comments = Array.isArray(input?.comments) ? input.comments.slice(0, 50) : [];
|
||||
const title = normalizeText(input?.title, 160);
|
||||
const body = normalizeText(input?.body);
|
||||
const commentTexts = comments.map((comment) => normalizeText(comment.body, 200)).filter(Boolean);
|
||||
const digestSource = redactSecrets([title, body, ...commentTexts].filter(Boolean).join("\n"));
|
||||
const botCommentCount = comments.filter((comment) => comment.isBot === true).length;
|
||||
const humanCommentCount = comments.length - botCommentCount;
|
||||
|
||||
return {
|
||||
issueTitle: title || null,
|
||||
commentCount: comments.length,
|
||||
humanCommentCount,
|
||||
botCommentCount,
|
||||
intent: classifyIntent(digestSource),
|
||||
redactedDigestSource: digestSource.slice(0, 1200),
|
||||
};
|
||||
}
|
||||
|
||||
export function createRecordedTriageRun(input: RecordedTriageInput): RecordedTriageRun {
|
||||
const issueUrl = typeof input.issueUrl === "string" ? input.issueUrl.trim() : "";
|
||||
const match = GITHUB_ISSUE_URL.exec(issueUrl);
|
||||
if (!match) {
|
||||
throw new Error("Expected a GitHub issue or pull request URL");
|
||||
}
|
||||
|
||||
const owner = match[1]!;
|
||||
const repo = match[2]!;
|
||||
const issueNumber = Number(match[3]);
|
||||
const repository = `${owner}/${repo}`;
|
||||
|
||||
return {
|
||||
accepted: true,
|
||||
mode: "recorded-triage",
|
||||
runner: "deterministic-recorded-triage",
|
||||
runId: buildRunId(repository, issueNumber),
|
||||
issueUrl: redactUrlCredentials(issueUrl),
|
||||
repository,
|
||||
issueNumber,
|
||||
dryRun: input.dryRun !== false,
|
||||
context: summarizeContext(input.recordedContext),
|
||||
steps: [...RECORDED_TRIAGE_STEPS],
|
||||
};
|
||||
}
|
||||
@@ -37,6 +37,7 @@ export const LOCAL_ONLY_API_PREFIXES: ReadonlyArray<string> = [
|
||||
"/api/copilot/", // unauthenticated LLM driver — CLI-only by default; admins can opt-in to remote access via manage-scope bypass
|
||||
"/api/tools/agent-bridge/", // AgentBridge: spawns MITM server + DNS edits (Hard Rules #15 + #17)
|
||||
"/api/tools/traffic-inspector/", // Traffic Inspector: http-proxy listener + system proxy (Hard Rules #15 + #17)
|
||||
"/api/issue-agent/", // Issue Agent: recorded/local triage executor surface; keep loopback/LAN until sandbox + audit hardening is complete
|
||||
"/api/plugins/", // plugins: load/execute via worker_threads + child_process (Hard Rules #15 + #17)
|
||||
"/api/plugins", // bare path: GET list + POST install also trigger plugin loading
|
||||
"/api/middleware/", // SECURITY_AUDIT M8: middleware hooks compile+run arbitrary JS via new vm.Script (src/lib/middleware/registry.ts) on the request hot path — same code-exec class as /api/plugins/, so loopback-gate it for parity (Hard Rules #15 + #17)
|
||||
|
||||
31
tests/unit/issue-agent-audit.test.ts
Normal file
31
tests/unit/issue-agent-audit.test.ts
Normal file
@@ -0,0 +1,31 @@
|
||||
import test from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { mkdtempSync, readFileSync } from "node:fs";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
|
||||
import { appendIssueAgentAuditRecord } from "../../src/lib/issueAgent/audit.ts";
|
||||
import { createRecordedTriageRun } from "../../src/lib/issueAgent/recordedTriage.ts";
|
||||
|
||||
test("appendIssueAgentAuditRecord writes redacted JSONL under explicit data dir", async () => {
|
||||
const dataDir = mkdtempSync(join(tmpdir(), "issue-agent-audit-"));
|
||||
const run = createRecordedTriageRun({
|
||||
issueUrl: "https://github.com/KooshaPari/OmniRoute/issues/42",
|
||||
recordedContext: {
|
||||
title: "Need fix",
|
||||
body: "Authorization: Bearer sk-secret1234567890abcd",
|
||||
},
|
||||
});
|
||||
|
||||
const result = await appendIssueAgentAuditRecord(run, { dataDir });
|
||||
const payload = readFileSync(result.path, "utf8");
|
||||
const row = JSON.parse(payload.trim()) as Record<string, unknown>;
|
||||
|
||||
assert.equal(result.path, join(dataDir, "issue-agent", "audit.jsonl"));
|
||||
assert.equal(row.runId, run.runId);
|
||||
assert.equal(row.repository, "KooshaPari/OmniRoute");
|
||||
assert.equal(row.issueNumber, 42);
|
||||
assert.equal(row.dryRun, true);
|
||||
assert.doesNotMatch(payload, /sk-secret/);
|
||||
assert.match(payload, /\[REDACTED\]/);
|
||||
});
|
||||
70
tests/unit/issue-agent-execution.test.ts
Normal file
70
tests/unit/issue-agent-execution.test.ts
Normal file
@@ -0,0 +1,70 @@
|
||||
import test from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
|
||||
import {
|
||||
executeRecordedTriageChatCompletion,
|
||||
RecordedTriageTimeoutError,
|
||||
} from "../../src/lib/issueAgent/execution.ts";
|
||||
import { createRecordedTriageRun } from "../../src/lib/issueAgent/recordedTriage.ts";
|
||||
|
||||
test("recorded triage invokes the normal chat-completions seam with configured routing", async () => {
|
||||
const run = createRecordedTriageRun({
|
||||
issueUrl: "https://github.com/KooshaPari/OmniRoute/issues/5980",
|
||||
recordedContext: {
|
||||
title: "Execute issue-agent runs through OmniRoute routing",
|
||||
body: "Use the configured provider and model.",
|
||||
},
|
||||
});
|
||||
let received: Request | undefined;
|
||||
|
||||
const response = await executeRecordedTriageChatCompletion(
|
||||
{
|
||||
run,
|
||||
model: "gpt-4.1-mini",
|
||||
provider: "openai",
|
||||
routingPolicy: "quality",
|
||||
timeoutMs: 5_000,
|
||||
},
|
||||
async (request) => {
|
||||
received = request;
|
||||
return new Response(JSON.stringify({ id: "chatcmpl-issue-agent" }), {
|
||||
headers: { "Content-Type": "application/json" },
|
||||
});
|
||||
}
|
||||
);
|
||||
|
||||
assert.equal(received?.url, "http://localhost/api/v1/chat/completions");
|
||||
assert.equal(received?.method, "POST");
|
||||
assert.equal(received?.headers.get("X-OmniRoute-Mode"), "quality");
|
||||
assert.ok(received?.signal, "the chat request must carry the timeout AbortSignal");
|
||||
|
||||
const body = (await received!.json()) as Record<string, unknown>;
|
||||
assert.equal(body.model, "openai/gpt-4.1-mini");
|
||||
assert.equal(body.stream, false);
|
||||
assert.equal(body.temperature, 0);
|
||||
assert.equal(body.max_tokens, 1200);
|
||||
assert.equal(response.status, 200);
|
||||
assert.deepEqual(response.body, { id: "chatcmpl-issue-agent" });
|
||||
assert.match(JSON.stringify(body.messages), /#5980/);
|
||||
});
|
||||
|
||||
test("recorded triage reports an aborted chat invocation as a timeout", async () => {
|
||||
const run = createRecordedTriageRun({
|
||||
issueUrl: "https://github.com/KooshaPari/OmniRoute/issues/5980",
|
||||
});
|
||||
|
||||
await assert.rejects(
|
||||
() =>
|
||||
executeRecordedTriageChatCompletion({ run, timeoutMs: 1 }, async (request) => {
|
||||
await new Promise<void>((_resolve, reject) => {
|
||||
request.signal.addEventListener("abort", () => {
|
||||
reject(Object.assign(new Error("aborted"), { name: "AbortError" }));
|
||||
});
|
||||
});
|
||||
return new Response();
|
||||
}),
|
||||
(error: unknown) =>
|
||||
error instanceof RecordedTriageTimeoutError &&
|
||||
error.message === "Issue Agent triage timed out after 1ms"
|
||||
);
|
||||
});
|
||||
48
tests/unit/issue-agent-github-export.test.ts
Normal file
48
tests/unit/issue-agent-github-export.test.ts
Normal file
@@ -0,0 +1,48 @@
|
||||
import test from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
|
||||
import { normalizeGitHubIssueExport } from "../../src/lib/issueAgent/githubExport.ts";
|
||||
|
||||
test("normalizeGitHubIssueExport reads GitHub REST issue plus comments shape", () => {
|
||||
const normalized = normalizeGitHubIssueExport({
|
||||
issue: {
|
||||
title: "Fix route guard bypass",
|
||||
body: "failing when Host is spoofed",
|
||||
html_url: "https://github.com/KooshaPari/OmniRoute/issues/6059",
|
||||
},
|
||||
comments: [
|
||||
{ user: { login: "renovate[bot]", type: "Bot" }, body: "automated" },
|
||||
{ user: { login: "maintainer", type: "User" }, body: "please patch" },
|
||||
],
|
||||
});
|
||||
|
||||
assert.equal(normalized.issueUrl, "https://github.com/KooshaPari/OmniRoute/issues/6059");
|
||||
assert.deepEqual(normalized.recordedContext, {
|
||||
title: "Fix route guard bypass",
|
||||
body: "failing when Host is spoofed",
|
||||
comments: [
|
||||
{ author: "renovate[bot]", body: "automated", isBot: true },
|
||||
{ author: "maintainer", body: "please patch", isBot: false },
|
||||
],
|
||||
});
|
||||
});
|
||||
|
||||
test("normalizeGitHubIssueExport reads flat issue export shape", () => {
|
||||
const normalized = normalizeGitHubIssueExport({
|
||||
title: "Question about native router",
|
||||
body: "how should cooldowns work?",
|
||||
url: "https://github.com/KooshaPari/OmniRoute/pull/6085",
|
||||
comments: [{ author: "human", body: "review please", isBot: false }],
|
||||
});
|
||||
|
||||
assert.equal(normalized.issueUrl, "https://github.com/KooshaPari/OmniRoute/pull/6085");
|
||||
assert.equal(normalized.recordedContext.title, "Question about native router");
|
||||
assert.equal(normalized.recordedContext.comments?.[0]?.author, "human");
|
||||
});
|
||||
|
||||
test("normalizeGitHubIssueExport rejects exports without a GitHub URL", () => {
|
||||
assert.throws(
|
||||
() => normalizeGitHubIssueExport({ title: "missing url" }),
|
||||
/Recorded GitHub export must include html_url or url/
|
||||
);
|
||||
});
|
||||
135
tests/unit/issue-agent-route-execution.test.ts
Normal file
135
tests/unit/issue-agent-route-execution.test.ts
Normal file
@@ -0,0 +1,135 @@
|
||||
import test from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import fs from "node:fs";
|
||||
import os from "node:os";
|
||||
import path from "node:path";
|
||||
|
||||
const TEST_DATA_DIR =
|
||||
process.env.DATA_DIR ??
|
||||
fs.mkdtempSync(path.join(os.tmpdir(), "omniroute-issue-agent-execution-"));
|
||||
process.env.DATA_DIR = TEST_DATA_DIR;
|
||||
process.env.APP_LOG_TO_FILE = "false";
|
||||
|
||||
const core = await import("../../src/lib/db/core.ts");
|
||||
const providersDb = await import("../../src/lib/db/providers.ts");
|
||||
const issueAgentRoute = await import("../../src/app/api/issue-agent/runs/route.ts");
|
||||
const originalFetch = globalThis.fetch;
|
||||
|
||||
async function resetStorage() {
|
||||
core.resetDbInstance();
|
||||
fs.rmSync(TEST_DATA_DIR, { recursive: true, force: true });
|
||||
fs.mkdirSync(TEST_DATA_DIR, { recursive: true });
|
||||
}
|
||||
|
||||
async function seedOpenAiConnection() {
|
||||
return providersDb.createProviderConnection({
|
||||
provider: "openai",
|
||||
authType: "apikey",
|
||||
name: "issue-agent-route-test",
|
||||
apiKey: "sk-issue-agent-route-test",
|
||||
isActive: true,
|
||||
testStatus: "active",
|
||||
});
|
||||
}
|
||||
|
||||
test.beforeEach(async () => {
|
||||
globalThis.fetch = originalFetch;
|
||||
process.env.OMNIROUTE_ISSUE_AGENT_ENABLED = "true";
|
||||
await resetStorage();
|
||||
await core.ensureDbInitialized();
|
||||
});
|
||||
|
||||
test.afterEach(() => {
|
||||
globalThis.fetch = originalFetch;
|
||||
delete process.env.OMNIROUTE_ISSUE_AGENT_ENABLED;
|
||||
});
|
||||
|
||||
test("issue-agent live triage traverses the normal chat-completions POST route", async () => {
|
||||
await seedOpenAiConnection();
|
||||
const fetchCalls: Array<{ url: string; init: RequestInit }> = [];
|
||||
globalThis.fetch = async (url, init = {}) => {
|
||||
fetchCalls.push({ url: String(url), init });
|
||||
return Response.json({
|
||||
id: "chatcmpl-issue-agent-route",
|
||||
choices: [{ message: { role: "assistant", content: "Triage response" } }],
|
||||
});
|
||||
};
|
||||
|
||||
const response = await issueAgentRoute.POST(
|
||||
new Request("http://localhost/api/issue-agent/runs", {
|
||||
method: "POST",
|
||||
body: JSON.stringify({
|
||||
mode: "recorded-triage",
|
||||
dryRun: false,
|
||||
issueUrl: "https://github.com/KooshaPari/OmniRoute/issues/5980",
|
||||
recordedContext: {
|
||||
title: "Execute issue-agent triage through the router",
|
||||
body: "Use the configured provider and routing policy.",
|
||||
},
|
||||
provider: "openai",
|
||||
model: "gpt-4.1",
|
||||
routingPolicy: "quality",
|
||||
}),
|
||||
})
|
||||
);
|
||||
const body = (await response.json()) as Record<string, unknown>;
|
||||
|
||||
assert.equal(response.status, 200);
|
||||
assert.equal(body.runner, "omniroute-chat-completions");
|
||||
assert.equal(fetchCalls.length, 1, "only the external provider boundary is mocked");
|
||||
assert.match(fetchCalls[0]!.url, /\/chat\/completions$/);
|
||||
|
||||
const providerRequest = JSON.parse(String(fetchCalls[0]!.init.body)) as Record<string, unknown>;
|
||||
assert.equal(providerRequest.model, "gpt-4.1");
|
||||
assert.equal(providerRequest.stream, false);
|
||||
assert.match(JSON.stringify(providerRequest.messages), /#5980/);
|
||||
assert.equal((body.completion as Record<string, unknown>).id, "chatcmpl-issue-agent-route");
|
||||
});
|
||||
|
||||
test("issue-agent preserves the normal chat route provider failure response", async () => {
|
||||
await seedOpenAiConnection();
|
||||
globalThis.fetch = async () =>
|
||||
Response.json(
|
||||
{ error: { message: "provider rate limited", type: "rate_limit_error" } },
|
||||
{ status: 429 }
|
||||
);
|
||||
|
||||
const response = await issueAgentRoute.POST(
|
||||
new Request("http://localhost/api/issue-agent/runs", {
|
||||
method: "POST",
|
||||
body: JSON.stringify({
|
||||
mode: "recorded-triage",
|
||||
// #7315: dryRun must be explicit false — createRecordedTriageRun()
|
||||
// (src/lib/issueAgent/recordedTriage.ts) computes `dryRun: input.dryRun
|
||||
// !== false`, so an OMITTED dryRun defaults to true (dry-run mode) and
|
||||
// the route returns the deterministic dry-run summary WITHOUT ever
|
||||
// calling executeRecordedTriageChatCompletion() — meaning the mocked
|
||||
// 429 fetch below is never invoked and this test always observed the
|
||||
// 200 dry-run response instead. Not a semantic-cache collision: the
|
||||
// sibling test above passes dryRun:false explicitly, this one didn't.
|
||||
dryRun: false,
|
||||
issueUrl: "https://github.com/KooshaPari/OmniRoute/issues/5980",
|
||||
recordedContext: { body: "Preserve upstream errors for triage." },
|
||||
provider: "openai",
|
||||
model: "gpt-4.1",
|
||||
}),
|
||||
})
|
||||
);
|
||||
const body = (await response.json()) as Record<string, unknown>;
|
||||
|
||||
// The normal chat-completions route enriches the raw upstream error (adds a
|
||||
// "[provider/model] [status]:" prefix and a connection-cooldown hint) rather
|
||||
// than passing the mocked JSON through byte-for-byte — that enrichment is the
|
||||
// normal route's own, deliberate behavior (see RESILIENCE_GUIDE.md connection
|
||||
// cooldown). This test's job is to prove the issue-agent execution wrapper
|
||||
// preserves and surfaces THAT real response (status + original message text)
|
||||
// rather than swallowing it or substituting its own error — so assert on the
|
||||
// stable, observable parts instead of the exact enrichment wording/timing.
|
||||
assert.equal(response.status, 429);
|
||||
const completion = body.completion as { error?: { message?: string } };
|
||||
assert.match(
|
||||
completion.error?.message ?? "",
|
||||
/provider rate limited/,
|
||||
"the original upstream error message must survive through the issue-agent execution wrapper"
|
||||
);
|
||||
});
|
||||
63
tests/unit/issue-agent-runner.test.ts
Normal file
63
tests/unit/issue-agent-runner.test.ts
Normal file
@@ -0,0 +1,63 @@
|
||||
import test from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
|
||||
import { createRecordedTriageRun } from "../../src/lib/issueAgent/recordedTriage.ts";
|
||||
|
||||
test("createRecordedTriageRun returns deterministic dry-run metadata", () => {
|
||||
const run = createRecordedTriageRun({
|
||||
issueUrl: "https://github.com/KooshaPari/OmniRoute/issues/6059",
|
||||
dryRun: true,
|
||||
});
|
||||
|
||||
assert.equal(run.mode, "recorded-triage");
|
||||
assert.equal(run.accepted, true);
|
||||
assert.equal(run.dryRun, true);
|
||||
assert.equal(run.repository, "KooshaPari/OmniRoute");
|
||||
assert.equal(run.issueNumber, 6059);
|
||||
assert.equal(run.runner, "deterministic-recorded-triage");
|
||||
assert.match(run.runId, /^issue-agent-recorded-triage-[a-f0-9]{16}$/);
|
||||
assert.deepEqual(run.steps, [
|
||||
"load-recorded-github-context",
|
||||
"classify-mention-intent",
|
||||
"draft-safe-response-plan",
|
||||
"emit-audit-record",
|
||||
]);
|
||||
});
|
||||
|
||||
test("createRecordedTriageRun redacts URL credentials before returning metadata", () => {
|
||||
const run = createRecordedTriageRun({
|
||||
issueUrl: "https://user:token12345678901234567890@github.com/KooshaPari/OmniRoute/issues/1",
|
||||
});
|
||||
|
||||
assert.equal(run.issueUrl, "https://[REDACTED]@github.com/KooshaPari/OmniRoute/issues/1");
|
||||
assert.doesNotMatch(run.issueUrl, /token123/);
|
||||
});
|
||||
|
||||
test("createRecordedTriageRun rejects non-GitHub issue URLs", () => {
|
||||
assert.throws(
|
||||
() => createRecordedTriageRun({ issueUrl: "https://example.com/not/github" }),
|
||||
/Expected a GitHub issue or pull request URL/
|
||||
);
|
||||
});
|
||||
|
||||
test("createRecordedTriageRun summarizes recorded context without leaking secrets", () => {
|
||||
const run = createRecordedTriageRun({
|
||||
issueUrl: "https://github.com/KooshaPari/OmniRoute/issues/42",
|
||||
recordedContext: {
|
||||
title: "Need fix for failing route guard",
|
||||
body: "Please inspect Authorization: Bearer sk-secret1234567890abcd",
|
||||
comments: [
|
||||
{ author: "octobot", body: "automated noise", isBot: true },
|
||||
{ author: "human", body: "mentions @KooshaPari and asks for patch", isBot: false },
|
||||
],
|
||||
},
|
||||
});
|
||||
|
||||
assert.equal(run.context.issueTitle, "Need fix for failing route guard");
|
||||
assert.equal(run.context.commentCount, 2);
|
||||
assert.equal(run.context.humanCommentCount, 1);
|
||||
assert.equal(run.context.botCommentCount, 1);
|
||||
assert.equal(run.context.intent, "bugfix");
|
||||
assert.doesNotMatch(run.context.redactedDigestSource, /sk-secret/);
|
||||
assert.match(run.context.redactedDigestSource, /\[REDACTED\]/);
|
||||
});
|
||||
285
tests/unit/issue-agent-runs-route.test.ts
Normal file
285
tests/unit/issue-agent-runs-route.test.ts
Normal file
@@ -0,0 +1,285 @@
|
||||
import test from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { mkdtempSync, writeFileSync } from "node:fs";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
|
||||
// Bun evaluates test files in one process. Set DATA_DIR before importing the
|
||||
// route so its import-time dependencies never probe the developer's real DB.
|
||||
const TEST_DATA_DIR =
|
||||
process.env.DATA_DIR ?? mkdtempSync(join(tmpdir(), "issue-agent-runs-route-"));
|
||||
process.env.DATA_DIR = TEST_DATA_DIR;
|
||||
process.env.APP_LOG_TO_FILE = "false";
|
||||
|
||||
const { GET, POST } = await import("../../src/app/api/issue-agent/runs/route.ts");
|
||||
|
||||
async function json(response: Response): Promise<Record<string, unknown>> {
|
||||
return (await response.json()) as Record<string, unknown>;
|
||||
}
|
||||
|
||||
test("issue-agent status reports default-off recorded triage support", async () => {
|
||||
const previous = process.env.OMNIROUTE_ISSUE_AGENT_ENABLED;
|
||||
delete process.env.OMNIROUTE_ISSUE_AGENT_ENABLED;
|
||||
try {
|
||||
const response = await GET();
|
||||
const body = await json(response);
|
||||
|
||||
assert.equal(response.status, 200);
|
||||
assert.equal(body.ok, true);
|
||||
assert.equal(body.enabled, false);
|
||||
assert.deepEqual(body.supportedModes, ["recorded-triage"]);
|
||||
} finally {
|
||||
if (previous === undefined) delete process.env.OMNIROUTE_ISSUE_AGENT_ENABLED;
|
||||
else process.env.OMNIROUTE_ISSUE_AGENT_ENABLED = previous;
|
||||
}
|
||||
});
|
||||
|
||||
test("issue-agent run rejects invalid JSON", async () => {
|
||||
const response = await POST(
|
||||
new Request("http://localhost/api/issue-agent/runs", {
|
||||
method: "POST",
|
||||
body: "{",
|
||||
})
|
||||
);
|
||||
const body = await json(response);
|
||||
|
||||
assert.equal(response.status, 400);
|
||||
assert.equal(body.error, "Invalid JSON body");
|
||||
});
|
||||
|
||||
test("issue-agent run rejects invalid request field types", async () => {
|
||||
const response = await POST(
|
||||
new Request("http://localhost/api/issue-agent/runs", {
|
||||
method: "POST",
|
||||
body: JSON.stringify({ mode: 42 }),
|
||||
})
|
||||
);
|
||||
|
||||
assert.equal(response.status, 400);
|
||||
});
|
||||
|
||||
test("issue-agent run accepts only recorded-triage mode", async () => {
|
||||
const response = await POST(
|
||||
new Request("http://localhost/api/issue-agent/runs", {
|
||||
method: "POST",
|
||||
body: JSON.stringify({ mode: "live-shell" }),
|
||||
})
|
||||
);
|
||||
const body = await json(response);
|
||||
|
||||
assert.equal(response.status, 400);
|
||||
assert.equal(body.error, "Unsupported issue-agent mode");
|
||||
});
|
||||
|
||||
test("issue-agent run rejects invalid recorded-triage field types", async () => {
|
||||
const response = await POST(
|
||||
new Request("http://localhost/api/issue-agent/runs", {
|
||||
method: "POST",
|
||||
body: JSON.stringify({ mode: "recorded-triage", dryRun: "true" }),
|
||||
})
|
||||
);
|
||||
const body = await json(response);
|
||||
const error = body.error as Record<string, unknown>;
|
||||
const details = error.details as Array<Record<string, unknown>>;
|
||||
|
||||
assert.equal(response.status, 400);
|
||||
assert.equal(error.message, "Invalid request");
|
||||
assert.deepEqual(details, [
|
||||
{ field: "dryRun", message: "Invalid input: expected boolean, received string" },
|
||||
]);
|
||||
});
|
||||
|
||||
test("issue-agent run is disabled by default", async () => {
|
||||
const previous = process.env.OMNIROUTE_ISSUE_AGENT_ENABLED;
|
||||
delete process.env.OMNIROUTE_ISSUE_AGENT_ENABLED;
|
||||
try {
|
||||
const response = await POST(
|
||||
new Request("http://localhost/api/issue-agent/runs", {
|
||||
method: "POST",
|
||||
body: JSON.stringify({
|
||||
mode: "recorded-triage",
|
||||
issueUrl: "https://github.com/x/y/issues/1",
|
||||
}),
|
||||
})
|
||||
);
|
||||
const body = await json(response);
|
||||
|
||||
assert.equal(response.status, 403);
|
||||
assert.equal(body.enabled, false);
|
||||
assert.equal(body.requiredEnv, "OMNIROUTE_ISSUE_AGENT_ENABLED=true");
|
||||
} finally {
|
||||
if (previous === undefined) delete process.env.OMNIROUTE_ISSUE_AGENT_ENABLED;
|
||||
else process.env.OMNIROUTE_ISSUE_AGENT_ENABLED = previous;
|
||||
}
|
||||
});
|
||||
|
||||
test("issue-agent run returns deterministic recorded-triage plan when enabled", async () => {
|
||||
const previous = process.env.OMNIROUTE_ISSUE_AGENT_ENABLED;
|
||||
process.env.OMNIROUTE_ISSUE_AGENT_ENABLED = "true";
|
||||
try {
|
||||
const response = await POST(
|
||||
new Request("http://localhost/api/issue-agent/runs", {
|
||||
method: "POST",
|
||||
body: JSON.stringify({
|
||||
mode: "recorded-triage",
|
||||
issueUrl: "https://github.com/KooshaPari/OmniRoute/issues/6059",
|
||||
dryRun: true,
|
||||
}),
|
||||
})
|
||||
);
|
||||
const body = await json(response);
|
||||
|
||||
assert.equal(response.status, 200);
|
||||
assert.equal(body.accepted, true);
|
||||
assert.equal(body.mode, "recorded-triage");
|
||||
assert.equal(body.repository, "KooshaPari/OmniRoute");
|
||||
assert.equal(body.issueNumber, 6059);
|
||||
assert.match(String(body.runId), /^issue-agent-recorded-triage-[a-f0-9]{16}$/);
|
||||
} finally {
|
||||
if (previous === undefined) delete process.env.OMNIROUTE_ISSUE_AGENT_ENABLED;
|
||||
else process.env.OMNIROUTE_ISSUE_AGENT_ENABLED = previous;
|
||||
}
|
||||
});
|
||||
|
||||
test("issue-agent run rejects invalid enabled recorded-triage URL", async () => {
|
||||
const previous = process.env.OMNIROUTE_ISSUE_AGENT_ENABLED;
|
||||
process.env.OMNIROUTE_ISSUE_AGENT_ENABLED = "true";
|
||||
try {
|
||||
const response = await POST(
|
||||
new Request("http://localhost/api/issue-agent/runs", {
|
||||
method: "POST",
|
||||
body: JSON.stringify({ mode: "recorded-triage", issueUrl: "https://example.com/nope" }),
|
||||
})
|
||||
);
|
||||
const body = await json(response);
|
||||
|
||||
assert.equal(response.status, 400);
|
||||
assert.equal(body.error, "Expected a GitHub issue or pull request URL");
|
||||
} finally {
|
||||
if (previous === undefined) delete process.env.OMNIROUTE_ISSUE_AGENT_ENABLED;
|
||||
else process.env.OMNIROUTE_ISSUE_AGENT_ENABLED = previous;
|
||||
}
|
||||
});
|
||||
|
||||
test("issue-agent run returns recorded context summary with redaction", async () => {
|
||||
const previous = process.env.OMNIROUTE_ISSUE_AGENT_ENABLED;
|
||||
const previousDataDir = process.env.DATA_DIR;
|
||||
process.env.OMNIROUTE_ISSUE_AGENT_ENABLED = "true";
|
||||
process.env.DATA_DIR = mkdtempSync(join(tmpdir(), "issue-agent-route-"));
|
||||
try {
|
||||
const response = await POST(
|
||||
new Request("http://localhost/api/issue-agent/runs", {
|
||||
method: "POST",
|
||||
body: JSON.stringify({
|
||||
mode: "recorded-triage",
|
||||
issueUrl: "https://github.com/KooshaPari/OmniRoute/issues/7",
|
||||
recordedContext: {
|
||||
title: "Review PR mention",
|
||||
body: "Authorization: Bearer sk-routeSecret1234567890",
|
||||
comments: [{ author: "maintainer", body: "please review", isBot: false }],
|
||||
},
|
||||
}),
|
||||
})
|
||||
);
|
||||
const body = await json(response);
|
||||
const context = body.context as Record<string, unknown>;
|
||||
|
||||
assert.equal(response.status, 200);
|
||||
assert.equal(body.auditPath, join(process.env.DATA_DIR, "issue-agent", "audit.jsonl"));
|
||||
assert.equal(context.issueTitle, "Review PR mention");
|
||||
assert.equal(context.intent, "review");
|
||||
assert.equal(context.humanCommentCount, 1);
|
||||
assert.doesNotMatch(String(context.redactedDigestSource), /sk-routeSecret/);
|
||||
assert.match(String(context.redactedDigestSource), /\[REDACTED\]/);
|
||||
} finally {
|
||||
if (previous === undefined) delete process.env.OMNIROUTE_ISSUE_AGENT_ENABLED;
|
||||
else process.env.OMNIROUTE_ISSUE_AGENT_ENABLED = previous;
|
||||
if (previousDataDir === undefined) delete process.env.DATA_DIR;
|
||||
else process.env.DATA_DIR = previousDataDir;
|
||||
}
|
||||
});
|
||||
|
||||
test("issue-agent run accepts recorded GitHub export payloads", async () => {
|
||||
const previous = process.env.OMNIROUTE_ISSUE_AGENT_ENABLED;
|
||||
const previousDataDir = process.env.DATA_DIR;
|
||||
process.env.OMNIROUTE_ISSUE_AGENT_ENABLED = "true";
|
||||
process.env.DATA_DIR = mkdtempSync(join(tmpdir(), "issue-agent-export-route-"));
|
||||
try {
|
||||
const response = await POST(
|
||||
new Request("http://localhost/api/issue-agent/runs", {
|
||||
method: "POST",
|
||||
body: JSON.stringify({
|
||||
mode: "recorded-triage",
|
||||
githubExport: {
|
||||
issue: {
|
||||
title: "Fix GitHub export importer",
|
||||
body: "bug in recorded context path",
|
||||
html_url: "https://github.com/KooshaPari/OmniRoute/issues/88",
|
||||
},
|
||||
comments: [{ user: { login: "maintainer", type: "User" }, body: "please patch" }],
|
||||
},
|
||||
}),
|
||||
})
|
||||
);
|
||||
const body = await json(response);
|
||||
const context = body.context as Record<string, unknown>;
|
||||
|
||||
assert.equal(response.status, 200);
|
||||
assert.equal(body.issueUrl, "https://github.com/KooshaPari/OmniRoute/issues/88");
|
||||
assert.equal(body.issueNumber, 88);
|
||||
assert.equal(context.issueTitle, "Fix GitHub export importer");
|
||||
assert.equal(context.intent, "bugfix");
|
||||
assert.equal(body.auditPath, join(process.env.DATA_DIR, "issue-agent", "audit.jsonl"));
|
||||
} finally {
|
||||
if (previous === undefined) delete process.env.OMNIROUTE_ISSUE_AGENT_ENABLED;
|
||||
else process.env.OMNIROUTE_ISSUE_AGENT_ENABLED = previous;
|
||||
if (previousDataDir === undefined) delete process.env.DATA_DIR;
|
||||
else process.env.DATA_DIR = previousDataDir;
|
||||
}
|
||||
});
|
||||
|
||||
test("issue-agent run sanitizes a forced audit-write failure instead of leaking its absolute path", async () => {
|
||||
const previous = process.env.OMNIROUTE_ISSUE_AGENT_ENABLED;
|
||||
const previousDataDir = process.env.DATA_DIR;
|
||||
process.env.OMNIROUTE_ISSUE_AGENT_ENABLED = "true";
|
||||
const auditBlockerDir = mkdtempSync(join(tmpdir(), "issue-agent-audit-blocker-"));
|
||||
// appendIssueAgentAuditRecord() (src/lib/issueAgent/audit.ts) does
|
||||
// `mkdir(join(DATA_DIR, "issue-agent"), { recursive: true })`. Pre-creating a
|
||||
// FILE at that exact path forces a real Node fs error (EEXIST) whose raw
|
||||
// `.message` embeds this directory's absolute path -- the same leak shape a
|
||||
// permission-denied or disk-full failure would produce (Hard Rule #12).
|
||||
writeFileSync(join(auditBlockerDir, "issue-agent"), "blocking file, not a directory");
|
||||
process.env.DATA_DIR = auditBlockerDir;
|
||||
|
||||
try {
|
||||
const response = await POST(
|
||||
new Request("http://localhost/api/issue-agent/runs", {
|
||||
method: "POST",
|
||||
body: JSON.stringify({
|
||||
mode: "recorded-triage",
|
||||
issueUrl: "https://github.com/KooshaPari/OmniRoute/issues/1",
|
||||
}),
|
||||
})
|
||||
);
|
||||
const body = await json(response);
|
||||
|
||||
assert.equal(response.status, 400);
|
||||
assert.equal(typeof body.error, "string");
|
||||
const errorMessage = String(body.error);
|
||||
assert.doesNotMatch(errorMessage, /EEXIST|ENOENT|EACCES/, "must not leak the raw errno code");
|
||||
assert.ok(
|
||||
!errorMessage.includes(auditBlockerDir),
|
||||
"must not leak the DATA_DIR absolute path"
|
||||
);
|
||||
assert.ok(
|
||||
!errorMessage.includes("/issue-agent/"),
|
||||
"must not leak the audit subdirectory path"
|
||||
);
|
||||
assert.equal(errorMessage, "Issue Agent request failed due to an internal error");
|
||||
} finally {
|
||||
if (previous === undefined) delete process.env.OMNIROUTE_ISSUE_AGENT_ENABLED;
|
||||
else process.env.OMNIROUTE_ISSUE_AGENT_ENABLED = previous;
|
||||
if (previousDataDir === undefined) delete process.env.DATA_DIR;
|
||||
else process.env.DATA_DIR = previousDataDir;
|
||||
}
|
||||
});
|
||||
@@ -85,6 +85,11 @@ test("services + traffic-inspector remain LOCAL_ONLY paths", () => {
|
||||
assert.equal(isLocalOnlyPath("/api/tools/traffic-inspector/sessions"), true);
|
||||
});
|
||||
|
||||
test("issue-agent routes are LOCAL_ONLY by default", () => {
|
||||
assert.equal(isLocalOnlyPath("/api/issue-agent/runs"), true);
|
||||
assert.equal(isLocalOnlyPath("/api/issue-agent/runs/recorded-triage"), true);
|
||||
});
|
||||
|
||||
test("management policy must NOT derive locality from the spoofable Host header", () => {
|
||||
const src = readFileSync(
|
||||
join(import.meta.dirname, "../../src/server/authz/policies/management.ts"),
|
||||
|
||||
Reference in New Issue
Block a user