docs: record Devin ACP live compatibility blocker

This commit is contained in:
Lucas Israel
2026-07-27 15:00:58 -03:00
parent 71af74bee4
commit 0dcac8bfc3
2 changed files with 47 additions and 14 deletions

View File

@@ -5,13 +5,21 @@ Messages endpoint while the official Devin CLI supplies model responses over ACP
It does not modify the existing Anthropic, Claude OAuth, Claude Web, or `devin-cli`
providers.
> **Current status: offline-complete, live-blocked.** Devin CLI `3000.2.17` does not
> expose a neutral no-tools inference mode over ACP. Its `summarizer` agent has a fixed
> summarization role and does not reliably follow the tool-envelope protocol; its default
> agent attempts to execute tools inside Devin. The adapter rejects those internal tool
> events with `502`, so it is fail-closed but the three required live scenarios do not pass.
> Do not use `launch` as a working live bridge until the official CLI provides a neutral
> generation mode or an equivalent supported API.
## Architecture
```text
Claude Code 2.1.220 (isolated Linux container)
-> http://omniroute:20128/v1/messages
-> devin-cli-agentic (Claude-format, no-auth provider)
-> devin acp --agent-type summarizer (ACP v1 over stdio)
-> devin acp (ACP v1 over stdio; internal Devin tool-call events are rejected)
-> Devin account in the dedicated devin-auth volume (live profile only)
```
@@ -26,6 +34,11 @@ tool's JSON Schema, rejects mixed narrative/actions, and permits one bounded rep
Claude Code executes the resulting Anthropic `tool_use`; Devin never executes those local
tools through this adapter.
The executor intentionally starts the default `devin acp` agent with every ACP client
capability disabled. Any `tool_call` or `tool_call_update` emitted by Devin aborts the turn
before OmniRoute can report success. This guard is required because allowing the default
agent to execute tools would make Devin, rather than Claude Code, the agentic runtime.
## Threat model and isolation
The bridge assumes the host contains an unrelated personal Claude installation and treats
@@ -82,6 +95,10 @@ unversioned in `.sandbox/evidence`.
## Devin login and live use
The commands below are retained for reproducing the live compatibility check. They are not
a claim that the bridge is live-ready; the current pinned CLI fails closed for the reason
documented above.
Authentication uses only the official CLI inside the dedicated volume. It never imports a
host session:
@@ -156,6 +173,11 @@ isolated databases, and evidence.
## Limits
- **Blocking limitation:** official Devin CLI `3000.2.17` offers `summarizer` (no tools,
fixed summarization behavior), `review` (read-only and shell tools), or the default agent.
There is no documented neutral text-generation agent that both follows the envelope and
structurally cannot execute tools. The default agent emitted internal tool calls in the
authorized live test, and OmniRoute rejected them with `502`.
- ACP context is reconstructed from each Anthropic request; there is no persistent process
or session affinity.
- One tool call is supported per model response; parallel tool calls are rejected.

View File

@@ -13,7 +13,7 @@ Updated: 2026-07-27
## Proved offline
- Focused unit and ACP suite: 20 tests passed.
- Focused unit and ACP suite: 27 tests passed.
- Anthropic wire suite: non-streaming JSON, SSE event order, `tool_use`, direct
`tool_result` continuation, ACP error, and early process exit passed.
- Final container build completed with the pinned CLIs and the production OmniRoute build.
@@ -31,7 +31,8 @@ Evidence is generated under `.sandbox/evidence` and is intentionally ignored by
## Regression status
- Focused ESLint and `typecheck:core` passed.
- Focused ESLint and `typecheck:core` passed; the final executor changes were rechecked with
`typecheck:core` and the 27-test bridge suite.
- The complete `npm run check` reached the unit suite after lint, but the repository test
runner did not terminate after `quota-redis-store.test.ts`: an `ioredis` client kept
reconnecting to an unavailable local Redis endpoint. The runner was interrupted after
@@ -41,19 +42,26 @@ Evidence is generated under `.sandbox/evidence` and is intentionally ignored by
## Live Devin
Not passed. After the ownership repair, a fresh `devin-auth` volume was mounted and read by
the non-root bridge user. The official CLI then reported that it was not logged in. Dynamic
model discovery and the three live scenarios therefore did not run, and no live model request
was made.
Not passed. The official Devin login succeeded inside the dedicated volume, and model
discovery selected `swe-1-7-lightning`. Authorized live runs established the following:
The login command resumes the complete live suite automatically after authentication:
1. `devin acp --agent-type summarizer` is not a neutral inference backend. The CLI's own
help identifies it as a no-tools summarizer, and its injected role caused future-action
narration, summaries, and malformed tool envelopes instead of a reliable Claude Code
tool loop.
2. The default `devin acp` agent can emit the required XML tool envelope in a minimal probe,
but the full Claude Code request caused it to emit ACP `tool_call` events and attempt to
own tool execution.
3. The adapter now rejects `tool_call` and `tool_call_update` with
`devin_internal_tool_execution` (`502`). Two live requests were observed failing closed;
a third in-flight request was aborted when the test was stopped to avoid automatic paid
retries.
4. No three-scenario live pass exists. The live test must remain red until the official CLI
exposes a neutral no-tools generation mode (without the summarizer role) or another
supported Devin API provides equivalent raw model inference.
```bash
ENABLE_LIVE_DEVIN_TESTS=1 ./scripts/devin-bridge/login-devin
```
Do not report the bridge as live-ready until `devin auth status`, dynamic model discovery,
and the three scenarios in `test-live-devin` all succeed.
The offline contract still proves `narrative -> single repair -> tool_use`, and a second
narrative now fails closed instead of being accepted as a successful final response.
## Safety record
@@ -61,3 +69,6 @@ No host Claude executable, configuration, login, OAuth token, Keychain, or Anthr
used. The first pre-isolation unit attempt initialized the repository's default OmniRoute
database at `/Users/lucasisrael/.omniroute/storage.sqlite`; it was not rolled back or touched
again. All subsequent bridge commands set isolated database paths under `.sandbox`.
The final live attempts used only the dedicated Docker volumes. Claude Code ran only inside
the non-root container; no host Claude configuration or credential path was mounted or read.