Files
OmniRoute/scripts/devin-bridge/test-live-devin
Praveen K Palaniswamy 65e81158ab fix(ollama): route models by advertised capability (#11088)
Landed with the design call resolved per the owner's pick — **option 1**: the synced store is now endpoint-agnostic (persistDiscoveredModels and managedModelImport no longer drop non-chat models at write time), and chat selectability moved to read time (auto-pool expansion in autoStrategy applies filterChatSelectableModels; the models-route projection already had its chatOnly filter). Your discovery test now passes end-to-end (3/3): /api/show capabilities persist per connection and image/embedding requests route through the advertising host.

Reconciliation notes: conflicted areas merged onto the current tip (adobe discovery import, requestedModel preflight signature, resolvedProvider fast-path coexists with the synced-route override — explicit resolution wins); carried base-red drains (#10055 memoization, #11071 test variants) dropped as already-landed; the managed-model-import exclusion test was propagated to the new contract (image/video models persist; the read filter still hides them from chat pickers — pinned by a new assertion). Full battery: 205/206 focused (the one red is a confirmed periodic-timer timing flake on the loaded devbox — 20/20 isolated), autoCombo vitest 30/30, combo suites 46/46, gates + typecheck clean.

Thank you @yourspraveen — the capability probe + routing design was right; it just needed the store contract opened up. Fixes #11087.
2026-08-23 11:45:01 -03:00

38 lines
1.6 KiB
Bash
Executable File

#!/usr/bin/env bash
set -euo pipefail
source "$(dirname "$0")/common"
[[ "${ENABLE_LIVE_DEVIN_TESTS:-}" == 1 ]] || { echo 'Set ENABLE_LIVE_DEVIN_TESTS=1' >&2; exit 1; }
trap bridge_cleanup_compose EXIT
bridge_cleanup_compose
bridge_reset_live_fixture
"$(dirname "$0")/verify-anthropic-isolation" --static
docker compose -f "$BRIDGE_COMPOSE" --profile live-devin up -d --wait network-guard
bridge_check_devin_auth
models_file="$BRIDGE_SANDBOX/evidence/live-models.json"
if [[ -n "${DEVIN_BRIDGE_DISCOVERED_MODEL:-}" ]]; then
devin_model="$DEVIN_BRIDGE_DISCOVERED_MODEL"
else
for attempt in 1 2 3; do
if bridge_run_devin models list --format json >"$models_file"; then
break
fi
[[ "$attempt" == 3 ]] && exit 1
sleep 1
done
devin_model="$(node --import tsx/esm "$BRIDGE_ROOT/scripts/devin-bridge/select-live-model.mjs" \
<"$models_file")"
fi
export DEVIN_BRIDGE_MODEL="devin-cli-agentic/$devin_model"
export DEVIN_BRIDGE_SONNET_MODEL="$DEVIN_BRIDGE_MODEL"
export DEVIN_BRIDGE_OPUS_MODEL="$DEVIN_BRIDGE_MODEL"
export DEVIN_BRIDGE_HAIKU_MODEL="$DEVIN_BRIDGE_MODEL"
export DEVIN_BRIDGE_SUBAGENT_MODEL="$DEVIN_BRIDGE_MODEL"
docker compose -f "$BRIDGE_COMPOSE" --profile live-devin up --abort-on-container-exit --exit-code-from claude-live claude-live
bridge_cleanup_compose
bridge_assert_devin_guard_audit "$BRIDGE_DEVIN_AUDIT"
bridge_assert_zero_claude_egress "$BRIDGE_CLAUDE_AUDIT"
bridge_export_guard_audit "$BRIDGE_DEVIN_AUDIT" egress.jsonl
bridge_export_guard_audit "$BRIDGE_CLAUDE_AUDIT" claude-egress.jsonl
trap - EXIT
printf 'PASS: live model %s was discovered and validated by three scenarios\n' "$devin_model"