1

Search Tools Studio

diegosouzapw edited this page 2026-06-15 12:01:41 -03:00

Table of Contents

• Search Tools Studio
• Overview
• Tabs
• Search Tab
• Scrape Tab
• Compare Tab
• Config Pane (Shared)
• SearchConceptCard
• Provider Catalog
• Export Code
• Backend Changes
• Extended GET /api/search/providers
• Key Files
• Troubleshooting
• References

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

🌍 View in other languages

Search Tools Studio

Feature: Search Tools Studio — unified web tools workspace for /dashboard/search-tools. Plans: 18-search-tools-studio-redesign.plan.md + _orchestration/master-plan-group-C.md Status: Released in v3.8.6

---

Overview

Search Tools Studio transforms /dashboard/search-tools from a basic search playground into a three-tab Studio unifying web search, web scraping, and side-by-side provider comparison.

┌ Search Tools ──────────────────────────────────────────────────────────┐
│ [🔍 Search] [📄 Scrape] [⚖ Compare]             142ms · $0.001  </>    │
│ ⓘ [Modalities guide]                                                    │
├──────────────────────────────────────────┬─────────────────────────────┤
│  {active tab content}                    │ ─ Config                    │
│                                          │ Provider [auto ∨]           │
│                                          │   🟢 Serper  $0.001         │
│                                          │   🟢 Tavily  $0.008         │
│                                          │   🔥 Firecrawl (fetch)      │
│                                          │ Type   [web | news]         │
│                                          │ Full page [ ] (scrape)      │
│                                          │ Format [md|text|html]       │
│                                          │ Rerank model [∨]            │
└──────────────────────────────────────────┴─────────────────────────────┘

---

Tabs

Search Tab

Evolves the existing SearchForm + ResultsPanel + RerankPanel into a tab:

• Query → results (title, URL, snippet, relevance score).
• Provider metadata in Config pane (cost, quota, status).
• Rerank section: pick a rerank model, reorder results, show positionDelta.
• Empty state with CTA when no search providers are configured.
• Search history via SearchHistory.tsx.
• Calls POST /v1/search (existing endpoint, no changes).

Scrape Tab

New tab for extracting content from a URL via POST /v1/web/fetch (created in plan 05):

• Input: URL + full-page toggle + format selector (markdown / text / HTML).
• Submit → fetch → render ScrapeResult.tsx.
• ScrapeResult renders markdown preview + raw toggle.
• Cap: if response body > 256 KB, UI shows (truncated, view raw) and opens raw in a Monaco modal (D21).
• Metadata panel: provider (firecrawl/jina-reader/tavily-search), latency, cost, response size, links count.
• Uses useScrapeFetch.ts hook.

Compare Tab

Runs the same query/URL across up to 4 providers in parallel (D22):

• Side-by-side columns per provider.
• Metrics: latency, cost, result count, response size.
• URL overlap calculation for search (number of shared URLs vs initial result).
• Calls POST /v1/search (search) or POST /v1/web/fetch (scrape) per provider.

---

Config Pane (Shared)

SearchToolsConfigPane.tsx — always visible, collapsible.

Field	Notes
Provider	Dropdown with status badge (configured / missing / rate limited)
Type	web or news (search only)
Full page	Toggle for scrape — fetches entire page vs first visible content
Format	markdown, text, or html (scrape only)
Rerank model	Optional model for post-search reranking
History	Collapsible search history section

---

SearchConceptCard

SearchConceptCard.tsx — always visible, collapsible accordion. Explains:

Concept	One-liner
Search	Fetches a list of web results (title, URL, snippet, relevance score)
Scrape	Extracts the full content of a URL (markdown, text or HTML)
Compare	Runs the same query in N providers side-by-side
Rerank	Reorders results via LLM to improve query relevance
Auto (cheapest)	Picks the cheapest available provider automatically

---

Provider Catalog

ProviderCatalog.tsx exposes the full provider list from GET /api/search/providers (extended in F4 to include fetch providers):

Field	Source
id, name	searchRegistry.ts
kind	"search" (12 providers) or "fetch" (firecrawl, jina-reader, tavily-search)
costPerQuery	Registry data
freeMonthlyQuota	Registry data
searchTypes / fetchFormats	Registry data
status	"configured" / "missing" / "rate_limited" — derived at runtime from credential store
configureHref	/dashboard/providers

The status is derived at request time by checking whether credentials exist and whether all keys are currently in cooldown.

---

Export Code

ExportCodeModal (imported from Playground Studio) + codeExport.ts generate curl / Python / TypeScript snippets for both /v1/search and /v1/web/fetch calls. API key placeholder is always $OMNIROUTE_API_KEY (D11, shared with Playground Studio).

---

Backend Changes

Only one backend change was needed for this feature:

Extended GET /api/search/providers

src/app/api/search/providers/route.ts was extended to:

• Include all 3 fetch providers (firecrawl, jina-reader, tavily-search) in the array.
• Add kind: "search" | "fetch" to every item.
• Add status: "configured" | "missing" | "rate_limited" derived from live credential state.
• Maintain backward compatibility — existing fields (id, name, etc.) unchanged.

---

Key Files

Path	Purpose
src/app/(dashboard)/dashboard/search-tools/SearchToolsClient.tsx	Studio shell, tab orchestrator
src/app/(dashboard)/dashboard/search-tools/components/SearchToolsTopBar.tsx	Tabs + metrics + export button
src/app/(dashboard)/dashboard/search-tools/components/SearchToolsConfigPane.tsx	Shared config panel
src/app/(dashboard)/dashboard/search-tools/components/SearchConceptCard.tsx	Explainer cards (always visible)
src/app/(dashboard)/dashboard/search-tools/components/ProviderCatalog.tsx	Provider list with metadata
src/app/(dashboard)/dashboard/search-tools/components/ScrapeResult.tsx	Markdown preview + raw toggle
src/app/(dashboard)/dashboard/search-tools/components/tabs/SearchTab.tsx	Search + rerank tab
src/app/(dashboard)/dashboard/search-tools/components/tabs/ScrapeTab.tsx	Scrape tab
src/app/(dashboard)/dashboard/search-tools/components/tabs/CompareTab.tsx	Multi-provider compare tab
src/app/(dashboard)/dashboard/search-tools/hooks/useScrapeFetch.ts	Scrape fetch hook
src/app/api/search/providers/route.ts	Extended with kind + status + fetch providers
open-sse/config/searchRegistry.ts	Source of truth for search provider metadata

---

Troubleshooting

Symptom	Cause	Fix
Scrape tab shows "endpoint not available"	/v1/web/fetch not wired	Verify plan 05 is merged; check src/app/api/v1/web/fetch/route.ts exists
Provider catalog shows all as "missing"	Credentials not configured	Add credentials in /dashboard/providers
Scrape content is truncated	Response > 256 KB cap	Expected behavior (D21). Use "view raw" button for full content
Compare tab only shows 2 providers	Rate limit active	Two or more providers may be in cooldown — check provider status in Config pane
"Size" shown as raw key in table	Missing i18n key	Verify search.size exists in the locale file; rebuild i18n

---

References

• Master plan: _tasks/features-v3.8.6/refactorpages/_orchestration/master-plan-group-C.md
• Feature plan: _tasks/features-v3.8.6/refactorpages/18-search-tools-studio-redesign.plan.md
• Search provider registry: open-sse/config/searchRegistry.ts
• Playground Studio (shared ExportCodeModal + codeExport.ts): docs/frameworks/PLAYGROUND_STUDIO.md
• Web fetch backend: src/app/api/v1/web/fetch/route.ts

🏠 Home

🌍 Languages (40+)

---

🚀 Getting Started

• Setup Guide
• User Guide
• Features
• Quick Start (Docker)
• Electron Desktop App
• Termux (Android)
• PWA Guide

🌐 Providers

• Provider Reference
• All Providers
• Free Tiers
• Kiro Setup

🎯 Routing & Combos

• Auto-Combo
• Reasoning Replay

🗜️ Compression

• Compression Guide
• Compression Engines
• RTK Compression
• Language Packs
• Rules Format

🔌 Integrations

• MCP Server
• A2A Server
• Agent Protocols
• OpenCode Plugin
• Webhooks
• Cloud Agents
• Skills
• Memory
• Evals
• Gamification

🏗️ Architecture

• Architecture Overview
• Codebase Documentation
• Repository Map
• Authorization Guide
• Resilience Guide

🔒 Security

• Guardrails
• Compliance
• Error Sanitization
• Public Credentials
• Route Guard Tiers
• Stealth Guide
• CLI Token Auth

📋 Reference

• API Reference
• CLI Tools
• Environment Variables

🚀 Operations

• VM Deployment
• Fly.io Deployment
• Tunnels Guide
• Proxy Guide
• SQLite Runtime
• Coverage Plan
• Release Checklist

ℹ️ More

• Troubleshooting
• i18n / Translations
• Uninstall
• Contributing
• Comparison vs Alternatives

OmniRoute · Website · npm · Docker Hub