planet/docs/plans/agents-light-orchestrator-websearch-plan.md

Lightweight Agent Orchestrator and WebSearch Evidence Plan

Overview

Planet should not turn aiprovider into a general-purpose agent runtime.

aiprovider should remain the model gateway:

• provider compatibility
• protocol adaptation
• model authentication
• request and response normalization

Agent behavior belongs in the backend, where Planet already owns business state, permissions, persistence, evidence records, and operator workflows.

The recommended direction is a lightweight backend Agent Orchestrator with a controlled tool layer. The first version should use fixed workflows instead of a free-form tool-calling loop.

Architecture Decision

Use this boundary:

aiprovider = model adapter only
backend Agent = task orchestration + tools + evidence + policy + business rules

This keeps model transport separate from Planet-specific behavior. It also lets OpenAI, MiniMax, Anthropic-compatible providers, Ollama, and later providers all reuse the same backend tools.

Recommended module shape:

backend/app/services/
  ai/
    agent_orchestrator.py
    tool_registry.py
    prompts.py
    schemas.py
  ai_tools/
    web_search.py
    web_fetch.py
    geo_resolve.py
    internal_data_query.py
    incident_query.py
    evidence_store.py
  situation/
    bgp_analyzer.py
    risk_scoring.py
    event_correlator.py
    alert_policy.py

aiprovider/
  provider_service.py
  main.py

Phase 1: Controlled Workflow Agent

The first implementation should not be a full OpenClaw/Codex-style agent loop. Planet's immediate needs are better served by explicit workflows:

1. tutorial_refresh
2. geo_correction
3. situation_brief

Each workflow should:

1. collect evidence with backend tools
2. normalize and store evidence
3. call AIProviderClient through the configured global provider/model/key
4. validate the result with Pydantic schemas
5. return a proposal, candidate, or brief instead of directly mutating critical state

For location correction, the flow should be:

object name / type / current coordinate / description
  -> web_search
  -> web_fetch for selected results
  -> geo_resolve for city/site coordinates
  -> LLM structured extraction
  -> schema validation and confidence scoring
  -> pending review candidate

The LLM output must be constrained to a schema such as:

{
  "object_id": "string",
  "object_type": "datacenter|ixp|submarine_cable|asn|city|facility|satellite",
  "current_location": {
    "lat": 0,
    "lon": 0
  },
  "suggested_location": {
    "lat": 0,
    "lon": 0
  },
  "confidence": 0.82,
  "reason": "short evidence-backed explanation",
  "evidence": [
    {
      "title": "source title",
      "url": "https://example.com/source",
      "quote": "short supporting excerpt",
      "retrieved_at": "2026-05-10T00:00:00Z"
    }
  ],
  "needs_human_review": true
}

The LLM may generate a suggestion, but it must not directly write final coordinates into the dimension tables.

Phase 2: Backend Tool Registry

Add a small Python tool interface in the backend:

class ToolResult(BaseModel):
    ok: bool
    data: Any = None
    error: str | None = None
    evidence: list[dict] = []

Register tools through a backend registry:

web_search
web_fetch
geo_resolve
internal_data_query
incident_query
evidence_store

Do not put WebSearch inside aiprovider.

Reasons:

• search is a business tool, not a model-provider feature
• search evidence must be stored and audited by the backend
• different LLM providers should share the same search pipeline
• Planet may switch between Tavily, Brave, Exa, SearXNG, or MiniMax MCP without changing model transport

The first WebSearch implementation should be an HTTP evidence provider. Tavily is the recommended first default because it is simple to call from the existing httpx backend stack and returns LLM/RAG-friendly search results. The interface should remain provider-neutral so Brave, Exa, SearXNG, or MiniMax MCP can be added later.

WebSearch configuration should live under PostgreSQL system_settings with the rest of external integrations:

external_integrations.web_search
  enabled
  provider
  api_key
  base_url
  max_results
  timeout_seconds

Secret resolution should follow the existing settings pattern:

1. saved PostgreSQL secret
2. provider-specific environment variable, for example TAVILY_API_KEY
3. generic fallback WEB_SEARCH_API_KEY

Common WebSearch Providers

The first implementation should model WebSearch as a provider-specific adapter behind one internal interface:

SearchEvidenceProvider.search(query, max_results, domains, freshness_days)
  -> list[SearchEvidence]

Recommended provider ids and environment variables:

Provider	Provider id	Env key	Default base URL	Primary use
Tavily	tavily	TAVILY_API_KEY	https://api.tavily.com	Default hosted search for agent/RAG style results
Brave Search API	brave	BRAVE_SEARCH_API_KEY	https://api.search.brave.com	Independent web index and low-level SERP results
SerpAPI	serpapi	SERPAPI_API_KEY	https://serpapi.com	Search-engine-backed SERP data with engine options
Exa	exa	EXA_API_KEY	https://api.exa.ai	Neural/semantic web search and result contents
Firecrawl Search / Scrape	firecrawl	FIRECRAWL_API_KEY	https://api.firecrawl.dev	Search plus page scrape/markdown extraction
SearXNG	searxng	optional SEARXNG_API_KEY	self-hosted instance URL	Self-hosted metasearch when external search APIs are undesirable

The normalized configuration should support per-provider defaults while keeping one active provider:

external_integrations.web_search
  enabled: true
  default_provider: tavily
  providers:
    tavily:
      base_url: https://api.tavily.com
      api_key: <secret>
      max_results: 5
      search_depth: basic
      include_answer: false
      include_raw_content: false
    brave:
      base_url: https://api.search.brave.com
      api_key: <secret>
      endpoint_path: /res/v1/web/search
      max_results: 5
    serpapi:
      base_url: https://serpapi.com
      api_key: <secret>
      endpoint_path: /search.json
      engine: google
      max_results: 5
    exa:
      base_url: https://api.exa.ai
      api_key: <secret>
      endpoint_path: /search
      max_results: 5
      include_text: false
    firecrawl:
      base_url: https://api.firecrawl.dev
      api_key: <secret>
      search_path: /v2/search
      scrape_path: /v2/scrape
      max_results: 5
      scrape_formats: [markdown]
    searxng:
      base_url: http://localhost:8080
      api_key: <optional secret>
      endpoint_path: /
      max_results: 5
      categories: general
      engines: []

Adapter notes:

• Tavily should call /search and normalize title, URL, snippet/content, score, and optional raw content.
• Brave should call /res/v1/web/search and map web results into the same SearchEvidence shape.
• SerpAPI should call /search.json, pass engine, and normalize organic results. Search-engine-specific fields should remain in provider metadata.
• Exa should call /search; optional result text should be treated as fetched content only when enabled.
• Firecrawl can be used both as web_search and web_fetch: /v2/search returns result URLs/descriptions and may include scrape options, while /v2/scrape can produce markdown for a selected URL.
• SearXNG should query the configured instance with q and format=json. Public instances should not be assumed reliable for production; a controlled self-hosted instance is preferred.

The settings UI should expose only provider, base URL, key, max results, and a test button in the first version. Provider-specific advanced fields can stay collapsed or backend-only until a real workflow needs them.

Frontend Configuration Window

Add a WebSearch configuration panel to the existing settings page, next to the LLM provider configuration. It should behave like the current AI provider secret controls: clear configured state, masked preview, explicit show/hide, test connection, and save feedback.

First-version visible fields:

WebSearch Provider
API Base URL
API Key
Max Results
Timeout Seconds
Enable WebSearch
Test Connection
Save

Provider dropdown options:

Tavily
Brave Search API
SerpAPI
Exa
Firecrawl Search / Scrape
SearXNG

Field behavior:

• Switching provider loads that provider's saved config and masked key preview.
• Empty key input means keep the existing saved or environment key.
• Typing a new key replaces only the selected provider's key.
• Show key reveals the full current input value when the backend reveal endpoint allows it; hide key returns to the prefix-preserving masked preview.
• The configured badge should only show 已配置 or 未配置, not repeat the masked key text.
• Test Connection sends the current unsaved draft to the backend and should not require a separate save first.
• A successful test may save the draft as the new WebSearch default only if the API endpoint is explicitly designed to mirror the AI provider test behavior. Otherwise, test should be read-only and the Save button should persist.
• Save success and test success must show visible feedback. Failures should show provider-specific but secret-safe error messages.

Provider-specific UI hints:

Provider	UI hint
Tavily	Good default for agent/RAG style search.
Brave Search API	Uses Brave's independent search index.
SerpAPI	Supports search-engine-specific parameters such as engine.
Exa	Good for semantic search and optional result text.
Firecrawl	Can search and scrape pages into markdown.
SearXNG	Requires a reachable self-hosted or trusted instance URL.

Advanced fields can live in a collapsed section:

Endpoint Path
Search Depth
Engine
Categories
Engines
Include Raw Content
Scrape Formats
Domain Allowlist

The first version should keep the UI conservative. It should not expose every provider knob until backend workflows use those knobs.

Web Fetch and Page Extraction

web_fetch is separate from web_search. Search finds candidate URLs; fetch turns selected pages into clean, citable evidence.

Recommended extraction chain:

1. plain httpx fetch
2. trafilatura extraction for static HTML
3. readability extraction as secondary cleanup
4. Playwright fetch only for allowlisted JS-heavy pages
5. Firecrawl scrape as hosted fallback when configured

Implementation guidance:

• Use trafilatura as the first local extractor because it is Python-native and matches the backend stack.
• Prefer a Python readability implementation for local cleanup. Do not introduce a Node-only readability dependency for backend fetch.
• Use Playwright sparingly for JavaScript-rendered pages. It should have domain allowlists, low concurrency, strict timeouts, response size limits, and no automatic form submission or login behavior.
• Store content_hash, retrieved_at, final URL, title, extracted text preview, and extractor name in ai_evidence.
• Keep short quotes for UI review, but do not store huge page bodies directly in every task record. Large extracted content should be truncated or stored once by hash.

The local/self-hosted stack should look like this:

SearXNG
  -> SearchEvidence URLs
  -> httpx fetch
  -> trafilatura / readability
  -> Playwright only when static extraction fails and the domain is allowed
  -> normalized evidence
  -> LLM structured output through AIProviderClient

This route gives Planet a lower-cost and more controllable search path, while hosted providers remain available when search quality or maintenance effort matters more than self-hosting.

Phase 3: Limited Agent Loop

After the fixed workflows are stable, the backend can add a limited agent loop:

LLM sees an allowed tool list
  -> LLM requests a tool call
  -> backend validates and executes the tool
  -> tool result is added to context
  -> LLM continues
  -> final structured output after at most N steps

Guardrails:

• max tool steps: 3 to 5
• only read-only tools may run automatically
• writes go to pending review first
• all web evidence must be persisted
• all final outputs must pass schema validation
• prompts must include explicit evidence boundaries

Permission levels:

L0: pure analysis, no tools
L1: read-only tools, web_search / web_fetch / internal_query
L2: proposal generation, write pending review records
L3: low-risk notifications and briefs
L4: database mutation or alert triggering, human confirmation required

Situational Awareness Boundary

Planet's situational-awareness layer should not rely on the LLM as the primary risk engine.

Use deterministic analysis for:

• anomaly type
• affected prefixes
• affected ASNs
• geographic scope
• duration
• severity score
• confidence
• related events
• raw evidence

Use the LLM for:

• readable summaries
• risk explanation
• likely impact narrative
• next recommended actions
• missing data requests

In short:

deterministic services compute the score
LLM explains the evidence and options

Proactive alerts should be triggered by deterministic rules or scheduled jobs, then optionally summarized by the Agent Orchestrator.

Persistence Model

Add lightweight persistence for auditability:

ai_tasks
  id
  task_type
  status
  input_json
  output_json
  model
  created_at
  finished_at
  error

ai_evidence
  id
  task_id
  source_type
  title
  url
  snippet
  content_hash
  retrieved_at
  credibility_score

ai_briefs
  id
  brief_type
  severity
  title
  summary
  evidence_ids
  related_entity_ids
  created_at
  acknowledged_at

ai_location_suggestions
  id
  object_type
  object_id
  old_lat
  old_lon
  new_lat
  new_lon
  confidence
  reason
  evidence_ids
  status

The tables can be introduced incrementally. The first implementation may start with ai_tasks and ai_evidence, then add specialized tables when the UI needs review queues and acknowledgement state.

MVP Scope

The MVP should deliver three fixed capabilities:

1. Tutorial Refresh

Input:

• provider or tutorial topic
• current tutorial text
• known stale point, when available

Tools:

• web_search
• web_fetch

Output:

• updated Markdown
• source list
• verification status

2. Geo Correction

Input:

• object id
• object name
• object type
• current coordinates
• source description

Tools:

• web_search
• web_fetch
• geo_resolve

Output:

• LocationCorrection JSON
• evidence list
• pending review candidate

3. Situation Brief

Input:

• anomaly event
• deterministic findings
• internal data summary

Tools:

• internal_data_query
• optional web_search

Output:

• SituationBrief JSON
• risk explanation
• recommended actions
• missing evidence list

Test Plan

Backend tests:

• WebSearch settings persist to system_settings and mask secrets in API responses.
• Env fallback resolves provider-specific keys before WEB_SEARCH_API_KEY.
• WebSearch provider normalizes success, empty results, 401, 429, and timeout responses.
• tutorial_refresh uses evidence when available and marks output unverified when no evidence exists.
• geo_correction returns pending review candidates and never writes final coordinates directly.
• situation_brief accepts deterministic findings and returns schema-valid summaries.
• Agent outputs fail closed when schema validation fails.

Frontend tests:

• WebSearch settings card shows configured state, masked key, connection test result, and save feedback.
• Candidate review UI can display evidence links and pending location suggestions.
• Situation brief UI can show evidence-backed summaries without exposing raw secrets.

Regression tests:

• existing aiprovider status and analysis calls remain unchanged
• current LLM provider configuration remains the global model source
• location pipeline tests continue to pass
• datasource credential guide tests continue to pass

Assumptions

• aiprovider remains model-adapter-only.
• Backend tools are implemented directly in Python first; MCP support is optional and later.
• Search is evidence collection, not model transport.
• Writes to important domain tables require human confirmation.
• Deterministic analysis owns risk scores; LLM output is explanatory and evidence-backed.