planet/docs/plans/agents-light-orchestrator-websearch-plan.md

# Lightweight Agent Orchestrator and WebSearch Evidence Plan

## Overview

Planet should not turn `aiprovider` into a general-purpose agent runtime.

`aiprovider` should remain the model gateway:

- provider compatibility
- protocol adaptation
- model authentication
- request and response normalization

Agent behavior belongs in the backend, where Planet already owns business state,
permissions, persistence, evidence records, and operator workflows.

The recommended direction is a lightweight backend Agent Orchestrator with a
controlled tool layer. The first version should use fixed workflows instead of a
free-form tool-calling loop.


## Architecture Decision

Use this boundary:

```text
aiprovider = model adapter only
backend Agent = task orchestration + tools + evidence + policy + business rules
```

This keeps model transport separate from Planet-specific behavior. It also lets
OpenAI, MiniMax, Anthropic-compatible providers, Ollama, and later providers all
reuse the same backend tools.

Recommended module shape:

```text
backend/app/services/
  ai/
    agent_orchestrator.py
    tool_registry.py
    prompts.py
    schemas.py
  ai_tools/
    web_search.py
    web_fetch.py
    geo_resolve.py
    internal_data_query.py
    incident_query.py
    evidence_store.py
  situation/
    bgp_analyzer.py
    risk_scoring.py
    event_correlator.py
    alert_policy.py

aiprovider/
  provider_service.py
  main.py
```


## Phase 1: Controlled Workflow Agent

The first implementation should not be a full OpenClaw/Codex-style agent loop.
Planet's immediate needs are better served by explicit workflows:

1. `tutorial_refresh`
2. `geo_correction`
3. `situation_brief`

Each workflow should:

1. collect evidence with backend tools
2. normalize and store evidence
3. call `AIProviderClient` through the configured global provider/model/key
4. validate the result with Pydantic schemas
5. return a proposal, candidate, or brief instead of directly mutating critical state

For location correction, the flow should be:

```text
object name / type / current coordinate / description
  -> web_search
  -> web_fetch for selected results
  -> geo_resolve for city/site coordinates
  -> LLM structured extraction
  -> schema validation and confidence scoring
  -> pending review candidate
```

The LLM output must be constrained to a schema such as:

```json
{
  "object_id": "string",
  "object_type": "datacenter|ixp|submarine_cable|asn|city|facility|satellite",
  "current_location": {
    "lat": 0,
    "lon": 0
  },
  "suggested_location": {
    "lat": 0,
    "lon": 0
  },
  "confidence": 0.82,
  "reason": "short evidence-backed explanation",
  "evidence": [
    {
      "title": "source title",
      "url": "https://example.com/source",
      "quote": "short supporting excerpt",
      "retrieved_at": "2026-05-10T00:00:00Z"
    }
  ],
  "needs_human_review": true
}
```

The LLM may generate a suggestion, but it must not directly write final
coordinates into the dimension tables.


## Phase 2: Backend Tool Registry

Add a small Python tool interface in the backend:

```python
class ToolResult(BaseModel):
    ok: bool
    data: Any = None
    error: str | None = None
    evidence: list[dict] = []
```

Register tools through a backend registry:

```text
web_search
web_fetch
geo_resolve
internal_data_query
incident_query
evidence_store
```

Do not put WebSearch inside `aiprovider`.

Reasons:

- search is a business tool, not a model-provider feature
- search evidence must be stored and audited by the backend
- different LLM providers should share the same search pipeline
- Planet may switch between Tavily, Brave, Exa, SearXNG, or MiniMax MCP without
  changing model transport

The first WebSearch implementation should be an HTTP evidence provider. Tavily is
the recommended first default because it is simple to call from the existing
`httpx` backend stack and returns LLM/RAG-friendly search results. The interface
should remain provider-neutral so Brave, Exa, SearXNG, or MiniMax MCP can be
added later.

WebSearch configuration should live under PostgreSQL `system_settings` with the
rest of external integrations:

```text
external_integrations.web_search
  enabled
  provider
  api_key
  base_url
  max_results
  timeout_seconds
```

Secret resolution should follow the existing settings pattern:

1. saved PostgreSQL secret
2. provider-specific environment variable, for example `TAVILY_API_KEY`
3. generic fallback `WEB_SEARCH_API_KEY`

### Common WebSearch Providers

The first implementation should model WebSearch as a provider-specific adapter
behind one internal interface:

```text
SearchEvidenceProvider.search(query, max_results, domains, freshness_days)
  -> list[SearchEvidence]
```

Recommended provider ids and environment variables:

| Provider | Provider id | Env key | Default base URL | Primary use |
| --- | --- | --- | --- | --- |
| Tavily | `tavily` | `TAVILY_API_KEY` | `https://api.tavily.com` | Default hosted search for agent/RAG style results |
| Brave Search API | `brave` | `BRAVE_SEARCH_API_KEY` | `https://api.search.brave.com` | Independent web index and low-level SERP results |
| SerpAPI | `serpapi` | `SERPAPI_API_KEY` | `https://serpapi.com` | Search-engine-backed SERP data with engine options |
| Exa | `exa` | `EXA_API_KEY` | `https://api.exa.ai` | Neural/semantic web search and result contents |
| Firecrawl Search / Scrape | `firecrawl` | `FIRECRAWL_API_KEY` | `https://api.firecrawl.dev` | Search plus page scrape/markdown extraction |
| SearXNG | `searxng` | optional `SEARXNG_API_KEY` | self-hosted instance URL | Self-hosted metasearch when external search APIs are undesirable |

The normalized configuration should support per-provider defaults while keeping
one active provider:

```text
external_integrations.web_search
  enabled: true
  default_provider: tavily
  providers:
    tavily:
      base_url: https://api.tavily.com
      api_key: <secret>
      max_results: 5
      search_depth: basic
      include_answer: false
      include_raw_content: false
    brave:
      base_url: https://api.search.brave.com
      api_key: <secret>
      endpoint_path: /res/v1/web/search
      max_results: 5
    serpapi:
      base_url: https://serpapi.com
      api_key: <secret>
      endpoint_path: /search.json
      engine: google
      max_results: 5
    exa:
      base_url: https://api.exa.ai
      api_key: <secret>
      endpoint_path: /search
      max_results: 5
      include_text: false
    firecrawl:
      base_url: https://api.firecrawl.dev
      api_key: <secret>
      search_path: /v2/search
      scrape_path: /v2/scrape
      max_results: 5
      scrape_formats: [markdown]
    searxng:
      base_url: http://localhost:8080
      api_key: <optional secret>
      endpoint_path: /
      max_results: 5
      categories: general
      engines: []
```

Adapter notes:

- Tavily should call `/search` and normalize title, URL, snippet/content, score,
  and optional raw content.
- Brave should call `/res/v1/web/search` and map web results into the same
  `SearchEvidence` shape.
- SerpAPI should call `/search.json`, pass `engine`, and normalize organic
  results. Search-engine-specific fields should remain in provider metadata.
- Exa should call `/search`; optional result text should be treated as fetched
  content only when enabled.
- Firecrawl can be used both as `web_search` and `web_fetch`: `/v2/search`
  returns result URLs/descriptions and may include scrape options, while
  `/v2/scrape` can produce markdown for a selected URL.
- SearXNG should query the configured instance with `q` and `format=json`.
  Public instances should not be assumed reliable for production; a controlled
  self-hosted instance is preferred.

The settings UI should expose only provider, base URL, key, max results, and a
test button in the first version. Provider-specific advanced fields can stay
collapsed or backend-only until a real workflow needs them.

### Frontend Configuration Window

Add a WebSearch configuration panel to the existing settings page, next to the
LLM provider configuration. It should behave like the current AI provider secret
controls: clear configured state, masked preview, explicit show/hide, test
connection, and save feedback.

First-version visible fields:

```text
WebSearch Provider
API Base URL
API Key
Max Results
Timeout Seconds
Enable WebSearch
Test Connection
Save
```

Provider dropdown options:

```text
Tavily
Brave Search API
SerpAPI
Exa
Firecrawl Search / Scrape
SearXNG
```

Field behavior:

- Switching provider loads that provider's saved config and masked key preview.
- Empty key input means keep the existing saved or environment key.
- Typing a new key replaces only the selected provider's key.
- Show key reveals the full current input value when the backend reveal endpoint
  allows it; hide key returns to the prefix-preserving masked preview.
- The configured badge should only show `已配置` or `未配置`, not repeat the
  masked key text.
- `Test Connection` sends the current unsaved draft to the backend and should
  not require a separate save first.
- A successful test may save the draft as the new WebSearch default only if the
  API endpoint is explicitly designed to mirror the AI provider test behavior.
  Otherwise, test should be read-only and the Save button should persist.
- Save success and test success must show visible feedback. Failures should show
  provider-specific but secret-safe error messages.

Provider-specific UI hints:

| Provider | UI hint |
| --- | --- |
| Tavily | Good default for agent/RAG style search. |
| Brave Search API | Uses Brave's independent search index. |
| SerpAPI | Supports search-engine-specific parameters such as `engine`. |
| Exa | Good for semantic search and optional result text. |
| Firecrawl | Can search and scrape pages into markdown. |
| SearXNG | Requires a reachable self-hosted or trusted instance URL. |

Advanced fields can live in a collapsed section:

```text
Endpoint Path
Search Depth
Engine
Categories
Engines
Include Raw Content
Scrape Formats
Domain Allowlist
```

The first version should keep the UI conservative. It should not expose every
provider knob until backend workflows use those knobs.

### Web Fetch and Page Extraction

`web_fetch` is separate from `web_search`. Search finds candidate URLs; fetch
turns selected pages into clean, citable evidence.

Recommended extraction chain:

```text
1. plain httpx fetch
2. trafilatura extraction for static HTML
3. readability extraction as secondary cleanup
4. Playwright fetch only for allowlisted JS-heavy pages
5. Firecrawl scrape as hosted fallback when configured
```

Implementation guidance:

- Use `trafilatura` as the first local extractor because it is Python-native and
  matches the backend stack.
- Prefer a Python readability implementation for local cleanup. Do not introduce
  a Node-only readability dependency for backend fetch.
- Use Playwright sparingly for JavaScript-rendered pages. It should have domain
  allowlists, low concurrency, strict timeouts, response size limits, and no
  automatic form submission or login behavior.
- Store `content_hash`, `retrieved_at`, final URL, title, extracted text
  preview, and extractor name in `ai_evidence`.
- Keep short quotes for UI review, but do not store huge page bodies directly in
  every task record. Large extracted content should be truncated or stored once
  by hash.

The local/self-hosted stack should look like this:

```text
SearXNG
  -> SearchEvidence URLs
  -> httpx fetch
  -> trafilatura / readability
  -> Playwright only when static extraction fails and the domain is allowed
  -> normalized evidence
  -> LLM structured output through AIProviderClient
```

This route gives Planet a lower-cost and more controllable search path, while
hosted providers remain available when search quality or maintenance effort
matters more than self-hosting.


## Phase 3: Limited Agent Loop

After the fixed workflows are stable, the backend can add a limited agent loop:

```text
LLM sees an allowed tool list
  -> LLM requests a tool call
  -> backend validates and executes the tool
  -> tool result is added to context
  -> LLM continues
  -> final structured output after at most N steps
```

Guardrails:

- max tool steps: 3 to 5
- only read-only tools may run automatically
- writes go to pending review first
- all web evidence must be persisted
- all final outputs must pass schema validation
- prompts must include explicit evidence boundaries

Permission levels:

```text
L0: pure analysis, no tools
L1: read-only tools, web_search / web_fetch / internal_query
L2: proposal generation, write pending review records
L3: low-risk notifications and briefs
L4: database mutation or alert triggering, human confirmation required
```


## Situational Awareness Boundary

Planet's situational-awareness layer should not rely on the LLM as the primary
risk engine.

Use deterministic analysis for:

- anomaly type
- affected prefixes
- affected ASNs
- geographic scope
- duration
- severity score
- confidence
- related events
- raw evidence

Use the LLM for:

- readable summaries
- risk explanation
- likely impact narrative
- next recommended actions
- missing data requests

In short:

```text
deterministic services compute the score
LLM explains the evidence and options
```

Proactive alerts should be triggered by deterministic rules or scheduled jobs,
then optionally summarized by the Agent Orchestrator.


## Persistence Model

Add lightweight persistence for auditability:

```text
ai_tasks
  id
  task_type
  status
  input_json
  output_json
  model
  created_at
  finished_at
  error

ai_evidence
  id
  task_id
  source_type
  title
  url
  snippet
  content_hash
  retrieved_at
  credibility_score

ai_briefs
  id
  brief_type
  severity
  title
  summary
  evidence_ids
  related_entity_ids
  created_at
  acknowledged_at

ai_location_suggestions
  id
  object_type
  object_id
  old_lat
  old_lon
  new_lat
  new_lon
  confidence
  reason
  evidence_ids
  status
```

The tables can be introduced incrementally. The first implementation may start
with `ai_tasks` and `ai_evidence`, then add specialized tables when the UI needs
review queues and acknowledgement state.


## MVP Scope

The MVP should deliver three fixed capabilities:

### 1. Tutorial Refresh

Input:

- provider or tutorial topic
- current tutorial text
- known stale point, when available

Tools:

- `web_search`
- `web_fetch`

Output:

- updated Markdown
- source list
- verification status

### 2. Geo Correction

Input:

- object id
- object name
- object type
- current coordinates
- source description

Tools:

- `web_search`
- `web_fetch`
- `geo_resolve`

Output:

- `LocationCorrection` JSON
- evidence list
- pending review candidate

### 3. Situation Brief

Input:

- anomaly event
- deterministic findings
- internal data summary

Tools:

- `internal_data_query`
- optional `web_search`

Output:

- `SituationBrief` JSON
- risk explanation
- recommended actions
- missing evidence list


## Test Plan

Backend tests:

- WebSearch settings persist to `system_settings` and mask secrets in API responses.
- Env fallback resolves provider-specific keys before `WEB_SEARCH_API_KEY`.
- WebSearch provider normalizes success, empty results, 401, 429, and timeout responses.
- `tutorial_refresh` uses evidence when available and marks output unverified when no evidence exists.
- `geo_correction` returns pending review candidates and never writes final coordinates directly.
- `situation_brief` accepts deterministic findings and returns schema-valid summaries.
- Agent outputs fail closed when schema validation fails.

Frontend tests:

- WebSearch settings card shows configured state, masked key, connection test result, and save feedback.
- Candidate review UI can display evidence links and pending location suggestions.
- Situation brief UI can show evidence-backed summaries without exposing raw secrets.

Regression tests:

- existing `aiprovider` status and analysis calls remain unchanged
- current LLM provider configuration remains the global model source
- location pipeline tests continue to pass
- datasource credential guide tests continue to pass


## Assumptions

- `aiprovider` remains model-adapter-only.
- Backend tools are implemented directly in Python first; MCP support is optional and later.
- Search is evidence collection, not model transport.
- Writes to important domain tables require human confirmation.
- Deterministic analysis owns risk scores; LLM output is explanatory and evidence-backed.