planet/docs/plans/agents-light-orchestrator-websearch-plan.md

# Lightweight Agent Orchestrator and WebSearch Evidence Plan

## Overview

Planet should not turn `aiprovider` into a general-purpose agent runtime.

`aiprovider` should remain the model gateway:

- provider compatibility
- protocol adaptation
- model authentication
- request and response normalization

Agent behavior belongs in the backend, where Planet already owns business state,
permissions, persistence, evidence records, and operator workflows.

The recommended direction is a lightweight backend Agent Orchestrator with a
controlled tool layer. The first version should use fixed workflows instead of a
free-form tool-calling loop.


## Architecture Decision

Use this boundary:

```text
aiprovider = model adapter only
backend Agent = task orchestration + tools + evidence + policy + business rules
```

This keeps model transport separate from Planet-specific behavior. It also lets
OpenAI, MiniMax, Anthropic-compatible providers, Ollama, and later providers all
reuse the same backend tools.

Recommended module shape:

```text
backend/app/services/
  ai/
    agent_orchestrator.py
    tool_registry.py
    prompts.py
    schemas.py
  ai_tools/
    web_search.py
    web_fetch.py
    geo_resolve.py
    internal_data_query.py
    incident_query.py
    evidence_store.py
  situation/
    bgp_analyzer.py
    risk_scoring.py
    event_correlator.py
    alert_policy.py

aiprovider/
  provider_service.py
  main.py
```


## Phase 1: Controlled Workflow Agent

The first implementation should not be a full OpenClaw/Codex-style agent loop.
Planet's immediate needs are better served by explicit workflows:

1. `tutorial_refresh`
2. `geo_correction`
3. `situation_brief`

Each workflow should:

1. collect evidence with backend tools
2. normalize and store evidence
3. call `AIProviderClient` through the configured global provider/model/key
4. validate the result with Pydantic schemas
5. return a proposal, candidate, or brief instead of directly mutating critical state

For location correction, the flow should be:

```text
object name / type / current coordinate / description
  -> web_search
  -> web_fetch for selected results
  -> geo_resolve for city/site coordinates
  -> LLM structured extraction
  -> schema validation and confidence scoring
  -> pending review candidate
```

The LLM output must be constrained to a schema such as:

```json
{
  "object_id": "string",
  "object_type": "datacenter|ixp|submarine_cable|asn|city|facility|satellite",
  "current_location": {
    "lat": 0,
    "lon": 0
  },
  "suggested_location": {
    "lat": 0,
    "lon": 0
  },
  "confidence": 0.82,
  "reason": "short evidence-backed explanation",
  "evidence": [
    {
      "title": "source title",
      "url": "https://example.com/source",
      "quote": "short supporting excerpt",
      "retrieved_at": "2026-05-10T00:00:00Z"
    }
  ],
  "needs_human_review": true
}
```

The LLM may generate a suggestion, but it must not directly write final
coordinates into the dimension tables.


## Phase 2: Backend Tool Registry

Add a small Python tool interface in the backend:

```python
class ToolResult(BaseModel):
    ok: bool
    data: Any = None
    error: str | None = None
    evidence: list[dict] = []
```

Register tools through a backend registry:

```text
web_search
web_fetch
geo_resolve
internal_data_query
incident_query
evidence_store
```

Do not put WebSearch inside `aiprovider`.

Reasons:

- search is a business tool, not a model-provider feature
- search evidence must be stored and audited by the backend
- different LLM providers should share the same search pipeline
- Planet may switch between Tavily, Brave, Exa, SearXNG, or MiniMax MCP without
  changing model transport

The first WebSearch implementation should be an HTTP evidence provider. Tavily is
the recommended first default because it is simple to call from the existing
`httpx` backend stack and returns LLM/RAG-friendly search results. The interface
should remain provider-neutral so Brave, Exa, SearXNG, or MiniMax MCP can be
added later.

WebSearch configuration should live under PostgreSQL `system_settings` with the
rest of external integrations:

```text
external_integrations.web_search
  enabled
  provider
  api_key
  base_url
  max_results
  timeout_seconds
```

Secret resolution should follow the existing settings pattern:

1. saved PostgreSQL secret
2. provider-specific environment variable, for example `TAVILY_API_KEY`
3. generic fallback `WEB_SEARCH_API_KEY`


## Phase 3: Limited Agent Loop

After the fixed workflows are stable, the backend can add a limited agent loop:

```text
LLM sees an allowed tool list
  -> LLM requests a tool call
  -> backend validates and executes the tool
  -> tool result is added to context
  -> LLM continues
  -> final structured output after at most N steps
```

Guardrails:

- max tool steps: 3 to 5
- only read-only tools may run automatically
- writes go to pending review first
- all web evidence must be persisted
- all final outputs must pass schema validation
- prompts must include explicit evidence boundaries

Permission levels:

```text
L0: pure analysis, no tools
L1: read-only tools, web_search / web_fetch / internal_query
L2: proposal generation, write pending review records
L3: low-risk notifications and briefs
L4: database mutation or alert triggering, human confirmation required
```


## Situational Awareness Boundary

Planet's situational-awareness layer should not rely on the LLM as the primary
risk engine.

Use deterministic analysis for:

- anomaly type
- affected prefixes
- affected ASNs
- geographic scope
- duration
- severity score
- confidence
- related events
- raw evidence

Use the LLM for:

- readable summaries
- risk explanation
- likely impact narrative
- next recommended actions
- missing data requests

In short:

```text
deterministic services compute the score
LLM explains the evidence and options
```

Proactive alerts should be triggered by deterministic rules or scheduled jobs,
then optionally summarized by the Agent Orchestrator.


## Persistence Model

Add lightweight persistence for auditability:

```text
ai_tasks
  id
  task_type
  status
  input_json
  output_json
  model
  created_at
  finished_at
  error

ai_evidence
  id
  task_id
  source_type
  title
  url
  snippet
  content_hash
  retrieved_at
  credibility_score

ai_briefs
  id
  brief_type
  severity
  title
  summary
  evidence_ids
  related_entity_ids
  created_at
  acknowledged_at

ai_location_suggestions
  id
  object_type
  object_id
  old_lat
  old_lon
  new_lat
  new_lon
  confidence
  reason
  evidence_ids
  status
```

The tables can be introduced incrementally. The first implementation may start
with `ai_tasks` and `ai_evidence`, then add specialized tables when the UI needs
review queues and acknowledgement state.


## MVP Scope

The MVP should deliver three fixed capabilities:

### 1. Tutorial Refresh

Input:

- provider or tutorial topic
- current tutorial text
- known stale point, when available

Tools:

- `web_search`
- `web_fetch`

Output:

- updated Markdown
- source list
- verification status

### 2. Geo Correction

Input:

- object id
- object name
- object type
- current coordinates
- source description

Tools:

- `web_search`
- `web_fetch`
- `geo_resolve`

Output:

- `LocationCorrection` JSON
- evidence list
- pending review candidate

### 3. Situation Brief

Input:

- anomaly event
- deterministic findings
- internal data summary

Tools:

- `internal_data_query`
- optional `web_search`

Output:

- `SituationBrief` JSON
- risk explanation
- recommended actions
- missing evidence list


## Test Plan

Backend tests:

- WebSearch settings persist to `system_settings` and mask secrets in API responses.
- Env fallback resolves provider-specific keys before `WEB_SEARCH_API_KEY`.
- WebSearch provider normalizes success, empty results, 401, 429, and timeout responses.
- `tutorial_refresh` uses evidence when available and marks output unverified when no evidence exists.
- `geo_correction` returns pending review candidates and never writes final coordinates directly.
- `situation_brief` accepts deterministic findings and returns schema-valid summaries.
- Agent outputs fail closed when schema validation fails.

Frontend tests:

- WebSearch settings card shows configured state, masked key, connection test result, and save feedback.
- Candidate review UI can display evidence links and pending location suggestions.
- Situation brief UI can show evidence-backed summaries without exposing raw secrets.

Regression tests:

- existing `aiprovider` status and analysis calls remain unchanged
- current LLM provider configuration remains the global model source
- location pipeline tests continue to pass
- datasource credential guide tests continue to pass


## Assumptions

- `aiprovider` remains model-adapter-only.
- Backend tools are implemented directly in Python first; MCP support is optional and later.
- Search is evidence collection, not model transport.
- Writes to important domain tables require human confirmation.
- Deterministic analysis owns risk scores; LLM output is explanatory and evidence-backed.