release: bump version to 0.50.0
This commit is contained in:
rayd1o
407
docs/plans/agents-light-orchestrator-websearch-plan.md
Normal file
407
docs/plans/agents-light-orchestrator-websearch-plan.md
Normal file
@@ -0,0 +1,407 @@
|
||||
# Lightweight Agent Orchestrator and WebSearch Evidence Plan
|
||||
|
||||
## Overview
|
||||
|
||||
Planet should not turn `aiprovider` into a general-purpose agent runtime.
|
||||
|
||||
`aiprovider` should remain the model gateway:
|
||||
|
||||
- provider compatibility
|
||||
- protocol adaptation
|
||||
- model authentication
|
||||
- request and response normalization
|
||||
|
||||
Agent behavior belongs in the backend, where Planet already owns business state,
|
||||
permissions, persistence, evidence records, and operator workflows.
|
||||
|
||||
The recommended direction is a lightweight backend Agent Orchestrator with a
|
||||
controlled tool layer. The first version should use fixed workflows instead of a
|
||||
free-form tool-calling loop.
|
||||
|
||||
|
||||
## Architecture Decision
|
||||
|
||||
Use this boundary:
|
||||
|
||||
```text
|
||||
aiprovider = model adapter only
|
||||
backend Agent = task orchestration + tools + evidence + policy + business rules
|
||||
```
|
||||
|
||||
This keeps model transport separate from Planet-specific behavior. It also lets
|
||||
OpenAI, MiniMax, Anthropic-compatible providers, Ollama, and later providers all
|
||||
reuse the same backend tools.
|
||||
|
||||
Recommended module shape:
|
||||
|
||||
```text
|
||||
backend/app/services/
|
||||
ai/
|
||||
agent_orchestrator.py
|
||||
tool_registry.py
|
||||
prompts.py
|
||||
schemas.py
|
||||
ai_tools/
|
||||
web_search.py
|
||||
web_fetch.py
|
||||
geo_resolve.py
|
||||
internal_data_query.py
|
||||
incident_query.py
|
||||
evidence_store.py
|
||||
situation/
|
||||
bgp_analyzer.py
|
||||
risk_scoring.py
|
||||
event_correlator.py
|
||||
alert_policy.py
|
||||
|
||||
aiprovider/
|
||||
provider_service.py
|
||||
main.py
|
||||
```
|
||||
|
||||
|
||||
## Phase 1: Controlled Workflow Agent
|
||||
|
||||
The first implementation should not be a full OpenClaw/Codex-style agent loop.
|
||||
Planet's immediate needs are better served by explicit workflows:
|
||||
|
||||
1. `tutorial_refresh`
|
||||
2. `geo_correction`
|
||||
3. `situation_brief`
|
||||
|
||||
Each workflow should:
|
||||
|
||||
1. collect evidence with backend tools
|
||||
2. normalize and store evidence
|
||||
3. call `AIProviderClient` through the configured global provider/model/key
|
||||
4. validate the result with Pydantic schemas
|
||||
5. return a proposal, candidate, or brief instead of directly mutating critical state
|
||||
|
||||
For location correction, the flow should be:
|
||||
|
||||
```text
|
||||
object name / type / current coordinate / description
|
||||
-> web_search
|
||||
-> web_fetch for selected results
|
||||
-> geo_resolve for city/site coordinates
|
||||
-> LLM structured extraction
|
||||
-> schema validation and confidence scoring
|
||||
-> pending review candidate
|
||||
```
|
||||
|
||||
The LLM output must be constrained to a schema such as:
|
||||
|
||||
```json
|
||||
{
|
||||
"object_id": "string",
|
||||
"object_type": "datacenter|ixp|submarine_cable|asn|city|facility|satellite",
|
||||
"current_location": {
|
||||
"lat": 0,
|
||||
"lon": 0
|
||||
},
|
||||
"suggested_location": {
|
||||
"lat": 0,
|
||||
"lon": 0
|
||||
},
|
||||
"confidence": 0.82,
|
||||
"reason": "short evidence-backed explanation",
|
||||
"evidence": [
|
||||
{
|
||||
"title": "source title",
|
||||
"url": "https://example.com/source",
|
||||
"quote": "short supporting excerpt",
|
||||
"retrieved_at": "2026-05-10T00:00:00Z"
|
||||
}
|
||||
],
|
||||
"needs_human_review": true
|
||||
}
|
||||
```
|
||||
|
||||
The LLM may generate a suggestion, but it must not directly write final
|
||||
coordinates into the dimension tables.
|
||||
|
||||
|
||||
## Phase 2: Backend Tool Registry
|
||||
|
||||
Add a small Python tool interface in the backend:
|
||||
|
||||
```python
|
||||
class ToolResult(BaseModel):
|
||||
ok: bool
|
||||
data: Any = None
|
||||
error: str | None = None
|
||||
evidence: list[dict] = []
|
||||
```
|
||||
|
||||
Register tools through a backend registry:
|
||||
|
||||
```text
|
||||
web_search
|
||||
web_fetch
|
||||
geo_resolve
|
||||
internal_data_query
|
||||
incident_query
|
||||
evidence_store
|
||||
```
|
||||
|
||||
Do not put WebSearch inside `aiprovider`.
|
||||
|
||||
Reasons:
|
||||
|
||||
- search is a business tool, not a model-provider feature
|
||||
- search evidence must be stored and audited by the backend
|
||||
- different LLM providers should share the same search pipeline
|
||||
- Planet may switch between Tavily, Brave, Exa, SearXNG, or MiniMax MCP without
|
||||
changing model transport
|
||||
|
||||
The first WebSearch implementation should be an HTTP evidence provider. Tavily is
|
||||
the recommended first default because it is simple to call from the existing
|
||||
`httpx` backend stack and returns LLM/RAG-friendly search results. The interface
|
||||
should remain provider-neutral so Brave, Exa, SearXNG, or MiniMax MCP can be
|
||||
added later.
|
||||
|
||||
WebSearch configuration should live under PostgreSQL `system_settings` with the
|
||||
rest of external integrations:
|
||||
|
||||
```text
|
||||
external_integrations.web_search
|
||||
enabled
|
||||
provider
|
||||
api_key
|
||||
base_url
|
||||
max_results
|
||||
timeout_seconds
|
||||
```
|
||||
|
||||
Secret resolution should follow the existing settings pattern:
|
||||
|
||||
1. saved PostgreSQL secret
|
||||
2. provider-specific environment variable, for example `TAVILY_API_KEY`
|
||||
3. generic fallback `WEB_SEARCH_API_KEY`
|
||||
|
||||
|
||||
## Phase 3: Limited Agent Loop
|
||||
|
||||
After the fixed workflows are stable, the backend can add a limited agent loop:
|
||||
|
||||
```text
|
||||
LLM sees an allowed tool list
|
||||
-> LLM requests a tool call
|
||||
-> backend validates and executes the tool
|
||||
-> tool result is added to context
|
||||
-> LLM continues
|
||||
-> final structured output after at most N steps
|
||||
```
|
||||
|
||||
Guardrails:
|
||||
|
||||
- max tool steps: 3 to 5
|
||||
- only read-only tools may run automatically
|
||||
- writes go to pending review first
|
||||
- all web evidence must be persisted
|
||||
- all final outputs must pass schema validation
|
||||
- prompts must include explicit evidence boundaries
|
||||
|
||||
Permission levels:
|
||||
|
||||
```text
|
||||
L0: pure analysis, no tools
|
||||
L1: read-only tools, web_search / web_fetch / internal_query
|
||||
L2: proposal generation, write pending review records
|
||||
L3: low-risk notifications and briefs
|
||||
L4: database mutation or alert triggering, human confirmation required
|
||||
```
|
||||
|
||||
|
||||
## Situational Awareness Boundary
|
||||
|
||||
Planet's situational-awareness layer should not rely on the LLM as the primary
|
||||
risk engine.
|
||||
|
||||
Use deterministic analysis for:
|
||||
|
||||
- anomaly type
|
||||
- affected prefixes
|
||||
- affected ASNs
|
||||
- geographic scope
|
||||
- duration
|
||||
- severity score
|
||||
- confidence
|
||||
- related events
|
||||
- raw evidence
|
||||
|
||||
Use the LLM for:
|
||||
|
||||
- readable summaries
|
||||
- risk explanation
|
||||
- likely impact narrative
|
||||
- next recommended actions
|
||||
- missing data requests
|
||||
|
||||
In short:
|
||||
|
||||
```text
|
||||
deterministic services compute the score
|
||||
LLM explains the evidence and options
|
||||
```
|
||||
|
||||
Proactive alerts should be triggered by deterministic rules or scheduled jobs,
|
||||
then optionally summarized by the Agent Orchestrator.
|
||||
|
||||
|
||||
## Persistence Model
|
||||
|
||||
Add lightweight persistence for auditability:
|
||||
|
||||
```text
|
||||
ai_tasks
|
||||
id
|
||||
task_type
|
||||
status
|
||||
input_json
|
||||
output_json
|
||||
model
|
||||
created_at
|
||||
finished_at
|
||||
error
|
||||
|
||||
ai_evidence
|
||||
id
|
||||
task_id
|
||||
source_type
|
||||
title
|
||||
url
|
||||
snippet
|
||||
content_hash
|
||||
retrieved_at
|
||||
credibility_score
|
||||
|
||||
ai_briefs
|
||||
id
|
||||
brief_type
|
||||
severity
|
||||
title
|
||||
summary
|
||||
evidence_ids
|
||||
related_entity_ids
|
||||
created_at
|
||||
acknowledged_at
|
||||
|
||||
ai_location_suggestions
|
||||
id
|
||||
object_type
|
||||
object_id
|
||||
old_lat
|
||||
old_lon
|
||||
new_lat
|
||||
new_lon
|
||||
confidence
|
||||
reason
|
||||
evidence_ids
|
||||
status
|
||||
```
|
||||
|
||||
The tables can be introduced incrementally. The first implementation may start
|
||||
with `ai_tasks` and `ai_evidence`, then add specialized tables when the UI needs
|
||||
review queues and acknowledgement state.
|
||||
|
||||
|
||||
## MVP Scope
|
||||
|
||||
The MVP should deliver three fixed capabilities:
|
||||
|
||||
### 1. Tutorial Refresh
|
||||
|
||||
Input:
|
||||
|
||||
- provider or tutorial topic
|
||||
- current tutorial text
|
||||
- known stale point, when available
|
||||
|
||||
Tools:
|
||||
|
||||
- `web_search`
|
||||
- `web_fetch`
|
||||
|
||||
Output:
|
||||
|
||||
- updated Markdown
|
||||
- source list
|
||||
- verification status
|
||||
|
||||
### 2. Geo Correction
|
||||
|
||||
Input:
|
||||
|
||||
- object id
|
||||
- object name
|
||||
- object type
|
||||
- current coordinates
|
||||
- source description
|
||||
|
||||
Tools:
|
||||
|
||||
- `web_search`
|
||||
- `web_fetch`
|
||||
- `geo_resolve`
|
||||
|
||||
Output:
|
||||
|
||||
- `LocationCorrection` JSON
|
||||
- evidence list
|
||||
- pending review candidate
|
||||
|
||||
### 3. Situation Brief
|
||||
|
||||
Input:
|
||||
|
||||
- anomaly event
|
||||
- deterministic findings
|
||||
- internal data summary
|
||||
|
||||
Tools:
|
||||
|
||||
- `internal_data_query`
|
||||
- optional `web_search`
|
||||
|
||||
Output:
|
||||
|
||||
- `SituationBrief` JSON
|
||||
- risk explanation
|
||||
- recommended actions
|
||||
- missing evidence list
|
||||
|
||||
|
||||
## Test Plan
|
||||
|
||||
Backend tests:
|
||||
|
||||
- WebSearch settings persist to `system_settings` and mask secrets in API responses.
|
||||
- Env fallback resolves provider-specific keys before `WEB_SEARCH_API_KEY`.
|
||||
- WebSearch provider normalizes success, empty results, 401, 429, and timeout responses.
|
||||
- `tutorial_refresh` uses evidence when available and marks output unverified when no evidence exists.
|
||||
- `geo_correction` returns pending review candidates and never writes final coordinates directly.
|
||||
- `situation_brief` accepts deterministic findings and returns schema-valid summaries.
|
||||
- Agent outputs fail closed when schema validation fails.
|
||||
|
||||
Frontend tests:
|
||||
|
||||
- WebSearch settings card shows configured state, masked key, connection test result, and save feedback.
|
||||
- Candidate review UI can display evidence links and pending location suggestions.
|
||||
- Situation brief UI can show evidence-backed summaries without exposing raw secrets.
|
||||
|
||||
Regression tests:
|
||||
|
||||
- existing `aiprovider` status and analysis calls remain unchanged
|
||||
- current LLM provider configuration remains the global model source
|
||||
- location pipeline tests continue to pass
|
||||
- datasource credential guide tests continue to pass
|
||||
|
||||
|
||||
## Assumptions
|
||||
|
||||
- `aiprovider` remains model-adapter-only.
|
||||
- Backend tools are implemented directly in Python first; MCP support is optional and later.
|
||||
- Search is evidence collection, not model transport.
|
||||
- Writes to important domain tables require human confirmation.
|
||||
- Deterministic analysis owns risk scores; LLM output is explanatory and evidence-backed.
|
||||
Reference in New Issue
Block a user