linkong/planet
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

docs: add agent runtime planning docs

Browse Source
This commit is contained in:
linkong
2026-04-08 12:49:49 +08:00
parent 8bd9d34376
commit d395769df6
3 changed files with 1471 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

647
docs/agent-architecture-plan.md Normal file
View File

@@ -0,0 +1,647 @@
# Agent Architecture Plan
## Overview
This document defines the agent architecture for Planet.
The architecture is intentionally broader than datasource health checking.
It is designed to support both:
- datasource health governance
- future situational-awareness workflows
The core idea is to avoid building a one-off "repair broken API links" agent.
Instead, Planet should grow a reusable agent runtime that can:
- collect evidence
- evaluate signals
- reason over incomplete information
- generate proposals
- produce assessments
- execute limited actions under policy
## Design Goal
Build an agent foundation that can evolve in this order:
1. datasource health checks
2. datasource repair proposals
3. signal correlation
4. situational assessments
5. controlled runtime actions
This means the architecture should treat datasource health as one use case of the larger agent system, not as the whole system.
## Core Principles
1. Separate evidence from reasoning
- raw signals should be gathered first
- deterministic checks should run before LLM reasoning
2. Agents do not own the defaults
- repository defaults remain human-owned
- agents operate on runtime state, proposals, and overrides
3. Reasoning and action are different responsibilities
- many agents should be read-only or propose-only
- only tightly controlled flows may apply changes
4. Shared runtime, specialized roles
- multiple agent roles should share the same object model and orchestration patterns
- health and situational-awareness agents should not invent incompatible payloads
5. Auditability is mandatory
- every proposal, assessment, and applied action should be attributable
## System Layers
Planet agent architecture should be split into four layers.
### 1. Signal Layer
Purpose:
- gather raw evidence from internal and external systems
Example sources:
- collector outputs
- datasource health checks
- logs
- snapshots
- alerts
- web search results
- scraped pages
- external APIs
- operator inputs
Responsibilities:
- fetch
- normalize
- timestamp
- tag with source and trust level
This layer should not make high-level judgments.
### 2. Evaluation Layer
Purpose:
- perform deterministic analysis
Examples:
- reachability checks
- schema validation
- threshold checks
- time-window comparisons
- anomaly counters
- completeness checks
Responsibilities:
- classify signals into machine-readable findings
- attach deterministic evidence
This layer should avoid LLM dependency whenever possible.
### 3. Reasoning Layer
Purpose:
- use LLMs when semantic interpretation or incomplete-information reasoning is needed
Examples:
- endpoint migration inference
- multi-source event correlation
- causality hypotheses
- ambiguity reduction
- assessment narrative generation
- action recommendation generation
Responsibilities:
- synthesize evidence
- produce hypotheses
- rank confidence
- explain reasoning boundaries
This is the main place where `aiprovider` and web search are used.
### 4. Action Layer
Purpose:
- convert proposals or assessments into controlled system actions
Examples:
- create runtime override
- create proposal
- publish alert
- update operator task queue
- generate summary artifact
- trigger follow-up verification
Responsibilities:
- enforce policy
- enforce approval requirements
- verify post-action outcomes
- record audit trails
## Architecture Sketch
```mermaid
flowchart TD
A["Collectors / Logs / Snapshots / External APIs"] --> B["Signal Layer"]
W["Web Search / Page Fetch / Docs"] --> B
B --> C["Evaluation Layer"]
C --> D["Findings"]
D --> E["Reasoning Layer (LLM + Tools)"]
E --> F["Proposals"]
E --> G["Assessments"]
F --> H["Action Layer"]
H --> I["Runtime Overrides / Alerts / Tasks"]
H --> J["Verification Loop"]
J --> B
K["Policy Engine"] --> H
L["Audit / History Store"] --> H
L --> E
L --> C
```
## Agent Roles
The first version should define these logical roles.
### 1. Health Agent
Primary use case:
- datasource health governance
Inputs:
- datasource metadata
- current endpoint
- latest health records
- latest failures
- deterministic findings
Outputs:
- health interpretation
- repair proposal
- confidence
- evidence references
Typical action level:
- propose-only
### 2. Correlation Agent
Primary use case:
- identify whether multiple signals describe the same event or related events
Inputs:
- findings from multiple collectors
- time windows
- region / ASN / prefix / cable relationships
- prior incidents
Outputs:
- grouped event candidates
- correlation rationale
- confidence per relationship
Typical action level:
- read-only
### 3. Assessment Agent
Primary use case:
- produce situational-awareness outputs
Inputs:
- grouped events
- findings
- current context
- historical context
- operator constraints
Outputs:
- structured assessment
- risk summary
- evidence-backed recommendations
- missing-information list
Typical action level:
- read-only or propose-only
### 4. Recovery Agent
Primary use case:
- carry low-risk proposals into controlled runtime actions
Inputs:
- approved proposal
- policy constraints
- trusted-domain rules
- verification checks
Outputs:
- applied override
- failed application
- rollback request
Typical action level:
- apply-limited
## Shared Object Model
All agents should work on a shared object model.
That prevents the health subsystem and situational-awareness subsystem from drifting into incompatible payloads.
### Signal
Represents a raw observed fact.
Examples:
- a datasource returned HTTP 404
- a collector returned empty results
- BGP updates spiked in one region
- a known endpoint now redirects elsewhere
Suggested shape:
```json
{
"id": "sig_123",
"type": "datasource.http_failure",
"source": "ris_live_bgp",
"occurred_at": "2026-04-08T10:00:00Z",
"severity": "medium",
"payload": {},
"trust": 0.95
}
```
### Finding
Represents a deterministic or semi-deterministic interpretation of one or more signals.
Examples:
- `schema_changed`
- `endpoint_unreachable`
- `data_volume_abnormally_low`
- `event_cluster_detected`
Suggested shape:
```json
{
"id": "find_123",
"type": "datasource.schema_changed",
"source_ids": ["sig_123"],
"confidence": 0.92,
"evidence": [],
"details": {}
}
```
### Proposal
Represents a recommended action, not an already-applied action.
Examples:
- switch endpoint to new URL
- disable bad override
- escalate issue for manual review
Suggested shape:
```json
{
"id": "prop_123",
"kind": "endpoint_override",
"target": "telegeography_cables",
"confidence": 0.84,
"reason": "Official docs now point to a new API path",
"payload": {},
"evidence_urls": [],
"status": "proposed"
}
```
### Assessment
Represents a structured situational-awareness output for operators or downstream systems.
Examples:
- current network posture summary
- incident impact assessment
- risk and response recommendations
Suggested shape:
```json
{
"id": "assess_123",
"scope": "regional-network",
"risk_level": "high",
"summary": "Regional routing instability is increasing.",
"key_risks": [],
"evidence": [],
"recommendations": [],
"missing_data": []
}
```
## State Machine
The shared orchestration flow should look like this:
```mermaid
stateDiagram-v2
[*] --> Collect
Collect --> Validate
Validate --> Classify
Classify --> Reason
Reason --> Propose
Reason --> Assess
Propose --> Review
Review --> Apply
Apply --> Verify
Verify --> Archive
Assess --> Archive
Archive --> [*]
```
Definitions:
- `Collect`: gather signals
- `Validate`: run deterministic checks
- `Classify`: create findings
- `Reason`: invoke LLM reasoning when needed
- `Propose`: create change proposals
- `Review`: policy or human approval
- `Apply`: perform limited runtime action
- `Verify`: confirm action effect
- `Archive`: store artifacts and decisions
## Permission Model
Each agent role should be assigned one of these action levels.
### `read-only`
Allowed:
- read signals
- search web
- fetch pages
- read internal state
- generate findings and assessments
Not allowed:
- mutate config
- write overrides
- change live runtime behavior
### `propose-only`
Allowed:
- everything in `read-only`
- create proposals
- create review tasks
Not allowed:
- apply live changes
### `apply-limited`
Allowed:
- everything in `propose-only`
- write approved runtime overrides
- trigger verification checks
Not allowed:
- mutate repository defaults
- make destructive data changes
- bypass policy engine
## Runtime Components
The first durable architecture should introduce these components.
### 1. Signal Store
Stores normalized evidence and health outputs.
### 2. Finding Store
Stores deterministic classifications that can be reused by multiple agents.
### 3. Proposal Store
Stores recommended actions with evidence and confidence.
### 4. Assessment Store
Stores structured situational-awareness outputs.
### 5. Policy Engine
Decides:
- whether agent may run
- whether proposal requires review
- whether proposal may auto-apply
- whether post-apply verification passed
### 6. Override Store
Stores runtime-only configuration changes.
This is where endpoint repairs should live.
## Relation To `aiprovider`
`aiprovider` should remain the model gateway.
It should not become the full agent runtime.
Recommended split:
- `aiprovider`
- provider adaptation
- prompt transport
- model execution
- protocol compatibility
- agent runtime
- orchestration
- signal handling
- tool selection
- proposal generation
- policy and audit
This keeps provider concerns and agent behavior concerns separate.
## Relation To Datasource Health
Datasource health becomes one vertical slice of this architecture.
Mapping:
- signal:
- endpoint unreachable
- schema mismatch
- bad content type
- finding:
- `failed`
- `schema_changed`
- `moved_endpoint_suspected`
- proposal:
- runtime override suggestion
- assessment:
- datasource health summary for operators
## Relation To Situational Awareness
Future situational-awareness capabilities should reuse the same flow:
- raw telemetry becomes signals
- anomaly detection becomes findings
- LLM correlation becomes reasoning
- operator-facing output becomes assessments
- policy-approved mitigations become actions
This lets the platform evolve from operational health governance into broader cyber/network posture workflows without changing the architecture.
## Suggested Delivery Sequence
### Phase A
- finalize shared object model
- implement health-oriented signal and finding storage
### Phase B
- implement Health Agent
- generate proposals only
### Phase C
- implement Assessment Agent
- expose structured assessments via API
### Phase D
- implement Correlation Agent
- support multi-source incident grouping
### Phase E
- implement Recovery Agent with policy-gated runtime actions
## Recommended First Build
The first build should not try to implement every agent role.
Recommended initial slice:
- shared object model
- health signals
- health findings
- Health Agent
- proposal generation only
This gives immediate value while preserving the longer-term architecture.
## Non-Goals For The First Iteration
- repository YAML auto-rewrites
- unrestricted autonomous action
- full incident graph reasoning
- automatic large-scale remediation
- agent-owned configuration source of truth
## Summary
Planet should treat agents as a reusable runtime for evidence, reasoning, proposals, and assessments.
The datasource health use case is the first practical entrypoint, but the architecture should already assume future situational-awareness expansion.
The safest path is:
- deterministic checks first
- agent reasoning second
- proposals before actions
- runtime overrides instead of default mutation

346
docs/agent-runtime-roadmap.md Normal file
View File

@@ -0,0 +1,346 @@
# Agent Runtime Roadmap
## Overview
This document connects three existing planning threads into one implementation roadmap:
- `aiprovider` as the model gateway
- datasource health governance as the first practical agent use case
- situational awareness as the broader long-term target
Related documents:
- [aiprovider](/home/ray/dev/linkong/planet/docs/aiprovider.md)
- [datasource-health-plan](/home/ray/dev/linkong/planet/docs/datasource-health-plan.md)
- [agent-architecture-plan](/home/ray/dev/linkong/planet/docs/agent-architecture-plan.md)
## Big Picture
Planet should evolve in layers:
1. stable model gateway
2. deterministic health and evidence collection
3. agent runtime for reasoning and proposal generation
4. situational-awareness assessments and controlled actions
This prevents the system from collapsing into a single giant "AI feature" with unclear boundaries.
## Architecture Overview
```mermaid
flowchart TD
U["Frontend / Backend APIs / Operators"] --> B["Planet Backend"]
B --> H["Datasource Health Services"]
B --> R["Agent Runtime"]
R --> P["aiprovider"]
P --> M["OpenAI / Anthropic / MiniMax / Ollama / Local Models"]
C["Collectors / Snapshots / Logs / Alerts / BGP Signals"] --> S["Signal Store"]
H --> S
S --> E["Evaluation Layer"]
E --> F["Findings"]
F --> R
W["Web Search / Page Fetch / Docs Fetch"] --> R
R --> PR["Proposals"]
R --> AS["Assessments"]
PR --> O["Runtime Overrides / Review Queue / Tasks"]
AS --> SA["Situational Awareness APIs / UI"]
O --> V["Verification Loop"]
V --> S
```
## Role Boundaries
### `aiprovider`
Responsibilities:
- provider compatibility
- protocol adaptation
- auth and model transport
- request/response normalization
Not responsible for:
- agent orchestration
- business workflows
- datasource repair policy
- situational-awareness domain logic
### Backend
Responsibilities:
- stable business APIs
- auth and permissions
- task orchestration
- health records
- proposal and override persistence
- assessment exposure
### Agent Runtime
Responsibilities:
- consume findings and context
- invoke LLMs via `aiprovider`
- invoke tools such as web search
- create proposals
- create assessments
- route to policy-controlled action paths
## Delivery Sequence
## Stage 1: Gateway Foundation
Status:
- already in place
Delivered by current work:
- `aiprovider`
- multi-provider compatibility
- backend AI facade
- MiniMax / Anthropic-compatible support
- request-id propagation
Primary outcome:
- the system already has a stable way to call models
## Stage 2: Datasource Health MVP
Goal:
- establish deterministic health observability
Key work:
- health check task runner
- health result table
- datasource health APIs
- UI visibility
- collector endpoint override precedence cleanup
Primary outcome:
- Planet knows which collectors are healthy before asking an LLM anything
## Stage 3: Health Agent
Goal:
- let the first agent role operate on health failures
Key work:
- convert health failures into signals/findings
- invoke agent only for failed or suspicious cases
- produce repair proposals with evidence and confidence
Primary outcome:
- Planet can suggest endpoint repairs without mutating defaults
## Stage 4: Runtime Repair Application
Goal:
- safely apply approved datasource repair proposals
Key work:
- override storage
- policy-gated apply flow
- verification after apply
- rollback path
Primary outcome:
- datasource repair becomes operationally useful without polluting repository defaults
## Stage 5: Situational Awareness Assessments
Goal:
- reuse the same runtime for broader operator-facing assessment
Key work:
- normalize telemetry and incident evidence into signals/findings
- build Assessment Agent
- expose structured assessments through backend APIs and UI
Primary outcome:
- LLM output becomes evidence-backed situational summary, not just ad hoc chat output
## Stage 6: Correlation and Controlled Actions
Goal:
- connect multiple sources into higher-level posture and event groupings
Key work:
- event correlation
- incident grouping
- recommendation scoring
- controlled action routing
Primary outcome:
- Planet becomes a true agent-assisted situational-awareness system
## Implementation Tracks
These tracks can progress in parallel, but they should stay loosely coupled.
### Track A: Config and Runtime Resolution
Scope:
- datasource defaults
- overrides
- runtime precedence
- audit trails
First milestone:
- health-safe override layer
### Track B: Health and Evidence
Scope:
- deterministic checks
- failure categorization
- signal and finding persistence
First milestone:
- datasource health record system
### Track C: Agent Runtime
Scope:
- shared object model
- orchestration flow
- prompt/tool pipeline
- policy integration
First milestone:
- Health Agent proposal pipeline
### Track D: Situational Awareness
Scope:
- assessment schema
- multi-source context assembly
- operator-facing outputs
First milestone:
- structured assessment API
## Shared Artifacts
To avoid fragmentation, these artifacts should be shared across all future agent work.
### Shared object model
- `Signal`
- `Finding`
- `Proposal`
- `Assessment`
### Shared orchestration flow
- collect
- validate
- classify
- reason
- propose or assess
- review or apply
- verify
- archive
### Shared policy model
- read-only
- propose-only
- apply-limited
## Recommended Next Concrete Steps
1. Build Stage 2 first
- datasource health records
- deterministic checks
- no automatic repair
2. Then build Stage 3
- Health Agent
- proposal generation only
3. Then Stage 4
- override apply flow
- rollback and verification
4. Only after that start Stage 5
- broader situational-awareness assessment workflows
## Why This Order
Because situational-awareness quality depends on reliable upstream data.
If datasource health is weak:
- agent reasoning quality will degrade
- false explanations will increase
- assessment trust will drop
So datasource health is not a side task.
It is the first operational foundation for the later situational-awareness system.
## Summary
Planet should be built as:
- `aiprovider` for model access
- backend services for orchestration and persistence
- datasource health as the first evidence-governance layer
- agent runtime as the reusable reasoning core
- situational awareness as the long-term application layer
That path keeps the architecture coherent and lets each phase produce useful functionality without forcing a rewrite later.

478
docs/datasource-health-stage2-tasks.md Normal file
View File

@@ -0,0 +1,478 @@
# Datasource Health Stage 2 Tasks
## Goal
Stage 2 focuses on the first practical operational layer:
- deterministic datasource health checks
- persisted health results
- health visibility through API and UI
- no agent-assisted repair yet
This stage should make Planet capable of answering:
- which collectors are healthy
- which collectors are degraded
- which collectors are failing
- why they are failing at a basic deterministic level
## Scope
Included:
- datasource health data model
- deterministic health check service
- manual and scheduled health check triggers
- health result APIs
- frontend visibility
Excluded:
- LLM reasoning
- web-search-based repair proposals
- automatic endpoint rewriting
- runtime override application
## Delivery Target
At the end of Stage 2, an operator should be able to:
1. see health status for each collector
2. trigger a health check manually
3. inspect the latest failure reason
4. inspect the last checked endpoint
5. understand whether the problem is:
- unreachable
- auth-related
- rate-limit-related
- schema-related
- empty-data-related
## Work Breakdown
## A. Data Model
### A1. Add datasource health record table
Create a new model, for example:
- `backend/app/models/datasource_health_check.py`
Suggested fields:
- `id`
- `datasource_id`
- `collector_name`
- `endpoint_checked`
- `status`
- `http_status`
- `content_type`
- `latency_ms`
- `sample_count`
- `error_message`
- `details`
- `checked_at`
Suggested status enum values:
- `healthy`
- `degraded`
- `failed`
- `schema_changed`
- `rate_limited`
- `auth_required`
- `empty_result`
### A2. Add datasource health summary fields
Option A:
- keep summary only in the health check table
Option B:
- also add summary fields on `data_sources`
Recommended first step:
- do not mutate `data_sources` schema yet
- derive summary from the latest health record
### A3. Migration task
Add migration for the health table.
Deliverables:
- migration file
- model registration
## B. Health Check Engine
### B1. Define health check service
Add a new service module, for example:
- `backend/app/services/datasource_health.py`
Responsibilities:
- resolve effective endpoint
- execute deterministic check
- classify result
- persist health record
### B2. Define shared result schema
Create a typed result object, for example:
- `HealthCheckResult`
Suggested fields:
- `status`
- `endpoint_checked`
- `http_status`
- `content_type`
- `latency_ms`
- `sample_count`
- `error_message`
- `details`
### B3. Implement base deterministic checks
Every datasource should go through a minimal baseline check:
1. resolve endpoint
2. perform request
3. measure latency
4. inspect status code
5. inspect content type
6. inspect body shape
Classification rules:
- network error -> `failed`
- HTTP 401/403 -> `auth_required`
- HTTP 429 -> `rate_limited`
- HTTP 404/410 -> `failed`
- parse failure -> `schema_changed`
- zero or suspiciously empty results -> `empty_result` or `degraded`
- valid parse -> `healthy`
### B4. Add collector-aware adapters
Some collectors do not use the same fetch semantics.
Add adapter profiles such as:
- `http_json`
- `http_csv`
- `html_scrape`
- `stream_probe`
- `auth_session_http`
Initial mapping suggestion:
- `huggingface`, `peeringdb`, `cloudflare` -> `http_json`
- `fao` -> `http_csv`
- `top500`, `epoch_ai`, `telegeography live_map` -> `html_scrape`
- `ris_live` -> `stream_probe`
- `spacetrack` -> `auth_session_http`
### B5. Add sample validation hooks
For each adapter, add a lightweight validation rule.
Examples:
- JSON array length > 0
- CSV rows > 1
- HTML page contains expected table or script patterns
- stream source yields at least one valid event within timeout
## C. Persistence and Query Layer
### C1. Save every check run
Each health check should insert a record.
Do not overwrite history in Stage 2.
### C2. Add latest-health query helpers
Add helper functions to fetch:
- latest health record by datasource
- latest failed health record
- recent health history
### C3. Optional retention policy
For Stage 2, retention can be deferred.
If desired, keep only:
- last N records per datasource
## D. API Layer
### D1. Add health list endpoint
Suggested endpoint:
- `GET /api/v1/datasources/health`
Returns:
- datasource id
- collector name
- current endpoint
- latest health status
- last checked time
- short reason
### D2. Add per-datasource health detail endpoint
Suggested endpoint:
- `GET /api/v1/datasources/{id}/health`
Returns:
- latest record
- recent history
- detailed classification fields
### D3. Add manual health trigger endpoint
Suggested endpoint:
- `POST /api/v1/datasources/{id}/health-check`
Behavior:
- run a health check now
- persist the result
- return the new record
### D4. Add bulk health trigger endpoint
Suggested endpoint:
- `POST /api/v1/datasources/health-check-all`
Behavior:
- enqueue or run health checks for all active datasources
## E. Scheduling
### E1. Add health scheduler task
Decide scheduling strategy.
Recommended first version:
- run collector jobs and health checks separately
- health checks run on a lower frequency
Suggested frequency:
- every 6h or 12h for most datasources
- optionally on-demand only in the very first cut
### E2. Prevent health check collision with collection
Rules:
- health checks should not disrupt active collection
- they should use light requests
- if a collector is currently running, health check may:
- skip
- or use a lightweight endpoint probe only
## F. Frontend
### F1. Add health columns to datasource list
Update:
- `frontend/src/pages/DataSources/DataSources.tsx`
Suggested new columns:
- health status
- last checked
- reason summary
### F2. Add manual health check action
Per datasource:
- button or dropdown action:
- `健康检查`
### F3. Add health detail drawer or modal
Show:
- endpoint checked
- status
- HTTP status
- content type
- sample count
- error message
- last few results
### F4. Add basic visual language
Suggested colors:
- green -> healthy
- yellow -> degraded
- orange -> rate-limited / auth-required
- red -> failed / schema-changed
## G. Observability
### G1. Structured logging
Every health check should log:
- datasource id
- collector name
- endpoint
- status
- latency
- failure class
### G2. Optional metrics
If metrics are added later, useful counters include:
- health checks total
- health checks failed
- schema changes detected
- rate limited checks
## H. Tests
### H1. Unit tests
Add tests for:
- status classification
- content type classification
- adapter behavior
- latest-health query helpers
### H2. API tests
Add tests for:
- health endpoints require auth
- manual trigger endpoint works
- list endpoint returns latest status
### H3. Failure-path tests
Add coverage for:
- HTTP 404
- HTTP 429
- invalid JSON
- empty response
- parse mismatch
## Suggested File Plan
Possible implementation files:
- `backend/app/models/datasource_health_check.py`
- `backend/app/services/datasource_health.py`
- `backend/app/schemas/datasource_health.py`
- `backend/app/api/v1/datasource_health.py`
- migration file under the project migration system
Likely touched existing files:
- `backend/app/api/main.py`
- `frontend/src/pages/DataSources/DataSources.tsx`
- `backend/tests/test_api.py`
## Suggested Execution Order
1. Add model and migration
2. Add service and result schema
3. Add deterministic adapters
4. Add manual trigger API
5. Add list/detail API
6. Add frontend visibility
7. Add scheduled checks
8. Expand tests
## Minimal First Milestone
If we want the fastest useful slice, do this first:
1. health table
2. deterministic check service
3. manual per-datasource health check API
4. latest health list API
5. frontend status badge column
That is enough to start operating the system and will provide the input layer for Stage 3.
## Dependency On Later Stages
Stage 2 outputs become direct inputs for Stage 3.
Specifically:
- failed or schema-changed health records become agent triggers
- health history becomes repair context
- endpoint_checked becomes proposal baseline
## Success Criteria
Stage 2 is done when:
- every active datasource can be health-checked deterministically
- the latest health state is visible in API and UI
- operators can manually trigger checks
- failures are categorized into stable machine-readable statuses
- no LLM is required for core health visibility