linkong/planet
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

release: bump version to 0.28.2

Browse Source
This commit is contained in:
linkong
2026-04-20 16:00:54 +08:00
parent 75cb214f23
commit b5dd4f12f8
31 changed files with 175 additions and 7995 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
VERSION
View File

@@ -1 +1 @@
0.28.1
0.28.2

16
docs/CHANGELOG.md
View File

@@ -8,6 +8,22 @@ This project follows the repository versioning rule:
- `improvement` -> `+0.0.1`（bugfix + 小功能混合）
- `bugfix` -> `+0.0.1`
## [0.28.2] — 2026-04-20
### ✨ Highlights
- 修正媒体情报面板在 `电视直播 / 态势聚合` tab 间切换时的尺寸记忆逻辑，切回原 tab 后可恢复各自大小状态
- 清理 `docs/` 根目录遗留的旧路径文档，只保留新的分组目录和归档目录，结束同一文档双路径并存状态
### 🔧 Improvements
- `media-panel` 切换逻辑改成按 tab 分别记忆尺寸状态，避免 `A -> B -> A` 时继续共用同一套外层尺寸
- 目录整理真正完成收尾：旧的 `docs/*.md` 平铺计划文档删除，继续以 `docs/agents / earth / backend / frontend / ops / ue5 / deprecated` 为唯一入口
### 🐛 Fixes
- 修复拉伸 `media-panel` 后切换 tab 时，`news-panel` 高度回退到旧默认值的问题
- 修复拉伸后切换 tab 导致面板视觉锚点异常的问题，切换时改为围绕当前卡片自身右下角进行尺寸恢复
---
## [0.28.1] — 2026-04-20
### ✨ Highlights

647
docs/agent-architecture-plan.md
View File

@@ -1,647 +0,0 @@
# Agent Architecture Plan
## Overview
This document defines the agent architecture for Planet.
The architecture is intentionally broader than datasource health checking.
It is designed to support both:
- datasource health governance
- future situational-awareness workflows
The core idea is to avoid building a one-off "repair broken API links" agent.
Instead, Planet should grow a reusable agent runtime that can:
- collect evidence
- evaluate signals
- reason over incomplete information
- generate proposals
- produce assessments
- execute limited actions under policy
## Design Goal
Build an agent foundation that can evolve in this order:
1. datasource health checks
2. datasource repair proposals
3. signal correlation
4. situational assessments
5. controlled runtime actions
This means the architecture should treat datasource health as one use case of the larger agent system, not as the whole system.
## Core Principles
1. Separate evidence from reasoning
- raw signals should be gathered first
- deterministic checks should run before LLM reasoning
2. Agents do not own the defaults
- repository defaults remain human-owned
- agents operate on runtime state, proposals, and overrides
3. Reasoning and action are different responsibilities
- many agents should be read-only or propose-only
- only tightly controlled flows may apply changes
4. Shared runtime, specialized roles
- multiple agent roles should share the same object model and orchestration patterns
- health and situational-awareness agents should not invent incompatible payloads
5. Auditability is mandatory
- every proposal, assessment, and applied action should be attributable
## System Layers
Planet agent architecture should be split into four layers.
### 1. Signal Layer
Purpose:
- gather raw evidence from internal and external systems
Example sources:
- collector outputs
- datasource health checks
- logs
- snapshots
- alerts
- web search results
- scraped pages
- external APIs
- operator inputs
Responsibilities:
- fetch
- normalize
- timestamp
- tag with source and trust level
This layer should not make high-level judgments.
### 2. Evaluation Layer
Purpose:
- perform deterministic analysis
Examples:
- reachability checks
- schema validation
- threshold checks
- time-window comparisons
- anomaly counters
- completeness checks
Responsibilities:
- classify signals into machine-readable findings
- attach deterministic evidence
This layer should avoid LLM dependency whenever possible.
### 3. Reasoning Layer
Purpose:
- use LLMs when semantic interpretation or incomplete-information reasoning is needed
Examples:
- endpoint migration inference
- multi-source event correlation
- causality hypotheses
- ambiguity reduction
- assessment narrative generation
- action recommendation generation
Responsibilities:
- synthesize evidence
- produce hypotheses
- rank confidence
- explain reasoning boundaries
This is the main place where `aiprovider` and web search are used.
### 4. Action Layer
Purpose:
- convert proposals or assessments into controlled system actions
Examples:
- create runtime override
- create proposal
- publish alert
- update operator task queue
- generate summary artifact
- trigger follow-up verification
Responsibilities:
- enforce policy
- enforce approval requirements
- verify post-action outcomes
- record audit trails
## Architecture Sketch
```mermaid
flowchart TD
A["Collectors / Logs / Snapshots / External APIs"] --> B["Signal Layer"]
W["Web Search / Page Fetch / Docs"] --> B
B --> C["Evaluation Layer"]
C --> D["Findings"]
D --> E["Reasoning Layer (LLM + Tools)"]
E --> F["Proposals"]
E --> G["Assessments"]
F --> H["Action Layer"]
H --> I["Runtime Overrides / Alerts / Tasks"]
H --> J["Verification Loop"]
J --> B
K["Policy Engine"] --> H
L["Audit / History Store"] --> H
L --> E
L --> C
```
## Agent Roles
The first version should define these logical roles.
### 1. Health Agent
Primary use case:
- datasource health governance
Inputs:
- datasource metadata
- current endpoint
- latest health records
- latest failures
- deterministic findings
Outputs:
- health interpretation
- repair proposal
- confidence
- evidence references
Typical action level:
- propose-only
### 2. Correlation Agent
Primary use case:
- identify whether multiple signals describe the same event or related events
Inputs:
- findings from multiple collectors
- time windows
- region / ASN / prefix / cable relationships
- prior incidents
Outputs:
- grouped event candidates
- correlation rationale
- confidence per relationship
Typical action level:
- read-only
### 3. Assessment Agent
Primary use case:
- produce situational-awareness outputs
Inputs:
- grouped events
- findings
- current context
- historical context
- operator constraints
Outputs:
- structured assessment
- risk summary
- evidence-backed recommendations
- missing-information list
Typical action level:
- read-only or propose-only
### 4. Recovery Agent
Primary use case:
- carry low-risk proposals into controlled runtime actions
Inputs:
- approved proposal
- policy constraints
- trusted-domain rules
- verification checks
Outputs:
- applied override
- failed application
- rollback request
Typical action level:
- apply-limited
## Shared Object Model
All agents should work on a shared object model.
That prevents the health subsystem and situational-awareness subsystem from drifting into incompatible payloads.
### Signal
Represents a raw observed fact.
Examples:
- a datasource returned HTTP 404
- a collector returned empty results
- BGP updates spiked in one region
- a known endpoint now redirects elsewhere
Suggested shape:
```json
{
"id": "sig_123",
"type": "datasource.http_failure",
"source": "ris_live_bgp",
"occurred_at": "2026-04-08T10:00:00Z",
"severity": "medium",
"payload": {},
"trust": 0.95
}
```
### Finding
Represents a deterministic or semi-deterministic interpretation of one or more signals.
Examples:
- `schema_changed`
- `endpoint_unreachable`
- `data_volume_abnormally_low`
- `event_cluster_detected`
Suggested shape:
```json
{
"id": "find_123",
"type": "datasource.schema_changed",
"source_ids": ["sig_123"],
"confidence": 0.92,
"evidence": [],
"details": {}
}
```
### Proposal
Represents a recommended action, not an already-applied action.
Examples:
- switch endpoint to new URL
- disable bad override
- escalate issue for manual review
Suggested shape:
```json
{
"id": "prop_123",
"kind": "endpoint_override",
"target": "telegeography_cables",
"confidence": 0.84,
"reason": "Official docs now point to a new API path",
"payload": {},
"evidence_urls": [],
"status": "proposed"
}
```
### Assessment
Represents a structured situational-awareness output for operators or downstream systems.
Examples:
- current network posture summary
- incident impact assessment
- risk and response recommendations
Suggested shape:
```json
{
"id": "assess_123",
"scope": "regional-network",
"risk_level": "high",
"summary": "Regional routing instability is increasing.",
"key_risks": [],
"evidence": [],
"recommendations": [],
"missing_data": []
}
```
## State Machine
The shared orchestration flow should look like this:
```mermaid
stateDiagram-v2
[*] --> Collect
Collect --> Validate
Validate --> Classify
Classify --> Reason
Reason --> Propose
Reason --> Assess
Propose --> Review
Review --> Apply
Apply --> Verify
Verify --> Archive
Assess --> Archive
Archive --> [*]
```
Definitions:
- `Collect`: gather signals
- `Validate`: run deterministic checks
- `Classify`: create findings
- `Reason`: invoke LLM reasoning when needed
- `Propose`: create change proposals
- `Review`: policy or human approval
- `Apply`: perform limited runtime action
- `Verify`: confirm action effect
- `Archive`: store artifacts and decisions
## Permission Model
Each agent role should be assigned one of these action levels.
### `read-only`
Allowed:
- read signals
- search web
- fetch pages
- read internal state
- generate findings and assessments
Not allowed:
- mutate config
- write overrides
- change live runtime behavior
### `propose-only`
Allowed:
- everything in `read-only`
- create proposals
- create review tasks
Not allowed:
- apply live changes
### `apply-limited`
Allowed:
- everything in `propose-only`
- write approved runtime overrides
- trigger verification checks
Not allowed:
- mutate repository defaults
- make destructive data changes
- bypass policy engine
## Runtime Components
The first durable architecture should introduce these components.
### 1. Signal Store
Stores normalized evidence and health outputs.
### 2. Finding Store
Stores deterministic classifications that can be reused by multiple agents.
### 3. Proposal Store
Stores recommended actions with evidence and confidence.
### 4. Assessment Store
Stores structured situational-awareness outputs.
### 5. Policy Engine
Decides:
- whether agent may run
- whether proposal requires review
- whether proposal may auto-apply
- whether post-apply verification passed
### 6. Override Store
Stores runtime-only configuration changes.
This is where endpoint repairs should live.
## Relation To `aiprovider`
`aiprovider` should remain the model gateway.
It should not become the full agent runtime.
Recommended split:
- `aiprovider`
- provider adaptation
- prompt transport
- model execution
- protocol compatibility
- agent runtime
- orchestration
- signal handling
- tool selection
- proposal generation
- policy and audit
This keeps provider concerns and agent behavior concerns separate.
## Relation To Datasource Health
Datasource health becomes one vertical slice of this architecture.
Mapping:
- signal:
- endpoint unreachable
- schema mismatch
- bad content type
- finding:
- `failed`
- `schema_changed`
- `moved_endpoint_suspected`
- proposal:
- runtime override suggestion
- assessment:
- datasource health summary for operators
## Relation To Situational Awareness
Future situational-awareness capabilities should reuse the same flow:
- raw telemetry becomes signals
- anomaly detection becomes findings
- LLM correlation becomes reasoning
- operator-facing output becomes assessments
- policy-approved mitigations become actions
This lets the platform evolve from operational health governance into broader cyber/network posture workflows without changing the architecture.
## Suggested Delivery Sequence
### Phase A
- finalize shared object model
- implement health-oriented signal and finding storage
### Phase B
- implement Health Agent
- generate proposals only
### Phase C
- implement Assessment Agent
- expose structured assessments via API
### Phase D
- implement Correlation Agent
- support multi-source incident grouping
### Phase E
- implement Recovery Agent with policy-gated runtime actions
## Recommended First Build
The first build should not try to implement every agent role.
Recommended initial slice:
- shared object model
- health signals
- health findings
- Health Agent
- proposal generation only
This gives immediate value while preserving the longer-term architecture.
## Non-Goals For The First Iteration
- repository YAML auto-rewrites
- unrestricted autonomous action
- full incident graph reasoning
- automatic large-scale remediation
- agent-owned configuration source of truth
## Summary
Planet should treat agents as a reusable runtime for evidence, reasoning, proposals, and assessments.
The datasource health use case is the first practical entrypoint, but the architecture should already assume future situational-awareness expansion.
The safest path is:
- deterministic checks first
- agent reasoning second
- proposals before actions
- runtime overrides instead of default mutation

346
docs/agent-runtime-roadmap.md
View File

@@ -1,346 +0,0 @@
# Agent Runtime Roadmap
## Overview
This document connects three existing planning threads into one implementation roadmap:
- `aiprovider` as the model gateway
- datasource health governance as the first practical agent use case
- situational awareness as the broader long-term target
Related documents:
- [aiprovider](/home/ray/dev/linkong/planet/docs/aiprovider.md)
- [datasource-health-plan](/home/ray/dev/linkong/planet/docs/datasource-health-plan.md)
- [agent-architecture-plan](/home/ray/dev/linkong/planet/docs/agent-architecture-plan.md)
## Big Picture
Planet should evolve in layers:
1. stable model gateway
2. deterministic health and evidence collection
3. agent runtime for reasoning and proposal generation
4. situational-awareness assessments and controlled actions
This prevents the system from collapsing into a single giant "AI feature" with unclear boundaries.
## Architecture Overview
```mermaid
flowchart TD
U["Frontend / Backend APIs / Operators"] --> B["Planet Backend"]
B --> H["Datasource Health Services"]
B --> R["Agent Runtime"]
R --> P["aiprovider"]
P --> M["OpenAI / Anthropic / MiniMax / Ollama / Local Models"]
C["Collectors / Snapshots / Logs / Alerts / BGP Signals"] --> S["Signal Store"]
H --> S
S --> E["Evaluation Layer"]
E --> F["Findings"]
F --> R
W["Web Search / Page Fetch / Docs Fetch"] --> R
R --> PR["Proposals"]
R --> AS["Assessments"]
PR --> O["Runtime Overrides / Review Queue / Tasks"]
AS --> SA["Situational Awareness APIs / UI"]
O --> V["Verification Loop"]
V --> S
```
## Role Boundaries
### `aiprovider`
Responsibilities:
- provider compatibility
- protocol adaptation
- auth and model transport
- request/response normalization
Not responsible for:
- agent orchestration
- business workflows
- datasource repair policy
- situational-awareness domain logic
### Backend
Responsibilities:
- stable business APIs
- auth and permissions
- task orchestration
- health records
- proposal and override persistence
- assessment exposure
### Agent Runtime
Responsibilities:
- consume findings and context
- invoke LLMs via `aiprovider`
- invoke tools such as web search
- create proposals
- create assessments
- route to policy-controlled action paths
## Delivery Sequence
## Stage 1: Gateway Foundation
Status:
- already in place
Delivered by current work:
- `aiprovider`
- multi-provider compatibility
- backend AI facade
- MiniMax / Anthropic-compatible support
- request-id propagation
Primary outcome:
- the system already has a stable way to call models
## Stage 2: Datasource Health MVP
Goal:
- establish deterministic health observability
Key work:
- health check task runner
- health result table
- datasource health APIs
- UI visibility
- collector endpoint override precedence cleanup
Primary outcome:
- Planet knows which collectors are healthy before asking an LLM anything
## Stage 3: Health Agent
Goal:
- let the first agent role operate on health failures
Key work:
- convert health failures into signals/findings
- invoke agent only for failed or suspicious cases
- produce repair proposals with evidence and confidence
Primary outcome:
- Planet can suggest endpoint repairs without mutating defaults
## Stage 4: Runtime Repair Application
Goal:
- safely apply approved datasource repair proposals
Key work:
- override storage
- policy-gated apply flow
- verification after apply
- rollback path
Primary outcome:
- datasource repair becomes operationally useful without polluting repository defaults
## Stage 5: Situational Awareness Assessments
Goal:
- reuse the same runtime for broader operator-facing assessment
Key work:
- normalize telemetry and incident evidence into signals/findings
- build Assessment Agent
- expose structured assessments through backend APIs and UI
Primary outcome:
- LLM output becomes evidence-backed situational summary, not just ad hoc chat output
## Stage 6: Correlation and Controlled Actions
Goal:
- connect multiple sources into higher-level posture and event groupings
Key work:
- event correlation
- incident grouping
- recommendation scoring
- controlled action routing
Primary outcome:
- Planet becomes a true agent-assisted situational-awareness system
## Implementation Tracks
These tracks can progress in parallel, but they should stay loosely coupled.
### Track A: Config and Runtime Resolution
Scope:
- datasource defaults
- overrides
- runtime precedence
- audit trails
First milestone:
- health-safe override layer
### Track B: Health and Evidence
Scope:
- deterministic checks
- failure categorization
- signal and finding persistence
First milestone:
- datasource health record system
### Track C: Agent Runtime
Scope:
- shared object model
- orchestration flow
- prompt/tool pipeline
- policy integration
First milestone:
- Health Agent proposal pipeline
### Track D: Situational Awareness
Scope:
- assessment schema
- multi-source context assembly
- operator-facing outputs
First milestone:
- structured assessment API
## Shared Artifacts
To avoid fragmentation, these artifacts should be shared across all future agent work.
### Shared object model
- `Signal`
- `Finding`
- `Proposal`
- `Assessment`
### Shared orchestration flow
- collect
- validate
- classify
- reason
- propose or assess
- review or apply
- verify
- archive
### Shared policy model
- read-only
- propose-only
- apply-limited
## Recommended Next Concrete Steps
1. Build Stage 2 first
- datasource health records
- deterministic checks
- no automatic repair
2. Then build Stage 3
- Health Agent
- proposal generation only
3. Then Stage 4
- override apply flow
- rollback and verification
4. Only after that start Stage 5
- broader situational-awareness assessment workflows
## Why This Order
Because situational-awareness quality depends on reliable upstream data.
If datasource health is weak:
- agent reasoning quality will degrade
- false explanations will increase
- assessment trust will drop
So datasource health is not a side task.
It is the first operational foundation for the later situational-awareness system.
## Summary
Planet should be built as:
- `aiprovider` for model access
- backend services for orchestration and persistence
- datasource health as the first evidence-governance layer
- agent runtime as the reusable reasoning core
- situational awareness as the long-term application layer
That path keeps the architecture coherent and lets each phase produce useful functionality without forcing a rewrite later.

361
docs/ai-playground-development-plan.md
View File

@@ -1,361 +0,0 @@
# AI Playground Development Plan
## 目标
这份计划用于统一 `aiprovider`、`backend AI facade`、`Playground` 页面，以及后续 `BGP / 告警 / 数据源健康` 等 AI 入口的演进方向。
当前原则：
- `aiprovider` 继续作为独立模型网关
- `backend` 继续作为稳定业务入口
- `frontend` 负责测试台和业务 UI
- 先做“可控、可验证、可解释”的 AI 能力，再逐步引入 agent/tool calling
## 当前已完成
### 1. AI 网关基础层
已完成：
- 独立 `aiprovider` 服务
- `backend -> aiprovider -> model provider` 调用链
- `provider/status` 与 `situational-awareness/analyze` 稳定接口
- `X-Request-ID` 透传
- 轻量超时与重试
- MiniMax / Anthropic-compatible / OpenAI-compatible / Ollama 适配
相关文件：
- [backend/app/api/v1/ai.py](/home/ray/dev/linkong/planet/backend/app/api/v1/ai.py)
- [backend/app/services/ai_client.py](/home/ray/dev/linkong/planet/backend/app/services/ai_client.py)
- [aiprovider/main.py](/home/ray/dev/linkong/planet/aiprovider/main.py)
- [aiprovider/provider_service.py](/home/ray/dev/linkong/planet/aiprovider/provider_service.py)
- [docs/aiprovider.md](/home/ray/dev/linkong/planet/docs/aiprovider.md)
### 2. 本地运行与配置打通
已完成：
- `planet.sh` 启动链路纳入 `aiprovider`
- `planet.sh` 启动完成后输出 Playground 入口
- `docker-compose.yml` 为 `aiprovider` 加入 `env_file`
- `backend/.env` 与 `aiprovider/.env` 两侧 service token 对齐
- `Playground` 状态缓存，避免页面切换时每次都重新请求 provider 状态
相关文件：
- [planet.sh](/home/ray/dev/linkong/planet/planet.sh)
- [docker-compose.yml](/home/ray/dev/linkong/planet/docker-compose.yml)
- [backend/.env.example](/home/ray/dev/linkong/planet/backend/.env.example)
- [aiprovider/.env.example](/home/ray/dev/linkong/planet/aiprovider/.env.example)
### 3. Playground UI 基础版
已完成：
- 新增前端路由 `/playground`
- 左侧 `Provider 状态 + 测试说明`
- 右侧 `请求 / 结果` Tabs
- `Provider 状态` 支持手动刷新
- `测试说明` 支持折叠
- 内部区域采用细滚动条
- 页面布局开始遵循“单屏工作区 + 模块内部滚动”规范
相关文件：
- [frontend/src/pages/Playground/Playground.tsx](/home/ray/dev/linkong/planet/frontend/src/pages/Playground/Playground.tsx)
- [frontend/src/App.tsx](/home/ray/dev/linkong/planet/frontend/src/App.tsx)
- [frontend/src/components/AppLayout/AppLayout.tsx](/home/ray/dev/linkong/planet/frontend/src/components/AppLayout/AppLayout.tsx)
- [frontend/src/index.css](/home/ray/dev/linkong/planet/frontend/src/index.css)
### 4. 前端布局规范沉淀
已完成：
- 把“一屏工作区、主模块优先、模块内部滚动”的规范文档化
- 明确 `BGP` 页面为当前参考实现
相关文件：
- [docs/frontend-layout-guidelines.md](/home/ray/dev/linkong/planet/docs/frontend-layout-guidelines.md)
- [frontend/src/pages/BGP/BGP.tsx](/home/ray/dev/linkong/planet/frontend/src/pages/BGP/BGP.tsx)
## 当前限制
### 1. Playground 还是 prompt playground，不是 agent playground
当前 `Playground` 的 `观察项 / 目标 / 约束条件` 都是人工输入。
模型现在拿到的是：
- 你手工输入的结构化字段
- 后端传递的少量静态上下文
模型现在拿不到：
- 实时 BGP 事件
- 真实告警列表
- 数据源健康状态
- 自动检索结果
- tool calling / skills / 自主取数
### 2. `situational-awareness/analyze` 还是通用提示词接口
当前更适合：
- 测试链路
- 测试模型输出风格
- 验证不同 provider 是否正常返回
当前还不适合：
- 直接当真实态势系统主入口
- 让用户手工维护长期分析模板
- 代替专用业务研判接口
### 3. 还没有可验证的真实业务输入注入
目前最缺的是：
- 从业务系统自动整理“事实输入”
- 再把这些事实喂给 AI
而不是继续让用户在 Playground 手工输入真实事件摘要。
## 短期计划
### Phase A: Playground 收敛为稳定测试台
目标：
- 保持 Playground 简洁可用
- 不再继续堆“高级参数”
工作项：
- 继续微调左侧 `Provider 状态` 与 `测试说明` 的空间策略
- 保持 `请求 / 结果` 为单一主工作区
- 不引入盲填式高级字段
- 统一滚动条、卡片、溢出行为
完成标准：
- 笔记本视口下依然可用
- 各模块标题可见
- 主要阅读区始终是右侧 Tabs
### Phase B: BGP AI 简报
目标：
- 不再依赖手工填写“观察项”
- 让系统自动把真实 BGP 数据注入 AI
- 让 BGP 页面逐步从“摘要汇总”升级为“证据驱动的区域态势分析”
建议实现：
- 新增专用后端接口，例如：
- `POST /api/v1/ai/bgp/brief`
- 后端自动读取：
- incidents summary
- anomalies
- recent events
- collector coverage summary
- 后端将结构化事实注入 `context / observations`
- 前端在 BGP 页面增加“生成 AI 简报”
当前阶段说明：
- 第一版 `BGP AI 简报` 允许先落地为“值班摘要生成器”
- 也就是先把 incidents / anomalies / events / collector coverage 自动注入
- 允许模型先做事实摘要、风险归纳、建议动作
但这不应被视为 Phase B 的最终形态。
Phase B 后续还需要补齐：
- prefix geography 证据注入
- `iptoasn`
- `opengeofeed`
- `nro_delegated`
- 基于 `affected_regions` 与 prefix geography 的区域聚合
- 区分“真实区域热度”与“collector coverage 偏差”
- 对高风险 prefix / ASN 给出更明确的国家、城市、运营商归属线索
- 让 AI 输出明确回答：
- 哪些区域正在异常升温
- 哪些结论只是观测站偏差
- 当前还缺哪些区域证据
完成标准：
- 用户不需要手工录入 BGP 观察项
- AI 输出能明确区分“事实”和“研判”
- AI 不只是复述总量和最近几条事件，还能利用 prefix geography 与 affected regions 做区域态势判断
- 输出中能明确指出：
- 高风险区域
- 区域证据来源
- collector coverage 偏差对判断的影响
### Phase C: 告警 / 数据源健康 AI 简报
目标：
- 复用同样模式，扩展到其他模块
建议入口：
- `Alerts` 页面：异常与告警摘要
- `DataSources` 页面：采集失败与健康状态总结
原则：
- 每个业务页优先做“专用 AI 简报”
- 不优先做“万能大聊天框”
## 中期计划
### 1. Assessment Layer
目标：
- 不只返回自由文本
- 返回结构化的 assessment
建议输出字段：
- summary
- key_risks
- evidence
- recommendations
- confidence
- missing_data
这样后续才能：
- 持久化
- 回看
- 对比不同时间的 AI 结论
- 在 Earth / Dashboard / BGP 页面稳定展示
### 2. Evidence-first Runtime
目标：
- 所有 AI 分析先取真实数据，再调模型
原则：
- 先 evidence
- 再 prompt
- 最后才是自由生成
优先要做的不是更强聊天，而是：
- 更稳定的数据注入
- 更一致的事实模板
- 更清晰的结果结构
### 3. 按页面提供专用入口
目标：
- 让 AI 成为业务视图的一部分，而不是孤立 playground
优先顺序建议：
1. `BGP` AI 简报
2. `Alerts` AI 简报
3. `DataSources` 健康研判
4. `Dashboard` 总览总结
## 长期计划
### 1. Tool Calling / Agent Runtime
只有在以下基础稳定后再推进：
- 数据源健康信号稳定
- BGP / Alerts / Datasource evidence 注入稳定
- assessment 结构稳定
长期可做能力：
- AI 调用受控工具查询业务数据
- AI 调用检索/web search 做外部验证
- AI 生成建议而不是直接修改系统
- 审核后触发受控动作
### 2. 受控动作与闭环
潜在方向：
- 根据健康异常生成修复建议
- 根据态势变化生成处理建议
- 进入 review queue
- 审批后执行
- 验证结果并形成闭环
### 3. 多模块统一 AI 体验
长期目标不是一个孤立 Playground，而是：
- 每个业务页都有自己的 AI 入口
- 共享统一的 backend AI facade
- 共享统一的 assessment 结构
- 共享统一的 evidence 注入与审计链路
## 设计决策总结
### 为什么保留 `aiprovider`
因为它已经很好地承担了：
- provider 适配
- 协议兼容
- service token 边界
- 独立重启与部署
因此短期内不建议把它并回 `backend`。
### 为什么 Playground 不做成万能聊天页
因为当前更需要的是：
- 稳定测试链路
- 可验证业务输入
- 专用分析入口
而不是一个泛化但没有真实数据支撑的聊天框。
### 为什么优先做专用 AI 简报
因为：
- 数据可以自动注入
- 用户心智更清晰
- 输出更容易结构化
- 更容易校验事实与研判是否一致
## 下一步建议
按优先级建议接下来这样做：
1. 稳住 `Playground` 当前布局，不再大幅重做
2. 在 `BGP` 页面新增专用 “AI 简报” 入口
3. 后端新增 `BGP brief` 专用接口，自动注入真实数据
4. 补齐 `BGP brief` 的区域态势证据层
5. 把 AI 输出逐步从自由文本升级为结构化 assessment
### BGP Brief 后续子项
为避免把“已有 AI 简报”误判成“区域分析已完成”，这里单独记录 `BGP brief` 的后续 backlog：
1. 把高风险 prefix 命中的 `iptoasn / opengeofeed / nro_delegated` 结果注入 brief context
2. 按国家/城市聚合 active incidents、anomalies、affected prefixes，生成区域热点事实层
3. 把 collector coverage 与区域热点并排注入，避免模型把观测偏差误判成区域风险
4. 对高风险 ASN / prefix 追加归属线索，如国家、城市、可能运营商或注册区域
5. 在输出结构中单独增加：
- 区域态势
- 证据来源
- 观测偏差说明
- 缺失区域证据

333
docs/aiprovider.md
View File

@@ -1,333 +0,0 @@
# AI Provider Guide
## Overview
`aiprovider` is the model-adapter service for Planet.
It isolates model-vendor details from the main backend so the rest of the system can call a stable business API:
- Caller service -> `planet backend`
- `planet backend` -> `aiprovider`
- `aiprovider` -> concrete model provider
The recommended default is:
- External and cross-service callers use `planet backend`
- Only infrastructure-grade internal jobs call `aiprovider` directly
## Responsibilities
`backend` is responsible for:
- authentication and authorization
- business-level request shaping
- stable `/api/v1/ai/...` endpoints
- internal service-to-service authentication toward `aiprovider`
`aiprovider` is responsible for:
- model protocol adaptation
- provider selection by `.env`
- timeout and lightweight retry
- request tracing via `X-Request-ID`
This now follows an OpenClaw-like seam:
- `AI_PROVIDER` identifies the vendor or logical provider
- `AI_PROVIDER_API` identifies the wire adapter
That split makes MiniMax, Claude-compatible gateways, and self-hosted OpenAI-compatible services easier to model without overloading one config field.
## Supported Providers
`aiprovider` currently supports these provider identities:
- `openai`
- `anthropic`
- `minimax`
- `ollama`
Supported request adapters:
- `openai-completions`
- `anthropic-messages`
- `ollama-generate`
Backward-compatible aliases still accepted:
- `openai_compatible`
- `anthropic_compatible`
- `claude_compatible`
Provider mapping:
- `vLLM`, `LM Studio`, `One API`: `AI_PROVIDER=openai`, `AI_PROVIDER_API=openai-completions`
- `MiniMax`: `AI_PROVIDER=minimax`, `AI_PROVIDER_API=anthropic-messages`
- Claude-compatible gateways: `AI_PROVIDER=anthropic`, `AI_PROVIDER_API=anthropic-messages`
- `Ollama`: `AI_PROVIDER=ollama`, `AI_PROVIDER_API=ollama-generate`
## API Surfaces
### Main backend API
Preferred stable entrypoints:
- `GET /api/v1/ai/provider/status`
- `POST /api/v1/ai/situational-awareness/analyze`
Authentication:
- `Authorization: Bearer <jwt>`
Optional tracing header:
- `X-Request-ID: <caller-generated-id>`
The backend will propagate `X-Request-ID` to `aiprovider` and return the same header in the response.
### AI provider internal API
Internal-only endpoints:
- `GET /v1/provider/status`
- `POST /v1/analyze`
Authentication:
- `X-Provider-Token: <shared-secret>`
Optional tracing header:
- `X-Request-ID: <caller-generated-id>`
## Request Example
### Call through backend
```bash
curl -X POST http://localhost:8000/api/v1/ai/situational-awareness/analyze \
-H "Authorization: Bearer <access_token>" \
-H "X-Request-ID: bgp-incident-20260407-001" \
-H "Content-Type: application/json" \
-d '{
"title": "BGP异常研判",
"objective": "总结当前风险并给出处置建议",
"observations": [
"collector A 在 5 分钟内出现多次 origin 变更",
"异常集中在同一地区前缀"
],
"constraints": [
"不要编造不存在的数据",
"区分事实和推断"
],
"context": {
"source": "bgp-monitor",
"severity": "high"
}
}'
```
### Call `aiprovider` directly
```bash
curl -X POST http://localhost:8010/v1/analyze \
-H "X-Provider-Token: change_me" \
-H "X-Request-ID: ai-batch-job-001" \
-H "Content-Type: application/json" \
-d '{
"title": "链路波动分析",
"objective": "给出简要态势摘要和下一步建议",
"observations": [
"多个节点出现延迟上升"
],
"constraints": [
"不要假设根因已经确认"
],
"context": {
"region": "APAC"
}
}'
```
## Response Shape
Both backend and `aiprovider` return the same payload shape:
```json
{
"provider": "minimax",
"api": "anthropic-messages",
"model": "MiniMax-M2.7",
"content": "1) 态势摘要 ...",
"content_blocks": [],
"text_blocks": [],
"thinking_blocks": [],
"raw_response": {}
}
```
Both services also return:
- `X-Request-ID: <id>`
## Configuration
### Backend
Recommended backend `.env`:
```env
AI_PROVIDER_SERVICE_URL=http://localhost:8010
AI_PROVIDER_SERVICE_TOKEN=change_me
AI_PROVIDER_TIMEOUT_SECONDS=60
AI_PROVIDER_RETRY_ATTEMPTS=2
```
Reference file:
- [backend/.env.example](/home/ray/dev/linkong/planet/backend/.env.example)
### AI Provider
Reference file:
- [aiprovider/.env.example](/home/ray/dev/linkong/planet/aiprovider/.env.example)
Frontend local reference:
- [frontend/.env.example](/home/ray/dev/linkong/planet/frontend/.env.example)
Common settings:
```env
SERVICE_NAME=planet-ai-provider
SERVICE_VERSION=0.1.0
AI_PROVIDER_SERVICE_TOKEN=change_me
AI_TIMEOUT_SECONDS=60
AI_HTTP_RETRY_ATTEMPTS=2
AI_ANALYSIS_SYSTEM_PROMPT=你是态势感知分析助手。请基于输入的上下文、观测与约束，输出结构化、克制、可执行的分析。
```
### OpenAI-compatible example
```env
AI_PROVIDER=openai
AI_PROVIDER_API=openai-completions
AI_BASE_URL=http://127.0.0.1:8001/v1
AI_API_KEY=local-key
AI_MODEL=your-local-model
```
### MiniMax CN example
```env
AI_PROVIDER=minimax
AI_PROVIDER_API=anthropic-messages
AI_BASE_URL=https://api.minimaxi.com/anthropic
AI_API_KEY=sk-cp-xxxxx
AI_MODEL=MiniMax-M2.7
AI_MAX_TOKENS=1200
AI_ANTHROPIC_VERSION=2023-06-01
```
MiniMax note:
- This follows the same Anthropic Messages request shape as the official MiniMax examples.
- For MiniMax, `aiprovider` now disables `thinking` by default unless the caller explicitly passes a `thinking` object.
- This mirrors OpenClaw's caution around MiniMax Anthropic-compatible behavior.
### Anthropic-compatible example
```env
AI_PROVIDER=anthropic
AI_PROVIDER_API=anthropic-messages
AI_BASE_URL=https://your-claude-compatible-endpoint.example.com/anthropic
AI_API_KEY=your_api_key
AI_MODEL=your-model
AI_MAX_TOKENS=1200
AI_ANTHROPIC_VERSION=2023-06-01
```
### Ollama example
```env
AI_PROVIDER=ollama
AI_PROVIDER_API=ollama-generate
AI_BASE_URL=http://127.0.0.1:11434
AI_API_KEY=
AI_MODEL=qwen2.5:7b
```
## Deployment Modes
### Single machine
Recommended local flow:
- `backend` on `localhost:8000`
- `aiprovider` on `localhost:8010`
- local model gateway on `localhost:11434` or another local port
Helpers already included:
- [planet.sh](/home/ray/dev/linkong/planet/planet.sh)
- [docker-compose.local-model.yml](/home/ray/dev/linkong/planet/docker-compose.local-model.yml)
### Multi-machine
Example topology:
- app machine: `backend`
- AI gateway machine: `aiprovider`
- model machine: local model service or cloud proxy
In that case, this becomes service-to-service HTTP RPC:
- caller -> backend
- backend -> `http://10.0.0.12:8010`
- `aiprovider` -> model endpoint
Recommended cross-machine backend config:
```env
AI_PROVIDER_SERVICE_URL=http://10.0.0.12:8010
AI_PROVIDER_SERVICE_TOKEN=change_me
AI_PROVIDER_TIMEOUT_SECONDS=60
AI_PROVIDER_RETRY_ATTEMPTS=2
```
Recommended operating rules:
- keep `aiprovider` on a private network
- protect it with `X-Provider-Token` at minimum
- always send `X-Request-ID`
- keep callers on the backend API unless they are infrastructure jobs
## Retry And Failure Behavior
`backend -> aiprovider`:
- retries lightweight network / 5xx failures
- returns `502` when the provider service is unavailable
`aiprovider -> model provider`:
- retries lightweight network / 5xx failures
- returns `502` when the model provider is unavailable
This is intentionally conservative. It avoids masking persistent errors while still absorbing short hiccups.
## Operational Notes
- `./planet.sh start` now starts `aiprovider` automatically
- `./planet.sh restart -a` restarts only `aiprovider`
- `./planet.sh log -a` tails `aiprovider` logs
- `./planet.sh health` reports `aiprovider` health
## Recommended Calling Policy
- Frontend and application services: call `backend`
- Scheduled infra jobs and diagnostics: optionally call `aiprovider`
- Do not let multiple business services integrate model vendors independently
That keeps provider switching centralized and avoids model-specific drift across the system.

355
docs/bgp-context.md
View File

@@ -1,355 +0,0 @@
# BGP Context
## Current Goal
The BGP module is being evolved from an anomaly-only demo into a layered observability pipeline:
`raw observations -> enrichment -> detectors -> incidents -> console/Earth visualization`
The practical product goal is no longer just to "show incidents on the globe". The current product objective is:
1. keep BGP visually present on Earth even when incident density is low
2. make incidents clearly feel like a higher-confidence layer than anomalies
3. show that the observation network is still active even when there are no active incidents
In practice, that means Earth should behave like an observability surface, not only an incident map:
- `collectors` show that observation is happening
- `activity` shows where routing state is currently active or noisy
- `incidents` become the highest-confidence focus layer
## Current Backend Architecture
### Data Layers
1. `BGPObservation`
- File: `backend/app/models/bgp_observation.py`
- Purpose: store normalized raw routing observations from live/history sources.
- Typical fields:
- `source`
- `collector`
- `peer_asn`
- `peer_ip`
- `prefix`
- `event_type`
- `as_path`
- `origin_asn`
- `next_hop`
- `communities`
- `observed_at`
- `raw_payload`
- `collector_geo`
- `ingest_batch_id`
2. `BGPAnomaly`
- File: `backend/app/models/bgp_anomaly.py`
- Purpose: hold atomic detector outputs.
- Current detector output types include:
- `origin_change`
- `more_specific_burst`
- `mass_withdrawal`
3. `BGPIncident`
- File: `backend/app/models/bgp_incident.py`
- Purpose: aggregate atomic anomalies into incident-level objects for humans and the UI.
### Pipeline
Main flow is currently anchored in:
- `backend/app/services/collectors/bgp_common.py`
- `backend/app/services/bgp_enrichment.py`
- `backend/app/services/bgp_detectors.py`
- `backend/app/services/bgp_incidents.py`
Operational flow:
1. collectors fetch raw BGP data
2. `normalize_bgp_event()` standardizes payloads
3. observations are persisted to `bgp_observations`
4. enrichment augments events with analysis context
5. detectors create `bgp_anomalies`
6. incident aggregation rolls anomalies up into `bgp_incidents`
### Current Ingest Sources
1. `RIPE RIS Live`
- Collector file: `backend/app/services/collectors/ris_live.py`
- Used for realtime observation flow.
2. `CAIDA BGPStream Backfill`
- Collector file: `backend/app/services/collectors/bgpstream.py`
- Used as history/backfill entry point.
## Current Enrichment Status
Implemented enrichment skeleton in:
- `backend/app/services/bgp_enrichment.py`
Current enrichments:
- prefix family / prefix length
- supernet / more-specific derivation
- deduplicated AS path
- path prepending hints
- collector region info
- prefix baseline hints
- new-origin detection
- ASN organization profile from PeeringDB where available
- prefix scope / impacted region hints
- prefix geography source priority:
- `OpenGeoFeed` (override/high confidence)
- `IPtoASN` (country-range baseline)
- `NRO delegated stats` (registry-allocation fallback)
Current limitation:
- `RPKI` is still placeholder-only and returns `unknown`
- no real ROA validation source is integrated yet
- `inetnum` / `inet6num` whois fallback is still pending
## Current API Surface
Primary API file:
- `backend/app/api/v1/bgp.py`
Available endpoints:
- `/api/v1/bgp/events`
- `/api/v1/bgp/events/summary`
- `/api/v1/bgp/events/{id}`
- `/api/v1/bgp/anomalies`
- `/api/v1/bgp/anomalies/summary`
- `/api/v1/bgp/anomalies/{id}`
- `/api/v1/bgp/incidents`
- `/api/v1/bgp/incidents/summary`
- `/api/v1/bgp/incidents/{id}`
Visualization GeoJSON endpoints:
- `backend/app/api/v1/visualization.py`
- `/api/v1/visualization/geo/bgp-collectors`
- `/api/v1/visualization/geo/bgp-anomalies`
- `/api/v1/visualization/geo/bgp-incidents`
## Current Earth Behavior
Relevant files:
- `frontend/public/earth/js/bgp.js`
- `frontend/public/earth/js/main.js`
- `frontend/public/earth/js/info-card.js`
- `frontend/public/earth/js/constants.js`
- `frontend/public/earth/index.html`
Current design:
1. Collectors are always shown when BGP is enabled.
2. Incident markers are now the primary Earth BGP markers.
3. If there are no incidents, Earth falls back to anomaly markers.
4. If there are no anomalies either, collectors still provide presence.
5. A dedicated `activity layer` now adds:
- per-collector recent 15-minute activity halos
- clustered regional activity hints derived from active collectors
6. Incident markers now use:
- symbol-driven event cores
- outward ring pulses
- reduced diffuse glow compared with older Earth builds
5. The right-side stats now show:
- BGP events
- collector count
- BGP status summary
This is directionally correct, but still incomplete for low-event-density periods. Right now Earth can still feel too quiet when incidents are sparse because the system lacks a dedicated `activity layer` between raw observation and incident focus.
Current BGP status strategy:
- incidents present: show active incident count
- no incidents but anomalies present: show active anomaly count, plus active observation regions when available
- no incidents/anomalies but activity present: show `观测网络运行中`
- no incidents/anomalies but collectors present: show `观测网络运行中 · 当前未发现聚合级事件`
- no BGP data at all: show `暂无观测数据`
Earth info-card strategy:
- `bgp` card is now incident-centric in wording
- `bgp_collector` card shows collector location and current event count
## Current Product Gap
The main product gap is not architecture correctness. It is low-density visualization strategy.
Current reality:
- incident count is naturally much lower than anomaly count
- that is expected, because incidents are aggregated and de-noised
- but incident-first rendering makes the Earth view look too quiet unless there is another always-available activity layer
Implementation detail for the recommended `activity layer` is expanded in [bgp-region-aggregation-plan.md](/home/ray/dev/linkong/planet/docs/bgp-region-aggregation-plan.md).
So the immediate next milestone is:
`event map -> observability map`
That means Earth needs three simultaneously readable layers:
1. `observation layer`
- collectors
- recent collector activity
- baseline coverage
2. `activity layer`
- recent event density
- anomaly/noise hotspots
- regional activity scoring
- incident presence bonus
3. `incident layer`
- sparse but highly legible, high-confidence event objects
- symbol-driven markers
- outward ring pulse instead of broad diffuse glow
## Incident Visual Direction
The Earth `incident` layer should not read like a large glowing patch. It should read like a compact, high-confidence event focus.
Design principles:
1. `incident` markers should use a strong primary symbol
- the symbol shape should carry type meaning where possible
- examples:
- `origin_change`: triangle-like warning marker
- `mass_withdrawal`: alert/exclamation-style marker
- `more_specific_burst`: split/radiating marker
2. emphasis should come from outward ring pulses, not area flooding
- use a compact hot core
- use one or more expanding ring pulses
- avoid broad luminous blobs that make the event center feel vague
3. `collector` and `incident` must stay visually distinct
- collectors are observation infrastructure
- incidents are extracted event focus
- collector activity should stay quieter than incident pulse language
4. calm periods still need observability presence
- collectors and activity layers should keep the map alive
- once incidents appear, they should clearly dominate nearby BGP visuals
5. incident geography should become `prefix-centric`
- collectors should remain evidence sources, not the primary event location
- preferred geography priority:
- `prefix_geography`
- `prefix_scope`
- `ASN organization region`
- `collector centroid` as final fallback
- `prefix_scope` should remain an observation-derived scope hint
- a new `prefix_geography` layer should be introduced for actual prefix-centric placement
Reference inspiration:
- `World Monitor`
- sparse event symbols
- compact centers
- ring-like outward pulses
- stronger incident legibility than diffuse glow
## Current Console Behavior
Relevant page:
- `frontend/src/pages/BGP/BGP.tsx`
Current BGP console page has three levels:
1. observation summary
- total events
- collector count
- prefix count
2. incident summary and incident table
3. anomaly detail table plus recent observation events
This means the BGP page still has useful signal even when there are zero anomalies.
## Known Product/Engineering Boundaries
1. The current system is still closer to an event board than a full BGP sensing platform.
2. RIS coverage still needs to expand beyond narrow subscription scope.
3. BGPStream history is still not full MRT-to-prefix decoded analytics.
4. Collector geography still depends heavily on static RIPE RIS mappings.
5. Incident-to-cable/IXP/region association is still weak and early-stage.
6. Earth currently visualizes logical observation/impact structure, not true physical traffic paths.
## Test Status
BGP-specific tests live in:
- `backend/tests/test_bgp.py`
Verified status at this point:
- `25 passed` for `backend/tests/test_bgp.py`
- `62 passed` for `backend/tests`
Covered areas include:
- normalization
- observation serialization
- enrichment
- detectors, including route leak candidate and path flap
- incident aggregation
- batch anomaly creation
- BGP events/incidents API
- summary endpoints
## Most Relevant Files
Backend:
- `backend/app/models/bgp_observation.py`
- `backend/app/models/bgp_anomaly.py`
- `backend/app/models/bgp_incident.py`
- `backend/app/services/collectors/bgp_common.py`
- `backend/app/services/bgp_enrichment.py`
- `backend/app/services/bgp_detectors.py`
- `backend/app/services/bgp_incidents.py`
- `backend/app/api/v1/bgp.py`
- `backend/app/api/v1/visualization.py`
Frontend:
- `frontend/src/pages/BGP/BGP.tsx`
- `frontend/public/earth/js/bgp.js`
- `frontend/public/earth/js/main.js`
- `frontend/public/earth/js/info-card.js`
- `frontend/public/earth/js/constants.js`
- `frontend/public/earth/index.html`
## Recommended Next Steps
### Next Backend / Detection Priority
1. Integrate real RPKI validation data.
2. Expand realtime collector coverage and include withdrawals more broadly.
3. Continue refining route leak and path instability detectors with stronger heuristics.
### Next Correlation / Storytelling Priority
4. Strengthen incident aggregation semantics and titles.
5. Add weak correlation from incidents to:
- cable corridors
- landing points
- IXPs
- other traffic anomaly sources
6. Refine Earth hover/click handoff between collectors and incidents.
### Next Visualization Priority
7. Refine regional activity scoring so the activity layer is informative without becoming noisy.
8. Add more incident symbol types as new detectors land.
9. Add a real prefix geography source:
- `IPtoASN / IPtoCountry` as the first practical dataset
- `OpenGeoFeed` as a higher-quality override layer
- registry/whois only as fallback

296
docs/bgp-earth-rendering-plan.md
View File

@@ -1,296 +0,0 @@
# BGP Earth Rendering Plan
## Goal
This document defines how the BGP `region activity layer` and `incident layer` should coexist on Earth without conflicting.
The main question it answers is:
- how to add a regional observability background layer
- without weakening the current incident-first event focus
## Core Principle
The Earth design should follow a strict semantic hierarchy:
- `collector layer` = observation infrastructure
- `region activity layer` = background situational awareness
- `incident layer` = focal high-confidence event objects
In short:
- collectors prove the network is observing
- regions show where routing behavior is active or abnormal
- incidents show the concrete event worth clicking
Region aggregation is therefore not a replacement for incident rendering.
It is the context layer that makes sparse incident markers legible.
## Rendering Hierarchy
Recommended visual stack order:
1. collector network / collector halos
2. region activity glow
3. incident markers and incident pulses
This ordering should always hold.
Why:
- collectors should stay visible but quiet
- regions should create ambient activity presence
- incidents must remain the first thing users notice as a concrete event
## Role Separation
### Region Layer
The region layer answers:
- where is routing activity building up
- where is there current noise or instability
- which part of the world is currently worth looking at
The region layer should feel:
- broad
- ambient
- low-frequency
- contextual
### Incident Layer
The incident layer answers:
- which exact event should the user inspect
- where is the highest-confidence routing event located right now
The incident layer should feel:
- sharp
- compact
- high-contrast
- intentionally clickable
## Non-Conflict Rules
To avoid visual and semantic conflict, these implementation rules should be treated as hard constraints:
1. region markers must not use the same symbol language as incidents
2. region emphasis must stay weaker than incident emphasis
3. region animation frequency must stay lower than incident animation frequency
4. incident markers must always render above region glows
5. region layer should support the event, not compete with it
If a user notices the region layer first but misses the incident marker, the region layer is too strong.
If a user only sees isolated incident points and cannot feel broader activity context, the region layer is too weak.
## Region Rendering Rules
The region layer should not be rendered as a second kind of incident point.
Recommended representation:
- diffuse glow
- halo
- low-detail pulse
- soft center, not a sharp icon
### Status Mapping
#### `observing`
- weak glow
- cool color, such as cyan or blue
- little to no pulse
- purpose: keep the globe alive during calm periods
#### `anomaly`
- stronger glow
- warmer color, such as amber
- gentle breathing or low-frequency pulse
- purpose: show that a region is experiencing abnormal routing noise
#### `incident`
- strongest regional background emphasis
- still clearly weaker than the incident marker itself
- purpose: lift the surrounding area so the focal event does not feel isolated
### Region Visual Characteristics
Recommended properties:
- large radius
- low opacity
- soft edge
- low-contrast outline or no outline
- low pulse amplitude
Avoid:
- sharp symbol shapes
- strong icon silhouettes
- bright hard-edged centers
- incident-like pulse language
## Incident Rendering Rules
The incident layer should remain visually sharper and more explicit than region activity.
Recommended qualities:
- clear event symbol
- compact hot core
- one or two outward ring pulses
- high contrast
- clear click target
The incident layer should read as:
- focal
- deliberate
- high-confidence
while the region layer should read as:
- contextual
- ambient
- supporting
## Region And Incident In The Same Area
When a region contains one or more incidents:
- the region glow may intensify
- but the incident marker must remain the dominant local feature
Interpretation should be:
- `region` says this area is in an event state
- `incident marker` says this is the concrete event object
So a region with `incident` status is not itself the event marker.
It is the background state around the event.
## Interaction Model
Interaction should also preserve hierarchy.
### Click Region
Open a regional situation view, such as:
- region name
- observation count
- anomaly count
- incident count
- affected prefix count
- affected ASN count
- recent incidents in the region
### Click Incident
Keep the current incident-focused detail interaction.
This creates a natural two-step flow:
1. region gives context
2. incident gives detail
## Layer Relationship To Existing BGP Elements
### Collector Layer
Collectors should remain:
- quieter than regions
- more infrastructural than semantic
- proof of coverage, not proof of incident
### Region Layer
Regions should become:
- the main ambient activity layer
- the bridge between collectors and incidents
- the answer to low-density map quietness
### Incident Layer
Incidents should remain:
- the most legible event layer
- sparse but dominant
- compact and symbol-driven
## Practical Visual Test
Use this test when tuning the Earth implementation:
### Calm Period
Expected result:
- collectors visible
- some weak region glows present
- no region feels alarm-heavy
- globe still feels alive
### Anomaly Period
Expected result:
- one or more regions brighten noticeably
- user can sense the active area before clicking
- still no confusion between region background and incident objects
### Incident Period
Expected result:
- region provides broader context
- incident marker is the first explicit focal object the eye lands on
- user can immediately tell both:
- which region is active
- which specific event to inspect
## Failure Modes To Avoid
### Region Too Strong
Symptoms:
- incident markers disappear into the glow
- users treat the region center as the main event
- the map feels like area flooding instead of event focus
### Region Too Weak
Symptoms:
- incident markers still feel isolated
- low-incident periods still look visually empty
- users cannot tell where routing activity is generally happening
### Region Uses Incident Language
Symptoms:
- region and incident both look like event markers
- users cannot distinguish context from event
## Final Design Rule
The desired reading order is:
1. see the specific incident marker
2. feel the active region around it
3. understand that collectors and background activity keep the globe alive even during quieter periods
In one sentence:
`incident is the point; region is the field.`

487
docs/bgp-observability-plan.md
View File

@@ -1,487 +0,0 @@
# BGP Observability Plan
## Goal
Build a global routing observability capability on top of:
- [RIPE RIS Live](https://ris-live.ripe.net/)
- [CAIDA BGPStream data access overview](https://bgpstream.caida.org/docs/overview/data-access)
The target is to support:
- real-time routing event ingestion
- historical replay and baseline analysis
- anomaly detection
- Earth big-screen visualization
## Important Scope Note
These data sources expose the BGP control plane, not user traffic itself.
That means the system can infer:
- route propagation direction
- prefix reachability changes
- AS path changes
- visibility changes across collectors
But it cannot directly measure:
- exact application traffic volume
- exact user packet path
- real bandwidth consumption between countries or operators
Product wording should therefore use phrases like:
- global routing propagation
- route visibility
- control-plane anomalies
- suspected path diversion
Instead of claiming direct traffic measurement.
## Data Source Roles
### RIS Live
Use RIS Live as the real-time feed.
Recommended usage:
- subscribe to update streams over WebSocket
- ingest announcements and withdrawals continuously
- trigger low-latency alerts
Best suited for:
- hijack suspicion
- withdrawal bursts
- real-time path changes
- live Earth event overlay
### BGPStream
Use BGPStream as the historical and replay layer.
Recommended usage:
- backfill time windows
- build normal baselines
- compare current events against history
- support investigations and playback
Best suited for:
- historical anomaly confirmation
- baseline path frequency
- visibility baselines
- postmortem analysis
## Recommended Architecture
```mermaid
flowchart LR
A["RIS Live WebSocket"] --> B["Realtime Collector"]
C["BGPStream Historical Access"] --> D["Backfill Collector"]
B --> E["Normalization Layer"]
D --> E
E --> F["data_snapshots"]
E --> G["collected_data"]
E --> H["bgp_anomalies"]
H --> I["Alerts API"]
G --> J["Visualization API"]
H --> J
J --> K["Earth Big Screen"]
```
## Storage Design
The current project already has:
- [data_snapshot.py](/home/ray/dev/linkong/planet/backend/app/models/data_snapshot.py)
- [collected_data.py](/home/ray/dev/linkong/planet/backend/app/models/collected_data.py)
So the lowest-risk path is:
1. keep raw and normalized BGP events in `collected_data`
2. use `data_snapshots` to group each ingest window
3. add a dedicated anomaly table for higher-value derived events
## Proposed Data Types
### `collected_data`
Use these `source` values:
- `ris_live_bgp`
- `bgpstream_bgp`
Use these `data_type` values:
- `bgp_update`
- `bgp_rib`
- `bgp_visibility`
- `bgp_path_change`
Recommended stable fields:
- `source`
- `source_id`
- `entity_key`
- `data_type`
- `name`
- `reference_date`
- `metadata`
Recommended `entity_key` strategy:
- event entity: `collector|peer|prefix|event_time`
- prefix state entity: `collector|peer|prefix`
- origin state entity: `prefix|origin_asn`
### `metadata` schema for raw events
Store the normalized event payload in `metadata`:
```json
{
"project": "ris-live",
"collector": "rrc00",
"peer_asn": 3333,
"peer_ip": "2001:db8::1",
"event_type": "announcement",
"prefix": "203.0.113.0/24",
"origin_asn": 64496,
"as_path": [3333, 64500, 64496],
"communities": ["3333:100", "64500:1"],
"next_hop": "192.0.2.1",
"med": 0,
"local_pref": null,
"timestamp": "2026-03-26T08:00:00Z",
"raw_message": {}
}
```
### New anomaly table
Add a new table, recommended name: `bgp_anomalies`
Suggested columns:
- `id`
- `snapshot_id`
- `task_id`
- `source`
- `anomaly_type`
- `severity`
- `status`
- `entity_key`
- `prefix`
- `origin_asn`
- `new_origin_asn`
- `peer_scope`
- `started_at`
- `ended_at`
- `confidence`
- `summary`
- `evidence`
- `created_at`
This table should represent derived intelligence, not raw updates.
## Collector Design
## 1. `RISLiveCollector`
Responsibility:
- maintain WebSocket connection
- subscribe to relevant message types
- normalize messages
- write event batches into snapshots
- optionally emit derived anomalies in near real time
Suggested runtime mode:
- long-running background task
Suggested snapshot strategy:
- one snapshot per rolling time window
- for example every 1 minute or every 5 minutes
## 2. `BGPStreamBackfillCollector`
Responsibility:
- fetch historical data windows
- normalize to the same schema as real-time data
- build baselines
- re-run anomaly rules on past windows if needed
Suggested runtime mode:
- scheduled task
- or ad hoc task for investigations
Suggested snapshot strategy:
- one snapshot per historical query window
## Normalization Rules
Normalize both sources into the same internal event model.
Required normalized fields:
- `collector`
- `peer_asn`
- `peer_ip`
- `event_type`
- `prefix`
- `origin_asn`
- `as_path`
- `timestamp`
Derived normalized fields:
- `as_path_length`
- `country_guess`
- `prefix_length`
- `is_more_specific`
- `visibility_weight`
## Anomaly Detection Rules
Start with these five rules first.
### 1. Origin ASN Change
Trigger when:
- the same prefix is announced by a new origin ASN not seen in the baseline window
Use for:
- hijack suspicion
- origin drift detection
### 2. More-Specific Burst
Trigger when:
- a more-specific prefix appears suddenly
- especially from an unexpected origin ASN
Use for:
- subprefix hijack suspicion
### 3. Mass Withdrawal
Trigger when:
- the same prefix or ASN sees many withdrawals across collectors within a short window
Use for:
- outage suspicion
- regional incident detection
### 4. Path Deviation
Trigger when:
- AS path length jumps sharply
- or a rarely seen transit ASN appears
- or path frequency drops below baseline norms
Use for:
- route leak suspicion
- unusual path diversion
### 5. Visibility Drop
Trigger when:
- a prefix is visible from far fewer collectors/peers than its baseline
Use for:
- regional reachability degradation
## Baseline Strategy
Use BGPStream historical data to build:
- common origin ASN per prefix
- common AS path patterns
- collector visibility distribution
- normal withdrawal frequency
Recommended baseline windows:
- short baseline: last 24 hours
- medium baseline: last 7 days
- long baseline: last 30 days
The first implementation can start with only the 7-day baseline.
## API Design
### Raw event API
Add endpoints like:
- `GET /api/v1/bgp/events`
- `GET /api/v1/bgp/events/{id}`
Suggested filters:
- `prefix`
- `origin_asn`
- `peer_asn`
- `collector`
- `event_type`
- `time_from`
- `time_to`
- `source`
### Anomaly API
Add endpoints like:
- `GET /api/v1/bgp/anomalies`
- `GET /api/v1/bgp/anomalies/{id}`
- `GET /api/v1/bgp/anomalies/summary`
Suggested filters:
- `severity`
- `anomaly_type`
- `status`
- `prefix`
- `origin_asn`
- `time_from`
- `time_to`
### Visualization API
Add an Earth-oriented endpoint like:
- `GET /api/v1/visualization/geo/bgp-anomalies`
Recommended feature shapes:
- point: collector locations
- arc: inferred propagation or suspicious path edge
- pulse point: active anomaly hotspot
## Earth Big-Screen Design
Recommended layers:
### Layer 1: Collector layer
Show known collector locations and current activity intensity.
### Layer 2: Route propagation arcs
Use arcs for:
- origin ASN country to collector country
- or collector-to-collector visibility edges
Important note:
This is an inferred propagation view, not real packet flow.
### Layer 3: Active anomaly overlay
Show:
- hijack suspicion in red
- mass withdrawal in orange
- visibility drop in yellow
- path deviation in blue
### Layer 4: Time playback
Use `data_snapshots` to replay:
- minute-by-minute route changes
- anomaly expansion
- recovery timeline
## Alerting Strategy
Map anomaly severity to the current alert system.
Recommended severity mapping:
- `critical`
- likely hijack
- very large withdrawal burst
- `high`
- clear origin change
- large visibility drop
- `medium`
- unusual path change
- moderate more-specific burst
- `low`
- weak or localized anomalies
## Delivery Plan
### Phase 1
- add `RISLiveCollector`
- normalize updates into `collected_data`
- create `bgp_anomalies`
- implement 3 rules:
- origin change
- more-specific burst
- mass withdrawal
### Phase 2
- add `BGPStreamBackfillCollector`
- build 7-day baseline
- implement:
- path deviation
- visibility drop
### Phase 3
- add Earth visualization layer
- add time playback
- add anomaly filtering and drilldown
## Practical Implementation Notes
- Start with IPv4 first, then add IPv6 after the event schema is stable.
- Store the original raw payload in `metadata.raw_message` for traceability.
- Deduplicate events by a stable hash of collector, peer, prefix, type, and timestamp.
- Keep anomaly generation idempotent so replay and backfill do not create duplicate alerts.
- Expect noisy data and partial views; confidence scoring matters.
## Recommended First Patch Set
The first code milestone should include:
1. `backend/app/services/collectors/ris_live.py`
2. `backend/app/services/collectors/bgpstream.py`
3. `backend/app/models/bgp_anomaly.py`
4. `backend/app/api/v1/bgp.py`
5. `backend/app/api/v1/visualization.py`
add BGP anomaly geo endpoint
6. `frontend/src/pages`
add a BGP anomaly list or summary page
7. `frontend/public/earth/js`
add BGP anomaly rendering layer
## Sources
- [RIPE RIS Live](https://ris-live.ripe.net/)
- [CAIDA BGPStream Data Access Overview](https://bgpstream.caida.org/docs/overview/data-access)

422
docs/bgp-region-aggregation-plan.md
View File

@@ -1,422 +0,0 @@
# BGP Region Aggregation Plan
## Goal
This document refines the current BGP `activity layer` into an implementation-ready regional aggregation design.
Primary product goal:
- turn sparse prefix-level observations, anomalies, and incidents into a readable `regional observability layer`
- keep Earth visually alive during low-incident periods
- make `incident markers` remain the highest-confidence foreground layer instead of replacing them
This layer is not a new collector, detector, or raw storage table.
It is an aggregation/view-model layer:
`observations -> enrichment -> anomalies/incidents -> geography mapping -> region aggregation -> Earth/UI activity layer`
## Why This Layer Exists
Current product gap from [bgp-context.md](/home/ray/dev/linkong/planet/docs/bgp-context.md):
- incident density is naturally low
- anomaly density is higher, but still not enough to keep the globe expressive all the time
- collector presence alone proves coverage, but does not communicate `where routing is currently active or noisy`
So the missing middle layer is:
- `collectors` show that observation exists
- `regions` show where activity is building up
- `incidents` show the specific high-confidence focus events
## Scope
This plan is specifically for:
- a backend aggregation service
- a summary API for console/stats
- a GeoJSON API for Earth rendering
- an Earth background activity layer that supports, but does not replace, incident markers
This plan does not attempt to solve:
- exact prefix geolocation quality
- polygon-heavy geopolitical visualization
- persistent materialized region tables in v1
## Region Layer Definition
Recommended conceptual model:
- `region layer` = background situational awareness
- `incident layer` = focal event markers
That means:
- region activity should answer `where is routing behavior currently active or abnormal`
- incident markers should answer `which concrete event should the user click`
## Recommended Output Model
Suggested backend output object:
## `BGPRegionActivity`
```json
{
"region_key": "sea",
"region_name": "Southeast Asia",
"center_lat": 1.3521,
"center_lon": 103.8198,
"observation_count": 128,
"anomaly_count": 9,
"incident_count": 2,
"activity_score": 17.6,
"status": "incident",
"affected_prefix_count": 14,
"affected_asn_count": 6,
"collector_count": 5,
"first_seen_at": "2026-04-02T10:00:00Z",
"last_seen_at": "2026-04-02T10:12:00Z"
}
```
### Fields To Keep In MVP
- `region_key`
- `region_name`
- `center_lat`
- `center_lon`
- `observation_count`
- `anomaly_count`
- `incident_count`
- `activity_score`
- `status`
- `affected_prefix_count`
- `affected_asn_count`
- `collector_count`
- `first_seen_at`
- `last_seen_at`
### Fields To Delay
These are useful, but not required for the first implementation:
- `bounding_box`
- `top_incident_types`
- `top_prefixes`
- polygon geometry
## Region Definition Strategy
### Recommendation
Use a static region-definition table first.
Examples:
- `north_america`
- `south_america`
- `western_europe`
- `eastern_europe`
- `east_asia`
- `southeast_asia`
- `south_asia`
- `middle_east`
- `north_africa`
- `sub_saharan_africa`
- `oceania`
Why this is the right v1 choice:
- stable UI semantics
- strong readability on Earth
- easier debugging and explanation
- lower implementation cost than geohash or H3 grids
### Not Recommended For V1
- geohash cell aggregation
- H3 aggregation
- fine-grained lat/lon bucket maps
Those are more flexible, but they make the map feel fragmented and less explainable.
## Geography Mapping Strategy
Do not reduce the implementation to only `prefix -> exact geo`.
The region layer should follow the same geography-priority logic already implied by the current BGP direction:
1. `prefix_geography`
2. `prefix_scope`
3. `ASN organization region`
4. `collector centroid` fallback
This matters because exact prefix geography will often be incomplete or approximate.
The region layer should stay robust even when only partial enrichment is available.
## Backend Design
Recommended new service file:
- [backend/app/services/bgp_regions.py](/home/ray/dev/linkong/planet/backend/app/services/bgp_regions.py)
Suggested responsibilities:
- `map_record_to_region(...)`
- `aggregate_region_activity(...)`
- `build_region_geojson(...)`
- `resolve_activity_status(...)`
- `compute_activity_score(...)`
### Data Source Inputs
Use a recent rolling window, default `15 minutes`, and aggregate from:
- `BGPObservation`
- `BGPAnomaly`
- active `BGPIncident`
### Aggregation Flow
1. query observations in the time window
2. query anomalies in the same window
3. query active incidents in the same window or active status set
4. resolve each record to a best-effort region
5. accumulate per-region counters
6. compute score and status
7. return region activity list
## Status Model
Recommended status buckets:
- `idle`
- `observing`
- `anomaly`
- `incident`
Suggested rule:
```text
if incident_count > 0: incident
elif anomaly_count > 0: anomaly
elif observation_count > 0: observing
else: idle
```
This aligns well with the current Earth status language and keeps the visual mapping simple.
## Activity Score
The score should be a tunable heuristic, not a fixed truth model.
Recommended v1 formula:
```text
activity_score =
min(observation_count, 50) * 0.03
+ anomaly_count * 1.2
+ incident_count * 5.0
```
Why cap observations:
- observation volume is usually much larger than anomaly or incident volume
- uncapped observation counts would overwhelm the score
- capped observation counts preserve baseline presence without drowning real abnormality
### Practical Guidance
- treat coefficients as configuration-like constants
- expect to retune after looking at real data
- keep `incident` weight dominant
## API Design
### 1. Summary/List API
Suggested endpoint:
- `/api/v1/bgp/regions/activity`
Response shape:
```json
{
"window_minutes": 15,
"regions": []
}
```
Use cases:
- BGP console summaries
- right-side Earth stats
- future region list panels
### 2. GeoJSON API
Suggested endpoint:
- `/api/v1/visualization/geo/bgp-regions`
Response shape:
```json
{
"type": "FeatureCollection",
"features": []
}
```
Each feature should include:
- `geometry`
- v1: `Point`
- later: optional `Polygon`
- `properties`
- `region_key`
- `region_name`
- `status`
- `activity_score`
- `observation_count`
- `anomaly_count`
- `incident_count`
- `affected_prefix_count`
- `affected_asn_count`
- `collector_count`
## Earth Rendering Plan
Detailed visual layering guidance is expanded in [bgp-earth-rendering-plan.md](/home/ray/dev/linkong/planet/docs/bgp-earth-rendering-plan.md).
### Layer Relationship
- `region layer` = ambient background activity
- `incident marker` = focal event object
Do not replace incident markers with region markers.
### Region Visual Rules
Suggested mapping:
- `observing`
- weak glow
- low pulse or no pulse
- `anomaly`
- stronger glow
- more visible pulse
- `incident`
- strongest regional emphasis
- but still visually secondary to the incident marker itself
### Region Labels
Good v2 enhancement:
- show region name
- show counts like `2 incidents / 5 anomalies`
This is useful, but should come after the core aggregation and Earth glow layer are working.
## Interaction Model
### Click Region
Recommended detail payload:
- region name
- observation/anomaly/incident counts in the selected window
- affected prefix count
- affected ASN count
- collector count
- recent incidents in the region
### Click Incident
Keep the current incident-detail flow.
Interaction should feel hierarchical:
1. region gives situational context
2. incident gives event focus
## MVP Implementation Order
### Step 1
Define static `REGIONS` in code or config.
### Step 2
Map geography-enriched BGP records into regions using the fallback chain.
### Step 3
Aggregate recent window counts:
- `observation_count`
- `anomaly_count`
- `incident_count`
### Step 4
Compute `activity_score` and `status`.
### Step 5
Expose:
- `/api/v1/bgp/regions/activity`
- `/api/v1/visualization/geo/bgp-regions`
### Step 6
Render region glows on Earth behind incident markers.
## Out Of Scope For MVP
- persistent materialized region tables
- geohash or H3 support
- polygon-filled regional overlays
- detailed top-prefix ranking in the first release
- complicated scoring personalization
## Risks And Constraints
### Geography Quality
Prefix geography is approximate and incomplete.
The region layer must tolerate fallback-based placement.
### Query Cost
Dynamic aggregation is the right v1 choice, but repeated short-window queries may eventually need:
- in-process caching
- scheduled pre-aggregation
- materialized summaries
### UI Overcrowding
If region glow, collector activity, and incidents all become too strong at once, Earth readability will regress.
The region layer must remain supportive, not dominant.
## Final Recommendation
The current BGP roadmap should explicitly add:
- `region aggregation` as the concrete implementation of the missing `activity layer`
The recommended product interpretation is:
- `collectors` prove observation coverage
- `regions` communicate live routing activity and abnormality
- `incidents` remain the clearest high-confidence event objects
In one sentence:
`region aggregation is not a replacement for incidents; it is the situational background that makes sparse incidents feel legible on Earth.`

207
docs/collected-data-column-removal-plan.md
View File

@@ -1,207 +0,0 @@
# collected_data 强耦合列拆除计划
## 背景
当前 `collected_data` 同时承担了两类职责：
1. 通用采集事实表
2. 少数数据源的宽表字段承载
典型强耦合列包括：
- `country`
- `city`
- `latitude`
- `longitude`
- `value`
- `unit`
以及 API 层临时平铺出来的：
- `cores`
- `rmax`
- `rpeak`
- `power`
这些字段并不适合作为统一事实表的长期 schema。
推荐方向是：
- 表内保留通用稳定字段
- 业务差异字段全部归入 `metadata`
- API 和前端动态读取 `metadata`
## 拆除目标
最终希望 `collected_data` 只保留：
- `id`
- `snapshot_id`
- `task_id`
- `source`
- `source_id`
- `entity_key`
- `data_type`
- `name`
- `title`
- `description`
- `metadata`
- `collected_at`
- `reference_date`
- `is_valid`
- `is_current`
- `previous_record_id`
- `change_type`
- `change_summary`
- `deleted_at`
## 计划阶段
### Phase 1：读取层去依赖
目标：
- API / 可视化 / 前端不再优先依赖宽列表字段
- 所有动态字段优先从 `metadata` 取
当前已完成：
- 新写入数据时，将 `country/city/latitude/longitude/value/unit` 自动镜像到 `metadata`
- `/api/v1/collected` 优先从 `metadata` 取动态字段
- `visualization` 接口优先从 `metadata` 取动态字段
- 国家筛选已改成只走 `metadata->>'country'`
- `CollectedData.to_dict()` 已切到 metadata-first
- 变更比较逻辑已切到 metadata-first
- 已新增历史回填脚本：
[scripts/backfill_collected_data_metadata.py](/home/ray/dev/linkong/planet/scripts/backfill_collected_data_metadata.py)
- 已新增删列脚本：
[scripts/drop_collected_data_legacy_columns.py](/home/ray/dev/linkong/planet/scripts/drop_collected_data_legacy_columns.py)
涉及文件：
- [backend/app/core/collected_data_fields.py](/home/ray/dev/linkong/planet/backend/app/core/collected_data_fields.py)
- [backend/app/services/collectors/base.py](/home/ray/dev/linkong/planet/backend/app/services/collectors/base.py)
- [backend/app/api/v1/collected_data.py](/home/ray/dev/linkong/planet/backend/app/api/v1/collected_data.py)
- [backend/app/api/v1/visualization.py](/home/ray/dev/linkong/planet/backend/app/api/v1/visualization.py)
### Phase 2：写入层去依赖
目标：
- 采集器内部不再把这些字段当作数据库一级列来理解
- 统一只写：
- 通用主字段
- `metadata`
建议动作：
1. Collector 内部仍可使用 `country/city/value` 这种临时字段作为采集过程变量
2. 进入 `BaseCollector._save_data()` 后统一归档到 `metadata`
3. `CollectedData` 模型中的强耦合列已从 ORM 移除，写入统一归档到 `metadata`
### Phase 3：数据库删列
目标：
- 从 `collected_data` 真正移除以下列：
- `country`
- `city`
- `latitude`
- `longitude`
- `value`
- `unit`
注意：
- `cores / rmax / rpeak / power` 当前本来就在 `metadata` 里，不是表列
- 这四个主要是 API 平铺字段，不需要数据库删列
## 当前阻塞点
在正式删列前，还需要确认这些地方已经完全不再直接依赖数据库列：
### 1. `CollectedData.to_dict()`
文件：
- [backend/app/models/collected_data.py](/home/ray/dev/linkong/planet/backend/app/models/collected_data.py)
状态：
- 已完成
### 2. 差异计算逻辑
文件：
- [backend/app/services/collectors/base.py](/home/ray/dev/linkong/planet/backend/app/services/collectors/base.py)
状态：
- 已完成
- 当前已改成比较归一化后的 metadata-first payload
### 3. 历史数据回填
问题：
- 老数据可能只有列值，没有对应 `metadata`
当前方案：
- 在删列前执行一次回填脚本：
- [scripts/backfill_collected_data_metadata.py](/home/ray/dev/linkong/planet/scripts/backfill_collected_data_metadata.py)
### 4. 导出格式兼容
文件：
- [backend/app/api/v1/collected_data.py](/home/ray/dev/linkong/planet/backend/app/api/v1/collected_data.py)
现状：
- CSV/JSON 导出已基本切成 metadata-first
建议：
- 删列前再回归检查一次导出字段是否一致
## 推荐执行顺序
1. 保持新数据写入时 `metadata` 完整
2. 把模型和 diff 逻辑完全切成 metadata-first
3. 写一条历史回填脚本
4. 回填后观察一轮
5. 正式执行删列迁移
## 推荐迁移 SQL
仅在确认全部读取链路已去依赖后执行：
```sql
ALTER TABLE collected_data
DROP COLUMN IF EXISTS country,
DROP COLUMN IF EXISTS city,
DROP COLUMN IF EXISTS latitude,
DROP COLUMN IF EXISTS longitude,
DROP COLUMN IF EXISTS value,
DROP COLUMN IF EXISTS unit;
```
## 风险提示
1. 地图类接口对经纬度最敏感
必须确保所有地图需要的记录，其 `metadata.latitude/longitude` 已回填完整。
2. 历史老数据如果没有回填，删列后会直接丢失这些信息。
3. 某些 collector 可能仍隐式依赖这些宽字段做差异比较，删列前必须做一次全量回归。
## 当前判断
当前项目已经完成“代码去依赖 + 历史回填 + readiness 检查”。
下一步执行顺序建议固定为：
1. 先部署当前代码版本并重启后端
2. 再做一轮功能回归
3. 最后执行：
`uv run python scripts/drop_collected_data_legacy_columns.py`

402
docs/collected-data-history-plan.md
View File

@@ -1,402 +0,0 @@
# 采集数据历史快照化改造方案
## 背景
当前系统的 `collected_data` 更接近“当前结果表”：
- 同一个 `source + source_id` 会被更新覆盖
- 前端列表页默认读取这张表
- `collection_tasks` 只记录任务执行状态，不直接承载数据版本语义
这套方式适合管理后台，但不利于后续做态势感知、时间回放、趋势分析和版本对比。
如果后面需要回答下面这类问题，当前模型会比较吃力：
- 某条实体在过去 7 天如何变化
- 某次采集相比上次新增了什么、删除了什么、值变了什么
- 某个时刻地图上“当时的世界状态”是什么
- 告警是在第几次采集后触发的
因此建议把采集数据改造成“历史快照 + 当前视图”模型。
## 目标
1. 每次触发采集都保留一份独立快照，历史可追溯。
2. 管理后台默认仍然只看“当前最新状态”，不增加使用复杂度。
3. 后续支持：
- 时间线回放
- 两次采集差异对比
- 趋势分析
- 按快照回溯告警和地图状态
4. 尽量兼容现有接口，降低改造成本。
## 结论
不建议继续用以下两种单一模式：
- 直接覆盖旧数据
问题：没有历史，无法回溯。
- 软删除旧数据再全量新增
问题：语义不清，历史和“当前无效”混在一起，后续统计复杂。
推荐方案：
- 保留历史事实表
- 维护当前视图
- 每次采集对应一个明确的快照批次
## 推荐数据模型
### 方案概览
建议拆成三层：
1. `collection_tasks`
继续作为采集任务表，表示“这次采集任务”。
2. `data_snapshots`
新增快照表，表示“某个数据源在某次任务中产出的一个快照批次”。
3. `collected_data`
从“当前结果表”升级为“历史事实表”，每一行归属于一个快照。
同时再提供一个“当前视图”：
- SQL View / 物化视图 / API 查询层封装均可
- 语义是“每个 `source + source_id` 的最新有效记录”
### 新增表：`data_snapshots`
建议字段：
| 字段 | 类型 | 含义 |
|---|---|---|
| `id` | bigint PK | 快照主键 |
| `datasource_id` | int | 对应数据源 |
| `task_id` | int | 对应采集任务 |
| `source` | varchar(100) | 数据源名，如 `top500` |
| `snapshot_key` | varchar(100) | 可选，业务快照标识 |
| `reference_date` | timestamptz nullable | 这批数据的参考时间 |
| `started_at` | timestamptz | 快照开始时间 |
| `completed_at` | timestamptz | 快照完成时间 |
| `record_count` | int | 快照总记录数 |
| `status` | varchar(20) | `running/success/failed/partial` |
| `is_current` | bool | 当前是否是该数据源最新快照 |
| `parent_snapshot_id` | bigint nullable | 上一版快照，可用于 diff |
| `summary` | jsonb | 本次快照统计摘要 |
说明：
- `collection_tasks` 偏“执行过程”
- `data_snapshots` 偏“数据版本”
- 一个任务通常对应一个快照，但保留分层更清晰
### 升级表：`collected_data`
建议新增字段：
| 字段 | 类型 | 含义 |
|---|---|---|
| `snapshot_id` | bigint not null | 归属快照 |
| `task_id` | int nullable | 归属任务，便于追查 |
| `entity_key` | varchar(255) | 实体稳定键，通常可由 `source + source_id` 派生 |
| `is_current` | bool | 当前是否为该实体最新记录 |
| `previous_record_id` | bigint nullable | 上一个版本的记录 |
| `change_type` | varchar(20) | `created/updated/unchanged/deleted` |
| `change_summary` | jsonb | 字段变化摘要 |
| `deleted_at` | timestamptz nullable | 对应“本次快照中消失”的实体 |
保留现有字段：
- `source`
- `source_id`
- `data_type`
- `name`
- `title`
- `description`
- `country`
- `city`
- `latitude`
- `longitude`
- `value`
- `unit`
- `metadata`
- `collected_at`
- `reference_date`
- `is_valid`
### 当前视图
建议新增一个只读视图：
`current_collected_data`
语义：
- 对每个 `source + source_id` 只保留最新一条 `is_current = true` 且 `deleted_at is null` 的记录
这样：
- 管理后台继续像现在一样查“当前数据”
- 历史分析查 `collected_data`
## 写入策略
### 触发按钮语义
“触发”不再理解为“覆盖旧表”，而是：
- 启动一次新的采集任务
- 生成一个新的快照
- 将本次结果写入历史事实表
- 再更新当前视图标记
### 写入流程
1. 创建 `collection_tasks` 记录，状态 `running`
2. 创建 `data_snapshots` 记录，状态 `running`
3. 采集器拉取原始数据并标准化
4. 为每条记录生成 `entity_key`
- 推荐：`{source}:{source_id}`
5. 将本次记录批量写入 `collected_data`
6. 与上一个快照做比对，计算：
- 新增
- 更新
- 未变
- 删除
7. 更新本批记录的：
- `change_type`
- `previous_record_id`
- `is_current`
8. 将上一批同实体记录的 `is_current` 置为 `false`
9. 将本次快照未出现但上一版存在的实体标记为 `deleted`
10. 更新 `data_snapshots.status = success`
11. 更新 `collection_tasks.status = success`
### 删除语义
这里不建议真的删记录。
建议采用“逻辑消失”模型：
- 历史行永远保留
- 如果某实体在新快照里消失：
- 上一条历史记录补一条“删除状态记录”或标记 `change_type = deleted`
- 同时该实体不再出现在当前视图
这样最适合态势感知。
## API 改造建议
### 保持现有接口默认行为
现有接口：
- `GET /api/v1/collected`
- `GET /api/v1/collected/{id}`
- `GET /api/v1/collected/summary`
建议默认仍返回“当前视图”，避免前端全面重写。
### 新增历史查询能力
建议新增参数或新接口：
#### 1. 当前/历史切换
`GET /api/v1/collected?mode=current|history`
- `current`：默认，查当前视图
- `history`：查历史事实表
#### 2. 按快照查询
`GET /api/v1/collected?snapshot_id=123`
#### 3. 快照列表
`GET /api/v1/snapshots`
支持筛选：
- `datasource_id`
- `source`
- `status`
- `date_from/date_to`
#### 4. 快照详情
`GET /api/v1/snapshots/{id}`
返回：
- 快照基础信息
- 统计摘要
- 与上一版的 diff 摘要
#### 5. 快照 diff
`GET /api/v1/snapshots/{id}/diff?base_snapshot_id=122`
返回：
- `created`
- `updated`
- `deleted`
- `unchanged`
## 前端改造建议
### 1. 数据列表页
默认仍看当前数据，不改用户使用习惯。
建议新增：
- “视图模式”
- 当前数据
- 历史数据
- “快照时间”筛选
- “只看变化项”筛选
### 2. 数据详情页
详情页建议展示：
- 当前记录基础信息
- 元数据动态字段
- 所属快照
- 上一版本对比入口
- 历史版本时间线
### 3. 数据源管理页
“触发”按钮文案建议改成更准确的：
- `立即采集`
并在详情里补：
- 最近一次快照时间
- 最近一次快照记录数
- 最近一次变化数
## 迁移方案
### Phase 1：兼容式落地
目标：先保留当前页面可用。
改动：
1. 新增 `data_snapshots`
2. 给 `collected_data` 增加：
- `snapshot_id`
- `task_id`
- `entity_key`
- `is_current`
- `previous_record_id`
- `change_type`
- `change_summary`
- `deleted_at`
3. 现有数据全部补成一个“初始化快照”
4. 现有 `/collected` 默认改查当前视图
优点：
- 前端几乎无感
- 风险最小
### Phase 2：启用差异计算
目标：采集后可知道本次改了什么。
改动：
1. 写入时做新旧快照比对
2. 写 `change_type`
3. 生成快照摘要
### Phase 3：前端态势感知能力
目标：支持历史回放和趋势分析。
改动：
1. 快照时间线
2. 版本 diff 页面
3. 地图时间回放
4. 告警和快照关联
## 唯一性与索引建议
### 建议保留的业务唯一性
在“同一个快照内部”，建议唯一：
- `(snapshot_id, source, source_id)`
不要在整张历史表上强加：
- `(source, source_id)` 唯一
因为历史表本来就应该允许同一实体跨快照存在多条版本。
### 建议索引
- `idx_collected_data_snapshot_id`
- `idx_collected_data_source_source_id`
- `idx_collected_data_entity_key`
- `idx_collected_data_is_current`
- `idx_collected_data_reference_date`
- `idx_snapshots_source_completed_at`
## 风险点
1. 存储量会明显增加
- 需要评估保留周期
- 可以考虑冷热分层
2. 写入复杂度上升
- 需要批量 upsert / diff 逻辑
3. 当前接口语义会从“表”变成“视图”
- 文档必须同步
4. 某些采集器缺稳定 `source_id`
- 需要补齐实体稳定键策略
## 对当前项目的具体建议
结合当前代码，推荐这样落地：
### 短期
1. 先设计并落表：
- `data_snapshots`
- `collected_data` 新字段
2. 采集完成后每次新增快照
3. `/api/v1/collected` 默认查 `is_current = true`
### 中期
1. 在 `BaseCollector._save_data()` 中改成：
- 生成快照
- 批量写历史
- 标记当前
2. 将 `CollectionTask.id` 关联到 `snapshot.task_id`
### 长期
1. 地图接口支持按 `snapshot_id` 查询
2. 仪表盘支持“最近一次快照变化量”
3. 告警支持绑定到快照版本
## 最终建议
最终建议采用：
- 历史事实表：保存每次采集结果
- 当前视图：服务管理后台默认查询
- 快照表：承载版本批次和 diff 语义
这样既能保留历史，又不会把当前页面全部推翻重做，是最适合后续做态势感知的一条路径。

263
docs/collectors.md
View File

@@ -1,263 +0,0 @@
# 数据采集系统 (Collectors)
## 一、系统架构
```
┌─────────────────────────────────────────────────────────────────┐
│ 数据采集系统架构 │
├─────────────────────────────────────────────────────────────────┤
│ │
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │
│ │ TOP500 │ │ Epoch AI │ │ HuggingFace │ │
│ │ 采集器 │ │ 采集器 │ │ 采集器 │ │
│ └──────┬──────┘ └──────┬──────┘ └──────┬──────┘ │
│ │ │ │ │
│ └───────────────────┼───────────────────┘ │
│ ▼ │
│ ┌─────────────────────┐ │
│ │ BaseCollector │◄── 基类 (统一处理) │
│ │ run() 方法 │ │
│ └─────────┬───────────┘ │
│ │ │
│ ┌─────────────────┼─────────────────┐ │
│ ▼ ▼ ▼ │
│ ┌───────────┐ ┌───────────┐ ┌───────────┐ │
│ │ fetch() │ │transform()│ │ _save_data│ │
│ │ 获取原始数据 │ │ 数据转换 │ │ 保存到DB │ │
│ └───────────┘ └───────────┘ └───────────┘ │
│ │ │
│ ▼ │
│ ┌─────────────────────┐ │
│ │ CollectedData 表 │◄── 统一存储 │
│ └─────────────────────┘ │
│ │
│ ┌─────────────────────────────────────────────────────────┐ │
│ │ Scheduler (APScheduler) │ │
│ │ 定时任务调度: 每4小时/6小时/12小时/1天 自动执行 │ │
│ └─────────────────────────────────────────────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────────┘
```
## 二、工作流程 (Pipeline)
```python
# 1. Scheduler 触发 (定时 或 手动触发)
# ↓
# 2. run() 方法执行完整流水线
async def run(self, db):
# 2.1 检查采集器是否启用
if not collector_registry.is_active(self.name):
return {"status": "skipped"}
# 2.2 记录任务开始
task = CollectionTask(status="running")
db.add(task)
await db.commit()
# 2.3 FETCH - 获取原始数据 (由子类实现)
raw_data = await self.fetch()
# 2.4 TRANSFORM - 转换为统一格式
data = self.transform(raw_data)
# 2.5 SAVE - 保存到数据库
records_count = await self._save_data(db, data)
# 2.6 记录任务完成
task.status = "success"
task.records_processed = records_count
await db.commit()
```
**核心文件**: `backend/app/services/collectors/base.py`
## 三、采集器列表
| 采集器 | 数据类型 | 数据内容 | 采集频率 |
|--------|----------|----------|----------|
| TOP500 | supercomputer | 全球超级计算机排名 (算力、性能) | 4小时 |
| Epoch AI | gpu_cluster | GPU算力集群信息 | 6小时 |
| HuggingFace Models | model | AI模型信息 | 12小时 |
| HuggingFace Datasets | dataset | 数据集信息 | 12小时 |
| HuggingFace Spaces | space | Demo应用 | 1天 |
| PeeringDB | ixp/network/facility | 互联网交换点/网络/机房 | 1-2天 |
| TeleGeography | submarine_cable | 海底光缆信息 | 7天 |
## 四、数据格式 (统一存储到 CollectedData 表)
```python
# 每个采集器 parse_response() 返回格式
{
"source_id": "top500_1", # 原始系统ID (必填)
"name": "El Capitan", # 名称 (必填)
"description": "系统描述...", # 描述
"country": "United States", # 国家
"city": "Livermore, CA", # 城市
"latitude": "37.6819", # 纬度 (字符串)
"longitude": "-121.7681", # 经度 (字符串)
"value": "1742.00", # 性能值 (如算力)
"unit": "PFlop/s", # 单位
"metadata": { # 额外数据 (JSON)
"rank": 1,
"r_peak": 2746.38,
"cores": 11039616
},
"reference_date": "2025-11-01" # 数据参考日期
}
```
## 五、数据库表结构
**CollectedData 表** (`collected_data`)
| 字段 | 类型 | 说明 |
|------|------|------|
| id | SERIAL | 主键 |
| source | VARCHAR(100) | 数据源名称 (top500, huggingface等) |
| source_id | VARCHAR(100) | 原始数据ID |
| data_type | VARCHAR(50) | 数据类型 (supercomputer, model等) |
| name | VARCHAR(500) | 名称 |
| title | VARCHAR(500) | 标题 |
| description | TEXT | 描述 |
| country | VARCHAR(100) | 国家 |
| city | VARCHAR(100) | 城市 |
| latitude | VARCHAR(50) | 纬度 |
| longitude | VARCHAR(50) | 经度 |
| value | VARCHAR(100) | 性能值 |
| unit | VARCHAR(20) | 单位 |
| metadata | JSONB | 额外元数据 |
| collected_at | TIMESTAMP | 采集时间 |
| reference_date | TIMESTAMP | 数据参考日期 |
| is_valid | INTEGER | 是否有效 |
**核心文件**: `backend/app/models/collected_data.py`
## 六、TOP500 采集器示例 (完整流程)
```python
# 1. fetch() - 从网页获取HTML
async def fetch(self):
url = "https://top500.org/lists/top500/list/2025/11/"
response = await client.get(url)
return response.text # 返回HTML
# 2. parse_response() - 解析HTML为统一格式
def parse_response(self, html):
soup = BeautifulSoup(html, "html.parser")
table = soup.find("table")
for row in table.find_all("tr")[1:]: # 跳过表头
cells = row.find_all("td")
entry = {
"source_id": f"top500_{cells[0].text}", # "top500_1"
"name": cells[1].text.strip(), # "El Capitan"
"country": cells[2].text.strip(), # "United States"
"city": "", # 城市
"latitude": "", # 需进一步解析
"longitude": "",
"value": "1742.00", # Rmax
"unit": "PFlop/s",
"metadata": {
"rank": 1,
"cores": "11340000"
},
"reference_date": "2025-11-01"
}
data.append(entry)
return data
# 3. run() 自动调用 _save_data() 保存到数据库
```
**核心文件**: `backend/app/services/collectors/top500.py`
## 七、调度机制
```python
# 启动时注册所有采集器到定时任务
def start_scheduler():
for name, collector in collectors.items():
if collector_registry.is_active(name):
scheduler.add_job(
run_collector_task,
trigger=IntervalTrigger(hours=collector.frequency_hours),
id=name,
name=name
)
```
| 采集器 | 采集频率 |
|--------|----------|
| TOP500 | 每4小时 |
| Epoch AI | 每6小时 |
| HuggingFace | 每12小时 |
| PeeringDB | 每1-2天 |
| TeleGeography | 每7天 |
**核心文件**: `backend/app/services/scheduler.py`
## 八、相关代码文件
```
backend/app/services/collectors/
├── base.py # 基类: run() 流水线, _save_data() 保存
├── registry.py # 采集器注册表
├── scheduler.py # 定时任务调度 (APScheduler)
├── top500.py # TOP500采集器
├── epoch_ai.py # Epoch AI采集器
├── huggingface.py # HuggingFace采集器
├── peeringdb.py # PeeringDB采集器
└── telegeraphy.py # TeleGeography海底光缆采集器
backend/app/models/
└── collected_data.py # 统一数据模型
```
## 九、数据使用场景
采集的数据最终会：
1. **可视化展示** - 在UE5大屏上显示超级计算机、GPU集群、海底光缆的地理位置
2. **态势分析** - 统计全球算力分布、增长趋势
3. **告警系统** - 检测重要节点变化
## 十、采集器注册机制
采集器在应用启动时自动注册：
```python
# backend/app/services/collectors/__init__.py
collector_registry.register(TOP500Collector())
collector_registry.register(EpochAIGPUCollector())
collector_registry.register(HuggingFaceModelCollector())
collector_registry.register(HuggingFaceDatasetCollector())
collector_registry.register(HuggingFaceSpacesCollector())
collector_registry.register(PeeringDBIXPCollector())
collector_registry.register(PeeringDBNetworkCollector())
collector_registry.register(PeeringDBFacilityCollector())
collector_registry.register(TeleGeographyCableCollector())
collector_registry.register(TeleGeographyLandingPointCollector())
collector_registry.register(TeleGeographyCableSystemCollector())
```
**核心文件**: `backend/app/services/collectors/registry.py`
## 十一、触发采集
### 方式一：定时触发
系统启动时，APScheduler会自动根据各采集器的`frequency_hours`设置定时任务。
### 方式二：手动触发 API
```bash
# 触发TOP500采集
curl -X POST http://localhost:8000/api/v1/datasources/1/trigger \
-H "Authorization: Bearer <token>"
```
**核心文件**: `backend/app/api/v1/datasources.py`

486
docs/datasource-health-plan.md
View File

@@ -1,486 +0,0 @@
# Datasource Health Plan
## Overview
This document defines a phased plan for datasource health governance.
The goal is to make collectors observable, diagnosable, and recoverable when upstream APIs change, while avoiding unsafe automatic mutation of repository defaults.
The key principle is:
- do not let runtime automation rewrite repository default config
Instead, split responsibilities across:
- default config
- runtime overrides
- health check records
- agent-generated repair proposals
## Problem Statement
Collectors currently depend on third-party APIs, data downloads, mirrored JSON files, archive links, and web pages.
These upstream dependencies can fail in several ways:
- endpoint becomes unreachable
- endpoint still responds but schema changes
- content-type changes
- website shuts down or moves
- mirror link disappears
- HTML structure changes and scraping fails
- endpoint requires a new path or new host
We want a system that can:
- detect datasource health degradation early
- identify likely cause
- search for updated endpoints when reasonable
- apply safe runtime fixes without polluting default repo config
- preserve auditability and rollback
## Design Principles
1. Default config is stable
- `backend/app/core/data_sources.yaml` remains the repository baseline.
- It should be changed intentionally through normal development flow, not by autonomous runtime agents.
2. Runtime fixes are isolated
- Emergency or adaptive fixes should live in a runtime override layer.
- Overrides should be reversible and auditable.
3. Deterministic checks come first
- Use normal programmatic health checks before using LLMs.
- Only call an agent when deterministic checks indicate a meaningful failure.
4. Agents suggest before they mutate
- Agents should produce proposals with evidence and confidence.
- Application of a proposal should be controlled by policy.
5. Every repair is attributable
- Store what changed, why, who or what suggested it, and when it was applied.
## Configuration Layers
Recommended runtime precedence:
1. datasource endpoint override
2. datasource DB endpoint override
3. repository default YAML
4. collector internal fallback logic
Definitions:
- repository default YAML:
- `backend/app/core/data_sources.yaml`
- versioned baseline
- datasource DB endpoint override:
- existing `DataSourceConfig.endpoint`
- current runtime override entrypoint
- datasource endpoint override:
- a dedicated new override table
- used for health-repair and proposal application
- collector internal fallback logic:
- final defensive fallback
- should be minimized over time
## Recommended Architecture
### 1. Deterministic Health Checks
Each collector gets a health profile with checks such as:
- endpoint resolves
- HTTP request succeeds
- status code is acceptable
- content-type is expected
- body parses successfully
- minimum structural fields exist
- sample item count is plausible
- latency is within threshold
Output states:
- `healthy`
- `degraded`
- `failed`
- `schema_changed`
- `rate_limited`
- `auth_required`
### 2. Agent-Assisted Repair Discovery
Only triggered when deterministic health checks fail or return suspicious structure.
Agent responsibilities:
- search for current official endpoint or replacement path
- inspect likely upstream documentation or landing pages
- compare candidate endpoint output to collector expectations
- produce a repair proposal with confidence and evidence
Agent should not directly modify repository defaults.
### 3. Safe Runtime Repair Application
Repair proposals can be:
- reviewed manually
- auto-applied only under strict low-risk policy
Auto-apply should be limited to cases like:
- same trusted domain
- highly similar response structure
- repeated successful verification
- confidence above threshold
## Phased Delivery Plan
## Phase 1: Deterministic Health MVP
Goal:
- build health observability without automated repair
Scope:
- datasource health check task runner
- datasource health result persistence
- endpoint reachability + parse checks
- dashboard or API visibility into health status
Deliverables:
- health check service
- health check record table
- status endpoint
- scheduled or manual check trigger
No agent usage yet.
## Phase 2: Agent Repair Proposals
Goal:
- let agent investigate failing sources and propose updated endpoints
Scope:
- invoke agent only when datasource health is `failed` or `schema_changed`
- web search + page inspection
- candidate endpoint extraction
- proposal persistence
Deliverables:
- repair proposal schema
- proposal generation pipeline
- confidence and evidence model
- operator review view or API
Still no automatic config mutation.
## Phase 3: Runtime Overrides
Goal:
- allow approved proposals to take effect safely at runtime
Scope:
- add dedicated override storage
- runtime resolution prefers override over default config
- proposal application writes override only
Deliverables:
- endpoint override table
- override-aware resolution logic
- apply/reject endpoints
- rollback endpoint
Repository default YAML remains untouched.
## Phase 4: Limited Auto-Apply
Goal:
- safely automate a narrow slice of low-risk repairs
Scope:
- policy engine for auto-apply
- same-domain or trusted-domain checks
- structure validation
- staged verification after apply
Deliverables:
- auto-apply rules
- audit logs
- automatic post-apply health verification
- auto-disable or rollback on regression
## Data Model Draft
### datasource_health_checks
Purpose:
- store each health evaluation result
Suggested fields:
- `id`
- `datasource_id`
- `collector_name`
- `endpoint_checked`
- `status`
- `http_status`
- `content_type`
- `latency_ms`
- `sample_count`
- `error_message`
- `details`
- `checked_at`
`details` can store structured diagnostic data such as:
- parsed fields
- schema mismatch summary
- retry count
- exception class
### datasource_repair_proposals
Purpose:
- store agent-generated repair suggestions
Suggested fields:
- `id`
- `datasource_id`
- `collector_name`
- `old_endpoint`
- `candidate_endpoint`
- `reason`
- `confidence`
- `evidence_urls`
- `evidence_summary`
- `status`
- `created_by`
- `created_at`
- `reviewed_at`
Suggested `status` values:
- `proposed`
- `approved`
- `rejected`
- `applied`
- `expired`
### datasource_endpoint_overrides
Purpose:
- runtime endpoint override layer
Suggested fields:
- `id`
- `datasource_id`
- `collector_name`
- `endpoint`
- `reason`
- `source`
- `proposal_id`
- `enabled`
- `created_at`
- `updated_at`
Suggested `source` values:
- `manual`
- `health-agent`
- `migration`
## API Draft
### Health
- `GET /api/v1/datasources/health`
- `GET /api/v1/datasources/{id}/health`
- `POST /api/v1/datasources/{id}/health-check`
- `POST /api/v1/datasources/health-check-all`
### Repair proposals
- `GET /api/v1/datasources/{id}/repair-proposals`
- `POST /api/v1/datasources/{id}/repair-proposals/generate`
- `POST /api/v1/datasources/{id}/repair-proposals/{proposal_id}/approve`
- `POST /api/v1/datasources/{id}/repair-proposals/{proposal_id}/reject`
- `POST /api/v1/datasources/{id}/repair-proposals/{proposal_id}/apply`
### Overrides
- `GET /api/v1/datasources/{id}/overrides`
- `POST /api/v1/datasources/{id}/overrides`
- `PUT /api/v1/datasources/{id}/overrides/{override_id}`
- `DELETE /api/v1/datasources/{id}/overrides/{override_id}`
## Agent Contract Draft
When deterministic health fails, the agent should receive:
- datasource name
- collector name
- current endpoint
- current failure mode
- expected response shape summary
- known trusted domains
Expected output:
```json
{
"status": "proposal",
"candidate_endpoint": "https://example.com/api/v2/data",
"confidence": 0.86,
"reason": "Official docs now point to v2 endpoint",
"evidence_urls": [
"https://example.com/docs/api",
"https://example.com/changelog"
],
"notes": "Response shape appears compatible after light field remapping"
}
```
The agent should never output "rewrite the default yaml" as its primary action.
## Risk Analysis
### Risk: wrong endpoint chosen by agent
Mitigation:
- use trusted-domain allowlists
- require evidence URLs
- require confidence threshold
- add manual review for medium-risk sources
### Risk: endpoint responds but schema silently changed
Mitigation:
- deterministic schema checks
- parse and sample validation
- content-type checks
- collector-specific required fields
### Risk: automatic runtime override causes hidden drift
Mitigation:
- store all overrides explicitly
- mark source of override
- keep default YAML unchanged
- expose active overrides in API/UI
### Risk: persistent bad override breaks data collection
Mitigation:
- allow rollback
- keep parent/default endpoint visible
- re-run verification after apply
- auto-disable override on repeated failure
## Operational Policy Recommendations
1. Do not auto-apply for high-value or high-fragility sources initially.
2. Use manual approval for:
- scraped HTML sources
- unofficial mirrors
- sources with auth or rate-limit complexity
- sources with legal or trust ambiguity
3. Allow auto-apply only for:
- same-domain version bumps
- obvious official migration paths
- repeated passing verification
4. Expose health + proposal + override state together in one operator view.
## Suggested Implementation Order
1. Phase 1
- health result table
- deterministic checks
- API and UI visibility
2. Phase 2
- proposal table
- agent prompt/output contract
- proposal generation job
3. Phase 3
- runtime override table
- resolver precedence update
- apply/reject endpoints
4. Phase 4
- auto-apply rules
- rollback policy
- operator automation
## Out Of Scope For The First Iteration
- direct automatic mutation of repository default YAML
- automatic git commits by repair agents
- unrestricted autonomous endpoint replacement
- fully generalized schema remapping engine
## Recommended First Milestone
The first milestone should be:
- deterministic datasource health checks
- persisted results
- manual visibility
- no automatic repair
This gives immediate operational value with low risk, and prepares clean inputs for the later agent phase.

478
docs/datasource-health-stage2-tasks.md
View File

@@ -1,478 +0,0 @@
# Datasource Health Stage 2 Tasks
## Goal
Stage 2 focuses on the first practical operational layer:
- deterministic datasource health checks
- persisted health results
- health visibility through API and UI
- no agent-assisted repair yet
This stage should make Planet capable of answering:
- which collectors are healthy
- which collectors are degraded
- which collectors are failing
- why they are failing at a basic deterministic level
## Scope
Included:
- datasource health data model
- deterministic health check service
- manual and scheduled health check triggers
- health result APIs
- frontend visibility
Excluded:
- LLM reasoning
- web-search-based repair proposals
- automatic endpoint rewriting
- runtime override application
## Delivery Target
At the end of Stage 2, an operator should be able to:
1. see health status for each collector
2. trigger a health check manually
3. inspect the latest failure reason
4. inspect the last checked endpoint
5. understand whether the problem is:
- unreachable
- auth-related
- rate-limit-related
- schema-related
- empty-data-related
## Work Breakdown
## A. Data Model
### A1. Add datasource health record table
Create a new model, for example:
- `backend/app/models/datasource_health_check.py`
Suggested fields:
- `id`
- `datasource_id`
- `collector_name`
- `endpoint_checked`
- `status`
- `http_status`
- `content_type`
- `latency_ms`
- `sample_count`
- `error_message`
- `details`
- `checked_at`
Suggested status enum values:
- `healthy`
- `degraded`
- `failed`
- `schema_changed`
- `rate_limited`
- `auth_required`
- `empty_result`
### A2. Add datasource health summary fields
Option A:
- keep summary only in the health check table
Option B:
- also add summary fields on `data_sources`
Recommended first step:
- do not mutate `data_sources` schema yet
- derive summary from the latest health record
### A3. Migration task
Add migration for the health table.
Deliverables:
- migration file
- model registration
## B. Health Check Engine
### B1. Define health check service
Add a new service module, for example:
- `backend/app/services/datasource_health.py`
Responsibilities:
- resolve effective endpoint
- execute deterministic check
- classify result
- persist health record
### B2. Define shared result schema
Create a typed result object, for example:
- `HealthCheckResult`
Suggested fields:
- `status`
- `endpoint_checked`
- `http_status`
- `content_type`
- `latency_ms`
- `sample_count`
- `error_message`
- `details`
### B3. Implement base deterministic checks
Every datasource should go through a minimal baseline check:
1. resolve endpoint
2. perform request
3. measure latency
4. inspect status code
5. inspect content type
6. inspect body shape
Classification rules:
- network error -> `failed`
- HTTP 401/403 -> `auth_required`
- HTTP 429 -> `rate_limited`
- HTTP 404/410 -> `failed`
- parse failure -> `schema_changed`
- zero or suspiciously empty results -> `empty_result` or `degraded`
- valid parse -> `healthy`
### B4. Add collector-aware adapters
Some collectors do not use the same fetch semantics.
Add adapter profiles such as:
- `http_json`
- `http_csv`
- `html_scrape`
- `stream_probe`
- `auth_session_http`
Initial mapping suggestion:
- `huggingface`, `peeringdb`, `cloudflare` -> `http_json`
- `fao` -> `http_csv`
- `top500`, `epoch_ai`, `telegeography live_map` -> `html_scrape`
- `ris_live` -> `stream_probe`
- `spacetrack` -> `auth_session_http`
### B5. Add sample validation hooks
For each adapter, add a lightweight validation rule.
Examples:
- JSON array length > 0
- CSV rows > 1
- HTML page contains expected table or script patterns
- stream source yields at least one valid event within timeout
## C. Persistence and Query Layer
### C1. Save every check run
Each health check should insert a record.
Do not overwrite history in Stage 2.
### C2. Add latest-health query helpers
Add helper functions to fetch:
- latest health record by datasource
- latest failed health record
- recent health history
### C3. Optional retention policy
For Stage 2, retention can be deferred.
If desired, keep only:
- last N records per datasource
## D. API Layer
### D1. Add health list endpoint
Suggested endpoint:
- `GET /api/v1/datasources/health`
Returns:
- datasource id
- collector name
- current endpoint
- latest health status
- last checked time
- short reason
### D2. Add per-datasource health detail endpoint
Suggested endpoint:
- `GET /api/v1/datasources/{id}/health`
Returns:
- latest record
- recent history
- detailed classification fields
### D3. Add manual health trigger endpoint
Suggested endpoint:
- `POST /api/v1/datasources/{id}/health-check`
Behavior:
- run a health check now
- persist the result
- return the new record
### D4. Add bulk health trigger endpoint
Suggested endpoint:
- `POST /api/v1/datasources/health-check-all`
Behavior:
- enqueue or run health checks for all active datasources
## E. Scheduling
### E1. Add health scheduler task
Decide scheduling strategy.
Recommended first version:
- run collector jobs and health checks separately
- health checks run on a lower frequency
Suggested frequency:
- every 6h or 12h for most datasources
- optionally on-demand only in the very first cut
### E2. Prevent health check collision with collection
Rules:
- health checks should not disrupt active collection
- they should use light requests
- if a collector is currently running, health check may:
- skip
- or use a lightweight endpoint probe only
## F. Frontend
### F1. Add health columns to datasource list
Update:
- `frontend/src/pages/DataSources/DataSources.tsx`
Suggested new columns:
- health status
- last checked
- reason summary
### F2. Add manual health check action
Per datasource:
- button or dropdown action:
- `健康检查`
### F3. Add health detail drawer or modal
Show:
- endpoint checked
- status
- HTTP status
- content type
- sample count
- error message
- last few results
### F4. Add basic visual language
Suggested colors:
- green -> healthy
- yellow -> degraded
- orange -> rate-limited / auth-required
- red -> failed / schema-changed
## G. Observability
### G1. Structured logging
Every health check should log:
- datasource id
- collector name
- endpoint
- status
- latency
- failure class
### G2. Optional metrics
If metrics are added later, useful counters include:
- health checks total
- health checks failed
- schema changes detected
- rate limited checks
## H. Tests
### H1. Unit tests
Add tests for:
- status classification
- content type classification
- adapter behavior
- latest-health query helpers
### H2. API tests
Add tests for:
- health endpoints require auth
- manual trigger endpoint works
- list endpoint returns latest status
### H3. Failure-path tests
Add coverage for:
- HTTP 404
- HTTP 429
- invalid JSON
- empty response
- parse mismatch
## Suggested File Plan
Possible implementation files:
- `backend/app/models/datasource_health_check.py`
- `backend/app/services/datasource_health.py`
- `backend/app/schemas/datasource_health.py`
- `backend/app/api/v1/datasource_health.py`
- migration file under the project migration system
Likely touched existing files:
- `backend/app/api/main.py`
- `frontend/src/pages/DataSources/DataSources.tsx`
- `backend/tests/test_api.py`
## Suggested Execution Order
1. Add model and migration
2. Add service and result schema
3. Add deterministic adapters
4. Add manual trigger API
5. Add list/detail API
6. Add frontend visibility
7. Add scheduled checks
8. Expand tests
## Minimal First Milestone
If we want the fastest useful slice, do this first:
1. health table
2. deterministic check service
3. manual per-datasource health check API
4. latest health list API
5. frontend status badge column
That is enough to start operating the system and will provide the input layer for Stage 3.
## Dependency On Later Stages
Stage 2 outputs become direct inputs for Stage 3.
Specifically:
- failed or schema-changed health records become agent triggers
- health history becomes repair context
- endpoint_checked becomes proposal baseline
## Success Criteria
Stage 2 is done when:
- every active datasource can be health-checked deterministically
- the latest health state is visible in API and UI
- operators can manually trigger checks
- failures are categorized into stable machine-readable statuses
- no LLM is required for core health visibility

105
docs/docker-compose-buildx-upgrade.md
View File

@@ -1,105 +0,0 @@
# Docker + Compose + Buildx 升级教程
流程：删除旧版 -> 安装新版 -> 验证
---
# 1. 删除旧版本
## 删除 apt 安装的旧包
```bash
sudo apt remove -y docker.io docker-compose docker-compose-v2 docker-doc podman-docker containerd runc
```
---
## 删除系统中的 `docker-compose`（V1）
```bash
sudo rm -f "$(which docker-compose 2>/dev/null)"
```
---
## 查找并删除手动安装的 Buildx 插件
```bash
docker info | sed -n '/Plugins:/,/^ Server:/p' | grep -A2 buildx
```
从输出中获取 `Path`，然后执行：
```bash
rm -f <Path中对应的docker-buildx文件>
```
---
## 清理无用依赖
```bash
sudo apt autoremove -y
```
---
# 2. 安装 Docker 官方版本
包含 Docker Engine、Docker Compose 插件、Docker Buildx 插件。
## 安装依赖
```bash
sudo apt update
sudo apt install -y ca-certificates curl gnupg
```
---
## 添加 Docker GPG key
```bash
sudo install -m 0755 -d /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | \
sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
sudo chmod a+r /etc/apt/keyrings/docker.gpg
```
---
## 添加官方仓库
```bash
echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \
sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
```
---
## 安装 Docker + Compose + Buildx
```bash
sudo apt update
sudo apt install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
```
---
# 3. 验证安装
```bash
docker --version
docker compose version
docker buildx version
```
---
# 4. 常用命令
```bash
docker compose up -d
docker compose down
docker buildx build .
```

210
docs/earth-module-plan.md
View File

@@ -1,210 +0,0 @@
# Earth 模块整治计划
## 背景
`planet` 前端中的 Earth 模块是当前最重要的大屏 3D 星球展示能力，但它仍以 legacy iframe 页面形式存在：
- React 页面入口仅为 [Earth.tsx](/home/ray/dev/linkong/planet/frontend/src/pages/Earth/Earth.tsx)
- 实际 3D 实现位于 [frontend/public/earth](/home/ray/dev/linkong/planet/frontend/public/earth)
当前模块已经具备基础展示能力，但在生命周期、性能、可恢复性、可维护性方面存在明显隐患，不适合长期无人值守的大屏场景直接扩展。
## 目标
本计划的目标不是立刻重写 Earth，而是分阶段把它从“能跑的 legacy 展示页”提升到“可稳定运行、可持续演进的大屏核心模块”。
核心目标：
1. 先止血，解决资源泄漏、重载污染、假性卡顿等稳定性问题
2. 再梳理数据加载、交互和渲染循环，降低性能风险
3. 最后逐步从 iframe legacy 向可控模块化架构迁移
## 现阶段主要问题
### 1. 生命周期缺失
- 没有统一 `destroy()` / 卸载清理逻辑
- `requestAnimationFrame`
- `window/document/dom listeners`
- `THREE` geometry / material / texture
- 运行时全局状态
都没有系统回收
### 2. 数据重载不完整
- `reloadData()` 没有彻底清理旧场景对象
- cable、landing point、satellite 相关缓存与对象存在累积风险
### 3. 渲染与命中检测成本高
- 鼠标移动时频繁创建 `Raycaster` / `Vector2`
- cable 命中前会重复做 bounding box 计算
- 卫星每帧计算量偏高
### 4. 状态管理分裂
- 大量依赖 `window.*` 全局桥接
- 模块之间靠隐式共享状态通信
- React 外层无法有效感知 Earth 内部状态
### 5. 错误恢复弱
- 数据加载失败主要依赖 `console` 和轻提示
- 缺少统一重试、降级、局部失败隔离机制
## 分阶段计划
## Phase 1：稳定性止血
目标：
- 不改视觉主形态
- 优先解决泄漏、卡死、重载污染
### 任务
1. 补 Earth 生命周期管理
- 为 [main.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/main.js) 增加：
- `init()`
- `destroy()`
- `reloadData()`
三类明确入口
- 统一记录并释放：
- animation frame id
- interval / timeout
- DOM 事件监听
- `window` 暴露对象
2. 增加场景对象清理层
- 为 cable / landing point / satellite sprite / orbit line 提供统一清理函数
- reload 前先 dispose 旧对象，再重新加载
3. 增加 stale 状态恢复
- 页面重新进入时，先清理上一次遗留选择态、hover 态、锁定态
- 避免 iframe reload 后出现旧状态残留
4. 加强失败提示
- 电缆、登陆点、卫星加载拆分为独立状态
- 某一类数据失败时，其它类型仍可继续显示
- 提供明确的页面内提示而不是只打 console
### 验收标准
- 页面重复进入 / 离开后内存不持续上涨
- 连续多次点“重新加载数据”后对象数量不异常增加
- 单一数据源加载失败时页面不整体失效
## Phase 2：性能优化
目标：
- 控制鼠标交互和动画循环成本
- 提升大屏长时间运行的稳定帧率
### 任务
1. 复用交互对象
- 复用 `Raycaster`、`Vector2`、中间 `Vector3`
- 避免 `mousemove` 热路径中频繁 new 对象
2. 优化 cable 命中逻辑
- 提前缓存 cable 中心点 / bounding 数据
- 移除 `mousemove` 内重复 `computeBoundingBox()`
- 必要时增加分层命中：
- 先粗筛
- 再精确相交
3. 改造动画循环
- 使用真实 `deltaTime`
- 把卫星位置更新、呼吸动画、视觉状态更新拆成独立阶段
- 为不可见对象减少无意义更新
4. 卫星轨迹与预测轨道优化
- 评估轨迹更新频率
- 对高开销几何计算增加缓存
- 限制预测轨道生成频次
### 验收标准
- 鼠标移动时不明显掉帧
- 中高数据量下动画速度不受帧率明显影响
- 长时间运行 CPU/GPU 占用更平稳
## Phase 3：架构收编
目标：
- 降低 legacy iframe 架构带来的维护成本
- 让 React 主应用重新获得对 Earth 模块的控制力
### 任务
1. 抽离 Earth App Shell
- 将数据加载、错误状态、控制面板状态抽到更明确的模块边界
- 减少 `window.*` 全局依赖
2. 规范模块通信
- 统一 `main / controls / cables / satellites / ui` 的状态流
- 明确只读配置、运行时状态、渲染对象的职责分层
3. 评估去 iframe 迁移
- 中期可以保留 public/legacy 资源目录
- 但逐步把 Earth 作为前端内嵌模块而不是完全孤立页面
### 验收标准
- Earth 内部状态不再大量依赖全局变量
- React 外层可以感知 Earth 加载状态和错误状态
- 后续功能开发不再必须修改多个 legacy 文件才能完成
## 优先级建议
### P0
- 生命周期清理
- reload 清理
- stale 状态恢复
### P1
- 命中检测优化
- 动画 `deltaTime`
- 数据加载失败隔离
### P2
- 全局状态收编
- iframe 架构迁移
## 推荐实施顺序
1. 先做 Phase 1
2. 再做交互热路径与动画循环优化
3. 最后再考虑架构迁移
## 风险提示
1. Earth 是 legacy 模块，修复时容易牵一发而动全身
2. 如果不先补清理逻辑，后续所有性能优化收益都会被泄漏问题吃掉
3. 如果过早重写而不先止血，短期会影响现有演示稳定性
## 当前建议
最值得马上启动的是一个小范围稳定性 sprint：
- 生命周期清理
- reload 全量清理
- 错误状态隔离
这个阶段不追求“更炫”，先追求“更稳”。稳定下来之后，再进入性能和架构层的优化。

117
docs/earth-tv-live-module-plan.md
View File

@@ -1,117 +0,0 @@
# Earth 电视直播模块计划
## 目标
为 `Earth` 页面增加一个可配置、可扩展、可拖拽的电视直播模块：
- 后台可配置新闻直播源
- 默认兜底源为央视 `CCTV-4`
- 未来可通过采集器接入世界各地新闻直播源
- Earth 工具栏 `显示控制` 子菜单新增电视按钮
- 点击后打开一个与其他 HUD 一致的可拖拽/可关闭窗口
- 窗口内部可播放或承载新闻直播页面
## 设计原则
- 第一阶段先交付“后台可配 + Earth 可用 + 默认可回退”的版本
- 公开读取接口与后台管理接口分离
- 手工配置源与采集器源共用统一的前端消费结构
- Earth 里的电视窗口必须复用现有 HUD 拖拽、关闭、布局最大化逻辑
- 小屏下优先保证窗口完整显示，超出部分在窗口内部滚动
## 分阶段实现
### Phase 1：后端配置与公开读取
- 在系统设置中新增 `tv` 分类
- 定义直播源配置结构：
- `default_source_id`
- `auto_fallback`
- `sources[]`
- 每个直播源至少包含：
- `id`
- `name`
- `provider`
- `region`
- `language`
- `source_type`
- `embed_url`
- `stream_url`
- `homepage_url`
- `is_enabled`
- `is_fallback`
- `sort_order`
- `collector_source`
- `notes`
- 默认兜底源使用央视官网 `CCTV-4` 直播页
- 新增公开读取接口，供 Earth 页面无登录态读取直播源配置
### Phase 2：采集器扩展位
- 新增 `news_live_streams` collector 占位
- 规范采集器入库数据结构，使其能与后台手工配置源合并
- TV 公开接口支持合并：
- 后台手工配置源
- 采集器入库源
- 保持手工配置源优先级更高，避免采集器覆盖人工兜底配置
### Phase 3：后台配置界面
- 在系统配置页新增 `电视直播` tab
- 支持：
- 查看当前默认源
- 开关自动回退
- 新增直播源
- 编辑直播源
- 删除直播源
- 启用/禁用直播源
- 将某个直播源设为默认源
- 明确区分：
- 手工配置源
- 采集器来源
### Phase 4：Earth HUD 集成
- 在 `显示控制` 子菜单加入电视按钮
- 新增 TV HUD 面板：
- 可拖拽
- 可关闭
- 支持显示/隐藏状态同步
- 参与布局最大化与恢复布局
- 面板内容至少包含：
- 当前频道标题
- 源切换下拉菜单
- 刷新按钮
- 打开官网按钮
- 播放区域
### Phase 5：播放策略
- 第一版优先支持 `iframe`/嵌入页类直播源
- 为未来扩展保留：
- `hls`
- `video`
- `external`
- 如果默认源不可用：
- 优先回退到标记为 `is_fallback=true` 的源
- 若无明确回退源，则回退到第一个可用源
- 面板内要有清晰的加载、错误、回退提示
### Phase 6：打磨与清理
- 统一 HUD 风格
- 小屏下限制窗口尺寸并启用内部滚动
- 避免窗口超出屏幕
- 补最小验证
- 清理临时代码、重复样式和无用资源
## 首版交付定义
当以下条件满足时，认为首版可用：
- 后台可以配置新闻直播源
- Earth 可以读取并显示默认直播源
- 工具栏可打开电视窗口
- 电视窗口可拖拽、可关闭
- 央视 `CCTV-4` 作为默认兜底源可被使用
- 代码结构已为后续采集器接入预留统一接口

309
docs/frontend-layout-guidelines.md
View File

@@ -1,309 +0,0 @@
# Frontend Layout Guidelines
本项目后台页面默认遵循“单屏工作区”布局规范。目标不是让页面永远不溢出，而是确保在常见桌面视口下：
- 页面主结构能在一屏内看清
- 用户能同时看到页头、摘要区和主工作区
- 超出的内容在模块内部滚动，而不是把整页纵向撑爆
当前推荐参考实现：
- [frontend/src/pages/BGP/BGP.tsx](/home/ray/dev/linkong/planet/frontend/src/pages/BGP/BGP.tsx)
- [frontend/src/index.css](/home/ray/dev/linkong/planet/frontend/src/index.css)
## 核心原则
### 1. 页面优先保证一屏工作区
管理页默认采用：
- 页头：标题、说明、主要操作
- 主工作区：统计卡、表格、图表、列表、标签页
推荐结构：
```tsx
<AppLayout>
<div className="page-shell">
<div className="page-shell__header">...</div>
<div className="page-shell__body">...</div>
</div>
</AppLayout>
```
页面总高度应被限制在 `AppLayout` 内容区内，而不是继续让整个页面自然向下增长。
### 2. 滚动优先发生在模块内部
如果表格、日志、长列表、图表明细超出空间：
- 让卡片内部滚动
- 让表格内部滚动
- 让标签页内容区内部滚动
不要默认依赖整个页面滚动去“解决”空间问题。
### 3. 主工作区必须拿到主要空间
页面里最重要的模块必须是视觉和空间上的主角。通常应保证：
- 页头始终可见
- 摘要区高度被控制
- 主表格 / 主图表 / 主分析区占据 50% 以上可视高度
如果一个页面有多个大模块，优先顺序是：
1. 先压缩说明区和摘要区
2. 再把次级模块收进标签页或切换视图
3. 最后才考虑继续增加整页滚动
### 4. 小屏幕和高缩放必须进入紧凑模式
在窗口高度较低、宽度较窄、或系统缩放较高时，应主动切换紧凑布局，例如：
- 缩小卡片 padding
- 缩小表头和单元格间距
- 将摘要区改为更紧凑的单行/横向滚动布局
- 将次级模块移入标签页、抽屉、折叠区
紧凑模式的目标是保持可用，不是单纯把文字和控件一股脑缩小。
### 5. overflow 责任必须明确
页面中的大块内容必须明确：
- 谁负责占满剩余高度
- 谁负责裁剪
- 谁负责滚动
常见要求：
- 父容器链路需要 `min-height: 0`
- 工作区容器通常需要 `display: flex`
- 真正的滚动节点要显式 `overflow: auto`
### 6. 卡片不能被压到不可读
历史上我们反复踩到的问题不是“没有滚动条”，而是：
- 卡片被 `flex` 压缩得只剩一小条可视区域
- 文字能渲染，但读不完整
- 内容其实存在，却被 `overflow: hidden` 裁掉
因此后续约束是：
- 先保证卡片有可读的最小高度
- 如果继续压缩会影响阅读，就切换成内部滚动
- 不要为了“保持一屏”而把正文、表格、描述区压成无法阅读的条状区域
### 7. Tabs 不是天然安全的布局容器
历史上 Tabs 相关回归非常多，典型问题包括：
- 隐藏 tab pane 因为自定义 `display: flex` 而重新露出来
- 所有 tab 被强行套用同一套高度/overflow 规则
- 表格 tab 能工作，但 markdown / help / diagnostics tab 被压坏
因此约束是：
- `Tabs` 里的每类内容都要单独定义自己的布局策略
- 表格 tab 可以是“固定高度 + 内部滚动”
- 文档/Markdown tab 更适合“tab pane 自身滚动 + 内容正常文档流”
- 如果覆盖组件库样式，必须同时检查 hidden 状态是否仍然成立
### 8. 摘要区优先进入紧凑模式，而不是挤压正文
历史经验表明，最容易被误处理的是顶部摘要卡：
- 它们经常为了“都放下”被强行压窄
- 然后正文、表格、AI 结果区一起失去主空间
后续统一约束：
- 小屏或高缩放时，摘要卡优先：
- 降低 padding
- 改成横向滚动
- 改成更紧凑的网格
- 不要优先牺牲主工作区的可视面积
### 9. 长文档类内容优先保证阅读体验
像下面这些内容，不能直接套用“表格工作区”的逻辑：
- AI 简报
- 运行日志
- 原始 JSON
- 帮助说明
- 多段描述性文本
这些区域应该优先满足：
- 标题和元信息稳定可见
- 正文有明确的最小可读高度
- 正文滚动策略单独定义
- 支持 Markdown 表格、分隔线、引用、代码块等结构
### 10. 高度关键路径要少包一层
历史上不少滚动问题不是组件本身错，而是多包了一层之后：
- 高度链路断掉
- `min-height: 0` 没传下去
- `overflow` 责任被吃掉
因此：
- 对高度关键区域，优先使用最直接的 DOM 结构
- 使用 `Space`、额外包装 `div`、第三方布局容器时，要确认它们不会改变滚动和高度语义
- 如果一个区域已经出现“内容明明有，但只剩一条缝”，优先怀疑中间包装层
## 历史坑位总结
从 Earth、Playground、BGP、DataSources 这些页面的 bugfix 可以归纳出几类高频坑：
### 1. 用 `overflow: hidden` 掩盖布局问题
表面上看页面“整齐了”，实际上会导致：
- 内容被裁掉
- tab 内容只剩一条缝
- 面板明明渲染成功，但用户看不见
正确做法：
- 让真正的内容节点滚动
- 不要让上层容器无差别裁剪所有子内容
### 2. 把所有 tab 当成同一种内容
表格、Markdown、帮助卡、日志流的空间需求完全不同。
正确做法：
- 表格：固定工作区 + 内部滚动
- 文档：普通流式内容 + pane 级滚动
- 侧边说明：内容驱动高度，不强行拉满
### 3. 只做视觉缩小，不做空间重分配
这会导致：
- 卡片文字被截断
- 表格只剩 1 到 2 行
- 按钮和筛选区挤成一团
正确做法：
- 紧凑模式优先重排
- 横向滚动摘要区
- 折叠/收纳次级模块
### 4. 父容器高度链不完整
这是最常见的内部滚动失效原因。
检查顺序：
1. 外层是否真的有确定高度
2. flex 父容器是否带了 `min-height: 0`
3. 真正滚动节点是否明确 `overflow: auto`
4. 中间包装层是否偷偷改了布局语义
### 5. UI 状态和显示状态不同步
Earth 相关改动里反复出现：
- 图层隐藏了，但 hover/lock 还在
- tooltip 还在显示旧对象
- legend 没跟着切换
这类约束同样适用于后台页面：
- 被隐藏、卸载、切换出视图的内容，不应继续保留活跃交互状态
## 推荐实现模式
### 页面骨架
优先复用项目里已有的通用结构：
- `.dashboard-content-inner`
- `.page-shell`
- `.page-shell__header`
- `.page-shell__body`
- `.table-scroll-region`
不要每个页面都重新发明一套完全不同的高度和滚动语义。
### 表格工作区
推荐模式：
```tsx
<Card>
<div className="table-scroll-region" ref={tableRegionRef}>
<Table
pagination={false}
scroll={{ x: 1200, y: tableHeight }}
/>
</div>
</Card>
```
要求：
- 表格尽量在卡片内部滚动
- `scroll.y` 应来自实际可用高度估算，而不是完全静态的魔法数字
- 父容器链路要保证 header、body、content 的 overflow 都在表格内部闭合
### 多模块页面
如果一个页面同时有：
- 摘要卡
- 表格
- 异常明细
- 最近事件
不建议简单纵向堆叠全部模块。优先使用：
- 顶部摘要 + 底部单一主工作区
- 标签页切换多个次级数据视图
- 左右分栏，并保证每栏内部独立滚动
## 不推荐的做法
以下模式默认视为不符合本项目页面规范：
- 依赖整页纵向滚动来显示主要工作区
- 一个页面纵向堆 3 到 4 个大卡片，每个都想完整展示
- 表格没有内部滚动，导致缩放后只能看到 1 到 2 行数据
- 父容器缺少 `min-height: 0`，导致内部滚动失效
- 只做视觉缩小，不处理真正的空间分配
## 页面验收检查清单
提交前至少检查：
- 页头、摘要区、主工作区能否同时出现
- 主工作区是否拿到了页面中最多的高度
- 表格或明细溢出时，滚动条是否出现在模块内部
- 卡片是否被压缩到文字显示不完整；如果会，是否已经切换为内部滚动
- 浏览器缩放到 `125%` / `150%` 时是否仍可用
- 低高度窗口下是否还保有合理的可见内容行数
- Tabs、Card、Table 在 overflow 时是否仍可操作
- 非表格 tab（Markdown、帮助说明、日志）是否有独立且合理的滚动策略
## 落地顺序
后续新增或重构后台页时，优先按这个顺序设计：
1. 先定义主工作区
2. 再确定哪些模块必须常驻可见
3. 最后再做样式和视觉层次
简单说：
- 先保证空间分配正确
- 再处理滚动边界
- 最后再做美化

165
docs/hud-panel-component-plan.md
View File

@@ -1,165 +0,0 @@
# HUD Panel Component Plan
## Goal
Unify Earth HUD panels into a reusable component layer so new panels can share:
- a consistent shell
- a consistent header
- a consistent action-button system
- a consistent body and collapse pattern
## Scope
Target panels:
- `tv-panel`
- `news-panel`
- `legend`
- `layer-panel`
- `earth-stats`
- `info-card`
- settings modal header/actions
## Component Model
### Base shell
- `.hud-panel`
- `.hud-panel--compact`
- `.hud-panel--media`
- `.hud-panel--collapsed`
- `.hud-panel-hidden`
- `.hud-panel.is-dragging`
- `.hud-panel.is-layout-animating`
### Header
- `.hud-panel__header`
- `.hud-panel__title-group`
- `.hud-panel__title`
- `.hud-panel__subtitle`
- `.hud-panel__chip`
- `.hud-panel__actions`
Header baseline rule:
- Header title styling is fixed by the component layer and should not drift per panel
- Title font size, font weight, letter spacing, line height, text color, and vertical alignment come from the shared header tokens and structure
- Header divider, border treatment, inner spacing, and title-to-actions alignment are part of the same shared baseline
- Panel-specific header differences should be limited to explicit variants such as `compact` or `media`, or token overrides with documented intent
- “Looks close enough” local header overrides should be treated as temporary compatibility code and removed during migration
### Actions
- `.hud-panel__action`
- `.hud-panel__action--icon`
- `.hud-panel__action--collapse`
- `.hud-panel__action--close`
- `.hud-panel__action--refresh`
- `.hud-panel__action--external`
Action-button baseline rule:
- Header action buttons must have one fixed default style baseline across all HUD panels
- Default width behavior, padding, icon size, radius, alignment, hover, and active feedback all come from `.hud-panel__action`
- Panel-specific differences must be expressed through explicit variants or token overrides, not ad-hoc local button rewrites
- `close` buttons are part of the same default action system and must not silently fall back to a separate legacy box model
### Body
- `.hud-panel__body`
- `.hud-panel__body--scroll`
- `.hud-panel__body--collapsible`
### Collapse behavior
- `.hud-panel--collapsed`
- `.hud-panel--expand-up`
- `.hud-panel--expand-down`
Adaptive collapse / expand rule:
- HUD panels support two expansion directions:
- top-to-bottom expansion
- bottom-to-top expansion
- Expansion direction should be decided at runtime from available viewport space rather than hardcoded per panel
- Use:
- `d` = available distance from the header anchor to the viewport bottom edge
- `h` = expected expanded panel height
- buffer = `20px`
- Collapsed-state direction rule:
- if `d > h + 20px`, the next action direction is `expand-up`
- if `d <= h + 20px`, the next action direction is `expand-down`
- To avoid jitter around the threshold, the shared controller should keep a small hysteresis band:
- if the current direction is already `up`, keep it until `d <= h`
- if the current direction is already `down`, keep it until `d > h + 20px`
- The opposite edge is still a safety guard:
- if the chosen side cannot fit at all, fall back to the other side if it can fit
- if neither side fully fits, choose the side with more space and let the body scroll
- If neither direction fully fits, choose the direction with more available space and let the body scroll
- Collapse icon direction must match the active expansion direction so the icon always describes the real open/close motion
- The collapse icon describes the next action, not the current state
- This mapping is fixed component behavior and must not drift per panel:
- collapsed + expand-down => `expand_more`
- expanded + expand-down => `expand_less`
- collapsed + expand-up => `expand_less`
- expanded + expand-up => `expand_more`
- Panels must not combine icon-name swapping with extra CSS rotation for the same collapse control
- Expansion direction and icon direction must come from one shared source of truth in the component controller
- The direction decision should be recomputed when opening, resizing the viewport, or restoring a dragged panel near another edge
## Tokens
Promote panel differences into CSS variables instead of duplicating selectors:
- `--hud-panel-padding`
- `--hud-header-padding`
- `--hud-header-gap`
- `--hud-action-padding`
- `--hud-action-gap`
- `--hud-action-icon-size`
- `--hud-body-gap`
- `--hud-body-max-height`
- `--hud-chip-radius`
- `--hud-title-font-size`
- `--hud-title-font-weight`
- `--hud-title-letter-spacing`
- `--hud-title-line-height`
- `--hud-title-color`
- `--hud-header-border-color`
- `--hud-header-divider-opacity`
- `--hud-expand-direction`
## Migration Order
1. Build the shared component layer in `frontend/public/earth/css/hud.css`
2. Migrate `tv-panel` and `news-panel` first as the reference implementation
3. Migrate `legend` and `layer-panel` into a compact variant
4. Migrate `earth-stats` and `info-card`
5. Align settings modal header/actions with the same action system
6. Remove legacy one-off button selectors after verification
## Guardrails
- Do not change panel behavior and data flow during the first pass
- Keep old class names temporarily as compatibility hooks
- Prefer variable overrides over per-panel reimplementation
- Treat header action-button default styling as fixed component API, not per-panel design space
- Treat header title typography, border, and divider styling as fixed component API, not per-panel design space
- Treat collapse direction as a component behavior contract, not a one-off panel trick
- Treat collapse icon semantics as a component behavior contract, not a per-panel visual preference
- Verify header alignment and drag/collapse behavior after each migration batch
## First Implementation Batch
Batch 1 should only do:
- shared header structure
- shared action-button system
- shared title typography and header border/divider baseline
- shared collapsible body pattern
- adaptive collapse direction logic and direction-aware collapse icons
- migration of `tv-panel` and `news-panel`
That keeps risk low while giving the rest of the HUD a stable target to migrate toward.

97
docs/news-live-streams-collector-format.md
View File

@@ -1,97 +0,0 @@
# News Live Streams Collector Format
`news_live_streams` 采集器面向“频道目录 JSON”输入，而不是直接抓网页。
这样做的目标是：
- 让后台能够稳定接入世界各地新闻直播源
- 让 `Earth` 页面电视模块始终消费统一结构
- 便于后续接入类似 `worldmonitor` 那种 YouTube / HLS / iframe 混合频道目录
## 推荐 JSON 结构
```json
{
"sources": [
{
"id": "bbc-world-news",
"name": "BBC World News",
"provider": "BBC",
"region": "UK",
"language": "en",
"source_type": "youtube",
"youtube_video_id": "dQw4w9WgXcQ",
"youtube_channel": "https://www.youtube.com/@BBCNews",
"embed_url": "",
"stream_url": "",
"homepage_url": "https://www.youtube.com/@BBCNews/live",
"poster_url": "",
"sort_order": 220,
"is_enabled": true,
"notes": "Primary English global news channel"
},
{
"id": "france24-en",
"name": "France 24 English",
"provider": "France 24",
"region": "France",
"language": "en",
"source_type": "hls",
"stream_url": "https://example.com/live.m3u8",
"homepage_url": "https://www.france24.com/en/live",
"sort_order": 230,
"is_enabled": true
},
{
"id": "cctv4-page",
"name": "CCTV-4 中文国际",
"provider": "CCTV",
"region": "China",
"language": "zh-CN",
"source_type": "iframe",
"embed_url": "https://tv.cctv.com/live/cctv4/",
"homepage_url": "https://tv.cctv.com/live/cctv4/",
"sort_order": 10,
"is_enabled": true
}
]
}
```
## 字段约定
- `id`: 唯一标识，建议稳定不变
- `name`: 频道显示名
- `provider`: 提供方
- `region`: 国家或地区
- `language`: 语言代码
- `source_type`: `iframe` / `hls` / `video` / `external` / `youtube`
- `embed_url`: 适合 iframe 内嵌的页面
- `stream_url`: 直接视频流地址
- `homepage_url`: 官网或频道页
- `youtube_video_id`: YouTube 直播视频 ID
- `youtube_channel`: YouTube 频道 handle 或频道 URL
- `poster_url`: 封面图，可选
- `sort_order`: 排序值，越小越靠前
- `is_enabled`: 是否启用
- `notes`: 简短备注
## 面板行为约定
- `youtube`
- 优先使用 `youtube_video_id`
- 无法内嵌时至少保留 `youtube_channel` 或 `homepage_url` 供外部打开
- `hls` / `video`
- 优先走 `stream_url`
- `iframe`
- 优先走 `embed_url`
- `external`
- 不尝试内嵌，只保留外部打开
## 当前实现状态
- 后台设置页可以手工维护频道目录
- `Earth` 电视模块会合并：
- 手工配置源
- `news_live_streams` 采集器采集源
- 当前默认兜底源为 `CCTV-4 中文国际`

216
docs/prefix-geography-plan.md
View File

@@ -1,216 +0,0 @@
# Prefix Geography Plan
## Goal
Make Earth BGP incidents `prefix-centric` instead of `collector-centric`.
The map should primarily answer:
- where a prefix-related event is likely centered
- which regions the prefix is likely associated with
- which collectors observed the event as evidence
It should not continue to imply that the event is located at the collector itself unless no better geography is available.
## Why Current Geography Is Not Enough
Current incident geography can still collapse back to collector-derived regions because:
1. `prefix_scope` is currently built mostly from observed collector regions and historical observation regions.
2. `origin_asn_profile` currently comes from `peeringdb_network`, which is useful for ASN footprint hints but not sufficient as a primary prefix location source.
3. `collector centroid` is still a common fallback and therefore dominates sparse incidents.
This makes Earth feel like a collector map with event decorations instead of a prefix impact map.
## Data Source Layers
Prefix geography should be built from four layers, ordered by confidence.
### Layer 1. Prefix-to-country / prefix-to-region
This is the primary source layer and the current missing piece.
Recommended sources:
1. `IPtoASN / IPtoCountry`
- URL: <https://iptoasn.com/>
- Good fit for this project because it provides downloadable IPv4/IPv6 range-to-ASN and range-to-country mappings.
- Best use:
- map a prefix to country code
- enrich prefixes with coarse regional placement
2. `OpenGeoFeed`
- URL: <https://opengeofeed.org/faq/>
- Best use:
- override coarse country mappings when the prefix holder publishes a geofeed
- provide a more realistic deployment/service region than whois-style registration country
### Layer 2. Registry allocation fallback
Use these only as fallback signals, not as a ground-truth physical location.
Candidate inputs:
- RIR delegated stats
- `inetnum` / `inet6num` whois
Best use:
- detect registration country / allocation region
- provide fallback when no direct prefix geolocation dataset is available
### Layer 3. ASN footprint hints
Existing in this project:
- `peeringdb_network`
- `peeringdb_facility`
- `peeringdb_ixp`
Best use:
- derive ASN city/country footprint
- identify likely exchange/facility regions
- act as secondary evidence when prefix-specific geography is unavailable
### Layer 4. Observation evidence
Existing in this project:
- `RIPE RIS Live`
- `CAIDA BGPStream Backfill`
Best use:
- prove who observed the event
- derive affected observation regions
- support impact evidence
This should remain the final fallback and evidence layer, not the primary event geography.
## Recommended Geography Priority
The backend should compute incident geography with this order:
1. `prefix_geography`
- prefix-to-country / region / geofeed-backed result
2. `asn_region`
- ASN organization / facility / IXP footprint
3. `collector_centroid`
- observed collector regions only as final fallback
Returned GeoJSON should keep exposing the selected mode through:
- `geography_mode = prefix_geography | asn_region | collector_centroid`
## Proposed Backend Changes
### 1. Add a dedicated prefix geography dataset
New datasource candidates:
- `ip2asn_prefix_geo`
- optionally `opengeofeed_prefix_geo`
Suggested storage model:
- keep downloaded rows in `CollectedData` first for speed of integration
- later move to a dedicated table if lookup volume grows
Minimum normalized fields:
- `range_start`
- `range_end`
- `prefix`
- `country`
- `continent`
- `asn`
- `as_name`
- `source`
- `confidence`
### 2. Add prefix geography enrichment
Extend:
- `backend/app/services/bgp_enrichment.py`
New enrichment payload should include:
- `prefix_geography`
- `country`
- `continent`
- `regions`
- `source`
- `confidence`
This should be separate from the current `prefix_scope`.
Suggested distinction:
- `prefix_scope`
- observation-derived scope hint
- `prefix_geography`
- prefix-centric geography estimate
### 3. Update incident visualization geography selection
Extend:
- `backend/app/api/v1/visualization.py`
Selection order:
1. `prefix_geography.regions`
2. ASN geography hints from PeeringDB-derived profile
3. observation-derived `affected_regions`
### 4. Keep evidence visible in the frontend
Earth should distinguish:
- event center = prefix geography estimate
- evidence lines / collectors = observation proof
This keeps the event meaningful for non-expert users without losing collector evidence.
## Earth UX Result
After this change, a user should see:
- an incident marker near the estimated affected prefix region
- collectors as supporting evidence, not as the event center itself
- cables / landing points / nearby infrastructure as weak correlation around the estimated region
This makes BGP incidents readable as “where the event is likely happening or affecting”, instead of “which station saw it”.
## Implementation Order
### Phase 1
1. Add `IPtoASN / IPtoCountry` datasource support
2. Normalize rows into lookup-friendly format
3. Enrich BGP events with `prefix_geography`
4. Switch incident geography priority to prefer `prefix_geography`
### Phase 2
5. Add `OpenGeoFeed` support
6. Let geofeed override coarse country-level prefix geography
7. Add confidence scoring per geography source
### Phase 3
8. Add RIR / whois fallback
9. Add better ASN regional footprint from PeeringDB facilities / IXPs
10. Refine Earth visual semantics for prefix geography vs observation evidence
## Recommendation
The best next engineering move is:
1. integrate `IPtoASN / IPtoCountry`
2. model `prefix_geography` separately from `prefix_scope`
3. only then continue refining incident map placement
Without this layer, any further Earth tuning will still be constrained by collector-centric data.

309
docs/situational-awareness-foundation-plan.md
View File

@@ -1,309 +0,0 @@
# Situational Awareness Foundation Plan
## 定位
当前这套 AI 能力应被视为 `态势感知服务底座`，而不是完整的态势感知产品。
也就是说，现阶段的目标不是：
- 做一个“什么都能分析”的万能 AI 页面
- 让模型在证据不足时替代人工研判
- 过早把页面做成完整指挥大屏
现阶段真正要做的是：
- 先把 `model gateway / backend facade / evidence injection / page-specific brief` 这几层边界搭稳
- 让系统能够在已有证据上稳定地产出“可读、可回看、可扩展”的摘要
- 为后续更强的数据联动、agent 推理和 assessment 结构化输出预留好接口与数据模型
## 当前现实约束
### 1. 数据维度不足
目前系统能提供的主要证据仍集中在：
- BGP incidents / anomalies / events
- collector coverage
- datasource health / platform alerts
- prefix geography 的部分归属信息
当前明显还缺：
- 流量异常与业务指标
- 电商、支付、物流等业务侧指标
- 更丰富的资产、链路、区域、行业画像
- 外部舆情、公告、运营商状态、基础设施事件等背景信息
这意味着：
- 模型现在可以做“基于现有证据的摘要与归纳”
- 但还不能可靠地做“跨维度因果研判”
### 2. 维度之间联动还弱
目前不同模块之间更多是“并列展示”，还不是“强关联分析”：
- 系统告警和 BGP 事件还没有统一事件模型
- collector bias 与真实区域热度还没有完全剥离
- datasource health 与 BGP 风险、业务影响之间还没有稳定映射
这意味着：
- 当前更适合做 `brief / overview / operator notes`
- 还不适合过度承诺“自动态势判断”
### 3. 结构化 assessment 还未成为主输出
虽然已经有 BGP brief、系统告警 brief、态势告警 brief，但目前主输出仍偏向：
- 文本摘要
- facts/context 附带证据
后续真正要服务态势感知，需要更稳定的结构化输出，例如：
- summary
- key risks
- evidence
- confidence
- recommendations
- missing data
## 当前基座已经具备的能力
### 1. AI 调用边界已经明确
- `aiprovider` 负责模型协议与 provider 兼容
- `backend` 负责业务 API、证据整合和鉴权
- `frontend` 负责页面入口与结果展示
### 2. 页面级 AI 入口已经开始成型
当前已经有或正在收口的入口：
- `Playground`
- 用于链路验证与 provider 诊断
- `BGP AI 简报`
- 用于 BGP 事实摘要和区域风险归纳
- `Alerts`
- 用于系统告警、BGP 告警、态势告警三类入口
### 3. 证据优先的方向已经建立
已经不再只依赖人工在 Playground 中手填 prompt，系统开始具备：
- 从真实业务数据生成事实输入
- 保存 facts/context 快照
- 回看 AI 输出时同时回看证据
这一步非常关键，因为它决定后面能否从“玩具 demo”走向“有运维价值的系统”。
## 近期收尾建议
这些事情都属于“底座收口”，值得做，但不应该再继续重产品包装。
### 1. 统一 Alerts 页面
已采用：
- 一个 `Alerts` 页面
- 三个 tab：
- `系统告警`
- `BGP 告警`
- `态势告警`
收尾重点：
- 保持 tab 的文案、摘要卡和 AI 简报交互一致
- 不额外扩展成多个独立二级页面
### 2. 保持 Playground 为测试台
原则：
- Playground 只承担链路验证、provider 状态诊断、请求结果观察
- 不继续堆“万能业务分析器”式交互
### 3. 把 brief 能力当服务能力而不是页面特效
页面现在能看到按钮和结果，这很好，但更重要的是：
- 后端接口稳定
- facts/context 可追踪
- 输出结构后续可升级
### 4. 导航结构先收口，不继续平铺一级菜单
随着后续能力扩展，系统很可能继续新增：
- 海缆
- 算力中心
- 战争信息
- 电商分析
- 其他专题观测页
如果继续把这些入口全部平铺在左侧一级菜单中，会带来两个问题：
- 一级菜单过长，用户难以判断先进入哪个上下文
- `观测页 / 告警页 / 研判页 / 运维页` 的职责边界会被混在一起
因此近期应明确采用分组导航，而不是继续扩展平铺菜单。
推荐的导航分组如下：
- `总览`
- 仪表盘
- Earth
- `专题观测`
- BGP 观测
- 采集数据
- 后续可扩展：海缆、算力中心、战争信息、电商分析
- `告警与研判`
- Alerts
- `运维与配置`
- 数据源
- AI Playground
- 用户管理
- 系统配置
这套结构的含义是：
- `专题观测` 页面负责看某个维度本身
- `Alerts` 负责跨模块风险与值班工作台
- `Playground` 保持为测试台，不挤占业务导航语义
短期收尾时，应优先重组现有入口，而不是继续增加新的一级菜单。
## 后续路线
## Phase 1：服务底座稳固
目标：
- 不追求“更炫的 AI 页面”
- 先把当前接口、证据、存储和页面入口收稳
工作项：
- 统一页面级 AI 入口模式
- 统一 brief response schema
- 保证 facts/context 在前后端都可回看
- 继续清理 mock 和临时分支逻辑
完成标准：
- 每个 AI 入口都是真实链路
- 每个 AI 结果都能追溯到证据输入
## Phase 2：Evidence-first Assessment
目标：
- 从“文本摘要”升级成“结构化 assessment”
工作项：
- 为 brief/assessment 定义统一 schema
- 固化：
- summary
- key_risks
- evidence
- confidence
- recommendations
- missing_data
- 页面以结构化区块展示，而不只是大段文本
完成标准：
- AI 输出可持久化、可比较、可审计
## Phase 3：多维证据接入
目标：
- 让“态势感知”真正拥有更多维度，而不是只靠 BGP 与系统告警
优先接入方向：
- datasource health findings
- 流量或业务指标
- 区域/资产/链路映射
- 外部事件与公告
- 业务垂直数据，例如电商分析相关指标
完成标准：
- AI 能基于多个维度做交叉说明
- 不再只围绕单一模块自说自话
## Phase 4：Correlation Layer
目标：
- 不同来源的信号不再只是并列，而是形成统一的事件关联
工作项：
- 统一 signal/finding 模型
- 跨模块事件聚合
- 证据来源权重
- collector bias 与真实热度分离
完成标准：
- 系统能回答“这些异常是不是同一件事”
- 系统能回答“哪些结论只是观测偏差”
## Phase 5：Agent-assisted Situational Awareness
目标：
- 在证据足够的前提下，再让 agent 负责更复杂的推理与建议
工作项：
- 复用现有 agent runtime 规划
- 引入 web search / docs fetch / repair proposal 等能力
- 但始终坚持：
- evidence first
- proposal before action
- no silent mutation of defaults
完成标准：
- agent 成为证据驱动的分析层
- 而不是一个“万能猜测层”
## 设计原则
### 1. 先底座，后产品化
先把服务链路和证据模型做好，再做更大的页面表达。
### 2. 先证据，后判断
事实输入应先稳定，再让模型做归纳。
### 3. 先专用 brief，后统一态势层
先让各业务页有各自可信的 AI 入口，再考虑统一态势页。
### 4. 先 proposal，后自动动作
涉及修复、覆盖、写配置、调任务的动作，都应经过 proposal 和审计。
## 当前建议结论
对现在这个项目，最合理的定位是：
- `Playground` 是测试台
- `BGP / Alerts` 是第一批业务 AI 入口
- `aiprovider + backend AI facade + evidence snapshots` 是核心服务底座
现阶段不需要追求“已经具备完整态势感知能力”。
现阶段真正的成功标准是：
- 这套底座可用
- 可回看
- 可扩展
- 不自欺欺人

347
docs/system-service-control.md
View File

@@ -1,347 +0,0 @@
# System Service Control
This document defines the fixed mapping between admin control-plane actions and
the existing `planet.sh` service-management commands.
The goal is to reuse the current operational script semantics without exposing
arbitrary shell execution to the frontend or API callers.
## Scope
- This mapping is for admin-side operational controls only.
- The control plane must submit a fixed action name, not a raw shell command.
- The backend is responsible for translating an allowed action into a fixed
`planet.sh` invocation.
## Design Rules
- Only whitelist actions may be executed.
- The frontend must never send arbitrary shell strings.
- The backend must build command arguments from a fixed mapping table.
- High-risk actions should be restricted to `super_admin`.
- Prefer partial restarts over full-stack restarts when UI continuity matters.
## Action Mapping
| Action name | Intended use | `planet.sh` command | Notes |
| --- | --- | --- | --- |
| `restart-backend` | Restart backend API only | `./planet.sh restart -b` | Recommended first implementation for UI-triggered restart flows. |
| `restart-database` | Restart PostgreSQL and Redis containers | `./planet.sh restart -d` | Useful when database/cache services need a controlled bounce without restarting the UI. |
| `restart-system` | Restart the whole application stack | `./planet.sh restart` | Frontend continuity breaks briefly; UI should switch to guided recovery mode. |
| `restart-frontend` | Restart frontend dev server only | `./planet.sh restart -f` | Use with caution; UI continuity is weaker than backend-only restart. |
| `restart-backend-port` | Restart backend on a specific port | `./planet.sh restart -b <port>` | Port must be backend-validated before execution. |
| `restart-frontend-port` | Restart frontend on a specific port | `./planet.sh restart -f <port>` | Port must be backend-validated before execution. |
| `health-check` | Read current service health | `./planet.sh health` | Safe read-only operational action. |
| `show-logs-backend` | Inspect backend logs | `./planet.sh log -b` | Best used for CLI/operator tooling, not normal Web UI streaming. |
| `show-logs-frontend` | Inspect frontend logs | `./planet.sh log -f` | Best used for CLI/operator tooling, not normal Web UI streaming. |
## Not Exposed In UI By Default
The following existing script capabilities should not be exposed directly in the
Web UI unless there is an explicit product need and an additional safety review:
- `./planet.sh restart`
- `./planet.sh start`
- `./planet.sh stop`
- `./planet.sh createuser`
- any future raw shell passthrough
Reason:
- full restart can break the current control session;
- stop/start have larger blast radius;
- user creation is not a service-control operation;
- raw shell passthrough creates unnecessary privilege risk.
## Recommended First-Phase UI Contract
### Frontend action payload
```json
{
"action": "restart-backend"
}
```
### Backend command resolution
```text
restart-backend -> ["./planet.sh", "restart", "-b"]
restart-database -> ["./planet.sh", "restart", "-d"]
restart-system -> ["./planet.sh", "restart"]
restart-frontend -> ["./planet.sh", "restart", "-f"]
health-check -> ["./planet.sh", "health"]
```
## API Draft
### Primary Endpoint
- `POST /api/v1/system/restart-tasks`
Purpose:
- create a controlled restart task;
- resolve a whitelist action into a fixed `planet.sh` command;
- hand execution off to an external runner or detached subprocess.
### Request Body
```json
{
"action": "restart-backend"
}
```
Optional future shape:
```json
{
"action": "restart-backend-port",
"port": 8000
}
```
### Response
```json
{
"task_id": "restart_20260331_153000_ab12cd",
"action": "restart-backend",
"status": "queued",
"stage": "accepted",
"message": "Restart task accepted"
}
```
### Task Query Endpoint
- `GET /api/v1/system/restart-tasks/{task_id}`
Response shape:
```json
{
"task_id": "restart_20260331_153000_ab12cd",
"action": "restart-backend",
"status": "queued",
"stage": "accepted",
"message": "Waiting for execution",
"requested_by": {
"id": 1,
"username": "admin"
},
"created_at": "2026-03-31T15:30:00+08:00",
"updated_at": "2026-03-31T15:30:02+08:00"
}
```
### Optional Log Endpoint
- `GET /api/v1/system/restart-tasks/{task_id}/logs`
Suggested response:
```json
{
"task_id": "restart_20260331_153000_ab12cd",
"lines": [
"accepted restart-backend request",
"spawning restart command",
"waiting for backend shutdown",
"waiting for backend health recovery"
]
}
```
This log endpoint is optional for phase one. The first version can work with
task state plus `/health` polling alone.
## Task State Model
### Status
- `queued`
- `running`
- `succeeded`
- `failed`
- `timeout`
### Stage
- `accepted`
- `spawning`
- `stopping`
- `starting`
- `waiting_for_health`
- `healthy`
- `failed`
### Interpretation
- `status` is the high-level terminal or non-terminal state.
- `stage` is the operator-facing execution phase for the UI.
- `message` is the short human-readable line shown in the modal or full-screen
overlay.
## Permission Model
- `restart-backend` should require `super_admin`.
- Permission checks should follow the same role pattern already used in
[users.py](/home/ray/dev/linkong/planet/backend/app/api/v1/users.py).
- Frontend visibility may hide controls for non-`super_admin`, but backend must
still enforce authorization.
## Storage Model
Recommended first implementation:
- store restart task state in Redis;
- keep task lifetime short;
- keep recent logs as a bounded list.
Suggested keys:
- `system:restart_task:{task_id}`
- `system:restart_task:{task_id}:logs`
Suggested stored fields:
- `task_id`
- `action`
- `status`
- `stage`
- `message`
- `requested_by_id`
- `requested_by_username`
- `created_at`
- `updated_at`
## Execution Model
The request-handling API process should not depend on itself surviving long
enough to stream the whole restart output.
Recommended execution flow:
1. validate caller and action
2. create task state in Redis
3. resolve action to fixed `planet.sh` argv
4. spawn detached executor
5. return `task_id`
6. executor updates task state while restart is in progress
7. frontend polls health and/or task state until recovery
Recommended command resolution examples:
```text
restart-backend -> ["./planet.sh", "restart", "-b"]
restart-frontend -> ["./planet.sh", "restart", "-f"]
restart-backend-port -> ["./planet.sh", "restart", "-b", "<port>"]
health-check -> ["./planet.sh", "health"]
```
## Frontend Polling Flow
Recommended first-phase UX:
1. user clicks `重启后端`
2. confirmation modal explains temporary unavailability
3. frontend calls `POST /api/v1/system/restart-tasks`
4. UI enters blocking restart state
5. frontend polls `/health` every `1-2s`
6. temporary request failures are treated as expected
7. after `2-3` consecutive successful health checks, frontend reloads page
Optional richer polling:
1. poll task status endpoint while backend is still reachable
2. switch to `/health` recovery polling after disconnect begins
3. refresh page after health recovery
## Frontend State Machine
- `idle`
- `confirming`
- `submitting`
- `waiting_for_shutdown`
- `waiting_for_recovery`
- `recovered`
- `failed`
- `timeout`
Suggested UI messages:
- `已发送重启指令`
- `正在停止后端服务`
- `正在等待服务恢复`
- `服务已恢复，正在刷新页面`
- `恢复超时，请手动检查服务状态`
## Phase-One Recommendation
Implement only the following in phase one:
- `restart-backend`
- `super_admin` permission gate
- task creation endpoint
- Redis-backed task state
- frontend confirmation modal
- frontend `/health` polling
- automatic page reload after recovery
Do not implement in phase one:
- full `./planet.sh restart`
- raw shell command passthrough
- arbitrary service control
- full terminal stdout streaming
- multi-action concurrent restart queueing
## Implementation Checklist
### Backend
1. add a dedicated system-control API module under `backend/app/api/v1/`
2. add a whitelist-based action resolver for `planet.sh`
3. store restart task state in Redis
4. add detached restart-runner script execution
5. expose:
- `POST /api/v1/system/restart-tasks`
- `GET /api/v1/system/restart-tasks/{task_id}`
- optional task log endpoint
6. enforce `super_admin` permission on all restart-task endpoints
### Frontend
1. add a `重启后端` control on the dashboard for `super_admin`
2. show a confirmation modal before dispatch
3. after submission, switch modal into blocking restart state
4. poll `/health` until backend recovery is confirmed
5. auto-refresh page after consecutive successful health checks
6. show short stage-oriented logs instead of raw terminal streaming
### Operational Notes
1. phase one should target backend-only restart
2. frontend restart should remain out of scope initially
3. command execution must always originate from repository root
4. only fixed action names may cross the API boundary
## Validation Requirements
- Reject any action not present in the whitelist.
- If a port-bearing action is added, validate the port as an integer in
`1..65535`.
- Resolve commands from the repository root so `planet.sh` runs with a stable
working directory.
- Record the requested action, operator identity, execution start time, and
result.
## Implementation Guidance
- For UI-triggered restart flows, prefer `restart-backend` first.
- Do not rely on the current API request process to stream full restart output
after it triggers its own restart.
- Use a task record plus polling/health-check recovery flow instead of raw
terminal streaming as the primary UX.

48
docs/system-settings-plan.md
View File

@@ -1,48 +0,0 @@
# 系统配置中心开发计划
## 目标
将当前仅保存于内存中的“系统配置”页面升级为真正可用的配置中心，优先服务以下两类能力：
1. 系统级配置持久化
2. 采集调度配置管理
## 第一阶段范围
### 1. 系统配置持久化
- 新增 `system_settings` 表，用于保存分类配置
- 将系统、通知、安全配置从进程内存迁移到数据库
- 提供统一读取接口，页面刷新和服务重启后保持不丢失
### 2. 采集调度配置接入真实数据源
- 统一内置采集器默认定义
- 启动时自动初始化 `data_sources` 表
- 配置页允许修改：
- 是否启用
- 采集频率（分钟）
- 优先级
- 修改后实时同步到调度器
### 3. 前端配置页重构
- 将当前通用模板页调整为项目专用配置中心
- 增加“采集调度”Tab
- 保留“系统显示 / 通知 / 安全”三类配置
- 将设置页正式接入主路由
## 非本阶段内容
- 邮件发送能力本身
- 配置审计历史
- 敏感凭证加密管理
- 多租户或按角色细粒度配置
## 验收标准
- 设置项修改后重启服务仍然存在
- 配置页可以查看并修改所有内置采集器的启停与采集频率
- 调整采集频率后，调度器任务随之更新
- `/settings` 页面可从主导航进入并正常工作

981
docs/ue5_mvp_fused_plan.md
View File

@@ -1,981 +0,0 @@
# 智能星球 UE5 客户端一期实施方案（融合版）
> 版本：v2.0
> 日期：2026-04-14
> 目标：把现有 Web Earth 项目，平滑推进到 **UE5 可用 MVP 客户端**
> 适用对象：**UE 零基础新手**
> 输出结果：一份 **能直接照着做** 的实施手册
> 策略：**保留原 MVP 方案里适合入门的部分，吸收更稳的工程做法，降低你第一次做 UE 时踩坑概率**
---
## 一、这份融合版方案解决什么问题
你原来的 MVP 方案是靠谱的，优点很明显：
- 范围克制
- 适合新手入门
- 目标明确
- 能较快做出“看得见、点得到”的成果
但它也有几个风险：
- 默认 `localhost` 一定通，这在 WSL2 + Windows + Docker 环境里不一定成立
- 默认 UE 蓝图里直接做 HTTP + JSON 解析会很顺，这一步其实很容易卡
- 默认“一上来就接真实后端”，新手会同时踩 UE、Cesium、网络、JSON、蓝图五个坑
- 时间估计略乐观
所以这份融合版方案的核心思路是：
## 核心原则
**先做“本地数据可交互地球”，再做“真实后端对接”。**
也就是把一期再拆成两个更稳的里程碑：
### 里程碑 A：本地演示版
先不接后端，只做：
- UE5 项目能打开
- Cesium 地球能显示
- 本地 JSON 里的点能正确落到地球
- 点击点能弹信息卡
- HUD 能正常显示假状态
### 里程碑 B：后端接入版
在 A 的基础上再做：
- HTTP 拉取真实后端数据
- 显示真实 TOP500 数据
- 显示后端在线状态
- 为后续扩展海缆/BGP/卫星打基础
这样做的好处是：
- 把问题拆开
- 更容易调试
- 更适合 UE 新手
- 不会因为后端联调没通就把整个 UE 开发节奏打断
---
# 二、一期目标：做什么，不做什么
## 这次一期一定要做的
做一个 **可用的 UE5 客户端 MVP**，达到以下 6 项：
1. 能打开 UE 项目并看到 3D 地球
2. 能在地球上显示超算数据点
3. 能点击数据点弹出信息卡
4. 能显示一个基础 HUD
5. 能通过 HTTP 接入后端数据
6. 能打包成 Windows 可执行程序
---
## 这次一期先不做的
这些全部放到后续阶段：
- 海缆路径渲染
- 卫星轨迹与卫星图层
- BGP 图层
- WebSocket 实时更新
- 粒子特效大升级
- 自动巡航
- 多屏/3D 偏振/大屏联动
一句话：
**一期不是“把 Web Earth 全搬到 UE”，而是“证明 UE 客户端链路能跑通”。**
---
# 三、UE 专有名词字典（零基础版）
这部分你最好先读一遍。后面所有步骤都围绕这些词。
## 1. Actor
**Actor = 场景里的一个对象**
你可以把它理解成：
- 一个地球控制器
- 一个超算点
- 一台相机
- 一条海缆
这些在 UE 里都可以是 Actor。
---
## 2. Component
**Component = 挂在 Actor 身上的功能零件**
比如一个超算点 Actor，可能有：
- 一个球形外观
- 一个碰撞盒
- 一个标签
- 一个发光效果
这些零件就是 Component。
一句话：
**Actor 是整台机器，Component 是机器上的零件。**
---
## 3. Blueprint（蓝图）
**Blueprint = UE 的可视化编程系统**
你不用先写代码，而是把很多“逻辑节点”拖出来，用线连接起来。
你可以把它理解成：
- 前端里的函数 + 事件监听
- 只不过不是写文本代码，而是连线
---
## 4. Level / Map（关卡）
**Level = 一个场景文件**
你可以把它理解成 Three.js 的一个 Scene。
本期只需要一个主场景：
- `Main`
---
## 5. Widget / UMG
**Widget = UI 组件**
**UMG = UE 的 UI 编辑系统**
比如：
- 信息卡
- 状态栏
- 右上角连接状态
- 图例
- HUD 面板
这些都用 Widget 做。
---
## 6. Material（材质）
**Material = 决定物体外观的系统**
比如：
- 球体是什么颜色
- 是否发光
- 是否透明
- 是否随性能大小变亮
这些都由材质控制。
---
## 7. Static Mesh
**Static Mesh = 不会变形的 3D 模型**
比如：
- 球
- 立方体
- 平面
- 某个固定模型
超算点一期里可以先直接用球体 Static Mesh。
---
## 8. Pawn
**Pawn = 玩家控制的对象**
一期里你可以把它理解成：
- 带相机的飞行控制器
---
## 9. PlayerController
**PlayerController = 处理输入的对象**
比如：
- 鼠标点击
- 拖拽
- 滚轮缩放
这些都由 PlayerController 或其相关逻辑来处理。
---
## 10. GameMode
**GameMode = 游戏/场景的主规则配置入口**
它决定：
- 默认用哪个 Pawn
- 默认用哪个 PlayerController
你可以把它理解成“主入口配置”。
---
## 11. Viewport
**Viewport = 你看 3D 场景的窗口**
就是 UE 编辑器中间那块 3D 视图。
---
## 12. Outliner
**Outliner = 当前场景对象列表**
你可以把它理解成：
- Scene 树
- DOM 树
- 资源树
---
## 13. Details Panel
**Details Panel = 选中对象后的属性面板**
相当于“右侧属性编辑器”。
---
## 14. Cesium for Unreal
**Cesium for Unreal = UE 里的地球插件**
它负责：
- 真实地球
- 卫星影像
- 地形
- 经纬度坐标和 UE 世界坐标的转换
如果没有它，你得自己处理地球和坐标系统，会非常难。
---
## 15. Struct（结构体）
**Struct = 数据结构定义**
你可以把它理解成 TypeScript 里的 `interface`。
比如：
```ts
interface ComputePoint {
id: string
name: string
latitude: number
longitude: number
performance: number
}
```
在 UE 里这类东西叫 Struct。
---
## 16. Event Dispatcher
**Event Dispatcher = 事件分发器**
你可以把它理解成：
- EventEmitter
- 发布订阅
比如：
“数据加载完毕”这个事件，就可以分发给其他蓝图。
---
## 17. Spline
**Spline = 一条平滑曲线**
后面做海缆、轨迹时非常有用。
一期可以先知道这个词，不一定马上用。
---
## 18. Niagara
**Niagara = UE 粒子特效系统**
比如：
- 流光
- 光晕
- 拖尾
- 火花
一期先不重点碰它。
---
# 四、你的真实开发策略：两阶段起步
这是这份融合版和原方案最大的区别。
---
## 阶段 A：本地演示版（先脱离后端）
### 目标
先把下面这些完全打通：
- UE 项目启动正常
- Cesium 地球正常
- 相机可操作
- 本地 JSON 文件能生成地球标记点
- 点击点能弹信息卡
- HUD 能显示假数据
### 为什么一定要先做这个
因为如果你一上来就接真实后端，你会同时碰到：
- WSL2 到 Windows 网络
- Docker 端口映射
- UE HTTP 请求
- 蓝图 JSON 解析
- Cesium 坐标转换
- 标记点生成
新手很容易直接乱掉。
---
## 阶段 B：后端接入版（再联调）
### 目标
在 A 的基础上，加上：
- HTTP 拉真实后端数据
- 显示真实 TOP500 点
- 右上角显示后端在线状态
- 为后续做更多图层留下数据接入层
---
# 五、环境准备
## 1. 你要安装的软件
### Epic Games Launcher
用来下载和启动 UE。
### Unreal Engine 5.4
建议直接用 5.4 稳定版。
### Visual Studio 2022
虽然一期主要用 Blueprint，但 UE 的很多项目依赖 VS 环境。
安装组件：
- Desktop development with C++
- Game development with C++
### Git
用来管理文档和后续工程。
### Cesium for Unreal
用来做地球。
---
## 2. 你的环境约束
你现在是：
- 后端可能跑在 WSL2 / Docker
- UE 必须跑在 Windows
所以你的真实运行方式通常会是：
- **Windows** 运行 UE5
- **WSL2** 运行后端
- 两者通过 HTTP 通信
这里最关键的一条是：
**不要默认 `localhost` 一定能通，必须先在 Windows 浏览器里验证。**
---
# 六、推荐的项目结构
## UE 项目目录内的 Content 结构
```text
Content/
Blueprints/
Data/
Widgets/
Materials/
Levels/
FX/
Textures/
```
建议说明：
- `Blueprints/` 放逻辑蓝图
- `Data/` 放本地 JSON、DataTable、Struct
- `Widgets/` 放 UI
- `Materials/` 放材质
- `Levels/` 放场景
- `FX/` 放特效
- `Textures/` 放贴图
---
# 七、一期最小蓝图清单
一期只需要这几个核心蓝图。
## 1. `BP_GlobeCamera`
作用：相机控制器
负责：
- 鼠标拖拽旋转
- 滚轮缩放
- 初始视角控制
---
## 2. `BP_PlanetGameMode`
作用：指定默认的 Pawn 等
---
## 3. `BP_DataLoader`
作用：负责读数据
一期建议支持两种来源：
- 本地 JSON
- HTTP 接口
这样调试更稳。
---
## 4. `BP_ComputePoint`
作用：一个超算点的显示对象
负责：
- 接收一条数据
- 放到正确经纬度位置
- 显示外观
- 处理点击
---
## 5. `WBP_InfoCard`
作用：点开后显示详情
显示：
- 名称
- 国家
- 算力
- 可选显示更多字段
---
## 6. `WBP_StatusBar`
作用：右上角状态栏
显示：
- 后端在线/离线
- 当前加载条数
- 当前模式（本地数据 / 真实后端）
---
# 八、数据层设计
一期不要一开始就完全照搬后端返回结构。
你要先定义一个 UE 友好的结构。
## `S_ComputePoint`
字段建议：
- `PointId`：字符串，唯一 ID
- `Name`：字符串
- `Latitude`：浮点
- `Longitude`：浮点
- `Performance`：浮点
- `CoreCount`：整数
- `Country`：字符串
- `Source`：字符串
这个结构同时适用于：
- 本地 JSON
- 后端 API 返回结果转换后的对象
---
# 九、最稳的执行路线
下面是整个实施计划最重要的部分。
---
# Phase 0：安装和验证环境
## 目标
确保你能：
- 安装 UE5.4
- 启用 Cesium
- 能打开一个空项目
- 能在 Windows 浏览器访问你的后端
## 验收
满足以下 4 条：
- UE 能打开
- Cesium 能启用
- 项目能创建
- Windows 浏览器能访问后端 summary 接口
如果第 4 条做不到，不要继续推进真实接口联调。
---
# Phase 1：创建项目并把地球显示出来
## 目标
打开项目后，能看到一个真实地球。
## 操作顺序
1. 新建 UE5 Blank Blueprint 项目
2. 创建 `Main` 场景
3. 启用 Cesium
4. 添加：
- `Cesium World Terrain`
- `Cesium Sun Sky`
- `CesiumGeoreference`
5. 调整视角，让你能看到整个地球
## 验收
能录一段短视频，里面能看到地球和镜头移动。
---
# Phase 2：做相机控制
## 目标
让地球可以：
- 鼠标拖拽旋转
- 滚轮缩放
## 说明
这里可以沿用原 MVP 方案的思路：
- `BP_GlobeCamera` 作为 Pawn
- Spring Arm + Camera 组成相机结构
- 用输入控制旋转和缩放
## 注意
这一版相机只是“一期可用版”，不是最终镜头系统。
## 验收
按 Play 后：
- 地球可旋转
- 可缩放
- 不会直接飞走或抖动失控
---
# Phase 3：先喂本地 JSON 数据
这是融合版方案里最关键的改动。
## 目标
不接后端，先验证：
- 数据结构正常
- JSON 能读
- 点能生成
- 点击交互正常
## 为什么先这么做
因为这样可以把问题收缩成 3 件事：
- Cesium 坐标转换
- 点渲染
- UI 弹窗
不牵涉后端联调。
## 本地 JSON 示例格式
建议放在 `Content/Data/compute_points.json`
```json
[
{
"PointId": "top500_1",
"Name": "Frontier",
"Latitude": 35.93,
"Longitude": -84.31,
"Performance": 1194.0,
"CoreCount": 8730624,
"Country": "US",
"Source": "top500"
},
{
"PointId": "top500_2",
"Name": "Fugaku",
"Latitude": 34.69,
"Longitude": 135.19,
"Performance": 442.0,
"CoreCount": 7630848,
"Country": "JP",
"Source": "top500"
}
]
```
## 推荐做法
先做一个“本地模式”开关。
在 `BP_DataLoader` 里支持：
- Mode = LocalJson
- Mode = HttpApi
先永远跑 `LocalJson`。
## 验收
你应该能看到：
- 多个点出现在地球上
- 大致位置正确
- 点击能弹信息卡
---
# Phase 4：做超算点蓝图
## 目标
完成 `BP_ComputePoint`
每个点要实现：
- 接收一条 `S_ComputePoint`
- 经度纬度转成 UE 世界坐标
- 在地球上显示为一个可见的发光球
- 支持被点击
## 显示建议
### 外观
先用最简单的球体 Static Mesh。
### 材质
做一个发光材质：
- 红橙色
- 自发光
- 不追求复杂效果
### 大小
球体要足够大，确保在地球尺度下看得见。
### 高度
不要贴地表太近，建议悬浮在地表上方一个固定高度。
## 验收
同一批数据点在地球上的位置大体合理。
---
# Phase 5：做信息卡
## 目标
点击一个点后，弹出一个简单的信息卡。
## `WBP_InfoCard` 要显示的内容
建议只显示最关键的 3 个字段：
- 名称
- 国家
- 算力
一期先不要堆太多字段。
## 验收
点击点 → 卡片出现
点击关闭 → 卡片消失
---
# Phase 6：做基础 HUD
## 目标
屏幕上始终有一个简单状态栏。
## `WBP_StatusBar` 显示内容建议
- 当前模式：Local / HTTP
- 已加载数据点数量
- 后端状态：Unknown / Online / Offline
在本地模式阶段，状态可以先写死或显示 `Local Demo`。
## 验收
不点击任何点时，屏幕右上角也有“系统正在工作”的感觉。
---
# Phase 7：再接真实后端
这是第二阶段开始。
## 目标
把数据源从本地 JSON 切到 HTTP。
## 正确做法
不要把 `BP_DataLoader` 重写。
而是让它支持：
- LocalJsonLoader
- HttpLoader
也就是：
**显示层不变，只替换数据来源。**
## 最重要的接口原则
如果后端已有接口字段非常杂，不一定要 UE 直接吃。
可以加一个“更适合 UE 的轻量接口”。
例如：
`/api/v1/ue/bootstrap/top500`
返回尽量扁平的数据：
```json
[
{
"PointId": "top500_1",
"Name": "Frontier",
"Latitude": 35.93,
"Longitude": -84.31,
"Performance": 1194.0,
"CoreCount": 8730624,
"Country": "US",
"Source": "top500"
}
]
```
## 为什么推荐 UE 轻量接口
因为 UE 不适合像前端 React 那样，层层解包一大堆复杂 JSON。
---
# Phase 8：做连接状态检测
## 目标
让 HUD 能显示：
- 在线
- 离线
- 本地模式
## 正确实现思路
建议用一个很小的状态请求，比如：
- summary 接口
- health 接口
- 或 UE 专用 ping 接口
不要让状态检测去依赖一个超大的数据接口。
## 验收
后端关掉时，状态栏能明显变成 Offline。
---
# Phase 9：打包发布
## 目标
把项目打包成 Windows 可执行程序。
## 注意
打包是一期必须尝试的，但不要让它阻塞前面所有开发。
也就是说：
- 编辑器里没稳定跑通前，不要反复纠结打包
- 等 LocalJson 版和 HTTP 版都能在编辑器 Play 模式稳定运行后，再打包
## 验收
双击 exe 可以运行，进入地球场景并正常展示数据。
---
# 十、建议的 14 天执行计划
这版比原 MVP 的时间估计更保守，也更适合新手。
## 第 1 天
- 安装 UE5.4
- 安装 Cesium
- 创建空项目
- 创建 Main 场景
## 第 2 天
- 启用 Cesium
- 把地球跑起来
- 保存项目结构
## 第 3 天
- 做 `BP_GlobeCamera`
- 跑通旋转和缩放
## 第 4 天
- 建 `S_ComputePoint`
- 准备本地 JSON 文件
- 做 `BP_DataLoader` 的本地模式
## 第 5 天
- 做 `BP_ComputePoint`
- 本地 JSON 批量生成点
## 第 6 天
- 调整点大小、颜色、高度
- 检查经纬度位置是否大致正确
## 第 7 天
- 做 `WBP_InfoCard`
- 跑通点击点弹卡片
## 第 8 天
- 做 `WBP_StatusBar`
- 显示本地模式状态和点数量
## 第 9 天
- Windows 浏览器验证后端接口
- 准备 HTTP 版加载逻辑
## 第 10 天
- 实现 HTTP 拉真实数据
- 先在日志里确认数据到了
## 第 11 天
- 把 HTTP 数据接到点渲染
- 切换 Local / HTTP 两种模式
## 第 12 天
- 做连接状态 Online / Offline
- 补错误提示
## 第 13 天
- 测试完整链路
- 修点选、缩放、HUD 细节
## 第 14 天
- 进行第一次打包
- 在 Windows 下运行 exe 验证
---
# 十一、这份方案和原 MVP 方案怎么融合
下面是合并关系。
## 保留原 MVP 方案的部分
这些内容很好，建议继续用：
- 术语表
- Phase 结构化写法
- `BP_GlobeCamera`
- `BP_ComputePoint`
- `WBP_InfoCard`
- `WBP_StatusBar`
- 相机、点、信息卡、状态栏这 4 个核心对象
- “先别做海缆、卫星、BGP”的范围控制
## 用融合版修正的部分
这些是这份新文档加进去的：
- 两阶段起步：先本地 JSON，再真实后端
- 不默认 `localhost` 一定通
- 推荐做 UE 轻量接口，而不是死扛原始接口
- 把打包放到后段，而不是过早纠结
- 时间预估更保守
- 明确“一期只是证明链路跑通”
---
# 十二、验收清单
## 环境
- [ ] UE5.4 安装成功
- [ ] Cesium 插件启用成功
- [ ] Windows 能访问后端接口
## 本地演示版
- [ ] 地球渲染正常
- [ ] 鼠标可旋转和缩放
- [ ] 本地 JSON 数据能生成点
- [ ] 点的位置大体正确
- [ ] 点击点能弹信息卡
- [ ] HUD 可显示本地模式和点数量
## 后端接入版
- [ ] HTTP 能拉取真实数据
- [ ] HTTP 数据能生成点
- [ ] HUD 能显示 Online/Offline
- [ ] 切换 Local / HTTP 模式不崩
- [ ] exe 能打包并运行
---
# 十三、后续路线（MVP 之后）
当这一期做完后，下一步顺序建议是：
1. 海缆路径
2. 卫星点或轨迹
3. 更稳的相机与巡航
4. WebSocket 增量更新
5. BGP 区域态势
6. BGP 事件点
7. 更强的粒子和视觉风格
也就是说：
**先补“静态层和镜头层”，再补“高频实时层”。**
---
# 十四、一句话总结
这份融合版方案的核心就是：
**保留原 MVP 的入门友好度，但改成“先本地 JSON、再真实后端”的两阶段实施路线，让你第一次做 UE 时更稳、更容易成功。**
如果你按这份方案推进，一期最现实的目标不是“立刻做出完整 UE 大屏”，而是：
**在 14 天左右，做出一个能显示真实地球、能显示超算点、能点击看详情、能接后端的可用 UE 客户端 MVP。**

3
docs/version-history.md
View File

@@ -16,12 +16,13 @@
## Current Version
- `main` 当前主线历史推导到：`0.16.5`
- `dev` 当前开发分支历史推导到：`0.28.1`
- `dev` 当前开发分支历史推导到：`0.28.2`
## Timeline
| Version | Type | Branch | Commit | Summary |
| --- | --- | --- | --- | --- |
| `0.28.2` | bugfix | `dev` | `pending` | 修正媒体情报 tab 尺寸记忆与切换锚点逻辑，并清理 docs 根目录遗留旧路径文档 |
| `0.28.1` | bugfix | `dev` | `pending` | 收口 Earth 媒体情报面板命名与 tab 文案，整理 docs 分组并归档已完成/废弃计划文档 |
| `0.28.0` | feature | `dev` | `pending` | 合并 Earth 媒体情报面板，整合新闻直播与态势聚合 tab，并稳定 TV/news 的 reform、resize 与共享 HUD 行为 |
| `0.27.8` | bugfix | `dev` | `pending` | 统一 Earth HUD 默认折叠逻辑，修复图例与图层面板箭头和底边阈值行为 |

2
frontend/package.json
View File

@@ -1,6 +1,6 @@
{
"name": "planet-frontend",
"version": "0.28.1",
"version": "0.28.2",
"private": true,
"packageManager": "bun@1",
"dependencies": {

156
frontend/public/earth/js/tv.js
View File

@@ -32,6 +32,10 @@ let activeTab = "live";
const failedSourceIds = new Set();
let probeTimer = null;
let reformCleanupTimer = null;
const tabPanelState = {
live: null,
news: null,
};
const META_AUTO_COLLAPSE_DELAY = 2500;
const PROBE_INTERVAL_MS = 2 * 60 * 1000;
@@ -129,6 +133,82 @@ function clearPanelPositioningForResize(panel) {
panel.dataset.dragged = "true";
}
function readPanelLayoutState(panel) {
return {
width: panel.style.width || "",
height: panel.style.height || "",
resized: panel.dataset.resized === "true",
};
}
function resetPanelLayoutState(panel) {
panel.style.width = "";
panel.style.height = "";
delete panel.dataset.resized;
}
function captureTabState(tab = activeTab) {
const { panel } = getElements();
if (!(panel instanceof HTMLElement)) return;
tabPanelState[tab] = {
layout: readPanelLayoutState(panel),
metaCollapsed:
tab === "live" ? (mediaPanel?.isCollapsed() ?? false) : null,
};
}
function restoreTabState(tab, panel, container, anchor = null) {
if (!(panel instanceof HTMLElement)) return;
const snapshot = tabPanelState[tab];
if (!snapshot?.layout) {
resetPanelLayoutState(panel);
return;
}
const { layout } = snapshot;
panel.style.width = layout.width;
panel.style.height = layout.height;
if (layout.resized) {
panel.dataset.resized = "true";
} else {
delete panel.dataset.resized;
}
if (tab === "live" && snapshot.metaCollapsed !== null) {
setMetaCollapsed(snapshot.metaCollapsed);
}
requestAnimationFrame(() => {
if (anchor) {
const panelRect = panel.getBoundingClientRect();
const containerRect = container.getBoundingClientRect();
const margin = Math.round(PANEL_RESIZE_MARGIN_PX * getHudScale());
const targetLeft = anchor.right - containerRect.left - panelRect.width;
const targetTop = anchor.bottom - containerRect.top - panelRect.height;
const maxLeft = Math.max(0, containerRect.width - panelRect.width - margin);
const maxTop = Math.max(0, containerRect.height - panelRect.height - margin);
const clampedLeft = Math.min(maxLeft, Math.max(0, targetLeft));
const clampedTop = Math.min(maxTop, Math.max(0, targetTop));
panel.style.left = `${clampedLeft}px`;
panel.style.top = `${clampedTop}px`;
panel.style.right = "auto";
panel.style.bottom = "auto";
panel.style.transform = "none";
panel.dataset.dragged = "true";
} else {
panel.style.right = "";
panel.style.bottom = "";
panel.style.left = "";
panel.style.top = "";
panel.style.transform = "";
delete panel.dataset.dragged;
}
});
}
function getHudScale() {
const scale = Number.parseFloat(
getComputedStyle(document.documentElement).getPropertyValue("--hud-scale"),
@@ -136,6 +216,27 @@ function getHudScale() {
return Number.isFinite(scale) && scale > 0 ? scale : 1;
}
function clampPanelToContainer(panel, container) {
if (!(panel instanceof HTMLElement) || !(container instanceof HTMLElement)) return;
if (panel.dataset.dragged !== "true") return;
const containerRect = container.getBoundingClientRect();
const panelRect = panel.getBoundingClientRect();
const margin = Math.round(PANEL_RESIZE_MARGIN_PX * getHudScale());
const maxLeft = Math.max(0, containerRect.width - panelRect.width - margin);
const maxTop = Math.max(0, containerRect.height - panelRect.height - margin);
const currentLeft = panelRect.left - containerRect.left;
const currentTop = panelRect.top - containerRect.top;
const clampedLeft = Math.min(maxLeft, Math.max(0, currentLeft));
const clampedTop = Math.min(maxTop, Math.max(0, currentTop));
panel.style.left = `${clampedLeft}px`;
panel.style.top = `${clampedTop}px`;
panel.style.right = "auto";
panel.style.bottom = "auto";
panel.style.transform = "none";
}
function setupResizeHandle() {
const { panel } = getElements();
const container = document.getElementById("container");
@@ -280,6 +381,14 @@ function animateTabReform(applyChange) {
return;
}
if (panel.dataset.resized === "true") {
applyChange();
requestAnimationFrame(() => {
clampPanelToContainer(panel, container);
});
return;
}
if (panel.classList.contains("is-dragging") || panel.classList.contains("is-resizing")) {
applyChange();
return;
@@ -351,7 +460,26 @@ function setActiveTab(tab) {
const nextTab = tab === "news" ? "news" : "live";
if (activeTab === nextTab) return;
animateTabReform(() => {
captureTabState(activeTab);
const { panel } = getElements();
const targetSnapshot = tabPanelState[nextTab];
const currentIsCustom =
panel instanceof HTMLElement && panel.dataset.resized === "true";
const targetIsCustom = Boolean(targetSnapshot?.layout?.resized);
const container = document.getElementById("container");
const currentAnchor =
panel instanceof HTMLElement && container instanceof HTMLElement
? (() => {
const panelRect = panel.getBoundingClientRect();
return {
right: panelRect.right,
bottom: panelRect.bottom,
};
})()
: null;
const applyTabSwitch = (restoreLayoutState = false) => {
activeTab = nextTab;
syncPanelActiveTab(nextTab);
syncNewsDefaultMaxHeight();
@@ -370,10 +498,30 @@ function setActiveTab(tab) {
updateTabState(newsHeaderControls, nextTab === "news");
updateTabState(livePane, nextTab === "live", "tv-tab-pane--active");
updateTabState(newsPane, nextTab === "news", "tv-tab-pane--active");
if (
restoreLayoutState &&
panel instanceof HTMLElement &&
container instanceof HTMLElement
) {
restoreTabState(nextTab, panel, container, currentAnchor);
}
requestAnimationFrame(() => {
captureTabState(nextTab);
});
window.dispatchEvent(new CustomEvent("earth:tv-tab-change", {
detail: { tab: nextTab },
}));
});
};
if (currentIsCustom || targetIsCustom) {
applyTabSwitch(true);
return;
}
animateTabReform(() => applyTabSwitch(false));
}
export function openTVPanelTab(tab = "live") {
@@ -951,7 +1099,9 @@ export function initTVPanel() {
setupResizeHandle();
syncPanelActiveTab("live");
syncNewsDefaultMaxHeight();
setActiveTab("live");
updateTabButtonState(liveTabBtn, true);
updateTabButtonState(newsTabBtn, false);
captureTabState("live");
window.addEventListener("resize", syncNewsDefaultMaxHeight);
}

2
pyproject.toml
View File

@@ -1,6 +1,6 @@
[project]
name = "planet"
version = "0.28.1"
version = "0.28.2"
description = "智能星球计划 - 态势感知系统"
requires-python = ">=3.14"
dependencies = [

2
uv.lock generated
View File

@@ -475,7 +475,7 @@ wheels = [
[[package]]
name = "planet"
version = "0.28.1"
version = "0.28.2"
source = { virtual = "." }
dependencies = [
{ name = "aiofiles" },