release: bump version to 0.31.1
This commit is contained in:
rayd1o
26
docs/technical/README.md
Normal file
26
docs/technical/README.md
Normal file
@@ -0,0 +1,26 @@
|
||||
# Technical Docs
|
||||
|
||||
这里放“当前实现和当前结构”的文档，重点回答：
|
||||
|
||||
- 现在代码是怎么组织的
|
||||
- 当前入口在哪
|
||||
- 状态和组件如何工作
|
||||
- 后续改动应该沿着哪条实现边界继续走
|
||||
|
||||
适合放入这里的内容：
|
||||
|
||||
- 前端上下文
|
||||
- Earth 前端结构
|
||||
- 后端运行控制
|
||||
- collector 现状
|
||||
- 采集格式约定
|
||||
|
||||
不适合放入这里的内容：
|
||||
|
||||
- 尚未完成的 roadmap
|
||||
- 未来迭代方案
|
||||
- 大范围重构计划
|
||||
|
||||
这些应放入：
|
||||
|
||||
- [docs/plans/README.md](/home/ray/dev/linkong/planet/docs/plans/README.md)
|
||||
333
docs/technical/agents-aiprovider.md
Normal file
333
docs/technical/agents-aiprovider.md
Normal file
@@ -0,0 +1,333 @@
|
||||
# AI Provider Guide
|
||||
|
||||
## Overview
|
||||
|
||||
`aiprovider` is the model-adapter service for Planet.
|
||||
|
||||
It isolates model-vendor details from the main backend so the rest of the system can call a stable business API:
|
||||
|
||||
- Caller service -> `planet backend`
|
||||
- `planet backend` -> `aiprovider`
|
||||
- `aiprovider` -> concrete model provider
|
||||
|
||||
The recommended default is:
|
||||
|
||||
- External and cross-service callers use `planet backend`
|
||||
- Only infrastructure-grade internal jobs call `aiprovider` directly
|
||||
|
||||
## Responsibilities
|
||||
|
||||
`backend` is responsible for:
|
||||
|
||||
- authentication and authorization
|
||||
- business-level request shaping
|
||||
- stable `/api/v1/ai/...` endpoints
|
||||
- internal service-to-service authentication toward `aiprovider`
|
||||
|
||||
`aiprovider` is responsible for:
|
||||
|
||||
- model protocol adaptation
|
||||
- provider selection by `.env`
|
||||
- timeout and lightweight retry
|
||||
- request tracing via `X-Request-ID`
|
||||
|
||||
This now follows an OpenClaw-like seam:
|
||||
|
||||
- `AI_PROVIDER` identifies the vendor or logical provider
|
||||
- `AI_PROVIDER_API` identifies the wire adapter
|
||||
|
||||
That split makes MiniMax, Claude-compatible gateways, and self-hosted OpenAI-compatible services easier to model without overloading one config field.
|
||||
|
||||
## Supported Providers
|
||||
|
||||
`aiprovider` currently supports these provider identities:
|
||||
|
||||
- `openai`
|
||||
- `anthropic`
|
||||
- `minimax`
|
||||
- `ollama`
|
||||
|
||||
Supported request adapters:
|
||||
|
||||
- `openai-completions`
|
||||
- `anthropic-messages`
|
||||
- `ollama-generate`
|
||||
|
||||
Backward-compatible aliases still accepted:
|
||||
|
||||
- `openai_compatible`
|
||||
- `anthropic_compatible`
|
||||
- `claude_compatible`
|
||||
|
||||
Provider mapping:
|
||||
|
||||
- `vLLM`, `LM Studio`, `One API`: `AI_PROVIDER=openai`, `AI_PROVIDER_API=openai-completions`
|
||||
- `MiniMax`: `AI_PROVIDER=minimax`, `AI_PROVIDER_API=anthropic-messages`
|
||||
- Claude-compatible gateways: `AI_PROVIDER=anthropic`, `AI_PROVIDER_API=anthropic-messages`
|
||||
- `Ollama`: `AI_PROVIDER=ollama`, `AI_PROVIDER_API=ollama-generate`
|
||||
|
||||
## API Surfaces
|
||||
|
||||
### Main backend API
|
||||
|
||||
Preferred stable entrypoints:
|
||||
|
||||
- `GET /api/v1/ai/provider/status`
|
||||
- `POST /api/v1/ai/situational-awareness/analyze`
|
||||
|
||||
Authentication:
|
||||
|
||||
- `Authorization: Bearer <jwt>`
|
||||
|
||||
Optional tracing header:
|
||||
|
||||
- `X-Request-ID: <caller-generated-id>`
|
||||
|
||||
The backend will propagate `X-Request-ID` to `aiprovider` and return the same header in the response.
|
||||
|
||||
### AI provider internal API
|
||||
|
||||
Internal-only endpoints:
|
||||
|
||||
- `GET /v1/provider/status`
|
||||
- `POST /v1/analyze`
|
||||
|
||||
Authentication:
|
||||
|
||||
- `X-Provider-Token: <shared-secret>`
|
||||
|
||||
Optional tracing header:
|
||||
|
||||
- `X-Request-ID: <caller-generated-id>`
|
||||
|
||||
## Request Example
|
||||
|
||||
### Call through backend
|
||||
|
||||
```bash
|
||||
curl -X POST http://localhost:8000/api/v1/ai/situational-awareness/analyze \
|
||||
-H "Authorization: Bearer <access_token>" \
|
||||
-H "X-Request-ID: bgp-incident-20260407-001" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"title": "BGP异常研判",
|
||||
"objective": "总结当前风险并给出处置建议",
|
||||
"observations": [
|
||||
"collector A 在 5 分钟内出现多次 origin 变更",
|
||||
"异常集中在同一地区前缀"
|
||||
],
|
||||
"constraints": [
|
||||
"不要编造不存在的数据",
|
||||
"区分事实和推断"
|
||||
],
|
||||
"context": {
|
||||
"source": "bgp-monitor",
|
||||
"severity": "high"
|
||||
}
|
||||
}'
|
||||
```
|
||||
|
||||
### Call `aiprovider` directly
|
||||
|
||||
```bash
|
||||
curl -X POST http://localhost:8010/v1/analyze \
|
||||
-H "X-Provider-Token: change_me" \
|
||||
-H "X-Request-ID: ai-batch-job-001" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"title": "链路波动分析",
|
||||
"objective": "给出简要态势摘要和下一步建议",
|
||||
"observations": [
|
||||
"多个节点出现延迟上升"
|
||||
],
|
||||
"constraints": [
|
||||
"不要假设根因已经确认"
|
||||
],
|
||||
"context": {
|
||||
"region": "APAC"
|
||||
}
|
||||
}'
|
||||
```
|
||||
|
||||
## Response Shape
|
||||
|
||||
Both backend and `aiprovider` return the same payload shape:
|
||||
|
||||
```json
|
||||
{
|
||||
"provider": "minimax",
|
||||
"api": "anthropic-messages",
|
||||
"model": "MiniMax-M2.7",
|
||||
"content": "1) 态势摘要 ...",
|
||||
"content_blocks": [],
|
||||
"text_blocks": [],
|
||||
"thinking_blocks": [],
|
||||
"raw_response": {}
|
||||
}
|
||||
```
|
||||
|
||||
Both services also return:
|
||||
|
||||
- `X-Request-ID: <id>`
|
||||
|
||||
## Configuration
|
||||
|
||||
### Backend
|
||||
|
||||
Recommended backend `.env`:
|
||||
|
||||
```env
|
||||
AI_PROVIDER_SERVICE_URL=http://localhost:8010
|
||||
AI_PROVIDER_SERVICE_TOKEN=change_me
|
||||
AI_PROVIDER_TIMEOUT_SECONDS=60
|
||||
AI_PROVIDER_RETRY_ATTEMPTS=2
|
||||
```
|
||||
|
||||
Reference file:
|
||||
|
||||
- [backend/.env.example](/home/ray/dev/linkong/planet/backend/.env.example)
|
||||
|
||||
### AI Provider
|
||||
|
||||
Reference file:
|
||||
|
||||
- [aiprovider/.env.example](/home/ray/dev/linkong/planet/aiprovider/.env.example)
|
||||
|
||||
Frontend local reference:
|
||||
|
||||
- [frontend/.env.example](/home/ray/dev/linkong/planet/frontend/.env.example)
|
||||
|
||||
Common settings:
|
||||
|
||||
```env
|
||||
SERVICE_NAME=planet-ai-provider
|
||||
SERVICE_VERSION=0.1.0
|
||||
AI_PROVIDER_SERVICE_TOKEN=change_me
|
||||
AI_TIMEOUT_SECONDS=60
|
||||
AI_HTTP_RETRY_ATTEMPTS=2
|
||||
AI_ANALYSIS_SYSTEM_PROMPT=你是态势感知分析助手。请基于输入的上下文、观测与约束，输出结构化、克制、可执行的分析。
|
||||
```
|
||||
|
||||
### OpenAI-compatible example
|
||||
|
||||
```env
|
||||
AI_PROVIDER=openai
|
||||
AI_PROVIDER_API=openai-completions
|
||||
AI_BASE_URL=http://127.0.0.1:8001/v1
|
||||
AI_API_KEY=local-key
|
||||
AI_MODEL=your-local-model
|
||||
```
|
||||
|
||||
### MiniMax CN example
|
||||
|
||||
```env
|
||||
AI_PROVIDER=minimax
|
||||
AI_PROVIDER_API=anthropic-messages
|
||||
AI_BASE_URL=https://api.minimaxi.com/anthropic
|
||||
AI_API_KEY=sk-cp-xxxxx
|
||||
AI_MODEL=MiniMax-M2.7
|
||||
AI_MAX_TOKENS=1200
|
||||
AI_ANTHROPIC_VERSION=2023-06-01
|
||||
```
|
||||
|
||||
MiniMax note:
|
||||
|
||||
- This follows the same Anthropic Messages request shape as the official MiniMax examples.
|
||||
- For MiniMax, `aiprovider` now disables `thinking` by default unless the caller explicitly passes a `thinking` object.
|
||||
- This mirrors OpenClaw's caution around MiniMax Anthropic-compatible behavior.
|
||||
|
||||
### Anthropic-compatible example
|
||||
|
||||
```env
|
||||
AI_PROVIDER=anthropic
|
||||
AI_PROVIDER_API=anthropic-messages
|
||||
AI_BASE_URL=https://your-claude-compatible-endpoint.example.com/anthropic
|
||||
AI_API_KEY=your_api_key
|
||||
AI_MODEL=your-model
|
||||
AI_MAX_TOKENS=1200
|
||||
AI_ANTHROPIC_VERSION=2023-06-01
|
||||
```
|
||||
|
||||
### Ollama example
|
||||
|
||||
```env
|
||||
AI_PROVIDER=ollama
|
||||
AI_PROVIDER_API=ollama-generate
|
||||
AI_BASE_URL=http://127.0.0.1:11434
|
||||
AI_API_KEY=
|
||||
AI_MODEL=qwen2.5:7b
|
||||
```
|
||||
|
||||
## Deployment Modes
|
||||
|
||||
### Single machine
|
||||
|
||||
Recommended local flow:
|
||||
|
||||
- `backend` on `localhost:8000`
|
||||
- `aiprovider` on `localhost:8010`
|
||||
- local model gateway on `localhost:11434` or another local port
|
||||
|
||||
Helpers already included:
|
||||
|
||||
- [planet.sh](/home/ray/dev/linkong/planet/planet.sh)
|
||||
- [docker-compose.local-model.yml](/home/ray/dev/linkong/planet/docker-compose.local-model.yml)
|
||||
|
||||
### Multi-machine
|
||||
|
||||
Example topology:
|
||||
|
||||
- app machine: `backend`
|
||||
- AI gateway machine: `aiprovider`
|
||||
- model machine: local model service or cloud proxy
|
||||
|
||||
In that case, this becomes service-to-service HTTP RPC:
|
||||
|
||||
- caller -> backend
|
||||
- backend -> `http://10.0.0.12:8010`
|
||||
- `aiprovider` -> model endpoint
|
||||
|
||||
Recommended cross-machine backend config:
|
||||
|
||||
```env
|
||||
AI_PROVIDER_SERVICE_URL=http://10.0.0.12:8010
|
||||
AI_PROVIDER_SERVICE_TOKEN=change_me
|
||||
AI_PROVIDER_TIMEOUT_SECONDS=60
|
||||
AI_PROVIDER_RETRY_ATTEMPTS=2
|
||||
```
|
||||
|
||||
Recommended operating rules:
|
||||
|
||||
- keep `aiprovider` on a private network
|
||||
- protect it with `X-Provider-Token` at minimum
|
||||
- always send `X-Request-ID`
|
||||
- keep callers on the backend API unless they are infrastructure jobs
|
||||
|
||||
## Retry And Failure Behavior
|
||||
|
||||
`backend -> aiprovider`:
|
||||
|
||||
- retries lightweight network / 5xx failures
|
||||
- returns `502` when the provider service is unavailable
|
||||
|
||||
`aiprovider -> model provider`:
|
||||
|
||||
- retries lightweight network / 5xx failures
|
||||
- returns `502` when the model provider is unavailable
|
||||
|
||||
This is intentionally conservative. It avoids masking persistent errors while still absorbing short hiccups.
|
||||
|
||||
## Operational Notes
|
||||
|
||||
- `./planet.sh start` now starts `aiprovider` automatically
|
||||
- `./planet.sh restart -a` restarts only `aiprovider`
|
||||
- `./planet.sh log -a` tails `aiprovider` logs
|
||||
- `./planet.sh health` reports `aiprovider` health
|
||||
|
||||
## Recommended Calling Policy
|
||||
|
||||
- Frontend and application services: call `backend`
|
||||
- Scheduled infra jobs and diagnostics: optionally call `aiprovider`
|
||||
- Do not let multiple business services integrate model vendors independently
|
||||
|
||||
That keeps provider switching centralized and avoids model-specific drift across the system.
|
||||
263
docs/technical/backend-collectors.md
Normal file
263
docs/technical/backend-collectors.md
Normal file
@@ -0,0 +1,263 @@
|
||||
# 数据采集系统 (Collectors)
|
||||
|
||||
## 一、系统架构
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ 数据采集系统架构 │
|
||||
├─────────────────────────────────────────────────────────────────┤
|
||||
│ │
|
||||
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │
|
||||
│ │ TOP500 │ │ Epoch AI │ │ HuggingFace │ │
|
||||
│ │ 采集器 │ │ 采集器 │ │ 采集器 │ │
|
||||
│ └──────┬──────┘ └──────┬──────┘ └──────┬──────┘ │
|
||||
│ │ │ │ │
|
||||
│ └───────────────────┼───────────────────┘ │
|
||||
│ ▼ │
|
||||
│ ┌─────────────────────┐ │
|
||||
│ │ BaseCollector │◄── 基类 (统一处理) │
|
||||
│ │ run() 方法 │ │
|
||||
│ └─────────┬───────────┘ │
|
||||
│ │ │
|
||||
│ ┌─────────────────┼─────────────────┐ │
|
||||
│ ▼ ▼ ▼ │
|
||||
│ ┌───────────┐ ┌───────────┐ ┌───────────┐ │
|
||||
│ │ fetch() │ │transform()│ │ _save_data│ │
|
||||
│ │ 获取原始数据 │ │ 数据转换 │ │ 保存到DB │ │
|
||||
│ └───────────┘ └───────────┘ └───────────┘ │
|
||||
│ │ │
|
||||
│ ▼ │
|
||||
│ ┌─────────────────────┐ │
|
||||
│ │ CollectedData 表 │◄── 统一存储 │
|
||||
│ └─────────────────────┘ │
|
||||
│ │
|
||||
│ ┌─────────────────────────────────────────────────────────┐ │
|
||||
│ │ Scheduler (APScheduler) │ │
|
||||
│ │ 定时任务调度: 每4小时/6小时/12小时/1天 自动执行 │ │
|
||||
│ └─────────────────────────────────────────────────────────┘ │
|
||||
│ │
|
||||
└─────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## 二、工作流程 (Pipeline)
|
||||
|
||||
```python
|
||||
# 1. Scheduler 触发 (定时 或 手动触发)
|
||||
# ↓
|
||||
|
||||
# 2. run() 方法执行完整流水线
|
||||
async def run(self, db):
|
||||
# 2.1 检查采集器是否启用
|
||||
if not collector_registry.is_active(self.name):
|
||||
return {"status": "skipped"}
|
||||
|
||||
# 2.2 记录任务开始
|
||||
task = CollectionTask(status="running")
|
||||
db.add(task)
|
||||
await db.commit()
|
||||
|
||||
# 2.3 FETCH - 获取原始数据 (由子类实现)
|
||||
raw_data = await self.fetch()
|
||||
|
||||
# 2.4 TRANSFORM - 转换为统一格式
|
||||
data = self.transform(raw_data)
|
||||
|
||||
# 2.5 SAVE - 保存到数据库
|
||||
records_count = await self._save_data(db, data)
|
||||
|
||||
# 2.6 记录任务完成
|
||||
task.status = "success"
|
||||
task.records_processed = records_count
|
||||
await db.commit()
|
||||
```
|
||||
|
||||
**核心文件**: `backend/app/services/collectors/base.py`
|
||||
|
||||
## 三、采集器列表
|
||||
|
||||
| 采集器 | 数据类型 | 数据内容 | 采集频率 |
|
||||
|--------|----------|----------|----------|
|
||||
| TOP500 | supercomputer | 全球超级计算机排名 (算力、性能) | 4小时 |
|
||||
| Epoch AI | gpu_cluster | GPU算力集群信息 | 6小时 |
|
||||
| HuggingFace Models | model | AI模型信息 | 12小时 |
|
||||
| HuggingFace Datasets | dataset | 数据集信息 | 12小时 |
|
||||
| HuggingFace Spaces | space | Demo应用 | 1天 |
|
||||
| PeeringDB | ixp/network/facility | 互联网交换点/网络/机房 | 1-2天 |
|
||||
| TeleGeography | submarine_cable | 海底光缆信息 | 7天 |
|
||||
|
||||
## 四、数据格式 (统一存储到 CollectedData 表)
|
||||
|
||||
```python
|
||||
# 每个采集器 parse_response() 返回格式
|
||||
{
|
||||
"source_id": "top500_1", # 原始系统ID (必填)
|
||||
"name": "El Capitan", # 名称 (必填)
|
||||
"description": "系统描述...", # 描述
|
||||
"country": "United States", # 国家
|
||||
"city": "Livermore, CA", # 城市
|
||||
"latitude": "37.6819", # 纬度 (字符串)
|
||||
"longitude": "-121.7681", # 经度 (字符串)
|
||||
"value": "1742.00", # 性能值 (如算力)
|
||||
"unit": "PFlop/s", # 单位
|
||||
"metadata": { # 额外数据 (JSON)
|
||||
"rank": 1,
|
||||
"r_peak": 2746.38,
|
||||
"cores": 11039616
|
||||
},
|
||||
"reference_date": "2025-11-01" # 数据参考日期
|
||||
}
|
||||
```
|
||||
|
||||
## 五、数据库表结构
|
||||
|
||||
**CollectedData 表** (`collected_data`)
|
||||
|
||||
| 字段 | 类型 | 说明 |
|
||||
|------|------|------|
|
||||
| id | SERIAL | 主键 |
|
||||
| source | VARCHAR(100) | 数据源名称 (top500, huggingface等) |
|
||||
| source_id | VARCHAR(100) | 原始数据ID |
|
||||
| data_type | VARCHAR(50) | 数据类型 (supercomputer, model等) |
|
||||
| name | VARCHAR(500) | 名称 |
|
||||
| title | VARCHAR(500) | 标题 |
|
||||
| description | TEXT | 描述 |
|
||||
| country | VARCHAR(100) | 国家 |
|
||||
| city | VARCHAR(100) | 城市 |
|
||||
| latitude | VARCHAR(50) | 纬度 |
|
||||
| longitude | VARCHAR(50) | 经度 |
|
||||
| value | VARCHAR(100) | 性能值 |
|
||||
| unit | VARCHAR(20) | 单位 |
|
||||
| metadata | JSONB | 额外元数据 |
|
||||
| collected_at | TIMESTAMP | 采集时间 |
|
||||
| reference_date | TIMESTAMP | 数据参考日期 |
|
||||
| is_valid | INTEGER | 是否有效 |
|
||||
|
||||
**核心文件**: `backend/app/models/collected_data.py`
|
||||
|
||||
## 六、TOP500 采集器示例 (完整流程)
|
||||
|
||||
```python
|
||||
# 1. fetch() - 从网页获取HTML
|
||||
async def fetch(self):
|
||||
url = "https://top500.org/lists/top500/list/2025/11/"
|
||||
response = await client.get(url)
|
||||
return response.text # 返回HTML
|
||||
|
||||
# 2. parse_response() - 解析HTML为统一格式
|
||||
def parse_response(self, html):
|
||||
soup = BeautifulSoup(html, "html.parser")
|
||||
table = soup.find("table")
|
||||
|
||||
for row in table.find_all("tr")[1:]: # 跳过表头
|
||||
cells = row.find_all("td")
|
||||
|
||||
entry = {
|
||||
"source_id": f"top500_{cells[0].text}", # "top500_1"
|
||||
"name": cells[1].text.strip(), # "El Capitan"
|
||||
"country": cells[2].text.strip(), # "United States"
|
||||
"city": "", # 城市
|
||||
"latitude": "", # 需进一步解析
|
||||
"longitude": "",
|
||||
"value": "1742.00", # Rmax
|
||||
"unit": "PFlop/s",
|
||||
"metadata": {
|
||||
"rank": 1,
|
||||
"cores": "11340000"
|
||||
},
|
||||
"reference_date": "2025-11-01"
|
||||
}
|
||||
data.append(entry)
|
||||
|
||||
return data
|
||||
|
||||
# 3. run() 自动调用 _save_data() 保存到数据库
|
||||
```
|
||||
|
||||
**核心文件**: `backend/app/services/collectors/top500.py`
|
||||
|
||||
## 七、调度机制
|
||||
|
||||
```python
|
||||
# 启动时注册所有采集器到定时任务
|
||||
def start_scheduler():
|
||||
for name, collector in collectors.items():
|
||||
if collector_registry.is_active(name):
|
||||
scheduler.add_job(
|
||||
run_collector_task,
|
||||
trigger=IntervalTrigger(hours=collector.frequency_hours),
|
||||
id=name,
|
||||
name=name
|
||||
)
|
||||
```
|
||||
|
||||
| 采集器 | 采集频率 |
|
||||
|--------|----------|
|
||||
| TOP500 | 每4小时 |
|
||||
| Epoch AI | 每6小时 |
|
||||
| HuggingFace | 每12小时 |
|
||||
| PeeringDB | 每1-2天 |
|
||||
| TeleGeography | 每7天 |
|
||||
|
||||
**核心文件**: `backend/app/services/scheduler.py`
|
||||
|
||||
## 八、相关代码文件
|
||||
|
||||
```
|
||||
backend/app/services/collectors/
|
||||
├── base.py # 基类: run() 流水线, _save_data() 保存
|
||||
├── registry.py # 采集器注册表
|
||||
├── scheduler.py # 定时任务调度 (APScheduler)
|
||||
├── top500.py # TOP500采集器
|
||||
├── epoch_ai.py # Epoch AI采集器
|
||||
├── huggingface.py # HuggingFace采集器
|
||||
├── peeringdb.py # PeeringDB采集器
|
||||
└── telegeraphy.py # TeleGeography海底光缆采集器
|
||||
|
||||
backend/app/models/
|
||||
└── collected_data.py # 统一数据模型
|
||||
```
|
||||
|
||||
## 九、数据使用场景
|
||||
|
||||
采集的数据最终会：
|
||||
|
||||
1. **可视化展示** - 在UE5大屏上显示超级计算机、GPU集群、海底光缆的地理位置
|
||||
2. **态势分析** - 统计全球算力分布、增长趋势
|
||||
3. **告警系统** - 检测重要节点变化
|
||||
|
||||
## 十、采集器注册机制
|
||||
|
||||
采集器在应用启动时自动注册：
|
||||
|
||||
```python
|
||||
# backend/app/services/collectors/__init__.py
|
||||
|
||||
collector_registry.register(TOP500Collector())
|
||||
collector_registry.register(EpochAIGPUCollector())
|
||||
collector_registry.register(HuggingFaceModelCollector())
|
||||
collector_registry.register(HuggingFaceDatasetCollector())
|
||||
collector_registry.register(HuggingFaceSpacesCollector())
|
||||
collector_registry.register(PeeringDBIXPCollector())
|
||||
collector_registry.register(PeeringDBNetworkCollector())
|
||||
collector_registry.register(PeeringDBFacilityCollector())
|
||||
collector_registry.register(TeleGeographyCableCollector())
|
||||
collector_registry.register(TeleGeographyLandingPointCollector())
|
||||
collector_registry.register(TeleGeographyCableSystemCollector())
|
||||
```
|
||||
|
||||
**核心文件**: `backend/app/services/collectors/registry.py`
|
||||
|
||||
## 十一、触发采集
|
||||
|
||||
### 方式一：定时触发
|
||||
系统启动时，APScheduler会自动根据各采集器的`frequency_hours`设置定时任务。
|
||||
|
||||
### 方式二：手动触发 API
|
||||
|
||||
```bash
|
||||
# 触发TOP500采集
|
||||
curl -X POST http://localhost:8000/api/v1/datasources/1/trigger \
|
||||
-H "Authorization: Bearer <token>"
|
||||
```
|
||||
|
||||
**核心文件**: `backend/app/api/v1/datasources.py`
|
||||
347
docs/technical/backend-system-service-control.md
Normal file
347
docs/technical/backend-system-service-control.md
Normal file
@@ -0,0 +1,347 @@
|
||||
# System Service Control
|
||||
|
||||
This document defines the fixed mapping between admin control-plane actions and
|
||||
the existing `planet.sh` service-management commands.
|
||||
|
||||
The goal is to reuse the current operational script semantics without exposing
|
||||
arbitrary shell execution to the frontend or API callers.
|
||||
|
||||
## Scope
|
||||
|
||||
- This mapping is for admin-side operational controls only.
|
||||
- The control plane must submit a fixed action name, not a raw shell command.
|
||||
- The backend is responsible for translating an allowed action into a fixed
|
||||
`planet.sh` invocation.
|
||||
|
||||
## Design Rules
|
||||
|
||||
- Only whitelist actions may be executed.
|
||||
- The frontend must never send arbitrary shell strings.
|
||||
- The backend must build command arguments from a fixed mapping table.
|
||||
- High-risk actions should be restricted to `super_admin`.
|
||||
- Prefer partial restarts over full-stack restarts when UI continuity matters.
|
||||
|
||||
## Action Mapping
|
||||
|
||||
| Action name | Intended use | `planet.sh` command | Notes |
|
||||
| --- | --- | --- | --- |
|
||||
| `restart-backend` | Restart backend API only | `./planet.sh restart -b` | Recommended first implementation for UI-triggered restart flows. |
|
||||
| `restart-database` | Restart PostgreSQL and Redis containers | `./planet.sh restart -d` | Useful when database/cache services need a controlled bounce without restarting the UI. |
|
||||
| `restart-system` | Restart the whole application stack | `./planet.sh restart` | Frontend continuity breaks briefly; UI should switch to guided recovery mode. |
|
||||
| `restart-frontend` | Restart frontend dev server only | `./planet.sh restart -f` | Use with caution; UI continuity is weaker than backend-only restart. |
|
||||
| `restart-backend-port` | Restart backend on a specific port | `./planet.sh restart -b <port>` | Port must be backend-validated before execution. |
|
||||
| `restart-frontend-port` | Restart frontend on a specific port | `./planet.sh restart -f <port>` | Port must be backend-validated before execution. |
|
||||
| `health-check` | Read current service health | `./planet.sh health` | Safe read-only operational action. |
|
||||
| `show-logs-backend` | Inspect backend logs | `./planet.sh log -b` | Best used for CLI/operator tooling, not normal Web UI streaming. |
|
||||
| `show-logs-frontend` | Inspect frontend logs | `./planet.sh log -f` | Best used for CLI/operator tooling, not normal Web UI streaming. |
|
||||
|
||||
## Not Exposed In UI By Default
|
||||
|
||||
The following existing script capabilities should not be exposed directly in the
|
||||
Web UI unless there is an explicit product need and an additional safety review:
|
||||
|
||||
- `./planet.sh restart`
|
||||
- `./planet.sh start`
|
||||
- `./planet.sh stop`
|
||||
- `./planet.sh createuser`
|
||||
- any future raw shell passthrough
|
||||
|
||||
Reason:
|
||||
|
||||
- full restart can break the current control session;
|
||||
- stop/start have larger blast radius;
|
||||
- user creation is not a service-control operation;
|
||||
- raw shell passthrough creates unnecessary privilege risk.
|
||||
|
||||
## Recommended First-Phase UI Contract
|
||||
|
||||
### Frontend action payload
|
||||
|
||||
```json
|
||||
{
|
||||
"action": "restart-backend"
|
||||
}
|
||||
```
|
||||
|
||||
### Backend command resolution
|
||||
|
||||
```text
|
||||
restart-backend -> ["./planet.sh", "restart", "-b"]
|
||||
restart-database -> ["./planet.sh", "restart", "-d"]
|
||||
restart-system -> ["./planet.sh", "restart"]
|
||||
restart-frontend -> ["./planet.sh", "restart", "-f"]
|
||||
health-check -> ["./planet.sh", "health"]
|
||||
```
|
||||
|
||||
## API Draft
|
||||
|
||||
### Primary Endpoint
|
||||
|
||||
- `POST /api/v1/system/restart-tasks`
|
||||
|
||||
Purpose:
|
||||
|
||||
- create a controlled restart task;
|
||||
- resolve a whitelist action into a fixed `planet.sh` command;
|
||||
- hand execution off to an external runner or detached subprocess.
|
||||
|
||||
### Request Body
|
||||
|
||||
```json
|
||||
{
|
||||
"action": "restart-backend"
|
||||
}
|
||||
```
|
||||
|
||||
Optional future shape:
|
||||
|
||||
```json
|
||||
{
|
||||
"action": "restart-backend-port",
|
||||
"port": 8000
|
||||
}
|
||||
```
|
||||
|
||||
### Response
|
||||
|
||||
```json
|
||||
{
|
||||
"task_id": "restart_20260331_153000_ab12cd",
|
||||
"action": "restart-backend",
|
||||
"status": "queued",
|
||||
"stage": "accepted",
|
||||
"message": "Restart task accepted"
|
||||
}
|
||||
```
|
||||
|
||||
### Task Query Endpoint
|
||||
|
||||
- `GET /api/v1/system/restart-tasks/{task_id}`
|
||||
|
||||
Response shape:
|
||||
|
||||
```json
|
||||
{
|
||||
"task_id": "restart_20260331_153000_ab12cd",
|
||||
"action": "restart-backend",
|
||||
"status": "queued",
|
||||
"stage": "accepted",
|
||||
"message": "Waiting for execution",
|
||||
"requested_by": {
|
||||
"id": 1,
|
||||
"username": "admin"
|
||||
},
|
||||
"created_at": "2026-03-31T15:30:00+08:00",
|
||||
"updated_at": "2026-03-31T15:30:02+08:00"
|
||||
}
|
||||
```
|
||||
|
||||
### Optional Log Endpoint
|
||||
|
||||
- `GET /api/v1/system/restart-tasks/{task_id}/logs`
|
||||
|
||||
Suggested response:
|
||||
|
||||
```json
|
||||
{
|
||||
"task_id": "restart_20260331_153000_ab12cd",
|
||||
"lines": [
|
||||
"accepted restart-backend request",
|
||||
"spawning restart command",
|
||||
"waiting for backend shutdown",
|
||||
"waiting for backend health recovery"
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
This log endpoint is optional for phase one. The first version can work with
|
||||
task state plus `/health` polling alone.
|
||||
|
||||
## Task State Model
|
||||
|
||||
### Status
|
||||
|
||||
- `queued`
|
||||
- `running`
|
||||
- `succeeded`
|
||||
- `failed`
|
||||
- `timeout`
|
||||
|
||||
### Stage
|
||||
|
||||
- `accepted`
|
||||
- `spawning`
|
||||
- `stopping`
|
||||
- `starting`
|
||||
- `waiting_for_health`
|
||||
- `healthy`
|
||||
- `failed`
|
||||
|
||||
### Interpretation
|
||||
|
||||
- `status` is the high-level terminal or non-terminal state.
|
||||
- `stage` is the operator-facing execution phase for the UI.
|
||||
- `message` is the short human-readable line shown in the modal or full-screen
|
||||
overlay.
|
||||
|
||||
## Permission Model
|
||||
|
||||
- `restart-backend` should require `super_admin`.
|
||||
- Permission checks should follow the same role pattern already used in
|
||||
[users.py](/home/ray/dev/linkong/planet/backend/app/api/v1/users.py).
|
||||
- Frontend visibility may hide controls for non-`super_admin`, but backend must
|
||||
still enforce authorization.
|
||||
|
||||
## Storage Model
|
||||
|
||||
Recommended first implementation:
|
||||
|
||||
- store restart task state in Redis;
|
||||
- keep task lifetime short;
|
||||
- keep recent logs as a bounded list.
|
||||
|
||||
Suggested keys:
|
||||
|
||||
- `system:restart_task:{task_id}`
|
||||
- `system:restart_task:{task_id}:logs`
|
||||
|
||||
Suggested stored fields:
|
||||
|
||||
- `task_id`
|
||||
- `action`
|
||||
- `status`
|
||||
- `stage`
|
||||
- `message`
|
||||
- `requested_by_id`
|
||||
- `requested_by_username`
|
||||
- `created_at`
|
||||
- `updated_at`
|
||||
|
||||
## Execution Model
|
||||
|
||||
The request-handling API process should not depend on itself surviving long
|
||||
enough to stream the whole restart output.
|
||||
|
||||
Recommended execution flow:
|
||||
|
||||
1. validate caller and action
|
||||
2. create task state in Redis
|
||||
3. resolve action to fixed `planet.sh` argv
|
||||
4. spawn detached executor
|
||||
5. return `task_id`
|
||||
6. executor updates task state while restart is in progress
|
||||
7. frontend polls health and/or task state until recovery
|
||||
|
||||
Recommended command resolution examples:
|
||||
|
||||
```text
|
||||
restart-backend -> ["./planet.sh", "restart", "-b"]
|
||||
restart-frontend -> ["./planet.sh", "restart", "-f"]
|
||||
restart-backend-port -> ["./planet.sh", "restart", "-b", "<port>"]
|
||||
health-check -> ["./planet.sh", "health"]
|
||||
```
|
||||
|
||||
## Frontend Polling Flow
|
||||
|
||||
Recommended first-phase UX:
|
||||
|
||||
1. user clicks `重启后端`
|
||||
2. confirmation modal explains temporary unavailability
|
||||
3. frontend calls `POST /api/v1/system/restart-tasks`
|
||||
4. UI enters blocking restart state
|
||||
5. frontend polls `/health` every `1-2s`
|
||||
6. temporary request failures are treated as expected
|
||||
7. after `2-3` consecutive successful health checks, frontend reloads page
|
||||
|
||||
Optional richer polling:
|
||||
|
||||
1. poll task status endpoint while backend is still reachable
|
||||
2. switch to `/health` recovery polling after disconnect begins
|
||||
3. refresh page after health recovery
|
||||
|
||||
## Frontend State Machine
|
||||
|
||||
- `idle`
|
||||
- `confirming`
|
||||
- `submitting`
|
||||
- `waiting_for_shutdown`
|
||||
- `waiting_for_recovery`
|
||||
- `recovered`
|
||||
- `failed`
|
||||
- `timeout`
|
||||
|
||||
Suggested UI messages:
|
||||
|
||||
- `已发送重启指令`
|
||||
- `正在停止后端服务`
|
||||
- `正在等待服务恢复`
|
||||
- `服务已恢复，正在刷新页面`
|
||||
- `恢复超时，请手动检查服务状态`
|
||||
|
||||
## Phase-One Recommendation
|
||||
|
||||
Implement only the following in phase one:
|
||||
|
||||
- `restart-backend`
|
||||
- `super_admin` permission gate
|
||||
- task creation endpoint
|
||||
- Redis-backed task state
|
||||
- frontend confirmation modal
|
||||
- frontend `/health` polling
|
||||
- automatic page reload after recovery
|
||||
|
||||
Do not implement in phase one:
|
||||
|
||||
- full `./planet.sh restart`
|
||||
- raw shell command passthrough
|
||||
- arbitrary service control
|
||||
- full terminal stdout streaming
|
||||
- multi-action concurrent restart queueing
|
||||
|
||||
## Implementation Checklist
|
||||
|
||||
### Backend
|
||||
|
||||
1. add a dedicated system-control API module under `backend/app/api/v1/`
|
||||
2. add a whitelist-based action resolver for `planet.sh`
|
||||
3. store restart task state in Redis
|
||||
4. add detached restart-runner script execution
|
||||
5. expose:
|
||||
- `POST /api/v1/system/restart-tasks`
|
||||
- `GET /api/v1/system/restart-tasks/{task_id}`
|
||||
- optional task log endpoint
|
||||
6. enforce `super_admin` permission on all restart-task endpoints
|
||||
|
||||
### Frontend
|
||||
|
||||
1. add a `重启后端` control on the dashboard for `super_admin`
|
||||
2. show a confirmation modal before dispatch
|
||||
3. after submission, switch modal into blocking restart state
|
||||
4. poll `/health` until backend recovery is confirmed
|
||||
5. auto-refresh page after consecutive successful health checks
|
||||
6. show short stage-oriented logs instead of raw terminal streaming
|
||||
|
||||
### Operational Notes
|
||||
|
||||
1. phase one should target backend-only restart
|
||||
2. frontend restart should remain out of scope initially
|
||||
3. command execution must always originate from repository root
|
||||
4. only fixed action names may cross the API boundary
|
||||
|
||||
## Validation Requirements
|
||||
|
||||
- Reject any action not present in the whitelist.
|
||||
- If a port-bearing action is added, validate the port as an integer in
|
||||
`1..65535`.
|
||||
- Resolve commands from the repository root so `planet.sh` runs with a stable
|
||||
working directory.
|
||||
- Record the requested action, operator identity, execution start time, and
|
||||
result.
|
||||
|
||||
## Implementation Guidance
|
||||
|
||||
- For UI-triggered restart flows, prefer `restart-backend` first.
|
||||
- Do not rely on the current API request process to stream full restart output
|
||||
after it triggers its own restart.
|
||||
- Use a task record plus polling/health-check recovery flow instead of raw
|
||||
terminal streaming as the primary UX.
|
||||
355
docs/technical/earth-bgp-context.md
Normal file
355
docs/technical/earth-bgp-context.md
Normal file
@@ -0,0 +1,355 @@
|
||||
# BGP Context
|
||||
|
||||
## Current Goal
|
||||
|
||||
The BGP module is being evolved from an anomaly-only demo into a layered observability pipeline:
|
||||
|
||||
`raw observations -> enrichment -> detectors -> incidents -> console/Earth visualization`
|
||||
|
||||
The practical product goal is no longer just to "show incidents on the globe". The current product objective is:
|
||||
|
||||
1. keep BGP visually present on Earth even when incident density is low
|
||||
2. make incidents clearly feel like a higher-confidence layer than anomalies
|
||||
3. show that the observation network is still active even when there are no active incidents
|
||||
|
||||
In practice, that means Earth should behave like an observability surface, not only an incident map:
|
||||
|
||||
- `collectors` show that observation is happening
|
||||
- `activity` shows where routing state is currently active or noisy
|
||||
- `incidents` become the highest-confidence focus layer
|
||||
|
||||
## Current Backend Architecture
|
||||
|
||||
### Data Layers
|
||||
|
||||
1. `BGPObservation`
|
||||
- File: `backend/app/models/bgp_observation.py`
|
||||
- Purpose: store normalized raw routing observations from live/history sources.
|
||||
- Typical fields:
|
||||
- `source`
|
||||
- `collector`
|
||||
- `peer_asn`
|
||||
- `peer_ip`
|
||||
- `prefix`
|
||||
- `event_type`
|
||||
- `as_path`
|
||||
- `origin_asn`
|
||||
- `next_hop`
|
||||
- `communities`
|
||||
- `observed_at`
|
||||
- `raw_payload`
|
||||
- `collector_geo`
|
||||
- `ingest_batch_id`
|
||||
|
||||
2. `BGPAnomaly`
|
||||
- File: `backend/app/models/bgp_anomaly.py`
|
||||
- Purpose: hold atomic detector outputs.
|
||||
- Current detector output types include:
|
||||
- `origin_change`
|
||||
- `more_specific_burst`
|
||||
- `mass_withdrawal`
|
||||
|
||||
3. `BGPIncident`
|
||||
- File: `backend/app/models/bgp_incident.py`
|
||||
- Purpose: aggregate atomic anomalies into incident-level objects for humans and the UI.
|
||||
|
||||
### Pipeline
|
||||
|
||||
Main flow is currently anchored in:
|
||||
|
||||
- `backend/app/services/collectors/bgp_common.py`
|
||||
- `backend/app/services/bgp_enrichment.py`
|
||||
- `backend/app/services/bgp_detectors.py`
|
||||
- `backend/app/services/bgp_incidents.py`
|
||||
|
||||
Operational flow:
|
||||
|
||||
1. collectors fetch raw BGP data
|
||||
2. `normalize_bgp_event()` standardizes payloads
|
||||
3. observations are persisted to `bgp_observations`
|
||||
4. enrichment augments events with analysis context
|
||||
5. detectors create `bgp_anomalies`
|
||||
6. incident aggregation rolls anomalies up into `bgp_incidents`
|
||||
|
||||
### Current Ingest Sources
|
||||
|
||||
1. `RIPE RIS Live`
|
||||
- Collector file: `backend/app/services/collectors/ris_live.py`
|
||||
- Used for realtime observation flow.
|
||||
|
||||
2. `CAIDA BGPStream Backfill`
|
||||
- Collector file: `backend/app/services/collectors/bgpstream.py`
|
||||
- Used as history/backfill entry point.
|
||||
|
||||
## Current Enrichment Status
|
||||
|
||||
Implemented enrichment skeleton in:
|
||||
|
||||
- `backend/app/services/bgp_enrichment.py`
|
||||
|
||||
Current enrichments:
|
||||
|
||||
- prefix family / prefix length
|
||||
- supernet / more-specific derivation
|
||||
- deduplicated AS path
|
||||
- path prepending hints
|
||||
- collector region info
|
||||
- prefix baseline hints
|
||||
- new-origin detection
|
||||
- ASN organization profile from PeeringDB where available
|
||||
- prefix scope / impacted region hints
|
||||
- prefix geography source priority:
|
||||
- `OpenGeoFeed` (override/high confidence)
|
||||
- `IPtoASN` (country-range baseline)
|
||||
- `NRO delegated stats` (registry-allocation fallback)
|
||||
|
||||
Current limitation:
|
||||
|
||||
- `RPKI` is still placeholder-only and returns `unknown`
|
||||
- no real ROA validation source is integrated yet
|
||||
- `inetnum` / `inet6num` whois fallback is still pending
|
||||
|
||||
## Current API Surface
|
||||
|
||||
Primary API file:
|
||||
|
||||
- `backend/app/api/v1/bgp.py`
|
||||
|
||||
Available endpoints:
|
||||
|
||||
- `/api/v1/bgp/events`
|
||||
- `/api/v1/bgp/events/summary`
|
||||
- `/api/v1/bgp/events/{id}`
|
||||
- `/api/v1/bgp/anomalies`
|
||||
- `/api/v1/bgp/anomalies/summary`
|
||||
- `/api/v1/bgp/anomalies/{id}`
|
||||
- `/api/v1/bgp/incidents`
|
||||
- `/api/v1/bgp/incidents/summary`
|
||||
- `/api/v1/bgp/incidents/{id}`
|
||||
|
||||
Visualization GeoJSON endpoints:
|
||||
|
||||
- `backend/app/api/v1/visualization.py`
|
||||
- `/api/v1/visualization/geo/bgp-collectors`
|
||||
- `/api/v1/visualization/geo/bgp-anomalies`
|
||||
- `/api/v1/visualization/geo/bgp-incidents`
|
||||
|
||||
## Current Earth Behavior
|
||||
|
||||
Relevant files:
|
||||
|
||||
- `frontend/public/earth/js/bgp.js`
|
||||
- `frontend/public/earth/js/main.js`
|
||||
- `frontend/public/earth/js/info-card.js`
|
||||
- `frontend/public/earth/js/constants.js`
|
||||
- `frontend/public/earth/index.html`
|
||||
|
||||
Current design:
|
||||
|
||||
1. Collectors are always shown when BGP is enabled.
|
||||
2. Incident markers are now the primary Earth BGP markers.
|
||||
3. If there are no incidents, Earth falls back to anomaly markers.
|
||||
4. If there are no anomalies either, collectors still provide presence.
|
||||
5. A dedicated `activity layer` now adds:
|
||||
- per-collector recent 15-minute activity halos
|
||||
- clustered regional activity hints derived from active collectors
|
||||
6. Incident markers now use:
|
||||
- symbol-driven event cores
|
||||
- outward ring pulses
|
||||
- reduced diffuse glow compared with older Earth builds
|
||||
5. The right-side stats now show:
|
||||
- BGP events
|
||||
- collector count
|
||||
- BGP status summary
|
||||
|
||||
This is directionally correct, but still incomplete for low-event-density periods. Right now Earth can still feel too quiet when incidents are sparse because the system lacks a dedicated `activity layer` between raw observation and incident focus.
|
||||
|
||||
Current BGP status strategy:
|
||||
|
||||
- incidents present: show active incident count
|
||||
- no incidents but anomalies present: show active anomaly count, plus active observation regions when available
|
||||
- no incidents/anomalies but activity present: show `观测网络运行中`
|
||||
- no incidents/anomalies but collectors present: show `观测网络运行中 · 当前未发现聚合级事件`
|
||||
- no BGP data at all: show `暂无观测数据`
|
||||
|
||||
Earth info-card strategy:
|
||||
|
||||
- `bgp` card is now incident-centric in wording
|
||||
- `bgp_collector` card shows collector location and current event count
|
||||
|
||||
## Current Product Gap
|
||||
|
||||
The main product gap is not architecture correctness. It is low-density visualization strategy.
|
||||
|
||||
Current reality:
|
||||
|
||||
- incident count is naturally much lower than anomaly count
|
||||
- that is expected, because incidents are aggregated and de-noised
|
||||
- but incident-first rendering makes the Earth view look too quiet unless there is another always-available activity layer
|
||||
|
||||
Implementation detail for the recommended `activity layer` is expanded in [bgp-region-aggregation-plan.md](/home/ray/dev/linkong/planet/docs/plans/earth-bgp-region-aggregation-plan.md).
|
||||
|
||||
So the immediate next milestone is:
|
||||
|
||||
`event map -> observability map`
|
||||
|
||||
That means Earth needs three simultaneously readable layers:
|
||||
|
||||
1. `observation layer`
|
||||
- collectors
|
||||
- recent collector activity
|
||||
- baseline coverage
|
||||
2. `activity layer`
|
||||
- recent event density
|
||||
- anomaly/noise hotspots
|
||||
- regional activity scoring
|
||||
- incident presence bonus
|
||||
3. `incident layer`
|
||||
- sparse but highly legible, high-confidence event objects
|
||||
- symbol-driven markers
|
||||
- outward ring pulse instead of broad diffuse glow
|
||||
|
||||
## Incident Visual Direction
|
||||
|
||||
The Earth `incident` layer should not read like a large glowing patch. It should read like a compact, high-confidence event focus.
|
||||
|
||||
Design principles:
|
||||
|
||||
1. `incident` markers should use a strong primary symbol
|
||||
- the symbol shape should carry type meaning where possible
|
||||
- examples:
|
||||
- `origin_change`: triangle-like warning marker
|
||||
- `mass_withdrawal`: alert/exclamation-style marker
|
||||
- `more_specific_burst`: split/radiating marker
|
||||
|
||||
2. emphasis should come from outward ring pulses, not area flooding
|
||||
- use a compact hot core
|
||||
- use one or more expanding ring pulses
|
||||
- avoid broad luminous blobs that make the event center feel vague
|
||||
|
||||
3. `collector` and `incident` must stay visually distinct
|
||||
- collectors are observation infrastructure
|
||||
- incidents are extracted event focus
|
||||
- collector activity should stay quieter than incident pulse language
|
||||
|
||||
4. calm periods still need observability presence
|
||||
- collectors and activity layers should keep the map alive
|
||||
- once incidents appear, they should clearly dominate nearby BGP visuals
|
||||
|
||||
5. incident geography should become `prefix-centric`
|
||||
- collectors should remain evidence sources, not the primary event location
|
||||
- preferred geography priority:
|
||||
- `prefix_geography`
|
||||
- `prefix_scope`
|
||||
- `ASN organization region`
|
||||
- `collector centroid` as final fallback
|
||||
- `prefix_scope` should remain an observation-derived scope hint
|
||||
- a new `prefix_geography` layer should be introduced for actual prefix-centric placement
|
||||
|
||||
Reference inspiration:
|
||||
|
||||
- `World Monitor`
|
||||
- sparse event symbols
|
||||
- compact centers
|
||||
- ring-like outward pulses
|
||||
- stronger incident legibility than diffuse glow
|
||||
|
||||
## Current Console Behavior
|
||||
|
||||
Relevant page:
|
||||
|
||||
- `frontend/src/pages/BGP/BGP.tsx`
|
||||
|
||||
Current BGP console page has three levels:
|
||||
|
||||
1. observation summary
|
||||
- total events
|
||||
- collector count
|
||||
- prefix count
|
||||
|
||||
2. incident summary and incident table
|
||||
|
||||
3. anomaly detail table plus recent observation events
|
||||
|
||||
This means the BGP page still has useful signal even when there are zero anomalies.
|
||||
|
||||
## Known Product/Engineering Boundaries
|
||||
|
||||
1. The current system is still closer to an event board than a full BGP sensing platform.
|
||||
2. RIS coverage still needs to expand beyond narrow subscription scope.
|
||||
3. BGPStream history is still not full MRT-to-prefix decoded analytics.
|
||||
4. Collector geography still depends heavily on static RIPE RIS mappings.
|
||||
5. Incident-to-cable/IXP/region association is still weak and early-stage.
|
||||
6. Earth currently visualizes logical observation/impact structure, not true physical traffic paths.
|
||||
|
||||
## Test Status
|
||||
|
||||
BGP-specific tests live in:
|
||||
|
||||
- `backend/tests/test_bgp.py`
|
||||
|
||||
Verified status at this point:
|
||||
|
||||
- `25 passed` for `backend/tests/test_bgp.py`
|
||||
- `62 passed` for `backend/tests`
|
||||
|
||||
Covered areas include:
|
||||
|
||||
- normalization
|
||||
- observation serialization
|
||||
- enrichment
|
||||
- detectors, including route leak candidate and path flap
|
||||
- incident aggregation
|
||||
- batch anomaly creation
|
||||
- BGP events/incidents API
|
||||
- summary endpoints
|
||||
|
||||
## Most Relevant Files
|
||||
|
||||
Backend:
|
||||
|
||||
- `backend/app/models/bgp_observation.py`
|
||||
- `backend/app/models/bgp_anomaly.py`
|
||||
- `backend/app/models/bgp_incident.py`
|
||||
- `backend/app/services/collectors/bgp_common.py`
|
||||
- `backend/app/services/bgp_enrichment.py`
|
||||
- `backend/app/services/bgp_detectors.py`
|
||||
- `backend/app/services/bgp_incidents.py`
|
||||
- `backend/app/api/v1/bgp.py`
|
||||
- `backend/app/api/v1/visualization.py`
|
||||
|
||||
Frontend:
|
||||
|
||||
- `frontend/src/pages/BGP/BGP.tsx`
|
||||
- `frontend/public/earth/js/bgp.js`
|
||||
- `frontend/public/earth/js/main.js`
|
||||
- `frontend/public/earth/js/info-card.js`
|
||||
- `frontend/public/earth/js/constants.js`
|
||||
- `frontend/public/earth/index.html`
|
||||
|
||||
## Recommended Next Steps
|
||||
|
||||
### Next Backend / Detection Priority
|
||||
|
||||
1. Integrate real RPKI validation data.
|
||||
2. Expand realtime collector coverage and include withdrawals more broadly.
|
||||
3. Continue refining route leak and path instability detectors with stronger heuristics.
|
||||
|
||||
### Next Correlation / Storytelling Priority
|
||||
|
||||
4. Strengthen incident aggregation semantics and titles.
|
||||
5. Add weak correlation from incidents to:
|
||||
- cable corridors
|
||||
- landing points
|
||||
- IXPs
|
||||
- other traffic anomaly sources
|
||||
6. Refine Earth hover/click handoff between collectors and incidents.
|
||||
|
||||
### Next Visualization Priority
|
||||
|
||||
7. Refine regional activity scoring so the activity layer is informative without becoming noisy.
|
||||
8. Add more incident symbol types as new detectors land.
|
||||
9. Add a real prefix geography source:
|
||||
- `IPtoASN / IPtoCountry` as the first practical dataset
|
||||
- `OpenGeoFeed` as a higher-quality override layer
|
||||
- registry/whois only as fallback
|
||||
263
docs/technical/earth-frontend-context.md
Normal file
263
docs/technical/earth-frontend-context.md
Normal file
@@ -0,0 +1,263 @@
|
||||
# Earth Frontend Context
|
||||
|
||||
本文件描述当前 Earth 大屏前端的真实结构，重点是帮助后续继续改 HUD、图层、媒体面板、真实地形、BGP 可视化时，不再重复踩结构和状态同步上的坑。
|
||||
|
||||
相关规则建议一起参考：
|
||||
|
||||
- [rules.md](/home/ray/dev/linkong/planet/rules.md)
|
||||
- [frontend-layout-guidelines.md](/home/ray/dev/linkong/planet/docs/technical/frontend-layout-guidelines.md)
|
||||
|
||||
## 当前目标
|
||||
|
||||
Earth 前端不是普通管理页，它是独立的大屏展示前端。当前产品目标是：
|
||||
|
||||
- 维持地球视图的空间感和可读性
|
||||
- 让 HUD、图层、媒体面板、BGP、卫星、海缆等保持统一交互
|
||||
- 把加载中、已启用、已隐藏、锁定中这类状态做清楚
|
||||
|
||||
## 当前入口
|
||||
|
||||
React 路由入口：
|
||||
|
||||
- [Earth.tsx](/home/ray/dev/linkong/planet/frontend/src/pages/Earth/Earth.tsx)
|
||||
|
||||
当前做法很简单：
|
||||
|
||||
- React 页面只负责提供一个全屏 `iframe`
|
||||
- 真正的 Earth 应用运行在：
|
||||
- [index.html](/home/ray/dev/linkong/planet/frontend/public/earth/index.html)
|
||||
|
||||
所以 Earth 前端本质上是 `public/earth` 下的一套独立静态应用。
|
||||
|
||||
## 当前文件分层
|
||||
|
||||
### 1. 页面入口与结构
|
||||
|
||||
- [index.html](/home/ray/dev/linkong/planet/frontend/public/earth/index.html)
|
||||
|
||||
职责：
|
||||
|
||||
- HUD 基础 DOM
|
||||
- 图层面板
|
||||
- 媒体面板
|
||||
- 工具栏
|
||||
- 设置弹窗
|
||||
- 兼容旧元素 id
|
||||
|
||||
### 2. 主运行时
|
||||
|
||||
- [main.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/main.js)
|
||||
|
||||
职责：
|
||||
|
||||
- 地球初始化
|
||||
- Three.js 场景组装
|
||||
- 数据加载与刷新
|
||||
- 各图层集成
|
||||
- Earth 级别状态同步
|
||||
|
||||
### 3. 地球控制层
|
||||
|
||||
- [controls.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/controls.js)
|
||||
|
||||
职责：
|
||||
|
||||
- 工具栏交互
|
||||
- 图层面板交互
|
||||
- 旋转/缩放/布局
|
||||
- HUD 面板拖拽
|
||||
- 图层开关状态机
|
||||
|
||||
这份文件是 Earth 前端当前最核心的 UI 控制入口。
|
||||
|
||||
### 4. UI 与状态消息
|
||||
|
||||
- [ui.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/ui.js)
|
||||
|
||||
职责：
|
||||
|
||||
- loading 面板
|
||||
- status message
|
||||
- tooltip / error / 清理逻辑
|
||||
|
||||
### 5. 地球与地形
|
||||
|
||||
- [earth.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/earth.js)
|
||||
- [terrain.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/terrain.js)
|
||||
|
||||
职责：
|
||||
|
||||
- 地球球体、云层、大气
|
||||
- 真实地形 mesh
|
||||
- terrain tile 拉取、解码、位移、着色
|
||||
|
||||
### 6. 图层模块
|
||||
|
||||
- [satellites.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/satellites.js)
|
||||
- [cables.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/cables.js)
|
||||
- [bgp.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/bgp.js)
|
||||
- [news.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/news.js)
|
||||
- [tv.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/tv.js)
|
||||
|
||||
职责：
|
||||
|
||||
- 各自的数据层
|
||||
- 开关行为
|
||||
- 面板内容
|
||||
- hover/lock/selection 语义
|
||||
|
||||
## 当前样式分层
|
||||
|
||||
Earth 的 CSS 不是一份大样式表，而是分层管理：
|
||||
|
||||
- [base.css](/home/ray/dev/linkong/planet/frontend/public/earth/css/base.css)
|
||||
- [hud.css](/home/ray/dev/linkong/planet/frontend/public/earth/css/hud.css)
|
||||
- [toolbar.css](/home/ray/dev/linkong/planet/frontend/public/earth/css/toolbar.css)
|
||||
- [layer-panel.css](/home/ray/dev/linkong/planet/frontend/public/earth/css/layer-panel.css)
|
||||
- [info-panel.css](/home/ray/dev/linkong/planet/frontend/public/earth/css/info-panel.css)
|
||||
- [legend.css](/home/ray/dev/linkong/planet/frontend/public/earth/css/legend.css)
|
||||
- [earth-stats.css](/home/ray/dev/linkong/planet/frontend/public/earth/css/earth-stats.css)
|
||||
- [coordinates-display.css](/home/ray/dev/linkong/planet/frontend/public/earth/css/coordinates-display.css)
|
||||
- [tv-panel.css](/home/ray/dev/linkong/planet/frontend/public/earth/css/tv-panel.css)
|
||||
|
||||
当前建议：
|
||||
|
||||
- 通用 HUD 壳层写进 `hud.css`
|
||||
- 单一面板特性写进各自子文件
|
||||
- 不要把业务状态样式再散回 `index.html`
|
||||
|
||||
## 当前图层开关状态语义
|
||||
|
||||
Earth 图层按钮现在不应再只有“开/关”两态，而应支持：
|
||||
|
||||
- `inactive`
|
||||
- `active`
|
||||
- `loading`
|
||||
|
||||
当前入口在：
|
||||
|
||||
- [controls.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/controls.js)
|
||||
- [layer-button-state.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/layer-button-state.js)
|
||||
|
||||
关键函数：
|
||||
|
||||
- `updateLayerButtonState(button, isActive)`
|
||||
- `setLayerButtonState(button, options)`
|
||||
|
||||
`setLayerButtonState` 负责：
|
||||
|
||||
- `loading` 样式
|
||||
- `aria-busy`
|
||||
- 按钮禁用
|
||||
- tooltip 更新
|
||||
- 绑定状态文本更新
|
||||
- 可选同步 `active`
|
||||
|
||||
因此后续如果别的图层也需要异步启用，应该直接走这套状态机，而不是再手写一套临时 loading class。
|
||||
|
||||
### `data-status-target`
|
||||
|
||||
图层按钮可以通过：
|
||||
|
||||
- `data-status-target`
|
||||
|
||||
指向一个状态文本节点。当前 terrain 已接入：
|
||||
|
||||
- 按钮：`#toggle-terrain`
|
||||
- 状态节点：`#terrain-status`
|
||||
|
||||
以后别的异步图层也可以沿用这套约定。
|
||||
|
||||
## 当前地形链路
|
||||
|
||||
真实地形首次启用会慢，原因不只是一个：
|
||||
|
||||
1. 需要拉取 Terrarium 瓦片
|
||||
2. 需要解码图片
|
||||
3. 需要按顶点采样高程
|
||||
4. 需要重新写入 geometry 和 color
|
||||
5. 需要重新计算法线与包围体
|
||||
|
||||
当前入口在：
|
||||
|
||||
- [terrain.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/terrain.js)
|
||||
|
||||
当前已经做了两层体验优化：
|
||||
|
||||
1. 图层开关 loading 状态持续可见
|
||||
2. 页面空闲时会预热 `ensureTerrainReady()`
|
||||
|
||||
也就是说，后续再继续优化 terrain 时，优先顺序应该是：
|
||||
|
||||
1. 先保证用户感知正确
|
||||
2. 再压缩首次等待
|
||||
3. 最后才做更激进的几何/瓦片优化
|
||||
|
||||
## 当前高频风险点
|
||||
|
||||
### 1. 视觉状态和业务状态不同步
|
||||
|
||||
Earth 里最常见的 bug 不是“没渲染”，而是：
|
||||
|
||||
- 图层关了，tooltip 还在
|
||||
- 锁定对象隐藏了，info card 还在
|
||||
- legend 没跟图层切换
|
||||
- loading 已结束，但按钮还像没开
|
||||
|
||||
后续改动必须优先检查状态同步。
|
||||
|
||||
### 2. HUD 布局问题先查结构，不要先打 CSS 补丁
|
||||
|
||||
Earth HUD 历史上反复出现：
|
||||
|
||||
- 面板只剩一条缝
|
||||
- markdown 被裁掉
|
||||
- tabs/iframe 被 `overflow: hidden` 吃掉
|
||||
|
||||
优先检查：
|
||||
|
||||
1. 谁负责高度
|
||||
2. 谁负责滚动
|
||||
3. 哪一层在裁剪
|
||||
|
||||
不要上来先加 `overflow: hidden` 或额外包装层。
|
||||
|
||||
### 3. Transitional path 必须收口
|
||||
|
||||
Earth 已经经历过多轮 HUD、toolbar、media panel 重构，所以最容易积累：
|
||||
|
||||
- 旧 helper
|
||||
- 旧 class
|
||||
- 旧 fallback 逻辑
|
||||
- 已废弃变体
|
||||
|
||||
每次大功能完成后，都要做一次 cleanup pass。
|
||||
|
||||
## 当前推荐改动方式
|
||||
|
||||
如果后续继续改 Earth，建议按这个顺序：
|
||||
|
||||
1. 先确认改的是：
|
||||
- Three.js 渲染层
|
||||
- HUD 结构层
|
||||
- 图层状态层
|
||||
- 面板内容层
|
||||
2. 如果涉及图层按钮，优先接入统一状态机
|
||||
3. 如果涉及可见性切换，检查 tooltip / legend / info-card / lock 是否一起收口
|
||||
4. 如果涉及面板布局，先查结构再动 CSS
|
||||
|
||||
## 当前与控制台前端的边界
|
||||
|
||||
Earth 前端和控制台前端不是同一套 UI 系统：
|
||||
|
||||
- 控制台前端：React + Ant Design 工作台
|
||||
- Earth 前端：`public/earth` 原生 HUD + Three.js 展示面
|
||||
|
||||
因此：
|
||||
|
||||
- Earth 不应该直接复用 Ant Table / AppLayout 语义
|
||||
- 控制台也不应该照搬 Earth HUD 动画和玻璃层语言
|
||||
|
||||
控制台相关结构见：
|
||||
|
||||
- [admin-frontend-context.md](/home/ray/dev/linkong/planet/docs/technical/frontend-admin-frontend-context.md)
|
||||
97
docs/technical/earth-news-live-streams-collector-format.md
Normal file
97
docs/technical/earth-news-live-streams-collector-format.md
Normal file
@@ -0,0 +1,97 @@
|
||||
# News Live Streams Collector Format
|
||||
|
||||
`news_live_streams` 采集器面向“频道目录 JSON”输入，而不是直接抓网页。
|
||||
|
||||
这样做的目标是：
|
||||
|
||||
- 让后台能够稳定接入世界各地新闻直播源
|
||||
- 让 `Earth` 页面电视模块始终消费统一结构
|
||||
- 便于后续接入类似 `worldmonitor` 那种 YouTube / HLS / iframe 混合频道目录
|
||||
|
||||
## 推荐 JSON 结构
|
||||
|
||||
```json
|
||||
{
|
||||
"sources": [
|
||||
{
|
||||
"id": "bbc-world-news",
|
||||
"name": "BBC World News",
|
||||
"provider": "BBC",
|
||||
"region": "UK",
|
||||
"language": "en",
|
||||
"source_type": "youtube",
|
||||
"youtube_video_id": "dQw4w9WgXcQ",
|
||||
"youtube_channel": "https://www.youtube.com/@BBCNews",
|
||||
"embed_url": "",
|
||||
"stream_url": "",
|
||||
"homepage_url": "https://www.youtube.com/@BBCNews/live",
|
||||
"poster_url": "",
|
||||
"sort_order": 220,
|
||||
"is_enabled": true,
|
||||
"notes": "Primary English global news channel"
|
||||
},
|
||||
{
|
||||
"id": "france24-en",
|
||||
"name": "France 24 English",
|
||||
"provider": "France 24",
|
||||
"region": "France",
|
||||
"language": "en",
|
||||
"source_type": "hls",
|
||||
"stream_url": "https://example.com/live.m3u8",
|
||||
"homepage_url": "https://www.france24.com/en/live",
|
||||
"sort_order": 230,
|
||||
"is_enabled": true
|
||||
},
|
||||
{
|
||||
"id": "cctv4-page",
|
||||
"name": "CCTV-4 中文国际",
|
||||
"provider": "CCTV",
|
||||
"region": "China",
|
||||
"language": "zh-CN",
|
||||
"source_type": "iframe",
|
||||
"embed_url": "https://tv.cctv.com/live/cctv4/",
|
||||
"homepage_url": "https://tv.cctv.com/live/cctv4/",
|
||||
"sort_order": 10,
|
||||
"is_enabled": true
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
## 字段约定
|
||||
|
||||
- `id`: 唯一标识，建议稳定不变
|
||||
- `name`: 频道显示名
|
||||
- `provider`: 提供方
|
||||
- `region`: 国家或地区
|
||||
- `language`: 语言代码
|
||||
- `source_type`: `iframe` / `hls` / `video` / `external` / `youtube`
|
||||
- `embed_url`: 适合 iframe 内嵌的页面
|
||||
- `stream_url`: 直接视频流地址
|
||||
- `homepage_url`: 官网或频道页
|
||||
- `youtube_video_id`: YouTube 直播视频 ID
|
||||
- `youtube_channel`: YouTube 频道 handle 或频道 URL
|
||||
- `poster_url`: 封面图，可选
|
||||
- `sort_order`: 排序值，越小越靠前
|
||||
- `is_enabled`: 是否启用
|
||||
- `notes`: 简短备注
|
||||
|
||||
## 面板行为约定
|
||||
|
||||
- `youtube`
|
||||
- 优先使用 `youtube_video_id`
|
||||
- 无法内嵌时至少保留 `youtube_channel` 或 `homepage_url` 供外部打开
|
||||
- `hls` / `video`
|
||||
- 优先走 `stream_url`
|
||||
- `iframe`
|
||||
- 优先走 `embed_url`
|
||||
- `external`
|
||||
- 不尝试内嵌，只保留外部打开
|
||||
|
||||
## 当前实现状态
|
||||
|
||||
- 后台设置页可以手工维护频道目录
|
||||
- `Earth` 电视模块会合并：
|
||||
- 手工配置源
|
||||
- `news_live_streams` 采集器采集源
|
||||
- 当前默认兜底源为 `CCTV-4 中文国际`
|
||||
236
docs/technical/frontend-admin-frontend-context.md
Normal file
236
docs/technical/frontend-admin-frontend-context.md
Normal file
@@ -0,0 +1,236 @@
|
||||
# Admin Frontend Context
|
||||
|
||||
本文件描述当前控制台前端的真实结构，目标是帮助后续页面开发、表格改造、布局治理和状态收口时快速找到正确入口。
|
||||
|
||||
相关规则建议一起参考：
|
||||
|
||||
- [rules.md](/home/ray/dev/linkong/planet/rules.md)
|
||||
- [frontend-layout-guidelines.md](/home/ray/dev/linkong/planet/docs/technical/frontend-layout-guidelines.md)
|
||||
|
||||
## 当前目标
|
||||
|
||||
控制台前端承担的是后台工作台，而不是展示型大屏。当前约束是：
|
||||
|
||||
- 页面默认遵循单屏工作区
|
||||
- 主交互在内部模块滚动，而不是依赖整页无限变长
|
||||
- 列表、表格、分析页优先保证主工作区可见
|
||||
- 通用布局、滚动条、表格滚动行为尽量复用，不要每页各写一套
|
||||
|
||||
## 当前路由入口
|
||||
|
||||
主入口在：
|
||||
|
||||
- [App.tsx](/home/ray/dev/linkong/planet/frontend/src/App.tsx)
|
||||
|
||||
当前后台相关路由包括：
|
||||
|
||||
- `/admin`
|
||||
- `/users`
|
||||
- `/datasources`
|
||||
- `/data`
|
||||
- `/alerts/system`
|
||||
- `/alerts/bgp`
|
||||
- `/alerts/situational`
|
||||
- `/bgp`
|
||||
- `/playground`
|
||||
- `/settings`
|
||||
|
||||
`/earth` 是独立展示页，不属于控制台骨架。
|
||||
|
||||
## 当前页面骨架
|
||||
|
||||
控制台公共壳层在：
|
||||
|
||||
- [AppLayout.tsx](/home/ray/dev/linkong/planet/frontend/src/components/AppLayout/AppLayout.tsx)
|
||||
|
||||
职责：
|
||||
|
||||
- 左侧导航
|
||||
- 折叠与展开
|
||||
- 当前账号/版本信息
|
||||
- 内容区高度闭合
|
||||
- 全站统一侧边栏滚动条
|
||||
|
||||
当前结构是：
|
||||
|
||||
```tsx
|
||||
<Layout className="dashboard-layout">
|
||||
<Sider className="dashboard-sider">...</Sider>
|
||||
<Layout>
|
||||
<Content className="dashboard-content">
|
||||
<div className="dashboard-content-inner">{children}</div>
|
||||
</Content>
|
||||
</Layout>
|
||||
</Layout>
|
||||
```
|
||||
|
||||
后续控制台页面应优先适配这套壳层，而不是重新定义全页高度语义。
|
||||
|
||||
## 当前共享组件
|
||||
|
||||
### 1. `Scrollbar`
|
||||
|
||||
文件：
|
||||
|
||||
- [Scrollbar.tsx](/home/ray/dev/linkong/planet/frontend/src/components/Scrollbar/Scrollbar.tsx)
|
||||
|
||||
用途：
|
||||
|
||||
- 控制台侧边栏这类普通内容容器
|
||||
- 组件内部管理可见性、thumb 尺寸、拖拽和双轴 overflow 判定
|
||||
|
||||
当前约束：
|
||||
|
||||
- 滚动条必须是浮层，不参与布局
|
||||
- 无 overflow 时不应留下可见痕迹
|
||||
- 真实滚动仍交给原生容器，只替换可见层和交互层
|
||||
|
||||
### 2. `ScrollbarOverlay`
|
||||
|
||||
文件：
|
||||
|
||||
- [ScrollbarOverlay.tsx](/home/ray/dev/linkong/planet/frontend/src/components/Scrollbar/ScrollbarOverlay.tsx)
|
||||
|
||||
用途：
|
||||
|
||||
- Ant Table 这类内部已有滚动容器的区域
|
||||
- 不接管滚动语义，只叠加新的滚动条可见层
|
||||
|
||||
当前使用场景：
|
||||
|
||||
- 数据源
|
||||
- 采集数据
|
||||
- 用户管理
|
||||
- 设置页
|
||||
- 告警页
|
||||
- BGP 页面
|
||||
|
||||
### 3. `TableScrollRegion`
|
||||
|
||||
文件：
|
||||
|
||||
- [TableScrollRegion.tsx](/home/ray/dev/linkong/planet/frontend/src/components/Scrollbar/TableScrollRegion.tsx)
|
||||
|
||||
用途：
|
||||
|
||||
- 为表格滚动区提供统一包裹层
|
||||
- 后续新表格页优先复用，不要重复写“表格区域 + overlay scrollbar”样板
|
||||
|
||||
### 4. 其他共享组件
|
||||
|
||||
- [MarkdownRenderer.tsx](/home/ray/dev/linkong/planet/frontend/src/components/MarkdownRenderer/MarkdownRenderer.tsx)
|
||||
- [TableActions.tsx](/home/ray/dev/linkong/planet/frontend/src/components/TableActions/TableActions.tsx)
|
||||
|
||||
## 当前状态来源
|
||||
|
||||
### 1. 认证状态
|
||||
|
||||
文件：
|
||||
|
||||
- [auth.ts](/home/ray/dev/linkong/planet/frontend/src/stores/auth.ts)
|
||||
|
||||
职责：
|
||||
|
||||
- token
|
||||
- 当前用户
|
||||
- 登录/退出
|
||||
|
||||
`App.tsx` 用它判断是否进入登录页。
|
||||
|
||||
### 2. 业务数据网关
|
||||
|
||||
目前 AI / 态势感知相关服务集中在：
|
||||
|
||||
- [http-gateway.ts](/home/ray/dev/linkong/planet/frontend/src/services/situational-awareness/http-gateway.ts)
|
||||
- [port.ts](/home/ray/dev/linkong/planet/frontend/src/services/situational-awareness/port.ts)
|
||||
- [types.ts](/home/ray/dev/linkong/planet/frontend/src/services/situational-awareness/types.ts)
|
||||
|
||||
约束：
|
||||
|
||||
- 页面不要直接散落拼 URL
|
||||
- 先通过 port/types 定义边界
|
||||
- 再由 http/mock gateway 实现
|
||||
|
||||
## 当前页面分层建议
|
||||
|
||||
### 1. 仪表盘和摘要型页面
|
||||
|
||||
例如：
|
||||
|
||||
- [Dashboard.tsx](/home/ray/dev/linkong/planet/frontend/src/pages/Dashboard/Dashboard.tsx)
|
||||
|
||||
优先目标：
|
||||
|
||||
- 页头稳定
|
||||
- 摘要卡片先紧凑化
|
||||
- 主工作区占据主要高度
|
||||
|
||||
### 2. 表格型页面
|
||||
|
||||
例如：
|
||||
|
||||
- [DataSources.tsx](/home/ray/dev/linkong/planet/frontend/src/pages/DataSources/DataSources.tsx)
|
||||
- [DataList.tsx](/home/ray/dev/linkong/planet/frontend/src/pages/DataList/DataList.tsx)
|
||||
- [Users.tsx](/home/ray/dev/linkong/planet/frontend/src/pages/Users/Users.tsx)
|
||||
- [Settings.tsx](/home/ray/dev/linkong/planet/frontend/src/pages/Settings/Settings.tsx)
|
||||
|
||||
约束：
|
||||
|
||||
- 优先内部滚动
|
||||
- 不要让表格撑爆整页
|
||||
- 新表格区域优先复用 `TableScrollRegion` / `ScrollbarOverlay`
|
||||
|
||||
### 3. 复杂工作区页面
|
||||
|
||||
例如：
|
||||
|
||||
- [BGP.tsx](/home/ray/dev/linkong/planet/frontend/src/pages/BGP/BGP.tsx)
|
||||
- [Playground.tsx](/home/ray/dev/linkong/planet/frontend/src/pages/Playground/Playground.tsx)
|
||||
|
||||
约束：
|
||||
|
||||
- Tabs 里的内容不能套同一套高度逻辑
|
||||
- 表格 tab、Markdown tab、配置 tab 要各自定义滚动责任
|
||||
- AI 结果区、长文本区优先保证最小可读高度
|
||||
|
||||
## 当前布局约束
|
||||
|
||||
这些原则已经在项目里反复验证过：
|
||||
|
||||
1. 父容器高度链要闭合
|
||||
2. `min-height: 0` 不能漏
|
||||
3. overflow 责任必须明确
|
||||
4. 不要用 `overflow: hidden` 掩盖结构问题
|
||||
5. 不要为了摘要卡完整显示去压缩主工作区
|
||||
6. 自定义滚动条必须是浮层，不得挤压内容宽度
|
||||
|
||||
详细经验见：
|
||||
|
||||
- [frontend-layout-guidelines.md](/home/ray/dev/linkong/planet/docs/technical/frontend-layout-guidelines.md)
|
||||
|
||||
## 当前推荐改动方式
|
||||
|
||||
如果后续继续改后台页面，建议按这个顺序：
|
||||
|
||||
1. 先确认页面属于摘要页、表格页还是复杂工作区
|
||||
2. 先接入现有壳层和滚动语义
|
||||
3. 优先复用共享滚动组件
|
||||
4. 最后再改视觉和细节交互
|
||||
|
||||
不要先写局部 CSS 补丁，再回头补结构。
|
||||
|
||||
## 当前明显边界
|
||||
|
||||
控制台前端和 Earth 前端不是一套系统：
|
||||
|
||||
- 控制台前端是 React + Ant Design 工作台
|
||||
- Earth 前端是 `public/earth` 下的独立原生 HUD 系统
|
||||
|
||||
因此：
|
||||
|
||||
- 不要把 Earth 的 HUD/动画/状态机直接挪进控制台
|
||||
- 不要把控制台表格/滚动策略硬套到 Earth HUD
|
||||
|
||||
Earth 相关结构见：
|
||||
|
||||
- [earth-frontend-context.md](/home/ray/dev/linkong/planet/docs/technical/earth-frontend-context.md)
|
||||
309
docs/technical/frontend-layout-guidelines.md
Normal file
309
docs/technical/frontend-layout-guidelines.md
Normal file
@@ -0,0 +1,309 @@
|
||||
# Frontend Layout Guidelines
|
||||
|
||||
本项目后台页面默认遵循“单屏工作区”布局规范。目标不是让页面永远不溢出，而是确保在常见桌面视口下：
|
||||
|
||||
- 页面主结构能在一屏内看清
|
||||
- 用户能同时看到页头、摘要区和主工作区
|
||||
- 超出的内容在模块内部滚动，而不是把整页纵向撑爆
|
||||
|
||||
当前推荐参考实现：
|
||||
|
||||
- [frontend/src/pages/BGP/BGP.tsx](/home/ray/dev/linkong/planet/frontend/src/pages/BGP/BGP.tsx)
|
||||
- [frontend/src/index.css](/home/ray/dev/linkong/planet/frontend/src/index.css)
|
||||
|
||||
## 核心原则
|
||||
|
||||
### 1. 页面优先保证一屏工作区
|
||||
|
||||
管理页默认采用：
|
||||
|
||||
- 页头：标题、说明、主要操作
|
||||
- 主工作区：统计卡、表格、图表、列表、标签页
|
||||
|
||||
推荐结构：
|
||||
|
||||
```tsx
|
||||
<AppLayout>
|
||||
<div className="page-shell">
|
||||
<div className="page-shell__header">...</div>
|
||||
<div className="page-shell__body">...</div>
|
||||
</div>
|
||||
</AppLayout>
|
||||
```
|
||||
|
||||
页面总高度应被限制在 `AppLayout` 内容区内，而不是继续让整个页面自然向下增长。
|
||||
|
||||
### 2. 滚动优先发生在模块内部
|
||||
|
||||
如果表格、日志、长列表、图表明细超出空间：
|
||||
|
||||
- 让卡片内部滚动
|
||||
- 让表格内部滚动
|
||||
- 让标签页内容区内部滚动
|
||||
|
||||
不要默认依赖整个页面滚动去“解决”空间问题。
|
||||
|
||||
### 3. 主工作区必须拿到主要空间
|
||||
|
||||
页面里最重要的模块必须是视觉和空间上的主角。通常应保证：
|
||||
|
||||
- 页头始终可见
|
||||
- 摘要区高度被控制
|
||||
- 主表格 / 主图表 / 主分析区占据 50% 以上可视高度
|
||||
|
||||
如果一个页面有多个大模块，优先顺序是：
|
||||
|
||||
1. 先压缩说明区和摘要区
|
||||
2. 再把次级模块收进标签页或切换视图
|
||||
3. 最后才考虑继续增加整页滚动
|
||||
|
||||
### 4. 小屏幕和高缩放必须进入紧凑模式
|
||||
|
||||
在窗口高度较低、宽度较窄、或系统缩放较高时，应主动切换紧凑布局，例如：
|
||||
|
||||
- 缩小卡片 padding
|
||||
- 缩小表头和单元格间距
|
||||
- 将摘要区改为更紧凑的单行/横向滚动布局
|
||||
- 将次级模块移入标签页、抽屉、折叠区
|
||||
|
||||
紧凑模式的目标是保持可用，不是单纯把文字和控件一股脑缩小。
|
||||
|
||||
### 5. overflow 责任必须明确
|
||||
|
||||
页面中的大块内容必须明确：
|
||||
|
||||
- 谁负责占满剩余高度
|
||||
- 谁负责裁剪
|
||||
- 谁负责滚动
|
||||
|
||||
常见要求：
|
||||
|
||||
- 父容器链路需要 `min-height: 0`
|
||||
- 工作区容器通常需要 `display: flex`
|
||||
- 真正的滚动节点要显式 `overflow: auto`
|
||||
|
||||
### 6. 卡片不能被压到不可读
|
||||
|
||||
历史上我们反复踩到的问题不是“没有滚动条”，而是：
|
||||
|
||||
- 卡片被 `flex` 压缩得只剩一小条可视区域
|
||||
- 文字能渲染，但读不完整
|
||||
- 内容其实存在，却被 `overflow: hidden` 裁掉
|
||||
|
||||
因此后续约束是：
|
||||
|
||||
- 先保证卡片有可读的最小高度
|
||||
- 如果继续压缩会影响阅读，就切换成内部滚动
|
||||
- 不要为了“保持一屏”而把正文、表格、描述区压成无法阅读的条状区域
|
||||
|
||||
### 7. Tabs 不是天然安全的布局容器
|
||||
|
||||
历史上 Tabs 相关回归非常多，典型问题包括：
|
||||
|
||||
- 隐藏 tab pane 因为自定义 `display: flex` 而重新露出来
|
||||
- 所有 tab 被强行套用同一套高度/overflow 规则
|
||||
- 表格 tab 能工作，但 markdown / help / diagnostics tab 被压坏
|
||||
|
||||
因此约束是：
|
||||
|
||||
- `Tabs` 里的每类内容都要单独定义自己的布局策略
|
||||
- 表格 tab 可以是“固定高度 + 内部滚动”
|
||||
- 文档/Markdown tab 更适合“tab pane 自身滚动 + 内容正常文档流”
|
||||
- 如果覆盖组件库样式，必须同时检查 hidden 状态是否仍然成立
|
||||
|
||||
### 8. 摘要区优先进入紧凑模式，而不是挤压正文
|
||||
|
||||
历史经验表明，最容易被误处理的是顶部摘要卡：
|
||||
|
||||
- 它们经常为了“都放下”被强行压窄
|
||||
- 然后正文、表格、AI 结果区一起失去主空间
|
||||
|
||||
后续统一约束：
|
||||
|
||||
- 小屏或高缩放时，摘要卡优先：
|
||||
- 降低 padding
|
||||
- 改成横向滚动
|
||||
- 改成更紧凑的网格
|
||||
- 不要优先牺牲主工作区的可视面积
|
||||
|
||||
### 9. 长文档类内容优先保证阅读体验
|
||||
|
||||
像下面这些内容，不能直接套用“表格工作区”的逻辑：
|
||||
|
||||
- AI 简报
|
||||
- 运行日志
|
||||
- 原始 JSON
|
||||
- 帮助说明
|
||||
- 多段描述性文本
|
||||
|
||||
这些区域应该优先满足：
|
||||
|
||||
- 标题和元信息稳定可见
|
||||
- 正文有明确的最小可读高度
|
||||
- 正文滚动策略单独定义
|
||||
- 支持 Markdown 表格、分隔线、引用、代码块等结构
|
||||
|
||||
### 10. 高度关键路径要少包一层
|
||||
|
||||
历史上不少滚动问题不是组件本身错，而是多包了一层之后：
|
||||
|
||||
- 高度链路断掉
|
||||
- `min-height: 0` 没传下去
|
||||
- `overflow` 责任被吃掉
|
||||
|
||||
因此：
|
||||
|
||||
- 对高度关键区域，优先使用最直接的 DOM 结构
|
||||
- 使用 `Space`、额外包装 `div`、第三方布局容器时，要确认它们不会改变滚动和高度语义
|
||||
- 如果一个区域已经出现“内容明明有，但只剩一条缝”，优先怀疑中间包装层
|
||||
|
||||
## 历史坑位总结
|
||||
|
||||
从 Earth、Playground、BGP、DataSources 这些页面的 bugfix 可以归纳出几类高频坑：
|
||||
|
||||
### 1. 用 `overflow: hidden` 掩盖布局问题
|
||||
|
||||
表面上看页面“整齐了”，实际上会导致：
|
||||
|
||||
- 内容被裁掉
|
||||
- tab 内容只剩一条缝
|
||||
- 面板明明渲染成功，但用户看不见
|
||||
|
||||
正确做法：
|
||||
|
||||
- 让真正的内容节点滚动
|
||||
- 不要让上层容器无差别裁剪所有子内容
|
||||
|
||||
### 2. 把所有 tab 当成同一种内容
|
||||
|
||||
表格、Markdown、帮助卡、日志流的空间需求完全不同。
|
||||
|
||||
正确做法：
|
||||
|
||||
- 表格：固定工作区 + 内部滚动
|
||||
- 文档：普通流式内容 + pane 级滚动
|
||||
- 侧边说明：内容驱动高度，不强行拉满
|
||||
|
||||
### 3. 只做视觉缩小，不做空间重分配
|
||||
|
||||
这会导致：
|
||||
|
||||
- 卡片文字被截断
|
||||
- 表格只剩 1 到 2 行
|
||||
- 按钮和筛选区挤成一团
|
||||
|
||||
正确做法：
|
||||
|
||||
- 紧凑模式优先重排
|
||||
- 横向滚动摘要区
|
||||
- 折叠/收纳次级模块
|
||||
|
||||
### 4. 父容器高度链不完整
|
||||
|
||||
这是最常见的内部滚动失效原因。
|
||||
|
||||
检查顺序：
|
||||
|
||||
1. 外层是否真的有确定高度
|
||||
2. flex 父容器是否带了 `min-height: 0`
|
||||
3. 真正滚动节点是否明确 `overflow: auto`
|
||||
4. 中间包装层是否偷偷改了布局语义
|
||||
|
||||
### 5. UI 状态和显示状态不同步
|
||||
|
||||
Earth 相关改动里反复出现：
|
||||
|
||||
- 图层隐藏了，但 hover/lock 还在
|
||||
- tooltip 还在显示旧对象
|
||||
- legend 没跟着切换
|
||||
|
||||
这类约束同样适用于后台页面：
|
||||
|
||||
- 被隐藏、卸载、切换出视图的内容，不应继续保留活跃交互状态
|
||||
|
||||
## 推荐实现模式
|
||||
|
||||
### 页面骨架
|
||||
|
||||
优先复用项目里已有的通用结构：
|
||||
|
||||
- `.dashboard-content-inner`
|
||||
- `.page-shell`
|
||||
- `.page-shell__header`
|
||||
- `.page-shell__body`
|
||||
- `.table-scroll-region`
|
||||
|
||||
不要每个页面都重新发明一套完全不同的高度和滚动语义。
|
||||
|
||||
### 表格工作区
|
||||
|
||||
推荐模式：
|
||||
|
||||
```tsx
|
||||
<Card>
|
||||
<div className="table-scroll-region" ref={tableRegionRef}>
|
||||
<Table
|
||||
pagination={false}
|
||||
scroll={{ x: 1200, y: tableHeight }}
|
||||
/>
|
||||
</div>
|
||||
</Card>
|
||||
```
|
||||
|
||||
要求：
|
||||
|
||||
- 表格尽量在卡片内部滚动
|
||||
- `scroll.y` 应来自实际可用高度估算，而不是完全静态的魔法数字
|
||||
- 父容器链路要保证 header、body、content 的 overflow 都在表格内部闭合
|
||||
|
||||
### 多模块页面
|
||||
|
||||
如果一个页面同时有：
|
||||
|
||||
- 摘要卡
|
||||
- 表格
|
||||
- 异常明细
|
||||
- 最近事件
|
||||
|
||||
不建议简单纵向堆叠全部模块。优先使用：
|
||||
|
||||
- 顶部摘要 + 底部单一主工作区
|
||||
- 标签页切换多个次级数据视图
|
||||
- 左右分栏，并保证每栏内部独立滚动
|
||||
|
||||
## 不推荐的做法
|
||||
|
||||
以下模式默认视为不符合本项目页面规范：
|
||||
|
||||
- 依赖整页纵向滚动来显示主要工作区
|
||||
- 一个页面纵向堆 3 到 4 个大卡片，每个都想完整展示
|
||||
- 表格没有内部滚动，导致缩放后只能看到 1 到 2 行数据
|
||||
- 父容器缺少 `min-height: 0`，导致内部滚动失效
|
||||
- 只做视觉缩小，不处理真正的空间分配
|
||||
|
||||
## 页面验收检查清单
|
||||
|
||||
提交前至少检查：
|
||||
|
||||
- 页头、摘要区、主工作区能否同时出现
|
||||
- 主工作区是否拿到了页面中最多的高度
|
||||
- 表格或明细溢出时，滚动条是否出现在模块内部
|
||||
- 卡片是否被压缩到文字显示不完整；如果会，是否已经切换为内部滚动
|
||||
- 浏览器缩放到 `125%` / `150%` 时是否仍可用
|
||||
- 低高度窗口下是否还保有合理的可见内容行数
|
||||
- Tabs、Card、Table 在 overflow 时是否仍可操作
|
||||
- 非表格 tab（Markdown、帮助说明、日志）是否有独立且合理的滚动策略
|
||||
|
||||
## 落地顺序
|
||||
|
||||
后续新增或重构后台页时，优先按这个顺序设计：
|
||||
|
||||
1. 先定义主工作区
|
||||
2. 再确定哪些模块必须常驻可见
|
||||
3. 最后再做样式和视觉层次
|
||||
|
||||
简单说：
|
||||
|
||||
- 先保证空间分配正确
|
||||
- 再处理滚动边界
|
||||
- 最后再做美化
|
||||
105
docs/technical/ops-docker-compose-buildx-upgrade.md
Normal file
105
docs/technical/ops-docker-compose-buildx-upgrade.md
Normal file
@@ -0,0 +1,105 @@
|
||||
# Docker + Compose + Buildx 升级教程
|
||||
|
||||
流程：删除旧版 -> 安装新版 -> 验证
|
||||
|
||||
---
|
||||
|
||||
# 1. 删除旧版本
|
||||
|
||||
## 删除 apt 安装的旧包
|
||||
|
||||
```bash
|
||||
sudo apt remove -y docker.io docker-compose docker-compose-v2 docker-doc podman-docker containerd runc
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 删除系统中的 `docker-compose`（V1）
|
||||
|
||||
```bash
|
||||
sudo rm -f "$(which docker-compose 2>/dev/null)"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 查找并删除手动安装的 Buildx 插件
|
||||
|
||||
```bash
|
||||
docker info | sed -n '/Plugins:/,/^ Server:/p' | grep -A2 buildx
|
||||
```
|
||||
|
||||
从输出中获取 `Path`，然后执行：
|
||||
|
||||
```bash
|
||||
rm -f <Path中对应的docker-buildx文件>
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 清理无用依赖
|
||||
|
||||
```bash
|
||||
sudo apt autoremove -y
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
# 2. 安装 Docker 官方版本
|
||||
|
||||
包含 Docker Engine、Docker Compose 插件、Docker Buildx 插件。
|
||||
|
||||
## 安装依赖
|
||||
|
||||
```bash
|
||||
sudo apt update
|
||||
sudo apt install -y ca-certificates curl gnupg
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 添加 Docker GPG key
|
||||
|
||||
```bash
|
||||
sudo install -m 0755 -d /etc/apt/keyrings
|
||||
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | \
|
||||
sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
|
||||
sudo chmod a+r /etc/apt/keyrings/docker.gpg
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 添加官方仓库
|
||||
|
||||
```bash
|
||||
echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \
|
||||
sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 安装 Docker + Compose + Buildx
|
||||
|
||||
```bash
|
||||
sudo apt update
|
||||
sudo apt install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
# 3. 验证安装
|
||||
|
||||
```bash
|
||||
docker --version
|
||||
docker compose version
|
||||
docker buildx version
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
# 4. 常用命令
|
||||
|
||||
```bash
|
||||
docker compose up -d
|
||||
docker compose down
|
||||
docker buildx build .
|
||||
```
|
||||
Reference in New Issue
Block a user