release: bump version to 0.43.0
This commit is contained in:
@@ -1,4 +1,4 @@
|
||||
# Technical Docs
|
||||
# 技术文档
|
||||
|
||||
这里放“当前实现和当前结构”的文档,重点回答:
|
||||
|
||||
@@ -9,24 +9,24 @@
|
||||
|
||||
适合放入这里的内容:
|
||||
|
||||
- Quickstart 和使用手册
|
||||
- 快速开始和使用手册
|
||||
- 前端上下文
|
||||
- Earth 前端结构
|
||||
- Earth 卫星 footprint 策略
|
||||
- Earth 卫星覆盖策略
|
||||
- Earth 渲染图层顺序
|
||||
- Earth 图层样式属性索引
|
||||
- 后端运行控制
|
||||
- collector 现状
|
||||
- 采集器现状
|
||||
- 采集格式约定
|
||||
|
||||
## 使用入口
|
||||
|
||||
- [quickstart.md](/home/ray/dev/linkong/planet/docs/technical/quickstart.md):从零启动 Planet 的最短路径
|
||||
- [manual.md](/home/ray/dev/linkong/planet/docs/technical/manual.md):控制台、`planet.sh`、Earth 和 Docs 的完整使用手册
|
||||
- [quickstart.md](/home/ray/dev/linkong/planet/docs/technical/zh/quickstart.md):从零启动 Planet 的最短路径
|
||||
- [manual.md](/home/ray/dev/linkong/planet/docs/technical/zh/manual.md):控制台、`planet.sh`、Earth 和 Docs 的完整使用手册
|
||||
|
||||
不适合放入这里的内容:
|
||||
|
||||
- 尚未完成的 roadmap
|
||||
- 尚未完成的路线图
|
||||
- 未来迭代方案
|
||||
- 大范围重构计划
|
||||
|
||||
|
||||
@@ -1,108 +1,108 @@
|
||||
# AI Provider Guide
|
||||
# AI Provider 指南
|
||||
|
||||
## Overview
|
||||
## 概览
|
||||
|
||||
`aiprovider` is the model-adapter service for Planet.
|
||||
`aiprovider` 是 Planet 的模型适配服务。
|
||||
|
||||
It isolates model-vendor details from the main backend so the rest of the system can call a stable business API:
|
||||
它把模型厂商差异隔离在主后端之外,让系统其它部分可以调用稳定的业务 API:
|
||||
|
||||
- Caller service -> `planet backend`
|
||||
- 调用方服务 -> `planet backend`
|
||||
- `planet backend` -> `aiprovider`
|
||||
- `aiprovider` -> concrete model provider
|
||||
- `aiprovider` -> 具体模型提供方
|
||||
|
||||
The recommended default is:
|
||||
推荐默认方式:
|
||||
|
||||
- External and cross-service callers use `planet backend`
|
||||
- Only infrastructure-grade internal jobs call `aiprovider` directly
|
||||
- 外部调用方和跨服务调用方统一调用 `planet backend`
|
||||
- 只有基础设施级内部任务才直接调用 `aiprovider`
|
||||
|
||||
## Responsibilities
|
||||
## 职责边界
|
||||
|
||||
`backend` is responsible for:
|
||||
`backend` 负责:
|
||||
|
||||
- authentication and authorization
|
||||
- business-level request shaping
|
||||
- stable `/api/v1/ai/...` endpoints
|
||||
- internal service-to-service authentication toward `aiprovider`
|
||||
- 身份认证和权限控制
|
||||
- 业务层请求整理
|
||||
- 稳定的 `/api/v1/ai/...` 接口
|
||||
- 面向 `aiprovider` 的内部服务认证
|
||||
|
||||
`aiprovider` is responsible for:
|
||||
`aiprovider` 负责:
|
||||
|
||||
- model protocol adaptation
|
||||
- provider selection by `.env`
|
||||
- timeout and lightweight retry
|
||||
- request tracing via `X-Request-ID`
|
||||
- 模型协议适配
|
||||
- 基于 `.env` 选择 provider
|
||||
- 超时和轻量重试
|
||||
- 通过 `X-Request-ID` 串联请求追踪
|
||||
|
||||
This now follows an OpenClaw-like seam:
|
||||
当前配置采用类似 OpenClaw 的拆分方式:
|
||||
|
||||
- `AI_PROVIDER` identifies the vendor or logical provider
|
||||
- `AI_PROVIDER_API` identifies the wire adapter
|
||||
- `AI_PROVIDER` 标识厂商或逻辑 provider
|
||||
- `AI_PROVIDER_API` 标识实际请求协议适配器
|
||||
|
||||
That split makes MiniMax, Claude-compatible gateways, and self-hosted OpenAI-compatible services easier to model without overloading one config field.
|
||||
这个拆分能更清楚地表达 MiniMax、Claude 兼容网关、自托管 OpenAI 兼容服务等情况,避免把所有含义塞进一个配置项。
|
||||
|
||||
## Supported Providers
|
||||
## 支持的 Provider
|
||||
|
||||
`aiprovider` currently supports these provider identities:
|
||||
`aiprovider` 当前支持以下 provider 标识:
|
||||
|
||||
- `openai`
|
||||
- `anthropic`
|
||||
- `minimax`
|
||||
- `ollama`
|
||||
|
||||
Supported request adapters:
|
||||
支持的请求适配器:
|
||||
|
||||
- `openai-completions`
|
||||
- `anthropic-messages`
|
||||
- `ollama-generate`
|
||||
|
||||
Backward-compatible aliases still accepted:
|
||||
仍然兼容的历史别名:
|
||||
|
||||
- `openai_compatible`
|
||||
- `anthropic_compatible`
|
||||
- `claude_compatible`
|
||||
|
||||
Provider mapping:
|
||||
推荐映射关系:
|
||||
|
||||
- `vLLM`, `LM Studio`, `One API`: `AI_PROVIDER=openai`, `AI_PROVIDER_API=openai-completions`
|
||||
- `MiniMax`: `AI_PROVIDER=minimax`, `AI_PROVIDER_API=anthropic-messages`
|
||||
- Claude-compatible gateways: `AI_PROVIDER=anthropic`, `AI_PROVIDER_API=anthropic-messages`
|
||||
- `Ollama`: `AI_PROVIDER=ollama`, `AI_PROVIDER_API=ollama-generate`
|
||||
- `vLLM`、`LM Studio`、`One API`:`AI_PROVIDER=openai`,`AI_PROVIDER_API=openai-completions`
|
||||
- `MiniMax`:`AI_PROVIDER=minimax`,`AI_PROVIDER_API=anthropic-messages`
|
||||
- Claude 兼容网关:`AI_PROVIDER=anthropic`,`AI_PROVIDER_API=anthropic-messages`
|
||||
- `Ollama`:`AI_PROVIDER=ollama`,`AI_PROVIDER_API=ollama-generate`
|
||||
|
||||
## API Surfaces
|
||||
## API 面
|
||||
|
||||
### Main backend API
|
||||
### 主后端 API
|
||||
|
||||
Preferred stable entrypoints:
|
||||
推荐使用的稳定入口:
|
||||
|
||||
- `GET /api/v1/ai/provider/status`
|
||||
- `POST /api/v1/ai/situational-awareness/analyze`
|
||||
|
||||
Authentication:
|
||||
认证方式:
|
||||
|
||||
- `Authorization: Bearer <jwt>`
|
||||
|
||||
Optional tracing header:
|
||||
可选追踪头:
|
||||
|
||||
- `X-Request-ID: <caller-generated-id>`
|
||||
|
||||
The backend will propagate `X-Request-ID` to `aiprovider` and return the same header in the response.
|
||||
后端会把 `X-Request-ID` 透传给 `aiprovider`,并在响应中返回同一个 header。
|
||||
|
||||
### AI provider internal API
|
||||
### AI Provider 内部 API
|
||||
|
||||
Internal-only endpoints:
|
||||
仅供内部调用的接口:
|
||||
|
||||
- `GET /v1/provider/status`
|
||||
- `POST /v1/analyze`
|
||||
|
||||
Authentication:
|
||||
认证方式:
|
||||
|
||||
- `X-Provider-Token: <shared-secret>`
|
||||
|
||||
Optional tracing header:
|
||||
可选追踪头:
|
||||
|
||||
- `X-Request-ID: <caller-generated-id>`
|
||||
|
||||
## Request Example
|
||||
## 请求示例
|
||||
|
||||
### Call through backend
|
||||
### 通过后端调用
|
||||
|
||||
```bash
|
||||
curl -X POST http://localhost:8000/api/v1/ai/situational-awareness/analyze \
|
||||
@@ -127,7 +127,7 @@ curl -X POST http://localhost:8000/api/v1/ai/situational-awareness/analyze \
|
||||
}'
|
||||
```
|
||||
|
||||
### Call `aiprovider` directly
|
||||
### 直接调用 `aiprovider`
|
||||
|
||||
```bash
|
||||
curl -X POST http://localhost:8010/v1/analyze \
|
||||
@@ -149,9 +149,9 @@ curl -X POST http://localhost:8010/v1/analyze \
|
||||
}'
|
||||
```
|
||||
|
||||
## Response Shape
|
||||
## 响应结构
|
||||
|
||||
Both backend and `aiprovider` return the same payload shape:
|
||||
后端和 `aiprovider` 返回相同的 payload 结构:
|
||||
|
||||
```json
|
||||
{
|
||||
@@ -166,15 +166,15 @@ Both backend and `aiprovider` return the same payload shape:
|
||||
}
|
||||
```
|
||||
|
||||
Both services also return:
|
||||
两个服务都会返回:
|
||||
|
||||
- `X-Request-ID: <id>`
|
||||
|
||||
## Configuration
|
||||
## 配置
|
||||
|
||||
### Backend
|
||||
### 后端
|
||||
|
||||
Recommended backend `.env`:
|
||||
推荐的后端 `.env`:
|
||||
|
||||
```env
|
||||
AI_PROVIDER_SERVICE_URL=http://localhost:8010
|
||||
@@ -183,21 +183,21 @@ AI_PROVIDER_TIMEOUT_SECONDS=60
|
||||
AI_PROVIDER_RETRY_ATTEMPTS=2
|
||||
```
|
||||
|
||||
Reference file:
|
||||
参考文件:
|
||||
|
||||
- [backend/.env.example](/home/ray/dev/linkong/planet/backend/.env.example)
|
||||
|
||||
### AI Provider
|
||||
|
||||
Reference file:
|
||||
参考文件:
|
||||
|
||||
- [aiprovider/.env.example](/home/ray/dev/linkong/planet/aiprovider/.env.example)
|
||||
|
||||
Frontend local reference:
|
||||
前端本地参考:
|
||||
|
||||
- [frontend/.env.example](/home/ray/dev/linkong/planet/frontend/.env.example)
|
||||
|
||||
Common settings:
|
||||
通用配置:
|
||||
|
||||
```env
|
||||
SERVICE_NAME=planet-ai-provider
|
||||
@@ -208,7 +208,7 @@ AI_HTTP_RETRY_ATTEMPTS=2
|
||||
AI_ANALYSIS_SYSTEM_PROMPT=你是态势感知分析助手。请基于输入的上下文、观测与约束,输出结构化、克制、可执行的分析。
|
||||
```
|
||||
|
||||
### OpenAI-compatible example
|
||||
### OpenAI 兼容示例
|
||||
|
||||
```env
|
||||
AI_PROVIDER=openai
|
||||
@@ -218,7 +218,7 @@ AI_API_KEY=local-key
|
||||
AI_MODEL=your-local-model
|
||||
```
|
||||
|
||||
### MiniMax CN example
|
||||
### MiniMax 中国区示例
|
||||
|
||||
```env
|
||||
AI_PROVIDER=minimax
|
||||
@@ -230,13 +230,13 @@ AI_MAX_TOKENS=1200
|
||||
AI_ANTHROPIC_VERSION=2023-06-01
|
||||
```
|
||||
|
||||
MiniMax note:
|
||||
MiniMax 说明:
|
||||
|
||||
- This follows the same Anthropic Messages request shape as the official MiniMax examples.
|
||||
- For MiniMax, `aiprovider` now disables `thinking` by default unless the caller explicitly passes a `thinking` object.
|
||||
- This mirrors OpenClaw's caution around MiniMax Anthropic-compatible behavior.
|
||||
- 这里使用官方 MiniMax 示例中的 Anthropic Messages 请求结构。
|
||||
- 对 MiniMax,`aiprovider` 默认不会开启 `thinking`,除非调用方显式传入 `thinking` 对象。
|
||||
- 这个行为和 OpenClaw 对 MiniMax Anthropic 兼容接口的谨慎处理保持一致。
|
||||
|
||||
### Anthropic-compatible example
|
||||
### Anthropic 兼容示例
|
||||
|
||||
```env
|
||||
AI_PROVIDER=anthropic
|
||||
@@ -248,7 +248,7 @@ AI_MAX_TOKENS=1200
|
||||
AI_ANTHROPIC_VERSION=2023-06-01
|
||||
```
|
||||
|
||||
### Ollama example
|
||||
### Ollama 示例
|
||||
|
||||
```env
|
||||
AI_PROVIDER=ollama
|
||||
@@ -258,36 +258,36 @@ AI_API_KEY=
|
||||
AI_MODEL=qwen2.5:7b
|
||||
```
|
||||
|
||||
## Deployment Modes
|
||||
## 部署模式
|
||||
|
||||
### Single machine
|
||||
### 单机部署
|
||||
|
||||
Recommended local flow:
|
||||
推荐的本地流程:
|
||||
|
||||
- `backend` on `localhost:8000`
|
||||
- `aiprovider` on `localhost:8010`
|
||||
- local model gateway on `localhost:11434` or another local port
|
||||
- `backend` 运行在 `localhost:8000`
|
||||
- `aiprovider` 运行在 `localhost:8010`
|
||||
- 本地模型网关运行在 `localhost:11434` 或其它本地端口
|
||||
|
||||
Helpers already included:
|
||||
仓库内已包含辅助入口:
|
||||
|
||||
- [planet.sh](/home/ray/dev/linkong/planet/planet.sh)
|
||||
- [docker-compose.local-model.yml](/home/ray/dev/linkong/planet/docker-compose.local-model.yml)
|
||||
|
||||
### Multi-machine
|
||||
### 多机部署
|
||||
|
||||
Example topology:
|
||||
示例拓扑:
|
||||
|
||||
- app machine: `backend`
|
||||
- AI gateway machine: `aiprovider`
|
||||
- model machine: local model service or cloud proxy
|
||||
- 应用机器:`backend`
|
||||
- AI 网关机器:`aiprovider`
|
||||
- 模型机器:本地模型服务或云代理
|
||||
|
||||
In that case, this becomes service-to-service HTTP RPC:
|
||||
此时链路变成服务间 HTTP RPC:
|
||||
|
||||
- caller -> backend
|
||||
- backend -> `http://10.0.0.12:8010`
|
||||
- `aiprovider` -> model endpoint
|
||||
- `aiprovider` -> 模型端点
|
||||
|
||||
Recommended cross-machine backend config:
|
||||
推荐的跨机器后端配置:
|
||||
|
||||
```env
|
||||
AI_PROVIDER_SERVICE_URL=http://10.0.0.12:8010
|
||||
@@ -296,38 +296,38 @@ AI_PROVIDER_TIMEOUT_SECONDS=60
|
||||
AI_PROVIDER_RETRY_ATTEMPTS=2
|
||||
```
|
||||
|
||||
Recommended operating rules:
|
||||
推荐运行规则:
|
||||
|
||||
- keep `aiprovider` on a private network
|
||||
- protect it with `X-Provider-Token` at minimum
|
||||
- always send `X-Request-ID`
|
||||
- keep callers on the backend API unless they are infrastructure jobs
|
||||
- 将 `aiprovider` 放在私有网络内
|
||||
- 至少用 `X-Provider-Token` 保护它
|
||||
- 始终发送 `X-Request-ID`
|
||||
- 除基础设施任务外,调用方优先走后端 API
|
||||
|
||||
## Retry And Failure Behavior
|
||||
## 重试和失败行为
|
||||
|
||||
`backend -> aiprovider`:
|
||||
`backend -> aiprovider`:
|
||||
|
||||
- retries lightweight network / 5xx failures
|
||||
- returns `502` when the provider service is unavailable
|
||||
- 对轻量网络错误和 5xx 失败进行重试
|
||||
- provider 服务不可用时返回 `502`
|
||||
|
||||
`aiprovider -> model provider`:
|
||||
`aiprovider -> model provider`:
|
||||
|
||||
- retries lightweight network / 5xx failures
|
||||
- returns `502` when the model provider is unavailable
|
||||
- 对轻量网络错误和 5xx 失败进行重试
|
||||
- 模型提供方不可用时返回 `502`
|
||||
|
||||
This is intentionally conservative. It avoids masking persistent errors while still absorbing short hiccups.
|
||||
这个策略故意保持保守:它能吸收短暂抖动,但不会掩盖持续性错误。
|
||||
|
||||
## Operational Notes
|
||||
## 运维说明
|
||||
|
||||
- `./planet.sh start` now starts `aiprovider` automatically
|
||||
- `./planet.sh restart -a` restarts only `aiprovider`
|
||||
- `./planet.sh log -a` tails `aiprovider` logs
|
||||
- `./planet.sh health` reports `aiprovider` health
|
||||
- `./planet.sh start` 会自动启动 `aiprovider`
|
||||
- `./planet.sh restart -a` 只重启 `aiprovider`
|
||||
- `./planet.sh log -a` 跟随查看 `aiprovider` 日志
|
||||
- `./planet.sh health` 会报告 `aiprovider` 健康状态
|
||||
|
||||
## Recommended Calling Policy
|
||||
## 推荐调用策略
|
||||
|
||||
- Frontend and application services: call `backend`
|
||||
- Scheduled infra jobs and diagnostics: optionally call `aiprovider`
|
||||
- Do not let multiple business services integrate model vendors independently
|
||||
- 前端和应用服务:调用 `backend`
|
||||
- 定时基础设施任务和诊断任务:可选直接调用 `aiprovider`
|
||||
- 不要让多个业务服务分别接入模型厂商
|
||||
|
||||
That keeps provider switching centralized and avoids model-specific drift across the system.
|
||||
这样可以集中管理 provider 切换,避免模型相关差异在系统里四处扩散。
|
||||
|
||||
100
docs/technical/zh/backend-datasources-api-performance.md
Normal file
100
docs/technical/zh/backend-datasources-api-performance.md
Normal file
@@ -0,0 +1,100 @@
|
||||
# DataSources 列表接口性能优化
|
||||
|
||||
## 背景
|
||||
|
||||
`GET /api/v1/datasources` 是数据源管理页面的核心接口,响应慢会直接阻塞页面渲染。
|
||||
|
||||
## 优化前的查询链路
|
||||
|
||||
`_load_datasource_list_context` 按顺序执行以下查询:
|
||||
|
||||
| 序号 | 函数 | 查询内容 | 瓶颈 |
|
||||
|------|------|---------|------|
|
||||
| 1 | `_load_latest_running_tasks` | collection_tasks 窗口函数,stale check 依赖此结果 | 必须串行 |
|
||||
| 2 | `_load_latest_completed_tasks` | collection_tasks 窗口函数(最近完成任务) | 串行等待 |
|
||||
| 3 | `_load_datasource_data_counts` | `COUNT(*) GROUP BY source` on collected_data | **最慢,全表扫描** |
|
||||
| 4 | `_load_datasource_endpoint_overrides` | datasource_configs 简单 SELECT | 串行等待 |
|
||||
|
||||
## 第一阶段:并行化
|
||||
|
||||
将 2/3/4 三个互不依赖的查询改为 `asyncio.gather` + 独立 session 并行执行:
|
||||
|
||||
```python
|
||||
async def _fetch_completed():
|
||||
async with async_session_factory() as s:
|
||||
return await _load_latest_completed_tasks(s, datasource_ids)
|
||||
|
||||
async def _fetch_counts():
|
||||
async with async_session_factory() as s:
|
||||
return await _load_datasource_data_counts(s, sources)
|
||||
|
||||
async def _fetch_overrides():
|
||||
async with async_session_factory() as s:
|
||||
return await _load_datasource_endpoint_overrides(s, sources)
|
||||
|
||||
completed_tasks, data_counts, endpoint_overrides = await asyncio.gather(
|
||||
_fetch_completed(), _fetch_counts(), _fetch_overrides(),
|
||||
)
|
||||
```
|
||||
|
||||
> **注意**:SQLAlchemy `AsyncSession` 不支持在同一 session 上并发,每个协程必须独立开 session。
|
||||
|
||||
## 第二阶段:删除重量级查询
|
||||
|
||||
### 删除 `_load_datasource_data_counts`
|
||||
|
||||
`data_count` 字段仅用于前端在"最近采集"列显示 `(0条)` 的边缘提示,不值得为此维持一次 `COUNT(*) GROUP BY` 全表扫描。
|
||||
|
||||
- 前端同步移除 `(0条)` 显示逻辑
|
||||
- 移除 `BuiltInDataSource` 接口中的 `data_count` 字段
|
||||
|
||||
### 删除 `_load_latest_completed_tasks`
|
||||
|
||||
`last_status` 和 `last_run_at` 已由 collector 在任务完成时直接更新到 `DataSource` 模型字段,不需要再 JOIN collection_tasks 获取:
|
||||
|
||||
```python
|
||||
# 优化前:需要查 completed_tasks
|
||||
last_run_at = datasource.last_run_at or (last_task.completed_at if last_task else None)
|
||||
last_status = datasource.last_status or (last_task.status if last_task else None)
|
||||
|
||||
# 优化后:直接读模型字段
|
||||
last_run_at = datasource.last_run_at
|
||||
last_status = datasource.last_status
|
||||
```
|
||||
|
||||
同步移除 `last_records_processed` 字段(来源是 completed_tasks,列表不显示此字段)。
|
||||
|
||||
## 优化后的查询链路
|
||||
|
||||
```
|
||||
datasources SELECT → 主数据,必须
|
||||
_load_latest_running_tasks → 必须(进行中状态 + stale check)
|
||||
_load_datasource_endpoint_overrides → 必须(endpoint 覆盖,编辑内置 collector 时需要默认值)
|
||||
```
|
||||
|
||||
3 个查询(原来 5 个),后两个顺序执行(running tasks 先完成用于 stale check,endpoint overrides 轻量)。
|
||||
|
||||
## 前端 triggerDatasource 双调修复
|
||||
|
||||
`triggerDatasource` 中存在双重 `fetchData()` 调用:
|
||||
|
||||
```typescript
|
||||
// 修复前
|
||||
} else {
|
||||
window.setTimeout(() => { fetchData() }, 800) // 无 task_id 时延迟刷
|
||||
}
|
||||
fetchData() // 总是立即刷 → 与上面的延迟刷重叠
|
||||
|
||||
// 修复后(二者互斥)
|
||||
if (res.data.task_id) {
|
||||
setTaskProgress(...)
|
||||
fetchData() // 有 task_id:立即刷一次
|
||||
} else {
|
||||
window.setTimeout(fetchData, 800) // 无 task_id:等 800ms 再刷一次
|
||||
}
|
||||
```
|
||||
|
||||
## 相关文件
|
||||
|
||||
- `backend/app/api/v1/datasources.py` — `_load_datasource_list_context`、`list_datasources`
|
||||
- `frontend/src/pages/DataSources/DataSources.tsx` — `BuiltInDataSource` interface、`triggerDatasource`
|
||||
@@ -1,61 +1,57 @@
|
||||
# System Service Control
|
||||
# 系统服务控制
|
||||
|
||||
This document defines the fixed mapping between admin control-plane actions and
|
||||
the existing `planet.sh` service-management commands.
|
||||
本文定义后台控制面动作与现有 `planet.sh` 服务管理命令之间的固定映射。
|
||||
|
||||
The goal is to reuse the current operational script semantics without exposing
|
||||
arbitrary shell execution to the frontend or API callers.
|
||||
目标是在复用当前运维脚本语义的同时,不向前端或 API 调用方暴露任意 shell 执行能力。
|
||||
|
||||
## Scope
|
||||
## 范围
|
||||
|
||||
- This mapping is for admin-side operational controls only.
|
||||
- The control plane must submit a fixed action name, not a raw shell command.
|
||||
- The backend is responsible for translating an allowed action into a fixed
|
||||
`planet.sh` invocation.
|
||||
- 这套映射只用于管理端运维控制。
|
||||
- 控制面必须提交固定 action 名称,而不是原始 shell 命令。
|
||||
- 后端负责把允许的 action 翻译成固定的 `planet.sh` 调用。
|
||||
|
||||
## Design Rules
|
||||
## 设计规则
|
||||
|
||||
- Only whitelist actions may be executed.
|
||||
- The frontend must never send arbitrary shell strings.
|
||||
- The backend must build command arguments from a fixed mapping table.
|
||||
- High-risk actions should be restricted to `super_admin`.
|
||||
- Prefer partial restarts over full-stack restarts when UI continuity matters.
|
||||
- 只允许执行白名单 action。
|
||||
- 前端绝不能发送任意 shell 字符串。
|
||||
- 后端必须从固定映射表构造命令参数。
|
||||
- 高风险 action 应限制为 `super_admin`。
|
||||
- 在 UI 连续性重要时,优先局部重启,而不是全栈重启。
|
||||
|
||||
## Action Mapping
|
||||
## Action 映射
|
||||
|
||||
| Action name | Intended use | `planet.sh` command | Notes |
|
||||
| Action 名称 | 用途 | `planet.sh` 命令 | 备注 |
|
||||
| --- | --- | --- | --- |
|
||||
| `restart-backend` | Restart backend API only | `./planet.sh restart -b` | Recommended first implementation for UI-triggered restart flows. |
|
||||
| `restart-database` | Restart PostgreSQL and Redis containers | `./planet.sh restart -d` | Useful when database/cache services need a controlled bounce without restarting the UI. |
|
||||
| `restart-system` | Restart the whole application stack | `./planet.sh restart` | Frontend continuity breaks briefly; UI should switch to guided recovery mode. |
|
||||
| `restart-frontend` | Restart frontend dev server only | `./planet.sh restart -f` | Use with caution; UI continuity is weaker than backend-only restart. |
|
||||
| `restart-backend-port` | Restart backend on a specific port | `./planet.sh restart -b <port>` | Port must be backend-validated before execution. |
|
||||
| `restart-frontend-port` | Restart frontend on a specific port | `./planet.sh restart -f <port>` | Port must be backend-validated before execution. |
|
||||
| `health-check` | Read current service health | `./planet.sh health` | Safe read-only operational action. |
|
||||
| `show-logs-backend` | Inspect backend logs | `./planet.sh log -b` | Best used for CLI/operator tooling, not normal Web UI streaming. |
|
||||
| `show-logs-frontend` | Inspect frontend logs | `./planet.sh log -f` | Best used for CLI/operator tooling, not normal Web UI streaming. |
|
||||
| `restart-backend` | 只重启后端 API | `./planet.sh restart -b` | 推荐作为 UI 触发重启流程的第一阶段实现。 |
|
||||
| `restart-database` | 重启 PostgreSQL 和 Redis 容器 | `./planet.sh restart -d` | 适合数据库/缓存需要受控重启但不希望重启 UI 的场景。 |
|
||||
| `restart-system` | 重启整个应用栈 | `./planet.sh restart` | 前端会短暂中断;UI 应进入引导恢复模式。 |
|
||||
| `restart-frontend` | 只重启前端开发服务器 | `./planet.sh restart -f` | 谨慎使用;UI 连续性弱于只重启后端。 |
|
||||
| `restart-backend-port` | 在指定端口重启后端 | `./planet.sh restart -b <port>` | 执行前必须由后端校验端口。 |
|
||||
| `restart-frontend-port` | 在指定端口重启前端 | `./planet.sh restart -f <port>` | 执行前必须由后端校验端口。 |
|
||||
| `health-check` | 读取当前服务健康状态 | `./planet.sh health` | 安全的只读运维动作。 |
|
||||
| `show-logs-backend` | 查看后端日志 | `./planet.sh log -b` | 更适合 CLI/运维工具,不建议作为普通 Web UI 日志流。 |
|
||||
| `show-logs-frontend` | 查看前端日志 | `./planet.sh log -f` | 更适合 CLI/运维工具,不建议作为普通 Web UI 日志流。 |
|
||||
|
||||
## Not Exposed In UI By Default
|
||||
## 默认不暴露到 UI 的能力
|
||||
|
||||
The following existing script capabilities should not be exposed directly in the
|
||||
Web UI unless there is an explicit product need and an additional safety review:
|
||||
除非有明确产品需求并经过额外安全评审,否则以下脚本能力不应直接暴露到 Web UI:
|
||||
|
||||
- `./planet.sh restart`
|
||||
- `./planet.sh start`
|
||||
- `./planet.sh stop`
|
||||
- `./planet.sh createuser`
|
||||
- any future raw shell passthrough
|
||||
- 任何未来的原始 shell 透传能力
|
||||
|
||||
Reason:
|
||||
原因:
|
||||
|
||||
- full restart can break the current control session;
|
||||
- stop/start have larger blast radius;
|
||||
- user creation is not a service-control operation;
|
||||
- raw shell passthrough creates unnecessary privilege risk.
|
||||
- 全量重启可能打断当前控制会话;
|
||||
- stop/start 影响面更大;
|
||||
- 用户创建不是服务控制操作;
|
||||
- 原始 shell 透传会引入不必要的权限风险。
|
||||
|
||||
## Recommended First-Phase UI Contract
|
||||
## 第一阶段推荐 UI 契约
|
||||
|
||||
### Frontend action payload
|
||||
### 前端 action payload
|
||||
|
||||
```json
|
||||
{
|
||||
@@ -63,7 +59,7 @@ Reason:
|
||||
}
|
||||
```
|
||||
|
||||
### Backend command resolution
|
||||
### 后端命令解析
|
||||
|
||||
```text
|
||||
restart-backend -> ["./planet.sh", "restart", "-b"]
|
||||
@@ -73,19 +69,19 @@ restart-frontend -> ["./planet.sh", "restart", "-f"]
|
||||
health-check -> ["./planet.sh", "health"]
|
||||
```
|
||||
|
||||
## API Draft
|
||||
## API 草案
|
||||
|
||||
### Primary Endpoint
|
||||
### 主接口
|
||||
|
||||
- `POST /api/v1/system/restart-tasks`
|
||||
|
||||
Purpose:
|
||||
用途:
|
||||
|
||||
- create a controlled restart task;
|
||||
- resolve a whitelist action into a fixed `planet.sh` command;
|
||||
- hand execution off to an external runner or detached subprocess.
|
||||
- 创建受控重启任务;
|
||||
- 将白名单 action 解析成固定 `planet.sh` 命令;
|
||||
- 把执行交给外部 runner 或 detached subprocess。
|
||||
|
||||
### Request Body
|
||||
### 请求体
|
||||
|
||||
```json
|
||||
{
|
||||
@@ -93,7 +89,7 @@ Purpose:
|
||||
}
|
||||
```
|
||||
|
||||
Optional future shape:
|
||||
未来可选形态:
|
||||
|
||||
```json
|
||||
{
|
||||
@@ -102,7 +98,7 @@ Optional future shape:
|
||||
}
|
||||
```
|
||||
|
||||
### Response
|
||||
### 响应
|
||||
|
||||
```json
|
||||
{
|
||||
@@ -114,11 +110,11 @@ Optional future shape:
|
||||
}
|
||||
```
|
||||
|
||||
### Task Query Endpoint
|
||||
### 任务查询接口
|
||||
|
||||
- `GET /api/v1/system/restart-tasks/{task_id}`
|
||||
|
||||
Response shape:
|
||||
响应结构:
|
||||
|
||||
```json
|
||||
{
|
||||
@@ -136,11 +132,11 @@ Response shape:
|
||||
}
|
||||
```
|
||||
|
||||
### Optional Log Endpoint
|
||||
### 可选日志接口
|
||||
|
||||
- `GET /api/v1/system/restart-tasks/{task_id}/logs`
|
||||
|
||||
Suggested response:
|
||||
建议响应:
|
||||
|
||||
```json
|
||||
{
|
||||
@@ -154,10 +150,9 @@ Suggested response:
|
||||
}
|
||||
```
|
||||
|
||||
This log endpoint is optional for phase one. The first version can work with
|
||||
task state plus `/health` polling alone.
|
||||
日志接口在第一阶段不是必需项。首版可以只依赖任务状态加 `/health` 轮询。
|
||||
|
||||
## Task State Model
|
||||
## 任务状态模型
|
||||
|
||||
### Status
|
||||
|
||||
@@ -177,35 +172,32 @@ task state plus `/health` polling alone.
|
||||
- `healthy`
|
||||
- `failed`
|
||||
|
||||
### Interpretation
|
||||
### 含义
|
||||
|
||||
- `status` is the high-level terminal or non-terminal state.
|
||||
- `stage` is the operator-facing execution phase for the UI.
|
||||
- `message` is the short human-readable line shown in the modal or full-screen
|
||||
overlay.
|
||||
- `status` 是高层终态/非终态状态。
|
||||
- `stage` 是面向运维人员和 UI 的执行阶段。
|
||||
- `message` 是 modal 或全屏遮罩中展示的短文本。
|
||||
|
||||
## Permission Model
|
||||
## 权限模型
|
||||
|
||||
- `restart-backend` should require `super_admin`.
|
||||
- Permission checks should follow the same role pattern already used in
|
||||
[users.py](/home/ray/dev/linkong/planet/backend/app/api/v1/users.py).
|
||||
- Frontend visibility may hide controls for non-`super_admin`, but backend must
|
||||
still enforce authorization.
|
||||
- `restart-backend` 应要求 `super_admin`。
|
||||
- 权限检查应沿用 [users.py](/home/ray/dev/linkong/planet/backend/app/api/v1/users.py) 中已有的角色模式。
|
||||
- 前端可以对非 `super_admin` 隐藏控件,但后端必须继续强制鉴权。
|
||||
|
||||
## Storage Model
|
||||
## 存储模型
|
||||
|
||||
Recommended first implementation:
|
||||
推荐第一阶段实现:
|
||||
|
||||
- store restart task state in Redis;
|
||||
- keep task lifetime short;
|
||||
- keep recent logs as a bounded list.
|
||||
- 将重启任务状态存入 Redis;
|
||||
- 任务生命周期保持较短;
|
||||
- 最近日志用有界列表保存。
|
||||
|
||||
Suggested keys:
|
||||
建议 key:
|
||||
|
||||
- `system:restart_task:{task_id}`
|
||||
- `system:restart_task:{task_id}:logs`
|
||||
|
||||
Suggested stored fields:
|
||||
建议字段:
|
||||
|
||||
- `task_id`
|
||||
- `action`
|
||||
@@ -217,22 +209,21 @@ Suggested stored fields:
|
||||
- `created_at`
|
||||
- `updated_at`
|
||||
|
||||
## Execution Model
|
||||
## 执行模型
|
||||
|
||||
The request-handling API process should not depend on itself surviving long
|
||||
enough to stream the whole restart output.
|
||||
处理请求的 API 进程不应依赖自身持续存活来流式输出完整重启日志。
|
||||
|
||||
Recommended execution flow:
|
||||
推荐执行流程:
|
||||
|
||||
1. validate caller and action
|
||||
2. create task state in Redis
|
||||
3. resolve action to fixed `planet.sh` argv
|
||||
4. spawn detached executor
|
||||
5. return `task_id`
|
||||
6. executor updates task state while restart is in progress
|
||||
7. frontend polls health and/or task state until recovery
|
||||
1. 校验调用方和 action
|
||||
2. 在 Redis 中创建任务状态
|
||||
3. 将 action 解析为固定 `planet.sh` argv
|
||||
4. 启动 detached executor
|
||||
5. 返回 `task_id`
|
||||
6. executor 在重启过程中更新任务状态
|
||||
7. 前端轮询健康状态和/或任务状态,直到服务恢复
|
||||
|
||||
Recommended command resolution examples:
|
||||
推荐命令解析示例:
|
||||
|
||||
```text
|
||||
restart-backend -> ["./planet.sh", "restart", "-b"]
|
||||
@@ -241,25 +232,25 @@ restart-backend-port -> ["./planet.sh", "restart", "-b", "<port>"]
|
||||
health-check -> ["./planet.sh", "health"]
|
||||
```
|
||||
|
||||
## Frontend Polling Flow
|
||||
## 前端轮询流程
|
||||
|
||||
Recommended first-phase UX:
|
||||
推荐第一阶段 UX:
|
||||
|
||||
1. user clicks `重启后端`
|
||||
2. confirmation modal explains temporary unavailability
|
||||
3. frontend calls `POST /api/v1/system/restart-tasks`
|
||||
4. UI enters blocking restart state
|
||||
5. frontend polls `/health` every `1-2s`
|
||||
6. temporary request failures are treated as expected
|
||||
7. after `2-3` consecutive successful health checks, frontend reloads page
|
||||
1. 用户点击 `重启后端`
|
||||
2. 确认 modal 说明服务会短暂不可用
|
||||
3. 前端调用 `POST /api/v1/system/restart-tasks`
|
||||
4. UI 进入阻塞式重启状态
|
||||
5. 前端每 `1-2s` 轮询 `/health`
|
||||
6. 临时请求失败视为预期现象
|
||||
7. 连续 `2-3` 次健康检查成功后,前端刷新页面
|
||||
|
||||
Optional richer polling:
|
||||
可选增强轮询:
|
||||
|
||||
1. poll task status endpoint while backend is still reachable
|
||||
2. switch to `/health` recovery polling after disconnect begins
|
||||
3. refresh page after health recovery
|
||||
1. 后端仍可达时轮询任务状态接口
|
||||
2. 断连开始后切换为 `/health` 恢复轮询
|
||||
3. 健康恢复后刷新页面
|
||||
|
||||
## Frontend State Machine
|
||||
## 前端状态机
|
||||
|
||||
- `idle`
|
||||
- `confirming`
|
||||
@@ -270,7 +261,7 @@ Optional richer polling:
|
||||
- `failed`
|
||||
- `timeout`
|
||||
|
||||
Suggested UI messages:
|
||||
建议 UI 文案:
|
||||
|
||||
- `已发送重启指令`
|
||||
- `正在停止后端服务`
|
||||
@@ -278,70 +269,65 @@ Suggested UI messages:
|
||||
- `服务已恢复,正在刷新页面`
|
||||
- `恢复超时,请手动检查服务状态`
|
||||
|
||||
## Phase-One Recommendation
|
||||
## 第一阶段建议
|
||||
|
||||
Implement only the following in phase one:
|
||||
第一阶段只实现:
|
||||
|
||||
- `restart-backend`
|
||||
- `super_admin` permission gate
|
||||
- task creation endpoint
|
||||
- Redis-backed task state
|
||||
- frontend confirmation modal
|
||||
- frontend `/health` polling
|
||||
- automatic page reload after recovery
|
||||
- `super_admin` 权限门禁
|
||||
- 任务创建接口
|
||||
- Redis 任务状态
|
||||
- 前端确认 modal
|
||||
- 前端 `/health` 轮询
|
||||
- 恢复后自动刷新页面
|
||||
|
||||
Do not implement in phase one:
|
||||
第一阶段不要实现:
|
||||
|
||||
- full `./planet.sh restart`
|
||||
- raw shell command passthrough
|
||||
- arbitrary service control
|
||||
- full terminal stdout streaming
|
||||
- multi-action concurrent restart queueing
|
||||
- 完整 `./planet.sh restart`
|
||||
- 原始 shell 命令透传
|
||||
- 任意服务控制
|
||||
- 完整终端 stdout 流式输出
|
||||
- 多 action 并发重启队列
|
||||
|
||||
## Implementation Checklist
|
||||
## 实现清单
|
||||
|
||||
### Backend
|
||||
### 后端
|
||||
|
||||
1. add a dedicated system-control API module under `backend/app/api/v1/`
|
||||
2. add a whitelist-based action resolver for `planet.sh`
|
||||
3. store restart task state in Redis
|
||||
4. add detached restart-runner script execution
|
||||
5. expose:
|
||||
1. 在 `backend/app/api/v1/` 下新增专用系统控制 API 模块
|
||||
2. 增加基于白名单的 `planet.sh` action 解析器
|
||||
3. 将重启任务状态存入 Redis
|
||||
4. 增加 detached restart-runner 脚本执行
|
||||
5. 暴露:
|
||||
- `POST /api/v1/system/restart-tasks`
|
||||
- `GET /api/v1/system/restart-tasks/{task_id}`
|
||||
- optional task log endpoint
|
||||
6. enforce `super_admin` permission on all restart-task endpoints
|
||||
- 可选任务日志接口
|
||||
6. 对所有 restart-task 接口强制 `super_admin` 权限
|
||||
|
||||
### Frontend
|
||||
### 前端
|
||||
|
||||
1. add a `重启后端` control on the dashboard for `super_admin`
|
||||
2. show a confirmation modal before dispatch
|
||||
3. after submission, switch modal into blocking restart state
|
||||
4. poll `/health` until backend recovery is confirmed
|
||||
5. auto-refresh page after consecutive successful health checks
|
||||
6. show short stage-oriented logs instead of raw terminal streaming
|
||||
1. 在 dashboard 为 `super_admin` 增加 `重启后端` 控件
|
||||
2. 发送前展示确认 modal
|
||||
3. 提交后将 modal 切换为阻塞式重启状态
|
||||
4. 轮询 `/health` 直到确认后端恢复
|
||||
5. 连续健康检查成功后自动刷新页面
|
||||
6. 展示简短阶段日志,而不是原始终端流
|
||||
|
||||
### Operational Notes
|
||||
### 运维说明
|
||||
|
||||
1. phase one should target backend-only restart
|
||||
2. frontend restart should remain out of scope initially
|
||||
3. command execution must always originate from repository root
|
||||
4. only fixed action names may cross the API boundary
|
||||
1. 第一阶段目标应限定为只重启后端
|
||||
2. 前端重启初期保持在范围外
|
||||
3. 命令执行必须始终从仓库根目录发起
|
||||
4. API 边界只能传递固定 action 名称
|
||||
|
||||
## Validation Requirements
|
||||
## 校验要求
|
||||
|
||||
- Reject any action not present in the whitelist.
|
||||
- If a port-bearing action is added, validate the port as an integer in
|
||||
`1..65535`.
|
||||
- Resolve commands from the repository root so `planet.sh` runs with a stable
|
||||
working directory.
|
||||
- Record the requested action, operator identity, execution start time, and
|
||||
result.
|
||||
- 拒绝任何不在白名单中的 action。
|
||||
- 如果增加带端口 action,端口必须校验为 `1..65535` 的整数。
|
||||
- 从仓库根目录解析命令,确保 `planet.sh` 的工作目录稳定。
|
||||
- 记录请求 action、操作者身份、执行开始时间和结果。
|
||||
|
||||
## Implementation Guidance
|
||||
## 实现建议
|
||||
|
||||
- For UI-triggered restart flows, prefer `restart-backend` first.
|
||||
- Do not rely on the current API request process to stream full restart output
|
||||
after it triggers its own restart.
|
||||
- Use a task record plus polling/health-check recovery flow instead of raw
|
||||
terminal streaming as the primary UX.
|
||||
- UI 触发重启流程时,优先实现 `restart-backend`。
|
||||
- 不要依赖当前 API 请求进程在触发自身重启后继续输出完整日志。
|
||||
- 主 UX 使用任务记录加轮询/健康检查恢复流程,而不是原始终端流。
|
||||
|
||||
@@ -1,31 +1,31 @@
|
||||
# BGP Context
|
||||
# BGP 态势上下文
|
||||
|
||||
## Current Goal
|
||||
## 当前目标
|
||||
|
||||
The BGP module is being evolved from an anomaly-only demo into a layered observability pipeline:
|
||||
BGP 模块正在从一个只展示异常的演示功能,演进为分层观测管线:
|
||||
|
||||
`raw observations -> enrichment -> detectors -> incidents -> console/Earth visualization`
|
||||
|
||||
The practical product goal is no longer just to "show incidents on the globe". The current product objective is:
|
||||
实际产品目标已经不只是“在地球上显示事件”。当前目标是:
|
||||
|
||||
1. keep BGP visually present on Earth even when incident density is low
|
||||
2. make incidents clearly feel like a higher-confidence layer than anomalies
|
||||
3. show that the observation network is still active even when there are no active incidents
|
||||
1. 即使 incident 密度很低,也让 BGP 在 Earth 上保持可见存在感
|
||||
2. 让 incident 明显比 anomaly 更像高置信度事件层
|
||||
3. 即使没有活跃 incident,也能表达观测网络仍在运行
|
||||
|
||||
In practice, that means Earth should behave like an observability surface, not only an incident map:
|
||||
换句话说,Earth 应该表现为观测面,而不只是事件地图:
|
||||
|
||||
- `collectors` show that observation is happening
|
||||
- `activity` shows where routing state is currently active or noisy
|
||||
- `incidents` become the highest-confidence focus layer
|
||||
- `collectors` 表达观测正在发生
|
||||
- `activity` 表达哪里的路由状态近期活跃或噪声较高
|
||||
- `incidents` 成为最高置信度的聚焦层
|
||||
|
||||
## Current Backend Architecture
|
||||
## 当前后端架构
|
||||
|
||||
### Data Layers
|
||||
### 数据层
|
||||
|
||||
1. `BGPObservation`
|
||||
- File: `backend/app/models/bgp_observation.py`
|
||||
- Purpose: store normalized raw routing observations from live/history sources.
|
||||
- Typical fields:
|
||||
- 文件:`backend/app/models/bgp_observation.py`
|
||||
- 用途:存储从实时/历史来源归一化后的原始路由观测。
|
||||
- 典型字段:
|
||||
- `source`
|
||||
- `collector`
|
||||
- `peer_asn`
|
||||
@@ -42,80 +42,80 @@ In practice, that means Earth should behave like an observability surface, not o
|
||||
- `ingest_batch_id`
|
||||
|
||||
2. `BGPAnomaly`
|
||||
- File: `backend/app/models/bgp_anomaly.py`
|
||||
- Purpose: hold atomic detector outputs.
|
||||
- Current detector output types include:
|
||||
- 文件:`backend/app/models/bgp_anomaly.py`
|
||||
- 用途:保存原子级 detector 输出。
|
||||
- 当前 detector 输出类型包括:
|
||||
- `origin_change`
|
||||
- `more_specific_burst`
|
||||
- `mass_withdrawal`
|
||||
|
||||
3. `BGPIncident`
|
||||
- File: `backend/app/models/bgp_incident.py`
|
||||
- Purpose: aggregate atomic anomalies into incident-level objects for humans and the UI.
|
||||
- 文件:`backend/app/models/bgp_incident.py`
|
||||
- 用途:把原子 anomaly 聚合成人类和 UI 可消费的 incident 对象。
|
||||
|
||||
### Pipeline
|
||||
### 管线
|
||||
|
||||
Main flow is currently anchored in:
|
||||
主流程目前集中在:
|
||||
|
||||
- `backend/app/services/collectors/bgp_common.py`
|
||||
- `backend/app/services/bgp_enrichment.py`
|
||||
- `backend/app/services/bgp_detectors.py`
|
||||
- `backend/app/services/bgp_incidents.py`
|
||||
|
||||
Operational flow:
|
||||
运行流程:
|
||||
|
||||
1. collectors fetch raw BGP data
|
||||
2. `normalize_bgp_event()` standardizes payloads
|
||||
3. observations are persisted to `bgp_observations`
|
||||
4. enrichment augments events with analysis context
|
||||
5. detectors create `bgp_anomalies`
|
||||
6. incident aggregation rolls anomalies up into `bgp_incidents`
|
||||
1. 采集器抓取原始 BGP 数据
|
||||
2. `normalize_bgp_event()` 规范化 payload
|
||||
3. observation 写入 `bgp_observations`
|
||||
4. enrichment 为事件补充分析上下文
|
||||
5. detector 创建 `bgp_anomalies`
|
||||
6. incident 聚合把 anomaly 汇总为 `bgp_incidents`
|
||||
|
||||
### Current Ingest Sources
|
||||
### 当前接入来源
|
||||
|
||||
1. `RIPE RIS Live`
|
||||
- Collector file: `backend/app/services/collectors/ris_live.py`
|
||||
- Used for realtime observation flow.
|
||||
- 采集器文件:`backend/app/services/collectors/ris_live.py`
|
||||
- 用于实时观测流。
|
||||
|
||||
2. `CAIDA BGPStream Backfill`
|
||||
- Collector file: `backend/app/services/collectors/bgpstream.py`
|
||||
- Used as history/backfill entry point.
|
||||
- 采集器文件:`backend/app/services/collectors/bgpstream.py`
|
||||
- 用作历史/回填入口。
|
||||
|
||||
## Current Enrichment Status
|
||||
## 当前 enrichment 状态
|
||||
|
||||
Implemented enrichment skeleton in:
|
||||
已在以下文件实现 enrichment 骨架:
|
||||
|
||||
- `backend/app/services/bgp_enrichment.py`
|
||||
|
||||
Current enrichments:
|
||||
当前 enrichment 内容:
|
||||
|
||||
- prefix family / prefix length
|
||||
- supernet / more-specific derivation
|
||||
- deduplicated AS path
|
||||
- path prepending hints
|
||||
- collector region info
|
||||
- prefix baseline hints
|
||||
- new-origin detection
|
||||
- ASN organization profile from PeeringDB where available
|
||||
- prefix scope / impacted region hints
|
||||
- prefix geography source priority:
|
||||
- `OpenGeoFeed` (override/high confidence)
|
||||
- `IPtoASN` (country-range baseline)
|
||||
- `NRO delegated stats` (registry-allocation fallback)
|
||||
- supernet / more-specific 推导
|
||||
- 去重 AS path
|
||||
- path prepending 提示
|
||||
- collector 区域信息
|
||||
- prefix baseline 提示
|
||||
- new-origin 检测
|
||||
- 可用时从 PeeringDB 获取 ASN 组织画像
|
||||
- prefix scope / 受影响区域提示
|
||||
- prefix 地理来源优先级:
|
||||
- `OpenGeoFeed`(override,高置信)
|
||||
- `IPtoASN`(国家范围 baseline)
|
||||
- `NRO delegated stats`(registry allocation fallback)
|
||||
|
||||
Current limitation:
|
||||
当前限制:
|
||||
|
||||
- `RPKI` is still placeholder-only and returns `unknown`
|
||||
- no real ROA validation source is integrated yet
|
||||
- `inetnum` / `inet6num` whois fallback is still pending
|
||||
- `RPKI` 仍只是占位,返回 `unknown`
|
||||
- 尚未集成真实 ROA 校验来源
|
||||
- `inetnum` / `inet6num` whois fallback 仍待实现
|
||||
|
||||
## Current API Surface
|
||||
## 当前 API 面
|
||||
|
||||
Primary API file:
|
||||
主 API 文件:
|
||||
|
||||
- `backend/app/api/v1/bgp.py`
|
||||
|
||||
Available endpoints:
|
||||
可用接口:
|
||||
|
||||
- `/api/v1/bgp/events`
|
||||
- `/api/v1/bgp/events/summary`
|
||||
@@ -127,16 +127,16 @@ Available endpoints:
|
||||
- `/api/v1/bgp/incidents/summary`
|
||||
- `/api/v1/bgp/incidents/{id}`
|
||||
|
||||
Visualization GeoJSON endpoints:
|
||||
可视化 GeoJSON 接口:
|
||||
|
||||
- `backend/app/api/v1/visualization.py`
|
||||
- `/api/v1/visualization/geo/bgp-collectors`
|
||||
- `/api/v1/visualization/geo/bgp-anomalies`
|
||||
- `/api/v1/visualization/geo/bgp-incidents`
|
||||
|
||||
## Current Earth Behavior
|
||||
## 当前 Earth 行为
|
||||
|
||||
Relevant files:
|
||||
相关文件:
|
||||
|
||||
- `frontend/public/earth/js/bgp.js`
|
||||
- `frontend/public/earth/js/main.js`
|
||||
@@ -144,56 +144,56 @@ Relevant files:
|
||||
- `frontend/public/earth/js/constants.js`
|
||||
- `frontend/public/earth/index.html`
|
||||
|
||||
Current design:
|
||||
当前设计:
|
||||
|
||||
1. Collectors are always shown when BGP is enabled.
|
||||
2. Incident markers are now the primary Earth BGP markers.
|
||||
3. If there are no incidents, Earth falls back to anomaly markers.
|
||||
4. If there are no anomalies either, collectors still provide presence.
|
||||
5. A dedicated `activity layer` now adds:
|
||||
- per-collector recent 15-minute activity halos
|
||||
- clustered regional activity hints derived from active collectors
|
||||
6. Incident markers now use:
|
||||
- symbol-driven event cores
|
||||
- outward ring pulses
|
||||
- reduced diffuse glow compared with older Earth builds
|
||||
5. The right-side stats now show:
|
||||
1. BGP 启用时始终显示 collectors。
|
||||
2. Incident marker 现在是 Earth BGP 的主 marker。
|
||||
3. 如果没有 incident,Earth 回退显示 anomaly marker。
|
||||
4. 如果也没有 anomaly,collector 仍然提供存在感。
|
||||
5. 专用 `activity layer` 现在增加:
|
||||
- 每个 collector 最近 15 分钟活动 halo
|
||||
- 基于活跃 collector 推导的区域聚合活动提示
|
||||
6. Incident marker 现在使用:
|
||||
- 由符号驱动的事件核心
|
||||
- 向外扩散的环形脉冲
|
||||
- 相比旧版 Earth 更少的弥散 glow
|
||||
7. 右侧统计现在显示:
|
||||
- BGP events
|
||||
- collector count
|
||||
- BGP status summary
|
||||
|
||||
This is directionally correct, but still incomplete for low-event-density periods. Right now Earth can still feel too quiet when incidents are sparse because the system lacks a dedicated `activity layer` between raw observation and incident focus.
|
||||
这个方向是对的,但在低事件密度时期仍不完整。当前 Earth 在 incident 稀疏时仍可能显得过于安静,因为系统还缺少位于原始观测和 incident 聚焦之间的专用 `activity layer`。
|
||||
|
||||
Current BGP status strategy:
|
||||
当前 BGP 状态策略:
|
||||
|
||||
- incidents present: show active incident count
|
||||
- no incidents but anomalies present: show active anomaly count, plus active observation regions when available
|
||||
- no incidents/anomalies but activity present: show `观测网络运行中`
|
||||
- no incidents/anomalies but collectors present: show `观测网络运行中 · 当前未发现聚合级事件`
|
||||
- no BGP data at all: show `暂无观测数据`
|
||||
- 有 incident:显示活跃 incident 数量
|
||||
- 无 incident 但有 anomaly:显示活跃 anomaly 数量,并在可用时显示活跃观测区域
|
||||
- 无 incident/anomaly 但有 activity:显示 `观测网络运行中`
|
||||
- 无 incident/anomaly 但有 collectors:显示 `观测网络运行中 · 当前未发现聚合级事件`
|
||||
- 完全无 BGP 数据:显示 `暂无观测数据`
|
||||
|
||||
Earth info-card strategy:
|
||||
Earth info-card 策略:
|
||||
|
||||
- `bgp` card is now incident-centric in wording
|
||||
- `bgp_collector` card shows collector location and current event count
|
||||
- `bgp` 卡片文案以 incident 为中心
|
||||
- `bgp_collector` 卡片显示 collector 位置和当前事件数
|
||||
|
||||
## Current Product Gap
|
||||
## 当前产品缺口
|
||||
|
||||
The main product gap is not architecture correctness. It is low-density visualization strategy.
|
||||
主要缺口不是架构正确性,而是低密度可视化策略。
|
||||
|
||||
Current reality:
|
||||
当前事实:
|
||||
|
||||
- incident count is naturally much lower than anomaly count
|
||||
- that is expected, because incidents are aggregated and de-noised
|
||||
- but incident-first rendering makes the Earth view look too quiet unless there is another always-available activity layer
|
||||
- incident 数量天然远低于 anomaly 数量
|
||||
- 这是预期行为,因为 incident 是聚合和去噪后的结果
|
||||
- 但 incident-first 渲染会让 Earth 显得过于安静,除非有另一层始终可用的 activity layer
|
||||
|
||||
Implementation detail for the recommended `activity layer` is expanded in [bgp-region-aggregation-plan.md](/home/ray/dev/linkong/planet/docs/plans/earth-bgp-region-aggregation-plan.md).
|
||||
推荐 `activity layer` 的实现细节在 [bgp-region-aggregation-plan.md](/home/ray/dev/linkong/planet/docs/plans/earth-bgp-region-aggregation-plan.md) 中展开。
|
||||
|
||||
So the immediate next milestone is:
|
||||
因此最近的里程碑是:
|
||||
|
||||
`event map -> observability map`
|
||||
|
||||
That means Earth needs three simultaneously readable layers:
|
||||
这意味着 Earth 需要三层同时可读:
|
||||
|
||||
1. `observation layer`
|
||||
- collectors
|
||||
@@ -205,108 +205,108 @@ That means Earth needs three simultaneously readable layers:
|
||||
- regional activity scoring
|
||||
- incident presence bonus
|
||||
3. `incident layer`
|
||||
- sparse but highly legible, high-confidence event objects
|
||||
- symbol-driven markers
|
||||
- outward ring pulse instead of broad diffuse glow
|
||||
- 稀疏但高度清晰的高置信事件对象
|
||||
- 符号化 marker
|
||||
- 向外环形脉冲,而不是大面积弥散 glow
|
||||
|
||||
## Incident Visual Direction
|
||||
## Incident 视觉方向
|
||||
|
||||
The Earth `incident` layer should not read like a large glowing patch. It should read like a compact, high-confidence event focus.
|
||||
Earth 的 `incident` 层不应该像一大片发光区域,而应该像紧凑、高置信度的事件焦点。
|
||||
|
||||
Design principles:
|
||||
设计原则:
|
||||
|
||||
1. `incident` markers should use a strong primary symbol
|
||||
- the symbol shape should carry type meaning where possible
|
||||
- examples:
|
||||
- `origin_change`: triangle-like warning marker
|
||||
- `mass_withdrawal`: alert/exclamation-style marker
|
||||
- `more_specific_burst`: split/radiating marker
|
||||
1. `incident` marker 应使用强主符号
|
||||
- 符号形状尽量承载类型含义
|
||||
- 示例:
|
||||
- `origin_change`:类似三角警告 marker
|
||||
- `mass_withdrawal`:告警/感叹号风格 marker
|
||||
- `more_specific_burst`:分裂/放射 marker
|
||||
|
||||
2. emphasis should come from outward ring pulses, not area flooding
|
||||
- use a compact hot core
|
||||
- use one or more expanding ring pulses
|
||||
- avoid broad luminous blobs that make the event center feel vague
|
||||
2. 强调应来自向外扩散的环形脉冲,而不是区域泛光
|
||||
- 使用紧凑高亮核心
|
||||
- 使用一个或多个扩张环形脉冲
|
||||
- 避免让事件中心变得模糊的大面积亮斑
|
||||
|
||||
3. `collector` and `incident` must stay visually distinct
|
||||
- collectors are observation infrastructure
|
||||
- incidents are extracted event focus
|
||||
- collector activity should stay quieter than incident pulse language
|
||||
3. `collector` 和 `incident` 必须保持视觉区别
|
||||
- collector 是观测基础设施
|
||||
- incident 是抽取后的事件焦点
|
||||
- collector activity 应比 incident pulse 更安静
|
||||
|
||||
4. calm periods still need observability presence
|
||||
- collectors and activity layers should keep the map alive
|
||||
- once incidents appear, they should clearly dominate nearby BGP visuals
|
||||
4. 平静期仍需要观测存在感
|
||||
- collectors 和 activity layer 应让地图保持活跃
|
||||
- 一旦出现 incident,它们应明确压过附近 BGP 视觉元素
|
||||
|
||||
5. incident geography should become `prefix-centric`
|
||||
- collectors should remain evidence sources, not the primary event location
|
||||
- preferred geography priority:
|
||||
5. incident 地理位置应转向 `prefix-centric`
|
||||
- collector 应保持证据来源身份,而不是主要事件位置
|
||||
- 推荐地理优先级:
|
||||
- `prefix_geography`
|
||||
- `prefix_scope`
|
||||
- `ASN organization region`
|
||||
- `collector centroid` as final fallback
|
||||
- `prefix_scope` should remain an observation-derived scope hint
|
||||
- a new `prefix_geography` layer should be introduced for actual prefix-centric placement
|
||||
- `collector centroid` 作为最终 fallback
|
||||
- `prefix_scope` 应保持为由观测推导出的范围提示
|
||||
- 应新增真正面向 prefix 位置的 `prefix_geography` 层
|
||||
|
||||
Reference inspiration:
|
||||
参考灵感:
|
||||
|
||||
- `World Monitor`
|
||||
- sparse event symbols
|
||||
- compact centers
|
||||
- ring-like outward pulses
|
||||
- stronger incident legibility than diffuse glow
|
||||
- 稀疏事件符号
|
||||
- 紧凑中心
|
||||
- 类似环形的向外脉冲
|
||||
- 比弥散 glow 更强的 incident 可读性
|
||||
|
||||
## Current Console Behavior
|
||||
## 当前控制台行为
|
||||
|
||||
Relevant page:
|
||||
相关页面:
|
||||
|
||||
- `frontend/src/pages/BGP/BGP.tsx`
|
||||
|
||||
Current BGP console page has three levels:
|
||||
当前 BGP 控制台页面有三层:
|
||||
|
||||
1. observation summary
|
||||
- total events
|
||||
- collector count
|
||||
- prefix count
|
||||
1. 观测摘要
|
||||
- 总事件数
|
||||
- collector 数量
|
||||
- prefix 数量
|
||||
|
||||
2. incident summary and incident table
|
||||
2. incident 摘要和 incident 表格
|
||||
|
||||
3. anomaly detail table plus recent observation events
|
||||
3. anomaly 详情表和最近 observation events
|
||||
|
||||
This means the BGP page still has useful signal even when there are zero anomalies.
|
||||
这意味着即使 anomaly 为零,BGP 页面仍有可用信号。
|
||||
|
||||
## Known Product/Engineering Boundaries
|
||||
## 已知产品/工程边界
|
||||
|
||||
1. The current system is still closer to an event board than a full BGP sensing platform.
|
||||
2. RIS coverage still needs to expand beyond narrow subscription scope.
|
||||
3. BGPStream history is still not full MRT-to-prefix decoded analytics.
|
||||
4. Collector geography still depends heavily on static RIPE RIS mappings.
|
||||
5. Incident-to-cable/IXP/region association is still weak and early-stage.
|
||||
6. Earth currently visualizes logical observation/impact structure, not true physical traffic paths.
|
||||
1. 当前系统仍更接近事件看板,而不是完整 BGP sensing platform。
|
||||
2. RIS 覆盖范围仍需从较窄订阅范围继续扩展。
|
||||
3. BGPStream 历史数据仍不是完整 MRT-to-prefix 解码分析。
|
||||
4. Collector 地理位置仍高度依赖静态 RIPE RIS 映射。
|
||||
5. Incident 与海缆、IXP、区域之间的关联仍较弱,且处于早期阶段。
|
||||
6. Earth 当前可视化的是逻辑观测/影响结构,而不是真实物理流量路径。
|
||||
|
||||
## Test Status
|
||||
## 测试状态
|
||||
|
||||
BGP-specific tests live in:
|
||||
BGP 专项测试位于:
|
||||
|
||||
- `backend/tests/test_bgp.py`
|
||||
|
||||
Verified status at this point:
|
||||
当前已验证状态:
|
||||
|
||||
- `25 passed` for `backend/tests/test_bgp.py`
|
||||
- `62 passed` for `backend/tests`
|
||||
- `backend/tests/test_bgp.py` 为 `25 passed`
|
||||
- `backend/tests` 为 `62 passed`
|
||||
|
||||
Covered areas include:
|
||||
覆盖范围包括:
|
||||
|
||||
- normalization
|
||||
- observation serialization
|
||||
- enrichment
|
||||
- detectors, including route leak candidate and path flap
|
||||
- detectors,包括 route leak candidate 和 path flap
|
||||
- incident aggregation
|
||||
- batch anomaly creation
|
||||
- BGP events/incidents API
|
||||
- summary endpoints
|
||||
|
||||
## Most Relevant Files
|
||||
## 最相关文件
|
||||
|
||||
Backend:
|
||||
后端:
|
||||
|
||||
- `backend/app/models/bgp_observation.py`
|
||||
- `backend/app/models/bgp_anomaly.py`
|
||||
@@ -318,7 +318,7 @@ Backend:
|
||||
- `backend/app/api/v1/bgp.py`
|
||||
- `backend/app/api/v1/visualization.py`
|
||||
|
||||
Frontend:
|
||||
前端:
|
||||
|
||||
- `frontend/src/pages/BGP/BGP.tsx`
|
||||
- `frontend/public/earth/js/bgp.js`
|
||||
@@ -327,29 +327,29 @@ Frontend:
|
||||
- `frontend/public/earth/js/constants.js`
|
||||
- `frontend/public/earth/index.html`
|
||||
|
||||
## Recommended Next Steps
|
||||
## 推荐下一步
|
||||
|
||||
### Next Backend / Detection Priority
|
||||
### 后端 / 检测优先级
|
||||
|
||||
1. Integrate real RPKI validation data.
|
||||
2. Expand realtime collector coverage and include withdrawals more broadly.
|
||||
3. Continue refining route leak and path instability detectors with stronger heuristics.
|
||||
1. 集成真实 RPKI 校验数据。
|
||||
2. 扩展实时 collector 覆盖范围,并更广泛纳入 withdrawals。
|
||||
3. 用更强启发式继续完善 route leak 和 path instability detector。
|
||||
|
||||
### Next Correlation / Storytelling Priority
|
||||
### 关联 / 叙事优先级
|
||||
|
||||
4. Strengthen incident aggregation semantics and titles.
|
||||
5. Add weak correlation from incidents to:
|
||||
- cable corridors
|
||||
- landing points
|
||||
4. 强化 incident 聚合语义和标题。
|
||||
5. 增加 incident 与以下对象的弱关联:
|
||||
- 海缆走廊
|
||||
- 登陆点
|
||||
- IXPs
|
||||
- other traffic anomaly sources
|
||||
6. Refine Earth hover/click handoff between collectors and incidents.
|
||||
- 其它流量异常来源
|
||||
6. 优化 Earth 中 collector 和 incident 之间的 hover/click 交接。
|
||||
|
||||
### Next Visualization Priority
|
||||
### 可视化优先级
|
||||
|
||||
7. Refine regional activity scoring so the activity layer is informative without becoming noisy.
|
||||
8. Add more incident symbol types as new detectors land.
|
||||
9. Add a real prefix geography source:
|
||||
- `IPtoASN / IPtoCountry` as the first practical dataset
|
||||
- `OpenGeoFeed` as a higher-quality override layer
|
||||
- registry/whois only as fallback
|
||||
7. 调整区域 activity scoring,让 activity layer 有信息量但不嘈杂。
|
||||
8. 随着新 detector 落地,增加更多 incident 符号类型。
|
||||
9. 增加真实 prefix geography 来源:
|
||||
- `IPtoASN / IPtoCountry` 作为第一阶段可用数据集
|
||||
- `OpenGeoFeed` 作为更高质量 override 层
|
||||
- registry/whois 只作为 fallback
|
||||
|
||||
@@ -1,11 +1,11 @@
|
||||
# Earth Frontend Context
|
||||
# Earth 前端结构
|
||||
|
||||
本文件描述当前 Earth 大屏前端的真实结构,重点是帮助后续继续改 HUD、图层、媒体面板、真实地形、BGP 可视化时,不再重复踩结构和状态同步上的坑。
|
||||
|
||||
相关规则建议一起参考:
|
||||
|
||||
- [rules.md](/home/ray/dev/linkong/planet/rules.md)
|
||||
- [frontend-layout-guidelines.md](/home/ray/dev/linkong/planet/docs/technical/frontend-layout-guidelines.md)
|
||||
- [frontend-layout-guidelines.md](/home/ray/dev/linkong/planet/docs/technical/zh/frontend-layout-guidelines.md)
|
||||
|
||||
## 当前目标
|
||||
|
||||
@@ -378,4 +378,4 @@ Earth 前端和控制台前端不是同一套 UI 系统:
|
||||
|
||||
控制台相关结构见:
|
||||
|
||||
- [admin-frontend-context.md](/home/ray/dev/linkong/planet/docs/technical/frontend-admin-frontend-context.md)
|
||||
- [admin-frontend-context.md](/home/ray/dev/linkong/planet/docs/technical/zh/frontend-admin-frontend-context.md)
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
本文记录当前 Earth 前端各图层的材质、颜色、透明度、线宽、半径偏移和
|
||||
`renderOrder` 等样式属性。层级关系请配合
|
||||
[earth-render-layer-order.md](/home/ray/dev/linkong/planet/docs/technical/earth-render-layer-order.md)
|
||||
[earth-render-layer-order.md](/home/ray/dev/linkong/planet/docs/technical/zh/earth-render-layer-order.md)
|
||||
查看。
|
||||
|
||||
## 命名约定
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
# News Live Streams Collector Format
|
||||
# 新闻直播采集格式
|
||||
|
||||
`news_live_streams` 采集器面向“频道目录 JSON”输入,而不是直接抓网页。
|
||||
|
||||
|
||||
@@ -1,11 +1,11 @@
|
||||
# Earth Satellite Footprint Policy
|
||||
# Earth 卫星覆盖策略
|
||||
|
||||
本文件记录 Earth 卫星图层当前关于 `footprint` 的产品边界、资料依据和已落地实现,目标是避免把 Starlink 这套专用地表覆盖模型误用到其它星座上。
|
||||
|
||||
相关上下文:
|
||||
|
||||
- [earth-frontend-context.md](/home/ray/dev/linkong/planet/docs/technical/earth-frontend-context.md)
|
||||
- [backend-collectors.md](/home/ray/dev/linkong/planet/docs/technical/backend-collectors.md)
|
||||
- [earth-frontend-context.md](/home/ray/dev/linkong/planet/docs/technical/zh/earth-frontend-context.md)
|
||||
- [backend-collectors.md](/home/ray/dev/linkong/planet/docs/technical/zh/backend-collectors.md)
|
||||
- [backend/app/services/collectors/celestrak.py](/home/ray/dev/linkong/planet/backend/app/services/collectors/celestrak.py)
|
||||
- [frontend/public/earth/js/satellites.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/satellites.js)
|
||||
|
||||
|
||||
@@ -1,11 +1,11 @@
|
||||
# Admin Frontend Context
|
||||
# 控制台前端结构
|
||||
|
||||
本文件描述当前控制台前端的真实结构,目标是帮助后续页面开发、表格改造、布局治理和状态收口时快速找到正确入口。
|
||||
|
||||
相关规则建议一起参考:
|
||||
|
||||
- [rules.md](/home/ray/dev/linkong/planet/rules.md)
|
||||
- [frontend-layout-guidelines.md](/home/ray/dev/linkong/planet/docs/technical/frontend-layout-guidelines.md)
|
||||
- [frontend-layout-guidelines.md](/home/ray/dev/linkong/planet/docs/technical/zh/frontend-layout-guidelines.md)
|
||||
|
||||
## 当前目标
|
||||
|
||||
@@ -263,7 +263,7 @@
|
||||
|
||||
详细经验见:
|
||||
|
||||
- [frontend-layout-guidelines.md](/home/ray/dev/linkong/planet/docs/technical/frontend-layout-guidelines.md)
|
||||
- [frontend-layout-guidelines.md](/home/ray/dev/linkong/planet/docs/technical/zh/frontend-layout-guidelines.md)
|
||||
|
||||
## 当前推荐改动方式
|
||||
|
||||
@@ -290,4 +290,4 @@
|
||||
|
||||
Earth 相关结构见:
|
||||
|
||||
- [earth-frontend-context.md](/home/ray/dev/linkong/planet/docs/technical/earth-frontend-context.md)
|
||||
- [earth-frontend-context.md](/home/ray/dev/linkong/planet/docs/technical/zh/earth-frontend-context.md)
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
# Frontend Layout Guidelines
|
||||
# 前端布局指南
|
||||
|
||||
本项目后台页面默认遵循“单屏工作区”布局规范。目标不是让页面永远不溢出,而是确保在常见桌面视口下:
|
||||
|
||||
|
||||
@@ -7,7 +7,7 @@
|
||||
- 控制台:登录后的管理后台
|
||||
- Docs:公开开发文档与使用手册
|
||||
|
||||
快速启动路径见 [quickstart.md](/home/ray/dev/linkong/planet/docs/technical/quickstart.md)。
|
||||
快速启动路径见 [quickstart.md](/home/ray/dev/linkong/planet/docs/technical/zh/quickstart.md)。
|
||||
|
||||
## 入口总览
|
||||
|
||||
@@ -480,9 +480,9 @@ source ~/.zshrc && bun run build
|
||||
|
||||
## 相关文档
|
||||
|
||||
- [quickstart.md](/home/ray/dev/linkong/planet/docs/technical/quickstart.md)
|
||||
- [frontend-admin-frontend-context.md](/home/ray/dev/linkong/planet/docs/technical/frontend-admin-frontend-context.md)
|
||||
- [earth-frontend-context.md](/home/ray/dev/linkong/planet/docs/technical/earth-frontend-context.md)
|
||||
- [earth-layer-style-reference.md](/home/ray/dev/linkong/planet/docs/technical/earth-layer-style-reference.md)
|
||||
- [backend-system-service-control.md](/home/ray/dev/linkong/planet/docs/technical/backend-system-service-control.md)
|
||||
- [backend-collectors.md](/home/ray/dev/linkong/planet/docs/technical/backend-collectors.md)
|
||||
- [quickstart.md](/home/ray/dev/linkong/planet/docs/technical/zh/quickstart.md)
|
||||
- [frontend-admin-frontend-context.md](/home/ray/dev/linkong/planet/docs/technical/zh/frontend-admin-frontend-context.md)
|
||||
- [earth-frontend-context.md](/home/ray/dev/linkong/planet/docs/technical/zh/earth-frontend-context.md)
|
||||
- [earth-layer-style-reference.md](/home/ray/dev/linkong/planet/docs/technical/zh/earth-layer-style-reference.md)
|
||||
- [backend-system-service-control.md](/home/ray/dev/linkong/planet/docs/technical/zh/backend-system-service-control.md)
|
||||
- [backend-collectors.md](/home/ray/dev/linkong/planet/docs/technical/zh/backend-collectors.md)
|
||||
|
||||
154
docs/technical/zh/ops-planet-sh-startup.md
Normal file
154
docs/technical/zh/ops-planet-sh-startup.md
Normal file
@@ -0,0 +1,154 @@
|
||||
# planet.sh 启动性能优化
|
||||
|
||||
## 背景
|
||||
|
||||
`planet.sh` 管理所有服务的启动/停止/重启。原有实现存在以下问题:
|
||||
|
||||
1. AI Provider 每次都重新构建(即使代码未变)
|
||||
2. 杀端口速度极慢(最长等 45 秒)
|
||||
3. 端口绑定检测用 Python 子进程(每次 ~300ms)
|
||||
4. 无参 `restart` 与 `restart -b` 行为不一致
|
||||
|
||||
## 问题一:AI Provider 每次重建
|
||||
|
||||
### 根因
|
||||
|
||||
构建戳文件存放在 `/tmp/`,WSL/Linux 重启后 `/tmp` 被清空,导致三个条件中的"戳文件非空"这一条始终不满足,进而判定需要重建:
|
||||
|
||||
```bash
|
||||
# 三个条件必须同时成立才跳过重建
|
||||
image_exists AND stamp_non_empty AND fingerprint_match
|
||||
```
|
||||
|
||||
### 修复
|
||||
|
||||
将戳文件路径从 `/tmp/` 改到持久路径:
|
||||
|
||||
```bash
|
||||
AI_PROVIDER_BUILD_STAMP_FILE="$HOME/.cache/planet/aiprovider_build.sha256"
|
||||
```
|
||||
|
||||
写入时确保目录存在:
|
||||
|
||||
```bash
|
||||
write_ai_provider_build_stamp() {
|
||||
mkdir -p "$(dirname "$AI_PROVIDER_BUILD_STAMP_FILE")"
|
||||
compute_ai_provider_build_fingerprint > "$AI_PROVIDER_BUILD_STAMP_FILE"
|
||||
}
|
||||
```
|
||||
|
||||
### fingerprint 计算提速
|
||||
|
||||
原实现对整个 `aiprovider/` 打 tar 包再算 SHA,大目录下耗时可达数秒。改为 `find + stat`(只读文件元信息,不读内容):
|
||||
|
||||
```bash
|
||||
compute_ai_provider_build_fingerprint() {
|
||||
find aiprovider \
|
||||
-type f \
|
||||
! -path '*/__pycache__/*' \
|
||||
! -name '*.pyc' \
|
||||
! -name '*.pyo' \
|
||||
| LC_ALL=C sort \
|
||||
| xargs -r stat --format="%Y %s %n" 2>/dev/null
|
||||
sha256sum docker-compose.yml docker-compose.simple.yml 2>/dev/null
|
||||
python3 "$SCRIPT_DIR/scripts/compute_aiprovider_dependency_fingerprint.py" 2>/dev/null
|
||||
}
|
||||
```
|
||||
|
||||
速度提升约 10 倍(大量小文件场景),误报率相同(mtime+size 变化 ≡ 文件被修改)。
|
||||
|
||||
### 跳过重建的原理
|
||||
|
||||
fingerprint 一致时不执行 `docker compose build`,而是:
|
||||
|
||||
```bash
|
||||
docker start planet_aiprovider # 启动已存在的容器,几秒内完成
|
||||
```
|
||||
|
||||
`docker stop` 停容器,不删镜像;`cleanup_exit_containers` 删已退出容器,不删镜像。下次 `docker start` 会从现有镜像直接创建并启动容器。
|
||||
|
||||
## 问题二:杀端口速度慢
|
||||
|
||||
### 原因
|
||||
|
||||
`wait_for_port_release` 默认最多等 45 秒(15 次 × 3 秒)。
|
||||
|
||||
### 修复
|
||||
|
||||
将后台进程清理场景的超时缩短至 3 秒(TERM→1.5s→KILL→1.5s):
|
||||
|
||||
```bash
|
||||
PORT_RELEASE_ATTEMPTS=15
|
||||
PORT_RELEASE_INTERVAL=0.2 # 每次等 0.2s,总计 3s
|
||||
|
||||
# cleanup_backend_processes / kill_port_if_requested
|
||||
wait_for_port_release "$port" 15 0.2
|
||||
```
|
||||
|
||||
`wait_for_port_release` 增加可选参数,允许不同场景使用不同超时:
|
||||
|
||||
```bash
|
||||
wait_for_port_release() {
|
||||
local port="$1"
|
||||
local max_attempts="${2:-$PORT_RELEASE_ATTEMPTS}"
|
||||
local interval="${3:-$PORT_RELEASE_INTERVAL}"
|
||||
...
|
||||
}
|
||||
```
|
||||
|
||||
## 问题三:端口检测用 Python
|
||||
|
||||
### 原因
|
||||
|
||||
`can_bind_port` 用 `python3 -c "import socket..."` 检测端口,每次调用约 300ms。
|
||||
|
||||
### 修复
|
||||
|
||||
优先使用系统工具(~10ms),Python 作为兜底:
|
||||
|
||||
```bash
|
||||
can_bind_port() {
|
||||
local port="$1"
|
||||
if command -v ss >/dev/null 2>&1; then
|
||||
! ss -tlnH 2>/dev/null | awk '{print $4}' | grep -qE ":${port}$"
|
||||
return
|
||||
fi
|
||||
if command -v lsof >/dev/null 2>&1; then
|
||||
[ -z "$(lsof -tiTCP:"${port}" -sTCP:LISTEN 2>/dev/null)" ]
|
||||
return
|
||||
fi
|
||||
python3 - "$port" <<'PY'
|
||||
import sys, socket
|
||||
p = int(sys.argv[1])
|
||||
s = socket.socket()
|
||||
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
|
||||
try:
|
||||
s.bind(("", p)); s.close(); sys.exit(0)
|
||||
except OSError:
|
||||
sys.exit(1)
|
||||
PY
|
||||
}
|
||||
```
|
||||
|
||||
## 问题四:restart 行为不一致
|
||||
|
||||
### 现象
|
||||
|
||||
- `restart -b`:停全部服务 → 检查 AI Provider fingerprint → 按需重建 → 启动
|
||||
- `restart`(无参):停全部服务 → AI Provider 总是判定需要重建(因戳文件在 /tmp)
|
||||
|
||||
### 修复
|
||||
|
||||
修复戳文件路径后,无参 `restart` 同样使用 `stop + start`,fingerprint 检查正常生效,行为与 `restart -b` 完全一致。无需额外代码变更。
|
||||
|
||||
## 其他:移除不必要的 sleep
|
||||
|
||||
启动链路中两处 `sleep 3` 在实际已有健康检查覆盖的情况下多余,已移除:
|
||||
|
||||
- `start_backend_service`:数据库健康检查通过后的 `sleep 3`
|
||||
- `restart_database_service`:重启后的等待 `sleep 3`
|
||||
|
||||
## 相关文件
|
||||
|
||||
- `planet.sh` — 全量修改
|
||||
- `scripts/compute_aiprovider_dependency_fingerprint.py` — 依赖 fingerprint(未改动)
|
||||
@@ -1,6 +1,6 @@
|
||||
# Quickstart
|
||||
# 快速开始
|
||||
|
||||
这份 Quickstart 面向第一次启动 Planet 的开发者或演示操作者。目标是用最短路径把服务跑起来,并知道应该打开哪些入口。
|
||||
这份快速开始面向第一次启动 Planet 的开发者或演示操作者。目标是用最短路径把服务跑起来,并知道应该打开哪些入口。
|
||||
|
||||
## 前置条件
|
||||
|
||||
@@ -187,7 +187,7 @@ ss -ltnp | grep -E ':3000|:8000'
|
||||
|
||||
## 下一步
|
||||
|
||||
- 完整操作说明见 [manual.md](/home/ray/dev/linkong/planet/docs/technical/manual.md)
|
||||
- 控制台结构见 [frontend-admin-frontend-context.md](/home/ray/dev/linkong/planet/docs/technical/frontend-admin-frontend-context.md)
|
||||
- Earth 结构见 [earth-frontend-context.md](/home/ray/dev/linkong/planet/docs/technical/earth-frontend-context.md)
|
||||
- 后端采集器见 [backend-collectors.md](/home/ray/dev/linkong/planet/docs/technical/backend-collectors.md)
|
||||
- 完整操作说明见 [manual.md](/home/ray/dev/linkong/planet/docs/technical/zh/manual.md)
|
||||
- 控制台结构见 [frontend-admin-frontend-context.md](/home/ray/dev/linkong/planet/docs/technical/zh/frontend-admin-frontend-context.md)
|
||||
- Earth 结构见 [earth-frontend-context.md](/home/ray/dev/linkong/planet/docs/technical/zh/earth-frontend-context.md)
|
||||
- 后端采集器见 [backend-collectors.md](/home/ray/dev/linkong/planet/docs/technical/zh/backend-collectors.md)
|
||||
|
||||
Reference in New Issue
Block a user