linkong/planet
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

release: bump version to 0.43.0

Browse Source
This commit is contained in:
linkong
2026-04-28 16:10:17 +08:00
parent 1cd2dab0ee
commit ac69d5d354
69 changed files with 6954 additions and 1141 deletions
Download Patch File Download Diff File Expand all files Collapse all files

344
docs/technical/zh/earth-bgp-context.md
View File

@@ -1,31 +1,31 @@
# BGP Context
# BGP 态势上下文
## Current Goal
## 当前目标
The BGP module is being evolved from an anomaly-only demo into a layered observability pipeline:
BGP 模块正在从一个只展示异常的演示功能，演进为分层观测管线：
`raw observations -> enrichment -> detectors -> incidents -> console/Earth visualization`
The practical product goal is no longer just to "show incidents on the globe". The current product objective is:
实际产品目标已经不只是“在地球上显示事件”。当前目标是：
1. keep BGP visually present on Earth even when incident density is low
2. make incidents clearly feel like a higher-confidence layer than anomalies
3. show that the observation network is still active even when there are no active incidents
1. 即使 incident 密度很低，也让 BGP 在 Earth 上保持可见存在感
2. 让 incident 明显比 anomaly 更像高置信度事件层
3. 即使没有活跃 incident，也能表达观测网络仍在运行
In practice, that means Earth should behave like an observability surface, not only an incident map:
换句话说，Earth 应该表现为观测面，而不只是事件地图：
- `collectors` show that observation is happening
- `activity` shows where routing state is currently active or noisy
- `incidents` become the highest-confidence focus layer
- `collectors` 表达观测正在发生
- `activity` 表达哪里的路由状态近期活跃或噪声较高
- `incidents` 成为最高置信度的聚焦层
## Current Backend Architecture
## 当前后端架构
### Data Layers
### 数据层
1. `BGPObservation`
- File: `backend/app/models/bgp_observation.py`
- Purpose: store normalized raw routing observations from live/history sources.
- Typical fields:
- 文件：`backend/app/models/bgp_observation.py`
- 用途：存储从实时/历史来源归一化后的原始路由观测。
- 典型字段：
- `source`
- `collector`
- `peer_asn`
@@ -42,80 +42,80 @@ In practice, that means Earth should behave like an observability surface, not o
- `ingest_batch_id`
2. `BGPAnomaly`
- File: `backend/app/models/bgp_anomaly.py`
- Purpose: hold atomic detector outputs.
- Current detector output types include:
- 文件：`backend/app/models/bgp_anomaly.py`
- 用途：保存原子级 detector 输出。
- 当前 detector 输出类型包括：
- `origin_change`
- `more_specific_burst`
- `mass_withdrawal`
3. `BGPIncident`
- File: `backend/app/models/bgp_incident.py`
- Purpose: aggregate atomic anomalies into incident-level objects for humans and the UI.
- 文件：`backend/app/models/bgp_incident.py`
- 用途：把原子 anomaly 聚合成人类和 UI 可消费的 incident 对象。
### Pipeline
### 管线
Main flow is currently anchored in:
主流程目前集中在：
- `backend/app/services/collectors/bgp_common.py`
- `backend/app/services/bgp_enrichment.py`
- `backend/app/services/bgp_detectors.py`
- `backend/app/services/bgp_incidents.py`
Operational flow:
运行流程：
1. collectors fetch raw BGP data
2. `normalize_bgp_event()` standardizes payloads
3. observations are persisted to `bgp_observations`
4. enrichment augments events with analysis context
5. detectors create `bgp_anomalies`
6. incident aggregation rolls anomalies up into `bgp_incidents`
1. 采集器抓取原始 BGP 数据
2. `normalize_bgp_event()` 规范化 payload
3. observation 写入 `bgp_observations`
4. enrichment 为事件补充分析上下文
5. detector 创建 `bgp_anomalies`
6. incident 聚合把 anomaly 汇总为 `bgp_incidents`
### Current Ingest Sources
### 当前接入来源
1. `RIPE RIS Live`
- Collector file: `backend/app/services/collectors/ris_live.py`
- Used for realtime observation flow.
- 采集器文件：`backend/app/services/collectors/ris_live.py`
- 用于实时观测流。
2. `CAIDA BGPStream Backfill`
- Collector file: `backend/app/services/collectors/bgpstream.py`
- Used as history/backfill entry point.
- 采集器文件：`backend/app/services/collectors/bgpstream.py`
- 用作历史/回填入口。
## Current Enrichment Status
## 当前 enrichment 状态
Implemented enrichment skeleton in:
已在以下文件实现 enrichment 骨架：
- `backend/app/services/bgp_enrichment.py`
Current enrichments:
当前 enrichment 内容：
- prefix family / prefix length
- supernet / more-specific derivation
- deduplicated AS path
- path prepending hints
- collector region info
- prefix baseline hints
- new-origin detection
- ASN organization profile from PeeringDB where available
- prefix scope / impacted region hints
- prefix geography source priority:
- `OpenGeoFeed` (override/high confidence)
- `IPtoASN` (country-range baseline)
- `NRO delegated stats` (registry-allocation fallback)
- supernet / more-specific 推导
- 去重 AS path
- path prepending 提示
- collector 区域信息
- prefix baseline 提示
- new-origin 检测
- 可用时从 PeeringDB 获取 ASN 组织画像
- prefix scope / 受影响区域提示
- prefix 地理来源优先级：
- `OpenGeoFeed`（override，高置信）
- `IPtoASN`（国家范围 baseline）
- `NRO delegated stats`（registry allocation fallback）
Current limitation:
当前限制：
- `RPKI` is still placeholder-only and returns `unknown`
- no real ROA validation source is integrated yet
- `inetnum` / `inet6num` whois fallback is still pending
- `RPKI` 仍只是占位，返回 `unknown`
- 尚未集成真实 ROA 校验来源
- `inetnum` / `inet6num` whois fallback 仍待实现
## Current API Surface
## 当前 API 面
Primary API file:
主 API 文件：
- `backend/app/api/v1/bgp.py`
Available endpoints:
可用接口：
- `/api/v1/bgp/events`
- `/api/v1/bgp/events/summary`
@@ -127,16 +127,16 @@ Available endpoints:
- `/api/v1/bgp/incidents/summary`
- `/api/v1/bgp/incidents/{id}`
Visualization GeoJSON endpoints:
可视化 GeoJSON 接口：
- `backend/app/api/v1/visualization.py`
- `/api/v1/visualization/geo/bgp-collectors`
- `/api/v1/visualization/geo/bgp-anomalies`
- `/api/v1/visualization/geo/bgp-incidents`
## Current Earth Behavior
## 当前 Earth 行为
Relevant files:
相关文件：
- `frontend/public/earth/js/bgp.js`
- `frontend/public/earth/js/main.js`
@@ -144,56 +144,56 @@ Relevant files:
- `frontend/public/earth/js/constants.js`
- `frontend/public/earth/index.html`
Current design:
当前设计：
1. Collectors are always shown when BGP is enabled.
2. Incident markers are now the primary Earth BGP markers.
3. If there are no incidents, Earth falls back to anomaly markers.
4. If there are no anomalies either, collectors still provide presence.
5. A dedicated `activity layer` now adds:
- per-collector recent 15-minute activity halos
- clustered regional activity hints derived from active collectors
6. Incident markers now use:
- symbol-driven event cores
- outward ring pulses
- reduced diffuse glow compared with older Earth builds
5. The right-side stats now show:
1. BGP 启用时始终显示 collectors。
2. Incident marker 现在是 Earth BGP 的主 marker。
3. 如果没有 incident，Earth 回退显示 anomaly marker。
4. 如果也没有 anomaly，collector 仍然提供存在感。
5. 专用 `activity layer` 现在增加：
- 每个 collector 最近 15 分钟活动 halo
- 基于活跃 collector 推导的区域聚合活动提示
6. Incident marker 现在使用：
- 由符号驱动的事件核心
- 向外扩散的环形脉冲
- 相比旧版 Earth 更少的弥散 glow
7. 右侧统计现在显示：
- BGP events
- collector count
- BGP status summary
This is directionally correct, but still incomplete for low-event-density periods. Right now Earth can still feel too quiet when incidents are sparse because the system lacks a dedicated `activity layer` between raw observation and incident focus.
这个方向是对的，但在低事件密度时期仍不完整。当前 Earth 在 incident 稀疏时仍可能显得过于安静，因为系统还缺少位于原始观测和 incident 聚焦之间的专用 `activity layer`。
Current BGP status strategy:
当前 BGP 状态策略：
- incidents present: show active incident count
- no incidents but anomalies present: show active anomaly count, plus active observation regions when available
- no incidents/anomalies but activity present: show `观测网络运行中`
- no incidents/anomalies but collectors present: show `观测网络运行中 · 当前未发现聚合级事件`
- no BGP data at all: show `暂无观测数据`
- 有 incident：显示活跃 incident 数量
- 无 incident 但有 anomaly：显示活跃 anomaly 数量，并在可用时显示活跃观测区域
- 无 incident/anomaly 但有 activity：显示 `观测网络运行中`
- 无 incident/anomaly 但有 collectors：显示 `观测网络运行中 · 当前未发现聚合级事件`
- 完全无 BGP 数据：显示 `暂无观测数据`
Earth info-card strategy:
Earth info-card 策略：
- `bgp` card is now incident-centric in wording
- `bgp_collector` card shows collector location and current event count
- `bgp` 卡片文案以 incident 为中心
- `bgp_collector` 卡片显示 collector 位置和当前事件数
## Current Product Gap
## 当前产品缺口
The main product gap is not architecture correctness. It is low-density visualization strategy.
主要缺口不是架构正确性，而是低密度可视化策略。
Current reality:
当前事实：
- incident count is naturally much lower than anomaly count
- that is expected, because incidents are aggregated and de-noised
- but incident-first rendering makes the Earth view look too quiet unless there is another always-available activity layer
- incident 数量天然远低于 anomaly 数量
- 这是预期行为，因为 incident 是聚合和去噪后的结果
- 但 incident-first 渲染会让 Earth 显得过于安静，除非有另一层始终可用的 activity layer
Implementation detail for the recommended `activity layer` is expanded in [bgp-region-aggregation-plan.md](/home/ray/dev/linkong/planet/docs/plans/earth-bgp-region-aggregation-plan.md).
推荐 `activity layer` 的实现细节在 [bgp-region-aggregation-plan.md](/home/ray/dev/linkong/planet/docs/plans/earth-bgp-region-aggregation-plan.md) 中展开。
So the immediate next milestone is:
因此最近的里程碑是：
`event map -> observability map`
That means Earth needs three simultaneously readable layers:
这意味着 Earth 需要三层同时可读：
1. `observation layer`
- collectors
@@ -205,108 +205,108 @@ That means Earth needs three simultaneously readable layers:
- regional activity scoring
- incident presence bonus
3. `incident layer`
- sparse but highly legible, high-confidence event objects
- symbol-driven markers
- outward ring pulse instead of broad diffuse glow
- 稀疏但高度清晰的高置信事件对象
- 符号化 marker
- 向外环形脉冲，而不是大面积弥散 glow
## Incident Visual Direction
## Incident 视觉方向
The Earth `incident` layer should not read like a large glowing patch. It should read like a compact, high-confidence event focus.
Earth 的 `incident` 层不应该像一大片发光区域，而应该像紧凑、高置信度的事件焦点。
Design principles:
设计原则：
1. `incident` markers should use a strong primary symbol
- the symbol shape should carry type meaning where possible
- examples:
- `origin_change`: triangle-like warning marker
- `mass_withdrawal`: alert/exclamation-style marker
- `more_specific_burst`: split/radiating marker
1. `incident` marker 应使用强主符号
- 符号形状尽量承载类型含义
- 示例：
- `origin_change`：类似三角警告 marker
- `mass_withdrawal`：告警/感叹号风格 marker
- `more_specific_burst`：分裂/放射 marker
2. emphasis should come from outward ring pulses, not area flooding
- use a compact hot core
- use one or more expanding ring pulses
- avoid broad luminous blobs that make the event center feel vague
2. 强调应来自向外扩散的环形脉冲，而不是区域泛光
- 使用紧凑高亮核心
- 使用一个或多个扩张环形脉冲
- 避免让事件中心变得模糊的大面积亮斑
3. `collector` and `incident` must stay visually distinct
- collectors are observation infrastructure
- incidents are extracted event focus
- collector activity should stay quieter than incident pulse language
3. `collector` 和 `incident` 必须保持视觉区别
- collector 是观测基础设施
- incident 是抽取后的事件焦点
- collector activity 应比 incident pulse 更安静
4. calm periods still need observability presence
- collectors and activity layers should keep the map alive
- once incidents appear, they should clearly dominate nearby BGP visuals
4. 平静期仍需要观测存在感
- collectors 和 activity layer 应让地图保持活跃
- 一旦出现 incident，它们应明确压过附近 BGP 视觉元素
5. incident geography should become `prefix-centric`
- collectors should remain evidence sources, not the primary event location
- preferred geography priority:
5. incident 地理位置应转向 `prefix-centric`
- collector 应保持证据来源身份，而不是主要事件位置
- 推荐地理优先级：
- `prefix_geography`
- `prefix_scope`
- `ASN organization region`
- `collector centroid` as final fallback
- `prefix_scope` should remain an observation-derived scope hint
- a new `prefix_geography` layer should be introduced for actual prefix-centric placement
- `collector centroid` 作为最终 fallback
- `prefix_scope` 应保持为由观测推导出的范围提示
- 应新增真正面向 prefix 位置的 `prefix_geography` 层
Reference inspiration:
参考灵感：
- `World Monitor`
- sparse event symbols
- compact centers
- ring-like outward pulses
- stronger incident legibility than diffuse glow
- 稀疏事件符号
- 紧凑中心
- 类似环形的向外脉冲
- 比弥散 glow 更强的 incident 可读性
## Current Console Behavior
## 当前控制台行为
Relevant page:
相关页面：
- `frontend/src/pages/BGP/BGP.tsx`
Current BGP console page has three levels:
当前 BGP 控制台页面有三层：
1. observation summary
- total events
- collector count
- prefix count
1. 观测摘要
- 总事件数
- collector 数量
- prefix 数量
2. incident summary and incident table
2. incident 摘要和 incident 表格
3. anomaly detail table plus recent observation events
3. anomaly 详情表和最近 observation events
This means the BGP page still has useful signal even when there are zero anomalies.
这意味着即使 anomaly 为零，BGP 页面仍有可用信号。
## Known Product/Engineering Boundaries
## 已知产品/工程边界
1. The current system is still closer to an event board than a full BGP sensing platform.
2. RIS coverage still needs to expand beyond narrow subscription scope.
3. BGPStream history is still not full MRT-to-prefix decoded analytics.
4. Collector geography still depends heavily on static RIPE RIS mappings.
5. Incident-to-cable/IXP/region association is still weak and early-stage.
6. Earth currently visualizes logical observation/impact structure, not true physical traffic paths.
1. 当前系统仍更接近事件看板，而不是完整 BGP sensing platform。
2. RIS 覆盖范围仍需从较窄订阅范围继续扩展。
3. BGPStream 历史数据仍不是完整 MRT-to-prefix 解码分析。
4. Collector 地理位置仍高度依赖静态 RIPE RIS 映射。
5. Incident 与海缆、IXP、区域之间的关联仍较弱，且处于早期阶段。
6. Earth 当前可视化的是逻辑观测/影响结构，而不是真实物理流量路径。
## Test Status
## 测试状态
BGP-specific tests live in:
BGP 专项测试位于：
- `backend/tests/test_bgp.py`
Verified status at this point:
当前已验证状态：
- `25 passed` for `backend/tests/test_bgp.py`
- `62 passed` for `backend/tests`
- `backend/tests/test_bgp.py` 为 `25 passed`
- `backend/tests` 为 `62 passed`
Covered areas include:
覆盖范围包括：
- normalization
- observation serialization
- enrichment
- detectors, including route leak candidate and path flap
- detectors，包括 route leak candidate 和 path flap
- incident aggregation
- batch anomaly creation
- BGP events/incidents API
- summary endpoints
## Most Relevant Files
## 最相关文件
Backend:
后端：
- `backend/app/models/bgp_observation.py`
- `backend/app/models/bgp_anomaly.py`
@@ -318,7 +318,7 @@ Backend:
- `backend/app/api/v1/bgp.py`
- `backend/app/api/v1/visualization.py`
Frontend:
前端：
- `frontend/src/pages/BGP/BGP.tsx`
- `frontend/public/earth/js/bgp.js`
@@ -327,29 +327,29 @@ Frontend:
- `frontend/public/earth/js/constants.js`
- `frontend/public/earth/index.html`
## Recommended Next Steps
## 推荐下一步
### Next Backend / Detection Priority
### 后端 / 检测优先级
1. Integrate real RPKI validation data.
2. Expand realtime collector coverage and include withdrawals more broadly.
3. Continue refining route leak and path instability detectors with stronger heuristics.
1. 集成真实 RPKI 校验数据。
2. 扩展实时 collector 覆盖范围，并更广泛纳入 withdrawals。
3. 用更强启发式继续完善 route leak 和 path instability detector。
### Next Correlation / Storytelling Priority
### 关联 / 叙事优先级
4. Strengthen incident aggregation semantics and titles.
5. Add weak correlation from incidents to:
- cable corridors
- landing points
4. 强化 incident 聚合语义和标题。
5. 增加 incident 与以下对象的弱关联：
- 海缆走廊
- 登陆点
- IXPs
- other traffic anomaly sources
6. Refine Earth hover/click handoff between collectors and incidents.
- 其它流量异常来源
6. 优化 Earth 中 collector 和 incident 之间的 hover/click 交接。
### Next Visualization Priority
### 可视化优先级
7. Refine regional activity scoring so the activity layer is informative without becoming noisy.
8. Add more incident symbol types as new detectors land.
9. Add a real prefix geography source:
- `IPtoASN / IPtoCountry` as the first practical dataset
- `OpenGeoFeed` as a higher-quality override layer
- registry/whois only as fallback
7. 调整区域 activity scoring，让 activity layer 有信息量但不嘈杂。
8. 随着新 detector 落地，增加更多 incident 符号类型。
9. 增加真实 prefix geography 来源：
- `IPtoASN / IPtoCountry` 作为第一阶段可用数据集
- `OpenGeoFeed` 作为更高质量 override 层
- registry/whois 只作为 fallback