fix: stabilize earth bgp geography and rendering

This commit is contained in:
linkong
2026-04-02 15:36:20 +08:00
parent 07e4f519a1
commit e5fec8ba3d
26 changed files with 1788 additions and 137 deletions

View File

@@ -6,7 +6,17 @@ The BGP module is being evolved from an anomaly-only demo into a layered observa
`raw observations -> enrichment -> detectors -> incidents -> console/Earth visualization`
The practical product goal is to turn low-level BGP control-plane changes into understandable network situation events with collector coverage, impact regions, and incident-centric visualization.
The practical product goal is no longer just to "show incidents on the globe". The current product objective is:
1. keep BGP visually present on Earth even when incident density is low
2. make incidents clearly feel like a higher-confidence layer than anomalies
3. show that the observation network is still active even when there are no active incidents
In practice, that means Earth should behave like an observability surface, not only an incident map:
- `collectors` show that observation is happening
- `activity` shows where routing state is currently active or noisy
- `incidents` become the highest-confidence focus layer
## Current Backend Architecture
@@ -135,16 +145,26 @@ Current design:
2. Incident markers are now the primary Earth BGP markers.
3. If there are no incidents, Earth falls back to anomaly markers.
4. If there are no anomalies either, collectors still provide presence.
5. A dedicated `activity layer` now adds:
- per-collector recent 15-minute activity halos
- clustered regional activity hints derived from active collectors
6. Incident markers now use:
- symbol-driven event cores
- outward ring pulses
- reduced diffuse glow compared with older Earth builds
5. The right-side stats now show:
- BGP events
- collector count
- BGP status summary
This is directionally correct, but still incomplete for low-event-density periods. Right now Earth can still feel too quiet when incidents are sparse because the system lacks a dedicated `activity layer` between raw observation and incident focus.
Current BGP status strategy:
- incidents present: show active incident count
- no incidents but anomalies present: show active anomaly count
- no incidents/anomalies but collectors present: show `当前无活跃事件`
- no incidents but anomalies present: show active anomaly count, plus active observation regions when available
- no incidents/anomalies but activity present: show `观测网络运行中`
- no incidents/anomalies but collectors present: show `观测网络运行中 · 当前未发现聚合级事件`
- no BGP data at all: show `暂无观测数据`
Earth info-card strategy:
@@ -152,6 +172,81 @@ Earth info-card strategy:
- `bgp` card is now incident-centric in wording
- `bgp_collector` card shows collector location and current event count
## Current Product Gap
The main product gap is not architecture correctness. It is low-density visualization strategy.
Current reality:
- incident count is naturally much lower than anomaly count
- that is expected, because incidents are aggregated and de-noised
- but incident-first rendering makes the Earth view look too quiet unless there is another always-available activity layer
So the immediate next milestone is:
`event map -> observability map`
That means Earth needs three simultaneously readable layers:
1. `observation layer`
- collectors
- recent collector activity
- baseline coverage
2. `activity layer`
- recent event density
- anomaly/noise hotspots
- regional activity scoring
- incident presence bonus
3. `incident layer`
- sparse but highly legible, high-confidence event objects
- symbol-driven markers
- outward ring pulse instead of broad diffuse glow
## Incident Visual Direction
The Earth `incident` layer should not read like a large glowing patch. It should read like a compact, high-confidence event focus.
Design principles:
1. `incident` markers should use a strong primary symbol
- the symbol shape should carry type meaning where possible
- examples:
- `origin_change`: triangle-like warning marker
- `mass_withdrawal`: alert/exclamation-style marker
- `more_specific_burst`: split/radiating marker
2. emphasis should come from outward ring pulses, not area flooding
- use a compact hot core
- use one or more expanding ring pulses
- avoid broad luminous blobs that make the event center feel vague
3. `collector` and `incident` must stay visually distinct
- collectors are observation infrastructure
- incidents are extracted event focus
- collector activity should stay quieter than incident pulse language
4. calm periods still need observability presence
- collectors and activity layers should keep the map alive
- once incidents appear, they should clearly dominate nearby BGP visuals
5. incident geography should become `prefix-centric`
- collectors should remain evidence sources, not the primary event location
- preferred geography priority:
- `prefix_geography`
- `prefix_scope`
- `ASN organization region`
- `collector centroid` as final fallback
- `prefix_scope` should remain an observation-derived scope hint
- a new `prefix_geography` layer should be introduced for actual prefix-centric placement
Reference inspiration:
- `World Monitor`
- sparse event symbols
- compact centers
- ring-like outward pulses
- stronger incident legibility than diffuse glow
## Current Console Behavior
Relevant page:
@@ -188,14 +283,15 @@ BGP-specific tests live in:
Verified status at this point:
- `17 passed`
- `25 passed` for `backend/tests/test_bgp.py`
- `62 passed` for `backend/tests`
Covered areas include:
- normalization
- observation serialization
- enrichment
- detectors
- detectors, including route leak candidate and path flap
- incident aggregation
- batch anomaly creation
- BGP events/incidents API
@@ -226,9 +322,14 @@ Frontend:
## Recommended Next Steps
1. Expand realtime collector coverage and include withdrawals more broadly.
2. Integrate real RPKI validation data.
3. Improve route leak and path instability detectors.
### Next Backend / Detection Priority
1. Integrate real RPKI validation data.
2. Expand realtime collector coverage and include withdrawals more broadly.
3. Continue refining route leak and path instability detectors with stronger heuristics.
### Next Correlation / Storytelling Priority
4. Strengthen incident aggregation semantics and titles.
5. Add weak correlation from incidents to:
- cable corridors
@@ -236,3 +337,12 @@ Frontend:
- IXPs
- other traffic anomaly sources
6. Refine Earth hover/click handoff between collectors and incidents.
### Next Visualization Priority
7. Refine regional activity scoring so the activity layer is informative without becoming noisy.
8. Add more incident symbol types as new detectors land.
9. Add a real prefix geography source:
- `IPtoASN / IPtoCountry` as the first practical dataset
- `OpenGeoFeed` as a higher-quality override layer
- registry/whois only as fallback