fix: stabilize earth bgp geography and rendering
This commit is contained in:
linkong
216
docs/prefix-geography-plan.md
Normal file
216
docs/prefix-geography-plan.md
Normal file
@@ -0,0 +1,216 @@
|
||||
# Prefix Geography Plan
|
||||
|
||||
## Goal
|
||||
|
||||
Make Earth BGP incidents `prefix-centric` instead of `collector-centric`.
|
||||
|
||||
The map should primarily answer:
|
||||
|
||||
- where a prefix-related event is likely centered
|
||||
- which regions the prefix is likely associated with
|
||||
- which collectors observed the event as evidence
|
||||
|
||||
It should not continue to imply that the event is located at the collector itself unless no better geography is available.
|
||||
|
||||
## Why Current Geography Is Not Enough
|
||||
|
||||
Current incident geography can still collapse back to collector-derived regions because:
|
||||
|
||||
1. `prefix_scope` is currently built mostly from observed collector regions and historical observation regions.
|
||||
2. `origin_asn_profile` currently comes from `peeringdb_network`, which is useful for ASN footprint hints but not sufficient as a primary prefix location source.
|
||||
3. `collector centroid` is still a common fallback and therefore dominates sparse incidents.
|
||||
|
||||
This makes Earth feel like a collector map with event decorations instead of a prefix impact map.
|
||||
|
||||
## Data Source Layers
|
||||
|
||||
Prefix geography should be built from four layers, ordered by confidence.
|
||||
|
||||
### Layer 1. Prefix-to-country / prefix-to-region
|
||||
|
||||
This is the primary source layer and the current missing piece.
|
||||
|
||||
Recommended sources:
|
||||
|
||||
1. `IPtoASN / IPtoCountry`
|
||||
- URL: <https://iptoasn.com/>
|
||||
- Good fit for this project because it provides downloadable IPv4/IPv6 range-to-ASN and range-to-country mappings.
|
||||
- Best use:
|
||||
- map a prefix to country code
|
||||
- enrich prefixes with coarse regional placement
|
||||
|
||||
2. `OpenGeoFeed`
|
||||
- URL: <https://opengeofeed.org/faq/>
|
||||
- Best use:
|
||||
- override coarse country mappings when the prefix holder publishes a geofeed
|
||||
- provide a more realistic deployment/service region than whois-style registration country
|
||||
|
||||
### Layer 2. Registry allocation fallback
|
||||
|
||||
Use these only as fallback signals, not as a ground-truth physical location.
|
||||
|
||||
Candidate inputs:
|
||||
|
||||
- RIR delegated stats
|
||||
- `inetnum` / `inet6num` whois
|
||||
|
||||
Best use:
|
||||
|
||||
- detect registration country / allocation region
|
||||
- provide fallback when no direct prefix geolocation dataset is available
|
||||
|
||||
### Layer 3. ASN footprint hints
|
||||
|
||||
Existing in this project:
|
||||
|
||||
- `peeringdb_network`
|
||||
- `peeringdb_facility`
|
||||
- `peeringdb_ixp`
|
||||
|
||||
Best use:
|
||||
|
||||
- derive ASN city/country footprint
|
||||
- identify likely exchange/facility regions
|
||||
- act as secondary evidence when prefix-specific geography is unavailable
|
||||
|
||||
### Layer 4. Observation evidence
|
||||
|
||||
Existing in this project:
|
||||
|
||||
- `RIPE RIS Live`
|
||||
- `CAIDA BGPStream Backfill`
|
||||
|
||||
Best use:
|
||||
|
||||
- prove who observed the event
|
||||
- derive affected observation regions
|
||||
- support impact evidence
|
||||
|
||||
This should remain the final fallback and evidence layer, not the primary event geography.
|
||||
|
||||
## Recommended Geography Priority
|
||||
|
||||
The backend should compute incident geography with this order:
|
||||
|
||||
1. `prefix_geography`
|
||||
- prefix-to-country / region / geofeed-backed result
|
||||
2. `asn_region`
|
||||
- ASN organization / facility / IXP footprint
|
||||
3. `collector_centroid`
|
||||
- observed collector regions only as final fallback
|
||||
|
||||
Returned GeoJSON should keep exposing the selected mode through:
|
||||
|
||||
- `geography_mode = prefix_geography | asn_region | collector_centroid`
|
||||
|
||||
## Proposed Backend Changes
|
||||
|
||||
### 1. Add a dedicated prefix geography dataset
|
||||
|
||||
New datasource candidates:
|
||||
|
||||
- `ip2asn_prefix_geo`
|
||||
- optionally `opengeofeed_prefix_geo`
|
||||
|
||||
Suggested storage model:
|
||||
|
||||
- keep downloaded rows in `CollectedData` first for speed of integration
|
||||
- later move to a dedicated table if lookup volume grows
|
||||
|
||||
Minimum normalized fields:
|
||||
|
||||
- `range_start`
|
||||
- `range_end`
|
||||
- `prefix`
|
||||
- `country`
|
||||
- `continent`
|
||||
- `asn`
|
||||
- `as_name`
|
||||
- `source`
|
||||
- `confidence`
|
||||
|
||||
### 2. Add prefix geography enrichment
|
||||
|
||||
Extend:
|
||||
|
||||
- `backend/app/services/bgp_enrichment.py`
|
||||
|
||||
New enrichment payload should include:
|
||||
|
||||
- `prefix_geography`
|
||||
- `country`
|
||||
- `continent`
|
||||
- `regions`
|
||||
- `source`
|
||||
- `confidence`
|
||||
|
||||
This should be separate from the current `prefix_scope`.
|
||||
|
||||
Suggested distinction:
|
||||
|
||||
- `prefix_scope`
|
||||
- observation-derived scope hint
|
||||
- `prefix_geography`
|
||||
- prefix-centric geography estimate
|
||||
|
||||
### 3. Update incident visualization geography selection
|
||||
|
||||
Extend:
|
||||
|
||||
- `backend/app/api/v1/visualization.py`
|
||||
|
||||
Selection order:
|
||||
|
||||
1. `prefix_geography.regions`
|
||||
2. ASN geography hints from PeeringDB-derived profile
|
||||
3. observation-derived `affected_regions`
|
||||
|
||||
### 4. Keep evidence visible in the frontend
|
||||
|
||||
Earth should distinguish:
|
||||
|
||||
- event center = prefix geography estimate
|
||||
- evidence lines / collectors = observation proof
|
||||
|
||||
This keeps the event meaningful for non-expert users without losing collector evidence.
|
||||
|
||||
## Earth UX Result
|
||||
|
||||
After this change, a user should see:
|
||||
|
||||
- an incident marker near the estimated affected prefix region
|
||||
- collectors as supporting evidence, not as the event center itself
|
||||
- cables / landing points / nearby infrastructure as weak correlation around the estimated region
|
||||
|
||||
This makes BGP incidents readable as “where the event is likely happening or affecting”, instead of “which station saw it”.
|
||||
|
||||
## Implementation Order
|
||||
|
||||
### Phase 1
|
||||
|
||||
1. Add `IPtoASN / IPtoCountry` datasource support
|
||||
2. Normalize rows into lookup-friendly format
|
||||
3. Enrich BGP events with `prefix_geography`
|
||||
4. Switch incident geography priority to prefer `prefix_geography`
|
||||
|
||||
### Phase 2
|
||||
|
||||
5. Add `OpenGeoFeed` support
|
||||
6. Let geofeed override coarse country-level prefix geography
|
||||
7. Add confidence scoring per geography source
|
||||
|
||||
### Phase 3
|
||||
|
||||
8. Add RIR / whois fallback
|
||||
9. Add better ASN regional footprint from PeeringDB facilities / IXPs
|
||||
10. Refine Earth visual semantics for prefix geography vs observation evidence
|
||||
|
||||
## Recommendation
|
||||
|
||||
The best next engineering move is:
|
||||
|
||||
1. integrate `IPtoASN / IPtoCountry`
|
||||
2. model `prefix_geography` separately from `prefix_scope`
|
||||
3. only then continue refining incident map placement
|
||||
|
||||
Without this layer, any further Earth tuning will still be constrained by collector-centric data.
|
||||
Reference in New Issue
Block a user