planet/docs/bgp-context.md

BGP Context

Current Goal

The BGP module is being evolved from an anomaly-only demo into a layered observability pipeline:

raw observations -> enrichment -> detectors -> incidents -> console/Earth visualization

The practical product goal is to turn low-level BGP control-plane changes into understandable network situation events with collector coverage, impact regions, and incident-centric visualization.

Current Backend Architecture

Data Layers

1. BGPObservation

   • File: backend/app/models/bgp_observation.py
   • Purpose: store normalized raw routing observations from live/history sources.
   • Typical fields:
     • source
     • collector
     • peer_asn
     • peer_ip
     • prefix
     • event_type
     • as_path
     • origin_asn
     • next_hop
     • communities
     • observed_at
     • raw_payload
     • collector_geo
     • ingest_batch_id

2. BGPAnomaly

   • File: backend/app/models/bgp_anomaly.py
   • Purpose: hold atomic detector outputs.
   • Current detector output types include:
     • origin_change
     • more_specific_burst
     • mass_withdrawal

3. BGPIncident

   • File: backend/app/models/bgp_incident.py
   • Purpose: aggregate atomic anomalies into incident-level objects for humans and the UI.

Pipeline

Main flow is currently anchored in:

• backend/app/services/collectors/bgp_common.py
• backend/app/services/bgp_enrichment.py
• backend/app/services/bgp_detectors.py
• backend/app/services/bgp_incidents.py

Operational flow:

1. collectors fetch raw BGP data
2. normalize_bgp_event() standardizes payloads
3. observations are persisted to bgp_observations
4. enrichment augments events with analysis context
5. detectors create bgp_anomalies
6. incident aggregation rolls anomalies up into bgp_incidents

Current Ingest Sources

1. RIPE RIS Live

   • Collector file: backend/app/services/collectors/ris_live.py
   • Used for realtime observation flow.

2. CAIDA BGPStream Backfill

   • Collector file: backend/app/services/collectors/bgpstream.py
   • Used as history/backfill entry point.

Current Enrichment Status

Implemented enrichment skeleton in:

• backend/app/services/bgp_enrichment.py

Current enrichments:

• prefix family / prefix length
• supernet / more-specific derivation
• deduplicated AS path
• path prepending hints
• collector region info
• prefix baseline hints
• new-origin detection
• ASN organization profile from PeeringDB where available
• prefix scope / impacted region hints

Current limitation:

• RPKI is still placeholder-only and returns unknown
• no real ROA validation source is integrated yet

Current API Surface

Primary API file:

• backend/app/api/v1/bgp.py

Available endpoints:

• /api/v1/bgp/events
• /api/v1/bgp/events/summary
• /api/v1/bgp/events/{id}
• /api/v1/bgp/anomalies
• /api/v1/bgp/anomalies/summary
• /api/v1/bgp/anomalies/{id}
• /api/v1/bgp/incidents
• /api/v1/bgp/incidents/summary
• /api/v1/bgp/incidents/{id}

Visualization GeoJSON endpoints:

• backend/app/api/v1/visualization.py
• /api/v1/visualization/geo/bgp-collectors
• /api/v1/visualization/geo/bgp-anomalies
• /api/v1/visualization/geo/bgp-incidents

Current Earth Behavior

Relevant files:

• frontend/public/earth/js/bgp.js
• frontend/public/earth/js/main.js
• frontend/public/earth/js/info-card.js
• frontend/public/earth/js/constants.js
• frontend/public/earth/index.html

Current design:

1. Collectors are always shown when BGP is enabled.
2. Incident markers are now the primary Earth BGP markers.
3. If there are no incidents, Earth falls back to anomaly markers.
4. If there are no anomalies either, collectors still provide presence.
5. The right-side stats now show:
   • BGP events
   • collector count
   • BGP status summary

Current BGP status strategy:

• incidents present: show active incident count
• no incidents but anomalies present: show active anomaly count
• no incidents/anomalies but collectors present: show 当前无活跃事件
• no BGP data at all: show 暂无观测数据

Earth info-card strategy:

• bgp card is now incident-centric in wording
• bgp_collector card shows collector location and current event count

Current Console Behavior

Relevant page:

• frontend/src/pages/BGP/BGP.tsx

Current BGP console page has three levels:

1. observation summary

   • total events
   • collector count
   • prefix count

2. incident summary and incident table

3. anomaly detail table plus recent observation events

This means the BGP page still has useful signal even when there are zero anomalies.

Known Product/Engineering Boundaries

1. The current system is still closer to an event board than a full BGP sensing platform.
2. RIS coverage still needs to expand beyond narrow subscription scope.
3. BGPStream history is still not full MRT-to-prefix decoded analytics.
4. Collector geography still depends heavily on static RIPE RIS mappings.
5. Incident-to-cable/IXP/region association is still weak and early-stage.
6. Earth currently visualizes logical observation/impact structure, not true physical traffic paths.

Test Status

BGP-specific tests live in:

• backend/tests/test_bgp.py

Verified status at this point:

• 17 passed

Covered areas include:

• normalization
• observation serialization
• enrichment
• detectors
• incident aggregation
• batch anomaly creation
• BGP events/incidents API
• summary endpoints

Most Relevant Files

Backend:

• backend/app/models/bgp_observation.py
• backend/app/models/bgp_anomaly.py
• backend/app/models/bgp_incident.py
• backend/app/services/collectors/bgp_common.py
• backend/app/services/bgp_enrichment.py
• backend/app/services/bgp_detectors.py
• backend/app/services/bgp_incidents.py
• backend/app/api/v1/bgp.py
• backend/app/api/v1/visualization.py

Frontend:

• frontend/src/pages/BGP/BGP.tsx
• frontend/public/earth/js/bgp.js
• frontend/public/earth/js/main.js
• frontend/public/earth/js/info-card.js
• frontend/public/earth/js/constants.js
• frontend/public/earth/index.html

Recommended Next Steps

1. Expand realtime collector coverage and include withdrawals more broadly.
2. Integrate real RPKI validation data.
3. Improve route leak and path instability detectors.
4. Strengthen incident aggregation semantics and titles.
5. Add weak correlation from incidents to:
   • cable corridors
   • landing points
   • IXPs
   • other traffic anomaly sources
6. Refine Earth hover/click handoff between collectors and incidents.