239 lines
6.4 KiB
Markdown
239 lines
6.4 KiB
Markdown
# BGP Context
|
|
|
|
## Current Goal
|
|
|
|
The BGP module is being evolved from an anomaly-only demo into a layered observability pipeline:
|
|
|
|
`raw observations -> enrichment -> detectors -> incidents -> console/Earth visualization`
|
|
|
|
The practical product goal is to turn low-level BGP control-plane changes into understandable network situation events with collector coverage, impact regions, and incident-centric visualization.
|
|
|
|
## Current Backend Architecture
|
|
|
|
### Data Layers
|
|
|
|
1. `BGPObservation`
|
|
- File: `backend/app/models/bgp_observation.py`
|
|
- Purpose: store normalized raw routing observations from live/history sources.
|
|
- Typical fields:
|
|
- `source`
|
|
- `collector`
|
|
- `peer_asn`
|
|
- `peer_ip`
|
|
- `prefix`
|
|
- `event_type`
|
|
- `as_path`
|
|
- `origin_asn`
|
|
- `next_hop`
|
|
- `communities`
|
|
- `observed_at`
|
|
- `raw_payload`
|
|
- `collector_geo`
|
|
- `ingest_batch_id`
|
|
|
|
2. `BGPAnomaly`
|
|
- File: `backend/app/models/bgp_anomaly.py`
|
|
- Purpose: hold atomic detector outputs.
|
|
- Current detector output types include:
|
|
- `origin_change`
|
|
- `more_specific_burst`
|
|
- `mass_withdrawal`
|
|
|
|
3. `BGPIncident`
|
|
- File: `backend/app/models/bgp_incident.py`
|
|
- Purpose: aggregate atomic anomalies into incident-level objects for humans and the UI.
|
|
|
|
### Pipeline
|
|
|
|
Main flow is currently anchored in:
|
|
|
|
- `backend/app/services/collectors/bgp_common.py`
|
|
- `backend/app/services/bgp_enrichment.py`
|
|
- `backend/app/services/bgp_detectors.py`
|
|
- `backend/app/services/bgp_incidents.py`
|
|
|
|
Operational flow:
|
|
|
|
1. collectors fetch raw BGP data
|
|
2. `normalize_bgp_event()` standardizes payloads
|
|
3. observations are persisted to `bgp_observations`
|
|
4. enrichment augments events with analysis context
|
|
5. detectors create `bgp_anomalies`
|
|
6. incident aggregation rolls anomalies up into `bgp_incidents`
|
|
|
|
### Current Ingest Sources
|
|
|
|
1. `RIPE RIS Live`
|
|
- Collector file: `backend/app/services/collectors/ris_live.py`
|
|
- Used for realtime observation flow.
|
|
|
|
2. `CAIDA BGPStream Backfill`
|
|
- Collector file: `backend/app/services/collectors/bgpstream.py`
|
|
- Used as history/backfill entry point.
|
|
|
|
## Current Enrichment Status
|
|
|
|
Implemented enrichment skeleton in:
|
|
|
|
- `backend/app/services/bgp_enrichment.py`
|
|
|
|
Current enrichments:
|
|
|
|
- prefix family / prefix length
|
|
- supernet / more-specific derivation
|
|
- deduplicated AS path
|
|
- path prepending hints
|
|
- collector region info
|
|
- prefix baseline hints
|
|
- new-origin detection
|
|
- ASN organization profile from PeeringDB where available
|
|
- prefix scope / impacted region hints
|
|
|
|
Current limitation:
|
|
|
|
- `RPKI` is still placeholder-only and returns `unknown`
|
|
- no real ROA validation source is integrated yet
|
|
|
|
## Current API Surface
|
|
|
|
Primary API file:
|
|
|
|
- `backend/app/api/v1/bgp.py`
|
|
|
|
Available endpoints:
|
|
|
|
- `/api/v1/bgp/events`
|
|
- `/api/v1/bgp/events/summary`
|
|
- `/api/v1/bgp/events/{id}`
|
|
- `/api/v1/bgp/anomalies`
|
|
- `/api/v1/bgp/anomalies/summary`
|
|
- `/api/v1/bgp/anomalies/{id}`
|
|
- `/api/v1/bgp/incidents`
|
|
- `/api/v1/bgp/incidents/summary`
|
|
- `/api/v1/bgp/incidents/{id}`
|
|
|
|
Visualization GeoJSON endpoints:
|
|
|
|
- `backend/app/api/v1/visualization.py`
|
|
- `/api/v1/visualization/geo/bgp-collectors`
|
|
- `/api/v1/visualization/geo/bgp-anomalies`
|
|
- `/api/v1/visualization/geo/bgp-incidents`
|
|
|
|
## Current Earth Behavior
|
|
|
|
Relevant files:
|
|
|
|
- `frontend/public/earth/js/bgp.js`
|
|
- `frontend/public/earth/js/main.js`
|
|
- `frontend/public/earth/js/info-card.js`
|
|
- `frontend/public/earth/js/constants.js`
|
|
- `frontend/public/earth/index.html`
|
|
|
|
Current design:
|
|
|
|
1. Collectors are always shown when BGP is enabled.
|
|
2. Incident markers are now the primary Earth BGP markers.
|
|
3. If there are no incidents, Earth falls back to anomaly markers.
|
|
4. If there are no anomalies either, collectors still provide presence.
|
|
5. The right-side stats now show:
|
|
- BGP events
|
|
- collector count
|
|
- BGP status summary
|
|
|
|
Current BGP status strategy:
|
|
|
|
- incidents present: show active incident count
|
|
- no incidents but anomalies present: show active anomaly count
|
|
- no incidents/anomalies but collectors present: show `当前无活跃事件`
|
|
- no BGP data at all: show `暂无观测数据`
|
|
|
|
Earth info-card strategy:
|
|
|
|
- `bgp` card is now incident-centric in wording
|
|
- `bgp_collector` card shows collector location and current event count
|
|
|
|
## Current Console Behavior
|
|
|
|
Relevant page:
|
|
|
|
- `frontend/src/pages/BGP/BGP.tsx`
|
|
|
|
Current BGP console page has three levels:
|
|
|
|
1. observation summary
|
|
- total events
|
|
- collector count
|
|
- prefix count
|
|
|
|
2. incident summary and incident table
|
|
|
|
3. anomaly detail table plus recent observation events
|
|
|
|
This means the BGP page still has useful signal even when there are zero anomalies.
|
|
|
|
## Known Product/Engineering Boundaries
|
|
|
|
1. The current system is still closer to an event board than a full BGP sensing platform.
|
|
2. RIS coverage still needs to expand beyond narrow subscription scope.
|
|
3. BGPStream history is still not full MRT-to-prefix decoded analytics.
|
|
4. Collector geography still depends heavily on static RIPE RIS mappings.
|
|
5. Incident-to-cable/IXP/region association is still weak and early-stage.
|
|
6. Earth currently visualizes logical observation/impact structure, not true physical traffic paths.
|
|
|
|
## Test Status
|
|
|
|
BGP-specific tests live in:
|
|
|
|
- `backend/tests/test_bgp.py`
|
|
|
|
Verified status at this point:
|
|
|
|
- `17 passed`
|
|
|
|
Covered areas include:
|
|
|
|
- normalization
|
|
- observation serialization
|
|
- enrichment
|
|
- detectors
|
|
- incident aggregation
|
|
- batch anomaly creation
|
|
- BGP events/incidents API
|
|
- summary endpoints
|
|
|
|
## Most Relevant Files
|
|
|
|
Backend:
|
|
|
|
- `backend/app/models/bgp_observation.py`
|
|
- `backend/app/models/bgp_anomaly.py`
|
|
- `backend/app/models/bgp_incident.py`
|
|
- `backend/app/services/collectors/bgp_common.py`
|
|
- `backend/app/services/bgp_enrichment.py`
|
|
- `backend/app/services/bgp_detectors.py`
|
|
- `backend/app/services/bgp_incidents.py`
|
|
- `backend/app/api/v1/bgp.py`
|
|
- `backend/app/api/v1/visualization.py`
|
|
|
|
Frontend:
|
|
|
|
- `frontend/src/pages/BGP/BGP.tsx`
|
|
- `frontend/public/earth/js/bgp.js`
|
|
- `frontend/public/earth/js/main.js`
|
|
- `frontend/public/earth/js/info-card.js`
|
|
- `frontend/public/earth/js/constants.js`
|
|
- `frontend/public/earth/index.html`
|
|
|
|
## Recommended Next Steps
|
|
|
|
1. Expand realtime collector coverage and include withdrawals more broadly.
|
|
2. Integrate real RPKI validation data.
|
|
3. Improve route leak and path instability detectors.
|
|
4. Strengthen incident aggregation semantics and titles.
|
|
5. Add weak correlation from incidents to:
|
|
- cable corridors
|
|
- landing points
|
|
- IXPs
|
|
- other traffic anomaly sources
|
|
6. Refine Earth hover/click handoff between collectors and incidents.
|