fix: expand bgp pipeline and stabilize backend tests

This commit is contained in:
linkong
2026-03-30 16:13:36 +08:00
parent 2015ab79bd
commit 945786cee5
22 changed files with 1787 additions and 228 deletions

View File

@@ -7,6 +7,39 @@ This project follows the repository versioning rule:
- `feature` -> `+0.1.0`
- `bugfix` -> `+0.0.1`
## 0.21.9
Released: 2026-03-30
### Highlights
- Upgraded BGP ingestion from anomaly-only output to a layered pipeline with raw observations, enrichment context, detector modules, and aggregated incidents.
- Stabilized Earth and visualization data feeds so repeated collections no longer inflate satellite and other entity counts in the globe view.
- Brought the backend test suite back to green and expanded BGP-specific coverage across helpers, aggregation, and API endpoints.
### Added
- Added raw BGP observation persistence in [bgp_observation.py](/home/ray/dev/linkong/planet/backend/app/models/bgp_observation.py).
- Added aggregated BGP incident persistence in [bgp_incident.py](/home/ray/dev/linkong/planet/backend/app/models/bgp_incident.py).
- Added BGP enrichment helpers in [bgp_enrichment.py](/home/ray/dev/linkong/planet/backend/app/services/bgp_enrichment.py) for prefix scope, AS path normalization, ASN organization context, and baseline tracking.
- Added modular BGP detector helpers in [bgp_detectors.py](/home/ray/dev/linkong/planet/backend/app/services/bgp_detectors.py).
- Added BGP incident aggregation helpers in [bgp_incidents.py](/home/ray/dev/linkong/planet/backend/app/services/bgp_incidents.py).
- Added `/api/v1/bgp/incidents` and `/api/v1/bgp/incidents/{id}` in [bgp.py](/home/ray/dev/linkong/planet/backend/app/api/v1/bgp.py).
### Improved
- Improved BGP collectors so [ris_live.py](/home/ray/dev/linkong/planet/backend/app/services/collectors/ris_live.py) and [bgpstream.py](/home/ray/dev/linkong/planet/backend/app/services/collectors/bgpstream.py) now write observations before deriving anomalies.
- Improved anomaly generation in [bgp_common.py](/home/ray/dev/linkong/planet/backend/app/services/collectors/bgp_common.py) by routing signals through enrichment and dedicated detector modules, then rolling them up into incidents.
- Improved the Earth cable layer in [main.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/main.js) so hide/show preserves loaded state correctly and stats remain visible while a layer is hidden.
- Improved backend visualization endpoints in [visualization.py](/home/ray/dev/linkong/planet/backend/app/api/v1/visualization.py) so repeated collections return only the latest unique entity records.
- Improved backend test coverage across [test_bgp.py](/home/ray/dev/linkong/planet/backend/tests/test_bgp.py), [test_api.py](/home/ray/dev/linkong/planet/backend/tests/test_api.py), [test_collectors.py](/home/ray/dev/linkong/planet/backend/tests/test_collectors.py), [test_models.py](/home/ray/dev/linkong/planet/backend/tests/test_models.py), and [test_security.py](/home/ray/dev/linkong/planet/backend/tests/test_security.py).
### Fixed
- Fixed Earth satellite overcounting caused by historical duplicate records being returned by visualization endpoints.
- Fixed BGP event APIs to read from observation records instead of the generic collected-data model.
- Fixed several stale backend tests so they match the current token, timestamp, collector, and API behavior.
## 0.21.8
Released: 2026-03-27