planet/docs/bgp-region-aggregation-plan.md

# BGP Region Aggregation Plan

## Goal

This document refines the current BGP `activity layer` into an implementation-ready regional aggregation design.

Primary product goal:

- turn sparse prefix-level observations, anomalies, and incidents into a readable `regional observability layer`
- keep Earth visually alive during low-incident periods
- make `incident markers` remain the highest-confidence foreground layer instead of replacing them

This layer is not a new collector, detector, or raw storage table.
It is an aggregation/view-model layer:

`observations -> enrichment -> anomalies/incidents -> geography mapping -> region aggregation -> Earth/UI activity layer`

## Why This Layer Exists

Current product gap from [bgp-context.md](/home/ray/dev/linkong/planet/docs/bgp-context.md):

- incident density is naturally low
- anomaly density is higher, but still not enough to keep the globe expressive all the time
- collector presence alone proves coverage, but does not communicate `where routing is currently active or noisy`

So the missing middle layer is:

- `collectors` show that observation exists
- `regions` show where activity is building up
- `incidents` show the specific high-confidence focus events

## Scope

This plan is specifically for:

- a backend aggregation service
- a summary API for console/stats
- a GeoJSON API for Earth rendering
- an Earth background activity layer that supports, but does not replace, incident markers

This plan does not attempt to solve:

- exact prefix geolocation quality
- polygon-heavy geopolitical visualization
- persistent materialized region tables in v1

## Region Layer Definition

Recommended conceptual model:

- `region layer` = background situational awareness
- `incident layer` = focal event markers

That means:

- region activity should answer `where is routing behavior currently active or abnormal`
- incident markers should answer `which concrete event should the user click`

## Recommended Output Model

Suggested backend output object:

## `BGPRegionActivity`

```json
{
  "region_key": "sea",
  "region_name": "Southeast Asia",
  "center_lat": 1.3521,
  "center_lon": 103.8198,
  "observation_count": 128,
  "anomaly_count": 9,
  "incident_count": 2,
  "activity_score": 17.6,
  "status": "incident",
  "affected_prefix_count": 14,
  "affected_asn_count": 6,
  "collector_count": 5,
  "first_seen_at": "2026-04-02T10:00:00Z",
  "last_seen_at": "2026-04-02T10:12:00Z"
}
```

### Fields To Keep In MVP

- `region_key`
- `region_name`
- `center_lat`
- `center_lon`
- `observation_count`
- `anomaly_count`
- `incident_count`
- `activity_score`
- `status`
- `affected_prefix_count`
- `affected_asn_count`
- `collector_count`
- `first_seen_at`
- `last_seen_at`

### Fields To Delay

These are useful, but not required for the first implementation:

- `bounding_box`
- `top_incident_types`
- `top_prefixes`
- polygon geometry

## Region Definition Strategy

### Recommendation

Use a static region-definition table first.

Examples:

- `north_america`
- `south_america`
- `western_europe`
- `eastern_europe`
- `east_asia`
- `southeast_asia`
- `south_asia`
- `middle_east`
- `north_africa`
- `sub_saharan_africa`
- `oceania`

Why this is the right v1 choice:

- stable UI semantics
- strong readability on Earth
- easier debugging and explanation
- lower implementation cost than geohash or H3 grids

### Not Recommended For V1

- geohash cell aggregation
- H3 aggregation
- fine-grained lat/lon bucket maps

Those are more flexible, but they make the map feel fragmented and less explainable.

## Geography Mapping Strategy

Do not reduce the implementation to only `prefix -> exact geo`.

The region layer should follow the same geography-priority logic already implied by the current BGP direction:

1. `prefix_geography`
2. `prefix_scope`
3. `ASN organization region`
4. `collector centroid` fallback

This matters because exact prefix geography will often be incomplete or approximate.
The region layer should stay robust even when only partial enrichment is available.

## Backend Design

Recommended new service file:

- [backend/app/services/bgp_regions.py](/home/ray/dev/linkong/planet/backend/app/services/bgp_regions.py)

Suggested responsibilities:

- `map_record_to_region(...)`
- `aggregate_region_activity(...)`
- `build_region_geojson(...)`
- `resolve_activity_status(...)`
- `compute_activity_score(...)`

### Data Source Inputs

Use a recent rolling window, default `15 minutes`, and aggregate from:

- `BGPObservation`
- `BGPAnomaly`
- active `BGPIncident`

### Aggregation Flow

1. query observations in the time window
2. query anomalies in the same window
3. query active incidents in the same window or active status set
4. resolve each record to a best-effort region
5. accumulate per-region counters
6. compute score and status
7. return region activity list

## Status Model

Recommended status buckets:

- `idle`
- `observing`
- `anomaly`
- `incident`

Suggested rule:

```text
if incident_count > 0: incident
elif anomaly_count > 0: anomaly
elif observation_count > 0: observing
else: idle
```

This aligns well with the current Earth status language and keeps the visual mapping simple.

## Activity Score

The score should be a tunable heuristic, not a fixed truth model.

Recommended v1 formula:

```text
activity_score =
  min(observation_count, 50) * 0.03
  + anomaly_count * 1.2
  + incident_count * 5.0
```

Why cap observations:

- observation volume is usually much larger than anomaly or incident volume
- uncapped observation counts would overwhelm the score
- capped observation counts preserve baseline presence without drowning real abnormality

### Practical Guidance

- treat coefficients as configuration-like constants
- expect to retune after looking at real data
- keep `incident` weight dominant

## API Design

### 1. Summary/List API

Suggested endpoint:

- `/api/v1/bgp/regions/activity`

Response shape:

```json
{
  "window_minutes": 15,
  "regions": []
}
```

Use cases:

- BGP console summaries
- right-side Earth stats
- future region list panels

### 2. GeoJSON API

Suggested endpoint:

- `/api/v1/visualization/geo/bgp-regions`

Response shape:

```json
{
  "type": "FeatureCollection",
  "features": []
}
```

Each feature should include:

- `geometry`
  - v1: `Point`
  - later: optional `Polygon`
- `properties`
  - `region_key`
  - `region_name`
  - `status`
  - `activity_score`
  - `observation_count`
  - `anomaly_count`
  - `incident_count`
  - `affected_prefix_count`
  - `affected_asn_count`
  - `collector_count`

## Earth Rendering Plan

Detailed visual layering guidance is expanded in [bgp-earth-rendering-plan.md](/home/ray/dev/linkong/planet/docs/bgp-earth-rendering-plan.md).

### Layer Relationship

- `region layer` = ambient background activity
- `incident marker` = focal event object

Do not replace incident markers with region markers.

### Region Visual Rules

Suggested mapping:

- `observing`
  - weak glow
  - low pulse or no pulse
- `anomaly`
  - stronger glow
  - more visible pulse
- `incident`
  - strongest regional emphasis
  - but still visually secondary to the incident marker itself

### Region Labels

Good v2 enhancement:

- show region name
- show counts like `2 incidents / 5 anomalies`

This is useful, but should come after the core aggregation and Earth glow layer are working.

## Interaction Model

### Click Region

Recommended detail payload:

- region name
- observation/anomaly/incident counts in the selected window
- affected prefix count
- affected ASN count
- collector count
- recent incidents in the region

### Click Incident

Keep the current incident-detail flow.

Interaction should feel hierarchical:

1. region gives situational context
2. incident gives event focus

## MVP Implementation Order

### Step 1

Define static `REGIONS` in code or config.

### Step 2

Map geography-enriched BGP records into regions using the fallback chain.

### Step 3

Aggregate recent window counts:

- `observation_count`
- `anomaly_count`
- `incident_count`

### Step 4

Compute `activity_score` and `status`.

### Step 5

Expose:

- `/api/v1/bgp/regions/activity`
- `/api/v1/visualization/geo/bgp-regions`

### Step 6

Render region glows on Earth behind incident markers.

## Out Of Scope For MVP

- persistent materialized region tables
- geohash or H3 support
- polygon-filled regional overlays
- detailed top-prefix ranking in the first release
- complicated scoring personalization

## Risks And Constraints

### Geography Quality

Prefix geography is approximate and incomplete.
The region layer must tolerate fallback-based placement.

### Query Cost

Dynamic aggregation is the right v1 choice, but repeated short-window queries may eventually need:

- in-process caching
- scheduled pre-aggregation
- materialized summaries

### UI Overcrowding

If region glow, collector activity, and incidents all become too strong at once, Earth readability will regress.
The region layer must remain supportive, not dominant.

## Final Recommendation

The current BGP roadmap should explicitly add:

- `region aggregation` as the concrete implementation of the missing `activity layer`

The recommended product interpretation is:

- `collectors` prove observation coverage
- `regions` communicate live routing activity and abnormality
- `incidents` remain the clearest high-confidence event objects

In one sentence:

`region aggregation is not a replacement for incidents; it is the situational background that makes sparse incidents feel legible on Earth.`