diff --git a/docs/bgp-context.md b/docs/bgp-context.md index 5f598755..99640091 100644 --- a/docs/bgp-context.md +++ b/docs/bgp-context.md @@ -182,6 +182,8 @@ Current reality: - that is expected, because incidents are aggregated and de-noised - but incident-first rendering makes the Earth view look too quiet unless there is another always-available activity layer +Implementation detail for the recommended `activity layer` is expanded in [bgp-region-aggregation-plan.md](/home/ray/dev/linkong/planet/docs/bgp-region-aggregation-plan.md). + So the immediate next milestone is: `event map -> observability map` diff --git a/docs/bgp-earth-rendering-plan.md b/docs/bgp-earth-rendering-plan.md new file mode 100644 index 00000000..c303a2dd --- /dev/null +++ b/docs/bgp-earth-rendering-plan.md @@ -0,0 +1,296 @@ +# BGP Earth Rendering Plan + +## Goal + +This document defines how the BGP `region activity layer` and `incident layer` should coexist on Earth without conflicting. + +The main question it answers is: + +- how to add a regional observability background layer +- without weakening the current incident-first event focus + +## Core Principle + +The Earth design should follow a strict semantic hierarchy: + +- `collector layer` = observation infrastructure +- `region activity layer` = background situational awareness +- `incident layer` = focal high-confidence event objects + +In short: + +- collectors prove the network is observing +- regions show where routing behavior is active or abnormal +- incidents show the concrete event worth clicking + +Region aggregation is therefore not a replacement for incident rendering. +It is the context layer that makes sparse incident markers legible. + +## Rendering Hierarchy + +Recommended visual stack order: + +1. collector network / collector halos +2. region activity glow +3. incident markers and incident pulses + +This ordering should always hold. + +Why: + +- collectors should stay visible but quiet +- regions should create ambient activity presence +- incidents must remain the first thing users notice as a concrete event + +## Role Separation + +### Region Layer + +The region layer answers: + +- where is routing activity building up +- where is there current noise or instability +- which part of the world is currently worth looking at + +The region layer should feel: + +- broad +- ambient +- low-frequency +- contextual + +### Incident Layer + +The incident layer answers: + +- which exact event should the user inspect +- where is the highest-confidence routing event located right now + +The incident layer should feel: + +- sharp +- compact +- high-contrast +- intentionally clickable + +## Non-Conflict Rules + +To avoid visual and semantic conflict, these implementation rules should be treated as hard constraints: + +1. region markers must not use the same symbol language as incidents +2. region emphasis must stay weaker than incident emphasis +3. region animation frequency must stay lower than incident animation frequency +4. incident markers must always render above region glows +5. region layer should support the event, not compete with it + +If a user notices the region layer first but misses the incident marker, the region layer is too strong. + +If a user only sees isolated incident points and cannot feel broader activity context, the region layer is too weak. + +## Region Rendering Rules + +The region layer should not be rendered as a second kind of incident point. + +Recommended representation: + +- diffuse glow +- halo +- low-detail pulse +- soft center, not a sharp icon + +### Status Mapping + +#### `observing` + +- weak glow +- cool color, such as cyan or blue +- little to no pulse +- purpose: keep the globe alive during calm periods + +#### `anomaly` + +- stronger glow +- warmer color, such as amber +- gentle breathing or low-frequency pulse +- purpose: show that a region is experiencing abnormal routing noise + +#### `incident` + +- strongest regional background emphasis +- still clearly weaker than the incident marker itself +- purpose: lift the surrounding area so the focal event does not feel isolated + +### Region Visual Characteristics + +Recommended properties: + +- large radius +- low opacity +- soft edge +- low-contrast outline or no outline +- low pulse amplitude + +Avoid: + +- sharp symbol shapes +- strong icon silhouettes +- bright hard-edged centers +- incident-like pulse language + +## Incident Rendering Rules + +The incident layer should remain visually sharper and more explicit than region activity. + +Recommended qualities: + +- clear event symbol +- compact hot core +- one or two outward ring pulses +- high contrast +- clear click target + +The incident layer should read as: + +- focal +- deliberate +- high-confidence + +while the region layer should read as: + +- contextual +- ambient +- supporting + +## Region And Incident In The Same Area + +When a region contains one or more incidents: + +- the region glow may intensify +- but the incident marker must remain the dominant local feature + +Interpretation should be: + +- `region` says this area is in an event state +- `incident marker` says this is the concrete event object + +So a region with `incident` status is not itself the event marker. +It is the background state around the event. + +## Interaction Model + +Interaction should also preserve hierarchy. + +### Click Region + +Open a regional situation view, such as: + +- region name +- observation count +- anomaly count +- incident count +- affected prefix count +- affected ASN count +- recent incidents in the region + +### Click Incident + +Keep the current incident-focused detail interaction. + +This creates a natural two-step flow: + +1. region gives context +2. incident gives detail + +## Layer Relationship To Existing BGP Elements + +### Collector Layer + +Collectors should remain: + +- quieter than regions +- more infrastructural than semantic +- proof of coverage, not proof of incident + +### Region Layer + +Regions should become: + +- the main ambient activity layer +- the bridge between collectors and incidents +- the answer to low-density map quietness + +### Incident Layer + +Incidents should remain: + +- the most legible event layer +- sparse but dominant +- compact and symbol-driven + +## Practical Visual Test + +Use this test when tuning the Earth implementation: + +### Calm Period + +Expected result: + +- collectors visible +- some weak region glows present +- no region feels alarm-heavy +- globe still feels alive + +### Anomaly Period + +Expected result: + +- one or more regions brighten noticeably +- user can sense the active area before clicking +- still no confusion between region background and incident objects + +### Incident Period + +Expected result: + +- region provides broader context +- incident marker is the first explicit focal object the eye lands on +- user can immediately tell both: + - which region is active + - which specific event to inspect + +## Failure Modes To Avoid + +### Region Too Strong + +Symptoms: + +- incident markers disappear into the glow +- users treat the region center as the main event +- the map feels like area flooding instead of event focus + +### Region Too Weak + +Symptoms: + +- incident markers still feel isolated +- low-incident periods still look visually empty +- users cannot tell where routing activity is generally happening + +### Region Uses Incident Language + +Symptoms: + +- region and incident both look like event markers +- users cannot distinguish context from event + +## Final Design Rule + +The desired reading order is: + +1. see the specific incident marker +2. feel the active region around it +3. understand that collectors and background activity keep the globe alive even during quieter periods + +In one sentence: + +`incident is the point; region is the field.` diff --git a/docs/bgp-region-aggregation-plan.md b/docs/bgp-region-aggregation-plan.md new file mode 100644 index 00000000..f31a20f3 --- /dev/null +++ b/docs/bgp-region-aggregation-plan.md @@ -0,0 +1,422 @@ +# BGP Region Aggregation Plan + +## Goal + +This document refines the current BGP `activity layer` into an implementation-ready regional aggregation design. + +Primary product goal: + +- turn sparse prefix-level observations, anomalies, and incidents into a readable `regional observability layer` +- keep Earth visually alive during low-incident periods +- make `incident markers` remain the highest-confidence foreground layer instead of replacing them + +This layer is not a new collector, detector, or raw storage table. +It is an aggregation/view-model layer: + +`observations -> enrichment -> anomalies/incidents -> geography mapping -> region aggregation -> Earth/UI activity layer` + +## Why This Layer Exists + +Current product gap from [bgp-context.md](/home/ray/dev/linkong/planet/docs/bgp-context.md): + +- incident density is naturally low +- anomaly density is higher, but still not enough to keep the globe expressive all the time +- collector presence alone proves coverage, but does not communicate `where routing is currently active or noisy` + +So the missing middle layer is: + +- `collectors` show that observation exists +- `regions` show where activity is building up +- `incidents` show the specific high-confidence focus events + +## Scope + +This plan is specifically for: + +- a backend aggregation service +- a summary API for console/stats +- a GeoJSON API for Earth rendering +- an Earth background activity layer that supports, but does not replace, incident markers + +This plan does not attempt to solve: + +- exact prefix geolocation quality +- polygon-heavy geopolitical visualization +- persistent materialized region tables in v1 + +## Region Layer Definition + +Recommended conceptual model: + +- `region layer` = background situational awareness +- `incident layer` = focal event markers + +That means: + +- region activity should answer `where is routing behavior currently active or abnormal` +- incident markers should answer `which concrete event should the user click` + +## Recommended Output Model + +Suggested backend output object: + +## `BGPRegionActivity` + +```json +{ + "region_key": "sea", + "region_name": "Southeast Asia", + "center_lat": 1.3521, + "center_lon": 103.8198, + "observation_count": 128, + "anomaly_count": 9, + "incident_count": 2, + "activity_score": 17.6, + "status": "incident", + "affected_prefix_count": 14, + "affected_asn_count": 6, + "collector_count": 5, + "first_seen_at": "2026-04-02T10:00:00Z", + "last_seen_at": "2026-04-02T10:12:00Z" +} +``` + +### Fields To Keep In MVP + +- `region_key` +- `region_name` +- `center_lat` +- `center_lon` +- `observation_count` +- `anomaly_count` +- `incident_count` +- `activity_score` +- `status` +- `affected_prefix_count` +- `affected_asn_count` +- `collector_count` +- `first_seen_at` +- `last_seen_at` + +### Fields To Delay + +These are useful, but not required for the first implementation: + +- `bounding_box` +- `top_incident_types` +- `top_prefixes` +- polygon geometry + +## Region Definition Strategy + +### Recommendation + +Use a static region-definition table first. + +Examples: + +- `north_america` +- `south_america` +- `western_europe` +- `eastern_europe` +- `east_asia` +- `southeast_asia` +- `south_asia` +- `middle_east` +- `north_africa` +- `sub_saharan_africa` +- `oceania` + +Why this is the right v1 choice: + +- stable UI semantics +- strong readability on Earth +- easier debugging and explanation +- lower implementation cost than geohash or H3 grids + +### Not Recommended For V1 + +- geohash cell aggregation +- H3 aggregation +- fine-grained lat/lon bucket maps + +Those are more flexible, but they make the map feel fragmented and less explainable. + +## Geography Mapping Strategy + +Do not reduce the implementation to only `prefix -> exact geo`. + +The region layer should follow the same geography-priority logic already implied by the current BGP direction: + +1. `prefix_geography` +2. `prefix_scope` +3. `ASN organization region` +4. `collector centroid` fallback + +This matters because exact prefix geography will often be incomplete or approximate. +The region layer should stay robust even when only partial enrichment is available. + +## Backend Design + +Recommended new service file: + +- [backend/app/services/bgp_regions.py](/home/ray/dev/linkong/planet/backend/app/services/bgp_regions.py) + +Suggested responsibilities: + +- `map_record_to_region(...)` +- `aggregate_region_activity(...)` +- `build_region_geojson(...)` +- `resolve_activity_status(...)` +- `compute_activity_score(...)` + +### Data Source Inputs + +Use a recent rolling window, default `15 minutes`, and aggregate from: + +- `BGPObservation` +- `BGPAnomaly` +- active `BGPIncident` + +### Aggregation Flow + +1. query observations in the time window +2. query anomalies in the same window +3. query active incidents in the same window or active status set +4. resolve each record to a best-effort region +5. accumulate per-region counters +6. compute score and status +7. return region activity list + +## Status Model + +Recommended status buckets: + +- `idle` +- `observing` +- `anomaly` +- `incident` + +Suggested rule: + +```text +if incident_count > 0: incident +elif anomaly_count > 0: anomaly +elif observation_count > 0: observing +else: idle +``` + +This aligns well with the current Earth status language and keeps the visual mapping simple. + +## Activity Score + +The score should be a tunable heuristic, not a fixed truth model. + +Recommended v1 formula: + +```text +activity_score = + min(observation_count, 50) * 0.03 + + anomaly_count * 1.2 + + incident_count * 5.0 +``` + +Why cap observations: + +- observation volume is usually much larger than anomaly or incident volume +- uncapped observation counts would overwhelm the score +- capped observation counts preserve baseline presence without drowning real abnormality + +### Practical Guidance + +- treat coefficients as configuration-like constants +- expect to retune after looking at real data +- keep `incident` weight dominant + +## API Design + +### 1. Summary/List API + +Suggested endpoint: + +- `/api/v1/bgp/regions/activity` + +Response shape: + +```json +{ + "window_minutes": 15, + "regions": [] +} +``` + +Use cases: + +- BGP console summaries +- right-side Earth stats +- future region list panels + +### 2. GeoJSON API + +Suggested endpoint: + +- `/api/v1/visualization/geo/bgp-regions` + +Response shape: + +```json +{ + "type": "FeatureCollection", + "features": [] +} +``` + +Each feature should include: + +- `geometry` + - v1: `Point` + - later: optional `Polygon` +- `properties` + - `region_key` + - `region_name` + - `status` + - `activity_score` + - `observation_count` + - `anomaly_count` + - `incident_count` + - `affected_prefix_count` + - `affected_asn_count` + - `collector_count` + +## Earth Rendering Plan + +Detailed visual layering guidance is expanded in [bgp-earth-rendering-plan.md](/home/ray/dev/linkong/planet/docs/bgp-earth-rendering-plan.md). + +### Layer Relationship + +- `region layer` = ambient background activity +- `incident marker` = focal event object + +Do not replace incident markers with region markers. + +### Region Visual Rules + +Suggested mapping: + +- `observing` + - weak glow + - low pulse or no pulse +- `anomaly` + - stronger glow + - more visible pulse +- `incident` + - strongest regional emphasis + - but still visually secondary to the incident marker itself + +### Region Labels + +Good v2 enhancement: + +- show region name +- show counts like `2 incidents / 5 anomalies` + +This is useful, but should come after the core aggregation and Earth glow layer are working. + +## Interaction Model + +### Click Region + +Recommended detail payload: + +- region name +- observation/anomaly/incident counts in the selected window +- affected prefix count +- affected ASN count +- collector count +- recent incidents in the region + +### Click Incident + +Keep the current incident-detail flow. + +Interaction should feel hierarchical: + +1. region gives situational context +2. incident gives event focus + +## MVP Implementation Order + +### Step 1 + +Define static `REGIONS` in code or config. + +### Step 2 + +Map geography-enriched BGP records into regions using the fallback chain. + +### Step 3 + +Aggregate recent window counts: + +- `observation_count` +- `anomaly_count` +- `incident_count` + +### Step 4 + +Compute `activity_score` and `status`. + +### Step 5 + +Expose: + +- `/api/v1/bgp/regions/activity` +- `/api/v1/visualization/geo/bgp-regions` + +### Step 6 + +Render region glows on Earth behind incident markers. + +## Out Of Scope For MVP + +- persistent materialized region tables +- geohash or H3 support +- polygon-filled regional overlays +- detailed top-prefix ranking in the first release +- complicated scoring personalization + +## Risks And Constraints + +### Geography Quality + +Prefix geography is approximate and incomplete. +The region layer must tolerate fallback-based placement. + +### Query Cost + +Dynamic aggregation is the right v1 choice, but repeated short-window queries may eventually need: + +- in-process caching +- scheduled pre-aggregation +- materialized summaries + +### UI Overcrowding + +If region glow, collector activity, and incidents all become too strong at once, Earth readability will regress. +The region layer must remain supportive, not dominant. + +## Final Recommendation + +The current BGP roadmap should explicitly add: + +- `region aggregation` as the concrete implementation of the missing `activity layer` + +The recommended product interpretation is: + +- `collectors` prove observation coverage +- `regions` communicate live routing activity and abnormality +- `incidents` remain the clearest high-confidence event objects + +In one sentence: + +`region aggregation is not a replacement for incidents; it is the situational background that makes sparse incidents feel legible on Earth.`