release: bump version to 0.49.0
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
linkong
200
docs/technical/en/location-pipeline-development.md
Normal file
200
docs/technical/en/location-pipeline-development.md
Normal file
@@ -0,0 +1,200 @@
|
||||
# Shared Location Resolution Pipeline Development Guide
|
||||
|
||||
`backend/app/services/location/` is the shared abstraction for any "given a record, decide its lat/lon" workflow. Compute centers, BGP collectors, and BGP events now run on this pipeline. Future entities such as satellite ground stations, user-claimed points, and IXP facilities should plug in here instead of creating another geocoding path.
|
||||
|
||||
For the user workflow, see [Earth Location Candidate Collection User Guide](/home/ray/dev/linkong/planet/docs/technical/en/location-pipeline-user.md).
|
||||
|
||||
## Design Goals
|
||||
|
||||
Historically compute centers had their own four-tier chain, BGP collectors used a hard-coded dictionary, and BGP events inherited collector coordinates. These implementations did not share code, and new algorithms had no stable insertion point.
|
||||
|
||||
The refactored rules:
|
||||
|
||||
- Share the `LocationResolver` protocol and `LocationPipeline` orchestrator.
|
||||
- Domain modules only build `LocationQuery` and choose resolver order.
|
||||
- New algorithms join by adding resolver classes, without changing ingestion, API, or frontend envelopes.
|
||||
- Earth renders only city-level or better locations.
|
||||
- Local JSON registries are not runtime candidate sources for compute centers or BGP collectors; persisted location facts live in database dimension tables.
|
||||
|
||||
## Core Interfaces
|
||||
|
||||
```python
|
||||
@dataclass(frozen=True)
|
||||
class LocationQuery:
|
||||
name: str | None
|
||||
aliases: tuple[str, ...]
|
||||
city: str | None
|
||||
country: str | None
|
||||
region: str | None
|
||||
source_latitude: float | None
|
||||
source_longitude: float | None
|
||||
extra: Mapping[str, Any]
|
||||
```
|
||||
|
||||
```python
|
||||
@dataclass(frozen=True)
|
||||
class LocationCandidate:
|
||||
latitude: float
|
||||
longitude: float
|
||||
display_name: str
|
||||
precision: str
|
||||
confidence: float
|
||||
source: str
|
||||
needs_confirmation: bool
|
||||
matched_fields: tuple[str, ...]
|
||||
suggested_registry_entry: dict | None
|
||||
```
|
||||
|
||||
```python
|
||||
class LocationResolver(Protocol):
|
||||
name: str
|
||||
def resolve(self, query: LocationQuery) -> ResolverOutput: ...
|
||||
```
|
||||
|
||||
`LocationPipeline.collect_candidates()` returns sorted candidates plus `attempted_queries`; `resolve_best()` returns the best candidate with diagnostics. The default sort key ranks source, precision, and confidence, then deduplicates candidates with the same source and rounded coordinates.
|
||||
|
||||
## Built-In Resolvers
|
||||
|
||||
| Resolver | File | Responsibility |
|
||||
| --- | --- | --- |
|
||||
| `SourceCoordinatesResolver` | `resolvers/source_coordinates.py` | Emits `precision="precise"` when the record already has lat/lon |
|
||||
| `RegistryResolver` | `resolvers/registry.py` | Legacy generic resolver; current compute-center and BGP runtime paths do not use it to generate candidates |
|
||||
| `NominatimResolver` | `resolvers/nominatim.py` | Runs a domain query plan against Nominatim with LRU cache and rate limiting |
|
||||
| `InheritFromAnotherEntityResolver` | `resolvers/inherit.py` | Wraps an externally resolved entity location as a candidate |
|
||||
|
||||
`RegistryResolver` remains available for future controlled import scenarios, but it should not be reconnected as a hard-coded hint source for compute centers or BGP. Matching common fields such as `operator` or `city` was the main reason multiple entities could collapse onto the same point.
|
||||
|
||||
## Current Domain Pipelines
|
||||
|
||||
### Compute Centers
|
||||
|
||||
Entry points:
|
||||
|
||||
- [compute_center_locations.py](/home/ray/dev/linkong/planet/backend/app/services/compute_center_locations.py)
|
||||
|
||||
Resolver order:
|
||||
|
||||
```python
|
||||
SourceCoordinatesResolver()
|
||||
StoredComputeCenterLocationResolver()
|
||||
```
|
||||
|
||||
The main map startup path is source coordinates first, then the database-backed current-location table. The table is `compute_center_locations`, keyed by `(source, source_id)`, and stores manually accepted locations or true coordinates migrated from source records. `init_db()` only migrates source records that already contain real coordinates; it does not import old hard-coded hints and does not run ROR, Nominatim, or LLM geocoding during startup.
|
||||
|
||||
Candidate collection is intentionally separate from rendering. `collect_location_candidates()` builds ROR and Nominatim/OpenStreetMap queries from source fields, but it does not emit the current `compute_center_locations` row as a candidate. After a user accepts a candidate, the save endpoint upserts it into the dimension table; the next map refresh renders it through `StoredComputeCenterLocationResolver`.
|
||||
|
||||
`resolve_compute_center_location()`, `resolve_compute_center_location_full()`, and `collect_location_candidates()` remain the domain API. `visualization.py` consumes that API and no longer owns coordinate hints, country-centroid fallbacks, or Nominatim details.
|
||||
|
||||
GeoJSON output includes only `RENDERABLE_PRECISIONS`. Unresolved records are returned in `unresolved` with `failure_reason`, `attempted_queries`, `source_id`, `record_id`, and related diagnostics.
|
||||
|
||||
### BGP Collectors
|
||||
|
||||
Entry points:
|
||||
|
||||
- [bgp_collector_locations.py](/home/ray/dev/linkong/planet/backend/app/services/bgp_collector_locations.py)
|
||||
- [bgp_collector_location.py](/home/ray/dev/linkong/planet/backend/app/models/bgp_collector_location.py)
|
||||
|
||||
Resolver order:
|
||||
|
||||
```python
|
||||
SourceCoordinatesResolver()
|
||||
StoredCollectorLocationResolver()
|
||||
NominatimResolver(_bgp_collector_query_plan)
|
||||
```
|
||||
|
||||
The 23 RIPE RIS collector coordinates moved from the old table into the `bgp_collector_locations` dimension table with `source=legacy_seed` and `needs_confirmation=true`. The legacy dictionary is still maintained from the DB-backed cache for compatibility; manual candidate collection uses stored site/city/country as context but does not emit stored rows as candidates.
|
||||
|
||||
### BGP Events
|
||||
|
||||
Entry point:
|
||||
|
||||
- [bgp_event_locations.py](/home/ray/dev/linkong/planet/backend/app/services/bgp_event_locations.py)
|
||||
|
||||
Resolver order:
|
||||
|
||||
```python
|
||||
SourceCoordinatesResolver()
|
||||
InheritFromAnotherEntityResolver(_inherit_from_owning_collector)
|
||||
```
|
||||
|
||||
Event inheritance uses a strict owning-collector lookup and does not run the full fuzzy collector registry. Future ASN facility, PrefixGeo, or PeeringDB resolvers can be inserted after inheritance.
|
||||
|
||||
## API Envelope
|
||||
|
||||
```http
|
||||
POST /api/v1/visualization/compute-centers/{source_id}/collect-location
|
||||
POST /api/v1/visualization/compute-centers/{source_id}/location
|
||||
POST /api/v1/bgp/collectors/{collector_id}/collect-location
|
||||
```
|
||||
|
||||
Both `collect-location` endpoints return the same envelope:
|
||||
|
||||
```json
|
||||
{
|
||||
"success": true,
|
||||
"candidates": [],
|
||||
"best_candidate": {},
|
||||
"attempted_queries": [],
|
||||
"context": {}
|
||||
}
|
||||
```
|
||||
|
||||
`POST /api/v1/visualization/compute-centers/{source_id}/location` upserts the candidate selected by the frontend into `compute_center_locations`. Manual saves default to `needs_confirmation=false`, `verification_status="verified"`, and a `verified_at` timestamp. Future automated staging can pass `needs_confirmation=true` explicitly.
|
||||
|
||||
The frontend [info-card.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/info-card.js) renders the shared candidate list and preview events. The compute-center layer button shows an `unresolved` badge; clicking it opens the unresolved queue. Row-level `采集` only fetches candidates. Header-level `一键采用` walks the queue top-to-bottom, picks the highest-confidence candidate with valid coordinates, saves it, removes the row, renumbers the list, and dispatches `earth:compute-center-unresolved-count-change` so the badge updates immediately. When the batch finishes, `earth:compute-center-location-saved` refreshes the real layer.
|
||||
|
||||
If the remaining records have no city-level candidates, the batch must not invent coordinates. The UI keeps those rows and shows the backend `failure_reason` plus attempted queries.
|
||||
|
||||
## Adding A Resolver
|
||||
|
||||
A resolver only needs `name` and `resolve()`, returning `ResolverOutput`.
|
||||
|
||||
```python
|
||||
class PeeringDBFacilityResolver:
|
||||
name = "peeringdb_facility"
|
||||
|
||||
def __init__(self, client):
|
||||
self._client = client
|
||||
|
||||
def resolve(self, query):
|
||||
asn = query.extra.get("origin_asn")
|
||||
if not asn:
|
||||
return ResolverOutput()
|
||||
return ResolverOutput(candidates=tuple(
|
||||
LocationCandidate(
|
||||
latitude=f.latitude,
|
||||
longitude=f.longitude,
|
||||
display_name=f.name,
|
||||
precision="site",
|
||||
confidence=0.78,
|
||||
query=f"peeringdb::{asn}",
|
||||
source=self.name,
|
||||
source_note=f"PeeringDB facility for AS{asn}",
|
||||
matched_fields=("origin_asn",),
|
||||
needs_confirmation=False,
|
||||
city=f.city,
|
||||
country=f.country,
|
||||
)
|
||||
for f in self._client.facilities_for_asn(asn)
|
||||
))
|
||||
```
|
||||
|
||||
Wire it in:
|
||||
|
||||
```python
|
||||
BGP_EVENT_PIPELINE = LocationPipeline([
|
||||
SourceCoordinatesResolver(),
|
||||
InheritFromAnotherEntityResolver(source_lookup=...),
|
||||
PeeringDBFacilityResolver(client=peeringdb_client),
|
||||
])
|
||||
```
|
||||
|
||||
## Test Coverage
|
||||
|
||||
Relevant tests:
|
||||
|
||||
- [test_location_pipeline.py](/home/ray/dev/linkong/planet/backend/tests/test_location_pipeline.py)
|
||||
- [test_bgp_collector_locations.py](/home/ray/dev/linkong/planet/backend/tests/test_bgp_collector_locations.py)
|
||||
- [test_visualization_compute_centers.py](/home/ray/dev/linkong/planet/backend/tests/test_visualization_compute_centers.py)
|
||||
|
||||
Coverage focuses on resolver pluggability, registry alias guards, BGP collector legacy dictionary compatibility, compute-center public API compatibility, and non-renderable locations being returned as `unresolved`.
|
||||
Reference in New Issue
Block a user