Files
planet/docs/technical/en/location-pipeline-development.md
T
2026-05-08 17:42:27 +08:00

9.2 KiB

Shared Location Resolution Pipeline Development Guide

backend/app/services/location/ is the shared abstraction for any "given a record, decide its lat/lon" workflow. Compute centers, BGP collectors, and BGP events now run on this pipeline. Future entities such as satellite ground stations, user-claimed points, and IXP facilities should plug in here instead of creating another geocoding path.

For the user workflow, see Earth Location Candidate Collection User Guide.

Design Goals

Historically compute centers had their own four-tier chain, BGP collectors used a hard-coded dictionary, and BGP events inherited collector coordinates. These implementations did not share code, and new algorithms had no stable insertion point.

The refactored rules:

  • Share the LocationResolver protocol and LocationPipeline orchestrator.
  • Domain modules only build LocationQuery and choose resolver order.
  • New algorithms join by adding resolver classes, without changing ingestion, API, or frontend envelopes.
  • Earth renders only city-level or better locations.
  • Local JSON registries are not runtime candidate sources for compute centers or BGP collectors; persisted location facts live in database dimension tables.

Core Interfaces

@dataclass(frozen=True)
class LocationQuery:
    name: str | None
    aliases: tuple[str, ...]
    city: str | None
    country: str | None
    region: str | None
    source_latitude: float | None
    source_longitude: float | None
    extra: Mapping[str, Any]
@dataclass(frozen=True)
class LocationCandidate:
    latitude: float
    longitude: float
    display_name: str
    precision: str
    confidence: float
    source: str
    needs_confirmation: bool
    matched_fields: tuple[str, ...]
    suggested_registry_entry: dict | None
class LocationResolver(Protocol):
    name: str
    def resolve(self, query: LocationQuery) -> ResolverOutput: ...

LocationPipeline.collect_candidates() returns sorted candidates plus attempted_queries; resolve_best() returns the best candidate with diagnostics. The default sort key ranks source, precision, and confidence, then deduplicates candidates with the same source and rounded coordinates.

Built-In Resolvers

Resolver File Responsibility
SourceCoordinatesResolver resolvers/source_coordinates.py Emits precision="precise" when the record already has lat/lon
RegistryResolver resolvers/registry.py Legacy generic resolver; current compute-center and BGP runtime paths do not use it to generate candidates
NominatimResolver resolvers/nominatim.py Runs a domain query plan against Nominatim with LRU cache and rate limiting
InheritFromAnotherEntityResolver resolvers/inherit.py Wraps an externally resolved entity location as a candidate

RegistryResolver remains available for future controlled import scenarios, but it should not be reconnected as a hard-coded hint source for compute centers or BGP. Matching common fields such as operator or city was the main reason multiple entities could collapse onto the same point.

Current Domain Pipelines

Compute Centers

Entry points:

Resolver order:

SourceCoordinatesResolver()
StoredComputeCenterLocationResolver()

The main map startup path is source coordinates first, then the database-backed current-location table. The table is compute_center_locations, keyed by (source, source_id), and stores manually accepted locations or true coordinates migrated from source records. init_db() only migrates source records that already contain real coordinates; it does not import old hard-coded hints and does not run ROR, Nominatim, or LLM geocoding during startup.

Candidate collection is intentionally separate from rendering. collect_location_candidates() builds ROR and Nominatim/OpenStreetMap queries from source fields, but it does not emit the current compute_center_locations row as a candidate. After a user accepts a candidate, the save endpoint upserts it into the dimension table; the next map refresh renders it through StoredComputeCenterLocationResolver.

resolve_compute_center_location(), resolve_compute_center_location_full(), and collect_location_candidates() remain the domain API. visualization.py consumes that API and no longer owns coordinate hints, country-centroid fallbacks, or Nominatim details.

GeoJSON output includes only RENDERABLE_PRECISIONS. Unresolved records are returned in unresolved with failure_reason, attempted_queries, source_id, record_id, and related diagnostics.

BGP Collectors

Entry points:

Resolver order:

SourceCoordinatesResolver()
StoredCollectorLocationResolver()
NominatimResolver(_bgp_collector_query_plan)

The 23 RIPE RIS collector coordinates moved from the old table into the bgp_collector_locations dimension table with source=legacy_seed and needs_confirmation=true. The legacy dictionary is still maintained from the DB-backed cache for compatibility; manual candidate collection uses stored site/city/country as context but does not emit stored rows as candidates.

BGP Events

Entry point:

Resolver order:

SourceCoordinatesResolver()
InheritFromAnotherEntityResolver(_inherit_from_owning_collector)

Event inheritance uses a strict owning-collector lookup and does not run the full fuzzy collector registry. Future ASN facility, PrefixGeo, or PeeringDB resolvers can be inserted after inheritance.

API Envelope

POST /api/v1/visualization/compute-centers/{source_id}/collect-location
POST /api/v1/visualization/compute-centers/{source_id}/location
POST /api/v1/bgp/collectors/{collector_id}/collect-location

Both collect-location endpoints return the same envelope:

{
  "success": true,
  "candidates": [],
  "best_candidate": {},
  "attempted_queries": [],
  "context": {}
}

POST /api/v1/visualization/compute-centers/{source_id}/location upserts the candidate selected by the frontend into compute_center_locations. Manual saves default to needs_confirmation=false, verification_status="verified", and a verified_at timestamp. Future automated staging can pass needs_confirmation=true explicitly.

The frontend info-card.js renders the shared candidate list and preview events. The compute-center layer button shows an unresolved badge; clicking it opens the unresolved queue. Row-level 采集 only fetches candidates. Header-level 一键采用 walks the queue top-to-bottom, picks the highest-confidence candidate with valid coordinates, saves it, removes the row, renumbers the list, and dispatches earth:compute-center-unresolved-count-change so the badge updates immediately. When the batch finishes, earth:compute-center-location-saved refreshes the real layer.

If the remaining records have no city-level candidates, the batch must not invent coordinates. The UI keeps those rows and shows the backend failure_reason plus attempted queries.

Adding A Resolver

A resolver only needs name and resolve(), returning ResolverOutput.

class PeeringDBFacilityResolver:
    name = "peeringdb_facility"

    def __init__(self, client):
        self._client = client

    def resolve(self, query):
        asn = query.extra.get("origin_asn")
        if not asn:
            return ResolverOutput()
        return ResolverOutput(candidates=tuple(
            LocationCandidate(
                latitude=f.latitude,
                longitude=f.longitude,
                display_name=f.name,
                precision="site",
                confidence=0.78,
                query=f"peeringdb::{asn}",
                source=self.name,
                source_note=f"PeeringDB facility for AS{asn}",
                matched_fields=("origin_asn",),
                needs_confirmation=False,
                city=f.city,
                country=f.country,
            )
            for f in self._client.facilities_for_asn(asn)
        ))

Wire it in:

BGP_EVENT_PIPELINE = LocationPipeline([
    SourceCoordinatesResolver(),
    InheritFromAnotherEntityResolver(source_lookup=...),
    PeeringDBFacilityResolver(client=peeringdb_client),
])

Test Coverage

Relevant tests:

Coverage focuses on resolver pluggability, registry alias guards, BGP collector legacy dictionary compatibility, compute-center public API compatibility, and non-renderable locations being returned as unresolved.