planet/docs/technical/en/location-pipeline-development.md

Shared Location Resolution Pipeline Development Guide

backend/app/services/location/ is the shared abstraction for any "given a record, decide its lat/lon" workflow. Compute centers, BGP collectors, and BGP events now run on this pipeline. Future entities such as satellite ground stations, user-claimed points, and IXP facilities should plug in here instead of creating another geocoding path.

For the user workflow, see the Earth coordinate-candidate section in Intelligent Planet Manual.

Design Goals

Historically compute centers had their own four-tier chain, BGP collectors used a hard-coded dictionary, and BGP events inherited collector coordinates. These implementations did not share code, and new algorithms had no stable insertion point.

The refactored rules:

• Share the LocationResolver protocol and LocationPipeline orchestrator.
• Domain modules only build LocationQuery and choose resolver order.
• New algorithms join by adding resolver classes, without changing ingestion, API, or frontend envelopes.
• Earth renders only city-level or better locations.
• Local JSON registries are not runtime candidate sources for compute centers or BGP collectors; persisted location facts live in database dimension tables.

Core Interfaces

@dataclass(frozen=True)
class LocationQuery:
    name: str | None
    aliases: tuple[str, ...]
    city: str | None
    country: str | None
    region: str | None
    source_latitude: float | None
    source_longitude: float | None
    extra: Mapping[str, Any]

@dataclass(frozen=True)
class LocationCandidate:
    latitude: float
    longitude: float
    display_name: str
    precision: str
    confidence: float
    source: str
    needs_confirmation: bool
    matched_fields: tuple[str, ...]
    suggested_registry_entry: dict | None

class LocationResolver(Protocol):
    name: str
    def resolve(self, query: LocationQuery) -> ResolverOutput: ...

LocationPipeline.collect_candidates() returns sorted candidates plus attempted_queries; resolve_best() returns the best candidate with diagnostics. The default sort key ranks source, precision, and confidence, then deduplicates candidates with the same source and rounded coordinates.

Built-In Resolvers

Resolver	File	Responsibility
SourceCoordinatesResolver	resolvers/source_coordinates.py	Emits precision="precise" when the record already has lat/lon
RegistryResolver	resolvers/registry.py	Legacy generic resolver; current compute-center and BGP runtime paths do not use it to generate candidates
NominatimResolver	resolvers/nominatim.py	Runs a domain query plan against Nominatim with LRU cache and rate limiting
InheritFromAnotherEntityResolver	resolvers/inherit.py	Wraps an externally resolved entity location as a candidate
LocationLLMFallback	location/llm_fallback.py	Generates a confirmation-required candidate through the current default AI Provider when user-triggered collection has no regular candidates

Nominatim is the geocoding service in the OpenStreetMap ecosystem. Given a place name, city, country, organization, or facility query, it returns possible coordinates, a display name, and structured address fields. It is useful for turning city/facility text into candidate coordinates, but it is not an authoritative fact registry and can match same-name places or broad administrative areas. Planet therefore treats Nominatim output as confirmation-required candidates and uses it with caching and rate limiting.

RegistryResolver remains available for future controlled import scenarios, but it should not be reconnected as a hard-coded hint source for compute centers or BGP. Matching common fields such as operator or city was the main reason multiple entities could collapse onto the same point.

Current Domain Pipelines

Compute Centers

Entry points:

• compute_center_locations.py

Resolver order:

SourceCoordinatesResolver()
StoredComputeCenterLocationResolver()

The main map startup path is source coordinates first, then the database-backed current-location table. The table is compute_center_locations, keyed by (source, source_id), and stores manually accepted locations or true coordinates migrated from source records. init_db() only migrates source records that already contain real coordinates; it does not import old hard-coded hints and does not run ROR, Nominatim, or LLM geocoding during startup.

Candidate collection is intentionally separate from rendering. collect_location_candidates() builds ROR and Nominatim/OpenStreetMap queries from source fields, but it does not emit the current compute_center_locations row as a candidate. If those regular candidates are empty, the API layer calls LocationLLMFallback through the current default AI Provider and only returns source="llm_location_factcheck" candidates with needs_confirmation=true. LLM candidates use a combined threshold made from the model self-score plus backend evidence scoring; when the LLM provides a credible city/country but no coordinates, the backend may fill city-level coordinates through Nominatim without increasing the evidence score. After a user accepts a candidate, the save endpoint upserts it into the dimension table; the next map refresh renders it through StoredComputeCenterLocationResolver.

resolve_compute_center_location(), resolve_compute_center_location_full(), and collect_location_candidates() remain the domain API. visualization.py consumes that API and no longer owns coordinate hints, country-centroid fallbacks, or Nominatim details.

GeoJSON output includes only RENDERABLE_PRECISIONS. Unresolved records are returned in unresolved with failure_reason, attempted_queries, source_id, record_id, and related diagnostics.

BGP Collectors

Entry points:

• bgp_collector_locations.py
• bgp_collector_location.py

Resolver order:

SourceCoordinatesResolver()
StoredCollectorLocationResolver()
NominatimResolver(_bgp_collector_query_plan)

The 23 RIPE RIS collector coordinates moved from the old table into the bgp_collector_locations dimension table with source=legacy_seed and needs_confirmation=true. The legacy dictionary is still maintained from the DB-backed cache for compatibility; manual candidate collection uses stored site/city/country as context but does not emit stored rows as candidates. If Nominatim cannot produce a city-level candidate, the collection endpoint uses the current default AI Provider as an LLM factcheck fallback and returns a confirmation-required candidate instead of saving automatically.

BGP Events

Entry point:

• bgp_event_locations.py

Resolver order:

SourceCoordinatesResolver()
InheritFromAnotherEntityResolver(_inherit_from_owning_collector)

Event inheritance uses a strict owning-collector lookup and does not run the full fuzzy collector registry. Future ASN facility, PrefixGeo, or PeeringDB resolvers can be inserted after inheritance.

API Envelope

POST /api/v1/visualization/compute-centers/{source_id}/collect-location
POST /api/v1/visualization/compute-centers/{source_id}/location
POST /api/v1/bgp/collectors/{collector_id}/collect-location

Both collect-location endpoints return the same envelope:

{
  "success": true,
  "candidates": [],
  "best_candidate": {},
  "attempted_queries": [],
  "context": {}
}

The LLM fallback only runs inside user-triggered collect-location requests, and only after regular candidates are empty. It does not run during /geo/compute-centers startup rendering, scheduled collection, or batch persistence, and it never writes directly to compute_center_locations or bgp_collector_locations. Internally it is no longer a single "strict JSON or fail" step. It first asks the LLM to factcheck the location; if the answer is not JSON, it makes a second normalization request that may only extract facts from the original text; if that still fails, it conservatively extracts a city/country pair from the prose. The backend then performs coordinate filling, combined scoring, and candidate creation through one shared path.

This lets an answer such as "DeepL Mercury is in Falun, Sweden" become a city-level candidate after backend Nominatim coordinate filling, and lets a prose first answer be normalized into JSON on the second pass. Regardless of the path, only precise, site, or city precision with non-zero coordinates and a sufficient combined score is converted to a candidate. Failed, low-score, country-only, or cityless responses stay as diagnostics.

The LLM-provided confidence is only the model's self-score. The backend recomputes a combined score and uses that value as the candidate confidence:

combined =
  0.25 * model_confidence
  + source_quality
  + entity_match
  + geography_match
  + precision_quality
  + name_location_hint
  - conflict_penalty
  - weak_evidence_penalty

Current component caps: authoritative/government/academic evidence can add up to 0.35, reputable databases or news up to 0.25, generic web evidence up to 0.15; evidence that clearly names the queried entity can add 0.25; city+country geography match adds 0.20, country-only match adds 0.05; precision adds precise=0.15, site=0.12, or city=0.08; name_location_hint adds signal when the entity name and candidate city overlap, such as TAIPEI-1 and Taipei; explicit conflicts can subtract up to 0.45; weak-evidence wording can subtract up to 0.30, capped at 0.15 when entity and city/country match and no conflict is present. Candidates below 0.55 are rejected. This lets cases such as Alem.Cloud and TAIPEI-1 recover from a low model self-score when entity and city evidence align, while genuinely weak or conflicting evidence still fails.

POST /api/v1/visualization/compute-centers/{source_id}/location upserts the candidate selected by the frontend into compute_center_locations. Manual saves default to needs_confirmation=false, verification_status="verified", and a verified_at timestamp. Future automated staging can pass needs_confirmation=true explicitly.

The frontend info-card.js renders the shared candidate list and preview events. The compute-center layer button shows an unresolved badge; clicking it opens the unresolved queue. Row-level 采集 only fetches candidates. Header-level 一键采用 walks the queue top-to-bottom, picks the highest-confidence candidate with valid coordinates, saves it, removes the row, renumbers the list, and dispatches earth:compute-center-unresolved-count-change so the badge updates immediately. When the batch finishes, earth:compute-center-location-saved refreshes the real layer.

If the remaining records have no city-level candidates, the batch must not invent coordinates. The UI keeps those rows and shows the backend failure_reason plus attempted queries.

Adding A Resolver

A resolver only needs name and resolve(), returning ResolverOutput.

class PeeringDBFacilityResolver:
    name = "peeringdb_facility"

    def __init__(self, client):
        self._client = client

    def resolve(self, query):
        asn = query.extra.get("origin_asn")
        if not asn:
            return ResolverOutput()
        return ResolverOutput(candidates=tuple(
            LocationCandidate(
                latitude=f.latitude,
                longitude=f.longitude,
                display_name=f.name,
                precision="site",
                confidence=0.78,
                query=f"peeringdb::{asn}",
                source=self.name,
                source_note=f"PeeringDB facility for AS{asn}",
                matched_fields=("origin_asn",),
                needs_confirmation=False,
                city=f.city,
                country=f.country,
            )
            for f in self._client.facilities_for_asn(asn)
        ))

Wire it in:

BGP_EVENT_PIPELINE = LocationPipeline([
    SourceCoordinatesResolver(),
    InheritFromAnotherEntityResolver(source_lookup=...),
    PeeringDBFacilityResolver(client=peeringdb_client),
])

Test Coverage

Relevant tests:

• test_location_pipeline.py
• test_bgp_collector_locations.py
• test_visualization_compute_centers.py

Coverage focuses on resolver pluggability, registry alias guards, BGP collector legacy dictionary compatibility, compute-center public API compatibility, and non-renderable locations being returned as unresolved.