release: bump version to 0.68.0

.gitignore

@@ -28,8 +28,8 @@ dist/
 downloads/
 eggs/
 .eggs/
-lib/
-lib64/
+/lib/
+/lib64/
 parts/
 sdist/
 var/

TODO.md

@@ -7,17 +7,12 @@ This file is the active backlog only. Completed history belongs in `docs/CHANGEL
 - [ ] Earth AI command entry: merge natural-language and speech-triggered LLM commands into the existing Earth search panel as described in [Agent Runtime, Earth LLM Command, And Speech Entry Plan](/home/ray/dev/linkong/planet/docs/plans/agents-earth-command-runtime-plan.md).
 - [ ] Earth action executor: implement safe visualization actions for layer toggles, batch highlights, filters, focus, result panels, and clear-highlight behavior.
 - [ ] Earth entity matching: support stable entity ids and batch matching for Beidou satellites, mainland China compute centers, BGP, news, vessels, and cables.
-- [x] High-precision country boundary tile framework: implement the static vector tile builder, versioned seed output, frontend bbox tile loader, debounce, in-flight dedupe, and LRU cache described in [Earth High Precision Boundary Tiles Plan](/home/ray/dev/linkong/planet/docs/plans/earth-high-precision-boundary-tiles-plan.md).
-- [x] Add the `pmtiles-mvt` frontend tile provider contract, MVT decoder dependencies, static PMTiles Nginx handling, collector artifact registration, production readiness check, and user operation docs for Earth boundaries.
-- [x] Split Earth boundary ingestion into standard source collectors (`earth_admin0_boundaries`, `earth_coastline`, `earth_claim_lines`) plus the downstream `earth_boundary_tiles` PMTiles builder.
 - [ ] Replace debug GeoJSON boundary tiles with the real `earth-boundaries-china-pov-v1.pmtiles` production artifact after audited admin-0 / coastline / claim-line sources and the PMTiles toolchain are available.
 - [ ] Import authoritative China POV / coastline / claim-line source packages through the three standard Earth boundary source collectors, then rebuild a versioned PMTiles artifact so highest zoom `8-10` preserves trusted source geometry instead of seed data.
 - [ ] Earth boundary data: acquire or generate auditable China POV geometry for Zangnan, Aksai Chin, Taiwan/Penghu, Diaoyu Dao and affiliated islands, Chiwei Yu, South China Sea islands, Kosovo, Gaza, and the official dashed maritime claim line before implementing final visual changes.
 - [ ] Earth high-resolution basemap tiles: implement the viewport-loaded imagery layer described in [Earth High Resolution Basemap Tiles Plan](/home/ray/dev/linkong/planet/docs/plans/earth-high-resolution-basemap-tiles-plan.md), using high-precision coastline as the alignment reference instead of replacing the globe with one huge texture.
-- [ ] Presentation controller ownership: replace the singleton card fallback in [presentation-controller.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/presentation-controller.js) with a presentation/card token check before BGP/News migrate onto the shared controller, so connectors only attach to their owning card.
 - [ ] BGP frontend maintainability: split [bgp.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/bgp.js) by responsibility into data loading, marker rendering, overlays, and animation once the current interaction behavior is stable.
 - [ ] Optional BGP marker experiment: evaluate HTML markers for BGP incident/collector points if WebGL marker density or fixed screen-size clickability becomes a real blocker.
-- [ ] Earth news cruise: connect Earth news to the generic cruise queue via a news adapter rather than coupling news-specific sequencing into `main.js`.
 
 ## Compute Centers And Location
 
@@ -48,19 +43,15 @@ This file is the active backlog only. Completed history belongs in `docs/CHANGEL
 - [ ] Compatibility schema: cover adapter type, base URL pattern, auth header, thinking/reasoning defaults, stream path, tool-call capability, multimodal capability, and provider-specific request patches.
 - [ ] BGP geography fallback: evaluate `inetnum` / `inet6num` whois as a finer fallback layer after `prefix_geography`, `OpenGeoFeed`, and RIR delegated data.
 
-## Platform
-
-- [ ] Earth preferences scope: keep current device-local Earth preferences in `localStorage`; only design backend user preferences if account-level synchronization becomes a real product requirement.
-- [ ] System logs: finish a usable Planet log viewing flow that covers backend, frontend, AI Provider, and collector/task logs, with filtering and tailing.
-- [ ] Console UI modernization: gradually replace Ant Design with Planet-owned components and a consistent Tabler Icons based icon system.
-- [ ] Earth live sync: design a unified realtime invalidation path for summary/BGP/satellite updates if polling and current WebSocket channels become insufficient.
-
 ## Archive
 
 Archived items stay here so old context is not lost. Completed items remain checked; obsolete, invalid, or superseded items stay unchecked and include the reason.
 
 ### Completed
 
+- [x] Implemented the high-precision country boundary tile framework from [Earth High Precision Boundary Tiles Plan](/home/ray/dev/linkong/planet/docs/plans/earth-high-precision-boundary-tiles-plan.md): static vector tile builder, versioned seed output, frontend bbox tile loader, debounce, in-flight dedupe, and LRU cache.
+- [x] Added the `pmtiles-mvt` frontend tile provider contract, MVT decoder dependencies, static PMTiles Nginx handling, collector artifact registration, production readiness check, and user operation docs for Earth boundaries.
+- [x] Split Earth boundary ingestion into standard source collectors (`earth_admin0_boundaries`, `earth_coastline`, `earth_claim_lines`) plus the downstream `earth_boundary_tiles` PMTiles builder.
 - [x] Refined BGP observer and anomaly `hover/click` feel.
 - [x] Added BGP anomaly relationship display with cables / regions.
 - [x] Added the Earth BGP activity layer so the map still feels alive when incident density is low.
@@ -75,6 +66,11 @@ Archived items stay here so old context is not lost. Completed items remain chec
 - [x] Added OpenGeoFeed as a high-quality prefix geography override source.
 - [x] Made RIR delegated data a prefix geography fallback rather than the primary source.
 - [x] Added route leak and path instability / flap detectors after the activity layer work.
+- [x] Console UI modernization. Admin is now the only console, legacy Ant Design / Admin Next code paths and dependencies have been removed, and current console UI uses Planet-owned components.
+- [x] Earth news cruise adapter. News cruise now uses `news-cruise-adapter.js` and is wired from `main.js` instead of keeping news-specific sequencing directly in the main Earth loop.
+- [x] Presentation controller ownership. `PresentationController` now guards async ownership through active request identity checks, and current callers pass per-request card targets so stale connector/card work cannot overwrite the active presentation.
+- [x] Earth live sync. Database writes now flow through `earth_data_change_events`, `earth_db_change_listener`, layer adapters, cache invalidation, and the `earth_updates` WebSocket channel; the Earth frontend debounces updates and refreshes BGP, cables, compute centers, satellites, vessels, news, and interactables by layer.
+- [x] System logs. Log sources now normalize into `LogEvent`, Admin supports snapshot filtering plus WebSocket tail/follow, task/detail views deep-link into prefiltered logs, and Admin runtime errors report through the `admin-client` log source.
 
 ### Obsolete Or Superseded
 
@@ -85,3 +81,4 @@ Archived items stay here so old context is not lost. Completed items remain chec
 - [ ] Earth surface material overlay for boundary calibration. Superseded by the high-precision boundary tile plan; future work must use source-faithful boundary/coastline data rather than overlay calibration against the coarse base map.
 - [ ] Hardcoded Earth news source extraction as a standalone task. Superseded by the broader Earth news source configuration and collector plans.
 - [ ] Country-level compute-center fallback placement as a standalone task. Superseded by the shared location pipeline and registry/manual-review backlog.
+- [ ] Earth preferences backend sync scope. Superseded by the current product decision to keep Earth preferences device-local in `localStorage` until account-level synchronization becomes a real requirement.

VERSION

@@ -1 +1 @@
-0.64.0
+0.68.0

aiprovider/Dockerfile

@@ -2,10 +2,14 @@
 
 ARG PYTHON_IMAGE=python:3.14-slim
 ARG UV_IMAGE=ghcr.io/astral-sh/uv:latest
+ARG AI_PROVIDER_BUILD_FINGERPRINT=unknown
 
 FROM ${UV_IMAGE} AS uv
 FROM ${PYTHON_IMAGE}
 
+ARG AI_PROVIDER_BUILD_FINGERPRINT
+LABEL planet.aiprovider.build-fingerprint="${AI_PROVIDER_BUILD_FINGERPRINT}"
+
 COPY --from=uv /uv /uvx /bin/
 
 WORKDIR /app
@@ -15,12 +19,15 @@ ENV PYTHONUNBUFFERED=1
 ENV UV_COMPILE_BYTECODE=1
 ENV UV_LINK_MODE=copy
 
+RUN mkdir -p /root/.config/uv
+
 RUN apt-get update && apt-get install -y --no-install-recommends \
     curl \
     && rm -rf /var/lib/apt/lists/*
 
 COPY pyproject.toml uv.lock /app/
 RUN --mount=type=cache,target=/root/.cache/uv \
+    --mount=type=secret,id=planet_uv_config,target=/root/.config/uv/uv.toml,required=false \
     uv sync --frozen --no-dev
 
 COPY aiprovider /app/aiprovider

backend/Dockerfile

@@ -1,3 +1,5 @@
+# syntax=docker/dockerfile:1.7
+
 ARG PYTHON_IMAGE=python:3.14-slim
 ARG UV_IMAGE=ghcr.io/astral-sh/uv:latest
 
@@ -14,12 +16,16 @@ ENV UV_COMPILE_BYTECODE=1
 ENV UV_LINK_MODE=copy
 ENV PYTHONPATH=/app/backend
 
+RUN mkdir -p /root/.config/uv
+
 RUN apt-get update && apt-get install -y --no-install-recommends \
     curl \
     && rm -rf /var/lib/apt/lists/*
 
 COPY pyproject.toml uv.lock /app/
-RUN uv sync --frozen --no-dev
+RUN --mount=type=cache,target=/root/.cache/uv \
+    --mount=type=secret,id=planet_uv_config,target=/root/.config/uv/uv.toml,required=false \
+    uv sync --frozen --no-dev
 
 COPY backend /app/backend
 COPY VERSION /app/VERSION

backend/app/api/main.py

@@ -19,6 +19,7 @@ from app.api.v1 import (
     vessels,
     bgp,
     news,
+    interactables,
     realtime_sources,
     system_control,
     tv,
@@ -53,4 +54,5 @@ api_router.include_router(vessels.router, prefix="/vessels", tags=["vessels"])
 api_router.include_router(bgp.router, prefix="/bgp", tags=["bgp"])
 api_router.include_router(tv.router, prefix="/tv", tags=["tv"])
 api_router.include_router(news.router, prefix="/news", tags=["news"])
+api_router.include_router(interactables.router, prefix="/interactables", tags=["interactables"])
 api_router.include_router(realtime_sources.router, prefix="/realtime-sources", tags=["realtime-sources"])

backend/app/api/v1/ai.py

@@ -3,6 +3,7 @@ from uuid import uuid4
 from fastapi import APIRouter, Depends, HTTPException, Request, Response
 from sqlalchemy.ext.asyncio import AsyncSession
 
+from app.core.logging import get_logger
 from app.core.security import get_current_user
 from app.db.session import get_db
 from app.models.user import User
@@ -47,8 +48,10 @@ from app.services.playground_chat_service import (
     stop_message,
 )
 from app.services.situational_alert_ai_brief import build_situational_alert_brief_request
+from app.services.business_logs import emit_business_log, exception_context
 
 router = APIRouter()
+logger = get_logger(__name__, service="api")
 
 
 @router.get("/provider/status", response_model=AIProviderStatusResponse)
@@ -122,6 +125,16 @@ async def create_playground_message(
     provider_client: AIProviderClient = Depends(get_ai_provider_client),
     db: AsyncSession = Depends(get_db),
 ):
+    await emit_business_log(
+        logger,
+        event="ai.playground.message.create",
+        message="Playground message creation requested",
+        category="ai",
+        service="api",
+        module=__name__,
+        user_id=current_user.id,
+        context={"session_key": payload.session_key, "preset": payload.selected_preset_key},
+    )
     return await create_turn(
         db,
         user_id=current_user.id,
@@ -136,6 +149,16 @@ async def stop_playground_message(
     current_user: User = Depends(get_current_user),
     db: AsyncSession = Depends(get_db),
 ):
+    await emit_business_log(
+        logger,
+        event="ai.playground.message.stop",
+        message="Playground message stop requested",
+        category="ai",
+        service="api",
+        module=__name__,
+        user_id=current_user.id,
+        context={"session_key": payload.session_key, "message_id": payload.message_id},
+    )
     return await stop_message(
         db,
         user_id=current_user.id,
@@ -150,6 +173,16 @@ async def resend_playground_message(
     provider_client: AIProviderClient = Depends(get_ai_provider_client),
     db: AsyncSession = Depends(get_db),
 ):
+    await emit_business_log(
+        logger,
+        event="ai.playground.message.resend",
+        message="Playground message resend requested",
+        category="ai",
+        service="api",
+        module=__name__,
+        user_id=current_user.id,
+        context={"session_key": payload.session_key, "user_message_id": payload.user_message_id},
+    )
     return await resend_turn(
         db,
         user_id=current_user.id,
@@ -214,16 +247,70 @@ async def analyze_bgp_brief(
         anomaly_limit=payload.anomaly_limit,
         collector_limit=payload.collector_limit,
     )
+    await emit_business_log(
+        logger,
+        event="ai.brief.bgp.facts_collected",
+        message="BGP brief facts collected",
+        category="ai",
+        service="api",
+        module=__name__,
+        request_id=request_id,
+        user_id=current_user.id,
+        context={
+            "incident_limit": payload.incident_limit,
+            "anomaly_limit": payload.anomaly_limit,
+            "collector_limit": payload.collector_limit,
+            "fact_count": len(facts or []),
+        },
+    )
     brief_request.preferred_model = payload.preferred_model
     brief_request.thinking = payload.thinking
 
-    analysis = await provider_client.analyze(brief_request, request_id=request_id)
-    return save_bgp_brief_record(
-        analysis,
+    await emit_business_log(
+        logger,
+        event="ai.brief.bgp.start",
+        message="BGP brief AI analysis started",
+        category="ai",
+        service="api",
+        module=__name__,
         request_id=request_id,
-        facts=facts,
-        context=context,
+        user_id=current_user.id,
+        context={"preferred_model": payload.preferred_model},
     )
+    try:
+        analysis = await provider_client.analyze(brief_request, request_id=request_id)
+        record = save_bgp_brief_record(
+            analysis,
+            request_id=request_id,
+            facts=facts,
+            context=context,
+        )
+        await emit_business_log(
+            logger,
+            event="ai.brief.bgp.completed",
+            message="BGP brief AI analysis saved",
+            category="ai",
+            service="api",
+            module=__name__,
+            request_id=request_id,
+            user_id=current_user.id,
+            context={"provider": analysis.provider, "model": analysis.model, "brief_id": record.id},
+        )
+        return record
+    except Exception as exc:
+        await emit_business_log(
+            logger,
+            event="ai.brief.bgp.failed",
+            message="BGP brief AI analysis failed",
+            category="ai",
+            level="error",
+            service="api",
+            module=__name__,
+            request_id=request_id,
+            user_id=current_user.id,
+            context=exception_context(exc, {"preferred_model": payload.preferred_model}),
+        )
+        raise
 
 
 @router.post("/alerts/brief", response_model=AlertBriefResponse)
@@ -242,17 +329,65 @@ async def analyze_alert_brief(
         db,
         alert_limit=payload.alert_limit,
     )
+    await emit_business_log(
+        logger,
+        event="ai.brief.alerts.facts_collected",
+        message="Alert brief facts collected",
+        category="ai",
+        service="api",
+        module=__name__,
+        request_id=request_id,
+        user_id=current_user.id,
+        context={"alert_limit": payload.alert_limit, "fact_count": len(facts or [])},
+    )
     brief_request.preferred_model = payload.preferred_model
     brief_request.thinking = payload.thinking
 
-    analysis = await provider_client.analyze(brief_request, request_id=request_id)
-    return AlertBriefResponse(
-        **analysis.model_dump(),
-        title=brief_request.title,
-        objective=brief_request.objective,
-        facts=facts,
-        context=context,
+    await emit_business_log(
+        logger,
+        event="ai.brief.alerts.start",
+        message="Alert brief AI analysis started",
+        category="ai",
+        service="api",
+        module=__name__,
+        request_id=request_id,
+        user_id=current_user.id,
+        context={"preferred_model": payload.preferred_model},
     )
+    try:
+        analysis = await provider_client.analyze(brief_request, request_id=request_id)
+        await emit_business_log(
+            logger,
+            event="ai.brief.alerts.completed",
+            message="Alert brief AI analysis completed",
+            category="ai",
+            service="api",
+            module=__name__,
+            request_id=request_id,
+            user_id=current_user.id,
+            context={"provider": analysis.provider, "model": analysis.model},
+        )
+        return AlertBriefResponse(
+            **analysis.model_dump(),
+            title=brief_request.title,
+            objective=brief_request.objective,
+            facts=facts,
+            context=context,
+        )
+    except Exception as exc:
+        await emit_business_log(
+            logger,
+            event="ai.brief.alerts.failed",
+            message="Alert brief AI analysis failed",
+            category="ai",
+            level="error",
+            service="api",
+            module=__name__,
+            request_id=request_id,
+            user_id=current_user.id,
+            context=exception_context(exc, {"preferred_model": payload.preferred_model}),
+        )
+        raise
 
 
 @router.post("/situational-alerts/brief", response_model=SituationalAlertBriefResponse)
@@ -268,14 +403,62 @@ async def analyze_situational_alert_brief(
     response.headers["X-Request-ID"] = request_id
 
     brief_request, facts, context = await build_situational_alert_brief_request(db)
+    await emit_business_log(
+        logger,
+        event="ai.brief.situational_alerts.facts_collected",
+        message="Situational alert brief facts collected",
+        category="ai",
+        service="api",
+        module=__name__,
+        request_id=request_id,
+        user_id=current_user.id,
+        context={"fact_count": len(facts or [])},
+    )
     brief_request.preferred_model = payload.preferred_model
     brief_request.thinking = payload.thinking
 
-    analysis = await provider_client.analyze(brief_request, request_id=request_id)
-    return SituationalAlertBriefResponse(
-        **analysis.model_dump(),
-        title=brief_request.title,
-        objective=brief_request.objective,
-        facts=facts,
-        context=context,
+    await emit_business_log(
+        logger,
+        event="ai.brief.situational_alerts.start",
+        message="Situational alert brief AI analysis started",
+        category="ai",
+        service="api",
+        module=__name__,
+        request_id=request_id,
+        user_id=current_user.id,
+        context={"preferred_model": payload.preferred_model},
     )
+    try:
+        analysis = await provider_client.analyze(brief_request, request_id=request_id)
+        await emit_business_log(
+            logger,
+            event="ai.brief.situational_alerts.completed",
+            message="Situational alert brief AI analysis completed",
+            category="ai",
+            service="api",
+            module=__name__,
+            request_id=request_id,
+            user_id=current_user.id,
+            context={"provider": analysis.provider, "model": analysis.model},
+        )
+        return SituationalAlertBriefResponse(
+            **analysis.model_dump(),
+            title=brief_request.title,
+            objective=brief_request.objective,
+            facts=facts,
+            context=context,
+        )
+    except Exception as exc:
+        await emit_business_log(
+            logger,
+            event="ai.brief.situational_alerts.failed",
+            message="Situational alert brief AI analysis failed",
+            category="ai",
+            level="error",
+            service="api",
+            module=__name__,
+            request_id=request_id,
+            user_id=current_user.id,
+            context=exception_context(exc, {"preferred_model": payload.preferred_model}),
+        )
+        raise

backend/app/api/v1/datasources.py

@@ -1,4 +1,3 @@
-import asyncio
 from datetime import datetime, timedelta, timezone
 from typing import Optional
 
@@ -7,7 +6,7 @@ from pydantic import BaseModel, Field
 from sqlalchemy import func, or_, select, text
 from sqlalchemy.ext.asyncio import AsyncSession
 
-from app.core.cache import cache
+from app.core.logging import get_logger
 from app.core.time import to_iso8601_utc
 from app.core.security import get_current_user
 from app.core.data_sources import get_data_sources_config
@@ -20,18 +19,27 @@ from app.models.datasource_config import DataSourceConfig
 from app.models.task import CollectionTask
 from app.models.user import User
 from app.models.vessel import AISRawObservation
-from app.services.vessel_ais_aggregation import VESSEL_AIS_SCHEMA
 from app.services.scheduler import (
-    cancel_running_collector_now,
-    get_latest_task_id_for_datasource,
-    run_collector_now,
     sync_datasource_job,
 )
-from app.services.earth_layer_cache import invalidate_earth_layer_cache_for_source
+from app.services.data_jobs import (
+    JOB_STATUS_CANCELLING,
+    JOB_STATUS_QUEUED,
+    JOB_STATUS_RUNNING,
+    JOB_TYPE_CLEAR_CACHE,
+    JOB_TYPE_CLEAR_DATA,
+    JOB_TYPE_COLLECT,
+    enqueue_datasource_job,
+    get_active_datasource_job,
+    request_cancel_datasource_task,
+)
+from app.services.business_logs import emit_business_log
 
 router = APIRouter()
+logger = get_logger(__name__, service="api")
 STALE_RUNNING_TASK_TIMEOUT_MINUTES = 90
 
+
 PRODUCT_SOURCE_KEYWORDS: tuple[tuple[str, tuple[str, ...]], ...] = (
     ("vessels", ("vessel", "ais")),
     ("cables", ("cable", "landing", "telegeography", "arcgis", "fao")),
@@ -115,7 +123,7 @@ async def _load_latest_running_tasks(
             _task_rank_column(CollectionTask.started_at),
         )
         .where(CollectionTask.datasource_id.in_(datasource_ids))
-        .where(CollectionTask.status == "running")
+        .where(CollectionTask.status.in_((JOB_STATUS_QUEUED, JOB_STATUS_RUNNING, JOB_STATUS_CANCELLING)))
         .subquery()
     )
     result = await db.execute(
@@ -126,32 +134,6 @@ async def _load_latest_running_tasks(
     return {task.datasource_id: task for task in result.scalars().all()}
 
 
-async def _load_latest_task_ids(
-    db: AsyncSession,
-    datasource_ids: list[int],
-) -> dict[int, int]:
-    if not datasource_ids:
-        return {}
-
-    ranked_tasks = (
-        select(
-            CollectionTask.id.label("task_id"),
-            CollectionTask.datasource_id.label("datasource_id"),
-            func.row_number().over(
-                partition_by=CollectionTask.datasource_id,
-                order_by=CollectionTask.id.desc(),
-            ).label("row_num"),
-        )
-        .where(CollectionTask.datasource_id.in_(datasource_ids))
-        .subquery()
-    )
-    result = await db.execute(
-        select(ranked_tasks.c.datasource_id, ranked_tasks.c.task_id)
-        .where(ranked_tasks.c.row_num == 1)
-    )
-    return {datasource_id: task_id for datasource_id, task_id in result.all()}
-
-
 async def _load_latest_tasks(
     db: AsyncSession,
     datasource_ids: list[int],
@@ -182,6 +164,8 @@ async def _load_latest_tasks(
 async def _load_collected_record_counts(
     db: AsyncSession,
     sources: list[str],
+    *,
+    exact_vessel_counts: bool = False,
 ) -> dict[str, int]:
     if not sources:
         return {}
@@ -202,14 +186,46 @@ async def _load_collected_record_counts(
         or "ais" in source
     ]
     if vessel_sources:
-        raw_result = await db.execute(
-            select(AISRawObservation.source, func.count(AISRawObservation.id))
-            .where(AISRawObservation.target_schema == VESSEL_AIS_SCHEMA)
-            .where(AISRawObservation.source.in_(vessel_sources))
-            .group_by(AISRawObservation.source)
+        if exact_vessel_counts:
+            exact_result = await db.execute(
+                select(AISRawObservation.source, func.count(AISRawObservation.id))
+                .where(AISRawObservation.source.in_(vessel_sources))
+                .group_by(AISRawObservation.source)
+            )
+            for source, count in exact_result.all():
+                counts[source] = max(counts.get(source, 0), int(count or 0))
+            return counts
+
+        # AIS raw observations can be tens of millions of rows. Use planner
+        # statistics for the datasource list instead of blocking page load on
+        # source-level count(*) scans.
+        stats_result = await db.execute(
+            text(
+                """
+                SELECT
+                    COALESCE(pg_class.reltuples, 0)::bigint AS total_rows,
+                    pg_stats.most_common_vals::text AS source_values,
+                    pg_stats.most_common_freqs::text AS source_freqs
+                FROM pg_class
+                LEFT JOIN pg_stats
+                  ON pg_stats.schemaname = 'public'
+                 AND pg_stats.tablename = 'ais_raw_observations'
+                 AND pg_stats.attname = 'source'
+                WHERE pg_class.relname = 'ais_raw_observations'
+                LIMIT 1
+                """
+            )
         )
-        for source, count in raw_result.all():
-            counts[source] = max(counts.get(source, 0), int(count or 0))
+        stats = stats_result.mappings().first()
+        if stats:
+            total_rows = int(stats["total_rows"] or 0)
+            values = str(stats["source_values"] or "").strip("{}")
+            freqs = str(stats["source_freqs"] or "").strip("{}")
+            source_values = [value.strip('"') for value in values.split(",") if value]
+            source_freqs = [float(value) for value in freqs.split(",") if value]
+            for source, freq in zip(source_values, source_freqs):
+                if source in vessel_sources:
+                    counts[source] = max(counts.get(source, 0), int(round(total_rows * freq)))
 
     return counts
 
@@ -303,8 +319,11 @@ def serialize_datasource_row(
         "last_run": to_iso8601_utc(last_run_at),
         "last_run_at": to_iso8601_utc(last_run_at),
         "last_status": last_status,
-        "is_running": running_task is not None,
+        "is_running": running_task is not None and running_task.task_type == JOB_TYPE_COLLECT,
+        "is_task_active": running_task is not None,
+        "task_status": running_task.status if running_task else None,
         "task_id": display_task.id if display_task else None,
+        "task_type": display_task.task_type if display_task else None,
         "progress": display_task.progress if display_task else None,
         "phase": display_task.phase if display_task else None,
         "phase_progress": display_task.phase_progress if display_task else None,
@@ -400,9 +419,25 @@ async def _trigger_datasource_batch(
     datasources: list[DataSource],
     *,
     force: bool,
+    actor_id: int | None = None,
+    trigger_kind: str = "batch",
 ) -> dict:
+    await emit_business_log(
+        logger,
+        event=f"collector.trigger.{trigger_kind}.start",
+        message="Datasource batch trigger started",
+        category="collector",
+        service="api",
+        module=__name__,
+        user_id=actor_id,
+        context={
+            "trigger_kind": trigger_kind,
+            "force": force,
+            "requested_count": len(datasources),
+        },
+    )
     if not datasources:
-        return {
+        result = {
             "status": "noop",
             "message": "No matching data sources to trigger",
             "force": force,
@@ -410,8 +445,18 @@ async def _trigger_datasource_batch(
             "skipped": [],
             "failed": [],
         }
+        await emit_business_log(
+            logger,
+            event=f"collector.trigger.{trigger_kind}.completed",
+            message="Datasource batch trigger completed with no matching sources",
+            category="collector",
+            service="api",
+            module=__name__,
+            user_id=actor_id,
+            context={"trigger_kind": trigger_kind, "force": force, "status": "noop", "triggered_count": 0},
+        )
+        return result
 
-    previous_task_ids: dict[int, Optional[int]] = {}
     triggered_sources: list[dict] = []
     skipped_sources: list[dict] = []
     failed_sources: list[dict] = []
@@ -446,9 +491,11 @@ async def _trigger_datasource_batch(
                     }
                 )
                 continue
-            cancelled = await cancel_running_collector_now(datasource.source)
-            if not cancelled:
-                await rollback_orphaned_running_task(db, datasource, running_task)
+            await request_cancel_datasource_task(
+                db,
+                running_task,
+                reason="superseded_by_forced_collection",
+            )
 
         if not force and not is_due_for_collection(datasource, now):
             skipped_sources.append(
@@ -465,57 +512,51 @@ async def _trigger_datasource_batch(
             )
             continue
 
-        previous_task_ids[datasource.id] = None
-        success = run_collector_now(datasource.source)
-        if not success:
-            failed_sources.append(
-                {
-                    "id": datasource.id,
-                    "source": datasource.source,
-                    "name": datasource.name,
-                    "reason": "trigger_failed",
-                }
-            )
-            continue
+        task = await enqueue_datasource_job(
+            db,
+            datasource,
+            JOB_TYPE_COLLECT,
+            payload={"force": force, "trigger": "batch"},
+        )
 
         triggered_sources.append(
             {
                 "id": datasource.id,
                 "source": datasource.source,
                 "name": datasource.name,
-                "task_id": None,
+                "task_id": task.id,
             }
         )
 
-    latest_task_ids = await _load_latest_task_ids(
-        db,
-        [datasource.id for datasource in datasources],
-    )
-    for datasource_id in previous_task_ids:
-        previous_task_ids[datasource_id] = latest_task_ids.get(datasource_id)
-
-    for _ in range(20):
-        await asyncio.sleep(0.1)
-        pending = [item for item in triggered_sources if item["task_id"] is None]
-        if not pending:
-            break
-        latest_task_ids = await _load_latest_task_ids(
-            db,
-            [item["id"] for item in pending],
-        )
-        for item in pending:
-            task_id = latest_task_ids.get(item["id"])
-            if task_id is not None and task_id != previous_task_ids.get(item["id"]):
-                item["task_id"] = task_id
-
-    return {
-        "status": "triggered" if triggered_sources else "partial",
-        "message": f"Triggered {len(triggered_sources)} data sources",
+    result = {
+        "status": "queued" if triggered_sources else "partial",
+        "message": f"Queued {len(triggered_sources)} data source jobs",
         "force": force,
         "triggered": triggered_sources,
         "skipped": skipped_sources,
         "failed": failed_sources,
     }
+    await emit_business_log(
+        logger,
+        event=f"collector.trigger.{trigger_kind}.completed",
+        message="Datasource batch trigger completed",
+        category="collector",
+        service="api",
+        module=__name__,
+        user_id=actor_id,
+        context={
+            "trigger_kind": trigger_kind,
+            "force": force,
+            "status": result["status"],
+            "requested_count": len(datasources),
+            "triggered_count": len(triggered_sources),
+            "skipped_count": len(skipped_sources),
+            "failed_count": len(failed_sources),
+            "triggered_sources": [item["source"] for item in triggered_sources],
+            "skipped_reasons": [item["reason"] for item in skipped_sources],
+        },
+    )
+    return result
 
 
 async def get_datasource_record(db: AsyncSession, source_id: str) -> Optional[DataSource]:
@@ -780,7 +821,13 @@ async def trigger_all_datasources(
         .order_by(DataSource.module, DataSource.id)
     )
     datasources = result.scalars().all()
-    return await _trigger_datasource_batch(db, datasources, force=force)
+    return await _trigger_datasource_batch(
+        db,
+        datasources,
+        force=force,
+        actor_id=current_user.id,
+        trigger_kind="all",
+    )
 
 
 @router.post("/trigger-batch")
@@ -816,7 +863,13 @@ async def trigger_datasource_batch(
         collected=None if payload.source_ids else payload.collected,
         credential_status=None if payload.source_ids else payload.credential_status,
     )
-    return await _trigger_datasource_batch(db, datasources, force=payload.force)
+    return await _trigger_datasource_batch(
+        db,
+        datasources,
+        force=payload.force,
+        actor_id=current_user.id,
+        trigger_kind="batch",
+    )
 
 
 @router.get("/snapshots")
@@ -909,7 +962,7 @@ async def get_datasource_row(
         [datasource],
         include_endpoint=include_endpoint,
     )
-    record_counts = await _load_collected_record_counts(db, [datasource.source])
+    record_counts = await _load_collected_record_counts(db, [datasource.source], exact_vessel_counts=True)
     return {
         "data": serialize_datasource_row(
             datasource,
@@ -992,8 +1045,24 @@ async def trigger_datasource(
     if not datasource.is_active:
         raise HTTPException(status_code=400, detail="Data source is disabled")
 
-    running_task = await get_running_task(db, datasource.id)
+    running_task = await get_active_datasource_job(db, datasource.id, task_types=(JOB_TYPE_COLLECT,))
     if running_task is not None and not force:
+        await emit_business_log(
+            logger,
+            event="collector.trigger.single.skipped_already_running",
+            message="Datasource trigger skipped because a task is already running",
+            category="collector",
+            level="warning",
+            service="api",
+            module=__name__,
+            user_id=current_user.id,
+            context={
+                "collector_name": datasource.source,
+                "datasource_id": datasource.id,
+                "task_id": running_task.id,
+                "status": "skipped",
+            },
+        )
         raise HTTPException(
             status_code=409,
             detail={
@@ -1013,31 +1082,42 @@ async def trigger_datasource(
         )
 
     if running_task is not None and force:
-        cancelled = await cancel_running_collector_now(datasource.source)
-        if not cancelled:
-            await rollback_orphaned_running_task(db, datasource, running_task)
+        await request_cancel_datasource_task(
+            db,
+            running_task,
+            reason="superseded_by_forced_collection",
+        )
 
-    previous_task_id = await get_latest_task_id_for_datasource(datasource.id)
-    success = run_collector_now(datasource.source)
-    if not success:
-        raise HTTPException(status_code=500, detail=f"Failed to trigger collector '{datasource.source}'")
-
-    task_id = None
-    for _ in range(20):
-        await asyncio.sleep(0.1)
-        task_id = await get_latest_task_id_for_datasource(datasource.id)
-        if task_id is not None and task_id != previous_task_id:
-            break
-    if task_id == previous_task_id:
-        task_id = None
+    task = await enqueue_datasource_job(
+        db,
+        datasource,
+        JOB_TYPE_COLLECT,
+        payload={"force": force, "trigger": "single"},
+    )
+    await emit_business_log(
+        logger,
+        event="collector.trigger.single.completed",
+        message="Datasource trigger queued",
+        category="collector",
+        service="api",
+        module=__name__,
+        user_id=current_user.id,
+        context={
+            "collector_name": datasource.source,
+            "datasource_id": datasource.id,
+            "task_id": task.id,
+            "force": force,
+            "status": "queued",
+        },
+    )
 
     return {
-        "status": "triggered",
+        "status": "queued",
         "source_id": datasource.id,
-        "task_id": task_id,
+        "task_id": task.id,
         "collector_name": datasource.source,
         "force": force,
-        "message": f"Collector '{datasource.source}' has been triggered",
+        "message": f"Collector '{datasource.source}' has been queued",
     }
 
 
@@ -1051,22 +1131,30 @@ async def clear_datasource_data(
     if not datasource:
         raise HTTPException(status_code=404, detail="Data source not found")
 
-    result = await db.execute(
-        select(func.count(CollectedData.id)).where(CollectedData.source == datasource.source)
+    active_task = await get_active_datasource_job(db, datasource.id)
+    if active_task is not None:
+        raise HTTPException(
+            status_code=409,
+            detail={
+                "reason": "datasource_job_in_progress",
+                "message": "当前数据源已有任务在执行，请等待完成或先取消任务。",
+                "task_id": active_task.id,
+                "task_type": active_task.task_type,
+                "status": active_task.status,
+            },
+        )
+    task = await enqueue_datasource_job(
+        db,
+        datasource,
+        JOB_TYPE_CLEAR_DATA,
+        payload={"source": datasource.source},
     )
-    count = result.scalar() or 0
-
-    if count == 0:
-        return {"status": "success", "message": "No data to clear", "deleted_count": 0}
-
-    delete_query = CollectedData.__table__.delete().where(CollectedData.source == datasource.source)
-    await db.execute(delete_query)
-    await db.commit()
 
     return {
-        "status": "success",
-        "message": f"Cleared {count} records for data source '{datasource.name}'",
-        "deleted_count": count,
+        "status": "queued",
+        "message": f"Queued data clearing for data source '{datasource.name}'",
+        "task_id": task.id,
+        "deleted_count": None,
     }
 
 
@@ -1080,16 +1168,44 @@ async def clear_datasource_cache(
     if not datasource:
         raise HTTPException(status_code=404, detail="Data source not found")
 
-    earth_deleted_count = invalidate_earth_layer_cache_for_source(datasource.source)
-    dashboard_deleted_count = int(cache.delete("dashboard:stats")) + int(cache.delete("dashboard:summary"))
-    deleted_count = earth_deleted_count + dashboard_deleted_count
+    task = await enqueue_datasource_job(
+        db,
+        datasource,
+        JOB_TYPE_CLEAR_CACHE,
+        payload={"source": datasource.source},
+        dedupe_key=f"clear_cache:{datasource.source}",
+    )
 
     return {
-        "status": "success",
-        "message": f"Cleared {deleted_count} cache keys for data source '{datasource.name}'",
-        "deleted_count": deleted_count,
-        "earth_layer_deleted_count": earth_deleted_count,
-        "dashboard_deleted_count": dashboard_deleted_count,
+        "status": "queued",
+        "message": f"Queued cache clearing for data source '{datasource.name}'",
+        "task_id": task.id,
+        "deleted_count": None,
+    }
+
+
+@router.post("/{source_id}/tasks/{task_id}/cancel")
+async def cancel_datasource_task(
+    source_id: str,
+    task_id: int,
+    current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    datasource = await get_datasource_record(db, source_id)
+    if not datasource:
+        raise HTTPException(status_code=404, detail="Data source not found")
+
+    task = await db.get(CollectionTask, task_id)
+    if not task or task.datasource_id != datasource.id:
+        raise HTTPException(status_code=404, detail="Task not found")
+
+    task = await request_cancel_datasource_task(db, task)
+    return {
+        "status": "cancelled" if task.completed_at else "cancelling",
+        "task_id": task.id,
+        "task_type": task.task_type,
+        "phase": task.phase,
+        "requested_cancel_at": to_iso8601_utc(task.requested_cancel_at),
     }
 
 
@@ -1109,7 +1225,7 @@ async def get_task_status(
         if not task or task.datasource_id != datasource.id:
             raise HTTPException(status_code=404, detail="Task not found")
     else:
-        task = await get_running_task(db, datasource.id)
+        task = await get_active_datasource_job(db, datasource.id)
         if task is None:
             result = await db.execute(
                 select(CollectionTask)
@@ -1134,8 +1250,12 @@ async def get_task_status(
         }
 
     return {
-        "is_running": task.status == "running",
+        "is_running": task.status in {JOB_STATUS_QUEUED, JOB_STATUS_RUNNING, JOB_STATUS_CANCELLING}
+        and task.task_type == JOB_TYPE_COLLECT,
+        "is_task_active": task.status in {JOB_STATUS_QUEUED, JOB_STATUS_RUNNING, JOB_STATUS_CANCELLING},
         "task_id": task.id,
+        "task_type": task.task_type,
+        "task_status": task.status,
         "progress": task.progress,
         "phase": task.phase,
         "phase_progress": task.phase_progress,
@@ -1146,5 +1266,6 @@ async def get_task_status(
         "records_processed": task.records_processed,
         "total_records": task.total_records,
         "status": task.status,
+        "requested_cancel_at": to_iso8601_utc(task.requested_cancel_at),
         "error_message": task.error_message,
     }

backend/app/api/v1/earth.py

@@ -12,6 +12,7 @@ from pydantic import BaseModel, Field
 from sqlalchemy import delete, func, select, text
 from sqlalchemy.ext.asyncio import AsyncSession
 
+from app.core.config import settings as app_settings
 from app.core.security import decode_token, get_current_user, redis_client
 from app.db.session import get_db
 from app.models.collected_data import CollectedData
@@ -36,9 +37,16 @@ EARTH_BRAND_ASSET_DIR = REPO_ROOT / "data" / "earth-brand"
 EARTH_BRAND_ASSET_URL_PREFIX = "/earth-brand-assets"
 EARTH_BRAND_CATEGORY = "earth_brand"
 EARTH_ABOUT_CATEGORY = "earth_about"
+SYSTEM_SETTINGS_CATEGORY = "system"
 MAX_EARTH_BRAND_ASSET_BYTES = 3 * 1024 * 1024
 ALLOWED_EARTH_BRAND_EXTENSIONS = {".png", ".jpg", ".jpeg", ".webp", ".svg"}
 
+
+def _app_version_label() -> str:
+    version = str(app_settings.VERSION or "").strip() or "0.0.0"
+    return version if version.startswith("v") else f"v{version}"
+
+
 DEFAULT_EARTH_BRAND = {
     "logo_src": "/earth/assets/brand/earth-logo.png",
     "title_src": "/earth/assets/brand/title-zh.png",
@@ -53,14 +61,15 @@ DEFAULT_EARTH_ABOUT = {
     "logo_src": "/earth/assets/brand/lim-logo.png",
     "kicker": "About",
     "title": "智能星球计划",
-    "version": "v0.64.0",
+    "version": _app_version_label(),
     "description": "面向临空场景下的智能媒体研究、全球态势感知与多源开放数据巡航，提供可视化观测、事件聚合与交互式探索能力。",
     "meta": [
         {"label": "出品方", "value": "浙江大学临空智能媒体研究院"},
-        {"label": "策划人", "value": "黄柳青"},
+        {"label": "策划人", "value": "方兴东、黄柳青"},
         {"label": "产品兼开发者", "value": "钱坤、张鸽、齐鹏"},
     ],
 }
+EARTH_ABOUT_LEGACY_PLANNER_VALUE = "黄柳青"
 
 
 class EarthBoundaryConfigPayload(BaseModel):
@@ -116,11 +125,12 @@ def _normalize_earth_about_payload(payload: dict[str, Any] | None) -> dict[str,
     }
     raw_meta = DEFAULT_EARTH_ABOUT["meta"]
     if payload:
-        for key in ("logo_src", "kicker", "title", "version", "description"):
+        for key in ("logo_src", "kicker", "title", "description"):
             value = payload.get(key)
             if value is not None:
                 merged[key] = str(value).strip()
         raw_meta = payload.get("meta") if isinstance(payload.get("meta"), list) else raw_meta
+    merged["version"] = _app_version_label()
 
     for key, default_value in DEFAULT_EARTH_ABOUT.items():
         if key == "meta":
@@ -134,6 +144,8 @@ def _normalize_earth_about_payload(payload: dict[str, Any] | None) -> dict[str,
             continue
         label = str(item.get("label") or "").strip()
         value = str(item.get("value") or "").strip()
+        if label == "策划人" and value == EARTH_ABOUT_LEGACY_PLANNER_VALUE:
+            value = "方兴东、黄柳青"
         if label or value:
             normalized_meta.append({"label": label, "value": value})
     if not normalized_meta:
@@ -142,6 +154,10 @@ def _normalize_earth_about_payload(payload: dict[str, Any] | None) -> dict[str,
     return merged
 
 
+def _is_demo_mode_enabled(payload: Any) -> bool:
+    return bool(payload.get("demo_mode")) if isinstance(payload, dict) else False
+
+
 async def _get_earth_brand_record(db: AsyncSession) -> SystemSetting | None:
     result = await db.execute(
         select(SystemSetting).where(SystemSetting.category == EARTH_BRAND_CATEGORY)
@@ -317,6 +333,11 @@ async def get_earth_oobe_status(
         select(func.count(CollectedData.id)).where(CollectedData.is_current.is_(True))
     )
     current_record_count = int(current_count_result.scalar() or 0)
+    system_result = await db.execute(
+        select(SystemSetting).where(SystemSetting.category == SYSTEM_SETTINGS_CATEGORY)
+    )
+    system_record = system_result.scalar_one_or_none()
+    demo_mode = _is_demo_mode_enabled(system_record.payload if system_record else None)
 
     datasource_count_result = await db.execute(select(func.count(DataSource.id)))
     datasource_count = int(datasource_count_result.scalar() or 0)
@@ -337,6 +358,8 @@ async def get_earth_oobe_status(
     ready = has_collected_data
 
     suggestions: list[str] = []
+    if demo_mode:
+        suggestions.append("演示模式已开启")
     if not current_user:
         suggestions.append("登录控制台")
     if not has_collected_data:
@@ -348,8 +371,9 @@ async def get_earth_oobe_status(
 
     return {
         "ready": ready,
+        "demo_mode": demo_mode,
         "authenticated": current_user is not None,
-        "needs_login": current_user is None and not ready,
+        "needs_login": current_user is None and not ready and not demo_mode,
         "has_collected_data": has_collected_data,
         "has_tv_sources": tv_source_count > 0,
         "has_core_layers": has_core_layers,

backend/app/api/v1/interactables.py

@@ -0,0 +1,190 @@
+"""CRUD APIs for persistent Earth interactables."""
+
+from __future__ import annotations
+
+from datetime import UTC, datetime
+from typing import Any
+
+from fastapi import APIRouter, Depends, HTTPException, Query, Response, status
+from pydantic import BaseModel, Field, field_validator
+from sqlalchemy.ext.asyncio import AsyncSession
+
+from app.core.security import get_current_user
+from app.db.session import get_db
+from app.models.earth_interactable import EarthInteractable
+from app.models.user import User
+from app.services.earth_interactables import (
+    build_interactable_event,
+    interactables_to_geojson,
+    invalidate_interactable_cache,
+    list_interactables,
+    normalize_interactable_id,
+    publish_interactable_event,
+    serialize_interactable,
+)
+from app.services.earth_layer_cache import (
+    EarthLayerCachePolicy,
+    earth_layer_cache,
+    get_or_build_layer_payload,
+)
+
+router = APIRouter()
+INTERACTABLE_CACHE_POLICY = EarthLayerCachePolicy(
+    fresh_ttl_seconds=60,
+    stale_ttl_seconds=10 * 60,
+    max_features=5000,
+)
+
+
+class InteractableCreate(BaseModel):
+    id: str | None = Field(default=None, max_length=160)
+    layer: str = Field(default="default", min_length=1, max_length=80)
+    kind: str = Field(default="default", min_length=1, max_length=80)
+    label: str = Field(default="", max_length=255)
+    description: str = Field(default="", max_length=4000)
+    latitude: float = Field(ge=-90, le=90)
+    longitude: float = Field(ge=-180, le=180)
+    altitude: float | None = None
+    properties: dict[str, Any] = Field(default_factory=dict)
+
+    @field_validator("layer", "kind")
+    @classmethod
+    def normalize_key(cls, value: str) -> str:
+        normalized = str(value or "").strip()
+        if not normalized:
+            raise ValueError("must not be empty")
+        return normalized
+
+
+class InteractableUpdate(BaseModel):
+    layer: str | None = Field(default=None, min_length=1, max_length=80)
+    kind: str | None = Field(default=None, min_length=1, max_length=80)
+    label: str | None = Field(default=None, max_length=255)
+    description: str | None = Field(default=None, max_length=4000)
+    latitude: float | None = Field(default=None, ge=-90, le=90)
+    longitude: float | None = Field(default=None, ge=-180, le=180)
+    altitude: float | None = None
+    properties: dict[str, Any] | None = None
+
+
+@router.get("")
+async def get_interactables(
+    response: Response,
+    layer: str | None = Query(default=None),
+    include_deleted: bool = Query(default=False),
+    db: AsyncSession = Depends(get_db),
+):
+    items = await list_interactables(db, layer=layer, include_deleted=include_deleted)
+    response.headers["X-Planet-Interactables-Count"] = str(len(items))
+    return {"items": [serialize_interactable(item) for item in items]}
+
+
+@router.get("/geojson")
+async def get_interactables_geojson(
+    response: Response,
+    layer: str | None = Query(default=None),
+    db: AsyncSession = Depends(get_db),
+):
+    async def build_payload() -> dict[str, Any]:
+        items = await list_interactables(db, layer=layer)
+        return interactables_to_geojson(items)
+
+    payload = await get_or_build_layer_payload(
+        key=earth_layer_cache.key("interactables", layer=layer or "all"),
+        policy=INTERACTABLE_CACHE_POLICY,
+        builder=build_payload,
+        response=response,
+    )
+    response.headers["X-Planet-Interactables-Count"] = str(len(payload.get("features") or []))
+    return payload
+
+
+@router.post("", status_code=status.HTTP_201_CREATED)
+async def create_interactable(
+    payload: InteractableCreate,
+    _current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    record_id = normalize_interactable_id(payload.id)
+    existing = await db.get(EarthInteractable, record_id)
+    if existing and not existing.is_deleted:
+        raise HTTPException(status_code=409, detail="Interactable already exists")
+
+    if existing is None:
+        record = EarthInteractable(id=record_id)
+        db.add(record)
+    else:
+        record = existing
+        record.is_deleted = False
+        record.deleted_at = None
+        record.revision += 1
+
+    record.layer = payload.layer
+    record.kind = payload.kind
+    record.label = payload.label
+    record.description = payload.description
+    record.latitude = payload.latitude
+    record.longitude = payload.longitude
+    record.altitude = payload.altitude
+    record.properties = payload.properties
+
+    await db.commit()
+    await db.refresh(record)
+    invalidate_interactable_cache(record.layer)
+    await publish_interactable_event("created", record)
+    return {"item": serialize_interactable(record)}
+
+
+@router.get("/{interactable_id}")
+async def get_interactable(interactable_id: str, db: AsyncSession = Depends(get_db)):
+    record = await db.get(EarthInteractable, interactable_id)
+    if record is None or record.is_deleted:
+        raise HTTPException(status_code=404, detail="Interactable not found")
+    return {"item": serialize_interactable(record)}
+
+
+@router.patch("/{interactable_id}")
+async def update_interactable(
+    interactable_id: str,
+    payload: InteractableUpdate,
+    _current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    record = await db.get(EarthInteractable, interactable_id)
+    if record is None or record.is_deleted:
+        raise HTTPException(status_code=404, detail="Interactable not found")
+
+    previous_layer = record.layer
+    patch = payload.model_dump(exclude_unset=True)
+    for key, value in patch.items():
+        setattr(record, key, value)
+    record.revision += 1
+
+    await db.commit()
+    await db.refresh(record)
+    invalidate_interactable_cache(previous_layer)
+    if record.layer != previous_layer:
+        invalidate_interactable_cache(record.layer)
+    await publish_interactable_event("updated", record)
+    return {"item": serialize_interactable(record)}
+
+
+@router.delete("/{interactable_id}")
+async def delete_interactable(
+    interactable_id: str,
+    _current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    record = await db.get(EarthInteractable, interactable_id)
+    if record is None or record.is_deleted:
+        raise HTTPException(status_code=404, detail="Interactable not found")
+
+    record.is_deleted = True
+    record.deleted_at = datetime.now(UTC)
+    record.revision += 1
+    await db.commit()
+    await db.refresh(record)
+    invalidate_interactable_cache(record.layer)
+    await publish_interactable_event("deleted", record)
+    event = build_interactable_event(action="deleted", record=record, include_item=True)
+    return {"deleted": True, "event": event}

backend/app/api/v1/settings.py

@@ -11,6 +11,7 @@ from dotenv import dotenv_values
 from sqlalchemy import select
 from sqlalchemy.ext.asyncio import AsyncSession
 
+from app.core.logging import get_logger
 from app.core.security import get_current_user
 from app.core.time import to_iso8601_utc
 from app.core.config import settings as app_settings
@@ -66,8 +67,10 @@ from app.services.llm_provider_catalog import (
 from app.services.scheduler import sync_datasource_job
 from app.services.tv_streams import DEFAULT_TV_SETTINGS, get_tv_settings_payload, normalize_tv_settings
 from app.services.persistent_logs import record_audit_log
+from app.services.business_logs import emit_business_log, exception_context
 
 router = APIRouter()
+logger = get_logger(__name__, service="api")
 AI_PROVIDER_QUICK_CONNECT_TIMEOUT_SECONDS = 5
 AI_CONNECTION_TEST_PROMPT_KEY = "ai.connection_test"
 SECRET_REVEAL_ROLES = {"admin", "super_admin"}
@@ -79,6 +82,7 @@ DEFAULT_SETTINGS = {
         "auto_refresh": True,
         "data_retention_days": 30,
         "max_concurrent_tasks": 5,
+        "demo_mode": False,
     },
     "notifications": {
         "email_enabled": False,
@@ -203,6 +207,7 @@ class SystemSettingsUpdate(BaseModel):
     auto_refresh: bool = True
     data_retention_days: int = Field(default=30, ge=1, le=3650)
     max_concurrent_tasks: int = Field(default=5, ge=1, le=50)
+    demo_mode: bool = False
 
 
 class NotificationSettingsUpdate(BaseModel):
@@ -699,6 +704,18 @@ async def _validate_ai_provider_full_connection(ai_payload: dict) -> dict:
         retry_attempts=runtime_config["retry_attempts"],
         llm_config=runtime_config.get("llm_config") or {},
     )
+    await emit_business_log(
+        logger,
+        event="settings.ai_provider.full_connection.start",
+        message="AI provider full connection validation started",
+        category="ai",
+        service="api",
+        module=__name__,
+        context={
+            "provider": runtime_config.get("llm_config", {}).get("provider"),
+            "model": runtime_config.get("llm_config", {}).get("model"),
+        },
+    )
     status_result = await client.get_status()
     if not status_result.configured:
         raise HTTPException(
@@ -715,6 +732,19 @@ async def _validate_ai_provider_full_connection(ai_payload: dict) -> dict:
             constraints=["回复尽量简短。"],
         )
     )
+    await emit_business_log(
+        logger,
+        event="settings.ai_provider.full_connection.success",
+        message="AI provider full connection validation completed",
+        category="ai",
+        service="api",
+        module=__name__,
+        context={
+            "provider": analysis_result.provider,
+            "model": analysis_result.model,
+            "configured": status_result.configured,
+        },
+    )
     return {
         "status": status_result.model_dump(),
         "provider": analysis_result.provider,
@@ -1612,9 +1642,34 @@ async def connect_ai_provider_integration(
         llm_config=quick_llm_config,
     )
 
+    await emit_business_log(
+        logger,
+        event="settings.ai_provider.connect.start",
+        message="AI provider connection test started",
+        category="ai",
+        service="api",
+        module=__name__,
+        user_id=current_user.id,
+        context={
+            "provider": payload.provider,
+            "model": payload.model,
+            "timeout_seconds": min(int(runtime_config["timeout_seconds"] or 60), AI_PROVIDER_QUICK_CONNECT_TIMEOUT_SECONDS),
+        },
+    )
     try:
         status_result = await client.get_status()
         if not status_result.configured:
+            await emit_business_log(
+                logger,
+                event="settings.ai_provider.connect.failed",
+                message="AI provider connection test failed because provider is incomplete",
+                category="ai",
+                level="warning",
+                service="api",
+                module=__name__,
+                user_id=current_user.id,
+                context={"provider": payload.provider, "model": payload.model, "configured": False},
+            )
             return {
                 "success": False,
                 "connected": False,
@@ -1628,6 +1683,21 @@ async def connect_ai_provider_integration(
                 AI_PROVIDER_QUICK_CONNECT_TIMEOUT_SECONDS,
             ),
         )
+        await emit_business_log(
+            logger,
+            event="settings.ai_provider.connect.success",
+            message="AI provider connection test completed",
+            category="ai",
+            service="api",
+            module=__name__,
+            user_id=current_user.id,
+            context={
+                "provider": payload.provider,
+                "model": payload.model,
+                "configured": True,
+                "lightweight_status": lightweight_result.get("status"),
+            },
+        )
         return {
             **lightweight_result,
             "status": status_result.model_dump(),
@@ -1639,6 +1709,17 @@ async def connect_ai_provider_integration(
             "message": str(exc.detail),
         }
     except Exception as exc:
+        await emit_business_log(
+            logger,
+            event="settings.ai_provider.connect.failed",
+            message="AI provider connection test failed",
+            category="ai",
+            level="error",
+            service="api",
+            module=__name__,
+            user_id=current_user.id,
+            context=exception_context(exc, {"provider": payload.provider, "model": payload.model}),
+        )
         return {
             "success": False,
             "connected": False,
@@ -1786,8 +1867,28 @@ async def connect_web_search_integration(
     runtime_config = _runtime_config_from_web_search_payload(draft_web_search_payload)
     client = WebSearchClient(runtime_config)
 
+    await emit_business_log(
+        logger,
+        event="settings.web_search.connect.start",
+        message="WebSearch connection test started",
+        category="ai_tool",
+        service="api",
+        module=__name__,
+        user_id=current_user.id,
+        context={"provider": runtime_config.default_provider},
+    )
     try:
         results = await client.test_connection()
+        await emit_business_log(
+            logger,
+            event="settings.web_search.connect.success",
+            message="WebSearch connection test completed",
+            category="ai_tool",
+            service="api",
+            module=__name__,
+            user_id=current_user.id,
+            context={"provider": runtime_config.default_provider, "result_count": len(results)},
+        )
         return {
             "success": True,
             "connected": True,
@@ -1796,18 +1897,51 @@ async def connect_web_search_integration(
             "results": [item.model_dump(mode="json") for item in results[:3]],
         }
     except WebSearchConfigurationError as exc:
+        await emit_business_log(
+            logger,
+            event="settings.web_search.connect.failed",
+            message="WebSearch connection test failed because configuration is incomplete",
+            category="ai_tool",
+            level="warning",
+            service="api",
+            module=__name__,
+            user_id=current_user.id,
+            context=exception_context(exc, {"provider": runtime_config.default_provider}),
+        )
         return {
             "success": False,
             "connected": False,
             "message": str(exc),
         }
     except WebSearchError as exc:
+        await emit_business_log(
+            logger,
+            event="settings.web_search.connect.failed",
+            message="WebSearch connection test failed",
+            category="ai_tool",
+            level="error",
+            service="api",
+            module=__name__,
+            user_id=current_user.id,
+            context=exception_context(exc, {"provider": runtime_config.default_provider}),
+        )
         return {
             "success": False,
             "connected": False,
             "message": str(exc),
         }
     except Exception as exc:
+        await emit_business_log(
+            logger,
+            event="settings.web_search.connect.failed",
+            message="WebSearch connection test failed",
+            category="ai_tool",
+            level="error",
+            service="api",
+            module=__name__,
+            user_id=current_user.id,
+            context=exception_context(exc, {"provider": runtime_config.default_provider}),
+        )
         return {
             "success": False,
             "connected": False,
@@ -1835,17 +1969,60 @@ async def generate_provider_credential_guide(
     ai_client: AIProviderClient = Depends(get_ai_provider_client),
 ):
     try:
+        await emit_business_log(
+            logger,
+            event="settings.credential_guide.generate.start",
+            message="Credential guide generation started",
+            category="ai",
+            service="api",
+            module=__name__,
+            user_id=current_user.id,
+            context={"provider": provider},
+        )
         web_search_client = await get_web_search_client(db)
-        return {
-            "guide": await generate_credential_guide(
+        guide = await generate_credential_guide(
                 db,
                 provider,
                 ai_client,
                 web_search_client,
             )
-        }
+        await emit_business_log(
+            logger,
+            event="settings.credential_guide.generate.success",
+            message="Credential guide generation completed",
+            category="ai",
+            service="api",
+            module=__name__,
+            user_id=current_user.id,
+            context={"provider": provider},
+        )
+        return {"guide": guide}
     except ValueError as exc:
+        await emit_business_log(
+            logger,
+            event="settings.credential_guide.generate.failed",
+            message="Credential guide generation failed",
+            category="ai",
+            level="warning",
+            service="api",
+            module=__name__,
+            user_id=current_user.id,
+            context=exception_context(exc, {"provider": provider}),
+        )
         raise HTTPException(status_code=404, detail=str(exc)) from exc
+    except Exception as exc:
+        await emit_business_log(
+            logger,
+            event="settings.credential_guide.generate.failed",
+            message="Credential guide generation failed",
+            category="ai",
+            level="error",
+            service="api",
+            module=__name__,
+            user_id=current_user.id,
+            context=exception_context(exc, {"provider": provider}),
+        )
+        raise
 
 
 @router.post("/credential-guides/{provider}/reset")

backend/app/api/v1/system_control.py

@@ -8,13 +8,11 @@ from datetime import datetime
 
 from fastapi import APIRouter, Depends, HTTPException, Query, Request, status
 from pydantic import BaseModel
-from sqlalchemy import select
 from sqlalchemy.ext.asyncio import AsyncSession
 
 from app.core.config import ROOT_DIR
 from app.core.security import get_current_user
 from app.db.session import get_db
-from app.models.system_log import AuditLog, SystemLog
 from app.models.user import User
 from app.services.persistent_logs import record_audit_log, record_system_log
 from app.services.system_control import (
@@ -37,6 +35,7 @@ from app.services.system_logs import (
     append_buffer_log,
     list_log_sources,
     normalize_log_level,
+    read_database_log_snapshot,
     read_log_snapshot,
 )
 from app.services.earth_layer_cache import earth_layer_cache
@@ -117,6 +116,46 @@ class EarthClientLogEventResponse(BaseModel):
     level: str
 
 
+async def ingest_client_log_event(
+    source_id: str,
+    *,
+    service: str,
+    event: str,
+    default_module: str,
+    default_category: str,
+    payload: EarthClientLogEventCreate,
+    request: Request,
+) -> EarthClientLogEventResponse:
+    normalized_level = normalize_log_level(payload.level)
+    append_buffer_log(
+        source_id,
+        level=normalized_level,
+        message=payload.message,
+        context={
+            "category": payload.category or "",
+            "url": payload.url or "",
+            "module": payload.module or "",
+            "detail": payload.detail or "",
+        },
+    )
+    await record_system_log(
+        source=source_id,
+        service=service,
+        module=payload.module or default_module,
+        event=event,
+        level=normalized_level,
+        message=payload.message,
+        category=payload.category or default_category,
+        context={
+            "url": payload.url or "",
+            "detail": payload.detail or "",
+            "module": payload.module or "",
+            "client_ip": request.client.host if request.client else "",
+        },
+    )
+    return EarthClientLogEventResponse(accepted=True, source_id=source_id, level=normalized_level)
+
+
 class EarthLayerCacheStatusResponse(BaseModel):
     prefix: str
     key_count: int
@@ -339,97 +378,6 @@ async def get_system_log_sources(
     }
 
 
-async def read_database_log_snapshot(
-    source_id: str,
-    *,
-    limit: int,
-    level: str,
-    levels: str | None,
-    start_date: str | None,
-    end_date: str | None,
-    search: str | None,
-    db: AsyncSession,
-) -> dict | None:
-    selected_levels = set(normalize_log_level(item) for item in (levels or level).split(",") if item.strip())
-    selected_levels.discard("all")
-    search_query = (search or "").strip().lower()
-    lines: list[str] = []
-
-    if source_id == "system-db":
-        query = select(SystemLog).order_by(SystemLog.occurred_at.desc().nullslast(), SystemLog.id.desc()).limit(limit * 5)
-        result = await db.execute(query)
-        records = result.scalars().all()
-        for record in records:
-            record_level = normalize_log_level(record.level)
-            if selected_levels and record_level not in selected_levels:
-                continue
-            occurred_at = record.occurred_at.date().isoformat() if record.occurred_at else ""
-            if start_date and occurred_at and occurred_at < start_date:
-                continue
-            if end_date and occurred_at and occurred_at > end_date:
-                continue
-            line = " ".join(
-                part
-                for part in [
-                    record.occurred_at.isoformat() if record.occurred_at else "",
-                    record_level.upper(),
-                    record.source,
-                    record.event or "",
-                    record.message,
-                ]
-                if part
-            )
-            if search_query and search_query not in line.lower():
-                continue
-            lines.append(line)
-    elif source_id == "audit-db":
-        query = select(AuditLog).order_by(AuditLog.occurred_at.desc().nullslast(), AuditLog.id.desc()).limit(limit * 5)
-        result = await db.execute(query)
-        records = result.scalars().all()
-        for record in records:
-            occurred_at = record.occurred_at.date().isoformat() if record.occurred_at else ""
-            if start_date and occurred_at and occurred_at < start_date:
-                continue
-            if end_date and occurred_at and occurred_at > end_date:
-                continue
-            line = " ".join(
-                part
-                for part in [
-                    record.occurred_at.isoformat() if record.occurred_at else "",
-                    "INFO",
-                    record.action,
-                    record.target_type or "",
-                    record.target_id or "",
-                    record.result or "",
-                ]
-                if part
-            )
-            if search_query and search_query not in line.lower():
-                continue
-            lines.append(line)
-    else:
-        return None
-
-    lines = list(reversed(lines[:limit]))
-    return {
-        "source_id": source_id,
-        "name": "系统事件" if source_id == "system-db" else "审计事件",
-        "kind": "database",
-        "location": "table://system_logs" if source_id == "system-db" else "table://audit_logs",
-        "description": "数据库持久化日志",
-        "category": "database" if source_id == "system-db" else "audit",
-        "status": "ok" if lines else "empty",
-        "level": level,
-        "selected_levels": sorted(selected_levels),
-        "search_query": search or "",
-        "available_levels": ["all", "error", "warning", "info", "debug"],
-        "daily_markers": [],
-        "line_limit": limit,
-        "line_count": len(lines),
-        "lines": lines,
-    }
-
-
 @router.get("/logs/{source_id}", response_model=SystemLogSnapshotResponse)
 async def get_system_log_snapshot(
     source_id: str,
@@ -493,31 +441,28 @@ async def ingest_earth_client_log(
     payload: EarthClientLogEventCreate,
     request: Request,
 ):
-    normalized_level = normalize_log_level(payload.level)
-    append_buffer_log(
+    return await ingest_client_log_event(
         "earth-client",
-        level=normalized_level,
-        message=payload.message,
-        context={
-            "category": payload.category or "",
-            "url": payload.url or "",
-            "module": payload.module or "",
-            "detail": payload.detail or "",
-        },
-    )
-    await record_system_log(
-        source="earth-client",
         service="earth",
-        module=payload.module or "earth-client",
         event="earth.client.runtime_log",
-        level=normalized_level,
-        message=payload.message,
-        category=payload.category or "client-runtime",
-        context={
-            "url": payload.url or "",
-            "detail": payload.detail or "",
-            "module": payload.module or "",
-            "client_ip": request.client.host if request.client else "",
-        },
+        default_module="earth-client",
+        default_category="client-runtime",
+        payload=payload,
+        request=request,
+    )
+
+
+@router.post("/logs/admin-client", response_model=EarthClientLogEventResponse)
+async def ingest_admin_client_log(
+    payload: EarthClientLogEventCreate,
+    request: Request,
+):
+    return await ingest_client_log_event(
+        "admin-client",
+        service="admin",
+        event="admin.client.runtime_log",
+        default_module="admin-client",
+        default_category="client-runtime",
+        payload=payload,
+        request=request,
     )
-    return {"accepted": True, "source_id": "earth-client", "level": normalized_level}

backend/app/api/v1/tasks.py

@@ -29,7 +29,8 @@ async def list_tasks(
         SELECT ct.id, ct.datasource_id, ds.name as datasource_name, ct.status,
                ct.started_at, ct.completed_at, ct.records_processed, ct.error_message,
                ct.phase, ct.phase_progress, ct.phase_message, ct.phase_current,
-               ct.phase_total, ct.phase_unit, ct.total_records, ct.progress
+               ct.phase_total, ct.phase_unit, ct.total_records, ct.progress,
+               ct.task_type, ct.source, ds.source as datasource_source
         FROM collection_tasks ct
         JOIN data_sources ds ON ct.datasource_id = ds.id
         WHERE 1=1
@@ -39,12 +40,19 @@ async def list_tasks(
 
     if datasource_id:
         query += " AND ct.datasource_id = :datasource_id"
-        count_query += " WHERE ct.datasource_id = :datasource_id"
+        count_query += " AND ct.datasource_id = :datasource_id"
         params["datasource_id"] = datasource_id
     if status:
-        query += " AND ct.status = :status"
-        count_query += " AND ct.status = :status"
-        params["status"] = status
+        statuses = [item.strip() for item in status.split(",") if item.strip()]
+        if len(statuses) > 1:
+            placeholders = ", ".join(f":status_{index}" for index, _item in enumerate(statuses))
+            query += f" AND ct.status IN ({placeholders})"
+            count_query += f" AND ct.status IN ({placeholders})"
+            params.update({f"status_{index}": item for index, item in enumerate(statuses)})
+        else:
+            query += " AND ct.status = :status"
+            count_query += " AND ct.status = :status"
+            params["status"] = statuses[0] if statuses else status
 
     query += f" ORDER BY ct.created_at DESC LIMIT {page_size} OFFSET {offset}"
 
@@ -76,6 +84,9 @@ async def list_tasks(
                 "phase_unit": t[13],
                 "total_records": t[14],
                 "progress": t[15],
+                "task_type": t[16],
+                "source": t[17] or t[18],
+                "datasource_source": t[18],
             }
             for t in tasks
         ],

backend/app/api/v1/visualization.py

@@ -38,7 +38,7 @@ from app.services.compute_center_locations import (
     upsert_compute_center_location,
 )
 from app.services.ai_client import get_ai_provider_client
-from app.api.v1.settings import get_web_search_client
+from app.api.v1.settings import get_runtime_web_search_config, get_web_search_client
 from app.services.collectors.bgp_common import RIPE_RIS_COLLECTOR_COORDS
 from app.services.location.llm_fallback import (
     collect_llm_location_fallback_candidate,
@@ -105,8 +105,8 @@ LANDING_POINT_CACHE_POLICY = EarthLayerCachePolicy(
 SATELLITE_CACHE_POLICY = EarthLayerCachePolicy(
     SATELLITE_CACHE_FRESH_SECONDS,
     SATELLITE_CACHE_STALE_SECONDS,
-    max_features=8000,
-    max_bytes=10 * BYTES_PER_MIB,
+    max_features=25000,
+    max_bytes=32 * BYTES_PER_MIB,
 )
 COMPUTE_CENTER_CACHE_POLICY = EarthLayerCachePolicy(
     COMPUTE_CENTER_CACHE_FRESH_SECONDS,
@@ -1563,15 +1563,7 @@ async def _build_cables_geojson(db: AsyncSession) -> dict[str, Any]:
     try:
         records = await _load_current_collected_data(db, "arcgis_cables")
 
-        if not records:
-            raise HTTPException(
-                status_code=404,
-                detail="No cable data found. Please run the arcgis_cables collector first.",
-            )
-
         return convert_cable_to_geojson(records)
-    except HTTPException:
-        raise
     except Exception as e:
         logger.exception_event(
             "Failed to build cables GeoJSON response",
@@ -1625,16 +1617,8 @@ async def _build_landing_points_geojson(db: AsyncSession) -> dict[str, Any]:
             relation_records,
             cable_records,
         )
-        
-        if not records:
-            raise HTTPException(
-                status_code=404,
-                detail="No landing point data found. Please run the arcgis_landing_points collector first.",
-            )
-        
+
         return convert_landing_point_to_geojson(records, city_to_cable_ids_map, cable_id_to_name_map)
-    except HTTPException:
-        raise
     except Exception as e:
         logger.exception_event(
             "Failed to build landing points GeoJSON response",
@@ -2071,6 +2055,44 @@ class SaveComputeCenterLocationRequest(BaseModel):
     model_config = {"populate_by_name": True}
 
 
+async def _compute_center_location_web_search_capability(db: AsyncSession) -> Dict[str, Any]:
+    try:
+        config = await get_runtime_web_search_config(db)
+    except Exception as exc:
+        return {
+            "enabled": False,
+            "provider": None,
+            "reason": f"WebSearch 配置读取失败：{exc}",
+        }
+    provider_config = config.active_provider_config
+    has_api_key = bool((provider_config.api_key or "").strip())
+    if not config.enabled:
+        return {
+            "enabled": False,
+            "provider": config.default_provider,
+            "reason": "WebSearch 未开启，无法进行事实核查定位。",
+        }
+    if not has_api_key:
+        return {
+            "enabled": False,
+            "provider": config.default_provider,
+            "reason": f"WebSearch Provider {config.default_provider} 未配置 API Key。",
+        }
+    return {
+        "enabled": True,
+        "provider": config.default_provider,
+        "reason": "",
+    }
+
+
+@router.get("/compute-centers/location-capability")
+async def get_compute_center_location_capability(
+    db: AsyncSession = Depends(get_db),
+):
+    """Return whether fact-checked compute-center location collection can run."""
+    return await _compute_center_location_web_search_capability(db)
+
+
 @router.post("/compute-centers/{source_id}/collect-location")
 async def collect_compute_center_location(
     source_id: str,
@@ -2089,6 +2111,9 @@ async def collect_compute_center_location(
     """
     if not source_id or not source_id.strip():
         raise HTTPException(status_code=400, detail="source_id is required")
+    capability = await _compute_center_location_web_search_capability(db)
+    if not capability.get("enabled"):
+        raise HTTPException(status_code=409, detail=capability)
 
     record = await _load_compute_center_record(db, source_id)
     name = payload.name or (record.name if record else None)
@@ -2696,6 +2721,8 @@ async def _build_bgp_collectors_geojson(db: AsyncSession) -> dict[str, Any]:
         db,
         source_filter=("ris_live_bgp", "bgpstream_bgp"),
     )
+    if not any(int(item.get("observation_count") or 0) > 0 for item in coverage):
+        return {"type": "FeatureCollection", "features": [], "count": 0}
     coverage_by_collector = {
         item["collector"]: item
         for item in coverage

backend/app/api/v1/websocket.py

@@ -6,11 +6,14 @@ from typing import Optional
 
 from fastapi import APIRouter, WebSocket, WebSocketDisconnect, Query
 from jose import jwt, JWTError
+from sqlalchemy import text
 
 from app.core.config import settings
 from app.core.logging import get_logger
 from app.core.time import to_iso8601_utc
 from app.core.websocket.manager import manager
+from app.db.session import async_session_factory
+from app.services.log_tail import LOG_TAIL_CHANNEL, log_tail_manager
 
 logger = get_logger(__name__, service="api")
 router = APIRouter()
@@ -37,6 +40,28 @@ async def authenticate_token(token: str) -> Optional[dict]:
         return None
 
 
+async def load_websocket_user_role(user_id: str | None) -> str | None:
+    if not user_id:
+        return None
+    try:
+        async with async_session_factory() as db:
+            result = await db.execute(
+                text("SELECT role, is_active FROM users WHERE id = :id"),
+                {"id": int(user_id)},
+            )
+            row = result.fetchone()
+    except Exception as exc:
+        logger.warning_event(
+            "WebSocket user role lookup failed",
+            event="auth.websocket.role_lookup_failed",
+            context={"user_id": user_id, "error": str(exc)},
+        )
+        return None
+    if row is None or not row[1]:
+        return None
+    return str(row[0] or "")
+
+
 @router.websocket("/ws")
 async def websocket_endpoint(
     websocket: WebSocket,
@@ -59,6 +84,7 @@ async def websocket_endpoint(
 
     is_anonymous = payload is None
     user_id = str(payload.get("sub")) if payload else f"anonymous:{id(websocket)}"
+    user_role = await load_websocket_user_role(user_id) if payload else None
     supported_channels = ["vessels", "earth_news", EARTH_UPDATES_CHANNEL] if is_anonymous else [
         "gpu_clusters",
         "submarine_cables",
@@ -70,6 +96,8 @@ async def websocket_endpoint(
         "earth_news",
         EARTH_UPDATES_CHANNEL,
     ]
+    if user_role == "super_admin":
+        supported_channels = [*supported_channels, LOG_TAIL_CHANNEL]
     await manager.connect(websocket, user_id)
 
     try:
@@ -100,6 +128,7 @@ async def websocket_endpoint(
                     payload_data = data.get("data", {})
                     if not isinstance(payload_data, dict):
                         payload_data = {}
+                    log_tail_config = None
                     channels = payload_data.get("channels", [])
                     if isinstance(channels, str):
                         channels = [channels]
@@ -108,6 +137,26 @@ async def websocket_endpoint(
                     channel = payload_data.get("channel")
                     if channel and channel not in channels:
                         channels = [*channels, channel]
+                    if LOG_TAIL_CHANNEL in channels:
+                        if user_role != "super_admin":
+                            await websocket.send_json(
+                                {
+                                    "type": "subscription_error",
+                                    "data": {"channel": LOG_TAIL_CHANNEL, "detail": "Only super_admin can subscribe logs"},
+                                }
+                            )
+                            channels = [item for item in channels if item != LOG_TAIL_CHANNEL]
+                        else:
+                            try:
+                                log_tail_config = await log_tail_manager.subscribe(websocket, payload_data)
+                            except ValueError as exc:
+                                await websocket.send_json(
+                                    {
+                                        "type": "subscription_error",
+                                        "data": {"channel": LOG_TAIL_CHANNEL, "detail": str(exc)},
+                                    }
+                                )
+                            channels = [item for item in channels if item != LOG_TAIL_CHANNEL]
                     if is_anonymous:
                         channels = [channel for channel in channels if channel in supported_channels]
                     vessel_subscription = None
@@ -131,14 +180,20 @@ async def websocket_endpoint(
                                 "action": "subscribe",
                                 "channels": [
                                     *channels,
+                                    *([LOG_TAIL_CHANNEL] if log_tail_config else []),
                                     *(["vessels"] if vessel_subscription else []),
                                 ],
                                 "vessels": vessel_subscription,
+                                "logs_tail": log_tail_config.__dict__ if log_tail_config else None,
                             },
                         }
                     )
                 elif data.get("type") == "unsubscribe":
                     channels = data.get("data", {}).get("channels", [])
+                    if isinstance(channels, str):
+                        channels = [channels]
+                    if LOG_TAIL_CHANNEL in channels:
+                        await log_tail_manager.unsubscribe(websocket)
                     manager.unsubscribe(websocket, channels)
                     await websocket.send_json(
                         {
@@ -159,4 +214,5 @@ async def websocket_endpoint(
     except WebSocketDisconnect:
         pass
     finally:
+        await log_tail_manager.disconnect(websocket)
         manager.disconnect(websocket, user_id)

backend/app/core/websocket/broadcaster.py

@@ -150,7 +150,7 @@ class DataBroadcaster:
                 "timestamp": to_iso8601_utc(datetime.now(UTC)),
                 "payload": data,
             },
-            channel="all",
+            channel="datasource_tasks",
         )
 
     def start(self):

backend/app/db/session.py

@@ -203,6 +203,7 @@ async def init_db():
     import app.models.vessel_enrichment  # noqa: F401
     import app.models.datasource_mapping  # noqa: F401
     import app.models.earth_news  # noqa: F401
+    import app.models.earth_interactable  # noqa: F401
 
     logger.warning_event(
         "Database pool settings active",
@@ -258,6 +259,406 @@ async def init_db():
                 """
             )
         )
+        await conn.execute(
+            text(
+                """
+                CREATE TABLE IF NOT EXISTS earth_data_change_events (
+                    id BIGSERIAL PRIMARY KEY,
+                    table_name VARCHAR(128) NOT NULL,
+                    operation VARCHAR(16) NOT NULL,
+                    source VARCHAR(128),
+                    entity_key VARCHAR(255),
+                    payload JSONB NOT NULL,
+                    occurred_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
+                    consumed_at TIMESTAMPTZ
+                )
+                """
+            )
+        )
+        await conn.execute(
+            text(
+                """
+                CREATE INDEX IF NOT EXISTS idx_earth_data_change_events_unconsumed
+                ON earth_data_change_events (consumed_at, id)
+                WHERE consumed_at IS NULL
+                """
+            )
+        )
+        await conn.execute(
+            text(
+                """
+                CREATE OR REPLACE FUNCTION planet_emit_earth_data_changed_statement(
+                    change_table TEXT,
+                    change_operation TEXT,
+                    change_source TEXT,
+                    source_record_count INTEGER,
+                    source_entity_keys TEXT[]
+                )
+                RETURNS VOID AS $$
+                DECLARE
+                    change_event_id BIGINT;
+                    change_payload JSONB;
+                BEGIN
+                    change_payload := jsonb_build_object(
+                        'event', 'earth.layer.changed',
+                        'table', change_table,
+                        'operation', change_operation,
+                        'source', change_source,
+                        'entity_key', NULL,
+                        'entity_keys', COALESCE(to_jsonb(source_entity_keys), '[]'::jsonb),
+                        'records_processed', COALESCE(source_record_count, 0),
+                        'occurred_at', NOW()
+                    );
+
+                    INSERT INTO earth_data_change_events (
+                        table_name,
+                        operation,
+                        source,
+                        entity_key,
+                        payload,
+                        occurred_at
+                    ) VALUES (
+                        change_table,
+                        change_operation,
+                        change_source,
+                        NULL,
+                        change_payload,
+                        NOW()
+                    )
+                    RETURNING id INTO change_event_id;
+
+                    change_payload := change_payload || jsonb_build_object(
+                        'event_id', change_event_id
+                    );
+
+                    UPDATE earth_data_change_events
+                    SET payload = change_payload
+                    WHERE id = change_event_id;
+
+                    PERFORM pg_notify(
+                        'planet_earth_data_changes',
+                        change_payload::text
+                    );
+                END;
+                $$ LANGUAGE plpgsql;
+                """
+            )
+        )
+        await conn.execute(
+            text(
+                """
+                CREATE OR REPLACE FUNCTION planet_emit_collected_data_changed_statement(
+                    change_operation TEXT,
+                    change_source TEXT,
+                    source_record_count INTEGER,
+                    source_entity_keys TEXT[]
+                )
+                RETURNS VOID AS $$
+                BEGIN
+                    PERFORM planet_emit_earth_data_changed_statement(
+                        'collected_data',
+                        change_operation,
+                        change_source,
+                        source_record_count,
+                        source_entity_keys
+                    );
+                END;
+                $$ LANGUAGE plpgsql;
+                """
+            )
+        )
+        await conn.execute(
+            text(
+                """
+                CREATE OR REPLACE FUNCTION planet_notify_earth_table_changed_statement()
+                RETURNS trigger AS $$
+                DECLARE
+                    change_source TEXT;
+                    source_record_count INTEGER;
+                    source_entity_keys TEXT[];
+                BEGIN
+                    IF TG_OP = 'INSERT' THEN
+                        FOR change_source IN
+                            SELECT DISTINCT COALESCE(NULLIF(row_data->>'source', ''), TG_TABLE_NAME)
+                            FROM (SELECT to_jsonb(t) AS row_data FROM new_rows AS t) changed_rows
+                        LOOP
+                            SELECT
+                                COUNT(*),
+                                ARRAY(
+                                    SELECT DISTINCT COALESCE(
+                                        NULLIF(row_data->>'entity_key', ''),
+                                        NULLIF(row_data->>'source_id', ''),
+                                        NULLIF(row_data->>'incident_key', ''),
+                                        NULLIF(row_data->>'id', ''),
+                                        NULLIF(row_data->>'mmsi', '')
+                                    )
+                                    FROM (SELECT to_jsonb(t) AS row_data FROM new_rows AS t) rows_for_keys
+                                    WHERE COALESCE(NULLIF(row_data->>'source', ''), TG_TABLE_NAME) = change_source
+                                    LIMIT 20
+                                )
+                            INTO source_record_count, source_entity_keys
+                            FROM (SELECT to_jsonb(t) AS row_data FROM new_rows AS t) rows_for_count
+                            WHERE COALESCE(NULLIF(row_data->>'source', ''), TG_TABLE_NAME) = change_source;
+
+                            PERFORM planet_emit_earth_data_changed_statement(
+                                TG_TABLE_NAME,
+                                TG_OP,
+                                change_source,
+                                source_record_count,
+                                source_entity_keys
+                            );
+                        END LOOP;
+                    ELSIF TG_OP = 'DELETE' THEN
+                        FOR change_source IN
+                            SELECT DISTINCT COALESCE(NULLIF(row_data->>'source', ''), TG_TABLE_NAME)
+                            FROM (SELECT to_jsonb(t) AS row_data FROM old_rows AS t) changed_rows
+                        LOOP
+                            SELECT
+                                COUNT(*),
+                                ARRAY(
+                                    SELECT DISTINCT COALESCE(
+                                        NULLIF(row_data->>'entity_key', ''),
+                                        NULLIF(row_data->>'source_id', ''),
+                                        NULLIF(row_data->>'incident_key', ''),
+                                        NULLIF(row_data->>'id', ''),
+                                        NULLIF(row_data->>'mmsi', '')
+                                    )
+                                    FROM (SELECT to_jsonb(t) AS row_data FROM old_rows AS t) rows_for_keys
+                                    WHERE COALESCE(NULLIF(row_data->>'source', ''), TG_TABLE_NAME) = change_source
+                                    LIMIT 20
+                                )
+                            INTO source_record_count, source_entity_keys
+                            FROM (SELECT to_jsonb(t) AS row_data FROM old_rows AS t) rows_for_count
+                            WHERE COALESCE(NULLIF(row_data->>'source', ''), TG_TABLE_NAME) = change_source;
+
+                            PERFORM planet_emit_earth_data_changed_statement(
+                                TG_TABLE_NAME,
+                                TG_OP,
+                                change_source,
+                                source_record_count,
+                                source_entity_keys
+                            );
+                        END LOOP;
+                    ELSIF TG_OP = 'UPDATE' THEN
+                        FOR change_source IN
+                            SELECT DISTINCT COALESCE(NULLIF(row_data->>'source', ''), TG_TABLE_NAME)
+                            FROM (
+                                SELECT to_jsonb(t) AS row_data FROM new_rows AS t
+                                UNION ALL
+                                SELECT to_jsonb(t) AS row_data FROM old_rows AS t
+                            ) changed_rows
+                        LOOP
+                            SELECT
+                                COUNT(*),
+                                ARRAY(
+                                    SELECT DISTINCT COALESCE(
+                                        NULLIF(row_data->>'entity_key', ''),
+                                        NULLIF(row_data->>'source_id', ''),
+                                        NULLIF(row_data->>'incident_key', ''),
+                                        NULLIF(row_data->>'id', ''),
+                                        NULLIF(row_data->>'mmsi', '')
+                                    )
+                                    FROM (
+                                        SELECT to_jsonb(t) AS row_data FROM new_rows AS t
+                                        UNION ALL
+                                        SELECT to_jsonb(t) AS row_data FROM old_rows AS t
+                                    ) rows_for_keys
+                                    WHERE COALESCE(NULLIF(row_data->>'source', ''), TG_TABLE_NAME) = change_source
+                                    LIMIT 20
+                                )
+                            INTO source_record_count, source_entity_keys
+                            FROM (
+                                SELECT to_jsonb(t) AS row_data FROM new_rows AS t
+                                UNION ALL
+                                SELECT to_jsonb(t) AS row_data FROM old_rows AS t
+                            ) rows_for_count
+                            WHERE COALESCE(NULLIF(row_data->>'source', ''), TG_TABLE_NAME) = change_source;
+
+                            PERFORM planet_emit_earth_data_changed_statement(
+                                TG_TABLE_NAME,
+                                TG_OP,
+                                change_source,
+                                source_record_count,
+                                source_entity_keys
+                            );
+                        END LOOP;
+                    END IF;
+
+                    RETURN NULL;
+                END;
+                $$ LANGUAGE plpgsql;
+                """
+            )
+        )
+        await conn.execute(
+            text(
+                """
+                CREATE OR REPLACE FUNCTION planet_notify_collected_data_changed_statement()
+                RETURNS trigger AS $$
+                DECLARE
+                    change_source TEXT;
+                    source_record_count INTEGER;
+                    source_entity_keys TEXT[];
+                BEGIN
+                    IF TG_OP = 'INSERT' THEN
+                        FOR change_source IN
+                            SELECT DISTINCT source FROM new_rows WHERE source IS NOT NULL
+                        LOOP
+                            SELECT
+                                COUNT(*),
+                                ARRAY(
+                                    SELECT DISTINCT COALESCE(entity_key, source_id, id::text)
+                                    FROM new_rows
+                                    WHERE source = change_source
+                                    LIMIT 20
+                                )
+                            INTO source_record_count, source_entity_keys
+                            FROM new_rows
+                            WHERE source = change_source;
+
+                            PERFORM planet_emit_collected_data_changed_statement(
+                                TG_OP,
+                                change_source,
+                                source_record_count,
+                                source_entity_keys
+                            );
+                        END LOOP;
+                    ELSIF TG_OP = 'DELETE' THEN
+                        FOR change_source IN
+                            SELECT DISTINCT source FROM old_rows WHERE source IS NOT NULL
+                        LOOP
+                            SELECT
+                                COUNT(*),
+                                ARRAY(
+                                    SELECT DISTINCT COALESCE(entity_key, source_id, id::text)
+                                    FROM old_rows
+                                    WHERE source = change_source
+                                    LIMIT 20
+                                )
+                            INTO source_record_count, source_entity_keys
+                            FROM old_rows
+                            WHERE source = change_source;
+
+                            PERFORM planet_emit_collected_data_changed_statement(
+                                TG_OP,
+                                change_source,
+                                source_record_count,
+                                source_entity_keys
+                            );
+                        END LOOP;
+                    ELSIF TG_OP = 'UPDATE' THEN
+                        FOR change_source IN
+                            SELECT DISTINCT source FROM (
+                                SELECT source FROM new_rows
+                                UNION
+                                SELECT source FROM old_rows
+                            ) changed_sources
+                            WHERE source IS NOT NULL
+                        LOOP
+                            SELECT
+                                COUNT(*),
+                                ARRAY(
+                                    SELECT DISTINCT COALESCE(entity_key, source_id, id::text)
+                                    FROM (
+                                        SELECT id, source_id, entity_key, source FROM new_rows
+                                        UNION ALL
+                                        SELECT id, source_id, entity_key, source FROM old_rows
+                                    ) changed_rows
+                                    WHERE source = change_source
+                                    LIMIT 20
+                                )
+                            INTO source_record_count, source_entity_keys
+                            FROM (
+                                SELECT id, source_id, entity_key, source FROM new_rows
+                                UNION ALL
+                                SELECT id, source_id, entity_key, source FROM old_rows
+                            ) changed_rows
+                            WHERE source = change_source;
+
+                            PERFORM planet_emit_collected_data_changed_statement(
+                                TG_OP,
+                                change_source,
+                                source_record_count,
+                                source_entity_keys
+                            );
+                        END LOOP;
+                    END IF;
+
+                    RETURN NULL;
+                END;
+                $$ LANGUAGE plpgsql;
+                """
+            )
+        )
+        for statement in (
+            "DROP TRIGGER IF EXISTS tr_planet_collected_data_changed ON collected_data",
+            "DROP TRIGGER IF EXISTS tr_planet_collected_data_changed_insert ON collected_data",
+            "DROP TRIGGER IF EXISTS tr_planet_collected_data_changed_update ON collected_data",
+            "DROP TRIGGER IF EXISTS tr_planet_collected_data_changed_delete ON collected_data",
+            "DROP FUNCTION IF EXISTS planet_notify_collected_data_changed()",
+            """
+            CREATE TRIGGER tr_planet_collected_data_changed_insert
+            AFTER INSERT ON collected_data
+            REFERENCING NEW TABLE AS new_rows
+            FOR EACH STATEMENT
+            EXECUTE FUNCTION planet_notify_collected_data_changed_statement()
+            """,
+            """
+            CREATE TRIGGER tr_planet_collected_data_changed_update
+            AFTER UPDATE ON collected_data
+            REFERENCING OLD TABLE AS old_rows NEW TABLE AS new_rows
+            FOR EACH STATEMENT
+            EXECUTE FUNCTION planet_notify_collected_data_changed_statement()
+            """,
+            """
+            CREATE TRIGGER tr_planet_collected_data_changed_delete
+            AFTER DELETE ON collected_data
+            REFERENCING OLD TABLE AS old_rows
+            FOR EACH STATEMENT
+            EXECUTE FUNCTION planet_notify_collected_data_changed_statement()
+            """,
+        ):
+            await conn.execute(text(statement))
+        for table_name in (
+            "bgp_observations",
+            "bgp_anomalies",
+            "bgp_incidents",
+            "bgp_collector_locations",
+            "vessel_static",
+            "vessel_position",
+            "ais_raw_observations",
+            "ais_source_health",
+            "compute_center_locations",
+            "earth_interactables",
+            "earth_news_items",
+        ):
+            for statement in (
+                f"DROP TRIGGER IF EXISTS tr_planet_{table_name}_changed_insert ON {table_name}",
+                f"DROP TRIGGER IF EXISTS tr_planet_{table_name}_changed_update ON {table_name}",
+                f"DROP TRIGGER IF EXISTS tr_planet_{table_name}_changed_delete ON {table_name}",
+                f"""
+                CREATE TRIGGER tr_planet_{table_name}_changed_insert
+                AFTER INSERT ON {table_name}
+                REFERENCING NEW TABLE AS new_rows
+                FOR EACH STATEMENT
+                EXECUTE FUNCTION planet_notify_earth_table_changed_statement()
+                """,
+                f"""
+                CREATE TRIGGER tr_planet_{table_name}_changed_update
+                AFTER UPDATE ON {table_name}
+                REFERENCING OLD TABLE AS old_rows NEW TABLE AS new_rows
+                FOR EACH STATEMENT
+                EXECUTE FUNCTION planet_notify_earth_table_changed_statement()
+                """,
+                f"""
+                CREATE TRIGGER tr_planet_{table_name}_changed_delete
+                AFTER DELETE ON {table_name}
+                REFERENCING OLD TABLE AS old_rows
+                FOR EACH STATEMENT
+                EXECUTE FUNCTION planet_notify_earth_table_changed_statement()
+                """,
+            ):
+                await conn.execute(text(statement))
         await conn.execute(
             text(
                 """
@@ -267,7 +668,16 @@ async def init_db():
                 ADD COLUMN IF NOT EXISTS phase_message VARCHAR(255),
                 ADD COLUMN IF NOT EXISTS phase_current BIGINT,
                 ADD COLUMN IF NOT EXISTS phase_total BIGINT,
-                ADD COLUMN IF NOT EXISTS phase_unit VARCHAR(30)
+                ADD COLUMN IF NOT EXISTS phase_unit VARCHAR(30),
+                ADD COLUMN IF NOT EXISTS source VARCHAR(100),
+                ADD COLUMN IF NOT EXISTS task_type VARCHAR(30) NOT NULL DEFAULT 'collect',
+                ADD COLUMN IF NOT EXISTS payload JSONB NOT NULL DEFAULT '{}'::jsonb,
+                ADD COLUMN IF NOT EXISTS rollback_policy VARCHAR(40) NOT NULL DEFAULT 'keep_committed_batches',
+                ADD COLUMN IF NOT EXISTS dedupe_key VARCHAR(180),
+                ADD COLUMN IF NOT EXISTS worker_id VARCHAR(120),
+                ADD COLUMN IF NOT EXISTS locked_at TIMESTAMPTZ,
+                ADD COLUMN IF NOT EXISTS requested_cancel_at TIMESTAMPTZ,
+                ADD COLUMN IF NOT EXISTS cancel_reason TEXT
                 """
             )
         )
@@ -283,6 +693,17 @@ async def init_db():
                 """
             )
         )
+        await conn.execute(
+            text(
+                """
+                ALTER TABLE earth_interactables
+                ADD COLUMN IF NOT EXISTS altitude DOUBLE PRECISION,
+                ADD COLUMN IF NOT EXISTS revision INTEGER NOT NULL DEFAULT 1,
+                ADD COLUMN IF NOT EXISTS is_deleted BOOLEAN NOT NULL DEFAULT FALSE,
+                ADD COLUMN IF NOT EXISTS deleted_at TIMESTAMPTZ
+                """
+            )
+        )
         await conn.execute(
             text(
                 """
@@ -307,6 +728,48 @@ async def init_db():
                 """
             )
         )
+        await conn.execute(
+            text(
+                """
+                CREATE INDEX IF NOT EXISTS idx_earth_interactables_layer_deleted
+                ON earth_interactables (layer, is_deleted)
+                """
+            )
+        )
+        await conn.execute(
+            text(
+                """
+                CREATE INDEX IF NOT EXISTS idx_earth_interactables_updated_at
+                ON earth_interactables (updated_at)
+                """
+            )
+        )
+        await conn.execute(
+            text(
+                """
+                CREATE INDEX IF NOT EXISTS idx_collection_tasks_source_status
+                ON collection_tasks (source, status)
+                """
+            )
+        )
+        await conn.execute(
+            text(
+                """
+                CREATE INDEX IF NOT EXISTS idx_collection_tasks_queue
+                ON collection_tasks (status, created_at, id)
+                WHERE status = 'queued'
+                """
+            )
+        )
+        await conn.execute(
+            text(
+                """
+                CREATE INDEX IF NOT EXISTS idx_collection_tasks_dedupe
+                ON collection_tasks (dedupe_key)
+                WHERE dedupe_key IS NOT NULL
+                """
+            )
+        )
         await conn.execute(
             text(
                 """

backend/app/main.py

@@ -24,6 +24,11 @@ from app.services.earth_news_worker import (
     start_earth_news_target_worker,
     stop_earth_news_target_worker,
 )
+from app.services.earth_db_change_listener import (
+    start_earth_db_change_listener,
+    stop_earth_db_change_listener,
+)
+from app.services.data_jobs import start_data_job_worker, stop_data_job_worker
 
 
 configure_logging()
@@ -59,9 +64,13 @@ async def lifespan(app: FastAPI):
     start_scheduler()
     await sync_scheduler_with_datasources()
     broadcaster.start()
+    start_data_job_worker()
+    start_earth_db_change_listener()
     start_earth_news_target_worker()
     yield
     await stop_earth_news_target_worker()
+    await stop_earth_db_change_listener()
+    await stop_data_job_worker()
     broadcaster.stop()
     stop_scheduler()
 

backend/app/models/__init__.py

@@ -17,6 +17,7 @@ from app.models.system_log import SystemLog, AuditLog
 from app.models.vessel import AISConflictRecord, AISRawObservation, AISSourceHealth, VesselPosition, VesselStatic
 from app.models.datasource_mapping import DataSourceMappingTemplate
 from app.models.earth_news import EarthNewsItem
+from app.models.earth_interactable import EarthInteractable
 
 __all__ = [
     "User",
@@ -45,4 +46,5 @@ __all__ = [
     "AISSourceHealth",
     "DataSourceMappingTemplate",
     "EarthNewsItem",
+    "EarthInteractable",
 ]

backend/app/models/earth_interactable.py

@@ -0,0 +1,30 @@
+"""Persistent Earth interactable objects."""
+
+from sqlalchemy import Boolean, Column, DateTime, Float, Index, Integer, JSON, String, Text
+from sqlalchemy.sql import func
+
+from app.db.session import Base
+
+
+class EarthInteractable(Base):
+    __tablename__ = "earth_interactables"
+
+    id = Column(String(160), primary_key=True)
+    layer = Column(String(80), nullable=False, default="interactables", index=True)
+    kind = Column(String(80), nullable=False, default="default", index=True)
+    label = Column(String(255), nullable=False, default="")
+    description = Column(Text, nullable=False, default="")
+    latitude = Column(Float, nullable=False)
+    longitude = Column(Float, nullable=False)
+    altitude = Column(Float, nullable=True)
+    revision = Column(Integer, nullable=False, default=1)
+    properties = Column(JSON, nullable=False, default=dict)
+    is_deleted = Column(Boolean, nullable=False, default=False, index=True)
+    created_at = Column(DateTime(timezone=True), server_default=func.now(), nullable=False)
+    updated_at = Column(DateTime(timezone=True), server_default=func.now(), onupdate=func.now())
+    deleted_at = Column(DateTime(timezone=True), nullable=True, index=True)
+
+    __table_args__ = (
+        Index("idx_earth_interactables_layer_deleted", "layer", "is_deleted"),
+        Index("idx_earth_interactables_updated_at", "updated_at"),
+    )

backend/app/models/task.py

@@ -1,6 +1,6 @@
-"""Collection Task model"""
+"""Datasource job model."""
 
-from sqlalchemy import BigInteger, Column, DateTime, Integer, String, Text, Float
+from sqlalchemy import BigInteger, Column, DateTime, Float, Integer, JSON, String, Text
 from sqlalchemy.sql import func
 
 from app.db.session import Base
@@ -11,7 +11,9 @@ class CollectionTask(Base):
 
     id = Column(Integer, primary_key=True, autoincrement=True)
     datasource_id = Column(Integer, nullable=False, index=True)
-    status = Column(String(20), nullable=False)  # pending, running, success, failed, cancelled
+    source = Column(String(100), nullable=True, index=True)
+    task_type = Column(String(30), nullable=False, default="collect", index=True)
+    status = Column(String(20), nullable=False)  # queued, running, cancelling, success, failed, cancelled
     phase = Column(String(30), default="queued")
     phase_progress = Column(Float)
     phase_message = Column(String(255))
@@ -24,6 +26,13 @@ class CollectionTask(Base):
     total_records = Column(Integer, default=0)  # Total records to process
     progress = Column(Float, default=0.0)  # Progress percentage (0-100)
     error_message = Column(Text)
+    payload = Column(JSON, default=dict)
+    rollback_policy = Column(String(40), nullable=False, default="keep_committed_batches")
+    dedupe_key = Column(String(180), nullable=True, index=True)
+    worker_id = Column(String(120), nullable=True, index=True)
+    locked_at = Column(DateTime(timezone=True), nullable=True, index=True)
+    requested_cancel_at = Column(DateTime(timezone=True), nullable=True)
+    cancel_reason = Column(Text)
     created_at = Column(DateTime(timezone=True), server_default=func.now())
 
     def __repr__(self):

backend/app/services/ai_client.py

@@ -2,18 +2,24 @@ from __future__ import annotations
 
 import asyncio
 import json
+from time import perf_counter
 
 import httpx
 from fastapi import Depends, HTTPException, status
 from sqlalchemy.ext.asyncio import AsyncSession
 
 from app.core.config import settings
+from app.core.logging import get_logger
 from app.db.session import get_db
 from app.schemas.ai import (
     AIProviderStatusResponse,
     SituationalAnalysisRequest,
     SituationalAnalysisResponse,
 )
+from app.services.business_logs import emit_business_log, exception_context
+
+
+logger = get_logger(__name__, service="ai")
 
 
 class AIProviderClient:
@@ -64,7 +70,19 @@ class AIProviderClient:
         return headers
 
     async def get_status(self, request_id: str | None = None) -> AIProviderStatusResponse:
+        context = self._base_log_context(operation="status")
         if not self.service_url:
+            await emit_business_log(
+                logger,
+                event="ai.provider.status.failed",
+                message="AI provider status skipped because service URL is not configured",
+                category="ai",
+                level="warning",
+                service="ai",
+                module=__name__,
+                request_id=request_id,
+                context={**context, "status": "unconfigured"},
+            )
             return AIProviderStatusResponse(
                 provider="unconfigured",
                 enabled=False,
@@ -73,27 +91,133 @@ class AIProviderClient:
                 base_url=None,
             )
 
-        data = await self._request("GET", "/v1/provider/status", request_id=request_id)
-        return AIProviderStatusResponse.model_validate(data)
+        started_at = perf_counter()
+        await emit_business_log(
+            logger,
+            event="ai.provider.status.start",
+            message="AI provider status request started",
+            category="ai",
+            service="ai",
+            module=__name__,
+            request_id=request_id,
+            context=context,
+        )
+        try:
+            data = await self._request("GET", "/v1/provider/status", request_id=request_id, operation="status")
+            result = AIProviderStatusResponse.model_validate(data)
+            await emit_business_log(
+                logger,
+                event="ai.provider.status.success",
+                message="AI provider status request completed",
+                category="ai",
+                service="ai",
+                module=__name__,
+                request_id=request_id,
+                context={
+                    **context,
+                    "status": "success",
+                    "duration_ms": self._duration_ms(started_at),
+                    "result_provider": result.provider,
+                    "result_model": result.model,
+                    "configured": result.configured,
+                    "enabled": result.enabled,
+                },
+            )
+            return result
+        except Exception as exc:
+            await emit_business_log(
+                logger,
+                event="ai.provider.status.failed",
+                message="AI provider status request failed",
+                category="ai",
+                level="error",
+                service="ai",
+                module=__name__,
+                request_id=request_id,
+                context=exception_context(exc, {**context, "status": "failed", "duration_ms": self._duration_ms(started_at)}),
+            )
+            raise
 
     async def analyze(
         self,
         payload: SituationalAnalysisRequest,
         request_id: str | None = None,
     ) -> SituationalAnalysisResponse:
+        context = self._base_log_context(
+            operation="analyze",
+            preferred_model=payload.preferred_model,
+            input_summary=self._summarize_analysis_payload(payload),
+        )
         if not self.service_url:
+            await emit_business_log(
+                logger,
+                event="ai.provider.analyze.failed",
+                message="AI provider analyze skipped because service URL is not configured",
+                category="ai",
+                level="warning",
+                service="ai",
+                module=__name__,
+                request_id=request_id,
+                context={**context, "status": "unconfigured"},
+            )
             raise HTTPException(
                 status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
                 detail="AI provider service URL is not configured.",
             )
 
-        data = await self._request(
-            "POST",
-            "/v1/analyze",
-            json=payload.model_dump(),
+        started_at = perf_counter()
+        await emit_business_log(
+            logger,
+            event="ai.provider.analyze.start",
+            message="AI provider analyze request started",
+            category="ai",
+            service="ai",
+            module=__name__,
             request_id=request_id,
+            context=context,
         )
-        return SituationalAnalysisResponse.model_validate(data)
+        try:
+            data = await self._request(
+                "POST",
+                "/v1/analyze",
+                json=payload.model_dump(),
+                request_id=request_id,
+                operation="analyze",
+                payload_summary=context["input_summary"],
+            )
+            result = SituationalAnalysisResponse.model_validate(data)
+            await emit_business_log(
+                logger,
+                event="ai.provider.analyze.success",
+                message="AI provider analyze request completed",
+                category="ai",
+                service="ai",
+                module=__name__,
+                request_id=request_id,
+                context={
+                    **context,
+                    "status": "success",
+                    "duration_ms": self._duration_ms(started_at),
+                    "result_provider": result.provider,
+                    "result_model": result.model,
+                    "content_block_count": len(result.content_blocks or []),
+                    "thinking_block_count": len(result.thinking_blocks or []),
+                },
+            )
+            return result
+        except Exception as exc:
+            await emit_business_log(
+                logger,
+                event="ai.provider.analyze.failed",
+                message="AI provider analyze request failed",
+                category="ai",
+                level="error",
+                service="ai",
+                module=__name__,
+                request_id=request_id,
+                context=exception_context(exc, {**context, "status": "failed", "duration_ms": self._duration_ms(started_at)}),
+            )
+            raise
 
     async def _request(
         self,
@@ -101,9 +225,12 @@ class AIProviderClient:
         path: str,
         json: dict | None = None,
         request_id: str | None = None,
+        operation: str = "request",
+        payload_summary: dict | None = None,
     ) -> dict:
         last_error: Exception | None = None
         for attempt in range(1, self.retry_attempts + 1):
+            attempt_started_at = perf_counter()
             try:
                 async with httpx.AsyncClient(timeout=self.timeout) as client:
                     response = await client.request(
@@ -117,6 +244,15 @@ class AIProviderClient:
             except httpx.HTTPStatusError as exc:
                 last_error = exc
                 if attempt < self.retry_attempts and exc.response.status_code >= 500:
+                    await self._log_retry(
+                        operation=operation,
+                        request_id=request_id,
+                        attempt=attempt,
+                        status_code=exc.response.status_code,
+                        duration_ms=self._duration_ms(attempt_started_at),
+                        error=exc,
+                        payload_summary=payload_summary,
+                    )
                     await asyncio.sleep(0.3 * attempt)
                     continue
                 detail = exc.response.text or "AI provider service returned an error"
@@ -127,6 +263,14 @@ class AIProviderClient:
             except httpx.HTTPError as exc:
                 last_error = exc
                 if attempt < self.retry_attempts:
+                    await self._log_retry(
+                        operation=operation,
+                        request_id=request_id,
+                        attempt=attempt,
+                        duration_ms=self._duration_ms(attempt_started_at),
+                        error=exc,
+                        payload_summary=payload_summary,
+                    )
                     await asyncio.sleep(0.3 * attempt)
                     continue
                 raise HTTPException(
@@ -139,6 +283,71 @@ class AIProviderClient:
             detail=f"AI provider service request failed: {last_error}",
         )
 
+    def _base_log_context(self, **extra: object) -> dict:
+        llm_provider_apis = self.llm_config.get("model_provider_apis")
+        return {
+            "provider": self.llm_config.get("provider") or "",
+            "provider_api": self.llm_config.get("provider_api") or "",
+            "model": self.llm_config.get("model") or "",
+            "base_url_configured": bool(self.llm_config.get("base_url")),
+            "service_url_configured": bool(self.service_url),
+            "timeout_seconds": self.timeout,
+            "retry_attempts": self.retry_attempts,
+            "model_provider_api_count": len(llm_provider_apis or {}) if isinstance(llm_provider_apis, dict) else 0,
+            **extra,
+        }
+
+    @staticmethod
+    def _duration_ms(started_at: float) -> int:
+        return int((perf_counter() - started_at) * 1000)
+
+    @staticmethod
+    def _summarize_analysis_payload(payload: SituationalAnalysisRequest) -> dict:
+        context = payload.context if isinstance(payload.context, dict) else {}
+        thinking = payload.thinking if isinstance(payload.thinking, dict) else payload.thinking
+        return {
+            "title_length": len(payload.title or ""),
+            "objective_length": len(payload.objective or ""),
+            "observation_count": len(payload.observations or []),
+            "constraint_count": len(payload.constraints or []),
+            "has_system_prompt": bool(payload.system_prompt),
+            "thinking_enabled": bool(thinking),
+            "context_keys": sorted(str(key) for key in context.keys()),
+        }
+
+    async def _log_retry(
+        self,
+        *,
+        operation: str,
+        request_id: str | None,
+        attempt: int,
+        duration_ms: int,
+        error: BaseException,
+        status_code: int | None = None,
+        payload_summary: dict | None = None,
+    ) -> None:
+        await emit_business_log(
+            logger,
+            event=f"ai.provider.{operation}.retry",
+            message="AI provider request will retry",
+            category="ai",
+            level="warning",
+            service="ai",
+            module=__name__,
+            request_id=request_id,
+            context=exception_context(
+                error,
+                {
+                    **self._base_log_context(operation=operation),
+                    "attempt": attempt,
+                    "next_attempt": attempt + 1,
+                    "status_code": status_code,
+                    "duration_ms": duration_ms,
+                    "input_summary": payload_summary,
+                },
+            ),
+        )
+
 
 async def get_ai_provider_client(db: AsyncSession = Depends(get_db)) -> AIProviderClient:
     from app.api.v1.settings import get_runtime_ai_provider_config

backend/app/services/ai_tools/web_fetch.py

@@ -1,11 +1,18 @@
 from __future__ import annotations
 
 import hashlib
+from time import perf_counter
+from urllib.parse import urlparse
 
 import httpx
 from bs4 import BeautifulSoup
 
+from app.core.logging import get_logger
 from app.services.ai_tools.schemas import FetchedEvidence
+from app.services.business_logs import emit_business_log, exception_context
+
+
+logger = get_logger(__name__, service="ai_tool")
 
 
 class WebFetchError(RuntimeError):
@@ -29,8 +36,33 @@ async def fetch_url_evidence(
     timeout_seconds: int = 20,
     max_bytes: int = 1_500_000,
 ) -> FetchedEvidence:
+    started_at = perf_counter()
     if not url:
+        await emit_business_log(
+            logger,
+            event="ai_tool.web_fetch.failed",
+            message="WebFetch failed because URL is empty",
+            category="ai_tool",
+            level="warning",
+            service="ai_tool",
+            module=__name__,
+            context={"reason": "empty_url"},
+        )
         raise WebFetchError("url is required")
+    request_host = urlparse(url).netloc
+    await emit_business_log(
+        logger,
+        event="ai_tool.web_fetch.start",
+        message="WebFetch request started",
+        category="ai_tool",
+        service="ai_tool",
+        module=__name__,
+        context={
+            "url_host": request_host,
+            "timeout_seconds": timeout_seconds,
+            "max_bytes": max_bytes,
+        },
+    )
     try:
         async with httpx.AsyncClient(
             timeout=timeout_seconds,
@@ -41,10 +73,45 @@ async def fetch_url_evidence(
             response.raise_for_status()
             content = response.content[:max_bytes]
     except httpx.HTTPError as exc:
+        await emit_business_log(
+            logger,
+            event="ai_tool.web_fetch.failed",
+            message="WebFetch request failed",
+            category="ai_tool",
+            level="error",
+            service="ai_tool",
+            module=__name__,
+            context=exception_context(
+                exc,
+                {
+                    "url_host": request_host,
+                    "status": "failed",
+                    "duration_ms": int((perf_counter() - started_at) * 1000),
+                },
+            ),
+        )
         raise WebFetchError(f"failed to fetch page: {exc}") from exc
 
     title, text = _extract_title_and_text(content.decode(response.encoding or "utf-8", errors="ignore"))
     content_hash = hashlib.sha256(text.encode("utf-8")).hexdigest()
+    await emit_business_log(
+        logger,
+        event="ai_tool.web_fetch.success",
+        message="WebFetch request completed",
+        category="ai_tool",
+        service="ai_tool",
+        module=__name__,
+        context={
+            "url_host": request_host,
+            "final_url_host": urlparse(str(response.url)).netloc,
+            "status": "success",
+            "status_code": response.status_code,
+            "bytes_read": len(content),
+            "content_hash": content_hash,
+            "duration_ms": int((perf_counter() - started_at) * 1000),
+            "extractor": "beautifulsoup_basic",
+        },
+    )
     return FetchedEvidence(
         url=url,
         final_url=str(response.url),
@@ -53,4 +120,3 @@ async def fetch_url_evidence(
         content_hash=content_hash,
         extractor="beautifulsoup_basic",
     )
-

backend/app/services/ai_tools/web_search.py

@@ -1,11 +1,18 @@
 from __future__ import annotations
 
 from copy import deepcopy
+import hashlib
+from time import perf_counter
 from typing import Any
 
 import httpx
 
+from app.core.logging import get_logger
 from app.services.ai_tools.schemas import SearchEvidence, WebSearchConfig, WebSearchProviderConfig
+from app.services.business_logs import emit_business_log, exception_context
+
+
+logger = get_logger(__name__, service="ai_tool")
 
 
 WEB_SEARCH_PROVIDER_PRESETS: dict[str, dict[str, Any]] = {
@@ -120,29 +127,108 @@ class WebSearchClient:
         domains: list[str] | None = None,
         freshness_days: int | None = None,
     ) -> list[SearchEvidence]:
+        started_at = perf_counter()
         if not self.config.enabled:
+            await emit_business_log(
+                logger,
+                event="ai_tool.web_search.unavailable",
+                message="WebSearch skipped because integration is disabled",
+                category="ai_tool",
+                level="warning",
+                service="ai_tool",
+                module=__name__,
+                context={"provider": self.config.default_provider, "reason": "disabled"},
+            )
             raise WebSearchConfigurationError("WebSearch is disabled.")
         provider_config = self.config.active_provider_config
         provider = normalize_web_search_provider(provider_config.provider)
         if provider != "searxng" and not provider_config.api_key:
+            await emit_business_log(
+                logger,
+                event="ai_tool.web_search.unavailable",
+                message="WebSearch skipped because API key is not configured",
+                category="ai_tool",
+                level="warning",
+                service="ai_tool",
+                module=__name__,
+                context={"provider": provider, "reason": "missing_api_key"},
+            )
             raise WebSearchConfigurationError(f"{provider} API key is not configured.")
         query = " ".join(str(query or "").split())
         if not query:
+            await emit_business_log(
+                logger,
+                event="ai_tool.web_search.failed",
+                message="WebSearch failed because query is empty",
+                category="ai_tool",
+                level="warning",
+                service="ai_tool",
+                module=__name__,
+                context={"provider": provider, "reason": "empty_query"},
+            )
             raise WebSearchConfigurationError("search query is required.")
         limit = max_results or provider_config.max_results
-        if provider == "tavily":
-            return await self._search_tavily(provider_config, query, limit, domains, freshness_days)
-        if provider == "brave":
-            return await self._search_brave(provider_config, query, limit, domains)
-        if provider == "serpapi":
-            return await self._search_serpapi(provider_config, query, limit)
-        if provider == "exa":
-            return await self._search_exa(provider_config, query, limit, domains)
-        if provider == "firecrawl":
-            return await self._search_firecrawl(provider_config, query, limit)
-        if provider == "searxng":
-            return await self._search_searxng(provider_config, query, limit, domains)
-        raise WebSearchConfigurationError(f"Unsupported web search provider: {provider}")
+        context = {
+            "provider": provider,
+            "query_hash": hashlib.sha256(query.encode("utf-8")).hexdigest(),
+            "query_length": len(query),
+            "max_results": limit,
+            "domain_count": len(domains or []),
+            "freshness_days": freshness_days,
+        }
+        await emit_business_log(
+            logger,
+            event="ai_tool.web_search.start",
+            message="WebSearch request started",
+            category="ai_tool",
+            service="ai_tool",
+            module=__name__,
+            context=context,
+        )
+        try:
+            if provider == "tavily":
+                results = await self._search_tavily(provider_config, query, limit, domains, freshness_days)
+            elif provider == "brave":
+                results = await self._search_brave(provider_config, query, limit, domains)
+            elif provider == "serpapi":
+                results = await self._search_serpapi(provider_config, query, limit)
+            elif provider == "exa":
+                results = await self._search_exa(provider_config, query, limit, domains)
+            elif provider == "firecrawl":
+                results = await self._search_firecrawl(provider_config, query, limit)
+            elif provider == "searxng":
+                results = await self._search_searxng(provider_config, query, limit, domains)
+            else:
+                raise WebSearchConfigurationError(f"Unsupported web search provider: {provider}")
+            event = "ai_tool.web_search.success" if results else "ai_tool.web_search.empty"
+            await emit_business_log(
+                logger,
+                event=event,
+                message="WebSearch request completed" if results else "WebSearch returned no results",
+                category="ai_tool",
+                level="info" if results else "warning",
+                service="ai_tool",
+                module=__name__,
+                context={
+                    **context,
+                    "status": "success" if results else "empty",
+                    "result_count": len(results),
+                    "duration_ms": int((perf_counter() - started_at) * 1000),
+                },
+            )
+            return results
+        except Exception as exc:
+            await emit_business_log(
+                logger,
+                event="ai_tool.web_search.failed",
+                message="WebSearch request failed",
+                category="ai_tool",
+                level="error",
+                service="ai_tool",
+                module=__name__,
+                context=exception_context(exc, {**context, "status": "failed", "duration_ms": int((perf_counter() - started_at) * 1000)}),
+            )
+            raise
 
     async def test_connection(self) -> list[SearchEvidence]:
         return await self.search("Planet WebSearch connectivity test", max_results=1)
@@ -388,4 +474,3 @@ def _float_or_none(value: Any) -> float | None:
         return float(value)
     except (TypeError, ValueError):
         return None
-

backend/app/services/business_logs.py

@@ -0,0 +1,117 @@
+from __future__ import annotations
+
+import asyncio
+from collections.abc import Mapping
+from typing import Any
+
+from app.core.logging import PlanetLoggerAdapter, sanitize_log_value
+from app.core.request_context import get_request_id
+from app.services.persistent_logs import record_system_log
+
+
+LEVEL_METHODS = {
+    "debug": "debug_event",
+    "info": "info_event",
+    "warning": "warning_event",
+    "error": "error_event",
+}
+
+
+def normalize_business_level(level: str | None) -> str:
+    normalized = str(level or "info").strip().lower()
+    if normalized in {"warn", "warning"}:
+        return "warning"
+    if normalized in {"err", "error", "critical", "fatal"}:
+        return "error"
+    if normalized == "debug":
+        return "debug"
+    return "info"
+
+
+def build_business_context(
+    context: Mapping[str, Any] | None = None,
+    **fields: Any,
+) -> dict[str, Any]:
+    payload = dict(context or {})
+    for key, value in fields.items():
+        if value is not None:
+            payload[key] = value
+    return sanitize_log_value(payload)
+
+
+async def emit_business_log(
+    logger: PlanetLoggerAdapter,
+    *,
+    event: str,
+    message: str,
+    category: str,
+    level: str = "info",
+    source: str = "backend",
+    service: str | None = None,
+    module: str | None = None,
+    request_id: str | None = None,
+    user_id: int | None = None,
+    context: Mapping[str, Any] | None = None,
+) -> None:
+    normalized_level = normalize_business_level(level)
+    safe_context = build_business_context(context)
+    log_method = getattr(logger, LEVEL_METHODS[normalized_level])
+    log_method(message, event=event, context=safe_context)
+    await record_system_log(
+        source=source,
+        level=normalized_level,
+        message=message,
+        service=service,
+        module=module,
+        event=event,
+        request_id=request_id or get_request_id(),
+        user_id=user_id,
+        category=category,
+        context=safe_context,
+    )
+
+
+def emit_business_log_background(
+    logger: PlanetLoggerAdapter,
+    *,
+    event: str,
+    message: str,
+    category: str,
+    level: str = "info",
+    source: str = "backend",
+    service: str | None = None,
+    module: str | None = None,
+    request_id: str | None = None,
+    user_id: int | None = None,
+    context: Mapping[str, Any] | None = None,
+) -> None:
+    normalized_level = normalize_business_level(level)
+    safe_context = build_business_context(context)
+    log_method = getattr(logger, LEVEL_METHODS[normalized_level])
+    log_method(message, event=event, context=safe_context)
+    try:
+        loop = asyncio.get_running_loop()
+    except RuntimeError:
+        return
+    loop.create_task(
+        record_system_log(
+            source=source,
+            level=normalized_level,
+            message=message,
+            service=service,
+            module=module,
+            event=event,
+            request_id=request_id or get_request_id(),
+            user_id=user_id,
+            category=category,
+            context=safe_context,
+        )
+    )
+
+
+def exception_context(exc: BaseException, context: Mapping[str, Any] | None = None) -> dict[str, Any]:
+    return build_business_context(
+        context,
+        error_type=type(exc).__name__,
+        error=str(exc),
+    )

backend/app/services/collectors/aisstream.py

@@ -322,6 +322,21 @@ class AISStreamCollector(BaseCollector):
             last_success_at=now if data else None,
             lag_seconds=max((now - latest_observed_at).total_seconds(), 0),
         )
+        if snapshot_id is not None:
+            from app.models.data_snapshot import DataSnapshot
+
+            snapshot = await db.get(DataSnapshot, snapshot_id)
+            if snapshot:
+                snapshot.record_count = records_added
+                snapshot.status = "success"
+                snapshot.completed_at = now
+                snapshot.summary = {
+                    "created": records_added,
+                    "updated": 0,
+                    "unchanged": 0,
+                    "deleted": 0,
+                    "storage": "ais_raw_observations",
+                }
         await db.commit()
         await self.update_progress(records_added, force=True)
         return records_added

backend/app/services/collectors/base.py

@@ -4,42 +4,22 @@ import asyncio
 from abc import ABC, abstractmethod
 from typing import Dict, List, Any, Optional
 from datetime import UTC, datetime
+from time import perf_counter
+from urllib.parse import urlparse
 import httpx
 from sqlalchemy import select, text
 from sqlalchemy.ext.asyncio import AsyncSession
 
 from app.core.collected_data_fields import build_dynamic_metadata, get_record_field
 from app.core.countries import normalize_country
+from app.core.logging import get_logger
 from app.core.time import to_iso8601_utc
 from app.core.websocket.broadcaster import broadcaster
-from app.services.earth_layer_cache import invalidate_earth_layer_cache_for_source
-
-EARTH_UPDATE_LAYER_HINTS: dict[str, list[str]] = {
-    "ris_live_bgp": ["bgp"],
-    "bgpstream_bgp": ["bgp"],
-    "top500_supercomputers": ["computeCenters"],
-    "epoch_ai_gpu": ["computeCenters"],
-    "huggingface_models": ["computeCenters"],
-    "huggingface_datasets": ["computeCenters"],
-    "huggingface_spaces": ["computeCenters"],
-    "telegeography_cables": ["cables"],
-    "telegeography_landing_points": ["cables"],
-    "telegeography_cable_systems": ["cables"],
-    "arcgis_cables": ["cables"],
-    "fao_landing_points": ["cables"],
-    "arcgis_landing_points": ["cables"],
-    "arcgis_cable_landing_relations": ["cables"],
-    "spacetrack_tle": ["satellites"],
-    "celestrak_tle": ["satellites"],
-    "barentswatch_vessels": ["vessels"],
-    "aisstream_vessels": ["vessels"],
-    "news_live_streams": ["media"],
-    "media_news_archive": ["news"],
-}
+from app.services.business_logs import emit_business_log, exception_context
+from app.services.earth_layer_adapters import get_earth_update_layers_for_source
 
 
-def get_earth_update_layers_for_source(source: str) -> list[str]:
-    return EARTH_UPDATE_LAYER_HINTS.get(source, [])
+logger = get_logger(__name__, service="collector")
 
 
 class BaseCollector(ABC):
@@ -58,6 +38,7 @@ class BaseCollector(ABC):
         self._datasource_id = 1
         self._resolved_url: Optional[str] = None
         self._last_broadcast_progress: Optional[int] = None
+        self._last_save_summary: dict[str, int] = {}
 
     async def resolve_url(self, db: AsyncSession) -> None:
         from app.core.data_sources import get_data_sources_config
@@ -96,29 +77,6 @@ class BaseCollector(ABC):
         )
         self._last_broadcast_progress = rounded_progress
 
-    async def _publish_earth_update(
-        self,
-        *,
-        action: str,
-        records_processed: int,
-        task_id: int | None = None,
-    ) -> None:
-        layers = get_earth_update_layers_for_source(self.name)
-        if not layers:
-            return
-        await broadcaster.broadcast_earth_update(
-            {
-                "action": action,
-                "source": self.name,
-                "data_type": self.data_type,
-                "layers": layers,
-                "datasource_id": getattr(self, "_datasource_id", None),
-                "task_id": task_id,
-                "records_processed": records_processed,
-                "timestamp": to_iso8601_utc(datetime.now(UTC)),
-            }
-        )
-
     async def update_progress(self, records_processed: int, *, commit: bool = False, force: bool = False):
         """Update task progress - call this during data processing"""
         if self._current_task and self._db_session:
@@ -322,19 +280,39 @@ class BaseCollector(ABC):
         from app.models.data_snapshot import DataSnapshot
 
         start_time = datetime.now(UTC)
+        started_at = perf_counter()
         datasource_id = getattr(self, "_datasource_id", 1)
         snapshot_id: Optional[int] = None
 
         if not collector_registry.is_active(self.name):
+            await self._log_collection_event(
+                "collector.run.skipped_disabled",
+                "Collector skipped because it is disabled",
+                level="info",
+                context={"status": "skipped", "reason": "disabled"},
+            )
             return {"status": "skipped", "reason": "Collector is disabled"}
 
-        task = CollectionTask(
-            datasource_id=datasource_id,
-            status="running",
-            phase="queued",
-            started_at=start_time,
-        )
-        db.add(task)
+        task = self._current_task if isinstance(self._current_task, CollectionTask) else None
+        if task is None:
+            task = CollectionTask(
+                datasource_id=datasource_id,
+                source=self.name,
+                task_type="collect",
+                status="running",
+                phase="queued",
+                started_at=start_time,
+            )
+            db.add(task)
+        else:
+            task.datasource_id = datasource_id
+            task.source = task.source or self.name
+            task.task_type = task.task_type or "collect"
+            task.status = "running"
+            task.phase = "queued"
+            task.started_at = task.started_at or start_time
+            task.completed_at = None
+            task.error_message = None
         await db.commit()
         task_id = task.id
 
@@ -344,23 +322,76 @@ class BaseCollector(ABC):
 
         await self.resolve_url(db)
         await self._publish_task_update(force=True)
+        await self._log_collection_event(
+            "collector.run.started",
+            "Collector run started",
+            context={"status": "running", "task_id": task_id},
+        )
 
         try:
+            phase_started_at = perf_counter()
             await self.set_phase("fetching", message="正在拉取原始数据")
+            await self._log_collection_event(
+                "collector.phase.fetching.start",
+                "Collector fetch phase started",
+                context={"task_id": task_id, "snapshot_id": snapshot_id},
+            )
             raw_data = await self.fetch()
             task.total_records = len(raw_data)
             await db.commit()
             await self._publish_task_update(force=True)
+            await self._log_collection_event(
+                "collector.phase.fetching.success",
+                "Collector fetch phase completed",
+                context={
+                    "task_id": task_id,
+                    "raw_count": len(raw_data),
+                    "duration_ms": self._duration_ms(phase_started_at),
+                },
+            )
 
             if self.fail_on_empty and not raw_data:
                 raise RuntimeError(f"Collector {self.name} returned no data")
 
+            phase_started_at = perf_counter()
             await self.set_phase("transforming", message="正在转换采集数据")
+            await self._log_collection_event(
+                "collector.phase.transforming.start",
+                "Collector transform phase started",
+                context={"task_id": task_id, "raw_count": len(raw_data)},
+            )
             data = self.transform(raw_data)
+            await self._log_collection_event(
+                "collector.phase.transforming.success",
+                "Collector transform phase completed",
+                context={
+                    "task_id": task_id,
+                    "raw_count": len(raw_data),
+                    "transformed_count": len(data),
+                    "duration_ms": self._duration_ms(phase_started_at),
+                },
+            )
             snapshot_id = await self._create_snapshot(db, task_id, data, start_time)
 
+            phase_started_at = perf_counter()
             await self.set_phase("saving", message="正在保存采集数据")
+            await self._log_collection_event(
+                "collector.phase.saving.start",
+                "Collector save phase started",
+                context={"task_id": task_id, "snapshot_id": snapshot_id, "transformed_count": len(data)},
+            )
             records_count = await self._save_data(db, data, task_id=task_id, snapshot_id=snapshot_id)
+            await self._log_collection_event(
+                "collector.phase.saving.success",
+                "Collector save phase completed",
+                context={
+                    "task_id": task_id,
+                    "snapshot_id": snapshot_id,
+                    "saved_count": records_count,
+                    **self._last_save_summary,
+                    "duration_ms": self._duration_ms(phase_started_at),
+                },
+            )
 
             task.status = "success"
             task.phase = "completed"
@@ -374,10 +405,19 @@ class BaseCollector(ABC):
             task.completed_at = datetime.now(UTC)
             await db.commit()
             await self._publish_task_update(force=True)
-            await self._publish_earth_update(
-                action="collector_completed",
-                records_processed=records_count,
-                task_id=task_id,
+            await self._log_collection_event(
+                "collector.run.completed",
+                "Collector run completed",
+                context={
+                    "status": "success",
+                    "task_id": task_id,
+                    "snapshot_id": snapshot_id,
+                    "raw_count": len(raw_data),
+                    "transformed_count": len(data),
+                    "saved_count": records_count,
+                    **self._last_save_summary,
+                    "duration_ms": self._duration_ms(started_at),
+                },
             )
 
             return {
@@ -402,6 +442,17 @@ class BaseCollector(ABC):
                 )
             await db.commit()
             await self._publish_task_update(force=True)
+            await self._log_collection_event(
+                "collector.run.cancelled",
+                "Collector run cancelled",
+                level="warning",
+                context={
+                    "status": "cancelled",
+                    "task_id": task_id,
+                    "snapshot_id": snapshot_id,
+                    "duration_ms": self._duration_ms(started_at),
+                },
+            )
             raise
         except Exception as e:
             await db.rollback()
@@ -418,6 +469,20 @@ class BaseCollector(ABC):
                     snapshot.summary = {"error": str(e)}
             await db.commit()
             await self._publish_task_update(force=True)
+            await self._log_collection_event(
+                "collector.run.failed",
+                "Collector run failed",
+                level="error",
+                context=exception_context(
+                    e,
+                    {
+                        "status": "failed",
+                        "task_id": task_id,
+                        "snapshot_id": snapshot_id,
+                        "duration_ms": self._duration_ms(started_at),
+                    },
+                ),
+            )
 
             return {
                 "status": "failed",
@@ -438,6 +503,7 @@ class BaseCollector(ABC):
         from app.models.data_snapshot import DataSnapshot
 
         if not data:
+            self._last_save_summary = {"created": 0, "updated": 0, "unchanged": 0, "deleted": 0}
             if snapshot_id is not None:
                 snapshot = await db.get(DataSnapshot, snapshot_id)
                 if snapshot:
@@ -584,12 +650,51 @@ class BaseCollector(ABC):
                     "unchanged": unchanged_count,
                     "deleted": len(deleted_keys),
                 }
+            self._last_save_summary = {
+                "created": created_count,
+                "updated": updated_count,
+                "unchanged": unchanged_count,
+                "deleted": len(deleted_keys),
+            }
+        else:
+            self._last_save_summary = {
+                "created": created_count,
+                "updated": updated_count,
+                "unchanged": unchanged_count,
+                "deleted": 0,
+            }
 
         await db.commit()
-        invalidate_earth_layer_cache_for_source(self.name)
         await self.update_progress(len(data), force=True)
         return records_added
 
+    @staticmethod
+    def _duration_ms(started_at: float) -> int:
+        return int((perf_counter() - started_at) * 1000)
+
+    async def _log_collection_event(
+        self,
+        event: str,
+        message: str,
+        *,
+        level: str = "info",
+        context: Dict[str, Any] | None = None,
+    ) -> None:
+        await emit_business_log(
+            logger,
+            event=event,
+            message=message,
+            category="collector",
+            level=level,
+            service="collector",
+            module=__name__,
+            context={
+                "collector_name": self.name,
+                "datasource_id": getattr(self, "_datasource_id", None),
+                **(context or {}),
+            },
+        )
+
     async def save(self, db: AsyncSession, data: List[Dict[str, Any]]) -> int:
         """Save data to database (legacy method, use _save_data instead)"""
         return await self._save_data(db, data)
@@ -602,10 +707,65 @@ class HTTPCollector(BaseCollector):
     headers: Dict[str, str] = {}
 
     async def fetch(self) -> List[Dict[str, Any]]:
+        started_at = perf_counter()
+        request_host = urlparse(self.base_url).netloc
+        await emit_business_log(
+            logger,
+            event="collector.http.fetch.start",
+            message="Collector HTTP request started",
+            category="collector",
+            service="collector",
+            module=__name__,
+            context={
+                "collector_name": self.name,
+                "datasource_id": getattr(self, "_datasource_id", None),
+                "url_host": request_host,
+            },
+        )
         async with httpx.AsyncClient(timeout=60.0) as client:
-            response = await client.get(self.base_url, headers=self.headers)
-            response.raise_for_status()
-            return self.parse_response(response.json())
+            try:
+                response = await client.get(self.base_url, headers=self.headers)
+                response.raise_for_status()
+                payload = response.json()
+                parsed = self.parse_response(payload)
+                await emit_business_log(
+                    logger,
+                    event="collector.http.fetch.success",
+                    message="Collector HTTP request completed",
+                    category="collector",
+                    service="collector",
+                    module=__name__,
+                    context={
+                        "collector_name": self.name,
+                        "datasource_id": getattr(self, "_datasource_id", None),
+                        "url_host": request_host,
+                        "status_code": response.status_code,
+                        "response_bytes": len(response.content or b""),
+                        "parsed_count": len(parsed),
+                        "duration_ms": BaseCollector._duration_ms(started_at),
+                    },
+                )
+                return parsed
+            except Exception as exc:
+                await emit_business_log(
+                    logger,
+                    event="collector.http.fetch.failed",
+                    message="Collector HTTP request failed",
+                    category="collector",
+                    level="error",
+                    service="collector",
+                    module=__name__,
+                    context=exception_context(
+                        exc,
+                        {
+                            "collector_name": self.name,
+                            "datasource_id": getattr(self, "_datasource_id", None),
+                            "url_host": request_host,
+                            "duration_ms": BaseCollector._duration_ms(started_at),
+                        },
+                    ),
+                )
+                raise
 
     @abstractmethod
     def parse_response(self, response: Dict[str, Any]) -> List[Dict[str, Any]]:

backend/app/services/collectors/celestrak.py

@@ -1,15 +1,39 @@
-"""CelesTrak TLE Collector
+"""CelesTrak TLE Collector.
 
-Collects satellite TLE (Two-Line Element) data from CelesTrak.org.
-Free, no authentication required.
+Collects the full active satellite GP element set from CelesTrak.
 """
 
+import asyncio
 import json
-from typing import Dict, Any, List
+from pathlib import Path
+from time import perf_counter
+from typing import Any, Dict, List
+from urllib.parse import urlencode, urlparse
+
 import httpx
 
+from app.core.logging import get_logger
 from app.core.satellite_tle import build_tle_lines_from_elements
+from app.services.business_logs import emit_business_log, exception_context
 from app.services.collectors.base import BaseCollector
+from app.services.collectors.downloads import DownloadHTTPStatusError, ResumableFileDownloader
+
+
+logger = get_logger(__name__, service="collector")
+ACTIVE_GROUP = "active"
+FALLBACK_GROUPS = (
+    "starlink",
+    "gps-ops",
+    "galileo",
+    "glonass",
+    "beidou",
+    "leo",
+    "geo",
+    "iridium-next",
+)
+FETCH_RETRY_ATTEMPTS = 3
+FETCH_RETRY_BASE_DELAY_SECONDS = 0.8
+CELESTRAK_NOT_UPDATED_MARKER = "GP data has not updated since your last successful"
 
 
 class CelesTrakTLECollector(BaseCollector):
@@ -18,55 +42,359 @@ class CelesTrakTLECollector(BaseCollector):
     module = "L3"
     frequency_hours = 24
     data_type = "satellite_tle"
+    _downloader = ResumableFileDownloader(
+        cache_namespace="celestrak",
+        default_accept="application/json",
+    )
 
     @property
     def base_url(self) -> str:
         return self._resolved_url or ""
 
+    def _active_url(self) -> str:
+        return self._group_url(ACTIVE_GROUP)
+
+    def _group_url(self, group: str) -> str:
+        if not self.base_url:
+            raise RuntimeError("CelesTrak base URL is not configured")
+        return f"{self.base_url}?{urlencode({'GROUP': group, 'FORMAT': 'json'})}"
+
     async def fetch(self) -> List[Dict[str, Any]]:
-        satellite_groups = [
-            "starlink",
-            "gps-ops",
-            "galileo",
-            "glonass",
-            "beidou",
-            "leo",
-            "geo",
-            "iridium-next",
-        ]
+        url = self._active_url()
+        last_error: Exception | None = None
 
-        all_satellites = []
-
-        async with httpx.AsyncClient(timeout=120.0) as client:
-            for group in satellite_groups:
+        async with httpx.AsyncClient(timeout=180.0, follow_redirects=True) as client:
+            for attempt in range(1, FETCH_RETRY_ATTEMPTS + 1):
+                started_at = perf_counter()
                 try:
-                    url = f"{self.base_url}?GROUP={group}&FORMAT=json"
-                    response = await client.get(url)
+                    await emit_business_log(
+                        logger,
+                        event="collector.celestrak.download.start",
+                        message="CelesTrak active satellite download started",
+                        category="collector",
+                        service="collector",
+                        module=__name__,
+                        context={
+                            "collector_name": self.name,
+                            "datasource_id": getattr(self, "_datasource_id", None),
+                            "group": ACTIVE_GROUP,
+                            "attempt": attempt,
+                            "url_host": urlparse(url).netloc,
+                        },
+                    )
+                    body_path = await self._downloader.download_file(
+                        client,
+                        url,
+                        extension=".json",
+                        accept="application/json",
+                        progress_callback=self._report_download_progress,
+                        validate_existing=self._validate_json_file,
+                    )
+                    data = await self._load_downloaded_payload(body_path, url)
+                    await emit_business_log(
+                        logger,
+                        event="collector.celestrak.download.success",
+                        message="CelesTrak active satellite download completed",
+                        category="collector",
+                        service="collector",
+                        module=__name__,
+                        context={
+                            "collector_name": self.name,
+                            "datasource_id": getattr(self, "_datasource_id", None),
+                            "group": ACTIVE_GROUP,
+                            "attempt": attempt,
+                            "record_count": len(data),
+                            "duration_ms": self._duration_ms(started_at),
+                        },
+                    )
+                    return data
+                except DownloadHTTPStatusError as exc:
+                    if self._is_not_updated_response(exc):
+                        cached_path = self._downloader.get_cached_file(
+                            url,
+                            ".json",
+                            validate_existing=self._validate_json_file,
+                        )
+                        if cached_path is not None:
+                            data = await self._load_downloaded_payload(cached_path, url)
+                            await emit_business_log(
+                                logger,
+                                event="collector.celestrak.download.cached_not_updated",
+                                message="CelesTrak active satellite data has not changed; using cached download",
+                                category="collector",
+                                level="warning",
+                                service="collector",
+                                module=__name__,
+                                context={
+                                    "collector_name": self.name,
+                                    "datasource_id": getattr(self, "_datasource_id", None),
+                                    "group": ACTIVE_GROUP,
+                                    "attempt": attempt,
+                                    "record_count": len(data),
+                                    "duration_ms": self._duration_ms(started_at),
+                                },
+                            )
+                            return data
+                        await emit_business_log(
+                            logger,
+                            event="collector.celestrak.download.not_updated_no_cache",
+                            message="CelesTrak active satellite data has not changed; trying fallback groups",
+                            category="collector",
+                            level="warning",
+                            service="collector",
+                            module=__name__,
+                            context=exception_context(
+                                exc,
+                                {
+                                    "collector_name": self.name,
+                                    "datasource_id": getattr(self, "_datasource_id", None),
+                                    "group": ACTIVE_GROUP,
+                                    "attempt": attempt,
+                                    "duration_ms": self._duration_ms(started_at),
+                                },
+                            ),
+                        )
+                        return await self._fetch_fallback_groups(client, active_error=exc)
+                    raise
+                except Exception as exc:
+                    last_error = exc
+                    is_final_attempt = attempt >= FETCH_RETRY_ATTEMPTS
+                    await emit_business_log(
+                        logger,
+                        event=(
+                            "collector.celestrak.download.failed"
+                            if is_final_attempt
+                            else "collector.celestrak.download.retry"
+                        ),
+                        message=(
+                            "CelesTrak active satellite download failed"
+                            if is_final_attempt
+                            else "CelesTrak active satellite download will retry"
+                        ),
+                        category="collector",
+                        level="error" if is_final_attempt else "warning",
+                        service="collector",
+                        module=__name__,
+                        context=exception_context(
+                            exc,
+                            {
+                                "collector_name": self.name,
+                                "datasource_id": getattr(self, "_datasource_id", None),
+                                "group": ACTIVE_GROUP,
+                                "attempt": attempt,
+                                "duration_ms": self._duration_ms(started_at),
+                            },
+                        ),
+                    )
+                    if not is_final_attempt:
+                        await asyncio.sleep(FETCH_RETRY_BASE_DELAY_SECONDS * attempt)
 
-                    if response.status_code == 200:
-                        data = response.json()
-                        if isinstance(data, list):
-                            for item in data:
-                                if isinstance(item, dict):
-                                    item["_celestrak_group"] = group
-                            all_satellites.extend(data)
-                            print(f"CelesTrak: Fetched {len(data)} satellites from group '{group}'")
-                except Exception as e:
-                    print(f"CelesTrak: Error fetching group '{group}': {e}")
+        raise RuntimeError(f"CelesTrak active satellite download failed after retries: {last_error}")
 
-        if not all_satellites:
-            return self._get_sample_data()
+    async def _fetch_fallback_groups(
+        self,
+        client: httpx.AsyncClient,
+        *,
+        active_error: DownloadHTTPStatusError,
+    ) -> List[Dict[str, Any]]:
+        started_at = perf_counter()
+        records_by_norad: dict[str, Dict[str, Any]] = {}
+        group_counts: dict[str, int] = {}
 
-        print(f"CelesTrak: Total satellites fetched: {len(all_satellites)}")
+        await emit_business_log(
+            logger,
+            event="collector.celestrak.fallback_groups.start",
+            message="CelesTrak fallback group download started",
+            category="collector",
+            level="warning",
+            service="collector",
+            module=__name__,
+            context={
+                "collector_name": self.name,
+                "datasource_id": getattr(self, "_datasource_id", None),
+                "groups": list(FALLBACK_GROUPS),
+                "reason": "active_not_updated_without_cache",
+            },
+        )
 
-        # Return raw data - base.run() will call transform()
-        return all_satellites
+        try:
+            for group in FALLBACK_GROUPS:
+                group_url = self._group_url(group)
+                try:
+                    body_path = await self._downloader.download_file(
+                        client,
+                        group_url,
+                        extension=".json",
+                        accept="application/json",
+                        validate_existing=self._validate_json_file,
+                    )
+                except DownloadHTTPStatusError as exc:
+                    if not self._is_not_updated_response(exc):
+                        raise RuntimeError(f"CelesTrak fallback group '{group}' download failed: {exc}") from exc
+                    cached_path = self._downloader.get_cached_file(
+                        group_url,
+                        ".json",
+                        validate_existing=self._validate_json_file,
+                    )
+                    if cached_path is None:
+                        raise RuntimeError(
+                            f"CelesTrak fallback group '{group}' has not updated and no local cached copy is available"
+                        ) from exc
+                    body_path = cached_path
+
+                group_records = await self._load_downloaded_payload(
+                    body_path,
+                    group_url,
+                    query_group=group,
+                    constellation_group=group,
+                )
+                group_counts[group] = len(group_records)
+                for item in group_records:
+                    norad_cat_id = item.get("NORAD_CAT_ID")
+                    if norad_cat_id is None:
+                        continue
+                    records_by_norad.setdefault(str(norad_cat_id), item)
+        except Exception as exc:
+            await emit_business_log(
+                logger,
+                event="collector.celestrak.fallback_groups.failed",
+                message="CelesTrak fallback group download failed",
+                category="collector",
+                level="error",
+                service="collector",
+                module=__name__,
+                context=exception_context(
+                    exc,
+                    {
+                        "collector_name": self.name,
+                        "datasource_id": getattr(self, "_datasource_id", None),
+                        "groups": list(FALLBACK_GROUPS),
+                        "completed_groups": list(group_counts),
+                        "duration_ms": self._duration_ms(started_at),
+                    },
+                ),
+            )
+            raise RuntimeError(
+                "CelesTrak active data has not updated since this network's last successful download, "
+                "no active cache is available, and fallback group mode failed. Wait until CelesTrak "
+                "publishes the next GP update, restore the Planet download cache, or use Space-Track."
+            ) from active_error
+
+        records = list(records_by_norad.values())
+        if not records:
+            raise RuntimeError("CelesTrak fallback group mode produced no satellite records")
+
+        await emit_business_log(
+            logger,
+            event="collector.celestrak.fallback_groups.success",
+            message="CelesTrak fallback group download completed",
+            category="collector",
+            level="warning",
+            service="collector",
+            module=__name__,
+            context={
+                "collector_name": self.name,
+                "datasource_id": getattr(self, "_datasource_id", None),
+                "groups": list(FALLBACK_GROUPS),
+                "group_counts": group_counts,
+                "record_count": len(records),
+                "duration_ms": self._duration_ms(started_at),
+            },
+        )
+        return records
+
+    async def _load_downloaded_payload(
+        self,
+        body_path: Path,
+        url: str,
+        *,
+        query_group: str = ACTIVE_GROUP,
+        constellation_group: str | None = None,
+    ) -> List[Dict[str, Any]]:
+        try:
+            data = self._load_active_payload(body_path)
+        except RuntimeError as exc:
+            await self._log_parse_failure(exc)
+            raise
+        for item in data:
+            item["_celestrak_query_group"] = query_group
+            item["_celestrak_source_url"] = url
+            if constellation_group:
+                item["_celestrak_group"] = constellation_group
+        return data
+
+    @staticmethod
+    def _is_not_updated_response(exc: DownloadHTTPStatusError) -> bool:
+        return exc.status_code == 403 and CELESTRAK_NOT_UPDATED_MARKER in exc.body
+
+    @staticmethod
+    def _duration_ms(started_at: float) -> int:
+        return int((perf_counter() - started_at) * 1000)
+
+    async def _report_download_progress(self, downloaded: int, total: int | None) -> None:
+        if total and total > 0:
+            await self.update_phase_progress(
+                current=min(downloaded, total),
+                total=total,
+                unit="bytes",
+                message=f"正在下载 CelesTrak active 卫星数据 {downloaded}/{total} bytes",
+                commit=True,
+            )
+
+    @staticmethod
+    def _validate_json_file(path: Path) -> bool:
+        try:
+            data = json.loads(path.read_text(encoding="utf-8"))
+        except (OSError, json.JSONDecodeError, UnicodeDecodeError):
+            return False
+        return isinstance(data, list)
+
+    async def _log_parse_failure(self, exc: Exception) -> None:
+        await emit_business_log(
+            logger,
+            event="collector.celestrak.parse.failed",
+            message="CelesTrak active satellite JSON parsing failed",
+            category="collector",
+            level="error",
+            service="collector",
+            module=__name__,
+            context=exception_context(
+                exc,
+                {
+                    "collector_name": self.name,
+                    "datasource_id": getattr(self, "_datasource_id", None),
+                    "group": ACTIVE_GROUP,
+                },
+            ),
+        )
+
+    def _load_active_payload(self, path: Path) -> List[Dict[str, Any]]:
+        try:
+            raw = json.loads(path.read_text(encoding="utf-8"))
+        except (OSError, json.JSONDecodeError, UnicodeDecodeError) as exc:
+            raise RuntimeError(f"CelesTrak active payload is not valid JSON: {exc}") from exc
+        if not isinstance(raw, list):
+            raise RuntimeError("CelesTrak active payload is not a JSON array")
+
+        records: List[Dict[str, Any]] = []
+        invalid_count = 0
+        for item in raw:
+            if isinstance(item, dict) and item.get("NORAD_CAT_ID") is not None:
+                records.append(item)
+            else:
+                invalid_count += 1
+        if invalid_count:
+            raise RuntimeError(f"CelesTrak active payload contains {invalid_count} invalid record(s)")
+        if not records:
+            raise RuntimeError("CelesTrak active payload contains no satellite records")
+        return records
 
     def transform(self, raw_data: List[Dict[str, Any]]) -> List[Dict[str, Any]]:
         transformed = []
         for item in raw_data:
+            norad_cat_id = item.get("NORAD_CAT_ID")
             tle_line1, tle_line2 = build_tle_lines_from_elements(
-                norad_cat_id=item.get("NORAD_CAT_ID"),
+                norad_cat_id=norad_cat_id,
                 epoch=item.get("EPOCH"),
                 inclination=item.get("INCLINATION"),
                 raan=item.get("RA_OF_ASC_NODE"),
@@ -75,14 +403,18 @@ class CelesTrakTLECollector(BaseCollector):
                 mean_anomaly=item.get("MEAN_ANOMALY"),
                 mean_motion=item.get("MEAN_MOTION"),
             )
+            constellation_group = self._infer_constellation_group(item)
 
             transformed.append(
                 {
+                    "source_id": str(norad_cat_id),
                     "name": item.get("OBJECT_NAME", "Unknown"),
                     "reference_date": item.get("EPOCH", ""),
                     "metadata": {
-                        "constellation_group": item.get("_celestrak_group"),
-                        "norad_cat_id": item.get("NORAD_CAT_ID"),
+                        "constellation_group": constellation_group,
+                        "celestrak_query_group": item.get("_celestrak_query_group") or ACTIVE_GROUP,
+                        "celestrak_source_url": item.get("_celestrak_source_url"),
+                        "norad_cat_id": norad_cat_id,
                         "international_designator": item.get("OBJECT_ID"),
                         "epoch": item.get("EPOCH"),
                         "mean_motion": item.get("MEAN_MOTION"),
@@ -105,6 +437,19 @@ class CelesTrakTLECollector(BaseCollector):
             )
         return transformed
 
+    @staticmethod
+    def _infer_constellation_group(item: Dict[str, Any]) -> str | None:
+        explicit_group = str(item.get("_celestrak_group") or "").strip().lower()
+        if explicit_group and explicit_group != ACTIVE_GROUP:
+            return explicit_group
+
+        name = str(item.get("OBJECT_NAME") or "").strip().upper()
+        if name.startswith("STARLINK"):
+            return "starlink"
+        if name.startswith("IRIDIUM"):
+            return "iridium-next"
+        return None
+
     def _get_sample_data(self) -> List[Dict[str, Any]]:
         return [
             {

backend/app/services/collectors/downloads.py

@@ -4,7 +4,7 @@ from __future__ import annotations
 
 import hashlib
 import json
-import tempfile
+import os
 import time
 from datetime import UTC, datetime
 from pathlib import Path
@@ -17,6 +17,31 @@ ProgressCallback = Callable[[int, int | None], Awaitable[None]]
 ValidateCallback = Callable[[Path], bool]
 
 
+class DownloadHTTPStatusError(RuntimeError):
+    """HTTP status error that keeps the upstream response body for caller-specific handling."""
+
+    def __init__(self, *, url: str, status_code: int, body: str) -> None:
+        self.url = url
+        self.status_code = status_code
+        self.body = body
+        preview = body.strip().replace("\r", " ").replace("\n", " ")[:240]
+        suffix = f": {preview}" if preview else ""
+        super().__init__(f"HTTP {status_code} while downloading {url}{suffix}")
+
+
+def default_download_cache_root() -> Path:
+    configured = os.getenv("PLANET_DOWNLOAD_CACHE_DIR")
+    if configured:
+        return Path(configured).expanduser()
+    planet_cache = os.getenv("PLANET_CACHE_DIR")
+    if planet_cache:
+        return Path(planet_cache).expanduser() / "downloads"
+    xdg_cache = os.getenv("XDG_CACHE_HOME")
+    if xdg_cache:
+        return Path(xdg_cache).expanduser() / "planet" / "downloads"
+    return Path.home() / ".cache" / "planet" / "downloads"
+
+
 class ResumableFileDownloader:
     """Download files with cache validators and byte-range resume support."""
 
@@ -26,8 +51,9 @@ class ResumableFileDownloader:
         cache_namespace: str,
         user_agent: str = "Planet-Intelligence-System/1.0 (Python/collector)",
         default_accept: str = "*/*",
+        cache_root: Path | None = None,
     ) -> None:
-        self._cache_dir = Path(tempfile.gettempdir()) / "planet-download-cache" / cache_namespace
+        self._cache_dir = (cache_root or default_download_cache_root()) / cache_namespace
         self._user_agent = user_agent
         self._default_accept = default_accept
 
@@ -43,6 +69,25 @@ class ResumableFileDownloader:
         meta_path = self._cache_dir / f"{key}.meta.json"
         return final_path, part_path, meta_path
 
+    def cached_file_path(self, url: str, extension: str) -> Path:
+        final_path, _, _ = self._cache_paths(url, extension)
+        return final_path
+
+    def get_cached_file(
+        self,
+        url: str,
+        extension: str,
+        *,
+        validate_existing: ValidateCallback | None = None,
+    ) -> Path | None:
+        final_path = self.cached_file_path(url, extension)
+        if not final_path.exists():
+            return None
+        if validate_existing and not validate_existing(final_path):
+            final_path.unlink(missing_ok=True)
+            return None
+        return final_path
+
     @staticmethod
     def _load_meta(meta_path: Path) -> dict[str, Any]:
         if not meta_path.exists():
@@ -140,7 +185,9 @@ class ResumableFileDownloader:
                 if progress_callback and expected_size and expected_size > 0:
                     await progress_callback(expected_size, expected_size)
                 return final_path
-            response.raise_for_status()
+            if response.status_code >= 400:
+                body = (await response.aread()).decode("utf-8", errors="replace")
+                raise DownloadHTTPStatusError(url=url, status_code=response.status_code, body=body)
 
             if response.status_code == 206 and resume_from > 0:
                 mode = "ab"

backend/app/services/collectors/telegeography.py

@@ -163,32 +163,64 @@ class TeleGeographyLandingPointCollector(BaseCollector):
     data_type = "landing_point"
 
     async def fetch(self) -> List[Dict[str, Any]]:
-        """Fetch landing point data from GitHub mirror"""
-        url = self._resolved_url or ""
+        """Fetch landing point data, falling back when the old mirror disappears."""
+        config = get_data_sources_config()
+        sources = [
+            self._resolved_url or "",
+            str(config.get_yaml_value("telegeography.landing_point_url") or ""),
+            str(config.get_yaml_value("arcgis.landing_point_url") or ""),
+        ]
 
-        async with httpx.AsyncClient(timeout=60.0) as client:
-            response = await client.get(url)
-            response.raise_for_status()
-            return self.parse_response(response.json())
+        last_error: Exception | None = None
+        async with httpx.AsyncClient(timeout=60.0, follow_redirects=True) as client:
+            for url in dict.fromkeys(source for source in sources if source):
+                try:
+                    params = (
+                        {"where": "1=1", "outFields": "*", "returnGeometry": "true", "f": "geojson"}
+                        if "FeatureServer" in url or url.endswith("/query")
+                        else None
+                    )
+                    response = await client.get(url, params=params)
+                    response.raise_for_status()
+                    records = self.parse_response(response.json())
+                    if records:
+                        return records
+                except Exception as exc:
+                    last_error = exc
+                    continue
 
-    def parse_response(self, data: List[Dict[str, Any]]) -> List[Dict[str, Any]]:
+        if last_error:
+            raise last_error
+        return self._get_sample_data()
+
+    def parse_response(self, data: List[Dict[str, Any]] | Dict[str, Any]) -> List[Dict[str, Any]]:
         """Parse landing point data"""
         result = []
+        items = data.get("features", []) if isinstance(data, dict) else data
 
-        for item in data:
+        for item in items:
+            props = item.get("properties", {}) if isinstance(item, dict) else {}
+            geometry = item.get("geometry", {}) if isinstance(item, dict) else {}
+            source = props or item
+            coords = geometry.get("coordinates", []) if isinstance(geometry, dict) else []
+            longitude = coords[0] if len(coords) > 0 else source.get("longitude")
+            latitude = coords[1] if len(coords) > 1 else source.get("latitude")
+            source_id = source.get("id") or source.get("OBJECTID") or source.get("city_id") or ""
             try:
                 entry = {
-                    "source_id": f"telegeo_lp_{item.get('id', '')}",
-                    "name": item.get("name", "Unknown"),
-                    "country": item.get("country", "Unknown"),
-                    "city": item.get("city", item.get("name", "")),
-                    "latitude": str(item.get("latitude", "")),
-                    "longitude": str(item.get("longitude", "")),
+                    "source_id": f"telegeo_lp_{source_id}",
+                    "name": source.get("name", source.get("Name", "Unknown")),
+                    "country": source.get("country", "Unknown"),
+                    "city": source.get("city", source.get("Name", source.get("name", ""))),
+                    "latitude": str(latitude or ""),
+                    "longitude": str(longitude or ""),
                     "value": "",
                     "unit": "",
                     "metadata": {
-                        "cable_count": len(item.get("cables", [])),
-                        "url": item.get("url"),
+                        "cable_count": len(source.get("cables", [])),
+                        "url": source.get("url"),
+                        "objectid": source.get("OBJECTID"),
+                        "city_id": source.get("city_id"),
                     },
                     "reference_date": datetime.now(UTC).strftime("%Y-%m-%d"),
                 }

backend/app/services/collectors/vessel_ais.py

@@ -119,6 +119,21 @@ class VesselAISCollector(BaseCollector):
             last_success_at=now if data else None,
             lag_seconds=max((now - latest_observed_at).total_seconds(), 0),
         )
+        if snapshot_id is not None:
+            from app.models.data_snapshot import DataSnapshot
+
+            snapshot = await db.get(DataSnapshot, snapshot_id)
+            if snapshot:
+                snapshot.record_count = records_added
+                snapshot.status = "success"
+                snapshot.completed_at = now
+                snapshot.summary = {
+                    "created": records_added,
+                    "updated": 0,
+                    "unchanged": 0,
+                    "deleted": 0,
+                    "storage": "ais_raw_observations",
+                }
         await db.commit()
         await self._broadcast_vessel_snapshot(data)
         await self.update_progress(records_added, force=True)

backend/app/services/compute_center_locations.py

@@ -11,7 +11,7 @@ startup, so it must stay local and deterministic.
 
 For the full design and the reason behind the abstraction (compute centers,
 BGP collectors, BGP events, and future entities all share one pipeline),
-see ``docs/plans/location-resolver-shared-pipeline-plan.md``.
+see ``docs/technical/zh/location-pipeline-development.md``.
 
 The ``ComputeCenterLocation`` dataclass and the public function signatures are
 preserved verbatim so existing callers and tests do not need to change.

backend/app/services/data_jobs.py

@@ -0,0 +1,719 @@
+"""Kafka-ready datasource job queue backed by PostgreSQL for v1."""
+
+from __future__ import annotations
+
+import asyncio
+from datetime import UTC, datetime, timedelta
+from uuid import uuid4
+from typing import Any
+
+from sqlalchemy import bindparam, select, text
+from sqlalchemy.ext.asyncio import AsyncSession
+
+from app.core.cache import cache
+from app.core.config import settings
+from app.core.logging import get_logger
+from app.core.time import to_iso8601_utc
+from app.core.websocket.broadcaster import broadcaster
+from app.db.session import async_session_factory
+from app.models.collected_data import CollectedData
+from app.models.data_snapshot import DataSnapshot
+from app.models.datasource import DataSource
+from app.models.task import CollectionTask
+from app.services.collectors.registry import collector_registry
+from app.services.datasource_connectivity import (
+    build_builtin_connectivity_checksum,
+    get_builtin_effective_candidate,
+    save_connectivity_success,
+)
+from app.services.earth_layer_adapters import (
+    clear_derived_datasource_data,
+    get_earth_refresh_strategy_for_change,
+    get_earth_update_layers_for_source,
+)
+from app.services.earth_layer_cache import invalidate_earth_layer_cache_for_source
+from app.services.scheduler import sync_datasource_job
+
+logger = get_logger(__name__)
+
+JOB_TYPE_COLLECT = "collect"
+JOB_TYPE_CLEAR_DATA = "clear_data"
+JOB_TYPE_CLEAR_CACHE = "clear_cache"
+JOB_TYPE_EARTH_REFRESH = "earth_refresh"
+
+JOB_STATUS_QUEUED = "queued"
+JOB_STATUS_RUNNING = "running"
+JOB_STATUS_CANCELLING = "cancelling"
+JOB_STATUS_SUCCESS = "success"
+JOB_STATUS_FAILED = "failed"
+JOB_STATUS_CANCELLED = "cancelled"
+
+ACTIVE_JOB_STATUSES = (JOB_STATUS_QUEUED, JOB_STATUS_RUNNING, JOB_STATUS_CANCELLING)
+TERMINAL_JOB_STATUSES = (JOB_STATUS_SUCCESS, JOB_STATUS_FAILED, JOB_STATUS_CANCELLED)
+DATA_WRITE_JOB_TYPES = (JOB_TYPE_COLLECT, JOB_TYPE_CLEAR_DATA, JOB_TYPE_CLEAR_CACHE)
+SOURCE_LOCK_JOB_STATUSES = (JOB_STATUS_RUNNING, JOB_STATUS_CANCELLING)
+QUEUE_POLL_SECONDS = 0.35
+JOB_STALE_LOCK_MINUTES = 90
+ORPHAN_CANCELLING_GRACE_SECONDS = 30
+JOB_RECOVERY_SWEEP_SECONDS = 15
+DATA_DELETE_BATCH_SIZE = 50_000
+DEFAULT_WORKER_CONCURRENCY = 2
+
+RUNNING_DATA_JOB_TASKS: dict[int, asyncio.Task[Any]] = {}
+
+
+def _utcnow() -> datetime:
+    return datetime.now(UTC)
+
+
+def _job_worker_id() -> str:
+    return f"{settings.PROJECT_NAME}:data-job-worker:{uuid4().hex[:8]}"
+
+
+def is_terminal_job_status(status: str | None) -> bool:
+    return status in TERMINAL_JOB_STATUSES
+
+
+async def enqueue_datasource_job(
+    db: AsyncSession,
+    datasource: DataSource,
+    task_type: str,
+    *,
+    payload: dict[str, Any] | None = None,
+    rollback_policy: str = "keep_committed_batches",
+    dedupe_key: str | None = None,
+) -> CollectionTask:
+    if dedupe_key:
+        existing = await _get_active_job_by_dedupe_key(db, dedupe_key)
+        if existing is not None:
+            return existing
+
+    task = CollectionTask(
+        datasource_id=datasource.id,
+        source=datasource.source,
+        task_type=task_type,
+        status=JOB_STATUS_QUEUED,
+        phase="queued",
+        phase_message="任务已进入队列",
+        payload=payload or {},
+        rollback_policy=rollback_policy,
+        dedupe_key=dedupe_key,
+    )
+    db.add(task)
+    await db.commit()
+    await db.refresh(task)
+    await _broadcast_task_update(task)
+    return task
+
+
+async def enqueue_earth_refresh_job(
+    db: AsyncSession,
+    *,
+    source: str,
+    payload: dict[str, Any] | None = None,
+) -> CollectionTask | None:
+    layers = list((payload or {}).get("layers") or get_earth_update_layers_for_source(source))
+    if not layers:
+        return None
+
+    datasource = await _get_or_create_virtual_datasource(db, source)
+    refresh_payload = {
+        "source": source,
+        "layers": layers,
+        "refresh_strategy": (payload or {}).get("refresh_strategy")
+        or get_earth_refresh_strategy_for_change((payload or {}).get("table"), source)
+        or "clear_then_reload",
+        **(payload or {}),
+    }
+    return await enqueue_datasource_job(
+        db,
+        datasource,
+        JOB_TYPE_EARTH_REFRESH,
+        payload=refresh_payload,
+        dedupe_key=f"earth_refresh:{source}",
+    )
+
+
+async def enqueue_earth_refresh_from_update(payload: dict[str, Any]) -> None:
+    source = str(payload.get("source") or "").strip()
+    if not source:
+        return
+    async with async_session_factory() as db:
+        await enqueue_earth_refresh_job(db, source=source, payload=payload)
+
+
+async def request_cancel_datasource_task(
+    db: AsyncSession,
+    task: CollectionTask,
+    *,
+    reason: str = "cancelled_by_operator",
+) -> CollectionTask:
+    if is_terminal_job_status(task.status):
+        return task
+
+    running_task = RUNNING_DATA_JOB_TASKS.get(task.id)
+    if task.status == JOB_STATUS_QUEUED or (
+        running_task is None
+        and (
+            task.status == JOB_STATUS_CANCELLING
+            or (task.status == JOB_STATUS_RUNNING and task.task_type != JOB_TYPE_COLLECT)
+        )
+    ):
+        return await _cancel_task_without_runner(db, task, reason=reason)
+
+    task.status = JOB_STATUS_CANCELLING
+    task.phase = JOB_STATUS_CANCELLING
+    task.phase_message = "正在停止任务"
+    task.requested_cancel_at = _utcnow()
+    task.cancel_reason = reason
+    await db.commit()
+    await db.refresh(task)
+
+    if running_task is not None and not running_task.done():
+        running_task.cancel()
+
+    await _broadcast_task_update(task)
+    return task
+
+
+async def _cancel_task_without_runner(
+    db: AsyncSession,
+    task: CollectionTask,
+    *,
+    reason: str,
+) -> CollectionTask:
+    if task.task_type == JOB_TYPE_COLLECT:
+        await db.execute(CollectedData.__table__.delete().where(CollectedData.task_id == task.id))
+        snapshot_result = await db.execute(select(DataSnapshot).where(DataSnapshot.task_id == task.id))
+        for snapshot in snapshot_result.scalars().all():
+            snapshot.status = JOB_STATUS_CANCELLED
+            snapshot.completed_at = _utcnow()
+            snapshot.error_message = "Cancelled after operator stop request; no active worker handle remained"
+        datasource = await db.get(DataSource, task.datasource_id)
+        if datasource is not None:
+            datasource.last_status = JOB_STATUS_CANCELLED
+
+    task.status = JOB_STATUS_CANCELLED
+    task.phase = JOB_STATUS_CANCELLED
+    task.phase_message = "任务已停止"
+    task.completed_at = _utcnow()
+    task.requested_cancel_at = task.requested_cancel_at or _utcnow()
+    task.cancel_reason = reason
+    await db.commit()
+    await db.refresh(task)
+    await _broadcast_task_update(task)
+    return task
+
+
+async def get_active_datasource_job(
+    db: AsyncSession,
+    datasource_id: int,
+    *,
+    task_types: tuple[str, ...] = DATA_WRITE_JOB_TYPES,
+) -> CollectionTask | None:
+    result = await db.execute(
+        select(CollectionTask)
+        .where(CollectionTask.datasource_id == datasource_id)
+        .where(CollectionTask.task_type.in_(task_types))
+        .where(CollectionTask.status.in_(ACTIVE_JOB_STATUSES))
+        .order_by(CollectionTask.created_at.desc().nullslast(), CollectionTask.id.desc())
+        .limit(1)
+    )
+    return result.scalar_one_or_none()
+
+
+async def _get_active_job_by_dedupe_key(db: AsyncSession, dedupe_key: str) -> CollectionTask | None:
+    result = await db.execute(
+        select(CollectionTask)
+        .where(CollectionTask.dedupe_key == dedupe_key)
+        .where(CollectionTask.status.in_(ACTIVE_JOB_STATUSES))
+        .order_by(CollectionTask.created_at.desc().nullslast(), CollectionTask.id.desc())
+        .limit(1)
+    )
+    return result.scalar_one_or_none()
+
+
+async def _get_or_create_virtual_datasource(db: AsyncSession, source: str) -> DataSource:
+    result = await db.execute(select(DataSource).where(DataSource.source == source))
+    datasource = result.scalar_one_or_none()
+    if datasource is not None:
+        return datasource
+
+    datasource = DataSource(
+        name=f"Earth refresh: {source}",
+        source=source,
+        module="SYS",
+        collector_class="EarthRefreshJob",
+        is_active=True,
+    )
+    db.add(datasource)
+    await db.commit()
+    await db.refresh(datasource)
+    return datasource
+
+
+async def _broadcast_task_update(task: CollectionTask) -> None:
+    await broadcaster.broadcast_datasource_task_update(
+        {
+            "datasource_id": task.datasource_id,
+            "collector_name": task.source,
+            "task_id": task.id,
+            "task_type": task.task_type,
+            "status": task.status,
+            "phase": task.phase,
+            "phase_progress": task.phase_progress,
+            "phase_message": task.phase_message,
+            "phase_current": task.phase_current,
+            "phase_total": task.phase_total,
+            "phase_unit": task.phase_unit,
+            "progress": task.progress,
+            "records_processed": task.records_processed,
+            "total_records": task.total_records,
+            "started_at": to_iso8601_utc(task.started_at),
+            "completed_at": to_iso8601_utc(task.completed_at),
+            "requested_cancel_at": to_iso8601_utc(task.requested_cancel_at),
+            "error_message": task.error_message,
+        }
+    )
+
+
+class DataJobWorker:
+    def __init__(self, *, concurrency: int = DEFAULT_WORKER_CONCURRENCY) -> None:
+        self.worker_id = _job_worker_id()
+        self.concurrency = max(1, concurrency)
+        self._task: asyncio.Task[None] | None = None
+        self._stop_event: asyncio.Event | None = None
+        self._running: set[asyncio.Task[Any]] = set()
+        self._last_recovery_sweep_at: datetime | None = None
+
+    def start(self) -> None:
+        if self._task and not self._task.done():
+            return
+        self._stop_event = asyncio.Event()
+        self._task = asyncio.create_task(self._run(), name="data-job-worker")
+
+    async def stop(self) -> None:
+        if self._stop_event:
+            self._stop_event.set()
+        for task in list(self._running):
+            task.cancel()
+        if self._task:
+            await asyncio.gather(self._task, return_exceptions=True)
+        if self._running:
+            await asyncio.gather(*self._running, return_exceptions=True)
+
+    async def _run(self) -> None:
+        assert self._stop_event is not None
+        await self._recover_stale_running_jobs()
+        while not self._stop_event.is_set():
+            self._running = {task for task in self._running if not task.done()}
+            if (
+                self._last_recovery_sweep_at is None
+                or (_utcnow() - self._last_recovery_sweep_at).total_seconds() >= JOB_RECOVERY_SWEEP_SECONDS
+            ):
+                await self._recover_stale_running_jobs()
+            if len(self._running) >= self.concurrency:
+                await asyncio.sleep(QUEUE_POLL_SECONDS)
+                continue
+
+            task_id = await self._claim_next_job()
+            if task_id is None:
+                await asyncio.sleep(QUEUE_POLL_SECONDS)
+                continue
+
+            runner = asyncio.create_task(self._run_claimed_job(task_id), name=f"data-job:{task_id}")
+            self._running.add(runner)
+
+    async def _recover_stale_running_jobs(self) -> None:
+        self._last_recovery_sweep_at = _utcnow()
+        cutoff = _utcnow() - timedelta(minutes=JOB_STALE_LOCK_MINUTES)
+        orphan_cancelling_cutoff = _utcnow() - timedelta(seconds=ORPHAN_CANCELLING_GRACE_SECONDS)
+        async with async_session_factory() as db:
+            result = await db.execute(
+                select(CollectionTask)
+                .where(CollectionTask.status.in_((JOB_STATUS_RUNNING, JOB_STATUS_CANCELLING)))
+                .where(CollectionTask.locked_at.is_not(None))
+                .where(CollectionTask.locked_at < cutoff)
+            )
+            stale_jobs = list(result.scalars().all())
+            for job in stale_jobs:
+                job.status = JOB_STATUS_FAILED
+                job.phase = JOB_STATUS_FAILED
+                job.completed_at = _utcnow()
+                job.error_message = "Marked failed after stale data job lock timeout"
+            if stale_jobs:
+                await db.commit()
+            orphan_result = await db.execute(
+                select(CollectionTask)
+                .where(CollectionTask.status == JOB_STATUS_CANCELLING)
+                .where(CollectionTask.locked_at.is_(None))
+                .where(CollectionTask.requested_cancel_at.is_not(None))
+                .where(CollectionTask.requested_cancel_at < orphan_cancelling_cutoff)
+            )
+            for job in orphan_result.scalars().all():
+                if job.id in RUNNING_DATA_JOB_TASKS:
+                    continue
+                await _cancel_task_without_runner(
+                    db,
+                    job,
+                    reason=job.cancel_reason or "cancelled_after_orphaned_runner",
+                )
+
+    async def _claim_next_job(self) -> int | None:
+        async with async_session_factory() as db:
+            row = await db.execute(
+                text(
+                    """
+                    SELECT queued.id
+                    FROM collection_tasks AS queued
+                    WHERE queued.status = :queued_status
+                      AND NOT EXISTS (
+                        SELECT 1
+                        FROM collection_tasks AS active
+                        WHERE active.source = queued.source
+                          AND active.id <> queued.id
+                          AND active.status IN :active_statuses
+                      )
+                    ORDER BY queued.created_at ASC NULLS FIRST, queued.id ASC
+                    LIMIT 1
+                    FOR UPDATE SKIP LOCKED
+                    """
+                ).bindparams(bindparam("active_statuses", expanding=True)),
+                {
+                    "queued_status": JOB_STATUS_QUEUED,
+                    "active_statuses": SOURCE_LOCK_JOB_STATUSES,
+                },
+            )
+            task_id = row.scalar_one_or_none()
+            if task_id is None:
+                return None
+
+            task = await db.get(CollectionTask, int(task_id))
+            if task is None:
+                return None
+            task.status = JOB_STATUS_RUNNING
+            task.phase = "starting"
+            task.phase_message = "任务开始执行"
+            task.started_at = task.started_at or _utcnow()
+            task.worker_id = self.worker_id
+            task.locked_at = _utcnow()
+            await db.commit()
+            await _broadcast_task_update(task)
+            return int(task_id)
+
+    async def _run_claimed_job(self, task_id: int) -> None:
+        current_task = asyncio.current_task()
+        if current_task is not None:
+            RUNNING_DATA_JOB_TASKS[task_id] = current_task
+        try:
+            async with async_session_factory() as db:
+                task = await db.get(CollectionTask, task_id)
+                if task is None:
+                    return
+                await self._execute_job(db, task)
+        except asyncio.CancelledError:
+            async with async_session_factory() as db:
+                task = await db.get(CollectionTask, task_id)
+                if task is not None and not is_terminal_job_status(task.status):
+                    task.status = JOB_STATUS_CANCELLED
+                    task.phase = JOB_STATUS_CANCELLED
+                    task.phase_message = "任务已停止"
+                    task.completed_at = _utcnow()
+                    await db.commit()
+                    await _broadcast_task_update(task)
+            raise
+        except Exception as exc:
+            logger.exception_event(
+                "Data job failed",
+                event="data_jobs.job_failed",
+                context={"task_id": task_id, "error": str(exc)},
+            )
+            async with async_session_factory() as db:
+                task = await db.get(CollectionTask, task_id)
+                if task is not None:
+                    task.status = JOB_STATUS_FAILED
+                    task.phase = JOB_STATUS_FAILED
+                    task.phase_message = str(exc)
+                    task.error_message = str(exc)
+                    task.completed_at = _utcnow()
+                    await db.commit()
+                    await _broadcast_task_update(task)
+        finally:
+            RUNNING_DATA_JOB_TASKS.pop(task_id, None)
+
+    async def _execute_job(self, db: AsyncSession, task: CollectionTask) -> None:
+        if task.task_type == JOB_TYPE_COLLECT:
+            await _run_collect_job(db, task)
+        elif task.task_type == JOB_TYPE_CLEAR_DATA:
+            await _run_clear_data_job(db, task)
+        elif task.task_type == JOB_TYPE_CLEAR_CACHE:
+            await _run_clear_cache_job(db, task)
+        elif task.task_type == JOB_TYPE_EARTH_REFRESH:
+            await _run_earth_refresh_job(db, task)
+        else:
+            raise RuntimeError(f"Unsupported data job type: {task.task_type}")
+
+
+async def _run_collect_job(db: AsyncSession, task: CollectionTask) -> None:
+    datasource = await db.get(DataSource, task.datasource_id)
+    if datasource is None:
+        raise RuntimeError("Data source not found")
+
+    collector = collector_registry.get(datasource.source)
+    if collector is None:
+        raise RuntimeError(f"Collector '{datasource.source}' not found")
+    if not datasource.is_active:
+        raise RuntimeError("Data source is disabled")
+
+    collector._datasource_id = datasource.id
+    collector._current_task = task
+    collector._db_session = db
+    result = await collector.run(db)
+
+    datasource.last_run_at = _utcnow()
+    datasource.last_status = result.get("status")
+    if datasource.last_status == JOB_STATUS_SUCCESS:
+        effective_candidate = await get_builtin_effective_candidate(db, datasource.source)
+        checksum, _credential_context = await build_builtin_connectivity_checksum(
+            datasource.source,
+            effective_candidate["endpoint"],
+            effective_candidate["auth_type"],
+            effective_candidate["headers"],
+            effective_candidate["config"],
+            db,
+        )
+        await save_connectivity_success(
+            db,
+            datasource.source,
+            checksum,
+            {"status_code": None},
+            connected_by="collection",
+        )
+    await db.commit()
+    await sync_datasource_job(datasource.id)
+
+
+async def _run_clear_data_job(db: AsyncSession, task: CollectionTask) -> None:
+    source = str(task.source or (task.payload or {}).get("source") or "").strip()
+    if not source:
+        raise RuntimeError("Clear data job has no source")
+
+    task.phase = "clearing_data"
+    task.phase_message = "正在删除数据库数据"
+    await db.commit()
+    await _broadcast_task_update(task)
+
+    deleted_count = await _delete_table_rows_by_source(
+        db,
+        task,
+        table_name="collected_data",
+        source_column="source",
+        source=source,
+    )
+    derived_deleted_counts = await _clear_derived_datasource_data_in_batches(
+        db,
+        task,
+        source,
+        progress_offset=deleted_count,
+    )
+    derived_deleted_count = sum(derived_deleted_counts.values())
+    if any(key.startswith("ais_") for key in derived_deleted_counts):
+        await db.execute(text("ANALYZE ais_raw_observations"))
+
+    task.records_processed = deleted_count + derived_deleted_count
+    task.total_records = task.records_processed
+    task.progress = 100.0
+    task.phase_progress = 100.0
+    task.phase_current = task.records_processed
+    task.phase_total = task.records_processed
+    task.phase_unit = "records"
+    task.payload = {
+        **(task.payload or {}),
+        "deleted_count": deleted_count,
+        "derived_deleted_count": derived_deleted_count,
+        "derived_deleted_counts": derived_deleted_counts,
+    }
+    task.status = JOB_STATUS_SUCCESS
+    task.phase = "completed"
+    task.phase_message = "数据库数据已清理"
+    task.completed_at = _utcnow()
+    datasource = await db.get(DataSource, task.datasource_id)
+    if datasource is not None:
+        datasource.last_status = JOB_STATUS_SUCCESS
+        datasource.last_run_at = task.completed_at
+    await db.execute(
+        DataSnapshot.__table__.update()
+        .where(DataSnapshot.source == source)
+        .values(is_current=False)
+    )
+    await db.commit()
+    await _broadcast_task_update(task)
+
+
+async def _delete_table_rows_by_source(
+    db: AsyncSession,
+    task: CollectionTask,
+    *,
+    table_name: str,
+    source_column: str,
+    source: str,
+    progress_offset: int = 0,
+) -> int:
+    deleted = 0
+    while True:
+        result = await db.execute(
+            text(
+                f"""
+                WITH doomed AS (
+                    SELECT ctid
+                    FROM {table_name}
+                    WHERE {source_column} = :source
+                    LIMIT :batch_size
+                ),
+                deleted_rows AS (
+                    DELETE FROM {table_name}
+                    USING doomed
+                    WHERE {table_name}.ctid = doomed.ctid
+                    RETURNING 1
+                )
+                SELECT COUNT(*) FROM deleted_rows
+                """
+            ),
+            {"source": source, "batch_size": DATA_DELETE_BATCH_SIZE},
+        )
+        batch_deleted = max(int(result.scalar_one() or 0), 0)
+        if batch_deleted <= 0:
+            break
+        deleted += batch_deleted
+        task.records_processed = progress_offset + deleted
+        task.phase_current = task.records_processed
+        task.phase_unit = "records"
+        task.phase_message = f"正在删除数据：{task.records_processed} 条"
+        await db.commit()
+        await _broadcast_task_update(task)
+    return deleted
+
+
+async def _clear_derived_datasource_data_in_batches(
+    db: AsyncSession,
+    task: CollectionTask,
+    source: str,
+    progress_offset: int = 0,
+) -> dict[str, int]:
+    deleted_counts: dict[str, int] = {}
+    if source in {"barentswatch_vessels", "aisstream_vessels"}:
+        deleted_counts["ais_conflict_records"] = await _delete_table_rows_by_source(
+            db,
+            task,
+            table_name="ais_conflict_records",
+            source_column="selected_source",
+            source=source,
+            progress_offset=progress_offset + sum(deleted_counts.values()),
+        )
+        deleted_counts["ais_source_health"] = await _delete_table_rows_by_source(
+            db,
+            task,
+            table_name="ais_source_health",
+            source_column="source",
+            source=source,
+            progress_offset=progress_offset + sum(deleted_counts.values()),
+        )
+        deleted_counts["ais_raw_observations"] = await _delete_table_rows_by_source(
+            db,
+            task,
+            table_name="ais_raw_observations",
+            source_column="source",
+            source=source,
+            progress_offset=progress_offset + sum(deleted_counts.values()),
+        )
+        return deleted_counts
+    return await clear_derived_datasource_data(db, source)
+
+
+async def _run_clear_cache_job(db: AsyncSession, task: CollectionTask) -> None:
+    source = str(task.source or (task.payload or {}).get("source") or "").strip()
+    if not source:
+        raise RuntimeError("Clear cache job has no source")
+
+    earth_deleted_count = invalidate_earth_layer_cache_for_source(source)
+    dashboard_deleted_count = int(cache.delete("dashboard:stats")) + int(cache.delete("dashboard:summary"))
+    deleted_count = earth_deleted_count + dashboard_deleted_count
+
+    task.records_processed = deleted_count
+    task.total_records = deleted_count
+    task.progress = 100.0
+    task.phase_progress = 100.0
+    task.phase = "completed"
+    task.phase_message = "缓存已清理"
+    task.payload = {
+        **(task.payload or {}),
+        "earth_layer_deleted_count": earth_deleted_count,
+        "dashboard_deleted_count": dashboard_deleted_count,
+    }
+    task.status = JOB_STATUS_SUCCESS
+    task.completed_at = _utcnow()
+    await db.commit()
+    await _broadcast_task_update(task)
+    await enqueue_earth_refresh_job(db, source=source, payload={"operation": "CACHE_INVALIDATED"})
+
+
+async def _run_earth_refresh_job(db: AsyncSession, task: CollectionTask) -> None:
+    payload = task.payload or {}
+    source = str(payload.get("source") or task.source or "").strip()
+    layers = list(payload.get("layers") or get_earth_update_layers_for_source(source))
+    if not source or not layers:
+        task.status = JOB_STATUS_SUCCESS
+        task.phase = "completed"
+        task.phase_message = "没有需要刷新的 Earth 图层"
+        task.completed_at = _utcnow()
+        await db.commit()
+        await _broadcast_task_update(task)
+        return
+
+    deleted_cache_entries = invalidate_earth_layer_cache_for_source(source)
+    update_payload = {
+        "event": "earth.layer.changed",
+        "action": "database_changed",
+        "source": source,
+        "table": payload.get("table"),
+        "data_type": source,
+        "layers": layers,
+        "refresh_strategy": payload.get("refresh_strategy") or "clear_then_reload",
+        "records_processed": payload.get("records_processed", 0),
+        "operations": payload.get("operations") or [payload.get("operation") or "CHANGE"],
+        "operation": payload.get("operation"),
+        "cache_entries_invalidated": deleted_cache_entries,
+        "timestamp": to_iso8601_utc(_utcnow()),
+    }
+    if payload.get("entity") == "interactable":
+        update_payload.update(
+            {
+                "entity": "interactable",
+                "action": payload.get("action") or "changed",
+                "ids": payload.get("ids") or payload.get("entity_keys") or [],
+                "item": payload.get("item"),
+            }
+        )
+    await broadcaster.broadcast_earth_update(update_payload)
+
+    task.records_processed = int(payload.get("records_processed") or 0)
+    task.progress = 100.0
+    task.phase_progress = 100.0
+    task.phase = "completed"
+    task.phase_message = "Earth 图层刷新通知已发送"
+    task.status = JOB_STATUS_SUCCESS
+    task.completed_at = _utcnow()
+    task.payload = {**payload, "cache_entries_invalidated": deleted_cache_entries}
+    await db.commit()
+    await _broadcast_task_update(task)
+
+
+_worker = DataJobWorker()
+
+
+def start_data_job_worker() -> None:
+    _worker.start()
+
+
+async def stop_data_job_worker() -> None:
+    await _worker.stop()

backend/app/services/docs_gatekeeper.py

@@ -32,28 +32,31 @@ class DocsMetadata:
 
 DOCS_METADATA: tuple[DocsMetadata, ...] = (
     DocsMetadata(DOCS_README_FILENAME, DEFAULT_DOCS_SLUG, "public", "Overview", 0, "技术文档", "Technical Docs"),
-    DocsMetadata("quickstart.md", "quickstart", "public", "Manual", 1, "快速开始", "Quickstart"),
-    DocsMetadata("manual.md", "manual", "public", "Manual", 2, "Planet 使用手册", "Planet Manual"),
+    DocsMetadata("manual.md", "manual", "public", "Manual", 1, "智能星球使用手册", "Intelligent Planet Manual"),
+    DocsMetadata("quickstart.md", "quickstart", "public", "Manual", 2, "快速开始", "Quickstart"),
     DocsMetadata("faq.md", "faq", "public", "Manual", 3, "常见问题", "FAQ"),
-    DocsMetadata("location-pipeline-user.md", "location-pipeline-user", "public", "Manual", 4, "Earth 位置候选采集使用手册", "Earth Location Candidate Collection User Guide"),
-    DocsMetadata("earth-frontend-context.md", "earth-frontend-context", "docs_developer", "Earth", 10, "Earth 前端结构", "Earth Frontend Context"),
-    DocsMetadata("earth-layer-style-reference.md", "earth-layer-style-reference", "docs_developer", "Earth", 11, "Earth 图层样式属性索引", "Earth Layer Style Reference"),
-    DocsMetadata("earth-render-layer-order.md", "earth-render-layer-order", "docs_developer", "Earth", 12, "Earth 渲染图层顺序", "Earth Render Layer Order"),
-    DocsMetadata("earth-satellite-footprint-policy.md", "earth-satellite-footprint-policy", "docs_developer", "Earth", 13, "Earth 卫星覆盖策略", "Earth Satellite Footprint Policy"),
+    DocsMetadata("platform-data-flows.md", "platform-data-flows", "docs_developer", "Architecture", 5, "业务架构与数据流转", "Business Architecture and Data Flows"),
+    DocsMetadata("naming-glossary.md", "naming-glossary", "docs_developer", "Architecture", 6, "命名与术语对照", "Naming Glossary"),
+    DocsMetadata("earth-frontend-context.md", "earth-frontend-context", "docs_developer", "Earth", 10, "智能星球前端结构", "Intelligent Planet Frontend Context"),
+    DocsMetadata("earth-layer-style-reference.md", "earth-layer-style-reference", "docs_developer", "Earth", 11, "智能星球图层样式属性索引", "Intelligent Planet Layer Style Reference"),
+    DocsMetadata("earth-render-layer-order.md", "earth-render-layer-order", "docs_developer", "Earth", 12, "智能星球渲染图层顺序", "Intelligent Planet Render Layer Order"),
+    DocsMetadata("earth-satellite-footprint-policy.md", "earth-satellite-footprint-policy", "docs_developer", "Earth", 13, "智能星球卫星覆盖策略", "Intelligent Planet Satellite Footprint Policy"),
     DocsMetadata("earth-bgp-context.md", "earth-bgp-context", "docs_developer", "Earth", 14, "BGP 态势上下文", "BGP Context"),
-    DocsMetadata("earth-news-live-streams-collector-format.md", "earth-news-live-streams-collector-format", "docs_developer", "Earth", 15, "新闻直播采集格式", "News Live Streams Collector Format"),
-    DocsMetadata("earth-interactable-usage.md", "earth-interactable-usage", "docs_developer", "Earth", 16, "Earth 可交互图标接入", "Earth Interactable Usage"),
-    DocsMetadata("earth-toolbar-overlay-coordination.md", "earth-toolbar-overlay-coordination", "docs_developer", "Earth", 17, "Earth 工具栏与浮层协同", "Earth Toolbar and Overlay Coordination"),
+    DocsMetadata("earth-interactable-usage.md", "earth-interactable-usage", "docs_developer", "Earth", 16, "智能星球可交互图标接入", "Intelligent Planet Interactable Usage"),
+    DocsMetadata("earth-toolbar-overlay-coordination.md", "earth-toolbar-overlay-coordination", "docs_developer", "Earth", 17, "智能星球工具栏与浮层协同", "Intelligent Planet Toolbar and Overlay Coordination"),
     DocsMetadata("frontend-admin-frontend-context.md", "frontend-admin-frontend-context", "docs_developer", "Frontend", 20, "控制台前端结构", "Admin Frontend Context"),
     DocsMetadata("frontend-layout-guidelines.md", "frontend-layout-guidelines", "docs_developer", "Frontend", 21, "前端布局指南", "Frontend Layout Guidelines"),
-    DocsMetadata("docs-gatekeeper-development.md", "docs-gatekeeper-development", "docs_developer", "Frontend", 22, "Docs Gatekeeper 开发说明", "Docs Gatekeeper Development Guide"),
+    DocsMetadata("tactile-ui-components.md", "tactile-ui-components", "docs_developer", "Frontend", 24, "Tactile UI 组件库", "Tactile UI Components"),
     DocsMetadata("backend-collectors.md", "backend-collectors", "docs_developer", "Backend", 30, "数据采集系统", "Data Collectors"),
     DocsMetadata("backend-system-service-control.md", "backend-system-service-control", "docs_admin", "Backend", 31, "系统服务控制", "System Service Control"),
     DocsMetadata("datasource-collector-settings-connectivity.md", "datasource-collector-settings-connectivity", "docs_developer", "Backend", 32, "数据源、采集器设置与连接验证", "Datasource Collector Settings and Connectivity"),
     DocsMetadata("backend-datasources-api-performance.md", "backend-datasources-api-performance", "docs_developer", "Backend", 33, "数据源 API 性能", "Datasource API Performance"),
-    DocsMetadata("location-pipeline-development.md", "location-pipeline-development", "docs_developer", "Backend", 34, "通用位置估算管线开发说明", "Shared Location Resolution Pipeline Development Guide"),
+    DocsMetadata("data-job-earth-sync-architecture.md", "data-job-earth-sync-architecture", "docs_developer", "Backend", 34, "数据作业与 Outbox 技术架构", "Data Jobs and Outbox Architecture"),
+    DocsMetadata("location-pipeline-development.md", "location-pipeline-development", "docs_developer", "Backend", 35, "通用位置估算管线开发说明", "Shared Location Resolution Pipeline Development Guide"),
+    DocsMetadata("earth-news-live-streams-collector-format.md", "earth-news-live-streams-collector-format", "docs_developer", "Backend", 36, "新闻直播采集格式", "News Live Streams Collector Format"),
+    DocsMetadata("docs-gatekeeper-development.md", "docs-gatekeeper-development", "docs_developer", "Backend", 37, "Docs Gatekeeper 开发说明", "Docs Gatekeeper Development Guide"),
     DocsMetadata("agents-aiprovider.md", "agents-aiprovider", "docs_developer", "Agents", 40, "AI Provider 指南", "AI Provider Guide"),
-    DocsMetadata("ops-runbook.md", "ops-runbook", "docs_admin", "Ops", 49, "Planet 运维手册", "Planet Ops Runbook"),
+    DocsMetadata("ops-runbook.md", "ops-runbook", "docs_admin", "Ops", 49, "智能星球运维手册", "Intelligent Planet Ops Runbook"),
     DocsMetadata("ops-docker-compose-buildx-upgrade.md", "ops-docker-compose-buildx-upgrade", "docs_admin", "Ops", 50, "Docker + Compose + Buildx 升级", "Docker + Compose + Buildx Upgrade"),
     DocsMetadata("ops-planet-sh-startup.md", "ops-planet-sh-startup", "docs_admin", "Ops", 51, "planet.sh 启动机制", "planet.sh Startup"),
 )

backend/app/services/earth_db_change_listener.py

@@ -0,0 +1,482 @@
+"""PostgreSQL LISTEN/NOTIFY bridge for Earth layer refresh events."""
+
+from __future__ import annotations
+
+import asyncio
+import json
+from collections import deque
+from dataclasses import dataclass, field
+from datetime import UTC, datetime
+from time import monotonic
+from typing import Any, Awaitable, Callable
+
+import asyncpg
+
+from app.core.config import settings
+from app.core.logging import get_logger
+from app.core.time import to_iso8601_utc
+from app.core.websocket.broadcaster import broadcaster
+from app.services.earth_layer_adapters import (
+    get_earth_refresh_strategy_for_change,
+    get_earth_update_layers_for_change,
+    get_earth_update_layers_for_source,
+)
+from app.services.earth_layer_cache import invalidate_earth_layer_cache_for_source
+
+logger = get_logger(__name__)
+
+EARTH_DATA_CHANGES_CHANNEL = "planet_earth_data_changes"
+DEFAULT_DEBOUNCE_SECONDS = 0.25
+DEFAULT_MAX_WAIT_SECONDS = 1.5
+DELETE_FAST_FLUSH_SECONDS = 0.05
+LISTEN_KEEPALIVE_SECONDS = 5.0
+OUTBOX_POLL_LIMIT = 5000
+MAX_ENTITY_KEY_SAMPLES = 20
+MAX_SEEN_EVENT_IDS = 20000
+
+BroadcastFn = Callable[[dict[str, Any]], Awaitable[None]]
+InvalidateFn = Callable[[str], int]
+
+
+def normalize_asyncpg_dsn(dsn: str) -> str:
+    """Convert SQLAlchemy asyncpg URLs into asyncpg-compatible URLs."""
+    return dsn.replace("postgresql+asyncpg://", "postgresql://", 1)
+
+
+def build_earth_update_from_db_payload(payload: dict[str, Any]) -> dict[str, Any] | None:
+    table = payload.get("table")
+    source = str(payload.get("source") or "").strip()
+    table_name = str(table or "").strip()
+    if not source and not table_name:
+        return None
+    layers = get_earth_update_layers_for_change(table_name, source)
+    if not layers:
+        return None
+    refresh_strategy = get_earth_refresh_strategy_for_change(table_name, source) or "clear_then_reload"
+    source_has_adapter = bool(get_earth_update_layers_for_source(source))
+    effective_source = source if source_has_adapter else (table_name if table_name else source)
+    operation = payload.get("operation")
+    update: dict[str, Any] = {
+        "event": "earth.layer.changed",
+        "action": "database_changed",
+        "source": effective_source,
+        "original_source": source or None,
+        "table": table_name or None,
+        "data_type": effective_source,
+        "layers": layers,
+        "refresh_strategy": refresh_strategy,
+        "operation": operation,
+        "entity_key": payload.get("entity_key"),
+        "timestamp": to_iso8601_utc(datetime.now(UTC)),
+    }
+    if table_name == "earth_interactables" and refresh_strategy == "delta":
+        ids = payload.get("entity_keys")
+        if not isinstance(ids, list):
+            ids = [payload.get("entity_key")] if payload.get("entity_key") else []
+        update.update(
+            {
+                "entity": "interactable",
+                "action": "deleted" if operation == "DELETE" else "changed",
+                "ids": [str(item) for item in ids if item],
+                "item": None,
+            }
+        )
+    return {
+        **update,
+    }
+
+
+@dataclass
+class PendingEarthDbChange:
+    source: str
+    layers: list[str]
+    table: str | None = None
+    refresh_strategy: str = "clear_then_reload"
+    entity: str | None = None
+    action: str = "database_changed"
+    records_processed: int = 0
+    operations: set[str] = field(default_factory=set)
+    entity_keys: list[str] = field(default_factory=list)
+    first_occurred_at: str | None = None
+    last_occurred_at: str | None = None
+    first_seen_monotonic: float = field(default_factory=monotonic)
+    last_seen_monotonic: float = field(default_factory=monotonic)
+
+    def add(self, payload: dict[str, Any]) -> None:
+        self.last_seen_monotonic = monotonic()
+        records_processed = payload.get("records_processed", 1)
+        try:
+            records_processed = int(records_processed)
+        except (TypeError, ValueError):
+            records_processed = 1
+        self.records_processed += max(records_processed, 1)
+        operation = payload.get("operation")
+        if operation:
+            self.operations.add(str(operation))
+        entity_keys = payload.get("entity_keys")
+        if not isinstance(entity_keys, list):
+            entity_key = payload.get("entity_key")
+            entity_keys = [entity_key] if entity_key else []
+        for entity_key in entity_keys:
+            if entity_key and len(self.entity_keys) < MAX_ENTITY_KEY_SAMPLES:
+                self.entity_keys.append(str(entity_key))
+        occurred_at = payload.get("occurred_at")
+        if occurred_at:
+            occurred_at = str(occurred_at)
+            self.first_occurred_at = self.first_occurred_at or occurred_at
+            self.last_occurred_at = occurred_at
+
+
+class EarthDbChangeDispatcher:
+    """Debounces database notifications and broadcasts Earth refresh hints."""
+
+    def __init__(
+        self,
+        *,
+        broadcast_earth_update: BroadcastFn | None = None,
+        invalidate_cache: InvalidateFn | None = None,
+        debounce_seconds: float = DEFAULT_DEBOUNCE_SECONDS,
+        max_wait_seconds: float = DEFAULT_MAX_WAIT_SECONDS,
+    ) -> None:
+        self._broadcast_earth_update = broadcast_earth_update or broadcaster.broadcast_earth_update
+        self._invalidate_cache = invalidate_cache or invalidate_earth_layer_cache_for_source
+        self._debounce_seconds = debounce_seconds
+        self._max_wait_seconds = max(max_wait_seconds, debounce_seconds)
+        self._pending: dict[str, PendingEarthDbChange] = {}
+        self._flush_tasks: dict[str, asyncio.Task[None]] = {}
+        self._seen_event_ids: set[int] = set()
+        self._seen_event_order: deque[int] = deque()
+
+    def handle_notification(self, payload_text: str) -> bool:
+        try:
+            payload = json.loads(payload_text)
+        except json.JSONDecodeError:
+            logger.warning_event(
+                "Ignoring malformed Earth database change notification",
+                event="earth.db_changes.notification_malformed",
+            )
+            return False
+        if not isinstance(payload, dict):
+            return False
+        return self.handle_payload(payload)
+
+    def handle_payload(self, payload: dict[str, Any]) -> bool:
+        event_id = payload.get("event_id")
+        if event_id is not None:
+            try:
+                normalized_event_id = int(event_id)
+            except (TypeError, ValueError):
+                normalized_event_id = None
+            if normalized_event_id is not None:
+                if normalized_event_id in self._seen_event_ids:
+                    return False
+                self._remember_event_id(normalized_event_id)
+
+        update = build_earth_update_from_db_payload(payload)
+        if not update:
+            return False
+
+        source = update["source"]
+        pending = self._pending.get(source)
+        if pending is None:
+            pending = PendingEarthDbChange(
+                source=source,
+                layers=list(update["layers"]),
+                table=update.get("table"),
+                refresh_strategy=str(update.get("refresh_strategy") or "clear_then_reload"),
+                entity=update.get("entity"),
+                action=str(update.get("action") or "database_changed"),
+            )
+            self._pending[source] = pending
+        pending.add(payload)
+
+        task = self._flush_tasks.pop(source, None)
+        if task and not task.done():
+            task.cancel()
+        self._flush_tasks[source] = asyncio.create_task(
+            self._flush_later(source, delay_seconds=self._next_flush_delay(pending))
+        )
+        return True
+
+    def _next_flush_delay(self, pending: PendingEarthDbChange) -> float:
+        if "DELETE" in pending.operations and pending.refresh_strategy == "clear_then_reload":
+            return DELETE_FAST_FLUSH_SECONDS
+        elapsed = max(0.0, monotonic() - pending.first_seen_monotonic)
+        remaining = self._max_wait_seconds - elapsed
+        if remaining <= 0:
+            return 0.0
+        return min(self._debounce_seconds, remaining)
+
+    def _remember_event_id(self, event_id: int) -> None:
+        self._seen_event_ids.add(event_id)
+        self._seen_event_order.append(event_id)
+        while len(self._seen_event_order) > MAX_SEEN_EVENT_IDS:
+            expired_event_id = self._seen_event_order.popleft()
+            self._seen_event_ids.discard(expired_event_id)
+
+    async def _flush_later(self, source: str, *, delay_seconds: float) -> None:
+        try:
+            if delay_seconds > 0:
+                await asyncio.sleep(delay_seconds)
+            await self.flush_source(source)
+        except asyncio.CancelledError:
+            raise
+        except Exception as exc:
+            logger.exception_event(
+                "Failed to broadcast debounced Earth database change",
+                event="earth.db_changes.flush_failed",
+                context={"source": source, "error": str(exc)},
+            )
+        finally:
+            current = self._flush_tasks.get(source)
+            if current is asyncio.current_task():
+                self._flush_tasks.pop(source, None)
+
+    async def flush_source(self, source: str) -> None:
+        pending = self._pending.get(source)
+        if pending is None:
+            return
+
+        flushed_at = datetime.now(UTC)
+        deleted_cache_entries = self._invalidate_cache(source)
+        operations = sorted(pending.operations)
+        payload: dict[str, Any] = {
+            "event": "earth.layer.changed",
+            "action": "database_changed",
+            "source": source,
+            "table": pending.table,
+            "data_type": source,
+            "layers": pending.layers,
+            "refresh_strategy": pending.refresh_strategy,
+            "records_processed": pending.records_processed,
+            "operations": operations,
+            "operation": operations[-1] if len(operations) == 1 else None,
+            "entity_keys": pending.entity_keys,
+            "entity_key_sample_size": len(pending.entity_keys),
+            "cache_entries_invalidated": deleted_cache_entries,
+            "first_occurred_at": pending.first_occurred_at,
+            "last_occurred_at": pending.last_occurred_at,
+            "debounce_ms": int((monotonic() - pending.first_seen_monotonic) * 1000),
+            "total_latency_ms": self._total_latency_ms(pending, flushed_at),
+            "timestamp": to_iso8601_utc(flushed_at),
+        }
+        if pending.entity == "interactable":
+            payload.update(
+                {
+                    "entity": "interactable",
+                    "action": "deleted" if "DELETE" in pending.operations else "changed",
+                    "ids": pending.entity_keys,
+                    "item": None,
+                }
+            )
+        await self._broadcast_earth_update(payload)
+        self._pending.pop(source, None)
+        logger.info_event(
+            "Broadcasted Earth database change",
+            event="earth.db_changes.broadcasted",
+            context={
+                "source": source,
+                "layers": pending.layers,
+                "records_processed": pending.records_processed,
+                "cache_entries_invalidated": deleted_cache_entries,
+                "debounce_ms": int((monotonic() - pending.first_seen_monotonic) * 1000),
+                "total_latency_ms": payload["total_latency_ms"],
+            },
+        )
+
+    @staticmethod
+    def _total_latency_ms(pending: PendingEarthDbChange, flushed_at: datetime) -> int | None:
+        occurred_at = pending.first_occurred_at
+        if not occurred_at:
+            return None
+        try:
+            normalized = occurred_at.replace("Z", "+00:00")
+            occurred = datetime.fromisoformat(normalized)
+            if occurred.tzinfo is None:
+                occurred = occurred.replace(tzinfo=UTC)
+            return max(0, int((flushed_at - occurred.astimezone(UTC)).total_seconds() * 1000))
+        except ValueError:
+            return None
+
+    async def flush_all(self) -> None:
+        sources = list(self._pending)
+        for source in sources:
+            task = self._flush_tasks.pop(source, None)
+            if task and not task.done():
+                task.cancel()
+            await self.flush_source(source)
+
+    async def stop(self) -> None:
+        tasks = [task for task in self._flush_tasks.values() if not task.done()]
+        for task in tasks:
+            task.cancel()
+        if tasks:
+            await asyncio.gather(*tasks, return_exceptions=True)
+        self._flush_tasks.clear()
+        await self.flush_all()
+
+
+class EarthDbChangeListener:
+    def __init__(
+        self,
+        *,
+        dsn: str,
+        dispatcher: EarthDbChangeDispatcher,
+        channel: str = EARTH_DATA_CHANGES_CHANNEL,
+    ) -> None:
+        self._dsn = normalize_asyncpg_dsn(dsn)
+        self._dispatcher = dispatcher
+        self._channel = channel
+        self._task: asyncio.Task[None] | None = None
+        self._stop_event: asyncio.Event | None = None
+        self._connection: asyncpg.Connection | None = None
+        self._loop: asyncio.AbstractEventLoop | None = None
+
+    def start(self) -> None:
+        if self._task and not self._task.done():
+            return
+        self._loop = asyncio.get_running_loop()
+        self._stop_event = asyncio.Event()
+        self._task = asyncio.create_task(self._run())
+
+    async def stop(self) -> None:
+        if self._stop_event:
+            self._stop_event.set()
+        if self._connection:
+            await self._connection.close()
+        if self._task:
+            await asyncio.gather(self._task, return_exceptions=True)
+        await self._dispatcher.stop()
+
+    async def _run(self) -> None:
+        backoff_seconds = 1.0
+        assert self._stop_event is not None
+
+        while not self._stop_event.is_set():
+            try:
+                self._connection = await asyncpg.connect(self._dsn)
+                await self._connection.add_listener(self._channel, self._on_notification)
+                logger.info_event(
+                    "Earth database change listener connected",
+                    event="earth.db_changes.connected",
+                    context={"channel": self._channel},
+                )
+                backoff_seconds = 1.0
+                while not self._stop_event.is_set():
+                    try:
+                        await asyncio.wait_for(
+                            self._stop_event.wait(),
+                            timeout=LISTEN_KEEPALIVE_SECONDS,
+                        )
+                    except asyncio.TimeoutError:
+                        await self._poll_outbox()
+            except asyncio.CancelledError:
+                raise
+            except Exception as exc:
+                logger.exception_event(
+                    "Earth database change listener failed",
+                    event="earth.db_changes.listener_failed",
+                    context={"channel": self._channel, "error": str(exc)},
+                )
+                try:
+                    await asyncio.wait_for(self._stop_event.wait(), timeout=backoff_seconds)
+                except asyncio.TimeoutError:
+                    pass
+                backoff_seconds = min(backoff_seconds * 2, 30.0)
+            finally:
+                if self._connection:
+                    try:
+                        await self._connection.remove_listener(self._channel, self._on_notification)
+                    except Exception:
+                        pass
+                    try:
+                        await self._connection.close()
+                    except Exception:
+                        pass
+                    self._connection = None
+
+    async def _poll_outbox(self) -> None:
+        if self._connection is None:
+            return
+
+        rows = await self._connection.fetch(
+            """
+            SELECT id, payload
+            FROM earth_data_change_events
+            WHERE consumed_at IS NULL
+            ORDER BY id
+            LIMIT $1
+            """,
+            OUTBOX_POLL_LIMIT,
+        )
+        accepted_count = 0
+        consumed_ids: list[int] = []
+        for row in rows:
+            payload = row["payload"]
+            if isinstance(payload, str):
+                try:
+                    payload = json.loads(payload)
+                except json.JSONDecodeError:
+                    consumed_ids.append(int(row["id"]))
+                    continue
+            if isinstance(payload, dict):
+                if self._dispatcher.handle_payload(payload):
+                    accepted_count += 1
+                consumed_ids.append(int(row["id"]))
+            else:
+                consumed_ids.append(int(row["id"]))
+        if consumed_ids:
+            await self._dispatcher.flush_all()
+        if consumed_ids:
+            await self._connection.execute(
+                """
+                UPDATE earth_data_change_events
+                SET consumed_at = NOW()
+                WHERE id = ANY($1::bigint[])
+                  AND consumed_at IS NULL
+                """,
+                consumed_ids,
+            )
+        if rows:
+            logger.info_event(
+                "Polled Earth database change outbox",
+                event="earth.db_changes.outbox_polled",
+                context={"events": len(rows), "accepted": accepted_count},
+            )
+
+    def _on_notification(
+        self,
+        _connection: asyncpg.Connection,
+        _pid: int,
+        _channel: str,
+        payload: str,
+    ) -> None:
+        if self._loop and self._loop.is_running():
+            self._loop.call_soon_threadsafe(self._dispatcher.handle_notification, payload)
+            return
+        self._dispatcher.handle_notification(payload)
+
+
+_dispatcher = EarthDbChangeDispatcher(
+    broadcast_earth_update=broadcaster.broadcast_earth_update,
+    invalidate_cache=invalidate_earth_layer_cache_for_source,
+)
+_listener: EarthDbChangeListener | None = None
+
+
+def start_earth_db_change_listener() -> None:
+    global _listener
+    if _listener is not None:
+        return
+    _listener = EarthDbChangeListener(dsn=settings.DATABASE_URL, dispatcher=_dispatcher)
+    _listener.start()
+
+
+async def stop_earth_db_change_listener() -> None:
+    global _listener
+    if _listener is None:
+        await _dispatcher.stop()
+        return
+    listener = _listener
+    _listener = None
+    await listener.stop()

backend/app/services/earth_interactables.py

@@ -0,0 +1,113 @@
+from __future__ import annotations
+
+from datetime import UTC, datetime
+from typing import Any
+from uuid import uuid4
+
+from sqlalchemy import select
+from sqlalchemy.ext.asyncio import AsyncSession
+
+from app.core.time import to_iso8601_utc
+from app.core.websocket.broadcaster import broadcaster
+from app.models.earth_interactable import EarthInteractable
+from app.services.earth_layer_cache import EARTH_LAYER_CACHE_PREFIX, earth_layer_cache
+
+INTERACTABLE_ENTITY = "interactable"
+INTERACTABLE_LAYER = "interactables"
+
+
+def normalize_interactable_id(value: str | None = None) -> str:
+    raw = str(value or "").strip()
+    return raw or f"interactable-{uuid4().hex}"
+
+
+def serialize_interactable(record: EarthInteractable) -> dict[str, Any]:
+    return {
+        "id": record.id,
+        "layer": record.layer,
+        "kind": record.kind,
+        "label": record.label,
+        "description": record.description,
+        "latitude": record.latitude,
+        "longitude": record.longitude,
+        "altitude": record.altitude,
+        "revision": record.revision,
+        "properties": record.properties or {},
+        "is_deleted": bool(record.is_deleted),
+        "created_at": to_iso8601_utc(record.created_at),
+        "updated_at": to_iso8601_utc(record.updated_at),
+        "deleted_at": to_iso8601_utc(record.deleted_at),
+    }
+
+
+def interactables_to_geojson(items: list[EarthInteractable]) -> dict[str, Any]:
+    return {
+        "type": "FeatureCollection",
+        "features": [
+            {
+                "type": "Feature",
+                "id": item.id,
+                "geometry": {
+                    "type": "Point",
+                    "coordinates": [item.longitude, item.latitude],
+                },
+                "properties": serialize_interactable(item),
+            }
+            for item in items
+            if not item.is_deleted
+        ],
+    }
+
+
+def invalidate_interactable_cache(layer: str | None = None) -> int:
+    layer_key = str(layer or "*").strip() or "*"
+    deleted = earth_layer_cache.delete_pattern(
+        f"{EARTH_LAYER_CACHE_PREFIX}:interactables:layer:{layer_key}*"
+    )
+    if layer_key != "all":
+        deleted += earth_layer_cache.delete_pattern(
+            f"{EARTH_LAYER_CACHE_PREFIX}:interactables:layer:all*"
+        )
+    return deleted
+
+
+def build_interactable_event(
+    *,
+    action: str,
+    record: EarthInteractable,
+    include_item: bool = True,
+) -> dict[str, Any]:
+    item = serialize_interactable(record)
+    return {
+        "entity": INTERACTABLE_ENTITY,
+        "action": action,
+        "layer": record.layer,
+        "layers": [INTERACTABLE_LAYER],
+        "ids": [record.id],
+        "revision": record.revision,
+        "changed_at": item["deleted_at"] or item["updated_at"] or to_iso8601_utc(datetime.now(UTC)),
+        "item": item if include_item else None,
+        "source": "earth_interactables",
+    }
+
+
+async def publish_interactable_event(action: str, record: EarthInteractable, *, include_item: bool = True) -> None:
+    await broadcaster.broadcast_earth_update(
+        build_interactable_event(action=action, record=record, include_item=include_item)
+    )
+
+
+async def list_interactables(
+    db: AsyncSession,
+    *,
+    layer: str | None = None,
+    include_deleted: bool = False,
+) -> list[EarthInteractable]:
+    stmt = select(EarthInteractable)
+    if layer:
+        stmt = stmt.where(EarthInteractable.layer == layer)
+    if not include_deleted:
+        stmt = stmt.where(EarthInteractable.is_deleted.is_(False))
+    stmt = stmt.order_by(EarthInteractable.updated_at.desc(), EarthInteractable.id.asc())
+    result = await db.execute(stmt)
+    return list(result.scalars().all())

backend/app/services/earth_layer_adapters.py

@@ -0,0 +1,187 @@
+"""Earth layer adapter registry for datasource-backed refresh behavior."""
+
+from __future__ import annotations
+
+from dataclasses import dataclass, field
+from typing import Any
+
+from sqlalchemy.ext.asyncio import AsyncSession
+
+
+@dataclass(frozen=True)
+class EarthLayerAdapter:
+    sources: frozenset[str]
+    layers: tuple[str, ...]
+    cache_patterns: tuple[str, ...]
+    tables: frozenset[str] = field(default_factory=frozenset)
+    derived_models: tuple[str, ...] = field(default_factory=tuple)
+    refresh_strategy: str = "clear_then_reload"
+
+
+EARTH_LAYER_ADAPTERS: tuple[EarthLayerAdapter, ...] = (
+    EarthLayerAdapter(
+        sources=frozenset({"barentswatch_vessels", "aisstream_vessels", "vessel_static", "vessel_position", "ais_raw_observations", "ais_source_health"}),
+        tables=frozenset({"vessel_static", "vessel_position", "ais_raw_observations", "ais_source_health"}),
+        layers=("vessels",),
+        cache_patterns=("vessels*", "summary*"),
+        derived_models=("ais_raw_observations", "ais_conflict_records", "ais_source_health"),
+    ),
+    EarthLayerAdapter(
+        sources=frozenset(
+            {
+                "telegeography_cables",
+                "telegeography_landing",
+                "telegeography_landing_points",
+                "telegeography_systems",
+                "telegeography_cable_systems",
+                "arcgis_cables",
+                "arcgis_landing_points",
+                "arcgis_cable_landing_relation",
+                "arcgis_cable_landing_relations",
+                "fao_landing_points",
+            }
+        ),
+        tables=frozenset({"collected_data"}),
+        layers=("cables",),
+        cache_patterns=("cables*", "landing-points*", "summary*"),
+    ),
+    EarthLayerAdapter(
+        sources=frozenset({"celestrak_tle", "spacetrack_tle"}),
+        tables=frozenset({"collected_data"}),
+        layers=("satellites",),
+        cache_patterns=("satellites*", "summary*"),
+    ),
+    EarthLayerAdapter(
+        sources=frozenset(
+            {
+                "top500",
+                "top500_supercomputers",
+                "epoch_ai_gpu",
+                "huggingface_models",
+                "huggingface_datasets",
+                "huggingface_spaces",
+                "compute_center_locations",
+            }
+        ),
+        tables=frozenset({"compute_center_locations"}),
+        layers=("computeCenters",),
+        cache_patterns=("compute-centers*", "summary*"),
+        refresh_strategy="reload",
+    ),
+    EarthLayerAdapter(
+        sources=frozenset(
+            {
+                "ris_live_bgp",
+                "bgpstream_bgp",
+                "iptoasn_prefix_geo",
+                "opengeofeed_prefix_geo",
+                "nro_delegated_prefix_geo",
+                "bgp_observations",
+                "bgp_anomalies",
+                "bgp_incidents",
+                "bgp_collector_locations",
+            }
+        ),
+        tables=frozenset({"bgp_observations", "bgp_anomalies", "bgp_incidents", "bgp_collector_locations"}),
+        layers=("bgp",),
+        cache_patterns=("bgp*", "summary*"),
+        derived_models=("bgp_observations", "bgp_anomalies", "bgp_incidents"),
+    ),
+    EarthLayerAdapter(
+        sources=frozenset({"news_live_streams"}),
+        tables=frozenset({"collected_data"}),
+        layers=("media",),
+        cache_patterns=("summary*",),
+        refresh_strategy="reload",
+    ),
+    EarthLayerAdapter(
+        sources=frozenset({"media_news_archive", "earth_news_items"}),
+        tables=frozenset({"earth_news_items"}),
+        layers=("news",),
+        cache_patterns=("summary*",),
+        refresh_strategy="reload",
+    ),
+    EarthLayerAdapter(
+        sources=frozenset({"earth_interactables"}),
+        tables=frozenset({"earth_interactables"}),
+        layers=("interactables",),
+        cache_patterns=("interactables*", "summary*"),
+        refresh_strategy="delta",
+    ),
+)
+
+_ADAPTERS_BY_SOURCE = {
+    source: adapter
+    for adapter in EARTH_LAYER_ADAPTERS
+    for source in adapter.sources
+}
+_ADAPTERS_BY_TABLE = {
+    table: adapter
+    for adapter in EARTH_LAYER_ADAPTERS
+    for table in adapter.tables
+}
+
+
+def get_earth_layer_adapter_for_source(source: str | None) -> EarthLayerAdapter | None:
+    return _ADAPTERS_BY_SOURCE.get(str(source or "").strip())
+
+
+def get_earth_layer_adapter_for_change(table: str | None, source: str | None) -> EarthLayerAdapter | None:
+    table_key = str(table or "").strip()
+    source_key = str(source or "").strip()
+    if table_key and table_key != "collected_data":
+        adapter = _ADAPTERS_BY_TABLE.get(table_key)
+        if adapter is not None:
+            return adapter
+    return get_earth_layer_adapter_for_source(source_key)
+
+
+def get_earth_update_layers_for_source(source: str | None) -> list[str]:
+    adapter = get_earth_layer_adapter_for_source(source)
+    return list(adapter.layers) if adapter else []
+
+
+def get_earth_update_layers_for_change(table: str | None, source: str | None) -> list[str]:
+    adapter = get_earth_layer_adapter_for_change(table, source)
+    return list(adapter.layers) if adapter else []
+
+
+def get_earth_refresh_strategy_for_change(table: str | None, source: str | None) -> str | None:
+    adapter = get_earth_layer_adapter_for_change(table, source)
+    return adapter.refresh_strategy if adapter else None
+
+
+def get_earth_cache_patterns_for_source(source: str | None) -> list[str]:
+    adapter = get_earth_layer_adapter_for_source(source)
+    return list(adapter.cache_patterns) if adapter else []
+
+
+async def clear_derived_datasource_data(db: AsyncSession, source: str) -> dict[str, int]:
+    adapter = get_earth_layer_adapter_for_source(source)
+    if adapter is None or not adapter.derived_models:
+        return {}
+
+    from app.models.bgp_anomaly import BGPAnomaly
+    from app.models.bgp_incident import BGPIncident
+    from app.models.bgp_observation import BGPObservation
+    from app.models.vessel import AISConflictRecord, AISRawObservation, AISSourceHealth
+
+    model_by_key: dict[str, Any] = {
+        "bgp_observations": BGPObservation,
+        "bgp_anomalies": BGPAnomaly,
+        "bgp_incidents": BGPIncident,
+        "ais_raw_observations": AISRawObservation,
+        "ais_conflict_records": AISConflictRecord,
+        "ais_source_health": AISSourceHealth,
+    }
+    deleted_counts: dict[str, int] = {}
+    for key in adapter.derived_models:
+        model = model_by_key.get(key)
+        if model is None:
+            continue
+        if key == "ais_conflict_records":
+            result = await db.execute(model.__table__.delete().where(model.selected_source == source))
+        else:
+            result = await db.execute(model.__table__.delete().where(model.source == source))
+        deleted_counts[key] = int(result.rowcount or 0)
+    return deleted_counts

backend/app/services/earth_layer_cache.py

@@ -268,34 +268,10 @@ def apply_payload_budget(payload: dict[str, Any], policy: EarthLayerCachePolicy)
 
 
 def invalidate_earth_layer_cache_for_source(source: str) -> int:
+    from app.services.earth_layer_adapters import get_earth_cache_patterns_for_source
+
     source_key = str(source or "").strip()
-    patterns = {
-        "barentswatch_vessels": ["vessels*", "summary*"],
-        "aisstream_vessels": ["vessels*", "summary*"],
-        "telegeography_cables": ["cables*", "landing-points*", "summary*"],
-        "telegeography_landing": ["landing-points*", "summary*"],
-        "telegeography_landing_points": ["landing-points*", "summary*"],
-        "telegeography_systems": ["cables*", "summary*"],
-        "telegeography_cable_systems": ["cables*", "summary*"],
-        "arcgis_cables": ["cables*", "landing-points*", "summary*"],
-        "arcgis_landing_points": ["landing-points*", "summary*"],
-        "arcgis_cable_landing_relation": ["landing-points*", "summary*"],
-        "arcgis_cable_landing_relations": ["landing-points*", "summary*"],
-        "fao_landing_points": ["landing-points*", "summary*"],
-        "celestrak_tle": ["satellites*", "summary*"],
-        "spacetrack_tle": ["satellites*", "summary*"],
-        "top500": ["compute-centers*", "summary*"],
-        "top500_supercomputers": ["compute-centers*", "summary*"],
-        "epoch_ai_gpu": ["compute-centers*", "summary*"],
-        "huggingface_models": ["compute-centers*", "summary*"],
-        "huggingface_datasets": ["compute-centers*", "summary*"],
-        "huggingface_spaces": ["compute-centers*", "summary*"],
-        "ris_live_bgp": ["bgp*", "summary*"],
-        "bgpstream_bgp": ["bgp*", "summary*"],
-        "iptoasn_prefix_geo": ["bgp*", "summary*"],
-        "opengeofeed_prefix_geo": ["bgp*", "summary*"],
-        "nro_delegated_prefix_geo": ["bgp*", "summary*"],
-    }.get(source_key, [])
+    patterns = get_earth_cache_patterns_for_source(source_key)
     deleted = 0
     for layer_pattern in patterns:
         deleted += earth_layer_cache.delete_pattern(f"{EARTH_LAYER_CACHE_PREFIX}:{layer_pattern}")

backend/app/services/earth_news.py

@@ -1063,6 +1063,7 @@ def _build_payload(
     lon: float | None,
     active_region: str,
     items: list[ParsedNewsItem],
+    cruise_items: list[ParsedNewsItem] | None = None,
     sources: list[NewsFeedSource],
     errors: list[str],
     stale: bool,
@@ -1082,6 +1083,10 @@ def _build_payload(
         },
         "sources": _serialize_sources(sources),
         "items": [_serialize_item(item, active_region=active_region) for item in items],
+        "cruise_items": [
+            _serialize_item(item, active_region=active_region)
+            for item in (cruise_items if cruise_items is not None else items)
+        ],
         "errors": errors,
         "stale": stale,
     }
@@ -1295,6 +1300,7 @@ async def get_earth_news_payload(
 
     from app.services.earth_news_store import (
         get_earth_news_freshness,
+        list_earth_news_cruise_items,
         list_earth_news_items,
         upsert_earth_news_items,
     )
@@ -1312,6 +1318,13 @@ async def get_earth_news_payload(
         active_region=active_region,
         limit=MAX_ITEMS_TOTAL,
     )
+    if hasattr(db, "execute"):
+        cruise_items = await list_earth_news_cruise_items(
+            db,
+            limit=MAX_ITEMS_TOTAL * len(REGION_ANCHORS),
+        )
+    else:
+        cruise_items = items
     await _enqueue_unverified_locations(items)
     stale = bool(errors and items)
 
@@ -1320,6 +1333,7 @@ async def get_earth_news_payload(
         lon=lon,
         active_region=active_region,
         items=items,
+        cruise_items=cruise_items,
         sources=sources,
         errors=errors,
         stale=stale,

backend/app/services/earth_news_store.py

@@ -78,6 +78,24 @@ async def list_earth_news_items(
     return [record_to_parsed_news_item(record) for record in result.scalars().all()]
 
 
+async def list_earth_news_cruise_items(
+    db: AsyncSession,
+    *,
+    limit: int,
+) -> list[ParsedNewsItem]:
+    result = await db.execute(
+        select(EarthNewsItem)
+        .order_by(
+            EarthNewsItem.region.asc(),
+            EarthNewsItem.published_at.desc().nullslast(),
+            EarthNewsItem.last_seen_at.desc(),
+            EarthNewsItem.feed_name.asc(),
+        )
+        .limit(limit)
+    )
+    return [record_to_parsed_news_item(record) for record in result.scalars().all()]
+
+
 async def get_earth_news_freshness(
     db: AsyncSession,
     *,

backend/app/services/log_tail.py

@@ -0,0 +1,161 @@
+from __future__ import annotations
+
+import asyncio
+from dataclasses import dataclass, field
+from datetime import UTC, datetime
+from typing import Any
+
+from fastapi import WebSocket
+
+from app.db.session import async_session_factory
+from app.services.system_logs import (
+    DEFAULT_LOG_LINE_LIMIT,
+    LOG_SOURCES,
+    MAX_LOG_LINE_LIMIT,
+    read_database_log_events,
+    read_log_events,
+)
+
+DATABASE_LOG_SOURCE_IDS = {"system-db", "audit-db"}
+LOG_TAIL_CHANNEL = "logs_tail"
+LOG_TAIL_INTERVAL_SECONDS = 1.5
+LOG_TAIL_SCAN_MULTIPLIER = 5
+
+
+@dataclass(frozen=True)
+class LogTailConfig:
+    source_id: str
+    limit: int = DEFAULT_LOG_LINE_LIMIT
+    level: str = "all"
+    levels: str | None = None
+    start_date: str | None = None
+    end_date: str | None = None
+    search: str | None = None
+
+
+@dataclass
+class LogTailSubscription:
+    config: LogTailConfig
+    emitted_cursors: set[str] = field(default_factory=set)
+    task: asyncio.Task | None = None
+
+
+class LogTailManager:
+    def __init__(self) -> None:
+        self._subscriptions: dict[WebSocket, LogTailSubscription] = {}
+
+    def normalize_config(self, payload: dict[str, Any]) -> LogTailConfig:
+        source_id = str(payload.get("source_id") or payload.get("source") or "").strip()
+        if not source_id:
+            raise ValueError("source_id is required")
+        if source_id not in LOG_SOURCES and source_id not in DATABASE_LOG_SOURCE_IDS:
+            raise ValueError("Log source not found")
+        try:
+            limit = int(payload.get("limit") or DEFAULT_LOG_LINE_LIMIT)
+        except (TypeError, ValueError) as exc:
+            raise ValueError("limit must be a number") from exc
+        if limit < 1 or limit > MAX_LOG_LINE_LIMIT:
+            raise ValueError(f"limit must be between 1 and {MAX_LOG_LINE_LIMIT}")
+        return LogTailConfig(
+            source_id=source_id,
+            limit=limit,
+            level=str(payload.get("level") or "all"),
+            levels=str(payload.get("levels")).strip() if payload.get("levels") else None,
+            start_date=str(payload.get("start_date")).strip() if payload.get("start_date") else None,
+            end_date=str(payload.get("end_date")).strip() if payload.get("end_date") else None,
+            search=str(payload.get("search")).strip() if payload.get("search") else None,
+        )
+
+    async def subscribe(self, websocket: WebSocket, payload: dict[str, Any]) -> LogTailConfig:
+        config = self.normalize_config(payload)
+        await self.unsubscribe(websocket)
+        subscription = LogTailSubscription(config=config)
+        subscription.task = asyncio.create_task(self._run_tail(websocket, subscription))
+        self._subscriptions[websocket] = subscription
+        return config
+
+    async def unsubscribe(self, websocket: WebSocket) -> None:
+        subscription = self._subscriptions.pop(websocket, None)
+        if subscription and subscription.task:
+            subscription.task.cancel()
+            try:
+                await subscription.task
+            except asyncio.CancelledError:
+                pass
+
+    async def disconnect(self, websocket: WebSocket) -> None:
+        await self.unsubscribe(websocket)
+
+    async def _run_tail(self, websocket: WebSocket, subscription: LogTailSubscription) -> None:
+        first_frame = True
+        while True:
+            events = await self._read_events(subscription.config)
+            if first_frame:
+                visible_events = events[-subscription.config.limit :]
+                subscription.emitted_cursors.update(event.cursor for event in visible_events)
+                await self._send_frame(websocket, subscription.config, "snapshot", visible_events)
+                first_frame = False
+            else:
+                new_events = [
+                    event
+                    for event in events
+                    if event.cursor not in subscription.emitted_cursors
+                ]
+                if new_events:
+                    visible_events = new_events[-subscription.config.limit :]
+                    subscription.emitted_cursors.update(event.cursor for event in visible_events)
+                    await self._send_frame(websocket, subscription.config, "append", visible_events)
+            await asyncio.sleep(LOG_TAIL_INTERVAL_SECONDS)
+
+    async def _read_events(self, config: LogTailConfig):
+        scan_limit = max(config.limit * LOG_TAIL_SCAN_MULTIPLIER, config.limit)
+        if config.source_id in DATABASE_LOG_SOURCE_IDS:
+            async with async_session_factory() as db:
+                events = await read_database_log_events(
+                    config.source_id,
+                    scan_limit=scan_limit,
+                    level=config.level,
+                    levels=config.levels,
+                    start_date=config.start_date,
+                    end_date=config.end_date,
+                    search=config.search,
+                    db=db,
+                )
+                return events or []
+        events = read_log_events(
+            config.source_id,
+            scan_limit=scan_limit,
+            level=config.level,
+            levels=config.levels,
+            start_date=config.start_date,
+            end_date=config.end_date,
+            search=config.search,
+        )
+        return events or []
+
+    async def _send_frame(self, websocket: WebSocket, config: LogTailConfig, mode: str, events) -> None:
+        await websocket.send_json(
+            {
+                "type": "data_frame",
+                "channel": LOG_TAIL_CHANNEL,
+                "timestamp": datetime.now(UTC).isoformat(),
+                "payload": {
+                    "mode": mode,
+                    "source_id": config.source_id,
+                    "line_count": len(events),
+                    "lines": [event.line for event in events],
+                    "filters": {
+                        "limit": config.limit,
+                        "level": config.level,
+                        "levels": config.levels,
+                        "start_date": config.start_date,
+                        "end_date": config.end_date,
+                        "search": config.search,
+                    },
+                    "status": "ok",
+                },
+            }
+        )
+
+
+log_tail_manager = LogTailManager()

backend/app/services/playground_chat_service.py

@@ -10,6 +10,7 @@ from fastapi import HTTPException, status
 from sqlalchemy import func, select, update
 from sqlalchemy.ext.asyncio import AsyncSession
 
+from app.core.logging import get_logger
 from app.db.session import async_session_factory
 from app.models.playground_message import PlaygroundMessage
 from app.models.playground_session import PlaygroundSession
@@ -27,9 +28,11 @@ from app.schemas.ai import (
     SituationalAnalysisRequest,
 )
 from app.services.ai_client import AIProviderClient
+from app.services.business_logs import emit_business_log, exception_context
 from app.services.playground_session_store import _to_response as session_to_response
 from app.services.playground_session_store import upsert_playground_session
 
+logger = get_logger(__name__, service="ai")
 STREAM_CHUNK_SIZE = 24
 STREAM_INTERVAL_SECONDS = 0.08
 THINKING_PREVIEW_SECONDS = 2.6
@@ -624,7 +627,42 @@ async def _run_assistant_message(
                 thinking={"type": "enabled"},
             )
 
+        await emit_business_log(
+            logger,
+            event="ai.playground.run.start",
+            message="Playground AI run started",
+            category="ai",
+            service="ai",
+            module=__name__,
+            request_id=request_id,
+            user_id=user_id,
+            context={
+                "session_id": session_id,
+                "session_key": session_key,
+                "user_message_id": user_message_id,
+                "assistant_message_id": assistant_message_id,
+                "preset": payload.selected_preset_key,
+            },
+        )
         analysis = await provider_client.analyze(request_payload, request_id=request_id)
+        await emit_business_log(
+            logger,
+            event="ai.playground.run.success",
+            message="Playground AI run completed",
+            category="ai",
+            service="ai",
+            module=__name__,
+            request_id=request_id,
+            user_id=user_id,
+            context={
+                "session_id": session_id,
+                "session_key": session_key,
+                "provider": analysis.provider,
+                "model": analysis.model,
+                "content_block_count": len(analysis.content_blocks or []),
+                "thinking_block_count": len(analysis.thinking_blocks or []),
+            },
+        )
 
         async with async_session_factory() as db:
             assistant_message = await _mark_message_state(
@@ -704,6 +742,23 @@ async def _run_assistant_message(
                 await db.flush()
             await db.commit()
     except asyncio.CancelledError:
+        await emit_business_log(
+            logger,
+            event="ai.playground.run.cancelled",
+            message="Playground AI run cancelled",
+            category="ai",
+            level="warning",
+            service="ai",
+            module=__name__,
+            request_id=request_id,
+            user_id=user_id,
+            context={
+                "session_id": session_id,
+                "session_key": session_key,
+                "assistant_message_id": assistant_message_id,
+                "duration_ms": round((perf_counter() - started_at) * 1000),
+            },
+        )
         async with async_session_factory() as db:
             result = await db.execute(select(PlaygroundMessage).where(PlaygroundMessage.id == assistant_message_id))
             message = result.scalar_one_or_none()
@@ -715,6 +770,26 @@ async def _run_assistant_message(
                 await db.commit()
         raise
     except Exception as exc:
+        await emit_business_log(
+            logger,
+            event="ai.playground.run.failed",
+            message="Playground AI run failed",
+            category="ai",
+            level="error",
+            service="ai",
+            module=__name__,
+            request_id=request_id,
+            user_id=user_id,
+            context=exception_context(
+                exc,
+                {
+                    "session_id": session_id,
+                    "session_key": session_key,
+                    "assistant_message_id": assistant_message_id,
+                    "duration_ms": round((perf_counter() - started_at) * 1000),
+                },
+            ),
+        )
         error_message = _format_run_exception(exc)
         async with async_session_factory() as db:
             result = await db.execute(select(PlaygroundMessage).where(PlaygroundMessage.id == assistant_message_id))

backend/app/services/scheduler.py

@@ -13,6 +13,7 @@ from app.db.session import async_session_factory
 from app.core.time import to_iso8601_utc
 from app.models.datasource import DataSource
 from app.models.task import CollectionTask
+from app.services.business_logs import emit_business_log, emit_business_log_background, exception_context
 from app.services.collectors.registry import collector_registry
 from app.services.datasource_connectivity import (
     build_builtin_connectivity_checksum,
@@ -124,6 +125,15 @@ async def run_collector_task(collector_name: str):
                 event="collector.run.skipped_disabled",
                 context={"collector_name": collector_name},
             )
+            await emit_business_log(
+                logger,
+                event="collector.run.skipped_disabled",
+                message="Skipping disabled collector",
+                category="collector",
+                service="scheduler",
+                module=__name__,
+                context={"collector_name": collector_name, "datasource_id": datasource.id, "status": "skipped"},
+            )
             return
 
         running_result = await db.execute(
@@ -152,6 +162,21 @@ async def run_collector_task(collector_name: str):
                     event="collector.run.skipped_already_running",
                     context={"collector_name": collector_name, "task_id": existing_running.id},
                 )
+                await emit_business_log(
+                    logger,
+                    event="collector.run.skipped_already_running",
+                    message="Skipping collector trigger because task is already running",
+                    category="collector",
+                    level="warning",
+                    service="scheduler",
+                    module=__name__,
+                    context={
+                        "collector_name": collector_name,
+                        "datasource_id": datasource.id,
+                        "task_id": existing_running.id,
+                        "status": "skipped",
+                    },
+                )
                 return
 
             existing_error = (existing_running.error_message or "").strip()
@@ -173,6 +198,21 @@ async def run_collector_task(collector_name: str):
                 event="collector.run.stale_task_failed",
                 context={"collector_name": collector_name, "task_id": existing_running.id},
             )
+            await emit_business_log(
+                logger,
+                event="collector.run.stale_task_failed",
+                message="Marked stale running task as failed before rerun",
+                category="collector",
+                level="warning",
+                service="scheduler",
+                module=__name__,
+                context={
+                    "collector_name": collector_name,
+                    "datasource_id": datasource.id,
+                    "task_id": existing_running.id,
+                    "status": "failed",
+                },
+            )
 
         try:
             datasource_id = datasource.id
@@ -183,6 +223,15 @@ async def run_collector_task(collector_name: str):
                 event="collector.run.started",
                 context={"collector_name": collector_name, "datasource_id": datasource_id},
             )
+            await emit_business_log(
+                logger,
+                event="collector.run.scheduled_started",
+                message="Scheduler started collector run",
+                category="collector",
+                service="scheduler",
+                module=__name__,
+                context={"collector_name": collector_name, "datasource_id": datasource_id, "status": "running"},
+            )
             task_result = await collector.run(db)
             datasource = await db.get(DataSource, datasource_id)
             if datasource is None:
@@ -217,6 +266,20 @@ async def run_collector_task(collector_name: str):
                 event="collector.run.completed",
                 context={"collector_name": collector_name, "datasource_id": datasource_id, "result": task_result},
             )
+            await emit_business_log(
+                logger,
+                event="collector.run.scheduled_completed",
+                message="Scheduler completed collector run",
+                category="collector",
+                service="scheduler",
+                module=__name__,
+                context={
+                    "collector_name": collector_name,
+                    "datasource_id": datasource_id,
+                    "status": task_result.get("status"),
+                    "result": task_result,
+                },
+            )
         except asyncio.CancelledError:
             await db.rollback()
             datasource = await db.get(DataSource, datasource_id)
@@ -228,6 +291,16 @@ async def run_collector_task(collector_name: str):
                 event="collector.run.cancelled",
                 context={"collector_name": collector_name, "datasource_id": datasource.id},
             )
+            await emit_business_log(
+                logger,
+                event="collector.run.cancelled",
+                message="Collector cancelled by operator",
+                category="collector",
+                level="warning",
+                service="scheduler",
+                module=__name__,
+                context={"collector_name": collector_name, "datasource_id": datasource.id, "status": "cancelled"},
+            )
             raise
         except Exception as exc:
             await db.rollback()
@@ -240,6 +313,19 @@ async def run_collector_task(collector_name: str):
                 event="collector.run.failed",
                 context={"collector_name": collector_name, "datasource_id": datasource.id, "error": str(exc)},
             )
+            await emit_business_log(
+                logger,
+                event="collector.run.failed",
+                message="Collector failed",
+                category="collector",
+                level="error",
+                service="scheduler",
+                module=__name__,
+                context=exception_context(
+                    exc,
+                    {"collector_name": collector_name, "datasource_id": datasource.id, "status": "failed"},
+                ),
+            )
 
 
 async def cleanup_stale_running_tasks(max_age_hours: int = 2) -> int:
@@ -361,6 +447,16 @@ def run_collector_now(collector_name: str) -> bool:
             event="collector.trigger.skipped_already_running",
             context={"collector_name": collector_name},
         )
+        emit_business_log_background(
+            logger,
+            event="collector.trigger.skipped_already_running",
+            message="Collector is already running in-memory; skipping duplicate trigger",
+            category="collector",
+            level="warning",
+            service="scheduler",
+            module=__name__,
+            context={"collector_name": collector_name, "status": "skipped"},
+        )
         return False
 
     try:
@@ -378,6 +474,15 @@ def run_collector_now(collector_name: str) -> bool:
             event="collector.trigger.started",
             context={"collector_name": collector_name},
         )
+        emit_business_log_background(
+            logger,
+            event="collector.trigger.started",
+            message="Triggered collector",
+            category="collector",
+            service="scheduler",
+            module=__name__,
+            context={"collector_name": collector_name, "status": "queued"},
+        )
         return True
     except Exception as exc:
         logger.error_event(
@@ -385,6 +490,16 @@ def run_collector_now(collector_name: str) -> bool:
             event="collector.trigger.failed",
             context={"collector_name": collector_name, "error": str(exc)},
         )
+        emit_business_log_background(
+            logger,
+            event="collector.trigger.failed",
+            message="Failed to trigger collector",
+            category="collector",
+            level="error",
+            service="scheduler",
+            module=__name__,
+            context=exception_context(exc, {"collector_name": collector_name, "status": "failed"}),
+        )
         return False
 
 

backend/app/services/system_logs.py

@@ -5,6 +5,7 @@ import os
 import re
 import shutil
 import subprocess
+import hashlib
 
 from collections import Counter, deque
 from dataclasses import dataclass
@@ -13,6 +14,9 @@ from pathlib import Path
 from typing import Any
 
 from app.core.security import redis_client
+from app.models.system_log import AuditLog, SystemLog
+from sqlalchemy import select
+from sqlalchemy.ext.asyncio import AsyncSession
 
 DEFAULT_LOG_LINE_LIMIT = 200
 MAX_LOG_LINE_LIMIT = 1000
@@ -99,6 +103,16 @@ class StructuredLogEntry:
     search_text: str
 
 
+@dataclass(frozen=True)
+class LogEvent:
+    source_id: str
+    cursor: str
+    timestamp: datetime | None
+    level: str | None
+    line: str
+    search_text: str
+
+
 @dataclass
 class DailyLogMarker:
     date_token: str
@@ -135,7 +149,7 @@ LOG_SOURCES: dict[str, LogSource] = {
         name="前端开发服务",
         kind="file",
         location=_state_log_path("frontend.log"),
-        description="控制台与 Earth 前端开发服务输出。",
+        description="控制台与智能星球前端开发服务输出。",
         category="service",
         fallback_locations=("/tmp/planet_frontend.log",),
     ),
@@ -150,13 +164,22 @@ LOG_SOURCES: dict[str, LogSource] = {
     ),
     "earth-client": LogSource(
         source_id="earth-client",
-        name="Earth 浏览器端",
+        name="智能星球浏览器端",
         kind="buffer",
         location="redis://planet:system_logs:earth-client",
-        description="Earth 浏览器端上报的运行时错误与关键业务日志。",
+        description="智能星球浏览器端上报的运行时错误与关键业务日志。",
         category="client",
         buffer_key=f"{LOG_BUFFER_KEY_PREFIX}:earth-client",
     ),
+    "admin-client": LogSource(
+        source_id="admin-client",
+        name="控制台浏览器端",
+        kind="buffer",
+        location="redis://planet:system_logs:admin-client",
+        description="控制台浏览器端上报的运行时错误。",
+        category="client",
+        buffer_key=f"{LOG_BUFFER_KEY_PREFIX}:admin-client",
+    ),
 }
 
 
@@ -365,6 +388,44 @@ def build_buffer_entry(payload: dict[str, Any]) -> StructuredLogEntry:
     )
 
 
+def compact_log_context(context: dict | None) -> str:
+    if not context:
+        return ""
+    allowed = {
+        key: value
+        for key, value in (context or {}).items()
+        if key
+        in {
+            "status",
+            "duration_ms",
+            "provider",
+            "model",
+            "result_provider",
+            "result_model",
+            "collector_name",
+            "datasource_id",
+            "task_id",
+            "snapshot_id",
+            "raw_count",
+            "transformed_count",
+            "saved_count",
+            "created",
+            "updated",
+            "unchanged",
+            "deleted",
+            "result_count",
+            "status_code",
+            "error_type",
+            "error",
+            "route",
+            "module",
+        }
+    }
+    if not allowed:
+        return ""
+    return json.dumps(allowed, ensure_ascii=False, sort_keys=True)
+
+
 def read_file_entries(source: LogSource, scan_limit: int) -> list[StructuredLogEntry]:
     path = resolve_file_log_path(source)
     if not path.exists():
@@ -437,6 +498,176 @@ def read_source_entries(source: LogSource, scan_limit: int) -> list[StructuredLo
     return []
 
 
+def _database_event_from_system_record(record: SystemLog) -> LogEvent:
+    record_level = normalize_log_level(record.level)
+    line = " ".join(
+        part
+        for part in [
+            record.occurred_at.isoformat() if record.occurred_at else "",
+            record_level.upper(),
+            record.source,
+            record.category or "",
+            record.event or "",
+            f"request_id={record.request_id}" if record.request_id else "",
+            record.message,
+            compact_log_context(record.context),
+        ]
+        if part
+    )
+    search_text = " ".join(
+        [
+            line,
+            f"id={record.id}",
+            f"user_id={record.user_id}" if record.user_id else "",
+            json.dumps(record.context or {}, ensure_ascii=False, sort_keys=True),
+        ]
+    ).lower()
+    return LogEvent(
+        source_id="system-db",
+        cursor=f"system-db:{record.id}",
+        timestamp=record.occurred_at,
+        level=None if record_level == LOG_LEVEL_ALL else record_level,
+        line=line,
+        search_text=search_text,
+    )
+
+
+def _database_event_from_audit_record(record: AuditLog) -> LogEvent:
+    line = " ".join(
+        part
+        for part in [
+            record.occurred_at.isoformat() if record.occurred_at else "",
+            "INFO",
+            record.action,
+            record.target_type or "",
+            record.target_id or "",
+            record.result or "",
+            f"request_id={record.request_id}" if record.request_id else "",
+        ]
+        if part
+    )
+    search_text = " ".join(
+        [
+            line,
+            f"id={record.id}",
+            f"actor_id={record.actor_id}" if record.actor_id else "",
+            record.actor_name or "",
+            json.dumps(record.details or {}, ensure_ascii=False, sort_keys=True),
+        ]
+    ).lower()
+    return LogEvent(
+        source_id="audit-db",
+        cursor=f"audit-db:{record.id}",
+        timestamp=record.occurred_at,
+        level=LOG_LEVEL_INFO,
+        line=line,
+        search_text=search_text,
+    )
+
+
+async def read_database_log_events(
+    source_id: str,
+    *,
+    scan_limit: int,
+    level: str = LOG_LEVEL_ALL,
+    levels: str | None = None,
+    start_date: str | None = None,
+    end_date: str | None = None,
+    search: str | None = None,
+    db: AsyncSession,
+) -> list[LogEvent] | None:
+    selected_levels = normalize_log_levels(level, levels)
+    search_query = (search or "").strip()
+    if source_id == "system-db":
+        query = select(SystemLog).order_by(SystemLog.occurred_at.desc().nullslast(), SystemLog.id.desc()).limit(scan_limit)
+        result = await db.execute(query)
+        events = [_database_event_from_system_record(record) for record in result.scalars().all()]
+    elif source_id == "audit-db":
+        query = select(AuditLog).order_by(AuditLog.occurred_at.desc().nullslast(), AuditLog.id.desc()).limit(scan_limit)
+        result = await db.execute(query)
+        events = [_database_event_from_audit_record(record) for record in result.scalars().all()]
+    else:
+        return None
+
+    events = list(reversed(events))
+    return [
+        event
+        for event in events
+        if event_matches_levels(event, selected_levels)
+        and event_matches_search(event, search_query)
+        and event_matches_date_range(event, start_date, end_date)
+    ]
+
+
+async def read_database_log_snapshot(
+    source_id: str,
+    *,
+    limit: int,
+    level: str,
+    levels: str | None,
+    start_date: str | None,
+    end_date: str | None,
+    search: str | None,
+    db: AsyncSession,
+) -> dict[str, Any] | None:
+    events = await read_database_log_events(
+        source_id,
+        scan_limit=limit * 5,
+        level=level,
+        levels=levels,
+        start_date=start_date,
+        end_date=end_date,
+        search=search,
+        db=db,
+    )
+    if events is None:
+        return None
+    visible_events = events[-limit:]
+    selected_levels = normalize_log_levels(level, levels)
+    return {
+        "source_id": source_id,
+        "name": "系统事件" if source_id == "system-db" else "审计事件",
+        "kind": "database",
+        "location": "table://system_logs" if source_id == "system-db" else "table://audit_logs",
+        "description": "数据库持久化日志",
+        "category": "database" if source_id == "system-db" else "audit",
+        "status": "ok" if visible_events else "empty",
+        "level": level,
+        "selected_levels": list(selected_levels),
+        "search_query": search or "",
+        "available_levels": ["all", "error", "warning", "info", "debug"],
+        "daily_markers": build_daily_log_markers_from_events(events),
+        "line_limit": limit,
+        "line_count": len(visible_events),
+        "lines": [event.line for event in visible_events],
+    }
+
+
+def _stable_hash(value: str) -> str:
+    return hashlib.sha1(value.encode("utf-8", errors="replace")).hexdigest()[:16]
+
+
+def build_log_events(source_id: str, entries: list[StructuredLogEntry]) -> list[LogEvent]:
+    events: list[LogEvent] = []
+    seen: dict[str, int] = {}
+    for entry in entries:
+        stable_value = entry.raw_line or entry.display_line
+        digest = _stable_hash(stable_value)
+        occurrence = seen.get(digest, 0) + 1
+        seen[digest] = occurrence
+        events.append(
+            LogEvent(
+                source_id=source_id,
+                cursor=f"{source_id}:{digest}:{occurrence}",
+                timestamp=entry.timestamp,
+                level=entry.level,
+                line=entry.display_line,
+                search_text=entry.search_text,
+            )
+        )
+    return events
+
+
 def matches_levels(entry: StructuredLogEntry, selected_levels: tuple[str, ...]) -> bool:
     if not selected_levels:
         return True
@@ -469,6 +700,34 @@ def matches_search(entry: StructuredLogEntry, search: str | None) -> bool:
     return query in entry.search_text
 
 
+def event_matches_levels(event: LogEvent, selected_levels: tuple[str, ...]) -> bool:
+    if not selected_levels:
+        return True
+    return event.level in selected_levels
+
+
+def event_matches_date_range(event: LogEvent, start_date: str | None, end_date: str | None) -> bool:
+    if not start_date and not end_date:
+        return True
+    if event.timestamp is None:
+        return False
+    date_token = event.timestamp.astimezone(UTC).date().isoformat()
+    if start_date and date_token < start_date:
+        return False
+    if end_date and date_token > end_date:
+        return False
+    return True
+
+
+def event_matches_search(event: LogEvent, search: str | None) -> bool:
+    if search is None:
+        return True
+    query = search.strip().lower()
+    if not query:
+        return True
+    return query in event.search_text
+
+
 def build_daily_log_markers(entries: list[StructuredLogEntry]) -> list[dict[str, Any]]:
     grouped: dict[str, list[StructuredLogEntry]] = {}
     for entry in entries:
@@ -503,6 +762,69 @@ def build_daily_log_markers(entries: list[StructuredLogEntry]) -> list[dict[str,
     return [marker.__dict__ for marker in markers]
 
 
+def build_daily_log_markers_from_events(events: list[LogEvent]) -> list[dict[str, Any]]:
+    grouped: dict[str, list[LogEvent]] = {}
+    for event in events:
+        if event.timestamp is None:
+            continue
+        date_token = event.timestamp.astimezone(UTC).date().isoformat()
+        grouped.setdefault(date_token, []).append(event)
+
+    markers: list[DailyLogMarker] = []
+    for date_token, group in sorted(grouped.items()):
+        level_counts = Counter(
+            event.level
+            for event in group
+            if event.level in SUPPORTED_LOG_LEVELS and event.level != LOG_LEVEL_ALL
+        )
+        dominant_level = LOG_LEVEL_INFO
+        if level_counts:
+            dominant_level = sorted(
+                level_counts.items(),
+                key=lambda item: (
+                    -item[1],
+                    ("error", "warning", "info", "debug").index(item[0]),
+                ),
+            )[0][0]
+        markers.append(
+            DailyLogMarker(
+                date_token=date_token,
+                total=len(group),
+                dominant_level=dominant_level,
+            )
+        )
+    return [marker.__dict__ for marker in markers]
+
+
+def read_log_events(
+    source_id: str,
+    scan_limit: int,
+    *,
+    level: str = LOG_LEVEL_ALL,
+    levels: str | None = None,
+    start_date: str | None = None,
+    end_date: str | None = None,
+    search: str | None = None,
+) -> list[LogEvent] | None:
+    source = LOG_SOURCES.get(source_id)
+    if source is None:
+        return None
+
+    selected_levels = normalize_log_levels(level, levels)
+    search_query = (search or "").strip()
+    events = build_log_events(source_id, read_source_entries(source, scan_limit))
+    marker_events = [
+        event
+        for event in events
+        if event_matches_levels(event, selected_levels) and event_matches_search(event, search_query)
+    ]
+    return [
+        event
+        for event in marker_events
+        if event_matches_date_range(event, start_date, end_date)
+    ]
+
+
 def read_log_snapshot(
     source_id: str,
     limit: int,
@@ -520,18 +842,18 @@ def read_log_snapshot(
     selected_levels = normalize_log_levels(level, levels)
     search_query = (search or "").strip()
     scan_limit = max(min(MAX_LOG_LINE_LIMIT * 5, 5000), limit * 5, BUFFER_LOG_LIMIT if source.kind == "buffer" else 1000)
-    all_entries = read_source_entries(source, scan_limit)
-    marker_entries = [
-        entry
-        for entry in all_entries
-        if matches_levels(entry, selected_levels) and matches_search(entry, search_query)
+    all_events = build_log_events(source_id, read_source_entries(source, scan_limit))
+    marker_events = [
+        event
+        for event in all_events
+        if event_matches_levels(event, selected_levels) and event_matches_search(event, search_query)
     ]
-    filtered_entries = [
-        entry
-        for entry in marker_entries
-        if matches_date_range(entry, start_date, end_date)
+    filtered_events = [
+        event
+        for event in marker_events
+        if event_matches_date_range(event, start_date, end_date)
     ]
-    visible_entries = filtered_entries[-limit:]
+    visible_events = filtered_events[-limit:]
 
     compatibility_level = selected_levels[0] if len(selected_levels) == 1 else LOG_LEVEL_ALL
     return {
@@ -552,8 +874,8 @@ def read_log_snapshot(
             LOG_LEVEL_INFO,
             LOG_LEVEL_DEBUG,
         ],
-        "daily_markers": build_daily_log_markers(marker_entries),
+        "daily_markers": build_daily_log_markers_from_events(marker_events),
         "line_limit": limit,
-        "line_count": len(visible_entries),
-        "lines": [entry.display_line for entry in visible_entries],
+        "line_count": len(visible_events),
+        "lines": [event.line for event in visible_events],
     }

backend/tests/test_ai_observability.py

@@ -0,0 +1,128 @@
+import pytest
+from fastapi import HTTPException
+
+from app.schemas.ai import SituationalAnalysisRequest
+from app.services.ai_tools import web_search as web_search_module
+from app.services.ai_tools.schemas import SearchEvidence, WebSearchConfig, WebSearchProviderConfig
+from app.services.ai_tools.web_search import WebSearchClient
+from app.services import ai_client as ai_client_module
+from app.services.ai_client import AIProviderClient
+
+
+@pytest.mark.asyncio
+async def test_ai_client_analyze_logs_summary_without_prompt(monkeypatch):
+    events = []
+
+    async def fake_emit_business_log(_logger, **payload):
+        events.append(payload)
+
+    async def fake_request(self, method, path, json=None, request_id=None, operation="request", payload_summary=None):
+        return {
+            "provider": "test-provider",
+            "model": "test-model",
+            "content": "ok",
+            "content_blocks": [],
+            "text_blocks": ["ok"],
+            "thinking_blocks": [],
+            "raw_response": {},
+        }
+
+    monkeypatch.setattr(ai_client_module, "emit_business_log", fake_emit_business_log)
+    monkeypatch.setattr(AIProviderClient, "_request", fake_request)
+
+    client = AIProviderClient(
+        service_url="http://provider.test",
+        llm_config={"provider": "openai", "provider_api": "openai-completions", "model": "gpt-test", "api_key": "sk-secret"},
+    )
+    result = await client.analyze(
+        SituationalAnalysisRequest(
+            title="Sensitive title",
+            objective="Do not store this full prompt",
+            observations=["secret observation"],
+            constraints=["secret constraint"],
+            context={"source": "test", "private": "value"},
+        ),
+        request_id="req-ai-test",
+    )
+
+    assert result.model == "test-model"
+    assert [event["event"] for event in events] == [
+        "ai.provider.analyze.start",
+        "ai.provider.analyze.success",
+    ]
+    serialized = str(events)
+    assert "Do not store this full prompt" not in serialized
+    assert "secret observation" not in serialized
+    assert "sk-secret" not in serialized
+    start_context = events[0]["context"]
+    assert start_context["model"] == "gpt-test"
+    assert start_context["input_summary"]["objective_length"] == len("Do not store this full prompt")
+    assert start_context["input_summary"]["observation_count"] == 1
+    assert start_context["input_summary"]["context_keys"] == ["private", "source"]
+
+
+@pytest.mark.asyncio
+async def test_ai_client_analyze_logs_failure(monkeypatch):
+    events = []
+
+    async def fake_emit_business_log(_logger, **payload):
+        events.append(payload)
+
+    async def fake_request(self, method, path, json=None, request_id=None, operation="request", payload_summary=None):
+        raise HTTPException(status_code=502, detail="provider failed")
+
+    monkeypatch.setattr(ai_client_module, "emit_business_log", fake_emit_business_log)
+    monkeypatch.setattr(AIProviderClient, "_request", fake_request)
+
+    client = AIProviderClient(service_url="http://provider.test", llm_config={"provider": "openai", "model": "gpt-test"})
+
+    with pytest.raises(HTTPException):
+        await client.analyze(
+            SituationalAnalysisRequest(title="T", objective="O", observations=["one"]),
+            request_id="req-ai-fail",
+        )
+
+    assert events[-1]["event"] == "ai.provider.analyze.failed"
+    assert events[-1]["level"] == "error"
+    assert events[-1]["context"]["error_type"] == "HTTPException"
+
+
+@pytest.mark.asyncio
+async def test_web_search_logs_query_hash_without_query(monkeypatch):
+    events = []
+
+    async def fake_emit_business_log(_logger, **payload):
+        events.append(payload)
+
+    async def fake_search_tavily(self, config, query, max_results, domains, freshness_days):
+        return [
+            SearchEvidence(
+                title="Example",
+                url="https://example.test",
+                snippet="result",
+                source_provider="tavily",
+            )
+        ]
+
+    monkeypatch.setattr(web_search_module, "emit_business_log", fake_emit_business_log)
+    monkeypatch.setattr(WebSearchClient, "_search_tavily", fake_search_tavily)
+
+    client = WebSearchClient(
+        WebSearchConfig(
+            enabled=True,
+            default_provider="tavily",
+            providers={"tavily": WebSearchProviderConfig(provider="tavily", api_key="secret-key")},
+        )
+    )
+    results = await client.search("secret query text", max_results=1)
+
+    assert len(results) == 1
+    assert [event["event"] for event in events] == [
+        "ai_tool.web_search.start",
+        "ai_tool.web_search.success",
+    ]
+    serialized = str(events)
+    assert "secret query text" not in serialized
+    assert "secret-key" not in serialized
+    assert events[0]["context"]["query_length"] == len("secret query text")
+    assert events[1]["context"]["result_count"] == 1

backend/tests/test_api.py

@@ -672,6 +672,39 @@ async def test_ingest_earth_client_log_accepts_public_events():
         app.dependency_overrides.clear()
 
 
+@pytest.mark.asyncio
+async def test_ingest_admin_client_log_accepts_public_events():
+    transport = ASGITransport(app=app)
+    try:
+        with patch("app.api.v1.system_control.append_buffer_log") as mock_append_buffer_log:
+            with patch("app.api.v1.system_control.record_system_log", new_callable=AsyncMock) as mock_record_system_log:
+                async with AsyncClient(transport=transport, base_url="http://test") as client:
+                    response = await client.post(
+                        "/api/v1/system/logs/admin-client",
+                        json={
+                            "level": "error",
+                            "message": "控制台发生未处理 Promise 错误",
+                            "category": "unhandledrejection",
+                            "module": "admin",
+                            "url": "http://test/logs",
+                            "detail": "stack preview",
+                        },
+                    )
+        assert response.status_code == 200
+        data = response.json()
+        assert data["accepted"] is True
+        assert data["source_id"] == "admin-client"
+        mock_append_buffer_log.assert_called_once()
+        mock_record_system_log.assert_awaited_once()
+        persisted_kwargs = mock_record_system_log.await_args.kwargs
+        assert persisted_kwargs["source"] == "admin-client"
+        assert persisted_kwargs["event"] == "admin.client.runtime_log"
+        assert persisted_kwargs["category"] == "unhandledrejection"
+        assert persisted_kwargs["context"]["url"] == "http://test/logs"
+    finally:
+        app.dependency_overrides.clear()
+
+
 @pytest.mark.asyncio
 async def test_earth_layer_cache_status_requires_super_admin(auth_headers, monkeypatch):
     def override_get_current_user():

backend/tests/test_collectors.py

@@ -1,9 +1,13 @@
 """Unit tests for data collectors"""
 
+import json
+
 import pytest
 from unittest.mock import AsyncMock, patch
 
 from app.core.datasource_defaults import DEFAULT_DATASOURCES
+from app.services.collectors.celestrak import CelesTrakTLECollector
+from app.services.collectors.downloads import DownloadHTTPStatusError, ResumableFileDownloader
 from app.services.credential_guides import DEFAULT_CREDENTIAL_GUIDES
 from app.services.collectors.top500 import TOP500Collector
 from app.services.collectors.registry import collector_registry
@@ -149,6 +153,172 @@ class TestHTTPCollector:
         assert callable(collector.parse_response)
 
 
+class TestCelesTrakTLECollector:
+    def test_transform_uses_norad_as_source_id_and_preserves_starlink_group(self):
+        collector = CelesTrakTLECollector()
+        result = collector.transform([
+            {
+                "NORAD_CAT_ID": 44720,
+                "OBJECT_NAME": "STARLINK-1000",
+                "OBJECT_ID": "2019-029AZ",
+                "EPOCH": "2026-03-13T00:00:00Z",
+                "MEAN_MOTION": 15.79234567,
+                "ECCENTRICITY": 0.0001234,
+                "INCLINATION": 53.0,
+                "RA_OF_ASC_NODE": 10.0,
+                "ARG_OF_PERICENTER": 20.0,
+                "MEAN_ANOMALY": 30.0,
+                "_celestrak_query_group": "active",
+                "_celestrak_source_url": "https://celestrak.example/gp.php?GROUP=active&FORMAT=json",
+            }
+        ])
+
+        assert result[0]["source_id"] == "44720"
+        assert result[0]["metadata"]["constellation_group"] == "starlink"
+        assert result[0]["metadata"]["celestrak_query_group"] == "active"
+        assert result[0]["metadata"]["norad_cat_id"] == 44720
+        assert result[0]["metadata"]["tle_line1"]
+        assert result[0]["metadata"]["tle_line2"]
+
+    def test_load_active_payload_rejects_invalid_records(self, tmp_path):
+        collector = CelesTrakTLECollector()
+        payload_path = tmp_path / "active.json"
+        payload_path.write_text(json.dumps([{"OBJECT_NAME": "missing norad"}]), encoding="utf-8")
+
+        with pytest.raises(RuntimeError, match="invalid record"):
+            collector._load_active_payload(payload_path)
+
+    def test_load_active_payload_accepts_complete_array(self, tmp_path):
+        collector = CelesTrakTLECollector()
+        payload_path = tmp_path / "active.json"
+        payload_path.write_text(
+            json.dumps([{"NORAD_CAT_ID": 25544, "OBJECT_NAME": "ISS (ZARYA)"}]),
+            encoding="utf-8",
+        )
+
+        records = collector._load_active_payload(payload_path)
+
+        assert records == [{"NORAD_CAT_ID": 25544, "OBJECT_NAME": "ISS (ZARYA)"}]
+
+    @pytest.mark.asyncio
+    async def test_fetch_retries_and_raises_instead_of_returning_partial_data(self, monkeypatch, tmp_path):
+        collector = CelesTrakTLECollector()
+        collector._resolved_url = "https://celestrak.example/NORAD/elements/gp.php"
+        attempts = 0
+
+        async def fake_download_file(*args, **kwargs):
+            nonlocal attempts
+            attempts += 1
+            raise RuntimeError("network interrupted")
+
+        async def fake_emit_business_log(*args, **kwargs):
+            return None
+
+        monkeypatch.setattr(collector._downloader, "download_file", fake_download_file)
+        monkeypatch.setattr("app.services.collectors.celestrak.emit_business_log", fake_emit_business_log)
+        monkeypatch.setattr("app.services.collectors.celestrak.asyncio.sleep", AsyncMock())
+
+        with pytest.raises(RuntimeError, match="failed after retries"):
+            await collector.fetch()
+
+        assert attempts == 3
+
+    @pytest.mark.asyncio
+    async def test_fetch_uses_cache_when_celestrak_reports_not_updated(self, monkeypatch, tmp_path):
+        collector = CelesTrakTLECollector()
+        collector._resolved_url = "https://celestrak.example/NORAD/elements/gp.php"
+        collector._downloader = ResumableFileDownloader(cache_namespace="celestrak-test", cache_root=tmp_path)
+        url = collector._active_url()
+        cached_path = collector._downloader.cached_file_path(url, ".json")
+        cached_path.parent.mkdir(parents=True, exist_ok=True)
+        cached_path.write_text(
+            json.dumps([{"NORAD_CAT_ID": 25544, "OBJECT_NAME": "ISS (ZARYA)"}]),
+            encoding="utf-8",
+        )
+
+        async def fake_download_file(*args, **kwargs):
+            raise DownloadHTTPStatusError(
+                url=url,
+                status_code=403,
+                body="GP data has not updated since your last successful download of GROUP=active.",
+            )
+
+        async def fake_emit_business_log(*args, **kwargs):
+            return None
+
+        monkeypatch.setattr(collector._downloader, "download_file", fake_download_file)
+        monkeypatch.setattr("app.services.collectors.celestrak.emit_business_log", fake_emit_business_log)
+
+        records = await collector.fetch()
+
+        assert records[0]["NORAD_CAT_ID"] == 25544
+        assert records[0]["_celestrak_query_group"] == "active"
+
+    @pytest.mark.asyncio
+    async def test_fetch_not_updated_without_cache_does_not_retry(self, monkeypatch, tmp_path):
+        collector = CelesTrakTLECollector()
+        collector._resolved_url = "https://celestrak.example/NORAD/elements/gp.php"
+        collector._downloader = ResumableFileDownloader(cache_namespace="celestrak-test", cache_root=tmp_path)
+        attempts = 0
+
+        async def fake_download_file(*args, **kwargs):
+            nonlocal attempts
+            attempts += 1
+            raise DownloadHTTPStatusError(
+                url=collector._active_url(),
+                status_code=403,
+                body="GP data has not updated since your last successful download of GROUP=active.",
+            )
+
+        async def fake_emit_business_log(*args, **kwargs):
+            return None
+
+        monkeypatch.setattr(collector._downloader, "download_file", fake_download_file)
+        monkeypatch.setattr("app.services.collectors.celestrak.FALLBACK_GROUPS", ("starlink",))
+        monkeypatch.setattr("app.services.collectors.celestrak.emit_business_log", fake_emit_business_log)
+        monkeypatch.setattr("app.services.collectors.celestrak.asyncio.sleep", AsyncMock())
+
+        with pytest.raises(RuntimeError, match="fallback group mode failed"):
+            await collector.fetch()
+
+        assert attempts == 2
+
+    @pytest.mark.asyncio
+    async def test_fetch_falls_back_to_all_groups_when_active_not_updated_without_cache(self, monkeypatch, tmp_path):
+        collector = CelesTrakTLECollector()
+        collector._resolved_url = "https://celestrak.example/NORAD/elements/gp.php"
+        collector._downloader = ResumableFileDownloader(cache_namespace="celestrak-test", cache_root=tmp_path)
+        payload_by_group = {
+            "starlink": [{"NORAD_CAT_ID": 100, "OBJECT_NAME": "STARLINK-100"}],
+            "gps-ops": [{"NORAD_CAT_ID": 200, "OBJECT_NAME": "GPS BIIR-2"}],
+        }
+
+        async def fake_download_file(_client, url, **_kwargs):
+            if "GROUP=active" in url:
+                raise DownloadHTTPStatusError(
+                    url=url,
+                    status_code=403,
+                    body="GP data has not updated since your last successful download of GROUP=active.",
+                )
+            group = "starlink" if "GROUP=starlink" in url else "gps-ops"
+            path = tmp_path / f"{group}.json"
+            path.write_text(json.dumps(payload_by_group[group]), encoding="utf-8")
+            return path
+
+        async def fake_emit_business_log(*args, **kwargs):
+            return None
+
+        monkeypatch.setattr(collector._downloader, "download_file", fake_download_file)
+        monkeypatch.setattr("app.services.collectors.celestrak.FALLBACK_GROUPS", tuple(payload_by_group))
+        monkeypatch.setattr("app.services.collectors.celestrak.emit_business_log", fake_emit_business_log)
+
+        records = await collector.fetch()
+
+        assert [item["NORAD_CAT_ID"] for item in records] == [100, 200]
+        assert records[0]["_celestrak_query_group"] == "starlink"
+        assert records[1]["_celestrak_group"] == "gps-ops"
+
+
 def test_aisstream_collector_is_registered():
     collector = collector_registry.get("aisstream_vessels")
 

backend/tests/test_datasources_batch.py

@@ -1,9 +1,11 @@
+import asyncio
+import inspect
 from datetime import datetime, timedelta, timezone
 
-import pytest
-
 from app.api.v1 import datasources as datasources_api
 from app.models.datasource import DataSource
+from app.models.task import CollectionTask
+from app.services import data_jobs
 from app.services import earth_layer_cache as earth_cache
 
 
@@ -100,6 +102,71 @@ def test_serialize_datasource_row_includes_endpoint_when_requested():
     assert row["endpoint"] == "https://example.test/arcgis_landing_points"
 
 
+def test_serialize_datasource_row_separates_collector_running_from_delete_task():
+    datasource = make_datasource(9, "top500", last_status="success")
+    task = CollectionTask(
+        id=44,
+        datasource_id=9,
+        source="top500",
+        task_type="clear_data",
+        status="running",
+        phase="clearing_data",
+    )
+
+    row = datasources_api.serialize_datasource_row(
+        datasource,
+        running_tasks={9: task},
+        latest_tasks={9: task},
+        record_counts={"top500": 500},
+        endpoint_overrides={},
+        config=object(),
+        include_endpoint=False,
+    )
+
+    assert row["is_task_active"] is True
+    assert row["is_running"] is False
+    assert row["task_type"] == "clear_data"
+    assert row["task_status"] == "running"
+
+
+def test_cancel_queued_delete_task_finishes_immediately(monkeypatch):
+    task = CollectionTask(
+        id=45,
+        datasource_id=9,
+        source="top500",
+        task_type="clear_data",
+        status="queued",
+        phase="queued",
+    )
+
+    class FakeDb:
+        async def commit(self):
+            return None
+
+        async def refresh(self, _task):
+            return None
+
+    async def fake_broadcast(_task):
+        return None
+
+    monkeypatch.setattr(data_jobs, "_broadcast_task_update", fake_broadcast)
+
+    async def run():
+        return await data_jobs.request_cancel_datasource_task(FakeDb(), task)
+
+    result = asyncio.run(run())
+
+    assert result.status == "cancelled"
+    assert result.phase == "cancelled"
+    assert result.completed_at is not None
+
+
+def test_clear_data_job_relies_on_db_outbox_instead_of_extra_earth_refresh_task():
+    source = inspect.getsource(data_jobs._run_clear_data_job)
+
+    assert "enqueue_earth_refresh_job" not in source
+
+
 def test_invalidate_earth_layer_cache_for_source_covers_datasource_aliases(monkeypatch):
     patterns: list[str] = []
 
@@ -119,34 +186,36 @@ def test_invalidate_earth_layer_cache_for_source_covers_datasource_aliases(monke
     ]
 
 
-@pytest.mark.asyncio
-async def test_trigger_datasource_batch_skips_disabled_and_frequency_window(monkeypatch):
-    now = datetime.now(timezone.utc)
-    disabled = make_datasource(1, "aisstream_vessels", is_active=False)
-    not_due = make_datasource(2, "telegeography_cables", last_run_at=now, frequency_minutes=120)
-    due = make_datasource(3, "ris_live_bgp", last_run_at=now - timedelta(hours=2))
-    triggered_sources: list[str] = []
+def test_trigger_datasource_batch_skips_disabled_and_frequency_window(monkeypatch):
+    async def run():
+        now = datetime.now(timezone.utc)
+        disabled = make_datasource(1, "aisstream_vessels", is_active=False)
+        not_due = make_datasource(2, "telegeography_cables", last_run_at=now, frequency_minutes=120)
+        due = make_datasource(3, "ris_live_bgp", last_run_at=now - timedelta(hours=2))
+        queued_sources: list[str] = []
 
-    async def fake_running_tasks(_db, _ids):
-        return {}
+        async def fake_running_tasks(_db, _ids):
+            return {}
 
-    async def fake_latest_task_ids(_db, _ids):
-        return {}
+        async def fake_enqueue(_db, datasource, task_type, **_kwargs):
+            queued_sources.append(datasource.source)
+            return CollectionTask(id=100 + datasource.id, datasource_id=datasource.id, source=datasource.source, task_type=task_type, status="queued")
 
-    monkeypatch.setattr(datasources_api, "_load_latest_running_tasks", fake_running_tasks)
-    monkeypatch.setattr(datasources_api, "_load_latest_task_ids", fake_latest_task_ids)
-    monkeypatch.setattr(
-        datasources_api,
-        "run_collector_now",
-        lambda source: triggered_sources.append(source) or True,
-    )
+        monkeypatch.setattr(datasources_api, "_load_latest_running_tasks", fake_running_tasks)
+        monkeypatch.setattr(
+            datasources_api,
+            "enqueue_datasource_job",
+            fake_enqueue,
+        )
 
-    result = await datasources_api._trigger_datasource_batch(
-        object(),
-        [disabled, not_due, due],
-        force=False,
-    )
+        result = await datasources_api._trigger_datasource_batch(
+            object(),
+            [disabled, not_due, due],
+            force=False,
+        )
 
-    assert [item["source"] for item in result["triggered"]] == ["ris_live_bgp"]
-    assert {item["reason"] for item in result["skipped"]} == {"disabled", "within_frequency_window"}
-    assert triggered_sources == ["ris_live_bgp"]
+        assert [item["source"] for item in result["triggered"]] == ["ris_live_bgp"]
+        assert {item["reason"] for item in result["skipped"]} == {"disabled", "within_frequency_window"}
+        assert queued_sources == ["ris_live_bgp"]
+
+    asyncio.run(run())

backend/tests/test_docs_gatekeeper.py

@@ -1,11 +1,15 @@
 """Docs Gatekeeper API tests."""
 
+import re
+from pathlib import Path
+
 import pytest
 from httpx import ASGITransport, AsyncClient
 
 from app.api.v1 import docs as docs_api
 from app.main import app
 from app.models.user import User
+from app.services.docs_gatekeeper import DOCS_METADATA
 
 
 def make_user(role: str = "viewer", groups: list[str] | None = None) -> User:
@@ -43,22 +47,45 @@ async def test_public_catalog_only_for_anonymous_user():
     assert response.status_code == 200
     items = response.json()["items"]
     assert {item["access"] for item in items} == {"public"}
-    assert {item["slug"] for item in items if item["lang"] == "zh"} == {
+    zh_items = [item for item in items if item["lang"] == "zh"]
+    assert [item["slug"] for item in zh_items] == [
         "overview",
-        "quickstart",
         "manual",
+        "quickstart",
         "faq",
-        "location-pipeline-user",
-    }
+    ]
+
+
+@pytest.mark.asyncio
+async def test_developer_catalog_includes_architecture_and_frontend_reference_docs():
+    response = await get_json(
+        "/api/v1/docs/catalog",
+        make_user(role="viewer", groups=["docs_developer"]),
+    )
+
+    assert response.status_code == 200
+    zh_slugs = {item["slug"] for item in response.json()["items"] if item["lang"] == "zh"}
+    zh_items = [item for item in response.json()["items"] if item["lang"] == "zh"]
+    assert [item["group"] for item in zh_items[:4]] == ["Overview", "Manual", "Manual", "Manual"]
+    assert zh_items[4]["group"] == "Architecture"
+    assert "platform-data-flows" in zh_slugs
+    assert "naming-glossary" in zh_slugs
+    assert "tactile-ui-components" in zh_slugs
 
 
 @pytest.mark.asyncio
 async def test_anonymous_can_read_public_doc():
     response = await get_json("/api/v1/docs/zh/quickstart")
+    manual_response = await get_json("/api/v1/docs/zh/manual")
+    overview_response = await get_json("/api/v1/docs/zh/overview")
 
     assert response.status_code == 200
     assert response.json()["access"] == "public"
     assert "快速开始" in response.json()["markdown"]
+    assert manual_response.status_code == 200
+    assert manual_response.json()["access"] == "public"
+    assert overview_response.status_code == 200
+    assert overview_response.json()["access"] == "public"
 
 
 @pytest.mark.asyncio
@@ -83,10 +110,16 @@ async def test_developer_group_can_read_developer_but_not_admin_doc():
     user = make_user(role="viewer", groups=["docs_developer"])
 
     developer_response = await get_json("/api/v1/docs/zh/backend-collectors", user)
+    tactile_response = await get_json("/api/v1/docs/zh/tactile-ui-components", user)
+    glossary_response = await get_json("/api/v1/docs/zh/naming-glossary", user)
     admin_response = await get_json("/api/v1/docs/zh/backend-system-service-control", user)
 
     assert developer_response.status_code == 200
     assert developer_response.json()["access"] == "docs_developer"
+    assert tactile_response.status_code == 200
+    assert tactile_response.json()["access"] == "docs_developer"
+    assert glossary_response.status_code == 200
+    assert glossary_response.json()["access"] == "docs_developer"
     assert admin_response.status_code == 403
 
 
@@ -114,3 +147,29 @@ async def test_unknown_language_slug_and_path_traversal_do_not_read_files():
     assert bad_lang.status_code == 404
     assert bad_slug.status_code == 404
     assert traversal.status_code == 404
+
+
+def test_public_docs_markdown_links_do_not_create_missing_docs_routes():
+    repo_root = Path(__file__).resolve().parents[2]
+    technical_root = repo_root / "docs" / "technical"
+    registered_filenames = {entry.filename for entry in DOCS_METADATA}
+    problems: list[str] = []
+
+    for markdown_path in sorted(technical_root.glob("*/*.md")):
+        markdown = markdown_path.read_text(encoding="utf-8")
+        for match in re.finditer(r"\[([^\]]+)]\(([^)]+\.md(?:#[^)]+)?)\)", markdown):
+            label, href = match.group(1), match.group(2)
+            if href.startswith(("http://", "https://", "mailto:")):
+                continue
+
+            href_without_hash = href.split("#", 1)[0].replace("\\", "/")
+            filename = Path(href_without_hash).name
+            if "/docs/technical/" in href_without_hash:
+                if filename not in registered_filenames:
+                    problems.append(f"{markdown_path.relative_to(repo_root)} links unregistered public doc {href!r} ({label})")
+                continue
+
+            if href_without_hash.endswith(".md"):
+                problems.append(f"{markdown_path.relative_to(repo_root)} links non-public markdown {href!r} ({label})")
+
+    assert problems == []

backend/tests/test_earth_db_change_listener.py

@@ -0,0 +1,454 @@
+import asyncio
+import json
+
+from app.services.earth_db_change_listener import (
+    EarthDbChangeDispatcher,
+    EarthDbChangeListener,
+    build_earth_update_from_db_payload,
+    normalize_asyncpg_dsn,
+)
+
+
+def test_normalize_asyncpg_dsn_strips_sqlalchemy_driver():
+    assert (
+        normalize_asyncpg_dsn("postgresql+asyncpg://postgres:postgres@localhost:5432/planet_db")
+        == "postgresql://postgres:postgres@localhost:5432/planet_db"
+    )
+
+
+def test_build_earth_update_maps_known_sources_to_layers():
+    satellite_update = build_earth_update_from_db_payload(
+        {"source": "celestrak_tle", "operation": "DELETE", "entity_key": "25544"}
+    )
+    cable_update = build_earth_update_from_db_payload(
+        {"source": "arcgis_cables", "operation": "UPDATE", "entity_key": "cable-1"}
+    )
+    compute_update = build_earth_update_from_db_payload(
+        {"source": "top500", "operation": "DELETE", "entity_key": None}
+    )
+    landing_update = build_earth_update_from_db_payload(
+        {"source": "telegeography_landing", "operation": "DELETE", "entity_key": None}
+    )
+
+    assert satellite_update is not None
+    assert satellite_update["layers"] == ["satellites"]
+    assert cable_update is not None
+    assert cable_update["layers"] == ["cables"]
+    assert compute_update is not None
+    assert compute_update["layers"] == ["computeCenters"]
+    assert landing_update is not None
+    assert landing_update["layers"] == ["cables"]
+    assert landing_update["refresh_strategy"] == "clear_then_reload"
+
+
+def test_build_earth_update_maps_derived_tables_to_layers():
+    bgp_update = build_earth_update_from_db_payload(
+        {"table": "bgp_anomalies", "source": "ris_live_bgp", "operation": "DELETE", "entity_key": "a1"}
+    )
+    compute_update = build_earth_update_from_db_payload(
+        {"table": "compute_center_locations", "source": "top500", "operation": "UPDATE", "entity_key": "top500_1"}
+    )
+    vessel_update = build_earth_update_from_db_payload(
+        {"table": "vessel_position", "operation": "DELETE", "entity_key": "123456789"}
+    )
+
+    assert bgp_update is not None
+    assert bgp_update["source"] == "ris_live_bgp"
+    assert bgp_update["table"] == "bgp_anomalies"
+    assert bgp_update["layers"] == ["bgp"]
+    assert bgp_update["refresh_strategy"] == "clear_then_reload"
+    assert compute_update is not None
+    assert compute_update["layers"] == ["computeCenters"]
+    assert compute_update["refresh_strategy"] == "reload"
+    assert vessel_update is not None
+    assert vessel_update["source"] == "vessel_position"
+    assert vessel_update["layers"] == ["vessels"]
+
+
+def test_build_earth_update_maps_interactable_delete_to_delta():
+    update = build_earth_update_from_db_payload(
+        {
+            "table": "earth_interactables",
+            "operation": "DELETE",
+            "entity_keys": ["note-1"],
+            "records_processed": 1,
+        }
+    )
+
+    assert update is not None
+    assert update["entity"] == "interactable"
+    assert update["action"] == "deleted"
+    assert update["ids"] == ["note-1"]
+    assert update["layers"] == ["interactables"]
+    assert update["refresh_strategy"] == "delta"
+
+
+def test_build_earth_update_ignores_unmapped_sources():
+    assert build_earth_update_from_db_payload({"source": "not_for_earth"}) is None
+
+
+def test_dispatcher_debounces_same_source_notifications():
+    async def run():
+        broadcasts = []
+        invalidated = []
+
+        async def broadcast(payload):
+            broadcasts.append(payload)
+
+        def invalidate(source):
+            invalidated.append(source)
+            return 2
+
+        dispatcher = EarthDbChangeDispatcher(
+            broadcast_earth_update=broadcast,
+            invalidate_cache=invalidate,
+            debounce_seconds=10,
+        )
+        dispatcher.handle_payload(
+            {
+                "table": "collected_data",
+                "operation": "INSERT",
+                "source": "arcgis_cables",
+                "entity_key": "cable-1",
+                "occurred_at": "2026-05-22T00:00:00Z",
+            }
+        )
+        dispatcher.handle_payload(
+            {
+                "table": "collected_data",
+                "operation": "UPDATE",
+                "source": "arcgis_cables",
+                "entity_key": "cable-2",
+                "occurred_at": "2026-05-22T00:00:01Z",
+            }
+        )
+
+        await dispatcher.flush_all()
+
+        assert invalidated == ["arcgis_cables"]
+        assert len(broadcasts) == 1
+        assert broadcasts[0]["action"] == "database_changed"
+        assert broadcasts[0]["source"] == "arcgis_cables"
+        assert broadcasts[0]["layers"] == ["cables"]
+        assert broadcasts[0]["records_processed"] == 2
+        assert broadcasts[0]["operations"] == ["INSERT", "UPDATE"]
+        assert broadcasts[0]["entity_keys"] == ["cable-1", "cable-2"]
+        assert broadcasts[0]["cache_entries_invalidated"] == 2
+
+    asyncio.run(run())
+
+
+def test_dispatcher_handles_delete_notification_as_earth_update():
+    async def run():
+        broadcasts = []
+
+        async def broadcast(payload):
+            broadcasts.append(payload)
+
+        dispatcher = EarthDbChangeDispatcher(
+            broadcast_earth_update=broadcast,
+            invalidate_cache=lambda source: 1,
+            debounce_seconds=10,
+        )
+        accepted = dispatcher.handle_notification(
+            json.dumps(
+                {
+                    "table": "collected_data",
+                    "operation": "DELETE",
+                    "source": "celestrak_tle",
+                    "entity_key": "sat:25544",
+                    "occurred_at": "2026-05-22T00:00:00Z",
+                }
+            )
+        )
+
+        await dispatcher.flush_all()
+
+        assert accepted is True
+        assert len(broadcasts) == 1
+        assert broadcasts[0]["action"] == "database_changed"
+        assert broadcasts[0]["source"] == "celestrak_tle"
+        assert broadcasts[0]["layers"] == ["satellites"]
+        assert broadcasts[0]["operation"] == "DELETE"
+        assert broadcasts[0]["refresh_strategy"] == "clear_then_reload"
+
+    asyncio.run(run())
+
+
+def test_dispatcher_debounces_bgp_derived_table_events_by_source():
+    async def run():
+        broadcasts = []
+
+        async def broadcast(payload):
+            broadcasts.append(payload)
+
+        dispatcher = EarthDbChangeDispatcher(
+            broadcast_earth_update=broadcast,
+            invalidate_cache=lambda source: 0,
+            debounce_seconds=10,
+        )
+        dispatcher.handle_payload(
+            {
+                "event_id": 200,
+                "table": "bgp_observations",
+                "operation": "DELETE",
+                "source": "ris_live_bgp",
+                "records_processed": 2,
+            }
+        )
+        dispatcher.handle_payload(
+            {
+                "event_id": 201,
+                "table": "bgp_anomalies",
+                "operation": "DELETE",
+                "source": "ris_live_bgp",
+                "records_processed": 3,
+            }
+        )
+
+        await dispatcher.flush_all()
+
+        assert len(broadcasts) == 1
+        assert broadcasts[0]["source"] == "ris_live_bgp"
+        assert broadcasts[0]["layers"] == ["bgp"]
+        assert broadcasts[0]["records_processed"] == 5
+        assert broadcasts[0]["refresh_strategy"] == "clear_then_reload"
+
+    asyncio.run(run())
+
+
+def test_dispatcher_deduplicates_notify_and_outbox_by_event_id():
+    async def run():
+        broadcasts = []
+
+        async def broadcast(payload):
+            broadcasts.append(payload)
+
+        dispatcher = EarthDbChangeDispatcher(
+            broadcast_earth_update=broadcast,
+            invalidate_cache=lambda source: 0,
+            debounce_seconds=10,
+        )
+        payload = {
+            "event_id": 42,
+            "table": "collected_data",
+            "operation": "INSERT",
+            "source": "celestrak_tle",
+            "entity_key": "sat:42",
+            "occurred_at": "2026-05-23T00:00:00Z",
+        }
+
+        assert dispatcher.handle_payload(payload) is True
+        assert dispatcher.handle_payload(dict(payload)) is False
+        await dispatcher.flush_all()
+
+        assert len(broadcasts) == 1
+        assert broadcasts[0]["records_processed"] == 1
+        assert broadcasts[0]["entity_keys"] == ["sat:42"]
+
+    asyncio.run(run())
+
+
+def test_dispatcher_flushes_continuous_events_at_max_wait():
+    async def run():
+        broadcasts = []
+        sleeps = []
+
+        async def broadcast(payload):
+            broadcasts.append(payload)
+
+        dispatcher = EarthDbChangeDispatcher(
+            broadcast_earth_update=broadcast,
+            invalidate_cache=lambda source: 0,
+            debounce_seconds=10,
+            max_wait_seconds=0.01,
+        )
+        dispatcher.handle_payload(
+            {
+                "event_id": 300,
+                "table": "collected_data",
+                "operation": "INSERT",
+                "source": "arcgis_cables",
+                "records_processed": 1,
+            }
+        )
+        await asyncio.sleep(0.02)
+        dispatcher.handle_payload(
+            {
+                "event_id": 301,
+                "table": "collected_data",
+                "operation": "UPDATE",
+                "source": "arcgis_cables",
+                "records_processed": 1,
+            }
+        )
+        await asyncio.sleep(0)
+        await dispatcher.flush_all()
+
+        assert len(broadcasts) == 1
+        assert broadcasts[0]["records_processed"] == 2
+        assert broadcasts[0]["debounce_ms"] >= 0
+
+    asyncio.run(run())
+
+
+def test_dispatcher_fast_flushes_delete_clear_then_reload():
+    async def run():
+        broadcasts = []
+
+        async def broadcast(payload):
+            broadcasts.append(payload)
+
+        dispatcher = EarthDbChangeDispatcher(
+            broadcast_earth_update=broadcast,
+            invalidate_cache=lambda source: 0,
+            debounce_seconds=10,
+            max_wait_seconds=10,
+        )
+        dispatcher.handle_payload(
+            {
+                "event_id": 310,
+                "table": "collected_data",
+                "operation": "DELETE",
+                "source": "celestrak_tle",
+                "records_processed": 10,
+            }
+        )
+        await asyncio.sleep(0.08)
+
+        assert len(broadcasts) == 1
+        assert broadcasts[0]["source"] == "celestrak_tle"
+        assert broadcasts[0]["refresh_strategy"] == "clear_then_reload"
+
+    asyncio.run(run())
+
+
+def test_outbox_rows_are_consumed_after_successful_flush():
+    async def run():
+        class FakeConnection:
+            def __init__(self):
+                self.consumed_ids = []
+
+            async def fetch(self, _query, _limit):
+                return [
+                    {
+                        "id": 501,
+                        "payload": {
+                            "event_id": 501,
+                            "table": "collected_data",
+                            "operation": "DELETE",
+                            "source": "celestrak_tle",
+                            "records_processed": 1,
+                        },
+                    }
+                ]
+
+            async def execute(self, _query, consumed_ids):
+                self.consumed_ids.extend(consumed_ids)
+
+        broadcasts = []
+
+        async def broadcast(payload):
+            broadcasts.append(payload)
+
+        dispatcher = EarthDbChangeDispatcher(
+            broadcast_earth_update=broadcast,
+            invalidate_cache=lambda source: 0,
+            debounce_seconds=10,
+        )
+        listener = EarthDbChangeListener(
+            dsn="postgresql://example/db",
+            dispatcher=dispatcher,
+        )
+        connection = FakeConnection()
+        listener._connection = connection
+
+        await listener._poll_outbox()
+
+        assert connection.consumed_ids == [501]
+        assert len(broadcasts) == 1
+
+    asyncio.run(run())
+
+
+def test_outbox_rows_remain_unconsumed_when_flush_fails():
+    async def run():
+        class FakeConnection:
+            def __init__(self):
+                self.consumed_ids = []
+
+            async def fetch(self, _query, _limit):
+                return [
+                    {
+                        "id": 601,
+                        "payload": {
+                            "event_id": 601,
+                            "table": "collected_data",
+                            "operation": "DELETE",
+                            "source": "celestrak_tle",
+                            "records_processed": 1,
+                        },
+                    }
+                ]
+
+            async def execute(self, _query, consumed_ids):
+                self.consumed_ids.extend(consumed_ids)
+
+        async def broadcast(_payload):
+            raise RuntimeError("ws down")
+
+        dispatcher = EarthDbChangeDispatcher(
+            broadcast_earth_update=broadcast,
+            invalidate_cache=lambda source: 0,
+            debounce_seconds=10,
+        )
+        listener = EarthDbChangeListener(
+            dsn="postgresql://example/db",
+            dispatcher=dispatcher,
+        )
+        connection = FakeConnection()
+        listener._connection = connection
+
+        try:
+            await listener._poll_outbox()
+        except RuntimeError:
+            pass
+
+        assert connection.consumed_ids == []
+
+    asyncio.run(run())
+
+
+def test_dispatcher_uses_aggregated_statement_record_count():
+    async def run():
+        broadcasts = []
+
+        async def broadcast(payload):
+            broadcasts.append(payload)
+
+        dispatcher = EarthDbChangeDispatcher(
+            broadcast_earth_update=broadcast,
+            invalidate_cache=lambda source: 0,
+            debounce_seconds=10,
+        )
+        dispatcher.handle_payload(
+            {
+                "event_id": 99,
+                "table": "collected_data",
+                "operation": "DELETE",
+                "source": "top500",
+                "records_processed": 100,
+                "entity_keys": ["top500:1", "top500:2"],
+                "occurred_at": "2026-05-23T00:00:00Z",
+            }
+        )
+
+        await dispatcher.flush_all()
+
+        assert len(broadcasts) == 1
+        assert broadcasts[0]["source"] == "top500"
+        assert broadcasts[0]["layers"] == ["computeCenters"]
+        assert broadcasts[0]["records_processed"] == 100
+        assert broadcasts[0]["entity_keys"] == ["top500:1", "top500:2"]
+
+    asyncio.run(run())

backend/tests/test_earth_interactables.py

@@ -0,0 +1,79 @@
+from datetime import UTC, datetime
+
+from app.models.earth_interactable import EarthInteractable
+from app.services import earth_interactables
+
+
+def make_interactable(**overrides):
+    values = {
+        "id": "poi-1",
+        "layer": "places",
+        "kind": "note",
+        "label": "Test POI",
+        "description": "A point on Earth",
+        "latitude": 30.25,
+        "longitude": 120.15,
+        "altitude": None,
+        "revision": 3,
+        "properties": {"owner": "test"},
+        "is_deleted": False,
+        "created_at": datetime(2026, 5, 22, 1, 0, tzinfo=UTC),
+        "updated_at": datetime(2026, 5, 22, 1, 5, tzinfo=UTC),
+        "deleted_at": None,
+    }
+    values.update(overrides)
+    return EarthInteractable(**values)
+
+
+def test_interactable_event_uses_object_delta_contract():
+    record = make_interactable()
+
+    event = earth_interactables.build_interactable_event(
+        action="updated",
+        record=record,
+    )
+
+    assert event["entity"] == "interactable"
+    assert event["action"] == "updated"
+    assert event["layer"] == "places"
+    assert event["layers"] == ["interactables"]
+    assert event["ids"] == ["poi-1"]
+    assert event["revision"] == 3
+    assert event["item"]["latitude"] == 30.25
+    assert event["item"]["properties"] == {"owner": "test"}
+
+
+def test_interactable_geojson_omits_deleted_records():
+    active = make_interactable(id="active")
+    deleted = make_interactable(id="deleted", is_deleted=True)
+
+    payload = earth_interactables.interactables_to_geojson([active, deleted])
+
+    assert payload["type"] == "FeatureCollection"
+    assert [feature["id"] for feature in payload["features"]] == ["active"]
+    assert payload["features"][0]["geometry"] == {
+        "type": "Point",
+        "coordinates": [120.15, 30.25],
+    }
+
+
+def test_interactable_cache_invalidation_clears_layer_and_all(monkeypatch):
+    patterns = []
+
+    def fake_delete_pattern(pattern):
+        patterns.append(pattern)
+        return 1
+
+    monkeypatch.setattr(
+        earth_interactables.earth_layer_cache,
+        "delete_pattern",
+        fake_delete_pattern,
+    )
+
+    deleted = earth_interactables.invalidate_interactable_cache("places")
+
+    assert deleted == 2
+    assert patterns == [
+        "earth:layer:v1:interactables:layer:places*",
+        "earth:layer:v1:interactables:layer:all*",
+    ]

backend/tests/test_earth_news.py

@@ -421,6 +421,62 @@ async def test_earth_news_payload_uses_fresh_database_items_without_rss(monkeypa
     assert payload["stale"] is False
 
 
+@pytest.mark.asyncio
+async def test_earth_news_payload_keeps_current_items_and_all_cruise_items(monkeypatch):
+    class FakeDb:
+        execute = object()
+
+    current_item = ParsedNewsItem(
+        id="db:current",
+        title="Current region story",
+        summary="Current summary",
+        url="https://example.com/current",
+        source="Stored Source",
+        feed_name="Stored Feed",
+        feed_region="americas",
+        homepage_url="https://example.com",
+        published_at=datetime(2026, 5, 15, 3, 0, tzinfo=UTC),
+    )
+    cruise_item = ParsedNewsItem(
+        id="db:apac",
+        title="APAC story",
+        summary="APAC summary",
+        url="https://example.com/apac",
+        source="Stored Source",
+        feed_name="Stored Feed",
+        feed_region="asia-pacific",
+        homepage_url="https://example.com",
+        published_at=datetime(2026, 5, 15, 4, 0, tzinfo=UTC),
+    )
+
+    async def fake_get_earth_news_freshness(_db, *, active_region):
+        return 12, datetime.now(UTC)
+
+    async def fake_list_earth_news_items(_db, *, active_region, limit):
+        return [current_item]
+
+    async def fake_list_earth_news_cruise_items(_db, *, limit):
+        return [current_item, cruise_item]
+
+    async def fake_enqueue_target_location_job(_payload, **_kwargs):
+        return True
+
+    async def fail_fetch(_sources):
+        raise AssertionError("fresh database items should not fetch RSS")
+
+    monkeypatch.setattr("app.services.earth_news_store.get_earth_news_freshness", fake_get_earth_news_freshness)
+    monkeypatch.setattr("app.services.earth_news_store.list_earth_news_items", fake_list_earth_news_items)
+    monkeypatch.setattr("app.services.earth_news_store.list_earth_news_cruise_items", fake_list_earth_news_cruise_items)
+    monkeypatch.setattr("app.services.earth_news_queue.enqueue_target_location_job", fake_enqueue_target_location_job)
+    monkeypatch.setattr("app.services.earth_news._fetch_rss_items_for_sources", fail_fetch)
+
+    payload = await get_earth_news_payload(lat=35.0, lon=-100.0, db=FakeDb())
+
+    assert [item["id"] for item in payload["items"]] == ["db:current"]
+    assert [item["id"] for item in payload["cruise_items"]] == ["db:current", "db:apac"]
+    assert payload["cruise_items"][1]["region"] == "asia-pacific"
+
+
 @pytest.mark.asyncio
 async def test_earth_news_payload_initializes_empty_database_from_rss(monkeypatch):
     db = object()

backend/tests/test_logging.py

@@ -4,8 +4,11 @@ import logging
 
 from io import StringIO
 
+import pytest
+
 from app.core.logging import PlanetContextFilter, PlanetFormatter, get_logger
 from app.core.request_context import set_request_id
+from app.services import business_logs
 
 
 def _capture_output(callback):
@@ -76,3 +79,43 @@ def test_structured_logger_redacts_sensitive_text_and_context():
     assert "hunter2" not in output
     assert "[REDACTED]" in output
     assert '"safe": "visible"' in output
+
+
+def test_business_context_redacts_nested_sensitive_values():
+    context = business_logs.build_business_context(
+        {
+            "provider": "openai",
+            "api_key": "sk-secret",
+            "nested": {
+                "token": "plain-token",
+                "safe": "visible",
+            },
+        }
+    )
+
+    assert context["api_key"] == "[REDACTED]"
+    assert context["nested"]["token"] == "[REDACTED]"
+    assert context["nested"]["safe"] == "visible"
+
+
+@pytest.mark.asyncio
+async def test_emit_business_log_persists_sanitized_system_event(monkeypatch):
+    events = []
+
+    async def fake_record_system_log(**payload):
+        events.append(payload)
+
+    monkeypatch.setattr(business_logs, "record_system_log", fake_record_system_log)
+
+    await business_logs.emit_business_log(
+        get_logger("tests.business"),
+        event="ai.provider.analyze.success",
+        message="AI request completed",
+        category="ai",
+        context={"model": "gpt-test", "api_key": "sk-secret"},
+    )
+
+    assert events[0]["event"] == "ai.provider.analyze.success"
+    assert events[0]["category"] == "ai"
+    assert events[0]["context"]["model"] == "gpt-test"
+    assert events[0]["context"]["api_key"] == "[REDACTED]"

backend/tests/test_system_logs.py

@@ -40,10 +40,10 @@ def test_read_log_snapshot_uses_structured_buffer_timestamp_level_and_search(mon
         {
             "earth-client": system_logs.LogSource(
                 source_id="earth-client",
-                name="Earth 浏览器端",
+                name="智能星球浏览器端",
                 kind="buffer",
                 location="redis://planet:system_logs:earth-client",
-                description="Earth 浏览器端上报日志",
+                description="智能星球浏览器端上报日志",
                 category="client",
                 buffer_key=system_logs.get_buffer_log_key("earth-client"),
             )
@@ -156,6 +156,52 @@ def test_append_buffer_log_persists_normalized_level(monkeypatch):
     assert payload["message"] == "feed delayed"
 
 
+def test_list_log_sources_includes_admin_client(monkeypatch):
+    fake_redis = FakeRedis()
+    monkeypatch.setattr(system_logs, "redis_client", fake_redis)
+
+    sources = system_logs.list_log_sources()
+
+    admin_source = next(item for item in sources if item["source_id"] == "admin-client")
+    assert admin_source["kind"] == "buffer"
+    assert admin_source["category"] == "client"
+
+
+def test_read_log_events_returns_stable_cursors(tmp_path: Path, monkeypatch):
+    log_path = tmp_path / "backend.log"
+    log_path.write_text(
+        "\n".join(
+            [
+                "2026-04-22 08:00:00 INFO service booted",
+                "2026-04-22 08:01:00 ERROR service failed",
+            ]
+        ),
+        encoding="utf-8",
+    )
+    monkeypatch.setattr(
+        system_logs,
+        "LOG_SOURCES",
+        {
+            "backend": system_logs.LogSource(
+                source_id="backend",
+                name="后端服务",
+                kind="file",
+                location=str(log_path),
+                description="测试文件日志",
+                category="service",
+            )
+        },
+    )
+
+    events = system_logs.read_log_events("backend", 50, level="error")
+
+    assert events is not None
+    assert len(events) == 1
+    assert events[0].source_id == "backend"
+    assert events[0].cursor.startswith("backend:")
+    assert events[0].line.endswith("ERROR service failed")
+
+
 def test_infer_log_level_prefers_leading_prefix_over_query_string():
     line = 'INFO: 127.0.0.1 - "GET /api/v1/system/logs/backend?limit=200&level=error&levels=error HTTP/1.1" 200 OK'
 

backend/tests/test_visualization_compute_centers.py

@@ -816,6 +816,27 @@ async def test_visualization_geo_summary_returns_counts(monkeypatch):
         app.dependency_overrides.clear()
 
 
+@pytest.mark.asyncio
+async def test_empty_cable_layers_return_empty_feature_collections():
+    class _ScalarResult:
+        def scalars(self):
+            class _Scalars:
+                def all(self):
+                    return []
+
+            return _Scalars()
+
+    class _FakeSession:
+        async def execute(self, _query):
+            return _ScalarResult()
+
+    cables = await visualization_api._build_cables_geojson(_FakeSession())
+    landing_points = await visualization_api._build_landing_points_geojson(_FakeSession())
+
+    assert cables == {"type": "FeatureCollection", "features": []}
+    assert landing_points == {"type": "FeatureCollection", "features": []}
+
+
 @pytest.mark.asyncio
 async def test_collect_location_endpoint_returns_candidates_for_known_record(monkeypatch):
     def _fake_ror(query):

docker-compose.local-model.yml

@@ -15,12 +15,16 @@ services:
       retries: 10
 
   aiprovider:
+    image: ${AI_PROVIDER_IMAGE_NAME:-planet-aiprovider:latest}
     build:
       context: .
       dockerfile: aiprovider/Dockerfile
       args:
         PYTHON_IMAGE: ${PYTHON_IMAGE:-python:3.14-slim}
         UV_IMAGE: ${UV_IMAGE:-ghcr.io/astral-sh/uv:latest}
+        AI_PROVIDER_BUILD_FINGERPRINT: ${AI_PROVIDER_BUILD_FINGERPRINT:-unknown}
+      secrets:
+        - planet_uv_config
     container_name: planet_aiprovider
     ports:
       - "8010:8010"
@@ -43,3 +47,7 @@ services:
 
 volumes:
   ollama_data:
+
+secrets:
+  planet_uv_config:
+    file: ${PLANET_UV_CONFIG_FILE:-/dev/null}

docker-compose.simple.yml

@@ -2,12 +2,16 @@ version: '3.8'
 
 services:
   aiprovider:
+    image: ${AI_PROVIDER_IMAGE_NAME:-planet-aiprovider:latest}
     build:
       context: .
       dockerfile: aiprovider/Dockerfile
       args:
         PYTHON_IMAGE: ${PYTHON_IMAGE:-python:3.14-slim}
         UV_IMAGE: ${UV_IMAGE:-ghcr.io/astral-sh/uv:latest}
+        AI_PROVIDER_BUILD_FINGERPRINT: ${AI_PROVIDER_BUILD_FINGERPRINT:-unknown}
+      secrets:
+        - planet_uv_config
     env_file:
       - ./aiprovider/.env
       - ${PLANET_AI_PROVIDER_RUNTIME_ENV_FILE:-./aiprovider/.env}
@@ -43,3 +47,7 @@ services:
 volumes:
   postgres_data:
   redis_data:
+
+secrets:
+  planet_uv_config:
+    file: ${PLANET_UV_CONFIG_FILE:-/dev/null}

docker-compose.yml

@@ -2,12 +2,16 @@ version: '3.8'
 
 services:
   aiprovider:
+    image: ${AI_PROVIDER_IMAGE_NAME:-planet-aiprovider:latest}
     build:
       context: .
       dockerfile: aiprovider/Dockerfile
       args:
         PYTHON_IMAGE: ${PYTHON_IMAGE:-python:3.14-slim}
         UV_IMAGE: ${UV_IMAGE:-ghcr.io/astral-sh/uv:latest}
+        AI_PROVIDER_BUILD_FINGERPRINT: ${AI_PROVIDER_BUILD_FINGERPRINT:-unknown}
+      secrets:
+        - planet_uv_config
     env_file:
       - ./aiprovider/.env
       - ${PLANET_AI_PROVIDER_RUNTIME_ENV_FILE:-./aiprovider/.env}
@@ -53,3 +57,7 @@ services:
 volumes:
   postgres_data:
   redis_data:
+
+secrets:
+  planet_uv_config:
+    file: ${PLANET_UV_CONFIG_FILE:-/dev/null}

docs/CHANGELOG.md

@@ -8,6 +8,153 @@ This project follows the repository versioning rule:
 - `improvement` -> `+0.0.1`（bugfix + 小功能混合）
 - `bugfix` -> `+0.0.1`
 
+## [0.68.0] — 2026-05-28
+
+Released: 2026-05-28
+
+### Highlights
+- 新增数据源任务队列的实时指标校准和批量删除进度，让大表清理、取消和完成状态在控制台中可感知。
+- 新增智能星球 interactable 可插拔聚类策略，支持稳定 3D 球面聚类、动态屏幕聚类和 250% 以上自动散开。
+- 改进新设备启动流程，`planet.sh` 会在启动前同步前端依赖，避免缺失依赖导致控制台动态导入失败。
+
+### Added / Fixed / Improved
+- 数据源列表改为中文记录数指标，并对 AIS 大表使用统计估算 + 单条详情精确校准，降低首次加载成本。
+- 数据删除任务改为分批删除并广播进度，清理 AIS 衍生表后自动 `ANALYZE`，同时修复取消中任务恢复和状态文案。
+- Earth BGP、算力中心和 interactable 图层默认使用 `stable-spherical` 聚类，船舶实时层保留 `dynamic-screen`。
+- 新增中英文 Earth interactable clustering 文档，并补充采集队列、后端删除语义和前端依赖同步说明。
+
+---
+
+## [0.67.0] — 2026-05-27
+
+Released: 2026-05-27
+
+### Highlights
+- 新增控制台日志实时跟随与前端运行时错误上报，帮助在控制台内直接排查 Admin / Earth 客户端异常。
+- 重构智能星球 Interactable 聚合和 wheel 缩放输入，保持真实地理锚点稳定，同时让鼠标滚轮和触控板拥有各自合适的缩放手感。
+
+### Added / Fixed / Improved
+- 新增 `/ws` 日志 tail 通道、数据库/文件日志统一事件读取，以及 Admin `error` / `unhandledrejection` / React ErrorBoundary 上报链路。
+- 优化控制台日志页状态颜色、跟随体验和运行时错误展示，并将 Admin 本地工具模块从 `lib` 命名迁移为局部 `utils`。
+- 修复智能星球国界/高清材质壳半径对齐、Interactable cluster 圆点显示、缩放目标累积和触控板连续缩放问题。
+- 更新 `planet.sh` 与 `.gitignore`，避免新环境构建污染锁文件并移除前端 `lib` 目录特殊放行。
+- 补充智能星球前端、渲染层级、控制台状态与运行时日志文档。
+
+---
+
+## [0.66.3] — 2026-05-26
+
+Released: 2026-05-26
+
+### Highlights
+- 修复控制台 Admin 动态导入时缺失 `../lib/utils` 导致 Vite 返回 500 的问题。
+- 补齐前端依赖安装状态，确保 Markdown Mermaid 渲染器可解析 `mermaid` 包。
+
+### Added / Fixed / Improved
+- 新增 Admin 本地 `cn` 与 `formatNumber` 工具模块，恢复布局、UI 组件和 Dashboard 的共享工具引用。
+- 放开 `.gitignore` 中 `frontend/src/admin/lib` 的源码例外，避免 utility module 再次被全局 `lib/` 规则漏提交。
+- 重启前端开发服务并验证 `/admin` 无 Vite overlay 和 console error。
+
+---
+
+## [0.66.2] — 2026-05-26
+
+Released: 2026-05-26
+
+### Highlights
+- 统一中文界面和公开文档中的产品命名：`Earth` 显示为“智能星球”，`Admin` 显示为“控制台”，`Docs` 显示为“文档”。
+- 修复文档手册、Docs catalog、控制台入口和智能星球设置中残留的中英混排标题。
+
+### Added / Fixed / Improved
+- 更新控制台侧边栏、智能星球系统设置、移动端抽屉和登录页品牌文案。
+- 同步前后端 Docs metadata、中文技术文档标题、使用手册、快速开始、术语表和运维手册中的产品命名。
+- 补充清理智能星球图层缓存、品牌配置、展示缓存等控制台 toast / dialog 文案。
+
+---
+
+## [0.66.1] — 2026-05-26
+
+Released: 2026-05-26
+
+### Highlights
+- 修复 CelesTrak active 更新窗口内清库后无法恢复的问题，新增持久原始下载缓存和完整 fallback group mode。
+- 避免数据源任务 WebSocket 与轮询同时完成时重复弹出采集失败 toast。
+- `planet.sh destroy` 保留上游原始下载缓存，数据库清空后仍可用缓存重灌。
+
+### Added / Fixed / Improved
+- CelesTrak 403 `GP data has not updated` 会先复用 active 缓存；无 active 缓存时完整拉取 `starlink/gps-ops/galileo/glonass/beidou/leo/geo/iridium-next`，任一 group 缺失则整体失败，不保存 partial。
+- 下载器缓存从 `/tmp` 迁到 `$PLANET_CACHE_DIR/downloads`，并保留 HTTP 错误响应正文以支持上游限频语义判断。
+- 补充 CelesTrak cache/fallback 回归测试和中英文运维/卫星策略文档。
+
+---
+
+## [0.66.0] — 2026-05-26
+
+Released: 2026-05-26
+
+### Highlights
+- 将 Admin 正式化为唯一控制台入口，移除旧 AntD 后台、Admin Next 路由痕迹和相关依赖。
+- 引入 PostgreSQL 数据作业队列、Earth outbox 同步和可交互对象管线，让采集、清理和 Earth 刷新进入可追踪异步链路。
+- 强化 AI、AI 工具、设置连接测试和采集任务的结构化业务日志，关键事件可在系统日志中检索。
+- CelesTrak 轨道根数采集改为完整 `active` 目录下载、续传和重试，避免部分分组失败时保存不完整卫星数据。
+
+### Added / Fixed / Improved
+- Admin 数据源页修复内置源调度启停按钮，使用 `is_active` 判断 `/enable` 与 `/disable`，避免采集状态误导调度开关。
+- Earth 新增通用 interactables 层和平台数据流文档，支持对象级 delta 同步与后续小图层扩展。
+- `planet.sh init/start/destroy` 补齐 uv 镜像回退、Bun unzip 依赖、HTTPS/LAN 跳转协议和 OOBE 清理语义。
+- Markdown code block、数据源队列、演示模式、About 版本展示、TV 链接协议和文档索引同步完成。
+
+---
+
+## [0.65.2] — 2026-05-22
+
+Released: 2026-05-22
+
+### Highlights
+- Stabilize AI Provider rebuild detection so unchanged source no longer rebuilds just because file timestamps or local cache state changed.
+- Keep `uv.lock` on the official registry while still allowing local and Docker builds to use the user's `uv.toml`.
+- Prevent local startup and bootstrap commands from rewriting `uv.lock` when a user-level mirror is configured.
+
+### Added / Fixed / Improved
+- AI Provider fingerprints now use content hashes, base image inputs, dependency metadata, and an image label instead of file mtimes.
+- `planet.sh` and `scripts/bootstrap-dev.sh` now use `uv sync --frozen`, and backend startup uses `uv run --frozen`.
+- Docker Compose passes the AI Provider fingerprint into the image build so future starts can inspect the image label directly.
+
+---
+
+## [0.65.1] — 2026-05-22
+
+Released: 2026-05-22
+
+### Highlights
+- 修复 `planet.sh` 与 Docker Compose 对 AI Provider 镜像名不一致导致的重复构建问题。
+- 让本地 `uv` 与 Docker build 统一使用用户机器上的 `uv.toml`，避免把镜像源写入 `uv.lock`。
+- 通过 BuildKit secret 向容器构建传入 uv 配置，保留用户源选择且不把配置写入镜像层。
+
+### Added / Fixed / Improved
+- `planet.sh` 自动解析 `UV_CONFIG_FILE`、仓库 `uv.toml` 与用户级 `~/.config/uv/uv.toml`，并清理会覆盖配置的 `UV_INDEX_URL` 环境变量。
+- AI Provider 和后端 Dockerfile 在 `uv sync` 阶段挂载 uv 配置 secret，并复用 BuildKit uv cache。
+- Docker Compose 三套配置统一声明 `planet-aiprovider:latest` 镜像名和 `planet_uv_config` build secret。
+
+---
+
+## [0.65.0] — 2026-05-21
+
+Released: 2026-05-21
+
+### Highlights
+- 收敛 Admin Next 数据源触发入口：主按钮在未勾选时触发全部，勾选内置源后切换为“触发已选 N”，并移除手填 ID 的批量触发弹窗。
+- 优化数据源采集队列入口：右上角按钮常驻，空态显示队列图标，有任务时显示纯圆环进度，队列改为浮层避免挤压表格。
+- 强化 `planet.sh destroy` 清理语义，销毁时先硬重置运行中的本地 Postgres `public` schema，避免残留采集数据让 OOBE 误判 ready。
+
+### Added / Fixed / Improved
+- Admin Next 表格新增可选选择列，仅在 `/datasources` 内置源分区启用，支持当前可见行全选并在筛选、切分区或刷新时清空选择。
+- 数据源批量触发复用 `/datasources/trigger-batch` 的 `source_ids`，成功后写入现有采集队列并清空勾选。
+- `destroy` 补充清理 `planet-aiprovider:latest` 镜像以及 Python/Vite 等本地编译缓存，同时保留源码和 `.env`。
+- Docs Gatekeeper 与 Tactile UI 文档/样式继续补齐，覆盖本轮按钮、队列、OOBE 和销毁流程说明。
+
+---
+
 ## [0.64.0] — 2026-05-21
 
 Released: 2026-05-21

docs/deprecated/README.md

@@ -16,7 +16,24 @@
 - 已被正式实现替代、继续放在 `docs/` 根目录会误导后续开发的计划，归档
 - 仍然指导未来开发、尚未完成或仍有明确执行价值的文档，继续保留在 `docs/`
 
+当前替代入口：
+
+- 业务和数据产品链路见 [业务架构与数据流转](/home/ray/dev/linkong/planet/docs/technical/zh/platform-data-flows.md)。
+- 当前后端作业、outbox 和 Earth 同步实现见 [数据作业与 Outbox 技术架构](/home/ray/dev/linkong/planet/docs/technical/zh/data-job-earth-sync-architecture.md)。
+- 用户操作流程见 [Planet 使用手册](/home/ray/dev/linkong/planet/docs/technical/zh/manual.md)。
+- 当前代码结构见 [技术文档索引](/home/ray/dev/linkong/planet/docs/technical/zh/README.md)。
+
 补充说明：
 
 - 一部分归档文档来自外部或临时工作流草案，例如 sisyphus 生成的初稿
 - 这类文档如果有可用内容，应先吸收到 `docs/plans/` 或 `docs/technical/`，再归档保留来源记录
+
+## 近期归档
+
+- [Docs Gatekeeper 鉴权系统计划](/home/ray/dev/linkong/planet/docs/deprecated/docs-gatekeeper-auth-plan.md)：已落地，当前实现见技术文档。
+- [Location Resolver 共享管线计划](/home/ray/dev/linkong/planet/docs/deprecated/location-resolver-shared-pipeline-plan.md)：已落地，当前实现见技术文档。
+- [Earth Surface Hover Info Plan](/home/ray/dev/linkong/planet/docs/deprecated/earth-surface-hover-info-plan.md)：已实现，保留为历史记录。
+- [Admin Next Dual Track Full Migration Plan](/home/ray/dev/linkong/planet/docs/deprecated/admin-next-dual-track-full-migration-plan.md)：已被当前 `frontend/src/admin/` 控制台结构替代。
+- [Admin Next Soft Glass Goal Driven Plan](/home/ray/dev/linkong/planet/docs/deprecated/admin-next-soft-glass-goal-driven-plan.md)：已被当前 `frontend/src/admin/` 控制台结构替代。
+- [Admin Next Parity Checklist](/home/ray/dev/linkong/planet/docs/deprecated/admin-next-parity-checklist.md)：旧 `/admin-next/*` 验收资料，保留为迁移历史。
+- [Admin Next Parity Audit Closeout](/home/ray/dev/linkong/planet/docs/deprecated/admin-next-parity-audit-closeout-plan.md)：旧 `/admin-next/*` 审计资料，保留为迁移历史。

docs/deprecated/admin-next-parity-audit-closeout-plan.md

@@ -104,7 +104,7 @@ The old AntD page itself treated these Earth content tabs as placeholder-level c
 - `models_3d`
 - `news_anchor_strategy`
 
-If backend endpoints are later added, these items must be promoted into `docs/plans/admin-next-parity-checklist.md` with concrete API and UI acceptance criteria.
+If backend endpoints are later added, these items must be promoted into a new active plan under `docs/plans/` with concrete API and UI acceptance criteria.
 
 ## Final Gate
 
@@ -114,7 +114,7 @@ After the route promotion, the final gate is no longer “switch old routes.”
 2. Run the static checks:
    - `rg "map: \\(\\) => \\[\\]|暂不支持保存|placeholder" frontend/src/admin-next`
    - `rg "ShadowPage|FeatureConsole|GlassPanel|InspectorDrawer" frontend/src/admin-next`
-3. Manually verify every official route listed in `docs/plans/admin-next-parity-checklist.md`.
+3. Manually verify every official route listed in `docs/deprecated/admin-next-parity-checklist.md`.
 4. Confirm `/legacy/admin/*` still opens old AntD pages during the validation window.
 5. Delete old AntD pages and remove AntD dependencies only as a separate final cleanup task after explicit confirmation.
 

docs/deprecated/admin-next-soft-glass-goal-driven-plan.md

@@ -8,7 +8,7 @@ The redesign must cover desktop and mobile. Data display, icon semantics, table
 
 ## Criteria For Success
 
-- This plan exists at `docs/plans/admin-next-soft-glass-goal-driven-plan.md`.
+- This archived plan exists at `docs/deprecated/admin-next-soft-glass-goal-driven-plan.md`.
 - `/admin-next/*` has real pages for every route; route usage of `ShadowPage` is removed.
 - Admin Next supports `system`, `light`, and `dark` theme modes using the same persistence and system-theme idea as Docs.
 - The visual language reads as soft-glass / light-neumorphic instead of an AntD reskin: translucent panels, fine borders, subtle glow, cool backgrounds, restrained accent colors, crisp icons, and tactile controls.
@@ -214,4 +214,3 @@ Manual viewport checks:
 - low height;
 - 125% / 150% browser zoom;
 - light / dark / system theme modes.
-

docs/deprecated/location-resolver-shared-pipeline-plan.md

@@ -1,6 +1,6 @@
 # Location Resolver Shared Pipeline Plan
 
-**状态**：已实现，当前用户流程见 [Earth 位置候选采集使用手册](/home/ray/dev/linkong/planet/docs/technical/zh/location-pipeline-user.md)，开发接口见 [通用位置估算管线开发说明](/home/ray/dev/linkong/planet/docs/technical/zh/location-pipeline-development.md)。
+**状态**：已实现，当前用户流程见 [Planet 使用手册](/home/ray/dev/linkong/planet/docs/technical/zh/manual.md) 的 Earth 位置候选采集章节，开发接口见 [通用位置估算管线开发说明](/home/ray/dev/linkong/planet/docs/technical/zh/location-pipeline-development.md)。
 
 ## Goal
 

docs/documentation-coverage-rules.md

@@ -37,7 +37,7 @@ If the same action has both a UI and a CLI path (e.g. user creation), describe t
   - a `credential_provider` in `backend/app/core/datasource_defaults.py`;
   - a default credential guide in `backend/app/services/credential_guides.py`;
   - a supported connectivity provider in `backend/app/services/datasource_connectivity.py`;
-  - settings UI guidance or a credential form in `frontend/src/pages/Settings/Settings.tsx`;
+  - settings UI guidance or a credential form in `frontend/src/admin/pages/PlainResourcePages.tsx`;
   - a regression test that fails if the guide/provider is missing.
 
 ## Recommended Checks

docs/plans/README.md

@@ -7,6 +7,12 @@
 - 分几期做
 - 当前差距和下一步是什么
 
+当前实现、业务架构和数据链路不放在这里。它们分别进入：
+
+- [业务架构与数据流转](/home/ray/dev/linkong/planet/docs/technical/zh/platform-data-flows.md)
+- [数据作业与 Outbox 技术架构](/home/ray/dev/linkong/planet/docs/technical/zh/data-job-earth-sync-architecture.md)
+- [技术文档索引](/home/ray/dev/linkong/planet/docs/technical/zh/README.md)
+
 适合放入这里的内容：
 
 - Earth / BGP / 地形 / 天球实施方案
@@ -16,38 +22,40 @@
 
 当前重点入口：
 
-- [earth-mobile-drawer-ui-plan.md](/home/ray/dev/linkong/planet/docs/plans/earth-mobile-drawer-ui-plan.md)
-- [earth-compute-center-bgp-style-plan.md](/home/ray/dev/linkong/planet/docs/plans/earth-compute-center-bgp-style-plan.md)
-- [earth-renderer-architecture-separation-plan.md](/home/ray/dev/linkong/planet/docs/plans/earth-renderer-architecture-separation-plan.md)
+- [Earth Mobile Drawer UI Plan](/home/ray/dev/linkong/planet/docs/plans/earth-mobile-drawer-ui-plan.md)
+- [Earth Compute Center BGP Style Plan](/home/ray/dev/linkong/planet/docs/plans/earth-compute-center-bgp-style-plan.md)
+- [Earth Renderer Architecture Separation Plan](/home/ray/dev/linkong/planet/docs/plans/earth-renderer-architecture-separation-plan.md)
 - [Earth 高精度国界静态瓦片计划](/home/ray/dev/linkong/planet/docs/plans/earth-high-precision-boundary-tiles-plan.md)
-- [earth-predicted-orbit-plan.md](/home/ray/dev/linkong/planet/docs/plans/earth-predicted-orbit-plan.md)
-- [earth-webgl-instancing-satellites-plan.md](/home/ray/dev/linkong/planet/docs/plans/earth-webgl-instancing-satellites-plan.md)
-- [earth-real-terrain-plan.md](/home/ray/dev/linkong/planet/docs/plans/earth-real-terrain-plan.md)
-- [earth-news-source-configuration-and-collector-plan.md](/home/ray/dev/linkong/planet/docs/plans/earth-news-source-configuration-and-collector-plan.md)
-- [earth-news-cruise-summary-plan.md](/home/ray/dev/linkong/planet/docs/plans/earth-news-cruise-summary-plan.md)
+- [Earth Predicted Orbit Plan](/home/ray/dev/linkong/planet/docs/plans/earth-predicted-orbit-plan.md)
+- [Earth WebGL Instancing Satellites Plan](/home/ray/dev/linkong/planet/docs/plans/earth-webgl-instancing-satellites-plan.md)
+- [Earth Real Terrain Plan](/home/ray/dev/linkong/planet/docs/plans/earth-real-terrain-plan.md)
+- [Earth News Source Configuration And Collector Plan](/home/ray/dev/linkong/planet/docs/plans/earth-news-source-configuration-and-collector-plan.md)
+- [Earth News Cruise Summary Plan](/home/ray/dev/linkong/planet/docs/plans/earth-news-cruise-summary-plan.md)
 - [Earth 动作捕捉手势控制计划](/home/ray/dev/linkong/planet/docs/plans/earth-motion-capture-gesture-control-plan.md)
 - [Earth 动捕交互语义 V2 计划](/home/ray/dev/linkong/planet/docs/plans/earth-motion-gesture-interaction-v2-plan.md)
 - [Earth Presentation 解耦架构计划](/home/ray/dev/linkong/planet/docs/plans/earth-presentation-decoupled-architecture-plan.md)
-- [earth-vessel-rendering-performance-plan.md](/home/ray/dev/linkong/planet/docs/plans/earth-vessel-rendering-performance-plan.md)
+- [Earth Vessel Rendering Performance Plan](/home/ray/dev/linkong/planet/docs/plans/earth-vessel-rendering-performance-plan.md)
 - [AIS 多源采集、冲突记录与聚合接口计划](/home/ray/dev/linkong/planet/docs/plans/earth-vessel-ais-aggregation-plan.md)
-- [earth-interactable-layer-plan.md](/home/ray/dev/linkong/planet/docs/plans/earth-interactable-layer-plan.md)
-- [frontend-public-docs-site-plan.md](/home/ray/dev/linkong/planet/docs/plans/frontend-public-docs-site-plan.md)
-- [Docs Gatekeeper 鉴权系统计划](/home/ray/dev/linkong/planet/docs/plans/docs-gatekeeper-auth-plan.md)
-- [Location Resolver 共享管线计划](/home/ray/dev/linkong/planet/docs/plans/location-resolver-shared-pipeline-plan.md)
-- [frontend-ai-playground-development-plan.md](/home/ray/dev/linkong/planet/docs/plans/frontend-ai-playground-development-plan.md)
+- [Earth Interactable Layer Plan](/home/ray/dev/linkong/planet/docs/plans/earth-interactable-layer-plan.md)
+- [Frontend Public Docs Site Plan](/home/ray/dev/linkong/planet/docs/plans/frontend-public-docs-site-plan.md)
+- [Frontend AI Playground Development Plan](/home/ray/dev/linkong/planet/docs/plans/frontend-ai-playground-development-plan.md)
 - [AI Provider OpenClaw-Style Routing Plan](/home/ray/dev/linkong/planet/docs/plans/ai-provider-openclaw-style-routing-plan.md)
 - [统一集成配置 Schema 系统计划](/home/ray/dev/linkong/planet/docs/plans/integration-config-schema-system-plan.md)
 - [Lightweight Agent Orchestrator 与 WebSearch 证据层计划](/home/ray/dev/linkong/planet/docs/plans/agents-light-orchestrator-websearch-plan.md)
-- [Admin Next Parity Checklist](/home/ray/dev/linkong/planet/docs/plans/admin-next-parity-checklist.md)
-- [Admin Next Parity Audit Closeout](/home/ray/dev/linkong/planet/docs/plans/admin-next-parity-audit-closeout-plan.md)
-- [ue5-mvp-fused-plan.md](/home/ray/dev/linkong/planet/docs/plans/ue5-mvp-fused-plan.md)
+- [UE5 MVP Fused Plan](/home/ray/dev/linkong/planet/docs/plans/ue5-mvp-fused-plan.md)
+
+历史计划入口：
+
+- 已完成、已替代或只作为历史决策背景保留的文档，统一放在 [Deprecated Docs](/home/ray/dev/linkong/planet/docs/deprecated/README.md)。
 
 不适合放入这里的内容：
 
 - 当前代码结构说明
 - 组件现状和实现入口
 - 已经落地的技术上下文说明
+- 已经成为当前行为的“架构说明”
 
 这些应放入：
 
-- [docs/technical/README.md](/home/ray/dev/linkong/planet/docs/technical/README.md)
+- [技术文档索引](/home/ray/dev/linkong/planet/docs/technical/zh/README.md)
+- [Technical Docs Index](/home/ray/dev/linkong/planet/docs/technical/en/README.md)

docs/plans/ai-provider-openclaw-style-routing-plan.md

@@ -178,5 +178,5 @@ Runtime 选择规则：
 - `backend/app/services/ai_client.py`
 - `aiprovider/main.py`
 - `aiprovider/provider_service.py`
-- `frontend/src/admin-next/pages/PlainResourcePages.tsx`
-- `docs/plans/admin-next-parity-audit-closeout-plan.md`
+- `frontend/src/admin/pages/PlainResourcePages.tsx`
+- `docs/deprecated/admin-next-parity-audit-closeout-plan.md`

docs/plans/earth-interactable-layer-plan.md

@@ -2,7 +2,7 @@
 
 ## 背景
 
-状态：Phase 1 已经开始落地，Phase 2 的 BGP 事件 / 观测站迁移和 Phase 3 的算力中心迁移也已完成。`frontend/public/earth/js/interactable.js` 已新增，AIS 船只、BGP 事件、BGP 观测站和算力中心图层已经改为通过 `createInteractableLayer()` 使用通用批量 `Points`、hover / locked overlay、默认 glow、状态更新、asset icon 预加载、屏幕空间 picking、固定 / 距离缩放和跨 Interactable 同坐标避让。登陆点因 `THREE.Points` 边缘深度裁切和贴地层级要求，已退回专用 `THREE.Sprite` 黄色球路径，并与海缆同高度同 renderOrder。后续阶段聚焦把可复用的扩圈 / 雷达扇形动画正式沉淀成 `animations` 扩展。
+状态：Phase 1 已经开始落地，Phase 2 的 BGP 事件 / 观测站迁移和 Phase 3 的算力中心迁移也已完成。`frontend/public/earth/js/interactable.js` 已新增，AIS 船只、BGP 事件、BGP 观测站和算力中心图层已经改为通过 `createInteractableLayer()` 使用通用批量 `Points`、hover / locked overlay、默认 glow、状态更新、asset icon 预加载、屏幕空间 picking、固定 / 距离缩放和跨 Interactable 同坐标关系元数据。登陆点因 `THREE.Points` 边缘深度裁切和贴地层级要求，已退回专用 `THREE.Sprite` 黄色球路径，并与海缆同高度同 renderOrder。后续阶段聚焦把可复用的扩圈 / 雷达扇形动画正式沉淀成 `animations` 扩展。
 
 当前实现说明和接入示例见：
 
@@ -100,9 +100,10 @@ createInteractableLayer({
 | `picking.throttleMs` | `100` | `80` | hover picking 节流。 |
 | `picking.skipWhileDragging` | `true` | `true` | 拖动和惯性期间跳过 hover picking。 |
 | `zIndexPolicy` | `"surface-icon"` | `"surface-icon"` | 预设层级策略，避免每个业务图层手写高度和 renderOrder。 |
-| `avoidance.enabled` | `true / false` | `true` | 是否参与跨 Interactable 的同坐标避让。默认开启，同一经纬度下的图标会沿地表切平面小幅排开，方便辨认和选择。 |
-| `avoidance.radius` | `number` | `1.1` | 同坐标避让的第一圈半径，单位为地球本地坐标单位。 |
+| `avoidance.enabled` | `true / false` | `true` | 是否记录跨 Interactable 的同坐标关系。该配置只生成 overlap 元数据，不允许移动真实 marker 坐标。 |
 | `avoidance.precision` | `number` | `4` | 经纬度归并精度，默认约等于只处理几乎完全重叠的图标。 |
+| `cluster.enabled` | `true / false` | 跟随 avoidance | 是否参与跨 Interactable 的当前帧屏幕重叠合并。只影响显示，不改变业务坐标。 |
+| `cluster.maxMarkersPerDot` | `number` | `14` | 单个聚合圆点的对象上限；超过后按屏幕局部邻近关系拆成多个较小圆点，避免一个点过大或跨区域串联。 |
 | `legend` | `{ label, color, shape }[]` | `[]` | 可选图例声明，业务层也可以继续自己导出。 |
 | `metadata` | object | `{}` | 业务扩展数据，不参与渲染但参与 tooltip / info-card / search。 |
 

docs/technical/en/README.md

@@ -1,46 +1,53 @@
 # Technical Docs
 
-This directory holds "current implementation and current structure" documentation, focusing on:
+This is the current Intelligent Planet documentation entry point. Docs are organized by reader path: start with business architecture to understand data products, then move into user manuals or implementation references.
 
-- How the code is organized right now
-- Where the current entry points are
-- How state and components work
-- Which implementation boundaries future changes should follow
+## Business Architecture
 
-What belongs here:
+- [Business Architecture and Data Flows](/home/ray/dev/linkong/planet/docs/technical/en/platform-data-flows.md): purpose, collection flow, fact tables, derived tables, cache, and WebSocket broadcast path for each Earth data product
+- [Naming Glossary](/home/ray/dev/linkong/planet/docs/technical/en/naming-glossary.md): English/Chinese terms used across the console, Earth, backend, and docs
 
-- Quickstart and user manual
-- Frontend context
-- Earth frontend structure
-- Earth satellite footprint policy
-- Earth render layer order
-- Earth layer style property index
-- Backend runtime control
-- Collector status
-- Collector settings and connectivity validation
-- Earth Interactable integration
-- Collection format conventions
+## Manual
 
-## Entry Points
+- [Quickstart](/home/ray/dev/linkong/planet/docs/technical/en/quickstart.md): shortest path to getting Intelligent Planet running from scratch
+- [Intelligent Planet Manual](/home/ray/dev/linkong/planet/docs/technical/en/manual.md): user workflows for the console, Earth, Docs, and common features
+- [FAQ](/home/ray/dev/linkong/planet/docs/technical/en/faq.md): troubleshooting for Windows / WSL, ports, dependencies, motion capture, credentials, and Docs permissions
 
-- [Quickstart](/home/ray/dev/linkong/planet/docs/technical/en/quickstart.md): The shortest path to getting Planet running from scratch
-- [Planet Manual](/home/ray/dev/linkong/planet/docs/technical/en/manual.md): Complete usage guide for the console, `planet.sh`, Earth, and Docs
-- [FAQ](/home/ray/dev/linkong/planet/docs/technical/en/faq.md): Central troubleshooting entry for Windows / WSL, ports, dependencies, motion capture, credentials, and Docs permissions
-- [Earth Location Candidate Collection User Guide](/home/ray/dev/linkong/planet/docs/technical/en/location-pipeline-user.md): Collect and preview coordinate candidates for compute centers and BGP collectors on Earth
-- [Collector Settings and Connectivity Validation](/home/ray/dev/linkong/planet/docs/technical/en/datasource-collector-settings-connectivity.md): Data source catalog, collector settings, connectivity validation, and BarentsWatch credentials
-- [Shared Location Resolution Pipeline Development Guide](/home/ray/dev/linkong/planet/docs/technical/en/location-pipeline-development.md): Backend location resolver / pipeline interfaces, registries, and extension points
-- [Docs Gatekeeper Development Guide](/home/ray/dev/linkong/planet/docs/technical/en/docs-gatekeeper-development.md): Backend Docs catalog, Markdown content loading, and Gatekeeper permission groups
-- [Naming Glossary](/home/ray/dev/linkong/planet/docs/technical/en/naming-glossary.md): English/Chinese term mapping for the console, Earth, backend, and docs
-- [Earth Interactable Usage](/home/ray/dev/linkong/planet/docs/technical/en/earth-interactable-usage.md): API, lifecycle, and integration examples for Earth surface icon Interactable
-- [Earth Toolbar and Overlay Coordination](/home/ray/dev/linkong/planet/docs/technical/en/earth-toolbar-overlay-coordination.md): Closing matrix and integration rules for toolbar buttons, search, settings, news, and layer overlays
-- [Tactile UI Components](/home/ray/dev/linkong/planet/docs/technical/en/tactile-ui-components.md): Portable button, switch, tooltip, and scrollbar APIs, theme tokens, and migration rules
+## Earth Implementation
 
-What does not belong here:
+- [Earth Frontend Context](/home/ray/dev/linkong/planet/docs/technical/en/earth-frontend-context.md): Earth modules, state, WebSocket refresh, and layer lifecycle
+- [Earth Layer Style Reference](/home/ray/dev/linkong/planet/docs/technical/en/earth-layer-style-reference.md): layer colors, symbols, materials, and visual parameters
+- [Earth Render Layer Order](/home/ray/dev/linkong/planet/docs/technical/en/earth-render-layer-order.md): renderOrder, depth strategy, picking, and collision avoidance
+- [Earth Satellite Footprint Policy](/home/ray/dev/linkong/planet/docs/technical/en/earth-satellite-footprint-policy.md): satellite footprint display boundaries and strategy
+- [BGP Context](/home/ray/dev/linkong/planet/docs/technical/en/earth-bgp-context.md): BGP rendering, aggregation, and collector implementation in Earth
+- [Earth Interactable Usage](/home/ray/dev/linkong/planet/docs/technical/en/earth-interactable-usage.md): `Interactable` API, lifecycle, and integration examples
+- [Earth Interactable Clustering](/home/ray/dev/linkong/planet/docs/technical/en/earth-interactable-clustering.md): pluggable cluster strategies, stable spherical clustering, and dynamic screen clustering boundaries
+- [Earth Toolbar and Overlay Coordination](/home/ray/dev/linkong/planet/docs/technical/en/earth-toolbar-overlay-coordination.md): close matrix for toolbar buttons, search, settings, news, and layer overlays
 
-- Incomplete roadmaps
-- Future iteration plans
-- Large-scale refactor proposals
+## Frontend Implementation
 
-Those belong in:
+- [Admin Frontend Context](/home/ray/dev/linkong/planet/docs/technical/en/frontend-admin-frontend-context.md): console page structure, state boundaries, and lazy loading
+- [Frontend Layout Guidelines](/home/ray/dev/linkong/planet/docs/technical/en/frontend-layout-guidelines.md): layout, tables, panels, and responsive constraints
+- [Tactile UI Components](/home/ray/dev/linkong/planet/docs/technical/en/tactile-ui-components.md): button, switch, tooltip, scrollbar APIs, and theme tokens
 
-- [Plans Index](/home/ray/dev/linkong/planet/docs/plans/README.md)
+## Backend Implementation
+
+- [Data Collectors](/home/ray/dev/linkong/planet/docs/technical/en/backend-collectors.md): collectors, task types, save layer, and status updates
+- [System Service Control](/home/ray/dev/linkong/planet/docs/technical/en/backend-system-service-control.md): backend service control and system operation APIs
+- [Datasource Collector Settings and Connectivity](/home/ray/dev/linkong/planet/docs/technical/en/datasource-collector-settings-connectivity.md): datasource catalog, collector settings, connectivity validation, and credentials
+- [Datasource API Performance](/home/ray/dev/linkong/planet/docs/technical/en/backend-datasources-api-performance.md): DataSources list API performance and caching
+- [Data Jobs and Outbox Architecture](/home/ray/dev/linkong/planet/docs/technical/en/data-job-earth-sync-architecture.md): PostgreSQL job queue, outbox, listener, and Kafka / Spark evolution boundaries
+- [Shared Location Resolution Pipeline Development Guide](/home/ray/dev/linkong/planet/docs/technical/en/location-pipeline-development.md): location resolver / pipeline interfaces, registries, and extension points
+- [News Live Streams Collector Format](/home/ray/dev/linkong/planet/docs/technical/en/earth-news-live-streams-collector-format.md): news, live stream, and media collection payload conventions
+- [Docs Gatekeeper Development Guide](/home/ray/dev/linkong/planet/docs/technical/en/docs-gatekeeper-development.md): backend Docs catalog, Markdown content loading, and Gatekeeper permission groups
+
+## Agents and Operations
+
+- [AI Provider Guide](/home/ray/dev/linkong/planet/docs/technical/en/agents-aiprovider.md): model provider adapters, task prompts, and invocation boundaries
+- [Intelligent Planet Ops Runbook](/home/ray/dev/linkong/planet/docs/technical/en/ops-runbook.md): deployment, startup, troubleshooting, and sensitive operations
+- [Docker + Compose + Buildx Upgrade](/home/ray/dev/linkong/planet/docs/technical/en/ops-docker-compose-buildx-upgrade.md): Docker toolchain upgrade steps
+- [planet.sh Startup](/home/ray/dev/linkong/planet/docs/technical/en/ops-planet-sh-startup.md): startup script, health checks, and performance optimization
+
+## Plans and History
+
+Incomplete roadmaps, large refactor proposals, and future plans live in the repository path `docs/plans/README.md`. Completed, outdated, or replaced designs live in `docs/deprecated/README.md`.

docs/technical/en/agents-aiprovider.md

@@ -101,7 +101,7 @@ The AI settings page uses:
 
 These endpoints require an authenticated user. The `secrets` endpoint is only used when the settings page reveals a key or token; hiding the field restores the masked preview.
 
-Admin Next keeps the AI page aligned with the legacy information architecture:
+Admin keeps the AI page aligned with the legacy information architecture:
 
 - `Model Providers`
   - Manages provider, wire adapter, default model, LLM API key, proxy URL, proxy token, model refresh, set-as-default, and lightweight connectivity testing.
@@ -272,7 +272,7 @@ Each provider has its own key slot. Resolution order is:
 
 `.env` is only a fallback. After the settings page saves successfully, or after the connection test succeeds, PostgreSQL becomes the global default source.
 
-Admin Next must compute key status per provider or tool:
+Admin must compute key status per provider or tool:
 
 - If the database has a key for the current provider/tool, show `configured`.
 - If the database has no key but the fallback provider, model, or tool matches the current item, show the fallback masked preview.
@@ -283,7 +283,7 @@ Tool keys follow the same rule. WebSearch and OCR must match the current tool an
 
 ### Lightweight Connectivity Testing
 
-The Admin Next plug button performs a lightweight connectivity check and does not save configuration. Common API-platform practice is two-tiered:
+The Admin plug button performs a lightweight connectivity check and does not save configuration. Common API-platform practice is two-tiered:
 
 - Check a provider catalog or low-cost endpoint to validate base URL, authentication, and model reachability.
 - Send full model requests only when the user explicitly runs Playground or a business task.

docs/technical/en/backend-collectors.md

@@ -73,6 +73,10 @@ async def run(self, db):
 
 **Core file**: `backend/app/services/collectors/base.py`
 
+Manual trigger, data clearing, and cache clearing now enter the PostgreSQL data job queue. `collection_tasks` remains the task ledger. Collectors only own `fetch -> transform -> save`; the `data_jobs.py` worker claims `collect` / `clear_data` / `clear_cache` / `earth_refresh` jobs and writes progress back. Earth layer refresh relationships live in `earth_layer_adapters.py`; do not hand-code cache invalidation or WebSocket broadcasts inside individual collectors or buttons.
+
+Data deletion runs in batches so AIS-scale tables are not locked by one huge statement. A `clear_data` job clears `collected_data`, then source-specific AIS derived tables, and broadcasts `records_processed` as it goes; the console queue renders only user-facing text such as `Deleting data` and `Delete complete`, while internal table names remain in logs and raw task details. After AIS cleanup, the backend runs `ANALYZE ais_raw_observations` so datasource-list estimates converge quickly. The datasource directory uses PostgreSQL statistics for AIS record counts by default to avoid a cold-start `count(*)`; opening a single datasource detail row requests the exact count for that source.
+
 ## III. Collector List
 
 | Collector | Data type | Content | Frequency |
@@ -93,7 +97,7 @@ Earth boundaries are no longer data collectors. They are Earth static rendering
 
 TOP500 and Epoch AI compute sources do not always provide usable coordinates. The unified Earth compute-center endpoint uses only valid source-provided coordinates or `compute_center_locations` dimension-table coordinates during the main map startup path; records without coordinates are returned as `unresolved` instead of being rendered from a local registry, country centroid, or guessed city. When users manually collect candidates, the backend queries ROR and Nominatim/OpenStreetMap from source fields; accepted candidates are saved into `compute_center_locations` and rendered from that table on the next layer refresh.
 
-Admin Next collection management follows the business hierarchy instead of flattening every endpoint into one table:
+Admin collection management follows the business hierarchy instead of flattening every endpoint into one table:
 
 - `Collectors`: endpoint, authentication, headers, base parameters, enabled state, and credential guides.
 - `Collection Schedule`: scheduler state and task controls.

docs/technical/en/backend-datasources-api-performance.md

@@ -96,4 +96,4 @@ if (res.data.task_id) {
 ## Related Files
 
 - [datasources.py](/home/ray/dev/linkong/planet/backend/app/api/v1/datasources.py): `_load_datasource_list_context`, `list_datasources`
-- [DataSources.tsx](/home/ray/dev/linkong/planet/frontend/src/pages/DataSources/DataSources.tsx): `BuiltInDataSource`, `triggerDatasource`
+- [DataSources.tsx](/home/ray/dev/linkong/planet/frontend/src/admin/pages/PlainResourcePages.tsx): `BuiltInDataSource`, `triggerDatasource`

docs/technical/en/data-job-earth-sync-architecture.md

@@ -0,0 +1,141 @@
+# Data Jobs and Outbox Architecture
+
+This document records the technical boundary for Planet v1 data jobs, database outbox, and Earth refresh. For each data product's business purpose and end-to-end flow, see [Business Architecture and Data Flows](/home/ray/dev/linkong/planet/docs/technical/en/platform-data-flows.md).
+
+## Architecture Boundary
+
+- PostgreSQL is the durable v1 job ledger and outbox. Kafka, Celery, and RQ are intentionally not part of v1.
+- `collection_tasks` records collection, data clearing, cache clearing, and non-database Earth refresh jobs.
+- `earth_data_change_events` records fact-table or derived-table changes and is the reliable source for Earth sync.
+- `LISTEN/NOTIFY` is only the low-latency wakeup path; the listener still polls unconsumed outbox rows.
+- Redis is mainly cache, auth helper, OTP / rate limit, temporary logs, and WebSocket support. It is not the durable queue.
+
+## Database Change Sync
+
+```mermaid
+flowchart LR
+  Write["Fact or derived table write"] --> Trigger["PostgreSQL trigger"]
+  Trigger --> Outbox["earth_data_change_events"]
+  Trigger --> Notify["planet_earth_data_changes"]
+  Outbox --> Listener["earth_db_change_listener"]
+  Notify --> Listener
+  Listener --> Adapter["earth_layer_adapters"]
+  Adapter --> Cache["cache invalidation"]
+  Cache --> WS["earth_updates"]
+```
+
+1. Collection, clearing, location resolution, or projection jobs write fact or derived tables.
+2. Statement triggers write outbox rows for `INSERT` / `UPDATE` / `DELETE`.
+3. The listener wakes through notify or finds pending rows through polling.
+4. The listener maps `table + source` to Earth layers, refresh strategy, and cache patterns through `earth_layer_adapters.py`.
+5. The listener merges short-window same-layer events, invalidates cache, and broadcasts `earth_updates`.
+6. The outbox row is marked consumed only after successful broadcast; failed rows stay retryable.
+
+DB changes no longer create default `earth_refresh` jobs, so they are not blocked by long same-source collection or clearing jobs. `earth_refresh` remains for manual cache clearing and non-DB refresh hints.
+
+## Data Job Queue
+
+`collection_tasks` is the unified job ledger. Workers claim `queued` jobs with PostgreSQL `FOR UPDATE SKIP LOCKED`; write jobs for the same `source` run serially, while different sources may run in parallel.
+
+| task_type | Purpose |
+| --- | --- |
+| `collect` | Run a built-in datasource collector |
+| `clear_data` | Delete collected rows and declared derived rows for the source |
+| `clear_cache` | Delete Earth / dashboard cache for the source |
+| `earth_refresh` | Invalidate Earth layer cache and broadcast a refresh hint for non-DB changes |
+
+API handlers only create jobs and return `task_id`. Execution, progress, cancellation, and terminal state are written back by workers and pushed to the frontend through the `datasource_tasks` channel.
+
+Cancellation means “keep committed batches”: clicking stop marks the job as `cancelling` and cancels the in-memory coroutine. Already committed batches remain; unfinished batches follow the collector or cleanup rollback path.
+
+## Earth Sync Event Model
+
+The unified event model is `earth.layer.changed`:
+
+```json
+{
+  "event": "earth.layer.changed",
+  "action": "database_changed",
+  "source": "celestrak_tle",
+  "table": "collected_data",
+  "operation": "DELETE",
+  "layers": ["satellites"],
+  "refresh_strategy": "clear_then_reload",
+  "records_processed": 11125,
+  "occurred_at": "2026-05-25T10:20:30Z"
+}
+```
+
+| strategy | Purpose |
+| --- | --- |
+| `clear_then_reload` | Clear local frontend layer objects first, then force a refetch. Prefer this for deletes. |
+| `reload` | Keep old objects until fresh data returns. Use it for location, metadata, or non-destructive updates. |
+| `delta` | Used only for `earth_interactables`; upsert or remove objects by id. |
+
+APIs must return HTTP 200 with an empty collection for real zero-data states; 5xx is reserved for real endpoint failures. After a delete event, if refetch fails, the frontend should keep the cleared state and show a lightweight error instead of restoring stale objects.
+
+## Layer Adapter Contract
+
+`earth_layer_adapters.py` is the single registry for sources, derived tables, Earth layers, cache patterns, and refresh strategy. New layers should be added through an adapter entry, not through one-off button handlers, collector branches, or frontend special cases.
+
+Each adapter must declare:
+
+- Which source or table feeds which Earth layer.
+- Which Earth cache key patterns must be invalidated.
+- Which owned derived tables must be removed during `clear_data`.
+- The default refresh strategy for that layer.
+
+When a source is cleared, the `clear_data` job first deletes `collected_data.source = <source>`, then deletes adapter-owned derived rows. Direct derived-table edits also trigger the outbox, so background jobs, admin APIs, and SQL repair scripts reach Earth as long as they mutate fact or derived tables.
+
+## Operations and Troubleshooting
+
+Check whether outbox rows are piling up:
+
+```sql
+SELECT id, table_name, operation, source, occurred_at
+FROM earth_data_change_events
+WHERE consumed_at IS NULL
+ORDER BY id
+LIMIT 20;
+```
+
+Check that triggers exist:
+
+```sql
+SELECT tgname, tgrelid::regclass
+FROM pg_trigger
+WHERE tgname LIKE 'tr_planet_%_changed_%'
+ORDER BY 2, 1;
+```
+
+Useful log events:
+
+- `earth.db_changes.connected`: the listener connected to PostgreSQL and started listening.
+- `earth.db_changes.outbox_polled`: polling found unconsumed outbox rows.
+- `earth.db_changes.broadcasted`: an Earth refresh broadcast was produced.
+- `data_job.started` / `data_job.completed`: job execution state.
+
+If Earth does not update, check in order: fact table changed, outbox was consumed, adapter covers the `source/table`, the listener is online, frontend WebSocket is connected, and the visualization API returns HTTP 200 with either an empty collection or fresh data.
+
+## Kafka-ready Boundaries
+
+Business code avoids depending on a concrete queue implementation by preserving these boundaries:
+
+- `JobQueue`: submit, claim, cancel, and complete data jobs.
+- `DataChangeBus`: publish database fact changes.
+- `EarthLayerAdapterRegistry`: declare source, layer, cache, and derived-data relationships.
+
+Kafka becomes appropriate when:
+
+- Several independent services must consume the same data-change stream.
+- AIS, BGP, or sensor streams become sustained high-throughput inputs.
+- Consumer groups, replay, and service decoupling are required.
+
+Spark becomes appropriate when:
+
+- Historical data reaches tens or hundreds of millions of rows and PostgreSQL aggregation becomes expensive.
+- Cross-source, long-window, spatiotemporal analysis is needed.
+- Raw data lands in Parquet / Iceberg / Delta and the system starts producing offline derived data products.
+
+For second-level continuous stream processing, evaluate Flink first. Spark is a better fit for batch or micro-batch analytics.
+

docs/technical/en/datasource-collector-settings-connectivity.md

@@ -32,8 +32,8 @@ If endpoint, headers, base configuration, or credential fingerprint changes afte
 
 Files:
 
-- [PlainResourcePages.tsx](/home/ray/dev/linkong/planet/frontend/src/admin-next/pages/PlainResourcePages.tsx)
-- [AdminNextRoutes.tsx](/home/ray/dev/linkong/planet/frontend/src/admin-next/AdminNextRoutes.tsx)
+- [PlainResourcePages.tsx](/home/ray/dev/linkong/planet/frontend/src/admin/pages/PlainResourcePages.tsx)
+- [AdminRoutes.tsx](/home/ray/dev/linkong/planet/frontend/src/admin/AdminRoutes.tsx)
 
 Current behavior:
 
@@ -57,7 +57,7 @@ Current behavior:
 
 File:
 
-- [PlainResourcePages.tsx](/home/ray/dev/linkong/planet/frontend/src/admin-next/pages/PlainResourcePages.tsx)
+- [PlainResourcePages.tsx](/home/ray/dev/linkong/planet/frontend/src/admin/pages/PlainResourcePages.tsx)
 
 Current behavior:
 

docs/technical/en/docs-gatekeeper-development.md

@@ -2,7 +2,7 @@
 
 Docs Gatekeeper moves `/docs` from "bundle all Markdown into the frontend" to "return catalog and content from the backend according to permissions." Its goal is to keep public manuals, user docs, developer docs, and admin/ops docs in one searchable Docs page while making every protected Markdown body pass through a server-side whitelist and authorization check.
 
-For the user workflow, see the Docs section in [Planet Manual](/home/ray/dev/linkong/planet/docs/technical/en/manual.md).
+For the user workflow, see the Docs section in [Intelligent Planet Manual](/home/ray/dev/linkong/planet/docs/technical/en/manual.md).
 
 ## Authorization Model
 
@@ -61,15 +61,15 @@ DocsMetadata(
     "public",
     "Manual",
     2,
-    "Planet 使用手册",
-    "Planet Manual",
+    "智能星球使用手册",
+    "Intelligent Planet Manual",
 )
 ```
 
-When adding a public technical doc:
+When adding a technical doc that should appear in the Docs page:
 
 - Add both Chinese and English Markdown files.
-- Add filename, slug, access, group, order, and titles to server `DOCS_METADATA`.
+- Add filename, slug, access, group, order, and titles to server `DOCS_METADATA`. The backend catalog endpoint is authoritative; frontend metadata alone does not publish a document into `/docs` navigation.
 - Add matching metadata to frontend [docs-content.ts](/home/ray/dev/linkong/planet/frontend/src/pages/Docs/docs-content.ts) so navigation titles and sorting stay aligned.
 - Update `docs/technical/zh/README.md` and `docs/technical/en/README.md` when the document should be discoverable from the README.
 
@@ -83,7 +83,7 @@ The user API:
 - Validates group names on update: only `docs_user`, `docs_developer`, and `docs_admin` are accepted.
 - Allows only `super_admin` to modify Gatekeeper groups.
 
-Frontend [Users.tsx](/home/ray/dev/linkong/planet/frontend/src/pages/Users/Users.tsx) displays group tags and provides a multi-select in the edit form. Non-`super_admin` users see the field disabled, and submission removes `gatekeeper_groups` before sending.
+Frontend [Users.tsx](/home/ray/dev/linkong/planet/frontend/src/admin/pages/Users.tsx) displays group tags and provides a multi-select in the edit form. Non-`super_admin` users see the field disabled, and submission removes `gatekeeper_groups` before sending.
 
 ## Frontend Docs Loading
 

docs/technical/en/earth-bgp-context.md

@@ -188,7 +188,7 @@ Current reality:
 - that is expected, because incidents are aggregated and de-noised
 - but incident-first rendering makes the Earth view look too quiet unless there is another always-available activity layer
 
-Implementation detail for the recommended `activity layer` is expanded in the [BGP Region Aggregation Plan](/home/ray/dev/linkong/planet/docs/plans/earth-bgp-region-aggregation-plan.md).
+Implementation detail for the recommended `activity layer` is kept in the repository path `docs/plans/earth-bgp-region-aggregation-plan.md`.
 
 So the immediate next milestone is:
 
@@ -259,7 +259,7 @@ Reference inspiration:
 
 Relevant page:
 
-- `frontend/src/pages/BGP/BGP.tsx`
+- `frontend/src/admin/pages/PlainResourcePages.tsx`
 
 Current BGP console page has three levels:
 
@@ -321,7 +321,7 @@ Backend:
 
 Frontend:
 
-- `frontend/src/pages/BGP/BGP.tsx`
+- `frontend/src/admin/pages/PlainResourcePages.tsx`
 - `frontend/public/earth/js/bgp.js`
 - `frontend/public/earth/js/main.js`
 - `frontend/public/earth/js/info-card.js`

docs/technical/en/earth-frontend-context.md

@@ -4,7 +4,7 @@ This document describes the current real structure of the Earth display frontend
 
 Related references:
 
-- [Project Rules](/home/ray/dev/linkong/planet/rules.md)
+- Repository root `rules.md`
 - [Frontend Layout Guidelines](/home/ray/dev/linkong/planet/docs/technical/en/frontend-layout-guidelines.md)
 
 ## Current Goal
@@ -56,6 +56,8 @@ Responsibilities:
 - Layer module integration
 - Earth-level state synchronization
 
+Earth treats `/ws` `earth_updates` as refresh hints only; real data is fetched again from `/api/v1/visualization/...`. Backend database-driven refresh now has the listener clear cache and broadcast directly instead of going through the default `earth_refresh` job queue. After `database_changed`, the frontend applies the per-layer `clear_then_reload`, `reload`, or `delta` strategy. `clear_then_reload` must clear Three.js objects before a no-store refetch, and summary is only a consistency check, not a reason to skip a layer reload when the count is `0`. See [Data Jobs and Outbox Architecture](/home/ray/dev/linkong/planet/docs/technical/en/data-job-earth-sync-architecture.md) for the technical pipeline and [Business Architecture and Data Flows](/home/ray/dev/linkong/planet/docs/technical/en/platform-data-flows.md) for business data flows.
+
 ### 3. Earth Control Layer
 
 - [controls.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/controls.js)
@@ -128,7 +130,7 @@ Responsibilities:
 - Terrain tile fetch, decode, displacement, and shading
 - Whole-globe land/ocean and border base overlays
 
-The Earth surface is a stack of near-concentric shells, not a single mesh. The base sphere and HD texture overlay in `earth.js`, plus the land/ocean base in `country-boundaries.js`, need explicit radius separation. At far zoom, GPU depth precision drops; neighboring shells that are too close can z-fight and show black flicker blocks or snow. The current stable spacing is `landAltitudeOffset = 0.32` and `textureOverlayAltitudeOffset = 0.48`. When adding or adjusting whole-globe surface overlays, update [Earth Render Layer Order](/home/ray/dev/linkong/planet/docs/technical/en/earth-render-layer-order.md) and verify at 50% zoom.
+The Earth surface is a stack of near-concentric shells, not a single mesh. The base sphere and HD texture overlay in `earth.js`, plus the land/ocean base in `country-boundaries.js`, need explicit radius separation. At far zoom, GPU depth precision drops; neighboring shells that are too close can z-fight and show black flicker blocks or snow. The current stable spacing is `landAltitudeOffset = 0.32` and `EARTH_SURFACE_TEXTURE_ALTITUDE_OFFSET = 0.48` for the HD texture shell. Country borders, coastlines, claim lines, and country hover lines must use that same HD texture shell radius so they do not drift relative to the texture while the globe rotates. When adding or adjusting whole-globe surface overlays, update [Earth Render Layer Order](/home/ray/dev/linkong/planet/docs/technical/en/earth-render-layer-order.md) and verify at 50% zoom.
 
 ### 7. Layer Modules
 
@@ -151,11 +153,11 @@ Each module is responsible for its own:
 
 `brand.js` manages Earth HUD brand resources. Static assets provide the default brand; runtime overrides come from `/api/v1/earth/brand`, and uploaded images are served from `/earth-brand-assets/...`. The frontend must treat logo/title images and text fallback separately: if an image fails, show the text title; if text fields are empty, rely on backend defaults so the HUD brand area never renders blank. The console Earth Content page owns saving and resetting brand configuration; the Earth frontend only consumes it.
 
-`about.js` manages the About card inside Earth settings. Frontend defaults remain as a fallback, while runtime content is loaded from `/api/v1/earth/about`. If the request fails or fields are missing, the renderer must fall back per field so the settings page never renders an empty card. Admin Next exposes an Earth Content `About` tab; saving uses `PUT /api/v1/earth/about`, and restoring defaults uses `DELETE /api/v1/earth/about`.
+`about.js` manages the About card inside Earth settings. Frontend defaults remain as a fallback, while runtime content is loaded from `/api/v1/earth/about`. If the request fails or fields are missing, the renderer must fall back per field so the settings page never renders an empty card. Admin exposes an Earth Content `About` tab; saving uses `PUT /api/v1/earth/about`, and restoring defaults uses `DELETE /api/v1/earth/about`.
 
 `oobe.js` manages the first-run Earth initialization guide. OOBE visibility must be driven by `/api/v1/earth/oobe-status` and its `ready` field, not by `localStorage`. `localStorage` may only store a short-lived "skip on this browser" flag; if the backend reports `ready: true`, logout, cleared browser storage, or a different browser must not show OOBE again. Desktop uses a dark starfield scrim and glass startup panel, while mobile uses a bottom sheet and respects `prefers-reduced-motion`.
 
-The Admin Next Earth Content page must preserve runtime semantics:
+The Admin Earth Content page must preserve runtime semantics:
 
 - `Brand`: brand preview should use the same dark starfield background, size, spacing, logo/title rendering, and text fallback as the Earth HUD top-left brand block, not a generic form preview.
 - `About`: configures the About card in Earth settings, including logo, kicker, title, version, description, and metadata items. Earth runtime reads `/earth/about` and falls back to defaults on failure.
@@ -311,6 +313,18 @@ If future cable, satellite, or news cruise is added, do not copy a new set of `m
 - [callout-connector.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/callout-connector.js)
 - The business adapter pattern from [bgp-cruise-adapter.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/bgp-cruise-adapter.js)
 
+## View Control Feedback
+
+[controls.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/controls.js) owns the Earth zoom state, and every zoom entry point must ultimately call `setZoomLevel()` to write the camera distance. Do not write `camera.position.z` from other modules, or the zoom percentage, drag sensitivity, and Interactable clustering thresholds will diverge again.
+
+Interactable clustering is selected per layer through `cluster.strategy`. `stable-spherical` uses discrete zoom bands and local 3D bucket clustering, so BGP, compute centers, and Earth interactables do not regroup while the globe rotates inside the same band. `dynamic-screen` keeps the projection-based behavior for high-frequency realtime layers such as vessels, and `none` disables clustering. Stable cluster dots stay rigidly aligned to their 3D centroid projection and do not participate in 2D avoidance. See [Earth Interactable Clustering](/home/ray/dev/linkong/planet/docs/technical/en/earth-interactable-clustering.md) for strategy configuration and tuning.
+
+Wheel input has two paths. Traditional mouse wheels keep the 10% step and short animation, using `wheelZoomTarget` as the logical base for continuous wheel input. Trackpads and high-precision wheels use the pixel delta for continuous zoom and call `setZoomLevel()` directly instead of passing through the 10% stepped animation. The trackpad path also filters a short-window, old-direction residual delta after a real direction change so inertia tails do not pull a just-reversed zoom back in the previous direction.
+
+The gesture capsule updates at most once every 90ms and fades after 760ms. It is view feedback, not data loading progress, and should not be written into layer loading state.
+
+[main.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/main.js) only calls `setZoomLevel()` and `showZoomStatusCapsule()` for pinch zoom and motion zoom. Mouse wheel and zoom-button capsule feedback should stay in `controls.js` so the same zoom feedback does not spread across modules.
+
 ## Recommended Change Approach
 
 For future Earth changes:

docs/technical/en/earth-interactable-clustering.md

@@ -0,0 +1,84 @@
+# Intelligent Planet Interactable Clustering
+
+Earth interactable icons are managed by `createInteractableLayer`. Rendering still uses Three.js `Points`, and picking, hover, locked state, tooltips, and cruise focus still depend on marker `userData`; the clustering strategy only decides which markers are represented by a cluster dot.
+
+## Strategies
+
+`cluster.strategy` supports three modes:
+
+- `stable-spherical`: stable spherical clustering. It clusters by local 3D positions on the globe and caches topology by zoom band. Rotation and small zoom changes within the same band only update projection and size. Use it for semi-static layers such as BGP, compute centers, and Earth interactables.
+- `dynamic-screen`: dynamic screen-space clustering. This keeps the previous projection-based behavior and evaluates visible relationships per frame. Use it for high-frequency realtime layers such as vessels, or as a fallback.
+- `none`: no clustering. Every marker is shown independently. Use it for low-count layers or precision-first views.
+
+`cluster: false` is equivalent to `strategy: "none"`. If a layer enables clustering without declaring a strategy, it keeps the compatible `dynamic-screen` behavior.
+
+## Stable Spherical
+
+`stable-spherical` moves cluster identity from screen distance to globe distance:
+
+- Each marker uses `icon_base_position` as its geographic anchor.
+- By default, zoom levels above `2.5` force clustering off and show every marker as its original icon.
+- The current zoom maps to a discrete band; rotation and small zoom changes inside the same band do not recompute topology.
+- Clusters are recomputed only when the band, data revision, visibility, or filter state changes.
+- A cluster centroid is computed from member 3D positions and normalized back to the globe shell, so the cluster dot stays rigidly aligned to its geographic center.
+- Cluster dots do not participate in 2D avoidance, so screen-space repulsion cannot push them away from their real geographic projection.
+- Band changes use a small hysteresis margin so zooming at a boundary does not repeatedly bounce between two bands.
+- Newly created marker and cluster dots run a short scale + opacity ease. This is only a rendering transition; it does not change marker coordinates, picking objects, or locked state.
+
+The stable strategy uses spherical bucket/hash neighbor lookup and must not use an all-pairs loop. Most frames only pay projection and material-size cost; topology cost is paid only on band or data changes.
+
+## Configuration
+
+```js
+const computeCenterIconLayer = createInteractableLayer({
+  id: "computeCenters",
+  // ...
+  avoidance: SURFACE_AVOIDANCE_PROFILES.city,
+  cluster: {
+    strategy: "stable-spherical",
+    minCount: 2,
+    maxMarkersPerDot: 14,
+    transitionMs: 220,
+    bandHysteresis: 0.08,
+    disableAboveZoom: 2.5,
+    bands: [
+      { key: "far", maxZoom: 1.7, distance: 15 },
+      { key: "mid", maxZoom: 2.6, distance: 8 },
+      { key: "near", maxZoom: 3.5, distance: 4 },
+      { key: "detail", maxZoom: Infinity, distance: 0 },
+    ],
+  },
+});
+```
+
+Realtime layers can stay dynamic:
+
+```js
+const vesselIconLayer = createInteractableLayer({
+  id: "vessels",
+  // ...
+  cluster: {
+    strategy: "dynamic-screen",
+    enabled: true,
+    maxMarkersPerDot: 10,
+  },
+});
+```
+
+## Tuning
+
+- `distance` is the 3D globe-distance threshold and uses the same unit as `CONFIG.earthRadius`. Larger values cluster more aggressively.
+- The farthest band usually uses a larger `distance`; the nearest detail band usually uses `0` to split clusters into original icons.
+- `bandHysteresis` controls band-boundary stickiness. Too little can flicker near thresholds; too much can make band changes feel late.
+- `transitionMs` controls cluster split/merge easing. Keep it around 160-260ms; longer durations can make realtime layers feel sluggish.
+- `disableAboveZoom` controls the precision-view threshold. It defaults to `2.5`; above that zoom, no cluster dots are generated. Set it to `false` to disable the hard threshold.
+- High-frequency realtime layers should prefer `dynamic-screen` so frequent data changes do not trigger stable topology recomputation.
+- If a layer behaves poorly, switch it back to `dynamic-screen` or use `cluster: false`.
+
+## Acceptance Checks
+
+- Rotating the globe inside one zoom band should not cause clusters to flicker or regroup.
+- Cluster dots should stay aligned with the globe-surface centroid and should not be pushed by avoidance.
+- Zooming into the detail band should restore the layer's original icon textures and click behavior.
+- Zoom levels above 250% should not show cluster dots.
+- Realtime layers such as vessels should reflect incoming updates immediately.

docs/technical/en/earth-render-layer-order.md

@@ -18,14 +18,14 @@ Note: the layer control panel order and the registration / startup load order ar
 | 0 | Earth base sphere | `earth.js` | `CONFIG.earthRadius` | Surface picking fallback target | Dark base; still visible when all optional map layers are off. |
 | 0.2 | Country dark tint | `country-boundaries.js` | `tintAltitudeOffset` | Raycast disabled | Used when HD texture is off. |
 | 0.86 | Land/ocean base fill | `country-boundaries.js` | `landAltitudeOffset = 0.32`; ocean `#010609`, land `#080f1b` | Raycast disabled | Base map remains usable even when country borders are off; radius is separated from the base sphere to avoid far-zoom z-fighting. |
-| 0.96 | HD Earth texture | `earth.js` | `textureOverlayAltitudeOffset = 0.48` | Surface picking target when visible | HD texture always overlays the land/ocean base fill; radius must stay above the land/ocean base and far enough from the base sphere. |
+| 0.96 | HD Earth texture | `earth.js` | `textureOverlayAltitudeOffset = EARTH_SURFACE_TEXTURE_ALTITUDE_OFFSET = 0.48` | Surface picking target when visible | HD texture always overlays the land/ocean base fill; radius must stay above the land/ocean base and far enough from the base sphere. |
 | 1 | Atmospheric glow and clouds | `earth.js` | Atmosphere / cloud spheres | Not in normal object selection path | Cloud layer controlled by the "Cloud Layer" toggle. |
 | 1 | Submarine cables | `cables.js` | `CABLE_CONFIG.line.renderOrder` | Cable picking path | Preserves existing cable layer level. |
 | 1.2 | Real terrain | `earth.js`, `terrain.js` | `TERRAIN_CONFIG.baseRadiusOffset` plus terrain displacement | Raycast disabled | Terrain overlays HD texture; temporarily hidden when HD texture is off, restores to prior state when re-enabled. |
 | 2.05 | Grid lines | `earth.js` | `CONFIG.earthRadius + 0.14` | Raycast disabled | Low-opacity lines over HD texture. |
-| 2.2 | Country borders | `country-boundaries.js` | `lineAltitudeOffset` | Raycast disabled | Only needs to stay above HD texture. |
+| 2.2 | Country borders | `country-boundaries.js` | `lineAltitudeOffset = EARTH_SURFACE_TEXTURE_ALTITUDE_OFFSET = 0.48`; claim lines have no extra lift | Raycast disabled | Line geometry still has its own `renderOrder`, but it shares the exact same radius as the HD texture shell to avoid parallax while the globe rotates. |
 | 2.29 | Country border hover glow | `country-boundaries.js` | Hover radius + glow offset | `depthTest: false`, raycast disabled | Additive glow to reinforce border edge and terrain hover visibility. |
-| 2.3 | Country border hover line | `country-boundaries.js` | `hoverAltitudeOffset` | `depthTest: false`, raycast disabled | Neon red-orange hover line; China and Taiwan share the same highlight group. |
+| 2.3 | Country border hover line | `country-boundaries.js` | `hoverAltitudeOffset = EARTH_SURFACE_TEXTURE_ALTITUDE_OFFSET = 0.48` | `depthTest: false`, raycast disabled | Neon red-orange hover line; aligned with the normal borders and HD texture shell to avoid ghosting or floating; China and Taiwan share the same highlight group. |
 | 3 | Satellite footprint fill / Iridium coverage ring | `satellites.js`, `iridium-footprint-adapter.js` | `GROUND_FOOTPRINT_RENDER_ORDER` | depth-tested; Iridium adapter fill / ring use the same renderOrder | Footprint above land / texture / terrain and country borders, below compute centers and satellites. |
 | 3-5 | BGP markers and overlays | `bgp.js` | Each marker's own renderOrder | BGP picking path | Preserves existing BGP visual level. |
 | 4.5 | Compute centers | `compute-centers.js` | `COMPUTE_CENTER_RENDER_ORDER` | Compute center picking path | Surface facilities, below satellites. |
@@ -52,7 +52,8 @@ The Earth surface is not a single mesh. It is a stack of near-concentric shells:
 Maintenance rules:
 
 - Do not reach first for hiding layers at far zoom. Check neighboring shell `altitudeOffset`, `renderOrder`, `depthTest`, and `depthWrite` first.
-- Whole-globe overlays such as the land/ocean base and HD texture must keep explicit separation from `CONFIG.earthRadius`; the current stable values are `landAltitudeOffset = 0.32` and `textureOverlayAltitudeOffset = 0.48`.
+- Whole-globe overlays such as the land/ocean base and HD texture must keep explicit separation from `CONFIG.earthRadius`; the current stable values are `landAltitudeOffset = 0.32` and `textureOverlayAltitudeOffset = EARTH_SURFACE_TEXTURE_ALTITUDE_OFFSET = 0.48`.
+- Country borders, coastlines, claim lines, and country hover lines must use `EARTH_SURFACE_TEXTURE_ALTITUDE_OFFSET`, exactly matching the HD texture shell. Do not give border lines an independent lower or higher offset, because globe rotation will make the lines appear to drift relative to the surface texture.
 - Any new whole-globe or near-whole-globe surface overlay must be screenshot-verified at 50% zoom and at common close zooms, with no black blocks, snow, flicker, or obvious floating.
 - If these radii change, update this document and the intent around the constants in `frontend/public/earth/js/constants.js`.
 

docs/technical/en/earth-satellite-footprint-policy.md

@@ -17,26 +17,21 @@ Related context:
 
 ## Current Local Categories
 
-Current CelesTrak satellite groups in [backend/app/services/collectors/celestrak.py](/home/ray/dev/linkong/planet/backend/app/services/collectors/celestrak.py) include:
+The CelesTrak collector now prefers the complete active satellite catalog from `GROUP=active&FORMAT=json`. This prevents one failed CelesTrak group request from being saved as a successful but incomplete batch. The collector only proceeds when the downloaded JSON is a parseable array and records include `NORAD_CAT_ID`; network, resume, or parsing failures are retried, and final failure preserves the previous current dataset.
 
-- `starlink`
-- `gps-ops`
-- `galileo`
-- `glonass`
-- `beidou`
-- `leo`
-- `geo`
-- `iridium-next`
+CelesTrak applies a repeat-download window to large groups such as `active`. To make database resets recoverable, Planet stores raw downloads under `$PLANET_CACHE_DIR/downloads`. When `active` returns the CelesTrak "GP data has not updated" HTTP 403:
 
-Non-Starlink categories:
+- If an `active` cache exists, the collector reuses it to repopulate the database.
+- If no `active` cache exists, the collector switches to fallback group mode and downloads `starlink`, `gps-ops`, `galileo`, `glonass`, `beidou`, `leo`, `geo`, and `iridium-next`.
+- Fallback group mode requires every group to succeed or have a reusable cache; if any group is missing, the whole collection fails and no partial dataset is saved.
 
-- `gps-ops`
-- `galileo`
-- `glonass`
-- `beidou`
-- `leo`
-- `geo`
-- `iridium-next`
+The collector still provides `metadata.constellation_group` to the frontend, but the value now comes from executable inference:
+
+- `OBJECT_NAME` starting with `STARLINK` is marked as `starlink`
+- `OBJECT_NAME` starting with `IRIDIUM` is marked as `iridium-next`
+- Other active satellites are not forced into the old CelesTrak small-group labels, because a broad source group is not an exact constellation
+
+The product policy therefore still discusses GNSS/RNSS, GEO, generic LEO, and Iridium NEXT semantics. However, only fallback group mode stores the old `gps-ops`, `galileo`, `glonass`, `beidou`, `leo`, or `geo` labels; the active primary path does not force every satellite into those labels.
 
 ## Research Conclusions
 
@@ -136,7 +131,8 @@ This implementation only does the minimum executable version and does not change
 
 1. Backend passes constellation group and footprint policy hint to the frontend
 
-- CelesTrak collector stores `GROUP` in `metadata.constellation_group`
+- CelesTrak collector stores the source query in `metadata.celestrak_query_group = active`
+- `metadata.constellation_group` only stores inferred business constellations such as `starlink` and `iridium-next`
 - Visualization API outputs:
   - `properties.constellation_group`
   - `properties.footprint_policy`

docs/technical/en/frontend-admin-frontend-context.md

@@ -4,7 +4,7 @@ This document describes the current real structure of the console frontend. The
 
 Related references:
 
-- [Project Rules](/home/ray/dev/linkong/planet/rules.md)
+- Repository root `rules.md`
 - [Frontend Layout Guidelines](/home/ray/dev/linkong/planet/docs/technical/en/frontend-layout-guidelines.md)
 
 ## Current Goal
@@ -22,7 +22,7 @@ Main entry point:
 
 - [App.tsx](/home/ray/dev/linkong/planet/frontend/src/App.tsx)
 
-Admin Next now owns the official admin routes:
+Admin now owns the official admin routes:
 
 - `/admin`
 - `/users`
@@ -37,23 +37,7 @@ Admin Next now owns the official admin routes:
 - `/collection-management`
 - `/settings`
 
-These routes render [AdminNextRoutes.tsx](/home/ray/dev/linkong/planet/frontend/src/admin-next/AdminNextRoutes.tsx). Page metadata and menu entries come from [manifest.tsx](/home/ray/dev/linkong/planet/frontend/src/admin-next/routes/manifest.tsx). `/admin-next/*` remains only as a compatibility entry and redirects to the official route; it is no longer a parallel primary entry.
-
-The old AntD console remains available under `/legacy/admin/*` for comparison and rollback:
-
-- `/legacy/admin`
-- `/legacy/admin/datasources`
-- `/legacy/admin/data`
-- `/legacy/admin/collection-management`
-- `/legacy/admin/earth-content`
-- `/legacy/admin/ai`
-- `/legacy/admin/logs`
-- `/legacy/admin/settings`
-- `/legacy/admin/users`
-- `/legacy/admin/bgp`
-- `/legacy/admin/alerts/*`
-
-Legacy pages, `AppLayout`, `antd`, and `@ant-design/icons` stay in place during the legacy validation window. Do not remove them before Admin Next parity is accepted.
+These routes render [AdminRoutes.tsx](/home/ray/dev/linkong/planet/frontend/src/admin/AdminRoutes.tsx). Page metadata and menu entries come from [manifest.tsx](/home/ray/dev/linkong/planet/frontend/src/admin/routes/manifest.tsx). Admin is the only console entry point; there is no parallel console or rollback route.
 
 `/earth` is a standalone display page and is not part of the console shell.
 
@@ -61,7 +45,7 @@ Legacy pages, `AppLayout`, `antd`, and `@ant-design/icons` stay in place during
 
 The official admin shell is at:
 
-- [AdminNextLayout.tsx](/home/ray/dev/linkong/planet/frontend/src/admin-next/components/layout/AdminNextLayout.tsx)
+- [AdminLayout.tsx](/home/ray/dev/linkong/planet/frontend/src/admin/components/layout/AdminLayout.tsx)
 
 Responsibilities:
 
@@ -69,38 +53,13 @@ Responsibilities:
 - Current account, version, logout, and theme switching
 - Top search, breadcrumbs, and page shortcuts
 - Single-screen content-area height closure
-- Coordination for Admin Next internal scrolling, tables, detail panels, and mobile detail views
+- Coordination for Admin internal scrolling, tables, detail panels, and mobile detail views
 
-The old AntD legacy shell remains at:
+Future official admin pages should adapt to `AdminLayout` and Admin page patterns; do not reintroduce a parallel admin shell.
 
-- [AppLayout.tsx](/home/ray/dev/linkong/planet/frontend/src/components/AppLayout/AppLayout.tsx)
+## Admin Section Loading
 
-Legacy responsibilities:
-
-- Left-side navigation
-- Collapse and expand
-- Current account / version information
-- Content area height closure
-- Site-wide unified sidebar scrollbar
-
-Current structure:
-
-```tsx
-<Layout className="dashboard-layout">
-  <Sider className="dashboard-sider">...</Sider>
-  <Layout>
-    <Content className="dashboard-content">
-      <div className="dashboard-content-inner">{children}</div>
-    </Content>
-  </Layout>
-</Layout>
-```
-
-Future official admin pages should adapt to `AdminNextLayout` and Admin Next page patterns rather than adding new capability to the old `AppLayout`. Only `/legacy/admin/*` maintenance should change the old shell.
-
-## Admin Next Section Loading
-
-Multi-tab pages are currently coordinated by [PlainResourcePages.tsx](/home/ray/dev/linkong/planet/frontend/src/admin-next/pages/PlainResourcePages.tsx), which hosts the current management and information workbench patterns. Section loading follows these rules:
+Multi-tab pages are currently coordinated by [PlainResourcePages.tsx](/home/ray/dev/linkong/planet/frontend/src/admin/pages/PlainResourcePages.tsx), which hosts the current management and information workbench patterns. Section loading follows these rules:
 
 - Initial page load requests only the active tab; it does not prefetch every tab endpoint.
 - Switching tabs lazily loads that tab. Loaded tabs stay cached in local `states`, so returning to a tab reuses the previous data.
@@ -112,24 +71,52 @@ This keeps Earth, AI, collection management, and other multi-section pages from
 
 ## Datasource Collection Queue
 
-The Admin Next datasource page routes single-source trigger, batch trigger, and trigger-all into a browser-download-list style collection queue:
+The Admin datasource page routes single-source trigger, table-selected trigger, and trigger-all into a browser-download-list style collection queue:
 
-- Queue state is managed by [PlainResourcePages.tsx](/home/ray/dev/linkong/planet/frontend/src/admin-next/pages/PlainResourcePages.tsx). It is a current-session visibility layer and does not fake task history in `localStorage`.
+- Queue state is managed by [PlainResourcePages.tsx](/home/ray/dev/linkong/planet/frontend/src/admin/pages/PlainResourcePages.tsx). It is a current-session visibility layer and does not fake task history in `localStorage`.
 - Progress first consumes the `/ws` `datasource_tasks` channel. If the socket is unavailable or stale, the page polls `/api/v1/datasources/{id}/task-status`.
 - Trigger responses immediately insert `triggered`, `skipped`, and `failed` items. After a refresh, the queue restores only real backend rows that are still `running`, `pending`, or `queued`.
-- The page shows a compact progress bar and counts above the table. The "view queue" action opens a bottom panel grouped by running, failed, completed, and skipped.
+- The `Built-in Sources` section uses the table selection column for selected-source triggering. With no rows selected, the primary button is `Trigger All`; after selection, the same button becomes `Trigger Selected N`, replacing the old manual-ID batch button.
+- The expanded queue no longer lives in the page content flow, so trigger-all cannot squeeze the table and detail panel. The top-right actions area uses the existing `Button` styling; the empty state shows a `ListChecks` icon, and active queues show only a pure circular total-progress indicator. Clicking it opens a floating panel grouped by running, failed, completed, and skipped.
 - Queue rows can jump to the datasource detail panel, and failed rows can retry. The detail panel's task summary only reports the selected source's latest task; it does not save configuration.
+- While a single-source collection is running, the action button becomes `Stop Collection`; unfinished queue rows expose a cancel action on the right. Cancellation calls `/api/v1/datasources/{source_id}/tasks/{task_id}/cancel`; the backend keeps committed batches and rolls back unfinished work.
+- Clearing database data and clearing display cache enter the same queue. The frontend shows task state and does not assume the API completed synchronously.
 
 This queue is a user-perception layer. Backend task status remains the only source of truth for running, completion, failure, and skipped decisions.
 
-## Admin Next Theme Slider
+## Admin Theme Slider
 
-The Admin Next sidebar theme switcher still reuses shared [SegmentedControl.tsx](/home/ray/dev/linkong/planet/frontend/src/components/SegmentedControl/SegmentedControl.tsx), while [styles.css](/home/ray/dev/linkong/planet/frontend/src/admin-next/styles.css) overrides the segment variables by `data-theme`:
+The Admin sidebar theme switcher still reuses shared [SegmentedControl.tsx](/home/ray/dev/linkong/planet/frontend/src/components/SegmentedControl/SegmentedControl.tsx), while [styles.css](/home/ray/dev/linkong/planet/frontend/src/admin/styles.css) overrides the segment variables by `data-theme`:
 
 - Light mode uses `--d-segment-bg: #eef3f9`, a white slider, and a light external shadow.
 - Dark mode uses the same dark base, `#202938` slider, and dark external shadow semantics as Docs.
 - Product code only hides the text label and keeps icon + tooltip behavior; it should not recreate the private slider DOM.
 
+## Admin Status Colors
+
+Admin status labels should use [StatusText](/home/ray/dev/linkong/planet/frontend/src/admin/patterns/patterns.tsx) or [Badge](/home/ray/dev/linkong/planet/frontend/src/admin/components/ui/badge.tsx). Color variables come from [styles.css](/home/ray/dev/linkong/planet/frontend/src/admin/styles.css). New states should map to an existing tone instead of adding page-local hex colors.
+
+`StatusText` is an indicator-light pill: the pill background and border stay on the component base color, while only the dot and text use the status color. `Badge` does not carry the indicator-light meaning, so it may use a light same-tone background and border for stronger hierarchy.
+
+| Tone | Color variable | Meaning | Examples |
+| --- | --- | --- | --- |
+| `success` | `--an-success` | available, successful, connected, enabled | log source `Available`, collection `Success` |
+| `warning` | `--an-warning` | needs attention but is not necessarily failed | missing log file, degraded or skipped state |
+| `danger` | `--an-danger` | failed, unavailable, permission/connection error | `Docker unavailable`, endpoint failure |
+| `info` / `running` | `--an-info` | in progress, syncing, informational | `Following`, `Syncing` |
+| `neutral` | `--an-muted` | empty, disabled, not reported yet, unknown | `No reports yet`, disabled |
+| `ai` | fixed purple | AI-specific emphasis | AI generation or model action |
+
+The log source list follows the same rule: `ok` is success; `empty` means the source exists but has not reported yet and is neutral; `missing` means an expected log file is not present and is warning; `docker_unavailable` / `source_unavailable` are danger.
+
+## Admin Runtime Logs
+
+Admin runtime errors are reported through [runtimeLogs.ts](/home/ray/dev/linkong/planet/frontend/src/admin/runtimeLogs.ts) to `/api/v1/system/logs/admin-client`. The reporter only runs on Admin routes and skips `/earth`, `/docs`, login, and registration pages so public-page noise does not enter the Admin client log source.
+
+[AdminErrorBoundary.tsx](/home/ray/dev/linkong/planet/frontend/src/admin/components/AdminErrorBoundary.tsx) catches React render failures and reuses the same reporter; global `error` and `unhandledrejection` events use that channel as well. Reporting failures must stay silent, because the logging path must not create another frontend error.
+
+The Logs page follows log increments through the `/ws` `logs_tail` channel. File logs and database logs are both normalized into line events by the backend. When adding a new log source, wire it through the backend source registry and tail manager instead of adding a page-local poller.
+
 ## Current Shared Components
 
 ### 1. `Scrollbar`
@@ -163,7 +150,7 @@ Purpose:
 
 Current usage:
 
-- Admin Next data sources, collected data, collection management, logs, alerts, and BGP pages
+- Admin data sources, collected data, collection management, logs, alerts, and BGP pages
 - Old AntD legacy pages continue using shared scrolling behavior through compatibility wrappers
 
 ### 3. `TableScrollRegion`
@@ -187,7 +174,7 @@ Files:
 
 Purpose:
 
-- Admin Next global tool buttons and detail-panel toolbars
+- Admin global tool buttons and detail-panel toolbars
 - Icon-only ordinary actions with tooltips
 - Strong-intent actions such as save, create, confirm, delete, and stop
 - Compact switches aligned with the Docs theme slider
@@ -245,39 +232,19 @@ Current constraints:
 - Internal document links should be converted to `/docs/:slug` through `transformLink`
 - Heading anchors are injected through `getHeadingId`, keeping route state outside the renderer
 
-### 7. `ConnectionTestInput`
+### 7. Admin UI Primitives
 
-File:
+Files:
 
-- [ConnectionTestInput.tsx](/home/ray/dev/linkong/planet/frontend/src/components/ConnectionTestInput/ConnectionTestInput.tsx)
+- [button.tsx](/home/ray/dev/linkong/planet/frontend/src/admin/components/ui/button.tsx)
+- [dialog.tsx](/home/ray/dev/linkong/planet/frontend/src/admin/components/ui/dialog.tsx)
+- [switch.tsx](/home/ray/dev/linkong/planet/frontend/src/admin/components/ui/switch.tsx)
 
-Purpose:
+Use:
 
-- Console form fields that combine an endpoint/Base URL value with a connection check
-- Connection-test entry points for AI Provider and WebSearch
-- Future collector configuration fields should reuse it when the test action belongs inside the input
-
-Current constraints:
-
-- The input suffix shows a single plug/connector icon, not an adjacent text button
-- Disabled integrations must grey out both the input and its connection-test action
-- The component only combines the input and action; callers still own form state, loading, disabled state, and the request itself
-
-### 8. `TableActions`
-
-File:
-
-- [TableActions.tsx](/home/ray/dev/linkong/planet/frontend/src/components/TableActions/TableActions.tsx)
-
-Purpose:
-
-- Shared action entry for table operation columns
-- Shows inline actions when expanded
-- Uses a more-actions dropdown when collapsed
-
-Companion export:
-
-- `actionCellProps`: for action-column `onCell`, preventing action buttons from being ellipsized or wrapped
+- Global tool buttons, detail actions, confirmation dialogs, and binary settings.
+- Aligned with Tactile UI tokens so Admin controls keep consistent size, hover, disabled, and dark-mode behavior.
+- Row actions should prefer icon buttons plus tooltip/title; do not reintroduce a separate action-menu component.
 
 ## Current State Sources
 
@@ -300,7 +267,7 @@ Responsibilities:
 
 File:
 
-- [AISettings.tsx](/home/ray/dev/linkong/planet/frontend/src/pages/AISettings/AISettings.tsx)
+- [AISettings.tsx](/home/ray/dev/linkong/planet/frontend/src/admin/pages/PlainResourcePages.tsx)
 
 Responsibilities:
 
@@ -336,7 +303,7 @@ Constraints:
 
 Example:
 
-- [Dashboard.tsx](/home/ray/dev/linkong/planet/frontend/src/pages/Dashboard/Dashboard.tsx)
+- [Dashboard.tsx](/home/ray/dev/linkong/planet/frontend/src/admin/pages/Dashboard.tsx)
 
 Priority goals:
 
@@ -348,10 +315,10 @@ Priority goals:
 
 Examples:
 
-- [DataSources.tsx](/home/ray/dev/linkong/planet/frontend/src/pages/DataSources/DataSources.tsx)
-- [DataList.tsx](/home/ray/dev/linkong/planet/frontend/src/pages/DataList/DataList.tsx)
-- [Users.tsx](/home/ray/dev/linkong/planet/frontend/src/pages/Users/Users.tsx)
-- [Settings.tsx](/home/ray/dev/linkong/planet/frontend/src/pages/Settings/Settings.tsx)
+- [DataSources.tsx](/home/ray/dev/linkong/planet/frontend/src/admin/pages/PlainResourcePages.tsx)
+- [DataList.tsx](/home/ray/dev/linkong/planet/frontend/src/admin/pages/DataList.tsx)
+- [Users.tsx](/home/ray/dev/linkong/planet/frontend/src/admin/pages/Users.tsx)
+- [Settings.tsx](/home/ray/dev/linkong/planet/frontend/src/admin/pages/PlainResourcePages.tsx)
 
 Constraints:
 
@@ -361,7 +328,7 @@ Constraints:
 
 ### Datasource Directory Page
 
-[DataSources.tsx](/home/ray/dev/linkong/planet/frontend/src/pages/DataSources/DataSources.tsx) is the datasource directory and collection operation page. It should not grow back into a configuration editor.
+[DataSources.tsx](/home/ray/dev/linkong/planet/frontend/src/admin/pages/PlainResourcePages.tsx) is the datasource directory and collection operation page. It should not grow back into a configuration editor.
 
 Current page boundary:
 
@@ -375,7 +342,9 @@ Keep this boundary: do not put custom datasource editing, built-in endpoint over
 
 ### Collectors Page
 
-[Settings.tsx](/home/ray/dev/linkong/planet/frontend/src/pages/Settings/Settings.tsx) has three route modes: `/settings` for System Settings, `/earth-content` for Earth Content, and `/collection-management` for Collection Management. The `collector_credentials` tab is shown as `Collectors` under `/collection-management`.
+[Settings.tsx](/home/ray/dev/linkong/planet/frontend/src/admin/pages/PlainResourcePages.tsx) has three route modes: `/settings` for System Settings, `/earth-content` for Earth Content, and `/collection-management` for Collection Management. The `collector_credentials` tab is shown as `Collectors` under `/collection-management`.
+
+The `System Display` section under `/settings` includes the `Demo Mode` switch. When enabled, Earth OOBE ignores existing current collected data and the local `browse first` temporary skip state, then opens the initialization guide directly. This switch is only for demos and acceptance checks; it does not change datasources, collection queues, or Earth content resources.
 
 Current boundary:
 
@@ -387,7 +356,7 @@ Current boundary:
 
 ### Earth Content Page
 
-`/earth-content` reuses the same single-screen tab container from [Settings.tsx](/home/ray/dev/linkong/planet/frontend/src/pages/Settings/Settings.tsx), but its ownership is separate from System Settings:
+`/earth-content` reuses the same single-screen tab container from [Settings.tsx](/home/ray/dev/linkong/planet/frontend/src/admin/pages/PlainResourcePages.tsx), but its ownership is separate from System Settings:
 
 - `TV Livestream` owns the Earth media-panel source configuration.
 - `Boundary Precision` owns the Earth static boundary asset state: provider, low-precision fallback, high-precision manifest/PMTiles, source JSON, and build action.
@@ -399,8 +368,8 @@ Do not add Earth experience resources or collection-lifecycle tabs back into `/s
 
 Examples:
 
-- [BGP.tsx](/home/ray/dev/linkong/planet/frontend/src/pages/BGP/BGP.tsx)
-- [Playground.tsx](/home/ray/dev/linkong/planet/frontend/src/pages/Playground/Playground.tsx)
+- [BGP.tsx](/home/ray/dev/linkong/planet/frontend/src/admin/pages/PlainResourcePages.tsx)
+- [Playground.tsx](/home/ray/dev/linkong/planet/frontend/src/admin/pages/PlainResourcePages.tsx)
 
 Constraints:
 

docs/technical/en/frontend-layout-guidelines.md

@@ -8,7 +8,8 @@ Admin pages in this project default to a "single-screen workspace" layout standa
 
 Current recommended reference implementations:
 
-- [frontend/src/pages/BGP/BGP.tsx](/home/ray/dev/linkong/planet/frontend/src/pages/BGP/BGP.tsx)
+- [frontend/src/admin/pages/PlainResourcePages.tsx](/home/ray/dev/linkong/planet/frontend/src/admin/pages/PlainResourcePages.tsx)
+- [frontend/src/admin/pages/DataList.tsx](/home/ray/dev/linkong/planet/frontend/src/admin/pages/DataList.tsx)
 - [frontend/src/index.css](/home/ray/dev/linkong/planet/frontend/src/index.css)
 
 ## Core Principles
@@ -23,15 +24,15 @@ Admin pages default to:
 Recommended structure:
 
 ```tsx
-<AppLayout>
+<AdminLayout>
   <div className="page-shell">
     <div className="page-shell__header">...</div>
     <div className="page-shell__body">...</div>
   </div>
-</AppLayout>
+</AdminLayout>
 ```
 
-Total page height should be bounded within the `AppLayout` content area, not allowed to grow naturally downward without limit.
+Total page height should be bounded within the `AdminLayout` content area, not allowed to grow naturally downward without limit.
 
 ### 2. Scrolling Should Happen Inside Modules
 

docs/technical/en/location-pipeline-development.md

@@ -2,7 +2,7 @@
 
 `backend/app/services/location/` is the shared abstraction for any "given a record, decide its lat/lon" workflow. Compute centers, BGP collectors, and BGP events now run on this pipeline. Future entities such as satellite ground stations, user-claimed points, and IXP facilities should plug in here instead of creating another geocoding path.
 
-For the user workflow, see [Earth Location Candidate Collection User Guide](/home/ray/dev/linkong/planet/docs/technical/en/location-pipeline-user.md).
+For the user workflow, see the Earth coordinate-candidate section in [Intelligent Planet Manual](/home/ray/dev/linkong/planet/docs/technical/en/manual.md).
 
 ## Design Goals
 

docs/technical/en/manual.md

@@ -1,8 +1,8 @@
-# Planet Manual
+# Intelligent Planet Manual
 
-This manual is for Planet end users. Starting from the browser, it covers account registration, login, configuring collectors, configuring AI, using Earth and the console, and reading the docs site. Every action happens in a browser.
+This manual is for Intelligent Planet end users. Starting from the browser, it covers account registration, login, configuring collectors, configuring AI, using Intelligent Planet and the console, and reading the docs site. Every action happens in a browser.
 
-If you are responsible for deployment or on-call duty, read the [Planet Ops Runbook](/home/ray/dev/linkong/planet/docs/technical/en/ops-runbook.md) instead — it covers shell commands, log paths, and CLI fallbacks for user creation.
+If you are responsible for deployment or on-call duty, read the [Intelligent Planet Ops Runbook](/home/ray/dev/linkong/planet/docs/technical/en/ops-runbook.md) instead — it covers shell commands, log paths, and CLI fallbacks for user creation.
 
 ## Entry Overview
 
@@ -233,7 +233,7 @@ To let a regular user read developer or operations docs, add `docs_developer` or
 
 ## Data Exploration
 
-- `/datasources`: source directory. The `Built-in Sources` tab can be filtered by product domain, layer/module, enabled state, last run status, whether collected records exist, and search text. Selecting rows triggers only those sources; `Trigger All` enters a browser-download-list style collection queue. The queue bar shows total progress plus running, completed, failed, and skipped counts; `View Queue` opens the bottom panel, failed rows can retry, and completed rows can jump to detail. `Realtime Sources` is for AISStream / WebSocket long connections and shows connection health, stored totals, time-window counters, and Start / Stop / Reconnect actions. Endpoint/credential/header editing happens at `/collection-management -> Collectors`.
+- `/datasources`: source directory. The `Built-in Sources` tab can be filtered by product domain, layer/module, enabled state, last run status, whether collected records exist, and search text. With no rows selected, the primary button shows `Trigger All`; after selecting rows, it becomes `Trigger Selected N` and submits only those sources. The top-right queue button shows a queue icon when empty and a pure circular total-progress indicator while tasks exist; it opens a floating panel grouped by running, completed, failed, and skipped. Failed rows can retry, and completed rows can jump to detail. `Realtime Sources` is for AISStream / WebSocket long connections and shows connection health, stored totals, time-window counters, and Start / Stop / Reconnect actions. Endpoint/credential/header editing happens at `/collection-management -> Collectors`.
 - `/data`: collected data table — used to verify "did data arrive", "is the freshness right", "does a source emit valid records"
 - `/bgp`: BGP detail page with list + detail + analysis; complements the BGP layer on Earth
 - `/alerts/system`, `/alerts/bgp`, `/alerts/situational`: system, BGP, and situational alerts
@@ -280,7 +280,17 @@ Search finds cables, landing points, satellites, compute centers, BGP events, BG
 
 Compute center and BGP observer detail cards support automatic coordinate-candidate collection. Click the object then use "Collect Coordinate Candidates" or "Re-collect Coordinates". The backend assembles candidates from source coordinates, public-org registry APIs, and online geocoders. When regular sources have no candidate, the current default AI Provider runs one LLM factcheck fallback. BGP observers' stored coordinates only fill query context; they are not returned as candidates.
 
-Candidates preview on Earth directly. Saving a compute-center candidate writes to the `compute_center_locations` dimension table and refreshes the layer immediately. The notification badge at the top-left of the compute-center layer shows the unresolved count; clicking it opens the queue, supports single collection, or "Adopt All" to save the top-confidence candidates from top to bottom. Records without candidates stay in the queue rather than being faked to country centroids. See [Earth Location Candidate Collection User Guide](/home/ray/dev/linkong/planet/docs/technical/en/location-pipeline-user.md).
+Candidates preview on Earth directly. Saving a compute-center candidate writes to the `compute_center_locations` dimension table and refreshes the layer immediately. The notification badge at the top-left of the compute-center layer shows the unresolved count; clicking it opens the queue, supports single collection, or "Adopt All" to save the top-confidence candidates from top to bottom. Records without candidates stay in the queue rather than being faked to country centroids.
+
+Recommended single-object flow:
+
+1. Open a compute center or BGP observer detail card.
+2. Click "Collect Coordinate Candidates".
+3. Wait for candidates; entries that depend on WebSearch / AI factcheck show collection state.
+4. Preview candidate positions on Earth.
+5. Save the candidate when it is credible; closing the card does not lose the current task state.
+
+Adopt All is for batch processing the compute-center unresolved queue. It starts from the top and adopts the highest-confidence candidate. Records without factual support remain in the queue. When WebSearch is disabled, single locate and Adopt All are disabled because location validation depends on factual lookup.
 
 ### Settings
 
@@ -351,7 +361,7 @@ Mobile uses a drawer layout: layer control moves into a drawer; search/settings/
 
 Docs at `http://localhost:3000/docs` are served by the backend with access control, not bundled into the frontend build.
 
-Anonymous visitors see only `public` docs: README, Quickstart, Manual, FAQ, Earth Location Candidate Collection User Guide. Authenticated users with Gatekeeper groups see more:
+Anonymous visitors see only `public` docs: README, Quickstart, Manual, and FAQ. Authenticated users with Gatekeeper groups see more:
 
 - `docs_user`: end-user operational docs
 - `docs_developer`: Earth, frontend, backend, collectors, AI Provider development docs
@@ -365,5 +375,4 @@ Docs supports: category navigation, Markdown rendering, tables and code blocks,
 
 - [Quickstart](/home/ray/dev/linkong/planet/docs/technical/en/quickstart.md)
 - [FAQ](/home/ray/dev/linkong/planet/docs/technical/en/faq.md)
-- [Earth Location Candidate Collection User Guide](/home/ray/dev/linkong/planet/docs/technical/en/location-pipeline-user.md)
-- [Planet Ops Runbook](/home/ray/dev/linkong/planet/docs/technical/en/ops-runbook.md)
+- [Intelligent Planet Ops Runbook](/home/ray/dev/linkong/planet/docs/technical/en/ops-runbook.md)

docs/technical/en/naming-glossary.md

@@ -1,6 +1,6 @@
 # Naming Glossary
 
-This document standardizes terms used across the Planet console, Earth, backend APIs, and documentation. When adding Chinese UI labels, chart labels, or translated documentation, prefer the Chinese display names listed here to avoid unnecessary mixed Chinese/English copy.
+This document standardizes terms used across Intelligent Planet, the console, backend APIs, and documentation. When adding Chinese UI labels, chart labels, or translated documentation, prefer the Chinese display names listed here to avoid unnecessary mixed Chinese/English copy.
 
 ## Usage Rules
 
@@ -14,9 +14,9 @@ This document standardizes terms used across the Planet console, Earth, backend
 
 | English / Key | Chinese Display Name | Usage |
 | --- | --- | --- |
-| Planet | Planet | Product name |
-| Admin Next | 新控制台 | New admin console |
-| Earth | Earth | Visualization product name |
+| Planet | 智能星球 | Product name and primary experience entry |
+| Admin | 控制台 | Admin console context |
+| Earth | 智能星球 | Visualization product name and `/earth` entry |
 | datasource | 数据源 | APIs, lists, filters |
 | collector | 采集器 | Collection jobs and credential configuration |
 | collected data | 采集数据 | Data list and statistics |

docs/technical/en/ops-runbook.md

@@ -1,6 +1,6 @@
-# Planet Ops Runbook
+# Intelligent Planet Ops Runbook
 
-This runbook is for deployment, on-call, and maintenance engineers. End-user UI flows live in the [Planet Manual](/home/ray/dev/linkong/planet/docs/technical/en/manual.md); this document only covers shell, Docker, logs, environment variables, and troubleshooting.
+This runbook is for deployment, on-call, and maintenance engineers. End-user UI flows live in the [Intelligent Planet Manual](/home/ray/dev/linkong/planet/docs/technical/en/manual.md); this document only covers shell, Docker, logs, environment variables, and troubleshooting.
 
 ## First Startup
 
@@ -61,6 +61,22 @@ Per-module restart:
 
 Per-module restart is preferred during development to avoid interrupting unrelated services.
 
+## Destructive Reset
+
+```bash
+./planet.sh destroy
+```
+
+`destroy` returns a local development environment to a near-empty project state. It requires typing `Y` before it runs; source files and existing `.env` files are preserved.
+
+Cleanup order and boundaries:
+
+- If `planet_postgres` is running, the script first clears the `public` schema in `planet_db`. This prevents old `collected_data.is_current = true` rows from making Earth OOBE report `ready=true` if Docker volume removal later fails.
+- Docker cleanup targets resources whose Compose project is `planet`, plus the explicit volumes `planet_postgres_data`, `planet_redis_data`, `postgres_data`, and `redis_data`; do not delete unlabeled volumes by a broad `planet_*` pattern, because another local project could own them.
+- Local build state removes `.venv`, frontend `node_modules` / `dist`, Planet state, and scattered Python / Vite cache directories. `$PLANET_CACHE_DIR/downloads` is preserved so upstream raw downloads such as CelesTrak can survive database resets and local rebuild cleanup.
+
+After the reset, run `./planet.sh init` again to recreate tables and default seed data. Old collected records are not restored, and Earth OOBE is evaluated from the backend's real collection state on the next visit. When CelesTrak later returns its "GP data has not updated" HTTP 403, the backend first reuses the preserved download cache to repopulate the database; if no cache exists, wait for the next CelesTrak update window or use Space-Track as a fallback.
+
 ## Health Check
 
 ```bash
@@ -120,6 +136,20 @@ Useful for:
 - Demoing Earth from a phone or tablet
 - Other LAN machines reaching the same dev instance
 
+On Windows, the repository-root `planet.cmd` can be used as a one-click entrypoint. It requests Administrator privileges, enters the `Ubuntu` WSL distribution at `/home/linkong/planet`, runs `./planet.sh restart --allow-lan`, opens `http://localhost:3000/earth` after a successful restart, and leaves the terminal inside a WSL shell for log inspection. If the local WSL distribution name or checkout path differs, adjust the `wsl.exe -d ... --cd ...` arguments in `planet.cmd` first.
+
+On a new Windows machine, check the WSL generation first:
+
+```powershell
+wsl -l -v
+```
+
+Planet development should use WSL2. WSL1 has different networking, filesystem, and process behavior, and can surface as Bun package-manager commands returning only `An unknown error occurred (Unexpected)`, unstable port release, or LAN behavior that does not match the script's assumptions. Convert the distribution if it still runs as WSL1:
+
+```powershell
+wsl --set-version Ubuntu 2
+```
+
 `--allow-lan` directly exposes the frontend, backend, and AI Provider from the development machine: frontend `3000`, backend `8000`, and AI Provider `8010`. Before startup, the script checks all three ports. If WSL/Linux cannot release a port and a Windows-side listener or stale `portproxy` rule owns it, the script requests Administrator PowerShell cleanup. When Planet runs in WSL, Windows can usually reach it through `localhost`; other LAN machines reaching the Windows LAN IP still need Windows Firewall allow rules.
 
 Diagnose in this order:
@@ -184,13 +214,32 @@ The AI Provider image only rebuilds when code, Dockerfile, Compose config, or Py
 ./planet.sh restart -a
 ```
 
+Rebuild detection is based on a content fingerprint rather than only file mtimes. `planet.sh` hashes the `aiprovider/` files, `aiprovider/Dockerfile`, `pyproject.toml`, `uv.lock`, `PYTHON_IMAGE`, `UV_IMAGE`, and the dependency fingerprint into `AI_PROVIDER_BUILD_FINGERPRINT`; Docker writes it into the image label `planet.aiprovider.build-fingerprint`. If the existing `planet-aiprovider:latest` image has a matching label, the script skips rebuild and refreshes the local stamp. Older images without the label fall back to the state/cache stamp.
+
+Docker builds use `uv sync --frozen`. To make container builds reuse the host uv mirror configuration, the script resolves the first config file in this order and mounts it into the build as a BuildKit secret at `/root/.config/uv/uv.toml`:
+
+1. The current `UV_CONFIG_FILE`
+2. Repository-root `uv.toml`
+3. `${XDG_CONFIG_HOME:-~/.config}/uv/uv.toml`
+4. `~/.uv/uv.toml`
+
+If none exists, the script creates an empty state file for the secret so Compose does not fail on a missing file. Before Docker build it unsets `UV_DEFAULT_INDEX`, `UV_INDEX_URL`, and `UV_EXTRA_INDEX_URL`, keeping the build tied to the explicit `UV_CONFIG_FILE`. For a temporary Tsinghua mirror, place this in repository-root `uv.toml`:
+
+```toml
+[[index]]
+name = "tsinghua"
+url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/simple/"
+default = true
+```
+
 Diagnose slow builds:
 
 | Symptom | Common cause | Fix |
 | --- | --- | --- |
 | Large `transferring context` | build context includes unrelated frontend / data files | `.dockerignore` ships only required files |
-| `uv sync` is slow | first build or cold cache | wait for the first build; later runs reuse BuildKit cache |
+| `uv sync --frozen` is slow | first build, cold cache, or missing uv mirror config | wait for the first build; later runs reuse BuildKit cache; configure `uv.toml` when needed |
 | Old keys still in effect after edit | container not restarted | `./planet.sh restart -a` |
+| Code changed but the image did not rebuild | fingerprint still matches the image label | Confirm the change is under `aiprovider/`, Dockerfile, or Python dependency inputs; delete `planet-aiprovider:latest` and retry if needed |
 
 ## SMTP Email (Required for Public Registration)
 
@@ -217,6 +266,15 @@ One-time codes are stored in Redis under `otp:{purpose}:{email}` with a 600-seco
 
 ## Development Command Conventions
 
+Backend setup and script initialization use the lockfile:
+
+```bash
+uv python install 3.14
+uv sync --frozen --group dev
+```
+
+`--frozen` rejects implicit `uv.lock` rewrites, which is the desired behavior on new machines, CI, and Docker builds. Dependency upgrades should explicitly update `pyproject.toml` / `uv.lock` on a development machine and commit the lockfile.
+
 Frontend must use Bun:
 
 ```bash
@@ -228,6 +286,8 @@ bun run build
 
 Do not use `npm run ...`. In the WSL / Windows mixed environment Bun avoids Node/npm path inconsistencies.
 
+`./planet.sh start` / `init` now runs `bun install` before startup instead of only checking whether the Vite entry file exists. This keeps new devices, cleaned `node_modules`, and lockfile changes synchronized before the console loads, avoiding dynamic-import 500s caused by missing frontend dependencies.
+
 Validate the frontend build:
 
 ```bash

docs/technical/en/platform-data-flows.md

@@ -0,0 +1,160 @@
+# Business Architecture and Data Flows
+
+This document is the business entry point for Intelligent Planet data products. It explains why each Earth data type exists, where it is collected from, which fact or derived tables it uses, and how cache invalidation plus WebSocket hints reach Earth. Frontend, backend, and Earth docs should focus on implementation details; start here when you need the cross-system data flow.
+
+## Overview
+
+Intelligent Planet data moves through three stages:
+
+1. **Collect and normalize**: built-in collectors, admin actions, or the location pipeline write PostgreSQL. Generic raw output lands in `collected_data`; layer-ready projections land in derived tables.
+2. **Project and broadcast**: database triggers write fact changes to the `earth_data_change_events` outbox and wake the backend listener with `LISTEN/NOTIFY`. The listener maps the change through layer adapters, invalidates cache, and broadcasts `earth_updates`.
+3. **Reload and render in Earth**: Earth receives layer-level refresh hints, applies `clear_then_reload`, `reload`, or `delta`, then refetches `/api/v1/visualization/...`.
+
+```mermaid
+flowchart LR
+  Source["External source / admin action"] --> Collector["Collector or data job"]
+  Collector --> Facts["collected_data"]
+  Collector --> Derived["Derived tables"]
+  Facts --> Outbox["earth_data_change_events"]
+  Derived --> Outbox
+  Outbox --> Listener["DB change listener"]
+  Listener --> Cache["Earth cache invalidation"]
+  Listener --> WS["earth_updates"]
+  WS --> Earth["Earth layer reload"]
+  Earth --> API["Visualization APIs"]
+```
+
+`LISTEN/NOTIFY` is only the low-latency wakeup path. The reliable source is the outbox. A real zero-data state is valid and APIs should return HTTP 200 with an empty collection; only actual endpoint failures should return 5xx.
+
+## Data Products
+
+```mermaid
+flowchart TB
+  Satellites["Satellite TLE"] --> SatelliteLayer["satellites layer"]
+  Cables["Cables + landing points"] --> CableLayer["cables layer"]
+  Compute["TOP500 / AI GPU / HF"] --> ComputeLocations["compute_center_locations"]
+  ComputeLocations --> ComputeLayer["computeCenters layer"]
+  BgpRaw["RIS Live / BGPStream / Prefix"] --> BgpDerived["bgp_observations / anomalies / incidents"]
+  BgpDerived --> BgpLayer["bgp layer"]
+  VesselRaw["AIS / BarentsWatch"] --> VesselDerived["vessel_static / vessel_position"]
+  VesselDerived --> VesselLayer["vessels layer"]
+  Interactables["earth_interactables"] --> InteractableLayer["interactables layer"]
+  NewsRaw["RSS / Live / News"] --> NewsItems["earth_news_items"]
+  NewsItems --> NewsLayer["news / media layer"]
+```
+
+| Data product | Business use | Fact sources | Derived / dimension tables | Earth layer | Strategy |
+| --- | --- | --- | --- | --- | --- |
+| Satellites | In-orbit objects, tracks, coverage, and cruise targets | `celestrak_tle`, `spacetrack_tle` | No long-lived projection; APIs convert TLE on demand | `satellites` | `clear_then_reload` |
+| Cables and landing points | Submarine connectivity, landing cities, cable detail, and search | `arcgis_cables`, `arcgis_landing_points`, TeleGeography / FAO landing sources | Cable relation and landing aggregation data | `cables` | `clear_then_reload` |
+| Compute centers | TOP500, AI GPU, model platform facilities, and location-completion state | `top500`, `epoch_ai_gpu`, HuggingFace sources | `compute_center_locations` | `computeCenters` | `reload` |
+| BGP context | Collectors, anomalies, incidents, route events, and regional context | `ris_live_bgp`, `bgpstream_bgp`, prefix geography sources | `bgp_observations`, `bgp_anomalies`, `bgp_incidents`, `bgp_collector_locations` | `bgp` | `clear_then_reload` |
+| Vessels | AIS vessels, positions, tracks, legend, and source health | AIS sources, `barentswatch_vessels` | `vessel_static`, `vessel_position`, `ais_raw_observations`, `ais_source_health` | `vessels` | `clear_then_reload` |
+| Interactables | Generic surface icons, manual objects, and future small layers | `earth_interactables` | None | `interactables` | `delta` |
+| News and media | Earth news, live streams, cruise summaries, and situation content | news sources | `earth_news_items` | `news` / `media` | `reload` |
+
+## Satellites
+
+Satellite data powers online satellite markers, tracks, footprint policy, and cruise lists. CelesTrak or Space-Track collectors fetch TLE and write `collected_data`. The visualization API converts TLE into current positions and trails on request.
+
+- **Collection entry**: CelesTrak TLE, Space-Track TLE.
+- **Fact table**: `collected_data.source IN ('celestrak_tle', 'spacetrack_tle')`.
+- **API**: satellite visualization API reads TLE and builds the Earth payload.
+- **Delete semantics**: deleting the source broadcasts `satellites` with `clear_then_reload`; the frontend clears satellite markers and trails before refetching. With no TLE, the API returns an empty list.
+- **Common failure**: summary is zero but Earth still shows satellites. Usually WS did not fire, the adapter did not cover the source, or the frontend did not clear existing Three.js objects.
+
+## Cables and Landing Points
+
+Cables and landing points show global network connectivity, landing cities, cable details, and search targets. Cable lines and landing points are one business layer; deleting either side must refresh `cables`.
+
+- **Collection entry**: ArcGIS cables, ArcGIS landing points, and TeleGeography / FAO landing related sources.
+- **Fact table**: cable and landing point sources in `collected_data`.
+- **Derived data**: cable relation tables, landing point aggregation, interface cache.
+- **API**: cable visualization API returns cables, landing points, and relation data.
+- **Delete semantics**: deleting cable or landing source clears owned projections and broadcasts `cables` with `clear_then_reload`. Empty data returns HTTP 200, not 404.
+- **Common failure**: cables disappear for a second and return. Usually a collection replacement or cache refresh window reuses old cache; check duplicate broadcasts and cache patterns.
+
+## Compute Centers
+
+Compute centers represent supercomputers, AI GPU sites, model-platform facilities, and unresolved-location state. Raw sources often only provide organization, country, site, or fuzzy location, so Earth rendering depends on `compute_center_locations`.
+
+- **Collection entry**: TOP500, Epoch AI GPU, HuggingFace related sources.
+- **Fact table**: compute sources in `collected_data`.
+- **Dimension table**: `compute_center_locations` stores adopted coordinate candidates.
+- **API**: compute center visualization API merges raw records with location rows.
+- **Delete semantics**: deleting TOP500 or similar sources refreshes `computeCenters`; changing the location table also refreshes the layer. The default is `reload`, because location updates do not always need an immediate clear.
+- **Common failure**: collection finished but Earth count did not change. Check visualization cache, location rows, and adapter coverage.
+
+## BGP
+
+BGP data shows route collectors, anomalies, incidents, diffusion rings, and situation summaries. Earth does not render raw BGP rows directly; it renders projected observations, anomalies, and incidents. This is the most common path where deleting raw data can leave old objects visible.
+
+```mermaid
+sequenceDiagram
+  participant UI as Admin delete button
+  participant Job as clear_data job
+  participant DB as PostgreSQL
+  participant Outbox as earth_data_change_events
+  participant Listener as DB change listener
+  participant Earth as Earth frontend
+
+  UI->>Job: Submit delete for ris_live_bgp / bgpstream_bgp
+  Job->>DB: Delete collected_data raw rows
+  Job->>DB: Delete BGP-owned derived tables
+  DB->>Outbox: trigger writes bgp layer changed
+  DB-->>Listener: LISTEN/NOTIFY wakeup
+  Listener->>Listener: merge events and clear bgp cache
+  Listener-->>Earth: broadcast earth_updates clear_then_reload
+  Earth->>Earth: clear BGP objects
+  Earth->>DB: refetch projections through visualization API
+```
+
+- **Collection entry**: RIPE RIS Live BGP, CAIDA BGPStream Backfill, IPtoASN / OpenGeoFeed / NRO prefix geography.
+- **Fact table**: BGP and prefix sources in `collected_data`.
+- **Derived tables**: `bgp_observations`, `bgp_anomalies`, `bgp_incidents`, `bgp_collector_locations`.
+- **API**: BGP visualization API reads projections and combines stored or candidate locations.
+- **Delete semantics**: deleting RIS Live or BGPStream raw sources must also clear BGP-owned projections and broadcast `bgp` with `clear_then_reload`. Direct derived-table deletes also write outbox events.
+- **Common failure**: collectors or incidents remain after raw rows are removed. Check derived tables first, not only `collected_data`.
+
+## Vessels
+
+Vessel data shows AIS vessels, navigation state, vessel-type legend, and source health. Earth rendering uses position snapshots and static vessel information; it should not render every raw AIS observation.
+
+- **Collection entry**: AIS sources, BarentsWatch vessels.
+- **Fact table**: `collected_data` or AIS raw observation tables.
+- **Derived tables**: `vessel_static`, `vessel_position`, `ais_raw_observations`, `ais_source_health`.
+- **API**: vessel visualization API returns current vessel markers and detail fields.
+- **Delete semantics**: deleting any vessel source lets owned projections broadcast `vessels` with `clear_then_reload`.
+- **Common failure**: the count panel changes but vessels remain. Summary and layer data are separate; Earth should clear objects based on layer updates.
+
+## Interactables
+
+`earth_interactables` is the generic surface-icon capability for manual objects, extensions, and future small layers. Unlike most layers, it keeps object-level delta.
+
+- **Fact table**: `earth_interactables`.
+- **API**: interactable API and Earth generic icon interface.
+- **Strategy**: upsert for create/update and removeItem for delete; no full layer reload.
+- **Delete semantics**: delete payloads must include a stable id so the frontend can remove the object.
+- **Common failure**: an object cannot be removed. Usually the stable id is missing, the delta handler did not match the object type, or an older duplicate render path still exists.
+
+## News and Media
+
+News and media support the Earth news ticker, live stream panel, news cruise, and situation summaries. They are content refresh paths rather than stable geographic object layers, so they default to `reload`.
+
+- **Collection entry**: RSS, live streams, news sources.
+- **Fact table**: news source rows in `collected_data`.
+- **Derived table**: `earth_news_items`.
+- **API**: news, live stream, and media content APIs.
+- **Delete semantics**: deleting news sources or `earth_news_items` broadcasts `news` / `media` reload; empty responses hide the corresponding content.
+- **Common failure**: the live panel shows stale content. Usually the media component ignored the layer update or the content API cache was not invalidated.
+
+## Adding a New Layer
+
+Add a new Earth data product in this order:
+
+1. Define the business purpose and Earth layer name.
+2. Identify fact sources, fact tables, and derived tables.
+3. Register source/table coverage, cache patterns, owned derived cleanup, and default strategy in the backend layer adapter.
+4. Ensure the visualization API returns HTTP 200 with an empty collection for zero data.
+5. Keep the Earth frontend driven only by layer and strategy, not database table names.
+6. Update this document with the data product flow, then document implementation details in the relevant frontend, backend, or Earth technical doc.

docs/technical/en/quickstart.md

@@ -1,8 +1,8 @@
 # Quickstart
 
-This quickstart is for Planet end users who just received an access URL and need the shortest path from "open the browser" to "first useful configuration done". Every action happens in the browser.
+This quickstart is for Intelligent Planet end users who just received an access URL and need the shortest path from "open the browser" to "first useful configuration done". Every action happens in the browser.
 
-If you are responsible for deployment or operations, read the [Planet Ops Runbook](/home/ray/dev/linkong/planet/docs/technical/en/ops-runbook.md) instead.
+If you are responsible for deployment or operations, read the [Intelligent Planet Ops Runbook](/home/ray/dev/linkong/planet/docs/technical/en/ops-runbook.md) instead.
 
 ## 1. Open the URL
 
@@ -34,7 +34,7 @@ After landing on the `/admin` dashboard, here's a recommended walk-through:
 
 1. `/collection-management?tab=collector_credentials`: pick a collector and click the plug icon to test connectivity. Free collectors (e.g. open BGP) usually work right away; credential-bearing ones like `AISStream` or `BarentsWatch` need an API key / client secret first
 2. `/ai?tab=providers`: fill an LLM provider (e.g. `minimax` / `openai`), model, base URL, API key, and click the plug at the end of the base URL to test. WebSearch / OCR tools are optional
-3. `/datasources` or `/data`: check whether collectors have produced data. Use `/datasources -> Built-in Sources` for finite collectors and open the collection queue after triggering; use `/datasources -> Realtime Sources` for AISStream / WebSocket health and counters
+3. `/datasources` or `/data`: check whether collectors have produced data. Use `/datasources -> Built-in Sources` for finite collectors: with no rows selected, click `Trigger All`; after selecting rows, the primary button becomes `Trigger Selected N`. The top-right queue button shows progress. Use `/datasources -> Realtime Sources` for AISStream / WebSocket health and counters
 4. `/alerts/system`: verify system alerts look right
 5. `/users` (super_admin only): open accounts for teammates or adjust their groups
 
@@ -62,7 +62,7 @@ Open `/forgot-password`, enter your email, receive a code, then enter the code p
 
 ## Next Steps
 
-- Full UI walkthrough: [Planet Manual](/home/ray/dev/linkong/planet/docs/technical/en/manual.md)
+- Full UI walkthrough: [Intelligent Planet Manual](/home/ray/dev/linkong/planet/docs/technical/en/manual.md)
 - Troubleshooting and configuration questions: [FAQ](/home/ray/dev/linkong/planet/docs/technical/en/faq.md)
-- Detailed Earth coordinate candidate flow: [Earth Location Candidate Collection User Guide](/home/ray/dev/linkong/planet/docs/technical/en/location-pipeline-user.md)
-- Deployment / operations commands: [Planet Ops Runbook](/home/ray/dev/linkong/planet/docs/technical/en/ops-runbook.md)
+- Detailed Earth coordinate candidate flow: see the Earth section in [Intelligent Planet Manual](/home/ray/dev/linkong/planet/docs/technical/en/manual.md)
+- Deployment / operations commands: [Intelligent Planet Ops Runbook](/home/ray/dev/linkong/planet/docs/technical/en/ops-runbook.md)

docs/technical/en/tactile-ui-components.md

@@ -1,6 +1,6 @@
 # Tactile UI Components
 
-Tactile UI is Planet's portable React control layer. It was extracted from the Admin Next button, switch, scrollbar, and tooltip work, but the components themselves do not depend on Admin Next, AntD, Radix, Tailwind, or `an-*` classes. The immediate goal is stable in-repo usage; the structure is intentionally close to something that can later be published as an npm package.
+Tactile UI is Planet's portable React control layer. It was extracted from the Admin button, switch, scrollbar, and tooltip work, but the components themselves do not depend on Admin, AntD, Radix, Tailwind, or `an-*` classes. The immediate goal is stable in-repo usage; the structure is intentionally close to something that can later be published as an npm package.
 
 ## Design Goals
 
@@ -49,6 +49,22 @@ Core tokens are exposed as `--tui-*` CSS variables. Product themes should overri
 
 Most controls also accept a `tactile` prop for local width, height, radius, background, border, and shadow overrides. Use local overrides for small special cases; use CSS variables for product-wide styling.
 
+## Portals and Dark Theme
+
+`TactileTooltip` and the Admin `Dialog`, `Select`, and toast controls that use Tactile UI may render through a portal attached to `document.body`. Those nodes are not descendants of `.admin-theme-root[data-theme='dark']`, so dark tokens cannot rely only on an ancestor selector inside the Admin root.
+
+The Admin theme provider mirrors the active theme to `body[data-admin-theme]`. Shared styles need to support both selector paths:
+
+```css
+[data-theme='dark'] .tui-button,
+body[data-admin-theme='dark'] .tui-button {
+  --tui-surface: #172033;
+  --tui-text: #e5edf8;
+}
+```
+
+When adding a portal-based control, first check whether it renders into body. If it does, add a `body[data-admin-theme='dark']` branch in that component's style entry, or reuse the already covered `--tui-*` / `--an-*` tokens. Avoid hard-coding a one-off dark modal style, because the same contrast problem can reappear in dropdowns, tooltips, toasts, and confirmation dialogs.
+
 ## `TactileButton`
 
 The button component covers regular buttons, icon buttons, strong-intent buttons, and link-like buttons.
@@ -74,6 +90,8 @@ Common props:
 
 `variant="neutral"` defaults to a white tactile button. Colored buttons should still keep the same height and external shadow instead of relying on page-specific CSS overrides.
 
+Colored button borders must not use the exact fill color. `primary`, `danger`, and future colored variants should use a lighter border from the same hue, such as `color-mix(in srgb, var(--tui-danger) 64%, white)`. The border still reads as part of the button color, but its visual weight is lower than the fill surface, so red or blue buttons do not look one outline larger than neutral buttons. Hover states should brighten rather than darken: mix a little white into the current fill color, and keep the hover border lighter than the hover fill.
+
 ## Icon Presets
 
 Preset icons are maintained in `tactileIconPresets`. Feature pages should call icons by semantic name so actions remain consistent across the console.
@@ -177,7 +195,7 @@ Do not force overlay scrollbars onto textareas. Text selection and the resize gr
 
 ## `TableScrollRegion`
 
-`TableScrollRegion` is a convenience wrapper for table scroll areas. The default target selector is `.tui-scroll-target`; Admin Next passes its table viewport selector explicitly so package code does not contain product-specific names.
+`TableScrollRegion` is a convenience wrapper for table scroll areas. The default target selector is `.tui-scroll-target`; Admin passes its table viewport selector explicitly so package code does not contain product-specific names.
 
 ```tsx
 <TableScrollRegion targetSelector=".table-viewport">
@@ -192,9 +210,9 @@ Do not force overlay scrollbars onto textareas. Text selection and the resize gr
 - Tooltip text is explanatory only; state must still be represented by text, badges, or `aria-*` attributes.
 - Disabled and loading states set `aria-disabled`; real `button` elements also receive `disabled`.
 
-## Admin Next Migration Rules
+## Admin Migration Rules
 
-Admin Next should use Tactile UI for global tool buttons, detail-panel toolbars, and list footer actions:
+Admin should use Tactile UI for global tool buttons, detail-panel toolbars, and list footer actions:
 
 - Unambiguous actions: `TactileButton iconOnly tooltip`
 - Save/create/confirm: `TactileButton variant="primary"`, usually with text

docs/technical/zh/README.md

@@ -1,44 +1,53 @@
 # 技术文档
 
-这里放“当前实现和当前结构”的文档，重点回答：
+这里是智能星球当前文档入口。文档按读者和问题类型分层：先看业务架构理解数据产品，再进入使用手册或技术实现文档。
 
-- 现在代码是怎么组织的
-- 当前入口在哪
-- 状态和组件如何工作
+## 业务架构
 
-适合放入这里的内容：
+- [业务架构与数据流转](/home/ray/dev/linkong/planet/docs/technical/zh/platform-data-flows.md)：每类智能星球数据的用途、采集链路、事实表、派生表、缓存和 WebSocket 广播链路
+- [命名与术语对照](/home/ray/dev/linkong/planet/docs/technical/zh/naming-glossary.md)：智能星球、控制台、后端和文档常见名词的中英对照
 
-- 快速开始和使用手册
-- 前端上下文
-- Earth 前端结构
-- Earth 卫星覆盖策略
-- Earth 渲染图层顺序
-- Earth 图层样式属性索引
-- 后端运行控制
-- 采集器现状
-- 采集器设置与连接验证
-- 采集格式约定
+## 使用手册
 
-## 使用入口
+- [快速开始](/home/ray/dev/linkong/planet/docs/technical/zh/quickstart.md)：从零启动智能星球的最短路径
+- [智能星球使用手册](/home/ray/dev/linkong/planet/docs/technical/zh/manual.md)：控制台、智能星球、文档和常用功能的用户操作说明
+- [常见问题](/home/ray/dev/linkong/planet/docs/technical/zh/faq.md)：Windows / WSL、端口、依赖、动捕、凭证和文档权限排障
 
-- [快速开始](/home/ray/dev/linkong/planet/docs/technical/zh/quickstart.md)：从零启动 Planet 的最短路径
-- [Planet 使用手册](/home/ray/dev/linkong/planet/docs/technical/zh/manual.md)：控制台、`planet.sh`、Earth 和 Docs 的完整使用手册
-- [常见问题](/home/ray/dev/linkong/planet/docs/technical/zh/faq.md)：Windows / WSL、端口、依赖、动捕、凭证和 Docs 权限的集中排障入口
-- [Earth 位置候选采集使用手册](/home/ray/dev/linkong/planet/docs/technical/zh/location-pipeline-user.md)：在 Earth 上为算力中心和 BGP 观测站采集、预览坐标候选
-- [数据源、采集器设置与连接验证](/home/ray/dev/linkong/planet/docs/technical/zh/datasource-collector-settings-connectivity.md)：数据源目录、采集器设置、连接验证、BarentsWatch 凭证链路
-- [通用位置估算管线开发说明](/home/ray/dev/linkong/planet/docs/technical/zh/location-pipeline-development.md)：后端 location resolver / pipeline 的接口、注册表和扩展方式
-- [Docs Gatekeeper 开发说明](/home/ray/dev/linkong/planet/docs/technical/zh/docs-gatekeeper-development.md)：后端 Docs 目录、正文读取和 Gatekeeper 权限组实现
-- [命名与术语对照](/home/ray/dev/linkong/planet/docs/technical/zh/naming-glossary.md)：控制台、Earth、后端和文档常见名词的中英对照
-- [Earth 可交互图标接入](/home/ray/dev/linkong/planet/docs/technical/zh/earth-interactable-usage.md)：Earth 地表可交互图标 `Interactable` 的接口、生命周期和接入示例
-- [Earth 工具栏与浮层协同](/home/ray/dev/linkong/planet/docs/technical/zh/earth-toolbar-overlay-coordination.md)：工具栏按钮与搜索 / 设置 / 新闻 / 图层浮层之间的关闭矩阵和接入规则
-- [Tactile UI 组件库](/home/ray/dev/linkong/planet/docs/technical/zh/tactile-ui-components.md)：可移植按钮、开关、tooltip 和滚动条组件的 API、主题 token 与迁移约定
+## 智能星球技术实现
 
-不适合放入这里的内容：
+- [智能星球前端结构](/home/ray/dev/linkong/planet/docs/technical/zh/earth-frontend-context.md)：智能星球页面模块、状态、WebSocket 刷新和图层生命周期
+- [智能星球图层样式属性索引](/home/ray/dev/linkong/planet/docs/technical/zh/earth-layer-style-reference.md)：图层颜色、符号、材质和视觉参数
+- [智能星球渲染图层顺序](/home/ray/dev/linkong/planet/docs/technical/zh/earth-render-layer-order.md)：renderOrder、深度策略、拾取和同坐标避让
+- [智能星球卫星覆盖策略](/home/ray/dev/linkong/planet/docs/technical/zh/earth-satellite-footprint-policy.md)：卫星 footprint 的显示边界和策略
+- [BGP 态势上下文](/home/ray/dev/linkong/planet/docs/technical/zh/earth-bgp-context.md)：BGP 在智能星球中的渲染、聚合和观测站实现
+- [智能星球可交互图标接入](/home/ray/dev/linkong/planet/docs/technical/zh/earth-interactable-usage.md)：`Interactable` 的接口、生命周期和接入示例
+- [智能星球可交互图标聚类策略](/home/ray/dev/linkong/planet/docs/technical/zh/earth-interactable-clustering.md)：可插拔 cluster strategy、稳定球面聚类和动态屏幕聚类的适用边界
+- [智能星球工具栏与浮层协同](/home/ray/dev/linkong/planet/docs/technical/zh/earth-toolbar-overlay-coordination.md)：工具栏按钮与搜索、设置、新闻、图层浮层的关闭矩阵
 
-- 尚未完成的路线图
-- 未来迭代方案
-- 大范围重构计划
+## 前端技术实现
 
-这些应放入：
+- [控制台前端结构](/home/ray/dev/linkong/planet/docs/technical/zh/frontend-admin-frontend-context.md)：控制台页面结构、状态边界和懒加载策略
+- [前端布局指南](/home/ray/dev/linkong/planet/docs/technical/zh/frontend-layout-guidelines.md)：布局、表格、面板和响应式约束
+- [Tactile UI 组件库](/home/ray/dev/linkong/planet/docs/technical/zh/tactile-ui-components.md)：按钮、开关、tooltip、滚动条组件 API 和主题 token
 
-- [计划文档索引](/home/ray/dev/linkong/planet/docs/plans/README.md)
+## 后端技术实现
+
+- [数据采集系统](/home/ray/dev/linkong/planet/docs/technical/zh/backend-collectors.md)：采集器、任务类型、保存层和状态回写
+- [系统服务控制](/home/ray/dev/linkong/planet/docs/technical/zh/backend-system-service-control.md)：后端服务控制和系统操作接口
+- [数据源、采集器设置与连接验证](/home/ray/dev/linkong/planet/docs/technical/zh/datasource-collector-settings-connectivity.md)：数据源目录、采集器设置、连接验证和凭证链路
+- [数据源 API 性能](/home/ray/dev/linkong/planet/docs/technical/zh/backend-datasources-api-performance.md)：DataSources 列表接口性能和缓存策略
+- [数据作业与 Outbox 技术架构](/home/ray/dev/linkong/planet/docs/technical/zh/data-job-earth-sync-architecture.md)：PostgreSQL 作业队列、outbox、listener 和 Kafka / Spark 演进边界
+- [通用位置估算管线开发说明](/home/ray/dev/linkong/planet/docs/technical/zh/location-pipeline-development.md)：location resolver / pipeline 的接口、注册表和扩展方式
+- [新闻直播采集格式](/home/ray/dev/linkong/planet/docs/technical/zh/earth-news-live-streams-collector-format.md)：新闻、直播和媒体采集 payload 约定
+- [Docs Gatekeeper 开发说明](/home/ray/dev/linkong/planet/docs/technical/zh/docs-gatekeeper-development.md)：后端文档目录、正文读取和 Gatekeeper 权限组实现
+
+## 智能体与运维
+
+- [AI Provider 指南](/home/ray/dev/linkong/planet/docs/technical/zh/agents-aiprovider.md)：模型供应商适配、任务 prompt 和调用边界
+- [智能星球运维手册](/home/ray/dev/linkong/planet/docs/technical/zh/ops-runbook.md)：部署、启动、排障和敏感操作
+- [Docker + Compose + Buildx 升级](/home/ray/dev/linkong/planet/docs/technical/zh/ops-docker-compose-buildx-upgrade.md)：Docker 工具链升级步骤
+- [planet.sh 启动机制](/home/ray/dev/linkong/planet/docs/technical/zh/ops-planet-sh-startup.md)：启动脚本、健康检查和性能优化
+
+## 计划与历史
+
+尚未完成的路线图、大范围重构方案和未来计划放在仓库路径 `docs/plans/README.md`。已完成、过时或被新实现替代的方案放在 `docs/deprecated/README.md`。

docs/technical/zh/agents-aiprovider.md

@@ -101,7 +101,7 @@ AI 配置页使用的接口：
 
 这些接口都需要用户登录。`secrets` 接口只用于配置页点击显示 key/token 时取回明文，隐藏时前端恢复为脱敏预览。
 
-Admin Next 的 AI 页面按旧版信息架构组织为：
+Admin 的 AI 页面按业务信息架构组织为：
 
 - `模型供应商`
   - 管理 provider、协议适配、默认模型、LLM API Key、代理地址、代理 token、模型列表刷新、设为默认和轻量连通性测试。
@@ -272,7 +272,7 @@ AI 配置仍保存在 PostgreSQL，不写入 JSON 文件。核心结构如下：
 
 `.env` 只是兜底。配置页保存或测试连接成功后，PostgreSQL 中的配置会成为全局默认。
 
-Admin Next 的密钥状态必须按 provider / tool 精确判断：
+Admin 的密钥状态必须按 provider / tool 精确判断：
 
 - 数据库中当前 provider/tool 有密钥时，显示为“已配置”。
 - 数据库没有密钥，但 fallback provider、model 或 tool 与当前项匹配时，可以显示 fallback 的脱敏预览。
@@ -283,7 +283,7 @@ Admin Next 的密钥状态必须按 provider / tool 精确判断：
 
 ### 轻量连通性测试
 
-Admin Next 的插头按钮走轻量连通性测试，不承担保存职责。业界常见做法是分两层：
+Admin 的插头按钮走轻量连通性测试，不承担保存职责。业界常见做法是分两层：
 
 - 快速检查 provider 目录或低成本 endpoint，确认 base URL、鉴权和当前模型是否可达。
 - 只有在用户明确运行 Playground 或业务任务时才发完整模型请求。

docs/technical/zh/backend-collectors.md

@@ -73,6 +73,10 @@ async def run(self, db):
 
 **核心文件**: `backend/app/services/collectors/base.py`
 
+手动触发、删除数据、清理缓存现在统一进入 PostgreSQL 数据作业队列，任务账本仍是 `collection_tasks`。采集器只负责 `fetch -> transform -> save`，由 `data_jobs.py` worker 领取 `collect` / `clear_data` / `clear_cache` / `earth_refresh` 任务并回写进度。Earth 图层刷新关系集中在 `earth_layer_adapters.py`，不要再在单个采集器或按钮里手写缓存失效和 WebSocket 广播。
+
+删除数据任务按批次执行，避免 AIS 这类千万级表一次性锁表。`clear_data` 会先清 `collected_data`，再按来源清理 AIS 衍生表，并持续广播 `records_processed`；前端任务队列只展示“正在删除数据 / 删除完成”，内部表名只保留在日志和原始任务详情。删除结束后后端会 `ANALYZE ais_raw_observations`，让数据源列表的估算指标尽快收敛。数据源目录页默认使用 PostgreSQL 统计信息估算 AIS 大表记录数，避免冷启动做 `count(*)`；打开单条详情时再用精确计数校准当前数据源。
+
 ## 三、采集器列表
 
 | 采集器 | 数据类型 | 数据内容 | 采集频率 |
@@ -90,7 +94,7 @@ async def run(self, db):
 
 AIS 船只类采集器和其它 `CollectedData` 采集器的落库路径不同。BarentsWatch、AISStream 和自定义 `vessel_ais` 源都会进入 AIS 原始观测层，随后由聚合服务合并成 Earth 船只图层使用的 GeoJSON 和详情数据。这样做可以保留来源、传输方式、字段冲突和观测时间，避免某个实时源直接覆盖最终展示表。
 
-Earth 国界不再属于采集器体系。它是 Earth 静态渲染资产，由控制台 `运维与配置 -> Earth 内容 -> 国界精度` 维护源配置，并由 `/api/v1/earth/boundaries/*` 构建 `frontend/public/earth/data/boundaries/earth-boundaries-china-pov-v1.pmtiles`。本地没有高精 PMTiles 时，前端会使用仓库内置的低精度 GeoJSON 作为 fallback，不会向 `CollectedData` 写入国界记录。
+智能星球国界不再属于采集器体系。它是智能星球静态渲染资产，由控制台 `运维与配置 -> 智能星球内容 -> 国界精度` 维护源配置，并由 `/api/v1/earth/boundaries/*` 构建 `frontend/public/earth/data/boundaries/earth-boundaries-china-pov-v1.pmtiles`。本地没有高精 PMTiles 时，前端会使用仓库内置的低精度 GeoJSON 作为 fallback，不会向 `CollectedData` 写入国界记录。
 
 TOP500 和 Epoch AI 算力数据的公开源不总是提供可用经纬度。Earth 统一算力中心接口在主地图启动链路中只使用源数据自带坐标或 `compute_center_locations` 维表坐标；缺少坐标的记录会进入 `unresolved`，不会通过本地注册表、国家质心或猜测城市自动渲染。用户手动采集候选时，后端会用源字段调用 ROR 组织注册 API 和 Nominatim/OpenStreetMap 在线搜索；候选经前端保存后写入 `compute_center_locations`，后续地图刷新再从维表渲染。
 
@@ -230,7 +234,7 @@ if datasource.last_status == "success":
 
 ### 采集管理与快照
 
-Admin Next 的采集管理入口按业务层级组织：
+Admin 的采集管理入口按业务层级组织：
 
 - `采集器`：配置 endpoint、认证方式、请求头、基础参数、启用状态和凭证教程。
 - `采集调度`：查看和调整调度状态，触发、停止或刷新采集任务。

docs/technical/zh/backend-datasources-api-performance.md

@@ -96,4 +96,4 @@ if (res.data.task_id) {
 ## 相关文件
 
 - `backend/app/api/v1/datasources.py` — `_load_datasource_list_context`、`list_datasources`
-- `frontend/src/pages/DataSources/DataSources.tsx` — `BuiltInDataSource` interface、`triggerDatasource`
+- `frontend/src/admin/pages/PlainResourcePages.tsx` — `BuiltInDataSource` interface、`triggerDatasource`

docs/technical/zh/data-job-earth-sync-architecture.md

@@ -0,0 +1,140 @@
+# 数据作业与 Outbox 技术架构
+
+本文记录 Planet v1 的任务队列、数据库 outbox 和 Earth 刷新技术边界。每类数据产品的业务用途和端到端链路见 [业务架构与数据流转](/home/ray/dev/linkong/planet/docs/technical/zh/platform-data-flows.md)。
+
+## 架构边界
+
+- PostgreSQL 是 v1 的可靠任务账本和 outbox，不引入 Kafka、Celery 或 RQ。
+- `collection_tasks` 记录采集、删除、清缓存和非数据库触发的 Earth refresh 任务。
+- `earth_data_change_events` 记录事实表或派生表变化，是 Earth 同步的可靠来源。
+- `LISTEN/NOTIFY` 只做低延迟唤醒；listener 仍会轮询未消费 outbox。
+- Redis 主要用于缓存、认证辅助、OTP / rate limit、临时日志和 WebSocket 辅助，不是可靠队列。
+
+## 数据库变化同步
+
+```mermaid
+flowchart LR
+  Write["Fact or derived table write"] --> Trigger["PostgreSQL trigger"]
+  Trigger --> Outbox["earth_data_change_events"]
+  Trigger --> Notify["planet_earth_data_changes"]
+  Outbox --> Listener["earth_db_change_listener"]
+  Notify --> Listener
+  Listener --> Adapter["earth_layer_adapters"]
+  Adapter --> Cache["cache invalidation"]
+  Cache --> WS["earth_updates"]
+```
+
+1. 采集、删除、定位或派生任务写入事实表或派生表。
+2. statement trigger 为 `INSERT` / `UPDATE` / `DELETE` 写入 outbox。
+3. listener 被 notify 唤醒，或通过轮询发现未消费事件。
+4. listener 使用 `earth_layer_adapters.py` 把 `table + source` 映射成 Earth layer、刷新策略和 cache pattern。
+5. listener 短窗口合并同 layer 事件，失效缓存并广播 `earth_updates`。
+6. 广播成功后标记 outbox consumed；失败时保留待重试。
+
+DB 变化不再默认创建 `earth_refresh` 任务，因此不会被同 source 的长采集或删除任务阻塞。`earth_refresh` 只保留给手动清缓存和非 DB 变化刷新提示。
+
+## 数据作业队列
+
+`collection_tasks` 是统一 job ledger。worker 使用 PostgreSQL `FOR UPDATE SKIP LOCKED` 领取 `queued` 任务；同一 `source` 的写任务串行，不同 source 可并行。
+
+| task_type | 作用 |
+| --- | --- |
+| `collect` | 执行内置 datasource 采集 |
+| `clear_data` | 删除该 source 的采集数据和声明过的派生数据 |
+| `clear_cache` | 删除该 source 对应的 Earth / dashboard 缓存 |
+| `earth_refresh` | 非 DB 变化场景下失效智能星球图层缓存并广播刷新提示 |
+
+接口只创建任务并返回 `task_id`。任务执行、进度、取消和终态由 worker 写回 `collection_tasks`，并通过 `datasource_tasks` channel 通知前端。
+
+取消语义是“保留已提交批次”：点击停止后，后端把任务标记为 `cancelling` 并取消内存中的执行协程；已经提交的批次保留，未完成批次按采集器或清理任务的回滚逻辑处理。
+
+## Earth 同步事件模型
+
+统一事件模型是 `earth.layer.changed`：
+
+```json
+{
+  "event": "earth.layer.changed",
+  "action": "database_changed",
+  "source": "celestrak_tle",
+  "table": "collected_data",
+  "operation": "DELETE",
+  "layers": ["satellites"],
+  "refresh_strategy": "clear_then_reload",
+  "records_processed": 11125,
+  "occurred_at": "2026-05-25T10:20:30Z"
+}
+```
+
+| strategy | 用途 |
+| --- | --- |
+| `clear_then_reload` | 先清前端本地图层对象，再强制重拉接口。删除数据时优先使用。 |
+| `reload` | 保留旧对象直到新数据返回，适合定位、元数据或非破坏性更新。 |
+| `delta` | 只用于 `earth_interactables`，按 id upsert 或 remove。 |
+
+接口在真实 0 数据时必须返回 200 和空集合；只有真实接口异常才返回 5xx。前端收到删除事件后，如果重拉失败，应保持已清空状态并显示轻量错误，不恢复旧对象。
+
+## Layer Adapter 约定
+
+`earth_layer_adapters.py` 是 source、派生表、Earth layer、缓存和刷新策略的唯一注册表。新图层只应新增 adapter，不应在按钮 handler、采集器或前端分支里手写同步逻辑。
+
+Adapter 必须声明：
+
+- source 或 table 由哪个 Earth layer 消费。
+- 需要清理哪些 Earth cache key pattern。
+- `clear_data` 删除 source 时是否需要同时删除 owned 派生表。
+- 该 layer 的默认刷新策略。
+
+删除 source 时，`clear_data` 作业先删除 `collected_data.source = <source>`，再根据 adapter 删除 owned 派生表。直接修改派生表也会触发 outbox，所以后台任务、管理接口和 SQL 修复脚本只要落到事实表或派生表，Earth 都能感知变化。
+
+## 运维排障
+
+检查 outbox 是否堆积：
+
+```sql
+SELECT id, table_name, operation, source, occurred_at
+FROM earth_data_change_events
+WHERE consumed_at IS NULL
+ORDER BY id
+LIMIT 20;
+```
+
+检查触发器是否存在：
+
+```sql
+SELECT tgname, tgrelid::regclass
+FROM pg_trigger
+WHERE tgname LIKE 'tr_planet_%_changed_%'
+ORDER BY 2, 1;
+```
+
+常用日志事件：
+
+- `earth.db_changes.connected`：listener 已连接 PostgreSQL 并开始监听。
+- `earth.db_changes.outbox_polled`：轮询到了未消费 outbox。
+- `earth.db_changes.broadcasted`：已产生 Earth 刷新广播。
+- `data_job.started` / `data_job.completed`：任务执行状态。
+
+如果 Earth 没更新，按顺序检查：事实表是否变化、outbox 是否消费、adapter 是否覆盖对应 `source/table`、listener 是否在线、前端 WebSocket 是否连接、visualization 接口是否返回 200 空集合或新数据。
+
+## Kafka-ready 边界
+
+业务代码不直接依赖具体队列实现，而是通过这些边界组织：
+
+- `JobQueue`：提交、领取、取消、完成数据作业。
+- `DataChangeBus`：发布数据库事实变化。
+- `EarthLayerAdapterRegistry`：声明 source、layer、缓存和派生数据关系。
+
+需要 Kafka 的信号：
+
+- 多个独立服务需要消费同一批数据变化。
+- AIS、BGP 或传感器流达到持续高吞吐。
+- 需要 consumer group、事件回放、跨服务解耦。
+
+需要 Spark 的信号：
+
+- 历史数据到千万或亿级，PostgreSQL 聚合开始吃力。
+- 需要跨源、长时间窗口、空间时间关联分析。
+- 原始数据进入 Parquet / Iceberg / Delta 等湖仓，并开始生产离线派生数据产品。
+
+若目标是秒级连续流计算，优先评估 Flink；Spark 更适合批量或微批分析。

docs/technical/zh/datasource-collector-settings-connectivity.md

@@ -32,8 +32,8 @@
 
 文件：
 
-- [PlainResourcePages.tsx](/home/ray/dev/linkong/planet/frontend/src/admin-next/pages/PlainResourcePages.tsx)
-- [AdminNextRoutes.tsx](/home/ray/dev/linkong/planet/frontend/src/admin-next/AdminNextRoutes.tsx)
+- [PlainResourcePages.tsx](/home/ray/dev/linkong/planet/frontend/src/admin/pages/PlainResourcePages.tsx)
+- [AdminRoutes.tsx](/home/ray/dev/linkong/planet/frontend/src/admin/AdminRoutes.tsx)
 
 当前行为：
 
@@ -57,11 +57,11 @@
 
 文件：
 
-- [PlainResourcePages.tsx](/home/ray/dev/linkong/planet/frontend/src/admin-next/pages/PlainResourcePages.tsx)
+- [PlainResourcePages.tsx](/home/ray/dev/linkong/planet/frontend/src/admin/pages/PlainResourcePages.tsx)
 
 当前行为：
 
-- `/collection-management` 按旧版层级收敛为 `采集器`、`采集调度`、`采集历史 / 快照`。
+- `/collection-management` 按业务层级收敛为 `采集器`、`采集调度`、`采集历史 / 快照`。
 - `采集器` 是配置页，左侧列表展示采集器配置；右侧表单编辑 endpoint、认证、请求头、采集参数和启用状态。
 - 新增采集器和目标 Schema 使用草稿详情页，不再用透明 JSON 弹窗；保存后才固化到列表，取消会销毁草稿。
 - 连接按钮只做连通性测试，不保存配置；保存按钮只持久化表单。
@@ -80,7 +80,7 @@
 - `目标 Schema` 维护可写入目标结构。
 - `run-mapped`、`stop-mapped`、`stream-status` 负责运行映射后的自定义采集器。
 
-这些入口在新版中需要表单化，只有高级字段才折叠为 JSON。不要把模板、Schema、运行状态和采集器配置拍平成同一张表。
+这些入口在 Admin 中需要表单化，只有高级字段才折叠为 JSON。不要把模板、Schema、运行状态和采集器配置拍平成同一张表。
 
 ## 后端接口
 
@@ -334,7 +334,7 @@ GET /api/v1/vessels/snapshot?bbox=lon_min,lat_min,lon_max,lat_max&zoom=12&limit=
 
 自定义源现在不是独立的新数据孤岛，而是作为内置数据源的补充源写入目标 schema。当前最完整的目标是 `vessel_ais`：自定义 REST 或 WebSocket 源经过确定性 mapping 后写入 AIS raw observations，再通过 `vessels` WebSocket channel 推送给 Earth。
 
-Earth 高精度边界不再使用自定义源目标 schema。国界是 Earth 静态资产，由控制台 `运维与配置 -> Earth 内容 -> 国界精度` 保存本机源配置并触发 PMTiles 构建，不写入 `CollectedData`。
+智能星球高精度边界不再使用自定义源目标 schema。国界是智能星球静态资产，由控制台 `运维与配置 -> 智能星球内容 -> 国界精度` 保存本机源配置并触发 PMTiles 构建，不写入 `CollectedData`。
 
 ### 配置语义
 

docs/technical/zh/docs-gatekeeper-development.md

@@ -2,7 +2,7 @@
 
 Docs Gatekeeper 把 `/docs` 从“前端构建时打包所有 Markdown”改成“后端按权限返回目录和正文”。它的目标是让公开使用手册、用户文档、开发文档和管理/运维文档在同一个 Docs 页面内可检索，但正文读取必须经过服务端白名单和用户权限检查。
 
-用户侧说明见 [Planet 使用手册](/home/ray/dev/linkong/planet/docs/technical/zh/manual.md) 的 Docs 章节。
+用户侧说明见 [智能星球使用手册](/home/ray/dev/linkong/planet/docs/technical/zh/manual.md) 的文档章节。
 
 ## 鉴权模型
 
@@ -16,7 +16,7 @@ Docs 使用两层权限：
 | 组 | 用途 |
 | --- | --- |
 | `docs_user` | 用户操作类文档 |
-| `docs_developer` | Earth、前端、后端、采集器和 AI Provider 开发文档 |
+| `docs_developer` | 智能星球、前端、后端、采集器和 AI Provider 开发文档 |
 | `docs_admin` | 服务控制、运维、环境变量和敏感操作文档 |
 
 继承规则：
@@ -24,7 +24,7 @@ Docs 使用两层权限：
 - 未登录用户只能读 `public`。
 - `docs_developer` 隐含 `docs_user`。
 - `docs_admin` 隐含 `docs_developer` 和 `docs_user`。
-- `admin` 和 `super_admin` 默认拥有全部 Docs 权限。
+- `admin` 和 `super_admin` 默认拥有全部文档权限。
 
 ## 后端入口
 
@@ -61,15 +61,15 @@ DocsMetadata(
     "public",
     "Manual",
     2,
-    "Planet 使用手册",
-    "Planet Manual",
+    "智能星球使用手册",
+    "Intelligent Planet Manual",
 )
 ```
 
-新增公开文档时，需要同步：
+新增可在 Docs 页面展示的技术文档时，需要同步：
 
 - 新增中英文 Markdown 文件。
-- 在服务端 `DOCS_METADATA` 添加 filename、slug、access、group、order、标题。
+- 在服务端 `DOCS_METADATA` 添加 filename、slug、access、group、order、标题。后端目录接口以这里为准，只改前端 metadata 不会让文档出现在 `/docs` 导航中。
 - 在前端 [docs-content.ts](/home/ray/dev/linkong/planet/frontend/src/pages/Docs/docs-content.ts) 添加同名 metadata，保持导航标题和排序一致。
 - 如果需要从 README 发现，更新 `docs/technical/zh/README.md` 和 `docs/technical/en/README.md`。
 
@@ -83,7 +83,7 @@ DocsMetadata(
 - 更新用户时校验组名只能是 `docs_user`、`docs_developer`、`docs_admin`。
 - 只有 `super_admin` 能修改 Gatekeeper 权限组。
 
-前端 [Users.tsx](/home/ray/dev/linkong/planet/frontend/src/pages/Users/Users.tsx) 展示权限组标签，并在编辑表单中提供多选框。非 `super_admin` 打开的表单会禁用该字段，并在提交前移除 `gatekeeper_groups`。
+前端 [Users.tsx](/home/ray/dev/linkong/planet/frontend/src/admin/pages/Users.tsx) 展示权限组标签，并在编辑表单中提供多选框。非 `super_admin` 打开的表单会禁用该字段，并在提交前移除 `gatekeeper_groups`。
 
 ## 前端 Docs 加载
 

docs/technical/zh/earth-bgp-context.md

@@ -188,7 +188,7 @@ Earth info-card 策略：
 - 这是预期行为，因为 incident 是聚合和去噪后的结果
 - 但 incident-first 渲染会让 Earth 显得过于安静，除非有另一层始终可用的 activity layer
 
-推荐 `activity layer` 的实现细节在 [BGP 区域聚合计划](/home/ray/dev/linkong/planet/docs/plans/earth-bgp-region-aggregation-plan.md) 中展开。
+推荐 `activity layer` 的未来实现细节保存在仓库路径 `docs/plans/earth-bgp-region-aggregation-plan.md`。
 
 因此最近的里程碑是：
 
@@ -259,7 +259,7 @@ Earth 的 `incident` 层不应该像一大片发光区域，而应该像紧凑
 
 相关页面：
 
-- `frontend/src/pages/BGP/BGP.tsx`
+- `frontend/src/admin/pages/PlainResourcePages.tsx`
 
 当前 BGP 控制台页面有三层：
 
@@ -321,7 +321,7 @@ BGP 专项测试位于：
 
 前端：
 
-- `frontend/src/pages/BGP/BGP.tsx`
+- `frontend/src/admin/pages/PlainResourcePages.tsx`
 - `frontend/public/earth/js/bgp.js`
 - `frontend/public/earth/js/main.js`
 - `frontend/public/earth/js/info-card.js`