release: bump version to 0.66.2

.claude/commands/docs.md

@@ -73,6 +73,7 @@ Style:
 - Use fenced code blocks with language tags.
 - Prefer tables for comparisons or parameter lists.
 - Keep snippets concise and relevant.
+- For UI labels, chart labels, feature names, datasource names, and other terms that may become mixed Chinese/English copy, check `docs/technical/{zh,en}/naming-glossary.md` and use the documented display name. If a confusing term is missing, update the glossary in both languages as part of the docs change.
 
 ### Step 4 — Verify
 

.codex/skills/docs/SKILL.md

@@ -52,6 +52,7 @@ rg -n "class |def |function |export |router|@router|interface |type " <path>
 - Keep code snippets short and directly relevant.
 - List related files only when they help future maintainers navigate.
 - Use the repository’s existing language, heading style, and naming conventions.
+- For UI labels, chart labels, feature names, datasource names, and other terms that may become mixed Chinese/English copy, check `docs/technical/{zh,en}/naming-glossary.md` and use the documented display name. If a confusing term is missing, update the glossary in both languages as part of the docs change.
 
 4. Verify:
 

README.md

@@ -169,10 +169,11 @@
 ## 快速启动
 
 ```bash
-# 新机器首次初始化
-./scripts/bootstrap-dev.sh
-# 会自动安装/检查 uv、bun，并同步 Python/前端依赖
+# 新机器或空项目首次初始化
+./planet.sh init
+# 会自动安装/检查 uv、bun，同步 Python/前端依赖
 # 会在缺少时生成 backend/.env、aiprovider/.env、frontend/.env.local
+# 会启动 PostgreSQL/Redis，并创建表、默认数据源和本地默认用户
 
 # 启动前后端服务
 ./planet.sh start
@@ -188,6 +189,9 @@
 
 # 查看服务状态
 ./planet.sh health
+
+# 删除容器、卷、镜像和本地编译状态，执行前需要输入 Y 确认
+./planet.sh destroy
 ```
 
 前端命令约定：

TODO.md

@@ -7,9 +7,6 @@ This file is the active backlog only. Completed history belongs in `docs/CHANGEL
 - [ ] Earth AI command entry: merge natural-language and speech-triggered LLM commands into the existing Earth search panel as described in [Agent Runtime, Earth LLM Command, And Speech Entry Plan](/home/ray/dev/linkong/planet/docs/plans/agents-earth-command-runtime-plan.md).
 - [ ] Earth action executor: implement safe visualization actions for layer toggles, batch highlights, filters, focus, result panels, and clear-highlight behavior.
 - [ ] Earth entity matching: support stable entity ids and batch matching for Beidou satellites, mainland China compute centers, BGP, news, vessels, and cables.
-- [x] High-precision country boundary tile framework: implement the static vector tile builder, versioned seed output, frontend bbox tile loader, debounce, in-flight dedupe, and LRU cache described in [Earth High Precision Boundary Tiles Plan](/home/ray/dev/linkong/planet/docs/plans/earth-high-precision-boundary-tiles-plan.md).
-- [x] Add the `pmtiles-mvt` frontend tile provider contract, MVT decoder dependencies, static PMTiles Nginx handling, collector artifact registration, production readiness check, and user operation docs for Earth boundaries.
-- [x] Split Earth boundary ingestion into standard source collectors (`earth_admin0_boundaries`, `earth_coastline`, `earth_claim_lines`) plus the downstream `earth_boundary_tiles` PMTiles builder.
 - [ ] Replace debug GeoJSON boundary tiles with the real `earth-boundaries-china-pov-v1.pmtiles` production artifact after audited admin-0 / coastline / claim-line sources and the PMTiles toolchain are available.
 - [ ] Import authoritative China POV / coastline / claim-line source packages through the three standard Earth boundary source collectors, then rebuild a versioned PMTiles artifact so highest zoom `8-10` preserves trusted source geometry instead of seed data.
 - [ ] Earth boundary data: acquire or generate auditable China POV geometry for Zangnan, Aksai Chin, Taiwan/Penghu, Diaoyu Dao and affiliated islands, Chiwei Yu, South China Sea islands, Kosovo, Gaza, and the official dashed maritime claim line before implementing final visual changes.
@@ -35,6 +32,10 @@ This file is the active backlog only. Completed history belongs in `docs/CHANGEL
 
 ## AI Provider And Agents
 
+- [ ] Unified integration config schema: implement the shared low-code schema engine for datasource, AI Provider, Web Search, and OCR configuration described in [Integration Config Schema System Plan](/home/ray/dev/linkong/planet/docs/plans/integration-config-schema-system-plan.md).
+- [ ] AI provider routing: finish the OpenClaw-style provider/model routing refactor described in [AI Provider OpenClaw-Style Routing Plan](/home/ray/dev/linkong/planet/docs/plans/ai-provider-openclaw-style-routing-plan.md), so model-specific transport rules live in provider metadata rather than runtime hardcoding.
+- [ ] AI provider catalog: replace the temporary `model_provider_apis` bridge with structured `models_metadata`, discovery descriptors, and incremental model sync with stale marking.
+- [ ] AI provider connectivity: keep the plug action as lightweight network/auth/model-directory validation only, and keep real generation tests inside Playground or explicit “trial run” actions.
 - [ ] Agent runtime foundation: add auditable agent runs, steps, evidence, proposals, and the Agent operations UI described in [Agent Runtime, Earth LLM Command, And Speech Entry Plan](/home/ray/dev/linkong/planet/docs/plans/agents-earth-command-runtime-plan.md).
 - [ ] Agent tool protocol: add backend JSON tool-call fallback, optional provider-native tool compatibility, tool whitelist validation, and policy-gated proposal application.
 - [ ] Speech/ASR integration for agents: add provider-neutral transcription settings and API, defaulting to Whisper-compatible API providers while keeping text commands usable when ASR is unavailable.
@@ -57,6 +58,9 @@ Archived items stay here so old context is not lost. Completed items remain chec
 
 ### Completed
 
+- [x] Implemented the high-precision country boundary tile framework from [Earth High Precision Boundary Tiles Plan](/home/ray/dev/linkong/planet/docs/plans/earth-high-precision-boundary-tiles-plan.md): static vector tile builder, versioned seed output, frontend bbox tile loader, debounce, in-flight dedupe, and LRU cache.
+- [x] Added the `pmtiles-mvt` frontend tile provider contract, MVT decoder dependencies, static PMTiles Nginx handling, collector artifact registration, production readiness check, and user operation docs for Earth boundaries.
+- [x] Split Earth boundary ingestion into standard source collectors (`earth_admin0_boundaries`, `earth_coastline`, `earth_claim_lines`) plus the downstream `earth_boundary_tiles` PMTiles builder.
 - [x] Refined BGP observer and anomaly `hover/click` feel.
 - [x] Added BGP anomaly relationship display with cables / regions.
 - [x] Added the Earth BGP activity layer so the map still feels alive when incident density is low.

VERSION

@@ -1 +1 @@
-0.61.0
+0.66.2

aiprovider/Dockerfile

@@ -2,10 +2,14 @@
 
 ARG PYTHON_IMAGE=python:3.14-slim
 ARG UV_IMAGE=ghcr.io/astral-sh/uv:latest
+ARG AI_PROVIDER_BUILD_FINGERPRINT=unknown
 
 FROM ${UV_IMAGE} AS uv
 FROM ${PYTHON_IMAGE}
 
+ARG AI_PROVIDER_BUILD_FINGERPRINT
+LABEL planet.aiprovider.build-fingerprint="${AI_PROVIDER_BUILD_FINGERPRINT}"
+
 COPY --from=uv /uv /uvx /bin/
 
 WORKDIR /app
@@ -15,12 +19,15 @@ ENV PYTHONUNBUFFERED=1
 ENV UV_COMPILE_BYTECODE=1
 ENV UV_LINK_MODE=copy
 
+RUN mkdir -p /root/.config/uv
+
 RUN apt-get update && apt-get install -y --no-install-recommends \
     curl \
     && rm -rf /var/lib/apt/lists/*
 
 COPY pyproject.toml uv.lock /app/
 RUN --mount=type=cache,target=/root/.cache/uv \
+    --mount=type=secret,id=planet_uv_config,target=/root/.config/uv/uv.toml,required=false \
     uv sync --frozen --no-dev
 
 COPY aiprovider /app/aiprovider

aiprovider/main.py

@@ -45,6 +45,7 @@ def get_provider_service(
     x_ai_model: str | None = Header(default=None),
     x_ai_max_tokens: str | None = Header(default=None),
     x_ai_anthropic_version: str | None = Header(default=None),
+    x_ai_model_provider_apis: str | None = Header(default=None),
 ) -> ProviderService:
     overrides = {
         "provider": x_ai_provider,
@@ -53,6 +54,7 @@ def get_provider_service(
         "api_key": x_ai_api_key,
         "model": x_ai_model,
         "anthropic_version": x_ai_anthropic_version,
+        "model_provider_apis": x_ai_model_provider_apis,
     }
     if x_ai_max_tokens:
         overrides["max_tokens"] = x_ai_max_tokens

aiprovider/provider_service.py

@@ -1,6 +1,7 @@
 from __future__ import annotations
 
 import asyncio
+import json
 from typing import Any
 
 import httpx
@@ -14,7 +15,6 @@ from aiprovider.schemas import (
     SituationalAnalysisResponse,
 )
 
-
 def _normalize_provider(value: str) -> str:
     return (value or "disabled").strip().lower()
 
@@ -62,6 +62,9 @@ class ProviderService:
         self.anthropic_version = str(
             overrides.get("anthropic_version") or settings.AI_ANTHROPIC_VERSION
         )
+        self.model_provider_apis = self._parse_model_provider_apis(
+            overrides.get("model_provider_apis")
+        )
 
     def get_status(self) -> AIProviderStatusResponse:
         enabled = self.provider != "disabled"
@@ -93,11 +96,13 @@ class ProviderService:
 
         prompt = self._build_prompt(payload)
 
-        if self.provider_api == "openai-completions":
+        provider_api = self._resolve_model_provider_api(model)
+
+        if provider_api == "openai-completions":
             data = await self._request_openai_compatible(model, prompt, payload.system_prompt)
             content = self._extract_openai_content(data)
             content_blocks = self._extract_openai_blocks(data)
-        elif self.provider_api == "anthropic-messages":
+        elif provider_api == "anthropic-messages":
             data = await self._request_anthropic_messages(
                 model,
                 prompt,
@@ -106,7 +111,7 @@ class ProviderService:
             )
             content = self._extract_anthropic_content(data)
             content_blocks = self._extract_anthropic_blocks(data)
-        elif self.provider_api == "ollama-generate":
+        elif provider_api == "ollama-generate":
             data = await self._request_ollama(model, prompt, payload.system_prompt)
             content = self._extract_ollama_content(data)
             content_blocks = self._extract_ollama_blocks(data)
@@ -132,6 +137,26 @@ class ProviderService:
     def _requires_api_key(self) -> bool:
         return self.provider_api != "ollama-generate"
 
+    def _resolve_model_provider_api(self, model: str) -> str:
+        return self.model_provider_apis.get(model) or self.provider_api
+
+    def _parse_model_provider_apis(self, value: Any) -> dict[str, str]:
+        if isinstance(value, dict):
+            raw = value
+        elif isinstance(value, str) and value.strip():
+            try:
+                parsed = json.loads(value)
+            except json.JSONDecodeError:
+                return {}
+            raw = parsed if isinstance(parsed, dict) else {}
+        else:
+            raw = {}
+        return {
+            str(model): _normalize_provider_api(str(provider_api))
+            for model, provider_api in raw.items()
+            if model and provider_api
+        }
+
     def _build_prompt(self, payload: SituationalAnalysisRequest) -> str:
         sections = [
             f"任务标题:\n{payload.title}",
@@ -314,13 +339,19 @@ class ProviderService:
         message = choices[0].get("message") or {}
         content = message.get("content")
         if isinstance(content, str):
-            return content
+            if content:
+                return content
+            reasoning_content = message.get("reasoning_content")
+            return reasoning_content if isinstance(reasoning_content, str) else ""
         if isinstance(content, list):
             return "".join(
                 item.get("text", "")
                 for item in content
                 if isinstance(item, dict)
             )
+        reasoning_content = message.get("reasoning_content")
+        if isinstance(reasoning_content, str):
+            return reasoning_content
         return ""
 
     def _extract_openai_blocks(self, payload: dict[str, Any]) -> list[AIContentBlock]:
@@ -331,9 +362,14 @@ class ProviderService:
         message = choices[0].get("message") or {}
         content = message.get("content")
         if isinstance(content, str):
-            return [AIContentBlock(type="text", text=content)]
+            blocks = [AIContentBlock(type="text", text=content)] if content else []
+            reasoning_content = message.get("reasoning_content")
+            if isinstance(reasoning_content, str) and reasoning_content:
+                blocks.append(AIContentBlock(type="thinking", thinking=reasoning_content))
+            return blocks
         if not isinstance(content, list):
-            return []
+            reasoning_content = message.get("reasoning_content")
+            return [AIContentBlock(type="thinking", thinking=reasoning_content)] if isinstance(reasoning_content, str) and reasoning_content else []
 
         blocks: list[AIContentBlock] = []
         for item in content:
@@ -346,7 +382,11 @@ class ProviderService:
                     metadata={k: v for k, v in item.items() if k not in {"type", "text"}},
                 )
             )
+        reasoning_content = message.get("reasoning_content")
+        if isinstance(reasoning_content, str) and reasoning_content:
+            blocks.append(AIContentBlock(type="thinking", thinking=reasoning_content))
         return blocks
+
     def _extract_anthropic_content(self, payload: dict[str, Any]) -> str:
         content = payload.get("content")
         if isinstance(content, str):

backend/Dockerfile

@@ -1,3 +1,5 @@
+# syntax=docker/dockerfile:1.7
+
 ARG PYTHON_IMAGE=python:3.14-slim
 ARG UV_IMAGE=ghcr.io/astral-sh/uv:latest
 
@@ -14,12 +16,16 @@ ENV UV_COMPILE_BYTECODE=1
 ENV UV_LINK_MODE=copy
 ENV PYTHONPATH=/app/backend
 
+RUN mkdir -p /root/.config/uv
+
 RUN apt-get update && apt-get install -y --no-install-recommends \
     curl \
     && rm -rf /var/lib/apt/lists/*
 
 COPY pyproject.toml uv.lock /app/
-RUN uv sync --frozen --no-dev
+RUN --mount=type=cache,target=/root/.cache/uv \
+    --mount=type=secret,id=planet_uv_config,target=/root/.config/uv/uv.toml,required=false \
+    uv sync --frozen --no-dev
 
 COPY backend /app/backend
 COPY VERSION /app/VERSION

backend/app/api/main.py

@@ -19,6 +19,7 @@ from app.api.v1 import (
     vessels,
     bgp,
     news,
+    interactables,
     realtime_sources,
     system_control,
     tv,
@@ -53,4 +54,5 @@ api_router.include_router(vessels.router, prefix="/vessels", tags=["vessels"])
 api_router.include_router(bgp.router, prefix="/bgp", tags=["bgp"])
 api_router.include_router(tv.router, prefix="/tv", tags=["tv"])
 api_router.include_router(news.router, prefix="/news", tags=["news"])
+api_router.include_router(interactables.router, prefix="/interactables", tags=["interactables"])
 api_router.include_router(realtime_sources.router, prefix="/realtime-sources", tags=["realtime-sources"])

backend/app/api/v1/ai.py

@@ -3,6 +3,7 @@ from uuid import uuid4
 from fastapi import APIRouter, Depends, HTTPException, Request, Response
 from sqlalchemy.ext.asyncio import AsyncSession
 
+from app.core.logging import get_logger
 from app.core.security import get_current_user
 from app.db.session import get_db
 from app.models.user import User
@@ -47,8 +48,10 @@ from app.services.playground_chat_service import (
     stop_message,
 )
 from app.services.situational_alert_ai_brief import build_situational_alert_brief_request
+from app.services.business_logs import emit_business_log, exception_context
 
 router = APIRouter()
+logger = get_logger(__name__, service="api")
 
 
 @router.get("/provider/status", response_model=AIProviderStatusResponse)
@@ -122,6 +125,16 @@ async def create_playground_message(
     provider_client: AIProviderClient = Depends(get_ai_provider_client),
     db: AsyncSession = Depends(get_db),
 ):
+    await emit_business_log(
+        logger,
+        event="ai.playground.message.create",
+        message="Playground message creation requested",
+        category="ai",
+        service="api",
+        module=__name__,
+        user_id=current_user.id,
+        context={"session_key": payload.session_key, "preset": payload.selected_preset_key},
+    )
     return await create_turn(
         db,
         user_id=current_user.id,
@@ -136,6 +149,16 @@ async def stop_playground_message(
     current_user: User = Depends(get_current_user),
     db: AsyncSession = Depends(get_db),
 ):
+    await emit_business_log(
+        logger,
+        event="ai.playground.message.stop",
+        message="Playground message stop requested",
+        category="ai",
+        service="api",
+        module=__name__,
+        user_id=current_user.id,
+        context={"session_key": payload.session_key, "message_id": payload.message_id},
+    )
     return await stop_message(
         db,
         user_id=current_user.id,
@@ -150,6 +173,16 @@ async def resend_playground_message(
     provider_client: AIProviderClient = Depends(get_ai_provider_client),
     db: AsyncSession = Depends(get_db),
 ):
+    await emit_business_log(
+        logger,
+        event="ai.playground.message.resend",
+        message="Playground message resend requested",
+        category="ai",
+        service="api",
+        module=__name__,
+        user_id=current_user.id,
+        context={"session_key": payload.session_key, "user_message_id": payload.user_message_id},
+    )
     return await resend_turn(
         db,
         user_id=current_user.id,
@@ -214,16 +247,70 @@ async def analyze_bgp_brief(
         anomaly_limit=payload.anomaly_limit,
         collector_limit=payload.collector_limit,
     )
+    await emit_business_log(
+        logger,
+        event="ai.brief.bgp.facts_collected",
+        message="BGP brief facts collected",
+        category="ai",
+        service="api",
+        module=__name__,
+        request_id=request_id,
+        user_id=current_user.id,
+        context={
+            "incident_limit": payload.incident_limit,
+            "anomaly_limit": payload.anomaly_limit,
+            "collector_limit": payload.collector_limit,
+            "fact_count": len(facts or []),
+        },
+    )
     brief_request.preferred_model = payload.preferred_model
     brief_request.thinking = payload.thinking
 
-    analysis = await provider_client.analyze(brief_request, request_id=request_id)
-    return save_bgp_brief_record(
-        analysis,
+    await emit_business_log(
+        logger,
+        event="ai.brief.bgp.start",
+        message="BGP brief AI analysis started",
+        category="ai",
+        service="api",
+        module=__name__,
         request_id=request_id,
-        facts=facts,
-        context=context,
+        user_id=current_user.id,
+        context={"preferred_model": payload.preferred_model},
     )
+    try:
+        analysis = await provider_client.analyze(brief_request, request_id=request_id)
+        record = save_bgp_brief_record(
+            analysis,
+            request_id=request_id,
+            facts=facts,
+            context=context,
+        )
+        await emit_business_log(
+            logger,
+            event="ai.brief.bgp.completed",
+            message="BGP brief AI analysis saved",
+            category="ai",
+            service="api",
+            module=__name__,
+            request_id=request_id,
+            user_id=current_user.id,
+            context={"provider": analysis.provider, "model": analysis.model, "brief_id": record.id},
+        )
+        return record
+    except Exception as exc:
+        await emit_business_log(
+            logger,
+            event="ai.brief.bgp.failed",
+            message="BGP brief AI analysis failed",
+            category="ai",
+            level="error",
+            service="api",
+            module=__name__,
+            request_id=request_id,
+            user_id=current_user.id,
+            context=exception_context(exc, {"preferred_model": payload.preferred_model}),
+        )
+        raise
 
 
 @router.post("/alerts/brief", response_model=AlertBriefResponse)
@@ -242,17 +329,65 @@ async def analyze_alert_brief(
         db,
         alert_limit=payload.alert_limit,
     )
+    await emit_business_log(
+        logger,
+        event="ai.brief.alerts.facts_collected",
+        message="Alert brief facts collected",
+        category="ai",
+        service="api",
+        module=__name__,
+        request_id=request_id,
+        user_id=current_user.id,
+        context={"alert_limit": payload.alert_limit, "fact_count": len(facts or [])},
+    )
     brief_request.preferred_model = payload.preferred_model
     brief_request.thinking = payload.thinking
 
-    analysis = await provider_client.analyze(brief_request, request_id=request_id)
-    return AlertBriefResponse(
-        **analysis.model_dump(),
-        title=brief_request.title,
-        objective=brief_request.objective,
-        facts=facts,
-        context=context,
+    await emit_business_log(
+        logger,
+        event="ai.brief.alerts.start",
+        message="Alert brief AI analysis started",
+        category="ai",
+        service="api",
+        module=__name__,
+        request_id=request_id,
+        user_id=current_user.id,
+        context={"preferred_model": payload.preferred_model},
     )
+    try:
+        analysis = await provider_client.analyze(brief_request, request_id=request_id)
+        await emit_business_log(
+            logger,
+            event="ai.brief.alerts.completed",
+            message="Alert brief AI analysis completed",
+            category="ai",
+            service="api",
+            module=__name__,
+            request_id=request_id,
+            user_id=current_user.id,
+            context={"provider": analysis.provider, "model": analysis.model},
+        )
+        return AlertBriefResponse(
+            **analysis.model_dump(),
+            title=brief_request.title,
+            objective=brief_request.objective,
+            facts=facts,
+            context=context,
+        )
+    except Exception as exc:
+        await emit_business_log(
+            logger,
+            event="ai.brief.alerts.failed",
+            message="Alert brief AI analysis failed",
+            category="ai",
+            level="error",
+            service="api",
+            module=__name__,
+            request_id=request_id,
+            user_id=current_user.id,
+            context=exception_context(exc, {"preferred_model": payload.preferred_model}),
+        )
+        raise
 
 
 @router.post("/situational-alerts/brief", response_model=SituationalAlertBriefResponse)
@@ -268,14 +403,62 @@ async def analyze_situational_alert_brief(
     response.headers["X-Request-ID"] = request_id
 
     brief_request, facts, context = await build_situational_alert_brief_request(db)
+    await emit_business_log(
+        logger,
+        event="ai.brief.situational_alerts.facts_collected",
+        message="Situational alert brief facts collected",
+        category="ai",
+        service="api",
+        module=__name__,
+        request_id=request_id,
+        user_id=current_user.id,
+        context={"fact_count": len(facts or [])},
+    )
     brief_request.preferred_model = payload.preferred_model
     brief_request.thinking = payload.thinking
 
-    analysis = await provider_client.analyze(brief_request, request_id=request_id)
-    return SituationalAlertBriefResponse(
-        **analysis.model_dump(),
-        title=brief_request.title,
-        objective=brief_request.objective,
-        facts=facts,
-        context=context,
+    await emit_business_log(
+        logger,
+        event="ai.brief.situational_alerts.start",
+        message="Situational alert brief AI analysis started",
+        category="ai",
+        service="api",
+        module=__name__,
+        request_id=request_id,
+        user_id=current_user.id,
+        context={"preferred_model": payload.preferred_model},
     )
+    try:
+        analysis = await provider_client.analyze(brief_request, request_id=request_id)
+        await emit_business_log(
+            logger,
+            event="ai.brief.situational_alerts.completed",
+            message="Situational alert brief AI analysis completed",
+            category="ai",
+            service="api",
+            module=__name__,
+            request_id=request_id,
+            user_id=current_user.id,
+            context={"provider": analysis.provider, "model": analysis.model},
+        )
+        return SituationalAlertBriefResponse(
+            **analysis.model_dump(),
+            title=brief_request.title,
+            objective=brief_request.objective,
+            facts=facts,
+            context=context,
+        )
+    except Exception as exc:
+        await emit_business_log(
+            logger,
+            event="ai.brief.situational_alerts.failed",
+            message="Situational alert brief AI analysis failed",
+            category="ai",
+            level="error",
+            service="api",
+            module=__name__,
+            request_id=request_id,
+            user_id=current_user.id,
+            context=exception_context(exc, {"preferred_model": payload.preferred_model}),
+        )
+        raise

backend/app/api/v1/datasource_config.py

@@ -5,7 +5,7 @@ from datetime import datetime
 import base64
 import json
 import re
-from fastapi import APIRouter, Depends, HTTPException, Query
+from fastapi import APIRouter, Depends, HTTPException, Query, Request, status
 from sqlalchemy import delete, select, func
 from sqlalchemy.ext.asyncio import AsyncSession
 from pydantic import BaseModel, Field
@@ -45,14 +45,70 @@ from app.services.custom_datasource_runtime import (
 
 DATASOURCE_MAPPING_PROMPT_KEY = "datasource.mapping"
 from app.services.datasource_connectivity import (
+    _resolve_aisstream_api_key,
+    _resolve_spacetrack_credentials_with_override,
     get_builtin_connection_status,
     save_connectivity_success,
     strip_connectivity_validation,
     test_builtin_connectivity,
 )
+from app.services.barentswatch import resolve_barentswatch_config
+from app.services.persistent_logs import record_audit_log
 
 router = APIRouter()
 
+SECRET_REVEAL_ROLES = {"admin", "super_admin"}
+
+
+def _user_role_value(user: User) -> str:
+    role = getattr(user, "role", "")
+    return str(getattr(role, "value", role) or "").lower()
+
+
+def _user_display_name(user: User) -> str:
+    return str(getattr(user, "username", None) or getattr(user, "email", None) or getattr(user, "id", ""))
+
+
+async def _record_datasource_secret_reveal(
+    *,
+    current_user: User,
+    request: Request,
+    target_id: str,
+    result: str,
+    details: dict[str, Any],
+) -> None:
+    await record_audit_log(
+        action="datasource_config.secret.reveal",
+        actor_id=getattr(current_user, "id", None),
+        actor_name=_user_display_name(current_user),
+        target_type="datasource_config_secret",
+        target_id=target_id,
+        result=result,
+        ip=request.client.host if request.client else None,
+        details=details,
+    )
+
+
+async def _ensure_datasource_secret_reveal_allowed(
+    current_user: User,
+    request: Request,
+    target_id: str,
+    details: dict[str, Any],
+) -> None:
+    if _user_role_value(current_user) in SECRET_REVEAL_ROLES:
+        return
+    await _record_datasource_secret_reveal(
+        current_user=current_user,
+        request=request,
+        target_id=target_id,
+        result="denied",
+        details={**details, "role": _user_role_value(current_user)},
+    )
+    raise HTTPException(
+        status_code=status.HTTP_403_FORBIDDEN,
+        detail="Only administrators can reveal datasource credentials",
+    )
+
 def _default_builtin_config(name: str) -> dict[str, Any]:
     return {"timeout": 30, "retry": 3}
 
@@ -389,10 +445,14 @@ async def list_all_datasources(
         db_config = db_configs.get(name)
         default_config = _default_builtin_config(name)
         default_url = yaml_url
+        db_auth_config = db_config.auth_config or {} if db_config else {}
 
         result.append(
             {
                 "name": name,
+                "requires_credentials": bool(metadata.get("requires_credentials", False)),
+                "credential_provider": metadata.get("credential_provider"),
+                "credential_status": metadata.get("credential_status", "none"),
                 "default_url": default_url,
                 "endpoint": db_config.endpoint if db_config else default_url,
                 "is_overridden": db_config is not None and db_config.endpoint != yaml_url
@@ -401,10 +461,20 @@ async def list_all_datasources(
                 "is_active": db_config.is_active if db_config else True,
                 "source_type": db_config.source_type if db_config else _default_builtin_source_type(name),
                 "auth_type": db_config.auth_type if db_config else "none",
+                "auth_config": {
+                    "client_id": db_auth_config.get("client_id") or "",
+                    "username": db_auth_config.get("username") or "",
+                    "key_name": db_auth_config.get("key_name") or db_auth_config.get("param_name") or "",
+                    "param_name": db_auth_config.get("param_name") or db_auth_config.get("key_name") or "",
+                    "location": db_auth_config.get("location") or db_auth_config.get("in") or "",
+                    "in": db_auth_config.get("in") or db_auth_config.get("location") or "",
+                },
                 "auth_configured": {
-                    "api_key": bool((db_config.auth_config or {}).get("api_key"))
-                    if db_config
-                    else False,
+                    "api_key": bool(db_auth_config.get("api_key")),
+                    "client_id": bool(db_auth_config.get("client_id")),
+                    "client_secret": bool(db_auth_config.get("client_secret")),
+                    "username": bool(db_auth_config.get("username")),
+                    "password": bool(db_auth_config.get("password")),
                 },
                 "headers": db_config.headers if db_config else {},
                 "config": strip_connectivity_validation(db_config.config if db_config else default_config),
@@ -418,6 +488,96 @@ async def list_all_datasources(
     return {"total": len(result), "data": result}
 
 
+@router.get("/configs/secrets")
+async def reveal_builtin_config_secrets(
+    request: Request,
+    name: str = Query(..., min_length=1),
+    current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    """Reveal configured built-in datasource credentials for admin editing."""
+    source = name.strip()
+    metadata = DEFAULT_DATASOURCES.get(source)
+    if not metadata or not metadata.get("requires_credentials"):
+        raise HTTPException(status_code=404, detail="Credentialed datasource config not found")
+
+    provider = str(metadata.get("credential_provider") or "")
+    target_id = f"datasource_config:{source}"
+    await _ensure_datasource_secret_reveal_allowed(
+        current_user,
+        request,
+        target_id,
+        {"source": source, "provider": provider},
+    )
+
+    result = await db.execute(select(DataSourceConfig).where(DataSourceConfig.name == source))
+    record = result.scalar_one_or_none()
+    auth_config = dict(record.auth_config or {}) if record else {}
+    payload: dict[str, Any] = {
+        "name": source,
+        "provider": provider,
+    }
+    details: dict[str, Any] = {"source": source, "provider": provider}
+
+    if provider == "barentswatch":
+        resolved = await resolve_barentswatch_config(db)
+        client_id = str(auth_config.get("client_id") or resolved.client_id or "")
+        client_secret = str(auth_config.get("client_secret") or resolved.client_secret or "")
+        source_label = "datasource_config" if auth_config.get("client_id") or auth_config.get("client_secret") else resolved.credential_source
+        payload.update(
+            {
+                "client_id": client_id,
+                "client_secret": client_secret,
+                "client_id_source": source_label if client_id else "missing",
+                "client_secret_source": source_label if client_secret else "missing",
+            }
+        )
+        details.update(
+            {
+                "client_id_configured": bool(client_id),
+                "client_secret_configured": bool(client_secret),
+                "credential_source": source_label,
+            }
+        )
+    elif provider == "aisstream":
+        api_key, api_key_source = await _resolve_aisstream_api_key(db)
+        payload.update({"api_key": api_key, "api_key_source": api_key_source})
+        details.update({"api_key_configured": bool(api_key), "api_key_source": api_key_source})
+    elif provider == "spacetrack":
+        if auth_config.get("username") or auth_config.get("password"):
+            username = str(auth_config.get("username") or "")
+            password = str(auth_config.get("password") or "")
+            credential_source = "datasource_config"
+        else:
+            username, password, credential_source = _resolve_spacetrack_credentials_with_override()
+        payload.update(
+            {
+                "username": username,
+                "password": password,
+                "username_source": credential_source if username else "missing",
+                "password_source": credential_source if password else "missing",
+            }
+        )
+        details.update(
+            {
+                "username_configured": bool(username),
+                "password_configured": bool(password),
+                "credential_source": credential_source,
+            }
+        )
+    else:
+        raise HTTPException(status_code=400, detail="Datasource credential provider is not supported")
+
+    await _record_datasource_secret_reveal(
+        current_user=current_user,
+        request=request,
+        target_id=target_id,
+        result="success",
+        details=details,
+    )
+    return payload
+
+
 @router.get("/configs/{config_id}")
 async def get_config(
     config_id: int,

backend/app/api/v1/datasources.py

@@ -1,4 +1,3 @@
-import asyncio
 from datetime import datetime, timedelta, timezone
 from typing import Optional
 
@@ -7,6 +6,7 @@ from pydantic import BaseModel, Field
 from sqlalchemy import func, or_, select, text
 from sqlalchemy.ext.asyncio import AsyncSession
 
+from app.core.logging import get_logger
 from app.core.time import to_iso8601_utc
 from app.core.security import get_current_user
 from app.core.data_sources import get_data_sources_config
@@ -21,15 +21,26 @@ from app.models.user import User
 from app.models.vessel import AISRawObservation
 from app.services.vessel_ais_aggregation import VESSEL_AIS_SCHEMA
 from app.services.scheduler import (
-    cancel_running_collector_now,
-    get_latest_task_id_for_datasource,
-    run_collector_now,
     sync_datasource_job,
 )
+from app.services.data_jobs import (
+    JOB_STATUS_CANCELLING,
+    JOB_STATUS_QUEUED,
+    JOB_STATUS_RUNNING,
+    JOB_TYPE_CLEAR_CACHE,
+    JOB_TYPE_CLEAR_DATA,
+    JOB_TYPE_COLLECT,
+    enqueue_datasource_job,
+    get_active_datasource_job,
+    request_cancel_datasource_task,
+)
+from app.services.business_logs import emit_business_log
 
 router = APIRouter()
+logger = get_logger(__name__, service="api")
 STALE_RUNNING_TASK_TIMEOUT_MINUTES = 90
 
+
 PRODUCT_SOURCE_KEYWORDS: tuple[tuple[str, tuple[str, ...]], ...] = (
     ("vessels", ("vessel", "ais")),
     ("cables", ("cable", "landing", "telegeography", "arcgis", "fao")),
@@ -113,7 +124,7 @@ async def _load_latest_running_tasks(
             _task_rank_column(CollectionTask.started_at),
         )
         .where(CollectionTask.datasource_id.in_(datasource_ids))
-        .where(CollectionTask.status == "running")
+        .where(CollectionTask.status.in_((JOB_STATUS_QUEUED, JOB_STATUS_RUNNING, JOB_STATUS_CANCELLING)))
         .subquery()
     )
     result = await db.execute(
@@ -124,32 +135,6 @@ async def _load_latest_running_tasks(
     return {task.datasource_id: task for task in result.scalars().all()}
 
 
-async def _load_latest_task_ids(
-    db: AsyncSession,
-    datasource_ids: list[int],
-) -> dict[int, int]:
-    if not datasource_ids:
-        return {}
-
-    ranked_tasks = (
-        select(
-            CollectionTask.id.label("task_id"),
-            CollectionTask.datasource_id.label("datasource_id"),
-            func.row_number().over(
-                partition_by=CollectionTask.datasource_id,
-                order_by=CollectionTask.id.desc(),
-            ).label("row_num"),
-        )
-        .where(CollectionTask.datasource_id.in_(datasource_ids))
-        .subquery()
-    )
-    result = await db.execute(
-        select(ranked_tasks.c.datasource_id, ranked_tasks.c.task_id)
-        .where(ranked_tasks.c.row_num == 1)
-    )
-    return {datasource_id: task_id for datasource_id, task_id in result.all()}
-
-
 async def _load_latest_tasks(
     db: AsyncSession,
     datasource_ids: list[int],
@@ -235,6 +220,8 @@ async def _load_datasource_endpoint_overrides(
 async def _load_datasource_list_context(
     db: AsyncSession,
     datasources: list[DataSource],
+    *,
+    include_endpoint: bool = True,
 ) -> tuple[dict[int, CollectionTask], dict[int, CollectionTask], dict[str, str]]:
     datasource_ids = [datasource.id for datasource in datasources]
     sources = [datasource.source for datasource in datasources]
@@ -260,10 +247,68 @@ async def _load_datasource_list_context(
         running_tasks = await _load_latest_running_tasks(db, datasource_ids)
 
     latest_tasks = await _load_latest_tasks(db, datasource_ids)
-    endpoint_overrides = await _load_datasource_endpoint_overrides(db, sources)
+    endpoint_overrides = await _load_datasource_endpoint_overrides(db, sources) if include_endpoint else {}
     return running_tasks, latest_tasks, endpoint_overrides
 
 
+def serialize_datasource_row(
+    datasource: DataSource,
+    *,
+    running_tasks: dict[int, CollectionTask],
+    latest_tasks: dict[int, CollectionTask],
+    record_counts: dict[str, int],
+    endpoint_overrides: dict[str, str],
+    config,
+    include_endpoint: bool,
+) -> dict:
+    running_task = running_tasks.get(datasource.id)
+    latest_task = latest_tasks.get(datasource.id)
+    display_task = running_task or latest_task
+    endpoint = None
+    if include_endpoint:
+        endpoint = endpoint_overrides.get(datasource.source) or config.get_yaml_url(datasource.source)
+    last_run_at = datasource.last_run_at or (latest_task.completed_at if latest_task else None)
+    last_status = datasource.last_status or (latest_task.status if latest_task else None)
+    collected_records = record_counts.get(datasource.source, 0)
+
+    row = {
+        "id": datasource.id,
+        "source": datasource.source,
+        "name": datasource.name,
+        **datasource_metadata(datasource.source),
+        "product": datasource_product_key(datasource),
+        "module": datasource.module,
+        "priority": datasource.priority,
+        "frequency": format_frequency_label(datasource.frequency_minutes),
+        "frequency_minutes": datasource.frequency_minutes,
+        "is_active": datasource.is_active,
+        "collector_class": datasource.collector_class,
+        "last_run": to_iso8601_utc(last_run_at),
+        "last_run_at": to_iso8601_utc(last_run_at),
+        "last_status": last_status,
+        "is_running": running_task is not None and running_task.task_type == JOB_TYPE_COLLECT,
+        "is_task_active": running_task is not None,
+        "task_status": running_task.status if running_task else None,
+        "task_id": display_task.id if display_task else None,
+        "task_type": display_task.task_type if display_task else None,
+        "progress": display_task.progress if display_task else None,
+        "phase": display_task.phase if display_task else None,
+        "phase_progress": display_task.phase_progress if display_task else None,
+        "phase_message": display_task.phase_message if display_task else None,
+        "phase_current": display_task.phase_current if display_task else None,
+        "phase_total": display_task.phase_total if display_task else None,
+        "phase_unit": display_task.phase_unit if display_task else None,
+        "records_processed": display_task.records_processed if display_task else None,
+        "total_records": display_task.total_records if display_task else None,
+        "error_message": display_task.error_message if display_task else None,
+        "collected_records": collected_records,
+        "has_collected_data": collected_records > 0,
+    }
+    if include_endpoint:
+        row["endpoint"] = endpoint
+    return row
+
+
 def _apply_datasource_query_filters(
     query,
     *,
@@ -341,9 +386,25 @@ async def _trigger_datasource_batch(
     datasources: list[DataSource],
     *,
     force: bool,
+    actor_id: int | None = None,
+    trigger_kind: str = "batch",
 ) -> dict:
+    await emit_business_log(
+        logger,
+        event=f"collector.trigger.{trigger_kind}.start",
+        message="Datasource batch trigger started",
+        category="collector",
+        service="api",
+        module=__name__,
+        user_id=actor_id,
+        context={
+            "trigger_kind": trigger_kind,
+            "force": force,
+            "requested_count": len(datasources),
+        },
+    )
     if not datasources:
-        return {
+        result = {
             "status": "noop",
             "message": "No matching data sources to trigger",
             "force": force,
@@ -351,8 +412,18 @@ async def _trigger_datasource_batch(
             "skipped": [],
             "failed": [],
         }
+        await emit_business_log(
+            logger,
+            event=f"collector.trigger.{trigger_kind}.completed",
+            message="Datasource batch trigger completed with no matching sources",
+            category="collector",
+            service="api",
+            module=__name__,
+            user_id=actor_id,
+            context={"trigger_kind": trigger_kind, "force": force, "status": "noop", "triggered_count": 0},
+        )
+        return result
 
-    previous_task_ids: dict[int, Optional[int]] = {}
     triggered_sources: list[dict] = []
     skipped_sources: list[dict] = []
     failed_sources: list[dict] = []
@@ -387,9 +458,11 @@ async def _trigger_datasource_batch(
                     }
                 )
                 continue
-            cancelled = await cancel_running_collector_now(datasource.source)
-            if not cancelled:
-                await rollback_orphaned_running_task(db, datasource, running_task)
+            await request_cancel_datasource_task(
+                db,
+                running_task,
+                reason="superseded_by_forced_collection",
+            )
 
         if not force and not is_due_for_collection(datasource, now):
             skipped_sources.append(
@@ -406,57 +479,51 @@ async def _trigger_datasource_batch(
             )
             continue
 
-        previous_task_ids[datasource.id] = None
-        success = run_collector_now(datasource.source)
-        if not success:
-            failed_sources.append(
-                {
-                    "id": datasource.id,
-                    "source": datasource.source,
-                    "name": datasource.name,
-                    "reason": "trigger_failed",
-                }
-            )
-            continue
+        task = await enqueue_datasource_job(
+            db,
+            datasource,
+            JOB_TYPE_COLLECT,
+            payload={"force": force, "trigger": "batch"},
+        )
 
         triggered_sources.append(
             {
                 "id": datasource.id,
                 "source": datasource.source,
                 "name": datasource.name,
-                "task_id": None,
+                "task_id": task.id,
             }
         )
 
-    latest_task_ids = await _load_latest_task_ids(
-        db,
-        [datasource.id for datasource in datasources],
-    )
-    for datasource_id in previous_task_ids:
-        previous_task_ids[datasource_id] = latest_task_ids.get(datasource_id)
-
-    for _ in range(20):
-        await asyncio.sleep(0.1)
-        pending = [item for item in triggered_sources if item["task_id"] is None]
-        if not pending:
-            break
-        latest_task_ids = await _load_latest_task_ids(
-            db,
-            [item["id"] for item in pending],
-        )
-        for item in pending:
-            task_id = latest_task_ids.get(item["id"])
-            if task_id is not None and task_id != previous_task_ids.get(item["id"]):
-                item["task_id"] = task_id
-
-    return {
-        "status": "triggered" if triggered_sources else "partial",
-        "message": f"Triggered {len(triggered_sources)} data sources",
+    result = {
+        "status": "queued" if triggered_sources else "partial",
+        "message": f"Queued {len(triggered_sources)} data source jobs",
         "force": force,
         "triggered": triggered_sources,
         "skipped": skipped_sources,
         "failed": failed_sources,
     }
+    await emit_business_log(
+        logger,
+        event=f"collector.trigger.{trigger_kind}.completed",
+        message="Datasource batch trigger completed",
+        category="collector",
+        service="api",
+        module=__name__,
+        user_id=actor_id,
+        context={
+            "trigger_kind": trigger_kind,
+            "force": force,
+            "status": result["status"],
+            "requested_count": len(datasources),
+            "triggered_count": len(triggered_sources),
+            "skipped_count": len(skipped_sources),
+            "failed_count": len(failed_sources),
+            "triggered_sources": [item["source"] for item in triggered_sources],
+            "skipped_reasons": [item["reason"] for item in skipped_sources],
+        },
+    )
+    return result
 
 
 async def get_datasource_record(db: AsyncSession, source_id: str) -> Optional[DataSource]:
@@ -658,6 +725,7 @@ async def list_datasources(
     collected: Optional[bool] = None,
     credential_status: Optional[str] = None,
     q: Optional[str] = None,
+    include_endpoint: bool = True,
     current_user: User = Depends(get_current_user),
     db: AsyncSession = Depends(get_db),
 ):
@@ -676,7 +744,11 @@ async def list_datasources(
 
     collector_list = []
     config = get_data_sources_config()
-    running_tasks, latest_tasks, endpoint_overrides = await _load_datasource_list_context(db, datasources)
+    running_tasks, latest_tasks, endpoint_overrides = await _load_datasource_list_context(
+        db,
+        datasources,
+        include_endpoint=include_endpoint,
+    )
     record_counts = await _load_collected_record_counts(db, [datasource.source for datasource in datasources])
     datasources = _filter_datasources_in_memory(
         datasources,
@@ -689,46 +761,16 @@ async def list_datasources(
         credential_status=credential_status,
     )
     for datasource in datasources:
-        running_task = running_tasks.get(datasource.id)
-        latest_task = latest_tasks.get(datasource.id)
-        display_task = running_task or latest_task
-        endpoint = endpoint_overrides.get(datasource.source) or config.get_yaml_url(datasource.source)
-        last_run_at = datasource.last_run_at or (latest_task.completed_at if latest_task else None)
-        last_status = datasource.last_status or (latest_task.status if latest_task else None)
-        collected_records = record_counts.get(datasource.source, 0)
-
         collector_list.append(
-            {
-                "id": datasource.id,
-                "source": datasource.source,
-                "name": datasource.name,
-                **datasource_metadata(datasource.source),
-                "product": datasource_product_key(datasource),
-                "module": datasource.module,
-                "priority": datasource.priority,
-                "frequency": format_frequency_label(datasource.frequency_minutes),
-                "frequency_minutes": datasource.frequency_minutes,
-                "is_active": datasource.is_active,
-                "collector_class": datasource.collector_class,
-                "endpoint": endpoint,
-                "last_run": to_iso8601_utc(last_run_at),
-                "last_run_at": to_iso8601_utc(last_run_at),
-                "last_status": last_status,
-                "is_running": running_task is not None,
-                "task_id": display_task.id if display_task else None,
-                "progress": display_task.progress if display_task else None,
-                "phase": display_task.phase if display_task else None,
-                "phase_progress": display_task.phase_progress if display_task else None,
-                "phase_message": display_task.phase_message if display_task else None,
-                "phase_current": display_task.phase_current if display_task else None,
-                "phase_total": display_task.phase_total if display_task else None,
-                "phase_unit": display_task.phase_unit if display_task else None,
-                "records_processed": display_task.records_processed if display_task else None,
-                "total_records": display_task.total_records if display_task else None,
-                "error_message": display_task.error_message if display_task else None,
-                "collected_records": collected_records,
-                "has_collected_data": collected_records > 0,
-            }
+            serialize_datasource_row(
+                datasource,
+                running_tasks=running_tasks,
+                latest_tasks=latest_tasks,
+                record_counts=record_counts,
+                endpoint_overrides=endpoint_overrides,
+                config=config,
+                include_endpoint=include_endpoint,
+            )
         )
 
     return {"total": len(collector_list), "data": collector_list}
@@ -746,7 +788,13 @@ async def trigger_all_datasources(
         .order_by(DataSource.module, DataSource.id)
     )
     datasources = result.scalars().all()
-    return await _trigger_datasource_batch(db, datasources, force=force)
+    return await _trigger_datasource_batch(
+        db,
+        datasources,
+        force=force,
+        actor_id=current_user.id,
+        trigger_kind="all",
+    )
 
 
 @router.post("/trigger-batch")
@@ -782,7 +830,58 @@ async def trigger_datasource_batch(
         collected=None if payload.source_ids else payload.collected,
         credential_status=None if payload.source_ids else payload.credential_status,
     )
-    return await _trigger_datasource_batch(db, datasources, force=payload.force)
+    return await _trigger_datasource_batch(
+        db,
+        datasources,
+        force=payload.force,
+        actor_id=current_user.id,
+        trigger_kind="batch",
+    )
+
+
+@router.get("/snapshots")
+async def list_datasource_snapshots(
+    source_id: Optional[str] = None,
+    current_only: Optional[bool] = None,
+    limit: int = Query(default=100, ge=1, le=500),
+    current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    query = (
+        select(DataSnapshot, DataSource.name, DataSource.module)
+        .outerjoin(DataSource, DataSource.id == DataSnapshot.datasource_id)
+        .order_by(DataSnapshot.created_at.desc().nullslast(), DataSnapshot.id.desc())
+        .limit(limit)
+    )
+    if source_id:
+        query = query.where(DataSnapshot.source == source_id)
+    if current_only is not None:
+        query = query.where(DataSnapshot.is_current.is_(current_only))
+
+    result = await db.execute(query)
+    rows = []
+    for snapshot, datasource_name, datasource_module in result.all():
+        rows.append(
+            {
+                "id": snapshot.id,
+                "datasource_id": snapshot.datasource_id,
+                "datasource_name": datasource_name,
+                "module": datasource_module,
+                "task_id": snapshot.task_id,
+                "source": snapshot.source,
+                "snapshot_key": snapshot.snapshot_key,
+                "reference_date": to_iso8601_utc(snapshot.reference_date),
+                "started_at": to_iso8601_utc(snapshot.started_at),
+                "completed_at": to_iso8601_utc(snapshot.completed_at),
+                "record_count": snapshot.record_count,
+                "status": snapshot.status,
+                "is_current": snapshot.is_current,
+                "parent_snapshot_id": snapshot.parent_snapshot_id,
+                "summary": snapshot.summary or {},
+                "created_at": to_iso8601_utc(snapshot.created_at),
+            }
+        )
+    return {"total": len(rows), "data": rows}
 
 
 @router.get("/{source_id}")
@@ -813,6 +912,37 @@ async def get_datasource(
     }
 
 
+@router.get("/{source_id}/row")
+async def get_datasource_row(
+    source_id: str,
+    include_endpoint: bool = True,
+    current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    datasource = await get_datasource_record(db, source_id)
+    if not datasource:
+        raise HTTPException(status_code=404, detail="Data source not found")
+
+    config = get_data_sources_config()
+    running_tasks, latest_tasks, endpoint_overrides = await _load_datasource_list_context(
+        db,
+        [datasource],
+        include_endpoint=include_endpoint,
+    )
+    record_counts = await _load_collected_record_counts(db, [datasource.source])
+    return {
+        "data": serialize_datasource_row(
+            datasource,
+            running_tasks=running_tasks,
+            latest_tasks=latest_tasks,
+            record_counts=record_counts,
+            endpoint_overrides=endpoint_overrides,
+            config=config,
+            include_endpoint=include_endpoint,
+        )
+    }
+
+
 @router.post("/{source_id}/enable")
 async def enable_datasource(
     source_id: str,
@@ -882,8 +1012,24 @@ async def trigger_datasource(
     if not datasource.is_active:
         raise HTTPException(status_code=400, detail="Data source is disabled")
 
-    running_task = await get_running_task(db, datasource.id)
+    running_task = await get_active_datasource_job(db, datasource.id, task_types=(JOB_TYPE_COLLECT,))
     if running_task is not None and not force:
+        await emit_business_log(
+            logger,
+            event="collector.trigger.single.skipped_already_running",
+            message="Datasource trigger skipped because a task is already running",
+            category="collector",
+            level="warning",
+            service="api",
+            module=__name__,
+            user_id=current_user.id,
+            context={
+                "collector_name": datasource.source,
+                "datasource_id": datasource.id,
+                "task_id": running_task.id,
+                "status": "skipped",
+            },
+        )
         raise HTTPException(
             status_code=409,
             detail={
@@ -903,31 +1049,42 @@ async def trigger_datasource(
         )
 
     if running_task is not None and force:
-        cancelled = await cancel_running_collector_now(datasource.source)
-        if not cancelled:
-            await rollback_orphaned_running_task(db, datasource, running_task)
+        await request_cancel_datasource_task(
+            db,
+            running_task,
+            reason="superseded_by_forced_collection",
+        )
 
-    previous_task_id = await get_latest_task_id_for_datasource(datasource.id)
-    success = run_collector_now(datasource.source)
-    if not success:
-        raise HTTPException(status_code=500, detail=f"Failed to trigger collector '{datasource.source}'")
-
-    task_id = None
-    for _ in range(20):
-        await asyncio.sleep(0.1)
-        task_id = await get_latest_task_id_for_datasource(datasource.id)
-        if task_id is not None and task_id != previous_task_id:
-            break
-    if task_id == previous_task_id:
-        task_id = None
+    task = await enqueue_datasource_job(
+        db,
+        datasource,
+        JOB_TYPE_COLLECT,
+        payload={"force": force, "trigger": "single"},
+    )
+    await emit_business_log(
+        logger,
+        event="collector.trigger.single.completed",
+        message="Datasource trigger queued",
+        category="collector",
+        service="api",
+        module=__name__,
+        user_id=current_user.id,
+        context={
+            "collector_name": datasource.source,
+            "datasource_id": datasource.id,
+            "task_id": task.id,
+            "force": force,
+            "status": "queued",
+        },
+    )
 
     return {
-        "status": "triggered",
+        "status": "queued",
         "source_id": datasource.id,
-        "task_id": task_id,
+        "task_id": task.id,
         "collector_name": datasource.source,
         "force": force,
-        "message": f"Collector '{datasource.source}' has been triggered",
+        "message": f"Collector '{datasource.source}' has been queued",
     }
 
 
@@ -941,22 +1098,81 @@ async def clear_datasource_data(
     if not datasource:
         raise HTTPException(status_code=404, detail="Data source not found")
 
-    result = await db.execute(
-        select(func.count(CollectedData.id)).where(CollectedData.source == datasource.source)
+    active_task = await get_active_datasource_job(db, datasource.id)
+    if active_task is not None:
+        raise HTTPException(
+            status_code=409,
+            detail={
+                "reason": "datasource_job_in_progress",
+                "message": "当前数据源已有任务在执行，请等待完成或先取消任务。",
+                "task_id": active_task.id,
+                "task_type": active_task.task_type,
+                "status": active_task.status,
+            },
+        )
+    task = await enqueue_datasource_job(
+        db,
+        datasource,
+        JOB_TYPE_CLEAR_DATA,
+        payload={"source": datasource.source},
     )
-    count = result.scalar() or 0
-
-    if count == 0:
-        return {"status": "success", "message": "No data to clear", "deleted_count": 0}
-
-    delete_query = CollectedData.__table__.delete().where(CollectedData.source == datasource.source)
-    await db.execute(delete_query)
-    await db.commit()
 
     return {
-        "status": "success",
-        "message": f"Cleared {count} records for data source '{datasource.name}'",
-        "deleted_count": count,
+        "status": "queued",
+        "message": f"Queued data clearing for data source '{datasource.name}'",
+        "task_id": task.id,
+        "deleted_count": None,
+    }
+
+
+@router.delete("/{source_id}/cache")
+async def clear_datasource_cache(
+    source_id: str,
+    current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    datasource = await get_datasource_record(db, source_id)
+    if not datasource:
+        raise HTTPException(status_code=404, detail="Data source not found")
+
+    task = await enqueue_datasource_job(
+        db,
+        datasource,
+        JOB_TYPE_CLEAR_CACHE,
+        payload={"source": datasource.source},
+        dedupe_key=f"clear_cache:{datasource.source}",
+    )
+
+    return {
+        "status": "queued",
+        "message": f"Queued cache clearing for data source '{datasource.name}'",
+        "task_id": task.id,
+        "deleted_count": None,
+    }
+
+
+@router.post("/{source_id}/tasks/{task_id}/cancel")
+async def cancel_datasource_task(
+    source_id: str,
+    task_id: int,
+    current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    datasource = await get_datasource_record(db, source_id)
+    if not datasource:
+        raise HTTPException(status_code=404, detail="Data source not found")
+
+    task = await db.get(CollectionTask, task_id)
+    if not task or task.datasource_id != datasource.id:
+        raise HTTPException(status_code=404, detail="Task not found")
+
+    task = await request_cancel_datasource_task(db, task)
+    return {
+        "status": "cancelled" if task.completed_at else "cancelling",
+        "task_id": task.id,
+        "task_type": task.task_type,
+        "phase": task.phase,
+        "requested_cancel_at": to_iso8601_utc(task.requested_cancel_at),
     }
 
 
@@ -976,7 +1192,7 @@ async def get_task_status(
         if not task or task.datasource_id != datasource.id:
             raise HTTPException(status_code=404, detail="Task not found")
     else:
-        task = await get_running_task(db, datasource.id)
+        task = await get_active_datasource_job(db, datasource.id)
         if task is None:
             result = await db.execute(
                 select(CollectionTask)
@@ -1001,8 +1217,12 @@ async def get_task_status(
         }
 
     return {
-        "is_running": task.status == "running",
+        "is_running": task.status in {JOB_STATUS_QUEUED, JOB_STATUS_RUNNING, JOB_STATUS_CANCELLING}
+        and task.task_type == JOB_TYPE_COLLECT,
+        "is_task_active": task.status in {JOB_STATUS_QUEUED, JOB_STATUS_RUNNING, JOB_STATUS_CANCELLING},
         "task_id": task.id,
+        "task_type": task.task_type,
+        "task_status": task.status,
         "progress": task.progress,
         "phase": task.phase,
         "phase_progress": task.phase_progress,
@@ -1013,5 +1233,6 @@ async def get_task_status(
         "records_processed": task.records_processed,
         "total_records": task.total_records,
         "status": task.status,
+        "requested_cancel_at": to_iso8601_utc(task.requested_cancel_at),
         "error_message": task.error_message,
     }

backend/app/api/v1/earth.py

@@ -9,13 +9,18 @@ from uuid import uuid4
 from fastapi import APIRouter, Depends, File, HTTPException, Request, UploadFile, status
 from fastapi.security import HTTPAuthorizationCredentials, HTTPBearer
 from pydantic import BaseModel, Field
-from sqlalchemy import delete, select, text
+from sqlalchemy import delete, func, select, text
 from sqlalchemy.ext.asyncio import AsyncSession
 
+from app.core.config import settings as app_settings
 from app.core.security import decode_token, get_current_user, redis_client
 from app.db.session import get_db
+from app.models.collected_data import CollectedData
+from app.models.datasource import DataSource
+from app.models.datasource_config import DataSourceConfig
 from app.models.system_setting import SystemSetting
 from app.models.user import User
+from app.services.tv_streams import get_tv_settings_payload
 from app.services.earth_boundaries import (
     EarthBoundaryBuildError,
     get_boundary_build_status,
@@ -31,9 +36,17 @@ REPO_ROOT = Path(__file__).resolve().parents[4]
 EARTH_BRAND_ASSET_DIR = REPO_ROOT / "data" / "earth-brand"
 EARTH_BRAND_ASSET_URL_PREFIX = "/earth-brand-assets"
 EARTH_BRAND_CATEGORY = "earth_brand"
+EARTH_ABOUT_CATEGORY = "earth_about"
+SYSTEM_SETTINGS_CATEGORY = "system"
 MAX_EARTH_BRAND_ASSET_BYTES = 3 * 1024 * 1024
 ALLOWED_EARTH_BRAND_EXTENSIONS = {".png", ".jpg", ".jpeg", ".webp", ".svg"}
 
+
+def _app_version_label() -> str:
+    version = str(app_settings.VERSION or "").strip() or "0.0.0"
+    return version if version.startswith("v") else f"v{version}"
+
+
 DEFAULT_EARTH_BRAND = {
     "logo_src": "/earth/assets/brand/earth-logo.png",
     "title_src": "/earth/assets/brand/title-zh.png",
@@ -44,6 +57,20 @@ DEFAULT_EARTH_BRAND = {
     "title_alt": "智能星球计划",
 }
 
+DEFAULT_EARTH_ABOUT = {
+    "logo_src": "/earth/assets/brand/lim-logo.png",
+    "kicker": "About",
+    "title": "智能星球计划",
+    "version": _app_version_label(),
+    "description": "面向临空场景下的智能媒体研究、全球态势感知与多源开放数据巡航，提供可视化观测、事件聚合与交互式探索能力。",
+    "meta": [
+        {"label": "出品方", "value": "浙江大学临空智能媒体研究院"},
+        {"label": "策划人", "value": "方兴东、黄柳青"},
+        {"label": "产品兼开发者", "value": "钱坤、张鸽、齐鹏"},
+    ],
+}
+EARTH_ABOUT_LEGACY_PLANNER_VALUE = "黄柳青"
+
 
 class EarthBoundaryConfigPayload(BaseModel):
     config: dict[str, Any] = Field(default_factory=dict)
@@ -59,6 +86,20 @@ class EarthBrandPayload(BaseModel):
     title_alt: str = Field(default=DEFAULT_EARTH_BRAND["title_alt"], max_length=200)
 
 
+class EarthAboutMetaItem(BaseModel):
+    label: str = Field(default="", max_length=80)
+    value: str = Field(default="", max_length=240)
+
+
+class EarthAboutPayload(BaseModel):
+    logo_src: str = Field(default=DEFAULT_EARTH_ABOUT["logo_src"], max_length=1000)
+    kicker: str = Field(default=DEFAULT_EARTH_ABOUT["kicker"], max_length=80)
+    title: str = Field(default=DEFAULT_EARTH_ABOUT["title"], max_length=160)
+    version: str = Field(default=DEFAULT_EARTH_ABOUT["version"], max_length=80)
+    description: str = Field(default=DEFAULT_EARTH_ABOUT["description"], max_length=800)
+    meta: list[EarthAboutMetaItem] = Field(default_factory=list)
+
+
 def _normalize_earth_brand_payload(payload: dict[str, Any] | None) -> dict[str, str]:
     merged = DEFAULT_EARTH_BRAND.copy()
     if payload:
@@ -76,6 +117,47 @@ def _normalize_earth_brand_payload(payload: dict[str, Any] | None) -> dict[str,
     return merged
 
 
+def _normalize_earth_about_payload(payload: dict[str, Any] | None) -> dict[str, Any]:
+    merged: dict[str, Any] = {
+        key: value
+        for key, value in DEFAULT_EARTH_ABOUT.items()
+        if key != "meta"
+    }
+    raw_meta = DEFAULT_EARTH_ABOUT["meta"]
+    if payload:
+        for key in ("logo_src", "kicker", "title", "description"):
+            value = payload.get(key)
+            if value is not None:
+                merged[key] = str(value).strip()
+        raw_meta = payload.get("meta") if isinstance(payload.get("meta"), list) else raw_meta
+    merged["version"] = _app_version_label()
+
+    for key, default_value in DEFAULT_EARTH_ABOUT.items():
+        if key == "meta":
+            continue
+        if not merged.get(key):
+            merged[key] = default_value
+
+    normalized_meta: list[dict[str, str]] = []
+    for item in raw_meta:
+        if not isinstance(item, dict):
+            continue
+        label = str(item.get("label") or "").strip()
+        value = str(item.get("value") or "").strip()
+        if label == "策划人" and value == EARTH_ABOUT_LEGACY_PLANNER_VALUE:
+            value = "方兴东、黄柳青"
+        if label or value:
+            normalized_meta.append({"label": label, "value": value})
+    if not normalized_meta:
+        normalized_meta = [dict(item) for item in DEFAULT_EARTH_ABOUT["meta"]]
+    merged["meta"] = normalized_meta
+    return merged
+
+
+def _is_demo_mode_enabled(payload: Any) -> bool:
+    return bool(payload.get("demo_mode")) if isinstance(payload, dict) else False
+
+
 async def _get_earth_brand_record(db: AsyncSession) -> SystemSetting | None:
     result = await db.execute(
         select(SystemSetting).where(SystemSetting.category == EARTH_BRAND_CATEGORY)
@@ -91,6 +173,56 @@ async def _get_earth_brand_payload(db: AsyncSession) -> dict[str, Any]:
     }
 
 
+async def _get_earth_about_record(db: AsyncSession) -> SystemSetting | None:
+    result = await db.execute(
+        select(SystemSetting).where(SystemSetting.category == EARTH_ABOUT_CATEGORY)
+    )
+    return result.scalar_one_or_none()
+
+
+async def _get_earth_about_payload(db: AsyncSession) -> dict[str, Any]:
+    record = await _get_earth_about_record(db)
+    return {
+        "about": _normalize_earth_about_payload(record.payload if record else None),
+        "is_default": record is None,
+    }
+
+
+async def _get_optional_current_user(
+    credentials: HTTPAuthorizationCredentials | None = Depends(optional_bearer),
+    db: AsyncSession = Depends(get_db),
+) -> User | None:
+    if credentials is None:
+        return None
+    token = credentials.credentials
+    if redis_client.sismember("blacklisted_tokens", token):
+        return None
+    payload = decode_token(token)
+    if payload is None or payload.get("type") != "access":
+        return None
+    user_id = payload.get("sub")
+    if user_id is None:
+        return None
+    result = await db.execute(
+        text(
+            "SELECT id, username, email, password_hash, role, is_active, gatekeeper_groups FROM users WHERE id = :id"
+        ),
+        {"id": int(user_id)},
+    )
+    row = result.fetchone()
+    if row is None or not row[5]:
+        return None
+    user = User()
+    user.id = row[0]
+    user.username = row[1]
+    user.email = row[2]
+    user.password_hash = row[3]
+    user.role = row[4]
+    user.is_active = row[5]
+    user.gatekeeper_groups = row[6] or []
+    return user
+
+
 @router.get("/brand")
 async def get_earth_brand(db: AsyncSession = Depends(get_db)):
     return await _get_earth_brand_payload(db)
@@ -159,46 +291,108 @@ async def upload_earth_brand_asset(
     return {"url": asset_url, "filename": safe_name, "content_type": file.content_type}
 
 
+@router.get("/about")
+async def get_earth_about(db: AsyncSession = Depends(get_db)):
+    return await _get_earth_about_payload(db)
+
+
+@router.put("/about")
+async def update_earth_about(
+    payload: EarthAboutPayload,
+    _current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    normalized = _normalize_earth_about_payload(payload.model_dump())
+    record = await _get_earth_about_record(db)
+    if record is None:
+        record = SystemSetting(category=EARTH_ABOUT_CATEGORY, payload=normalized)
+        db.add(record)
+    else:
+        record.payload = normalized
+    await db.commit()
+    await db.refresh(record)
+    return {"status": "updated", "about": _normalize_earth_about_payload(record.payload), "is_default": False}
+
+
+@router.delete("/about")
+async def reset_earth_about(
+    _current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    await db.execute(delete(SystemSetting).where(SystemSetting.category == EARTH_ABOUT_CATEGORY))
+    await db.commit()
+    return {"status": "reset", "about": _normalize_earth_about_payload(None), "is_default": True}
+
+
+@router.get("/oobe-status")
+async def get_earth_oobe_status(
+    current_user: User | None = Depends(_get_optional_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    current_count_result = await db.execute(
+        select(func.count(CollectedData.id)).where(CollectedData.is_current.is_(True))
+    )
+    current_record_count = int(current_count_result.scalar() or 0)
+    system_result = await db.execute(
+        select(SystemSetting).where(SystemSetting.category == SYSTEM_SETTINGS_CATEGORY)
+    )
+    system_record = system_result.scalar_one_or_none()
+    demo_mode = _is_demo_mode_enabled(system_record.payload if system_record else None)
+
+    datasource_count_result = await db.execute(select(func.count(DataSource.id)))
+    datasource_count = int(datasource_count_result.scalar() or 0)
+    active_datasource_count_result = await db.execute(
+        select(func.count(DataSource.id)).where(DataSource.is_active.is_(True))
+    )
+    active_datasource_count = int(active_datasource_count_result.scalar() or 0)
+    config_result = await db.execute(select(func.count(DataSourceConfig.id)))
+    custom_config_count = int(config_result.scalar() or 0)
+
+    tv_payload = await get_tv_settings_payload(db)
+    tv_sources = tv_payload.get("sources") if isinstance(tv_payload, dict) else []
+    tv_source_count = len(tv_sources) if isinstance(tv_sources, list) else 0
+
+    boundary_status = get_boundary_status()
+    has_core_layers = bool(boundary_status.get("ready") or boundary_status.get("available") or boundary_status.get("status") in {"ready", "built", "ok"})
+    has_collected_data = current_record_count > 0
+    ready = has_collected_data
+
+    suggestions: list[str] = []
+    if demo_mode:
+        suggestions.append("演示模式已开启")
+    if not current_user:
+        suggestions.append("登录控制台")
+    if not has_collected_data:
+        suggestions.append("触发数据源采集")
+    if not custom_config_count:
+        suggestions.append("确认采集器配置")
+    if not has_core_layers:
+        suggestions.append("构建或启用 Earth 图层")
+
+    return {
+        "ready": ready,
+        "demo_mode": demo_mode,
+        "authenticated": current_user is not None,
+        "needs_login": current_user is None and not ready and not demo_mode,
+        "has_collected_data": has_collected_data,
+        "has_tv_sources": tv_source_count > 0,
+        "has_core_layers": has_core_layers,
+        "current_record_count": current_record_count,
+        "datasource_count": datasource_count,
+        "active_datasource_count": active_datasource_count,
+        "custom_config_count": custom_config_count,
+        "tv_source_count": tv_source_count,
+        "suggestions": suggestions,
+        "login_url": "/login?next=/datasources",
+        "datasources_url": "/datasources",
+        "collection_url": "/collection-management",
+    }
+
+
 @router.get("/boundaries/status")
 async def get_earth_boundary_status():
     return get_boundary_status()
 
-
-async def _get_optional_current_user(
-    credentials: HTTPAuthorizationCredentials | None = Depends(optional_bearer),
-    db: AsyncSession = Depends(get_db),
-) -> User | None:
-    if credentials is None:
-        return None
-    token = credentials.credentials
-    if redis_client.sismember("blacklisted_tokens", token):
-        return None
-    payload = decode_token(token)
-    if payload is None or payload.get("type") != "access":
-        return None
-    user_id = payload.get("sub")
-    if user_id is None:
-        return None
-    result = await db.execute(
-        text(
-            "SELECT id, username, email, password_hash, role, is_active, gatekeeper_groups FROM users WHERE id = :id"
-        ),
-        {"id": int(user_id)},
-    )
-    row = result.fetchone()
-    if row is None or not row[5]:
-        return None
-    user = User()
-    user.id = row[0]
-    user.username = row[1]
-    user.email = row[2]
-    user.password_hash = row[3]
-    user.role = row[4]
-    user.is_active = row[5]
-    user.gatekeeper_groups = row[6] or []
-    return user
-
-
 def _is_loopback_request(request: Request) -> bool:
     host = request.client.host if request.client else ""
     return host in {"127.0.0.1", "::1", "localhost"} or host.startswith("127.")

backend/app/api/v1/interactables.py

@@ -0,0 +1,190 @@
+"""CRUD APIs for persistent Earth interactables."""
+
+from __future__ import annotations
+
+from datetime import UTC, datetime
+from typing import Any
+
+from fastapi import APIRouter, Depends, HTTPException, Query, Response, status
+from pydantic import BaseModel, Field, field_validator
+from sqlalchemy.ext.asyncio import AsyncSession
+
+from app.core.security import get_current_user
+from app.db.session import get_db
+from app.models.earth_interactable import EarthInteractable
+from app.models.user import User
+from app.services.earth_interactables import (
+    build_interactable_event,
+    interactables_to_geojson,
+    invalidate_interactable_cache,
+    list_interactables,
+    normalize_interactable_id,
+    publish_interactable_event,
+    serialize_interactable,
+)
+from app.services.earth_layer_cache import (
+    EarthLayerCachePolicy,
+    earth_layer_cache,
+    get_or_build_layer_payload,
+)
+
+router = APIRouter()
+INTERACTABLE_CACHE_POLICY = EarthLayerCachePolicy(
+    fresh_ttl_seconds=60,
+    stale_ttl_seconds=10 * 60,
+    max_features=5000,
+)
+
+
+class InteractableCreate(BaseModel):
+    id: str | None = Field(default=None, max_length=160)
+    layer: str = Field(default="default", min_length=1, max_length=80)
+    kind: str = Field(default="default", min_length=1, max_length=80)
+    label: str = Field(default="", max_length=255)
+    description: str = Field(default="", max_length=4000)
+    latitude: float = Field(ge=-90, le=90)
+    longitude: float = Field(ge=-180, le=180)
+    altitude: float | None = None
+    properties: dict[str, Any] = Field(default_factory=dict)
+
+    @field_validator("layer", "kind")
+    @classmethod
+    def normalize_key(cls, value: str) -> str:
+        normalized = str(value or "").strip()
+        if not normalized:
+            raise ValueError("must not be empty")
+        return normalized
+
+
+class InteractableUpdate(BaseModel):
+    layer: str | None = Field(default=None, min_length=1, max_length=80)
+    kind: str | None = Field(default=None, min_length=1, max_length=80)
+    label: str | None = Field(default=None, max_length=255)
+    description: str | None = Field(default=None, max_length=4000)
+    latitude: float | None = Field(default=None, ge=-90, le=90)
+    longitude: float | None = Field(default=None, ge=-180, le=180)
+    altitude: float | None = None
+    properties: dict[str, Any] | None = None
+
+
+@router.get("")
+async def get_interactables(
+    response: Response,
+    layer: str | None = Query(default=None),
+    include_deleted: bool = Query(default=False),
+    db: AsyncSession = Depends(get_db),
+):
+    items = await list_interactables(db, layer=layer, include_deleted=include_deleted)
+    response.headers["X-Planet-Interactables-Count"] = str(len(items))
+    return {"items": [serialize_interactable(item) for item in items]}
+
+
+@router.get("/geojson")
+async def get_interactables_geojson(
+    response: Response,
+    layer: str | None = Query(default=None),
+    db: AsyncSession = Depends(get_db),
+):
+    async def build_payload() -> dict[str, Any]:
+        items = await list_interactables(db, layer=layer)
+        return interactables_to_geojson(items)
+
+    payload = await get_or_build_layer_payload(
+        key=earth_layer_cache.key("interactables", layer=layer or "all"),
+        policy=INTERACTABLE_CACHE_POLICY,
+        builder=build_payload,
+        response=response,
+    )
+    response.headers["X-Planet-Interactables-Count"] = str(len(payload.get("features") or []))
+    return payload
+
+
+@router.post("", status_code=status.HTTP_201_CREATED)
+async def create_interactable(
+    payload: InteractableCreate,
+    _current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    record_id = normalize_interactable_id(payload.id)
+    existing = await db.get(EarthInteractable, record_id)
+    if existing and not existing.is_deleted:
+        raise HTTPException(status_code=409, detail="Interactable already exists")
+
+    if existing is None:
+        record = EarthInteractable(id=record_id)
+        db.add(record)
+    else:
+        record = existing
+        record.is_deleted = False
+        record.deleted_at = None
+        record.revision += 1
+
+    record.layer = payload.layer
+    record.kind = payload.kind
+    record.label = payload.label
+    record.description = payload.description
+    record.latitude = payload.latitude
+    record.longitude = payload.longitude
+    record.altitude = payload.altitude
+    record.properties = payload.properties
+
+    await db.commit()
+    await db.refresh(record)
+    invalidate_interactable_cache(record.layer)
+    await publish_interactable_event("created", record)
+    return {"item": serialize_interactable(record)}
+
+
+@router.get("/{interactable_id}")
+async def get_interactable(interactable_id: str, db: AsyncSession = Depends(get_db)):
+    record = await db.get(EarthInteractable, interactable_id)
+    if record is None or record.is_deleted:
+        raise HTTPException(status_code=404, detail="Interactable not found")
+    return {"item": serialize_interactable(record)}
+
+
+@router.patch("/{interactable_id}")
+async def update_interactable(
+    interactable_id: str,
+    payload: InteractableUpdate,
+    _current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    record = await db.get(EarthInteractable, interactable_id)
+    if record is None or record.is_deleted:
+        raise HTTPException(status_code=404, detail="Interactable not found")
+
+    previous_layer = record.layer
+    patch = payload.model_dump(exclude_unset=True)
+    for key, value in patch.items():
+        setattr(record, key, value)
+    record.revision += 1
+
+    await db.commit()
+    await db.refresh(record)
+    invalidate_interactable_cache(previous_layer)
+    if record.layer != previous_layer:
+        invalidate_interactable_cache(record.layer)
+    await publish_interactable_event("updated", record)
+    return {"item": serialize_interactable(record)}
+
+
+@router.delete("/{interactable_id}")
+async def delete_interactable(
+    interactable_id: str,
+    _current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    record = await db.get(EarthInteractable, interactable_id)
+    if record is None or record.is_deleted:
+        raise HTTPException(status_code=404, detail="Interactable not found")
+
+    record.is_deleted = True
+    record.deleted_at = datetime.now(UTC)
+    record.revision += 1
+    await db.commit()
+    await db.refresh(record)
+    invalidate_interactable_cache(record.layer)
+    await publish_interactable_event("deleted", record)
+    event = build_interactable_event(action="deleted", record=record, include_item=True)
+    return {"deleted": True, "event": event}

backend/app/api/v1/settings.py

@@ -4,12 +4,14 @@ import os
 from pathlib import Path
 from typing import Optional
 
-from fastapi import APIRouter, Depends, HTTPException, Query
+from fastapi import APIRouter, Depends, HTTPException, Query, Request, status
+import httpx
 from pydantic import BaseModel, EmailStr, Field
 from dotenv import dotenv_values
 from sqlalchemy import select
 from sqlalchemy.ext.asyncio import AsyncSession
 
+from app.core.logging import get_logger
 from app.core.security import get_current_user
 from app.core.time import to_iso8601_utc
 from app.core.config import settings as app_settings
@@ -64,10 +66,14 @@ from app.services.llm_provider_catalog import (
 )
 from app.services.scheduler import sync_datasource_job
 from app.services.tv_streams import DEFAULT_TV_SETTINGS, get_tv_settings_payload, normalize_tv_settings
+from app.services.persistent_logs import record_audit_log
+from app.services.business_logs import emit_business_log, exception_context
 
 router = APIRouter()
+logger = get_logger(__name__, service="api")
 AI_PROVIDER_QUICK_CONNECT_TIMEOUT_SECONDS = 5
 AI_CONNECTION_TEST_PROMPT_KEY = "ai.connection_test"
+SECRET_REVEAL_ROLES = {"admin", "super_admin"}
 
 DEFAULT_SETTINGS = {
     "system": {
@@ -76,6 +82,7 @@ DEFAULT_SETTINGS = {
         "auto_refresh": True,
         "data_retention_days": 30,
         "max_concurrent_tasks": 5,
+        "demo_mode": False,
     },
     "notifications": {
         "email_enabled": False,
@@ -130,12 +137,77 @@ DEFAULT_SETTINGS = {
 }
 
 
+def _user_role_value(user: User) -> str:
+    role = getattr(user, "role", "")
+    return role.value if hasattr(role, "value") else str(role or "")
+
+
+def _user_display_name(user: User) -> str | None:
+    return getattr(user, "username", None) or getattr(user, "email", None)
+
+
+def _request_client_ip(request: Request | None) -> str | None:
+    if request is None or request.client is None:
+        return None
+    return request.client.host
+
+
+def _can_reveal_integration_secrets(user: User) -> bool:
+    return _user_role_value(user) in SECRET_REVEAL_ROLES
+
+
+async def _record_integration_secret_reveal(
+    *,
+    current_user: User,
+    request: Request | None,
+    target_id: str,
+    result: str,
+    details: dict,
+) -> None:
+    await record_audit_log(
+        action="settings.integration_secret.reveal",
+        actor_id=getattr(current_user, "id", None),
+        actor_name=_user_display_name(current_user),
+        target_type="integration_secret",
+        target_id=target_id,
+        result=result,
+        ip=_request_client_ip(request),
+        details=details,
+    )
+
+
+async def _ensure_secret_reveal_allowed(
+    *,
+    current_user: User,
+    request: Request | None,
+    target_id: str,
+    details: dict | None = None,
+) -> None:
+    if _can_reveal_integration_secrets(current_user):
+        return
+    await _record_integration_secret_reveal(
+        current_user=current_user,
+        request=request,
+        target_id=target_id,
+        result="denied",
+        details={
+            **(details or {}),
+            "role": _user_role_value(current_user),
+        },
+    )
+    raise HTTPException(
+        status_code=status.HTTP_403_FORBIDDEN,
+        detail="Only administrators can reveal integration secrets",
+    )
+
+
 class SystemSettingsUpdate(BaseModel):
     system_name: str = "智能星球"
     refresh_interval: int = Field(default=60, ge=10, le=3600)
     auto_refresh: bool = True
     data_retention_days: int = Field(default=30, ge=1, le=3650)
     max_concurrent_tasks: int = Field(default=5, ge=1, le=50)
+    demo_mode: bool = False
 
 
 class NotificationSettingsUpdate(BaseModel):
@@ -353,9 +425,10 @@ def _get_provider_preset(provider: str) -> dict:
             "provider": provider,
             "provider_api": "openai-completions",
             "base_url": "",
-            "model": "",
-            "models": [],
-            "api_key_env": "",
+        "model": "",
+        "models": [],
+        "model_provider_apis": {},
+        "api_key_env": "",
         }
 
 
@@ -377,6 +450,9 @@ def _resolve_env_secret(*names: str) -> tuple[str, str]:
         value = env_file_values.get(name)
         if value:
             return value, "env_file"
+        value = os.environ.get(name)
+        if value:
+            return value, "env"
     return "", ""
 
 
@@ -419,9 +495,16 @@ def _provider_defaults(provider: str) -> dict:
             1200 if preset.get("provider_api") == "anthropic-messages" else 4096
         ),
         "anthropic_version": "2023-06-01",
+        "model_provider_apis": preset.get("model_provider_apis") or {},
     }
 
 
+def _selected_ai_env_provider() -> str:
+    env_file_values = _read_ai_provider_env_file()
+    provider = env_file_values.get("AI_PROVIDER") or os.environ.get("AI_PROVIDER") or "minimax"
+    return _normalize_provider_id(provider)
+
+
 def _normalize_ai_provider_payload(ai_payload: dict | None) -> dict:
     raw = dict(ai_payload or {})
     default_provider = _normalize_provider_id(raw.get("default_provider") or raw.get("provider"))
@@ -440,6 +523,7 @@ def _normalize_ai_provider_payload(ai_payload: dict | None) -> dict:
             "api_key",
             "max_tokens",
             "anthropic_version",
+            "model_provider_apis",
         )
         if raw.get(key) not in (None, "")
     }
@@ -477,7 +561,12 @@ def _resolve_provider_api_key(provider: str, provider_config: dict) -> tuple[str
         return str(saved_key), "runtime"
     preset = _get_provider_preset(provider)
     api_key_env = preset.get("api_key_env") or ""
-    return _resolve_env_secret(api_key_env, "AI_API_KEY")
+    value, source = _resolve_env_secret(api_key_env)
+    if value:
+        return value, source
+    if _normalize_provider_id(provider) == _selected_ai_env_provider():
+        return _resolve_env_secret("AI_API_KEY")
+    return "", ""
 
 
 def _resolve_service_token(ai_payload: dict) -> tuple[str, str]:
@@ -509,7 +598,12 @@ def _is_secret_placeholder(value: Optional[str], current_preview: str = "") -> b
 
 def _build_ai_provider_payload(current_payload: dict, update: AIProviderIntegrationUpdate) -> dict:
     current_ai = _normalize_ai_provider_payload(current_payload.get("ai_provider") or {})
-    provider_id = _normalize_provider_id(update.default_provider or update.provider)
+    provider_id = _normalize_provider_id(update.provider)
+    default_provider = (
+        _normalize_provider_id(update.default_provider)
+        if update.default_provider is not None
+        else current_ai["default_provider"]
+    )
     current_providers = {
         provider: dict(config or {})
         for provider, config in current_ai.get("providers", {}).items()
@@ -543,7 +637,7 @@ def _build_ai_provider_payload(current_payload: dict, update: AIProviderIntegrat
         "service_url": update.service_url.strip()
         or app_settings.AI_PROVIDER_SERVICE_URL,
         "service_token": current_ai.get("service_token") or "",
-        "default_provider": provider_id,
+        "default_provider": default_provider,
         "providers": current_providers,
         "timeout_seconds": update.timeout_seconds,
         "retry_attempts": update.retry_attempts,
@@ -577,6 +671,8 @@ def _runtime_config_from_ai_payload(ai_payload: dict) -> dict:
             "api_key": api_key,
             "max_tokens": int(provider_config.get("max_tokens") or 1200),
             "anthropic_version": provider_config.get("anthropic_version") or "2023-06-01",
+            "model_provider_apis": provider_config.get("model_provider_apis") or {},
+            "preset_models": _get_provider_preset(default_provider).get("models") or [],
         },
     }
 
@@ -608,6 +704,18 @@ async def _validate_ai_provider_full_connection(ai_payload: dict) -> dict:
         retry_attempts=runtime_config["retry_attempts"],
         llm_config=runtime_config.get("llm_config") or {},
     )
+    await emit_business_log(
+        logger,
+        event="settings.ai_provider.full_connection.start",
+        message="AI provider full connection validation started",
+        category="ai",
+        service="api",
+        module=__name__,
+        context={
+            "provider": runtime_config.get("llm_config", {}).get("provider"),
+            "model": runtime_config.get("llm_config", {}).get("model"),
+        },
+    )
     status_result = await client.get_status()
     if not status_result.configured:
         raise HTTPException(
@@ -624,6 +732,19 @@ async def _validate_ai_provider_full_connection(ai_payload: dict) -> dict:
             constraints=["回复尽量简短。"],
         )
     )
+    await emit_business_log(
+        logger,
+        event="settings.ai_provider.full_connection.success",
+        message="AI provider full connection validation completed",
+        category="ai",
+        service="api",
+        module=__name__,
+        context={
+            "provider": analysis_result.provider,
+            "model": analysis_result.model,
+            "configured": status_result.configured,
+        },
+    )
     return {
         "status": status_result.model_dump(),
         "provider": analysis_result.provider,
@@ -631,6 +752,159 @@ async def _validate_ai_provider_full_connection(ai_payload: dict) -> dict:
     }
 
 
+def _join_provider_url(base_url: str, path: str) -> str:
+    return f"{base_url.rstrip('/')}/{path.lstrip('/')}"
+
+
+def _extract_model_ids(payload: dict) -> list[str]:
+    data = payload.get("data") if isinstance(payload, dict) else None
+    if isinstance(data, list):
+        return [
+            str(item.get("id"))
+            for item in data
+            if isinstance(item, dict) and item.get("id")
+        ]
+    models = payload.get("models") if isinstance(payload, dict) else None
+    if isinstance(models, list):
+        return [
+            str(item.get("name") or item.get("model") or item.get("id") or item)
+            for item in models
+            if item
+        ]
+    return []
+
+
+def _contains_model(model_ids: list[str], model: str) -> bool:
+    normalized_model = model.strip().lower()
+    return any(str(item).strip().lower() == normalized_model for item in model_ids)
+
+
+async def _check_ai_provider_lightweight(llm_config: dict, timeout_seconds: int) -> dict:
+    provider = _normalize_provider_id(llm_config.get("provider") or "")
+    configured_api = str(llm_config.get("provider_api") or "").strip() or "openai-completions"
+    model = str(llm_config.get("model") or "").strip()
+    base_url = str(llm_config.get("base_url") or "").strip().rstrip("/")
+    api_key = str(llm_config.get("api_key") or "").strip()
+    provider_api = configured_api
+    model_provider_apis = llm_config.get("model_provider_apis")
+    if isinstance(model_provider_apis, dict):
+        provider_api = str(model_provider_apis.get(model) or provider_api)
+    preset_models = [
+        str(item)
+        for item in (llm_config.get("preset_models") or [])
+        if str(item).strip()
+    ]
+
+    if not provider or not base_url or not model:
+        return {
+            "success": False,
+            "connected": False,
+            "message": "当前 provider/base_url/model 未完整配置。",
+            "mode": "lightweight_config",
+        }
+    if provider_api != "ollama-generate" and not api_key:
+        return {
+            "success": False,
+            "connected": False,
+            "message": "当前 provider 未配置 API Key。",
+            "mode": "lightweight_config",
+        }
+
+    if provider == "opencode-go":
+        url = _join_provider_url(base_url, "/models")
+        headers = {"Authorization": f"Bearer {api_key}"}
+    elif provider_api == "ollama-generate":
+        url = _join_provider_url(base_url, "/api/tags")
+        headers: dict[str, str] = {}
+    elif provider_api == "openai-completions":
+        url = _join_provider_url(base_url, "/models")
+        headers = {"Authorization": f"Bearer {api_key}"}
+    elif provider_api == "anthropic-messages":
+        url = _join_provider_url(base_url, "/models")
+        headers = {
+            "x-api-key": api_key,
+            "anthropic-version": str(llm_config.get("anthropic_version") or "2023-06-01"),
+        }
+    else:
+        return {
+            "success": False,
+            "connected": False,
+            "message": f"当前 provider_api 不支持轻量连通性测试: {provider_api}",
+            "mode": "lightweight_unsupported",
+        }
+
+    try:
+        async with httpx.AsyncClient(timeout=min(timeout_seconds, AI_PROVIDER_QUICK_CONNECT_TIMEOUT_SECONDS)) as client:
+            response = await client.get(url, headers=headers)
+            response.raise_for_status()
+            payload = response.json()
+    except httpx.HTTPStatusError as exc:
+        detail = exc.response.text or exc.response.reason_phrase
+        if exc.response.status_code == 404 and _contains_model(preset_models, model):
+            return {
+                "success": True,
+                "connected": True,
+                "message": "轻量连通性测试通过；当前 provider 不提供可用的模型目录，已按内置模型预设确认。",
+                "mode": "lightweight_preset",
+                "provider": provider,
+                "provider_api": provider_api,
+                "model": model,
+                "url": url,
+            }
+        return {
+            "success": False,
+            "connected": False,
+            "message": f"轻量连通性测试失败: HTTP {exc.response.status_code} {detail}",
+            "mode": "lightweight_models",
+            "url": url,
+        }
+    except Exception as exc:
+        return {
+            "success": False,
+            "connected": False,
+            "message": f"轻量连通性测试失败: {exc}",
+            "mode": "lightweight_models",
+            "url": url,
+        }
+
+    model_ids = _extract_model_ids(payload)
+    if model_ids and not _contains_model(model_ids, model):
+        if _contains_model(preset_models, model):
+            return {
+                "success": True,
+                "connected": True,
+                "message": "轻量连通性测试通过；provider 模型目录未返回当前别名，已按内置模型预设确认。",
+                "mode": "lightweight_models_with_preset_alias",
+                "provider": provider,
+                "provider_api": provider_api,
+                "model": model,
+                "models_count": len(model_ids),
+                "url": url,
+            }
+        return {
+            "success": False,
+            "connected": False,
+            "message": f"连接可用，但模型目录中没有当前模型: {model}",
+            "mode": "lightweight_models",
+            "provider": provider,
+            "model": model,
+            "models_count": len(model_ids),
+            "url": url,
+        }
+
+    return {
+        "success": True,
+        "connected": True,
+        "message": "轻量连通性测试通过",
+        "mode": "lightweight_models",
+        "provider": provider,
+        "provider_api": provider_api,
+        "model": model,
+        "models_count": len(model_ids),
+        "url": url,
+    }
+
+
 def _web_search_provider_defaults(provider: str) -> dict:
     return web_search_provider_defaults(provider).model_dump()
 
@@ -691,12 +965,21 @@ def _normalize_web_search_payload(web_search_payload: dict | None) -> dict:
     }
 
 
-def _resolve_web_search_api_key(provider: str, provider_config: dict) -> tuple[str, str]:
+def _resolve_web_search_api_key(
+    provider: str,
+    provider_config: dict,
+    default_provider: str | None = None,
+) -> tuple[str, str]:
     saved_key = provider_config.get("api_key") or ""
     if saved_key:
         return str(saved_key), "runtime"
     preset = get_web_search_provider_preset(provider)
-    return _resolve_web_search_env_secret(preset.get("api_key_env") or "", "WEB_SEARCH_API_KEY")
+    value, source = _resolve_web_search_env_secret(preset.get("api_key_env") or "")
+    if value:
+        return value, source
+    if normalize_web_search_provider(provider) == normalize_web_search_provider(default_provider or "tavily"):
+        return _resolve_web_search_env_secret("WEB_SEARCH_API_KEY")
+    return "", ""
 
 
 def _build_web_search_payload(
@@ -706,13 +989,22 @@ def _build_web_search_payload(
     current_web_search = _normalize_web_search_payload(current_payload.get("web_search") or {})
     if update is None:
         return current_web_search
-    provider_id = normalize_web_search_provider(update.default_provider or update.provider)
+    provider_id = normalize_web_search_provider(update.provider)
+    default_provider = (
+        normalize_web_search_provider(update.default_provider)
+        if update.default_provider is not None
+        else current_web_search["default_provider"]
+    )
     current_providers = {
         provider: dict(config or {})
         for provider, config in current_web_search.get("providers", {}).items()
     }
     current_provider = current_providers.get(provider_id) or _web_search_provider_defaults(provider_id)
-    current_key, current_key_source = _resolve_web_search_api_key(provider_id, current_provider)
+    current_key, current_key_source = _resolve_web_search_api_key(
+        provider_id,
+        current_provider,
+        current_web_search["default_provider"],
+    )
     current_key_preview = _mask_secret(current_key, current_key_source)["preview"]
     provider_payload = {
         **_web_search_provider_defaults(provider_id),
@@ -745,7 +1037,7 @@ def _build_web_search_payload(
     current_providers[provider_id] = provider_payload
     return {
         "enabled": update.enabled,
-        "default_provider": provider_id,
+        "default_provider": default_provider,
         "providers": current_providers,
     }
 
@@ -754,12 +1046,12 @@ def _runtime_config_from_web_search_payload(web_search_payload: dict) -> WebSear
     normalized = _normalize_web_search_payload(web_search_payload)
     provider_id = normalized["default_provider"]
     provider_config = normalized["providers"].get(provider_id) or _web_search_provider_defaults(provider_id)
-    api_key, _source = _resolve_web_search_api_key(provider_id, provider_config)
+    api_key, _source = _resolve_web_search_api_key(provider_id, provider_config, provider_id)
     provider_models = {
         provider: WebSearchProviderConfig(**{
             **config,
             "api_key": (
-                api_key if provider == provider_id else _resolve_web_search_api_key(provider, config)[0]
+                api_key if provider == provider_id else _resolve_web_search_api_key(provider, config, provider_id)[0]
             ),
         })
         for provider, config in normalized["providers"].items()
@@ -893,7 +1185,11 @@ async def serialize_external_integrations(db: AsyncSession) -> dict:
             normalized_web_search["providers"].get(provider_id)
             or _web_search_provider_defaults(provider_id)
         )
-        api_key, api_key_source = _resolve_web_search_api_key(provider_id, provider_config)
+        api_key, api_key_source = _resolve_web_search_api_key(
+            provider_id,
+            provider_config,
+            normalized_web_search["default_provider"],
+        )
         web_search_providers_payload[provider_id] = {
             **{
                 key: value
@@ -980,10 +1276,7 @@ async def save_external_integrations_payload(
     update: ExternalIntegrationsUpdate,
 ) -> dict:
     current_payload = await get_setting_payload(db, "external_integrations")
-    current_ai_payload = _normalize_ai_provider_payload(current_payload.get("ai_provider") or {})
     ai_payload = _build_ai_provider_payload(current_payload, update.ai_provider)
-    if _ai_provider_runtime_fingerprint(ai_payload) != _ai_provider_runtime_fingerprint(current_ai_payload):
-        await _validate_ai_provider_full_connection(ai_payload)
     web_search_payload = _build_web_search_payload(current_payload, update.web_search)
     ocr_payload = _build_ocr_payload(current_payload, update.ocr)
 
@@ -1331,6 +1624,10 @@ async def connect_ai_provider_integration(
     db: AsyncSession = Depends(get_db),
 ):
     current_payload = await get_setting_payload(db, "external_integrations")
+    # Connection testing should validate the provider being edited, not the
+    # currently saved default provider. This is a transient draft only and is
+    # intentionally not persisted.
+    payload = payload.model_copy(update={"default_provider": payload.provider})
     draft_ai_payload = _build_ai_provider_payload(current_payload, payload)
     runtime_config = _runtime_config_from_ai_payload(draft_ai_payload)
     quick_llm_config = {
@@ -1345,33 +1642,65 @@ async def connect_ai_provider_integration(
         llm_config=quick_llm_config,
     )
 
+    await emit_business_log(
+        logger,
+        event="settings.ai_provider.connect.start",
+        message="AI provider connection test started",
+        category="ai",
+        service="api",
+        module=__name__,
+        user_id=current_user.id,
+        context={
+            "provider": payload.provider,
+            "model": payload.model,
+            "timeout_seconds": min(int(runtime_config["timeout_seconds"] or 60), AI_PROVIDER_QUICK_CONNECT_TIMEOUT_SECONDS),
+        },
+    )
     try:
         status_result = await client.get_status()
         if not status_result.configured:
+            await emit_business_log(
+                logger,
+                event="settings.ai_provider.connect.failed",
+                message="AI provider connection test failed because provider is incomplete",
+                category="ai",
+                level="warning",
+                service="api",
+                module=__name__,
+                user_id=current_user.id,
+                context={"provider": payload.provider, "model": payload.model, "configured": False},
+            )
             return {
                 "success": False,
                 "connected": False,
                 "message": "AI Provider 可访问，但当前 provider/model/key 未完整配置。",
                 "status": status_result.model_dump(),
             }
-        prompt = await get_effective_prompt(db, AI_CONNECTION_TEST_PROMPT_KEY)
-        probe_result = await client.analyze(
-            SituationalAnalysisRequest(
-                title="快速连接测试",
-                objective=prompt.prompt,
-                system_prompt=prompt.system_prompt or None,
-                observations=[],
-                constraints=["Output only OK."],
-            )
+        lightweight_result = await _check_ai_provider_lightweight(
+            quick_llm_config,
+            timeout_seconds=min(
+                int(runtime_config["timeout_seconds"] or 60),
+                AI_PROVIDER_QUICK_CONNECT_TIMEOUT_SECONDS,
+            ),
+        )
+        await emit_business_log(
+            logger,
+            event="settings.ai_provider.connect.success",
+            message="AI provider connection test completed",
+            category="ai",
+            service="api",
+            module=__name__,
+            user_id=current_user.id,
+            context={
+                "provider": payload.provider,
+                "model": payload.model,
+                "configured": True,
+                "lightweight_status": lightweight_result.get("status"),
+            },
         )
         return {
-            "success": True,
-            "connected": True,
-            "message": "连接测试通过",
+            **lightweight_result,
             "status": status_result.model_dump(),
-            "provider": probe_result.provider,
-            "model": probe_result.model,
-            "mode": "quick_probe",
         }
     except HTTPException as exc:
         return {
@@ -1380,6 +1709,17 @@ async def connect_ai_provider_integration(
             "message": str(exc.detail),
         }
     except Exception as exc:
+        await emit_business_log(
+            logger,
+            event="settings.ai_provider.connect.failed",
+            message="AI provider connection test failed",
+            category="ai",
+            level="error",
+            service="api",
+            module=__name__,
+            user_id=current_user.id,
+            context=exception_context(exc, {"provider": payload.provider, "model": payload.model}),
+        )
         return {
             "success": False,
             "connected": False,
@@ -1389,16 +1729,38 @@ async def connect_ai_provider_integration(
 
 @router.get("/integrations/ai-provider/secrets")
 async def reveal_ai_provider_secrets(
+    request: Request,
     provider: str = Query(default=""),
     current_user: User = Depends(get_current_user),
     db: AsyncSession = Depends(get_db),
 ):
+    requested_provider = _normalize_provider_id(provider) if provider else "default"
+    await _ensure_secret_reveal_allowed(
+        current_user=current_user,
+        request=request,
+        target_id=f"ai_provider:{requested_provider}",
+        details={"kind": "ai_provider", "provider": requested_provider},
+    )
     current_payload = await get_setting_payload(db, "external_integrations")
     ai_payload = _normalize_ai_provider_payload(current_payload.get("ai_provider") or {})
     provider_id = _normalize_provider_id(provider or ai_payload["default_provider"])
     provider_config = ai_payload["providers"].get(provider_id) or _provider_defaults(provider_id)
     api_key, api_key_source = _resolve_provider_api_key(provider_id, provider_config)
     service_token, service_token_source = _resolve_service_token(ai_payload)
+    await _record_integration_secret_reveal(
+        current_user=current_user,
+        request=request,
+        target_id=f"ai_provider:{provider_id}",
+        result="success",
+        details={
+            "kind": "ai_provider",
+            "provider": provider_id,
+            "api_key_configured": bool(api_key),
+            "api_key_source": api_key_source,
+            "service_token_configured": bool(service_token),
+            "service_token_source": service_token_source,
+        },
+    )
     return {
         "provider": provider_id,
         "api_key": api_key,
@@ -1417,10 +1779,18 @@ async def get_web_search_presets(
 
 @router.get("/integrations/web-search/secrets")
 async def reveal_web_search_secrets(
+    request: Request,
     provider: str = Query(default=""),
     current_user: User = Depends(get_current_user),
     db: AsyncSession = Depends(get_db),
 ):
+    requested_provider = normalize_web_search_provider(provider) if provider else "default"
+    await _ensure_secret_reveal_allowed(
+        current_user=current_user,
+        request=request,
+        target_id=f"web_search:{requested_provider}",
+        details={"kind": "web_search", "provider": requested_provider},
+    )
     current_payload = await get_setting_payload(db, "external_integrations")
     web_search_payload = _normalize_web_search_payload(current_payload.get("web_search") or {})
     provider_id = normalize_web_search_provider(provider or web_search_payload["default_provider"])
@@ -1428,7 +1798,23 @@ async def reveal_web_search_secrets(
         web_search_payload["providers"].get(provider_id)
         or _web_search_provider_defaults(provider_id)
     )
-    api_key, api_key_source = _resolve_web_search_api_key(provider_id, provider_config)
+    api_key, api_key_source = _resolve_web_search_api_key(
+        provider_id,
+        provider_config,
+        web_search_payload["default_provider"],
+    )
+    await _record_integration_secret_reveal(
+        current_user=current_user,
+        request=request,
+        target_id=f"web_search:{provider_id}",
+        result="success",
+        details={
+            "kind": "web_search",
+            "provider": provider_id,
+            "api_key_configured": bool(api_key),
+            "api_key_source": api_key_source,
+        },
+    )
     return {
         "provider": provider_id,
         "api_key": api_key,
@@ -1438,12 +1824,31 @@ async def reveal_web_search_secrets(
 
 @router.get("/integrations/ocr/secrets")
 async def reveal_ocr_secrets(
+    request: Request,
     current_user: User = Depends(get_current_user),
     db: AsyncSession = Depends(get_db),
 ):
+    await _ensure_secret_reveal_allowed(
+        current_user=current_user,
+        request=request,
+        target_id="ocr:default",
+        details={"kind": "ocr", "provider": "default"},
+    )
     current_payload = await get_setting_payload(db, "external_integrations")
     ocr_payload = _normalize_ocr_payload(current_payload.get("ocr") or {})
     api_key, api_key_source = _resolve_ocr_api_key(ocr_payload)
+    await _record_integration_secret_reveal(
+        current_user=current_user,
+        request=request,
+        target_id=f"ocr:{ocr_payload['provider']}",
+        result="success",
+        details={
+            "kind": "ocr",
+            "provider": ocr_payload["provider"],
+            "api_key_configured": bool(api_key),
+            "api_key_source": api_key_source,
+        },
+    )
     return {
         "provider": ocr_payload["provider"],
         "api_key": api_key,
@@ -1462,8 +1867,28 @@ async def connect_web_search_integration(
     runtime_config = _runtime_config_from_web_search_payload(draft_web_search_payload)
     client = WebSearchClient(runtime_config)
 
+    await emit_business_log(
+        logger,
+        event="settings.web_search.connect.start",
+        message="WebSearch connection test started",
+        category="ai_tool",
+        service="api",
+        module=__name__,
+        user_id=current_user.id,
+        context={"provider": runtime_config.default_provider},
+    )
     try:
         results = await client.test_connection()
+        await emit_business_log(
+            logger,
+            event="settings.web_search.connect.success",
+            message="WebSearch connection test completed",
+            category="ai_tool",
+            service="api",
+            module=__name__,
+            user_id=current_user.id,
+            context={"provider": runtime_config.default_provider, "result_count": len(results)},
+        )
         return {
             "success": True,
             "connected": True,
@@ -1472,18 +1897,51 @@ async def connect_web_search_integration(
             "results": [item.model_dump(mode="json") for item in results[:3]],
         }
     except WebSearchConfigurationError as exc:
+        await emit_business_log(
+            logger,
+            event="settings.web_search.connect.failed",
+            message="WebSearch connection test failed because configuration is incomplete",
+            category="ai_tool",
+            level="warning",
+            service="api",
+            module=__name__,
+            user_id=current_user.id,
+            context=exception_context(exc, {"provider": runtime_config.default_provider}),
+        )
         return {
             "success": False,
             "connected": False,
             "message": str(exc),
         }
     except WebSearchError as exc:
+        await emit_business_log(
+            logger,
+            event="settings.web_search.connect.failed",
+            message="WebSearch connection test failed",
+            category="ai_tool",
+            level="error",
+            service="api",
+            module=__name__,
+            user_id=current_user.id,
+            context=exception_context(exc, {"provider": runtime_config.default_provider}),
+        )
         return {
             "success": False,
             "connected": False,
             "message": str(exc),
         }
     except Exception as exc:
+        await emit_business_log(
+            logger,
+            event="settings.web_search.connect.failed",
+            message="WebSearch connection test failed",
+            category="ai_tool",
+            level="error",
+            service="api",
+            module=__name__,
+            user_id=current_user.id,
+            context=exception_context(exc, {"provider": runtime_config.default_provider}),
+        )
         return {
             "success": False,
             "connected": False,
@@ -1511,17 +1969,60 @@ async def generate_provider_credential_guide(
     ai_client: AIProviderClient = Depends(get_ai_provider_client),
 ):
     try:
+        await emit_business_log(
+            logger,
+            event="settings.credential_guide.generate.start",
+            message="Credential guide generation started",
+            category="ai",
+            service="api",
+            module=__name__,
+            user_id=current_user.id,
+            context={"provider": provider},
+        )
         web_search_client = await get_web_search_client(db)
-        return {
-            "guide": await generate_credential_guide(
+        guide = await generate_credential_guide(
                 db,
                 provider,
                 ai_client,
                 web_search_client,
             )
-        }
+        await emit_business_log(
+            logger,
+            event="settings.credential_guide.generate.success",
+            message="Credential guide generation completed",
+            category="ai",
+            service="api",
+            module=__name__,
+            user_id=current_user.id,
+            context={"provider": provider},
+        )
+        return {"guide": guide}
     except ValueError as exc:
+        await emit_business_log(
+            logger,
+            event="settings.credential_guide.generate.failed",
+            message="Credential guide generation failed",
+            category="ai",
+            level="warning",
+            service="api",
+            module=__name__,
+            user_id=current_user.id,
+            context=exception_context(exc, {"provider": provider}),
+        )
         raise HTTPException(status_code=404, detail=str(exc)) from exc
+    except Exception as exc:
+        await emit_business_log(
+            logger,
+            event="settings.credential_guide.generate.failed",
+            message="Credential guide generation failed",
+            category="ai",
+            level="error",
+            service="api",
+            module=__name__,
+            user_id=current_user.id,
+            context=exception_context(exc, {"provider": provider}),
+        )
+        raise
 
 
 @router.post("/credential-guides/{provider}/reset")
@@ -1547,9 +2048,17 @@ async def get_ai_provider_presets(
 async def refresh_ai_provider_preset(
     provider: str,
     current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
 ):
     try:
-        return {"data": await refresh_llm_provider_preset(provider)}
+        provider_id = _normalize_provider_id(provider)
+        api_key = None
+        if provider_id == "opencode-go":
+            current_payload = await get_setting_payload(db, "external_integrations")
+            ai_payload = _normalize_ai_provider_payload(current_payload.get("ai_provider") or {})
+            provider_config = ai_payload["providers"].get(provider_id) or _provider_defaults(provider_id)
+            api_key, _api_key_source = _resolve_provider_api_key(provider_id, provider_config)
+        return {"data": await refresh_llm_provider_preset(provider_id, api_key=api_key)}
     except ValueError as exc:
         raise HTTPException(status_code=404, detail=str(exc)) from exc
     except Exception as exc:

backend/app/api/v1/system_control.py

@@ -1,6 +1,7 @@
 from __future__ import annotations
 
 import os
+import json
 import subprocess
 import sys
 
@@ -8,9 +9,13 @@ from datetime import datetime
 
 from fastapi import APIRouter, Depends, HTTPException, Query, Request, status
 from pydantic import BaseModel
+from sqlalchemy import select
+from sqlalchemy.ext.asyncio import AsyncSession
 
 from app.core.config import ROOT_DIR
 from app.core.security import get_current_user
+from app.db.session import get_db
+from app.models.system_log import AuditLog, SystemLog
 from app.models.user import User
 from app.services.persistent_logs import record_audit_log, record_system_log
 from app.services.system_control import (
@@ -40,6 +45,42 @@ from app.services.earth_layer_cache import earth_layer_cache
 router = APIRouter()
 
 
+def _compact_log_context(context: dict | None) -> str:
+    if not context:
+        return ""
+    allowed = {
+        key: value
+        for key, value in (context or {}).items()
+        if key
+        in {
+            "status",
+            "duration_ms",
+            "provider",
+            "model",
+            "result_provider",
+            "result_model",
+            "collector_name",
+            "datasource_id",
+            "task_id",
+            "snapshot_id",
+            "raw_count",
+            "transformed_count",
+            "saved_count",
+            "created",
+            "updated",
+            "unchanged",
+            "deleted",
+            "result_count",
+            "status_code",
+            "error_type",
+            "error",
+        }
+    }
+    if not allowed:
+        return ""
+    return json.dumps(allowed, ensure_ascii=False, sort_keys=True)
+
+
 class RestartTaskCreate(BaseModel):
     action: str
 
@@ -310,7 +351,123 @@ async def get_system_log_sources(
     current_user: User = Depends(get_current_user),
 ):
     ensure_super_admin(current_user)
-    return {"items": list_log_sources()}
+    return {
+        "items": [
+            *list_log_sources(),
+            {
+                "source_id": "system-db",
+                "name": "系统事件",
+                "kind": "database",
+                "location": "table://system_logs",
+                "description": "后端持久化系统事件、AI 和采集器操作日志。",
+                "category": "database",
+                "status": "ok",
+            },
+            {
+                "source_id": "audit-db",
+                "name": "审计事件",
+                "kind": "database",
+                "location": "table://audit_logs",
+                "description": "管理员敏感操作和密钥 reveal 审计记录。",
+                "category": "audit",
+                "status": "ok",
+            },
+        ]
+    }
+
+
+async def read_database_log_snapshot(
+    source_id: str,
+    *,
+    limit: int,
+    level: str,
+    levels: str | None,
+    start_date: str | None,
+    end_date: str | None,
+    search: str | None,
+    db: AsyncSession,
+) -> dict | None:
+    selected_levels = set(normalize_log_level(item) for item in (levels or level).split(",") if item.strip())
+    selected_levels.discard("all")
+    search_query = (search or "").strip().lower()
+    lines: list[str] = []
+
+    if source_id == "system-db":
+        query = select(SystemLog).order_by(SystemLog.occurred_at.desc().nullslast(), SystemLog.id.desc()).limit(limit * 5)
+        result = await db.execute(query)
+        records = result.scalars().all()
+        for record in records:
+            record_level = normalize_log_level(record.level)
+            if selected_levels and record_level not in selected_levels:
+                continue
+            occurred_at = record.occurred_at.date().isoformat() if record.occurred_at else ""
+            if start_date and occurred_at and occurred_at < start_date:
+                continue
+            if end_date and occurred_at and occurred_at > end_date:
+                continue
+            line = " ".join(
+                part
+                for part in [
+                    record.occurred_at.isoformat() if record.occurred_at else "",
+                    record_level.upper(),
+                    record.source,
+                    record.category or "",
+                    record.event or "",
+                    f"request_id={record.request_id}" if record.request_id else "",
+                    record.message,
+                    _compact_log_context(record.context),
+                ]
+                if part
+            )
+            if search_query and search_query not in line.lower():
+                continue
+            lines.append(line)
+    elif source_id == "audit-db":
+        query = select(AuditLog).order_by(AuditLog.occurred_at.desc().nullslast(), AuditLog.id.desc()).limit(limit * 5)
+        result = await db.execute(query)
+        records = result.scalars().all()
+        for record in records:
+            occurred_at = record.occurred_at.date().isoformat() if record.occurred_at else ""
+            if start_date and occurred_at and occurred_at < start_date:
+                continue
+            if end_date and occurred_at and occurred_at > end_date:
+                continue
+            line = " ".join(
+                part
+                for part in [
+                    record.occurred_at.isoformat() if record.occurred_at else "",
+                    "INFO",
+                    record.action,
+                    record.target_type or "",
+                    record.target_id or "",
+                    record.result or "",
+                ]
+                if part
+            )
+            if search_query and search_query not in line.lower():
+                continue
+            lines.append(line)
+    else:
+        return None
+
+    lines = list(reversed(lines[:limit]))
+    return {
+        "source_id": source_id,
+        "name": "系统事件" if source_id == "system-db" else "审计事件",
+        "kind": "database",
+        "location": "table://system_logs" if source_id == "system-db" else "table://audit_logs",
+        "description": "数据库持久化日志",
+        "category": "database" if source_id == "system-db" else "audit",
+        "status": "ok" if lines else "empty",
+        "level": level,
+        "selected_levels": sorted(selected_levels),
+        "search_query": search or "",
+        "available_levels": ["all", "error", "warning", "info", "debug"],
+        "daily_markers": [],
+        "line_limit": limit,
+        "line_count": len(lines),
+        "lines": lines,
+    }
 
 
 @router.get("/logs/{source_id}", response_model=SystemLogSnapshotResponse)
@@ -323,6 +480,7 @@ async def get_system_log_snapshot(
     end_date: str | None = Query(None, description="Filter logs until this date (YYYY-MM-DD)"),
     search: str | None = Query(None, description="Case-insensitive substring search"),
     current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
 ):
     ensure_super_admin(current_user)
 
@@ -345,15 +503,26 @@ async def get_system_log_snapshot(
     if normalized_start_date and normalized_end_date and normalized_start_date > normalized_end_date:
         raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail="start_date must be earlier than or equal to end_date")
 
-    snapshot = read_log_snapshot(
+    snapshot = await read_database_log_snapshot(
         source_id,
-        limit,
+        limit=limit,
         level=level,
         levels=levels,
         start_date=normalized_start_date,
         end_date=normalized_end_date,
         search=search,
+        db=db,
     )
+    if snapshot is None:
+        snapshot = read_log_snapshot(
+            source_id,
+            limit,
+            level=level,
+            levels=levels,
+            start_date=normalized_start_date,
+            end_date=normalized_end_date,
+            search=search,
+        )
     if snapshot is None:
         raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Log source not found")
     return snapshot

backend/app/api/v1/visualization.py

@@ -38,7 +38,7 @@ from app.services.compute_center_locations import (
     upsert_compute_center_location,
 )
 from app.services.ai_client import get_ai_provider_client
-from app.api.v1.settings import get_web_search_client
+from app.api.v1.settings import get_runtime_web_search_config, get_web_search_client
 from app.services.collectors.bgp_common import RIPE_RIS_COLLECTOR_COORDS
 from app.services.location.llm_fallback import (
     collect_llm_location_fallback_candidate,
@@ -105,8 +105,8 @@ LANDING_POINT_CACHE_POLICY = EarthLayerCachePolicy(
 SATELLITE_CACHE_POLICY = EarthLayerCachePolicy(
     SATELLITE_CACHE_FRESH_SECONDS,
     SATELLITE_CACHE_STALE_SECONDS,
-    max_features=8000,
-    max_bytes=10 * BYTES_PER_MIB,
+    max_features=25000,
+    max_bytes=32 * BYTES_PER_MIB,
 )
 COMPUTE_CENTER_CACHE_POLICY = EarthLayerCachePolicy(
     COMPUTE_CENTER_CACHE_FRESH_SECONDS,
@@ -1563,15 +1563,7 @@ async def _build_cables_geojson(db: AsyncSession) -> dict[str, Any]:
     try:
         records = await _load_current_collected_data(db, "arcgis_cables")
 
-        if not records:
-            raise HTTPException(
-                status_code=404,
-                detail="No cable data found. Please run the arcgis_cables collector first.",
-            )
-
         return convert_cable_to_geojson(records)
-    except HTTPException:
-        raise
     except Exception as e:
         logger.exception_event(
             "Failed to build cables GeoJSON response",
@@ -1625,16 +1617,8 @@ async def _build_landing_points_geojson(db: AsyncSession) -> dict[str, Any]:
             relation_records,
             cable_records,
         )
-        
-        if not records:
-            raise HTTPException(
-                status_code=404,
-                detail="No landing point data found. Please run the arcgis_landing_points collector first.",
-            )
-        
+
         return convert_landing_point_to_geojson(records, city_to_cable_ids_map, cable_id_to_name_map)
-    except HTTPException:
-        raise
     except Exception as e:
         logger.exception_event(
             "Failed to build landing points GeoJSON response",
@@ -2071,6 +2055,44 @@ class SaveComputeCenterLocationRequest(BaseModel):
     model_config = {"populate_by_name": True}
 
 
+async def _compute_center_location_web_search_capability(db: AsyncSession) -> Dict[str, Any]:
+    try:
+        config = await get_runtime_web_search_config(db)
+    except Exception as exc:
+        return {
+            "enabled": False,
+            "provider": None,
+            "reason": f"WebSearch 配置读取失败：{exc}",
+        }
+    provider_config = config.active_provider_config
+    has_api_key = bool((provider_config.api_key or "").strip())
+    if not config.enabled:
+        return {
+            "enabled": False,
+            "provider": config.default_provider,
+            "reason": "WebSearch 未开启，无法进行事实核查定位。",
+        }
+    if not has_api_key:
+        return {
+            "enabled": False,
+            "provider": config.default_provider,
+            "reason": f"WebSearch Provider {config.default_provider} 未配置 API Key。",
+        }
+    return {
+        "enabled": True,
+        "provider": config.default_provider,
+        "reason": "",
+    }
+
+
+@router.get("/compute-centers/location-capability")
+async def get_compute_center_location_capability(
+    db: AsyncSession = Depends(get_db),
+):
+    """Return whether fact-checked compute-center location collection can run."""
+    return await _compute_center_location_web_search_capability(db)
+
+
 @router.post("/compute-centers/{source_id}/collect-location")
 async def collect_compute_center_location(
     source_id: str,
@@ -2089,6 +2111,9 @@ async def collect_compute_center_location(
     """
     if not source_id or not source_id.strip():
         raise HTTPException(status_code=400, detail="source_id is required")
+    capability = await _compute_center_location_web_search_capability(db)
+    if not capability.get("enabled"):
+        raise HTTPException(status_code=409, detail=capability)
 
     record = await _load_compute_center_record(db, source_id)
     name = payload.name or (record.name if record else None)
@@ -2159,6 +2184,24 @@ async def collect_compute_center_location(
             llm_failure_reason = llm_result.failure_reason
 
     if not candidates:
+        logger.warning_event(
+            "Compute center location collection returned no candidates",
+            event="visualization.compute_center.location_collect.completed",
+            context={
+                "source_id": source_id,
+                "record_id": record_id,
+                "name": name,
+                "success": False,
+                "llm_failure_reason": llm_failure_reason,
+                "attempted_queries": list(attempted_queries),
+                "context": {
+                    "operator": operator,
+                    "site": site,
+                    "city": city,
+                    "country": country,
+                },
+            },
+        )
         return {
             "source_id": source_id,
             "record_id": record_id,
@@ -2180,13 +2223,34 @@ async def collect_compute_center_location(
             },
         }
 
+    best_candidate = candidates[0].to_dict()
+    logger.info_event(
+        "Compute center location collection returned candidates",
+        event="visualization.compute_center.location_collect.completed",
+        context={
+            "source_id": source_id,
+            "record_id": record_id,
+            "name": name,
+            "success": True,
+            "candidate_count": len(candidates),
+            "best_candidate": best_candidate,
+            "llm_failure_reason": llm_failure_reason,
+            "attempted_queries": list(attempted_queries),
+            "context": {
+                "operator": operator,
+                "site": site,
+                "city": city,
+                "country": country,
+            },
+        },
+    )
     return {
         "source_id": source_id,
         "record_id": record_id,
         "name": name,
         "success": True,
         "candidates": [candidate.to_dict() for candidate in candidates],
-        "best_candidate": candidates[0].to_dict(),
+        "best_candidate": best_candidate,
         "attempted_queries": list(attempted_queries),
         "context": {
             "name": name,
@@ -2657,6 +2721,8 @@ async def _build_bgp_collectors_geojson(db: AsyncSession) -> dict[str, Any]:
         db,
         source_filter=("ris_live_bgp", "bgpstream_bgp"),
     )
+    if not any(int(item.get("observation_count") or 0) > 0 for item in coverage):
+        return {"type": "FeatureCollection", "features": [], "count": 0}
     coverage_by_collector = {
         item["collector"]: item
         for item in coverage

backend/app/api/v1/websocket.py

@@ -14,6 +14,7 @@ from app.core.websocket.manager import manager
 
 logger = get_logger(__name__, service="api")
 router = APIRouter()
+EARTH_UPDATES_CHANNEL = "earth_updates"
 
 
 async def authenticate_token(token: str) -> Optional[dict]:
@@ -58,7 +59,7 @@ async def websocket_endpoint(
 
     is_anonymous = payload is None
     user_id = str(payload.get("sub")) if payload else f"anonymous:{id(websocket)}"
-    supported_channels = ["vessels", "earth_news"] if is_anonymous else [
+    supported_channels = ["vessels", "earth_news", EARTH_UPDATES_CHANNEL] if is_anonymous else [
         "gpu_clusters",
         "submarine_cables",
         "ixp_nodes",
@@ -67,6 +68,7 @@ async def websocket_endpoint(
         "datasource_tasks",
         "vessels",
         "earth_news",
+        EARTH_UPDATES_CHANNEL,
     ]
     await manager.connect(websocket, user_id)
 

backend/app/core/websocket/broadcaster.py

@@ -8,6 +8,8 @@ from app.core.time import to_iso8601_utc
 from app.core.websocket.manager import manager
 
 
+EARTH_UPDATES_CHANNEL = "earth_updates"
+
 
 class DataBroadcaster:
     """Periodically broadcasts data to connected WebSocket clients"""
@@ -83,6 +85,10 @@ class DataBroadcaster:
             channel=channel,
         )
 
+    async def broadcast_earth_update(self, data: Dict[str, Any]):
+        """Broadcast Earth visualization refresh hints to connected clients."""
+        await self.broadcast_custom(EARTH_UPDATES_CHANNEL, data)
+
     def enqueue_vessel_update(self, data: Dict[str, Any]):
         vessels = data.get("vessels") if isinstance(data, dict) else None
         if not isinstance(vessels, list):
@@ -144,7 +150,7 @@ class DataBroadcaster:
                 "timestamp": to_iso8601_utc(datetime.now(UTC)),
                 "payload": data,
             },
-            channel="all",
+            channel="datasource_tasks",
         )
 
     def start(self):

backend/app/db/session.py

@@ -150,7 +150,7 @@ DEFAULT_LOGIN_USERS = (
     {
         "username": "linkong",
         "email": "linkong@planet.local",
-        "password": "12345678",
+        "password": "LK12345678",
         "role": "super_admin",
     },
 )
@@ -203,6 +203,7 @@ async def init_db():
     import app.models.vessel_enrichment  # noqa: F401
     import app.models.datasource_mapping  # noqa: F401
     import app.models.earth_news  # noqa: F401
+    import app.models.earth_interactable  # noqa: F401
 
     logger.warning_event(
         "Database pool settings active",
@@ -258,6 +259,406 @@ async def init_db():
                 """
             )
         )
+        await conn.execute(
+            text(
+                """
+                CREATE TABLE IF NOT EXISTS earth_data_change_events (
+                    id BIGSERIAL PRIMARY KEY,
+                    table_name VARCHAR(128) NOT NULL,
+                    operation VARCHAR(16) NOT NULL,
+                    source VARCHAR(128),
+                    entity_key VARCHAR(255),
+                    payload JSONB NOT NULL,
+                    occurred_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
+                    consumed_at TIMESTAMPTZ
+                )
+                """
+            )
+        )
+        await conn.execute(
+            text(
+                """
+                CREATE INDEX IF NOT EXISTS idx_earth_data_change_events_unconsumed
+                ON earth_data_change_events (consumed_at, id)
+                WHERE consumed_at IS NULL
+                """
+            )
+        )
+        await conn.execute(
+            text(
+                """
+                CREATE OR REPLACE FUNCTION planet_emit_earth_data_changed_statement(
+                    change_table TEXT,
+                    change_operation TEXT,
+                    change_source TEXT,
+                    source_record_count INTEGER,
+                    source_entity_keys TEXT[]
+                )
+                RETURNS VOID AS $$
+                DECLARE
+                    change_event_id BIGINT;
+                    change_payload JSONB;
+                BEGIN
+                    change_payload := jsonb_build_object(
+                        'event', 'earth.layer.changed',
+                        'table', change_table,
+                        'operation', change_operation,
+                        'source', change_source,
+                        'entity_key', NULL,
+                        'entity_keys', COALESCE(to_jsonb(source_entity_keys), '[]'::jsonb),
+                        'records_processed', COALESCE(source_record_count, 0),
+                        'occurred_at', NOW()
+                    );
+
+                    INSERT INTO earth_data_change_events (
+                        table_name,
+                        operation,
+                        source,
+                        entity_key,
+                        payload,
+                        occurred_at
+                    ) VALUES (
+                        change_table,
+                        change_operation,
+                        change_source,
+                        NULL,
+                        change_payload,
+                        NOW()
+                    )
+                    RETURNING id INTO change_event_id;
+
+                    change_payload := change_payload || jsonb_build_object(
+                        'event_id', change_event_id
+                    );
+
+                    UPDATE earth_data_change_events
+                    SET payload = change_payload
+                    WHERE id = change_event_id;
+
+                    PERFORM pg_notify(
+                        'planet_earth_data_changes',
+                        change_payload::text
+                    );
+                END;
+                $$ LANGUAGE plpgsql;
+                """
+            )
+        )
+        await conn.execute(
+            text(
+                """
+                CREATE OR REPLACE FUNCTION planet_emit_collected_data_changed_statement(
+                    change_operation TEXT,
+                    change_source TEXT,
+                    source_record_count INTEGER,
+                    source_entity_keys TEXT[]
+                )
+                RETURNS VOID AS $$
+                BEGIN
+                    PERFORM planet_emit_earth_data_changed_statement(
+                        'collected_data',
+                        change_operation,
+                        change_source,
+                        source_record_count,
+                        source_entity_keys
+                    );
+                END;
+                $$ LANGUAGE plpgsql;
+                """
+            )
+        )
+        await conn.execute(
+            text(
+                """
+                CREATE OR REPLACE FUNCTION planet_notify_earth_table_changed_statement()
+                RETURNS trigger AS $$
+                DECLARE
+                    change_source TEXT;
+                    source_record_count INTEGER;
+                    source_entity_keys TEXT[];
+                BEGIN
+                    IF TG_OP = 'INSERT' THEN
+                        FOR change_source IN
+                            SELECT DISTINCT COALESCE(NULLIF(row_data->>'source', ''), TG_TABLE_NAME)
+                            FROM (SELECT to_jsonb(t) AS row_data FROM new_rows AS t) changed_rows
+                        LOOP
+                            SELECT
+                                COUNT(*),
+                                ARRAY(
+                                    SELECT DISTINCT COALESCE(
+                                        NULLIF(row_data->>'entity_key', ''),
+                                        NULLIF(row_data->>'source_id', ''),
+                                        NULLIF(row_data->>'incident_key', ''),
+                                        NULLIF(row_data->>'id', ''),
+                                        NULLIF(row_data->>'mmsi', '')
+                                    )
+                                    FROM (SELECT to_jsonb(t) AS row_data FROM new_rows AS t) rows_for_keys
+                                    WHERE COALESCE(NULLIF(row_data->>'source', ''), TG_TABLE_NAME) = change_source
+                                    LIMIT 20
+                                )
+                            INTO source_record_count, source_entity_keys
+                            FROM (SELECT to_jsonb(t) AS row_data FROM new_rows AS t) rows_for_count
+                            WHERE COALESCE(NULLIF(row_data->>'source', ''), TG_TABLE_NAME) = change_source;
+
+                            PERFORM planet_emit_earth_data_changed_statement(
+                                TG_TABLE_NAME,
+                                TG_OP,
+                                change_source,
+                                source_record_count,
+                                source_entity_keys
+                            );
+                        END LOOP;
+                    ELSIF TG_OP = 'DELETE' THEN
+                        FOR change_source IN
+                            SELECT DISTINCT COALESCE(NULLIF(row_data->>'source', ''), TG_TABLE_NAME)
+                            FROM (SELECT to_jsonb(t) AS row_data FROM old_rows AS t) changed_rows
+                        LOOP
+                            SELECT
+                                COUNT(*),
+                                ARRAY(
+                                    SELECT DISTINCT COALESCE(
+                                        NULLIF(row_data->>'entity_key', ''),
+                                        NULLIF(row_data->>'source_id', ''),
+                                        NULLIF(row_data->>'incident_key', ''),
+                                        NULLIF(row_data->>'id', ''),
+                                        NULLIF(row_data->>'mmsi', '')
+                                    )
+                                    FROM (SELECT to_jsonb(t) AS row_data FROM old_rows AS t) rows_for_keys
+                                    WHERE COALESCE(NULLIF(row_data->>'source', ''), TG_TABLE_NAME) = change_source
+                                    LIMIT 20
+                                )
+                            INTO source_record_count, source_entity_keys
+                            FROM (SELECT to_jsonb(t) AS row_data FROM old_rows AS t) rows_for_count
+                            WHERE COALESCE(NULLIF(row_data->>'source', ''), TG_TABLE_NAME) = change_source;
+
+                            PERFORM planet_emit_earth_data_changed_statement(
+                                TG_TABLE_NAME,
+                                TG_OP,
+                                change_source,
+                                source_record_count,
+                                source_entity_keys
+                            );
+                        END LOOP;
+                    ELSIF TG_OP = 'UPDATE' THEN
+                        FOR change_source IN
+                            SELECT DISTINCT COALESCE(NULLIF(row_data->>'source', ''), TG_TABLE_NAME)
+                            FROM (
+                                SELECT to_jsonb(t) AS row_data FROM new_rows AS t
+                                UNION ALL
+                                SELECT to_jsonb(t) AS row_data FROM old_rows AS t
+                            ) changed_rows
+                        LOOP
+                            SELECT
+                                COUNT(*),
+                                ARRAY(
+                                    SELECT DISTINCT COALESCE(
+                                        NULLIF(row_data->>'entity_key', ''),
+                                        NULLIF(row_data->>'source_id', ''),
+                                        NULLIF(row_data->>'incident_key', ''),
+                                        NULLIF(row_data->>'id', ''),
+                                        NULLIF(row_data->>'mmsi', '')
+                                    )
+                                    FROM (
+                                        SELECT to_jsonb(t) AS row_data FROM new_rows AS t
+                                        UNION ALL
+                                        SELECT to_jsonb(t) AS row_data FROM old_rows AS t
+                                    ) rows_for_keys
+                                    WHERE COALESCE(NULLIF(row_data->>'source', ''), TG_TABLE_NAME) = change_source
+                                    LIMIT 20
+                                )
+                            INTO source_record_count, source_entity_keys
+                            FROM (
+                                SELECT to_jsonb(t) AS row_data FROM new_rows AS t
+                                UNION ALL
+                                SELECT to_jsonb(t) AS row_data FROM old_rows AS t
+                            ) rows_for_count
+                            WHERE COALESCE(NULLIF(row_data->>'source', ''), TG_TABLE_NAME) = change_source;
+
+                            PERFORM planet_emit_earth_data_changed_statement(
+                                TG_TABLE_NAME,
+                                TG_OP,
+                                change_source,
+                                source_record_count,
+                                source_entity_keys
+                            );
+                        END LOOP;
+                    END IF;
+
+                    RETURN NULL;
+                END;
+                $$ LANGUAGE plpgsql;
+                """
+            )
+        )
+        await conn.execute(
+            text(
+                """
+                CREATE OR REPLACE FUNCTION planet_notify_collected_data_changed_statement()
+                RETURNS trigger AS $$
+                DECLARE
+                    change_source TEXT;
+                    source_record_count INTEGER;
+                    source_entity_keys TEXT[];
+                BEGIN
+                    IF TG_OP = 'INSERT' THEN
+                        FOR change_source IN
+                            SELECT DISTINCT source FROM new_rows WHERE source IS NOT NULL
+                        LOOP
+                            SELECT
+                                COUNT(*),
+                                ARRAY(
+                                    SELECT DISTINCT COALESCE(entity_key, source_id, id::text)
+                                    FROM new_rows
+                                    WHERE source = change_source
+                                    LIMIT 20
+                                )
+                            INTO source_record_count, source_entity_keys
+                            FROM new_rows
+                            WHERE source = change_source;
+
+                            PERFORM planet_emit_collected_data_changed_statement(
+                                TG_OP,
+                                change_source,
+                                source_record_count,
+                                source_entity_keys
+                            );
+                        END LOOP;
+                    ELSIF TG_OP = 'DELETE' THEN
+                        FOR change_source IN
+                            SELECT DISTINCT source FROM old_rows WHERE source IS NOT NULL
+                        LOOP
+                            SELECT
+                                COUNT(*),
+                                ARRAY(
+                                    SELECT DISTINCT COALESCE(entity_key, source_id, id::text)
+                                    FROM old_rows
+                                    WHERE source = change_source
+                                    LIMIT 20
+                                )
+                            INTO source_record_count, source_entity_keys
+                            FROM old_rows
+                            WHERE source = change_source;
+
+                            PERFORM planet_emit_collected_data_changed_statement(
+                                TG_OP,
+                                change_source,
+                                source_record_count,
+                                source_entity_keys
+                            );
+                        END LOOP;
+                    ELSIF TG_OP = 'UPDATE' THEN
+                        FOR change_source IN
+                            SELECT DISTINCT source FROM (
+                                SELECT source FROM new_rows
+                                UNION
+                                SELECT source FROM old_rows
+                            ) changed_sources
+                            WHERE source IS NOT NULL
+                        LOOP
+                            SELECT
+                                COUNT(*),
+                                ARRAY(
+                                    SELECT DISTINCT COALESCE(entity_key, source_id, id::text)
+                                    FROM (
+                                        SELECT id, source_id, entity_key, source FROM new_rows
+                                        UNION ALL
+                                        SELECT id, source_id, entity_key, source FROM old_rows
+                                    ) changed_rows
+                                    WHERE source = change_source
+                                    LIMIT 20
+                                )
+                            INTO source_record_count, source_entity_keys
+                            FROM (
+                                SELECT id, source_id, entity_key, source FROM new_rows
+                                UNION ALL
+                                SELECT id, source_id, entity_key, source FROM old_rows
+                            ) changed_rows
+                            WHERE source = change_source;
+
+                            PERFORM planet_emit_collected_data_changed_statement(
+                                TG_OP,
+                                change_source,
+                                source_record_count,
+                                source_entity_keys
+                            );
+                        END LOOP;
+                    END IF;
+
+                    RETURN NULL;
+                END;
+                $$ LANGUAGE plpgsql;
+                """
+            )
+        )
+        for statement in (
+            "DROP TRIGGER IF EXISTS tr_planet_collected_data_changed ON collected_data",
+            "DROP TRIGGER IF EXISTS tr_planet_collected_data_changed_insert ON collected_data",
+            "DROP TRIGGER IF EXISTS tr_planet_collected_data_changed_update ON collected_data",
+            "DROP TRIGGER IF EXISTS tr_planet_collected_data_changed_delete ON collected_data",
+            "DROP FUNCTION IF EXISTS planet_notify_collected_data_changed()",
+            """
+            CREATE TRIGGER tr_planet_collected_data_changed_insert
+            AFTER INSERT ON collected_data
+            REFERENCING NEW TABLE AS new_rows
+            FOR EACH STATEMENT
+            EXECUTE FUNCTION planet_notify_collected_data_changed_statement()
+            """,
+            """
+            CREATE TRIGGER tr_planet_collected_data_changed_update
+            AFTER UPDATE ON collected_data
+            REFERENCING OLD TABLE AS old_rows NEW TABLE AS new_rows
+            FOR EACH STATEMENT
+            EXECUTE FUNCTION planet_notify_collected_data_changed_statement()
+            """,
+            """
+            CREATE TRIGGER tr_planet_collected_data_changed_delete
+            AFTER DELETE ON collected_data
+            REFERENCING OLD TABLE AS old_rows
+            FOR EACH STATEMENT
+            EXECUTE FUNCTION planet_notify_collected_data_changed_statement()
+            """,
+        ):
+            await conn.execute(text(statement))
+        for table_name in (
+            "bgp_observations",
+            "bgp_anomalies",
+            "bgp_incidents",
+            "bgp_collector_locations",
+            "vessel_static",
+            "vessel_position",
+            "ais_raw_observations",
+            "ais_source_health",
+            "compute_center_locations",
+            "earth_interactables",
+            "earth_news_items",
+        ):
+            for statement in (
+                f"DROP TRIGGER IF EXISTS tr_planet_{table_name}_changed_insert ON {table_name}",
+                f"DROP TRIGGER IF EXISTS tr_planet_{table_name}_changed_update ON {table_name}",
+                f"DROP TRIGGER IF EXISTS tr_planet_{table_name}_changed_delete ON {table_name}",
+                f"""
+                CREATE TRIGGER tr_planet_{table_name}_changed_insert
+                AFTER INSERT ON {table_name}
+                REFERENCING NEW TABLE AS new_rows
+                FOR EACH STATEMENT
+                EXECUTE FUNCTION planet_notify_earth_table_changed_statement()
+                """,
+                f"""
+                CREATE TRIGGER tr_planet_{table_name}_changed_update
+                AFTER UPDATE ON {table_name}
+                REFERENCING OLD TABLE AS old_rows NEW TABLE AS new_rows
+                FOR EACH STATEMENT
+                EXECUTE FUNCTION planet_notify_earth_table_changed_statement()
+                """,
+                f"""
+                CREATE TRIGGER tr_planet_{table_name}_changed_delete
+                AFTER DELETE ON {table_name}
+                REFERENCING OLD TABLE AS old_rows
+                FOR EACH STATEMENT
+                EXECUTE FUNCTION planet_notify_earth_table_changed_statement()
+                """,
+            ):
+                await conn.execute(text(statement))
         await conn.execute(
             text(
                 """
@@ -267,7 +668,16 @@ async def init_db():
                 ADD COLUMN IF NOT EXISTS phase_message VARCHAR(255),
                 ADD COLUMN IF NOT EXISTS phase_current BIGINT,
                 ADD COLUMN IF NOT EXISTS phase_total BIGINT,
-                ADD COLUMN IF NOT EXISTS phase_unit VARCHAR(30)
+                ADD COLUMN IF NOT EXISTS phase_unit VARCHAR(30),
+                ADD COLUMN IF NOT EXISTS source VARCHAR(100),
+                ADD COLUMN IF NOT EXISTS task_type VARCHAR(30) NOT NULL DEFAULT 'collect',
+                ADD COLUMN IF NOT EXISTS payload JSONB NOT NULL DEFAULT '{}'::jsonb,
+                ADD COLUMN IF NOT EXISTS rollback_policy VARCHAR(40) NOT NULL DEFAULT 'keep_committed_batches',
+                ADD COLUMN IF NOT EXISTS dedupe_key VARCHAR(180),
+                ADD COLUMN IF NOT EXISTS worker_id VARCHAR(120),
+                ADD COLUMN IF NOT EXISTS locked_at TIMESTAMPTZ,
+                ADD COLUMN IF NOT EXISTS requested_cancel_at TIMESTAMPTZ,
+                ADD COLUMN IF NOT EXISTS cancel_reason TEXT
                 """
             )
         )
@@ -283,6 +693,17 @@ async def init_db():
                 """
             )
         )
+        await conn.execute(
+            text(
+                """
+                ALTER TABLE earth_interactables
+                ADD COLUMN IF NOT EXISTS altitude DOUBLE PRECISION,
+                ADD COLUMN IF NOT EXISTS revision INTEGER NOT NULL DEFAULT 1,
+                ADD COLUMN IF NOT EXISTS is_deleted BOOLEAN NOT NULL DEFAULT FALSE,
+                ADD COLUMN IF NOT EXISTS deleted_at TIMESTAMPTZ
+                """
+            )
+        )
         await conn.execute(
             text(
                 """
@@ -307,6 +728,48 @@ async def init_db():
                 """
             )
         )
+        await conn.execute(
+            text(
+                """
+                CREATE INDEX IF NOT EXISTS idx_earth_interactables_layer_deleted
+                ON earth_interactables (layer, is_deleted)
+                """
+            )
+        )
+        await conn.execute(
+            text(
+                """
+                CREATE INDEX IF NOT EXISTS idx_earth_interactables_updated_at
+                ON earth_interactables (updated_at)
+                """
+            )
+        )
+        await conn.execute(
+            text(
+                """
+                CREATE INDEX IF NOT EXISTS idx_collection_tasks_source_status
+                ON collection_tasks (source, status)
+                """
+            )
+        )
+        await conn.execute(
+            text(
+                """
+                CREATE INDEX IF NOT EXISTS idx_collection_tasks_queue
+                ON collection_tasks (status, created_at, id)
+                WHERE status = 'queued'
+                """
+            )
+        )
+        await conn.execute(
+            text(
+                """
+                CREATE INDEX IF NOT EXISTS idx_collection_tasks_dedupe
+                ON collection_tasks (dedupe_key)
+                WHERE dedupe_key IS NOT NULL
+                """
+            )
+        )
         await conn.execute(
             text(
                 """

backend/app/main.py

@@ -24,6 +24,11 @@ from app.services.earth_news_worker import (
     start_earth_news_target_worker,
     stop_earth_news_target_worker,
 )
+from app.services.earth_db_change_listener import (
+    start_earth_db_change_listener,
+    stop_earth_db_change_listener,
+)
+from app.services.data_jobs import start_data_job_worker, stop_data_job_worker
 
 
 configure_logging()
@@ -59,9 +64,13 @@ async def lifespan(app: FastAPI):
     start_scheduler()
     await sync_scheduler_with_datasources()
     broadcaster.start()
+    start_data_job_worker()
+    start_earth_db_change_listener()
     start_earth_news_target_worker()
     yield
     await stop_earth_news_target_worker()
+    await stop_earth_db_change_listener()
+    await stop_data_job_worker()
     broadcaster.stop()
     stop_scheduler()
 

backend/app/models/__init__.py

@@ -17,6 +17,7 @@ from app.models.system_log import SystemLog, AuditLog
 from app.models.vessel import AISConflictRecord, AISRawObservation, AISSourceHealth, VesselPosition, VesselStatic
 from app.models.datasource_mapping import DataSourceMappingTemplate
 from app.models.earth_news import EarthNewsItem
+from app.models.earth_interactable import EarthInteractable
 
 __all__ = [
     "User",
@@ -45,4 +46,5 @@ __all__ = [
     "AISSourceHealth",
     "DataSourceMappingTemplate",
     "EarthNewsItem",
+    "EarthInteractable",
 ]

backend/app/models/earth_interactable.py

@@ -0,0 +1,30 @@
+"""Persistent Earth interactable objects."""
+
+from sqlalchemy import Boolean, Column, DateTime, Float, Index, Integer, JSON, String, Text
+from sqlalchemy.sql import func
+
+from app.db.session import Base
+
+
+class EarthInteractable(Base):
+    __tablename__ = "earth_interactables"
+
+    id = Column(String(160), primary_key=True)
+    layer = Column(String(80), nullable=False, default="interactables", index=True)
+    kind = Column(String(80), nullable=False, default="default", index=True)
+    label = Column(String(255), nullable=False, default="")
+    description = Column(Text, nullable=False, default="")
+    latitude = Column(Float, nullable=False)
+    longitude = Column(Float, nullable=False)
+    altitude = Column(Float, nullable=True)
+    revision = Column(Integer, nullable=False, default=1)
+    properties = Column(JSON, nullable=False, default=dict)
+    is_deleted = Column(Boolean, nullable=False, default=False, index=True)
+    created_at = Column(DateTime(timezone=True), server_default=func.now(), nullable=False)
+    updated_at = Column(DateTime(timezone=True), server_default=func.now(), onupdate=func.now())
+    deleted_at = Column(DateTime(timezone=True), nullable=True, index=True)
+
+    __table_args__ = (
+        Index("idx_earth_interactables_layer_deleted", "layer", "is_deleted"),
+        Index("idx_earth_interactables_updated_at", "updated_at"),
+    )

backend/app/models/task.py

@@ -1,6 +1,6 @@
-"""Collection Task model"""
+"""Datasource job model."""
 
-from sqlalchemy import BigInteger, Column, DateTime, Integer, String, Text, Float
+from sqlalchemy import BigInteger, Column, DateTime, Float, Integer, JSON, String, Text
 from sqlalchemy.sql import func
 
 from app.db.session import Base
@@ -11,7 +11,9 @@ class CollectionTask(Base):
 
     id = Column(Integer, primary_key=True, autoincrement=True)
     datasource_id = Column(Integer, nullable=False, index=True)
-    status = Column(String(20), nullable=False)  # pending, running, success, failed, cancelled
+    source = Column(String(100), nullable=True, index=True)
+    task_type = Column(String(30), nullable=False, default="collect", index=True)
+    status = Column(String(20), nullable=False)  # queued, running, cancelling, success, failed, cancelled
     phase = Column(String(30), default="queued")
     phase_progress = Column(Float)
     phase_message = Column(String(255))
@@ -24,6 +26,13 @@ class CollectionTask(Base):
     total_records = Column(Integer, default=0)  # Total records to process
     progress = Column(Float, default=0.0)  # Progress percentage (0-100)
     error_message = Column(Text)
+    payload = Column(JSON, default=dict)
+    rollback_policy = Column(String(40), nullable=False, default="keep_committed_batches")
+    dedupe_key = Column(String(180), nullable=True, index=True)
+    worker_id = Column(String(120), nullable=True, index=True)
+    locked_at = Column(DateTime(timezone=True), nullable=True, index=True)
+    requested_cancel_at = Column(DateTime(timezone=True), nullable=True)
+    cancel_reason = Column(Text)
     created_at = Column(DateTime(timezone=True), server_default=func.now())
 
     def __repr__(self):

backend/app/services/ai_client.py

@@ -1,18 +1,25 @@
 from __future__ import annotations
 
 import asyncio
+import json
+from time import perf_counter
 
 import httpx
 from fastapi import Depends, HTTPException, status
 from sqlalchemy.ext.asyncio import AsyncSession
 
 from app.core.config import settings
+from app.core.logging import get_logger
 from app.db.session import get_db
 from app.schemas.ai import (
     AIProviderStatusResponse,
     SituationalAnalysisRequest,
     SituationalAnalysisResponse,
 )
+from app.services.business_logs import emit_business_log, exception_context
+
+
+logger = get_logger(__name__, service="ai")
 
 
 class AIProviderClient:
@@ -57,10 +64,25 @@ class AIProviderClient:
             value = self.llm_config.get(key)
             if value not in (None, ""):
                 headers[header_name] = str(value)
+        model_provider_apis = self.llm_config.get("model_provider_apis")
+        if isinstance(model_provider_apis, dict) and model_provider_apis:
+            headers["X-AI-Model-Provider-APIs"] = json.dumps(model_provider_apis)
         return headers
 
     async def get_status(self, request_id: str | None = None) -> AIProviderStatusResponse:
+        context = self._base_log_context(operation="status")
         if not self.service_url:
+            await emit_business_log(
+                logger,
+                event="ai.provider.status.failed",
+                message="AI provider status skipped because service URL is not configured",
+                category="ai",
+                level="warning",
+                service="ai",
+                module=__name__,
+                request_id=request_id,
+                context={**context, "status": "unconfigured"},
+            )
             return AIProviderStatusResponse(
                 provider="unconfigured",
                 enabled=False,
@@ -69,27 +91,133 @@ class AIProviderClient:
                 base_url=None,
             )
 
-        data = await self._request("GET", "/v1/provider/status", request_id=request_id)
-        return AIProviderStatusResponse.model_validate(data)
+        started_at = perf_counter()
+        await emit_business_log(
+            logger,
+            event="ai.provider.status.start",
+            message="AI provider status request started",
+            category="ai",
+            service="ai",
+            module=__name__,
+            request_id=request_id,
+            context=context,
+        )
+        try:
+            data = await self._request("GET", "/v1/provider/status", request_id=request_id, operation="status")
+            result = AIProviderStatusResponse.model_validate(data)
+            await emit_business_log(
+                logger,
+                event="ai.provider.status.success",
+                message="AI provider status request completed",
+                category="ai",
+                service="ai",
+                module=__name__,
+                request_id=request_id,
+                context={
+                    **context,
+                    "status": "success",
+                    "duration_ms": self._duration_ms(started_at),
+                    "result_provider": result.provider,
+                    "result_model": result.model,
+                    "configured": result.configured,
+                    "enabled": result.enabled,
+                },
+            )
+            return result
+        except Exception as exc:
+            await emit_business_log(
+                logger,
+                event="ai.provider.status.failed",
+                message="AI provider status request failed",
+                category="ai",
+                level="error",
+                service="ai",
+                module=__name__,
+                request_id=request_id,
+                context=exception_context(exc, {**context, "status": "failed", "duration_ms": self._duration_ms(started_at)}),
+            )
+            raise
 
     async def analyze(
         self,
         payload: SituationalAnalysisRequest,
         request_id: str | None = None,
     ) -> SituationalAnalysisResponse:
+        context = self._base_log_context(
+            operation="analyze",
+            preferred_model=payload.preferred_model,
+            input_summary=self._summarize_analysis_payload(payload),
+        )
         if not self.service_url:
+            await emit_business_log(
+                logger,
+                event="ai.provider.analyze.failed",
+                message="AI provider analyze skipped because service URL is not configured",
+                category="ai",
+                level="warning",
+                service="ai",
+                module=__name__,
+                request_id=request_id,
+                context={**context, "status": "unconfigured"},
+            )
             raise HTTPException(
                 status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
                 detail="AI provider service URL is not configured.",
             )
 
-        data = await self._request(
-            "POST",
-            "/v1/analyze",
-            json=payload.model_dump(),
+        started_at = perf_counter()
+        await emit_business_log(
+            logger,
+            event="ai.provider.analyze.start",
+            message="AI provider analyze request started",
+            category="ai",
+            service="ai",
+            module=__name__,
             request_id=request_id,
+            context=context,
         )
-        return SituationalAnalysisResponse.model_validate(data)
+        try:
+            data = await self._request(
+                "POST",
+                "/v1/analyze",
+                json=payload.model_dump(),
+                request_id=request_id,
+                operation="analyze",
+                payload_summary=context["input_summary"],
+            )
+            result = SituationalAnalysisResponse.model_validate(data)
+            await emit_business_log(
+                logger,
+                event="ai.provider.analyze.success",
+                message="AI provider analyze request completed",
+                category="ai",
+                service="ai",
+                module=__name__,
+                request_id=request_id,
+                context={
+                    **context,
+                    "status": "success",
+                    "duration_ms": self._duration_ms(started_at),
+                    "result_provider": result.provider,
+                    "result_model": result.model,
+                    "content_block_count": len(result.content_blocks or []),
+                    "thinking_block_count": len(result.thinking_blocks or []),
+                },
+            )
+            return result
+        except Exception as exc:
+            await emit_business_log(
+                logger,
+                event="ai.provider.analyze.failed",
+                message="AI provider analyze request failed",
+                category="ai",
+                level="error",
+                service="ai",
+                module=__name__,
+                request_id=request_id,
+                context=exception_context(exc, {**context, "status": "failed", "duration_ms": self._duration_ms(started_at)}),
+            )
+            raise
 
     async def _request(
         self,
@@ -97,9 +225,12 @@ class AIProviderClient:
         path: str,
         json: dict | None = None,
         request_id: str | None = None,
+        operation: str = "request",
+        payload_summary: dict | None = None,
     ) -> dict:
         last_error: Exception | None = None
         for attempt in range(1, self.retry_attempts + 1):
+            attempt_started_at = perf_counter()
             try:
                 async with httpx.AsyncClient(timeout=self.timeout) as client:
                     response = await client.request(
@@ -113,6 +244,15 @@ class AIProviderClient:
             except httpx.HTTPStatusError as exc:
                 last_error = exc
                 if attempt < self.retry_attempts and exc.response.status_code >= 500:
+                    await self._log_retry(
+                        operation=operation,
+                        request_id=request_id,
+                        attempt=attempt,
+                        status_code=exc.response.status_code,
+                        duration_ms=self._duration_ms(attempt_started_at),
+                        error=exc,
+                        payload_summary=payload_summary,
+                    )
                     await asyncio.sleep(0.3 * attempt)
                     continue
                 detail = exc.response.text or "AI provider service returned an error"
@@ -123,6 +263,14 @@ class AIProviderClient:
             except httpx.HTTPError as exc:
                 last_error = exc
                 if attempt < self.retry_attempts:
+                    await self._log_retry(
+                        operation=operation,
+                        request_id=request_id,
+                        attempt=attempt,
+                        duration_ms=self._duration_ms(attempt_started_at),
+                        error=exc,
+                        payload_summary=payload_summary,
+                    )
                     await asyncio.sleep(0.3 * attempt)
                     continue
                 raise HTTPException(
@@ -135,6 +283,71 @@ class AIProviderClient:
             detail=f"AI provider service request failed: {last_error}",
         )
 
+    def _base_log_context(self, **extra: object) -> dict:
+        llm_provider_apis = self.llm_config.get("model_provider_apis")
+        return {
+            "provider": self.llm_config.get("provider") or "",
+            "provider_api": self.llm_config.get("provider_api") or "",
+            "model": self.llm_config.get("model") or "",
+            "base_url_configured": bool(self.llm_config.get("base_url")),
+            "service_url_configured": bool(self.service_url),
+            "timeout_seconds": self.timeout,
+            "retry_attempts": self.retry_attempts,
+            "model_provider_api_count": len(llm_provider_apis or {}) if isinstance(llm_provider_apis, dict) else 0,
+            **extra,
+        }
+
+    @staticmethod
+    def _duration_ms(started_at: float) -> int:
+        return int((perf_counter() - started_at) * 1000)
+
+    @staticmethod
+    def _summarize_analysis_payload(payload: SituationalAnalysisRequest) -> dict:
+        context = payload.context if isinstance(payload.context, dict) else {}
+        thinking = payload.thinking if isinstance(payload.thinking, dict) else payload.thinking
+        return {
+            "title_length": len(payload.title or ""),
+            "objective_length": len(payload.objective or ""),
+            "observation_count": len(payload.observations or []),
+            "constraint_count": len(payload.constraints or []),
+            "has_system_prompt": bool(payload.system_prompt),
+            "thinking_enabled": bool(thinking),
+            "context_keys": sorted(str(key) for key in context.keys()),
+        }
+
+    async def _log_retry(
+        self,
+        *,
+        operation: str,
+        request_id: str | None,
+        attempt: int,
+        duration_ms: int,
+        error: BaseException,
+        status_code: int | None = None,
+        payload_summary: dict | None = None,
+    ) -> None:
+        await emit_business_log(
+            logger,
+            event=f"ai.provider.{operation}.retry",
+            message="AI provider request will retry",
+            category="ai",
+            level="warning",
+            service="ai",
+            module=__name__,
+            request_id=request_id,
+            context=exception_context(
+                error,
+                {
+                    **self._base_log_context(operation=operation),
+                    "attempt": attempt,
+                    "next_attempt": attempt + 1,
+                    "status_code": status_code,
+                    "duration_ms": duration_ms,
+                    "input_summary": payload_summary,
+                },
+            ),
+        )
+
 
 async def get_ai_provider_client(db: AsyncSession = Depends(get_db)) -> AIProviderClient:
     from app.api.v1.settings import get_runtime_ai_provider_config

backend/app/services/ai_tools/web_fetch.py

@@ -1,11 +1,18 @@
 from __future__ import annotations
 
 import hashlib
+from time import perf_counter
+from urllib.parse import urlparse
 
 import httpx
 from bs4 import BeautifulSoup
 
+from app.core.logging import get_logger
 from app.services.ai_tools.schemas import FetchedEvidence
+from app.services.business_logs import emit_business_log, exception_context
+
+
+logger = get_logger(__name__, service="ai_tool")
 
 
 class WebFetchError(RuntimeError):
@@ -29,8 +36,33 @@ async def fetch_url_evidence(
     timeout_seconds: int = 20,
     max_bytes: int = 1_500_000,
 ) -> FetchedEvidence:
+    started_at = perf_counter()
     if not url:
+        await emit_business_log(
+            logger,
+            event="ai_tool.web_fetch.failed",
+            message="WebFetch failed because URL is empty",
+            category="ai_tool",
+            level="warning",
+            service="ai_tool",
+            module=__name__,
+            context={"reason": "empty_url"},
+        )
         raise WebFetchError("url is required")
+    request_host = urlparse(url).netloc
+    await emit_business_log(
+        logger,
+        event="ai_tool.web_fetch.start",
+        message="WebFetch request started",
+        category="ai_tool",
+        service="ai_tool",
+        module=__name__,
+        context={
+            "url_host": request_host,
+            "timeout_seconds": timeout_seconds,
+            "max_bytes": max_bytes,
+        },
+    )
     try:
         async with httpx.AsyncClient(
             timeout=timeout_seconds,
@@ -41,10 +73,45 @@ async def fetch_url_evidence(
             response.raise_for_status()
             content = response.content[:max_bytes]
     except httpx.HTTPError as exc:
+        await emit_business_log(
+            logger,
+            event="ai_tool.web_fetch.failed",
+            message="WebFetch request failed",
+            category="ai_tool",
+            level="error",
+            service="ai_tool",
+            module=__name__,
+            context=exception_context(
+                exc,
+                {
+                    "url_host": request_host,
+                    "status": "failed",
+                    "duration_ms": int((perf_counter() - started_at) * 1000),
+                },
+            ),
+        )
         raise WebFetchError(f"failed to fetch page: {exc}") from exc
 
     title, text = _extract_title_and_text(content.decode(response.encoding or "utf-8", errors="ignore"))
     content_hash = hashlib.sha256(text.encode("utf-8")).hexdigest()
+    await emit_business_log(
+        logger,
+        event="ai_tool.web_fetch.success",
+        message="WebFetch request completed",
+        category="ai_tool",
+        service="ai_tool",
+        module=__name__,
+        context={
+            "url_host": request_host,
+            "final_url_host": urlparse(str(response.url)).netloc,
+            "status": "success",
+            "status_code": response.status_code,
+            "bytes_read": len(content),
+            "content_hash": content_hash,
+            "duration_ms": int((perf_counter() - started_at) * 1000),
+            "extractor": "beautifulsoup_basic",
+        },
+    )
     return FetchedEvidence(
         url=url,
         final_url=str(response.url),
@@ -53,4 +120,3 @@ async def fetch_url_evidence(
         content_hash=content_hash,
         extractor="beautifulsoup_basic",
     )
-

backend/app/services/ai_tools/web_search.py

@@ -1,11 +1,18 @@
 from __future__ import annotations
 
 from copy import deepcopy
+import hashlib
+from time import perf_counter
 from typing import Any
 
 import httpx
 
+from app.core.logging import get_logger
 from app.services.ai_tools.schemas import SearchEvidence, WebSearchConfig, WebSearchProviderConfig
+from app.services.business_logs import emit_business_log, exception_context
+
+
+logger = get_logger(__name__, service="ai_tool")
 
 
 WEB_SEARCH_PROVIDER_PRESETS: dict[str, dict[str, Any]] = {
@@ -120,29 +127,108 @@ class WebSearchClient:
         domains: list[str] | None = None,
         freshness_days: int | None = None,
     ) -> list[SearchEvidence]:
+        started_at = perf_counter()
         if not self.config.enabled:
+            await emit_business_log(
+                logger,
+                event="ai_tool.web_search.unavailable",
+                message="WebSearch skipped because integration is disabled",
+                category="ai_tool",
+                level="warning",
+                service="ai_tool",
+                module=__name__,
+                context={"provider": self.config.default_provider, "reason": "disabled"},
+            )
             raise WebSearchConfigurationError("WebSearch is disabled.")
         provider_config = self.config.active_provider_config
         provider = normalize_web_search_provider(provider_config.provider)
         if provider != "searxng" and not provider_config.api_key:
+            await emit_business_log(
+                logger,
+                event="ai_tool.web_search.unavailable",
+                message="WebSearch skipped because API key is not configured",
+                category="ai_tool",
+                level="warning",
+                service="ai_tool",
+                module=__name__,
+                context={"provider": provider, "reason": "missing_api_key"},
+            )
             raise WebSearchConfigurationError(f"{provider} API key is not configured.")
         query = " ".join(str(query or "").split())
         if not query:
+            await emit_business_log(
+                logger,
+                event="ai_tool.web_search.failed",
+                message="WebSearch failed because query is empty",
+                category="ai_tool",
+                level="warning",
+                service="ai_tool",
+                module=__name__,
+                context={"provider": provider, "reason": "empty_query"},
+            )
             raise WebSearchConfigurationError("search query is required.")
         limit = max_results or provider_config.max_results
-        if provider == "tavily":
-            return await self._search_tavily(provider_config, query, limit, domains, freshness_days)
-        if provider == "brave":
-            return await self._search_brave(provider_config, query, limit, domains)
-        if provider == "serpapi":
-            return await self._search_serpapi(provider_config, query, limit)
-        if provider == "exa":
-            return await self._search_exa(provider_config, query, limit, domains)
-        if provider == "firecrawl":
-            return await self._search_firecrawl(provider_config, query, limit)
-        if provider == "searxng":
-            return await self._search_searxng(provider_config, query, limit, domains)
-        raise WebSearchConfigurationError(f"Unsupported web search provider: {provider}")
+        context = {
+            "provider": provider,
+            "query_hash": hashlib.sha256(query.encode("utf-8")).hexdigest(),
+            "query_length": len(query),
+            "max_results": limit,
+            "domain_count": len(domains or []),
+            "freshness_days": freshness_days,
+        }
+        await emit_business_log(
+            logger,
+            event="ai_tool.web_search.start",
+            message="WebSearch request started",
+            category="ai_tool",
+            service="ai_tool",
+            module=__name__,
+            context=context,
+        )
+        try:
+            if provider == "tavily":
+                results = await self._search_tavily(provider_config, query, limit, domains, freshness_days)
+            elif provider == "brave":
+                results = await self._search_brave(provider_config, query, limit, domains)
+            elif provider == "serpapi":
+                results = await self._search_serpapi(provider_config, query, limit)
+            elif provider == "exa":
+                results = await self._search_exa(provider_config, query, limit, domains)
+            elif provider == "firecrawl":
+                results = await self._search_firecrawl(provider_config, query, limit)
+            elif provider == "searxng":
+                results = await self._search_searxng(provider_config, query, limit, domains)
+            else:
+                raise WebSearchConfigurationError(f"Unsupported web search provider: {provider}")
+            event = "ai_tool.web_search.success" if results else "ai_tool.web_search.empty"
+            await emit_business_log(
+                logger,
+                event=event,
+                message="WebSearch request completed" if results else "WebSearch returned no results",
+                category="ai_tool",
+                level="info" if results else "warning",
+                service="ai_tool",
+                module=__name__,
+                context={
+                    **context,
+                    "status": "success" if results else "empty",
+                    "result_count": len(results),
+                    "duration_ms": int((perf_counter() - started_at) * 1000),
+                },
+            )
+            return results
+        except Exception as exc:
+            await emit_business_log(
+                logger,
+                event="ai_tool.web_search.failed",
+                message="WebSearch request failed",
+                category="ai_tool",
+                level="error",
+                service="ai_tool",
+                module=__name__,
+                context=exception_context(exc, {**context, "status": "failed", "duration_ms": int((perf_counter() - started_at) * 1000)}),
+            )
+            raise
 
     async def test_connection(self) -> list[SearchEvidence]:
         return await self.search("Planet WebSearch connectivity test", max_results=1)
@@ -388,4 +474,3 @@ def _float_or_none(value: Any) -> float | None:
         return float(value)
     except (TypeError, ValueError):
         return None
-

backend/app/services/business_logs.py

@@ -0,0 +1,117 @@
+from __future__ import annotations
+
+import asyncio
+from collections.abc import Mapping
+from typing import Any
+
+from app.core.logging import PlanetLoggerAdapter, sanitize_log_value
+from app.core.request_context import get_request_id
+from app.services.persistent_logs import record_system_log
+
+
+LEVEL_METHODS = {
+    "debug": "debug_event",
+    "info": "info_event",
+    "warning": "warning_event",
+    "error": "error_event",
+}
+
+
+def normalize_business_level(level: str | None) -> str:
+    normalized = str(level or "info").strip().lower()
+    if normalized in {"warn", "warning"}:
+        return "warning"
+    if normalized in {"err", "error", "critical", "fatal"}:
+        return "error"
+    if normalized == "debug":
+        return "debug"
+    return "info"
+
+
+def build_business_context(
+    context: Mapping[str, Any] | None = None,
+    **fields: Any,
+) -> dict[str, Any]:
+    payload = dict(context or {})
+    for key, value in fields.items():
+        if value is not None:
+            payload[key] = value
+    return sanitize_log_value(payload)
+
+
+async def emit_business_log(
+    logger: PlanetLoggerAdapter,
+    *,
+    event: str,
+    message: str,
+    category: str,
+    level: str = "info",
+    source: str = "backend",
+    service: str | None = None,
+    module: str | None = None,
+    request_id: str | None = None,
+    user_id: int | None = None,
+    context: Mapping[str, Any] | None = None,
+) -> None:
+    normalized_level = normalize_business_level(level)
+    safe_context = build_business_context(context)
+    log_method = getattr(logger, LEVEL_METHODS[normalized_level])
+    log_method(message, event=event, context=safe_context)
+    await record_system_log(
+        source=source,
+        level=normalized_level,
+        message=message,
+        service=service,
+        module=module,
+        event=event,
+        request_id=request_id or get_request_id(),
+        user_id=user_id,
+        category=category,
+        context=safe_context,
+    )
+
+
+def emit_business_log_background(
+    logger: PlanetLoggerAdapter,
+    *,
+    event: str,
+    message: str,
+    category: str,
+    level: str = "info",
+    source: str = "backend",
+    service: str | None = None,
+    module: str | None = None,
+    request_id: str | None = None,
+    user_id: int | None = None,
+    context: Mapping[str, Any] | None = None,
+) -> None:
+    normalized_level = normalize_business_level(level)
+    safe_context = build_business_context(context)
+    log_method = getattr(logger, LEVEL_METHODS[normalized_level])
+    log_method(message, event=event, context=safe_context)
+    try:
+        loop = asyncio.get_running_loop()
+    except RuntimeError:
+        return
+    loop.create_task(
+        record_system_log(
+            source=source,
+            level=normalized_level,
+            message=message,
+            service=service,
+            module=module,
+            event=event,
+            request_id=request_id or get_request_id(),
+            user_id=user_id,
+            category=category,
+            context=safe_context,
+        )
+    )
+
+
+def exception_context(exc: BaseException, context: Mapping[str, Any] | None = None) -> dict[str, Any]:
+    return build_business_context(
+        context,
+        error_type=type(exc).__name__,
+        error=str(exc),
+    )

backend/app/services/collectors/base.py

@@ -4,16 +4,22 @@ import asyncio
 from abc import ABC, abstractmethod
 from typing import Dict, List, Any, Optional
 from datetime import UTC, datetime
+from time import perf_counter
+from urllib.parse import urlparse
 import httpx
 from sqlalchemy import select, text
 from sqlalchemy.ext.asyncio import AsyncSession
 
 from app.core.collected_data_fields import build_dynamic_metadata, get_record_field
-from app.core.config import settings
 from app.core.countries import normalize_country
+from app.core.logging import get_logger
 from app.core.time import to_iso8601_utc
 from app.core.websocket.broadcaster import broadcaster
-from app.services.earth_layer_cache import invalidate_earth_layer_cache_for_source
+from app.services.business_logs import emit_business_log, exception_context
+from app.services.earth_layer_adapters import get_earth_update_layers_for_source
+
+
+logger = get_logger(__name__, service="collector")
 
 
 class BaseCollector(ABC):
@@ -32,6 +38,7 @@ class BaseCollector(ABC):
         self._datasource_id = 1
         self._resolved_url: Optional[str] = None
         self._last_broadcast_progress: Optional[int] = None
+        self._last_save_summary: dict[str, int] = {}
 
     async def resolve_url(self, db: AsyncSession) -> None:
         from app.core.data_sources import get_data_sources_config
@@ -187,7 +194,7 @@ class BaseCollector(ABC):
 
         result = await db.execute(
             select(DataSnapshot)
-            .where(DataSnapshot.source == self.name, DataSnapshot.is_current == True)
+            .where(DataSnapshot.source == self.name, DataSnapshot.is_current.is_(True))
             .order_by(DataSnapshot.completed_at.desc().nullslast(), DataSnapshot.id.desc())
             .limit(1)
         )
@@ -273,19 +280,39 @@ class BaseCollector(ABC):
         from app.models.data_snapshot import DataSnapshot
 
         start_time = datetime.now(UTC)
+        started_at = perf_counter()
         datasource_id = getattr(self, "_datasource_id", 1)
         snapshot_id: Optional[int] = None
 
         if not collector_registry.is_active(self.name):
+            await self._log_collection_event(
+                "collector.run.skipped_disabled",
+                "Collector skipped because it is disabled",
+                level="info",
+                context={"status": "skipped", "reason": "disabled"},
+            )
             return {"status": "skipped", "reason": "Collector is disabled"}
 
-        task = CollectionTask(
-            datasource_id=datasource_id,
-            status="running",
-            phase="queued",
-            started_at=start_time,
-        )
-        db.add(task)
+        task = self._current_task if isinstance(self._current_task, CollectionTask) else None
+        if task is None:
+            task = CollectionTask(
+                datasource_id=datasource_id,
+                source=self.name,
+                task_type="collect",
+                status="running",
+                phase="queued",
+                started_at=start_time,
+            )
+            db.add(task)
+        else:
+            task.datasource_id = datasource_id
+            task.source = task.source or self.name
+            task.task_type = task.task_type or "collect"
+            task.status = "running"
+            task.phase = "queued"
+            task.started_at = task.started_at or start_time
+            task.completed_at = None
+            task.error_message = None
         await db.commit()
         task_id = task.id
 
@@ -295,23 +322,76 @@ class BaseCollector(ABC):
 
         await self.resolve_url(db)
         await self._publish_task_update(force=True)
+        await self._log_collection_event(
+            "collector.run.started",
+            "Collector run started",
+            context={"status": "running", "task_id": task_id},
+        )
 
         try:
+            phase_started_at = perf_counter()
             await self.set_phase("fetching", message="正在拉取原始数据")
+            await self._log_collection_event(
+                "collector.phase.fetching.start",
+                "Collector fetch phase started",
+                context={"task_id": task_id, "snapshot_id": snapshot_id},
+            )
             raw_data = await self.fetch()
             task.total_records = len(raw_data)
             await db.commit()
             await self._publish_task_update(force=True)
+            await self._log_collection_event(
+                "collector.phase.fetching.success",
+                "Collector fetch phase completed",
+                context={
+                    "task_id": task_id,
+                    "raw_count": len(raw_data),
+                    "duration_ms": self._duration_ms(phase_started_at),
+                },
+            )
 
             if self.fail_on_empty and not raw_data:
                 raise RuntimeError(f"Collector {self.name} returned no data")
 
+            phase_started_at = perf_counter()
             await self.set_phase("transforming", message="正在转换采集数据")
+            await self._log_collection_event(
+                "collector.phase.transforming.start",
+                "Collector transform phase started",
+                context={"task_id": task_id, "raw_count": len(raw_data)},
+            )
             data = self.transform(raw_data)
+            await self._log_collection_event(
+                "collector.phase.transforming.success",
+                "Collector transform phase completed",
+                context={
+                    "task_id": task_id,
+                    "raw_count": len(raw_data),
+                    "transformed_count": len(data),
+                    "duration_ms": self._duration_ms(phase_started_at),
+                },
+            )
             snapshot_id = await self._create_snapshot(db, task_id, data, start_time)
 
+            phase_started_at = perf_counter()
             await self.set_phase("saving", message="正在保存采集数据")
+            await self._log_collection_event(
+                "collector.phase.saving.start",
+                "Collector save phase started",
+                context={"task_id": task_id, "snapshot_id": snapshot_id, "transformed_count": len(data)},
+            )
             records_count = await self._save_data(db, data, task_id=task_id, snapshot_id=snapshot_id)
+            await self._log_collection_event(
+                "collector.phase.saving.success",
+                "Collector save phase completed",
+                context={
+                    "task_id": task_id,
+                    "snapshot_id": snapshot_id,
+                    "saved_count": records_count,
+                    **self._last_save_summary,
+                    "duration_ms": self._duration_ms(phase_started_at),
+                },
+            )
 
             task.status = "success"
             task.phase = "completed"
@@ -325,6 +405,20 @@ class BaseCollector(ABC):
             task.completed_at = datetime.now(UTC)
             await db.commit()
             await self._publish_task_update(force=True)
+            await self._log_collection_event(
+                "collector.run.completed",
+                "Collector run completed",
+                context={
+                    "status": "success",
+                    "task_id": task_id,
+                    "snapshot_id": snapshot_id,
+                    "raw_count": len(raw_data),
+                    "transformed_count": len(data),
+                    "saved_count": records_count,
+                    **self._last_save_summary,
+                    "duration_ms": self._duration_ms(started_at),
+                },
+            )
 
             return {
                 "status": "success",
@@ -348,6 +442,17 @@ class BaseCollector(ABC):
                 )
             await db.commit()
             await self._publish_task_update(force=True)
+            await self._log_collection_event(
+                "collector.run.cancelled",
+                "Collector run cancelled",
+                level="warning",
+                context={
+                    "status": "cancelled",
+                    "task_id": task_id,
+                    "snapshot_id": snapshot_id,
+                    "duration_ms": self._duration_ms(started_at),
+                },
+            )
             raise
         except Exception as e:
             await db.rollback()
@@ -364,6 +469,20 @@ class BaseCollector(ABC):
                     snapshot.summary = {"error": str(e)}
             await db.commit()
             await self._publish_task_update(force=True)
+            await self._log_collection_event(
+                "collector.run.failed",
+                "Collector run failed",
+                level="error",
+                context=exception_context(
+                    e,
+                    {
+                        "status": "failed",
+                        "task_id": task_id,
+                        "snapshot_id": snapshot_id,
+                        "duration_ms": self._duration_ms(started_at),
+                    },
+                ),
+            )
 
             return {
                 "status": "failed",
@@ -384,6 +503,7 @@ class BaseCollector(ABC):
         from app.models.data_snapshot import DataSnapshot
 
         if not data:
+            self._last_save_summary = {"created": 0, "updated": 0, "unchanged": 0, "deleted": 0}
             if snapshot_id is not None:
                 snapshot = await db.get(DataSnapshot, snapshot_id)
                 if snapshot:
@@ -406,7 +526,7 @@ class BaseCollector(ABC):
             select(CollectedData)
             .where(
                 CollectedData.source == self.name,
-                CollectedData.is_current == True,
+                CollectedData.is_current.is_(True),
             )
             .order_by(CollectedData.entity_key.asc(), CollectedData.collected_at.desc().nullslast(), CollectedData.id.desc())
         )
@@ -530,12 +650,51 @@ class BaseCollector(ABC):
                     "unchanged": unchanged_count,
                     "deleted": len(deleted_keys),
                 }
+            self._last_save_summary = {
+                "created": created_count,
+                "updated": updated_count,
+                "unchanged": unchanged_count,
+                "deleted": len(deleted_keys),
+            }
+        else:
+            self._last_save_summary = {
+                "created": created_count,
+                "updated": updated_count,
+                "unchanged": unchanged_count,
+                "deleted": 0,
+            }
 
         await db.commit()
-        invalidate_earth_layer_cache_for_source(self.name)
         await self.update_progress(len(data), force=True)
         return records_added
 
+    @staticmethod
+    def _duration_ms(started_at: float) -> int:
+        return int((perf_counter() - started_at) * 1000)
+
+    async def _log_collection_event(
+        self,
+        event: str,
+        message: str,
+        *,
+        level: str = "info",
+        context: Dict[str, Any] | None = None,
+    ) -> None:
+        await emit_business_log(
+            logger,
+            event=event,
+            message=message,
+            category="collector",
+            level=level,
+            service="collector",
+            module=__name__,
+            context={
+                "collector_name": self.name,
+                "datasource_id": getattr(self, "_datasource_id", None),
+                **(context or {}),
+            },
+        )
+
     async def save(self, db: AsyncSession, data: List[Dict[str, Any]]) -> int:
         """Save data to database (legacy method, use _save_data instead)"""
         return await self._save_data(db, data)
@@ -548,10 +707,65 @@ class HTTPCollector(BaseCollector):
     headers: Dict[str, str] = {}
 
     async def fetch(self) -> List[Dict[str, Any]]:
+        started_at = perf_counter()
+        request_host = urlparse(self.base_url).netloc
+        await emit_business_log(
+            logger,
+            event="collector.http.fetch.start",
+            message="Collector HTTP request started",
+            category="collector",
+            service="collector",
+            module=__name__,
+            context={
+                "collector_name": self.name,
+                "datasource_id": getattr(self, "_datasource_id", None),
+                "url_host": request_host,
+            },
+        )
         async with httpx.AsyncClient(timeout=60.0) as client:
-            response = await client.get(self.base_url, headers=self.headers)
-            response.raise_for_status()
-            return self.parse_response(response.json())
+            try:
+                response = await client.get(self.base_url, headers=self.headers)
+                response.raise_for_status()
+                payload = response.json()
+                parsed = self.parse_response(payload)
+                await emit_business_log(
+                    logger,
+                    event="collector.http.fetch.success",
+                    message="Collector HTTP request completed",
+                    category="collector",
+                    service="collector",
+                    module=__name__,
+                    context={
+                        "collector_name": self.name,
+                        "datasource_id": getattr(self, "_datasource_id", None),
+                        "url_host": request_host,
+                        "status_code": response.status_code,
+                        "response_bytes": len(response.content or b""),
+                        "parsed_count": len(parsed),
+                        "duration_ms": BaseCollector._duration_ms(started_at),
+                    },
+                )
+                return parsed
+            except Exception as exc:
+                await emit_business_log(
+                    logger,
+                    event="collector.http.fetch.failed",
+                    message="Collector HTTP request failed",
+                    category="collector",
+                    level="error",
+                    service="collector",
+                    module=__name__,
+                    context=exception_context(
+                        exc,
+                        {
+                            "collector_name": self.name,
+                            "datasource_id": getattr(self, "_datasource_id", None),
+                            "url_host": request_host,
+                            "duration_ms": BaseCollector._duration_ms(started_at),
+                        },
+                    ),
+                )
+                raise
 
     @abstractmethod
     def parse_response(self, response: Dict[str, Any]) -> List[Dict[str, Any]]:

backend/app/services/collectors/celestrak.py

@@ -1,15 +1,39 @@
-"""CelesTrak TLE Collector
+"""CelesTrak TLE Collector.
 
-Collects satellite TLE (Two-Line Element) data from CelesTrak.org.
-Free, no authentication required.
+Collects the full active satellite GP element set from CelesTrak.
 """
 
+import asyncio
 import json
-from typing import Dict, Any, List
+from pathlib import Path
+from time import perf_counter
+from typing import Any, Dict, List
+from urllib.parse import urlencode, urlparse
+
 import httpx
 
+from app.core.logging import get_logger
 from app.core.satellite_tle import build_tle_lines_from_elements
+from app.services.business_logs import emit_business_log, exception_context
 from app.services.collectors.base import BaseCollector
+from app.services.collectors.downloads import DownloadHTTPStatusError, ResumableFileDownloader
+
+
+logger = get_logger(__name__, service="collector")
+ACTIVE_GROUP = "active"
+FALLBACK_GROUPS = (
+    "starlink",
+    "gps-ops",
+    "galileo",
+    "glonass",
+    "beidou",
+    "leo",
+    "geo",
+    "iridium-next",
+)
+FETCH_RETRY_ATTEMPTS = 3
+FETCH_RETRY_BASE_DELAY_SECONDS = 0.8
+CELESTRAK_NOT_UPDATED_MARKER = "GP data has not updated since your last successful"
 
 
 class CelesTrakTLECollector(BaseCollector):
@@ -18,55 +42,359 @@ class CelesTrakTLECollector(BaseCollector):
     module = "L3"
     frequency_hours = 24
     data_type = "satellite_tle"
+    _downloader = ResumableFileDownloader(
+        cache_namespace="celestrak",
+        default_accept="application/json",
+    )
 
     @property
     def base_url(self) -> str:
         return self._resolved_url or ""
 
+    def _active_url(self) -> str:
+        return self._group_url(ACTIVE_GROUP)
+
+    def _group_url(self, group: str) -> str:
+        if not self.base_url:
+            raise RuntimeError("CelesTrak base URL is not configured")
+        return f"{self.base_url}?{urlencode({'GROUP': group, 'FORMAT': 'json'})}"
+
     async def fetch(self) -> List[Dict[str, Any]]:
-        satellite_groups = [
-            "starlink",
-            "gps-ops",
-            "galileo",
-            "glonass",
-            "beidou",
-            "leo",
-            "geo",
-            "iridium-next",
-        ]
+        url = self._active_url()
+        last_error: Exception | None = None
 
-        all_satellites = []
-
-        async with httpx.AsyncClient(timeout=120.0) as client:
-            for group in satellite_groups:
+        async with httpx.AsyncClient(timeout=180.0, follow_redirects=True) as client:
+            for attempt in range(1, FETCH_RETRY_ATTEMPTS + 1):
+                started_at = perf_counter()
                 try:
-                    url = f"{self.base_url}?GROUP={group}&FORMAT=json"
-                    response = await client.get(url)
+                    await emit_business_log(
+                        logger,
+                        event="collector.celestrak.download.start",
+                        message="CelesTrak active satellite download started",
+                        category="collector",
+                        service="collector",
+                        module=__name__,
+                        context={
+                            "collector_name": self.name,
+                            "datasource_id": getattr(self, "_datasource_id", None),
+                            "group": ACTIVE_GROUP,
+                            "attempt": attempt,
+                            "url_host": urlparse(url).netloc,
+                        },
+                    )
+                    body_path = await self._downloader.download_file(
+                        client,
+                        url,
+                        extension=".json",
+                        accept="application/json",
+                        progress_callback=self._report_download_progress,
+                        validate_existing=self._validate_json_file,
+                    )
+                    data = await self._load_downloaded_payload(body_path, url)
+                    await emit_business_log(
+                        logger,
+                        event="collector.celestrak.download.success",
+                        message="CelesTrak active satellite download completed",
+                        category="collector",
+                        service="collector",
+                        module=__name__,
+                        context={
+                            "collector_name": self.name,
+                            "datasource_id": getattr(self, "_datasource_id", None),
+                            "group": ACTIVE_GROUP,
+                            "attempt": attempt,
+                            "record_count": len(data),
+                            "duration_ms": self._duration_ms(started_at),
+                        },
+                    )
+                    return data
+                except DownloadHTTPStatusError as exc:
+                    if self._is_not_updated_response(exc):
+                        cached_path = self._downloader.get_cached_file(
+                            url,
+                            ".json",
+                            validate_existing=self._validate_json_file,
+                        )
+                        if cached_path is not None:
+                            data = await self._load_downloaded_payload(cached_path, url)
+                            await emit_business_log(
+                                logger,
+                                event="collector.celestrak.download.cached_not_updated",
+                                message="CelesTrak active satellite data has not changed; using cached download",
+                                category="collector",
+                                level="warning",
+                                service="collector",
+                                module=__name__,
+                                context={
+                                    "collector_name": self.name,
+                                    "datasource_id": getattr(self, "_datasource_id", None),
+                                    "group": ACTIVE_GROUP,
+                                    "attempt": attempt,
+                                    "record_count": len(data),
+                                    "duration_ms": self._duration_ms(started_at),
+                                },
+                            )
+                            return data
+                        await emit_business_log(
+                            logger,
+                            event="collector.celestrak.download.not_updated_no_cache",
+                            message="CelesTrak active satellite data has not changed; trying fallback groups",
+                            category="collector",
+                            level="warning",
+                            service="collector",
+                            module=__name__,
+                            context=exception_context(
+                                exc,
+                                {
+                                    "collector_name": self.name,
+                                    "datasource_id": getattr(self, "_datasource_id", None),
+                                    "group": ACTIVE_GROUP,
+                                    "attempt": attempt,
+                                    "duration_ms": self._duration_ms(started_at),
+                                },
+                            ),
+                        )
+                        return await self._fetch_fallback_groups(client, active_error=exc)
+                    raise
+                except Exception as exc:
+                    last_error = exc
+                    is_final_attempt = attempt >= FETCH_RETRY_ATTEMPTS
+                    await emit_business_log(
+                        logger,
+                        event=(
+                            "collector.celestrak.download.failed"
+                            if is_final_attempt
+                            else "collector.celestrak.download.retry"
+                        ),
+                        message=(
+                            "CelesTrak active satellite download failed"
+                            if is_final_attempt
+                            else "CelesTrak active satellite download will retry"
+                        ),
+                        category="collector",
+                        level="error" if is_final_attempt else "warning",
+                        service="collector",
+                        module=__name__,
+                        context=exception_context(
+                            exc,
+                            {
+                                "collector_name": self.name,
+                                "datasource_id": getattr(self, "_datasource_id", None),
+                                "group": ACTIVE_GROUP,
+                                "attempt": attempt,
+                                "duration_ms": self._duration_ms(started_at),
+                            },
+                        ),
+                    )
+                    if not is_final_attempt:
+                        await asyncio.sleep(FETCH_RETRY_BASE_DELAY_SECONDS * attempt)
 
-                    if response.status_code == 200:
-                        data = response.json()
-                        if isinstance(data, list):
-                            for item in data:
-                                if isinstance(item, dict):
-                                    item["_celestrak_group"] = group
-                            all_satellites.extend(data)
-                            print(f"CelesTrak: Fetched {len(data)} satellites from group '{group}'")
-                except Exception as e:
-                    print(f"CelesTrak: Error fetching group '{group}': {e}")
+        raise RuntimeError(f"CelesTrak active satellite download failed after retries: {last_error}")
 
-        if not all_satellites:
-            return self._get_sample_data()
+    async def _fetch_fallback_groups(
+        self,
+        client: httpx.AsyncClient,
+        *,
+        active_error: DownloadHTTPStatusError,
+    ) -> List[Dict[str, Any]]:
+        started_at = perf_counter()
+        records_by_norad: dict[str, Dict[str, Any]] = {}
+        group_counts: dict[str, int] = {}
 
-        print(f"CelesTrak: Total satellites fetched: {len(all_satellites)}")
+        await emit_business_log(
+            logger,
+            event="collector.celestrak.fallback_groups.start",
+            message="CelesTrak fallback group download started",
+            category="collector",
+            level="warning",
+            service="collector",
+            module=__name__,
+            context={
+                "collector_name": self.name,
+                "datasource_id": getattr(self, "_datasource_id", None),
+                "groups": list(FALLBACK_GROUPS),
+                "reason": "active_not_updated_without_cache",
+            },
+        )
 
-        # Return raw data - base.run() will call transform()
-        return all_satellites
+        try:
+            for group in FALLBACK_GROUPS:
+                group_url = self._group_url(group)
+                try:
+                    body_path = await self._downloader.download_file(
+                        client,
+                        group_url,
+                        extension=".json",
+                        accept="application/json",
+                        validate_existing=self._validate_json_file,
+                    )
+                except DownloadHTTPStatusError as exc:
+                    if not self._is_not_updated_response(exc):
+                        raise RuntimeError(f"CelesTrak fallback group '{group}' download failed: {exc}") from exc
+                    cached_path = self._downloader.get_cached_file(
+                        group_url,
+                        ".json",
+                        validate_existing=self._validate_json_file,
+                    )
+                    if cached_path is None:
+                        raise RuntimeError(
+                            f"CelesTrak fallback group '{group}' has not updated and no local cached copy is available"
+                        ) from exc
+                    body_path = cached_path
+
+                group_records = await self._load_downloaded_payload(
+                    body_path,
+                    group_url,
+                    query_group=group,
+                    constellation_group=group,
+                )
+                group_counts[group] = len(group_records)
+                for item in group_records:
+                    norad_cat_id = item.get("NORAD_CAT_ID")
+                    if norad_cat_id is None:
+                        continue
+                    records_by_norad.setdefault(str(norad_cat_id), item)
+        except Exception as exc:
+            await emit_business_log(
+                logger,
+                event="collector.celestrak.fallback_groups.failed",
+                message="CelesTrak fallback group download failed",
+                category="collector",
+                level="error",
+                service="collector",
+                module=__name__,
+                context=exception_context(
+                    exc,
+                    {
+                        "collector_name": self.name,
+                        "datasource_id": getattr(self, "_datasource_id", None),
+                        "groups": list(FALLBACK_GROUPS),
+                        "completed_groups": list(group_counts),
+                        "duration_ms": self._duration_ms(started_at),
+                    },
+                ),
+            )
+            raise RuntimeError(
+                "CelesTrak active data has not updated since this network's last successful download, "
+                "no active cache is available, and fallback group mode failed. Wait until CelesTrak "
+                "publishes the next GP update, restore the Planet download cache, or use Space-Track."
+            ) from active_error
+
+        records = list(records_by_norad.values())
+        if not records:
+            raise RuntimeError("CelesTrak fallback group mode produced no satellite records")
+
+        await emit_business_log(
+            logger,
+            event="collector.celestrak.fallback_groups.success",
+            message="CelesTrak fallback group download completed",
+            category="collector",
+            level="warning",
+            service="collector",
+            module=__name__,
+            context={
+                "collector_name": self.name,
+                "datasource_id": getattr(self, "_datasource_id", None),
+                "groups": list(FALLBACK_GROUPS),
+                "group_counts": group_counts,
+                "record_count": len(records),
+                "duration_ms": self._duration_ms(started_at),
+            },
+        )
+        return records
+
+    async def _load_downloaded_payload(
+        self,
+        body_path: Path,
+        url: str,
+        *,
+        query_group: str = ACTIVE_GROUP,
+        constellation_group: str | None = None,
+    ) -> List[Dict[str, Any]]:
+        try:
+            data = self._load_active_payload(body_path)
+        except RuntimeError as exc:
+            await self._log_parse_failure(exc)
+            raise
+        for item in data:
+            item["_celestrak_query_group"] = query_group
+            item["_celestrak_source_url"] = url
+            if constellation_group:
+                item["_celestrak_group"] = constellation_group
+        return data
+
+    @staticmethod
+    def _is_not_updated_response(exc: DownloadHTTPStatusError) -> bool:
+        return exc.status_code == 403 and CELESTRAK_NOT_UPDATED_MARKER in exc.body
+
+    @staticmethod
+    def _duration_ms(started_at: float) -> int:
+        return int((perf_counter() - started_at) * 1000)
+
+    async def _report_download_progress(self, downloaded: int, total: int | None) -> None:
+        if total and total > 0:
+            await self.update_phase_progress(
+                current=min(downloaded, total),
+                total=total,
+                unit="bytes",
+                message=f"正在下载 CelesTrak active 卫星数据 {downloaded}/{total} bytes",
+                commit=True,
+            )
+
+    @staticmethod
+    def _validate_json_file(path: Path) -> bool:
+        try:
+            data = json.loads(path.read_text(encoding="utf-8"))
+        except (OSError, json.JSONDecodeError, UnicodeDecodeError):
+            return False
+        return isinstance(data, list)
+
+    async def _log_parse_failure(self, exc: Exception) -> None:
+        await emit_business_log(
+            logger,
+            event="collector.celestrak.parse.failed",
+            message="CelesTrak active satellite JSON parsing failed",
+            category="collector",
+            level="error",
+            service="collector",
+            module=__name__,
+            context=exception_context(
+                exc,
+                {
+                    "collector_name": self.name,
+                    "datasource_id": getattr(self, "_datasource_id", None),
+                    "group": ACTIVE_GROUP,
+                },
+            ),
+        )
+
+    def _load_active_payload(self, path: Path) -> List[Dict[str, Any]]:
+        try:
+            raw = json.loads(path.read_text(encoding="utf-8"))
+        except (OSError, json.JSONDecodeError, UnicodeDecodeError) as exc:
+            raise RuntimeError(f"CelesTrak active payload is not valid JSON: {exc}") from exc
+        if not isinstance(raw, list):
+            raise RuntimeError("CelesTrak active payload is not a JSON array")
+
+        records: List[Dict[str, Any]] = []
+        invalid_count = 0
+        for item in raw:
+            if isinstance(item, dict) and item.get("NORAD_CAT_ID") is not None:
+                records.append(item)
+            else:
+                invalid_count += 1
+        if invalid_count:
+            raise RuntimeError(f"CelesTrak active payload contains {invalid_count} invalid record(s)")
+        if not records:
+            raise RuntimeError("CelesTrak active payload contains no satellite records")
+        return records
 
     def transform(self, raw_data: List[Dict[str, Any]]) -> List[Dict[str, Any]]:
         transformed = []
         for item in raw_data:
+            norad_cat_id = item.get("NORAD_CAT_ID")
             tle_line1, tle_line2 = build_tle_lines_from_elements(
-                norad_cat_id=item.get("NORAD_CAT_ID"),
+                norad_cat_id=norad_cat_id,
                 epoch=item.get("EPOCH"),
                 inclination=item.get("INCLINATION"),
                 raan=item.get("RA_OF_ASC_NODE"),
@@ -75,14 +403,18 @@ class CelesTrakTLECollector(BaseCollector):
                 mean_anomaly=item.get("MEAN_ANOMALY"),
                 mean_motion=item.get("MEAN_MOTION"),
             )
+            constellation_group = self._infer_constellation_group(item)
 
             transformed.append(
                 {
+                    "source_id": str(norad_cat_id),
                     "name": item.get("OBJECT_NAME", "Unknown"),
                     "reference_date": item.get("EPOCH", ""),
                     "metadata": {
-                        "constellation_group": item.get("_celestrak_group"),
-                        "norad_cat_id": item.get("NORAD_CAT_ID"),
+                        "constellation_group": constellation_group,
+                        "celestrak_query_group": item.get("_celestrak_query_group") or ACTIVE_GROUP,
+                        "celestrak_source_url": item.get("_celestrak_source_url"),
+                        "norad_cat_id": norad_cat_id,
                         "international_designator": item.get("OBJECT_ID"),
                         "epoch": item.get("EPOCH"),
                         "mean_motion": item.get("MEAN_MOTION"),
@@ -105,6 +437,19 @@ class CelesTrakTLECollector(BaseCollector):
             )
         return transformed
 
+    @staticmethod
+    def _infer_constellation_group(item: Dict[str, Any]) -> str | None:
+        explicit_group = str(item.get("_celestrak_group") or "").strip().lower()
+        if explicit_group and explicit_group != ACTIVE_GROUP:
+            return explicit_group
+
+        name = str(item.get("OBJECT_NAME") or "").strip().upper()
+        if name.startswith("STARLINK"):
+            return "starlink"
+        if name.startswith("IRIDIUM"):
+            return "iridium-next"
+        return None
+
     def _get_sample_data(self) -> List[Dict[str, Any]]:
         return [
             {

backend/app/services/collectors/downloads.py

@@ -4,7 +4,7 @@ from __future__ import annotations
 
 import hashlib
 import json
-import tempfile
+import os
 import time
 from datetime import UTC, datetime
 from pathlib import Path
@@ -17,6 +17,31 @@ ProgressCallback = Callable[[int, int | None], Awaitable[None]]
 ValidateCallback = Callable[[Path], bool]
 
 
+class DownloadHTTPStatusError(RuntimeError):
+    """HTTP status error that keeps the upstream response body for caller-specific handling."""
+
+    def __init__(self, *, url: str, status_code: int, body: str) -> None:
+        self.url = url
+        self.status_code = status_code
+        self.body = body
+        preview = body.strip().replace("\r", " ").replace("\n", " ")[:240]
+        suffix = f": {preview}" if preview else ""
+        super().__init__(f"HTTP {status_code} while downloading {url}{suffix}")
+
+
+def default_download_cache_root() -> Path:
+    configured = os.getenv("PLANET_DOWNLOAD_CACHE_DIR")
+    if configured:
+        return Path(configured).expanduser()
+    planet_cache = os.getenv("PLANET_CACHE_DIR")
+    if planet_cache:
+        return Path(planet_cache).expanduser() / "downloads"
+    xdg_cache = os.getenv("XDG_CACHE_HOME")
+    if xdg_cache:
+        return Path(xdg_cache).expanduser() / "planet" / "downloads"
+    return Path.home() / ".cache" / "planet" / "downloads"
+
+
 class ResumableFileDownloader:
     """Download files with cache validators and byte-range resume support."""
 
@@ -26,8 +51,9 @@ class ResumableFileDownloader:
         cache_namespace: str,
         user_agent: str = "Planet-Intelligence-System/1.0 (Python/collector)",
         default_accept: str = "*/*",
+        cache_root: Path | None = None,
     ) -> None:
-        self._cache_dir = Path(tempfile.gettempdir()) / "planet-download-cache" / cache_namespace
+        self._cache_dir = (cache_root or default_download_cache_root()) / cache_namespace
         self._user_agent = user_agent
         self._default_accept = default_accept
 
@@ -43,6 +69,25 @@ class ResumableFileDownloader:
         meta_path = self._cache_dir / f"{key}.meta.json"
         return final_path, part_path, meta_path
 
+    def cached_file_path(self, url: str, extension: str) -> Path:
+        final_path, _, _ = self._cache_paths(url, extension)
+        return final_path
+
+    def get_cached_file(
+        self,
+        url: str,
+        extension: str,
+        *,
+        validate_existing: ValidateCallback | None = None,
+    ) -> Path | None:
+        final_path = self.cached_file_path(url, extension)
+        if not final_path.exists():
+            return None
+        if validate_existing and not validate_existing(final_path):
+            final_path.unlink(missing_ok=True)
+            return None
+        return final_path
+
     @staticmethod
     def _load_meta(meta_path: Path) -> dict[str, Any]:
         if not meta_path.exists():
@@ -140,7 +185,9 @@ class ResumableFileDownloader:
                 if progress_callback and expected_size and expected_size > 0:
                     await progress_callback(expected_size, expected_size)
                 return final_path
-            response.raise_for_status()
+            if response.status_code >= 400:
+                body = (await response.aread()).decode("utf-8", errors="replace")
+                raise DownloadHTTPStatusError(url=url, status_code=response.status_code, body=body)
 
             if response.status_code == 206 and resume_from > 0:
                 mode = "ab"

backend/app/services/collectors/telegeography.py

@@ -163,32 +163,64 @@ class TeleGeographyLandingPointCollector(BaseCollector):
     data_type = "landing_point"
 
     async def fetch(self) -> List[Dict[str, Any]]:
-        """Fetch landing point data from GitHub mirror"""
-        url = self._resolved_url or ""
+        """Fetch landing point data, falling back when the old mirror disappears."""
+        config = get_data_sources_config()
+        sources = [
+            self._resolved_url or "",
+            str(config.get_yaml_value("telegeography.landing_point_url") or ""),
+            str(config.get_yaml_value("arcgis.landing_point_url") or ""),
+        ]
 
-        async with httpx.AsyncClient(timeout=60.0) as client:
-            response = await client.get(url)
-            response.raise_for_status()
-            return self.parse_response(response.json())
+        last_error: Exception | None = None
+        async with httpx.AsyncClient(timeout=60.0, follow_redirects=True) as client:
+            for url in dict.fromkeys(source for source in sources if source):
+                try:
+                    params = (
+                        {"where": "1=1", "outFields": "*", "returnGeometry": "true", "f": "geojson"}
+                        if "FeatureServer" in url or url.endswith("/query")
+                        else None
+                    )
+                    response = await client.get(url, params=params)
+                    response.raise_for_status()
+                    records = self.parse_response(response.json())
+                    if records:
+                        return records
+                except Exception as exc:
+                    last_error = exc
+                    continue
 
-    def parse_response(self, data: List[Dict[str, Any]]) -> List[Dict[str, Any]]:
+        if last_error:
+            raise last_error
+        return self._get_sample_data()
+
+    def parse_response(self, data: List[Dict[str, Any]] | Dict[str, Any]) -> List[Dict[str, Any]]:
         """Parse landing point data"""
         result = []
+        items = data.get("features", []) if isinstance(data, dict) else data
 
-        for item in data:
+        for item in items:
+            props = item.get("properties", {}) if isinstance(item, dict) else {}
+            geometry = item.get("geometry", {}) if isinstance(item, dict) else {}
+            source = props or item
+            coords = geometry.get("coordinates", []) if isinstance(geometry, dict) else []
+            longitude = coords[0] if len(coords) > 0 else source.get("longitude")
+            latitude = coords[1] if len(coords) > 1 else source.get("latitude")
+            source_id = source.get("id") or source.get("OBJECTID") or source.get("city_id") or ""
             try:
                 entry = {
-                    "source_id": f"telegeo_lp_{item.get('id', '')}",
-                    "name": item.get("name", "Unknown"),
-                    "country": item.get("country", "Unknown"),
-                    "city": item.get("city", item.get("name", "")),
-                    "latitude": str(item.get("latitude", "")),
-                    "longitude": str(item.get("longitude", "")),
+                    "source_id": f"telegeo_lp_{source_id}",
+                    "name": source.get("name", source.get("Name", "Unknown")),
+                    "country": source.get("country", "Unknown"),
+                    "city": source.get("city", source.get("Name", source.get("name", ""))),
+                    "latitude": str(latitude or ""),
+                    "longitude": str(longitude or ""),
                     "value": "",
                     "unit": "",
                     "metadata": {
-                        "cable_count": len(item.get("cables", [])),
-                        "url": item.get("url"),
+                        "cable_count": len(source.get("cables", [])),
+                        "url": source.get("url"),
+                        "objectid": source.get("OBJECTID"),
+                        "city_id": source.get("city_id"),
                     },
                     "reference_date": datetime.now(UTC).strftime("%Y-%m-%d"),
                 }

backend/app/services/compute_center_locations.py

@@ -11,7 +11,7 @@ startup, so it must stay local and deterministic.
 
 For the full design and the reason behind the abstraction (compute centers,
 BGP collectors, BGP events, and future entities all share one pipeline),
-see ``docs/plans/location-resolver-shared-pipeline-plan.md``.
+see ``docs/technical/zh/location-pipeline-development.md``.
 
 The ``ComputeCenterLocation`` dataclass and the public function signatures are
 preserved verbatim so existing callers and tests do not need to change.

backend/app/services/credential_guides.py

@@ -3,9 +3,11 @@
 from __future__ import annotations
 
 from dataclasses import dataclass
+from copy import deepcopy
 from typing import Any
 
 from sqlalchemy import select
+from sqlalchemy.orm.attributes import flag_modified
 
 from app.ai_tasks.prompts import get_effective_prompt
 from app.models.system_setting import SystemSetting
@@ -135,33 +137,57 @@ DEFAULT_CREDENTIAL_GUIDES = {
 }
 
 
+def _normalize_provider(provider: str) -> str:
+    return provider.strip().lower().replace(" ", "_")
+
+
+def _credential_guide_default(provider: str) -> CredentialGuideDefault:
+    normalized = _normalize_provider(provider)
+    known = DEFAULT_CREDENTIAL_GUIDES.get(normalized)
+    if known is not None:
+        return known
+    title = f"{normalized or 'collector'} 凭证配置教程"
+    return CredentialGuideDefault(
+        provider=normalized,
+        title=title,
+        prompt=(
+            f"请生成一份中文教程，指导开发者为 Planet 采集器配置 {normalized} 凭证。"
+            "教程要面向已经有本地开发环境的人，包含官方入口或文档查找方式、"
+            "获取 API Key / Token / Client credentials 的通用步骤、在 Planet 采集器配置中"
+            "填写凭证字段、连接测试、保存、常见失败排查。不要编造具体页面按钮文案；"
+            "如果公开资料不足，必须明确提醒以 provider 官方文档和当前控制台页面为准。"
+        ),
+        markdown="",
+    )
+
+
 async def _get_guide_store(db) -> tuple[SystemSetting | None, dict[str, Any]]:
     result = await db.execute(
         select(SystemSetting).where(SystemSetting.category == CREDENTIAL_GUIDES_CATEGORY)
     )
     record = result.scalar_one_or_none()
-    payload = dict(record.payload or {}) if record and isinstance(record.payload, dict) else {}
+    payload = deepcopy(record.payload) if record and isinstance(record.payload, dict) else {}
     return record, payload
 
 
 async def get_credential_guide(db, provider: str) -> dict[str, Any]:
-    default = DEFAULT_CREDENTIAL_GUIDES.get(provider)
-    if default is None:
-        raise ValueError(f"Unsupported credential guide provider: {provider}")
+    provider = _normalize_provider(provider)
+    default = _credential_guide_default(provider)
 
     _record, store = await _get_guide_store(db)
     custom = store.get(provider) if isinstance(store.get(provider), dict) else None
+    has_default_markdown = bool(default.markdown.strip())
     return {
         "provider": provider,
         "title": custom.get("title") if custom else default.title,
         "markdown": custom.get("markdown") if custom else default.markdown,
         "prompt": default.prompt,
-        "source": "ai" if custom else "default",
+        "source": "ai" if custom else "default" if has_default_markdown else "missing",
         "sources": custom.get("sources", []) if custom else [],
         "verification_status": (
             custom.get("verification_status", "verified_with_search_evidence")
             if custom
-            else "default_unverified"
+            else "default_unverified" if has_default_markdown else "missing"
         ),
         "verification_error": custom.get("verification_error") if custom else None,
     }
@@ -177,9 +203,8 @@ async def save_credential_guide(
     verification_status: str = "verified_with_search_evidence",
     verification_error: str | None = None,
 ) -> dict[str, Any]:
-    default = DEFAULT_CREDENTIAL_GUIDES.get(provider)
-    if default is None:
-        raise ValueError(f"Unsupported credential guide provider: {provider}")
+    provider = _normalize_provider(provider)
+    default = _credential_guide_default(provider)
 
     record, store = await _get_guide_store(db)
     store[provider] = {
@@ -192,21 +217,21 @@ async def save_credential_guide(
     if record is None:
         db.add(SystemSetting(category=CREDENTIAL_GUIDES_CATEGORY, payload=store))
     else:
-        record.payload = store
+        record.payload = deepcopy(store)
+        flag_modified(record, "payload")
     await db.commit()
     return await get_credential_guide(db, provider)
 
 
 async def reset_credential_guide(db, provider: str) -> dict[str, Any]:
-    default = DEFAULT_CREDENTIAL_GUIDES.get(provider)
-    if default is None:
-        raise ValueError(f"Unsupported credential guide provider: {provider}")
+    provider = _normalize_provider(provider)
 
     record, store = await _get_guide_store(db)
     if provider in store:
         store.pop(provider, None)
         if record is not None:
-            record.payload = store
+            record.payload = deepcopy(store)
+            flag_modified(record, "payload")
             await db.commit()
     return await get_credential_guide(db, provider)
 
@@ -217,9 +242,8 @@ async def generate_credential_guide(
     ai_client: AIProviderClient,
     web_search_client: WebSearchClient | None = None,
 ) -> dict[str, Any]:
-    default = DEFAULT_CREDENTIAL_GUIDES.get(provider)
-    if default is None:
-        raise ValueError(f"Unsupported credential guide provider: {provider}")
+    provider = _normalize_provider(provider)
+    default = _credential_guide_default(provider)
 
     search_evidence: list[dict[str, Any]] = []
     search_error: str | None = None

backend/app/services/data_jobs.py

@@ -0,0 +1,597 @@
+"""Kafka-ready datasource job queue backed by PostgreSQL for v1."""
+
+from __future__ import annotations
+
+import asyncio
+from datetime import UTC, datetime, timedelta
+from uuid import uuid4
+from typing import Any
+
+from sqlalchemy import bindparam, select, text
+from sqlalchemy.ext.asyncio import AsyncSession
+
+from app.core.cache import cache
+from app.core.config import settings
+from app.core.logging import get_logger
+from app.core.time import to_iso8601_utc
+from app.core.websocket.broadcaster import broadcaster
+from app.db.session import async_session_factory
+from app.models.collected_data import CollectedData
+from app.models.data_snapshot import DataSnapshot
+from app.models.datasource import DataSource
+from app.models.task import CollectionTask
+from app.services.collectors.registry import collector_registry
+from app.services.datasource_connectivity import (
+    build_builtin_connectivity_checksum,
+    get_builtin_effective_candidate,
+    save_connectivity_success,
+)
+from app.services.earth_layer_adapters import (
+    clear_derived_datasource_data,
+    get_earth_refresh_strategy_for_change,
+    get_earth_update_layers_for_source,
+)
+from app.services.earth_layer_cache import invalidate_earth_layer_cache_for_source
+from app.services.scheduler import sync_datasource_job
+
+logger = get_logger(__name__)
+
+JOB_TYPE_COLLECT = "collect"
+JOB_TYPE_CLEAR_DATA = "clear_data"
+JOB_TYPE_CLEAR_CACHE = "clear_cache"
+JOB_TYPE_EARTH_REFRESH = "earth_refresh"
+
+JOB_STATUS_QUEUED = "queued"
+JOB_STATUS_RUNNING = "running"
+JOB_STATUS_CANCELLING = "cancelling"
+JOB_STATUS_SUCCESS = "success"
+JOB_STATUS_FAILED = "failed"
+JOB_STATUS_CANCELLED = "cancelled"
+
+ACTIVE_JOB_STATUSES = (JOB_STATUS_QUEUED, JOB_STATUS_RUNNING, JOB_STATUS_CANCELLING)
+TERMINAL_JOB_STATUSES = (JOB_STATUS_SUCCESS, JOB_STATUS_FAILED, JOB_STATUS_CANCELLED)
+DATA_WRITE_JOB_TYPES = (JOB_TYPE_COLLECT, JOB_TYPE_CLEAR_DATA, JOB_TYPE_CLEAR_CACHE)
+SOURCE_LOCK_JOB_STATUSES = (JOB_STATUS_RUNNING, JOB_STATUS_CANCELLING)
+QUEUE_POLL_SECONDS = 0.35
+JOB_STALE_LOCK_MINUTES = 90
+DEFAULT_WORKER_CONCURRENCY = 2
+
+RUNNING_DATA_JOB_TASKS: dict[int, asyncio.Task[Any]] = {}
+
+
+def _utcnow() -> datetime:
+    return datetime.now(UTC)
+
+
+def _job_worker_id() -> str:
+    return f"{settings.PROJECT_NAME}:data-job-worker:{uuid4().hex[:8]}"
+
+
+def is_terminal_job_status(status: str | None) -> bool:
+    return status in TERMINAL_JOB_STATUSES
+
+
+async def enqueue_datasource_job(
+    db: AsyncSession,
+    datasource: DataSource,
+    task_type: str,
+    *,
+    payload: dict[str, Any] | None = None,
+    rollback_policy: str = "keep_committed_batches",
+    dedupe_key: str | None = None,
+) -> CollectionTask:
+    if dedupe_key:
+        existing = await _get_active_job_by_dedupe_key(db, dedupe_key)
+        if existing is not None:
+            return existing
+
+    task = CollectionTask(
+        datasource_id=datasource.id,
+        source=datasource.source,
+        task_type=task_type,
+        status=JOB_STATUS_QUEUED,
+        phase="queued",
+        phase_message="任务已进入队列",
+        payload=payload or {},
+        rollback_policy=rollback_policy,
+        dedupe_key=dedupe_key,
+    )
+    db.add(task)
+    await db.commit()
+    await db.refresh(task)
+    await _broadcast_task_update(task)
+    return task
+
+
+async def enqueue_earth_refresh_job(
+    db: AsyncSession,
+    *,
+    source: str,
+    payload: dict[str, Any] | None = None,
+) -> CollectionTask | None:
+    layers = list((payload or {}).get("layers") or get_earth_update_layers_for_source(source))
+    if not layers:
+        return None
+
+    datasource = await _get_or_create_virtual_datasource(db, source)
+    refresh_payload = {
+        "source": source,
+        "layers": layers,
+        "refresh_strategy": (payload or {}).get("refresh_strategy")
+        or get_earth_refresh_strategy_for_change((payload or {}).get("table"), source)
+        or "clear_then_reload",
+        **(payload or {}),
+    }
+    return await enqueue_datasource_job(
+        db,
+        datasource,
+        JOB_TYPE_EARTH_REFRESH,
+        payload=refresh_payload,
+        dedupe_key=f"earth_refresh:{source}",
+    )
+
+
+async def enqueue_earth_refresh_from_update(payload: dict[str, Any]) -> None:
+    source = str(payload.get("source") or "").strip()
+    if not source:
+        return
+    async with async_session_factory() as db:
+        await enqueue_earth_refresh_job(db, source=source, payload=payload)
+
+
+async def request_cancel_datasource_task(
+    db: AsyncSession,
+    task: CollectionTask,
+    *,
+    reason: str = "cancelled_by_operator",
+) -> CollectionTask:
+    if is_terminal_job_status(task.status):
+        return task
+
+    running_task = RUNNING_DATA_JOB_TASKS.get(task.id)
+    if task.status == JOB_STATUS_QUEUED or (
+        running_task is None
+        and (
+            task.status == JOB_STATUS_CANCELLING
+            or (task.status == JOB_STATUS_RUNNING and task.task_type != JOB_TYPE_COLLECT)
+        )
+    ):
+        return await _cancel_task_without_runner(db, task, reason=reason)
+
+    task.status = JOB_STATUS_CANCELLING
+    task.phase = JOB_STATUS_CANCELLING
+    task.phase_message = "正在停止任务"
+    task.requested_cancel_at = _utcnow()
+    task.cancel_reason = reason
+    await db.commit()
+    await db.refresh(task)
+
+    if running_task is not None and not running_task.done():
+        running_task.cancel()
+
+    await _broadcast_task_update(task)
+    return task
+
+
+async def _cancel_task_without_runner(
+    db: AsyncSession,
+    task: CollectionTask,
+    *,
+    reason: str,
+) -> CollectionTask:
+    if task.task_type == JOB_TYPE_COLLECT:
+        await db.execute(CollectedData.__table__.delete().where(CollectedData.task_id == task.id))
+        snapshot_result = await db.execute(select(DataSnapshot).where(DataSnapshot.task_id == task.id))
+        for snapshot in snapshot_result.scalars().all():
+            snapshot.status = JOB_STATUS_CANCELLED
+            snapshot.completed_at = _utcnow()
+            snapshot.error_message = "Cancelled after operator stop request; no active worker handle remained"
+        datasource = await db.get(DataSource, task.datasource_id)
+        if datasource is not None:
+            datasource.last_status = JOB_STATUS_CANCELLED
+
+    task.status = JOB_STATUS_CANCELLED
+    task.phase = JOB_STATUS_CANCELLED
+    task.phase_message = "任务已停止"
+    task.completed_at = _utcnow()
+    task.requested_cancel_at = task.requested_cancel_at or _utcnow()
+    task.cancel_reason = reason
+    await db.commit()
+    await db.refresh(task)
+    await _broadcast_task_update(task)
+    return task
+
+
+async def get_active_datasource_job(
+    db: AsyncSession,
+    datasource_id: int,
+    *,
+    task_types: tuple[str, ...] = DATA_WRITE_JOB_TYPES,
+) -> CollectionTask | None:
+    result = await db.execute(
+        select(CollectionTask)
+        .where(CollectionTask.datasource_id == datasource_id)
+        .where(CollectionTask.task_type.in_(task_types))
+        .where(CollectionTask.status.in_(ACTIVE_JOB_STATUSES))
+        .order_by(CollectionTask.created_at.desc().nullslast(), CollectionTask.id.desc())
+        .limit(1)
+    )
+    return result.scalar_one_or_none()
+
+
+async def _get_active_job_by_dedupe_key(db: AsyncSession, dedupe_key: str) -> CollectionTask | None:
+    result = await db.execute(
+        select(CollectionTask)
+        .where(CollectionTask.dedupe_key == dedupe_key)
+        .where(CollectionTask.status.in_(ACTIVE_JOB_STATUSES))
+        .order_by(CollectionTask.created_at.desc().nullslast(), CollectionTask.id.desc())
+        .limit(1)
+    )
+    return result.scalar_one_or_none()
+
+
+async def _get_or_create_virtual_datasource(db: AsyncSession, source: str) -> DataSource:
+    result = await db.execute(select(DataSource).where(DataSource.source == source))
+    datasource = result.scalar_one_or_none()
+    if datasource is not None:
+        return datasource
+
+    datasource = DataSource(
+        name=f"Earth refresh: {source}",
+        source=source,
+        module="SYS",
+        collector_class="EarthRefreshJob",
+        is_active=True,
+    )
+    db.add(datasource)
+    await db.commit()
+    await db.refresh(datasource)
+    return datasource
+
+
+async def _broadcast_task_update(task: CollectionTask) -> None:
+    await broadcaster.broadcast_datasource_task_update(
+        {
+            "datasource_id": task.datasource_id,
+            "collector_name": task.source,
+            "task_id": task.id,
+            "task_type": task.task_type,
+            "status": task.status,
+            "phase": task.phase,
+            "phase_progress": task.phase_progress,
+            "phase_message": task.phase_message,
+            "phase_current": task.phase_current,
+            "phase_total": task.phase_total,
+            "phase_unit": task.phase_unit,
+            "progress": task.progress,
+            "records_processed": task.records_processed,
+            "total_records": task.total_records,
+            "started_at": to_iso8601_utc(task.started_at),
+            "completed_at": to_iso8601_utc(task.completed_at),
+            "requested_cancel_at": to_iso8601_utc(task.requested_cancel_at),
+            "error_message": task.error_message,
+        }
+    )
+
+
+class DataJobWorker:
+    def __init__(self, *, concurrency: int = DEFAULT_WORKER_CONCURRENCY) -> None:
+        self.worker_id = _job_worker_id()
+        self.concurrency = max(1, concurrency)
+        self._task: asyncio.Task[None] | None = None
+        self._stop_event: asyncio.Event | None = None
+        self._running: set[asyncio.Task[Any]] = set()
+
+    def start(self) -> None:
+        if self._task and not self._task.done():
+            return
+        self._stop_event = asyncio.Event()
+        self._task = asyncio.create_task(self._run(), name="data-job-worker")
+
+    async def stop(self) -> None:
+        if self._stop_event:
+            self._stop_event.set()
+        for task in list(self._running):
+            task.cancel()
+        if self._task:
+            await asyncio.gather(self._task, return_exceptions=True)
+        if self._running:
+            await asyncio.gather(*self._running, return_exceptions=True)
+
+    async def _run(self) -> None:
+        assert self._stop_event is not None
+        await self._recover_stale_running_jobs()
+        while not self._stop_event.is_set():
+            self._running = {task for task in self._running if not task.done()}
+            if len(self._running) >= self.concurrency:
+                await asyncio.sleep(QUEUE_POLL_SECONDS)
+                continue
+
+            task_id = await self._claim_next_job()
+            if task_id is None:
+                await asyncio.sleep(QUEUE_POLL_SECONDS)
+                continue
+
+            runner = asyncio.create_task(self._run_claimed_job(task_id), name=f"data-job:{task_id}")
+            self._running.add(runner)
+
+    async def _recover_stale_running_jobs(self) -> None:
+        cutoff = _utcnow() - timedelta(minutes=JOB_STALE_LOCK_MINUTES)
+        async with async_session_factory() as db:
+            result = await db.execute(
+                select(CollectionTask)
+                .where(CollectionTask.status.in_((JOB_STATUS_RUNNING, JOB_STATUS_CANCELLING)))
+                .where(CollectionTask.locked_at.is_not(None))
+                .where(CollectionTask.locked_at < cutoff)
+            )
+            stale_jobs = list(result.scalars().all())
+            for job in stale_jobs:
+                job.status = JOB_STATUS_FAILED
+                job.phase = JOB_STATUS_FAILED
+                job.completed_at = _utcnow()
+                job.error_message = "Marked failed after stale data job lock timeout"
+            if stale_jobs:
+                await db.commit()
+
+    async def _claim_next_job(self) -> int | None:
+        async with async_session_factory() as db:
+            row = await db.execute(
+                text(
+                    """
+                    SELECT queued.id
+                    FROM collection_tasks AS queued
+                    WHERE queued.status = :queued_status
+                      AND NOT EXISTS (
+                        SELECT 1
+                        FROM collection_tasks AS active
+                        WHERE active.source = queued.source
+                          AND active.id <> queued.id
+                          AND active.status IN :active_statuses
+                      )
+                    ORDER BY queued.created_at ASC NULLS FIRST, queued.id ASC
+                    LIMIT 1
+                    FOR UPDATE SKIP LOCKED
+                    """
+                ).bindparams(bindparam("active_statuses", expanding=True)),
+                {
+                    "queued_status": JOB_STATUS_QUEUED,
+                    "active_statuses": SOURCE_LOCK_JOB_STATUSES,
+                },
+            )
+            task_id = row.scalar_one_or_none()
+            if task_id is None:
+                return None
+
+            task = await db.get(CollectionTask, int(task_id))
+            if task is None:
+                return None
+            task.status = JOB_STATUS_RUNNING
+            task.phase = "starting"
+            task.phase_message = "任务开始执行"
+            task.started_at = task.started_at or _utcnow()
+            task.worker_id = self.worker_id
+            task.locked_at = _utcnow()
+            await db.commit()
+            await _broadcast_task_update(task)
+            return int(task_id)
+
+    async def _run_claimed_job(self, task_id: int) -> None:
+        current_task = asyncio.current_task()
+        if current_task is not None:
+            RUNNING_DATA_JOB_TASKS[task_id] = current_task
+        try:
+            async with async_session_factory() as db:
+                task = await db.get(CollectionTask, task_id)
+                if task is None:
+                    return
+                await self._execute_job(db, task)
+        except asyncio.CancelledError:
+            async with async_session_factory() as db:
+                task = await db.get(CollectionTask, task_id)
+                if task is not None and not is_terminal_job_status(task.status):
+                    task.status = JOB_STATUS_CANCELLED
+                    task.phase = JOB_STATUS_CANCELLED
+                    task.phase_message = "任务已停止"
+                    task.completed_at = _utcnow()
+                    await db.commit()
+                    await _broadcast_task_update(task)
+            raise
+        except Exception as exc:
+            logger.exception_event(
+                "Data job failed",
+                event="data_jobs.job_failed",
+                context={"task_id": task_id, "error": str(exc)},
+            )
+            async with async_session_factory() as db:
+                task = await db.get(CollectionTask, task_id)
+                if task is not None:
+                    task.status = JOB_STATUS_FAILED
+                    task.phase = JOB_STATUS_FAILED
+                    task.phase_message = str(exc)
+                    task.error_message = str(exc)
+                    task.completed_at = _utcnow()
+                    await db.commit()
+                    await _broadcast_task_update(task)
+        finally:
+            RUNNING_DATA_JOB_TASKS.pop(task_id, None)
+
+    async def _execute_job(self, db: AsyncSession, task: CollectionTask) -> None:
+        if task.task_type == JOB_TYPE_COLLECT:
+            await _run_collect_job(db, task)
+        elif task.task_type == JOB_TYPE_CLEAR_DATA:
+            await _run_clear_data_job(db, task)
+        elif task.task_type == JOB_TYPE_CLEAR_CACHE:
+            await _run_clear_cache_job(db, task)
+        elif task.task_type == JOB_TYPE_EARTH_REFRESH:
+            await _run_earth_refresh_job(db, task)
+        else:
+            raise RuntimeError(f"Unsupported data job type: {task.task_type}")
+
+
+async def _run_collect_job(db: AsyncSession, task: CollectionTask) -> None:
+    datasource = await db.get(DataSource, task.datasource_id)
+    if datasource is None:
+        raise RuntimeError("Data source not found")
+
+    collector = collector_registry.get(datasource.source)
+    if collector is None:
+        raise RuntimeError(f"Collector '{datasource.source}' not found")
+    if not datasource.is_active:
+        raise RuntimeError("Data source is disabled")
+
+    collector._datasource_id = datasource.id
+    collector._current_task = task
+    collector._db_session = db
+    result = await collector.run(db)
+
+    datasource.last_run_at = _utcnow()
+    datasource.last_status = result.get("status")
+    if datasource.last_status == JOB_STATUS_SUCCESS:
+        effective_candidate = await get_builtin_effective_candidate(db, datasource.source)
+        checksum, _credential_context = await build_builtin_connectivity_checksum(
+            datasource.source,
+            effective_candidate["endpoint"],
+            effective_candidate["auth_type"],
+            effective_candidate["headers"],
+            effective_candidate["config"],
+            db,
+        )
+        await save_connectivity_success(
+            db,
+            datasource.source,
+            checksum,
+            {"status_code": None},
+            connected_by="collection",
+        )
+    await db.commit()
+    await sync_datasource_job(datasource.id)
+
+
+async def _run_clear_data_job(db: AsyncSession, task: CollectionTask) -> None:
+    source = str(task.source or (task.payload or {}).get("source") or "").strip()
+    if not source:
+        raise RuntimeError("Clear data job has no source")
+
+    task.phase = "clearing_data"
+    task.phase_message = "正在删除数据库数据"
+    await db.commit()
+    await _broadcast_task_update(task)
+
+    count_result = await db.execute(
+        select(CollectedData.id).where(CollectedData.source == source)
+    )
+    collected_ids = [row[0] for row in count_result.all()]
+    derived_deleted_counts = await clear_derived_datasource_data(db, source)
+    if collected_ids:
+        await db.execute(CollectedData.__table__.delete().where(CollectedData.id.in_(collected_ids)))
+    deleted_count = len(collected_ids)
+    derived_deleted_count = sum(derived_deleted_counts.values())
+
+    task.records_processed = deleted_count + derived_deleted_count
+    task.total_records = task.records_processed
+    task.progress = 100.0
+    task.phase_progress = 100.0
+    task.phase_current = task.records_processed
+    task.phase_total = task.records_processed
+    task.phase_unit = "records"
+    task.payload = {
+        **(task.payload or {}),
+        "deleted_count": deleted_count,
+        "derived_deleted_count": derived_deleted_count,
+        "derived_deleted_counts": derived_deleted_counts,
+    }
+    task.status = JOB_STATUS_SUCCESS
+    task.phase = "completed"
+    task.phase_message = "数据库数据已清理"
+    task.completed_at = _utcnow()
+    await db.commit()
+    await _broadcast_task_update(task)
+
+
+async def _run_clear_cache_job(db: AsyncSession, task: CollectionTask) -> None:
+    source = str(task.source or (task.payload or {}).get("source") or "").strip()
+    if not source:
+        raise RuntimeError("Clear cache job has no source")
+
+    earth_deleted_count = invalidate_earth_layer_cache_for_source(source)
+    dashboard_deleted_count = int(cache.delete("dashboard:stats")) + int(cache.delete("dashboard:summary"))
+    deleted_count = earth_deleted_count + dashboard_deleted_count
+
+    task.records_processed = deleted_count
+    task.total_records = deleted_count
+    task.progress = 100.0
+    task.phase_progress = 100.0
+    task.phase = "completed"
+    task.phase_message = "缓存已清理"
+    task.payload = {
+        **(task.payload or {}),
+        "earth_layer_deleted_count": earth_deleted_count,
+        "dashboard_deleted_count": dashboard_deleted_count,
+    }
+    task.status = JOB_STATUS_SUCCESS
+    task.completed_at = _utcnow()
+    await db.commit()
+    await _broadcast_task_update(task)
+    await enqueue_earth_refresh_job(db, source=source, payload={"operation": "CACHE_INVALIDATED"})
+
+
+async def _run_earth_refresh_job(db: AsyncSession, task: CollectionTask) -> None:
+    payload = task.payload or {}
+    source = str(payload.get("source") or task.source or "").strip()
+    layers = list(payload.get("layers") or get_earth_update_layers_for_source(source))
+    if not source or not layers:
+        task.status = JOB_STATUS_SUCCESS
+        task.phase = "completed"
+        task.phase_message = "没有需要刷新的 Earth 图层"
+        task.completed_at = _utcnow()
+        await db.commit()
+        await _broadcast_task_update(task)
+        return
+
+    deleted_cache_entries = invalidate_earth_layer_cache_for_source(source)
+    update_payload = {
+        "event": "earth.layer.changed",
+        "action": "database_changed",
+        "source": source,
+        "table": payload.get("table"),
+        "data_type": source,
+        "layers": layers,
+        "refresh_strategy": payload.get("refresh_strategy") or "clear_then_reload",
+        "records_processed": payload.get("records_processed", 0),
+        "operations": payload.get("operations") or [payload.get("operation") or "CHANGE"],
+        "operation": payload.get("operation"),
+        "cache_entries_invalidated": deleted_cache_entries,
+        "timestamp": to_iso8601_utc(_utcnow()),
+    }
+    if payload.get("entity") == "interactable":
+        update_payload.update(
+            {
+                "entity": "interactable",
+                "action": payload.get("action") or "changed",
+                "ids": payload.get("ids") or payload.get("entity_keys") or [],
+                "item": payload.get("item"),
+            }
+        )
+    await broadcaster.broadcast_earth_update(update_payload)
+
+    task.records_processed = int(payload.get("records_processed") or 0)
+    task.progress = 100.0
+    task.phase_progress = 100.0
+    task.phase = "completed"
+    task.phase_message = "Earth 图层刷新通知已发送"
+    task.status = JOB_STATUS_SUCCESS
+    task.completed_at = _utcnow()
+    task.payload = {**payload, "cache_entries_invalidated": deleted_cache_entries}
+    await db.commit()
+    await _broadcast_task_update(task)
+
+
+_worker = DataJobWorker()
+
+
+def start_data_job_worker() -> None:
+    _worker.start()
+
+
+async def stop_data_job_worker() -> None:
+    await _worker.stop()

backend/app/services/datasource_connectivity.py

@@ -18,6 +18,7 @@ from app.models.datasource import DataSource
 from app.models.datasource_config import DataSourceConfig
 from app.models.system_setting import SystemSetting
 from app.services.barentswatch import (
+    BarentsWatchConfig,
     _read_zshrc_env,
     fetch_barentswatch_access_token,
     resolve_barentswatch_config,
@@ -44,6 +45,20 @@ def _resolve_spacetrack_credentials() -> tuple[str, str, str]:
     return username, password, source or "missing"
 
 
+def _resolve_spacetrack_credentials_with_override(
+    credential_override: dict[str, str] | None = None,
+) -> tuple[str, str, str]:
+    if credential_override and (
+        credential_override.get("username") or credential_override.get("password")
+    ):
+        return (
+            str(credential_override.get("username") or ""),
+            str(credential_override.get("password") or ""),
+            "draft",
+        )
+    return _resolve_spacetrack_credentials()
+
+
 async def _resolve_aisstream_api_key(
     db=None,
     credential_override: dict[str, str] | None = None,
@@ -126,7 +141,9 @@ async def build_builtin_connectivity_checksum(
             }
         )
     elif credential_provider == "spacetrack":
-        username, password, credential_source = _resolve_spacetrack_credentials()
+        username, password, credential_source = _resolve_spacetrack_credentials_with_override(
+            credential_override
+        )
         has_credentials = bool(username and password)
         credential_fingerprint = _sha256_json(
             {
@@ -230,7 +247,16 @@ async def test_builtin_connectivity(
     try:
         async with httpx.AsyncClient(timeout=timeout, follow_redirects=True) as client:
             if credential_context["credential_provider"] == "barentswatch":
-                barentswatch_config = await resolve_barentswatch_config(db)
+                if credential_override:
+                    barentswatch_config = BarentsWatchConfig(
+                        endpoint=str(request_endpoint or ""),
+                        client_id=str(credential_override.get("client_id") or ""),
+                        client_secret=str(credential_override.get("client_secret") or ""),
+                        credential_source="draft",
+                        endpoint_source="draft",
+                    )
+                else:
+                    barentswatch_config = await resolve_barentswatch_config(db)
                 token = await fetch_barentswatch_access_token(client, barentswatch_config)
                 if not token:
                     return {
@@ -243,7 +269,9 @@ async def test_builtin_connectivity(
                     }
                 request_headers["Authorization"] = f"Bearer {token}"
             elif credential_context["credential_provider"] == "spacetrack":
-                username, password, _source = _resolve_spacetrack_credentials()
+                username, password, _source = _resolve_spacetrack_credentials_with_override(
+                    credential_override
+                )
                 login_url = "https://www.space-track.org/ajaxauth/login"
                 login_response = await client.post(
                     login_url,

backend/app/services/docs_gatekeeper.py

@@ -32,28 +32,31 @@ class DocsMetadata:
 
 DOCS_METADATA: tuple[DocsMetadata, ...] = (
     DocsMetadata(DOCS_README_FILENAME, DEFAULT_DOCS_SLUG, "public", "Overview", 0, "技术文档", "Technical Docs"),
-    DocsMetadata("quickstart.md", "quickstart", "public", "Manual", 1, "快速开始", "Quickstart"),
-    DocsMetadata("manual.md", "manual", "public", "Manual", 2, "Planet 使用手册", "Planet Manual"),
+    DocsMetadata("manual.md", "manual", "public", "Manual", 1, "智能星球使用手册", "Intelligent Planet Manual"),
+    DocsMetadata("quickstart.md", "quickstart", "public", "Manual", 2, "快速开始", "Quickstart"),
     DocsMetadata("faq.md", "faq", "public", "Manual", 3, "常见问题", "FAQ"),
-    DocsMetadata("location-pipeline-user.md", "location-pipeline-user", "public", "Manual", 4, "Earth 位置候选采集使用手册", "Earth Location Candidate Collection User Guide"),
-    DocsMetadata("earth-frontend-context.md", "earth-frontend-context", "docs_developer", "Earth", 10, "Earth 前端结构", "Earth Frontend Context"),
-    DocsMetadata("earth-layer-style-reference.md", "earth-layer-style-reference", "docs_developer", "Earth", 11, "Earth 图层样式属性索引", "Earth Layer Style Reference"),
-    DocsMetadata("earth-render-layer-order.md", "earth-render-layer-order", "docs_developer", "Earth", 12, "Earth 渲染图层顺序", "Earth Render Layer Order"),
-    DocsMetadata("earth-satellite-footprint-policy.md", "earth-satellite-footprint-policy", "docs_developer", "Earth", 13, "Earth 卫星覆盖策略", "Earth Satellite Footprint Policy"),
+    DocsMetadata("platform-data-flows.md", "platform-data-flows", "docs_developer", "Architecture", 5, "业务架构与数据流转", "Business Architecture and Data Flows"),
+    DocsMetadata("naming-glossary.md", "naming-glossary", "docs_developer", "Architecture", 6, "命名与术语对照", "Naming Glossary"),
+    DocsMetadata("earth-frontend-context.md", "earth-frontend-context", "docs_developer", "Earth", 10, "智能星球前端结构", "Intelligent Planet Frontend Context"),
+    DocsMetadata("earth-layer-style-reference.md", "earth-layer-style-reference", "docs_developer", "Earth", 11, "智能星球图层样式属性索引", "Intelligent Planet Layer Style Reference"),
+    DocsMetadata("earth-render-layer-order.md", "earth-render-layer-order", "docs_developer", "Earth", 12, "智能星球渲染图层顺序", "Intelligent Planet Render Layer Order"),
+    DocsMetadata("earth-satellite-footprint-policy.md", "earth-satellite-footprint-policy", "docs_developer", "Earth", 13, "智能星球卫星覆盖策略", "Intelligent Planet Satellite Footprint Policy"),
     DocsMetadata("earth-bgp-context.md", "earth-bgp-context", "docs_developer", "Earth", 14, "BGP 态势上下文", "BGP Context"),
-    DocsMetadata("earth-news-live-streams-collector-format.md", "earth-news-live-streams-collector-format", "docs_developer", "Earth", 15, "新闻直播采集格式", "News Live Streams Collector Format"),
-    DocsMetadata("earth-interactable-usage.md", "earth-interactable-usage", "docs_developer", "Earth", 16, "Earth 可交互图标接入", "Earth Interactable Usage"),
-    DocsMetadata("earth-toolbar-overlay-coordination.md", "earth-toolbar-overlay-coordination", "docs_developer", "Earth", 17, "Earth 工具栏与浮层协同", "Earth Toolbar and Overlay Coordination"),
+    DocsMetadata("earth-interactable-usage.md", "earth-interactable-usage", "docs_developer", "Earth", 16, "智能星球可交互图标接入", "Intelligent Planet Interactable Usage"),
+    DocsMetadata("earth-toolbar-overlay-coordination.md", "earth-toolbar-overlay-coordination", "docs_developer", "Earth", 17, "智能星球工具栏与浮层协同", "Intelligent Planet Toolbar and Overlay Coordination"),
     DocsMetadata("frontend-admin-frontend-context.md", "frontend-admin-frontend-context", "docs_developer", "Frontend", 20, "控制台前端结构", "Admin Frontend Context"),
     DocsMetadata("frontend-layout-guidelines.md", "frontend-layout-guidelines", "docs_developer", "Frontend", 21, "前端布局指南", "Frontend Layout Guidelines"),
-    DocsMetadata("docs-gatekeeper-development.md", "docs-gatekeeper-development", "docs_developer", "Frontend", 22, "Docs Gatekeeper 开发说明", "Docs Gatekeeper Development Guide"),
+    DocsMetadata("tactile-ui-components.md", "tactile-ui-components", "docs_developer", "Frontend", 24, "Tactile UI 组件库", "Tactile UI Components"),
     DocsMetadata("backend-collectors.md", "backend-collectors", "docs_developer", "Backend", 30, "数据采集系统", "Data Collectors"),
     DocsMetadata("backend-system-service-control.md", "backend-system-service-control", "docs_admin", "Backend", 31, "系统服务控制", "System Service Control"),
     DocsMetadata("datasource-collector-settings-connectivity.md", "datasource-collector-settings-connectivity", "docs_developer", "Backend", 32, "数据源、采集器设置与连接验证", "Datasource Collector Settings and Connectivity"),
     DocsMetadata("backend-datasources-api-performance.md", "backend-datasources-api-performance", "docs_developer", "Backend", 33, "数据源 API 性能", "Datasource API Performance"),
-    DocsMetadata("location-pipeline-development.md", "location-pipeline-development", "docs_developer", "Backend", 34, "通用位置估算管线开发说明", "Shared Location Resolution Pipeline Development Guide"),
+    DocsMetadata("data-job-earth-sync-architecture.md", "data-job-earth-sync-architecture", "docs_developer", "Backend", 34, "数据作业与 Outbox 技术架构", "Data Jobs and Outbox Architecture"),
+    DocsMetadata("location-pipeline-development.md", "location-pipeline-development", "docs_developer", "Backend", 35, "通用位置估算管线开发说明", "Shared Location Resolution Pipeline Development Guide"),
+    DocsMetadata("earth-news-live-streams-collector-format.md", "earth-news-live-streams-collector-format", "docs_developer", "Backend", 36, "新闻直播采集格式", "News Live Streams Collector Format"),
+    DocsMetadata("docs-gatekeeper-development.md", "docs-gatekeeper-development", "docs_developer", "Backend", 37, "Docs Gatekeeper 开发说明", "Docs Gatekeeper Development Guide"),
     DocsMetadata("agents-aiprovider.md", "agents-aiprovider", "docs_developer", "Agents", 40, "AI Provider 指南", "AI Provider Guide"),
-    DocsMetadata("ops-runbook.md", "ops-runbook", "docs_admin", "Ops", 49, "Planet 运维手册", "Planet Ops Runbook"),
+    DocsMetadata("ops-runbook.md", "ops-runbook", "docs_admin", "Ops", 49, "智能星球运维手册", "Intelligent Planet Ops Runbook"),
     DocsMetadata("ops-docker-compose-buildx-upgrade.md", "ops-docker-compose-buildx-upgrade", "docs_admin", "Ops", 50, "Docker + Compose + Buildx 升级", "Docker + Compose + Buildx Upgrade"),
     DocsMetadata("ops-planet-sh-startup.md", "ops-planet-sh-startup", "docs_admin", "Ops", 51, "planet.sh 启动机制", "planet.sh Startup"),
 )

backend/app/services/earth_db_change_listener.py

@@ -0,0 +1,482 @@
+"""PostgreSQL LISTEN/NOTIFY bridge for Earth layer refresh events."""
+
+from __future__ import annotations
+
+import asyncio
+import json
+from collections import deque
+from dataclasses import dataclass, field
+from datetime import UTC, datetime
+from time import monotonic
+from typing import Any, Awaitable, Callable
+
+import asyncpg
+
+from app.core.config import settings
+from app.core.logging import get_logger
+from app.core.time import to_iso8601_utc
+from app.core.websocket.broadcaster import broadcaster
+from app.services.earth_layer_adapters import (
+    get_earth_refresh_strategy_for_change,
+    get_earth_update_layers_for_change,
+    get_earth_update_layers_for_source,
+)
+from app.services.earth_layer_cache import invalidate_earth_layer_cache_for_source
+
+logger = get_logger(__name__)
+
+EARTH_DATA_CHANGES_CHANNEL = "planet_earth_data_changes"
+DEFAULT_DEBOUNCE_SECONDS = 0.25
+DEFAULT_MAX_WAIT_SECONDS = 1.5
+DELETE_FAST_FLUSH_SECONDS = 0.05
+LISTEN_KEEPALIVE_SECONDS = 5.0
+OUTBOX_POLL_LIMIT = 5000
+MAX_ENTITY_KEY_SAMPLES = 20
+MAX_SEEN_EVENT_IDS = 20000
+
+BroadcastFn = Callable[[dict[str, Any]], Awaitable[None]]
+InvalidateFn = Callable[[str], int]
+
+
+def normalize_asyncpg_dsn(dsn: str) -> str:
+    """Convert SQLAlchemy asyncpg URLs into asyncpg-compatible URLs."""
+    return dsn.replace("postgresql+asyncpg://", "postgresql://", 1)
+
+
+def build_earth_update_from_db_payload(payload: dict[str, Any]) -> dict[str, Any] | None:
+    table = payload.get("table")
+    source = str(payload.get("source") or "").strip()
+    table_name = str(table or "").strip()
+    if not source and not table_name:
+        return None
+    layers = get_earth_update_layers_for_change(table_name, source)
+    if not layers:
+        return None
+    refresh_strategy = get_earth_refresh_strategy_for_change(table_name, source) or "clear_then_reload"
+    source_has_adapter = bool(get_earth_update_layers_for_source(source))
+    effective_source = source if source_has_adapter else (table_name if table_name else source)
+    operation = payload.get("operation")
+    update: dict[str, Any] = {
+        "event": "earth.layer.changed",
+        "action": "database_changed",
+        "source": effective_source,
+        "original_source": source or None,
+        "table": table_name or None,
+        "data_type": effective_source,
+        "layers": layers,
+        "refresh_strategy": refresh_strategy,
+        "operation": operation,
+        "entity_key": payload.get("entity_key"),
+        "timestamp": to_iso8601_utc(datetime.now(UTC)),
+    }
+    if table_name == "earth_interactables" and refresh_strategy == "delta":
+        ids = payload.get("entity_keys")
+        if not isinstance(ids, list):
+            ids = [payload.get("entity_key")] if payload.get("entity_key") else []
+        update.update(
+            {
+                "entity": "interactable",
+                "action": "deleted" if operation == "DELETE" else "changed",
+                "ids": [str(item) for item in ids if item],
+                "item": None,
+            }
+        )
+    return {
+        **update,
+    }
+
+
+@dataclass
+class PendingEarthDbChange:
+    source: str
+    layers: list[str]
+    table: str | None = None
+    refresh_strategy: str = "clear_then_reload"
+    entity: str | None = None
+    action: str = "database_changed"
+    records_processed: int = 0
+    operations: set[str] = field(default_factory=set)
+    entity_keys: list[str] = field(default_factory=list)
+    first_occurred_at: str | None = None
+    last_occurred_at: str | None = None
+    first_seen_monotonic: float = field(default_factory=monotonic)
+    last_seen_monotonic: float = field(default_factory=monotonic)
+
+    def add(self, payload: dict[str, Any]) -> None:
+        self.last_seen_monotonic = monotonic()
+        records_processed = payload.get("records_processed", 1)
+        try:
+            records_processed = int(records_processed)
+        except (TypeError, ValueError):
+            records_processed = 1
+        self.records_processed += max(records_processed, 1)
+        operation = payload.get("operation")
+        if operation:
+            self.operations.add(str(operation))
+        entity_keys = payload.get("entity_keys")
+        if not isinstance(entity_keys, list):
+            entity_key = payload.get("entity_key")
+            entity_keys = [entity_key] if entity_key else []
+        for entity_key in entity_keys:
+            if entity_key and len(self.entity_keys) < MAX_ENTITY_KEY_SAMPLES:
+                self.entity_keys.append(str(entity_key))
+        occurred_at = payload.get("occurred_at")
+        if occurred_at:
+            occurred_at = str(occurred_at)
+            self.first_occurred_at = self.first_occurred_at or occurred_at
+            self.last_occurred_at = occurred_at
+
+
+class EarthDbChangeDispatcher:
+    """Debounces database notifications and broadcasts Earth refresh hints."""
+
+    def __init__(
+        self,
+        *,
+        broadcast_earth_update: BroadcastFn | None = None,
+        invalidate_cache: InvalidateFn | None = None,
+        debounce_seconds: float = DEFAULT_DEBOUNCE_SECONDS,
+        max_wait_seconds: float = DEFAULT_MAX_WAIT_SECONDS,
+    ) -> None:
+        self._broadcast_earth_update = broadcast_earth_update or broadcaster.broadcast_earth_update
+        self._invalidate_cache = invalidate_cache or invalidate_earth_layer_cache_for_source
+        self._debounce_seconds = debounce_seconds
+        self._max_wait_seconds = max(max_wait_seconds, debounce_seconds)
+        self._pending: dict[str, PendingEarthDbChange] = {}
+        self._flush_tasks: dict[str, asyncio.Task[None]] = {}
+        self._seen_event_ids: set[int] = set()
+        self._seen_event_order: deque[int] = deque()
+
+    def handle_notification(self, payload_text: str) -> bool:
+        try:
+            payload = json.loads(payload_text)
+        except json.JSONDecodeError:
+            logger.warning_event(
+                "Ignoring malformed Earth database change notification",
+                event="earth.db_changes.notification_malformed",
+            )
+            return False
+        if not isinstance(payload, dict):
+            return False
+        return self.handle_payload(payload)
+
+    def handle_payload(self, payload: dict[str, Any]) -> bool:
+        event_id = payload.get("event_id")
+        if event_id is not None:
+            try:
+                normalized_event_id = int(event_id)
+            except (TypeError, ValueError):
+                normalized_event_id = None
+            if normalized_event_id is not None:
+                if normalized_event_id in self._seen_event_ids:
+                    return False
+                self._remember_event_id(normalized_event_id)
+
+        update = build_earth_update_from_db_payload(payload)
+        if not update:
+            return False
+
+        source = update["source"]
+        pending = self._pending.get(source)
+        if pending is None:
+            pending = PendingEarthDbChange(
+                source=source,
+                layers=list(update["layers"]),
+                table=update.get("table"),
+                refresh_strategy=str(update.get("refresh_strategy") or "clear_then_reload"),
+                entity=update.get("entity"),
+                action=str(update.get("action") or "database_changed"),
+            )
+            self._pending[source] = pending
+        pending.add(payload)
+
+        task = self._flush_tasks.pop(source, None)
+        if task and not task.done():
+            task.cancel()
+        self._flush_tasks[source] = asyncio.create_task(
+            self._flush_later(source, delay_seconds=self._next_flush_delay(pending))
+        )
+        return True
+
+    def _next_flush_delay(self, pending: PendingEarthDbChange) -> float:
+        if "DELETE" in pending.operations and pending.refresh_strategy == "clear_then_reload":
+            return DELETE_FAST_FLUSH_SECONDS
+        elapsed = max(0.0, monotonic() - pending.first_seen_monotonic)
+        remaining = self._max_wait_seconds - elapsed
+        if remaining <= 0:
+            return 0.0
+        return min(self._debounce_seconds, remaining)
+
+    def _remember_event_id(self, event_id: int) -> None:
+        self._seen_event_ids.add(event_id)
+        self._seen_event_order.append(event_id)
+        while len(self._seen_event_order) > MAX_SEEN_EVENT_IDS:
+            expired_event_id = self._seen_event_order.popleft()
+            self._seen_event_ids.discard(expired_event_id)
+
+    async def _flush_later(self, source: str, *, delay_seconds: float) -> None:
+        try:
+            if delay_seconds > 0:
+                await asyncio.sleep(delay_seconds)
+            await self.flush_source(source)
+        except asyncio.CancelledError:
+            raise
+        except Exception as exc:
+            logger.exception_event(
+                "Failed to broadcast debounced Earth database change",
+                event="earth.db_changes.flush_failed",
+                context={"source": source, "error": str(exc)},
+            )
+        finally:
+            current = self._flush_tasks.get(source)
+            if current is asyncio.current_task():
+                self._flush_tasks.pop(source, None)
+
+    async def flush_source(self, source: str) -> None:
+        pending = self._pending.get(source)
+        if pending is None:
+            return
+
+        flushed_at = datetime.now(UTC)
+        deleted_cache_entries = self._invalidate_cache(source)
+        operations = sorted(pending.operations)
+        payload: dict[str, Any] = {
+            "event": "earth.layer.changed",
+            "action": "database_changed",
+            "source": source,
+            "table": pending.table,
+            "data_type": source,
+            "layers": pending.layers,
+            "refresh_strategy": pending.refresh_strategy,
+            "records_processed": pending.records_processed,
+            "operations": operations,
+            "operation": operations[-1] if len(operations) == 1 else None,
+            "entity_keys": pending.entity_keys,
+            "entity_key_sample_size": len(pending.entity_keys),
+            "cache_entries_invalidated": deleted_cache_entries,
+            "first_occurred_at": pending.first_occurred_at,
+            "last_occurred_at": pending.last_occurred_at,
+            "debounce_ms": int((monotonic() - pending.first_seen_monotonic) * 1000),
+            "total_latency_ms": self._total_latency_ms(pending, flushed_at),
+            "timestamp": to_iso8601_utc(flushed_at),
+        }
+        if pending.entity == "interactable":
+            payload.update(
+                {
+                    "entity": "interactable",
+                    "action": "deleted" if "DELETE" in pending.operations else "changed",
+                    "ids": pending.entity_keys,
+                    "item": None,
+                }
+            )
+        await self._broadcast_earth_update(payload)
+        self._pending.pop(source, None)
+        logger.info_event(
+            "Broadcasted Earth database change",
+            event="earth.db_changes.broadcasted",
+            context={
+                "source": source,
+                "layers": pending.layers,
+                "records_processed": pending.records_processed,
+                "cache_entries_invalidated": deleted_cache_entries,
+                "debounce_ms": int((monotonic() - pending.first_seen_monotonic) * 1000),
+                "total_latency_ms": payload["total_latency_ms"],
+            },
+        )
+
+    @staticmethod
+    def _total_latency_ms(pending: PendingEarthDbChange, flushed_at: datetime) -> int | None:
+        occurred_at = pending.first_occurred_at
+        if not occurred_at:
+            return None
+        try:
+            normalized = occurred_at.replace("Z", "+00:00")
+            occurred = datetime.fromisoformat(normalized)
+            if occurred.tzinfo is None:
+                occurred = occurred.replace(tzinfo=UTC)
+            return max(0, int((flushed_at - occurred.astimezone(UTC)).total_seconds() * 1000))
+        except ValueError:
+            return None
+
+    async def flush_all(self) -> None:
+        sources = list(self._pending)
+        for source in sources:
+            task = self._flush_tasks.pop(source, None)
+            if task and not task.done():
+                task.cancel()
+            await self.flush_source(source)
+
+    async def stop(self) -> None:
+        tasks = [task for task in self._flush_tasks.values() if not task.done()]
+        for task in tasks:
+            task.cancel()
+        if tasks:
+            await asyncio.gather(*tasks, return_exceptions=True)
+        self._flush_tasks.clear()
+        await self.flush_all()
+
+
+class EarthDbChangeListener:
+    def __init__(
+        self,
+        *,
+        dsn: str,
+        dispatcher: EarthDbChangeDispatcher,
+        channel: str = EARTH_DATA_CHANGES_CHANNEL,
+    ) -> None:
+        self._dsn = normalize_asyncpg_dsn(dsn)
+        self._dispatcher = dispatcher
+        self._channel = channel
+        self._task: asyncio.Task[None] | None = None
+        self._stop_event: asyncio.Event | None = None
+        self._connection: asyncpg.Connection | None = None
+        self._loop: asyncio.AbstractEventLoop | None = None
+
+    def start(self) -> None:
+        if self._task and not self._task.done():
+            return
+        self._loop = asyncio.get_running_loop()
+        self._stop_event = asyncio.Event()
+        self._task = asyncio.create_task(self._run())
+
+    async def stop(self) -> None:
+        if self._stop_event:
+            self._stop_event.set()
+        if self._connection:
+            await self._connection.close()
+        if self._task:
+            await asyncio.gather(self._task, return_exceptions=True)
+        await self._dispatcher.stop()
+
+    async def _run(self) -> None:
+        backoff_seconds = 1.0
+        assert self._stop_event is not None
+
+        while not self._stop_event.is_set():
+            try:
+                self._connection = await asyncpg.connect(self._dsn)
+                await self._connection.add_listener(self._channel, self._on_notification)
+                logger.info_event(
+                    "Earth database change listener connected",
+                    event="earth.db_changes.connected",
+                    context={"channel": self._channel},
+                )
+                backoff_seconds = 1.0
+                while not self._stop_event.is_set():
+                    try:
+                        await asyncio.wait_for(
+                            self._stop_event.wait(),
+                            timeout=LISTEN_KEEPALIVE_SECONDS,
+                        )
+                    except asyncio.TimeoutError:
+                        await self._poll_outbox()
+            except asyncio.CancelledError:
+                raise
+            except Exception as exc:
+                logger.exception_event(
+                    "Earth database change listener failed",
+                    event="earth.db_changes.listener_failed",
+                    context={"channel": self._channel, "error": str(exc)},
+                )
+                try:
+                    await asyncio.wait_for(self._stop_event.wait(), timeout=backoff_seconds)
+                except asyncio.TimeoutError:
+                    pass
+                backoff_seconds = min(backoff_seconds * 2, 30.0)
+            finally:
+                if self._connection:
+                    try:
+                        await self._connection.remove_listener(self._channel, self._on_notification)
+                    except Exception:
+                        pass
+                    try:
+                        await self._connection.close()
+                    except Exception:
+                        pass
+                    self._connection = None
+
+    async def _poll_outbox(self) -> None:
+        if self._connection is None:
+            return
+
+        rows = await self._connection.fetch(
+            """
+            SELECT id, payload
+            FROM earth_data_change_events
+            WHERE consumed_at IS NULL
+            ORDER BY id
+            LIMIT $1
+            """,
+            OUTBOX_POLL_LIMIT,
+        )
+        accepted_count = 0
+        consumed_ids: list[int] = []
+        for row in rows:
+            payload = row["payload"]
+            if isinstance(payload, str):
+                try:
+                    payload = json.loads(payload)
+                except json.JSONDecodeError:
+                    consumed_ids.append(int(row["id"]))
+                    continue
+            if isinstance(payload, dict):
+                if self._dispatcher.handle_payload(payload):
+                    accepted_count += 1
+                consumed_ids.append(int(row["id"]))
+            else:
+                consumed_ids.append(int(row["id"]))
+        if consumed_ids:
+            await self._dispatcher.flush_all()
+        if consumed_ids:
+            await self._connection.execute(
+                """
+                UPDATE earth_data_change_events
+                SET consumed_at = NOW()
+                WHERE id = ANY($1::bigint[])
+                  AND consumed_at IS NULL
+                """,
+                consumed_ids,
+            )
+        if rows:
+            logger.info_event(
+                "Polled Earth database change outbox",
+                event="earth.db_changes.outbox_polled",
+                context={"events": len(rows), "accepted": accepted_count},
+            )
+
+    def _on_notification(
+        self,
+        _connection: asyncpg.Connection,
+        _pid: int,
+        _channel: str,
+        payload: str,
+    ) -> None:
+        if self._loop and self._loop.is_running():
+            self._loop.call_soon_threadsafe(self._dispatcher.handle_notification, payload)
+            return
+        self._dispatcher.handle_notification(payload)
+
+
+_dispatcher = EarthDbChangeDispatcher(
+    broadcast_earth_update=broadcaster.broadcast_earth_update,
+    invalidate_cache=invalidate_earth_layer_cache_for_source,
+)
+_listener: EarthDbChangeListener | None = None
+
+
+def start_earth_db_change_listener() -> None:
+    global _listener
+    if _listener is not None:
+        return
+    _listener = EarthDbChangeListener(dsn=settings.DATABASE_URL, dispatcher=_dispatcher)
+    _listener.start()
+
+
+async def stop_earth_db_change_listener() -> None:
+    global _listener
+    if _listener is None:
+        await _dispatcher.stop()
+        return
+    listener = _listener
+    _listener = None
+    await listener.stop()

backend/app/services/earth_interactables.py

@@ -0,0 +1,113 @@
+from __future__ import annotations
+
+from datetime import UTC, datetime
+from typing import Any
+from uuid import uuid4
+
+from sqlalchemy import select
+from sqlalchemy.ext.asyncio import AsyncSession
+
+from app.core.time import to_iso8601_utc
+from app.core.websocket.broadcaster import broadcaster
+from app.models.earth_interactable import EarthInteractable
+from app.services.earth_layer_cache import EARTH_LAYER_CACHE_PREFIX, earth_layer_cache
+
+INTERACTABLE_ENTITY = "interactable"
+INTERACTABLE_LAYER = "interactables"
+
+
+def normalize_interactable_id(value: str | None = None) -> str:
+    raw = str(value or "").strip()
+    return raw or f"interactable-{uuid4().hex}"
+
+
+def serialize_interactable(record: EarthInteractable) -> dict[str, Any]:
+    return {
+        "id": record.id,
+        "layer": record.layer,
+        "kind": record.kind,
+        "label": record.label,
+        "description": record.description,
+        "latitude": record.latitude,
+        "longitude": record.longitude,
+        "altitude": record.altitude,
+        "revision": record.revision,
+        "properties": record.properties or {},
+        "is_deleted": bool(record.is_deleted),
+        "created_at": to_iso8601_utc(record.created_at),
+        "updated_at": to_iso8601_utc(record.updated_at),
+        "deleted_at": to_iso8601_utc(record.deleted_at),
+    }
+
+
+def interactables_to_geojson(items: list[EarthInteractable]) -> dict[str, Any]:
+    return {
+        "type": "FeatureCollection",
+        "features": [
+            {
+                "type": "Feature",
+                "id": item.id,
+                "geometry": {
+                    "type": "Point",
+                    "coordinates": [item.longitude, item.latitude],
+                },
+                "properties": serialize_interactable(item),
+            }
+            for item in items
+            if not item.is_deleted
+        ],
+    }
+
+
+def invalidate_interactable_cache(layer: str | None = None) -> int:
+    layer_key = str(layer or "*").strip() or "*"
+    deleted = earth_layer_cache.delete_pattern(
+        f"{EARTH_LAYER_CACHE_PREFIX}:interactables:layer:{layer_key}*"
+    )
+    if layer_key != "all":
+        deleted += earth_layer_cache.delete_pattern(
+            f"{EARTH_LAYER_CACHE_PREFIX}:interactables:layer:all*"
+        )
+    return deleted
+
+
+def build_interactable_event(
+    *,
+    action: str,
+    record: EarthInteractable,
+    include_item: bool = True,
+) -> dict[str, Any]:
+    item = serialize_interactable(record)
+    return {
+        "entity": INTERACTABLE_ENTITY,
+        "action": action,
+        "layer": record.layer,
+        "layers": [INTERACTABLE_LAYER],
+        "ids": [record.id],
+        "revision": record.revision,
+        "changed_at": item["deleted_at"] or item["updated_at"] or to_iso8601_utc(datetime.now(UTC)),
+        "item": item if include_item else None,
+        "source": "earth_interactables",
+    }
+
+
+async def publish_interactable_event(action: str, record: EarthInteractable, *, include_item: bool = True) -> None:
+    await broadcaster.broadcast_earth_update(
+        build_interactable_event(action=action, record=record, include_item=include_item)
+    )
+
+
+async def list_interactables(
+    db: AsyncSession,
+    *,
+    layer: str | None = None,
+    include_deleted: bool = False,
+) -> list[EarthInteractable]:
+    stmt = select(EarthInteractable)
+    if layer:
+        stmt = stmt.where(EarthInteractable.layer == layer)
+    if not include_deleted:
+        stmt = stmt.where(EarthInteractable.is_deleted.is_(False))
+    stmt = stmt.order_by(EarthInteractable.updated_at.desc(), EarthInteractable.id.asc())
+    result = await db.execute(stmt)
+    return list(result.scalars().all())

backend/app/services/earth_layer_adapters.py

@@ -0,0 +1,179 @@
+"""Earth layer adapter registry for datasource-backed refresh behavior."""
+
+from __future__ import annotations
+
+from dataclasses import dataclass, field
+from typing import Any
+
+from sqlalchemy.ext.asyncio import AsyncSession
+
+
+@dataclass(frozen=True)
+class EarthLayerAdapter:
+    sources: frozenset[str]
+    layers: tuple[str, ...]
+    cache_patterns: tuple[str, ...]
+    tables: frozenset[str] = field(default_factory=frozenset)
+    derived_models: tuple[str, ...] = field(default_factory=tuple)
+    refresh_strategy: str = "clear_then_reload"
+
+
+EARTH_LAYER_ADAPTERS: tuple[EarthLayerAdapter, ...] = (
+    EarthLayerAdapter(
+        sources=frozenset({"barentswatch_vessels", "aisstream_vessels", "vessel_static", "vessel_position", "ais_raw_observations", "ais_source_health"}),
+        tables=frozenset({"vessel_static", "vessel_position", "ais_raw_observations", "ais_source_health"}),
+        layers=("vessels",),
+        cache_patterns=("vessels*", "summary*"),
+    ),
+    EarthLayerAdapter(
+        sources=frozenset(
+            {
+                "telegeography_cables",
+                "telegeography_landing",
+                "telegeography_landing_points",
+                "telegeography_systems",
+                "telegeography_cable_systems",
+                "arcgis_cables",
+                "arcgis_landing_points",
+                "arcgis_cable_landing_relation",
+                "arcgis_cable_landing_relations",
+                "fao_landing_points",
+            }
+        ),
+        tables=frozenset({"collected_data"}),
+        layers=("cables",),
+        cache_patterns=("cables*", "landing-points*", "summary*"),
+    ),
+    EarthLayerAdapter(
+        sources=frozenset({"celestrak_tle", "spacetrack_tle"}),
+        tables=frozenset({"collected_data"}),
+        layers=("satellites",),
+        cache_patterns=("satellites*", "summary*"),
+    ),
+    EarthLayerAdapter(
+        sources=frozenset(
+            {
+                "top500",
+                "top500_supercomputers",
+                "epoch_ai_gpu",
+                "huggingface_models",
+                "huggingface_datasets",
+                "huggingface_spaces",
+                "compute_center_locations",
+            }
+        ),
+        tables=frozenset({"compute_center_locations"}),
+        layers=("computeCenters",),
+        cache_patterns=("compute-centers*", "summary*"),
+        refresh_strategy="reload",
+    ),
+    EarthLayerAdapter(
+        sources=frozenset(
+            {
+                "ris_live_bgp",
+                "bgpstream_bgp",
+                "iptoasn_prefix_geo",
+                "opengeofeed_prefix_geo",
+                "nro_delegated_prefix_geo",
+                "bgp_observations",
+                "bgp_anomalies",
+                "bgp_incidents",
+                "bgp_collector_locations",
+            }
+        ),
+        tables=frozenset({"bgp_observations", "bgp_anomalies", "bgp_incidents", "bgp_collector_locations"}),
+        layers=("bgp",),
+        cache_patterns=("bgp*", "summary*"),
+        derived_models=("bgp_observations", "bgp_anomalies", "bgp_incidents"),
+    ),
+    EarthLayerAdapter(
+        sources=frozenset({"news_live_streams"}),
+        tables=frozenset({"collected_data"}),
+        layers=("media",),
+        cache_patterns=("summary*",),
+        refresh_strategy="reload",
+    ),
+    EarthLayerAdapter(
+        sources=frozenset({"media_news_archive", "earth_news_items"}),
+        tables=frozenset({"earth_news_items"}),
+        layers=("news",),
+        cache_patterns=("summary*",),
+        refresh_strategy="reload",
+    ),
+    EarthLayerAdapter(
+        sources=frozenset({"earth_interactables"}),
+        tables=frozenset({"earth_interactables"}),
+        layers=("interactables",),
+        cache_patterns=("interactables*", "summary*"),
+        refresh_strategy="delta",
+    ),
+)
+
+_ADAPTERS_BY_SOURCE = {
+    source: adapter
+    for adapter in EARTH_LAYER_ADAPTERS
+    for source in adapter.sources
+}
+_ADAPTERS_BY_TABLE = {
+    table: adapter
+    for adapter in EARTH_LAYER_ADAPTERS
+    for table in adapter.tables
+}
+
+
+def get_earth_layer_adapter_for_source(source: str | None) -> EarthLayerAdapter | None:
+    return _ADAPTERS_BY_SOURCE.get(str(source or "").strip())
+
+
+def get_earth_layer_adapter_for_change(table: str | None, source: str | None) -> EarthLayerAdapter | None:
+    table_key = str(table or "").strip()
+    source_key = str(source or "").strip()
+    if table_key and table_key != "collected_data":
+        adapter = _ADAPTERS_BY_TABLE.get(table_key)
+        if adapter is not None:
+            return adapter
+    return get_earth_layer_adapter_for_source(source_key)
+
+
+def get_earth_update_layers_for_source(source: str | None) -> list[str]:
+    adapter = get_earth_layer_adapter_for_source(source)
+    return list(adapter.layers) if adapter else []
+
+
+def get_earth_update_layers_for_change(table: str | None, source: str | None) -> list[str]:
+    adapter = get_earth_layer_adapter_for_change(table, source)
+    return list(adapter.layers) if adapter else []
+
+
+def get_earth_refresh_strategy_for_change(table: str | None, source: str | None) -> str | None:
+    adapter = get_earth_layer_adapter_for_change(table, source)
+    return adapter.refresh_strategy if adapter else None
+
+
+def get_earth_cache_patterns_for_source(source: str | None) -> list[str]:
+    adapter = get_earth_layer_adapter_for_source(source)
+    return list(adapter.cache_patterns) if adapter else []
+
+
+async def clear_derived_datasource_data(db: AsyncSession, source: str) -> dict[str, int]:
+    adapter = get_earth_layer_adapter_for_source(source)
+    if adapter is None or not adapter.derived_models:
+        return {}
+
+    from app.models.bgp_anomaly import BGPAnomaly
+    from app.models.bgp_incident import BGPIncident
+    from app.models.bgp_observation import BGPObservation
+
+    model_by_key: dict[str, Any] = {
+        "bgp_observations": BGPObservation,
+        "bgp_anomalies": BGPAnomaly,
+        "bgp_incidents": BGPIncident,
+    }
+    deleted_counts: dict[str, int] = {}
+    for key in adapter.derived_models:
+        model = model_by_key.get(key)
+        if model is None:
+            continue
+        result = await db.execute(model.__table__.delete().where(model.source == source))
+        deleted_counts[key] = int(result.rowcount or 0)
+    return deleted_counts

backend/app/services/earth_layer_cache.py

@@ -268,18 +268,10 @@ def apply_payload_budget(payload: dict[str, Any], policy: EarthLayerCachePolicy)
 
 
 def invalidate_earth_layer_cache_for_source(source: str) -> int:
+    from app.services.earth_layer_adapters import get_earth_cache_patterns_for_source
+
     source_key = str(source or "").strip()
-    patterns = {
-        "arcgis_cables": ["cables*", "landing-points*", "summary*"],
-        "arcgis_landing_points": ["landing-points*", "summary*"],
-        "arcgis_cable_landing_relation": ["landing-points*", "summary*"],
-        "celestrak_tle": ["satellites*", "summary*"],
-        "spacetrack_tle": ["satellites*", "summary*"],
-        "top500": ["compute-centers*", "summary*"],
-        "epoch_ai_gpu": ["compute-centers*", "summary*"],
-        "ris_live_bgp": ["bgp*", "summary*"],
-        "bgpstream_bgp": ["bgp*", "summary*"],
-    }.get(source_key, [])
+    patterns = get_earth_cache_patterns_for_source(source_key)
     deleted = 0
     for layer_pattern in patterns:
         deleted += earth_layer_cache.delete_pattern(f"{EARTH_LAYER_CACHE_PREFIX}:{layer_pattern}")

backend/app/services/earth_news.py

@@ -853,7 +853,7 @@ def _parse_feed_entries(xml_text: str, source: NewsFeedSource) -> list[ParsedNew
         if not clean_title or not link:
             continue
 
-        item_source = _normalize_source_name(clean_title, source.name)
+        item_source = source.name
         display_title = clean_title
         if source.source_type == "aggregated" and " - " in clean_title:
             parts = clean_title.rsplit(" - ", 1)
@@ -1063,6 +1063,7 @@ def _build_payload(
     lon: float | None,
     active_region: str,
     items: list[ParsedNewsItem],
+    cruise_items: list[ParsedNewsItem] | None = None,
     sources: list[NewsFeedSource],
     errors: list[str],
     stale: bool,
@@ -1082,6 +1083,10 @@ def _build_payload(
         },
         "sources": _serialize_sources(sources),
         "items": [_serialize_item(item, active_region=active_region) for item in items],
+        "cruise_items": [
+            _serialize_item(item, active_region=active_region)
+            for item in (cruise_items if cruise_items is not None else items)
+        ],
         "errors": errors,
         "stale": stale,
     }
@@ -1295,6 +1300,7 @@ async def get_earth_news_payload(
 
     from app.services.earth_news_store import (
         get_earth_news_freshness,
+        list_earth_news_cruise_items,
         list_earth_news_items,
         upsert_earth_news_items,
     )
@@ -1312,6 +1318,13 @@ async def get_earth_news_payload(
         active_region=active_region,
         limit=MAX_ITEMS_TOTAL,
     )
+    if hasattr(db, "execute"):
+        cruise_items = await list_earth_news_cruise_items(
+            db,
+            limit=MAX_ITEMS_TOTAL * len(REGION_ANCHORS),
+        )
+    else:
+        cruise_items = items
     await _enqueue_unverified_locations(items)
     stale = bool(errors and items)
 
@@ -1320,6 +1333,7 @@ async def get_earth_news_payload(
         lon=lon,
         active_region=active_region,
         items=items,
+        cruise_items=cruise_items,
         sources=sources,
         errors=errors,
         stale=stale,

backend/app/services/earth_news_store.py

@@ -78,6 +78,24 @@ async def list_earth_news_items(
     return [record_to_parsed_news_item(record) for record in result.scalars().all()]
 
 
+async def list_earth_news_cruise_items(
+    db: AsyncSession,
+    *,
+    limit: int,
+) -> list[ParsedNewsItem]:
+    result = await db.execute(
+        select(EarthNewsItem)
+        .order_by(
+            EarthNewsItem.region.asc(),
+            EarthNewsItem.published_at.desc().nullslast(),
+            EarthNewsItem.last_seen_at.desc(),
+            EarthNewsItem.feed_name.asc(),
+        )
+        .limit(limit)
+    )
+    return [record_to_parsed_news_item(record) for record in result.scalars().all()]
+
+
 async def get_earth_news_freshness(
     db: AsyncSession,
     *,

backend/app/services/llm_provider_catalog.py

@@ -7,6 +7,26 @@ from typing import Any
 import httpx
 
 MODELS_DEV_URL = "https://models.dev/api.json"
+OPENCODE_GO_MODELS_URL = "https://opencode.ai/zen/go/v1/models"
+
+OPENCODE_GO_MODEL_PROVIDER_APIS = {
+    "minimax-m2.7": "anthropic-messages",
+    "minimax-m2.5": "anthropic-messages",
+}
+OPENCODE_GO_FALLBACK_MODELS = [
+    "minimax-m2.7",
+    "minimax-m2.5",
+    "kimi-k2.6",
+    "kimi-k2.5",
+    "glm-5.1",
+    "glm-5",
+    "deepseek-v4-pro",
+    "deepseek-v4-flash",
+    "qwen3.6-plus",
+    "qwen3.5-plus",
+    "mimo-v2.5-pro",
+    "mimo-v2.5",
+]
 
 
 FALLBACK_LLM_PROVIDER_PRESETS: dict[str, dict[str, Any]] = {
@@ -80,6 +100,17 @@ FALLBACK_LLM_PROVIDER_PRESETS: dict[str, dict[str, Any]] = {
         "api_key_env": "OPENROUTER_API_KEY",
         "source": "fallback",
     },
+    "opencode-go": {
+        "provider": "opencode-go",
+        "label": "OpenCode Go",
+        "provider_api": "openai-completions",
+        "base_url": "https://opencode.ai/zen/go/v1",
+        "model": "glm-5.1",
+        "models": OPENCODE_GO_FALLBACK_MODELS,
+        "model_provider_apis": OPENCODE_GO_MODEL_PROVIDER_APIS,
+        "api_key_env": "OPENCODE_GO_API_KEY",
+        "source": "fallback",
+    },
     "ollama": {
         "provider": "ollama",
         "label": "Ollama Local",
@@ -114,8 +145,43 @@ def get_fallback_llm_provider_preset(provider: str) -> dict[str, Any]:
     return dict(FALLBACK_LLM_PROVIDER_PRESETS[key])
 
 
-async def refresh_llm_provider_preset(provider: str) -> dict[str, Any]:
+def _opencode_go_model_provider_apis(model_ids: list[str]) -> dict[str, str]:
+    return {
+        model_id: OPENCODE_GO_MODEL_PROVIDER_APIS.get(model_id, "openai-completions")
+        for model_id in model_ids
+    }
+
+
+async def refresh_llm_provider_preset(provider: str, api_key: str | None = None) -> dict[str, Any]:
     fallback = get_fallback_llm_provider_preset(provider)
+    if fallback["provider"] == "opencode-go":
+        headers = {"User-Agent": "Planet/1.0"}
+        if api_key:
+            headers["Authorization"] = f"Bearer {api_key}"
+        async with httpx.AsyncClient(timeout=15.0, follow_redirects=True) as client:
+            response = await client.get(
+                OPENCODE_GO_MODELS_URL,
+                headers=headers,
+            )
+            response.raise_for_status()
+            payload = response.json()
+
+        data = payload.get("data") if isinstance(payload, dict) else []
+        model_ids = [
+            str(item.get("id"))
+            for item in data
+            if isinstance(item, dict) and item.get("id")
+        ][:120]
+        if not model_ids:
+            model_ids = fallback["models"]
+        return {
+            **fallback,
+            "model": fallback["model"] if fallback["model"] in model_ids else model_ids[0],
+            "models": model_ids,
+            "model_provider_apis": _opencode_go_model_provider_apis(model_ids),
+            "source": OPENCODE_GO_MODELS_URL,
+        }
+
     models_dev_key = MODELS_DEV_PROVIDER_KEYS.get(fallback["provider"])
     if not models_dev_key:
         return fallback

backend/app/services/location/llm_fallback.py

@@ -9,6 +9,7 @@ from typing import Any, Iterable
 
 from sqlalchemy.ext.asyncio import AsyncSession
 
+from app.core.logging import get_logger
 from app.core.countries import COUNTRY_ENTRIES, normalize_country
 from app.schemas.ai import SituationalAnalysisRequest
 from app.ai_tasks.prompts import get_effective_prompt
@@ -29,6 +30,9 @@ DEFAULT_MIN_CONFIDENCE = 0.55
 LOCATION_NORMALIZE_PROMPT_KEY = "location.factcheck.normalize"
 LOCATION_RESOLVE_PROMPT_KEY = "location.factcheck.resolve"
 MODEL_CONFIDENCE_WEIGHT = 0.25
+LOG_TEXT_LIMIT = 1200
+LOG_EVIDENCE_LIMIT = 5
+logger = get_logger(__name__, service="location")
 _geocode_llm_city = build_default_nominatim_geocoder()
 _LLM_LOCATION_NAME_KEYS = (
     "matched_location_name",
@@ -97,6 +101,35 @@ class LocationEvidenceScore:
     summary: str
 
 
+def _truncate_log_text(value: Any, limit: int = LOG_TEXT_LIMIT) -> str:
+    text = coerce_str(value)
+    if len(text) <= limit:
+        return text
+    return f"{text[:limit]}…"
+
+
+def _summarize_search_evidence(evidence: list[dict[str, Any]] | None) -> list[dict[str, Any]]:
+    items: list[dict[str, Any]] = []
+    for item in (evidence or [])[:LOG_EVIDENCE_LIMIT]:
+        if not isinstance(item, dict):
+            continue
+        items.append(
+            {
+                "title": _truncate_log_text(item.get("title"), 180),
+                "source": _truncate_log_text(item.get("source") or item.get("name"), 120),
+                "url": _truncate_log_text(item.get("url"), 240),
+                "snippet": _truncate_log_text(
+                    item.get("snippet")
+                    or item.get("content")
+                    or item.get("text")
+                    or item.get("summary"),
+                    360,
+                ),
+            }
+        )
+    return items
+
+
 def _first_json_object(text: str) -> dict[str, Any] | None:
     stripped = text.strip()
     if not stripped:
@@ -162,6 +195,64 @@ def _evidence_label(item: Any) -> str:
     return coerce_str(item)
 
 
+def _evidence_text(item: dict[str, Any]) -> str:
+    return " ".join(
+        coerce_str(item.get(key))
+        for key in ("title", "source", "name", "url", "snippet", "content", "text", "quote", "summary")
+        if coerce_str(item.get(key))
+    )
+
+
+def _search_evidence_entity_match(item: dict[str, Any], query: LocationQuery) -> bool:
+    haystack = normalize_text(_evidence_text(item))
+    if not haystack:
+        return False
+    needles = [
+        coerce_str(query.name),
+        *[coerce_str(alias) for alias in query.aliases],
+    ]
+    return any(normalize_text(needle) and normalize_text(needle) in haystack for needle in needles)
+
+
+def _evidence_has_location_assertion(item: dict[str, Any], city: str) -> bool:
+    normalized_city = normalize_text(city)
+    text = normalize_text(_evidence_text(item))
+    if not normalized_city or normalized_city not in text:
+        return False
+    assertion_terms = (
+        "located",
+        "situated",
+        "built",
+        "hosted",
+        "deployed",
+        "installed",
+        "facility",
+        "campus",
+        "site",
+        "data center",
+        "datacenter",
+        "supercomputer center",
+        "位于",
+        "位於",
+        "坐落",
+        "建置",
+        "設置",
+        "设置",
+    )
+    return any(term in text for term in assertion_terms)
+
+
+def _city_is_unsupported_name_hint(payload: dict[str, Any], query: LocationQuery, evidence_items: list[dict[str, Any]]) -> bool:
+    city = coerce_str(payload.get("city") or query.city)
+    if not city:
+        return False
+    normalized_city = normalize_text(city)
+    normalized_name = normalize_text(query.name)
+    if not normalized_city or not normalized_name or normalized_city not in normalized_name:
+        return False
+    return not any(_evidence_has_location_assertion(item, city) for item in evidence_items)
+
+
 def _normalize_llm_precision(value: Any) -> str:
     text = coerce_str(value).lower()
     return LLM_PRECISION_ALIASES.get(text, text)
@@ -588,6 +679,7 @@ def _weak_evidence_penalty(
     payload: dict[str, Any],
     evidence_items: list[dict[str, Any]],
     *,
+    query: LocationQuery,
     entity_match: float,
     geography_match: float,
     conflict_penalty: float,
@@ -598,6 +690,8 @@ def _weak_evidence_penalty(
         penalty += 0.20
     if any(_truthy_evidence_field(item, "ambiguous") for item in evidence_items):
         penalty += 0.15
+    if _city_is_unsupported_name_hint(payload, query, evidence_items):
+        penalty += 0.10
     if conflict_penalty == 0.0 and entity_match > 0 and geography_match >= 0.20:
         return min(penalty, 0.15)
     return min(penalty, 0.30)
@@ -620,6 +714,7 @@ def _score_llm_location_payload(
     weak_evidence_penalty = _weak_evidence_penalty(
         payload,
         evidence_items,
+        query=query,
         entity_match=entity_match,
         geography_match=geography_match,
         conflict_penalty=conflict_penalty,
@@ -636,6 +731,8 @@ def _score_llm_location_payload(
         - weak_evidence_penalty
     )
     score = min(max(score, 0.0), 1.0)
+    if _city_is_unsupported_name_hint(payload, query, evidence_items):
+        score = min(score, 0.54)
     summary = (
         f"combined={score:.2f}; model={model_confidence:.2f}; "
         f"source={source_quality:.2f}; entity={entity_match:.2f}; "
@@ -847,15 +944,43 @@ async def collect_location_search_evidence(
 ) -> LocationSearchEvidenceResult:
     search_query = _location_search_query(query, entity_type)
     attempt = f"web_search:{entity_type}:{search_query}"
+    logger.info_event(
+        "Collecting location search evidence",
+        event="location.factcheck.web_search.start",
+        context={
+            "entity_type": entity_type,
+            "search_query": search_query,
+            "location_query": _query_context(query),
+            "max_results": max_results,
+        },
+    )
     try:
         evidence = await web_search_client.search(search_query, max_results=max_results)
     except WebSearchError as exc:
+        logger.warning_event(
+            "Location search evidence failed",
+            event="location.factcheck.web_search.failed",
+            context={
+                "entity_type": entity_type,
+                "search_query": search_query,
+                "error": str(exc),
+            },
+        )
         return LocationSearchEvidenceResult(
             evidence=[],
             attempted_queries=[attempt],
             failure_reason=f"WebSearch location evidence failed: {exc}",
         )
     except Exception as exc:
+        logger.warning_event(
+            "Location search evidence unavailable",
+            event="location.factcheck.web_search.unavailable",
+            context={
+                "entity_type": entity_type,
+                "search_query": search_query,
+                "error": str(exc),
+            },
+        )
         return LocationSearchEvidenceResult(
             evidence=[],
             attempted_queries=[attempt],
@@ -863,11 +988,29 @@ async def collect_location_search_evidence(
         )
     normalized = normalize_search_evidence(evidence, limit=max_results)
     if not normalized:
+        logger.warning_event(
+            "Location search returned no usable evidence",
+            event="location.factcheck.web_search.empty",
+            context={
+                "entity_type": entity_type,
+                "search_query": search_query,
+            },
+        )
         return LocationSearchEvidenceResult(
             evidence=[],
             attempted_queries=[attempt],
             failure_reason="WebSearch returned no usable location evidence.",
         )
+    logger.info_event(
+        "Collected location search evidence",
+        event="location.factcheck.web_search.result",
+        context={
+            "entity_type": entity_type,
+            "search_query": search_query,
+            "evidence_count": len(normalized),
+            "evidence": _summarize_search_evidence(normalized),
+        },
+    )
     return LocationSearchEvidenceResult(
         evidence=normalized,
         attempted_queries=[attempt],
@@ -947,6 +1090,15 @@ async def collect_llm_location_fallback_candidate(
     """
     attempt = f"llm_factcheck:{entity_type}:{coerce_str(query.name) or 'unknown'}"
     if search_evidence is not None and not search_evidence:
+        logger.warning_event(
+            "Skipping LLM location factcheck because search evidence is empty",
+            event="location.factcheck.llm.skipped_no_evidence",
+            context={
+                "entity_type": entity_type,
+                "attempt": attempt,
+                "location_query": _query_context(query),
+            },
+        )
         return LocationLLMFallbackResult(
             candidates=[],
             attempted_queries=[attempt],
@@ -986,20 +1138,66 @@ async def collect_llm_location_fallback_candidate(
             "Return evidence as objects when possible, including source, url, source_type, and entity_match.",
             "Include source names or URLs in evidence when known. The backend will recompute the final confidence from model confidence plus evidence quality.",
             "If search_evidence is provided, use only that evidence as factual support.",
+            "Do not treat a website footer, office address, publisher address, or contact address as the entity's physical location.",
+            "If the entity name contains a city name, do not choose that city unless evidence explicitly says the entity/facility/supercomputer is located, hosted, built, deployed, or installed there.",
             "Prefer the facility/site if known; otherwise use the best supported city.",
         ],
     )
+    logger.info_event(
+        "Sending location factcheck request to LLM",
+        event="location.factcheck.llm.request",
+        context={
+            "entity_type": entity_type,
+            "attempt": attempt,
+            "title": request.title,
+            "objective": request.objective,
+            "location_query": request.context.get("location_query"),
+            "observations": request.observations,
+            "constraints": request.constraints,
+            "search_evidence_count": len(search_evidence or []),
+            "search_evidence": _summarize_search_evidence(search_evidence),
+        },
+    )
     try:
         response = await provider_client.analyze(request)
     except Exception as exc:
+        logger.warning_event(
+            "LLM location factcheck failed",
+            event="location.factcheck.llm.failed",
+            context={
+                "entity_type": entity_type,
+                "attempt": attempt,
+                "error": str(exc),
+            },
+        )
         return LocationLLMFallbackResult(
             candidates=[],
             attempted_queries=[attempt],
             failure_reason=f"LLM location factcheck failed: {exc}",
         )
 
+    logger.info_event(
+        "Received location factcheck response from LLM",
+        event="location.factcheck.llm.response",
+        context={
+            "entity_type": entity_type,
+            "attempt": attempt,
+            "provider": response.provider,
+            "model": response.model,
+            "content": _truncate_log_text(response.content, 2000),
+        },
+    )
     payload = _first_json_object(response.content)
     if payload is None:
+        logger.warning_event(
+            "LLM location factcheck response was not strict JSON; attempting repair",
+            event="location.factcheck.llm.non_json",
+            context={
+                "entity_type": entity_type,
+                "attempt": attempt,
+                "content": _truncate_log_text(response.content, 1200),
+            },
+        )
         payload = await _repair_location_payload_from_text(
             provider_client=provider_client,
             raw_text=response.content,
@@ -1009,9 +1207,17 @@ async def collect_llm_location_fallback_candidate(
         )
     if payload is None:
         payload = _payload_from_free_text(response.content, query=query)
-    if payload is None:
+    if payload is None and entity_type != "compute_center":
         payload = _payload_from_query_name_geocode(query)
     if payload is None:
+        logger.warning_event(
+            "LLM location factcheck produced no parseable payload",
+            event="location.factcheck.llm.unparseable",
+            context={
+                "entity_type": entity_type,
+                "attempt": attempt,
+            },
+        )
         return LocationLLMFallbackResult(
             candidates=[],
             attempted_queries=[attempt],
@@ -1032,7 +1238,7 @@ async def collect_llm_location_fallback_candidate(
                     "url": item.get("url"),
                     "text": item.get("snippet") or item.get("content"),
                     "source_type": "web_search",
-                    "entity_match": True,
+                    "entity_match": _search_evidence_entity_match(item, query),
                 }
                 for item in search_evidence
                 if isinstance(item, dict)
@@ -1054,6 +1260,18 @@ async def collect_llm_location_fallback_candidate(
     if candidate is None:
         if city_geocode_failure and rejection_reason == "missing, invalid, or zero latitude/longitude":
             rejection_reason = f"{rejection_reason}; {city_geocode_failure}"
+        logger.warning_event(
+            "Rejected LLM location factcheck candidate",
+            event="location.factcheck.llm.rejected",
+            context={
+                "entity_type": entity_type,
+                "attempt": attempt,
+                "reason": rejection_reason,
+                "payload": payload,
+                "search_evidence_count": len(search_evidence or []),
+                "search_evidence": _summarize_search_evidence(search_evidence),
+            },
+        )
         return LocationLLMFallbackResult(
             candidates=[],
             attempted_queries=[attempt],
@@ -1062,6 +1280,18 @@ async def collect_llm_location_fallback_candidate(
                 + (f": {rejection_reason}." if rejection_reason else ".")
             ),
         )
+    logger.info_event(
+        "Accepted LLM location factcheck candidate",
+        event="location.factcheck.llm.accepted",
+        context={
+            "entity_type": entity_type,
+            "attempt": attempt,
+            "candidate": candidate.to_dict(),
+            "payload": payload,
+            "search_evidence_count": len(search_evidence or []),
+            "search_evidence": _summarize_search_evidence(search_evidence),
+        },
+    )
     return LocationLLMFallbackResult(
         candidates=[candidate],
         attempted_queries=[attempt],

backend/app/services/playground_chat_service.py

@@ -10,6 +10,7 @@ from fastapi import HTTPException, status
 from sqlalchemy import func, select, update
 from sqlalchemy.ext.asyncio import AsyncSession
 
+from app.core.logging import get_logger
 from app.db.session import async_session_factory
 from app.models.playground_message import PlaygroundMessage
 from app.models.playground_session import PlaygroundSession
@@ -27,9 +28,11 @@ from app.schemas.ai import (
     SituationalAnalysisRequest,
 )
 from app.services.ai_client import AIProviderClient
+from app.services.business_logs import emit_business_log, exception_context
 from app.services.playground_session_store import _to_response as session_to_response
 from app.services.playground_session_store import upsert_playground_session
 
+logger = get_logger(__name__, service="ai")
 STREAM_CHUNK_SIZE = 24
 STREAM_INTERVAL_SECONDS = 0.08
 THINKING_PREVIEW_SECONDS = 2.6
@@ -624,7 +627,42 @@ async def _run_assistant_message(
                 thinking={"type": "enabled"},
             )
 
+        await emit_business_log(
+            logger,
+            event="ai.playground.run.start",
+            message="Playground AI run started",
+            category="ai",
+            service="ai",
+            module=__name__,
+            request_id=request_id,
+            user_id=user_id,
+            context={
+                "session_id": session_id,
+                "session_key": session_key,
+                "user_message_id": user_message_id,
+                "assistant_message_id": assistant_message_id,
+                "preset": payload.selected_preset_key,
+            },
+        )
         analysis = await provider_client.analyze(request_payload, request_id=request_id)
+        await emit_business_log(
+            logger,
+            event="ai.playground.run.success",
+            message="Playground AI run completed",
+            category="ai",
+            service="ai",
+            module=__name__,
+            request_id=request_id,
+            user_id=user_id,
+            context={
+                "session_id": session_id,
+                "session_key": session_key,
+                "provider": analysis.provider,
+                "model": analysis.model,
+                "content_block_count": len(analysis.content_blocks or []),
+                "thinking_block_count": len(analysis.thinking_blocks or []),
+            },
+        )
 
         async with async_session_factory() as db:
             assistant_message = await _mark_message_state(
@@ -704,6 +742,23 @@ async def _run_assistant_message(
                 await db.flush()
             await db.commit()
     except asyncio.CancelledError:
+        await emit_business_log(
+            logger,
+            event="ai.playground.run.cancelled",
+            message="Playground AI run cancelled",
+            category="ai",
+            level="warning",
+            service="ai",
+            module=__name__,
+            request_id=request_id,
+            user_id=user_id,
+            context={
+                "session_id": session_id,
+                "session_key": session_key,
+                "assistant_message_id": assistant_message_id,
+                "duration_ms": round((perf_counter() - started_at) * 1000),
+            },
+        )
         async with async_session_factory() as db:
             result = await db.execute(select(PlaygroundMessage).where(PlaygroundMessage.id == assistant_message_id))
             message = result.scalar_one_or_none()
@@ -715,6 +770,26 @@ async def _run_assistant_message(
                 await db.commit()
         raise
     except Exception as exc:
+        await emit_business_log(
+            logger,
+            event="ai.playground.run.failed",
+            message="Playground AI run failed",
+            category="ai",
+            level="error",
+            service="ai",
+            module=__name__,
+            request_id=request_id,
+            user_id=user_id,
+            context=exception_context(
+                exc,
+                {
+                    "session_id": session_id,
+                    "session_key": session_key,
+                    "assistant_message_id": assistant_message_id,
+                    "duration_ms": round((perf_counter() - started_at) * 1000),
+                },
+            ),
+        )
         error_message = _format_run_exception(exc)
         async with async_session_factory() as db:
             result = await db.execute(select(PlaygroundMessage).where(PlaygroundMessage.id == assistant_message_id))

backend/app/services/scheduler.py

@@ -13,6 +13,7 @@ from app.db.session import async_session_factory
 from app.core.time import to_iso8601_utc
 from app.models.datasource import DataSource
 from app.models.task import CollectionTask
+from app.services.business_logs import emit_business_log, emit_business_log_background, exception_context
 from app.services.collectors.registry import collector_registry
 from app.services.datasource_connectivity import (
     build_builtin_connectivity_checksum,
@@ -124,6 +125,15 @@ async def run_collector_task(collector_name: str):
                 event="collector.run.skipped_disabled",
                 context={"collector_name": collector_name},
             )
+            await emit_business_log(
+                logger,
+                event="collector.run.skipped_disabled",
+                message="Skipping disabled collector",
+                category="collector",
+                service="scheduler",
+                module=__name__,
+                context={"collector_name": collector_name, "datasource_id": datasource.id, "status": "skipped"},
+            )
             return
 
         running_result = await db.execute(
@@ -152,6 +162,21 @@ async def run_collector_task(collector_name: str):
                     event="collector.run.skipped_already_running",
                     context={"collector_name": collector_name, "task_id": existing_running.id},
                 )
+                await emit_business_log(
+                    logger,
+                    event="collector.run.skipped_already_running",
+                    message="Skipping collector trigger because task is already running",
+                    category="collector",
+                    level="warning",
+                    service="scheduler",
+                    module=__name__,
+                    context={
+                        "collector_name": collector_name,
+                        "datasource_id": datasource.id,
+                        "task_id": existing_running.id,
+                        "status": "skipped",
+                    },
+                )
                 return
 
             existing_error = (existing_running.error_message or "").strip()
@@ -173,6 +198,21 @@ async def run_collector_task(collector_name: str):
                 event="collector.run.stale_task_failed",
                 context={"collector_name": collector_name, "task_id": existing_running.id},
             )
+            await emit_business_log(
+                logger,
+                event="collector.run.stale_task_failed",
+                message="Marked stale running task as failed before rerun",
+                category="collector",
+                level="warning",
+                service="scheduler",
+                module=__name__,
+                context={
+                    "collector_name": collector_name,
+                    "datasource_id": datasource.id,
+                    "task_id": existing_running.id,
+                    "status": "failed",
+                },
+            )
 
         try:
             datasource_id = datasource.id
@@ -183,6 +223,15 @@ async def run_collector_task(collector_name: str):
                 event="collector.run.started",
                 context={"collector_name": collector_name, "datasource_id": datasource_id},
             )
+            await emit_business_log(
+                logger,
+                event="collector.run.scheduled_started",
+                message="Scheduler started collector run",
+                category="collector",
+                service="scheduler",
+                module=__name__,
+                context={"collector_name": collector_name, "datasource_id": datasource_id, "status": "running"},
+            )
             task_result = await collector.run(db)
             datasource = await db.get(DataSource, datasource_id)
             if datasource is None:
@@ -217,6 +266,20 @@ async def run_collector_task(collector_name: str):
                 event="collector.run.completed",
                 context={"collector_name": collector_name, "datasource_id": datasource_id, "result": task_result},
             )
+            await emit_business_log(
+                logger,
+                event="collector.run.scheduled_completed",
+                message="Scheduler completed collector run",
+                category="collector",
+                service="scheduler",
+                module=__name__,
+                context={
+                    "collector_name": collector_name,
+                    "datasource_id": datasource_id,
+                    "status": task_result.get("status"),
+                    "result": task_result,
+                },
+            )
         except asyncio.CancelledError:
             await db.rollback()
             datasource = await db.get(DataSource, datasource_id)
@@ -228,6 +291,16 @@ async def run_collector_task(collector_name: str):
                 event="collector.run.cancelled",
                 context={"collector_name": collector_name, "datasource_id": datasource.id},
             )
+            await emit_business_log(
+                logger,
+                event="collector.run.cancelled",
+                message="Collector cancelled by operator",
+                category="collector",
+                level="warning",
+                service="scheduler",
+                module=__name__,
+                context={"collector_name": collector_name, "datasource_id": datasource.id, "status": "cancelled"},
+            )
             raise
         except Exception as exc:
             await db.rollback()
@@ -240,6 +313,19 @@ async def run_collector_task(collector_name: str):
                 event="collector.run.failed",
                 context={"collector_name": collector_name, "datasource_id": datasource.id, "error": str(exc)},
             )
+            await emit_business_log(
+                logger,
+                event="collector.run.failed",
+                message="Collector failed",
+                category="collector",
+                level="error",
+                service="scheduler",
+                module=__name__,
+                context=exception_context(
+                    exc,
+                    {"collector_name": collector_name, "datasource_id": datasource.id, "status": "failed"},
+                ),
+            )
 
 
 async def cleanup_stale_running_tasks(max_age_hours: int = 2) -> int:
@@ -361,6 +447,16 @@ def run_collector_now(collector_name: str) -> bool:
             event="collector.trigger.skipped_already_running",
             context={"collector_name": collector_name},
         )
+        emit_business_log_background(
+            logger,
+            event="collector.trigger.skipped_already_running",
+            message="Collector is already running in-memory; skipping duplicate trigger",
+            category="collector",
+            level="warning",
+            service="scheduler",
+            module=__name__,
+            context={"collector_name": collector_name, "status": "skipped"},
+        )
         return False
 
     try:
@@ -378,6 +474,15 @@ def run_collector_now(collector_name: str) -> bool:
             event="collector.trigger.started",
             context={"collector_name": collector_name},
         )
+        emit_business_log_background(
+            logger,
+            event="collector.trigger.started",
+            message="Triggered collector",
+            category="collector",
+            service="scheduler",
+            module=__name__,
+            context={"collector_name": collector_name, "status": "queued"},
+        )
         return True
     except Exception as exc:
         logger.error_event(
@@ -385,6 +490,16 @@ def run_collector_now(collector_name: str) -> bool:
             event="collector.trigger.failed",
             context={"collector_name": collector_name, "error": str(exc)},
         )
+        emit_business_log_background(
+            logger,
+            event="collector.trigger.failed",
+            message="Failed to trigger collector",
+            category="collector",
+            level="error",
+            service="scheduler",
+            module=__name__,
+            context=exception_context(exc, {"collector_name": collector_name, "status": "failed"}),
+        )
         return False
 
 

backend/app/services/system_logs.py

@@ -1,6 +1,7 @@
 from __future__ import annotations
 
 import json
+import os
 import re
 import shutil
 import subprocess
@@ -86,6 +87,7 @@ class LogSource:
     status: str = "ok"
     buffer_key: str | None = None
     container_name: str | None = None
+    fallback_locations: tuple[str, ...] = ()
 
 
 @dataclass
@@ -104,22 +106,38 @@ class DailyLogMarker:
     dominant_level: str
 
 
+def _planet_state_dir() -> Path:
+    configured = os.getenv("PLANET_STATE_DIR")
+    if configured:
+        return Path(configured).expanduser()
+    xdg_state = os.getenv("XDG_STATE_HOME")
+    if xdg_state:
+        return Path(xdg_state).expanduser() / "planet"
+    return Path.home() / ".local" / "state" / "planet"
+
+
+def _state_log_path(filename: str) -> str:
+    return str(_planet_state_dir() / filename)
+
+
 LOG_SOURCES: dict[str, LogSource] = {
     "backend": LogSource(
         source_id="backend",
         name="后端服务",
         kind="file",
-        location="/tmp/planet_backend.log",
+        location=_state_log_path("backend.log"),
         description="FastAPI 后端、调度器和采集任务共享日志。",
         category="service",
+        fallback_locations=("/tmp/planet_backend.log",),
     ),
     "frontend": LogSource(
         source_id="frontend",
         name="前端开发服务",
         kind="file",
-        location="/tmp/planet_frontend.log",
+        location=_state_log_path("frontend.log"),
         description="控制台与 Earth 前端开发服务输出。",
         category="service",
+        fallback_locations=("/tmp/planet_frontend.log",),
     ),
     "ai-provider": LogSource(
         source_id="ai-provider",
@@ -164,9 +182,18 @@ def normalize_log_levels(level: str | None = None, levels: str | None = None) ->
     return tuple(normalized_levels)
 
 
+def resolve_file_log_path(source: LogSource) -> Path:
+    primary = Path(source.location).expanduser()
+    candidates = (primary, *(Path(item).expanduser() for item in source.fallback_locations))
+    for candidate in candidates:
+        if candidate.exists():
+            return candidate
+    return primary
+
+
 def get_source_status(source: LogSource) -> str:
     if source.kind == "file":
-        path = Path(source.location)
+        path = resolve_file_log_path(source)
         if not path.exists():
             return "missing"
         return "ok" if path.stat().st_size > 0 else "empty"
@@ -190,7 +217,7 @@ def list_log_sources() -> list[dict[str, str]]:
                 "source_id": source.source_id,
                 "name": source.name,
                 "kind": source.kind,
-                "location": source.location,
+                "location": str(resolve_file_log_path(source)) if source.kind == "file" else source.location,
                 "description": source.description,
                 "category": source.category,
                 "status": get_source_status(source),
@@ -339,7 +366,7 @@ def build_buffer_entry(payload: dict[str, Any]) -> StructuredLogEntry:
 
 
 def read_file_entries(source: LogSource, scan_limit: int) -> list[StructuredLogEntry]:
-    path = Path(source.location)
+    path = resolve_file_log_path(source)
     if not path.exists():
         return []
     with path.open("r", encoding="utf-8", errors="replace") as handle:
@@ -511,7 +538,7 @@ def read_log_snapshot(
         "source_id": source.source_id,
         "name": source.name,
         "kind": source.kind,
-        "location": source.location,
+        "location": str(resolve_file_log_path(source)) if source.kind == "file" else source.location,
         "description": source.description,
         "category": source.category,
         "status": get_source_status(source),

backend/app/services/tv_streams.py

@@ -385,13 +385,18 @@ def build_public_tv_payload(
     settings_payload: dict[str, Any],
     collected_sources: list[dict[str, Any]],
 ) -> dict[str, Any]:
+    configured_by_id = {
+        source["id"]: source
+        for source in settings_payload["sources"]
+        if source.get("id")
+    }
     configured_sources = [
         source for source in settings_payload["sources"] if source["is_enabled"]
     ]
 
     merged_by_id = {source["id"]: source for source in configured_sources}
     for source in collected_sources:
-        if source["id"] in merged_by_id or not source["is_enabled"]:
+        if source["id"] in configured_by_id or not source["is_enabled"]:
             continue
         merged_by_id[source["id"]] = source
 

backend/scripts/create_admin.py

@@ -18,7 +18,7 @@ async def create_admin():
 
         if existing_user:
             print("用户 linkong 已存在，更新密码...")
-            existing_user.set_password("12345678")
+            existing_user.set_password("LK12345678")
             existing_user.role = "super_admin"
             existing_user.email = "linkong@planet.local"
         else:
@@ -26,7 +26,7 @@ async def create_admin():
             user = User(
                 username="linkong",
                 email="linkong@planet.local",
-                password_hash=get_password_hash("12345678"),
+                password_hash=get_password_hash("LK12345678"),
                 role="super_admin",
                 is_active=True,
             )

backend/scripts/init_admin.py

@@ -19,7 +19,7 @@ DEFAULT_LOGIN_USERS = (
     {
         "username": "linkong",
         "email": "linkong@planet.local",
-        "password": "12345678",
+        "password": "LK12345678",
         "role": "super_admin",
     },
 )

backend/scripts/init_db.py

@@ -22,7 +22,7 @@ DEFAULT_LOGIN_USERS = (
     {
         "username": "linkong",
         "email": "linkong@planet.local",
-        "password": "12345678",
+        "password": "LK12345678",
         "role": "super_admin",
     },
 )

backend/tests/test_ai_observability.py

@@ -0,0 +1,128 @@
+import pytest
+from fastapi import HTTPException
+
+from app.schemas.ai import SituationalAnalysisRequest
+from app.services.ai_tools import web_search as web_search_module
+from app.services.ai_tools.schemas import SearchEvidence, WebSearchConfig, WebSearchProviderConfig
+from app.services.ai_tools.web_search import WebSearchClient
+from app.services import ai_client as ai_client_module
+from app.services.ai_client import AIProviderClient
+
+
+@pytest.mark.asyncio
+async def test_ai_client_analyze_logs_summary_without_prompt(monkeypatch):
+    events = []
+
+    async def fake_emit_business_log(_logger, **payload):
+        events.append(payload)
+
+    async def fake_request(self, method, path, json=None, request_id=None, operation="request", payload_summary=None):
+        return {
+            "provider": "test-provider",
+            "model": "test-model",
+            "content": "ok",
+            "content_blocks": [],
+            "text_blocks": ["ok"],
+            "thinking_blocks": [],
+            "raw_response": {},
+        }
+
+    monkeypatch.setattr(ai_client_module, "emit_business_log", fake_emit_business_log)
+    monkeypatch.setattr(AIProviderClient, "_request", fake_request)
+
+    client = AIProviderClient(
+        service_url="http://provider.test",
+        llm_config={"provider": "openai", "provider_api": "openai-completions", "model": "gpt-test", "api_key": "sk-secret"},
+    )
+    result = await client.analyze(
+        SituationalAnalysisRequest(
+            title="Sensitive title",
+            objective="Do not store this full prompt",
+            observations=["secret observation"],
+            constraints=["secret constraint"],
+            context={"source": "test", "private": "value"},
+        ),
+        request_id="req-ai-test",
+    )
+
+    assert result.model == "test-model"
+    assert [event["event"] for event in events] == [
+        "ai.provider.analyze.start",
+        "ai.provider.analyze.success",
+    ]
+    serialized = str(events)
+    assert "Do not store this full prompt" not in serialized
+    assert "secret observation" not in serialized
+    assert "sk-secret" not in serialized
+    start_context = events[0]["context"]
+    assert start_context["model"] == "gpt-test"
+    assert start_context["input_summary"]["objective_length"] == len("Do not store this full prompt")
+    assert start_context["input_summary"]["observation_count"] == 1
+    assert start_context["input_summary"]["context_keys"] == ["private", "source"]
+
+
+@pytest.mark.asyncio
+async def test_ai_client_analyze_logs_failure(monkeypatch):
+    events = []
+
+    async def fake_emit_business_log(_logger, **payload):
+        events.append(payload)
+
+    async def fake_request(self, method, path, json=None, request_id=None, operation="request", payload_summary=None):
+        raise HTTPException(status_code=502, detail="provider failed")
+
+    monkeypatch.setattr(ai_client_module, "emit_business_log", fake_emit_business_log)
+    monkeypatch.setattr(AIProviderClient, "_request", fake_request)
+
+    client = AIProviderClient(service_url="http://provider.test", llm_config={"provider": "openai", "model": "gpt-test"})
+
+    with pytest.raises(HTTPException):
+        await client.analyze(
+            SituationalAnalysisRequest(title="T", objective="O", observations=["one"]),
+            request_id="req-ai-fail",
+        )
+
+    assert events[-1]["event"] == "ai.provider.analyze.failed"
+    assert events[-1]["level"] == "error"
+    assert events[-1]["context"]["error_type"] == "HTTPException"
+
+
+@pytest.mark.asyncio
+async def test_web_search_logs_query_hash_without_query(monkeypatch):
+    events = []
+
+    async def fake_emit_business_log(_logger, **payload):
+        events.append(payload)
+
+    async def fake_search_tavily(self, config, query, max_results, domains, freshness_days):
+        return [
+            SearchEvidence(
+                title="Example",
+                url="https://example.test",
+                snippet="result",
+                source_provider="tavily",
+            )
+        ]
+
+    monkeypatch.setattr(web_search_module, "emit_business_log", fake_emit_business_log)
+    monkeypatch.setattr(WebSearchClient, "_search_tavily", fake_search_tavily)
+
+    client = WebSearchClient(
+        WebSearchConfig(
+            enabled=True,
+            default_provider="tavily",
+            providers={"tavily": WebSearchProviderConfig(provider="tavily", api_key="secret-key")},
+        )
+    )
+    results = await client.search("secret query text", max_results=1)
+
+    assert len(results) == 1
+    assert [event["event"] for event in events] == [
+        "ai_tool.web_search.start",
+        "ai_tool.web_search.success",
+    ]
+    serialized = str(events)
+    assert "secret query text" not in serialized
+    assert "secret-key" not in serialized
+    assert events[0]["context"]["query_length"] == len("secret query text")
+    assert events[1]["context"]["result_count"] == 1

backend/tests/test_collectors.py

@@ -1,9 +1,13 @@
 """Unit tests for data collectors"""
 
+import json
+
 import pytest
 from unittest.mock import AsyncMock, patch
 
 from app.core.datasource_defaults import DEFAULT_DATASOURCES
+from app.services.collectors.celestrak import CelesTrakTLECollector
+from app.services.collectors.downloads import DownloadHTTPStatusError, ResumableFileDownloader
 from app.services.credential_guides import DEFAULT_CREDENTIAL_GUIDES
 from app.services.collectors.top500 import TOP500Collector
 from app.services.collectors.registry import collector_registry
@@ -149,6 +153,172 @@ class TestHTTPCollector:
         assert callable(collector.parse_response)
 
 
+class TestCelesTrakTLECollector:
+    def test_transform_uses_norad_as_source_id_and_preserves_starlink_group(self):
+        collector = CelesTrakTLECollector()
+        result = collector.transform([
+            {
+                "NORAD_CAT_ID": 44720,
+                "OBJECT_NAME": "STARLINK-1000",
+                "OBJECT_ID": "2019-029AZ",
+                "EPOCH": "2026-03-13T00:00:00Z",
+                "MEAN_MOTION": 15.79234567,
+                "ECCENTRICITY": 0.0001234,
+                "INCLINATION": 53.0,
+                "RA_OF_ASC_NODE": 10.0,
+                "ARG_OF_PERICENTER": 20.0,
+                "MEAN_ANOMALY": 30.0,
+                "_celestrak_query_group": "active",
+                "_celestrak_source_url": "https://celestrak.example/gp.php?GROUP=active&FORMAT=json",
+            }
+        ])
+
+        assert result[0]["source_id"] == "44720"
+        assert result[0]["metadata"]["constellation_group"] == "starlink"
+        assert result[0]["metadata"]["celestrak_query_group"] == "active"
+        assert result[0]["metadata"]["norad_cat_id"] == 44720
+        assert result[0]["metadata"]["tle_line1"]
+        assert result[0]["metadata"]["tle_line2"]
+
+    def test_load_active_payload_rejects_invalid_records(self, tmp_path):
+        collector = CelesTrakTLECollector()
+        payload_path = tmp_path / "active.json"
+        payload_path.write_text(json.dumps([{"OBJECT_NAME": "missing norad"}]), encoding="utf-8")
+
+        with pytest.raises(RuntimeError, match="invalid record"):
+            collector._load_active_payload(payload_path)
+
+    def test_load_active_payload_accepts_complete_array(self, tmp_path):
+        collector = CelesTrakTLECollector()
+        payload_path = tmp_path / "active.json"
+        payload_path.write_text(
+            json.dumps([{"NORAD_CAT_ID": 25544, "OBJECT_NAME": "ISS (ZARYA)"}]),
+            encoding="utf-8",
+        )
+
+        records = collector._load_active_payload(payload_path)
+
+        assert records == [{"NORAD_CAT_ID": 25544, "OBJECT_NAME": "ISS (ZARYA)"}]
+
+    @pytest.mark.asyncio
+    async def test_fetch_retries_and_raises_instead_of_returning_partial_data(self, monkeypatch, tmp_path):
+        collector = CelesTrakTLECollector()
+        collector._resolved_url = "https://celestrak.example/NORAD/elements/gp.php"
+        attempts = 0
+
+        async def fake_download_file(*args, **kwargs):
+            nonlocal attempts
+            attempts += 1
+            raise RuntimeError("network interrupted")
+
+        async def fake_emit_business_log(*args, **kwargs):
+            return None
+
+        monkeypatch.setattr(collector._downloader, "download_file", fake_download_file)
+        monkeypatch.setattr("app.services.collectors.celestrak.emit_business_log", fake_emit_business_log)
+        monkeypatch.setattr("app.services.collectors.celestrak.asyncio.sleep", AsyncMock())
+
+        with pytest.raises(RuntimeError, match="failed after retries"):
+            await collector.fetch()
+
+        assert attempts == 3
+
+    @pytest.mark.asyncio
+    async def test_fetch_uses_cache_when_celestrak_reports_not_updated(self, monkeypatch, tmp_path):
+        collector = CelesTrakTLECollector()
+        collector._resolved_url = "https://celestrak.example/NORAD/elements/gp.php"
+        collector._downloader = ResumableFileDownloader(cache_namespace="celestrak-test", cache_root=tmp_path)
+        url = collector._active_url()
+        cached_path = collector._downloader.cached_file_path(url, ".json")
+        cached_path.parent.mkdir(parents=True, exist_ok=True)
+        cached_path.write_text(
+            json.dumps([{"NORAD_CAT_ID": 25544, "OBJECT_NAME": "ISS (ZARYA)"}]),
+            encoding="utf-8",
+        )
+
+        async def fake_download_file(*args, **kwargs):
+            raise DownloadHTTPStatusError(
+                url=url,
+                status_code=403,
+                body="GP data has not updated since your last successful download of GROUP=active.",
+            )
+
+        async def fake_emit_business_log(*args, **kwargs):
+            return None
+
+        monkeypatch.setattr(collector._downloader, "download_file", fake_download_file)
+        monkeypatch.setattr("app.services.collectors.celestrak.emit_business_log", fake_emit_business_log)
+
+        records = await collector.fetch()
+
+        assert records[0]["NORAD_CAT_ID"] == 25544
+        assert records[0]["_celestrak_query_group"] == "active"
+
+    @pytest.mark.asyncio
+    async def test_fetch_not_updated_without_cache_does_not_retry(self, monkeypatch, tmp_path):
+        collector = CelesTrakTLECollector()
+        collector._resolved_url = "https://celestrak.example/NORAD/elements/gp.php"
+        collector._downloader = ResumableFileDownloader(cache_namespace="celestrak-test", cache_root=tmp_path)
+        attempts = 0
+
+        async def fake_download_file(*args, **kwargs):
+            nonlocal attempts
+            attempts += 1
+            raise DownloadHTTPStatusError(
+                url=collector._active_url(),
+                status_code=403,
+                body="GP data has not updated since your last successful download of GROUP=active.",
+            )
+
+        async def fake_emit_business_log(*args, **kwargs):
+            return None
+
+        monkeypatch.setattr(collector._downloader, "download_file", fake_download_file)
+        monkeypatch.setattr("app.services.collectors.celestrak.FALLBACK_GROUPS", ("starlink",))
+        monkeypatch.setattr("app.services.collectors.celestrak.emit_business_log", fake_emit_business_log)
+        monkeypatch.setattr("app.services.collectors.celestrak.asyncio.sleep", AsyncMock())
+
+        with pytest.raises(RuntimeError, match="fallback group mode failed"):
+            await collector.fetch()
+
+        assert attempts == 2
+
+    @pytest.mark.asyncio
+    async def test_fetch_falls_back_to_all_groups_when_active_not_updated_without_cache(self, monkeypatch, tmp_path):
+        collector = CelesTrakTLECollector()
+        collector._resolved_url = "https://celestrak.example/NORAD/elements/gp.php"
+        collector._downloader = ResumableFileDownloader(cache_namespace="celestrak-test", cache_root=tmp_path)
+        payload_by_group = {
+            "starlink": [{"NORAD_CAT_ID": 100, "OBJECT_NAME": "STARLINK-100"}],
+            "gps-ops": [{"NORAD_CAT_ID": 200, "OBJECT_NAME": "GPS BIIR-2"}],
+        }
+
+        async def fake_download_file(_client, url, **_kwargs):
+            if "GROUP=active" in url:
+                raise DownloadHTTPStatusError(
+                    url=url,
+                    status_code=403,
+                    body="GP data has not updated since your last successful download of GROUP=active.",
+                )
+            group = "starlink" if "GROUP=starlink" in url else "gps-ops"
+            path = tmp_path / f"{group}.json"
+            path.write_text(json.dumps(payload_by_group[group]), encoding="utf-8")
+            return path
+
+        async def fake_emit_business_log(*args, **kwargs):
+            return None
+
+        monkeypatch.setattr(collector._downloader, "download_file", fake_download_file)
+        monkeypatch.setattr("app.services.collectors.celestrak.FALLBACK_GROUPS", tuple(payload_by_group))
+        monkeypatch.setattr("app.services.collectors.celestrak.emit_business_log", fake_emit_business_log)
+
+        records = await collector.fetch()
+
+        assert [item["NORAD_CAT_ID"] for item in records] == [100, 200]
+        assert records[0]["_celestrak_query_group"] == "starlink"
+        assert records[1]["_celestrak_group"] == "gps-ops"
+
+
 def test_aisstream_collector_is_registered():
     collector = collector_registry.get("aisstream_vessels")
 

backend/tests/test_datasources_batch.py

@@ -1,9 +1,12 @@
+import asyncio
+import inspect
 from datetime import datetime, timedelta, timezone
 
-import pytest
-
 from app.api.v1 import datasources as datasources_api
 from app.models.datasource import DataSource
+from app.models.task import CollectionTask
+from app.services import data_jobs
+from app.services import earth_layer_cache as earth_cache
 
 
 def make_datasource(
@@ -53,34 +56,166 @@ def test_filter_datasources_by_product_status_and_collected_state():
     assert filtered == [vessels]
 
 
-@pytest.mark.asyncio
-async def test_trigger_datasource_batch_skips_disabled_and_frequency_window(monkeypatch):
-    now = datetime.now(timezone.utc)
-    disabled = make_datasource(1, "aisstream_vessels", is_active=False)
-    not_due = make_datasource(2, "telegeography_cables", last_run_at=now, frequency_minutes=120)
-    due = make_datasource(3, "ris_live_bgp", last_run_at=now - timedelta(hours=2))
-    triggered_sources: list[str] = []
+def test_serialize_datasource_row_can_skip_endpoint_resolution():
+    datasource = make_datasource(7, "arcgis_cables", last_status="success", module="L2")
 
-    async def fake_running_tasks(_db, _ids):
-        return {}
+    class ExplodingConfig:
+        def get_yaml_url(self, _source):
+            raise AssertionError("endpoint resolution should be skipped")
 
-    async def fake_latest_task_ids(_db, _ids):
-        return {}
-
-    monkeypatch.setattr(datasources_api, "_load_latest_running_tasks", fake_running_tasks)
-    monkeypatch.setattr(datasources_api, "_load_latest_task_ids", fake_latest_task_ids)
-    monkeypatch.setattr(
-        datasources_api,
-        "run_collector_now",
-        lambda source: triggered_sources.append(source) or True,
+    row = datasources_api.serialize_datasource_row(
+        datasource,
+        running_tasks={},
+        latest_tasks={},
+        record_counts={"arcgis_cables": 343},
+        endpoint_overrides={},
+        config=ExplodingConfig(),
+        include_endpoint=False,
     )
 
-    result = await datasources_api._trigger_datasource_batch(
-        object(),
-        [disabled, not_due, due],
-        force=False,
+    assert row["id"] == 7
+    assert row["source"] == "arcgis_cables"
+    assert row["module"] == "L2"
+    assert row["last_status"] == "success"
+    assert row["collected_records"] == 343
+    assert row["has_collected_data"] is True
+    assert "endpoint" not in row
+
+
+def test_serialize_datasource_row_includes_endpoint_when_requested():
+    datasource = make_datasource(8, "arcgis_landing_points")
+
+    class Config:
+        def get_yaml_url(self, source):
+            return f"https://example.test/{source}"
+
+    row = datasources_api.serialize_datasource_row(
+        datasource,
+        running_tasks={},
+        latest_tasks={},
+        record_counts={},
+        endpoint_overrides={},
+        config=Config(),
+        include_endpoint=True,
     )
 
-    assert [item["source"] for item in result["triggered"]] == ["ris_live_bgp"]
-    assert {item["reason"] for item in result["skipped"]} == {"disabled", "within_frequency_window"}
-    assert triggered_sources == ["ris_live_bgp"]
+    assert row["endpoint"] == "https://example.test/arcgis_landing_points"
+
+
+def test_serialize_datasource_row_separates_collector_running_from_delete_task():
+    datasource = make_datasource(9, "top500", last_status="success")
+    task = CollectionTask(
+        id=44,
+        datasource_id=9,
+        source="top500",
+        task_type="clear_data",
+        status="running",
+        phase="clearing_data",
+    )
+
+    row = datasources_api.serialize_datasource_row(
+        datasource,
+        running_tasks={9: task},
+        latest_tasks={9: task},
+        record_counts={"top500": 500},
+        endpoint_overrides={},
+        config=object(),
+        include_endpoint=False,
+    )
+
+    assert row["is_task_active"] is True
+    assert row["is_running"] is False
+    assert row["task_type"] == "clear_data"
+    assert row["task_status"] == "running"
+
+
+def test_cancel_queued_delete_task_finishes_immediately(monkeypatch):
+    task = CollectionTask(
+        id=45,
+        datasource_id=9,
+        source="top500",
+        task_type="clear_data",
+        status="queued",
+        phase="queued",
+    )
+
+    class FakeDb:
+        async def commit(self):
+            return None
+
+        async def refresh(self, _task):
+            return None
+
+    async def fake_broadcast(_task):
+        return None
+
+    monkeypatch.setattr(data_jobs, "_broadcast_task_update", fake_broadcast)
+
+    async def run():
+        return await data_jobs.request_cancel_datasource_task(FakeDb(), task)
+
+    result = asyncio.run(run())
+
+    assert result.status == "cancelled"
+    assert result.phase == "cancelled"
+    assert result.completed_at is not None
+
+
+def test_clear_data_job_relies_on_db_outbox_instead_of_extra_earth_refresh_task():
+    source = inspect.getsource(data_jobs._run_clear_data_job)
+
+    assert "enqueue_earth_refresh_job" not in source
+
+
+def test_invalidate_earth_layer_cache_for_source_covers_datasource_aliases(monkeypatch):
+    patterns: list[str] = []
+
+    def fake_delete_pattern(pattern: str) -> int:
+        patterns.append(pattern)
+        return 1
+
+    monkeypatch.setattr(earth_cache.earth_layer_cache, "delete_pattern", fake_delete_pattern)
+
+    deleted = earth_cache.invalidate_earth_layer_cache_for_source("telegeography_cables")
+
+    assert deleted == 3
+    assert patterns == [
+        "earth:layer:v1:cables*",
+        "earth:layer:v1:landing-points*",
+        "earth:layer:v1:summary*",
+    ]
+
+
+def test_trigger_datasource_batch_skips_disabled_and_frequency_window(monkeypatch):
+    async def run():
+        now = datetime.now(timezone.utc)
+        disabled = make_datasource(1, "aisstream_vessels", is_active=False)
+        not_due = make_datasource(2, "telegeography_cables", last_run_at=now, frequency_minutes=120)
+        due = make_datasource(3, "ris_live_bgp", last_run_at=now - timedelta(hours=2))
+        queued_sources: list[str] = []
+
+        async def fake_running_tasks(_db, _ids):
+            return {}
+
+        async def fake_enqueue(_db, datasource, task_type, **_kwargs):
+            queued_sources.append(datasource.source)
+            return CollectionTask(id=100 + datasource.id, datasource_id=datasource.id, source=datasource.source, task_type=task_type, status="queued")
+
+        monkeypatch.setattr(datasources_api, "_load_latest_running_tasks", fake_running_tasks)
+        monkeypatch.setattr(
+            datasources_api,
+            "enqueue_datasource_job",
+            fake_enqueue,
+        )
+
+        result = await datasources_api._trigger_datasource_batch(
+            object(),
+            [disabled, not_due, due],
+            force=False,
+        )
+
+        assert [item["source"] for item in result["triggered"]] == ["ris_live_bgp"]
+        assert {item["reason"] for item in result["skipped"]} == {"disabled", "within_frequency_window"}
+        assert queued_sources == ["ris_live_bgp"]
+
+    asyncio.run(run())

backend/tests/test_docs_gatekeeper.py

@@ -1,11 +1,15 @@
 """Docs Gatekeeper API tests."""
 
+import re
+from pathlib import Path
+
 import pytest
 from httpx import ASGITransport, AsyncClient
 
 from app.api.v1 import docs as docs_api
 from app.main import app
 from app.models.user import User
+from app.services.docs_gatekeeper import DOCS_METADATA
 
 
 def make_user(role: str = "viewer", groups: list[str] | None = None) -> User:
@@ -43,22 +47,45 @@ async def test_public_catalog_only_for_anonymous_user():
     assert response.status_code == 200
     items = response.json()["items"]
     assert {item["access"] for item in items} == {"public"}
-    assert {item["slug"] for item in items if item["lang"] == "zh"} == {
+    zh_items = [item for item in items if item["lang"] == "zh"]
+    assert [item["slug"] for item in zh_items] == [
         "overview",
-        "quickstart",
         "manual",
+        "quickstart",
         "faq",
-        "location-pipeline-user",
-    }
+    ]
+
+
+@pytest.mark.asyncio
+async def test_developer_catalog_includes_architecture_and_frontend_reference_docs():
+    response = await get_json(
+        "/api/v1/docs/catalog",
+        make_user(role="viewer", groups=["docs_developer"]),
+    )
+
+    assert response.status_code == 200
+    zh_slugs = {item["slug"] for item in response.json()["items"] if item["lang"] == "zh"}
+    zh_items = [item for item in response.json()["items"] if item["lang"] == "zh"]
+    assert [item["group"] for item in zh_items[:4]] == ["Overview", "Manual", "Manual", "Manual"]
+    assert zh_items[4]["group"] == "Architecture"
+    assert "platform-data-flows" in zh_slugs
+    assert "naming-glossary" in zh_slugs
+    assert "tactile-ui-components" in zh_slugs
 
 
 @pytest.mark.asyncio
 async def test_anonymous_can_read_public_doc():
     response = await get_json("/api/v1/docs/zh/quickstart")
+    manual_response = await get_json("/api/v1/docs/zh/manual")
+    overview_response = await get_json("/api/v1/docs/zh/overview")
 
     assert response.status_code == 200
     assert response.json()["access"] == "public"
     assert "快速开始" in response.json()["markdown"]
+    assert manual_response.status_code == 200
+    assert manual_response.json()["access"] == "public"
+    assert overview_response.status_code == 200
+    assert overview_response.json()["access"] == "public"
 
 
 @pytest.mark.asyncio
@@ -83,10 +110,16 @@ async def test_developer_group_can_read_developer_but_not_admin_doc():
     user = make_user(role="viewer", groups=["docs_developer"])
 
     developer_response = await get_json("/api/v1/docs/zh/backend-collectors", user)
+    tactile_response = await get_json("/api/v1/docs/zh/tactile-ui-components", user)
+    glossary_response = await get_json("/api/v1/docs/zh/naming-glossary", user)
     admin_response = await get_json("/api/v1/docs/zh/backend-system-service-control", user)
 
     assert developer_response.status_code == 200
     assert developer_response.json()["access"] == "docs_developer"
+    assert tactile_response.status_code == 200
+    assert tactile_response.json()["access"] == "docs_developer"
+    assert glossary_response.status_code == 200
+    assert glossary_response.json()["access"] == "docs_developer"
     assert admin_response.status_code == 403
 
 
@@ -114,3 +147,29 @@ async def test_unknown_language_slug_and_path_traversal_do_not_read_files():
     assert bad_lang.status_code == 404
     assert bad_slug.status_code == 404
     assert traversal.status_code == 404
+
+
+def test_public_docs_markdown_links_do_not_create_missing_docs_routes():
+    repo_root = Path(__file__).resolve().parents[2]
+    technical_root = repo_root / "docs" / "technical"
+    registered_filenames = {entry.filename for entry in DOCS_METADATA}
+    problems: list[str] = []
+
+    for markdown_path in sorted(technical_root.glob("*/*.md")):
+        markdown = markdown_path.read_text(encoding="utf-8")
+        for match in re.finditer(r"\[([^\]]+)]\(([^)]+\.md(?:#[^)]+)?)\)", markdown):
+            label, href = match.group(1), match.group(2)
+            if href.startswith(("http://", "https://", "mailto:")):
+                continue
+
+            href_without_hash = href.split("#", 1)[0].replace("\\", "/")
+            filename = Path(href_without_hash).name
+            if "/docs/technical/" in href_without_hash:
+                if filename not in registered_filenames:
+                    problems.append(f"{markdown_path.relative_to(repo_root)} links unregistered public doc {href!r} ({label})")
+                continue
+
+            if href_without_hash.endswith(".md"):
+                problems.append(f"{markdown_path.relative_to(repo_root)} links non-public markdown {href!r} ({label})")
+
+    assert problems == []

backend/tests/test_earth_db_change_listener.py

@@ -0,0 +1,454 @@
+import asyncio
+import json
+
+from app.services.earth_db_change_listener import (
+    EarthDbChangeDispatcher,
+    EarthDbChangeListener,
+    build_earth_update_from_db_payload,
+    normalize_asyncpg_dsn,
+)
+
+
+def test_normalize_asyncpg_dsn_strips_sqlalchemy_driver():
+    assert (
+        normalize_asyncpg_dsn("postgresql+asyncpg://postgres:postgres@localhost:5432/planet_db")
+        == "postgresql://postgres:postgres@localhost:5432/planet_db"
+    )
+
+
+def test_build_earth_update_maps_known_sources_to_layers():
+    satellite_update = build_earth_update_from_db_payload(
+        {"source": "celestrak_tle", "operation": "DELETE", "entity_key": "25544"}
+    )
+    cable_update = build_earth_update_from_db_payload(
+        {"source": "arcgis_cables", "operation": "UPDATE", "entity_key": "cable-1"}
+    )
+    compute_update = build_earth_update_from_db_payload(
+        {"source": "top500", "operation": "DELETE", "entity_key": None}
+    )
+    landing_update = build_earth_update_from_db_payload(
+        {"source": "telegeography_landing", "operation": "DELETE", "entity_key": None}
+    )
+
+    assert satellite_update is not None
+    assert satellite_update["layers"] == ["satellites"]
+    assert cable_update is not None
+    assert cable_update["layers"] == ["cables"]
+    assert compute_update is not None
+    assert compute_update["layers"] == ["computeCenters"]
+    assert landing_update is not None
+    assert landing_update["layers"] == ["cables"]
+    assert landing_update["refresh_strategy"] == "clear_then_reload"
+
+
+def test_build_earth_update_maps_derived_tables_to_layers():
+    bgp_update = build_earth_update_from_db_payload(
+        {"table": "bgp_anomalies", "source": "ris_live_bgp", "operation": "DELETE", "entity_key": "a1"}
+    )
+    compute_update = build_earth_update_from_db_payload(
+        {"table": "compute_center_locations", "source": "top500", "operation": "UPDATE", "entity_key": "top500_1"}
+    )
+    vessel_update = build_earth_update_from_db_payload(
+        {"table": "vessel_position", "operation": "DELETE", "entity_key": "123456789"}
+    )
+
+    assert bgp_update is not None
+    assert bgp_update["source"] == "ris_live_bgp"
+    assert bgp_update["table"] == "bgp_anomalies"
+    assert bgp_update["layers"] == ["bgp"]
+    assert bgp_update["refresh_strategy"] == "clear_then_reload"
+    assert compute_update is not None
+    assert compute_update["layers"] == ["computeCenters"]
+    assert compute_update["refresh_strategy"] == "reload"
+    assert vessel_update is not None
+    assert vessel_update["source"] == "vessel_position"
+    assert vessel_update["layers"] == ["vessels"]
+
+
+def test_build_earth_update_maps_interactable_delete_to_delta():
+    update = build_earth_update_from_db_payload(
+        {
+            "table": "earth_interactables",
+            "operation": "DELETE",
+            "entity_keys": ["note-1"],
+            "records_processed": 1,
+        }
+    )
+
+    assert update is not None
+    assert update["entity"] == "interactable"
+    assert update["action"] == "deleted"
+    assert update["ids"] == ["note-1"]
+    assert update["layers"] == ["interactables"]
+    assert update["refresh_strategy"] == "delta"
+
+
+def test_build_earth_update_ignores_unmapped_sources():
+    assert build_earth_update_from_db_payload({"source": "not_for_earth"}) is None
+
+
+def test_dispatcher_debounces_same_source_notifications():
+    async def run():
+        broadcasts = []
+        invalidated = []
+
+        async def broadcast(payload):
+            broadcasts.append(payload)
+
+        def invalidate(source):
+            invalidated.append(source)
+            return 2
+
+        dispatcher = EarthDbChangeDispatcher(
+            broadcast_earth_update=broadcast,
+            invalidate_cache=invalidate,
+            debounce_seconds=10,
+        )
+        dispatcher.handle_payload(
+            {
+                "table": "collected_data",
+                "operation": "INSERT",
+                "source": "arcgis_cables",
+                "entity_key": "cable-1",
+                "occurred_at": "2026-05-22T00:00:00Z",
+            }
+        )
+        dispatcher.handle_payload(
+            {
+                "table": "collected_data",
+                "operation": "UPDATE",
+                "source": "arcgis_cables",
+                "entity_key": "cable-2",
+                "occurred_at": "2026-05-22T00:00:01Z",
+            }
+        )
+
+        await dispatcher.flush_all()
+
+        assert invalidated == ["arcgis_cables"]
+        assert len(broadcasts) == 1
+        assert broadcasts[0]["action"] == "database_changed"
+        assert broadcasts[0]["source"] == "arcgis_cables"
+        assert broadcasts[0]["layers"] == ["cables"]
+        assert broadcasts[0]["records_processed"] == 2
+        assert broadcasts[0]["operations"] == ["INSERT", "UPDATE"]
+        assert broadcasts[0]["entity_keys"] == ["cable-1", "cable-2"]
+        assert broadcasts[0]["cache_entries_invalidated"] == 2
+
+    asyncio.run(run())
+
+
+def test_dispatcher_handles_delete_notification_as_earth_update():
+    async def run():
+        broadcasts = []
+
+        async def broadcast(payload):
+            broadcasts.append(payload)
+
+        dispatcher = EarthDbChangeDispatcher(
+            broadcast_earth_update=broadcast,
+            invalidate_cache=lambda source: 1,
+            debounce_seconds=10,
+        )
+        accepted = dispatcher.handle_notification(
+            json.dumps(
+                {
+                    "table": "collected_data",
+                    "operation": "DELETE",
+                    "source": "celestrak_tle",
+                    "entity_key": "sat:25544",
+                    "occurred_at": "2026-05-22T00:00:00Z",
+                }
+            )
+        )
+
+        await dispatcher.flush_all()
+
+        assert accepted is True
+        assert len(broadcasts) == 1
+        assert broadcasts[0]["action"] == "database_changed"
+        assert broadcasts[0]["source"] == "celestrak_tle"
+        assert broadcasts[0]["layers"] == ["satellites"]
+        assert broadcasts[0]["operation"] == "DELETE"
+        assert broadcasts[0]["refresh_strategy"] == "clear_then_reload"
+
+    asyncio.run(run())
+
+
+def test_dispatcher_debounces_bgp_derived_table_events_by_source():
+    async def run():
+        broadcasts = []
+
+        async def broadcast(payload):
+            broadcasts.append(payload)
+
+        dispatcher = EarthDbChangeDispatcher(
+            broadcast_earth_update=broadcast,
+            invalidate_cache=lambda source: 0,
+            debounce_seconds=10,
+        )
+        dispatcher.handle_payload(
+            {
+                "event_id": 200,
+                "table": "bgp_observations",
+                "operation": "DELETE",
+                "source": "ris_live_bgp",
+                "records_processed": 2,
+            }
+        )
+        dispatcher.handle_payload(
+            {
+                "event_id": 201,
+                "table": "bgp_anomalies",
+                "operation": "DELETE",
+                "source": "ris_live_bgp",
+                "records_processed": 3,
+            }
+        )
+
+        await dispatcher.flush_all()
+
+        assert len(broadcasts) == 1
+        assert broadcasts[0]["source"] == "ris_live_bgp"
+        assert broadcasts[0]["layers"] == ["bgp"]
+        assert broadcasts[0]["records_processed"] == 5
+        assert broadcasts[0]["refresh_strategy"] == "clear_then_reload"
+
+    asyncio.run(run())
+
+
+def test_dispatcher_deduplicates_notify_and_outbox_by_event_id():
+    async def run():
+        broadcasts = []
+
+        async def broadcast(payload):
+            broadcasts.append(payload)
+
+        dispatcher = EarthDbChangeDispatcher(
+            broadcast_earth_update=broadcast,
+            invalidate_cache=lambda source: 0,
+            debounce_seconds=10,
+        )
+        payload = {
+            "event_id": 42,
+            "table": "collected_data",
+            "operation": "INSERT",
+            "source": "celestrak_tle",
+            "entity_key": "sat:42",
+            "occurred_at": "2026-05-23T00:00:00Z",
+        }
+
+        assert dispatcher.handle_payload(payload) is True
+        assert dispatcher.handle_payload(dict(payload)) is False
+        await dispatcher.flush_all()
+
+        assert len(broadcasts) == 1
+        assert broadcasts[0]["records_processed"] == 1
+        assert broadcasts[0]["entity_keys"] == ["sat:42"]
+
+    asyncio.run(run())
+
+
+def test_dispatcher_flushes_continuous_events_at_max_wait():
+    async def run():
+        broadcasts = []
+        sleeps = []
+
+        async def broadcast(payload):
+            broadcasts.append(payload)
+
+        dispatcher = EarthDbChangeDispatcher(
+            broadcast_earth_update=broadcast,
+            invalidate_cache=lambda source: 0,
+            debounce_seconds=10,
+            max_wait_seconds=0.01,
+        )
+        dispatcher.handle_payload(
+            {
+                "event_id": 300,
+                "table": "collected_data",
+                "operation": "INSERT",
+                "source": "arcgis_cables",
+                "records_processed": 1,
+            }
+        )
+        await asyncio.sleep(0.02)
+        dispatcher.handle_payload(
+            {
+                "event_id": 301,
+                "table": "collected_data",
+                "operation": "UPDATE",
+                "source": "arcgis_cables",
+                "records_processed": 1,
+            }
+        )
+        await asyncio.sleep(0)
+        await dispatcher.flush_all()
+
+        assert len(broadcasts) == 1
+        assert broadcasts[0]["records_processed"] == 2
+        assert broadcasts[0]["debounce_ms"] >= 0
+
+    asyncio.run(run())
+
+
+def test_dispatcher_fast_flushes_delete_clear_then_reload():
+    async def run():
+        broadcasts = []
+
+        async def broadcast(payload):
+            broadcasts.append(payload)
+
+        dispatcher = EarthDbChangeDispatcher(
+            broadcast_earth_update=broadcast,
+            invalidate_cache=lambda source: 0,
+            debounce_seconds=10,
+            max_wait_seconds=10,
+        )
+        dispatcher.handle_payload(
+            {
+                "event_id": 310,
+                "table": "collected_data",
+                "operation": "DELETE",
+                "source": "celestrak_tle",
+                "records_processed": 10,
+            }
+        )
+        await asyncio.sleep(0.08)
+
+        assert len(broadcasts) == 1
+        assert broadcasts[0]["source"] == "celestrak_tle"
+        assert broadcasts[0]["refresh_strategy"] == "clear_then_reload"
+
+    asyncio.run(run())
+
+
+def test_outbox_rows_are_consumed_after_successful_flush():
+    async def run():
+        class FakeConnection:
+            def __init__(self):
+                self.consumed_ids = []
+
+            async def fetch(self, _query, _limit):
+                return [
+                    {
+                        "id": 501,
+                        "payload": {
+                            "event_id": 501,
+                            "table": "collected_data",
+                            "operation": "DELETE",
+                            "source": "celestrak_tle",
+                            "records_processed": 1,
+                        },
+                    }
+                ]
+
+            async def execute(self, _query, consumed_ids):
+                self.consumed_ids.extend(consumed_ids)
+
+        broadcasts = []
+
+        async def broadcast(payload):
+            broadcasts.append(payload)
+
+        dispatcher = EarthDbChangeDispatcher(
+            broadcast_earth_update=broadcast,
+            invalidate_cache=lambda source: 0,
+            debounce_seconds=10,
+        )
+        listener = EarthDbChangeListener(
+            dsn="postgresql://example/db",
+            dispatcher=dispatcher,
+        )
+        connection = FakeConnection()
+        listener._connection = connection
+
+        await listener._poll_outbox()
+
+        assert connection.consumed_ids == [501]
+        assert len(broadcasts) == 1
+
+    asyncio.run(run())
+
+
+def test_outbox_rows_remain_unconsumed_when_flush_fails():
+    async def run():
+        class FakeConnection:
+            def __init__(self):
+                self.consumed_ids = []
+
+            async def fetch(self, _query, _limit):
+                return [
+                    {
+                        "id": 601,
+                        "payload": {
+                            "event_id": 601,
+                            "table": "collected_data",
+                            "operation": "DELETE",
+                            "source": "celestrak_tle",
+                            "records_processed": 1,
+                        },
+                    }
+                ]
+
+            async def execute(self, _query, consumed_ids):
+                self.consumed_ids.extend(consumed_ids)
+
+        async def broadcast(_payload):
+            raise RuntimeError("ws down")
+
+        dispatcher = EarthDbChangeDispatcher(
+            broadcast_earth_update=broadcast,
+            invalidate_cache=lambda source: 0,
+            debounce_seconds=10,
+        )
+        listener = EarthDbChangeListener(
+            dsn="postgresql://example/db",
+            dispatcher=dispatcher,
+        )
+        connection = FakeConnection()
+        listener._connection = connection
+
+        try:
+            await listener._poll_outbox()
+        except RuntimeError:
+            pass
+
+        assert connection.consumed_ids == []
+
+    asyncio.run(run())
+
+
+def test_dispatcher_uses_aggregated_statement_record_count():
+    async def run():
+        broadcasts = []
+
+        async def broadcast(payload):
+            broadcasts.append(payload)
+
+        dispatcher = EarthDbChangeDispatcher(
+            broadcast_earth_update=broadcast,
+            invalidate_cache=lambda source: 0,
+            debounce_seconds=10,
+        )
+        dispatcher.handle_payload(
+            {
+                "event_id": 99,
+                "table": "collected_data",
+                "operation": "DELETE",
+                "source": "top500",
+                "records_processed": 100,
+                "entity_keys": ["top500:1", "top500:2"],
+                "occurred_at": "2026-05-23T00:00:00Z",
+            }
+        )
+
+        await dispatcher.flush_all()
+
+        assert len(broadcasts) == 1
+        assert broadcasts[0]["source"] == "top500"
+        assert broadcasts[0]["layers"] == ["computeCenters"]
+        assert broadcasts[0]["records_processed"] == 100
+        assert broadcasts[0]["entity_keys"] == ["top500:1", "top500:2"]
+
+    asyncio.run(run())

backend/tests/test_earth_interactables.py

@@ -0,0 +1,79 @@
+from datetime import UTC, datetime
+
+from app.models.earth_interactable import EarthInteractable
+from app.services import earth_interactables
+
+
+def make_interactable(**overrides):
+    values = {
+        "id": "poi-1",
+        "layer": "places",
+        "kind": "note",
+        "label": "Test POI",
+        "description": "A point on Earth",
+        "latitude": 30.25,
+        "longitude": 120.15,
+        "altitude": None,
+        "revision": 3,
+        "properties": {"owner": "test"},
+        "is_deleted": False,
+        "created_at": datetime(2026, 5, 22, 1, 0, tzinfo=UTC),
+        "updated_at": datetime(2026, 5, 22, 1, 5, tzinfo=UTC),
+        "deleted_at": None,
+    }
+    values.update(overrides)
+    return EarthInteractable(**values)
+
+
+def test_interactable_event_uses_object_delta_contract():
+    record = make_interactable()
+
+    event = earth_interactables.build_interactable_event(
+        action="updated",
+        record=record,
+    )
+
+    assert event["entity"] == "interactable"
+    assert event["action"] == "updated"
+    assert event["layer"] == "places"
+    assert event["layers"] == ["interactables"]
+    assert event["ids"] == ["poi-1"]
+    assert event["revision"] == 3
+    assert event["item"]["latitude"] == 30.25
+    assert event["item"]["properties"] == {"owner": "test"}
+
+
+def test_interactable_geojson_omits_deleted_records():
+    active = make_interactable(id="active")
+    deleted = make_interactable(id="deleted", is_deleted=True)
+
+    payload = earth_interactables.interactables_to_geojson([active, deleted])
+
+    assert payload["type"] == "FeatureCollection"
+    assert [feature["id"] for feature in payload["features"]] == ["active"]
+    assert payload["features"][0]["geometry"] == {
+        "type": "Point",
+        "coordinates": [120.15, 30.25],
+    }
+
+
+def test_interactable_cache_invalidation_clears_layer_and_all(monkeypatch):
+    patterns = []
+
+    def fake_delete_pattern(pattern):
+        patterns.append(pattern)
+        return 1
+
+    monkeypatch.setattr(
+        earth_interactables.earth_layer_cache,
+        "delete_pattern",
+        fake_delete_pattern,
+    )
+
+    deleted = earth_interactables.invalidate_interactable_cache("places")
+
+    assert deleted == 2
+    assert patterns == [
+        "earth:layer:v1:interactables:layer:places*",
+        "earth:layer:v1:interactables:layer:all*",
+    ]

backend/tests/test_earth_news.py

@@ -9,6 +9,7 @@ from app.services.earth_news import (
     ParsedNewsItem,
     _enrich_items_with_target_locations,
     _extract_target_location_from_text,
+    _parse_feed_entries,
     _serialize_item,
     get_earth_news_payload,
 )
@@ -109,6 +110,60 @@ def test_serialize_item_includes_inferred_target_location():
     assert payload["location_meta"]["ai_error"] is None
 
 
+def test_parse_plain_rss_uses_feed_name_as_source():
+    source = NewsFeedSource(
+        id="bbc-world",
+        name="BBC World",
+        region="global",
+        feed_url="https://feeds.bbci.co.uk/news/world/rss.xml",
+        homepage_url="https://www.bbc.com/news/world",
+    )
+    xml = """
+    <rss>
+      <channel>
+        <item>
+          <title>This may be the last time you hear my voice: Political executions surge in Iran since start of war</title>
+          <description>Story summary</description>
+          <link>https://www.bbc.com/news/example</link>
+          <pubDate>Fri, 15 May 2026 03:00:00 GMT</pubDate>
+        </item>
+      </channel>
+    </rss>
+    """
+
+    items = _parse_feed_entries(xml, source)
+
+    assert items[0].title == "This may be the last time you hear my voice: Political executions surge in Iran since start of war"
+    assert items[0].source == "BBC World"
+
+
+def test_parse_aggregated_rss_splits_publisher_from_title():
+    source = NewsFeedSource(
+        id="global-scan",
+        name="Global Monitor / World",
+        region="global",
+        feed_url="https://news.google.com/rss",
+        homepage_url="https://news.google.com/",
+        source_type="aggregated",
+    )
+    xml = """
+    <rss>
+      <channel>
+        <item>
+          <title>Example headline - Reuters</title>
+          <description>Story summary</description>
+          <link>https://news.google.com/example</link>
+        </item>
+      </channel>
+    </rss>
+    """
+
+    items = _parse_feed_entries(xml, source)
+
+    assert items[0].title == "Example headline"
+    assert items[0].source == "Reuters"
+
+
 @pytest.mark.asyncio
 async def test_enrich_items_with_target_locations_uses_ai_and_geocode(monkeypatch):
     item = ParsedNewsItem(
@@ -366,6 +421,62 @@ async def test_earth_news_payload_uses_fresh_database_items_without_rss(monkeypa
     assert payload["stale"] is False
 
 
+@pytest.mark.asyncio
+async def test_earth_news_payload_keeps_current_items_and_all_cruise_items(monkeypatch):
+    class FakeDb:
+        execute = object()
+
+    current_item = ParsedNewsItem(
+        id="db:current",
+        title="Current region story",
+        summary="Current summary",
+        url="https://example.com/current",
+        source="Stored Source",
+        feed_name="Stored Feed",
+        feed_region="americas",
+        homepage_url="https://example.com",
+        published_at=datetime(2026, 5, 15, 3, 0, tzinfo=UTC),
+    )
+    cruise_item = ParsedNewsItem(
+        id="db:apac",
+        title="APAC story",
+        summary="APAC summary",
+        url="https://example.com/apac",
+        source="Stored Source",
+        feed_name="Stored Feed",
+        feed_region="asia-pacific",
+        homepage_url="https://example.com",
+        published_at=datetime(2026, 5, 15, 4, 0, tzinfo=UTC),
+    )
+
+    async def fake_get_earth_news_freshness(_db, *, active_region):
+        return 12, datetime.now(UTC)
+
+    async def fake_list_earth_news_items(_db, *, active_region, limit):
+        return [current_item]
+
+    async def fake_list_earth_news_cruise_items(_db, *, limit):
+        return [current_item, cruise_item]
+
+    async def fake_enqueue_target_location_job(_payload, **_kwargs):
+        return True
+
+    async def fail_fetch(_sources):
+        raise AssertionError("fresh database items should not fetch RSS")
+
+    monkeypatch.setattr("app.services.earth_news_store.get_earth_news_freshness", fake_get_earth_news_freshness)
+    monkeypatch.setattr("app.services.earth_news_store.list_earth_news_items", fake_list_earth_news_items)
+    monkeypatch.setattr("app.services.earth_news_store.list_earth_news_cruise_items", fake_list_earth_news_cruise_items)
+    monkeypatch.setattr("app.services.earth_news_queue.enqueue_target_location_job", fake_enqueue_target_location_job)
+    monkeypatch.setattr("app.services.earth_news._fetch_rss_items_for_sources", fail_fetch)
+
+    payload = await get_earth_news_payload(lat=35.0, lon=-100.0, db=FakeDb())
+
+    assert [item["id"] for item in payload["items"]] == ["db:current"]
+    assert [item["id"] for item in payload["cruise_items"]] == ["db:current", "db:apac"]
+    assert payload["cruise_items"][1]["region"] == "asia-pacific"
+
+
 @pytest.mark.asyncio
 async def test_earth_news_payload_initializes_empty_database_from_rss(monkeypatch):
     db = object()

backend/tests/test_location_pipeline.py

@@ -700,7 +700,7 @@ async def test_llm_location_fallback_repairs_non_json_answer(monkeypatch):
 
 
 @pytest.mark.asyncio
-async def test_llm_location_fallback_accepts_taipei_name_hint_with_weak_wording(monkeypatch):
+async def test_llm_location_fallback_rejects_city_from_name_without_location_evidence(monkeypatch):
     monkeypatch.setattr(
         llm_fallback,
         "_geocode_llm_city",
@@ -740,18 +740,60 @@ async def test_llm_location_fallback_accepts_taipei_name_hint_with_weak_wording(
         entity_type="compute_center",
     )
 
-    assert result.failure_reason is None
-    candidate = result.candidates[0]
-    assert candidate.city == "Taipei"
-    assert candidate.confidence >= 0.55
-    breakdown = candidate.suggested_registry_entry["llm_score_breakdown"]
-    assert breakdown["weak_evidence_penalty"] <= 0.15
-    assert breakdown["conflict_penalty"] == 0
-    assert breakdown["name_location_hint"] > 0
+    assert result.candidates == []
+    assert result.failure_reason is not None
+    assert "below minimum" in result.failure_reason
 
 
 @pytest.mark.asyncio
-async def test_llm_location_fallback_geocodes_city_from_entity_name_when_llm_unparseable(monkeypatch):
+async def test_llm_location_fallback_accepts_explicit_facility_location_for_name_city_conflict(monkeypatch):
+    monkeypatch.setattr(
+        llm_fallback,
+        "_geocode_llm_city",
+        lambda query: {
+            "lat": "22.6048",
+            "lon": "120.3000",
+            "display_name": "Kaohsiung, Taiwan",
+            "address": {"city": "Kaohsiung", "country": "Taiwan"},
+        },
+    )
+    client = _FakeAIProviderClient(
+        json.dumps(
+            {
+                "latitude": None,
+                "longitude": None,
+                "precision": "city",
+                "confidence": 0.72,
+                "city": "Kaohsiung",
+                "country": "Taiwan",
+                "matched_location_name": "Kaohsiung, Taiwan",
+                "evidence": [
+                    {
+                        "source": "Taiwan News",
+                        "source_type": "news",
+                        "entity_match": True,
+                        "text": "Nvidia's first AI supercomputer center, Taipei-1, is located in Kaohsiung.",
+                    }
+                ],
+                "reasoning_summary": "Explicit facility location evidence overrides the city-like system name.",
+            }
+        )
+    )
+
+    result = await collect_llm_location_fallback_candidate(
+        provider_client=client,
+        query=LocationQuery(name="TAIPEI-1", country="Taiwan"),
+        entity_type="compute_center",
+    )
+
+    assert result.failure_reason is None
+    candidate = result.candidates[0]
+    assert candidate.city == "Kaohsiung"
+    assert candidate.confidence >= 0.55
+
+
+@pytest.mark.asyncio
+async def test_llm_location_fallback_rejects_compute_center_city_from_entity_name_when_llm_unparseable(monkeypatch):
     def _fake_geocode(query):
         if query != "Taipei, 中国（台湾）":
             return None
@@ -772,12 +814,9 @@ async def test_llm_location_fallback_geocodes_city_from_entity_name_when_llm_unp
     )
 
     assert client.calls == 2
-    assert result.failure_reason is None
-    candidate = result.candidates[0]
-    assert candidate.city == "Taipei"
-    assert candidate.latitude == pytest.approx(25.033)
-    assert candidate.longitude == pytest.approx(121.5654)
-    assert "Entity name city hint" in candidate.source_note
+    assert result.candidates == []
+    assert result.failure_reason is not None
+    assert "parseable city-level location fact" in result.failure_reason
 
 
 @pytest.mark.asyncio

backend/tests/test_logging.py

@@ -4,8 +4,11 @@ import logging
 
 from io import StringIO
 
+import pytest
+
 from app.core.logging import PlanetContextFilter, PlanetFormatter, get_logger
 from app.core.request_context import set_request_id
+from app.services import business_logs
 
 
 def _capture_output(callback):
@@ -76,3 +79,43 @@ def test_structured_logger_redacts_sensitive_text_and_context():
     assert "hunter2" not in output
     assert "[REDACTED]" in output
     assert '"safe": "visible"' in output
+
+
+def test_business_context_redacts_nested_sensitive_values():
+    context = business_logs.build_business_context(
+        {
+            "provider": "openai",
+            "api_key": "sk-secret",
+            "nested": {
+                "token": "plain-token",
+                "safe": "visible",
+            },
+        }
+    )
+
+    assert context["api_key"] == "[REDACTED]"
+    assert context["nested"]["token"] == "[REDACTED]"
+    assert context["nested"]["safe"] == "visible"
+
+
+@pytest.mark.asyncio
+async def test_emit_business_log_persists_sanitized_system_event(monkeypatch):
+    events = []
+
+    async def fake_record_system_log(**payload):
+        events.append(payload)
+
+    monkeypatch.setattr(business_logs, "record_system_log", fake_record_system_log)
+
+    await business_logs.emit_business_log(
+        get_logger("tests.business"),
+        event="ai.provider.analyze.success",
+        message="AI request completed",
+        category="ai",
+        context={"model": "gpt-test", "api_key": "sk-secret"},
+    )
+
+    assert events[0]["event"] == "ai.provider.analyze.success"
+    assert events[0]["category"] == "ai"
+    assert events[0]["context"]["model"] == "gpt-test"
+    assert events[0]["context"]["api_key"] == "[REDACTED]"

backend/tests/test_settings_ai_provider.py

@@ -5,21 +5,42 @@ import pytest
 from app.api.v1 import settings as settings_api
 from app.api.v1.settings import (
     AIProviderIntegrationUpdate,
+    BarentsWatchIntegrationUpdate,
+    ExternalIntegrationsUpdate,
     OCRIntegrationUpdate,
+    WebSearchIntegrationUpdate,
     _build_ai_provider_payload,
     _build_ocr_payload,
+    _can_reveal_integration_secrets,
+    _ensure_secret_reveal_allowed,
     _mask_secret,
     _normalize_ai_provider_payload,
     _normalize_ocr_payload,
+    _record_integration_secret_reveal,
     _resolve_provider_api_key,
     get_runtime_ai_provider_config,
+    save_external_integrations_payload,
 )
+from app.services.llm_provider_catalog import get_fallback_llm_provider_preset
 
 
 @pytest.fixture(autouse=True)
 def isolated_ai_provider_env_file(monkeypatch, tmp_path):
     env_file = tmp_path / ".env"
     monkeypatch.setattr(settings_api, "AI_PROVIDER_ENV_FILE", env_file)
+    for name in (
+        "AI_PROVIDER",
+        "AI_API_KEY",
+        "MINIMAX_API_KEY",
+        "OPENAI_API_KEY",
+        "ANTHROPIC_API_KEY",
+        "DEEPSEEK_API_KEY",
+        "DASHSCOPE_API_KEY",
+        "MOONSHOT_API_KEY",
+        "OPENROUTER_API_KEY",
+        "OPENCODE_GO_API_KEY",
+    ):
+        monkeypatch.delenv(name, raising=False)
     return env_file
 
 
@@ -56,7 +77,7 @@ def test_provider_key_prefers_specific_env_file_key(isolated_ai_provider_env_fil
 
 def test_provider_key_falls_back_to_generic_ai_api_key(isolated_ai_provider_env_file):
     isolated_ai_provider_env_file.write_text(
-        "AI_API_KEY=generic-env-file-key\n",
+        "AI_PROVIDER=openai\nAI_API_KEY=generic-env-file-key\n",
         encoding="utf-8",
     )
 
@@ -66,11 +87,101 @@ def test_provider_key_falls_back_to_generic_ai_api_key(isolated_ai_provider_env_
     assert source == "env_file"
 
 
+def test_generic_ai_api_key_only_applies_to_selected_env_provider(isolated_ai_provider_env_file):
+    isolated_ai_provider_env_file.write_text(
+        "AI_PROVIDER=minimax\nAI_API_KEY=generic-env-file-key\n",
+        encoding="utf-8",
+    )
+
+    openai_value, openai_source = _resolve_provider_api_key("openai", {"api_key": ""})
+    minimax_value, minimax_source = _resolve_provider_api_key("minimax", {"api_key": ""})
+
+    assert openai_value == ""
+    assert openai_source == ""
+    assert minimax_value == "generic-env-file-key"
+    assert minimax_source == "env_file"
+
+
+def test_opencode_go_provider_preset_is_openai_compatible():
+    preset = get_fallback_llm_provider_preset("opencode-go")
+
+    assert preset["label"] == "OpenCode Go"
+    assert preset["provider_api"] == "openai-completions"
+    assert preset["base_url"] == "https://opencode.ai/zen/go/v1"
+    assert preset["model"] == "glm-5.1"
+    assert "glm-5.1" in preset["models"]
+    assert "deepseek-v4-flash" in preset["models"]
+    assert preset["model_provider_apis"]["minimax-m2.7"] == "anthropic-messages"
+    assert preset["api_key_env"] == "OPENCODE_GO_API_KEY"
+
+
 def test_mask_secret_without_prefix_is_fully_masked():
     assert _mask_secret("plainsecret")["preview"] == "***********"
     assert _mask_secret("sk-prefixed")["preview"] == "sk-********"
 
 
+def test_secret_reveal_permission_is_admin_only():
+    assert _can_reveal_integration_secrets(SimpleNamespace(role="super_admin")) is True
+    assert _can_reveal_integration_secrets(SimpleNamespace(role="admin")) is True
+    assert _can_reveal_integration_secrets(SimpleNamespace(role="viewer")) is False
+
+
+@pytest.mark.asyncio
+async def test_secret_reveal_denial_is_audited(monkeypatch):
+    events = []
+
+    async def fake_record_audit_log(**payload):
+        events.append(payload)
+
+    monkeypatch.setattr(settings_api, "record_audit_log", fake_record_audit_log)
+    user = SimpleNamespace(id=7, username="viewer", email="viewer@example.test", role="viewer")
+
+    with pytest.raises(settings_api.HTTPException) as exc:
+        await _ensure_secret_reveal_allowed(
+            current_user=user,
+            request=None,
+            target_id="ai_provider:openai",
+            details={"kind": "ai_provider", "provider": "openai"},
+        )
+
+    assert exc.value.status_code == 403
+    assert events[0]["action"] == "settings.integration_secret.reveal"
+    assert events[0]["result"] == "denied"
+    assert events[0]["actor_id"] == 7
+    assert events[0]["target_id"] == "ai_provider:openai"
+
+
+@pytest.mark.asyncio
+async def test_secret_reveal_audit_does_not_store_plaintext(monkeypatch):
+    events = []
+
+    async def fake_record_audit_log(**payload):
+        events.append(payload)
+
+    monkeypatch.setattr(settings_api, "record_audit_log", fake_record_audit_log)
+    user = SimpleNamespace(id=1, username="admin", email="admin@example.test", role="admin")
+
+    await _record_integration_secret_reveal(
+        current_user=user,
+        request=None,
+        target_id="ai_provider:minimax",
+        result="success",
+        details={
+            "kind": "ai_provider",
+            "provider": "minimax",
+            "api_key_configured": True,
+            "api_key_source": "env_file",
+            "service_token_configured": True,
+            "service_token_source": "runtime",
+        },
+    )
+
+    serialized = str(events[0])
+    assert "secret-value" not in serialized
+    assert events[0]["details"]["api_key_source"] == "env_file"
+    assert events[0]["details"]["service_token_configured"] is True
+
+
 def test_build_payload_updates_only_selected_provider_key():
     current = {
         "ai_provider": {
@@ -109,6 +220,30 @@ def test_build_payload_updates_only_selected_provider_key():
     assert payload["providers"]["minimax"]["api_key"] == "minimax-old-key"
 
 
+def test_build_payload_saves_provider_without_changing_default():
+    current = {
+        "ai_provider": {
+            "default_provider": "minimax",
+            "providers": {
+                "minimax": {"provider": "minimax", "api_key": "minimax-key"},
+                "openai": {"provider": "openai", "api_key": ""},
+            },
+        }
+    }
+    update = AIProviderIntegrationUpdate(
+        provider="openai",
+        provider_api="openai-completions",
+        base_url="https://api.openai.com/v1",
+        model="gpt-test",
+        api_key="openai-new-key",
+    )
+
+    payload = _build_ai_provider_payload(current, update)
+
+    assert payload["default_provider"] == "minimax"
+    assert payload["providers"]["openai"]["api_key"] == "openai-new-key"
+
+
 def test_build_payload_keeps_saved_key_when_preview_submitted():
     current = {
         "ai_provider": {
@@ -207,3 +342,78 @@ async def test_runtime_config_uses_default_provider_specific_key(monkeypatch):
     assert runtime_config["llm_config"]["provider"] == "minimax"
     assert runtime_config["llm_config"]["api_key"] == "minimax-key"
     assert runtime_config["llm_config"]["model"] == "MiniMax-test"
+
+
+@pytest.mark.asyncio
+async def test_saving_ai_provider_does_not_run_connection_validation(monkeypatch):
+    calls = {"validated": 0, "saved": 0}
+
+    async def fake_get_setting_payload(_db, category):
+        assert category == "external_integrations"
+        return {
+            "ai_provider": {
+                "default_provider": "minimax",
+                "providers": {
+                    "minimax": {
+                        "provider": "minimax",
+                        "api_key": "minimax-key",
+                    },
+                },
+            },
+            "web_search": {},
+            "ocr": {},
+        }
+
+    async def fake_validate(_payload):
+        calls["validated"] += 1
+        raise AssertionError("save should not run AI provider connection validation")
+
+    async def fake_save_setting_payload(_db, category, payload):
+        assert category == "external_integrations"
+        calls["saved"] += 1
+        return payload
+
+    async def fake_get_barentswatch_config_record(_db):
+        return SimpleNamespace(
+            endpoint="",
+            auth_config={},
+        )
+
+    async def fake_serialize_external_integrations(_db):
+        return {"ai_provider": {"default_provider": "minimax"}}
+
+    monkeypatch.setattr(settings_api, "get_setting_payload", fake_get_setting_payload)
+    monkeypatch.setattr(settings_api, "_validate_ai_provider_full_connection", fake_validate)
+    monkeypatch.setattr(settings_api, "save_setting_payload", fake_save_setting_payload)
+    monkeypatch.setattr(settings_api, "get_barentswatch_config_record", fake_get_barentswatch_config_record)
+    monkeypatch.setattr(settings_api, "serialize_external_integrations", fake_serialize_external_integrations)
+
+    update = ExternalIntegrationsUpdate(
+        ai_provider=AIProviderIntegrationUpdate(
+            provider="minimax",
+            default_provider="minimax",
+            provider_api="anthropic-messages",
+            base_url="https://api.minimaxi.com/anthropic",
+            model="MiniMax-M2.7",
+            api_key="sk-new-key",
+        ),
+        barentswatch=BarentsWatchIntegrationUpdate(),
+        web_search=WebSearchIntegrationUpdate(),
+        ocr=OCRIntegrationUpdate(),
+    )
+
+    async def fake_commit():
+        return None
+
+    async def fake_refresh(_record):
+        return None
+
+    db = SimpleNamespace(
+        add=lambda _record: None,
+        commit=fake_commit,
+        refresh=fake_refresh,
+    )
+
+    await save_external_integrations_payload(db, update)
+
+    assert calls == {"validated": 0, "saved": 1}

backend/tests/test_visualization_compute_centers.py

@@ -816,6 +816,27 @@ async def test_visualization_geo_summary_returns_counts(monkeypatch):
         app.dependency_overrides.clear()
 
 
+@pytest.mark.asyncio
+async def test_empty_cable_layers_return_empty_feature_collections():
+    class _ScalarResult:
+        def scalars(self):
+            class _Scalars:
+                def all(self):
+                    return []
+
+            return _Scalars()
+
+    class _FakeSession:
+        async def execute(self, _query):
+            return _ScalarResult()
+
+    cables = await visualization_api._build_cables_geojson(_FakeSession())
+    landing_points = await visualization_api._build_landing_points_geojson(_FakeSession())
+
+    assert cables == {"type": "FeatureCollection", "features": []}
+    assert landing_points == {"type": "FeatureCollection", "features": []}
+
+
 @pytest.mark.asyncio
 async def test_collect_location_endpoint_returns_candidates_for_known_record(monkeypatch):
     def _fake_ror(query):

backend/tests/test_web_search_tools.py

@@ -46,6 +46,21 @@ def test_web_search_key_prefers_provider_env(isolated_web_search_env_files):
     assert source == "env_file"
 
 
+def test_web_search_generic_key_only_applies_to_default_provider(isolated_web_search_env_files):
+    isolated_web_search_env_files.write_text(
+        "WEB_SEARCH_API_KEY=generic-search-key\n",
+        encoding="utf-8",
+    )
+
+    default_value, default_source = _resolve_web_search_api_key("tavily", {"api_key": ""}, "tavily")
+    other_value, other_source = _resolve_web_search_api_key("brave", {"api_key": ""}, "tavily")
+
+    assert default_value == "generic-search-key"
+    assert default_source == "env_file"
+    assert other_value == ""
+    assert other_source == ""
+
+
 def test_build_web_search_payload_keeps_saved_key_when_preview_submitted():
     current = {
         "web_search": {
@@ -71,6 +86,35 @@ def test_build_web_search_payload_keeps_saved_key_when_preview_submitted():
     assert payload["providers"]["tavily"]["api_key"] == "tvly-old-secret"
 
 
+def test_build_web_search_payload_saves_provider_without_changing_default():
+    current = {
+        "web_search": {
+            "default_provider": "tavily",
+            "providers": {
+                "tavily": {
+                    "provider": "tavily",
+                    "api_key": "tvly-key",
+                },
+                "brave": {
+                    "provider": "brave",
+                    "api_key": "",
+                },
+            },
+        }
+    }
+    update = WebSearchIntegrationUpdate(
+        enabled=True,
+        provider="brave",
+        base_url="https://api.search.brave.com/res/v1/web/search",
+        api_key="brave-new-key",
+    )
+
+    payload = _build_web_search_payload(current, update)
+
+    assert payload["default_provider"] == "tavily"
+    assert payload["providers"]["brave"]["api_key"] == "brave-new-key"
+
+
 @pytest.mark.asyncio
 async def test_tavily_adapter_normalizes_results(monkeypatch):
     config = WebSearchConfig(

docker-compose.local-model.yml

@@ -15,12 +15,16 @@ services:
       retries: 10
 
   aiprovider:
+    image: ${AI_PROVIDER_IMAGE_NAME:-planet-aiprovider:latest}
     build:
       context: .
       dockerfile: aiprovider/Dockerfile
       args:
         PYTHON_IMAGE: ${PYTHON_IMAGE:-python:3.14-slim}
         UV_IMAGE: ${UV_IMAGE:-ghcr.io/astral-sh/uv:latest}
+        AI_PROVIDER_BUILD_FINGERPRINT: ${AI_PROVIDER_BUILD_FINGERPRINT:-unknown}
+      secrets:
+        - planet_uv_config
     container_name: planet_aiprovider
     ports:
       - "8010:8010"
@@ -43,3 +47,7 @@ services:
 
 volumes:
   ollama_data:
+
+secrets:
+  planet_uv_config:
+    file: ${PLANET_UV_CONFIG_FILE:-/dev/null}

docker-compose.simple.yml

@@ -2,12 +2,16 @@ version: '3.8'
 
 services:
   aiprovider:
+    image: ${AI_PROVIDER_IMAGE_NAME:-planet-aiprovider:latest}
     build:
       context: .
       dockerfile: aiprovider/Dockerfile
       args:
         PYTHON_IMAGE: ${PYTHON_IMAGE:-python:3.14-slim}
         UV_IMAGE: ${UV_IMAGE:-ghcr.io/astral-sh/uv:latest}
+        AI_PROVIDER_BUILD_FINGERPRINT: ${AI_PROVIDER_BUILD_FINGERPRINT:-unknown}
+      secrets:
+        - planet_uv_config
     env_file:
       - ./aiprovider/.env
       - ${PLANET_AI_PROVIDER_RUNTIME_ENV_FILE:-./aiprovider/.env}
@@ -43,3 +47,7 @@ services:
 volumes:
   postgres_data:
   redis_data:
+
+secrets:
+  planet_uv_config:
+    file: ${PLANET_UV_CONFIG_FILE:-/dev/null}

docker-compose.yml

@@ -2,12 +2,16 @@ version: '3.8'
 
 services:
   aiprovider:
+    image: ${AI_PROVIDER_IMAGE_NAME:-planet-aiprovider:latest}
     build:
       context: .
       dockerfile: aiprovider/Dockerfile
       args:
         PYTHON_IMAGE: ${PYTHON_IMAGE:-python:3.14-slim}
         UV_IMAGE: ${UV_IMAGE:-ghcr.io/astral-sh/uv:latest}
+        AI_PROVIDER_BUILD_FINGERPRINT: ${AI_PROVIDER_BUILD_FINGERPRINT:-unknown}
+      secrets:
+        - planet_uv_config
     env_file:
       - ./aiprovider/.env
       - ${PLANET_AI_PROVIDER_RUNTIME_ENV_FILE:-./aiprovider/.env}
@@ -53,3 +57,7 @@ services:
 volumes:
   postgres_data:
   redis_data:
+
+secrets:
+  planet_uv_config:
+    file: ${PLANET_UV_CONFIG_FILE:-/dev/null}

docs/CHANGELOG.md

@@ -8,6 +8,167 @@ This project follows the repository versioning rule:
 - `improvement` -> `+0.0.1`（bugfix + 小功能混合）
 - `bugfix` -> `+0.0.1`
 
+## [0.66.2] — 2026-05-26
+
+Released: 2026-05-26
+
+### Highlights
+- 统一中文界面和公开文档中的产品命名：`Earth` 显示为“智能星球”，`Admin` 显示为“控制台”，`Docs` 显示为“文档”。
+- 修复文档手册、Docs catalog、控制台入口和智能星球设置中残留的中英混排标题。
+
+### Added / Fixed / Improved
+- 更新控制台侧边栏、智能星球系统设置、移动端抽屉和登录页品牌文案。
+- 同步前后端 Docs metadata、中文技术文档标题、使用手册、快速开始、术语表和运维手册中的产品命名。
+- 补充清理智能星球图层缓存、品牌配置、展示缓存等控制台 toast / dialog 文案。
+
+---
+
+## [0.66.1] — 2026-05-26
+
+Released: 2026-05-26
+
+### Highlights
+- 修复 CelesTrak active 更新窗口内清库后无法恢复的问题，新增持久原始下载缓存和完整 fallback group mode。
+- 避免数据源任务 WebSocket 与轮询同时完成时重复弹出采集失败 toast。
+- `planet.sh destroy` 保留上游原始下载缓存，数据库清空后仍可用缓存重灌。
+
+### Added / Fixed / Improved
+- CelesTrak 403 `GP data has not updated` 会先复用 active 缓存；无 active 缓存时完整拉取 `starlink/gps-ops/galileo/glonass/beidou/leo/geo/iridium-next`，任一 group 缺失则整体失败，不保存 partial。
+- 下载器缓存从 `/tmp` 迁到 `$PLANET_CACHE_DIR/downloads`，并保留 HTTP 错误响应正文以支持上游限频语义判断。
+- 补充 CelesTrak cache/fallback 回归测试和中英文运维/卫星策略文档。
+
+---
+
+## [0.66.0] — 2026-05-26
+
+Released: 2026-05-26
+
+### Highlights
+- 将 Admin 正式化为唯一控制台入口，移除旧 AntD 后台、Admin Next 路由痕迹和相关依赖。
+- 引入 PostgreSQL 数据作业队列、Earth outbox 同步和可交互对象管线，让采集、清理和 Earth 刷新进入可追踪异步链路。
+- 强化 AI、AI 工具、设置连接测试和采集任务的结构化业务日志，关键事件可在系统日志中检索。
+- CelesTrak 轨道根数采集改为完整 `active` 目录下载、续传和重试，避免部分分组失败时保存不完整卫星数据。
+
+### Added / Fixed / Improved
+- Admin 数据源页修复内置源调度启停按钮，使用 `is_active` 判断 `/enable` 与 `/disable`，避免采集状态误导调度开关。
+- Earth 新增通用 interactables 层和平台数据流文档，支持对象级 delta 同步与后续小图层扩展。
+- `planet.sh init/start/destroy` 补齐 uv 镜像回退、Bun unzip 依赖、HTTPS/LAN 跳转协议和 OOBE 清理语义。
+- Markdown code block、数据源队列、演示模式、About 版本展示、TV 链接协议和文档索引同步完成。
+
+---
+
+## [0.65.2] — 2026-05-22
+
+Released: 2026-05-22
+
+### Highlights
+- Stabilize AI Provider rebuild detection so unchanged source no longer rebuilds just because file timestamps or local cache state changed.
+- Keep `uv.lock` on the official registry while still allowing local and Docker builds to use the user's `uv.toml`.
+- Prevent local startup and bootstrap commands from rewriting `uv.lock` when a user-level mirror is configured.
+
+### Added / Fixed / Improved
+- AI Provider fingerprints now use content hashes, base image inputs, dependency metadata, and an image label instead of file mtimes.
+- `planet.sh` and `scripts/bootstrap-dev.sh` now use `uv sync --frozen`, and backend startup uses `uv run --frozen`.
+- Docker Compose passes the AI Provider fingerprint into the image build so future starts can inspect the image label directly.
+
+---
+
+## [0.65.1] — 2026-05-22
+
+Released: 2026-05-22
+
+### Highlights
+- 修复 `planet.sh` 与 Docker Compose 对 AI Provider 镜像名不一致导致的重复构建问题。
+- 让本地 `uv` 与 Docker build 统一使用用户机器上的 `uv.toml`，避免把镜像源写入 `uv.lock`。
+- 通过 BuildKit secret 向容器构建传入 uv 配置，保留用户源选择且不把配置写入镜像层。
+
+### Added / Fixed / Improved
+- `planet.sh` 自动解析 `UV_CONFIG_FILE`、仓库 `uv.toml` 与用户级 `~/.config/uv/uv.toml`，并清理会覆盖配置的 `UV_INDEX_URL` 环境变量。
+- AI Provider 和后端 Dockerfile 在 `uv sync` 阶段挂载 uv 配置 secret，并复用 BuildKit uv cache。
+- Docker Compose 三套配置统一声明 `planet-aiprovider:latest` 镜像名和 `planet_uv_config` build secret。
+
+---
+
+## [0.65.0] — 2026-05-21
+
+Released: 2026-05-21
+
+### Highlights
+- 收敛 Admin Next 数据源触发入口：主按钮在未勾选时触发全部，勾选内置源后切换为“触发已选 N”，并移除手填 ID 的批量触发弹窗。
+- 优化数据源采集队列入口：右上角按钮常驻，空态显示队列图标，有任务时显示纯圆环进度，队列改为浮层避免挤压表格。
+- 强化 `planet.sh destroy` 清理语义，销毁时先硬重置运行中的本地 Postgres `public` schema，避免残留采集数据让 OOBE 误判 ready。
+
+### Added / Fixed / Improved
+- Admin Next 表格新增可选选择列，仅在 `/datasources` 内置源分区启用，支持当前可见行全选并在筛选、切分区或刷新时清空选择。
+- 数据源批量触发复用 `/datasources/trigger-batch` 的 `source_ids`，成功后写入现有采集队列并清空勾选。
+- `destroy` 补充清理 `planet-aiprovider:latest` 镜像以及 Python/Vite 等本地编译缓存，同时保留源码和 `.env`。
+- Docs Gatekeeper 与 Tactile UI 文档/样式继续补齐，覆盖本轮按钮、队列、OOBE 和销毁流程说明。
+
+---
+
+## [0.64.0] — 2026-05-21
+
+Released: 2026-05-21
+
+### Highlights
+- 新增 Earth 首次初始化 OOBE，由后端真实采集状态决定是否显示，避免 localStorage 清空后误弹，并提供桌面毛玻璃引导与移动端 bottom sheet。
+- 数据源页新增下载列表式采集队列，把单源、批量和触发全部的任务进度统一展示，并支持失败重试与跳转详情。
+- Earth 内容新增“关于”配置接口和后台 tab，Earth 设置页 About 卡片改为运行时读取配置并带默认 fallback。
+
+### Added / Fixed / Improved
+- 新增 `/api/v1/earth/oobe-status`、`/api/v1/earth/about` GET/PUT/DELETE，并让 Earth 前端加载 `about.js` 与 `oobe.js`。
+- Admin Next 数据源队列优先消费 `datasource_tasks` WebSocket，断线时轮询 `/datasources/{id}/task-status`，刷新后只恢复后端仍在运行的真实任务。
+- Admin Next 深色主题滑块补齐 Docs 同款 dark token，侧栏主题控件在 dark 模式下不再保持浅色底座。
+- 用户手册、快速开始、Earth 前端上下文和 Admin 前端上下文同步记录 OOBE、采集队列、About 配置与主题滑块行为。
+
+---
+
+## [0.63.1] — 2026-05-21
+
+Released: 2026-05-21
+
+### Highlights
+- 修复新设备初始化后 Admin Next 无法打开的问题：补上被 `.gitignore` 的 `lib/` 规则误忽略的 Admin Next utility module。
+
+### Added / Fixed / Improved
+- 将 `frontend/src/admin-next/lib/utils.ts` 纳入版本控制，恢复 `DashboardNext`、`AdminNextLayout` 等页面对 `cn` 和 `formatNumber` 的运行时依赖。
+- 避免 Vite dev server 在新 clone 环境中因缺失源码模块而对 `/src/admin-next/*` 返回 500，并导致动态导入 AdminNextRoutes 失败。
+
+---
+
+## [0.63.0] — 2026-05-21
+
+Released: 2026-05-21
+
+### Highlights
+- 新增 `./planet.sh init` 首次初始化入口，将 uv/bun 依赖同步、env 模板补齐、数据库容器启动、建表 seed 和默认用户生成串成一条空项目引导路径。
+- 新增 `./planet.sh destroy` 破坏性重置入口，带 CLI 确认保护，可清理 Planet 容器、卷、镜像和本地编译/运行状态，同时保留源码与 `.env` 配置。
+- 改进 `planet.sh` 日志体验，只在带状态标签的输出行末尾追加时间戳，并让 `init` 在应用服务已运行时自动跳过重复初始化。
+
+### Added / Fixed / Improved
+- `init` 复用现有 Docker/依赖 helper，支持重复执行时不覆盖 env、不清空数据，并在完成后提示默认本地登录账号。
+- `destroy` 停止本地服务后清理 Docker compose 状态、项目镜像、数据卷、`.venv`、`node_modules`、前端构建产物和 Planet state/cache。
+- README 快速启动补充 `init` 和 `destroy` 命令说明，明确首次引导和重置路径。
+
+---
+
+## [0.62.0] — 2026-05-21
+
+Released: 2026-05-21
+
+### Highlights
+- Admin Next 转正为正式后台入口，旧 AntD 控制台完整迁移到 `/legacy/admin/*` 作为回退和对照，并保留 `/admin-next/*` 兼容重定向。
+- 新后台完成采集、AI Provider、Earth 内容、日志、BGP/告警、设置和认证链路的全量收口，新增 lazy tab loading、层级配置、移动端详情和 Markdown 文档渲染体验。
+- 抽出 Tactile UI 按钮、开关、tooltip、滚动条和表格滚动组件，为未来独立 npm 组件库打基础，同时补齐中英文开发文档。
+
+### Added / Fixed / Improved
+- 新增采集管理分层工作台，覆盖采集器配置、采集调度、采集历史/快照、凭证教程生成/重置、映射 propose/preview/create/activate 和真实连接/运行状态。
+- 改进 AI Provider 与工具调用配置，修复 key fallback、脱敏显示、轻量连通性测试、默认 provider、OpenCode Go 路由和 Playground Markdown 输出。
+- 更新 Earth 内容管理和 Earth 前台体验，支持品牌预览/上传/重置、TV 默认源与新增草稿、图层/新闻/3D 模型配置入口以及 live TV 预览。
+- 强化后端数据源、系统日志、WebSocket、AI client、位置 LLM fallback、Earth layer cache 和 datasource connectivity，并补充相关回归测试与运维脚本能力。
+
+---
+
 ## [0.61.0] — 2026-05-18
 
 Released: 2026-05-18

docs/deprecated/README.md

@@ -16,7 +16,24 @@
 - 已被正式实现替代、继续放在 `docs/` 根目录会误导后续开发的计划，归档
 - 仍然指导未来开发、尚未完成或仍有明确执行价值的文档，继续保留在 `docs/`
 
+当前替代入口：
+
+- 业务和数据产品链路见 [业务架构与数据流转](/home/ray/dev/linkong/planet/docs/technical/zh/platform-data-flows.md)。
+- 当前后端作业、outbox 和 Earth 同步实现见 [数据作业与 Outbox 技术架构](/home/ray/dev/linkong/planet/docs/technical/zh/data-job-earth-sync-architecture.md)。
+- 用户操作流程见 [Planet 使用手册](/home/ray/dev/linkong/planet/docs/technical/zh/manual.md)。
+- 当前代码结构见 [技术文档索引](/home/ray/dev/linkong/planet/docs/technical/zh/README.md)。
+
 补充说明：
 
 - 一部分归档文档来自外部或临时工作流草案，例如 sisyphus 生成的初稿
 - 这类文档如果有可用内容，应先吸收到 `docs/plans/` 或 `docs/technical/`，再归档保留来源记录
+
+## 近期归档
+
+- [Docs Gatekeeper 鉴权系统计划](/home/ray/dev/linkong/planet/docs/deprecated/docs-gatekeeper-auth-plan.md)：已落地，当前实现见技术文档。
+- [Location Resolver 共享管线计划](/home/ray/dev/linkong/planet/docs/deprecated/location-resolver-shared-pipeline-plan.md)：已落地，当前实现见技术文档。
+- [Earth Surface Hover Info Plan](/home/ray/dev/linkong/planet/docs/deprecated/earth-surface-hover-info-plan.md)：已实现，保留为历史记录。
+- [Admin Next Dual Track Full Migration Plan](/home/ray/dev/linkong/planet/docs/deprecated/admin-next-dual-track-full-migration-plan.md)：已被当前 `frontend/src/admin/` 控制台结构替代。
+- [Admin Next Soft Glass Goal Driven Plan](/home/ray/dev/linkong/planet/docs/deprecated/admin-next-soft-glass-goal-driven-plan.md)：已被当前 `frontend/src/admin/` 控制台结构替代。
+- [Admin Next Parity Checklist](/home/ray/dev/linkong/planet/docs/deprecated/admin-next-parity-checklist.md)：旧 `/admin-next/*` 验收资料，保留为迁移历史。
+- [Admin Next Parity Audit Closeout](/home/ray/dev/linkong/planet/docs/deprecated/admin-next-parity-audit-closeout-plan.md)：旧 `/admin-next/*` 审计资料，保留为迁移历史。

docs/deprecated/admin-next-dual-track-full-migration-plan.md

@@ -0,0 +1,128 @@
+# Admin Next 双轨全量迁移修正计划
+
+## Summary
+
+`/admin-next/*` 继续作为新版影子路由开发，旧 AntD 控制台必须完整保留作为生产回退和行为对照。只有新版完成全功能 parity 并通过验收后，才能逐个切换旧路由；删除旧页面和移除 AntD 必须作为最后独立步骤，并等待明确确认。
+
+## Hard Rules
+
+- 未完成 parity checklist 前，不删除旧页面、不移除 AntD、不切旧入口。
+- “旧版能删除”只表示新版能力达到替代标准，不表示立即删除旧版。
+- 新版不能只做看板；旧版里可新增、编辑、删除、测试、触发、批量、轮询、WebSocket、权限控制、错误提示和确认弹窗的能力，都必须迁移。
+- `/admin-next/*` 可以重做交互和视觉，但业务语义、API payload、权限和危险操作保护必须对齐旧版。
+
+## Design Direction
+
+- 风格采用“朴素工具化 + 轻微立体触感”：灰白/暗色低对比背景、清晰 1px 边框、轻微凸起/按下态、克制圆角、图标少量彩色。
+- 禁止回到 soft-glass、hero、大发光、大渐变、大色块和装饰性卡片。
+- 常规动作默认 icon-only + tooltip：刷新、重启、退出、复制、查看、编辑、删除、关闭、设置。
+- 强意图动作保留实心文字按钮：保存、创建、确认、执行、测试连接。
+- 页面统一 `16px` 外边距、`16px` 主区块间距、`12px` 面板 header/body 间距；表格标题和表体不能贴边。
+- 所有页面遵守一屏工作台：`PageHeader` 之外只允许一个主内容区域吃满剩余高度；详情区、表格区和 Playground 消息区内部滚动，不能被父级裁掉。
+- 页内 tabs 必须按旧版用户心智组织，而不是按后端接口名拆散；接口状态只能作为分区内部信息。
+
+## Implementation Plan
+
+1. 恢复旧版安全网：恢复旧 AntD 页面、旧 `AppLayout`、旧 helper、AntD 依赖和旧路由；保留 `/admin-next/*` 影子路由。
+2. 建立新版基础层：整理 admin-next token、按钮、tooltip、theme switch、sidebar account、`EntityTable`、`FormDialog`、`ConfirmDialog`、`DetailPanel`、移动端表格/卡片切换。
+3. 拆掉通用看板页：每个模块建立真实业务页面、API adapter、form/mutation、轮询或 WebSocket 逻辑，不再用万能 Resource table 聚合展示。
+4. 按模块迁移：
+   - Dashboard：统计、WebSocket、健康检查、重启任务、任务日志、恢复探测。
+   - DataSources：内置源/自定义源/实时源、详情、stats、task-status、启停、触发、批量触发、清理数据。
+   - DataList：列表、summary、sources/types/countries、搜索、筛选、分页、详情、导出、分布。
+   - Collection Management：configs CRUD、builtin connect/test、target schemas、mappings、run/stop、stream status。
+   - Settings：system、notifications、security、SMTP、TV、integrations、collectors、测试连接、凭证指南。
+   - AI：provider、web search、OCR、prompt registry、playground 完整会话操作。
+   - Earth Content：brand、upload、delete/reset、boundary config/build/status、TV/content。
+   - Logs：sources、level/date/search、刷新、详情、复制、tail/snapshot。
+   - BGP：overview、collectors、incidents、anomalies、events、AI briefs。
+   - Alerts：系统告警、BGP 告警、态势告警，包含 ack/resolve/stats/brief。
+   - Users：list/create/edit/delete、role、Gatekeeper groups、权限显示。
+   - Auth：登录、注册、验证邮箱、重发、找回、重置、logout。
+5. 每完成一个模块，对照旧页面跑 checklist；只有全部模块通过后，才提出旧路由切换和旧版删除。
+
+## Test Plan
+
+- 恢复阶段：`cd frontend && /home/ray/.bun/bin/bun install && /home/ray/.bun/bin/bun run build`。
+- 旧路由检查：`/admin`、`/datasources`、`/settings`、`/ai`、`/logs` 必须继续打开旧版。
+- 新路由检查：`/admin-next/*` 必须继续打开新版。
+- 每模块 parity：读写删、测试连接、触发、批量、轮询、WebSocket、权限、错误提示、确认弹窗、移动端和滚动。
+- 视觉检查：light/dark/system、低高度窗口、125%/150% 缩放、按钮/状态/badge 等高、无双滚动条。
+
+## Parity Checklist
+
+### Dashboard
+- [x] `/dashboard/stats` 数据展示与旧版一致。
+- [x] WebSocket 连接状态、健康状态、自动刷新语义一致。
+- [x] 重启任务创建、轮询、日志查看、失败提示和恢复探测一致。
+
+### DataSources
+- [x] 内置源、自定义源、实时源分区清晰，不混表。
+- [x] 详情、stats、task-status、enable/disable、trigger、trigger-batch、trigger-all、delete-data 与旧版一致。
+- [x] 实时源 start/stop/restart、连接状态、任务状态与旧版一致。
+
+### DataList
+- [x] 列表、summary、sources/types/countries、搜索、筛选、分页参数与旧版一致。
+- [x] 详情 metadata、分布摘要、JSON/CSV 导出与旧版一致。
+- [x] 表格固定列、横向滚动、移动端卡片模式可用。
+
+### Collection Management
+- [x] configs/all、configs CRUD、builtin connect/test、custom sample 与旧版一致。
+- [x] target schemas、mappings preview/list/update/activate 与旧版一致；propose/create 已提供表单化入口。
+- [x] run-mapped、stop-mapped、stream-status 与旧版一致。
+
+### Settings
+- [x] system、notifications、security、SMTP、TV、integrations、collectors 表单字段完整。
+- [x] SMTP test、AI/Web/OCR secrets、connect tests、preset refresh 与旧版一致。
+- [x] credential guides、cache 清理、危险操作确认一致。
+
+### AI
+- [x] Provider、Web Search、OCR、Prompt Registry 配置保存/重置完整。
+- [x] Playground thread/session/messages/status/stop/resend/edit 完整。
+- [x] provider refresh、connect test、secrets 读取、错误提示和 loading 状态完整。
+- [x] 页内 tabs 按旧版心智恢复为模型供应商、工具调用、提示词、Playground。
+
+### Earth Content
+- [x] brand get/save/delete/reset/upload 完整。
+- [x] boundary status/config/build/build-status 完整。
+- [x] TV/content 配置与旧版 Settings 入口一致。
+
+### Logs
+- [x] sources、level/date/search、limit、刷新参数与旧版一致。
+- [x] 日志正文、详情展开、复制、空态和错误态完整。
+
+### BGP
+- [x] overview、collectors、incidents、anomalies、events 的 summary/list/detail 完整。
+- [x] AI brief list/detail/generate 完整。
+- [x] collector collect-location、刷新、结果详情和错误提示完整。
+
+### Alerts
+- [x] 系统告警 list/detail/ack/resolve/stats/AI brief 完整。
+- [x] BGP incidents/anomalies/brief 完整。
+- [x] 态势告警 stats/list/brief 完整。
+
+### Users
+- [x] list/create/edit/delete、role、Gatekeeper groups 完整。
+- [x] 权限显示、普通用户限制和 super admin 行为一致。
+
+### Auth
+- [x] login/register/verify/resend cooldown/forgot/reset/logout/me 完整。
+- [x] 未验证邮箱跳转、错误提示、成功跳转和 token store 行为一致。
+
+### Hierarchy Repair
+- [x] AI 不再按接口/状态拍平成一张表；模型供应商恢复为 provider -> model/API/key 层级。
+- [x] 工具调用恢复为 Web Search/OCR -> provider/API/高级参数层级。
+- [x] 提示词恢复为 group -> prompt 入口层级，保存/重置只作用于当前入口。
+- [x] Settings/Earth 配置型页面使用父级列表 + 右侧正式表单，长内容在一屏内内部滚动。
+
+### Interaction Polish
+- [x] 不会产生歧义的刷新、复制、详情、批量、状态、清理等动作优先改为 icon-only + tooltip/title。
+- [x] 强意图动作保留实心文字按钮，例如保存、创建、触发全部、生成简报、确认执行。
+- [x] 按钮字号、高度、图标容器和 hover 触感统一，接近侧栏深色模式滑块的轻微立体风格。
+- [x] 表格与详情之间增加可拖动竖向 resize handle，低高度和窄屏下不制造额外滚动条。
+
+### Page Information Architecture
+- [x] 页内 tab 默认使用中文；BGP、AI、OCR、Web Search、Playground、Schema 等专有名词按可识别性保留。
+- [x] 信息观测类页面使用列表 + 详情，例如 BGP、Alerts、Logs、DataSources。
+- [x] 配置管理类页面使用分层结构，例如 AI、Settings、Earth Content、Collection Management。
+- [x] 采集管理已从纯表格看板改为分层管理：采集器、映射模板、目标 Schema、运行状态先选父级，再编辑或执行动作。

docs/deprecated/admin-next-parity-audit-closeout-plan.md

@@ -0,0 +1,135 @@
+# Admin Next Parity Audit Closeout
+
+Last updated: 2026-05-21
+
+## Status
+
+Admin Next has been promoted to the official admin route family. The old AntD admin, old layout helpers, `antd`, and `@ant-design/icons` remain available under `/legacy/admin/*` as the rollback and comparison surface.
+
+`/admin-next/*` is now compatibility-only: old test links redirect to the official route. Do not add new capabilities there as a separate entry point.
+
+This closeout document is the final work log for the second parity audit. It records what has been fixed, what still requires manual verification, and what must not be treated as complete.
+
+## Fixed In This Audit
+
+- Restored the dual-track rule: old AntD pages and routes are not deleted and are not replaced before parity sign-off.
+- Reworked management pages away from flat status tables toward business hierarchy:
+  - AI: `模型供应商 / 工具调用 / 提示词 / Playground`.
+  - Settings: system display, notification policy, security policy, SMTP only.
+  - Earth Content: brand, boundary precision, TV content.
+  - Collection: collector, collection schedule, history/snapshot.
+- Repaired AI provider credential semantics:
+  - Provider status now comes from that provider's stored key or a runtime key that is explicitly scoped to that provider.
+  - Generic `.env` fallback no longer marks every provider as configured.
+  - Runtime fallback secrets stay masked by default; authorized administrators can reveal them through the console and each reveal must be audited without storing plaintext.
+  - Secret previews preserve the prefix before the first `-` when the backend preview exposes it, for example `sk-**********`.
+  - Connect-test buttons are icon-only plug actions and do not save configuration.
+- Repaired key UI gaps:
+  - LLM API Key, proxy token, Web Search API Key, and OCR API Key use masked input with inline eye toggle.
+  - Save buttons use the disk icon.
+  - Default action is disabled when the item is already default.
+- Repaired SMTP test flow:
+  - SMTP test has a recipient input.
+  - Request payload is `{ to, settings }`.
+- Repaired Earth Content actions:
+  - Brand save, upload, delete/reset actions are available in the brand section.
+  - Brand preview renders with the same Earth left-top brand structure and starfield-style background.
+  - Boundary build is scoped to the boundary precision section, not the global page toolbar.
+  - TV default configuration is folded into item-level configuration; TV items support add/default/reset/delete semantics.
+  - TV default source now follows the same default-state semantics as AI Provider: if the response does not expose an explicit default, `cgtn-en` is treated as the runtime default; setting a default promotes that item visually and disables the redundant default action.
+- Repaired Collection actions:
+  - Credential guide read/generate/reset entry points are available from collector-related panels.
+  - Create collector config uses field-first form controls with advanced JSON as a secondary path.
+  - Mapping propose/preview/create/activate flow is represented in the mapping workflow.
+- Repaired shared UI rules:
+  - Management list names can wrap and remain readable instead of being squeezed by status tags.
+  - Status tags are fixed width and color-coded: default blue, configured green, unconfigured gray, error red.
+  - Configuration lists with a default item sort the default item to the top after save/default changes instead of leaving it in the previous alphabetical or API order.
+  - List footer actions are part of the scroll content; users see them after scrolling to the bottom.
+  - Footer icon buttons use the current tactile surface direction: external shadow first, without a separate inset/base-plate treatment.
+  - Detail panels use a dark neutral resize handle.
+  - Tooltip, dialog, modal backdrop, textarea scrollbar, custom scrollbar, mobile detail, and one-screen layout issues from the audit have been addressed in the shared admin-next layer.
+  - Multi-tab Admin Next pages now use lazy active-tab loading with local cache. Initial page load no longer requests every section endpoint; manual refresh and mutating actions refresh only the current section.
+  - Markdown-producing details, AI brief content, credential tutorials, and Playground assistant output use the shared Markdown renderer where the content is meant to be read as a document. Raw metadata remains available only where the page is explicitly showing original payloads.
+- Added developer workflow scripts:
+  - `build:watch`
+  - `preview:auto`
+  - `bun run build` remains a production artifact build and does not reload an already-open dev page by itself.
+
+## Manual Verification Status
+
+The code has been updated. The list below records which areas are already manually checked and which still need route-level regression before replacement:
+
+- AI:
+  - Latest manual pass marked provider status, secret reveal, and Playground behavior as checked.
+- DataSources:
+  - Single trigger performs old task-status precheck, handles running-conflict confirmation, supports force recollect, and refreshes task status afterward.
+  - Batch trigger uses old `/datasources/trigger-batch` semantics for selected IDs, filters, and force.
+  - Builtin, custom, and realtime source actions match old enable/disable/start/stop/restart/clear-data behavior.
+  - 2026-05-20 implementation pass: Admin Next builtin datasource rows now expose task/collection status and metrics, and single trigger now performs task-status precheck with running-conflict force confirmation plus post-trigger task refresh.
+  - 2026-05-20 manual pass: builtin trigger, task-status precheck, force recollect, batch trigger, enable/disable, and clear-data were verified in Admin Next.
+- Collection:
+  - Collector config create/update/delete, builtin connect, custom test/sample/run/stop/status, schedule save, history/snapshot, mapping propose/preview/create/activate, and credential guide read/generate/reset all work with real backend payloads.
+  - 2026-05-20 implementation pass: `采集历史 / 快照` now reads real datasource snapshots from `/datasources/snapshots` instead of config rows.
+  - 2026-05-20 implementation pass: default credential guide actions no longer call unsupported providers blindly; unsupported collectors show a clear empty state and supported providers use `barentswatch` / `aisstream`.
+  - Collection action buttons must follow the AI page's single-responsibility rule: navigation/link actions do not save, save actions do not connect/test, and connect/test actions do not persist drafts.
+  - 2026-05-20 manual pass: collector create/edit `auth_config`, mapping propose/preview/create/activate, custom test/sample/run/stop/status, credential guide read/generate/reset, and history/snapshot behavior were verified in Admin Next.
+  - 2026-05-20 follow-up: dead JSON modal create/mapping code was removed so Collection creation now goes through the list-bottom draft workflow only.
+- Earth Content:
+  - Brand preview visually matches the Earth runtime left-top brand at the same structural level.
+  - Brand upload writes back `logo_src` or `title_src` correctly.
+  - 2026-05-20 implementation pass: Brand upload validates and displays accepted suffixes (`png`, `jpg`, `jpeg`, `webp`, `svg`) and supports drag-and-drop.
+  - Boundary status/config/build and TV add/save/default/delete/reset match old behavior.
+- Logs:
+  - Sources, snapshot/tail, level/date/search, copy, empty/error states, and internal scrolling all work.
+  - 2026-05-20 implementation pass: Logs now include database-backed `system-db` and `audit-db` sources as a fallback when file/docker/buffer logs are empty or missing, and the UI auto-selects an available source. Manual regression must still verify snapshot/tail-like reading, filters, copy, and scrolling.
+- BGP / Alerts:
+  - BGP overview, collectors, collect-location detail, incidents, anomalies, events, latest/detail/generate AI brief.
+  - System/BGP/situational alert detail, ack, resolve with note, refresh after mutation, and AI brief details.
+- Users/Auth:
+  - Users CRUD, role, Gatekeeper groups.
+  - 2026-05-20 implementation pass: User edit select dropdowns now use an opaque high-z-index menu surface, and Gatekeeper checkbox groups use an explicit surface background. Manual visual regression is still required.
+  - Login, register, verify email, resend cooldown, forgot/reset password.
+- Mobile:
+  - 375px, 430px, and 768px widths default to list/group first.
+  - Detail opens only after tap/click and has a visible back action.
+  - Long forms, JSON, logs, and Playground messages scroll internally without body-level double scrollbars.
+
+## Explicit Non-Production Items
+
+The old AntD page itself treated these Earth content tabs as placeholder-level capabilities. Admin Next must show a visible “backend capability pending” empty state and must not mix in unrelated data:
+
+- `basemap`
+- `layer_resources`
+- `models_3d`
+- `news_anchor_strategy`
+
+If backend endpoints are later added, these items must be promoted into a new active plan under `docs/plans/` with concrete API and UI acceptance criteria.
+
+## Final Gate
+
+After the route promotion, the final gate is no longer “switch old routes.” The remaining gate is “keep legacy available until the promoted routes are stable enough to remove AntD.”
+
+1. Run `cd frontend && /home/ray/.bun/bin/bun run build`.
+2. Run the static checks:
+   - `rg "map: \\(\\) => \\[\\]|暂不支持保存|placeholder" frontend/src/admin-next`
+   - `rg "ShadowPage|FeatureConsole|GlassPanel|InspectorDrawer" frontend/src/admin-next`
+3. Manually verify every official route listed in `docs/deprecated/admin-next-parity-checklist.md`.
+4. Confirm `/legacy/admin/*` still opens old AntD pages during the validation window.
+5. Delete old AntD pages and remove AntD dependencies only as a separate final cleanup task after explicit confirmation.
+
+## Current Build Verification
+
+Last successful local build during this closeout, after the 2026-05-20 follow-up fixes:
+
+```bash
+cd frontend && /home/ray/.bun/bin/bun run build
+```
+
+Result: passed.
+
+Backend syntax check also passed:
+
+```bash
+/home/ray/.local/bin/uv run python -m py_compile backend/app/api/v1/datasources.py backend/app/api/v1/datasource_config.py backend/app/api/v1/system_control.py
+```

docs/deprecated/admin-next-parity-checklist.md

@@ -0,0 +1,43 @@
+# Admin Next Parity Checklist
+
+Last updated: 2026-05-21
+
+This checklist is the hard gate for deleting the old AntD admin. Admin Next now owns the official admin routes; old AntD pages stay available under `/legacy/admin/*` until every production capability below is verified.
+
+## Route Gate
+
+- `/admin` Dashboard: stats, health probe, WebSocket status, restart task, restart logs, recovery probe.
+- `/datasources`: builtin sources, custom sources, realtime sources, detail/stat/task-status, trigger, batch trigger, enable/disable, clear data, realtime start/stop/restart, custom source test/sample/run/stop/status link.
+- `/data`: list, summary, source/type/country filters, search, pagination, detail metadata, export, distribution chart.
+- `/collection-management`: collector config, mapping templates, target schemas, collection schedule, history/snapshot, create config, propose/preview/create/activate mapping, builtin connect, run/stop/stream-status, credential guide read/generate/reset.
+- `/settings`: system display, notifications, security, SMTP get/save/test with recipient.
+- `/ai`: model providers, tool calling, prompts, key reveal, provider refresh, connect tests, OCR/WebSearch secrets, Playground thread/chat/edit/resend/stop.
+- `/earth-content`: brand preview as Earth renders it, upload/save/delete/reset, boundary status/build/config, TV source add/save/default/delete/reset.
+- `/logs`: sources, filters, snapshot, empty/error states, copy.
+- `/bgp`: overview, collectors, collect-location, incidents, anomalies, events, AI brief list/latest/detail/generate.
+- `/alerts/system`, `/alerts/bgp`, `/alerts/situational`: list/stat/detail, ack, resolve with note, AI brief generation.
+- `/users`: list/create/edit/delete, role, Gatekeeper permission groups.
+- Auth pages: login, register, verify email, resend cooldown, forgot/reset password.
+
+Compatibility routes under `/admin-next/*` should redirect to these official paths and are not a separate validation target.
+
+## Verified Manual Passes
+
+- 2026-05-20 `/admin-next/collection-management`: user verified collector create/edit payloads, mapping propose/preview/create/activate, custom collector test/sample/run/stop/status, credential guide read/generate/reset, and collection history/snapshot behavior after the draft-form and Time Capsule updates.
+- 2026-05-20 `/admin-next/datasources`: user verified builtin source trigger, task-status precheck, force recollect, batch trigger, enable/disable, and clear-data behavior.
+
+## Known Non-Production Tabs
+
+The old AntD page also marked these Earth content tabs as placeholder-level capabilities. Admin Next must show them as “backend capability pending” and must not mix in unrelated data:
+
+- `basemap`
+- `layer_resources`
+- `models_3d`
+- `news_anchor_strategy`
+
+## Replacement Rules
+
+- Keep `/legacy/admin/*` available until the matching route gate above is manually verified after promotion.
+- Do not delete old AntD pages, old layout helpers, `antd`, or `@ant-design/icons` until final explicit confirmation.
+- Do not use fake rows, unrelated endpoint data, or empty adapters for a real old capability.
+- Any backend-missing capability must be recorded here with the missing endpoint and visible UI empty state.

docs/deprecated/admin-next-soft-glass-goal-driven-plan.md

@@ -0,0 +1,216 @@
+# Admin Next Soft Glass Goal-Driven Plan
+
+## Goal
+
+Rebuild `/admin-next/*` into a modern soft-glass / light-neumorphic console while keeping the legacy Ant Design admin routes available for comparison. The new console must use visual references only as design anchors, not as imported templates, and must be implemented as Planet-owned reusable components and page patterns.
+
+The redesign must cover desktop and mobile. Data display, icon semantics, table readability, and scroll behavior are first-class acceptance criteria.
+
+## Criteria For Success
+
+- This archived plan exists at `docs/deprecated/admin-next-soft-glass-goal-driven-plan.md`.
+- `/admin-next/*` has real pages for every route; route usage of `ShadowPage` is removed.
+- Admin Next supports `system`, `light`, and `dark` theme modes using the same persistence and system-theme idea as Docs.
+- The visual language reads as soft-glass / light-neumorphic instead of an AntD reskin: translucent panels, fine borders, subtle glow, cool backgrounds, restrained accent colors, crisp icons, and tactile controls.
+- Mobile is explicitly designed: mobile navigation, filter sheets/chips, card-list data views, full-screen detail sheets, and bottom action bars instead of squeezed desktop tables.
+- Tables do not show double scrollbars. Long fields are inspectable, copyable, or expandable; important columns stay readable.
+- Existing custom scrollbar components remain the default scroll experience.
+- `cd frontend && bun run build` passes.
+- Static checks can prove no route-level placeholder remains.
+
+## Reference Strategy
+
+Use references to calibrate feel, not to take over code:
+
+- User-provided soft-glass / light-neumorphic dashboard images define the desired mood.
+- Shadcn / Tailwind / glassmorphism admin templates may be inspected for shadow, spacing, panel, dark-mode, and app-shell ideas.
+- Do not import a complete template, router, auth layer, state layer, or business page.
+- Planet-specific interactions such as datasource tasks, BGP briefs, AI settings, log streams, Earth resources, and custom scrollbars are implemented in this repo.
+
+## Source-Of-Truth Mining
+
+Design content and page boundaries from existing repo history and docs:
+
+- `docs/CHANGELOG.md`
+- `docs/plans/*`
+- `docs/technical/*frontend*`
+- `TODO.md`
+
+Important current boundaries:
+
+- `DataSources` is a datasource directory and runtime-control page, not a configuration editor.
+- `Collection Management` owns collector endpoint, credential, custom source, mapping, and runtime configuration.
+- `Earth Content` owns TV livestreams, brand assets, boundary precision, basemap, and Earth display resources.
+- `AI` owns provider settings, tools, prompts, and Playground.
+- `BGP` is an observability and evidence workspace, not just tables.
+- `Alerts` is a duty/analysis workspace for system, BGP, and situational risk.
+- `Logs` should become a usable log workbench with filtering and structured detail.
+- `Settings` should keep platform settings only.
+
+## Reusable Architecture
+
+Recommended structure:
+
+- `admin-next/design`: theme tokens, theme mode, status colors, icon semantics, chart themes, scrollbar themes.
+- `admin-next/components`: primitive UI such as button, input, select, tabs, dialog, drawer, badge, status pill, icon badge.
+- `admin-next/patterns`: page patterns such as `PageFrame`, `GlassPanel`, `MetricCard`, `CommandBar`, `FilterRail`, `InspectorDrawer`, `DataWorkspace`, `SettingsWorkspace`, `ScrollableRegion`, `TableRegion`, and `MobileSheet`.
+- `admin-next/features`: business modules such as datasources, data-list, bgp, alerts, ai, logs, settings, earth-content, collection-management, users.
+- `admin-next/services`: feature API gateways and hooks. Pages should not scatter URL construction.
+- `admin-next/routes`: route manifest used by navigation, command search, breadcrumbs, and legacy links.
+
+## Interaction Rules
+
+- Row click opens an inspector; frequent row actions live in the inspector header.
+- Bulk selection opens a floating command bar.
+- Common filters stay visible; advanced filters open a sheet/drawer.
+- Detail, JSON, log context, and AI brief use inspector or reading panels instead of stacked modal dialogs.
+- Settings are split into `FormSection` blocks with local save, dirty state, connection test, and reset.
+- Dangerous actions use a confirm sheet.
+- Loading, empty, error, and retry states are shared through reusable patterns.
+
+## Mobile Rules
+
+- Do not squeeze desktop layout into mobile.
+- Use mobile navigation and sheets instead of a permanent desktop sidebar.
+- Filters collapse into a filter sheet; active filters remain visible as chips.
+- Data workspaces default to card-list mode on mobile and allow explicit table mode for dense comparison.
+- Details open in full-screen mobile sheets.
+- Batch actions use a sticky bottom command bar.
+- Long logs and JSON use full-screen reading/editing surfaces with custom scrollbars.
+
+## Table And Data Display Rules
+
+- Tables must prioritize complete data display:
+  - key columns get stable widths and priority;
+  - long fields have peek/expand or inspector detail;
+  - IDs, URLs, metadata, and errors are copyable;
+  - complex fields are rendered fully in inspector.
+- Desktop tables use one `TableRegion` scroll container.
+- Mobile defaults to card-list display and supports dense table mode.
+- Native scrollbar visuals are hidden where custom scrollbars apply.
+
+## Icon Rules
+
+- Use `lucide-react`.
+- Define icon semantics by category:
+  - navigation icon;
+  - entity icon;
+  - status icon;
+  - action icon;
+  - severity icon.
+- Icons must not be the only semantic carrier unless the icon is universally obvious; use labels or tooltips.
+- Status icons pair with `StatusPill`.
+- Colors come from tokens, not ad hoc hard-coded values.
+
+## Page Design
+
+### Dashboard
+
+- System health, realtime connection, datasource activity, task trend, alert summary, recent events, and quick commands.
+- Restart flow uses a confirm sheet and log console.
+- Mobile uses horizontal metric cards and collapsible event/command panels.
+
+### DataSources
+
+- Datasource directory and runtime control.
+- Desktop: overview rail, datasource table/list, inspector.
+- Mobile: datasource cards with status, latest task, trigger/toggle actions.
+- Batch trigger uses command bar.
+- Endpoint/config/task details are shown fully in inspector.
+
+### DataList
+
+- Collection result browsing, search, source/type filters, paging, distribution chart, and detail inspector.
+- Fix double scrollbars.
+- Mobile defaults to cards and allows table mode.
+
+### BGP
+
+- Views: collectors, incidents, anomalies, events, AI brief.
+- Emphasize severity, region, ASN/prefix, collector coverage, and evidence.
+- AI brief shows facts, judgment, and evidence gaps.
+
+### Alerts
+
+- Real pages for system, BGP, and situational alerts.
+- Shared alert workspace with stats, severity/status filters, list, and inspector.
+- AI brief appears in inspector/mobile sheet.
+
+### AI
+
+- Provider, tools, prompts, and Playground.
+- Provider and tool settings use local save and connection tests.
+- Prompt registry is grouped by task.
+- Mobile Playground uses step tabs rather than cramped columns.
+
+### Logs
+
+- Log workbench with source, level, date, search, refresh, copy, and structured detail.
+- Desktop uses a terminal-like log stream.
+- Mobile rows expand or open a full-screen reading sheet.
+
+### Users
+
+- Search, role filter, create/edit/delete, Gatekeeper groups.
+- Edit/create in drawer/sheet.
+- Gatekeeper groups use checkbox chips.
+
+### Settings
+
+- Platform settings only: system, notification, security, SMTP.
+- Each section saves independently.
+
+### Earth Content
+
+- TV livestreams, brand assets, boundary precision, basemap, and layer resources.
+- Resource previews, upload state, and boundary build status are emphasized.
+
+### Collection Management
+
+- Collector config, custom sources, mapping, runtime control.
+- Custom source creation uses a step sheet.
+- Advanced JSON is isolated in a collapsible editor.
+
+## Milestones
+
+1. Save this plan and establish goal-driven criteria.
+2. Build design/theme/pattern foundations.
+3. Remove route-level placeholders and create real pages for all `/admin-next/*` routes.
+4. Redesign layout, theme, mobile shell, scroll behavior, and table behavior.
+5. Fill core workspaces with real API calls and reusable patterns.
+6. Verify build, no `ShadowPage` route usage, and core route availability.
+
+## Verification
+
+Run:
+
+```bash
+cd frontend && bun run build
+rg "ShadowPage" frontend/src/admin-next
+rg "axios\\." frontend/src/admin-next/pages
+rg "overflow: auto|overflow-y: auto" frontend/src/admin-next
+```
+
+Manual route checks:
+
+- `/admin-next`
+- `/admin-next/datasources`
+- `/admin-next/data`
+- `/admin-next/bgp`
+- `/admin-next/alerts/system`
+- `/admin-next/alerts/bgp`
+- `/admin-next/alerts/situational`
+- `/admin-next/ai`
+- `/admin-next/logs`
+- `/admin-next/users`
+- `/admin-next/settings`
+- `/admin-next/earth-content`
+- `/admin-next/collection-management`
+
+Manual viewport checks:
+
+- desktop;
+- mobile width;
+- low height;
+- 125% / 150% browser zoom;
+- light / dark / system theme modes.

docs/deprecated/location-resolver-shared-pipeline-plan.md

@@ -1,6 +1,6 @@
 # Location Resolver Shared Pipeline Plan
 
-**状态**：已实现，当前用户流程见 [Earth 位置候选采集使用手册](/home/ray/dev/linkong/planet/docs/technical/zh/location-pipeline-user.md)，开发接口见 [通用位置估算管线开发说明](/home/ray/dev/linkong/planet/docs/technical/zh/location-pipeline-development.md)。
+**状态**：已实现，当前用户流程见 [Planet 使用手册](/home/ray/dev/linkong/planet/docs/technical/zh/manual.md) 的 Earth 位置候选采集章节，开发接口见 [通用位置估算管线开发说明](/home/ray/dev/linkong/planet/docs/technical/zh/location-pipeline-development.md)。
 
 ## Goal
 

docs/documentation-coverage-rules.md

@@ -37,7 +37,7 @@ If the same action has both a UI and a CLI path (e.g. user creation), describe t
   - a `credential_provider` in `backend/app/core/datasource_defaults.py`;
   - a default credential guide in `backend/app/services/credential_guides.py`;
   - a supported connectivity provider in `backend/app/services/datasource_connectivity.py`;
-  - settings UI guidance or a credential form in `frontend/src/pages/Settings/Settings.tsx`;
+  - settings UI guidance or a credential form in `frontend/src/admin/pages/PlainResourcePages.tsx`;
   - a regression test that fails if the guide/provider is missing.
 
 ## Recommended Checks

docs/plans/README.md

@@ -7,6 +7,12 @@
 - 分几期做
 - 当前差距和下一步是什么
 
+当前实现、业务架构和数据链路不放在这里。它们分别进入：
+
+- [业务架构与数据流转](/home/ray/dev/linkong/planet/docs/technical/zh/platform-data-flows.md)
+- [数据作业与 Outbox 技术架构](/home/ray/dev/linkong/planet/docs/technical/zh/data-job-earth-sync-architecture.md)
+- [技术文档索引](/home/ray/dev/linkong/planet/docs/technical/zh/README.md)
+
 适合放入这里的内容：
 
 - Earth / BGP / 地形 / 天球实施方案
@@ -16,34 +22,40 @@
 
 当前重点入口：
 
-- [earth-mobile-drawer-ui-plan.md](/home/ray/dev/linkong/planet/docs/plans/earth-mobile-drawer-ui-plan.md)
-- [earth-compute-center-bgp-style-plan.md](/home/ray/dev/linkong/planet/docs/plans/earth-compute-center-bgp-style-plan.md)
-- [earth-renderer-architecture-separation-plan.md](/home/ray/dev/linkong/planet/docs/plans/earth-renderer-architecture-separation-plan.md)
+- [Earth Mobile Drawer UI Plan](/home/ray/dev/linkong/planet/docs/plans/earth-mobile-drawer-ui-plan.md)
+- [Earth Compute Center BGP Style Plan](/home/ray/dev/linkong/planet/docs/plans/earth-compute-center-bgp-style-plan.md)
+- [Earth Renderer Architecture Separation Plan](/home/ray/dev/linkong/planet/docs/plans/earth-renderer-architecture-separation-plan.md)
 - [Earth 高精度国界静态瓦片计划](/home/ray/dev/linkong/planet/docs/plans/earth-high-precision-boundary-tiles-plan.md)
-- [earth-predicted-orbit-plan.md](/home/ray/dev/linkong/planet/docs/plans/earth-predicted-orbit-plan.md)
-- [earth-webgl-instancing-satellites-plan.md](/home/ray/dev/linkong/planet/docs/plans/earth-webgl-instancing-satellites-plan.md)
-- [earth-real-terrain-plan.md](/home/ray/dev/linkong/planet/docs/plans/earth-real-terrain-plan.md)
-- [earth-news-source-configuration-and-collector-plan.md](/home/ray/dev/linkong/planet/docs/plans/earth-news-source-configuration-and-collector-plan.md)
-- [earth-news-cruise-summary-plan.md](/home/ray/dev/linkong/planet/docs/plans/earth-news-cruise-summary-plan.md)
+- [Earth Predicted Orbit Plan](/home/ray/dev/linkong/planet/docs/plans/earth-predicted-orbit-plan.md)
+- [Earth WebGL Instancing Satellites Plan](/home/ray/dev/linkong/planet/docs/plans/earth-webgl-instancing-satellites-plan.md)
+- [Earth Real Terrain Plan](/home/ray/dev/linkong/planet/docs/plans/earth-real-terrain-plan.md)
+- [Earth News Source Configuration And Collector Plan](/home/ray/dev/linkong/planet/docs/plans/earth-news-source-configuration-and-collector-plan.md)
+- [Earth News Cruise Summary Plan](/home/ray/dev/linkong/planet/docs/plans/earth-news-cruise-summary-plan.md)
 - [Earth 动作捕捉手势控制计划](/home/ray/dev/linkong/planet/docs/plans/earth-motion-capture-gesture-control-plan.md)
 - [Earth 动捕交互语义 V2 计划](/home/ray/dev/linkong/planet/docs/plans/earth-motion-gesture-interaction-v2-plan.md)
 - [Earth Presentation 解耦架构计划](/home/ray/dev/linkong/planet/docs/plans/earth-presentation-decoupled-architecture-plan.md)
-- [earth-vessel-rendering-performance-plan.md](/home/ray/dev/linkong/planet/docs/plans/earth-vessel-rendering-performance-plan.md)
+- [Earth Vessel Rendering Performance Plan](/home/ray/dev/linkong/planet/docs/plans/earth-vessel-rendering-performance-plan.md)
 - [AIS 多源采集、冲突记录与聚合接口计划](/home/ray/dev/linkong/planet/docs/plans/earth-vessel-ais-aggregation-plan.md)
-- [earth-interactable-layer-plan.md](/home/ray/dev/linkong/planet/docs/plans/earth-interactable-layer-plan.md)
-- [frontend-public-docs-site-plan.md](/home/ray/dev/linkong/planet/docs/plans/frontend-public-docs-site-plan.md)
-- [Docs Gatekeeper 鉴权系统计划](/home/ray/dev/linkong/planet/docs/plans/docs-gatekeeper-auth-plan.md)
-- [Location Resolver 共享管线计划](/home/ray/dev/linkong/planet/docs/plans/location-resolver-shared-pipeline-plan.md)
-- [frontend-ai-playground-development-plan.md](/home/ray/dev/linkong/planet/docs/plans/frontend-ai-playground-development-plan.md)
+- [Earth Interactable Layer Plan](/home/ray/dev/linkong/planet/docs/plans/earth-interactable-layer-plan.md)
+- [Frontend Public Docs Site Plan](/home/ray/dev/linkong/planet/docs/plans/frontend-public-docs-site-plan.md)
+- [Frontend AI Playground Development Plan](/home/ray/dev/linkong/planet/docs/plans/frontend-ai-playground-development-plan.md)
+- [AI Provider OpenClaw-Style Routing Plan](/home/ray/dev/linkong/planet/docs/plans/ai-provider-openclaw-style-routing-plan.md)
+- [统一集成配置 Schema 系统计划](/home/ray/dev/linkong/planet/docs/plans/integration-config-schema-system-plan.md)
 - [Lightweight Agent Orchestrator 与 WebSearch 证据层计划](/home/ray/dev/linkong/planet/docs/plans/agents-light-orchestrator-websearch-plan.md)
-- [ue5-mvp-fused-plan.md](/home/ray/dev/linkong/planet/docs/plans/ue5-mvp-fused-plan.md)
+- [UE5 MVP Fused Plan](/home/ray/dev/linkong/planet/docs/plans/ue5-mvp-fused-plan.md)
+
+历史计划入口：
+
+- 已完成、已替代或只作为历史决策背景保留的文档，统一放在 [Deprecated Docs](/home/ray/dev/linkong/planet/docs/deprecated/README.md)。
 
 不适合放入这里的内容：
 
 - 当前代码结构说明
 - 组件现状和实现入口
 - 已经落地的技术上下文说明
+- 已经成为当前行为的“架构说明”
 
 这些应放入：
 
-- [docs/technical/README.md](/home/ray/dev/linkong/planet/docs/technical/README.md)
+- [技术文档索引](/home/ray/dev/linkong/planet/docs/technical/zh/README.md)
+- [Technical Docs Index](/home/ray/dev/linkong/planet/docs/technical/en/README.md)

docs/plans/ai-provider-openclaw-style-routing-plan.md

@@ -0,0 +1,182 @@
+# AI Provider OpenClaw-Style Routing Plan
+
+Last updated: 2026-05-20
+
+## Summary
+
+Planet 的 AI Provider 路由要从“运行时识别特殊 provider / 特殊模型”收敛到 OpenClaw 风格的配置驱动模型：模型引用、协议、鉴权、轻量探测、真实调用和模型级例外都由 provider catalog / preset / runtime metadata 描述，运行时只解释这些元数据，不再散落 `if provider == ... and model == ...` 这类硬编码。
+
+这份计划覆盖 Admin Next 的 Provider 配置体验、backend settings API、`aiprovider` 适配服务和未来模型目录同步方式。目标是让 OpenCode Go、MiniMax、DeepSeek、OpenAI-compatible、Anthropic-compatible、Ollama、OpenRouter / One API 类代理都能用同一套规则扩展。
+
+## Background
+
+当前实现已经完成了两步临时修正：
+
+- OpenCode Go 模型目录不再使用普通 Zen free 列表，而是使用 `https://opencode.ai/zen/go/v1/models`。
+- `minimax-m2.7` / `minimax-m2.5` 的协议例外已从 `aiprovider` 运行逻辑移到 `model_provider_apis` 元数据中。
+
+但整体还没有完全达到 OpenClaw 式结构。OpenClaw 的关键思想是：
+
+- 模型引用使用 `provider/model`，由 provider 前缀确定 runtime provider。
+- provider 插件或 catalog 拥有 `normalizeModelId`、`normalizeTransport`、`normalizeConfig`、`prepareRuntimeAuth`、`createStreamFn` 等 provider 行为。
+- 主推理循环不认识具体模型名，只使用解析后的 provider config、transport 和 request adapter。
+- 上游网关能自己路由时，尽量透传 provider routing metadata，不在本地复制上游逻辑。
+
+Planet 不需要完整复制 OpenClaw 插件系统，但需要学习它的边界划分。
+
+## Design Principles
+
+- Provider catalog 是路由事实来源，runtime 不是。
+- 模型级协议例外必须是 metadata，例如 `model_provider_apis`，不能是 Python set / if 分支。
+- 轻量连通性测试只验证网络、鉴权和模型目录，不发真实 prompt。
+- 真实模型调用只发生在 Playground、AI brief、分析任务等明确需要生成的路径。
+- 保存配置不自动设为默认，不自动触发连接测试；保存、设默认、测试三种按钮职责分离。
+- 目录刷新使用增量合并语义：发现新模型，标记旧模型 stale，不直接删除用户选择或自定义模型。
+- 如果 provider 不提供可靠 `/models`，可以用内置 preset 确认已知模型，但 UI 必须说清楚这是 preset confirmation，不是假装 provider 返回了目录。
+
+## Target Data Model
+
+Provider preset / runtime config 应逐步收敛为类似结构：
+
+```json
+{
+  "provider": "opencode-go",
+  "label": "OpenCode Go",
+  "default_transport": "openai-completions",
+  "base_url": "https://opencode.ai/zen/go/v1",
+  "auth": {
+    "type": "bearer",
+    "api_key_env": "OPENCODE_GO_API_KEY"
+  },
+  "models": [
+    {
+      "id": "glm-5.1",
+      "label": "GLM 5.1",
+      "transport": "openai-completions",
+      "context_window": null,
+      "capabilities": ["text"]
+    },
+    {
+      "id": "minimax-m2.7",
+      "label": "MiniMax M2.7",
+      "transport": "anthropic-messages",
+      "capabilities": ["text", "reasoning"]
+    }
+  ],
+  "discovery": {
+    "type": "openai-models",
+    "url": "https://opencode.ai/zen/go/v1/models",
+    "auth": "provider-api-key"
+  }
+}
+```
+
+Runtime 选择规则：
+
+1. 解析 provider。
+2. 解析 model。
+3. 从 `models[].transport` 找模型级 transport。
+4. 若没有模型级 transport，使用 provider `default_transport`。
+5. 将解析结果传给 `aiprovider`。
+6. `aiprovider` 只按 `transport` 组装请求，不认识 provider 专属模型名。
+
+## Implementation Plan
+
+### Phase 1: Stabilize Current Metadata Path
+
+- Keep `model_provider_apis` as the immediate compatibility bridge.
+- Ensure `_provider_defaults()` includes provider metadata such as `model_provider_apis`.
+- Ensure `_runtime_config_from_ai_payload()` sends the resolved metadata through `AIProviderClient`.
+- Ensure `AIProviderClient` forwards metadata to `aiprovider` with a structured header.
+- Ensure `aiprovider.ProviderService` reads model metadata and resolves `provider_api = model_provider_apis[model] ?? provider_api`.
+- Add tests proving `aiprovider` does not contain provider/model-specific literals for routing decisions.
+
+### Phase 2: Replace `model_provider_apis` With Structured Model Catalog
+
+- Extend `backend/app/services/llm_provider_catalog.py` preset shape with `models_metadata`.
+- Preserve old `models` as a compatibility list for the UI.
+- Add helpers:
+  - `get_provider_model_metadata(provider, model)`
+  - `resolve_provider_transport(provider_config, model)`
+  - `merge_discovered_models(existing, discovered)`
+- Return both `models` and `models_metadata` from refresh endpoints.
+- Admin Next should render model labels, capabilities and transport hints from metadata.
+
+### Phase 3: Provider Discovery And Incremental Sync
+
+- Add provider discovery descriptors:
+  - OpenAI-compatible `/models`
+  - Anthropic-compatible no-models / preset-confirmed path
+  - Ollama `/api/tags`
+  - OpenCode Go `/zen/go/v1/models`
+  - OpenRouter / One API passthrough model discovery
+- Add incremental merge behavior:
+  - New discovered model: add.
+  - Existing discovered model: update `last_seen_at`, metadata.
+  - Missing discovered model: mark `stale`, do not delete.
+  - User custom model: keep unless explicitly removed.
+- Surface discovery source in Admin Next: `实时发现 / 内置预设 / 用户自定义 / 已过期`.
+
+### Phase 4: Transport Adapters
+
+- Replace provider-specific request decisions with adapter descriptors:
+  - `openai-completions`
+  - `anthropic-messages`
+  - `ollama-generate`
+  - future `openai-responses`
+  - future `gemini-generate-content`
+- Each adapter owns:
+  - path
+  - auth header format
+  - request body transform
+  - response text extraction
+  - reasoning/thinking block extraction
+  - models endpoint strategy
+- `ProviderService.analyze()` should select adapter by resolved transport and call the adapter.
+
+### Phase 5: Admin Next UX
+
+- Model provider page should show:
+  - provider status tag
+  - default model tag
+  - source tag: env / runtime / preset / discovered
+  - model list with transport/capability hint
+  - separate buttons for save, set default, refresh model catalog, lightweight test
+- The connect plug button remains lightweight.
+- Full generation test lives only in Playground or a clearly named “试运行” action.
+- If lightweight test falls back to preset confirmation, toast must say so explicitly.
+
+## TODO
+
+- [ ] Add `models_metadata` to provider presets and refresh responses.
+- [ ] Add runtime resolver helper for provider/model transport selection.
+- [ ] Remove any remaining provider/model-specific literals from `aiprovider` runtime routing.
+- [ ] Add tests that `opencode-go/minimax-m2.7` resolves through metadata, not through runtime hardcode.
+- [ ] Add tests for lightweight connectivity:
+  - [ ] 401 / 403 fail as auth error.
+  - [ ] 404 with known preset model passes as preset-confirmed.
+  - [ ] `/models` missing alias passes only when preset contains the alias.
+  - [ ] unknown model fails.
+- [ ] Add discovery descriptors for OpenCode Go, OpenAI-compatible, Anthropic-compatible, Ollama, OpenRouter / One API.
+- [ ] Add incremental model catalog merge semantics with stale marking.
+- [ ] Update Admin Next model list to show model source, transport and capability.
+- [ ] Keep save / set default / lightweight test / full test as separate actions.
+- [ ] Document the final provider catalog schema in technical docs after implementation.
+
+## Current Acceptance Criteria
+
+- No runtime routing branch may depend on concrete model names like `minimax-m2.7`.
+- OpenCode Go model refresh must not use the ordinary Zen free-model endpoint.
+- Lightweight connect must not call `analyze()` or consume generation quota.
+- Saving a provider must not automatically set it as default.
+- Provider UI must distinguish configured key, fallback key, preset model and live-discovered model.
+
+## Related Files
+
+- `backend/app/services/llm_provider_catalog.py`
+- `backend/app/api/v1/settings.py`
+- `backend/app/services/ai_client.py`
+- `aiprovider/main.py`
+- `aiprovider/provider_service.py`
+- `frontend/src/admin/pages/PlainResourcePages.tsx`
+- `docs/deprecated/admin-next-parity-audit-closeout-plan.md`

docs/plans/docs-audience-split-plan.md

@@ -57,7 +57,7 @@
 
 抽自现 manual.md，重新组织：
 
-1. 首次启动 — `./planet.sh start`、默认账号（`admin/admin123`、`linkong/12345678`，引用 `b15d097b` 引入的 `DEFAULT_LOGIN_USERS`）
+1. 首次启动 — `./planet.sh start`、默认账号（`admin/admin123`、`linkong/LK12345678`，引用 `b15d097b` 引入的 `DEFAULT_LOGIN_USERS`）
 2. 启停与按模块重启 — `start/stop/restart` 及 `-b -f -a -d`
 3. 健康检查 — `./planet.sh health`
 4. 日志 — `./planet.sh log` 及 `-f -b -a`，日志文件路径

docs/plans/integration-config-schema-system-plan.md

@@ -0,0 +1,349 @@
+# 统一集成配置 Schema 系统计划
+
+Last updated: 2026-05-20
+
+## Summary
+
+Planet 的采集器、AI Provider 和工具调用配置需要从“页面各自硬编码字段”收敛到同一套低代码 schema 系统。系统负责两件事：
+
+- 用后台可编辑 schema 生成配置表单。
+- 按字段 target 把表单值组装成后端运行时需要的请求、凭证和 JSON 配置。
+
+这套 schema 不替代 `target_schema_registry`。`target_schema_registry` 继续负责采集结果映射和校验；本计划中的 `integration_config_schemas` 负责“怎么配置一个集成”。
+
+## Current Problems
+
+- Admin Next 的采集器配置曾把不同凭证形态压成通用 `api_key`，导致 `barentswatch_vessels` 这种 OAuth client credentials 丢失 `client_id` 字段。
+- AI Provider、Web Search、OCR 和 DataSource 配置各自维护表单字段、secret 处理和 payload 组装逻辑，重复且容易漂移。
+- 新增字段时需要改前端字段列表和保存逻辑，无法做到后台配置化扩展。
+
+## Target Model
+
+新增统一 registry：`integration_config_schemas`，存储在 `SystemSetting.payload`。
+
+Registry 包含：
+
+- `fragments`：可复用字段片段，例如 endpoint、API Key、OAuth Client、HTTP 请求、WebSocket 订阅、AI Provider 基础字段、工具超时字段。
+- `auth_schemas`：可复用认证编排，例如 API Key、Bearer Token、Basic、OAuth2 Client Credentials、OAuth2 Authorization Code、Session Cookie Login。
+- `schemas`：具体配置对象使用的 schema，例如 `datasource:barentswatch_vessels`、`ai_provider:minimax`、`tool:web_search:tavily`。
+- `defaults`：每类集成的默认 schema，例如 `datasource`、`ai_provider`、`tool`。
+
+Schema 必须声明 `kind`：
+
+- `datasource`
+- `ai_provider`
+- `tool`
+
+字段定义统一使用：
+
+```json
+{
+  "key": "client_secret",
+  "label": "Client Secret",
+  "type": "secret",
+  "secret": true,
+  "target": "auth_config.client_secret",
+  "required": true,
+  "wide": true
+}
+```
+
+字段 target 支持写入：
+
+- DataSource：`root.*`、`auth_config.*`、`headers.*`、`config.*`
+- AI Provider：`ai_provider.*`、`ai_provider.providers.{provider}.*`
+- Tool：`web_search.*`、`web_search.providers.{provider}.*`、`ocr.*`
+
+## Auth Schema Model
+
+认证必须成为 schema 系统的一等能力，不能再把所有凭证强行压成 `api_key`。每个配置 schema 可以引用一个 `auth_schema`，也可以内联声明认证编排。
+
+Auth schema 描述：
+
+- 凭证字段：哪些字段是 secret、是否必填、写入哪个 target。
+- 凭证来源：DB、env fallback、运行时草稿、用户授权回调。
+- 预认证请求：例如登录接口、token endpoint、OAuth callback。
+- 凭证注入方式：header、query、form、JSON body、cookie jar、WebSocket subscription payload。
+- reveal 策略：管理员可 reveal 并写 audit log；无 DB/env 值时显示空。
+- 测试策略：连接测试必须使用当前表单草稿优先，再 fallback 到已保存/env。
+
+v1 需要支持的认证类型：
+
+- `none`：无认证。
+- `api_key`：API Key 写入 header/query/form/body。
+- `bearer_token`：Bearer token header。
+- `basic`：username/password，支持直接 Basic header 或 provider 特定登录。
+- `oauth2_client_credentials`：client_id/client_secret 换 access_token。
+- `oauth2_authorization_code`：第三方登录授权，包含 authorize URL、callback、token exchange、refresh。
+- `session_cookie_login`：用户名密码登录后保存 cookie jar，再访问数据接口。
+- `custom_auth_preflight`：无法归类时，用声明式 preflight 请求生成后续请求上下文。
+
+Auth schema 示例：
+
+```json
+{
+  "key": "spacetrack_session",
+  "type": "session_cookie_login",
+  "fields": [
+    {
+      "key": "username",
+      "label": "Username",
+      "target": "auth_config.username",
+      "required": true
+    },
+    {
+      "key": "password",
+      "label": "Password",
+      "type": "secret",
+      "secret": true,
+      "target": "auth_config.password",
+      "required": true
+    }
+  ],
+  "preflight": {
+    "method": "POST",
+    "url": "https://www.space-track.org/ajaxauth/login",
+    "body_type": "form",
+    "body": {
+      "identity": "{{auth_config.username}}",
+      "password": "{{auth_config.password}}"
+    },
+    "success": {
+      "type": "cookie"
+    }
+  },
+  "inject": {
+    "type": "cookie_jar"
+  }
+}
+```
+
+OAuth Authorization Code 示例：
+
+```json
+{
+  "key": "github_oauth",
+  "type": "oauth2_authorization_code",
+  "fields": [
+    { "key": "client_id", "target": "auth_config.client_id", "required": true },
+    { "key": "client_secret", "type": "secret", "secret": true, "target": "auth_config.client_secret", "required": true },
+    { "key": "scopes", "type": "tags", "target": "auth_config.scopes" }
+  ],
+  "authorization": {
+    "url": "https://github.com/login/oauth/authorize",
+    "client_id": "{{auth_config.client_id}}",
+    "scopes": "{{auth_config.scopes}}",
+    "redirect_uri": "{{system.callback_base_url}}/api/v1/integrations/oauth/github/callback"
+  },
+  "token": {
+    "method": "POST",
+    "url": "https://github.com/login/oauth/access_token",
+    "body_type": "form",
+    "body": {
+      "client_id": "{{auth_config.client_id}}",
+      "client_secret": "{{auth_config.client_secret}}",
+      "code": "{{oauth.code}}",
+      "redirect_uri": "{{oauth.redirect_uri}}"
+    }
+  },
+  "inject": {
+    "type": "bearer_header",
+    "token_path": "access_token"
+  }
+}
+```
+
+## Request And Runtime Assembly
+
+这套系统本质是面向 Planet 集成的低代码 Postman，但目标不是临时发请求，而是沉淀成可保存、可测试、可调度、可审计的运行配置。
+
+Schema 支持描述：
+
+- HTTP method：`GET`、`POST`
+- endpoint
+- headers
+- query params
+- JSON body / form body
+- auth schema / auth config / preflight auth flow
+- WebSocket endpoint 和 subscription payload
+- AI Provider 的 `provider_api`、`base_url`、`model`、`api_key`、`service_token`
+- Tool 的 provider、base_url、api_key、timeout 和工具专属参数
+
+请求执行顺序：
+
+1. 从 schema 字段 target 组装 root/auth_config/headers/config。
+2. 如果存在 auth schema，先解析凭证来源和草稿覆盖。
+3. 需要 preflight 时执行认证请求，例如 token exchange 或 login。
+4. 把认证结果注入正式请求，例如 bearer header、cookie jar、query token。
+5. 执行连接测试、采样、采集、AI Provider connect 或 tool connect。
+
+## Default Schemas
+
+### DataSource
+
+默认采集器：
+
+- endpoint
+- method
+- headers JSON
+- query/body/config JSON
+- API Key auth
+- advanced JSON
+
+`barentswatch_vessels`：
+
+- endpoint
+- Client ID -> `auth_config.client_id`
+- Client Secret -> `auth_config.client_secret`
+- fixed `auth_type = oauth_client`
+
+`aisstream_vessels`：
+
+- WebSocket endpoint
+- API Key -> `auth_config.api_key`
+- subscription / bounding boxes config
+
+`spacetrack_tle`：
+
+- API Base URL / endpoint
+- Username -> `auth_config.username`
+- Password -> `auth_config.password`
+- auth schema = `session_cookie_login`
+- login endpoint = `https://www.space-track.org/ajaxauth/login`
+- login body fields：`identity` / `password`
+- run/test request uses returned session cookie
+
+### AI Provider
+
+默认字段：
+
+- provider
+- provider_api
+- base_url
+- model
+- api_key
+- max_tokens
+- anthropic_version
+- service_url
+- service_token
+- timeout_seconds
+- retry_attempts
+- model_provider_apis
+
+Provider presets supply initial defaults, but the editable schema controls which fields appear and where values are saved.
+
+AI Provider auth variants：
+
+- OpenAI-compatible providers：`api_key` or `bearer_token`。
+- Local/sidecar services：`none`、`service_token` or custom header。
+- OAuth-backed providers：`oauth2_authorization_code`，适用于需要用户授权登录的 provider。
+- CLI/session-backed tools such as Codex：优先作为 `tool` 或本机 runner 集成；如果作为 provider，必须显式声明会话来源、权限边界、不可多用户复用的限制。
+
+### Tool
+
+`web_search`：
+
+- enabled
+- provider / default_provider
+- base_url
+- api_key
+- max_results
+- timeout_seconds
+- endpoint_path
+- search_depth
+- engine
+- include_answer / include_raw_content / include_text
+- search_path / scrape_path / scrape_formats
+
+`ocr`：
+
+- enabled
+- provider
+- base_url
+- api_key
+- model
+- languages
+- timeout_seconds
+- max_file_size_mb
+- output_format
+
+Tool auth variants：
+
+- GitHub PAT：`bearer_token`。
+- GitHub OAuth App：`oauth2_authorization_code`，适合用户授权登录和代表用户访问。
+- GitHub App：`app_installation`，需要 app id/private key/installation id，并通过 schema 声明 installation token exchange。
+- Browser/session tools：必须显式标记为 `session_local_only`，不能作为后台多用户稳定凭证。
+
+## API Plan
+
+- `GET /api/v1/integration-config-schemas`
+  - Return the full registry.
+- `PUT /api/v1/integration-config-schemas`
+  - Save the registry. Admin only.
+- `POST /api/v1/integration-config-schemas/validate`
+  - Validate full registry or one schema.
+- `POST /api/v1/integration-config-schemas/auth/test`
+  - Test auth schema with draft credentials without saving.
+- `GET /api/v1/integration-config-schemas/auth/secrets`
+  - Reveal stored/env-backed secret fields for admins; write audit log.
+- `POST /api/v1/integrations/oauth/{provider}/start`
+  - Start OAuth Authorization Code flow.
+- `GET /api/v1/integrations/oauth/{provider}/callback`
+  - Complete OAuth callback and store token material according to schema.
+- `GET /api/v1/datasources/configs/all`
+  - Add `form_schema` to each row.
+- `GET /api/v1/settings/integrations`
+  - Add `form_schema` for AI Provider and tools.
+
+Existing save APIs remain compatible:
+
+- DataSource saves to `DataSourceConfig`.
+- AI Provider, Web Search and OCR save to `external_integrations`.
+
+Secret fields never return plaintext through list/config endpoints. They return configured state and masked preview only.
+
+Secret reveal endpoints return plaintext only on explicit administrator action and must log target, actor, source, result and timestamp. List/config endpoints must never leak secret plaintext.
+
+## Frontend Plan
+
+Admin Next extracts a reusable `SchemaForm`:
+
+- Render fields from `form_schema.fields`.
+- Build payload by writing values to each field `target`.
+- Preserve masked secret semantics: unchanged masked values do not overwrite stored secrets.
+- Reveal secrets through a schema-aware reveal action; if neither DB nor env has a value, show an empty editable input.
+- Support text, secret, number, boolean, select, textarea, JSON and tags controls.
+- Support auth controls for API key, username/password, OAuth connect/disconnect, session cookie login and custom preflight status.
+- Validate schema before saving registry changes.
+
+Pages migrated in v1:
+
+- Collection Management / collector configs.
+- AI / Provider configuration.
+- AI / Tools configuration for Web Search and OCR.
+
+Each detail page gets a schema editing action for admins. The editor saves registry JSON after validation.
+
+## Test Plan
+
+- Default registry initializes with datasource, AI Provider and tool schemas.
+- Validation rejects duplicate keys, illegal targets, illegal field types and secret plaintext defaults.
+- `barentswatch_vessels` renders and saves `client_id` / `client_secret`.
+- `spacetrack_tle` renders username/password, tests with draft credentials, and does not require API Key.
+- `aisstream_vessels` renders API Key and WebSocket subscription fields.
+- Session cookie login auth executes preflight before sample/run and uses the resulting cookie jar.
+- OAuth Authorization Code schema can start callback flow, store token metadata and inject bearer token.
+- Secret reveal returns DB value, env fallback or empty value according to source, and writes audit log.
+- Connection tests always prefer current draft credentials over saved/env credentials.
+- AI Provider renders and saves `provider_api`, `base_url`, `model`, `api_key` and `service_token`.
+- AI Provider and tool schemas can reuse the same auth schema primitives as DataSource.
+- Web Search and OCR render and save provider-specific tool fields.
+- Adding a schema field in the registry makes it appear in Admin Next without frontend code changes.
+- Existing connection tests, datasource sampling, datasource run, AI Provider connect/reveal/refresh, and Web Search connect keep working.
+
+## Assumptions
+
+- `target_schema_registry` remains separate because it describes collected result shape, not configuration forms.
+- The first implementation stores schema registry in `SystemSetting`; no new database table is required.
+- Old AntD Settings pages stay compatible but are not migrated in v1.
+- Tool scope in v1 is Web Search and OCR.

docs/technical/en/README.md

@@ -1,44 +1,52 @@
 # Technical Docs
 
-This directory holds "current implementation and current structure" documentation, focusing on:
+This is the current Intelligent Planet documentation entry point. Docs are organized by reader path: start with business architecture to understand data products, then move into user manuals or implementation references.
 
-- How the code is organized right now
-- Where the current entry points are
-- How state and components work
-- Which implementation boundaries future changes should follow
+## Business Architecture
 
-What belongs here:
+- [Business Architecture and Data Flows](/home/ray/dev/linkong/planet/docs/technical/en/platform-data-flows.md): purpose, collection flow, fact tables, derived tables, cache, and WebSocket broadcast path for each Earth data product
+- [Naming Glossary](/home/ray/dev/linkong/planet/docs/technical/en/naming-glossary.md): English/Chinese terms used across the console, Earth, backend, and docs
 
-- Quickstart and user manual
-- Frontend context
-- Earth frontend structure
-- Earth satellite footprint policy
-- Earth render layer order
-- Earth layer style property index
-- Backend runtime control
-- Collector status
-- Collector settings and connectivity validation
-- Earth Interactable integration
-- Collection format conventions
+## Manual
 
-## Entry Points
+- [Quickstart](/home/ray/dev/linkong/planet/docs/technical/en/quickstart.md): shortest path to getting Intelligent Planet running from scratch
+- [Intelligent Planet Manual](/home/ray/dev/linkong/planet/docs/technical/en/manual.md): user workflows for the console, Earth, Docs, and common features
+- [FAQ](/home/ray/dev/linkong/planet/docs/technical/en/faq.md): troubleshooting for Windows / WSL, ports, dependencies, motion capture, credentials, and Docs permissions
 
-- [Quickstart](/home/ray/dev/linkong/planet/docs/technical/en/quickstart.md): The shortest path to getting Planet running from scratch
-- [Planet Manual](/home/ray/dev/linkong/planet/docs/technical/en/manual.md): Complete usage guide for the console, `planet.sh`, Earth, and Docs
-- [FAQ](/home/ray/dev/linkong/planet/docs/technical/en/faq.md): Central troubleshooting entry for Windows / WSL, ports, dependencies, motion capture, credentials, and Docs permissions
-- [Earth Location Candidate Collection User Guide](/home/ray/dev/linkong/planet/docs/technical/en/location-pipeline-user.md): Collect and preview coordinate candidates for compute centers and BGP collectors on Earth
-- [Collector Settings and Connectivity Validation](/home/ray/dev/linkong/planet/docs/technical/en/datasource-collector-settings-connectivity.md): Data source catalog, collector settings, connectivity validation, and BarentsWatch credentials
-- [Shared Location Resolution Pipeline Development Guide](/home/ray/dev/linkong/planet/docs/technical/en/location-pipeline-development.md): Backend location resolver / pipeline interfaces, registries, and extension points
-- [Docs Gatekeeper Development Guide](/home/ray/dev/linkong/planet/docs/technical/en/docs-gatekeeper-development.md): Backend Docs catalog, Markdown content loading, and Gatekeeper permission groups
-- [Earth Interactable Usage](/home/ray/dev/linkong/planet/docs/technical/en/earth-interactable-usage.md): API, lifecycle, and integration examples for Earth surface icon Interactable
-- [Earth Toolbar and Overlay Coordination](/home/ray/dev/linkong/planet/docs/technical/en/earth-toolbar-overlay-coordination.md): Closing matrix and integration rules for toolbar buttons, search, settings, news, and layer overlays
+## Earth Implementation
 
-What does not belong here:
+- [Earth Frontend Context](/home/ray/dev/linkong/planet/docs/technical/en/earth-frontend-context.md): Earth modules, state, WebSocket refresh, and layer lifecycle
+- [Earth Layer Style Reference](/home/ray/dev/linkong/planet/docs/technical/en/earth-layer-style-reference.md): layer colors, symbols, materials, and visual parameters
+- [Earth Render Layer Order](/home/ray/dev/linkong/planet/docs/technical/en/earth-render-layer-order.md): renderOrder, depth strategy, picking, and collision avoidance
+- [Earth Satellite Footprint Policy](/home/ray/dev/linkong/planet/docs/technical/en/earth-satellite-footprint-policy.md): satellite footprint display boundaries and strategy
+- [BGP Context](/home/ray/dev/linkong/planet/docs/technical/en/earth-bgp-context.md): BGP rendering, aggregation, and collector implementation in Earth
+- [Earth Interactable Usage](/home/ray/dev/linkong/planet/docs/technical/en/earth-interactable-usage.md): `Interactable` API, lifecycle, and integration examples
+- [Earth Toolbar and Overlay Coordination](/home/ray/dev/linkong/planet/docs/technical/en/earth-toolbar-overlay-coordination.md): close matrix for toolbar buttons, search, settings, news, and layer overlays
 
-- Incomplete roadmaps
-- Future iteration plans
-- Large-scale refactor proposals
+## Frontend Implementation
 
-Those belong in:
+- [Admin Frontend Context](/home/ray/dev/linkong/planet/docs/technical/en/frontend-admin-frontend-context.md): console page structure, state boundaries, and lazy loading
+- [Frontend Layout Guidelines](/home/ray/dev/linkong/planet/docs/technical/en/frontend-layout-guidelines.md): layout, tables, panels, and responsive constraints
+- [Tactile UI Components](/home/ray/dev/linkong/planet/docs/technical/en/tactile-ui-components.md): button, switch, tooltip, scrollbar APIs, and theme tokens
 
-- [Plans Index](/home/ray/dev/linkong/planet/docs/plans/README.md)
+## Backend Implementation
+
+- [Data Collectors](/home/ray/dev/linkong/planet/docs/technical/en/backend-collectors.md): collectors, task types, save layer, and status updates
+- [System Service Control](/home/ray/dev/linkong/planet/docs/technical/en/backend-system-service-control.md): backend service control and system operation APIs
+- [Datasource Collector Settings and Connectivity](/home/ray/dev/linkong/planet/docs/technical/en/datasource-collector-settings-connectivity.md): datasource catalog, collector settings, connectivity validation, and credentials
+- [Datasource API Performance](/home/ray/dev/linkong/planet/docs/technical/en/backend-datasources-api-performance.md): DataSources list API performance and caching
+- [Data Jobs and Outbox Architecture](/home/ray/dev/linkong/planet/docs/technical/en/data-job-earth-sync-architecture.md): PostgreSQL job queue, outbox, listener, and Kafka / Spark evolution boundaries
+- [Shared Location Resolution Pipeline Development Guide](/home/ray/dev/linkong/planet/docs/technical/en/location-pipeline-development.md): location resolver / pipeline interfaces, registries, and extension points
+- [News Live Streams Collector Format](/home/ray/dev/linkong/planet/docs/technical/en/earth-news-live-streams-collector-format.md): news, live stream, and media collection payload conventions
+- [Docs Gatekeeper Development Guide](/home/ray/dev/linkong/planet/docs/technical/en/docs-gatekeeper-development.md): backend Docs catalog, Markdown content loading, and Gatekeeper permission groups
+
+## Agents and Operations
+
+- [AI Provider Guide](/home/ray/dev/linkong/planet/docs/technical/en/agents-aiprovider.md): model provider adapters, task prompts, and invocation boundaries
+- [Intelligent Planet Ops Runbook](/home/ray/dev/linkong/planet/docs/technical/en/ops-runbook.md): deployment, startup, troubleshooting, and sensitive operations
+- [Docker + Compose + Buildx Upgrade](/home/ray/dev/linkong/planet/docs/technical/en/ops-docker-compose-buildx-upgrade.md): Docker toolchain upgrade steps
+- [planet.sh Startup](/home/ray/dev/linkong/planet/docs/technical/en/ops-planet-sh-startup.md): startup script, health checks, and performance optimization
+
+## Plans and History
+
+Incomplete roadmaps, large refactor proposals, and future plans live in the repository path `docs/plans/README.md`. Completed, outdated, or replaced designs live in `docs/deprecated/README.md`.

docs/technical/en/agents-aiprovider.md

@@ -101,6 +101,19 @@ The AI settings page uses:
 
 These endpoints require an authenticated user. The `secrets` endpoint is only used when the settings page reveals a key or token; hiding the field restores the masked preview.
 
+Admin keeps the AI page aligned with the legacy information architecture:
+
+- `Model Providers`
+  - Manages provider, wire adapter, default model, LLM API key, proxy URL, proxy token, model refresh, set-as-default, and lightweight connectivity testing.
+- `Tool Calling`
+  - Manages tools such as WebSearch and OCR. Each tool first selects a provider, then edits that provider's API, key, and advanced parameters.
+- `Prompts`
+  - Edits system/user prompts by prompt group and task key. Save and reset only affect the current task.
+- `Playground`
+  - Runs real conversations with the active provider and prompt configuration. AI responses are rendered as Markdown.
+
+Save, set-as-default, and connectivity testing are separate responsibilities: save only persists the form, set-as-default only changes the active provider/tool, and connectivity testing only validates the current draft. It must not implicitly save or switch defaults.
+
 The `ai-prompts` endpoints back the Prompts tab in AI settings. Shipped defaults come from versioned backend resources, while business code references stable task keys. The API stores only operator overrides. Resetting a prompt removes the override and falls back to the current shipped default.
 
 ### Prompt Boundary
@@ -259,6 +272,34 @@ Each provider has its own key slot. Resolution order is:
 
 `.env` is only a fallback. After the settings page saves successfully, or after the connection test succeeds, PostgreSQL becomes the global default source.
 
+Admin must compute key status per provider or tool:
+
+- If the database has a key for the current provider/tool, show `configured`.
+- If the database has no key but the fallback provider, model, or tool matches the current item, show the fallback masked preview.
+- If neither database nor matching fallback exists, show `not configured`; a generic `.env` key for another provider must not make this item appear configured.
+- Masking keeps the prefix before the first `-`, for example `sk-********`. Plaintext reveal is only available inside the authorized configuration page.
+
+Tool keys follow the same rule. WebSearch and OCR must match the current tool and provider before they can use fallback credentials.
+
+### Lightweight Connectivity Testing
+
+The Admin plug button performs a lightweight connectivity check and does not save configuration. Common API-platform practice is two-tiered:
+
+- Check a provider catalog or low-cost endpoint to validate base URL, authentication, and model reachability.
+- Send full model requests only when the user explicitly runs Playground or a business task.
+
+Connectivity results should be explicit:
+
+- `ok`: authentication, route, and model catalog are usable.
+- `warning`: service is reachable, but the current model is missing from the catalog or capability metadata is incomplete.
+- `error`: authentication, network, protocol, or model lookup failed.
+
+Toast titles must match the result; failures must not be titled as a successful connection.
+
+### OpenCode Go Routing Model
+
+Subscription channels such as OpenCode Go should not be handled by hard-coded frontend model sets. Prefer provider catalog or backend capability discovery that records per-model capabilities such as `chat_completions`, `anthropic_messages`, `models_endpoint`, and whether a subscription key is required. The frontend should display capabilities; the backend should map provider, base URL, model, and adapter into the real request.
+
 #### Settings Page Behavior
 
 - The Provider select controls the global default provider.

docs/technical/en/backend-collectors.md

@@ -73,6 +73,8 @@ async def run(self, db):
 
 **Core file**: `backend/app/services/collectors/base.py`
 
+Manual trigger, data clearing, and cache clearing now enter the PostgreSQL data job queue. `collection_tasks` remains the task ledger. Collectors only own `fetch -> transform -> save`; the `data_jobs.py` worker claims `collect` / `clear_data` / `clear_cache` / `earth_refresh` jobs and writes progress back. Earth layer refresh relationships live in `earth_layer_adapters.py`; do not hand-code cache invalidation or WebSocket broadcasts inside individual collectors or buttons.
+
 ## III. Collector List
 
 | Collector | Data type | Content | Frequency |
@@ -93,6 +95,16 @@ Earth boundaries are no longer data collectors. They are Earth static rendering
 
 TOP500 and Epoch AI compute sources do not always provide usable coordinates. The unified Earth compute-center endpoint uses only valid source-provided coordinates or `compute_center_locations` dimension-table coordinates during the main map startup path; records without coordinates are returned as `unresolved` instead of being rendered from a local registry, country centroid, or guessed city. When users manually collect candidates, the backend queries ROR and Nominatim/OpenStreetMap from source fields; accepted candidates are saved into `compute_center_locations` and rendered from that table on the next layer refresh.
 
+Admin collection management follows the business hierarchy instead of flattening every endpoint into one table:
+
+- `Collectors`: endpoint, authentication, headers, base parameters, enabled state, and credential guides.
+- `Collection Schedule`: scheduler state and task controls.
+- `Collection History / Snapshots`: history grouped by collector, with a detail-side snapshot selector for versions.
+
+Snapshot lists should not show every snapshot of the same collector as separate top-level records. The top-level list selects a collector; the detail area switches between time versions.
+
+Credential guides are maintained by `backend/app/services/credential_guides.py`. The console uses read / generate / reset actions to load or create Markdown instructions. The frontend should render the guide Markdown for operators, not expose generation prompts or raw metadata.
+
 ## IV. Data Format (stored in CollectedData table)
 
 ```python

docs/technical/en/backend-datasources-api-performance.md

@@ -96,4 +96,4 @@ if (res.data.task_id) {
 ## Related Files
 
 - [datasources.py](/home/ray/dev/linkong/planet/backend/app/api/v1/datasources.py): `_load_datasource_list_context`, `list_datasources`
-- [DataSources.tsx](/home/ray/dev/linkong/planet/frontend/src/pages/DataSources/DataSources.tsx): `BuiltInDataSource`, `triggerDatasource`
+- [DataSources.tsx](/home/ray/dev/linkong/planet/frontend/src/admin/pages/PlainResourcePages.tsx): `BuiltInDataSource`, `triggerDatasource`

docs/technical/en/data-job-earth-sync-architecture.md

@@ -0,0 +1,141 @@
+# Data Jobs and Outbox Architecture
+
+This document records the technical boundary for Planet v1 data jobs, database outbox, and Earth refresh. For each data product's business purpose and end-to-end flow, see [Business Architecture and Data Flows](/home/ray/dev/linkong/planet/docs/technical/en/platform-data-flows.md).
+
+## Architecture Boundary
+
+- PostgreSQL is the durable v1 job ledger and outbox. Kafka, Celery, and RQ are intentionally not part of v1.
+- `collection_tasks` records collection, data clearing, cache clearing, and non-database Earth refresh jobs.
+- `earth_data_change_events` records fact-table or derived-table changes and is the reliable source for Earth sync.
+- `LISTEN/NOTIFY` is only the low-latency wakeup path; the listener still polls unconsumed outbox rows.
+- Redis is mainly cache, auth helper, OTP / rate limit, temporary logs, and WebSocket support. It is not the durable queue.
+
+## Database Change Sync
+
+```mermaid
+flowchart LR
+  Write["Fact or derived table write"] --> Trigger["PostgreSQL trigger"]
+  Trigger --> Outbox["earth_data_change_events"]
+  Trigger --> Notify["planet_earth_data_changes"]
+  Outbox --> Listener["earth_db_change_listener"]
+  Notify --> Listener
+  Listener --> Adapter["earth_layer_adapters"]
+  Adapter --> Cache["cache invalidation"]
+  Cache --> WS["earth_updates"]
+```
+
+1. Collection, clearing, location resolution, or projection jobs write fact or derived tables.
+2. Statement triggers write outbox rows for `INSERT` / `UPDATE` / `DELETE`.
+3. The listener wakes through notify or finds pending rows through polling.
+4. The listener maps `table + source` to Earth layers, refresh strategy, and cache patterns through `earth_layer_adapters.py`.
+5. The listener merges short-window same-layer events, invalidates cache, and broadcasts `earth_updates`.
+6. The outbox row is marked consumed only after successful broadcast; failed rows stay retryable.
+
+DB changes no longer create default `earth_refresh` jobs, so they are not blocked by long same-source collection or clearing jobs. `earth_refresh` remains for manual cache clearing and non-DB refresh hints.
+
+## Data Job Queue
+
+`collection_tasks` is the unified job ledger. Workers claim `queued` jobs with PostgreSQL `FOR UPDATE SKIP LOCKED`; write jobs for the same `source` run serially, while different sources may run in parallel.
+
+| task_type | Purpose |
+| --- | --- |
+| `collect` | Run a built-in datasource collector |
+| `clear_data` | Delete collected rows and declared derived rows for the source |
+| `clear_cache` | Delete Earth / dashboard cache for the source |
+| `earth_refresh` | Invalidate Earth layer cache and broadcast a refresh hint for non-DB changes |
+
+API handlers only create jobs and return `task_id`. Execution, progress, cancellation, and terminal state are written back by workers and pushed to the frontend through the `datasource_tasks` channel.
+
+Cancellation means “keep committed batches”: clicking stop marks the job as `cancelling` and cancels the in-memory coroutine. Already committed batches remain; unfinished batches follow the collector or cleanup rollback path.
+
+## Earth Sync Event Model
+
+The unified event model is `earth.layer.changed`:
+
+```json
+{
+  "event": "earth.layer.changed",
+  "action": "database_changed",
+  "source": "celestrak_tle",
+  "table": "collected_data",
+  "operation": "DELETE",
+  "layers": ["satellites"],
+  "refresh_strategy": "clear_then_reload",
+  "records_processed": 11125,
+  "occurred_at": "2026-05-25T10:20:30Z"
+}
+```
+
+| strategy | Purpose |
+| --- | --- |
+| `clear_then_reload` | Clear local frontend layer objects first, then force a refetch. Prefer this for deletes. |
+| `reload` | Keep old objects until fresh data returns. Use it for location, metadata, or non-destructive updates. |
+| `delta` | Used only for `earth_interactables`; upsert or remove objects by id. |
+
+APIs must return HTTP 200 with an empty collection for real zero-data states; 5xx is reserved for real endpoint failures. After a delete event, if refetch fails, the frontend should keep the cleared state and show a lightweight error instead of restoring stale objects.
+
+## Layer Adapter Contract
+
+`earth_layer_adapters.py` is the single registry for sources, derived tables, Earth layers, cache patterns, and refresh strategy. New layers should be added through an adapter entry, not through one-off button handlers, collector branches, or frontend special cases.
+
+Each adapter must declare:
+
+- Which source or table feeds which Earth layer.
+- Which Earth cache key patterns must be invalidated.
+- Which owned derived tables must be removed during `clear_data`.
+- The default refresh strategy for that layer.
+
+When a source is cleared, the `clear_data` job first deletes `collected_data.source = <source>`, then deletes adapter-owned derived rows. Direct derived-table edits also trigger the outbox, so background jobs, admin APIs, and SQL repair scripts reach Earth as long as they mutate fact or derived tables.
+
+## Operations and Troubleshooting
+
+Check whether outbox rows are piling up:
+
+```sql
+SELECT id, table_name, operation, source, occurred_at
+FROM earth_data_change_events
+WHERE consumed_at IS NULL
+ORDER BY id
+LIMIT 20;
+```
+
+Check that triggers exist:
+
+```sql
+SELECT tgname, tgrelid::regclass
+FROM pg_trigger
+WHERE tgname LIKE 'tr_planet_%_changed_%'
+ORDER BY 2, 1;
+```
+
+Useful log events:
+
+- `earth.db_changes.connected`: the listener connected to PostgreSQL and started listening.
+- `earth.db_changes.outbox_polled`: polling found unconsumed outbox rows.
+- `earth.db_changes.broadcasted`: an Earth refresh broadcast was produced.
+- `data_job.started` / `data_job.completed`: job execution state.
+
+If Earth does not update, check in order: fact table changed, outbox was consumed, adapter covers the `source/table`, the listener is online, frontend WebSocket is connected, and the visualization API returns HTTP 200 with either an empty collection or fresh data.
+
+## Kafka-ready Boundaries
+
+Business code avoids depending on a concrete queue implementation by preserving these boundaries:
+
+- `JobQueue`: submit, claim, cancel, and complete data jobs.
+- `DataChangeBus`: publish database fact changes.
+- `EarthLayerAdapterRegistry`: declare source, layer, cache, and derived-data relationships.
+
+Kafka becomes appropriate when:
+
+- Several independent services must consume the same data-change stream.
+- AIS, BGP, or sensor streams become sustained high-throughput inputs.
+- Consumer groups, replay, and service decoupling are required.
+
+Spark becomes appropriate when:
+
+- Historical data reaches tens or hundreds of millions of rows and PostgreSQL aggregation becomes expensive.
+- Cross-source, long-window, spatiotemporal analysis is needed.
+- Raw data lands in Parquet / Iceberg / Delta and the system starts producing offline derived data products.
+
+For second-level continuous stream processing, evaluate Flink first. Spark is a better fit for batch or micro-batch analytics.
+

docs/technical/en/datasource-collector-settings-connectivity.md

@@ -32,8 +32,8 @@ If endpoint, headers, base configuration, or credential fingerprint changes afte
 
 Files:
 
-- [DataSources.tsx](/home/ray/dev/linkong/planet/frontend/src/pages/DataSources/DataSources.tsx)
-- [index.css](/home/ray/dev/linkong/planet/frontend/src/index.css)
+- [PlainResourcePages.tsx](/home/ray/dev/linkong/planet/frontend/src/admin/pages/PlainResourcePages.tsx)
+- [AdminRoutes.tsx](/home/ray/dev/linkong/planet/frontend/src/admin/AdminRoutes.tsx)
 
 Current behavior:
 
@@ -53,30 +53,35 @@ Current behavior:
 
 `data-source-bulk-toolbar__running-pill` is the styling entry point for the "Collecting" pill. It is aligned with other status tags, while hover treatment, arrow affordance, and blue outline indicate interactivity.
 
-### Collectors
+### Collection Management
 
 File:
 
-- [Settings.tsx](/home/ray/dev/linkong/planet/frontend/src/pages/Settings/Settings.tsx)
+- [PlainResourcePages.tsx](/home/ray/dev/linkong/planet/frontend/src/admin/pages/PlainResourcePages.tsx)
 
 Current behavior:
 
-- The `collector_credentials` tab is displayed as "Collectors" under `/collection-management`.
-- A select lists built-in collectors and supports maintaining custom supplemental sources that merge into built-in data.
-- The only button beside the select is a plug icon for health checks.
-- Status tags below the select show:
-  - `Credentials required` / `No credentials required`
-  - Module
-  - `Enabled` / `Disabled`
-  - `Unchecked` / `Available` / `Unavailable`
-  - Whether the endpoint is overridden
-- Collectors that require credentials place the credential card above base configuration.
-- Collectors without credentials only show base configuration.
+- `/collection-management` follows the legacy hierarchy: `Collectors`, `Collection Schedule`, and `Collection History / Snapshots`.
+- `Collectors` is the configuration page. The left list shows collector configs; the right form edits endpoint, authentication, headers, collection parameters, and enabled state.
+- New collectors and target schemas use draft detail pages instead of transparent JSON modals; save persists the draft, while cancel destroys it.
+- The connection button only tests connectivity and does not save. The save button only persists the form.
+- Credential guides open in a draggable Markdown modal. When no guide exists, the modal still opens and offers a generate action. Generation shows a centered waiting state; reset confirmation must render above the guide modal.
+- `Collection History / Snapshots` groups by collector. The list must not repeat every snapshot for the same datasource; detail view switches versions through a Time Capsule / Time Machine style selector.
 - The AISStream collector uses WebSocket semantics: connecting, streaming, reconnecting, or stopped. It does not use a fixed completion percentage.
 - Custom source editing lives in collector settings. The data source catalog keeps overview, run controls, and read-only drawers.
 
 The connection button uses an inline Tabler-style plug icon with `plug-connected` semantics, avoiding the older refresh icon for a connection action.
 
+### Mapping Templates and Target Schemas
+
+Mapping remains part of collection management, but not the main collector tab:
+
+- `Mapping Templates` owns sample payload, AI propose, preview, create/update, and activate.
+- `Target Schemas` maintains writable target structures.
+- `run-mapped`, `stop-mapped`, and `stream-status` run mapped custom collectors.
+
+The new UI should keep these paths form-first. Only advanced fields should collapse into JSON. Do not flatten templates, schemas, runtime status, and collector configuration into one table.
+
 ## Backend APIs
 
 ### Data Source Configuration List
@@ -140,6 +145,26 @@ Successful responses include:
 - `credential_provider`
 - `credential_source`
 
+### Credential Guides
+
+```http
+GET  /api/v1/datasources/credential-guides/{provider}
+POST /api/v1/datasources/credential-guides/{provider}/generate
+POST /api/v1/datasources/credential-guides/{provider}/reset
+```
+
+Purpose:
+
+- Read the Markdown credential guide for a provider.
+- Generate a guide from collector metadata when none exists.
+- Reset back to the backend default guide.
+
+Frontend rules:
+
+- Render Markdown, not backend prompts or metadata.
+- Generate/reset actions belong inside the guide modal, not in the collector configuration toolbar.
+- Missing guides still open a modal so the user can generate one from there.
+
 ### BarentsWatch AIS Connectivity Validation
 
 ```http

docs/technical/en/docs-gatekeeper-development.md

@@ -2,7 +2,7 @@
 
 Docs Gatekeeper moves `/docs` from "bundle all Markdown into the frontend" to "return catalog and content from the backend according to permissions." Its goal is to keep public manuals, user docs, developer docs, and admin/ops docs in one searchable Docs page while making every protected Markdown body pass through a server-side whitelist and authorization check.
 
-For the user workflow, see the Docs section in [Planet Manual](/home/ray/dev/linkong/planet/docs/technical/en/manual.md).
+For the user workflow, see the Docs section in [Intelligent Planet Manual](/home/ray/dev/linkong/planet/docs/technical/en/manual.md).
 
 ## Authorization Model
 
@@ -61,15 +61,15 @@ DocsMetadata(
     "public",
     "Manual",
     2,
-    "Planet 使用手册",
-    "Planet Manual",
+    "智能星球使用手册",
+    "Intelligent Planet Manual",
 )
 ```
 
-When adding a public technical doc:
+When adding a technical doc that should appear in the Docs page:
 
 - Add both Chinese and English Markdown files.
-- Add filename, slug, access, group, order, and titles to server `DOCS_METADATA`.
+- Add filename, slug, access, group, order, and titles to server `DOCS_METADATA`. The backend catalog endpoint is authoritative; frontend metadata alone does not publish a document into `/docs` navigation.
 - Add matching metadata to frontend [docs-content.ts](/home/ray/dev/linkong/planet/frontend/src/pages/Docs/docs-content.ts) so navigation titles and sorting stay aligned.
 - Update `docs/technical/zh/README.md` and `docs/technical/en/README.md` when the document should be discoverable from the README.
 
@@ -83,7 +83,7 @@ The user API:
 - Validates group names on update: only `docs_user`, `docs_developer`, and `docs_admin` are accepted.
 - Allows only `super_admin` to modify Gatekeeper groups.
 
-Frontend [Users.tsx](/home/ray/dev/linkong/planet/frontend/src/pages/Users/Users.tsx) displays group tags and provides a multi-select in the edit form. Non-`super_admin` users see the field disabled, and submission removes `gatekeeper_groups` before sending.
+Frontend [Users.tsx](/home/ray/dev/linkong/planet/frontend/src/admin/pages/Users.tsx) displays group tags and provides a multi-select in the edit form. Non-`super_admin` users see the field disabled, and submission removes `gatekeeper_groups` before sending.
 
 ## Frontend Docs Loading
 

docs/technical/en/earth-bgp-context.md

@@ -188,7 +188,7 @@ Current reality:
 - that is expected, because incidents are aggregated and de-noised
 - but incident-first rendering makes the Earth view look too quiet unless there is another always-available activity layer
 
-Implementation detail for the recommended `activity layer` is expanded in the [BGP Region Aggregation Plan](/home/ray/dev/linkong/planet/docs/plans/earth-bgp-region-aggregation-plan.md).
+Implementation detail for the recommended `activity layer` is kept in the repository path `docs/plans/earth-bgp-region-aggregation-plan.md`.
 
 So the immediate next milestone is:
 
@@ -259,7 +259,7 @@ Reference inspiration:
 
 Relevant page:
 
-- `frontend/src/pages/BGP/BGP.tsx`
+- `frontend/src/admin/pages/PlainResourcePages.tsx`
 
 Current BGP console page has three levels:
 
@@ -321,7 +321,7 @@ Backend:
 
 Frontend:
 
-- `frontend/src/pages/BGP/BGP.tsx`
+- `frontend/src/admin/pages/PlainResourcePages.tsx`
 - `frontend/public/earth/js/bgp.js`
 - `frontend/public/earth/js/main.js`
 - `frontend/public/earth/js/info-card.js`

docs/technical/en/earth-frontend-context.md

@@ -4,7 +4,7 @@ This document describes the current real structure of the Earth display frontend
 
 Related references:
 
-- [Project Rules](/home/ray/dev/linkong/planet/rules.md)
+- Repository root `rules.md`
 - [Frontend Layout Guidelines](/home/ray/dev/linkong/planet/docs/technical/en/frontend-layout-guidelines.md)
 
 ## Current Goal
@@ -56,6 +56,8 @@ Responsibilities:
 - Layer module integration
 - Earth-level state synchronization
 
+Earth treats `/ws` `earth_updates` as refresh hints only; real data is fetched again from `/api/v1/visualization/...`. Backend database-driven refresh now has the listener clear cache and broadcast directly instead of going through the default `earth_refresh` job queue. After `database_changed`, the frontend applies the per-layer `clear_then_reload`, `reload`, or `delta` strategy. `clear_then_reload` must clear Three.js objects before a no-store refetch, and summary is only a consistency check, not a reason to skip a layer reload when the count is `0`. See [Data Jobs and Outbox Architecture](/home/ray/dev/linkong/planet/docs/technical/en/data-job-earth-sync-architecture.md) for the technical pipeline and [Business Architecture and Data Flows](/home/ray/dev/linkong/planet/docs/technical/en/platform-data-flows.md) for business data flows.
+
 ### 3. Earth Control Layer
 
 - [controls.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/controls.js)
@@ -151,6 +153,20 @@ Each module is responsible for its own:
 
 `brand.js` manages Earth HUD brand resources. Static assets provide the default brand; runtime overrides come from `/api/v1/earth/brand`, and uploaded images are served from `/earth-brand-assets/...`. The frontend must treat logo/title images and text fallback separately: if an image fails, show the text title; if text fields are empty, rely on backend defaults so the HUD brand area never renders blank. The console Earth Content page owns saving and resetting brand configuration; the Earth frontend only consumes it.
 
+`about.js` manages the About card inside Earth settings. Frontend defaults remain as a fallback, while runtime content is loaded from `/api/v1/earth/about`. If the request fails or fields are missing, the renderer must fall back per field so the settings page never renders an empty card. Admin exposes an Earth Content `About` tab; saving uses `PUT /api/v1/earth/about`, and restoring defaults uses `DELETE /api/v1/earth/about`.
+
+`oobe.js` manages the first-run Earth initialization guide. OOBE visibility must be driven by `/api/v1/earth/oobe-status` and its `ready` field, not by `localStorage`. `localStorage` may only store a short-lived "skip on this browser" flag; if the backend reports `ready: true`, logout, cleared browser storage, or a different browser must not show OOBE again. Desktop uses a dark starfield scrim and glass startup panel, while mobile uses a bottom sheet and respects `prefers-reduced-motion`.
+
+The Admin Earth Content page must preserve runtime semantics:
+
+- `Brand`: brand preview should use the same dark starfield background, size, spacing, logo/title rendering, and text fallback as the Earth HUD top-left brand block, not a generic form preview.
+- `About`: configures the About card in Earth settings, including logo, kicker, title, version, description, and metadata items. Earth runtime reads `/earth/about` and falls back to defaults on failure.
+- `Boundary Precision`: build boundary, refresh status, and restore defaults belong inside this section, not in the global page toolbar.
+- `TV`: the list distinguishes built-in, collected, and custom sources. Card state represents enabled, disabled, draft, or error. Built-in sources cannot be deleted; collected and custom sources can. A new live source only enters draft state after the plus button is clicked; save persists it into the list, while cancel destroys the draft.
+- `Basemap`, `Layer Resources`, `3D Models`, and `News Anchor Strategy`: if backend capability is not available yet, the console should show an explicit pending state instead of mixing those items into TV or brand configuration.
+
+TV preview should reuse the Earth runtime live-card structure and state labels as closely as possible so built-in markers, live loading state, stream source, region, and language match what users see on Earth.
+
 The compute-center layer row has a notification badge for GeoJSON `unresolved` records. The badge means "no trustworthy coordinates, cannot render on the globe"; it is different from the `?` marker drawn on already positioned but unconfirmed compute centers. Clicking the badge opens a fixed info card beside the layer panel. Row-level `采集` fetches candidates only. Header-level `一键采用` processes the queue top-to-bottom, saves the highest-confidence valid candidate, removes successful rows, renumbers the list, and dispatches `earth:compute-center-unresolved-count-change` so the badge updates immediately. When the batch ends, `earth:compute-center-location-saved` refreshes the real layer.
 
 Location candidate state in the details card is cached in [info-card.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/info-card.js) by `entityType:entityId`. If the user closes the details card or unresolved queue and reopens the same compute center / BGP collector, previously collected candidates and status text are restored. Header-level `一键采用` prefers cached candidates, avoiding repeated online geocoding or LLM factcheck calls. After a location is saved, that entity's candidate list is cleared to a "refreshing layer" status so stale candidates do not keep misleading the user.

docs/technical/en/earth-satellite-footprint-policy.md

@@ -17,26 +17,21 @@ Related context:
 
 ## Current Local Categories
 
-Current CelesTrak satellite groups in [backend/app/services/collectors/celestrak.py](/home/ray/dev/linkong/planet/backend/app/services/collectors/celestrak.py) include:
+The CelesTrak collector now prefers the complete active satellite catalog from `GROUP=active&FORMAT=json`. This prevents one failed CelesTrak group request from being saved as a successful but incomplete batch. The collector only proceeds when the downloaded JSON is a parseable array and records include `NORAD_CAT_ID`; network, resume, or parsing failures are retried, and final failure preserves the previous current dataset.
 
-- `starlink`
-- `gps-ops`
-- `galileo`
-- `glonass`
-- `beidou`
-- `leo`
-- `geo`
-- `iridium-next`
+CelesTrak applies a repeat-download window to large groups such as `active`. To make database resets recoverable, Planet stores raw downloads under `$PLANET_CACHE_DIR/downloads`. When `active` returns the CelesTrak "GP data has not updated" HTTP 403:
 
-Non-Starlink categories:
+- If an `active` cache exists, the collector reuses it to repopulate the database.
+- If no `active` cache exists, the collector switches to fallback group mode and downloads `starlink`, `gps-ops`, `galileo`, `glonass`, `beidou`, `leo`, `geo`, and `iridium-next`.
+- Fallback group mode requires every group to succeed or have a reusable cache; if any group is missing, the whole collection fails and no partial dataset is saved.
 
-- `gps-ops`
-- `galileo`
-- `glonass`
-- `beidou`
-- `leo`
-- `geo`
-- `iridium-next`
+The collector still provides `metadata.constellation_group` to the frontend, but the value now comes from executable inference:
+
+- `OBJECT_NAME` starting with `STARLINK` is marked as `starlink`
+- `OBJECT_NAME` starting with `IRIDIUM` is marked as `iridium-next`
+- Other active satellites are not forced into the old CelesTrak small-group labels, because a broad source group is not an exact constellation
+
+The product policy therefore still discusses GNSS/RNSS, GEO, generic LEO, and Iridium NEXT semantics. However, only fallback group mode stores the old `gps-ops`, `galileo`, `glonass`, `beidou`, `leo`, or `geo` labels; the active primary path does not force every satellite into those labels.
 
 ## Research Conclusions
 
@@ -136,7 +131,8 @@ This implementation only does the minimum executable version and does not change
 
 1. Backend passes constellation group and footprint policy hint to the frontend
 
-- CelesTrak collector stores `GROUP` in `metadata.constellation_group`
+- CelesTrak collector stores the source query in `metadata.celestrak_query_group = active`
+- `metadata.constellation_group` only stores inferred business constellations such as `starlink` and `iridium-next`
 - Visualization API outputs:
   - `properties.constellation_group`
   - `properties.footprint_policy`

docs/technical/en/frontend-admin-frontend-context.md

@@ -4,7 +4,7 @@ This document describes the current real structure of the console frontend. The
 
 Related references:
 
-- [Project Rules](/home/ray/dev/linkong/planet/rules.md)
+- Repository root `rules.md`
 - [Frontend Layout Guidelines](/home/ray/dev/linkong/planet/docs/technical/en/frontend-layout-guidelines.md)
 
 ## Current Goal
@@ -22,7 +22,7 @@ Main entry point:
 
 - [App.tsx](/home/ray/dev/linkong/planet/frontend/src/App.tsx)
 
-Current admin-related routes:
+Admin now owns the official admin routes:
 
 - `/admin`
 - `/users`
@@ -37,36 +37,60 @@ Current admin-related routes:
 - `/collection-management`
 - `/settings`
 
+These routes render [AdminRoutes.tsx](/home/ray/dev/linkong/planet/frontend/src/admin/AdminRoutes.tsx). Page metadata and menu entries come from [manifest.tsx](/home/ray/dev/linkong/planet/frontend/src/admin/routes/manifest.tsx). Admin is the only console entry point; there is no parallel console or rollback route.
+
 `/earth` is a standalone display page and is not part of the console shell.
 
 ## Current Page Shell
 
-The console shared shell is at:
+The official admin shell is at:
 
-- [AppLayout.tsx](/home/ray/dev/linkong/planet/frontend/src/components/AppLayout/AppLayout.tsx)
+- [AdminLayout.tsx](/home/ray/dev/linkong/planet/frontend/src/admin/components/layout/AdminLayout.tsx)
 
 Responsibilities:
 
-- Left-side navigation
-- Collapse and expand
-- Current account / version information
-- Content area height closure
-- Site-wide unified sidebar scrollbar
+- Left navigation, grouped collapse, and mobile drawer
+- Current account, version, logout, and theme switching
+- Top search, breadcrumbs, and page shortcuts
+- Single-screen content-area height closure
+- Coordination for Admin internal scrolling, tables, detail panels, and mobile detail views
 
-Current structure:
+Future official admin pages should adapt to `AdminLayout` and Admin page patterns; do not reintroduce a parallel admin shell.
 
-```tsx
-<Layout className="dashboard-layout">
-  <Sider className="dashboard-sider">...</Sider>
-  <Layout>
-    <Content className="dashboard-content">
-      <div className="dashboard-content-inner">{children}</div>
-    </Content>
-  </Layout>
-</Layout>
-```
+## Admin Section Loading
 
-Future console pages should adapt to this shell rather than redefining full-page height semantics.
+Multi-tab pages are currently coordinated by [PlainResourcePages.tsx](/home/ray/dev/linkong/planet/frontend/src/admin/pages/PlainResourcePages.tsx), which hosts the current management and information workbench patterns. Section loading follows these rules:
+
+- Initial page load requests only the active tab; it does not prefetch every tab endpoint.
+- Switching tabs lazily loads that tab. Loaded tabs stay cached in local `states`, so returning to a tab reuses the previous data.
+- Explicit actions such as refresh, save, test, upload, and credential-guide generation refresh only the current section instead of fanning out to unrelated sections.
+- Datasource directory filter changes clear the built-in-source section cache and reload the current section with the new filters.
+- Top summary metrics count loaded sections only, so unopened tabs are not reported as failed interfaces.
+
+This keeps Earth, AI, collection management, and other multi-section pages from flooding backend APIs on cold start while preserving a fast cached tab-switching experience. Full health checks should use backend health endpoints or explicit refresh flows rather than relying on page initialization to touch every business endpoint.
+
+## Datasource Collection Queue
+
+The Admin datasource page routes single-source trigger, table-selected trigger, and trigger-all into a browser-download-list style collection queue:
+
+- Queue state is managed by [PlainResourcePages.tsx](/home/ray/dev/linkong/planet/frontend/src/admin/pages/PlainResourcePages.tsx). It is a current-session visibility layer and does not fake task history in `localStorage`.
+- Progress first consumes the `/ws` `datasource_tasks` channel. If the socket is unavailable or stale, the page polls `/api/v1/datasources/{id}/task-status`.
+- Trigger responses immediately insert `triggered`, `skipped`, and `failed` items. After a refresh, the queue restores only real backend rows that are still `running`, `pending`, or `queued`.
+- The `Built-in Sources` section uses the table selection column for selected-source triggering. With no rows selected, the primary button is `Trigger All`; after selection, the same button becomes `Trigger Selected N`, replacing the old manual-ID batch button.
+- The expanded queue no longer lives in the page content flow, so trigger-all cannot squeeze the table and detail panel. The top-right actions area uses the existing `Button` styling; the empty state shows a `ListChecks` icon, and active queues show only a pure circular total-progress indicator. Clicking it opens a floating panel grouped by running, failed, completed, and skipped.
+- Queue rows can jump to the datasource detail panel, and failed rows can retry. The detail panel's task summary only reports the selected source's latest task; it does not save configuration.
+- While a single-source collection is running, the action button becomes `Stop Collection`; unfinished queue rows expose a cancel action on the right. Cancellation calls `/api/v1/datasources/{source_id}/tasks/{task_id}/cancel`; the backend keeps committed batches and rolls back unfinished work.
+- Clearing database data and clearing display cache enter the same queue. The frontend shows task state and does not assume the API completed synchronously.
+
+This queue is a user-perception layer. Backend task status remains the only source of truth for running, completion, failure, and skipped decisions.
+
+## Admin Theme Slider
+
+The Admin sidebar theme switcher still reuses shared [SegmentedControl.tsx](/home/ray/dev/linkong/planet/frontend/src/components/SegmentedControl/SegmentedControl.tsx), while [styles.css](/home/ray/dev/linkong/planet/frontend/src/admin/styles.css) overrides the segment variables by `data-theme`:
+
+- Light mode uses `--d-segment-bg: #eef3f9`, a white slider, and a light external shadow.
+- Dark mode uses the same dark base, `#202938` slider, and dark external shadow semantics as Docs.
+- Product code only hides the text label and keeps icon + tooltip behavior; it should not recreate the private slider DOM.
 
 ## Current Shared Components
 
@@ -74,7 +98,8 @@ Future console pages should adapt to this shell rather than redefining full-page
 
 File:
 
-- [Scrollbar.tsx](/home/ray/dev/linkong/planet/frontend/src/components/Scrollbar/Scrollbar.tsx)
+- [Scrollbar.tsx](/home/ray/dev/linkong/planet/frontend/src/components/tactile-ui/Scrollbar.tsx)
+- [Tactile UI Components](/home/ray/dev/linkong/planet/docs/technical/en/tactile-ui-components.md)
 
 Purpose:
 
@@ -91,7 +116,7 @@ Current constraint:
 
 File:
 
-- [ScrollbarOverlay.tsx](/home/ray/dev/linkong/planet/frontend/src/components/Scrollbar/ScrollbarOverlay.tsx)
+- [ScrollbarOverlay.tsx](/home/ray/dev/linkong/planet/frontend/src/components/tactile-ui/ScrollbarOverlay.tsx)
 
 Purpose:
 
@@ -100,25 +125,42 @@ Purpose:
 
 Current usage:
 
-- Data sources
-- Collected data
-- User management
-- Settings page
-- Alerts page
-- BGP page
+- Admin data sources, collected data, collection management, logs, alerts, and BGP pages
+- Old AntD legacy pages continue using shared scrolling behavior through compatibility wrappers
 
 ### 3. `TableScrollRegion`
 
 File:
 
-- [TableScrollRegion.tsx](/home/ray/dev/linkong/planet/frontend/src/components/Scrollbar/TableScrollRegion.tsx)
+- [TableScrollRegion.tsx](/home/ray/dev/linkong/planet/frontend/src/components/tactile-ui/TableScrollRegion.tsx)
 
 Purpose:
 
 - Provides a unified wrapper for table scroll areas
 - New table pages should reuse this rather than repeating the "table area + overlay scrollbar" boilerplate
 
-### 4. `SegmentedControl`
+### 4. `TactileButton` / `TactileSwitch` / `ControlGroup`
+
+Files:
+
+- [Button.tsx](/home/ray/dev/linkong/planet/frontend/src/components/tactile-ui/Button.tsx)
+- [Switch.tsx](/home/ray/dev/linkong/planet/frontend/src/components/tactile-ui/Switch.tsx)
+- [ControlGroup.tsx](/home/ray/dev/linkong/planet/frontend/src/components/tactile-ui/ControlGroup.tsx)
+
+Purpose:
+
+- Admin global tool buttons and detail-panel toolbars
+- Icon-only ordinary actions with tooltips
+- Strong-intent actions such as save, create, confirm, delete, and stop
+- Compact switches aligned with the Docs theme slider
+
+Current constraints:
+
+- Neutral buttons default to a white tactile surface with external shadow
+- Unambiguous actions prefer icon + tooltip; strong-intent actions like save may keep text
+- Feature pages should customize through props and CSS variables, not by rewriting core button CSS
+
+### 5. `SegmentedControl`
 
 Files:
 
@@ -146,7 +188,7 @@ Current constraints:
 - Prefer CSS variable overrides for colors instead of hard-coding theme colors in feature components
 - Best for a small set of mutually exclusive choices; do not use it as a long list, navigation menu, or select replacement
 
-### 5. `MarkdownRenderer`
+### 6. `MarkdownRenderer`
 
 File:
 
@@ -165,39 +207,19 @@ Current constraints:
 - Internal document links should be converted to `/docs/:slug` through `transformLink`
 - Heading anchors are injected through `getHeadingId`, keeping route state outside the renderer
 
-### 6. `ConnectionTestInput`
+### 7. Admin UI Primitives
 
-File:
+Files:
 
-- [ConnectionTestInput.tsx](/home/ray/dev/linkong/planet/frontend/src/components/ConnectionTestInput/ConnectionTestInput.tsx)
+- [button.tsx](/home/ray/dev/linkong/planet/frontend/src/admin/components/ui/button.tsx)
+- [dialog.tsx](/home/ray/dev/linkong/planet/frontend/src/admin/components/ui/dialog.tsx)
+- [switch.tsx](/home/ray/dev/linkong/planet/frontend/src/admin/components/ui/switch.tsx)
 
-Purpose:
+Use:
 
-- Console form fields that combine an endpoint/Base URL value with a connection check
-- Connection-test entry points for AI Provider and WebSearch
-- Future collector configuration fields should reuse it when the test action belongs inside the input
-
-Current constraints:
-
-- The input suffix shows a single plug/connector icon, not an adjacent text button
-- Disabled integrations must grey out both the input and its connection-test action
-- The component only combines the input and action; callers still own form state, loading, disabled state, and the request itself
-
-### 7. `TableActions`
-
-File:
-
-- [TableActions.tsx](/home/ray/dev/linkong/planet/frontend/src/components/TableActions/TableActions.tsx)
-
-Purpose:
-
-- Shared action entry for table operation columns
-- Shows inline actions when expanded
-- Uses a more-actions dropdown when collapsed
-
-Companion export:
-
-- `actionCellProps`: for action-column `onCell`, preventing action buttons from being ellipsized or wrapped
+- Global tool buttons, detail actions, confirmation dialogs, and binary settings.
+- Aligned with Tactile UI tokens so Admin controls keep consistent size, hover, disabled, and dark-mode behavior.
+- Row actions should prefer icon buttons plus tooltip/title; do not reintroduce a separate action-menu component.
 
 ## Current State Sources
 
@@ -220,7 +242,7 @@ Responsibilities:
 
 File:
 
-- [AISettings.tsx](/home/ray/dev/linkong/planet/frontend/src/pages/AISettings/AISettings.tsx)
+- [AISettings.tsx](/home/ray/dev/linkong/planet/frontend/src/admin/pages/PlainResourcePages.tsx)
 
 Responsibilities:
 
@@ -256,7 +278,7 @@ Constraints:
 
 Example:
 
-- [Dashboard.tsx](/home/ray/dev/linkong/planet/frontend/src/pages/Dashboard/Dashboard.tsx)
+- [Dashboard.tsx](/home/ray/dev/linkong/planet/frontend/src/admin/pages/Dashboard.tsx)
 
 Priority goals:
 
@@ -268,10 +290,10 @@ Priority goals:
 
 Examples:
 
-- [DataSources.tsx](/home/ray/dev/linkong/planet/frontend/src/pages/DataSources/DataSources.tsx)
-- [DataList.tsx](/home/ray/dev/linkong/planet/frontend/src/pages/DataList/DataList.tsx)
-- [Users.tsx](/home/ray/dev/linkong/planet/frontend/src/pages/Users/Users.tsx)
-- [Settings.tsx](/home/ray/dev/linkong/planet/frontend/src/pages/Settings/Settings.tsx)
+- [DataSources.tsx](/home/ray/dev/linkong/planet/frontend/src/admin/pages/PlainResourcePages.tsx)
+- [DataList.tsx](/home/ray/dev/linkong/planet/frontend/src/admin/pages/DataList.tsx)
+- [Users.tsx](/home/ray/dev/linkong/planet/frontend/src/admin/pages/Users.tsx)
+- [Settings.tsx](/home/ray/dev/linkong/planet/frontend/src/admin/pages/PlainResourcePages.tsx)
 
 Constraints:
 
@@ -281,7 +303,7 @@ Constraints:
 
 ### Datasource Directory Page
 
-[DataSources.tsx](/home/ray/dev/linkong/planet/frontend/src/pages/DataSources/DataSources.tsx) is the datasource directory and collection operation page. It should not grow back into a configuration editor.
+[DataSources.tsx](/home/ray/dev/linkong/planet/frontend/src/admin/pages/PlainResourcePages.tsx) is the datasource directory and collection operation page. It should not grow back into a configuration editor.
 
 Current page boundary:
 
@@ -295,7 +317,9 @@ Keep this boundary: do not put custom datasource editing, built-in endpoint over
 
 ### Collectors Page
 
-[Settings.tsx](/home/ray/dev/linkong/planet/frontend/src/pages/Settings/Settings.tsx) has three route modes: `/settings` for System Settings, `/earth-content` for Earth Content, and `/collection-management` for Collection Management. The `collector_credentials` tab is shown as `Collectors` under `/collection-management`.
+[Settings.tsx](/home/ray/dev/linkong/planet/frontend/src/admin/pages/PlainResourcePages.tsx) has three route modes: `/settings` for System Settings, `/earth-content` for Earth Content, and `/collection-management` for Collection Management. The `collector_credentials` tab is shown as `Collectors` under `/collection-management`.
+
+The `System Display` section under `/settings` includes the `Demo Mode` switch. When enabled, Earth OOBE ignores existing current collected data and the local `browse first` temporary skip state, then opens the initialization guide directly. This switch is only for demos and acceptance checks; it does not change datasources, collection queues, or Earth content resources.
 
 Current boundary:
 
@@ -307,7 +331,7 @@ Current boundary:
 
 ### Earth Content Page
 
-`/earth-content` reuses the same single-screen tab container from [Settings.tsx](/home/ray/dev/linkong/planet/frontend/src/pages/Settings/Settings.tsx), but its ownership is separate from System Settings:
+`/earth-content` reuses the same single-screen tab container from [Settings.tsx](/home/ray/dev/linkong/planet/frontend/src/admin/pages/PlainResourcePages.tsx), but its ownership is separate from System Settings:
 
 - `TV Livestream` owns the Earth media-panel source configuration.
 - `Boundary Precision` owns the Earth static boundary asset state: provider, low-precision fallback, high-precision manifest/PMTiles, source JSON, and build action.
@@ -319,8 +343,8 @@ Do not add Earth experience resources or collection-lifecycle tabs back into `/s
 
 Examples:
 
-- [BGP.tsx](/home/ray/dev/linkong/planet/frontend/src/pages/BGP/BGP.tsx)
-- [Playground.tsx](/home/ray/dev/linkong/planet/frontend/src/pages/Playground/Playground.tsx)
+- [BGP.tsx](/home/ray/dev/linkong/planet/frontend/src/admin/pages/PlainResourcePages.tsx)
+- [Playground.tsx](/home/ray/dev/linkong/planet/frontend/src/admin/pages/PlainResourcePages.tsx)
 
 Constraints:
 

docs/technical/en/frontend-layout-guidelines.md

@@ -8,7 +8,8 @@ Admin pages in this project default to a "single-screen workspace" layout standa
 
 Current recommended reference implementations:
 
-- [frontend/src/pages/BGP/BGP.tsx](/home/ray/dev/linkong/planet/frontend/src/pages/BGP/BGP.tsx)
+- [frontend/src/admin/pages/PlainResourcePages.tsx](/home/ray/dev/linkong/planet/frontend/src/admin/pages/PlainResourcePages.tsx)
+- [frontend/src/admin/pages/DataList.tsx](/home/ray/dev/linkong/planet/frontend/src/admin/pages/DataList.tsx)
 - [frontend/src/index.css](/home/ray/dev/linkong/planet/frontend/src/index.css)
 
 ## Core Principles
@@ -23,15 +24,15 @@ Admin pages default to:
 Recommended structure:
 
 ```tsx
-<AppLayout>
+<AdminLayout>
   <div className="page-shell">
     <div className="page-shell__header">...</div>
     <div className="page-shell__body">...</div>
   </div>
-</AppLayout>
+</AdminLayout>
 ```
 
-Total page height should be bounded within the `AppLayout` content area, not allowed to grow naturally downward without limit.
+Total page height should be bounded within the `AdminLayout` content area, not allowed to grow naturally downward without limit.
 
 ### 2. Scrolling Should Happen Inside Modules
 

docs/technical/en/location-pipeline-development.md

@@ -2,7 +2,7 @@
 
 `backend/app/services/location/` is the shared abstraction for any "given a record, decide its lat/lon" workflow. Compute centers, BGP collectors, and BGP events now run on this pipeline. Future entities such as satellite ground stations, user-claimed points, and IXP facilities should plug in here instead of creating another geocoding path.
 
-For the user workflow, see [Earth Location Candidate Collection User Guide](/home/ray/dev/linkong/planet/docs/technical/en/location-pipeline-user.md).
+For the user workflow, see the Earth coordinate-candidate section in [Intelligent Planet Manual](/home/ray/dev/linkong/planet/docs/technical/en/manual.md).
 
 ## Design Goals
 

docs/technical/en/manual.md

@@ -1,8 +1,8 @@
-# Planet Manual
+# Intelligent Planet Manual
 
-This manual is for Planet end users. Starting from the browser, it covers account registration, login, configuring collectors, configuring AI, using Earth and the console, and reading the docs site. Every action happens in a browser.
+This manual is for Intelligent Planet end users. Starting from the browser, it covers account registration, login, configuring collectors, configuring AI, using Intelligent Planet and the console, and reading the docs site. Every action happens in a browser.
 
-If you are responsible for deployment or on-call duty, read the [Planet Ops Runbook](/home/ray/dev/linkong/planet/docs/technical/en/ops-runbook.md) instead — it covers shell commands, log paths, and CLI fallbacks for user creation.
+If you are responsible for deployment or on-call duty, read the [Intelligent Planet Ops Runbook](/home/ray/dev/linkong/planet/docs/technical/en/ops-runbook.md) instead — it covers shell commands, log paths, and CLI fallbacks for user creation.
 
 ## Entry Overview
 
@@ -188,19 +188,22 @@ TV livestreams and boundary precision moved to `/earth-content`; collectors and
 `/earth-content` is under the console's Operations and Configuration group and owns resources used by the Earth frontend:
 
 - **Brand Assets**: manages the logo, title image, title text, subtitle, and description used by the Earth HUD. Uploaded images are saved as Earth brand assets and read by the Earth page immediately.
+- **About**: manages the About card shown in Earth settings, including logo, kicker, title, version, description, and metadata.
 - **TV Livestream**: manages sources shown in the Earth media panel.
 - **Boundary Precision**: shows the current provider, low-precision fallback, high-precision PMTiles/manifest status, local source JSON, and manual build action.
 - **Base Map**, **Layer Resources**, **3D Assets**, and **News Anchor Strategy**: placeholder tabs for future configuration. They do not display fake data.
 
 The Earth page settings gear also includes Boundary Precision. Switching to High Precision starts a local background download/build, like a game update package, when no high-precision asset exists yet. Progress is shown as a percentage, and the result applies automatically after success without a page reload. Switching back to Low Precision only changes the local display preference.
 
+If the backend decides Earth is not initialized yet, the first visit to `/earth` shows a glassy startup guide that points the user to sign in and collect data. This is based on backend data state; once the system has collected data, clearing browser storage does not make the guide reappear.
+
 ### Collection Management
 
 `/collection-management` is also under Operations and Configuration and owns the collection lifecycle:
 
 - **Collectors**: endpoint, headers, credentials, timeout, retry, and connection checks.
 - **Collection Scheduling**: the existing scheduling configuration.
-- **Collection History / Snapshots**: a placeholder for future collection task, snapshot, and collected-data browsing.
+- **Collection History / Snapshots**: groups snapshots by datasource and lets the detail panel switch versions through a Time Capsule selector.
 
 ### SMTP Email Settings
 
@@ -230,7 +233,7 @@ To let a regular user read developer or operations docs, add `docs_developer` or
 
 ## Data Exploration
 
-- `/datasources`: source directory. The `Collection Tasks` tab is for one-shot, scheduled, and finite collectors; it can be filtered by product domain, layer/module, enabled state, last run status, whether collected records exist, and search text. Selecting rows triggers only those sources; with no selected rows, `Collect current filter` triggers the filtered scope. The `Realtime Streams` tab is for AISStream / WebSocket long connections and shows connection health, stored totals, time-window counters, and Start / Stop / Reconnect actions. Clicking a name opens an info drawer showing endpoint, headers, base config, and built-in flag; endpoint/credentials editing happens at `/collection-management -> Collectors`. The `Collecting N` tag under the overall progress can be clicked to expand the current collection task list
+- `/datasources`: source directory. The `Built-in Sources` tab can be filtered by product domain, layer/module, enabled state, last run status, whether collected records exist, and search text. With no rows selected, the primary button shows `Trigger All`; after selecting rows, it becomes `Trigger Selected N` and submits only those sources. The top-right queue button shows a queue icon when empty and a pure circular total-progress indicator while tasks exist; it opens a floating panel grouped by running, completed, failed, and skipped. Failed rows can retry, and completed rows can jump to detail. `Realtime Sources` is for AISStream / WebSocket long connections and shows connection health, stored totals, time-window counters, and Start / Stop / Reconnect actions. Endpoint/credential/header editing happens at `/collection-management -> Collectors`.
 - `/data`: collected data table — used to verify "did data arrive", "is the freshness right", "does a source emit valid records"
 - `/bgp`: BGP detail page with list + detail + analysis; complements the BGP layer on Earth
 - `/alerts/system`, `/alerts/bgp`, `/alerts/situational`: system, BGP, and situational alerts
@@ -277,7 +280,17 @@ Search finds cables, landing points, satellites, compute centers, BGP events, BG
 
 Compute center and BGP observer detail cards support automatic coordinate-candidate collection. Click the object then use "Collect Coordinate Candidates" or "Re-collect Coordinates". The backend assembles candidates from source coordinates, public-org registry APIs, and online geocoders. When regular sources have no candidate, the current default AI Provider runs one LLM factcheck fallback. BGP observers' stored coordinates only fill query context; they are not returned as candidates.
 
-Candidates preview on Earth directly. Saving a compute-center candidate writes to the `compute_center_locations` dimension table and refreshes the layer immediately. The notification badge at the top-left of the compute-center layer shows the unresolved count; clicking it opens the queue, supports single collection, or "Adopt All" to save the top-confidence candidates from top to bottom. Records without candidates stay in the queue rather than being faked to country centroids. See [Earth Location Candidate Collection User Guide](/home/ray/dev/linkong/planet/docs/technical/en/location-pipeline-user.md).
+Candidates preview on Earth directly. Saving a compute-center candidate writes to the `compute_center_locations` dimension table and refreshes the layer immediately. The notification badge at the top-left of the compute-center layer shows the unresolved count; clicking it opens the queue, supports single collection, or "Adopt All" to save the top-confidence candidates from top to bottom. Records without candidates stay in the queue rather than being faked to country centroids.
+
+Recommended single-object flow:
+
+1. Open a compute center or BGP observer detail card.
+2. Click "Collect Coordinate Candidates".
+3. Wait for candidates; entries that depend on WebSearch / AI factcheck show collection state.
+4. Preview candidate positions on Earth.
+5. Save the candidate when it is credible; closing the card does not lose the current task state.
+
+Adopt All is for batch processing the compute-center unresolved queue. It starts from the top and adopts the highest-confidence candidate. Records without factual support remain in the queue. When WebSearch is disabled, single locate and Adopt All are disabled because location validation depends on factual lookup.
 
 ### Settings
 
@@ -348,7 +361,7 @@ Mobile uses a drawer layout: layer control moves into a drawer; search/settings/
 
 Docs at `http://localhost:3000/docs` are served by the backend with access control, not bundled into the frontend build.
 
-Anonymous visitors see only `public` docs: README, Quickstart, Manual, FAQ, Earth Location Candidate Collection User Guide. Authenticated users with Gatekeeper groups see more:
+Anonymous visitors see only `public` docs: README, Quickstart, Manual, and FAQ. Authenticated users with Gatekeeper groups see more:
 
 - `docs_user`: end-user operational docs
 - `docs_developer`: Earth, frontend, backend, collectors, AI Provider development docs
@@ -362,5 +375,4 @@ Docs supports: category navigation, Markdown rendering, tables and code blocks,
 
 - [Quickstart](/home/ray/dev/linkong/planet/docs/technical/en/quickstart.md)
 - [FAQ](/home/ray/dev/linkong/planet/docs/technical/en/faq.md)
-- [Earth Location Candidate Collection User Guide](/home/ray/dev/linkong/planet/docs/technical/en/location-pipeline-user.md)
-- [Planet Ops Runbook](/home/ray/dev/linkong/planet/docs/technical/en/ops-runbook.md)
+- [Intelligent Planet Ops Runbook](/home/ray/dev/linkong/planet/docs/technical/en/ops-runbook.md)

docs/technical/en/naming-glossary.md

@@ -0,0 +1,99 @@
+# Naming Glossary
+
+This document standardizes terms used across Intelligent Planet, the console, backend APIs, and documentation. When adding Chinese UI labels, chart labels, or translated documentation, prefer the Chinese display names listed here to avoid unnecessary mixed Chinese/English copy.
+
+## Usage Rules
+
+- Chinese UI should use the Chinese display name.
+- API fields, database fields, code identifiers, and external brand names keep their original English or snake_case form.
+- On first mention in long-form docs, use “Chinese Display Name (English Term)” when clarification helps; subsequent mentions can use the Chinese display name.
+- Industry abbreviations such as BGP, AI, API, URL, ID, TLE, ASN, GPU, and AIS may remain English.
+- External service brands keep their official English names, such as PeeringDB, Space-Track, BarentsWatch, AISStream, and GitHub.
+
+## Core Product Terms
+
+| English / Key | Chinese Display Name | Usage |
+| --- | --- | --- |
+| Planet | 智能星球 | Product name and primary experience entry |
+| Admin | 控制台 | Admin console context |
+| Earth | 智能星球 | Visualization product name and `/earth` entry |
+| datasource | 数据源 | APIs, lists, filters |
+| collector | 采集器 | Collection jobs and credential configuration |
+| collected data | 采集数据 | Data list and statistics |
+| snapshot | 快照 | Collection history |
+| mapping | 映射 | Field mapping |
+| target schema | 目标 Schema | Mapping target structure |
+| credential | 凭证 | Secrets, account passwords, OAuth material |
+| secret reveal | 密钥 reveal | Explicit administrator secret reveal action |
+| AI Provider | AI Provider | Service name |
+| tool | 工具 | Web Search, OCR, and similar integrations |
+| Playground | Playground | Interactive debugging entry |
+
+## Data Types
+
+| data_type | Chinese Display Name | Meaning |
+| --- | --- | --- |
+| `supercomputer` | 超算 | TOP500 supercomputer record |
+| `gpu_cluster` | GPU 集群 | Epoch AI GPU cluster |
+| `model` | 模型 | Hugging Face model |
+| `dataset` | 数据集 | Hugging Face dataset |
+| `space` | 空间 | Hugging Face Space |
+| `ixp` | 互联网交换点 | PeeringDB IXP |
+| `network` | 网络 | PeeringDB network |
+| `facility` | 设施 | PeeringDB facility |
+| `submarine_cable` | 海底光缆 | Submarine cable route |
+| `landing_point` | 登陆点 | Cable landing point |
+| `cable_system` | 海缆系统 | TeleGeography cable system |
+| `cable_landing_relation` | 海缆登陆关系 | Cable-to-landing-point relation |
+| `satellite_tle` | 卫星轨道根数 | CelesTrak / Space-Track TLE |
+| `bgp_rib` | BGP 路由表 | BGP RIB snapshot |
+| `bgp_update` | BGP 更新 | BGP update event |
+| `prefix_geography` | 前缀地理位置 | IP prefix geography |
+| `vessel_ais` | AIS 船舶 | AIS vessel observation |
+| `vessel` | 船舶 | Aggregated vessel object |
+| `news_live_stream` | 新闻直播源 | News live stream source |
+| `news_item` | 新闻条目 | Archived news item |
+| `device_stats` | 设备统计 | Cloudflare Radar device statistics |
+| `traffic_stats` | 流量统计 | Cloudflare Radar traffic statistics |
+| `as_stats` | 自治系统统计 | Cloudflare Radar AS statistics |
+| `compute_center` | 算力中心 | Aggregated Earth compute-center object |
+| `generic` | 通用数据 | Generic collector output |
+| `generic_records` | 通用记录 | Generic mapping output |
+
+## Datasources And External Services
+
+| English / Key | Chinese Display Name | Notes |
+| --- | --- | --- |
+| TOP500 | TOP500 超算榜单 | Keep TOP500 as the external list name |
+| Epoch AI | Epoch AI GPU 集群 | Keep Epoch AI as the service name |
+| Hugging Face | Hugging Face | Official brand name |
+| PeeringDB IXP | PeeringDB 交换中心 | IXP can also be explained as 互联网交换点 |
+| PeeringDB Networks | PeeringDB 网络 | Network entity |
+| PeeringDB Facilities | PeeringDB 设施 | Facility entity |
+| Submarine Cables | 海底光缆 | Prefer Chinese in charts and lists |
+| Cable Landing Points | 海缆登陆点 | Use 登陆点 consistently |
+| CelesTrak TLE | CelesTrak 轨道根数 | TLE may remain abbreviated |
+| Space-Track TLE | Space-Track 轨道根数 | Keep Space-Track as brand name |
+| RIPE RIS Live BGP | RIPE RIS Live BGP | Keep service name and BGP abbreviation |
+| IPtoASN Prefix Geography | IPtoASN 前缀地理位置 | Keep IPtoASN as dataset name |
+| OpenGeoFeed Prefix Geography | OpenGeoFeed 前缀地理位置 | Keep OpenGeoFeed as brand name |
+| NRO Delegated Prefix Geography | NRO Delegated 前缀地理位置 | Keep NRO Delegated as dataset name |
+| BarentsWatch AIS Vessels | BarentsWatch AIS 船舶 | Keep BarentsWatch / AIS |
+| AISStream Vessels | AISStream 船舶 | Keep AISStream as brand name |
+
+## UI Actions And Statuses
+
+| English / Key | Chinese Display Name | Usage |
+| --- | --- | --- |
+| success | 成功 | Status badge |
+| failed | 失败 | Status badge |
+| running | 运行中 | Task status |
+| completed | 已完成 | Task status |
+| cancelled | 已取消 | Task status |
+| enabled | 已启用 | Toggle state |
+| disabled | 已停用 | Toggle state |
+| configured | 已配置 | Credential / integration state |
+| unconfigured | 未配置 | Credential / integration state |
+| refresh | 刷新 | Button |
+| copy | 复制 | Button |
+| reveal | reveal | Keep English for audited secret reveal semantics |

docs/technical/en/ops-runbook.md

@@ -1,6 +1,6 @@
-# Planet Ops Runbook
+# Intelligent Planet Ops Runbook
 
-This runbook is for deployment, on-call, and maintenance engineers. End-user UI flows live in the [Planet Manual](/home/ray/dev/linkong/planet/docs/technical/en/manual.md); this document only covers shell, Docker, logs, environment variables, and troubleshooting.
+This runbook is for deployment, on-call, and maintenance engineers. End-user UI flows live in the [Intelligent Planet Manual](/home/ray/dev/linkong/planet/docs/technical/en/manual.md); this document only covers shell, Docker, logs, environment variables, and troubleshooting.
 
 ## First Startup
 
@@ -21,7 +21,7 @@ First startup seeds two default accounts (see `DEFAULT_LOGIN_USERS` in `backend/
 | Username | Password | Role |
 | --- | --- | --- |
 | `admin` | `admin123` | `super_admin` |
-| `linkong` | `12345678` | `super_admin` |
+| `linkong` | `LK12345678` | `super_admin` |
 
 Both seed accounts are created with `email_verified = TRUE` and can log into the console immediately. Any other account must either go through the public registration flow described in the Manual, or be created via `./planet.sh createuser`.
 
@@ -61,6 +61,22 @@ Per-module restart:
 
 Per-module restart is preferred during development to avoid interrupting unrelated services.
 
+## Destructive Reset
+
+```bash
+./planet.sh destroy
+```
+
+`destroy` returns a local development environment to a near-empty project state. It requires typing `Y` before it runs; source files and existing `.env` files are preserved.
+
+Cleanup order and boundaries:
+
+- If `planet_postgres` is running, the script first clears the `public` schema in `planet_db`. This prevents old `collected_data.is_current = true` rows from making Earth OOBE report `ready=true` if Docker volume removal later fails.
+- Docker cleanup targets resources whose Compose project is `planet`, plus the explicit volumes `planet_postgres_data`, `planet_redis_data`, `postgres_data`, and `redis_data`; do not delete unlabeled volumes by a broad `planet_*` pattern, because another local project could own them.
+- Local build state removes `.venv`, frontend `node_modules` / `dist`, Planet state, and scattered Python / Vite cache directories. `$PLANET_CACHE_DIR/downloads` is preserved so upstream raw downloads such as CelesTrak can survive database resets and local rebuild cleanup.
+
+After the reset, run `./planet.sh init` again to recreate tables and default seed data. Old collected records are not restored, and Earth OOBE is evaluated from the backend's real collection state on the next visit. When CelesTrak later returns its "GP data has not updated" HTTP 403, the backend first reuses the preserved download cache to repopulate the database; if no cache exists, wait for the next CelesTrak update window or use Space-Track as a fallback.
+
 ## Health Check
 
 ```bash
@@ -87,8 +103,8 @@ Recent logs:
 Follow:
 
 ```bash
-./planet.sh log -f   # frontend: /tmp/planet_frontend.log
-./planet.sh log -b   # backend:  /tmp/planet_backend.log
+./planet.sh log -f   # frontend: ~/.local/state/planet/frontend.log
+./planet.sh log -b   # backend:  ~/.local/state/planet/backend.log
 ./planet.sh log -a   # AI Provider: planet_aiprovider container logs
 ```
 
@@ -120,6 +136,20 @@ Useful for:
 - Demoing Earth from a phone or tablet
 - Other LAN machines reaching the same dev instance
 
+On Windows, the repository-root `planet.cmd` can be used as a one-click entrypoint. It requests Administrator privileges, enters the `Ubuntu` WSL distribution at `/home/linkong/planet`, runs `./planet.sh restart --allow-lan`, opens `http://localhost:3000/earth` after a successful restart, and leaves the terminal inside a WSL shell for log inspection. If the local WSL distribution name or checkout path differs, adjust the `wsl.exe -d ... --cd ...` arguments in `planet.cmd` first.
+
+On a new Windows machine, check the WSL generation first:
+
+```powershell
+wsl -l -v
+```
+
+Planet development should use WSL2. WSL1 has different networking, filesystem, and process behavior, and can surface as Bun package-manager commands returning only `An unknown error occurred (Unexpected)`, unstable port release, or LAN behavior that does not match the script's assumptions. Convert the distribution if it still runs as WSL1:
+
+```powershell
+wsl --set-version Ubuntu 2
+```
+
 `--allow-lan` directly exposes the frontend, backend, and AI Provider from the development machine: frontend `3000`, backend `8000`, and AI Provider `8010`. Before startup, the script checks all three ports. If WSL/Linux cannot release a port and a Windows-side listener or stale `portproxy` rule owns it, the script requests Administrator PowerShell cleanup. When Planet runs in WSL, Windows can usually reach it through `localhost`; other LAN machines reaching the Windows LAN IP still need Windows Firewall allow rules.
 
 Diagnose in this order:
@@ -184,13 +214,32 @@ The AI Provider image only rebuilds when code, Dockerfile, Compose config, or Py
 ./planet.sh restart -a
 ```
 
+Rebuild detection is based on a content fingerprint rather than only file mtimes. `planet.sh` hashes the `aiprovider/` files, `aiprovider/Dockerfile`, `pyproject.toml`, `uv.lock`, `PYTHON_IMAGE`, `UV_IMAGE`, and the dependency fingerprint into `AI_PROVIDER_BUILD_FINGERPRINT`; Docker writes it into the image label `planet.aiprovider.build-fingerprint`. If the existing `planet-aiprovider:latest` image has a matching label, the script skips rebuild and refreshes the local stamp. Older images without the label fall back to the state/cache stamp.
+
+Docker builds use `uv sync --frozen`. To make container builds reuse the host uv mirror configuration, the script resolves the first config file in this order and mounts it into the build as a BuildKit secret at `/root/.config/uv/uv.toml`:
+
+1. The current `UV_CONFIG_FILE`
+2. Repository-root `uv.toml`
+3. `${XDG_CONFIG_HOME:-~/.config}/uv/uv.toml`
+4. `~/.uv/uv.toml`
+
+If none exists, the script creates an empty state file for the secret so Compose does not fail on a missing file. Before Docker build it unsets `UV_DEFAULT_INDEX`, `UV_INDEX_URL`, and `UV_EXTRA_INDEX_URL`, keeping the build tied to the explicit `UV_CONFIG_FILE`. For a temporary Tsinghua mirror, place this in repository-root `uv.toml`:
+
+```toml
+[[index]]
+name = "tsinghua"
+url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/simple/"
+default = true
+```
+
 Diagnose slow builds:
 
 | Symptom | Common cause | Fix |
 | --- | --- | --- |
 | Large `transferring context` | build context includes unrelated frontend / data files | `.dockerignore` ships only required files |
-| `uv sync` is slow | first build or cold cache | wait for the first build; later runs reuse BuildKit cache |
+| `uv sync --frozen` is slow | first build, cold cache, or missing uv mirror config | wait for the first build; later runs reuse BuildKit cache; configure `uv.toml` when needed |
 | Old keys still in effect after edit | container not restarted | `./planet.sh restart -a` |
+| Code changed but the image did not rebuild | fingerprint still matches the image label | Confirm the change is under `aiprovider/`, Dockerfile, or Python dependency inputs; delete `planet-aiprovider:latest` and retry if needed |
 
 ## SMTP Email (Required for Public Registration)
 
@@ -217,6 +266,15 @@ One-time codes are stored in Redis under `otp:{purpose}:{email}` with a 600-seco
 
 ## Development Command Conventions
 
+Backend setup and script initialization use the lockfile:
+
+```bash
+uv python install 3.14
+uv sync --frozen --group dev
+```
+
+`--frozen` rejects implicit `uv.lock` rewrites, which is the desired behavior on new machines, CI, and Docker builds. Dependency upgrades should explicitly update `pyproject.toml` / `uv.lock` on a development machine and commit the lockfile.
+
 Frontend must use Bun:
 
 ```bash
@@ -234,6 +292,20 @@ Validate the frontend build:
 source ~/.zshrc && bun run build
 ```
 
+Use `bun run dev` during development; Vite HMR refreshes the browser after source saves. `bun run build` only writes the `dist` artifact and does not refresh an already-open dev page.
+
+To inspect the production bundle with automatic reload after successful builds:
+
+```bash
+bun run preview:auto
+```
+
+To watch sources and rebuild continuously without starting the preview server:
+
+```bash
+bun run build:watch
+```
+
 Backend dependencies are managed with uv:
 
 ```bash

docs/technical/en/platform-data-flows.md

@@ -0,0 +1,160 @@
+# Business Architecture and Data Flows
+
+This document is the business entry point for Intelligent Planet data products. It explains why each Earth data type exists, where it is collected from, which fact or derived tables it uses, and how cache invalidation plus WebSocket hints reach Earth. Frontend, backend, and Earth docs should focus on implementation details; start here when you need the cross-system data flow.
+
+## Overview
+
+Intelligent Planet data moves through three stages:
+
+1. **Collect and normalize**: built-in collectors, admin actions, or the location pipeline write PostgreSQL. Generic raw output lands in `collected_data`; layer-ready projections land in derived tables.
+2. **Project and broadcast**: database triggers write fact changes to the `earth_data_change_events` outbox and wake the backend listener with `LISTEN/NOTIFY`. The listener maps the change through layer adapters, invalidates cache, and broadcasts `earth_updates`.
+3. **Reload and render in Earth**: Earth receives layer-level refresh hints, applies `clear_then_reload`, `reload`, or `delta`, then refetches `/api/v1/visualization/...`.
+
+```mermaid
+flowchart LR
+  Source["External source / admin action"] --> Collector["Collector or data job"]
+  Collector --> Facts["collected_data"]
+  Collector --> Derived["Derived tables"]
+  Facts --> Outbox["earth_data_change_events"]
+  Derived --> Outbox
+  Outbox --> Listener["DB change listener"]
+  Listener --> Cache["Earth cache invalidation"]
+  Listener --> WS["earth_updates"]
+  WS --> Earth["Earth layer reload"]
+  Earth --> API["Visualization APIs"]
+```
+
+`LISTEN/NOTIFY` is only the low-latency wakeup path. The reliable source is the outbox. A real zero-data state is valid and APIs should return HTTP 200 with an empty collection; only actual endpoint failures should return 5xx.
+
+## Data Products
+
+```mermaid
+flowchart TB
+  Satellites["Satellite TLE"] --> SatelliteLayer["satellites layer"]
+  Cables["Cables + landing points"] --> CableLayer["cables layer"]
+  Compute["TOP500 / AI GPU / HF"] --> ComputeLocations["compute_center_locations"]
+  ComputeLocations --> ComputeLayer["computeCenters layer"]
+  BgpRaw["RIS Live / BGPStream / Prefix"] --> BgpDerived["bgp_observations / anomalies / incidents"]
+  BgpDerived --> BgpLayer["bgp layer"]
+  VesselRaw["AIS / BarentsWatch"] --> VesselDerived["vessel_static / vessel_position"]
+  VesselDerived --> VesselLayer["vessels layer"]
+  Interactables["earth_interactables"] --> InteractableLayer["interactables layer"]
+  NewsRaw["RSS / Live / News"] --> NewsItems["earth_news_items"]
+  NewsItems --> NewsLayer["news / media layer"]
+```
+
+| Data product | Business use | Fact sources | Derived / dimension tables | Earth layer | Strategy |
+| --- | --- | --- | --- | --- | --- |
+| Satellites | In-orbit objects, tracks, coverage, and cruise targets | `celestrak_tle`, `spacetrack_tle` | No long-lived projection; APIs convert TLE on demand | `satellites` | `clear_then_reload` |
+| Cables and landing points | Submarine connectivity, landing cities, cable detail, and search | `arcgis_cables`, `arcgis_landing_points`, TeleGeography / FAO landing sources | Cable relation and landing aggregation data | `cables` | `clear_then_reload` |
+| Compute centers | TOP500, AI GPU, model platform facilities, and location-completion state | `top500`, `epoch_ai_gpu`, HuggingFace sources | `compute_center_locations` | `computeCenters` | `reload` |
+| BGP context | Collectors, anomalies, incidents, route events, and regional context | `ris_live_bgp`, `bgpstream_bgp`, prefix geography sources | `bgp_observations`, `bgp_anomalies`, `bgp_incidents`, `bgp_collector_locations` | `bgp` | `clear_then_reload` |
+| Vessels | AIS vessels, positions, tracks, legend, and source health | AIS sources, `barentswatch_vessels` | `vessel_static`, `vessel_position`, `ais_raw_observations`, `ais_source_health` | `vessels` | `clear_then_reload` |
+| Interactables | Generic surface icons, manual objects, and future small layers | `earth_interactables` | None | `interactables` | `delta` |
+| News and media | Earth news, live streams, cruise summaries, and situation content | news sources | `earth_news_items` | `news` / `media` | `reload` |
+
+## Satellites
+
+Satellite data powers online satellite markers, tracks, footprint policy, and cruise lists. CelesTrak or Space-Track collectors fetch TLE and write `collected_data`. The visualization API converts TLE into current positions and trails on request.
+
+- **Collection entry**: CelesTrak TLE, Space-Track TLE.
+- **Fact table**: `collected_data.source IN ('celestrak_tle', 'spacetrack_tle')`.
+- **API**: satellite visualization API reads TLE and builds the Earth payload.
+- **Delete semantics**: deleting the source broadcasts `satellites` with `clear_then_reload`; the frontend clears satellite markers and trails before refetching. With no TLE, the API returns an empty list.
+- **Common failure**: summary is zero but Earth still shows satellites. Usually WS did not fire, the adapter did not cover the source, or the frontend did not clear existing Three.js objects.
+
+## Cables and Landing Points
+
+Cables and landing points show global network connectivity, landing cities, cable details, and search targets. Cable lines and landing points are one business layer; deleting either side must refresh `cables`.
+
+- **Collection entry**: ArcGIS cables, ArcGIS landing points, and TeleGeography / FAO landing related sources.
+- **Fact table**: cable and landing point sources in `collected_data`.
+- **Derived data**: cable relation tables, landing point aggregation, interface cache.
+- **API**: cable visualization API returns cables, landing points, and relation data.
+- **Delete semantics**: deleting cable or landing source clears owned projections and broadcasts `cables` with `clear_then_reload`. Empty data returns HTTP 200, not 404.
+- **Common failure**: cables disappear for a second and return. Usually a collection replacement or cache refresh window reuses old cache; check duplicate broadcasts and cache patterns.
+
+## Compute Centers
+
+Compute centers represent supercomputers, AI GPU sites, model-platform facilities, and unresolved-location state. Raw sources often only provide organization, country, site, or fuzzy location, so Earth rendering depends on `compute_center_locations`.
+
+- **Collection entry**: TOP500, Epoch AI GPU, HuggingFace related sources.
+- **Fact table**: compute sources in `collected_data`.
+- **Dimension table**: `compute_center_locations` stores adopted coordinate candidates.
+- **API**: compute center visualization API merges raw records with location rows.
+- **Delete semantics**: deleting TOP500 or similar sources refreshes `computeCenters`; changing the location table also refreshes the layer. The default is `reload`, because location updates do not always need an immediate clear.
+- **Common failure**: collection finished but Earth count did not change. Check visualization cache, location rows, and adapter coverage.
+
+## BGP
+
+BGP data shows route collectors, anomalies, incidents, diffusion rings, and situation summaries. Earth does not render raw BGP rows directly; it renders projected observations, anomalies, and incidents. This is the most common path where deleting raw data can leave old objects visible.
+
+```mermaid
+sequenceDiagram
+  participant UI as Admin delete button
+  participant Job as clear_data job
+  participant DB as PostgreSQL
+  participant Outbox as earth_data_change_events
+  participant Listener as DB change listener
+  participant Earth as Earth frontend
+
+  UI->>Job: Submit delete for ris_live_bgp / bgpstream_bgp
+  Job->>DB: Delete collected_data raw rows
+  Job->>DB: Delete BGP-owned derived tables
+  DB->>Outbox: trigger writes bgp layer changed
+  DB-->>Listener: LISTEN/NOTIFY wakeup
+  Listener->>Listener: merge events and clear bgp cache
+  Listener-->>Earth: broadcast earth_updates clear_then_reload
+  Earth->>Earth: clear BGP objects
+  Earth->>DB: refetch projections through visualization API
+```
+
+- **Collection entry**: RIPE RIS Live BGP, CAIDA BGPStream Backfill, IPtoASN / OpenGeoFeed / NRO prefix geography.
+- **Fact table**: BGP and prefix sources in `collected_data`.
+- **Derived tables**: `bgp_observations`, `bgp_anomalies`, `bgp_incidents`, `bgp_collector_locations`.
+- **API**: BGP visualization API reads projections and combines stored or candidate locations.
+- **Delete semantics**: deleting RIS Live or BGPStream raw sources must also clear BGP-owned projections and broadcast `bgp` with `clear_then_reload`. Direct derived-table deletes also write outbox events.
+- **Common failure**: collectors or incidents remain after raw rows are removed. Check derived tables first, not only `collected_data`.
+
+## Vessels
+
+Vessel data shows AIS vessels, navigation state, vessel-type legend, and source health. Earth rendering uses position snapshots and static vessel information; it should not render every raw AIS observation.
+
+- **Collection entry**: AIS sources, BarentsWatch vessels.
+- **Fact table**: `collected_data` or AIS raw observation tables.
+- **Derived tables**: `vessel_static`, `vessel_position`, `ais_raw_observations`, `ais_source_health`.
+- **API**: vessel visualization API returns current vessel markers and detail fields.
+- **Delete semantics**: deleting any vessel source lets owned projections broadcast `vessels` with `clear_then_reload`.
+- **Common failure**: the count panel changes but vessels remain. Summary and layer data are separate; Earth should clear objects based on layer updates.
+
+## Interactables
+
+`earth_interactables` is the generic surface-icon capability for manual objects, extensions, and future small layers. Unlike most layers, it keeps object-level delta.
+
+- **Fact table**: `earth_interactables`.
+- **API**: interactable API and Earth generic icon interface.
+- **Strategy**: upsert for create/update and removeItem for delete; no full layer reload.
+- **Delete semantics**: delete payloads must include a stable id so the frontend can remove the object.
+- **Common failure**: an object cannot be removed. Usually the stable id is missing, the delta handler did not match the object type, or an older duplicate render path still exists.
+
+## News and Media
+
+News and media support the Earth news ticker, live stream panel, news cruise, and situation summaries. They are content refresh paths rather than stable geographic object layers, so they default to `reload`.
+
+- **Collection entry**: RSS, live streams, news sources.
+- **Fact table**: news source rows in `collected_data`.
+- **Derived table**: `earth_news_items`.
+- **API**: news, live stream, and media content APIs.
+- **Delete semantics**: deleting news sources or `earth_news_items` broadcasts `news` / `media` reload; empty responses hide the corresponding content.
+- **Common failure**: the live panel shows stale content. Usually the media component ignored the layer update or the content API cache was not invalidated.
+
+## Adding a New Layer
+
+Add a new Earth data product in this order:
+
+1. Define the business purpose and Earth layer name.
+2. Identify fact sources, fact tables, and derived tables.
+3. Register source/table coverage, cache patterns, owned derived cleanup, and default strategy in the backend layer adapter.
+4. Ensure the visualization API returns HTTP 200 with an empty collection for zero data.
+5. Keep the Earth frontend driven only by layer and strategy, not database table names.
+6. Update this document with the data product flow, then document implementation details in the relevant frontend, backend, or Earth technical doc.

docs/technical/en/quickstart.md

@@ -1,8 +1,8 @@
 # Quickstart
 
-This quickstart is for Planet end users who just received an access URL and need the shortest path from "open the browser" to "first useful configuration done". Every action happens in the browser.
+This quickstart is for Intelligent Planet end users who just received an access URL and need the shortest path from "open the browser" to "first useful configuration done". Every action happens in the browser.
 
-If you are responsible for deployment or operations, read the [Planet Ops Runbook](/home/ray/dev/linkong/planet/docs/technical/en/ops-runbook.md) instead.
+If you are responsible for deployment or operations, read the [Intelligent Planet Ops Runbook](/home/ray/dev/linkong/planet/docs/technical/en/ops-runbook.md) instead.
 
 ## 1. Open the URL
 
@@ -34,7 +34,7 @@ After landing on the `/admin` dashboard, here's a recommended walk-through:
 
 1. `/collection-management?tab=collector_credentials`: pick a collector and click the plug icon to test connectivity. Free collectors (e.g. open BGP) usually work right away; credential-bearing ones like `AISStream` or `BarentsWatch` need an API key / client secret first
 2. `/ai?tab=providers`: fill an LLM provider (e.g. `minimax` / `openai`), model, base URL, API key, and click the plug at the end of the base URL to test. WebSearch / OCR tools are optional
-3. `/datasources` or `/data`: check whether collectors have produced data. Use `/datasources -> Collection Tasks` for finite collectors, and `/datasources -> Realtime Streams` for AISStream / WebSocket health and counters
+3. `/datasources` or `/data`: check whether collectors have produced data. Use `/datasources -> Built-in Sources` for finite collectors: with no rows selected, click `Trigger All`; after selecting rows, the primary button becomes `Trigger Selected N`. The top-right queue button shows progress. Use `/datasources -> Realtime Sources` for AISStream / WebSocket health and counters
 4. `/alerts/system`: verify system alerts look right
 5. `/users` (super_admin only): open accounts for teammates or adjust their groups
 
@@ -42,6 +42,8 @@ After landing on the `/admin` dashboard, here's a recommended walk-through:
 
 Visit `/earth`. This is a public page — no login required.
 
+If the system has no collected data yet, Earth shows an initialization guide that points you to sign in and trigger collection. This guide is backend-state driven, so clearing browser storage does not make it appear once the system is ready.
+
 Once in, verify:
 
 - The globe renders, and the right-side layer panel can toggle layers
@@ -60,7 +62,7 @@ Open `/forgot-password`, enter your email, receive a code, then enter the code p
 
 ## Next Steps
 
-- Full UI walkthrough: [Planet Manual](/home/ray/dev/linkong/planet/docs/technical/en/manual.md)
+- Full UI walkthrough: [Intelligent Planet Manual](/home/ray/dev/linkong/planet/docs/technical/en/manual.md)
 - Troubleshooting and configuration questions: [FAQ](/home/ray/dev/linkong/planet/docs/technical/en/faq.md)
-- Detailed Earth coordinate candidate flow: [Earth Location Candidate Collection User Guide](/home/ray/dev/linkong/planet/docs/technical/en/location-pipeline-user.md)
-- Deployment / operations commands: [Planet Ops Runbook](/home/ray/dev/linkong/planet/docs/technical/en/ops-runbook.md)
+- Detailed Earth coordinate candidate flow: see the Earth section in [Intelligent Planet Manual](/home/ray/dev/linkong/planet/docs/technical/en/manual.md)
+- Deployment / operations commands: [Intelligent Planet Ops Runbook](/home/ray/dev/linkong/planet/docs/technical/en/ops-runbook.md)