release: bump version to 0.74.7

Release 0.58.0 includes the Earth high-precision boundary PMTiles/MVT pipeline, standardized Earth boundary source collectors, China POV boundary configuration templates, and removal of the legacy low-precision GeoJSON fallback. It also adds Earth news target-location queueing/archive support, fixes datasource task status visibility, documents the Earth surface depth-spacing rules that prevent far-zoom z-fighting snow/black blocks, and updates bilingual operations/developer docs.

.claude/commands/cleanup.md

@@ -1,139 +0,0 @@
----
-description: 审查当前工作区未提交代码中的垃圾代码，并在不影响逻辑的前提下自动清理
-argument-hint: 可选：指定要检查的文件或目录（默认检查所有未提交修改）
-allowed-tools: ["Read", "Edit", "Bash", "Grep", "Glob"]
----
-
-# /cleanup — 垃圾代码审查与清理
-
-分析当前工作区（git diff）中的未提交代码，找出并修复常见垃圾代码，**不得改变任何运行逻辑**。
-
-## 检查范围
-
-若 `$ARGUMENTS` 非空，则只检查指定文件/目录；否则检查所有未提交修改（`git diff HEAD`）。
-
-## 节省上下文规则
-
-优先用确定性的 CLI 检查缩小范围，不要一上来把完整文件或大 diff 读入上下文：
-
-```bash
-git diff --name-only HEAD
-git diff --unified=0 HEAD -- <path>
-git diff --check
-rg -n "TODO|FIXME|console\.log|debugger|print\(" <changed-paths>
-```
-
-只有 focused diff 不足以安全判断或修改时，才读取完整文件。
-
-## 审查清单
-
-按优先级检查以下问题（只报告在本次 diff 中**新增或修改**的代码里存在的问题）：
-
-### 1. 重复逻辑 (Duplicate Logic)
-- 完全相同或高度相似的代码块在多处出现
-- 同一函数/方法被多个地方各自实现，已有公共版本未被复用
-- 相同的 DOM 查询、正则、模板字符串在同一文件重复
-
-### 2. Magic Numbers / Magic Strings
-- 裸数字直接参与计算（如偏移量、时间、尺寸、阈值），没有命名常量
-- 硬编码字符串（如 id 名、状态值、URL 片段）散落在逻辑中
-- 例外：`0`, `1`, `-1`, `100`, `""` 等语义明确的惯用值不算
-
-### 3. 命名问题
-- 含义不明的缩写变量（如 `or_`, `tmp2`, `x2`）
-- 命名与实际用途不符
-- 同一概念在不同地方用不同名字表达
-
-### 4. 死代码 / 无效代码
-- 注释掉的旧代码块（3行以上）
-- 声明后从未使用的变量/参数/导入
-- 永远不会执行的条件分支
-
-### 5. 代码风格问题
-- 尾部空白字符（trailing whitespace）
-- 同一文件内风格不一致（如混用单双引号、缩进不统一）
-- 空行使用不一致（连续多个空行等）
-
-### 6. 其他常见问题
-- 私有辅助函数应被 export 但没有，导致调用方重复实现
-- 类型/接口重复定义
-- 过于冗长的条件表达式可以简化（不改逻辑）
-
-## 执行步骤
-
-### Step 1 — 获取待检查文件列表
-
-```bash
-# 无参数时：获取所有未提交修改
-git diff HEAD --name-only
-
-# 有参数时：用 $ARGUMENTS 过滤
-```
-
-### Step 2 — 逐文件阅读并分析
-
-先从 focused diff 开始：
-
-```bash
-git diff --unified=0 HEAD -- <file>
-```
-
-用 `rg`、`git diff --check`、编译器或 linter 输出确认确定性问题。只有需要上下文时才用 Read 读取完整文件。对照审查清单，记录每个问题：文件名、行号、问题类型、建议修复方式。
-
-### Step 3 — 报告问题清单
-
-在修改前，先以列表形式输出所有发现的问题：
-
-```
-发现 N 个问题：
-
-[文件] js/foo.js
-  · L34, L78: 重复逻辑 — 两处都实现了相同的 DOM 查询，可提取到 getPanel()
-  · L91: Magic number — 硬编码 14 作为偏移量，应命名为 TOOLTIP_OFFSET
-
-[文件] js/bar.js
-  · L12: 命名问题 — 变量 `or_` 语义不明，应命名为 outerR/outerG/outerB
-  ...
-```
-
-如果没有发现问题，直接输出"未发现垃圾代码，当前代码质量良好。"并停止。
-
-### Step 4 — 执行修复
-
-对每个问题，使用 Edit 工具进行**最小化修改**：
-
-- **重复逻辑**：提取为共享常量/函数，更新所有调用点
-- **Magic number**：在文件顶部或逻辑附近声明 `const NAME = value`，替换所有引用
-- **命名问题**：重命名变量，更新所有使用处
-- **死代码**：直接删除
-- **尾部空白/风格**：修正
-- **未 export 的函数**：添加 `export`，在调用方改为导入（不重复实现）
-
-**修复原则：**
-- 只改在审查清单中发现的问题，不做额外优化
-- 每次 Edit 只修改确实有问题的行，保持 diff 最小
-- 改完后用 `grep` 验证旧的坏代码已消失
-- 优先做精确补丁；只有仓库已有对应格式化流程时，才运行格式化工具
-
-### Step 5 — 输出总结
-
-```
-清理完成：
-
-修复了 N 个问题：
-  ✓ earth.js — 提取重复 vertexShader 为 ATMOS_VERTEX_SHADER 常量
-  ✓ main.js — 提取 TOOLTIP_CURSOR_OFFSET = 14（4处引用）
-  ✓ controls.js — export updateLayerButtonState，移除 main.js 中的重复实现
-  ...
-
-未修改的问题（需人工确认）：
-  ! foo.js L45 — 注释代码块较长，建议手动确认是否可删除
-```
-
-## 约束
-
-- **禁止**改变函数签名、接口定义、导出 API（除非问题正是私有函数应被 export）
-- **禁止**添加新功能、新抽象、新参数
-- **禁止**修改注释内容（只删除注释掉的死代码）
-- **禁止**修改测试文件逻辑
-- 如果一个 Magic number 的语义不完全确定，**跳过**，在总结中标记为"需人工确认"

.claude/commands/docs.md

@@ -1,103 +0,0 @@
----
-description: Create or update repository documentation from current code changes
-argument-hint: Optional: topic to document, or leave empty to infer from git diff
-allowed-tools: ["Read", "Edit", "Write", "Bash", "Glob", "Grep"]
----
-
-# /docs — Documentation Workflow
-
-## Goal
-
-Create or update documentation that explains why a change exists, how it behaves, and what maintainers need to know. Keep this command generic. Repository-specific coverage rules live in the repository and must be loaded separately.
-
-## Repository Rules
-
-Before deciding scope, check whether the repository has a documentation rules file:
-
-```bash
-test -f docs/documentation-coverage-rules.md && sed -n '1,240p' docs/documentation-coverage-rules.md
-```
-
-If it exists, apply it as the project-specific coverage checklist. If it does not exist, continue with the generic workflow below.
-
-## Workflow
-
-### Step 1 — Understand The Change
-
-```bash
-git diff HEAD --stat
-git diff HEAD --name-only
-git log --oneline -10
-rg --files docs
-```
-
-If `$ARGUMENTS` specifies a topic, focus on that topic. Otherwise infer the documentation topic from the changed files. Do not read the full repository diff by default; inspect focused files only:
-
-```bash
-git diff HEAD -- <path>
-rg -n "class |def |function |export |router|@router|interface |type " <path>
-```
-
-### Step 2 — Decide Scope
-
-- Prefer updating an existing relevant document over creating a duplicate.
-- Use one document for one coherent topic.
-- Split documents only when the change crosses meaningful domains.
-- Keep filenames lowercase and hyphenated.
-- Apply the repository-specific rules file before writing.
-
-#### Document Audience Routing (Planet)
-
-In this repository, classify the action's performer before picking a target file:
-
-- Browser/UI end user → `docs/technical/{zh,en}/manual.md` or `quickstart.md`.
-- Shell / Docker / log paths / `planet.sh` / SMTP fallbacks / port forwarding → `docs/technical/{zh,en}/ops-runbook.md` (or an existing `ops-*.md`).
-- Second-party developers → existing `*-context.md` / `backend-*.md` / `earth-*.md` files.
-
-Never put shell commands, log paths, or Docker operations into `manual.md` / `quickstart.md`. Never put UI button labels or screenshots into `ops-*.md`. When the same action has both a UI and a CLI path, write each in its own home and cross-link them with one sentence.
-
-For ambiguous or large documentation changes, briefly state the intended doc plan before editing. For clear small changes, proceed directly.
-
-### Step 3 — Write
-
-Explain:
-
-- Background/problem: what was wrong or missing before.
-- Core design decisions and rationale.
-- Operational or user-facing impact.
-- Relevant code paths, only when useful for future maintainers.
-
-Style:
-
-- Follow the repository’s existing language and heading conventions.
-- Use fenced code blocks with language tags.
-- Prefer tables for comparisons or parameter lists.
-- Keep snippets concise and relevant.
-
-### Step 4 — Verify
-
-- Read the completed docs once for clarity and stale statements.
-- Verify referenced paths exist with `test -e` or `rg --files`.
-- Run applicable checks from `docs/documentation-coverage-rules.md`.
-- Check Markdown links use readable user-facing titles unless repository rules allow otherwise.
-
-### Step 5 — Report
-
-Summarize changed docs and verification:
-
-```md
-Updated:
-- path/to/doc.md — what changed
-
-Verified:
-- checks that passed
-- checks that could not be run, if any
-```
-
-## Hard Constraints
-
-- Do not leave placeholder docs.
-- Do not duplicate bilingual files byte-for-byte.
-- Do not reference PR numbers, issue numbers, or the current conversation unless explicitly requested.
-- Do not write changelog-style lists without the reasoning and tradeoffs behind the change.
-- Keep docs maintainable and concise.

.claude/commands/goal-driven.md

@@ -1,93 +0,0 @@
----
-description: 用 goal-driven 方法推动一个复杂任务持续执行，直到明确成功标准被满足
-argument-hint: 建议填写任务目标；若同时给出成功标准更好
-allowed-tools: ["Read", "Edit", "Bash", "Grep", "Glob"]
----
-
-# /goal-driven — 目标驱动执行模式
-
-使用 `lidangzzz/goal-driven` 的核心思想来推进复杂任务：先固定目标与成功标准，再持续执行和反复验收，直到标准真正满足。
-
-适用场景：
-
-- 长周期实现任务
-- 高复杂度工程任务
-- 可被明确验收的研究、实现、迁移、验证类工作
-
-不适用场景：
-
-- 纯脑暴
-- 无法定义成功标准的模糊任务
-- 很小的一次性修改
-
-## 输入要求
-
-若 `$ARGUMENTS` 只包含目标，没有成功标准，先补全一版可执行的成功标准再开始。
-
-启动时先输出：
-
-```md
-Goal
-- ...
-
-Criteria for success
-- ...
-
-Plan
-1. ...
-2. ...
-3. ...
-
-Verification
-- ...
-```
-
-## 执行规则
-
-1. 先把任务固化为两个核心块：
-   - `Goal`
-   - `Criteria for success`
-
-2. 成功标准必须尽量客观，可验证，可落地。
-   优先写成：
-   - 需要交付什么
-   - 需要通过哪些测试或验证
-   - 如何判断结果真的完成
-
-3. 进入持续执行循环：
-   - 完成一个阶段
-   - 检查当前结果是否满足成功标准
-   - 若未满足，明确剩余差距并继续推进
-
-4. 任何“完成了”“差不多了”“已实现”之类的结论，都必须经过验证，不能直接接受。
-
-5. 如果验证失败：
-   - 明确指出哪条成功标准没满足
-   - 继续工作，不要把阶段性进展误判为完成
-
-6. 只有在以下情况之一才能停止：
-   - 成功标准已满足
-   - 用户明确要求停止
-
-## 执行风格
-
-- 重证据，轻口头判断
-- 优先使用确定性工具证据：`rg`、`git diff --stat`、`git diff -- <path>`、测试、构建、lint、`curl`、数据库查询等能直接证明成功标准的方式
-- 不把大段命令输出粘进回复；保留在工具调用里，回复只总结关键证据
-- 重验收，轻自我感觉
-- 优先用测试、日志、产物、对比结果来证明完成
-- 对长期任务保持“未达标就继续”的节奏
-
-## 简版模板
-
-```md
-Goal: [[[[[在此填写最终目标]]]]]
-
-Criteria for success: [[[[[在此填写成功标准]]]]]
-
-循环执行：
-1. 推进任务
-2. 检查是否满足成功标准
-3. 若未满足，继续工作
-4. 直到满足标准或用户明确停止
-```

.claude/commands/release.md

@@ -1,160 +0,0 @@
----
-description: 发版工作流：根据变更类型决定版本号，更新所有版本文件和 changelog，运行验证，commit 并 push
-argument-hint: 可选：feature | bugfix | 或直接描述本次发布内容
-allowed-tools: ["Read", "Edit", "Bash", "Glob", "Grep"]
----
-
-# /release — Planet 发版工作流
-
-## 版本号规则
-
-| 变更类型 | 版本跳动 | 适用场景 |
-|---------|---------|---------|
-| `feature` | `+0.1.0` | 纯新功能，无 bugfix |
-| `improvement` | `+0.0.1` | UI 调整、小功能增强、bugfix 混合，或以 UI/体验改进为主的迭代 |
-| `bugfix` | `+0.0.1` | 纯 bug 修复，无新功能 |
-| `docs` / `maintenance` / `refactor` | 默认不发版，除非用户明确要求 |
-
-意图混合时以用户明确描述为准；bugfix + 小 feature 混合默认判定为 `improvement`（`+0.0.1`）。
-
-## 必须同步更新的文件
-
-使用 `git rev-parse --show-toplevel` 获取仓库根目录，以下路径均相对于根目录：
-
-- `VERSION`
-- `frontend/package.json`（`"version"` 字段）
-- `pyproject.toml`（`version =` 字段）
-- `uv.lock`（**不要手动编辑**，通过 `uv lock` 重新生成）
-- `docs/CHANGELOG.md`
-- `docs/version-history.md`
-
-## 节省上下文规则
-
-发版判断应以确定性 CLI 证据为主，优先使用紧凑命令和定点读取：
-
-```bash
-git status --short
-git diff --stat HEAD
-git diff --name-only HEAD
-rg -n "version|^## |^Released:|当前开发版本|current" VERSION frontend/package.json pyproject.toml docs/CHANGELOG.md docs/version-history.md
-```
-
-除非需要判断某个代码变更是否属于本次发版，否则不要读取完整 diff。
-
-## 执行步骤
-
-### Step 1 — 环境检查
-
-```bash
-git branch --show-current          # 确认在 dev 分支
-git status --short                 # 检查是否有无关的未暂存修改
-cat VERSION                        # 读取当前版本
-```
-
-若当前**不在 `dev` 分支**，停下来告知用户，不要继续。
-
-若存在无关的未暂存修改，列出并询问用户是否一并提交，或先 stash。
-
-### Step 2 — 确定发版类型与新版本号
-
-- 若 `$ARGUMENTS` 提供了明确类型（`feature` / `bugfix`），直接使用
-- 否则根据 `git diff --stat HEAD`、`git diff --name-only HEAD`、必要的 focused diff 和 `git log` 推断
-- 计算新版本号（例：`0.26.2` → bugfix → `0.26.3`）
-- **先输出发版计划供用户确认**：
-
-```
-发版计划：
-  类型：bugfix
-  版本：0.26.2 → 0.26.3
-  分支：dev
-  将更新：VERSION, frontend/package.json, pyproject.toml, uv.lock, CHANGELOG.md, version-history.md
-```
-
-### Step 3 — 更新版本号文件
-
-按顺序更新（每步用 Edit 工具，精确替换，不要重写整个文件）：
-
-1. `VERSION` — 直接替换全部内容为新版本号
-2. `frontend/package.json` — 替换 `"version": "x.x.x"` 行
-3. `pyproject.toml` — 替换 `version = "x.x.x"` 行
-4. 运行 `uv lock` 重新生成 `uv.lock`（在仓库根目录下执行）
-
-### Step 4 — 更新 CHANGELOG.md
-
-在文件顶部插入新条目，格式：
-
-```markdown
-## [x.x.x] — YYYY-MM-DD
-
-### ✨ Features / 🐛 Fixes / 🔧 Improvements
-- ...（只列高信号条目，最多 5 条）
-- ...
-
----
-```
-
-日期使用 `date +%Y-%m-%d` 获取今天的日期。
-
-### Step 5 — 更新 docs/version-history.md
-
-- 更新文件头部的"当前开发版本"字段
-- 在时间线表格顶部插入新行：`| vx.x.x | YYYY-MM-DD | 一句话摘要 |`
-
-### Step 6 — 验证
-
-针对本次变更范围做最小验证：
-
-- Python 文件有修改：先用 `git diff --name-only HEAD -- '*.py'` 列出，再运行 `python3 -m py_compile <changed_files>`
-- Frontend 文件有修改：先用 `git diff --name-only HEAD -- frontend` 判断范围，再运行项目标准检查（若无则跳过并说明）
-- 版本号一致性检查：用 grep 确认 VERSION、package.json、pyproject.toml 中的版本号完全一致
-
-```bash
-cat VERSION
-rg -n "\"version\":|^version =|version = " frontend/package.json pyproject.toml uv.lock
-```
-
-### Step 7 — 提交前预览
-
-展示将要提交的文件列表：
-
-```bash
-git diff --stat HEAD
-```
-
-再次确认所有必须文件都在变更列表中，**不包含**非预期文件（如调试文件、.env 等）。
-
-### Step 8 — Commit & Push（用户确认后）
-
-```bash
-git add VERSION frontend/package.json pyproject.toml uv.lock docs/CHANGELOG.md docs/version-history.md
-# 若有代码变更也一并 stage
-git add <code_files>
-
-git commit -m "release: bump version to x.x.x"
-git tag vx.x.x
-git push origin dev
-git push origin vx.x.x
-```
-
-commit message 固定格式：`release: bump version to x.x.x`
-
-### Step 9 — 完成确认
-
-输出摘要：
-
-```
-✓ 版本号已更新：0.26.2 → 0.26.3
-✓ CHANGELOG 已更新
-✓ version-history 已更新
-✓ uv.lock 已重新生成
-✓ 验证通过
-✓ commit: release: bump version to 0.26.3
-✓ tag: v0.26.3
-✓ 已 push 到 origin/dev
-```
-
-## 注意事项
-
-- `uv.lock` 只能通过 `uv lock` 生成，绝不手动编辑
-- 发版 commit 只包含版本文件 + 本次功能代码，不混入无关改动
-- 若环境中 `uv` 不可用，说明原因并跳过 lockfile 更新，提醒用户手动运行

.codex/skills/docs/SKILL.md

@@ -52,6 +52,7 @@ rg -n "class |def |function |export |router|@router|interface |type " <path>
 - Keep code snippets short and directly relevant.
 - List related files only when they help future maintainers navigate.
 - Use the repository’s existing language, heading style, and naming conventions.
+- For UI labels, chart labels, feature names, datasource names, and other terms that may become mixed Chinese/English copy, check `docs/technical/{zh,en}/naming-glossary.md` and use the documented display name. If a confusing term is missing, update the glossary in both languages as part of the docs change.
 
 4. Verify:
 

.gitignore

@@ -8,6 +8,7 @@
 .env
 .env.local
 .env.*.local
+config/earth-boundary-sources.local.json
 *.pem
 *.key
 *.crt
@@ -24,11 +25,14 @@ __pycache__/
 build/
 develop-eggs/
 dist/
-downloads/
+downloads/*
+!downloads/usbipd-win/
+downloads/usbipd-win/*
+!downloads/usbipd-win/usbipd-win-5.3.0.msi
 eggs/
 .eggs/
-lib/
-lib64/
+/lib/
+/lib64/
 parts/
 sdist/
 var/
@@ -150,3 +154,9 @@ temp/
 # Runtime Data
 # ----------------------
 data/ai/bgp-briefs/
+data/earth-boundary-sources/
+
+# Generated Earth boundary tile artifacts. Keep source configs and builders in
+# Git; publish PMTiles/MVT artifacts through release/deploy storage instead of
+# committing thousands of generated tile files.
+frontend/public/earth/data/boundaries/

AGENTS.md

@@ -0,0 +1,180 @@
+# AGENTS.md
+
+**Planet agent harness. Defines behavior for coding agents working in this repository.**
+
+---
+
+## Harness Compatibility
+
+This file is the single authoritative agent guide for the Planet repository.
+The older lowercase `agents.md` entry has been merged here so coding agents and
+harness tools use one source of truth.
+
+### Source Of Truth
+
+- `rules.md` is the mandatory repository rule source. Always load `core`,
+  `security`, and `workflow`; load only task-relevant modules after that.
+- `AGENTS.md` defines the local agent operating mode and evidence gates.
+- `project_context.md` is background, not a rule source. Prefer newer
+  implementation docs when it disagrees with current code.
+- `.codex/skills/` is the active specialized workflow layer for cleanup, docs,
+  goal-driven work, and release.
+- Do not duplicate long workflow text across harness files. Durable constraints
+  belong in `rules.md`; task procedures belong in skills or scripts.
+
+Read these files before changing code:
+
+1. `rules.md`
+2. `AGENTS.md`
+3. `project_context.md`
+4. `README.md`
+5. `docs/HARNESS.md`
+6. `CODEMAP.md`
+
+For documentation work, also read `docs/documentation-coverage-rules.md`.
+
+### Start Safely
+
+Before broad edits:
+
+```bash
+git status --short
+scripts/harness/doctor.sh
+```
+
+Use focused context commands before reading large files:
+
+```bash
+rg -n "<symbol-or-term>" <path>
+git diff --stat HEAD
+git diff --name-only HEAD
+git diff --unified=0 HEAD -- <path>
+```
+
+Preserve user changes already present in the worktree.
+
+### Validation
+
+Fast local harness validation:
+
+```bash
+scripts/harness/quick-check.sh
+```
+
+Full local validation:
+
+```bash
+scripts/harness/validate.sh
+```
+
+`validate.sh` includes quick checks, frontend Bun build, and frontend smoke
+unless disabled by its documented environment flags. Docker image smoke builds
+are intentionally opt-in:
+
+```bash
+PLANET_HARNESS_DOCKER_SMOKE=1 scripts/harness/validate.sh
+```
+
+Harness scripts resolve `bun`, `uv`, and optional delivery tools from the
+current non-interactive environment first. If a tool is missing there, they ask
+the user's login interactive shell instead of assuming a specific dotfile.
+
+### High-Risk Areas
+
+- `planet.sh` owns local lifecycle, ports, WSL/LAN behavior, and destructive
+  `destroy` cleanup.
+- Frontend package management is Bun-only. Do not use npm, pnpm, or yarn.
+- Frontend changes must satisfy `scripts/harness/frontend-rules-check.sh`; use
+  rendered smoke evidence for public pages, auth guards, authenticated admin
+  route/section availability, safe navigation/search/tab interactions, mobile
+  layout, and 125% / 150% zoom, not only a build.
+- Admin or Docs layout changes must load `rules.md` `uiux` and preserve the
+  one-screen (`一屏` / `首屏`) height chain: route roots use `height: 100%`,
+  intermediate wrappers keep `min-height: 0`, and only the intended child owns
+  scrolling.
+- `aiprovider` is a protocol/provider adapter; keep business prompts and product
+  workflows in the backend.
+- Earth rendering depends on layer order, depth behavior, picking, and
+  performance-sensitive Three.js code.
+- Secrets belong in environment files or configured settings stores, never in
+  committed files.
+- Backend service code must use structured logging instead of `print()` or
+  debugger calls; `scripts/harness/backend-rules-check.sh` enforces this.
+
+### Conflict Policy
+
+Existing project rules and workflows win. If new harness guidance conflicts with
+`rules.md`, `AGENTS.md`, current docs, scripts, or CI, keep the existing
+behavior and document the compatibility note in `docs/harness-audit.md` or
+`docs/HARNESS.md`.
+
+---
+
+## Operating Mode
+
+- Default to acting directly when the user gives a clear task.
+- Ask before acting only when the missing decision is risky, cannot be
+  discovered from repository context, and no conservative assumption is safe.
+- Read relevant files before editing.
+- Prefer focused CLI evidence: `rg`, `git diff --stat`, `git diff --name-only`,
+  focused file reads, tests, builds, linters, and harness scripts.
+- Keep changes scoped to the requested area. Do not mix cleanup, feature work,
+  release work, and documentation unless the task requires it.
+
+---
+
+## Evidence Gates
+
+- Visual inputs are blocking evidence. If the user provides a screenshot, image,
+  mock, browser capture, or visual reference, obtain evidence from the artifact
+  before interpreting intent or editing code.
+- Path resolution is part of the task. If the path cannot be opened, first try
+  reasonable local equivalents such as WSL/Windows path conversion,
+  workspace-relative lookup, absolute paths, and attached-file locations.
+- Never guess from prompt text, filenames, previous context, logs, OCR, or
+  memory when a visual artifact was provided but cannot be accessed.
+- OCR is acceptable evidence for text-only visual questions or non-multimodal
+  environments; state that OCR was used as the fallback. Layout, color, spacing,
+  pixel, and rendering issues need real visual inspection or a clear limitation
+  note.
+- If a visual artifact still cannot be inspected, say so and pause that
+  visual-dependent part of the work.
+- Claims of completion need evidence: a relevant test, build, lint, screenshot,
+  diff, direct file check, or harness result.
+- For UI and rendering changes, verify the rendered result when local tooling
+  allows it.
+
+---
+
+## Communication
+
+- Match the user's language. Use Chinese for Chinese requests unless the user
+  asks otherwise.
+- Keep updates short and specific: what is being inspected, edited, or verified.
+- Final responses should summarize changed files and verification, with blockers
+  stated plainly.
+- Use file references with line numbers when explaining code or review findings.
+
+---
+
+## Quality Bar
+
+- Prefer existing project patterns over new abstractions.
+- Remove stale branches, mocks, compatibility paths, and duplicated helpers once
+  a stable path exists.
+- Centralize prompts, constants, defaults, and shared request/response handling.
+- Do not add secrets, generated runtime output, or local environment files.
+- Frontend commands use Bun only. Do not use `npm`, `pnpm`, or `yarn`.
+- Run the smallest relevant verification for the changed scope and report
+  anything skipped.
+
+---
+
+## Prohibited
+
+- Do not skip visual evidence handling when a visual artifact was provided.
+- Do not preserve obsolete harness files just because they already exist.
+- Do not invent behavior not present in code, docs, or verified external
+  sources.
+- Do not rewrite unrelated files during cleanup.
+- Do not mark a task complete without checking concrete success criteria.

CODEMAP.md

@@ -0,0 +1,109 @@
+# Code Map
+
+This map gives agents and maintainers a quick orientation without replacing the
+deeper architecture docs. Current implementation docs under `docs/technical/`
+are the source of detail for specific subsystems.
+
+## Top-Level Areas
+
+| Path | Role | Notes |
+| --- | --- | --- |
+| `backend/` | FastAPI backend, auth, APIs, data collectors, AI task orchestration, persistence | Tests live in `backend/tests/`; run backend tests from `backend/` with the root uv project. |
+| `frontend/` | React admin console, Docs UI, Web Earth shell, Vite build | Use Bun only. Public Earth assets live under `frontend/public/earth/`. |
+| `aiprovider/` | Model provider/protocol adapter service | Keep it free of product-specific prompts and workflows. |
+| `motion_agent/` | Motion capture protocol service used by `planet.sh` | Often dry-runs when cameras are unavailable, especially in WSL. |
+| `scripts/` | Utility scripts and harness wrappers | Harness commands live in `scripts/harness/`. |
+| `docs/` | Plans, technical docs, changelog, harness docs | Public technical docs are explicitly registered by the frontend Docs catalog. |
+| `deploy/helm/planet/` | Helm chart for staging/deployment smoke paths | CI runs helm lint/template when delivery checks are available. |
+| `.gitea/workflows/` | CI, release image build, staging deploy workflows | This repository uses Gitea workflow files, not `.github/workflows/`. |
+| `planet.sh` | Main local lifecycle script | Owns init/start/restart/stop/health/log/createuser/destroy. |
+
+## Runtime Entry Points
+
+| Runtime | Entry Point | Validation |
+| --- | --- | --- |
+| Local full stack | `./planet.sh start` | `./planet.sh health` |
+| Backend API | `backend/app/main.py` | `cd backend && uv run --frozen --group dev --project .. python -m pytest -q` |
+| Frontend app | `frontend/src/main.tsx` and `frontend/vite.config.mts` | `cd frontend && bun run build` |
+| AI Provider | `aiprovider/main.py` | `curl http://localhost:8010/health` after startup |
+| Motion Agent | `python -m motion_agent` via `planet.sh` | `./planet.sh health` or dry-run startup |
+| Docs UI | `frontend/src/pages/Docs/` | Docs catalog metadata plus frontend build |
+
+## Ownership Boundaries
+
+- Backend owns business state, auth, evidence collection, prompt selection, AI
+  task orchestration, and database persistence.
+- `aiprovider` owns provider identity, request adapter style, model gateway
+  retries, and health/status endpoints only.
+- Frontend owns operator workflows, Docs presentation, Web Earth orchestration,
+  and client-side state that mirrors backend truth.
+- Web Earth rendering changes must preserve documented layer order, altitude
+  offsets, picking behavior, legend semantics, and performance constraints.
+- `planet.sh` owns local environment bootstrap and service lifecycle. Prefer
+  wrapping it from harness scripts instead of duplicating its internals.
+- Harness scripts source `scripts/harness/lib.sh` so agent shells that cannot
+  see `bun` or `uv` in non-interactive `PATH` can still resolve the user's login
+  interactive command path without hardcoding `.zshrc`.
+
+## Validation Commands
+
+```bash
+scripts/harness/doctor.sh
+scripts/harness/security-check.sh
+scripts/harness/backend-rules-check.sh
+scripts/harness/frontend-rules-check.sh
+scripts/harness/docs-consistency-check.sh
+scripts/harness/quick-check.sh
+scripts/harness/validate.sh
+./planet.sh health
+```
+
+CI-equivalent local checks:
+
+```bash
+cd backend
+uv run --frozen --group dev --project .. python -m pytest -s tests/test_api.py tests/test_realtime_sources.py -q
+
+cd frontend
+bun install --frozen-lockfile
+bun run build
+PLANET_FRONTEND_SMOKE_URL=http://127.0.0.1:4173 bun ../scripts/harness/frontend-smoke.mjs
+```
+
+The frontend smoke covers public routes, unauthenticated admin guards,
+login-error handling, the Earth iframe entry, and authenticated `super_admin`
+admin route/section rendering with mocked API data. Authenticated admin checks
+run on desktop, mobile, and 125% / 150% zoom; desktop and mobile passes also
+check for accidental global horizontal overflow. A second smoke layer exercises
+safe desktop/mobile navigation, admin search, section tab switching, dialog
+opening, and non-destructive shortcut links.
+
+Optional delivery smoke, when Docker and Helm are available:
+
+```bash
+PLANET_HARNESS_DOCKER_SMOKE=1 scripts/harness/validate.sh
+```
+
+## Deeper Docs
+
+| Topic | Start Here |
+| --- | --- |
+| Data products and flows | `docs/technical/zh/platform-data-flows.md` and `docs/technical/en/platform-data-flows.md` |
+| Operations and local lifecycle | `docs/technical/zh/ops-runbook.md` and `docs/technical/en/ops-runbook.md` |
+| `planet.sh` startup behavior | `docs/technical/zh/ops-planet-sh-startup.md` and `docs/technical/en/ops-planet-sh-startup.md` |
+| AI Provider | `docs/technical/zh/agents-aiprovider.md` and `docs/technical/en/agents-aiprovider.md` |
+| Admin frontend | `docs/technical/zh/frontend-admin-frontend-context.md` and `docs/technical/en/frontend-admin-frontend-context.md` |
+| Earth frontend | `docs/technical/zh/earth-frontend-context.md` and `docs/technical/en/earth-frontend-context.md` |
+| Earth render order | `docs/technical/zh/earth-render-layer-order.md` and `docs/technical/en/earth-render-layer-order.md` |
+| Documentation rules | `docs/documentation-coverage-rules.md` |
+| Harness workflow | `docs/HARNESS.md` |
+
+## Known Sharp Edges
+
+- `project_context.md` is static background for agents. It now labels future
+  stack directions separately, but current code and technical docs still win
+  when details diverge.
+- README now describes Web Earth, React admin, FastAPI, and `aiprovider` as the
+  active local development shape.
+- Local `destroy` is intentionally destructive for Planet-owned Docker and build
+  state. Never run it as a validation shortcut.

README.md

@@ -8,68 +8,54 @@
 
 ## 系统架构
 
+当前仓库的核心形态是“Web Earth 可视化 + React 运维台 + FastAPI 数据与 AI 编排后端 + 独立模型适配层”。物理大屏与 UE 客户端仍是长期方向，但不再作为本地开发和当前发布的必需运行单元。
+
 ```
-┌─────────────────────────────────────────────────────────────────────────┐
-│                           物理大屏展示层                                  │
-│  ┌─────────────────────────────────────────────────────────────────┐    │
-│  │         偏振片3D大屏 (2m×3m, 4K, 120Hz, 眼镜式)                 │    │
-│  │  ┌─────────────────────────────────────────────────────────┐    │    │
-│  │  │              虚幻引擎 UE5 客户端                          │    │    │
-│  │  │  ├── 3D地球渲染 (Cesium for UE)                          │    │    │
-│  │  │  ├── 算力点可视化 (GPU集群、智算中心)                     │    │    │
-│  │  │  ├── 连接弧线 (光缆、路由、数据流向)                      │    │    │
-│  │  │  ├── 粒子效果 (数据流动、告警提示)                        │    │    │
-│  │  │  └── 自动巡航相机 + 交互控制                              │    │    │
-│  │  └─────────────────────────────────────────────────────────┘    │    │
-│  └─────────────────────────────────────────────────────────────────┘    │
-└─────────────────────────────────────────────────────────────────────────┘
-                                    ▲
-                                    │ WebSocket (实时推送)
-                                    │ 120Hz 心跳 / 数据帧同步
-                                    ▼
-┌─────────────────────────────────────────────────────────────────────────┐
-│                           数据中台服务层 (FastAPI)                        │
-│  ┌─────────────────────────────────────────────────────────────────┐    │
-│  │                      API Gateway (Redis 限流)                    │    │
-│  └─────────────────────────────────────────────────────────────────┘    │
-│                                    │                                    │
-│  ┌───────────────────┬──────────────────────────┬──────────────────┐   │
-│  │    数据采集服务    │      核心业务服务         │    运维管理服务   │   │
-│  │  ┌─────────────┐  │  ┌─────────────────┐    │  ┌─────────────┐ │   │
-│  │  │ 调度中心    │  │  │ WebSocket 服务  │    │  │ 用户管理    │ │   │
-│  │  │ (Celery)   │  │  │ (FastAPI)       │    │  │ (JWT Auth)  │ │   │
-│  │  └─────────────┘  │  └─────────────────┘    │  └─────────────┘ │   │
-│  │  ┌─────────────┐  │  ┌─────────────────┐    │  ┌─────────────┐ │   │
-│  │  │ 采集器池    │  │  │ 数据查询 API    │    │  │ 数据源配置  │ │   │
-│  │  │ (10+源)    │  │  │ (REST)          │    │  │ 监控告警    │ │   │
-│  │  └─────────────┘  │  └─────────────────┘    │  └─────────────┘ │   │
-│  │  ┌─────────────┐  │  ┌─────────────────┐    │  ┌─────────────┐ │   │
-│  │  │ 消息队列    │  │  │ 态势分析引擎    │    │  │ 系统配置    │ │   │
-│  │  │ (Kafka)    │  │  │ (计算/聚合)     │    │  │ 日志审计    │ │   │
-│  │  └─────────────┘  │  └─────────────────┘    │  └─────────────┘ │   │
-│  └───────────────────┴──────────────────────────┴──────────────────┘   │
-└─────────────────────────────────────────────────────────────────────────┘
-                                    ▲
-                                    │ 内部 API 调用
-                                    ▼
-┌─────────────────────────────────────────────────────────────────────────┐
-│                           Web管理端 (React Admin)                        │
-│  ┌─────────────────────────────────────────────────────────────────┐    │
-│  │  登录页 │ 仪表盘 │ 用户管理 │ 数据源配置 │ 任务监控 │ 系统配置   │    │
-│  └─────────────────────────────────────────────────────────────────┘    │
-└─────────────────────────────────────────────────────────────────────────┘
-                                    ▲
-                                    │ PostgreSQL / Redis
-                                    ▼
-┌─────────────────────────────────────────────────────────────────────────┐
-│                           数据存储层                                      │
-│  ┌─────────────┐  ┌─────────────┐  ┌─────────────┐  ┌─────────────┐    │
-│  │ PostgreSQL  │  │ TimescaleDB │  │    Redis    │  │  MinIO      │    │
-│  │ (用户/配置) │  │ (时序数据)  │  │ (缓存/会话) │  │ (文件存储) │    │
-│  └─────────────┘  └─────────────┘  └─────────────┘  └─────────────┘    │
-└─────────────────────────────────────────────────────────────────────────┘
+┌─────────────────────────────────────────────────────────────────────┐
+│                         浏览器展示与运维层                            │
+│  ┌──────────────────────────────┐  ┌──────────────────────────────┐ │
+│  │ Web Earth                    │  │ React 运维台                  │ │
+│  │ frontend/public/earth        │  │ frontend/src                  │ │
+│  │ Three.js 地球 / HUD / 新闻    │  │ 数据源 / 告警 / AI 设置       │ │
+│  │ 国界精度 / 品牌内容配置       │  │ 提示词配置 / 用户与系统配置   │ │
+│  └──────────────────────────────┘  └──────────────────────────────┘ │
+└─────────────────────────────────────────────────────────────────────┘
+                       │ REST / WebSocket
+                       ▼
+┌─────────────────────────────────────────────────────────────────────┐
+│                       FastAPI 业务与编排后端                          │
+│  ┌────────────────────┐ ┌────────────────────┐ ┌─────────────────┐ │
+│  │ 数据 API 与认证     │ │ Earth 新闻增强      │ │ 告警与态势简报   │ │
+│  │ JWT / 权限 / 审计   │ │ 位置推断 / 本地化   │ │ BGP / 告警研判   │ │
+│  └────────────────────┘ └────────────────────┘ └─────────────────┘ │
+│  ┌────────────────────┐ ┌────────────────────┐ ┌─────────────────┐ │
+│  │ 系统运行配置        │ │ 默认提示词注册表     │ │ 未来 Agent Runtime│ │
+│  │ system_settings     │ │ 代码发布 + DB 覆盖  │ │ 工具/证据/工作流   │ │
+│  └────────────────────┘ └────────────────────┘ └─────────────────┘ │
+└─────────────────────────────────────────────────────────────────────┘
+        │ SQLAlchemy / Redis Stream            │ 纯净 LLM 调用
+        ▼                                      ▼
+┌──────────────────────────────┐      ┌──────────────────────────────┐
+│ PostgreSQL / Redis            │      │ aiprovider                   │
+│ 用户、配置、采集结果、新闻     │      │ provider + protocol adapter  │
+│ Stream、缓存、运行状态         │      │ OpenAI / MiniMax / Ollama 等 │
+└──────────────────────────────┘      └──────────────────────────────┘
+        ▲
+        │ 采集器 / 外部数据源
+        ▼
+┌─────────────────────────────────────────────────────────────────────┐
+│ RSS 新闻、BGP 观测、公开数据源、后续 WebSearch/OCR/语音识别等工具      │
+└─────────────────────────────────────────────────────────────────────┘
 ```
 
+架构边界:
+
+- `backend` 负责业务语义、证据收集、提示词选择、AI 任务编排、权限和数据落库。
+- `aiprovider` 只负责把纯净模型请求适配到不同供应商或协议，不内置具体业务提示词。
+- 默认提示词随代码发布并保存在 `backend/app/ai_tasks/default_prompts.json`，运维台可在数据库中保存覆盖值，重置时回到当前代码版本的默认提示词。
+- Earth 新闻保留英文原文，中文展示结果存入 `localizations`，前端默认展示 `zh-CN` 的 `display_title`、`display_summary` 和中文地域/状态文案。
+- Earth LLM 指令、语音识别、多角色态势研判属于后续 Agent Runtime 方向，计划见 [docs/plans/agents-earth-command-runtime-plan.md](/home/ray/dev/linkong/planet/docs/plans/agents-earth-command-runtime-plan.md)。
+
 ## 四大核心要素
 
 | 层级 | 要素 | 描述 |
@@ -87,21 +73,21 @@
 |------|------|------|
 | FastAPI | 0.109+ | Web 框架 |
 | SQLAlchemy | 2.0+ | ORM |
-| Alembic | - | 数据库迁移 |
-| Celery | 5.3+ | 任务队列 |
-| Redis | 7.0+ | 缓存/消息 |
-| Kafka | 3.0+ | 事件流 |
+| uv | - | Python 依赖与命令运行 |
+| Redis | 7.0+ | 缓存、Stream 与运行协调 |
 | PyJWT | - | 认证 |
+| APScheduler / 后台任务 | - | 采集、增强与运行时任务 |
 
 ### 前端 (React Admin)
 
 | 组件 | 用途 |
 |------|------|
 | React 18 | UI 框架 |
-| Ant Design Pro | 管理后台组件 |
+| Tactile UI / Radix primitives / lucide-react | 管理后台组件、基础交互与图标 |
 | Axios | HTTP 客户端 |
 | Socket.io-client | WebSocket 客户端 |
 | ECharts | 统计图表 |
+| Three.js | Earth 3D 地球渲染 |
 | Bun | 前端包管理与脚本运行 |
 
 前端工程统一使用 Bun：
@@ -110,22 +96,16 @@
 - 运行脚本使用 `bun run <script>`
 - 不使用 `npm`、`pnpm`、`yarn`
 
-### 虚幻引擎客户端
+### 大屏与 3D 展示方向
 
-| 组件 | 版本 | 用途 |
-|------|------|------|
-| Unreal Engine 5 | 5.3+ | 3D 渲染引擎 |
-| Cesium for Unreal | 1.5+ | 地理可视化 |
-| Niagara | - | 粒子系统 |
+当前发布优先使用浏览器 Web Earth。UE5 / Cesium for Unreal / Niagara 可作为后续物理大屏方向接入，但不是本地开发闭环的必需组件。
 
 ### 数据库
 
 | 组件 | 用途 |
 |------|------|
 | PostgreSQL 15+ | 关系数据 |
-| TimescaleDB | 时序数据扩展 |
-| Redis 7+ | 缓存/会话 |
-| MinIO | S3 兼容存储 |
+| Redis 7+ | 缓存、Stream、运行状态 |
 
 ### 部署
 
@@ -152,9 +132,9 @@
 | P0 | Epoch AI | 每小时 |
 | P0 | Hugging Face | 每 2 小时 |
 | P0 | GitHub | 每 4 小时 |
-| P0 每日 |
+| P0 | 海底光缆 / IXP / 卫星等基础设施数据 | 每日或按源刷新 |
 | P0 | PeeringDB | 每 2 小时 |
-| P1 | Cloudflare Radar | | TeleGeography | 每小时 |
+| P1 | Cloudflare Radar / TeleGeography | 每小时 |
 | P1 | CAIDA BGPStream | 每 15 分钟 |
 
 ## 项目结构
@@ -166,20 +146,18 @@
 │   │   ├── core/         # 核心配置
 │   │   ├── models/       # 数据模型
 │   │   ├── schemas/      # Pydantic 模型
-│   │   ├── services/     # 业务逻辑
-│   │   └── tasks/        # Celery 任务
+│   │   ├── services/     # 业务逻辑与 AI 任务编排
+│   │   └── ai_tasks/     # 默认提示词与 AI 任务定义
 │   └── tests/
+├── aiprovider/            # 独立模型供应商适配层
 ├── frontend/              # React 管理后台
 │   ├── src/
 │   │   ├── components/   # 组件
 │   │   ├── pages/        # 页面
 │   │   ├── services/     # API 服务
 │   │   └── store/        # 状态管理
-│   └── tests/
-├── unreal/                # UE5 大屏客户端
-│   ├── Content/
-│   ├── Source/
-│   └── Plugins/
+│   ├── public/earth/      # Web Earth 静态应用
+│   └── tests/             # 前端测试
 ├── data/                  # 数据文件
 ├── docs/                  # 文档
 ├── scripts/               # 脚本
@@ -190,11 +168,14 @@
 
 ## 快速启动
 
+入口需要先具备 `zsh`、`curl` 和可访问的软件源。Ubuntu / Ubuntu WSL 上，`init` 会自动检测并补装 Docker Engine、Compose v2 和 Buildx，启动 Docker 服务并配置当前用户的访问权限；需要系统权限时会提示输入 sudo 密码。其他系统请先准备可用的 Docker 环境。
+
 ```bash
-# 新机器首次初始化
-./scripts/bootstrap-dev.sh
-# 会自动安装/检查 uv、bun，并同步 Python/前端依赖
+# 新机器或空项目首次初始化
+./planet.sh init
+# 会先准备 Docker / Compose / Buildx，再安装/检查 uv、bun 并同步 Python/前端依赖
 # 会在缺少时生成 backend/.env、aiprovider/.env、frontend/.env.local
+# 会启动 PostgreSQL/Redis，并创建表、默认数据源和本地默认用户
 
 # 启动前后端服务
 ./planet.sh start
@@ -210,6 +191,9 @@
 
 # 查看服务状态
 ./planet.sh health
+
+# 删除容器、卷、镜像和本地编译状态，执行前需要输入 Y 确认
+./planet.sh destroy
 ```
 
 前端命令约定：
@@ -244,7 +228,7 @@ bun run build
 ./planet.sh start --allow-lan
 ```
 
-这会让前端监听 `0.0.0.0:3000`，后端监听 `0.0.0.0:8000`。
+这会让前端监听 `0.0.0.0:3000`，后端监听 `0.0.0.0:8000`，AI Provider 通过 Docker 发布到 `0.0.0.0:8010`。启动前脚本会检查这三个端口；如果 WSL/Linux 侧无法释放端口，并检测到 Windows 侧 listener 或旧 `portproxy`，会请求管理员 PowerShell 清理。
 
 ### 2. 先确认 WSL 内部服务正常
 
@@ -253,14 +237,16 @@ bun run build
 ```bash
 curl http://localhost:3000
 curl http://localhost:8000/health
-ss -ltnp | grep -E ':3000|:8000'
+curl http://localhost:8010/health
+ss -ltnp | grep -E ':3000|:8000|:8010'
 ```
 
 预期：
 
 - `3000` 返回前端 HTML
 - `8000/health` 返回健康检查 JSON
-- `ss` 中能看到 `0.0.0.0:3000` 和 `0.0.0.0:8000`
+- `8010/health` 返回 AI Provider 健康检查 JSON
+- `ss` 中能看到 `0.0.0.0:3000`、`0.0.0.0:8000` 和 `0.0.0.0:8010`，或 Docker 已发布 `8010`
 
 如果这一步不通，先不要继续做 Windows 转发。
 
@@ -271,42 +257,31 @@ ss -ltnp | grep -E ':3000|:8000'
 ```powershell
 curl http://localhost:3000
 curl http://localhost:8000/health
+curl http://localhost:8010/health
 ```
 
 在常见的 WSL2 开发环境下，Windows 通常可以直接通过 `localhost` 访问 WSL 中的服务。
 
-### 4. 如果需要让局域网设备访问，再做 Windows 端口转发
+### 4. 如果需要让局域网设备访问，清理端口和防火墙
 
-注意：下面的命令必须在“以管理员身份运行”的 PowerShell 中执行。
+`./planet.sh start --allow-lan` 不再启动额外的 Windows 端口转发进程。它直接让开发服务对 `3000` / `8000` / `8010` 开放，并在启动前尝试释放这些端口。端口被 Windows 侧 listener 或旧 `portproxy` 占用时，脚本会请求一次管理员 PowerShell 清理。
 
-先把 Windows 对外网卡上的 `3000` / `8000` 转发到 Windows 本机 `127.0.0.1`：
+如果以前手动配置过持久 `portproxy`，若自动请求被取消，可以手动清理，避免 `iphlpsvc` 继续占用端口：
 
 ```powershell
 netsh interface portproxy delete v4tov4 listenaddress=0.0.0.0 listenport=3000
 netsh interface portproxy delete v4tov4 listenaddress=0.0.0.0 listenport=8000
-
-netsh interface portproxy add v4tov4 listenaddress=0.0.0.0 listenport=3000 connectaddress=127.0.0.1 connectport=3000
-netsh interface portproxy add v4tov4 listenaddress=0.0.0.0 listenport=8000 connectaddress=127.0.0.1 connectport=8000
+netsh interface portproxy delete v4tov4 listenaddress=0.0.0.0 listenport=8010
 ```
 
-再放行 Windows 防火墙：
+脚本会检测 Windows 防火墙是否已放行 `3000` / `8000` / `8010`。如果缺少规则，会触发一次 Windows UAC 管理员 PowerShell 请求来自动创建。若自动请求被取消，也可以手动执行：
 
 ```powershell
 New-NetFirewallRule -DisplayName "WSL Planet 3000" -Direction Inbound -Action Allow -Protocol TCP -LocalPort 3000
 New-NetFirewallRule -DisplayName "WSL Planet 8000" -Direction Inbound -Action Allow -Protocol TCP -LocalPort 8000
+New-NetFirewallRule -DisplayName "WSL Planet 8010" -Direction Inbound -Action Allow -Protocol TCP -LocalPort 8010
 ```
 
-检查转发规则是否生效：
-
-```powershell
-netsh interface portproxy show all
-```
-
-预期能看到：
-
-- `0.0.0.0:3000 -> 127.0.0.1:3000`
-- `0.0.0.0:8000 -> 127.0.0.1:8000`
-
 ### 5. 查 Windows 局域网 IP，并让其他设备访问
 
 在 Windows PowerShell 中执行：
@@ -321,6 +296,8 @@ ipconfig
 
 - `http://<Windows局域网IP>:3000/earth`
 - `http://<Windows局域网IP>:3000/admin`
+- `http://<Windows局域网IP>:8000/health`
+- `http://<Windows局域网IP>:8010/health`
 
 例如：
 
@@ -329,7 +306,7 @@ ipconfig
 ### 6. 常见现象与判断
 
 - WSL 中 `curl localhost:3000` 能通，但 Windows 访问 `WSL 的局域网 IP:3000` 不通：这是正常现象之一，优先验证 Windows 的 `localhost:3000`
-- Windows `localhost:3000` 能通，但局域网设备访问 `Windows 局域网 IP:3000` 不通：通常缺少 `portproxy` 或防火墙放行
+- Windows `localhost:3000` 能通，但局域网设备访问 `Windows 局域网 IP:3000` 不通：通常是 Windows 防火墙、网络配置或旧 `portproxy` 残留
 - `whoami /groups` 中 `S-1-5-32-544` 显示 `deny only`：说明当前 PowerShell 不是提权管理员窗口
 
 ### 7. 本项目一次性验证顺序
@@ -338,14 +315,16 @@ ipconfig
 
 1. WSL 中执行 `curl http://localhost:3000`
 2. WSL 中执行 `curl http://localhost:8000/health`
-3. Windows 中执行 `curl http://localhost:3000`
-4. Windows 中执行 `curl http://localhost:8000/health`
-5. 管理员 PowerShell 配置 `portproxy` 和防火墙
-6. 用手机或其他电脑访问 `http://<Windows局域网IP>:3000/earth`
+3. WSL 中执行 `curl http://localhost:8010/health`
+4. Windows 中执行 `curl http://localhost:3000`
+5. Windows 中执行 `curl http://localhost:8000/health`
+6. Windows 中执行 `curl http://localhost:8010/health`
+7. 按脚本提示完成 Windows 防火墙或端口清理 UAC 请求
+8. 用手机或其他电脑访问 Windows 对外端口，例如 `http://<Windows局域网IP>:3000/earth`
 
 ## 启动容错参数
 
-`planet.sh` 现在为依赖安装、数据库、AI Provider 启动加入了有限次重试，并会在数据库与 `aiprovider` 启动后额外等待 Docker healthcheck。
+`planet.sh` 为依赖安装、数据库、AI Provider 启动提供有限次重试。数据库先检查容器健康，再验证后端实际连接；`aiprovider` 直接以宿主机 `/health` 就绪为准。后端进程退出或应用初始化失败时立即停止等待，避免重复消耗健康检查预算。
 
 可通过环境变量临时调整:
 
@@ -369,14 +348,23 @@ DATABASE_RETRY_INTERVAL=10 \
 - `FRONTEND_HEALTH_CHECK_ATTEMPTS` / `FRONTEND_HEALTH_CHECK_INTERVAL`: 控制前端 HTTP 可访问检查等待次数与间隔，默认 `10` 次、`2` 秒
 - `AI_PROVIDER_HEALTH_CHECK_ATTEMPTS` / `AI_PROVIDER_HEALTH_CHECK_INTERVAL`: 控制 `aiprovider` HTTP 健康检查等待次数与间隔，默认 `10` 次、`2` 秒
 
-## AI 接口预留
+## AI 与智能体接口
 
-项目现在采用“两层”设计:
+项目现在采用“三段式”边界:
 
-- 主后端暴露稳定业务接口: `GET /api/v1/ai/provider/status`、`POST /api/v1/ai/situational-awareness/analyze`
-- 独立 `aiprovider` 服务负责适配具体模型供应商
+- `backend`: 暴露业务接口，负责选择任务提示词、组织证据、调用工具、保存 AI 设置和结果。
+- `aiprovider`: 暴露模型网关接口，只负责 provider / protocol 适配，不写入 BGP、新闻、告警等业务提示词。
+- 模型供应商: OpenAI 兼容、MiniMax、Anthropic、Ollama 或其他兼容网关。
 
-这样前端和业务代码不直接依赖 OpenAI、本地模型网关或其他订阅服务，后续切换部署方式只需要调整环境变量。
+这样前端和业务代码不直接依赖某个模型供应商，后续增加 Agent Runtime、Earth 一键 LLM 指令、语音识别或多角色态势研判时，也可以把业务工作流放在后端，而不是污染模型适配层。
+
+当前已落地的 AI 配置能力:
+
+- 运维台 AI 设置可维护 provider、模型、协议、超时、token 等运行配置。
+- 运维台 AI 设置中的“提示词”页可选择不同功能入口，手动覆盖提示词，并一键重置到默认值。
+- 默认提示词随代码发布，位于 [backend/app/ai_tasks/default_prompts.json](/home/ray/dev/linkong/planet/backend/app/ai_tasks/default_prompts.json)。
+- 覆盖值保存在数据库运行配置中，升级代码后可继续保留现场配置，也可重置到新版本默认提示词。
+- 态势摘要、告警研判、新闻本地化等入口应使用各自任务提示词；调用 `aiprovider` 时只传递当前任务所需的 `prompt` / `system_prompt`。
 
 主后端建议配置:
 
@@ -389,39 +377,33 @@ AI_PROVIDER_TIMEOUT_SECONDS=60
 `aiprovider` 服务建议配置:
 
 ```env
-AI_PROVIDER=openai_compatible
-AI_BASE_URL=https://api.openai.com/v1
+AI_PROVIDER=minimax
+AI_PROVIDER_API=anthropic-messages
+AI_BASE_URL=https://api.minimaxi.com/anthropic
 AI_API_KEY=your_api_key
-AI_MODEL=gpt-4o-mini
+AI_MODEL=MiniMax-M2.7
 AI_TIMEOUT_SECONDS=60
 AI_PROVIDER_SERVICE_TOKEN=change_me
 ```
 
-OpenAI 兼容场景推荐使用:
+推荐映射关系:
 
-- `AI_PROVIDER=openai_compatible`
+- `vLLM` / `LM Studio` / `One API`: `AI_PROVIDER=openai` + `AI_PROVIDER_API=openai-completions`
+- `MiniMax`: `AI_PROVIDER=minimax` + `AI_PROVIDER_API=anthropic-messages`
+- Claude 兼容网关: `AI_PROVIDER=anthropic` + `AI_PROVIDER_API=anthropic-messages`
+- `Ollama`: `AI_PROVIDER=ollama` + `AI_PROVIDER_API=ollama-generate`
 
-Claude 兼容场景推荐使用:
-
-- `AI_PROVIDER=anthropic`
-- `AI_PROVIDER=anthropic_compatible`
-- `AI_PROVIDER=claude_compatible`
-
-Ollama 原生场景推荐使用:
-
-- `AI_PROVIDER=ollama`
-
-比如 MiniMax 或其他 Claude 兼容网关，可以这样配置:
+比如 MiniMax 可以这样配置:
 
 ```env
-AI_PROVIDER=claude_compatible
-AI_BASE_URL=https://your-claude-compatible-endpoint.example.com
+AI_PROVIDER=minimax
+AI_PROVIDER_API=anthropic-messages
+AI_BASE_URL=https://api.minimaxi.com/anthropic
 AI_API_KEY=your_api_key
-AI_MODEL=your-claude-compatible-model
+AI_MODEL=MiniMax-M2.7
 AI_TIMEOUT_SECONDS=60
 AI_MAX_TOKENS=1200
 AI_ANTHROPIC_VERSION=2023-06-01
-AI_PROVIDER_SERVICE_TOKEN=change_me
 ```
 
 如果你要本地直接起模型适配层，项目里已经补了模板:
@@ -429,12 +411,6 @@ AI_PROVIDER_SERVICE_TOKEN=change_me
 - [aiprovider/.env.example](/home/ray/dev/linkong/planet/aiprovider/.env.example)
 - [docker-compose.local-model.yml](/home/ray/dev/linkong/planet/docker-compose.local-model.yml)
 
-推荐映射关系:
-
-- `vLLM` / `LM Studio` / `One API`: `AI_PROVIDER=openai_compatible`
-- `MiniMax` / Claude 兼容网关: `AI_PROVIDER=claude_compatible`
-- `Ollama`: `AI_PROVIDER=ollama`
-
 运行与调用补充:
 
 - `./planet.sh start` 默认会启动 `aiprovider`
@@ -444,11 +420,13 @@ AI_PROVIDER_SERVICE_TOKEN=change_me
 
 详细文档:
 
-- [docs/technical/agents-aiprovider.md](/home/ray/dev/linkong/planet/docs/technical/agents-aiprovider.md)
+- [docs/technical/zh/agents-aiprovider.md](/home/ray/dev/linkong/planet/docs/technical/zh/agents-aiprovider.md)
+- [docs/technical/en/agents-aiprovider.md](/home/ray/dev/linkong/planet/docs/technical/en/agents-aiprovider.md)
 - [aiprovider/README.md](/home/ray/dev/linkong/planet/aiprovider/README.md)
 - [docs/technical/frontend-layout-guidelines.md](/home/ray/dev/linkong/planet/docs/technical/frontend-layout-guidelines.md)
 - [docs/plans/frontend-ai-playground-development-plan.md](/home/ray/dev/linkong/planet/docs/plans/frontend-ai-playground-development-plan.md)
 - [docs/plans/agents-situational-awareness-foundation-plan.md](/home/ray/dev/linkong/planet/docs/plans/agents-situational-awareness-foundation-plan.md)
+- [docs/plans/agents-earth-command-runtime-plan.md](/home/ray/dev/linkong/planet/docs/plans/agents-earth-command-runtime-plan.md)
 
 ## 前端页面布局规范
 

TODO.md

@@ -4,12 +4,16 @@ This file is the active backlog only. Completed history belongs in `docs/CHANGEL
 
 ## Earth
 
-- [ ] High-precision country boundaries: implement the source-faithful static vector tile pipeline described in [Earth High Precision Boundary Tiles Plan](/home/ray/dev/linkong/planet/docs/plans/earth-high-precision-boundary-tiles-plan.md). Highest zoom must preserve trusted source geometry instead of smoothing or hand-drawing borders.
-- [ ] Earth boundary data: acquire or generate auditable China POV geometry for Zangnan, Aksai Chin, Taiwan/Penghu, Diaoyu Dao and affiliated islands, Chiwei Yu, South China Sea islands, and the official dashed maritime claim line before implementing visual changes.
-- [ ] Presentation controller ownership: replace the singleton card fallback in [presentation-controller.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/presentation-controller.js) with a presentation/card token check before BGP/News migrate onto the shared controller, so connectors only attach to their owning card.
+- [ ] Motion Agent v2 hardening: tune the implemented MediaPipe gesture recognizer across camera placements, exercise the UE command/control client, run reconnect and dual-camera soak tests, and continue the v3 calibrated 3D roadmap described in [Motion Agent v2 Control Protocol And 3D Calibration Roadmap](/home/ray/dev/linkong/planet/docs/plans/motion-agent-v2-control-protocol-plan.md).
+- [ ] Earth AI command entry: merge natural-language and speech-triggered LLM commands into the existing Earth search panel as described in [Agent Runtime, Earth LLM Command, And Speech Entry Plan](/home/ray/dev/linkong/planet/docs/plans/agents-earth-command-runtime-plan.md).
+- [ ] Earth action executor: implement safe visualization actions for layer toggles, batch highlights, filters, focus, result panels, and clear-highlight behavior.
+- [ ] Earth entity matching: support stable entity ids and batch matching for Beidou satellites, mainland China compute centers, BGP, news, vessels, and cables.
+- [ ] Replace debug GeoJSON boundary tiles with the real `earth-boundaries-china-pov-v1.pmtiles` production artifact after audited admin-0 / coastline / claim-line sources and the PMTiles toolchain are available.
+- [ ] Import authoritative China POV / coastline / claim-line source packages through the three standard Earth boundary source collectors, then rebuild a versioned PMTiles artifact so highest zoom `8-10` preserves trusted source geometry instead of seed data.
+- [ ] Earth boundary data: acquire or generate auditable China POV geometry for Zangnan, Aksai Chin, Taiwan/Penghu, Diaoyu Dao and affiliated islands, Chiwei Yu, South China Sea islands, Kosovo, Gaza, and the official dashed maritime claim line before implementing final visual changes.
+- [ ] Earth high-resolution basemap tiles: implement the viewport-loaded imagery layer described in [Earth High Resolution Basemap Tiles Plan](/home/ray/dev/linkong/planet/docs/plans/earth-high-resolution-basemap-tiles-plan.md), using high-precision coastline as the alignment reference instead of replacing the globe with one huge texture.
 - [ ] BGP frontend maintainability: split [bgp.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/bgp.js) by responsibility into data loading, marker rendering, overlays, and animation once the current interaction behavior is stable.
 - [ ] Optional BGP marker experiment: evaluate HTML markers for BGP incident/collector points if WebGL marker density or fixed screen-size clickability becomes a real blocker.
-- [ ] Earth news cruise: connect Earth news to the generic cruise queue via a news adapter rather than coupling news-specific sequencing into `main.js`.
 
 ## Compute Centers And Location
 
@@ -27,24 +31,28 @@ This file is the active backlog only. Completed history belongs in `docs/CHANGEL
 
 ## AI Provider And Agents
 
+- [ ] Unified integration config schema: implement the shared low-code schema engine for datasource, AI Provider, Web Search, and OCR configuration described in [Integration Config Schema System Plan](/home/ray/dev/linkong/planet/docs/plans/integration-config-schema-system-plan.md).
+- [ ] AI provider routing: finish the OpenClaw-style provider/model routing refactor described in [AI Provider OpenClaw-Style Routing Plan](/home/ray/dev/linkong/planet/docs/plans/ai-provider-openclaw-style-routing-plan.md), so model-specific transport rules live in provider metadata rather than runtime hardcoding.
+- [ ] AI provider catalog: replace the temporary `model_provider_apis` bridge with structured `models_metadata`, discovery descriptors, and incremental model sync with stale marking.
+- [ ] AI provider connectivity: keep the plug action as lightweight network/auth/model-directory validation only, and keep real generation tests inside Playground or explicit “trial run” actions.
+- [ ] Agent runtime foundation: add auditable agent runs, steps, evidence, proposals, and the Agent operations UI described in [Agent Runtime, Earth LLM Command, And Speech Entry Plan](/home/ray/dev/linkong/planet/docs/plans/agents-earth-command-runtime-plan.md).
+- [ ] Agent tool protocol: add backend JSON tool-call fallback, optional provider-native tool compatibility, tool whitelist validation, and policy-gated proposal application.
+- [ ] Speech/ASR integration for agents: add provider-neutral transcription settings and API, defaulting to Whisper-compatible API providers while keeping text commands usable when ASR is unavailable.
+- [ ] Earth voice wake: add device-local configurable wake-word preferences, microphone fallback states, and post-wake instruction upload for Earth commands.
 - [ ] AI provider compatibility center: move provider/model compatibility rules into a JSON/YAML config read by runtime, instead of continuing to scatter provider-specific branches through Python code.
 - [ ] Provider compatibility coverage: add explicit config for OpenAI, Anthropic, MiniMax, Ollama, Moonshot, DeepSeek, Qwen, GLM, Gemini, OpenRouter, vLLM, LM Studio, and One API.
 - [ ] Compatibility schema: cover adapter type, base URL pattern, auth header, thinking/reasoning defaults, stream path, tool-call capability, multimodal capability, and provider-specific request patches.
 - [ ] BGP geography fallback: evaluate `inetnum` / `inet6num` whois as a finer fallback layer after `prefix_geography`, `OpenGeoFeed`, and RIR delegated data.
 
-## Platform
-
-- [ ] Earth preferences scope: keep current device-local Earth preferences in `localStorage`; only design backend user preferences if account-level synchronization becomes a real product requirement.
-- [ ] System logs: finish a usable Planet log viewing flow that covers backend, frontend, AI Provider, and collector/task logs, with filtering and tailing.
-- [ ] Console UI modernization: gradually replace Ant Design with Planet-owned components and a consistent Tabler Icons based icon system.
-- [ ] Earth live sync: design a unified realtime invalidation path for summary/BGP/satellite updates if polling and current WebSocket channels become insufficient.
-
 ## Archive
 
 Archived items stay here so old context is not lost. Completed items remain checked; obsolete, invalid, or superseded items stay unchecked and include the reason.
 
 ### Completed
 
+- [x] Implemented the high-precision country boundary tile framework from [Earth High Precision Boundary Tiles Plan](/home/ray/dev/linkong/planet/docs/plans/earth-high-precision-boundary-tiles-plan.md): static vector tile builder, versioned seed output, frontend bbox tile loader, debounce, in-flight dedupe, and LRU cache.
+- [x] Added the `pmtiles-mvt` frontend tile provider contract, MVT decoder dependencies, static PMTiles Nginx handling, collector artifact registration, production readiness check, and user operation docs for Earth boundaries.
+- [x] Split Earth boundary ingestion into standard source collectors (`earth_admin0_boundaries`, `earth_coastline`, `earth_claim_lines`) plus the downstream `earth_boundary_tiles` PMTiles builder.
 - [x] Refined BGP observer and anomaly `hover/click` feel.
 - [x] Added BGP anomaly relationship display with cables / regions.
 - [x] Added the Earth BGP activity layer so the map still feels alive when incident density is low.
@@ -59,6 +67,11 @@ Archived items stay here so old context is not lost. Completed items remain chec
 - [x] Added OpenGeoFeed as a high-quality prefix geography override source.
 - [x] Made RIR delegated data a prefix geography fallback rather than the primary source.
 - [x] Added route leak and path instability / flap detectors after the activity layer work.
+- [x] Console UI modernization. Admin is now the only console, legacy Ant Design / Admin Next code paths and dependencies have been removed, and current console UI uses Planet-owned components.
+- [x] Earth news cruise adapter. News cruise now uses `news-cruise-adapter.js` and is wired from `main.js` instead of keeping news-specific sequencing directly in the main Earth loop.
+- [x] Presentation controller ownership. `PresentationController` now guards async ownership through active request identity checks, and current callers pass per-request card targets so stale connector/card work cannot overwrite the active presentation.
+- [x] Earth live sync. Database writes now flow through `earth_data_change_events`, `earth_db_change_listener`, layer adapters, cache invalidation, and the `earth_updates` WebSocket channel; the Earth frontend debounces updates and refreshes BGP, cables, compute centers, satellites, vessels, news, and interactables by layer.
+- [x] System logs. Log sources now normalize into `LogEvent`, Admin supports snapshot filtering plus WebSocket tail/follow, task/detail views deep-link into prefiltered logs, and Admin runtime errors report through the `admin-client` log source.
 
 ### Obsolete Or Superseded
 
@@ -69,3 +82,4 @@ Archived items stay here so old context is not lost. Completed items remain chec
 - [ ] Earth surface material overlay for boundary calibration. Superseded by the high-precision boundary tile plan; future work must use source-faithful boundary/coastline data rather than overlay calibration against the coarse base map.
 - [ ] Hardcoded Earth news source extraction as a standalone task. Superseded by the broader Earth news source configuration and collector plans.
 - [ ] Country-level compute-center fallback placement as a standalone task. Superseded by the shared location pipeline and registry/manual-review backlog.
+- [ ] Earth preferences backend sync scope. Superseded by the current product decision to keep Earth preferences device-local in `localStorage` until account-level synchronization becomes a real requirement.

VERSION

@@ -1 +1 @@
-0.56.0
+0.74.7

agents.md

@@ -1,231 +0,0 @@
-# agents.md
-
-**AI Agent 角色设定。定义 AI 如何行为、沟通和工作。**
-
----
-
-## Identity
-
-You are **opencode**, an AI coding assistant specialized in enterprise-level systems.
-
-You are working on the **智能星球计划 (Intelligent Planet Plan)** - a situational awareness system for data-centric competition featuring:
-- Python FastAPI backend
-- React Admin dashboard
-- Unreal Engine 5 3D visualization
-- Multi-source data collection
-- Polarized 3D large display (4K, 120Hz)
-
----
-
-## Communication Style
-
-### Tone
-- **Professional but concise**
-- Technical accuracy with clarity
-- No unnecessary verbosity
-- Use code comments sparingly (explain **why**, not **what**)
-
-### When Responding
-1. **Answer directly** - 1-3 sentences for simple questions
-2. **Use code blocks** for all code snippets
-3. **Include file:line_number** references when discussing code
-4. **Never** start with "I am an AI assistant" or similar phrases
-5. **Never** add unnecessary preambles/postambles
-
-### Examples
-
-**Good:**
-```
-GPU clusters are stored in `backend/app/services/collectors/top500.py:45`.
-```
-
-**Bad:**
-```
-Based on the information you provided, I can see that the GPU clusters are stored in the top500.py file at line 45. Let me explain more about this...
-```
-
----
-
-## Operational Mode
-
-### Plan Mode (default for complex tasks)
-- Analyze requirements
-- Propose architecture
-- Confirm with user before execution
-- **DO NOT** write code until approved
-
-### Build Mode (after user approval)
-- Execute the approved plan
-- Write code, run commands
-- Verify results
-- Report completion concisely
-
-### Read-Only Mode
-- Analyze code
-- Explain functionality
-- Answer questions
-- **DO NOT** modify files
-
----
-
-## Decision Framework
-
-### When to Ask Before Acting
-- Unclear requirements
-- Multiple implementation approaches
-- Architecture changes
-- Dependency additions
-- Anything that could break existing functionality
-
-### When to Act Directly
-- Clear, approved requirements
-- Routine tasks (linting, formatting, running tests)
-- Following established patterns
-- Fixing obvious bugs
-
-### When to Refuse
-- Malicious code requests
-- Security violations (secrets, credentials)
-- Anything that violates `rules.md`
-
----
-
-## Working Principles
-
-### 1. First Understand, Then Act
-- Read relevant files before editing
-- Understand existing patterns and conventions
-- Follow the code style in the codebase
-- Match the project's technology choices
-
-### 2. Incremental Progress
-- Break large tasks into smaller PRs
-- Complete one feature before starting the next
-- Run tests after each significant change
-- Commit frequently with clear messages
-
-### 3. Quality First
-- Write tests for new functionality
-- Run linters before committing
-- Fix warnings, don't ignore them
-- Document non-obvious decisions
-
-### 4. Communication Clarity
-- Use precise technical language
-- Show relevant code, not explanations
-- Report errors with context
-- Confirm understanding of requirements
-
----
-
-## Code Review Checklist
-
-Before marking a task complete:
-
-- [ ] Code follows `rules.md` style guidelines
-- [ ] Type hints are correct and complete
-- [ ] Error handling is proper (no silent failures)
-- [ ] Tests pass locally
-- [ ] Linting passes
-- [ ] No TODO comments left behind
-- [ ] Documentation updated if needed
-- [ ] Commit message is clear
-
----
-
-## Common Workflows
-
-### Feature Development
-```
-1. Understand requirements
-2. Check existing patterns in codebase
-3. Design solution (brief mental model)
-4. Write code following rules.md
-5. Write/run tests
-6. Lint and format
-7. Commit with clear message
-8. Report completion
-```
-
-### Bug Fix
-```
-1. Reproduce the bug (write failing test)
-2. Locate the source
-3. Fix the issue
-4. Verify test passes
-5. Check for regressions
-6. Commit fix
-```
-
-### Refactoring
-```
-1. Understand current behavior
-2. Design target state
-3. Make incremental changes
-4. Preserve tests
-5. Verify functionality
-6. Clean up dead code
-```
-
----
-
-## Special Considerations
-
-### WebSocket Services
-- Implement heartbeat mechanism (30-second intervals)
-- Handle disconnection gracefully
-- Include camera position in control frames
-- Support both update and full sync modes
-
-### Data Collectors
-- Inherit from BaseCollector
-- Implement fetch() and transform() methods
-- Support incremental updates
-- Handle API changes gracefully
-
-### UE5 Integration
-- Communicate via WebSocket
-- Send data frames at configurable intervals (default 5 min)
-- Support auto-cruise and manual modes
-- Optimize for 4K@120Hz rendering
-
-### Multi-User Security
-- JWT tokens with 15-minute expiration
-- Redis token blacklist for logout
-- Role-based access control (RBAC)
-- Audit logging for all actions
-
----
-
-## Output Format
-
-### When Writing Code
-```python
-# File: backend/app/services/collectors/top500.py
-from typing import List, Dict
-
-class TOP500Collector:
-    async def fetch(self) -> List[Dict]:
-        ...
-```
-
-### When Explaining
-- Use concise paragraphs
-- Include code references
-- No conversational filler
-
-### When Reporting Progress
-- What was done
-- What remains
-- Any blockers
-- Next action
-
----
-
-## Remember
-
-1. **Rules are hard constraints** - follow `rules.md` absolutely
-2. **Context provides understanding** - use `project_context.md` for background
-3. **Role defines behavior** - follow `agents.md` for how to work
-4. **Quality over speed** - Enterprise systems require precision
-5. **Communicate clearly** - Precision in, precision out

aiprovider/Dockerfile

@@ -1,7 +1,8 @@
-# syntax=docker/dockerfile:1.7
+# Use BuildKit's bundled frontend to avoid a separate Docker Hub fetch.
 
 ARG PYTHON_IMAGE=python:3.14-slim
 ARG UV_IMAGE=ghcr.io/astral-sh/uv:latest
+ARG AI_PROVIDER_BUILD_FINGERPRINT=unknown
 
 FROM ${UV_IMAGE} AS uv
 FROM ${PYTHON_IMAGE}
@@ -15,19 +16,25 @@ ENV PYTHONUNBUFFERED=1
 ENV UV_COMPILE_BYTECODE=1
 ENV UV_LINK_MODE=copy
 
+RUN mkdir -p /root/.config/uv
+
 RUN apt-get update && apt-get install -y --no-install-recommends \
     curl \
     && rm -rf /var/lib/apt/lists/*
 
 COPY pyproject.toml uv.lock /app/
 RUN --mount=type=cache,target=/root/.cache/uv \
-    uv sync --frozen --no-dev
+    --mount=type=secret,id=planet_uv_config,target=/root/.config/uv/uv.toml,required=false \
+    uv sync --frozen --only-group aiprovider
 
 COPY aiprovider /app/aiprovider
 
+ARG AI_PROVIDER_BUILD_FINGERPRINT
+LABEL planet.aiprovider.build-fingerprint="${AI_PROVIDER_BUILD_FINGERPRINT}"
+
 EXPOSE 8010
 
 HEALTHCHECK --interval=30s --timeout=5s --retries=3 \
     CMD curl -fsS http://127.0.0.1:8010/health >/dev/null || exit 1
 
-CMD ["uv", "run", "--frozen", "--no-dev", "--project", "/app", "python", "-m", "uvicorn", "aiprovider.main:app", "--host", "0.0.0.0", "--port", "8010"]
+CMD ["/app/.venv/bin/python", "-m", "uvicorn", "aiprovider.main:app", "--host", "0.0.0.0", "--port", "8010"]

aiprovider/README.md

@@ -4,7 +4,7 @@
 
 完整使用说明见:
 
-- [docs/agents/aiprovider.md](/home/ray/dev/linkong/planet/docs/agents/aiprovider.md)
+- [AI Provider 指南](../docs/technical/zh/agents-aiprovider.md)
 
 当前支持:
 
@@ -15,6 +15,7 @@
   - `AI_PROVIDER=ollama`
 - request adapter:
   - `AI_PROVIDER_API=openai-completions`
+  - `AI_PROVIDER_API=openai-responses`
   - `AI_PROVIDER_API=anthropic-messages`
   - `AI_PROVIDER_API=ollama-generate`
 

aiprovider/config.py

@@ -17,9 +17,6 @@ class Settings(BaseSettings):
     AI_HTTP_RETRY_ATTEMPTS: int = 2
     AI_MAX_TOKENS: int = 1200
     AI_ANTHROPIC_VERSION: str = "2023-06-01"
-    AI_ANALYSIS_SYSTEM_PROMPT: str = (
-        "你是态势感知分析助手。请基于输入的上下文、观测与约束，输出结构化、克制、可执行的分析。"
-    )
 
     AI_PROVIDER_SERVICE_TOKEN: str = ""
 

aiprovider/main.py

@@ -45,6 +45,7 @@ def get_provider_service(
     x_ai_model: str | None = Header(default=None),
     x_ai_max_tokens: str | None = Header(default=None),
     x_ai_anthropic_version: str | None = Header(default=None),
+    x_ai_model_provider_apis: str | None = Header(default=None),
 ) -> ProviderService:
     overrides = {
         "provider": x_ai_provider,
@@ -53,6 +54,7 @@ def get_provider_service(
         "api_key": x_ai_api_key,
         "model": x_ai_model,
         "anthropic_version": x_ai_anthropic_version,
+        "model_provider_apis": x_ai_model_provider_apis,
     }
     if x_ai_max_tokens:
         overrides["max_tokens"] = x_ai_max_tokens

aiprovider/provider_service.py

@@ -1,7 +1,9 @@
 from __future__ import annotations
 
 import asyncio
+import json
 from typing import Any
+from uuid import NAMESPACE_URL, uuid4, uuid5
 
 import httpx
 from fastapi import HTTPException, status
@@ -14,7 +16,6 @@ from aiprovider.schemas import (
     SituationalAnalysisResponse,
 )
 
-
 def _normalize_provider(value: str) -> str:
     return (value or "disabled").strip().lower()
 
@@ -62,7 +63,10 @@ class ProviderService:
         self.anthropic_version = str(
             overrides.get("anthropic_version") or settings.AI_ANTHROPIC_VERSION
         )
-        self.system_prompt = settings.AI_ANALYSIS_SYSTEM_PROMPT
+        self.model_provider_apis = self._parse_model_provider_apis(
+            overrides.get("model_provider_apis")
+        )
+        self.session_id = str(uuid4())
 
     def get_status(self) -> AIProviderStatusResponse:
         enabled = self.provider != "disabled"
@@ -93,17 +97,30 @@ class ProviderService:
             )
 
         prompt = self._build_prompt(payload)
+        if payload.context.get("session_id") is not None:
+            self.session_id = str(uuid5(NAMESPACE_URL, f"planet:{payload.context['session_id']}"))
 
-        if self.provider_api == "openai-completions":
-            data = await self._request_openai_compatible(model, prompt)
+        provider_api = self._resolve_model_provider_api(model)
+
+        if provider_api == "openai-completions":
+            data = await self._request_openai_compatible(model, prompt, payload.system_prompt)
             content = self._extract_openai_content(data)
             content_blocks = self._extract_openai_blocks(data)
-        elif self.provider_api == "anthropic-messages":
-            data = await self._request_anthropic_messages(model, prompt, payload.thinking)
+        elif provider_api == "openai-responses":
+            data = await self._request_openai_responses(model, prompt, payload.system_prompt)
+            content_blocks = self._extract_responses_blocks(data)
+            content = "".join(block.text for block in content_blocks if block.text)
+        elif provider_api == "anthropic-messages":
+            data = await self._request_anthropic_messages(
+                model,
+                prompt,
+                payload.thinking,
+                payload.system_prompt,
+            )
             content = self._extract_anthropic_content(data)
             content_blocks = self._extract_anthropic_blocks(data)
-        elif self.provider_api == "ollama-generate":
-            data = await self._request_ollama(model, prompt)
+        elif provider_api == "ollama-generate":
+            data = await self._request_ollama(model, prompt, payload.system_prompt)
             content = self._extract_ollama_content(data)
             content_blocks = self._extract_ollama_blocks(data)
         else:
@@ -128,6 +145,26 @@ class ProviderService:
     def _requires_api_key(self) -> bool:
         return self.provider_api != "ollama-generate"
 
+    def _resolve_model_provider_api(self, model: str) -> str:
+        return self.model_provider_apis.get(model) or self.provider_api
+
+    def _parse_model_provider_apis(self, value: Any) -> dict[str, str]:
+        if isinstance(value, dict):
+            raw = value
+        elif isinstance(value, str) and value.strip():
+            try:
+                parsed = json.loads(value)
+            except json.JSONDecodeError:
+                return {}
+            raw = parsed if isinstance(parsed, dict) else {}
+        else:
+            raw = {}
+        return {
+            str(model): _normalize_provider_api(str(provider_api))
+            for model, provider_api in raw.items()
+            if model and provider_api
+        }
+
     def _build_prompt(self, payload: SituationalAnalysisRequest) -> str:
         sections = [
             f"任务标题:\n{payload.title}",
@@ -139,19 +176,28 @@ class ProviderService:
             sections.append("约束条件:\n" + "\n".join(f"- {item}" for item in payload.constraints))
         if payload.context:
             sections.append(f"附加上下文:\n{payload.context}")
-        sections.append(
-            "请输出: 1) 态势摘要 2) 关键风险 3) 研判依据 4) 建议动作 5) 还缺少的数据。"
-        )
         return "\n\n".join(sections)
 
-    async def _request_openai_compatible(self, model: str, prompt: str) -> dict[str, Any]:
+    def _resolve_system_prompt(self, system_prompt: str | None) -> str | None:
+        resolved = str(system_prompt or "").strip()
+        return resolved or None
+
+    async def _request_openai_compatible(
+        self,
+        model: str,
+        prompt: str,
+        system_prompt: str | None = None,
+    ) -> dict[str, Any]:
+        messages = []
+        resolved_system_prompt = self._resolve_system_prompt(system_prompt)
+        if resolved_system_prompt:
+            messages.append({"role": "system", "content": resolved_system_prompt})
+        messages.append({"role": "user", "content": prompt})
         request_body = {
             "model": model,
-            "messages": [
-                {"role": "system", "content": self.system_prompt},
-                {"role": "user", "content": prompt},
-            ],
+            "messages": messages,
             "temperature": 0.2,
+            "max_tokens": self.max_tokens,
         }
         return await self._post(
             path="/chat/completions",
@@ -162,15 +208,48 @@ class ProviderService:
             request_body=request_body,
         )
 
+    async def _request_openai_responses(
+        self, model: str, prompt: str, system_prompt: str | None = None,
+    ) -> dict[str, Any]:
+        request_body: dict[str, Any] = {
+            "model": model, "input": prompt, "max_output_tokens": self.max_tokens, "store": False,
+        }
+        resolved_system_prompt = self._resolve_system_prompt(system_prompt)
+        if resolved_system_prompt:
+            request_body["instructions"] = resolved_system_prompt
+        return await self._post(
+            path="/responses",
+            headers={"Authorization": f"Bearer {self.api_key}", "Content-Type": "application/json"},
+            request_body=request_body,
+        )
+
+    def _extract_responses_blocks(self, payload: dict[str, Any]) -> list[AIContentBlock]:
+        blocks: list[AIContentBlock] = []
+        for item in payload.get("output") or []:
+            if not isinstance(item, dict):
+                continue
+            if item.get("type") == "message":
+                for part in item.get("content") or []:
+                    if not isinstance(part, dict):
+                        continue
+                    text = part.get("text") or part.get("refusal")
+                    if isinstance(text, str) and text:
+                        blocks.append(AIContentBlock(type="text", text=text))
+            elif item.get("type") == "reasoning":
+                for part in item.get("summary") or []:
+                    if isinstance(part, dict) and isinstance(part.get("text"), str):
+                        blocks.append(AIContentBlock(type="thinking", thinking=part["text"]))
+        return blocks
+
     async def _request_anthropic_messages(
         self,
         model: str,
         prompt: str,
         thinking: dict[str, Any] | None = None,
+        system_prompt: str | None = None,
     ) -> dict[str, Any]:
         request_body = {
             "model": model,
-            "system": self.system_prompt,
             "messages": [
                 {
                     "role": "user",
@@ -185,7 +264,10 @@ class ProviderService:
             "max_tokens": self.max_tokens,
             "temperature": 0.2,
         }
-        resolved_thinking = self._resolve_anthropic_thinking(thinking)
+        resolved_system_prompt = self._resolve_system_prompt(system_prompt)
+        if resolved_system_prompt:
+            request_body["system"] = resolved_system_prompt
+        resolved_thinking = self._resolve_anthropic_thinking(thinking, model)
         if resolved_thinking:
             request_body["thinking"] = resolved_thinking
         if self.provider == "minimax" and self.base_url.endswith("/anthropic"):
@@ -202,8 +284,12 @@ class ProviderService:
             request_body=request_body,
         )
 
-    def _resolve_anthropic_thinking(self, thinking: dict[str, Any] | None) -> dict[str, Any] | None:
+    def _resolve_anthropic_thinking(
+        self, thinking: dict[str, Any] | None, model: str,
+    ) -> dict[str, Any] | None:
         if thinking:
+            if model.casefold() == "minimax-m3" and thinking.get("type") == "enabled":
+                return {"type": "adaptive"}
             return thinking
 
         # OpenClaw treats MiniMax's Anthropic-compatible path specially:
@@ -218,19 +304,28 @@ class ProviderService:
         model: str,
         prompt: str,
         thinking: dict[str, Any] | None = None,
+        system_prompt: str | None = None,
     ) -> dict[str, Any]:
-        return await self._request_anthropic_messages(model, prompt, thinking)
+        return await self._request_anthropic_messages(model, prompt, thinking, system_prompt)
 
-    async def _request_ollama(self, model: str, prompt: str) -> dict[str, Any]:
+    async def _request_ollama(
+        self,
+        model: str,
+        prompt: str,
+        system_prompt: str | None = None,
+    ) -> dict[str, Any]:
         request_body = {
             "model": model,
             "stream": False,
-            "system": self.system_prompt,
             "prompt": prompt,
             "options": {
                 "temperature": 0.2,
+                "num_predict": self.max_tokens,
             },
         }
+        resolved_system_prompt = self._resolve_system_prompt(system_prompt)
+        if resolved_system_prompt:
+            request_body["system"] = resolved_system_prompt
         return await self._post(
             path="/api/generate",
             headers={
@@ -245,6 +340,9 @@ class ProviderService:
         headers: dict[str, str],
         request_body: dict[str, Any],
     ) -> dict[str, Any]:
+        headers = {"User-Agent": "Planet/1.0", **headers}
+        if self.provider == "opencode-go":
+            headers["x-opencode-session"] = self.session_id
         last_error: Exception | None = None
         for attempt in range(1, self.http_retry_attempts + 1):
             try:
@@ -289,13 +387,19 @@ class ProviderService:
         message = choices[0].get("message") or {}
         content = message.get("content")
         if isinstance(content, str):
-            return content
+            if content:
+                return content
+            reasoning_content = message.get("reasoning_content")
+            return reasoning_content if isinstance(reasoning_content, str) else ""
         if isinstance(content, list):
             return "".join(
                 item.get("text", "")
                 for item in content
                 if isinstance(item, dict)
             )
+        reasoning_content = message.get("reasoning_content")
+        if isinstance(reasoning_content, str):
+            return reasoning_content
         return ""
 
     def _extract_openai_blocks(self, payload: dict[str, Any]) -> list[AIContentBlock]:
@@ -306,9 +410,14 @@ class ProviderService:
         message = choices[0].get("message") or {}
         content = message.get("content")
         if isinstance(content, str):
-            return [AIContentBlock(type="text", text=content)]
+            blocks = [AIContentBlock(type="text", text=content)] if content else []
+            reasoning_content = message.get("reasoning_content")
+            if isinstance(reasoning_content, str) and reasoning_content:
+                blocks.append(AIContentBlock(type="thinking", thinking=reasoning_content))
+            return blocks
         if not isinstance(content, list):
-            return []
+            reasoning_content = message.get("reasoning_content")
+            return [AIContentBlock(type="thinking", thinking=reasoning_content)] if isinstance(reasoning_content, str) and reasoning_content else []
 
         blocks: list[AIContentBlock] = []
         for item in content:
@@ -321,7 +430,11 @@ class ProviderService:
                     metadata={k: v for k, v in item.items() if k not in {"type", "text"}},
                 )
             )
+        reasoning_content = message.get("reasoning_content")
+        if isinstance(reasoning_content, str) and reasoning_content:
+            blocks.append(AIContentBlock(type="thinking", thinking=reasoning_content))
         return blocks
+
     def _extract_anthropic_content(self, payload: dict[str, Any]) -> str:
         content = payload.get("content")
         if isinstance(content, str):

aiprovider/schemas.py

@@ -13,10 +13,11 @@ class AIContentBlock(BaseModel):
 
 class SituationalAnalysisRequest(BaseModel):
     title: str = Field(..., min_length=1, max_length=200)
-    objective: str = Field(..., min_length=1, max_length=1000)
+    objective: str = Field(..., min_length=1, max_length=20000)
     context: dict[str, Any] = Field(default_factory=dict)
     observations: list[str] = Field(default_factory=list)
     constraints: list[str] = Field(default_factory=list)
+    system_prompt: str | None = Field(default=None, max_length=8000)
     preferred_model: str | None = Field(default=None, max_length=200)
     thinking: dict[str, Any] | None = None
 

backend/Dockerfile

@@ -1,3 +1,5 @@
+# syntax=docker/dockerfile:1.7
+
 ARG PYTHON_IMAGE=python:3.14-slim
 ARG UV_IMAGE=ghcr.io/astral-sh/uv:latest
 
@@ -14,12 +16,16 @@ ENV UV_COMPILE_BYTECODE=1
 ENV UV_LINK_MODE=copy
 ENV PYTHONPATH=/app/backend
 
+RUN mkdir -p /root/.config/uv
+
 RUN apt-get update && apt-get install -y --no-install-recommends \
     curl \
     && rm -rf /var/lib/apt/lists/*
 
 COPY pyproject.toml uv.lock /app/
-RUN uv sync --frozen --no-dev
+RUN --mount=type=cache,target=/root/.cache/uv \
+    --mount=type=secret,id=planet_uv_config,target=/root/.config/uv/uv.toml,required=false \
+    uv sync --frozen --no-dev
 
 COPY backend /app/backend
 COPY VERSION /app/VERSION

backend/app/ai_tasks/__init__.py

@@ -0,0 +1,2 @@
+"""AI task prompt registry and runtime helpers."""
+

backend/app/ai_tasks/default_prompts.json

@@ -0,0 +1,74 @@
+[
+  {
+    "key": "earth.news.enrich",
+    "label": "Earth 新闻汉化与定位",
+    "group": "Earth 新闻",
+    "version": "2026-05-16.2",
+    "system_prompt": "",
+    "prompt": "Return exactly one strict JSON object with a location object and a localizations object. Infer the most likely physical event location and produce a faithful Simplified Chinese title plus a one-sentence newswire-style Chinese summary based only on the supplied RSS headline, description, source, and date. The summary should read like a concise breaking-news lead, not a label, slogan, or keyword headline."
+  },
+  {
+    "key": "alerts.brief",
+    "label": "系统告警研判",
+    "group": "告警研判",
+    "version": "2026-05-16.1",
+    "system_prompt": "你是告警研判助手。请基于输入的告警事实、上下文与约束，输出结构化、克制、可执行的值班研判；明确区分事实、推断与建议，不要夸大证据不足的风险。",
+    "prompt": "基于当前告警总量、严重度、状态、数据源分布与最近告警摘录，生成一份面向值班人员的简明告警态势简报，突出待处理风险、告警集中点和优先动作。"
+  },
+  {
+    "key": "alerts.situational.brief",
+    "label": "跨模块态势告警研判",
+    "group": "告警研判",
+    "version": "2026-05-16.1",
+    "system_prompt": "你是告警研判助手。请基于输入的告警事实、上下文与约束，输出结构化、克制、可执行的值班研判；明确区分事实、推断与建议，不要夸大证据不足的风险。",
+    "prompt": "综合系统告警、BGP incidents、BGP anomalies 与近期 BGP AI 简报，生成一份面向值班人员的态势告警简报，指出当前最需要关注的风险域、跨模块联动迹象和优先动作。"
+  },
+  {
+    "key": "bgp.brief",
+    "label": "BGP 态势简报",
+    "group": "BGP",
+    "version": "2026-05-16.2",
+    "system_prompt": "你是 BGP 值班分析师。请直接输出面向值班人员的中文 Markdown 简报，只写最终研判内容；不要复述用户需求、提示词、写作计划、字段清单或“我将如何回答”。",
+    "prompt": "基于当前 BGP incidents、anomalies、原始观测事件、观测站覆盖与 prefix geography 证据，生成一份面向操作员的简明态势简报，突出区域热点、观测偏差、当前风险、证据和优先动作。"
+  },
+  {
+    "key": "location.factcheck.normalize",
+    "label": "位置事实核查结构化",
+    "group": "位置解析",
+    "version": "2026-05-16.1",
+    "system_prompt": "",
+    "prompt": "Convert the supplied location factcheck text into exactly one strict JSON object. Extract only facts present in the text or original query."
+  },
+  {
+    "key": "location.factcheck.resolve",
+    "label": "位置事实核查兜底",
+    "group": "位置解析",
+    "version": "2026-05-16.1",
+    "system_prompt": "",
+    "prompt": "Return exactly one JSON object for the most likely physical location. Use only fact-checkable public knowledge; return null fields rather than guessing when evidence is weak."
+  },
+  {
+    "key": "datasource.mapping",
+    "label": "数据源映射生成",
+    "group": "采集配置",
+    "version": "2026-05-16.1",
+    "system_prompt": "",
+    "prompt": "Return only JSON for a deterministic mapping DSL. The JSON must contain source.items_path and fields. Do not include prose or code."
+  },
+  {
+    "key": "credential.guide",
+    "label": "采集器凭据教程",
+    "group": "采集配置",
+    "version": "2026-05-16.1",
+    "system_prompt": "",
+    "prompt": "生成一份中文采集器凭据配置教程。只能根据 context.search_evidence 中的来源生成教程；如果证据不足，明确说明需要以官方页面为准。"
+  },
+  {
+    "key": "ai.connection_test",
+    "label": "AI Provider 连接测试",
+    "group": "运维测试",
+    "version": "2026-05-16.1",
+    "system_prompt": "",
+    "prompt": "Reply OK."
+  }
+]

backend/app/ai_tasks/prompts.py

@@ -0,0 +1,182 @@
+from __future__ import annotations
+
+from dataclasses import dataclass
+from datetime import UTC, datetime
+import json
+from functools import lru_cache
+from pathlib import Path
+from typing import Any
+
+from sqlalchemy import select
+from sqlalchemy.ext.asyncio import AsyncSession
+
+from app.models.system_setting import SystemSetting
+
+AI_PROMPTS_CATEGORY = "ai_prompts"
+DEFAULT_PROMPTS_PATH = Path(__file__).with_name("default_prompts.json")
+
+
+@dataclass(frozen=True)
+class AIPromptDefinition:
+    key: str
+    label: str
+    group: str
+    version: str
+    system_prompt: str
+    prompt: str
+
+
+@dataclass(frozen=True)
+class EffectiveAIPrompt:
+    key: str
+    label: str
+    group: str
+    version: str
+    default_system_prompt: str
+    default_prompt: str
+    system_prompt: str
+    prompt: str
+    is_custom: bool
+    updated_at: str | None = None
+
+
+@lru_cache(maxsize=1)
+def list_prompt_definitions() -> tuple[AIPromptDefinition, ...]:
+    raw_items = json.loads(DEFAULT_PROMPTS_PATH.read_text(encoding="utf-8"))
+    return tuple(
+        AIPromptDefinition(
+            key=str(item["key"]),
+            label=str(item["label"]),
+            group=str(item["group"]),
+            version=str(item["version"]),
+            system_prompt=str(item.get("system_prompt") or ""),
+            prompt=str(item.get("prompt") or ""),
+        )
+        for item in raw_items
+    )
+
+
+def get_prompt_definition(task_key: str) -> AIPromptDefinition:
+    for definition in list_prompt_definitions():
+        if definition.key == task_key:
+            return definition
+    raise KeyError(task_key)
+
+
+async def _get_prompt_setting(db: AsyncSession) -> SystemSetting | None:
+    result = await db.execute(
+        select(SystemSetting).where(SystemSetting.category == AI_PROMPTS_CATEGORY)
+    )
+    return result.scalar_one_or_none()
+
+
+def _normalize_overrides(payload: dict[str, Any] | None) -> dict[str, dict[str, Any]]:
+    raw = (payload or {}).get("overrides")
+    if not isinstance(raw, dict):
+        return {}
+    return {
+        str(key): dict(value)
+        for key, value in raw.items()
+        if isinstance(value, dict)
+    }
+
+
+async def get_prompt_overrides(db: AsyncSession) -> dict[str, dict[str, Any]]:
+    if not hasattr(db, "execute"):
+        return {}
+    setting = await _get_prompt_setting(db)
+    return _normalize_overrides(setting.payload if setting else None)
+
+
+def _effective_prompt(
+    definition: AIPromptDefinition,
+    override: dict[str, Any] | None,
+) -> EffectiveAIPrompt:
+    override = override or {}
+    custom_system = override.get("system_prompt")
+    custom_prompt = override.get("prompt")
+    has_custom_system = isinstance(custom_system, str)
+    has_custom_prompt = isinstance(custom_prompt, str)
+    return EffectiveAIPrompt(
+        key=definition.key,
+        label=definition.label,
+        group=definition.group,
+        version=definition.version,
+        default_system_prompt=definition.system_prompt,
+        default_prompt=definition.prompt,
+        system_prompt=custom_system if has_custom_system else definition.system_prompt,
+        prompt=custom_prompt if has_custom_prompt else definition.prompt,
+        is_custom=has_custom_system or has_custom_prompt,
+        updated_at=str(override.get("updated_at") or "") or None,
+    )
+
+
+async def list_effective_prompts(db: AsyncSession) -> list[EffectiveAIPrompt]:
+    overrides = await get_prompt_overrides(db)
+    return [
+        _effective_prompt(definition, overrides.get(definition.key))
+        for definition in list_prompt_definitions()
+    ]
+
+
+async def get_effective_prompt(db: AsyncSession | None, task_key: str) -> EffectiveAIPrompt:
+    definition = get_prompt_definition(task_key)
+    if db is None:
+        return _effective_prompt(definition, None)
+    overrides = await get_prompt_overrides(db)
+    return _effective_prompt(definition, overrides.get(task_key))
+
+
+async def save_prompt_override(
+    db: AsyncSession,
+    task_key: str,
+    *,
+    system_prompt: str,
+    prompt: str,
+) -> EffectiveAIPrompt:
+    definition = get_prompt_definition(task_key)
+    setting = await _get_prompt_setting(db)
+    payload = dict(setting.payload or {}) if setting else {}
+    overrides = _normalize_overrides(payload)
+    overrides[definition.key] = {
+        "system_prompt": system_prompt,
+        "prompt": prompt,
+        "updated_at": datetime.now(UTC).isoformat().replace("+00:00", "Z"),
+    }
+    payload["overrides"] = overrides
+    if setting is None:
+        setting = SystemSetting(category=AI_PROMPTS_CATEGORY, payload=payload)
+        db.add(setting)
+    else:
+        setting.payload = payload
+    await db.commit()
+    return _effective_prompt(definition, overrides[definition.key])
+
+
+async def reset_prompt_override(db: AsyncSession, task_key: str) -> EffectiveAIPrompt:
+    definition = get_prompt_definition(task_key)
+    setting = await _get_prompt_setting(db)
+    if setting is None:
+        return _effective_prompt(definition, None)
+    payload = dict(setting.payload or {})
+    overrides = _normalize_overrides(payload)
+    overrides.pop(definition.key, None)
+    payload["overrides"] = overrides
+    setting.payload = payload
+    await db.commit()
+    return _effective_prompt(definition, None)
+
+
+def serialize_effective_prompt(prompt: EffectiveAIPrompt) -> dict[str, Any]:
+    return {
+        "key": prompt.key,
+        "label": prompt.label,
+        "group": prompt.group,
+        "version": prompt.version,
+        "default_system_prompt": prompt.default_system_prompt,
+        "default_prompt": prompt.default_prompt,
+        "system_prompt": prompt.system_prompt,
+        "prompt": prompt.prompt,
+        "is_custom": prompt.is_custom,
+        "updated_at": prompt.updated_at,
+    }

backend/app/api/main.py

@@ -6,6 +6,7 @@ from app.api.v1 import (
     datasource_config,
     datasources,
     docs,
+    earth,
     tasks,
     dashboard,
     alerts,
@@ -18,6 +19,7 @@ from app.api.v1 import (
     vessels,
     bgp,
     news,
+    interactables,
     realtime_sources,
     system_control,
     tv,
@@ -34,6 +36,7 @@ api_router.include_router(
 api_router.include_router(datasources.router, prefix="/datasources", tags=["datasources"])
 api_router.include_router(collected_data.router, prefix="/collected", tags=["collected-data"])
 api_router.include_router(docs.router, prefix="/docs", tags=["docs"])
+api_router.include_router(earth.router, prefix="/earth", tags=["earth"])
 api_router.include_router(tasks.router, prefix="/tasks", tags=["tasks"])
 api_router.include_router(dashboard.router, prefix="/dashboard", tags=["dashboard"])
 api_router.include_router(alerts.router, prefix="/alerts", tags=["alerts"])
@@ -51,4 +54,5 @@ api_router.include_router(vessels.router, prefix="/vessels", tags=["vessels"])
 api_router.include_router(bgp.router, prefix="/bgp", tags=["bgp"])
 api_router.include_router(tv.router, prefix="/tv", tags=["tv"])
 api_router.include_router(news.router, prefix="/news", tags=["news"])
+api_router.include_router(interactables.router, prefix="/interactables", tags=["interactables"])
 api_router.include_router(realtime_sources.router, prefix="/realtime-sources", tags=["realtime-sources"])

backend/app/api/v1/ai.py

@@ -3,6 +3,7 @@ from uuid import uuid4
 from fastapi import APIRouter, Depends, HTTPException, Request, Response
 from sqlalchemy.ext.asyncio import AsyncSession
 
+from app.core.logging import get_logger
 from app.core.security import get_current_user
 from app.db.session import get_db
 from app.models.user import User
@@ -47,8 +48,10 @@ from app.services.playground_chat_service import (
     stop_message,
 )
 from app.services.situational_alert_ai_brief import build_situational_alert_brief_request
+from app.services.business_logs import emit_business_log, exception_context
 
 router = APIRouter()
+logger = get_logger(__name__, service="api")
 
 
 @router.get("/provider/status", response_model=AIProviderStatusResponse)
@@ -122,6 +125,16 @@ async def create_playground_message(
     provider_client: AIProviderClient = Depends(get_ai_provider_client),
     db: AsyncSession = Depends(get_db),
 ):
+    await emit_business_log(
+        logger,
+        event="ai.playground.message.create",
+        message="Playground message creation requested",
+        category="ai",
+        service="api",
+        module=__name__,
+        user_id=current_user.id,
+        context={"session_key": payload.session_key, "preset": payload.selected_preset_key},
+    )
     return await create_turn(
         db,
         user_id=current_user.id,
@@ -136,6 +149,16 @@ async def stop_playground_message(
     current_user: User = Depends(get_current_user),
     db: AsyncSession = Depends(get_db),
 ):
+    await emit_business_log(
+        logger,
+        event="ai.playground.message.stop",
+        message="Playground message stop requested",
+        category="ai",
+        service="api",
+        module=__name__,
+        user_id=current_user.id,
+        context={"session_key": payload.session_key, "message_id": payload.message_id},
+    )
     return await stop_message(
         db,
         user_id=current_user.id,
@@ -150,6 +173,16 @@ async def resend_playground_message(
     provider_client: AIProviderClient = Depends(get_ai_provider_client),
     db: AsyncSession = Depends(get_db),
 ):
+    await emit_business_log(
+        logger,
+        event="ai.playground.message.resend",
+        message="Playground message resend requested",
+        category="ai",
+        service="api",
+        module=__name__,
+        user_id=current_user.id,
+        context={"session_key": payload.session_key, "user_message_id": payload.user_message_id},
+    )
     return await resend_turn(
         db,
         user_id=current_user.id,
@@ -214,16 +247,70 @@ async def analyze_bgp_brief(
         anomaly_limit=payload.anomaly_limit,
         collector_limit=payload.collector_limit,
     )
+    await emit_business_log(
+        logger,
+        event="ai.brief.bgp.facts_collected",
+        message="BGP brief facts collected",
+        category="ai",
+        service="api",
+        module=__name__,
+        request_id=request_id,
+        user_id=current_user.id,
+        context={
+            "incident_limit": payload.incident_limit,
+            "anomaly_limit": payload.anomaly_limit,
+            "collector_limit": payload.collector_limit,
+            "fact_count": len(facts or []),
+        },
+    )
     brief_request.preferred_model = payload.preferred_model
     brief_request.thinking = payload.thinking
 
-    analysis = await provider_client.analyze(brief_request, request_id=request_id)
-    return save_bgp_brief_record(
-        analysis,
+    await emit_business_log(
+        logger,
+        event="ai.brief.bgp.start",
+        message="BGP brief AI analysis started",
+        category="ai",
+        service="api",
+        module=__name__,
         request_id=request_id,
-        facts=facts,
-        context=context,
+        user_id=current_user.id,
+        context={"preferred_model": payload.preferred_model},
     )
+    try:
+        analysis = await provider_client.analyze(brief_request, request_id=request_id)
+        record = save_bgp_brief_record(
+            analysis,
+            request_id=request_id,
+            facts=facts,
+            context=context,
+        )
+        await emit_business_log(
+            logger,
+            event="ai.brief.bgp.completed",
+            message="BGP brief AI analysis saved",
+            category="ai",
+            service="api",
+            module=__name__,
+            request_id=request_id,
+            user_id=current_user.id,
+            context={"provider": analysis.provider, "model": analysis.model, "brief_id": record.id},
+        )
+        return record
+    except Exception as exc:
+        await emit_business_log(
+            logger,
+            event="ai.brief.bgp.failed",
+            message="BGP brief AI analysis failed",
+            category="ai",
+            level="error",
+            service="api",
+            module=__name__,
+            request_id=request_id,
+            user_id=current_user.id,
+            context=exception_context(exc, {"preferred_model": payload.preferred_model}),
+        )
+        raise
 
 
 @router.post("/alerts/brief", response_model=AlertBriefResponse)
@@ -242,17 +329,65 @@ async def analyze_alert_brief(
         db,
         alert_limit=payload.alert_limit,
     )
+    await emit_business_log(
+        logger,
+        event="ai.brief.alerts.facts_collected",
+        message="Alert brief facts collected",
+        category="ai",
+        service="api",
+        module=__name__,
+        request_id=request_id,
+        user_id=current_user.id,
+        context={"alert_limit": payload.alert_limit, "fact_count": len(facts or [])},
+    )
     brief_request.preferred_model = payload.preferred_model
     brief_request.thinking = payload.thinking
 
-    analysis = await provider_client.analyze(brief_request, request_id=request_id)
-    return AlertBriefResponse(
-        **analysis.model_dump(),
-        title=brief_request.title,
-        objective=brief_request.objective,
-        facts=facts,
-        context=context,
+    await emit_business_log(
+        logger,
+        event="ai.brief.alerts.start",
+        message="Alert brief AI analysis started",
+        category="ai",
+        service="api",
+        module=__name__,
+        request_id=request_id,
+        user_id=current_user.id,
+        context={"preferred_model": payload.preferred_model},
     )
+    try:
+        analysis = await provider_client.analyze(brief_request, request_id=request_id)
+        await emit_business_log(
+            logger,
+            event="ai.brief.alerts.completed",
+            message="Alert brief AI analysis completed",
+            category="ai",
+            service="api",
+            module=__name__,
+            request_id=request_id,
+            user_id=current_user.id,
+            context={"provider": analysis.provider, "model": analysis.model},
+        )
+        return AlertBriefResponse(
+            **analysis.model_dump(),
+            title=brief_request.title,
+            objective=brief_request.objective,
+            facts=facts,
+            context=context,
+        )
+    except Exception as exc:
+        await emit_business_log(
+            logger,
+            event="ai.brief.alerts.failed",
+            message="Alert brief AI analysis failed",
+            category="ai",
+            level="error",
+            service="api",
+            module=__name__,
+            request_id=request_id,
+            user_id=current_user.id,
+            context=exception_context(exc, {"preferred_model": payload.preferred_model}),
+        )
+        raise
 
 
 @router.post("/situational-alerts/brief", response_model=SituationalAlertBriefResponse)
@@ -268,14 +403,62 @@ async def analyze_situational_alert_brief(
     response.headers["X-Request-ID"] = request_id
 
     brief_request, facts, context = await build_situational_alert_brief_request(db)
+    await emit_business_log(
+        logger,
+        event="ai.brief.situational_alerts.facts_collected",
+        message="Situational alert brief facts collected",
+        category="ai",
+        service="api",
+        module=__name__,
+        request_id=request_id,
+        user_id=current_user.id,
+        context={"fact_count": len(facts or [])},
+    )
     brief_request.preferred_model = payload.preferred_model
     brief_request.thinking = payload.thinking
 
-    analysis = await provider_client.analyze(brief_request, request_id=request_id)
-    return SituationalAlertBriefResponse(
-        **analysis.model_dump(),
-        title=brief_request.title,
-        objective=brief_request.objective,
-        facts=facts,
-        context=context,
+    await emit_business_log(
+        logger,
+        event="ai.brief.situational_alerts.start",
+        message="Situational alert brief AI analysis started",
+        category="ai",
+        service="api",
+        module=__name__,
+        request_id=request_id,
+        user_id=current_user.id,
+        context={"preferred_model": payload.preferred_model},
     )
+    try:
+        analysis = await provider_client.analyze(brief_request, request_id=request_id)
+        await emit_business_log(
+            logger,
+            event="ai.brief.situational_alerts.completed",
+            message="Situational alert brief AI analysis completed",
+            category="ai",
+            service="api",
+            module=__name__,
+            request_id=request_id,
+            user_id=current_user.id,
+            context={"provider": analysis.provider, "model": analysis.model},
+        )
+        return SituationalAlertBriefResponse(
+            **analysis.model_dump(),
+            title=brief_request.title,
+            objective=brief_request.objective,
+            facts=facts,
+            context=context,
+        )
+    except Exception as exc:
+        await emit_business_log(
+            logger,
+            event="ai.brief.situational_alerts.failed",
+            message="Situational alert brief AI analysis failed",
+            category="ai",
+            level="error",
+            service="api",
+            module=__name__,
+            request_id=request_id,
+            user_id=current_user.id,
+            context=exception_context(exc, {"preferred_model": payload.preferred_model}),
+        )
+        raise

backend/app/api/v1/auth.py

@@ -4,6 +4,7 @@ from sqlalchemy.ext.asyncio import AsyncSession
 from sqlalchemy import text
 
 from app.core.config import settings
+from app.core.enums import OtpPurpose, UserRole
 from app.core.logging import get_logger
 from app.core.security import (
     create_access_token,
@@ -170,7 +171,7 @@ async def get_me(current_user: User = Depends(get_current_user)):
     }
 
 
-async def _send_code_or_raise(db: AsyncSession, email: str, code: str, purpose: str) -> None:
+async def _send_code_or_raise(db: AsyncSession, email: str, code: str, purpose: OtpPurpose) -> None:
     try:
         await send_verification_email(db, to=email, code=code, purpose=purpose)
     except EmailNotConfiguredError as exc:
@@ -207,7 +208,7 @@ async def register(payload: UserRegister, db: AsyncSession = Depends(get_db)):
         username=payload.username,
         email=payload.email,
         password_hash=get_password_hash(payload.password),
-        role="viewer",
+        role=UserRole.VIEWER.value,
         is_active=True,
         email_verified=False,
     )
@@ -215,13 +216,13 @@ async def register(payload: UserRegister, db: AsyncSession = Depends(get_db)):
     await db.commit()
 
     try:
-        code = otp.issue_code(payload.email, "register")
+        code = otp.issue_code(payload.email, OtpPurpose.REGISTER)
     except otp.OtpResendRateLimited as exc:
         raise HTTPException(
             status_code=status.HTTP_429_TOO_MANY_REQUESTS,
             detail={"code": exc.code, "retry_after_seconds": exc.retry_after_seconds},
         ) from exc
-    await _send_code_or_raise(db, payload.email, code, "register")
+    await _send_code_or_raise(db, payload.email, code, OtpPurpose.REGISTER)
     return {"status": "pending_verification", "email": payload.email}
 
 
@@ -266,7 +267,7 @@ async def resend_code(payload: ResendCodeRequest, db: AsyncSession = Depends(get
     if user is None:
         # Avoid email enumeration; pretend success.
         return {"status": "ok"}
-    if payload.purpose == "register" and user.email_verified:
+    if payload.purpose is OtpPurpose.REGISTER and user.email_verified:
         raise HTTPException(
             status_code=status.HTTP_409_CONFLICT,
             detail={"code": "ALREADY_VERIFIED"},
@@ -289,12 +290,17 @@ async def forgot_password(payload: ForgotPasswordRequest, db: AsyncSession = Dep
         # Don't leak whether an email is registered.
         return {"status": "ok"}
     try:
-        code = otp.issue_code(payload.email, "reset_password")
+        code = otp.issue_code(payload.email, OtpPurpose.RESET_PASSWORD)
     except otp.OtpResendRateLimited:
         # Silently accept; the user can retry after the cooldown.
         return {"status": "ok"}
     try:
-        await send_verification_email(db, to=payload.email, code=code, purpose="reset_password")
+        await send_verification_email(
+            db,
+            to=payload.email,
+            code=code,
+            purpose=OtpPurpose.RESET_PASSWORD,
+        )
     except EmailNotConfiguredError as exc:
         raise HTTPException(
             status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
@@ -304,7 +310,11 @@ async def forgot_password(payload: ForgotPasswordRequest, db: AsyncSession = Dep
         logger.warning_event(
             "SMTP send failed",
             event="auth.email.send_failed",
-            context={"email": payload.email, "purpose": "reset_password", "error": str(exc)},
+            context={
+                "email": payload.email,
+                "purpose": OtpPurpose.RESET_PASSWORD.value,
+                "error": str(exc),
+            },
         )
     return {"status": "ok"}
 
@@ -318,7 +328,7 @@ async def reset_password(payload: ResetPasswordRequest, db: AsyncSession = Depen
             detail={"code": "OTP_INVALID"},
         )
     try:
-        otp.verify_code(payload.email, "reset_password", payload.code)
+        otp.verify_code(payload.email, OtpPurpose.RESET_PASSWORD, payload.code)
     except otp.OtpExpired as exc:
         raise HTTPException(
             status_code=status.HTTP_410_GONE,

backend/app/api/v1/bgp.py

@@ -341,6 +341,7 @@ async def collect_bgp_collector_location(
                 provider_client=provider_client,
                 query=query,
                 entity_type="bgp_collector",
+                db=db,
                 attempted_queries=attempted_queries,
                 search_evidence=search_result.evidence,
             )

backend/app/api/v1/collected_data.py

@@ -102,7 +102,12 @@ def build_search_rank_sql(search: Optional[str]) -> str:
     """
 
 
-def serialize_collected_row(row, source_name_map: dict[str, str] | None = None) -> dict:
+def serialize_collected_row(
+    row,
+    source_name_map: dict[str, str] | None = None,
+    *,
+    include_metadata: bool = True,
+) -> dict:
     metadata = row[7]
     source = row[1]
     return {
@@ -120,7 +125,7 @@ def serialize_collected_row(row, source_name_map: dict[str, str] | None = None)
         "longitude": get_metadata_field(metadata, "longitude"),
         "value": get_metadata_field(metadata, "value"),
         "unit": get_metadata_field(metadata, "unit"),
-        "metadata": metadata,
+        "metadata": metadata if include_metadata else None,
         "cores": get_metadata_field(metadata, "cores"),
         "rmax": get_metadata_field(metadata, "rmax"),
         "rpeak": get_metadata_field(metadata, "rpeak"),
@@ -145,6 +150,7 @@ async def list_collected_data(
     search: Optional[str] = Query(None, description="搜索名称"),
     page: int = Query(1, ge=1, description="页码"),
     page_size: int = Query(20, ge=1, le=100, description="每页数量"),
+    include_metadata: bool = Query(True, description="是否返回完整 metadata 字段"),
     current_user: User = Depends(get_current_user),
     db: AsyncSession = Depends(get_db),
 ):
@@ -201,7 +207,7 @@ async def list_collected_data(
 
     data = []
     for row in rows:
-        data.append(serialize_collected_row(row[:11], source_name_map))
+        data.append(serialize_collected_row(row[:11], source_name_map, include_metadata=include_metadata))
 
     return {
         "total": total,

backend/app/api/v1/datasource_config.py

@@ -5,7 +5,7 @@ from datetime import datetime
 import base64
 import json
 import re
-from fastapi import APIRouter, Depends, HTTPException, Query
+from fastapi import APIRouter, Depends, HTTPException, Query, Request, status
 from sqlalchemy import delete, select, func
 from sqlalchemy.ext.asyncio import AsyncSession
 from pydantic import BaseModel, Field
@@ -13,6 +13,7 @@ import httpx
 
 from app.core.target_schema_registry import get_target_schema, list_target_schemas
 from app.core.datasource_defaults import DEFAULT_DATASOURCES
+from app.core.enums import AuthType, MappingValidationStatus, UserRole
 from app.db.session import get_db
 from app.models.user import User
 from app.models.datasource_config import DataSourceConfig
@@ -22,6 +23,7 @@ from app.models.vessel import AISRawObservation, AISSourceHealth
 from app.core.security import get_current_user
 from app.core.cache import cache
 from app.core.time import to_iso8601_utc
+from app.ai_tasks.prompts import get_effective_prompt
 from app.schemas.ai import SituationalAnalysisRequest
 from app.services.ai_client import AIProviderClient, get_ai_provider_client
 from app.services.datasource_mapping import (
@@ -33,7 +35,6 @@ from app.services.datasource_mapping import (
 )
 from app.services.custom_datasource_runtime import (
     CustomDatasourceRuntimeError,
-    fetch_rest_payload,
     get_custom_stream_status,
     run_mapped_rest_config,
     run_mapped_websocket_config,
@@ -42,21 +43,87 @@ from app.services.custom_datasource_runtime import (
     test_websocket_config,
 )
 from app.services.datasource_connectivity import (
+    _resolve_aisstream_api_key,
+    _resolve_spacetrack_credentials_with_override,
     get_builtin_connection_status,
     save_connectivity_success,
     strip_connectivity_validation,
     test_builtin_connectivity,
 )
+from app.services.barentswatch import resolve_barentswatch_config
+from app.services.persistent_logs import record_audit_log
 
 router = APIRouter()
 
+DATASOURCE_MAPPING_PROMPT_KEY = "datasource.mapping"
+SECRET_REVEAL_ROLES = {UserRole.ADMIN.value, UserRole.SUPER_ADMIN.value}
+
+
+def _user_role_value(user: User) -> str:
+    role = getattr(user, "role", "")
+    return str(getattr(role, "value", role) or "").lower()
+
+
+def _user_display_name(user: User) -> str:
+    return str(getattr(user, "username", None) or getattr(user, "email", None) or getattr(user, "id", ""))
+
+
+async def _record_datasource_secret_reveal(
+    *,
+    current_user: User,
+    request: Request,
+    target_id: str,
+    result: str,
+    details: dict[str, Any],
+) -> None:
+    await record_audit_log(
+        action="datasource_config.secret.reveal",
+        actor_id=getattr(current_user, "id", None),
+        actor_name=_user_display_name(current_user),
+        target_type="datasource_config_secret",
+        target_id=target_id,
+        result=result,
+        ip=request.client.host if request.client else None,
+        details=details,
+    )
+
+
+async def _ensure_datasource_secret_reveal_allowed(
+    current_user: User,
+    request: Request,
+    target_id: str,
+    details: dict[str, Any],
+) -> None:
+    if _user_role_value(current_user) in SECRET_REVEAL_ROLES:
+        return
+    await _record_datasource_secret_reveal(
+        current_user=current_user,
+        request=request,
+        target_id=target_id,
+        result="denied",
+        details={**details, "role": _user_role_value(current_user)},
+    )
+    raise HTTPException(
+        status_code=status.HTTP_403_FORBIDDEN,
+        detail="Only administrators can reveal datasource credentials",
+    )
+
+def _default_builtin_config(name: str) -> dict[str, Any]:
+    return {"timeout": 30, "retry": 3}
+
+
+def _default_builtin_source_type(name: str) -> str:
+    if name == "aisstream_vessels":
+        return "websocket"
+    return "http"
+
 
 class DataSourceConfigCreate(BaseModel):
     name: str = Field(..., min_length=1, max_length=100)
     description: Optional[str] = None
     source_type: str = Field(..., description="rest, websocket, http, api, database")
     endpoint: str = Field(..., max_length=500)
-    auth_type: str = Field(default="none", description="none, bearer, api_key, basic")
+    auth_type: AuthType = Field(default=AuthType.NONE, description="none, bearer, api_key, basic")
     auth_config: dict = Field(default={})
     headers: dict = Field(default={})
     config: dict = Field(default={"timeout": 30, "retry": 3})
@@ -67,7 +134,7 @@ class DataSourceConfigUpdate(BaseModel):
     description: Optional[str] = None
     source_type: Optional[str] = None
     endpoint: Optional[str] = Field(None, max_length=500)
-    auth_type: Optional[str] = None
+    auth_type: Optional[AuthType] = None
     auth_config: Optional[dict] = None
     headers: Optional[dict] = None
     config: Optional[dict] = None
@@ -142,7 +209,7 @@ class MappingTemplateCreate(BaseModel):
     mapping_json: dict
     sample_payload: Any | None = None
     sample_payload_hash: Optional[str] = None
-    validation_status: str = Field(default="draft", pattern="^(draft|valid|invalid)$")
+    validation_status: MappingValidationStatus = MappingValidationStatus.DRAFT
     is_active: bool = False
 
 
@@ -151,7 +218,7 @@ class MappingTemplateUpdate(BaseModel):
     mapping_json: Optional[dict] = None
     sample_payload: Any | None = None
     sample_payload_hash: Optional[str] = None
-    validation_status: Optional[str] = Field(default=None, pattern="^(draft|valid|invalid)$")
+    validation_status: Optional[MappingValidationStatus] = None
     is_active: Optional[bool] = None
 
 
@@ -364,7 +431,7 @@ async def list_all_datasources(
     db: AsyncSession = Depends(get_db),
 ):
     """List all data sources: YAML defaults + DB overrides"""
-    from app.core.data_sources import COLLECTOR_URL_KEYS, get_data_sources_config
+    from app.core.data_sources import get_data_sources_config
 
     config = get_data_sources_config()
 
@@ -372,38 +439,144 @@ async def list_all_datasources(
     db_configs = {c.name: c for c in db_query.scalars().all()}
 
     result = []
-    for name, yaml_key in COLLECTOR_URL_KEYS.items():
+    for name, metadata in DEFAULT_DATASOURCES.items():
         yaml_url = config.get_yaml_url(name)
         db_config = db_configs.get(name)
+        default_config = _default_builtin_config(name)
+        default_url = yaml_url
+        db_auth_config = db_config.auth_config or {} if db_config else {}
 
         result.append(
             {
                 "name": name,
-                "default_url": yaml_url,
-                "endpoint": db_config.endpoint if db_config else yaml_url,
+                "requires_credentials": bool(metadata.get("requires_credentials", False)),
+                "credential_provider": metadata.get("credential_provider"),
+                "credential_status": metadata.get("credential_status", "none"),
+                "default_url": default_url,
+                "endpoint": db_config.endpoint if db_config else default_url,
                 "is_overridden": db_config is not None and db_config.endpoint != yaml_url
-                if yaml_url
+                if default_url
                 else db_config is not None,
                 "is_active": db_config.is_active if db_config else True,
-                "source_type": db_config.source_type if db_config else "http",
+                "source_type": db_config.source_type if db_config else _default_builtin_source_type(name),
                 "auth_type": db_config.auth_type if db_config else "none",
+                "auth_config": {
+                    "client_id": db_auth_config.get("client_id") or "",
+                    "username": db_auth_config.get("username") or "",
+                    "key_name": db_auth_config.get("key_name") or db_auth_config.get("param_name") or "",
+                    "param_name": db_auth_config.get("param_name") or db_auth_config.get("key_name") or "",
+                    "location": db_auth_config.get("location") or db_auth_config.get("in") or "",
+                    "in": db_auth_config.get("in") or db_auth_config.get("location") or "",
+                },
                 "auth_configured": {
-                    "api_key": bool((db_config.auth_config or {}).get("api_key"))
-                    if db_config
-                    else False,
+                    "api_key": bool(db_auth_config.get("api_key")),
+                    "client_id": bool(db_auth_config.get("client_id")),
+                    "client_secret": bool(db_auth_config.get("client_secret")),
+                    "username": bool(db_auth_config.get("username")),
+                    "password": bool(db_auth_config.get("password")),
                 },
                 "headers": db_config.headers if db_config else {},
-                "config": strip_connectivity_validation(db_config.config if db_config else {}),
+                "config": strip_connectivity_validation(db_config.config if db_config else default_config),
                 "config_id": db_config.id if db_config else None,
                 "description": db_config.description
                 if db_config
-                else f"Data source from YAML: {yaml_key}",
+                else f"内置采集器默认配置：{metadata.get('display_name') or metadata.get('name') or name}",
             }
         )
 
     return {"total": len(result), "data": result}
 
 
+@router.get("/configs/secrets")
+async def reveal_builtin_config_secrets(
+    request: Request,
+    name: str = Query(..., min_length=1),
+    current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    """Reveal configured built-in datasource credentials for admin editing."""
+    source = name.strip()
+    metadata = DEFAULT_DATASOURCES.get(source)
+    if not metadata or not metadata.get("requires_credentials"):
+        raise HTTPException(status_code=404, detail="Credentialed datasource config not found")
+
+    provider = str(metadata.get("credential_provider") or "")
+    target_id = f"datasource_config:{source}"
+    await _ensure_datasource_secret_reveal_allowed(
+        current_user,
+        request,
+        target_id,
+        {"source": source, "provider": provider},
+    )
+
+    result = await db.execute(select(DataSourceConfig).where(DataSourceConfig.name == source))
+    record = result.scalar_one_or_none()
+    auth_config = dict(record.auth_config or {}) if record else {}
+    payload: dict[str, Any] = {
+        "name": source,
+        "provider": provider,
+    }
+    details: dict[str, Any] = {"source": source, "provider": provider}
+
+    if provider == "barentswatch":
+        resolved = await resolve_barentswatch_config(db)
+        client_id = str(auth_config.get("client_id") or resolved.client_id or "")
+        client_secret = str(auth_config.get("client_secret") or resolved.client_secret or "")
+        source_label = "datasource_config" if auth_config.get("client_id") or auth_config.get("client_secret") else resolved.credential_source
+        payload.update(
+            {
+                "client_id": client_id,
+                "client_secret": client_secret,
+                "client_id_source": source_label if client_id else "missing",
+                "client_secret_source": source_label if client_secret else "missing",
+            }
+        )
+        details.update(
+            {
+                "client_id_configured": bool(client_id),
+                "client_secret_configured": bool(client_secret),
+                "credential_source": source_label,
+            }
+        )
+    elif provider == "aisstream":
+        api_key, api_key_source = await _resolve_aisstream_api_key(db)
+        payload.update({"api_key": api_key, "api_key_source": api_key_source})
+        details.update({"api_key_configured": bool(api_key), "api_key_source": api_key_source})
+    elif provider == "spacetrack":
+        if auth_config.get("username") or auth_config.get("password"):
+            username = str(auth_config.get("username") or "")
+            password = str(auth_config.get("password") or "")
+            credential_source = "datasource_config"
+        else:
+            username, password, credential_source = _resolve_spacetrack_credentials_with_override()
+        payload.update(
+            {
+                "username": username,
+                "password": password,
+                "username_source": credential_source if username else "missing",
+                "password_source": credential_source if password else "missing",
+            }
+        )
+        details.update(
+            {
+                "username_configured": bool(username),
+                "password_configured": bool(password),
+                "credential_source": credential_source,
+            }
+        )
+    else:
+        raise HTTPException(status_code=400, detail="Datasource credential provider is not supported")
+
+    await _record_datasource_secret_reveal(
+        current_user=current_user,
+        request=request,
+        target_id=target_id,
+        result="success",
+        details=details,
+    )
+    return payload
+
+
 @router.get("/configs/{config_id}")
 async def get_config(
     config_id: int,
@@ -744,6 +917,7 @@ async def get_datasource_target_schemas(
 @router.post("/mappings/propose")
 async def propose_datasource_mapping(
     payload: MappingProposeRequest,
+    db: AsyncSession = Depends(get_db),
     current_user: User = Depends(get_current_user),
     ai_client: AIProviderClient = Depends(get_ai_provider_client),
 ):
@@ -757,14 +931,12 @@ async def propose_datasource_mapping(
     generated_by = "heuristic"
     if payload.use_ai:
         try:
+            prompt = await get_effective_prompt(db, DATASOURCE_MAPPING_PROMPT_KEY)
             response = await ai_client.analyze(
                 SituationalAnalysisRequest(
                     title=f"Generate datasource mapping for {schema.key}",
-                    objective=(
-                        "Return only JSON for a deterministic mapping DSL. "
-                        "The JSON must contain source.items_path and fields. "
-                        "Do not include prose or code."
-                    ),
+                    objective=prompt.prompt,
+                    system_prompt=prompt.system_prompt or None,
                     context={
                         "target_schema": schema.to_dict(),
                         "sample_payload": redacted_sample,

backend/app/api/v1/datasources.py

@@ -1,4 +1,3 @@
-import asyncio
 from datetime import datetime, timedelta, timezone
 from typing import Optional
 
@@ -7,6 +6,8 @@ from pydantic import BaseModel, Field
 from sqlalchemy import func, or_, select, text
 from sqlalchemy.ext.asyncio import AsyncSession
 
+from app.core.logging import get_logger
+from app.core.enums import JobStatus, SnapshotStatus
 from app.core.time import to_iso8601_utc
 from app.core.security import get_current_user
 from app.core.data_sources import get_data_sources_config
@@ -19,17 +20,27 @@ from app.models.datasource_config import DataSourceConfig
 from app.models.task import CollectionTask
 from app.models.user import User
 from app.models.vessel import AISRawObservation
-from app.services.vessel_ais_aggregation import VESSEL_AIS_SCHEMA
 from app.services.scheduler import (
-    cancel_running_collector_now,
-    get_latest_task_id_for_datasource,
-    run_collector_now,
     sync_datasource_job,
 )
+from app.services.data_jobs import (
+    JOB_STATUS_CANCELLING,
+    JOB_STATUS_QUEUED,
+    JOB_STATUS_RUNNING,
+    JOB_TYPE_CLEAR_CACHE,
+    JOB_TYPE_CLEAR_DATA,
+    JOB_TYPE_COLLECT,
+    enqueue_datasource_job,
+    get_active_datasource_job,
+    request_cancel_datasource_task,
+)
+from app.services.business_logs import emit_business_log
 
 router = APIRouter()
+logger = get_logger(__name__, service="api")
 STALE_RUNNING_TASK_TIMEOUT_MINUTES = 90
 
+
 PRODUCT_SOURCE_KEYWORDS: tuple[tuple[str, tuple[str, ...]], ...] = (
     ("vessels", ("vessel", "ais")),
     ("cables", ("cable", "landing", "telegeography", "arcgis", "fao")),
@@ -113,7 +124,7 @@ async def _load_latest_running_tasks(
             _task_rank_column(CollectionTask.started_at),
         )
         .where(CollectionTask.datasource_id.in_(datasource_ids))
-        .where(CollectionTask.status == "running")
+        .where(CollectionTask.status.in_((JOB_STATUS_QUEUED, JOB_STATUS_RUNNING, JOB_STATUS_CANCELLING)))
         .subquery()
     )
     result = await db.execute(
@@ -124,10 +135,10 @@ async def _load_latest_running_tasks(
     return {task.datasource_id: task for task in result.scalars().all()}
 
 
-async def _load_latest_task_ids(
+async def _load_latest_tasks(
     db: AsyncSession,
     datasource_ids: list[int],
-) -> dict[int, int]:
+) -> dict[int, CollectionTask]:
     if not datasource_ids:
         return {}
 
@@ -137,22 +148,25 @@ async def _load_latest_task_ids(
             CollectionTask.datasource_id.label("datasource_id"),
             func.row_number().over(
                 partition_by=CollectionTask.datasource_id,
-                order_by=CollectionTask.id.desc(),
+                order_by=(CollectionTask.created_at.desc().nullslast(), CollectionTask.id.desc()),
             ).label("row_num"),
         )
         .where(CollectionTask.datasource_id.in_(datasource_ids))
         .subquery()
     )
     result = await db.execute(
-        select(ranked_tasks.c.datasource_id, ranked_tasks.c.task_id)
+        select(CollectionTask)
+        .join(ranked_tasks, CollectionTask.id == ranked_tasks.c.task_id)
         .where(ranked_tasks.c.row_num == 1)
     )
-    return {datasource_id: task_id for datasource_id, task_id in result.all()}
+    return {task.datasource_id: task for task in result.scalars().all()}
 
 
 async def _load_collected_record_counts(
     db: AsyncSession,
     sources: list[str],
+    *,
+    exact_vessel_counts: bool = False,
 ) -> dict[str, int]:
     if not sources:
         return {}
@@ -173,14 +187,46 @@ async def _load_collected_record_counts(
         or "ais" in source
     ]
     if vessel_sources:
-        raw_result = await db.execute(
-            select(AISRawObservation.source, func.count(AISRawObservation.id))
-            .where(AISRawObservation.target_schema == VESSEL_AIS_SCHEMA)
-            .where(AISRawObservation.source.in_(vessel_sources))
-            .group_by(AISRawObservation.source)
+        if exact_vessel_counts:
+            exact_result = await db.execute(
+                select(AISRawObservation.source, func.count(AISRawObservation.id))
+                .where(AISRawObservation.source.in_(vessel_sources))
+                .group_by(AISRawObservation.source)
+            )
+            for source, count in exact_result.all():
+                counts[source] = max(counts.get(source, 0), int(count or 0))
+            return counts
+
+        # AIS raw observations can be tens of millions of rows. Use planner
+        # statistics for the datasource list instead of blocking page load on
+        # source-level count(*) scans.
+        stats_result = await db.execute(
+            text(
+                """
+                SELECT
+                    COALESCE(pg_class.reltuples, 0)::bigint AS total_rows,
+                    pg_stats.most_common_vals::text AS source_values,
+                    pg_stats.most_common_freqs::text AS source_freqs
+                FROM pg_class
+                LEFT JOIN pg_stats
+                  ON pg_stats.schemaname = 'public'
+                 AND pg_stats.tablename = 'ais_raw_observations'
+                 AND pg_stats.attname = 'source'
+                WHERE pg_class.relname = 'ais_raw_observations'
+                LIMIT 1
+                """
+            )
         )
-        for source, count in raw_result.all():
-            counts[source] = max(counts.get(source, 0), int(count or 0))
+        stats = stats_result.mappings().first()
+        if stats:
+            total_rows = int(stats["total_rows"] or 0)
+            values = str(stats["source_values"] or "").strip("{}")
+            freqs = str(stats["source_freqs"] or "").strip("{}")
+            source_values = [value.strip('"') for value in values.split(",") if value]
+            source_freqs = [float(value) for value in freqs.split(",") if value]
+            for source, freq in zip(source_values, source_freqs):
+                if source in vessel_sources:
+                    counts[source] = max(counts.get(source, 0), int(round(total_rows * freq)))
 
     return counts
 
@@ -208,7 +254,9 @@ async def _load_datasource_endpoint_overrides(
 async def _load_datasource_list_context(
     db: AsyncSession,
     datasources: list[DataSource],
-) -> tuple[dict[int, CollectionTask], dict[str, str]]:
+    *,
+    include_endpoint: bool = True,
+) -> tuple[dict[int, CollectionTask], dict[int, CollectionTask], dict[str, str]]:
     datasource_ids = [datasource.id for datasource in datasources]
     sources = [datasource.source for datasource in datasources]
 
@@ -232,8 +280,67 @@ async def _load_datasource_list_context(
     if stale_datasource_ids:
         running_tasks = await _load_latest_running_tasks(db, datasource_ids)
 
-    endpoint_overrides = await _load_datasource_endpoint_overrides(db, sources)
-    return running_tasks, endpoint_overrides
+    latest_tasks = await _load_latest_tasks(db, datasource_ids)
+    endpoint_overrides = await _load_datasource_endpoint_overrides(db, sources) if include_endpoint else {}
+    return running_tasks, latest_tasks, endpoint_overrides
+
+
+def serialize_datasource_row(
+    datasource: DataSource,
+    *,
+    running_tasks: dict[int, CollectionTask],
+    latest_tasks: dict[int, CollectionTask],
+    record_counts: dict[str, int],
+    endpoint_overrides: dict[str, str],
+    config,
+    include_endpoint: bool,
+) -> dict:
+    running_task = running_tasks.get(datasource.id)
+    latest_task = latest_tasks.get(datasource.id)
+    display_task = running_task or latest_task
+    endpoint = None
+    if include_endpoint:
+        endpoint = endpoint_overrides.get(datasource.source) or config.get_yaml_url(datasource.source)
+    last_run_at = datasource.last_run_at or (latest_task.completed_at if latest_task else None)
+    last_status = datasource.last_status or (latest_task.status if latest_task else None)
+    collected_records = record_counts.get(datasource.source, 0)
+
+    row = {
+        "id": datasource.id,
+        "source": datasource.source,
+        "name": datasource.name,
+        **datasource_metadata(datasource.source),
+        "product": datasource_product_key(datasource),
+        "module": datasource.module,
+        "priority": datasource.priority,
+        "frequency": format_frequency_label(datasource.frequency_minutes),
+        "frequency_minutes": datasource.frequency_minutes,
+        "is_active": datasource.is_active,
+        "collector_class": datasource.collector_class,
+        "last_run": to_iso8601_utc(last_run_at),
+        "last_run_at": to_iso8601_utc(last_run_at),
+        "last_status": last_status,
+        "is_running": running_task is not None and running_task.task_type == JOB_TYPE_COLLECT,
+        "is_task_active": running_task is not None,
+        "task_status": running_task.status if running_task else None,
+        "task_id": display_task.id if display_task else None,
+        "task_type": display_task.task_type if display_task else None,
+        "progress": display_task.progress if display_task else None,
+        "phase": display_task.phase if display_task else None,
+        "phase_progress": display_task.phase_progress if display_task else None,
+        "phase_message": display_task.phase_message if display_task else None,
+        "phase_current": display_task.phase_current if display_task else None,
+        "phase_total": display_task.phase_total if display_task else None,
+        "phase_unit": display_task.phase_unit if display_task else None,
+        "records_processed": display_task.records_processed if display_task else None,
+        "total_records": display_task.total_records if display_task else None,
+        "error_message": display_task.error_message if display_task else None,
+        "collected_records": collected_records,
+        "has_collected_data": collected_records > 0,
+    }
+    if include_endpoint:
+        row["endpoint"] = endpoint
+    return row
 
 
 def _apply_datasource_query_filters(
@@ -251,11 +358,6 @@ def _apply_datasource_query_filters(
         query = query.where(DataSource.is_active == is_active)
     if priority:
         query = query.where(DataSource.priority == priority)
-    if run_status and run_status not in {"running", "collected", "uncollected"}:
-        if run_status == "not_run":
-            query = query.where(DataSource.last_status.is_(None))
-        else:
-            query = query.where(DataSource.last_status == run_status)
     if q:
         like_value = f"%{q.strip()}%"
         query = query.where(
@@ -272,15 +374,25 @@ def _filter_datasources_in_memory(
     datasources: list[DataSource],
     *,
     running_tasks: dict[int, CollectionTask],
+    latest_tasks: dict[int, CollectionTask] | None = None,
     record_counts: dict[str, int],
     product: Optional[str] = None,
     run_status: Optional[str] = None,
     collected: Optional[bool] = None,
     credential_status: Optional[str] = None,
 ) -> list[DataSource]:
+    latest_tasks = latest_tasks or {}
     filtered: list[DataSource] = []
     for datasource in datasources:
         record_count = record_counts.get(datasource.source, 0)
+        latest_task = latest_tasks.get(datasource.id)
+        effective_status = (
+            "running"
+            if datasource.id in running_tasks
+            else latest_task.status
+            if latest_task is not None
+            else datasource.last_status
+        )
         if product and datasource_product_key(datasource) != product:
             continue
         if collected is not None and (record_count > 0) != collected:
@@ -291,6 +403,10 @@ def _filter_datasources_in_memory(
                 continue
         if run_status == "running" and datasource.id not in running_tasks:
             continue
+        if run_status == "not_run" and effective_status is not None:
+            continue
+        if run_status not in {None, "running", "not_run", "collected", "uncollected"} and effective_status != run_status:
+            continue
         if run_status == "collected" and record_count <= 0:
             continue
         if run_status == "uncollected" and record_count > 0:
@@ -304,9 +420,25 @@ async def _trigger_datasource_batch(
     datasources: list[DataSource],
     *,
     force: bool,
+    actor_id: int | None = None,
+    trigger_kind: str = "batch",
 ) -> dict:
+    await emit_business_log(
+        logger,
+        event=f"collector.trigger.{trigger_kind}.start",
+        message="Datasource batch trigger started",
+        category="collector",
+        service="api",
+        module=__name__,
+        user_id=actor_id,
+        context={
+            "trigger_kind": trigger_kind,
+            "force": force,
+            "requested_count": len(datasources),
+        },
+    )
     if not datasources:
-        return {
+        result = {
             "status": "noop",
             "message": "No matching data sources to trigger",
             "force": force,
@@ -314,8 +446,18 @@ async def _trigger_datasource_batch(
             "skipped": [],
             "failed": [],
         }
+        await emit_business_log(
+            logger,
+            event=f"collector.trigger.{trigger_kind}.completed",
+            message="Datasource batch trigger completed with no matching sources",
+            category="collector",
+            service="api",
+            module=__name__,
+            user_id=actor_id,
+            context={"trigger_kind": trigger_kind, "force": force, "status": "noop", "triggered_count": 0},
+        )
+        return result
 
-    previous_task_ids: dict[int, Optional[int]] = {}
     triggered_sources: list[dict] = []
     skipped_sources: list[dict] = []
     failed_sources: list[dict] = []
@@ -350,9 +492,11 @@ async def _trigger_datasource_batch(
                     }
                 )
                 continue
-            cancelled = await cancel_running_collector_now(datasource.source)
-            if not cancelled:
-                await rollback_orphaned_running_task(db, datasource, running_task)
+            await request_cancel_datasource_task(
+                db,
+                running_task,
+                reason="superseded_by_forced_collection",
+            )
 
         if not force and not is_due_for_collection(datasource, now):
             skipped_sources.append(
@@ -369,57 +513,51 @@ async def _trigger_datasource_batch(
             )
             continue
 
-        previous_task_ids[datasource.id] = None
-        success = run_collector_now(datasource.source)
-        if not success:
-            failed_sources.append(
-                {
-                    "id": datasource.id,
-                    "source": datasource.source,
-                    "name": datasource.name,
-                    "reason": "trigger_failed",
-                }
-            )
-            continue
+        task = await enqueue_datasource_job(
+            db,
+            datasource,
+            JOB_TYPE_COLLECT,
+            payload={"force": force, "trigger": "batch"},
+        )
 
         triggered_sources.append(
             {
                 "id": datasource.id,
                 "source": datasource.source,
                 "name": datasource.name,
-                "task_id": None,
+                "task_id": task.id,
             }
         )
 
-    latest_task_ids = await _load_latest_task_ids(
-        db,
-        [datasource.id for datasource in datasources],
-    )
-    for datasource_id in previous_task_ids:
-        previous_task_ids[datasource_id] = latest_task_ids.get(datasource_id)
-
-    for _ in range(20):
-        await asyncio.sleep(0.1)
-        pending = [item for item in triggered_sources if item["task_id"] is None]
-        if not pending:
-            break
-        latest_task_ids = await _load_latest_task_ids(
-            db,
-            [item["id"] for item in pending],
-        )
-        for item in pending:
-            task_id = latest_task_ids.get(item["id"])
-            if task_id is not None and task_id != previous_task_ids.get(item["id"]):
-                item["task_id"] = task_id
-
-    return {
-        "status": "triggered" if triggered_sources else "partial",
-        "message": f"Triggered {len(triggered_sources)} data sources",
+    result = {
+        "status": "queued" if triggered_sources else "partial",
+        "message": f"Queued {len(triggered_sources)} data source jobs",
         "force": force,
         "triggered": triggered_sources,
         "skipped": skipped_sources,
         "failed": failed_sources,
     }
+    await emit_business_log(
+        logger,
+        event=f"collector.trigger.{trigger_kind}.completed",
+        message="Datasource batch trigger completed",
+        category="collector",
+        service="api",
+        module=__name__,
+        user_id=actor_id,
+        context={
+            "trigger_kind": trigger_kind,
+            "force": force,
+            "status": result["status"],
+            "requested_count": len(datasources),
+            "triggered_count": len(triggered_sources),
+            "skipped_count": len(skipped_sources),
+            "failed_count": len(failed_sources),
+            "triggered_sources": [item["source"] for item in triggered_sources],
+            "skipped_reasons": [item["reason"] for item in skipped_sources],
+        },
+    )
+    return result
 
 
 async def get_datasource_record(db: AsyncSession, source_id: str) -> Optional[DataSource]:
@@ -444,7 +582,7 @@ async def get_running_task(db: AsyncSession, datasource_id: int) -> Optional[Col
     result = await db.execute(
         select(CollectionTask)
         .where(CollectionTask.datasource_id == datasource_id)
-        .where(CollectionTask.status == "running")
+        .where(CollectionTask.status == JobStatus.RUNNING.value)
         .order_by(CollectionTask.started_at.desc())
         .limit(1)
     )
@@ -472,8 +610,8 @@ async def get_running_task(db: AsyncSession, datasource_id: int) -> Optional[Col
             f"Marked failed automatically after stale running timeout "
             f"({STALE_RUNNING_TASK_TIMEOUT_MINUTES}m)"
         )
-        task.status = "failed"
-        task.phase = "failed"
+        task.status = JobStatus.FAILED.value
+        task.phase = JobStatus.FAILED.value
         task.completed_at = now
         task.error_message = f"{existing_error}\n{stale_reason}".strip() if existing_error else stale_reason
         await db.commit()
@@ -510,7 +648,7 @@ async def rollback_orphaned_running_task(
     )
 
     if snapshot is not None:
-        snapshot.status = "cancelled"
+        snapshot.status = SnapshotStatus.CANCELLED.value
         snapshot.is_current = False
         snapshot.completed_at = datetime.now(timezone.utc)
         summary = dict(snapshot.summary or {})
@@ -533,13 +671,13 @@ async def rollback_orphaned_running_task(
                 {"snapshot_id": snapshot.parent_snapshot_id},
             )
 
-    running_task.status = "cancelled"
-    running_task.phase = "cancelled"
+    running_task.status = JobStatus.CANCELLED.value
+    running_task.phase = JobStatus.CANCELLED.value
     running_task.completed_at = datetime.now(timezone.utc)
     existing_error = (running_task.error_message or "").strip()
     cancel_reason = "Cancelled after backend restart because the running task handle was lost; incomplete writes rolled back"
     running_task.error_message = f"{existing_error}\n{cancel_reason}".strip() if existing_error else cancel_reason
-    datasource.last_status = "cancelled"
+    datasource.last_status = JobStatus.CANCELLED.value
     datasource.last_run_at = datetime.now(timezone.utc)
     await db.commit()
 
@@ -574,7 +712,7 @@ async def fail_and_rollback_stale_running_task(
     )
 
     if snapshot is not None:
-        snapshot.status = "failed"
+        snapshot.status = SnapshotStatus.FAILED.value
         snapshot.is_current = False
         snapshot.completed_at = datetime.now(timezone.utc)
         summary = dict(snapshot.summary or {})
@@ -602,11 +740,11 @@ async def fail_and_rollback_stale_running_task(
         f"Marked failed automatically after stale running timeout "
         f"({STALE_RUNNING_TASK_TIMEOUT_MINUTES}m); incomplete writes rolled back"
     )
-    running_task.status = "failed"
-    running_task.phase = "failed"
+    running_task.status = JobStatus.FAILED.value
+    running_task.phase = JobStatus.FAILED.value
     running_task.completed_at = datetime.now(timezone.utc)
     running_task.error_message = f"{existing_error}\n{stale_reason}".strip() if existing_error else stale_reason
-    datasource.last_status = "failed"
+    datasource.last_status = JobStatus.FAILED.value
     datasource.last_run_at = datetime.now(timezone.utc)
     await db.commit()
 
@@ -621,6 +759,7 @@ async def list_datasources(
     collected: Optional[bool] = None,
     credential_status: Optional[str] = None,
     q: Optional[str] = None,
+    include_endpoint: bool = True,
     current_user: User = Depends(get_current_user),
     db: AsyncSession = Depends(get_db),
 ):
@@ -639,11 +778,16 @@ async def list_datasources(
 
     collector_list = []
     config = get_data_sources_config()
-    running_tasks, endpoint_overrides = await _load_datasource_list_context(db, datasources)
+    running_tasks, latest_tasks, endpoint_overrides = await _load_datasource_list_context(
+        db,
+        datasources,
+        include_endpoint=include_endpoint,
+    )
     record_counts = await _load_collected_record_counts(db, [datasource.source for datasource in datasources])
     datasources = _filter_datasources_in_memory(
         datasources,
         running_tasks=running_tasks,
+        latest_tasks=latest_tasks,
         record_counts=record_counts,
         product=product,
         run_status=run_status,
@@ -651,43 +795,16 @@ async def list_datasources(
         credential_status=credential_status,
     )
     for datasource in datasources:
-        running_task = running_tasks.get(datasource.id)
-        endpoint = endpoint_overrides.get(datasource.source) or config.get_yaml_url(datasource.source)
-        last_run_at = datasource.last_run_at
-        last_status = datasource.last_status
-        collected_records = record_counts.get(datasource.source, 0)
-
         collector_list.append(
-            {
-                "id": datasource.id,
-                "source": datasource.source,
-                "name": datasource.name,
-                **datasource_metadata(datasource.source),
-                "product": datasource_product_key(datasource),
-                "module": datasource.module,
-                "priority": datasource.priority,
-                "frequency": format_frequency_label(datasource.frequency_minutes),
-                "frequency_minutes": datasource.frequency_minutes,
-                "is_active": datasource.is_active,
-                "collector_class": datasource.collector_class,
-                "endpoint": endpoint,
-                "last_run": to_iso8601_utc(last_run_at),
-                "last_run_at": to_iso8601_utc(last_run_at),
-                "last_status": last_status,
-                "is_running": running_task is not None,
-                "task_id": running_task.id if running_task else None,
-                "progress": running_task.progress if running_task else None,
-                "phase": running_task.phase if running_task else None,
-                "phase_progress": running_task.phase_progress if running_task else None,
-                "phase_message": running_task.phase_message if running_task else None,
-                "phase_current": running_task.phase_current if running_task else None,
-                "phase_total": running_task.phase_total if running_task else None,
-                "phase_unit": running_task.phase_unit if running_task else None,
-                "records_processed": running_task.records_processed if running_task else None,
-                "total_records": running_task.total_records if running_task else None,
-                "collected_records": collected_records,
-                "has_collected_data": collected_records > 0,
-            }
+            serialize_datasource_row(
+                datasource,
+                running_tasks=running_tasks,
+                latest_tasks=latest_tasks,
+                record_counts=record_counts,
+                endpoint_overrides=endpoint_overrides,
+                config=config,
+                include_endpoint=include_endpoint,
+            )
         )
 
     return {"total": len(collector_list), "data": collector_list}
@@ -705,7 +822,13 @@ async def trigger_all_datasources(
         .order_by(DataSource.module, DataSource.id)
     )
     datasources = result.scalars().all()
-    return await _trigger_datasource_batch(db, datasources, force=force)
+    return await _trigger_datasource_batch(
+        db,
+        datasources,
+        force=force,
+        actor_id=current_user.id,
+        trigger_kind="all",
+    )
 
 
 @router.post("/trigger-batch")
@@ -729,18 +852,70 @@ async def trigger_datasource_batch(
 
     result = await db.execute(query)
     datasources = result.scalars().all()
-    running_tasks, _ = await _load_datasource_list_context(db, datasources)
+    running_tasks, latest_tasks, _ = await _load_datasource_list_context(db, datasources)
     record_counts = await _load_collected_record_counts(db, [datasource.source for datasource in datasources])
     datasources = _filter_datasources_in_memory(
         datasources,
         running_tasks=running_tasks,
+        latest_tasks=latest_tasks,
         record_counts=record_counts,
         product=None if payload.source_ids else payload.product,
         run_status=None if payload.source_ids else payload.run_status,
         collected=None if payload.source_ids else payload.collected,
         credential_status=None if payload.source_ids else payload.credential_status,
     )
-    return await _trigger_datasource_batch(db, datasources, force=payload.force)
+    return await _trigger_datasource_batch(
+        db,
+        datasources,
+        force=payload.force,
+        actor_id=current_user.id,
+        trigger_kind="batch",
+    )
+
+
+@router.get("/snapshots")
+async def list_datasource_snapshots(
+    source_id: Optional[str] = None,
+    current_only: Optional[bool] = None,
+    limit: int = Query(default=100, ge=1, le=500),
+    current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    query = (
+        select(DataSnapshot, DataSource.name, DataSource.module)
+        .outerjoin(DataSource, DataSource.id == DataSnapshot.datasource_id)
+        .order_by(DataSnapshot.created_at.desc().nullslast(), DataSnapshot.id.desc())
+        .limit(limit)
+    )
+    if source_id:
+        query = query.where(DataSnapshot.source == source_id)
+    if current_only is not None:
+        query = query.where(DataSnapshot.is_current.is_(current_only))
+
+    result = await db.execute(query)
+    rows = []
+    for snapshot, datasource_name, datasource_module in result.all():
+        rows.append(
+            {
+                "id": snapshot.id,
+                "datasource_id": snapshot.datasource_id,
+                "datasource_name": datasource_name,
+                "module": datasource_module,
+                "task_id": snapshot.task_id,
+                "source": snapshot.source,
+                "snapshot_key": snapshot.snapshot_key,
+                "reference_date": to_iso8601_utc(snapshot.reference_date),
+                "started_at": to_iso8601_utc(snapshot.started_at),
+                "completed_at": to_iso8601_utc(snapshot.completed_at),
+                "record_count": snapshot.record_count,
+                "status": snapshot.status,
+                "is_current": snapshot.is_current,
+                "parent_snapshot_id": snapshot.parent_snapshot_id,
+                "summary": snapshot.summary or {},
+                "created_at": to_iso8601_utc(snapshot.created_at),
+            }
+        )
+    return {"total": len(rows), "data": rows}
 
 
 @router.get("/{source_id}")
@@ -771,6 +946,37 @@ async def get_datasource(
     }
 
 
+@router.get("/{source_id}/row")
+async def get_datasource_row(
+    source_id: str,
+    include_endpoint: bool = True,
+    current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    datasource = await get_datasource_record(db, source_id)
+    if not datasource:
+        raise HTTPException(status_code=404, detail="Data source not found")
+
+    config = get_data_sources_config()
+    running_tasks, latest_tasks, endpoint_overrides = await _load_datasource_list_context(
+        db,
+        [datasource],
+        include_endpoint=include_endpoint,
+    )
+    record_counts = await _load_collected_record_counts(db, [datasource.source], exact_vessel_counts=True)
+    return {
+        "data": serialize_datasource_row(
+            datasource,
+            running_tasks=running_tasks,
+            latest_tasks=latest_tasks,
+            record_counts=record_counts,
+            endpoint_overrides=endpoint_overrides,
+            config=config,
+            include_endpoint=include_endpoint,
+        )
+    }
+
+
 @router.post("/{source_id}/enable")
 async def enable_datasource(
     source_id: str,
@@ -840,8 +1046,24 @@ async def trigger_datasource(
     if not datasource.is_active:
         raise HTTPException(status_code=400, detail="Data source is disabled")
 
-    running_task = await get_running_task(db, datasource.id)
+    running_task = await get_active_datasource_job(db, datasource.id, task_types=(JOB_TYPE_COLLECT,))
     if running_task is not None and not force:
+        await emit_business_log(
+            logger,
+            event="collector.trigger.single.skipped_already_running",
+            message="Datasource trigger skipped because a task is already running",
+            category="collector",
+            level="warning",
+            service="api",
+            module=__name__,
+            user_id=current_user.id,
+            context={
+                "collector_name": datasource.source,
+                "datasource_id": datasource.id,
+                "task_id": running_task.id,
+                "status": "skipped",
+            },
+        )
         raise HTTPException(
             status_code=409,
             detail={
@@ -861,31 +1083,42 @@ async def trigger_datasource(
         )
 
     if running_task is not None and force:
-        cancelled = await cancel_running_collector_now(datasource.source)
-        if not cancelled:
-            await rollback_orphaned_running_task(db, datasource, running_task)
+        await request_cancel_datasource_task(
+            db,
+            running_task,
+            reason="superseded_by_forced_collection",
+        )
 
-    previous_task_id = await get_latest_task_id_for_datasource(datasource.id)
-    success = run_collector_now(datasource.source)
-    if not success:
-        raise HTTPException(status_code=500, detail=f"Failed to trigger collector '{datasource.source}'")
-
-    task_id = None
-    for _ in range(20):
-        await asyncio.sleep(0.1)
-        task_id = await get_latest_task_id_for_datasource(datasource.id)
-        if task_id is not None and task_id != previous_task_id:
-            break
-    if task_id == previous_task_id:
-        task_id = None
+    task = await enqueue_datasource_job(
+        db,
+        datasource,
+        JOB_TYPE_COLLECT,
+        payload={"force": force, "trigger": "single"},
+    )
+    await emit_business_log(
+        logger,
+        event="collector.trigger.single.completed",
+        message="Datasource trigger queued",
+        category="collector",
+        service="api",
+        module=__name__,
+        user_id=current_user.id,
+        context={
+            "collector_name": datasource.source,
+            "datasource_id": datasource.id,
+            "task_id": task.id,
+            "force": force,
+            "status": "queued",
+        },
+    )
 
     return {
-        "status": "triggered",
+        "status": "queued",
         "source_id": datasource.id,
-        "task_id": task_id,
+        "task_id": task.id,
         "collector_name": datasource.source,
         "force": force,
-        "message": f"Collector '{datasource.source}' has been triggered",
+        "message": f"Collector '{datasource.source}' has been queued",
     }
 
 
@@ -899,22 +1132,81 @@ async def clear_datasource_data(
     if not datasource:
         raise HTTPException(status_code=404, detail="Data source not found")
 
-    result = await db.execute(
-        select(func.count(CollectedData.id)).where(CollectedData.source == datasource.source)
+    active_task = await get_active_datasource_job(db, datasource.id)
+    if active_task is not None:
+        raise HTTPException(
+            status_code=409,
+            detail={
+                "reason": "datasource_job_in_progress",
+                "message": "当前数据源已有任务在执行，请等待完成或先取消任务。",
+                "task_id": active_task.id,
+                "task_type": active_task.task_type,
+                "status": active_task.status,
+            },
+        )
+    task = await enqueue_datasource_job(
+        db,
+        datasource,
+        JOB_TYPE_CLEAR_DATA,
+        payload={"source": datasource.source},
     )
-    count = result.scalar() or 0
-
-    if count == 0:
-        return {"status": "success", "message": "No data to clear", "deleted_count": 0}
-
-    delete_query = CollectedData.__table__.delete().where(CollectedData.source == datasource.source)
-    await db.execute(delete_query)
-    await db.commit()
 
     return {
-        "status": "success",
-        "message": f"Cleared {count} records for data source '{datasource.name}'",
-        "deleted_count": count,
+        "status": "queued",
+        "message": f"Queued data clearing for data source '{datasource.name}'",
+        "task_id": task.id,
+        "deleted_count": None,
+    }
+
+
+@router.delete("/{source_id}/cache")
+async def clear_datasource_cache(
+    source_id: str,
+    current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    datasource = await get_datasource_record(db, source_id)
+    if not datasource:
+        raise HTTPException(status_code=404, detail="Data source not found")
+
+    task = await enqueue_datasource_job(
+        db,
+        datasource,
+        JOB_TYPE_CLEAR_CACHE,
+        payload={"source": datasource.source},
+        dedupe_key=f"clear_cache:{datasource.source}",
+    )
+
+    return {
+        "status": "queued",
+        "message": f"Queued cache clearing for data source '{datasource.name}'",
+        "task_id": task.id,
+        "deleted_count": None,
+    }
+
+
+@router.post("/{source_id}/tasks/{task_id}/cancel")
+async def cancel_datasource_task(
+    source_id: str,
+    task_id: int,
+    current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    datasource = await get_datasource_record(db, source_id)
+    if not datasource:
+        raise HTTPException(status_code=404, detail="Data source not found")
+
+    task = await db.get(CollectionTask, task_id)
+    if not task or task.datasource_id != datasource.id:
+        raise HTTPException(status_code=404, detail="Task not found")
+
+    task = await request_cancel_datasource_task(db, task)
+    return {
+        "status": "cancelled" if task.completed_at else "cancelling",
+        "task_id": task.id,
+        "task_type": task.task_type,
+        "phase": task.phase,
+        "requested_cancel_at": to_iso8601_utc(task.requested_cancel_at),
     }
 
 
@@ -934,7 +1226,15 @@ async def get_task_status(
         if not task or task.datasource_id != datasource.id:
             raise HTTPException(status_code=404, detail="Task not found")
     else:
-        task = await get_running_task(db, datasource.id)
+        task = await get_active_datasource_job(db, datasource.id)
+        if task is None:
+            result = await db.execute(
+                select(CollectionTask)
+                .where(CollectionTask.datasource_id == datasource.id)
+                .order_by(CollectionTask.created_at.desc().nullslast(), CollectionTask.id.desc())
+                .limit(1)
+            )
+            task = result.scalar_one_or_none()
 
     if not task:
         return {
@@ -951,8 +1251,12 @@ async def get_task_status(
         }
 
     return {
-        "is_running": task.status == "running",
+        "is_running": task.status in {JOB_STATUS_QUEUED, JOB_STATUS_RUNNING, JOB_STATUS_CANCELLING}
+        and task.task_type == JOB_TYPE_COLLECT,
+        "is_task_active": task.status in {JOB_STATUS_QUEUED, JOB_STATUS_RUNNING, JOB_STATUS_CANCELLING},
         "task_id": task.id,
+        "task_type": task.task_type,
+        "task_status": task.status,
         "progress": task.progress,
         "phase": task.phase,
         "phase_progress": task.phase_progress,
@@ -963,4 +1267,6 @@ async def get_task_status(
         "records_processed": task.records_processed,
         "total_records": task.total_records,
         "status": task.status,
+        "requested_cancel_at": to_iso8601_utc(task.requested_cancel_at),
+        "error_message": task.error_message,
     }

backend/app/api/v1/earth.py

@@ -0,0 +1,682 @@
+"""Earth asset management APIs."""
+
+from __future__ import annotations
+
+from pathlib import Path
+from typing import Any
+from uuid import uuid4
+
+from fastapi import APIRouter, Depends, File, Form, HTTPException, Query, Request, UploadFile, status
+from fastapi.security import HTTPAuthorizationCredentials, HTTPBearer
+from pydantic import BaseModel, Field
+from sqlalchemy import delete, func, select, text
+from sqlalchemy.ext.asyncio import AsyncSession
+
+from app.core.config import settings as app_settings
+from app.core.security import decode_token, get_current_user, redis_client
+from app.db.session import get_db
+from app.models.collected_data import CollectedData
+from app.models.datasource import DataSource
+from app.models.datasource_config import DataSourceConfig
+from app.models.system_setting import SystemSetting
+from app.models.user import User
+from app.services.tv_streams import get_tv_settings_payload
+from app.services.earth_news import (
+    get_earth_news_sources_payload,
+    reset_earth_news_sources_payload,
+    save_earth_news_sources_payload,
+    test_news_source_config,
+)
+from app.services.earth_news_manual import (
+    broadcast_manual_news_changed,
+    create_manual_news_group,
+    delete_manual_news_item,
+    get_news_record_or_404,
+    import_manual_news_items,
+    list_news_groups,
+    list_news_records,
+    parse_manual_news_import_upload,
+    rename_manual_news_group,
+    reprocess_manual_news_item,
+    serialize_news_record,
+    upsert_manual_news_item,
+)
+from app.services.earth_boundaries import (
+    EarthBoundaryBuildError,
+    get_boundary_build_status,
+    get_boundary_status,
+    save_boundary_config,
+    start_boundary_build_job,
+)
+
+
+router = APIRouter()
+optional_bearer = HTTPBearer(auto_error=False)
+REPO_ROOT = Path(__file__).resolve().parents[4]
+EARTH_BRAND_ASSET_DIR = REPO_ROOT / "data" / "earth-brand"
+EARTH_BRAND_ASSET_URL_PREFIX = "/earth-brand-assets"
+EARTH_BRAND_CATEGORY = "earth_brand"
+EARTH_ABOUT_CATEGORY = "earth_about"
+SYSTEM_SETTINGS_CATEGORY = "system"
+MAX_EARTH_BRAND_ASSET_BYTES = 3 * 1024 * 1024
+ALLOWED_EARTH_BRAND_EXTENSIONS = {".png", ".jpg", ".jpeg", ".webp", ".svg"}
+
+
+def _app_version_label() -> str:
+    version = str(app_settings.VERSION or "").strip() or "0.0.0"
+    return version if version.startswith("v") else f"v{version}"
+
+
+DEFAULT_EARTH_BRAND = {
+    "logo_src": "/earth/assets/brand/earth-logo.png",
+    "title_src": "/earth/assets/brand/title-zh.png",
+    "title_text": "智能星球计划",
+    "subtitle": "现实层宇宙全息感知系统",
+    "description": "卫星 · 海底光缆 · 算力基础设施",
+    "aria_label": "智能星球计划品牌标识",
+    "title_alt": "智能星球计划",
+}
+
+DEFAULT_EARTH_ABOUT = {
+    "logo_src": "/earth/assets/brand/lim-logo.png",
+    "kicker": "About",
+    "title": "智能星球计划",
+    "version": _app_version_label(),
+    "description": "面向临空场景下的智能媒体研究、全球态势感知与多源开放数据巡航，提供可视化观测、事件聚合与交互式探索能力。",
+    "meta": [
+        {"label": "出品方", "value": "浙江大学临空智能媒体研究院"},
+        {"label": "策划人", "value": "方兴东、黄柳青"},
+        {"label": "产品兼开发者", "value": "钱坤、张鸽、齐鹏"},
+    ],
+}
+EARTH_ABOUT_LEGACY_PLANNER_VALUE = "黄柳青"
+
+
+class EarthBoundaryConfigPayload(BaseModel):
+    config: dict[str, Any] = Field(default_factory=dict)
+
+
+class EarthBrandPayload(BaseModel):
+    logo_src: str = Field(default=DEFAULT_EARTH_BRAND["logo_src"], max_length=1000)
+    title_src: str = Field(default=DEFAULT_EARTH_BRAND["title_src"], max_length=1000)
+    title_text: str = Field(default=DEFAULT_EARTH_BRAND["title_text"], max_length=120)
+    subtitle: str = Field(default=DEFAULT_EARTH_BRAND["subtitle"], max_length=160)
+    description: str = Field(default=DEFAULT_EARTH_BRAND["description"], max_length=200)
+    aria_label: str = Field(default=DEFAULT_EARTH_BRAND["aria_label"], max_length=200)
+    title_alt: str = Field(default=DEFAULT_EARTH_BRAND["title_alt"], max_length=200)
+
+
+class EarthAboutMetaItem(BaseModel):
+    label: str = Field(default="", max_length=80)
+    value: str = Field(default="", max_length=240)
+
+
+class EarthAboutPayload(BaseModel):
+    logo_src: str = Field(default=DEFAULT_EARTH_ABOUT["logo_src"], max_length=1000)
+    kicker: str = Field(default=DEFAULT_EARTH_ABOUT["kicker"], max_length=80)
+    title: str = Field(default=DEFAULT_EARTH_ABOUT["title"], max_length=160)
+    version: str = Field(default=DEFAULT_EARTH_ABOUT["version"], max_length=80)
+    description: str = Field(default=DEFAULT_EARTH_ABOUT["description"], max_length=800)
+    meta: list[EarthAboutMetaItem] = Field(default_factory=list)
+
+
+class EarthNewsSourcesPayload(BaseModel):
+    cache_version: int | None = None
+    source_tags: list[dict[str, Any]] = Field(default_factory=list)
+    categories: list[dict[str, Any]] = Field(default_factory=list)
+    item_tag_rules: list[dict[str, Any]] = Field(default_factory=list)
+    sources: list[dict[str, Any]] = Field(default_factory=list)
+    health: dict[str, Any] = Field(default_factory=dict)
+
+
+class EarthNewsSourceTestPayload(BaseModel):
+    source: dict[str, Any] = Field(default_factory=dict)
+
+
+class EarthNewsManualItemPayload(BaseModel):
+    title: str = Field(default="", max_length=500)
+    summary: str = Field(default="", max_length=1200)
+    content: str = Field(default="", max_length=12000)
+    url: str = Field(default="", max_length=2000)
+    source: str = Field(default="", max_length=255)
+    region: str = Field(default="global", max_length=80)
+    published_at: str | None = None
+    category: str = Field(default="other", max_length=80)
+    tags: list[str] = Field(default_factory=list)
+    location: dict[str, Any] | None = None
+    homepage_url: str = Field(default="", max_length=2000)
+    content_language: str = Field(default="", max_length=32)
+    group_id: str | None = Field(default=None, max_length=120)
+
+
+class EarthNewsManualGroupPayload(BaseModel):
+    name: str = Field(default="", max_length=120)
+
+
+def _normalize_earth_brand_payload(payload: dict[str, Any] | None) -> dict[str, str]:
+    merged = DEFAULT_EARTH_BRAND.copy()
+    if payload:
+        for key in DEFAULT_EARTH_BRAND:
+            value = payload.get(key)
+            if value is not None:
+                merged[key] = str(value).strip()
+
+    if not merged["title_text"]:
+        merged["title_text"] = DEFAULT_EARTH_BRAND["title_text"]
+    if not merged["aria_label"]:
+        merged["aria_label"] = merged["title_text"]
+    if not merged["title_alt"]:
+        merged["title_alt"] = merged["title_text"]
+    return merged
+
+
+def _normalize_earth_about_payload(payload: dict[str, Any] | None) -> dict[str, Any]:
+    merged: dict[str, Any] = {
+        key: value
+        for key, value in DEFAULT_EARTH_ABOUT.items()
+        if key != "meta"
+    }
+    raw_meta = DEFAULT_EARTH_ABOUT["meta"]
+    if payload:
+        for key in ("logo_src", "kicker", "title", "description"):
+            value = payload.get(key)
+            if value is not None:
+                merged[key] = str(value).strip()
+        raw_meta = payload.get("meta") if isinstance(payload.get("meta"), list) else raw_meta
+    merged["version"] = _app_version_label()
+
+    for key, default_value in DEFAULT_EARTH_ABOUT.items():
+        if key == "meta":
+            continue
+        if not merged.get(key):
+            merged[key] = default_value
+
+    normalized_meta: list[dict[str, str]] = []
+    for item in raw_meta:
+        if not isinstance(item, dict):
+            continue
+        label = str(item.get("label") or "").strip()
+        value = str(item.get("value") or "").strip()
+        if label == "策划人" and value == EARTH_ABOUT_LEGACY_PLANNER_VALUE:
+            value = "方兴东、黄柳青"
+        if label or value:
+            normalized_meta.append({"label": label, "value": value})
+    if not normalized_meta:
+        normalized_meta = [dict(item) for item in DEFAULT_EARTH_ABOUT["meta"]]
+    merged["meta"] = normalized_meta
+    return merged
+
+
+def _is_demo_mode_enabled(payload: Any) -> bool:
+    return bool(payload.get("demo_mode")) if isinstance(payload, dict) else False
+
+
+async def _get_earth_brand_record(db: AsyncSession) -> SystemSetting | None:
+    result = await db.execute(
+        select(SystemSetting).where(SystemSetting.category == EARTH_BRAND_CATEGORY)
+    )
+    return result.scalar_one_or_none()
+
+
+async def _get_earth_brand_payload(db: AsyncSession) -> dict[str, Any]:
+    record = await _get_earth_brand_record(db)
+    return {
+        "brand": _normalize_earth_brand_payload(record.payload if record else None),
+        "is_default": record is None,
+    }
+
+
+async def _get_earth_about_record(db: AsyncSession) -> SystemSetting | None:
+    result = await db.execute(
+        select(SystemSetting).where(SystemSetting.category == EARTH_ABOUT_CATEGORY)
+    )
+    return result.scalar_one_or_none()
+
+
+async def _get_earth_about_payload(db: AsyncSession) -> dict[str, Any]:
+    record = await _get_earth_about_record(db)
+    return {
+        "about": _normalize_earth_about_payload(record.payload if record else None),
+        "is_default": record is None,
+    }
+
+
+async def _get_optional_current_user(
+    credentials: HTTPAuthorizationCredentials | None = Depends(optional_bearer),
+    db: AsyncSession = Depends(get_db),
+) -> User | None:
+    if credentials is None:
+        return None
+    token = credentials.credentials
+    if redis_client.sismember("blacklisted_tokens", token):
+        return None
+    payload = decode_token(token)
+    if payload is None or payload.get("type") != "access":
+        return None
+    user_id = payload.get("sub")
+    if user_id is None:
+        return None
+    result = await db.execute(
+        text(
+            "SELECT id, username, email, password_hash, role, is_active, gatekeeper_groups FROM users WHERE id = :id"
+        ),
+        {"id": int(user_id)},
+    )
+    row = result.fetchone()
+    if row is None or not row[5]:
+        return None
+    user = User()
+    user.id = row[0]
+    user.username = row[1]
+    user.email = row[2]
+    user.password_hash = row[3]
+    user.role = row[4]
+    user.is_active = row[5]
+    user.gatekeeper_groups = row[6] or []
+    return user
+
+
+@router.get("/brand")
+async def get_earth_brand(db: AsyncSession = Depends(get_db)):
+    return await _get_earth_brand_payload(db)
+
+
+@router.put("/brand")
+async def update_earth_brand(
+    payload: EarthBrandPayload,
+    _current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    normalized = _normalize_earth_brand_payload(payload.model_dump())
+    record = await _get_earth_brand_record(db)
+    if record is None:
+        record = SystemSetting(category=EARTH_BRAND_CATEGORY, payload=normalized)
+        db.add(record)
+    else:
+        record.payload = normalized
+    await db.commit()
+    await db.refresh(record)
+    return {"status": "updated", "brand": _normalize_earth_brand_payload(record.payload), "is_default": False}
+
+
+@router.delete("/brand")
+@router.post("/brand/reset")
+async def reset_earth_brand(
+    _current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    await db.execute(delete(SystemSetting).where(SystemSetting.category == EARTH_BRAND_CATEGORY))
+    await db.commit()
+    return {"status": "reset", "brand": DEFAULT_EARTH_BRAND.copy(), "is_default": True}
+
+
+@router.post("/brand/assets")
+async def upload_earth_brand_asset(
+    file: UploadFile = File(...),
+    _current_user: User = Depends(get_current_user),
+):
+    original_name = file.filename or ""
+    extension = Path(original_name).suffix.lower()
+    if extension not in ALLOWED_EARTH_BRAND_EXTENSIONS:
+        raise HTTPException(
+            status_code=400,
+            detail={
+                "code": "unsupported_file_type",
+                "message": "Only png, jpg, jpeg, webp, and svg brand assets are supported.",
+            },
+        )
+
+    content = await file.read(MAX_EARTH_BRAND_ASSET_BYTES + 1)
+    if len(content) > MAX_EARTH_BRAND_ASSET_BYTES:
+        raise HTTPException(
+            status_code=400,
+            detail={
+                "code": "file_too_large",
+                "message": "Brand asset must be 3 MB or smaller.",
+            },
+        )
+
+    EARTH_BRAND_ASSET_DIR.mkdir(parents=True, exist_ok=True)
+    safe_name = f"{uuid4().hex}{extension}"
+    destination = EARTH_BRAND_ASSET_DIR / safe_name
+    destination.write_bytes(content)
+    asset_url = f"{EARTH_BRAND_ASSET_URL_PREFIX}/{safe_name}"
+    return {"url": asset_url, "filename": safe_name, "content_type": file.content_type}
+
+
+@router.get("/about")
+async def get_earth_about(db: AsyncSession = Depends(get_db)):
+    return await _get_earth_about_payload(db)
+
+
+@router.put("/about")
+async def update_earth_about(
+    payload: EarthAboutPayload,
+    _current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    normalized = _normalize_earth_about_payload(payload.model_dump())
+    record = await _get_earth_about_record(db)
+    if record is None:
+        record = SystemSetting(category=EARTH_ABOUT_CATEGORY, payload=normalized)
+        db.add(record)
+    else:
+        record.payload = normalized
+    await db.commit()
+    await db.refresh(record)
+    return {"status": "updated", "about": _normalize_earth_about_payload(record.payload), "is_default": False}
+
+
+@router.delete("/about")
+async def reset_earth_about(
+    _current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    await db.execute(delete(SystemSetting).where(SystemSetting.category == EARTH_ABOUT_CATEGORY))
+    await db.commit()
+    return {"status": "reset", "about": _normalize_earth_about_payload(None), "is_default": True}
+
+
+@router.get("/news-sources")
+async def get_earth_news_sources(db: AsyncSession = Depends(get_db)):
+    return await get_earth_news_sources_payload(db)
+
+
+@router.put("/news-sources")
+async def update_earth_news_sources(
+    payload: EarthNewsSourcesPayload,
+    _current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    return await save_earth_news_sources_payload(db, payload.model_dump())
+
+
+@router.delete("/news-sources")
+@router.post("/news-sources/reset")
+async def reset_earth_news_sources(
+    _current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    return await reset_earth_news_sources_payload(db)
+
+
+@router.post("/news-sources/test")
+async def test_earth_news_source(
+    payload: EarthNewsSourceTestPayload,
+    _current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    return await test_news_source_config(payload.source, db=db)
+
+
+@router.get("/news-groups")
+async def list_earth_news_groups_admin(
+    _current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    return await list_news_groups(db)
+
+
+@router.post("/news-groups")
+async def create_earth_news_group_admin(
+    payload: EarthNewsManualGroupPayload,
+    _current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    try:
+        group = await create_manual_news_group(db, payload.name)
+    except ValueError as exc:
+        raise HTTPException(status_code=422, detail=str(exc)) from exc
+    await db.commit()
+    return {"status": "ok", "group": group}
+
+
+@router.put("/news-groups/{group_id:path}")
+async def rename_earth_news_group_admin(
+    group_id: str,
+    payload: EarthNewsManualGroupPayload,
+    _current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    try:
+        group = await rename_manual_news_group(db, group_id, payload.name)
+    except ValueError as exc:
+        raise HTTPException(status_code=422, detail=str(exc)) from exc
+    await db.commit()
+    await broadcast_manual_news_changed()
+    return {"status": "ok", "group": group}
+
+
+@router.get("/news-items")
+async def list_earth_news_items_admin(
+    page: int = Query(1, ge=1),
+    page_size: int = Query(50, ge=1, le=100),
+    source_type: str | None = Query(None),
+    region: str | None = Query(None),
+    category: str | None = Query(None),
+    status_filter: str | None = Query(None, alias="status"),
+    group_id: str | None = Query(None),
+    _current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    return await list_news_records(
+        db,
+        page=page,
+        page_size=page_size,
+        source_type=source_type,
+        region=region,
+        category=category,
+        status_filter=status_filter,
+        group_id=group_id,
+    )
+
+
+@router.post("/news-items")
+async def create_earth_news_item_admin(
+    payload: EarthNewsManualItemPayload,
+    _current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    try:
+        result = await upsert_manual_news_item(db, payload.model_dump())
+    except ValueError as exc:
+        raise HTTPException(status_code=422, detail=str(exc)) from exc
+    except PermissionError as exc:
+        raise HTTPException(status_code=403, detail=str(exc)) from exc
+    await db.commit()
+    await broadcast_manual_news_changed()
+    return {"status": "ok", "created": result.created, "queued": result.queued, "item": serialize_news_record(result.item)}
+
+
+@router.post("/news-items/import")
+async def import_earth_news_items_admin(
+    file: UploadFile = File(...),
+    group_id: str | None = Form(default=None),
+    _current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    try:
+        payload = await parse_manual_news_import_upload(await file.read())
+        result = await import_manual_news_items(db, payload, group_id=group_id)
+    except ValueError as exc:
+        raise HTTPException(status_code=422, detail=str(exc)) from exc
+    await db.commit()
+    await broadcast_manual_news_changed()
+    return {"status": "ok", **result}
+
+
+@router.put("/news-items/{item_id:path}")
+async def update_earth_news_item_admin(
+    item_id: str,
+    payload: EarthNewsManualItemPayload,
+    _current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    existing = await get_news_record_or_404(db, item_id)
+    if existing is None:
+        raise HTTPException(status_code=404, detail="News item not found.")
+    try:
+        result = await upsert_manual_news_item(
+            db,
+            payload.model_dump(),
+            item_id_override=item_id,
+        )
+    except ValueError as exc:
+        raise HTTPException(status_code=422, detail=str(exc)) from exc
+    except PermissionError as exc:
+        raise HTTPException(status_code=403, detail=str(exc)) from exc
+    await db.commit()
+    await broadcast_manual_news_changed()
+    return {"status": "ok", "created": result.created, "queued": result.queued, "item": serialize_news_record(result.item)}
+
+
+@router.delete("/news-items/{item_id:path}")
+async def delete_earth_news_item_admin(
+    item_id: str,
+    _current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    try:
+        deleted = await delete_manual_news_item(db, item_id)
+    except PermissionError as exc:
+        raise HTTPException(status_code=403, detail=str(exc)) from exc
+    if not deleted:
+        raise HTTPException(status_code=404, detail="News item not found.")
+    await db.commit()
+    await broadcast_manual_news_changed()
+    return {"status": "deleted", "id": item_id}
+
+
+@router.post("/news-items/{item_id:path}/reprocess")
+async def reprocess_earth_news_item_admin(
+    item_id: str,
+    _current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    existing = await get_news_record_or_404(db, item_id)
+    if existing is None:
+        raise HTTPException(status_code=404, detail="News item not found.")
+    try:
+        queued = await reprocess_manual_news_item(db, item_id)
+    except PermissionError as exc:
+        raise HTTPException(status_code=403, detail=str(exc)) from exc
+    await db.commit()
+    await broadcast_manual_news_changed()
+    return {"status": "queued" if queued else "not_queued", "queued": queued, "id": item_id}
+
+
+@router.get("/oobe-status")
+async def get_earth_oobe_status(
+    current_user: User | None = Depends(_get_optional_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    current_count_result = await db.execute(
+        select(func.count(CollectedData.id)).where(CollectedData.is_current.is_(True))
+    )
+    current_record_count = int(current_count_result.scalar() or 0)
+    system_result = await db.execute(
+        select(SystemSetting).where(SystemSetting.category == SYSTEM_SETTINGS_CATEGORY)
+    )
+    system_record = system_result.scalar_one_or_none()
+    demo_mode = _is_demo_mode_enabled(system_record.payload if system_record else None)
+
+    datasource_count_result = await db.execute(select(func.count(DataSource.id)))
+    datasource_count = int(datasource_count_result.scalar() or 0)
+    active_datasource_count_result = await db.execute(
+        select(func.count(DataSource.id)).where(DataSource.is_active.is_(True))
+    )
+    active_datasource_count = int(active_datasource_count_result.scalar() or 0)
+    config_result = await db.execute(select(func.count(DataSourceConfig.id)))
+    custom_config_count = int(config_result.scalar() or 0)
+
+    tv_payload = await get_tv_settings_payload(db)
+    tv_sources = tv_payload.get("sources") if isinstance(tv_payload, dict) else []
+    tv_source_count = len(tv_sources) if isinstance(tv_sources, list) else 0
+
+    boundary_status = get_boundary_status()
+    has_core_layers = bool(boundary_status.get("ready") or boundary_status.get("available") or boundary_status.get("status") in {"ready", "built", "ok"})
+    has_collected_data = current_record_count > 0
+    ready = has_collected_data
+
+    suggestions: list[str] = []
+    if demo_mode:
+        suggestions.append("演示模式已开启")
+    if not current_user:
+        suggestions.append("登录控制台")
+    if not has_collected_data:
+        suggestions.append("触发数据源采集")
+    if not custom_config_count:
+        suggestions.append("确认采集器配置")
+    if not has_core_layers:
+        suggestions.append("构建或启用 Earth 图层")
+
+    return {
+        "ready": ready,
+        "demo_mode": demo_mode,
+        "authenticated": current_user is not None,
+        "needs_login": current_user is None and not ready and not demo_mode,
+        "has_collected_data": has_collected_data,
+        "has_tv_sources": tv_source_count > 0,
+        "has_core_layers": has_core_layers,
+        "current_record_count": current_record_count,
+        "datasource_count": datasource_count,
+        "active_datasource_count": active_datasource_count,
+        "custom_config_count": custom_config_count,
+        "tv_source_count": tv_source_count,
+        "suggestions": suggestions,
+        "login_url": "/login?next=/datasources",
+        "datasources_url": "/datasources",
+        "collection_url": "/collection-management",
+    }
+
+
+@router.get("/boundaries/status")
+async def get_earth_boundary_status():
+    return get_boundary_status()
+
+def _is_loopback_request(request: Request) -> bool:
+    host = request.client.host if request.client else ""
+    return host in {"127.0.0.1", "::1", "localhost"} or host.startswith("127.")
+
+
+def _require_local_or_user(request: Request, user: User | None) -> None:
+    if user is not None or _is_loopback_request(request):
+        return
+    raise HTTPException(
+        status_code=status.HTTP_401_UNAUTHORIZED,
+        detail="Authentication required outside localhost",
+    )
+
+
+@router.put("/boundaries/config")
+async def update_earth_boundary_config(
+    payload: EarthBoundaryConfigPayload,
+    _current_user: User = Depends(get_current_user),
+):
+    try:
+        return save_boundary_config(payload.config)
+    except EarthBoundaryBuildError as exc:
+        raise HTTPException(
+            status_code=400,
+            detail={"code": exc.code, "message": str(exc), "details": exc.details},
+        ) from exc
+
+
+@router.post("/boundaries/build")
+async def build_earth_boundary_assets(
+    request: Request,
+    current_user: User | None = Depends(_get_optional_current_user),
+):
+    _require_local_or_user(request, current_user)
+    try:
+        return await start_boundary_build_job()
+    except EarthBoundaryBuildError as exc:
+        raise HTTPException(
+            status_code=400,
+            detail={"code": exc.code, "message": str(exc), "details": exc.details},
+        ) from exc
+
+
+@router.get("/boundaries/build/status")
+async def get_earth_boundary_build_status():
+    return get_boundary_build_status()

backend/app/api/v1/interactables.py

@@ -0,0 +1,190 @@
+"""CRUD APIs for persistent Earth interactables."""
+
+from __future__ import annotations
+
+from datetime import UTC, datetime
+from typing import Any
+
+from fastapi import APIRouter, Depends, HTTPException, Query, Response, status
+from pydantic import BaseModel, Field, field_validator
+from sqlalchemy.ext.asyncio import AsyncSession
+
+from app.core.security import get_current_user
+from app.db.session import get_db
+from app.models.earth_interactable import EarthInteractable
+from app.models.user import User
+from app.services.earth_interactables import (
+    build_interactable_event,
+    interactables_to_geojson,
+    invalidate_interactable_cache,
+    list_interactables,
+    normalize_interactable_id,
+    publish_interactable_event,
+    serialize_interactable,
+)
+from app.services.earth_layer_cache import (
+    EarthLayerCachePolicy,
+    earth_layer_cache,
+    get_or_build_layer_payload,
+)
+
+router = APIRouter()
+INTERACTABLE_CACHE_POLICY = EarthLayerCachePolicy(
+    fresh_ttl_seconds=60,
+    stale_ttl_seconds=10 * 60,
+    max_features=5000,
+)
+
+
+class InteractableCreate(BaseModel):
+    id: str | None = Field(default=None, max_length=160)
+    layer: str = Field(default="default", min_length=1, max_length=80)
+    kind: str = Field(default="default", min_length=1, max_length=80)
+    label: str = Field(default="", max_length=255)
+    description: str = Field(default="", max_length=4000)
+    latitude: float = Field(ge=-90, le=90)
+    longitude: float = Field(ge=-180, le=180)
+    altitude: float | None = None
+    properties: dict[str, Any] = Field(default_factory=dict)
+
+    @field_validator("layer", "kind")
+    @classmethod
+    def normalize_key(cls, value: str) -> str:
+        normalized = str(value or "").strip()
+        if not normalized:
+            raise ValueError("must not be empty")
+        return normalized
+
+
+class InteractableUpdate(BaseModel):
+    layer: str | None = Field(default=None, min_length=1, max_length=80)
+    kind: str | None = Field(default=None, min_length=1, max_length=80)
+    label: str | None = Field(default=None, max_length=255)
+    description: str | None = Field(default=None, max_length=4000)
+    latitude: float | None = Field(default=None, ge=-90, le=90)
+    longitude: float | None = Field(default=None, ge=-180, le=180)
+    altitude: float | None = None
+    properties: dict[str, Any] | None = None
+
+
+@router.get("")
+async def get_interactables(
+    response: Response,
+    layer: str | None = Query(default=None),
+    include_deleted: bool = Query(default=False),
+    db: AsyncSession = Depends(get_db),
+):
+    items = await list_interactables(db, layer=layer, include_deleted=include_deleted)
+    response.headers["X-Planet-Interactables-Count"] = str(len(items))
+    return {"items": [serialize_interactable(item) for item in items]}
+
+
+@router.get("/geojson")
+async def get_interactables_geojson(
+    response: Response,
+    layer: str | None = Query(default=None),
+    db: AsyncSession = Depends(get_db),
+):
+    async def build_payload() -> dict[str, Any]:
+        items = await list_interactables(db, layer=layer)
+        return interactables_to_geojson(items)
+
+    payload = await get_or_build_layer_payload(
+        key=earth_layer_cache.key("interactables", interactable_layer=layer or "all"),
+        policy=INTERACTABLE_CACHE_POLICY,
+        builder=build_payload,
+        response=response,
+    )
+    response.headers["X-Planet-Interactables-Count"] = str(len(payload.get("features") or []))
+    return payload
+
+
+@router.post("", status_code=status.HTTP_201_CREATED)
+async def create_interactable(
+    payload: InteractableCreate,
+    _current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    record_id = normalize_interactable_id(payload.id)
+    existing = await db.get(EarthInteractable, record_id)
+    if existing and not existing.is_deleted:
+        raise HTTPException(status_code=409, detail="Interactable already exists")
+
+    if existing is None:
+        record = EarthInteractable(id=record_id)
+        db.add(record)
+    else:
+        record = existing
+        record.is_deleted = False
+        record.deleted_at = None
+        record.revision += 1
+
+    record.layer = payload.layer
+    record.kind = payload.kind
+    record.label = payload.label
+    record.description = payload.description
+    record.latitude = payload.latitude
+    record.longitude = payload.longitude
+    record.altitude = payload.altitude
+    record.properties = payload.properties
+
+    await db.commit()
+    await db.refresh(record)
+    invalidate_interactable_cache(record.layer)
+    await publish_interactable_event("created", record)
+    return {"item": serialize_interactable(record)}
+
+
+@router.get("/{interactable_id}")
+async def get_interactable(interactable_id: str, db: AsyncSession = Depends(get_db)):
+    record = await db.get(EarthInteractable, interactable_id)
+    if record is None or record.is_deleted:
+        raise HTTPException(status_code=404, detail="Interactable not found")
+    return {"item": serialize_interactable(record)}
+
+
+@router.patch("/{interactable_id}")
+async def update_interactable(
+    interactable_id: str,
+    payload: InteractableUpdate,
+    _current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    record = await db.get(EarthInteractable, interactable_id)
+    if record is None or record.is_deleted:
+        raise HTTPException(status_code=404, detail="Interactable not found")
+
+    previous_layer = record.layer
+    patch = payload.model_dump(exclude_unset=True)
+    for key, value in patch.items():
+        setattr(record, key, value)
+    record.revision += 1
+
+    await db.commit()
+    await db.refresh(record)
+    invalidate_interactable_cache(previous_layer)
+    if record.layer != previous_layer:
+        invalidate_interactable_cache(record.layer)
+    await publish_interactable_event("updated", record)
+    return {"item": serialize_interactable(record)}
+
+
+@router.delete("/{interactable_id}")
+async def delete_interactable(
+    interactable_id: str,
+    _current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    record = await db.get(EarthInteractable, interactable_id)
+    if record is None or record.is_deleted:
+        raise HTTPException(status_code=404, detail="Interactable not found")
+
+    record.is_deleted = True
+    record.deleted_at = datetime.now(UTC)
+    record.revision += 1
+    await db.commit()
+    await db.refresh(record)
+    invalidate_interactable_cache(record.layer)
+    await publish_interactable_event("deleted", record)
+    event = build_interactable_event(action="deleted", record=record, include_item=True)
+    return {"deleted": True, "event": event}

backend/app/api/v1/layers.py

@@ -1,6 +1,6 @@
 from typing import Any, Optional
 
-from fastapi import APIRouter, Depends, HTTPException, Query
+from fastapi import APIRouter, Depends, HTTPException, Query, Response
 from sqlalchemy.ext.asyncio import AsyncSession
 
 from app.api.v1.visualization import (
@@ -105,11 +105,12 @@ def _parse_layer_bbox(bbox: str) -> tuple[float, float, float, float]:
 @router.get("/vessels/snapshot")
 async def get_vessel_layer_snapshot(
     bbox: str = Query(..., description="lon_min,lat_min,lon_max,lat_max"),
-    zoom: int = Query(..., ge=1, le=20),
+    zoom: float = Query(..., ge=1, le=20),
     limit: int = Query(DEFAULT_LAYER_LIMIT, ge=1),
     vessel_type: Optional[str] = Query(None, alias="type"),
     since_minutes: int = Query(60, ge=1, le=1440),
     db: AsyncSession = Depends(get_db),
+    response: Response = None,
 ):
     parsed_bbox = _parse_layer_bbox(bbox)
     return await build_vessel_snapshot_response(
@@ -119,6 +120,7 @@ async def get_vessel_layer_snapshot(
         limit=limit,
         type_filter=vessel_type,
         since_minutes=since_minutes,
+        response=response,
     )
 
 

backend/app/api/v1/news.py

@@ -1,13 +1,92 @@
-from fastapi import APIRouter, Query
+from fastapi import APIRouter, Depends, HTTPException, Query
+from sqlalchemy.ext.asyncio import AsyncSession
 
-from app.services.earth_news import get_earth_news_payload
+from app.db.session import get_db
+from app.services.earth_news import (
+    ALLOWED_NEWS_CATEGORY_KEYS,
+    SUPPORTED_NEWS_LOCALES,
+    REGION_ANCHORS,
+    get_earth_news_payload,
+)
 
 router = APIRouter()
 
 
+def _parse_categories(raw: str | None) -> set[str] | None:
+    if raw is None or not raw.strip():
+        return None
+    requested = {item.strip().lower() for item in raw.split(",") if item.strip()}
+    invalid = sorted(requested - set(ALLOWED_NEWS_CATEGORY_KEYS))
+    if invalid:
+        raise HTTPException(
+            status_code=422,
+            detail={
+                "message": "Unsupported news categories.",
+                "invalid_categories": invalid,
+                "allowed_categories": list(ALLOWED_NEWS_CATEGORY_KEYS),
+            },
+        )
+    return requested or None
+
+
+def _parse_source_ids(raw: str | None) -> set[str] | None:
+    if raw is None or not raw.strip():
+        return None
+    return {item.strip() for item in raw.split(",") if item.strip()} or None
+
+
+def _parse_limit(raw: int | None) -> int:
+    if raw is None:
+        return 12
+    if raw < 1:
+        raise HTTPException(status_code=422, detail={"message": "News limit must be greater than 0."})
+    return min(raw, 100)
+
+
+def _parse_locale(raw: str | None) -> str:
+    if raw is None or not raw.strip():
+        return "zh-CN"
+    requested = raw.strip()
+    if requested not in SUPPORTED_NEWS_LOCALES:
+        raise HTTPException(
+            status_code=422,
+            detail={
+                "message": "Unsupported news locale.",
+                "invalid_locale": requested,
+                "allowed_locales": sorted(SUPPORTED_NEWS_LOCALES),
+            },
+        )
+    return requested
+
+
 @router.get("/earth-feed")
 async def get_earth_feed(
     lat: float | None = Query(None, description="Current Earth view center latitude"),
     lon: float | None = Query(None, description="Current Earth view center longitude"),
+    region: str | None = Query(None, description="Explicit Earth news region for UE/client integrations"),
+    categories: str | None = Query(None, description="Comma-separated news category keys"),
+    sources: str | None = Query(None, description="Comma-separated news source ids"),
+    limit: int | None = Query(None, description="Maximum news items to return, capped at 100"),
+    locale: str | None = Query(None, description="Display locale, zh-CN or en-US"),
+    db: AsyncSession = Depends(get_db),
 ):
-    return await get_earth_news_payload(lat=lat, lon=lon)
+    normalized_region = region.strip().lower() if isinstance(region, str) and region.strip() else None
+    if normalized_region is not None and normalized_region not in REGION_ANCHORS:
+        raise HTTPException(
+            status_code=422,
+            detail={
+                "message": "Unsupported news region.",
+                "invalid_region": normalized_region,
+                "allowed_regions": list(REGION_ANCHORS.keys()),
+            },
+        )
+    return await get_earth_news_payload(
+        lat=lat,
+        lon=lon,
+        region=normalized_region,
+        categories=_parse_categories(categories),
+        source_ids=_parse_source_ids(sources),
+        limit=_parse_limit(limit),
+        locale=_parse_locale(locale),
+        db=db,
+    )

backend/app/api/v1/system_control.py

@@ -1,16 +1,19 @@
 from __future__ import annotations
 
 import os
+import secrets
 import subprocess
 import sys
 
 from datetime import datetime
 
-from fastapi import APIRouter, Depends, HTTPException, Query, Request, status
+from fastapi import APIRouter, Depends, Header, HTTPException, Query, Request, status
 from pydantic import BaseModel
+from sqlalchemy.ext.asyncio import AsyncSession
 
-from app.core.config import ROOT_DIR
+from app.core.config import ROOT_DIR, settings
 from app.core.security import get_current_user
+from app.db.session import get_db
 from app.models.user import User
 from app.services.persistent_logs import record_audit_log, record_system_log
 from app.services.system_control import (
@@ -33,8 +36,13 @@ from app.services.system_logs import (
     append_buffer_log,
     list_log_sources,
     normalize_log_level,
+    read_database_log_snapshot,
     read_log_snapshot,
+    read_observability_group_events,
+    read_observability_groups,
+    read_observability_raw_events,
 )
+from app.services.earth_layer_cache import earth_layer_cache
 
 router = APIRouter()
 
@@ -104,12 +112,115 @@ class EarthClientLogEventCreate(BaseModel):
     url: str | None = None
     module: str | None = None
     detail: str | None = None
+    fingerprint: str | None = None
+    occurrence_count: int = 1
+    metadata: dict[str, object] | None = None
 
 
 class EarthClientLogEventResponse(BaseModel):
     accepted: bool
     source_id: str
     level: str
+    fingerprint: str | None = None
+
+
+class ServiceLogEventCreate(BaseModel):
+    source: str = "ai-provider"
+    service: str = "ai-provider"
+    module: str | None = None
+    category: str | None = None
+    event: str = "service.runtime_log"
+    level: str = "error"
+    message: str
+    fingerprint: str | None = None
+    occurrence_count: int = 1
+    request_id: str | None = None
+    trace_id: str | None = None
+    task_id: str | None = None
+    source_id: int | str | None = None
+    provider: str | None = None
+    context: dict[str, object] | None = None
+
+
+async def ingest_client_log_event(
+    source_id: str,
+    *,
+    service: str,
+    event: str,
+    default_module: str,
+    default_category: str,
+    payload: EarthClientLogEventCreate,
+    request: Request,
+) -> EarthClientLogEventResponse:
+    normalized_level = normalize_log_level(payload.level)
+    append_buffer_log(
+        source_id,
+        level=normalized_level,
+        message=payload.message,
+        context={
+            "category": payload.category or "",
+            "url": payload.url or "",
+            "module": payload.module or "",
+            "detail": payload.detail or "",
+            "fingerprint": payload.fingerprint or "",
+            "occurrence_count": max(1, int(payload.occurrence_count or 1)),
+            "metadata": payload.metadata or {},
+        },
+    )
+    await record_system_log(
+        source=source_id,
+        service=service,
+        module=payload.module or default_module,
+        event=event,
+        level=normalized_level,
+        message=payload.message,
+        category=payload.category or default_category,
+        context={
+            "url": payload.url or "",
+            "detail": payload.detail or "",
+            "module": payload.module or "",
+            "client_ip": request.client.host if request.client else "",
+            "metadata": payload.metadata or {},
+        },
+        fingerprint=payload.fingerprint,
+        occurrence_count=max(1, int(payload.occurrence_count or 1)),
+    )
+    return EarthClientLogEventResponse(accepted=True, source_id=source_id, level=normalized_level, fingerprint=payload.fingerprint)
+
+
+def require_observability_ingest_token(
+    authorization: str | None,
+    ingest_token: str | None,
+) -> None:
+    expected_token = settings.OBSERVABILITY_INGEST_TOKEN.strip()
+    if not expected_token:
+        raise HTTPException(
+            status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
+            detail="Observability service ingestion is not configured",
+        )
+    provided = ""
+    if ingest_token:
+        provided = ingest_token.strip()
+    elif authorization:
+        scheme, _, token = authorization.partition(" ")
+        if scheme.lower() == "bearer":
+            provided = token.strip()
+    if not provided or not secrets.compare_digest(provided, expected_token):
+        raise HTTPException(
+            status_code=status.HTTP_403_FORBIDDEN,
+            detail="Invalid observability ingestion token",
+        )
+
+
+class EarthLayerCacheStatusResponse(BaseModel):
+    prefix: str
+    key_count: int
+    memory_bytes: int
+    layers: dict[str, dict[str, int]]
+
+
+class EarthLayerCacheClearResponse(BaseModel):
+    deleted: int
 
 
 def ensure_super_admin(current_user: User) -> None:
@@ -132,6 +243,34 @@ def validate_log_date(raw_value: str | None, field_name: str) -> str | None:
         ) from exc
 
 
+@router.get("/cache/earth-layers", response_model=EarthLayerCacheStatusResponse)
+async def get_earth_layer_cache_status(
+    current_user: User = Depends(get_current_user),
+):
+    ensure_super_admin(current_user)
+    try:
+        return earth_layer_cache.status()
+    except Exception as exc:
+        raise HTTPException(
+            status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
+            detail=f"Unable to read Earth layer cache status: {exc}",
+        ) from exc
+
+
+@router.delete("/cache/earth-layers", response_model=EarthLayerCacheClearResponse)
+async def clear_earth_layer_cache(
+    current_user: User = Depends(get_current_user),
+):
+    ensure_super_admin(current_user)
+    try:
+        return {"deleted": earth_layer_cache.delete_pattern()}
+    except Exception as exc:
+        raise HTTPException(
+            status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
+            detail=f"Unable to clear Earth layer cache: {exc}",
+        ) from exc
+
+
 @router.post("/restart-tasks", response_model=RestartTaskResponse)
 async def create_restart_task(
     payload: RestartTaskCreate,
@@ -270,7 +409,141 @@ async def get_system_log_sources(
     current_user: User = Depends(get_current_user),
 ):
     ensure_super_admin(current_user)
-    return {"items": list_log_sources()}
+    return {
+        "items": [
+            *list_log_sources(),
+            {
+                "source_id": "system-db",
+                "name": "系统事件",
+                "kind": "database",
+                "location": "table://system_logs",
+                "description": "后端持久化系统事件、AI 和采集器操作日志。",
+                "category": "database",
+                "status": "ok",
+            },
+            {
+                "source_id": "audit-db",
+                "name": "审计事件",
+                "kind": "database",
+                "location": "table://audit_logs",
+                "description": "管理员敏感操作和密钥 reveal 审计记录。",
+                "category": "audit",
+                "status": "ok",
+            },
+        ]
+    }
+
+
+@router.get("/logs/observability/groups")
+async def get_observability_log_groups(
+    limit: int = DEFAULT_LOG_LINE_LIMIT,
+    level: str = "all",
+    levels: str | None = Query(None, description="Comma-separated log levels"),
+    start_date: str | None = Query(None, description="Filter logs from this date (YYYY-MM-DD)"),
+    end_date: str | None = Query(None, description="Filter logs until this date (YYYY-MM-DD)"),
+    search: str | None = Query(None, description="Case-insensitive substring search"),
+    current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    ensure_super_admin(current_user)
+    if limit < 1 or limit > MAX_LOG_LINE_LIMIT:
+        raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail=f"limit must be between 1 and {MAX_LOG_LINE_LIMIT}")
+    normalized_start_date = validate_log_date(start_date, "start_date")
+    normalized_end_date = validate_log_date(end_date, "end_date")
+    if normalized_start_date and normalized_end_date and normalized_start_date > normalized_end_date:
+        raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail="start_date must be earlier than or equal to end_date")
+    return await read_observability_groups(
+        limit=limit,
+        level=level,
+        levels=levels,
+        start_date=normalized_start_date,
+        end_date=normalized_end_date,
+        search=search,
+        db=db,
+    )
+
+
+@router.get("/logs/observability/groups/{fingerprint}/events")
+async def get_observability_group_events(
+    fingerprint: str,
+    limit: int = DEFAULT_LOG_LINE_LIMIT,
+    current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    ensure_super_admin(current_user)
+    if limit < 1 or limit > MAX_LOG_LINE_LIMIT:
+        raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail=f"limit must be between 1 and {MAX_LOG_LINE_LIMIT}")
+    payload = await read_observability_group_events(fingerprint, limit=limit, db=db)
+    if payload is None:
+        raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Observability group not found")
+    return payload
+
+
+@router.get("/logs/observability/raw")
+async def get_observability_raw_events(
+    limit: int = DEFAULT_LOG_LINE_LIMIT,
+    level: str = "all",
+    levels: str | None = Query(None, description="Comma-separated log levels"),
+    start_date: str | None = Query(None, description="Filter logs from this date (YYYY-MM-DD)"),
+    end_date: str | None = Query(None, description="Filter logs until this date (YYYY-MM-DD)"),
+    search: str | None = Query(None, description="Case-insensitive substring search"),
+    current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    ensure_super_admin(current_user)
+    if limit < 1 or limit > MAX_LOG_LINE_LIMIT:
+        raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail=f"limit must be between 1 and {MAX_LOG_LINE_LIMIT}")
+    normalized_start_date = validate_log_date(start_date, "start_date")
+    normalized_end_date = validate_log_date(end_date, "end_date")
+    return await read_observability_raw_events(
+        limit=limit,
+        level=level,
+        levels=levels,
+        start_date=normalized_start_date,
+        end_date=normalized_end_date,
+        search=search,
+        db=db,
+    )
+
+
+@router.post("/logs/service", response_model=EarthClientLogEventResponse)
+async def ingest_service_log(
+    payload: ServiceLogEventCreate,
+    authorization: str | None = Header(default=None),
+    ingest_token: str | None = Header(default=None, alias="X-Planet-Observability-Token"),
+):
+    require_observability_ingest_token(authorization, ingest_token)
+    normalized_level = normalize_log_level(payload.level)
+    source = (payload.source or "ai-provider").strip() or "ai-provider"
+    context = dict(payload.context or {})
+    if payload.request_id:
+        context["request_id"] = payload.request_id
+    if payload.trace_id:
+        context["trace_id"] = payload.trace_id
+    if payload.task_id:
+        context["task_id"] = payload.task_id
+    if payload.source_id is not None:
+        context["source_id"] = payload.source_id
+    if payload.provider:
+        context["provider"] = payload.provider
+    await record_system_log(
+        source=source,
+        service=(payload.service or source).strip() or source,
+        module=payload.module or source,
+        event=(payload.event or "service.runtime_log").strip() or "service.runtime_log",
+        level=normalized_level,
+        message=payload.message,
+        category=payload.category or "service-runtime",
+        context=context,
+        fingerprint=payload.fingerprint,
+        occurrence_count=max(1, int(payload.occurrence_count or 1)),
+    )
+    return EarthClientLogEventResponse(
+        accepted=True,
+        source_id=source,
+        level=normalized_level,
+        fingerprint=payload.fingerprint,
+    )
 
 
 @router.get("/logs/{source_id}", response_model=SystemLogSnapshotResponse)
@@ -283,6 +556,7 @@ async def get_system_log_snapshot(
     end_date: str | None = Query(None, description="Filter logs until this date (YYYY-MM-DD)"),
     search: str | None = Query(None, description="Case-insensitive substring search"),
     current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
 ):
     ensure_super_admin(current_user)
 
@@ -305,15 +579,26 @@ async def get_system_log_snapshot(
     if normalized_start_date and normalized_end_date and normalized_start_date > normalized_end_date:
         raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail="start_date must be earlier than or equal to end_date")
 
-    snapshot = read_log_snapshot(
+    snapshot = await read_database_log_snapshot(
         source_id,
-        limit,
+        limit=limit,
         level=level,
         levels=levels,
         start_date=normalized_start_date,
         end_date=normalized_end_date,
         search=search,
+        db=db,
     )
+    if snapshot is None:
+        snapshot = read_log_snapshot(
+            source_id,
+            limit,
+            level=level,
+            levels=levels,
+            start_date=normalized_start_date,
+            end_date=normalized_end_date,
+            search=search,
+        )
     if snapshot is None:
         raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Log source not found")
     return snapshot
@@ -324,31 +609,28 @@ async def ingest_earth_client_log(
     payload: EarthClientLogEventCreate,
     request: Request,
 ):
-    normalized_level = normalize_log_level(payload.level)
-    append_buffer_log(
+    return await ingest_client_log_event(
         "earth-client",
-        level=normalized_level,
-        message=payload.message,
-        context={
-            "category": payload.category or "",
-            "url": payload.url or "",
-            "module": payload.module or "",
-            "detail": payload.detail or "",
-        },
-    )
-    await record_system_log(
-        source="earth-client",
         service="earth",
-        module=payload.module or "earth-client",
         event="earth.client.runtime_log",
-        level=normalized_level,
-        message=payload.message,
-        category=payload.category or "client-runtime",
-        context={
-            "url": payload.url or "",
-            "detail": payload.detail or "",
-            "module": payload.module or "",
-            "client_ip": request.client.host if request.client else "",
-        },
+        default_module="earth-client",
+        default_category="client-runtime",
+        payload=payload,
+        request=request,
+    )
+
+
+@router.post("/logs/admin-client", response_model=EarthClientLogEventResponse)
+async def ingest_admin_client_log(
+    payload: EarthClientLogEventCreate,
+    request: Request,
+):
+    return await ingest_client_log_event(
+        "admin-client",
+        service="admin",
+        event="admin.client.runtime_log",
+        default_module="admin-client",
+        default_category="client-runtime",
+        payload=payload,
+        request=request,
     )
-    return {"accepted": True, "source_id": "earth-client", "level": normalized_level}

backend/app/api/v1/tasks.py

@@ -29,7 +29,8 @@ async def list_tasks(
         SELECT ct.id, ct.datasource_id, ds.name as datasource_name, ct.status,
                ct.started_at, ct.completed_at, ct.records_processed, ct.error_message,
                ct.phase, ct.phase_progress, ct.phase_message, ct.phase_current,
-               ct.phase_total, ct.phase_unit, ct.total_records, ct.progress
+               ct.phase_total, ct.phase_unit, ct.total_records, ct.progress,
+               ct.task_type, ct.source, ds.source as datasource_source
         FROM collection_tasks ct
         JOIN data_sources ds ON ct.datasource_id = ds.id
         WHERE 1=1
@@ -39,12 +40,19 @@ async def list_tasks(
 
     if datasource_id:
         query += " AND ct.datasource_id = :datasource_id"
-        count_query += " WHERE ct.datasource_id = :datasource_id"
+        count_query += " AND ct.datasource_id = :datasource_id"
         params["datasource_id"] = datasource_id
     if status:
-        query += " AND ct.status = :status"
-        count_query += " AND ct.status = :status"
-        params["status"] = status
+        statuses = [item.strip() for item in status.split(",") if item.strip()]
+        if len(statuses) > 1:
+            placeholders = ", ".join(f":status_{index}" for index, _item in enumerate(statuses))
+            query += f" AND ct.status IN ({placeholders})"
+            count_query += f" AND ct.status IN ({placeholders})"
+            params.update({f"status_{index}": item for index, item in enumerate(statuses)})
+        else:
+            query += " AND ct.status = :status"
+            count_query += " AND ct.status = :status"
+            params["status"] = statuses[0] if statuses else status
 
     query += f" ORDER BY ct.created_at DESC LIMIT {page_size} OFFSET {offset}"
 
@@ -76,6 +84,9 @@ async def list_tasks(
                 "phase_unit": t[13],
                 "total_records": t[14],
                 "progress": t[15],
+                "task_type": t[16],
+                "source": t[17] or t[18],
+                "datasource_source": t[18],
             }
             for t in tasks
         ],

backend/app/api/v1/tv.py

@@ -1,3 +1,4 @@
+import re
 from urllib.parse import quote, urljoin
 
 import httpx
@@ -6,16 +7,41 @@ from fastapi.responses import Response
 from sqlalchemy.ext.asyncio import AsyncSession
 
 from app.db.session import get_db
+from app.services.tv_catalog import get_tv_catalog_page
 from app.services.tv_streams import get_public_tv_payload, is_allowed_tv_proxy_url
 
 router = APIRouter()
 
+_HLS_URI_ATTRIBUTE_RE = re.compile(r'URI="([^"]+)"')
+
+
+def _proxied_tv_url(url: str) -> str:
+    return f"/api/v1/tv/proxy?url={quote(url, safe='')}"
+
+
+def _rewrite_hls_uri_attributes(line: str, *, base_url: str) -> str:
+    def replace(match: re.Match[str]) -> str:
+        uri = match.group(1)
+        absolute_url = urljoin(base_url, uri)
+        return f'URI="{_proxied_tv_url(absolute_url)}"'
+
+    return _HLS_URI_ATTRIBUTE_RE.sub(replace, line)
+
+
+def _should_strip_hls_metadata_line(line: str) -> bool:
+    normalized = line.strip().upper()
+    return normalized.startswith("#EXT-X-MEDIA:") and "TYPE=SUBTITLES" in normalized
+
 
 @router.get("/streams")
 async def list_public_tv_streams(
+    offset: int = Query(0, ge=0),
+    limit: int = Query(50, ge=1, le=100),
+    q: str = Query("", max_length=200),
+    selected_id: str | None = Query(None, max_length=200),
     db: AsyncSession = Depends(get_db),
 ):
-    return await get_public_tv_payload(db)
+    return await get_tv_catalog_page(db, offset=offset, limit=limit, q=q, selected_id=selected_id)
 
 
 @router.get("/proxy")
@@ -56,11 +82,16 @@ async def proxy_tv_stream(
         rewritten_lines: list[str] = []
         for line in manifest_text.splitlines():
             stripped = line.strip()
-            if not stripped or stripped.startswith("#"):
+            if not stripped:
                 rewritten_lines.append(line)
                 continue
+            if stripped.startswith("#"):
+                if _should_strip_hls_metadata_line(line):
+                    continue
+                rewritten_lines.append(_rewrite_hls_uri_attributes(line, base_url=response_url))
+                continue
             absolute_url = urljoin(response_url, stripped)
-            rewritten_lines.append(f"/api/v1/tv/proxy?url={quote(absolute_url, safe='')}")
+            rewritten_lines.append(_proxied_tv_url(absolute_url))
         return Response(
             content="\n".join(rewritten_lines),
             media_type="application/vnd.apple.mpegurl",

backend/app/api/v1/users.py

@@ -5,6 +5,7 @@ from fastapi import APIRouter, Depends, HTTPException, status
 from sqlalchemy.ext.asyncio import AsyncSession
 from sqlalchemy import text
 
+from app.core.enums import UserRole
 from app.core.security import get_current_user, get_password_hash
 from app.db.session import get_db
 from app.models.user import User
@@ -13,6 +14,8 @@ from app.schemas.user import UserCreate, UserUpdate
 router = APIRouter()
 
 VALID_GATEKEEPER_GROUPS = {"docs_user", "docs_developer", "docs_admin"}
+ADMIN_ROLES = [UserRole.SUPER_ADMIN.value, UserRole.ADMIN.value]
+SUPER_ADMIN_ROLES = [UserRole.SUPER_ADMIN.value]
 
 
 def check_permission(current_user: User, required_roles: List[str]) -> bool:
@@ -32,7 +35,7 @@ async def list_users(
     current_user: User = Depends(get_current_user),
     db: AsyncSession = Depends(get_db),
 ):
-    if not check_permission(current_user, ["super_admin", "admin"]):
+    if not check_permission(current_user, ADMIN_ROLES):
         raise HTTPException(
             status_code=status.HTTP_403_FORBIDDEN,
             detail="Insufficient permissions",
@@ -91,7 +94,7 @@ async def get_user(
     current_user: User = Depends(get_current_user),
     db: AsyncSession = Depends(get_db),
 ):
-    if not check_permission(current_user, ["super_admin", "admin"]) and current_user.id != user_id:
+    if not check_permission(current_user, ADMIN_ROLES) and current_user.id != user_id:
         raise HTTPException(
             status_code=status.HTTP_403_FORBIDDEN,
             detail="Insufficient permissions",
@@ -128,7 +131,7 @@ async def create_user(
     current_user: User = Depends(get_current_user),
     db: AsyncSession = Depends(get_db),
 ):
-    if not check_permission(current_user, ["super_admin"]):
+    if not check_permission(current_user, SUPER_ADMIN_ROLES):
         raise HTTPException(
             status_code=status.HTTP_403_FORBIDDEN,
             detail="Only super_admin can create users",
@@ -196,18 +199,18 @@ async def update_user(
     current_user: User = Depends(get_current_user),
     db: AsyncSession = Depends(get_db),
 ):
-    if not check_permission(current_user, ["super_admin", "admin"]) and current_user.id != user_id:
+    if not check_permission(current_user, ADMIN_ROLES) and current_user.id != user_id:
         raise HTTPException(
             status_code=status.HTTP_403_FORBIDDEN,
             detail="Insufficient permissions",
         )
 
-    if not check_permission(current_user, ["super_admin"]) and user_data.role is not None:
+    if not check_permission(current_user, SUPER_ADMIN_ROLES) and user_data.role is not None:
         raise HTTPException(
             status_code=status.HTTP_403_FORBIDDEN,
             detail="Only super_admin can change user role",
         )
-    if not check_permission(current_user, ["super_admin"]) and user_data.gatekeeper_groups is not None:
+    if not check_permission(current_user, SUPER_ADMIN_ROLES) and user_data.gatekeeper_groups is not None:
         raise HTTPException(
             status_code=status.HTTP_403_FORBIDDEN,
             detail="Only super_admin can change Gatekeeper groups",
@@ -260,7 +263,7 @@ async def delete_user(
     current_user: User = Depends(get_current_user),
     db: AsyncSession = Depends(get_db),
 ):
-    if not check_permission(current_user, ["super_admin"]):
+    if not check_permission(current_user, SUPER_ADMIN_ROLES):
         raise HTTPException(
             status_code=status.HTTP_403_FORBIDDEN,
             detail="Only super_admin can delete users",

backend/app/api/v1/vessels.py

@@ -1,8 +1,8 @@
-"""Bounded vessel snapshot APIs for viewport-first consumers."""
+"""Bounded vessel snapshot APIs backed by the latest vessel state table."""
 
 from typing import Optional
 
-from fastapi import APIRouter, Depends, HTTPException, Query
+from fastapi import APIRouter, Depends, HTTPException, Query, Response
 from sqlalchemy.ext.asyncio import AsyncSession
 
 from app.api.v1.visualization import _parse_bbox, build_vessel_snapshot_response
@@ -14,8 +14,8 @@ router = APIRouter()
 
 @router.get("/snapshot")
 async def get_vessel_snapshot(
-    bbox: Optional[str] = Query(None, description="Viewport bbox as lon_min,lat_min,lon_max,lat_max"),
-    zoom: int = Query(..., ge=1, le=20, description="Current map zoom level"),
+    bbox: Optional[str] = Query(None, description="Snapshot bbox as lon_min,lat_min,lon_max,lat_max"),
+    zoom: float = Query(..., ge=1, le=20, description="Current map zoom level"),
     type: Optional[str] = Query(
         None,
         description="Comma-separated vessel types: cargo,tanker,passenger,fishing,military,other",
@@ -23,6 +23,7 @@ async def get_vessel_snapshot(
     limit: int = Query(1000, ge=1, le=MAX_SNAPSHOT_LIMIT),
     since_minutes: int = Query(60, ge=1, le=1440),
     db: AsyncSession = Depends(get_db),
+    response: Response = None,
 ):
     if not bbox:
         raise HTTPException(status_code=400, detail="bbox is required")
@@ -36,4 +37,5 @@ async def get_vessel_snapshot(
         type_filter=type,
         limit=limit,
         since_minutes=since_minutes,
+        response=response,
     )

backend/app/api/v1/visualization.py

@@ -18,6 +18,7 @@ from sqlalchemy import select, func
 from typing import List, Dict, Any, Optional
 
 from app.core.collected_data_fields import get_record_field
+from app.core.enums import BGPStatus
 from app.core.satellite_tle import build_tle_lines_from_elements
 from app.core.time import to_iso8601_utc
 from app.db.session import get_db
@@ -25,7 +26,7 @@ from app.models.bgp_anomaly import BGPAnomaly
 from app.models.bgp_incident import BGPIncident
 from app.models.bgp_observation import BGPObservation
 from app.models.collected_data import CollectedData
-from app.models.vessel import AISSourceHealth, VesselPosition, VesselStatic
+from app.models.vessel import AISSourceHealth, VesselCurrentState, VesselPosition, VesselStatic
 from app.services.bgp_collectors import build_bgp_collector_coverage
 from app.services.cable_graph import build_graph_from_data, CableGraph, haversine_distance
 from app.services.compute_center_locations import (
@@ -38,7 +39,7 @@ from app.services.compute_center_locations import (
     upsert_compute_center_location,
 )
 from app.services.ai_client import get_ai_provider_client
-from app.api.v1.settings import get_web_search_client
+from app.api.v1.settings import get_runtime_web_search_config, get_web_search_client
 from app.services.collectors.bgp_common import RIPE_RIS_COLLECTOR_COORDS
 from app.services.location.llm_fallback import (
     collect_llm_location_fallback_candidate,
@@ -47,15 +48,21 @@ from app.services.location.llm_fallback import (
 from app.services.persistent_logs import record_system_log
 from app.services.vessel_ais_aggregation import (
     build_field_conflict_candidates,
-    count_unique_raw_vessel_mmsi,
     get_aggregated_vessel,
     get_aggregated_vessel_track,
     get_aggregated_vessels,
-    get_aggregated_vessels_snapshot,
+    get_current_vessels_snapshot,
     get_vessel_conflict_records,
     get_vessel_raw_observations,
     MAX_SNAPSHOT_LIMIT,
 )
+from app.services.earth_layer_cache import (
+    EarthLayerCachePolicy,
+    earth_layer_cache,
+    format_bbox_key,
+    get_or_build_layer_payload,
+    quantize_bbox,
+)
 from app.core.logging import get_logger
 
 router = APIRouter()
@@ -68,7 +75,68 @@ TERRAIN_TILE_BATCH_MAX_ITEMS = 128
 TERRAIN_TILE_BATCH_CONCURRENCY = 16
 _terrain_tile_cache: OrderedDict[tuple[int, int, int], tuple[bytes, str, dict[str, str]]] = OrderedDict()
 VESSEL_NAME_FALLBACK_PATTERN = re.compile(r"^mmsi\s*\d+$", re.IGNORECASE)
-VESSEL_SNAPSHOT_LEGACY_FALLBACK_ENABLED = True
+SECONDS_PER_MINUTE = 60
+BYTES_PER_MIB = 1024 * 1024
+CABLE_CACHE_FRESH_SECONDS = 6 * 60 * SECONDS_PER_MINUTE
+CABLE_CACHE_STALE_SECONDS = 24 * 60 * SECONDS_PER_MINUTE
+SATELLITE_CACHE_FRESH_SECONDS = 15 * SECONDS_PER_MINUTE
+SATELLITE_CACHE_STALE_SECONDS = 2 * 60 * SECONDS_PER_MINUTE
+COMPUTE_CENTER_CACHE_FRESH_SECONDS = 10 * SECONDS_PER_MINUTE
+COMPUTE_CENTER_CACHE_STALE_SECONDS = 60 * SECONDS_PER_MINUTE
+BGP_CACHE_FRESH_SECONDS = 60
+BGP_EVENT_CACHE_FRESH_SECONDS = 30
+BGP_CACHE_STALE_SECONDS = 10 * SECONDS_PER_MINUTE
+VESSEL_SNAPSHOT_CACHE_FRESH_SECONDS = 5
+VESSEL_SNAPSHOT_CACHE_STALE_SECONDS = 30
+
+CABLE_CACHE_POLICY = EarthLayerCachePolicy(
+    CABLE_CACHE_FRESH_SECONDS,
+    CABLE_CACHE_STALE_SECONDS,
+    max_features=6000,
+    max_bytes=10 * BYTES_PER_MIB,
+)
+LANDING_POINT_CACHE_POLICY = EarthLayerCachePolicy(
+    CABLE_CACHE_FRESH_SECONDS,
+    CABLE_CACHE_STALE_SECONDS,
+    max_features=6000,
+    max_bytes=8 * BYTES_PER_MIB,
+)
+SATELLITE_CACHE_POLICY = EarthLayerCachePolicy(
+    SATELLITE_CACHE_FRESH_SECONDS,
+    SATELLITE_CACHE_STALE_SECONDS,
+    max_features=25000,
+    max_bytes=32 * BYTES_PER_MIB,
+)
+COMPUTE_CENTER_CACHE_POLICY = EarthLayerCachePolicy(
+    COMPUTE_CENTER_CACHE_FRESH_SECONDS,
+    COMPUTE_CENTER_CACHE_STALE_SECONDS,
+    max_features=1000,
+    max_bytes=4 * BYTES_PER_MIB,
+)
+BGP_CACHE_POLICY = EarthLayerCachePolicy(
+    BGP_CACHE_FRESH_SECONDS,
+    BGP_CACHE_STALE_SECONDS,
+    max_features=1000,
+    max_bytes=3 * BYTES_PER_MIB,
+)
+BGP_EVENT_CACHE_POLICY = EarthLayerCachePolicy(
+    BGP_EVENT_CACHE_FRESH_SECONDS,
+    BGP_CACHE_STALE_SECONDS,
+    max_features=1000,
+    max_bytes=3 * BYTES_PER_MIB,
+)
+SUMMARY_CACHE_POLICY = EarthLayerCachePolicy(
+    BGP_EVENT_CACHE_FRESH_SECONDS,
+    BGP_CACHE_STALE_SECONDS,
+    max_features=0,
+    max_bytes=512 * 1024,
+)
+VESSEL_SNAPSHOT_CACHE_POLICY = EarthLayerCachePolicy(
+    VESSEL_SNAPSHOT_CACHE_FRESH_SECONDS,
+    VESSEL_SNAPSHOT_CACHE_STALE_SECONDS,
+    max_features=1500,
+    max_bytes=3 * BYTES_PER_MIB,
+)
 
 
 class TerrariumTileRequest(BaseModel):
@@ -770,9 +838,14 @@ def convert_aggregated_vessels_to_geojson(vessels: List[dict[str, Any]]) -> Dict
             continue
         source_summary = {}
         for source, summary in (vessel.get("source_summary") or {}).items():
+            latest_observed_at = summary.get("latest_observed_at")
             source_summary[source] = {
                 **summary,
-                "latest_observed_at": to_iso8601_utc(summary.get("latest_observed_at")),
+                "latest_observed_at": (
+                    to_iso8601_utc(latest_observed_at)
+                    if isinstance(latest_observed_at, datetime)
+                    else latest_observed_at
+                ),
             }
         props = {
             "mmsi": vessel["mmsi"],
@@ -1006,15 +1079,49 @@ async def build_vessel_snapshot_response(
     db: AsyncSession,
     *,
     bbox: tuple[float, float, float, float] | None,
-    zoom: int | None,
+    zoom: float | None,
     type_filter: str | None,
     limit: int | None,
     since_minutes: int = 60,
+    response: Response | None = None,
+    use_cache: bool = True,
 ) -> dict[str, Any]:
+    if use_cache and bbox is not None:
+        safe_limit_for_key = _safe_vessel_limit(limit)
+        safe_since_for_key = min(max(int(since_minutes or 60), 1), 1440)
+        cache_key = earth_layer_cache.key(
+            "vessels-snapshot",
+            bbox=format_bbox_key(quantize_bbox(bbox)),
+            zoom=zoom or "none",
+            type=type_filter or "all",
+            limit=safe_limit_for_key,
+            since=safe_since_for_key,
+        )
+
+        async def build_uncached() -> dict[str, Any]:
+            return await build_vessel_snapshot_response(
+                db,
+                bbox=bbox,
+                zoom=zoom,
+                type_filter=type_filter,
+                limit=limit,
+                since_minutes=since_minutes,
+                response=None,
+                use_cache=False,
+            )
+
+        return await get_or_build_layer_payload(
+            key=cache_key,
+            policy=VESSEL_SNAPSHOT_CACHE_POLICY,
+            builder=build_uncached,
+            response=response,
+        )
+
     requested_types = _requested_vessel_types(type_filter)
     safe_limit = _safe_vessel_limit(limit)
     safe_since_minutes = min(max(int(since_minutes or 60), 1), 1440)
     observed_since = datetime.now(UTC) - timedelta(minutes=safe_since_minutes)
+    snapshot_started_at = datetime.now(UTC)
     features, diagnostics = await _load_raw_vessel_snapshot_features(
         db,
         bbox=bbox,
@@ -1031,6 +1138,7 @@ async def build_vessel_snapshot_response(
         "features": features,
         "count": len(features),
         "stats": _build_vessel_stats(features),
+        "generated_at": to_iso8601_utc(snapshot_started_at),
         "diagnostics": {
             **diagnostics,
             "filtered_count": len(features),
@@ -1444,20 +1552,24 @@ def convert_bgp_incidents_to_geojson(
 
 
 @router.get("/geo/cables")
-async def get_cables_geojson(db: AsyncSession = Depends(get_db)):
+async def get_cables_geojson(db: AsyncSession = Depends(get_db), response: Response = None):
     """获取海底电缆 GeoJSON 数据 (LineString)"""
+    async def build_payload() -> dict[str, Any]:
+        return await _build_cables_geojson(db)
+
+    return await get_or_build_layer_payload(
+        key=earth_layer_cache.key("cables"),
+        policy=CABLE_CACHE_POLICY,
+        builder=build_payload,
+        response=response,
+    )
+
+
+async def _build_cables_geojson(db: AsyncSession) -> dict[str, Any]:
     try:
         records = await _load_current_collected_data(db, "arcgis_cables")
 
-        if not records:
-            raise HTTPException(
-                status_code=404,
-                detail="No cable data found. Please run the arcgis_cables collector first.",
-            )
-
         return convert_cable_to_geojson(records)
-    except HTTPException:
-        raise
     except Exception as e:
         logger.exception_event(
             "Failed to build cables GeoJSON response",
@@ -1478,7 +1590,19 @@ async def get_cables_geojson(db: AsyncSession = Depends(get_db)):
 
 
 @router.get("/geo/landing-points")
-async def get_landing_points_geojson(db: AsyncSession = Depends(get_db)):
+async def get_landing_points_geojson(db: AsyncSession = Depends(get_db), response: Response = None):
+    async def build_payload() -> dict[str, Any]:
+        return await _build_landing_points_geojson(db)
+
+    return await get_or_build_layer_payload(
+        key=earth_layer_cache.key("landing-points"),
+        policy=LANDING_POINT_CACHE_POLICY,
+        builder=build_payload,
+        response=response,
+    )
+
+
+async def _build_landing_points_geojson(db: AsyncSession) -> dict[str, Any]:
     try:
         records_by_source = await _load_current_collected_data_by_sources(
             db,
@@ -1499,16 +1623,8 @@ async def get_landing_points_geojson(db: AsyncSession = Depends(get_db)):
             relation_records,
             cable_records,
         )
-        
-        if not records:
-            raise HTTPException(
-                status_code=404,
-                detail="No landing point data found. Please run the arcgis_landing_points collector first.",
-            )
-        
+
         return convert_landing_point_to_geojson(records, city_to_cable_ids_map, cable_id_to_name_map)
-    except HTTPException:
-        raise
     except Exception as e:
         logger.exception_event(
             "Failed to build landing points GeoJSON response",
@@ -1731,8 +1847,25 @@ async def get_satellites_geojson(
         description="Maximum number of satellites to return. Omit for no limit.",
     ),
     db: AsyncSession = Depends(get_db),
+    response: Response = None,
 ):
     """获取卫星 TLE GeoJSON 数据"""
+    async def build_payload() -> dict[str, Any]:
+        return await _build_satellites_geojson(limit=limit, db=db)
+
+    return await get_or_build_layer_payload(
+        key=earth_layer_cache.key("satellites", limit=limit or "all"),
+        policy=SATELLITE_CACHE_POLICY,
+        builder=build_payload,
+        response=response,
+    )
+
+
+async def _build_satellites_geojson(
+    *,
+    limit: int | None,
+    db: AsyncSession,
+) -> dict[str, Any]:
     records = await _load_current_or_latest_task_data(
         db,
         "celestrak_tle",
@@ -1800,8 +1933,25 @@ async def get_gpu_clusters_geojson(
 async def get_compute_centers_geojson(
     limit: int = Query(200, ge=1, le=1000),
     db: AsyncSession = Depends(get_db),
+    response: Response = None,
 ):
     """获取统一算力中心 GeoJSON 数据"""
+    async def build_payload() -> dict[str, Any]:
+        return await _build_compute_centers_geojson(limit=limit, db=db)
+
+    return await get_or_build_layer_payload(
+        key=earth_layer_cache.key("compute-centers", limit=limit),
+        policy=COMPUTE_CENTER_CACHE_POLICY,
+        builder=build_payload,
+        response=response,
+    )
+
+
+async def _build_compute_centers_geojson(
+    *,
+    limit: int,
+    db: AsyncSession,
+) -> dict[str, Any]:
     records_by_source = await _load_current_collected_data_by_sources(
         db,
         ["top500", "epoch_ai_gpu"],
@@ -1911,6 +2061,44 @@ class SaveComputeCenterLocationRequest(BaseModel):
     model_config = {"populate_by_name": True}
 
 
+async def _compute_center_location_web_search_capability(db: AsyncSession) -> Dict[str, Any]:
+    try:
+        config = await get_runtime_web_search_config(db)
+    except Exception as exc:
+        return {
+            "enabled": False,
+            "provider": None,
+            "reason": f"WebSearch 配置读取失败：{exc}",
+        }
+    provider_config = config.active_provider_config
+    has_api_key = bool((provider_config.api_key or "").strip())
+    if not config.enabled:
+        return {
+            "enabled": False,
+            "provider": config.default_provider,
+            "reason": "WebSearch 未开启，无法进行事实核查定位。",
+        }
+    if not has_api_key:
+        return {
+            "enabled": False,
+            "provider": config.default_provider,
+            "reason": f"WebSearch Provider {config.default_provider} 未配置 API Key。",
+        }
+    return {
+        "enabled": True,
+        "provider": config.default_provider,
+        "reason": "",
+    }
+
+
+@router.get("/compute-centers/location-capability")
+async def get_compute_center_location_capability(
+    db: AsyncSession = Depends(get_db),
+):
+    """Return whether fact-checked compute-center location collection can run."""
+    return await _compute_center_location_web_search_capability(db)
+
+
 @router.post("/compute-centers/{source_id}/collect-location")
 async def collect_compute_center_location(
     source_id: str,
@@ -1929,6 +2117,9 @@ async def collect_compute_center_location(
     """
     if not source_id or not source_id.strip():
         raise HTTPException(status_code=400, detail="source_id is required")
+    capability = await _compute_center_location_web_search_capability(db)
+    if not capability.get("enabled"):
+        raise HTTPException(status_code=409, detail=capability)
 
     record = await _load_compute_center_record(db, source_id)
     name = payload.name or (record.name if record else None)
@@ -1982,6 +2173,7 @@ async def collect_compute_center_location(
                 provider_client=provider_client,
                 query=query,
                 entity_type="compute_center",
+                db=db,
                 attempted_queries=attempted_queries,
                 search_evidence=search_result.evidence,
             )
@@ -1998,6 +2190,24 @@ async def collect_compute_center_location(
             llm_failure_reason = llm_result.failure_reason
 
     if not candidates:
+        logger.warning_event(
+            "Compute center location collection returned no candidates",
+            event="visualization.compute_center.location_collect.completed",
+            context={
+                "source_id": source_id,
+                "record_id": record_id,
+                "name": name,
+                "success": False,
+                "llm_failure_reason": llm_failure_reason,
+                "attempted_queries": list(attempted_queries),
+                "context": {
+                    "operator": operator,
+                    "site": site,
+                    "city": city,
+                    "country": country,
+                },
+            },
+        )
         return {
             "source_id": source_id,
             "record_id": record_id,
@@ -2019,13 +2229,34 @@ async def collect_compute_center_location(
             },
         }
 
+    best_candidate = candidates[0].to_dict()
+    logger.info_event(
+        "Compute center location collection returned candidates",
+        event="visualization.compute_center.location_collect.completed",
+        context={
+            "source_id": source_id,
+            "record_id": record_id,
+            "name": name,
+            "success": True,
+            "candidate_count": len(candidates),
+            "best_candidate": best_candidate,
+            "llm_failure_reason": llm_failure_reason,
+            "attempted_queries": list(attempted_queries),
+            "context": {
+                "operator": operator,
+                "site": site,
+                "city": city,
+                "country": country,
+            },
+        },
+    )
     return {
         "source_id": source_id,
         "record_id": record_id,
         "name": name,
         "success": True,
         "candidates": [candidate.to_dict() for candidate in candidates],
-        "best_candidate": candidates[0].to_dict(),
+        "best_candidate": best_candidate,
         "attempted_queries": list(attempted_queries),
         "context": {
             "name": name,
@@ -2115,58 +2346,35 @@ async def _load_raw_vessel_snapshot_features(
     observed_since: datetime,
 ) -> tuple[list[dict[str, Any]], dict[str, Any]]:
     if bbox is None:
-        aggregated_vessels = await get_aggregated_vessels(
-            db,
-            limit=limit,
-            observed_since=observed_since,
-        )
-    else:
-        aggregated_vessels = await get_aggregated_vessels_snapshot(
-            db,
-            bbox=bbox,
-            limit=limit,
-            observed_since=observed_since,
-        )
-    raw_geojson = convert_aggregated_vessels_to_geojson(aggregated_vessels)
-    raw_features = raw_geojson.get("features", [])
-    features = raw_features
-    legacy_features: list[dict[str, Any]] = []
-    legacy_fallback_used = False
-    if not raw_features and VESSEL_SNAPSHOT_LEGACY_FALLBACK_ENABLED:
-        legacy_features = await _load_legacy_vessel_snapshot_features(
-            db,
-            bbox=bbox,
-            limit=limit,
-        )
-        features, _merge_diagnostics = _merge_vessel_features(raw_features, legacy_features)
-        legacy_fallback_used = bool(legacy_features)
-
+        return [], {
+            "source": "vessel_current_state",
+            "current_state_count": 0,
+            "final_unique_mmsi": 0,
+        }
+    current_vessels = await get_current_vessels_snapshot(
+        db,
+        bbox=bbox,
+        limit=limit,
+        observed_since=observed_since,
+    )
+    features = convert_aggregated_vessels_to_geojson(current_vessels).get("features", [])
+    unique_mmsi = len(
+        {
+            key
+            for key in (_feature_mmsi_key(feature) for feature in features)
+            if key is not None
+        }
+    )
     return features, {
-        "raw_feature_count": len(raw_features),
-        "raw_unique_mmsi": len(
-            {
-                key
-                for key in (_feature_mmsi_key(feature) for feature in raw_features)
-                if key is not None
-            }
-        ),
-        "legacy_feature_count": len(legacy_features),
-        "legacy_backfilled_mmsi": len(
-            {
-                key
-                for key in (_feature_mmsi_key(feature) for feature in legacy_features)
-                if key is not None
-            }
-        ),
-        "legacy_fallback_enabled": VESSEL_SNAPSHOT_LEGACY_FALLBACK_ENABLED,
-        "legacy_fallback_used": legacy_fallback_used,
-        "final_unique_mmsi": len(
-            {
-                key
-                for key in (_feature_mmsi_key(feature) for feature in features)
-                if key is not None
-            }
-        ),
+        "source": "vessel_current_state",
+        "current_state_count": len(features),
+        "final_unique_mmsi": unique_mmsi,
+        "raw_feature_count": 0,
+        "raw_unique_mmsi": 0,
+        "legacy_feature_count": 0,
+        "legacy_backfilled_mmsi": 0,
+        "legacy_fallback_enabled": False,
+        "legacy_fallback_used": False,
     }
 
 @router.get("/vessels/custom-supplements")
@@ -2396,7 +2604,31 @@ async def get_bgp_anomalies_geojson(
     status: Optional[str] = Query("active"),
     limit: int = Query(200, ge=1, le=1000),
     db: AsyncSession = Depends(get_db),
+    response: Response = None,
 ):
+    async def build_payload() -> dict[str, Any]:
+        return await _build_bgp_anomalies_geojson(
+            severity=severity,
+            status=status,
+            limit=limit,
+            db=db,
+        )
+
+    return await get_or_build_layer_payload(
+        key=earth_layer_cache.key("bgp-anomalies", severity=severity or "all", status=status or "all", limit=limit),
+        policy=BGP_EVENT_CACHE_POLICY,
+        builder=build_payload,
+        response=response,
+    )
+
+
+async def _build_bgp_anomalies_geojson(
+    *,
+    severity: str | None,
+    status: str | None,
+    limit: int,
+    db: AsyncSession,
+) -> dict[str, Any]:
     stmt = select(BGPAnomaly).order_by(BGPAnomaly.created_at.desc()).limit(limit)
     if severity:
         stmt = stmt.where(BGPAnomaly.severity == severity)
@@ -2416,7 +2648,31 @@ async def get_bgp_incidents_geojson(
     status: Optional[str] = Query("active"),
     limit: int = Query(100, ge=1, le=500),
     db: AsyncSession = Depends(get_db),
+    response: Response = None,
 ):
+    async def build_payload() -> dict[str, Any]:
+        return await _build_bgp_incidents_geojson(
+            severity=severity,
+            status=status,
+            limit=limit,
+            db=db,
+        )
+
+    return await get_or_build_layer_payload(
+        key=earth_layer_cache.key("bgp-incidents", severity=severity or "all", status=status or "all", limit=limit),
+        policy=BGP_EVENT_CACHE_POLICY,
+        builder=build_payload,
+        response=response,
+    )
+
+
+async def _build_bgp_incidents_geojson(
+    *,
+    severity: str | None,
+    status: str | None,
+    limit: int,
+    db: AsyncSession,
+) -> dict[str, Any]:
     stmt = select(BGPIncident).order_by(BGPIncident.created_at.desc()).limit(limit)
     if severity:
         stmt = stmt.where(BGPIncident.severity == severity)
@@ -2431,11 +2687,25 @@ async def get_bgp_incidents_geojson(
 
 
 @router.get("/geo/bgp-collectors")
-async def get_bgp_collectors_geojson(db: AsyncSession = Depends(get_db)):
+async def get_bgp_collectors_geojson(db: AsyncSession = Depends(get_db), response: Response = None):
+    async def build_payload() -> dict[str, Any]:
+        return await _build_bgp_collectors_geojson(db)
+
+    return await get_or_build_layer_payload(
+        key=earth_layer_cache.key("bgp-collectors"),
+        policy=BGP_CACHE_POLICY,
+        builder=build_payload,
+        response=response,
+    )
+
+
+async def _build_bgp_collectors_geojson(db: AsyncSession) -> dict[str, Any]:
     coverage = await build_bgp_collector_coverage(
         db,
         source_filter=("ris_live_bgp", "bgpstream_bgp"),
     )
+    if not any(int(item.get("observation_count") or 0) > 0 for item in coverage):
+        return {"type": "FeatureCollection", "features": [], "count": 0}
     coverage_by_collector = {
         item["collector"]: item
         for item in coverage
@@ -2446,8 +2716,20 @@ async def get_bgp_collectors_geojson(db: AsyncSession = Depends(get_db)):
 
 
 @router.get("/geo/summary")
-async def get_visualization_geo_summary(db: AsyncSession = Depends(get_db)):
+async def get_visualization_geo_summary(db: AsyncSession = Depends(get_db), response: Response = None):
     """Return lightweight Earth HUD counts without loading layer GeoJSON payloads."""
+    async def build_payload() -> dict[str, Any]:
+        return await _build_visualization_geo_summary(db)
+
+    return await get_or_build_layer_payload(
+        key=earth_layer_cache.key("summary"),
+        policy=SUMMARY_CACHE_POLICY,
+        builder=build_payload,
+        response=response,
+    )
+
+
+async def _build_visualization_geo_summary(db: AsyncSession) -> dict[str, Any]:
     cable_count = await _count_current_or_latest_task_data(db, "arcgis_cables")
     landing_point_count = await _count_current_or_latest_task_data(db, "arcgis_landing_points")
     satellite_count = await _count_current_or_latest_task_data(
@@ -2460,10 +2742,10 @@ async def get_visualization_geo_summary(db: AsyncSession = Depends(get_db)):
     compute_center_count = supercomputer_count + gpu_cluster_count
 
     active_incident_result = await db.execute(
-        select(func.count(BGPIncident.id)).where(BGPIncident.status == "active"),
+        select(func.count(BGPIncident.id)).where(BGPIncident.status == BGPStatus.ACTIVE.value),
     )
     active_anomaly_result = await db.execute(
-        select(func.count(BGPAnomaly.id)).where(BGPAnomaly.status == "active"),
+        select(func.count(BGPAnomaly.id)).where(BGPAnomaly.status == BGPStatus.ACTIVE.value),
     )
     active_incident_count = int(active_incident_result.scalar() or 0)
     active_anomaly_count = int(active_anomaly_result.scalar() or 0)
@@ -2484,21 +2766,14 @@ async def get_visualization_geo_summary(db: AsyncSession = Depends(get_db)):
         )
     else:
         bgp_collector_count = int(bgp_collector_scalar or 0)
-    raw_unique_window_hours = 24
-    raw_unique_mmsi = await count_unique_raw_vessel_mmsi(
-        db,
-        observed_since=datetime.now(UTC) - timedelta(hours=raw_unique_window_hours),
+    vessel_current_window_minutes = 60
+    vessel_current_result = await db.execute(
+        select(func.count(VesselCurrentState.mmsi)).where(
+            VesselCurrentState.observed_at
+            >= datetime.now(UTC) - timedelta(minutes=vessel_current_window_minutes)
+        )
     )
-    legacy_unique_result = await db.execute(
-        select(func.count(func.distinct(VesselPosition.mmsi)))
-    )
-    legacy_unique_mmsi = int(legacy_unique_result.scalar() or 0)
-    legacy_fallback_active = (
-        VESSEL_SNAPSHOT_LEGACY_FALLBACK_ENABLED
-        and raw_unique_mmsi == 0
-        and legacy_unique_mmsi > 0
-    )
-    vessel_count = legacy_unique_mmsi if legacy_fallback_active else raw_unique_mmsi
+    vessel_count = int(vessel_current_result.scalar() or 0)
     aisstream_health = await db.get(AISSourceHealth, "aisstream_vessels")
 
     return {
@@ -2509,11 +2784,10 @@ async def get_visualization_geo_summary(db: AsyncSession = Depends(get_db)):
             "satellite_count": satellite_count,
             "compute_center_count": compute_center_count,
             "vessel_count": vessel_count,
-            "vessel_count_source": "legacy_fallback" if legacy_fallback_active else "raw_recent",
-            "vessel_legacy_fallback_enabled": VESSEL_SNAPSHOT_LEGACY_FALLBACK_ENABLED,
-            "vessel_raw_unique_mmsi": raw_unique_mmsi,
-            "vessel_raw_unique_window_hours": raw_unique_window_hours,
-            "vessel_legacy_unique_mmsi": legacy_unique_mmsi,
+            "vessel_count_source": "vessel_current_state",
+            "vessel_current_window_minutes": vessel_current_window_minutes,
+            "vessel_raw_unique_mmsi": 0,
+            "vessel_legacy_unique_mmsi": 0,
             "aisstream_connection_state": aisstream_health.connection_state if aisstream_health else None,
             "aisstream_last_seen_at": to_iso8601_utc(aisstream_health.last_seen_at) if aisstream_health else None,
             "aisstream_message_rate": aisstream_health.message_rate if aisstream_health else None,

backend/app/api/v1/websocket.py

@@ -6,14 +6,19 @@ from typing import Optional
 
 from fastapi import APIRouter, WebSocket, WebSocketDisconnect, Query
 from jose import jwt, JWTError
+from sqlalchemy import text
 
 from app.core.config import settings
+from app.core.enums import UserRole
 from app.core.logging import get_logger
 from app.core.time import to_iso8601_utc
 from app.core.websocket.manager import manager
+from app.db.session import async_session_factory
+from app.services.log_tail import LOG_TAIL_CHANNEL, log_tail_manager
 
 logger = get_logger(__name__, service="api")
 router = APIRouter()
+EARTH_UPDATES_CHANNEL = "earth_updates"
 
 
 async def authenticate_token(token: str) -> Optional[dict]:
@@ -36,6 +41,28 @@ async def authenticate_token(token: str) -> Optional[dict]:
         return None
 
 
+async def load_websocket_user_role(user_id: str | None) -> str | None:
+    if not user_id:
+        return None
+    try:
+        async with async_session_factory() as db:
+            result = await db.execute(
+                text("SELECT role, is_active FROM users WHERE id = :id"),
+                {"id": int(user_id)},
+            )
+            row = result.fetchone()
+    except Exception as exc:
+        logger.warning_event(
+            "WebSocket user role lookup failed",
+            event="auth.websocket.role_lookup_failed",
+            context={"user_id": user_id, "error": str(exc)},
+        )
+        return None
+    if row is None or not row[1]:
+        return None
+    return str(row[0] or "")
+
+
 @router.websocket("/ws")
 async def websocket_endpoint(
     websocket: WebSocket,
@@ -58,7 +85,8 @@ async def websocket_endpoint(
 
     is_anonymous = payload is None
     user_id = str(payload.get("sub")) if payload else f"anonymous:{id(websocket)}"
-    supported_channels = ["vessels"] if is_anonymous else [
+    user_role = await load_websocket_user_role(user_id) if payload else None
+    supported_channels = ["vessels", "earth_news", EARTH_UPDATES_CHANNEL] if is_anonymous else [
         "gpu_clusters",
         "submarine_cables",
         "ixp_nodes",
@@ -66,7 +94,11 @@ async def websocket_endpoint(
         "dashboard",
         "datasource_tasks",
         "vessels",
+        "earth_news",
+        EARTH_UPDATES_CHANNEL,
     ]
+    if user_role == UserRole.SUPER_ADMIN.value:
+        supported_channels = [*supported_channels, LOG_TAIL_CHANNEL]
     await manager.connect(websocket, user_id)
 
     try:
@@ -97,6 +129,7 @@ async def websocket_endpoint(
                     payload_data = data.get("data", {})
                     if not isinstance(payload_data, dict):
                         payload_data = {}
+                    log_tail_config = None
                     channels = payload_data.get("channels", [])
                     if isinstance(channels, str):
                         channels = [channels]
@@ -105,10 +138,30 @@ async def websocket_endpoint(
                     channel = payload_data.get("channel")
                     if channel and channel not in channels:
                         channels = [*channels, channel]
+                    if LOG_TAIL_CHANNEL in channels:
+                        if user_role != UserRole.SUPER_ADMIN.value:
+                            await websocket.send_json(
+                                {
+                                    "type": "subscription_error",
+                                    "data": {"channel": LOG_TAIL_CHANNEL, "detail": "Only super_admin can subscribe logs"},
+                                }
+                            )
+                            channels = [item for item in channels if item != LOG_TAIL_CHANNEL]
+                        else:
+                            try:
+                                log_tail_config = await log_tail_manager.subscribe(websocket, payload_data)
+                            except ValueError as exc:
+                                await websocket.send_json(
+                                    {
+                                        "type": "subscription_error",
+                                        "data": {"channel": LOG_TAIL_CHANNEL, "detail": str(exc)},
+                                    }
+                                )
+                            channels = [item for item in channels if item != LOG_TAIL_CHANNEL]
                     if is_anonymous:
                         channels = [channel for channel in channels if channel in supported_channels]
                     vessel_subscription = None
-                    if "vessels" in channels and "bbox" in payload_data:
+                    if "vessels" in channels:
                         try:
                             vessel_subscription = manager.subscribe_vessels(websocket, payload_data)
                         except ValueError as exc:
@@ -128,14 +181,20 @@ async def websocket_endpoint(
                                 "action": "subscribe",
                                 "channels": [
                                     *channels,
+                                    *([LOG_TAIL_CHANNEL] if log_tail_config else []),
                                     *(["vessels"] if vessel_subscription else []),
                                 ],
                                 "vessels": vessel_subscription,
+                                "logs_tail": log_tail_config.__dict__ if log_tail_config else None,
                             },
                         }
                     )
                 elif data.get("type") == "unsubscribe":
                     channels = data.get("data", {}).get("channels", [])
+                    if isinstance(channels, str):
+                        channels = [channels]
+                    if LOG_TAIL_CHANNEL in channels:
+                        await log_tail_manager.unsubscribe(websocket)
                     manager.unsubscribe(websocket, channels)
                     await websocket.send_json(
                         {
@@ -156,4 +215,5 @@ async def websocket_endpoint(
     except WebSocketDisconnect:
         pass
     finally:
+        await log_tail_manager.disconnect(websocket)
         manager.disconnect(websocket, user_id)

backend/app/core/config.py

@@ -41,6 +41,7 @@ class Settings(BaseSettings):
     AI_PROVIDER_SERVICE_TOKEN: str = ""
     AI_PROVIDER_TIMEOUT_SECONDS: int = 60
     AI_PROVIDER_RETRY_ATTEMPTS: int = 2
+    OBSERVABILITY_INGEST_TOKEN: str = ""
 
     @property
     def REDIS_URL(self) -> str:

backend/app/core/datasource_defaults.py

@@ -257,6 +257,16 @@ DEFAULT_DATASOURCES = {
         "credential_provider": "aisstream",
         "credential_status": "supported",
     },
+    "media_news_archive": {
+        "id": 33,
+        "name": "Media News Archive",
+        "display_name": "媒体新闻归档",
+        "module": "L4",
+        "priority": "P2",
+        "frequency_minutes": 720,
+        "is_free": True,
+        "requires_credentials": False,
+    },
 }
 
 ID_TO_COLLECTOR = {info["id"]: name for name, info in DEFAULT_DATASOURCES.items()}

backend/app/core/enums.py

@@ -0,0 +1,227 @@
+"""Stable backend protocol enums.
+
+Database columns and JSON payloads continue to store the enum string values.
+Configurable identifiers, user-authored values, and open-ended taxonomies do
+not belong in this module.
+"""
+
+from __future__ import annotations
+
+import logging
+from enum import StrEnum
+from typing import TypeVar
+
+logger = logging.getLogger(__name__)
+
+EnumT = TypeVar("EnumT", bound=StrEnum)
+
+
+def parse_enum(enum_type: type[EnumT], value: object, default: EnumT) -> EnumT:
+    """Parse an external value without breaking reads of legacy data."""
+
+    if value is None or str(value).strip() == "":
+        return default
+    if isinstance(value, enum_type):
+        return value
+    try:
+        return enum_type(str(value).strip().lower())
+    except (TypeError, ValueError):
+        logger.warning(
+            "Unknown %s value %r; falling back to %s",
+            enum_type.__name__,
+            value,
+            default.value,
+        )
+        return default
+
+
+class NewsImportanceLevel(StrEnum):
+    LOW = "low"
+    MEDIUM = "medium"
+    HIGH = "high"
+    CRITICAL = "critical"
+
+
+class BreakingLevel(StrEnum):
+    NONE = "none"
+    WATCH = "watch"
+    BREAKING = "breaking"
+    CRITICAL = "critical"
+
+
+class BreakingScope(StrEnum):
+    REGIONAL = "regional"
+    GLOBAL = "global"
+
+
+class BreakingSource(StrEnum):
+    RULES = "rules"
+    AI = "ai"
+    MANUAL = "manual"
+    MULTI_SOURCE = "multi_source"
+
+
+class NewsSourceType(StrEnum):
+    RSS = "rss"
+    ATOM = "atom"
+    AGGREGATED = "aggregated"
+    REFERENCE = "reference"
+    MANUAL = "manual"
+
+
+class NewsEnrichmentStatus(StrEnum):
+    PENDING = "pending"
+    QUEUED = "queued"
+    ATTEMPTED = "attempted"
+    SUCCESS = "success"
+    CONTENT_ONLY = "content_only"
+    LOCATION_ONLY = "location_only"
+    UNAVAILABLE = "unavailable"
+    PROVIDER_ERROR = "provider_error"
+    PARSE_ERROR = "parse_error"
+    NO_RESULT = "no_result"
+
+
+class NewsMarketImpact(StrEnum):
+    NONE = "none"
+    SECTOR = "sector"
+    NATIONAL = "national"
+    GLOBAL = "global"
+
+
+class NewsTaggingSource(StrEnum):
+    RULES = "rules"
+    AI = "ai"
+    MANUAL = "manual"
+
+
+class JobType(StrEnum):
+    COLLECT = "collect"
+    CLEAR_DATA = "clear_data"
+    CLEAR_CACHE = "clear_cache"
+    EARTH_REFRESH = "earth_refresh"
+
+
+class JobStatus(StrEnum):
+    QUEUED = "queued"
+    RUNNING = "running"
+    CANCELLING = "cancelling"
+    SUCCESS = "success"
+    FAILED = "failed"
+    CANCELLED = "cancelled"
+
+
+class RollbackPolicy(StrEnum):
+    KEEP_COMMITTED_BATCHES = "keep_committed_batches"
+
+
+class MappingValidationStatus(StrEnum):
+    DRAFT = "draft"
+    VALID = "valid"
+    INVALID = "invalid"
+
+
+class SnapshotStatus(StrEnum):
+    RUNNING = "running"
+    SUCCESS = "success"
+    FAILED = "failed"
+    CANCELLED = "cancelled"
+
+
+class DatasourceRunStatus(StrEnum):
+    RUNNING = "running"
+    NOT_RUN = "not_run"
+    COLLECTED = "collected"
+    UNCOLLECTED = "uncollected"
+
+
+class ProviderApi(StrEnum):
+    ANTHROPIC_MESSAGES = "anthropic-messages"
+    OPENAI_COMPLETIONS = "openai-completions"
+    OPENAI_RESPONSES = "openai-responses"
+    OLLAMA_GENERATE = "ollama-generate"
+
+
+class PlaygroundMessageRole(StrEnum):
+    SYSTEM = "system"
+    USER = "user"
+    ASSISTANT = "assistant"
+    TOOL = "tool"
+
+
+class PlaygroundMessageKind(StrEnum):
+    MESSAGE = "message"
+    THINKING = "thinking"
+    ERROR = "error"
+    STATUS = "status"
+
+
+class PlaygroundMessageStatus(StrEnum):
+    PENDING = "pending"
+    THINKING = "thinking"
+    ANSWERING = "answering"
+    DONE = "done"
+    FAILED = "failed"
+    CANCELLED = "cancelled"
+    ERROR = "error"
+    STOPPED = "stopped"
+
+
+class OtpPurpose(StrEnum):
+    REGISTER = "register"
+    VERIFY_EMAIL = "verify_email"
+    RESET_PASSWORD = "reset_password"
+
+
+class UserRole(StrEnum):
+    VIEWER = "viewer"
+    ADMIN = "admin"
+    SUPER_ADMIN = "super_admin"
+
+
+class AlertSeverity(StrEnum):
+    CRITICAL = "critical"
+    WARNING = "warning"
+    INFO = "info"
+
+
+class AlertStatus(StrEnum):
+    ACTIVE = "active"
+    ACKNOWLEDGED = "acknowledged"
+    RESOLVED = "resolved"
+
+
+class BGPStatus(StrEnum):
+    ACTIVE = "active"
+    ACKNOWLEDGED = "acknowledged"
+    RESOLVED = "resolved"
+
+
+class LogLevel(StrEnum):
+    ALL = "all"
+    ERROR = "error"
+    WARNING = "warning"
+    INFO = "info"
+    DEBUG = "debug"
+
+
+class ConnectionState(StrEnum):
+    DISCONNECTED = "disconnected"
+    CONNECTING = "connecting"
+    CONNECTED = "connected"
+    ERROR = "error"
+
+
+class AuthType(StrEnum):
+    NONE = "none"
+    BEARER = "bearer"
+    API_KEY = "api_key"
+    BASIC = "basic"
+
+
+class TVSourceType(StrEnum):
+    IFRAME = "iframe"
+    HLS = "hls"
+    VIDEO = "video"
+    EXTERNAL = "external"
+    YOUTUBE = "youtube"

backend/app/core/websocket/broadcaster.py

@@ -4,10 +4,15 @@ import asyncio
 from datetime import UTC, datetime
 from typing import Dict, Any
 
+from app.core.logging import get_logger
 from app.core.time import to_iso8601_utc
 from app.core.websocket.manager import manager
 
 
+EARTH_UPDATES_CHANNEL = "earth_updates"
+VESSEL_STATE_QUERY_BATCH_SIZE = 1000
+logger = get_logger(__name__, service="websocket")
+
 
 class DataBroadcaster:
     """Periodically broadcasts data to connected WebSocket clients"""
@@ -83,6 +88,10 @@ class DataBroadcaster:
             channel=channel,
         )
 
+    async def broadcast_earth_update(self, data: Dict[str, Any]):
+        """Broadcast Earth visualization refresh hints to connected clients."""
+        await self.broadcast_custom(EARTH_UPDATES_CHANNEL, data)
+
     def enqueue_vessel_update(self, data: Dict[str, Any]):
         vessels = data.get("vessels") if isinstance(data, dict) else None
         if not isinstance(vessels, list):
@@ -108,16 +117,22 @@ class DataBroadcaster:
             return
         pending = self._pending_vessel_updates
         self._pending_vessel_updates = {}
-        vessels = []
-        for item in pending.values():
-            vessel = dict(item)
-            source = vessel.pop("_source", None)
-            action = vessel.pop("_action", "upsert")
-            created = vessel.pop("_created", None)
-            vessel["source"] = source
-            vessel["action"] = action
-            vessel["created"] = created
-            vessels.append(vessel)
+        try:
+            vessels = await self._load_current_vessel_updates(list(pending))
+        except Exception:
+            # Preserve newer updates that arrived during the failed database read.
+            self._pending_vessel_updates = {**pending, **self._pending_vessel_updates}
+            raise
+        if not vessels:
+            return
+        vessels = [
+            {
+                **vessel,
+                "action": vessel.get("action", "upsert"),
+                "created": pending.get(str(vessel["mmsi"]), {}).get("_created"),
+            }
+            for vessel in vessels
+        ]
         await manager.broadcast_vessels(
             {
                 "action": "upsert",
@@ -127,12 +142,31 @@ class DataBroadcaster:
             }
         )
 
+    async def _load_current_vessel_updates(self, keys: list[str]) -> list[Dict[str, Any]]:
+        from app.db.session import async_session_factory
+        from app.services.vessel_ais_aggregation import get_current_vessels_by_mmsi
+
+        mmsis = [int(key) for key in keys if key.isdigit()]
+        vessels = []
+        async with async_session_factory() as db:
+            for offset in range(0, len(mmsis), VESSEL_STATE_QUERY_BATCH_SIZE):
+                vessels.extend(await get_current_vessels_by_mmsi(
+                    db, mmsis[offset:offset + VESSEL_STATE_QUERY_BATCH_SIZE]
+                ))
+        present = {int(vessel["mmsi"]) for vessel in vessels}
+        vessels.extend({"mmsi": mmsi, "action": "remove"} for mmsi in mmsis if mmsi not in present)
+        return vessels
+
     async def broadcast_vessels_periodically(self):
         while self.running:
             try:
                 await self.flush_vessel_updates()
-            except Exception:
-                pass
+            except Exception as exc:
+                logger.exception_event(
+                    "Failed to flush vessel updates",
+                    event="vessels.broadcast.failed",
+                    context={"error": str(exc)},
+                )
             await asyncio.sleep(self._vessel_flush_interval)
 
     async def broadcast_datasource_task_update(self, data: Dict[str, Any]):
@@ -144,7 +178,7 @@ class DataBroadcaster:
                 "timestamp": to_iso8601_utc(datetime.now(UTC)),
                 "payload": data,
             },
-            channel="all",
+            channel="datasource_tasks",
         )
 
     def start(self):

backend/app/core/websocket/manager.py

@@ -63,6 +63,8 @@ class ConnectionManager:
 
     def unsubscribe(self, websocket: WebSocket, channels: list[str]):
         for channel in {str(channel).strip() for channel in channels if str(channel).strip()}:
+            if channel == "vessels":
+                self.vessel_subscriptions.pop(websocket, None)
             subscribers = self.channel_subscriptions.get(channel)
             if subscribers is not None:
                 subscribers.discard(websocket)
@@ -88,7 +90,8 @@ class ConnectionManager:
         return subscription
 
     def _normalize_vessel_subscription(self, config: dict[str, Any]) -> dict[str, Any]:
-        bbox = config.get("bbox")
+        global_scope = config.get("scope") == "global"
+        bbox = [-180, -90, 180, 90] if global_scope else config.get("bbox")
         if not isinstance(bbox, (list, tuple)) or len(bbox) != 4:
             raise ValueError("vessels subscription requires bbox=[lon_min,lat_min,lon_max,lat_max]")
         try:
@@ -103,7 +106,7 @@ class ConnectionManager:
             raise ValueError("bbox longitude values must be between -180 and 180")
         if not (-90 <= lat_min <= 90 and -90 <= lat_max <= 90):
             raise ValueError("bbox latitude values must be between -90 and 90")
-        if (lon_max - lon_min) * (lat_max - lat_min) > MAX_VESSEL_BBOX_AREA:
+        if not global_scope and (lon_max - lon_min) * (lat_max - lat_min) > MAX_VESSEL_BBOX_AREA:
             raise ValueError("bbox is too large; zoom in or request a smaller viewport")
 
         zoom = int(config.get("zoom") or 1)
@@ -116,10 +119,11 @@ class ConnectionManager:
             if str(item).strip()
         }
         return {
+            "scope": "global" if global_scope else "viewport",
             "bbox": (lon_min, lat_min, lon_max, lat_max),
             "zoom": zoom,
             "limit": limit,
-            "type": vessel_types,
+            "type": sorted(vessel_types),
             "last_sent_at": None,
         }
 
@@ -152,7 +156,9 @@ class ConnectionManager:
                 vessel
                 for vessel in vessels
                 if self._vessel_matches_subscription(vessel, subscription)
-            ][: min(subscription["limit"], MAX_VESSEL_WS_MESSAGE_ITEMS)]
+            ]
+            if subscription.get("scope") != "global":
+                matched = matched[:subscription["limit"]]
             if not matched:
                 continue
             subscription["last_sent_at"] = datetime.now(UTC)
@@ -162,16 +168,24 @@ class ConnectionManager:
                 "timestamp": subscription["last_sent_at"].isoformat(),
                 "payload": {
                     **data,
-                    "vessels": matched,
+                    "vessels": [],
                     "subscription": {
                         "bbox": list(subscription["bbox"]),
                         "zoom": subscription["zoom"],
                         "limit": subscription["limit"],
+                        "scope": subscription.get("scope", "viewport"),
                     },
                 },
             }
             try:
-                await connection.send_json(message)
+                for offset in range(0, len(matched), MAX_VESSEL_WS_MESSAGE_ITEMS):
+                    await connection.send_json({
+                        **message,
+                        "payload": {
+                            **message["payload"],
+                            "vessels": matched[offset:offset + MAX_VESSEL_WS_MESSAGE_ITEMS],
+                        },
+                    })
             except Exception:
                 self.unsubscribe_all(connection)
 
@@ -180,6 +194,8 @@ class ConnectionManager:
         vessel: dict[str, Any],
         subscription: dict[str, Any],
     ) -> bool:
+        if vessel.get("action") == "remove" and subscription.get("scope") == "global":
+            return True
         try:
             lon = float(vessel.get("lon"))
             lat = float(vessel.get("lat"))

backend/app/db/session.py

@@ -1,6 +1,6 @@
 from typing import AsyncGenerator
 
-from sqlalchemy import text
+from sqlalchemy import bindparam, text
 from sqlalchemy.ext.asyncio import AsyncSession, create_async_engine, async_sessionmaker
 from sqlalchemy.orm import declarative_base
 
@@ -72,6 +72,74 @@ async def seed_default_datasources(session: AsyncSession):
     await session.commit()
 
 
+LEGACY_EARTH_BOUNDARY_SOURCES = (
+    "earth_admin0_boundaries",
+    "earth_coastline",
+    "earth_claim_lines",
+    "earth_boundary_tiles",
+)
+LEGACY_EARTH_BOUNDARY_DATATYPES = (
+    "earth_boundary_source",
+    "earth_boundary_tiles",
+)
+LEGACY_EARTH_BOUNDARY_IDS = (29, 30, 31, 32)
+
+
+async def purge_legacy_earth_boundary_datasources(session: AsyncSession) -> None:
+    source_names = tuple(LEGACY_EARTH_BOUNDARY_SOURCES)
+    source_ids = tuple(LEGACY_EARTH_BOUNDARY_IDS)
+    data_types = tuple(LEGACY_EARTH_BOUNDARY_DATATYPES)
+    await session.execute(
+        text(
+            """
+            DELETE FROM datasource_mapping_templates
+            WHERE target_schema IN :data_types
+               OR datasource_config_id IN (
+                    SELECT id FROM datasource_configs WHERE name IN :source_names
+               )
+            """
+        ).bindparams(bindparam("source_names", expanding=True), bindparam("data_types", expanding=True)),
+        {"source_names": list(source_names), "data_types": list(data_types)},
+    )
+    await session.execute(
+        text("DELETE FROM datasource_configs WHERE name IN :source_names").bindparams(
+            bindparam("source_names", expanding=True)
+        ),
+        {"source_names": list(source_names)},
+    )
+    await session.execute(
+        text(
+            """
+            DELETE FROM collected_data
+            WHERE source IN :source_names OR data_type IN :data_types
+            """
+        ).bindparams(bindparam("source_names", expanding=True), bindparam("data_types", expanding=True)),
+        {"source_names": list(source_names), "data_types": list(data_types)},
+    )
+    await session.execute(
+        text(
+            """
+            DELETE FROM data_snapshots
+            WHERE source IN :source_names OR datasource_id IN :source_ids
+            """
+        ).bindparams(bindparam("source_names", expanding=True), bindparam("source_ids", expanding=True)),
+        {"source_names": list(source_names), "source_ids": list(source_ids)},
+    )
+    await session.execute(
+        text("DELETE FROM collection_tasks WHERE datasource_id IN :source_ids").bindparams(
+            bindparam("source_ids", expanding=True)
+        ),
+        {"source_ids": list(source_ids)},
+    )
+    await session.execute(
+        text("DELETE FROM data_sources WHERE source IN :source_names OR id IN :source_ids").bindparams(
+            bindparam("source_names", expanding=True), bindparam("source_ids", expanding=True)
+        ),
+        {"source_names": list(source_names), "source_ids": list(source_ids)},
+    )
+    await session.commit()
+
+
 DEFAULT_LOGIN_USERS = (
     {
         "username": "admin",
@@ -82,7 +150,7 @@ DEFAULT_LOGIN_USERS = (
     {
         "username": "linkong",
         "email": "linkong@planet.local",
-        "password": "12345678",
+        "password": "LK12345678",
         "role": "super_admin",
     },
 )
@@ -134,6 +202,8 @@ async def init_db():
     import app.models.vessel  # noqa: F401
     import app.models.vessel_enrichment  # noqa: F401
     import app.models.datasource_mapping  # noqa: F401
+    import app.models.earth_news  # noqa: F401
+    import app.models.earth_interactable  # noqa: F401
 
     logger.warning_event(
         "Database pool settings active",
@@ -189,6 +259,407 @@ async def init_db():
                 """
             )
         )
+        await conn.execute(
+            text(
+                """
+                CREATE TABLE IF NOT EXISTS earth_data_change_events (
+                    id BIGSERIAL PRIMARY KEY,
+                    table_name VARCHAR(128) NOT NULL,
+                    operation VARCHAR(16) NOT NULL,
+                    source VARCHAR(128),
+                    entity_key VARCHAR(255),
+                    payload JSONB NOT NULL,
+                    occurred_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
+                    consumed_at TIMESTAMPTZ
+                )
+                """
+            )
+        )
+        await conn.execute(
+            text(
+                """
+                CREATE INDEX IF NOT EXISTS idx_earth_data_change_events_unconsumed
+                ON earth_data_change_events (consumed_at, id)
+                WHERE consumed_at IS NULL
+                """
+            )
+        )
+        await conn.execute(
+            text(
+                """
+                CREATE OR REPLACE FUNCTION planet_emit_earth_data_changed_statement(
+                    change_table TEXT,
+                    change_operation TEXT,
+                    change_source TEXT,
+                    source_record_count INTEGER,
+                    source_entity_keys TEXT[]
+                )
+                RETURNS VOID AS $$
+                DECLARE
+                    change_event_id BIGINT;
+                    change_payload JSONB;
+                BEGIN
+                    change_payload := jsonb_build_object(
+                        'event', 'earth.layer.changed',
+                        'table', change_table,
+                        'operation', change_operation,
+                        'source', change_source,
+                        'entity_key', NULL,
+                        'entity_keys', COALESCE(to_jsonb(source_entity_keys), '[]'::jsonb),
+                        'records_processed', COALESCE(source_record_count, 0),
+                        'occurred_at', NOW()
+                    );
+
+                    INSERT INTO earth_data_change_events (
+                        table_name,
+                        operation,
+                        source,
+                        entity_key,
+                        payload,
+                        occurred_at
+                    ) VALUES (
+                        change_table,
+                        change_operation,
+                        change_source,
+                        NULL,
+                        change_payload,
+                        NOW()
+                    )
+                    RETURNING id INTO change_event_id;
+
+                    change_payload := change_payload || jsonb_build_object(
+                        'event_id', change_event_id
+                    );
+
+                    UPDATE earth_data_change_events
+                    SET payload = change_payload
+                    WHERE id = change_event_id;
+
+                    PERFORM pg_notify(
+                        'planet_earth_data_changes',
+                        change_payload::text
+                    );
+                END;
+                $$ LANGUAGE plpgsql;
+                """
+            )
+        )
+        await conn.execute(
+            text(
+                """
+                CREATE OR REPLACE FUNCTION planet_emit_collected_data_changed_statement(
+                    change_operation TEXT,
+                    change_source TEXT,
+                    source_record_count INTEGER,
+                    source_entity_keys TEXT[]
+                )
+                RETURNS VOID AS $$
+                BEGIN
+                    PERFORM planet_emit_earth_data_changed_statement(
+                        'collected_data',
+                        change_operation,
+                        change_source,
+                        source_record_count,
+                        source_entity_keys
+                    );
+                END;
+                $$ LANGUAGE plpgsql;
+                """
+            )
+        )
+        await conn.execute(
+            text(
+                """
+                CREATE OR REPLACE FUNCTION planet_notify_earth_table_changed_statement()
+                RETURNS trigger AS $$
+                DECLARE
+                    change_source TEXT;
+                    source_record_count INTEGER;
+                    source_entity_keys TEXT[];
+                BEGIN
+                    IF TG_OP = 'INSERT' THEN
+                        FOR change_source IN
+                            SELECT DISTINCT COALESCE(NULLIF(row_data->>'source', ''), TG_TABLE_NAME)
+                            FROM (SELECT to_jsonb(t) AS row_data FROM new_rows AS t) changed_rows
+                        LOOP
+                            SELECT
+                                COUNT(*),
+                                ARRAY(
+                                    SELECT DISTINCT COALESCE(
+                                        NULLIF(row_data->>'entity_key', ''),
+                                        NULLIF(row_data->>'source_id', ''),
+                                        NULLIF(row_data->>'incident_key', ''),
+                                        NULLIF(row_data->>'id', ''),
+                                        NULLIF(row_data->>'mmsi', '')
+                                    )
+                                    FROM (SELECT to_jsonb(t) AS row_data FROM new_rows AS t) rows_for_keys
+                                    WHERE COALESCE(NULLIF(row_data->>'source', ''), TG_TABLE_NAME) = change_source
+                                    LIMIT 20
+                                )
+                            INTO source_record_count, source_entity_keys
+                            FROM (SELECT to_jsonb(t) AS row_data FROM new_rows AS t) rows_for_count
+                            WHERE COALESCE(NULLIF(row_data->>'source', ''), TG_TABLE_NAME) = change_source;
+
+                            PERFORM planet_emit_earth_data_changed_statement(
+                                TG_TABLE_NAME,
+                                TG_OP,
+                                change_source,
+                                source_record_count,
+                                source_entity_keys
+                            );
+                        END LOOP;
+                    ELSIF TG_OP = 'DELETE' THEN
+                        FOR change_source IN
+                            SELECT DISTINCT COALESCE(NULLIF(row_data->>'source', ''), TG_TABLE_NAME)
+                            FROM (SELECT to_jsonb(t) AS row_data FROM old_rows AS t) changed_rows
+                        LOOP
+                            SELECT
+                                COUNT(*),
+                                ARRAY(
+                                    SELECT DISTINCT COALESCE(
+                                        NULLIF(row_data->>'entity_key', ''),
+                                        NULLIF(row_data->>'source_id', ''),
+                                        NULLIF(row_data->>'incident_key', ''),
+                                        NULLIF(row_data->>'id', ''),
+                                        NULLIF(row_data->>'mmsi', '')
+                                    )
+                                    FROM (SELECT to_jsonb(t) AS row_data FROM old_rows AS t) rows_for_keys
+                                    WHERE COALESCE(NULLIF(row_data->>'source', ''), TG_TABLE_NAME) = change_source
+                                    LIMIT 20
+                                )
+                            INTO source_record_count, source_entity_keys
+                            FROM (SELECT to_jsonb(t) AS row_data FROM old_rows AS t) rows_for_count
+                            WHERE COALESCE(NULLIF(row_data->>'source', ''), TG_TABLE_NAME) = change_source;
+
+                            PERFORM planet_emit_earth_data_changed_statement(
+                                TG_TABLE_NAME,
+                                TG_OP,
+                                change_source,
+                                source_record_count,
+                                source_entity_keys
+                            );
+                        END LOOP;
+                    ELSIF TG_OP = 'UPDATE' THEN
+                        FOR change_source IN
+                            SELECT DISTINCT COALESCE(NULLIF(row_data->>'source', ''), TG_TABLE_NAME)
+                            FROM (
+                                SELECT to_jsonb(t) AS row_data FROM new_rows AS t
+                                UNION ALL
+                                SELECT to_jsonb(t) AS row_data FROM old_rows AS t
+                            ) changed_rows
+                        LOOP
+                            SELECT
+                                COUNT(*),
+                                ARRAY(
+                                    SELECT DISTINCT COALESCE(
+                                        NULLIF(row_data->>'entity_key', ''),
+                                        NULLIF(row_data->>'source_id', ''),
+                                        NULLIF(row_data->>'incident_key', ''),
+                                        NULLIF(row_data->>'id', ''),
+                                        NULLIF(row_data->>'mmsi', '')
+                                    )
+                                    FROM (
+                                        SELECT to_jsonb(t) AS row_data FROM new_rows AS t
+                                        UNION ALL
+                                        SELECT to_jsonb(t) AS row_data FROM old_rows AS t
+                                    ) rows_for_keys
+                                    WHERE COALESCE(NULLIF(row_data->>'source', ''), TG_TABLE_NAME) = change_source
+                                    LIMIT 20
+                                )
+                            INTO source_record_count, source_entity_keys
+                            FROM (
+                                SELECT to_jsonb(t) AS row_data FROM new_rows AS t
+                                UNION ALL
+                                SELECT to_jsonb(t) AS row_data FROM old_rows AS t
+                            ) rows_for_count
+                            WHERE COALESCE(NULLIF(row_data->>'source', ''), TG_TABLE_NAME) = change_source;
+
+                            PERFORM planet_emit_earth_data_changed_statement(
+                                TG_TABLE_NAME,
+                                TG_OP,
+                                change_source,
+                                source_record_count,
+                                source_entity_keys
+                            );
+                        END LOOP;
+                    END IF;
+
+                    RETURN NULL;
+                END;
+                $$ LANGUAGE plpgsql;
+                """
+            )
+        )
+        await conn.execute(
+            text(
+                """
+                CREATE OR REPLACE FUNCTION planet_notify_collected_data_changed_statement()
+                RETURNS trigger AS $$
+                DECLARE
+                    change_source TEXT;
+                    source_record_count INTEGER;
+                    source_entity_keys TEXT[];
+                BEGIN
+                    IF TG_OP = 'INSERT' THEN
+                        FOR change_source IN
+                            SELECT DISTINCT source FROM new_rows WHERE source IS NOT NULL
+                        LOOP
+                            SELECT
+                                COUNT(*),
+                                ARRAY(
+                                    SELECT DISTINCT COALESCE(entity_key, source_id, id::text)
+                                    FROM new_rows
+                                    WHERE source = change_source
+                                    LIMIT 20
+                                )
+                            INTO source_record_count, source_entity_keys
+                            FROM new_rows
+                            WHERE source = change_source;
+
+                            PERFORM planet_emit_collected_data_changed_statement(
+                                TG_OP,
+                                change_source,
+                                source_record_count,
+                                source_entity_keys
+                            );
+                        END LOOP;
+                    ELSIF TG_OP = 'DELETE' THEN
+                        FOR change_source IN
+                            SELECT DISTINCT source FROM old_rows WHERE source IS NOT NULL
+                        LOOP
+                            SELECT
+                                COUNT(*),
+                                ARRAY(
+                                    SELECT DISTINCT COALESCE(entity_key, source_id, id::text)
+                                    FROM old_rows
+                                    WHERE source = change_source
+                                    LIMIT 20
+                                )
+                            INTO source_record_count, source_entity_keys
+                            FROM old_rows
+                            WHERE source = change_source;
+
+                            PERFORM planet_emit_collected_data_changed_statement(
+                                TG_OP,
+                                change_source,
+                                source_record_count,
+                                source_entity_keys
+                            );
+                        END LOOP;
+                    ELSIF TG_OP = 'UPDATE' THEN
+                        FOR change_source IN
+                            SELECT DISTINCT source FROM (
+                                SELECT source FROM new_rows
+                                UNION
+                                SELECT source FROM old_rows
+                            ) changed_sources
+                            WHERE source IS NOT NULL
+                        LOOP
+                            SELECT
+                                COUNT(*),
+                                ARRAY(
+                                    SELECT DISTINCT COALESCE(entity_key, source_id, id::text)
+                                    FROM (
+                                        SELECT id, source_id, entity_key, source FROM new_rows
+                                        UNION ALL
+                                        SELECT id, source_id, entity_key, source FROM old_rows
+                                    ) changed_rows
+                                    WHERE source = change_source
+                                    LIMIT 20
+                                )
+                            INTO source_record_count, source_entity_keys
+                            FROM (
+                                SELECT id, source_id, entity_key, source FROM new_rows
+                                UNION ALL
+                                SELECT id, source_id, entity_key, source FROM old_rows
+                            ) changed_rows
+                            WHERE source = change_source;
+
+                            PERFORM planet_emit_collected_data_changed_statement(
+                                TG_OP,
+                                change_source,
+                                source_record_count,
+                                source_entity_keys
+                            );
+                        END LOOP;
+                    END IF;
+
+                    RETURN NULL;
+                END;
+                $$ LANGUAGE plpgsql;
+                """
+            )
+        )
+        for statement in (
+            "DROP TRIGGER IF EXISTS tr_planet_collected_data_changed ON collected_data",
+            "DROP TRIGGER IF EXISTS tr_planet_collected_data_changed_insert ON collected_data",
+            "DROP TRIGGER IF EXISTS tr_planet_collected_data_changed_update ON collected_data",
+            "DROP TRIGGER IF EXISTS tr_planet_collected_data_changed_delete ON collected_data",
+            "DROP FUNCTION IF EXISTS planet_notify_collected_data_changed()",
+            """
+            CREATE TRIGGER tr_planet_collected_data_changed_insert
+            AFTER INSERT ON collected_data
+            REFERENCING NEW TABLE AS new_rows
+            FOR EACH STATEMENT
+            EXECUTE FUNCTION planet_notify_collected_data_changed_statement()
+            """,
+            """
+            CREATE TRIGGER tr_planet_collected_data_changed_update
+            AFTER UPDATE ON collected_data
+            REFERENCING OLD TABLE AS old_rows NEW TABLE AS new_rows
+            FOR EACH STATEMENT
+            EXECUTE FUNCTION planet_notify_collected_data_changed_statement()
+            """,
+            """
+            CREATE TRIGGER tr_planet_collected_data_changed_delete
+            AFTER DELETE ON collected_data
+            REFERENCING OLD TABLE AS old_rows
+            FOR EACH STATEMENT
+            EXECUTE FUNCTION planet_notify_collected_data_changed_statement()
+            """,
+        ):
+            await conn.execute(text(statement))
+        for table_name in (
+            "bgp_observations",
+            "bgp_anomalies",
+            "bgp_incidents",
+            "bgp_collector_locations",
+            "vessel_static",
+            "vessel_position",
+            "vessel_current_state",
+            "ais_raw_observations",
+            "ais_source_health",
+            "compute_center_locations",
+            "earth_interactables",
+            "earth_news_items",
+        ):
+            for statement in (
+                f"DROP TRIGGER IF EXISTS tr_planet_{table_name}_changed_insert ON {table_name}",
+                f"DROP TRIGGER IF EXISTS tr_planet_{table_name}_changed_update ON {table_name}",
+                f"DROP TRIGGER IF EXISTS tr_planet_{table_name}_changed_delete ON {table_name}",
+                f"""
+                CREATE TRIGGER tr_planet_{table_name}_changed_insert
+                AFTER INSERT ON {table_name}
+                REFERENCING NEW TABLE AS new_rows
+                FOR EACH STATEMENT
+                EXECUTE FUNCTION planet_notify_earth_table_changed_statement()
+                """,
+                f"""
+                CREATE TRIGGER tr_planet_{table_name}_changed_update
+                AFTER UPDATE ON {table_name}
+                REFERENCING OLD TABLE AS old_rows NEW TABLE AS new_rows
+                FOR EACH STATEMENT
+                EXECUTE FUNCTION planet_notify_earth_table_changed_statement()
+                """,
+                f"""
+                CREATE TRIGGER tr_planet_{table_name}_changed_delete
+                AFTER DELETE ON {table_name}
+                REFERENCING OLD TABLE AS old_rows
+                FOR EACH STATEMENT
+                EXECUTE FUNCTION planet_notify_earth_table_changed_statement()
+                """,
+            ):
+                await conn.execute(text(statement))
         await conn.execute(
             text(
                 """
@@ -198,7 +669,39 @@ async def init_db():
                 ADD COLUMN IF NOT EXISTS phase_message VARCHAR(255),
                 ADD COLUMN IF NOT EXISTS phase_current BIGINT,
                 ADD COLUMN IF NOT EXISTS phase_total BIGINT,
-                ADD COLUMN IF NOT EXISTS phase_unit VARCHAR(30)
+                ADD COLUMN IF NOT EXISTS phase_unit VARCHAR(30),
+                ADD COLUMN IF NOT EXISTS source VARCHAR(100),
+                ADD COLUMN IF NOT EXISTS task_type VARCHAR(30) NOT NULL DEFAULT 'collect',
+                ADD COLUMN IF NOT EXISTS payload JSONB NOT NULL DEFAULT '{}'::jsonb,
+                ADD COLUMN IF NOT EXISTS rollback_policy VARCHAR(40) NOT NULL DEFAULT 'keep_committed_batches',
+                ADD COLUMN IF NOT EXISTS dedupe_key VARCHAR(180),
+                ADD COLUMN IF NOT EXISTS worker_id VARCHAR(120),
+                ADD COLUMN IF NOT EXISTS locked_at TIMESTAMPTZ,
+                ADD COLUMN IF NOT EXISTS requested_cancel_at TIMESTAMPTZ,
+                ADD COLUMN IF NOT EXISTS cancel_reason TEXT
+                """
+            )
+        )
+        await conn.execute(
+            text(
+                """
+                ALTER TABLE earth_news_items
+                ADD COLUMN IF NOT EXISTS content_language VARCHAR(32) NOT NULL DEFAULT 'en',
+                ADD COLUMN IF NOT EXISTS localizations JSONB NOT NULL DEFAULT '{}'::jsonb,
+                ADD COLUMN IF NOT EXISTS enrichment_status VARCHAR(80) NOT NULL DEFAULT 'pending',
+                ADD COLUMN IF NOT EXISTS enrichment_error TEXT,
+                ADD COLUMN IF NOT EXISTS enriched_at TIMESTAMPTZ
+                """
+            )
+        )
+        await conn.execute(
+            text(
+                """
+                ALTER TABLE earth_interactables
+                ADD COLUMN IF NOT EXISTS altitude DOUBLE PRECISION,
+                ADD COLUMN IF NOT EXISTS revision INTEGER NOT NULL DEFAULT 1,
+                ADD COLUMN IF NOT EXISTS is_deleted BOOLEAN NOT NULL DEFAULT FALSE,
+                ADD COLUMN IF NOT EXISTS deleted_at TIMESTAMPTZ
                 """
             )
         )
@@ -210,6 +713,64 @@ async def init_db():
                 """
             )
         )
+        await conn.execute(
+            text(
+                """
+                CREATE INDEX IF NOT EXISTS idx_earth_news_enrichment_status
+                ON earth_news_items (enrichment_status)
+                """
+            )
+        )
+        await conn.execute(
+            text(
+                """
+                CREATE INDEX IF NOT EXISTS idx_earth_news_enriched_at
+                ON earth_news_items (enriched_at)
+                """
+            )
+        )
+        await conn.execute(
+            text(
+                """
+                CREATE INDEX IF NOT EXISTS idx_earth_interactables_layer_deleted
+                ON earth_interactables (layer, is_deleted)
+                """
+            )
+        )
+        await conn.execute(
+            text(
+                """
+                CREATE INDEX IF NOT EXISTS idx_earth_interactables_updated_at
+                ON earth_interactables (updated_at)
+                """
+            )
+        )
+        await conn.execute(
+            text(
+                """
+                CREATE INDEX IF NOT EXISTS idx_collection_tasks_source_status
+                ON collection_tasks (source, status)
+                """
+            )
+        )
+        await conn.execute(
+            text(
+                """
+                CREATE INDEX IF NOT EXISTS idx_collection_tasks_queue
+                ON collection_tasks (status, created_at, id)
+                WHERE status = 'queued'
+                """
+            )
+        )
+        await conn.execute(
+            text(
+                """
+                CREATE INDEX IF NOT EXISTS idx_collection_tasks_dedupe
+                ON collection_tasks (dedupe_key)
+                WHERE dedupe_key IS NOT NULL
+                """
+            )
+        )
         await conn.execute(
             text(
                 """
@@ -226,6 +787,22 @@ async def init_db():
                 """
             )
         )
+        await conn.execute(
+            text(
+                """
+                CREATE INDEX IF NOT EXISTS idx_vessel_current_bbox
+                ON vessel_current_state (lon, lat)
+                """
+            )
+        )
+        await conn.execute(
+            text(
+                """
+                CREATE INDEX IF NOT EXISTS idx_vessel_current_observed
+                ON vessel_current_state (observed_at DESC)
+                """
+            )
+        )
         await conn.execute(
             text(
                 """
@@ -284,4 +861,5 @@ async def init_db():
         await seed_default_bgp_collector_locations(session)
         await seed_compute_center_locations_from_source_coords(session)
         await seed_default_datasources(session)
+        await purge_legacy_earth_boundary_datasources(session)
         await ensure_default_admin_user(session)

backend/app/main.py

@@ -1,8 +1,10 @@
 from contextlib import asynccontextmanager
+from pathlib import Path
 from uuid import uuid4
 
 from fastapi import FastAPI
 from fastapi.middleware.cors import CORSMiddleware
+from fastapi.staticfiles import StaticFiles
 from starlette.middleware.base import BaseHTTPMiddleware
 
 from app.api.main import api_router
@@ -18,6 +20,15 @@ from app.services.scheduler import (
     stop_scheduler,
     sync_scheduler_with_datasources,
 )
+from app.services.earth_news_worker import (
+    start_earth_news_target_worker,
+    stop_earth_news_target_worker,
+)
+from app.services.earth_db_change_listener import (
+    start_earth_db_change_listener,
+    stop_earth_db_change_listener,
+)
+from app.services.data_jobs import start_data_job_worker, stop_data_job_worker
 
 
 configure_logging()
@@ -53,7 +64,13 @@ async def lifespan(app: FastAPI):
     start_scheduler()
     await sync_scheduler_with_datasources()
     broadcaster.start()
+    start_data_job_worker()
+    start_earth_db_change_listener()
+    start_earth_news_target_worker()
     yield
+    await stop_earth_news_target_worker()
+    await stop_earth_db_change_listener()
+    await stop_data_job_worker()
     broadcaster.stop()
     stop_scheduler()
 
@@ -82,6 +99,14 @@ app.add_middleware(WebSocketCORSMiddleware)
 app.include_router(api_router, prefix="/api/v1")
 app.include_router(websocket.router)
 
+EARTH_BRAND_ASSET_DIR = Path(__file__).resolve().parents[2] / "data" / "earth-brand"
+EARTH_BRAND_ASSET_DIR.mkdir(parents=True, exist_ok=True)
+app.mount(
+    "/earth-brand-assets",
+    StaticFiles(directory=str(EARTH_BRAND_ASSET_DIR)),
+    name="earth-brand-assets",
+)
+
 
 @app.get("/health")
 async def health_check():

backend/app/models/__init__.py

@@ -13,9 +13,11 @@ from app.models.compute_center_location import ComputeCenterLocationRecord
 from app.models.system_setting import SystemSetting
 from app.models.playground_session import PlaygroundSession
 from app.models.playground_message import PlaygroundMessage
-from app.models.system_log import SystemLog, AuditLog
+from app.models.system_log import AuditLog, ObservabilityEvent, ObservabilityEventGroup, SystemLog
 from app.models.vessel import AISConflictRecord, AISRawObservation, AISSourceHealth, VesselPosition, VesselStatic
 from app.models.datasource_mapping import DataSourceMappingTemplate
+from app.models.earth_news import EarthNewsItem
+from app.models.earth_interactable import EarthInteractable
 
 __all__ = [
     "User",
@@ -35,6 +37,8 @@ __all__ = [
     "ComputeCenterLocationRecord",
     "SystemLog",
     "AuditLog",
+    "ObservabilityEvent",
+    "ObservabilityEventGroup",
     "PlaygroundSession",
     "PlaygroundMessage",
     "VesselPosition",
@@ -43,4 +47,6 @@ __all__ = [
     "AISConflictRecord",
     "AISSourceHealth",
     "DataSourceMappingTemplate",
+    "EarthNewsItem",
+    "EarthInteractable",
 ]

backend/app/models/alert.py

@@ -1,26 +1,12 @@
 from datetime import datetime
-from enum import Enum
-from typing import Optional
 
-from sqlalchemy import Column, Integer, String, DateTime, Text, ForeignKey, Enum as SQLEnum
-from sqlalchemy.orm import relationship
+from sqlalchemy import Column, Integer, String, DateTime, Text, Enum as SQLEnum
 
+from app.core.enums import AlertSeverity, AlertStatus
 from app.core.time import to_iso8601_utc
 from app.db.session import Base
 
 
-class AlertSeverity(str, Enum):
-    CRITICAL = "critical"
-    WARNING = "warning"
-    INFO = "info"
-
-
-class AlertStatus(str, Enum):
-    ACTIVE = "active"
-    ACKNOWLEDGED = "acknowledged"
-    RESOLVED = "resolved"
-
-
 class Alert(Base):
     __tablename__ = "alerts"
 

backend/app/models/bgp_anomaly.py

@@ -4,6 +4,7 @@ from datetime import datetime
 
 from sqlalchemy import Column, DateTime, Float, ForeignKey, Index, Integer, JSON, String, Text
 
+from app.core.enums import BGPStatus
 from app.core.time import to_iso8601_utc
 from app.db.session import Base
 
@@ -17,7 +18,7 @@ class BGPAnomaly(Base):
     source = Column(String(100), nullable=False, index=True)
     anomaly_type = Column(String(50), nullable=False, index=True)
     severity = Column(String(20), nullable=False, index=True)
-    status = Column(String(20), nullable=False, default="active", index=True)
+    status = Column(String(20), nullable=False, default=BGPStatus.ACTIVE.value, index=True)
     entity_key = Column(String(255), nullable=False, index=True)
     prefix = Column(String(64), nullable=True, index=True)
     origin_asn = Column(Integer, nullable=True, index=True)

backend/app/models/bgp_incident.py

@@ -4,6 +4,7 @@ from datetime import datetime
 
 from sqlalchemy import Column, DateTime, Float, ForeignKey, Index, Integer, JSON, String, Text
 
+from app.core.enums import BGPStatus
 from app.core.time import to_iso8601_utc
 from app.db.session import Base
 
@@ -20,7 +21,7 @@ class BGPIncident(Base):
     title = Column(String(255), nullable=False)
     summary = Column(Text, nullable=False)
     severity = Column(String(20), nullable=False, index=True)
-    status = Column(String(20), nullable=False, default="active", index=True)
+    status = Column(String(20), nullable=False, default=BGPStatus.ACTIVE.value, index=True)
     confidence = Column(Float, nullable=False, default=0.5)
     started_at = Column(DateTime(timezone=True), nullable=False, default=datetime.utcnow, index=True)
     ended_at = Column(DateTime(timezone=True), nullable=True)

backend/app/models/data_snapshot.py

@@ -1,6 +1,7 @@
 from sqlalchemy import Boolean, Column, DateTime, ForeignKey, Integer, JSON, String
 from sqlalchemy.sql import func
 
+from app.core.enums import SnapshotStatus
 from app.db.session import Base
 
 
@@ -16,7 +17,7 @@ class DataSnapshot(Base):
     started_at = Column(DateTime(timezone=True), server_default=func.now())
     completed_at = Column(DateTime(timezone=True), nullable=True)
     record_count = Column(Integer, default=0)
-    status = Column(String(20), nullable=False, default="running")
+    status = Column(String(20), nullable=False, default=SnapshotStatus.RUNNING.value)
     is_current = Column(Boolean, default=True, index=True)
     parent_snapshot_id = Column(Integer, ForeignKey("data_snapshots.id"), nullable=True, index=True)
     summary = Column(JSON, default={})

backend/app/models/datasource_mapping.py

@@ -3,6 +3,7 @@
 from sqlalchemy import Boolean, Column, DateTime, ForeignKey, Integer, JSON, String
 from sqlalchemy.sql import func
 
+from app.core.enums import MappingValidationStatus
 from app.db.session import Base
 
 
@@ -19,7 +20,7 @@ class DataSourceMappingTemplate(Base):
     target_schema = Column(String(80), nullable=False, index=True)
     mapping_json = Column(JSON, nullable=False, default={})
     sample_payload_hash = Column(String(64), nullable=True)
-    validation_status = Column(String(30), nullable=False, default="draft")
+    validation_status = Column(String(30), nullable=False, default=MappingValidationStatus.DRAFT.value)
     version = Column(Integer, nullable=False, default=1)
     is_active = Column(Boolean, nullable=False, default=False, index=True)
     created_at = Column(DateTime(timezone=True), server_default=func.now())

backend/app/models/earth_interactable.py

@@ -0,0 +1,30 @@
+"""Persistent Earth interactable objects."""
+
+from sqlalchemy import Boolean, Column, DateTime, Float, Index, Integer, JSON, String, Text
+from sqlalchemy.sql import func
+
+from app.db.session import Base
+
+
+class EarthInteractable(Base):
+    __tablename__ = "earth_interactables"
+
+    id = Column(String(160), primary_key=True)
+    layer = Column(String(80), nullable=False, default="interactables", index=True)
+    kind = Column(String(80), nullable=False, default="default", index=True)
+    label = Column(String(255), nullable=False, default="")
+    description = Column(Text, nullable=False, default="")
+    latitude = Column(Float, nullable=False)
+    longitude = Column(Float, nullable=False)
+    altitude = Column(Float, nullable=True)
+    revision = Column(Integer, nullable=False, default=1)
+    properties = Column(JSON, nullable=False, default=dict)
+    is_deleted = Column(Boolean, nullable=False, default=False, index=True)
+    created_at = Column(DateTime(timezone=True), server_default=func.now(), nullable=False)
+    updated_at = Column(DateTime(timezone=True), server_default=func.now(), onupdate=func.now())
+    deleted_at = Column(DateTime(timezone=True), nullable=True, index=True)
+
+    __table_args__ = (
+        Index("idx_earth_interactables_layer_deleted", "layer", "is_deleted"),
+        Index("idx_earth_interactables_updated_at", "updated_at"),
+    )

backend/app/models/earth_news.py

@@ -0,0 +1,40 @@
+from sqlalchemy import Boolean, Column, DateTime, Float, Index, JSON, String, Text
+from sqlalchemy.sql import func
+
+from app.db.session import Base
+
+
+class EarthNewsItem(Base):
+    __tablename__ = "earth_news_items"
+
+    id = Column(String(160), primary_key=True)
+    title = Column(String(500), nullable=False)
+    summary = Column(Text, nullable=False, default="")
+    content_language = Column(String(32), nullable=False, default="en")
+    localizations = Column(JSON, nullable=False, default=dict)
+    url = Column(Text, nullable=False)
+    source = Column(String(255), nullable=False, default="")
+    feed_name = Column(String(255), nullable=False, default="")
+    region = Column(String(80), nullable=False, index=True)
+    homepage_url = Column(Text, nullable=False, default="")
+    published_at = Column(DateTime(timezone=True), nullable=True, index=True)
+
+    latitude = Column(Float, nullable=False)
+    longitude = Column(Float, nullable=False)
+    location_label = Column(String(255), nullable=False)
+    location_source = Column(String(80), nullable=False, default="region_anchor")
+    verified = Column(Boolean, nullable=False, default=False, index=True)
+    location_meta = Column(JSON, nullable=False, default=dict)
+
+    first_seen_at = Column(DateTime(timezone=True), server_default=func.now(), nullable=False)
+    last_seen_at = Column(DateTime(timezone=True), server_default=func.now(), nullable=False, index=True)
+    resolved_at = Column(DateTime(timezone=True), nullable=True, index=True)
+    enrichment_status = Column(String(80), nullable=False, default="pending", index=True)
+    enrichment_error = Column(Text, nullable=True)
+    enriched_at = Column(DateTime(timezone=True), nullable=True, index=True)
+    updated_at = Column(DateTime(timezone=True), server_default=func.now(), onupdate=func.now())
+
+    __table_args__ = (
+        Index("idx_earth_news_region_published", "region", "published_at"),
+        Index("idx_earth_news_region_seen", "region", "last_seen_at"),
+    )

backend/app/models/playground_message.py

@@ -1,6 +1,7 @@
 from sqlalchemy import JSON, Boolean, Column, DateTime, ForeignKey, Integer, String, Text
 from sqlalchemy.sql import func
 
+from app.core.enums import PlaygroundMessageKind, PlaygroundMessageStatus
 from app.db.session import Base
 
 
@@ -13,8 +14,8 @@ class PlaygroundMessage(Base):
     user_id = Column(Integer, ForeignKey("users.id", ondelete="CASCADE"), nullable=False, index=True)
     parent_message_id = Column(Integer, ForeignKey("playground_messages.id", ondelete="SET NULL"), nullable=True)
     role = Column(String(20), nullable=False)
-    kind = Column(String(20), nullable=False, default="message")
-    status = Column(String(20), nullable=False, default="done")
+    kind = Column(String(20), nullable=False, default=PlaygroundMessageKind.MESSAGE.value)
+    status = Column(String(20), nullable=False, default=PlaygroundMessageStatus.DONE.value)
     title = Column(String(255), nullable=True)
     content = Column(Text, nullable=False, default="")
     thinking_content = Column(Text, nullable=False, default="")

backend/app/models/system_log.py

@@ -38,3 +38,46 @@ class AuditLog(Base):
     ip = Column(String(64), nullable=True)
     details = Column(JSON, nullable=False, default=dict)
     created_at = Column(DateTime(timezone=True), server_default=func.now())
+
+
+class ObservabilityEvent(Base):
+    __tablename__ = "observability_events"
+
+    id = Column(Integer, primary_key=True, autoincrement=True)
+    occurred_at = Column(DateTime(timezone=True), server_default=func.now(), index=True)
+    source = Column(String(50), nullable=False, index=True)
+    service = Column(String(50), nullable=True, index=True)
+    module = Column(String(120), nullable=True, index=True)
+    category = Column(String(80), nullable=True, index=True)
+    event = Column(String(160), nullable=True, index=True)
+    level = Column(String(20), nullable=False, index=True)
+    message = Column(Text, nullable=False)
+    fingerprint = Column(String(80), nullable=False, index=True)
+    request_id = Column(String(64), nullable=True, index=True)
+    trace_id = Column(String(64), nullable=True, index=True)
+    task_id = Column(String(120), nullable=True, index=True)
+    source_ref_id = Column(String(120), nullable=True, index=True)
+    provider = Column(String(120), nullable=True, index=True)
+    user_id = Column(Integer, nullable=True, index=True)
+    context = Column(JSON, nullable=False, default=dict)
+    occurrence_count = Column(Integer, nullable=False, default=1)
+    created_at = Column(DateTime(timezone=True), server_default=func.now())
+
+
+class ObservabilityEventGroup(Base):
+    __tablename__ = "observability_event_groups"
+
+    fingerprint = Column(String(80), primary_key=True)
+    source = Column(String(50), nullable=False, index=True)
+    service = Column(String(50), nullable=True, index=True)
+    module = Column(String(120), nullable=True, index=True)
+    category = Column(String(80), nullable=True, index=True)
+    event = Column(String(160), nullable=True, index=True)
+    last_level = Column(String(20), nullable=False, index=True)
+    sample_message = Column(Text, nullable=False)
+    sample_detail = Column(Text, nullable=True)
+    affected_sources = Column(JSON, nullable=False, default=list)
+    count = Column(Integer, nullable=False, default=0)
+    first_seen_at = Column(DateTime(timezone=True), nullable=False, index=True)
+    last_seen_at = Column(DateTime(timezone=True), nullable=False, index=True)
+    updated_at = Column(DateTime(timezone=True), server_default=func.now(), onupdate=func.now())

backend/app/models/task.py

@@ -1,8 +1,9 @@
-"""Collection Task model"""
+"""Datasource job model."""
 
-from sqlalchemy import BigInteger, Column, DateTime, Integer, String, Text, Float
+from sqlalchemy import BigInteger, Column, DateTime, Float, Integer, JSON, String, Text
 from sqlalchemy.sql import func
 
+from app.core.enums import JobStatus, JobType, RollbackPolicy
 from app.db.session import Base
 
 
@@ -11,8 +12,10 @@ class CollectionTask(Base):
 
     id = Column(Integer, primary_key=True, autoincrement=True)
     datasource_id = Column(Integer, nullable=False, index=True)
-    status = Column(String(20), nullable=False)  # pending, running, success, failed, cancelled
-    phase = Column(String(30), default="queued")
+    source = Column(String(100), nullable=True, index=True)
+    task_type = Column(String(30), nullable=False, default=JobType.COLLECT.value, index=True)
+    status = Column(String(20), nullable=False)  # queued, running, cancelling, success, failed, cancelled
+    phase = Column(String(30), default=JobStatus.QUEUED.value)
     phase_progress = Column(Float)
     phase_message = Column(String(255))
     phase_current = Column(BigInteger)
@@ -24,6 +27,13 @@ class CollectionTask(Base):
     total_records = Column(Integer, default=0)  # Total records to process
     progress = Column(Float, default=0.0)  # Progress percentage (0-100)
     error_message = Column(Text)
+    payload = Column(JSON, default=dict)
+    rollback_policy = Column(String(40), nullable=False, default=RollbackPolicy.KEEP_COMMITTED_BATCHES.value)
+    dedupe_key = Column(String(180), nullable=True, index=True)
+    worker_id = Column(String(120), nullable=True, index=True)
+    locked_at = Column(DateTime(timezone=True), nullable=True, index=True)
+    requested_cancel_at = Column(DateTime(timezone=True), nullable=True)
+    cancel_reason = Column(Text)
     created_at = Column(DateTime(timezone=True), server_default=func.now())
 
     def __repr__(self):

backend/app/models/user.py

@@ -1,6 +1,7 @@
 from sqlalchemy import Boolean, Column, DateTime, Integer, JSON, String
 from sqlalchemy.sql import func
 
+from app.core.enums import UserRole
 from app.db.session import Base
 
 
@@ -11,7 +12,7 @@ class User(Base):
     username = Column(String(50), unique=True, index=True, nullable=False)
     email = Column(String(255), unique=True, index=True, nullable=False)
     password_hash = Column(String(255), nullable=False)
-    role = Column(String(20), default="viewer")
+    role = Column(String(20), default=UserRole.VIEWER.value)
     gatekeeper_groups = Column(JSON, default=list)
     is_active = Column(Boolean, default=True)
     email_verified = Column(Boolean, default=False, nullable=False)

backend/app/models/vessel.py

@@ -3,6 +3,7 @@
 from sqlalchemy import BigInteger, Column, DateTime, Float, Index, Integer, JSON, SmallInteger, String
 from sqlalchemy.sql import func
 
+from app.core.enums import ConnectionState
 from app.core.time import to_iso8601_utc
 from app.db.session import Base
 
@@ -75,6 +76,67 @@ class VesselPosition(Base):
         }
 
 
+class VesselCurrentState(Base):
+    """Latest renderable state for one vessel, independent from AIS history."""
+
+    __tablename__ = "vessel_current_state"
+
+    mmsi = Column(BigInteger, primary_key=True)
+    lat = Column(Float, nullable=False)
+    lon = Column(Float, nullable=False)
+    sog = Column(Float, nullable=True)
+    cog = Column(Float, nullable=True)
+    heading = Column(SmallInteger, nullable=True)
+    nav_status = Column(SmallInteger, nullable=True, index=True)
+    name = Column(String(128), nullable=True)
+    callsign = Column(String(16), nullable=True)
+    vessel_type = Column(SmallInteger, nullable=True, index=True)
+    vessel_type_name = Column(String(64), nullable=True, index=True)
+    flag = Column(String(4), nullable=True, index=True)
+    length = Column(Float, nullable=True)
+    width = Column(Float, nullable=True)
+    draught = Column(Float, nullable=True)
+    imo = Column(BigInteger, nullable=True)
+    source = Column(String(100), nullable=False, index=True)
+    observed_at = Column(DateTime(timezone=True), nullable=False)
+    field_sources = Column(JSON, default=dict)
+    selected_reasons = Column(JSON, default=dict)
+    source_summary = Column(JSON, default=dict)
+    quality_flags = Column(JSON, default=list)
+    updated_at = Column(DateTime(timezone=True), nullable=False, server_default=func.now())
+
+    __table_args__ = (
+        Index("idx_vessel_current_bbox", "lon", "lat"),
+        Index("idx_vessel_current_observed", "observed_at"),
+    )
+
+    def to_dict(self) -> dict:
+        return {
+            "mmsi": self.mmsi,
+            "lat": self.lat,
+            "lon": self.lon,
+            "sog": self.sog,
+            "cog": self.cog,
+            "heading": self.heading,
+            "nav_status": self.nav_status,
+            "name": self.name,
+            "callsign": self.callsign,
+            "vessel_type": self.vessel_type,
+            "vessel_type_name": self.vessel_type_name,
+            "flag": self.flag,
+            "length": self.length,
+            "width": self.width,
+            "draught": self.draught,
+            "imo": self.imo,
+            "source": self.source,
+            "received_at": self.observed_at,
+            "field_sources": self.field_sources or {},
+            "selected_reasons": self.selected_reasons or {},
+            "source_summary": self.source_summary or {},
+            "quality_flags": self.quality_flags or [],
+        }
+
+
 class AISRawObservation(Base):
     """Source-level AIS fact before aggregation and conflict resolution."""
 
@@ -165,7 +227,7 @@ class AISSourceHealth(Base):
     __tablename__ = "ais_source_health"
 
     source = Column(String(100), primary_key=True)
-    connection_state = Column(String(32), nullable=False, default="disconnected", index=True)
+    connection_state = Column(String(32), nullable=False, default=ConnectionState.DISCONNECTED.value, index=True)
     last_seen_at = Column(DateTime(timezone=True), nullable=True, index=True)
     last_success_at = Column(DateTime(timezone=True), nullable=True, index=True)
     last_error = Column(String(500), nullable=True)

backend/app/schemas/ai.py

@@ -2,6 +2,7 @@ from typing import Any
 
 from pydantic import BaseModel, Field
 
+from app.core.enums import PlaygroundMessageKind, PlaygroundMessageRole, PlaygroundMessageStatus
 
 class AIContentBlock(BaseModel):
     type: str
@@ -13,10 +14,11 @@ class AIContentBlock(BaseModel):
 
 class SituationalAnalysisRequest(BaseModel):
     title: str = Field(..., min_length=1, max_length=200)
-    objective: str = Field(..., min_length=1, max_length=1000)
+    objective: str = Field(..., min_length=1, max_length=20000)
     context: dict[str, Any] = Field(default_factory=dict)
     observations: list[str] = Field(default_factory=list)
     constraints: list[str] = Field(default_factory=list)
+    system_prompt: str | None = Field(default=None, max_length=8000)
     preferred_model: str | None = Field(default=None, max_length=200)
     thinking: dict[str, Any] | None = None
 
@@ -109,9 +111,9 @@ class PlaygroundSessionUpsertRequest(BaseModel):
 
 class PlaygroundMessageRecord(BaseModel):
     id: str
-    role: str
-    kind: str = "message"
-    status: str = "done"
+    role: PlaygroundMessageRole
+    kind: PlaygroundMessageKind = PlaygroundMessageKind.MESSAGE
+    status: PlaygroundMessageStatus = PlaygroundMessageStatus.DONE
     title: str | None = None
     content: str = ""
     thinking_content: str = ""

backend/app/schemas/user.py

@@ -3,6 +3,7 @@ from typing import Optional
 
 from pydantic import BaseModel, EmailStr, Field
 
+from app.core.enums import OtpPurpose, UserRole
 
 class UserBase(BaseModel):
     username: str
@@ -11,13 +12,13 @@ class UserBase(BaseModel):
 
 class UserCreate(UserBase):
     password: str = Field(..., min_length=8)
-    role: str = "viewer"
+    role: UserRole = UserRole.VIEWER
     gatekeeper_groups: list[str] = Field(default_factory=list)
 
 
 class UserUpdate(BaseModel):
     email: Optional[EmailStr] = None
-    role: Optional[str] = None
+    role: Optional[UserRole] = None
     gatekeeper_groups: Optional[list[str]] = None
     is_active: Optional[bool] = None
 
@@ -59,7 +60,7 @@ class VerifyEmailRequest(BaseModel):
 
 class ResendCodeRequest(BaseModel):
     email: EmailStr
-    purpose: str = Field(default="register", pattern="^(register|verify_email|reset_password)$")
+    purpose: OtpPurpose = OtpPurpose.REGISTER
 
 
 class ForgotPasswordRequest(BaseModel):

backend/app/services/ai_client.py

@@ -1,18 +1,25 @@
 from __future__ import annotations
 
 import asyncio
+import json
+from time import perf_counter
 
 import httpx
 from fastapi import Depends, HTTPException, status
 from sqlalchemy.ext.asyncio import AsyncSession
 
 from app.core.config import settings
+from app.core.logging import get_logger
 from app.db.session import get_db
 from app.schemas.ai import (
     AIProviderStatusResponse,
     SituationalAnalysisRequest,
     SituationalAnalysisResponse,
 )
+from app.services.business_logs import emit_business_log, exception_context
+
+
+logger = get_logger(__name__, service="ai")
 
 
 class AIProviderClient:
@@ -57,10 +64,25 @@ class AIProviderClient:
             value = self.llm_config.get(key)
             if value not in (None, ""):
                 headers[header_name] = str(value)
+        model_provider_apis = self.llm_config.get("model_provider_apis")
+        if isinstance(model_provider_apis, dict) and model_provider_apis:
+            headers["X-AI-Model-Provider-APIs"] = json.dumps(model_provider_apis)
         return headers
 
     async def get_status(self, request_id: str | None = None) -> AIProviderStatusResponse:
+        context = self._base_log_context(operation="status")
         if not self.service_url:
+            await emit_business_log(
+                logger,
+                event="ai.provider.status.failed",
+                message="AI provider status skipped because service URL is not configured",
+                category="ai",
+                level="warning",
+                service="ai",
+                module=__name__,
+                request_id=request_id,
+                context={**context, "status": "unconfigured"},
+            )
             return AIProviderStatusResponse(
                 provider="unconfigured",
                 enabled=False,
@@ -69,27 +91,133 @@ class AIProviderClient:
                 base_url=None,
             )
 
-        data = await self._request("GET", "/v1/provider/status", request_id=request_id)
-        return AIProviderStatusResponse.model_validate(data)
+        started_at = perf_counter()
+        await emit_business_log(
+            logger,
+            event="ai.provider.status.start",
+            message="AI provider status request started",
+            category="ai",
+            service="ai",
+            module=__name__,
+            request_id=request_id,
+            context=context,
+        )
+        try:
+            data = await self._request("GET", "/v1/provider/status", request_id=request_id, operation="status")
+            result = AIProviderStatusResponse.model_validate(data)
+            await emit_business_log(
+                logger,
+                event="ai.provider.status.success",
+                message="AI provider status request completed",
+                category="ai",
+                service="ai",
+                module=__name__,
+                request_id=request_id,
+                context={
+                    **context,
+                    "status": "success",
+                    "duration_ms": self._duration_ms(started_at),
+                    "result_provider": result.provider,
+                    "result_model": result.model,
+                    "configured": result.configured,
+                    "enabled": result.enabled,
+                },
+            )
+            return result
+        except Exception as exc:
+            await emit_business_log(
+                logger,
+                event="ai.provider.status.failed",
+                message="AI provider status request failed",
+                category="ai",
+                level="error",
+                service="ai",
+                module=__name__,
+                request_id=request_id,
+                context=exception_context(exc, {**context, "status": "failed", "duration_ms": self._duration_ms(started_at)}),
+            )
+            raise
 
     async def analyze(
         self,
         payload: SituationalAnalysisRequest,
         request_id: str | None = None,
     ) -> SituationalAnalysisResponse:
+        context = self._base_log_context(
+            operation="analyze",
+            preferred_model=payload.preferred_model,
+            input_summary=self._summarize_analysis_payload(payload),
+        )
         if not self.service_url:
+            await emit_business_log(
+                logger,
+                event="ai.provider.analyze.failed",
+                message="AI provider analyze skipped because service URL is not configured",
+                category="ai",
+                level="warning",
+                service="ai",
+                module=__name__,
+                request_id=request_id,
+                context={**context, "status": "unconfigured"},
+            )
             raise HTTPException(
                 status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
                 detail="AI provider service URL is not configured.",
             )
 
-        data = await self._request(
-            "POST",
-            "/v1/analyze",
-            json=payload.model_dump(),
+        started_at = perf_counter()
+        await emit_business_log(
+            logger,
+            event="ai.provider.analyze.start",
+            message="AI provider analyze request started",
+            category="ai",
+            service="ai",
+            module=__name__,
             request_id=request_id,
+            context=context,
         )
-        return SituationalAnalysisResponse.model_validate(data)
+        try:
+            data = await self._request(
+                "POST",
+                "/v1/analyze",
+                json=payload.model_dump(),
+                request_id=request_id,
+                operation="analyze",
+                payload_summary=context["input_summary"],
+            )
+            result = SituationalAnalysisResponse.model_validate(data)
+            await emit_business_log(
+                logger,
+                event="ai.provider.analyze.success",
+                message="AI provider analyze request completed",
+                category="ai",
+                service="ai",
+                module=__name__,
+                request_id=request_id,
+                context={
+                    **context,
+                    "status": "success",
+                    "duration_ms": self._duration_ms(started_at),
+                    "result_provider": result.provider,
+                    "result_model": result.model,
+                    "content_block_count": len(result.content_blocks or []),
+                    "thinking_block_count": len(result.thinking_blocks or []),
+                },
+            )
+            return result
+        except Exception as exc:
+            await emit_business_log(
+                logger,
+                event="ai.provider.analyze.failed",
+                message="AI provider analyze request failed",
+                category="ai",
+                level="error",
+                service="ai",
+                module=__name__,
+                request_id=request_id,
+                context=exception_context(exc, {**context, "status": "failed", "duration_ms": self._duration_ms(started_at)}),
+            )
+            raise
 
     async def _request(
         self,
@@ -97,9 +225,12 @@ class AIProviderClient:
         path: str,
         json: dict | None = None,
         request_id: str | None = None,
+        operation: str = "request",
+        payload_summary: dict | None = None,
     ) -> dict:
         last_error: Exception | None = None
         for attempt in range(1, self.retry_attempts + 1):
+            attempt_started_at = perf_counter()
             try:
                 async with httpx.AsyncClient(timeout=self.timeout) as client:
                     response = await client.request(
@@ -113,6 +244,15 @@ class AIProviderClient:
             except httpx.HTTPStatusError as exc:
                 last_error = exc
                 if attempt < self.retry_attempts and exc.response.status_code >= 500:
+                    await self._log_retry(
+                        operation=operation,
+                        request_id=request_id,
+                        attempt=attempt,
+                        status_code=exc.response.status_code,
+                        duration_ms=self._duration_ms(attempt_started_at),
+                        error=exc,
+                        payload_summary=payload_summary,
+                    )
                     await asyncio.sleep(0.3 * attempt)
                     continue
                 detail = exc.response.text or "AI provider service returned an error"
@@ -123,6 +263,14 @@ class AIProviderClient:
             except httpx.HTTPError as exc:
                 last_error = exc
                 if attempt < self.retry_attempts:
+                    await self._log_retry(
+                        operation=operation,
+                        request_id=request_id,
+                        attempt=attempt,
+                        duration_ms=self._duration_ms(attempt_started_at),
+                        error=exc,
+                        payload_summary=payload_summary,
+                    )
                     await asyncio.sleep(0.3 * attempt)
                     continue
                 raise HTTPException(
@@ -135,6 +283,71 @@ class AIProviderClient:
             detail=f"AI provider service request failed: {last_error}",
         )
 
+    def _base_log_context(self, **extra: object) -> dict:
+        llm_provider_apis = self.llm_config.get("model_provider_apis")
+        return {
+            "provider": self.llm_config.get("provider") or "",
+            "provider_api": self.llm_config.get("provider_api") or "",
+            "model": self.llm_config.get("model") or "",
+            "base_url_configured": bool(self.llm_config.get("base_url")),
+            "service_url_configured": bool(self.service_url),
+            "timeout_seconds": self.timeout,
+            "retry_attempts": self.retry_attempts,
+            "model_provider_api_count": len(llm_provider_apis or {}) if isinstance(llm_provider_apis, dict) else 0,
+            **extra,
+        }
+
+    @staticmethod
+    def _duration_ms(started_at: float) -> int:
+        return int((perf_counter() - started_at) * 1000)
+
+    @staticmethod
+    def _summarize_analysis_payload(payload: SituationalAnalysisRequest) -> dict:
+        context = payload.context if isinstance(payload.context, dict) else {}
+        thinking = payload.thinking if isinstance(payload.thinking, dict) else payload.thinking
+        return {
+            "title_length": len(payload.title or ""),
+            "objective_length": len(payload.objective or ""),
+            "observation_count": len(payload.observations or []),
+            "constraint_count": len(payload.constraints or []),
+            "has_system_prompt": bool(payload.system_prompt),
+            "thinking_enabled": bool(thinking),
+            "context_keys": sorted(str(key) for key in context.keys()),
+        }
+
+    async def _log_retry(
+        self,
+        *,
+        operation: str,
+        request_id: str | None,
+        attempt: int,
+        duration_ms: int,
+        error: BaseException,
+        status_code: int | None = None,
+        payload_summary: dict | None = None,
+    ) -> None:
+        await emit_business_log(
+            logger,
+            event=f"ai.provider.{operation}.retry",
+            message="AI provider request will retry",
+            category="ai",
+            level="warning",
+            service="ai",
+            module=__name__,
+            request_id=request_id,
+            context=exception_context(
+                error,
+                {
+                    **self._base_log_context(operation=operation),
+                    "attempt": attempt,
+                    "next_attempt": attempt + 1,
+                    "status_code": status_code,
+                    "duration_ms": duration_ms,
+                    "input_summary": payload_summary,
+                },
+            ),
+        )
+
 
 async def get_ai_provider_client(db: AsyncSession = Depends(get_db)) -> AIProviderClient:
     from app.api.v1.settings import get_runtime_ai_provider_config

backend/app/services/ai_tools/web_fetch.py

@@ -1,11 +1,18 @@
 from __future__ import annotations
 
 import hashlib
+from time import perf_counter
+from urllib.parse import urlparse
 
 import httpx
 from bs4 import BeautifulSoup
 
+from app.core.logging import get_logger
 from app.services.ai_tools.schemas import FetchedEvidence
+from app.services.business_logs import emit_business_log, exception_context
+
+
+logger = get_logger(__name__, service="ai_tool")
 
 
 class WebFetchError(RuntimeError):
@@ -29,8 +36,33 @@ async def fetch_url_evidence(
     timeout_seconds: int = 20,
     max_bytes: int = 1_500_000,
 ) -> FetchedEvidence:
+    started_at = perf_counter()
     if not url:
+        await emit_business_log(
+            logger,
+            event="ai_tool.web_fetch.failed",
+            message="WebFetch failed because URL is empty",
+            category="ai_tool",
+            level="warning",
+            service="ai_tool",
+            module=__name__,
+            context={"reason": "empty_url"},
+        )
         raise WebFetchError("url is required")
+    request_host = urlparse(url).netloc
+    await emit_business_log(
+        logger,
+        event="ai_tool.web_fetch.start",
+        message="WebFetch request started",
+        category="ai_tool",
+        service="ai_tool",
+        module=__name__,
+        context={
+            "url_host": request_host,
+            "timeout_seconds": timeout_seconds,
+            "max_bytes": max_bytes,
+        },
+    )
     try:
         async with httpx.AsyncClient(
             timeout=timeout_seconds,
@@ -41,10 +73,45 @@ async def fetch_url_evidence(
             response.raise_for_status()
             content = response.content[:max_bytes]
     except httpx.HTTPError as exc:
+        await emit_business_log(
+            logger,
+            event="ai_tool.web_fetch.failed",
+            message="WebFetch request failed",
+            category="ai_tool",
+            level="error",
+            service="ai_tool",
+            module=__name__,
+            context=exception_context(
+                exc,
+                {
+                    "url_host": request_host,
+                    "status": "failed",
+                    "duration_ms": int((perf_counter() - started_at) * 1000),
+                },
+            ),
+        )
         raise WebFetchError(f"failed to fetch page: {exc}") from exc
 
     title, text = _extract_title_and_text(content.decode(response.encoding or "utf-8", errors="ignore"))
     content_hash = hashlib.sha256(text.encode("utf-8")).hexdigest()
+    await emit_business_log(
+        logger,
+        event="ai_tool.web_fetch.success",
+        message="WebFetch request completed",
+        category="ai_tool",
+        service="ai_tool",
+        module=__name__,
+        context={
+            "url_host": request_host,
+            "final_url_host": urlparse(str(response.url)).netloc,
+            "status": "success",
+            "status_code": response.status_code,
+            "bytes_read": len(content),
+            "content_hash": content_hash,
+            "duration_ms": int((perf_counter() - started_at) * 1000),
+            "extractor": "beautifulsoup_basic",
+        },
+    )
     return FetchedEvidence(
         url=url,
         final_url=str(response.url),
@@ -53,4 +120,3 @@ async def fetch_url_evidence(
         content_hash=content_hash,
         extractor="beautifulsoup_basic",
     )
-

backend/app/services/ai_tools/web_search.py

@@ -1,11 +1,18 @@
 from __future__ import annotations
 
 from copy import deepcopy
+import hashlib
+from time import perf_counter
 from typing import Any
 
 import httpx
 
+from app.core.logging import get_logger
 from app.services.ai_tools.schemas import SearchEvidence, WebSearchConfig, WebSearchProviderConfig
+from app.services.business_logs import emit_business_log, exception_context
+
+
+logger = get_logger(__name__, service="ai_tool")
 
 
 WEB_SEARCH_PROVIDER_PRESETS: dict[str, dict[str, Any]] = {
@@ -120,29 +127,108 @@ class WebSearchClient:
         domains: list[str] | None = None,
         freshness_days: int | None = None,
     ) -> list[SearchEvidence]:
+        started_at = perf_counter()
         if not self.config.enabled:
+            await emit_business_log(
+                logger,
+                event="ai_tool.web_search.unavailable",
+                message="WebSearch skipped because integration is disabled",
+                category="ai_tool",
+                level="warning",
+                service="ai_tool",
+                module=__name__,
+                context={"provider": self.config.default_provider, "reason": "disabled"},
+            )
             raise WebSearchConfigurationError("WebSearch is disabled.")
         provider_config = self.config.active_provider_config
         provider = normalize_web_search_provider(provider_config.provider)
         if provider != "searxng" and not provider_config.api_key:
+            await emit_business_log(
+                logger,
+                event="ai_tool.web_search.unavailable",
+                message="WebSearch skipped because API key is not configured",
+                category="ai_tool",
+                level="warning",
+                service="ai_tool",
+                module=__name__,
+                context={"provider": provider, "reason": "missing_api_key"},
+            )
             raise WebSearchConfigurationError(f"{provider} API key is not configured.")
         query = " ".join(str(query or "").split())
         if not query:
+            await emit_business_log(
+                logger,
+                event="ai_tool.web_search.failed",
+                message="WebSearch failed because query is empty",
+                category="ai_tool",
+                level="warning",
+                service="ai_tool",
+                module=__name__,
+                context={"provider": provider, "reason": "empty_query"},
+            )
             raise WebSearchConfigurationError("search query is required.")
         limit = max_results or provider_config.max_results
-        if provider == "tavily":
-            return await self._search_tavily(provider_config, query, limit, domains, freshness_days)
-        if provider == "brave":
-            return await self._search_brave(provider_config, query, limit, domains)
-        if provider == "serpapi":
-            return await self._search_serpapi(provider_config, query, limit)
-        if provider == "exa":
-            return await self._search_exa(provider_config, query, limit, domains)
-        if provider == "firecrawl":
-            return await self._search_firecrawl(provider_config, query, limit)
-        if provider == "searxng":
-            return await self._search_searxng(provider_config, query, limit, domains)
-        raise WebSearchConfigurationError(f"Unsupported web search provider: {provider}")
+        context = {
+            "provider": provider,
+            "query_hash": hashlib.sha256(query.encode("utf-8")).hexdigest(),
+            "query_length": len(query),
+            "max_results": limit,
+            "domain_count": len(domains or []),
+            "freshness_days": freshness_days,
+        }
+        await emit_business_log(
+            logger,
+            event="ai_tool.web_search.start",
+            message="WebSearch request started",
+            category="ai_tool",
+            service="ai_tool",
+            module=__name__,
+            context=context,
+        )
+        try:
+            if provider == "tavily":
+                results = await self._search_tavily(provider_config, query, limit, domains, freshness_days)
+            elif provider == "brave":
+                results = await self._search_brave(provider_config, query, limit, domains)
+            elif provider == "serpapi":
+                results = await self._search_serpapi(provider_config, query, limit)
+            elif provider == "exa":
+                results = await self._search_exa(provider_config, query, limit, domains)
+            elif provider == "firecrawl":
+                results = await self._search_firecrawl(provider_config, query, limit)
+            elif provider == "searxng":
+                results = await self._search_searxng(provider_config, query, limit, domains)
+            else:
+                raise WebSearchConfigurationError(f"Unsupported web search provider: {provider}")
+            event = "ai_tool.web_search.success" if results else "ai_tool.web_search.empty"
+            await emit_business_log(
+                logger,
+                event=event,
+                message="WebSearch request completed" if results else "WebSearch returned no results",
+                category="ai_tool",
+                level="info" if results else "warning",
+                service="ai_tool",
+                module=__name__,
+                context={
+                    **context,
+                    "status": "success" if results else "empty",
+                    "result_count": len(results),
+                    "duration_ms": int((perf_counter() - started_at) * 1000),
+                },
+            )
+            return results
+        except Exception as exc:
+            await emit_business_log(
+                logger,
+                event="ai_tool.web_search.failed",
+                message="WebSearch request failed",
+                category="ai_tool",
+                level="error",
+                service="ai_tool",
+                module=__name__,
+                context=exception_context(exc, {**context, "status": "failed", "duration_ms": int((perf_counter() - started_at) * 1000)}),
+            )
+            raise
 
     async def test_connection(self) -> list[SearchEvidence]:
         return await self.search("Planet WebSearch connectivity test", max_results=1)
@@ -388,4 +474,3 @@ def _float_or_none(value: Any) -> float | None:
         return float(value)
     except (TypeError, ValueError):
         return None
-

backend/app/services/alert_ai_brief.py

@@ -8,6 +8,9 @@ from sqlalchemy.ext.asyncio import AsyncSession
 
 from app.models.alert import Alert, AlertSeverity, AlertStatus
 from app.schemas.ai import AlertBriefRequest, SituationalAnalysisRequest
+from app.ai_tasks.prompts import get_effective_prompt
+
+ALERT_BRIEF_PROMPT_KEY = "alerts.brief"
 
 
 def _format_counter(counter: Counter[str], empty_text: str = "无") -> str:
@@ -84,11 +87,13 @@ async def build_alert_brief_request(
         "top_datasources": dict(datasource_counts.most_common(6)),
         "top_active_datasources": dict(active_datasource_counts.most_common(5)),
     }
+    prompt = await get_effective_prompt(db, ALERT_BRIEF_PROMPT_KEY)
 
     return (
         SituationalAnalysisRequest(
             title="告警态势 AI 简报",
-            objective="基于当前告警总量、严重度、状态、数据源分布与最近告警摘录，生成一份面向值班人员的简明告警态势简报，突出待处理风险、告警集中点和优先动作。",
+            objective=prompt.prompt,
+            system_prompt=prompt.system_prompt or None,
             observations=facts,
             constraints=[
                 "明确区分事实、推断与建议。",

backend/app/services/bgp_ai_brief.py

@@ -11,9 +11,12 @@ from app.models.bgp_anomaly import BGPAnomaly
 from app.models.bgp_incident import BGPIncident
 from app.models.bgp_observation import BGPObservation
 from app.schemas.ai import SituationalAnalysisRequest
+from app.ai_tasks.prompts import get_effective_prompt
 from app.services.bgp_collectors import build_bgp_collector_coverage
 from app.services.bgp_enrichment import lookup_prefix_geography
 
+BGP_BRIEF_PROMPT_KEY = "bgp.brief"
+
 
 def _format_counter(counter: dict[str, int], empty_text: str = "无") -> str:
     if not counter:
@@ -243,12 +246,15 @@ async def build_bgp_brief_request(
             for prefix, item in list(prefix_geographies.items())[:8]
         },
     }
+    prompt = await get_effective_prompt(db, BGP_BRIEF_PROMPT_KEY)
 
     return SituationalAnalysisRequest(
         title="BGP 态势 AI 简报",
-        objective="基于当前 BGP incidents、anomalies、原始观测事件、观测站覆盖与 prefix geography 证据，生成一份面向操作员的简明态势简报，突出区域热点、观测偏差、当前风险、证据和优先动作。",
+        objective=prompt.prompt,
+        system_prompt=prompt.system_prompt or None,
         observations=observations_lines,
         constraints=[
+            "直接输出中文 Markdown 简报正文，不要输出英文写作计划、提示词复述、字段说明或元评论。",
             "明确区分事实、推断与建议。",
             "优先指出需要立即关注的高严重度 incident 或异常模式。",
             "需要单独指出哪些区域结论来自 prefix geography / affected regions，哪些可能受 collector coverage 偏差影响。",

backend/app/services/bgp_detectors.py

@@ -6,6 +6,7 @@ from collections import Counter, defaultdict
 from datetime import UTC, datetime
 from typing import Any
 
+from app.core.enums import BGPStatus
 from app.models.bgp_anomaly import BGPAnomaly
 
 
@@ -127,7 +128,7 @@ def detect_origin_change_anomalies(
                     source=source,
                     anomaly_type=anomaly_type,
                     severity=severity,
-                    status="active",
+                    status=BGPStatus.ACTIVE.value,
                     entity_key=f"{anomaly_type}:{prefix}:{new_origin}",
                     prefix=prefix,
                     origin_asn=sorted(historic)[0] if historic else None,
@@ -197,7 +198,7 @@ def detect_more_specific_burst_anomalies(
                 source=source,
                 anomaly_type="more_specific_burst",
                 severity="high",
-                status="active",
+                status=BGPStatus.ACTIVE.value,
                 entity_key=f"more_specific_burst:{root_prefix}:{len(unique_prefixes)}:{len(related_collectors)}",
                 prefix=sample.get("prefix"),
                 origin_asn=sample.get("origin_asn"),
@@ -267,7 +268,7 @@ def detect_mass_withdrawal_anomalies(
                 source=source,
                 anomaly_type="mass_withdrawal",
                 severity=severity,
-                status="active",
+                status=BGPStatus.ACTIVE.value,
                 entity_key=f"mass_withdrawal:{prefix}:{origin_asn}:{len(related_collectors)}:{count}",
                 prefix=prefix,
                 origin_asn=origin_asn,
@@ -354,7 +355,7 @@ def detect_route_leak_anomalies(
                 source=source,
                 anomaly_type="route_leak_candidate",
                 severity="high" if max_path_length >= dominant_length + 3 else "medium",
-                status="active",
+                status=BGPStatus.ACTIVE.value,
                 entity_key=f"route_leak_candidate:{prefix}:{max_path_length}:{len(related_collectors)}",
                 prefix=prefix,
                 origin_asn=sample_metadata.get("origin_asn"),
@@ -435,7 +436,7 @@ def detect_path_flap_anomalies(
                 source=source,
                 anomaly_type="path_flap",
                 severity=severity,
-                status="active",
+                status=BGPStatus.ACTIVE.value,
                 entity_key=f"path_flap:{prefix}:{transitions}:{len(distinct_paths)}",
                 prefix=prefix,
                 origin_asn=sample_metadata.get("origin_asn"),

backend/app/services/bgp_incidents.py

@@ -9,6 +9,7 @@ from sqlalchemy import select
 from sqlalchemy.ext.asyncio import AsyncSession
 
 from app.core.collected_data_fields import get_record_field
+from app.core.enums import BGPStatus
 from app.models.bgp_anomaly import BGPAnomaly
 from app.models.bgp_incident import BGPIncident
 from app.models.collected_data import CollectedData
@@ -290,7 +291,7 @@ async def create_bgp_incidents_for_anomalies(
             existing.title = title
             existing.summary = summary
             existing.severity = severity
-            existing.status = "active"
+            existing.status = BGPStatus.ACTIVE.value
             existing.confidence = confidence
             existing.started_at = primary.started_at or existing.started_at or datetime.now(UTC)
             existing.ended_at = None
@@ -313,7 +314,7 @@ async def create_bgp_incidents_for_anomalies(
                 title=title,
                 summary=summary,
                 severity=severity,
-                status="active",
+                status=BGPStatus.ACTIVE.value,
                 confidence=confidence,
                 started_at=primary.started_at or datetime.now(UTC),
                 affected_prefixes=prefixes,

backend/app/services/business_logs.py

@@ -0,0 +1,117 @@
+from __future__ import annotations
+
+import asyncio
+from collections.abc import Mapping
+from typing import Any
+
+from app.core.logging import PlanetLoggerAdapter, sanitize_log_value
+from app.core.request_context import get_request_id
+from app.services.persistent_logs import record_system_log
+
+
+LEVEL_METHODS = {
+    "debug": "debug_event",
+    "info": "info_event",
+    "warning": "warning_event",
+    "error": "error_event",
+}
+
+
+def normalize_business_level(level: str | None) -> str:
+    normalized = str(level or "info").strip().lower()
+    if normalized in {"warn", "warning"}:
+        return "warning"
+    if normalized in {"err", "error", "critical", "fatal"}:
+        return "error"
+    if normalized == "debug":
+        return "debug"
+    return "info"
+
+
+def build_business_context(
+    context: Mapping[str, Any] | None = None,
+    **fields: Any,
+) -> dict[str, Any]:
+    payload = dict(context or {})
+    for key, value in fields.items():
+        if value is not None:
+            payload[key] = value
+    return sanitize_log_value(payload)
+
+
+async def emit_business_log(
+    logger: PlanetLoggerAdapter,
+    *,
+    event: str,
+    message: str,
+    category: str,
+    level: str = "info",
+    source: str = "backend",
+    service: str | None = None,
+    module: str | None = None,
+    request_id: str | None = None,
+    user_id: int | None = None,
+    context: Mapping[str, Any] | None = None,
+) -> None:
+    normalized_level = normalize_business_level(level)
+    safe_context = build_business_context(context)
+    log_method = getattr(logger, LEVEL_METHODS[normalized_level])
+    log_method(message, event=event, context=safe_context)
+    await record_system_log(
+        source=source,
+        level=normalized_level,
+        message=message,
+        service=service,
+        module=module,
+        event=event,
+        request_id=request_id or get_request_id(),
+        user_id=user_id,
+        category=category,
+        context=safe_context,
+    )
+
+
+def emit_business_log_background(
+    logger: PlanetLoggerAdapter,
+    *,
+    event: str,
+    message: str,
+    category: str,
+    level: str = "info",
+    source: str = "backend",
+    service: str | None = None,
+    module: str | None = None,
+    request_id: str | None = None,
+    user_id: int | None = None,
+    context: Mapping[str, Any] | None = None,
+) -> None:
+    normalized_level = normalize_business_level(level)
+    safe_context = build_business_context(context)
+    log_method = getattr(logger, LEVEL_METHODS[normalized_level])
+    log_method(message, event=event, context=safe_context)
+    try:
+        loop = asyncio.get_running_loop()
+    except RuntimeError:
+        return
+    loop.create_task(
+        record_system_log(
+            source=source,
+            level=normalized_level,
+            message=message,
+            service=service,
+            module=module,
+            event=event,
+            request_id=request_id or get_request_id(),
+            user_id=user_id,
+            category=category,
+            context=safe_context,
+        )
+    )
+
+
+def exception_context(exc: BaseException, context: Mapping[str, Any] | None = None) -> dict[str, Any]:
+    return build_business_context(
+        context,
+        error_type=type(exc).__name__,
+        error=str(exc),
+    )

backend/app/services/collectors/__init__.py

@@ -36,6 +36,7 @@ from app.services.collectors.iptoasn import IPtoASNPrefixGeoCollector
 from app.services.collectors.opengeofeed import OpenGeoFeedPrefixGeoCollector
 from app.services.collectors.nro_delegated import NRODelegatedPrefixGeoCollector
 from app.services.collectors.news_live_streams import NewsLiveStreamsCollector
+from app.services.collectors.media_news_archive import MediaNewsArchiveCollector
 from app.services.collectors.aisstream import AISStreamCollector
 from app.services.collectors.vessel_ais import VesselAISCollector
 
@@ -65,6 +66,7 @@ collector_registry.register(IPtoASNPrefixGeoCollector())
 collector_registry.register(OpenGeoFeedPrefixGeoCollector())
 collector_registry.register(NRODelegatedPrefixGeoCollector())
 collector_registry.register(NewsLiveStreamsCollector())
+collector_registry.register(MediaNewsArchiveCollector())
 collector_registry.register(VesselAISCollector())
 collector_registry.register(AISStreamCollector())
 
@@ -100,6 +102,7 @@ __all__ = [
     "OpenGeoFeedPrefixGeoCollector",
     "NRODelegatedPrefixGeoCollector",
     "NewsLiveStreamsCollector",
+    "MediaNewsArchiveCollector",
     "VesselAISCollector",
     "AISStreamCollector",
 ]

backend/app/services/collectors/aisstream.py

@@ -322,6 +322,21 @@ class AISStreamCollector(BaseCollector):
             last_success_at=now if data else None,
             lag_seconds=max((now - latest_observed_at).total_seconds(), 0),
         )
+        if snapshot_id is not None:
+            from app.models.data_snapshot import DataSnapshot
+
+            snapshot = await db.get(DataSnapshot, snapshot_id)
+            if snapshot:
+                snapshot.record_count = records_added
+                snapshot.status = "success"
+                snapshot.completed_at = now
+                snapshot.summary = {
+                    "created": records_added,
+                    "updated": 0,
+                    "unchanged": 0,
+                    "deleted": 0,
+                    "storage": "ais_raw_observations",
+                }
         await db.commit()
         await self.update_progress(records_added, force=True)
         return records_added

backend/app/services/collectors/base.py

@@ -4,15 +4,22 @@ import asyncio
 from abc import ABC, abstractmethod
 from typing import Dict, List, Any, Optional
 from datetime import UTC, datetime
+from time import perf_counter
+from urllib.parse import urlparse
 import httpx
 from sqlalchemy import select, text
 from sqlalchemy.ext.asyncio import AsyncSession
 
 from app.core.collected_data_fields import build_dynamic_metadata, get_record_field
-from app.core.config import settings
 from app.core.countries import normalize_country
+from app.core.enums import JobStatus, SnapshotStatus
+from app.core.logging import get_logger
 from app.core.time import to_iso8601_utc
 from app.core.websocket.broadcaster import broadcaster
+from app.services.business_logs import emit_business_log, exception_context
+
+
+logger = get_logger(__name__, service="collector")
 
 
 class BaseCollector(ABC):
@@ -31,6 +38,7 @@ class BaseCollector(ABC):
         self._datasource_id = 1
         self._resolved_url: Optional[str] = None
         self._last_broadcast_progress: Optional[int] = None
+        self._last_save_summary: dict[str, int] = {}
 
     async def resolve_url(self, db: AsyncSession) -> None:
         from app.core.data_sources import get_data_sources_config
@@ -186,7 +194,7 @@ class BaseCollector(ABC):
 
         result = await db.execute(
             select(DataSnapshot)
-            .where(DataSnapshot.source == self.name, DataSnapshot.is_current == True)
+            .where(DataSnapshot.source == self.name, DataSnapshot.is_current.is_(True))
             .order_by(DataSnapshot.completed_at.desc().nullslast(), DataSnapshot.id.desc())
             .limit(1)
         )
@@ -230,7 +238,7 @@ class BaseCollector(ABC):
             snapshot = await db.get(DataSnapshot, snapshot_id)
             if snapshot:
                 parent_snapshot_id = snapshot.parent_snapshot_id
-                snapshot.status = "cancelled"
+                snapshot.status = SnapshotStatus.CANCELLED.value
                 snapshot.is_current = False
                 snapshot.completed_at = datetime.now(UTC)
                 summary = dict(snapshot.summary or {})
@@ -272,19 +280,39 @@ class BaseCollector(ABC):
         from app.models.data_snapshot import DataSnapshot
 
         start_time = datetime.now(UTC)
+        started_at = perf_counter()
         datasource_id = getattr(self, "_datasource_id", 1)
         snapshot_id: Optional[int] = None
 
         if not collector_registry.is_active(self.name):
+            await self._log_collection_event(
+                "collector.run.skipped_disabled",
+                "Collector skipped because it is disabled",
+                level="info",
+                context={"status": "skipped", "reason": "disabled"},
+            )
             return {"status": "skipped", "reason": "Collector is disabled"}
 
-        task = CollectionTask(
-            datasource_id=datasource_id,
-            status="running",
-            phase="queued",
-            started_at=start_time,
-        )
-        db.add(task)
+        task = self._current_task if isinstance(self._current_task, CollectionTask) else None
+        if task is None:
+            task = CollectionTask(
+                datasource_id=datasource_id,
+                source=self.name,
+                task_type="collect",
+                status="running",
+                phase="queued",
+                started_at=start_time,
+            )
+            db.add(task)
+        else:
+            task.datasource_id = datasource_id
+            task.source = task.source or self.name
+            task.task_type = task.task_type or "collect"
+            task.status = JobStatus.RUNNING.value
+            task.phase = "queued"
+            task.started_at = task.started_at or start_time
+            task.completed_at = None
+            task.error_message = None
         await db.commit()
         task_id = task.id
 
@@ -294,25 +322,78 @@ class BaseCollector(ABC):
 
         await self.resolve_url(db)
         await self._publish_task_update(force=True)
+        await self._log_collection_event(
+            "collector.run.started",
+            "Collector run started",
+            context={"status": "running", "task_id": task_id},
+        )
 
         try:
+            phase_started_at = perf_counter()
             await self.set_phase("fetching", message="正在拉取原始数据")
+            await self._log_collection_event(
+                "collector.phase.fetching.start",
+                "Collector fetch phase started",
+                context={"task_id": task_id, "snapshot_id": snapshot_id},
+            )
             raw_data = await self.fetch()
             task.total_records = len(raw_data)
             await db.commit()
             await self._publish_task_update(force=True)
+            await self._log_collection_event(
+                "collector.phase.fetching.success",
+                "Collector fetch phase completed",
+                context={
+                    "task_id": task_id,
+                    "raw_count": len(raw_data),
+                    "duration_ms": self._duration_ms(phase_started_at),
+                },
+            )
 
             if self.fail_on_empty and not raw_data:
                 raise RuntimeError(f"Collector {self.name} returned no data")
 
+            phase_started_at = perf_counter()
             await self.set_phase("transforming", message="正在转换采集数据")
+            await self._log_collection_event(
+                "collector.phase.transforming.start",
+                "Collector transform phase started",
+                context={"task_id": task_id, "raw_count": len(raw_data)},
+            )
             data = self.transform(raw_data)
+            await self._log_collection_event(
+                "collector.phase.transforming.success",
+                "Collector transform phase completed",
+                context={
+                    "task_id": task_id,
+                    "raw_count": len(raw_data),
+                    "transformed_count": len(data),
+                    "duration_ms": self._duration_ms(phase_started_at),
+                },
+            )
             snapshot_id = await self._create_snapshot(db, task_id, data, start_time)
 
+            phase_started_at = perf_counter()
             await self.set_phase("saving", message="正在保存采集数据")
+            await self._log_collection_event(
+                "collector.phase.saving.start",
+                "Collector save phase started",
+                context={"task_id": task_id, "snapshot_id": snapshot_id, "transformed_count": len(data)},
+            )
             records_count = await self._save_data(db, data, task_id=task_id, snapshot_id=snapshot_id)
+            await self._log_collection_event(
+                "collector.phase.saving.success",
+                "Collector save phase completed",
+                context={
+                    "task_id": task_id,
+                    "snapshot_id": snapshot_id,
+                    "saved_count": records_count,
+                    **self._last_save_summary,
+                    "duration_ms": self._duration_ms(phase_started_at),
+                },
+            )
 
-            task.status = "success"
+            task.status = JobStatus.SUCCESS.value
             task.phase = "completed"
             task.phase_progress = 100.0
             task.phase_message = "采集完成"
@@ -324,6 +405,20 @@ class BaseCollector(ABC):
             task.completed_at = datetime.now(UTC)
             await db.commit()
             await self._publish_task_update(force=True)
+            await self._log_collection_event(
+                "collector.run.completed",
+                "Collector run completed",
+                context={
+                    "status": "success",
+                    "task_id": task_id,
+                    "snapshot_id": snapshot_id,
+                    "raw_count": len(raw_data),
+                    "transformed_count": len(data),
+                    "saved_count": records_count,
+                    **self._last_save_summary,
+                    "duration_ms": self._duration_ms(started_at),
+                },
+            )
 
             return {
                 "status": "success",
@@ -333,7 +428,7 @@ class BaseCollector(ABC):
             }
         except asyncio.CancelledError:
             await db.rollback()
-            task.status = "cancelled"
+            task.status = JobStatus.CANCELLED.value
             task.phase = "cancelled"
             task.phase_message = "采集已取消"
             task.error_message = "Collection cancelled by operator and rolled back"
@@ -347,10 +442,21 @@ class BaseCollector(ABC):
                 )
             await db.commit()
             await self._publish_task_update(force=True)
+            await self._log_collection_event(
+                "collector.run.cancelled",
+                "Collector run cancelled",
+                level="warning",
+                context={
+                    "status": "cancelled",
+                    "task_id": task_id,
+                    "snapshot_id": snapshot_id,
+                    "duration_ms": self._duration_ms(started_at),
+                },
+            )
             raise
         except Exception as e:
             await db.rollback()
-            task.status = "failed"
+            task.status = JobStatus.FAILED.value
             task.phase = "failed"
             task.phase_message = str(e)
             task.error_message = str(e)
@@ -358,11 +464,25 @@ class BaseCollector(ABC):
             if snapshot_id is not None:
                 snapshot = await db.get(DataSnapshot, snapshot_id)
                 if snapshot:
-                    snapshot.status = "failed"
+                    snapshot.status = SnapshotStatus.FAILED.value
                     snapshot.completed_at = datetime.now(UTC)
                     snapshot.summary = {"error": str(e)}
             await db.commit()
             await self._publish_task_update(force=True)
+            await self._log_collection_event(
+                "collector.run.failed",
+                "Collector run failed",
+                level="error",
+                context=exception_context(
+                    e,
+                    {
+                        "status": "failed",
+                        "task_id": task_id,
+                        "snapshot_id": snapshot_id,
+                        "duration_ms": self._duration_ms(started_at),
+                    },
+                ),
+            )
 
             return {
                 "status": "failed",
@@ -383,12 +503,13 @@ class BaseCollector(ABC):
         from app.models.data_snapshot import DataSnapshot
 
         if not data:
+            self._last_save_summary = {"created": 0, "updated": 0, "unchanged": 0, "deleted": 0}
             if snapshot_id is not None:
                 snapshot = await db.get(DataSnapshot, snapshot_id)
                 if snapshot:
                     snapshot.record_count = 0
                     snapshot.summary = {"created": 0, "updated": 0, "unchanged": 0}
-                    snapshot.status = "success"
+                    snapshot.status = SnapshotStatus.SUCCESS.value
                     snapshot.completed_at = datetime.now(UTC)
                     await db.commit()
             return 0
@@ -405,7 +526,7 @@ class BaseCollector(ABC):
             select(CollectedData)
             .where(
                 CollectedData.source == self.name,
-                CollectedData.is_current == True,
+                CollectedData.is_current.is_(True),
             )
             .order_by(CollectedData.entity_key.asc(), CollectedData.collected_at.desc().nullslast(), CollectedData.id.desc())
         )
@@ -521,7 +642,7 @@ class BaseCollector(ABC):
             snapshot = await db.get(DataSnapshot, snapshot_id)
             if snapshot:
                 snapshot.record_count = records_added
-                snapshot.status = "success"
+                snapshot.status = SnapshotStatus.SUCCESS.value
                 snapshot.completed_at = datetime.now(UTC)
                 snapshot.summary = {
                     "created": created_count,
@@ -529,11 +650,51 @@ class BaseCollector(ABC):
                     "unchanged": unchanged_count,
                     "deleted": len(deleted_keys),
                 }
+            self._last_save_summary = {
+                "created": created_count,
+                "updated": updated_count,
+                "unchanged": unchanged_count,
+                "deleted": len(deleted_keys),
+            }
+        else:
+            self._last_save_summary = {
+                "created": created_count,
+                "updated": updated_count,
+                "unchanged": unchanged_count,
+                "deleted": 0,
+            }
 
         await db.commit()
         await self.update_progress(len(data), force=True)
         return records_added
 
+    @staticmethod
+    def _duration_ms(started_at: float) -> int:
+        return int((perf_counter() - started_at) * 1000)
+
+    async def _log_collection_event(
+        self,
+        event: str,
+        message: str,
+        *,
+        level: str = "info",
+        context: Dict[str, Any] | None = None,
+    ) -> None:
+        await emit_business_log(
+            logger,
+            event=event,
+            message=message,
+            category="collector",
+            level=level,
+            service="collector",
+            module=__name__,
+            context={
+                "collector_name": self.name,
+                "datasource_id": getattr(self, "_datasource_id", None),
+                **(context or {}),
+            },
+        )
+
     async def save(self, db: AsyncSession, data: List[Dict[str, Any]]) -> int:
         """Save data to database (legacy method, use _save_data instead)"""
         return await self._save_data(db, data)
@@ -546,10 +707,65 @@ class HTTPCollector(BaseCollector):
     headers: Dict[str, str] = {}
 
     async def fetch(self) -> List[Dict[str, Any]]:
+        started_at = perf_counter()
+        request_host = urlparse(self.base_url).netloc
+        await emit_business_log(
+            logger,
+            event="collector.http.fetch.start",
+            message="Collector HTTP request started",
+            category="collector",
+            service="collector",
+            module=__name__,
+            context={
+                "collector_name": self.name,
+                "datasource_id": getattr(self, "_datasource_id", None),
+                "url_host": request_host,
+            },
+        )
         async with httpx.AsyncClient(timeout=60.0) as client:
-            response = await client.get(self.base_url, headers=self.headers)
-            response.raise_for_status()
-            return self.parse_response(response.json())
+            try:
+                response = await client.get(self.base_url, headers=self.headers)
+                response.raise_for_status()
+                payload = response.json()
+                parsed = self.parse_response(payload)
+                await emit_business_log(
+                    logger,
+                    event="collector.http.fetch.success",
+                    message="Collector HTTP request completed",
+                    category="collector",
+                    service="collector",
+                    module=__name__,
+                    context={
+                        "collector_name": self.name,
+                        "datasource_id": getattr(self, "_datasource_id", None),
+                        "url_host": request_host,
+                        "status_code": response.status_code,
+                        "response_bytes": len(response.content or b""),
+                        "parsed_count": len(parsed),
+                        "duration_ms": BaseCollector._duration_ms(started_at),
+                    },
+                )
+                return parsed
+            except Exception as exc:
+                await emit_business_log(
+                    logger,
+                    event="collector.http.fetch.failed",
+                    message="Collector HTTP request failed",
+                    category="collector",
+                    level="error",
+                    service="collector",
+                    module=__name__,
+                    context=exception_context(
+                        exc,
+                        {
+                            "collector_name": self.name,
+                            "datasource_id": getattr(self, "_datasource_id", None),
+                            "url_host": request_host,
+                            "duration_ms": BaseCollector._duration_ms(started_at),
+                        },
+                    ),
+                )
+                raise
 
     @abstractmethod
     def parse_response(self, response: Dict[str, Any]) -> List[Dict[str, Any]]:

backend/app/services/collectors/bgp_common.py

@@ -19,6 +19,7 @@ from app.services.bgp_collector_locations import (
 )
 from app.services.bgp_event_locations import resolve_bgp_event_geo_dict
 from app.services.bgp_incidents import create_bgp_incidents_for_anomalies
+from app.services.earth_layer_cache import invalidate_earth_layer_cache_for_source
 from app.services.bgp_detectors import (
     detect_mass_withdrawal_anomalies,
     detect_more_specific_burst_anomalies,
@@ -223,6 +224,8 @@ async def save_bgp_observations_for_batch(
 
     if created:
         await db.commit()
+        for source in {"ris_live_bgp", "bgpstream_bgp"}:
+            invalidate_earth_layer_cache_for_source(source)
 
     return created
 

backend/app/services/collectors/celestrak.py

@@ -1,15 +1,45 @@
-"""CelesTrak TLE Collector
+"""CelesTrak TLE Collector.
 
-Collects satellite TLE (Two-Line Element) data from CelesTrak.org.
-Free, no authentication required.
+Collects the full active satellite GP element set from CelesTrak.
 """
 
+import asyncio
 import json
-from typing import Dict, Any, List
+from pathlib import Path
+from time import perf_counter
+from typing import Any, Dict, List
+from urllib.parse import urlencode, urlparse
+
 import httpx
 
+from app.core.logging import get_logger
 from app.core.satellite_tle import build_tle_lines_from_elements
+from app.services.business_logs import emit_business_log, exception_context
 from app.services.collectors.base import BaseCollector
+from app.services.collectors.downloads import DownloadHTTPStatusError, ResumableFileDownloader
+
+
+logger = get_logger(__name__, service="collector")
+ACTIVE_GROUP = "active"
+FALLBACK_GROUPS = (
+    "starlink",
+    "gps-ops",
+    "galileo",
+    "glo-ops",
+    "beidou",
+    "geo",
+    "iridium-next",
+    "stations",
+    "visual",
+    "weather",
+    "science",
+    "cubesat",
+    "amateur",
+    "last-30-days",
+)
+FETCH_RETRY_ATTEMPTS = 3
+FETCH_RETRY_BASE_DELAY_SECONDS = 0.8
+CELESTRAK_NOT_UPDATED_MARKER = "GP data has not updated since your last successful"
 
 
 class CelesTrakTLECollector(BaseCollector):
@@ -18,55 +48,360 @@ class CelesTrakTLECollector(BaseCollector):
     module = "L3"
     frequency_hours = 24
     data_type = "satellite_tle"
+    _downloader = ResumableFileDownloader(
+        cache_namespace="celestrak",
+        default_accept="application/json",
+    )
 
     @property
     def base_url(self) -> str:
         return self._resolved_url or ""
 
+    def _active_url(self) -> str:
+        return self._group_url(ACTIVE_GROUP)
+
+    def _group_url(self, group: str) -> str:
+        if not self.base_url:
+            raise RuntimeError("CelesTrak base URL is not configured")
+        return f"{self.base_url}?{urlencode({'GROUP': group, 'FORMAT': 'json'})}"
+
     async def fetch(self) -> List[Dict[str, Any]]:
-        satellite_groups = [
-            "starlink",
-            "gps-ops",
-            "galileo",
-            "glonass",
-            "beidou",
-            "leo",
-            "geo",
-            "iridium-next",
-        ]
+        url = self._active_url()
+        last_error: Exception | None = None
 
-        all_satellites = []
-
-        async with httpx.AsyncClient(timeout=120.0) as client:
-            for group in satellite_groups:
+        async with httpx.AsyncClient(timeout=180.0, follow_redirects=True) as client:
+            for attempt in range(1, FETCH_RETRY_ATTEMPTS + 1):
+                started_at = perf_counter()
                 try:
-                    url = f"{self.base_url}?GROUP={group}&FORMAT=json"
-                    response = await client.get(url)
+                    await emit_business_log(
+                        logger,
+                        event="collector.celestrak.download.start",
+                        message="CelesTrak active satellite download started",
+                        category="collector",
+                        service="collector",
+                        module=__name__,
+                        context={
+                            "collector_name": self.name,
+                            "datasource_id": getattr(self, "_datasource_id", None),
+                            "group": ACTIVE_GROUP,
+                            "attempt": attempt,
+                            "url_host": urlparse(url).netloc,
+                        },
+                    )
+                    body_path = await self._downloader.download_file(
+                        client,
+                        url,
+                        extension=".json",
+                        accept="application/json",
+                        progress_callback=self._report_download_progress,
+                        validate_existing=self._validate_json_file,
+                    )
+                    data = await self._load_downloaded_payload(body_path, url)
+                    await emit_business_log(
+                        logger,
+                        event="collector.celestrak.download.success",
+                        message="CelesTrak active satellite download completed",
+                        category="collector",
+                        service="collector",
+                        module=__name__,
+                        context={
+                            "collector_name": self.name,
+                            "datasource_id": getattr(self, "_datasource_id", None),
+                            "group": ACTIVE_GROUP,
+                            "attempt": attempt,
+                            "record_count": len(data),
+                            "duration_ms": self._duration_ms(started_at),
+                        },
+                    )
+                    return data
+                except DownloadHTTPStatusError as exc:
+                    if self._is_not_updated_response(exc):
+                        cached_path = self._downloader.get_cached_file(
+                            url,
+                            ".json",
+                            validate_existing=self._validate_json_file,
+                        )
+                        if cached_path is not None:
+                            data = await self._load_downloaded_payload(cached_path, url)
+                            await emit_business_log(
+                                logger,
+                                event="collector.celestrak.download.cached_not_updated",
+                                message="CelesTrak active satellite data has not changed; using cached download",
+                                category="collector",
+                                level="warning",
+                                service="collector",
+                                module=__name__,
+                                context={
+                                    "collector_name": self.name,
+                                    "datasource_id": getattr(self, "_datasource_id", None),
+                                    "group": ACTIVE_GROUP,
+                                    "attempt": attempt,
+                                    "record_count": len(data),
+                                    "duration_ms": self._duration_ms(started_at),
+                                },
+                            )
+                            return data
+                        await emit_business_log(
+                            logger,
+                            event="collector.celestrak.download.not_updated_no_cache",
+                            message="CelesTrak active satellite data has not changed; trying fallback groups",
+                            category="collector",
+                            level="warning",
+                            service="collector",
+                            module=__name__,
+                            context=exception_context(
+                                exc,
+                                {
+                                    "collector_name": self.name,
+                                    "datasource_id": getattr(self, "_datasource_id", None),
+                                    "group": ACTIVE_GROUP,
+                                    "attempt": attempt,
+                                    "duration_ms": self._duration_ms(started_at),
+                                },
+                            ),
+                        )
+                        return await self._fetch_fallback_groups(client, active_error=exc)
+                    raise
+                except Exception as exc:
+                    last_error = exc
+                    is_final_attempt = attempt >= FETCH_RETRY_ATTEMPTS
+                    await emit_business_log(
+                        logger,
+                        event=(
+                            "collector.celestrak.download.failed"
+                            if is_final_attempt
+                            else "collector.celestrak.download.retry"
+                        ),
+                        message=(
+                            "CelesTrak active satellite download failed"
+                            if is_final_attempt
+                            else "CelesTrak active satellite download will retry"
+                        ),
+                        category="collector",
+                        level="error" if is_final_attempt else "warning",
+                        service="collector",
+                        module=__name__,
+                        context=exception_context(
+                            exc,
+                            {
+                                "collector_name": self.name,
+                                "datasource_id": getattr(self, "_datasource_id", None),
+                                "group": ACTIVE_GROUP,
+                                "attempt": attempt,
+                                "duration_ms": self._duration_ms(started_at),
+                            },
+                        ),
+                    )
+                    if not is_final_attempt:
+                        await asyncio.sleep(FETCH_RETRY_BASE_DELAY_SECONDS * attempt)
 
-                    if response.status_code == 200:
-                        data = response.json()
-                        if isinstance(data, list):
-                            for item in data:
-                                if isinstance(item, dict):
-                                    item["_celestrak_group"] = group
-                            all_satellites.extend(data)
-                            print(f"CelesTrak: Fetched {len(data)} satellites from group '{group}'")
-                except Exception as e:
-                    print(f"CelesTrak: Error fetching group '{group}': {e}")
+        raise RuntimeError(f"CelesTrak active satellite download failed after retries: {last_error}")
 
-        if not all_satellites:
-            return self._get_sample_data()
+    async def _fetch_fallback_groups(
+        self,
+        client: httpx.AsyncClient,
+        *,
+        active_error: DownloadHTTPStatusError,
+    ) -> List[Dict[str, Any]]:
+        started_at = perf_counter()
+        records_by_norad: dict[str, Dict[str, Any]] = {}
+        group_counts: dict[str, int] = {}
 
-        print(f"CelesTrak: Total satellites fetched: {len(all_satellites)}")
+        await emit_business_log(
+            logger,
+            event="collector.celestrak.fallback_groups.start",
+            message="CelesTrak fallback group download started",
+            category="collector",
+            level="warning",
+            service="collector",
+            module=__name__,
+            context={
+                "collector_name": self.name,
+                "datasource_id": getattr(self, "_datasource_id", None),
+                "groups": list(FALLBACK_GROUPS),
+                "reason": "active_not_updated_without_cache",
+            },
+        )
 
-        # Return raw data - base.run() will call transform()
-        return all_satellites
+        try:
+            for group in FALLBACK_GROUPS:
+                group_url = self._group_url(group)
+                cached_path = self._downloader.get_cached_file(
+                    group_url,
+                    ".json",
+                    validate_existing=self._validate_json_file,
+                )
+                if cached_path is not None:
+                    body_path = cached_path
+                else:
+                    try:
+                        body_path = await self._downloader.download_file(
+                            client,
+                            group_url,
+                            extension=".json",
+                            accept="application/json",
+                            validate_existing=self._validate_json_file,
+                        )
+                    except DownloadHTTPStatusError as exc:
+                        if not self._is_not_updated_response(exc):
+                            raise RuntimeError(f"CelesTrak fallback group '{group}' download failed: {exc}") from exc
+                        raise RuntimeError(
+                            f"CelesTrak fallback group '{group}' has not updated and no local cached copy is available"
+                        ) from exc
+
+                group_records = await self._load_downloaded_payload(
+                    body_path,
+                    group_url,
+                    query_group=group,
+                    constellation_group=group,
+                )
+                group_counts[group] = len(group_records)
+                for item in group_records:
+                    norad_cat_id = item.get("NORAD_CAT_ID")
+                    if norad_cat_id is None:
+                        continue
+                    records_by_norad.setdefault(str(norad_cat_id), item)
+        except Exception as exc:
+            await emit_business_log(
+                logger,
+                event="collector.celestrak.fallback_groups.failed",
+                message="CelesTrak fallback group download failed",
+                category="collector",
+                level="error",
+                service="collector",
+                module=__name__,
+                context=exception_context(
+                    exc,
+                    {
+                        "collector_name": self.name,
+                        "datasource_id": getattr(self, "_datasource_id", None),
+                        "groups": list(FALLBACK_GROUPS),
+                        "completed_groups": list(group_counts),
+                        "duration_ms": self._duration_ms(started_at),
+                    },
+                ),
+            )
+            raise RuntimeError(
+                "CelesTrak active data has not updated since this network's last successful download, "
+                "no active cache is available, and fallback group mode failed. Wait until CelesTrak "
+                "publishes the next GP update, restore the Planet download cache, or use Space-Track."
+            ) from active_error
+
+        records = list(records_by_norad.values())
+        if not records:
+            raise RuntimeError("CelesTrak fallback group mode produced no satellite records")
+
+        await emit_business_log(
+            logger,
+            event="collector.celestrak.fallback_groups.success",
+            message="CelesTrak fallback group download completed",
+            category="collector",
+            level="warning",
+            service="collector",
+            module=__name__,
+            context={
+                "collector_name": self.name,
+                "datasource_id": getattr(self, "_datasource_id", None),
+                "groups": list(FALLBACK_GROUPS),
+                "group_counts": group_counts,
+                "record_count": len(records),
+                "duration_ms": self._duration_ms(started_at),
+            },
+        )
+        return records
+
+    async def _load_downloaded_payload(
+        self,
+        body_path: Path,
+        url: str,
+        *,
+        query_group: str = ACTIVE_GROUP,
+        constellation_group: str | None = None,
+    ) -> List[Dict[str, Any]]:
+        try:
+            data = self._load_active_payload(body_path)
+        except RuntimeError as exc:
+            await self._log_parse_failure(exc)
+            raise
+        for item in data:
+            item["_celestrak_query_group"] = query_group
+            item["_celestrak_source_url"] = url
+            if constellation_group:
+                item["_celestrak_group"] = constellation_group
+        return data
+
+    @staticmethod
+    def _is_not_updated_response(exc: DownloadHTTPStatusError) -> bool:
+        return exc.status_code == 403 and CELESTRAK_NOT_UPDATED_MARKER in exc.body
+
+    @staticmethod
+    def _duration_ms(started_at: float) -> int:
+        return int((perf_counter() - started_at) * 1000)
+
+    async def _report_download_progress(self, downloaded: int, total: int | None) -> None:
+        if total and total > 0:
+            await self.update_phase_progress(
+                current=min(downloaded, total),
+                total=total,
+                unit="bytes",
+                message=f"正在下载 CelesTrak active 卫星数据 {downloaded}/{total} bytes",
+                commit=True,
+            )
+
+    @staticmethod
+    def _validate_json_file(path: Path) -> bool:
+        try:
+            data = json.loads(path.read_text(encoding="utf-8"))
+        except (OSError, json.JSONDecodeError, UnicodeDecodeError):
+            return False
+        return isinstance(data, list)
+
+    async def _log_parse_failure(self, exc: Exception) -> None:
+        await emit_business_log(
+            logger,
+            event="collector.celestrak.parse.failed",
+            message="CelesTrak active satellite JSON parsing failed",
+            category="collector",
+            level="error",
+            service="collector",
+            module=__name__,
+            context=exception_context(
+                exc,
+                {
+                    "collector_name": self.name,
+                    "datasource_id": getattr(self, "_datasource_id", None),
+                    "group": ACTIVE_GROUP,
+                },
+            ),
+        )
+
+    def _load_active_payload(self, path: Path) -> List[Dict[str, Any]]:
+        try:
+            raw = json.loads(path.read_text(encoding="utf-8"))
+        except (OSError, json.JSONDecodeError, UnicodeDecodeError) as exc:
+            raise RuntimeError(f"CelesTrak active payload is not valid JSON: {exc}") from exc
+        if not isinstance(raw, list):
+            raise RuntimeError("CelesTrak active payload is not a JSON array")
+
+        records: List[Dict[str, Any]] = []
+        invalid_count = 0
+        for item in raw:
+            if isinstance(item, dict) and item.get("NORAD_CAT_ID") is not None:
+                records.append(item)
+            else:
+                invalid_count += 1
+        if invalid_count:
+            raise RuntimeError(f"CelesTrak active payload contains {invalid_count} invalid record(s)")
+        if not records:
+            raise RuntimeError("CelesTrak active payload contains no satellite records")
+        return records
 
     def transform(self, raw_data: List[Dict[str, Any]]) -> List[Dict[str, Any]]:
         transformed = []
         for item in raw_data:
+            norad_cat_id = item.get("NORAD_CAT_ID")
             tle_line1, tle_line2 = build_tle_lines_from_elements(
-                norad_cat_id=item.get("NORAD_CAT_ID"),
+                norad_cat_id=norad_cat_id,
                 epoch=item.get("EPOCH"),
                 inclination=item.get("INCLINATION"),
                 raan=item.get("RA_OF_ASC_NODE"),
@@ -75,14 +410,18 @@ class CelesTrakTLECollector(BaseCollector):
                 mean_anomaly=item.get("MEAN_ANOMALY"),
                 mean_motion=item.get("MEAN_MOTION"),
             )
+            constellation_group = self._infer_constellation_group(item)
 
             transformed.append(
                 {
+                    "source_id": str(norad_cat_id),
                     "name": item.get("OBJECT_NAME", "Unknown"),
                     "reference_date": item.get("EPOCH", ""),
                     "metadata": {
-                        "constellation_group": item.get("_celestrak_group"),
-                        "norad_cat_id": item.get("NORAD_CAT_ID"),
+                        "constellation_group": constellation_group,
+                        "celestrak_query_group": item.get("_celestrak_query_group") or ACTIVE_GROUP,
+                        "celestrak_source_url": item.get("_celestrak_source_url"),
+                        "norad_cat_id": norad_cat_id,
                         "international_designator": item.get("OBJECT_ID"),
                         "epoch": item.get("EPOCH"),
                         "mean_motion": item.get("MEAN_MOTION"),
@@ -105,6 +444,19 @@ class CelesTrakTLECollector(BaseCollector):
             )
         return transformed
 
+    @staticmethod
+    def _infer_constellation_group(item: Dict[str, Any]) -> str | None:
+        explicit_group = str(item.get("_celestrak_group") or "").strip().lower()
+        if explicit_group and explicit_group != ACTIVE_GROUP:
+            return explicit_group
+
+        name = str(item.get("OBJECT_NAME") or "").strip().upper()
+        if name.startswith("STARLINK"):
+            return "starlink"
+        if name.startswith("IRIDIUM"):
+            return "iridium-next"
+        return None
+
     def _get_sample_data(self) -> List[Dict[str, Any]]:
         return [
             {

backend/app/services/collectors/downloads.py

@@ -4,7 +4,7 @@ from __future__ import annotations
 
 import hashlib
 import json
-import tempfile
+import os
 import time
 from datetime import UTC, datetime
 from pathlib import Path
@@ -17,6 +17,31 @@ ProgressCallback = Callable[[int, int | None], Awaitable[None]]
 ValidateCallback = Callable[[Path], bool]
 
 
+class DownloadHTTPStatusError(RuntimeError):
+    """HTTP status error that keeps the upstream response body for caller-specific handling."""
+
+    def __init__(self, *, url: str, status_code: int, body: str) -> None:
+        self.url = url
+        self.status_code = status_code
+        self.body = body
+        preview = body.strip().replace("\r", " ").replace("\n", " ")[:240]
+        suffix = f": {preview}" if preview else ""
+        super().__init__(f"HTTP {status_code} while downloading {url}{suffix}")
+
+
+def default_download_cache_root() -> Path:
+    configured = os.getenv("PLANET_DOWNLOAD_CACHE_DIR")
+    if configured:
+        return Path(configured).expanduser()
+    planet_cache = os.getenv("PLANET_CACHE_DIR")
+    if planet_cache:
+        return Path(planet_cache).expanduser() / "downloads"
+    xdg_cache = os.getenv("XDG_CACHE_HOME")
+    if xdg_cache:
+        return Path(xdg_cache).expanduser() / "planet" / "downloads"
+    return Path.home() / ".cache" / "planet" / "downloads"
+
+
 class ResumableFileDownloader:
     """Download files with cache validators and byte-range resume support."""
 
@@ -26,8 +51,9 @@ class ResumableFileDownloader:
         cache_namespace: str,
         user_agent: str = "Planet-Intelligence-System/1.0 (Python/collector)",
         default_accept: str = "*/*",
+        cache_root: Path | None = None,
     ) -> None:
-        self._cache_dir = Path(tempfile.gettempdir()) / "planet-download-cache" / cache_namespace
+        self._cache_dir = (cache_root or default_download_cache_root()) / cache_namespace
         self._user_agent = user_agent
         self._default_accept = default_accept
 
@@ -43,6 +69,25 @@ class ResumableFileDownloader:
         meta_path = self._cache_dir / f"{key}.meta.json"
         return final_path, part_path, meta_path
 
+    def cached_file_path(self, url: str, extension: str) -> Path:
+        final_path, _, _ = self._cache_paths(url, extension)
+        return final_path
+
+    def get_cached_file(
+        self,
+        url: str,
+        extension: str,
+        *,
+        validate_existing: ValidateCallback | None = None,
+    ) -> Path | None:
+        final_path = self.cached_file_path(url, extension)
+        if not final_path.exists():
+            return None
+        if validate_existing and not validate_existing(final_path):
+            final_path.unlink(missing_ok=True)
+            return None
+        return final_path
+
     @staticmethod
     def _load_meta(meta_path: Path) -> dict[str, Any]:
         if not meta_path.exists():
@@ -140,7 +185,9 @@ class ResumableFileDownloader:
                 if progress_callback and expected_size and expected_size > 0:
                     await progress_callback(expected_size, expected_size)
                 return final_path
-            response.raise_for_status()
+            if response.status_code >= 400:
+                body = (await response.aread()).decode("utf-8", errors="replace")
+                raise DownloadHTTPStatusError(url=url, status_code=response.status_code, body=body)
 
             if response.status_code == 206 and resume_from > 0:
                 mode = "ab"

backend/app/services/collectors/media_news_archive.py

@@ -0,0 +1,57 @@
+from __future__ import annotations
+
+from typing import Any
+
+from app.services.collectors.base import BaseCollector
+from app.services.earth_news_store import list_all_earth_news_records
+
+
+class MediaNewsArchiveCollector(BaseCollector):
+    name = "media_news_archive"
+    priority = "P2"
+    module = "L4"
+    frequency_hours = 12
+    data_type = "news_item"
+    fail_on_empty = False
+
+    async def fetch(self) -> list[dict[str, Any]]:
+        if not self._db_session:
+            return []
+
+        records = await list_all_earth_news_records(self._db_session)
+        items: list[dict[str, Any]] = []
+        for record in records:
+            location_meta = dict(record.location_meta or {})
+            target = location_meta.get("target") if isinstance(location_meta.get("target"), dict) else {}
+            country = target.get("country")
+            city = target.get("city")
+            items.append(
+                {
+                    "id": record.id,
+                    "source_id": record.id,
+                    "name": record.title,
+                    "title": record.title,
+                    "description": record.summary,
+                    "country": country,
+                    "city": city,
+                    "latitude": record.latitude,
+                    "longitude": record.longitude,
+                    "reference_date": record.published_at,
+                    "metadata": {
+                        "url": record.url,
+                        "source": record.source,
+                        "feed_name": record.feed_name,
+                        "region": record.region,
+                        "homepage_url": record.homepage_url,
+                        "published_at": record.published_at.isoformat() if record.published_at else None,
+                        "location_label": record.location_label,
+                        "location_source": record.location_source,
+                        "verified": record.verified,
+                        "location_meta": location_meta,
+                        "first_seen_at": record.first_seen_at.isoformat() if record.first_seen_at else None,
+                        "last_seen_at": record.last_seen_at.isoformat() if record.last_seen_at else None,
+                        "resolved_at": record.resolved_at.isoformat() if record.resolved_at else None,
+                    },
+                }
+            )
+        return items

backend/app/services/collectors/news_live_streams.py

@@ -35,7 +35,7 @@ class NewsLiveStreamsCollector(BaseCollector):
     DEFAULT_IPTV_ORG_LOGOS_URL = "https://iptv-org.github.io/api/logos.json"
     DEFAULT_IPTV_ORG_NEWS_CATEGORIES = ("news", "business", "weather")
     DEFAULT_IPTV_ORG_EXCLUDE_CATEGORIES = ("music", "sports", "kids", "entertainment")
-    DEFAULT_IPTV_ORG_MAX_SOURCES = 120
+    DEFAULT_IPTV_ORG_MAX_SOURCES = 0  # Zero keeps the complete matching channel catalog.
 
     async def fetch(self) -> list[dict[str, Any]]:
         request_url = (self._resolved_url or "").strip()
@@ -445,7 +445,7 @@ class NewsLiveStreamsCollector(BaseCollector):
                     "reference_date": datetime.now(UTC).isoformat(),
                 }
             )
-            if len(normalized) >= max_sources:
+            if max_sources > 0 and len(normalized) >= max_sources:
                 break
 
         return normalized

backend/app/services/collectors/peeringdb.py

@@ -11,17 +11,20 @@ To get higher limits, set PEERINGDB_API_KEY environment variable.
 """
 
 import asyncio
-import os
-from typing import Dict, Any, List
 from datetime import UTC, datetime
+import os
+from typing import Any, Dict, List
+from urllib.parse import urlencode
 
 import httpx
-from urllib.parse import urlencode
+
+from app.core.logging import get_logger
 from app.services.collectors.base import HTTPCollector
 
 
 # PeeringDB API key - read from environment variable
 PEERINGDB_API_KEY = os.environ.get("PEERINGDB_API_KEY", "")
+logger = get_logger(__name__, service="collector")
 
 
 class PeeringDBIXPCollector(HTTPCollector):
@@ -39,6 +42,7 @@ class PeeringDBIXPCollector(HTTPCollector):
             "User-Agent": "Planet-Intelligence-System/1.0 (Python/collector)",
             "Accept": "application/json",
         }
+
     @property
     def request_url(self) -> str:
         base = self._resolved_url or self.base_url
@@ -61,7 +65,11 @@ class PeeringDBIXPCollector(HTTPCollector):
                     if response.status_code == 429:
                         # Rate limited - wait and retry with exponential backoff
                         delay = base_delay * (2**attempt)
-                        print(f"PeeringDB rate limited, waiting {delay}s before retry...")
+                        logger.warning_event(
+                            "PeeringDB rate limited; retrying after delay",
+                            event="collector.peeringdb.rate_limited",
+                            context={"delay_seconds": delay, "attempt": attempt + 1},
+                        )
                         await asyncio.sleep(delay)
                         last_error = "Rate limited"
                         continue
@@ -72,13 +80,21 @@ class PeeringDBIXPCollector(HTTPCollector):
             except httpx.HTTPStatusError as e:
                 if e.response.status_code == 429:
                     delay = base_delay * (2**attempt)
-                    print(f"PeeringDB rate limited, waiting {delay}s before retry...")
+                    logger.warning_event(
+                        "PeeringDB rate limited; retrying after delay",
+                        event="collector.peeringdb.rate_limited",
+                        context={"delay_seconds": delay, "attempt": attempt + 1},
+                    )
                     await asyncio.sleep(delay)
                     last_error = "Rate limited"
                     continue
                 raise
 
-        print(f"Warning: PeeringDB collection failed after {max_retries} retries: {last_error}")
+        logger.warning_event(
+            "PeeringDB collection failed after retries",
+            event="collector.peeringdb.retries_exhausted",
+            context={"max_retries": max_retries, "last_error": last_error},
+        )
         return {}
 
     async def fetch(self) -> List[Dict[str, Any]]:
@@ -146,6 +162,7 @@ class PeeringDBNetworkCollector(HTTPCollector):
             "User-Agent": "Planet-Intelligence-System/1.0 (Python/collector)",
             "Accept": "application/json",
         }
+
     @property
     def request_url(self) -> str:
         base = self._resolved_url or self.base_url
@@ -167,7 +184,11 @@ class PeeringDBNetworkCollector(HTTPCollector):
 
                     if response.status_code == 429:
                         delay = base_delay * (2**attempt)
-                        print(f"PeeringDB rate limited, waiting {delay}s before retry...")
+                        logger.warning_event(
+                            "PeeringDB rate limited; retrying after delay",
+                            event="collector.peeringdb.rate_limited",
+                            context={"delay_seconds": delay, "attempt": attempt + 1},
+                        )
                         await asyncio.sleep(delay)
                         last_error = "Rate limited"
                         continue
@@ -178,13 +199,21 @@ class PeeringDBNetworkCollector(HTTPCollector):
             except httpx.HTTPStatusError as e:
                 if e.response.status_code == 429:
                     delay = base_delay * (2**attempt)
-                    print(f"PeeringDB rate limited, waiting {delay}s before retry...")
+                    logger.warning_event(
+                        "PeeringDB rate limited; retrying after delay",
+                        event="collector.peeringdb.rate_limited",
+                        context={"delay_seconds": delay, "attempt": attempt + 1},
+                    )
                     await asyncio.sleep(delay)
                     last_error = "Rate limited"
                     continue
                 raise
 
-        print(f"Warning: PeeringDB collection failed after {max_retries} retries: {last_error}")
+        logger.warning_event(
+            "PeeringDB collection failed after retries",
+            event="collector.peeringdb.retries_exhausted",
+            context={"max_retries": max_retries, "last_error": last_error},
+        )
         return {}
 
     async def fetch(self) -> List[Dict[str, Any]]:
@@ -254,6 +283,7 @@ class PeeringDBFacilityCollector(HTTPCollector):
             "User-Agent": "Planet-Intelligence-System/1.0 (Python/collector)",
             "Accept": "application/json",
         }
+
     @property
     def request_url(self) -> str:
         base = self._resolved_url or self.base_url
@@ -275,7 +305,11 @@ class PeeringDBFacilityCollector(HTTPCollector):
 
                     if response.status_code == 429:
                         delay = base_delay * (2**attempt)
-                        print(f"PeeringDB rate limited, waiting {delay}s before retry...")
+                        logger.warning_event(
+                            "PeeringDB rate limited; retrying after delay",
+                            event="collector.peeringdb.rate_limited",
+                            context={"delay_seconds": delay, "attempt": attempt + 1},
+                        )
                         await asyncio.sleep(delay)
                         last_error = "Rate limited"
                         continue
@@ -286,13 +320,21 @@ class PeeringDBFacilityCollector(HTTPCollector):
             except httpx.HTTPStatusError as e:
                 if e.response.status_code == 429:
                     delay = base_delay * (2**attempt)
-                    print(f"PeeringDB rate limited, waiting {delay}s before retry...")
+                    logger.warning_event(
+                        "PeeringDB rate limited; retrying after delay",
+                        event="collector.peeringdb.rate_limited",
+                        context={"delay_seconds": delay, "attempt": attempt + 1},
+                    )
                     await asyncio.sleep(delay)
                     last_error = "Rate limited"
                     continue
                 raise
 
-        print(f"Warning: PeeringDB collection failed after {max_retries} retries: {last_error}")
+        logger.warning_event(
+            "PeeringDB collection failed after retries",
+            event="collector.peeringdb.retries_exhausted",
+            context={"max_retries": max_retries, "last_error": last_error},
+        )
         return {}
 
     async def fetch(self) -> List[Dict[str, Any]]:

backend/app/services/collectors/spacetrack.py

@@ -1,17 +1,21 @@
-"""Space-Track TLE Collector
+"""Space-Track TLE Collector.
 
 Collects satellite TLE (Two-Line Element) data from Space-Track.org.
 API documentation: https://www.space-track.org/documentation
 """
 
-import json
-from typing import Dict, Any, List
-import httpx
+from typing import Any, Dict, List
 from urllib.parse import urlparse
 
-from app.services.collectors.base import BaseCollector
+import httpx
+
 from app.core.data_sources import get_data_sources_config
+from app.core.logging import get_logger
 from app.core.satellite_tle import build_tle_lines_from_elements
+from app.services.collectors.base import BaseCollector
+
+
+logger = get_logger(__name__, service="collector")
 
 
 class SpaceTrackTLECollector(BaseCollector):
@@ -53,10 +57,16 @@ class SpaceTrackTLECollector(BaseCollector):
         password = settings.SPACETRACK_PASSWORD
 
         if not username or not password:
-            print("SPACETRACK: No credentials configured, using sample data")
+            logger.warning_event(
+                "Space-Track credentials are not configured; using sample data",
+                event="collector.spacetrack.credentials_missing",
+            )
             return self._get_sample_data()
 
-        print(f"SPACETRACK: Attempting to fetch TLE data with username: {username}")
+        logger.info_event(
+            "Space-Track TLE fetch started",
+            event="collector.spacetrack.fetch.start",
+        )
 
         try:
             async with httpx.AsyncClient(
@@ -78,11 +88,17 @@ class SpaceTrackTLECollector(BaseCollector):
                         "password": password,
                     },
                 )
-                print(f"SPACETRACK: Login response status: {login_response.status_code}")
-                print(f"SPACETRACK: Login response URL: {login_response.url}")
+                logger.info_event(
+                    "Space-Track login response received",
+                    event="collector.spacetrack.login.response",
+                    context={"status_code": login_response.status_code},
+                )
 
                 if login_response.status_code == 403:
-                    print("SPACETRACK: Trying alternate login method...")
+                    logger.warning_event(
+                        "Space-Track login returned forbidden; trying alternate method",
+                        event="collector.spacetrack.login.forbidden",
+                    )
 
                     async with httpx.AsyncClient(
                         timeout=120.0,
@@ -90,11 +106,6 @@ class SpaceTrackTLECollector(BaseCollector):
                     ) as alt_client:
                         await alt_client.get(f"{self.site_root}/")
 
-                        form_data = {
-                            "username": username,
-                            "password": password,
-                            "query": "class/gp/NORAD_CAT_ID/25544/format/json",
-                        }
                         alt_login = await alt_client.post(
                             self.login_url,
                             data={
@@ -102,77 +113,59 @@ class SpaceTrackTLECollector(BaseCollector):
                                 "password": password,
                             },
                         )
-                        print(f"SPACETRACK: Alt login status: {alt_login.status_code}")
+                        logger.info_event(
+                            "Space-Track alternate login response received",
+                            event="collector.spacetrack.alt_login.response",
+                            context={"status_code": alt_login.status_code},
+                        )
 
                         if alt_login.status_code == 200:
                             tle_response = await alt_client.get(self.probe_url)
                             if tle_response.status_code == 200:
                                 data = tle_response.json()
-                                print(f"SPACETRACK: Received {len(data)} records via alt method")
+                                logger.info_event(
+                                    "Space-Track alternate query completed",
+                                    event="collector.spacetrack.alt_query.completed",
+                                    context={"record_count": len(data)},
+                                )
                                 return data
 
                 if login_response.status_code != 200:
-                    print(f"SPACETRACK: Login failed, using sample data")
+                    logger.warning_event(
+                        "Space-Track login failed; using sample data",
+                        event="collector.spacetrack.login.failed",
+                        context={"status_code": login_response.status_code},
+                    )
                     return self._get_sample_data()
 
                 tle_response = await client.get(self.probe_url)
-                print(f"SPACETRACK: TLE query status: {tle_response.status_code}")
-
-                if tle_response.status_code != 200:
-                    print(f"SPACETRACK: Query failed, using sample data")
-                    return self._get_sample_data()
-
-                data = tle_response.json()
-                print(f"SPACETRACK: Received {len(data)} records")
-                return data
-        except Exception as e:
-            print(f"SPACETRACK: Error - {e}, using sample data")
-            return self._get_sample_data()
-
-        print(f"SPACETRACK: Attempting to fetch TLE data with username: {username}")
-
-        try:
-            async with httpx.AsyncClient(
-                timeout=120.0,
-                follow_redirects=True,
-                headers={
-                    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36",
-                    "Accept": "application/json, text/html, */*",
-                    "Accept-Language": "en-US,en;q=0.9",
-                },
-            ) as client:
-                # First, visit the main page to get any cookies
-                await client.get(f"{self.site_root}/")
-
-                # Login to get session cookie
-                login_response = await client.post(
-                    self.login_url,
-                    data={
-                        "identity": username,
-                        "password": password,
-                    },
+                logger.info_event(
+                    "Space-Track TLE query response received",
+                    event="collector.spacetrack.query.response",
+                    context={"status_code": tle_response.status_code},
                 )
-                print(f"SPACETRACK: Login response status: {login_response.status_code}")
-                print(f"SPACETRACK: Login response URL: {login_response.url}")
-                print(f"SPACETRACK: Login response body: {login_response.text[:500]}")
-
-                if login_response.status_code != 200:
-                    print(f"SPACETRACK: Login failed, using sample data")
-                    return self._get_sample_data()
-
-                # Query for TLE data (get first 1000 satellites)
-                tle_response = await client.get(self.query_url)
-                print(f"SPACETRACK: TLE query status: {tle_response.status_code}")
 
                 if tle_response.status_code != 200:
-                    print(f"SPACETRACK: Query failed, using sample data")
+                    logger.warning_event(
+                        "Space-Track TLE query failed; using sample data",
+                        event="collector.spacetrack.query.failed",
+                        context={"status_code": tle_response.status_code},
+                    )
                     return self._get_sample_data()
 
                 data = tle_response.json()
-                print(f"SPACETRACK: Received {len(data)} records")
+                logger.info_event(
+                    "Space-Track TLE fetch completed",
+                    event="collector.spacetrack.fetch.completed",
+                    context={"record_count": len(data)},
+                )
                 return data
         except Exception as e:
-            print(f"SPACETRACK: Error - {e}, using sample data")
+            logger.warning_event(
+                "Space-Track TLE fetch failed; using sample data",
+                event="collector.spacetrack.fetch.failed",
+                context={"error": str(e)},
+            )
             return self._get_sample_data()
 
     def transform(self, raw_data: List[Dict[str, Any]]) -> List[Dict[str, Any]]:

backend/app/services/collectors/telegeography.py

@@ -163,32 +163,64 @@ class TeleGeographyLandingPointCollector(BaseCollector):
     data_type = "landing_point"
 
     async def fetch(self) -> List[Dict[str, Any]]:
-        """Fetch landing point data from GitHub mirror"""
-        url = self._resolved_url or ""
+        """Fetch landing point data, falling back when the old mirror disappears."""
+        config = get_data_sources_config()
+        sources = [
+            self._resolved_url or "",
+            str(config.get_yaml_value("telegeography.landing_point_url") or ""),
+            str(config.get_yaml_value("arcgis.landing_point_url") or ""),
+        ]
 
-        async with httpx.AsyncClient(timeout=60.0) as client:
-            response = await client.get(url)
-            response.raise_for_status()
-            return self.parse_response(response.json())
+        last_error: Exception | None = None
+        async with httpx.AsyncClient(timeout=60.0, follow_redirects=True) as client:
+            for url in dict.fromkeys(source for source in sources if source):
+                try:
+                    params = (
+                        {"where": "1=1", "outFields": "*", "returnGeometry": "true", "f": "geojson"}
+                        if "FeatureServer" in url or url.endswith("/query")
+                        else None
+                    )
+                    response = await client.get(url, params=params)
+                    response.raise_for_status()
+                    records = self.parse_response(response.json())
+                    if records:
+                        return records
+                except Exception as exc:
+                    last_error = exc
+                    continue
 
-    def parse_response(self, data: List[Dict[str, Any]]) -> List[Dict[str, Any]]:
+        if last_error:
+            raise last_error
+        return self._get_sample_data()
+
+    def parse_response(self, data: List[Dict[str, Any]] | Dict[str, Any]) -> List[Dict[str, Any]]:
         """Parse landing point data"""
         result = []
+        items = data.get("features", []) if isinstance(data, dict) else data
 
-        for item in data:
+        for item in items:
+            props = item.get("properties", {}) if isinstance(item, dict) else {}
+            geometry = item.get("geometry", {}) if isinstance(item, dict) else {}
+            source = props or item
+            coords = geometry.get("coordinates", []) if isinstance(geometry, dict) else []
+            longitude = coords[0] if len(coords) > 0 else source.get("longitude")
+            latitude = coords[1] if len(coords) > 1 else source.get("latitude")
+            source_id = source.get("id") or source.get("OBJECTID") or source.get("city_id") or ""
             try:
                 entry = {
-                    "source_id": f"telegeo_lp_{item.get('id', '')}",
-                    "name": item.get("name", "Unknown"),
-                    "country": item.get("country", "Unknown"),
-                    "city": item.get("city", item.get("name", "")),
-                    "latitude": str(item.get("latitude", "")),
-                    "longitude": str(item.get("longitude", "")),
+                    "source_id": f"telegeo_lp_{source_id}",
+                    "name": source.get("name", source.get("Name", "Unknown")),
+                    "country": source.get("country", "Unknown"),
+                    "city": source.get("city", source.get("Name", source.get("name", ""))),
+                    "latitude": str(latitude or ""),
+                    "longitude": str(longitude or ""),
                     "value": "",
                     "unit": "",
                     "metadata": {
-                        "cable_count": len(item.get("cables", [])),
-                        "url": item.get("url"),
+                        "cable_count": len(source.get("cables", [])),
+                        "url": source.get("url"),
+                        "objectid": source.get("OBJECTID"),
+                        "city_id": source.get("city_id"),
                     },
                     "reference_date": datetime.now(UTC).strftime("%Y-%m-%d"),
                 }

backend/app/services/collectors/vessel_ais.py

@@ -6,6 +6,7 @@ from typing import Any
 import httpx
 from sqlalchemy.ext.asyncio import AsyncSession
 
+from app.core.enums import SnapshotStatus
 from app.core.time import to_iso8601_utc
 from app.core.websocket.broadcaster import broadcaster
 from app.services.barentswatch import (
@@ -119,6 +120,21 @@ class VesselAISCollector(BaseCollector):
             last_success_at=now if data else None,
             lag_seconds=max((now - latest_observed_at).total_seconds(), 0),
         )
+        if snapshot_id is not None:
+            from app.models.data_snapshot import DataSnapshot
+
+            snapshot = await db.get(DataSnapshot, snapshot_id)
+            if snapshot:
+                snapshot.record_count = records_added
+                snapshot.status = SnapshotStatus.SUCCESS.value
+                snapshot.completed_at = now
+                snapshot.summary = {
+                    "created": records_added,
+                    "updated": 0,
+                    "unchanged": 0,
+                    "deleted": 0,
+                    "storage": "ais_raw_observations",
+                }
         await db.commit()
         await self._broadcast_vessel_snapshot(data)
         await self.update_progress(records_added, force=True)

backend/app/services/compute_center_locations.py

@@ -11,7 +11,7 @@ startup, so it must stay local and deterministic.
 
 For the full design and the reason behind the abstraction (compute centers,
 BGP collectors, BGP events, and future entities all share one pipeline),
-see ``docs/plans/location-resolver-shared-pipeline-plan.md``.
+see ``docs/technical/zh/location-pipeline-development.md``.
 
 The ``ComputeCenterLocation`` dataclass and the public function signatures are
 preserved verbatim so existing callers and tests do not need to change.

backend/app/services/credential_guides.py

@@ -3,10 +3,13 @@
 from __future__ import annotations
 
 from dataclasses import dataclass
+from copy import deepcopy
 from typing import Any
 
 from sqlalchemy import select
+from sqlalchemy.orm.attributes import flag_modified
 
+from app.ai_tasks.prompts import get_effective_prompt
 from app.models.system_setting import SystemSetting
 from app.schemas.ai import SituationalAnalysisRequest
 from app.services.ai_client import AIProviderClient
@@ -15,6 +18,7 @@ from app.services.ai_tools.web_search import WebSearchClient, WebSearchError
 
 
 CREDENTIAL_GUIDES_CATEGORY = "collector_credential_guides"
+CREDENTIAL_GUIDE_PROMPT_KEY = "credential.guide"
 
 
 @dataclass(frozen=True)
@@ -133,33 +137,57 @@ DEFAULT_CREDENTIAL_GUIDES = {
 }
 
 
+def _normalize_provider(provider: str) -> str:
+    return provider.strip().lower().replace(" ", "_")
+
+
+def _credential_guide_default(provider: str) -> CredentialGuideDefault:
+    normalized = _normalize_provider(provider)
+    known = DEFAULT_CREDENTIAL_GUIDES.get(normalized)
+    if known is not None:
+        return known
+    title = f"{normalized or 'collector'} 凭证配置教程"
+    return CredentialGuideDefault(
+        provider=normalized,
+        title=title,
+        prompt=(
+            f"请生成一份中文教程，指导开发者为 Planet 采集器配置 {normalized} 凭证。"
+            "教程要面向已经有本地开发环境的人，包含官方入口或文档查找方式、"
+            "获取 API Key / Token / Client credentials 的通用步骤、在 Planet 采集器配置中"
+            "填写凭证字段、连接测试、保存、常见失败排查。不要编造具体页面按钮文案；"
+            "如果公开资料不足，必须明确提醒以 provider 官方文档和当前控制台页面为准。"
+        ),
+        markdown="",
+    )
+
+
 async def _get_guide_store(db) -> tuple[SystemSetting | None, dict[str, Any]]:
     result = await db.execute(
         select(SystemSetting).where(SystemSetting.category == CREDENTIAL_GUIDES_CATEGORY)
     )
     record = result.scalar_one_or_none()
-    payload = dict(record.payload or {}) if record and isinstance(record.payload, dict) else {}
+    payload = deepcopy(record.payload) if record and isinstance(record.payload, dict) else {}
     return record, payload
 
 
 async def get_credential_guide(db, provider: str) -> dict[str, Any]:
-    default = DEFAULT_CREDENTIAL_GUIDES.get(provider)
-    if default is None:
-        raise ValueError(f"Unsupported credential guide provider: {provider}")
+    provider = _normalize_provider(provider)
+    default = _credential_guide_default(provider)
 
     _record, store = await _get_guide_store(db)
     custom = store.get(provider) if isinstance(store.get(provider), dict) else None
+    has_default_markdown = bool(default.markdown.strip())
     return {
         "provider": provider,
         "title": custom.get("title") if custom else default.title,
         "markdown": custom.get("markdown") if custom else default.markdown,
         "prompt": default.prompt,
-        "source": "ai" if custom else "default",
+        "source": "ai" if custom else "default" if has_default_markdown else "missing",
         "sources": custom.get("sources", []) if custom else [],
         "verification_status": (
             custom.get("verification_status", "verified_with_search_evidence")
             if custom
-            else "default_unverified"
+            else "default_unverified" if has_default_markdown else "missing"
         ),
         "verification_error": custom.get("verification_error") if custom else None,
     }
@@ -175,9 +203,8 @@ async def save_credential_guide(
     verification_status: str = "verified_with_search_evidence",
     verification_error: str | None = None,
 ) -> dict[str, Any]:
-    default = DEFAULT_CREDENTIAL_GUIDES.get(provider)
-    if default is None:
-        raise ValueError(f"Unsupported credential guide provider: {provider}")
+    provider = _normalize_provider(provider)
+    default = _credential_guide_default(provider)
 
     record, store = await _get_guide_store(db)
     store[provider] = {
@@ -190,21 +217,21 @@ async def save_credential_guide(
     if record is None:
         db.add(SystemSetting(category=CREDENTIAL_GUIDES_CATEGORY, payload=store))
     else:
-        record.payload = store
+        record.payload = deepcopy(store)
+        flag_modified(record, "payload")
     await db.commit()
     return await get_credential_guide(db, provider)
 
 
 async def reset_credential_guide(db, provider: str) -> dict[str, Any]:
-    default = DEFAULT_CREDENTIAL_GUIDES.get(provider)
-    if default is None:
-        raise ValueError(f"Unsupported credential guide provider: {provider}")
+    provider = _normalize_provider(provider)
 
     record, store = await _get_guide_store(db)
     if provider in store:
         store.pop(provider, None)
         if record is not None:
-            record.payload = store
+            record.payload = deepcopy(store)
+            flag_modified(record, "payload")
             await db.commit()
     return await get_credential_guide(db, provider)
 
@@ -215,9 +242,8 @@ async def generate_credential_guide(
     ai_client: AIProviderClient,
     web_search_client: WebSearchClient | None = None,
 ) -> dict[str, Any]:
-    default = DEFAULT_CREDENTIAL_GUIDES.get(provider)
-    if default is None:
-        raise ValueError(f"Unsupported credential guide provider: {provider}")
+    provider = _normalize_provider(provider)
+    default = _credential_guide_default(provider)
 
     search_evidence: list[dict[str, Any]] = []
     search_error: str | None = None
@@ -240,14 +266,12 @@ async def generate_credential_guide(
         guide["sources"] = []
         return guide
 
+    prompt = await get_effective_prompt(db, CREDENTIAL_GUIDE_PROMPT_KEY)
     response = await ai_client.analyze(
         SituationalAnalysisRequest(
             title=f"Generate credential guide for {provider}",
-            objective=(
-                default.prompt
-                + "\n只能根据 context.search_evidence 中的来源生成教程；"
-                + "如果证据不足，明确说明需要以官方页面为准。"
-            ),
+            objective=f"{default.prompt}\n{prompt.prompt}",
+            system_prompt=prompt.system_prompt or None,
             context={
                 "provider": provider,
                 "current_default_guide": default.markdown,

backend/app/services/data_jobs.py

@@ -0,0 +1,720 @@
+"""Kafka-ready datasource job queue backed by PostgreSQL for v1."""
+
+from __future__ import annotations
+
+import asyncio
+from datetime import UTC, datetime, timedelta
+from uuid import uuid4
+from typing import Any
+
+from sqlalchemy import bindparam, select, text
+from sqlalchemy.ext.asyncio import AsyncSession
+
+from app.core.cache import cache
+from app.core.config import settings
+from app.core.enums import JobStatus, JobType, RollbackPolicy
+from app.core.logging import get_logger
+from app.core.time import to_iso8601_utc
+from app.core.websocket.broadcaster import broadcaster
+from app.db.session import async_session_factory
+from app.models.collected_data import CollectedData
+from app.models.data_snapshot import DataSnapshot
+from app.models.datasource import DataSource
+from app.models.task import CollectionTask
+from app.services.collectors.registry import collector_registry
+from app.services.datasource_connectivity import (
+    build_builtin_connectivity_checksum,
+    get_builtin_effective_candidate,
+    save_connectivity_success,
+)
+from app.services.earth_layer_adapters import (
+    clear_derived_datasource_data,
+    get_earth_refresh_strategy_for_change,
+    get_earth_update_layers_for_source,
+)
+from app.services.earth_layer_cache import invalidate_earth_layer_cache_for_source
+from app.services.scheduler import sync_datasource_job
+
+logger = get_logger(__name__)
+
+JOB_TYPE_COLLECT = JobType.COLLECT.value
+JOB_TYPE_CLEAR_DATA = JobType.CLEAR_DATA.value
+JOB_TYPE_CLEAR_CACHE = JobType.CLEAR_CACHE.value
+JOB_TYPE_EARTH_REFRESH = JobType.EARTH_REFRESH.value
+
+JOB_STATUS_QUEUED = JobStatus.QUEUED.value
+JOB_STATUS_RUNNING = JobStatus.RUNNING.value
+JOB_STATUS_CANCELLING = JobStatus.CANCELLING.value
+JOB_STATUS_SUCCESS = JobStatus.SUCCESS.value
+JOB_STATUS_FAILED = JobStatus.FAILED.value
+JOB_STATUS_CANCELLED = JobStatus.CANCELLED.value
+
+ACTIVE_JOB_STATUSES = (JOB_STATUS_QUEUED, JOB_STATUS_RUNNING, JOB_STATUS_CANCELLING)
+TERMINAL_JOB_STATUSES = (JOB_STATUS_SUCCESS, JOB_STATUS_FAILED, JOB_STATUS_CANCELLED)
+DATA_WRITE_JOB_TYPES = (JOB_TYPE_COLLECT, JOB_TYPE_CLEAR_DATA, JOB_TYPE_CLEAR_CACHE)
+SOURCE_LOCK_JOB_STATUSES = (JOB_STATUS_RUNNING, JOB_STATUS_CANCELLING)
+QUEUE_POLL_SECONDS = 0.35
+JOB_STALE_LOCK_MINUTES = 90
+ORPHAN_CANCELLING_GRACE_SECONDS = 30
+JOB_RECOVERY_SWEEP_SECONDS = 15
+DATA_DELETE_BATCH_SIZE = 50_000
+DEFAULT_WORKER_CONCURRENCY = 2
+
+RUNNING_DATA_JOB_TASKS: dict[int, asyncio.Task[Any]] = {}
+
+
+def _utcnow() -> datetime:
+    return datetime.now(UTC)
+
+
+def _job_worker_id() -> str:
+    return f"{settings.PROJECT_NAME}:data-job-worker:{uuid4().hex[:8]}"
+
+
+def is_terminal_job_status(status: str | None) -> bool:
+    return status in TERMINAL_JOB_STATUSES
+
+
+async def enqueue_datasource_job(
+    db: AsyncSession,
+    datasource: DataSource,
+    task_type: str,
+    *,
+    payload: dict[str, Any] | None = None,
+    rollback_policy: str = RollbackPolicy.KEEP_COMMITTED_BATCHES.value,
+    dedupe_key: str | None = None,
+) -> CollectionTask:
+    if dedupe_key:
+        existing = await _get_active_job_by_dedupe_key(db, dedupe_key)
+        if existing is not None:
+            return existing
+
+    task = CollectionTask(
+        datasource_id=datasource.id,
+        source=datasource.source,
+        task_type=task_type,
+        status=JOB_STATUS_QUEUED,
+        phase="queued",
+        phase_message="任务已进入队列",
+        payload=payload or {},
+        rollback_policy=rollback_policy,
+        dedupe_key=dedupe_key,
+    )
+    db.add(task)
+    await db.commit()
+    await db.refresh(task)
+    await _broadcast_task_update(task)
+    return task
+
+
+async def enqueue_earth_refresh_job(
+    db: AsyncSession,
+    *,
+    source: str,
+    payload: dict[str, Any] | None = None,
+) -> CollectionTask | None:
+    layers = list((payload or {}).get("layers") or get_earth_update_layers_for_source(source))
+    if not layers:
+        return None
+
+    datasource = await _get_or_create_virtual_datasource(db, source)
+    refresh_payload = {
+        "source": source,
+        "layers": layers,
+        "refresh_strategy": (payload or {}).get("refresh_strategy")
+        or get_earth_refresh_strategy_for_change((payload or {}).get("table"), source)
+        or "clear_then_reload",
+        **(payload or {}),
+    }
+    return await enqueue_datasource_job(
+        db,
+        datasource,
+        JOB_TYPE_EARTH_REFRESH,
+        payload=refresh_payload,
+        dedupe_key=f"earth_refresh:{source}",
+    )
+
+
+async def enqueue_earth_refresh_from_update(payload: dict[str, Any]) -> None:
+    source = str(payload.get("source") or "").strip()
+    if not source:
+        return
+    async with async_session_factory() as db:
+        await enqueue_earth_refresh_job(db, source=source, payload=payload)
+
+
+async def request_cancel_datasource_task(
+    db: AsyncSession,
+    task: CollectionTask,
+    *,
+    reason: str = "cancelled_by_operator",
+) -> CollectionTask:
+    if is_terminal_job_status(task.status):
+        return task
+
+    running_task = RUNNING_DATA_JOB_TASKS.get(task.id)
+    if task.status == JOB_STATUS_QUEUED or (
+        running_task is None
+        and (
+            task.status == JOB_STATUS_CANCELLING
+            or (task.status == JOB_STATUS_RUNNING and task.task_type != JOB_TYPE_COLLECT)
+        )
+    ):
+        return await _cancel_task_without_runner(db, task, reason=reason)
+
+    task.status = JOB_STATUS_CANCELLING
+    task.phase = JOB_STATUS_CANCELLING
+    task.phase_message = "正在停止任务"
+    task.requested_cancel_at = _utcnow()
+    task.cancel_reason = reason
+    await db.commit()
+    await db.refresh(task)
+
+    if running_task is not None and not running_task.done():
+        running_task.cancel()
+
+    await _broadcast_task_update(task)
+    return task
+
+
+async def _cancel_task_without_runner(
+    db: AsyncSession,
+    task: CollectionTask,
+    *,
+    reason: str,
+) -> CollectionTask:
+    if task.task_type == JOB_TYPE_COLLECT:
+        await db.execute(CollectedData.__table__.delete().where(CollectedData.task_id == task.id))
+        snapshot_result = await db.execute(select(DataSnapshot).where(DataSnapshot.task_id == task.id))
+        for snapshot in snapshot_result.scalars().all():
+            snapshot.status = JOB_STATUS_CANCELLED
+            snapshot.completed_at = _utcnow()
+            snapshot.error_message = "Cancelled after operator stop request; no active worker handle remained"
+        datasource = await db.get(DataSource, task.datasource_id)
+        if datasource is not None:
+            datasource.last_status = JOB_STATUS_CANCELLED
+
+    task.status = JOB_STATUS_CANCELLED
+    task.phase = JOB_STATUS_CANCELLED
+    task.phase_message = "任务已停止"
+    task.completed_at = _utcnow()
+    task.requested_cancel_at = task.requested_cancel_at or _utcnow()
+    task.cancel_reason = reason
+    await db.commit()
+    await db.refresh(task)
+    await _broadcast_task_update(task)
+    return task
+
+
+async def get_active_datasource_job(
+    db: AsyncSession,
+    datasource_id: int,
+    *,
+    task_types: tuple[str, ...] = DATA_WRITE_JOB_TYPES,
+) -> CollectionTask | None:
+    result = await db.execute(
+        select(CollectionTask)
+        .where(CollectionTask.datasource_id == datasource_id)
+        .where(CollectionTask.task_type.in_(task_types))
+        .where(CollectionTask.status.in_(ACTIVE_JOB_STATUSES))
+        .order_by(CollectionTask.created_at.desc().nullslast(), CollectionTask.id.desc())
+        .limit(1)
+    )
+    return result.scalar_one_or_none()
+
+
+async def _get_active_job_by_dedupe_key(db: AsyncSession, dedupe_key: str) -> CollectionTask | None:
+    result = await db.execute(
+        select(CollectionTask)
+        .where(CollectionTask.dedupe_key == dedupe_key)
+        .where(CollectionTask.status.in_(ACTIVE_JOB_STATUSES))
+        .order_by(CollectionTask.created_at.desc().nullslast(), CollectionTask.id.desc())
+        .limit(1)
+    )
+    return result.scalar_one_or_none()
+
+
+async def _get_or_create_virtual_datasource(db: AsyncSession, source: str) -> DataSource:
+    result = await db.execute(select(DataSource).where(DataSource.source == source))
+    datasource = result.scalar_one_or_none()
+    if datasource is not None:
+        return datasource
+
+    datasource = DataSource(
+        name=f"Earth refresh: {source}",
+        source=source,
+        module="SYS",
+        collector_class="EarthRefreshJob",
+        is_active=True,
+    )
+    db.add(datasource)
+    await db.commit()
+    await db.refresh(datasource)
+    return datasource
+
+
+async def _broadcast_task_update(task: CollectionTask) -> None:
+    await broadcaster.broadcast_datasource_task_update(
+        {
+            "datasource_id": task.datasource_id,
+            "collector_name": task.source,
+            "task_id": task.id,
+            "task_type": task.task_type,
+            "status": task.status,
+            "phase": task.phase,
+            "phase_progress": task.phase_progress,
+            "phase_message": task.phase_message,
+            "phase_current": task.phase_current,
+            "phase_total": task.phase_total,
+            "phase_unit": task.phase_unit,
+            "progress": task.progress,
+            "records_processed": task.records_processed,
+            "total_records": task.total_records,
+            "started_at": to_iso8601_utc(task.started_at),
+            "completed_at": to_iso8601_utc(task.completed_at),
+            "requested_cancel_at": to_iso8601_utc(task.requested_cancel_at),
+            "error_message": task.error_message,
+        }
+    )
+
+
+class DataJobWorker:
+    def __init__(self, *, concurrency: int = DEFAULT_WORKER_CONCURRENCY) -> None:
+        self.worker_id = _job_worker_id()
+        self.concurrency = max(1, concurrency)
+        self._task: asyncio.Task[None] | None = None
+        self._stop_event: asyncio.Event | None = None
+        self._running: set[asyncio.Task[Any]] = set()
+        self._last_recovery_sweep_at: datetime | None = None
+
+    def start(self) -> None:
+        if self._task and not self._task.done():
+            return
+        self._stop_event = asyncio.Event()
+        self._task = asyncio.create_task(self._run(), name="data-job-worker")
+
+    async def stop(self) -> None:
+        if self._stop_event:
+            self._stop_event.set()
+        for task in list(self._running):
+            task.cancel()
+        if self._task:
+            await asyncio.gather(self._task, return_exceptions=True)
+        if self._running:
+            await asyncio.gather(*self._running, return_exceptions=True)
+
+    async def _run(self) -> None:
+        assert self._stop_event is not None
+        await self._recover_stale_running_jobs()
+        while not self._stop_event.is_set():
+            self._running = {task for task in self._running if not task.done()}
+            if (
+                self._last_recovery_sweep_at is None
+                or (_utcnow() - self._last_recovery_sweep_at).total_seconds() >= JOB_RECOVERY_SWEEP_SECONDS
+            ):
+                await self._recover_stale_running_jobs()
+            if len(self._running) >= self.concurrency:
+                await asyncio.sleep(QUEUE_POLL_SECONDS)
+                continue
+
+            task_id = await self._claim_next_job()
+            if task_id is None:
+                await asyncio.sleep(QUEUE_POLL_SECONDS)
+                continue
+
+            runner = asyncio.create_task(self._run_claimed_job(task_id), name=f"data-job:{task_id}")
+            self._running.add(runner)
+
+    async def _recover_stale_running_jobs(self) -> None:
+        self._last_recovery_sweep_at = _utcnow()
+        cutoff = _utcnow() - timedelta(minutes=JOB_STALE_LOCK_MINUTES)
+        orphan_cancelling_cutoff = _utcnow() - timedelta(seconds=ORPHAN_CANCELLING_GRACE_SECONDS)
+        async with async_session_factory() as db:
+            result = await db.execute(
+                select(CollectionTask)
+                .where(CollectionTask.status.in_((JOB_STATUS_RUNNING, JOB_STATUS_CANCELLING)))
+                .where(CollectionTask.locked_at.is_not(None))
+                .where(CollectionTask.locked_at < cutoff)
+            )
+            stale_jobs = list(result.scalars().all())
+            for job in stale_jobs:
+                job.status = JOB_STATUS_FAILED
+                job.phase = JOB_STATUS_FAILED
+                job.completed_at = _utcnow()
+                job.error_message = "Marked failed after stale data job lock timeout"
+            if stale_jobs:
+                await db.commit()
+            orphan_result = await db.execute(
+                select(CollectionTask)
+                .where(CollectionTask.status == JOB_STATUS_CANCELLING)
+                .where(CollectionTask.locked_at.is_(None))
+                .where(CollectionTask.requested_cancel_at.is_not(None))
+                .where(CollectionTask.requested_cancel_at < orphan_cancelling_cutoff)
+            )
+            for job in orphan_result.scalars().all():
+                if job.id in RUNNING_DATA_JOB_TASKS:
+                    continue
+                await _cancel_task_without_runner(
+                    db,
+                    job,
+                    reason=job.cancel_reason or "cancelled_after_orphaned_runner",
+                )
+
+    async def _claim_next_job(self) -> int | None:
+        async with async_session_factory() as db:
+            row = await db.execute(
+                text(
+                    """
+                    SELECT queued.id
+                    FROM collection_tasks AS queued
+                    WHERE queued.status = :queued_status
+                      AND NOT EXISTS (
+                        SELECT 1
+                        FROM collection_tasks AS active
+                        WHERE active.source = queued.source
+                          AND active.id <> queued.id
+                          AND active.status IN :active_statuses
+                      )
+                    ORDER BY queued.created_at ASC NULLS FIRST, queued.id ASC
+                    LIMIT 1
+                    FOR UPDATE SKIP LOCKED
+                    """
+                ).bindparams(bindparam("active_statuses", expanding=True)),
+                {
+                    "queued_status": JOB_STATUS_QUEUED,
+                    "active_statuses": SOURCE_LOCK_JOB_STATUSES,
+                },
+            )
+            task_id = row.scalar_one_or_none()
+            if task_id is None:
+                return None
+
+            task = await db.get(CollectionTask, int(task_id))
+            if task is None:
+                return None
+            task.status = JOB_STATUS_RUNNING
+            task.phase = "starting"
+            task.phase_message = "任务开始执行"
+            task.started_at = task.started_at or _utcnow()
+            task.worker_id = self.worker_id
+            task.locked_at = _utcnow()
+            await db.commit()
+            await _broadcast_task_update(task)
+            return int(task_id)
+
+    async def _run_claimed_job(self, task_id: int) -> None:
+        current_task = asyncio.current_task()
+        if current_task is not None:
+            RUNNING_DATA_JOB_TASKS[task_id] = current_task
+        try:
+            async with async_session_factory() as db:
+                task = await db.get(CollectionTask, task_id)
+                if task is None:
+                    return
+                await self._execute_job(db, task)
+        except asyncio.CancelledError:
+            async with async_session_factory() as db:
+                task = await db.get(CollectionTask, task_id)
+                if task is not None and not is_terminal_job_status(task.status):
+                    task.status = JOB_STATUS_CANCELLED
+                    task.phase = JOB_STATUS_CANCELLED
+                    task.phase_message = "任务已停止"
+                    task.completed_at = _utcnow()
+                    await db.commit()
+                    await _broadcast_task_update(task)
+            raise
+        except Exception as exc:
+            logger.exception_event(
+                "Data job failed",
+                event="data_jobs.job_failed",
+                context={"task_id": task_id, "error": str(exc)},
+            )
+            async with async_session_factory() as db:
+                task = await db.get(CollectionTask, task_id)
+                if task is not None:
+                    task.status = JOB_STATUS_FAILED
+                    task.phase = JOB_STATUS_FAILED
+                    task.phase_message = str(exc)
+                    task.error_message = str(exc)
+                    task.completed_at = _utcnow()
+                    await db.commit()
+                    await _broadcast_task_update(task)
+        finally:
+            RUNNING_DATA_JOB_TASKS.pop(task_id, None)
+
+    async def _execute_job(self, db: AsyncSession, task: CollectionTask) -> None:
+        if task.task_type == JOB_TYPE_COLLECT:
+            await _run_collect_job(db, task)
+        elif task.task_type == JOB_TYPE_CLEAR_DATA:
+            await _run_clear_data_job(db, task)
+        elif task.task_type == JOB_TYPE_CLEAR_CACHE:
+            await _run_clear_cache_job(db, task)
+        elif task.task_type == JOB_TYPE_EARTH_REFRESH:
+            await _run_earth_refresh_job(db, task)
+        else:
+            raise RuntimeError(f"Unsupported data job type: {task.task_type}")
+
+
+async def _run_collect_job(db: AsyncSession, task: CollectionTask) -> None:
+    datasource = await db.get(DataSource, task.datasource_id)
+    if datasource is None:
+        raise RuntimeError("Data source not found")
+
+    collector = collector_registry.get(datasource.source)
+    if collector is None:
+        raise RuntimeError(f"Collector '{datasource.source}' not found")
+    if not datasource.is_active:
+        raise RuntimeError("Data source is disabled")
+
+    collector._datasource_id = datasource.id
+    collector._current_task = task
+    collector._db_session = db
+    result = await collector.run(db)
+
+    datasource.last_run_at = _utcnow()
+    datasource.last_status = result.get("status")
+    if datasource.last_status == JOB_STATUS_SUCCESS:
+        effective_candidate = await get_builtin_effective_candidate(db, datasource.source)
+        checksum, _credential_context = await build_builtin_connectivity_checksum(
+            datasource.source,
+            effective_candidate["endpoint"],
+            effective_candidate["auth_type"],
+            effective_candidate["headers"],
+            effective_candidate["config"],
+            db,
+        )
+        await save_connectivity_success(
+            db,
+            datasource.source,
+            checksum,
+            {"status_code": None},
+            connected_by="collection",
+        )
+    await db.commit()
+    await sync_datasource_job(datasource.id)
+
+
+async def _run_clear_data_job(db: AsyncSession, task: CollectionTask) -> None:
+    source = str(task.source or (task.payload or {}).get("source") or "").strip()
+    if not source:
+        raise RuntimeError("Clear data job has no source")
+
+    task.phase = "clearing_data"
+    task.phase_message = "正在删除数据库数据"
+    await db.commit()
+    await _broadcast_task_update(task)
+
+    deleted_count = await _delete_table_rows_by_source(
+        db,
+        task,
+        table_name="collected_data",
+        source_column="source",
+        source=source,
+    )
+    derived_deleted_counts = await _clear_derived_datasource_data_in_batches(
+        db,
+        task,
+        source,
+        progress_offset=deleted_count,
+    )
+    derived_deleted_count = sum(derived_deleted_counts.values())
+    if any(key.startswith("ais_") for key in derived_deleted_counts):
+        await db.execute(text("ANALYZE ais_raw_observations"))
+
+    task.records_processed = deleted_count + derived_deleted_count
+    task.total_records = task.records_processed
+    task.progress = 100.0
+    task.phase_progress = 100.0
+    task.phase_current = task.records_processed
+    task.phase_total = task.records_processed
+    task.phase_unit = "records"
+    task.payload = {
+        **(task.payload or {}),
+        "deleted_count": deleted_count,
+        "derived_deleted_count": derived_deleted_count,
+        "derived_deleted_counts": derived_deleted_counts,
+    }
+    task.status = JOB_STATUS_SUCCESS
+    task.phase = "completed"
+    task.phase_message = "数据库数据已清理"
+    task.completed_at = _utcnow()
+    datasource = await db.get(DataSource, task.datasource_id)
+    if datasource is not None:
+        datasource.last_status = JOB_STATUS_SUCCESS
+        datasource.last_run_at = task.completed_at
+    await db.execute(
+        DataSnapshot.__table__.update()
+        .where(DataSnapshot.source == source)
+        .values(is_current=False)
+    )
+    await db.commit()
+    await _broadcast_task_update(task)
+
+
+async def _delete_table_rows_by_source(
+    db: AsyncSession,
+    task: CollectionTask,
+    *,
+    table_name: str,
+    source_column: str,
+    source: str,
+    progress_offset: int = 0,
+) -> int:
+    deleted = 0
+    while True:
+        result = await db.execute(
+            text(
+                f"""
+                WITH doomed AS (
+                    SELECT ctid
+                    FROM {table_name}
+                    WHERE {source_column} = :source
+                    LIMIT :batch_size
+                ),
+                deleted_rows AS (
+                    DELETE FROM {table_name}
+                    USING doomed
+                    WHERE {table_name}.ctid = doomed.ctid
+                    RETURNING 1
+                )
+                SELECT COUNT(*) FROM deleted_rows
+                """
+            ),
+            {"source": source, "batch_size": DATA_DELETE_BATCH_SIZE},
+        )
+        batch_deleted = max(int(result.scalar_one() or 0), 0)
+        if batch_deleted <= 0:
+            break
+        deleted += batch_deleted
+        task.records_processed = progress_offset + deleted
+        task.phase_current = task.records_processed
+        task.phase_unit = "records"
+        task.phase_message = f"正在删除数据：{task.records_processed} 条"
+        await db.commit()
+        await _broadcast_task_update(task)
+    return deleted
+
+
+async def _clear_derived_datasource_data_in_batches(
+    db: AsyncSession,
+    task: CollectionTask,
+    source: str,
+    progress_offset: int = 0,
+) -> dict[str, int]:
+    deleted_counts: dict[str, int] = {}
+    if source in {"barentswatch_vessels", "aisstream_vessels"}:
+        deleted_counts["ais_conflict_records"] = await _delete_table_rows_by_source(
+            db,
+            task,
+            table_name="ais_conflict_records",
+            source_column="selected_source",
+            source=source,
+            progress_offset=progress_offset + sum(deleted_counts.values()),
+        )
+        deleted_counts["ais_source_health"] = await _delete_table_rows_by_source(
+            db,
+            task,
+            table_name="ais_source_health",
+            source_column="source",
+            source=source,
+            progress_offset=progress_offset + sum(deleted_counts.values()),
+        )
+        deleted_counts["ais_raw_observations"] = await _delete_table_rows_by_source(
+            db,
+            task,
+            table_name="ais_raw_observations",
+            source_column="source",
+            source=source,
+            progress_offset=progress_offset + sum(deleted_counts.values()),
+        )
+        return deleted_counts
+    return await clear_derived_datasource_data(db, source)
+
+
+async def _run_clear_cache_job(db: AsyncSession, task: CollectionTask) -> None:
+    source = str(task.source or (task.payload or {}).get("source") or "").strip()
+    if not source:
+        raise RuntimeError("Clear cache job has no source")
+
+    earth_deleted_count = invalidate_earth_layer_cache_for_source(source)
+    dashboard_deleted_count = int(cache.delete("dashboard:stats")) + int(cache.delete("dashboard:summary"))
+    deleted_count = earth_deleted_count + dashboard_deleted_count
+
+    task.records_processed = deleted_count
+    task.total_records = deleted_count
+    task.progress = 100.0
+    task.phase_progress = 100.0
+    task.phase = "completed"
+    task.phase_message = "缓存已清理"
+    task.payload = {
+        **(task.payload or {}),
+        "earth_layer_deleted_count": earth_deleted_count,
+        "dashboard_deleted_count": dashboard_deleted_count,
+    }
+    task.status = JOB_STATUS_SUCCESS
+    task.completed_at = _utcnow()
+    await db.commit()
+    await _broadcast_task_update(task)
+    await enqueue_earth_refresh_job(db, source=source, payload={"operation": "CACHE_INVALIDATED"})
+
+
+async def _run_earth_refresh_job(db: AsyncSession, task: CollectionTask) -> None:
+    payload = task.payload or {}
+    source = str(payload.get("source") or task.source or "").strip()
+    layers = list(payload.get("layers") or get_earth_update_layers_for_source(source))
+    if not source or not layers:
+        task.status = JOB_STATUS_SUCCESS
+        task.phase = "completed"
+        task.phase_message = "没有需要刷新的 Earth 图层"
+        task.completed_at = _utcnow()
+        await db.commit()
+        await _broadcast_task_update(task)
+        return
+
+    deleted_cache_entries = invalidate_earth_layer_cache_for_source(source)
+    update_payload = {
+        "event": "earth.layer.changed",
+        "action": "database_changed",
+        "source": source,
+        "table": payload.get("table"),
+        "data_type": source,
+        "layers": layers,
+        "refresh_strategy": payload.get("refresh_strategy") or "clear_then_reload",
+        "records_processed": payload.get("records_processed", 0),
+        "operations": payload.get("operations") or [payload.get("operation") or "CHANGE"],
+        "operation": payload.get("operation"),
+        "cache_entries_invalidated": deleted_cache_entries,
+        "timestamp": to_iso8601_utc(_utcnow()),
+    }
+    if payload.get("entity") == "interactable":
+        update_payload.update(
+            {
+                "entity": "interactable",
+                "action": payload.get("action") or "changed",
+                "ids": payload.get("ids") or payload.get("entity_keys") or [],
+                "item": payload.get("item"),
+            }
+        )
+    await broadcaster.broadcast_earth_update(update_payload)
+
+    task.records_processed = int(payload.get("records_processed") or 0)
+    task.progress = 100.0
+    task.phase_progress = 100.0
+    task.phase = "completed"
+    task.phase_message = "Earth 图层刷新通知已发送"
+    task.status = JOB_STATUS_SUCCESS
+    task.completed_at = _utcnow()
+    task.payload = {**payload, "cache_entries_invalidated": deleted_cache_entries}
+    await db.commit()
+    await _broadcast_task_update(task)
+
+
+_worker = DataJobWorker()
+
+
+def start_data_job_worker() -> None:
+    _worker.start()
+
+
+async def stop_data_job_worker() -> None:
+    await _worker.stop()

backend/app/services/datasource_connectivity.py

@@ -13,11 +13,13 @@ from sqlalchemy import func, select
 
 from app.core.data_sources import get_data_sources_config
 from app.core.datasource_defaults import DEFAULT_DATASOURCES
+from app.core.enums import JobStatus
 from app.models.collected_data import CollectedData
 from app.models.datasource import DataSource
 from app.models.datasource_config import DataSourceConfig
 from app.models.system_setting import SystemSetting
 from app.services.barentswatch import (
+    BarentsWatchConfig,
     _read_zshrc_env,
     fetch_barentswatch_access_token,
     resolve_barentswatch_config,
@@ -44,6 +46,20 @@ def _resolve_spacetrack_credentials() -> tuple[str, str, str]:
     return username, password, source or "missing"
 
 
+def _resolve_spacetrack_credentials_with_override(
+    credential_override: dict[str, str] | None = None,
+) -> tuple[str, str, str]:
+    if credential_override and (
+        credential_override.get("username") or credential_override.get("password")
+    ):
+        return (
+            str(credential_override.get("username") or ""),
+            str(credential_override.get("password") or ""),
+            "draft",
+        )
+    return _resolve_spacetrack_credentials()
+
+
 async def _resolve_aisstream_api_key(
     db=None,
     credential_override: dict[str, str] | None = None,
@@ -126,7 +142,9 @@ async def build_builtin_connectivity_checksum(
             }
         )
     elif credential_provider == "spacetrack":
-        username, password, credential_source = _resolve_spacetrack_credentials()
+        username, password, credential_source = _resolve_spacetrack_credentials_with_override(
+            credential_override
+        )
         has_credentials = bool(username and password)
         credential_fingerprint = _sha256_json(
             {
@@ -230,7 +248,16 @@ async def test_builtin_connectivity(
     try:
         async with httpx.AsyncClient(timeout=timeout, follow_redirects=True) as client:
             if credential_context["credential_provider"] == "barentswatch":
-                barentswatch_config = await resolve_barentswatch_config(db)
+                if credential_override:
+                    barentswatch_config = BarentsWatchConfig(
+                        endpoint=str(request_endpoint or ""),
+                        client_id=str(credential_override.get("client_id") or ""),
+                        client_secret=str(credential_override.get("client_secret") or ""),
+                        credential_source="draft",
+                        endpoint_source="draft",
+                    )
+                else:
+                    barentswatch_config = await resolve_barentswatch_config(db)
                 token = await fetch_barentswatch_access_token(client, barentswatch_config)
                 if not token:
                     return {
@@ -243,7 +270,9 @@ async def test_builtin_connectivity(
                     }
                 request_headers["Authorization"] = f"Bearer {token}"
             elif credential_context["credential_provider"] == "spacetrack":
-                username, password, _source = _resolve_spacetrack_credentials()
+                username, password, _source = _resolve_spacetrack_credentials_with_override(
+                    credential_override
+                )
                 login_url = "https://www.space-track.org/ajaxauth/login"
                 login_response = await client.post(
                     login_url,
@@ -370,7 +399,7 @@ async def has_collected_data(db, source: str) -> bool:
 
     datasource_result = await db.execute(select(DataSource).where(DataSource.source == source))
     datasource = datasource_result.scalar_one_or_none()
-    return bool(datasource and datasource.last_status == "success")
+    return bool(datasource and datasource.last_status == JobStatus.SUCCESS.value)
 
 
 async def get_builtin_connection_status(

backend/app/services/datasource_mapping.py

@@ -8,6 +8,7 @@ import re
 from datetime import UTC, datetime
 from typing import Any
 
+from sqlalchemy import select
 from sqlalchemy.ext.asyncio import AsyncSession
 
 from app.core.target_schema_registry import TargetSchema, get_target_schema
@@ -254,6 +255,12 @@ def _best_field_match(field_name: str, candidates: list[str]) -> str | None:
         "lat": ("lat", "latitude", "y"),
         "lon": ("lon", "lng", "longitude", "x"),
         "mmsi": ("mmsi",),
+        "geometry": ("geometry", "geom"),
+        "properties": ("properties", "props"),
+        "source_kind": ("source_kind", "kind", "type"),
+        "feature_count": ("feature_count", "features_count", "count"),
+        "artifact_path": ("artifact_path", "path", "file"),
+        "sha256": ("sha256", "hash", "checksum"),
         "sog": ("sog", "speed", "speedOverGround"),
         "cog": ("cog", "course", "courseOverGround"),
         "received_at": ("received_at", "timestamp", "time", "updated_at"),

backend/app/services/docs_gatekeeper.py

@@ -6,6 +6,7 @@ from dataclasses import dataclass
 from pathlib import Path
 from typing import Literal
 
+from app.core.enums import UserRole
 from app.models.user import User
 
 DocsAccess = Literal["public", "docs_user", "docs_developer", "docs_admin"]
@@ -32,28 +33,34 @@ class DocsMetadata:
 
 DOCS_METADATA: tuple[DocsMetadata, ...] = (
     DocsMetadata(DOCS_README_FILENAME, DEFAULT_DOCS_SLUG, "public", "Overview", 0, "技术文档", "Technical Docs"),
-    DocsMetadata("quickstart.md", "quickstart", "public", "Manual", 1, "快速开始", "Quickstart"),
-    DocsMetadata("manual.md", "manual", "public", "Manual", 2, "Planet 使用手册", "Planet Manual"),
+    DocsMetadata("manual.md", "manual", "public", "Manual", 1, "智能星球使用手册", "Intelligent Planet Manual"),
+    DocsMetadata("quickstart.md", "quickstart", "public", "Manual", 2, "快速开始", "Quickstart"),
     DocsMetadata("faq.md", "faq", "public", "Manual", 3, "常见问题", "FAQ"),
-    DocsMetadata("location-pipeline-user.md", "location-pipeline-user", "public", "Manual", 4, "Earth 位置候选采集使用手册", "Earth Location Candidate Collection User Guide"),
-    DocsMetadata("earth-frontend-context.md", "earth-frontend-context", "docs_developer", "Earth", 10, "Earth 前端结构", "Earth Frontend Context"),
-    DocsMetadata("earth-layer-style-reference.md", "earth-layer-style-reference", "docs_developer", "Earth", 11, "Earth 图层样式属性索引", "Earth Layer Style Reference"),
-    DocsMetadata("earth-render-layer-order.md", "earth-render-layer-order", "docs_developer", "Earth", 12, "Earth 渲染图层顺序", "Earth Render Layer Order"),
-    DocsMetadata("earth-satellite-footprint-policy.md", "earth-satellite-footprint-policy", "docs_developer", "Earth", 13, "Earth 卫星覆盖策略", "Earth Satellite Footprint Policy"),
+    DocsMetadata("platform-data-flows.md", "platform-data-flows", "docs_developer", "Architecture", 5, "业务架构与数据流转", "Business Architecture and Data Flows"),
+    DocsMetadata("naming-glossary.md", "naming-glossary", "docs_developer", "Architecture", 6, "命名与术语对照", "Naming Glossary"),
+    DocsMetadata("earth-frontend-context.md", "earth-frontend-context", "docs_developer", "Earth", 10, "智能星球前端结构", "Intelligent Planet Frontend Context"),
+    DocsMetadata("earth-layer-style-reference.md", "earth-layer-style-reference", "docs_developer", "Earth", 11, "智能星球图层样式属性索引", "Intelligent Planet Layer Style Reference"),
+    DocsMetadata("earth-render-layer-order.md", "earth-render-layer-order", "docs_developer", "Earth", 12, "智能星球渲染图层顺序", "Intelligent Planet Render Layer Order"),
+    DocsMetadata("earth-satellite-footprint-policy.md", "earth-satellite-footprint-policy", "docs_developer", "Earth", 13, "智能星球卫星覆盖策略", "Intelligent Planet Satellite Footprint Policy"),
     DocsMetadata("earth-bgp-context.md", "earth-bgp-context", "docs_developer", "Earth", 14, "BGP 态势上下文", "BGP Context"),
-    DocsMetadata("earth-news-live-streams-collector-format.md", "earth-news-live-streams-collector-format", "docs_developer", "Earth", 15, "新闻直播采集格式", "News Live Streams Collector Format"),
-    DocsMetadata("earth-interactable-usage.md", "earth-interactable-usage", "docs_developer", "Earth", 16, "Earth 可交互图标接入", "Earth Interactable Usage"),
-    DocsMetadata("earth-toolbar-overlay-coordination.md", "earth-toolbar-overlay-coordination", "docs_developer", "Earth", 17, "Earth 工具栏与浮层协同", "Earth Toolbar and Overlay Coordination"),
+    DocsMetadata("earth-interactable-usage.md", "earth-interactable-usage", "docs_developer", "Earth", 16, "智能星球可交互图标接入", "Intelligent Planet Interactable Usage"),
+    DocsMetadata("earth-interactable-clustering.md", "earth-interactable-clustering", "docs_developer", "Earth", 17, "智能星球可交互图标聚类策略", "Intelligent Planet Interactable Clustering"),
+    DocsMetadata("earth-toolbar-overlay-coordination.md", "earth-toolbar-overlay-coordination", "docs_developer", "Earth", 18, "智能星球工具栏与浮层协同", "Intelligent Planet Toolbar and Overlay Coordination"),
+    DocsMetadata("earth-news-sources.md", "earth-news-sources", "docs_developer", "Earth", 19, "智能星球新闻源配置", "Intelligent Planet News Source Configuration"),
     DocsMetadata("frontend-admin-frontend-context.md", "frontend-admin-frontend-context", "docs_developer", "Frontend", 20, "控制台前端结构", "Admin Frontend Context"),
     DocsMetadata("frontend-layout-guidelines.md", "frontend-layout-guidelines", "docs_developer", "Frontend", 21, "前端布局指南", "Frontend Layout Guidelines"),
-    DocsMetadata("docs-gatekeeper-development.md", "docs-gatekeeper-development", "docs_developer", "Frontend", 22, "Docs Gatekeeper 开发说明", "Docs Gatekeeper Development Guide"),
+    DocsMetadata("tactile-ui-components.md", "tactile-ui-components", "docs_developer", "Frontend", 24, "Tactile UI 组件库", "Tactile UI Components"),
     DocsMetadata("backend-collectors.md", "backend-collectors", "docs_developer", "Backend", 30, "数据采集系统", "Data Collectors"),
     DocsMetadata("backend-system-service-control.md", "backend-system-service-control", "docs_admin", "Backend", 31, "系统服务控制", "System Service Control"),
     DocsMetadata("datasource-collector-settings-connectivity.md", "datasource-collector-settings-connectivity", "docs_developer", "Backend", 32, "数据源、采集器设置与连接验证", "Datasource Collector Settings and Connectivity"),
     DocsMetadata("backend-datasources-api-performance.md", "backend-datasources-api-performance", "docs_developer", "Backend", 33, "数据源 API 性能", "Datasource API Performance"),
-    DocsMetadata("location-pipeline-development.md", "location-pipeline-development", "docs_developer", "Backend", 34, "通用位置估算管线开发说明", "Shared Location Resolution Pipeline Development Guide"),
+    DocsMetadata("data-job-earth-sync-architecture.md", "data-job-earth-sync-architecture", "docs_developer", "Backend", 34, "数据作业与 Outbox 技术架构", "Data Jobs and Outbox Architecture"),
+    DocsMetadata("backend-enum-contracts.md", "backend-enum-contracts", "docs_developer", "Backend", 35, "后端枚举与字符串兼容契约", "Backend Enum and String Compatibility Contract"),
+    DocsMetadata("location-pipeline-development.md", "location-pipeline-development", "docs_developer", "Backend", 35, "通用位置估算管线开发说明", "Shared Location Resolution Pipeline Development Guide"),
+    DocsMetadata("earth-news-live-streams-collector-format.md", "earth-news-live-streams-collector-format", "docs_developer", "Backend", 36, "新闻直播采集格式", "News Live Streams Collector Format"),
+    DocsMetadata("docs-gatekeeper-development.md", "docs-gatekeeper-development", "docs_developer", "Backend", 37, "Docs Gatekeeper 开发说明", "Docs Gatekeeper Development Guide"),
     DocsMetadata("agents-aiprovider.md", "agents-aiprovider", "docs_developer", "Agents", 40, "AI Provider 指南", "AI Provider Guide"),
-    DocsMetadata("ops-runbook.md", "ops-runbook", "docs_admin", "Ops", 49, "Planet 运维手册", "Planet Ops Runbook"),
+    DocsMetadata("ops-runbook.md", "ops-runbook", "docs_admin", "Ops", 49, "智能星球运维手册", "Intelligent Planet Ops Runbook"),
     DocsMetadata("ops-docker-compose-buildx-upgrade.md", "ops-docker-compose-buildx-upgrade", "docs_admin", "Ops", 50, "Docker + Compose + Buildx 升级", "Docker + Compose + Buildx Upgrade"),
     DocsMetadata("ops-planet-sh-startup.md", "ops-planet-sh-startup", "docs_admin", "Ops", 51, "planet.sh 启动机制", "planet.sh Startup"),
 )
@@ -66,9 +73,9 @@ def get_user_gatekeeper_groups(user: User | None) -> set[str]:
         return set()
 
     role = user.role.value if hasattr(user.role, "value") else str(user.role or "")
-    if role == "super_admin":
+    if role == UserRole.SUPER_ADMIN.value:
         return {"docs_user", "docs_developer", "docs_admin"}
-    if role == "admin":
+    if role == UserRole.ADMIN.value:
         return {"docs_user", "docs_developer", "docs_admin"}
 
     groups = set()

backend/app/services/earth_boundaries.py

@@ -0,0 +1,671 @@
+"""Earth boundary static asset service."""
+
+from __future__ import annotations
+
+import asyncio
+import hashlib
+import json
+import shutil
+import sys
+from datetime import UTC, datetime
+from pathlib import Path
+from typing import Any
+from uuid import uuid4
+
+import httpx
+
+
+REPO_ROOT = Path(__file__).resolve().parents[3]
+SOURCE_OUTPUT_DIR = REPO_ROOT / "data/earth-boundary-sources"
+SOURCE_MANIFEST_PATH = SOURCE_OUTPUT_DIR / "manifest.json"
+BUILD_RESULT_PATH = SOURCE_OUTPUT_DIR / "build-result.json"
+BUILD_JOB_PATH = SOURCE_OUTPUT_DIR / "build-job.json"
+BOUNDARY_OUTPUT_DIR = REPO_ROOT / "frontend/public/earth/data/boundaries/v1"
+BOUNDARY_MANIFEST_PATH = BOUNDARY_OUTPUT_DIR / "manifest.json"
+PMTILES_ARTIFACT_PATH = (
+    REPO_ROOT / "frontend/public/earth/data/boundaries/earth-boundaries-china-pov-v1.pmtiles"
+)
+LEGACY_GEOJSON_PATH = REPO_ROOT / "frontend/public/earth/data/countries-admin0.min.geojson"
+POV_POLICY_PATH = REPO_ROOT / "config/earth-boundary-pov-policy.china-v1.json"
+LOCAL_CONFIG_PATH = REPO_ROOT / "config/earth-boundary-sources.local.json"
+EXAMPLE_CONFIG_PATH = REPO_ROOT / "config/earth-boundary-sources.example.json"
+
+BOUNDARY_SOURCE_KINDS = {
+    "earth_admin0_boundaries": "admin0-boundaries",
+    "earth_coastline": "coastline",
+    "earth_claim_lines": "claim-lines",
+}
+
+DEFAULT_PUBLIC_BOUNDARY_SOURCES = {
+    "earth_admin0_boundaries": {
+        "displayName": "Natural Earth Admin-0 Countries",
+        "sourceKind": "admin0-boundaries",
+        "endpoint": "https://raw.githubusercontent.com/nvkelso/natural-earth-vector/master/geojson/ne_10m_admin_0_countries.geojson",
+        "method": "GET",
+        "headers": {},
+        "auth_type": "none",
+        "license": "Natural Earth public domain",
+    },
+    "earth_coastline": {
+        "displayName": "Natural Earth Coastline",
+        "sourceKind": "coastline",
+        "endpoint": "https://raw.githubusercontent.com/nvkelso/natural-earth-vector/master/geojson/ne_10m_coastline.geojson",
+        "method": "GET",
+        "headers": {},
+        "auth_type": "none",
+        "license": "Natural Earth public domain",
+    },
+    "earth_claim_lines": {
+        "displayName": "Natural Earth Disputed Boundaries",
+        "sourceKind": "claim-lines",
+        "endpoint": "https://raw.githubusercontent.com/nvkelso/natural-earth-vector/master/geojson/ne_10m_admin_0_boundary_lines_disputed_areas.geojson",
+        "method": "GET",
+        "headers": {},
+        "auth_type": "none",
+        "license": "Natural Earth public domain",
+    },
+}
+
+BUILD_CONFIG = {
+    "builder": "scripts/build_earth_boundary_pmtiles.py",
+    "format": "pmtiles+mvt",
+    "production_target": "pmtiles-mvt",
+}
+
+
+class EarthBoundaryBuildError(RuntimeError):
+    def __init__(self, message: str, *, code: str = "build_failed", details: Any = None) -> None:
+        super().__init__(message)
+        self.code = code
+        self.details = details
+
+
+_build_job_lock = asyncio.Lock()
+_build_task: asyncio.Task | None = None
+_build_job_state: dict[str, Any] = {}
+
+
+def _utc_now_iso() -> str:
+    return datetime.now(UTC).isoformat()
+
+
+def _public_job_state() -> dict[str, Any]:
+    if _build_job_state:
+        return dict(_build_job_state)
+    return _read_json(BUILD_JOB_PATH)
+
+
+def get_boundary_build_status() -> dict[str, Any]:
+    return {"job": _public_job_state()}
+
+
+def _set_job_state(**updates: Any) -> dict[str, Any]:
+    global _build_job_state
+    current = dict(_build_job_state)
+    current.update(updates)
+    current["updated_at"] = _utc_now_iso()
+    _build_job_state = current
+    _write_json(BUILD_JOB_PATH, current)
+    return current
+
+
+def _append_job_log(message: str) -> None:
+    logs = list(_build_job_state.get("logs") or [])
+    logs.append({"time": _utc_now_iso(), "message": message})
+    _set_job_state(logs=logs[-40:])
+
+
+def _update_job_progress(progress: float, phase: str, message: str, **extra: Any) -> None:
+    bounded_progress = max(0, min(100, int(round(progress))))
+    _set_job_state(
+        status="running",
+        progress=bounded_progress,
+        phase=phase,
+        message=message,
+        **extra,
+    )
+
+
+def _read_json(path: Path) -> dict[str, Any]:
+    if not path.exists():
+        return {}
+    with path.open("r", encoding="utf-8") as f:
+        payload = json.load(f)
+    return payload if isinstance(payload, dict) else {}
+
+
+def _write_json(path: Path, payload: dict[str, Any]) -> None:
+    path.parent.mkdir(parents=True, exist_ok=True)
+    with path.open("w", encoding="utf-8") as f:
+        json.dump(payload, f, ensure_ascii=False, indent=2)
+        f.write("\n")
+
+
+def _sha256_bytes(payload: bytes) -> str:
+    return hashlib.sha256(payload).hexdigest()
+
+
+def _stable_json_hash(payload: Any) -> str:
+    encoded = json.dumps(payload, ensure_ascii=False, sort_keys=True, separators=(",", ":"))
+    return hashlib.sha256(encoded.encode("utf-8")).hexdigest()
+
+
+def _artifact_extension(endpoint: str, content_type: str, payload: bytes) -> str:
+    suffix = Path(endpoint.split("?", 1)[0]).suffix.lower()
+    if suffix in {".json", ".geojson", ".zip", ".pbf"}:
+        return suffix
+    if "geo+json" in content_type or b'"FeatureCollection"' in payload[:4096]:
+        return ".geojson"
+    if "json" in content_type:
+        return ".json"
+    return ".dat"
+
+
+def _json_feature_count(payload: Any) -> int:
+    if isinstance(payload, dict) and isinstance(payload.get("features"), list):
+        return len(payload["features"])
+    if isinstance(payload, list):
+        return len(payload)
+    return 1 if payload else 0
+
+
+def _directory_stats(path: Path) -> dict[str, int]:
+    if not path.exists():
+        return {"file_count": 0, "size_bytes": 0}
+    files = [item for item in path.rglob("*") if item.is_file()]
+    return {"file_count": len(files), "size_bytes": sum(item.stat().st_size for item in files)}
+
+
+def _load_source_feature_collection(source: dict[str, Any]) -> dict[str, Any]:
+    path = REPO_ROOT / source["path"]
+    payload = _read_json(path)
+    features = payload.get("features") if isinstance(payload, dict) else None
+    return {
+        "type": "FeatureCollection",
+        "features": features if isinstance(features, list) else [],
+    }
+
+
+def _write_high_precision_geojson_manifest(
+    sources: list[dict[str, Any]],
+    build_input_hash: str,
+    missing_tools: list[str],
+) -> dict[str, Any]:
+    BOUNDARY_OUTPUT_DIR.mkdir(parents=True, exist_ok=True)
+    admin0 = next(source for source in sources if source["kind"] == "admin0-boundaries")
+    coastline = next(source for source in sources if source["kind"] == "coastline")
+    claim_lines = next(source for source in sources if source["kind"] == "claim-lines")
+
+    admin0_payload = _load_source_feature_collection(admin0)
+    coastline_payload = _load_source_feature_collection(coastline)
+    claim_payload = _load_source_feature_collection(claim_lines)
+    for feature in coastline_payload["features"]:
+        props = feature.setdefault("properties", {})
+        if isinstance(props, dict):
+            props["PLANET_LAYER"] = "coastline"
+
+    base_payload = {
+        "type": "FeatureCollection",
+        "features": [*admin0_payload["features"], *coastline_payload["features"]],
+    }
+    base_path = BOUNDARY_OUTPUT_DIR / "earth-boundaries-high-precision-base.geojson"
+    hover_path = BOUNDARY_OUTPUT_DIR / "earth-boundaries-high-precision-hover.geojson"
+    claim_path = BOUNDARY_OUTPUT_DIR / "earth-boundaries-high-precision-claims.geojson"
+    _write_json(base_path, base_payload)
+    _write_json(hover_path, admin0_payload)
+    _write_json(claim_path, claim_payload)
+
+    manifest = {
+        "version": "natural-earth-v1",
+        "builtAt": _utc_now_iso(),
+        "tileProvider": "geojson-high-precision",
+        "format": "geojson-directory",
+        "buildInputHash": build_input_hash,
+        "base": base_path.name,
+        "hoverIndex": hover_path.name,
+        "claimLine": claim_path.name,
+        "sourceFeatureCount": {
+            "admin0": len(admin0_payload["features"]),
+            "coastline": len(coastline_payload["features"]),
+            "claimLines": len(claim_payload["features"]),
+        },
+        "pmtiles": None,
+        "missingTools": missing_tools,
+    }
+    _write_json(BOUNDARY_MANIFEST_PATH, manifest)
+    return manifest
+
+
+def _relative(path: Path) -> str:
+    return str(path.relative_to(REPO_ROOT))
+
+
+def load_boundary_config() -> tuple[dict[str, Any], str]:
+    if LOCAL_CONFIG_PATH.exists():
+        return _read_json(LOCAL_CONFIG_PATH), "local"
+    return _read_json(EXAMPLE_CONFIG_PATH), "example"
+
+
+def save_boundary_config(payload: dict[str, Any]) -> dict[str, Any]:
+    if not isinstance(payload, dict):
+        raise EarthBoundaryBuildError("Earth boundary config must be a JSON object", code="invalid_config")
+    _write_json(LOCAL_CONFIG_PATH, payload)
+    return get_boundary_status()
+
+
+def _source_configs(payload: dict[str, Any]) -> dict[str, Any]:
+    raw_sources = payload.get("collectorConfigs") or payload.get("sources") or {}
+    return raw_sources if isinstance(raw_sources, dict) else {}
+
+
+def _is_placeholder_endpoint(endpoint: Any) -> bool:
+    value = str(endpoint or "").strip()
+    return not value or "example.com" in value
+
+
+def _source_configs_with_defaults(payload: dict[str, Any]) -> dict[str, Any]:
+    raw_sources = _source_configs(payload)
+    merged: dict[str, Any] = {}
+    for source_key, default_config in DEFAULT_PUBLIC_BOUNDARY_SOURCES.items():
+        configured = raw_sources.get(source_key)
+        if not isinstance(configured, dict) or _is_placeholder_endpoint(configured.get("endpoint")):
+            merged[source_key] = dict(default_config)
+        else:
+            merged[source_key] = {**default_config, **configured}
+    for source_key, source_config in raw_sources.items():
+        if source_key not in merged:
+            merged[source_key] = source_config
+    return merged
+
+
+def _build_input_hash(source_manifest: dict[str, Any]) -> str:
+    return _stable_json_hash(
+        {
+            "source_manifest_schema": source_manifest.get("schema"),
+            "sources": [
+                {
+                    "id": source.get("id"),
+                    "sha256": source.get("sha256"),
+                    "kind": source.get("kind"),
+                }
+                for source in source_manifest.get("sources", [])
+            ],
+            "pov_policy": source_manifest.get("povPolicy"),
+            "build_config": BUILD_CONFIG,
+        }
+    )
+
+
+def _has_current_artifacts(boundary_manifest: dict[str, Any], build_input_hash: str) -> bool:
+    return (
+        bool(boundary_manifest)
+        and boundary_manifest.get("buildInputHash") == build_input_hash
+        and boundary_manifest.get("tileProvider") == "pmtiles-mvt"
+        and PMTILES_ARTIFACT_PATH.exists()
+    )
+
+
+def get_boundary_status() -> dict[str, Any]:
+    config_payload, config_source = load_boundary_config()
+    effective_source_configs = _source_configs_with_defaults(config_payload)
+    boundary_manifest = _read_json(BOUNDARY_MANIFEST_PATH)
+    pmtiles_exists = PMTILES_ARTIFACT_PATH.exists()
+    manifest_exists = BOUNDARY_MANIFEST_PATH.exists()
+    high_precision_ready = (
+        manifest_exists
+        and (
+            (
+                boundary_manifest.get("tileProvider") == "pmtiles-mvt"
+                and pmtiles_exists
+            )
+            or boundary_manifest.get("tileProvider") == "geojson-high-precision"
+        )
+    )
+    legacy_exists = LEGACY_GEOJSON_PATH.exists()
+    provider = (
+        boundary_manifest.get("tileProvider")
+        if high_precision_ready
+        else "legacy-geojson" if legacy_exists else "missing"
+    )
+    return {
+        "provider": provider,
+        "high_precision_ready": high_precision_ready,
+        "fallback_available": legacy_exists,
+        "config_source": config_source,
+        "config_path": _relative(LOCAL_CONFIG_PATH),
+        "config_exists": LOCAL_CONFIG_PATH.exists(),
+        "config": config_payload,
+        "effective_default_sources": [
+            source_key
+            for source_key, source_config in effective_source_configs.items()
+            if source_key in DEFAULT_PUBLIC_BOUNDARY_SOURCES
+            and source_config.get("endpoint") == DEFAULT_PUBLIC_BOUNDARY_SOURCES[source_key]["endpoint"]
+        ],
+        "manifest": {
+            "path": _relative(BOUNDARY_MANIFEST_PATH),
+            "exists": manifest_exists,
+            "tileProvider": boundary_manifest.get("tileProvider"),
+            "buildInputHash": boundary_manifest.get("buildInputHash"),
+            "builtAt": boundary_manifest.get("builtAt"),
+        },
+        "pmtiles": {
+            "path": _relative(PMTILES_ARTIFACT_PATH),
+            "exists": pmtiles_exists,
+            "size_bytes": PMTILES_ARTIFACT_PATH.stat().st_size if pmtiles_exists else 0,
+        },
+        "legacy": {
+            "path": _relative(LEGACY_GEOJSON_PATH),
+            "exists": legacy_exists,
+            "size_bytes": LEGACY_GEOJSON_PATH.stat().st_size if legacy_exists else 0,
+        },
+        "source_manifest": {
+            "path": _relative(SOURCE_MANIFEST_PATH),
+            "exists": SOURCE_MANIFEST_PATH.exists(),
+        },
+        "last_build": _read_json(BUILD_RESULT_PATH),
+        "current_job": _public_job_state(),
+    }
+
+
+async def _download_source(
+    source_key: str,
+    source_config: dict[str, Any],
+    progress_callback: Any = None,
+) -> dict[str, Any]:
+    endpoint = str(source_config.get("endpoint") or "").strip()
+    if _is_placeholder_endpoint(endpoint):
+        raise EarthBoundaryBuildError(
+            f"{source_key} endpoint is not configured",
+            code="source_not_configured",
+            details={"source": source_key},
+        )
+    method = str(source_config.get("method") or "GET").upper()
+    if method not in {"GET", "POST"}:
+        raise EarthBoundaryBuildError(
+            f"{source_key} method must be GET or POST",
+            code="invalid_config",
+            details={"source": source_key, "method": method},
+        )
+
+    if endpoint.startswith("file://") or Path(endpoint).expanduser().exists():
+        payload = Path(endpoint.removeprefix("file://")).expanduser().read_bytes()
+        content_type = "application/octet-stream"
+        if progress_callback:
+            progress_callback(1, len(payload), len(payload))
+    else:
+        timeout = float(source_config.get("timeout") or 120)
+        headers = source_config.get("headers") if isinstance(source_config.get("headers"), dict) else {}
+        async with httpx.AsyncClient(timeout=timeout, follow_redirects=True) as client:
+            async with client.stream(method, endpoint, headers=headers) as response:
+                response.raise_for_status()
+                content_type = response.headers.get("content-type", "")
+                total = int(response.headers.get("content-length") or 0)
+                chunks = []
+                downloaded = 0
+                async for chunk in response.aiter_bytes():
+                    if not chunk:
+                        continue
+                    chunks.append(chunk)
+                    downloaded += len(chunk)
+                    if progress_callback:
+                        progress_callback(
+                            (downloaded / total) if total else None,
+                            downloaded,
+                            total,
+                        )
+                payload = b"".join(chunks)
+
+    extension = _artifact_extension(endpoint, content_type, payload)
+    parsed: Any = None
+    if extension in {".json", ".geojson"}:
+        parsed = json.loads(payload.decode("utf-8"))
+    feature_count = _json_feature_count(parsed)
+    if feature_count <= 0:
+        raise EarthBoundaryBuildError(
+            f"{source_key} downloaded payload contains no features",
+            code="empty_source",
+            details={"source": source_key},
+        )
+
+    sha256 = _sha256_bytes(payload)
+    source_dir = SOURCE_OUTPUT_DIR / source_key
+    source_dir.mkdir(parents=True, exist_ok=True)
+    artifact_path = source_dir / f"{sha256}{extension}"
+    artifact_path.write_bytes(payload)
+    return {
+        "id": source_key,
+        "kind": source_config.get("sourceKind") or BOUNDARY_SOURCE_KINDS[source_key],
+        "path": _relative(artifact_path),
+        "sha256": sha256,
+        "featureCount": feature_count,
+        "license": source_config.get("license"),
+    }
+
+
+async def _run_step(args: list[str]) -> dict[str, Any]:
+    process = await asyncio.create_subprocess_exec(
+        sys.executable,
+        *args,
+        cwd=REPO_ROOT,
+        stdout=asyncio.subprocess.PIPE,
+        stderr=asyncio.subprocess.PIPE,
+    )
+    stdout_bytes, stderr_bytes = await process.communicate()
+    stdout = stdout_bytes.decode("utf-8", errors="replace").strip()
+    stderr = stderr_bytes.decode("utf-8", errors="replace").strip()
+    payload: dict[str, Any] = {"stdout": stdout, "stderr": stderr, "returncode": process.returncode}
+    last_line = stdout.splitlines()[-1:] or []
+    if last_line:
+        try:
+            payload["result"] = json.loads(last_line[0])
+        except json.JSONDecodeError:
+            payload["result"] = last_line[0]
+    if process.returncode != 0:
+        raise EarthBoundaryBuildError(
+            stderr or stdout or f"command failed: {' '.join(args)}",
+            code="build_command_failed",
+            details=payload,
+        )
+    return payload
+
+
+async def build_boundary_assets(progress_callback: Any = None) -> dict[str, Any]:
+    config_payload, config_source = load_boundary_config()
+
+    source_configs = _source_configs_with_defaults(config_payload)
+    missing = [source for source in BOUNDARY_SOURCE_KINDS if source not in source_configs]
+    if missing:
+        raise EarthBoundaryBuildError(
+            f"Missing Earth boundary source configs: {', '.join(missing)}",
+            code="missing_sources",
+            details={"missing": missing},
+        )
+
+    sources = []
+    source_keys = list(BOUNDARY_SOURCE_KINDS)
+    for index, source_key in enumerate(source_keys):
+        source_config = source_configs[source_key]
+        if not isinstance(source_config, dict):
+            raise EarthBoundaryBuildError(
+                f"{source_key} config must be an object",
+                code="invalid_config",
+                details={"source": source_key},
+            )
+        source_start = 8 + index * 18
+        source_end = source_start + 18
+        if progress_callback:
+            progress_callback(source_start, "download", f"正在下载 {source_key}")
+
+        def report_download_progress(ratio: float | None, downloaded: int, total: int) -> None:
+            if not progress_callback:
+                return
+            if ratio is None:
+                progress_callback(source_start + 8, "download", f"{source_key} 已下载 {downloaded} bytes")
+                return
+            progress_callback(
+                source_start + (source_end - source_start) * ratio,
+                "download",
+                f"{source_key} 下载 {int(ratio * 100)}%",
+                downloaded_bytes=downloaded,
+                total_bytes=total,
+            )
+
+        sources.append(await _download_source(source_key, source_config, report_download_progress))
+
+    source_manifest = {
+        "schema": "planet-earth-boundary-sources/v2",
+        "sources": sources,
+        "povPolicy": _read_json(POV_POLICY_PATH),
+    }
+    if progress_callback:
+        progress_callback(65, "manifest", "正在写入边界源 manifest")
+    _write_json(SOURCE_MANIFEST_PATH, source_manifest)
+    build_input_hash = _build_input_hash(source_manifest)
+
+    boundary_manifest = _read_json(BOUNDARY_MANIFEST_PATH)
+    build_skipped = _has_current_artifacts(boundary_manifest, build_input_hash)
+    missing_tools = [tool for tool in ("tippecanoe", "pmtiles") if shutil.which(tool) is None]
+    if missing_tools and not build_skipped:
+        if progress_callback:
+            progress_callback(82, "build", "缺少 PMTiles 工具，正在生成 GeoJSON 高清包")
+        boundary_manifest = _write_high_precision_geojson_manifest(
+            sources,
+            build_input_hash,
+            missing_tools,
+        )
+        result = {
+            "status": "built_geojson_fallback",
+            "code": "missing_tools",
+            "missing_tools": missing_tools,
+            "sources": sources,
+            "boundary_manifest": _relative(BOUNDARY_MANIFEST_PATH),
+            "manifest": boundary_manifest,
+        }
+        _write_json(BUILD_RESULT_PATH, result)
+        if progress_callback:
+            progress_callback(96, "finalize", "GeoJSON 高清国界包已生成")
+        return {**get_boundary_status(), "build": result}
+
+    if build_skipped:
+        if progress_callback:
+            progress_callback(96, "unchanged", "高精国界已是最新")
+        build_result = {
+            "status": "unchanged",
+            "reason": "source manifest and build config hash unchanged",
+            "buildInputHash": build_input_hash,
+        }
+    else:
+        if progress_callback:
+            progress_callback(72, "build", "正在构建 PMTiles/MVT")
+        admin0 = next(source for source in sources if source["kind"] == "admin0-boundaries")
+        coastline = next(source for source in sources if source["kind"] == "coastline")
+        claim_lines = next(source for source in sources if source["kind"] == "claim-lines")
+        build_result = await _run_step(
+            [
+                "scripts/build_earth_boundary_pmtiles.py",
+                "--admin0-source",
+                admin0["path"],
+                "--coastline-source",
+                coastline["path"],
+                "--claims-source",
+                claim_lines["path"],
+                "--output",
+                _relative(PMTILES_ARTIFACT_PATH),
+                "--manifest",
+                _relative(BOUNDARY_MANIFEST_PATH),
+                "--build-input-hash",
+                build_input_hash,
+                "--pov-policy",
+                _relative(POV_POLICY_PATH),
+            ]
+        )
+        if progress_callback:
+            progress_callback(95, "finalize", "正在校验构建产物")
+
+    boundary_manifest = _read_json(BOUNDARY_MANIFEST_PATH)
+    boundary_stats = _directory_stats(BOUNDARY_OUTPUT_DIR)
+    result = {
+        "status": "unchanged" if build_skipped else "built",
+        "sources": sources,
+        "source_manifest": _relative(SOURCE_MANIFEST_PATH),
+        "boundary_manifest": _relative(BOUNDARY_MANIFEST_PATH),
+        "pmtiles_artifact": _relative(PMTILES_ARTIFACT_PATH),
+        "pmtiles_exists": PMTILES_ARTIFACT_PATH.exists(),
+        "boundary_stats": boundary_stats,
+        "manifest": boundary_manifest,
+        "build_result": build_result,
+    }
+    _write_json(BUILD_RESULT_PATH, result)
+    return {**get_boundary_status(), "build": result}
+
+
+async def _run_boundary_build_job(job_id: str) -> None:
+    def report(progress: float, phase: str, message: str, **extra: Any) -> None:
+        if _build_job_state.get("id") != job_id:
+            return
+        _update_job_progress(progress, phase, message, **extra)
+
+    try:
+        report(3, "prepare", "正在准备高精国界构建")
+        result = await build_boundary_assets(report)
+        _set_job_state(
+            id=job_id,
+            status="succeeded",
+            progress=100,
+            phase="complete",
+            message="高精国界构建完成",
+            finished_at=_utc_now_iso(),
+            result={
+                "provider": result.get("provider"),
+                "high_precision_ready": result.get("high_precision_ready"),
+                "pmtiles": result.get("pmtiles"),
+                "manifest": result.get("manifest"),
+            },
+        )
+        _append_job_log("高精国界构建完成")
+    except EarthBoundaryBuildError as exc:
+        _set_job_state(
+            id=job_id,
+            status="failed",
+            progress=_build_job_state.get("progress", 0),
+            phase="failed",
+            message=str(exc),
+            code=exc.code,
+            details=exc.details,
+            finished_at=_utc_now_iso(),
+        )
+        _append_job_log(str(exc))
+    except Exception as exc:  # pragma: no cover - defensive guard for background task
+        _set_job_state(
+            id=job_id,
+            status="failed",
+            progress=_build_job_state.get("progress", 0),
+            phase="failed",
+            message=str(exc),
+            code="build_failed",
+            finished_at=_utc_now_iso(),
+        )
+        _append_job_log(str(exc))
+
+
+async def start_boundary_build_job() -> dict[str, Any]:
+    global _build_task
+    async with _build_job_lock:
+        if _build_task and not _build_task.done():
+            return {"accepted": False, "job": _public_job_state()}
+        job_id = uuid4().hex
+        _set_job_state(
+            id=job_id,
+            status="queued",
+            progress=0,
+            phase="queued",
+            message="高精国界构建已加入队列",
+            logs=[],
+            started_at=_utc_now_iso(),
+            finished_at=None,
+            code=None,
+            details=None,
+        )
+        _append_job_log("高精国界构建已启动")
+        _build_task = asyncio.create_task(_run_boundary_build_job(job_id))
+        return {"accepted": True, "job": _public_job_state()}

backend/app/services/earth_db_change_listener.py

@@ -0,0 +1,497 @@
+"""PostgreSQL LISTEN/NOTIFY bridge for Earth layer refresh events."""
+
+from __future__ import annotations
+
+import asyncio
+import json
+from collections import deque
+from dataclasses import dataclass, field
+from datetime import UTC, datetime
+from time import monotonic
+from typing import Any, Awaitable, Callable
+
+import asyncpg
+
+from app.core.config import settings
+from app.core.logging import get_logger
+from app.core.time import to_iso8601_utc
+from app.core.websocket.broadcaster import broadcaster
+from app.services.earth_layer_adapters import (
+    get_earth_refresh_strategy_for_change,
+    get_earth_update_layers_for_change,
+    get_earth_update_layers_for_source,
+)
+from app.services.earth_layer_cache import invalidate_earth_layer_cache_for_source
+
+logger = get_logger(__name__)
+
+EARTH_DATA_CHANGES_CHANNEL = "planet_earth_data_changes"
+DEFAULT_DEBOUNCE_SECONDS = 0.25
+DEFAULT_MAX_WAIT_SECONDS = 1.5
+DELETE_FAST_FLUSH_SECONDS = 0.05
+LISTEN_KEEPALIVE_SECONDS = 5.0
+OUTBOX_POLL_LIMIT = 5000
+MAX_ENTITY_KEY_SAMPLES = 20
+MAX_SEEN_EVENT_IDS = 20000
+
+BroadcastFn = Callable[[dict[str, Any]], Awaitable[None]]
+InvalidateFn = Callable[[str], int]
+
+
+def normalize_asyncpg_dsn(dsn: str) -> str:
+    """Convert SQLAlchemy asyncpg URLs into asyncpg-compatible URLs."""
+    return dsn.replace("postgresql+asyncpg://", "postgresql://", 1)
+
+
+def build_earth_update_from_db_payload(payload: dict[str, Any]) -> dict[str, Any] | None:
+    table = payload.get("table")
+    source = str(payload.get("source") or "").strip()
+    table_name = str(table or "").strip()
+    if not source and not table_name:
+        return None
+    layers = get_earth_update_layers_for_change(table_name, source)
+    if not layers:
+        return None
+    refresh_strategy = get_earth_refresh_strategy_for_change(table_name, source) or "clear_then_reload"
+    source_has_adapter = bool(get_earth_update_layers_for_source(source))
+    effective_source = source if source_has_adapter else (table_name if table_name else source)
+    operation = payload.get("operation")
+    if "vessels" in layers and operation in {"DELETE", "TRUNCATE"}:
+        refresh_strategy = "reload"
+    update: dict[str, Any] = {
+        "event": "earth.layer.changed",
+        "action": "database_changed",
+        "source": effective_source,
+        "original_source": source or None,
+        "table": table_name or None,
+        "data_type": effective_source,
+        "layers": layers,
+        "refresh_strategy": refresh_strategy,
+        "operation": operation,
+        "entity_key": payload.get("entity_key"),
+        "timestamp": to_iso8601_utc(datetime.now(UTC)),
+    }
+    if table_name == "earth_interactables" and refresh_strategy == "delta":
+        ids = payload.get("entity_keys")
+        if not isinstance(ids, list):
+            ids = [payload.get("entity_key")] if payload.get("entity_key") else []
+        update.update(
+            {
+                "entity": "interactable",
+                "action": "deleted" if operation == "DELETE" else "changed",
+                "ids": [str(item) for item in ids if item],
+                "item": None,
+            }
+        )
+    return {
+        **update,
+    }
+
+
+@dataclass
+class PendingEarthDbChange:
+    source: str
+    layers: list[str]
+    table: str | None = None
+    refresh_strategy: str = "clear_then_reload"
+    entity: str | None = None
+    action: str = "database_changed"
+    records_processed: int = 0
+    operations: set[str] = field(default_factory=set)
+    entity_keys: list[str] = field(default_factory=list)
+    first_occurred_at: str | None = None
+    last_occurred_at: str | None = None
+    first_seen_monotonic: float = field(default_factory=monotonic)
+    last_seen_monotonic: float = field(default_factory=monotonic)
+
+    def add(self, payload: dict[str, Any]) -> None:
+        self.last_seen_monotonic = monotonic()
+        records_processed = payload.get("records_processed", 1)
+        try:
+            records_processed = int(records_processed)
+        except (TypeError, ValueError):
+            records_processed = 1
+        self.records_processed += max(records_processed, 1)
+        operation = payload.get("operation")
+        if operation:
+            self.operations.add(str(operation))
+            if "vessels" in self.layers and operation in {"DELETE", "TRUNCATE"}:
+                self.refresh_strategy = "reload"
+        entity_keys = payload.get("entity_keys")
+        if not isinstance(entity_keys, list):
+            entity_key = payload.get("entity_key")
+            entity_keys = [entity_key] if entity_key else []
+        for entity_key in entity_keys:
+            if entity_key and len(self.entity_keys) < MAX_ENTITY_KEY_SAMPLES:
+                self.entity_keys.append(str(entity_key))
+        occurred_at = payload.get("occurred_at")
+        if occurred_at:
+            occurred_at = str(occurred_at)
+            self.first_occurred_at = self.first_occurred_at or occurred_at
+            self.last_occurred_at = occurred_at
+
+
+class EarthDbChangeDispatcher:
+    """Debounces database notifications and broadcasts Earth refresh hints."""
+
+    def __init__(
+        self,
+        *,
+        broadcast_earth_update: BroadcastFn | None = None,
+        invalidate_cache: InvalidateFn | None = None,
+        debounce_seconds: float = DEFAULT_DEBOUNCE_SECONDS,
+        max_wait_seconds: float = DEFAULT_MAX_WAIT_SECONDS,
+    ) -> None:
+        self._broadcast_earth_update = broadcast_earth_update or broadcaster.broadcast_earth_update
+        self._invalidate_cache = invalidate_cache or invalidate_earth_layer_cache_for_source
+        self._debounce_seconds = debounce_seconds
+        self._max_wait_seconds = max(max_wait_seconds, debounce_seconds)
+        self._pending: dict[str, PendingEarthDbChange] = {}
+        self._flush_tasks: dict[str, asyncio.Task[None]] = {}
+        self._seen_event_ids: set[int] = set()
+        self._seen_event_order: deque[int] = deque()
+
+    def handle_notification(self, payload_text: str) -> bool:
+        try:
+            payload = json.loads(payload_text)
+        except json.JSONDecodeError:
+            logger.warning_event(
+                "Ignoring malformed Earth database change notification",
+                event="earth.db_changes.notification_malformed",
+            )
+            return False
+        if not isinstance(payload, dict):
+            return False
+        return self.handle_payload(payload)
+
+    def handle_payload(self, payload: dict[str, Any]) -> bool:
+        event_id = payload.get("event_id")
+        if event_id is not None:
+            try:
+                normalized_event_id = int(event_id)
+            except (TypeError, ValueError):
+                normalized_event_id = None
+            if normalized_event_id is not None:
+                if normalized_event_id in self._seen_event_ids:
+                    return False
+                self._remember_event_id(normalized_event_id)
+
+        update = build_earth_update_from_db_payload(payload)
+        if not update:
+            return False
+
+        if payload.get("table") == "vessel_current_state" and payload.get("operation") == "DELETE":
+            from app.core.websocket.broadcaster import broadcaster
+
+            keys = payload.get("entity_keys")
+            if not isinstance(keys, list):
+                keys = [payload.get("entity_key")]
+            broadcaster.enqueue_vessel_update({
+                "action": "remove",
+                "vessels": [{"mmsi": key} for key in keys if key is not None],
+            })
+
+        source = update["source"]
+        pending = self._pending.get(source)
+        if pending is None:
+            pending = PendingEarthDbChange(
+                source=source,
+                layers=list(update["layers"]),
+                table=update.get("table"),
+                refresh_strategy=str(update.get("refresh_strategy") or "clear_then_reload"),
+                entity=update.get("entity"),
+                action=str(update.get("action") or "database_changed"),
+            )
+            self._pending[source] = pending
+        pending.add(payload)
+
+        task = self._flush_tasks.pop(source, None)
+        if task and not task.done():
+            task.cancel()
+        self._flush_tasks[source] = asyncio.create_task(
+            self._flush_later(source, delay_seconds=self._next_flush_delay(pending))
+        )
+        return True
+
+    def _next_flush_delay(self, pending: PendingEarthDbChange) -> float:
+        if "DELETE" in pending.operations and pending.refresh_strategy == "clear_then_reload":
+            return DELETE_FAST_FLUSH_SECONDS
+        elapsed = max(0.0, monotonic() - pending.first_seen_monotonic)
+        remaining = self._max_wait_seconds - elapsed
+        if remaining <= 0:
+            return 0.0
+        return min(self._debounce_seconds, remaining)
+
+    def _remember_event_id(self, event_id: int) -> None:
+        self._seen_event_ids.add(event_id)
+        self._seen_event_order.append(event_id)
+        while len(self._seen_event_order) > MAX_SEEN_EVENT_IDS:
+            expired_event_id = self._seen_event_order.popleft()
+            self._seen_event_ids.discard(expired_event_id)
+
+    async def _flush_later(self, source: str, *, delay_seconds: float) -> None:
+        try:
+            if delay_seconds > 0:
+                await asyncio.sleep(delay_seconds)
+            await self.flush_source(source)
+        except asyncio.CancelledError:
+            raise
+        except Exception as exc:
+            logger.exception_event(
+                "Failed to broadcast debounced Earth database change",
+                event="earth.db_changes.flush_failed",
+                context={"source": source, "error": str(exc)},
+            )
+        finally:
+            current = self._flush_tasks.get(source)
+            if current is asyncio.current_task():
+                self._flush_tasks.pop(source, None)
+
+    async def flush_source(self, source: str) -> None:
+        pending = self._pending.get(source)
+        if pending is None:
+            return
+
+        flushed_at = datetime.now(UTC)
+        deleted_cache_entries = self._invalidate_cache(source)
+        operations = sorted(pending.operations)
+        payload: dict[str, Any] = {
+            "event": "earth.layer.changed",
+            "action": "database_changed",
+            "source": source,
+            "table": pending.table,
+            "data_type": source,
+            "layers": pending.layers,
+            "refresh_strategy": pending.refresh_strategy,
+            "records_processed": pending.records_processed,
+            "operations": operations,
+            "operation": operations[-1] if len(operations) == 1 else None,
+            "entity_keys": pending.entity_keys,
+            "entity_key_sample_size": len(pending.entity_keys),
+            "cache_entries_invalidated": deleted_cache_entries,
+            "first_occurred_at": pending.first_occurred_at,
+            "last_occurred_at": pending.last_occurred_at,
+            "debounce_ms": int((monotonic() - pending.first_seen_monotonic) * 1000),
+            "total_latency_ms": self._total_latency_ms(pending, flushed_at),
+            "timestamp": to_iso8601_utc(flushed_at),
+        }
+        if pending.entity == "interactable":
+            payload.update(
+                {
+                    "entity": "interactable",
+                    "action": "deleted" if "DELETE" in pending.operations else "changed",
+                    "ids": pending.entity_keys,
+                    "item": None,
+                }
+            )
+        await self._broadcast_earth_update(payload)
+        self._pending.pop(source, None)
+        logger.info_event(
+            "Broadcasted Earth database change",
+            event="earth.db_changes.broadcasted",
+            context={
+                "source": source,
+                "layers": pending.layers,
+                "records_processed": pending.records_processed,
+                "cache_entries_invalidated": deleted_cache_entries,
+                "debounce_ms": int((monotonic() - pending.first_seen_monotonic) * 1000),
+                "total_latency_ms": payload["total_latency_ms"],
+            },
+        )
+
+    @staticmethod
+    def _total_latency_ms(pending: PendingEarthDbChange, flushed_at: datetime) -> int | None:
+        occurred_at = pending.first_occurred_at
+        if not occurred_at:
+            return None
+        try:
+            normalized = occurred_at.replace("Z", "+00:00")
+            occurred = datetime.fromisoformat(normalized)
+            if occurred.tzinfo is None:
+                occurred = occurred.replace(tzinfo=UTC)
+            return max(0, int((flushed_at - occurred.astimezone(UTC)).total_seconds() * 1000))
+        except ValueError:
+            return None
+
+    async def flush_all(self) -> None:
+        sources = list(self._pending)
+        for source in sources:
+            task = self._flush_tasks.pop(source, None)
+            if task and not task.done():
+                task.cancel()
+            await self.flush_source(source)
+
+    async def stop(self) -> None:
+        tasks = [task for task in self._flush_tasks.values() if not task.done()]
+        for task in tasks:
+            task.cancel()
+        if tasks:
+            await asyncio.gather(*tasks, return_exceptions=True)
+        self._flush_tasks.clear()
+        await self.flush_all()
+
+
+class EarthDbChangeListener:
+    def __init__(
+        self,
+        *,
+        dsn: str,
+        dispatcher: EarthDbChangeDispatcher,
+        channel: str = EARTH_DATA_CHANGES_CHANNEL,
+    ) -> None:
+        self._dsn = normalize_asyncpg_dsn(dsn)
+        self._dispatcher = dispatcher
+        self._channel = channel
+        self._task: asyncio.Task[None] | None = None
+        self._stop_event: asyncio.Event | None = None
+        self._connection: asyncpg.Connection | None = None
+        self._loop: asyncio.AbstractEventLoop | None = None
+
+    def start(self) -> None:
+        if self._task and not self._task.done():
+            return
+        self._loop = asyncio.get_running_loop()
+        self._stop_event = asyncio.Event()
+        self._task = asyncio.create_task(self._run())
+
+    async def stop(self) -> None:
+        if self._stop_event:
+            self._stop_event.set()
+        if self._connection:
+            await self._connection.close()
+        if self._task:
+            await asyncio.gather(self._task, return_exceptions=True)
+        await self._dispatcher.stop()
+
+    async def _run(self) -> None:
+        backoff_seconds = 1.0
+        assert self._stop_event is not None
+
+        while not self._stop_event.is_set():
+            try:
+                self._connection = await asyncpg.connect(self._dsn)
+                await self._connection.add_listener(self._channel, self._on_notification)
+                logger.info_event(
+                    "Earth database change listener connected",
+                    event="earth.db_changes.connected",
+                    context={"channel": self._channel},
+                )
+                backoff_seconds = 1.0
+                while not self._stop_event.is_set():
+                    try:
+                        await asyncio.wait_for(
+                            self._stop_event.wait(),
+                            timeout=LISTEN_KEEPALIVE_SECONDS,
+                        )
+                    except asyncio.TimeoutError:
+                        await self._poll_outbox()
+            except asyncio.CancelledError:
+                raise
+            except Exception as exc:
+                logger.exception_event(
+                    "Earth database change listener failed",
+                    event="earth.db_changes.listener_failed",
+                    context={"channel": self._channel, "error": str(exc)},
+                )
+                try:
+                    await asyncio.wait_for(self._stop_event.wait(), timeout=backoff_seconds)
+                except asyncio.TimeoutError:
+                    pass
+                backoff_seconds = min(backoff_seconds * 2, 30.0)
+            finally:
+                if self._connection:
+                    try:
+                        await self._connection.remove_listener(self._channel, self._on_notification)
+                    except Exception:
+                        pass
+                    try:
+                        await self._connection.close()
+                    except Exception:
+                        pass
+                    self._connection = None
+
+    async def _poll_outbox(self) -> None:
+        if self._connection is None:
+            return
+
+        rows = await self._connection.fetch(
+            """
+            SELECT id, payload
+            FROM earth_data_change_events
+            WHERE consumed_at IS NULL
+            ORDER BY id
+            LIMIT $1
+            """,
+            OUTBOX_POLL_LIMIT,
+        )
+        accepted_count = 0
+        consumed_ids: list[int] = []
+        for row in rows:
+            payload = row["payload"]
+            if isinstance(payload, str):
+                try:
+                    payload = json.loads(payload)
+                except json.JSONDecodeError:
+                    consumed_ids.append(int(row["id"]))
+                    continue
+            if isinstance(payload, dict):
+                if self._dispatcher.handle_payload(payload):
+                    accepted_count += 1
+                consumed_ids.append(int(row["id"]))
+            else:
+                consumed_ids.append(int(row["id"]))
+        if consumed_ids:
+            await self._dispatcher.flush_all()
+        if consumed_ids:
+            await self._connection.execute(
+                """
+                UPDATE earth_data_change_events
+                SET consumed_at = NOW()
+                WHERE id = ANY($1::bigint[])
+                  AND consumed_at IS NULL
+                """,
+                consumed_ids,
+            )
+        if rows:
+            logger.info_event(
+                "Polled Earth database change outbox",
+                event="earth.db_changes.outbox_polled",
+                context={"events": len(rows), "accepted": accepted_count},
+            )
+
+    def _on_notification(
+        self,
+        _connection: asyncpg.Connection,
+        _pid: int,
+        _channel: str,
+        payload: str,
+    ) -> None:
+        if self._loop and self._loop.is_running():
+            self._loop.call_soon_threadsafe(self._dispatcher.handle_notification, payload)
+            return
+        self._dispatcher.handle_notification(payload)
+
+
+_dispatcher = EarthDbChangeDispatcher(
+    broadcast_earth_update=broadcaster.broadcast_earth_update,
+    invalidate_cache=invalidate_earth_layer_cache_for_source,
+)
+_listener: EarthDbChangeListener | None = None
+
+
+def start_earth_db_change_listener() -> None:
+    global _listener
+    if _listener is not None:
+        return
+    _listener = EarthDbChangeListener(dsn=settings.DATABASE_URL, dispatcher=_dispatcher)
+    _listener.start()
+
+
+async def stop_earth_db_change_listener() -> None:
+    global _listener
+    if _listener is None:
+        await _dispatcher.stop()
+        return
+    listener = _listener
+    _listener = None
+    await listener.stop()