release: bump version to 0.76.19

.claude/commands/cleanup.md

@@ -1,139 +0,0 @@
----
-description: 审查当前工作区未提交代码中的垃圾代码，并在不影响逻辑的前提下自动清理
-argument-hint: 可选：指定要检查的文件或目录（默认检查所有未提交修改）
-allowed-tools: ["Read", "Edit", "Bash", "Grep", "Glob"]
----
-
-# /cleanup — 垃圾代码审查与清理
-
-分析当前工作区（git diff）中的未提交代码，找出并修复常见垃圾代码，**不得改变任何运行逻辑**。
-
-## 检查范围
-
-若 `$ARGUMENTS` 非空，则只检查指定文件/目录；否则检查所有未提交修改（`git diff HEAD`）。
-
-## 节省上下文规则
-
-优先用确定性的 CLI 检查缩小范围，不要一上来把完整文件或大 diff 读入上下文：
-
-```bash
-git diff --name-only HEAD
-git diff --unified=0 HEAD -- <path>
-git diff --check
-rg -n "TODO|FIXME|console\.log|debugger|print\(" <changed-paths>
-```
-
-只有 focused diff 不足以安全判断或修改时，才读取完整文件。
-
-## 审查清单
-
-按优先级检查以下问题（只报告在本次 diff 中**新增或修改**的代码里存在的问题）：
-
-### 1. 重复逻辑 (Duplicate Logic)
-- 完全相同或高度相似的代码块在多处出现
-- 同一函数/方法被多个地方各自实现，已有公共版本未被复用
-- 相同的 DOM 查询、正则、模板字符串在同一文件重复
-
-### 2. Magic Numbers / Magic Strings
-- 裸数字直接参与计算（如偏移量、时间、尺寸、阈值），没有命名常量
-- 硬编码字符串（如 id 名、状态值、URL 片段）散落在逻辑中
-- 例外：`0`, `1`, `-1`, `100`, `""` 等语义明确的惯用值不算
-
-### 3. 命名问题
-- 含义不明的缩写变量（如 `or_`, `tmp2`, `x2`）
-- 命名与实际用途不符
-- 同一概念在不同地方用不同名字表达
-
-### 4. 死代码 / 无效代码
-- 注释掉的旧代码块（3行以上）
-- 声明后从未使用的变量/参数/导入
-- 永远不会执行的条件分支
-
-### 5. 代码风格问题
-- 尾部空白字符（trailing whitespace）
-- 同一文件内风格不一致（如混用单双引号、缩进不统一）
-- 空行使用不一致（连续多个空行等）
-
-### 6. 其他常见问题
-- 私有辅助函数应被 export 但没有，导致调用方重复实现
-- 类型/接口重复定义
-- 过于冗长的条件表达式可以简化（不改逻辑）
-
-## 执行步骤
-
-### Step 1 — 获取待检查文件列表
-
-```bash
-# 无参数时：获取所有未提交修改
-git diff HEAD --name-only
-
-# 有参数时：用 $ARGUMENTS 过滤
-```
-
-### Step 2 — 逐文件阅读并分析
-
-先从 focused diff 开始：
-
-```bash
-git diff --unified=0 HEAD -- <file>
-```
-
-用 `rg`、`git diff --check`、编译器或 linter 输出确认确定性问题。只有需要上下文时才用 Read 读取完整文件。对照审查清单，记录每个问题：文件名、行号、问题类型、建议修复方式。
-
-### Step 3 — 报告问题清单
-
-在修改前，先以列表形式输出所有发现的问题：
-
-```
-发现 N 个问题：
-
-[文件] js/foo.js
-  · L34, L78: 重复逻辑 — 两处都实现了相同的 DOM 查询，可提取到 getPanel()
-  · L91: Magic number — 硬编码 14 作为偏移量，应命名为 TOOLTIP_OFFSET
-
-[文件] js/bar.js
-  · L12: 命名问题 — 变量 `or_` 语义不明，应命名为 outerR/outerG/outerB
-  ...
-```
-
-如果没有发现问题，直接输出"未发现垃圾代码，当前代码质量良好。"并停止。
-
-### Step 4 — 执行修复
-
-对每个问题，使用 Edit 工具进行**最小化修改**：
-
-- **重复逻辑**：提取为共享常量/函数，更新所有调用点
-- **Magic number**：在文件顶部或逻辑附近声明 `const NAME = value`，替换所有引用
-- **命名问题**：重命名变量，更新所有使用处
-- **死代码**：直接删除
-- **尾部空白/风格**：修正
-- **未 export 的函数**：添加 `export`，在调用方改为导入（不重复实现）
-
-**修复原则：**
-- 只改在审查清单中发现的问题，不做额外优化
-- 每次 Edit 只修改确实有问题的行，保持 diff 最小
-- 改完后用 `grep` 验证旧的坏代码已消失
-- 优先做精确补丁；只有仓库已有对应格式化流程时，才运行格式化工具
-
-### Step 5 — 输出总结
-
-```
-清理完成：
-
-修复了 N 个问题：
-  ✓ earth.js — 提取重复 vertexShader 为 ATMOS_VERTEX_SHADER 常量
-  ✓ main.js — 提取 TOOLTIP_CURSOR_OFFSET = 14（4处引用）
-  ✓ controls.js — export updateLayerButtonState，移除 main.js 中的重复实现
-  ...
-
-未修改的问题（需人工确认）：
-  ! foo.js L45 — 注释代码块较长，建议手动确认是否可删除
-```
-
-## 约束
-
-- **禁止**改变函数签名、接口定义、导出 API（除非问题正是私有函数应被 export）
-- **禁止**添加新功能、新抽象、新参数
-- **禁止**修改注释内容（只删除注释掉的死代码）
-- **禁止**修改测试文件逻辑
-- 如果一个 Magic number 的语义不完全确定，**跳过**，在总结中标记为"需人工确认"

.claude/commands/docs.md

@@ -1,104 +0,0 @@
----
-description: Create or update repository documentation from current code changes
-argument-hint: Optional: topic to document, or leave empty to infer from git diff
-allowed-tools: ["Read", "Edit", "Write", "Bash", "Glob", "Grep"]
----
-
-# /docs — Documentation Workflow
-
-## Goal
-
-Create or update documentation that explains why a change exists, how it behaves, and what maintainers need to know. Keep this command generic. Repository-specific coverage rules live in the repository and must be loaded separately.
-
-## Repository Rules
-
-Before deciding scope, check whether the repository has a documentation rules file:
-
-```bash
-test -f docs/documentation-coverage-rules.md && sed -n '1,240p' docs/documentation-coverage-rules.md
-```
-
-If it exists, apply it as the project-specific coverage checklist. If it does not exist, continue with the generic workflow below.
-
-## Workflow
-
-### Step 1 — Understand The Change
-
-```bash
-git diff HEAD --stat
-git diff HEAD --name-only
-git log --oneline -10
-rg --files docs
-```
-
-If `$ARGUMENTS` specifies a topic, focus on that topic. Otherwise infer the documentation topic from the changed files. Do not read the full repository diff by default; inspect focused files only:
-
-```bash
-git diff HEAD -- <path>
-rg -n "class |def |function |export |router|@router|interface |type " <path>
-```
-
-### Step 2 — Decide Scope
-
-- Prefer updating an existing relevant document over creating a duplicate.
-- Use one document for one coherent topic.
-- Split documents only when the change crosses meaningful domains.
-- Keep filenames lowercase and hyphenated.
-- Apply the repository-specific rules file before writing.
-
-#### Document Audience Routing (Planet)
-
-In this repository, classify the action's performer before picking a target file:
-
-- Browser/UI end user → `docs/technical/{zh,en}/manual.md` or `quickstart.md`.
-- Shell / Docker / log paths / `planet.sh` / SMTP fallbacks / port forwarding → `docs/technical/{zh,en}/ops-runbook.md` (or an existing `ops-*.md`).
-- Second-party developers → existing `*-context.md` / `backend-*.md` / `earth-*.md` files.
-
-Never put shell commands, log paths, or Docker operations into `manual.md` / `quickstart.md`. Never put UI button labels or screenshots into `ops-*.md`. When the same action has both a UI and a CLI path, write each in its own home and cross-link them with one sentence.
-
-For ambiguous or large documentation changes, briefly state the intended doc plan before editing. For clear small changes, proceed directly.
-
-### Step 3 — Write
-
-Explain:
-
-- Background/problem: what was wrong or missing before.
-- Core design decisions and rationale.
-- Operational or user-facing impact.
-- Relevant code paths, only when useful for future maintainers.
-
-Style:
-
-- Follow the repository’s existing language and heading conventions.
-- Use fenced code blocks with language tags.
-- Prefer tables for comparisons or parameter lists.
-- Keep snippets concise and relevant.
-- For UI labels, chart labels, feature names, datasource names, and other terms that may become mixed Chinese/English copy, check `docs/technical/{zh,en}/naming-glossary.md` and use the documented display name. If a confusing term is missing, update the glossary in both languages as part of the docs change.
-
-### Step 4 — Verify
-
-- Read the completed docs once for clarity and stale statements.
-- Verify referenced paths exist with `test -e` or `rg --files`.
-- Run applicable checks from `docs/documentation-coverage-rules.md`.
-- Check Markdown links use readable user-facing titles unless repository rules allow otherwise.
-
-### Step 5 — Report
-
-Summarize changed docs and verification:
-
-```md
-Updated:
-- path/to/doc.md — what changed
-
-Verified:
-- checks that passed
-- checks that could not be run, if any
-```
-
-## Hard Constraints
-
-- Do not leave placeholder docs.
-- Do not duplicate bilingual files byte-for-byte.
-- Do not reference PR numbers, issue numbers, or the current conversation unless explicitly requested.
-- Do not write changelog-style lists without the reasoning and tradeoffs behind the change.
-- Keep docs maintainable and concise.

.claude/commands/goal-driven.md

@@ -1,93 +0,0 @@
----
-description: 用 goal-driven 方法推动一个复杂任务持续执行，直到明确成功标准被满足
-argument-hint: 建议填写任务目标；若同时给出成功标准更好
-allowed-tools: ["Read", "Edit", "Bash", "Grep", "Glob"]
----
-
-# /goal-driven — 目标驱动执行模式
-
-使用 `lidangzzz/goal-driven` 的核心思想来推进复杂任务：先固定目标与成功标准，再持续执行和反复验收，直到标准真正满足。
-
-适用场景：
-
-- 长周期实现任务
-- 高复杂度工程任务
-- 可被明确验收的研究、实现、迁移、验证类工作
-
-不适用场景：
-
-- 纯脑暴
-- 无法定义成功标准的模糊任务
-- 很小的一次性修改
-
-## 输入要求
-
-若 `$ARGUMENTS` 只包含目标，没有成功标准，先补全一版可执行的成功标准再开始。
-
-启动时先输出：
-
-```md
-Goal
-- ...
-
-Criteria for success
-- ...
-
-Plan
-1. ...
-2. ...
-3. ...
-
-Verification
-- ...
-```
-
-## 执行规则
-
-1. 先把任务固化为两个核心块：
-   - `Goal`
-   - `Criteria for success`
-
-2. 成功标准必须尽量客观，可验证，可落地。
-   优先写成：
-   - 需要交付什么
-   - 需要通过哪些测试或验证
-   - 如何判断结果真的完成
-
-3. 进入持续执行循环：
-   - 完成一个阶段
-   - 检查当前结果是否满足成功标准
-   - 若未满足，明确剩余差距并继续推进
-
-4. 任何“完成了”“差不多了”“已实现”之类的结论，都必须经过验证，不能直接接受。
-
-5. 如果验证失败：
-   - 明确指出哪条成功标准没满足
-   - 继续工作，不要把阶段性进展误判为完成
-
-6. 只有在以下情况之一才能停止：
-   - 成功标准已满足
-   - 用户明确要求停止
-
-## 执行风格
-
-- 重证据，轻口头判断
-- 优先使用确定性工具证据：`rg`、`git diff --stat`、`git diff -- <path>`、测试、构建、lint、`curl`、数据库查询等能直接证明成功标准的方式
-- 不把大段命令输出粘进回复；保留在工具调用里，回复只总结关键证据
-- 重验收，轻自我感觉
-- 优先用测试、日志、产物、对比结果来证明完成
-- 对长期任务保持“未达标就继续”的节奏
-
-## 简版模板
-
-```md
-Goal: [[[[[在此填写最终目标]]]]]
-
-Criteria for success: [[[[[在此填写成功标准]]]]]
-
-循环执行：
-1. 推进任务
-2. 检查是否满足成功标准
-3. 若未满足，继续工作
-4. 直到满足标准或用户明确停止
-```

.claude/commands/release.md

@@ -1,160 +0,0 @@
----
-description: 发版工作流：根据变更类型决定版本号，更新所有版本文件和 changelog，运行验证，commit 并 push
-argument-hint: 可选：feature | bugfix | 或直接描述本次发布内容
-allowed-tools: ["Read", "Edit", "Bash", "Glob", "Grep"]
----
-
-# /release — Planet 发版工作流
-
-## 版本号规则
-
-| 变更类型 | 版本跳动 | 适用场景 |
-|---------|---------|---------|
-| `feature` | `+0.1.0` | 纯新功能，无 bugfix |
-| `improvement` | `+0.0.1` | UI 调整、小功能增强、bugfix 混合，或以 UI/体验改进为主的迭代 |
-| `bugfix` | `+0.0.1` | 纯 bug 修复，无新功能 |
-| `docs` / `maintenance` / `refactor` | 默认不发版，除非用户明确要求 |
-
-意图混合时以用户明确描述为准；bugfix + 小 feature 混合默认判定为 `improvement`（`+0.0.1`）。
-
-## 必须同步更新的文件
-
-使用 `git rev-parse --show-toplevel` 获取仓库根目录，以下路径均相对于根目录：
-
-- `VERSION`
-- `frontend/package.json`（`"version"` 字段）
-- `pyproject.toml`（`version =` 字段）
-- `uv.lock`（**不要手动编辑**，通过 `uv lock` 重新生成）
-- `docs/CHANGELOG.md`
-- `docs/version-history.md`
-
-## 节省上下文规则
-
-发版判断应以确定性 CLI 证据为主，优先使用紧凑命令和定点读取：
-
-```bash
-git status --short
-git diff --stat HEAD
-git diff --name-only HEAD
-rg -n "version|^## |^Released:|当前开发版本|current" VERSION frontend/package.json pyproject.toml docs/CHANGELOG.md docs/version-history.md
-```
-
-除非需要判断某个代码变更是否属于本次发版，否则不要读取完整 diff。
-
-## 执行步骤
-
-### Step 1 — 环境检查
-
-```bash
-git branch --show-current          # 确认在 dev 分支
-git status --short                 # 检查是否有无关的未暂存修改
-cat VERSION                        # 读取当前版本
-```
-
-若当前**不在 `dev` 分支**，停下来告知用户，不要继续。
-
-若存在无关的未暂存修改，列出并询问用户是否一并提交，或先 stash。
-
-### Step 2 — 确定发版类型与新版本号
-
-- 若 `$ARGUMENTS` 提供了明确类型（`feature` / `bugfix`），直接使用
-- 否则根据 `git diff --stat HEAD`、`git diff --name-only HEAD`、必要的 focused diff 和 `git log` 推断
-- 计算新版本号（例：`0.26.2` → bugfix → `0.26.3`）
-- **先输出发版计划供用户确认**：
-
-```
-发版计划：
-  类型：bugfix
-  版本：0.26.2 → 0.26.3
-  分支：dev
-  将更新：VERSION, frontend/package.json, pyproject.toml, uv.lock, CHANGELOG.md, version-history.md
-```
-
-### Step 3 — 更新版本号文件
-
-按顺序更新（每步用 Edit 工具，精确替换，不要重写整个文件）：
-
-1. `VERSION` — 直接替换全部内容为新版本号
-2. `frontend/package.json` — 替换 `"version": "x.x.x"` 行
-3. `pyproject.toml` — 替换 `version = "x.x.x"` 行
-4. 运行 `uv lock` 重新生成 `uv.lock`（在仓库根目录下执行）
-
-### Step 4 — 更新 CHANGELOG.md
-
-在文件顶部插入新条目，格式：
-
-```markdown
-## [x.x.x] — YYYY-MM-DD
-
-### ✨ Features / 🐛 Fixes / 🔧 Improvements
-- ...（只列高信号条目，最多 5 条）
-- ...
-
----
-```
-
-日期使用 `date +%Y-%m-%d` 获取今天的日期。
-
-### Step 5 — 更新 docs/version-history.md
-
-- 更新文件头部的"当前开发版本"字段
-- 在时间线表格顶部插入新行：`| vx.x.x | YYYY-MM-DD | 一句话摘要 |`
-
-### Step 6 — 验证
-
-针对本次变更范围做最小验证：
-
-- Python 文件有修改：先用 `git diff --name-only HEAD -- '*.py'` 列出，再运行 `python3 -m py_compile <changed_files>`
-- Frontend 文件有修改：先用 `git diff --name-only HEAD -- frontend` 判断范围，再运行项目标准检查（若无则跳过并说明）
-- 版本号一致性检查：用 grep 确认 VERSION、package.json、pyproject.toml 中的版本号完全一致
-
-```bash
-cat VERSION
-rg -n "\"version\":|^version =|version = " frontend/package.json pyproject.toml uv.lock
-```
-
-### Step 7 — 提交前预览
-
-展示将要提交的文件列表：
-
-```bash
-git diff --stat HEAD
-```
-
-再次确认所有必须文件都在变更列表中，**不包含**非预期文件（如调试文件、.env 等）。
-
-### Step 8 — Commit & Push（用户确认后）
-
-```bash
-git add VERSION frontend/package.json pyproject.toml uv.lock docs/CHANGELOG.md docs/version-history.md
-# 若有代码变更也一并 stage
-git add <code_files>
-
-git commit -m "release: bump version to x.x.x"
-git tag vx.x.x
-git push origin dev
-git push origin vx.x.x
-```
-
-commit message 固定格式：`release: bump version to x.x.x`
-
-### Step 9 — 完成确认
-
-输出摘要：
-
-```
-✓ 版本号已更新：0.26.2 → 0.26.3
-✓ CHANGELOG 已更新
-✓ version-history 已更新
-✓ uv.lock 已重新生成
-✓ 验证通过
-✓ commit: release: bump version to 0.26.3
-✓ tag: v0.26.3
-✓ 已 push 到 origin/dev
-```
-
-## 注意事项
-
-- `uv.lock` 只能通过 `uv lock` 生成，绝不手动编辑
-- 发版 commit 只包含版本文件 + 本次功能代码，不混入无关改动
-- 若环境中 `uv` 不可用，说明原因并跳过 lockfile 更新，提醒用户手动运行

.codex/skills/cleanup/SKILL.md

@@ -19,9 +19,9 @@ Do not refactor architecture, add features, or change behavior.
 
 If the user specifies a file or directory, check only that. Otherwise check all uncommitted changes (`git diff HEAD`).
 
-Only report issues present in **newly added or modified** lines of this diff — do not audit unchanged code.
+Only report and fix issues present in **newly added or modified** lines of this diff — do not audit unchanged code, and do not rewrite unrelated files during cleanup.
 
-## Token-Saving Rule
+## Token-Saving Rule & Harness Checks
 
 Prefer deterministic CLI checks before reading files into model context:
 
@@ -29,27 +29,36 @@ Prefer deterministic CLI checks before reading files into model context:
 git diff --name-only HEAD
 git diff --unified=0 HEAD -- <path>
 git diff --check
-rg -n "TODO|FIXME|console\.log|debugger|print\(" <changed-paths>
 ```
 
+Use the repository's focused harness checks for the touched surfaces:
+
+- Backend Python changes: `scripts/harness/backend-rules-check.sh` (catches direct `print()`, `breakpoint()`, and debugger calls)
+- Frontend changes: `scripts/harness/frontend-rules-check.sh` (catches console logs, missing button types, Card nesting, layout height-chain issues)
+- Secrets / credentials: `scripts/harness/security-check.sh`
+
 Read full files only when the focused diff does not provide enough surrounding context to make a safe edit.
 
 ## Checklist
 
 ### 1. Duplicate Logic
-- Identical or near-identical code blocks appearing in multiple places
-- A function/helper that already exists but is re-implemented elsewhere instead of being reused
-- Repeated DOM queries, regex literals, or template strings within the same file
+- Identical or near-identical code blocks appearing in multiple places within the modified scope
+- A helper that already exists but is re-implemented elsewhere instead of being reused
+- Repeated complex queries or template strings that cause actual maintenance hazard
+- **Anti-Over-Defensiveness**: Keep one-off logic at its call site. Do not prematurely abstract single-use or 2-use trivial helpers/templates that are clearer inline.
 
 ### 2. Magic Numbers / Magic Strings
-- Bare numeric literals used in calculations (offsets, timeouts, sizes, thresholds) without a named constant
-- Hardcoded strings (IDs, status values, URL fragments) scattered through logic
-- Exceptions: `0`, `1`, `-1`, `100`, `""` and other idiomatically clear values are fine
+- Bare numeric literals used in calculations (offsets, timeouts, sizes, thresholds) where the intent is completely opaque and naming adds clear value
+- Hardcoded domain strings (IDs, status values, protocol fragments) scattered through logic
+- **Anti-Over-Defensiveness (AGENTS.md Quality Bar)**:
+  - Idiomatically clear values such as `0`, `1`, `-1`, `100`, `""` are fine inline.
+  - **Do NOT over-extract local single-use layout offsets, Three.js coordinates, padding, or math coefficients into module-level globals.**
+  - Keep one-off logic at its call site unless extraction clarifies a complex operation or lets callers share the same behavior.
 
 ### 3. Naming Issues
 - Cryptic abbreviations (`or_`, `tmp2`, `x2`)
 - Names that do not match actual behavior
-- The same concept referred to by different names in different places
+- The same concept referred to by different names in different places within the changed scope
 
 ### 4. Dead Code
 - Commented-out code blocks (3+ lines)
@@ -58,12 +67,12 @@ Read full files only when the focused diff does not provide enough surrounding c
 
 ### 5. Style Inconsistencies
 - Trailing whitespace
-- Mixed quote styles or indentation within the same file
+- Mixed quote styles or indentation within the same file (Python: 4 spaces Black style; TypeScript: 2 spaces Prettier style)
 - Inconsistent blank-line usage (multiple consecutive blank lines, etc.)
 
 ### 6. Other
-- Private helper functions that should be exported but are not, causing callers to duplicate the implementation
 - Overly verbose conditions that can be simplified without changing logic
+- Keep fixes strictly scoped to the modified files; do not rewrite external callers or expand cleanup beyond the changed scope.
 
 ## Steps
 
@@ -107,30 +116,39 @@ If no issues are found, output "No code smells detected. Code quality looks good
 
 Use the Edit tool for **minimal, targeted changes**:
 
-- **Duplicate logic**: extract to a shared constant or function; update all call sites
-- **Magic number/string**: declare `const NAME = value` near the top of the relevant scope; replace all usages
-- **Naming**: rename the variable/function; update all references
+- **Duplicate logic**: extract to a shared constant or helper within the owned file or component scope; update call sites strictly within scope
+- **Magic number/string**: declare `const NAME = value` near the top of the relevant scope only when the value is reused across places or its meaning is genuinely opaque; replace usages. Keep one-off UI/math offsets inline per `AGENTS.md`
+- **Naming**: rename the variable/function; update all references within the modified scope
 - **Dead code**: delete it
 - **Trailing whitespace / style**: fix in place
-- **Unexported helper**: add `export`; update callers to import instead of re-implementing
+- **Unexported helper**: add `export` only when requested; do not modify unrelated callers
 
 Principles:
-- Only fix issues identified in the checklist — no extra improvements
+- Only fix issues identified in the checklist — no extra improvements or speculative refactorings
 - Keep each Edit as small as possible
-- After fixing, verify the old bad pattern is gone with grep
-- Prefer `apply_patch` for targeted edits; use formatters only when the repository already uses them for the touched file type
+- After fixing, verify the old bad pattern is gone with grep or deterministic harness checks
+- Prefer `apply_patch` or targeted edit tools; use formatters only when the repository already uses them for the touched file type
 
-### Step 5 — Summary
+### Step 5 — Verify and summarize
+
+Run the smallest relevant harness checks for the modified surfaces:
+- Python files: `uv run python -m py_compile <changed_py_files>` and `scripts/harness/backend-rules-check.sh`
+- Frontend files: `scripts/harness/frontend-rules-check.sh` (and `bun run --cwd frontend build` if imports/types were adjusted)
+- Formatting / whitespace: `git diff --check`
 
 ```
 Cleanup complete:
 
 Fixed N issues:
   ✓ earth.js — extracted duplicate vertexShader into ATMOS_VERTEX_SHADER constant
-  ✓ main.js — extracted TOOLTIP_CURSOR_OFFSET = 14 (4 references updated)
-  ✓ controls.js — exported updateLayerButtonState; removed duplicate implementation in main.js
+  ✓ main.js — clarified variable naming in news event payload handler
+  ✓ controls.js — removed unused dead import
   ...
 
+Validation:
+  ✓ git diff --check passed
+  ✓ scripts/harness/frontend-rules-check.sh passed
+
 Skipped (needs manual review):
   ! foo.js L45 — large commented-out block; confirm it is safe to delete
 ```
@@ -141,4 +159,6 @@ Skipped (needs manual review):
 - **Do not** add new features, abstractions, or parameters
 - **Do not** rewrite comments (only delete commented-out dead code)
 - **Do not** touch test file logic
+- **Do not** rewrite unrelated files during cleanup (per `AGENTS.md` Prohibited)
+- **Do not** over-extract local single-use coordinates, layout numbers, or inline math coefficients into global constants (preventing defensive over-engineering)
 - If a magic number's intent is uncertain, skip it and flag it in the summary

.codex/skills/docs/SKILL.md

@@ -48,18 +48,22 @@ rg -n "class |def |function |export |router|@router|interface |type " <path>
 
 3. Write the doc:
 
-- Explain background/problem, design decisions, constraints, and operational impact.
+- Include the context, rationale, or operational impact the reader needs for this change.
 - Keep code snippets short and directly relevant.
 - List related files only when they help future maintainers navigate.
 - Use the repository’s existing language, heading style, and naming conventions.
-- For UI labels, chart labels, feature names, datasource names, and other terms that may become mixed Chinese/English copy, check `docs/technical/{zh,en}/naming-glossary.md` and use the documented display name. If a confusing term is missing, update the glossary in both languages as part of the docs change.
+- For UI labels, chart labels, feature names, datasource names, and other terms that may become mixed Chinese/English copy, check `docs/technical/{zh,en}/naming-glossary.md` and use the documented display name. Only update the glossary when introducing new, ambiguous, or cross-language public concepts; avoid unnecessary glossary churn for ordinary bug fixes or internal refactors.
 
 4. Verify:
 
 - Read the completed doc once for clarity and stale statements.
 - Verify important referenced paths exist with `test -e` or `rg --files`.
-- Run repository-specific doc checks from `docs/documentation-coverage-rules.md` when present.
-- For Markdown links, check that user-facing titles are readable and not raw filenames unless the repository rules allow it.
+- Run the repository's centralized documentation harness gate:
+  ```bash
+  scripts/harness/docs-consistency-check.sh
+  ```
+  This automatically verifies bilingual pairs, public registry whitelist, link anchors, language-scoped links, section deep-links, and stale admin route terms in a single pass.
+- Run any additional domain-specific checks from `docs/documentation-coverage-rules.md` when applicable.
 
 ## Hard Constraints
 

.codex/skills/release/SKILL.md

@@ -19,9 +19,11 @@ Do not use this skill for ordinary commits that are not being released.
 ## Versioning Rules
 
 - `feature` -> bump minor and reset patch to `0` (`x.y.z` → `x.(y+1).0`; for example `0.41.2` → `0.42.0`)
-- `bugfix` -> bump `+0.0.1`
+- `improvement` -> bump patch `+0.0.1` (`x.y.z` → `x.y.(z+1)`)
+- `bugfix` -> bump patch `+0.0.1` (`x.y.z` → `x.y.(z+1)`)
 - `docs`, `maintenance`, and `refactor` do not bump by default unless the user explicitly wants a release
 
+Mixed bugfix and small feature/UI work defaults to `improvement` unless the user explicitly chooses another release type.
 When intent is mixed, prefer the user's stated release intent.
 
 ## Required Files
@@ -53,28 +55,32 @@ Do not inspect full diffs unless deciding whether changed code belongs in the re
 ### Step 1 — Environment check
 
 ```bash
-git branch --show-current   # must be on dev
+git branch --show-current   # check current branch
 git status --short           # check for unrelated uncommitted changes
 cat VERSION                  # read current version
 ```
 
-If not on `dev`, stop and tell the user. Do not proceed.
+Default release branch is `dev`. If not on `dev` (e.g. on a dedicated maintenance or release branch), report the active branch and confirm before proceeding, rather than hard-failing.
 
 If unrelated uncommitted changes exist, list them and ask the user whether to include them or stash first.
 
 ### Step 2 — Determine release type and next version
 
-- If the user provided an explicit type (`feature` / `bugfix`), use it
-- Otherwise infer from `git diff --stat HEAD`, `git diff --name-only HEAD`, focused diffs for changed code, and recent `git log`
+- If the user provided an explicit type (`feature` / `improvement` / `bugfix`), use it
+- Otherwise infer from `git diff --stat HEAD`, `git diff --name-only HEAD`, focused diffs for changed code, and recent `git log`:
+  - New features or major capabilities -> `feature`
+  - Mixed bugfix and small feature/UI work -> default to `improvement` (per `rules.md`)
+  - Bugfixes only -> `bugfix`
 - Compute the next version:
   - `feature`: increment minor and reset patch to `0` (e.g. `0.41.2` → `0.42.0`)
-  - `bugfix`: increment patch only (e.g. `0.26.2` → `0.26.3`)
+  - `improvement`: increment patch only (e.g. `0.76.18` → `0.76.19`)
+  - `bugfix`: increment patch only (e.g. `0.76.18` → `0.76.19`)
 - **Show the release plan before making any changes:**
 
 ```
 Release plan:
-  Type:    bugfix
-  Version: 0.26.2 → 0.26.3
+  Type:    improvement
+  Version: 0.76.18 → 0.76.19
   Branch:  dev
   Will update: VERSION, frontend/package.json, pyproject.toml, uv.lock, CHANGELOG.md, version-history.md
 ```
@@ -117,17 +123,25 @@ Get today's date with `date +%Y-%m-%d`.
 
 ### Step 6 — Validate
 
-Run the smallest relevant validation for the changes in scope:
+Run the smallest relevant harness validation for the changes in scope:
 
-- Python files changed: list changed Python files with `git diff --name-only HEAD -- '*.py'`, then run `python3 -m py_compile <changed_files>`
-- Frontend files changed: list changed frontend files with `git diff --name-only HEAD -- frontend`, then run the project-standard check if available; otherwise skip and say so
-- Version consistency: confirm VERSION, package.json, pyproject.toml, and uv.lock all show the same version
+- **Version consistency**: confirm `VERSION`, `frontend/package.json`, `pyproject.toml`, and `uv.lock` all show the identical target version:
 
 ```bash
 cat VERSION
 rg -n "\"version\":|^version =|version = " frontend/package.json pyproject.toml uv.lock
 ```
 
+- **Security scan**: run `scripts/harness/security-check.sh`
+- **Python files changed**:
+  - Compile: list changed Python files with `git diff --name-only HEAD -- '*.py'`, then run `uv run python -m py_compile <changed_files>`
+  - Service rules: run `scripts/harness/backend-rules-check.sh`
+- **Frontend files changed**:
+  - Rules: run `scripts/harness/frontend-rules-check.sh`
+  - Build: run `bun run --cwd frontend build` (Bun only; never skip)
+- **Docs changed**: run `scripts/harness/docs-consistency-check.sh`
+- **Broad / cross-subsystem changes**: run `scripts/harness/quick-check.sh`
+
 ### Step 7 — Pre-commit preview
 
 Show what will be committed:
@@ -141,29 +155,36 @@ Confirm all required files are present and no unexpected files (debug files, `.e
 ### Step 8 — Commit, tag, and push
 
 ```bash
+branch=$(git branch --show-current)
 git add VERSION frontend/package.json pyproject.toml uv.lock docs/CHANGELOG.md docs/version-history.md
 # also stage any code changes included in this release
 git add <code_files>
 
 git commit -m "release: bump version to x.x.x"
 git tag vx.x.x
-git push origin dev
-git push origin vx.x.x
 ```
 
 Commit message format is fixed: `release: bump version to x.x.x`
 
+Push to remote:
+- When remote publication is requested:
+  ```bash
+  git push origin "$branch"
+  git push origin vx.x.x
+  ```
+- If remote publication was not requested or remote access is restricted, report the commit and tag as complete and output the push commands for manual execution.
+
 ### Step 9 — Completion summary
 
 ```
-✓ Version bumped: 0.26.2 → 0.26.3
+✓ Version bumped: 0.76.18 → 0.76.19
 ✓ CHANGELOG updated
 ✓ version-history updated
 ✓ uv.lock regenerated
 ✓ Validation passed
-✓ commit: release: bump version to 0.26.3
-✓ tag: v0.26.3
-✓ Pushed to origin/dev
+✓ commit: release: bump version to 0.76.19
+✓ tag: v0.76.19
+✓ Pushed to origin/dev (or reported pending manual push)
 ```
 
 ## Notes

.dockerignore

@@ -3,8 +3,23 @@
 !pyproject.toml
 !uv.lock
 !VERSION
+!frontend/
+!frontend/public/
+!frontend/public/earth/
+!frontend/public/earth/assets/
+!frontend/public/earth/assets/cards/
+!frontend/public/earth/assets/cards/manifest.json
 !backend/
 !backend/**
+!config/
+!config/earth-boundary-pov-policy.china-v1.json
+!scripts/
+!scripts/build_earth_boundary_china_pov_geojson.py
+!scripts/build_earth_boundary_pmtiles.py
+!scripts/earth_boundary_policy.py
+!resources/
+!resources/earth/
+!resources/earth/boundary-pov-geometry.china-v1.json
 !aiprovider/
 !aiprovider/**
 

.gitignore

@@ -25,11 +25,14 @@ __pycache__/
 build/
 develop-eggs/
 dist/
-downloads/
+downloads/*
+!downloads/usbipd-win/
+downloads/usbipd-win/*
+!downloads/usbipd-win/usbipd-win-5.3.0.msi
 eggs/
 .eggs/
-lib/
-lib64/
+/lib/
+/lib64/
 parts/
 sdist/
 var/
@@ -157,3 +160,8 @@ data/earth-boundary-sources/
 # Git; publish PMTiles/MVT artifacts through release/deploy storage instead of
 # committing thousands of generated tile files.
 frontend/public/earth/data/boundaries/
+
+# Offline comparison originals; runtime 8K maps and provenance remain versioned.
+resources/earth/candidates/nasa-blue-marble/*.jpg
+resources/earth/candidates/natural-earth-3/*.jpg
+*:Zone.Identifier

AGENTS.md

@@ -0,0 +1,204 @@
+# AGENTS.md
+
+**Planet agent harness. Defines behavior for coding agents working in this repository.**
+
+---
+
+## Harness Compatibility
+
+This file is the single authoritative agent guide for the Planet repository.
+The older lowercase `agents.md` entry has been merged here so coding agents and
+harness tools use one source of truth.
+
+### Source Of Truth
+
+- `rules.md` is the mandatory repository rule source. Always load `core`,
+  `security`, and `workflow`; load only task-relevant modules after that.
+- `AGENTS.md` defines the local agent operating mode and evidence gates.
+- `project_context.md` is background, not a rule source. Prefer newer
+  implementation docs when it disagrees with current code.
+- `.codex/skills/` is the active specialized workflow layer for cleanup, docs,
+  goal-driven work, and release.
+- Do not duplicate long workflow text across harness files. Durable constraints
+  belong in `rules.md`; task procedures belong in skills or scripts.
+
+Read these files before changing code:
+
+1. `rules.md`
+2. `AGENTS.md`
+3. `project_context.md`
+4. `README.md`
+5. `docs/HARNESS.md`
+6. `CODEMAP.md`
+
+For documentation work, also read `docs/documentation-coverage-rules.md`.
+
+### Start Safely
+
+Before broad edits:
+
+```bash
+git status --short
+scripts/harness/doctor.sh
+```
+
+Use focused context commands before reading large files:
+
+```bash
+rg -n "<symbol-or-term>" <path>
+git diff --stat HEAD
+git diff --name-only HEAD
+git diff --unified=0 HEAD -- <path>
+```
+
+Preserve user changes already present in the worktree.
+
+### Validation
+
+For an isolated change, run the smallest check that exercises the affected
+behavior, along with any gates required for that area. For example, run the
+docs consistency check for a link change, or reproduce a bug at the entry
+point where it occurred.
+
+Use the quick suite when a change affects shared harness wiring or needs
+regression checks across areas:
+
+```bash
+scripts/harness/quick-check.sh
+```
+
+Use full validation for larger cross-subsystem changes:
+
+```bash
+scripts/harness/validate.sh
+```
+
+`validate.sh` includes quick checks, frontend Bun build, and frontend smoke
+unless disabled by its documented environment flags. Docker image smoke builds
+are intentionally opt-in:
+
+```bash
+PLANET_HARNESS_DOCKER_SMOKE=1 scripts/harness/validate.sh
+```
+
+Harness scripts resolve `bun`, `uv`, and optional delivery tools from the
+current non-interactive environment first. If a tool is missing there, they ask
+the user's login interactive shell instead of assuming a specific dotfile.
+
+### High-Risk Areas
+
+- `planet.sh` owns local lifecycle, ports, WSL/LAN behavior, and destructive
+  `destroy` cleanup.
+- Frontend package management is Bun-only. Do not use npm, pnpm, or yarn.
+- Frontend changes must satisfy `scripts/harness/frontend-rules-check.sh`; use
+  rendered smoke evidence for public pages, auth guards, authenticated admin
+  route/section availability, safe navigation/search/tab interactions, mobile
+  layout, and 125% / 150% zoom, not only a build.
+- Admin or Docs layout changes must load `rules.md` `uiux` and preserve the
+  one-screen (`一屏` / `首屏`) height chain: route roots use `height: 100%`,
+  intermediate wrappers keep `min-height: 0`, and only the intended child owns
+  scrolling.
+- `aiprovider` is a protocol/provider adapter; keep business prompts and product
+  workflows in the backend.
+- Earth rendering depends on layer order, depth behavior, picking, and
+  performance-sensitive Three.js code.
+- Secrets belong in environment files or configured settings stores, never in
+  committed files.
+- Backend service code must use structured logging instead of `print()` or
+  debugger calls; `scripts/harness/backend-rules-check.sh` enforces this.
+
+### Conflict Policy
+
+Existing project rules and workflows win. If new harness guidance conflicts with
+`rules.md`, `AGENTS.md`, current docs, scripts, or CI, keep the existing
+behavior and document the compatibility note in `docs/harness-audit.md` or
+`docs/HARNESS.md`.
+
+---
+
+## Operating Mode
+
+- Default to acting directly when the user gives a clear task.
+- For authorized multi-agent work, follow the Coding Agent Model Strategy in
+  `rules.md` workflow: Sol for implementation, Luna for bounded supporting work,
+  and Astra for complex reasoning; the main agent owns final verification.
+- Ask before acting only when the missing decision is risky, cannot be
+  discovered from repository context, and no conservative assumption is safe.
+- Read relevant files before editing.
+- Prefer focused CLI evidence: `rg`, `git diff --stat`, `git diff --name-only`,
+  focused file reads, tests, builds, linters, and harness scripts.
+- Keep changes scoped to the requested area. Do not mix cleanup, feature work,
+  release work, and documentation unless the task requires it.
+- When the user corrects a detail, apply it to the original request. Carry the
+  correction into the result without turning discarded ideas into new requirements.
+
+---
+
+## Evidence Gates
+
+- Visual inputs are blocking evidence. If the user provides a screenshot, image,
+  mock, browser capture, or visual reference, obtain evidence from the artifact
+  before interpreting intent or editing code.
+- Path resolution is part of the task. If the path cannot be opened, first try
+  reasonable local equivalents such as WSL/Windows path conversion,
+  workspace-relative lookup, absolute paths, and attached-file locations.
+- Never guess from prompt text, filenames, previous context, logs, OCR, or
+  memory when a visual artifact was provided but cannot be accessed.
+- OCR is acceptable evidence for text-only visual questions or non-multimodal
+  environments; state that OCR was used as the fallback. Layout, color, spacing,
+  pixel, and rendering issues need real visual inspection or a clear limitation
+  note.
+- If a visual artifact still cannot be inspected, say so and pause that
+  visual-dependent part of the work.
+- Claims of completion need evidence: a relevant test, build, lint, screenshot,
+  diff, direct file check, or harness result.
+- Verify the outcome through the entry point the user relies on. A source check
+  can confirm a code convention. A UI test with a mocked API can confirm what
+  the user sees for that response.
+- When a test fails, identify the behavior or invariant it protects. Fix the
+  implementation if that regressed. If the behavior is correct and the assertion
+  pins an obsolete detail, update the test to exercise the behavior.
+- For UI and rendering changes, verify the rendered result when local tooling
+  allows it.
+
+---
+
+## Communication
+
+- Match the user's language. Use Chinese for Chinese requests unless the user
+  asks otherwise.
+- Keep updates short and specific: what is being inspected, edited, or verified.
+- Final responses should summarize changed files and verification, with blockers
+  stated plainly.
+- Use file references with line numbers when explaining code or review findings.
+
+---
+
+## Quality Bar
+
+- Prefer existing project patterns and the smallest change that satisfies the
+  request. Keep one-off logic at its call site unless extraction clarifies a
+  complex operation or lets callers share the same behavior.
+- Validate and normalize external data at the boundary that owns it. Recheck a
+  guarantee when data crosses a new trust boundary or an operation can invalidate it.
+- Before adding a retry, lock, or fallback, identify the failure it addresses
+  from a requirement, an existing contract, or observed behavior. Catch errors
+  where the caller can recover, translate them, or add context and propagate them.
+- Remove stale branches, mocks, compatibility paths, and duplicated helpers once
+  a stable path exists.
+- Centralize prompts, constants, defaults, and shared request/response handling.
+- Do not add secrets, generated runtime output, or local environment files.
+- Frontend commands use Bun only. Do not use `npm`, `pnpm`, or `yarn`.
+- Run the smallest relevant verification for the changed scope and report
+  anything skipped.
+
+---
+
+## Prohibited
+
+- Do not skip visual evidence handling when a visual artifact was provided.
+- Do not preserve obsolete harness files just because they already exist.
+- Do not invent behavior not present in code, docs, or verified external
+  sources.
+- Do not rewrite unrelated files during cleanup.
+- Do not mark a task complete without checking concrete success criteria.

CODEMAP.md

@@ -0,0 +1,110 @@
+# Code Map
+
+This map gives agents and maintainers a quick orientation without replacing the
+deeper architecture docs. Current implementation docs under `docs/technical/`
+are the source of detail for specific subsystems.
+
+## Top-Level Areas
+
+| Path | Role | Notes |
+| --- | --- | --- |
+| `backend/` | FastAPI backend, auth, APIs, data collectors, AI task orchestration, persistence | Tests live in `backend/tests/`; run backend tests from `backend/` with the root uv project. |
+| `frontend/` | React admin console, Docs UI, Web Earth shell, Vite build | Use Bun only. Public Earth assets live under `frontend/public/earth/`. |
+| `aiprovider/` | Model provider/protocol adapter service | Keep it free of product-specific prompts and workflows. |
+| `motion_agent/` | Motion capture protocol service used by `planet.sh` | Often dry-runs when cameras are unavailable, especially in WSL. |
+| `scripts/` | Utility scripts and harness wrappers | Harness commands live in `scripts/harness/`. |
+| `docs/` | Plans, technical docs, changelog, harness docs | Public technical docs are explicitly registered by the frontend Docs catalog. |
+| `deploy/helm/planet/` | Helm chart for staging/deployment smoke paths | CI runs helm lint/template when delivery checks are available. |
+| `.gitea/workflows/` | CI, release image build, staging deploy workflows | This repository uses Gitea workflow files, not `.github/workflows/`. |
+| `planet.sh` | Main local lifecycle script | Owns init/start/restart/stop/health/log/createuser/destroy. |
+| `planet.cmd` and `scripts/windows/` | Portable Windows entry, bound desktop launcher generation, and WSL restart bridge | Delegates service lifecycle to `planet.sh`; Windows and isolated WSL regression tests live under `scripts/harness/`. |
+
+## Runtime Entry Points
+
+| Runtime | Entry Point | Validation |
+| --- | --- | --- |
+| Local full stack | `./planet.sh start` | `./planet.sh health` |
+| Backend API | `backend/app/main.py` | `cd backend && uv run --frozen --group dev --project .. python -m pytest -q` |
+| Frontend app | `frontend/src/main.tsx` and `frontend/vite.config.mts` | `cd frontend && bun run build` |
+| AI Provider | `aiprovider/main.py` | `curl http://localhost:8010/health` after startup |
+| Motion Agent | `python -m motion_agent` via `planet.sh` | `./planet.sh health` or dry-run startup |
+| Docs UI | `frontend/src/pages/Docs/` | Docs catalog metadata plus frontend build |
+
+## Ownership Boundaries
+
+- Backend owns business state, auth, evidence collection, prompt selection, AI
+  task orchestration, and database persistence.
+- `aiprovider` owns provider identity, request adapter style, model gateway
+  retries, and health/status endpoints only.
+- Frontend owns operator workflows, Docs presentation, Web Earth orchestration,
+  and client-side state that mirrors backend truth.
+- Web Earth rendering changes must preserve documented layer order, altitude
+  offsets, picking behavior, legend semantics, and performance constraints.
+- `planet.sh` owns local environment bootstrap and service lifecycle. Prefer
+  wrapping it from harness scripts instead of duplicating its internals.
+- Harness scripts source `scripts/harness/lib.sh` so agent shells that cannot
+  see `bun` or `uv` in non-interactive `PATH` can still resolve the user's login
+  interactive command path without hardcoding `.zshrc`.
+
+## Validation Commands
+
+```bash
+scripts/harness/doctor.sh
+scripts/harness/security-check.sh
+scripts/harness/backend-rules-check.sh
+scripts/harness/frontend-rules-check.sh
+scripts/harness/docs-consistency-check.sh
+scripts/harness/quick-check.sh
+scripts/harness/validate.sh
+./planet.sh health
+```
+
+CI-equivalent local checks:
+
+```bash
+cd backend
+uv run --frozen --group dev --project .. python -m pytest -s tests/test_api.py tests/test_realtime_sources.py -q
+
+cd frontend
+bun install --frozen-lockfile
+bun run build
+PLANET_FRONTEND_SMOKE_URL=http://127.0.0.1:4173 bun ../scripts/harness/frontend-smoke.mjs
+```
+
+The frontend smoke covers public routes, unauthenticated admin guards,
+login-error handling, the Earth iframe entry, and authenticated `super_admin`
+admin route/section rendering with mocked API data. Authenticated admin checks
+run on desktop, mobile, and 125% / 150% zoom; desktop and mobile passes also
+check for accidental global horizontal overflow. A second smoke layer exercises
+safe desktop/mobile navigation, admin search, section tab switching, dialog
+opening, and non-destructive shortcut links.
+
+Optional delivery smoke, when Docker and Helm are available:
+
+```bash
+PLANET_HARNESS_DOCKER_SMOKE=1 scripts/harness/validate.sh
+```
+
+## Deeper Docs
+
+| Topic | Start Here |
+| --- | --- |
+| Data products and flows | `docs/technical/zh/platform-data-flows.md` and `docs/technical/en/platform-data-flows.md` |
+| Operations and local lifecycle | `docs/technical/zh/ops-runbook.md` and `docs/technical/en/ops-runbook.md` |
+| `planet.sh` startup behavior | `docs/technical/zh/ops-planet-sh-startup.md` and `docs/technical/en/ops-planet-sh-startup.md` |
+| AI Provider | `docs/technical/zh/agents-aiprovider.md` and `docs/technical/en/agents-aiprovider.md` |
+| Admin frontend | `docs/technical/zh/frontend-admin-frontend-context.md` and `docs/technical/en/frontend-admin-frontend-context.md` |
+| Earth frontend | `docs/technical/zh/earth-frontend-context.md` and `docs/technical/en/earth-frontend-context.md` |
+| Earth render order | `docs/technical/zh/earth-render-layer-order.md` and `docs/technical/en/earth-render-layer-order.md` |
+| Documentation rules | `docs/documentation-coverage-rules.md` |
+| Harness workflow | `docs/HARNESS.md` |
+
+## Known Sharp Edges
+
+- `project_context.md` is static background for agents. It now labels future
+  stack directions separately, but current code and technical docs still win
+  when details diverge.
+- README now describes Web Earth, React admin, FastAPI, and `aiprovider` as the
+  active local development shape.
+- Local `destroy` is intentionally destructive for Planet-owned Docker and build
+  state. Never run it as a validation shortcut.

README.md

@@ -83,7 +83,7 @@
 | 组件 | 用途 |
 |------|------|
 | React 18 | UI 框架 |
-| Ant Design Pro | 管理后台组件 |
+| Tactile UI / Radix primitives / lucide-react | 管理后台组件、基础交互与图标 |
 | Axios | HTTP 客户端 |
 | Socket.io-client | WebSocket 客户端 |
 | ECharts | 统计图表 |
@@ -168,10 +168,12 @@
 
 ## 快速启动
 
+入口需要先具备 `zsh`、`curl` 和可访问的软件源。Ubuntu / Ubuntu WSL 上，`init` 会自动检测并补装 Docker Engine、Compose v2 和 Buildx，启动 Docker 服务并配置当前用户的访问权限；需要系统权限时会提示输入 sudo 密码。其他系统请先准备可用的 Docker 环境。
+
 ```bash
 # 新机器或空项目首次初始化
 ./planet.sh init
-# 会自动安装/检查 uv、bun，同步 Python/前端依赖
+# 会先准备 Docker / Compose / Buildx，再安装/检查 uv、bun 并同步 Python/前端依赖
 # 会在缺少时生成 backend/.env、aiprovider/.env、frontend/.env.local
 # 会启动 PostgreSQL/Redis，并创建表、默认数据源和本地默认用户
 
@@ -322,7 +324,7 @@ ipconfig
 
 ## 启动容错参数
 
-`planet.sh` 现在为依赖安装、数据库、AI Provider 启动加入了有限次重试，并会在数据库与 `aiprovider` 启动后额外等待 Docker healthcheck。
+`planet.sh` 为依赖安装、数据库、AI Provider 启动提供有限次重试。数据库先检查容器健康，再验证后端实际连接；`aiprovider` 直接以宿主机 `/health` 就绪为准。后端进程退出或应用初始化失败时立即停止等待，避免重复消耗健康检查预算。
 
 可通过环境变量临时调整:
 

TODO.md

@@ -4,20 +4,16 @@ This file is the active backlog only. Completed history belongs in `docs/CHANGEL
 
 ## Earth
 
+- [ ] Motion Agent v2 hardening: tune the implemented MediaPipe gesture recognizer across camera placements, exercise the UE command/control client, run reconnect and dual-camera soak tests, and continue the v3 calibrated 3D roadmap described in [Motion Agent v2 Control Protocol And 3D Calibration Roadmap](/home/ray/dev/linkong/planet/docs/plans/motion-agent-v2-control-protocol-plan.md).
 - [ ] Earth AI command entry: merge natural-language and speech-triggered LLM commands into the existing Earth search panel as described in [Agent Runtime, Earth LLM Command, And Speech Entry Plan](/home/ray/dev/linkong/planet/docs/plans/agents-earth-command-runtime-plan.md).
 - [ ] Earth action executor: implement safe visualization actions for layer toggles, batch highlights, filters, focus, result panels, and clear-highlight behavior.
 - [ ] Earth entity matching: support stable entity ids and batch matching for Beidou satellites, mainland China compute centers, BGP, news, vessels, and cables.
-- [x] High-precision country boundary tile framework: implement the static vector tile builder, versioned seed output, frontend bbox tile loader, debounce, in-flight dedupe, and LRU cache described in [Earth High Precision Boundary Tiles Plan](/home/ray/dev/linkong/planet/docs/plans/earth-high-precision-boundary-tiles-plan.md).
-- [x] Add the `pmtiles-mvt` frontend tile provider contract, MVT decoder dependencies, static PMTiles Nginx handling, collector artifact registration, production readiness check, and user operation docs for Earth boundaries.
-- [x] Split Earth boundary ingestion into standard source collectors (`earth_admin0_boundaries`, `earth_coastline`, `earth_claim_lines`) plus the downstream `earth_boundary_tiles` PMTiles builder.
 - [ ] Replace debug GeoJSON boundary tiles with the real `earth-boundaries-china-pov-v1.pmtiles` production artifact after audited admin-0 / coastline / claim-line sources and the PMTiles toolchain are available.
 - [ ] Import authoritative China POV / coastline / claim-line source packages through the three standard Earth boundary source collectors, then rebuild a versioned PMTiles artifact so highest zoom `8-10` preserves trusted source geometry instead of seed data.
 - [ ] Earth boundary data: acquire or generate auditable China POV geometry for Zangnan, Aksai Chin, Taiwan/Penghu, Diaoyu Dao and affiliated islands, Chiwei Yu, South China Sea islands, Kosovo, Gaza, and the official dashed maritime claim line before implementing final visual changes.
 - [ ] Earth high-resolution basemap tiles: implement the viewport-loaded imagery layer described in [Earth High Resolution Basemap Tiles Plan](/home/ray/dev/linkong/planet/docs/plans/earth-high-resolution-basemap-tiles-plan.md), using high-precision coastline as the alignment reference instead of replacing the globe with one huge texture.
-- [ ] Presentation controller ownership: replace the singleton card fallback in [presentation-controller.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/presentation-controller.js) with a presentation/card token check before BGP/News migrate onto the shared controller, so connectors only attach to their owning card.
 - [ ] BGP frontend maintainability: split [bgp.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/bgp.js) by responsibility into data loading, marker rendering, overlays, and animation once the current interaction behavior is stable.
 - [ ] Optional BGP marker experiment: evaluate HTML markers for BGP incident/collector points if WebGL marker density or fixed screen-size clickability becomes a real blocker.
-- [ ] Earth news cruise: connect Earth news to the generic cruise queue via a news adapter rather than coupling news-specific sequencing into `main.js`.
 
 ## Compute Centers And Location
 
@@ -48,19 +44,15 @@ This file is the active backlog only. Completed history belongs in `docs/CHANGEL
 - [ ] Compatibility schema: cover adapter type, base URL pattern, auth header, thinking/reasoning defaults, stream path, tool-call capability, multimodal capability, and provider-specific request patches.
 - [ ] BGP geography fallback: evaluate `inetnum` / `inet6num` whois as a finer fallback layer after `prefix_geography`, `OpenGeoFeed`, and RIR delegated data.
 
-## Platform
-
-- [ ] Earth preferences scope: keep current device-local Earth preferences in `localStorage`; only design backend user preferences if account-level synchronization becomes a real product requirement.
-- [ ] System logs: finish a usable Planet log viewing flow that covers backend, frontend, AI Provider, and collector/task logs, with filtering and tailing.
-- [ ] Console UI modernization: gradually replace Ant Design with Planet-owned components and a consistent Tabler Icons based icon system.
-- [ ] Earth live sync: design a unified realtime invalidation path for summary/BGP/satellite updates if polling and current WebSocket channels become insufficient.
-
 ## Archive
 
 Archived items stay here so old context is not lost. Completed items remain checked; obsolete, invalid, or superseded items stay unchecked and include the reason.
 
 ### Completed
 
+- [x] Implemented the high-precision country boundary tile framework from [Earth High Precision Boundary Tiles Plan](/home/ray/dev/linkong/planet/docs/plans/earth-high-precision-boundary-tiles-plan.md): static vector tile builder, versioned seed output, frontend bbox tile loader, debounce, in-flight dedupe, and LRU cache.
+- [x] Added the `pmtiles-mvt` frontend tile provider contract, MVT decoder dependencies, static PMTiles Nginx handling, collector artifact registration, production readiness check, and user operation docs for Earth boundaries.
+- [x] Split Earth boundary ingestion into standard source collectors (`earth_admin0_boundaries`, `earth_coastline`, `earth_claim_lines`) plus the downstream `earth_boundary_tiles` PMTiles builder.
 - [x] Refined BGP observer and anomaly `hover/click` feel.
 - [x] Added BGP anomaly relationship display with cables / regions.
 - [x] Added the Earth BGP activity layer so the map still feels alive when incident density is low.
@@ -75,6 +67,11 @@ Archived items stay here so old context is not lost. Completed items remain chec
 - [x] Added OpenGeoFeed as a high-quality prefix geography override source.
 - [x] Made RIR delegated data a prefix geography fallback rather than the primary source.
 - [x] Added route leak and path instability / flap detectors after the activity layer work.
+- [x] Console UI modernization. Admin is now the only console, legacy Ant Design / Admin Next code paths and dependencies have been removed, and current console UI uses Planet-owned components.
+- [x] Earth news cruise adapter. News cruise now uses `news-cruise-adapter.js` and is wired from `main.js` instead of keeping news-specific sequencing directly in the main Earth loop.
+- [x] Presentation controller ownership. `PresentationController` now guards async ownership through active request identity checks, and current callers pass per-request card targets so stale connector/card work cannot overwrite the active presentation.
+- [x] Earth live sync. Database writes now flow through `earth_data_change_events`, `earth_db_change_listener`, layer adapters, cache invalidation, and the `earth_updates` WebSocket channel; the Earth frontend debounces updates and refreshes BGP, cables, compute centers, satellites, vessels, news, and interactables by layer.
+- [x] System logs. Log sources now normalize into `LogEvent`, Admin supports snapshot filtering plus WebSocket tail/follow, task/detail views deep-link into prefiltered logs, and Admin runtime errors report through the `admin-client` log source.
 
 ### Obsolete Or Superseded
 
@@ -85,3 +82,4 @@ Archived items stay here so old context is not lost. Completed items remain chec
 - [ ] Earth surface material overlay for boundary calibration. Superseded by the high-precision boundary tile plan; future work must use source-faithful boundary/coastline data rather than overlay calibration against the coarse base map.
 - [ ] Hardcoded Earth news source extraction as a standalone task. Superseded by the broader Earth news source configuration and collector plans.
 - [ ] Country-level compute-center fallback placement as a standalone task. Superseded by the shared location pipeline and registry/manual-review backlog.
+- [ ] Earth preferences backend sync scope. Superseded by the current product decision to keep Earth preferences device-local in `localStorage` until account-level synchronization becomes a real requirement.

VERSION

@@ -1 +1 @@
-0.63.0
+0.76.19

agents.md

@@ -1,231 +0,0 @@
-# agents.md
-
-**AI Agent 角色设定。定义 AI 如何行为、沟通和工作。**
-
----
-
-## Identity
-
-You are **opencode**, an AI coding assistant specialized in enterprise-level systems.
-
-You are working on the **智能星球计划 (Intelligent Planet Plan)** - a situational awareness system for data-centric competition featuring:
-- Python FastAPI backend
-- React Admin dashboard
-- Unreal Engine 5 3D visualization
-- Multi-source data collection
-- Polarized 3D large display (4K, 120Hz)
-
----
-
-## Communication Style
-
-### Tone
-- **Professional but concise**
-- Technical accuracy with clarity
-- No unnecessary verbosity
-- Use code comments sparingly (explain **why**, not **what**)
-
-### When Responding
-1. **Answer directly** - 1-3 sentences for simple questions
-2. **Use code blocks** for all code snippets
-3. **Include file:line_number** references when discussing code
-4. **Never** start with "I am an AI assistant" or similar phrases
-5. **Never** add unnecessary preambles/postambles
-
-### Examples
-
-**Good:**
-```
-GPU clusters are stored in `backend/app/services/collectors/top500.py:45`.
-```
-
-**Bad:**
-```
-Based on the information you provided, I can see that the GPU clusters are stored in the top500.py file at line 45. Let me explain more about this...
-```
-
----
-
-## Operational Mode
-
-### Plan Mode (default for complex tasks)
-- Analyze requirements
-- Propose architecture
-- Confirm with user before execution
-- **DO NOT** write code until approved
-
-### Build Mode (after user approval)
-- Execute the approved plan
-- Write code, run commands
-- Verify results
-- Report completion concisely
-
-### Read-Only Mode
-- Analyze code
-- Explain functionality
-- Answer questions
-- **DO NOT** modify files
-
----
-
-## Decision Framework
-
-### When to Ask Before Acting
-- Unclear requirements
-- Multiple implementation approaches
-- Architecture changes
-- Dependency additions
-- Anything that could break existing functionality
-
-### When to Act Directly
-- Clear, approved requirements
-- Routine tasks (linting, formatting, running tests)
-- Following established patterns
-- Fixing obvious bugs
-
-### When to Refuse
-- Malicious code requests
-- Security violations (secrets, credentials)
-- Anything that violates `rules.md`
-
----
-
-## Working Principles
-
-### 1. First Understand, Then Act
-- Read relevant files before editing
-- Understand existing patterns and conventions
-- Follow the code style in the codebase
-- Match the project's technology choices
-
-### 2. Incremental Progress
-- Break large tasks into smaller PRs
-- Complete one feature before starting the next
-- Run tests after each significant change
-- Commit frequently with clear messages
-
-### 3. Quality First
-- Write tests for new functionality
-- Run linters before committing
-- Fix warnings, don't ignore them
-- Document non-obvious decisions
-
-### 4. Communication Clarity
-- Use precise technical language
-- Show relevant code, not explanations
-- Report errors with context
-- Confirm understanding of requirements
-
----
-
-## Code Review Checklist
-
-Before marking a task complete:
-
-- [ ] Code follows `rules.md` style guidelines
-- [ ] Type hints are correct and complete
-- [ ] Error handling is proper (no silent failures)
-- [ ] Tests pass locally
-- [ ] Linting passes
-- [ ] No TODO comments left behind
-- [ ] Documentation updated if needed
-- [ ] Commit message is clear
-
----
-
-## Common Workflows
-
-### Feature Development
-```
-1. Understand requirements
-2. Check existing patterns in codebase
-3. Design solution (brief mental model)
-4. Write code following rules.md
-5. Write/run tests
-6. Lint and format
-7. Commit with clear message
-8. Report completion
-```
-
-### Bug Fix
-```
-1. Reproduce the bug (write failing test)
-2. Locate the source
-3. Fix the issue
-4. Verify test passes
-5. Check for regressions
-6. Commit fix
-```
-
-### Refactoring
-```
-1. Understand current behavior
-2. Design target state
-3. Make incremental changes
-4. Preserve tests
-5. Verify functionality
-6. Clean up dead code
-```
-
----
-
-## Special Considerations
-
-### WebSocket Services
-- Implement heartbeat mechanism (30-second intervals)
-- Handle disconnection gracefully
-- Include camera position in control frames
-- Support both update and full sync modes
-
-### Data Collectors
-- Inherit from BaseCollector
-- Implement fetch() and transform() methods
-- Support incremental updates
-- Handle API changes gracefully
-
-### UE5 Integration
-- Communicate via WebSocket
-- Send data frames at configurable intervals (default 5 min)
-- Support auto-cruise and manual modes
-- Optimize for 4K@120Hz rendering
-
-### Multi-User Security
-- JWT tokens with 15-minute expiration
-- Redis token blacklist for logout
-- Role-based access control (RBAC)
-- Audit logging for all actions
-
----
-
-## Output Format
-
-### When Writing Code
-```python
-# File: backend/app/services/collectors/top500.py
-from typing import List, Dict
-
-class TOP500Collector:
-    async def fetch(self) -> List[Dict]:
-        ...
-```
-
-### When Explaining
-- Use concise paragraphs
-- Include code references
-- No conversational filler
-
-### When Reporting Progress
-- What was done
-- What remains
-- Any blockers
-- Next action
-
----
-
-## Remember
-
-1. **Rules are hard constraints** - follow `rules.md` absolutely
-2. **Context provides understanding** - use `project_context.md` for background
-3. **Role defines behavior** - follow `agents.md` for how to work
-4. **Quality over speed** - Enterprise systems require precision
-5. **Communicate clearly** - Precision in, precision out

aiprovider/Dockerfile

@@ -1,7 +1,8 @@
-# syntax=docker/dockerfile:1.7
+# Use BuildKit's bundled frontend to avoid a separate Docker Hub fetch.
 
 ARG PYTHON_IMAGE=python:3.14-slim
 ARG UV_IMAGE=ghcr.io/astral-sh/uv:latest
+ARG AI_PROVIDER_BUILD_FINGERPRINT=unknown
 
 FROM ${UV_IMAGE} AS uv
 FROM ${PYTHON_IMAGE}
@@ -15,19 +16,25 @@ ENV PYTHONUNBUFFERED=1
 ENV UV_COMPILE_BYTECODE=1
 ENV UV_LINK_MODE=copy
 
+RUN mkdir -p /root/.config/uv
+
 RUN apt-get update && apt-get install -y --no-install-recommends \
     curl \
     && rm -rf /var/lib/apt/lists/*
 
 COPY pyproject.toml uv.lock /app/
 RUN --mount=type=cache,target=/root/.cache/uv \
-    uv sync --frozen --no-dev
+    --mount=type=secret,id=planet_uv_config,target=/root/.config/uv/uv.toml,required=false \
+    uv sync --frozen --only-group aiprovider
 
 COPY aiprovider /app/aiprovider
 
+ARG AI_PROVIDER_BUILD_FINGERPRINT
+LABEL planet.aiprovider.build-fingerprint="${AI_PROVIDER_BUILD_FINGERPRINT}"
+
 EXPOSE 8010
 
 HEALTHCHECK --interval=30s --timeout=5s --retries=3 \
     CMD curl -fsS http://127.0.0.1:8010/health >/dev/null || exit 1
 
-CMD ["uv", "run", "--frozen", "--no-dev", "--project", "/app", "python", "-m", "uvicorn", "aiprovider.main:app", "--host", "0.0.0.0", "--port", "8010"]
+CMD ["/app/.venv/bin/python", "-m", "uvicorn", "aiprovider.main:app", "--host", "0.0.0.0", "--port", "8010"]

aiprovider/README.md

@@ -4,7 +4,7 @@
 
 完整使用说明见:
 
-- [docs/agents/aiprovider.md](/home/ray/dev/linkong/planet/docs/agents/aiprovider.md)
+- [AI Provider 指南](../docs/technical/zh/agents-aiprovider.md)
 
 当前支持:
 
@@ -15,6 +15,7 @@
   - `AI_PROVIDER=ollama`
 - request adapter:
   - `AI_PROVIDER_API=openai-completions`
+  - `AI_PROVIDER_API=openai-responses`
   - `AI_PROVIDER_API=anthropic-messages`
   - `AI_PROVIDER_API=ollama-generate`
 

aiprovider/provider_service.py

@@ -3,6 +3,7 @@ from __future__ import annotations
 import asyncio
 import json
 from typing import Any
+from uuid import NAMESPACE_URL, uuid4, uuid5
 
 import httpx
 from fastapi import HTTPException, status
@@ -65,6 +66,7 @@ class ProviderService:
         self.model_provider_apis = self._parse_model_provider_apis(
             overrides.get("model_provider_apis")
         )
+        self.session_id = str(uuid4())
 
     def get_status(self) -> AIProviderStatusResponse:
         enabled = self.provider != "disabled"
@@ -95,6 +97,8 @@ class ProviderService:
             )
 
         prompt = self._build_prompt(payload)
+        if payload.context.get("session_id") is not None:
+            self.session_id = str(uuid5(NAMESPACE_URL, f"planet:{payload.context['session_id']}"))
 
         provider_api = self._resolve_model_provider_api(model)
 
@@ -102,6 +106,10 @@ class ProviderService:
             data = await self._request_openai_compatible(model, prompt, payload.system_prompt)
             content = self._extract_openai_content(data)
             content_blocks = self._extract_openai_blocks(data)
+        elif provider_api == "openai-responses":
+            data = await self._request_openai_responses(model, prompt, payload.system_prompt)
+            content_blocks = self._extract_responses_blocks(data)
+            content = "".join(block.text for block in content_blocks if block.text)
         elif provider_api == "anthropic-messages":
             data = await self._request_anthropic_messages(
                 model,
@@ -200,6 +208,39 @@ class ProviderService:
             request_body=request_body,
         )
 
+    async def _request_openai_responses(
+        self, model: str, prompt: str, system_prompt: str | None = None,
+    ) -> dict[str, Any]:
+        request_body: dict[str, Any] = {
+            "model": model, "input": prompt, "max_output_tokens": self.max_tokens, "store": False,
+        }
+        resolved_system_prompt = self._resolve_system_prompt(system_prompt)
+        if resolved_system_prompt:
+            request_body["instructions"] = resolved_system_prompt
+        return await self._post(
+            path="/responses",
+            headers={"Authorization": f"Bearer {self.api_key}", "Content-Type": "application/json"},
+            request_body=request_body,
+        )
+
+    def _extract_responses_blocks(self, payload: dict[str, Any]) -> list[AIContentBlock]:
+        blocks: list[AIContentBlock] = []
+        for item in payload.get("output") or []:
+            if not isinstance(item, dict):
+                continue
+            if item.get("type") == "message":
+                for part in item.get("content") or []:
+                    if not isinstance(part, dict):
+                        continue
+                    text = part.get("text") or part.get("refusal")
+                    if isinstance(text, str) and text:
+                        blocks.append(AIContentBlock(type="text", text=text))
+            elif item.get("type") == "reasoning":
+                for part in item.get("summary") or []:
+                    if isinstance(part, dict) and isinstance(part.get("text"), str):
+                        blocks.append(AIContentBlock(type="thinking", thinking=part["text"]))
+        return blocks
+
     async def _request_anthropic_messages(
         self,
         model: str,
@@ -226,7 +267,7 @@ class ProviderService:
         resolved_system_prompt = self._resolve_system_prompt(system_prompt)
         if resolved_system_prompt:
             request_body["system"] = resolved_system_prompt
-        resolved_thinking = self._resolve_anthropic_thinking(thinking)
+        resolved_thinking = self._resolve_anthropic_thinking(thinking, model)
         if resolved_thinking:
             request_body["thinking"] = resolved_thinking
         if self.provider == "minimax" and self.base_url.endswith("/anthropic"):
@@ -243,8 +284,12 @@ class ProviderService:
             request_body=request_body,
         )
 
-    def _resolve_anthropic_thinking(self, thinking: dict[str, Any] | None) -> dict[str, Any] | None:
+    def _resolve_anthropic_thinking(
+        self, thinking: dict[str, Any] | None, model: str,
+    ) -> dict[str, Any] | None:
         if thinking:
+            if model.casefold() == "minimax-m3" and thinking.get("type") == "enabled":
+                return {"type": "adaptive"}
             return thinking
 
         # OpenClaw treats MiniMax's Anthropic-compatible path specially:
@@ -295,6 +340,9 @@ class ProviderService:
         headers: dict[str, str],
         request_body: dict[str, Any],
     ) -> dict[str, Any]:
+        headers = {"User-Agent": "Planet/1.0", **headers}
+        if self.provider == "opencode-go":
+            headers["x-opencode-session"] = self.session_id
         last_error: Exception | None = None
         for attempt in range(1, self.http_retry_attempts + 1):
             try:

backend/Dockerfile

@@ -1,3 +1,5 @@
+# syntax=docker/dockerfile:1.7
+
 ARG PYTHON_IMAGE=python:3.14-slim
 ARG UV_IMAGE=ghcr.io/astral-sh/uv:latest
 
@@ -14,14 +16,24 @@ ENV UV_COMPILE_BYTECODE=1
 ENV UV_LINK_MODE=copy
 ENV PYTHONPATH=/app/backend
 
+RUN mkdir -p /root/.config/uv
+
 RUN apt-get update && apt-get install -y --no-install-recommends \
     curl \
     && rm -rf /var/lib/apt/lists/*
 
 COPY pyproject.toml uv.lock /app/
-RUN uv sync --frozen --no-dev
+RUN --mount=type=cache,target=/root/.cache/uv \
+    --mount=type=secret,id=planet_uv_config,target=/root/.config/uv/uv.toml,required=false \
+    uv sync --frozen --no-dev
 
 COPY backend /app/backend
+COPY config/earth-boundary-pov-policy.china-v1.json /app/config/earth-boundary-pov-policy.china-v1.json
+COPY scripts/build_earth_boundary_china_pov_geojson.py /app/scripts/build_earth_boundary_china_pov_geojson.py
+COPY scripts/build_earth_boundary_pmtiles.py /app/scripts/build_earth_boundary_pmtiles.py
+COPY scripts/earth_boundary_policy.py /app/scripts/earth_boundary_policy.py
+COPY resources/earth/boundary-pov-geometry.china-v1.json /app/resources/earth/boundary-pov-geometry.china-v1.json
+COPY frontend/public/earth/assets/cards/manifest.json /app/frontend/public/earth/assets/cards/manifest.json
 COPY VERSION /app/VERSION
 
 EXPOSE 8000

backend/app/api/main.py

@@ -1,3 +1,4 @@
+from app.api.v1 import card_images
 from fastapi import APIRouter
 from app.api.v1 import (
     ai,
@@ -12,6 +13,7 @@ from app.api.v1 import (
     alerts,
     settings,
     collected_data,
+    compute_intelligence,
     data_products,
     layers,
     visualization,
@@ -19,6 +21,7 @@ from app.api.v1 import (
     vessels,
     bgp,
     news,
+    interactables,
     realtime_sources,
     system_control,
     tv,
@@ -42,6 +45,7 @@ api_router.include_router(alerts.router, prefix="/alerts", tags=["alerts"])
 api_router.include_router(settings.router, prefix="/settings", tags=["settings"])
 api_router.include_router(system_control.router, prefix="/system", tags=["system"])
 api_router.include_router(data_products.router, prefix="/data-products", tags=["data-products"])
+api_router.include_router(compute_intelligence.router, prefix="/data-products/compute", tags=["compute"])
 api_router.include_router(layers.router, prefix="/layers", tags=["layers"])
 api_router.include_router(visualization.router, prefix="/visualization", tags=["visualization"])
 api_router.include_router(
@@ -53,4 +57,7 @@ api_router.include_router(vessels.router, prefix="/vessels", tags=["vessels"])
 api_router.include_router(bgp.router, prefix="/bgp", tags=["bgp"])
 api_router.include_router(tv.router, prefix="/tv", tags=["tv"])
 api_router.include_router(news.router, prefix="/news", tags=["news"])
+api_router.include_router(interactables.router, prefix="/interactables", tags=["interactables"])
 api_router.include_router(realtime_sources.router, prefix="/realtime-sources", tags=["realtime-sources"])
+
+api_router.include_router(card_images.router, prefix="/earth/card-images", tags=["card-images"])

backend/app/api/v1/ai.py

@@ -3,6 +3,7 @@ from uuid import uuid4
 from fastapi import APIRouter, Depends, HTTPException, Request, Response
 from sqlalchemy.ext.asyncio import AsyncSession
 
+from app.core.logging import get_logger
 from app.core.security import get_current_user
 from app.db.session import get_db
 from app.models.user import User
@@ -47,8 +48,10 @@ from app.services.playground_chat_service import (
     stop_message,
 )
 from app.services.situational_alert_ai_brief import build_situational_alert_brief_request
+from app.services.business_logs import emit_business_log, exception_context
 
 router = APIRouter()
+logger = get_logger(__name__, service="api")
 
 
 @router.get("/provider/status", response_model=AIProviderStatusResponse)
@@ -122,6 +125,16 @@ async def create_playground_message(
     provider_client: AIProviderClient = Depends(get_ai_provider_client),
     db: AsyncSession = Depends(get_db),
 ):
+    await emit_business_log(
+        logger,
+        event="ai.playground.message.create",
+        message="Playground message creation requested",
+        category="ai",
+        service="api",
+        module=__name__,
+        user_id=current_user.id,
+        context={"session_key": payload.session_key, "preset": payload.selected_preset_key},
+    )
     return await create_turn(
         db,
         user_id=current_user.id,
@@ -136,6 +149,16 @@ async def stop_playground_message(
     current_user: User = Depends(get_current_user),
     db: AsyncSession = Depends(get_db),
 ):
+    await emit_business_log(
+        logger,
+        event="ai.playground.message.stop",
+        message="Playground message stop requested",
+        category="ai",
+        service="api",
+        module=__name__,
+        user_id=current_user.id,
+        context={"session_key": payload.session_key, "message_id": payload.message_id},
+    )
     return await stop_message(
         db,
         user_id=current_user.id,
@@ -150,6 +173,16 @@ async def resend_playground_message(
     provider_client: AIProviderClient = Depends(get_ai_provider_client),
     db: AsyncSession = Depends(get_db),
 ):
+    await emit_business_log(
+        logger,
+        event="ai.playground.message.resend",
+        message="Playground message resend requested",
+        category="ai",
+        service="api",
+        module=__name__,
+        user_id=current_user.id,
+        context={"session_key": payload.session_key, "user_message_id": payload.user_message_id},
+    )
     return await resend_turn(
         db,
         user_id=current_user.id,
@@ -214,16 +247,70 @@ async def analyze_bgp_brief(
         anomaly_limit=payload.anomaly_limit,
         collector_limit=payload.collector_limit,
     )
+    await emit_business_log(
+        logger,
+        event="ai.brief.bgp.facts_collected",
+        message="BGP brief facts collected",
+        category="ai",
+        service="api",
+        module=__name__,
+        request_id=request_id,
+        user_id=current_user.id,
+        context={
+            "incident_limit": payload.incident_limit,
+            "anomaly_limit": payload.anomaly_limit,
+            "collector_limit": payload.collector_limit,
+            "fact_count": len(facts or []),
+        },
+    )
     brief_request.preferred_model = payload.preferred_model
     brief_request.thinking = payload.thinking
 
-    analysis = await provider_client.analyze(brief_request, request_id=request_id)
-    return save_bgp_brief_record(
-        analysis,
+    await emit_business_log(
+        logger,
+        event="ai.brief.bgp.start",
+        message="BGP brief AI analysis started",
+        category="ai",
+        service="api",
+        module=__name__,
         request_id=request_id,
-        facts=facts,
-        context=context,
+        user_id=current_user.id,
+        context={"preferred_model": payload.preferred_model},
     )
+    try:
+        analysis = await provider_client.analyze(brief_request, request_id=request_id)
+        record = save_bgp_brief_record(
+            analysis,
+            request_id=request_id,
+            facts=facts,
+            context=context,
+        )
+        await emit_business_log(
+            logger,
+            event="ai.brief.bgp.completed",
+            message="BGP brief AI analysis saved",
+            category="ai",
+            service="api",
+            module=__name__,
+            request_id=request_id,
+            user_id=current_user.id,
+            context={"provider": analysis.provider, "model": analysis.model, "brief_id": record.id},
+        )
+        return record
+    except Exception as exc:
+        await emit_business_log(
+            logger,
+            event="ai.brief.bgp.failed",
+            message="BGP brief AI analysis failed",
+            category="ai",
+            level="error",
+            service="api",
+            module=__name__,
+            request_id=request_id,
+            user_id=current_user.id,
+            context=exception_context(exc, {"preferred_model": payload.preferred_model}),
+        )
+        raise
 
 
 @router.post("/alerts/brief", response_model=AlertBriefResponse)
@@ -242,17 +329,65 @@ async def analyze_alert_brief(
         db,
         alert_limit=payload.alert_limit,
     )
+    await emit_business_log(
+        logger,
+        event="ai.brief.alerts.facts_collected",
+        message="Alert brief facts collected",
+        category="ai",
+        service="api",
+        module=__name__,
+        request_id=request_id,
+        user_id=current_user.id,
+        context={"alert_limit": payload.alert_limit, "fact_count": len(facts or [])},
+    )
     brief_request.preferred_model = payload.preferred_model
     brief_request.thinking = payload.thinking
 
-    analysis = await provider_client.analyze(brief_request, request_id=request_id)
-    return AlertBriefResponse(
-        **analysis.model_dump(),
-        title=brief_request.title,
-        objective=brief_request.objective,
-        facts=facts,
-        context=context,
+    await emit_business_log(
+        logger,
+        event="ai.brief.alerts.start",
+        message="Alert brief AI analysis started",
+        category="ai",
+        service="api",
+        module=__name__,
+        request_id=request_id,
+        user_id=current_user.id,
+        context={"preferred_model": payload.preferred_model},
     )
+    try:
+        analysis = await provider_client.analyze(brief_request, request_id=request_id)
+        await emit_business_log(
+            logger,
+            event="ai.brief.alerts.completed",
+            message="Alert brief AI analysis completed",
+            category="ai",
+            service="api",
+            module=__name__,
+            request_id=request_id,
+            user_id=current_user.id,
+            context={"provider": analysis.provider, "model": analysis.model},
+        )
+        return AlertBriefResponse(
+            **analysis.model_dump(),
+            title=brief_request.title,
+            objective=brief_request.objective,
+            facts=facts,
+            context=context,
+        )
+    except Exception as exc:
+        await emit_business_log(
+            logger,
+            event="ai.brief.alerts.failed",
+            message="Alert brief AI analysis failed",
+            category="ai",
+            level="error",
+            service="api",
+            module=__name__,
+            request_id=request_id,
+            user_id=current_user.id,
+            context=exception_context(exc, {"preferred_model": payload.preferred_model}),
+        )
+        raise
 
 
 @router.post("/situational-alerts/brief", response_model=SituationalAlertBriefResponse)
@@ -268,14 +403,62 @@ async def analyze_situational_alert_brief(
     response.headers["X-Request-ID"] = request_id
 
     brief_request, facts, context = await build_situational_alert_brief_request(db)
+    await emit_business_log(
+        logger,
+        event="ai.brief.situational_alerts.facts_collected",
+        message="Situational alert brief facts collected",
+        category="ai",
+        service="api",
+        module=__name__,
+        request_id=request_id,
+        user_id=current_user.id,
+        context={"fact_count": len(facts or [])},
+    )
     brief_request.preferred_model = payload.preferred_model
     brief_request.thinking = payload.thinking
 
-    analysis = await provider_client.analyze(brief_request, request_id=request_id)
-    return SituationalAlertBriefResponse(
-        **analysis.model_dump(),
-        title=brief_request.title,
-        objective=brief_request.objective,
-        facts=facts,
-        context=context,
+    await emit_business_log(
+        logger,
+        event="ai.brief.situational_alerts.start",
+        message="Situational alert brief AI analysis started",
+        category="ai",
+        service="api",
+        module=__name__,
+        request_id=request_id,
+        user_id=current_user.id,
+        context={"preferred_model": payload.preferred_model},
     )
+    try:
+        analysis = await provider_client.analyze(brief_request, request_id=request_id)
+        await emit_business_log(
+            logger,
+            event="ai.brief.situational_alerts.completed",
+            message="Situational alert brief AI analysis completed",
+            category="ai",
+            service="api",
+            module=__name__,
+            request_id=request_id,
+            user_id=current_user.id,
+            context={"provider": analysis.provider, "model": analysis.model},
+        )
+        return SituationalAlertBriefResponse(
+            **analysis.model_dump(),
+            title=brief_request.title,
+            objective=brief_request.objective,
+            facts=facts,
+            context=context,
+        )
+    except Exception as exc:
+        await emit_business_log(
+            logger,
+            event="ai.brief.situational_alerts.failed",
+            message="Situational alert brief AI analysis failed",
+            category="ai",
+            level="error",
+            service="api",
+            module=__name__,
+            request_id=request_id,
+            user_id=current_user.id,
+            context=exception_context(exc, {"preferred_model": payload.preferred_model}),
+        )
+        raise

backend/app/api/v1/auth.py

@@ -4,6 +4,7 @@ from sqlalchemy.ext.asyncio import AsyncSession
 from sqlalchemy import text
 
 from app.core.config import settings
+from app.core.enums import OtpPurpose, UserRole
 from app.core.logging import get_logger
 from app.core.security import (
     create_access_token,
@@ -170,7 +171,7 @@ async def get_me(current_user: User = Depends(get_current_user)):
     }
 
 
-async def _send_code_or_raise(db: AsyncSession, email: str, code: str, purpose: str) -> None:
+async def _send_code_or_raise(db: AsyncSession, email: str, code: str, purpose: OtpPurpose) -> None:
     try:
         await send_verification_email(db, to=email, code=code, purpose=purpose)
     except EmailNotConfiguredError as exc:
@@ -207,7 +208,7 @@ async def register(payload: UserRegister, db: AsyncSession = Depends(get_db)):
         username=payload.username,
         email=payload.email,
         password_hash=get_password_hash(payload.password),
-        role="viewer",
+        role=UserRole.VIEWER.value,
         is_active=True,
         email_verified=False,
     )
@@ -215,13 +216,13 @@ async def register(payload: UserRegister, db: AsyncSession = Depends(get_db)):
     await db.commit()
 
     try:
-        code = otp.issue_code(payload.email, "register")
+        code = otp.issue_code(payload.email, OtpPurpose.REGISTER)
     except otp.OtpResendRateLimited as exc:
         raise HTTPException(
             status_code=status.HTTP_429_TOO_MANY_REQUESTS,
             detail={"code": exc.code, "retry_after_seconds": exc.retry_after_seconds},
         ) from exc
-    await _send_code_or_raise(db, payload.email, code, "register")
+    await _send_code_or_raise(db, payload.email, code, OtpPurpose.REGISTER)
     return {"status": "pending_verification", "email": payload.email}
 
 
@@ -266,7 +267,7 @@ async def resend_code(payload: ResendCodeRequest, db: AsyncSession = Depends(get
     if user is None:
         # Avoid email enumeration; pretend success.
         return {"status": "ok"}
-    if payload.purpose == "register" and user.email_verified:
+    if payload.purpose is OtpPurpose.REGISTER and user.email_verified:
         raise HTTPException(
             status_code=status.HTTP_409_CONFLICT,
             detail={"code": "ALREADY_VERIFIED"},
@@ -289,12 +290,17 @@ async def forgot_password(payload: ForgotPasswordRequest, db: AsyncSession = Dep
         # Don't leak whether an email is registered.
         return {"status": "ok"}
     try:
-        code = otp.issue_code(payload.email, "reset_password")
+        code = otp.issue_code(payload.email, OtpPurpose.RESET_PASSWORD)
     except otp.OtpResendRateLimited:
         # Silently accept; the user can retry after the cooldown.
         return {"status": "ok"}
     try:
-        await send_verification_email(db, to=payload.email, code=code, purpose="reset_password")
+        await send_verification_email(
+            db,
+            to=payload.email,
+            code=code,
+            purpose=OtpPurpose.RESET_PASSWORD,
+        )
     except EmailNotConfiguredError as exc:
         raise HTTPException(
             status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
@@ -304,7 +310,11 @@ async def forgot_password(payload: ForgotPasswordRequest, db: AsyncSession = Dep
         logger.warning_event(
             "SMTP send failed",
             event="auth.email.send_failed",
-            context={"email": payload.email, "purpose": "reset_password", "error": str(exc)},
+            context={
+                "email": payload.email,
+                "purpose": OtpPurpose.RESET_PASSWORD.value,
+                "error": str(exc),
+            },
         )
     return {"status": "ok"}
 
@@ -318,7 +328,7 @@ async def reset_password(payload: ResetPasswordRequest, db: AsyncSession = Depen
             detail={"code": "OTP_INVALID"},
         )
     try:
-        otp.verify_code(payload.email, "reset_password", payload.code)
+        otp.verify_code(payload.email, OtpPurpose.RESET_PASSWORD, payload.code)
     except otp.OtpExpired as exc:
         raise HTTPException(
             status_code=status.HTTP_410_GONE,

backend/app/api/v1/card_images.py

@@ -0,0 +1,108 @@
+"""Card-image configuration, upload, catalog selection and public resolution."""
+
+from fastapi import APIRouter, Depends, File, HTTPException, Query, Response, UploadFile
+from fastapi.responses import FileResponse
+from sqlalchemy.ext.asyncio import AsyncSession
+from starlette.concurrency import run_in_threadpool
+
+from app.core.enums import UserRole
+from app.core.security import get_current_user
+from app.db.session import get_db
+from app.models.user import User
+from app.schemas.card_images import (
+    CardImageConfiguration,
+    CardImagePreview,
+    CardImagePublish,
+    EntityType,
+    MatchScope,
+)
+from app.services.card_image_configuration import (
+    publish_configuration,
+    read_configuration,
+    resolve_image,
+)
+from app.services.card_image_files import MAX_UPLOAD_BYTES, store_image, uploaded_path
+from app.services.card_image_targets import get_target, search_targets
+
+router = APIRouter()
+
+
+async def image_admin(user: User = Depends(get_current_user)) -> User:
+    if user.role not in {UserRole.ADMIN.value, UserRole.SUPER_ADMIN.value}:
+        raise HTTPException(403, "Administrator access required")
+    return user
+
+
+@router.get("/configuration")
+async def get_configuration(
+    response: Response, _actor: User = Depends(image_admin), db: AsyncSession = Depends(get_db)
+):
+    response.headers["Cache-Control"] = "no-store"
+    return await read_configuration(db)
+
+
+@router.put("/configuration")
+async def save_configuration(
+    payload: CardImagePublish,
+    actor: User = Depends(image_admin),
+    db: AsyncSession = Depends(get_db),
+):
+    return await publish_configuration(db, payload.revision, payload.config, actor)
+
+
+@router.post("/upload")
+async def upload_image(file: UploadFile = File(...), _actor: User = Depends(image_admin)):
+    content = await file.read(MAX_UPLOAD_BYTES + 1)
+    await file.close()
+    return await run_in_threadpool(store_image, content, (file.filename or "Image")[:200])
+
+
+@router.get("/files/{filename}")
+async def image_file(filename: str):
+    path = uploaded_path(filename)
+    if not path.is_file():
+        raise HTTPException(404, "Image not found")
+    return FileResponse(
+        path,
+        headers={
+            "Cache-Control": "public, max-age=31536000, immutable",
+            "X-Content-Type-Options": "nosniff",
+        },
+    )
+
+
+@router.get("/targets")
+async def targets(
+    entity_type: EntityType,
+    scope: MatchScope = "object",
+    q: str = Query(default="", max_length=100),
+    limit: int = Query(default=25, ge=1, le=50),
+    _actor: User = Depends(image_admin),
+    db: AsyncSession = Depends(get_db),
+):
+    return {"items": await search_targets(db, entity_type, scope, q.strip(), limit)}
+
+
+@router.get("/resolve")
+async def resolve(
+    response: Response,
+    entity_type: EntityType,
+    entity_id: str = Query(min_length=1, max_length=300),
+    db: AsyncSession = Depends(get_db),
+):
+    response.headers["Cache-Control"] = "no-store"
+    envelope = await read_configuration(db)
+    config = CardImageConfiguration.model_validate(envelope["config"])
+    context = await get_target(db, entity_type, entity_id)
+    result = resolve_image(config, context)
+    return {**result, "revision": envelope["revision"], "entity": context}
+
+
+@router.post("/preview")
+async def preview(
+    payload: CardImagePreview,
+    _actor: User = Depends(image_admin),
+    db: AsyncSession = Depends(get_db),
+):
+    context = await get_target(db, payload.entity_type, payload.entity_id)
+    return {**resolve_image(payload.config, context), "entity": context}

backend/app/api/v1/compute_intelligence.py

@@ -0,0 +1,235 @@
+"""Compute observations, administrator configuration and independent facility samples."""
+
+from datetime import UTC, datetime
+from typing import Literal
+
+from app.core.compute_types import COMPUTE_SITE_TYPES, compute_record_filter, compute_site_type_expression
+from app.api.v1.visualization import convert_compute_centers_to_geojson
+from app.core.earth_geography_policy import display_country
+from app.core.enums import UserRole
+from app.core.security import get_current_user
+from app.db.session import get_db
+from app.models.collected_data import CollectedData
+from app.models.data_snapshot import DataSnapshot
+from app.models.user import User
+from app.schemas.compute_configuration import (
+    ComputeConfiguration,
+    ComputeConfigurationReset,
+    ComputeConfigurationWrite,
+)
+from app.services.compute_catalog import (
+    PARTICIPANTS,
+    participant_members,
+    source_country_names,
+    source_country_participants,
+)
+from app.services.compute_center_locations import refresh_compute_center_location_cache
+from app.services.compute_configuration import (
+    DEFAULT_CONFIGURATION,
+    configuration_response,
+    publish_configuration,
+    read_configuration,
+)
+from app.services.compute_history import compute_history
+from fastapi import APIRouter, Depends, HTTPException, Query, Response
+from sqlalchemy import and_, case, func, or_, select
+from sqlalchemy.ext.asyncio import AsyncSession
+
+router = APIRouter()
+
+
+def selected_countries(countries: str) -> list[str]:
+    codes = list(dict.fromkeys(code.strip().upper() for code in countries.split(",")))
+    if not codes or len(codes) > 12 or any(code not in PARTICIPANTS for code in codes):
+        raise HTTPException(422, "Select 1–12 participants from the compute catalog")
+    members = participant_members(codes)
+    flattened = [country for participant in members.values() for country in participant]
+    if len(flattened) != len(set(flattened)):
+        raise HTTPException(422, "Comparison participants cannot have overlapping country members")
+    return codes
+
+
+@router.get("/series")
+async def get_series(
+    response: Response,
+    countries: str = "CN,US",
+    mode: Literal["global_share", "bilateral_share", "absolute"] = "global_share",
+    db: AsyncSession = Depends(get_db),
+):
+    codes = selected_countries(countries)
+    envelope = await read_configuration(db)
+    response.headers["Cache-Control"] = "no-store"
+    result = compute_history(codes, mode, ComputeConfiguration.model_validate(envelope["config"]))
+    result.update(
+        version=envelope["revision"],
+        provenance="bundled_reports" if envelope["is_default"] else "admin_curated_reports",
+    )
+    return result
+
+
+async def configuration_admin(user: User = Depends(get_current_user)) -> User:
+    if user.role not in {UserRole.ADMIN.value, UserRole.SUPER_ADMIN.value}:
+        raise HTTPException(403, "Administrator access required")
+    return user
+
+
+@router.get("/configuration")
+async def get_configuration(
+    response: Response,
+    _actor: User = Depends(configuration_admin),
+    db: AsyncSession = Depends(get_db),
+):
+    response.headers["Cache-Control"] = "no-store"
+    return configuration_response(await read_configuration(db))
+
+
+@router.put("/configuration")
+async def update_configuration(
+    payload: ComputeConfigurationWrite,
+    actor: User = Depends(configuration_admin),
+    db: AsyncSession = Depends(get_db),
+):
+    envelope = await publish_configuration(
+        db, revision=payload.revision, config=payload.config, actor=actor,
+    )
+    return configuration_response(envelope)
+
+
+@router.post("/configuration/reset")
+async def reset_configuration(
+    payload: ComputeConfigurationReset,
+    actor: User = Depends(configuration_admin),
+    db: AsyncSession = Depends(get_db),
+):
+    envelope = await publish_configuration(
+        db, revision=payload.revision, config=DEFAULT_CONFIGURATION, actor=actor, reset=True,
+    )
+    return configuration_response(envelope)
+
+
+async def sample_query(db: AsyncSession, countries: list[str], year: int | None):
+    now = datetime.now(UTC)
+    cutoff = min(datetime(year + 1, 1, 1, tzinfo=UTC), now) if year else now
+    ranked = (
+        select(
+            DataSnapshot.id,
+            func.row_number()
+            .over(
+                partition_by=DataSnapshot.source,
+                order_by=(DataSnapshot.completed_at.desc(), DataSnapshot.id.desc()),
+            )
+            .label("rank"),
+        )
+        .where(
+            DataSnapshot.source.in_(select(CollectedData.source).where(compute_record_filter(CollectedData)).distinct()),
+            DataSnapshot.status == "success",
+            DataSnapshot.completed_at < cutoff,
+        )
+        .subquery()
+    )
+    snapshots = select(ranked.c.id).where(ranked.c.rank == 1)
+    infrastructure = compute_site_type_expression(CollectedData).in_(COMPUTE_SITE_TYPES[2:])
+    # Old collector rows have ambiguous units/identity and must not become
+    # national evidence merely because a new UI can display them.
+    query = select(CollectedData).where(
+        compute_record_filter(CollectedData),
+        CollectedData.deleted_at.is_(None),
+        CollectedData.snapshot_id.in_(snapshots),
+        CollectedData.is_valid == 1,
+        or_(CollectedData.extra_data["compute_contract"].as_string() == "v1", infrastructure),
+        or_(
+            infrastructure,
+            CollectedData.source == "top500",
+            and_(
+                CollectedData.extra_data["status"].as_string() == "Existing",
+                func.coalesce(CollectedData.extra_data["superseded_by"].as_string(), "") == "",
+                CollectedData.extra_data["possible_duplicate"].as_boolean().is_not(True),
+            ),
+        ),
+        CollectedData.extra_data["country"].as_string().in_(source_country_names(countries)),
+    )
+    return query
+
+
+@router.get("/facilities")
+async def get_facilities(
+    countries: str = "CN,US",
+    year: int | None = Query(None, ge=2000, le=2100),
+    limit: int = Query(1000, ge=1, le=2000),
+    db: AsyncSession = Depends(get_db),
+):
+    codes = selected_countries(countries)
+    query = await sample_query(db, codes, year)
+    country = CollectedData.extra_data["country"].as_string()
+    source_participants = source_country_participants(codes)
+    # Canonical source subdivisions retain their raw label on each record but
+    # share one owner in the comparison, using the same policy as the globe.
+    owner_cases = [
+        (country == name, display_country(name))
+        for name in source_country_names(codes)
+        if display_country(name) != name
+    ]
+    display = case(*owner_cases, else_=country) if owner_cases else country
+    participant_cases = [(country == name, code) for name, code in source_participants.items()]
+    participant = case(*participant_cases, else_=None)
+    counts = (
+        (
+            await db.execute(
+                query.with_only_columns(
+                    display.label("country"),
+                    participant.label("participant_code"),
+                    CollectedData.source,
+                    func.count().label("count"),
+                ).group_by(display, participant, CollectedData.source)
+            )
+        )
+        .mappings()
+        .all()
+    )
+    participant_counts = (
+        (
+            await db.execute(
+                query.with_only_columns(
+                    participant.label("participant_code"),
+                    CollectedData.source,
+                    func.count().label("count"),
+                ).group_by(participant, CollectedData.source)
+            )
+        )
+        .mappings()
+        .all()
+    )
+    records = (
+        (await db.execute(query.order_by(CollectedData.name, CollectedData.id).limit(limit)))
+        .scalars()
+        .all()
+    )
+    if year:
+        await refresh_compute_center_location_cache(db, as_of=datetime(year + 1, 1, 1, tzinfo=UTC))
+    else:
+        await refresh_compute_center_location_cache(db)
+    payload = convert_compute_centers_to_geojson(records)
+    for feature in payload["features"]:
+        feature["properties"]["historical_snapshot"] = year is not None
+        properties = feature["properties"]
+        properties["source_country"] = properties.get("country")
+        properties["country"] = display_country(properties.get("country"))
+    total = sum(row["count"] for row in counts)
+    payload.update(
+        {
+            "counts": [dict(row) for row in counts],
+            "participant_counts": [dict(row) for row in participant_counts],
+            "participants": [PARTICIPANTS[code] for code in codes],
+            "total_count": total,
+            "returned_count": len(records),
+            "truncated": total > limit,
+            "reference_year": year,
+            "scope": "public_source_records",
+            "history_basis": (
+                "snapshots_collected_by_year_end" if year else "latest_successful_snapshot"
+            ),
+            "coverage": "available" if total else "no_verified_snapshot",
+            "cross_source_deduplicated": False,
+        }
+    )
+    return payload

backend/app/api/v1/data_products.py

@@ -52,7 +52,7 @@ PRODUCT_DEFINITIONS: dict[str, dict] = {
         "name": "算力",
         "sources": ["top500", "epoch_ai_gpu"],
         "primary_stat_key": "compute_center_count",
-        "stat_keys": ["compute_center_count", "supercomputer_count", "gpu_cluster_count"],
+        "stat_keys": ["compute_center_count", "supercomputer_count", "gpu_cluster_count", "data_center_count", "cloud_center_count", "edge_node_count"],
     },
 }
 

backend/app/api/v1/datasource_config.py

@@ -13,6 +13,7 @@ import httpx
 
 from app.core.target_schema_registry import get_target_schema, list_target_schemas
 from app.core.datasource_defaults import DEFAULT_DATASOURCES
+from app.core.enums import AuthType, MappingValidationStatus, UserRole
 from app.db.session import get_db
 from app.models.user import User
 from app.models.datasource_config import DataSourceConfig
@@ -34,7 +35,6 @@ from app.services.datasource_mapping import (
 )
 from app.services.custom_datasource_runtime import (
     CustomDatasourceRuntimeError,
-    fetch_rest_payload,
     get_custom_stream_status,
     run_mapped_rest_config,
     run_mapped_websocket_config,
@@ -42,8 +42,6 @@ from app.services.custom_datasource_runtime import (
     stop_custom_stream,
     test_websocket_config,
 )
-
-DATASOURCE_MAPPING_PROMPT_KEY = "datasource.mapping"
 from app.services.datasource_connectivity import (
     _resolve_aisstream_api_key,
     _resolve_spacetrack_credentials_with_override,
@@ -57,7 +55,8 @@ from app.services.persistent_logs import record_audit_log
 
 router = APIRouter()
 
-SECRET_REVEAL_ROLES = {"admin", "super_admin"}
+DATASOURCE_MAPPING_PROMPT_KEY = "datasource.mapping"
+SECRET_REVEAL_ROLES = {UserRole.ADMIN.value, UserRole.SUPER_ADMIN.value}
 
 
 def _user_role_value(user: User) -> str:
@@ -124,7 +123,7 @@ class DataSourceConfigCreate(BaseModel):
     description: Optional[str] = None
     source_type: str = Field(..., description="rest, websocket, http, api, database")
     endpoint: str = Field(..., max_length=500)
-    auth_type: str = Field(default="none", description="none, bearer, api_key, basic")
+    auth_type: AuthType = Field(default=AuthType.NONE, description="none, bearer, api_key, basic")
     auth_config: dict = Field(default={})
     headers: dict = Field(default={})
     config: dict = Field(default={"timeout": 30, "retry": 3})
@@ -135,7 +134,7 @@ class DataSourceConfigUpdate(BaseModel):
     description: Optional[str] = None
     source_type: Optional[str] = None
     endpoint: Optional[str] = Field(None, max_length=500)
-    auth_type: Optional[str] = None
+    auth_type: Optional[AuthType] = None
     auth_config: Optional[dict] = None
     headers: Optional[dict] = None
     config: Optional[dict] = None
@@ -210,7 +209,7 @@ class MappingTemplateCreate(BaseModel):
     mapping_json: dict
     sample_payload: Any | None = None
     sample_payload_hash: Optional[str] = None
-    validation_status: str = Field(default="draft", pattern="^(draft|valid|invalid)$")
+    validation_status: MappingValidationStatus = MappingValidationStatus.DRAFT
     is_active: bool = False
 
 
@@ -219,7 +218,7 @@ class MappingTemplateUpdate(BaseModel):
     mapping_json: Optional[dict] = None
     sample_payload: Any | None = None
     sample_payload_hash: Optional[str] = None
-    validation_status: Optional[str] = Field(default=None, pattern="^(draft|valid|invalid)$")
+    validation_status: Optional[MappingValidationStatus] = None
     is_active: Optional[bool] = None
 
 
@@ -918,6 +917,7 @@ async def get_datasource_target_schemas(
 @router.post("/mappings/propose")
 async def propose_datasource_mapping(
     payload: MappingProposeRequest,
+    db: AsyncSession = Depends(get_db),
     current_user: User = Depends(get_current_user),
     ai_client: AIProviderClient = Depends(get_ai_provider_client),
 ):

backend/app/api/v1/datasources.py

@@ -1,4 +1,3 @@
-import asyncio
 from datetime import datetime, timedelta, timezone
 from typing import Optional
 
@@ -7,7 +6,8 @@ from pydantic import BaseModel, Field
 from sqlalchemy import func, or_, select, text
 from sqlalchemy.ext.asyncio import AsyncSession
 
-from app.core.cache import cache
+from app.core.logging import get_logger
+from app.core.enums import JobStatus, SnapshotStatus
 from app.core.time import to_iso8601_utc
 from app.core.security import get_current_user
 from app.core.data_sources import get_data_sources_config
@@ -20,18 +20,27 @@ from app.models.datasource_config import DataSourceConfig
 from app.models.task import CollectionTask
 from app.models.user import User
 from app.models.vessel import AISRawObservation
-from app.services.vessel_ais_aggregation import VESSEL_AIS_SCHEMA
 from app.services.scheduler import (
-    cancel_running_collector_now,
-    get_latest_task_id_for_datasource,
-    run_collector_now,
     sync_datasource_job,
 )
-from app.services.earth_layer_cache import invalidate_earth_layer_cache_for_source
+from app.services.data_jobs import (
+    JOB_STATUS_CANCELLING,
+    JOB_STATUS_QUEUED,
+    JOB_STATUS_RUNNING,
+    JOB_TYPE_CLEAR_CACHE,
+    JOB_TYPE_CLEAR_DATA,
+    JOB_TYPE_COLLECT,
+    enqueue_datasource_job,
+    get_active_datasource_job,
+    request_cancel_datasource_task,
+)
+from app.services.business_logs import emit_business_log
 
 router = APIRouter()
+logger = get_logger(__name__, service="api")
 STALE_RUNNING_TASK_TIMEOUT_MINUTES = 90
 
+
 PRODUCT_SOURCE_KEYWORDS: tuple[tuple[str, tuple[str, ...]], ...] = (
     ("vessels", ("vessel", "ais")),
     ("cables", ("cable", "landing", "telegeography", "arcgis", "fao")),
@@ -115,7 +124,7 @@ async def _load_latest_running_tasks(
             _task_rank_column(CollectionTask.started_at),
         )
         .where(CollectionTask.datasource_id.in_(datasource_ids))
-        .where(CollectionTask.status == "running")
+        .where(CollectionTask.status.in_((JOB_STATUS_QUEUED, JOB_STATUS_RUNNING, JOB_STATUS_CANCELLING)))
         .subquery()
     )
     result = await db.execute(
@@ -126,32 +135,6 @@ async def _load_latest_running_tasks(
     return {task.datasource_id: task for task in result.scalars().all()}
 
 
-async def _load_latest_task_ids(
-    db: AsyncSession,
-    datasource_ids: list[int],
-) -> dict[int, int]:
-    if not datasource_ids:
-        return {}
-
-    ranked_tasks = (
-        select(
-            CollectionTask.id.label("task_id"),
-            CollectionTask.datasource_id.label("datasource_id"),
-            func.row_number().over(
-                partition_by=CollectionTask.datasource_id,
-                order_by=CollectionTask.id.desc(),
-            ).label("row_num"),
-        )
-        .where(CollectionTask.datasource_id.in_(datasource_ids))
-        .subquery()
-    )
-    result = await db.execute(
-        select(ranked_tasks.c.datasource_id, ranked_tasks.c.task_id)
-        .where(ranked_tasks.c.row_num == 1)
-    )
-    return {datasource_id: task_id for datasource_id, task_id in result.all()}
-
-
 async def _load_latest_tasks(
     db: AsyncSession,
     datasource_ids: list[int],
@@ -182,6 +165,8 @@ async def _load_latest_tasks(
 async def _load_collected_record_counts(
     db: AsyncSession,
     sources: list[str],
+    *,
+    exact_vessel_counts: bool = False,
 ) -> dict[str, int]:
     if not sources:
         return {}
@@ -202,14 +187,46 @@ async def _load_collected_record_counts(
         or "ais" in source
     ]
     if vessel_sources:
-        raw_result = await db.execute(
-            select(AISRawObservation.source, func.count(AISRawObservation.id))
-            .where(AISRawObservation.target_schema == VESSEL_AIS_SCHEMA)
-            .where(AISRawObservation.source.in_(vessel_sources))
-            .group_by(AISRawObservation.source)
+        if exact_vessel_counts:
+            exact_result = await db.execute(
+                select(AISRawObservation.source, func.count(AISRawObservation.id))
+                .where(AISRawObservation.source.in_(vessel_sources))
+                .group_by(AISRawObservation.source)
+            )
+            for source, count in exact_result.all():
+                counts[source] = max(counts.get(source, 0), int(count or 0))
+            return counts
+
+        # AIS raw observations can be tens of millions of rows. Use planner
+        # statistics for the datasource list instead of blocking page load on
+        # source-level count(*) scans.
+        stats_result = await db.execute(
+            text(
+                """
+                SELECT
+                    COALESCE(pg_class.reltuples, 0)::bigint AS total_rows,
+                    pg_stats.most_common_vals::text AS source_values,
+                    pg_stats.most_common_freqs::text AS source_freqs
+                FROM pg_class
+                LEFT JOIN pg_stats
+                  ON pg_stats.schemaname = 'public'
+                 AND pg_stats.tablename = 'ais_raw_observations'
+                 AND pg_stats.attname = 'source'
+                WHERE pg_class.relname = 'ais_raw_observations'
+                LIMIT 1
+                """
+            )
         )
-        for source, count in raw_result.all():
-            counts[source] = max(counts.get(source, 0), int(count or 0))
+        stats = stats_result.mappings().first()
+        if stats:
+            total_rows = int(stats["total_rows"] or 0)
+            values = str(stats["source_values"] or "").strip("{}")
+            freqs = str(stats["source_freqs"] or "").strip("{}")
+            source_values = [value.strip('"') for value in values.split(",") if value]
+            source_freqs = [float(value) for value in freqs.split(",") if value]
+            for source, freq in zip(source_values, source_freqs):
+                if source in vessel_sources:
+                    counts[source] = max(counts.get(source, 0), int(round(total_rows * freq)))
 
     return counts
 
@@ -303,8 +320,11 @@ def serialize_datasource_row(
         "last_run": to_iso8601_utc(last_run_at),
         "last_run_at": to_iso8601_utc(last_run_at),
         "last_status": last_status,
-        "is_running": running_task is not None,
+        "is_running": running_task is not None and running_task.task_type == JOB_TYPE_COLLECT,
+        "is_task_active": running_task is not None,
+        "task_status": running_task.status if running_task else None,
         "task_id": display_task.id if display_task else None,
+        "task_type": display_task.task_type if display_task else None,
         "progress": display_task.progress if display_task else None,
         "phase": display_task.phase if display_task else None,
         "phase_progress": display_task.phase_progress if display_task else None,
@@ -400,9 +420,25 @@ async def _trigger_datasource_batch(
     datasources: list[DataSource],
     *,
     force: bool,
+    actor_id: int | None = None,
+    trigger_kind: str = "batch",
 ) -> dict:
+    await emit_business_log(
+        logger,
+        event=f"collector.trigger.{trigger_kind}.start",
+        message="Datasource batch trigger started",
+        category="collector",
+        service="api",
+        module=__name__,
+        user_id=actor_id,
+        context={
+            "trigger_kind": trigger_kind,
+            "force": force,
+            "requested_count": len(datasources),
+        },
+    )
     if not datasources:
-        return {
+        result = {
             "status": "noop",
             "message": "No matching data sources to trigger",
             "force": force,
@@ -410,8 +446,18 @@ async def _trigger_datasource_batch(
             "skipped": [],
             "failed": [],
         }
+        await emit_business_log(
+            logger,
+            event=f"collector.trigger.{trigger_kind}.completed",
+            message="Datasource batch trigger completed with no matching sources",
+            category="collector",
+            service="api",
+            module=__name__,
+            user_id=actor_id,
+            context={"trigger_kind": trigger_kind, "force": force, "status": "noop", "triggered_count": 0},
+        )
+        return result
 
-    previous_task_ids: dict[int, Optional[int]] = {}
     triggered_sources: list[dict] = []
     skipped_sources: list[dict] = []
     failed_sources: list[dict] = []
@@ -446,9 +492,11 @@ async def _trigger_datasource_batch(
                     }
                 )
                 continue
-            cancelled = await cancel_running_collector_now(datasource.source)
-            if not cancelled:
-                await rollback_orphaned_running_task(db, datasource, running_task)
+            await request_cancel_datasource_task(
+                db,
+                running_task,
+                reason="superseded_by_forced_collection",
+            )
 
         if not force and not is_due_for_collection(datasource, now):
             skipped_sources.append(
@@ -465,57 +513,51 @@ async def _trigger_datasource_batch(
             )
             continue
 
-        previous_task_ids[datasource.id] = None
-        success = run_collector_now(datasource.source)
-        if not success:
-            failed_sources.append(
-                {
-                    "id": datasource.id,
-                    "source": datasource.source,
-                    "name": datasource.name,
-                    "reason": "trigger_failed",
-                }
-            )
-            continue
+        task = await enqueue_datasource_job(
+            db,
+            datasource,
+            JOB_TYPE_COLLECT,
+            payload={"force": force, "trigger": "batch"},
+        )
 
         triggered_sources.append(
             {
                 "id": datasource.id,
                 "source": datasource.source,
                 "name": datasource.name,
-                "task_id": None,
+                "task_id": task.id,
             }
         )
 
-    latest_task_ids = await _load_latest_task_ids(
-        db,
-        [datasource.id for datasource in datasources],
-    )
-    for datasource_id in previous_task_ids:
-        previous_task_ids[datasource_id] = latest_task_ids.get(datasource_id)
-
-    for _ in range(20):
-        await asyncio.sleep(0.1)
-        pending = [item for item in triggered_sources if item["task_id"] is None]
-        if not pending:
-            break
-        latest_task_ids = await _load_latest_task_ids(
-            db,
-            [item["id"] for item in pending],
-        )
-        for item in pending:
-            task_id = latest_task_ids.get(item["id"])
-            if task_id is not None and task_id != previous_task_ids.get(item["id"]):
-                item["task_id"] = task_id
-
-    return {
-        "status": "triggered" if triggered_sources else "partial",
-        "message": f"Triggered {len(triggered_sources)} data sources",
+    result = {
+        "status": "queued" if triggered_sources else "partial",
+        "message": f"Queued {len(triggered_sources)} data source jobs",
         "force": force,
         "triggered": triggered_sources,
         "skipped": skipped_sources,
         "failed": failed_sources,
     }
+    await emit_business_log(
+        logger,
+        event=f"collector.trigger.{trigger_kind}.completed",
+        message="Datasource batch trigger completed",
+        category="collector",
+        service="api",
+        module=__name__,
+        user_id=actor_id,
+        context={
+            "trigger_kind": trigger_kind,
+            "force": force,
+            "status": result["status"],
+            "requested_count": len(datasources),
+            "triggered_count": len(triggered_sources),
+            "skipped_count": len(skipped_sources),
+            "failed_count": len(failed_sources),
+            "triggered_sources": [item["source"] for item in triggered_sources],
+            "skipped_reasons": [item["reason"] for item in skipped_sources],
+        },
+    )
+    return result
 
 
 async def get_datasource_record(db: AsyncSession, source_id: str) -> Optional[DataSource]:
@@ -540,7 +582,7 @@ async def get_running_task(db: AsyncSession, datasource_id: int) -> Optional[Col
     result = await db.execute(
         select(CollectionTask)
         .where(CollectionTask.datasource_id == datasource_id)
-        .where(CollectionTask.status == "running")
+        .where(CollectionTask.status == JobStatus.RUNNING.value)
         .order_by(CollectionTask.started_at.desc())
         .limit(1)
     )
@@ -568,8 +610,8 @@ async def get_running_task(db: AsyncSession, datasource_id: int) -> Optional[Col
             f"Marked failed automatically after stale running timeout "
             f"({STALE_RUNNING_TASK_TIMEOUT_MINUTES}m)"
         )
-        task.status = "failed"
-        task.phase = "failed"
+        task.status = JobStatus.FAILED.value
+        task.phase = JobStatus.FAILED.value
         task.completed_at = now
         task.error_message = f"{existing_error}\n{stale_reason}".strip() if existing_error else stale_reason
         await db.commit()
@@ -606,7 +648,7 @@ async def rollback_orphaned_running_task(
     )
 
     if snapshot is not None:
-        snapshot.status = "cancelled"
+        snapshot.status = SnapshotStatus.CANCELLED.value
         snapshot.is_current = False
         snapshot.completed_at = datetime.now(timezone.utc)
         summary = dict(snapshot.summary or {})
@@ -629,13 +671,13 @@ async def rollback_orphaned_running_task(
                 {"snapshot_id": snapshot.parent_snapshot_id},
             )
 
-    running_task.status = "cancelled"
-    running_task.phase = "cancelled"
+    running_task.status = JobStatus.CANCELLED.value
+    running_task.phase = JobStatus.CANCELLED.value
     running_task.completed_at = datetime.now(timezone.utc)
     existing_error = (running_task.error_message or "").strip()
     cancel_reason = "Cancelled after backend restart because the running task handle was lost; incomplete writes rolled back"
     running_task.error_message = f"{existing_error}\n{cancel_reason}".strip() if existing_error else cancel_reason
-    datasource.last_status = "cancelled"
+    datasource.last_status = JobStatus.CANCELLED.value
     datasource.last_run_at = datetime.now(timezone.utc)
     await db.commit()
 
@@ -670,7 +712,7 @@ async def fail_and_rollback_stale_running_task(
     )
 
     if snapshot is not None:
-        snapshot.status = "failed"
+        snapshot.status = SnapshotStatus.FAILED.value
         snapshot.is_current = False
         snapshot.completed_at = datetime.now(timezone.utc)
         summary = dict(snapshot.summary or {})
@@ -698,11 +740,11 @@ async def fail_and_rollback_stale_running_task(
         f"Marked failed automatically after stale running timeout "
         f"({STALE_RUNNING_TASK_TIMEOUT_MINUTES}m); incomplete writes rolled back"
     )
-    running_task.status = "failed"
-    running_task.phase = "failed"
+    running_task.status = JobStatus.FAILED.value
+    running_task.phase = JobStatus.FAILED.value
     running_task.completed_at = datetime.now(timezone.utc)
     running_task.error_message = f"{existing_error}\n{stale_reason}".strip() if existing_error else stale_reason
-    datasource.last_status = "failed"
+    datasource.last_status = JobStatus.FAILED.value
     datasource.last_run_at = datetime.now(timezone.utc)
     await db.commit()
 
@@ -780,7 +822,13 @@ async def trigger_all_datasources(
         .order_by(DataSource.module, DataSource.id)
     )
     datasources = result.scalars().all()
-    return await _trigger_datasource_batch(db, datasources, force=force)
+    return await _trigger_datasource_batch(
+        db,
+        datasources,
+        force=force,
+        actor_id=current_user.id,
+        trigger_kind="all",
+    )
 
 
 @router.post("/trigger-batch")
@@ -816,7 +864,13 @@ async def trigger_datasource_batch(
         collected=None if payload.source_ids else payload.collected,
         credential_status=None if payload.source_ids else payload.credential_status,
     )
-    return await _trigger_datasource_batch(db, datasources, force=payload.force)
+    return await _trigger_datasource_batch(
+        db,
+        datasources,
+        force=payload.force,
+        actor_id=current_user.id,
+        trigger_kind="batch",
+    )
 
 
 @router.get("/snapshots")
@@ -909,7 +963,7 @@ async def get_datasource_row(
         [datasource],
         include_endpoint=include_endpoint,
     )
-    record_counts = await _load_collected_record_counts(db, [datasource.source])
+    record_counts = await _load_collected_record_counts(db, [datasource.source], exact_vessel_counts=True)
     return {
         "data": serialize_datasource_row(
             datasource,
@@ -992,8 +1046,24 @@ async def trigger_datasource(
     if not datasource.is_active:
         raise HTTPException(status_code=400, detail="Data source is disabled")
 
-    running_task = await get_running_task(db, datasource.id)
+    running_task = await get_active_datasource_job(db, datasource.id, task_types=(JOB_TYPE_COLLECT,))
     if running_task is not None and not force:
+        await emit_business_log(
+            logger,
+            event="collector.trigger.single.skipped_already_running",
+            message="Datasource trigger skipped because a task is already running",
+            category="collector",
+            level="warning",
+            service="api",
+            module=__name__,
+            user_id=current_user.id,
+            context={
+                "collector_name": datasource.source,
+                "datasource_id": datasource.id,
+                "task_id": running_task.id,
+                "status": "skipped",
+            },
+        )
         raise HTTPException(
             status_code=409,
             detail={
@@ -1013,31 +1083,42 @@ async def trigger_datasource(
         )
 
     if running_task is not None and force:
-        cancelled = await cancel_running_collector_now(datasource.source)
-        if not cancelled:
-            await rollback_orphaned_running_task(db, datasource, running_task)
+        await request_cancel_datasource_task(
+            db,
+            running_task,
+            reason="superseded_by_forced_collection",
+        )
 
-    previous_task_id = await get_latest_task_id_for_datasource(datasource.id)
-    success = run_collector_now(datasource.source)
-    if not success:
-        raise HTTPException(status_code=500, detail=f"Failed to trigger collector '{datasource.source}'")
-
-    task_id = None
-    for _ in range(20):
-        await asyncio.sleep(0.1)
-        task_id = await get_latest_task_id_for_datasource(datasource.id)
-        if task_id is not None and task_id != previous_task_id:
-            break
-    if task_id == previous_task_id:
-        task_id = None
+    task = await enqueue_datasource_job(
+        db,
+        datasource,
+        JOB_TYPE_COLLECT,
+        payload={"force": force, "trigger": "single"},
+    )
+    await emit_business_log(
+        logger,
+        event="collector.trigger.single.completed",
+        message="Datasource trigger queued",
+        category="collector",
+        service="api",
+        module=__name__,
+        user_id=current_user.id,
+        context={
+            "collector_name": datasource.source,
+            "datasource_id": datasource.id,
+            "task_id": task.id,
+            "force": force,
+            "status": "queued",
+        },
+    )
 
     return {
-        "status": "triggered",
+        "status": "queued",
         "source_id": datasource.id,
-        "task_id": task_id,
+        "task_id": task.id,
         "collector_name": datasource.source,
         "force": force,
-        "message": f"Collector '{datasource.source}' has been triggered",
+        "message": f"Collector '{datasource.source}' has been queued",
     }
 
 
@@ -1051,22 +1132,30 @@ async def clear_datasource_data(
     if not datasource:
         raise HTTPException(status_code=404, detail="Data source not found")
 
-    result = await db.execute(
-        select(func.count(CollectedData.id)).where(CollectedData.source == datasource.source)
+    active_task = await get_active_datasource_job(db, datasource.id)
+    if active_task is not None:
+        raise HTTPException(
+            status_code=409,
+            detail={
+                "reason": "datasource_job_in_progress",
+                "message": "当前数据源已有任务在执行，请等待完成或先取消任务。",
+                "task_id": active_task.id,
+                "task_type": active_task.task_type,
+                "status": active_task.status,
+            },
+        )
+    task = await enqueue_datasource_job(
+        db,
+        datasource,
+        JOB_TYPE_CLEAR_DATA,
+        payload={"source": datasource.source},
     )
-    count = result.scalar() or 0
-
-    if count == 0:
-        return {"status": "success", "message": "No data to clear", "deleted_count": 0}
-
-    delete_query = CollectedData.__table__.delete().where(CollectedData.source == datasource.source)
-    await db.execute(delete_query)
-    await db.commit()
 
     return {
-        "status": "success",
-        "message": f"Cleared {count} records for data source '{datasource.name}'",
-        "deleted_count": count,
+        "status": "queued",
+        "message": f"Queued data clearing for data source '{datasource.name}'",
+        "task_id": task.id,
+        "deleted_count": None,
     }
 
 
@@ -1080,16 +1169,44 @@ async def clear_datasource_cache(
     if not datasource:
         raise HTTPException(status_code=404, detail="Data source not found")
 
-    earth_deleted_count = invalidate_earth_layer_cache_for_source(datasource.source)
-    dashboard_deleted_count = int(cache.delete("dashboard:stats")) + int(cache.delete("dashboard:summary"))
-    deleted_count = earth_deleted_count + dashboard_deleted_count
+    task = await enqueue_datasource_job(
+        db,
+        datasource,
+        JOB_TYPE_CLEAR_CACHE,
+        payload={"source": datasource.source},
+        dedupe_key=f"clear_cache:{datasource.source}",
+    )
 
     return {
-        "status": "success",
-        "message": f"Cleared {deleted_count} cache keys for data source '{datasource.name}'",
-        "deleted_count": deleted_count,
-        "earth_layer_deleted_count": earth_deleted_count,
-        "dashboard_deleted_count": dashboard_deleted_count,
+        "status": "queued",
+        "message": f"Queued cache clearing for data source '{datasource.name}'",
+        "task_id": task.id,
+        "deleted_count": None,
+    }
+
+
+@router.post("/{source_id}/tasks/{task_id}/cancel")
+async def cancel_datasource_task(
+    source_id: str,
+    task_id: int,
+    current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    datasource = await get_datasource_record(db, source_id)
+    if not datasource:
+        raise HTTPException(status_code=404, detail="Data source not found")
+
+    task = await db.get(CollectionTask, task_id)
+    if not task or task.datasource_id != datasource.id:
+        raise HTTPException(status_code=404, detail="Task not found")
+
+    task = await request_cancel_datasource_task(db, task)
+    return {
+        "status": "cancelled" if task.completed_at else "cancelling",
+        "task_id": task.id,
+        "task_type": task.task_type,
+        "phase": task.phase,
+        "requested_cancel_at": to_iso8601_utc(task.requested_cancel_at),
     }
 
 
@@ -1109,7 +1226,7 @@ async def get_task_status(
         if not task or task.datasource_id != datasource.id:
             raise HTTPException(status_code=404, detail="Task not found")
     else:
-        task = await get_running_task(db, datasource.id)
+        task = await get_active_datasource_job(db, datasource.id)
         if task is None:
             result = await db.execute(
                 select(CollectionTask)
@@ -1134,8 +1251,12 @@ async def get_task_status(
         }
 
     return {
-        "is_running": task.status == "running",
+        "is_running": task.status in {JOB_STATUS_QUEUED, JOB_STATUS_RUNNING, JOB_STATUS_CANCELLING}
+        and task.task_type == JOB_TYPE_COLLECT,
+        "is_task_active": task.status in {JOB_STATUS_QUEUED, JOB_STATUS_RUNNING, JOB_STATUS_CANCELLING},
         "task_id": task.id,
+        "task_type": task.task_type,
+        "task_status": task.status,
         "progress": task.progress,
         "phase": task.phase,
         "phase_progress": task.phase_progress,
@@ -1146,5 +1267,6 @@ async def get_task_status(
         "records_processed": task.records_processed,
         "total_records": task.total_records,
         "status": task.status,
+        "requested_cancel_at": to_iso8601_utc(task.requested_cancel_at),
         "error_message": task.error_message,
     }

backend/app/api/v1/earth.py

@@ -6,16 +6,42 @@ from pathlib import Path
 from typing import Any
 from uuid import uuid4
 
-from fastapi import APIRouter, Depends, File, HTTPException, Request, UploadFile, status
+from fastapi import APIRouter, Depends, File, Form, HTTPException, Query, Request, UploadFile, status
 from fastapi.security import HTTPAuthorizationCredentials, HTTPBearer
 from pydantic import BaseModel, Field
-from sqlalchemy import delete, select, text
+from sqlalchemy import delete, func, select, text
 from sqlalchemy.ext.asyncio import AsyncSession
 
+from app.core.config import settings as app_settings
+from app.core.logging import get_logger
 from app.core.security import decode_token, get_current_user, redis_client
 from app.db.session import get_db
+from app.models.collected_data import CollectedData
+from app.models.datasource import DataSource
+from app.models.datasource_config import DataSourceConfig
 from app.models.system_setting import SystemSetting
 from app.models.user import User
+from app.services.tv_streams import get_tv_settings_payload
+from app.services.earth_news import (
+    get_earth_news_sources_payload,
+    reset_earth_news_sources_payload,
+    save_earth_news_sources_payload,
+    test_news_source_config,
+)
+from app.services.earth_news_manual import (
+    broadcast_manual_news_changed,
+    create_manual_news_group,
+    delete_manual_news_item,
+    get_news_record_or_404,
+    import_manual_news_items,
+    list_news_groups,
+    list_news_records,
+    parse_manual_news_import_upload,
+    rename_manual_news_group,
+    reprocess_manual_news_item,
+    serialize_news_record,
+    upsert_manual_news_item,
+)
 from app.services.earth_boundaries import (
     EarthBoundaryBuildError,
     get_boundary_build_status,
@@ -26,14 +52,23 @@ from app.services.earth_boundaries import (
 
 
 router = APIRouter()
+logger = get_logger(__name__)
 optional_bearer = HTTPBearer(auto_error=False)
 REPO_ROOT = Path(__file__).resolve().parents[4]
 EARTH_BRAND_ASSET_DIR = REPO_ROOT / "data" / "earth-brand"
 EARTH_BRAND_ASSET_URL_PREFIX = "/earth-brand-assets"
 EARTH_BRAND_CATEGORY = "earth_brand"
+EARTH_ABOUT_CATEGORY = "earth_about"
+SYSTEM_SETTINGS_CATEGORY = "system"
 MAX_EARTH_BRAND_ASSET_BYTES = 3 * 1024 * 1024
 ALLOWED_EARTH_BRAND_EXTENSIONS = {".png", ".jpg", ".jpeg", ".webp", ".svg"}
 
+
+def _app_version_label() -> str:
+    version = str(app_settings.VERSION or "").strip() or "0.0.0"
+    return version if version.startswith("v") else f"v{version}"
+
+
 DEFAULT_EARTH_BRAND = {
     "logo_src": "/earth/assets/brand/earth-logo.png",
     "title_src": "/earth/assets/brand/title-zh.png",
@@ -44,6 +79,20 @@ DEFAULT_EARTH_BRAND = {
     "title_alt": "智能星球计划",
 }
 
+DEFAULT_EARTH_ABOUT = {
+    "logo_src": "/earth/assets/brand/lim-logo.png",
+    "kicker": "About",
+    "title": "智能星球计划",
+    "version": _app_version_label(),
+    "description": "面向临空场景下的智能媒体研究、全球态势感知与多源开放数据巡航，提供可视化观测、事件聚合与交互式探索能力。",
+    "meta": [
+        {"label": "出品方", "value": "浙江大学临空智能媒体研究院"},
+        {"label": "策划人", "value": "方兴东、黄柳青"},
+        {"label": "产品兼开发者", "value": "钱坤、张鸽、齐鹏"},
+    ],
+}
+EARTH_ABOUT_LEGACY_PLANNER_VALUE = "黄柳青"
+
 
 class EarthBoundaryConfigPayload(BaseModel):
     config: dict[str, Any] = Field(default_factory=dict)
@@ -59,6 +108,53 @@ class EarthBrandPayload(BaseModel):
     title_alt: str = Field(default=DEFAULT_EARTH_BRAND["title_alt"], max_length=200)
 
 
+class EarthAboutMetaItem(BaseModel):
+    label: str = Field(default="", max_length=80)
+    value: str = Field(default="", max_length=240)
+
+
+class EarthAboutPayload(BaseModel):
+    logo_src: str = Field(default=DEFAULT_EARTH_ABOUT["logo_src"], max_length=1000)
+    kicker: str = Field(default=DEFAULT_EARTH_ABOUT["kicker"], max_length=80)
+    title: str = Field(default=DEFAULT_EARTH_ABOUT["title"], max_length=160)
+    version: str = Field(default=DEFAULT_EARTH_ABOUT["version"], max_length=80)
+    description: str = Field(default=DEFAULT_EARTH_ABOUT["description"], max_length=800)
+    meta: list[EarthAboutMetaItem] = Field(default_factory=list)
+
+
+class EarthNewsSourcesPayload(BaseModel):
+    cache_version: int | None = None
+    source_tags: list[dict[str, Any]] = Field(default_factory=list)
+    categories: list[dict[str, Any]] = Field(default_factory=list)
+    item_tag_rules: list[dict[str, Any]] = Field(default_factory=list)
+    sources: list[dict[str, Any]] = Field(default_factory=list)
+    health: dict[str, Any] = Field(default_factory=dict)
+
+
+class EarthNewsSourceTestPayload(BaseModel):
+    source: dict[str, Any] = Field(default_factory=dict)
+
+
+class EarthNewsManualItemPayload(BaseModel):
+    title: str = Field(default="", max_length=500)
+    summary: str = Field(default="", max_length=1200)
+    content: str = Field(default="", max_length=12000)
+    url: str = Field(default="", max_length=2000)
+    source: str = Field(default="", max_length=255)
+    region: str = Field(default="global", max_length=80)
+    published_at: str | None = None
+    category: str = Field(default="other", max_length=80)
+    tags: list[str] = Field(default_factory=list)
+    location: dict[str, Any] | None = None
+    homepage_url: str = Field(default="", max_length=2000)
+    content_language: str = Field(default="", max_length=32)
+    group_id: str | None = Field(default=None, max_length=120)
+
+
+class EarthNewsManualGroupPayload(BaseModel):
+    name: str = Field(default="", max_length=120)
+
+
 def _normalize_earth_brand_payload(payload: dict[str, Any] | None) -> dict[str, str]:
     merged = DEFAULT_EARTH_BRAND.copy()
     if payload:
@@ -76,6 +172,47 @@ def _normalize_earth_brand_payload(payload: dict[str, Any] | None) -> dict[str,
     return merged
 
 
+def _normalize_earth_about_payload(payload: dict[str, Any] | None) -> dict[str, Any]:
+    merged: dict[str, Any] = {
+        key: value
+        for key, value in DEFAULT_EARTH_ABOUT.items()
+        if key != "meta"
+    }
+    raw_meta = DEFAULT_EARTH_ABOUT["meta"]
+    if payload:
+        for key in ("logo_src", "kicker", "title", "description"):
+            value = payload.get(key)
+            if value is not None:
+                merged[key] = str(value).strip()
+        raw_meta = payload.get("meta") if isinstance(payload.get("meta"), list) else raw_meta
+    merged["version"] = _app_version_label()
+
+    for key, default_value in DEFAULT_EARTH_ABOUT.items():
+        if key == "meta":
+            continue
+        if not merged.get(key):
+            merged[key] = default_value
+
+    normalized_meta: list[dict[str, str]] = []
+    for item in raw_meta:
+        if not isinstance(item, dict):
+            continue
+        label = str(item.get("label") or "").strip()
+        value = str(item.get("value") or "").strip()
+        if label == "策划人" and value == EARTH_ABOUT_LEGACY_PLANNER_VALUE:
+            value = "方兴东、黄柳青"
+        if label or value:
+            normalized_meta.append({"label": label, "value": value})
+    if not normalized_meta:
+        normalized_meta = [dict(item) for item in DEFAULT_EARTH_ABOUT["meta"]]
+    merged["meta"] = normalized_meta
+    return merged
+
+
+def _is_demo_mode_enabled(payload: Any) -> bool:
+    return bool(payload.get("demo_mode")) if isinstance(payload, dict) else False
+
+
 async def _get_earth_brand_record(db: AsyncSession) -> SystemSetting | None:
     result = await db.execute(
         select(SystemSetting).where(SystemSetting.category == EARTH_BRAND_CATEGORY)
@@ -91,6 +228,56 @@ async def _get_earth_brand_payload(db: AsyncSession) -> dict[str, Any]:
     }
 
 
+async def _get_earth_about_record(db: AsyncSession) -> SystemSetting | None:
+    result = await db.execute(
+        select(SystemSetting).where(SystemSetting.category == EARTH_ABOUT_CATEGORY)
+    )
+    return result.scalar_one_or_none()
+
+
+async def _get_earth_about_payload(db: AsyncSession) -> dict[str, Any]:
+    record = await _get_earth_about_record(db)
+    return {
+        "about": _normalize_earth_about_payload(record.payload if record else None),
+        "is_default": record is None,
+    }
+
+
+async def _get_optional_current_user(
+    credentials: HTTPAuthorizationCredentials | None = Depends(optional_bearer),
+    db: AsyncSession = Depends(get_db),
+) -> User | None:
+    if credentials is None:
+        return None
+    token = credentials.credentials
+    if redis_client.sismember("blacklisted_tokens", token):
+        return None
+    payload = decode_token(token)
+    if payload is None or payload.get("type") != "access":
+        return None
+    user_id = payload.get("sub")
+    if user_id is None:
+        return None
+    result = await db.execute(
+        text(
+            "SELECT id, username, email, password_hash, role, is_active, gatekeeper_groups FROM users WHERE id = :id"
+        ),
+        {"id": int(user_id)},
+    )
+    row = result.fetchone()
+    if row is None or not row[5]:
+        return None
+    user = User()
+    user.id = row[0]
+    user.username = row[1]
+    user.email = row[2]
+    user.password_hash = row[3]
+    user.role = row[4]
+    user.is_active = row[5]
+    user.gatekeeper_groups = row[6] or []
+    return user
+
+
 @router.get("/brand")
 async def get_earth_brand(db: AsyncSession = Depends(get_db)):
     return await _get_earth_brand_payload(db)
@@ -159,46 +346,313 @@ async def upload_earth_brand_asset(
     return {"url": asset_url, "filename": safe_name, "content_type": file.content_type}
 
 
+@router.get("/about")
+async def get_earth_about(db: AsyncSession = Depends(get_db)):
+    return await _get_earth_about_payload(db)
+
+
+@router.put("/about")
+async def update_earth_about(
+    payload: EarthAboutPayload,
+    _current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    normalized = _normalize_earth_about_payload(payload.model_dump())
+    record = await _get_earth_about_record(db)
+    if record is None:
+        record = SystemSetting(category=EARTH_ABOUT_CATEGORY, payload=normalized)
+        db.add(record)
+    else:
+        record.payload = normalized
+    await db.commit()
+    await db.refresh(record)
+    return {"status": "updated", "about": _normalize_earth_about_payload(record.payload), "is_default": False}
+
+
+@router.delete("/about")
+async def reset_earth_about(
+    _current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    await db.execute(delete(SystemSetting).where(SystemSetting.category == EARTH_ABOUT_CATEGORY))
+    await db.commit()
+    return {"status": "reset", "about": _normalize_earth_about_payload(None), "is_default": True}
+
+
+@router.get("/news-sources")
+async def get_earth_news_sources(db: AsyncSession = Depends(get_db)):
+    return await get_earth_news_sources_payload(db)
+
+
+def _schedule_earth_news_source_refresh() -> None:
+    try:
+        from app.services.scheduler import schedule_earth_news_refresh_for_all_regions
+
+        schedule_earth_news_refresh_for_all_regions(force=True)
+    except Exception as exc:
+        logger.warning_event(
+            "Failed to schedule Earth news refresh after source configuration change",
+            event="earth_news.rss.config_refresh_schedule_failed",
+            context={"error": str(exc)},
+        )
+
+
+@router.put("/news-sources")
+async def update_earth_news_sources(
+    payload: EarthNewsSourcesPayload,
+    _current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    result = await save_earth_news_sources_payload(db, payload.model_dump())
+    _schedule_earth_news_source_refresh()
+    return result
+
+
+@router.delete("/news-sources")
+@router.post("/news-sources/reset")
+async def reset_earth_news_sources(
+    _current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    result = await reset_earth_news_sources_payload(db)
+    _schedule_earth_news_source_refresh()
+    return result
+
+
+@router.post("/news-sources/test")
+async def test_earth_news_source(
+    payload: EarthNewsSourceTestPayload,
+    _current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    return await test_news_source_config(payload.source, db=db)
+
+
+@router.get("/news-groups")
+async def list_earth_news_groups_admin(
+    _current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    return await list_news_groups(db)
+
+
+@router.post("/news-groups")
+async def create_earth_news_group_admin(
+    payload: EarthNewsManualGroupPayload,
+    _current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    try:
+        group = await create_manual_news_group(db, payload.name)
+    except ValueError as exc:
+        raise HTTPException(status_code=422, detail=str(exc)) from exc
+    await db.commit()
+    return {"status": "ok", "group": group}
+
+
+@router.put("/news-groups/{group_id:path}")
+async def rename_earth_news_group_admin(
+    group_id: str,
+    payload: EarthNewsManualGroupPayload,
+    _current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    try:
+        group = await rename_manual_news_group(db, group_id, payload.name)
+    except ValueError as exc:
+        raise HTTPException(status_code=422, detail=str(exc)) from exc
+    await db.commit()
+    await broadcast_manual_news_changed()
+    return {"status": "ok", "group": group}
+
+
+@router.get("/news-items")
+async def list_earth_news_items_admin(
+    page: int = Query(1, ge=1),
+    page_size: int = Query(50, ge=1, le=100),
+    source_type: str | None = Query(None),
+    region: str | None = Query(None),
+    category: str | None = Query(None),
+    status_filter: str | None = Query(None, alias="status"),
+    group_id: str | None = Query(None),
+    _current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    return await list_news_records(
+        db,
+        page=page,
+        page_size=page_size,
+        source_type=source_type,
+        region=region,
+        category=category,
+        status_filter=status_filter,
+        group_id=group_id,
+    )
+
+
+@router.post("/news-items")
+async def create_earth_news_item_admin(
+    payload: EarthNewsManualItemPayload,
+    _current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    try:
+        result = await upsert_manual_news_item(db, payload.model_dump())
+    except ValueError as exc:
+        raise HTTPException(status_code=422, detail=str(exc)) from exc
+    except PermissionError as exc:
+        raise HTTPException(status_code=403, detail=str(exc)) from exc
+    await db.commit()
+    await broadcast_manual_news_changed()
+    return {"status": "ok", "created": result.created, "queued": result.queued, "item": serialize_news_record(result.item)}
+
+
+@router.post("/news-items/import")
+async def import_earth_news_items_admin(
+    file: UploadFile = File(...),
+    group_id: str | None = Form(default=None),
+    _current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    try:
+        payload = await parse_manual_news_import_upload(await file.read())
+        result = await import_manual_news_items(db, payload, group_id=group_id)
+    except ValueError as exc:
+        raise HTTPException(status_code=422, detail=str(exc)) from exc
+    await db.commit()
+    await broadcast_manual_news_changed()
+    return {"status": "ok", **result}
+
+
+@router.put("/news-items/{item_id:path}")
+async def update_earth_news_item_admin(
+    item_id: str,
+    payload: EarthNewsManualItemPayload,
+    _current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    existing = await get_news_record_or_404(db, item_id)
+    if existing is None:
+        raise HTTPException(status_code=404, detail="News item not found.")
+    try:
+        result = await upsert_manual_news_item(
+            db,
+            payload.model_dump(),
+            item_id_override=item_id,
+        )
+    except ValueError as exc:
+        raise HTTPException(status_code=422, detail=str(exc)) from exc
+    except PermissionError as exc:
+        raise HTTPException(status_code=403, detail=str(exc)) from exc
+    await db.commit()
+    await broadcast_manual_news_changed()
+    return {"status": "ok", "created": result.created, "queued": result.queued, "item": serialize_news_record(result.item)}
+
+
+@router.delete("/news-items/{item_id:path}")
+async def delete_earth_news_item_admin(
+    item_id: str,
+    _current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    try:
+        deleted = await delete_manual_news_item(db, item_id)
+    except PermissionError as exc:
+        raise HTTPException(status_code=403, detail=str(exc)) from exc
+    if not deleted:
+        raise HTTPException(status_code=404, detail="News item not found.")
+    await db.commit()
+    await broadcast_manual_news_changed()
+    return {"status": "deleted", "id": item_id}
+
+
+@router.post("/news-items/{item_id:path}/reprocess")
+async def reprocess_earth_news_item_admin(
+    item_id: str,
+    _current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    existing = await get_news_record_or_404(db, item_id)
+    if existing is None:
+        raise HTTPException(status_code=404, detail="News item not found.")
+    try:
+        queued = await reprocess_manual_news_item(db, item_id)
+    except PermissionError as exc:
+        raise HTTPException(status_code=403, detail=str(exc)) from exc
+    await db.commit()
+    await broadcast_manual_news_changed()
+    return {"status": "queued" if queued else "not_queued", "queued": queued, "id": item_id}
+
+
+@router.get("/oobe-status")
+async def get_earth_oobe_status(
+    current_user: User | None = Depends(_get_optional_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    current_count_result = await db.execute(
+        select(func.count(CollectedData.id)).where(CollectedData.is_current.is_(True))
+    )
+    current_record_count = int(current_count_result.scalar() or 0)
+    system_result = await db.execute(
+        select(SystemSetting).where(SystemSetting.category == SYSTEM_SETTINGS_CATEGORY)
+    )
+    system_record = system_result.scalar_one_or_none()
+    demo_mode = _is_demo_mode_enabled(system_record.payload if system_record else None)
+
+    datasource_count_result = await db.execute(select(func.count(DataSource.id)))
+    datasource_count = int(datasource_count_result.scalar() or 0)
+    active_datasource_count_result = await db.execute(
+        select(func.count(DataSource.id)).where(DataSource.is_active.is_(True))
+    )
+    active_datasource_count = int(active_datasource_count_result.scalar() or 0)
+    config_result = await db.execute(select(func.count(DataSourceConfig.id)))
+    custom_config_count = int(config_result.scalar() or 0)
+
+    tv_payload = await get_tv_settings_payload(db)
+    tv_sources = tv_payload.get("sources") if isinstance(tv_payload, dict) else []
+    tv_source_count = len(tv_sources) if isinstance(tv_sources, list) else 0
+
+    boundary_status = get_boundary_status()
+    has_core_layers = bool(boundary_status.get("ready") or boundary_status.get("available") or boundary_status.get("status") in {"ready", "built", "ok"})
+    has_collected_data = current_record_count > 0
+    ready = has_collected_data
+
+    suggestions: list[str] = []
+    if demo_mode:
+        suggestions.append("演示模式已开启")
+    if not current_user:
+        suggestions.append("登录控制台")
+    if not has_collected_data:
+        suggestions.append("触发数据源采集")
+    if not custom_config_count:
+        suggestions.append("确认采集器配置")
+    if not has_core_layers:
+        suggestions.append("构建或启用 Earth 图层")
+
+    return {
+        "ready": ready,
+        "demo_mode": demo_mode,
+        "authenticated": current_user is not None,
+        "needs_login": current_user is None and not ready and not demo_mode,
+        "has_collected_data": has_collected_data,
+        "has_tv_sources": tv_source_count > 0,
+        "has_core_layers": has_core_layers,
+        "current_record_count": current_record_count,
+        "datasource_count": datasource_count,
+        "active_datasource_count": active_datasource_count,
+        "custom_config_count": custom_config_count,
+        "tv_source_count": tv_source_count,
+        "suggestions": suggestions,
+        "login_url": "/login?next=/datasources",
+        "datasources_url": "/datasources",
+        "collection_url": "/collection-management",
+    }
+
+
 @router.get("/boundaries/status")
 async def get_earth_boundary_status():
     return get_boundary_status()
 
-
-async def _get_optional_current_user(
-    credentials: HTTPAuthorizationCredentials | None = Depends(optional_bearer),
-    db: AsyncSession = Depends(get_db),
-) -> User | None:
-    if credentials is None:
-        return None
-    token = credentials.credentials
-    if redis_client.sismember("blacklisted_tokens", token):
-        return None
-    payload = decode_token(token)
-    if payload is None or payload.get("type") != "access":
-        return None
-    user_id = payload.get("sub")
-    if user_id is None:
-        return None
-    result = await db.execute(
-        text(
-            "SELECT id, username, email, password_hash, role, is_active, gatekeeper_groups FROM users WHERE id = :id"
-        ),
-        {"id": int(user_id)},
-    )
-    row = result.fetchone()
-    if row is None or not row[5]:
-        return None
-    user = User()
-    user.id = row[0]
-    user.username = row[1]
-    user.email = row[2]
-    user.password_hash = row[3]
-    user.role = row[4]
-    user.is_active = row[5]
-    user.gatekeeper_groups = row[6] or []
-    return user
-
-
 def _is_loopback_request(request: Request) -> bool:
     host = request.client.host if request.client else ""
     return host in {"127.0.0.1", "::1", "localhost"} or host.startswith("127.")

backend/app/api/v1/interactables.py

@@ -0,0 +1,190 @@
+"""CRUD APIs for persistent Earth interactables."""
+
+from __future__ import annotations
+
+from datetime import UTC, datetime
+from typing import Any
+
+from fastapi import APIRouter, Depends, HTTPException, Query, Response, status
+from pydantic import BaseModel, Field, field_validator
+from sqlalchemy.ext.asyncio import AsyncSession
+
+from app.core.security import get_current_user
+from app.db.session import get_db
+from app.models.earth_interactable import EarthInteractable
+from app.models.user import User
+from app.services.earth_interactables import (
+    build_interactable_event,
+    interactables_to_geojson,
+    invalidate_interactable_cache,
+    list_interactables,
+    normalize_interactable_id,
+    publish_interactable_event,
+    serialize_interactable,
+)
+from app.services.earth_layer_cache import (
+    EarthLayerCachePolicy,
+    earth_layer_cache,
+    get_or_build_layer_payload,
+)
+
+router = APIRouter()
+INTERACTABLE_CACHE_POLICY = EarthLayerCachePolicy(
+    fresh_ttl_seconds=60,
+    stale_ttl_seconds=10 * 60,
+    max_features=5000,
+)
+
+
+class InteractableCreate(BaseModel):
+    id: str | None = Field(default=None, max_length=160)
+    layer: str = Field(default="default", min_length=1, max_length=80)
+    kind: str = Field(default="default", min_length=1, max_length=80)
+    label: str = Field(default="", max_length=255)
+    description: str = Field(default="", max_length=4000)
+    latitude: float = Field(ge=-90, le=90)
+    longitude: float = Field(ge=-180, le=180)
+    altitude: float | None = None
+    properties: dict[str, Any] = Field(default_factory=dict)
+
+    @field_validator("layer", "kind")
+    @classmethod
+    def normalize_key(cls, value: str) -> str:
+        normalized = str(value or "").strip()
+        if not normalized:
+            raise ValueError("must not be empty")
+        return normalized
+
+
+class InteractableUpdate(BaseModel):
+    layer: str | None = Field(default=None, min_length=1, max_length=80)
+    kind: str | None = Field(default=None, min_length=1, max_length=80)
+    label: str | None = Field(default=None, max_length=255)
+    description: str | None = Field(default=None, max_length=4000)
+    latitude: float | None = Field(default=None, ge=-90, le=90)
+    longitude: float | None = Field(default=None, ge=-180, le=180)
+    altitude: float | None = None
+    properties: dict[str, Any] | None = None
+
+
+@router.get("")
+async def get_interactables(
+    response: Response,
+    layer: str | None = Query(default=None),
+    include_deleted: bool = Query(default=False),
+    db: AsyncSession = Depends(get_db),
+):
+    items = await list_interactables(db, layer=layer, include_deleted=include_deleted)
+    response.headers["X-Planet-Interactables-Count"] = str(len(items))
+    return {"items": [serialize_interactable(item) for item in items]}
+
+
+@router.get("/geojson")
+async def get_interactables_geojson(
+    response: Response,
+    layer: str | None = Query(default=None),
+    db: AsyncSession = Depends(get_db),
+):
+    async def build_payload() -> dict[str, Any]:
+        items = await list_interactables(db, layer=layer)
+        return interactables_to_geojson(items)
+
+    payload = await get_or_build_layer_payload(
+        key=earth_layer_cache.key("interactables", interactable_layer=layer or "all"),
+        policy=INTERACTABLE_CACHE_POLICY,
+        builder=build_payload,
+        response=response,
+    )
+    response.headers["X-Planet-Interactables-Count"] = str(len(payload.get("features") or []))
+    return payload
+
+
+@router.post("", status_code=status.HTTP_201_CREATED)
+async def create_interactable(
+    payload: InteractableCreate,
+    _current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    record_id = normalize_interactable_id(payload.id)
+    existing = await db.get(EarthInteractable, record_id)
+    if existing and not existing.is_deleted:
+        raise HTTPException(status_code=409, detail="Interactable already exists")
+
+    if existing is None:
+        record = EarthInteractable(id=record_id)
+        db.add(record)
+    else:
+        record = existing
+        record.is_deleted = False
+        record.deleted_at = None
+        record.revision += 1
+
+    record.layer = payload.layer
+    record.kind = payload.kind
+    record.label = payload.label
+    record.description = payload.description
+    record.latitude = payload.latitude
+    record.longitude = payload.longitude
+    record.altitude = payload.altitude
+    record.properties = payload.properties
+
+    await db.commit()
+    await db.refresh(record)
+    invalidate_interactable_cache(record.layer)
+    await publish_interactable_event("created", record)
+    return {"item": serialize_interactable(record)}
+
+
+@router.get("/{interactable_id}")
+async def get_interactable(interactable_id: str, db: AsyncSession = Depends(get_db)):
+    record = await db.get(EarthInteractable, interactable_id)
+    if record is None or record.is_deleted:
+        raise HTTPException(status_code=404, detail="Interactable not found")
+    return {"item": serialize_interactable(record)}
+
+
+@router.patch("/{interactable_id}")
+async def update_interactable(
+    interactable_id: str,
+    payload: InteractableUpdate,
+    _current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    record = await db.get(EarthInteractable, interactable_id)
+    if record is None or record.is_deleted:
+        raise HTTPException(status_code=404, detail="Interactable not found")
+
+    previous_layer = record.layer
+    patch = payload.model_dump(exclude_unset=True)
+    for key, value in patch.items():
+        setattr(record, key, value)
+    record.revision += 1
+
+    await db.commit()
+    await db.refresh(record)
+    invalidate_interactable_cache(previous_layer)
+    if record.layer != previous_layer:
+        invalidate_interactable_cache(record.layer)
+    await publish_interactable_event("updated", record)
+    return {"item": serialize_interactable(record)}
+
+
+@router.delete("/{interactable_id}")
+async def delete_interactable(
+    interactable_id: str,
+    _current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    record = await db.get(EarthInteractable, interactable_id)
+    if record is None or record.is_deleted:
+        raise HTTPException(status_code=404, detail="Interactable not found")
+
+    record.is_deleted = True
+    record.deleted_at = datetime.now(UTC)
+    record.revision += 1
+    await db.commit()
+    await db.refresh(record)
+    invalidate_interactable_cache(record.layer)
+    await publish_interactable_event("deleted", record)
+    event = build_interactable_event(action="deleted", record=record, include_item=True)
+    return {"deleted": True, "event": event}

backend/app/api/v1/layers.py

@@ -105,7 +105,7 @@ def _parse_layer_bbox(bbox: str) -> tuple[float, float, float, float]:
 @router.get("/vessels/snapshot")
 async def get_vessel_layer_snapshot(
     bbox: str = Query(..., description="lon_min,lat_min,lon_max,lat_max"),
-    zoom: int = Query(..., ge=1, le=20),
+    zoom: float = Query(..., ge=1, le=20),
     limit: int = Query(DEFAULT_LAYER_LIMIT, ge=1),
     vessel_type: Optional[str] = Query(None, alias="type"),
     since_minutes: int = Query(60, ge=1, le=1440),

backend/app/api/v1/news.py

@@ -1,16 +1,161 @@
-from fastapi import APIRouter, Depends, Query
+from fastapi import APIRouter, Depends, HTTPException, Query, status
+from pydantic import BaseModel, Field
 from sqlalchemy.ext.asyncio import AsyncSession
+from uuid import UUID
 
 from app.db.session import get_db
-from app.services.earth_news import get_earth_news_payload
+from app.services.earth_news import (
+    ALLOWED_NEWS_CATEGORY_KEYS,
+    SUPPORTED_NEWS_LOCALES,
+    REGION_ANCHORS,
+    determine_focus_region,
+    get_earth_news_payload,
+)
 
 router = APIRouter()
 
 
+class EarthNewsRefreshRequest(BaseModel):
+    lat: float | None = Field(default=None, ge=-90, le=90)
+    lon: float | None = Field(default=None, ge=-180, le=180)
+    region: str | None = Field(default=None, max_length=64)
+
+
+def _parse_categories(raw: str | None) -> set[str] | None:
+    if raw is None or not raw.strip():
+        return None
+    requested = {item.strip().lower() for item in raw.split(",") if item.strip()}
+    invalid = sorted(requested - set(ALLOWED_NEWS_CATEGORY_KEYS))
+    if invalid:
+        raise HTTPException(
+            status_code=422,
+            detail={
+                "message": "Unsupported news categories.",
+                "invalid_categories": invalid,
+                "allowed_categories": list(ALLOWED_NEWS_CATEGORY_KEYS),
+            },
+        )
+    return requested or None
+
+
+def _parse_source_ids(raw: str | None) -> set[str] | None:
+    if raw is None or not raw.strip():
+        return None
+    return {item.strip() for item in raw.split(",") if item.strip()} or None
+
+
+def _parse_limit(raw: int | None) -> int:
+    if raw is None:
+        return 12
+    if raw < 1:
+        raise HTTPException(
+            status_code=422, detail={"message": "News limit must be greater than 0."}
+        )
+    return min(raw, 100)
+
+
+def _parse_locale(raw: str | None) -> str:
+    if raw is None or not raw.strip():
+        return "zh-CN"
+    requested = raw.strip()
+    if requested not in SUPPORTED_NEWS_LOCALES:
+        raise HTTPException(
+            status_code=422,
+            detail={
+                "message": "Unsupported news locale.",
+                "invalid_locale": requested,
+                "allowed_locales": sorted(SUPPORTED_NEWS_LOCALES),
+            },
+        )
+    return requested
+
+
+@router.post("/refresh", status_code=status.HTTP_202_ACCEPTED)
+async def request_earth_news_refresh(payload: EarthNewsRefreshRequest):
+    requested_region = payload.region.strip().lower() if payload.region else None
+    if requested_region and requested_region not in REGION_ANCHORS:
+        raise HTTPException(
+            status_code=422,
+            detail={
+                "message": "Unsupported news region.",
+                "invalid_region": requested_region,
+                "allowed_regions": list(REGION_ANCHORS.keys()),
+            },
+        )
+    region = requested_region or determine_focus_region(payload.lat, payload.lon)
+    from app.services.scheduler import request_earth_news_region_refresh
+
+    try:
+        return request_earth_news_region_refresh(region)
+    except Exception as exc:
+        raise HTTPException(
+            status_code=503,
+            detail={
+                "code": "refresh_failed",
+                "message": "Could not start a news refresh. Please try again.",
+            },
+        ) from exc
+
+
+@router.get("/refresh/{refresh_id}")
+async def get_earth_news_refresh(refresh_id: UUID):
+    from app.services.earth_news_refresh import get_earth_news_refresh_status
+
+    try:
+        result = get_earth_news_refresh_status(refresh_id.hex)
+    except Exception as exc:
+        raise HTTPException(
+            status_code=503,
+            detail={
+                "code": "refresh_failed",
+                "message": "Refresh status is temporarily unavailable. Please retry.",
+            },
+        ) from exc
+    if result is not None:
+        return result
+    return {
+        "refresh_id": refresh_id.hex,
+        "region": None,
+        "status": "failed",
+        "error": {
+            "code": "expired",
+            "message": "Refresh status expired. Submit a new refresh.",
+        },
+    }
+
+
 @router.get("/earth-feed")
 async def get_earth_feed(
     lat: float | None = Query(None, description="Current Earth view center latitude"),
     lon: float | None = Query(None, description="Current Earth view center longitude"),
+    region: str | None = Query(
+        None, description="Explicit Earth news region for UE/client integrations"
+    ),
+    categories: str | None = Query(None, description="Comma-separated news category keys"),
+    sources: str | None = Query(None, description="Comma-separated news source ids"),
+    limit: int | None = Query(None, description="Maximum news items to return, capped at 100"),
+    locale: str | None = Query(None, description="Display locale, zh-CN or en-US"),
     db: AsyncSession = Depends(get_db),
 ):
-    return await get_earth_news_payload(lat=lat, lon=lon, db=db)
+    normalized_region = (
+        region.strip().lower() if isinstance(region, str) and region.strip() else None
+    )
+    if normalized_region is not None and normalized_region not in REGION_ANCHORS:
+        raise HTTPException(
+            status_code=422,
+            detail={
+                "message": "Unsupported news region.",
+                "invalid_region": normalized_region,
+                "allowed_regions": list(REGION_ANCHORS.keys()),
+            },
+        )
+    return await get_earth_news_payload(
+        lat=lat,
+        lon=lon,
+        region=normalized_region,
+        categories=_parse_categories(categories),
+        source_ids=_parse_source_ids(sources),
+        limit=_parse_limit(limit),
+        locale=_parse_locale(locale),
+        db=db,
+    )

backend/app/api/v1/settings.py

@@ -3,14 +3,16 @@ from datetime import UTC, datetime
 import os
 from pathlib import Path
 from typing import Optional
+from urllib.parse import urlsplit
 
 from fastapi import APIRouter, Depends, HTTPException, Query, Request, status
-import httpx
 from pydantic import BaseModel, EmailStr, Field
 from dotenv import dotenv_values
 from sqlalchemy import select
 from sqlalchemy.ext.asyncio import AsyncSession
 
+from app.core.logging import get_logger
+from app.core.enums import ProviderApi, TVSourceType, UserRole
 from app.core.security import get_current_user
 from app.core.time import to_iso8601_utc
 from app.core.config import settings as app_settings
@@ -63,14 +65,18 @@ from app.services.llm_provider_catalog import (
     list_fallback_llm_provider_presets,
     refresh_llm_provider_preset,
 )
+from app.services.llm_model_catalog import catalog_error_message, fetch_model_catalog
 from app.services.scheduler import sync_datasource_job
 from app.services.tv_streams import DEFAULT_TV_SETTINGS, get_tv_settings_payload, normalize_tv_settings
 from app.services.persistent_logs import record_audit_log
+from app.services.business_logs import emit_business_log, exception_context
 
 router = APIRouter()
+logger = get_logger(__name__, service="api")
 AI_PROVIDER_QUICK_CONNECT_TIMEOUT_SECONDS = 5
 AI_CONNECTION_TEST_PROMPT_KEY = "ai.connection_test"
-SECRET_REVEAL_ROLES = {"admin", "super_admin"}
+SECRET_REVEAL_ROLES = {UserRole.ADMIN.value, UserRole.SUPER_ADMIN.value}
+LLM_PROVIDER_PRESET_CATEGORY_PREFIX = "llm_provider_preset:"
 
 DEFAULT_SETTINGS = {
     "system": {
@@ -79,6 +85,7 @@ DEFAULT_SETTINGS = {
         "auto_refresh": True,
         "data_retention_days": 30,
         "max_concurrent_tasks": 5,
+        "demo_mode": False,
     },
     "notifications": {
         "email_enabled": False,
@@ -203,6 +210,7 @@ class SystemSettingsUpdate(BaseModel):
     auto_refresh: bool = True
     data_retention_days: int = Field(default=30, ge=1, le=3650)
     max_concurrent_tasks: int = Field(default=5, ge=1, le=50)
+    demo_mode: bool = False
 
 
 class NotificationSettingsUpdate(BaseModel):
@@ -231,7 +239,7 @@ class TVStreamSourceUpdate(BaseModel):
     provider: str = Field(default="Unknown", max_length=100)
     region: str = Field(default="Global", max_length=100)
     language: str = Field(default="und", max_length=32)
-    source_type: str = Field(default="iframe", pattern="^(iframe|hls|video|external|youtube)$")
+    source_type: TVSourceType = TVSourceType.IFRAME
     embed_url: str = ""
     stream_url: str = ""
     homepage_url: str = ""
@@ -274,7 +282,7 @@ class AIProviderIntegrationUpdate(BaseModel):
     service_token: Optional[str] = None
     default_provider: Optional[str] = None
     provider: str = Field(default="minimax", max_length=80)
-    provider_api: str = Field(default="anthropic-messages", max_length=80)
+    provider_api: ProviderApi = ProviderApi.ANTHROPIC_MESSAGES
     base_url: str = Field(default="", max_length=500)
     model: str = Field(default="", max_length=200)
     api_key: Optional[str] = None
@@ -339,7 +347,13 @@ class ExternalIntegrationsUpdate(BaseModel):
 
 
 def merge_with_defaults(category: str, payload: Optional[dict]) -> dict:
-    merged = deepcopy(DEFAULT_SETTINGS[category])
+    if category.startswith(LLM_PROVIDER_PRESET_CATEGORY_PREFIX):
+        defaults = get_fallback_llm_provider_preset(
+            category.removeprefix(LLM_PROVIDER_PRESET_CATEGORY_PREFIX)
+        )
+    else:
+        defaults = DEFAULT_SETTINGS[category]
+    merged = deepcopy(defaults)
     if payload:
         merged.update(payload)
     return merged
@@ -376,6 +390,7 @@ async def get_setting_payload(db: AsyncSession, category: str) -> dict:
 
 
 async def save_setting_payload(db: AsyncSession, category: str, payload: dict) -> dict:
+    merged = merge_with_defaults(category, payload)
     record = await get_setting_record(db, category)
     if record is None:
         record = SystemSetting(category=category, payload=payload)
@@ -385,7 +400,7 @@ async def save_setting_payload(db: AsyncSession, category: str, payload: dict) -
 
     await db.commit()
     await db.refresh(record)
-    return merge_with_defaults(category, record.payload)
+    return merged
 
 
 AI_PROVIDER_ENV_FILE = Path(__file__).resolve().parents[4] / "aiprovider" / ".env"
@@ -418,7 +433,7 @@ def _get_provider_preset(provider: str) -> dict:
     except ValueError:
         return {
             "provider": provider,
-            "provider_api": "openai-completions",
+            "provider_api": ProviderApi.OPENAI_COMPLETIONS.value,
             "base_url": "",
         "model": "",
         "models": [],
@@ -482,12 +497,12 @@ def _provider_defaults(provider: str) -> dict:
     preset = _get_provider_preset(provider)
     return {
         "provider": provider,
-        "provider_api": preset.get("provider_api") or "openai-completions",
+        "provider_api": preset.get("provider_api") or ProviderApi.OPENAI_COMPLETIONS.value,
         "base_url": preset.get("base_url") or "",
         "model": preset.get("model") or "",
         "api_key": "",
         "max_tokens": (
-            1200 if preset.get("provider_api") == "anthropic-messages" else 4096
+            1200 if preset.get("provider_api") == ProviderApi.ANTHROPIC_MESSAGES.value else 4096
         ),
         "anthropic_version": "2023-06-01",
         "model_provider_apis": preset.get("model_provider_apis") or {},
@@ -649,6 +664,12 @@ def _runtime_config_from_ai_payload(ai_payload: dict) -> dict:
         normalized_ai["providers"].get(default_provider) or _provider_defaults(default_provider)
     )
     api_key, _api_key_source = _resolve_provider_api_key(default_provider, provider_config)
+    model_provider_apis = {
+        **(provider_config.get("model_provider_apis") or {}),
+        **(_get_provider_preset(default_provider).get("model_provider_apis") or {}),
+    }
+    if default_provider == "openai" and urlsplit(provider_config.get("base_url") or "").hostname != "api.openai.com":
+        model_provider_apis = {}
     return {
         "service_url": normalized_ai.get("service_url") or app_settings.AI_PROVIDER_SERVICE_URL,
         "service_token": _resolve_service_token(normalized_ai)[0],
@@ -660,14 +681,13 @@ def _runtime_config_from_ai_payload(ai_payload: dict) -> dict:
         ),
         "llm_config": {
             "provider": default_provider,
-            "provider_api": provider_config.get("provider_api") or "anthropic-messages",
+            "provider_api": provider_config.get("provider_api") or ProviderApi.ANTHROPIC_MESSAGES.value,
             "base_url": provider_config.get("base_url") or "",
             "model": provider_config.get("model") or "",
             "api_key": api_key,
             "max_tokens": int(provider_config.get("max_tokens") or 1200),
             "anthropic_version": provider_config.get("anthropic_version") or "2023-06-01",
-            "model_provider_apis": provider_config.get("model_provider_apis") or {},
-            "preset_models": _get_provider_preset(default_provider).get("models") or [],
+            "model_provider_apis": model_provider_apis,
         },
     }
 
@@ -699,6 +719,18 @@ async def _validate_ai_provider_full_connection(ai_payload: dict) -> dict:
         retry_attempts=runtime_config["retry_attempts"],
         llm_config=runtime_config.get("llm_config") or {},
     )
+    await emit_business_log(
+        logger,
+        event="settings.ai_provider.full_connection.start",
+        message="AI provider full connection validation started",
+        category="ai",
+        service="api",
+        module=__name__,
+        context={
+            "provider": runtime_config.get("llm_config", {}).get("provider"),
+            "model": runtime_config.get("llm_config", {}).get("model"),
+        },
+    )
     status_result = await client.get_status()
     if not status_result.configured:
         raise HTTPException(
@@ -715,6 +747,19 @@ async def _validate_ai_provider_full_connection(ai_payload: dict) -> dict:
             constraints=["回复尽量简短。"],
         )
     )
+    await emit_business_log(
+        logger,
+        event="settings.ai_provider.full_connection.success",
+        message="AI provider full connection validation completed",
+        category="ai",
+        service="api",
+        module=__name__,
+        context={
+            "provider": analysis_result.provider,
+            "model": analysis_result.model,
+            "configured": status_result.configured,
+        },
+    )
     return {
         "status": status_result.model_dump(),
         "provider": analysis_result.provider,
@@ -722,156 +767,35 @@ async def _validate_ai_provider_full_connection(ai_payload: dict) -> dict:
     }
 
 
-def _join_provider_url(base_url: str, path: str) -> str:
-    return f"{base_url.rstrip('/')}/{path.lstrip('/')}"
-
-
-def _extract_model_ids(payload: dict) -> list[str]:
-    data = payload.get("data") if isinstance(payload, dict) else None
-    if isinstance(data, list):
-        return [
-            str(item.get("id"))
-            for item in data
-            if isinstance(item, dict) and item.get("id")
-        ]
-    models = payload.get("models") if isinstance(payload, dict) else None
-    if isinstance(models, list):
-        return [
-            str(item.get("name") or item.get("model") or item.get("id") or item)
-            for item in models
-            if item
-        ]
-    return []
-
-
-def _contains_model(model_ids: list[str], model: str) -> bool:
-    normalized_model = model.strip().lower()
-    return any(str(item).strip().lower() == normalized_model for item in model_ids)
-
-
 async def _check_ai_provider_lightweight(llm_config: dict, timeout_seconds: int) -> dict:
     provider = _normalize_provider_id(llm_config.get("provider") or "")
-    configured_api = str(llm_config.get("provider_api") or "").strip() or "openai-completions"
+    provider_api = str(llm_config.get("provider_api") or ProviderApi.OPENAI_COMPLETIONS.value)
     model = str(llm_config.get("model") or "").strip()
-    base_url = str(llm_config.get("base_url") or "").strip().rstrip("/")
+    base_url = str(llm_config.get("base_url") or "").strip()
     api_key = str(llm_config.get("api_key") or "").strip()
-    provider_api = configured_api
-    model_provider_apis = llm_config.get("model_provider_apis")
-    if isinstance(model_provider_apis, dict):
-        provider_api = str(model_provider_apis.get(model) or provider_api)
-    preset_models = [
-        str(item)
-        for item in (llm_config.get("preset_models") or [])
-        if str(item).strip()
-    ]
-
-    if not provider or not base_url or not model:
-        return {
-            "success": False,
-            "connected": False,
-            "message": "当前 provider/base_url/model 未完整配置。",
-            "mode": "lightweight_config",
-        }
-    if provider_api != "ollama-generate" and not api_key:
-        return {
-            "success": False,
-            "connected": False,
-            "message": "当前 provider 未配置 API Key。",
-            "mode": "lightweight_config",
-        }
-
-    if provider == "opencode-go":
-        url = _join_provider_url(base_url, "/models")
-        headers = {"Authorization": f"Bearer {api_key}"}
-    elif provider_api == "ollama-generate":
-        url = _join_provider_url(base_url, "/api/tags")
-        headers: dict[str, str] = {}
-    elif provider_api == "openai-completions":
-        url = _join_provider_url(base_url, "/models")
-        headers = {"Authorization": f"Bearer {api_key}"}
-    elif provider_api == "anthropic-messages":
-        url = _join_provider_url(base_url, "/models")
-        headers = {
-            "x-api-key": api_key,
-            "anthropic-version": str(llm_config.get("anthropic_version") or "2023-06-01"),
-        }
-    else:
-        return {
-            "success": False,
-            "connected": False,
-            "message": f"当前 provider_api 不支持轻量连通性测试: {provider_api}",
-            "mode": "lightweight_unsupported",
-        }
-
+    result = {
+        "success": False, "connected": False, "mode": "lightweight_models",
+        "provider": provider, "model": model,
+    }
+    if not base_url or not model:
+        return {**result, "message": "当前 provider/base_url/model 未完整配置。"}
+    if provider_api != ProviderApi.OLLAMA_GENERATE.value and not api_key:
+        return {**result, "message": "当前 provider 未配置 API Key。"}
     try:
-        async with httpx.AsyncClient(timeout=min(timeout_seconds, AI_PROVIDER_QUICK_CONNECT_TIMEOUT_SECONDS)) as client:
-            response = await client.get(url, headers=headers)
-            response.raise_for_status()
-            payload = response.json()
-    except httpx.HTTPStatusError as exc:
-        detail = exc.response.text or exc.response.reason_phrase
-        if exc.response.status_code == 404 and _contains_model(preset_models, model):
-            return {
-                "success": True,
-                "connected": True,
-                "message": "轻量连通性测试通过；当前 provider 不提供可用的模型目录，已按内置模型预设确认。",
-                "mode": "lightweight_preset",
-                "provider": provider,
-                "provider_api": provider_api,
-                "model": model,
-                "url": url,
-            }
-        return {
-            "success": False,
-            "connected": False,
-            "message": f"轻量连通性测试失败: HTTP {exc.response.status_code} {detail}",
-            "mode": "lightweight_models",
-            "url": url,
-        }
+        catalog = await fetch_model_catalog(
+            provider, base_url, provider_api, api_key,
+            str(llm_config.get("anthropic_version") or "2023-06-01"), timeout_seconds,
+        )
     except Exception as exc:
-        return {
-            "success": False,
-            "connected": False,
-            "message": f"轻量连通性测试失败: {exc}",
-            "mode": "lightweight_models",
-            "url": url,
-        }
-
-    model_ids = _extract_model_ids(payload)
-    if model_ids and not _contains_model(model_ids, model):
-        if _contains_model(preset_models, model):
-            return {
-                "success": True,
-                "connected": True,
-                "message": "轻量连通性测试通过；provider 模型目录未返回当前别名，已按内置模型预设确认。",
-                "mode": "lightweight_models_with_preset_alias",
-                "provider": provider,
-                "provider_api": provider_api,
-                "model": model,
-                "models_count": len(model_ids),
-                "url": url,
-            }
-        return {
-            "success": False,
-            "connected": False,
-            "message": f"连接可用，但模型目录中没有当前模型: {model}",
-            "mode": "lightweight_models",
-            "provider": provider,
-            "model": model,
-            "models_count": len(model_ids),
-            "url": url,
-        }
-
+        return {**result, "message": catalog_error_message(exc)}
+    result.update({"url": catalog.url, "models_count": len(catalog.models)})
+    if model.casefold() not in {item.casefold() for item in catalog.models}:
+        return {**result, "message": f"模型目录查询成功，但当前模型不在目录中：{model}"}
+    model_apis = llm_config.get("model_provider_apis") or {}
     return {
-        "success": True,
-        "connected": True,
-        "message": "轻量连通性测试通过",
-        "mode": "lightweight_models",
-        "provider": provider,
-        "provider_api": provider_api,
-        "model": model,
-        "models_count": len(model_ids),
-        "url": url,
+        **result, "success": True, "connected": True,
+        "provider_api": model_apis.get(model) or provider_api,
+        "message": "模型目录查询成功，当前模型已找到；尚未执行生成调用。",
     }
 
 
@@ -1135,7 +1059,7 @@ async def serialize_external_integrations(db: AsyncSession) -> dict:
         api_key, api_key_source = _resolve_provider_api_key(provider_id, provider_config)
         providers_payload[provider_id] = {
             "provider": provider_id,
-            "provider_api": provider_config.get("provider_api") or "openai-completions",
+            "provider_api": provider_config.get("provider_api") or ProviderApi.OPENAI_COMPLETIONS.value,
             "base_url": provider_config.get("base_url") or "",
             "model": provider_config.get("model") or "",
             "api_key": _mask_secret(api_key, api_key_source),
@@ -1185,7 +1109,7 @@ async def serialize_external_integrations(db: AsyncSession) -> dict:
             "service_token": _mask_secret(*_resolve_service_token(normalized_ai)),
             "default_provider": default_provider,
             "provider": default_provider,
-            "provider_api": display_llm_config.get("provider_api") or "anthropic-messages",
+            "provider_api": display_llm_config.get("provider_api") or ProviderApi.ANTHROPIC_MESSAGES.value,
             "base_url": display_llm_config.get("base_url") or "https://api.minimaxi.com/anthropic",
             "model": display_llm_config.get("model") or "MiniMax-M2.7",
             "api_key": display_llm_config.get("api_key") or _mask_secret(None),
@@ -1428,7 +1352,7 @@ async def update_smtp_settings(
     current_user: User = Depends(get_current_user),
     db: AsyncSession = Depends(get_db),
 ):
-    if current_user.role not in ("admin", "super_admin"):
+    if current_user.role not in (UserRole.ADMIN.value, UserRole.SUPER_ADMIN.value):
         raise HTTPException(status_code=403, detail="Only administrators can change SMTP settings")
     current = await get_setting_payload(db, "smtp")
     merged = _build_smtp_payload(current, payload)
@@ -1442,7 +1366,7 @@ async def test_smtp_settings(
     current_user: User = Depends(get_current_user),
     db: AsyncSession = Depends(get_db),
 ):
-    if current_user.role not in ("admin", "super_admin"):
+    if current_user.role not in (UserRole.ADMIN.value, UserRole.SUPER_ADMIN.value):
         raise HTTPException(status_code=403, detail="Only administrators can test SMTP settings")
     from app.services.email import EmailError, send_email
 
@@ -1612,9 +1536,34 @@ async def connect_ai_provider_integration(
         llm_config=quick_llm_config,
     )
 
+    await emit_business_log(
+        logger,
+        event="settings.ai_provider.connect.start",
+        message="AI provider connection test started",
+        category="ai",
+        service="api",
+        module=__name__,
+        user_id=current_user.id,
+        context={
+            "provider": payload.provider,
+            "model": payload.model,
+            "timeout_seconds": min(int(runtime_config["timeout_seconds"] or 60), AI_PROVIDER_QUICK_CONNECT_TIMEOUT_SECONDS),
+        },
+    )
     try:
         status_result = await client.get_status()
         if not status_result.configured:
+            await emit_business_log(
+                logger,
+                event="settings.ai_provider.connect.failed",
+                message="AI provider connection test failed because provider is incomplete",
+                category="ai",
+                level="warning",
+                service="api",
+                module=__name__,
+                user_id=current_user.id,
+                context={"provider": payload.provider, "model": payload.model, "configured": False},
+            )
             return {
                 "success": False,
                 "connected": False,
@@ -1628,6 +1577,22 @@ async def connect_ai_provider_integration(
                 AI_PROVIDER_QUICK_CONNECT_TIMEOUT_SECONDS,
             ),
         )
+        await emit_business_log(
+            logger,
+            event=("settings.ai_provider.connect.success" if lightweight_result["success"]
+                   else "settings.ai_provider.connect.failed"),
+            message="AI provider connection test completed",
+            category="ai",
+            service="api",
+            module=__name__,
+            user_id=current_user.id,
+            context={
+                "provider": payload.provider,
+                "model": payload.model,
+                "configured": True,
+                "connected": lightweight_result["connected"],
+            },
+        )
         return {
             **lightweight_result,
             "status": status_result.model_dump(),
@@ -1639,6 +1604,17 @@ async def connect_ai_provider_integration(
             "message": str(exc.detail),
         }
     except Exception as exc:
+        await emit_business_log(
+            logger,
+            event="settings.ai_provider.connect.failed",
+            message="AI provider connection test failed",
+            category="ai",
+            level="error",
+            service="api",
+            module=__name__,
+            user_id=current_user.id,
+            context=exception_context(exc, {"provider": payload.provider, "model": payload.model}),
+        )
         return {
             "success": False,
             "connected": False,
@@ -1786,8 +1762,28 @@ async def connect_web_search_integration(
     runtime_config = _runtime_config_from_web_search_payload(draft_web_search_payload)
     client = WebSearchClient(runtime_config)
 
+    await emit_business_log(
+        logger,
+        event="settings.web_search.connect.start",
+        message="WebSearch connection test started",
+        category="ai_tool",
+        service="api",
+        module=__name__,
+        user_id=current_user.id,
+        context={"provider": runtime_config.default_provider},
+    )
     try:
         results = await client.test_connection()
+        await emit_business_log(
+            logger,
+            event="settings.web_search.connect.success",
+            message="WebSearch connection test completed",
+            category="ai_tool",
+            service="api",
+            module=__name__,
+            user_id=current_user.id,
+            context={"provider": runtime_config.default_provider, "result_count": len(results)},
+        )
         return {
             "success": True,
             "connected": True,
@@ -1796,18 +1792,51 @@ async def connect_web_search_integration(
             "results": [item.model_dump(mode="json") for item in results[:3]],
         }
     except WebSearchConfigurationError as exc:
+        await emit_business_log(
+            logger,
+            event="settings.web_search.connect.failed",
+            message="WebSearch connection test failed because configuration is incomplete",
+            category="ai_tool",
+            level="warning",
+            service="api",
+            module=__name__,
+            user_id=current_user.id,
+            context=exception_context(exc, {"provider": runtime_config.default_provider}),
+        )
         return {
             "success": False,
             "connected": False,
             "message": str(exc),
         }
     except WebSearchError as exc:
+        await emit_business_log(
+            logger,
+            event="settings.web_search.connect.failed",
+            message="WebSearch connection test failed",
+            category="ai_tool",
+            level="error",
+            service="api",
+            module=__name__,
+            user_id=current_user.id,
+            context=exception_context(exc, {"provider": runtime_config.default_provider}),
+        )
         return {
             "success": False,
             "connected": False,
             "message": str(exc),
         }
     except Exception as exc:
+        await emit_business_log(
+            logger,
+            event="settings.web_search.connect.failed",
+            message="WebSearch connection test failed",
+            category="ai_tool",
+            level="error",
+            service="api",
+            module=__name__,
+            user_id=current_user.id,
+            context=exception_context(exc, {"provider": runtime_config.default_provider}),
+        )
         return {
             "success": False,
             "connected": False,
@@ -1835,17 +1864,60 @@ async def generate_provider_credential_guide(
     ai_client: AIProviderClient = Depends(get_ai_provider_client),
 ):
     try:
+        await emit_business_log(
+            logger,
+            event="settings.credential_guide.generate.start",
+            message="Credential guide generation started",
+            category="ai",
+            service="api",
+            module=__name__,
+            user_id=current_user.id,
+            context={"provider": provider},
+        )
         web_search_client = await get_web_search_client(db)
-        return {
-            "guide": await generate_credential_guide(
+        guide = await generate_credential_guide(
                 db,
                 provider,
                 ai_client,
                 web_search_client,
             )
-        }
+        await emit_business_log(
+            logger,
+            event="settings.credential_guide.generate.success",
+            message="Credential guide generation completed",
+            category="ai",
+            service="api",
+            module=__name__,
+            user_id=current_user.id,
+            context={"provider": provider},
+        )
+        return {"guide": guide}
     except ValueError as exc:
+        await emit_business_log(
+            logger,
+            event="settings.credential_guide.generate.failed",
+            message="Credential guide generation failed",
+            category="ai",
+            level="warning",
+            service="api",
+            module=__name__,
+            user_id=current_user.id,
+            context=exception_context(exc, {"provider": provider}),
+        )
         raise HTTPException(status_code=404, detail=str(exc)) from exc
+    except Exception as exc:
+        await emit_business_log(
+            logger,
+            event="settings.credential_guide.generate.failed",
+            message="Credential guide generation failed",
+            category="ai",
+            level="error",
+            service="api",
+            module=__name__,
+            user_id=current_user.id,
+            context=exception_context(exc, {"provider": provider}),
+        )
+        raise
 
 
 @router.post("/credential-guides/{provider}/reset")
@@ -1863,8 +1935,18 @@ async def reset_provider_credential_guide(
 @router.get("/integrations/ai-provider/presets")
 async def get_ai_provider_presets(
     current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
 ):
-    return {"data": list_fallback_llm_provider_presets()}
+    presets = list_fallback_llm_provider_presets()
+    saved = await get_setting_payloads(
+        db, [f"{LLM_PROVIDER_PRESET_CATEGORY_PREFIX}{preset['provider']}" for preset in presets]
+    )
+    return {
+        "data": [
+            {**preset, **saved[f"{LLM_PROVIDER_PRESET_CATEGORY_PREFIX}{preset['provider']}"]}
+            for preset in presets
+        ]
+    }
 
 
 @router.post("/integrations/ai-provider/presets/{provider}/refresh")
@@ -1872,22 +1954,52 @@ async def refresh_ai_provider_preset(
     provider: str,
     current_user: User = Depends(get_current_user),
     db: AsyncSession = Depends(get_db),
+    payload: AIProviderIntegrationUpdate | None = None,
 ):
     try:
         provider_id = _normalize_provider_id(provider)
-        api_key = None
-        if provider_id == "opencode-go":
-            current_payload = await get_setting_payload(db, "external_integrations")
-            ai_payload = _normalize_ai_provider_payload(current_payload.get("ai_provider") or {})
-            provider_config = ai_payload["providers"].get(provider_id) or _provider_defaults(provider_id)
-            api_key, _api_key_source = _resolve_provider_api_key(provider_id, provider_config)
-        return {"data": await refresh_llm_provider_preset(provider_id, api_key=api_key)}
+        get_fallback_llm_provider_preset(provider_id)
     except ValueError as exc:
         raise HTTPException(status_code=404, detail=str(exc)) from exc
+
+    current_payload = await get_setting_payload(db, "external_integrations")
+    if payload is not None:
+        if _normalize_provider_id(payload.provider) != provider_id:
+            raise HTTPException(status_code=400, detail="刷新供应商与表单供应商不一致。")
+        ai_payload = _build_ai_provider_payload(current_payload, payload)
+    else:
+        ai_payload = _normalize_ai_provider_payload(current_payload.get("ai_provider") or {})
+    provider_config = ai_payload["providers"].get(provider_id) or _provider_defaults(provider_id)
+    api_key, _api_key_source = _resolve_provider_api_key(provider_id, provider_config)
+    if not api_key and provider_id not in {"ollama", "opencode-go", "openrouter"}:
+        raise HTTPException(
+            status_code=400, detail="请先配置当前供应商的 API Key，再刷新模型列表。",
+        )
+    try:
+        refreshed = await refresh_llm_provider_preset(
+            provider_id, api_key=api_key, base_url=provider_config.get("base_url"),
+            provider_api=provider_config.get("provider_api"),
+            anthropic_version=provider_config.get("anthropic_version") or "2023-06-01",
+        )
     except Exception as exc:
-        fallback = get_fallback_llm_provider_preset(provider)
-        fallback["refresh_error"] = str(exc)
-        return {"data": fallback}
+        logger.warning(
+            "LLM provider catalog refresh failed",
+            extra={
+                "event": "settings.ai_provider.catalog.failed",
+                "context": {
+                    "provider": provider_id,
+                    "error_type": type(exc).__name__,
+                },
+            },
+        )
+        raise HTTPException(
+            status_code=502,
+            detail=f"{catalog_error_message(exc)} 已保留上次模型列表。",
+        ) from exc
+
+    refreshed["refreshed_at"] = to_iso8601_utc(datetime.now(UTC))
+    await save_setting_payload(db, f"{LLM_PROVIDER_PRESET_CATEGORY_PREFIX}{provider_id}", refreshed)
+    return {"data": refreshed}
 
 
 @router.put("/integrations")

backend/app/api/v1/system_control.py

@@ -1,20 +1,19 @@
 from __future__ import annotations
 
 import os
+import secrets
 import subprocess
 import sys
 
 from datetime import datetime
 
-from fastapi import APIRouter, Depends, HTTPException, Query, Request, status
+from fastapi import APIRouter, Depends, Header, HTTPException, Query, Request, status
 from pydantic import BaseModel
-from sqlalchemy import select
 from sqlalchemy.ext.asyncio import AsyncSession
 
-from app.core.config import ROOT_DIR
+from app.core.config import ROOT_DIR, settings
 from app.core.security import get_current_user
 from app.db.session import get_db
-from app.models.system_log import AuditLog, SystemLog
 from app.models.user import User
 from app.services.persistent_logs import record_audit_log, record_system_log
 from app.services.system_control import (
@@ -37,7 +36,11 @@ from app.services.system_logs import (
     append_buffer_log,
     list_log_sources,
     normalize_log_level,
+    read_database_log_snapshot,
     read_log_snapshot,
+    read_observability_group_events,
+    read_observability_groups,
+    read_observability_raw_events,
 )
 from app.services.earth_layer_cache import earth_layer_cache
 
@@ -109,12 +112,104 @@ class EarthClientLogEventCreate(BaseModel):
     url: str | None = None
     module: str | None = None
     detail: str | None = None
+    fingerprint: str | None = None
+    occurrence_count: int = 1
+    metadata: dict[str, object] | None = None
 
 
 class EarthClientLogEventResponse(BaseModel):
     accepted: bool
     source_id: str
     level: str
+    fingerprint: str | None = None
+
+
+class ServiceLogEventCreate(BaseModel):
+    source: str = "ai-provider"
+    service: str = "ai-provider"
+    module: str | None = None
+    category: str | None = None
+    event: str = "service.runtime_log"
+    level: str = "error"
+    message: str
+    fingerprint: str | None = None
+    occurrence_count: int = 1
+    request_id: str | None = None
+    trace_id: str | None = None
+    task_id: str | None = None
+    source_id: int | str | None = None
+    provider: str | None = None
+    context: dict[str, object] | None = None
+
+
+async def ingest_client_log_event(
+    source_id: str,
+    *,
+    service: str,
+    event: str,
+    default_module: str,
+    default_category: str,
+    payload: EarthClientLogEventCreate,
+    request: Request,
+) -> EarthClientLogEventResponse:
+    normalized_level = normalize_log_level(payload.level)
+    append_buffer_log(
+        source_id,
+        level=normalized_level,
+        message=payload.message,
+        context={
+            "category": payload.category or "",
+            "url": payload.url or "",
+            "module": payload.module or "",
+            "detail": payload.detail or "",
+            "fingerprint": payload.fingerprint or "",
+            "occurrence_count": max(1, int(payload.occurrence_count or 1)),
+            "metadata": payload.metadata or {},
+        },
+    )
+    await record_system_log(
+        source=source_id,
+        service=service,
+        module=payload.module or default_module,
+        event=event,
+        level=normalized_level,
+        message=payload.message,
+        category=payload.category or default_category,
+        context={
+            "url": payload.url or "",
+            "detail": payload.detail or "",
+            "module": payload.module or "",
+            "client_ip": request.client.host if request.client else "",
+            "metadata": payload.metadata or {},
+        },
+        fingerprint=payload.fingerprint,
+        occurrence_count=max(1, int(payload.occurrence_count or 1)),
+    )
+    return EarthClientLogEventResponse(accepted=True, source_id=source_id, level=normalized_level, fingerprint=payload.fingerprint)
+
+
+def require_observability_ingest_token(
+    authorization: str | None,
+    ingest_token: str | None,
+) -> None:
+    expected_token = settings.OBSERVABILITY_INGEST_TOKEN.strip()
+    if not expected_token:
+        raise HTTPException(
+            status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
+            detail="Observability service ingestion is not configured",
+        )
+    provided = ""
+    if ingest_token:
+        provided = ingest_token.strip()
+    elif authorization:
+        scheme, _, token = authorization.partition(" ")
+        if scheme.lower() == "bearer":
+            provided = token.strip()
+    if not provided or not secrets.compare_digest(provided, expected_token):
+        raise HTTPException(
+            status_code=status.HTTP_403_FORBIDDEN,
+            detail="Invalid observability ingestion token",
+        )
 
 
 class EarthLayerCacheStatusResponse(BaseModel):
@@ -339,95 +434,116 @@ async def get_system_log_sources(
     }
 
 
-async def read_database_log_snapshot(
-    source_id: str,
-    *,
-    limit: int,
-    level: str,
-    levels: str | None,
-    start_date: str | None,
-    end_date: str | None,
-    search: str | None,
-    db: AsyncSession,
-) -> dict | None:
-    selected_levels = set(normalize_log_level(item) for item in (levels or level).split(",") if item.strip())
-    selected_levels.discard("all")
-    search_query = (search or "").strip().lower()
-    lines: list[str] = []
+@router.get("/logs/observability/groups")
+async def get_observability_log_groups(
+    limit: int = DEFAULT_LOG_LINE_LIMIT,
+    level: str = "all",
+    levels: str | None = Query(None, description="Comma-separated log levels"),
+    start_date: str | None = Query(None, description="Filter logs from this date (YYYY-MM-DD)"),
+    end_date: str | None = Query(None, description="Filter logs until this date (YYYY-MM-DD)"),
+    search: str | None = Query(None, description="Case-insensitive substring search"),
+    current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    ensure_super_admin(current_user)
+    if limit < 1 or limit > MAX_LOG_LINE_LIMIT:
+        raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail=f"limit must be between 1 and {MAX_LOG_LINE_LIMIT}")
+    normalized_start_date = validate_log_date(start_date, "start_date")
+    normalized_end_date = validate_log_date(end_date, "end_date")
+    if normalized_start_date and normalized_end_date and normalized_start_date > normalized_end_date:
+        raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail="start_date must be earlier than or equal to end_date")
+    return await read_observability_groups(
+        limit=limit,
+        level=level,
+        levels=levels,
+        start_date=normalized_start_date,
+        end_date=normalized_end_date,
+        search=search,
+        db=db,
+    )
 
-    if source_id == "system-db":
-        query = select(SystemLog).order_by(SystemLog.occurred_at.desc().nullslast(), SystemLog.id.desc()).limit(limit * 5)
-        result = await db.execute(query)
-        records = result.scalars().all()
-        for record in records:
-            record_level = normalize_log_level(record.level)
-            if selected_levels and record_level not in selected_levels:
-                continue
-            occurred_at = record.occurred_at.date().isoformat() if record.occurred_at else ""
-            if start_date and occurred_at and occurred_at < start_date:
-                continue
-            if end_date and occurred_at and occurred_at > end_date:
-                continue
-            line = " ".join(
-                part
-                for part in [
-                    record.occurred_at.isoformat() if record.occurred_at else "",
-                    record_level.upper(),
-                    record.source,
-                    record.event or "",
-                    record.message,
-                ]
-                if part
-            )
-            if search_query and search_query not in line.lower():
-                continue
-            lines.append(line)
-    elif source_id == "audit-db":
-        query = select(AuditLog).order_by(AuditLog.occurred_at.desc().nullslast(), AuditLog.id.desc()).limit(limit * 5)
-        result = await db.execute(query)
-        records = result.scalars().all()
-        for record in records:
-            occurred_at = record.occurred_at.date().isoformat() if record.occurred_at else ""
-            if start_date and occurred_at and occurred_at < start_date:
-                continue
-            if end_date and occurred_at and occurred_at > end_date:
-                continue
-            line = " ".join(
-                part
-                for part in [
-                    record.occurred_at.isoformat() if record.occurred_at else "",
-                    "INFO",
-                    record.action,
-                    record.target_type or "",
-                    record.target_id or "",
-                    record.result or "",
-                ]
-                if part
-            )
-            if search_query and search_query not in line.lower():
-                continue
-            lines.append(line)
-    else:
-        return None
 
-    lines = list(reversed(lines[:limit]))
-    return {
-        "source_id": source_id,
-        "name": "系统事件" if source_id == "system-db" else "审计事件",
-        "kind": "database",
-        "location": "table://system_logs" if source_id == "system-db" else "table://audit_logs",
-        "description": "数据库持久化日志",
-        "category": "database" if source_id == "system-db" else "audit",
-        "status": "ok" if lines else "empty",
-        "level": level,
-        "selected_levels": sorted(selected_levels),
-        "search_query": search or "",
-        "available_levels": ["all", "error", "warning", "info", "debug"],
-        "daily_markers": [],
-        "line_limit": limit,
-        "line_count": len(lines),
-        "lines": lines,
-    }
+@router.get("/logs/observability/groups/{fingerprint}/events")
+async def get_observability_group_events(
+    fingerprint: str,
+    limit: int = DEFAULT_LOG_LINE_LIMIT,
+    current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    ensure_super_admin(current_user)
+    if limit < 1 or limit > MAX_LOG_LINE_LIMIT:
+        raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail=f"limit must be between 1 and {MAX_LOG_LINE_LIMIT}")
+    payload = await read_observability_group_events(fingerprint, limit=limit, db=db)
+    if payload is None:
+        raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Observability group not found")
+    return payload
+
+
+@router.get("/logs/observability/raw")
+async def get_observability_raw_events(
+    limit: int = DEFAULT_LOG_LINE_LIMIT,
+    level: str = "all",
+    levels: str | None = Query(None, description="Comma-separated log levels"),
+    start_date: str | None = Query(None, description="Filter logs from this date (YYYY-MM-DD)"),
+    end_date: str | None = Query(None, description="Filter logs until this date (YYYY-MM-DD)"),
+    search: str | None = Query(None, description="Case-insensitive substring search"),
+    current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    ensure_super_admin(current_user)
+    if limit < 1 or limit > MAX_LOG_LINE_LIMIT:
+        raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail=f"limit must be between 1 and {MAX_LOG_LINE_LIMIT}")
+    normalized_start_date = validate_log_date(start_date, "start_date")
+    normalized_end_date = validate_log_date(end_date, "end_date")
+    return await read_observability_raw_events(
+        limit=limit,
+        level=level,
+        levels=levels,
+        start_date=normalized_start_date,
+        end_date=normalized_end_date,
+        search=search,
+        db=db,
+    )
+
+
+@router.post("/logs/service", response_model=EarthClientLogEventResponse)
+async def ingest_service_log(
+    payload: ServiceLogEventCreate,
+    authorization: str | None = Header(default=None),
+    ingest_token: str | None = Header(default=None, alias="X-Planet-Observability-Token"),
+):
+    require_observability_ingest_token(authorization, ingest_token)
+    normalized_level = normalize_log_level(payload.level)
+    source = (payload.source or "ai-provider").strip() or "ai-provider"
+    context = dict(payload.context or {})
+    if payload.request_id:
+        context["request_id"] = payload.request_id
+    if payload.trace_id:
+        context["trace_id"] = payload.trace_id
+    if payload.task_id:
+        context["task_id"] = payload.task_id
+    if payload.source_id is not None:
+        context["source_id"] = payload.source_id
+    if payload.provider:
+        context["provider"] = payload.provider
+    await record_system_log(
+        source=source,
+        service=(payload.service or source).strip() or source,
+        module=payload.module or source,
+        event=(payload.event or "service.runtime_log").strip() or "service.runtime_log",
+        level=normalized_level,
+        message=payload.message,
+        category=payload.category or "service-runtime",
+        context=context,
+        fingerprint=payload.fingerprint,
+        occurrence_count=max(1, int(payload.occurrence_count or 1)),
+    )
+    return EarthClientLogEventResponse(
+        accepted=True,
+        source_id=source,
+        level=normalized_level,
+        fingerprint=payload.fingerprint,
+    )
 
 
 @router.get("/logs/{source_id}", response_model=SystemLogSnapshotResponse)
@@ -493,31 +609,28 @@ async def ingest_earth_client_log(
     payload: EarthClientLogEventCreate,
     request: Request,
 ):
-    normalized_level = normalize_log_level(payload.level)
-    append_buffer_log(
+    return await ingest_client_log_event(
         "earth-client",
-        level=normalized_level,
-        message=payload.message,
-        context={
-            "category": payload.category or "",
-            "url": payload.url or "",
-            "module": payload.module or "",
-            "detail": payload.detail or "",
-        },
-    )
-    await record_system_log(
-        source="earth-client",
         service="earth",
-        module=payload.module or "earth-client",
         event="earth.client.runtime_log",
-        level=normalized_level,
-        message=payload.message,
-        category=payload.category or "client-runtime",
-        context={
-            "url": payload.url or "",
-            "detail": payload.detail or "",
-            "module": payload.module or "",
-            "client_ip": request.client.host if request.client else "",
-        },
+        default_module="earth-client",
+        default_category="client-runtime",
+        payload=payload,
+        request=request,
+    )
+
+
+@router.post("/logs/admin-client", response_model=EarthClientLogEventResponse)
+async def ingest_admin_client_log(
+    payload: EarthClientLogEventCreate,
+    request: Request,
+):
+    return await ingest_client_log_event(
+        "admin-client",
+        service="admin",
+        event="admin.client.runtime_log",
+        default_module="admin-client",
+        default_category="client-runtime",
+        payload=payload,
+        request=request,
     )
-    return {"accepted": True, "source_id": "earth-client", "level": normalized_level}

backend/app/api/v1/tasks.py

@@ -29,7 +29,8 @@ async def list_tasks(
         SELECT ct.id, ct.datasource_id, ds.name as datasource_name, ct.status,
                ct.started_at, ct.completed_at, ct.records_processed, ct.error_message,
                ct.phase, ct.phase_progress, ct.phase_message, ct.phase_current,
-               ct.phase_total, ct.phase_unit, ct.total_records, ct.progress
+               ct.phase_total, ct.phase_unit, ct.total_records, ct.progress,
+               ct.task_type, ct.source, ds.source as datasource_source
         FROM collection_tasks ct
         JOIN data_sources ds ON ct.datasource_id = ds.id
         WHERE 1=1
@@ -39,12 +40,19 @@ async def list_tasks(
 
     if datasource_id:
         query += " AND ct.datasource_id = :datasource_id"
-        count_query += " WHERE ct.datasource_id = :datasource_id"
+        count_query += " AND ct.datasource_id = :datasource_id"
         params["datasource_id"] = datasource_id
     if status:
-        query += " AND ct.status = :status"
-        count_query += " AND ct.status = :status"
-        params["status"] = status
+        statuses = [item.strip() for item in status.split(",") if item.strip()]
+        if len(statuses) > 1:
+            placeholders = ", ".join(f":status_{index}" for index, _item in enumerate(statuses))
+            query += f" AND ct.status IN ({placeholders})"
+            count_query += f" AND ct.status IN ({placeholders})"
+            params.update({f"status_{index}": item for index, item in enumerate(statuses)})
+        else:
+            query += " AND ct.status = :status"
+            count_query += " AND ct.status = :status"
+            params["status"] = statuses[0] if statuses else status
 
     query += f" ORDER BY ct.created_at DESC LIMIT {page_size} OFFSET {offset}"
 
@@ -76,6 +84,9 @@ async def list_tasks(
                 "phase_unit": t[13],
                 "total_records": t[14],
                 "progress": t[15],
+                "task_type": t[16],
+                "source": t[17] or t[18],
+                "datasource_source": t[18],
             }
             for t in tasks
         ],

backend/app/api/v1/tv.py

@@ -1,3 +1,4 @@
+import re
 from urllib.parse import quote, urljoin
 
 import httpx
@@ -6,16 +7,41 @@ from fastapi.responses import Response
 from sqlalchemy.ext.asyncio import AsyncSession
 
 from app.db.session import get_db
+from app.services.tv_catalog import get_tv_catalog_page
 from app.services.tv_streams import get_public_tv_payload, is_allowed_tv_proxy_url
 
 router = APIRouter()
 
+_HLS_URI_ATTRIBUTE_RE = re.compile(r'URI="([^"]+)"')
+
+
+def _proxied_tv_url(url: str) -> str:
+    return f"/api/v1/tv/proxy?url={quote(url, safe='')}"
+
+
+def _rewrite_hls_uri_attributes(line: str, *, base_url: str) -> str:
+    def replace(match: re.Match[str]) -> str:
+        uri = match.group(1)
+        absolute_url = urljoin(base_url, uri)
+        return f'URI="{_proxied_tv_url(absolute_url)}"'
+
+    return _HLS_URI_ATTRIBUTE_RE.sub(replace, line)
+
+
+def _should_strip_hls_metadata_line(line: str) -> bool:
+    normalized = line.strip().upper()
+    return normalized.startswith("#EXT-X-MEDIA:") and "TYPE=SUBTITLES" in normalized
+
 
 @router.get("/streams")
 async def list_public_tv_streams(
+    offset: int = Query(0, ge=0),
+    limit: int = Query(50, ge=1, le=100),
+    q: str = Query("", max_length=200),
+    selected_id: str | None = Query(None, max_length=200),
     db: AsyncSession = Depends(get_db),
 ):
-    return await get_public_tv_payload(db)
+    return await get_tv_catalog_page(db, offset=offset, limit=limit, q=q, selected_id=selected_id)
 
 
 @router.get("/proxy")
@@ -56,11 +82,16 @@ async def proxy_tv_stream(
         rewritten_lines: list[str] = []
         for line in manifest_text.splitlines():
             stripped = line.strip()
-            if not stripped or stripped.startswith("#"):
+            if not stripped:
                 rewritten_lines.append(line)
                 continue
+            if stripped.startswith("#"):
+                if _should_strip_hls_metadata_line(line):
+                    continue
+                rewritten_lines.append(_rewrite_hls_uri_attributes(line, base_url=response_url))
+                continue
             absolute_url = urljoin(response_url, stripped)
-            rewritten_lines.append(f"/api/v1/tv/proxy?url={quote(absolute_url, safe='')}")
+            rewritten_lines.append(_proxied_tv_url(absolute_url))
         return Response(
             content="\n".join(rewritten_lines),
             media_type="application/vnd.apple.mpegurl",

backend/app/api/v1/users.py

@@ -5,6 +5,7 @@ from fastapi import APIRouter, Depends, HTTPException, status
 from sqlalchemy.ext.asyncio import AsyncSession
 from sqlalchemy import text
 
+from app.core.enums import UserRole
 from app.core.security import get_current_user, get_password_hash
 from app.db.session import get_db
 from app.models.user import User
@@ -13,6 +14,8 @@ from app.schemas.user import UserCreate, UserUpdate
 router = APIRouter()
 
 VALID_GATEKEEPER_GROUPS = {"docs_user", "docs_developer", "docs_admin"}
+ADMIN_ROLES = [UserRole.SUPER_ADMIN.value, UserRole.ADMIN.value]
+SUPER_ADMIN_ROLES = [UserRole.SUPER_ADMIN.value]
 
 
 def check_permission(current_user: User, required_roles: List[str]) -> bool:
@@ -32,7 +35,7 @@ async def list_users(
     current_user: User = Depends(get_current_user),
     db: AsyncSession = Depends(get_db),
 ):
-    if not check_permission(current_user, ["super_admin", "admin"]):
+    if not check_permission(current_user, ADMIN_ROLES):
         raise HTTPException(
             status_code=status.HTTP_403_FORBIDDEN,
             detail="Insufficient permissions",
@@ -91,7 +94,7 @@ async def get_user(
     current_user: User = Depends(get_current_user),
     db: AsyncSession = Depends(get_db),
 ):
-    if not check_permission(current_user, ["super_admin", "admin"]) and current_user.id != user_id:
+    if not check_permission(current_user, ADMIN_ROLES) and current_user.id != user_id:
         raise HTTPException(
             status_code=status.HTTP_403_FORBIDDEN,
             detail="Insufficient permissions",
@@ -128,7 +131,7 @@ async def create_user(
     current_user: User = Depends(get_current_user),
     db: AsyncSession = Depends(get_db),
 ):
-    if not check_permission(current_user, ["super_admin"]):
+    if not check_permission(current_user, SUPER_ADMIN_ROLES):
         raise HTTPException(
             status_code=status.HTTP_403_FORBIDDEN,
             detail="Only super_admin can create users",
@@ -196,18 +199,18 @@ async def update_user(
     current_user: User = Depends(get_current_user),
     db: AsyncSession = Depends(get_db),
 ):
-    if not check_permission(current_user, ["super_admin", "admin"]) and current_user.id != user_id:
+    if not check_permission(current_user, ADMIN_ROLES) and current_user.id != user_id:
         raise HTTPException(
             status_code=status.HTTP_403_FORBIDDEN,
             detail="Insufficient permissions",
         )
 
-    if not check_permission(current_user, ["super_admin"]) and user_data.role is not None:
+    if not check_permission(current_user, SUPER_ADMIN_ROLES) and user_data.role is not None:
         raise HTTPException(
             status_code=status.HTTP_403_FORBIDDEN,
             detail="Only super_admin can change user role",
         )
-    if not check_permission(current_user, ["super_admin"]) and user_data.gatekeeper_groups is not None:
+    if not check_permission(current_user, SUPER_ADMIN_ROLES) and user_data.gatekeeper_groups is not None:
         raise HTTPException(
             status_code=status.HTTP_403_FORBIDDEN,
             detail="Only super_admin can change Gatekeeper groups",
@@ -260,7 +263,7 @@ async def delete_user(
     current_user: User = Depends(get_current_user),
     db: AsyncSession = Depends(get_db),
 ):
-    if not check_permission(current_user, ["super_admin"]):
+    if not check_permission(current_user, SUPER_ADMIN_ROLES):
         raise HTTPException(
             status_code=status.HTTP_403_FORBIDDEN,
             detail="Only super_admin can delete users",

backend/app/api/v1/vessels.py

@@ -1,4 +1,4 @@
-"""Bounded vessel snapshot APIs for viewport-first consumers."""
+"""Bounded vessel snapshot APIs backed by the latest vessel state table."""
 
 from typing import Optional
 
@@ -14,8 +14,8 @@ router = APIRouter()
 
 @router.get("/snapshot")
 async def get_vessel_snapshot(
-    bbox: Optional[str] = Query(None, description="Viewport bbox as lon_min,lat_min,lon_max,lat_max"),
-    zoom: int = Query(..., ge=1, le=20, description="Current map zoom level"),
+    bbox: Optional[str] = Query(None, description="Snapshot bbox as lon_min,lat_min,lon_max,lat_max"),
+    zoom: float = Query(..., ge=1, le=20, description="Current map zoom level"),
     type: Optional[str] = Query(
         None,
         description="Comma-separated vessel types: cargo,tanker,passenger,fishing,military,other",

backend/app/api/v1/visualization.py

@@ -17,15 +17,18 @@ from sqlalchemy.ext.asyncio import AsyncSession
 from sqlalchemy import select, func
 from typing import List, Dict, Any, Optional
 
+from app.core.compute_types import COMPUTE_SITE_TYPES, COMPUTE_SOURCES, compute_site_type, compute_record_filter, compute_site_type_expression
 from app.core.collected_data_fields import get_record_field
+from app.core.enums import BGPStatus
 from app.core.satellite_tle import build_tle_lines_from_elements
 from app.core.time import to_iso8601_utc
 from app.db.session import get_db
+from app.core.satellite_presentation import satellite_constellation_group as _normalize_satellite_constellation_group
 from app.models.bgp_anomaly import BGPAnomaly
 from app.models.bgp_incident import BGPIncident
 from app.models.bgp_observation import BGPObservation
 from app.models.collected_data import CollectedData
-from app.models.vessel import AISSourceHealth, VesselPosition, VesselStatic
+from app.models.vessel import AISSourceHealth, VesselCurrentState, VesselPosition, VesselStatic
 from app.services.bgp_collectors import build_bgp_collector_coverage
 from app.services.cable_graph import build_graph_from_data, CableGraph, haversine_distance
 from app.services.compute_center_locations import (
@@ -38,7 +41,7 @@ from app.services.compute_center_locations import (
     upsert_compute_center_location,
 )
 from app.services.ai_client import get_ai_provider_client
-from app.api.v1.settings import get_web_search_client
+from app.api.v1.settings import get_runtime_web_search_config, get_web_search_client
 from app.services.collectors.bgp_common import RIPE_RIS_COLLECTOR_COORDS
 from app.services.location.llm_fallback import (
     collect_llm_location_fallback_candidate,
@@ -47,11 +50,10 @@ from app.services.location.llm_fallback import (
 from app.services.persistent_logs import record_system_log
 from app.services.vessel_ais_aggregation import (
     build_field_conflict_candidates,
-    count_unique_raw_vessel_mmsi,
     get_aggregated_vessel,
     get_aggregated_vessel_track,
     get_aggregated_vessels,
-    get_aggregated_vessels_snapshot,
+    get_current_vessels_snapshot,
     get_vessel_conflict_records,
     get_vessel_raw_observations,
     MAX_SNAPSHOT_LIMIT,
@@ -75,7 +77,6 @@ TERRAIN_TILE_BATCH_MAX_ITEMS = 128
 TERRAIN_TILE_BATCH_CONCURRENCY = 16
 _terrain_tile_cache: OrderedDict[tuple[int, int, int], tuple[bytes, str, dict[str, str]]] = OrderedDict()
 VESSEL_NAME_FALLBACK_PATTERN = re.compile(r"^mmsi\s*\d+$", re.IGNORECASE)
-VESSEL_SNAPSHOT_LEGACY_FALLBACK_ENABLED = True
 SECONDS_PER_MINUTE = 60
 BYTES_PER_MIB = 1024 * 1024
 CABLE_CACHE_FRESH_SECONDS = 6 * 60 * SECONDS_PER_MINUTE
@@ -105,8 +106,8 @@ LANDING_POINT_CACHE_POLICY = EarthLayerCachePolicy(
 SATELLITE_CACHE_POLICY = EarthLayerCachePolicy(
     SATELLITE_CACHE_FRESH_SECONDS,
     SATELLITE_CACHE_STALE_SECONDS,
-    max_features=8000,
-    max_bytes=10 * BYTES_PER_MIB,
+    max_features=25000,
+    max_bytes=32 * BYTES_PER_MIB,
 )
 COMPUTE_CENTER_CACHE_POLICY = EarthLayerCachePolicy(
     COMPUTE_CENTER_CACHE_FRESH_SECONDS,
@@ -338,22 +339,6 @@ def convert_satellite_to_geojson(records: List[CollectedData]) -> Dict[str, Any]
     return {"type": "FeatureCollection", "features": features}
 
 
-def _normalize_satellite_constellation_group(
-    raw_group: Any,
-    name: Optional[str],
-) -> Optional[str]:
-    normalized_group = str(raw_group or "").strip().lower()
-    if normalized_group:
-        return normalized_group
-
-    normalized_name = str(name or "").strip().upper()
-    if normalized_name.startswith("STARLINK"):
-        return "starlink"
-    if normalized_name.startswith("IRIDIUM"):
-        return "iridium-next"
-
-    return None
-
 
 def _get_satellite_footprint_policy(constellation_group: Optional[str]) -> str:
     if constellation_group == "starlink":
@@ -506,16 +491,19 @@ async def _count_current_or_latest_task_data(
 async def _load_current_collected_data_by_sources(
     db: AsyncSession,
     sources: List[str],
+    *, include_compute_types: bool = False,
 ) -> Dict[str, List[CollectedData]]:
     if not sources:
         return {}
 
     stmt = (
         select(CollectedData)
-        .where(CollectedData.source.in_(sources))
+        .where(compute_record_filter(CollectedData) if include_compute_types else CollectedData.source.in_(sources))
         .where(CollectedData.is_current.is_(True))
         .order_by(CollectedData.source.asc(), CollectedData.id.desc())
     )
+    if include_compute_types:
+        stmt = stmt.where(CollectedData.deleted_at.is_(None))
     result = await db.execute(stmt)
 
     grouped_records: Dict[str, List[CollectedData]] = {source: [] for source in sources}
@@ -645,7 +633,9 @@ def _normalize_capacity_band(capacity_value: Optional[float], capacity_unit: str
         return "unknown"
 
     unit = str(capacity_unit or "").strip().lower()
-    if unit in {"pflop/s", "pflops", "pflop"}:
+    if unit in {"flop/s", "flops"}:
+        normalized_tflops = capacity_value / 1e12
+    elif unit in {"pflop/s", "pflops", "pflop"}:
         normalized_tflops = capacity_value * 1000
     elif unit in {"gflop/s", "gflops", "gflop"}:
         normalized_tflops = capacity_value
@@ -673,15 +663,15 @@ def convert_compute_centers_to_geojson(records: List[CollectedData]) -> Dict[str
     """
     features: List[Dict[str, Any]] = []
     unresolved: List[Dict[str, Any]] = []
+    withheld_count = 0
 
     for record in records:
         metadata = record.extra_data or {}
+        if metadata.get("compute_contract") == "v1" and metadata.get("anonymous"):
+            withheld_count += 1
+            continue
         result = resolve_compute_center_location_full(record, metadata)
-        site_type = (
-            "supercomputer"
-            if record.source == "top500" or record.data_type == "supercomputer"
-            else "gpu_cluster"
-        )
+        site_type = compute_site_type(record)
 
         if not result.is_resolved:
             diagnostic = result.diagnostic or ResolutionDiagnostic(
@@ -707,7 +697,10 @@ def convert_compute_centers_to_geojson(records: List[CollectedData]) -> Dict[str
         latitude = location.latitude
         longitude = location.longitude
 
-        if site_type == "supercomputer":
+        if metadata.get("compute_contract") == "v1":
+            capacity_value = _parse_float(get_record_field(record, "value"))
+            capacity_unit = str(get_record_field(record, "unit") or "FLOP/s")
+        elif site_type == "supercomputer":
             capacity_value = _parse_float(get_record_field(record, "rmax"))
             capacity_unit = "GFlops"
         else:
@@ -768,7 +761,7 @@ def convert_compute_centers_to_geojson(records: List[CollectedData]) -> Dict[str
             }
         )
 
-    return {"type": "FeatureCollection", "features": features, "unresolved": unresolved}
+    return {"type": "FeatureCollection", "features": features, "unresolved": unresolved, "withheld_count": withheld_count}
 
 
 VESSEL_TYPE_FILTERS = {
@@ -839,9 +832,14 @@ def convert_aggregated_vessels_to_geojson(vessels: List[dict[str, Any]]) -> Dict
             continue
         source_summary = {}
         for source, summary in (vessel.get("source_summary") or {}).items():
+            latest_observed_at = summary.get("latest_observed_at")
             source_summary[source] = {
                 **summary,
-                "latest_observed_at": to_iso8601_utc(summary.get("latest_observed_at")),
+                "latest_observed_at": (
+                    to_iso8601_utc(latest_observed_at)
+                    if isinstance(latest_observed_at, datetime)
+                    else latest_observed_at
+                ),
             }
         props = {
             "mmsi": vessel["mmsi"],
@@ -1075,7 +1073,7 @@ async def build_vessel_snapshot_response(
     db: AsyncSession,
     *,
     bbox: tuple[float, float, float, float] | None,
-    zoom: int | None,
+    zoom: float | None,
     type_filter: str | None,
     limit: int | None,
     since_minutes: int = 60,
@@ -1117,6 +1115,7 @@ async def build_vessel_snapshot_response(
     safe_limit = _safe_vessel_limit(limit)
     safe_since_minutes = min(max(int(since_minutes or 60), 1), 1440)
     observed_since = datetime.now(UTC) - timedelta(minutes=safe_since_minutes)
+    snapshot_started_at = datetime.now(UTC)
     features, diagnostics = await _load_raw_vessel_snapshot_features(
         db,
         bbox=bbox,
@@ -1133,6 +1132,7 @@ async def build_vessel_snapshot_response(
         "features": features,
         "count": len(features),
         "stats": _build_vessel_stats(features),
+        "generated_at": to_iso8601_utc(snapshot_started_at),
         "diagnostics": {
             **diagnostics,
             "filtered_count": len(features),
@@ -1563,15 +1563,7 @@ async def _build_cables_geojson(db: AsyncSession) -> dict[str, Any]:
     try:
         records = await _load_current_collected_data(db, "arcgis_cables")
 
-        if not records:
-            raise HTTPException(
-                status_code=404,
-                detail="No cable data found. Please run the arcgis_cables collector first.",
-            )
-
         return convert_cable_to_geojson(records)
-    except HTTPException:
-        raise
     except Exception as e:
         logger.exception_event(
             "Failed to build cables GeoJSON response",
@@ -1625,16 +1617,8 @@ async def _build_landing_points_geojson(db: AsyncSession) -> dict[str, Any]:
             relation_records,
             cable_records,
         )
-        
-        if not records:
-            raise HTTPException(
-                status_code=404,
-                detail="No landing point data found. Please run the arcgis_landing_points collector first.",
-            )
-        
+
         return convert_landing_point_to_geojson(records, city_to_cable_ids_map, cable_id_to_name_map)
-    except HTTPException:
-        raise
     except Exception as e:
         logger.exception_event(
             "Failed to build landing points GeoJSON response",
@@ -1965,9 +1949,10 @@ async def _build_compute_centers_geojson(
     records_by_source = await _load_current_collected_data_by_sources(
         db,
         ["top500", "epoch_ai_gpu"],
+        include_compute_types=True,
     )
     records = _filter_known_records(
-        records_by_source.get("top500", []) + records_by_source.get("epoch_ai_gpu", []),
+        [record for group in records_by_source.values() for record in group],
     )
     if limit is not None:
         records = records[:limit]
@@ -1982,6 +1967,7 @@ async def _build_compute_centers_geojson(
                 "total": 0,
                 "supercomputers": 0,
                 "gpu_clusters": 0,
+                "by_type": dict.fromkeys(COMPUTE_SITE_TYPES, 0),
                 "unresolved": 0,
             },
         }
@@ -2024,6 +2010,7 @@ async def _build_compute_centers_geojson(
         "count": len(sanitized_features),
         "stats": {
             "total": len(sanitized_features),
+            "by_type": {kind: sum(feature["properties"]["site_type"] == kind for feature in sanitized_features) for kind in COMPUTE_SITE_TYPES},
             "supercomputers": sum(
                 1 for feature in sanitized_features
                 if feature.get("properties", {}).get("site_type") == "supercomputer"
@@ -2071,6 +2058,44 @@ class SaveComputeCenterLocationRequest(BaseModel):
     model_config = {"populate_by_name": True}
 
 
+async def _compute_center_location_web_search_capability(db: AsyncSession) -> Dict[str, Any]:
+    try:
+        config = await get_runtime_web_search_config(db)
+    except Exception as exc:
+        return {
+            "enabled": False,
+            "provider": None,
+            "reason": f"WebSearch 配置读取失败：{exc}",
+        }
+    provider_config = config.active_provider_config
+    has_api_key = bool((provider_config.api_key or "").strip())
+    if not config.enabled:
+        return {
+            "enabled": False,
+            "provider": config.default_provider,
+            "reason": "WebSearch 未开启，无法进行事实核查定位。",
+        }
+    if not has_api_key:
+        return {
+            "enabled": False,
+            "provider": config.default_provider,
+            "reason": f"WebSearch Provider {config.default_provider} 未配置 API Key。",
+        }
+    return {
+        "enabled": True,
+        "provider": config.default_provider,
+        "reason": "",
+    }
+
+
+@router.get("/compute-centers/location-capability")
+async def get_compute_center_location_capability(
+    db: AsyncSession = Depends(get_db),
+):
+    """Return whether fact-checked compute-center location collection can run."""
+    return await _compute_center_location_web_search_capability(db)
+
+
 @router.post("/compute-centers/{source_id}/collect-location")
 async def collect_compute_center_location(
     source_id: str,
@@ -2089,6 +2114,9 @@ async def collect_compute_center_location(
     """
     if not source_id or not source_id.strip():
         raise HTTPException(status_code=400, detail="source_id is required")
+    capability = await _compute_center_location_web_search_capability(db)
+    if not capability.get("enabled"):
+        raise HTTPException(status_code=409, detail=capability)
 
     record = await _load_compute_center_record(db, source_id)
     name = payload.name or (record.name if record else None)
@@ -2299,7 +2327,8 @@ async def _load_compute_center_record(db: AsyncSession, source_id: str) -> Colle
     stmt = (
         select(CollectedData)
         .where(CollectedData.source_id == source_id)
-        .where(CollectedData.source.in_(["top500", "epoch_ai_gpu"]))
+        .where(compute_record_filter(CollectedData))
+        .where(CollectedData.deleted_at.is_(None))
         .order_by(CollectedData.is_current.desc(), CollectedData.id.desc())
         .limit(1)
     )
@@ -2315,58 +2344,35 @@ async def _load_raw_vessel_snapshot_features(
     observed_since: datetime,
 ) -> tuple[list[dict[str, Any]], dict[str, Any]]:
     if bbox is None:
-        aggregated_vessels = await get_aggregated_vessels(
-            db,
-            limit=limit,
-            observed_since=observed_since,
-        )
-    else:
-        aggregated_vessels = await get_aggregated_vessels_snapshot(
-            db,
-            bbox=bbox,
-            limit=limit,
-            observed_since=observed_since,
-        )
-    raw_geojson = convert_aggregated_vessels_to_geojson(aggregated_vessels)
-    raw_features = raw_geojson.get("features", [])
-    features = raw_features
-    legacy_features: list[dict[str, Any]] = []
-    legacy_fallback_used = False
-    if not raw_features and VESSEL_SNAPSHOT_LEGACY_FALLBACK_ENABLED:
-        legacy_features = await _load_legacy_vessel_snapshot_features(
-            db,
-            bbox=bbox,
-            limit=limit,
-        )
-        features, _merge_diagnostics = _merge_vessel_features(raw_features, legacy_features)
-        legacy_fallback_used = bool(legacy_features)
-
+        return [], {
+            "source": "vessel_current_state",
+            "current_state_count": 0,
+            "final_unique_mmsi": 0,
+        }
+    current_vessels = await get_current_vessels_snapshot(
+        db,
+        bbox=bbox,
+        limit=limit,
+        observed_since=observed_since,
+    )
+    features = convert_aggregated_vessels_to_geojson(current_vessels).get("features", [])
+    unique_mmsi = len(
+        {
+            key
+            for key in (_feature_mmsi_key(feature) for feature in features)
+            if key is not None
+        }
+    )
     return features, {
-        "raw_feature_count": len(raw_features),
-        "raw_unique_mmsi": len(
-            {
-                key
-                for key in (_feature_mmsi_key(feature) for feature in raw_features)
-                if key is not None
-            }
-        ),
-        "legacy_feature_count": len(legacy_features),
-        "legacy_backfilled_mmsi": len(
-            {
-                key
-                for key in (_feature_mmsi_key(feature) for feature in legacy_features)
-                if key is not None
-            }
-        ),
-        "legacy_fallback_enabled": VESSEL_SNAPSHOT_LEGACY_FALLBACK_ENABLED,
-        "legacy_fallback_used": legacy_fallback_used,
-        "final_unique_mmsi": len(
-            {
-                key
-                for key in (_feature_mmsi_key(feature) for feature in features)
-                if key is not None
-            }
-        ),
+        "source": "vessel_current_state",
+        "current_state_count": len(features),
+        "final_unique_mmsi": unique_mmsi,
+        "raw_feature_count": 0,
+        "raw_unique_mmsi": 0,
+        "legacy_feature_count": 0,
+        "legacy_backfilled_mmsi": 0,
+        "legacy_fallback_enabled": False,
+        "legacy_fallback_used": False,
     }
 
 @router.get("/vessels/custom-supplements")
@@ -2696,6 +2702,8 @@ async def _build_bgp_collectors_geojson(db: AsyncSession) -> dict[str, Any]:
         db,
         source_filter=("ris_live_bgp", "bgpstream_bgp"),
     )
+    if not any(int(item.get("observation_count") or 0) > 0 for item in coverage):
+        return {"type": "FeatureCollection", "features": [], "count": 0}
     coverage_by_collector = {
         item["collector"]: item
         for item in coverage
@@ -2719,6 +2727,25 @@ async def get_visualization_geo_summary(db: AsyncSession = Depends(get_db), resp
     )
 
 
+async def _compute_facility_counts(db: AsyncSession) -> dict[str, int]:
+    kind = compute_site_type_expression(CollectedData)
+    rows = (await db.execute(select(CollectedData.source, kind, func.count(CollectedData.id))
+        .where(compute_record_filter(CollectedData), CollectedData.is_current.is_(True),
+               CollectedData.deleted_at.is_(None))
+        .group_by(CollectedData.source, kind))).all()
+    counts = dict.fromkeys(COMPUTE_SITE_TYPES, 0)
+    present_sources = set()
+    for source, site_type, count in rows:
+        present_sources.add(source)
+        counts[site_type] += int(count or 0)
+    # Retain the existing last-task fallback for the two collected catalogs.
+    for source in COMPUTE_SOURCES:
+        if source not in present_sources:
+            key = "supercomputer" if source == "top500" else "gpu_cluster"
+            counts[key] += await _count_current_or_latest_task_data(db, source)
+    return counts
+
+
 async def _build_visualization_geo_summary(db: AsyncSession) -> dict[str, Any]:
     cable_count = await _count_current_or_latest_task_data(db, "arcgis_cables")
     landing_point_count = await _count_current_or_latest_task_data(db, "arcgis_landing_points")
@@ -2727,15 +2754,16 @@ async def _build_visualization_geo_summary(db: AsyncSession) -> dict[str, Any]:
         "celestrak_tle",
         exclude_unknown_name=True,
     )
-    supercomputer_count = await _count_current_or_latest_task_data(db, "top500")
-    gpu_cluster_count = await _count_current_or_latest_task_data(db, "epoch_ai_gpu")
-    compute_center_count = supercomputer_count + gpu_cluster_count
+    compute_counts = await _compute_facility_counts(db)
+    supercomputer_count = compute_counts["supercomputer"]
+    gpu_cluster_count = compute_counts["gpu_cluster"]
+    compute_center_count = sum(compute_counts.values())
 
     active_incident_result = await db.execute(
-        select(func.count(BGPIncident.id)).where(BGPIncident.status == "active"),
+        select(func.count(BGPIncident.id)).where(BGPIncident.status == BGPStatus.ACTIVE.value),
     )
     active_anomaly_result = await db.execute(
-        select(func.count(BGPAnomaly.id)).where(BGPAnomaly.status == "active"),
+        select(func.count(BGPAnomaly.id)).where(BGPAnomaly.status == BGPStatus.ACTIVE.value),
     )
     active_incident_count = int(active_incident_result.scalar() or 0)
     active_anomaly_count = int(active_anomaly_result.scalar() or 0)
@@ -2756,21 +2784,14 @@ async def _build_visualization_geo_summary(db: AsyncSession) -> dict[str, Any]:
         )
     else:
         bgp_collector_count = int(bgp_collector_scalar or 0)
-    raw_unique_window_hours = 24
-    raw_unique_mmsi = await count_unique_raw_vessel_mmsi(
-        db,
-        observed_since=datetime.now(UTC) - timedelta(hours=raw_unique_window_hours),
+    vessel_current_window_minutes = 60
+    vessel_current_result = await db.execute(
+        select(func.count(VesselCurrentState.mmsi)).where(
+            VesselCurrentState.observed_at
+            >= datetime.now(UTC) - timedelta(minutes=vessel_current_window_minutes)
+        )
     )
-    legacy_unique_result = await db.execute(
-        select(func.count(func.distinct(VesselPosition.mmsi)))
-    )
-    legacy_unique_mmsi = int(legacy_unique_result.scalar() or 0)
-    legacy_fallback_active = (
-        VESSEL_SNAPSHOT_LEGACY_FALLBACK_ENABLED
-        and raw_unique_mmsi == 0
-        and legacy_unique_mmsi > 0
-    )
-    vessel_count = legacy_unique_mmsi if legacy_fallback_active else raw_unique_mmsi
+    vessel_count = int(vessel_current_result.scalar() or 0)
     aisstream_health = await db.get(AISSourceHealth, "aisstream_vessels")
 
     return {
@@ -2781,17 +2802,19 @@ async def _build_visualization_geo_summary(db: AsyncSession) -> dict[str, Any]:
             "satellite_count": satellite_count,
             "compute_center_count": compute_center_count,
             "vessel_count": vessel_count,
-            "vessel_count_source": "legacy_fallback" if legacy_fallback_active else "raw_recent",
-            "vessel_legacy_fallback_enabled": VESSEL_SNAPSHOT_LEGACY_FALLBACK_ENABLED,
-            "vessel_raw_unique_mmsi": raw_unique_mmsi,
-            "vessel_raw_unique_window_hours": raw_unique_window_hours,
-            "vessel_legacy_unique_mmsi": legacy_unique_mmsi,
+            "vessel_count_source": "vessel_current_state",
+            "vessel_current_window_minutes": vessel_current_window_minutes,
+            "vessel_raw_unique_mmsi": 0,
+            "vessel_legacy_unique_mmsi": 0,
             "aisstream_connection_state": aisstream_health.connection_state if aisstream_health else None,
             "aisstream_last_seen_at": to_iso8601_utc(aisstream_health.last_seen_at) if aisstream_health else None,
             "aisstream_message_rate": aisstream_health.message_rate if aisstream_health else None,
             "aisstream_lag_seconds": aisstream_health.lag_seconds if aisstream_health else None,
             "supercomputer_count": supercomputer_count,
             "gpu_cluster_count": gpu_cluster_count,
+            "data_center_count": compute_counts["data_center"],
+            "cloud_center_count": compute_counts["cloud_center"],
+            "edge_node_count": compute_counts["edge_node"],
             "bgp_event_count": active_incident_count or active_anomaly_count,
             "bgp_incident_count": active_incident_count,
             "bgp_anomaly_count": active_anomaly_count,

backend/app/api/v1/websocket.py

@@ -6,11 +6,15 @@ from typing import Optional
 
 from fastapi import APIRouter, WebSocket, WebSocketDisconnect, Query
 from jose import jwt, JWTError
+from sqlalchemy import text
 
 from app.core.config import settings
+from app.core.enums import UserRole
 from app.core.logging import get_logger
 from app.core.time import to_iso8601_utc
 from app.core.websocket.manager import manager
+from app.db.session import async_session_factory
+from app.services.log_tail import LOG_TAIL_CHANNEL, log_tail_manager
 
 logger = get_logger(__name__, service="api")
 router = APIRouter()
@@ -37,6 +41,28 @@ async def authenticate_token(token: str) -> Optional[dict]:
         return None
 
 
+async def load_websocket_user_role(user_id: str | None) -> str | None:
+    if not user_id:
+        return None
+    try:
+        async with async_session_factory() as db:
+            result = await db.execute(
+                text("SELECT role, is_active FROM users WHERE id = :id"),
+                {"id": int(user_id)},
+            )
+            row = result.fetchone()
+    except Exception as exc:
+        logger.warning_event(
+            "WebSocket user role lookup failed",
+            event="auth.websocket.role_lookup_failed",
+            context={"user_id": user_id, "error": str(exc)},
+        )
+        return None
+    if row is None or not row[1]:
+        return None
+    return str(row[0] or "")
+
+
 @router.websocket("/ws")
 async def websocket_endpoint(
     websocket: WebSocket,
@@ -59,6 +85,7 @@ async def websocket_endpoint(
 
     is_anonymous = payload is None
     user_id = str(payload.get("sub")) if payload else f"anonymous:{id(websocket)}"
+    user_role = await load_websocket_user_role(user_id) if payload else None
     supported_channels = ["vessels", "earth_news", EARTH_UPDATES_CHANNEL] if is_anonymous else [
         "gpu_clusters",
         "submarine_cables",
@@ -70,6 +97,8 @@ async def websocket_endpoint(
         "earth_news",
         EARTH_UPDATES_CHANNEL,
     ]
+    if user_role == UserRole.SUPER_ADMIN.value:
+        supported_channels = [*supported_channels, LOG_TAIL_CHANNEL]
     await manager.connect(websocket, user_id)
 
     try:
@@ -100,6 +129,7 @@ async def websocket_endpoint(
                     payload_data = data.get("data", {})
                     if not isinstance(payload_data, dict):
                         payload_data = {}
+                    log_tail_config = None
                     channels = payload_data.get("channels", [])
                     if isinstance(channels, str):
                         channels = [channels]
@@ -108,10 +138,30 @@ async def websocket_endpoint(
                     channel = payload_data.get("channel")
                     if channel and channel not in channels:
                         channels = [*channels, channel]
+                    if LOG_TAIL_CHANNEL in channels:
+                        if user_role != UserRole.SUPER_ADMIN.value:
+                            await websocket.send_json(
+                                {
+                                    "type": "subscription_error",
+                                    "data": {"channel": LOG_TAIL_CHANNEL, "detail": "Only super_admin can subscribe logs"},
+                                }
+                            )
+                            channels = [item for item in channels if item != LOG_TAIL_CHANNEL]
+                        else:
+                            try:
+                                log_tail_config = await log_tail_manager.subscribe(websocket, payload_data)
+                            except ValueError as exc:
+                                await websocket.send_json(
+                                    {
+                                        "type": "subscription_error",
+                                        "data": {"channel": LOG_TAIL_CHANNEL, "detail": str(exc)},
+                                    }
+                                )
+                            channels = [item for item in channels if item != LOG_TAIL_CHANNEL]
                     if is_anonymous:
                         channels = [channel for channel in channels if channel in supported_channels]
                     vessel_subscription = None
-                    if "vessels" in channels and "bbox" in payload_data:
+                    if "vessels" in channels:
                         try:
                             vessel_subscription = manager.subscribe_vessels(websocket, payload_data)
                         except ValueError as exc:
@@ -131,14 +181,20 @@ async def websocket_endpoint(
                                 "action": "subscribe",
                                 "channels": [
                                     *channels,
+                                    *([LOG_TAIL_CHANNEL] if log_tail_config else []),
                                     *(["vessels"] if vessel_subscription else []),
                                 ],
                                 "vessels": vessel_subscription,
+                                "logs_tail": log_tail_config.__dict__ if log_tail_config else None,
                             },
                         }
                     )
                 elif data.get("type") == "unsubscribe":
                     channels = data.get("data", {}).get("channels", [])
+                    if isinstance(channels, str):
+                        channels = [channels]
+                    if LOG_TAIL_CHANNEL in channels:
+                        await log_tail_manager.unsubscribe(websocket)
                     manager.unsubscribe(websocket, channels)
                     await websocket.send_json(
                         {
@@ -159,4 +215,5 @@ async def websocket_endpoint(
     except WebSocketDisconnect:
         pass
     finally:
+        await log_tail_manager.disconnect(websocket)
         manager.disconnect(websocket, user_id)

backend/app/core/compute_types.py

@@ -0,0 +1,35 @@
+"""Compute facility classification shared by the map and image resolver."""
+
+from sqlalchemy import case, or_
+
+COMPUTE_SITE_TYPES = ("supercomputer", "gpu_cluster", "data_center", "cloud_center", "edge_node")
+COMPUTE_SOURCES = ("top500", "epoch_ai_gpu")
+
+
+def compute_site_type(record) -> str:
+    metadata = record.extra_data or {}
+    for value in (metadata.get("site_type"), record.data_type):
+        if value in COMPUTE_SITE_TYPES:
+            return value
+    return {"top500": "supercomputer", "epoch_ai_gpu": "gpu_cluster"}.get(
+        record.source, "data_center"
+    )
+
+
+def compute_site_type_expression(record):
+    declared = record.extra_data["site_type"].as_string()
+    return case(
+        (declared.in_(COMPUTE_SITE_TYPES), declared),
+        (record.data_type.in_(COMPUTE_SITE_TYPES), record.data_type),
+        (record.source == "top500", "supercomputer"),
+        (record.source == "epoch_ai_gpu", "gpu_cluster"),
+        else_="data_center",
+    )
+
+
+def compute_record_filter(record):
+    return or_(
+        record.source.in_(COMPUTE_SOURCES),
+        record.data_type.in_(COMPUTE_SITE_TYPES),
+        record.extra_data["site_type"].as_string().in_(COMPUTE_SITE_TYPES),
+    )

backend/app/core/config.py

@@ -41,6 +41,7 @@ class Settings(BaseSettings):
     AI_PROVIDER_SERVICE_TOKEN: str = ""
     AI_PROVIDER_TIMEOUT_SECONDS: int = 60
     AI_PROVIDER_RETRY_ATTEMPTS: int = 2
+    OBSERVABILITY_INGEST_TOKEN: str = ""
 
     @property
     def REDIS_URL(self) -> str:

backend/app/core/earth_geography_policy.py

@@ -0,0 +1,57 @@
+"""Country display ownership for Earth comparisons and cards.
+
+Keep source-country evidence separate; use this projection only for display
+grouping so regional source records cannot become competing national keys.
+"""
+
+from __future__ import annotations
+
+import json
+from functools import lru_cache
+from pathlib import Path
+
+from app.core.countries import COUNTRY_ENTRIES, normalize_country
+
+
+_DISPLAY_ISO_A3 = {
+    canonical: next(
+        (alias for alias in aliases if len(alias) == 3 and alias.isascii() and alias.isupper()),
+        None,
+    )
+    for canonical, aliases in COUNTRY_ENTRIES
+}
+_COUNTRY_BY_ISO_A3 = {code: country for country, code in _DISPLAY_ISO_A3.items() if code}
+_POLICY_PATH = Path(__file__).resolve().parents[3] / "config/earth-boundary-pov-policy.china-v1.json"
+
+
+@lru_cache(maxsize=1)
+def _policy_country_by_code() -> dict[str, str]:
+    try:
+        content = _POLICY_PATH.read_text(encoding="utf-8")
+    except FileNotFoundError as error:
+        raise FileNotFoundError(f"EARTH_DISPLAY_POLICY_MISSING: {_POLICY_PATH}") from error
+    policy = json.loads(content)
+    if policy.get("profile") != "china-pov-v1":
+        raise ValueError("unsupported Earth display-policy profile")
+    mappings = {}
+    for rule in policy["rules"]:
+        target = rule.get("targetIsoA3") or rule.get("parentIsoA3")
+        if target not in _COUNTRY_BY_ISO_A3:
+            continue
+        for code in rule.get("sourceIsoA3", []):
+            mappings[code] = _COUNTRY_BY_ISO_A3[target]
+    return mappings
+
+
+def display_country(value: str | None) -> str | None:
+    country = normalize_country(value)
+    code = _DISPLAY_ISO_A3.get(country) if country else str(value or "").strip().upper()
+    owner = _policy_country_by_code().get(code)
+    if owner:
+        return owner
+    return country
+
+
+def display_iso_a3(value: str | None) -> str | None:
+    country = display_country(value)
+    return _DISPLAY_ISO_A3.get(country) if country else None

backend/app/core/enums.py

@@ -0,0 +1,227 @@
+"""Stable backend protocol enums.
+
+Database columns and JSON payloads continue to store the enum string values.
+Configurable identifiers, user-authored values, and open-ended taxonomies do
+not belong in this module.
+"""
+
+from __future__ import annotations
+
+import logging
+from enum import StrEnum
+from typing import TypeVar
+
+logger = logging.getLogger(__name__)
+
+EnumT = TypeVar("EnumT", bound=StrEnum)
+
+
+def parse_enum(enum_type: type[EnumT], value: object, default: EnumT) -> EnumT:
+    """Parse an external value without breaking reads of legacy data."""
+
+    if value is None or str(value).strip() == "":
+        return default
+    if isinstance(value, enum_type):
+        return value
+    try:
+        return enum_type(str(value).strip().lower())
+    except (TypeError, ValueError):
+        logger.warning(
+            "Unknown %s value %r; falling back to %s",
+            enum_type.__name__,
+            value,
+            default.value,
+        )
+        return default
+
+
+class NewsImportanceLevel(StrEnum):
+    LOW = "low"
+    MEDIUM = "medium"
+    HIGH = "high"
+    CRITICAL = "critical"
+
+
+class BreakingLevel(StrEnum):
+    NONE = "none"
+    WATCH = "watch"
+    BREAKING = "breaking"
+    CRITICAL = "critical"
+
+
+class BreakingScope(StrEnum):
+    REGIONAL = "regional"
+    GLOBAL = "global"
+
+
+class BreakingSource(StrEnum):
+    RULES = "rules"
+    AI = "ai"
+    MANUAL = "manual"
+    MULTI_SOURCE = "multi_source"
+
+
+class NewsSourceType(StrEnum):
+    RSS = "rss"
+    ATOM = "atom"
+    AGGREGATED = "aggregated"
+    REFERENCE = "reference"
+    MANUAL = "manual"
+
+
+class NewsEnrichmentStatus(StrEnum):
+    PENDING = "pending"
+    QUEUED = "queued"
+    ATTEMPTED = "attempted"
+    SUCCESS = "success"
+    CONTENT_ONLY = "content_only"
+    LOCATION_ONLY = "location_only"
+    UNAVAILABLE = "unavailable"
+    PROVIDER_ERROR = "provider_error"
+    PARSE_ERROR = "parse_error"
+    NO_RESULT = "no_result"
+
+
+class NewsMarketImpact(StrEnum):
+    NONE = "none"
+    SECTOR = "sector"
+    NATIONAL = "national"
+    GLOBAL = "global"
+
+
+class NewsTaggingSource(StrEnum):
+    RULES = "rules"
+    AI = "ai"
+    MANUAL = "manual"
+
+
+class JobType(StrEnum):
+    COLLECT = "collect"
+    CLEAR_DATA = "clear_data"
+    CLEAR_CACHE = "clear_cache"
+    EARTH_REFRESH = "earth_refresh"
+
+
+class JobStatus(StrEnum):
+    QUEUED = "queued"
+    RUNNING = "running"
+    CANCELLING = "cancelling"
+    SUCCESS = "success"
+    FAILED = "failed"
+    CANCELLED = "cancelled"
+
+
+class RollbackPolicy(StrEnum):
+    KEEP_COMMITTED_BATCHES = "keep_committed_batches"
+
+
+class MappingValidationStatus(StrEnum):
+    DRAFT = "draft"
+    VALID = "valid"
+    INVALID = "invalid"
+
+
+class SnapshotStatus(StrEnum):
+    RUNNING = "running"
+    SUCCESS = "success"
+    FAILED = "failed"
+    CANCELLED = "cancelled"
+
+
+class DatasourceRunStatus(StrEnum):
+    RUNNING = "running"
+    NOT_RUN = "not_run"
+    COLLECTED = "collected"
+    UNCOLLECTED = "uncollected"
+
+
+class ProviderApi(StrEnum):
+    ANTHROPIC_MESSAGES = "anthropic-messages"
+    OPENAI_COMPLETIONS = "openai-completions"
+    OPENAI_RESPONSES = "openai-responses"
+    OLLAMA_GENERATE = "ollama-generate"
+
+
+class PlaygroundMessageRole(StrEnum):
+    SYSTEM = "system"
+    USER = "user"
+    ASSISTANT = "assistant"
+    TOOL = "tool"
+
+
+class PlaygroundMessageKind(StrEnum):
+    MESSAGE = "message"
+    THINKING = "thinking"
+    ERROR = "error"
+    STATUS = "status"
+
+
+class PlaygroundMessageStatus(StrEnum):
+    PENDING = "pending"
+    THINKING = "thinking"
+    ANSWERING = "answering"
+    DONE = "done"
+    FAILED = "failed"
+    CANCELLED = "cancelled"
+    ERROR = "error"
+    STOPPED = "stopped"
+
+
+class OtpPurpose(StrEnum):
+    REGISTER = "register"
+    VERIFY_EMAIL = "verify_email"
+    RESET_PASSWORD = "reset_password"
+
+
+class UserRole(StrEnum):
+    VIEWER = "viewer"
+    ADMIN = "admin"
+    SUPER_ADMIN = "super_admin"
+
+
+class AlertSeverity(StrEnum):
+    CRITICAL = "critical"
+    WARNING = "warning"
+    INFO = "info"
+
+
+class AlertStatus(StrEnum):
+    ACTIVE = "active"
+    ACKNOWLEDGED = "acknowledged"
+    RESOLVED = "resolved"
+
+
+class BGPStatus(StrEnum):
+    ACTIVE = "active"
+    ACKNOWLEDGED = "acknowledged"
+    RESOLVED = "resolved"
+
+
+class LogLevel(StrEnum):
+    ALL = "all"
+    ERROR = "error"
+    WARNING = "warning"
+    INFO = "info"
+    DEBUG = "debug"
+
+
+class ConnectionState(StrEnum):
+    DISCONNECTED = "disconnected"
+    CONNECTING = "connecting"
+    CONNECTED = "connected"
+    ERROR = "error"
+
+
+class AuthType(StrEnum):
+    NONE = "none"
+    BEARER = "bearer"
+    API_KEY = "api_key"
+    BASIC = "basic"
+
+
+class TVSourceType(StrEnum):
+    IFRAME = "iframe"
+    HLS = "hls"
+    VIDEO = "video"
+    EXTERNAL = "external"
+    YOUTUBE = "youtube"

backend/app/core/satellite_presentation.py

@@ -0,0 +1,13 @@
+"""Shared source-group normalization; never infer hardware generations."""
+
+
+def satellite_constellation_group(raw_group: object, name: str | None) -> str | None:
+    group = str(raw_group or "").strip().lower()
+    if group:
+        return group
+    normalized_name = str(name or "").strip().upper()
+    if normalized_name.startswith("STARLINK"):
+        return "starlink"
+    if normalized_name.startswith("IRIDIUM"):
+        return "iridium-next"
+    return None

backend/app/core/security.py

@@ -6,7 +6,7 @@ import redis
 from fastapi import Depends, HTTPException, status
 from fastapi.security import HTTPAuthorizationCredentials, HTTPBearer
 from jose import JWTError, jwt
-from sqlalchemy import text
+from sqlalchemy import select
 from sqlalchemy.ext.asyncio import AsyncSession
 
 from app.core.config import settings
@@ -81,6 +81,21 @@ def decode_token(token: str) -> Optional[dict]:
         return None
 
 
+async def _load_active_user(user_id: str, db: AsyncSession) -> User:
+    try:
+        parsed_id = int(user_id)
+    except (TypeError, ValueError):
+        raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail="Invalid token")
+    result = await db.execute(select(User).where(User.id == parsed_id))
+    user = result.scalar_one_or_none()
+    if user is None or not user.is_active:
+        raise HTTPException(
+            status_code=status.HTTP_401_UNAUTHORIZED,
+            detail="User not found or inactive",
+        )
+    return user
+
+
 async def get_current_user(
     credentials: HTTPAuthorizationCredentials = Depends(oauth2_scheme),
     db: AsyncSession = Depends(get_db),
@@ -103,27 +118,7 @@ async def get_current_user(
             status_code=status.HTTP_401_UNAUTHORIZED,
             detail="Invalid token",
         )
-    result = await db.execute(
-        text(
-            "SELECT id, username, email, password_hash, role, is_active, gatekeeper_groups FROM users WHERE id = :id"
-        ),
-        {"id": int(user_id)},
-    )
-    row = result.fetchone()
-    if row is None or not row[5]:
-        raise HTTPException(
-            status_code=status.HTTP_401_UNAUTHORIZED,
-            detail="User not found or inactive",
-        )
-    user = User()
-    user.id = row[0]
-    user.username = row[1]
-    user.email = row[2]
-    user.password_hash = row[3]
-    user.role = row[4]
-    user.is_active = row[5]
-    user.gatekeeper_groups = row[6] or []
-    return user
+    return await _load_active_user(user_id, db)
 
 
 async def get_current_user_refresh(
@@ -143,27 +138,7 @@ async def get_current_user_refresh(
             status_code=status.HTTP_401_UNAUTHORIZED,
             detail="Invalid token",
         )
-    result = await db.execute(
-        text(
-            "SELECT id, username, email, password_hash, role, is_active, gatekeeper_groups FROM users WHERE id = :id"
-        ),
-        {"id": int(user_id)},
-    )
-    row = result.fetchone()
-    if row is None or not row[5]:
-        raise HTTPException(
-            status_code=status.HTTP_401_UNAUTHORIZED,
-            detail="User not found or inactive",
-        )
-    user = User()
-    user.id = row[0]
-    user.username = row[1]
-    user.email = row[2]
-    user.password_hash = row[3]
-    user.role = row[4]
-    user.is_active = row[5]
-    user.gatekeeper_groups = row[6] or []
-    return user
+    return await _load_active_user(user_id, db)
 
 
 def blacklist_token(token: str) -> None:

backend/app/core/websocket/broadcaster.py

@@ -4,11 +4,14 @@ import asyncio
 from datetime import UTC, datetime
 from typing import Dict, Any
 
+from app.core.logging import get_logger
 from app.core.time import to_iso8601_utc
 from app.core.websocket.manager import manager
 
 
 EARTH_UPDATES_CHANNEL = "earth_updates"
+VESSEL_STATE_QUERY_BATCH_SIZE = 1000
+logger = get_logger(__name__, service="websocket")
 
 
 class DataBroadcaster:
@@ -114,16 +117,22 @@ class DataBroadcaster:
             return
         pending = self._pending_vessel_updates
         self._pending_vessel_updates = {}
-        vessels = []
-        for item in pending.values():
-            vessel = dict(item)
-            source = vessel.pop("_source", None)
-            action = vessel.pop("_action", "upsert")
-            created = vessel.pop("_created", None)
-            vessel["source"] = source
-            vessel["action"] = action
-            vessel["created"] = created
-            vessels.append(vessel)
+        try:
+            vessels = await self._load_current_vessel_updates(list(pending))
+        except Exception:
+            # Preserve newer updates that arrived during the failed database read.
+            self._pending_vessel_updates = {**pending, **self._pending_vessel_updates}
+            raise
+        if not vessels:
+            return
+        vessels = [
+            {
+                **vessel,
+                "action": vessel.get("action", "upsert"),
+                "created": pending.get(str(vessel["mmsi"]), {}).get("_created"),
+            }
+            for vessel in vessels
+        ]
         await manager.broadcast_vessels(
             {
                 "action": "upsert",
@@ -133,12 +142,31 @@ class DataBroadcaster:
             }
         )
 
+    async def _load_current_vessel_updates(self, keys: list[str]) -> list[Dict[str, Any]]:
+        from app.db.session import async_session_factory
+        from app.services.vessel_ais_aggregation import get_current_vessels_by_mmsi
+
+        mmsis = [int(key) for key in keys if key.isdigit()]
+        vessels = []
+        async with async_session_factory() as db:
+            for offset in range(0, len(mmsis), VESSEL_STATE_QUERY_BATCH_SIZE):
+                vessels.extend(await get_current_vessels_by_mmsi(
+                    db, mmsis[offset:offset + VESSEL_STATE_QUERY_BATCH_SIZE]
+                ))
+        present = {int(vessel["mmsi"]) for vessel in vessels}
+        vessels.extend({"mmsi": mmsi, "action": "remove"} for mmsi in mmsis if mmsi not in present)
+        return vessels
+
     async def broadcast_vessels_periodically(self):
         while self.running:
             try:
                 await self.flush_vessel_updates()
-            except Exception:
-                pass
+            except Exception as exc:
+                logger.exception_event(
+                    "Failed to flush vessel updates",
+                    event="vessels.broadcast.failed",
+                    context={"error": str(exc)},
+                )
             await asyncio.sleep(self._vessel_flush_interval)
 
     async def broadcast_datasource_task_update(self, data: Dict[str, Any]):
@@ -150,7 +178,7 @@ class DataBroadcaster:
                 "timestamp": to_iso8601_utc(datetime.now(UTC)),
                 "payload": data,
             },
-            channel="all",
+            channel="datasource_tasks",
         )
 
     def start(self):

backend/app/core/websocket/manager.py

@@ -63,6 +63,8 @@ class ConnectionManager:
 
     def unsubscribe(self, websocket: WebSocket, channels: list[str]):
         for channel in {str(channel).strip() for channel in channels if str(channel).strip()}:
+            if channel == "vessels":
+                self.vessel_subscriptions.pop(websocket, None)
             subscribers = self.channel_subscriptions.get(channel)
             if subscribers is not None:
                 subscribers.discard(websocket)
@@ -88,7 +90,8 @@ class ConnectionManager:
         return subscription
 
     def _normalize_vessel_subscription(self, config: dict[str, Any]) -> dict[str, Any]:
-        bbox = config.get("bbox")
+        global_scope = config.get("scope") == "global"
+        bbox = [-180, -90, 180, 90] if global_scope else config.get("bbox")
         if not isinstance(bbox, (list, tuple)) or len(bbox) != 4:
             raise ValueError("vessels subscription requires bbox=[lon_min,lat_min,lon_max,lat_max]")
         try:
@@ -103,7 +106,7 @@ class ConnectionManager:
             raise ValueError("bbox longitude values must be between -180 and 180")
         if not (-90 <= lat_min <= 90 and -90 <= lat_max <= 90):
             raise ValueError("bbox latitude values must be between -90 and 90")
-        if (lon_max - lon_min) * (lat_max - lat_min) > MAX_VESSEL_BBOX_AREA:
+        if not global_scope and (lon_max - lon_min) * (lat_max - lat_min) > MAX_VESSEL_BBOX_AREA:
             raise ValueError("bbox is too large; zoom in or request a smaller viewport")
 
         zoom = int(config.get("zoom") or 1)
@@ -116,10 +119,11 @@ class ConnectionManager:
             if str(item).strip()
         }
         return {
+            "scope": "global" if global_scope else "viewport",
             "bbox": (lon_min, lat_min, lon_max, lat_max),
             "zoom": zoom,
             "limit": limit,
-            "type": vessel_types,
+            "type": sorted(vessel_types),
             "last_sent_at": None,
         }
 
@@ -152,7 +156,9 @@ class ConnectionManager:
                 vessel
                 for vessel in vessels
                 if self._vessel_matches_subscription(vessel, subscription)
-            ][: min(subscription["limit"], MAX_VESSEL_WS_MESSAGE_ITEMS)]
+            ]
+            if subscription.get("scope") != "global":
+                matched = matched[:subscription["limit"]]
             if not matched:
                 continue
             subscription["last_sent_at"] = datetime.now(UTC)
@@ -162,16 +168,24 @@ class ConnectionManager:
                 "timestamp": subscription["last_sent_at"].isoformat(),
                 "payload": {
                     **data,
-                    "vessels": matched,
+                    "vessels": [],
                     "subscription": {
                         "bbox": list(subscription["bbox"]),
                         "zoom": subscription["zoom"],
                         "limit": subscription["limit"],
+                        "scope": subscription.get("scope", "viewport"),
                     },
                 },
             }
             try:
-                await connection.send_json(message)
+                for offset in range(0, len(matched), MAX_VESSEL_WS_MESSAGE_ITEMS):
+                    await connection.send_json({
+                        **message,
+                        "payload": {
+                            **message["payload"],
+                            "vessels": matched[offset:offset + MAX_VESSEL_WS_MESSAGE_ITEMS],
+                        },
+                    })
             except Exception:
                 self.unsubscribe_all(connection)
 
@@ -180,6 +194,8 @@ class ConnectionManager:
         vessel: dict[str, Any],
         subscription: dict[str, Any],
     ) -> bool:
+        if vessel.get("action") == "remove" and subscription.get("scope") == "global":
+            return True
         try:
             lon = float(vessel.get("lon"))
             lat = float(vessel.get("lat"))

backend/app/db/bgp_prefix_geo.py

@@ -0,0 +1,51 @@
+"""SQL objects supporting indexed BGP prefix geography lookup."""
+
+from sqlalchemy import text
+from sqlalchemy.ext.asyncio import AsyncConnection
+
+BGP_PREFIX_GEO_FUNCTION_SQL = (
+    """
+    CREATE OR REPLACE FUNCTION planet_bgp_try_inet(value text)
+    RETURNS inet
+    LANGUAGE plpgsql IMMUTABLE STRICT PARALLEL SAFE
+    AS $$
+    BEGIN
+        RETURN value::inet;
+    EXCEPTION WHEN data_exception THEN
+        RETURN NULL;
+    END;
+    $$
+    """,
+    """
+    CREATE OR REPLACE FUNCTION planet_bgp_try_masklen(value text)
+    RETURNS integer
+    LANGUAGE plpgsql IMMUTABLE STRICT PARALLEL SAFE
+    AS $$
+    BEGIN
+        RETURN masklen(value::cidr);
+    EXCEPTION WHEN data_exception THEN
+        RETURN NULL;
+    END;
+    $$
+    """,
+)
+
+BGP_PREFIX_GEO_INDEX_NAME = "idx_bgp_prefix_geo_range_current"
+BGP_PREFIX_GEO_INDEX_SQL = f"""
+CREATE INDEX CONCURRENTLY IF NOT EXISTS {BGP_PREFIX_GEO_INDEX_NAME}
+ON collected_data USING gist (
+    planet_bgp_try_inet(metadata->>'range_start') inet_ops,
+    planet_bgp_try_inet(metadata->>'range_end') inet_ops
+)
+WHERE source IN (
+    'opengeofeed_prefix_geo',
+    'iptoasn_prefix_geo',
+    'nro_delegated_prefix_geo'
+)
+  AND COALESCE(is_current, TRUE) = TRUE
+"""
+
+
+async def install_bgp_prefix_geo_functions(connection: AsyncConnection) -> None:
+    for statement in BGP_PREFIX_GEO_FUNCTION_SQL:
+        await connection.execute(text(statement))

backend/app/db/session.py

@@ -6,6 +6,7 @@ from sqlalchemy.orm import declarative_base
 
 from app.core.config import settings
 from app.core.logging import get_logger
+from app.db.bgp_prefix_geo import install_bgp_prefix_geo_functions
 
 logger = get_logger(__name__)
 
@@ -150,7 +151,7 @@ DEFAULT_LOGIN_USERS = (
     {
         "username": "linkong",
         "email": "linkong@planet.local",
-        "password": "12345678",
+        "password": "LK12345678",
         "role": "super_admin",
     },
 )
@@ -203,6 +204,7 @@ async def init_db():
     import app.models.vessel_enrichment  # noqa: F401
     import app.models.datasource_mapping  # noqa: F401
     import app.models.earth_news  # noqa: F401
+    import app.models.earth_interactable  # noqa: F401
 
     logger.warning_event(
         "Database pool settings active",
@@ -218,6 +220,7 @@ async def init_db():
 
     async with engine.begin() as conn:
         await conn.run_sync(Base.metadata.create_all)
+        await install_bgp_prefix_geo_functions(conn)
         users_email_verified_existed = (
             await conn.execute(
                 text(
@@ -258,6 +261,414 @@ async def init_db():
                 """
             )
         )
+        await conn.execute(
+            text(
+                """
+                CREATE TABLE IF NOT EXISTS earth_data_change_events (
+                    id BIGSERIAL PRIMARY KEY,
+                    table_name VARCHAR(128) NOT NULL,
+                    operation VARCHAR(16) NOT NULL,
+                    source VARCHAR(128),
+                    entity_key VARCHAR(255),
+                    payload JSONB NOT NULL,
+                    occurred_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
+                    consumed_at TIMESTAMPTZ
+                )
+                """
+            )
+        )
+        await conn.execute(
+            text(
+                """
+                CREATE INDEX IF NOT EXISTS idx_earth_data_change_events_unconsumed
+                ON earth_data_change_events (consumed_at, id)
+                WHERE consumed_at IS NULL
+                """
+            )
+        )
+        await conn.execute(
+            text(
+                """
+                CREATE OR REPLACE FUNCTION planet_emit_earth_data_changed_statement(
+                    change_table TEXT,
+                    change_operation TEXT,
+                    change_source TEXT,
+                    source_record_count INTEGER,
+                    source_entity_keys TEXT[]
+                )
+                RETURNS VOID AS $$
+                DECLARE
+                    change_event_id BIGINT;
+                    change_payload JSONB;
+                BEGIN
+                    change_event_id := nextval(
+                        pg_get_serial_sequence('earth_data_change_events', 'id')::regclass
+                    );
+                    change_payload := jsonb_build_object(
+                        'event', 'earth.layer.changed',
+                        'event_id', change_event_id,
+                        'table', change_table,
+                        'operation', change_operation,
+                        'source', change_source,
+                        'entity_key', NULL,
+                        'entity_keys', COALESCE(to_jsonb(source_entity_keys), '[]'::jsonb),
+                        'records_processed', COALESCE(source_record_count, 0),
+                        'occurred_at', NOW()
+                    );
+
+                    INSERT INTO earth_data_change_events (
+                        id,
+                        table_name,
+                        operation,
+                        source,
+                        entity_key,
+                        payload,
+                        occurred_at
+                    ) VALUES (
+                        change_event_id,
+                        change_table,
+                        change_operation,
+                        change_source,
+                        NULL,
+                        change_payload,
+                        NOW()
+                    );
+
+                    PERFORM pg_notify(
+                        'planet_earth_data_changes',
+                        change_payload::text
+                    );
+                END;
+                $$ LANGUAGE plpgsql;
+                """
+            )
+        )
+        await conn.execute(
+            text(
+                """
+                CREATE OR REPLACE FUNCTION planet_emit_collected_data_changed_statement(
+                    change_operation TEXT,
+                    change_source TEXT,
+                    source_record_count INTEGER,
+                    source_entity_keys TEXT[]
+                )
+                RETURNS VOID AS $$
+                BEGIN
+                    PERFORM planet_emit_earth_data_changed_statement(
+                        'collected_data',
+                        change_operation,
+                        change_source,
+                        source_record_count,
+                        source_entity_keys
+                    );
+                END;
+                $$ LANGUAGE plpgsql;
+                """
+            )
+        )
+        await conn.execute(
+            text(
+                """
+                CREATE OR REPLACE FUNCTION planet_notify_earth_table_changed_statement()
+                RETURNS trigger AS $$
+                DECLARE
+                    change_source TEXT;
+                    source_record_count INTEGER;
+                    source_entity_keys TEXT[];
+                BEGIN
+                    IF TG_OP = 'INSERT' THEN
+                        FOR change_source IN
+                            SELECT DISTINCT COALESCE(NULLIF(row_data->>'source', ''), TG_TABLE_NAME)
+                            FROM (SELECT to_jsonb(t) AS row_data FROM new_rows AS t) changed_rows
+                        LOOP
+                            SELECT
+                                COUNT(*),
+                                ARRAY(
+                                    SELECT DISTINCT COALESCE(
+                                        NULLIF(row_data->>'entity_key', ''),
+                                        NULLIF(row_data->>'source_id', ''),
+                                        NULLIF(row_data->>'incident_key', ''),
+                                        NULLIF(row_data->>'id', ''),
+                                        NULLIF(row_data->>'mmsi', '')
+                                    )
+                                    FROM (SELECT to_jsonb(t) AS row_data FROM new_rows AS t) rows_for_keys
+                                    WHERE COALESCE(NULLIF(row_data->>'source', ''), TG_TABLE_NAME) = change_source
+                                    LIMIT 20
+                                )
+                            INTO source_record_count, source_entity_keys
+                            FROM (SELECT to_jsonb(t) AS row_data FROM new_rows AS t) rows_for_count
+                            WHERE COALESCE(NULLIF(row_data->>'source', ''), TG_TABLE_NAME) = change_source;
+
+                            PERFORM planet_emit_earth_data_changed_statement(
+                                TG_TABLE_NAME,
+                                TG_OP,
+                                change_source,
+                                source_record_count,
+                                source_entity_keys
+                            );
+                        END LOOP;
+                    ELSIF TG_OP = 'DELETE' THEN
+                        FOR change_source IN
+                            SELECT DISTINCT COALESCE(NULLIF(row_data->>'source', ''), TG_TABLE_NAME)
+                            FROM (SELECT to_jsonb(t) AS row_data FROM old_rows AS t) changed_rows
+                        LOOP
+                            SELECT
+                                COUNT(*),
+                                ARRAY(
+                                    SELECT DISTINCT COALESCE(
+                                        NULLIF(row_data->>'entity_key', ''),
+                                        NULLIF(row_data->>'source_id', ''),
+                                        NULLIF(row_data->>'incident_key', ''),
+                                        NULLIF(row_data->>'id', ''),
+                                        NULLIF(row_data->>'mmsi', '')
+                                    )
+                                    FROM (SELECT to_jsonb(t) AS row_data FROM old_rows AS t) rows_for_keys
+                                    WHERE COALESCE(NULLIF(row_data->>'source', ''), TG_TABLE_NAME) = change_source
+                                    LIMIT 20
+                                )
+                            INTO source_record_count, source_entity_keys
+                            FROM (SELECT to_jsonb(t) AS row_data FROM old_rows AS t) rows_for_count
+                            WHERE COALESCE(NULLIF(row_data->>'source', ''), TG_TABLE_NAME) = change_source;
+
+                            PERFORM planet_emit_earth_data_changed_statement(
+                                TG_TABLE_NAME,
+                                TG_OP,
+                                change_source,
+                                source_record_count,
+                                source_entity_keys
+                            );
+                        END LOOP;
+                    ELSIF TG_OP = 'UPDATE' THEN
+                        IF TG_TABLE_NAME = 'earth_news_items' AND NOT EXISTS (
+                            SELECT 1
+                            FROM new_rows AS current_row
+                            FULL JOIN old_rows AS previous_row USING (id)
+                            WHERE (to_jsonb(current_row) - 'last_seen_at' - 'updated_at')
+                                IS DISTINCT FROM
+                                  (to_jsonb(previous_row) - 'last_seen_at' - 'updated_at')
+                        ) THEN
+                            RETURN NULL;
+                        END IF;
+                        FOR change_source IN
+                            SELECT DISTINCT COALESCE(NULLIF(row_data->>'source', ''), TG_TABLE_NAME)
+                            FROM (
+                                SELECT to_jsonb(t) AS row_data FROM new_rows AS t
+                                UNION ALL
+                                SELECT to_jsonb(t) AS row_data FROM old_rows AS t
+                            ) changed_rows
+                        LOOP
+                            SELECT
+                                COUNT(*),
+                                ARRAY(
+                                    SELECT DISTINCT COALESCE(
+                                        NULLIF(row_data->>'entity_key', ''),
+                                        NULLIF(row_data->>'source_id', ''),
+                                        NULLIF(row_data->>'incident_key', ''),
+                                        NULLIF(row_data->>'id', ''),
+                                        NULLIF(row_data->>'mmsi', '')
+                                    )
+                                    FROM (
+                                        SELECT to_jsonb(t) AS row_data FROM new_rows AS t
+                                        UNION ALL
+                                        SELECT to_jsonb(t) AS row_data FROM old_rows AS t
+                                    ) rows_for_keys
+                                    WHERE COALESCE(NULLIF(row_data->>'source', ''), TG_TABLE_NAME) = change_source
+                                    LIMIT 20
+                                )
+                            INTO source_record_count, source_entity_keys
+                            FROM (
+                                SELECT to_jsonb(t) AS row_data FROM new_rows AS t
+                                UNION ALL
+                                SELECT to_jsonb(t) AS row_data FROM old_rows AS t
+                            ) rows_for_count
+                            WHERE COALESCE(NULLIF(row_data->>'source', ''), TG_TABLE_NAME) = change_source;
+
+                            PERFORM planet_emit_earth_data_changed_statement(
+                                TG_TABLE_NAME,
+                                TG_OP,
+                                change_source,
+                                source_record_count,
+                                source_entity_keys
+                            );
+                        END LOOP;
+                    END IF;
+
+                    RETURN NULL;
+                END;
+                $$ LANGUAGE plpgsql;
+                """
+            )
+        )
+        await conn.execute(
+            text(
+                """
+                CREATE OR REPLACE FUNCTION planet_notify_collected_data_changed_statement()
+                RETURNS trigger AS $$
+                DECLARE
+                    change_source TEXT;
+                    source_record_count INTEGER;
+                    source_entity_keys TEXT[];
+                BEGIN
+                    IF TG_OP = 'INSERT' THEN
+                        FOR change_source IN
+                            SELECT DISTINCT source FROM new_rows WHERE source IS NOT NULL
+                        LOOP
+                            SELECT
+                                COUNT(*),
+                                ARRAY(
+                                    SELECT DISTINCT COALESCE(entity_key, source_id, id::text)
+                                    FROM new_rows
+                                    WHERE source = change_source
+                                    LIMIT 20
+                                )
+                            INTO source_record_count, source_entity_keys
+                            FROM new_rows
+                            WHERE source = change_source;
+
+                            PERFORM planet_emit_collected_data_changed_statement(
+                                TG_OP,
+                                change_source,
+                                source_record_count,
+                                source_entity_keys
+                            );
+                        END LOOP;
+                    ELSIF TG_OP = 'DELETE' THEN
+                        FOR change_source IN
+                            SELECT DISTINCT source FROM old_rows WHERE source IS NOT NULL
+                        LOOP
+                            SELECT
+                                COUNT(*),
+                                ARRAY(
+                                    SELECT DISTINCT COALESCE(entity_key, source_id, id::text)
+                                    FROM old_rows
+                                    WHERE source = change_source
+                                    LIMIT 20
+                                )
+                            INTO source_record_count, source_entity_keys
+                            FROM old_rows
+                            WHERE source = change_source;
+
+                            PERFORM planet_emit_collected_data_changed_statement(
+                                TG_OP,
+                                change_source,
+                                source_record_count,
+                                source_entity_keys
+                            );
+                        END LOOP;
+                    ELSIF TG_OP = 'UPDATE' THEN
+                        FOR change_source IN
+                            SELECT DISTINCT source FROM (
+                                SELECT source FROM new_rows
+                                UNION
+                                SELECT source FROM old_rows
+                            ) changed_sources
+                            WHERE source IS NOT NULL
+                        LOOP
+                            SELECT
+                                COUNT(*),
+                                ARRAY(
+                                    SELECT DISTINCT COALESCE(entity_key, source_id, id::text)
+                                    FROM (
+                                        SELECT id, source_id, entity_key, source FROM new_rows
+                                        UNION ALL
+                                        SELECT id, source_id, entity_key, source FROM old_rows
+                                    ) changed_rows
+                                    WHERE source = change_source
+                                    LIMIT 20
+                                )
+                            INTO source_record_count, source_entity_keys
+                            FROM (
+                                SELECT id, source_id, entity_key, source FROM new_rows
+                                UNION ALL
+                                SELECT id, source_id, entity_key, source FROM old_rows
+                            ) changed_rows
+                            WHERE source = change_source;
+
+                            PERFORM planet_emit_collected_data_changed_statement(
+                                TG_OP,
+                                change_source,
+                                source_record_count,
+                                source_entity_keys
+                            );
+                        END LOOP;
+                    END IF;
+
+                    RETURN NULL;
+                END;
+                $$ LANGUAGE plpgsql;
+                """
+            )
+        )
+        for statement in (
+            "DROP TRIGGER IF EXISTS tr_planet_collected_data_changed ON collected_data",
+            "DROP TRIGGER IF EXISTS tr_planet_collected_data_changed_insert ON collected_data",
+            "DROP TRIGGER IF EXISTS tr_planet_collected_data_changed_update ON collected_data",
+            "DROP TRIGGER IF EXISTS tr_planet_collected_data_changed_delete ON collected_data",
+            "DROP FUNCTION IF EXISTS planet_notify_collected_data_changed()",
+            """
+            CREATE TRIGGER tr_planet_collected_data_changed_insert
+            AFTER INSERT ON collected_data
+            REFERENCING NEW TABLE AS new_rows
+            FOR EACH STATEMENT
+            EXECUTE FUNCTION planet_notify_collected_data_changed_statement()
+            """,
+            """
+            CREATE TRIGGER tr_planet_collected_data_changed_update
+            AFTER UPDATE ON collected_data
+            REFERENCING OLD TABLE AS old_rows NEW TABLE AS new_rows
+            FOR EACH STATEMENT
+            EXECUTE FUNCTION planet_notify_collected_data_changed_statement()
+            """,
+            """
+            CREATE TRIGGER tr_planet_collected_data_changed_delete
+            AFTER DELETE ON collected_data
+            REFERENCING OLD TABLE AS old_rows
+            FOR EACH STATEMENT
+            EXECUTE FUNCTION planet_notify_collected_data_changed_statement()
+            """,
+        ):
+            await conn.execute(text(statement))
+        for table_name in (
+            "bgp_observations",
+            "bgp_anomalies",
+            "bgp_incidents",
+            "bgp_collector_locations",
+            "vessel_static",
+            "vessel_position",
+            "vessel_current_state",
+            "ais_raw_observations",
+            "ais_source_health",
+            "compute_center_locations",
+            "earth_interactables",
+            "earth_news_items",
+        ):
+            for statement in (
+                f"DROP TRIGGER IF EXISTS tr_planet_{table_name}_changed_insert ON {table_name}",
+                f"DROP TRIGGER IF EXISTS tr_planet_{table_name}_changed_update ON {table_name}",
+                f"DROP TRIGGER IF EXISTS tr_planet_{table_name}_changed_delete ON {table_name}",
+                f"""
+                CREATE TRIGGER tr_planet_{table_name}_changed_insert
+                AFTER INSERT ON {table_name}
+                REFERENCING NEW TABLE AS new_rows
+                FOR EACH STATEMENT
+                EXECUTE FUNCTION planet_notify_earth_table_changed_statement()
+                """,
+                f"""
+                CREATE TRIGGER tr_planet_{table_name}_changed_update
+                AFTER UPDATE ON {table_name}
+                REFERENCING OLD TABLE AS old_rows NEW TABLE AS new_rows
+                FOR EACH STATEMENT
+                EXECUTE FUNCTION planet_notify_earth_table_changed_statement()
+                """,
+                f"""
+                CREATE TRIGGER tr_planet_{table_name}_changed_delete
+                AFTER DELETE ON {table_name}
+                REFERENCING OLD TABLE AS old_rows
+                FOR EACH STATEMENT
+                EXECUTE FUNCTION planet_notify_earth_table_changed_statement()
+                """,
+            ):
+                await conn.execute(text(statement))
         await conn.execute(
             text(
                 """
@@ -267,7 +678,16 @@ async def init_db():
                 ADD COLUMN IF NOT EXISTS phase_message VARCHAR(255),
                 ADD COLUMN IF NOT EXISTS phase_current BIGINT,
                 ADD COLUMN IF NOT EXISTS phase_total BIGINT,
-                ADD COLUMN IF NOT EXISTS phase_unit VARCHAR(30)
+                ADD COLUMN IF NOT EXISTS phase_unit VARCHAR(30),
+                ADD COLUMN IF NOT EXISTS source VARCHAR(100),
+                ADD COLUMN IF NOT EXISTS task_type VARCHAR(30) NOT NULL DEFAULT 'collect',
+                ADD COLUMN IF NOT EXISTS payload JSONB NOT NULL DEFAULT '{}'::jsonb,
+                ADD COLUMN IF NOT EXISTS rollback_policy VARCHAR(40) NOT NULL DEFAULT 'keep_committed_batches',
+                ADD COLUMN IF NOT EXISTS dedupe_key VARCHAR(180),
+                ADD COLUMN IF NOT EXISTS worker_id VARCHAR(120),
+                ADD COLUMN IF NOT EXISTS locked_at TIMESTAMPTZ,
+                ADD COLUMN IF NOT EXISTS requested_cancel_at TIMESTAMPTZ,
+                ADD COLUMN IF NOT EXISTS cancel_reason TEXT
                 """
             )
         )
@@ -283,6 +703,17 @@ async def init_db():
                 """
             )
         )
+        await conn.execute(
+            text(
+                """
+                ALTER TABLE earth_interactables
+                ADD COLUMN IF NOT EXISTS altitude DOUBLE PRECISION,
+                ADD COLUMN IF NOT EXISTS revision INTEGER NOT NULL DEFAULT 1,
+                ADD COLUMN IF NOT EXISTS is_deleted BOOLEAN NOT NULL DEFAULT FALSE,
+                ADD COLUMN IF NOT EXISTS deleted_at TIMESTAMPTZ
+                """
+            )
+        )
         await conn.execute(
             text(
                 """
@@ -307,6 +738,48 @@ async def init_db():
                 """
             )
         )
+        await conn.execute(
+            text(
+                """
+                CREATE INDEX IF NOT EXISTS idx_earth_interactables_layer_deleted
+                ON earth_interactables (layer, is_deleted)
+                """
+            )
+        )
+        await conn.execute(
+            text(
+                """
+                CREATE INDEX IF NOT EXISTS idx_earth_interactables_updated_at
+                ON earth_interactables (updated_at)
+                """
+            )
+        )
+        await conn.execute(
+            text(
+                """
+                CREATE INDEX IF NOT EXISTS idx_collection_tasks_source_status
+                ON collection_tasks (source, status)
+                """
+            )
+        )
+        await conn.execute(
+            text(
+                """
+                CREATE INDEX IF NOT EXISTS idx_collection_tasks_queue
+                ON collection_tasks (status, created_at, id)
+                WHERE status = 'queued'
+                """
+            )
+        )
+        await conn.execute(
+            text(
+                """
+                CREATE INDEX IF NOT EXISTS idx_collection_tasks_dedupe
+                ON collection_tasks (dedupe_key)
+                WHERE dedupe_key IS NOT NULL
+                """
+            )
+        )
         await conn.execute(
             text(
                 """
@@ -323,6 +796,22 @@ async def init_db():
                 """
             )
         )
+        await conn.execute(
+            text(
+                """
+                CREATE INDEX IF NOT EXISTS idx_vessel_current_bbox
+                ON vessel_current_state (lon, lat)
+                """
+            )
+        )
+        await conn.execute(
+            text(
+                """
+                CREATE INDEX IF NOT EXISTS idx_vessel_current_observed
+                ON vessel_current_state (observed_at DESC)
+                """
+            )
+        )
         await conn.execute(
             text(
                 """

backend/app/main.py

@@ -24,6 +24,11 @@ from app.services.earth_news_worker import (
     start_earth_news_target_worker,
     stop_earth_news_target_worker,
 )
+from app.services.earth_db_change_listener import (
+    start_earth_db_change_listener,
+    stop_earth_db_change_listener,
+)
+from app.services.data_jobs import start_data_job_worker, stop_data_job_worker
 
 
 configure_logging()
@@ -59,9 +64,13 @@ async def lifespan(app: FastAPI):
     start_scheduler()
     await sync_scheduler_with_datasources()
     broadcaster.start()
+    start_data_job_worker()
+    start_earth_db_change_listener()
     start_earth_news_target_worker()
     yield
     await stop_earth_news_target_worker()
+    await stop_earth_db_change_listener()
+    await stop_data_job_worker()
     broadcaster.stop()
     stop_scheduler()
 

backend/app/models/__init__.py

@@ -13,10 +13,11 @@ from app.models.compute_center_location import ComputeCenterLocationRecord
 from app.models.system_setting import SystemSetting
 from app.models.playground_session import PlaygroundSession
 from app.models.playground_message import PlaygroundMessage
-from app.models.system_log import SystemLog, AuditLog
+from app.models.system_log import AuditLog, ObservabilityEvent, ObservabilityEventGroup, SystemLog
 from app.models.vessel import AISConflictRecord, AISRawObservation, AISSourceHealth, VesselPosition, VesselStatic
 from app.models.datasource_mapping import DataSourceMappingTemplate
 from app.models.earth_news import EarthNewsItem
+from app.models.earth_interactable import EarthInteractable
 
 __all__ = [
     "User",
@@ -36,6 +37,8 @@ __all__ = [
     "ComputeCenterLocationRecord",
     "SystemLog",
     "AuditLog",
+    "ObservabilityEvent",
+    "ObservabilityEventGroup",
     "PlaygroundSession",
     "PlaygroundMessage",
     "VesselPosition",
@@ -45,4 +48,5 @@ __all__ = [
     "AISSourceHealth",
     "DataSourceMappingTemplate",
     "EarthNewsItem",
+    "EarthInteractable",
 ]

backend/app/models/alert.py

@@ -1,26 +1,12 @@
 from datetime import datetime
-from enum import Enum
-from typing import Optional
 
-from sqlalchemy import Column, Integer, String, DateTime, Text, ForeignKey, Enum as SQLEnum
-from sqlalchemy.orm import relationship
+from sqlalchemy import Column, Integer, String, DateTime, Text, Enum as SQLEnum
 
+from app.core.enums import AlertSeverity, AlertStatus
 from app.core.time import to_iso8601_utc
 from app.db.session import Base
 
 
-class AlertSeverity(str, Enum):
-    CRITICAL = "critical"
-    WARNING = "warning"
-    INFO = "info"
-
-
-class AlertStatus(str, Enum):
-    ACTIVE = "active"
-    ACKNOWLEDGED = "acknowledged"
-    RESOLVED = "resolved"
-
-
 class Alert(Base):
     __tablename__ = "alerts"
 

backend/app/models/bgp_anomaly.py

@@ -4,6 +4,7 @@ from datetime import datetime
 
 from sqlalchemy import Column, DateTime, Float, ForeignKey, Index, Integer, JSON, String, Text
 
+from app.core.enums import BGPStatus
 from app.core.time import to_iso8601_utc
 from app.db.session import Base
 
@@ -17,7 +18,7 @@ class BGPAnomaly(Base):
     source = Column(String(100), nullable=False, index=True)
     anomaly_type = Column(String(50), nullable=False, index=True)
     severity = Column(String(20), nullable=False, index=True)
-    status = Column(String(20), nullable=False, default="active", index=True)
+    status = Column(String(20), nullable=False, default=BGPStatus.ACTIVE.value, index=True)
     entity_key = Column(String(255), nullable=False, index=True)
     prefix = Column(String(64), nullable=True, index=True)
     origin_asn = Column(Integer, nullable=True, index=True)

backend/app/models/bgp_incident.py

@@ -4,6 +4,7 @@ from datetime import datetime
 
 from sqlalchemy import Column, DateTime, Float, ForeignKey, Index, Integer, JSON, String, Text
 
+from app.core.enums import BGPStatus
 from app.core.time import to_iso8601_utc
 from app.db.session import Base
 
@@ -20,7 +21,7 @@ class BGPIncident(Base):
     title = Column(String(255), nullable=False)
     summary = Column(Text, nullable=False)
     severity = Column(String(20), nullable=False, index=True)
-    status = Column(String(20), nullable=False, default="active", index=True)
+    status = Column(String(20), nullable=False, default=BGPStatus.ACTIVE.value, index=True)
     confidence = Column(Float, nullable=False, default=0.5)
     started_at = Column(DateTime(timezone=True), nullable=False, default=datetime.utcnow, index=True)
     ended_at = Column(DateTime(timezone=True), nullable=True)

backend/app/models/data_snapshot.py

@@ -1,6 +1,7 @@
 from sqlalchemy import Boolean, Column, DateTime, ForeignKey, Integer, JSON, String
 from sqlalchemy.sql import func
 
+from app.core.enums import SnapshotStatus
 from app.db.session import Base
 
 
@@ -16,7 +17,7 @@ class DataSnapshot(Base):
     started_at = Column(DateTime(timezone=True), server_default=func.now())
     completed_at = Column(DateTime(timezone=True), nullable=True)
     record_count = Column(Integer, default=0)
-    status = Column(String(20), nullable=False, default="running")
+    status = Column(String(20), nullable=False, default=SnapshotStatus.RUNNING.value)
     is_current = Column(Boolean, default=True, index=True)
     parent_snapshot_id = Column(Integer, ForeignKey("data_snapshots.id"), nullable=True, index=True)
     summary = Column(JSON, default={})

backend/app/models/datasource_mapping.py

@@ -3,6 +3,7 @@
 from sqlalchemy import Boolean, Column, DateTime, ForeignKey, Integer, JSON, String
 from sqlalchemy.sql import func
 
+from app.core.enums import MappingValidationStatus
 from app.db.session import Base
 
 
@@ -19,7 +20,7 @@ class DataSourceMappingTemplate(Base):
     target_schema = Column(String(80), nullable=False, index=True)
     mapping_json = Column(JSON, nullable=False, default={})
     sample_payload_hash = Column(String(64), nullable=True)
-    validation_status = Column(String(30), nullable=False, default="draft")
+    validation_status = Column(String(30), nullable=False, default=MappingValidationStatus.DRAFT.value)
     version = Column(Integer, nullable=False, default=1)
     is_active = Column(Boolean, nullable=False, default=False, index=True)
     created_at = Column(DateTime(timezone=True), server_default=func.now())

backend/app/models/earth_interactable.py

@@ -0,0 +1,30 @@
+"""Persistent Earth interactable objects."""
+
+from sqlalchemy import Boolean, Column, DateTime, Float, Index, Integer, JSON, String, Text
+from sqlalchemy.sql import func
+
+from app.db.session import Base
+
+
+class EarthInteractable(Base):
+    __tablename__ = "earth_interactables"
+
+    id = Column(String(160), primary_key=True)
+    layer = Column(String(80), nullable=False, default="interactables", index=True)
+    kind = Column(String(80), nullable=False, default="default", index=True)
+    label = Column(String(255), nullable=False, default="")
+    description = Column(Text, nullable=False, default="")
+    latitude = Column(Float, nullable=False)
+    longitude = Column(Float, nullable=False)
+    altitude = Column(Float, nullable=True)
+    revision = Column(Integer, nullable=False, default=1)
+    properties = Column(JSON, nullable=False, default=dict)
+    is_deleted = Column(Boolean, nullable=False, default=False, index=True)
+    created_at = Column(DateTime(timezone=True), server_default=func.now(), nullable=False)
+    updated_at = Column(DateTime(timezone=True), server_default=func.now(), onupdate=func.now())
+    deleted_at = Column(DateTime(timezone=True), nullable=True, index=True)
+
+    __table_args__ = (
+        Index("idx_earth_interactables_layer_deleted", "layer", "is_deleted"),
+        Index("idx_earth_interactables_updated_at", "updated_at"),
+    )

backend/app/models/playground_message.py

@@ -1,6 +1,7 @@
 from sqlalchemy import JSON, Boolean, Column, DateTime, ForeignKey, Integer, String, Text
 from sqlalchemy.sql import func
 
+from app.core.enums import PlaygroundMessageKind, PlaygroundMessageStatus
 from app.db.session import Base
 
 
@@ -13,8 +14,8 @@ class PlaygroundMessage(Base):
     user_id = Column(Integer, ForeignKey("users.id", ondelete="CASCADE"), nullable=False, index=True)
     parent_message_id = Column(Integer, ForeignKey("playground_messages.id", ondelete="SET NULL"), nullable=True)
     role = Column(String(20), nullable=False)
-    kind = Column(String(20), nullable=False, default="message")
-    status = Column(String(20), nullable=False, default="done")
+    kind = Column(String(20), nullable=False, default=PlaygroundMessageKind.MESSAGE.value)
+    status = Column(String(20), nullable=False, default=PlaygroundMessageStatus.DONE.value)
     title = Column(String(255), nullable=True)
     content = Column(Text, nullable=False, default="")
     thinking_content = Column(Text, nullable=False, default="")

backend/app/models/system_log.py

@@ -38,3 +38,46 @@ class AuditLog(Base):
     ip = Column(String(64), nullable=True)
     details = Column(JSON, nullable=False, default=dict)
     created_at = Column(DateTime(timezone=True), server_default=func.now())
+
+
+class ObservabilityEvent(Base):
+    __tablename__ = "observability_events"
+
+    id = Column(Integer, primary_key=True, autoincrement=True)
+    occurred_at = Column(DateTime(timezone=True), server_default=func.now(), index=True)
+    source = Column(String(50), nullable=False, index=True)
+    service = Column(String(50), nullable=True, index=True)
+    module = Column(String(120), nullable=True, index=True)
+    category = Column(String(80), nullable=True, index=True)
+    event = Column(String(160), nullable=True, index=True)
+    level = Column(String(20), nullable=False, index=True)
+    message = Column(Text, nullable=False)
+    fingerprint = Column(String(80), nullable=False, index=True)
+    request_id = Column(String(64), nullable=True, index=True)
+    trace_id = Column(String(64), nullable=True, index=True)
+    task_id = Column(String(120), nullable=True, index=True)
+    source_ref_id = Column(String(120), nullable=True, index=True)
+    provider = Column(String(120), nullable=True, index=True)
+    user_id = Column(Integer, nullable=True, index=True)
+    context = Column(JSON, nullable=False, default=dict)
+    occurrence_count = Column(Integer, nullable=False, default=1)
+    created_at = Column(DateTime(timezone=True), server_default=func.now())
+
+
+class ObservabilityEventGroup(Base):
+    __tablename__ = "observability_event_groups"
+
+    fingerprint = Column(String(80), primary_key=True)
+    source = Column(String(50), nullable=False, index=True)
+    service = Column(String(50), nullable=True, index=True)
+    module = Column(String(120), nullable=True, index=True)
+    category = Column(String(80), nullable=True, index=True)
+    event = Column(String(160), nullable=True, index=True)
+    last_level = Column(String(20), nullable=False, index=True)
+    sample_message = Column(Text, nullable=False)
+    sample_detail = Column(Text, nullable=True)
+    affected_sources = Column(JSON, nullable=False, default=list)
+    count = Column(Integer, nullable=False, default=0)
+    first_seen_at = Column(DateTime(timezone=True), nullable=False, index=True)
+    last_seen_at = Column(DateTime(timezone=True), nullable=False, index=True)
+    updated_at = Column(DateTime(timezone=True), server_default=func.now(), onupdate=func.now())

backend/app/models/task.py

@@ -1,8 +1,9 @@
-"""Collection Task model"""
+"""Datasource job model."""
 
-from sqlalchemy import BigInteger, Column, DateTime, Integer, String, Text, Float
+from sqlalchemy import BigInteger, Column, DateTime, Float, Integer, JSON, String, Text
 from sqlalchemy.sql import func
 
+from app.core.enums import JobStatus, JobType, RollbackPolicy
 from app.db.session import Base
 
 
@@ -11,8 +12,10 @@ class CollectionTask(Base):
 
     id = Column(Integer, primary_key=True, autoincrement=True)
     datasource_id = Column(Integer, nullable=False, index=True)
-    status = Column(String(20), nullable=False)  # pending, running, success, failed, cancelled
-    phase = Column(String(30), default="queued")
+    source = Column(String(100), nullable=True, index=True)
+    task_type = Column(String(30), nullable=False, default=JobType.COLLECT.value, index=True)
+    status = Column(String(20), nullable=False)  # queued, running, cancelling, success, failed, cancelled
+    phase = Column(String(30), default=JobStatus.QUEUED.value)
     phase_progress = Column(Float)
     phase_message = Column(String(255))
     phase_current = Column(BigInteger)
@@ -24,6 +27,13 @@ class CollectionTask(Base):
     total_records = Column(Integer, default=0)  # Total records to process
     progress = Column(Float, default=0.0)  # Progress percentage (0-100)
     error_message = Column(Text)
+    payload = Column(JSON, default=dict)
+    rollback_policy = Column(String(40), nullable=False, default=RollbackPolicy.KEEP_COMMITTED_BATCHES.value)
+    dedupe_key = Column(String(180), nullable=True, index=True)
+    worker_id = Column(String(120), nullable=True, index=True)
+    locked_at = Column(DateTime(timezone=True), nullable=True, index=True)
+    requested_cancel_at = Column(DateTime(timezone=True), nullable=True)
+    cancel_reason = Column(Text)
     created_at = Column(DateTime(timezone=True), server_default=func.now())
 
     def __repr__(self):

backend/app/models/user.py

@@ -1,6 +1,7 @@
 from sqlalchemy import Boolean, Column, DateTime, Integer, JSON, String
 from sqlalchemy.sql import func
 
+from app.core.enums import UserRole
 from app.db.session import Base
 
 
@@ -11,7 +12,7 @@ class User(Base):
     username = Column(String(50), unique=True, index=True, nullable=False)
     email = Column(String(255), unique=True, index=True, nullable=False)
     password_hash = Column(String(255), nullable=False)
-    role = Column(String(20), default="viewer")
+    role = Column(String(20), default=UserRole.VIEWER.value)
     gatekeeper_groups = Column(JSON, default=list)
     is_active = Column(Boolean, default=True)
     email_verified = Column(Boolean, default=False, nullable=False)

backend/app/models/vessel.py

@@ -3,6 +3,7 @@
 from sqlalchemy import BigInteger, Column, DateTime, Float, Index, Integer, JSON, SmallInteger, String
 from sqlalchemy.sql import func
 
+from app.core.enums import ConnectionState
 from app.core.time import to_iso8601_utc
 from app.db.session import Base
 
@@ -75,6 +76,67 @@ class VesselPosition(Base):
         }
 
 
+class VesselCurrentState(Base):
+    """Latest renderable state for one vessel, independent from AIS history."""
+
+    __tablename__ = "vessel_current_state"
+
+    mmsi = Column(BigInteger, primary_key=True)
+    lat = Column(Float, nullable=False)
+    lon = Column(Float, nullable=False)
+    sog = Column(Float, nullable=True)
+    cog = Column(Float, nullable=True)
+    heading = Column(SmallInteger, nullable=True)
+    nav_status = Column(SmallInteger, nullable=True, index=True)
+    name = Column(String(128), nullable=True)
+    callsign = Column(String(16), nullable=True)
+    vessel_type = Column(SmallInteger, nullable=True, index=True)
+    vessel_type_name = Column(String(64), nullable=True, index=True)
+    flag = Column(String(4), nullable=True, index=True)
+    length = Column(Float, nullable=True)
+    width = Column(Float, nullable=True)
+    draught = Column(Float, nullable=True)
+    imo = Column(BigInteger, nullable=True)
+    source = Column(String(100), nullable=False, index=True)
+    observed_at = Column(DateTime(timezone=True), nullable=False)
+    field_sources = Column(JSON, default=dict)
+    selected_reasons = Column(JSON, default=dict)
+    source_summary = Column(JSON, default=dict)
+    quality_flags = Column(JSON, default=list)
+    updated_at = Column(DateTime(timezone=True), nullable=False, server_default=func.now())
+
+    __table_args__ = (
+        Index("idx_vessel_current_bbox", "lon", "lat"),
+        Index("idx_vessel_current_observed", "observed_at"),
+    )
+
+    def to_dict(self) -> dict:
+        return {
+            "mmsi": self.mmsi,
+            "lat": self.lat,
+            "lon": self.lon,
+            "sog": self.sog,
+            "cog": self.cog,
+            "heading": self.heading,
+            "nav_status": self.nav_status,
+            "name": self.name,
+            "callsign": self.callsign,
+            "vessel_type": self.vessel_type,
+            "vessel_type_name": self.vessel_type_name,
+            "flag": self.flag,
+            "length": self.length,
+            "width": self.width,
+            "draught": self.draught,
+            "imo": self.imo,
+            "source": self.source,
+            "received_at": self.observed_at,
+            "field_sources": self.field_sources or {},
+            "selected_reasons": self.selected_reasons or {},
+            "source_summary": self.source_summary or {},
+            "quality_flags": self.quality_flags or [],
+        }
+
+
 class AISRawObservation(Base):
     """Source-level AIS fact before aggregation and conflict resolution."""
 
@@ -165,7 +227,7 @@ class AISSourceHealth(Base):
     __tablename__ = "ais_source_health"
 
     source = Column(String(100), primary_key=True)
-    connection_state = Column(String(32), nullable=False, default="disconnected", index=True)
+    connection_state = Column(String(32), nullable=False, default=ConnectionState.DISCONNECTED.value, index=True)
     last_seen_at = Column(DateTime(timezone=True), nullable=True, index=True)
     last_success_at = Column(DateTime(timezone=True), nullable=True, index=True)
     last_error = Column(String(500), nullable=True)

backend/app/schemas/ai.py

@@ -2,6 +2,7 @@ from typing import Any
 
 from pydantic import BaseModel, Field
 
+from app.core.enums import PlaygroundMessageKind, PlaygroundMessageRole, PlaygroundMessageStatus
 
 class AIContentBlock(BaseModel):
     type: str
@@ -110,9 +111,9 @@ class PlaygroundSessionUpsertRequest(BaseModel):
 
 class PlaygroundMessageRecord(BaseModel):
     id: str
-    role: str
-    kind: str = "message"
-    status: str = "done"
+    role: PlaygroundMessageRole
+    kind: PlaygroundMessageKind = PlaygroundMessageKind.MESSAGE
+    status: PlaygroundMessageStatus = PlaygroundMessageStatus.DONE
     title: str | None = None
     content: str = ""
     thinking_content: str = ""

backend/app/schemas/card_images.py

@@ -0,0 +1,128 @@
+"""Typed, validated configuration for optional object-card images."""
+
+from typing import Literal, Self
+from urllib.parse import urlsplit
+
+from pydantic import BaseModel, ConfigDict, Field, field_validator, model_validator
+
+EntityType = Literal["satellite", "compute_center", "cable", "landing_point", "vessel", "bgp", "bgp_collector", "earth_interactable"]
+MatchScope = Literal["object", "model", "group", "category"]
+
+
+def safe_url(value: str, *, local: bool = False) -> str:
+    if not value:
+        return value
+    if any(ord(char) < 32 for char in value) or "\\" in value:
+        raise ValueError("Invalid URL")
+    parts = urlsplit(value)
+    if local and value.startswith(("/earth/assets/cards/", "/api/v1/earth/card-images/files/")):
+        if (
+            ".." not in parts.path
+            and "%" not in parts.path
+            and not parts.netloc
+            and not parts.query
+            and not parts.fragment
+        ):
+            return value
+    if (
+        parts.scheme in {"http", "https"}
+        and parts.hostname
+        and not parts.username
+        and not parts.password
+    ):
+        return value
+    raise ValueError("Use an HTTP(S) URL without credentials or a managed image path")
+
+
+class CardImageAsset(BaseModel):
+    model_config = ConfigDict(extra="forbid", str_strip_whitespace=True)
+    id: str = Field(min_length=1, max_length=100, pattern=r"^[a-zA-Z0-9_-]+$")
+    title: str = Field(min_length=1, max_length=200)
+    url: str = Field(min_length=1, max_length=2000)
+    variants: dict[str, str] = Field(default_factory=dict, max_length=4)
+    width: int = Field(default=0, ge=0, le=20000)
+    height: int = Field(default=0, ge=0, le=20000)
+    credit: str = Field(default="", max_length=500)
+    source_url: str = Field(default="", max_length=2000)
+    license: str = Field(default="", max_length=300)
+    license_url: str = Field(default="", max_length=2000)
+    kind: Literal["photo", "model_photo", "render", "animation_still"] = "photo"
+    rights_status: Literal["available", "permission_pending"] = "available"
+    enabled: bool = True
+    focus_x: float = Field(default=50, ge=0, le=100, allow_inf_nan=False)
+    focus_y: float = Field(default=50, ge=0, le=100, allow_inf_nan=False)
+
+    @field_validator("url")
+    @classmethod
+    def validate_image_url(cls, value: str) -> str:
+        return safe_url(value, local=True)
+
+    @field_validator("source_url", "license_url")
+    @classmethod
+    def validate_source_url(cls, value: str) -> str:
+        return safe_url(value)
+
+    @field_validator("variants")
+    @classmethod
+    def validate_variants(cls, value: dict[str, str]) -> dict[str, str]:
+        for width, url in value.items():
+            if width not in {"640", "1280"}:
+                raise ValueError("Unsupported image variant")
+            safe_url(url, local=True)
+        return value
+
+
+class CardImageRule(BaseModel):
+    model_config = ConfigDict(extra="forbid", str_strip_whitespace=True)
+    id: str = Field(min_length=1, max_length=100, pattern=r"^[a-zA-Z0-9_-]+$")
+    entity_type: EntityType
+    scope: MatchScope
+    value: str = Field(min_length=1, max_length=300)
+    label: str = Field(default="", max_length=500)
+    asset_id: str | None = Field(default=None, max_length=100)
+    suppress: bool = False
+    enabled: bool = True
+
+    @model_validator(mode="after")
+    def validate_action(self) -> Self:
+        if self.scope != "object":
+            self.value = self.value.casefold()
+        if self.suppress and (self.scope != "object" or self.asset_id):
+            raise ValueError("Only an object rule may suppress its image, without an asset")
+        if not self.suppress and not self.asset_id:
+            raise ValueError("Choose an image or suppress this object")
+        return self
+
+    def target(self) -> tuple[str, str, str]:
+        return self.entity_type, self.scope, self.value
+
+
+class CardImageConfiguration(BaseModel):
+    model_config = ConfigDict(extra="forbid")
+    assets: list[CardImageAsset] = Field(default_factory=list, max_length=1000)
+    rules: list[CardImageRule] = Field(default_factory=list, max_length=3000)
+
+    @model_validator(mode="after")
+    def validate_relations(self) -> Self:
+        assets = {asset.id for asset in self.assets}
+        if len(assets) != len(self.assets) or len({rule.id for rule in self.rules}) != len(
+            self.rules
+        ):
+            raise ValueError("Image and rule IDs must be unique")
+        targets = [rule.target() for rule in self.rules]
+        if len(set(targets)) != len(targets):
+            raise ValueError("Each target may have only one rule")
+        if any(rule.asset_id and rule.asset_id not in assets for rule in self.rules):
+            raise ValueError("Remove image associations before deleting the image")
+        return self
+
+
+class CardImagePublish(BaseModel):
+    revision: str = Field(max_length=200)
+    config: CardImageConfiguration
+
+
+class CardImagePreview(BaseModel):
+    entity_type: EntityType
+    entity_id: str = Field(min_length=1, max_length=300)
+    config: CardImageConfiguration

backend/app/schemas/compute_configuration.py

@@ -0,0 +1,94 @@
+"""Validated, source-backed observations for the compute comparison cards."""
+
+from datetime import date
+from typing import Annotated, Literal, Self
+
+from app.services.compute_catalog import COUNTRY_PARTICIPANTS
+from pydantic import BaseModel, ConfigDict, Field, HttpUrl, field_validator, model_validator
+
+Year = Annotated[int, Field(strict=True, ge=2000, le=2100)]
+Value = Annotated[float, Field(strict=True, allow_inf_nan=False, ge=0, le=1e12)]
+Identifier = Annotated[str, Field(min_length=1, max_length=80, pattern=r"^[a-zA-Z0-9_-]+$")]
+Text = Annotated[str, Field(min_length=1, max_length=500)]
+
+
+class ConfigurationModel(BaseModel):
+    model_config = ConfigDict(extra="forbid", str_strip_whitespace=True)
+
+
+class ComputeSource(ConfigurationModel):
+    id: Identifier
+    title: Text
+    url: HttpUrl
+    publisher: Text
+    report_year: Year
+    published_at: date | None = None
+    locator: Text
+
+    @field_validator("url")
+    @classmethod
+    def public_reference(cls, value: HttpUrl) -> HttpUrl:
+        if value.username or value.password or len(str(value)) > 2000:
+            raise ValueError("Use a report URL without credentials (maximum 2000 characters)")
+        return value
+
+
+class ComputeObservation(ConfigurationModel):
+    year: Year
+    metric: Literal["global_share", "absolute"]
+    source_id: Identifier
+    values: dict[str, Value] = Field(min_length=1, max_length=250)
+
+    @model_validator(mode="after")
+    def comparable_values(self) -> Self:
+        if any(code not in COUNTRY_PARTICIPANTS for code in self.values):
+            raise ValueError(
+                "Use countries from the canonical compute catalog, not regional totals"
+            )
+        if self.metric == "global_share" and sum(self.values.values()) > 100.000001:
+            raise ValueError("Global shares for one year must total at most 100 percent")
+        return self
+
+
+class InnovationObservation(ConfigurationModel):
+    year: Year
+    value: Annotated[float, Field(strict=True, allow_inf_nan=False, ge=-100, le=100)]
+    source_id: Identifier
+
+
+class ComputeConfiguration(ConfigurationModel):
+    reviewed_at: date
+    sources: list[ComputeSource] = Field(max_length=200)
+    observations: list[ComputeObservation] = Field(max_length=202)
+    innovation: list[InnovationObservation] = Field(max_length=101)
+
+    @model_validator(mode="after")
+    def consistent_evidence(self) -> Self:
+        ids = [source.id for source in self.sources]
+        if len(ids) != len(set(ids)):
+            raise ValueError("Report source IDs must be unique")
+        keys = [(row.metric, row.year) for row in self.observations]
+        if len(keys) != len(set(keys)):
+            raise ValueError("Each metric/year may have only one observation")
+        years = [row.year for row in self.innovation]
+        if len(years) != len(set(years)):
+            raise ValueError("Innovation observation years must be unique")
+        sources = {source.id: source for source in self.sources}
+        for row in [*self.observations, *self.innovation]:
+            source = sources.get(row.source_id)
+            if source is None:
+                raise ValueError(f"Missing report source: {row.source_id}")
+            if row.year > source.report_year:
+                raise ValueError("Observation year must not be after its report year")
+        if self.reviewed_at > date.today():
+            raise ValueError("Review date must not be in the future")
+        return self
+
+
+class ComputeConfigurationWrite(ConfigurationModel):
+    revision: str = Field(min_length=1, max_length=80)
+    config: ComputeConfiguration
+
+
+class ComputeConfigurationReset(ConfigurationModel):
+    revision: str = Field(min_length=1, max_length=80)

backend/app/schemas/user.py

@@ -3,6 +3,7 @@ from typing import Optional
 
 from pydantic import BaseModel, EmailStr, Field
 
+from app.core.enums import OtpPurpose, UserRole
 
 class UserBase(BaseModel):
     username: str
@@ -11,13 +12,13 @@ class UserBase(BaseModel):
 
 class UserCreate(UserBase):
     password: str = Field(..., min_length=8)
-    role: str = "viewer"
+    role: UserRole = UserRole.VIEWER
     gatekeeper_groups: list[str] = Field(default_factory=list)
 
 
 class UserUpdate(BaseModel):
     email: Optional[EmailStr] = None
-    role: Optional[str] = None
+    role: Optional[UserRole] = None
     gatekeeper_groups: Optional[list[str]] = None
     is_active: Optional[bool] = None
 
@@ -35,6 +36,9 @@ class UserInDB(UserBase):
 
 
 class UserResponse(UserBase):
+    # Bootstrap accounts have reserved .local email addresses. Validate email
+    # on writes, while allowing existing local identities to be serialized.
+    email: str
     id: int
     role: str
     gatekeeper_groups: list[str] = Field(default_factory=list)
@@ -59,7 +63,7 @@ class VerifyEmailRequest(BaseModel):
 
 class ResendCodeRequest(BaseModel):
     email: EmailStr
-    purpose: str = Field(default="register", pattern="^(register|verify_email|reset_password)$")
+    purpose: OtpPurpose = OtpPurpose.REGISTER
 
 
 class ForgotPasswordRequest(BaseModel):

backend/app/services/ai_client.py

@@ -2,18 +2,24 @@ from __future__ import annotations
 
 import asyncio
 import json
+from time import perf_counter
 
 import httpx
 from fastapi import Depends, HTTPException, status
 from sqlalchemy.ext.asyncio import AsyncSession
 
 from app.core.config import settings
+from app.core.logging import get_logger
 from app.db.session import get_db
 from app.schemas.ai import (
     AIProviderStatusResponse,
     SituationalAnalysisRequest,
     SituationalAnalysisResponse,
 )
+from app.services.business_logs import emit_business_log, exception_context
+
+
+logger = get_logger(__name__, service="ai")
 
 
 class AIProviderClient:
@@ -64,7 +70,19 @@ class AIProviderClient:
         return headers
 
     async def get_status(self, request_id: str | None = None) -> AIProviderStatusResponse:
+        context = self._base_log_context(operation="status")
         if not self.service_url:
+            await emit_business_log(
+                logger,
+                event="ai.provider.status.failed",
+                message="AI provider status skipped because service URL is not configured",
+                category="ai",
+                level="warning",
+                service="ai",
+                module=__name__,
+                request_id=request_id,
+                context={**context, "status": "unconfigured"},
+            )
             return AIProviderStatusResponse(
                 provider="unconfigured",
                 enabled=False,
@@ -73,27 +91,133 @@ class AIProviderClient:
                 base_url=None,
             )
 
-        data = await self._request("GET", "/v1/provider/status", request_id=request_id)
-        return AIProviderStatusResponse.model_validate(data)
+        started_at = perf_counter()
+        await emit_business_log(
+            logger,
+            event="ai.provider.status.start",
+            message="AI provider status request started",
+            category="ai",
+            service="ai",
+            module=__name__,
+            request_id=request_id,
+            context=context,
+        )
+        try:
+            data = await self._request("GET", "/v1/provider/status", request_id=request_id, operation="status")
+            result = AIProviderStatusResponse.model_validate(data)
+            await emit_business_log(
+                logger,
+                event="ai.provider.status.success",
+                message="AI provider status request completed",
+                category="ai",
+                service="ai",
+                module=__name__,
+                request_id=request_id,
+                context={
+                    **context,
+                    "status": "success",
+                    "duration_ms": self._duration_ms(started_at),
+                    "result_provider": result.provider,
+                    "result_model": result.model,
+                    "configured": result.configured,
+                    "enabled": result.enabled,
+                },
+            )
+            return result
+        except Exception as exc:
+            await emit_business_log(
+                logger,
+                event="ai.provider.status.failed",
+                message="AI provider status request failed",
+                category="ai",
+                level="error",
+                service="ai",
+                module=__name__,
+                request_id=request_id,
+                context=exception_context(exc, {**context, "status": "failed", "duration_ms": self._duration_ms(started_at)}),
+            )
+            raise
 
     async def analyze(
         self,
         payload: SituationalAnalysisRequest,
         request_id: str | None = None,
     ) -> SituationalAnalysisResponse:
+        context = self._base_log_context(
+            operation="analyze",
+            preferred_model=payload.preferred_model,
+            input_summary=self._summarize_analysis_payload(payload),
+        )
         if not self.service_url:
+            await emit_business_log(
+                logger,
+                event="ai.provider.analyze.failed",
+                message="AI provider analyze skipped because service URL is not configured",
+                category="ai",
+                level="warning",
+                service="ai",
+                module=__name__,
+                request_id=request_id,
+                context={**context, "status": "unconfigured"},
+            )
             raise HTTPException(
                 status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
                 detail="AI provider service URL is not configured.",
             )
 
-        data = await self._request(
-            "POST",
-            "/v1/analyze",
-            json=payload.model_dump(),
+        started_at = perf_counter()
+        await emit_business_log(
+            logger,
+            event="ai.provider.analyze.start",
+            message="AI provider analyze request started",
+            category="ai",
+            service="ai",
+            module=__name__,
             request_id=request_id,
+            context=context,
         )
-        return SituationalAnalysisResponse.model_validate(data)
+        try:
+            data = await self._request(
+                "POST",
+                "/v1/analyze",
+                json=payload.model_dump(),
+                request_id=request_id,
+                operation="analyze",
+                payload_summary=context["input_summary"],
+            )
+            result = SituationalAnalysisResponse.model_validate(data)
+            await emit_business_log(
+                logger,
+                event="ai.provider.analyze.success",
+                message="AI provider analyze request completed",
+                category="ai",
+                service="ai",
+                module=__name__,
+                request_id=request_id,
+                context={
+                    **context,
+                    "status": "success",
+                    "duration_ms": self._duration_ms(started_at),
+                    "result_provider": result.provider,
+                    "result_model": result.model,
+                    "content_block_count": len(result.content_blocks or []),
+                    "thinking_block_count": len(result.thinking_blocks or []),
+                },
+            )
+            return result
+        except Exception as exc:
+            await emit_business_log(
+                logger,
+                event="ai.provider.analyze.failed",
+                message="AI provider analyze request failed",
+                category="ai",
+                level="error",
+                service="ai",
+                module=__name__,
+                request_id=request_id,
+                context=exception_context(exc, {**context, "status": "failed", "duration_ms": self._duration_ms(started_at)}),
+            )
+            raise
 
     async def _request(
         self,
@@ -101,9 +225,12 @@ class AIProviderClient:
         path: str,
         json: dict | None = None,
         request_id: str | None = None,
+        operation: str = "request",
+        payload_summary: dict | None = None,
     ) -> dict:
         last_error: Exception | None = None
         for attempt in range(1, self.retry_attempts + 1):
+            attempt_started_at = perf_counter()
             try:
                 async with httpx.AsyncClient(timeout=self.timeout) as client:
                     response = await client.request(
@@ -117,6 +244,15 @@ class AIProviderClient:
             except httpx.HTTPStatusError as exc:
                 last_error = exc
                 if attempt < self.retry_attempts and exc.response.status_code >= 500:
+                    await self._log_retry(
+                        operation=operation,
+                        request_id=request_id,
+                        attempt=attempt,
+                        status_code=exc.response.status_code,
+                        duration_ms=self._duration_ms(attempt_started_at),
+                        error=exc,
+                        payload_summary=payload_summary,
+                    )
                     await asyncio.sleep(0.3 * attempt)
                     continue
                 detail = exc.response.text or "AI provider service returned an error"
@@ -127,6 +263,14 @@ class AIProviderClient:
             except httpx.HTTPError as exc:
                 last_error = exc
                 if attempt < self.retry_attempts:
+                    await self._log_retry(
+                        operation=operation,
+                        request_id=request_id,
+                        attempt=attempt,
+                        duration_ms=self._duration_ms(attempt_started_at),
+                        error=exc,
+                        payload_summary=payload_summary,
+                    )
                     await asyncio.sleep(0.3 * attempt)
                     continue
                 raise HTTPException(
@@ -139,6 +283,71 @@ class AIProviderClient:
             detail=f"AI provider service request failed: {last_error}",
         )
 
+    def _base_log_context(self, **extra: object) -> dict:
+        llm_provider_apis = self.llm_config.get("model_provider_apis")
+        return {
+            "provider": self.llm_config.get("provider") or "",
+            "provider_api": self.llm_config.get("provider_api") or "",
+            "model": self.llm_config.get("model") or "",
+            "base_url_configured": bool(self.llm_config.get("base_url")),
+            "service_url_configured": bool(self.service_url),
+            "timeout_seconds": self.timeout,
+            "retry_attempts": self.retry_attempts,
+            "model_provider_api_count": len(llm_provider_apis or {}) if isinstance(llm_provider_apis, dict) else 0,
+            **extra,
+        }
+
+    @staticmethod
+    def _duration_ms(started_at: float) -> int:
+        return int((perf_counter() - started_at) * 1000)
+
+    @staticmethod
+    def _summarize_analysis_payload(payload: SituationalAnalysisRequest) -> dict:
+        context = payload.context if isinstance(payload.context, dict) else {}
+        thinking = payload.thinking if isinstance(payload.thinking, dict) else payload.thinking
+        return {
+            "title_length": len(payload.title or ""),
+            "objective_length": len(payload.objective or ""),
+            "observation_count": len(payload.observations or []),
+            "constraint_count": len(payload.constraints or []),
+            "has_system_prompt": bool(payload.system_prompt),
+            "thinking_enabled": bool(thinking),
+            "context_keys": sorted(str(key) for key in context.keys()),
+        }
+
+    async def _log_retry(
+        self,
+        *,
+        operation: str,
+        request_id: str | None,
+        attempt: int,
+        duration_ms: int,
+        error: BaseException,
+        status_code: int | None = None,
+        payload_summary: dict | None = None,
+    ) -> None:
+        await emit_business_log(
+            logger,
+            event=f"ai.provider.{operation}.retry",
+            message="AI provider request will retry",
+            category="ai",
+            level="warning",
+            service="ai",
+            module=__name__,
+            request_id=request_id,
+            context=exception_context(
+                error,
+                {
+                    **self._base_log_context(operation=operation),
+                    "attempt": attempt,
+                    "next_attempt": attempt + 1,
+                    "status_code": status_code,
+                    "duration_ms": duration_ms,
+                    "input_summary": payload_summary,
+                },
+            ),
+        )
+
 
 async def get_ai_provider_client(db: AsyncSession = Depends(get_db)) -> AIProviderClient:
     from app.api.v1.settings import get_runtime_ai_provider_config

backend/app/services/ai_tools/web_fetch.py

@@ -1,11 +1,18 @@
 from __future__ import annotations
 
 import hashlib
+from time import perf_counter
+from urllib.parse import urlparse
 
 import httpx
 from bs4 import BeautifulSoup
 
+from app.core.logging import get_logger
 from app.services.ai_tools.schemas import FetchedEvidence
+from app.services.business_logs import emit_business_log, exception_context
+
+
+logger = get_logger(__name__, service="ai_tool")
 
 
 class WebFetchError(RuntimeError):
@@ -29,8 +36,33 @@ async def fetch_url_evidence(
     timeout_seconds: int = 20,
     max_bytes: int = 1_500_000,
 ) -> FetchedEvidence:
+    started_at = perf_counter()
     if not url:
+        await emit_business_log(
+            logger,
+            event="ai_tool.web_fetch.failed",
+            message="WebFetch failed because URL is empty",
+            category="ai_tool",
+            level="warning",
+            service="ai_tool",
+            module=__name__,
+            context={"reason": "empty_url"},
+        )
         raise WebFetchError("url is required")
+    request_host = urlparse(url).netloc
+    await emit_business_log(
+        logger,
+        event="ai_tool.web_fetch.start",
+        message="WebFetch request started",
+        category="ai_tool",
+        service="ai_tool",
+        module=__name__,
+        context={
+            "url_host": request_host,
+            "timeout_seconds": timeout_seconds,
+            "max_bytes": max_bytes,
+        },
+    )
     try:
         async with httpx.AsyncClient(
             timeout=timeout_seconds,
@@ -41,10 +73,45 @@ async def fetch_url_evidence(
             response.raise_for_status()
             content = response.content[:max_bytes]
     except httpx.HTTPError as exc:
+        await emit_business_log(
+            logger,
+            event="ai_tool.web_fetch.failed",
+            message="WebFetch request failed",
+            category="ai_tool",
+            level="error",
+            service="ai_tool",
+            module=__name__,
+            context=exception_context(
+                exc,
+                {
+                    "url_host": request_host,
+                    "status": "failed",
+                    "duration_ms": int((perf_counter() - started_at) * 1000),
+                },
+            ),
+        )
         raise WebFetchError(f"failed to fetch page: {exc}") from exc
 
     title, text = _extract_title_and_text(content.decode(response.encoding or "utf-8", errors="ignore"))
     content_hash = hashlib.sha256(text.encode("utf-8")).hexdigest()
+    await emit_business_log(
+        logger,
+        event="ai_tool.web_fetch.success",
+        message="WebFetch request completed",
+        category="ai_tool",
+        service="ai_tool",
+        module=__name__,
+        context={
+            "url_host": request_host,
+            "final_url_host": urlparse(str(response.url)).netloc,
+            "status": "success",
+            "status_code": response.status_code,
+            "bytes_read": len(content),
+            "content_hash": content_hash,
+            "duration_ms": int((perf_counter() - started_at) * 1000),
+            "extractor": "beautifulsoup_basic",
+        },
+    )
     return FetchedEvidence(
         url=url,
         final_url=str(response.url),
@@ -53,4 +120,3 @@ async def fetch_url_evidence(
         content_hash=content_hash,
         extractor="beautifulsoup_basic",
     )
-

backend/app/services/ai_tools/web_search.py

@@ -1,11 +1,18 @@
 from __future__ import annotations
 
 from copy import deepcopy
+import hashlib
+from time import perf_counter
 from typing import Any
 
 import httpx
 
+from app.core.logging import get_logger
 from app.services.ai_tools.schemas import SearchEvidence, WebSearchConfig, WebSearchProviderConfig
+from app.services.business_logs import emit_business_log, exception_context
+
+
+logger = get_logger(__name__, service="ai_tool")
 
 
 WEB_SEARCH_PROVIDER_PRESETS: dict[str, dict[str, Any]] = {
@@ -120,29 +127,108 @@ class WebSearchClient:
         domains: list[str] | None = None,
         freshness_days: int | None = None,
     ) -> list[SearchEvidence]:
+        started_at = perf_counter()
         if not self.config.enabled:
+            await emit_business_log(
+                logger,
+                event="ai_tool.web_search.unavailable",
+                message="WebSearch skipped because integration is disabled",
+                category="ai_tool",
+                level="warning",
+                service="ai_tool",
+                module=__name__,
+                context={"provider": self.config.default_provider, "reason": "disabled"},
+            )
             raise WebSearchConfigurationError("WebSearch is disabled.")
         provider_config = self.config.active_provider_config
         provider = normalize_web_search_provider(provider_config.provider)
         if provider != "searxng" and not provider_config.api_key:
+            await emit_business_log(
+                logger,
+                event="ai_tool.web_search.unavailable",
+                message="WebSearch skipped because API key is not configured",
+                category="ai_tool",
+                level="warning",
+                service="ai_tool",
+                module=__name__,
+                context={"provider": provider, "reason": "missing_api_key"},
+            )
             raise WebSearchConfigurationError(f"{provider} API key is not configured.")
         query = " ".join(str(query or "").split())
         if not query:
+            await emit_business_log(
+                logger,
+                event="ai_tool.web_search.failed",
+                message="WebSearch failed because query is empty",
+                category="ai_tool",
+                level="warning",
+                service="ai_tool",
+                module=__name__,
+                context={"provider": provider, "reason": "empty_query"},
+            )
             raise WebSearchConfigurationError("search query is required.")
         limit = max_results or provider_config.max_results
-        if provider == "tavily":
-            return await self._search_tavily(provider_config, query, limit, domains, freshness_days)
-        if provider == "brave":
-            return await self._search_brave(provider_config, query, limit, domains)
-        if provider == "serpapi":
-            return await self._search_serpapi(provider_config, query, limit)
-        if provider == "exa":
-            return await self._search_exa(provider_config, query, limit, domains)
-        if provider == "firecrawl":
-            return await self._search_firecrawl(provider_config, query, limit)
-        if provider == "searxng":
-            return await self._search_searxng(provider_config, query, limit, domains)
-        raise WebSearchConfigurationError(f"Unsupported web search provider: {provider}")
+        context = {
+            "provider": provider,
+            "query_hash": hashlib.sha256(query.encode("utf-8")).hexdigest(),
+            "query_length": len(query),
+            "max_results": limit,
+            "domain_count": len(domains or []),
+            "freshness_days": freshness_days,
+        }
+        await emit_business_log(
+            logger,
+            event="ai_tool.web_search.start",
+            message="WebSearch request started",
+            category="ai_tool",
+            service="ai_tool",
+            module=__name__,
+            context=context,
+        )
+        try:
+            if provider == "tavily":
+                results = await self._search_tavily(provider_config, query, limit, domains, freshness_days)
+            elif provider == "brave":
+                results = await self._search_brave(provider_config, query, limit, domains)
+            elif provider == "serpapi":
+                results = await self._search_serpapi(provider_config, query, limit)
+            elif provider == "exa":
+                results = await self._search_exa(provider_config, query, limit, domains)
+            elif provider == "firecrawl":
+                results = await self._search_firecrawl(provider_config, query, limit)
+            elif provider == "searxng":
+                results = await self._search_searxng(provider_config, query, limit, domains)
+            else:
+                raise WebSearchConfigurationError(f"Unsupported web search provider: {provider}")
+            event = "ai_tool.web_search.success" if results else "ai_tool.web_search.empty"
+            await emit_business_log(
+                logger,
+                event=event,
+                message="WebSearch request completed" if results else "WebSearch returned no results",
+                category="ai_tool",
+                level="info" if results else "warning",
+                service="ai_tool",
+                module=__name__,
+                context={
+                    **context,
+                    "status": "success" if results else "empty",
+                    "result_count": len(results),
+                    "duration_ms": int((perf_counter() - started_at) * 1000),
+                },
+            )
+            return results
+        except Exception as exc:
+            await emit_business_log(
+                logger,
+                event="ai_tool.web_search.failed",
+                message="WebSearch request failed",
+                category="ai_tool",
+                level="error",
+                service="ai_tool",
+                module=__name__,
+                context=exception_context(exc, {**context, "status": "failed", "duration_ms": int((perf_counter() - started_at) * 1000)}),
+            )
+            raise
 
     async def test_connection(self) -> list[SearchEvidence]:
         return await self.search("Planet WebSearch connectivity test", max_results=1)
@@ -388,4 +474,3 @@ def _float_or_none(value: Any) -> float | None:
         return float(value)
     except (TypeError, ValueError):
         return None
-

backend/app/services/bgp_detectors.py

@@ -6,6 +6,7 @@ from collections import Counter, defaultdict
 from datetime import UTC, datetime
 from typing import Any
 
+from app.core.enums import BGPStatus
 from app.models.bgp_anomaly import BGPAnomaly
 
 
@@ -127,7 +128,7 @@ def detect_origin_change_anomalies(
                     source=source,
                     anomaly_type=anomaly_type,
                     severity=severity,
-                    status="active",
+                    status=BGPStatus.ACTIVE.value,
                     entity_key=f"{anomaly_type}:{prefix}:{new_origin}",
                     prefix=prefix,
                     origin_asn=sorted(historic)[0] if historic else None,
@@ -197,7 +198,7 @@ def detect_more_specific_burst_anomalies(
                 source=source,
                 anomaly_type="more_specific_burst",
                 severity="high",
-                status="active",
+                status=BGPStatus.ACTIVE.value,
                 entity_key=f"more_specific_burst:{root_prefix}:{len(unique_prefixes)}:{len(related_collectors)}",
                 prefix=sample.get("prefix"),
                 origin_asn=sample.get("origin_asn"),
@@ -267,7 +268,7 @@ def detect_mass_withdrawal_anomalies(
                 source=source,
                 anomaly_type="mass_withdrawal",
                 severity=severity,
-                status="active",
+                status=BGPStatus.ACTIVE.value,
                 entity_key=f"mass_withdrawal:{prefix}:{origin_asn}:{len(related_collectors)}:{count}",
                 prefix=prefix,
                 origin_asn=origin_asn,
@@ -354,7 +355,7 @@ def detect_route_leak_anomalies(
                 source=source,
                 anomaly_type="route_leak_candidate",
                 severity="high" if max_path_length >= dominant_length + 3 else "medium",
-                status="active",
+                status=BGPStatus.ACTIVE.value,
                 entity_key=f"route_leak_candidate:{prefix}:{max_path_length}:{len(related_collectors)}",
                 prefix=prefix,
                 origin_asn=sample_metadata.get("origin_asn"),
@@ -435,7 +436,7 @@ def detect_path_flap_anomalies(
                 source=source,
                 anomaly_type="path_flap",
                 severity=severity,
-                status="active",
+                status=BGPStatus.ACTIVE.value,
                 entity_key=f"path_flap:{prefix}:{transitions}:{len(distinct_paths)}",
                 prefix=prefix,
                 origin_asn=sample_metadata.get("origin_asn"),

backend/app/services/bgp_enrichment.py

@@ -111,15 +111,24 @@ async def _lookup_prefix_geography(
         result = await db.execute(
             text(
                 """
+                WITH candidates AS MATERIALIZED (
+                    SELECT source, id, metadata
+                    FROM collected_data
+                    WHERE source IN (
+                        'opengeofeed_prefix_geo',
+                        'iptoasn_prefix_geo',
+                        'nro_delegated_prefix_geo'
+                    )
+                      AND COALESCE(is_current, TRUE) = TRUE
+                      AND planet_bgp_try_inet(metadata->>'range_start') <= CAST(:range_start AS inet)
+                      AND planet_bgp_try_inet(metadata->>'range_end') >= CAST(:range_end AS inet)
+                )
                 SELECT metadata
-                FROM collected_data
+                FROM candidates
                 WHERE source = :source
-                  AND COALESCE(is_current, TRUE) = TRUE
                   AND metadata->>'family' = :family
-                  AND CAST(metadata->>'range_start' AS inet) <= CAST(:range_start AS inet)
-                  AND CAST(metadata->>'range_end' AS inet) >= CAST(:range_end AS inet)
                 ORDER BY
-                  masklen(CAST(metadata->>'prefix' AS cidr)) DESC NULLS LAST,
+                  planet_bgp_try_masklen(metadata->>'prefix') DESC NULLS LAST,
                   id DESC
                 LIMIT 1
                 """
@@ -146,6 +155,10 @@ async def _lookup_prefix_geography(
 
     results: dict[str, dict[str, Any]] = {}
 
+    # The GiST range predicate is selective, but PostgreSQL 15 greatly overestimates its cost.
+    # Avoid spending more time compiling JIT code than fetching the small candidate set.
+    await db.execute(text("SET LOCAL jit = off"))
+
     for prefix in prefix_values:
         try:
             network = ipaddress.ip_network(prefix, strict=False)

backend/app/services/bgp_incidents.py

@@ -9,6 +9,7 @@ from sqlalchemy import select
 from sqlalchemy.ext.asyncio import AsyncSession
 
 from app.core.collected_data_fields import get_record_field
+from app.core.enums import BGPStatus
 from app.models.bgp_anomaly import BGPAnomaly
 from app.models.bgp_incident import BGPIncident
 from app.models.collected_data import CollectedData
@@ -290,7 +291,7 @@ async def create_bgp_incidents_for_anomalies(
             existing.title = title
             existing.summary = summary
             existing.severity = severity
-            existing.status = "active"
+            existing.status = BGPStatus.ACTIVE.value
             existing.confidence = confidence
             existing.started_at = primary.started_at or existing.started_at or datetime.now(UTC)
             existing.ended_at = None
@@ -313,7 +314,7 @@ async def create_bgp_incidents_for_anomalies(
                 title=title,
                 summary=summary,
                 severity=severity,
-                status="active",
+                status=BGPStatus.ACTIVE.value,
                 confidence=confidence,
                 started_at=primary.started_at or datetime.now(UTC),
                 affected_prefixes=prefixes,

backend/app/services/business_logs.py

@@ -0,0 +1,117 @@
+from __future__ import annotations
+
+import asyncio
+from collections.abc import Mapping
+from typing import Any
+
+from app.core.logging import PlanetLoggerAdapter, sanitize_log_value
+from app.core.request_context import get_request_id
+from app.services.persistent_logs import record_system_log
+
+
+LEVEL_METHODS = {
+    "debug": "debug_event",
+    "info": "info_event",
+    "warning": "warning_event",
+    "error": "error_event",
+}
+
+
+def normalize_business_level(level: str | None) -> str:
+    normalized = str(level or "info").strip().lower()
+    if normalized in {"warn", "warning"}:
+        return "warning"
+    if normalized in {"err", "error", "critical", "fatal"}:
+        return "error"
+    if normalized == "debug":
+        return "debug"
+    return "info"
+
+
+def build_business_context(
+    context: Mapping[str, Any] | None = None,
+    **fields: Any,
+) -> dict[str, Any]:
+    payload = dict(context or {})
+    for key, value in fields.items():
+        if value is not None:
+            payload[key] = value
+    return sanitize_log_value(payload)
+
+
+async def emit_business_log(
+    logger: PlanetLoggerAdapter,
+    *,
+    event: str,
+    message: str,
+    category: str,
+    level: str = "info",
+    source: str = "backend",
+    service: str | None = None,
+    module: str | None = None,
+    request_id: str | None = None,
+    user_id: int | None = None,
+    context: Mapping[str, Any] | None = None,
+) -> None:
+    normalized_level = normalize_business_level(level)
+    safe_context = build_business_context(context)
+    log_method = getattr(logger, LEVEL_METHODS[normalized_level])
+    log_method(message, event=event, context=safe_context)
+    await record_system_log(
+        source=source,
+        level=normalized_level,
+        message=message,
+        service=service,
+        module=module,
+        event=event,
+        request_id=request_id or get_request_id(),
+        user_id=user_id,
+        category=category,
+        context=safe_context,
+    )
+
+
+def emit_business_log_background(
+    logger: PlanetLoggerAdapter,
+    *,
+    event: str,
+    message: str,
+    category: str,
+    level: str = "info",
+    source: str = "backend",
+    service: str | None = None,
+    module: str | None = None,
+    request_id: str | None = None,
+    user_id: int | None = None,
+    context: Mapping[str, Any] | None = None,
+) -> None:
+    normalized_level = normalize_business_level(level)
+    safe_context = build_business_context(context)
+    log_method = getattr(logger, LEVEL_METHODS[normalized_level])
+    log_method(message, event=event, context=safe_context)
+    try:
+        loop = asyncio.get_running_loop()
+    except RuntimeError:
+        return
+    loop.create_task(
+        record_system_log(
+            source=source,
+            level=normalized_level,
+            message=message,
+            service=service,
+            module=module,
+            event=event,
+            request_id=request_id or get_request_id(),
+            user_id=user_id,
+            category=category,
+            context=safe_context,
+        )
+    )
+
+
+def exception_context(exc: BaseException, context: Mapping[str, Any] | None = None) -> dict[str, Any]:
+    return build_business_context(
+        context,
+        error_type=type(exc).__name__,
+        error=str(exc),
+    )

backend/app/services/card_image_configuration.py

@@ -0,0 +1,174 @@
+"""Revision-checked image settings; new bundled assets never replace user edits."""
+
+from uuid import uuid4
+
+from fastapi import HTTPException
+from sqlalchemy import select
+from sqlalchemy.dialects.postgresql import insert
+from sqlalchemy.ext.asyncio import AsyncSession
+
+from app.core.request_context import get_request_id
+from app.models.system_log import AuditLog
+from app.models.system_setting import SystemSetting
+from app.models.user import User
+from app.schemas.card_images import CardImageConfiguration
+from app.services.card_image_defaults import (
+    bundled_configuration,
+    previous_bundled_assets,
+    previous_bundled_rules,
+)
+
+CATEGORY = "earth_card_images"
+
+
+def effective_envelope(payload: dict | None) -> dict:
+    defaults, version = bundled_configuration()
+    config = (
+        CardImageConfiguration.model_validate(payload["config"])
+        if payload
+        else defaults.model_copy(deep=True)
+    )
+    if payload:
+        previous = previous_bundled_assets()
+        latest = {asset.id: asset for asset in defaults.assets}
+        for index, asset in enumerate(config.assets):
+            old = next((item for item in previous.get(asset.id, []) if item.url == asset.url), None)
+            # Replace the old bundled source while preserving fields the user
+            # changed (crop, title, enabled state, etc.). Custom URLs are untouched.
+            if old is not None and asset.url == old.url and asset.id in latest:
+                overrides = {
+                    key: value
+                    for key, value in asset.model_dump().items()
+                    if value != getattr(old, key)
+                }
+                config.assets[index] = latest[asset.id].model_copy(update=overrides, deep=True)
+        known_assets = set(payload.get("known_assets", []))
+        asset_ids = {asset.id for asset in config.assets}
+        config.assets.extend(
+            asset.model_copy(deep=True)
+            for asset in defaults.assets
+            if asset.id not in known_assets and asset.id not in asset_ids
+        )
+        prior_rules = previous_bundled_rules()
+        latest_rules = {rule.id: rule for rule in defaults.rules}
+        for index, rule in enumerate(config.rules):
+            old_rule = prior_rules.get(rule.id)
+            if (
+                old_rule
+                and rule.id in latest_rules
+                and rule.target() == old_rule.target()
+                and rule.asset_id == old_rule.asset_id
+            ):
+                overrides = {
+                    key: value
+                    for key, value in rule.model_dump().items()
+                    if value != getattr(old_rule, key)
+                }
+                config.rules[index] = latest_rules[rule.id].model_copy(update=overrides, deep=True)
+        known_rules = set(payload.get("known_rules", []))
+        rule_ids = {rule.id for rule in config.rules}
+        targets = {rule.target() for rule in config.rules}
+        available = {asset.id for asset in config.assets}
+        config.rules.extend(
+            rule.model_copy(deep=True)
+            for rule in defaults.rules
+            if rule.id not in known_rules
+            and rule.id not in rule_ids
+            and rule.target() not in targets
+            and rule.asset_id in available
+        )
+    return {
+        "config": config.model_dump(mode="json"),
+        "revision": f"{version}:{payload['revision'] if payload else 'default'}",
+        "is_default": payload is None,
+    }
+
+
+async def read_configuration(db: AsyncSession) -> dict:
+    record = (
+        await db.execute(select(SystemSetting).where(SystemSetting.category == CATEGORY))
+    ).scalar_one_or_none()
+    return effective_envelope(record.payload if record else None)
+
+
+async def publish_configuration(
+    db: AsyncSession, revision: str, config: CardImageConfiguration, actor: User
+) -> dict:
+    defaults, _ = bundled_configuration()
+    pending = {asset.id for asset in defaults.assets if asset.rights_status == "permission_pending"}
+    if any(
+        asset.id in pending and asset.rights_status != "permission_pending"
+        for asset in config.assets
+    ):
+        raise HTTPException(422, "Bundled image permission is still unverified")
+    initial = {
+        "config": defaults.model_dump(mode="json"),
+        "revision": "default",
+        "known_assets": [],
+        "known_rules": [],
+    }
+    await db.execute(
+        insert(SystemSetting)
+        .values(category=CATEGORY, payload=initial)
+        .on_conflict_do_nothing(index_elements=[SystemSetting.category])
+    )
+    record = (
+        await db.execute(
+            select(SystemSetting).where(SystemSetting.category == CATEGORY).with_for_update()
+        )
+    ).scalar_one()
+    if effective_envelope(record.payload)["revision"] != revision:
+        await db.rollback()
+        raise HTTPException(409, "Configuration changed. Reload before saving your edits.")
+    before = record.payload
+    record.payload = {
+        "config": config.model_dump(mode="json"),
+        "revision": uuid4().hex,
+        "known_assets": [asset.id for asset in defaults.assets],
+        "known_rules": [rule.id for rule in defaults.rules],
+    }
+    db.add(
+        AuditLog(
+            actor_id=actor.id,
+            actor_name=actor.username,
+            action="card_images.update",
+            target_type="system_setting",
+            target_id=CATEGORY,
+            result="success",
+            request_id=get_request_id(),
+            details={"before": before, "after": record.payload},
+        )
+    )
+    await db.commit()
+    return effective_envelope(record.payload)
+
+
+def resolve_image(config: CardImageConfiguration, context: dict[str, str]) -> dict:
+    assets = {asset.id: asset for asset in config.assets}
+    for scope in ("object", "model", "group", "category"):
+        value = context.get("entity_id" if scope == "object" else scope)
+        values = [value] if value else []
+        if scope == "category" and value != "*":
+            values.append("*")
+        for value in values:
+            value = value if scope == "object" else value.casefold()
+            rule = next(
+                (
+                    rule
+                    for rule in config.rules
+                    if rule.enabled
+                    and rule.entity_type == context["entity_type"]
+                    and rule.scope == scope
+                    and rule.value == value
+                ),
+                None,
+            )
+            if not rule:
+                continue
+            match = {"scope": scope, "value": value, "rule_id": rule.id}
+            if rule.suppress:
+                return {"image": None, "match": match, "suppressed": True}
+            asset = assets.get(rule.asset_id)
+            if asset and asset.enabled:
+                return {"image": asset.model_dump(mode="json"), "match": match, "suppressed": False}
+    return {"image": None, "match": None, "suppressed": False}

backend/app/services/card_image_defaults.py

@@ -0,0 +1,108 @@
+"""Read immutable bundled assets without inserting machine-specific database rows."""
+
+import hashlib
+import json
+from functools import lru_cache
+from pathlib import Path
+
+from app.schemas.card_images import CardImageAsset, CardImageConfiguration, CardImageRule
+
+REPO_ROOT = Path(__file__).resolve().parents[3]
+MANIFEST_PATH = REPO_ROOT / "frontend/public/earth/assets/cards/manifest.json"
+
+
+def _asset_from_manifest(item: dict) -> CardImageAsset:
+    pending = item.get("redistribution_status") == "permission_not_verified"
+    return CardImageAsset(
+        id=item["id"],
+        title=item["label"],
+        url=item["url"],
+        width=item["width"],
+        height=item["height"],
+        credit=item["credit"],
+        source_url=item.get("source_page") or "",
+        license=item["license"],
+        license_url=item.get("license_url") or "",
+        kind=item["kind"],
+        rights_status="permission_pending" if pending else "available",
+        enabled=item.get("enabled", not pending),
+        focus_x=item.get("focus_x", 50),
+        focus_y=item.get("focus_y", 50),
+        variants=item.get("variants", {}),
+    )
+
+
+@lru_cache(maxsize=1)
+def previous_bundled_assets() -> dict[str, list[CardImageAsset]]:
+    manifest = json.loads(MANIFEST_PATH.read_bytes())
+    return {
+        key: [CardImageAsset.model_validate(item) for item in value]
+        for key, value in manifest.get("previous_defaults", {}).items()
+    }
+
+
+@lru_cache(maxsize=1)
+def previous_bundled_rules() -> dict[str, CardImageRule]:
+    manifest = json.loads(MANIFEST_PATH.read_bytes())
+    return {
+        key: CardImageRule.model_validate(value)
+        for key, value in manifest.get("previous_rules", {}).items()
+    }
+
+
+@lru_cache(maxsize=1)
+def bundled_configuration() -> tuple[CardImageConfiguration, str]:
+    content = MANIFEST_PATH.read_bytes()
+    manifest = json.loads(content)
+    assets, rules = [], []
+    for item in manifest["assets"]:
+        asset = _asset_from_manifest(item)
+        assets.append(asset)
+        if not asset.enabled:
+            continue
+        hints = item.get("binding_hints", {})
+        for index, target in enumerate(hints.get("defaults", [])):
+            rules.append(
+                CardImageRule(
+                    id=target.get("id", f"builtin-{asset.id}-default-{index}"),
+                    entity_type=target["entity_type"],
+                    scope=target["scope"],
+                    value=target["value"],
+                    label=asset.title,
+                    asset_id=asset.id,
+                )
+            )
+        for index, entity_id in enumerate(hints.get("entity_ids", [])):
+            rules.append(
+                CardImageRule(
+                    id=f"builtin-{asset.id}-entity-{index}",
+                    entity_type=item["entity_type"],
+                    scope="object",
+                    value=entity_id,
+                    label=asset.title,
+                    asset_id=asset.id,
+                )
+            )
+        for norad_id in hints.get("norad_ids", []):
+            rules.append(
+                CardImageRule(
+                    id=f"builtin-{asset.id}-{norad_id}",
+                    entity_type="satellite",
+                    scope="object",
+                    value=str(norad_id),
+                    label=f"{asset.title} · NORAD {norad_id}",
+                    asset_id=asset.id,
+                )
+            )
+        if group := hints.get("constellation_group"):
+            rules.append(
+                CardImageRule(
+                    id=f"builtin-{asset.id}-group",
+                    entity_type="satellite",
+                    scope="group",
+                    value=group,
+                    label=asset.title,
+                    asset_id=asset.id,
+                )
+            )
+    return CardImageConfiguration(assets=assets, rules=rules), hashlib.sha256(content).hexdigest()

backend/app/services/card_image_files.py

@@ -0,0 +1,95 @@
+"""Validated raster uploads and content-addressed, bounded card-size derivatives."""
+
+import hashlib
+import io
+import re
+import warnings
+from uuid import uuid4
+from pathlib import Path
+import os
+
+from fastapi import HTTPException
+from PIL import Image, ImageOps, UnidentifiedImageError
+
+from app.schemas.card_images import CardImageAsset
+from app.services.card_image_defaults import REPO_ROOT
+
+ASSET_DIR = Path(
+    os.environ.get("PLANET_CARD_IMAGE_DIR", str(REPO_ROOT / "data/earth-card-images"))
+).resolve()
+MAX_UPLOAD_BYTES = 12 * 1024 * 1024
+MAX_PIXELS = 32_000_000
+FILE_PATTERN = re.compile(r"^[a-f0-9]{64}-(?:original\.(?:jpg|png|webp)|(?:640|1280)\.webp)$")
+FORMATS = {"JPEG": "jpg", "PNG": "png", "WEBP": "webp"}
+
+
+def uploaded_path(filename: str) -> Path:
+    if not FILE_PATTERN.fullmatch(filename):
+        raise HTTPException(404, "Image not found")
+    return ASSET_DIR / filename
+
+
+def store_image(content: bytes, title: str) -> CardImageAsset:
+    if len(content) > MAX_UPLOAD_BYTES:
+        raise HTTPException(413, "Image must be 12 MiB or smaller")
+    try:
+        with warnings.catch_warnings():
+            warnings.simplefilter("error", Image.DecompressionBombWarning)
+            image = Image.open(io.BytesIO(content))
+            extension = FORMATS.get(image.format)
+            if (
+                not extension
+                or image.width * image.height > MAX_PIXELS
+                or getattr(image, "is_animated", False)
+            ):
+                raise ValueError("Unsupported or oversized raster")
+            image.load()
+            image = ImageOps.exif_transpose(image).convert(
+                "RGBA" if "A" in image.getbands() or "transparency" in image.info else "RGB"
+            )
+    except (
+        UnidentifiedImageError,
+        OSError,
+        ValueError,
+        Image.DecompressionBombError,
+        Image.DecompressionBombWarning,
+    ) as error:
+        raise HTTPException(
+            422, "Use a valid, still JPEG, PNG or WebP image below 32 megapixels"
+        ) from error
+    digest = hashlib.sha256(content).hexdigest()
+    ASSET_DIR.mkdir(parents=True, exist_ok=True)
+    original = f"{digest}-original.{extension}"
+    original_path = uploaded_path(original)
+    if not original_path.exists():
+        temporary = original_path.with_suffix(f".{uuid4().hex}.tmp")
+        temporary.write_bytes(content)
+        temporary.replace(original_path)
+    variants = {}
+    for size, content in render_variants(image).items():
+        filename = f"{digest}-{size}.webp"
+        path = uploaded_path(filename)
+        if not path.exists():
+            temporary = path.with_suffix(f".{uuid4().hex}.tmp")
+            temporary.write_bytes(content)
+            temporary.replace(path)
+        variants[size] = f"/api/v1/earth/card-images/files/{filename}"
+    return CardImageAsset(
+        id=f"upload-{digest[:24]}",
+        title=title or "Image",
+        url=f"/api/v1/earth/card-images/files/{original}",
+        variants=variants,
+        width=image.width,
+        height=image.height,
+    )
+
+
+def render_variants(image: Image.Image) -> dict[str, bytes]:
+    result = {}
+    for size in (640, 1280):
+        resized = image.copy()
+        resized.thumbnail((size, size), Image.Resampling.LANCZOS)
+        buffer = io.BytesIO()
+        resized.save(buffer, format="WEBP", quality=93, method=4)
+        result[str(size)] = buffer.getvalue()
+    return result

backend/app/services/card_image_targets.py

@@ -0,0 +1,180 @@
+"""Bounded catalog queries and type defaults for every Earth object card."""
+
+from sqlalchemy import String, case, cast, func, literal, or_, select
+from sqlalchemy.ext.asyncio import AsyncSession
+
+from app.core.compute_types import (
+    COMPUTE_SITE_TYPES,
+    compute_site_type_expression,
+    compute_record_filter,
+)
+from app.models.bgp_incident import BGPIncident
+from app.models.bgp_collector_location import BGPCollectorLocation
+from app.models.collected_data import CollectedData
+from app.models.earth_interactable import EarthInteractable
+from app.models.vessel import VesselCurrentState
+from app.schemas.card_images import EntityType, MatchScope
+
+SOURCES = {
+    "satellite": ("celestrak_tle", "spacetrack_tle"),
+    "compute_center": ("top500", "epoch_ai_gpu"),
+    "cable": ("arcgis_cables",),
+    "landing_point": ("arcgis_landing_points",),
+}
+# Named constellation families, not inferred hardware models or generations.
+SATELLITE_GROUP_PATTERNS = (
+    ("STARLINK%", "starlink"),
+    ("IRIDIUM%", "iridium-next"),
+    ("BEIDOU-3%", "beidou-3"),
+    ("BEIDOU%", "beidou"),
+    ("QIANFAN%", "qianfan"),
+    ("ONEWEB%", "oneweb"),
+    ("KUIPER%", "kuiper"),
+)
+
+
+def collected_query(entity_type: EntityType):
+    record, metadata = CollectedData, CollectedData.extra_data
+    model = func.coalesce(
+        metadata["satellite_model" if entity_type == "satellite" else "model"].as_string(), ""
+    )
+    category = (
+        compute_site_type_expression(record)
+        if entity_type == "compute_center"
+        else literal(entity_type)
+    )
+    if entity_type == "satellite":
+        group = func.coalesce(
+            func.nullif(func.lower(metadata["constellation_group"].as_string()), ""),
+            case(
+                *[
+                    (func.upper(record.name).like(pattern), value)
+                    for pattern, value in SATELLITE_GROUP_PATTERNS
+                ],
+                else_="",
+            ),
+        )
+        entity_id = metadata["norad_cat_id"].as_string()
+    else:
+        group = func.coalesce(
+            metadata["operator"].as_string(), metadata["organization"].as_string(), ""
+        )
+        entity_id = record.source + ":" + record.source_id
+    return select(
+        record.name.label("name"),
+        cast(entity_id, String).label("entity_id"),
+        model.label("model"),
+        group.label("group"),
+        category.label("category"),
+    ).where(
+        (
+            compute_record_filter(record)
+            if entity_type == "compute_center"
+            else record.source.in_(SOURCES[entity_type])
+        ),
+        record.is_current.is_(True),
+        record.deleted_at.is_(None),
+        metadata["anonymous"].as_boolean().is_not(True),
+    )
+
+
+def target_query(entity_type: EntityType):
+    if entity_type in SOURCES:
+        return collected_query(entity_type)
+    if entity_type == "vessel":
+        model = VesselCurrentState
+        identity, name, group, category = model.mmsi, model.name, model.flag, model.vessel_type_name
+        conditions = []
+    elif entity_type == "bgp":
+        model = BGPIncident
+        identity, name, group, category = (
+            model.incident_key,
+            model.title,
+            model.source,
+            model.incident_type,
+        )
+        conditions = []
+    elif entity_type == "bgp_collector":
+        model = BGPCollectorLocation
+        identity, name, group, category = (
+            model.collector_id,
+            model.collector_id,
+            model.operator,
+            literal("bgp_collector"),
+        )
+        conditions = []
+    else:
+        model = EarthInteractable
+        identity, name, group, category = model.id, model.label, model.layer, model.kind
+        conditions = [model.is_deleted.is_(False)]
+    return select(
+        name.label("name"),
+        cast(identity, String).label("entity_id"),
+        literal("").label("model"),
+        func.coalesce(group, "").label("group"),
+        func.coalesce(category, entity_type).label("category"),
+    ).where(*conditions)
+
+
+def category_context(entity_type: EntityType) -> dict[str, str]:
+    return {
+        "entity_type": entity_type,
+        "entity_id": "",
+        "name": "",
+        "model": "",
+        "group": "",
+        "category": entity_type,
+    }
+
+
+async def get_target(db: AsyncSession, entity_type: EntityType, entity_id: str) -> dict[str, str]:
+    if entity_type == "compute_center" and entity_id.startswith("__category__:"):
+        category = entity_id.partition(":")[2]
+        if category in COMPUTE_SITE_TYPES:
+            return {**category_context(entity_type), "category": category}
+    if entity_id == "__category__":
+        return category_context(entity_type)
+    if entity_type in SOURCES and entity_type != "satellite":
+        source, separator, source_id = entity_id.partition(":")
+        if not separator or not source or not source_id:
+            return category_context(entity_type)
+        # Match the indexed identity columns before projecting JSON metadata.
+        statement = collected_query(entity_type).where(
+            CollectedData.source == source, CollectedData.source_id == source_id
+        )
+    else:
+        catalog = target_query(entity_type).subquery()
+        statement = select(catalog).where(catalog.c.entity_id == entity_id)
+    row = (await db.execute(statement.limit(1))).mappings().first()
+    if not row:
+        # Old map snapshots can outlive a catalog refresh. They may use the type
+        # illustration, but must not claim an unverified exact-object identity.
+        return category_context(entity_type)
+    context = {key: str(value or "") for key, value in row.items()}
+    return {**context, "entity_type": entity_type}
+
+
+async def search_targets(
+    db: AsyncSession, entity_type: EntityType, scope: MatchScope, query: str, limit: int
+) -> list[dict[str, str]]:
+    catalog = target_query(entity_type).subquery()
+    value = catalog.c["entity_id" if scope == "object" else scope]
+    label = catalog.c.name if scope == "object" else value
+    statement = select(value.label("value"), label.label("label")).where(
+        value.is_not(None), value != ""
+    )
+    if query:
+        escaped = query.replace("\\", "\\\\").replace("%", "\\%").replace("_", "\\_")
+        statement = statement.where(
+            or_(value.ilike(f"%{escaped}%", escape="\\"), label.ilike(f"%{escaped}%", escape="\\"))
+        )
+    rows = (await db.execute(statement.distinct().order_by(label, value).limit(limit))).mappings()
+    items = [{"value": row["value"], "label": row["label"] or row["value"]} for row in rows]
+    if scope == "category" and not query:
+        known = {item["value"] for item in items}
+        if entity_type == "compute_center":
+            items.extend(
+                {"value": kind, "label": kind} for kind in COMPUTE_SITE_TYPES if kind not in known
+            )
+        items.insert(0, {"value": "*", "label": "*"})
+    return items

backend/app/services/collectors/aisstream.py

@@ -322,6 +322,21 @@ class AISStreamCollector(BaseCollector):
             last_success_at=now if data else None,
             lag_seconds=max((now - latest_observed_at).total_seconds(), 0),
         )
+        if snapshot_id is not None:
+            from app.models.data_snapshot import DataSnapshot
+
+            snapshot = await db.get(DataSnapshot, snapshot_id)
+            if snapshot:
+                snapshot.record_count = records_added
+                snapshot.status = "success"
+                snapshot.completed_at = now
+                snapshot.summary = {
+                    "created": records_added,
+                    "updated": 0,
+                    "unchanged": 0,
+                    "deleted": 0,
+                    "storage": "ais_raw_observations",
+                }
         await db.commit()
         await self.update_progress(records_added, force=True)
         return records_added

backend/app/services/collectors/base.py

@@ -4,42 +4,22 @@ import asyncio
 from abc import ABC, abstractmethod
 from typing import Dict, List, Any, Optional
 from datetime import UTC, datetime
+from time import perf_counter
+from urllib.parse import urlparse
 import httpx
 from sqlalchemy import select, text
 from sqlalchemy.ext.asyncio import AsyncSession
 
 from app.core.collected_data_fields import build_dynamic_metadata, get_record_field
 from app.core.countries import normalize_country
+from app.core.enums import JobStatus, SnapshotStatus
+from app.core.logging import get_logger
 from app.core.time import to_iso8601_utc
 from app.core.websocket.broadcaster import broadcaster
-from app.services.earth_layer_cache import invalidate_earth_layer_cache_for_source
-
-EARTH_UPDATE_LAYER_HINTS: dict[str, list[str]] = {
-    "ris_live_bgp": ["bgp"],
-    "bgpstream_bgp": ["bgp"],
-    "top500_supercomputers": ["computeCenters"],
-    "epoch_ai_gpu": ["computeCenters"],
-    "huggingface_models": ["computeCenters"],
-    "huggingface_datasets": ["computeCenters"],
-    "huggingface_spaces": ["computeCenters"],
-    "telegeography_cables": ["cables"],
-    "telegeography_landing_points": ["cables"],
-    "telegeography_cable_systems": ["cables"],
-    "arcgis_cables": ["cables"],
-    "fao_landing_points": ["cables"],
-    "arcgis_landing_points": ["cables"],
-    "arcgis_cable_landing_relations": ["cables"],
-    "spacetrack_tle": ["satellites"],
-    "celestrak_tle": ["satellites"],
-    "barentswatch_vessels": ["vessels"],
-    "aisstream_vessels": ["vessels"],
-    "news_live_streams": ["media"],
-    "media_news_archive": ["news"],
-}
+from app.services.business_logs import emit_business_log, exception_context
 
 
-def get_earth_update_layers_for_source(source: str) -> list[str]:
-    return EARTH_UPDATE_LAYER_HINTS.get(source, [])
+logger = get_logger(__name__, service="collector")
 
 
 class BaseCollector(ABC):
@@ -51,6 +31,7 @@ class BaseCollector(ABC):
     frequency_hours: int = 4
     data_type: str = "generic"
     fail_on_empty: bool = False
+    preserve_snapshot_on_failure: bool = False
 
     def __init__(self):
         self._current_task = None
@@ -58,6 +39,7 @@ class BaseCollector(ABC):
         self._datasource_id = 1
         self._resolved_url: Optional[str] = None
         self._last_broadcast_progress: Optional[int] = None
+        self._last_save_summary: dict[str, int] = {}
 
     async def resolve_url(self, db: AsyncSession) -> None:
         from app.core.data_sources import get_data_sources_config
@@ -96,29 +78,6 @@ class BaseCollector(ABC):
         )
         self._last_broadcast_progress = rounded_progress
 
-    async def _publish_earth_update(
-        self,
-        *,
-        action: str,
-        records_processed: int,
-        task_id: int | None = None,
-    ) -> None:
-        layers = get_earth_update_layers_for_source(self.name)
-        if not layers:
-            return
-        await broadcaster.broadcast_earth_update(
-            {
-                "action": action,
-                "source": self.name,
-                "data_type": self.data_type,
-                "layers": layers,
-                "datasource_id": getattr(self, "_datasource_id", None),
-                "task_id": task_id,
-                "records_processed": records_processed,
-                "timestamp": to_iso8601_utc(datetime.now(UTC)),
-            }
-        )
-
     async def update_progress(self, records_processed: int, *, commit: bool = False, force: bool = False):
         """Update task progress - call this during data processing"""
         if self._current_task and self._db_session:
@@ -280,7 +239,7 @@ class BaseCollector(ABC):
             snapshot = await db.get(DataSnapshot, snapshot_id)
             if snapshot:
                 parent_snapshot_id = snapshot.parent_snapshot_id
-                snapshot.status = "cancelled"
+                snapshot.status = SnapshotStatus.CANCELLED.value
                 snapshot.is_current = False
                 snapshot.completed_at = datetime.now(UTC)
                 summary = dict(snapshot.summary or {})
@@ -322,19 +281,39 @@ class BaseCollector(ABC):
         from app.models.data_snapshot import DataSnapshot
 
         start_time = datetime.now(UTC)
+        started_at = perf_counter()
         datasource_id = getattr(self, "_datasource_id", 1)
         snapshot_id: Optional[int] = None
 
         if not collector_registry.is_active(self.name):
+            await self._log_collection_event(
+                "collector.run.skipped_disabled",
+                "Collector skipped because it is disabled",
+                level="info",
+                context={"status": "skipped", "reason": "disabled"},
+            )
             return {"status": "skipped", "reason": "Collector is disabled"}
 
-        task = CollectionTask(
-            datasource_id=datasource_id,
-            status="running",
-            phase="queued",
-            started_at=start_time,
-        )
-        db.add(task)
+        task = self._current_task if isinstance(self._current_task, CollectionTask) else None
+        if task is None:
+            task = CollectionTask(
+                datasource_id=datasource_id,
+                source=self.name,
+                task_type="collect",
+                status="running",
+                phase="queued",
+                started_at=start_time,
+            )
+            db.add(task)
+        else:
+            task.datasource_id = datasource_id
+            task.source = task.source or self.name
+            task.task_type = task.task_type or "collect"
+            task.status = JobStatus.RUNNING.value
+            task.phase = "queued"
+            task.started_at = task.started_at or start_time
+            task.completed_at = None
+            task.error_message = None
         await db.commit()
         task_id = task.id
 
@@ -344,25 +323,80 @@ class BaseCollector(ABC):
 
         await self.resolve_url(db)
         await self._publish_task_update(force=True)
+        await self._log_collection_event(
+            "collector.run.started",
+            "Collector run started",
+            context={"status": "running", "task_id": task_id},
+        )
 
         try:
+            phase_started_at = perf_counter()
             await self.set_phase("fetching", message="正在拉取原始数据")
+            await self._log_collection_event(
+                "collector.phase.fetching.start",
+                "Collector fetch phase started",
+                context={"task_id": task_id, "snapshot_id": snapshot_id},
+            )
             raw_data = await self.fetch()
             task.total_records = len(raw_data)
             await db.commit()
             await self._publish_task_update(force=True)
+            await self._log_collection_event(
+                "collector.phase.fetching.success",
+                "Collector fetch phase completed",
+                context={
+                    "task_id": task_id,
+                    "raw_count": len(raw_data),
+                    "duration_ms": self._duration_ms(phase_started_at),
+                },
+            )
 
             if self.fail_on_empty and not raw_data:
                 raise RuntimeError(f"Collector {self.name} returned no data")
 
+            phase_started_at = perf_counter()
             await self.set_phase("transforming", message="正在转换采集数据")
+            await self._log_collection_event(
+                "collector.phase.transforming.start",
+                "Collector transform phase started",
+                context={"task_id": task_id, "raw_count": len(raw_data)},
+            )
             data = self.transform(raw_data)
+            if self.fail_on_empty and not data:
+                raise RuntimeError(f"Collector {self.name} produced no valid records")
+            await self._log_collection_event(
+                "collector.phase.transforming.success",
+                "Collector transform phase completed",
+                context={
+                    "task_id": task_id,
+                    "raw_count": len(raw_data),
+                    "transformed_count": len(data),
+                    "duration_ms": self._duration_ms(phase_started_at),
+                },
+            )
             snapshot_id = await self._create_snapshot(db, task_id, data, start_time)
 
+            phase_started_at = perf_counter()
             await self.set_phase("saving", message="正在保存采集数据")
+            await self._log_collection_event(
+                "collector.phase.saving.start",
+                "Collector save phase started",
+                context={"task_id": task_id, "snapshot_id": snapshot_id, "transformed_count": len(data)},
+            )
             records_count = await self._save_data(db, data, task_id=task_id, snapshot_id=snapshot_id)
+            await self._log_collection_event(
+                "collector.phase.saving.success",
+                "Collector save phase completed",
+                context={
+                    "task_id": task_id,
+                    "snapshot_id": snapshot_id,
+                    "saved_count": records_count,
+                    **self._last_save_summary,
+                    "duration_ms": self._duration_ms(phase_started_at),
+                },
+            )
 
-            task.status = "success"
+            task.status = JobStatus.SUCCESS.value
             task.phase = "completed"
             task.phase_progress = 100.0
             task.phase_message = "采集完成"
@@ -374,10 +408,19 @@ class BaseCollector(ABC):
             task.completed_at = datetime.now(UTC)
             await db.commit()
             await self._publish_task_update(force=True)
-            await self._publish_earth_update(
-                action="collector_completed",
-                records_processed=records_count,
-                task_id=task_id,
+            await self._log_collection_event(
+                "collector.run.completed",
+                "Collector run completed",
+                context={
+                    "status": "success",
+                    "task_id": task_id,
+                    "snapshot_id": snapshot_id,
+                    "raw_count": len(raw_data),
+                    "transformed_count": len(data),
+                    "saved_count": records_count,
+                    **self._last_save_summary,
+                    "duration_ms": self._duration_ms(started_at),
+                },
             )
 
             return {
@@ -388,7 +431,7 @@ class BaseCollector(ABC):
             }
         except asyncio.CancelledError:
             await db.rollback()
-            task.status = "cancelled"
+            task.status = JobStatus.CANCELLED.value
             task.phase = "cancelled"
             task.phase_message = "采集已取消"
             task.error_message = "Collection cancelled by operator and rolled back"
@@ -402,10 +445,25 @@ class BaseCollector(ABC):
                 )
             await db.commit()
             await self._publish_task_update(force=True)
+            await self._log_collection_event(
+                "collector.run.cancelled",
+                "Collector run cancelled",
+                level="warning",
+                context={
+                    "status": "cancelled",
+                    "task_id": task_id,
+                    "snapshot_id": snapshot_id,
+                    "duration_ms": self._duration_ms(started_at),
+                },
+            )
             raise
         except Exception as e:
             await db.rollback()
-            task.status = "failed"
+            if snapshot_id is not None and self.preserve_snapshot_on_failure:
+                await self._rollback_incomplete_run(
+                    db, task_id=task_id, snapshot_id=snapshot_id, reason=str(e)
+                )
+            task.status = JobStatus.FAILED.value
             task.phase = "failed"
             task.phase_message = str(e)
             task.error_message = str(e)
@@ -413,11 +471,27 @@ class BaseCollector(ABC):
             if snapshot_id is not None:
                 snapshot = await db.get(DataSnapshot, snapshot_id)
                 if snapshot:
-                    snapshot.status = "failed"
+                    snapshot.status = SnapshotStatus.FAILED.value
+                    if self.preserve_snapshot_on_failure:
+                        snapshot.is_current = False
                     snapshot.completed_at = datetime.now(UTC)
                     snapshot.summary = {"error": str(e)}
             await db.commit()
             await self._publish_task_update(force=True)
+            await self._log_collection_event(
+                "collector.run.failed",
+                "Collector run failed",
+                level="error",
+                context=exception_context(
+                    e,
+                    {
+                        "status": "failed",
+                        "task_id": task_id,
+                        "snapshot_id": snapshot_id,
+                        "duration_ms": self._duration_ms(started_at),
+                    },
+                ),
+            )
 
             return {
                 "status": "failed",
@@ -438,12 +512,13 @@ class BaseCollector(ABC):
         from app.models.data_snapshot import DataSnapshot
 
         if not data:
+            self._last_save_summary = {"created": 0, "updated": 0, "unchanged": 0, "deleted": 0}
             if snapshot_id is not None:
                 snapshot = await db.get(DataSnapshot, snapshot_id)
                 if snapshot:
                     snapshot.record_count = 0
                     snapshot.summary = {"created": 0, "updated": 0, "unchanged": 0}
-                    snapshot.status = "success"
+                    snapshot.status = SnapshotStatus.SUCCESS.value
                     snapshot.completed_at = datetime.now(UTC)
                     await db.commit()
             return 0
@@ -576,7 +651,7 @@ class BaseCollector(ABC):
             snapshot = await db.get(DataSnapshot, snapshot_id)
             if snapshot:
                 snapshot.record_count = records_added
-                snapshot.status = "success"
+                snapshot.status = SnapshotStatus.SUCCESS.value
                 snapshot.completed_at = datetime.now(UTC)
                 snapshot.summary = {
                     "created": created_count,
@@ -584,12 +659,51 @@ class BaseCollector(ABC):
                     "unchanged": unchanged_count,
                     "deleted": len(deleted_keys),
                 }
+            self._last_save_summary = {
+                "created": created_count,
+                "updated": updated_count,
+                "unchanged": unchanged_count,
+                "deleted": len(deleted_keys),
+            }
+        else:
+            self._last_save_summary = {
+                "created": created_count,
+                "updated": updated_count,
+                "unchanged": unchanged_count,
+                "deleted": 0,
+            }
 
         await db.commit()
-        invalidate_earth_layer_cache_for_source(self.name)
         await self.update_progress(len(data), force=True)
         return records_added
 
+    @staticmethod
+    def _duration_ms(started_at: float) -> int:
+        return int((perf_counter() - started_at) * 1000)
+
+    async def _log_collection_event(
+        self,
+        event: str,
+        message: str,
+        *,
+        level: str = "info",
+        context: Dict[str, Any] | None = None,
+    ) -> None:
+        await emit_business_log(
+            logger,
+            event=event,
+            message=message,
+            category="collector",
+            level=level,
+            service="collector",
+            module=__name__,
+            context={
+                "collector_name": self.name,
+                "datasource_id": getattr(self, "_datasource_id", None),
+                **(context or {}),
+            },
+        )
+
     async def save(self, db: AsyncSession, data: List[Dict[str, Any]]) -> int:
         """Save data to database (legacy method, use _save_data instead)"""
         return await self._save_data(db, data)
@@ -602,10 +716,65 @@ class HTTPCollector(BaseCollector):
     headers: Dict[str, str] = {}
 
     async def fetch(self) -> List[Dict[str, Any]]:
+        started_at = perf_counter()
+        request_host = urlparse(self.base_url).netloc
+        await emit_business_log(
+            logger,
+            event="collector.http.fetch.start",
+            message="Collector HTTP request started",
+            category="collector",
+            service="collector",
+            module=__name__,
+            context={
+                "collector_name": self.name,
+                "datasource_id": getattr(self, "_datasource_id", None),
+                "url_host": request_host,
+            },
+        )
         async with httpx.AsyncClient(timeout=60.0) as client:
-            response = await client.get(self.base_url, headers=self.headers)
-            response.raise_for_status()
-            return self.parse_response(response.json())
+            try:
+                response = await client.get(self.base_url, headers=self.headers)
+                response.raise_for_status()
+                payload = response.json()
+                parsed = self.parse_response(payload)
+                await emit_business_log(
+                    logger,
+                    event="collector.http.fetch.success",
+                    message="Collector HTTP request completed",
+                    category="collector",
+                    service="collector",
+                    module=__name__,
+                    context={
+                        "collector_name": self.name,
+                        "datasource_id": getattr(self, "_datasource_id", None),
+                        "url_host": request_host,
+                        "status_code": response.status_code,
+                        "response_bytes": len(response.content or b""),
+                        "parsed_count": len(parsed),
+                        "duration_ms": BaseCollector._duration_ms(started_at),
+                    },
+                )
+                return parsed
+            except Exception as exc:
+                await emit_business_log(
+                    logger,
+                    event="collector.http.fetch.failed",
+                    message="Collector HTTP request failed",
+                    category="collector",
+                    level="error",
+                    service="collector",
+                    module=__name__,
+                    context=exception_context(
+                        exc,
+                        {
+                            "collector_name": self.name,
+                            "datasource_id": getattr(self, "_datasource_id", None),
+                            "url_host": request_host,
+                            "duration_ms": BaseCollector._duration_ms(started_at),
+                        },
+                    ),
+                )
+                raise
 
     @abstractmethod
     def parse_response(self, response: Dict[str, Any]) -> List[Dict[str, Any]]:

backend/app/services/collectors/celestrak.py

@@ -1,15 +1,45 @@
-"""CelesTrak TLE Collector
+"""CelesTrak TLE Collector.
 
-Collects satellite TLE (Two-Line Element) data from CelesTrak.org.
-Free, no authentication required.
+Collects the full active satellite GP element set from CelesTrak.
 """
 
+import asyncio
 import json
-from typing import Dict, Any, List
+from pathlib import Path
+from time import perf_counter
+from typing import Any, Dict, List
+from urllib.parse import urlencode, urlparse
+
 import httpx
 
+from app.core.logging import get_logger
 from app.core.satellite_tle import build_tle_lines_from_elements
+from app.services.business_logs import emit_business_log, exception_context
 from app.services.collectors.base import BaseCollector
+from app.services.collectors.downloads import DownloadHTTPStatusError, ResumableFileDownloader
+
+logger = get_logger(__name__, service="collector")
+ACTIVE_GROUP = "active"
+FALLBACK_GROUPS = (
+    "starlink",
+    "gps-ops",
+    "galileo",
+    "glo-ops",
+    "beidou",
+    "geo",
+    "iridium-next",
+    "stations",
+    "visual",
+    "weather",
+    "science",
+    "cubesat",
+    "amateur",
+    "last-30-days",
+)
+FETCH_RETRY_ATTEMPTS = 3
+FETCH_RETRY_BASE_DELAY_SECONDS = 0.8
+CELESTRAK_CACHE_TTL_SECONDS = 2 * 60 * 60
+CELESTRAK_NOT_UPDATED_MARKER = "GP data has not updated since your last successful"
 
 
 class CelesTrakTLECollector(BaseCollector):
@@ -18,55 +48,367 @@ class CelesTrakTLECollector(BaseCollector):
     module = "L3"
     frequency_hours = 24
     data_type = "satellite_tle"
+    _downloader = ResumableFileDownloader(
+        cache_namespace="celestrak",
+        default_accept="application/json",
+        preflight_head=False,
+        cache_ttl_seconds=CELESTRAK_CACHE_TTL_SECONDS,
+    )
 
     @property
     def base_url(self) -> str:
         return self._resolved_url or ""
 
+    def _active_url(self) -> str:
+        return self._group_url(ACTIVE_GROUP)
+
+    def _group_url(self, group: str) -> str:
+        if not self.base_url:
+            raise RuntimeError("CelesTrak base URL is not configured")
+        return f"{self.base_url}?{urlencode({'GROUP': group, 'FORMAT': 'json'})}"
+
     async def fetch(self) -> List[Dict[str, Any]]:
-        satellite_groups = [
-            "starlink",
-            "gps-ops",
-            "galileo",
-            "glonass",
-            "beidou",
-            "leo",
-            "geo",
-            "iridium-next",
-        ]
+        url = self._active_url()
+        last_error: Exception | None = None
 
-        all_satellites = []
-
-        async with httpx.AsyncClient(timeout=120.0) as client:
-            for group in satellite_groups:
+        async with httpx.AsyncClient(timeout=180.0, follow_redirects=True) as client:
+            for attempt in range(1, FETCH_RETRY_ATTEMPTS + 1):
+                started_at = perf_counter()
                 try:
-                    url = f"{self.base_url}?GROUP={group}&FORMAT=json"
-                    response = await client.get(url)
+                    await emit_business_log(
+                        logger,
+                        event="collector.celestrak.download.start",
+                        message="CelesTrak active satellite download started",
+                        category="collector",
+                        service="collector",
+                        module=__name__,
+                        context={
+                            "collector_name": self.name,
+                            "datasource_id": getattr(self, "_datasource_id", None),
+                            "group": ACTIVE_GROUP,
+                            "attempt": attempt,
+                            "url_host": urlparse(url).netloc,
+                        },
+                    )
+                    body_path = await self._downloader.download_file(
+                        client,
+                        url,
+                        extension=".json",
+                        accept="application/json",
+                        progress_callback=self._report_download_progress,
+                        validate_existing=self._validate_json_file,
+                    )
+                    data = await self._load_downloaded_payload(body_path, url)
+                    await emit_business_log(
+                        logger,
+                        event="collector.celestrak.download.success",
+                        message="CelesTrak active satellite download completed",
+                        category="collector",
+                        service="collector",
+                        module=__name__,
+                        context={
+                            "collector_name": self.name,
+                            "datasource_id": getattr(self, "_datasource_id", None),
+                            "group": ACTIVE_GROUP,
+                            "attempt": attempt,
+                            "record_count": len(data),
+                            "duration_ms": self._duration_ms(started_at),
+                        },
+                    )
+                    return data
+                except DownloadHTTPStatusError as exc:
+                    if self._is_not_updated_response(exc):
+                        cached_path = self._downloader.get_cached_file(
+                            url,
+                            ".json",
+                            validate_existing=self._validate_json_file,
+                        )
+                        if cached_path is not None:
+                            data = await self._load_downloaded_payload(cached_path, url)
+                            await emit_business_log(
+                                logger,
+                                event="collector.celestrak.download.cached_not_updated",
+                                message="CelesTrak active satellite data has not changed; using cached download",
+                                category="collector",
+                                level="warning",
+                                service="collector",
+                                module=__name__,
+                                context={
+                                    "collector_name": self.name,
+                                    "datasource_id": getattr(self, "_datasource_id", None),
+                                    "group": ACTIVE_GROUP,
+                                    "attempt": attempt,
+                                    "record_count": len(data),
+                                    "duration_ms": self._duration_ms(started_at),
+                                },
+                            )
+                            return data
+                        await emit_business_log(
+                            logger,
+                            event="collector.celestrak.download.not_updated_no_cache",
+                            message="CelesTrak active satellite data has not changed; trying fallback groups",
+                            category="collector",
+                            level="warning",
+                            service="collector",
+                            module=__name__,
+                            context=exception_context(
+                                exc,
+                                {
+                                    "collector_name": self.name,
+                                    "datasource_id": getattr(self, "_datasource_id", None),
+                                    "group": ACTIVE_GROUP,
+                                    "attempt": attempt,
+                                    "duration_ms": self._duration_ms(started_at),
+                                },
+                            ),
+                        )
+                        return await self._fetch_fallback_groups(client, active_error=exc)
+                    raise
+                except Exception as exc:
+                    last_error = exc
+                    is_final_attempt = attempt >= FETCH_RETRY_ATTEMPTS
+                    await emit_business_log(
+                        logger,
+                        event=(
+                            "collector.celestrak.download.failed"
+                            if is_final_attempt
+                            else "collector.celestrak.download.retry"
+                        ),
+                        message=(
+                            "CelesTrak active satellite download failed"
+                            if is_final_attempt
+                            else "CelesTrak active satellite download will retry"
+                        ),
+                        category="collector",
+                        level="error" if is_final_attempt else "warning",
+                        service="collector",
+                        module=__name__,
+                        context=exception_context(
+                            exc,
+                            {
+                                "collector_name": self.name,
+                                "datasource_id": getattr(self, "_datasource_id", None),
+                                "group": ACTIVE_GROUP,
+                                "attempt": attempt,
+                                "duration_ms": self._duration_ms(started_at),
+                            },
+                        ),
+                    )
+                    if not is_final_attempt:
+                        await asyncio.sleep(FETCH_RETRY_BASE_DELAY_SECONDS * attempt)
 
-                    if response.status_code == 200:
-                        data = response.json()
-                        if isinstance(data, list):
-                            for item in data:
-                                if isinstance(item, dict):
-                                    item["_celestrak_group"] = group
-                            all_satellites.extend(data)
-                            print(f"CelesTrak: Fetched {len(data)} satellites from group '{group}'")
-                except Exception as e:
-                    print(f"CelesTrak: Error fetching group '{group}': {e}")
+        raise RuntimeError(
+            f"CelesTrak active satellite download failed after retries: {last_error}"
+        )
 
-        if not all_satellites:
-            return self._get_sample_data()
+    async def _fetch_fallback_groups(
+        self,
+        client: httpx.AsyncClient,
+        *,
+        active_error: DownloadHTTPStatusError,
+    ) -> List[Dict[str, Any]]:
+        started_at = perf_counter()
+        records_by_norad: dict[str, Dict[str, Any]] = {}
+        group_counts: dict[str, int] = {}
 
-        print(f"CelesTrak: Total satellites fetched: {len(all_satellites)}")
+        await emit_business_log(
+            logger,
+            event="collector.celestrak.fallback_groups.start",
+            message="CelesTrak fallback group download started",
+            category="collector",
+            level="warning",
+            service="collector",
+            module=__name__,
+            context={
+                "collector_name": self.name,
+                "datasource_id": getattr(self, "_datasource_id", None),
+                "groups": list(FALLBACK_GROUPS),
+                "reason": "active_not_updated_without_cache",
+            },
+        )
 
-        # Return raw data - base.run() will call transform()
-        return all_satellites
+        try:
+            for group in FALLBACK_GROUPS:
+                group_url = self._group_url(group)
+                try:
+                    body_path = await self._downloader.download_file(
+                        client,
+                        group_url,
+                        extension=".json",
+                        accept="application/json",
+                        validate_existing=self._validate_json_file,
+                    )
+                except DownloadHTTPStatusError as exc:
+                    if not self._is_not_updated_response(exc):
+                        raise RuntimeError(
+                            f"CelesTrak fallback group '{group}' download failed: {exc}"
+                        ) from exc
+                    body_path = self._downloader.get_cached_file(
+                        group_url,
+                        ".json",
+                        validate_existing=self._validate_json_file,
+                    )
+                    if body_path is None:
+                        raise RuntimeError(
+                            f"CelesTrak fallback group '{group}' has not updated and no local cached copy is available"
+                        ) from exc
+
+                group_records = await self._load_downloaded_payload(
+                    body_path,
+                    group_url,
+                    query_group=group,
+                    constellation_group=group,
+                )
+                group_counts[group] = len(group_records)
+                for item in group_records:
+                    norad_cat_id = item.get("NORAD_CAT_ID")
+                    if norad_cat_id is None:
+                        continue
+                    records_by_norad.setdefault(str(norad_cat_id), item)
+        except Exception as exc:
+            await emit_business_log(
+                logger,
+                event="collector.celestrak.fallback_groups.failed",
+                message="CelesTrak fallback group download failed",
+                category="collector",
+                level="error",
+                service="collector",
+                module=__name__,
+                context=exception_context(
+                    exc,
+                    {
+                        "collector_name": self.name,
+                        "datasource_id": getattr(self, "_datasource_id", None),
+                        "groups": list(FALLBACK_GROUPS),
+                        "completed_groups": list(group_counts),
+                        "duration_ms": self._duration_ms(started_at),
+                    },
+                ),
+            )
+            raise RuntimeError(
+                "CelesTrak active data has not updated since this network's last successful download, "
+                "no active cache is available, and fallback group mode failed. Wait until CelesTrak "
+                "publishes the next GP update, restore the Planet download cache, or use Space-Track."
+            ) from active_error
+
+        records = list(records_by_norad.values())
+        if not records:
+            raise RuntimeError("CelesTrak fallback group mode produced no satellite records")
+
+        await emit_business_log(
+            logger,
+            event="collector.celestrak.fallback_groups.success",
+            message="CelesTrak fallback group download completed",
+            category="collector",
+            level="warning",
+            service="collector",
+            module=__name__,
+            context={
+                "collector_name": self.name,
+                "datasource_id": getattr(self, "_datasource_id", None),
+                "groups": list(FALLBACK_GROUPS),
+                "group_counts": group_counts,
+                "record_count": len(records),
+                "duration_ms": self._duration_ms(started_at),
+            },
+        )
+        return records
+
+    async def _load_downloaded_payload(
+        self,
+        body_path: Path,
+        url: str,
+        *,
+        query_group: str = ACTIVE_GROUP,
+        constellation_group: str | None = None,
+    ) -> List[Dict[str, Any]]:
+        try:
+            data = self._load_active_payload(body_path)
+        except RuntimeError as exc:
+            await self._log_parse_failure(exc)
+            raise
+        for item in data:
+            item["_celestrak_query_group"] = query_group
+            item["_celestrak_source_url"] = url
+            if constellation_group:
+                item["_celestrak_group"] = constellation_group
+        return data
+
+    @staticmethod
+    def _is_not_updated_response(exc: DownloadHTTPStatusError) -> bool:
+        return exc.status_code == 403 and CELESTRAK_NOT_UPDATED_MARKER in exc.body
+
+    @staticmethod
+    def _duration_ms(started_at: float) -> int:
+        return int((perf_counter() - started_at) * 1000)
+
+    async def _report_download_progress(self, downloaded: int, total: int | None) -> None:
+        if total and total > 0:
+            await self.update_phase_progress(
+                current=min(downloaded, total),
+                total=total,
+                unit="bytes",
+                message=f"正在下载 CelesTrak active 卫星数据 {downloaded}/{total} bytes",
+                commit=True,
+            )
+
+    @classmethod
+    def _validate_json_file(cls, path: Path) -> bool:
+        try:
+            cls._load_active_payload(path)
+        except RuntimeError:
+            return False
+        return True
+
+    async def _log_parse_failure(self, exc: Exception) -> None:
+        await emit_business_log(
+            logger,
+            event="collector.celestrak.parse.failed",
+            message="CelesTrak active satellite JSON parsing failed",
+            category="collector",
+            level="error",
+            service="collector",
+            module=__name__,
+            context=exception_context(
+                exc,
+                {
+                    "collector_name": self.name,
+                    "datasource_id": getattr(self, "_datasource_id", None),
+                    "group": ACTIVE_GROUP,
+                },
+            ),
+        )
+
+    @staticmethod
+    def _load_active_payload(path: Path) -> List[Dict[str, Any]]:
+        try:
+            raw = json.loads(path.read_text(encoding="utf-8"))
+        except (OSError, json.JSONDecodeError, UnicodeDecodeError) as exc:
+            raise RuntimeError(f"CelesTrak active payload is not valid JSON: {exc}") from exc
+        if not isinstance(raw, list):
+            raise RuntimeError("CelesTrak active payload is not a JSON array")
+
+        records: List[Dict[str, Any]] = []
+        invalid_count = 0
+        for item in raw:
+            if isinstance(item, dict) and item.get("NORAD_CAT_ID") is not None:
+                records.append(item)
+            else:
+                invalid_count += 1
+        if invalid_count:
+            raise RuntimeError(
+                f"CelesTrak active payload contains {invalid_count} invalid record(s)"
+            )
+        if not records:
+            raise RuntimeError("CelesTrak active payload contains no satellite records")
+        return records
 
     def transform(self, raw_data: List[Dict[str, Any]]) -> List[Dict[str, Any]]:
         transformed = []
         for item in raw_data:
+            norad_cat_id = item.get("NORAD_CAT_ID")
             tle_line1, tle_line2 = build_tle_lines_from_elements(
-                norad_cat_id=item.get("NORAD_CAT_ID"),
+                norad_cat_id=norad_cat_id,
                 epoch=item.get("EPOCH"),
                 inclination=item.get("INCLINATION"),
                 raan=item.get("RA_OF_ASC_NODE"),
@@ -75,14 +417,18 @@ class CelesTrakTLECollector(BaseCollector):
                 mean_anomaly=item.get("MEAN_ANOMALY"),
                 mean_motion=item.get("MEAN_MOTION"),
             )
+            constellation_group = self._infer_constellation_group(item)
 
             transformed.append(
                 {
+                    "source_id": str(norad_cat_id),
                     "name": item.get("OBJECT_NAME", "Unknown"),
                     "reference_date": item.get("EPOCH", ""),
                     "metadata": {
-                        "constellation_group": item.get("_celestrak_group"),
-                        "norad_cat_id": item.get("NORAD_CAT_ID"),
+                        "constellation_group": constellation_group,
+                        "celestrak_query_group": item.get("_celestrak_query_group") or ACTIVE_GROUP,
+                        "celestrak_source_url": item.get("_celestrak_source_url"),
+                        "norad_cat_id": norad_cat_id,
                         "international_designator": item.get("OBJECT_ID"),
                         "epoch": item.get("EPOCH"),
                         "mean_motion": item.get("MEAN_MOTION"),
@@ -105,6 +451,19 @@ class CelesTrakTLECollector(BaseCollector):
             )
         return transformed
 
+    @staticmethod
+    def _infer_constellation_group(item: Dict[str, Any]) -> str | None:
+        explicit_group = str(item.get("_celestrak_group") or "").strip().lower()
+        if explicit_group and explicit_group != ACTIVE_GROUP:
+            return explicit_group
+
+        name = str(item.get("OBJECT_NAME") or "").strip().upper()
+        if name.startswith("STARLINK"):
+            return "starlink"
+        if name.startswith("IRIDIUM"):
+            return "iridium-next"
+        return None
+
     def _get_sample_data(self) -> List[Dict[str, Any]]:
         return [
             {

backend/app/services/collectors/compute_base.py

@@ -0,0 +1,82 @@
+"""Quality gates shared by the two compute sources."""
+
+from datetime import datetime
+
+from sqlalchemy import select
+from sqlalchemy.ext.asyncio import AsyncSession
+
+from app.core.countries import normalize_country
+from app.models.compute_center_location import ComputeCenterLocationRecord
+from app.models.data_snapshot import DataSnapshot
+from app.services.collectors.base import BaseCollector
+
+
+class ComputeCollector(BaseCollector):
+    fail_on_empty = True
+    preserve_snapshot_on_failure = True
+
+    async def _create_snapshot(
+        self, db: AsyncSession, task_id: int, data: list[dict], started_at: datetime
+    ) -> int:
+        previous_count = await db.scalar(
+            select(DataSnapshot.record_count)
+            .where(
+                DataSnapshot.source == self.name,
+                DataSnapshot.is_current.is_(True),
+                DataSnapshot.status == "success",
+            )
+            .order_by(DataSnapshot.id.desc())
+            .limit(1)
+        )
+        if previous_count and len(data) < previous_count * 0.6:
+            raise ValueError("Compute source lost over 40% of rows; previous snapshot retained")
+        return await super()._create_snapshot(db, task_id, data, started_at)
+
+    async def _save_data(self, db: AsyncSession, data: list[dict], **kwargs) -> int:
+        if self.name == "top500":
+            await self._copy_verified_locations(db, data)
+        return await super()._save_data(db, data, **kwargs)
+
+    async def _copy_verified_locations(self, db: AsyncSession, data: list[dict]) -> None:
+        """Keep historical location keys; copy only unambiguous exact identities.
+
+        Never migrate by rank: the machine occupying rank 1 changes over time.
+        """
+        names = [item["name"] for item in data]
+        locations = (
+            (
+                await db.execute(
+                    select(ComputeCenterLocationRecord).where(
+                        ComputeCenterLocationRecord.source == self.name,
+                        ComputeCenterLocationRecord.name.in_(names),
+                    )
+                )
+            )
+            .scalars()
+            .all()
+        )
+        existing = {location.source_id for location in locations}
+        for item in data:
+            if item["source_id"] in existing:
+                continue
+            country = normalize_country(item["country"])
+            if not country:
+                continue
+            matches = [
+                location
+                for location in locations
+                if location.name == item["name"]
+                and normalize_country(location.country) == country
+                and (not location.site or location.site == item.get("metadata", {}).get("site"))
+                and not location.source_id.startswith("top500_system_")
+            ]
+            if len(matches) != 1:
+                continue
+            old = matches[0]
+            values = {
+                column.name: getattr(old, column.name)
+                for column in old.__table__.columns
+                if column.name not in {"id", "source_id", "created_at", "updated_at"}
+            }
+            db.add(ComputeCenterLocationRecord(**values, source_id=item["source_id"]))
+            existing.add(item["source_id"])

backend/app/services/collectors/downloads.py

@@ -4,7 +4,7 @@ from __future__ import annotations
 
 import hashlib
 import json
-import tempfile
+import os
 import time
 from datetime import UTC, datetime
 from pathlib import Path
@@ -12,11 +12,35 @@ from typing import Any, Awaitable, Callable
 
 import httpx
 
-
 ProgressCallback = Callable[[int, int | None], Awaitable[None]]
 ValidateCallback = Callable[[Path], bool]
 
 
+class DownloadHTTPStatusError(RuntimeError):
+    """HTTP status error that keeps the upstream response body for caller-specific handling."""
+
+    def __init__(self, *, url: str, status_code: int, body: str) -> None:
+        self.url = url
+        self.status_code = status_code
+        self.body = body
+        preview = body.strip().replace("\r", " ").replace("\n", " ")[:240]
+        suffix = f": {preview}" if preview else ""
+        super().__init__(f"HTTP {status_code} while downloading {url}{suffix}")
+
+
+def default_download_cache_root() -> Path:
+    configured = os.getenv("PLANET_DOWNLOAD_CACHE_DIR")
+    if configured:
+        return Path(configured).expanduser()
+    planet_cache = os.getenv("PLANET_CACHE_DIR")
+    if planet_cache:
+        return Path(planet_cache).expanduser() / "downloads"
+    xdg_cache = os.getenv("XDG_CACHE_HOME")
+    if xdg_cache:
+        return Path(xdg_cache).expanduser() / "planet" / "downloads"
+    return Path.home() / ".cache" / "planet" / "downloads"
+
+
 class ResumableFileDownloader:
     """Download files with cache validators and byte-range resume support."""
 
@@ -26,10 +50,15 @@ class ResumableFileDownloader:
         cache_namespace: str,
         user_agent: str = "Planet-Intelligence-System/1.0 (Python/collector)",
         default_accept: str = "*/*",
+        cache_root: Path | None = None,
+        preflight_head: bool = True,
+        cache_ttl_seconds: float = 0,
     ) -> None:
-        self._cache_dir = Path(tempfile.gettempdir()) / "planet-download-cache" / cache_namespace
+        self._cache_dir = (cache_root or default_download_cache_root()) / cache_namespace
         self._user_agent = user_agent
         self._default_accept = default_accept
+        self._preflight_head = preflight_head
+        self._cache_ttl_seconds = cache_ttl_seconds
 
     @staticmethod
     def _cache_key(url: str) -> str:
@@ -43,13 +72,33 @@ class ResumableFileDownloader:
         meta_path = self._cache_dir / f"{key}.meta.json"
         return final_path, part_path, meta_path
 
+    def cached_file_path(self, url: str, extension: str) -> Path:
+        final_path, _, _ = self._cache_paths(url, extension)
+        return final_path
+
+    def get_cached_file(
+        self,
+        url: str,
+        extension: str,
+        *,
+        validate_existing: ValidateCallback | None = None,
+    ) -> Path | None:
+        final_path = self.cached_file_path(url, extension)
+        if not final_path.exists():
+            return None
+        if validate_existing and not validate_existing(final_path):
+            final_path.unlink(missing_ok=True)
+            return None
+        return final_path
+
     @staticmethod
     def _load_meta(meta_path: Path) -> dict[str, Any]:
         if not meta_path.exists():
             return {}
         try:
-            return json.loads(meta_path.read_text(encoding="utf-8"))
-        except (json.JSONDecodeError, OSError):
+            data = json.loads(meta_path.read_text(encoding="utf-8"))
+            return data if isinstance(data, dict) else {}
+        except json.JSONDecodeError, OSError:
             return {}
 
     @staticmethod
@@ -66,20 +115,25 @@ class ResumableFileDownloader:
             return last_modified == str(meta.get("last_modified") or "").strip()
         return True
 
+    @staticmethod
+    def _response_info(response: httpx.Response) -> dict[str, Any]:
+        # httpx yields decoded bytes; encoded Content-Length cannot validate them.
+        raw_length = response.headers.get("content-length")
+        encoded = response.headers.get("content-encoding", "identity") != "identity"
+        return {
+            "etag": response.headers.get("etag"),
+            "last_modified": response.headers.get("last-modified"),
+            "content_length": int(raw_length) if raw_length and not encoded else None,
+            "accept_ranges": (response.headers.get("accept-ranges") or "").lower(),
+        }
+
     async def fetch_remote_info(self, client: httpx.AsyncClient, url: str) -> dict[str, Any]:
         try:
             response = await client.head(url)
             if response.status_code >= 400:
                 return {}
-            content_length_raw = response.headers.get("content-length")
-            content_length = int(content_length_raw) if content_length_raw else None
-            return {
-                "etag": response.headers.get("etag"),
-                "last_modified": response.headers.get("last-modified"),
-                "content_length": content_length,
-                "accept_ranges": (response.headers.get("accept-ranges") or "").lower(),
-            }
-        except (httpx.HTTPError, ValueError):
+            return self._response_info(response)
+        except httpx.HTTPError, ValueError:
             return {}
 
     async def download_file(
@@ -94,14 +148,24 @@ class ResumableFileDownloader:
     ) -> Path:
         self._cache_dir.mkdir(parents=True, exist_ok=True)
         final_path, part_path, meta_path = self._cache_paths(url, extension)
+        cached_path = self.get_cached_file(url, extension, validate_existing=validate_existing)
+        if cached_path and 0 <= time.time() - cached_path.stat().st_mtime < self._cache_ttl_seconds:
+            size = cached_path.stat().st_size
+            if progress_callback:
+                await progress_callback(size, size)
+            return cached_path
         meta = self._load_meta(meta_path)
-        remote = await self.fetch_remote_info(client, url)
+        remote = await self.fetch_remote_info(client, url) if self._preflight_head else {}
         expected_size = remote.get("content_length")
 
         if final_path.exists():
             local_size = final_path.stat().st_size
             size_match = expected_size is None or local_size == expected_size
-            if self._validators_match(meta, remote) and size_match:
+            if (
+                (remote.get("etag") or remote.get("last_modified"))
+                and self._validators_match(meta, remote)
+                and size_match
+            ):
                 if validate_existing and not validate_existing(final_path):
                     final_path.unlink(missing_ok=True)
                 else:
@@ -136,23 +200,33 @@ class ResumableFileDownloader:
                 headers["If-Range"] = str(remote.get("last_modified"))
 
         async with client.stream("GET", url, headers=headers) as response:
-            if response.status_code == 304 and final_path.exists():
-                if progress_callback and expected_size and expected_size > 0:
-                    await progress_callback(expected_size, expected_size)
+            if response.status_code == 304:
+                if cached_path is None:
+                    raise RuntimeError(f"Server returned 304 without a valid local cache for {url}")
+                final_path.touch()
+                size = final_path.stat().st_size
+                if progress_callback:
+                    await progress_callback(size, size)
                 return final_path
-            response.raise_for_status()
+            if response.status_code >= 400:
+                body = (await response.aread()).decode("utf-8", errors="replace")
+                raise DownloadHTTPStatusError(url=url, status_code=response.status_code, body=body)
 
             if response.status_code == 206 and resume_from > 0:
                 mode = "ab"
             else:
                 mode = "wb"
                 resume_from = 0
+                remote = self._response_info(response)
+                expected_size = remote.get("content_length")
 
             downloaded = resume_from
             last_emit_bytes = 0
             last_emit_time = time.monotonic()
             min_emit_bytes = (
-                max(expected_size // 150, 512 * 1024) if expected_size and expected_size > 0 else 1024 * 1024
+                max(expected_size // 150, 512 * 1024)
+                if expected_size and expected_size > 0
+                else 1024 * 1024
             )
 
             with part_path.open(mode) as f:
@@ -183,6 +257,10 @@ class ResumableFileDownloader:
                 f"Resumable download incomplete for {url}: expected={expected_size}, got={final_size}"
             )
 
+        # Never replace the last good payload with an HTML error or invalid JSON.
+        if validate_existing and not validate_existing(part_path):
+            part_path.unlink(missing_ok=True)
+            raise RuntimeError(f"Downloaded file validation failed for {url}")
         part_path.replace(final_path)
         self._save_meta(
             meta_path,
@@ -195,9 +273,6 @@ class ResumableFileDownloader:
             },
         )
 
-        if validate_existing and not validate_existing(final_path):
-            raise RuntimeError(f"Downloaded file validation failed for {url}")
-
         if progress_callback and expected_size and expected_size > 0:
             await progress_callback(expected_size, expected_size)
 

backend/app/services/collectors/epoch_ai.py

@@ -1,119 +1,135 @@
-"""Epoch AI GPU Clusters Collector
-
-Collects data from Epoch AI GPU clusters tracking.
-https://epoch.ai/data/gpu-clusters
-"""
+"""Epoch's official CSV; preserve precision, source dates and anonymous records."""
 
+import csv
+import hashlib
+import io
+import json
+import math
 import re
-from typing import Dict, Any, List
-from datetime import UTC, datetime
-from bs4 import BeautifulSoup
+from urllib.parse import urlparse
+
 import httpx
 
-from app.services.collectors.base import BaseCollector
+from app.core.countries import normalize_country
+from app.services.collectors.compute_base import ComputeCollector
+from app.services.epoch_location import assess_epoch_location
+
+EPOCH_CSV_URL = "https://epoch.ai/data/gpu_clusters.csv"
 
 
-
-class EpochAIGPUCollector(BaseCollector):
+class EpochAIGPUCollector(ComputeCollector):
     name = "epoch_ai_gpu"
     priority = "P0"
     module = "L1"
     frequency_hours = 6
     data_type = "gpu_cluster"
 
-    async def fetch(self) -> List[Dict[str, Any]]:
-        """Fetch Epoch AI GPU clusters data from webpage"""
-        url = self._resolved_url or ""
-
-        async with httpx.AsyncClient(timeout=60.0) as client:
+    async def fetch(self) -> list[dict]:
+        configured = self._resolved_url or EPOCH_CSV_URL
+        parsed = urlparse(configured)
+        url = (
+            EPOCH_CSV_URL
+            if parsed.hostname == "epoch.ai" and not parsed.path.endswith(".csv")
+            else configured
+        )
+        async with httpx.AsyncClient(timeout=60, follow_redirects=True) as client:
             response = await client.get(url)
             response.raise_for_status()
-            return self.parse_response(response.text)
+        return self.parse_response(response.text)
 
-    def parse_response(self, html: str) -> List[Dict[str, Any]]:
-        """Parse Epoch AI webpage to extract GPU cluster data"""
-        data = []
-        soup = BeautifulSoup(html, "html.parser")
+    def parse_response(self, text: str) -> list[dict]:
+        reader = csv.DictReader(io.StringIO(text.lstrip("\ufeff")))
+        required = {"Name", "Country", "Status", "First Operational Date", "16-bit OP/s (log)"}
+        if not required.issubset(reader.fieldnames or []):
+            raise ValueError(
+                "Epoch CSV schema changed or HTML returned; previous snapshot retained"
+            )
+        entries = []
+        seen = set()
+        anonymous_counts = {}
+        for row in reader:
+            name = (row.get("Name") or "").strip()
+            country = normalize_country(row.get("Country"))
+            if not name:
+                raise ValueError("Epoch row has missing identity")
+            source_id = f"epoch_{re.sub(r'[^a-zA-Z0-9]', '_', name.lower())}"
+            anonymous = "anonym" in name.lower()
+            if anonymous:
+                # Public anonymous rows have no facility identity. Preserve
+                # multiplicity within snapshots, without guessing names.
+                digest = hashlib.sha256(json.dumps(row, sort_keys=True).encode()).hexdigest()[:24]
+                anonymous_counts[digest] = anonymous_counts.get(digest, 0) + 1
+                source_id = f"epoch_anonymous_{digest}_{anonymous_counts[digest]}"
+            if source_id in seen:
+                raise ValueError("Epoch identity collision; previous snapshot retained")
+            seen.add(source_id)
+            raw_date = row.get("First Operational Date", "")
+            reference_date = raw_date if re.fullmatch(r"\d{4}-\d{2}-\d{2}", raw_date) else None
+            value = None
+            raw_value = row.get("16-bit OP/s (log)", "")
+            if raw_value:
+                exponent = float(raw_value)
+                if not math.isfinite(exponent) or not 0 <= exponent <= 30:
+                    raise ValueError("Epoch performance exponent out of bounds")
+                value = 10**exponent
+            anonymous = anonymous or (country == "中国" and not row.get("Location"))
+            location = row.get("Location")
+            location_assessment = assess_epoch_location(
+                location,
+                country=country,
+                raw_country=row.get("Country"),
+            )
+            entries.append(
+                {
+                    "source_id": source_id,
+                    "name": name,
+                    "country": country,
+                    "value": value,
+                    "unit": "FLOP/s",
+                    "reference_date": reference_date,
+                    "latitude": None if anonymous else self.coordinate(row.get("latitude"), 90),
+                    "longitude": None if anonymous else self.coordinate(row.get("longitude"), 180),
+                    "metadata": {
+                        "compute_contract": "v1",
+                        "precision": "FP16",
+                        "measurement": "theoretical",
+                        "source_url": EPOCH_CSV_URL,
+                        "raw_reference_date": raw_date,
+                        "status": row.get("Status"),
+                        "certainty": row.get("Certainty"),
+                        "operator": row.get("Owner"),
+                        "gpu_type": row.get("Chip type (primary)"),
+                        "gpu_count": row.get("Chip quantity (primary)"),
+                        "location": location,
+                        "location_precision": location_assessment.precision,
+                        "location_precision_reason": location_assessment.reason,
+                        "raw_country": row.get("Country"),
+                        "raw_performance": raw_value,
+                        "performance_encoding": "log10",
+                        "builds_upon": row.get("Builds Upon"),
+                        "superseded_by": row.get("Superseded by"),
+                        "possible_duplicate": row.get("Possible Duplicate", "").lower() == "true",
+                        "include_in_standard_analysis": row.get(
+                            "Include in Standard Analysis", ""
+                        ).lower()
+                        == "true",
+                        "rounded": country == "中国",
+                        "anonymous": anonymous,
+                        "identity_scope": "snapshot_record" if anonymous else "named_cluster",
+                        "country_unresolved": country is None,
+                        "sources": [
+                            row[f"Source {i}"] for i in range(1, 6) if row.get(f"Source {i}")
+                        ],
+                    },
+                }
+            )
+        return entries
 
-        # Try to find data table on the page
-        tables = soup.find_all("table")
-        for table in tables:
-            rows = table.find_all("tr")
-            for row in rows[1:]:  # Skip header
-                cells = row.find_all(["td", "th"])
-                if len(cells) >= 5:
-                    try:
-                        cluster_name = cells[0].get_text(strip=True)
-                        if not cluster_name or cluster_name in ["Cluster", "System", "Name"]:
-                            continue
-
-                        location_cell = cells[1].get_text(strip=True) if len(cells) > 1 else ""
-                        country, city = self._parse_location(location_cell)
-
-                        perf_cell = cells[2].get_text(strip=True) if len(cells) > 2 else ""
-
-                        entry = {
-                            "source_id": f"epoch_{re.sub(r'[^a-zA-Z0-9]', '_', cluster_name.lower())}",
-                            "name": cluster_name,
-                            "country": country,
-                            "city": city,
-                            "latitude": "",
-                            "longitude": "",
-                            "value": self._parse_performance(perf_cell),
-                            "unit": "TFlop/s",
-                            "metadata": {
-                                "raw_data": perf_cell,
-                            },
-                            "reference_date": datetime.now(UTC).strftime("%Y-%m-%d"),
-                        }
-                        data.append(entry)
-                    except (ValueError, IndexError, AttributeError):
-                        continue
-
-        # If no table found, return sample data
-        if not data:
-            data = self._get_sample_data()
-
-        return data
-
-    def _parse_location(self, location: str) -> tuple:
-        """Parse location string into country and city"""
-        if not location:
-            return "", ""
-        if "," in location:
-            parts = location.rsplit(",", 1)
-            city = parts[0].strip()
-            country = parts[1].strip() if len(parts) > 1 else ""
-            return country, city
-        return location, ""
-
-    def _parse_performance(self, perf: str) -> str:
-        """Parse performance string to extract value"""
-        if not perf:
-            return "0"
-        match = re.search(r"([\d,.]+)\s*(TFlop/s|PFlop/s|GFlop/s)?", perf, re.I)
-        if match:
-            return match.group(1).replace(",", "")
-        match = re.search(r"([\d,.]+)", perf)
-        if match:
-            return match.group(1).replace(",", "")
-        return "0"
-
-    def _get_sample_data(self) -> List[Dict[str, Any]]:
-        """Return sample data for testing when scraping fails"""
-        return [
-            {
-                "source_id": "epoch_sample_1",
-                "name": "Sample GPU Cluster",
-                "country": "United States",
-                "city": "San Francisco, CA",
-                "latitude": "",
-                "longitude": "",
-                "value": "1000",
-                "unit": "TFlop/s",
-                "metadata": {
-                    "note": "Sample data - Epoch AI page structure may vary",
-                },
-                "reference_date": datetime.now(UTC).strftime("%Y-%m-%d"),
-            },
-        ]
+    @staticmethod
+    def coordinate(value: str | None, bound: int) -> float | None:
+        if not value:
+            return None
+        number = float(value)
+        if not math.isfinite(number) or abs(number) > bound:
+            raise ValueError("Epoch coordinate out of bounds")
+        return number

backend/app/services/collectors/news_live_streams.py

@@ -35,7 +35,7 @@ class NewsLiveStreamsCollector(BaseCollector):
     DEFAULT_IPTV_ORG_LOGOS_URL = "https://iptv-org.github.io/api/logos.json"
     DEFAULT_IPTV_ORG_NEWS_CATEGORIES = ("news", "business", "weather")
     DEFAULT_IPTV_ORG_EXCLUDE_CATEGORIES = ("music", "sports", "kids", "entertainment")
-    DEFAULT_IPTV_ORG_MAX_SOURCES = 120
+    DEFAULT_IPTV_ORG_MAX_SOURCES = 0  # Zero keeps the complete matching channel catalog.
 
     async def fetch(self) -> list[dict[str, Any]]:
         request_url = (self._resolved_url or "").strip()
@@ -445,7 +445,7 @@ class NewsLiveStreamsCollector(BaseCollector):
                     "reference_date": datetime.now(UTC).isoformat(),
                 }
             )
-            if len(normalized) >= max_sources:
+            if max_sources > 0 and len(normalized) >= max_sources:
                 break
 
         return normalized

backend/app/services/collectors/peeringdb.py

@@ -11,17 +11,20 @@ To get higher limits, set PEERINGDB_API_KEY environment variable.
 """
 
 import asyncio
-import os
-from typing import Dict, Any, List
 from datetime import UTC, datetime
+import os
+from typing import Any, Dict, List
+from urllib.parse import urlencode
 
 import httpx
-from urllib.parse import urlencode
+
+from app.core.logging import get_logger
 from app.services.collectors.base import HTTPCollector
 
 
 # PeeringDB API key - read from environment variable
 PEERINGDB_API_KEY = os.environ.get("PEERINGDB_API_KEY", "")
+logger = get_logger(__name__, service="collector")
 
 
 class PeeringDBIXPCollector(HTTPCollector):
@@ -39,6 +42,7 @@ class PeeringDBIXPCollector(HTTPCollector):
             "User-Agent": "Planet-Intelligence-System/1.0 (Python/collector)",
             "Accept": "application/json",
         }
+
     @property
     def request_url(self) -> str:
         base = self._resolved_url or self.base_url
@@ -61,7 +65,11 @@ class PeeringDBIXPCollector(HTTPCollector):
                     if response.status_code == 429:
                         # Rate limited - wait and retry with exponential backoff
                         delay = base_delay * (2**attempt)
-                        print(f"PeeringDB rate limited, waiting {delay}s before retry...")
+                        logger.warning_event(
+                            "PeeringDB rate limited; retrying after delay",
+                            event="collector.peeringdb.rate_limited",
+                            context={"delay_seconds": delay, "attempt": attempt + 1},
+                        )
                         await asyncio.sleep(delay)
                         last_error = "Rate limited"
                         continue
@@ -72,13 +80,21 @@ class PeeringDBIXPCollector(HTTPCollector):
             except httpx.HTTPStatusError as e:
                 if e.response.status_code == 429:
                     delay = base_delay * (2**attempt)
-                    print(f"PeeringDB rate limited, waiting {delay}s before retry...")
+                    logger.warning_event(
+                        "PeeringDB rate limited; retrying after delay",
+                        event="collector.peeringdb.rate_limited",
+                        context={"delay_seconds": delay, "attempt": attempt + 1},
+                    )
                     await asyncio.sleep(delay)
                     last_error = "Rate limited"
                     continue
                 raise
 
-        print(f"Warning: PeeringDB collection failed after {max_retries} retries: {last_error}")
+        logger.warning_event(
+            "PeeringDB collection failed after retries",
+            event="collector.peeringdb.retries_exhausted",
+            context={"max_retries": max_retries, "last_error": last_error},
+        )
         return {}
 
     async def fetch(self) -> List[Dict[str, Any]]:
@@ -146,6 +162,7 @@ class PeeringDBNetworkCollector(HTTPCollector):
             "User-Agent": "Planet-Intelligence-System/1.0 (Python/collector)",
             "Accept": "application/json",
         }
+
     @property
     def request_url(self) -> str:
         base = self._resolved_url or self.base_url
@@ -167,7 +184,11 @@ class PeeringDBNetworkCollector(HTTPCollector):
 
                     if response.status_code == 429:
                         delay = base_delay * (2**attempt)
-                        print(f"PeeringDB rate limited, waiting {delay}s before retry...")
+                        logger.warning_event(
+                            "PeeringDB rate limited; retrying after delay",
+                            event="collector.peeringdb.rate_limited",
+                            context={"delay_seconds": delay, "attempt": attempt + 1},
+                        )
                         await asyncio.sleep(delay)
                         last_error = "Rate limited"
                         continue
@@ -178,13 +199,21 @@ class PeeringDBNetworkCollector(HTTPCollector):
             except httpx.HTTPStatusError as e:
                 if e.response.status_code == 429:
                     delay = base_delay * (2**attempt)
-                    print(f"PeeringDB rate limited, waiting {delay}s before retry...")
+                    logger.warning_event(
+                        "PeeringDB rate limited; retrying after delay",
+                        event="collector.peeringdb.rate_limited",
+                        context={"delay_seconds": delay, "attempt": attempt + 1},
+                    )
                     await asyncio.sleep(delay)
                     last_error = "Rate limited"
                     continue
                 raise
 
-        print(f"Warning: PeeringDB collection failed after {max_retries} retries: {last_error}")
+        logger.warning_event(
+            "PeeringDB collection failed after retries",
+            event="collector.peeringdb.retries_exhausted",
+            context={"max_retries": max_retries, "last_error": last_error},
+        )
         return {}
 
     async def fetch(self) -> List[Dict[str, Any]]:
@@ -254,6 +283,7 @@ class PeeringDBFacilityCollector(HTTPCollector):
             "User-Agent": "Planet-Intelligence-System/1.0 (Python/collector)",
             "Accept": "application/json",
         }
+
     @property
     def request_url(self) -> str:
         base = self._resolved_url or self.base_url
@@ -275,7 +305,11 @@ class PeeringDBFacilityCollector(HTTPCollector):
 
                     if response.status_code == 429:
                         delay = base_delay * (2**attempt)
-                        print(f"PeeringDB rate limited, waiting {delay}s before retry...")
+                        logger.warning_event(
+                            "PeeringDB rate limited; retrying after delay",
+                            event="collector.peeringdb.rate_limited",
+                            context={"delay_seconds": delay, "attempt": attempt + 1},
+                        )
                         await asyncio.sleep(delay)
                         last_error = "Rate limited"
                         continue
@@ -286,13 +320,21 @@ class PeeringDBFacilityCollector(HTTPCollector):
             except httpx.HTTPStatusError as e:
                 if e.response.status_code == 429:
                     delay = base_delay * (2**attempt)
-                    print(f"PeeringDB rate limited, waiting {delay}s before retry...")
+                    logger.warning_event(
+                        "PeeringDB rate limited; retrying after delay",
+                        event="collector.peeringdb.rate_limited",
+                        context={"delay_seconds": delay, "attempt": attempt + 1},
+                    )
                     await asyncio.sleep(delay)
                     last_error = "Rate limited"
                     continue
                 raise
 
-        print(f"Warning: PeeringDB collection failed after {max_retries} retries: {last_error}")
+        logger.warning_event(
+            "PeeringDB collection failed after retries",
+            event="collector.peeringdb.retries_exhausted",
+            context={"max_retries": max_retries, "last_error": last_error},
+        )
         return {}
 
     async def fetch(self) -> List[Dict[str, Any]]:

backend/app/services/collectors/spacetrack.py

@@ -1,17 +1,21 @@
-"""Space-Track TLE Collector
+"""Space-Track TLE Collector.
 
 Collects satellite TLE (Two-Line Element) data from Space-Track.org.
 API documentation: https://www.space-track.org/documentation
 """
 
-import json
-from typing import Dict, Any, List
-import httpx
+from typing import Any, Dict, List
 from urllib.parse import urlparse
 
-from app.services.collectors.base import BaseCollector
+import httpx
+
 from app.core.data_sources import get_data_sources_config
+from app.core.logging import get_logger
 from app.core.satellite_tle import build_tle_lines_from_elements
+from app.services.collectors.base import BaseCollector
+
+
+logger = get_logger(__name__, service="collector")
 
 
 class SpaceTrackTLECollector(BaseCollector):
@@ -53,10 +57,16 @@ class SpaceTrackTLECollector(BaseCollector):
         password = settings.SPACETRACK_PASSWORD
 
         if not username or not password:
-            print("SPACETRACK: No credentials configured, using sample data")
+            logger.warning_event(
+                "Space-Track credentials are not configured; using sample data",
+                event="collector.spacetrack.credentials_missing",
+            )
             return self._get_sample_data()
 
-        print(f"SPACETRACK: Attempting to fetch TLE data with username: {username}")
+        logger.info_event(
+            "Space-Track TLE fetch started",
+            event="collector.spacetrack.fetch.start",
+        )
 
         try:
             async with httpx.AsyncClient(
@@ -78,11 +88,17 @@ class SpaceTrackTLECollector(BaseCollector):
                         "password": password,
                     },
                 )
-                print(f"SPACETRACK: Login response status: {login_response.status_code}")
-                print(f"SPACETRACK: Login response URL: {login_response.url}")
+                logger.info_event(
+                    "Space-Track login response received",
+                    event="collector.spacetrack.login.response",
+                    context={"status_code": login_response.status_code},
+                )
 
                 if login_response.status_code == 403:
-                    print("SPACETRACK: Trying alternate login method...")
+                    logger.warning_event(
+                        "Space-Track login returned forbidden; trying alternate method",
+                        event="collector.spacetrack.login.forbidden",
+                    )
 
                     async with httpx.AsyncClient(
                         timeout=120.0,
@@ -90,11 +106,6 @@ class SpaceTrackTLECollector(BaseCollector):
                     ) as alt_client:
                         await alt_client.get(f"{self.site_root}/")
 
-                        form_data = {
-                            "username": username,
-                            "password": password,
-                            "query": "class/gp/NORAD_CAT_ID/25544/format/json",
-                        }
                         alt_login = await alt_client.post(
                             self.login_url,
                             data={
@@ -102,77 +113,59 @@ class SpaceTrackTLECollector(BaseCollector):
                                 "password": password,
                             },
                         )
-                        print(f"SPACETRACK: Alt login status: {alt_login.status_code}")
+                        logger.info_event(
+                            "Space-Track alternate login response received",
+                            event="collector.spacetrack.alt_login.response",
+                            context={"status_code": alt_login.status_code},
+                        )
 
                         if alt_login.status_code == 200:
                             tle_response = await alt_client.get(self.probe_url)
                             if tle_response.status_code == 200:
                                 data = tle_response.json()
-                                print(f"SPACETRACK: Received {len(data)} records via alt method")
+                                logger.info_event(
+                                    "Space-Track alternate query completed",
+                                    event="collector.spacetrack.alt_query.completed",
+                                    context={"record_count": len(data)},
+                                )
                                 return data
 
                 if login_response.status_code != 200:
-                    print(f"SPACETRACK: Login failed, using sample data")
+                    logger.warning_event(
+                        "Space-Track login failed; using sample data",
+                        event="collector.spacetrack.login.failed",
+                        context={"status_code": login_response.status_code},
+                    )
                     return self._get_sample_data()
 
                 tle_response = await client.get(self.probe_url)
-                print(f"SPACETRACK: TLE query status: {tle_response.status_code}")
-
-                if tle_response.status_code != 200:
-                    print(f"SPACETRACK: Query failed, using sample data")
-                    return self._get_sample_data()
-
-                data = tle_response.json()
-                print(f"SPACETRACK: Received {len(data)} records")
-                return data
-        except Exception as e:
-            print(f"SPACETRACK: Error - {e}, using sample data")
-            return self._get_sample_data()
-
-        print(f"SPACETRACK: Attempting to fetch TLE data with username: {username}")
-
-        try:
-            async with httpx.AsyncClient(
-                timeout=120.0,
-                follow_redirects=True,
-                headers={
-                    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36",
-                    "Accept": "application/json, text/html, */*",
-                    "Accept-Language": "en-US,en;q=0.9",
-                },
-            ) as client:
-                # First, visit the main page to get any cookies
-                await client.get(f"{self.site_root}/")
-
-                # Login to get session cookie
-                login_response = await client.post(
-                    self.login_url,
-                    data={
-                        "identity": username,
-                        "password": password,
-                    },
+                logger.info_event(
+                    "Space-Track TLE query response received",
+                    event="collector.spacetrack.query.response",
+                    context={"status_code": tle_response.status_code},
                 )
-                print(f"SPACETRACK: Login response status: {login_response.status_code}")
-                print(f"SPACETRACK: Login response URL: {login_response.url}")
-                print(f"SPACETRACK: Login response body: {login_response.text[:500]}")
-
-                if login_response.status_code != 200:
-                    print(f"SPACETRACK: Login failed, using sample data")
-                    return self._get_sample_data()
-
-                # Query for TLE data (get first 1000 satellites)
-                tle_response = await client.get(self.query_url)
-                print(f"SPACETRACK: TLE query status: {tle_response.status_code}")
 
                 if tle_response.status_code != 200:
-                    print(f"SPACETRACK: Query failed, using sample data")
+                    logger.warning_event(
+                        "Space-Track TLE query failed; using sample data",
+                        event="collector.spacetrack.query.failed",
+                        context={"status_code": tle_response.status_code},
+                    )
                     return self._get_sample_data()
 
                 data = tle_response.json()
-                print(f"SPACETRACK: Received {len(data)} records")
+                logger.info_event(
+                    "Space-Track TLE fetch completed",
+                    event="collector.spacetrack.fetch.completed",
+                    context={"record_count": len(data)},
+                )
                 return data
         except Exception as e:
-            print(f"SPACETRACK: Error - {e}, using sample data")
+            logger.warning_event(
+                "Space-Track TLE fetch failed; using sample data",
+                event="collector.spacetrack.fetch.failed",
+                context={"error": str(e)},
+            )
             return self._get_sample_data()
 
     def transform(self, raw_data: List[Dict[str, Any]]) -> List[Dict[str, Any]]:

backend/app/services/collectors/telegeography.py

@@ -163,32 +163,64 @@ class TeleGeographyLandingPointCollector(BaseCollector):
     data_type = "landing_point"
 
     async def fetch(self) -> List[Dict[str, Any]]:
-        """Fetch landing point data from GitHub mirror"""
-        url = self._resolved_url or ""
+        """Fetch landing point data, falling back when the old mirror disappears."""
+        config = get_data_sources_config()
+        sources = [
+            self._resolved_url or "",
+            str(config.get_yaml_value("telegeography.landing_point_url") or ""),
+            str(config.get_yaml_value("arcgis.landing_point_url") or ""),
+        ]
 
-        async with httpx.AsyncClient(timeout=60.0) as client:
-            response = await client.get(url)
-            response.raise_for_status()
-            return self.parse_response(response.json())
+        last_error: Exception | None = None
+        async with httpx.AsyncClient(timeout=60.0, follow_redirects=True) as client:
+            for url in dict.fromkeys(source for source in sources if source):
+                try:
+                    params = (
+                        {"where": "1=1", "outFields": "*", "returnGeometry": "true", "f": "geojson"}
+                        if "FeatureServer" in url or url.endswith("/query")
+                        else None
+                    )
+                    response = await client.get(url, params=params)
+                    response.raise_for_status()
+                    records = self.parse_response(response.json())
+                    if records:
+                        return records
+                except Exception as exc:
+                    last_error = exc
+                    continue
 
-    def parse_response(self, data: List[Dict[str, Any]]) -> List[Dict[str, Any]]:
+        if last_error:
+            raise last_error
+        return self._get_sample_data()
+
+    def parse_response(self, data: List[Dict[str, Any]] | Dict[str, Any]) -> List[Dict[str, Any]]:
         """Parse landing point data"""
         result = []
+        items = data.get("features", []) if isinstance(data, dict) else data
 
-        for item in data:
+        for item in items:
+            props = item.get("properties", {}) if isinstance(item, dict) else {}
+            geometry = item.get("geometry", {}) if isinstance(item, dict) else {}
+            source = props or item
+            coords = geometry.get("coordinates", []) if isinstance(geometry, dict) else []
+            longitude = coords[0] if len(coords) > 0 else source.get("longitude")
+            latitude = coords[1] if len(coords) > 1 else source.get("latitude")
+            source_id = source.get("id") or source.get("OBJECTID") or source.get("city_id") or ""
             try:
                 entry = {
-                    "source_id": f"telegeo_lp_{item.get('id', '')}",
-                    "name": item.get("name", "Unknown"),
-                    "country": item.get("country", "Unknown"),
-                    "city": item.get("city", item.get("name", "")),
-                    "latitude": str(item.get("latitude", "")),
-                    "longitude": str(item.get("longitude", "")),
+                    "source_id": f"telegeo_lp_{source_id}",
+                    "name": source.get("name", source.get("Name", "Unknown")),
+                    "country": source.get("country", "Unknown"),
+                    "city": source.get("city", source.get("Name", source.get("name", ""))),
+                    "latitude": str(latitude or ""),
+                    "longitude": str(longitude or ""),
                     "value": "",
                     "unit": "",
                     "metadata": {
-                        "cable_count": len(item.get("cables", [])),
-                        "url": item.get("url"),
+                        "cable_count": len(source.get("cables", [])),
+                        "url": source.get("url"),
+                        "objectid": source.get("OBJECTID"),
+                        "city_id": source.get("city_id"),
                     },
                     "reference_date": datetime.now(UTC).strftime("%Y-%m-%d"),
                 }

backend/app/services/collectors/top500.py

@@ -5,16 +5,18 @@ https://top500.org/lists/top500/
 """
 
 import asyncio
+import math
 import re
+from urllib.parse import urljoin
 from typing import Dict, Any, List
 from bs4 import BeautifulSoup
 import httpx
 
 from app.core.data_sources import get_data_sources_config
-from app.services.collectors.base import BaseCollector
+from app.services.collectors.compute_base import ComputeCollector
 
 
-class TOP500Collector(BaseCollector):
+class TOP500Collector(ComputeCollector):
     name = "top500"
     priority = "P0"
     module = "L1"
@@ -28,7 +30,7 @@ class TOP500Collector(BaseCollector):
         async with httpx.AsyncClient(timeout=60.0, follow_redirects=True) as client:
             response = await client.get(url)
             response.raise_for_status()
-            entries = self.parse_response(response.text)
+            entries = self.parse_response(response.text, str(response.url))
 
             semaphore = asyncio.Semaphore(8)
 
@@ -55,7 +57,7 @@ class TOP500Collector(BaseCollector):
         system_name = link.get_text(" ", strip=True) if link else system_cell.get_text(" ", strip=True)
         detail_url = ""
         if link and link.get("href"):
-            detail_url = f"{str(top500_base_url).rstrip('/')}{link.get('href')}"
+            detail_url = urljoin(str(top500_base_url), link.get("href"))
 
         manufacturer = ""
         if link and link.next_sibling:
@@ -127,10 +129,14 @@ class TOP500Collector(BaseCollector):
 
         return detail_map
 
-    def parse_response(self, html: str) -> List[Dict[str, Any]]:
+    def parse_response(self, html: str, source_url: str = "") -> List[Dict[str, Any]]:
         """Parse TOP500 HTML response"""
         data = []
         soup = BeautifulSoup(html, "html.parser")
+        canonical = soup.find("link", rel="canonical")
+        date_url = source_url or (canonical.get("href", "") if canonical else "") or self._resolved_url or ""
+        date_match = re.search(r"/lists/top500/(\d{4})/(06|11)/", date_url)
+        reference_date = f"{date_match[1]}-{date_match[2]}-01" if date_match else None
 
         # Find the table with TOP500 data
         table = None
@@ -152,6 +158,9 @@ class TOP500Collector(BaseCollector):
             table = soup.find("table", {"class": "top500-table"}) or soup.find("table", {"id": "top500"})
 
         if table:
+            headers = [cell.get_text(" ", strip=True) for cell in table.select("tr")[0].find_all(["th", "td"])]
+            if not reference_date:
+                raise ValueError("TOP500 list date missing; previous snapshot retained")
             rows = table.select("tr")
             for row in rows[1:]:  # Skip header row
                 cells = row.find_all(["td", "th"])
@@ -176,25 +185,32 @@ class TOP500Collector(BaseCollector):
                         cores = cells[2].get_text(strip=True).replace(",", "")
 
                         rmax_text = cells[3].get_text(strip=True)
-                        rmax = self._parse_performance(rmax_text)
+                        rmax = self._parse_performance(rmax_text, headers[3])
 
                         rpeak_text = cells[4].get_text(strip=True)
-                        rpeak = self._parse_performance(rpeak_text)
+                        rpeak = self._parse_performance(rpeak_text, headers[4])
 
                         power = ""
                         if len(cells) >= 6:
                             power = cells[5].get_text(strip=True).replace(",", "")
 
+                        identity = re.search(r"/system/(\d+)", detail_url)
+                        if not identity:
+                            raise ValueError("TOP500 stable system ID missing")
                         entry = {
-                            "source_id": f"top500_{rank}",
+                            "source_id": f"top500_system_{identity[1]}",
                             "name": system_name,
                             "country": country,
                             "city": city,
-                            "latitude": 0.0,
-                            "longitude": 0.0,
+                            "latitude": None,
+                            "longitude": None,
                             "value": str(rmax),
-                            "unit": "PFlop/s",
+                            "unit": "FLOP/s",
                             "metadata": {
+                                "compute_contract": "v1",
+                                "precision": "FP64", "measurement": "LINPACK Rmax",
+                                "source_url": detail_url, "list_url": date_url,
+                                "status": "listed", "rmax_flops": rmax, "rpeak_flops": rpeak,
                                 "rank": rank,
                                 "cores": cores,
                                 "rmax": rmax_text,
@@ -204,15 +220,11 @@ class TOP500Collector(BaseCollector):
                                 "site": site,
                             },
                             "_detail_url": detail_url,
-                            "reference_date": "2025-11-01",
+                            "reference_date": reference_date,
                         }
                         data.append(entry)
                     except (ValueError, IndexError, AttributeError) as e:
-                        continue
-
-        # If scraping failed, return sample data for testing
-        if not data:
-            data = self._get_sample_data()
+                        raise ValueError("Invalid TOP500 row; previous snapshot retained") from e
 
         return data
 
@@ -227,9 +239,14 @@ class TOP500Collector(BaseCollector):
                 return 0.0
         return 0.0
 
-    def _parse_performance(self, text: str) -> float:
+    def _parse_performance(self, text: str, header: str = "") -> float:
         """Parse performance value from text (handles E, P, T suffixes)"""
-        text = text.strip().upper()
+        text = text.strip().upper().replace(",", "")
+        if not re.search(r"[EPTGMK]F", text):
+            unit = re.search(r"\b([EPTGMK])FLOP/?S", header.upper())
+            if not unit:
+                raise ValueError("TOP500 performance unit missing")
+            text = f"{text} {unit[1]}FLOP/S"
         multipliers = {
             "E": 1e18,
             "P": 1e15,
@@ -249,73 +266,9 @@ class TOP500Collector(BaseCollector):
 
         # Try simple float parsing
         try:
-            return float(text.replace(",", ""))
+            value = float(text.replace(",", ""))
+            if not math.isfinite(value) or value < 0:
+                raise ValueError("Invalid TOP500 performance")
+            return value
         except ValueError:
-            return 0.0
-
-    def _get_sample_data(self) -> List[Dict[str, Any]]:
-        """Return sample data for testing when scraping fails"""
-        return [
-            {
-                "source_id": "top500_1",
-                "name": "El Capitan - HPE Cray EX255a, AMD 4th Gen EPYC 24C 1.8GHz, AMD Instinct MI300A",
-                "country": "United States",
-                "city": "Livermore, CA",
-                "latitude": 37.6819,
-                "longitude": -121.7681,
-                "value": "1742.00",
-                "unit": "PFlop/s",
-                "metadata": {
-                    "rank": 1,
-                    "cores": "11039616",
-                    "rmax": "1742.00",
-                    "rpeak": "2746.38",
-                    "power": "29581",
-                    "manufacturer": "HPE",
-                    "site": "DOE/NNSA/LLNL",
-                    "processor": "AMD 4th Gen EPYC 24C 1.8GHz",
-                    "interconnect": "Slingshot-11",
-                    "installation_year": "2025",
-                },
-                "reference_date": "2025-11-01",
-            },
-            {
-                "source_id": "top500_2",
-                "name": "Frontier - HPE Cray EX235a, AMD Optimized 3rd Generation EPYC 64C 2GHz, AMD Instinct MI250X",
-                "country": "United States",
-                "city": "Oak Ridge, TN",
-                "latitude": 36.0107,
-                "longitude": -84.2663,
-                "value": "1353.00",
-                "unit": "PFlop/s",
-                "metadata": {
-                    "rank": 2,
-                    "cores": "9066176",
-                    "rmax": "1353.00",
-                    "rpeak": "2055.72",
-                    "power": "24607",
-                    "manufacturer": "HPE",
-                    "site": "DOE/SC/Oak Ridge National Laboratory",
-                },
-                "reference_date": "2025-11-01",
-            },
-            {
-                "source_id": "top500_3",
-                "name": "Aurora - HPE Cray EX - Intel Exascale Compute Blade, Xeon CPU Max 9470 52C 2.4GHz, Intel Data Center GPU Max",
-                "country": "United States",
-                "city": "Argonne, IL",
-                "latitude": 41.3784,
-                "longitude": -87.8600,
-                "value": "1012.00",
-                "unit": "PFlop/s",
-                "metadata": {
-                    "rank": 3,
-                    "cores": "9264128",
-                    "rmax": "1012.00",
-                    "rpeak": "1980.01",
-                    "power": "38698",
-                    "manufacturer": "Intel",
-                },
-                "reference_date": "2025-11-01",
-            },
-        ]
+            raise ValueError("Invalid TOP500 performance") from None

backend/app/services/collectors/vessel_ais.py

@@ -6,6 +6,7 @@ from typing import Any
 import httpx
 from sqlalchemy.ext.asyncio import AsyncSession
 
+from app.core.enums import SnapshotStatus
 from app.core.time import to_iso8601_utc
 from app.core.websocket.broadcaster import broadcaster
 from app.services.barentswatch import (
@@ -119,6 +120,21 @@ class VesselAISCollector(BaseCollector):
             last_success_at=now if data else None,
             lag_seconds=max((now - latest_observed_at).total_seconds(), 0),
         )
+        if snapshot_id is not None:
+            from app.models.data_snapshot import DataSnapshot
+
+            snapshot = await db.get(DataSnapshot, snapshot_id)
+            if snapshot:
+                snapshot.record_count = records_added
+                snapshot.status = SnapshotStatus.SUCCESS.value
+                snapshot.completed_at = now
+                snapshot.summary = {
+                    "created": records_added,
+                    "updated": 0,
+                    "unchanged": 0,
+                    "deleted": 0,
+                    "storage": "ais_raw_observations",
+                }
         await db.commit()
         await self._broadcast_vessel_snapshot(data)
         await self.update_progress(records_added, force=True)

backend/app/services/compute_catalog.py

@@ -0,0 +1,200 @@
+"""Canonical country and mutually exclusive regional compute participants."""
+
+import re
+from collections.abc import Iterable
+
+from app.core.countries import COUNTRY_CENTROIDS, COUNTRY_ENTRIES
+from app.core.earth_geography_policy import display_country
+
+PALETTE = ("#91baff", "#f0c88a", "#89d2c2", "#c5abdf", "#eda9b3", "#b6c99a")
+DEFAULT_PARTICIPANTS = ("CN", "US", "EUROPE", "APAC", "OTHER")
+
+# Explicit membership prevents a regional total from double-counting a country.
+# APAC deliberately excludes China.
+COMPARISON_GROUPS = (
+    ("CN", "中国", "China", ("CN",), {"latitude": 35.8617, "longitude": 104.1954}),
+    ("US", "美国", "United States", ("US",), {"latitude": 39.8283, "longitude": -98.5795}),
+    (
+        "EUROPE",
+        "欧洲",
+        "Europe",
+        (
+            "GB",
+            "DE",
+            "FR",
+            "CH",
+            "ES",
+            "FI",
+            "HU",
+            "IE",
+            "IT",
+            "LU",
+            "MD",
+            "NL",
+            "PL",
+            "RO",
+            "RU",
+            "UA",
+        ),
+        {"latitude": 54.0, "longitude": 20.0},
+    ),
+    (
+        "APAC",
+        "亚太（不含中国）",
+        "Asia Pacific (excluding China)",
+        ("JP", "KR", "IN", "AU", "AF", "ID", "NP", "PH", "PK", "SG", "TH"),
+        {"latitude": 36.0, "longitude": 130.5},
+    ),
+    (
+        "OTHER",
+        "其他地区",
+        "Other",
+        (
+            "CA",
+            "AO",
+            "EG",
+            "KE",
+            "MZ",
+            "NG",
+            "UG",
+            "ZA",
+            "BR",
+            "CO",
+            "CR",
+            "DO",
+            "MX",
+            "AE",
+            "SY",
+            "TR",
+        ),
+        {"latitude": 12.0, "longitude": 10.0},
+    ),
+)
+
+
+def _country_code(canonical: str, aliases: list[str]) -> str | None:
+    code = next((alias for alias in aliases if re.fullmatch(r"[A-Z]{2}", alias)), None)
+    return "GB" if canonical == "英国" else code
+
+
+def build_country_catalog() -> list[dict]:
+    """Build the policy-projected canonical country drilldown catalog."""
+    countries = []
+    for canonical, aliases in COUNTRY_ENTRIES:
+        if display_country(canonical) != canonical or canonical not in COUNTRY_CENTROIDS:
+            continue
+        code = _country_code(canonical, aliases)
+        if code:
+            countries.append(
+                {
+                    "code": code,
+                    "name": canonical,
+                    "name_en": aliases[0],
+                    "kind": "country",
+                    "members": [code],
+                    "anchor": dict(COUNTRY_CENTROIDS[canonical]),
+                    "anchor_kind": "regional_label",
+                }
+            )
+    return countries
+
+
+COUNTRY_CATALOG = build_country_catalog()
+COUNTRIES = {entry["code"]: entry["name"] for entry in COUNTRY_CATALOG}
+COUNTRY_PARTICIPANTS = {entry["code"]: entry for entry in COUNTRY_CATALOG}
+
+
+def _validate_group_members() -> None:
+    members = [
+        member for _code, _name, _name_en, codes, _anchor in COMPARISON_GROUPS for member in codes
+    ]
+    if len(members) != len(set(members)) or set(members) != set(COUNTRIES):
+        raise ValueError("compute comparison groups must cover country catalog exactly once")
+
+
+_validate_group_members()
+GROUP_BY_MEMBER = {
+    member: code
+    for code, _name, _name_en, members, _anchor in COMPARISON_GROUPS
+    for member in members
+}
+
+
+def build_catalog() -> list[dict]:
+    """Return the five default participants before country drilldown entries."""
+    catalog: list[dict] = []
+    for index, (code, name, name_en, members, anchor) in enumerate(COMPARISON_GROUPS):
+        entry = (
+            dict(COUNTRY_PARTICIPANTS[code])
+            if code in COUNTRY_PARTICIPANTS
+            else {
+                "code": code,
+                "name": name,
+                "name_en": name_en,
+                "kind": "region",
+                "members": list(members),
+                "anchor": dict(anchor),
+                "anchor_kind": "regional_label",
+            }
+        )
+        entry.update(
+            {
+                "members": list(members),
+                "parent_group": name,
+                "parent_group_code": code,
+                "order": index,
+                "default_selected": True,
+                "color": PALETTE[index % len(PALETTE)],
+            }
+        )
+        catalog.append(entry)
+
+    for entry in COUNTRY_CATALOG:
+        if entry["code"] in DEFAULT_PARTICIPANTS:
+            continue
+        country = dict(entry)
+        group_code = GROUP_BY_MEMBER[country["code"]]
+        group_name = next(group[1] for group in COMPARISON_GROUPS if group[0] == group_code)
+        country.update(
+            {
+                "parent_group": group_name,
+                "parent_group_code": group_code,
+                "order": len(catalog),
+                "default_selected": False,
+                "color": PALETTE[len(catalog) % len(PALETTE)],
+            }
+        )
+        catalog.append(country)
+    return catalog
+
+
+CATALOG = build_catalog()
+PARTICIPANTS = {entry["code"]: entry for entry in CATALOG}
+
+
+def participant_members(codes: Iterable[str]) -> dict[str, tuple[str, ...]]:
+    """Resolve country or region participant codes to canonical country codes."""
+    return {code: tuple(PARTICIPANTS[code]["members"]) for code in codes}
+
+
+def source_country_names(codes: Iterable[str]) -> list[str]:
+    """Return unique canonical source labels, respecting China and Serbia ownership."""
+    selected_members = {
+        member for members in participant_members(codes).values() for member in members
+    }
+    selected_names = {COUNTRIES[code] for code in selected_members}
+    return [name for name, _aliases in COUNTRY_ENTRIES if display_country(name) in selected_names]
+
+
+def source_country_participants(codes: Iterable[str]) -> dict[str, str]:
+    """Map stored source-country labels to their requested participant code."""
+    owner_by_country = {
+        member: participant
+        for participant, members in participant_members(codes).items()
+        for member in members
+    }
+    code_by_name = {name: code for code, name in COUNTRIES.items()}
+    return {
+        name: owner_by_country[code_by_name[display_country(name)]]
+        for name in source_country_names(codes)
+    }

backend/app/services/compute_center_locations.py

@@ -3,7 +3,12 @@
 This module is a thin domain wrapper that wires up
 :mod:`app.services.location` for compute centers:
 
-    SourceCoordinates
+    Confirmed stored location -> trustworthy source coordinates -> exact
+    official facility registration -> source city/site coordinates.
+
+Epoch AI coordinates are admitted only at the grain asserted by its official
+``Location`` column. Country/region/empty locations remain unresolved even
+when the CSV happens to contain a latitude/longitude pair.
 
 The online Nominatim step is intentionally reserved for the user-triggered
 ``collect-location`` flow. The regular GeoJSON endpoint runs during Earth
@@ -11,7 +16,7 @@ startup, so it must stay local and deterministic.
 
 For the full design and the reason behind the abstraction (compute centers,
 BGP collectors, BGP events, and future entities all share one pipeline),
-see ``docs/plans/location-resolver-shared-pipeline-plan.md``.
+see ``docs/technical/zh/location-pipeline-development.md``.
 
 The ``ComputeCenterLocation`` dataclass and the public function signatures are
 preserved verbatim so existing callers and tests do not need to change.
@@ -28,9 +33,12 @@ import httpx
 from sqlalchemy import select
 from sqlalchemy.ext.asyncio import AsyncSession
 
+from app.core.compute_types import compute_record_filter
 from app.core.collected_data_fields import get_record_field
 from app.models.collected_data import CollectedData
 from app.models.compute_center_location import ComputeCenterLocationRecord
+from app.services.compute_center_verified_sites import get_verified_compute_site
+from app.services.epoch_location import assess_epoch_location
 
 from app.services.location import (
     LocationCandidate,
@@ -166,8 +174,13 @@ def set_compute_center_location_cache(
 
 async def refresh_compute_center_location_cache(
     session: AsyncSession,
+    *,
+    as_of: datetime | None = None,
 ) -> dict[str, dict[str, Any]]:
-    result = await session.execute(select(ComputeCenterLocationRecord))
+    query = select(ComputeCenterLocationRecord)
+    if as_of is not None:
+        query = query.where(ComputeCenterLocationRecord.verified_at < as_of)
+    result = await session.execute(query)
     records = result.scalars().all()
     cache = {}
     for record in records:
@@ -350,6 +363,58 @@ class ROROrganizationResolver:
         )
 
 
+class ComputeCenterSourceCoordinatesResolver:
+    """Use source coordinates only at the grain asserted by the source text."""
+
+    name = "source_coordinates"
+
+    def __init__(self) -> None:
+        self._default = SourceCoordinatesResolver()
+
+    def resolve(self, query: LocationQuery) -> ResolverOutput:
+        extra = query.extra or {}
+        if extra.get("source") != "epoch_ai_gpu":
+            return self._default.resolve(query)
+        if query.source_latitude in (None, 0.0) or query.source_longitude in (None, 0.0):
+            return ResolverOutput()
+        assessment = assess_epoch_location(
+            extra.get("source_location"),
+            country=query.country,
+            raw_country=extra.get("source_raw_country"),
+        )
+        if not assessment.is_renderable:
+            return ResolverOutput(
+                attempted_queries=(f"source_coordinates:{assessment.precision}",)
+            )
+        output = self._default.resolve(query)
+        if not output.candidates:
+            return output
+        candidate = output.candidates[0]
+        source_location = extra.get("source_location")
+        source_location_text = (
+            source_location.strip() if isinstance(source_location, str) else ""
+        )
+        source_note = assessment.reason
+        if source_location_text:
+            source_note = f"{source_note}; Epoch Location: {source_location_text}"
+        return ResolverOutput(
+            candidates=(
+                LocationCandidate(
+                    **{
+                        **candidate.__dict__,
+                        "precision": assessment.precision,
+                        "confidence": 1.0 if assessment.precision == "precise" else 0.9,
+                        "source_note": source_note,
+                        "needs_confirmation": assessment.precision != "precise",
+                        "matched_location_name": source_location_text
+                        or candidate.matched_location_name,
+                    }
+                ),
+            ),
+            attempted_queries=output.attempted_queries,
+        )
+
+
 class StoredComputeCenterLocationResolver:
     """Resolve a compute center through the DB-backed current-location cache."""
 
@@ -367,13 +432,26 @@ class StoredComputeCenterLocationResolver:
         longitude = parse_float(stored.get("longitude"))
         if latitude in (None, 0.0) or longitude in (None, 0.0):
             return ResolverOutput()
+        precision = stored.get("precision") or "city"
+        if (
+            extra.get("source") == "epoch_ai_gpu"
+            and stored.get("location_source") == "source_coordinates"
+        ):
+            assessment = assess_epoch_location(
+                extra.get("source_location"),
+                country=query.country,
+                raw_country=extra.get("source_raw_country"),
+            )
+            if not assessment.is_renderable:
+                return ResolverOutput()
+            precision = assessment.precision
         return ResolverOutput(
             candidates=(
                 LocationCandidate(
                     latitude=latitude,
                     longitude=longitude,
                     display_name=stored.get("name") or query.name or "Compute center",
-                    precision=stored.get("precision") or "city",
+                    precision=precision,
                     confidence=float(stored.get("confidence") or 0.85),
                     query=f"stored_compute_center_location::{stored.get('source')}:{stored.get('source_id')}",
                     source=self.name,
@@ -386,6 +464,60 @@ class StoredComputeCenterLocationResolver:
                     matched_location_name=stored.get("site") or stored.get("name") or query.name,
                     location_verified_at=stored.get("verified_at"),
                     suggested_registry_entry=None,
+                    raw_payload={
+                        "stored_location_source": stored.get("location_source"),
+                        "verification_status": stored.get("verification_status"),
+                    },
+                ),
+            )
+        )
+
+
+class VerifiedComputeSiteResolver:
+    """Resolve an exact source identity to an evidence-backed facility site."""
+
+    name = "official_facility_registry"
+
+    def resolve(self, query: LocationQuery) -> ResolverOutput:
+        extra = query.extra or {}
+        source = coerce_str(extra.get("source"))
+        source_id = coerce_str(extra.get("source_id"))
+        if not source or not source_id:
+            return ResolverOutput()
+        site = get_verified_compute_site(source, source_id)
+        if not site:
+            return ResolverOutput()
+        source_urls = [coerce_str(url) for url in site.get("sources", []) if coerce_str(url)]
+        source_note = coerce_str(site.get("source_note"))
+        if source_urls:
+            source_note = f"{source_note} Sources: {'; '.join(source_urls)}".strip()
+        return ResolverOutput(
+            candidates=(
+                LocationCandidate(
+                    latitude=float(site["latitude"]),
+                    longitude=float(site["longitude"]),
+                    display_name=coerce_str(site.get("matched_location_name"))
+                    or query.name
+                    or "Compute center",
+                    precision=coerce_str(site.get("precision")) or "site",
+                    confidence=1.0,
+                    query=f"official_facility_registry::{source}:{source_id}",
+                    source=self.name,
+                    source_note=source_note or None,
+                    matched_fields=("source", "source_id"),
+                    needs_confirmation=False,
+                    city=coerce_str(site.get("city")) or None,
+                    region=None,
+                    country=coerce_str(site.get("country")) or query.country,
+                    matched_location_name=coerce_str(site.get("matched_location_name"))
+                    or None,
+                    location_verified_at=coerce_str(site.get("verified_at")) or None,
+                    suggested_registry_entry=None,
+                    raw_payload={
+                        "address": site.get("address"),
+                        "coordinate_basis": site.get("coordinate_basis"),
+                        "sources": source_urls,
+                    },
                 ),
             )
         )
@@ -400,7 +532,7 @@ def _short_system_name(name: Any) -> str:
     return head or text
 
 
-def _record_context(record: Any, metadata: dict[str, Any]) -> dict[str, str]:
+def _record_context(record: Any, metadata: dict[str, Any]) -> dict[str, Any]:
     name = coerce_str(getattr(record, "name", None))
     return {
         "source": coerce_str(getattr(record, "source", None)),
@@ -417,11 +549,13 @@ def _record_context(record: Any, metadata: dict[str, Any]) -> dict[str, str]:
             or metadata.get("manufacturer")
         ),
         "organization": coerce_str(metadata.get("organization")),
+        "source_location": metadata.get("location"),
+        "source_raw_country": metadata.get("raw_country"),
     }
 
 
 def _context_to_query(
-    context: dict[str, str],
+    context: dict[str, Any],
     *,
     source_lat: float | None = None,
     source_lon: float | None = None,
@@ -444,6 +578,8 @@ def _context_to_query(
             "site": context.get("site") or "",
             "operator": context.get("operator") or "",
             "organization": context.get("organization") or "",
+            "source_location": context.get("source_location"),
+            "source_raw_country": context.get("source_raw_country"),
         },
     )
 
@@ -497,11 +633,36 @@ def _compute_center_query_plan(
     return plan
 
 
+def _compute_center_candidate_sort_key(candidate: LocationCandidate) -> tuple[int, int, float]:
+    stored_origin = (candidate.raw_payload or {}).get("stored_location_source")
+    stored_confirmed = (
+        candidate.source == "stored_compute_center_location"
+        and not candidate.needs_confirmation
+        and stored_origin != "source_coordinates"
+    )
+    if stored_confirmed:
+        source_rank = 0
+    elif candidate.source == "source_coordinates" and candidate.precision == "precise":
+        source_rank = 1
+    elif candidate.source == "official_facility_registry":
+        source_rank = 2
+    elif candidate.source == "source_coordinates":
+        source_rank = 3
+    elif candidate.source == "stored_compute_center_location":
+        source_rank = 4
+    else:
+        source_rank = 5
+    precision_rank = {"precise": 0, "site": 1, "city": 2}.get(candidate.precision, 9)
+    return (source_rank, precision_rank, -float(candidate.confidence or 0))
+
+
 COMPUTE_CENTER_PIPELINE = LocationPipeline(
     [
-        SourceCoordinatesResolver(),
+        ComputeCenterSourceCoordinatesResolver(),
         StoredComputeCenterLocationResolver(),
+        VerifiedComputeSiteResolver(),
     ],
+    sort_key=_compute_center_candidate_sort_key,
     failure_reason=(
         "Could not resolve to city-level coordinates from source coords"
         " or stored compute-center location."
@@ -510,7 +671,8 @@ COMPUTE_CENTER_PIPELINE = LocationPipeline(
 
 COMPUTE_CENTER_COLLECTION_PIPELINE = LocationPipeline(
     [
-        SourceCoordinatesResolver(),
+        ComputeCenterSourceCoordinatesResolver(),
+        VerifiedComputeSiteResolver(),
         ROROrganizationResolver(),
         NominatimResolver(
             query_plan_builder=_compute_center_query_plan,
@@ -518,6 +680,7 @@ COMPUTE_CENTER_COLLECTION_PIPELINE = LocationPipeline(
             geocoder=lambda q: _geocode_online(q),
         ),
     ],
+    sort_key=_compute_center_candidate_sort_key,
     failure_reason=(
         "Could not resolve to city-level coordinates from source coords"
         ", ROR organization lookup, or online geocoding."
@@ -531,6 +694,7 @@ COMPUTE_CENTER_COLLECTION_PIPELINE = LocationPipeline(
 _GEOGRAPHY_MODE_BY_SOURCE = {
     "source_coordinates": "source_coordinates",
     "stored_compute_center_location": "stored_compute_center_location",
+    "official_facility_registry": "official_facility_registry",
     "ror_organization_registry": "ror_organization",
     "nominatim_online_geocode": "online_geocode",
 }
@@ -539,7 +703,7 @@ _GEOGRAPHY_MODE_BY_SOURCE = {
 def _candidate_to_location(
     candidate: LocationCandidate,
     *,
-    context: dict[str, str],
+    context: dict[str, Any],
 ) -> ComputeCenterLocation:
     geography_mode = _GEOGRAPHY_MODE_BY_SOURCE.get(candidate.source, "online_geocode")
     is_estimated = candidate.needs_confirmation or candidate.source.startswith(
@@ -594,7 +758,7 @@ def _candidate_to_location(
 
 def _diagnostic_for(
     record: Any,
-    context: dict[str, str],
+    context: dict[str, Any],
     *,
     failure_reason: str,
     attempted_queries: tuple[str, ...] = (),
@@ -738,7 +902,7 @@ def build_compute_center_location_query(
     organization: str | None = None,
 ) -> LocationQuery:
     name_value = coerce_str(name)
-    context: dict[str, str] = {
+    context: dict[str, Any] = {
         "source": coerce_str(source),
         "source_id": coerce_str(source_id),
         "name": name_value,
@@ -767,7 +931,7 @@ async def seed_compute_center_locations_from_source_coords(
     """Seed stored compute-center locations only from real source coordinates."""
     stmt = (
         select(CollectedData)
-        .where(CollectedData.source.in_(["top500", "epoch_ai_gpu"]))
+        .where(compute_record_filter(CollectedData))
         .where(CollectedData.is_current.is_(True))
     )
     result = await session.execute(stmt)
@@ -783,6 +947,23 @@ async def seed_compute_center_locations_from_source_coords(
         longitude = parse_float(get_record_field(record, "longitude"))
         if latitude in (None, 0.0) or longitude in (None, 0.0):
             continue
+        metadata = record.extra_data or {}
+        precision = "precise"
+        confidence = 1.0
+        needs_confirmation = False
+        source_note = "Seeded from source-provided compute-center coordinates"
+        if source_value == "epoch_ai_gpu":
+            assessment = assess_epoch_location(
+                metadata.get("location"),
+                country=get_record_field(record, "country"),
+                raw_country=metadata.get("raw_country"),
+            )
+            if not assessment.is_renderable:
+                continue
+            precision = assessment.precision
+            confidence = 1.0 if precision == "precise" else 0.9
+            needs_confirmation = precision != "precise"
+            source_note = assessment.reason
         existing = await session.scalar(
             select(ComputeCenterLocationRecord)
             .where(ComputeCenterLocationRecord.source == source_value)
@@ -790,7 +971,6 @@ async def seed_compute_center_locations_from_source_coords(
         )
         if existing:
             continue
-        metadata = record.extra_data or {}
         session.add(
             ComputeCenterLocationRecord(
                 source=source_value,
@@ -802,16 +982,16 @@ async def seed_compute_center_locations_from_source_coords(
                 country=coerce_str(get_record_field(record, "country")) or None,
                 latitude=latitude,
                 longitude=longitude,
-                precision="precise",
-                confidence=1.0,
+                precision=precision,
+                confidence=confidence,
                 location_source="source_coordinates",
-                source_note="Seeded from source-provided compute-center coordinates",
+                source_note=source_note,
                 raw_payload={
                     "record_id": getattr(record, "id", None),
                     "source": source_value,
                     "source_id": source_id,
                 },
-                needs_confirmation=False,
+                needs_confirmation=needs_confirmation,
                 verification_status="source_provided",
                 verified_at=None,
             )