release: bump version to 0.76.19

.claude/commands/cleanup.md

@@ -1,139 +0,0 @@
----
-description: 审查当前工作区未提交代码中的垃圾代码，并在不影响逻辑的前提下自动清理
-argument-hint: 可选：指定要检查的文件或目录（默认检查所有未提交修改）
-allowed-tools: ["Read", "Edit", "Bash", "Grep", "Glob"]
----
-
-# /cleanup — 垃圾代码审查与清理
-
-分析当前工作区（git diff）中的未提交代码，找出并修复常见垃圾代码，**不得改变任何运行逻辑**。
-
-## 检查范围
-
-若 `$ARGUMENTS` 非空，则只检查指定文件/目录；否则检查所有未提交修改（`git diff HEAD`）。
-
-## 节省上下文规则
-
-优先用确定性的 CLI 检查缩小范围，不要一上来把完整文件或大 diff 读入上下文：
-
-```bash
-git diff --name-only HEAD
-git diff --unified=0 HEAD -- <path>
-git diff --check
-rg -n "TODO|FIXME|console\.log|debugger|print\(" <changed-paths>
-```
-
-只有 focused diff 不足以安全判断或修改时，才读取完整文件。
-
-## 审查清单
-
-按优先级检查以下问题（只报告在本次 diff 中**新增或修改**的代码里存在的问题）：
-
-### 1. 重复逻辑 (Duplicate Logic)
-- 完全相同或高度相似的代码块在多处出现
-- 同一函数/方法被多个地方各自实现，已有公共版本未被复用
-- 相同的 DOM 查询、正则、模板字符串在同一文件重复
-
-### 2. Magic Numbers / Magic Strings
-- 裸数字直接参与计算（如偏移量、时间、尺寸、阈值），没有命名常量
-- 硬编码字符串（如 id 名、状态值、URL 片段）散落在逻辑中
-- 例外：`0`, `1`, `-1`, `100`, `""` 等语义明确的惯用值不算
-
-### 3. 命名问题
-- 含义不明的缩写变量（如 `or_`, `tmp2`, `x2`）
-- 命名与实际用途不符
-- 同一概念在不同地方用不同名字表达
-
-### 4. 死代码 / 无效代码
-- 注释掉的旧代码块（3行以上）
-- 声明后从未使用的变量/参数/导入
-- 永远不会执行的条件分支
-
-### 5. 代码风格问题
-- 尾部空白字符（trailing whitespace）
-- 同一文件内风格不一致（如混用单双引号、缩进不统一）
-- 空行使用不一致（连续多个空行等）
-
-### 6. 其他常见问题
-- 私有辅助函数应被 export 但没有，导致调用方重复实现
-- 类型/接口重复定义
-- 过于冗长的条件表达式可以简化（不改逻辑）
-
-## 执行步骤
-
-### Step 1 — 获取待检查文件列表
-
-```bash
-# 无参数时：获取所有未提交修改
-git diff HEAD --name-only
-
-# 有参数时：用 $ARGUMENTS 过滤
-```
-
-### Step 2 — 逐文件阅读并分析
-
-先从 focused diff 开始：
-
-```bash
-git diff --unified=0 HEAD -- <file>
-```
-
-用 `rg`、`git diff --check`、编译器或 linter 输出确认确定性问题。只有需要上下文时才用 Read 读取完整文件。对照审查清单，记录每个问题：文件名、行号、问题类型、建议修复方式。
-
-### Step 3 — 报告问题清单
-
-在修改前，先以列表形式输出所有发现的问题：
-
-```
-发现 N 个问题：
-
-[文件] js/foo.js
-  · L34, L78: 重复逻辑 — 两处都实现了相同的 DOM 查询，可提取到 getPanel()
-  · L91: Magic number — 硬编码 14 作为偏移量，应命名为 TOOLTIP_OFFSET
-
-[文件] js/bar.js
-  · L12: 命名问题 — 变量 `or_` 语义不明，应命名为 outerR/outerG/outerB
-  ...
-```
-
-如果没有发现问题，直接输出"未发现垃圾代码，当前代码质量良好。"并停止。
-
-### Step 4 — 执行修复
-
-对每个问题，使用 Edit 工具进行**最小化修改**：
-
-- **重复逻辑**：提取为共享常量/函数，更新所有调用点
-- **Magic number**：在文件顶部或逻辑附近声明 `const NAME = value`，替换所有引用
-- **命名问题**：重命名变量，更新所有使用处
-- **死代码**：直接删除
-- **尾部空白/风格**：修正
-- **未 export 的函数**：添加 `export`，在调用方改为导入（不重复实现）
-
-**修复原则：**
-- 只改在审查清单中发现的问题，不做额外优化
-- 每次 Edit 只修改确实有问题的行，保持 diff 最小
-- 改完后用 `grep` 验证旧的坏代码已消失
-- 优先做精确补丁；只有仓库已有对应格式化流程时，才运行格式化工具
-
-### Step 5 — 输出总结
-
-```
-清理完成：
-
-修复了 N 个问题：
-  ✓ earth.js — 提取重复 vertexShader 为 ATMOS_VERTEX_SHADER 常量
-  ✓ main.js — 提取 TOOLTIP_CURSOR_OFFSET = 14（4处引用）
-  ✓ controls.js — export updateLayerButtonState，移除 main.js 中的重复实现
-  ...
-
-未修改的问题（需人工确认）：
-  ! foo.js L45 — 注释代码块较长，建议手动确认是否可删除
-```
-
-## 约束
-
-- **禁止**改变函数签名、接口定义、导出 API（除非问题正是私有函数应被 export）
-- **禁止**添加新功能、新抽象、新参数
-- **禁止**修改注释内容（只删除注释掉的死代码）
-- **禁止**修改测试文件逻辑
-- 如果一个 Magic number 的语义不完全确定，**跳过**，在总结中标记为"需人工确认"

.claude/commands/docs.md

@@ -1,104 +0,0 @@
----
-description: Create or update repository documentation from current code changes
-argument-hint: Optional: topic to document, or leave empty to infer from git diff
-allowed-tools: ["Read", "Edit", "Write", "Bash", "Glob", "Grep"]
----
-
-# /docs — Documentation Workflow
-
-## Goal
-
-Create or update documentation that explains why a change exists, how it behaves, and what maintainers need to know. Keep this command generic. Repository-specific coverage rules live in the repository and must be loaded separately.
-
-## Repository Rules
-
-Before deciding scope, check whether the repository has a documentation rules file:
-
-```bash
-test -f docs/documentation-coverage-rules.md && sed -n '1,240p' docs/documentation-coverage-rules.md
-```
-
-If it exists, apply it as the project-specific coverage checklist. If it does not exist, continue with the generic workflow below.
-
-## Workflow
-
-### Step 1 — Understand The Change
-
-```bash
-git diff HEAD --stat
-git diff HEAD --name-only
-git log --oneline -10
-rg --files docs
-```
-
-If `$ARGUMENTS` specifies a topic, focus on that topic. Otherwise infer the documentation topic from the changed files. Do not read the full repository diff by default; inspect focused files only:
-
-```bash
-git diff HEAD -- <path>
-rg -n "class |def |function |export |router|@router|interface |type " <path>
-```
-
-### Step 2 — Decide Scope
-
-- Prefer updating an existing relevant document over creating a duplicate.
-- Use one document for one coherent topic.
-- Split documents only when the change crosses meaningful domains.
-- Keep filenames lowercase and hyphenated.
-- Apply the repository-specific rules file before writing.
-
-#### Document Audience Routing (Planet)
-
-In this repository, classify the action's performer before picking a target file:
-
-- Browser/UI end user → `docs/technical/{zh,en}/manual.md` or `quickstart.md`.
-- Shell / Docker / log paths / `planet.sh` / SMTP fallbacks / port forwarding → `docs/technical/{zh,en}/ops-runbook.md` (or an existing `ops-*.md`).
-- Second-party developers → existing `*-context.md` / `backend-*.md` / `earth-*.md` files.
-
-Never put shell commands, log paths, or Docker operations into `manual.md` / `quickstart.md`. Never put UI button labels or screenshots into `ops-*.md`. When the same action has both a UI and a CLI path, write each in its own home and cross-link them with one sentence.
-
-For ambiguous or large documentation changes, briefly state the intended doc plan before editing. For clear small changes, proceed directly.
-
-### Step 3 — Write
-
-Explain:
-
-- Background/problem: what was wrong or missing before.
-- Core design decisions and rationale.
-- Operational or user-facing impact.
-- Relevant code paths, only when useful for future maintainers.
-
-Style:
-
-- Follow the repository’s existing language and heading conventions.
-- Use fenced code blocks with language tags.
-- Prefer tables for comparisons or parameter lists.
-- Keep snippets concise and relevant.
-- For UI labels, chart labels, feature names, datasource names, and other terms that may become mixed Chinese/English copy, check `docs/technical/{zh,en}/naming-glossary.md` and use the documented display name. If a confusing term is missing, update the glossary in both languages as part of the docs change.
-
-### Step 4 — Verify
-
-- Read the completed docs once for clarity and stale statements.
-- Verify referenced paths exist with `test -e` or `rg --files`.
-- Run applicable checks from `docs/documentation-coverage-rules.md`.
-- Check Markdown links use readable user-facing titles unless repository rules allow otherwise.
-
-### Step 5 — Report
-
-Summarize changed docs and verification:
-
-```md
-Updated:
-- path/to/doc.md — what changed
-
-Verified:
-- checks that passed
-- checks that could not be run, if any
-```
-
-## Hard Constraints
-
-- Do not leave placeholder docs.
-- Do not duplicate bilingual files byte-for-byte.
-- Do not reference PR numbers, issue numbers, or the current conversation unless explicitly requested.
-- Do not write changelog-style lists without the reasoning and tradeoffs behind the change.
-- Keep docs maintainable and concise.

.claude/commands/goal-driven.md

@@ -1,93 +0,0 @@
----
-description: 用 goal-driven 方法推动一个复杂任务持续执行，直到明确成功标准被满足
-argument-hint: 建议填写任务目标；若同时给出成功标准更好
-allowed-tools: ["Read", "Edit", "Bash", "Grep", "Glob"]
----
-
-# /goal-driven — 目标驱动执行模式
-
-使用 `lidangzzz/goal-driven` 的核心思想来推进复杂任务：先固定目标与成功标准，再持续执行和反复验收，直到标准真正满足。
-
-适用场景：
-
-- 长周期实现任务
-- 高复杂度工程任务
-- 可被明确验收的研究、实现、迁移、验证类工作
-
-不适用场景：
-
-- 纯脑暴
-- 无法定义成功标准的模糊任务
-- 很小的一次性修改
-
-## 输入要求
-
-若 `$ARGUMENTS` 只包含目标，没有成功标准，先补全一版可执行的成功标准再开始。
-
-启动时先输出：
-
-```md
-Goal
-- ...
-
-Criteria for success
-- ...
-
-Plan
-1. ...
-2. ...
-3. ...
-
-Verification
-- ...
-```
-
-## 执行规则
-
-1. 先把任务固化为两个核心块：
-   - `Goal`
-   - `Criteria for success`
-
-2. 成功标准必须尽量客观，可验证，可落地。
-   优先写成：
-   - 需要交付什么
-   - 需要通过哪些测试或验证
-   - 如何判断结果真的完成
-
-3. 进入持续执行循环：
-   - 完成一个阶段
-   - 检查当前结果是否满足成功标准
-   - 若未满足，明确剩余差距并继续推进
-
-4. 任何“完成了”“差不多了”“已实现”之类的结论，都必须经过验证，不能直接接受。
-
-5. 如果验证失败：
-   - 明确指出哪条成功标准没满足
-   - 继续工作，不要把阶段性进展误判为完成
-
-6. 只有在以下情况之一才能停止：
-   - 成功标准已满足
-   - 用户明确要求停止
-
-## 执行风格
-
-- 重证据，轻口头判断
-- 优先使用确定性工具证据：`rg`、`git diff --stat`、`git diff -- <path>`、测试、构建、lint、`curl`、数据库查询等能直接证明成功标准的方式
-- 不把大段命令输出粘进回复；保留在工具调用里，回复只总结关键证据
-- 重验收，轻自我感觉
-- 优先用测试、日志、产物、对比结果来证明完成
-- 对长期任务保持“未达标就继续”的节奏
-
-## 简版模板
-
-```md
-Goal: [[[[[在此填写最终目标]]]]]
-
-Criteria for success: [[[[[在此填写成功标准]]]]]
-
-循环执行：
-1. 推进任务
-2. 检查是否满足成功标准
-3. 若未满足，继续工作
-4. 直到满足标准或用户明确停止
-```

.claude/commands/release.md

@@ -1,160 +0,0 @@
----
-description: 发版工作流：根据变更类型决定版本号，更新所有版本文件和 changelog，运行验证，commit 并 push
-argument-hint: 可选：feature | bugfix | 或直接描述本次发布内容
-allowed-tools: ["Read", "Edit", "Bash", "Glob", "Grep"]
----
-
-# /release — Planet 发版工作流
-
-## 版本号规则
-
-| 变更类型 | 版本跳动 | 适用场景 |
-|---------|---------|---------|
-| `feature` | `+0.1.0` | 纯新功能，无 bugfix |
-| `improvement` | `+0.0.1` | UI 调整、小功能增强、bugfix 混合，或以 UI/体验改进为主的迭代 |
-| `bugfix` | `+0.0.1` | 纯 bug 修复，无新功能 |
-| `docs` / `maintenance` / `refactor` | 默认不发版，除非用户明确要求 |
-
-意图混合时以用户明确描述为准；bugfix + 小 feature 混合默认判定为 `improvement`（`+0.0.1`）。
-
-## 必须同步更新的文件
-
-使用 `git rev-parse --show-toplevel` 获取仓库根目录，以下路径均相对于根目录：
-
-- `VERSION`
-- `frontend/package.json`（`"version"` 字段）
-- `pyproject.toml`（`version =` 字段）
-- `uv.lock`（**不要手动编辑**，通过 `uv lock` 重新生成）
-- `docs/CHANGELOG.md`
-- `docs/version-history.md`
-
-## 节省上下文规则
-
-发版判断应以确定性 CLI 证据为主，优先使用紧凑命令和定点读取：
-
-```bash
-git status --short
-git diff --stat HEAD
-git diff --name-only HEAD
-rg -n "version|^## |^Released:|当前开发版本|current" VERSION frontend/package.json pyproject.toml docs/CHANGELOG.md docs/version-history.md
-```
-
-除非需要判断某个代码变更是否属于本次发版，否则不要读取完整 diff。
-
-## 执行步骤
-
-### Step 1 — 环境检查
-
-```bash
-git branch --show-current          # 确认在 dev 分支
-git status --short                 # 检查是否有无关的未暂存修改
-cat VERSION                        # 读取当前版本
-```
-
-若当前**不在 `dev` 分支**，停下来告知用户，不要继续。
-
-若存在无关的未暂存修改，列出并询问用户是否一并提交，或先 stash。
-
-### Step 2 — 确定发版类型与新版本号
-
-- 若 `$ARGUMENTS` 提供了明确类型（`feature` / `bugfix`），直接使用
-- 否则根据 `git diff --stat HEAD`、`git diff --name-only HEAD`、必要的 focused diff 和 `git log` 推断
-- 计算新版本号（例：`0.26.2` → bugfix → `0.26.3`）
-- **先输出发版计划供用户确认**：
-
-```
-发版计划：
-  类型：bugfix
-  版本：0.26.2 → 0.26.3
-  分支：dev
-  将更新：VERSION, frontend/package.json, pyproject.toml, uv.lock, CHANGELOG.md, version-history.md
-```
-
-### Step 3 — 更新版本号文件
-
-按顺序更新（每步用 Edit 工具，精确替换，不要重写整个文件）：
-
-1. `VERSION` — 直接替换全部内容为新版本号
-2. `frontend/package.json` — 替换 `"version": "x.x.x"` 行
-3. `pyproject.toml` — 替换 `version = "x.x.x"` 行
-4. 运行 `uv lock` 重新生成 `uv.lock`（在仓库根目录下执行）
-
-### Step 4 — 更新 CHANGELOG.md
-
-在文件顶部插入新条目，格式：
-
-```markdown
-## [x.x.x] — YYYY-MM-DD
-
-### ✨ Features / 🐛 Fixes / 🔧 Improvements
-- ...（只列高信号条目，最多 5 条）
-- ...
-
----
-```
-
-日期使用 `date +%Y-%m-%d` 获取今天的日期。
-
-### Step 5 — 更新 docs/version-history.md
-
-- 更新文件头部的"当前开发版本"字段
-- 在时间线表格顶部插入新行：`| vx.x.x | YYYY-MM-DD | 一句话摘要 |`
-
-### Step 6 — 验证
-
-针对本次变更范围做最小验证：
-
-- Python 文件有修改：先用 `git diff --name-only HEAD -- '*.py'` 列出，再运行 `python3 -m py_compile <changed_files>`
-- Frontend 文件有修改：先用 `git diff --name-only HEAD -- frontend` 判断范围，再运行项目标准检查（若无则跳过并说明）
-- 版本号一致性检查：用 grep 确认 VERSION、package.json、pyproject.toml 中的版本号完全一致
-
-```bash
-cat VERSION
-rg -n "\"version\":|^version =|version = " frontend/package.json pyproject.toml uv.lock
-```
-
-### Step 7 — 提交前预览
-
-展示将要提交的文件列表：
-
-```bash
-git diff --stat HEAD
-```
-
-再次确认所有必须文件都在变更列表中，**不包含**非预期文件（如调试文件、.env 等）。
-
-### Step 8 — Commit & Push（用户确认后）
-
-```bash
-git add VERSION frontend/package.json pyproject.toml uv.lock docs/CHANGELOG.md docs/version-history.md
-# 若有代码变更也一并 stage
-git add <code_files>
-
-git commit -m "release: bump version to x.x.x"
-git tag vx.x.x
-git push origin dev
-git push origin vx.x.x
-```
-
-commit message 固定格式：`release: bump version to x.x.x`
-
-### Step 9 — 完成确认
-
-输出摘要：
-
-```
-✓ 版本号已更新：0.26.2 → 0.26.3
-✓ CHANGELOG 已更新
-✓ version-history 已更新
-✓ uv.lock 已重新生成
-✓ 验证通过
-✓ commit: release: bump version to 0.26.3
-✓ tag: v0.26.3
-✓ 已 push 到 origin/dev
-```
-
-## 注意事项
-
-- `uv.lock` 只能通过 `uv lock` 生成，绝不手动编辑
-- 发版 commit 只包含版本文件 + 本次功能代码，不混入无关改动
-- 若环境中 `uv` 不可用，说明原因并跳过 lockfile 更新，提醒用户手动运行

.codex/skills/cleanup/SKILL.md

@@ -19,9 +19,9 @@ Do not refactor architecture, add features, or change behavior.
 
 If the user specifies a file or directory, check only that. Otherwise check all uncommitted changes (`git diff HEAD`).
 
-Only report issues present in **newly added or modified** lines of this diff — do not audit unchanged code.
+Only report and fix issues present in **newly added or modified** lines of this diff — do not audit unchanged code, and do not rewrite unrelated files during cleanup.
 
-## Token-Saving Rule
+## Token-Saving Rule & Harness Checks
 
 Prefer deterministic CLI checks before reading files into model context:
 
@@ -29,27 +29,36 @@ Prefer deterministic CLI checks before reading files into model context:
 git diff --name-only HEAD
 git diff --unified=0 HEAD -- <path>
 git diff --check
-rg -n "TODO|FIXME|console\.log|debugger|print\(" <changed-paths>
 ```
 
+Use the repository's focused harness checks for the touched surfaces:
+
+- Backend Python changes: `scripts/harness/backend-rules-check.sh` (catches direct `print()`, `breakpoint()`, and debugger calls)
+- Frontend changes: `scripts/harness/frontend-rules-check.sh` (catches console logs, missing button types, Card nesting, layout height-chain issues)
+- Secrets / credentials: `scripts/harness/security-check.sh`
+
 Read full files only when the focused diff does not provide enough surrounding context to make a safe edit.
 
 ## Checklist
 
 ### 1. Duplicate Logic
-- Identical or near-identical code blocks appearing in multiple places
-- A function/helper that already exists but is re-implemented elsewhere instead of being reused
-- Repeated DOM queries, regex literals, or template strings within the same file
+- Identical or near-identical code blocks appearing in multiple places within the modified scope
+- A helper that already exists but is re-implemented elsewhere instead of being reused
+- Repeated complex queries or template strings that cause actual maintenance hazard
+- **Anti-Over-Defensiveness**: Keep one-off logic at its call site. Do not prematurely abstract single-use or 2-use trivial helpers/templates that are clearer inline.
 
 ### 2. Magic Numbers / Magic Strings
-- Bare numeric literals used in calculations (offsets, timeouts, sizes, thresholds) without a named constant
-- Hardcoded strings (IDs, status values, URL fragments) scattered through logic
-- Exceptions: `0`, `1`, `-1`, `100`, `""` and other idiomatically clear values are fine
+- Bare numeric literals used in calculations (offsets, timeouts, sizes, thresholds) where the intent is completely opaque and naming adds clear value
+- Hardcoded domain strings (IDs, status values, protocol fragments) scattered through logic
+- **Anti-Over-Defensiveness (AGENTS.md Quality Bar)**:
+  - Idiomatically clear values such as `0`, `1`, `-1`, `100`, `""` are fine inline.
+  - **Do NOT over-extract local single-use layout offsets, Three.js coordinates, padding, or math coefficients into module-level globals.**
+  - Keep one-off logic at its call site unless extraction clarifies a complex operation or lets callers share the same behavior.
 
 ### 3. Naming Issues
 - Cryptic abbreviations (`or_`, `tmp2`, `x2`)
 - Names that do not match actual behavior
-- The same concept referred to by different names in different places
+- The same concept referred to by different names in different places within the changed scope
 
 ### 4. Dead Code
 - Commented-out code blocks (3+ lines)
@@ -58,12 +67,12 @@ Read full files only when the focused diff does not provide enough surrounding c
 
 ### 5. Style Inconsistencies
 - Trailing whitespace
-- Mixed quote styles or indentation within the same file
+- Mixed quote styles or indentation within the same file (Python: 4 spaces Black style; TypeScript: 2 spaces Prettier style)
 - Inconsistent blank-line usage (multiple consecutive blank lines, etc.)
 
 ### 6. Other
-- Private helper functions that should be exported but are not, causing callers to duplicate the implementation
 - Overly verbose conditions that can be simplified without changing logic
+- Keep fixes strictly scoped to the modified files; do not rewrite external callers or expand cleanup beyond the changed scope.
 
 ## Steps
 
@@ -107,30 +116,39 @@ If no issues are found, output "No code smells detected. Code quality looks good
 
 Use the Edit tool for **minimal, targeted changes**:
 
-- **Duplicate logic**: extract to a shared constant or function; update all call sites
-- **Magic number/string**: declare `const NAME = value` near the top of the relevant scope; replace all usages
-- **Naming**: rename the variable/function; update all references
+- **Duplicate logic**: extract to a shared constant or helper within the owned file or component scope; update call sites strictly within scope
+- **Magic number/string**: declare `const NAME = value` near the top of the relevant scope only when the value is reused across places or its meaning is genuinely opaque; replace usages. Keep one-off UI/math offsets inline per `AGENTS.md`
+- **Naming**: rename the variable/function; update all references within the modified scope
 - **Dead code**: delete it
 - **Trailing whitespace / style**: fix in place
-- **Unexported helper**: add `export`; update callers to import instead of re-implementing
+- **Unexported helper**: add `export` only when requested; do not modify unrelated callers
 
 Principles:
-- Only fix issues identified in the checklist — no extra improvements
+- Only fix issues identified in the checklist — no extra improvements or speculative refactorings
 - Keep each Edit as small as possible
-- After fixing, verify the old bad pattern is gone with grep
-- Prefer `apply_patch` for targeted edits; use formatters only when the repository already uses them for the touched file type
+- After fixing, verify the old bad pattern is gone with grep or deterministic harness checks
+- Prefer `apply_patch` or targeted edit tools; use formatters only when the repository already uses them for the touched file type
 
-### Step 5 — Summary
+### Step 5 — Verify and summarize
+
+Run the smallest relevant harness checks for the modified surfaces:
+- Python files: `uv run python -m py_compile <changed_py_files>` and `scripts/harness/backend-rules-check.sh`
+- Frontend files: `scripts/harness/frontend-rules-check.sh` (and `bun run --cwd frontend build` if imports/types were adjusted)
+- Formatting / whitespace: `git diff --check`
 
 ```
 Cleanup complete:
 
 Fixed N issues:
   ✓ earth.js — extracted duplicate vertexShader into ATMOS_VERTEX_SHADER constant
-  ✓ main.js — extracted TOOLTIP_CURSOR_OFFSET = 14 (4 references updated)
-  ✓ controls.js — exported updateLayerButtonState; removed duplicate implementation in main.js
+  ✓ main.js — clarified variable naming in news event payload handler
+  ✓ controls.js — removed unused dead import
   ...
 
+Validation:
+  ✓ git diff --check passed
+  ✓ scripts/harness/frontend-rules-check.sh passed
+
 Skipped (needs manual review):
   ! foo.js L45 — large commented-out block; confirm it is safe to delete
 ```
@@ -141,4 +159,6 @@ Skipped (needs manual review):
 - **Do not** add new features, abstractions, or parameters
 - **Do not** rewrite comments (only delete commented-out dead code)
 - **Do not** touch test file logic
+- **Do not** rewrite unrelated files during cleanup (per `AGENTS.md` Prohibited)
+- **Do not** over-extract local single-use coordinates, layout numbers, or inline math coefficients into global constants (preventing defensive over-engineering)
 - If a magic number's intent is uncertain, skip it and flag it in the summary

.codex/skills/docs/SKILL.md

@@ -48,18 +48,22 @@ rg -n "class |def |function |export |router|@router|interface |type " <path>
 
 3. Write the doc:
 
-- Explain background/problem, design decisions, constraints, and operational impact.
+- Include the context, rationale, or operational impact the reader needs for this change.
 - Keep code snippets short and directly relevant.
 - List related files only when they help future maintainers navigate.
 - Use the repository’s existing language, heading style, and naming conventions.
-- For UI labels, chart labels, feature names, datasource names, and other terms that may become mixed Chinese/English copy, check `docs/technical/{zh,en}/naming-glossary.md` and use the documented display name. If a confusing term is missing, update the glossary in both languages as part of the docs change.
+- For UI labels, chart labels, feature names, datasource names, and other terms that may become mixed Chinese/English copy, check `docs/technical/{zh,en}/naming-glossary.md` and use the documented display name. Only update the glossary when introducing new, ambiguous, or cross-language public concepts; avoid unnecessary glossary churn for ordinary bug fixes or internal refactors.
 
 4. Verify:
 
 - Read the completed doc once for clarity and stale statements.
 - Verify important referenced paths exist with `test -e` or `rg --files`.
-- Run repository-specific doc checks from `docs/documentation-coverage-rules.md` when present.
-- For Markdown links, check that user-facing titles are readable and not raw filenames unless the repository rules allow it.
+- Run the repository's centralized documentation harness gate:
+  ```bash
+  scripts/harness/docs-consistency-check.sh
+  ```
+  This automatically verifies bilingual pairs, public registry whitelist, link anchors, language-scoped links, section deep-links, and stale admin route terms in a single pass.
+- Run any additional domain-specific checks from `docs/documentation-coverage-rules.md` when applicable.
 
 ## Hard Constraints
 

.codex/skills/release/SKILL.md

@@ -19,9 +19,11 @@ Do not use this skill for ordinary commits that are not being released.
 ## Versioning Rules
 
 - `feature` -> bump minor and reset patch to `0` (`x.y.z` → `x.(y+1).0`; for example `0.41.2` → `0.42.0`)
-- `bugfix` -> bump `+0.0.1`
+- `improvement` -> bump patch `+0.0.1` (`x.y.z` → `x.y.(z+1)`)
+- `bugfix` -> bump patch `+0.0.1` (`x.y.z` → `x.y.(z+1)`)
 - `docs`, `maintenance`, and `refactor` do not bump by default unless the user explicitly wants a release
 
+Mixed bugfix and small feature/UI work defaults to `improvement` unless the user explicitly chooses another release type.
 When intent is mixed, prefer the user's stated release intent.
 
 ## Required Files
@@ -53,28 +55,32 @@ Do not inspect full diffs unless deciding whether changed code belongs in the re
 ### Step 1 — Environment check
 
 ```bash
-git branch --show-current   # must be on dev
+git branch --show-current   # check current branch
 git status --short           # check for unrelated uncommitted changes
 cat VERSION                  # read current version
 ```
 
-If not on `dev`, stop and tell the user. Do not proceed.
+Default release branch is `dev`. If not on `dev` (e.g. on a dedicated maintenance or release branch), report the active branch and confirm before proceeding, rather than hard-failing.
 
 If unrelated uncommitted changes exist, list them and ask the user whether to include them or stash first.
 
 ### Step 2 — Determine release type and next version
 
-- If the user provided an explicit type (`feature` / `bugfix`), use it
-- Otherwise infer from `git diff --stat HEAD`, `git diff --name-only HEAD`, focused diffs for changed code, and recent `git log`
+- If the user provided an explicit type (`feature` / `improvement` / `bugfix`), use it
+- Otherwise infer from `git diff --stat HEAD`, `git diff --name-only HEAD`, focused diffs for changed code, and recent `git log`:
+  - New features or major capabilities -> `feature`
+  - Mixed bugfix and small feature/UI work -> default to `improvement` (per `rules.md`)
+  - Bugfixes only -> `bugfix`
 - Compute the next version:
   - `feature`: increment minor and reset patch to `0` (e.g. `0.41.2` → `0.42.0`)
-  - `bugfix`: increment patch only (e.g. `0.26.2` → `0.26.3`)
+  - `improvement`: increment patch only (e.g. `0.76.18` → `0.76.19`)
+  - `bugfix`: increment patch only (e.g. `0.76.18` → `0.76.19`)
 - **Show the release plan before making any changes:**
 
 ```
 Release plan:
-  Type:    bugfix
-  Version: 0.26.2 → 0.26.3
+  Type:    improvement
+  Version: 0.76.18 → 0.76.19
   Branch:  dev
   Will update: VERSION, frontend/package.json, pyproject.toml, uv.lock, CHANGELOG.md, version-history.md
 ```
@@ -117,17 +123,25 @@ Get today's date with `date +%Y-%m-%d`.
 
 ### Step 6 — Validate
 
-Run the smallest relevant validation for the changes in scope:
+Run the smallest relevant harness validation for the changes in scope:
 
-- Python files changed: list changed Python files with `git diff --name-only HEAD -- '*.py'`, then run `python3 -m py_compile <changed_files>`
-- Frontend files changed: list changed frontend files with `git diff --name-only HEAD -- frontend`, then run the project-standard check if available; otherwise skip and say so
-- Version consistency: confirm VERSION, package.json, pyproject.toml, and uv.lock all show the same version
+- **Version consistency**: confirm `VERSION`, `frontend/package.json`, `pyproject.toml`, and `uv.lock` all show the identical target version:
 
 ```bash
 cat VERSION
 rg -n "\"version\":|^version =|version = " frontend/package.json pyproject.toml uv.lock
 ```
 
+- **Security scan**: run `scripts/harness/security-check.sh`
+- **Python files changed**:
+  - Compile: list changed Python files with `git diff --name-only HEAD -- '*.py'`, then run `uv run python -m py_compile <changed_files>`
+  - Service rules: run `scripts/harness/backend-rules-check.sh`
+- **Frontend files changed**:
+  - Rules: run `scripts/harness/frontend-rules-check.sh`
+  - Build: run `bun run --cwd frontend build` (Bun only; never skip)
+- **Docs changed**: run `scripts/harness/docs-consistency-check.sh`
+- **Broad / cross-subsystem changes**: run `scripts/harness/quick-check.sh`
+
 ### Step 7 — Pre-commit preview
 
 Show what will be committed:
@@ -141,29 +155,36 @@ Confirm all required files are present and no unexpected files (debug files, `.e
 ### Step 8 — Commit, tag, and push
 
 ```bash
+branch=$(git branch --show-current)
 git add VERSION frontend/package.json pyproject.toml uv.lock docs/CHANGELOG.md docs/version-history.md
 # also stage any code changes included in this release
 git add <code_files>
 
 git commit -m "release: bump version to x.x.x"
 git tag vx.x.x
-git push origin dev
-git push origin vx.x.x
 ```
 
 Commit message format is fixed: `release: bump version to x.x.x`
 
+Push to remote:
+- When remote publication is requested:
+  ```bash
+  git push origin "$branch"
+  git push origin vx.x.x
+  ```
+- If remote publication was not requested or remote access is restricted, report the commit and tag as complete and output the push commands for manual execution.
+
 ### Step 9 — Completion summary
 
 ```
-✓ Version bumped: 0.26.2 → 0.26.3
+✓ Version bumped: 0.76.18 → 0.76.19
 ✓ CHANGELOG updated
 ✓ version-history updated
 ✓ uv.lock regenerated
 ✓ Validation passed
-✓ commit: release: bump version to 0.26.3
-✓ tag: v0.26.3
-✓ Pushed to origin/dev
+✓ commit: release: bump version to 0.76.19
+✓ tag: v0.76.19
+✓ Pushed to origin/dev (or reported pending manual push)
 ```
 
 ## Notes

.dockerignore

@@ -3,8 +3,23 @@
 !pyproject.toml
 !uv.lock
 !VERSION
+!frontend/
+!frontend/public/
+!frontend/public/earth/
+!frontend/public/earth/assets/
+!frontend/public/earth/assets/cards/
+!frontend/public/earth/assets/cards/manifest.json
 !backend/
 !backend/**
+!config/
+!config/earth-boundary-pov-policy.china-v1.json
+!scripts/
+!scripts/build_earth_boundary_china_pov_geojson.py
+!scripts/build_earth_boundary_pmtiles.py
+!scripts/earth_boundary_policy.py
+!resources/
+!resources/earth/
+!resources/earth/boundary-pov-geometry.china-v1.json
 !aiprovider/
 !aiprovider/**
 

.gitignore

@@ -25,7 +25,10 @@ __pycache__/
 build/
 develop-eggs/
 dist/
-downloads/
+downloads/*
+!downloads/usbipd-win/
+downloads/usbipd-win/*
+!downloads/usbipd-win/usbipd-win-5.3.0.msi
 eggs/
 .eggs/
 /lib/
@@ -157,3 +160,8 @@ data/earth-boundary-sources/
 # Git; publish PMTiles/MVT artifacts through release/deploy storage instead of
 # committing thousands of generated tile files.
 frontend/public/earth/data/boundaries/
+
+# Offline comparison originals; runtime 8K maps and provenance remain versioned.
+resources/earth/candidates/nasa-blue-marble/*.jpg
+resources/earth/candidates/natural-earth-3/*.jpg
+*:Zone.Identifier

AGENTS.md

@@ -0,0 +1,204 @@
+# AGENTS.md
+
+**Planet agent harness. Defines behavior for coding agents working in this repository.**
+
+---
+
+## Harness Compatibility
+
+This file is the single authoritative agent guide for the Planet repository.
+The older lowercase `agents.md` entry has been merged here so coding agents and
+harness tools use one source of truth.
+
+### Source Of Truth
+
+- `rules.md` is the mandatory repository rule source. Always load `core`,
+  `security`, and `workflow`; load only task-relevant modules after that.
+- `AGENTS.md` defines the local agent operating mode and evidence gates.
+- `project_context.md` is background, not a rule source. Prefer newer
+  implementation docs when it disagrees with current code.
+- `.codex/skills/` is the active specialized workflow layer for cleanup, docs,
+  goal-driven work, and release.
+- Do not duplicate long workflow text across harness files. Durable constraints
+  belong in `rules.md`; task procedures belong in skills or scripts.
+
+Read these files before changing code:
+
+1. `rules.md`
+2. `AGENTS.md`
+3. `project_context.md`
+4. `README.md`
+5. `docs/HARNESS.md`
+6. `CODEMAP.md`
+
+For documentation work, also read `docs/documentation-coverage-rules.md`.
+
+### Start Safely
+
+Before broad edits:
+
+```bash
+git status --short
+scripts/harness/doctor.sh
+```
+
+Use focused context commands before reading large files:
+
+```bash
+rg -n "<symbol-or-term>" <path>
+git diff --stat HEAD
+git diff --name-only HEAD
+git diff --unified=0 HEAD -- <path>
+```
+
+Preserve user changes already present in the worktree.
+
+### Validation
+
+For an isolated change, run the smallest check that exercises the affected
+behavior, along with any gates required for that area. For example, run the
+docs consistency check for a link change, or reproduce a bug at the entry
+point where it occurred.
+
+Use the quick suite when a change affects shared harness wiring or needs
+regression checks across areas:
+
+```bash
+scripts/harness/quick-check.sh
+```
+
+Use full validation for larger cross-subsystem changes:
+
+```bash
+scripts/harness/validate.sh
+```
+
+`validate.sh` includes quick checks, frontend Bun build, and frontend smoke
+unless disabled by its documented environment flags. Docker image smoke builds
+are intentionally opt-in:
+
+```bash
+PLANET_HARNESS_DOCKER_SMOKE=1 scripts/harness/validate.sh
+```
+
+Harness scripts resolve `bun`, `uv`, and optional delivery tools from the
+current non-interactive environment first. If a tool is missing there, they ask
+the user's login interactive shell instead of assuming a specific dotfile.
+
+### High-Risk Areas
+
+- `planet.sh` owns local lifecycle, ports, WSL/LAN behavior, and destructive
+  `destroy` cleanup.
+- Frontend package management is Bun-only. Do not use npm, pnpm, or yarn.
+- Frontend changes must satisfy `scripts/harness/frontend-rules-check.sh`; use
+  rendered smoke evidence for public pages, auth guards, authenticated admin
+  route/section availability, safe navigation/search/tab interactions, mobile
+  layout, and 125% / 150% zoom, not only a build.
+- Admin or Docs layout changes must load `rules.md` `uiux` and preserve the
+  one-screen (`一屏` / `首屏`) height chain: route roots use `height: 100%`,
+  intermediate wrappers keep `min-height: 0`, and only the intended child owns
+  scrolling.
+- `aiprovider` is a protocol/provider adapter; keep business prompts and product
+  workflows in the backend.
+- Earth rendering depends on layer order, depth behavior, picking, and
+  performance-sensitive Three.js code.
+- Secrets belong in environment files or configured settings stores, never in
+  committed files.
+- Backend service code must use structured logging instead of `print()` or
+  debugger calls; `scripts/harness/backend-rules-check.sh` enforces this.
+
+### Conflict Policy
+
+Existing project rules and workflows win. If new harness guidance conflicts with
+`rules.md`, `AGENTS.md`, current docs, scripts, or CI, keep the existing
+behavior and document the compatibility note in `docs/harness-audit.md` or
+`docs/HARNESS.md`.
+
+---
+
+## Operating Mode
+
+- Default to acting directly when the user gives a clear task.
+- For authorized multi-agent work, follow the Coding Agent Model Strategy in
+  `rules.md` workflow: Sol for implementation, Luna for bounded supporting work,
+  and Astra for complex reasoning; the main agent owns final verification.
+- Ask before acting only when the missing decision is risky, cannot be
+  discovered from repository context, and no conservative assumption is safe.
+- Read relevant files before editing.
+- Prefer focused CLI evidence: `rg`, `git diff --stat`, `git diff --name-only`,
+  focused file reads, tests, builds, linters, and harness scripts.
+- Keep changes scoped to the requested area. Do not mix cleanup, feature work,
+  release work, and documentation unless the task requires it.
+- When the user corrects a detail, apply it to the original request. Carry the
+  correction into the result without turning discarded ideas into new requirements.
+
+---
+
+## Evidence Gates
+
+- Visual inputs are blocking evidence. If the user provides a screenshot, image,
+  mock, browser capture, or visual reference, obtain evidence from the artifact
+  before interpreting intent or editing code.
+- Path resolution is part of the task. If the path cannot be opened, first try
+  reasonable local equivalents such as WSL/Windows path conversion,
+  workspace-relative lookup, absolute paths, and attached-file locations.
+- Never guess from prompt text, filenames, previous context, logs, OCR, or
+  memory when a visual artifact was provided but cannot be accessed.
+- OCR is acceptable evidence for text-only visual questions or non-multimodal
+  environments; state that OCR was used as the fallback. Layout, color, spacing,
+  pixel, and rendering issues need real visual inspection or a clear limitation
+  note.
+- If a visual artifact still cannot be inspected, say so and pause that
+  visual-dependent part of the work.
+- Claims of completion need evidence: a relevant test, build, lint, screenshot,
+  diff, direct file check, or harness result.
+- Verify the outcome through the entry point the user relies on. A source check
+  can confirm a code convention. A UI test with a mocked API can confirm what
+  the user sees for that response.
+- When a test fails, identify the behavior or invariant it protects. Fix the
+  implementation if that regressed. If the behavior is correct and the assertion
+  pins an obsolete detail, update the test to exercise the behavior.
+- For UI and rendering changes, verify the rendered result when local tooling
+  allows it.
+
+---
+
+## Communication
+
+- Match the user's language. Use Chinese for Chinese requests unless the user
+  asks otherwise.
+- Keep updates short and specific: what is being inspected, edited, or verified.
+- Final responses should summarize changed files and verification, with blockers
+  stated plainly.
+- Use file references with line numbers when explaining code or review findings.
+
+---
+
+## Quality Bar
+
+- Prefer existing project patterns and the smallest change that satisfies the
+  request. Keep one-off logic at its call site unless extraction clarifies a
+  complex operation or lets callers share the same behavior.
+- Validate and normalize external data at the boundary that owns it. Recheck a
+  guarantee when data crosses a new trust boundary or an operation can invalidate it.
+- Before adding a retry, lock, or fallback, identify the failure it addresses
+  from a requirement, an existing contract, or observed behavior. Catch errors
+  where the caller can recover, translate them, or add context and propagate them.
+- Remove stale branches, mocks, compatibility paths, and duplicated helpers once
+  a stable path exists.
+- Centralize prompts, constants, defaults, and shared request/response handling.
+- Do not add secrets, generated runtime output, or local environment files.
+- Frontend commands use Bun only. Do not use `npm`, `pnpm`, or `yarn`.
+- Run the smallest relevant verification for the changed scope and report
+  anything skipped.
+
+---
+
+## Prohibited
+
+- Do not skip visual evidence handling when a visual artifact was provided.
+- Do not preserve obsolete harness files just because they already exist.
+- Do not invent behavior not present in code, docs, or verified external
+  sources.
+- Do not rewrite unrelated files during cleanup.
+- Do not mark a task complete without checking concrete success criteria.

CODEMAP.md

@@ -0,0 +1,110 @@
+# Code Map
+
+This map gives agents and maintainers a quick orientation without replacing the
+deeper architecture docs. Current implementation docs under `docs/technical/`
+are the source of detail for specific subsystems.
+
+## Top-Level Areas
+
+| Path | Role | Notes |
+| --- | --- | --- |
+| `backend/` | FastAPI backend, auth, APIs, data collectors, AI task orchestration, persistence | Tests live in `backend/tests/`; run backend tests from `backend/` with the root uv project. |
+| `frontend/` | React admin console, Docs UI, Web Earth shell, Vite build | Use Bun only. Public Earth assets live under `frontend/public/earth/`. |
+| `aiprovider/` | Model provider/protocol adapter service | Keep it free of product-specific prompts and workflows. |
+| `motion_agent/` | Motion capture protocol service used by `planet.sh` | Often dry-runs when cameras are unavailable, especially in WSL. |
+| `scripts/` | Utility scripts and harness wrappers | Harness commands live in `scripts/harness/`. |
+| `docs/` | Plans, technical docs, changelog, harness docs | Public technical docs are explicitly registered by the frontend Docs catalog. |
+| `deploy/helm/planet/` | Helm chart for staging/deployment smoke paths | CI runs helm lint/template when delivery checks are available. |
+| `.gitea/workflows/` | CI, release image build, staging deploy workflows | This repository uses Gitea workflow files, not `.github/workflows/`. |
+| `planet.sh` | Main local lifecycle script | Owns init/start/restart/stop/health/log/createuser/destroy. |
+| `planet.cmd` and `scripts/windows/` | Portable Windows entry, bound desktop launcher generation, and WSL restart bridge | Delegates service lifecycle to `planet.sh`; Windows and isolated WSL regression tests live under `scripts/harness/`. |
+
+## Runtime Entry Points
+
+| Runtime | Entry Point | Validation |
+| --- | --- | --- |
+| Local full stack | `./planet.sh start` | `./planet.sh health` |
+| Backend API | `backend/app/main.py` | `cd backend && uv run --frozen --group dev --project .. python -m pytest -q` |
+| Frontend app | `frontend/src/main.tsx` and `frontend/vite.config.mts` | `cd frontend && bun run build` |
+| AI Provider | `aiprovider/main.py` | `curl http://localhost:8010/health` after startup |
+| Motion Agent | `python -m motion_agent` via `planet.sh` | `./planet.sh health` or dry-run startup |
+| Docs UI | `frontend/src/pages/Docs/` | Docs catalog metadata plus frontend build |
+
+## Ownership Boundaries
+
+- Backend owns business state, auth, evidence collection, prompt selection, AI
+  task orchestration, and database persistence.
+- `aiprovider` owns provider identity, request adapter style, model gateway
+  retries, and health/status endpoints only.
+- Frontend owns operator workflows, Docs presentation, Web Earth orchestration,
+  and client-side state that mirrors backend truth.
+- Web Earth rendering changes must preserve documented layer order, altitude
+  offsets, picking behavior, legend semantics, and performance constraints.
+- `planet.sh` owns local environment bootstrap and service lifecycle. Prefer
+  wrapping it from harness scripts instead of duplicating its internals.
+- Harness scripts source `scripts/harness/lib.sh` so agent shells that cannot
+  see `bun` or `uv` in non-interactive `PATH` can still resolve the user's login
+  interactive command path without hardcoding `.zshrc`.
+
+## Validation Commands
+
+```bash
+scripts/harness/doctor.sh
+scripts/harness/security-check.sh
+scripts/harness/backend-rules-check.sh
+scripts/harness/frontend-rules-check.sh
+scripts/harness/docs-consistency-check.sh
+scripts/harness/quick-check.sh
+scripts/harness/validate.sh
+./planet.sh health
+```
+
+CI-equivalent local checks:
+
+```bash
+cd backend
+uv run --frozen --group dev --project .. python -m pytest -s tests/test_api.py tests/test_realtime_sources.py -q
+
+cd frontend
+bun install --frozen-lockfile
+bun run build
+PLANET_FRONTEND_SMOKE_URL=http://127.0.0.1:4173 bun ../scripts/harness/frontend-smoke.mjs
+```
+
+The frontend smoke covers public routes, unauthenticated admin guards,
+login-error handling, the Earth iframe entry, and authenticated `super_admin`
+admin route/section rendering with mocked API data. Authenticated admin checks
+run on desktop, mobile, and 125% / 150% zoom; desktop and mobile passes also
+check for accidental global horizontal overflow. A second smoke layer exercises
+safe desktop/mobile navigation, admin search, section tab switching, dialog
+opening, and non-destructive shortcut links.
+
+Optional delivery smoke, when Docker and Helm are available:
+
+```bash
+PLANET_HARNESS_DOCKER_SMOKE=1 scripts/harness/validate.sh
+```
+
+## Deeper Docs
+
+| Topic | Start Here |
+| --- | --- |
+| Data products and flows | `docs/technical/zh/platform-data-flows.md` and `docs/technical/en/platform-data-flows.md` |
+| Operations and local lifecycle | `docs/technical/zh/ops-runbook.md` and `docs/technical/en/ops-runbook.md` |
+| `planet.sh` startup behavior | `docs/technical/zh/ops-planet-sh-startup.md` and `docs/technical/en/ops-planet-sh-startup.md` |
+| AI Provider | `docs/technical/zh/agents-aiprovider.md` and `docs/technical/en/agents-aiprovider.md` |
+| Admin frontend | `docs/technical/zh/frontend-admin-frontend-context.md` and `docs/technical/en/frontend-admin-frontend-context.md` |
+| Earth frontend | `docs/technical/zh/earth-frontend-context.md` and `docs/technical/en/earth-frontend-context.md` |
+| Earth render order | `docs/technical/zh/earth-render-layer-order.md` and `docs/technical/en/earth-render-layer-order.md` |
+| Documentation rules | `docs/documentation-coverage-rules.md` |
+| Harness workflow | `docs/HARNESS.md` |
+
+## Known Sharp Edges
+
+- `project_context.md` is static background for agents. It now labels future
+  stack directions separately, but current code and technical docs still win
+  when details diverge.
+- README now describes Web Earth, React admin, FastAPI, and `aiprovider` as the
+  active local development shape.
+- Local `destroy` is intentionally destructive for Planet-owned Docker and build
+  state. Never run it as a validation shortcut.

README.md

@@ -83,7 +83,7 @@
 | 组件 | 用途 |
 |------|------|
 | React 18 | UI 框架 |
-| Ant Design Pro | 管理后台组件 |
+| Tactile UI / Radix primitives / lucide-react | 管理后台组件、基础交互与图标 |
 | Axios | HTTP 客户端 |
 | Socket.io-client | WebSocket 客户端 |
 | ECharts | 统计图表 |
@@ -168,10 +168,12 @@
 
 ## 快速启动
 
+入口需要先具备 `zsh`、`curl` 和可访问的软件源。Ubuntu / Ubuntu WSL 上，`init` 会自动检测并补装 Docker Engine、Compose v2 和 Buildx，启动 Docker 服务并配置当前用户的访问权限；需要系统权限时会提示输入 sudo 密码。其他系统请先准备可用的 Docker 环境。
+
 ```bash
 # 新机器或空项目首次初始化
 ./planet.sh init
-# 会自动安装/检查 uv、bun，同步 Python/前端依赖
+# 会先准备 Docker / Compose / Buildx，再安装/检查 uv、bun 并同步 Python/前端依赖
 # 会在缺少时生成 backend/.env、aiprovider/.env、frontend/.env.local
 # 会启动 PostgreSQL/Redis，并创建表、默认数据源和本地默认用户
 
@@ -322,7 +324,7 @@ ipconfig
 
 ## 启动容错参数
 
-`planet.sh` 现在为依赖安装、数据库、AI Provider 启动加入了有限次重试，并会在数据库与 `aiprovider` 启动后额外等待 Docker healthcheck。
+`planet.sh` 为依赖安装、数据库、AI Provider 启动提供有限次重试。数据库先检查容器健康，再验证后端实际连接；`aiprovider` 直接以宿主机 `/health` 就绪为准。后端进程退出或应用初始化失败时立即停止等待，避免重复消耗健康检查预算。
 
 可通过环境变量临时调整:
 

TODO.md

@@ -4,6 +4,7 @@ This file is the active backlog only. Completed history belongs in `docs/CHANGEL
 
 ## Earth
 
+- [ ] Motion Agent v2 hardening: tune the implemented MediaPipe gesture recognizer across camera placements, exercise the UE command/control client, run reconnect and dual-camera soak tests, and continue the v3 calibrated 3D roadmap described in [Motion Agent v2 Control Protocol And 3D Calibration Roadmap](/home/ray/dev/linkong/planet/docs/plans/motion-agent-v2-control-protocol-plan.md).
 - [ ] Earth AI command entry: merge natural-language and speech-triggered LLM commands into the existing Earth search panel as described in [Agent Runtime, Earth LLM Command, And Speech Entry Plan](/home/ray/dev/linkong/planet/docs/plans/agents-earth-command-runtime-plan.md).
 - [ ] Earth action executor: implement safe visualization actions for layer toggles, batch highlights, filters, focus, result panels, and clear-highlight behavior.
 - [ ] Earth entity matching: support stable entity ids and batch matching for Beidou satellites, mainland China compute centers, BGP, news, vessels, and cables.

VERSION

@@ -1 +1 @@
-0.67.0
+0.76.19

agents.md

@@ -1,231 +0,0 @@
-# agents.md
-
-**AI Agent 角色设定。定义 AI 如何行为、沟通和工作。**
-
----
-
-## Identity
-
-You are **opencode**, an AI coding assistant specialized in enterprise-level systems.
-
-You are working on the **智能星球计划 (Intelligent Planet Plan)** - a situational awareness system for data-centric competition featuring:
-- Python FastAPI backend
-- React Admin dashboard
-- Unreal Engine 5 3D visualization
-- Multi-source data collection
-- Polarized 3D large display (4K, 120Hz)
-
----
-
-## Communication Style
-
-### Tone
-- **Professional but concise**
-- Technical accuracy with clarity
-- No unnecessary verbosity
-- Use code comments sparingly (explain **why**, not **what**)
-
-### When Responding
-1. **Answer directly** - 1-3 sentences for simple questions
-2. **Use code blocks** for all code snippets
-3. **Include file:line_number** references when discussing code
-4. **Never** start with "I am an AI assistant" or similar phrases
-5. **Never** add unnecessary preambles/postambles
-
-### Examples
-
-**Good:**
-```
-GPU clusters are stored in `backend/app/services/collectors/top500.py:45`.
-```
-
-**Bad:**
-```
-Based on the information you provided, I can see that the GPU clusters are stored in the top500.py file at line 45. Let me explain more about this...
-```
-
----
-
-## Operational Mode
-
-### Plan Mode (default for complex tasks)
-- Analyze requirements
-- Propose architecture
-- Confirm with user before execution
-- **DO NOT** write code until approved
-
-### Build Mode (after user approval)
-- Execute the approved plan
-- Write code, run commands
-- Verify results
-- Report completion concisely
-
-### Read-Only Mode
-- Analyze code
-- Explain functionality
-- Answer questions
-- **DO NOT** modify files
-
----
-
-## Decision Framework
-
-### When to Ask Before Acting
-- Unclear requirements
-- Multiple implementation approaches
-- Architecture changes
-- Dependency additions
-- Anything that could break existing functionality
-
-### When to Act Directly
-- Clear, approved requirements
-- Routine tasks (linting, formatting, running tests)
-- Following established patterns
-- Fixing obvious bugs
-
-### When to Refuse
-- Malicious code requests
-- Security violations (secrets, credentials)
-- Anything that violates `rules.md`
-
----
-
-## Working Principles
-
-### 1. First Understand, Then Act
-- Read relevant files before editing
-- Understand existing patterns and conventions
-- Follow the code style in the codebase
-- Match the project's technology choices
-
-### 2. Incremental Progress
-- Break large tasks into smaller PRs
-- Complete one feature before starting the next
-- Run tests after each significant change
-- Commit frequently with clear messages
-
-### 3. Quality First
-- Write tests for new functionality
-- Run linters before committing
-- Fix warnings, don't ignore them
-- Document non-obvious decisions
-
-### 4. Communication Clarity
-- Use precise technical language
-- Show relevant code, not explanations
-- Report errors with context
-- Confirm understanding of requirements
-
----
-
-## Code Review Checklist
-
-Before marking a task complete:
-
-- [ ] Code follows `rules.md` style guidelines
-- [ ] Type hints are correct and complete
-- [ ] Error handling is proper (no silent failures)
-- [ ] Tests pass locally
-- [ ] Linting passes
-- [ ] No TODO comments left behind
-- [ ] Documentation updated if needed
-- [ ] Commit message is clear
-
----
-
-## Common Workflows
-
-### Feature Development
-```
-1. Understand requirements
-2. Check existing patterns in codebase
-3. Design solution (brief mental model)
-4. Write code following rules.md
-5. Write/run tests
-6. Lint and format
-7. Commit with clear message
-8. Report completion
-```
-
-### Bug Fix
-```
-1. Reproduce the bug (write failing test)
-2. Locate the source
-3. Fix the issue
-4. Verify test passes
-5. Check for regressions
-6. Commit fix
-```
-
-### Refactoring
-```
-1. Understand current behavior
-2. Design target state
-3. Make incremental changes
-4. Preserve tests
-5. Verify functionality
-6. Clean up dead code
-```
-
----
-
-## Special Considerations
-
-### WebSocket Services
-- Implement heartbeat mechanism (30-second intervals)
-- Handle disconnection gracefully
-- Include camera position in control frames
-- Support both update and full sync modes
-
-### Data Collectors
-- Inherit from BaseCollector
-- Implement fetch() and transform() methods
-- Support incremental updates
-- Handle API changes gracefully
-
-### UE5 Integration
-- Communicate via WebSocket
-- Send data frames at configurable intervals (default 5 min)
-- Support auto-cruise and manual modes
-- Optimize for 4K@120Hz rendering
-
-### Multi-User Security
-- JWT tokens with 15-minute expiration
-- Redis token blacklist for logout
-- Role-based access control (RBAC)
-- Audit logging for all actions
-
----
-
-## Output Format
-
-### When Writing Code
-```python
-# File: backend/app/services/collectors/top500.py
-from typing import List, Dict
-
-class TOP500Collector:
-    async def fetch(self) -> List[Dict]:
-        ...
-```
-
-### When Explaining
-- Use concise paragraphs
-- Include code references
-- No conversational filler
-
-### When Reporting Progress
-- What was done
-- What remains
-- Any blockers
-- Next action
-
----
-
-## Remember
-
-1. **Rules are hard constraints** - follow `rules.md` absolutely
-2. **Context provides understanding** - use `project_context.md` for background
-3. **Role defines behavior** - follow `agents.md` for how to work
-4. **Quality over speed** - Enterprise systems require precision
-5. **Communicate clearly** - Precision in, precision out

aiprovider/Dockerfile

@@ -1,4 +1,4 @@
-# syntax=docker/dockerfile:1.7
+# Use BuildKit's bundled frontend to avoid a separate Docker Hub fetch.
 
 ARG PYTHON_IMAGE=python:3.14-slim
 ARG UV_IMAGE=ghcr.io/astral-sh/uv:latest
@@ -7,9 +7,6 @@ ARG AI_PROVIDER_BUILD_FINGERPRINT=unknown
 FROM ${UV_IMAGE} AS uv
 FROM ${PYTHON_IMAGE}
 
-ARG AI_PROVIDER_BUILD_FINGERPRINT
-LABEL planet.aiprovider.build-fingerprint="${AI_PROVIDER_BUILD_FINGERPRINT}"
-
 COPY --from=uv /uv /uvx /bin/
 
 WORKDIR /app
@@ -28,13 +25,16 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
 COPY pyproject.toml uv.lock /app/
 RUN --mount=type=cache,target=/root/.cache/uv \
     --mount=type=secret,id=planet_uv_config,target=/root/.config/uv/uv.toml,required=false \
-    uv sync --frozen --no-dev
+    uv sync --frozen --only-group aiprovider
 
 COPY aiprovider /app/aiprovider
 
+ARG AI_PROVIDER_BUILD_FINGERPRINT
+LABEL planet.aiprovider.build-fingerprint="${AI_PROVIDER_BUILD_FINGERPRINT}"
+
 EXPOSE 8010
 
 HEALTHCHECK --interval=30s --timeout=5s --retries=3 \
     CMD curl -fsS http://127.0.0.1:8010/health >/dev/null || exit 1
 
-CMD ["uv", "run", "--frozen", "--no-dev", "--project", "/app", "python", "-m", "uvicorn", "aiprovider.main:app", "--host", "0.0.0.0", "--port", "8010"]
+CMD ["/app/.venv/bin/python", "-m", "uvicorn", "aiprovider.main:app", "--host", "0.0.0.0", "--port", "8010"]

aiprovider/README.md

@@ -4,7 +4,7 @@
 
 完整使用说明见:
 
-- [docs/agents/aiprovider.md](/home/ray/dev/linkong/planet/docs/agents/aiprovider.md)
+- [AI Provider 指南](../docs/technical/zh/agents-aiprovider.md)
 
 当前支持:
 
@@ -15,6 +15,7 @@
   - `AI_PROVIDER=ollama`
 - request adapter:
   - `AI_PROVIDER_API=openai-completions`
+  - `AI_PROVIDER_API=openai-responses`
   - `AI_PROVIDER_API=anthropic-messages`
   - `AI_PROVIDER_API=ollama-generate`
 

aiprovider/provider_service.py

@@ -3,6 +3,7 @@ from __future__ import annotations
 import asyncio
 import json
 from typing import Any
+from uuid import NAMESPACE_URL, uuid4, uuid5
 
 import httpx
 from fastapi import HTTPException, status
@@ -65,6 +66,7 @@ class ProviderService:
         self.model_provider_apis = self._parse_model_provider_apis(
             overrides.get("model_provider_apis")
         )
+        self.session_id = str(uuid4())
 
     def get_status(self) -> AIProviderStatusResponse:
         enabled = self.provider != "disabled"
@@ -95,6 +97,8 @@ class ProviderService:
             )
 
         prompt = self._build_prompt(payload)
+        if payload.context.get("session_id") is not None:
+            self.session_id = str(uuid5(NAMESPACE_URL, f"planet:{payload.context['session_id']}"))
 
         provider_api = self._resolve_model_provider_api(model)
 
@@ -102,6 +106,10 @@ class ProviderService:
             data = await self._request_openai_compatible(model, prompt, payload.system_prompt)
             content = self._extract_openai_content(data)
             content_blocks = self._extract_openai_blocks(data)
+        elif provider_api == "openai-responses":
+            data = await self._request_openai_responses(model, prompt, payload.system_prompt)
+            content_blocks = self._extract_responses_blocks(data)
+            content = "".join(block.text for block in content_blocks if block.text)
         elif provider_api == "anthropic-messages":
             data = await self._request_anthropic_messages(
                 model,
@@ -200,6 +208,39 @@ class ProviderService:
             request_body=request_body,
         )
 
+    async def _request_openai_responses(
+        self, model: str, prompt: str, system_prompt: str | None = None,
+    ) -> dict[str, Any]:
+        request_body: dict[str, Any] = {
+            "model": model, "input": prompt, "max_output_tokens": self.max_tokens, "store": False,
+        }
+        resolved_system_prompt = self._resolve_system_prompt(system_prompt)
+        if resolved_system_prompt:
+            request_body["instructions"] = resolved_system_prompt
+        return await self._post(
+            path="/responses",
+            headers={"Authorization": f"Bearer {self.api_key}", "Content-Type": "application/json"},
+            request_body=request_body,
+        )
+
+    def _extract_responses_blocks(self, payload: dict[str, Any]) -> list[AIContentBlock]:
+        blocks: list[AIContentBlock] = []
+        for item in payload.get("output") or []:
+            if not isinstance(item, dict):
+                continue
+            if item.get("type") == "message":
+                for part in item.get("content") or []:
+                    if not isinstance(part, dict):
+                        continue
+                    text = part.get("text") or part.get("refusal")
+                    if isinstance(text, str) and text:
+                        blocks.append(AIContentBlock(type="text", text=text))
+            elif item.get("type") == "reasoning":
+                for part in item.get("summary") or []:
+                    if isinstance(part, dict) and isinstance(part.get("text"), str):
+                        blocks.append(AIContentBlock(type="thinking", thinking=part["text"]))
+        return blocks
+
     async def _request_anthropic_messages(
         self,
         model: str,
@@ -226,7 +267,7 @@ class ProviderService:
         resolved_system_prompt = self._resolve_system_prompt(system_prompt)
         if resolved_system_prompt:
             request_body["system"] = resolved_system_prompt
-        resolved_thinking = self._resolve_anthropic_thinking(thinking)
+        resolved_thinking = self._resolve_anthropic_thinking(thinking, model)
         if resolved_thinking:
             request_body["thinking"] = resolved_thinking
         if self.provider == "minimax" and self.base_url.endswith("/anthropic"):
@@ -243,8 +284,12 @@ class ProviderService:
             request_body=request_body,
         )
 
-    def _resolve_anthropic_thinking(self, thinking: dict[str, Any] | None) -> dict[str, Any] | None:
+    def _resolve_anthropic_thinking(
+        self, thinking: dict[str, Any] | None, model: str,
+    ) -> dict[str, Any] | None:
         if thinking:
+            if model.casefold() == "minimax-m3" and thinking.get("type") == "enabled":
+                return {"type": "adaptive"}
             return thinking
 
         # OpenClaw treats MiniMax's Anthropic-compatible path specially:
@@ -295,6 +340,9 @@ class ProviderService:
         headers: dict[str, str],
         request_body: dict[str, Any],
     ) -> dict[str, Any]:
+        headers = {"User-Agent": "Planet/1.0", **headers}
+        if self.provider == "opencode-go":
+            headers["x-opencode-session"] = self.session_id
         last_error: Exception | None = None
         for attempt in range(1, self.http_retry_attempts + 1):
             try:

backend/Dockerfile

@@ -28,6 +28,12 @@ RUN --mount=type=cache,target=/root/.cache/uv \
     uv sync --frozen --no-dev
 
 COPY backend /app/backend
+COPY config/earth-boundary-pov-policy.china-v1.json /app/config/earth-boundary-pov-policy.china-v1.json
+COPY scripts/build_earth_boundary_china_pov_geojson.py /app/scripts/build_earth_boundary_china_pov_geojson.py
+COPY scripts/build_earth_boundary_pmtiles.py /app/scripts/build_earth_boundary_pmtiles.py
+COPY scripts/earth_boundary_policy.py /app/scripts/earth_boundary_policy.py
+COPY resources/earth/boundary-pov-geometry.china-v1.json /app/resources/earth/boundary-pov-geometry.china-v1.json
+COPY frontend/public/earth/assets/cards/manifest.json /app/frontend/public/earth/assets/cards/manifest.json
 COPY VERSION /app/VERSION
 
 EXPOSE 8000

backend/app/api/main.py

@@ -1,3 +1,4 @@
+from app.api.v1 import card_images
 from fastapi import APIRouter
 from app.api.v1 import (
     ai,
@@ -12,6 +13,7 @@ from app.api.v1 import (
     alerts,
     settings,
     collected_data,
+    compute_intelligence,
     data_products,
     layers,
     visualization,
@@ -43,6 +45,7 @@ api_router.include_router(alerts.router, prefix="/alerts", tags=["alerts"])
 api_router.include_router(settings.router, prefix="/settings", tags=["settings"])
 api_router.include_router(system_control.router, prefix="/system", tags=["system"])
 api_router.include_router(data_products.router, prefix="/data-products", tags=["data-products"])
+api_router.include_router(compute_intelligence.router, prefix="/data-products/compute", tags=["compute"])
 api_router.include_router(layers.router, prefix="/layers", tags=["layers"])
 api_router.include_router(visualization.router, prefix="/visualization", tags=["visualization"])
 api_router.include_router(
@@ -56,3 +59,5 @@ api_router.include_router(tv.router, prefix="/tv", tags=["tv"])
 api_router.include_router(news.router, prefix="/news", tags=["news"])
 api_router.include_router(interactables.router, prefix="/interactables", tags=["interactables"])
 api_router.include_router(realtime_sources.router, prefix="/realtime-sources", tags=["realtime-sources"])
+
+api_router.include_router(card_images.router, prefix="/earth/card-images", tags=["card-images"])

backend/app/api/v1/auth.py

@@ -4,6 +4,7 @@ from sqlalchemy.ext.asyncio import AsyncSession
 from sqlalchemy import text
 
 from app.core.config import settings
+from app.core.enums import OtpPurpose, UserRole
 from app.core.logging import get_logger
 from app.core.security import (
     create_access_token,
@@ -170,7 +171,7 @@ async def get_me(current_user: User = Depends(get_current_user)):
     }
 
 
-async def _send_code_or_raise(db: AsyncSession, email: str, code: str, purpose: str) -> None:
+async def _send_code_or_raise(db: AsyncSession, email: str, code: str, purpose: OtpPurpose) -> None:
     try:
         await send_verification_email(db, to=email, code=code, purpose=purpose)
     except EmailNotConfiguredError as exc:
@@ -207,7 +208,7 @@ async def register(payload: UserRegister, db: AsyncSession = Depends(get_db)):
         username=payload.username,
         email=payload.email,
         password_hash=get_password_hash(payload.password),
-        role="viewer",
+        role=UserRole.VIEWER.value,
         is_active=True,
         email_verified=False,
     )
@@ -215,13 +216,13 @@ async def register(payload: UserRegister, db: AsyncSession = Depends(get_db)):
     await db.commit()
 
     try:
-        code = otp.issue_code(payload.email, "register")
+        code = otp.issue_code(payload.email, OtpPurpose.REGISTER)
     except otp.OtpResendRateLimited as exc:
         raise HTTPException(
             status_code=status.HTTP_429_TOO_MANY_REQUESTS,
             detail={"code": exc.code, "retry_after_seconds": exc.retry_after_seconds},
         ) from exc
-    await _send_code_or_raise(db, payload.email, code, "register")
+    await _send_code_or_raise(db, payload.email, code, OtpPurpose.REGISTER)
     return {"status": "pending_verification", "email": payload.email}
 
 
@@ -266,7 +267,7 @@ async def resend_code(payload: ResendCodeRequest, db: AsyncSession = Depends(get
     if user is None:
         # Avoid email enumeration; pretend success.
         return {"status": "ok"}
-    if payload.purpose == "register" and user.email_verified:
+    if payload.purpose is OtpPurpose.REGISTER and user.email_verified:
         raise HTTPException(
             status_code=status.HTTP_409_CONFLICT,
             detail={"code": "ALREADY_VERIFIED"},
@@ -289,12 +290,17 @@ async def forgot_password(payload: ForgotPasswordRequest, db: AsyncSession = Dep
         # Don't leak whether an email is registered.
         return {"status": "ok"}
     try:
-        code = otp.issue_code(payload.email, "reset_password")
+        code = otp.issue_code(payload.email, OtpPurpose.RESET_PASSWORD)
     except otp.OtpResendRateLimited:
         # Silently accept; the user can retry after the cooldown.
         return {"status": "ok"}
     try:
-        await send_verification_email(db, to=payload.email, code=code, purpose="reset_password")
+        await send_verification_email(
+            db,
+            to=payload.email,
+            code=code,
+            purpose=OtpPurpose.RESET_PASSWORD,
+        )
     except EmailNotConfiguredError as exc:
         raise HTTPException(
             status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
@@ -304,7 +310,11 @@ async def forgot_password(payload: ForgotPasswordRequest, db: AsyncSession = Dep
         logger.warning_event(
             "SMTP send failed",
             event="auth.email.send_failed",
-            context={"email": payload.email, "purpose": "reset_password", "error": str(exc)},
+            context={
+                "email": payload.email,
+                "purpose": OtpPurpose.RESET_PASSWORD.value,
+                "error": str(exc),
+            },
         )
     return {"status": "ok"}
 
@@ -318,7 +328,7 @@ async def reset_password(payload: ResetPasswordRequest, db: AsyncSession = Depen
             detail={"code": "OTP_INVALID"},
         )
     try:
-        otp.verify_code(payload.email, "reset_password", payload.code)
+        otp.verify_code(payload.email, OtpPurpose.RESET_PASSWORD, payload.code)
     except otp.OtpExpired as exc:
         raise HTTPException(
             status_code=status.HTTP_410_GONE,

backend/app/api/v1/card_images.py

@@ -0,0 +1,108 @@
+"""Card-image configuration, upload, catalog selection and public resolution."""
+
+from fastapi import APIRouter, Depends, File, HTTPException, Query, Response, UploadFile
+from fastapi.responses import FileResponse
+from sqlalchemy.ext.asyncio import AsyncSession
+from starlette.concurrency import run_in_threadpool
+
+from app.core.enums import UserRole
+from app.core.security import get_current_user
+from app.db.session import get_db
+from app.models.user import User
+from app.schemas.card_images import (
+    CardImageConfiguration,
+    CardImagePreview,
+    CardImagePublish,
+    EntityType,
+    MatchScope,
+)
+from app.services.card_image_configuration import (
+    publish_configuration,
+    read_configuration,
+    resolve_image,
+)
+from app.services.card_image_files import MAX_UPLOAD_BYTES, store_image, uploaded_path
+from app.services.card_image_targets import get_target, search_targets
+
+router = APIRouter()
+
+
+async def image_admin(user: User = Depends(get_current_user)) -> User:
+    if user.role not in {UserRole.ADMIN.value, UserRole.SUPER_ADMIN.value}:
+        raise HTTPException(403, "Administrator access required")
+    return user
+
+
+@router.get("/configuration")
+async def get_configuration(
+    response: Response, _actor: User = Depends(image_admin), db: AsyncSession = Depends(get_db)
+):
+    response.headers["Cache-Control"] = "no-store"
+    return await read_configuration(db)
+
+
+@router.put("/configuration")
+async def save_configuration(
+    payload: CardImagePublish,
+    actor: User = Depends(image_admin),
+    db: AsyncSession = Depends(get_db),
+):
+    return await publish_configuration(db, payload.revision, payload.config, actor)
+
+
+@router.post("/upload")
+async def upload_image(file: UploadFile = File(...), _actor: User = Depends(image_admin)):
+    content = await file.read(MAX_UPLOAD_BYTES + 1)
+    await file.close()
+    return await run_in_threadpool(store_image, content, (file.filename or "Image")[:200])
+
+
+@router.get("/files/{filename}")
+async def image_file(filename: str):
+    path = uploaded_path(filename)
+    if not path.is_file():
+        raise HTTPException(404, "Image not found")
+    return FileResponse(
+        path,
+        headers={
+            "Cache-Control": "public, max-age=31536000, immutable",
+            "X-Content-Type-Options": "nosniff",
+        },
+    )
+
+
+@router.get("/targets")
+async def targets(
+    entity_type: EntityType,
+    scope: MatchScope = "object",
+    q: str = Query(default="", max_length=100),
+    limit: int = Query(default=25, ge=1, le=50),
+    _actor: User = Depends(image_admin),
+    db: AsyncSession = Depends(get_db),
+):
+    return {"items": await search_targets(db, entity_type, scope, q.strip(), limit)}
+
+
+@router.get("/resolve")
+async def resolve(
+    response: Response,
+    entity_type: EntityType,
+    entity_id: str = Query(min_length=1, max_length=300),
+    db: AsyncSession = Depends(get_db),
+):
+    response.headers["Cache-Control"] = "no-store"
+    envelope = await read_configuration(db)
+    config = CardImageConfiguration.model_validate(envelope["config"])
+    context = await get_target(db, entity_type, entity_id)
+    result = resolve_image(config, context)
+    return {**result, "revision": envelope["revision"], "entity": context}
+
+
+@router.post("/preview")
+async def preview(
+    payload: CardImagePreview,
+    _actor: User = Depends(image_admin),
+    db: AsyncSession = Depends(get_db),
+):
+    context = await get_target(db, payload.entity_type, payload.entity_id)
+    return {**resolve_image(payload.config, context), "entity": context}

backend/app/api/v1/compute_intelligence.py

@@ -0,0 +1,235 @@
+"""Compute observations, administrator configuration and independent facility samples."""
+
+from datetime import UTC, datetime
+from typing import Literal
+
+from app.core.compute_types import COMPUTE_SITE_TYPES, compute_record_filter, compute_site_type_expression
+from app.api.v1.visualization import convert_compute_centers_to_geojson
+from app.core.earth_geography_policy import display_country
+from app.core.enums import UserRole
+from app.core.security import get_current_user
+from app.db.session import get_db
+from app.models.collected_data import CollectedData
+from app.models.data_snapshot import DataSnapshot
+from app.models.user import User
+from app.schemas.compute_configuration import (
+    ComputeConfiguration,
+    ComputeConfigurationReset,
+    ComputeConfigurationWrite,
+)
+from app.services.compute_catalog import (
+    PARTICIPANTS,
+    participant_members,
+    source_country_names,
+    source_country_participants,
+)
+from app.services.compute_center_locations import refresh_compute_center_location_cache
+from app.services.compute_configuration import (
+    DEFAULT_CONFIGURATION,
+    configuration_response,
+    publish_configuration,
+    read_configuration,
+)
+from app.services.compute_history import compute_history
+from fastapi import APIRouter, Depends, HTTPException, Query, Response
+from sqlalchemy import and_, case, func, or_, select
+from sqlalchemy.ext.asyncio import AsyncSession
+
+router = APIRouter()
+
+
+def selected_countries(countries: str) -> list[str]:
+    codes = list(dict.fromkeys(code.strip().upper() for code in countries.split(",")))
+    if not codes or len(codes) > 12 or any(code not in PARTICIPANTS for code in codes):
+        raise HTTPException(422, "Select 1–12 participants from the compute catalog")
+    members = participant_members(codes)
+    flattened = [country for participant in members.values() for country in participant]
+    if len(flattened) != len(set(flattened)):
+        raise HTTPException(422, "Comparison participants cannot have overlapping country members")
+    return codes
+
+
+@router.get("/series")
+async def get_series(
+    response: Response,
+    countries: str = "CN,US",
+    mode: Literal["global_share", "bilateral_share", "absolute"] = "global_share",
+    db: AsyncSession = Depends(get_db),
+):
+    codes = selected_countries(countries)
+    envelope = await read_configuration(db)
+    response.headers["Cache-Control"] = "no-store"
+    result = compute_history(codes, mode, ComputeConfiguration.model_validate(envelope["config"]))
+    result.update(
+        version=envelope["revision"],
+        provenance="bundled_reports" if envelope["is_default"] else "admin_curated_reports",
+    )
+    return result
+
+
+async def configuration_admin(user: User = Depends(get_current_user)) -> User:
+    if user.role not in {UserRole.ADMIN.value, UserRole.SUPER_ADMIN.value}:
+        raise HTTPException(403, "Administrator access required")
+    return user
+
+
+@router.get("/configuration")
+async def get_configuration(
+    response: Response,
+    _actor: User = Depends(configuration_admin),
+    db: AsyncSession = Depends(get_db),
+):
+    response.headers["Cache-Control"] = "no-store"
+    return configuration_response(await read_configuration(db))
+
+
+@router.put("/configuration")
+async def update_configuration(
+    payload: ComputeConfigurationWrite,
+    actor: User = Depends(configuration_admin),
+    db: AsyncSession = Depends(get_db),
+):
+    envelope = await publish_configuration(
+        db, revision=payload.revision, config=payload.config, actor=actor,
+    )
+    return configuration_response(envelope)
+
+
+@router.post("/configuration/reset")
+async def reset_configuration(
+    payload: ComputeConfigurationReset,
+    actor: User = Depends(configuration_admin),
+    db: AsyncSession = Depends(get_db),
+):
+    envelope = await publish_configuration(
+        db, revision=payload.revision, config=DEFAULT_CONFIGURATION, actor=actor, reset=True,
+    )
+    return configuration_response(envelope)
+
+
+async def sample_query(db: AsyncSession, countries: list[str], year: int | None):
+    now = datetime.now(UTC)
+    cutoff = min(datetime(year + 1, 1, 1, tzinfo=UTC), now) if year else now
+    ranked = (
+        select(
+            DataSnapshot.id,
+            func.row_number()
+            .over(
+                partition_by=DataSnapshot.source,
+                order_by=(DataSnapshot.completed_at.desc(), DataSnapshot.id.desc()),
+            )
+            .label("rank"),
+        )
+        .where(
+            DataSnapshot.source.in_(select(CollectedData.source).where(compute_record_filter(CollectedData)).distinct()),
+            DataSnapshot.status == "success",
+            DataSnapshot.completed_at < cutoff,
+        )
+        .subquery()
+    )
+    snapshots = select(ranked.c.id).where(ranked.c.rank == 1)
+    infrastructure = compute_site_type_expression(CollectedData).in_(COMPUTE_SITE_TYPES[2:])
+    # Old collector rows have ambiguous units/identity and must not become
+    # national evidence merely because a new UI can display them.
+    query = select(CollectedData).where(
+        compute_record_filter(CollectedData),
+        CollectedData.deleted_at.is_(None),
+        CollectedData.snapshot_id.in_(snapshots),
+        CollectedData.is_valid == 1,
+        or_(CollectedData.extra_data["compute_contract"].as_string() == "v1", infrastructure),
+        or_(
+            infrastructure,
+            CollectedData.source == "top500",
+            and_(
+                CollectedData.extra_data["status"].as_string() == "Existing",
+                func.coalesce(CollectedData.extra_data["superseded_by"].as_string(), "") == "",
+                CollectedData.extra_data["possible_duplicate"].as_boolean().is_not(True),
+            ),
+        ),
+        CollectedData.extra_data["country"].as_string().in_(source_country_names(countries)),
+    )
+    return query
+
+
+@router.get("/facilities")
+async def get_facilities(
+    countries: str = "CN,US",
+    year: int | None = Query(None, ge=2000, le=2100),
+    limit: int = Query(1000, ge=1, le=2000),
+    db: AsyncSession = Depends(get_db),
+):
+    codes = selected_countries(countries)
+    query = await sample_query(db, codes, year)
+    country = CollectedData.extra_data["country"].as_string()
+    source_participants = source_country_participants(codes)
+    # Canonical source subdivisions retain their raw label on each record but
+    # share one owner in the comparison, using the same policy as the globe.
+    owner_cases = [
+        (country == name, display_country(name))
+        for name in source_country_names(codes)
+        if display_country(name) != name
+    ]
+    display = case(*owner_cases, else_=country) if owner_cases else country
+    participant_cases = [(country == name, code) for name, code in source_participants.items()]
+    participant = case(*participant_cases, else_=None)
+    counts = (
+        (
+            await db.execute(
+                query.with_only_columns(
+                    display.label("country"),
+                    participant.label("participant_code"),
+                    CollectedData.source,
+                    func.count().label("count"),
+                ).group_by(display, participant, CollectedData.source)
+            )
+        )
+        .mappings()
+        .all()
+    )
+    participant_counts = (
+        (
+            await db.execute(
+                query.with_only_columns(
+                    participant.label("participant_code"),
+                    CollectedData.source,
+                    func.count().label("count"),
+                ).group_by(participant, CollectedData.source)
+            )
+        )
+        .mappings()
+        .all()
+    )
+    records = (
+        (await db.execute(query.order_by(CollectedData.name, CollectedData.id).limit(limit)))
+        .scalars()
+        .all()
+    )
+    if year:
+        await refresh_compute_center_location_cache(db, as_of=datetime(year + 1, 1, 1, tzinfo=UTC))
+    else:
+        await refresh_compute_center_location_cache(db)
+    payload = convert_compute_centers_to_geojson(records)
+    for feature in payload["features"]:
+        feature["properties"]["historical_snapshot"] = year is not None
+        properties = feature["properties"]
+        properties["source_country"] = properties.get("country")
+        properties["country"] = display_country(properties.get("country"))
+    total = sum(row["count"] for row in counts)
+    payload.update(
+        {
+            "counts": [dict(row) for row in counts],
+            "participant_counts": [dict(row) for row in participant_counts],
+            "participants": [PARTICIPANTS[code] for code in codes],
+            "total_count": total,
+            "returned_count": len(records),
+            "truncated": total > limit,
+            "reference_year": year,
+            "scope": "public_source_records",
+            "history_basis": (
+                "snapshots_collected_by_year_end" if year else "latest_successful_snapshot"
+            ),
+            "coverage": "available" if total else "no_verified_snapshot",
+            "cross_source_deduplicated": False,
+        }
+    )
+    return payload

backend/app/api/v1/data_products.py

@@ -52,7 +52,7 @@ PRODUCT_DEFINITIONS: dict[str, dict] = {
         "name": "算力",
         "sources": ["top500", "epoch_ai_gpu"],
         "primary_stat_key": "compute_center_count",
-        "stat_keys": ["compute_center_count", "supercomputer_count", "gpu_cluster_count"],
+        "stat_keys": ["compute_center_count", "supercomputer_count", "gpu_cluster_count", "data_center_count", "cloud_center_count", "edge_node_count"],
     },
 }
 

backend/app/api/v1/datasource_config.py

@@ -13,6 +13,7 @@ import httpx
 
 from app.core.target_schema_registry import get_target_schema, list_target_schemas
 from app.core.datasource_defaults import DEFAULT_DATASOURCES
+from app.core.enums import AuthType, MappingValidationStatus, UserRole
 from app.db.session import get_db
 from app.models.user import User
 from app.models.datasource_config import DataSourceConfig
@@ -34,7 +35,6 @@ from app.services.datasource_mapping import (
 )
 from app.services.custom_datasource_runtime import (
     CustomDatasourceRuntimeError,
-    fetch_rest_payload,
     get_custom_stream_status,
     run_mapped_rest_config,
     run_mapped_websocket_config,
@@ -42,8 +42,6 @@ from app.services.custom_datasource_runtime import (
     stop_custom_stream,
     test_websocket_config,
 )
-
-DATASOURCE_MAPPING_PROMPT_KEY = "datasource.mapping"
 from app.services.datasource_connectivity import (
     _resolve_aisstream_api_key,
     _resolve_spacetrack_credentials_with_override,
@@ -57,7 +55,8 @@ from app.services.persistent_logs import record_audit_log
 
 router = APIRouter()
 
-SECRET_REVEAL_ROLES = {"admin", "super_admin"}
+DATASOURCE_MAPPING_PROMPT_KEY = "datasource.mapping"
+SECRET_REVEAL_ROLES = {UserRole.ADMIN.value, UserRole.SUPER_ADMIN.value}
 
 
 def _user_role_value(user: User) -> str:
@@ -124,7 +123,7 @@ class DataSourceConfigCreate(BaseModel):
     description: Optional[str] = None
     source_type: str = Field(..., description="rest, websocket, http, api, database")
     endpoint: str = Field(..., max_length=500)
-    auth_type: str = Field(default="none", description="none, bearer, api_key, basic")
+    auth_type: AuthType = Field(default=AuthType.NONE, description="none, bearer, api_key, basic")
     auth_config: dict = Field(default={})
     headers: dict = Field(default={})
     config: dict = Field(default={"timeout": 30, "retry": 3})
@@ -135,7 +134,7 @@ class DataSourceConfigUpdate(BaseModel):
     description: Optional[str] = None
     source_type: Optional[str] = None
     endpoint: Optional[str] = Field(None, max_length=500)
-    auth_type: Optional[str] = None
+    auth_type: Optional[AuthType] = None
     auth_config: Optional[dict] = None
     headers: Optional[dict] = None
     config: Optional[dict] = None
@@ -210,7 +209,7 @@ class MappingTemplateCreate(BaseModel):
     mapping_json: dict
     sample_payload: Any | None = None
     sample_payload_hash: Optional[str] = None
-    validation_status: str = Field(default="draft", pattern="^(draft|valid|invalid)$")
+    validation_status: MappingValidationStatus = MappingValidationStatus.DRAFT
     is_active: bool = False
 
 
@@ -219,7 +218,7 @@ class MappingTemplateUpdate(BaseModel):
     mapping_json: Optional[dict] = None
     sample_payload: Any | None = None
     sample_payload_hash: Optional[str] = None
-    validation_status: Optional[str] = Field(default=None, pattern="^(draft|valid|invalid)$")
+    validation_status: Optional[MappingValidationStatus] = None
     is_active: Optional[bool] = None
 
 
@@ -918,6 +917,7 @@ async def get_datasource_target_schemas(
 @router.post("/mappings/propose")
 async def propose_datasource_mapping(
     payload: MappingProposeRequest,
+    db: AsyncSession = Depends(get_db),
     current_user: User = Depends(get_current_user),
     ai_client: AIProviderClient = Depends(get_ai_provider_client),
 ):

backend/app/api/v1/datasources.py

@@ -7,6 +7,7 @@ from sqlalchemy import func, or_, select, text
 from sqlalchemy.ext.asyncio import AsyncSession
 
 from app.core.logging import get_logger
+from app.core.enums import JobStatus, SnapshotStatus
 from app.core.time import to_iso8601_utc
 from app.core.security import get_current_user
 from app.core.data_sources import get_data_sources_config
@@ -19,7 +20,6 @@ from app.models.datasource_config import DataSourceConfig
 from app.models.task import CollectionTask
 from app.models.user import User
 from app.models.vessel import AISRawObservation
-from app.services.vessel_ais_aggregation import VESSEL_AIS_SCHEMA
 from app.services.scheduler import (
     sync_datasource_job,
 )
@@ -165,6 +165,8 @@ async def _load_latest_tasks(
 async def _load_collected_record_counts(
     db: AsyncSession,
     sources: list[str],
+    *,
+    exact_vessel_counts: bool = False,
 ) -> dict[str, int]:
     if not sources:
         return {}
@@ -185,14 +187,46 @@ async def _load_collected_record_counts(
         or "ais" in source
     ]
     if vessel_sources:
-        raw_result = await db.execute(
-            select(AISRawObservation.source, func.count(AISRawObservation.id))
-            .where(AISRawObservation.target_schema == VESSEL_AIS_SCHEMA)
-            .where(AISRawObservation.source.in_(vessel_sources))
-            .group_by(AISRawObservation.source)
+        if exact_vessel_counts:
+            exact_result = await db.execute(
+                select(AISRawObservation.source, func.count(AISRawObservation.id))
+                .where(AISRawObservation.source.in_(vessel_sources))
+                .group_by(AISRawObservation.source)
+            )
+            for source, count in exact_result.all():
+                counts[source] = max(counts.get(source, 0), int(count or 0))
+            return counts
+
+        # AIS raw observations can be tens of millions of rows. Use planner
+        # statistics for the datasource list instead of blocking page load on
+        # source-level count(*) scans.
+        stats_result = await db.execute(
+            text(
+                """
+                SELECT
+                    COALESCE(pg_class.reltuples, 0)::bigint AS total_rows,
+                    pg_stats.most_common_vals::text AS source_values,
+                    pg_stats.most_common_freqs::text AS source_freqs
+                FROM pg_class
+                LEFT JOIN pg_stats
+                  ON pg_stats.schemaname = 'public'
+                 AND pg_stats.tablename = 'ais_raw_observations'
+                 AND pg_stats.attname = 'source'
+                WHERE pg_class.relname = 'ais_raw_observations'
+                LIMIT 1
+                """
+            )
         )
-        for source, count in raw_result.all():
-            counts[source] = max(counts.get(source, 0), int(count or 0))
+        stats = stats_result.mappings().first()
+        if stats:
+            total_rows = int(stats["total_rows"] or 0)
+            values = str(stats["source_values"] or "").strip("{}")
+            freqs = str(stats["source_freqs"] or "").strip("{}")
+            source_values = [value.strip('"') for value in values.split(",") if value]
+            source_freqs = [float(value) for value in freqs.split(",") if value]
+            for source, freq in zip(source_values, source_freqs):
+                if source in vessel_sources:
+                    counts[source] = max(counts.get(source, 0), int(round(total_rows * freq)))
 
     return counts
 
@@ -548,7 +582,7 @@ async def get_running_task(db: AsyncSession, datasource_id: int) -> Optional[Col
     result = await db.execute(
         select(CollectionTask)
         .where(CollectionTask.datasource_id == datasource_id)
-        .where(CollectionTask.status == "running")
+        .where(CollectionTask.status == JobStatus.RUNNING.value)
         .order_by(CollectionTask.started_at.desc())
         .limit(1)
     )
@@ -576,8 +610,8 @@ async def get_running_task(db: AsyncSession, datasource_id: int) -> Optional[Col
             f"Marked failed automatically after stale running timeout "
             f"({STALE_RUNNING_TASK_TIMEOUT_MINUTES}m)"
         )
-        task.status = "failed"
-        task.phase = "failed"
+        task.status = JobStatus.FAILED.value
+        task.phase = JobStatus.FAILED.value
         task.completed_at = now
         task.error_message = f"{existing_error}\n{stale_reason}".strip() if existing_error else stale_reason
         await db.commit()
@@ -614,7 +648,7 @@ async def rollback_orphaned_running_task(
     )
 
     if snapshot is not None:
-        snapshot.status = "cancelled"
+        snapshot.status = SnapshotStatus.CANCELLED.value
         snapshot.is_current = False
         snapshot.completed_at = datetime.now(timezone.utc)
         summary = dict(snapshot.summary or {})
@@ -637,13 +671,13 @@ async def rollback_orphaned_running_task(
                 {"snapshot_id": snapshot.parent_snapshot_id},
             )
 
-    running_task.status = "cancelled"
-    running_task.phase = "cancelled"
+    running_task.status = JobStatus.CANCELLED.value
+    running_task.phase = JobStatus.CANCELLED.value
     running_task.completed_at = datetime.now(timezone.utc)
     existing_error = (running_task.error_message or "").strip()
     cancel_reason = "Cancelled after backend restart because the running task handle was lost; incomplete writes rolled back"
     running_task.error_message = f"{existing_error}\n{cancel_reason}".strip() if existing_error else cancel_reason
-    datasource.last_status = "cancelled"
+    datasource.last_status = JobStatus.CANCELLED.value
     datasource.last_run_at = datetime.now(timezone.utc)
     await db.commit()
 
@@ -678,7 +712,7 @@ async def fail_and_rollback_stale_running_task(
     )
 
     if snapshot is not None:
-        snapshot.status = "failed"
+        snapshot.status = SnapshotStatus.FAILED.value
         snapshot.is_current = False
         snapshot.completed_at = datetime.now(timezone.utc)
         summary = dict(snapshot.summary or {})
@@ -706,11 +740,11 @@ async def fail_and_rollback_stale_running_task(
         f"Marked failed automatically after stale running timeout "
         f"({STALE_RUNNING_TASK_TIMEOUT_MINUTES}m); incomplete writes rolled back"
     )
-    running_task.status = "failed"
-    running_task.phase = "failed"
+    running_task.status = JobStatus.FAILED.value
+    running_task.phase = JobStatus.FAILED.value
     running_task.completed_at = datetime.now(timezone.utc)
     running_task.error_message = f"{existing_error}\n{stale_reason}".strip() if existing_error else stale_reason
-    datasource.last_status = "failed"
+    datasource.last_status = JobStatus.FAILED.value
     datasource.last_run_at = datetime.now(timezone.utc)
     await db.commit()
 
@@ -929,7 +963,7 @@ async def get_datasource_row(
         [datasource],
         include_endpoint=include_endpoint,
     )
-    record_counts = await _load_collected_record_counts(db, [datasource.source])
+    record_counts = await _load_collected_record_counts(db, [datasource.source], exact_vessel_counts=True)
     return {
         "data": serialize_datasource_row(
             datasource,

backend/app/api/v1/earth.py

@@ -6,13 +6,14 @@ from pathlib import Path
 from typing import Any
 from uuid import uuid4
 
-from fastapi import APIRouter, Depends, File, HTTPException, Request, UploadFile, status
+from fastapi import APIRouter, Depends, File, Form, HTTPException, Query, Request, UploadFile, status
 from fastapi.security import HTTPAuthorizationCredentials, HTTPBearer
 from pydantic import BaseModel, Field
 from sqlalchemy import delete, func, select, text
 from sqlalchemy.ext.asyncio import AsyncSession
 
 from app.core.config import settings as app_settings
+from app.core.logging import get_logger
 from app.core.security import decode_token, get_current_user, redis_client
 from app.db.session import get_db
 from app.models.collected_data import CollectedData
@@ -21,6 +22,26 @@ from app.models.datasource_config import DataSourceConfig
 from app.models.system_setting import SystemSetting
 from app.models.user import User
 from app.services.tv_streams import get_tv_settings_payload
+from app.services.earth_news import (
+    get_earth_news_sources_payload,
+    reset_earth_news_sources_payload,
+    save_earth_news_sources_payload,
+    test_news_source_config,
+)
+from app.services.earth_news_manual import (
+    broadcast_manual_news_changed,
+    create_manual_news_group,
+    delete_manual_news_item,
+    get_news_record_or_404,
+    import_manual_news_items,
+    list_news_groups,
+    list_news_records,
+    parse_manual_news_import_upload,
+    rename_manual_news_group,
+    reprocess_manual_news_item,
+    serialize_news_record,
+    upsert_manual_news_item,
+)
 from app.services.earth_boundaries import (
     EarthBoundaryBuildError,
     get_boundary_build_status,
@@ -31,6 +52,7 @@ from app.services.earth_boundaries import (
 
 
 router = APIRouter()
+logger = get_logger(__name__)
 optional_bearer = HTTPBearer(auto_error=False)
 REPO_ROOT = Path(__file__).resolve().parents[4]
 EARTH_BRAND_ASSET_DIR = REPO_ROOT / "data" / "earth-brand"
@@ -100,6 +122,39 @@ class EarthAboutPayload(BaseModel):
     meta: list[EarthAboutMetaItem] = Field(default_factory=list)
 
 
+class EarthNewsSourcesPayload(BaseModel):
+    cache_version: int | None = None
+    source_tags: list[dict[str, Any]] = Field(default_factory=list)
+    categories: list[dict[str, Any]] = Field(default_factory=list)
+    item_tag_rules: list[dict[str, Any]] = Field(default_factory=list)
+    sources: list[dict[str, Any]] = Field(default_factory=list)
+    health: dict[str, Any] = Field(default_factory=dict)
+
+
+class EarthNewsSourceTestPayload(BaseModel):
+    source: dict[str, Any] = Field(default_factory=dict)
+
+
+class EarthNewsManualItemPayload(BaseModel):
+    title: str = Field(default="", max_length=500)
+    summary: str = Field(default="", max_length=1200)
+    content: str = Field(default="", max_length=12000)
+    url: str = Field(default="", max_length=2000)
+    source: str = Field(default="", max_length=255)
+    region: str = Field(default="global", max_length=80)
+    published_at: str | None = None
+    category: str = Field(default="other", max_length=80)
+    tags: list[str] = Field(default_factory=list)
+    location: dict[str, Any] | None = None
+    homepage_url: str = Field(default="", max_length=2000)
+    content_language: str = Field(default="", max_length=32)
+    group_id: str | None = Field(default=None, max_length=120)
+
+
+class EarthNewsManualGroupPayload(BaseModel):
+    name: str = Field(default="", max_length=120)
+
+
 def _normalize_earth_brand_payload(payload: dict[str, Any] | None) -> dict[str, str]:
     merged = DEFAULT_EARTH_BRAND.copy()
     if payload:
@@ -324,6 +379,211 @@ async def reset_earth_about(
     return {"status": "reset", "about": _normalize_earth_about_payload(None), "is_default": True}
 
 
+@router.get("/news-sources")
+async def get_earth_news_sources(db: AsyncSession = Depends(get_db)):
+    return await get_earth_news_sources_payload(db)
+
+
+def _schedule_earth_news_source_refresh() -> None:
+    try:
+        from app.services.scheduler import schedule_earth_news_refresh_for_all_regions
+
+        schedule_earth_news_refresh_for_all_regions(force=True)
+    except Exception as exc:
+        logger.warning_event(
+            "Failed to schedule Earth news refresh after source configuration change",
+            event="earth_news.rss.config_refresh_schedule_failed",
+            context={"error": str(exc)},
+        )
+
+
+@router.put("/news-sources")
+async def update_earth_news_sources(
+    payload: EarthNewsSourcesPayload,
+    _current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    result = await save_earth_news_sources_payload(db, payload.model_dump())
+    _schedule_earth_news_source_refresh()
+    return result
+
+
+@router.delete("/news-sources")
+@router.post("/news-sources/reset")
+async def reset_earth_news_sources(
+    _current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    result = await reset_earth_news_sources_payload(db)
+    _schedule_earth_news_source_refresh()
+    return result
+
+
+@router.post("/news-sources/test")
+async def test_earth_news_source(
+    payload: EarthNewsSourceTestPayload,
+    _current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    return await test_news_source_config(payload.source, db=db)
+
+
+@router.get("/news-groups")
+async def list_earth_news_groups_admin(
+    _current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    return await list_news_groups(db)
+
+
+@router.post("/news-groups")
+async def create_earth_news_group_admin(
+    payload: EarthNewsManualGroupPayload,
+    _current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    try:
+        group = await create_manual_news_group(db, payload.name)
+    except ValueError as exc:
+        raise HTTPException(status_code=422, detail=str(exc)) from exc
+    await db.commit()
+    return {"status": "ok", "group": group}
+
+
+@router.put("/news-groups/{group_id:path}")
+async def rename_earth_news_group_admin(
+    group_id: str,
+    payload: EarthNewsManualGroupPayload,
+    _current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    try:
+        group = await rename_manual_news_group(db, group_id, payload.name)
+    except ValueError as exc:
+        raise HTTPException(status_code=422, detail=str(exc)) from exc
+    await db.commit()
+    await broadcast_manual_news_changed()
+    return {"status": "ok", "group": group}
+
+
+@router.get("/news-items")
+async def list_earth_news_items_admin(
+    page: int = Query(1, ge=1),
+    page_size: int = Query(50, ge=1, le=100),
+    source_type: str | None = Query(None),
+    region: str | None = Query(None),
+    category: str | None = Query(None),
+    status_filter: str | None = Query(None, alias="status"),
+    group_id: str | None = Query(None),
+    _current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    return await list_news_records(
+        db,
+        page=page,
+        page_size=page_size,
+        source_type=source_type,
+        region=region,
+        category=category,
+        status_filter=status_filter,
+        group_id=group_id,
+    )
+
+
+@router.post("/news-items")
+async def create_earth_news_item_admin(
+    payload: EarthNewsManualItemPayload,
+    _current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    try:
+        result = await upsert_manual_news_item(db, payload.model_dump())
+    except ValueError as exc:
+        raise HTTPException(status_code=422, detail=str(exc)) from exc
+    except PermissionError as exc:
+        raise HTTPException(status_code=403, detail=str(exc)) from exc
+    await db.commit()
+    await broadcast_manual_news_changed()
+    return {"status": "ok", "created": result.created, "queued": result.queued, "item": serialize_news_record(result.item)}
+
+
+@router.post("/news-items/import")
+async def import_earth_news_items_admin(
+    file: UploadFile = File(...),
+    group_id: str | None = Form(default=None),
+    _current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    try:
+        payload = await parse_manual_news_import_upload(await file.read())
+        result = await import_manual_news_items(db, payload, group_id=group_id)
+    except ValueError as exc:
+        raise HTTPException(status_code=422, detail=str(exc)) from exc
+    await db.commit()
+    await broadcast_manual_news_changed()
+    return {"status": "ok", **result}
+
+
+@router.put("/news-items/{item_id:path}")
+async def update_earth_news_item_admin(
+    item_id: str,
+    payload: EarthNewsManualItemPayload,
+    _current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    existing = await get_news_record_or_404(db, item_id)
+    if existing is None:
+        raise HTTPException(status_code=404, detail="News item not found.")
+    try:
+        result = await upsert_manual_news_item(
+            db,
+            payload.model_dump(),
+            item_id_override=item_id,
+        )
+    except ValueError as exc:
+        raise HTTPException(status_code=422, detail=str(exc)) from exc
+    except PermissionError as exc:
+        raise HTTPException(status_code=403, detail=str(exc)) from exc
+    await db.commit()
+    await broadcast_manual_news_changed()
+    return {"status": "ok", "created": result.created, "queued": result.queued, "item": serialize_news_record(result.item)}
+
+
+@router.delete("/news-items/{item_id:path}")
+async def delete_earth_news_item_admin(
+    item_id: str,
+    _current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    try:
+        deleted = await delete_manual_news_item(db, item_id)
+    except PermissionError as exc:
+        raise HTTPException(status_code=403, detail=str(exc)) from exc
+    if not deleted:
+        raise HTTPException(status_code=404, detail="News item not found.")
+    await db.commit()
+    await broadcast_manual_news_changed()
+    return {"status": "deleted", "id": item_id}
+
+
+@router.post("/news-items/{item_id:path}/reprocess")
+async def reprocess_earth_news_item_admin(
+    item_id: str,
+    _current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    existing = await get_news_record_or_404(db, item_id)
+    if existing is None:
+        raise HTTPException(status_code=404, detail="News item not found.")
+    try:
+        queued = await reprocess_manual_news_item(db, item_id)
+    except PermissionError as exc:
+        raise HTTPException(status_code=403, detail=str(exc)) from exc
+    await db.commit()
+    await broadcast_manual_news_changed()
+    return {"status": "queued" if queued else "not_queued", "queued": queued, "id": item_id}
+
+
 @router.get("/oobe-status")
 async def get_earth_oobe_status(
     current_user: User | None = Depends(_get_optional_current_user),

backend/app/api/v1/interactables.py

@@ -90,7 +90,7 @@ async def get_interactables_geojson(
         return interactables_to_geojson(items)
 
     payload = await get_or_build_layer_payload(
-        key=earth_layer_cache.key("interactables", layer=layer or "all"),
+        key=earth_layer_cache.key("interactables", interactable_layer=layer or "all"),
         policy=INTERACTABLE_CACHE_POLICY,
         builder=build_payload,
         response=response,

backend/app/api/v1/layers.py

@@ -105,7 +105,7 @@ def _parse_layer_bbox(bbox: str) -> tuple[float, float, float, float]:
 @router.get("/vessels/snapshot")
 async def get_vessel_layer_snapshot(
     bbox: str = Query(..., description="lon_min,lat_min,lon_max,lat_max"),
-    zoom: int = Query(..., ge=1, le=20),
+    zoom: float = Query(..., ge=1, le=20),
     limit: int = Query(DEFAULT_LAYER_LIMIT, ge=1),
     vessel_type: Optional[str] = Query(None, alias="type"),
     since_minutes: int = Query(60, ge=1, le=1440),

backend/app/api/v1/news.py

@@ -1,16 +1,161 @@
-from fastapi import APIRouter, Depends, Query
+from fastapi import APIRouter, Depends, HTTPException, Query, status
+from pydantic import BaseModel, Field
 from sqlalchemy.ext.asyncio import AsyncSession
+from uuid import UUID
 
 from app.db.session import get_db
-from app.services.earth_news import get_earth_news_payload
+from app.services.earth_news import (
+    ALLOWED_NEWS_CATEGORY_KEYS,
+    SUPPORTED_NEWS_LOCALES,
+    REGION_ANCHORS,
+    determine_focus_region,
+    get_earth_news_payload,
+)
 
 router = APIRouter()
 
 
+class EarthNewsRefreshRequest(BaseModel):
+    lat: float | None = Field(default=None, ge=-90, le=90)
+    lon: float | None = Field(default=None, ge=-180, le=180)
+    region: str | None = Field(default=None, max_length=64)
+
+
+def _parse_categories(raw: str | None) -> set[str] | None:
+    if raw is None or not raw.strip():
+        return None
+    requested = {item.strip().lower() for item in raw.split(",") if item.strip()}
+    invalid = sorted(requested - set(ALLOWED_NEWS_CATEGORY_KEYS))
+    if invalid:
+        raise HTTPException(
+            status_code=422,
+            detail={
+                "message": "Unsupported news categories.",
+                "invalid_categories": invalid,
+                "allowed_categories": list(ALLOWED_NEWS_CATEGORY_KEYS),
+            },
+        )
+    return requested or None
+
+
+def _parse_source_ids(raw: str | None) -> set[str] | None:
+    if raw is None or not raw.strip():
+        return None
+    return {item.strip() for item in raw.split(",") if item.strip()} or None
+
+
+def _parse_limit(raw: int | None) -> int:
+    if raw is None:
+        return 12
+    if raw < 1:
+        raise HTTPException(
+            status_code=422, detail={"message": "News limit must be greater than 0."}
+        )
+    return min(raw, 100)
+
+
+def _parse_locale(raw: str | None) -> str:
+    if raw is None or not raw.strip():
+        return "zh-CN"
+    requested = raw.strip()
+    if requested not in SUPPORTED_NEWS_LOCALES:
+        raise HTTPException(
+            status_code=422,
+            detail={
+                "message": "Unsupported news locale.",
+                "invalid_locale": requested,
+                "allowed_locales": sorted(SUPPORTED_NEWS_LOCALES),
+            },
+        )
+    return requested
+
+
+@router.post("/refresh", status_code=status.HTTP_202_ACCEPTED)
+async def request_earth_news_refresh(payload: EarthNewsRefreshRequest):
+    requested_region = payload.region.strip().lower() if payload.region else None
+    if requested_region and requested_region not in REGION_ANCHORS:
+        raise HTTPException(
+            status_code=422,
+            detail={
+                "message": "Unsupported news region.",
+                "invalid_region": requested_region,
+                "allowed_regions": list(REGION_ANCHORS.keys()),
+            },
+        )
+    region = requested_region or determine_focus_region(payload.lat, payload.lon)
+    from app.services.scheduler import request_earth_news_region_refresh
+
+    try:
+        return request_earth_news_region_refresh(region)
+    except Exception as exc:
+        raise HTTPException(
+            status_code=503,
+            detail={
+                "code": "refresh_failed",
+                "message": "Could not start a news refresh. Please try again.",
+            },
+        ) from exc
+
+
+@router.get("/refresh/{refresh_id}")
+async def get_earth_news_refresh(refresh_id: UUID):
+    from app.services.earth_news_refresh import get_earth_news_refresh_status
+
+    try:
+        result = get_earth_news_refresh_status(refresh_id.hex)
+    except Exception as exc:
+        raise HTTPException(
+            status_code=503,
+            detail={
+                "code": "refresh_failed",
+                "message": "Refresh status is temporarily unavailable. Please retry.",
+            },
+        ) from exc
+    if result is not None:
+        return result
+    return {
+        "refresh_id": refresh_id.hex,
+        "region": None,
+        "status": "failed",
+        "error": {
+            "code": "expired",
+            "message": "Refresh status expired. Submit a new refresh.",
+        },
+    }
+
+
 @router.get("/earth-feed")
 async def get_earth_feed(
     lat: float | None = Query(None, description="Current Earth view center latitude"),
     lon: float | None = Query(None, description="Current Earth view center longitude"),
+    region: str | None = Query(
+        None, description="Explicit Earth news region for UE/client integrations"
+    ),
+    categories: str | None = Query(None, description="Comma-separated news category keys"),
+    sources: str | None = Query(None, description="Comma-separated news source ids"),
+    limit: int | None = Query(None, description="Maximum news items to return, capped at 100"),
+    locale: str | None = Query(None, description="Display locale, zh-CN or en-US"),
     db: AsyncSession = Depends(get_db),
 ):
-    return await get_earth_news_payload(lat=lat, lon=lon, db=db)
+    normalized_region = (
+        region.strip().lower() if isinstance(region, str) and region.strip() else None
+    )
+    if normalized_region is not None and normalized_region not in REGION_ANCHORS:
+        raise HTTPException(
+            status_code=422,
+            detail={
+                "message": "Unsupported news region.",
+                "invalid_region": normalized_region,
+                "allowed_regions": list(REGION_ANCHORS.keys()),
+            },
+        )
+    return await get_earth_news_payload(
+        lat=lat,
+        lon=lon,
+        region=normalized_region,
+        categories=_parse_categories(categories),
+        source_ids=_parse_source_ids(sources),
+        limit=_parse_limit(limit),
+        locale=_parse_locale(locale),
+        db=db,
+    )

backend/app/api/v1/settings.py

@@ -3,15 +3,16 @@ from datetime import UTC, datetime
 import os
 from pathlib import Path
 from typing import Optional
+from urllib.parse import urlsplit
 
 from fastapi import APIRouter, Depends, HTTPException, Query, Request, status
-import httpx
 from pydantic import BaseModel, EmailStr, Field
 from dotenv import dotenv_values
 from sqlalchemy import select
 from sqlalchemy.ext.asyncio import AsyncSession
 
 from app.core.logging import get_logger
+from app.core.enums import ProviderApi, TVSourceType, UserRole
 from app.core.security import get_current_user
 from app.core.time import to_iso8601_utc
 from app.core.config import settings as app_settings
@@ -64,6 +65,7 @@ from app.services.llm_provider_catalog import (
     list_fallback_llm_provider_presets,
     refresh_llm_provider_preset,
 )
+from app.services.llm_model_catalog import catalog_error_message, fetch_model_catalog
 from app.services.scheduler import sync_datasource_job
 from app.services.tv_streams import DEFAULT_TV_SETTINGS, get_tv_settings_payload, normalize_tv_settings
 from app.services.persistent_logs import record_audit_log
@@ -73,7 +75,8 @@ router = APIRouter()
 logger = get_logger(__name__, service="api")
 AI_PROVIDER_QUICK_CONNECT_TIMEOUT_SECONDS = 5
 AI_CONNECTION_TEST_PROMPT_KEY = "ai.connection_test"
-SECRET_REVEAL_ROLES = {"admin", "super_admin"}
+SECRET_REVEAL_ROLES = {UserRole.ADMIN.value, UserRole.SUPER_ADMIN.value}
+LLM_PROVIDER_PRESET_CATEGORY_PREFIX = "llm_provider_preset:"
 
 DEFAULT_SETTINGS = {
     "system": {
@@ -236,7 +239,7 @@ class TVStreamSourceUpdate(BaseModel):
     provider: str = Field(default="Unknown", max_length=100)
     region: str = Field(default="Global", max_length=100)
     language: str = Field(default="und", max_length=32)
-    source_type: str = Field(default="iframe", pattern="^(iframe|hls|video|external|youtube)$")
+    source_type: TVSourceType = TVSourceType.IFRAME
     embed_url: str = ""
     stream_url: str = ""
     homepage_url: str = ""
@@ -279,7 +282,7 @@ class AIProviderIntegrationUpdate(BaseModel):
     service_token: Optional[str] = None
     default_provider: Optional[str] = None
     provider: str = Field(default="minimax", max_length=80)
-    provider_api: str = Field(default="anthropic-messages", max_length=80)
+    provider_api: ProviderApi = ProviderApi.ANTHROPIC_MESSAGES
     base_url: str = Field(default="", max_length=500)
     model: str = Field(default="", max_length=200)
     api_key: Optional[str] = None
@@ -344,7 +347,13 @@ class ExternalIntegrationsUpdate(BaseModel):
 
 
 def merge_with_defaults(category: str, payload: Optional[dict]) -> dict:
-    merged = deepcopy(DEFAULT_SETTINGS[category])
+    if category.startswith(LLM_PROVIDER_PRESET_CATEGORY_PREFIX):
+        defaults = get_fallback_llm_provider_preset(
+            category.removeprefix(LLM_PROVIDER_PRESET_CATEGORY_PREFIX)
+        )
+    else:
+        defaults = DEFAULT_SETTINGS[category]
+    merged = deepcopy(defaults)
     if payload:
         merged.update(payload)
     return merged
@@ -381,6 +390,7 @@ async def get_setting_payload(db: AsyncSession, category: str) -> dict:
 
 
 async def save_setting_payload(db: AsyncSession, category: str, payload: dict) -> dict:
+    merged = merge_with_defaults(category, payload)
     record = await get_setting_record(db, category)
     if record is None:
         record = SystemSetting(category=category, payload=payload)
@@ -390,7 +400,7 @@ async def save_setting_payload(db: AsyncSession, category: str, payload: dict) -
 
     await db.commit()
     await db.refresh(record)
-    return merge_with_defaults(category, record.payload)
+    return merged
 
 
 AI_PROVIDER_ENV_FILE = Path(__file__).resolve().parents[4] / "aiprovider" / ".env"
@@ -423,7 +433,7 @@ def _get_provider_preset(provider: str) -> dict:
     except ValueError:
         return {
             "provider": provider,
-            "provider_api": "openai-completions",
+            "provider_api": ProviderApi.OPENAI_COMPLETIONS.value,
             "base_url": "",
         "model": "",
         "models": [],
@@ -487,12 +497,12 @@ def _provider_defaults(provider: str) -> dict:
     preset = _get_provider_preset(provider)
     return {
         "provider": provider,
-        "provider_api": preset.get("provider_api") or "openai-completions",
+        "provider_api": preset.get("provider_api") or ProviderApi.OPENAI_COMPLETIONS.value,
         "base_url": preset.get("base_url") or "",
         "model": preset.get("model") or "",
         "api_key": "",
         "max_tokens": (
-            1200 if preset.get("provider_api") == "anthropic-messages" else 4096
+            1200 if preset.get("provider_api") == ProviderApi.ANTHROPIC_MESSAGES.value else 4096
         ),
         "anthropic_version": "2023-06-01",
         "model_provider_apis": preset.get("model_provider_apis") or {},
@@ -654,6 +664,12 @@ def _runtime_config_from_ai_payload(ai_payload: dict) -> dict:
         normalized_ai["providers"].get(default_provider) or _provider_defaults(default_provider)
     )
     api_key, _api_key_source = _resolve_provider_api_key(default_provider, provider_config)
+    model_provider_apis = {
+        **(provider_config.get("model_provider_apis") or {}),
+        **(_get_provider_preset(default_provider).get("model_provider_apis") or {}),
+    }
+    if default_provider == "openai" and urlsplit(provider_config.get("base_url") or "").hostname != "api.openai.com":
+        model_provider_apis = {}
     return {
         "service_url": normalized_ai.get("service_url") or app_settings.AI_PROVIDER_SERVICE_URL,
         "service_token": _resolve_service_token(normalized_ai)[0],
@@ -665,14 +681,13 @@ def _runtime_config_from_ai_payload(ai_payload: dict) -> dict:
         ),
         "llm_config": {
             "provider": default_provider,
-            "provider_api": provider_config.get("provider_api") or "anthropic-messages",
+            "provider_api": provider_config.get("provider_api") or ProviderApi.ANTHROPIC_MESSAGES.value,
             "base_url": provider_config.get("base_url") or "",
             "model": provider_config.get("model") or "",
             "api_key": api_key,
             "max_tokens": int(provider_config.get("max_tokens") or 1200),
             "anthropic_version": provider_config.get("anthropic_version") or "2023-06-01",
-            "model_provider_apis": provider_config.get("model_provider_apis") or {},
-            "preset_models": _get_provider_preset(default_provider).get("models") or [],
+            "model_provider_apis": model_provider_apis,
         },
     }
 
@@ -752,156 +767,35 @@ async def _validate_ai_provider_full_connection(ai_payload: dict) -> dict:
     }
 
 
-def _join_provider_url(base_url: str, path: str) -> str:
-    return f"{base_url.rstrip('/')}/{path.lstrip('/')}"
-
-
-def _extract_model_ids(payload: dict) -> list[str]:
-    data = payload.get("data") if isinstance(payload, dict) else None
-    if isinstance(data, list):
-        return [
-            str(item.get("id"))
-            for item in data
-            if isinstance(item, dict) and item.get("id")
-        ]
-    models = payload.get("models") if isinstance(payload, dict) else None
-    if isinstance(models, list):
-        return [
-            str(item.get("name") or item.get("model") or item.get("id") or item)
-            for item in models
-            if item
-        ]
-    return []
-
-
-def _contains_model(model_ids: list[str], model: str) -> bool:
-    normalized_model = model.strip().lower()
-    return any(str(item).strip().lower() == normalized_model for item in model_ids)
-
-
 async def _check_ai_provider_lightweight(llm_config: dict, timeout_seconds: int) -> dict:
     provider = _normalize_provider_id(llm_config.get("provider") or "")
-    configured_api = str(llm_config.get("provider_api") or "").strip() or "openai-completions"
+    provider_api = str(llm_config.get("provider_api") or ProviderApi.OPENAI_COMPLETIONS.value)
     model = str(llm_config.get("model") or "").strip()
-    base_url = str(llm_config.get("base_url") or "").strip().rstrip("/")
+    base_url = str(llm_config.get("base_url") or "").strip()
     api_key = str(llm_config.get("api_key") or "").strip()
-    provider_api = configured_api
-    model_provider_apis = llm_config.get("model_provider_apis")
-    if isinstance(model_provider_apis, dict):
-        provider_api = str(model_provider_apis.get(model) or provider_api)
-    preset_models = [
-        str(item)
-        for item in (llm_config.get("preset_models") or [])
-        if str(item).strip()
-    ]
-
-    if not provider or not base_url or not model:
-        return {
-            "success": False,
-            "connected": False,
-            "message": "当前 provider/base_url/model 未完整配置。",
-            "mode": "lightweight_config",
-        }
-    if provider_api != "ollama-generate" and not api_key:
-        return {
-            "success": False,
-            "connected": False,
-            "message": "当前 provider 未配置 API Key。",
-            "mode": "lightweight_config",
-        }
-
-    if provider == "opencode-go":
-        url = _join_provider_url(base_url, "/models")
-        headers = {"Authorization": f"Bearer {api_key}"}
-    elif provider_api == "ollama-generate":
-        url = _join_provider_url(base_url, "/api/tags")
-        headers: dict[str, str] = {}
-    elif provider_api == "openai-completions":
-        url = _join_provider_url(base_url, "/models")
-        headers = {"Authorization": f"Bearer {api_key}"}
-    elif provider_api == "anthropic-messages":
-        url = _join_provider_url(base_url, "/models")
-        headers = {
-            "x-api-key": api_key,
-            "anthropic-version": str(llm_config.get("anthropic_version") or "2023-06-01"),
-        }
-    else:
-        return {
-            "success": False,
-            "connected": False,
-            "message": f"当前 provider_api 不支持轻量连通性测试: {provider_api}",
-            "mode": "lightweight_unsupported",
-        }
-
+    result = {
+        "success": False, "connected": False, "mode": "lightweight_models",
+        "provider": provider, "model": model,
+    }
+    if not base_url or not model:
+        return {**result, "message": "当前 provider/base_url/model 未完整配置。"}
+    if provider_api != ProviderApi.OLLAMA_GENERATE.value and not api_key:
+        return {**result, "message": "当前 provider 未配置 API Key。"}
     try:
-        async with httpx.AsyncClient(timeout=min(timeout_seconds, AI_PROVIDER_QUICK_CONNECT_TIMEOUT_SECONDS)) as client:
-            response = await client.get(url, headers=headers)
-            response.raise_for_status()
-            payload = response.json()
-    except httpx.HTTPStatusError as exc:
-        detail = exc.response.text or exc.response.reason_phrase
-        if exc.response.status_code == 404 and _contains_model(preset_models, model):
-            return {
-                "success": True,
-                "connected": True,
-                "message": "轻量连通性测试通过；当前 provider 不提供可用的模型目录，已按内置模型预设确认。",
-                "mode": "lightweight_preset",
-                "provider": provider,
-                "provider_api": provider_api,
-                "model": model,
-                "url": url,
-            }
-        return {
-            "success": False,
-            "connected": False,
-            "message": f"轻量连通性测试失败: HTTP {exc.response.status_code} {detail}",
-            "mode": "lightweight_models",
-            "url": url,
-        }
+        catalog = await fetch_model_catalog(
+            provider, base_url, provider_api, api_key,
+            str(llm_config.get("anthropic_version") or "2023-06-01"), timeout_seconds,
+        )
     except Exception as exc:
-        return {
-            "success": False,
-            "connected": False,
-            "message": f"轻量连通性测试失败: {exc}",
-            "mode": "lightweight_models",
-            "url": url,
-        }
-
-    model_ids = _extract_model_ids(payload)
-    if model_ids and not _contains_model(model_ids, model):
-        if _contains_model(preset_models, model):
-            return {
-                "success": True,
-                "connected": True,
-                "message": "轻量连通性测试通过；provider 模型目录未返回当前别名，已按内置模型预设确认。",
-                "mode": "lightweight_models_with_preset_alias",
-                "provider": provider,
-                "provider_api": provider_api,
-                "model": model,
-                "models_count": len(model_ids),
-                "url": url,
-            }
-        return {
-            "success": False,
-            "connected": False,
-            "message": f"连接可用，但模型目录中没有当前模型: {model}",
-            "mode": "lightweight_models",
-            "provider": provider,
-            "model": model,
-            "models_count": len(model_ids),
-            "url": url,
-        }
-
+        return {**result, "message": catalog_error_message(exc)}
+    result.update({"url": catalog.url, "models_count": len(catalog.models)})
+    if model.casefold() not in {item.casefold() for item in catalog.models}:
+        return {**result, "message": f"模型目录查询成功，但当前模型不在目录中：{model}"}
+    model_apis = llm_config.get("model_provider_apis") or {}
     return {
-        "success": True,
-        "connected": True,
-        "message": "轻量连通性测试通过",
-        "mode": "lightweight_models",
-        "provider": provider,
-        "provider_api": provider_api,
-        "model": model,
-        "models_count": len(model_ids),
-        "url": url,
+        **result, "success": True, "connected": True,
+        "provider_api": model_apis.get(model) or provider_api,
+        "message": "模型目录查询成功，当前模型已找到；尚未执行生成调用。",
     }
 
 
@@ -1165,7 +1059,7 @@ async def serialize_external_integrations(db: AsyncSession) -> dict:
         api_key, api_key_source = _resolve_provider_api_key(provider_id, provider_config)
         providers_payload[provider_id] = {
             "provider": provider_id,
-            "provider_api": provider_config.get("provider_api") or "openai-completions",
+            "provider_api": provider_config.get("provider_api") or ProviderApi.OPENAI_COMPLETIONS.value,
             "base_url": provider_config.get("base_url") or "",
             "model": provider_config.get("model") or "",
             "api_key": _mask_secret(api_key, api_key_source),
@@ -1215,7 +1109,7 @@ async def serialize_external_integrations(db: AsyncSession) -> dict:
             "service_token": _mask_secret(*_resolve_service_token(normalized_ai)),
             "default_provider": default_provider,
             "provider": default_provider,
-            "provider_api": display_llm_config.get("provider_api") or "anthropic-messages",
+            "provider_api": display_llm_config.get("provider_api") or ProviderApi.ANTHROPIC_MESSAGES.value,
             "base_url": display_llm_config.get("base_url") or "https://api.minimaxi.com/anthropic",
             "model": display_llm_config.get("model") or "MiniMax-M2.7",
             "api_key": display_llm_config.get("api_key") or _mask_secret(None),
@@ -1458,7 +1352,7 @@ async def update_smtp_settings(
     current_user: User = Depends(get_current_user),
     db: AsyncSession = Depends(get_db),
 ):
-    if current_user.role not in ("admin", "super_admin"):
+    if current_user.role not in (UserRole.ADMIN.value, UserRole.SUPER_ADMIN.value):
         raise HTTPException(status_code=403, detail="Only administrators can change SMTP settings")
     current = await get_setting_payload(db, "smtp")
     merged = _build_smtp_payload(current, payload)
@@ -1472,7 +1366,7 @@ async def test_smtp_settings(
     current_user: User = Depends(get_current_user),
     db: AsyncSession = Depends(get_db),
 ):
-    if current_user.role not in ("admin", "super_admin"):
+    if current_user.role not in (UserRole.ADMIN.value, UserRole.SUPER_ADMIN.value):
         raise HTTPException(status_code=403, detail="Only administrators can test SMTP settings")
     from app.services.email import EmailError, send_email
 
@@ -1685,7 +1579,8 @@ async def connect_ai_provider_integration(
         )
         await emit_business_log(
             logger,
-            event="settings.ai_provider.connect.success",
+            event=("settings.ai_provider.connect.success" if lightweight_result["success"]
+                   else "settings.ai_provider.connect.failed"),
             message="AI provider connection test completed",
             category="ai",
             service="api",
@@ -1695,7 +1590,7 @@ async def connect_ai_provider_integration(
                 "provider": payload.provider,
                 "model": payload.model,
                 "configured": True,
-                "lightweight_status": lightweight_result.get("status"),
+                "connected": lightweight_result["connected"],
             },
         )
         return {
@@ -2040,8 +1935,18 @@ async def reset_provider_credential_guide(
 @router.get("/integrations/ai-provider/presets")
 async def get_ai_provider_presets(
     current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
 ):
-    return {"data": list_fallback_llm_provider_presets()}
+    presets = list_fallback_llm_provider_presets()
+    saved = await get_setting_payloads(
+        db, [f"{LLM_PROVIDER_PRESET_CATEGORY_PREFIX}{preset['provider']}" for preset in presets]
+    )
+    return {
+        "data": [
+            {**preset, **saved[f"{LLM_PROVIDER_PRESET_CATEGORY_PREFIX}{preset['provider']}"]}
+            for preset in presets
+        ]
+    }
 
 
 @router.post("/integrations/ai-provider/presets/{provider}/refresh")
@@ -2049,22 +1954,52 @@ async def refresh_ai_provider_preset(
     provider: str,
     current_user: User = Depends(get_current_user),
     db: AsyncSession = Depends(get_db),
+    payload: AIProviderIntegrationUpdate | None = None,
 ):
     try:
         provider_id = _normalize_provider_id(provider)
-        api_key = None
-        if provider_id == "opencode-go":
-            current_payload = await get_setting_payload(db, "external_integrations")
-            ai_payload = _normalize_ai_provider_payload(current_payload.get("ai_provider") or {})
-            provider_config = ai_payload["providers"].get(provider_id) or _provider_defaults(provider_id)
-            api_key, _api_key_source = _resolve_provider_api_key(provider_id, provider_config)
-        return {"data": await refresh_llm_provider_preset(provider_id, api_key=api_key)}
+        get_fallback_llm_provider_preset(provider_id)
     except ValueError as exc:
         raise HTTPException(status_code=404, detail=str(exc)) from exc
+
+    current_payload = await get_setting_payload(db, "external_integrations")
+    if payload is not None:
+        if _normalize_provider_id(payload.provider) != provider_id:
+            raise HTTPException(status_code=400, detail="刷新供应商与表单供应商不一致。")
+        ai_payload = _build_ai_provider_payload(current_payload, payload)
+    else:
+        ai_payload = _normalize_ai_provider_payload(current_payload.get("ai_provider") or {})
+    provider_config = ai_payload["providers"].get(provider_id) or _provider_defaults(provider_id)
+    api_key, _api_key_source = _resolve_provider_api_key(provider_id, provider_config)
+    if not api_key and provider_id not in {"ollama", "opencode-go", "openrouter"}:
+        raise HTTPException(
+            status_code=400, detail="请先配置当前供应商的 API Key，再刷新模型列表。",
+        )
+    try:
+        refreshed = await refresh_llm_provider_preset(
+            provider_id, api_key=api_key, base_url=provider_config.get("base_url"),
+            provider_api=provider_config.get("provider_api"),
+            anthropic_version=provider_config.get("anthropic_version") or "2023-06-01",
+        )
     except Exception as exc:
-        fallback = get_fallback_llm_provider_preset(provider)
-        fallback["refresh_error"] = str(exc)
-        return {"data": fallback}
+        logger.warning(
+            "LLM provider catalog refresh failed",
+            extra={
+                "event": "settings.ai_provider.catalog.failed",
+                "context": {
+                    "provider": provider_id,
+                    "error_type": type(exc).__name__,
+                },
+            },
+        )
+        raise HTTPException(
+            status_code=502,
+            detail=f"{catalog_error_message(exc)} 已保留上次模型列表。",
+        ) from exc
+
+    refreshed["refreshed_at"] = to_iso8601_utc(datetime.now(UTC))
+    await save_setting_payload(db, f"{LLM_PROVIDER_PRESET_CATEGORY_PREFIX}{provider_id}", refreshed)
+    return {"data": refreshed}
 
 
 @router.put("/integrations")

backend/app/api/v1/system_control.py

@@ -1,16 +1,17 @@
 from __future__ import annotations
 
 import os
+import secrets
 import subprocess
 import sys
 
 from datetime import datetime
 
-from fastapi import APIRouter, Depends, HTTPException, Query, Request, status
+from fastapi import APIRouter, Depends, Header, HTTPException, Query, Request, status
 from pydantic import BaseModel
 from sqlalchemy.ext.asyncio import AsyncSession
 
-from app.core.config import ROOT_DIR
+from app.core.config import ROOT_DIR, settings
 from app.core.security import get_current_user
 from app.db.session import get_db
 from app.models.user import User
@@ -37,6 +38,9 @@ from app.services.system_logs import (
     normalize_log_level,
     read_database_log_snapshot,
     read_log_snapshot,
+    read_observability_group_events,
+    read_observability_groups,
+    read_observability_raw_events,
 )
 from app.services.earth_layer_cache import earth_layer_cache
 
@@ -108,12 +112,34 @@ class EarthClientLogEventCreate(BaseModel):
     url: str | None = None
     module: str | None = None
     detail: str | None = None
+    fingerprint: str | None = None
+    occurrence_count: int = 1
+    metadata: dict[str, object] | None = None
 
 
 class EarthClientLogEventResponse(BaseModel):
     accepted: bool
     source_id: str
     level: str
+    fingerprint: str | None = None
+
+
+class ServiceLogEventCreate(BaseModel):
+    source: str = "ai-provider"
+    service: str = "ai-provider"
+    module: str | None = None
+    category: str | None = None
+    event: str = "service.runtime_log"
+    level: str = "error"
+    message: str
+    fingerprint: str | None = None
+    occurrence_count: int = 1
+    request_id: str | None = None
+    trace_id: str | None = None
+    task_id: str | None = None
+    source_id: int | str | None = None
+    provider: str | None = None
+    context: dict[str, object] | None = None
 
 
 async def ingest_client_log_event(
@@ -136,6 +162,9 @@ async def ingest_client_log_event(
             "url": payload.url or "",
             "module": payload.module or "",
             "detail": payload.detail or "",
+            "fingerprint": payload.fingerprint or "",
+            "occurrence_count": max(1, int(payload.occurrence_count or 1)),
+            "metadata": payload.metadata or {},
         },
     )
     await record_system_log(
@@ -151,9 +180,36 @@ async def ingest_client_log_event(
             "detail": payload.detail or "",
             "module": payload.module or "",
             "client_ip": request.client.host if request.client else "",
+            "metadata": payload.metadata or {},
         },
+        fingerprint=payload.fingerprint,
+        occurrence_count=max(1, int(payload.occurrence_count or 1)),
     )
-    return EarthClientLogEventResponse(accepted=True, source_id=source_id, level=normalized_level)
+    return EarthClientLogEventResponse(accepted=True, source_id=source_id, level=normalized_level, fingerprint=payload.fingerprint)
+
+
+def require_observability_ingest_token(
+    authorization: str | None,
+    ingest_token: str | None,
+) -> None:
+    expected_token = settings.OBSERVABILITY_INGEST_TOKEN.strip()
+    if not expected_token:
+        raise HTTPException(
+            status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
+            detail="Observability service ingestion is not configured",
+        )
+    provided = ""
+    if ingest_token:
+        provided = ingest_token.strip()
+    elif authorization:
+        scheme, _, token = authorization.partition(" ")
+        if scheme.lower() == "bearer":
+            provided = token.strip()
+    if not provided or not secrets.compare_digest(provided, expected_token):
+        raise HTTPException(
+            status_code=status.HTTP_403_FORBIDDEN,
+            detail="Invalid observability ingestion token",
+        )
 
 
 class EarthLayerCacheStatusResponse(BaseModel):
@@ -378,6 +434,118 @@ async def get_system_log_sources(
     }
 
 
+@router.get("/logs/observability/groups")
+async def get_observability_log_groups(
+    limit: int = DEFAULT_LOG_LINE_LIMIT,
+    level: str = "all",
+    levels: str | None = Query(None, description="Comma-separated log levels"),
+    start_date: str | None = Query(None, description="Filter logs from this date (YYYY-MM-DD)"),
+    end_date: str | None = Query(None, description="Filter logs until this date (YYYY-MM-DD)"),
+    search: str | None = Query(None, description="Case-insensitive substring search"),
+    current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    ensure_super_admin(current_user)
+    if limit < 1 or limit > MAX_LOG_LINE_LIMIT:
+        raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail=f"limit must be between 1 and {MAX_LOG_LINE_LIMIT}")
+    normalized_start_date = validate_log_date(start_date, "start_date")
+    normalized_end_date = validate_log_date(end_date, "end_date")
+    if normalized_start_date and normalized_end_date and normalized_start_date > normalized_end_date:
+        raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail="start_date must be earlier than or equal to end_date")
+    return await read_observability_groups(
+        limit=limit,
+        level=level,
+        levels=levels,
+        start_date=normalized_start_date,
+        end_date=normalized_end_date,
+        search=search,
+        db=db,
+    )
+
+
+@router.get("/logs/observability/groups/{fingerprint}/events")
+async def get_observability_group_events(
+    fingerprint: str,
+    limit: int = DEFAULT_LOG_LINE_LIMIT,
+    current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    ensure_super_admin(current_user)
+    if limit < 1 or limit > MAX_LOG_LINE_LIMIT:
+        raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail=f"limit must be between 1 and {MAX_LOG_LINE_LIMIT}")
+    payload = await read_observability_group_events(fingerprint, limit=limit, db=db)
+    if payload is None:
+        raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Observability group not found")
+    return payload
+
+
+@router.get("/logs/observability/raw")
+async def get_observability_raw_events(
+    limit: int = DEFAULT_LOG_LINE_LIMIT,
+    level: str = "all",
+    levels: str | None = Query(None, description="Comma-separated log levels"),
+    start_date: str | None = Query(None, description="Filter logs from this date (YYYY-MM-DD)"),
+    end_date: str | None = Query(None, description="Filter logs until this date (YYYY-MM-DD)"),
+    search: str | None = Query(None, description="Case-insensitive substring search"),
+    current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    ensure_super_admin(current_user)
+    if limit < 1 or limit > MAX_LOG_LINE_LIMIT:
+        raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail=f"limit must be between 1 and {MAX_LOG_LINE_LIMIT}")
+    normalized_start_date = validate_log_date(start_date, "start_date")
+    normalized_end_date = validate_log_date(end_date, "end_date")
+    return await read_observability_raw_events(
+        limit=limit,
+        level=level,
+        levels=levels,
+        start_date=normalized_start_date,
+        end_date=normalized_end_date,
+        search=search,
+        db=db,
+    )
+
+
+@router.post("/logs/service", response_model=EarthClientLogEventResponse)
+async def ingest_service_log(
+    payload: ServiceLogEventCreate,
+    authorization: str | None = Header(default=None),
+    ingest_token: str | None = Header(default=None, alias="X-Planet-Observability-Token"),
+):
+    require_observability_ingest_token(authorization, ingest_token)
+    normalized_level = normalize_log_level(payload.level)
+    source = (payload.source or "ai-provider").strip() or "ai-provider"
+    context = dict(payload.context or {})
+    if payload.request_id:
+        context["request_id"] = payload.request_id
+    if payload.trace_id:
+        context["trace_id"] = payload.trace_id
+    if payload.task_id:
+        context["task_id"] = payload.task_id
+    if payload.source_id is not None:
+        context["source_id"] = payload.source_id
+    if payload.provider:
+        context["provider"] = payload.provider
+    await record_system_log(
+        source=source,
+        service=(payload.service or source).strip() or source,
+        module=payload.module or source,
+        event=(payload.event or "service.runtime_log").strip() or "service.runtime_log",
+        level=normalized_level,
+        message=payload.message,
+        category=payload.category or "service-runtime",
+        context=context,
+        fingerprint=payload.fingerprint,
+        occurrence_count=max(1, int(payload.occurrence_count or 1)),
+    )
+    return EarthClientLogEventResponse(
+        accepted=True,
+        source_id=source,
+        level=normalized_level,
+        fingerprint=payload.fingerprint,
+    )
+
+
 @router.get("/logs/{source_id}", response_model=SystemLogSnapshotResponse)
 async def get_system_log_snapshot(
     source_id: str,

backend/app/api/v1/tasks.py

@@ -29,7 +29,8 @@ async def list_tasks(
         SELECT ct.id, ct.datasource_id, ds.name as datasource_name, ct.status,
                ct.started_at, ct.completed_at, ct.records_processed, ct.error_message,
                ct.phase, ct.phase_progress, ct.phase_message, ct.phase_current,
-               ct.phase_total, ct.phase_unit, ct.total_records, ct.progress
+               ct.phase_total, ct.phase_unit, ct.total_records, ct.progress,
+               ct.task_type, ct.source, ds.source as datasource_source
         FROM collection_tasks ct
         JOIN data_sources ds ON ct.datasource_id = ds.id
         WHERE 1=1
@@ -39,12 +40,19 @@ async def list_tasks(
 
     if datasource_id:
         query += " AND ct.datasource_id = :datasource_id"
-        count_query += " WHERE ct.datasource_id = :datasource_id"
+        count_query += " AND ct.datasource_id = :datasource_id"
         params["datasource_id"] = datasource_id
     if status:
-        query += " AND ct.status = :status"
-        count_query += " AND ct.status = :status"
-        params["status"] = status
+        statuses = [item.strip() for item in status.split(",") if item.strip()]
+        if len(statuses) > 1:
+            placeholders = ", ".join(f":status_{index}" for index, _item in enumerate(statuses))
+            query += f" AND ct.status IN ({placeholders})"
+            count_query += f" AND ct.status IN ({placeholders})"
+            params.update({f"status_{index}": item for index, item in enumerate(statuses)})
+        else:
+            query += " AND ct.status = :status"
+            count_query += " AND ct.status = :status"
+            params["status"] = statuses[0] if statuses else status
 
     query += f" ORDER BY ct.created_at DESC LIMIT {page_size} OFFSET {offset}"
 
@@ -76,6 +84,9 @@ async def list_tasks(
                 "phase_unit": t[13],
                 "total_records": t[14],
                 "progress": t[15],
+                "task_type": t[16],
+                "source": t[17] or t[18],
+                "datasource_source": t[18],
             }
             for t in tasks
         ],

backend/app/api/v1/tv.py

@@ -1,3 +1,4 @@
+import re
 from urllib.parse import quote, urljoin
 
 import httpx
@@ -6,16 +7,41 @@ from fastapi.responses import Response
 from sqlalchemy.ext.asyncio import AsyncSession
 
 from app.db.session import get_db
+from app.services.tv_catalog import get_tv_catalog_page
 from app.services.tv_streams import get_public_tv_payload, is_allowed_tv_proxy_url
 
 router = APIRouter()
 
+_HLS_URI_ATTRIBUTE_RE = re.compile(r'URI="([^"]+)"')
+
+
+def _proxied_tv_url(url: str) -> str:
+    return f"/api/v1/tv/proxy?url={quote(url, safe='')}"
+
+
+def _rewrite_hls_uri_attributes(line: str, *, base_url: str) -> str:
+    def replace(match: re.Match[str]) -> str:
+        uri = match.group(1)
+        absolute_url = urljoin(base_url, uri)
+        return f'URI="{_proxied_tv_url(absolute_url)}"'
+
+    return _HLS_URI_ATTRIBUTE_RE.sub(replace, line)
+
+
+def _should_strip_hls_metadata_line(line: str) -> bool:
+    normalized = line.strip().upper()
+    return normalized.startswith("#EXT-X-MEDIA:") and "TYPE=SUBTITLES" in normalized
+
 
 @router.get("/streams")
 async def list_public_tv_streams(
+    offset: int = Query(0, ge=0),
+    limit: int = Query(50, ge=1, le=100),
+    q: str = Query("", max_length=200),
+    selected_id: str | None = Query(None, max_length=200),
     db: AsyncSession = Depends(get_db),
 ):
-    return await get_public_tv_payload(db)
+    return await get_tv_catalog_page(db, offset=offset, limit=limit, q=q, selected_id=selected_id)
 
 
 @router.get("/proxy")
@@ -56,11 +82,16 @@ async def proxy_tv_stream(
         rewritten_lines: list[str] = []
         for line in manifest_text.splitlines():
             stripped = line.strip()
-            if not stripped or stripped.startswith("#"):
+            if not stripped:
                 rewritten_lines.append(line)
                 continue
+            if stripped.startswith("#"):
+                if _should_strip_hls_metadata_line(line):
+                    continue
+                rewritten_lines.append(_rewrite_hls_uri_attributes(line, base_url=response_url))
+                continue
             absolute_url = urljoin(response_url, stripped)
-            rewritten_lines.append(f"/api/v1/tv/proxy?url={quote(absolute_url, safe='')}")
+            rewritten_lines.append(_proxied_tv_url(absolute_url))
         return Response(
             content="\n".join(rewritten_lines),
             media_type="application/vnd.apple.mpegurl",

backend/app/api/v1/users.py

@@ -5,6 +5,7 @@ from fastapi import APIRouter, Depends, HTTPException, status
 from sqlalchemy.ext.asyncio import AsyncSession
 from sqlalchemy import text
 
+from app.core.enums import UserRole
 from app.core.security import get_current_user, get_password_hash
 from app.db.session import get_db
 from app.models.user import User
@@ -13,6 +14,8 @@ from app.schemas.user import UserCreate, UserUpdate
 router = APIRouter()
 
 VALID_GATEKEEPER_GROUPS = {"docs_user", "docs_developer", "docs_admin"}
+ADMIN_ROLES = [UserRole.SUPER_ADMIN.value, UserRole.ADMIN.value]
+SUPER_ADMIN_ROLES = [UserRole.SUPER_ADMIN.value]
 
 
 def check_permission(current_user: User, required_roles: List[str]) -> bool:
@@ -32,7 +35,7 @@ async def list_users(
     current_user: User = Depends(get_current_user),
     db: AsyncSession = Depends(get_db),
 ):
-    if not check_permission(current_user, ["super_admin", "admin"]):
+    if not check_permission(current_user, ADMIN_ROLES):
         raise HTTPException(
             status_code=status.HTTP_403_FORBIDDEN,
             detail="Insufficient permissions",
@@ -91,7 +94,7 @@ async def get_user(
     current_user: User = Depends(get_current_user),
     db: AsyncSession = Depends(get_db),
 ):
-    if not check_permission(current_user, ["super_admin", "admin"]) and current_user.id != user_id:
+    if not check_permission(current_user, ADMIN_ROLES) and current_user.id != user_id:
         raise HTTPException(
             status_code=status.HTTP_403_FORBIDDEN,
             detail="Insufficient permissions",
@@ -128,7 +131,7 @@ async def create_user(
     current_user: User = Depends(get_current_user),
     db: AsyncSession = Depends(get_db),
 ):
-    if not check_permission(current_user, ["super_admin"]):
+    if not check_permission(current_user, SUPER_ADMIN_ROLES):
         raise HTTPException(
             status_code=status.HTTP_403_FORBIDDEN,
             detail="Only super_admin can create users",
@@ -196,18 +199,18 @@ async def update_user(
     current_user: User = Depends(get_current_user),
     db: AsyncSession = Depends(get_db),
 ):
-    if not check_permission(current_user, ["super_admin", "admin"]) and current_user.id != user_id:
+    if not check_permission(current_user, ADMIN_ROLES) and current_user.id != user_id:
         raise HTTPException(
             status_code=status.HTTP_403_FORBIDDEN,
             detail="Insufficient permissions",
         )
 
-    if not check_permission(current_user, ["super_admin"]) and user_data.role is not None:
+    if not check_permission(current_user, SUPER_ADMIN_ROLES) and user_data.role is not None:
         raise HTTPException(
             status_code=status.HTTP_403_FORBIDDEN,
             detail="Only super_admin can change user role",
         )
-    if not check_permission(current_user, ["super_admin"]) and user_data.gatekeeper_groups is not None:
+    if not check_permission(current_user, SUPER_ADMIN_ROLES) and user_data.gatekeeper_groups is not None:
         raise HTTPException(
             status_code=status.HTTP_403_FORBIDDEN,
             detail="Only super_admin can change Gatekeeper groups",
@@ -260,7 +263,7 @@ async def delete_user(
     current_user: User = Depends(get_current_user),
     db: AsyncSession = Depends(get_db),
 ):
-    if not check_permission(current_user, ["super_admin"]):
+    if not check_permission(current_user, SUPER_ADMIN_ROLES):
         raise HTTPException(
             status_code=status.HTTP_403_FORBIDDEN,
             detail="Only super_admin can delete users",

backend/app/api/v1/vessels.py

@@ -1,4 +1,4 @@
-"""Bounded vessel snapshot APIs for viewport-first consumers."""
+"""Bounded vessel snapshot APIs backed by the latest vessel state table."""
 
 from typing import Optional
 
@@ -14,8 +14,8 @@ router = APIRouter()
 
 @router.get("/snapshot")
 async def get_vessel_snapshot(
-    bbox: Optional[str] = Query(None, description="Viewport bbox as lon_min,lat_min,lon_max,lat_max"),
-    zoom: int = Query(..., ge=1, le=20, description="Current map zoom level"),
+    bbox: Optional[str] = Query(None, description="Snapshot bbox as lon_min,lat_min,lon_max,lat_max"),
+    zoom: float = Query(..., ge=1, le=20, description="Current map zoom level"),
     type: Optional[str] = Query(
         None,
         description="Comma-separated vessel types: cargo,tanker,passenger,fishing,military,other",

backend/app/api/v1/visualization.py

@@ -17,15 +17,18 @@ from sqlalchemy.ext.asyncio import AsyncSession
 from sqlalchemy import select, func
 from typing import List, Dict, Any, Optional
 
+from app.core.compute_types import COMPUTE_SITE_TYPES, COMPUTE_SOURCES, compute_site_type, compute_record_filter, compute_site_type_expression
 from app.core.collected_data_fields import get_record_field
+from app.core.enums import BGPStatus
 from app.core.satellite_tle import build_tle_lines_from_elements
 from app.core.time import to_iso8601_utc
 from app.db.session import get_db
+from app.core.satellite_presentation import satellite_constellation_group as _normalize_satellite_constellation_group
 from app.models.bgp_anomaly import BGPAnomaly
 from app.models.bgp_incident import BGPIncident
 from app.models.bgp_observation import BGPObservation
 from app.models.collected_data import CollectedData
-from app.models.vessel import AISSourceHealth, VesselPosition, VesselStatic
+from app.models.vessel import AISSourceHealth, VesselCurrentState, VesselPosition, VesselStatic
 from app.services.bgp_collectors import build_bgp_collector_coverage
 from app.services.cable_graph import build_graph_from_data, CableGraph, haversine_distance
 from app.services.compute_center_locations import (
@@ -47,11 +50,10 @@ from app.services.location.llm_fallback import (
 from app.services.persistent_logs import record_system_log
 from app.services.vessel_ais_aggregation import (
     build_field_conflict_candidates,
-    count_unique_raw_vessel_mmsi,
     get_aggregated_vessel,
     get_aggregated_vessel_track,
     get_aggregated_vessels,
-    get_aggregated_vessels_snapshot,
+    get_current_vessels_snapshot,
     get_vessel_conflict_records,
     get_vessel_raw_observations,
     MAX_SNAPSHOT_LIMIT,
@@ -75,7 +77,6 @@ TERRAIN_TILE_BATCH_MAX_ITEMS = 128
 TERRAIN_TILE_BATCH_CONCURRENCY = 16
 _terrain_tile_cache: OrderedDict[tuple[int, int, int], tuple[bytes, str, dict[str, str]]] = OrderedDict()
 VESSEL_NAME_FALLBACK_PATTERN = re.compile(r"^mmsi\s*\d+$", re.IGNORECASE)
-VESSEL_SNAPSHOT_LEGACY_FALLBACK_ENABLED = True
 SECONDS_PER_MINUTE = 60
 BYTES_PER_MIB = 1024 * 1024
 CABLE_CACHE_FRESH_SECONDS = 6 * 60 * SECONDS_PER_MINUTE
@@ -338,22 +339,6 @@ def convert_satellite_to_geojson(records: List[CollectedData]) -> Dict[str, Any]
     return {"type": "FeatureCollection", "features": features}
 
 
-def _normalize_satellite_constellation_group(
-    raw_group: Any,
-    name: Optional[str],
-) -> Optional[str]:
-    normalized_group = str(raw_group or "").strip().lower()
-    if normalized_group:
-        return normalized_group
-
-    normalized_name = str(name or "").strip().upper()
-    if normalized_name.startswith("STARLINK"):
-        return "starlink"
-    if normalized_name.startswith("IRIDIUM"):
-        return "iridium-next"
-
-    return None
-
 
 def _get_satellite_footprint_policy(constellation_group: Optional[str]) -> str:
     if constellation_group == "starlink":
@@ -506,16 +491,19 @@ async def _count_current_or_latest_task_data(
 async def _load_current_collected_data_by_sources(
     db: AsyncSession,
     sources: List[str],
+    *, include_compute_types: bool = False,
 ) -> Dict[str, List[CollectedData]]:
     if not sources:
         return {}
 
     stmt = (
         select(CollectedData)
-        .where(CollectedData.source.in_(sources))
+        .where(compute_record_filter(CollectedData) if include_compute_types else CollectedData.source.in_(sources))
         .where(CollectedData.is_current.is_(True))
         .order_by(CollectedData.source.asc(), CollectedData.id.desc())
     )
+    if include_compute_types:
+        stmt = stmt.where(CollectedData.deleted_at.is_(None))
     result = await db.execute(stmt)
 
     grouped_records: Dict[str, List[CollectedData]] = {source: [] for source in sources}
@@ -645,7 +633,9 @@ def _normalize_capacity_band(capacity_value: Optional[float], capacity_unit: str
         return "unknown"
 
     unit = str(capacity_unit or "").strip().lower()
-    if unit in {"pflop/s", "pflops", "pflop"}:
+    if unit in {"flop/s", "flops"}:
+        normalized_tflops = capacity_value / 1e12
+    elif unit in {"pflop/s", "pflops", "pflop"}:
         normalized_tflops = capacity_value * 1000
     elif unit in {"gflop/s", "gflops", "gflop"}:
         normalized_tflops = capacity_value
@@ -673,15 +663,15 @@ def convert_compute_centers_to_geojson(records: List[CollectedData]) -> Dict[str
     """
     features: List[Dict[str, Any]] = []
     unresolved: List[Dict[str, Any]] = []
+    withheld_count = 0
 
     for record in records:
         metadata = record.extra_data or {}
+        if metadata.get("compute_contract") == "v1" and metadata.get("anonymous"):
+            withheld_count += 1
+            continue
         result = resolve_compute_center_location_full(record, metadata)
-        site_type = (
-            "supercomputer"
-            if record.source == "top500" or record.data_type == "supercomputer"
-            else "gpu_cluster"
-        )
+        site_type = compute_site_type(record)
 
         if not result.is_resolved:
             diagnostic = result.diagnostic or ResolutionDiagnostic(
@@ -707,7 +697,10 @@ def convert_compute_centers_to_geojson(records: List[CollectedData]) -> Dict[str
         latitude = location.latitude
         longitude = location.longitude
 
-        if site_type == "supercomputer":
+        if metadata.get("compute_contract") == "v1":
+            capacity_value = _parse_float(get_record_field(record, "value"))
+            capacity_unit = str(get_record_field(record, "unit") or "FLOP/s")
+        elif site_type == "supercomputer":
             capacity_value = _parse_float(get_record_field(record, "rmax"))
             capacity_unit = "GFlops"
         else:
@@ -768,7 +761,7 @@ def convert_compute_centers_to_geojson(records: List[CollectedData]) -> Dict[str
             }
         )
 
-    return {"type": "FeatureCollection", "features": features, "unresolved": unresolved}
+    return {"type": "FeatureCollection", "features": features, "unresolved": unresolved, "withheld_count": withheld_count}
 
 
 VESSEL_TYPE_FILTERS = {
@@ -839,9 +832,14 @@ def convert_aggregated_vessels_to_geojson(vessels: List[dict[str, Any]]) -> Dict
             continue
         source_summary = {}
         for source, summary in (vessel.get("source_summary") or {}).items():
+            latest_observed_at = summary.get("latest_observed_at")
             source_summary[source] = {
                 **summary,
-                "latest_observed_at": to_iso8601_utc(summary.get("latest_observed_at")),
+                "latest_observed_at": (
+                    to_iso8601_utc(latest_observed_at)
+                    if isinstance(latest_observed_at, datetime)
+                    else latest_observed_at
+                ),
             }
         props = {
             "mmsi": vessel["mmsi"],
@@ -1075,7 +1073,7 @@ async def build_vessel_snapshot_response(
     db: AsyncSession,
     *,
     bbox: tuple[float, float, float, float] | None,
-    zoom: int | None,
+    zoom: float | None,
     type_filter: str | None,
     limit: int | None,
     since_minutes: int = 60,
@@ -1117,6 +1115,7 @@ async def build_vessel_snapshot_response(
     safe_limit = _safe_vessel_limit(limit)
     safe_since_minutes = min(max(int(since_minutes or 60), 1), 1440)
     observed_since = datetime.now(UTC) - timedelta(minutes=safe_since_minutes)
+    snapshot_started_at = datetime.now(UTC)
     features, diagnostics = await _load_raw_vessel_snapshot_features(
         db,
         bbox=bbox,
@@ -1133,6 +1132,7 @@ async def build_vessel_snapshot_response(
         "features": features,
         "count": len(features),
         "stats": _build_vessel_stats(features),
+        "generated_at": to_iso8601_utc(snapshot_started_at),
         "diagnostics": {
             **diagnostics,
             "filtered_count": len(features),
@@ -1949,9 +1949,10 @@ async def _build_compute_centers_geojson(
     records_by_source = await _load_current_collected_data_by_sources(
         db,
         ["top500", "epoch_ai_gpu"],
+        include_compute_types=True,
     )
     records = _filter_known_records(
-        records_by_source.get("top500", []) + records_by_source.get("epoch_ai_gpu", []),
+        [record for group in records_by_source.values() for record in group],
     )
     if limit is not None:
         records = records[:limit]
@@ -1966,6 +1967,7 @@ async def _build_compute_centers_geojson(
                 "total": 0,
                 "supercomputers": 0,
                 "gpu_clusters": 0,
+                "by_type": dict.fromkeys(COMPUTE_SITE_TYPES, 0),
                 "unresolved": 0,
             },
         }
@@ -2008,6 +2010,7 @@ async def _build_compute_centers_geojson(
         "count": len(sanitized_features),
         "stats": {
             "total": len(sanitized_features),
+            "by_type": {kind: sum(feature["properties"]["site_type"] == kind for feature in sanitized_features) for kind in COMPUTE_SITE_TYPES},
             "supercomputers": sum(
                 1 for feature in sanitized_features
                 if feature.get("properties", {}).get("site_type") == "supercomputer"
@@ -2324,7 +2327,8 @@ async def _load_compute_center_record(db: AsyncSession, source_id: str) -> Colle
     stmt = (
         select(CollectedData)
         .where(CollectedData.source_id == source_id)
-        .where(CollectedData.source.in_(["top500", "epoch_ai_gpu"]))
+        .where(compute_record_filter(CollectedData))
+        .where(CollectedData.deleted_at.is_(None))
         .order_by(CollectedData.is_current.desc(), CollectedData.id.desc())
         .limit(1)
     )
@@ -2340,58 +2344,35 @@ async def _load_raw_vessel_snapshot_features(
     observed_since: datetime,
 ) -> tuple[list[dict[str, Any]], dict[str, Any]]:
     if bbox is None:
-        aggregated_vessels = await get_aggregated_vessels(
-            db,
-            limit=limit,
-            observed_since=observed_since,
-        )
-    else:
-        aggregated_vessels = await get_aggregated_vessels_snapshot(
-            db,
-            bbox=bbox,
-            limit=limit,
-            observed_since=observed_since,
-        )
-    raw_geojson = convert_aggregated_vessels_to_geojson(aggregated_vessels)
-    raw_features = raw_geojson.get("features", [])
-    features = raw_features
-    legacy_features: list[dict[str, Any]] = []
-    legacy_fallback_used = False
-    if not raw_features and VESSEL_SNAPSHOT_LEGACY_FALLBACK_ENABLED:
-        legacy_features = await _load_legacy_vessel_snapshot_features(
-            db,
-            bbox=bbox,
-            limit=limit,
-        )
-        features, _merge_diagnostics = _merge_vessel_features(raw_features, legacy_features)
-        legacy_fallback_used = bool(legacy_features)
-
+        return [], {
+            "source": "vessel_current_state",
+            "current_state_count": 0,
+            "final_unique_mmsi": 0,
+        }
+    current_vessels = await get_current_vessels_snapshot(
+        db,
+        bbox=bbox,
+        limit=limit,
+        observed_since=observed_since,
+    )
+    features = convert_aggregated_vessels_to_geojson(current_vessels).get("features", [])
+    unique_mmsi = len(
+        {
+            key
+            for key in (_feature_mmsi_key(feature) for feature in features)
+            if key is not None
+        }
+    )
     return features, {
-        "raw_feature_count": len(raw_features),
-        "raw_unique_mmsi": len(
-            {
-                key
-                for key in (_feature_mmsi_key(feature) for feature in raw_features)
-                if key is not None
-            }
-        ),
-        "legacy_feature_count": len(legacy_features),
-        "legacy_backfilled_mmsi": len(
-            {
-                key
-                for key in (_feature_mmsi_key(feature) for feature in legacy_features)
-                if key is not None
-            }
-        ),
-        "legacy_fallback_enabled": VESSEL_SNAPSHOT_LEGACY_FALLBACK_ENABLED,
-        "legacy_fallback_used": legacy_fallback_used,
-        "final_unique_mmsi": len(
-            {
-                key
-                for key in (_feature_mmsi_key(feature) for feature in features)
-                if key is not None
-            }
-        ),
+        "source": "vessel_current_state",
+        "current_state_count": len(features),
+        "final_unique_mmsi": unique_mmsi,
+        "raw_feature_count": 0,
+        "raw_unique_mmsi": 0,
+        "legacy_feature_count": 0,
+        "legacy_backfilled_mmsi": 0,
+        "legacy_fallback_enabled": False,
+        "legacy_fallback_used": False,
     }
 
 @router.get("/vessels/custom-supplements")
@@ -2746,6 +2727,25 @@ async def get_visualization_geo_summary(db: AsyncSession = Depends(get_db), resp
     )
 
 
+async def _compute_facility_counts(db: AsyncSession) -> dict[str, int]:
+    kind = compute_site_type_expression(CollectedData)
+    rows = (await db.execute(select(CollectedData.source, kind, func.count(CollectedData.id))
+        .where(compute_record_filter(CollectedData), CollectedData.is_current.is_(True),
+               CollectedData.deleted_at.is_(None))
+        .group_by(CollectedData.source, kind))).all()
+    counts = dict.fromkeys(COMPUTE_SITE_TYPES, 0)
+    present_sources = set()
+    for source, site_type, count in rows:
+        present_sources.add(source)
+        counts[site_type] += int(count or 0)
+    # Retain the existing last-task fallback for the two collected catalogs.
+    for source in COMPUTE_SOURCES:
+        if source not in present_sources:
+            key = "supercomputer" if source == "top500" else "gpu_cluster"
+            counts[key] += await _count_current_or_latest_task_data(db, source)
+    return counts
+
+
 async def _build_visualization_geo_summary(db: AsyncSession) -> dict[str, Any]:
     cable_count = await _count_current_or_latest_task_data(db, "arcgis_cables")
     landing_point_count = await _count_current_or_latest_task_data(db, "arcgis_landing_points")
@@ -2754,15 +2754,16 @@ async def _build_visualization_geo_summary(db: AsyncSession) -> dict[str, Any]:
         "celestrak_tle",
         exclude_unknown_name=True,
     )
-    supercomputer_count = await _count_current_or_latest_task_data(db, "top500")
-    gpu_cluster_count = await _count_current_or_latest_task_data(db, "epoch_ai_gpu")
-    compute_center_count = supercomputer_count + gpu_cluster_count
+    compute_counts = await _compute_facility_counts(db)
+    supercomputer_count = compute_counts["supercomputer"]
+    gpu_cluster_count = compute_counts["gpu_cluster"]
+    compute_center_count = sum(compute_counts.values())
 
     active_incident_result = await db.execute(
-        select(func.count(BGPIncident.id)).where(BGPIncident.status == "active"),
+        select(func.count(BGPIncident.id)).where(BGPIncident.status == BGPStatus.ACTIVE.value),
     )
     active_anomaly_result = await db.execute(
-        select(func.count(BGPAnomaly.id)).where(BGPAnomaly.status == "active"),
+        select(func.count(BGPAnomaly.id)).where(BGPAnomaly.status == BGPStatus.ACTIVE.value),
     )
     active_incident_count = int(active_incident_result.scalar() or 0)
     active_anomaly_count = int(active_anomaly_result.scalar() or 0)
@@ -2783,21 +2784,14 @@ async def _build_visualization_geo_summary(db: AsyncSession) -> dict[str, Any]:
         )
     else:
         bgp_collector_count = int(bgp_collector_scalar or 0)
-    raw_unique_window_hours = 24
-    raw_unique_mmsi = await count_unique_raw_vessel_mmsi(
-        db,
-        observed_since=datetime.now(UTC) - timedelta(hours=raw_unique_window_hours),
+    vessel_current_window_minutes = 60
+    vessel_current_result = await db.execute(
+        select(func.count(VesselCurrentState.mmsi)).where(
+            VesselCurrentState.observed_at
+            >= datetime.now(UTC) - timedelta(minutes=vessel_current_window_minutes)
+        )
     )
-    legacy_unique_result = await db.execute(
-        select(func.count(func.distinct(VesselPosition.mmsi)))
-    )
-    legacy_unique_mmsi = int(legacy_unique_result.scalar() or 0)
-    legacy_fallback_active = (
-        VESSEL_SNAPSHOT_LEGACY_FALLBACK_ENABLED
-        and raw_unique_mmsi == 0
-        and legacy_unique_mmsi > 0
-    )
-    vessel_count = legacy_unique_mmsi if legacy_fallback_active else raw_unique_mmsi
+    vessel_count = int(vessel_current_result.scalar() or 0)
     aisstream_health = await db.get(AISSourceHealth, "aisstream_vessels")
 
     return {
@@ -2808,17 +2802,19 @@ async def _build_visualization_geo_summary(db: AsyncSession) -> dict[str, Any]:
             "satellite_count": satellite_count,
             "compute_center_count": compute_center_count,
             "vessel_count": vessel_count,
-            "vessel_count_source": "legacy_fallback" if legacy_fallback_active else "raw_recent",
-            "vessel_legacy_fallback_enabled": VESSEL_SNAPSHOT_LEGACY_FALLBACK_ENABLED,
-            "vessel_raw_unique_mmsi": raw_unique_mmsi,
-            "vessel_raw_unique_window_hours": raw_unique_window_hours,
-            "vessel_legacy_unique_mmsi": legacy_unique_mmsi,
+            "vessel_count_source": "vessel_current_state",
+            "vessel_current_window_minutes": vessel_current_window_minutes,
+            "vessel_raw_unique_mmsi": 0,
+            "vessel_legacy_unique_mmsi": 0,
             "aisstream_connection_state": aisstream_health.connection_state if aisstream_health else None,
             "aisstream_last_seen_at": to_iso8601_utc(aisstream_health.last_seen_at) if aisstream_health else None,
             "aisstream_message_rate": aisstream_health.message_rate if aisstream_health else None,
             "aisstream_lag_seconds": aisstream_health.lag_seconds if aisstream_health else None,
             "supercomputer_count": supercomputer_count,
             "gpu_cluster_count": gpu_cluster_count,
+            "data_center_count": compute_counts["data_center"],
+            "cloud_center_count": compute_counts["cloud_center"],
+            "edge_node_count": compute_counts["edge_node"],
             "bgp_event_count": active_incident_count or active_anomaly_count,
             "bgp_incident_count": active_incident_count,
             "bgp_anomaly_count": active_anomaly_count,

backend/app/api/v1/websocket.py

@@ -9,6 +9,7 @@ from jose import jwt, JWTError
 from sqlalchemy import text
 
 from app.core.config import settings
+from app.core.enums import UserRole
 from app.core.logging import get_logger
 from app.core.time import to_iso8601_utc
 from app.core.websocket.manager import manager
@@ -96,7 +97,7 @@ async def websocket_endpoint(
         "earth_news",
         EARTH_UPDATES_CHANNEL,
     ]
-    if user_role == "super_admin":
+    if user_role == UserRole.SUPER_ADMIN.value:
         supported_channels = [*supported_channels, LOG_TAIL_CHANNEL]
     await manager.connect(websocket, user_id)
 
@@ -138,7 +139,7 @@ async def websocket_endpoint(
                     if channel and channel not in channels:
                         channels = [*channels, channel]
                     if LOG_TAIL_CHANNEL in channels:
-                        if user_role != "super_admin":
+                        if user_role != UserRole.SUPER_ADMIN.value:
                             await websocket.send_json(
                                 {
                                     "type": "subscription_error",
@@ -160,7 +161,7 @@ async def websocket_endpoint(
                     if is_anonymous:
                         channels = [channel for channel in channels if channel in supported_channels]
                     vessel_subscription = None
-                    if "vessels" in channels and "bbox" in payload_data:
+                    if "vessels" in channels:
                         try:
                             vessel_subscription = manager.subscribe_vessels(websocket, payload_data)
                         except ValueError as exc:

backend/app/core/compute_types.py

@@ -0,0 +1,35 @@
+"""Compute facility classification shared by the map and image resolver."""
+
+from sqlalchemy import case, or_
+
+COMPUTE_SITE_TYPES = ("supercomputer", "gpu_cluster", "data_center", "cloud_center", "edge_node")
+COMPUTE_SOURCES = ("top500", "epoch_ai_gpu")
+
+
+def compute_site_type(record) -> str:
+    metadata = record.extra_data or {}
+    for value in (metadata.get("site_type"), record.data_type):
+        if value in COMPUTE_SITE_TYPES:
+            return value
+    return {"top500": "supercomputer", "epoch_ai_gpu": "gpu_cluster"}.get(
+        record.source, "data_center"
+    )
+
+
+def compute_site_type_expression(record):
+    declared = record.extra_data["site_type"].as_string()
+    return case(
+        (declared.in_(COMPUTE_SITE_TYPES), declared),
+        (record.data_type.in_(COMPUTE_SITE_TYPES), record.data_type),
+        (record.source == "top500", "supercomputer"),
+        (record.source == "epoch_ai_gpu", "gpu_cluster"),
+        else_="data_center",
+    )
+
+
+def compute_record_filter(record):
+    return or_(
+        record.source.in_(COMPUTE_SOURCES),
+        record.data_type.in_(COMPUTE_SITE_TYPES),
+        record.extra_data["site_type"].as_string().in_(COMPUTE_SITE_TYPES),
+    )

backend/app/core/config.py

@@ -41,6 +41,7 @@ class Settings(BaseSettings):
     AI_PROVIDER_SERVICE_TOKEN: str = ""
     AI_PROVIDER_TIMEOUT_SECONDS: int = 60
     AI_PROVIDER_RETRY_ATTEMPTS: int = 2
+    OBSERVABILITY_INGEST_TOKEN: str = ""
 
     @property
     def REDIS_URL(self) -> str:

backend/app/core/earth_geography_policy.py

@@ -0,0 +1,57 @@
+"""Country display ownership for Earth comparisons and cards.
+
+Keep source-country evidence separate; use this projection only for display
+grouping so regional source records cannot become competing national keys.
+"""
+
+from __future__ import annotations
+
+import json
+from functools import lru_cache
+from pathlib import Path
+
+from app.core.countries import COUNTRY_ENTRIES, normalize_country
+
+
+_DISPLAY_ISO_A3 = {
+    canonical: next(
+        (alias for alias in aliases if len(alias) == 3 and alias.isascii() and alias.isupper()),
+        None,
+    )
+    for canonical, aliases in COUNTRY_ENTRIES
+}
+_COUNTRY_BY_ISO_A3 = {code: country for country, code in _DISPLAY_ISO_A3.items() if code}
+_POLICY_PATH = Path(__file__).resolve().parents[3] / "config/earth-boundary-pov-policy.china-v1.json"
+
+
+@lru_cache(maxsize=1)
+def _policy_country_by_code() -> dict[str, str]:
+    try:
+        content = _POLICY_PATH.read_text(encoding="utf-8")
+    except FileNotFoundError as error:
+        raise FileNotFoundError(f"EARTH_DISPLAY_POLICY_MISSING: {_POLICY_PATH}") from error
+    policy = json.loads(content)
+    if policy.get("profile") != "china-pov-v1":
+        raise ValueError("unsupported Earth display-policy profile")
+    mappings = {}
+    for rule in policy["rules"]:
+        target = rule.get("targetIsoA3") or rule.get("parentIsoA3")
+        if target not in _COUNTRY_BY_ISO_A3:
+            continue
+        for code in rule.get("sourceIsoA3", []):
+            mappings[code] = _COUNTRY_BY_ISO_A3[target]
+    return mappings
+
+
+def display_country(value: str | None) -> str | None:
+    country = normalize_country(value)
+    code = _DISPLAY_ISO_A3.get(country) if country else str(value or "").strip().upper()
+    owner = _policy_country_by_code().get(code)
+    if owner:
+        return owner
+    return country
+
+
+def display_iso_a3(value: str | None) -> str | None:
+    country = display_country(value)
+    return _DISPLAY_ISO_A3.get(country) if country else None

backend/app/core/enums.py

@@ -0,0 +1,227 @@
+"""Stable backend protocol enums.
+
+Database columns and JSON payloads continue to store the enum string values.
+Configurable identifiers, user-authored values, and open-ended taxonomies do
+not belong in this module.
+"""
+
+from __future__ import annotations
+
+import logging
+from enum import StrEnum
+from typing import TypeVar
+
+logger = logging.getLogger(__name__)
+
+EnumT = TypeVar("EnumT", bound=StrEnum)
+
+
+def parse_enum(enum_type: type[EnumT], value: object, default: EnumT) -> EnumT:
+    """Parse an external value without breaking reads of legacy data."""
+
+    if value is None or str(value).strip() == "":
+        return default
+    if isinstance(value, enum_type):
+        return value
+    try:
+        return enum_type(str(value).strip().lower())
+    except (TypeError, ValueError):
+        logger.warning(
+            "Unknown %s value %r; falling back to %s",
+            enum_type.__name__,
+            value,
+            default.value,
+        )
+        return default
+
+
+class NewsImportanceLevel(StrEnum):
+    LOW = "low"
+    MEDIUM = "medium"
+    HIGH = "high"
+    CRITICAL = "critical"
+
+
+class BreakingLevel(StrEnum):
+    NONE = "none"
+    WATCH = "watch"
+    BREAKING = "breaking"
+    CRITICAL = "critical"
+
+
+class BreakingScope(StrEnum):
+    REGIONAL = "regional"
+    GLOBAL = "global"
+
+
+class BreakingSource(StrEnum):
+    RULES = "rules"
+    AI = "ai"
+    MANUAL = "manual"
+    MULTI_SOURCE = "multi_source"
+
+
+class NewsSourceType(StrEnum):
+    RSS = "rss"
+    ATOM = "atom"
+    AGGREGATED = "aggregated"
+    REFERENCE = "reference"
+    MANUAL = "manual"
+
+
+class NewsEnrichmentStatus(StrEnum):
+    PENDING = "pending"
+    QUEUED = "queued"
+    ATTEMPTED = "attempted"
+    SUCCESS = "success"
+    CONTENT_ONLY = "content_only"
+    LOCATION_ONLY = "location_only"
+    UNAVAILABLE = "unavailable"
+    PROVIDER_ERROR = "provider_error"
+    PARSE_ERROR = "parse_error"
+    NO_RESULT = "no_result"
+
+
+class NewsMarketImpact(StrEnum):
+    NONE = "none"
+    SECTOR = "sector"
+    NATIONAL = "national"
+    GLOBAL = "global"
+
+
+class NewsTaggingSource(StrEnum):
+    RULES = "rules"
+    AI = "ai"
+    MANUAL = "manual"
+
+
+class JobType(StrEnum):
+    COLLECT = "collect"
+    CLEAR_DATA = "clear_data"
+    CLEAR_CACHE = "clear_cache"
+    EARTH_REFRESH = "earth_refresh"
+
+
+class JobStatus(StrEnum):
+    QUEUED = "queued"
+    RUNNING = "running"
+    CANCELLING = "cancelling"
+    SUCCESS = "success"
+    FAILED = "failed"
+    CANCELLED = "cancelled"
+
+
+class RollbackPolicy(StrEnum):
+    KEEP_COMMITTED_BATCHES = "keep_committed_batches"
+
+
+class MappingValidationStatus(StrEnum):
+    DRAFT = "draft"
+    VALID = "valid"
+    INVALID = "invalid"
+
+
+class SnapshotStatus(StrEnum):
+    RUNNING = "running"
+    SUCCESS = "success"
+    FAILED = "failed"
+    CANCELLED = "cancelled"
+
+
+class DatasourceRunStatus(StrEnum):
+    RUNNING = "running"
+    NOT_RUN = "not_run"
+    COLLECTED = "collected"
+    UNCOLLECTED = "uncollected"
+
+
+class ProviderApi(StrEnum):
+    ANTHROPIC_MESSAGES = "anthropic-messages"
+    OPENAI_COMPLETIONS = "openai-completions"
+    OPENAI_RESPONSES = "openai-responses"
+    OLLAMA_GENERATE = "ollama-generate"
+
+
+class PlaygroundMessageRole(StrEnum):
+    SYSTEM = "system"
+    USER = "user"
+    ASSISTANT = "assistant"
+    TOOL = "tool"
+
+
+class PlaygroundMessageKind(StrEnum):
+    MESSAGE = "message"
+    THINKING = "thinking"
+    ERROR = "error"
+    STATUS = "status"
+
+
+class PlaygroundMessageStatus(StrEnum):
+    PENDING = "pending"
+    THINKING = "thinking"
+    ANSWERING = "answering"
+    DONE = "done"
+    FAILED = "failed"
+    CANCELLED = "cancelled"
+    ERROR = "error"
+    STOPPED = "stopped"
+
+
+class OtpPurpose(StrEnum):
+    REGISTER = "register"
+    VERIFY_EMAIL = "verify_email"
+    RESET_PASSWORD = "reset_password"
+
+
+class UserRole(StrEnum):
+    VIEWER = "viewer"
+    ADMIN = "admin"
+    SUPER_ADMIN = "super_admin"
+
+
+class AlertSeverity(StrEnum):
+    CRITICAL = "critical"
+    WARNING = "warning"
+    INFO = "info"
+
+
+class AlertStatus(StrEnum):
+    ACTIVE = "active"
+    ACKNOWLEDGED = "acknowledged"
+    RESOLVED = "resolved"
+
+
+class BGPStatus(StrEnum):
+    ACTIVE = "active"
+    ACKNOWLEDGED = "acknowledged"
+    RESOLVED = "resolved"
+
+
+class LogLevel(StrEnum):
+    ALL = "all"
+    ERROR = "error"
+    WARNING = "warning"
+    INFO = "info"
+    DEBUG = "debug"
+
+
+class ConnectionState(StrEnum):
+    DISCONNECTED = "disconnected"
+    CONNECTING = "connecting"
+    CONNECTED = "connected"
+    ERROR = "error"
+
+
+class AuthType(StrEnum):
+    NONE = "none"
+    BEARER = "bearer"
+    API_KEY = "api_key"
+    BASIC = "basic"
+
+
+class TVSourceType(StrEnum):
+    IFRAME = "iframe"
+    HLS = "hls"
+    VIDEO = "video"
+    EXTERNAL = "external"
+    YOUTUBE = "youtube"

backend/app/core/satellite_presentation.py

@@ -0,0 +1,13 @@
+"""Shared source-group normalization; never infer hardware generations."""
+
+
+def satellite_constellation_group(raw_group: object, name: str | None) -> str | None:
+    group = str(raw_group or "").strip().lower()
+    if group:
+        return group
+    normalized_name = str(name or "").strip().upper()
+    if normalized_name.startswith("STARLINK"):
+        return "starlink"
+    if normalized_name.startswith("IRIDIUM"):
+        return "iridium-next"
+    return None

backend/app/core/security.py

@@ -6,7 +6,7 @@ import redis
 from fastapi import Depends, HTTPException, status
 from fastapi.security import HTTPAuthorizationCredentials, HTTPBearer
 from jose import JWTError, jwt
-from sqlalchemy import text
+from sqlalchemy import select
 from sqlalchemy.ext.asyncio import AsyncSession
 
 from app.core.config import settings
@@ -81,6 +81,21 @@ def decode_token(token: str) -> Optional[dict]:
         return None
 
 
+async def _load_active_user(user_id: str, db: AsyncSession) -> User:
+    try:
+        parsed_id = int(user_id)
+    except (TypeError, ValueError):
+        raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail="Invalid token")
+    result = await db.execute(select(User).where(User.id == parsed_id))
+    user = result.scalar_one_or_none()
+    if user is None or not user.is_active:
+        raise HTTPException(
+            status_code=status.HTTP_401_UNAUTHORIZED,
+            detail="User not found or inactive",
+        )
+    return user
+
+
 async def get_current_user(
     credentials: HTTPAuthorizationCredentials = Depends(oauth2_scheme),
     db: AsyncSession = Depends(get_db),
@@ -103,27 +118,7 @@ async def get_current_user(
             status_code=status.HTTP_401_UNAUTHORIZED,
             detail="Invalid token",
         )
-    result = await db.execute(
-        text(
-            "SELECT id, username, email, password_hash, role, is_active, gatekeeper_groups FROM users WHERE id = :id"
-        ),
-        {"id": int(user_id)},
-    )
-    row = result.fetchone()
-    if row is None or not row[5]:
-        raise HTTPException(
-            status_code=status.HTTP_401_UNAUTHORIZED,
-            detail="User not found or inactive",
-        )
-    user = User()
-    user.id = row[0]
-    user.username = row[1]
-    user.email = row[2]
-    user.password_hash = row[3]
-    user.role = row[4]
-    user.is_active = row[5]
-    user.gatekeeper_groups = row[6] or []
-    return user
+    return await _load_active_user(user_id, db)
 
 
 async def get_current_user_refresh(
@@ -143,27 +138,7 @@ async def get_current_user_refresh(
             status_code=status.HTTP_401_UNAUTHORIZED,
             detail="Invalid token",
         )
-    result = await db.execute(
-        text(
-            "SELECT id, username, email, password_hash, role, is_active, gatekeeper_groups FROM users WHERE id = :id"
-        ),
-        {"id": int(user_id)},
-    )
-    row = result.fetchone()
-    if row is None or not row[5]:
-        raise HTTPException(
-            status_code=status.HTTP_401_UNAUTHORIZED,
-            detail="User not found or inactive",
-        )
-    user = User()
-    user.id = row[0]
-    user.username = row[1]
-    user.email = row[2]
-    user.password_hash = row[3]
-    user.role = row[4]
-    user.is_active = row[5]
-    user.gatekeeper_groups = row[6] or []
-    return user
+    return await _load_active_user(user_id, db)
 
 
 def blacklist_token(token: str) -> None:

backend/app/core/websocket/broadcaster.py

@@ -4,11 +4,14 @@ import asyncio
 from datetime import UTC, datetime
 from typing import Dict, Any
 
+from app.core.logging import get_logger
 from app.core.time import to_iso8601_utc
 from app.core.websocket.manager import manager
 
 
 EARTH_UPDATES_CHANNEL = "earth_updates"
+VESSEL_STATE_QUERY_BATCH_SIZE = 1000
+logger = get_logger(__name__, service="websocket")
 
 
 class DataBroadcaster:
@@ -114,16 +117,22 @@ class DataBroadcaster:
             return
         pending = self._pending_vessel_updates
         self._pending_vessel_updates = {}
-        vessels = []
-        for item in pending.values():
-            vessel = dict(item)
-            source = vessel.pop("_source", None)
-            action = vessel.pop("_action", "upsert")
-            created = vessel.pop("_created", None)
-            vessel["source"] = source
-            vessel["action"] = action
-            vessel["created"] = created
-            vessels.append(vessel)
+        try:
+            vessels = await self._load_current_vessel_updates(list(pending))
+        except Exception:
+            # Preserve newer updates that arrived during the failed database read.
+            self._pending_vessel_updates = {**pending, **self._pending_vessel_updates}
+            raise
+        if not vessels:
+            return
+        vessels = [
+            {
+                **vessel,
+                "action": vessel.get("action", "upsert"),
+                "created": pending.get(str(vessel["mmsi"]), {}).get("_created"),
+            }
+            for vessel in vessels
+        ]
         await manager.broadcast_vessels(
             {
                 "action": "upsert",
@@ -133,12 +142,31 @@ class DataBroadcaster:
             }
         )
 
+    async def _load_current_vessel_updates(self, keys: list[str]) -> list[Dict[str, Any]]:
+        from app.db.session import async_session_factory
+        from app.services.vessel_ais_aggregation import get_current_vessels_by_mmsi
+
+        mmsis = [int(key) for key in keys if key.isdigit()]
+        vessels = []
+        async with async_session_factory() as db:
+            for offset in range(0, len(mmsis), VESSEL_STATE_QUERY_BATCH_SIZE):
+                vessels.extend(await get_current_vessels_by_mmsi(
+                    db, mmsis[offset:offset + VESSEL_STATE_QUERY_BATCH_SIZE]
+                ))
+        present = {int(vessel["mmsi"]) for vessel in vessels}
+        vessels.extend({"mmsi": mmsi, "action": "remove"} for mmsi in mmsis if mmsi not in present)
+        return vessels
+
     async def broadcast_vessels_periodically(self):
         while self.running:
             try:
                 await self.flush_vessel_updates()
-            except Exception:
-                pass
+            except Exception as exc:
+                logger.exception_event(
+                    "Failed to flush vessel updates",
+                    event="vessels.broadcast.failed",
+                    context={"error": str(exc)},
+                )
             await asyncio.sleep(self._vessel_flush_interval)
 
     async def broadcast_datasource_task_update(self, data: Dict[str, Any]):

backend/app/core/websocket/manager.py

@@ -63,6 +63,8 @@ class ConnectionManager:
 
     def unsubscribe(self, websocket: WebSocket, channels: list[str]):
         for channel in {str(channel).strip() for channel in channels if str(channel).strip()}:
+            if channel == "vessels":
+                self.vessel_subscriptions.pop(websocket, None)
             subscribers = self.channel_subscriptions.get(channel)
             if subscribers is not None:
                 subscribers.discard(websocket)
@@ -88,7 +90,8 @@ class ConnectionManager:
         return subscription
 
     def _normalize_vessel_subscription(self, config: dict[str, Any]) -> dict[str, Any]:
-        bbox = config.get("bbox")
+        global_scope = config.get("scope") == "global"
+        bbox = [-180, -90, 180, 90] if global_scope else config.get("bbox")
         if not isinstance(bbox, (list, tuple)) or len(bbox) != 4:
             raise ValueError("vessels subscription requires bbox=[lon_min,lat_min,lon_max,lat_max]")
         try:
@@ -103,7 +106,7 @@ class ConnectionManager:
             raise ValueError("bbox longitude values must be between -180 and 180")
         if not (-90 <= lat_min <= 90 and -90 <= lat_max <= 90):
             raise ValueError("bbox latitude values must be between -90 and 90")
-        if (lon_max - lon_min) * (lat_max - lat_min) > MAX_VESSEL_BBOX_AREA:
+        if not global_scope and (lon_max - lon_min) * (lat_max - lat_min) > MAX_VESSEL_BBOX_AREA:
             raise ValueError("bbox is too large; zoom in or request a smaller viewport")
 
         zoom = int(config.get("zoom") or 1)
@@ -116,10 +119,11 @@ class ConnectionManager:
             if str(item).strip()
         }
         return {
+            "scope": "global" if global_scope else "viewport",
             "bbox": (lon_min, lat_min, lon_max, lat_max),
             "zoom": zoom,
             "limit": limit,
-            "type": vessel_types,
+            "type": sorted(vessel_types),
             "last_sent_at": None,
         }
 
@@ -152,7 +156,9 @@ class ConnectionManager:
                 vessel
                 for vessel in vessels
                 if self._vessel_matches_subscription(vessel, subscription)
-            ][: min(subscription["limit"], MAX_VESSEL_WS_MESSAGE_ITEMS)]
+            ]
+            if subscription.get("scope") != "global":
+                matched = matched[:subscription["limit"]]
             if not matched:
                 continue
             subscription["last_sent_at"] = datetime.now(UTC)
@@ -162,16 +168,24 @@ class ConnectionManager:
                 "timestamp": subscription["last_sent_at"].isoformat(),
                 "payload": {
                     **data,
-                    "vessels": matched,
+                    "vessels": [],
                     "subscription": {
                         "bbox": list(subscription["bbox"]),
                         "zoom": subscription["zoom"],
                         "limit": subscription["limit"],
+                        "scope": subscription.get("scope", "viewport"),
                     },
                 },
             }
             try:
-                await connection.send_json(message)
+                for offset in range(0, len(matched), MAX_VESSEL_WS_MESSAGE_ITEMS):
+                    await connection.send_json({
+                        **message,
+                        "payload": {
+                            **message["payload"],
+                            "vessels": matched[offset:offset + MAX_VESSEL_WS_MESSAGE_ITEMS],
+                        },
+                    })
             except Exception:
                 self.unsubscribe_all(connection)
 
@@ -180,6 +194,8 @@ class ConnectionManager:
         vessel: dict[str, Any],
         subscription: dict[str, Any],
     ) -> bool:
+        if vessel.get("action") == "remove" and subscription.get("scope") == "global":
+            return True
         try:
             lon = float(vessel.get("lon"))
             lat = float(vessel.get("lat"))

backend/app/db/bgp_prefix_geo.py

@@ -0,0 +1,51 @@
+"""SQL objects supporting indexed BGP prefix geography lookup."""
+
+from sqlalchemy import text
+from sqlalchemy.ext.asyncio import AsyncConnection
+
+BGP_PREFIX_GEO_FUNCTION_SQL = (
+    """
+    CREATE OR REPLACE FUNCTION planet_bgp_try_inet(value text)
+    RETURNS inet
+    LANGUAGE plpgsql IMMUTABLE STRICT PARALLEL SAFE
+    AS $$
+    BEGIN
+        RETURN value::inet;
+    EXCEPTION WHEN data_exception THEN
+        RETURN NULL;
+    END;
+    $$
+    """,
+    """
+    CREATE OR REPLACE FUNCTION planet_bgp_try_masklen(value text)
+    RETURNS integer
+    LANGUAGE plpgsql IMMUTABLE STRICT PARALLEL SAFE
+    AS $$
+    BEGIN
+        RETURN masklen(value::cidr);
+    EXCEPTION WHEN data_exception THEN
+        RETURN NULL;
+    END;
+    $$
+    """,
+)
+
+BGP_PREFIX_GEO_INDEX_NAME = "idx_bgp_prefix_geo_range_current"
+BGP_PREFIX_GEO_INDEX_SQL = f"""
+CREATE INDEX CONCURRENTLY IF NOT EXISTS {BGP_PREFIX_GEO_INDEX_NAME}
+ON collected_data USING gist (
+    planet_bgp_try_inet(metadata->>'range_start') inet_ops,
+    planet_bgp_try_inet(metadata->>'range_end') inet_ops
+)
+WHERE source IN (
+    'opengeofeed_prefix_geo',
+    'iptoasn_prefix_geo',
+    'nro_delegated_prefix_geo'
+)
+  AND COALESCE(is_current, TRUE) = TRUE
+"""
+
+
+async def install_bgp_prefix_geo_functions(connection: AsyncConnection) -> None:
+    for statement in BGP_PREFIX_GEO_FUNCTION_SQL:
+        await connection.execute(text(statement))

backend/app/db/session.py

@@ -6,6 +6,7 @@ from sqlalchemy.orm import declarative_base
 
 from app.core.config import settings
 from app.core.logging import get_logger
+from app.db.bgp_prefix_geo import install_bgp_prefix_geo_functions
 
 logger = get_logger(__name__)
 
@@ -219,6 +220,7 @@ async def init_db():
 
     async with engine.begin() as conn:
         await conn.run_sync(Base.metadata.create_all)
+        await install_bgp_prefix_geo_functions(conn)
         users_email_verified_existed = (
             await conn.execute(
                 text(
@@ -299,8 +301,12 @@ async def init_db():
                     change_event_id BIGINT;
                     change_payload JSONB;
                 BEGIN
+                    change_event_id := nextval(
+                        pg_get_serial_sequence('earth_data_change_events', 'id')::regclass
+                    );
                     change_payload := jsonb_build_object(
                         'event', 'earth.layer.changed',
+                        'event_id', change_event_id,
                         'table', change_table,
                         'operation', change_operation,
                         'source', change_source,
@@ -311,6 +317,7 @@ async def init_db():
                     );
 
                     INSERT INTO earth_data_change_events (
+                        id,
                         table_name,
                         operation,
                         source,
@@ -318,23 +325,15 @@ async def init_db():
                         payload,
                         occurred_at
                     ) VALUES (
+                        change_event_id,
                         change_table,
                         change_operation,
                         change_source,
                         NULL,
                         change_payload,
                         NOW()
-                    )
-                    RETURNING id INTO change_event_id;
-
-                    change_payload := change_payload || jsonb_build_object(
-                        'event_id', change_event_id
                     );
 
-                    UPDATE earth_data_change_events
-                    SET payload = change_payload
-                    WHERE id = change_event_id;
-
                     PERFORM pg_notify(
                         'planet_earth_data_changes',
                         change_payload::text
@@ -440,6 +439,16 @@ async def init_db():
                             );
                         END LOOP;
                     ELSIF TG_OP = 'UPDATE' THEN
+                        IF TG_TABLE_NAME = 'earth_news_items' AND NOT EXISTS (
+                            SELECT 1
+                            FROM new_rows AS current_row
+                            FULL JOIN old_rows AS previous_row USING (id)
+                            WHERE (to_jsonb(current_row) - 'last_seen_at' - 'updated_at')
+                                IS DISTINCT FROM
+                                  (to_jsonb(previous_row) - 'last_seen_at' - 'updated_at')
+                        ) THEN
+                            RETURN NULL;
+                        END IF;
                         FOR change_source IN
                             SELECT DISTINCT COALESCE(NULLIF(row_data->>'source', ''), TG_TABLE_NAME)
                             FROM (
@@ -626,6 +635,7 @@ async def init_db():
             "bgp_collector_locations",
             "vessel_static",
             "vessel_position",
+            "vessel_current_state",
             "ais_raw_observations",
             "ais_source_health",
             "compute_center_locations",
@@ -786,6 +796,22 @@ async def init_db():
                 """
             )
         )
+        await conn.execute(
+            text(
+                """
+                CREATE INDEX IF NOT EXISTS idx_vessel_current_bbox
+                ON vessel_current_state (lon, lat)
+                """
+            )
+        )
+        await conn.execute(
+            text(
+                """
+                CREATE INDEX IF NOT EXISTS idx_vessel_current_observed
+                ON vessel_current_state (observed_at DESC)
+                """
+            )
+        )
         await conn.execute(
             text(
                 """

backend/app/models/__init__.py

@@ -13,7 +13,7 @@ from app.models.compute_center_location import ComputeCenterLocationRecord
 from app.models.system_setting import SystemSetting
 from app.models.playground_session import PlaygroundSession
 from app.models.playground_message import PlaygroundMessage
-from app.models.system_log import SystemLog, AuditLog
+from app.models.system_log import AuditLog, ObservabilityEvent, ObservabilityEventGroup, SystemLog
 from app.models.vessel import AISConflictRecord, AISRawObservation, AISSourceHealth, VesselPosition, VesselStatic
 from app.models.datasource_mapping import DataSourceMappingTemplate
 from app.models.earth_news import EarthNewsItem
@@ -37,6 +37,8 @@ __all__ = [
     "ComputeCenterLocationRecord",
     "SystemLog",
     "AuditLog",
+    "ObservabilityEvent",
+    "ObservabilityEventGroup",
     "PlaygroundSession",
     "PlaygroundMessage",
     "VesselPosition",

backend/app/models/alert.py

@@ -1,26 +1,12 @@
 from datetime import datetime
-from enum import Enum
-from typing import Optional
 
-from sqlalchemy import Column, Integer, String, DateTime, Text, ForeignKey, Enum as SQLEnum
-from sqlalchemy.orm import relationship
+from sqlalchemy import Column, Integer, String, DateTime, Text, Enum as SQLEnum
 
+from app.core.enums import AlertSeverity, AlertStatus
 from app.core.time import to_iso8601_utc
 from app.db.session import Base
 
 
-class AlertSeverity(str, Enum):
-    CRITICAL = "critical"
-    WARNING = "warning"
-    INFO = "info"
-
-
-class AlertStatus(str, Enum):
-    ACTIVE = "active"
-    ACKNOWLEDGED = "acknowledged"
-    RESOLVED = "resolved"
-
-
 class Alert(Base):
     __tablename__ = "alerts"
 

backend/app/models/bgp_anomaly.py

@@ -4,6 +4,7 @@ from datetime import datetime
 
 from sqlalchemy import Column, DateTime, Float, ForeignKey, Index, Integer, JSON, String, Text
 
+from app.core.enums import BGPStatus
 from app.core.time import to_iso8601_utc
 from app.db.session import Base
 
@@ -17,7 +18,7 @@ class BGPAnomaly(Base):
     source = Column(String(100), nullable=False, index=True)
     anomaly_type = Column(String(50), nullable=False, index=True)
     severity = Column(String(20), nullable=False, index=True)
-    status = Column(String(20), nullable=False, default="active", index=True)
+    status = Column(String(20), nullable=False, default=BGPStatus.ACTIVE.value, index=True)
     entity_key = Column(String(255), nullable=False, index=True)
     prefix = Column(String(64), nullable=True, index=True)
     origin_asn = Column(Integer, nullable=True, index=True)

backend/app/models/bgp_incident.py

@@ -4,6 +4,7 @@ from datetime import datetime
 
 from sqlalchemy import Column, DateTime, Float, ForeignKey, Index, Integer, JSON, String, Text
 
+from app.core.enums import BGPStatus
 from app.core.time import to_iso8601_utc
 from app.db.session import Base
 
@@ -20,7 +21,7 @@ class BGPIncident(Base):
     title = Column(String(255), nullable=False)
     summary = Column(Text, nullable=False)
     severity = Column(String(20), nullable=False, index=True)
-    status = Column(String(20), nullable=False, default="active", index=True)
+    status = Column(String(20), nullable=False, default=BGPStatus.ACTIVE.value, index=True)
     confidence = Column(Float, nullable=False, default=0.5)
     started_at = Column(DateTime(timezone=True), nullable=False, default=datetime.utcnow, index=True)
     ended_at = Column(DateTime(timezone=True), nullable=True)

backend/app/models/data_snapshot.py

@@ -1,6 +1,7 @@
 from sqlalchemy import Boolean, Column, DateTime, ForeignKey, Integer, JSON, String
 from sqlalchemy.sql import func
 
+from app.core.enums import SnapshotStatus
 from app.db.session import Base
 
 
@@ -16,7 +17,7 @@ class DataSnapshot(Base):
     started_at = Column(DateTime(timezone=True), server_default=func.now())
     completed_at = Column(DateTime(timezone=True), nullable=True)
     record_count = Column(Integer, default=0)
-    status = Column(String(20), nullable=False, default="running")
+    status = Column(String(20), nullable=False, default=SnapshotStatus.RUNNING.value)
     is_current = Column(Boolean, default=True, index=True)
     parent_snapshot_id = Column(Integer, ForeignKey("data_snapshots.id"), nullable=True, index=True)
     summary = Column(JSON, default={})

backend/app/models/datasource_mapping.py

@@ -3,6 +3,7 @@
 from sqlalchemy import Boolean, Column, DateTime, ForeignKey, Integer, JSON, String
 from sqlalchemy.sql import func
 
+from app.core.enums import MappingValidationStatus
 from app.db.session import Base
 
 
@@ -19,7 +20,7 @@ class DataSourceMappingTemplate(Base):
     target_schema = Column(String(80), nullable=False, index=True)
     mapping_json = Column(JSON, nullable=False, default={})
     sample_payload_hash = Column(String(64), nullable=True)
-    validation_status = Column(String(30), nullable=False, default="draft")
+    validation_status = Column(String(30), nullable=False, default=MappingValidationStatus.DRAFT.value)
     version = Column(Integer, nullable=False, default=1)
     is_active = Column(Boolean, nullable=False, default=False, index=True)
     created_at = Column(DateTime(timezone=True), server_default=func.now())

backend/app/models/playground_message.py

@@ -1,6 +1,7 @@
 from sqlalchemy import JSON, Boolean, Column, DateTime, ForeignKey, Integer, String, Text
 from sqlalchemy.sql import func
 
+from app.core.enums import PlaygroundMessageKind, PlaygroundMessageStatus
 from app.db.session import Base
 
 
@@ -13,8 +14,8 @@ class PlaygroundMessage(Base):
     user_id = Column(Integer, ForeignKey("users.id", ondelete="CASCADE"), nullable=False, index=True)
     parent_message_id = Column(Integer, ForeignKey("playground_messages.id", ondelete="SET NULL"), nullable=True)
     role = Column(String(20), nullable=False)
-    kind = Column(String(20), nullable=False, default="message")
-    status = Column(String(20), nullable=False, default="done")
+    kind = Column(String(20), nullable=False, default=PlaygroundMessageKind.MESSAGE.value)
+    status = Column(String(20), nullable=False, default=PlaygroundMessageStatus.DONE.value)
     title = Column(String(255), nullable=True)
     content = Column(Text, nullable=False, default="")
     thinking_content = Column(Text, nullable=False, default="")

backend/app/models/system_log.py

@@ -38,3 +38,46 @@ class AuditLog(Base):
     ip = Column(String(64), nullable=True)
     details = Column(JSON, nullable=False, default=dict)
     created_at = Column(DateTime(timezone=True), server_default=func.now())
+
+
+class ObservabilityEvent(Base):
+    __tablename__ = "observability_events"
+
+    id = Column(Integer, primary_key=True, autoincrement=True)
+    occurred_at = Column(DateTime(timezone=True), server_default=func.now(), index=True)
+    source = Column(String(50), nullable=False, index=True)
+    service = Column(String(50), nullable=True, index=True)
+    module = Column(String(120), nullable=True, index=True)
+    category = Column(String(80), nullable=True, index=True)
+    event = Column(String(160), nullable=True, index=True)
+    level = Column(String(20), nullable=False, index=True)
+    message = Column(Text, nullable=False)
+    fingerprint = Column(String(80), nullable=False, index=True)
+    request_id = Column(String(64), nullable=True, index=True)
+    trace_id = Column(String(64), nullable=True, index=True)
+    task_id = Column(String(120), nullable=True, index=True)
+    source_ref_id = Column(String(120), nullable=True, index=True)
+    provider = Column(String(120), nullable=True, index=True)
+    user_id = Column(Integer, nullable=True, index=True)
+    context = Column(JSON, nullable=False, default=dict)
+    occurrence_count = Column(Integer, nullable=False, default=1)
+    created_at = Column(DateTime(timezone=True), server_default=func.now())
+
+
+class ObservabilityEventGroup(Base):
+    __tablename__ = "observability_event_groups"
+
+    fingerprint = Column(String(80), primary_key=True)
+    source = Column(String(50), nullable=False, index=True)
+    service = Column(String(50), nullable=True, index=True)
+    module = Column(String(120), nullable=True, index=True)
+    category = Column(String(80), nullable=True, index=True)
+    event = Column(String(160), nullable=True, index=True)
+    last_level = Column(String(20), nullable=False, index=True)
+    sample_message = Column(Text, nullable=False)
+    sample_detail = Column(Text, nullable=True)
+    affected_sources = Column(JSON, nullable=False, default=list)
+    count = Column(Integer, nullable=False, default=0)
+    first_seen_at = Column(DateTime(timezone=True), nullable=False, index=True)
+    last_seen_at = Column(DateTime(timezone=True), nullable=False, index=True)
+    updated_at = Column(DateTime(timezone=True), server_default=func.now(), onupdate=func.now())

backend/app/models/task.py

@@ -3,6 +3,7 @@
 from sqlalchemy import BigInteger, Column, DateTime, Float, Integer, JSON, String, Text
 from sqlalchemy.sql import func
 
+from app.core.enums import JobStatus, JobType, RollbackPolicy
 from app.db.session import Base
 
 
@@ -12,9 +13,9 @@ class CollectionTask(Base):
     id = Column(Integer, primary_key=True, autoincrement=True)
     datasource_id = Column(Integer, nullable=False, index=True)
     source = Column(String(100), nullable=True, index=True)
-    task_type = Column(String(30), nullable=False, default="collect", index=True)
+    task_type = Column(String(30), nullable=False, default=JobType.COLLECT.value, index=True)
     status = Column(String(20), nullable=False)  # queued, running, cancelling, success, failed, cancelled
-    phase = Column(String(30), default="queued")
+    phase = Column(String(30), default=JobStatus.QUEUED.value)
     phase_progress = Column(Float)
     phase_message = Column(String(255))
     phase_current = Column(BigInteger)
@@ -27,7 +28,7 @@ class CollectionTask(Base):
     progress = Column(Float, default=0.0)  # Progress percentage (0-100)
     error_message = Column(Text)
     payload = Column(JSON, default=dict)
-    rollback_policy = Column(String(40), nullable=False, default="keep_committed_batches")
+    rollback_policy = Column(String(40), nullable=False, default=RollbackPolicy.KEEP_COMMITTED_BATCHES.value)
     dedupe_key = Column(String(180), nullable=True, index=True)
     worker_id = Column(String(120), nullable=True, index=True)
     locked_at = Column(DateTime(timezone=True), nullable=True, index=True)

backend/app/models/user.py

@@ -1,6 +1,7 @@
 from sqlalchemy import Boolean, Column, DateTime, Integer, JSON, String
 from sqlalchemy.sql import func
 
+from app.core.enums import UserRole
 from app.db.session import Base
 
 
@@ -11,7 +12,7 @@ class User(Base):
     username = Column(String(50), unique=True, index=True, nullable=False)
     email = Column(String(255), unique=True, index=True, nullable=False)
     password_hash = Column(String(255), nullable=False)
-    role = Column(String(20), default="viewer")
+    role = Column(String(20), default=UserRole.VIEWER.value)
     gatekeeper_groups = Column(JSON, default=list)
     is_active = Column(Boolean, default=True)
     email_verified = Column(Boolean, default=False, nullable=False)

backend/app/models/vessel.py

@@ -3,6 +3,7 @@
 from sqlalchemy import BigInteger, Column, DateTime, Float, Index, Integer, JSON, SmallInteger, String
 from sqlalchemy.sql import func
 
+from app.core.enums import ConnectionState
 from app.core.time import to_iso8601_utc
 from app.db.session import Base
 
@@ -75,6 +76,67 @@ class VesselPosition(Base):
         }
 
 
+class VesselCurrentState(Base):
+    """Latest renderable state for one vessel, independent from AIS history."""
+
+    __tablename__ = "vessel_current_state"
+
+    mmsi = Column(BigInteger, primary_key=True)
+    lat = Column(Float, nullable=False)
+    lon = Column(Float, nullable=False)
+    sog = Column(Float, nullable=True)
+    cog = Column(Float, nullable=True)
+    heading = Column(SmallInteger, nullable=True)
+    nav_status = Column(SmallInteger, nullable=True, index=True)
+    name = Column(String(128), nullable=True)
+    callsign = Column(String(16), nullable=True)
+    vessel_type = Column(SmallInteger, nullable=True, index=True)
+    vessel_type_name = Column(String(64), nullable=True, index=True)
+    flag = Column(String(4), nullable=True, index=True)
+    length = Column(Float, nullable=True)
+    width = Column(Float, nullable=True)
+    draught = Column(Float, nullable=True)
+    imo = Column(BigInteger, nullable=True)
+    source = Column(String(100), nullable=False, index=True)
+    observed_at = Column(DateTime(timezone=True), nullable=False)
+    field_sources = Column(JSON, default=dict)
+    selected_reasons = Column(JSON, default=dict)
+    source_summary = Column(JSON, default=dict)
+    quality_flags = Column(JSON, default=list)
+    updated_at = Column(DateTime(timezone=True), nullable=False, server_default=func.now())
+
+    __table_args__ = (
+        Index("idx_vessel_current_bbox", "lon", "lat"),
+        Index("idx_vessel_current_observed", "observed_at"),
+    )
+
+    def to_dict(self) -> dict:
+        return {
+            "mmsi": self.mmsi,
+            "lat": self.lat,
+            "lon": self.lon,
+            "sog": self.sog,
+            "cog": self.cog,
+            "heading": self.heading,
+            "nav_status": self.nav_status,
+            "name": self.name,
+            "callsign": self.callsign,
+            "vessel_type": self.vessel_type,
+            "vessel_type_name": self.vessel_type_name,
+            "flag": self.flag,
+            "length": self.length,
+            "width": self.width,
+            "draught": self.draught,
+            "imo": self.imo,
+            "source": self.source,
+            "received_at": self.observed_at,
+            "field_sources": self.field_sources or {},
+            "selected_reasons": self.selected_reasons or {},
+            "source_summary": self.source_summary or {},
+            "quality_flags": self.quality_flags or [],
+        }
+
+
 class AISRawObservation(Base):
     """Source-level AIS fact before aggregation and conflict resolution."""
 
@@ -165,7 +227,7 @@ class AISSourceHealth(Base):
     __tablename__ = "ais_source_health"
 
     source = Column(String(100), primary_key=True)
-    connection_state = Column(String(32), nullable=False, default="disconnected", index=True)
+    connection_state = Column(String(32), nullable=False, default=ConnectionState.DISCONNECTED.value, index=True)
     last_seen_at = Column(DateTime(timezone=True), nullable=True, index=True)
     last_success_at = Column(DateTime(timezone=True), nullable=True, index=True)
     last_error = Column(String(500), nullable=True)

backend/app/schemas/ai.py

@@ -2,6 +2,7 @@ from typing import Any
 
 from pydantic import BaseModel, Field
 
+from app.core.enums import PlaygroundMessageKind, PlaygroundMessageRole, PlaygroundMessageStatus
 
 class AIContentBlock(BaseModel):
     type: str
@@ -110,9 +111,9 @@ class PlaygroundSessionUpsertRequest(BaseModel):
 
 class PlaygroundMessageRecord(BaseModel):
     id: str
-    role: str
-    kind: str = "message"
-    status: str = "done"
+    role: PlaygroundMessageRole
+    kind: PlaygroundMessageKind = PlaygroundMessageKind.MESSAGE
+    status: PlaygroundMessageStatus = PlaygroundMessageStatus.DONE
     title: str | None = None
     content: str = ""
     thinking_content: str = ""

backend/app/schemas/card_images.py

@@ -0,0 +1,128 @@
+"""Typed, validated configuration for optional object-card images."""
+
+from typing import Literal, Self
+from urllib.parse import urlsplit
+
+from pydantic import BaseModel, ConfigDict, Field, field_validator, model_validator
+
+EntityType = Literal["satellite", "compute_center", "cable", "landing_point", "vessel", "bgp", "bgp_collector", "earth_interactable"]
+MatchScope = Literal["object", "model", "group", "category"]
+
+
+def safe_url(value: str, *, local: bool = False) -> str:
+    if not value:
+        return value
+    if any(ord(char) < 32 for char in value) or "\\" in value:
+        raise ValueError("Invalid URL")
+    parts = urlsplit(value)
+    if local and value.startswith(("/earth/assets/cards/", "/api/v1/earth/card-images/files/")):
+        if (
+            ".." not in parts.path
+            and "%" not in parts.path
+            and not parts.netloc
+            and not parts.query
+            and not parts.fragment
+        ):
+            return value
+    if (
+        parts.scheme in {"http", "https"}
+        and parts.hostname
+        and not parts.username
+        and not parts.password
+    ):
+        return value
+    raise ValueError("Use an HTTP(S) URL without credentials or a managed image path")
+
+
+class CardImageAsset(BaseModel):
+    model_config = ConfigDict(extra="forbid", str_strip_whitespace=True)
+    id: str = Field(min_length=1, max_length=100, pattern=r"^[a-zA-Z0-9_-]+$")
+    title: str = Field(min_length=1, max_length=200)
+    url: str = Field(min_length=1, max_length=2000)
+    variants: dict[str, str] = Field(default_factory=dict, max_length=4)
+    width: int = Field(default=0, ge=0, le=20000)
+    height: int = Field(default=0, ge=0, le=20000)
+    credit: str = Field(default="", max_length=500)
+    source_url: str = Field(default="", max_length=2000)
+    license: str = Field(default="", max_length=300)
+    license_url: str = Field(default="", max_length=2000)
+    kind: Literal["photo", "model_photo", "render", "animation_still"] = "photo"
+    rights_status: Literal["available", "permission_pending"] = "available"
+    enabled: bool = True
+    focus_x: float = Field(default=50, ge=0, le=100, allow_inf_nan=False)
+    focus_y: float = Field(default=50, ge=0, le=100, allow_inf_nan=False)
+
+    @field_validator("url")
+    @classmethod
+    def validate_image_url(cls, value: str) -> str:
+        return safe_url(value, local=True)
+
+    @field_validator("source_url", "license_url")
+    @classmethod
+    def validate_source_url(cls, value: str) -> str:
+        return safe_url(value)
+
+    @field_validator("variants")
+    @classmethod
+    def validate_variants(cls, value: dict[str, str]) -> dict[str, str]:
+        for width, url in value.items():
+            if width not in {"640", "1280"}:
+                raise ValueError("Unsupported image variant")
+            safe_url(url, local=True)
+        return value
+
+
+class CardImageRule(BaseModel):
+    model_config = ConfigDict(extra="forbid", str_strip_whitespace=True)
+    id: str = Field(min_length=1, max_length=100, pattern=r"^[a-zA-Z0-9_-]+$")
+    entity_type: EntityType
+    scope: MatchScope
+    value: str = Field(min_length=1, max_length=300)
+    label: str = Field(default="", max_length=500)
+    asset_id: str | None = Field(default=None, max_length=100)
+    suppress: bool = False
+    enabled: bool = True
+
+    @model_validator(mode="after")
+    def validate_action(self) -> Self:
+        if self.scope != "object":
+            self.value = self.value.casefold()
+        if self.suppress and (self.scope != "object" or self.asset_id):
+            raise ValueError("Only an object rule may suppress its image, without an asset")
+        if not self.suppress and not self.asset_id:
+            raise ValueError("Choose an image or suppress this object")
+        return self
+
+    def target(self) -> tuple[str, str, str]:
+        return self.entity_type, self.scope, self.value
+
+
+class CardImageConfiguration(BaseModel):
+    model_config = ConfigDict(extra="forbid")
+    assets: list[CardImageAsset] = Field(default_factory=list, max_length=1000)
+    rules: list[CardImageRule] = Field(default_factory=list, max_length=3000)
+
+    @model_validator(mode="after")
+    def validate_relations(self) -> Self:
+        assets = {asset.id for asset in self.assets}
+        if len(assets) != len(self.assets) or len({rule.id for rule in self.rules}) != len(
+            self.rules
+        ):
+            raise ValueError("Image and rule IDs must be unique")
+        targets = [rule.target() for rule in self.rules]
+        if len(set(targets)) != len(targets):
+            raise ValueError("Each target may have only one rule")
+        if any(rule.asset_id and rule.asset_id not in assets for rule in self.rules):
+            raise ValueError("Remove image associations before deleting the image")
+        return self
+
+
+class CardImagePublish(BaseModel):
+    revision: str = Field(max_length=200)
+    config: CardImageConfiguration
+
+
+class CardImagePreview(BaseModel):
+    entity_type: EntityType
+    entity_id: str = Field(min_length=1, max_length=300)
+    config: CardImageConfiguration

backend/app/schemas/compute_configuration.py

@@ -0,0 +1,94 @@
+"""Validated, source-backed observations for the compute comparison cards."""
+
+from datetime import date
+from typing import Annotated, Literal, Self
+
+from app.services.compute_catalog import COUNTRY_PARTICIPANTS
+from pydantic import BaseModel, ConfigDict, Field, HttpUrl, field_validator, model_validator
+
+Year = Annotated[int, Field(strict=True, ge=2000, le=2100)]
+Value = Annotated[float, Field(strict=True, allow_inf_nan=False, ge=0, le=1e12)]
+Identifier = Annotated[str, Field(min_length=1, max_length=80, pattern=r"^[a-zA-Z0-9_-]+$")]
+Text = Annotated[str, Field(min_length=1, max_length=500)]
+
+
+class ConfigurationModel(BaseModel):
+    model_config = ConfigDict(extra="forbid", str_strip_whitespace=True)
+
+
+class ComputeSource(ConfigurationModel):
+    id: Identifier
+    title: Text
+    url: HttpUrl
+    publisher: Text
+    report_year: Year
+    published_at: date | None = None
+    locator: Text
+
+    @field_validator("url")
+    @classmethod
+    def public_reference(cls, value: HttpUrl) -> HttpUrl:
+        if value.username or value.password or len(str(value)) > 2000:
+            raise ValueError("Use a report URL without credentials (maximum 2000 characters)")
+        return value
+
+
+class ComputeObservation(ConfigurationModel):
+    year: Year
+    metric: Literal["global_share", "absolute"]
+    source_id: Identifier
+    values: dict[str, Value] = Field(min_length=1, max_length=250)
+
+    @model_validator(mode="after")
+    def comparable_values(self) -> Self:
+        if any(code not in COUNTRY_PARTICIPANTS for code in self.values):
+            raise ValueError(
+                "Use countries from the canonical compute catalog, not regional totals"
+            )
+        if self.metric == "global_share" and sum(self.values.values()) > 100.000001:
+            raise ValueError("Global shares for one year must total at most 100 percent")
+        return self
+
+
+class InnovationObservation(ConfigurationModel):
+    year: Year
+    value: Annotated[float, Field(strict=True, allow_inf_nan=False, ge=-100, le=100)]
+    source_id: Identifier
+
+
+class ComputeConfiguration(ConfigurationModel):
+    reviewed_at: date
+    sources: list[ComputeSource] = Field(max_length=200)
+    observations: list[ComputeObservation] = Field(max_length=202)
+    innovation: list[InnovationObservation] = Field(max_length=101)
+
+    @model_validator(mode="after")
+    def consistent_evidence(self) -> Self:
+        ids = [source.id for source in self.sources]
+        if len(ids) != len(set(ids)):
+            raise ValueError("Report source IDs must be unique")
+        keys = [(row.metric, row.year) for row in self.observations]
+        if len(keys) != len(set(keys)):
+            raise ValueError("Each metric/year may have only one observation")
+        years = [row.year for row in self.innovation]
+        if len(years) != len(set(years)):
+            raise ValueError("Innovation observation years must be unique")
+        sources = {source.id: source for source in self.sources}
+        for row in [*self.observations, *self.innovation]:
+            source = sources.get(row.source_id)
+            if source is None:
+                raise ValueError(f"Missing report source: {row.source_id}")
+            if row.year > source.report_year:
+                raise ValueError("Observation year must not be after its report year")
+        if self.reviewed_at > date.today():
+            raise ValueError("Review date must not be in the future")
+        return self
+
+
+class ComputeConfigurationWrite(ConfigurationModel):
+    revision: str = Field(min_length=1, max_length=80)
+    config: ComputeConfiguration
+
+
+class ComputeConfigurationReset(ConfigurationModel):
+    revision: str = Field(min_length=1, max_length=80)

backend/app/schemas/user.py

@@ -3,6 +3,7 @@ from typing import Optional
 
 from pydantic import BaseModel, EmailStr, Field
 
+from app.core.enums import OtpPurpose, UserRole
 
 class UserBase(BaseModel):
     username: str
@@ -11,13 +12,13 @@ class UserBase(BaseModel):
 
 class UserCreate(UserBase):
     password: str = Field(..., min_length=8)
-    role: str = "viewer"
+    role: UserRole = UserRole.VIEWER
     gatekeeper_groups: list[str] = Field(default_factory=list)
 
 
 class UserUpdate(BaseModel):
     email: Optional[EmailStr] = None
-    role: Optional[str] = None
+    role: Optional[UserRole] = None
     gatekeeper_groups: Optional[list[str]] = None
     is_active: Optional[bool] = None
 
@@ -35,6 +36,9 @@ class UserInDB(UserBase):
 
 
 class UserResponse(UserBase):
+    # Bootstrap accounts have reserved .local email addresses. Validate email
+    # on writes, while allowing existing local identities to be serialized.
+    email: str
     id: int
     role: str
     gatekeeper_groups: list[str] = Field(default_factory=list)
@@ -59,7 +63,7 @@ class VerifyEmailRequest(BaseModel):
 
 class ResendCodeRequest(BaseModel):
     email: EmailStr
-    purpose: str = Field(default="register", pattern="^(register|verify_email|reset_password)$")
+    purpose: OtpPurpose = OtpPurpose.REGISTER
 
 
 class ForgotPasswordRequest(BaseModel):

backend/app/services/bgp_detectors.py

@@ -6,6 +6,7 @@ from collections import Counter, defaultdict
 from datetime import UTC, datetime
 from typing import Any
 
+from app.core.enums import BGPStatus
 from app.models.bgp_anomaly import BGPAnomaly
 
 
@@ -127,7 +128,7 @@ def detect_origin_change_anomalies(
                     source=source,
                     anomaly_type=anomaly_type,
                     severity=severity,
-                    status="active",
+                    status=BGPStatus.ACTIVE.value,
                     entity_key=f"{anomaly_type}:{prefix}:{new_origin}",
                     prefix=prefix,
                     origin_asn=sorted(historic)[0] if historic else None,
@@ -197,7 +198,7 @@ def detect_more_specific_burst_anomalies(
                 source=source,
                 anomaly_type="more_specific_burst",
                 severity="high",
-                status="active",
+                status=BGPStatus.ACTIVE.value,
                 entity_key=f"more_specific_burst:{root_prefix}:{len(unique_prefixes)}:{len(related_collectors)}",
                 prefix=sample.get("prefix"),
                 origin_asn=sample.get("origin_asn"),
@@ -267,7 +268,7 @@ def detect_mass_withdrawal_anomalies(
                 source=source,
                 anomaly_type="mass_withdrawal",
                 severity=severity,
-                status="active",
+                status=BGPStatus.ACTIVE.value,
                 entity_key=f"mass_withdrawal:{prefix}:{origin_asn}:{len(related_collectors)}:{count}",
                 prefix=prefix,
                 origin_asn=origin_asn,
@@ -354,7 +355,7 @@ def detect_route_leak_anomalies(
                 source=source,
                 anomaly_type="route_leak_candidate",
                 severity="high" if max_path_length >= dominant_length + 3 else "medium",
-                status="active",
+                status=BGPStatus.ACTIVE.value,
                 entity_key=f"route_leak_candidate:{prefix}:{max_path_length}:{len(related_collectors)}",
                 prefix=prefix,
                 origin_asn=sample_metadata.get("origin_asn"),
@@ -435,7 +436,7 @@ def detect_path_flap_anomalies(
                 source=source,
                 anomaly_type="path_flap",
                 severity=severity,
-                status="active",
+                status=BGPStatus.ACTIVE.value,
                 entity_key=f"path_flap:{prefix}:{transitions}:{len(distinct_paths)}",
                 prefix=prefix,
                 origin_asn=sample_metadata.get("origin_asn"),

backend/app/services/bgp_enrichment.py

@@ -111,15 +111,24 @@ async def _lookup_prefix_geography(
         result = await db.execute(
             text(
                 """
+                WITH candidates AS MATERIALIZED (
+                    SELECT source, id, metadata
+                    FROM collected_data
+                    WHERE source IN (
+                        'opengeofeed_prefix_geo',
+                        'iptoasn_prefix_geo',
+                        'nro_delegated_prefix_geo'
+                    )
+                      AND COALESCE(is_current, TRUE) = TRUE
+                      AND planet_bgp_try_inet(metadata->>'range_start') <= CAST(:range_start AS inet)
+                      AND planet_bgp_try_inet(metadata->>'range_end') >= CAST(:range_end AS inet)
+                )
                 SELECT metadata
-                FROM collected_data
+                FROM candidates
                 WHERE source = :source
-                  AND COALESCE(is_current, TRUE) = TRUE
                   AND metadata->>'family' = :family
-                  AND CAST(metadata->>'range_start' AS inet) <= CAST(:range_start AS inet)
-                  AND CAST(metadata->>'range_end' AS inet) >= CAST(:range_end AS inet)
                 ORDER BY
-                  masklen(CAST(metadata->>'prefix' AS cidr)) DESC NULLS LAST,
+                  planet_bgp_try_masklen(metadata->>'prefix') DESC NULLS LAST,
                   id DESC
                 LIMIT 1
                 """
@@ -146,6 +155,10 @@ async def _lookup_prefix_geography(
 
     results: dict[str, dict[str, Any]] = {}
 
+    # The GiST range predicate is selective, but PostgreSQL 15 greatly overestimates its cost.
+    # Avoid spending more time compiling JIT code than fetching the small candidate set.
+    await db.execute(text("SET LOCAL jit = off"))
+
     for prefix in prefix_values:
         try:
             network = ipaddress.ip_network(prefix, strict=False)

backend/app/services/bgp_incidents.py

@@ -9,6 +9,7 @@ from sqlalchemy import select
 from sqlalchemy.ext.asyncio import AsyncSession
 
 from app.core.collected_data_fields import get_record_field
+from app.core.enums import BGPStatus
 from app.models.bgp_anomaly import BGPAnomaly
 from app.models.bgp_incident import BGPIncident
 from app.models.collected_data import CollectedData
@@ -290,7 +291,7 @@ async def create_bgp_incidents_for_anomalies(
             existing.title = title
             existing.summary = summary
             existing.severity = severity
-            existing.status = "active"
+            existing.status = BGPStatus.ACTIVE.value
             existing.confidence = confidence
             existing.started_at = primary.started_at or existing.started_at or datetime.now(UTC)
             existing.ended_at = None
@@ -313,7 +314,7 @@ async def create_bgp_incidents_for_anomalies(
                 title=title,
                 summary=summary,
                 severity=severity,
-                status="active",
+                status=BGPStatus.ACTIVE.value,
                 confidence=confidence,
                 started_at=primary.started_at or datetime.now(UTC),
                 affected_prefixes=prefixes,

backend/app/services/card_image_configuration.py

@@ -0,0 +1,174 @@
+"""Revision-checked image settings; new bundled assets never replace user edits."""
+
+from uuid import uuid4
+
+from fastapi import HTTPException
+from sqlalchemy import select
+from sqlalchemy.dialects.postgresql import insert
+from sqlalchemy.ext.asyncio import AsyncSession
+
+from app.core.request_context import get_request_id
+from app.models.system_log import AuditLog
+from app.models.system_setting import SystemSetting
+from app.models.user import User
+from app.schemas.card_images import CardImageConfiguration
+from app.services.card_image_defaults import (
+    bundled_configuration,
+    previous_bundled_assets,
+    previous_bundled_rules,
+)
+
+CATEGORY = "earth_card_images"
+
+
+def effective_envelope(payload: dict | None) -> dict:
+    defaults, version = bundled_configuration()
+    config = (
+        CardImageConfiguration.model_validate(payload["config"])
+        if payload
+        else defaults.model_copy(deep=True)
+    )
+    if payload:
+        previous = previous_bundled_assets()
+        latest = {asset.id: asset for asset in defaults.assets}
+        for index, asset in enumerate(config.assets):
+            old = next((item for item in previous.get(asset.id, []) if item.url == asset.url), None)
+            # Replace the old bundled source while preserving fields the user
+            # changed (crop, title, enabled state, etc.). Custom URLs are untouched.
+            if old is not None and asset.url == old.url and asset.id in latest:
+                overrides = {
+                    key: value
+                    for key, value in asset.model_dump().items()
+                    if value != getattr(old, key)
+                }
+                config.assets[index] = latest[asset.id].model_copy(update=overrides, deep=True)
+        known_assets = set(payload.get("known_assets", []))
+        asset_ids = {asset.id for asset in config.assets}
+        config.assets.extend(
+            asset.model_copy(deep=True)
+            for asset in defaults.assets
+            if asset.id not in known_assets and asset.id not in asset_ids
+        )
+        prior_rules = previous_bundled_rules()
+        latest_rules = {rule.id: rule for rule in defaults.rules}
+        for index, rule in enumerate(config.rules):
+            old_rule = prior_rules.get(rule.id)
+            if (
+                old_rule
+                and rule.id in latest_rules
+                and rule.target() == old_rule.target()
+                and rule.asset_id == old_rule.asset_id
+            ):
+                overrides = {
+                    key: value
+                    for key, value in rule.model_dump().items()
+                    if value != getattr(old_rule, key)
+                }
+                config.rules[index] = latest_rules[rule.id].model_copy(update=overrides, deep=True)
+        known_rules = set(payload.get("known_rules", []))
+        rule_ids = {rule.id for rule in config.rules}
+        targets = {rule.target() for rule in config.rules}
+        available = {asset.id for asset in config.assets}
+        config.rules.extend(
+            rule.model_copy(deep=True)
+            for rule in defaults.rules
+            if rule.id not in known_rules
+            and rule.id not in rule_ids
+            and rule.target() not in targets
+            and rule.asset_id in available
+        )
+    return {
+        "config": config.model_dump(mode="json"),
+        "revision": f"{version}:{payload['revision'] if payload else 'default'}",
+        "is_default": payload is None,
+    }
+
+
+async def read_configuration(db: AsyncSession) -> dict:
+    record = (
+        await db.execute(select(SystemSetting).where(SystemSetting.category == CATEGORY))
+    ).scalar_one_or_none()
+    return effective_envelope(record.payload if record else None)
+
+
+async def publish_configuration(
+    db: AsyncSession, revision: str, config: CardImageConfiguration, actor: User
+) -> dict:
+    defaults, _ = bundled_configuration()
+    pending = {asset.id for asset in defaults.assets if asset.rights_status == "permission_pending"}
+    if any(
+        asset.id in pending and asset.rights_status != "permission_pending"
+        for asset in config.assets
+    ):
+        raise HTTPException(422, "Bundled image permission is still unverified")
+    initial = {
+        "config": defaults.model_dump(mode="json"),
+        "revision": "default",
+        "known_assets": [],
+        "known_rules": [],
+    }
+    await db.execute(
+        insert(SystemSetting)
+        .values(category=CATEGORY, payload=initial)
+        .on_conflict_do_nothing(index_elements=[SystemSetting.category])
+    )
+    record = (
+        await db.execute(
+            select(SystemSetting).where(SystemSetting.category == CATEGORY).with_for_update()
+        )
+    ).scalar_one()
+    if effective_envelope(record.payload)["revision"] != revision:
+        await db.rollback()
+        raise HTTPException(409, "Configuration changed. Reload before saving your edits.")
+    before = record.payload
+    record.payload = {
+        "config": config.model_dump(mode="json"),
+        "revision": uuid4().hex,
+        "known_assets": [asset.id for asset in defaults.assets],
+        "known_rules": [rule.id for rule in defaults.rules],
+    }
+    db.add(
+        AuditLog(
+            actor_id=actor.id,
+            actor_name=actor.username,
+            action="card_images.update",
+            target_type="system_setting",
+            target_id=CATEGORY,
+            result="success",
+            request_id=get_request_id(),
+            details={"before": before, "after": record.payload},
+        )
+    )
+    await db.commit()
+    return effective_envelope(record.payload)
+
+
+def resolve_image(config: CardImageConfiguration, context: dict[str, str]) -> dict:
+    assets = {asset.id: asset for asset in config.assets}
+    for scope in ("object", "model", "group", "category"):
+        value = context.get("entity_id" if scope == "object" else scope)
+        values = [value] if value else []
+        if scope == "category" and value != "*":
+            values.append("*")
+        for value in values:
+            value = value if scope == "object" else value.casefold()
+            rule = next(
+                (
+                    rule
+                    for rule in config.rules
+                    if rule.enabled
+                    and rule.entity_type == context["entity_type"]
+                    and rule.scope == scope
+                    and rule.value == value
+                ),
+                None,
+            )
+            if not rule:
+                continue
+            match = {"scope": scope, "value": value, "rule_id": rule.id}
+            if rule.suppress:
+                return {"image": None, "match": match, "suppressed": True}
+            asset = assets.get(rule.asset_id)
+            if asset and asset.enabled:
+                return {"image": asset.model_dump(mode="json"), "match": match, "suppressed": False}
+    return {"image": None, "match": None, "suppressed": False}

backend/app/services/card_image_defaults.py

@@ -0,0 +1,108 @@
+"""Read immutable bundled assets without inserting machine-specific database rows."""
+
+import hashlib
+import json
+from functools import lru_cache
+from pathlib import Path
+
+from app.schemas.card_images import CardImageAsset, CardImageConfiguration, CardImageRule
+
+REPO_ROOT = Path(__file__).resolve().parents[3]
+MANIFEST_PATH = REPO_ROOT / "frontend/public/earth/assets/cards/manifest.json"
+
+
+def _asset_from_manifest(item: dict) -> CardImageAsset:
+    pending = item.get("redistribution_status") == "permission_not_verified"
+    return CardImageAsset(
+        id=item["id"],
+        title=item["label"],
+        url=item["url"],
+        width=item["width"],
+        height=item["height"],
+        credit=item["credit"],
+        source_url=item.get("source_page") or "",
+        license=item["license"],
+        license_url=item.get("license_url") or "",
+        kind=item["kind"],
+        rights_status="permission_pending" if pending else "available",
+        enabled=item.get("enabled", not pending),
+        focus_x=item.get("focus_x", 50),
+        focus_y=item.get("focus_y", 50),
+        variants=item.get("variants", {}),
+    )
+
+
+@lru_cache(maxsize=1)
+def previous_bundled_assets() -> dict[str, list[CardImageAsset]]:
+    manifest = json.loads(MANIFEST_PATH.read_bytes())
+    return {
+        key: [CardImageAsset.model_validate(item) for item in value]
+        for key, value in manifest.get("previous_defaults", {}).items()
+    }
+
+
+@lru_cache(maxsize=1)
+def previous_bundled_rules() -> dict[str, CardImageRule]:
+    manifest = json.loads(MANIFEST_PATH.read_bytes())
+    return {
+        key: CardImageRule.model_validate(value)
+        for key, value in manifest.get("previous_rules", {}).items()
+    }
+
+
+@lru_cache(maxsize=1)
+def bundled_configuration() -> tuple[CardImageConfiguration, str]:
+    content = MANIFEST_PATH.read_bytes()
+    manifest = json.loads(content)
+    assets, rules = [], []
+    for item in manifest["assets"]:
+        asset = _asset_from_manifest(item)
+        assets.append(asset)
+        if not asset.enabled:
+            continue
+        hints = item.get("binding_hints", {})
+        for index, target in enumerate(hints.get("defaults", [])):
+            rules.append(
+                CardImageRule(
+                    id=target.get("id", f"builtin-{asset.id}-default-{index}"),
+                    entity_type=target["entity_type"],
+                    scope=target["scope"],
+                    value=target["value"],
+                    label=asset.title,
+                    asset_id=asset.id,
+                )
+            )
+        for index, entity_id in enumerate(hints.get("entity_ids", [])):
+            rules.append(
+                CardImageRule(
+                    id=f"builtin-{asset.id}-entity-{index}",
+                    entity_type=item["entity_type"],
+                    scope="object",
+                    value=entity_id,
+                    label=asset.title,
+                    asset_id=asset.id,
+                )
+            )
+        for norad_id in hints.get("norad_ids", []):
+            rules.append(
+                CardImageRule(
+                    id=f"builtin-{asset.id}-{norad_id}",
+                    entity_type="satellite",
+                    scope="object",
+                    value=str(norad_id),
+                    label=f"{asset.title} · NORAD {norad_id}",
+                    asset_id=asset.id,
+                )
+            )
+        if group := hints.get("constellation_group"):
+            rules.append(
+                CardImageRule(
+                    id=f"builtin-{asset.id}-group",
+                    entity_type="satellite",
+                    scope="group",
+                    value=group,
+                    label=asset.title,
+                    asset_id=asset.id,
+                )
+            )
+    return CardImageConfiguration(assets=assets, rules=rules), hashlib.sha256(content).hexdigest()

backend/app/services/card_image_files.py

@@ -0,0 +1,95 @@
+"""Validated raster uploads and content-addressed, bounded card-size derivatives."""
+
+import hashlib
+import io
+import re
+import warnings
+from uuid import uuid4
+from pathlib import Path
+import os
+
+from fastapi import HTTPException
+from PIL import Image, ImageOps, UnidentifiedImageError
+
+from app.schemas.card_images import CardImageAsset
+from app.services.card_image_defaults import REPO_ROOT
+
+ASSET_DIR = Path(
+    os.environ.get("PLANET_CARD_IMAGE_DIR", str(REPO_ROOT / "data/earth-card-images"))
+).resolve()
+MAX_UPLOAD_BYTES = 12 * 1024 * 1024
+MAX_PIXELS = 32_000_000
+FILE_PATTERN = re.compile(r"^[a-f0-9]{64}-(?:original\.(?:jpg|png|webp)|(?:640|1280)\.webp)$")
+FORMATS = {"JPEG": "jpg", "PNG": "png", "WEBP": "webp"}
+
+
+def uploaded_path(filename: str) -> Path:
+    if not FILE_PATTERN.fullmatch(filename):
+        raise HTTPException(404, "Image not found")
+    return ASSET_DIR / filename
+
+
+def store_image(content: bytes, title: str) -> CardImageAsset:
+    if len(content) > MAX_UPLOAD_BYTES:
+        raise HTTPException(413, "Image must be 12 MiB or smaller")
+    try:
+        with warnings.catch_warnings():
+            warnings.simplefilter("error", Image.DecompressionBombWarning)
+            image = Image.open(io.BytesIO(content))
+            extension = FORMATS.get(image.format)
+            if (
+                not extension
+                or image.width * image.height > MAX_PIXELS
+                or getattr(image, "is_animated", False)
+            ):
+                raise ValueError("Unsupported or oversized raster")
+            image.load()
+            image = ImageOps.exif_transpose(image).convert(
+                "RGBA" if "A" in image.getbands() or "transparency" in image.info else "RGB"
+            )
+    except (
+        UnidentifiedImageError,
+        OSError,
+        ValueError,
+        Image.DecompressionBombError,
+        Image.DecompressionBombWarning,
+    ) as error:
+        raise HTTPException(
+            422, "Use a valid, still JPEG, PNG or WebP image below 32 megapixels"
+        ) from error
+    digest = hashlib.sha256(content).hexdigest()
+    ASSET_DIR.mkdir(parents=True, exist_ok=True)
+    original = f"{digest}-original.{extension}"
+    original_path = uploaded_path(original)
+    if not original_path.exists():
+        temporary = original_path.with_suffix(f".{uuid4().hex}.tmp")
+        temporary.write_bytes(content)
+        temporary.replace(original_path)
+    variants = {}
+    for size, content in render_variants(image).items():
+        filename = f"{digest}-{size}.webp"
+        path = uploaded_path(filename)
+        if not path.exists():
+            temporary = path.with_suffix(f".{uuid4().hex}.tmp")
+            temporary.write_bytes(content)
+            temporary.replace(path)
+        variants[size] = f"/api/v1/earth/card-images/files/{filename}"
+    return CardImageAsset(
+        id=f"upload-{digest[:24]}",
+        title=title or "Image",
+        url=f"/api/v1/earth/card-images/files/{original}",
+        variants=variants,
+        width=image.width,
+        height=image.height,
+    )
+
+
+def render_variants(image: Image.Image) -> dict[str, bytes]:
+    result = {}
+    for size in (640, 1280):
+        resized = image.copy()
+        resized.thumbnail((size, size), Image.Resampling.LANCZOS)
+        buffer = io.BytesIO()
+        resized.save(buffer, format="WEBP", quality=93, method=4)
+        result[str(size)] = buffer.getvalue()
+    return result

backend/app/services/card_image_targets.py

@@ -0,0 +1,180 @@
+"""Bounded catalog queries and type defaults for every Earth object card."""
+
+from sqlalchemy import String, case, cast, func, literal, or_, select
+from sqlalchemy.ext.asyncio import AsyncSession
+
+from app.core.compute_types import (
+    COMPUTE_SITE_TYPES,
+    compute_site_type_expression,
+    compute_record_filter,
+)
+from app.models.bgp_incident import BGPIncident
+from app.models.bgp_collector_location import BGPCollectorLocation
+from app.models.collected_data import CollectedData
+from app.models.earth_interactable import EarthInteractable
+from app.models.vessel import VesselCurrentState
+from app.schemas.card_images import EntityType, MatchScope
+
+SOURCES = {
+    "satellite": ("celestrak_tle", "spacetrack_tle"),
+    "compute_center": ("top500", "epoch_ai_gpu"),
+    "cable": ("arcgis_cables",),
+    "landing_point": ("arcgis_landing_points",),
+}
+# Named constellation families, not inferred hardware models or generations.
+SATELLITE_GROUP_PATTERNS = (
+    ("STARLINK%", "starlink"),
+    ("IRIDIUM%", "iridium-next"),
+    ("BEIDOU-3%", "beidou-3"),
+    ("BEIDOU%", "beidou"),
+    ("QIANFAN%", "qianfan"),
+    ("ONEWEB%", "oneweb"),
+    ("KUIPER%", "kuiper"),
+)
+
+
+def collected_query(entity_type: EntityType):
+    record, metadata = CollectedData, CollectedData.extra_data
+    model = func.coalesce(
+        metadata["satellite_model" if entity_type == "satellite" else "model"].as_string(), ""
+    )
+    category = (
+        compute_site_type_expression(record)
+        if entity_type == "compute_center"
+        else literal(entity_type)
+    )
+    if entity_type == "satellite":
+        group = func.coalesce(
+            func.nullif(func.lower(metadata["constellation_group"].as_string()), ""),
+            case(
+                *[
+                    (func.upper(record.name).like(pattern), value)
+                    for pattern, value in SATELLITE_GROUP_PATTERNS
+                ],
+                else_="",
+            ),
+        )
+        entity_id = metadata["norad_cat_id"].as_string()
+    else:
+        group = func.coalesce(
+            metadata["operator"].as_string(), metadata["organization"].as_string(), ""
+        )
+        entity_id = record.source + ":" + record.source_id
+    return select(
+        record.name.label("name"),
+        cast(entity_id, String).label("entity_id"),
+        model.label("model"),
+        group.label("group"),
+        category.label("category"),
+    ).where(
+        (
+            compute_record_filter(record)
+            if entity_type == "compute_center"
+            else record.source.in_(SOURCES[entity_type])
+        ),
+        record.is_current.is_(True),
+        record.deleted_at.is_(None),
+        metadata["anonymous"].as_boolean().is_not(True),
+    )
+
+
+def target_query(entity_type: EntityType):
+    if entity_type in SOURCES:
+        return collected_query(entity_type)
+    if entity_type == "vessel":
+        model = VesselCurrentState
+        identity, name, group, category = model.mmsi, model.name, model.flag, model.vessel_type_name
+        conditions = []
+    elif entity_type == "bgp":
+        model = BGPIncident
+        identity, name, group, category = (
+            model.incident_key,
+            model.title,
+            model.source,
+            model.incident_type,
+        )
+        conditions = []
+    elif entity_type == "bgp_collector":
+        model = BGPCollectorLocation
+        identity, name, group, category = (
+            model.collector_id,
+            model.collector_id,
+            model.operator,
+            literal("bgp_collector"),
+        )
+        conditions = []
+    else:
+        model = EarthInteractable
+        identity, name, group, category = model.id, model.label, model.layer, model.kind
+        conditions = [model.is_deleted.is_(False)]
+    return select(
+        name.label("name"),
+        cast(identity, String).label("entity_id"),
+        literal("").label("model"),
+        func.coalesce(group, "").label("group"),
+        func.coalesce(category, entity_type).label("category"),
+    ).where(*conditions)
+
+
+def category_context(entity_type: EntityType) -> dict[str, str]:
+    return {
+        "entity_type": entity_type,
+        "entity_id": "",
+        "name": "",
+        "model": "",
+        "group": "",
+        "category": entity_type,
+    }
+
+
+async def get_target(db: AsyncSession, entity_type: EntityType, entity_id: str) -> dict[str, str]:
+    if entity_type == "compute_center" and entity_id.startswith("__category__:"):
+        category = entity_id.partition(":")[2]
+        if category in COMPUTE_SITE_TYPES:
+            return {**category_context(entity_type), "category": category}
+    if entity_id == "__category__":
+        return category_context(entity_type)
+    if entity_type in SOURCES and entity_type != "satellite":
+        source, separator, source_id = entity_id.partition(":")
+        if not separator or not source or not source_id:
+            return category_context(entity_type)
+        # Match the indexed identity columns before projecting JSON metadata.
+        statement = collected_query(entity_type).where(
+            CollectedData.source == source, CollectedData.source_id == source_id
+        )
+    else:
+        catalog = target_query(entity_type).subquery()
+        statement = select(catalog).where(catalog.c.entity_id == entity_id)
+    row = (await db.execute(statement.limit(1))).mappings().first()
+    if not row:
+        # Old map snapshots can outlive a catalog refresh. They may use the type
+        # illustration, but must not claim an unverified exact-object identity.
+        return category_context(entity_type)
+    context = {key: str(value or "") for key, value in row.items()}
+    return {**context, "entity_type": entity_type}
+
+
+async def search_targets(
+    db: AsyncSession, entity_type: EntityType, scope: MatchScope, query: str, limit: int
+) -> list[dict[str, str]]:
+    catalog = target_query(entity_type).subquery()
+    value = catalog.c["entity_id" if scope == "object" else scope]
+    label = catalog.c.name if scope == "object" else value
+    statement = select(value.label("value"), label.label("label")).where(
+        value.is_not(None), value != ""
+    )
+    if query:
+        escaped = query.replace("\\", "\\\\").replace("%", "\\%").replace("_", "\\_")
+        statement = statement.where(
+            or_(value.ilike(f"%{escaped}%", escape="\\"), label.ilike(f"%{escaped}%", escape="\\"))
+        )
+    rows = (await db.execute(statement.distinct().order_by(label, value).limit(limit))).mappings()
+    items = [{"value": row["value"], "label": row["label"] or row["value"]} for row in rows]
+    if scope == "category" and not query:
+        known = {item["value"] for item in items}
+        if entity_type == "compute_center":
+            items.extend(
+                {"value": kind, "label": kind} for kind in COMPUTE_SITE_TYPES if kind not in known
+            )
+        items.insert(0, {"value": "*", "label": "*"})
+    return items

backend/app/services/collectors/aisstream.py

@@ -322,6 +322,21 @@ class AISStreamCollector(BaseCollector):
             last_success_at=now if data else None,
             lag_seconds=max((now - latest_observed_at).total_seconds(), 0),
         )
+        if snapshot_id is not None:
+            from app.models.data_snapshot import DataSnapshot
+
+            snapshot = await db.get(DataSnapshot, snapshot_id)
+            if snapshot:
+                snapshot.record_count = records_added
+                snapshot.status = "success"
+                snapshot.completed_at = now
+                snapshot.summary = {
+                    "created": records_added,
+                    "updated": 0,
+                    "unchanged": 0,
+                    "deleted": 0,
+                    "storage": "ais_raw_observations",
+                }
         await db.commit()
         await self.update_progress(records_added, force=True)
         return records_added

backend/app/services/collectors/base.py

@@ -12,11 +12,11 @@ from sqlalchemy.ext.asyncio import AsyncSession
 
 from app.core.collected_data_fields import build_dynamic_metadata, get_record_field
 from app.core.countries import normalize_country
+from app.core.enums import JobStatus, SnapshotStatus
 from app.core.logging import get_logger
 from app.core.time import to_iso8601_utc
 from app.core.websocket.broadcaster import broadcaster
 from app.services.business_logs import emit_business_log, exception_context
-from app.services.earth_layer_adapters import get_earth_update_layers_for_source
 
 
 logger = get_logger(__name__, service="collector")
@@ -31,6 +31,7 @@ class BaseCollector(ABC):
     frequency_hours: int = 4
     data_type: str = "generic"
     fail_on_empty: bool = False
+    preserve_snapshot_on_failure: bool = False
 
     def __init__(self):
         self._current_task = None
@@ -238,7 +239,7 @@ class BaseCollector(ABC):
             snapshot = await db.get(DataSnapshot, snapshot_id)
             if snapshot:
                 parent_snapshot_id = snapshot.parent_snapshot_id
-                snapshot.status = "cancelled"
+                snapshot.status = SnapshotStatus.CANCELLED.value
                 snapshot.is_current = False
                 snapshot.completed_at = datetime.now(UTC)
                 summary = dict(snapshot.summary or {})
@@ -308,7 +309,7 @@ class BaseCollector(ABC):
             task.datasource_id = datasource_id
             task.source = task.source or self.name
             task.task_type = task.task_type or "collect"
-            task.status = "running"
+            task.status = JobStatus.RUNNING.value
             task.phase = "queued"
             task.started_at = task.started_at or start_time
             task.completed_at = None
@@ -361,6 +362,8 @@ class BaseCollector(ABC):
                 context={"task_id": task_id, "raw_count": len(raw_data)},
             )
             data = self.transform(raw_data)
+            if self.fail_on_empty and not data:
+                raise RuntimeError(f"Collector {self.name} produced no valid records")
             await self._log_collection_event(
                 "collector.phase.transforming.success",
                 "Collector transform phase completed",
@@ -393,7 +396,7 @@ class BaseCollector(ABC):
                 },
             )
 
-            task.status = "success"
+            task.status = JobStatus.SUCCESS.value
             task.phase = "completed"
             task.phase_progress = 100.0
             task.phase_message = "采集完成"
@@ -428,7 +431,7 @@ class BaseCollector(ABC):
             }
         except asyncio.CancelledError:
             await db.rollback()
-            task.status = "cancelled"
+            task.status = JobStatus.CANCELLED.value
             task.phase = "cancelled"
             task.phase_message = "采集已取消"
             task.error_message = "Collection cancelled by operator and rolled back"
@@ -456,7 +459,11 @@ class BaseCollector(ABC):
             raise
         except Exception as e:
             await db.rollback()
-            task.status = "failed"
+            if snapshot_id is not None and self.preserve_snapshot_on_failure:
+                await self._rollback_incomplete_run(
+                    db, task_id=task_id, snapshot_id=snapshot_id, reason=str(e)
+                )
+            task.status = JobStatus.FAILED.value
             task.phase = "failed"
             task.phase_message = str(e)
             task.error_message = str(e)
@@ -464,7 +471,9 @@ class BaseCollector(ABC):
             if snapshot_id is not None:
                 snapshot = await db.get(DataSnapshot, snapshot_id)
                 if snapshot:
-                    snapshot.status = "failed"
+                    snapshot.status = SnapshotStatus.FAILED.value
+                    if self.preserve_snapshot_on_failure:
+                        snapshot.is_current = False
                     snapshot.completed_at = datetime.now(UTC)
                     snapshot.summary = {"error": str(e)}
             await db.commit()
@@ -509,7 +518,7 @@ class BaseCollector(ABC):
                 if snapshot:
                     snapshot.record_count = 0
                     snapshot.summary = {"created": 0, "updated": 0, "unchanged": 0}
-                    snapshot.status = "success"
+                    snapshot.status = SnapshotStatus.SUCCESS.value
                     snapshot.completed_at = datetime.now(UTC)
                     await db.commit()
             return 0
@@ -642,7 +651,7 @@ class BaseCollector(ABC):
             snapshot = await db.get(DataSnapshot, snapshot_id)
             if snapshot:
                 snapshot.record_count = records_added
-                snapshot.status = "success"
+                snapshot.status = SnapshotStatus.SUCCESS.value
                 snapshot.completed_at = datetime.now(UTC)
                 snapshot.summary = {
                     "created": created_count,

backend/app/services/collectors/celestrak.py

@@ -18,21 +18,27 @@ from app.services.business_logs import emit_business_log, exception_context
 from app.services.collectors.base import BaseCollector
 from app.services.collectors.downloads import DownloadHTTPStatusError, ResumableFileDownloader
 
-
 logger = get_logger(__name__, service="collector")
 ACTIVE_GROUP = "active"
 FALLBACK_GROUPS = (
     "starlink",
     "gps-ops",
     "galileo",
-    "glonass",
+    "glo-ops",
     "beidou",
-    "leo",
     "geo",
     "iridium-next",
+    "stations",
+    "visual",
+    "weather",
+    "science",
+    "cubesat",
+    "amateur",
+    "last-30-days",
 )
 FETCH_RETRY_ATTEMPTS = 3
 FETCH_RETRY_BASE_DELAY_SECONDS = 0.8
+CELESTRAK_CACHE_TTL_SECONDS = 2 * 60 * 60
 CELESTRAK_NOT_UPDATED_MARKER = "GP data has not updated since your last successful"
 
 
@@ -45,6 +51,8 @@ class CelesTrakTLECollector(BaseCollector):
     _downloader = ResumableFileDownloader(
         cache_namespace="celestrak",
         default_accept="application/json",
+        preflight_head=False,
+        cache_ttl_seconds=CELESTRAK_CACHE_TTL_SECONDS,
     )
 
     @property
@@ -189,7 +197,9 @@ class CelesTrakTLECollector(BaseCollector):
                     if not is_final_attempt:
                         await asyncio.sleep(FETCH_RETRY_BASE_DELAY_SECONDS * attempt)
 
-        raise RuntimeError(f"CelesTrak active satellite download failed after retries: {last_error}")
+        raise RuntimeError(
+            f"CelesTrak active satellite download failed after retries: {last_error}"
+        )
 
     async def _fetch_fallback_groups(
         self,
@@ -230,17 +240,18 @@ class CelesTrakTLECollector(BaseCollector):
                     )
                 except DownloadHTTPStatusError as exc:
                     if not self._is_not_updated_response(exc):
-                        raise RuntimeError(f"CelesTrak fallback group '{group}' download failed: {exc}") from exc
-                    cached_path = self._downloader.get_cached_file(
+                        raise RuntimeError(
+                            f"CelesTrak fallback group '{group}' download failed: {exc}"
+                        ) from exc
+                    body_path = self._downloader.get_cached_file(
                         group_url,
                         ".json",
                         validate_existing=self._validate_json_file,
                     )
-                    if cached_path is None:
+                    if body_path is None:
                         raise RuntimeError(
                             f"CelesTrak fallback group '{group}' has not updated and no local cached copy is available"
                         ) from exc
-                    body_path = cached_path
 
                 group_records = await self._load_downloaded_payload(
                     body_path,
@@ -341,13 +352,13 @@ class CelesTrakTLECollector(BaseCollector):
                 commit=True,
             )
 
-    @staticmethod
-    def _validate_json_file(path: Path) -> bool:
+    @classmethod
+    def _validate_json_file(cls, path: Path) -> bool:
         try:
-            data = json.loads(path.read_text(encoding="utf-8"))
-        except (OSError, json.JSONDecodeError, UnicodeDecodeError):
+            cls._load_active_payload(path)
+        except RuntimeError:
             return False
-        return isinstance(data, list)
+        return True
 
     async def _log_parse_failure(self, exc: Exception) -> None:
         await emit_business_log(
@@ -368,7 +379,8 @@ class CelesTrakTLECollector(BaseCollector):
             ),
         )
 
-    def _load_active_payload(self, path: Path) -> List[Dict[str, Any]]:
+    @staticmethod
+    def _load_active_payload(path: Path) -> List[Dict[str, Any]]:
         try:
             raw = json.loads(path.read_text(encoding="utf-8"))
         except (OSError, json.JSONDecodeError, UnicodeDecodeError) as exc:
@@ -384,7 +396,9 @@ class CelesTrakTLECollector(BaseCollector):
             else:
                 invalid_count += 1
         if invalid_count:
-            raise RuntimeError(f"CelesTrak active payload contains {invalid_count} invalid record(s)")
+            raise RuntimeError(
+                f"CelesTrak active payload contains {invalid_count} invalid record(s)"
+            )
         if not records:
             raise RuntimeError("CelesTrak active payload contains no satellite records")
         return records

backend/app/services/collectors/compute_base.py

@@ -0,0 +1,82 @@
+"""Quality gates shared by the two compute sources."""
+
+from datetime import datetime
+
+from sqlalchemy import select
+from sqlalchemy.ext.asyncio import AsyncSession
+
+from app.core.countries import normalize_country
+from app.models.compute_center_location import ComputeCenterLocationRecord
+from app.models.data_snapshot import DataSnapshot
+from app.services.collectors.base import BaseCollector
+
+
+class ComputeCollector(BaseCollector):
+    fail_on_empty = True
+    preserve_snapshot_on_failure = True
+
+    async def _create_snapshot(
+        self, db: AsyncSession, task_id: int, data: list[dict], started_at: datetime
+    ) -> int:
+        previous_count = await db.scalar(
+            select(DataSnapshot.record_count)
+            .where(
+                DataSnapshot.source == self.name,
+                DataSnapshot.is_current.is_(True),
+                DataSnapshot.status == "success",
+            )
+            .order_by(DataSnapshot.id.desc())
+            .limit(1)
+        )
+        if previous_count and len(data) < previous_count * 0.6:
+            raise ValueError("Compute source lost over 40% of rows; previous snapshot retained")
+        return await super()._create_snapshot(db, task_id, data, started_at)
+
+    async def _save_data(self, db: AsyncSession, data: list[dict], **kwargs) -> int:
+        if self.name == "top500":
+            await self._copy_verified_locations(db, data)
+        return await super()._save_data(db, data, **kwargs)
+
+    async def _copy_verified_locations(self, db: AsyncSession, data: list[dict]) -> None:
+        """Keep historical location keys; copy only unambiguous exact identities.
+
+        Never migrate by rank: the machine occupying rank 1 changes over time.
+        """
+        names = [item["name"] for item in data]
+        locations = (
+            (
+                await db.execute(
+                    select(ComputeCenterLocationRecord).where(
+                        ComputeCenterLocationRecord.source == self.name,
+                        ComputeCenterLocationRecord.name.in_(names),
+                    )
+                )
+            )
+            .scalars()
+            .all()
+        )
+        existing = {location.source_id for location in locations}
+        for item in data:
+            if item["source_id"] in existing:
+                continue
+            country = normalize_country(item["country"])
+            if not country:
+                continue
+            matches = [
+                location
+                for location in locations
+                if location.name == item["name"]
+                and normalize_country(location.country) == country
+                and (not location.site or location.site == item.get("metadata", {}).get("site"))
+                and not location.source_id.startswith("top500_system_")
+            ]
+            if len(matches) != 1:
+                continue
+            old = matches[0]
+            values = {
+                column.name: getattr(old, column.name)
+                for column in old.__table__.columns
+                if column.name not in {"id", "source_id", "created_at", "updated_at"}
+            }
+            db.add(ComputeCenterLocationRecord(**values, source_id=item["source_id"]))
+            existing.add(item["source_id"])

backend/app/services/collectors/downloads.py

@@ -12,7 +12,6 @@ from typing import Any, Awaitable, Callable
 
 import httpx
 
-
 ProgressCallback = Callable[[int, int | None], Awaitable[None]]
 ValidateCallback = Callable[[Path], bool]
 
@@ -52,10 +51,14 @@ class ResumableFileDownloader:
         user_agent: str = "Planet-Intelligence-System/1.0 (Python/collector)",
         default_accept: str = "*/*",
         cache_root: Path | None = None,
+        preflight_head: bool = True,
+        cache_ttl_seconds: float = 0,
     ) -> None:
         self._cache_dir = (cache_root or default_download_cache_root()) / cache_namespace
         self._user_agent = user_agent
         self._default_accept = default_accept
+        self._preflight_head = preflight_head
+        self._cache_ttl_seconds = cache_ttl_seconds
 
     @staticmethod
     def _cache_key(url: str) -> str:
@@ -93,8 +96,9 @@ class ResumableFileDownloader:
         if not meta_path.exists():
             return {}
         try:
-            return json.loads(meta_path.read_text(encoding="utf-8"))
-        except (json.JSONDecodeError, OSError):
+            data = json.loads(meta_path.read_text(encoding="utf-8"))
+            return data if isinstance(data, dict) else {}
+        except json.JSONDecodeError, OSError:
             return {}
 
     @staticmethod
@@ -111,20 +115,25 @@ class ResumableFileDownloader:
             return last_modified == str(meta.get("last_modified") or "").strip()
         return True
 
+    @staticmethod
+    def _response_info(response: httpx.Response) -> dict[str, Any]:
+        # httpx yields decoded bytes; encoded Content-Length cannot validate them.
+        raw_length = response.headers.get("content-length")
+        encoded = response.headers.get("content-encoding", "identity") != "identity"
+        return {
+            "etag": response.headers.get("etag"),
+            "last_modified": response.headers.get("last-modified"),
+            "content_length": int(raw_length) if raw_length and not encoded else None,
+            "accept_ranges": (response.headers.get("accept-ranges") or "").lower(),
+        }
+
     async def fetch_remote_info(self, client: httpx.AsyncClient, url: str) -> dict[str, Any]:
         try:
             response = await client.head(url)
             if response.status_code >= 400:
                 return {}
-            content_length_raw = response.headers.get("content-length")
-            content_length = int(content_length_raw) if content_length_raw else None
-            return {
-                "etag": response.headers.get("etag"),
-                "last_modified": response.headers.get("last-modified"),
-                "content_length": content_length,
-                "accept_ranges": (response.headers.get("accept-ranges") or "").lower(),
-            }
-        except (httpx.HTTPError, ValueError):
+            return self._response_info(response)
+        except httpx.HTTPError, ValueError:
             return {}
 
     async def download_file(
@@ -139,14 +148,24 @@ class ResumableFileDownloader:
     ) -> Path:
         self._cache_dir.mkdir(parents=True, exist_ok=True)
         final_path, part_path, meta_path = self._cache_paths(url, extension)
+        cached_path = self.get_cached_file(url, extension, validate_existing=validate_existing)
+        if cached_path and 0 <= time.time() - cached_path.stat().st_mtime < self._cache_ttl_seconds:
+            size = cached_path.stat().st_size
+            if progress_callback:
+                await progress_callback(size, size)
+            return cached_path
         meta = self._load_meta(meta_path)
-        remote = await self.fetch_remote_info(client, url)
+        remote = await self.fetch_remote_info(client, url) if self._preflight_head else {}
         expected_size = remote.get("content_length")
 
         if final_path.exists():
             local_size = final_path.stat().st_size
             size_match = expected_size is None or local_size == expected_size
-            if self._validators_match(meta, remote) and size_match:
+            if (
+                (remote.get("etag") or remote.get("last_modified"))
+                and self._validators_match(meta, remote)
+                and size_match
+            ):
                 if validate_existing and not validate_existing(final_path):
                     final_path.unlink(missing_ok=True)
                 else:
@@ -181,9 +200,13 @@ class ResumableFileDownloader:
                 headers["If-Range"] = str(remote.get("last_modified"))
 
         async with client.stream("GET", url, headers=headers) as response:
-            if response.status_code == 304 and final_path.exists():
-                if progress_callback and expected_size and expected_size > 0:
-                    await progress_callback(expected_size, expected_size)
+            if response.status_code == 304:
+                if cached_path is None:
+                    raise RuntimeError(f"Server returned 304 without a valid local cache for {url}")
+                final_path.touch()
+                size = final_path.stat().st_size
+                if progress_callback:
+                    await progress_callback(size, size)
                 return final_path
             if response.status_code >= 400:
                 body = (await response.aread()).decode("utf-8", errors="replace")
@@ -194,12 +217,16 @@ class ResumableFileDownloader:
             else:
                 mode = "wb"
                 resume_from = 0
+                remote = self._response_info(response)
+                expected_size = remote.get("content_length")
 
             downloaded = resume_from
             last_emit_bytes = 0
             last_emit_time = time.monotonic()
             min_emit_bytes = (
-                max(expected_size // 150, 512 * 1024) if expected_size and expected_size > 0 else 1024 * 1024
+                max(expected_size // 150, 512 * 1024)
+                if expected_size and expected_size > 0
+                else 1024 * 1024
             )
 
             with part_path.open(mode) as f:
@@ -230,6 +257,10 @@ class ResumableFileDownloader:
                 f"Resumable download incomplete for {url}: expected={expected_size}, got={final_size}"
             )
 
+        # Never replace the last good payload with an HTML error or invalid JSON.
+        if validate_existing and not validate_existing(part_path):
+            part_path.unlink(missing_ok=True)
+            raise RuntimeError(f"Downloaded file validation failed for {url}")
         part_path.replace(final_path)
         self._save_meta(
             meta_path,
@@ -242,9 +273,6 @@ class ResumableFileDownloader:
             },
         )
 
-        if validate_existing and not validate_existing(final_path):
-            raise RuntimeError(f"Downloaded file validation failed for {url}")
-
         if progress_callback and expected_size and expected_size > 0:
             await progress_callback(expected_size, expected_size)
 

backend/app/services/collectors/epoch_ai.py

@@ -1,119 +1,135 @@
-"""Epoch AI GPU Clusters Collector
-
-Collects data from Epoch AI GPU clusters tracking.
-https://epoch.ai/data/gpu-clusters
-"""
+"""Epoch's official CSV; preserve precision, source dates and anonymous records."""
 
+import csv
+import hashlib
+import io
+import json
+import math
 import re
-from typing import Dict, Any, List
-from datetime import UTC, datetime
-from bs4 import BeautifulSoup
+from urllib.parse import urlparse
+
 import httpx
 
-from app.services.collectors.base import BaseCollector
+from app.core.countries import normalize_country
+from app.services.collectors.compute_base import ComputeCollector
+from app.services.epoch_location import assess_epoch_location
+
+EPOCH_CSV_URL = "https://epoch.ai/data/gpu_clusters.csv"
 
 
-
-class EpochAIGPUCollector(BaseCollector):
+class EpochAIGPUCollector(ComputeCollector):
     name = "epoch_ai_gpu"
     priority = "P0"
     module = "L1"
     frequency_hours = 6
     data_type = "gpu_cluster"
 
-    async def fetch(self) -> List[Dict[str, Any]]:
-        """Fetch Epoch AI GPU clusters data from webpage"""
-        url = self._resolved_url or ""
-
-        async with httpx.AsyncClient(timeout=60.0) as client:
+    async def fetch(self) -> list[dict]:
+        configured = self._resolved_url or EPOCH_CSV_URL
+        parsed = urlparse(configured)
+        url = (
+            EPOCH_CSV_URL
+            if parsed.hostname == "epoch.ai" and not parsed.path.endswith(".csv")
+            else configured
+        )
+        async with httpx.AsyncClient(timeout=60, follow_redirects=True) as client:
             response = await client.get(url)
             response.raise_for_status()
-            return self.parse_response(response.text)
+        return self.parse_response(response.text)
 
-    def parse_response(self, html: str) -> List[Dict[str, Any]]:
-        """Parse Epoch AI webpage to extract GPU cluster data"""
-        data = []
-        soup = BeautifulSoup(html, "html.parser")
+    def parse_response(self, text: str) -> list[dict]:
+        reader = csv.DictReader(io.StringIO(text.lstrip("\ufeff")))
+        required = {"Name", "Country", "Status", "First Operational Date", "16-bit OP/s (log)"}
+        if not required.issubset(reader.fieldnames or []):
+            raise ValueError(
+                "Epoch CSV schema changed or HTML returned; previous snapshot retained"
+            )
+        entries = []
+        seen = set()
+        anonymous_counts = {}
+        for row in reader:
+            name = (row.get("Name") or "").strip()
+            country = normalize_country(row.get("Country"))
+            if not name:
+                raise ValueError("Epoch row has missing identity")
+            source_id = f"epoch_{re.sub(r'[^a-zA-Z0-9]', '_', name.lower())}"
+            anonymous = "anonym" in name.lower()
+            if anonymous:
+                # Public anonymous rows have no facility identity. Preserve
+                # multiplicity within snapshots, without guessing names.
+                digest = hashlib.sha256(json.dumps(row, sort_keys=True).encode()).hexdigest()[:24]
+                anonymous_counts[digest] = anonymous_counts.get(digest, 0) + 1
+                source_id = f"epoch_anonymous_{digest}_{anonymous_counts[digest]}"
+            if source_id in seen:
+                raise ValueError("Epoch identity collision; previous snapshot retained")
+            seen.add(source_id)
+            raw_date = row.get("First Operational Date", "")
+            reference_date = raw_date if re.fullmatch(r"\d{4}-\d{2}-\d{2}", raw_date) else None
+            value = None
+            raw_value = row.get("16-bit OP/s (log)", "")
+            if raw_value:
+                exponent = float(raw_value)
+                if not math.isfinite(exponent) or not 0 <= exponent <= 30:
+                    raise ValueError("Epoch performance exponent out of bounds")
+                value = 10**exponent
+            anonymous = anonymous or (country == "中国" and not row.get("Location"))
+            location = row.get("Location")
+            location_assessment = assess_epoch_location(
+                location,
+                country=country,
+                raw_country=row.get("Country"),
+            )
+            entries.append(
+                {
+                    "source_id": source_id,
+                    "name": name,
+                    "country": country,
+                    "value": value,
+                    "unit": "FLOP/s",
+                    "reference_date": reference_date,
+                    "latitude": None if anonymous else self.coordinate(row.get("latitude"), 90),
+                    "longitude": None if anonymous else self.coordinate(row.get("longitude"), 180),
+                    "metadata": {
+                        "compute_contract": "v1",
+                        "precision": "FP16",
+                        "measurement": "theoretical",
+                        "source_url": EPOCH_CSV_URL,
+                        "raw_reference_date": raw_date,
+                        "status": row.get("Status"),
+                        "certainty": row.get("Certainty"),
+                        "operator": row.get("Owner"),
+                        "gpu_type": row.get("Chip type (primary)"),
+                        "gpu_count": row.get("Chip quantity (primary)"),
+                        "location": location,
+                        "location_precision": location_assessment.precision,
+                        "location_precision_reason": location_assessment.reason,
+                        "raw_country": row.get("Country"),
+                        "raw_performance": raw_value,
+                        "performance_encoding": "log10",
+                        "builds_upon": row.get("Builds Upon"),
+                        "superseded_by": row.get("Superseded by"),
+                        "possible_duplicate": row.get("Possible Duplicate", "").lower() == "true",
+                        "include_in_standard_analysis": row.get(
+                            "Include in Standard Analysis", ""
+                        ).lower()
+                        == "true",
+                        "rounded": country == "中国",
+                        "anonymous": anonymous,
+                        "identity_scope": "snapshot_record" if anonymous else "named_cluster",
+                        "country_unresolved": country is None,
+                        "sources": [
+                            row[f"Source {i}"] for i in range(1, 6) if row.get(f"Source {i}")
+                        ],
+                    },
+                }
+            )
+        return entries
 
-        # Try to find data table on the page
-        tables = soup.find_all("table")
-        for table in tables:
-            rows = table.find_all("tr")
-            for row in rows[1:]:  # Skip header
-                cells = row.find_all(["td", "th"])
-                if len(cells) >= 5:
-                    try:
-                        cluster_name = cells[0].get_text(strip=True)
-                        if not cluster_name or cluster_name in ["Cluster", "System", "Name"]:
-                            continue
-
-                        location_cell = cells[1].get_text(strip=True) if len(cells) > 1 else ""
-                        country, city = self._parse_location(location_cell)
-
-                        perf_cell = cells[2].get_text(strip=True) if len(cells) > 2 else ""
-
-                        entry = {
-                            "source_id": f"epoch_{re.sub(r'[^a-zA-Z0-9]', '_', cluster_name.lower())}",
-                            "name": cluster_name,
-                            "country": country,
-                            "city": city,
-                            "latitude": "",
-                            "longitude": "",
-                            "value": self._parse_performance(perf_cell),
-                            "unit": "TFlop/s",
-                            "metadata": {
-                                "raw_data": perf_cell,
-                            },
-                            "reference_date": datetime.now(UTC).strftime("%Y-%m-%d"),
-                        }
-                        data.append(entry)
-                    except (ValueError, IndexError, AttributeError):
-                        continue
-
-        # If no table found, return sample data
-        if not data:
-            data = self._get_sample_data()
-
-        return data
-
-    def _parse_location(self, location: str) -> tuple:
-        """Parse location string into country and city"""
-        if not location:
-            return "", ""
-        if "," in location:
-            parts = location.rsplit(",", 1)
-            city = parts[0].strip()
-            country = parts[1].strip() if len(parts) > 1 else ""
-            return country, city
-        return location, ""
-
-    def _parse_performance(self, perf: str) -> str:
-        """Parse performance string to extract value"""
-        if not perf:
-            return "0"
-        match = re.search(r"([\d,.]+)\s*(TFlop/s|PFlop/s|GFlop/s)?", perf, re.I)
-        if match:
-            return match.group(1).replace(",", "")
-        match = re.search(r"([\d,.]+)", perf)
-        if match:
-            return match.group(1).replace(",", "")
-        return "0"
-
-    def _get_sample_data(self) -> List[Dict[str, Any]]:
-        """Return sample data for testing when scraping fails"""
-        return [
-            {
-                "source_id": "epoch_sample_1",
-                "name": "Sample GPU Cluster",
-                "country": "United States",
-                "city": "San Francisco, CA",
-                "latitude": "",
-                "longitude": "",
-                "value": "1000",
-                "unit": "TFlop/s",
-                "metadata": {
-                    "note": "Sample data - Epoch AI page structure may vary",
-                },
-                "reference_date": datetime.now(UTC).strftime("%Y-%m-%d"),
-            },
-        ]
+    @staticmethod
+    def coordinate(value: str | None, bound: int) -> float | None:
+        if not value:
+            return None
+        number = float(value)
+        if not math.isfinite(number) or abs(number) > bound:
+            raise ValueError("Epoch coordinate out of bounds")
+        return number

backend/app/services/collectors/news_live_streams.py

@@ -35,7 +35,7 @@ class NewsLiveStreamsCollector(BaseCollector):
     DEFAULT_IPTV_ORG_LOGOS_URL = "https://iptv-org.github.io/api/logos.json"
     DEFAULT_IPTV_ORG_NEWS_CATEGORIES = ("news", "business", "weather")
     DEFAULT_IPTV_ORG_EXCLUDE_CATEGORIES = ("music", "sports", "kids", "entertainment")
-    DEFAULT_IPTV_ORG_MAX_SOURCES = 120
+    DEFAULT_IPTV_ORG_MAX_SOURCES = 0  # Zero keeps the complete matching channel catalog.
 
     async def fetch(self) -> list[dict[str, Any]]:
         request_url = (self._resolved_url or "").strip()
@@ -445,7 +445,7 @@ class NewsLiveStreamsCollector(BaseCollector):
                     "reference_date": datetime.now(UTC).isoformat(),
                 }
             )
-            if len(normalized) >= max_sources:
+            if max_sources > 0 and len(normalized) >= max_sources:
                 break
 
         return normalized

backend/app/services/collectors/peeringdb.py

@@ -11,17 +11,20 @@ To get higher limits, set PEERINGDB_API_KEY environment variable.
 """
 
 import asyncio
-import os
-from typing import Dict, Any, List
 from datetime import UTC, datetime
+import os
+from typing import Any, Dict, List
+from urllib.parse import urlencode
 
 import httpx
-from urllib.parse import urlencode
+
+from app.core.logging import get_logger
 from app.services.collectors.base import HTTPCollector
 
 
 # PeeringDB API key - read from environment variable
 PEERINGDB_API_KEY = os.environ.get("PEERINGDB_API_KEY", "")
+logger = get_logger(__name__, service="collector")
 
 
 class PeeringDBIXPCollector(HTTPCollector):
@@ -39,6 +42,7 @@ class PeeringDBIXPCollector(HTTPCollector):
             "User-Agent": "Planet-Intelligence-System/1.0 (Python/collector)",
             "Accept": "application/json",
         }
+
     @property
     def request_url(self) -> str:
         base = self._resolved_url or self.base_url
@@ -61,7 +65,11 @@ class PeeringDBIXPCollector(HTTPCollector):
                     if response.status_code == 429:
                         # Rate limited - wait and retry with exponential backoff
                         delay = base_delay * (2**attempt)
-                        print(f"PeeringDB rate limited, waiting {delay}s before retry...")
+                        logger.warning_event(
+                            "PeeringDB rate limited; retrying after delay",
+                            event="collector.peeringdb.rate_limited",
+                            context={"delay_seconds": delay, "attempt": attempt + 1},
+                        )
                         await asyncio.sleep(delay)
                         last_error = "Rate limited"
                         continue
@@ -72,13 +80,21 @@ class PeeringDBIXPCollector(HTTPCollector):
             except httpx.HTTPStatusError as e:
                 if e.response.status_code == 429:
                     delay = base_delay * (2**attempt)
-                    print(f"PeeringDB rate limited, waiting {delay}s before retry...")
+                    logger.warning_event(
+                        "PeeringDB rate limited; retrying after delay",
+                        event="collector.peeringdb.rate_limited",
+                        context={"delay_seconds": delay, "attempt": attempt + 1},
+                    )
                     await asyncio.sleep(delay)
                     last_error = "Rate limited"
                     continue
                 raise
 
-        print(f"Warning: PeeringDB collection failed after {max_retries} retries: {last_error}")
+        logger.warning_event(
+            "PeeringDB collection failed after retries",
+            event="collector.peeringdb.retries_exhausted",
+            context={"max_retries": max_retries, "last_error": last_error},
+        )
         return {}
 
     async def fetch(self) -> List[Dict[str, Any]]:
@@ -146,6 +162,7 @@ class PeeringDBNetworkCollector(HTTPCollector):
             "User-Agent": "Planet-Intelligence-System/1.0 (Python/collector)",
             "Accept": "application/json",
         }
+
     @property
     def request_url(self) -> str:
         base = self._resolved_url or self.base_url
@@ -167,7 +184,11 @@ class PeeringDBNetworkCollector(HTTPCollector):
 
                     if response.status_code == 429:
                         delay = base_delay * (2**attempt)
-                        print(f"PeeringDB rate limited, waiting {delay}s before retry...")
+                        logger.warning_event(
+                            "PeeringDB rate limited; retrying after delay",
+                            event="collector.peeringdb.rate_limited",
+                            context={"delay_seconds": delay, "attempt": attempt + 1},
+                        )
                         await asyncio.sleep(delay)
                         last_error = "Rate limited"
                         continue
@@ -178,13 +199,21 @@ class PeeringDBNetworkCollector(HTTPCollector):
             except httpx.HTTPStatusError as e:
                 if e.response.status_code == 429:
                     delay = base_delay * (2**attempt)
-                    print(f"PeeringDB rate limited, waiting {delay}s before retry...")
+                    logger.warning_event(
+                        "PeeringDB rate limited; retrying after delay",
+                        event="collector.peeringdb.rate_limited",
+                        context={"delay_seconds": delay, "attempt": attempt + 1},
+                    )
                     await asyncio.sleep(delay)
                     last_error = "Rate limited"
                     continue
                 raise
 
-        print(f"Warning: PeeringDB collection failed after {max_retries} retries: {last_error}")
+        logger.warning_event(
+            "PeeringDB collection failed after retries",
+            event="collector.peeringdb.retries_exhausted",
+            context={"max_retries": max_retries, "last_error": last_error},
+        )
         return {}
 
     async def fetch(self) -> List[Dict[str, Any]]:
@@ -254,6 +283,7 @@ class PeeringDBFacilityCollector(HTTPCollector):
             "User-Agent": "Planet-Intelligence-System/1.0 (Python/collector)",
             "Accept": "application/json",
         }
+
     @property
     def request_url(self) -> str:
         base = self._resolved_url or self.base_url
@@ -275,7 +305,11 @@ class PeeringDBFacilityCollector(HTTPCollector):
 
                     if response.status_code == 429:
                         delay = base_delay * (2**attempt)
-                        print(f"PeeringDB rate limited, waiting {delay}s before retry...")
+                        logger.warning_event(
+                            "PeeringDB rate limited; retrying after delay",
+                            event="collector.peeringdb.rate_limited",
+                            context={"delay_seconds": delay, "attempt": attempt + 1},
+                        )
                         await asyncio.sleep(delay)
                         last_error = "Rate limited"
                         continue
@@ -286,13 +320,21 @@ class PeeringDBFacilityCollector(HTTPCollector):
             except httpx.HTTPStatusError as e:
                 if e.response.status_code == 429:
                     delay = base_delay * (2**attempt)
-                    print(f"PeeringDB rate limited, waiting {delay}s before retry...")
+                    logger.warning_event(
+                        "PeeringDB rate limited; retrying after delay",
+                        event="collector.peeringdb.rate_limited",
+                        context={"delay_seconds": delay, "attempt": attempt + 1},
+                    )
                     await asyncio.sleep(delay)
                     last_error = "Rate limited"
                     continue
                 raise
 
-        print(f"Warning: PeeringDB collection failed after {max_retries} retries: {last_error}")
+        logger.warning_event(
+            "PeeringDB collection failed after retries",
+            event="collector.peeringdb.retries_exhausted",
+            context={"max_retries": max_retries, "last_error": last_error},
+        )
         return {}
 
     async def fetch(self) -> List[Dict[str, Any]]:

backend/app/services/collectors/spacetrack.py

@@ -1,17 +1,21 @@
-"""Space-Track TLE Collector
+"""Space-Track TLE Collector.
 
 Collects satellite TLE (Two-Line Element) data from Space-Track.org.
 API documentation: https://www.space-track.org/documentation
 """
 
-import json
-from typing import Dict, Any, List
-import httpx
+from typing import Any, Dict, List
 from urllib.parse import urlparse
 
-from app.services.collectors.base import BaseCollector
+import httpx
+
 from app.core.data_sources import get_data_sources_config
+from app.core.logging import get_logger
 from app.core.satellite_tle import build_tle_lines_from_elements
+from app.services.collectors.base import BaseCollector
+
+
+logger = get_logger(__name__, service="collector")
 
 
 class SpaceTrackTLECollector(BaseCollector):
@@ -53,10 +57,16 @@ class SpaceTrackTLECollector(BaseCollector):
         password = settings.SPACETRACK_PASSWORD
 
         if not username or not password:
-            print("SPACETRACK: No credentials configured, using sample data")
+            logger.warning_event(
+                "Space-Track credentials are not configured; using sample data",
+                event="collector.spacetrack.credentials_missing",
+            )
             return self._get_sample_data()
 
-        print(f"SPACETRACK: Attempting to fetch TLE data with username: {username}")
+        logger.info_event(
+            "Space-Track TLE fetch started",
+            event="collector.spacetrack.fetch.start",
+        )
 
         try:
             async with httpx.AsyncClient(
@@ -78,11 +88,17 @@ class SpaceTrackTLECollector(BaseCollector):
                         "password": password,
                     },
                 )
-                print(f"SPACETRACK: Login response status: {login_response.status_code}")
-                print(f"SPACETRACK: Login response URL: {login_response.url}")
+                logger.info_event(
+                    "Space-Track login response received",
+                    event="collector.spacetrack.login.response",
+                    context={"status_code": login_response.status_code},
+                )
 
                 if login_response.status_code == 403:
-                    print("SPACETRACK: Trying alternate login method...")
+                    logger.warning_event(
+                        "Space-Track login returned forbidden; trying alternate method",
+                        event="collector.spacetrack.login.forbidden",
+                    )
 
                     async with httpx.AsyncClient(
                         timeout=120.0,
@@ -90,11 +106,6 @@ class SpaceTrackTLECollector(BaseCollector):
                     ) as alt_client:
                         await alt_client.get(f"{self.site_root}/")
 
-                        form_data = {
-                            "username": username,
-                            "password": password,
-                            "query": "class/gp/NORAD_CAT_ID/25544/format/json",
-                        }
                         alt_login = await alt_client.post(
                             self.login_url,
                             data={
@@ -102,77 +113,59 @@ class SpaceTrackTLECollector(BaseCollector):
                                 "password": password,
                             },
                         )
-                        print(f"SPACETRACK: Alt login status: {alt_login.status_code}")
+                        logger.info_event(
+                            "Space-Track alternate login response received",
+                            event="collector.spacetrack.alt_login.response",
+                            context={"status_code": alt_login.status_code},
+                        )
 
                         if alt_login.status_code == 200:
                             tle_response = await alt_client.get(self.probe_url)
                             if tle_response.status_code == 200:
                                 data = tle_response.json()
-                                print(f"SPACETRACK: Received {len(data)} records via alt method")
+                                logger.info_event(
+                                    "Space-Track alternate query completed",
+                                    event="collector.spacetrack.alt_query.completed",
+                                    context={"record_count": len(data)},
+                                )
                                 return data
 
                 if login_response.status_code != 200:
-                    print(f"SPACETRACK: Login failed, using sample data")
+                    logger.warning_event(
+                        "Space-Track login failed; using sample data",
+                        event="collector.spacetrack.login.failed",
+                        context={"status_code": login_response.status_code},
+                    )
                     return self._get_sample_data()
 
                 tle_response = await client.get(self.probe_url)
-                print(f"SPACETRACK: TLE query status: {tle_response.status_code}")
-
-                if tle_response.status_code != 200:
-                    print(f"SPACETRACK: Query failed, using sample data")
-                    return self._get_sample_data()
-
-                data = tle_response.json()
-                print(f"SPACETRACK: Received {len(data)} records")
-                return data
-        except Exception as e:
-            print(f"SPACETRACK: Error - {e}, using sample data")
-            return self._get_sample_data()
-
-        print(f"SPACETRACK: Attempting to fetch TLE data with username: {username}")
-
-        try:
-            async with httpx.AsyncClient(
-                timeout=120.0,
-                follow_redirects=True,
-                headers={
-                    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36",
-                    "Accept": "application/json, text/html, */*",
-                    "Accept-Language": "en-US,en;q=0.9",
-                },
-            ) as client:
-                # First, visit the main page to get any cookies
-                await client.get(f"{self.site_root}/")
-
-                # Login to get session cookie
-                login_response = await client.post(
-                    self.login_url,
-                    data={
-                        "identity": username,
-                        "password": password,
-                    },
+                logger.info_event(
+                    "Space-Track TLE query response received",
+                    event="collector.spacetrack.query.response",
+                    context={"status_code": tle_response.status_code},
                 )
-                print(f"SPACETRACK: Login response status: {login_response.status_code}")
-                print(f"SPACETRACK: Login response URL: {login_response.url}")
-                print(f"SPACETRACK: Login response body: {login_response.text[:500]}")
-
-                if login_response.status_code != 200:
-                    print(f"SPACETRACK: Login failed, using sample data")
-                    return self._get_sample_data()
-
-                # Query for TLE data (get first 1000 satellites)
-                tle_response = await client.get(self.query_url)
-                print(f"SPACETRACK: TLE query status: {tle_response.status_code}")
 
                 if tle_response.status_code != 200:
-                    print(f"SPACETRACK: Query failed, using sample data")
+                    logger.warning_event(
+                        "Space-Track TLE query failed; using sample data",
+                        event="collector.spacetrack.query.failed",
+                        context={"status_code": tle_response.status_code},
+                    )
                     return self._get_sample_data()
 
                 data = tle_response.json()
-                print(f"SPACETRACK: Received {len(data)} records")
+                logger.info_event(
+                    "Space-Track TLE fetch completed",
+                    event="collector.spacetrack.fetch.completed",
+                    context={"record_count": len(data)},
+                )
                 return data
         except Exception as e:
-            print(f"SPACETRACK: Error - {e}, using sample data")
+            logger.warning_event(
+                "Space-Track TLE fetch failed; using sample data",
+                event="collector.spacetrack.fetch.failed",
+                context={"error": str(e)},
+            )
             return self._get_sample_data()
 
     def transform(self, raw_data: List[Dict[str, Any]]) -> List[Dict[str, Any]]:

backend/app/services/collectors/top500.py

@@ -5,16 +5,18 @@ https://top500.org/lists/top500/
 """
 
 import asyncio
+import math
 import re
+from urllib.parse import urljoin
 from typing import Dict, Any, List
 from bs4 import BeautifulSoup
 import httpx
 
 from app.core.data_sources import get_data_sources_config
-from app.services.collectors.base import BaseCollector
+from app.services.collectors.compute_base import ComputeCollector
 
 
-class TOP500Collector(BaseCollector):
+class TOP500Collector(ComputeCollector):
     name = "top500"
     priority = "P0"
     module = "L1"
@@ -28,7 +30,7 @@ class TOP500Collector(BaseCollector):
         async with httpx.AsyncClient(timeout=60.0, follow_redirects=True) as client:
             response = await client.get(url)
             response.raise_for_status()
-            entries = self.parse_response(response.text)
+            entries = self.parse_response(response.text, str(response.url))
 
             semaphore = asyncio.Semaphore(8)
 
@@ -55,7 +57,7 @@ class TOP500Collector(BaseCollector):
         system_name = link.get_text(" ", strip=True) if link else system_cell.get_text(" ", strip=True)
         detail_url = ""
         if link and link.get("href"):
-            detail_url = f"{str(top500_base_url).rstrip('/')}{link.get('href')}"
+            detail_url = urljoin(str(top500_base_url), link.get("href"))
 
         manufacturer = ""
         if link and link.next_sibling:
@@ -127,10 +129,14 @@ class TOP500Collector(BaseCollector):
 
         return detail_map
 
-    def parse_response(self, html: str) -> List[Dict[str, Any]]:
+    def parse_response(self, html: str, source_url: str = "") -> List[Dict[str, Any]]:
         """Parse TOP500 HTML response"""
         data = []
         soup = BeautifulSoup(html, "html.parser")
+        canonical = soup.find("link", rel="canonical")
+        date_url = source_url or (canonical.get("href", "") if canonical else "") or self._resolved_url or ""
+        date_match = re.search(r"/lists/top500/(\d{4})/(06|11)/", date_url)
+        reference_date = f"{date_match[1]}-{date_match[2]}-01" if date_match else None
 
         # Find the table with TOP500 data
         table = None
@@ -152,6 +158,9 @@ class TOP500Collector(BaseCollector):
             table = soup.find("table", {"class": "top500-table"}) or soup.find("table", {"id": "top500"})
 
         if table:
+            headers = [cell.get_text(" ", strip=True) for cell in table.select("tr")[0].find_all(["th", "td"])]
+            if not reference_date:
+                raise ValueError("TOP500 list date missing; previous snapshot retained")
             rows = table.select("tr")
             for row in rows[1:]:  # Skip header row
                 cells = row.find_all(["td", "th"])
@@ -176,25 +185,32 @@ class TOP500Collector(BaseCollector):
                         cores = cells[2].get_text(strip=True).replace(",", "")
 
                         rmax_text = cells[3].get_text(strip=True)
-                        rmax = self._parse_performance(rmax_text)
+                        rmax = self._parse_performance(rmax_text, headers[3])
 
                         rpeak_text = cells[4].get_text(strip=True)
-                        rpeak = self._parse_performance(rpeak_text)
+                        rpeak = self._parse_performance(rpeak_text, headers[4])
 
                         power = ""
                         if len(cells) >= 6:
                             power = cells[5].get_text(strip=True).replace(",", "")
 
+                        identity = re.search(r"/system/(\d+)", detail_url)
+                        if not identity:
+                            raise ValueError("TOP500 stable system ID missing")
                         entry = {
-                            "source_id": f"top500_{rank}",
+                            "source_id": f"top500_system_{identity[1]}",
                             "name": system_name,
                             "country": country,
                             "city": city,
-                            "latitude": 0.0,
-                            "longitude": 0.0,
+                            "latitude": None,
+                            "longitude": None,
                             "value": str(rmax),
-                            "unit": "PFlop/s",
+                            "unit": "FLOP/s",
                             "metadata": {
+                                "compute_contract": "v1",
+                                "precision": "FP64", "measurement": "LINPACK Rmax",
+                                "source_url": detail_url, "list_url": date_url,
+                                "status": "listed", "rmax_flops": rmax, "rpeak_flops": rpeak,
                                 "rank": rank,
                                 "cores": cores,
                                 "rmax": rmax_text,
@@ -204,15 +220,11 @@ class TOP500Collector(BaseCollector):
                                 "site": site,
                             },
                             "_detail_url": detail_url,
-                            "reference_date": "2025-11-01",
+                            "reference_date": reference_date,
                         }
                         data.append(entry)
                     except (ValueError, IndexError, AttributeError) as e:
-                        continue
-
-        # If scraping failed, return sample data for testing
-        if not data:
-            data = self._get_sample_data()
+                        raise ValueError("Invalid TOP500 row; previous snapshot retained") from e
 
         return data
 
@@ -227,9 +239,14 @@ class TOP500Collector(BaseCollector):
                 return 0.0
         return 0.0
 
-    def _parse_performance(self, text: str) -> float:
+    def _parse_performance(self, text: str, header: str = "") -> float:
         """Parse performance value from text (handles E, P, T suffixes)"""
-        text = text.strip().upper()
+        text = text.strip().upper().replace(",", "")
+        if not re.search(r"[EPTGMK]F", text):
+            unit = re.search(r"\b([EPTGMK])FLOP/?S", header.upper())
+            if not unit:
+                raise ValueError("TOP500 performance unit missing")
+            text = f"{text} {unit[1]}FLOP/S"
         multipliers = {
             "E": 1e18,
             "P": 1e15,
@@ -249,73 +266,9 @@ class TOP500Collector(BaseCollector):
 
         # Try simple float parsing
         try:
-            return float(text.replace(",", ""))
+            value = float(text.replace(",", ""))
+            if not math.isfinite(value) or value < 0:
+                raise ValueError("Invalid TOP500 performance")
+            return value
         except ValueError:
-            return 0.0
-
-    def _get_sample_data(self) -> List[Dict[str, Any]]:
-        """Return sample data for testing when scraping fails"""
-        return [
-            {
-                "source_id": "top500_1",
-                "name": "El Capitan - HPE Cray EX255a, AMD 4th Gen EPYC 24C 1.8GHz, AMD Instinct MI300A",
-                "country": "United States",
-                "city": "Livermore, CA",
-                "latitude": 37.6819,
-                "longitude": -121.7681,
-                "value": "1742.00",
-                "unit": "PFlop/s",
-                "metadata": {
-                    "rank": 1,
-                    "cores": "11039616",
-                    "rmax": "1742.00",
-                    "rpeak": "2746.38",
-                    "power": "29581",
-                    "manufacturer": "HPE",
-                    "site": "DOE/NNSA/LLNL",
-                    "processor": "AMD 4th Gen EPYC 24C 1.8GHz",
-                    "interconnect": "Slingshot-11",
-                    "installation_year": "2025",
-                },
-                "reference_date": "2025-11-01",
-            },
-            {
-                "source_id": "top500_2",
-                "name": "Frontier - HPE Cray EX235a, AMD Optimized 3rd Generation EPYC 64C 2GHz, AMD Instinct MI250X",
-                "country": "United States",
-                "city": "Oak Ridge, TN",
-                "latitude": 36.0107,
-                "longitude": -84.2663,
-                "value": "1353.00",
-                "unit": "PFlop/s",
-                "metadata": {
-                    "rank": 2,
-                    "cores": "9066176",
-                    "rmax": "1353.00",
-                    "rpeak": "2055.72",
-                    "power": "24607",
-                    "manufacturer": "HPE",
-                    "site": "DOE/SC/Oak Ridge National Laboratory",
-                },
-                "reference_date": "2025-11-01",
-            },
-            {
-                "source_id": "top500_3",
-                "name": "Aurora - HPE Cray EX - Intel Exascale Compute Blade, Xeon CPU Max 9470 52C 2.4GHz, Intel Data Center GPU Max",
-                "country": "United States",
-                "city": "Argonne, IL",
-                "latitude": 41.3784,
-                "longitude": -87.8600,
-                "value": "1012.00",
-                "unit": "PFlop/s",
-                "metadata": {
-                    "rank": 3,
-                    "cores": "9264128",
-                    "rmax": "1012.00",
-                    "rpeak": "1980.01",
-                    "power": "38698",
-                    "manufacturer": "Intel",
-                },
-                "reference_date": "2025-11-01",
-            },
-        ]
+            raise ValueError("Invalid TOP500 performance") from None

backend/app/services/collectors/vessel_ais.py

@@ -6,6 +6,7 @@ from typing import Any
 import httpx
 from sqlalchemy.ext.asyncio import AsyncSession
 
+from app.core.enums import SnapshotStatus
 from app.core.time import to_iso8601_utc
 from app.core.websocket.broadcaster import broadcaster
 from app.services.barentswatch import (
@@ -119,6 +120,21 @@ class VesselAISCollector(BaseCollector):
             last_success_at=now if data else None,
             lag_seconds=max((now - latest_observed_at).total_seconds(), 0),
         )
+        if snapshot_id is not None:
+            from app.models.data_snapshot import DataSnapshot
+
+            snapshot = await db.get(DataSnapshot, snapshot_id)
+            if snapshot:
+                snapshot.record_count = records_added
+                snapshot.status = SnapshotStatus.SUCCESS.value
+                snapshot.completed_at = now
+                snapshot.summary = {
+                    "created": records_added,
+                    "updated": 0,
+                    "unchanged": 0,
+                    "deleted": 0,
+                    "storage": "ais_raw_observations",
+                }
         await db.commit()
         await self._broadcast_vessel_snapshot(data)
         await self.update_progress(records_added, force=True)

backend/app/services/compute_catalog.py

@@ -0,0 +1,200 @@
+"""Canonical country and mutually exclusive regional compute participants."""
+
+import re
+from collections.abc import Iterable
+
+from app.core.countries import COUNTRY_CENTROIDS, COUNTRY_ENTRIES
+from app.core.earth_geography_policy import display_country
+
+PALETTE = ("#91baff", "#f0c88a", "#89d2c2", "#c5abdf", "#eda9b3", "#b6c99a")
+DEFAULT_PARTICIPANTS = ("CN", "US", "EUROPE", "APAC", "OTHER")
+
+# Explicit membership prevents a regional total from double-counting a country.
+# APAC deliberately excludes China.
+COMPARISON_GROUPS = (
+    ("CN", "中国", "China", ("CN",), {"latitude": 35.8617, "longitude": 104.1954}),
+    ("US", "美国", "United States", ("US",), {"latitude": 39.8283, "longitude": -98.5795}),
+    (
+        "EUROPE",
+        "欧洲",
+        "Europe",
+        (
+            "GB",
+            "DE",
+            "FR",
+            "CH",
+            "ES",
+            "FI",
+            "HU",
+            "IE",
+            "IT",
+            "LU",
+            "MD",
+            "NL",
+            "PL",
+            "RO",
+            "RU",
+            "UA",
+        ),
+        {"latitude": 54.0, "longitude": 20.0},
+    ),
+    (
+        "APAC",
+        "亚太（不含中国）",
+        "Asia Pacific (excluding China)",
+        ("JP", "KR", "IN", "AU", "AF", "ID", "NP", "PH", "PK", "SG", "TH"),
+        {"latitude": 36.0, "longitude": 130.5},
+    ),
+    (
+        "OTHER",
+        "其他地区",
+        "Other",
+        (
+            "CA",
+            "AO",
+            "EG",
+            "KE",
+            "MZ",
+            "NG",
+            "UG",
+            "ZA",
+            "BR",
+            "CO",
+            "CR",
+            "DO",
+            "MX",
+            "AE",
+            "SY",
+            "TR",
+        ),
+        {"latitude": 12.0, "longitude": 10.0},
+    ),
+)
+
+
+def _country_code(canonical: str, aliases: list[str]) -> str | None:
+    code = next((alias for alias in aliases if re.fullmatch(r"[A-Z]{2}", alias)), None)
+    return "GB" if canonical == "英国" else code
+
+
+def build_country_catalog() -> list[dict]:
+    """Build the policy-projected canonical country drilldown catalog."""
+    countries = []
+    for canonical, aliases in COUNTRY_ENTRIES:
+        if display_country(canonical) != canonical or canonical not in COUNTRY_CENTROIDS:
+            continue
+        code = _country_code(canonical, aliases)
+        if code:
+            countries.append(
+                {
+                    "code": code,
+                    "name": canonical,
+                    "name_en": aliases[0],
+                    "kind": "country",
+                    "members": [code],
+                    "anchor": dict(COUNTRY_CENTROIDS[canonical]),
+                    "anchor_kind": "regional_label",
+                }
+            )
+    return countries
+
+
+COUNTRY_CATALOG = build_country_catalog()
+COUNTRIES = {entry["code"]: entry["name"] for entry in COUNTRY_CATALOG}
+COUNTRY_PARTICIPANTS = {entry["code"]: entry for entry in COUNTRY_CATALOG}
+
+
+def _validate_group_members() -> None:
+    members = [
+        member for _code, _name, _name_en, codes, _anchor in COMPARISON_GROUPS for member in codes
+    ]
+    if len(members) != len(set(members)) or set(members) != set(COUNTRIES):
+        raise ValueError("compute comparison groups must cover country catalog exactly once")
+
+
+_validate_group_members()
+GROUP_BY_MEMBER = {
+    member: code
+    for code, _name, _name_en, members, _anchor in COMPARISON_GROUPS
+    for member in members
+}
+
+
+def build_catalog() -> list[dict]:
+    """Return the five default participants before country drilldown entries."""
+    catalog: list[dict] = []
+    for index, (code, name, name_en, members, anchor) in enumerate(COMPARISON_GROUPS):
+        entry = (
+            dict(COUNTRY_PARTICIPANTS[code])
+            if code in COUNTRY_PARTICIPANTS
+            else {
+                "code": code,
+                "name": name,
+                "name_en": name_en,
+                "kind": "region",
+                "members": list(members),
+                "anchor": dict(anchor),
+                "anchor_kind": "regional_label",
+            }
+        )
+        entry.update(
+            {
+                "members": list(members),
+                "parent_group": name,
+                "parent_group_code": code,
+                "order": index,
+                "default_selected": True,
+                "color": PALETTE[index % len(PALETTE)],
+            }
+        )
+        catalog.append(entry)
+
+    for entry in COUNTRY_CATALOG:
+        if entry["code"] in DEFAULT_PARTICIPANTS:
+            continue
+        country = dict(entry)
+        group_code = GROUP_BY_MEMBER[country["code"]]
+        group_name = next(group[1] for group in COMPARISON_GROUPS if group[0] == group_code)
+        country.update(
+            {
+                "parent_group": group_name,
+                "parent_group_code": group_code,
+                "order": len(catalog),
+                "default_selected": False,
+                "color": PALETTE[len(catalog) % len(PALETTE)],
+            }
+        )
+        catalog.append(country)
+    return catalog
+
+
+CATALOG = build_catalog()
+PARTICIPANTS = {entry["code"]: entry for entry in CATALOG}
+
+
+def participant_members(codes: Iterable[str]) -> dict[str, tuple[str, ...]]:
+    """Resolve country or region participant codes to canonical country codes."""
+    return {code: tuple(PARTICIPANTS[code]["members"]) for code in codes}
+
+
+def source_country_names(codes: Iterable[str]) -> list[str]:
+    """Return unique canonical source labels, respecting China and Serbia ownership."""
+    selected_members = {
+        member for members in participant_members(codes).values() for member in members
+    }
+    selected_names = {COUNTRIES[code] for code in selected_members}
+    return [name for name, _aliases in COUNTRY_ENTRIES if display_country(name) in selected_names]
+
+
+def source_country_participants(codes: Iterable[str]) -> dict[str, str]:
+    """Map stored source-country labels to their requested participant code."""
+    owner_by_country = {
+        member: participant
+        for participant, members in participant_members(codes).items()
+        for member in members
+    }
+    code_by_name = {name: code for code, name in COUNTRIES.items()}
+    return {
+        name: owner_by_country[code_by_name[display_country(name)]]
+        for name in source_country_names(codes)
+    }

backend/app/services/compute_center_locations.py

@@ -3,7 +3,12 @@
 This module is a thin domain wrapper that wires up
 :mod:`app.services.location` for compute centers:
 
-    SourceCoordinates
+    Confirmed stored location -> trustworthy source coordinates -> exact
+    official facility registration -> source city/site coordinates.
+
+Epoch AI coordinates are admitted only at the grain asserted by its official
+``Location`` column. Country/region/empty locations remain unresolved even
+when the CSV happens to contain a latitude/longitude pair.
 
 The online Nominatim step is intentionally reserved for the user-triggered
 ``collect-location`` flow. The regular GeoJSON endpoint runs during Earth
@@ -28,9 +33,12 @@ import httpx
 from sqlalchemy import select
 from sqlalchemy.ext.asyncio import AsyncSession
 
+from app.core.compute_types import compute_record_filter
 from app.core.collected_data_fields import get_record_field
 from app.models.collected_data import CollectedData
 from app.models.compute_center_location import ComputeCenterLocationRecord
+from app.services.compute_center_verified_sites import get_verified_compute_site
+from app.services.epoch_location import assess_epoch_location
 
 from app.services.location import (
     LocationCandidate,
@@ -166,8 +174,13 @@ def set_compute_center_location_cache(
 
 async def refresh_compute_center_location_cache(
     session: AsyncSession,
+    *,
+    as_of: datetime | None = None,
 ) -> dict[str, dict[str, Any]]:
-    result = await session.execute(select(ComputeCenterLocationRecord))
+    query = select(ComputeCenterLocationRecord)
+    if as_of is not None:
+        query = query.where(ComputeCenterLocationRecord.verified_at < as_of)
+    result = await session.execute(query)
     records = result.scalars().all()
     cache = {}
     for record in records:
@@ -350,6 +363,58 @@ class ROROrganizationResolver:
         )
 
 
+class ComputeCenterSourceCoordinatesResolver:
+    """Use source coordinates only at the grain asserted by the source text."""
+
+    name = "source_coordinates"
+
+    def __init__(self) -> None:
+        self._default = SourceCoordinatesResolver()
+
+    def resolve(self, query: LocationQuery) -> ResolverOutput:
+        extra = query.extra or {}
+        if extra.get("source") != "epoch_ai_gpu":
+            return self._default.resolve(query)
+        if query.source_latitude in (None, 0.0) or query.source_longitude in (None, 0.0):
+            return ResolverOutput()
+        assessment = assess_epoch_location(
+            extra.get("source_location"),
+            country=query.country,
+            raw_country=extra.get("source_raw_country"),
+        )
+        if not assessment.is_renderable:
+            return ResolverOutput(
+                attempted_queries=(f"source_coordinates:{assessment.precision}",)
+            )
+        output = self._default.resolve(query)
+        if not output.candidates:
+            return output
+        candidate = output.candidates[0]
+        source_location = extra.get("source_location")
+        source_location_text = (
+            source_location.strip() if isinstance(source_location, str) else ""
+        )
+        source_note = assessment.reason
+        if source_location_text:
+            source_note = f"{source_note}; Epoch Location: {source_location_text}"
+        return ResolverOutput(
+            candidates=(
+                LocationCandidate(
+                    **{
+                        **candidate.__dict__,
+                        "precision": assessment.precision,
+                        "confidence": 1.0 if assessment.precision == "precise" else 0.9,
+                        "source_note": source_note,
+                        "needs_confirmation": assessment.precision != "precise",
+                        "matched_location_name": source_location_text
+                        or candidate.matched_location_name,
+                    }
+                ),
+            ),
+            attempted_queries=output.attempted_queries,
+        )
+
+
 class StoredComputeCenterLocationResolver:
     """Resolve a compute center through the DB-backed current-location cache."""
 
@@ -367,13 +432,26 @@ class StoredComputeCenterLocationResolver:
         longitude = parse_float(stored.get("longitude"))
         if latitude in (None, 0.0) or longitude in (None, 0.0):
             return ResolverOutput()
+        precision = stored.get("precision") or "city"
+        if (
+            extra.get("source") == "epoch_ai_gpu"
+            and stored.get("location_source") == "source_coordinates"
+        ):
+            assessment = assess_epoch_location(
+                extra.get("source_location"),
+                country=query.country,
+                raw_country=extra.get("source_raw_country"),
+            )
+            if not assessment.is_renderable:
+                return ResolverOutput()
+            precision = assessment.precision
         return ResolverOutput(
             candidates=(
                 LocationCandidate(
                     latitude=latitude,
                     longitude=longitude,
                     display_name=stored.get("name") or query.name or "Compute center",
-                    precision=stored.get("precision") or "city",
+                    precision=precision,
                     confidence=float(stored.get("confidence") or 0.85),
                     query=f"stored_compute_center_location::{stored.get('source')}:{stored.get('source_id')}",
                     source=self.name,
@@ -386,6 +464,60 @@ class StoredComputeCenterLocationResolver:
                     matched_location_name=stored.get("site") or stored.get("name") or query.name,
                     location_verified_at=stored.get("verified_at"),
                     suggested_registry_entry=None,
+                    raw_payload={
+                        "stored_location_source": stored.get("location_source"),
+                        "verification_status": stored.get("verification_status"),
+                    },
+                ),
+            )
+        )
+
+
+class VerifiedComputeSiteResolver:
+    """Resolve an exact source identity to an evidence-backed facility site."""
+
+    name = "official_facility_registry"
+
+    def resolve(self, query: LocationQuery) -> ResolverOutput:
+        extra = query.extra or {}
+        source = coerce_str(extra.get("source"))
+        source_id = coerce_str(extra.get("source_id"))
+        if not source or not source_id:
+            return ResolverOutput()
+        site = get_verified_compute_site(source, source_id)
+        if not site:
+            return ResolverOutput()
+        source_urls = [coerce_str(url) for url in site.get("sources", []) if coerce_str(url)]
+        source_note = coerce_str(site.get("source_note"))
+        if source_urls:
+            source_note = f"{source_note} Sources: {'; '.join(source_urls)}".strip()
+        return ResolverOutput(
+            candidates=(
+                LocationCandidate(
+                    latitude=float(site["latitude"]),
+                    longitude=float(site["longitude"]),
+                    display_name=coerce_str(site.get("matched_location_name"))
+                    or query.name
+                    or "Compute center",
+                    precision=coerce_str(site.get("precision")) or "site",
+                    confidence=1.0,
+                    query=f"official_facility_registry::{source}:{source_id}",
+                    source=self.name,
+                    source_note=source_note or None,
+                    matched_fields=("source", "source_id"),
+                    needs_confirmation=False,
+                    city=coerce_str(site.get("city")) or None,
+                    region=None,
+                    country=coerce_str(site.get("country")) or query.country,
+                    matched_location_name=coerce_str(site.get("matched_location_name"))
+                    or None,
+                    location_verified_at=coerce_str(site.get("verified_at")) or None,
+                    suggested_registry_entry=None,
+                    raw_payload={
+                        "address": site.get("address"),
+                        "coordinate_basis": site.get("coordinate_basis"),
+                        "sources": source_urls,
+                    },
                 ),
             )
         )
@@ -400,7 +532,7 @@ def _short_system_name(name: Any) -> str:
     return head or text
 
 
-def _record_context(record: Any, metadata: dict[str, Any]) -> dict[str, str]:
+def _record_context(record: Any, metadata: dict[str, Any]) -> dict[str, Any]:
     name = coerce_str(getattr(record, "name", None))
     return {
         "source": coerce_str(getattr(record, "source", None)),
@@ -417,11 +549,13 @@ def _record_context(record: Any, metadata: dict[str, Any]) -> dict[str, str]:
             or metadata.get("manufacturer")
         ),
         "organization": coerce_str(metadata.get("organization")),
+        "source_location": metadata.get("location"),
+        "source_raw_country": metadata.get("raw_country"),
     }
 
 
 def _context_to_query(
-    context: dict[str, str],
+    context: dict[str, Any],
     *,
     source_lat: float | None = None,
     source_lon: float | None = None,
@@ -444,6 +578,8 @@ def _context_to_query(
             "site": context.get("site") or "",
             "operator": context.get("operator") or "",
             "organization": context.get("organization") or "",
+            "source_location": context.get("source_location"),
+            "source_raw_country": context.get("source_raw_country"),
         },
     )
 
@@ -497,11 +633,36 @@ def _compute_center_query_plan(
     return plan
 
 
+def _compute_center_candidate_sort_key(candidate: LocationCandidate) -> tuple[int, int, float]:
+    stored_origin = (candidate.raw_payload or {}).get("stored_location_source")
+    stored_confirmed = (
+        candidate.source == "stored_compute_center_location"
+        and not candidate.needs_confirmation
+        and stored_origin != "source_coordinates"
+    )
+    if stored_confirmed:
+        source_rank = 0
+    elif candidate.source == "source_coordinates" and candidate.precision == "precise":
+        source_rank = 1
+    elif candidate.source == "official_facility_registry":
+        source_rank = 2
+    elif candidate.source == "source_coordinates":
+        source_rank = 3
+    elif candidate.source == "stored_compute_center_location":
+        source_rank = 4
+    else:
+        source_rank = 5
+    precision_rank = {"precise": 0, "site": 1, "city": 2}.get(candidate.precision, 9)
+    return (source_rank, precision_rank, -float(candidate.confidence or 0))
+
+
 COMPUTE_CENTER_PIPELINE = LocationPipeline(
     [
-        SourceCoordinatesResolver(),
+        ComputeCenterSourceCoordinatesResolver(),
         StoredComputeCenterLocationResolver(),
+        VerifiedComputeSiteResolver(),
     ],
+    sort_key=_compute_center_candidate_sort_key,
     failure_reason=(
         "Could not resolve to city-level coordinates from source coords"
         " or stored compute-center location."
@@ -510,7 +671,8 @@ COMPUTE_CENTER_PIPELINE = LocationPipeline(
 
 COMPUTE_CENTER_COLLECTION_PIPELINE = LocationPipeline(
     [
-        SourceCoordinatesResolver(),
+        ComputeCenterSourceCoordinatesResolver(),
+        VerifiedComputeSiteResolver(),
         ROROrganizationResolver(),
         NominatimResolver(
             query_plan_builder=_compute_center_query_plan,
@@ -518,6 +680,7 @@ COMPUTE_CENTER_COLLECTION_PIPELINE = LocationPipeline(
             geocoder=lambda q: _geocode_online(q),
         ),
     ],
+    sort_key=_compute_center_candidate_sort_key,
     failure_reason=(
         "Could not resolve to city-level coordinates from source coords"
         ", ROR organization lookup, or online geocoding."
@@ -531,6 +694,7 @@ COMPUTE_CENTER_COLLECTION_PIPELINE = LocationPipeline(
 _GEOGRAPHY_MODE_BY_SOURCE = {
     "source_coordinates": "source_coordinates",
     "stored_compute_center_location": "stored_compute_center_location",
+    "official_facility_registry": "official_facility_registry",
     "ror_organization_registry": "ror_organization",
     "nominatim_online_geocode": "online_geocode",
 }
@@ -539,7 +703,7 @@ _GEOGRAPHY_MODE_BY_SOURCE = {
 def _candidate_to_location(
     candidate: LocationCandidate,
     *,
-    context: dict[str, str],
+    context: dict[str, Any],
 ) -> ComputeCenterLocation:
     geography_mode = _GEOGRAPHY_MODE_BY_SOURCE.get(candidate.source, "online_geocode")
     is_estimated = candidate.needs_confirmation or candidate.source.startswith(
@@ -594,7 +758,7 @@ def _candidate_to_location(
 
 def _diagnostic_for(
     record: Any,
-    context: dict[str, str],
+    context: dict[str, Any],
     *,
     failure_reason: str,
     attempted_queries: tuple[str, ...] = (),
@@ -738,7 +902,7 @@ def build_compute_center_location_query(
     organization: str | None = None,
 ) -> LocationQuery:
     name_value = coerce_str(name)
-    context: dict[str, str] = {
+    context: dict[str, Any] = {
         "source": coerce_str(source),
         "source_id": coerce_str(source_id),
         "name": name_value,
@@ -767,7 +931,7 @@ async def seed_compute_center_locations_from_source_coords(
     """Seed stored compute-center locations only from real source coordinates."""
     stmt = (
         select(CollectedData)
-        .where(CollectedData.source.in_(["top500", "epoch_ai_gpu"]))
+        .where(compute_record_filter(CollectedData))
         .where(CollectedData.is_current.is_(True))
     )
     result = await session.execute(stmt)
@@ -783,6 +947,23 @@ async def seed_compute_center_locations_from_source_coords(
         longitude = parse_float(get_record_field(record, "longitude"))
         if latitude in (None, 0.0) or longitude in (None, 0.0):
             continue
+        metadata = record.extra_data or {}
+        precision = "precise"
+        confidence = 1.0
+        needs_confirmation = False
+        source_note = "Seeded from source-provided compute-center coordinates"
+        if source_value == "epoch_ai_gpu":
+            assessment = assess_epoch_location(
+                metadata.get("location"),
+                country=get_record_field(record, "country"),
+                raw_country=metadata.get("raw_country"),
+            )
+            if not assessment.is_renderable:
+                continue
+            precision = assessment.precision
+            confidence = 1.0 if precision == "precise" else 0.9
+            needs_confirmation = precision != "precise"
+            source_note = assessment.reason
         existing = await session.scalar(
             select(ComputeCenterLocationRecord)
             .where(ComputeCenterLocationRecord.source == source_value)
@@ -790,7 +971,6 @@ async def seed_compute_center_locations_from_source_coords(
         )
         if existing:
             continue
-        metadata = record.extra_data or {}
         session.add(
             ComputeCenterLocationRecord(
                 source=source_value,
@@ -802,16 +982,16 @@ async def seed_compute_center_locations_from_source_coords(
                 country=coerce_str(get_record_field(record, "country")) or None,
                 latitude=latitude,
                 longitude=longitude,
-                precision="precise",
-                confidence=1.0,
+                precision=precision,
+                confidence=confidence,
                 location_source="source_coordinates",
-                source_note="Seeded from source-provided compute-center coordinates",
+                source_note=source_note,
                 raw_payload={
                     "record_id": getattr(record, "id", None),
                     "source": source_value,
                     "source_id": source_id,
                 },
-                needs_confirmation=False,
+                needs_confirmation=needs_confirmation,
                 verification_status="source_provided",
                 verified_at=None,
             )

backend/app/services/compute_center_verified_sites.json

@@ -0,0 +1,44 @@
+{
+  "version": 1,
+  "sites": [
+    {
+      "source": "epoch_ai_gpu",
+      "source_ids": ["epoch_jamstec_zettascaler_2_0_gyoukou", "epoch_jamstec_zettascaler_2_2_gyoukou"],
+      "latitude": 35.380924,
+      "longitude": 139.625008,
+      "precision": "site",
+      "location_source": "official_facility_registry",
+      "country": "日本",
+      "city": "Yokohama",
+      "matched_location_name": "JAMSTEC Yokohama Institute for Earth Sciences",
+      "address": "3173-25 Showa-machi, Kanazawa-ku, Yokohama, Kanagawa, Japan",
+      "source_note": "Gyoukou deployment confirmed by JAMSTEC's 2017 release. Campus-level anchor from its official access map center, not a surveyed machine-room or building coordinate.",
+      "verified_at": "2026-09-21T17:06:31Z",
+      "coordinate_basis": "official_access_map_center",
+      "sources": [
+        "https://www.jamstec.go.jp/j/about/press_release/archive/2017/20171114.pdf",
+        "https://www.jamstec.go.jp/e/about/access/yokohama.html"
+      ]
+    },
+    {
+      "source": "epoch_ai_gpu",
+      "source_ids": ["epoch_argonne_nl_sophia__formerly_theta_"],
+      "latitude": 41.718109383287235,
+      "longitude": -87.984498648496,
+      "precision": "site",
+      "location_source": "official_facility_registry",
+      "country": "美国",
+      "city": "Argonne",
+      "matched_location_name": "Argonne Leadership Computing Facility",
+      "address": "9700 South Cass Avenue, Building 240, Argonne, IL 60439, USA",
+      "source_note": "Sophia is an ALCF system. Campus-level anchor from ALCF's official visitor map center; the published address identifies Building 240 but the coordinate is not a surveyed building or rack location.",
+      "verified_at": "2026-09-21T17:06:31Z",
+      "coordinate_basis": "official_access_map_center",
+      "sources": [
+        "https://www.alcf.anl.gov/systems/sophia",
+        "https://docs.alcf.anl.gov/sophia/",
+        "https://www.alcf.anl.gov/about/visiting-alcf"
+      ]
+    }
+  ]
+}

backend/app/services/compute_center_verified_sites.py

@@ -0,0 +1,26 @@
+"""Evidence-backed facility anchors; never match an operator's headquarters by name."""
+
+import json
+from copy import deepcopy
+from functools import lru_cache
+from pathlib import Path
+
+
+@lru_cache(maxsize=1)
+def _site_index() -> dict[tuple[str, str], dict]:
+    payload = json.loads(Path(__file__).with_suffix(".json").read_text(encoding="utf-8"))
+    index = {}
+    for site in payload["sites"]:
+        if not (-90 <= site["latitude"] <= 90 and -180 <= site["longitude"] <= 180):
+            raise ValueError("Invalid verified compute facility coordinate")
+        for source_id in site["source_ids"]:
+            key = (site["source"], source_id)
+            if key in index:
+                raise ValueError("Duplicate verified compute facility identity")
+            index[key] = site
+    return index
+
+
+def get_verified_compute_site(source: str, source_id: str) -> dict | None:
+    site = _site_index().get((source, source_id))
+    return deepcopy(site) if site else None

backend/app/services/compute_configuration.py

@@ -0,0 +1,99 @@
+"""Database overrides with bundled defaults and atomic, revision-checked publishing."""
+
+import hashlib
+from pathlib import Path
+from uuid import uuid4
+
+from app.core.request_context import get_request_id
+from app.models.system_log import AuditLog
+from app.models.system_setting import SystemSetting
+from app.models.user import User
+from app.schemas.compute_configuration import ComputeConfiguration
+from app.services.compute_catalog import COUNTRY_CATALOG
+from fastapi import HTTPException
+from sqlalchemy import select
+from sqlalchemy.dialects.postgresql import insert
+from sqlalchemy.ext.asyncio import AsyncSession
+
+CATEGORY = "compute_comparison"
+DEFAULT_PATH = Path(__file__).with_name("compute_history_defaults.json")
+DEFAULT_CONFIGURATION = ComputeConfiguration.model_validate_json(DEFAULT_PATH.read_text())
+DEFAULT_REVISION = hashlib.sha256(DEFAULT_PATH.read_bytes()).hexdigest()
+
+
+def default_envelope() -> dict:
+    return {
+        "config": DEFAULT_CONFIGURATION.model_dump(mode="json"),
+        "revision": DEFAULT_REVISION,
+        "is_default": True,
+    }
+
+
+async def read_configuration(db: AsyncSession) -> dict:
+    record = (
+        await db.execute(select(SystemSetting).where(SystemSetting.category == CATEGORY))
+    ).scalar_one_or_none()
+    return dict(record.payload) if record else default_envelope()
+
+
+def configuration_response(envelope: dict) -> dict:
+    return {
+        **envelope,
+        "countries": [
+            {key: country[key] for key in ("code", "name", "name_en")}
+            for country in COUNTRY_CATALOG
+        ],
+    }
+
+
+async def publish_configuration(
+    db: AsyncSession,
+    *,
+    revision: str,
+    config: ComputeConfiguration,
+    actor: User,
+    reset: bool = False,
+) -> dict:
+    # The unique category plus row lock also serializes concurrent first writes.
+    await db.execute(
+        insert(SystemSetting)
+        .values(category=CATEGORY, payload=default_envelope())
+        .on_conflict_do_nothing(index_elements=[SystemSetting.category])
+    )
+    record = (
+        await db.execute(
+            select(SystemSetting).where(SystemSetting.category == CATEGORY).with_for_update()
+        )
+    ).scalar_one()
+    if record.payload["revision"] != revision:
+        await db.rollback()
+        raise HTTPException(409, "Configuration changed. Reload before saving your edits.")
+    envelope = (
+        default_envelope()
+        if reset
+        else {
+            "config": config.model_dump(mode="json"),
+            "revision": uuid4().hex,
+            "is_default": reset,
+        }
+    )
+    before = record.payload
+    if reset:
+        await db.delete(record)
+    else:
+        record.payload = envelope
+    # Audit and configuration commit together: a published edit must be attributable.
+    db.add(
+        AuditLog(
+            actor_id=actor.id,
+            actor_name=actor.username,
+            action="compute_comparison.reset" if reset else "compute_comparison.update",
+            target_type="system_setting",
+            target_id=CATEGORY,
+            result="success",
+            request_id=get_request_id(),
+            details={"before": before, "after": envelope},
+        )
+    )
+    await db.commit()
+    return envelope

backend/app/services/compute_history.py

@@ -0,0 +1,122 @@
+"""Versioned, source-backed annual observations for compute comparisons."""
+
+from copy import deepcopy
+from datetime import date
+
+from app.schemas.compute_configuration import ComputeConfiguration
+from app.services.compute_catalog import (
+    CATALOG,
+    DEFAULT_PARTICIPANTS,
+    PARTICIPANTS,
+    participant_members,
+)
+from app.services.compute_configuration import DEFAULT_CONFIGURATION
+
+
+def _participant_value(
+    members: tuple[str, ...], observations: dict[str, float | int], mode: str
+) -> tuple[float | int | None, dict]:
+    covered = [member for member in members if member in observations]
+    missing = [member for member in members if member not in observations]
+    coverage = {
+        "status": "complete" if not missing else "incomplete",
+        "required_members": list(members),
+        "covered_members": covered,
+        "missing_members": missing,
+    }
+    if missing:
+        return None, coverage
+    if mode == "bilateral_share":
+        if (
+            not set(members).issubset({"CN", "US"})
+            or not {"CN", "US"}.issubset(observations)
+            or observations["US"] + observations["CN"] <= 0
+        ):
+            coverage.update(
+                {
+                    "status": "incompatible",
+                    "reason": "bilateral_share_requires_CN_and_US_observations",
+                }
+            )
+            return None, coverage
+        return (
+            round(
+                sum(observations[member] for member in members)
+                / (observations["US"] + observations["CN"])
+                * 100,
+                2,
+            ),
+            coverage,
+        )
+    return sum(observations[member] for member in members), coverage
+
+
+def compute_history(
+    countries: list[str],
+    mode: str = "global_share",
+    config: ComputeConfiguration | None = None,
+) -> dict:
+    """Aggregate only fully covered, same-year compatible observations."""
+    config = config if config is not None else DEFAULT_CONFIGURATION
+    metric = "global_share" if mode == "bilateral_share" else mode
+    annual = {row.year: row for row in config.observations if row.metric == metric}
+    gaps = {row.year: row for row in config.innovation}
+    observed_years = [row.year for row in config.observations] + list(gaps)
+    years = tuple(
+        range(
+            min(observed_years, default=date.today().year),
+            max([date.today().year, *observed_years]) + 1,
+        )
+    )
+    members_by_participant = participant_members(countries)
+    rows = []
+    for year in years:
+        record = annual.get(year)
+        observations = record.values if record else {}
+        values, coverage = {}, {}
+        for code, members in members_by_participant.items():
+            values[code], coverage[code] = _participant_value(members, observations, mode)
+        evidence_id = record.source_id if record else None
+        rows.append(
+            {"year": year, "values": values, "coverage": coverage, "evidence_id": evidence_id}
+        )
+
+    selected_members = {member for members in members_by_participant.values() for member in members}
+    has_innovation_scope = {"CN", "US"}.issubset(selected_members)
+    return {
+        "version": f"compute-history-{config.reviewed_at}",
+        "provenance": "curated_reports",
+        "reviewed_at": config.reviewed_at.isoformat(),
+        "time_basis": "retrospective",
+        "years": years,
+        "mode": mode,
+        "unit": "EFlop/s (FP32)" if mode == "absolute" else "%",
+        "denominator": {"global_share": "global", "bilateral_share": "US+CN", "absolute": None}[
+            mode
+        ],
+        "catalog": deepcopy(CATALOG),
+        "default_participants": list(DEFAULT_PARTICIPANTS),
+        "countries": [deepcopy(PARTICIPANTS[code]) for code in countries],
+        "compute": rows,
+        "innovation": [
+            {
+                "year": year,
+                "value": gaps[year].value if year in gaps and has_innovation_scope else None,
+                "evidence_id": gaps[year].source_id
+                if year in gaps and has_innovation_scope
+                else None,
+            }
+            for year in years
+        ],
+        "evidence": {
+            source.id: source.model_dump(mode="json", exclude={"id"}) for source in config.sources
+        },
+        "innovation_scope": ["US", "CN"],
+        "limitations": [
+            "公开设施样本不代表全国总算力；FP64、FP16 与 FP32 不可直接相加。",
+            "区域值只在全部成员有同年、同指标、可比较观测时汇总；缺失成员时返回 null。",
+            "全球份额不从中美或其他局部成员反推剩余地区。",
+            "年度资料按已配置报告引用展示，观测年与报告年份分列；跨年方法可比性仍需核验。",
+            "创新分差为美国减中国；只有分差，不反推单国得分，不证明算力与创新的因果关系。",
+        ],
+    }

backend/app/services/compute_history_defaults.json

@@ -0,0 +1,125 @@
+{
+  "reviewed_at": "2026-09-22",
+  "sources": [
+    {
+      "id": "caict-2021",
+      "title": "中国算力发展指数白皮书（2021）",
+      "url": "https://www.caict.ac.cn/kxyj/qwfb/bps/202109/P020210918521091309950.pdf",
+      "publisher": "中国信息通信研究院",
+      "report_year": 2021,
+      "published_at": null,
+      "locator": "正文 11 页图 4（2020 年全球份额）"
+    },
+    {
+      "id": "caict-2022",
+      "title": "中国算力发展指数白皮书（2022）",
+      "url": "https://www.caict.ac.cn/kxyj/qwfb/bps/202211/P020221105727522653499.pdf",
+      "publisher": "中国信息通信研究院",
+      "report_year": 2022,
+      "published_at": null,
+      "locator": "正文 13 页图 5（2021 年全球份额）"
+    },
+    {
+      "id": "caict-2023",
+      "title": "中国算力发展指数白皮书（2023）",
+      "url": "https://www.caict.ac.cn/kxyj/qwfb/bps/202309/P020240326630458153765.pdf",
+      "publisher": "中国信息通信研究院",
+      "report_year": 2023,
+      "published_at": null,
+      "locator": "正文 13 页图 5（2022 年全球份额）；16 页（中国总规模）"
+    },
+    {
+      "id": "caict-2024",
+      "title": "先进计算暨算力发展指数蓝皮书（2024年）",
+      "url": "https://www.caict.ac.cn/kxyj/qwfb/bps/202501/P020250117503621662777.pdf",
+      "publisher": "中国信息通信研究院",
+      "report_year": 2024,
+      "published_at": null,
+      "locator": "正文 10 页图 3（2023 年全球份额）；12 页（中国总规模）"
+    },
+    {
+      "id": "innovation-2026",
+      "title": "全球人工智能创新指数报告 2026：科技日报报道",
+      "url": "https://finance.sina.com.cn/tech/roll/2026-07-17/doc-iniickaz8224252.shtml",
+      "publisher": "科技日报（新浪转载）",
+      "report_year": 2026,
+      "published_at": "2026-07-17",
+      "locator": "2023–2025 年美国减中国的指数分差"
+    }
+  ],
+  "observations": [
+    {
+      "year": 2020,
+      "metric": "global_share",
+      "source_id": "caict-2021",
+      "values": {
+        "US": 36,
+        "CN": 31,
+        "JP": 6
+      }
+    },
+    {
+      "year": 2021,
+      "metric": "global_share",
+      "source_id": "caict-2022",
+      "values": {
+        "US": 34,
+        "CN": 33,
+        "JP": 5
+      }
+    },
+    {
+      "year": 2022,
+      "metric": "global_share",
+      "source_id": "caict-2023",
+      "values": {
+        "US": 34,
+        "CN": 33,
+        "JP": 4
+      }
+    },
+    {
+      "year": 2023,
+      "metric": "global_share",
+      "source_id": "caict-2024",
+      "values": {
+        "US": 41,
+        "CN": 31,
+        "JP": 4
+      }
+    },
+    {
+      "year": 2022,
+      "metric": "absolute",
+      "source_id": "caict-2023",
+      "values": {
+        "CN": 302
+      }
+    },
+    {
+      "year": 2023,
+      "metric": "absolute",
+      "source_id": "caict-2024",
+      "values": {
+        "CN": 435
+      }
+    }
+  ],
+  "innovation": [
+    {
+      "year": 2023,
+      "value": 22.02,
+      "source_id": "innovation-2026"
+    },
+    {
+      "year": 2024,
+      "value": 19.96,
+      "source_id": "innovation-2026"
+    },
+    {
+      "year": 2025,
+      "value": 17.95,
+      "source_id": "innovation-2026"
+    }
+  ]
+}

backend/app/services/data_jobs.py

@@ -12,6 +12,7 @@ from sqlalchemy.ext.asyncio import AsyncSession
 
 from app.core.cache import cache
 from app.core.config import settings
+from app.core.enums import JobStatus, JobType, RollbackPolicy
 from app.core.logging import get_logger
 from app.core.time import to_iso8601_utc
 from app.core.websocket.broadcaster import broadcaster
@@ -36,17 +37,17 @@ from app.services.scheduler import sync_datasource_job
 
 logger = get_logger(__name__)
 
-JOB_TYPE_COLLECT = "collect"
-JOB_TYPE_CLEAR_DATA = "clear_data"
-JOB_TYPE_CLEAR_CACHE = "clear_cache"
-JOB_TYPE_EARTH_REFRESH = "earth_refresh"
+JOB_TYPE_COLLECT = JobType.COLLECT.value
+JOB_TYPE_CLEAR_DATA = JobType.CLEAR_DATA.value
+JOB_TYPE_CLEAR_CACHE = JobType.CLEAR_CACHE.value
+JOB_TYPE_EARTH_REFRESH = JobType.EARTH_REFRESH.value
 
-JOB_STATUS_QUEUED = "queued"
-JOB_STATUS_RUNNING = "running"
-JOB_STATUS_CANCELLING = "cancelling"
-JOB_STATUS_SUCCESS = "success"
-JOB_STATUS_FAILED = "failed"
-JOB_STATUS_CANCELLED = "cancelled"
+JOB_STATUS_QUEUED = JobStatus.QUEUED.value
+JOB_STATUS_RUNNING = JobStatus.RUNNING.value
+JOB_STATUS_CANCELLING = JobStatus.CANCELLING.value
+JOB_STATUS_SUCCESS = JobStatus.SUCCESS.value
+JOB_STATUS_FAILED = JobStatus.FAILED.value
+JOB_STATUS_CANCELLED = JobStatus.CANCELLED.value
 
 ACTIVE_JOB_STATUSES = (JOB_STATUS_QUEUED, JOB_STATUS_RUNNING, JOB_STATUS_CANCELLING)
 TERMINAL_JOB_STATUSES = (JOB_STATUS_SUCCESS, JOB_STATUS_FAILED, JOB_STATUS_CANCELLED)
@@ -54,6 +55,9 @@ DATA_WRITE_JOB_TYPES = (JOB_TYPE_COLLECT, JOB_TYPE_CLEAR_DATA, JOB_TYPE_CLEAR_CA
 SOURCE_LOCK_JOB_STATUSES = (JOB_STATUS_RUNNING, JOB_STATUS_CANCELLING)
 QUEUE_POLL_SECONDS = 0.35
 JOB_STALE_LOCK_MINUTES = 90
+ORPHAN_CANCELLING_GRACE_SECONDS = 30
+JOB_RECOVERY_SWEEP_SECONDS = 15
+DATA_DELETE_BATCH_SIZE = 50_000
 DEFAULT_WORKER_CONCURRENCY = 2
 
 RUNNING_DATA_JOB_TASKS: dict[int, asyncio.Task[Any]] = {}
@@ -77,7 +81,7 @@ async def enqueue_datasource_job(
     task_type: str,
     *,
     payload: dict[str, Any] | None = None,
-    rollback_policy: str = "keep_committed_batches",
+    rollback_policy: str = RollbackPolicy.KEEP_COMMITTED_BATCHES.value,
     dedupe_key: str | None = None,
 ) -> CollectionTask:
     if dedupe_key:
@@ -281,6 +285,7 @@ class DataJobWorker:
         self._task: asyncio.Task[None] | None = None
         self._stop_event: asyncio.Event | None = None
         self._running: set[asyncio.Task[Any]] = set()
+        self._last_recovery_sweep_at: datetime | None = None
 
     def start(self) -> None:
         if self._task and not self._task.done():
@@ -303,6 +308,11 @@ class DataJobWorker:
         await self._recover_stale_running_jobs()
         while not self._stop_event.is_set():
             self._running = {task for task in self._running if not task.done()}
+            if (
+                self._last_recovery_sweep_at is None
+                or (_utcnow() - self._last_recovery_sweep_at).total_seconds() >= JOB_RECOVERY_SWEEP_SECONDS
+            ):
+                await self._recover_stale_running_jobs()
             if len(self._running) >= self.concurrency:
                 await asyncio.sleep(QUEUE_POLL_SECONDS)
                 continue
@@ -316,7 +326,9 @@ class DataJobWorker:
             self._running.add(runner)
 
     async def _recover_stale_running_jobs(self) -> None:
+        self._last_recovery_sweep_at = _utcnow()
         cutoff = _utcnow() - timedelta(minutes=JOB_STALE_LOCK_MINUTES)
+        orphan_cancelling_cutoff = _utcnow() - timedelta(seconds=ORPHAN_CANCELLING_GRACE_SECONDS)
         async with async_session_factory() as db:
             result = await db.execute(
                 select(CollectionTask)
@@ -332,6 +344,21 @@ class DataJobWorker:
                 job.error_message = "Marked failed after stale data job lock timeout"
             if stale_jobs:
                 await db.commit()
+            orphan_result = await db.execute(
+                select(CollectionTask)
+                .where(CollectionTask.status == JOB_STATUS_CANCELLING)
+                .where(CollectionTask.locked_at.is_(None))
+                .where(CollectionTask.requested_cancel_at.is_not(None))
+                .where(CollectionTask.requested_cancel_at < orphan_cancelling_cutoff)
+            )
+            for job in orphan_result.scalars().all():
+                if job.id in RUNNING_DATA_JOB_TASKS:
+                    continue
+                await _cancel_task_without_runner(
+                    db,
+                    job,
+                    reason=job.cancel_reason or "cancelled_after_orphaned_runner",
+                )
 
     async def _claim_next_job(self) -> int | None:
         async with async_session_factory() as db:
@@ -477,15 +504,22 @@ async def _run_clear_data_job(db: AsyncSession, task: CollectionTask) -> None:
     await db.commit()
     await _broadcast_task_update(task)
 
-    count_result = await db.execute(
-        select(CollectedData.id).where(CollectedData.source == source)
+    deleted_count = await _delete_table_rows_by_source(
+        db,
+        task,
+        table_name="collected_data",
+        source_column="source",
+        source=source,
+    )
+    derived_deleted_counts = await _clear_derived_datasource_data_in_batches(
+        db,
+        task,
+        source,
+        progress_offset=deleted_count,
     )
-    collected_ids = [row[0] for row in count_result.all()]
-    derived_deleted_counts = await clear_derived_datasource_data(db, source)
-    if collected_ids:
-        await db.execute(CollectedData.__table__.delete().where(CollectedData.id.in_(collected_ids)))
-    deleted_count = len(collected_ids)
     derived_deleted_count = sum(derived_deleted_counts.values())
+    if any(key.startswith("ais_") for key in derived_deleted_counts):
+        await db.execute(text("ANALYZE ais_raw_observations"))
 
     task.records_processed = deleted_count + derived_deleted_count
     task.total_records = task.records_processed
@@ -504,10 +538,99 @@ async def _run_clear_data_job(db: AsyncSession, task: CollectionTask) -> None:
     task.phase = "completed"
     task.phase_message = "数据库数据已清理"
     task.completed_at = _utcnow()
+    datasource = await db.get(DataSource, task.datasource_id)
+    if datasource is not None:
+        datasource.last_status = JOB_STATUS_SUCCESS
+        datasource.last_run_at = task.completed_at
+    await db.execute(
+        DataSnapshot.__table__.update()
+        .where(DataSnapshot.source == source)
+        .values(is_current=False)
+    )
     await db.commit()
     await _broadcast_task_update(task)
 
 
+async def _delete_table_rows_by_source(
+    db: AsyncSession,
+    task: CollectionTask,
+    *,
+    table_name: str,
+    source_column: str,
+    source: str,
+    progress_offset: int = 0,
+) -> int:
+    deleted = 0
+    while True:
+        result = await db.execute(
+            text(
+                f"""
+                WITH doomed AS (
+                    SELECT ctid
+                    FROM {table_name}
+                    WHERE {source_column} = :source
+                    LIMIT :batch_size
+                ),
+                deleted_rows AS (
+                    DELETE FROM {table_name}
+                    USING doomed
+                    WHERE {table_name}.ctid = doomed.ctid
+                    RETURNING 1
+                )
+                SELECT COUNT(*) FROM deleted_rows
+                """
+            ),
+            {"source": source, "batch_size": DATA_DELETE_BATCH_SIZE},
+        )
+        batch_deleted = max(int(result.scalar_one() or 0), 0)
+        if batch_deleted <= 0:
+            break
+        deleted += batch_deleted
+        task.records_processed = progress_offset + deleted
+        task.phase_current = task.records_processed
+        task.phase_unit = "records"
+        task.phase_message = f"正在删除数据：{task.records_processed} 条"
+        await db.commit()
+        await _broadcast_task_update(task)
+    return deleted
+
+
+async def _clear_derived_datasource_data_in_batches(
+    db: AsyncSession,
+    task: CollectionTask,
+    source: str,
+    progress_offset: int = 0,
+) -> dict[str, int]:
+    deleted_counts: dict[str, int] = {}
+    if source in {"barentswatch_vessels", "aisstream_vessels"}:
+        deleted_counts["ais_conflict_records"] = await _delete_table_rows_by_source(
+            db,
+            task,
+            table_name="ais_conflict_records",
+            source_column="selected_source",
+            source=source,
+            progress_offset=progress_offset + sum(deleted_counts.values()),
+        )
+        deleted_counts["ais_source_health"] = await _delete_table_rows_by_source(
+            db,
+            task,
+            table_name="ais_source_health",
+            source_column="source",
+            source=source,
+            progress_offset=progress_offset + sum(deleted_counts.values()),
+        )
+        deleted_counts["ais_raw_observations"] = await _delete_table_rows_by_source(
+            db,
+            task,
+            table_name="ais_raw_observations",
+            source_column="source",
+            source=source,
+            progress_offset=progress_offset + sum(deleted_counts.values()),
+        )
+        return deleted_counts
+    return await clear_derived_datasource_data(db, source)
+
+
 async def _run_clear_cache_job(db: AsyncSession, task: CollectionTask) -> None:
     source = str(task.source or (task.payload or {}).get("source") or "").strip()
     if not source:

backend/app/services/datasource_connectivity.py

@@ -13,6 +13,7 @@ from sqlalchemy import func, select
 
 from app.core.data_sources import get_data_sources_config
 from app.core.datasource_defaults import DEFAULT_DATASOURCES
+from app.core.enums import JobStatus
 from app.models.collected_data import CollectedData
 from app.models.datasource import DataSource
 from app.models.datasource_config import DataSourceConfig
@@ -398,7 +399,7 @@ async def has_collected_data(db, source: str) -> bool:
 
     datasource_result = await db.execute(select(DataSource).where(DataSource.source == source))
     datasource = datasource_result.scalar_one_or_none()
-    return bool(datasource and datasource.last_status == "success")
+    return bool(datasource and datasource.last_status == JobStatus.SUCCESS.value)
 
 
 async def get_builtin_connection_status(

backend/app/services/docs_gatekeeper.py

@@ -6,6 +6,7 @@ from dataclasses import dataclass
 from pathlib import Path
 from typing import Literal
 
+from app.core.enums import UserRole
 from app.models.user import User
 
 DocsAccess = Literal["public", "docs_user", "docs_developer", "docs_admin"]
@@ -43,7 +44,9 @@ DOCS_METADATA: tuple[DocsMetadata, ...] = (
     DocsMetadata("earth-satellite-footprint-policy.md", "earth-satellite-footprint-policy", "docs_developer", "Earth", 13, "智能星球卫星覆盖策略", "Intelligent Planet Satellite Footprint Policy"),
     DocsMetadata("earth-bgp-context.md", "earth-bgp-context", "docs_developer", "Earth", 14, "BGP 态势上下文", "BGP Context"),
     DocsMetadata("earth-interactable-usage.md", "earth-interactable-usage", "docs_developer", "Earth", 16, "智能星球可交互图标接入", "Intelligent Planet Interactable Usage"),
-    DocsMetadata("earth-toolbar-overlay-coordination.md", "earth-toolbar-overlay-coordination", "docs_developer", "Earth", 17, "智能星球工具栏与浮层协同", "Intelligent Planet Toolbar and Overlay Coordination"),
+    DocsMetadata("earth-interactable-clustering.md", "earth-interactable-clustering", "docs_developer", "Earth", 17, "智能星球可交互图标聚类策略", "Intelligent Planet Interactable Clustering"),
+    DocsMetadata("earth-toolbar-overlay-coordination.md", "earth-toolbar-overlay-coordination", "docs_developer", "Earth", 18, "智能星球工具栏与浮层协同", "Intelligent Planet Toolbar and Overlay Coordination"),
+    DocsMetadata("earth-news-sources.md", "earth-news-sources", "docs_developer", "Earth", 19, "智能星球新闻源配置", "Intelligent Planet News Source Configuration"),
     DocsMetadata("frontend-admin-frontend-context.md", "frontend-admin-frontend-context", "docs_developer", "Frontend", 20, "控制台前端结构", "Admin Frontend Context"),
     DocsMetadata("frontend-layout-guidelines.md", "frontend-layout-guidelines", "docs_developer", "Frontend", 21, "前端布局指南", "Frontend Layout Guidelines"),
     DocsMetadata("tactile-ui-components.md", "tactile-ui-components", "docs_developer", "Frontend", 24, "Tactile UI 组件库", "Tactile UI Components"),
@@ -52,6 +55,7 @@ DOCS_METADATA: tuple[DocsMetadata, ...] = (
     DocsMetadata("datasource-collector-settings-connectivity.md", "datasource-collector-settings-connectivity", "docs_developer", "Backend", 32, "数据源、采集器设置与连接验证", "Datasource Collector Settings and Connectivity"),
     DocsMetadata("backend-datasources-api-performance.md", "backend-datasources-api-performance", "docs_developer", "Backend", 33, "数据源 API 性能", "Datasource API Performance"),
     DocsMetadata("data-job-earth-sync-architecture.md", "data-job-earth-sync-architecture", "docs_developer", "Backend", 34, "数据作业与 Outbox 技术架构", "Data Jobs and Outbox Architecture"),
+    DocsMetadata("backend-enum-contracts.md", "backend-enum-contracts", "docs_developer", "Backend", 35, "后端枚举与字符串兼容契约", "Backend Enum and String Compatibility Contract"),
     DocsMetadata("location-pipeline-development.md", "location-pipeline-development", "docs_developer", "Backend", 35, "通用位置估算管线开发说明", "Shared Location Resolution Pipeline Development Guide"),
     DocsMetadata("earth-news-live-streams-collector-format.md", "earth-news-live-streams-collector-format", "docs_developer", "Backend", 36, "新闻直播采集格式", "News Live Streams Collector Format"),
     DocsMetadata("docs-gatekeeper-development.md", "docs-gatekeeper-development", "docs_developer", "Backend", 37, "Docs Gatekeeper 开发说明", "Docs Gatekeeper Development Guide"),
@@ -69,9 +73,9 @@ def get_user_gatekeeper_groups(user: User | None) -> set[str]:
         return set()
 
     role = user.role.value if hasattr(user.role, "value") else str(user.role or "")
-    if role == "super_admin":
+    if role == UserRole.SUPER_ADMIN.value:
         return {"docs_user", "docs_developer", "docs_admin"}
-    if role == "admin":
+    if role == UserRole.ADMIN.value:
         return {"docs_user", "docs_developer", "docs_admin"}
 
     groups = set()