release: bump version to 0.76.2

.claude/commands/cleanup.md

@@ -1,139 +0,0 @@
----
-description: 审查当前工作区未提交代码中的垃圾代码，并在不影响逻辑的前提下自动清理
-argument-hint: 可选：指定要检查的文件或目录（默认检查所有未提交修改）
-allowed-tools: ["Read", "Edit", "Bash", "Grep", "Glob"]
----
-
-# /cleanup — 垃圾代码审查与清理
-
-分析当前工作区（git diff）中的未提交代码，找出并修复常见垃圾代码，**不得改变任何运行逻辑**。
-
-## 检查范围
-
-若 `$ARGUMENTS` 非空，则只检查指定文件/目录；否则检查所有未提交修改（`git diff HEAD`）。
-
-## 节省上下文规则
-
-优先用确定性的 CLI 检查缩小范围，不要一上来把完整文件或大 diff 读入上下文：
-
-```bash
-git diff --name-only HEAD
-git diff --unified=0 HEAD -- <path>
-git diff --check
-rg -n "TODO|FIXME|console\.log|debugger|print\(" <changed-paths>
-```
-
-只有 focused diff 不足以安全判断或修改时，才读取完整文件。
-
-## 审查清单
-
-按优先级检查以下问题（只报告在本次 diff 中**新增或修改**的代码里存在的问题）：
-
-### 1. 重复逻辑 (Duplicate Logic)
-- 完全相同或高度相似的代码块在多处出现
-- 同一函数/方法被多个地方各自实现，已有公共版本未被复用
-- 相同的 DOM 查询、正则、模板字符串在同一文件重复
-
-### 2. Magic Numbers / Magic Strings
-- 裸数字直接参与计算（如偏移量、时间、尺寸、阈值），没有命名常量
-- 硬编码字符串（如 id 名、状态值、URL 片段）散落在逻辑中
-- 例外：`0`, `1`, `-1`, `100`, `""` 等语义明确的惯用值不算
-
-### 3. 命名问题
-- 含义不明的缩写变量（如 `or_`, `tmp2`, `x2`）
-- 命名与实际用途不符
-- 同一概念在不同地方用不同名字表达
-
-### 4. 死代码 / 无效代码
-- 注释掉的旧代码块（3行以上）
-- 声明后从未使用的变量/参数/导入
-- 永远不会执行的条件分支
-
-### 5. 代码风格问题
-- 尾部空白字符（trailing whitespace）
-- 同一文件内风格不一致（如混用单双引号、缩进不统一）
-- 空行使用不一致（连续多个空行等）
-
-### 6. 其他常见问题
-- 私有辅助函数应被 export 但没有，导致调用方重复实现
-- 类型/接口重复定义
-- 过于冗长的条件表达式可以简化（不改逻辑）
-
-## 执行步骤
-
-### Step 1 — 获取待检查文件列表
-
-```bash
-# 无参数时：获取所有未提交修改
-git diff HEAD --name-only
-
-# 有参数时：用 $ARGUMENTS 过滤
-```
-
-### Step 2 — 逐文件阅读并分析
-
-先从 focused diff 开始：
-
-```bash
-git diff --unified=0 HEAD -- <file>
-```
-
-用 `rg`、`git diff --check`、编译器或 linter 输出确认确定性问题。只有需要上下文时才用 Read 读取完整文件。对照审查清单，记录每个问题：文件名、行号、问题类型、建议修复方式。
-
-### Step 3 — 报告问题清单
-
-在修改前，先以列表形式输出所有发现的问题：
-
-```
-发现 N 个问题：
-
-[文件] js/foo.js
-  · L34, L78: 重复逻辑 — 两处都实现了相同的 DOM 查询，可提取到 getPanel()
-  · L91: Magic number — 硬编码 14 作为偏移量，应命名为 TOOLTIP_OFFSET
-
-[文件] js/bar.js
-  · L12: 命名问题 — 变量 `or_` 语义不明，应命名为 outerR/outerG/outerB
-  ...
-```
-
-如果没有发现问题，直接输出"未发现垃圾代码，当前代码质量良好。"并停止。
-
-### Step 4 — 执行修复
-
-对每个问题，使用 Edit 工具进行**最小化修改**：
-
-- **重复逻辑**：提取为共享常量/函数，更新所有调用点
-- **Magic number**：在文件顶部或逻辑附近声明 `const NAME = value`，替换所有引用
-- **命名问题**：重命名变量，更新所有使用处
-- **死代码**：直接删除
-- **尾部空白/风格**：修正
-- **未 export 的函数**：添加 `export`，在调用方改为导入（不重复实现）
-
-**修复原则：**
-- 只改在审查清单中发现的问题，不做额外优化
-- 每次 Edit 只修改确实有问题的行，保持 diff 最小
-- 改完后用 `grep` 验证旧的坏代码已消失
-- 优先做精确补丁；只有仓库已有对应格式化流程时，才运行格式化工具
-
-### Step 5 — 输出总结
-
-```
-清理完成：
-
-修复了 N 个问题：
-  ✓ earth.js — 提取重复 vertexShader 为 ATMOS_VERTEX_SHADER 常量
-  ✓ main.js — 提取 TOOLTIP_CURSOR_OFFSET = 14（4处引用）
-  ✓ controls.js — export updateLayerButtonState，移除 main.js 中的重复实现
-  ...
-
-未修改的问题（需人工确认）：
-  ! foo.js L45 — 注释代码块较长，建议手动确认是否可删除
-```
-
-## 约束
-
-- **禁止**改变函数签名、接口定义、导出 API（除非问题正是私有函数应被 export）
-- **禁止**添加新功能、新抽象、新参数
-- **禁止**修改注释内容（只删除注释掉的死代码）
-- **禁止**修改测试文件逻辑
-- 如果一个 Magic number 的语义不完全确定，**跳过**，在总结中标记为"需人工确认"

.claude/commands/docs.md

@@ -1,104 +0,0 @@
----
-description: Create or update repository documentation from current code changes
-argument-hint: Optional: topic to document, or leave empty to infer from git diff
-allowed-tools: ["Read", "Edit", "Write", "Bash", "Glob", "Grep"]
----
-
-# /docs — Documentation Workflow
-
-## Goal
-
-Create or update documentation that explains why a change exists, how it behaves, and what maintainers need to know. Keep this command generic. Repository-specific coverage rules live in the repository and must be loaded separately.
-
-## Repository Rules
-
-Before deciding scope, check whether the repository has a documentation rules file:
-
-```bash
-test -f docs/documentation-coverage-rules.md && sed -n '1,240p' docs/documentation-coverage-rules.md
-```
-
-If it exists, apply it as the project-specific coverage checklist. If it does not exist, continue with the generic workflow below.
-
-## Workflow
-
-### Step 1 — Understand The Change
-
-```bash
-git diff HEAD --stat
-git diff HEAD --name-only
-git log --oneline -10
-rg --files docs
-```
-
-If `$ARGUMENTS` specifies a topic, focus on that topic. Otherwise infer the documentation topic from the changed files. Do not read the full repository diff by default; inspect focused files only:
-
-```bash
-git diff HEAD -- <path>
-rg -n "class |def |function |export |router|@router|interface |type " <path>
-```
-
-### Step 2 — Decide Scope
-
-- Prefer updating an existing relevant document over creating a duplicate.
-- Use one document for one coherent topic.
-- Split documents only when the change crosses meaningful domains.
-- Keep filenames lowercase and hyphenated.
-- Apply the repository-specific rules file before writing.
-
-#### Document Audience Routing (Planet)
-
-In this repository, classify the action's performer before picking a target file:
-
-- Browser/UI end user → `docs/technical/{zh,en}/manual.md` or `quickstart.md`.
-- Shell / Docker / log paths / `planet.sh` / SMTP fallbacks / port forwarding → `docs/technical/{zh,en}/ops-runbook.md` (or an existing `ops-*.md`).
-- Second-party developers → existing `*-context.md` / `backend-*.md` / `earth-*.md` files.
-
-Never put shell commands, log paths, or Docker operations into `manual.md` / `quickstart.md`. Never put UI button labels or screenshots into `ops-*.md`. When the same action has both a UI and a CLI path, write each in its own home and cross-link them with one sentence.
-
-For ambiguous or large documentation changes, briefly state the intended doc plan before editing. For clear small changes, proceed directly.
-
-### Step 3 — Write
-
-Explain:
-
-- Background/problem: what was wrong or missing before.
-- Core design decisions and rationale.
-- Operational or user-facing impact.
-- Relevant code paths, only when useful for future maintainers.
-
-Style:
-
-- Follow the repository’s existing language and heading conventions.
-- Use fenced code blocks with language tags.
-- Prefer tables for comparisons or parameter lists.
-- Keep snippets concise and relevant.
-- For UI labels, chart labels, feature names, datasource names, and other terms that may become mixed Chinese/English copy, check `docs/technical/{zh,en}/naming-glossary.md` and use the documented display name. If a confusing term is missing, update the glossary in both languages as part of the docs change.
-
-### Step 4 — Verify
-
-- Read the completed docs once for clarity and stale statements.
-- Verify referenced paths exist with `test -e` or `rg --files`.
-- Run applicable checks from `docs/documentation-coverage-rules.md`.
-- Check Markdown links use readable user-facing titles unless repository rules allow otherwise.
-
-### Step 5 — Report
-
-Summarize changed docs and verification:
-
-```md
-Updated:
-- path/to/doc.md — what changed
-
-Verified:
-- checks that passed
-- checks that could not be run, if any
-```
-
-## Hard Constraints
-
-- Do not leave placeholder docs.
-- Do not duplicate bilingual files byte-for-byte.
-- Do not reference PR numbers, issue numbers, or the current conversation unless explicitly requested.
-- Do not write changelog-style lists without the reasoning and tradeoffs behind the change.
-- Keep docs maintainable and concise.

.claude/commands/goal-driven.md

@@ -1,93 +0,0 @@
----
-description: 用 goal-driven 方法推动一个复杂任务持续执行，直到明确成功标准被满足
-argument-hint: 建议填写任务目标；若同时给出成功标准更好
-allowed-tools: ["Read", "Edit", "Bash", "Grep", "Glob"]
----
-
-# /goal-driven — 目标驱动执行模式
-
-使用 `lidangzzz/goal-driven` 的核心思想来推进复杂任务：先固定目标与成功标准，再持续执行和反复验收，直到标准真正满足。
-
-适用场景：
-
-- 长周期实现任务
-- 高复杂度工程任务
-- 可被明确验收的研究、实现、迁移、验证类工作
-
-不适用场景：
-
-- 纯脑暴
-- 无法定义成功标准的模糊任务
-- 很小的一次性修改
-
-## 输入要求
-
-若 `$ARGUMENTS` 只包含目标，没有成功标准，先补全一版可执行的成功标准再开始。
-
-启动时先输出：
-
-```md
-Goal
-- ...
-
-Criteria for success
-- ...
-
-Plan
-1. ...
-2. ...
-3. ...
-
-Verification
-- ...
-```
-
-## 执行规则
-
-1. 先把任务固化为两个核心块：
-   - `Goal`
-   - `Criteria for success`
-
-2. 成功标准必须尽量客观，可验证，可落地。
-   优先写成：
-   - 需要交付什么
-   - 需要通过哪些测试或验证
-   - 如何判断结果真的完成
-
-3. 进入持续执行循环：
-   - 完成一个阶段
-   - 检查当前结果是否满足成功标准
-   - 若未满足，明确剩余差距并继续推进
-
-4. 任何“完成了”“差不多了”“已实现”之类的结论，都必须经过验证，不能直接接受。
-
-5. 如果验证失败：
-   - 明确指出哪条成功标准没满足
-   - 继续工作，不要把阶段性进展误判为完成
-
-6. 只有在以下情况之一才能停止：
-   - 成功标准已满足
-   - 用户明确要求停止
-
-## 执行风格
-
-- 重证据，轻口头判断
-- 优先使用确定性工具证据：`rg`、`git diff --stat`、`git diff -- <path>`、测试、构建、lint、`curl`、数据库查询等能直接证明成功标准的方式
-- 不把大段命令输出粘进回复；保留在工具调用里，回复只总结关键证据
-- 重验收，轻自我感觉
-- 优先用测试、日志、产物、对比结果来证明完成
-- 对长期任务保持“未达标就继续”的节奏
-
-## 简版模板
-
-```md
-Goal: [[[[[在此填写最终目标]]]]]
-
-Criteria for success: [[[[[在此填写成功标准]]]]]
-
-循环执行：
-1. 推进任务
-2. 检查是否满足成功标准
-3. 若未满足，继续工作
-4. 直到满足标准或用户明确停止
-```

.claude/commands/release.md

@@ -1,160 +0,0 @@
----
-description: 发版工作流：根据变更类型决定版本号，更新所有版本文件和 changelog，运行验证，commit 并 push
-argument-hint: 可选：feature | bugfix | 或直接描述本次发布内容
-allowed-tools: ["Read", "Edit", "Bash", "Glob", "Grep"]
----
-
-# /release — Planet 发版工作流
-
-## 版本号规则
-
-| 变更类型 | 版本跳动 | 适用场景 |
-|---------|---------|---------|
-| `feature` | `+0.1.0` | 纯新功能，无 bugfix |
-| `improvement` | `+0.0.1` | UI 调整、小功能增强、bugfix 混合，或以 UI/体验改进为主的迭代 |
-| `bugfix` | `+0.0.1` | 纯 bug 修复，无新功能 |
-| `docs` / `maintenance` / `refactor` | 默认不发版，除非用户明确要求 |
-
-意图混合时以用户明确描述为准；bugfix + 小 feature 混合默认判定为 `improvement`（`+0.0.1`）。
-
-## 必须同步更新的文件
-
-使用 `git rev-parse --show-toplevel` 获取仓库根目录，以下路径均相对于根目录：
-
-- `VERSION`
-- `frontend/package.json`（`"version"` 字段）
-- `pyproject.toml`（`version =` 字段）
-- `uv.lock`（**不要手动编辑**，通过 `uv lock` 重新生成）
-- `docs/CHANGELOG.md`
-- `docs/version-history.md`
-
-## 节省上下文规则
-
-发版判断应以确定性 CLI 证据为主，优先使用紧凑命令和定点读取：
-
-```bash
-git status --short
-git diff --stat HEAD
-git diff --name-only HEAD
-rg -n "version|^## |^Released:|当前开发版本|current" VERSION frontend/package.json pyproject.toml docs/CHANGELOG.md docs/version-history.md
-```
-
-除非需要判断某个代码变更是否属于本次发版，否则不要读取完整 diff。
-
-## 执行步骤
-
-### Step 1 — 环境检查
-
-```bash
-git branch --show-current          # 确认在 dev 分支
-git status --short                 # 检查是否有无关的未暂存修改
-cat VERSION                        # 读取当前版本
-```
-
-若当前**不在 `dev` 分支**，停下来告知用户，不要继续。
-
-若存在无关的未暂存修改，列出并询问用户是否一并提交，或先 stash。
-
-### Step 2 — 确定发版类型与新版本号
-
-- 若 `$ARGUMENTS` 提供了明确类型（`feature` / `bugfix`），直接使用
-- 否则根据 `git diff --stat HEAD`、`git diff --name-only HEAD`、必要的 focused diff 和 `git log` 推断
-- 计算新版本号（例：`0.26.2` → bugfix → `0.26.3`）
-- **先输出发版计划供用户确认**：
-
-```
-发版计划：
-  类型：bugfix
-  版本：0.26.2 → 0.26.3
-  分支：dev
-  将更新：VERSION, frontend/package.json, pyproject.toml, uv.lock, CHANGELOG.md, version-history.md
-```
-
-### Step 3 — 更新版本号文件
-
-按顺序更新（每步用 Edit 工具，精确替换，不要重写整个文件）：
-
-1. `VERSION` — 直接替换全部内容为新版本号
-2. `frontend/package.json` — 替换 `"version": "x.x.x"` 行
-3. `pyproject.toml` — 替换 `version = "x.x.x"` 行
-4. 运行 `uv lock` 重新生成 `uv.lock`（在仓库根目录下执行）
-
-### Step 4 — 更新 CHANGELOG.md
-
-在文件顶部插入新条目，格式：
-
-```markdown
-## [x.x.x] — YYYY-MM-DD
-
-### ✨ Features / 🐛 Fixes / 🔧 Improvements
-- ...（只列高信号条目，最多 5 条）
-- ...
-
----
-```
-
-日期使用 `date +%Y-%m-%d` 获取今天的日期。
-
-### Step 5 — 更新 docs/version-history.md
-
-- 更新文件头部的"当前开发版本"字段
-- 在时间线表格顶部插入新行：`| vx.x.x | YYYY-MM-DD | 一句话摘要 |`
-
-### Step 6 — 验证
-
-针对本次变更范围做最小验证：
-
-- Python 文件有修改：先用 `git diff --name-only HEAD -- '*.py'` 列出，再运行 `python3 -m py_compile <changed_files>`
-- Frontend 文件有修改：先用 `git diff --name-only HEAD -- frontend` 判断范围，再运行项目标准检查（若无则跳过并说明）
-- 版本号一致性检查：用 grep 确认 VERSION、package.json、pyproject.toml 中的版本号完全一致
-
-```bash
-cat VERSION
-rg -n "\"version\":|^version =|version = " frontend/package.json pyproject.toml uv.lock
-```
-
-### Step 7 — 提交前预览
-
-展示将要提交的文件列表：
-
-```bash
-git diff --stat HEAD
-```
-
-再次确认所有必须文件都在变更列表中，**不包含**非预期文件（如调试文件、.env 等）。
-
-### Step 8 — Commit & Push（用户确认后）
-
-```bash
-git add VERSION frontend/package.json pyproject.toml uv.lock docs/CHANGELOG.md docs/version-history.md
-# 若有代码变更也一并 stage
-git add <code_files>
-
-git commit -m "release: bump version to x.x.x"
-git tag vx.x.x
-git push origin dev
-git push origin vx.x.x
-```
-
-commit message 固定格式：`release: bump version to x.x.x`
-
-### Step 9 — 完成确认
-
-输出摘要：
-
-```
-✓ 版本号已更新：0.26.2 → 0.26.3
-✓ CHANGELOG 已更新
-✓ version-history 已更新
-✓ uv.lock 已重新生成
-✓ 验证通过
-✓ commit: release: bump version to 0.26.3
-✓ tag: v0.26.3
-✓ 已 push 到 origin/dev
-```
-
-## 注意事项
-
-- `uv.lock` 只能通过 `uv lock` 生成，绝不手动编辑
-- 发版 commit 只包含版本文件 + 本次功能代码，不混入无关改动
-- 若环境中 `uv` 不可用，说明原因并跳过 lockfile 更新，提醒用户手动运行

.dockerignore

@@ -3,8 +3,23 @@
 !pyproject.toml
 !uv.lock
 !VERSION
+!frontend/
+!frontend/public/
+!frontend/public/earth/
+!frontend/public/earth/assets/
+!frontend/public/earth/assets/cards/
+!frontend/public/earth/assets/cards/manifest.json
 !backend/
 !backend/**
+!config/
+!config/earth-boundary-pov-policy.china-v1.json
+!scripts/
+!scripts/build_earth_boundary_china_pov_geojson.py
+!scripts/build_earth_boundary_pmtiles.py
+!scripts/earth_boundary_policy.py
+!resources/
+!resources/earth/
+!resources/earth/boundary-pov-geometry.china-v1.json
 !aiprovider/
 !aiprovider/**
 

.gitignore

@@ -25,7 +25,10 @@ __pycache__/
 build/
 develop-eggs/
 dist/
-downloads/
+downloads/*
+!downloads/usbipd-win/
+downloads/usbipd-win/*
+!downloads/usbipd-win/usbipd-win-5.3.0.msi
 eggs/
 .eggs/
 /lib/
@@ -157,3 +160,8 @@ data/earth-boundary-sources/
 # Git; publish PMTiles/MVT artifacts through release/deploy storage instead of
 # committing thousands of generated tile files.
 frontend/public/earth/data/boundaries/
+
+# Offline comparison originals; runtime 8K maps and provenance remain versioned.
+resources/earth/candidates/nasa-blue-marble/*.jpg
+resources/earth/candidates/natural-earth-3/*.jpg
+*:Zone.Identifier

AGENTS.md

@@ -0,0 +1,183 @@
+# AGENTS.md
+
+**Planet agent harness. Defines behavior for coding agents working in this repository.**
+
+---
+
+## Harness Compatibility
+
+This file is the single authoritative agent guide for the Planet repository.
+The older lowercase `agents.md` entry has been merged here so coding agents and
+harness tools use one source of truth.
+
+### Source Of Truth
+
+- `rules.md` is the mandatory repository rule source. Always load `core`,
+  `security`, and `workflow`; load only task-relevant modules after that.
+- `AGENTS.md` defines the local agent operating mode and evidence gates.
+- `project_context.md` is background, not a rule source. Prefer newer
+  implementation docs when it disagrees with current code.
+- `.codex/skills/` is the active specialized workflow layer for cleanup, docs,
+  goal-driven work, and release.
+- Do not duplicate long workflow text across harness files. Durable constraints
+  belong in `rules.md`; task procedures belong in skills or scripts.
+
+Read these files before changing code:
+
+1. `rules.md`
+2. `AGENTS.md`
+3. `project_context.md`
+4. `README.md`
+5. `docs/HARNESS.md`
+6. `CODEMAP.md`
+
+For documentation work, also read `docs/documentation-coverage-rules.md`.
+
+### Start Safely
+
+Before broad edits:
+
+```bash
+git status --short
+scripts/harness/doctor.sh
+```
+
+Use focused context commands before reading large files:
+
+```bash
+rg -n "<symbol-or-term>" <path>
+git diff --stat HEAD
+git diff --name-only HEAD
+git diff --unified=0 HEAD -- <path>
+```
+
+Preserve user changes already present in the worktree.
+
+### Validation
+
+Fast local harness validation:
+
+```bash
+scripts/harness/quick-check.sh
+```
+
+Full local validation:
+
+```bash
+scripts/harness/validate.sh
+```
+
+`validate.sh` includes quick checks, frontend Bun build, and frontend smoke
+unless disabled by its documented environment flags. Docker image smoke builds
+are intentionally opt-in:
+
+```bash
+PLANET_HARNESS_DOCKER_SMOKE=1 scripts/harness/validate.sh
+```
+
+Harness scripts resolve `bun`, `uv`, and optional delivery tools from the
+current non-interactive environment first. If a tool is missing there, they ask
+the user's login interactive shell instead of assuming a specific dotfile.
+
+### High-Risk Areas
+
+- `planet.sh` owns local lifecycle, ports, WSL/LAN behavior, and destructive
+  `destroy` cleanup.
+- Frontend package management is Bun-only. Do not use npm, pnpm, or yarn.
+- Frontend changes must satisfy `scripts/harness/frontend-rules-check.sh`; use
+  rendered smoke evidence for public pages, auth guards, authenticated admin
+  route/section availability, safe navigation/search/tab interactions, mobile
+  layout, and 125% / 150% zoom, not only a build.
+- Admin or Docs layout changes must load `rules.md` `uiux` and preserve the
+  one-screen (`一屏` / `首屏`) height chain: route roots use `height: 100%`,
+  intermediate wrappers keep `min-height: 0`, and only the intended child owns
+  scrolling.
+- `aiprovider` is a protocol/provider adapter; keep business prompts and product
+  workflows in the backend.
+- Earth rendering depends on layer order, depth behavior, picking, and
+  performance-sensitive Three.js code.
+- Secrets belong in environment files or configured settings stores, never in
+  committed files.
+- Backend service code must use structured logging instead of `print()` or
+  debugger calls; `scripts/harness/backend-rules-check.sh` enforces this.
+
+### Conflict Policy
+
+Existing project rules and workflows win. If new harness guidance conflicts with
+`rules.md`, `AGENTS.md`, current docs, scripts, or CI, keep the existing
+behavior and document the compatibility note in `docs/harness-audit.md` or
+`docs/HARNESS.md`.
+
+---
+
+## Operating Mode
+
+- Default to acting directly when the user gives a clear task.
+- For authorized multi-agent work, follow the Coding Agent Model Strategy in
+  `rules.md` workflow: Sol for implementation, Luna for bounded supporting work,
+  and Astra for complex reasoning; the main agent owns final verification.
+- Ask before acting only when the missing decision is risky, cannot be
+  discovered from repository context, and no conservative assumption is safe.
+- Read relevant files before editing.
+- Prefer focused CLI evidence: `rg`, `git diff --stat`, `git diff --name-only`,
+  focused file reads, tests, builds, linters, and harness scripts.
+- Keep changes scoped to the requested area. Do not mix cleanup, feature work,
+  release work, and documentation unless the task requires it.
+
+---
+
+## Evidence Gates
+
+- Visual inputs are blocking evidence. If the user provides a screenshot, image,
+  mock, browser capture, or visual reference, obtain evidence from the artifact
+  before interpreting intent or editing code.
+- Path resolution is part of the task. If the path cannot be opened, first try
+  reasonable local equivalents such as WSL/Windows path conversion,
+  workspace-relative lookup, absolute paths, and attached-file locations.
+- Never guess from prompt text, filenames, previous context, logs, OCR, or
+  memory when a visual artifact was provided but cannot be accessed.
+- OCR is acceptable evidence for text-only visual questions or non-multimodal
+  environments; state that OCR was used as the fallback. Layout, color, spacing,
+  pixel, and rendering issues need real visual inspection or a clear limitation
+  note.
+- If a visual artifact still cannot be inspected, say so and pause that
+  visual-dependent part of the work.
+- Claims of completion need evidence: a relevant test, build, lint, screenshot,
+  diff, direct file check, or harness result.
+- For UI and rendering changes, verify the rendered result when local tooling
+  allows it.
+
+---
+
+## Communication
+
+- Match the user's language. Use Chinese for Chinese requests unless the user
+  asks otherwise.
+- Keep updates short and specific: what is being inspected, edited, or verified.
+- Final responses should summarize changed files and verification, with blockers
+  stated plainly.
+- Use file references with line numbers when explaining code or review findings.
+
+---
+
+## Quality Bar
+
+- Prefer existing project patterns over new abstractions.
+- Remove stale branches, mocks, compatibility paths, and duplicated helpers once
+  a stable path exists.
+- Centralize prompts, constants, defaults, and shared request/response handling.
+- Do not add secrets, generated runtime output, or local environment files.
+- Frontend commands use Bun only. Do not use `npm`, `pnpm`, or `yarn`.
+- Run the smallest relevant verification for the changed scope and report
+  anything skipped.
+
+---
+
+## Prohibited
+
+- Do not skip visual evidence handling when a visual artifact was provided.
+- Do not preserve obsolete harness files just because they already exist.
+- Do not invent behavior not present in code, docs, or verified external
+  sources.
+- Do not rewrite unrelated files during cleanup.
+- Do not mark a task complete without checking concrete success criteria.

CODEMAP.md

@@ -0,0 +1,110 @@
+# Code Map
+
+This map gives agents and maintainers a quick orientation without replacing the
+deeper architecture docs. Current implementation docs under `docs/technical/`
+are the source of detail for specific subsystems.
+
+## Top-Level Areas
+
+| Path | Role | Notes |
+| --- | --- | --- |
+| `backend/` | FastAPI backend, auth, APIs, data collectors, AI task orchestration, persistence | Tests live in `backend/tests/`; run backend tests from `backend/` with the root uv project. |
+| `frontend/` | React admin console, Docs UI, Web Earth shell, Vite build | Use Bun only. Public Earth assets live under `frontend/public/earth/`. |
+| `aiprovider/` | Model provider/protocol adapter service | Keep it free of product-specific prompts and workflows. |
+| `motion_agent/` | Motion capture protocol service used by `planet.sh` | Often dry-runs when cameras are unavailable, especially in WSL. |
+| `scripts/` | Utility scripts and harness wrappers | Harness commands live in `scripts/harness/`. |
+| `docs/` | Plans, technical docs, changelog, harness docs | Public technical docs are explicitly registered by the frontend Docs catalog. |
+| `deploy/helm/planet/` | Helm chart for staging/deployment smoke paths | CI runs helm lint/template when delivery checks are available. |
+| `.gitea/workflows/` | CI, release image build, staging deploy workflows | This repository uses Gitea workflow files, not `.github/workflows/`. |
+| `planet.sh` | Main local lifecycle script | Owns init/start/restart/stop/health/log/createuser/destroy. |
+| `planet.cmd` and `scripts/windows/` | Portable Windows entry, bound desktop launcher generation, and WSL restart bridge | Delegates service lifecycle to `planet.sh`; Windows and isolated WSL regression tests live under `scripts/harness/`. |
+
+## Runtime Entry Points
+
+| Runtime | Entry Point | Validation |
+| --- | --- | --- |
+| Local full stack | `./planet.sh start` | `./planet.sh health` |
+| Backend API | `backend/app/main.py` | `cd backend && uv run --frozen --group dev --project .. python -m pytest -q` |
+| Frontend app | `frontend/src/main.tsx` and `frontend/vite.config.mts` | `cd frontend && bun run build` |
+| AI Provider | `aiprovider/main.py` | `curl http://localhost:8010/health` after startup |
+| Motion Agent | `python -m motion_agent` via `planet.sh` | `./planet.sh health` or dry-run startup |
+| Docs UI | `frontend/src/pages/Docs/` | Docs catalog metadata plus frontend build |
+
+## Ownership Boundaries
+
+- Backend owns business state, auth, evidence collection, prompt selection, AI
+  task orchestration, and database persistence.
+- `aiprovider` owns provider identity, request adapter style, model gateway
+  retries, and health/status endpoints only.
+- Frontend owns operator workflows, Docs presentation, Web Earth orchestration,
+  and client-side state that mirrors backend truth.
+- Web Earth rendering changes must preserve documented layer order, altitude
+  offsets, picking behavior, legend semantics, and performance constraints.
+- `planet.sh` owns local environment bootstrap and service lifecycle. Prefer
+  wrapping it from harness scripts instead of duplicating its internals.
+- Harness scripts source `scripts/harness/lib.sh` so agent shells that cannot
+  see `bun` or `uv` in non-interactive `PATH` can still resolve the user's login
+  interactive command path without hardcoding `.zshrc`.
+
+## Validation Commands
+
+```bash
+scripts/harness/doctor.sh
+scripts/harness/security-check.sh
+scripts/harness/backend-rules-check.sh
+scripts/harness/frontend-rules-check.sh
+scripts/harness/docs-consistency-check.sh
+scripts/harness/quick-check.sh
+scripts/harness/validate.sh
+./planet.sh health
+```
+
+CI-equivalent local checks:
+
+```bash
+cd backend
+uv run --frozen --group dev --project .. python -m pytest -s tests/test_api.py tests/test_realtime_sources.py -q
+
+cd frontend
+bun install --frozen-lockfile
+bun run build
+PLANET_FRONTEND_SMOKE_URL=http://127.0.0.1:4173 bun ../scripts/harness/frontend-smoke.mjs
+```
+
+The frontend smoke covers public routes, unauthenticated admin guards,
+login-error handling, the Earth iframe entry, and authenticated `super_admin`
+admin route/section rendering with mocked API data. Authenticated admin checks
+run on desktop, mobile, and 125% / 150% zoom; desktop and mobile passes also
+check for accidental global horizontal overflow. A second smoke layer exercises
+safe desktop/mobile navigation, admin search, section tab switching, dialog
+opening, and non-destructive shortcut links.
+
+Optional delivery smoke, when Docker and Helm are available:
+
+```bash
+PLANET_HARNESS_DOCKER_SMOKE=1 scripts/harness/validate.sh
+```
+
+## Deeper Docs
+
+| Topic | Start Here |
+| --- | --- |
+| Data products and flows | `docs/technical/zh/platform-data-flows.md` and `docs/technical/en/platform-data-flows.md` |
+| Operations and local lifecycle | `docs/technical/zh/ops-runbook.md` and `docs/technical/en/ops-runbook.md` |
+| `planet.sh` startup behavior | `docs/technical/zh/ops-planet-sh-startup.md` and `docs/technical/en/ops-planet-sh-startup.md` |
+| AI Provider | `docs/technical/zh/agents-aiprovider.md` and `docs/technical/en/agents-aiprovider.md` |
+| Admin frontend | `docs/technical/zh/frontend-admin-frontend-context.md` and `docs/technical/en/frontend-admin-frontend-context.md` |
+| Earth frontend | `docs/technical/zh/earth-frontend-context.md` and `docs/technical/en/earth-frontend-context.md` |
+| Earth render order | `docs/technical/zh/earth-render-layer-order.md` and `docs/technical/en/earth-render-layer-order.md` |
+| Documentation rules | `docs/documentation-coverage-rules.md` |
+| Harness workflow | `docs/HARNESS.md` |
+
+## Known Sharp Edges
+
+- `project_context.md` is static background for agents. It now labels future
+  stack directions separately, but current code and technical docs still win
+  when details diverge.
+- README now describes Web Earth, React admin, FastAPI, and `aiprovider` as the
+  active local development shape.
+- Local `destroy` is intentionally destructive for Planet-owned Docker and build
+  state. Never run it as a validation shortcut.

README.md

@@ -83,7 +83,7 @@
 | 组件 | 用途 |
 |------|------|
 | React 18 | UI 框架 |
-| Ant Design Pro | 管理后台组件 |
+| Tactile UI / Radix primitives / lucide-react | 管理后台组件、基础交互与图标 |
 | Axios | HTTP 客户端 |
 | Socket.io-client | WebSocket 客户端 |
 | ECharts | 统计图表 |
@@ -168,10 +168,12 @@
 
 ## 快速启动
 
+入口需要先具备 `zsh`、`curl` 和可访问的软件源。Ubuntu / Ubuntu WSL 上，`init` 会自动检测并补装 Docker Engine、Compose v2 和 Buildx，启动 Docker 服务并配置当前用户的访问权限；需要系统权限时会提示输入 sudo 密码。其他系统请先准备可用的 Docker 环境。
+
 ```bash
 # 新机器或空项目首次初始化
 ./planet.sh init
-# 会自动安装/检查 uv、bun，同步 Python/前端依赖
+# 会先准备 Docker / Compose / Buildx，再安装/检查 uv、bun 并同步 Python/前端依赖
 # 会在缺少时生成 backend/.env、aiprovider/.env、frontend/.env.local
 # 会启动 PostgreSQL/Redis，并创建表、默认数据源和本地默认用户
 
@@ -322,7 +324,7 @@ ipconfig
 
 ## 启动容错参数
 
-`planet.sh` 现在为依赖安装、数据库、AI Provider 启动加入了有限次重试，并会在数据库与 `aiprovider` 启动后额外等待 Docker healthcheck。
+`planet.sh` 为依赖安装、数据库、AI Provider 启动提供有限次重试。数据库先检查容器健康，再验证后端实际连接；`aiprovider` 直接以宿主机 `/health` 就绪为准。后端进程退出或应用初始化失败时立即停止等待，避免重复消耗健康检查预算。
 
 可通过环境变量临时调整:
 

TODO.md

@@ -4,6 +4,7 @@ This file is the active backlog only. Completed history belongs in `docs/CHANGEL
 
 ## Earth
 
+- [ ] Motion Agent v2 hardening: tune the implemented MediaPipe gesture recognizer across camera placements, exercise the UE command/control client, run reconnect and dual-camera soak tests, and continue the v3 calibrated 3D roadmap described in [Motion Agent v2 Control Protocol And 3D Calibration Roadmap](/home/ray/dev/linkong/planet/docs/plans/motion-agent-v2-control-protocol-plan.md).
 - [ ] Earth AI command entry: merge natural-language and speech-triggered LLM commands into the existing Earth search panel as described in [Agent Runtime, Earth LLM Command, And Speech Entry Plan](/home/ray/dev/linkong/planet/docs/plans/agents-earth-command-runtime-plan.md).
 - [ ] Earth action executor: implement safe visualization actions for layer toggles, batch highlights, filters, focus, result panels, and clear-highlight behavior.
 - [ ] Earth entity matching: support stable entity ids and batch matching for Beidou satellites, mainland China compute centers, BGP, news, vessels, and cables.

VERSION

@@ -1 +1 @@
-0.67.0
+0.76.2

agents.md

@@ -1,231 +0,0 @@
-# agents.md
-
-**AI Agent 角色设定。定义 AI 如何行为、沟通和工作。**
-
----
-
-## Identity
-
-You are **opencode**, an AI coding assistant specialized in enterprise-level systems.
-
-You are working on the **智能星球计划 (Intelligent Planet Plan)** - a situational awareness system for data-centric competition featuring:
-- Python FastAPI backend
-- React Admin dashboard
-- Unreal Engine 5 3D visualization
-- Multi-source data collection
-- Polarized 3D large display (4K, 120Hz)
-
----
-
-## Communication Style
-
-### Tone
-- **Professional but concise**
-- Technical accuracy with clarity
-- No unnecessary verbosity
-- Use code comments sparingly (explain **why**, not **what**)
-
-### When Responding
-1. **Answer directly** - 1-3 sentences for simple questions
-2. **Use code blocks** for all code snippets
-3. **Include file:line_number** references when discussing code
-4. **Never** start with "I am an AI assistant" or similar phrases
-5. **Never** add unnecessary preambles/postambles
-
-### Examples
-
-**Good:**
-```
-GPU clusters are stored in `backend/app/services/collectors/top500.py:45`.
-```
-
-**Bad:**
-```
-Based on the information you provided, I can see that the GPU clusters are stored in the top500.py file at line 45. Let me explain more about this...
-```
-
----
-
-## Operational Mode
-
-### Plan Mode (default for complex tasks)
-- Analyze requirements
-- Propose architecture
-- Confirm with user before execution
-- **DO NOT** write code until approved
-
-### Build Mode (after user approval)
-- Execute the approved plan
-- Write code, run commands
-- Verify results
-- Report completion concisely
-
-### Read-Only Mode
-- Analyze code
-- Explain functionality
-- Answer questions
-- **DO NOT** modify files
-
----
-
-## Decision Framework
-
-### When to Ask Before Acting
-- Unclear requirements
-- Multiple implementation approaches
-- Architecture changes
-- Dependency additions
-- Anything that could break existing functionality
-
-### When to Act Directly
-- Clear, approved requirements
-- Routine tasks (linting, formatting, running tests)
-- Following established patterns
-- Fixing obvious bugs
-
-### When to Refuse
-- Malicious code requests
-- Security violations (secrets, credentials)
-- Anything that violates `rules.md`
-
----
-
-## Working Principles
-
-### 1. First Understand, Then Act
-- Read relevant files before editing
-- Understand existing patterns and conventions
-- Follow the code style in the codebase
-- Match the project's technology choices
-
-### 2. Incremental Progress
-- Break large tasks into smaller PRs
-- Complete one feature before starting the next
-- Run tests after each significant change
-- Commit frequently with clear messages
-
-### 3. Quality First
-- Write tests for new functionality
-- Run linters before committing
-- Fix warnings, don't ignore them
-- Document non-obvious decisions
-
-### 4. Communication Clarity
-- Use precise technical language
-- Show relevant code, not explanations
-- Report errors with context
-- Confirm understanding of requirements
-
----
-
-## Code Review Checklist
-
-Before marking a task complete:
-
-- [ ] Code follows `rules.md` style guidelines
-- [ ] Type hints are correct and complete
-- [ ] Error handling is proper (no silent failures)
-- [ ] Tests pass locally
-- [ ] Linting passes
-- [ ] No TODO comments left behind
-- [ ] Documentation updated if needed
-- [ ] Commit message is clear
-
----
-
-## Common Workflows
-
-### Feature Development
-```
-1. Understand requirements
-2. Check existing patterns in codebase
-3. Design solution (brief mental model)
-4. Write code following rules.md
-5. Write/run tests
-6. Lint and format
-7. Commit with clear message
-8. Report completion
-```
-
-### Bug Fix
-```
-1. Reproduce the bug (write failing test)
-2. Locate the source
-3. Fix the issue
-4. Verify test passes
-5. Check for regressions
-6. Commit fix
-```
-
-### Refactoring
-```
-1. Understand current behavior
-2. Design target state
-3. Make incremental changes
-4. Preserve tests
-5. Verify functionality
-6. Clean up dead code
-```
-
----
-
-## Special Considerations
-
-### WebSocket Services
-- Implement heartbeat mechanism (30-second intervals)
-- Handle disconnection gracefully
-- Include camera position in control frames
-- Support both update and full sync modes
-
-### Data Collectors
-- Inherit from BaseCollector
-- Implement fetch() and transform() methods
-- Support incremental updates
-- Handle API changes gracefully
-
-### UE5 Integration
-- Communicate via WebSocket
-- Send data frames at configurable intervals (default 5 min)
-- Support auto-cruise and manual modes
-- Optimize for 4K@120Hz rendering
-
-### Multi-User Security
-- JWT tokens with 15-minute expiration
-- Redis token blacklist for logout
-- Role-based access control (RBAC)
-- Audit logging for all actions
-
----
-
-## Output Format
-
-### When Writing Code
-```python
-# File: backend/app/services/collectors/top500.py
-from typing import List, Dict
-
-class TOP500Collector:
-    async def fetch(self) -> List[Dict]:
-        ...
-```
-
-### When Explaining
-- Use concise paragraphs
-- Include code references
-- No conversational filler
-
-### When Reporting Progress
-- What was done
-- What remains
-- Any blockers
-- Next action
-
----
-
-## Remember
-
-1. **Rules are hard constraints** - follow `rules.md` absolutely
-2. **Context provides understanding** - use `project_context.md` for background
-3. **Role defines behavior** - follow `agents.md` for how to work
-4. **Quality over speed** - Enterprise systems require precision
-5. **Communicate clearly** - Precision in, precision out

aiprovider/Dockerfile

@@ -1,4 +1,4 @@
-# syntax=docker/dockerfile:1.7
+# Use BuildKit's bundled frontend to avoid a separate Docker Hub fetch.
 
 ARG PYTHON_IMAGE=python:3.14-slim
 ARG UV_IMAGE=ghcr.io/astral-sh/uv:latest
@@ -7,9 +7,6 @@ ARG AI_PROVIDER_BUILD_FINGERPRINT=unknown
 FROM ${UV_IMAGE} AS uv
 FROM ${PYTHON_IMAGE}
 
-ARG AI_PROVIDER_BUILD_FINGERPRINT
-LABEL planet.aiprovider.build-fingerprint="${AI_PROVIDER_BUILD_FINGERPRINT}"
-
 COPY --from=uv /uv /uvx /bin/
 
 WORKDIR /app
@@ -28,13 +25,16 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
 COPY pyproject.toml uv.lock /app/
 RUN --mount=type=cache,target=/root/.cache/uv \
     --mount=type=secret,id=planet_uv_config,target=/root/.config/uv/uv.toml,required=false \
-    uv sync --frozen --no-dev
+    uv sync --frozen --only-group aiprovider
 
 COPY aiprovider /app/aiprovider
 
+ARG AI_PROVIDER_BUILD_FINGERPRINT
+LABEL planet.aiprovider.build-fingerprint="${AI_PROVIDER_BUILD_FINGERPRINT}"
+
 EXPOSE 8010
 
 HEALTHCHECK --interval=30s --timeout=5s --retries=3 \
     CMD curl -fsS http://127.0.0.1:8010/health >/dev/null || exit 1
 
-CMD ["uv", "run", "--frozen", "--no-dev", "--project", "/app", "python", "-m", "uvicorn", "aiprovider.main:app", "--host", "0.0.0.0", "--port", "8010"]
+CMD ["/app/.venv/bin/python", "-m", "uvicorn", "aiprovider.main:app", "--host", "0.0.0.0", "--port", "8010"]

aiprovider/README.md

@@ -4,7 +4,7 @@
 
 完整使用说明见:
 
-- [docs/agents/aiprovider.md](/home/ray/dev/linkong/planet/docs/agents/aiprovider.md)
+- [AI Provider 指南](../docs/technical/zh/agents-aiprovider.md)
 
 当前支持:
 
@@ -15,6 +15,7 @@
   - `AI_PROVIDER=ollama`
 - request adapter:
   - `AI_PROVIDER_API=openai-completions`
+  - `AI_PROVIDER_API=openai-responses`
   - `AI_PROVIDER_API=anthropic-messages`
   - `AI_PROVIDER_API=ollama-generate`
 

aiprovider/provider_service.py

@@ -3,6 +3,7 @@ from __future__ import annotations
 import asyncio
 import json
 from typing import Any
+from uuid import NAMESPACE_URL, uuid4, uuid5
 
 import httpx
 from fastapi import HTTPException, status
@@ -65,6 +66,7 @@ class ProviderService:
         self.model_provider_apis = self._parse_model_provider_apis(
             overrides.get("model_provider_apis")
         )
+        self.session_id = str(uuid4())
 
     def get_status(self) -> AIProviderStatusResponse:
         enabled = self.provider != "disabled"
@@ -95,6 +97,8 @@ class ProviderService:
             )
 
         prompt = self._build_prompt(payload)
+        if payload.context.get("session_id") is not None:
+            self.session_id = str(uuid5(NAMESPACE_URL, f"planet:{payload.context['session_id']}"))
 
         provider_api = self._resolve_model_provider_api(model)
 
@@ -102,6 +106,10 @@ class ProviderService:
             data = await self._request_openai_compatible(model, prompt, payload.system_prompt)
             content = self._extract_openai_content(data)
             content_blocks = self._extract_openai_blocks(data)
+        elif provider_api == "openai-responses":
+            data = await self._request_openai_responses(model, prompt, payload.system_prompt)
+            content_blocks = self._extract_responses_blocks(data)
+            content = "".join(block.text for block in content_blocks if block.text)
         elif provider_api == "anthropic-messages":
             data = await self._request_anthropic_messages(
                 model,
@@ -200,6 +208,39 @@ class ProviderService:
             request_body=request_body,
         )
 
+    async def _request_openai_responses(
+        self, model: str, prompt: str, system_prompt: str | None = None,
+    ) -> dict[str, Any]:
+        request_body: dict[str, Any] = {
+            "model": model, "input": prompt, "max_output_tokens": self.max_tokens, "store": False,
+        }
+        resolved_system_prompt = self._resolve_system_prompt(system_prompt)
+        if resolved_system_prompt:
+            request_body["instructions"] = resolved_system_prompt
+        return await self._post(
+            path="/responses",
+            headers={"Authorization": f"Bearer {self.api_key}", "Content-Type": "application/json"},
+            request_body=request_body,
+        )
+
+    def _extract_responses_blocks(self, payload: dict[str, Any]) -> list[AIContentBlock]:
+        blocks: list[AIContentBlock] = []
+        for item in payload.get("output") or []:
+            if not isinstance(item, dict):
+                continue
+            if item.get("type") == "message":
+                for part in item.get("content") or []:
+                    if not isinstance(part, dict):
+                        continue
+                    text = part.get("text") or part.get("refusal")
+                    if isinstance(text, str) and text:
+                        blocks.append(AIContentBlock(type="text", text=text))
+            elif item.get("type") == "reasoning":
+                for part in item.get("summary") or []:
+                    if isinstance(part, dict) and isinstance(part.get("text"), str):
+                        blocks.append(AIContentBlock(type="thinking", thinking=part["text"]))
+        return blocks
+
     async def _request_anthropic_messages(
         self,
         model: str,
@@ -226,7 +267,7 @@ class ProviderService:
         resolved_system_prompt = self._resolve_system_prompt(system_prompt)
         if resolved_system_prompt:
             request_body["system"] = resolved_system_prompt
-        resolved_thinking = self._resolve_anthropic_thinking(thinking)
+        resolved_thinking = self._resolve_anthropic_thinking(thinking, model)
         if resolved_thinking:
             request_body["thinking"] = resolved_thinking
         if self.provider == "minimax" and self.base_url.endswith("/anthropic"):
@@ -243,8 +284,12 @@ class ProviderService:
             request_body=request_body,
         )
 
-    def _resolve_anthropic_thinking(self, thinking: dict[str, Any] | None) -> dict[str, Any] | None:
+    def _resolve_anthropic_thinking(
+        self, thinking: dict[str, Any] | None, model: str,
+    ) -> dict[str, Any] | None:
         if thinking:
+            if model.casefold() == "minimax-m3" and thinking.get("type") == "enabled":
+                return {"type": "adaptive"}
             return thinking
 
         # OpenClaw treats MiniMax's Anthropic-compatible path specially:
@@ -295,6 +340,9 @@ class ProviderService:
         headers: dict[str, str],
         request_body: dict[str, Any],
     ) -> dict[str, Any]:
+        headers = {"User-Agent": "Planet/1.0", **headers}
+        if self.provider == "opencode-go":
+            headers["x-opencode-session"] = self.session_id
         last_error: Exception | None = None
         for attempt in range(1, self.http_retry_attempts + 1):
             try:

backend/Dockerfile

@@ -28,6 +28,12 @@ RUN --mount=type=cache,target=/root/.cache/uv \
     uv sync --frozen --no-dev
 
 COPY backend /app/backend
+COPY config/earth-boundary-pov-policy.china-v1.json /app/config/earth-boundary-pov-policy.china-v1.json
+COPY scripts/build_earth_boundary_china_pov_geojson.py /app/scripts/build_earth_boundary_china_pov_geojson.py
+COPY scripts/build_earth_boundary_pmtiles.py /app/scripts/build_earth_boundary_pmtiles.py
+COPY scripts/earth_boundary_policy.py /app/scripts/earth_boundary_policy.py
+COPY resources/earth/boundary-pov-geometry.china-v1.json /app/resources/earth/boundary-pov-geometry.china-v1.json
+COPY frontend/public/earth/assets/cards/manifest.json /app/frontend/public/earth/assets/cards/manifest.json
 COPY VERSION /app/VERSION
 
 EXPOSE 8000

backend/app/api/main.py

@@ -1,3 +1,4 @@
+from app.api.v1 import card_images
 from fastapi import APIRouter
 from app.api.v1 import (
     ai,
@@ -12,6 +13,7 @@ from app.api.v1 import (
     alerts,
     settings,
     collected_data,
+    compute_intelligence,
     data_products,
     layers,
     visualization,
@@ -43,6 +45,7 @@ api_router.include_router(alerts.router, prefix="/alerts", tags=["alerts"])
 api_router.include_router(settings.router, prefix="/settings", tags=["settings"])
 api_router.include_router(system_control.router, prefix="/system", tags=["system"])
 api_router.include_router(data_products.router, prefix="/data-products", tags=["data-products"])
+api_router.include_router(compute_intelligence.router, prefix="/data-products/compute", tags=["compute"])
 api_router.include_router(layers.router, prefix="/layers", tags=["layers"])
 api_router.include_router(visualization.router, prefix="/visualization", tags=["visualization"])
 api_router.include_router(
@@ -56,3 +59,5 @@ api_router.include_router(tv.router, prefix="/tv", tags=["tv"])
 api_router.include_router(news.router, prefix="/news", tags=["news"])
 api_router.include_router(interactables.router, prefix="/interactables", tags=["interactables"])
 api_router.include_router(realtime_sources.router, prefix="/realtime-sources", tags=["realtime-sources"])
+
+api_router.include_router(card_images.router, prefix="/earth/card-images", tags=["card-images"])

backend/app/api/v1/auth.py

@@ -4,6 +4,7 @@ from sqlalchemy.ext.asyncio import AsyncSession
 from sqlalchemy import text
 
 from app.core.config import settings
+from app.core.enums import OtpPurpose, UserRole
 from app.core.logging import get_logger
 from app.core.security import (
     create_access_token,
@@ -170,7 +171,7 @@ async def get_me(current_user: User = Depends(get_current_user)):
     }
 
 
-async def _send_code_or_raise(db: AsyncSession, email: str, code: str, purpose: str) -> None:
+async def _send_code_or_raise(db: AsyncSession, email: str, code: str, purpose: OtpPurpose) -> None:
     try:
         await send_verification_email(db, to=email, code=code, purpose=purpose)
     except EmailNotConfiguredError as exc:
@@ -207,7 +208,7 @@ async def register(payload: UserRegister, db: AsyncSession = Depends(get_db)):
         username=payload.username,
         email=payload.email,
         password_hash=get_password_hash(payload.password),
-        role="viewer",
+        role=UserRole.VIEWER.value,
         is_active=True,
         email_verified=False,
     )
@@ -215,13 +216,13 @@ async def register(payload: UserRegister, db: AsyncSession = Depends(get_db)):
     await db.commit()
 
     try:
-        code = otp.issue_code(payload.email, "register")
+        code = otp.issue_code(payload.email, OtpPurpose.REGISTER)
     except otp.OtpResendRateLimited as exc:
         raise HTTPException(
             status_code=status.HTTP_429_TOO_MANY_REQUESTS,
             detail={"code": exc.code, "retry_after_seconds": exc.retry_after_seconds},
         ) from exc
-    await _send_code_or_raise(db, payload.email, code, "register")
+    await _send_code_or_raise(db, payload.email, code, OtpPurpose.REGISTER)
     return {"status": "pending_verification", "email": payload.email}
 
 
@@ -266,7 +267,7 @@ async def resend_code(payload: ResendCodeRequest, db: AsyncSession = Depends(get
     if user is None:
         # Avoid email enumeration; pretend success.
         return {"status": "ok"}
-    if payload.purpose == "register" and user.email_verified:
+    if payload.purpose is OtpPurpose.REGISTER and user.email_verified:
         raise HTTPException(
             status_code=status.HTTP_409_CONFLICT,
             detail={"code": "ALREADY_VERIFIED"},
@@ -289,12 +290,17 @@ async def forgot_password(payload: ForgotPasswordRequest, db: AsyncSession = Dep
         # Don't leak whether an email is registered.
         return {"status": "ok"}
     try:
-        code = otp.issue_code(payload.email, "reset_password")
+        code = otp.issue_code(payload.email, OtpPurpose.RESET_PASSWORD)
     except otp.OtpResendRateLimited:
         # Silently accept; the user can retry after the cooldown.
         return {"status": "ok"}
     try:
-        await send_verification_email(db, to=payload.email, code=code, purpose="reset_password")
+        await send_verification_email(
+            db,
+            to=payload.email,
+            code=code,
+            purpose=OtpPurpose.RESET_PASSWORD,
+        )
     except EmailNotConfiguredError as exc:
         raise HTTPException(
             status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
@@ -304,7 +310,11 @@ async def forgot_password(payload: ForgotPasswordRequest, db: AsyncSession = Dep
         logger.warning_event(
             "SMTP send failed",
             event="auth.email.send_failed",
-            context={"email": payload.email, "purpose": "reset_password", "error": str(exc)},
+            context={
+                "email": payload.email,
+                "purpose": OtpPurpose.RESET_PASSWORD.value,
+                "error": str(exc),
+            },
         )
     return {"status": "ok"}
 
@@ -318,7 +328,7 @@ async def reset_password(payload: ResetPasswordRequest, db: AsyncSession = Depen
             detail={"code": "OTP_INVALID"},
         )
     try:
-        otp.verify_code(payload.email, "reset_password", payload.code)
+        otp.verify_code(payload.email, OtpPurpose.RESET_PASSWORD, payload.code)
     except otp.OtpExpired as exc:
         raise HTTPException(
             status_code=status.HTTP_410_GONE,

backend/app/api/v1/card_images.py

@@ -0,0 +1,108 @@
+"""Card-image configuration, upload, catalog selection and public resolution."""
+
+from fastapi import APIRouter, Depends, File, HTTPException, Query, Response, UploadFile
+from fastapi.responses import FileResponse
+from sqlalchemy.ext.asyncio import AsyncSession
+from starlette.concurrency import run_in_threadpool
+
+from app.core.enums import UserRole
+from app.core.security import get_current_user
+from app.db.session import get_db
+from app.models.user import User
+from app.schemas.card_images import (
+    CardImageConfiguration,
+    CardImagePreview,
+    CardImagePublish,
+    EntityType,
+    MatchScope,
+)
+from app.services.card_image_configuration import (
+    publish_configuration,
+    read_configuration,
+    resolve_image,
+)
+from app.services.card_image_files import MAX_UPLOAD_BYTES, store_image, uploaded_path
+from app.services.card_image_targets import get_target, search_targets
+
+router = APIRouter()
+
+
+async def image_admin(user: User = Depends(get_current_user)) -> User:
+    if user.role not in {UserRole.ADMIN.value, UserRole.SUPER_ADMIN.value}:
+        raise HTTPException(403, "Administrator access required")
+    return user
+
+
+@router.get("/configuration")
+async def get_configuration(
+    response: Response, _actor: User = Depends(image_admin), db: AsyncSession = Depends(get_db)
+):
+    response.headers["Cache-Control"] = "no-store"
+    return await read_configuration(db)
+
+
+@router.put("/configuration")
+async def save_configuration(
+    payload: CardImagePublish,
+    actor: User = Depends(image_admin),
+    db: AsyncSession = Depends(get_db),
+):
+    return await publish_configuration(db, payload.revision, payload.config, actor)
+
+
+@router.post("/upload")
+async def upload_image(file: UploadFile = File(...), _actor: User = Depends(image_admin)):
+    content = await file.read(MAX_UPLOAD_BYTES + 1)
+    await file.close()
+    return await run_in_threadpool(store_image, content, (file.filename or "Image")[:200])
+
+
+@router.get("/files/{filename}")
+async def image_file(filename: str):
+    path = uploaded_path(filename)
+    if not path.is_file():
+        raise HTTPException(404, "Image not found")
+    return FileResponse(
+        path,
+        headers={
+            "Cache-Control": "public, max-age=31536000, immutable",
+            "X-Content-Type-Options": "nosniff",
+        },
+    )
+
+
+@router.get("/targets")
+async def targets(
+    entity_type: EntityType,
+    scope: MatchScope = "object",
+    q: str = Query(default="", max_length=100),
+    limit: int = Query(default=25, ge=1, le=50),
+    _actor: User = Depends(image_admin),
+    db: AsyncSession = Depends(get_db),
+):
+    return {"items": await search_targets(db, entity_type, scope, q.strip(), limit)}
+
+
+@router.get("/resolve")
+async def resolve(
+    response: Response,
+    entity_type: EntityType,
+    entity_id: str = Query(min_length=1, max_length=300),
+    db: AsyncSession = Depends(get_db),
+):
+    response.headers["Cache-Control"] = "no-store"
+    envelope = await read_configuration(db)
+    config = CardImageConfiguration.model_validate(envelope["config"])
+    context = await get_target(db, entity_type, entity_id)
+    result = resolve_image(config, context)
+    return {**result, "revision": envelope["revision"], "entity": context}
+
+
+@router.post("/preview")
+async def preview(
+    payload: CardImagePreview,
+    _actor: User = Depends(image_admin),
+    db: AsyncSession = Depends(get_db),
+):
+    context = await get_target(db, payload.entity_type, payload.entity_id)
+    return {**resolve_image(payload.config, context), "entity": context}

backend/app/api/v1/compute_intelligence.py

@@ -0,0 +1,235 @@
+"""Compute observations, administrator configuration and independent facility samples."""
+
+from datetime import UTC, datetime
+from typing import Literal
+
+from app.core.compute_types import COMPUTE_SITE_TYPES, compute_record_filter, compute_site_type_expression
+from app.api.v1.visualization import convert_compute_centers_to_geojson
+from app.core.earth_geography_policy import display_country
+from app.core.enums import UserRole
+from app.core.security import get_current_user
+from app.db.session import get_db
+from app.models.collected_data import CollectedData
+from app.models.data_snapshot import DataSnapshot
+from app.models.user import User
+from app.schemas.compute_configuration import (
+    ComputeConfiguration,
+    ComputeConfigurationReset,
+    ComputeConfigurationWrite,
+)
+from app.services.compute_catalog import (
+    PARTICIPANTS,
+    participant_members,
+    source_country_names,
+    source_country_participants,
+)
+from app.services.compute_center_locations import refresh_compute_center_location_cache
+from app.services.compute_configuration import (
+    DEFAULT_CONFIGURATION,
+    configuration_response,
+    publish_configuration,
+    read_configuration,
+)
+from app.services.compute_history import compute_history
+from fastapi import APIRouter, Depends, HTTPException, Query, Response
+from sqlalchemy import and_, case, func, or_, select
+from sqlalchemy.ext.asyncio import AsyncSession
+
+router = APIRouter()
+
+
+def selected_countries(countries: str) -> list[str]:
+    codes = list(dict.fromkeys(code.strip().upper() for code in countries.split(",")))
+    if not codes or len(codes) > 12 or any(code not in PARTICIPANTS for code in codes):
+        raise HTTPException(422, "Select 1–12 participants from the compute catalog")
+    members = participant_members(codes)
+    flattened = [country for participant in members.values() for country in participant]
+    if len(flattened) != len(set(flattened)):
+        raise HTTPException(422, "Comparison participants cannot have overlapping country members")
+    return codes
+
+
+@router.get("/series")
+async def get_series(
+    response: Response,
+    countries: str = "CN,US",
+    mode: Literal["global_share", "bilateral_share", "absolute"] = "global_share",
+    db: AsyncSession = Depends(get_db),
+):
+    codes = selected_countries(countries)
+    envelope = await read_configuration(db)
+    response.headers["Cache-Control"] = "no-store"
+    result = compute_history(codes, mode, ComputeConfiguration.model_validate(envelope["config"]))
+    result.update(
+        version=envelope["revision"],
+        provenance="bundled_reports" if envelope["is_default"] else "admin_curated_reports",
+    )
+    return result
+
+
+async def configuration_admin(user: User = Depends(get_current_user)) -> User:
+    if user.role not in {UserRole.ADMIN.value, UserRole.SUPER_ADMIN.value}:
+        raise HTTPException(403, "Administrator access required")
+    return user
+
+
+@router.get("/configuration")
+async def get_configuration(
+    response: Response,
+    _actor: User = Depends(configuration_admin),
+    db: AsyncSession = Depends(get_db),
+):
+    response.headers["Cache-Control"] = "no-store"
+    return configuration_response(await read_configuration(db))
+
+
+@router.put("/configuration")
+async def update_configuration(
+    payload: ComputeConfigurationWrite,
+    actor: User = Depends(configuration_admin),
+    db: AsyncSession = Depends(get_db),
+):
+    envelope = await publish_configuration(
+        db, revision=payload.revision, config=payload.config, actor=actor,
+    )
+    return configuration_response(envelope)
+
+
+@router.post("/configuration/reset")
+async def reset_configuration(
+    payload: ComputeConfigurationReset,
+    actor: User = Depends(configuration_admin),
+    db: AsyncSession = Depends(get_db),
+):
+    envelope = await publish_configuration(
+        db, revision=payload.revision, config=DEFAULT_CONFIGURATION, actor=actor, reset=True,
+    )
+    return configuration_response(envelope)
+
+
+async def sample_query(db: AsyncSession, countries: list[str], year: int | None):
+    now = datetime.now(UTC)
+    cutoff = min(datetime(year + 1, 1, 1, tzinfo=UTC), now) if year else now
+    ranked = (
+        select(
+            DataSnapshot.id,
+            func.row_number()
+            .over(
+                partition_by=DataSnapshot.source,
+                order_by=(DataSnapshot.completed_at.desc(), DataSnapshot.id.desc()),
+            )
+            .label("rank"),
+        )
+        .where(
+            DataSnapshot.source.in_(select(CollectedData.source).where(compute_record_filter(CollectedData)).distinct()),
+            DataSnapshot.status == "success",
+            DataSnapshot.completed_at < cutoff,
+        )
+        .subquery()
+    )
+    snapshots = select(ranked.c.id).where(ranked.c.rank == 1)
+    infrastructure = compute_site_type_expression(CollectedData).in_(COMPUTE_SITE_TYPES[2:])
+    # Old collector rows have ambiguous units/identity and must not become
+    # national evidence merely because a new UI can display them.
+    query = select(CollectedData).where(
+        compute_record_filter(CollectedData),
+        CollectedData.deleted_at.is_(None),
+        CollectedData.snapshot_id.in_(snapshots),
+        CollectedData.is_valid == 1,
+        or_(CollectedData.extra_data["compute_contract"].as_string() == "v1", infrastructure),
+        or_(
+            infrastructure,
+            CollectedData.source == "top500",
+            and_(
+                CollectedData.extra_data["status"].as_string() == "Existing",
+                func.coalesce(CollectedData.extra_data["superseded_by"].as_string(), "") == "",
+                CollectedData.extra_data["possible_duplicate"].as_boolean().is_not(True),
+            ),
+        ),
+        CollectedData.extra_data["country"].as_string().in_(source_country_names(countries)),
+    )
+    return query
+
+
+@router.get("/facilities")
+async def get_facilities(
+    countries: str = "CN,US",
+    year: int | None = Query(None, ge=2000, le=2100),
+    limit: int = Query(1000, ge=1, le=2000),
+    db: AsyncSession = Depends(get_db),
+):
+    codes = selected_countries(countries)
+    query = await sample_query(db, codes, year)
+    country = CollectedData.extra_data["country"].as_string()
+    source_participants = source_country_participants(codes)
+    # Canonical source subdivisions retain their raw label on each record but
+    # share one owner in the comparison, using the same policy as the globe.
+    owner_cases = [
+        (country == name, display_country(name))
+        for name in source_country_names(codes)
+        if display_country(name) != name
+    ]
+    display = case(*owner_cases, else_=country) if owner_cases else country
+    participant_cases = [(country == name, code) for name, code in source_participants.items()]
+    participant = case(*participant_cases, else_=None)
+    counts = (
+        (
+            await db.execute(
+                query.with_only_columns(
+                    display.label("country"),
+                    participant.label("participant_code"),
+                    CollectedData.source,
+                    func.count().label("count"),
+                ).group_by(display, participant, CollectedData.source)
+            )
+        )
+        .mappings()
+        .all()
+    )
+    participant_counts = (
+        (
+            await db.execute(
+                query.with_only_columns(
+                    participant.label("participant_code"),
+                    CollectedData.source,
+                    func.count().label("count"),
+                ).group_by(participant, CollectedData.source)
+            )
+        )
+        .mappings()
+        .all()
+    )
+    records = (
+        (await db.execute(query.order_by(CollectedData.name, CollectedData.id).limit(limit)))
+        .scalars()
+        .all()
+    )
+    if year:
+        await refresh_compute_center_location_cache(db, as_of=datetime(year + 1, 1, 1, tzinfo=UTC))
+    else:
+        await refresh_compute_center_location_cache(db)
+    payload = convert_compute_centers_to_geojson(records)
+    for feature in payload["features"]:
+        feature["properties"]["historical_snapshot"] = year is not None
+        properties = feature["properties"]
+        properties["source_country"] = properties.get("country")
+        properties["country"] = display_country(properties.get("country"))
+    total = sum(row["count"] for row in counts)
+    payload.update(
+        {
+            "counts": [dict(row) for row in counts],
+            "participant_counts": [dict(row) for row in participant_counts],
+            "participants": [PARTICIPANTS[code] for code in codes],
+            "total_count": total,
+            "returned_count": len(records),
+            "truncated": total > limit,
+            "reference_year": year,
+            "scope": "public_source_records",
+            "history_basis": (
+                "snapshots_collected_by_year_end" if year else "latest_successful_snapshot"
+            ),
+            "coverage": "available" if total else "no_verified_snapshot",
+            "cross_source_deduplicated": False,
+        }
+    )
+    return payload

backend/app/api/v1/data_products.py

@@ -52,7 +52,7 @@ PRODUCT_DEFINITIONS: dict[str, dict] = {
         "name": "算力",
         "sources": ["top500", "epoch_ai_gpu"],
         "primary_stat_key": "compute_center_count",
-        "stat_keys": ["compute_center_count", "supercomputer_count", "gpu_cluster_count"],
+        "stat_keys": ["compute_center_count", "supercomputer_count", "gpu_cluster_count", "data_center_count", "cloud_center_count", "edge_node_count"],
     },
 }
 

backend/app/api/v1/datasource_config.py

@@ -13,6 +13,7 @@ import httpx
 
 from app.core.target_schema_registry import get_target_schema, list_target_schemas
 from app.core.datasource_defaults import DEFAULT_DATASOURCES
+from app.core.enums import AuthType, MappingValidationStatus, UserRole
 from app.db.session import get_db
 from app.models.user import User
 from app.models.datasource_config import DataSourceConfig
@@ -34,7 +35,6 @@ from app.services.datasource_mapping import (
 )
 from app.services.custom_datasource_runtime import (
     CustomDatasourceRuntimeError,
-    fetch_rest_payload,
     get_custom_stream_status,
     run_mapped_rest_config,
     run_mapped_websocket_config,
@@ -42,8 +42,6 @@ from app.services.custom_datasource_runtime import (
     stop_custom_stream,
     test_websocket_config,
 )
-
-DATASOURCE_MAPPING_PROMPT_KEY = "datasource.mapping"
 from app.services.datasource_connectivity import (
     _resolve_aisstream_api_key,
     _resolve_spacetrack_credentials_with_override,
@@ -57,7 +55,8 @@ from app.services.persistent_logs import record_audit_log
 
 router = APIRouter()
 
-SECRET_REVEAL_ROLES = {"admin", "super_admin"}
+DATASOURCE_MAPPING_PROMPT_KEY = "datasource.mapping"
+SECRET_REVEAL_ROLES = {UserRole.ADMIN.value, UserRole.SUPER_ADMIN.value}
 
 
 def _user_role_value(user: User) -> str:
@@ -124,7 +123,7 @@ class DataSourceConfigCreate(BaseModel):
     description: Optional[str] = None
     source_type: str = Field(..., description="rest, websocket, http, api, database")
     endpoint: str = Field(..., max_length=500)
-    auth_type: str = Field(default="none", description="none, bearer, api_key, basic")
+    auth_type: AuthType = Field(default=AuthType.NONE, description="none, bearer, api_key, basic")
     auth_config: dict = Field(default={})
     headers: dict = Field(default={})
     config: dict = Field(default={"timeout": 30, "retry": 3})
@@ -135,7 +134,7 @@ class DataSourceConfigUpdate(BaseModel):
     description: Optional[str] = None
     source_type: Optional[str] = None
     endpoint: Optional[str] = Field(None, max_length=500)
-    auth_type: Optional[str] = None
+    auth_type: Optional[AuthType] = None
     auth_config: Optional[dict] = None
     headers: Optional[dict] = None
     config: Optional[dict] = None
@@ -210,7 +209,7 @@ class MappingTemplateCreate(BaseModel):
     mapping_json: dict
     sample_payload: Any | None = None
     sample_payload_hash: Optional[str] = None
-    validation_status: str = Field(default="draft", pattern="^(draft|valid|invalid)$")
+    validation_status: MappingValidationStatus = MappingValidationStatus.DRAFT
     is_active: bool = False
 
 
@@ -219,7 +218,7 @@ class MappingTemplateUpdate(BaseModel):
     mapping_json: Optional[dict] = None
     sample_payload: Any | None = None
     sample_payload_hash: Optional[str] = None
-    validation_status: Optional[str] = Field(default=None, pattern="^(draft|valid|invalid)$")
+    validation_status: Optional[MappingValidationStatus] = None
     is_active: Optional[bool] = None
 
 
@@ -918,6 +917,7 @@ async def get_datasource_target_schemas(
 @router.post("/mappings/propose")
 async def propose_datasource_mapping(
     payload: MappingProposeRequest,
+    db: AsyncSession = Depends(get_db),
     current_user: User = Depends(get_current_user),
     ai_client: AIProviderClient = Depends(get_ai_provider_client),
 ):

backend/app/api/v1/datasources.py

@@ -7,6 +7,7 @@ from sqlalchemy import func, or_, select, text
 from sqlalchemy.ext.asyncio import AsyncSession
 
 from app.core.logging import get_logger
+from app.core.enums import JobStatus, SnapshotStatus
 from app.core.time import to_iso8601_utc
 from app.core.security import get_current_user
 from app.core.data_sources import get_data_sources_config
@@ -19,7 +20,6 @@ from app.models.datasource_config import DataSourceConfig
 from app.models.task import CollectionTask
 from app.models.user import User
 from app.models.vessel import AISRawObservation
-from app.services.vessel_ais_aggregation import VESSEL_AIS_SCHEMA
 from app.services.scheduler import (
     sync_datasource_job,
 )
@@ -165,6 +165,8 @@ async def _load_latest_tasks(
 async def _load_collected_record_counts(
     db: AsyncSession,
     sources: list[str],
+    *,
+    exact_vessel_counts: bool = False,
 ) -> dict[str, int]:
     if not sources:
         return {}
@@ -185,14 +187,46 @@ async def _load_collected_record_counts(
         or "ais" in source
     ]
     if vessel_sources:
-        raw_result = await db.execute(
-            select(AISRawObservation.source, func.count(AISRawObservation.id))
-            .where(AISRawObservation.target_schema == VESSEL_AIS_SCHEMA)
-            .where(AISRawObservation.source.in_(vessel_sources))
-            .group_by(AISRawObservation.source)
+        if exact_vessel_counts:
+            exact_result = await db.execute(
+                select(AISRawObservation.source, func.count(AISRawObservation.id))
+                .where(AISRawObservation.source.in_(vessel_sources))
+                .group_by(AISRawObservation.source)
+            )
+            for source, count in exact_result.all():
+                counts[source] = max(counts.get(source, 0), int(count or 0))
+            return counts
+
+        # AIS raw observations can be tens of millions of rows. Use planner
+        # statistics for the datasource list instead of blocking page load on
+        # source-level count(*) scans.
+        stats_result = await db.execute(
+            text(
+                """
+                SELECT
+                    COALESCE(pg_class.reltuples, 0)::bigint AS total_rows,
+                    pg_stats.most_common_vals::text AS source_values,
+                    pg_stats.most_common_freqs::text AS source_freqs
+                FROM pg_class
+                LEFT JOIN pg_stats
+                  ON pg_stats.schemaname = 'public'
+                 AND pg_stats.tablename = 'ais_raw_observations'
+                 AND pg_stats.attname = 'source'
+                WHERE pg_class.relname = 'ais_raw_observations'
+                LIMIT 1
+                """
+            )
         )
-        for source, count in raw_result.all():
-            counts[source] = max(counts.get(source, 0), int(count or 0))
+        stats = stats_result.mappings().first()
+        if stats:
+            total_rows = int(stats["total_rows"] or 0)
+            values = str(stats["source_values"] or "").strip("{}")
+            freqs = str(stats["source_freqs"] or "").strip("{}")
+            source_values = [value.strip('"') for value in values.split(",") if value]
+            source_freqs = [float(value) for value in freqs.split(",") if value]
+            for source, freq in zip(source_values, source_freqs):
+                if source in vessel_sources:
+                    counts[source] = max(counts.get(source, 0), int(round(total_rows * freq)))
 
     return counts
 
@@ -548,7 +582,7 @@ async def get_running_task(db: AsyncSession, datasource_id: int) -> Optional[Col
     result = await db.execute(
         select(CollectionTask)
         .where(CollectionTask.datasource_id == datasource_id)
-        .where(CollectionTask.status == "running")
+        .where(CollectionTask.status == JobStatus.RUNNING.value)
         .order_by(CollectionTask.started_at.desc())
         .limit(1)
     )
@@ -576,8 +610,8 @@ async def get_running_task(db: AsyncSession, datasource_id: int) -> Optional[Col
             f"Marked failed automatically after stale running timeout "
             f"({STALE_RUNNING_TASK_TIMEOUT_MINUTES}m)"
         )
-        task.status = "failed"
-        task.phase = "failed"
+        task.status = JobStatus.FAILED.value
+        task.phase = JobStatus.FAILED.value
         task.completed_at = now
         task.error_message = f"{existing_error}\n{stale_reason}".strip() if existing_error else stale_reason
         await db.commit()
@@ -614,7 +648,7 @@ async def rollback_orphaned_running_task(
     )
 
     if snapshot is not None:
-        snapshot.status = "cancelled"
+        snapshot.status = SnapshotStatus.CANCELLED.value
         snapshot.is_current = False
         snapshot.completed_at = datetime.now(timezone.utc)
         summary = dict(snapshot.summary or {})
@@ -637,13 +671,13 @@ async def rollback_orphaned_running_task(
                 {"snapshot_id": snapshot.parent_snapshot_id},
             )
 
-    running_task.status = "cancelled"
-    running_task.phase = "cancelled"
+    running_task.status = JobStatus.CANCELLED.value
+    running_task.phase = JobStatus.CANCELLED.value
     running_task.completed_at = datetime.now(timezone.utc)
     existing_error = (running_task.error_message or "").strip()
     cancel_reason = "Cancelled after backend restart because the running task handle was lost; incomplete writes rolled back"
     running_task.error_message = f"{existing_error}\n{cancel_reason}".strip() if existing_error else cancel_reason
-    datasource.last_status = "cancelled"
+    datasource.last_status = JobStatus.CANCELLED.value
     datasource.last_run_at = datetime.now(timezone.utc)
     await db.commit()
 
@@ -678,7 +712,7 @@ async def fail_and_rollback_stale_running_task(
     )
 
     if snapshot is not None:
-        snapshot.status = "failed"
+        snapshot.status = SnapshotStatus.FAILED.value
         snapshot.is_current = False
         snapshot.completed_at = datetime.now(timezone.utc)
         summary = dict(snapshot.summary or {})
@@ -706,11 +740,11 @@ async def fail_and_rollback_stale_running_task(
         f"Marked failed automatically after stale running timeout "
         f"({STALE_RUNNING_TASK_TIMEOUT_MINUTES}m); incomplete writes rolled back"
     )
-    running_task.status = "failed"
-    running_task.phase = "failed"
+    running_task.status = JobStatus.FAILED.value
+    running_task.phase = JobStatus.FAILED.value
     running_task.completed_at = datetime.now(timezone.utc)
     running_task.error_message = f"{existing_error}\n{stale_reason}".strip() if existing_error else stale_reason
-    datasource.last_status = "failed"
+    datasource.last_status = JobStatus.FAILED.value
     datasource.last_run_at = datetime.now(timezone.utc)
     await db.commit()
 
@@ -929,7 +963,7 @@ async def get_datasource_row(
         [datasource],
         include_endpoint=include_endpoint,
     )
-    record_counts = await _load_collected_record_counts(db, [datasource.source])
+    record_counts = await _load_collected_record_counts(db, [datasource.source], exact_vessel_counts=True)
     return {
         "data": serialize_datasource_row(
             datasource,

backend/app/api/v1/earth.py

@@ -6,7 +6,7 @@ from pathlib import Path
 from typing import Any
 from uuid import uuid4
 
-from fastapi import APIRouter, Depends, File, HTTPException, Request, UploadFile, status
+from fastapi import APIRouter, Depends, File, Form, HTTPException, Query, Request, UploadFile, status
 from fastapi.security import HTTPAuthorizationCredentials, HTTPBearer
 from pydantic import BaseModel, Field
 from sqlalchemy import delete, func, select, text
@@ -21,6 +21,26 @@ from app.models.datasource_config import DataSourceConfig
 from app.models.system_setting import SystemSetting
 from app.models.user import User
 from app.services.tv_streams import get_tv_settings_payload
+from app.services.earth_news import (
+    get_earth_news_sources_payload,
+    reset_earth_news_sources_payload,
+    save_earth_news_sources_payload,
+    test_news_source_config,
+)
+from app.services.earth_news_manual import (
+    broadcast_manual_news_changed,
+    create_manual_news_group,
+    delete_manual_news_item,
+    get_news_record_or_404,
+    import_manual_news_items,
+    list_news_groups,
+    list_news_records,
+    parse_manual_news_import_upload,
+    rename_manual_news_group,
+    reprocess_manual_news_item,
+    serialize_news_record,
+    upsert_manual_news_item,
+)
 from app.services.earth_boundaries import (
     EarthBoundaryBuildError,
     get_boundary_build_status,
@@ -100,6 +120,39 @@ class EarthAboutPayload(BaseModel):
     meta: list[EarthAboutMetaItem] = Field(default_factory=list)
 
 
+class EarthNewsSourcesPayload(BaseModel):
+    cache_version: int | None = None
+    source_tags: list[dict[str, Any]] = Field(default_factory=list)
+    categories: list[dict[str, Any]] = Field(default_factory=list)
+    item_tag_rules: list[dict[str, Any]] = Field(default_factory=list)
+    sources: list[dict[str, Any]] = Field(default_factory=list)
+    health: dict[str, Any] = Field(default_factory=dict)
+
+
+class EarthNewsSourceTestPayload(BaseModel):
+    source: dict[str, Any] = Field(default_factory=dict)
+
+
+class EarthNewsManualItemPayload(BaseModel):
+    title: str = Field(default="", max_length=500)
+    summary: str = Field(default="", max_length=1200)
+    content: str = Field(default="", max_length=12000)
+    url: str = Field(default="", max_length=2000)
+    source: str = Field(default="", max_length=255)
+    region: str = Field(default="global", max_length=80)
+    published_at: str | None = None
+    category: str = Field(default="other", max_length=80)
+    tags: list[str] = Field(default_factory=list)
+    location: dict[str, Any] | None = None
+    homepage_url: str = Field(default="", max_length=2000)
+    content_language: str = Field(default="", max_length=32)
+    group_id: str | None = Field(default=None, max_length=120)
+
+
+class EarthNewsManualGroupPayload(BaseModel):
+    name: str = Field(default="", max_length=120)
+
+
 def _normalize_earth_brand_payload(payload: dict[str, Any] | None) -> dict[str, str]:
     merged = DEFAULT_EARTH_BRAND.copy()
     if payload:
@@ -324,6 +377,194 @@ async def reset_earth_about(
     return {"status": "reset", "about": _normalize_earth_about_payload(None), "is_default": True}
 
 
+@router.get("/news-sources")
+async def get_earth_news_sources(db: AsyncSession = Depends(get_db)):
+    return await get_earth_news_sources_payload(db)
+
+
+@router.put("/news-sources")
+async def update_earth_news_sources(
+    payload: EarthNewsSourcesPayload,
+    _current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    return await save_earth_news_sources_payload(db, payload.model_dump())
+
+
+@router.delete("/news-sources")
+@router.post("/news-sources/reset")
+async def reset_earth_news_sources(
+    _current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    return await reset_earth_news_sources_payload(db)
+
+
+@router.post("/news-sources/test")
+async def test_earth_news_source(
+    payload: EarthNewsSourceTestPayload,
+    _current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    return await test_news_source_config(payload.source, db=db)
+
+
+@router.get("/news-groups")
+async def list_earth_news_groups_admin(
+    _current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    return await list_news_groups(db)
+
+
+@router.post("/news-groups")
+async def create_earth_news_group_admin(
+    payload: EarthNewsManualGroupPayload,
+    _current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    try:
+        group = await create_manual_news_group(db, payload.name)
+    except ValueError as exc:
+        raise HTTPException(status_code=422, detail=str(exc)) from exc
+    await db.commit()
+    return {"status": "ok", "group": group}
+
+
+@router.put("/news-groups/{group_id:path}")
+async def rename_earth_news_group_admin(
+    group_id: str,
+    payload: EarthNewsManualGroupPayload,
+    _current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    try:
+        group = await rename_manual_news_group(db, group_id, payload.name)
+    except ValueError as exc:
+        raise HTTPException(status_code=422, detail=str(exc)) from exc
+    await db.commit()
+    await broadcast_manual_news_changed()
+    return {"status": "ok", "group": group}
+
+
+@router.get("/news-items")
+async def list_earth_news_items_admin(
+    page: int = Query(1, ge=1),
+    page_size: int = Query(50, ge=1, le=100),
+    source_type: str | None = Query(None),
+    region: str | None = Query(None),
+    category: str | None = Query(None),
+    status_filter: str | None = Query(None, alias="status"),
+    group_id: str | None = Query(None),
+    _current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    return await list_news_records(
+        db,
+        page=page,
+        page_size=page_size,
+        source_type=source_type,
+        region=region,
+        category=category,
+        status_filter=status_filter,
+        group_id=group_id,
+    )
+
+
+@router.post("/news-items")
+async def create_earth_news_item_admin(
+    payload: EarthNewsManualItemPayload,
+    _current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    try:
+        result = await upsert_manual_news_item(db, payload.model_dump())
+    except ValueError as exc:
+        raise HTTPException(status_code=422, detail=str(exc)) from exc
+    except PermissionError as exc:
+        raise HTTPException(status_code=403, detail=str(exc)) from exc
+    await db.commit()
+    await broadcast_manual_news_changed()
+    return {"status": "ok", "created": result.created, "queued": result.queued, "item": serialize_news_record(result.item)}
+
+
+@router.post("/news-items/import")
+async def import_earth_news_items_admin(
+    file: UploadFile = File(...),
+    group_id: str | None = Form(default=None),
+    _current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    try:
+        payload = await parse_manual_news_import_upload(await file.read())
+        result = await import_manual_news_items(db, payload, group_id=group_id)
+    except ValueError as exc:
+        raise HTTPException(status_code=422, detail=str(exc)) from exc
+    await db.commit()
+    await broadcast_manual_news_changed()
+    return {"status": "ok", **result}
+
+
+@router.put("/news-items/{item_id:path}")
+async def update_earth_news_item_admin(
+    item_id: str,
+    payload: EarthNewsManualItemPayload,
+    _current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    existing = await get_news_record_or_404(db, item_id)
+    if existing is None:
+        raise HTTPException(status_code=404, detail="News item not found.")
+    try:
+        result = await upsert_manual_news_item(
+            db,
+            payload.model_dump(),
+            item_id_override=item_id,
+        )
+    except ValueError as exc:
+        raise HTTPException(status_code=422, detail=str(exc)) from exc
+    except PermissionError as exc:
+        raise HTTPException(status_code=403, detail=str(exc)) from exc
+    await db.commit()
+    await broadcast_manual_news_changed()
+    return {"status": "ok", "created": result.created, "queued": result.queued, "item": serialize_news_record(result.item)}
+
+
+@router.delete("/news-items/{item_id:path}")
+async def delete_earth_news_item_admin(
+    item_id: str,
+    _current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    try:
+        deleted = await delete_manual_news_item(db, item_id)
+    except PermissionError as exc:
+        raise HTTPException(status_code=403, detail=str(exc)) from exc
+    if not deleted:
+        raise HTTPException(status_code=404, detail="News item not found.")
+    await db.commit()
+    await broadcast_manual_news_changed()
+    return {"status": "deleted", "id": item_id}
+
+
+@router.post("/news-items/{item_id:path}/reprocess")
+async def reprocess_earth_news_item_admin(
+    item_id: str,
+    _current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    existing = await get_news_record_or_404(db, item_id)
+    if existing is None:
+        raise HTTPException(status_code=404, detail="News item not found.")
+    try:
+        queued = await reprocess_manual_news_item(db, item_id)
+    except PermissionError as exc:
+        raise HTTPException(status_code=403, detail=str(exc)) from exc
+    await db.commit()
+    await broadcast_manual_news_changed()
+    return {"status": "queued" if queued else "not_queued", "queued": queued, "id": item_id}
+
+
 @router.get("/oobe-status")
 async def get_earth_oobe_status(
     current_user: User | None = Depends(_get_optional_current_user),

backend/app/api/v1/interactables.py

@@ -90,7 +90,7 @@ async def get_interactables_geojson(
         return interactables_to_geojson(items)
 
     payload = await get_or_build_layer_payload(
-        key=earth_layer_cache.key("interactables", layer=layer or "all"),
+        key=earth_layer_cache.key("interactables", interactable_layer=layer or "all"),
         policy=INTERACTABLE_CACHE_POLICY,
         builder=build_payload,
         response=response,

backend/app/api/v1/layers.py

@@ -105,7 +105,7 @@ def _parse_layer_bbox(bbox: str) -> tuple[float, float, float, float]:
 @router.get("/vessels/snapshot")
 async def get_vessel_layer_snapshot(
     bbox: str = Query(..., description="lon_min,lat_min,lon_max,lat_max"),
-    zoom: int = Query(..., ge=1, le=20),
+    zoom: float = Query(..., ge=1, le=20),
     limit: int = Query(DEFAULT_LAYER_LIMIT, ge=1),
     vessel_type: Optional[str] = Query(None, alias="type"),
     since_minutes: int = Query(60, ge=1, le=1440),

backend/app/api/v1/news.py

@@ -1,16 +1,92 @@
-from fastapi import APIRouter, Depends, Query
+from fastapi import APIRouter, Depends, HTTPException, Query
 from sqlalchemy.ext.asyncio import AsyncSession
 
 from app.db.session import get_db
-from app.services.earth_news import get_earth_news_payload
+from app.services.earth_news import (
+    ALLOWED_NEWS_CATEGORY_KEYS,
+    SUPPORTED_NEWS_LOCALES,
+    REGION_ANCHORS,
+    get_earth_news_payload,
+)
 
 router = APIRouter()
 
 
+def _parse_categories(raw: str | None) -> set[str] | None:
+    if raw is None or not raw.strip():
+        return None
+    requested = {item.strip().lower() for item in raw.split(",") if item.strip()}
+    invalid = sorted(requested - set(ALLOWED_NEWS_CATEGORY_KEYS))
+    if invalid:
+        raise HTTPException(
+            status_code=422,
+            detail={
+                "message": "Unsupported news categories.",
+                "invalid_categories": invalid,
+                "allowed_categories": list(ALLOWED_NEWS_CATEGORY_KEYS),
+            },
+        )
+    return requested or None
+
+
+def _parse_source_ids(raw: str | None) -> set[str] | None:
+    if raw is None or not raw.strip():
+        return None
+    return {item.strip() for item in raw.split(",") if item.strip()} or None
+
+
+def _parse_limit(raw: int | None) -> int:
+    if raw is None:
+        return 12
+    if raw < 1:
+        raise HTTPException(status_code=422, detail={"message": "News limit must be greater than 0."})
+    return min(raw, 100)
+
+
+def _parse_locale(raw: str | None) -> str:
+    if raw is None or not raw.strip():
+        return "zh-CN"
+    requested = raw.strip()
+    if requested not in SUPPORTED_NEWS_LOCALES:
+        raise HTTPException(
+            status_code=422,
+            detail={
+                "message": "Unsupported news locale.",
+                "invalid_locale": requested,
+                "allowed_locales": sorted(SUPPORTED_NEWS_LOCALES),
+            },
+        )
+    return requested
+
+
 @router.get("/earth-feed")
 async def get_earth_feed(
     lat: float | None = Query(None, description="Current Earth view center latitude"),
     lon: float | None = Query(None, description="Current Earth view center longitude"),
+    region: str | None = Query(None, description="Explicit Earth news region for UE/client integrations"),
+    categories: str | None = Query(None, description="Comma-separated news category keys"),
+    sources: str | None = Query(None, description="Comma-separated news source ids"),
+    limit: int | None = Query(None, description="Maximum news items to return, capped at 100"),
+    locale: str | None = Query(None, description="Display locale, zh-CN or en-US"),
     db: AsyncSession = Depends(get_db),
 ):
-    return await get_earth_news_payload(lat=lat, lon=lon, db=db)
+    normalized_region = region.strip().lower() if isinstance(region, str) and region.strip() else None
+    if normalized_region is not None and normalized_region not in REGION_ANCHORS:
+        raise HTTPException(
+            status_code=422,
+            detail={
+                "message": "Unsupported news region.",
+                "invalid_region": normalized_region,
+                "allowed_regions": list(REGION_ANCHORS.keys()),
+            },
+        )
+    return await get_earth_news_payload(
+        lat=lat,
+        lon=lon,
+        region=normalized_region,
+        categories=_parse_categories(categories),
+        source_ids=_parse_source_ids(sources),
+        limit=_parse_limit(limit),
+        locale=_parse_locale(locale),
+        db=db,
+    )

backend/app/api/v1/settings.py

@@ -3,15 +3,16 @@ from datetime import UTC, datetime
 import os
 from pathlib import Path
 from typing import Optional
+from urllib.parse import urlsplit
 
 from fastapi import APIRouter, Depends, HTTPException, Query, Request, status
-import httpx
 from pydantic import BaseModel, EmailStr, Field
 from dotenv import dotenv_values
 from sqlalchemy import select
 from sqlalchemy.ext.asyncio import AsyncSession
 
 from app.core.logging import get_logger
+from app.core.enums import ProviderApi, TVSourceType, UserRole
 from app.core.security import get_current_user
 from app.core.time import to_iso8601_utc
 from app.core.config import settings as app_settings
@@ -64,6 +65,7 @@ from app.services.llm_provider_catalog import (
     list_fallback_llm_provider_presets,
     refresh_llm_provider_preset,
 )
+from app.services.llm_model_catalog import catalog_error_message, fetch_model_catalog
 from app.services.scheduler import sync_datasource_job
 from app.services.tv_streams import DEFAULT_TV_SETTINGS, get_tv_settings_payload, normalize_tv_settings
 from app.services.persistent_logs import record_audit_log
@@ -73,7 +75,8 @@ router = APIRouter()
 logger = get_logger(__name__, service="api")
 AI_PROVIDER_QUICK_CONNECT_TIMEOUT_SECONDS = 5
 AI_CONNECTION_TEST_PROMPT_KEY = "ai.connection_test"
-SECRET_REVEAL_ROLES = {"admin", "super_admin"}
+SECRET_REVEAL_ROLES = {UserRole.ADMIN.value, UserRole.SUPER_ADMIN.value}
+LLM_PROVIDER_PRESET_CATEGORY_PREFIX = "llm_provider_preset:"
 
 DEFAULT_SETTINGS = {
     "system": {
@@ -236,7 +239,7 @@ class TVStreamSourceUpdate(BaseModel):
     provider: str = Field(default="Unknown", max_length=100)
     region: str = Field(default="Global", max_length=100)
     language: str = Field(default="und", max_length=32)
-    source_type: str = Field(default="iframe", pattern="^(iframe|hls|video|external|youtube)$")
+    source_type: TVSourceType = TVSourceType.IFRAME
     embed_url: str = ""
     stream_url: str = ""
     homepage_url: str = ""
@@ -279,7 +282,7 @@ class AIProviderIntegrationUpdate(BaseModel):
     service_token: Optional[str] = None
     default_provider: Optional[str] = None
     provider: str = Field(default="minimax", max_length=80)
-    provider_api: str = Field(default="anthropic-messages", max_length=80)
+    provider_api: ProviderApi = ProviderApi.ANTHROPIC_MESSAGES
     base_url: str = Field(default="", max_length=500)
     model: str = Field(default="", max_length=200)
     api_key: Optional[str] = None
@@ -344,7 +347,13 @@ class ExternalIntegrationsUpdate(BaseModel):
 
 
 def merge_with_defaults(category: str, payload: Optional[dict]) -> dict:
-    merged = deepcopy(DEFAULT_SETTINGS[category])
+    if category.startswith(LLM_PROVIDER_PRESET_CATEGORY_PREFIX):
+        defaults = get_fallback_llm_provider_preset(
+            category.removeprefix(LLM_PROVIDER_PRESET_CATEGORY_PREFIX)
+        )
+    else:
+        defaults = DEFAULT_SETTINGS[category]
+    merged = deepcopy(defaults)
     if payload:
         merged.update(payload)
     return merged
@@ -381,6 +390,7 @@ async def get_setting_payload(db: AsyncSession, category: str) -> dict:
 
 
 async def save_setting_payload(db: AsyncSession, category: str, payload: dict) -> dict:
+    merged = merge_with_defaults(category, payload)
     record = await get_setting_record(db, category)
     if record is None:
         record = SystemSetting(category=category, payload=payload)
@@ -390,7 +400,7 @@ async def save_setting_payload(db: AsyncSession, category: str, payload: dict) -
 
     await db.commit()
     await db.refresh(record)
-    return merge_with_defaults(category, record.payload)
+    return merged
 
 
 AI_PROVIDER_ENV_FILE = Path(__file__).resolve().parents[4] / "aiprovider" / ".env"
@@ -423,7 +433,7 @@ def _get_provider_preset(provider: str) -> dict:
     except ValueError:
         return {
             "provider": provider,
-            "provider_api": "openai-completions",
+            "provider_api": ProviderApi.OPENAI_COMPLETIONS.value,
             "base_url": "",
         "model": "",
         "models": [],
@@ -487,12 +497,12 @@ def _provider_defaults(provider: str) -> dict:
     preset = _get_provider_preset(provider)
     return {
         "provider": provider,
-        "provider_api": preset.get("provider_api") or "openai-completions",
+        "provider_api": preset.get("provider_api") or ProviderApi.OPENAI_COMPLETIONS.value,
         "base_url": preset.get("base_url") or "",
         "model": preset.get("model") or "",
         "api_key": "",
         "max_tokens": (
-            1200 if preset.get("provider_api") == "anthropic-messages" else 4096
+            1200 if preset.get("provider_api") == ProviderApi.ANTHROPIC_MESSAGES.value else 4096
         ),
         "anthropic_version": "2023-06-01",
         "model_provider_apis": preset.get("model_provider_apis") or {},
@@ -654,6 +664,12 @@ def _runtime_config_from_ai_payload(ai_payload: dict) -> dict:
         normalized_ai["providers"].get(default_provider) or _provider_defaults(default_provider)
     )
     api_key, _api_key_source = _resolve_provider_api_key(default_provider, provider_config)
+    model_provider_apis = {
+        **(provider_config.get("model_provider_apis") or {}),
+        **(_get_provider_preset(default_provider).get("model_provider_apis") or {}),
+    }
+    if default_provider == "openai" and urlsplit(provider_config.get("base_url") or "").hostname != "api.openai.com":
+        model_provider_apis = {}
     return {
         "service_url": normalized_ai.get("service_url") or app_settings.AI_PROVIDER_SERVICE_URL,
         "service_token": _resolve_service_token(normalized_ai)[0],
@@ -665,14 +681,13 @@ def _runtime_config_from_ai_payload(ai_payload: dict) -> dict:
         ),
         "llm_config": {
             "provider": default_provider,
-            "provider_api": provider_config.get("provider_api") or "anthropic-messages",
+            "provider_api": provider_config.get("provider_api") or ProviderApi.ANTHROPIC_MESSAGES.value,
             "base_url": provider_config.get("base_url") or "",
             "model": provider_config.get("model") or "",
             "api_key": api_key,
             "max_tokens": int(provider_config.get("max_tokens") or 1200),
             "anthropic_version": provider_config.get("anthropic_version") or "2023-06-01",
-            "model_provider_apis": provider_config.get("model_provider_apis") or {},
-            "preset_models": _get_provider_preset(default_provider).get("models") or [],
+            "model_provider_apis": model_provider_apis,
         },
     }
 
@@ -752,156 +767,35 @@ async def _validate_ai_provider_full_connection(ai_payload: dict) -> dict:
     }
 
 
-def _join_provider_url(base_url: str, path: str) -> str:
-    return f"{base_url.rstrip('/')}/{path.lstrip('/')}"
-
-
-def _extract_model_ids(payload: dict) -> list[str]:
-    data = payload.get("data") if isinstance(payload, dict) else None
-    if isinstance(data, list):
-        return [
-            str(item.get("id"))
-            for item in data
-            if isinstance(item, dict) and item.get("id")
-        ]
-    models = payload.get("models") if isinstance(payload, dict) else None
-    if isinstance(models, list):
-        return [
-            str(item.get("name") or item.get("model") or item.get("id") or item)
-            for item in models
-            if item
-        ]
-    return []
-
-
-def _contains_model(model_ids: list[str], model: str) -> bool:
-    normalized_model = model.strip().lower()
-    return any(str(item).strip().lower() == normalized_model for item in model_ids)
-
-
 async def _check_ai_provider_lightweight(llm_config: dict, timeout_seconds: int) -> dict:
     provider = _normalize_provider_id(llm_config.get("provider") or "")
-    configured_api = str(llm_config.get("provider_api") or "").strip() or "openai-completions"
+    provider_api = str(llm_config.get("provider_api") or ProviderApi.OPENAI_COMPLETIONS.value)
     model = str(llm_config.get("model") or "").strip()
-    base_url = str(llm_config.get("base_url") or "").strip().rstrip("/")
+    base_url = str(llm_config.get("base_url") or "").strip()
     api_key = str(llm_config.get("api_key") or "").strip()
-    provider_api = configured_api
-    model_provider_apis = llm_config.get("model_provider_apis")
-    if isinstance(model_provider_apis, dict):
-        provider_api = str(model_provider_apis.get(model) or provider_api)
-    preset_models = [
-        str(item)
-        for item in (llm_config.get("preset_models") or [])
-        if str(item).strip()
-    ]
-
-    if not provider or not base_url or not model:
-        return {
-            "success": False,
-            "connected": False,
-            "message": "当前 provider/base_url/model 未完整配置。",
-            "mode": "lightweight_config",
-        }
-    if provider_api != "ollama-generate" and not api_key:
-        return {
-            "success": False,
-            "connected": False,
-            "message": "当前 provider 未配置 API Key。",
-            "mode": "lightweight_config",
-        }
-
-    if provider == "opencode-go":
-        url = _join_provider_url(base_url, "/models")
-        headers = {"Authorization": f"Bearer {api_key}"}
-    elif provider_api == "ollama-generate":
-        url = _join_provider_url(base_url, "/api/tags")
-        headers: dict[str, str] = {}
-    elif provider_api == "openai-completions":
-        url = _join_provider_url(base_url, "/models")
-        headers = {"Authorization": f"Bearer {api_key}"}
-    elif provider_api == "anthropic-messages":
-        url = _join_provider_url(base_url, "/models")
-        headers = {
-            "x-api-key": api_key,
-            "anthropic-version": str(llm_config.get("anthropic_version") or "2023-06-01"),
-        }
-    else:
-        return {
-            "success": False,
-            "connected": False,
-            "message": f"当前 provider_api 不支持轻量连通性测试: {provider_api}",
-            "mode": "lightweight_unsupported",
-        }
-
+    result = {
+        "success": False, "connected": False, "mode": "lightweight_models",
+        "provider": provider, "model": model,
+    }
+    if not base_url or not model:
+        return {**result, "message": "当前 provider/base_url/model 未完整配置。"}
+    if provider_api != ProviderApi.OLLAMA_GENERATE.value and not api_key:
+        return {**result, "message": "当前 provider 未配置 API Key。"}
     try:
-        async with httpx.AsyncClient(timeout=min(timeout_seconds, AI_PROVIDER_QUICK_CONNECT_TIMEOUT_SECONDS)) as client:
-            response = await client.get(url, headers=headers)
-            response.raise_for_status()
-            payload = response.json()
-    except httpx.HTTPStatusError as exc:
-        detail = exc.response.text or exc.response.reason_phrase
-        if exc.response.status_code == 404 and _contains_model(preset_models, model):
-            return {
-                "success": True,
-                "connected": True,
-                "message": "轻量连通性测试通过；当前 provider 不提供可用的模型目录，已按内置模型预设确认。",
-                "mode": "lightweight_preset",
-                "provider": provider,
-                "provider_api": provider_api,
-                "model": model,
-                "url": url,
-            }
-        return {
-            "success": False,
-            "connected": False,
-            "message": f"轻量连通性测试失败: HTTP {exc.response.status_code} {detail}",
-            "mode": "lightweight_models",
-            "url": url,
-        }
+        catalog = await fetch_model_catalog(
+            provider, base_url, provider_api, api_key,
+            str(llm_config.get("anthropic_version") or "2023-06-01"), timeout_seconds,
+        )
     except Exception as exc:
-        return {
-            "success": False,
-            "connected": False,
-            "message": f"轻量连通性测试失败: {exc}",
-            "mode": "lightweight_models",
-            "url": url,
-        }
-
-    model_ids = _extract_model_ids(payload)
-    if model_ids and not _contains_model(model_ids, model):
-        if _contains_model(preset_models, model):
-            return {
-                "success": True,
-                "connected": True,
-                "message": "轻量连通性测试通过；provider 模型目录未返回当前别名，已按内置模型预设确认。",
-                "mode": "lightweight_models_with_preset_alias",
-                "provider": provider,
-                "provider_api": provider_api,
-                "model": model,
-                "models_count": len(model_ids),
-                "url": url,
-            }
-        return {
-            "success": False,
-            "connected": False,
-            "message": f"连接可用，但模型目录中没有当前模型: {model}",
-            "mode": "lightweight_models",
-            "provider": provider,
-            "model": model,
-            "models_count": len(model_ids),
-            "url": url,
-        }
-
+        return {**result, "message": catalog_error_message(exc)}
+    result.update({"url": catalog.url, "models_count": len(catalog.models)})
+    if model.casefold() not in {item.casefold() for item in catalog.models}:
+        return {**result, "message": f"模型目录查询成功，但当前模型不在目录中：{model}"}
+    model_apis = llm_config.get("model_provider_apis") or {}
     return {
-        "success": True,
-        "connected": True,
-        "message": "轻量连通性测试通过",
-        "mode": "lightweight_models",
-        "provider": provider,
-        "provider_api": provider_api,
-        "model": model,
-        "models_count": len(model_ids),
-        "url": url,
+        **result, "success": True, "connected": True,
+        "provider_api": model_apis.get(model) or provider_api,
+        "message": "模型目录查询成功，当前模型已找到；尚未执行生成调用。",
     }
 
 
@@ -1165,7 +1059,7 @@ async def serialize_external_integrations(db: AsyncSession) -> dict:
         api_key, api_key_source = _resolve_provider_api_key(provider_id, provider_config)
         providers_payload[provider_id] = {
             "provider": provider_id,
-            "provider_api": provider_config.get("provider_api") or "openai-completions",
+            "provider_api": provider_config.get("provider_api") or ProviderApi.OPENAI_COMPLETIONS.value,
             "base_url": provider_config.get("base_url") or "",
             "model": provider_config.get("model") or "",
             "api_key": _mask_secret(api_key, api_key_source),
@@ -1215,7 +1109,7 @@ async def serialize_external_integrations(db: AsyncSession) -> dict:
             "service_token": _mask_secret(*_resolve_service_token(normalized_ai)),
             "default_provider": default_provider,
             "provider": default_provider,
-            "provider_api": display_llm_config.get("provider_api") or "anthropic-messages",
+            "provider_api": display_llm_config.get("provider_api") or ProviderApi.ANTHROPIC_MESSAGES.value,
             "base_url": display_llm_config.get("base_url") or "https://api.minimaxi.com/anthropic",
             "model": display_llm_config.get("model") or "MiniMax-M2.7",
             "api_key": display_llm_config.get("api_key") or _mask_secret(None),
@@ -1458,7 +1352,7 @@ async def update_smtp_settings(
     current_user: User = Depends(get_current_user),
     db: AsyncSession = Depends(get_db),
 ):
-    if current_user.role not in ("admin", "super_admin"):
+    if current_user.role not in (UserRole.ADMIN.value, UserRole.SUPER_ADMIN.value):
         raise HTTPException(status_code=403, detail="Only administrators can change SMTP settings")
     current = await get_setting_payload(db, "smtp")
     merged = _build_smtp_payload(current, payload)
@@ -1472,7 +1366,7 @@ async def test_smtp_settings(
     current_user: User = Depends(get_current_user),
     db: AsyncSession = Depends(get_db),
 ):
-    if current_user.role not in ("admin", "super_admin"):
+    if current_user.role not in (UserRole.ADMIN.value, UserRole.SUPER_ADMIN.value):
         raise HTTPException(status_code=403, detail="Only administrators can test SMTP settings")
     from app.services.email import EmailError, send_email
 
@@ -1685,7 +1579,8 @@ async def connect_ai_provider_integration(
         )
         await emit_business_log(
             logger,
-            event="settings.ai_provider.connect.success",
+            event=("settings.ai_provider.connect.success" if lightweight_result["success"]
+                   else "settings.ai_provider.connect.failed"),
             message="AI provider connection test completed",
             category="ai",
             service="api",
@@ -1695,7 +1590,7 @@ async def connect_ai_provider_integration(
                 "provider": payload.provider,
                 "model": payload.model,
                 "configured": True,
-                "lightweight_status": lightweight_result.get("status"),
+                "connected": lightweight_result["connected"],
             },
         )
         return {
@@ -2040,8 +1935,18 @@ async def reset_provider_credential_guide(
 @router.get("/integrations/ai-provider/presets")
 async def get_ai_provider_presets(
     current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
 ):
-    return {"data": list_fallback_llm_provider_presets()}
+    presets = list_fallback_llm_provider_presets()
+    saved = await get_setting_payloads(
+        db, [f"{LLM_PROVIDER_PRESET_CATEGORY_PREFIX}{preset['provider']}" for preset in presets]
+    )
+    return {
+        "data": [
+            {**preset, **saved[f"{LLM_PROVIDER_PRESET_CATEGORY_PREFIX}{preset['provider']}"]}
+            for preset in presets
+        ]
+    }
 
 
 @router.post("/integrations/ai-provider/presets/{provider}/refresh")
@@ -2049,22 +1954,52 @@ async def refresh_ai_provider_preset(
     provider: str,
     current_user: User = Depends(get_current_user),
     db: AsyncSession = Depends(get_db),
+    payload: AIProviderIntegrationUpdate | None = None,
 ):
     try:
         provider_id = _normalize_provider_id(provider)
-        api_key = None
-        if provider_id == "opencode-go":
-            current_payload = await get_setting_payload(db, "external_integrations")
-            ai_payload = _normalize_ai_provider_payload(current_payload.get("ai_provider") or {})
-            provider_config = ai_payload["providers"].get(provider_id) or _provider_defaults(provider_id)
-            api_key, _api_key_source = _resolve_provider_api_key(provider_id, provider_config)
-        return {"data": await refresh_llm_provider_preset(provider_id, api_key=api_key)}
+        get_fallback_llm_provider_preset(provider_id)
     except ValueError as exc:
         raise HTTPException(status_code=404, detail=str(exc)) from exc
+
+    current_payload = await get_setting_payload(db, "external_integrations")
+    if payload is not None:
+        if _normalize_provider_id(payload.provider) != provider_id:
+            raise HTTPException(status_code=400, detail="刷新供应商与表单供应商不一致。")
+        ai_payload = _build_ai_provider_payload(current_payload, payload)
+    else:
+        ai_payload = _normalize_ai_provider_payload(current_payload.get("ai_provider") or {})
+    provider_config = ai_payload["providers"].get(provider_id) or _provider_defaults(provider_id)
+    api_key, _api_key_source = _resolve_provider_api_key(provider_id, provider_config)
+    if not api_key and provider_id not in {"ollama", "opencode-go", "openrouter"}:
+        raise HTTPException(
+            status_code=400, detail="请先配置当前供应商的 API Key，再刷新模型列表。",
+        )
+    try:
+        refreshed = await refresh_llm_provider_preset(
+            provider_id, api_key=api_key, base_url=provider_config.get("base_url"),
+            provider_api=provider_config.get("provider_api"),
+            anthropic_version=provider_config.get("anthropic_version") or "2023-06-01",
+        )
     except Exception as exc:
-        fallback = get_fallback_llm_provider_preset(provider)
-        fallback["refresh_error"] = str(exc)
-        return {"data": fallback}
+        logger.warning(
+            "LLM provider catalog refresh failed",
+            extra={
+                "event": "settings.ai_provider.catalog.failed",
+                "context": {
+                    "provider": provider_id,
+                    "error_type": type(exc).__name__,
+                },
+            },
+        )
+        raise HTTPException(
+            status_code=502,
+            detail=f"{catalog_error_message(exc)} 已保留上次模型列表。",
+        ) from exc
+
+    refreshed["refreshed_at"] = to_iso8601_utc(datetime.now(UTC))
+    await save_setting_payload(db, f"{LLM_PROVIDER_PRESET_CATEGORY_PREFIX}{provider_id}", refreshed)
+    return {"data": refreshed}
 
 
 @router.put("/integrations")

backend/app/api/v1/system_control.py

@@ -1,16 +1,17 @@
 from __future__ import annotations
 
 import os
+import secrets
 import subprocess
 import sys
 
 from datetime import datetime
 
-from fastapi import APIRouter, Depends, HTTPException, Query, Request, status
+from fastapi import APIRouter, Depends, Header, HTTPException, Query, Request, status
 from pydantic import BaseModel
 from sqlalchemy.ext.asyncio import AsyncSession
 
-from app.core.config import ROOT_DIR
+from app.core.config import ROOT_DIR, settings
 from app.core.security import get_current_user
 from app.db.session import get_db
 from app.models.user import User
@@ -37,6 +38,9 @@ from app.services.system_logs import (
     normalize_log_level,
     read_database_log_snapshot,
     read_log_snapshot,
+    read_observability_group_events,
+    read_observability_groups,
+    read_observability_raw_events,
 )
 from app.services.earth_layer_cache import earth_layer_cache
 
@@ -108,12 +112,34 @@ class EarthClientLogEventCreate(BaseModel):
     url: str | None = None
     module: str | None = None
     detail: str | None = None
+    fingerprint: str | None = None
+    occurrence_count: int = 1
+    metadata: dict[str, object] | None = None
 
 
 class EarthClientLogEventResponse(BaseModel):
     accepted: bool
     source_id: str
     level: str
+    fingerprint: str | None = None
+
+
+class ServiceLogEventCreate(BaseModel):
+    source: str = "ai-provider"
+    service: str = "ai-provider"
+    module: str | None = None
+    category: str | None = None
+    event: str = "service.runtime_log"
+    level: str = "error"
+    message: str
+    fingerprint: str | None = None
+    occurrence_count: int = 1
+    request_id: str | None = None
+    trace_id: str | None = None
+    task_id: str | None = None
+    source_id: int | str | None = None
+    provider: str | None = None
+    context: dict[str, object] | None = None
 
 
 async def ingest_client_log_event(
@@ -136,6 +162,9 @@ async def ingest_client_log_event(
             "url": payload.url or "",
             "module": payload.module or "",
             "detail": payload.detail or "",
+            "fingerprint": payload.fingerprint or "",
+            "occurrence_count": max(1, int(payload.occurrence_count or 1)),
+            "metadata": payload.metadata or {},
         },
     )
     await record_system_log(
@@ -151,9 +180,36 @@ async def ingest_client_log_event(
             "detail": payload.detail or "",
             "module": payload.module or "",
             "client_ip": request.client.host if request.client else "",
+            "metadata": payload.metadata or {},
         },
+        fingerprint=payload.fingerprint,
+        occurrence_count=max(1, int(payload.occurrence_count or 1)),
     )
-    return EarthClientLogEventResponse(accepted=True, source_id=source_id, level=normalized_level)
+    return EarthClientLogEventResponse(accepted=True, source_id=source_id, level=normalized_level, fingerprint=payload.fingerprint)
+
+
+def require_observability_ingest_token(
+    authorization: str | None,
+    ingest_token: str | None,
+) -> None:
+    expected_token = settings.OBSERVABILITY_INGEST_TOKEN.strip()
+    if not expected_token:
+        raise HTTPException(
+            status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
+            detail="Observability service ingestion is not configured",
+        )
+    provided = ""
+    if ingest_token:
+        provided = ingest_token.strip()
+    elif authorization:
+        scheme, _, token = authorization.partition(" ")
+        if scheme.lower() == "bearer":
+            provided = token.strip()
+    if not provided or not secrets.compare_digest(provided, expected_token):
+        raise HTTPException(
+            status_code=status.HTTP_403_FORBIDDEN,
+            detail="Invalid observability ingestion token",
+        )
 
 
 class EarthLayerCacheStatusResponse(BaseModel):
@@ -378,6 +434,118 @@ async def get_system_log_sources(
     }
 
 
+@router.get("/logs/observability/groups")
+async def get_observability_log_groups(
+    limit: int = DEFAULT_LOG_LINE_LIMIT,
+    level: str = "all",
+    levels: str | None = Query(None, description="Comma-separated log levels"),
+    start_date: str | None = Query(None, description="Filter logs from this date (YYYY-MM-DD)"),
+    end_date: str | None = Query(None, description="Filter logs until this date (YYYY-MM-DD)"),
+    search: str | None = Query(None, description="Case-insensitive substring search"),
+    current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    ensure_super_admin(current_user)
+    if limit < 1 or limit > MAX_LOG_LINE_LIMIT:
+        raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail=f"limit must be between 1 and {MAX_LOG_LINE_LIMIT}")
+    normalized_start_date = validate_log_date(start_date, "start_date")
+    normalized_end_date = validate_log_date(end_date, "end_date")
+    if normalized_start_date and normalized_end_date and normalized_start_date > normalized_end_date:
+        raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail="start_date must be earlier than or equal to end_date")
+    return await read_observability_groups(
+        limit=limit,
+        level=level,
+        levels=levels,
+        start_date=normalized_start_date,
+        end_date=normalized_end_date,
+        search=search,
+        db=db,
+    )
+
+
+@router.get("/logs/observability/groups/{fingerprint}/events")
+async def get_observability_group_events(
+    fingerprint: str,
+    limit: int = DEFAULT_LOG_LINE_LIMIT,
+    current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    ensure_super_admin(current_user)
+    if limit < 1 or limit > MAX_LOG_LINE_LIMIT:
+        raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail=f"limit must be between 1 and {MAX_LOG_LINE_LIMIT}")
+    payload = await read_observability_group_events(fingerprint, limit=limit, db=db)
+    if payload is None:
+        raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Observability group not found")
+    return payload
+
+
+@router.get("/logs/observability/raw")
+async def get_observability_raw_events(
+    limit: int = DEFAULT_LOG_LINE_LIMIT,
+    level: str = "all",
+    levels: str | None = Query(None, description="Comma-separated log levels"),
+    start_date: str | None = Query(None, description="Filter logs from this date (YYYY-MM-DD)"),
+    end_date: str | None = Query(None, description="Filter logs until this date (YYYY-MM-DD)"),
+    search: str | None = Query(None, description="Case-insensitive substring search"),
+    current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    ensure_super_admin(current_user)
+    if limit < 1 or limit > MAX_LOG_LINE_LIMIT:
+        raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail=f"limit must be between 1 and {MAX_LOG_LINE_LIMIT}")
+    normalized_start_date = validate_log_date(start_date, "start_date")
+    normalized_end_date = validate_log_date(end_date, "end_date")
+    return await read_observability_raw_events(
+        limit=limit,
+        level=level,
+        levels=levels,
+        start_date=normalized_start_date,
+        end_date=normalized_end_date,
+        search=search,
+        db=db,
+    )
+
+
+@router.post("/logs/service", response_model=EarthClientLogEventResponse)
+async def ingest_service_log(
+    payload: ServiceLogEventCreate,
+    authorization: str | None = Header(default=None),
+    ingest_token: str | None = Header(default=None, alias="X-Planet-Observability-Token"),
+):
+    require_observability_ingest_token(authorization, ingest_token)
+    normalized_level = normalize_log_level(payload.level)
+    source = (payload.source or "ai-provider").strip() or "ai-provider"
+    context = dict(payload.context or {})
+    if payload.request_id:
+        context["request_id"] = payload.request_id
+    if payload.trace_id:
+        context["trace_id"] = payload.trace_id
+    if payload.task_id:
+        context["task_id"] = payload.task_id
+    if payload.source_id is not None:
+        context["source_id"] = payload.source_id
+    if payload.provider:
+        context["provider"] = payload.provider
+    await record_system_log(
+        source=source,
+        service=(payload.service or source).strip() or source,
+        module=payload.module or source,
+        event=(payload.event or "service.runtime_log").strip() or "service.runtime_log",
+        level=normalized_level,
+        message=payload.message,
+        category=payload.category or "service-runtime",
+        context=context,
+        fingerprint=payload.fingerprint,
+        occurrence_count=max(1, int(payload.occurrence_count or 1)),
+    )
+    return EarthClientLogEventResponse(
+        accepted=True,
+        source_id=source,
+        level=normalized_level,
+        fingerprint=payload.fingerprint,
+    )
+
+
 @router.get("/logs/{source_id}", response_model=SystemLogSnapshotResponse)
 async def get_system_log_snapshot(
     source_id: str,

backend/app/api/v1/tasks.py

@@ -29,7 +29,8 @@ async def list_tasks(
         SELECT ct.id, ct.datasource_id, ds.name as datasource_name, ct.status,
                ct.started_at, ct.completed_at, ct.records_processed, ct.error_message,
                ct.phase, ct.phase_progress, ct.phase_message, ct.phase_current,
-               ct.phase_total, ct.phase_unit, ct.total_records, ct.progress
+               ct.phase_total, ct.phase_unit, ct.total_records, ct.progress,
+               ct.task_type, ct.source, ds.source as datasource_source
         FROM collection_tasks ct
         JOIN data_sources ds ON ct.datasource_id = ds.id
         WHERE 1=1
@@ -39,12 +40,19 @@ async def list_tasks(
 
     if datasource_id:
         query += " AND ct.datasource_id = :datasource_id"
-        count_query += " WHERE ct.datasource_id = :datasource_id"
+        count_query += " AND ct.datasource_id = :datasource_id"
         params["datasource_id"] = datasource_id
     if status:
-        query += " AND ct.status = :status"
-        count_query += " AND ct.status = :status"
-        params["status"] = status
+        statuses = [item.strip() for item in status.split(",") if item.strip()]
+        if len(statuses) > 1:
+            placeholders = ", ".join(f":status_{index}" for index, _item in enumerate(statuses))
+            query += f" AND ct.status IN ({placeholders})"
+            count_query += f" AND ct.status IN ({placeholders})"
+            params.update({f"status_{index}": item for index, item in enumerate(statuses)})
+        else:
+            query += " AND ct.status = :status"
+            count_query += " AND ct.status = :status"
+            params["status"] = statuses[0] if statuses else status
 
     query += f" ORDER BY ct.created_at DESC LIMIT {page_size} OFFSET {offset}"
 
@@ -76,6 +84,9 @@ async def list_tasks(
                 "phase_unit": t[13],
                 "total_records": t[14],
                 "progress": t[15],
+                "task_type": t[16],
+                "source": t[17] or t[18],
+                "datasource_source": t[18],
             }
             for t in tasks
         ],

backend/app/api/v1/tv.py

@@ -1,3 +1,4 @@
+import re
 from urllib.parse import quote, urljoin
 
 import httpx
@@ -6,16 +7,41 @@ from fastapi.responses import Response
 from sqlalchemy.ext.asyncio import AsyncSession
 
 from app.db.session import get_db
+from app.services.tv_catalog import get_tv_catalog_page
 from app.services.tv_streams import get_public_tv_payload, is_allowed_tv_proxy_url
 
 router = APIRouter()
 
+_HLS_URI_ATTRIBUTE_RE = re.compile(r'URI="([^"]+)"')
+
+
+def _proxied_tv_url(url: str) -> str:
+    return f"/api/v1/tv/proxy?url={quote(url, safe='')}"
+
+
+def _rewrite_hls_uri_attributes(line: str, *, base_url: str) -> str:
+    def replace(match: re.Match[str]) -> str:
+        uri = match.group(1)
+        absolute_url = urljoin(base_url, uri)
+        return f'URI="{_proxied_tv_url(absolute_url)}"'
+
+    return _HLS_URI_ATTRIBUTE_RE.sub(replace, line)
+
+
+def _should_strip_hls_metadata_line(line: str) -> bool:
+    normalized = line.strip().upper()
+    return normalized.startswith("#EXT-X-MEDIA:") and "TYPE=SUBTITLES" in normalized
+
 
 @router.get("/streams")
 async def list_public_tv_streams(
+    offset: int = Query(0, ge=0),
+    limit: int = Query(50, ge=1, le=100),
+    q: str = Query("", max_length=200),
+    selected_id: str | None = Query(None, max_length=200),
     db: AsyncSession = Depends(get_db),
 ):
-    return await get_public_tv_payload(db)
+    return await get_tv_catalog_page(db, offset=offset, limit=limit, q=q, selected_id=selected_id)
 
 
 @router.get("/proxy")
@@ -56,11 +82,16 @@ async def proxy_tv_stream(
         rewritten_lines: list[str] = []
         for line in manifest_text.splitlines():
             stripped = line.strip()
-            if not stripped or stripped.startswith("#"):
+            if not stripped:
                 rewritten_lines.append(line)
                 continue
+            if stripped.startswith("#"):
+                if _should_strip_hls_metadata_line(line):
+                    continue
+                rewritten_lines.append(_rewrite_hls_uri_attributes(line, base_url=response_url))
+                continue
             absolute_url = urljoin(response_url, stripped)
-            rewritten_lines.append(f"/api/v1/tv/proxy?url={quote(absolute_url, safe='')}")
+            rewritten_lines.append(_proxied_tv_url(absolute_url))
         return Response(
             content="\n".join(rewritten_lines),
             media_type="application/vnd.apple.mpegurl",

backend/app/api/v1/users.py

@@ -5,6 +5,7 @@ from fastapi import APIRouter, Depends, HTTPException, status
 from sqlalchemy.ext.asyncio import AsyncSession
 from sqlalchemy import text
 
+from app.core.enums import UserRole
 from app.core.security import get_current_user, get_password_hash
 from app.db.session import get_db
 from app.models.user import User
@@ -13,6 +14,8 @@ from app.schemas.user import UserCreate, UserUpdate
 router = APIRouter()
 
 VALID_GATEKEEPER_GROUPS = {"docs_user", "docs_developer", "docs_admin"}
+ADMIN_ROLES = [UserRole.SUPER_ADMIN.value, UserRole.ADMIN.value]
+SUPER_ADMIN_ROLES = [UserRole.SUPER_ADMIN.value]
 
 
 def check_permission(current_user: User, required_roles: List[str]) -> bool:
@@ -32,7 +35,7 @@ async def list_users(
     current_user: User = Depends(get_current_user),
     db: AsyncSession = Depends(get_db),
 ):
-    if not check_permission(current_user, ["super_admin", "admin"]):
+    if not check_permission(current_user, ADMIN_ROLES):
         raise HTTPException(
             status_code=status.HTTP_403_FORBIDDEN,
             detail="Insufficient permissions",
@@ -91,7 +94,7 @@ async def get_user(
     current_user: User = Depends(get_current_user),
     db: AsyncSession = Depends(get_db),
 ):
-    if not check_permission(current_user, ["super_admin", "admin"]) and current_user.id != user_id:
+    if not check_permission(current_user, ADMIN_ROLES) and current_user.id != user_id:
         raise HTTPException(
             status_code=status.HTTP_403_FORBIDDEN,
             detail="Insufficient permissions",
@@ -128,7 +131,7 @@ async def create_user(
     current_user: User = Depends(get_current_user),
     db: AsyncSession = Depends(get_db),
 ):
-    if not check_permission(current_user, ["super_admin"]):
+    if not check_permission(current_user, SUPER_ADMIN_ROLES):
         raise HTTPException(
             status_code=status.HTTP_403_FORBIDDEN,
             detail="Only super_admin can create users",
@@ -196,18 +199,18 @@ async def update_user(
     current_user: User = Depends(get_current_user),
     db: AsyncSession = Depends(get_db),
 ):
-    if not check_permission(current_user, ["super_admin", "admin"]) and current_user.id != user_id:
+    if not check_permission(current_user, ADMIN_ROLES) and current_user.id != user_id:
         raise HTTPException(
             status_code=status.HTTP_403_FORBIDDEN,
             detail="Insufficient permissions",
         )
 
-    if not check_permission(current_user, ["super_admin"]) and user_data.role is not None:
+    if not check_permission(current_user, SUPER_ADMIN_ROLES) and user_data.role is not None:
         raise HTTPException(
             status_code=status.HTTP_403_FORBIDDEN,
             detail="Only super_admin can change user role",
         )
-    if not check_permission(current_user, ["super_admin"]) and user_data.gatekeeper_groups is not None:
+    if not check_permission(current_user, SUPER_ADMIN_ROLES) and user_data.gatekeeper_groups is not None:
         raise HTTPException(
             status_code=status.HTTP_403_FORBIDDEN,
             detail="Only super_admin can change Gatekeeper groups",
@@ -260,7 +263,7 @@ async def delete_user(
     current_user: User = Depends(get_current_user),
     db: AsyncSession = Depends(get_db),
 ):
-    if not check_permission(current_user, ["super_admin"]):
+    if not check_permission(current_user, SUPER_ADMIN_ROLES):
         raise HTTPException(
             status_code=status.HTTP_403_FORBIDDEN,
             detail="Only super_admin can delete users",

backend/app/api/v1/vessels.py

@@ -1,4 +1,4 @@
-"""Bounded vessel snapshot APIs for viewport-first consumers."""
+"""Bounded vessel snapshot APIs backed by the latest vessel state table."""
 
 from typing import Optional
 
@@ -14,8 +14,8 @@ router = APIRouter()
 
 @router.get("/snapshot")
 async def get_vessel_snapshot(
-    bbox: Optional[str] = Query(None, description="Viewport bbox as lon_min,lat_min,lon_max,lat_max"),
-    zoom: int = Query(..., ge=1, le=20, description="Current map zoom level"),
+    bbox: Optional[str] = Query(None, description="Snapshot bbox as lon_min,lat_min,lon_max,lat_max"),
+    zoom: float = Query(..., ge=1, le=20, description="Current map zoom level"),
     type: Optional[str] = Query(
         None,
         description="Comma-separated vessel types: cargo,tanker,passenger,fishing,military,other",

backend/app/api/v1/visualization.py

@@ -17,15 +17,18 @@ from sqlalchemy.ext.asyncio import AsyncSession
 from sqlalchemy import select, func
 from typing import List, Dict, Any, Optional
 
+from app.core.compute_types import COMPUTE_SITE_TYPES, COMPUTE_SOURCES, compute_site_type, compute_record_filter, compute_site_type_expression
 from app.core.collected_data_fields import get_record_field
+from app.core.enums import BGPStatus
 from app.core.satellite_tle import build_tle_lines_from_elements
 from app.core.time import to_iso8601_utc
 from app.db.session import get_db
+from app.core.satellite_presentation import satellite_constellation_group as _normalize_satellite_constellation_group
 from app.models.bgp_anomaly import BGPAnomaly
 from app.models.bgp_incident import BGPIncident
 from app.models.bgp_observation import BGPObservation
 from app.models.collected_data import CollectedData
-from app.models.vessel import AISSourceHealth, VesselPosition, VesselStatic
+from app.models.vessel import AISSourceHealth, VesselCurrentState, VesselPosition, VesselStatic
 from app.services.bgp_collectors import build_bgp_collector_coverage
 from app.services.cable_graph import build_graph_from_data, CableGraph, haversine_distance
 from app.services.compute_center_locations import (
@@ -47,11 +50,10 @@ from app.services.location.llm_fallback import (
 from app.services.persistent_logs import record_system_log
 from app.services.vessel_ais_aggregation import (
     build_field_conflict_candidates,
-    count_unique_raw_vessel_mmsi,
     get_aggregated_vessel,
     get_aggregated_vessel_track,
     get_aggregated_vessels,
-    get_aggregated_vessels_snapshot,
+    get_current_vessels_snapshot,
     get_vessel_conflict_records,
     get_vessel_raw_observations,
     MAX_SNAPSHOT_LIMIT,
@@ -75,7 +77,6 @@ TERRAIN_TILE_BATCH_MAX_ITEMS = 128
 TERRAIN_TILE_BATCH_CONCURRENCY = 16
 _terrain_tile_cache: OrderedDict[tuple[int, int, int], tuple[bytes, str, dict[str, str]]] = OrderedDict()
 VESSEL_NAME_FALLBACK_PATTERN = re.compile(r"^mmsi\s*\d+$", re.IGNORECASE)
-VESSEL_SNAPSHOT_LEGACY_FALLBACK_ENABLED = True
 SECONDS_PER_MINUTE = 60
 BYTES_PER_MIB = 1024 * 1024
 CABLE_CACHE_FRESH_SECONDS = 6 * 60 * SECONDS_PER_MINUTE
@@ -338,22 +339,6 @@ def convert_satellite_to_geojson(records: List[CollectedData]) -> Dict[str, Any]
     return {"type": "FeatureCollection", "features": features}
 
 
-def _normalize_satellite_constellation_group(
-    raw_group: Any,
-    name: Optional[str],
-) -> Optional[str]:
-    normalized_group = str(raw_group or "").strip().lower()
-    if normalized_group:
-        return normalized_group
-
-    normalized_name = str(name or "").strip().upper()
-    if normalized_name.startswith("STARLINK"):
-        return "starlink"
-    if normalized_name.startswith("IRIDIUM"):
-        return "iridium-next"
-
-    return None
-
 
 def _get_satellite_footprint_policy(constellation_group: Optional[str]) -> str:
     if constellation_group == "starlink":
@@ -506,16 +491,19 @@ async def _count_current_or_latest_task_data(
 async def _load_current_collected_data_by_sources(
     db: AsyncSession,
     sources: List[str],
+    *, include_compute_types: bool = False,
 ) -> Dict[str, List[CollectedData]]:
     if not sources:
         return {}
 
     stmt = (
         select(CollectedData)
-        .where(CollectedData.source.in_(sources))
+        .where(compute_record_filter(CollectedData) if include_compute_types else CollectedData.source.in_(sources))
         .where(CollectedData.is_current.is_(True))
         .order_by(CollectedData.source.asc(), CollectedData.id.desc())
     )
+    if include_compute_types:
+        stmt = stmt.where(CollectedData.deleted_at.is_(None))
     result = await db.execute(stmt)
 
     grouped_records: Dict[str, List[CollectedData]] = {source: [] for source in sources}
@@ -645,7 +633,9 @@ def _normalize_capacity_band(capacity_value: Optional[float], capacity_unit: str
         return "unknown"
 
     unit = str(capacity_unit or "").strip().lower()
-    if unit in {"pflop/s", "pflops", "pflop"}:
+    if unit in {"flop/s", "flops"}:
+        normalized_tflops = capacity_value / 1e12
+    elif unit in {"pflop/s", "pflops", "pflop"}:
         normalized_tflops = capacity_value * 1000
     elif unit in {"gflop/s", "gflops", "gflop"}:
         normalized_tflops = capacity_value
@@ -673,15 +663,15 @@ def convert_compute_centers_to_geojson(records: List[CollectedData]) -> Dict[str
     """
     features: List[Dict[str, Any]] = []
     unresolved: List[Dict[str, Any]] = []
+    withheld_count = 0
 
     for record in records:
         metadata = record.extra_data or {}
+        if metadata.get("compute_contract") == "v1" and metadata.get("anonymous"):
+            withheld_count += 1
+            continue
         result = resolve_compute_center_location_full(record, metadata)
-        site_type = (
-            "supercomputer"
-            if record.source == "top500" or record.data_type == "supercomputer"
-            else "gpu_cluster"
-        )
+        site_type = compute_site_type(record)
 
         if not result.is_resolved:
             diagnostic = result.diagnostic or ResolutionDiagnostic(
@@ -707,7 +697,10 @@ def convert_compute_centers_to_geojson(records: List[CollectedData]) -> Dict[str
         latitude = location.latitude
         longitude = location.longitude
 
-        if site_type == "supercomputer":
+        if metadata.get("compute_contract") == "v1":
+            capacity_value = _parse_float(get_record_field(record, "value"))
+            capacity_unit = str(get_record_field(record, "unit") or "FLOP/s")
+        elif site_type == "supercomputer":
             capacity_value = _parse_float(get_record_field(record, "rmax"))
             capacity_unit = "GFlops"
         else:
@@ -768,7 +761,7 @@ def convert_compute_centers_to_geojson(records: List[CollectedData]) -> Dict[str
             }
         )
 
-    return {"type": "FeatureCollection", "features": features, "unresolved": unresolved}
+    return {"type": "FeatureCollection", "features": features, "unresolved": unresolved, "withheld_count": withheld_count}
 
 
 VESSEL_TYPE_FILTERS = {
@@ -839,9 +832,14 @@ def convert_aggregated_vessels_to_geojson(vessels: List[dict[str, Any]]) -> Dict
             continue
         source_summary = {}
         for source, summary in (vessel.get("source_summary") or {}).items():
+            latest_observed_at = summary.get("latest_observed_at")
             source_summary[source] = {
                 **summary,
-                "latest_observed_at": to_iso8601_utc(summary.get("latest_observed_at")),
+                "latest_observed_at": (
+                    to_iso8601_utc(latest_observed_at)
+                    if isinstance(latest_observed_at, datetime)
+                    else latest_observed_at
+                ),
             }
         props = {
             "mmsi": vessel["mmsi"],
@@ -1075,7 +1073,7 @@ async def build_vessel_snapshot_response(
     db: AsyncSession,
     *,
     bbox: tuple[float, float, float, float] | None,
-    zoom: int | None,
+    zoom: float | None,
     type_filter: str | None,
     limit: int | None,
     since_minutes: int = 60,
@@ -1117,6 +1115,7 @@ async def build_vessel_snapshot_response(
     safe_limit = _safe_vessel_limit(limit)
     safe_since_minutes = min(max(int(since_minutes or 60), 1), 1440)
     observed_since = datetime.now(UTC) - timedelta(minutes=safe_since_minutes)
+    snapshot_started_at = datetime.now(UTC)
     features, diagnostics = await _load_raw_vessel_snapshot_features(
         db,
         bbox=bbox,
@@ -1133,6 +1132,7 @@ async def build_vessel_snapshot_response(
         "features": features,
         "count": len(features),
         "stats": _build_vessel_stats(features),
+        "generated_at": to_iso8601_utc(snapshot_started_at),
         "diagnostics": {
             **diagnostics,
             "filtered_count": len(features),
@@ -1949,9 +1949,10 @@ async def _build_compute_centers_geojson(
     records_by_source = await _load_current_collected_data_by_sources(
         db,
         ["top500", "epoch_ai_gpu"],
+        include_compute_types=True,
     )
     records = _filter_known_records(
-        records_by_source.get("top500", []) + records_by_source.get("epoch_ai_gpu", []),
+        [record for group in records_by_source.values() for record in group],
     )
     if limit is not None:
         records = records[:limit]
@@ -1966,6 +1967,7 @@ async def _build_compute_centers_geojson(
                 "total": 0,
                 "supercomputers": 0,
                 "gpu_clusters": 0,
+                "by_type": dict.fromkeys(COMPUTE_SITE_TYPES, 0),
                 "unresolved": 0,
             },
         }
@@ -2008,6 +2010,7 @@ async def _build_compute_centers_geojson(
         "count": len(sanitized_features),
         "stats": {
             "total": len(sanitized_features),
+            "by_type": {kind: sum(feature["properties"]["site_type"] == kind for feature in sanitized_features) for kind in COMPUTE_SITE_TYPES},
             "supercomputers": sum(
                 1 for feature in sanitized_features
                 if feature.get("properties", {}).get("site_type") == "supercomputer"
@@ -2324,7 +2327,8 @@ async def _load_compute_center_record(db: AsyncSession, source_id: str) -> Colle
     stmt = (
         select(CollectedData)
         .where(CollectedData.source_id == source_id)
-        .where(CollectedData.source.in_(["top500", "epoch_ai_gpu"]))
+        .where(compute_record_filter(CollectedData))
+        .where(CollectedData.deleted_at.is_(None))
         .order_by(CollectedData.is_current.desc(), CollectedData.id.desc())
         .limit(1)
     )
@@ -2340,58 +2344,35 @@ async def _load_raw_vessel_snapshot_features(
     observed_since: datetime,
 ) -> tuple[list[dict[str, Any]], dict[str, Any]]:
     if bbox is None:
-        aggregated_vessels = await get_aggregated_vessels(
-            db,
-            limit=limit,
-            observed_since=observed_since,
-        )
-    else:
-        aggregated_vessels = await get_aggregated_vessels_snapshot(
-            db,
-            bbox=bbox,
-            limit=limit,
-            observed_since=observed_since,
-        )
-    raw_geojson = convert_aggregated_vessels_to_geojson(aggregated_vessels)
-    raw_features = raw_geojson.get("features", [])
-    features = raw_features
-    legacy_features: list[dict[str, Any]] = []
-    legacy_fallback_used = False
-    if not raw_features and VESSEL_SNAPSHOT_LEGACY_FALLBACK_ENABLED:
-        legacy_features = await _load_legacy_vessel_snapshot_features(
-            db,
-            bbox=bbox,
-            limit=limit,
-        )
-        features, _merge_diagnostics = _merge_vessel_features(raw_features, legacy_features)
-        legacy_fallback_used = bool(legacy_features)
-
+        return [], {
+            "source": "vessel_current_state",
+            "current_state_count": 0,
+            "final_unique_mmsi": 0,
+        }
+    current_vessels = await get_current_vessels_snapshot(
+        db,
+        bbox=bbox,
+        limit=limit,
+        observed_since=observed_since,
+    )
+    features = convert_aggregated_vessels_to_geojson(current_vessels).get("features", [])
+    unique_mmsi = len(
+        {
+            key
+            for key in (_feature_mmsi_key(feature) for feature in features)
+            if key is not None
+        }
+    )
     return features, {
-        "raw_feature_count": len(raw_features),
-        "raw_unique_mmsi": len(
-            {
-                key
-                for key in (_feature_mmsi_key(feature) for feature in raw_features)
-                if key is not None
-            }
-        ),
-        "legacy_feature_count": len(legacy_features),
-        "legacy_backfilled_mmsi": len(
-            {
-                key
-                for key in (_feature_mmsi_key(feature) for feature in legacy_features)
-                if key is not None
-            }
-        ),
-        "legacy_fallback_enabled": VESSEL_SNAPSHOT_LEGACY_FALLBACK_ENABLED,
-        "legacy_fallback_used": legacy_fallback_used,
-        "final_unique_mmsi": len(
-            {
-                key
-                for key in (_feature_mmsi_key(feature) for feature in features)
-                if key is not None
-            }
-        ),
+        "source": "vessel_current_state",
+        "current_state_count": len(features),
+        "final_unique_mmsi": unique_mmsi,
+        "raw_feature_count": 0,
+        "raw_unique_mmsi": 0,
+        "legacy_feature_count": 0,
+        "legacy_backfilled_mmsi": 0,
+        "legacy_fallback_enabled": False,
+        "legacy_fallback_used": False,
     }
 
 @router.get("/vessels/custom-supplements")
@@ -2746,6 +2727,25 @@ async def get_visualization_geo_summary(db: AsyncSession = Depends(get_db), resp
     )
 
 
+async def _compute_facility_counts(db: AsyncSession) -> dict[str, int]:
+    kind = compute_site_type_expression(CollectedData)
+    rows = (await db.execute(select(CollectedData.source, kind, func.count(CollectedData.id))
+        .where(compute_record_filter(CollectedData), CollectedData.is_current.is_(True),
+               CollectedData.deleted_at.is_(None))
+        .group_by(CollectedData.source, kind))).all()
+    counts = dict.fromkeys(COMPUTE_SITE_TYPES, 0)
+    present_sources = set()
+    for source, site_type, count in rows:
+        present_sources.add(source)
+        counts[site_type] += int(count or 0)
+    # Retain the existing last-task fallback for the two collected catalogs.
+    for source in COMPUTE_SOURCES:
+        if source not in present_sources:
+            key = "supercomputer" if source == "top500" else "gpu_cluster"
+            counts[key] += await _count_current_or_latest_task_data(db, source)
+    return counts
+
+
 async def _build_visualization_geo_summary(db: AsyncSession) -> dict[str, Any]:
     cable_count = await _count_current_or_latest_task_data(db, "arcgis_cables")
     landing_point_count = await _count_current_or_latest_task_data(db, "arcgis_landing_points")
@@ -2754,15 +2754,16 @@ async def _build_visualization_geo_summary(db: AsyncSession) -> dict[str, Any]:
         "celestrak_tle",
         exclude_unknown_name=True,
     )
-    supercomputer_count = await _count_current_or_latest_task_data(db, "top500")
-    gpu_cluster_count = await _count_current_or_latest_task_data(db, "epoch_ai_gpu")
-    compute_center_count = supercomputer_count + gpu_cluster_count
+    compute_counts = await _compute_facility_counts(db)
+    supercomputer_count = compute_counts["supercomputer"]
+    gpu_cluster_count = compute_counts["gpu_cluster"]
+    compute_center_count = sum(compute_counts.values())
 
     active_incident_result = await db.execute(
-        select(func.count(BGPIncident.id)).where(BGPIncident.status == "active"),
+        select(func.count(BGPIncident.id)).where(BGPIncident.status == BGPStatus.ACTIVE.value),
     )
     active_anomaly_result = await db.execute(
-        select(func.count(BGPAnomaly.id)).where(BGPAnomaly.status == "active"),
+        select(func.count(BGPAnomaly.id)).where(BGPAnomaly.status == BGPStatus.ACTIVE.value),
     )
     active_incident_count = int(active_incident_result.scalar() or 0)
     active_anomaly_count = int(active_anomaly_result.scalar() or 0)
@@ -2783,21 +2784,14 @@ async def _build_visualization_geo_summary(db: AsyncSession) -> dict[str, Any]:
         )
     else:
         bgp_collector_count = int(bgp_collector_scalar or 0)
-    raw_unique_window_hours = 24
-    raw_unique_mmsi = await count_unique_raw_vessel_mmsi(
-        db,
-        observed_since=datetime.now(UTC) - timedelta(hours=raw_unique_window_hours),
+    vessel_current_window_minutes = 60
+    vessel_current_result = await db.execute(
+        select(func.count(VesselCurrentState.mmsi)).where(
+            VesselCurrentState.observed_at
+            >= datetime.now(UTC) - timedelta(minutes=vessel_current_window_minutes)
+        )
     )
-    legacy_unique_result = await db.execute(
-        select(func.count(func.distinct(VesselPosition.mmsi)))
-    )
-    legacy_unique_mmsi = int(legacy_unique_result.scalar() or 0)
-    legacy_fallback_active = (
-        VESSEL_SNAPSHOT_LEGACY_FALLBACK_ENABLED
-        and raw_unique_mmsi == 0
-        and legacy_unique_mmsi > 0
-    )
-    vessel_count = legacy_unique_mmsi if legacy_fallback_active else raw_unique_mmsi
+    vessel_count = int(vessel_current_result.scalar() or 0)
     aisstream_health = await db.get(AISSourceHealth, "aisstream_vessels")
 
     return {
@@ -2808,17 +2802,19 @@ async def _build_visualization_geo_summary(db: AsyncSession) -> dict[str, Any]:
             "satellite_count": satellite_count,
             "compute_center_count": compute_center_count,
             "vessel_count": vessel_count,
-            "vessel_count_source": "legacy_fallback" if legacy_fallback_active else "raw_recent",
-            "vessel_legacy_fallback_enabled": VESSEL_SNAPSHOT_LEGACY_FALLBACK_ENABLED,
-            "vessel_raw_unique_mmsi": raw_unique_mmsi,
-            "vessel_raw_unique_window_hours": raw_unique_window_hours,
-            "vessel_legacy_unique_mmsi": legacy_unique_mmsi,
+            "vessel_count_source": "vessel_current_state",
+            "vessel_current_window_minutes": vessel_current_window_minutes,
+            "vessel_raw_unique_mmsi": 0,
+            "vessel_legacy_unique_mmsi": 0,
             "aisstream_connection_state": aisstream_health.connection_state if aisstream_health else None,
             "aisstream_last_seen_at": to_iso8601_utc(aisstream_health.last_seen_at) if aisstream_health else None,
             "aisstream_message_rate": aisstream_health.message_rate if aisstream_health else None,
             "aisstream_lag_seconds": aisstream_health.lag_seconds if aisstream_health else None,
             "supercomputer_count": supercomputer_count,
             "gpu_cluster_count": gpu_cluster_count,
+            "data_center_count": compute_counts["data_center"],
+            "cloud_center_count": compute_counts["cloud_center"],
+            "edge_node_count": compute_counts["edge_node"],
             "bgp_event_count": active_incident_count or active_anomaly_count,
             "bgp_incident_count": active_incident_count,
             "bgp_anomaly_count": active_anomaly_count,

backend/app/api/v1/websocket.py

@@ -9,6 +9,7 @@ from jose import jwt, JWTError
 from sqlalchemy import text
 
 from app.core.config import settings
+from app.core.enums import UserRole
 from app.core.logging import get_logger
 from app.core.time import to_iso8601_utc
 from app.core.websocket.manager import manager
@@ -96,7 +97,7 @@ async def websocket_endpoint(
         "earth_news",
         EARTH_UPDATES_CHANNEL,
     ]
-    if user_role == "super_admin":
+    if user_role == UserRole.SUPER_ADMIN.value:
         supported_channels = [*supported_channels, LOG_TAIL_CHANNEL]
     await manager.connect(websocket, user_id)
 
@@ -138,7 +139,7 @@ async def websocket_endpoint(
                     if channel and channel not in channels:
                         channels = [*channels, channel]
                     if LOG_TAIL_CHANNEL in channels:
-                        if user_role != "super_admin":
+                        if user_role != UserRole.SUPER_ADMIN.value:
                             await websocket.send_json(
                                 {
                                     "type": "subscription_error",
@@ -160,7 +161,7 @@ async def websocket_endpoint(
                     if is_anonymous:
                         channels = [channel for channel in channels if channel in supported_channels]
                     vessel_subscription = None
-                    if "vessels" in channels and "bbox" in payload_data:
+                    if "vessels" in channels:
                         try:
                             vessel_subscription = manager.subscribe_vessels(websocket, payload_data)
                         except ValueError as exc:

backend/app/core/compute_types.py

@@ -0,0 +1,35 @@
+"""Compute facility classification shared by the map and image resolver."""
+
+from sqlalchemy import case, or_
+
+COMPUTE_SITE_TYPES = ("supercomputer", "gpu_cluster", "data_center", "cloud_center", "edge_node")
+COMPUTE_SOURCES = ("top500", "epoch_ai_gpu")
+
+
+def compute_site_type(record) -> str:
+    metadata = record.extra_data or {}
+    for value in (metadata.get("site_type"), record.data_type):
+        if value in COMPUTE_SITE_TYPES:
+            return value
+    return {"top500": "supercomputer", "epoch_ai_gpu": "gpu_cluster"}.get(
+        record.source, "data_center"
+    )
+
+
+def compute_site_type_expression(record):
+    declared = record.extra_data["site_type"].as_string()
+    return case(
+        (declared.in_(COMPUTE_SITE_TYPES), declared),
+        (record.data_type.in_(COMPUTE_SITE_TYPES), record.data_type),
+        (record.source == "top500", "supercomputer"),
+        (record.source == "epoch_ai_gpu", "gpu_cluster"),
+        else_="data_center",
+    )
+
+
+def compute_record_filter(record):
+    return or_(
+        record.source.in_(COMPUTE_SOURCES),
+        record.data_type.in_(COMPUTE_SITE_TYPES),
+        record.extra_data["site_type"].as_string().in_(COMPUTE_SITE_TYPES),
+    )

backend/app/core/config.py

@@ -41,6 +41,7 @@ class Settings(BaseSettings):
     AI_PROVIDER_SERVICE_TOKEN: str = ""
     AI_PROVIDER_TIMEOUT_SECONDS: int = 60
     AI_PROVIDER_RETRY_ATTEMPTS: int = 2
+    OBSERVABILITY_INGEST_TOKEN: str = ""
 
     @property
     def REDIS_URL(self) -> str:

backend/app/core/earth_geography_policy.py

@@ -0,0 +1,57 @@
+"""Country display ownership for Earth comparisons and cards.
+
+Keep source-country evidence separate; use this projection only for display
+grouping so regional source records cannot become competing national keys.
+"""
+
+from __future__ import annotations
+
+import json
+from functools import lru_cache
+from pathlib import Path
+
+from app.core.countries import COUNTRY_ENTRIES, normalize_country
+
+
+_DISPLAY_ISO_A3 = {
+    canonical: next(
+        (alias for alias in aliases if len(alias) == 3 and alias.isascii() and alias.isupper()),
+        None,
+    )
+    for canonical, aliases in COUNTRY_ENTRIES
+}
+_COUNTRY_BY_ISO_A3 = {code: country for country, code in _DISPLAY_ISO_A3.items() if code}
+_POLICY_PATH = Path(__file__).resolve().parents[3] / "config/earth-boundary-pov-policy.china-v1.json"
+
+
+@lru_cache(maxsize=1)
+def _policy_country_by_code() -> dict[str, str]:
+    try:
+        content = _POLICY_PATH.read_text(encoding="utf-8")
+    except FileNotFoundError as error:
+        raise FileNotFoundError(f"EARTH_DISPLAY_POLICY_MISSING: {_POLICY_PATH}") from error
+    policy = json.loads(content)
+    if policy.get("profile") != "china-pov-v1":
+        raise ValueError("unsupported Earth display-policy profile")
+    mappings = {}
+    for rule in policy["rules"]:
+        target = rule.get("targetIsoA3") or rule.get("parentIsoA3")
+        if target not in _COUNTRY_BY_ISO_A3:
+            continue
+        for code in rule.get("sourceIsoA3", []):
+            mappings[code] = _COUNTRY_BY_ISO_A3[target]
+    return mappings
+
+
+def display_country(value: str | None) -> str | None:
+    country = normalize_country(value)
+    code = _DISPLAY_ISO_A3.get(country) if country else str(value or "").strip().upper()
+    owner = _policy_country_by_code().get(code)
+    if owner:
+        return owner
+    return country
+
+
+def display_iso_a3(value: str | None) -> str | None:
+    country = display_country(value)
+    return _DISPLAY_ISO_A3.get(country) if country else None

backend/app/core/enums.py

@@ -0,0 +1,227 @@
+"""Stable backend protocol enums.
+
+Database columns and JSON payloads continue to store the enum string values.
+Configurable identifiers, user-authored values, and open-ended taxonomies do
+not belong in this module.
+"""
+
+from __future__ import annotations
+
+import logging
+from enum import StrEnum
+from typing import TypeVar
+
+logger = logging.getLogger(__name__)
+
+EnumT = TypeVar("EnumT", bound=StrEnum)
+
+
+def parse_enum(enum_type: type[EnumT], value: object, default: EnumT) -> EnumT:
+    """Parse an external value without breaking reads of legacy data."""
+
+    if value is None or str(value).strip() == "":
+        return default
+    if isinstance(value, enum_type):
+        return value
+    try:
+        return enum_type(str(value).strip().lower())
+    except (TypeError, ValueError):
+        logger.warning(
+            "Unknown %s value %r; falling back to %s",
+            enum_type.__name__,
+            value,
+            default.value,
+        )
+        return default
+
+
+class NewsImportanceLevel(StrEnum):
+    LOW = "low"
+    MEDIUM = "medium"
+    HIGH = "high"
+    CRITICAL = "critical"
+
+
+class BreakingLevel(StrEnum):
+    NONE = "none"
+    WATCH = "watch"
+    BREAKING = "breaking"
+    CRITICAL = "critical"
+
+
+class BreakingScope(StrEnum):
+    REGIONAL = "regional"
+    GLOBAL = "global"
+
+
+class BreakingSource(StrEnum):
+    RULES = "rules"
+    AI = "ai"
+    MANUAL = "manual"
+    MULTI_SOURCE = "multi_source"
+
+
+class NewsSourceType(StrEnum):
+    RSS = "rss"
+    ATOM = "atom"
+    AGGREGATED = "aggregated"
+    REFERENCE = "reference"
+    MANUAL = "manual"
+
+
+class NewsEnrichmentStatus(StrEnum):
+    PENDING = "pending"
+    QUEUED = "queued"
+    ATTEMPTED = "attempted"
+    SUCCESS = "success"
+    CONTENT_ONLY = "content_only"
+    LOCATION_ONLY = "location_only"
+    UNAVAILABLE = "unavailable"
+    PROVIDER_ERROR = "provider_error"
+    PARSE_ERROR = "parse_error"
+    NO_RESULT = "no_result"
+
+
+class NewsMarketImpact(StrEnum):
+    NONE = "none"
+    SECTOR = "sector"
+    NATIONAL = "national"
+    GLOBAL = "global"
+
+
+class NewsTaggingSource(StrEnum):
+    RULES = "rules"
+    AI = "ai"
+    MANUAL = "manual"
+
+
+class JobType(StrEnum):
+    COLLECT = "collect"
+    CLEAR_DATA = "clear_data"
+    CLEAR_CACHE = "clear_cache"
+    EARTH_REFRESH = "earth_refresh"
+
+
+class JobStatus(StrEnum):
+    QUEUED = "queued"
+    RUNNING = "running"
+    CANCELLING = "cancelling"
+    SUCCESS = "success"
+    FAILED = "failed"
+    CANCELLED = "cancelled"
+
+
+class RollbackPolicy(StrEnum):
+    KEEP_COMMITTED_BATCHES = "keep_committed_batches"
+
+
+class MappingValidationStatus(StrEnum):
+    DRAFT = "draft"
+    VALID = "valid"
+    INVALID = "invalid"
+
+
+class SnapshotStatus(StrEnum):
+    RUNNING = "running"
+    SUCCESS = "success"
+    FAILED = "failed"
+    CANCELLED = "cancelled"
+
+
+class DatasourceRunStatus(StrEnum):
+    RUNNING = "running"
+    NOT_RUN = "not_run"
+    COLLECTED = "collected"
+    UNCOLLECTED = "uncollected"
+
+
+class ProviderApi(StrEnum):
+    ANTHROPIC_MESSAGES = "anthropic-messages"
+    OPENAI_COMPLETIONS = "openai-completions"
+    OPENAI_RESPONSES = "openai-responses"
+    OLLAMA_GENERATE = "ollama-generate"
+
+
+class PlaygroundMessageRole(StrEnum):
+    SYSTEM = "system"
+    USER = "user"
+    ASSISTANT = "assistant"
+    TOOL = "tool"
+
+
+class PlaygroundMessageKind(StrEnum):
+    MESSAGE = "message"
+    THINKING = "thinking"
+    ERROR = "error"
+    STATUS = "status"
+
+
+class PlaygroundMessageStatus(StrEnum):
+    PENDING = "pending"
+    THINKING = "thinking"
+    ANSWERING = "answering"
+    DONE = "done"
+    FAILED = "failed"
+    CANCELLED = "cancelled"
+    ERROR = "error"
+    STOPPED = "stopped"
+
+
+class OtpPurpose(StrEnum):
+    REGISTER = "register"
+    VERIFY_EMAIL = "verify_email"
+    RESET_PASSWORD = "reset_password"
+
+
+class UserRole(StrEnum):
+    VIEWER = "viewer"
+    ADMIN = "admin"
+    SUPER_ADMIN = "super_admin"
+
+
+class AlertSeverity(StrEnum):
+    CRITICAL = "critical"
+    WARNING = "warning"
+    INFO = "info"
+
+
+class AlertStatus(StrEnum):
+    ACTIVE = "active"
+    ACKNOWLEDGED = "acknowledged"
+    RESOLVED = "resolved"
+
+
+class BGPStatus(StrEnum):
+    ACTIVE = "active"
+    ACKNOWLEDGED = "acknowledged"
+    RESOLVED = "resolved"
+
+
+class LogLevel(StrEnum):
+    ALL = "all"
+    ERROR = "error"
+    WARNING = "warning"
+    INFO = "info"
+    DEBUG = "debug"
+
+
+class ConnectionState(StrEnum):
+    DISCONNECTED = "disconnected"
+    CONNECTING = "connecting"
+    CONNECTED = "connected"
+    ERROR = "error"
+
+
+class AuthType(StrEnum):
+    NONE = "none"
+    BEARER = "bearer"
+    API_KEY = "api_key"
+    BASIC = "basic"
+
+
+class TVSourceType(StrEnum):
+    IFRAME = "iframe"
+    HLS = "hls"
+    VIDEO = "video"
+    EXTERNAL = "external"
+    YOUTUBE = "youtube"

backend/app/core/satellite_presentation.py

@@ -0,0 +1,13 @@
+"""Shared source-group normalization; never infer hardware generations."""
+
+
+def satellite_constellation_group(raw_group: object, name: str | None) -> str | None:
+    group = str(raw_group or "").strip().lower()
+    if group:
+        return group
+    normalized_name = str(name or "").strip().upper()
+    if normalized_name.startswith("STARLINK"):
+        return "starlink"
+    if normalized_name.startswith("IRIDIUM"):
+        return "iridium-next"
+    return None

backend/app/core/security.py

@@ -6,7 +6,7 @@ import redis
 from fastapi import Depends, HTTPException, status
 from fastapi.security import HTTPAuthorizationCredentials, HTTPBearer
 from jose import JWTError, jwt
-from sqlalchemy import text
+from sqlalchemy import select
 from sqlalchemy.ext.asyncio import AsyncSession
 
 from app.core.config import settings
@@ -81,6 +81,21 @@ def decode_token(token: str) -> Optional[dict]:
         return None
 
 
+async def _load_active_user(user_id: str, db: AsyncSession) -> User:
+    try:
+        parsed_id = int(user_id)
+    except (TypeError, ValueError):
+        raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail="Invalid token")
+    result = await db.execute(select(User).where(User.id == parsed_id))
+    user = result.scalar_one_or_none()
+    if user is None or not user.is_active:
+        raise HTTPException(
+            status_code=status.HTTP_401_UNAUTHORIZED,
+            detail="User not found or inactive",
+        )
+    return user
+
+
 async def get_current_user(
     credentials: HTTPAuthorizationCredentials = Depends(oauth2_scheme),
     db: AsyncSession = Depends(get_db),
@@ -103,27 +118,7 @@ async def get_current_user(
             status_code=status.HTTP_401_UNAUTHORIZED,
             detail="Invalid token",
         )
-    result = await db.execute(
-        text(
-            "SELECT id, username, email, password_hash, role, is_active, gatekeeper_groups FROM users WHERE id = :id"
-        ),
-        {"id": int(user_id)},
-    )
-    row = result.fetchone()
-    if row is None or not row[5]:
-        raise HTTPException(
-            status_code=status.HTTP_401_UNAUTHORIZED,
-            detail="User not found or inactive",
-        )
-    user = User()
-    user.id = row[0]
-    user.username = row[1]
-    user.email = row[2]
-    user.password_hash = row[3]
-    user.role = row[4]
-    user.is_active = row[5]
-    user.gatekeeper_groups = row[6] or []
-    return user
+    return await _load_active_user(user_id, db)
 
 
 async def get_current_user_refresh(
@@ -143,27 +138,7 @@ async def get_current_user_refresh(
             status_code=status.HTTP_401_UNAUTHORIZED,
             detail="Invalid token",
         )
-    result = await db.execute(
-        text(
-            "SELECT id, username, email, password_hash, role, is_active, gatekeeper_groups FROM users WHERE id = :id"
-        ),
-        {"id": int(user_id)},
-    )
-    row = result.fetchone()
-    if row is None or not row[5]:
-        raise HTTPException(
-            status_code=status.HTTP_401_UNAUTHORIZED,
-            detail="User not found or inactive",
-        )
-    user = User()
-    user.id = row[0]
-    user.username = row[1]
-    user.email = row[2]
-    user.password_hash = row[3]
-    user.role = row[4]
-    user.is_active = row[5]
-    user.gatekeeper_groups = row[6] or []
-    return user
+    return await _load_active_user(user_id, db)
 
 
 def blacklist_token(token: str) -> None:

backend/app/core/websocket/broadcaster.py

@@ -4,11 +4,14 @@ import asyncio
 from datetime import UTC, datetime
 from typing import Dict, Any
 
+from app.core.logging import get_logger
 from app.core.time import to_iso8601_utc
 from app.core.websocket.manager import manager
 
 
 EARTH_UPDATES_CHANNEL = "earth_updates"
+VESSEL_STATE_QUERY_BATCH_SIZE = 1000
+logger = get_logger(__name__, service="websocket")
 
 
 class DataBroadcaster:
@@ -114,16 +117,22 @@ class DataBroadcaster:
             return
         pending = self._pending_vessel_updates
         self._pending_vessel_updates = {}
-        vessels = []
-        for item in pending.values():
-            vessel = dict(item)
-            source = vessel.pop("_source", None)
-            action = vessel.pop("_action", "upsert")
-            created = vessel.pop("_created", None)
-            vessel["source"] = source
-            vessel["action"] = action
-            vessel["created"] = created
-            vessels.append(vessel)
+        try:
+            vessels = await self._load_current_vessel_updates(list(pending))
+        except Exception:
+            # Preserve newer updates that arrived during the failed database read.
+            self._pending_vessel_updates = {**pending, **self._pending_vessel_updates}
+            raise
+        if not vessels:
+            return
+        vessels = [
+            {
+                **vessel,
+                "action": vessel.get("action", "upsert"),
+                "created": pending.get(str(vessel["mmsi"]), {}).get("_created"),
+            }
+            for vessel in vessels
+        ]
         await manager.broadcast_vessels(
             {
                 "action": "upsert",
@@ -133,12 +142,31 @@ class DataBroadcaster:
             }
         )
 
+    async def _load_current_vessel_updates(self, keys: list[str]) -> list[Dict[str, Any]]:
+        from app.db.session import async_session_factory
+        from app.services.vessel_ais_aggregation import get_current_vessels_by_mmsi
+
+        mmsis = [int(key) for key in keys if key.isdigit()]
+        vessels = []
+        async with async_session_factory() as db:
+            for offset in range(0, len(mmsis), VESSEL_STATE_QUERY_BATCH_SIZE):
+                vessels.extend(await get_current_vessels_by_mmsi(
+                    db, mmsis[offset:offset + VESSEL_STATE_QUERY_BATCH_SIZE]
+                ))
+        present = {int(vessel["mmsi"]) for vessel in vessels}
+        vessels.extend({"mmsi": mmsi, "action": "remove"} for mmsi in mmsis if mmsi not in present)
+        return vessels
+
     async def broadcast_vessels_periodically(self):
         while self.running:
             try:
                 await self.flush_vessel_updates()
-            except Exception:
-                pass
+            except Exception as exc:
+                logger.exception_event(
+                    "Failed to flush vessel updates",
+                    event="vessels.broadcast.failed",
+                    context={"error": str(exc)},
+                )
             await asyncio.sleep(self._vessel_flush_interval)
 
     async def broadcast_datasource_task_update(self, data: Dict[str, Any]):

backend/app/core/websocket/manager.py

@@ -63,6 +63,8 @@ class ConnectionManager:
 
     def unsubscribe(self, websocket: WebSocket, channels: list[str]):
         for channel in {str(channel).strip() for channel in channels if str(channel).strip()}:
+            if channel == "vessels":
+                self.vessel_subscriptions.pop(websocket, None)
             subscribers = self.channel_subscriptions.get(channel)
             if subscribers is not None:
                 subscribers.discard(websocket)
@@ -88,7 +90,8 @@ class ConnectionManager:
         return subscription
 
     def _normalize_vessel_subscription(self, config: dict[str, Any]) -> dict[str, Any]:
-        bbox = config.get("bbox")
+        global_scope = config.get("scope") == "global"
+        bbox = [-180, -90, 180, 90] if global_scope else config.get("bbox")
         if not isinstance(bbox, (list, tuple)) or len(bbox) != 4:
             raise ValueError("vessels subscription requires bbox=[lon_min,lat_min,lon_max,lat_max]")
         try:
@@ -103,7 +106,7 @@ class ConnectionManager:
             raise ValueError("bbox longitude values must be between -180 and 180")
         if not (-90 <= lat_min <= 90 and -90 <= lat_max <= 90):
             raise ValueError("bbox latitude values must be between -90 and 90")
-        if (lon_max - lon_min) * (lat_max - lat_min) > MAX_VESSEL_BBOX_AREA:
+        if not global_scope and (lon_max - lon_min) * (lat_max - lat_min) > MAX_VESSEL_BBOX_AREA:
             raise ValueError("bbox is too large; zoom in or request a smaller viewport")
 
         zoom = int(config.get("zoom") or 1)
@@ -116,10 +119,11 @@ class ConnectionManager:
             if str(item).strip()
         }
         return {
+            "scope": "global" if global_scope else "viewport",
             "bbox": (lon_min, lat_min, lon_max, lat_max),
             "zoom": zoom,
             "limit": limit,
-            "type": vessel_types,
+            "type": sorted(vessel_types),
             "last_sent_at": None,
         }
 
@@ -152,7 +156,9 @@ class ConnectionManager:
                 vessel
                 for vessel in vessels
                 if self._vessel_matches_subscription(vessel, subscription)
-            ][: min(subscription["limit"], MAX_VESSEL_WS_MESSAGE_ITEMS)]
+            ]
+            if subscription.get("scope") != "global":
+                matched = matched[:subscription["limit"]]
             if not matched:
                 continue
             subscription["last_sent_at"] = datetime.now(UTC)
@@ -162,16 +168,24 @@ class ConnectionManager:
                 "timestamp": subscription["last_sent_at"].isoformat(),
                 "payload": {
                     **data,
-                    "vessels": matched,
+                    "vessels": [],
                     "subscription": {
                         "bbox": list(subscription["bbox"]),
                         "zoom": subscription["zoom"],
                         "limit": subscription["limit"],
+                        "scope": subscription.get("scope", "viewport"),
                     },
                 },
             }
             try:
-                await connection.send_json(message)
+                for offset in range(0, len(matched), MAX_VESSEL_WS_MESSAGE_ITEMS):
+                    await connection.send_json({
+                        **message,
+                        "payload": {
+                            **message["payload"],
+                            "vessels": matched[offset:offset + MAX_VESSEL_WS_MESSAGE_ITEMS],
+                        },
+                    })
             except Exception:
                 self.unsubscribe_all(connection)
 
@@ -180,6 +194,8 @@ class ConnectionManager:
         vessel: dict[str, Any],
         subscription: dict[str, Any],
     ) -> bool:
+        if vessel.get("action") == "remove" and subscription.get("scope") == "global":
+            return True
         try:
             lon = float(vessel.get("lon"))
             lat = float(vessel.get("lat"))

backend/app/db/session.py

@@ -626,6 +626,7 @@ async def init_db():
             "bgp_collector_locations",
             "vessel_static",
             "vessel_position",
+            "vessel_current_state",
             "ais_raw_observations",
             "ais_source_health",
             "compute_center_locations",
@@ -786,6 +787,22 @@ async def init_db():
                 """
             )
         )
+        await conn.execute(
+            text(
+                """
+                CREATE INDEX IF NOT EXISTS idx_vessel_current_bbox
+                ON vessel_current_state (lon, lat)
+                """
+            )
+        )
+        await conn.execute(
+            text(
+                """
+                CREATE INDEX IF NOT EXISTS idx_vessel_current_observed
+                ON vessel_current_state (observed_at DESC)
+                """
+            )
+        )
         await conn.execute(
             text(
                 """

backend/app/models/__init__.py

@@ -13,7 +13,7 @@ from app.models.compute_center_location import ComputeCenterLocationRecord
 from app.models.system_setting import SystemSetting
 from app.models.playground_session import PlaygroundSession
 from app.models.playground_message import PlaygroundMessage
-from app.models.system_log import SystemLog, AuditLog
+from app.models.system_log import AuditLog, ObservabilityEvent, ObservabilityEventGroup, SystemLog
 from app.models.vessel import AISConflictRecord, AISRawObservation, AISSourceHealth, VesselPosition, VesselStatic
 from app.models.datasource_mapping import DataSourceMappingTemplate
 from app.models.earth_news import EarthNewsItem
@@ -37,6 +37,8 @@ __all__ = [
     "ComputeCenterLocationRecord",
     "SystemLog",
     "AuditLog",
+    "ObservabilityEvent",
+    "ObservabilityEventGroup",
     "PlaygroundSession",
     "PlaygroundMessage",
     "VesselPosition",

backend/app/models/alert.py

@@ -1,26 +1,12 @@
 from datetime import datetime
-from enum import Enum
-from typing import Optional
 
-from sqlalchemy import Column, Integer, String, DateTime, Text, ForeignKey, Enum as SQLEnum
-from sqlalchemy.orm import relationship
+from sqlalchemy import Column, Integer, String, DateTime, Text, Enum as SQLEnum
 
+from app.core.enums import AlertSeverity, AlertStatus
 from app.core.time import to_iso8601_utc
 from app.db.session import Base
 
 
-class AlertSeverity(str, Enum):
-    CRITICAL = "critical"
-    WARNING = "warning"
-    INFO = "info"
-
-
-class AlertStatus(str, Enum):
-    ACTIVE = "active"
-    ACKNOWLEDGED = "acknowledged"
-    RESOLVED = "resolved"
-
-
 class Alert(Base):
     __tablename__ = "alerts"
 

backend/app/models/bgp_anomaly.py

@@ -4,6 +4,7 @@ from datetime import datetime
 
 from sqlalchemy import Column, DateTime, Float, ForeignKey, Index, Integer, JSON, String, Text
 
+from app.core.enums import BGPStatus
 from app.core.time import to_iso8601_utc
 from app.db.session import Base
 
@@ -17,7 +18,7 @@ class BGPAnomaly(Base):
     source = Column(String(100), nullable=False, index=True)
     anomaly_type = Column(String(50), nullable=False, index=True)
     severity = Column(String(20), nullable=False, index=True)
-    status = Column(String(20), nullable=False, default="active", index=True)
+    status = Column(String(20), nullable=False, default=BGPStatus.ACTIVE.value, index=True)
     entity_key = Column(String(255), nullable=False, index=True)
     prefix = Column(String(64), nullable=True, index=True)
     origin_asn = Column(Integer, nullable=True, index=True)

backend/app/models/bgp_incident.py

@@ -4,6 +4,7 @@ from datetime import datetime
 
 from sqlalchemy import Column, DateTime, Float, ForeignKey, Index, Integer, JSON, String, Text
 
+from app.core.enums import BGPStatus
 from app.core.time import to_iso8601_utc
 from app.db.session import Base
 
@@ -20,7 +21,7 @@ class BGPIncident(Base):
     title = Column(String(255), nullable=False)
     summary = Column(Text, nullable=False)
     severity = Column(String(20), nullable=False, index=True)
-    status = Column(String(20), nullable=False, default="active", index=True)
+    status = Column(String(20), nullable=False, default=BGPStatus.ACTIVE.value, index=True)
     confidence = Column(Float, nullable=False, default=0.5)
     started_at = Column(DateTime(timezone=True), nullable=False, default=datetime.utcnow, index=True)
     ended_at = Column(DateTime(timezone=True), nullable=True)

backend/app/models/data_snapshot.py

@@ -1,6 +1,7 @@
 from sqlalchemy import Boolean, Column, DateTime, ForeignKey, Integer, JSON, String
 from sqlalchemy.sql import func
 
+from app.core.enums import SnapshotStatus
 from app.db.session import Base
 
 
@@ -16,7 +17,7 @@ class DataSnapshot(Base):
     started_at = Column(DateTime(timezone=True), server_default=func.now())
     completed_at = Column(DateTime(timezone=True), nullable=True)
     record_count = Column(Integer, default=0)
-    status = Column(String(20), nullable=False, default="running")
+    status = Column(String(20), nullable=False, default=SnapshotStatus.RUNNING.value)
     is_current = Column(Boolean, default=True, index=True)
     parent_snapshot_id = Column(Integer, ForeignKey("data_snapshots.id"), nullable=True, index=True)
     summary = Column(JSON, default={})

backend/app/models/datasource_mapping.py

@@ -3,6 +3,7 @@
 from sqlalchemy import Boolean, Column, DateTime, ForeignKey, Integer, JSON, String
 from sqlalchemy.sql import func
 
+from app.core.enums import MappingValidationStatus
 from app.db.session import Base
 
 
@@ -19,7 +20,7 @@ class DataSourceMappingTemplate(Base):
     target_schema = Column(String(80), nullable=False, index=True)
     mapping_json = Column(JSON, nullable=False, default={})
     sample_payload_hash = Column(String(64), nullable=True)
-    validation_status = Column(String(30), nullable=False, default="draft")
+    validation_status = Column(String(30), nullable=False, default=MappingValidationStatus.DRAFT.value)
     version = Column(Integer, nullable=False, default=1)
     is_active = Column(Boolean, nullable=False, default=False, index=True)
     created_at = Column(DateTime(timezone=True), server_default=func.now())

backend/app/models/playground_message.py

@@ -1,6 +1,7 @@
 from sqlalchemy import JSON, Boolean, Column, DateTime, ForeignKey, Integer, String, Text
 from sqlalchemy.sql import func
 
+from app.core.enums import PlaygroundMessageKind, PlaygroundMessageStatus
 from app.db.session import Base
 
 
@@ -13,8 +14,8 @@ class PlaygroundMessage(Base):
     user_id = Column(Integer, ForeignKey("users.id", ondelete="CASCADE"), nullable=False, index=True)
     parent_message_id = Column(Integer, ForeignKey("playground_messages.id", ondelete="SET NULL"), nullable=True)
     role = Column(String(20), nullable=False)
-    kind = Column(String(20), nullable=False, default="message")
-    status = Column(String(20), nullable=False, default="done")
+    kind = Column(String(20), nullable=False, default=PlaygroundMessageKind.MESSAGE.value)
+    status = Column(String(20), nullable=False, default=PlaygroundMessageStatus.DONE.value)
     title = Column(String(255), nullable=True)
     content = Column(Text, nullable=False, default="")
     thinking_content = Column(Text, nullable=False, default="")

backend/app/models/system_log.py

@@ -38,3 +38,46 @@ class AuditLog(Base):
     ip = Column(String(64), nullable=True)
     details = Column(JSON, nullable=False, default=dict)
     created_at = Column(DateTime(timezone=True), server_default=func.now())
+
+
+class ObservabilityEvent(Base):
+    __tablename__ = "observability_events"
+
+    id = Column(Integer, primary_key=True, autoincrement=True)
+    occurred_at = Column(DateTime(timezone=True), server_default=func.now(), index=True)
+    source = Column(String(50), nullable=False, index=True)
+    service = Column(String(50), nullable=True, index=True)
+    module = Column(String(120), nullable=True, index=True)
+    category = Column(String(80), nullable=True, index=True)
+    event = Column(String(160), nullable=True, index=True)
+    level = Column(String(20), nullable=False, index=True)
+    message = Column(Text, nullable=False)
+    fingerprint = Column(String(80), nullable=False, index=True)
+    request_id = Column(String(64), nullable=True, index=True)
+    trace_id = Column(String(64), nullable=True, index=True)
+    task_id = Column(String(120), nullable=True, index=True)
+    source_ref_id = Column(String(120), nullable=True, index=True)
+    provider = Column(String(120), nullable=True, index=True)
+    user_id = Column(Integer, nullable=True, index=True)
+    context = Column(JSON, nullable=False, default=dict)
+    occurrence_count = Column(Integer, nullable=False, default=1)
+    created_at = Column(DateTime(timezone=True), server_default=func.now())
+
+
+class ObservabilityEventGroup(Base):
+    __tablename__ = "observability_event_groups"
+
+    fingerprint = Column(String(80), primary_key=True)
+    source = Column(String(50), nullable=False, index=True)
+    service = Column(String(50), nullable=True, index=True)
+    module = Column(String(120), nullable=True, index=True)
+    category = Column(String(80), nullable=True, index=True)
+    event = Column(String(160), nullable=True, index=True)
+    last_level = Column(String(20), nullable=False, index=True)
+    sample_message = Column(Text, nullable=False)
+    sample_detail = Column(Text, nullable=True)
+    affected_sources = Column(JSON, nullable=False, default=list)
+    count = Column(Integer, nullable=False, default=0)
+    first_seen_at = Column(DateTime(timezone=True), nullable=False, index=True)
+    last_seen_at = Column(DateTime(timezone=True), nullable=False, index=True)
+    updated_at = Column(DateTime(timezone=True), server_default=func.now(), onupdate=func.now())

backend/app/models/task.py

@@ -3,6 +3,7 @@
 from sqlalchemy import BigInteger, Column, DateTime, Float, Integer, JSON, String, Text
 from sqlalchemy.sql import func
 
+from app.core.enums import JobStatus, JobType, RollbackPolicy
 from app.db.session import Base
 
 
@@ -12,9 +13,9 @@ class CollectionTask(Base):
     id = Column(Integer, primary_key=True, autoincrement=True)
     datasource_id = Column(Integer, nullable=False, index=True)
     source = Column(String(100), nullable=True, index=True)
-    task_type = Column(String(30), nullable=False, default="collect", index=True)
+    task_type = Column(String(30), nullable=False, default=JobType.COLLECT.value, index=True)
     status = Column(String(20), nullable=False)  # queued, running, cancelling, success, failed, cancelled
-    phase = Column(String(30), default="queued")
+    phase = Column(String(30), default=JobStatus.QUEUED.value)
     phase_progress = Column(Float)
     phase_message = Column(String(255))
     phase_current = Column(BigInteger)
@@ -27,7 +28,7 @@ class CollectionTask(Base):
     progress = Column(Float, default=0.0)  # Progress percentage (0-100)
     error_message = Column(Text)
     payload = Column(JSON, default=dict)
-    rollback_policy = Column(String(40), nullable=False, default="keep_committed_batches")
+    rollback_policy = Column(String(40), nullable=False, default=RollbackPolicy.KEEP_COMMITTED_BATCHES.value)
     dedupe_key = Column(String(180), nullable=True, index=True)
     worker_id = Column(String(120), nullable=True, index=True)
     locked_at = Column(DateTime(timezone=True), nullable=True, index=True)

backend/app/models/user.py

@@ -1,6 +1,7 @@
 from sqlalchemy import Boolean, Column, DateTime, Integer, JSON, String
 from sqlalchemy.sql import func
 
+from app.core.enums import UserRole
 from app.db.session import Base
 
 
@@ -11,7 +12,7 @@ class User(Base):
     username = Column(String(50), unique=True, index=True, nullable=False)
     email = Column(String(255), unique=True, index=True, nullable=False)
     password_hash = Column(String(255), nullable=False)
-    role = Column(String(20), default="viewer")
+    role = Column(String(20), default=UserRole.VIEWER.value)
     gatekeeper_groups = Column(JSON, default=list)
     is_active = Column(Boolean, default=True)
     email_verified = Column(Boolean, default=False, nullable=False)

backend/app/models/vessel.py

@@ -3,6 +3,7 @@
 from sqlalchemy import BigInteger, Column, DateTime, Float, Index, Integer, JSON, SmallInteger, String
 from sqlalchemy.sql import func
 
+from app.core.enums import ConnectionState
 from app.core.time import to_iso8601_utc
 from app.db.session import Base
 
@@ -75,6 +76,67 @@ class VesselPosition(Base):
         }
 
 
+class VesselCurrentState(Base):
+    """Latest renderable state for one vessel, independent from AIS history."""
+
+    __tablename__ = "vessel_current_state"
+
+    mmsi = Column(BigInteger, primary_key=True)
+    lat = Column(Float, nullable=False)
+    lon = Column(Float, nullable=False)
+    sog = Column(Float, nullable=True)
+    cog = Column(Float, nullable=True)
+    heading = Column(SmallInteger, nullable=True)
+    nav_status = Column(SmallInteger, nullable=True, index=True)
+    name = Column(String(128), nullable=True)
+    callsign = Column(String(16), nullable=True)
+    vessel_type = Column(SmallInteger, nullable=True, index=True)
+    vessel_type_name = Column(String(64), nullable=True, index=True)
+    flag = Column(String(4), nullable=True, index=True)
+    length = Column(Float, nullable=True)
+    width = Column(Float, nullable=True)
+    draught = Column(Float, nullable=True)
+    imo = Column(BigInteger, nullable=True)
+    source = Column(String(100), nullable=False, index=True)
+    observed_at = Column(DateTime(timezone=True), nullable=False)
+    field_sources = Column(JSON, default=dict)
+    selected_reasons = Column(JSON, default=dict)
+    source_summary = Column(JSON, default=dict)
+    quality_flags = Column(JSON, default=list)
+    updated_at = Column(DateTime(timezone=True), nullable=False, server_default=func.now())
+
+    __table_args__ = (
+        Index("idx_vessel_current_bbox", "lon", "lat"),
+        Index("idx_vessel_current_observed", "observed_at"),
+    )
+
+    def to_dict(self) -> dict:
+        return {
+            "mmsi": self.mmsi,
+            "lat": self.lat,
+            "lon": self.lon,
+            "sog": self.sog,
+            "cog": self.cog,
+            "heading": self.heading,
+            "nav_status": self.nav_status,
+            "name": self.name,
+            "callsign": self.callsign,
+            "vessel_type": self.vessel_type,
+            "vessel_type_name": self.vessel_type_name,
+            "flag": self.flag,
+            "length": self.length,
+            "width": self.width,
+            "draught": self.draught,
+            "imo": self.imo,
+            "source": self.source,
+            "received_at": self.observed_at,
+            "field_sources": self.field_sources or {},
+            "selected_reasons": self.selected_reasons or {},
+            "source_summary": self.source_summary or {},
+            "quality_flags": self.quality_flags or [],
+        }
+
+
 class AISRawObservation(Base):
     """Source-level AIS fact before aggregation and conflict resolution."""
 
@@ -165,7 +227,7 @@ class AISSourceHealth(Base):
     __tablename__ = "ais_source_health"
 
     source = Column(String(100), primary_key=True)
-    connection_state = Column(String(32), nullable=False, default="disconnected", index=True)
+    connection_state = Column(String(32), nullable=False, default=ConnectionState.DISCONNECTED.value, index=True)
     last_seen_at = Column(DateTime(timezone=True), nullable=True, index=True)
     last_success_at = Column(DateTime(timezone=True), nullable=True, index=True)
     last_error = Column(String(500), nullable=True)

backend/app/schemas/ai.py

@@ -2,6 +2,7 @@ from typing import Any
 
 from pydantic import BaseModel, Field
 
+from app.core.enums import PlaygroundMessageKind, PlaygroundMessageRole, PlaygroundMessageStatus
 
 class AIContentBlock(BaseModel):
     type: str
@@ -110,9 +111,9 @@ class PlaygroundSessionUpsertRequest(BaseModel):
 
 class PlaygroundMessageRecord(BaseModel):
     id: str
-    role: str
-    kind: str = "message"
-    status: str = "done"
+    role: PlaygroundMessageRole
+    kind: PlaygroundMessageKind = PlaygroundMessageKind.MESSAGE
+    status: PlaygroundMessageStatus = PlaygroundMessageStatus.DONE
     title: str | None = None
     content: str = ""
     thinking_content: str = ""

backend/app/schemas/card_images.py

@@ -0,0 +1,128 @@
+"""Typed, validated configuration for optional object-card images."""
+
+from typing import Literal, Self
+from urllib.parse import urlsplit
+
+from pydantic import BaseModel, ConfigDict, Field, field_validator, model_validator
+
+EntityType = Literal["satellite", "compute_center", "cable", "landing_point", "vessel", "bgp", "bgp_collector", "earth_interactable"]
+MatchScope = Literal["object", "model", "group", "category"]
+
+
+def safe_url(value: str, *, local: bool = False) -> str:
+    if not value:
+        return value
+    if any(ord(char) < 32 for char in value) or "\\" in value:
+        raise ValueError("Invalid URL")
+    parts = urlsplit(value)
+    if local and value.startswith(("/earth/assets/cards/", "/api/v1/earth/card-images/files/")):
+        if (
+            ".." not in parts.path
+            and "%" not in parts.path
+            and not parts.netloc
+            and not parts.query
+            and not parts.fragment
+        ):
+            return value
+    if (
+        parts.scheme in {"http", "https"}
+        and parts.hostname
+        and not parts.username
+        and not parts.password
+    ):
+        return value
+    raise ValueError("Use an HTTP(S) URL without credentials or a managed image path")
+
+
+class CardImageAsset(BaseModel):
+    model_config = ConfigDict(extra="forbid", str_strip_whitespace=True)
+    id: str = Field(min_length=1, max_length=100, pattern=r"^[a-zA-Z0-9_-]+$")
+    title: str = Field(min_length=1, max_length=200)
+    url: str = Field(min_length=1, max_length=2000)
+    variants: dict[str, str] = Field(default_factory=dict, max_length=4)
+    width: int = Field(default=0, ge=0, le=20000)
+    height: int = Field(default=0, ge=0, le=20000)
+    credit: str = Field(default="", max_length=500)
+    source_url: str = Field(default="", max_length=2000)
+    license: str = Field(default="", max_length=300)
+    license_url: str = Field(default="", max_length=2000)
+    kind: Literal["photo", "model_photo", "render", "animation_still"] = "photo"
+    rights_status: Literal["available", "permission_pending"] = "available"
+    enabled: bool = True
+    focus_x: float = Field(default=50, ge=0, le=100, allow_inf_nan=False)
+    focus_y: float = Field(default=50, ge=0, le=100, allow_inf_nan=False)
+
+    @field_validator("url")
+    @classmethod
+    def validate_image_url(cls, value: str) -> str:
+        return safe_url(value, local=True)
+
+    @field_validator("source_url", "license_url")
+    @classmethod
+    def validate_source_url(cls, value: str) -> str:
+        return safe_url(value)
+
+    @field_validator("variants")
+    @classmethod
+    def validate_variants(cls, value: dict[str, str]) -> dict[str, str]:
+        for width, url in value.items():
+            if width not in {"640", "1280"}:
+                raise ValueError("Unsupported image variant")
+            safe_url(url, local=True)
+        return value
+
+
+class CardImageRule(BaseModel):
+    model_config = ConfigDict(extra="forbid", str_strip_whitespace=True)
+    id: str = Field(min_length=1, max_length=100, pattern=r"^[a-zA-Z0-9_-]+$")
+    entity_type: EntityType
+    scope: MatchScope
+    value: str = Field(min_length=1, max_length=300)
+    label: str = Field(default="", max_length=500)
+    asset_id: str | None = Field(default=None, max_length=100)
+    suppress: bool = False
+    enabled: bool = True
+
+    @model_validator(mode="after")
+    def validate_action(self) -> Self:
+        if self.scope != "object":
+            self.value = self.value.casefold()
+        if self.suppress and (self.scope != "object" or self.asset_id):
+            raise ValueError("Only an object rule may suppress its image, without an asset")
+        if not self.suppress and not self.asset_id:
+            raise ValueError("Choose an image or suppress this object")
+        return self
+
+    def target(self) -> tuple[str, str, str]:
+        return self.entity_type, self.scope, self.value
+
+
+class CardImageConfiguration(BaseModel):
+    model_config = ConfigDict(extra="forbid")
+    assets: list[CardImageAsset] = Field(default_factory=list, max_length=1000)
+    rules: list[CardImageRule] = Field(default_factory=list, max_length=3000)
+
+    @model_validator(mode="after")
+    def validate_relations(self) -> Self:
+        assets = {asset.id for asset in self.assets}
+        if len(assets) != len(self.assets) or len({rule.id for rule in self.rules}) != len(
+            self.rules
+        ):
+            raise ValueError("Image and rule IDs must be unique")
+        targets = [rule.target() for rule in self.rules]
+        if len(set(targets)) != len(targets):
+            raise ValueError("Each target may have only one rule")
+        if any(rule.asset_id and rule.asset_id not in assets for rule in self.rules):
+            raise ValueError("Remove image associations before deleting the image")
+        return self
+
+
+class CardImagePublish(BaseModel):
+    revision: str = Field(max_length=200)
+    config: CardImageConfiguration
+
+
+class CardImagePreview(BaseModel):
+    entity_type: EntityType
+    entity_id: str = Field(min_length=1, max_length=300)
+    config: CardImageConfiguration

backend/app/schemas/compute_configuration.py

@@ -0,0 +1,94 @@
+"""Validated, source-backed observations for the compute comparison cards."""
+
+from datetime import date
+from typing import Annotated, Literal, Self
+
+from app.services.compute_catalog import COUNTRY_PARTICIPANTS
+from pydantic import BaseModel, ConfigDict, Field, HttpUrl, field_validator, model_validator
+
+Year = Annotated[int, Field(strict=True, ge=2000, le=2100)]
+Value = Annotated[float, Field(strict=True, allow_inf_nan=False, ge=0, le=1e12)]
+Identifier = Annotated[str, Field(min_length=1, max_length=80, pattern=r"^[a-zA-Z0-9_-]+$")]
+Text = Annotated[str, Field(min_length=1, max_length=500)]
+
+
+class ConfigurationModel(BaseModel):
+    model_config = ConfigDict(extra="forbid", str_strip_whitespace=True)
+
+
+class ComputeSource(ConfigurationModel):
+    id: Identifier
+    title: Text
+    url: HttpUrl
+    publisher: Text
+    report_year: Year
+    published_at: date | None = None
+    locator: Text
+
+    @field_validator("url")
+    @classmethod
+    def public_reference(cls, value: HttpUrl) -> HttpUrl:
+        if value.username or value.password or len(str(value)) > 2000:
+            raise ValueError("Use a report URL without credentials (maximum 2000 characters)")
+        return value
+
+
+class ComputeObservation(ConfigurationModel):
+    year: Year
+    metric: Literal["global_share", "absolute"]
+    source_id: Identifier
+    values: dict[str, Value] = Field(min_length=1, max_length=250)
+
+    @model_validator(mode="after")
+    def comparable_values(self) -> Self:
+        if any(code not in COUNTRY_PARTICIPANTS for code in self.values):
+            raise ValueError(
+                "Use countries from the canonical compute catalog, not regional totals"
+            )
+        if self.metric == "global_share" and sum(self.values.values()) > 100.000001:
+            raise ValueError("Global shares for one year must total at most 100 percent")
+        return self
+
+
+class InnovationObservation(ConfigurationModel):
+    year: Year
+    value: Annotated[float, Field(strict=True, allow_inf_nan=False, ge=-100, le=100)]
+    source_id: Identifier
+
+
+class ComputeConfiguration(ConfigurationModel):
+    reviewed_at: date
+    sources: list[ComputeSource] = Field(max_length=200)
+    observations: list[ComputeObservation] = Field(max_length=202)
+    innovation: list[InnovationObservation] = Field(max_length=101)
+
+    @model_validator(mode="after")
+    def consistent_evidence(self) -> Self:
+        ids = [source.id for source in self.sources]
+        if len(ids) != len(set(ids)):
+            raise ValueError("Report source IDs must be unique")
+        keys = [(row.metric, row.year) for row in self.observations]
+        if len(keys) != len(set(keys)):
+            raise ValueError("Each metric/year may have only one observation")
+        years = [row.year for row in self.innovation]
+        if len(years) != len(set(years)):
+            raise ValueError("Innovation observation years must be unique")
+        sources = {source.id: source for source in self.sources}
+        for row in [*self.observations, *self.innovation]:
+            source = sources.get(row.source_id)
+            if source is None:
+                raise ValueError(f"Missing report source: {row.source_id}")
+            if row.year > source.report_year:
+                raise ValueError("Observation year must not be after its report year")
+        if self.reviewed_at > date.today():
+            raise ValueError("Review date must not be in the future")
+        return self
+
+
+class ComputeConfigurationWrite(ConfigurationModel):
+    revision: str = Field(min_length=1, max_length=80)
+    config: ComputeConfiguration
+
+
+class ComputeConfigurationReset(ConfigurationModel):
+    revision: str = Field(min_length=1, max_length=80)

backend/app/schemas/user.py

@@ -3,6 +3,7 @@ from typing import Optional
 
 from pydantic import BaseModel, EmailStr, Field
 
+from app.core.enums import OtpPurpose, UserRole
 
 class UserBase(BaseModel):
     username: str
@@ -11,13 +12,13 @@ class UserBase(BaseModel):
 
 class UserCreate(UserBase):
     password: str = Field(..., min_length=8)
-    role: str = "viewer"
+    role: UserRole = UserRole.VIEWER
     gatekeeper_groups: list[str] = Field(default_factory=list)
 
 
 class UserUpdate(BaseModel):
     email: Optional[EmailStr] = None
-    role: Optional[str] = None
+    role: Optional[UserRole] = None
     gatekeeper_groups: Optional[list[str]] = None
     is_active: Optional[bool] = None
 
@@ -35,6 +36,9 @@ class UserInDB(UserBase):
 
 
 class UserResponse(UserBase):
+    # Bootstrap accounts have reserved .local email addresses. Validate email
+    # on writes, while allowing existing local identities to be serialized.
+    email: str
     id: int
     role: str
     gatekeeper_groups: list[str] = Field(default_factory=list)
@@ -59,7 +63,7 @@ class VerifyEmailRequest(BaseModel):
 
 class ResendCodeRequest(BaseModel):
     email: EmailStr
-    purpose: str = Field(default="register", pattern="^(register|verify_email|reset_password)$")
+    purpose: OtpPurpose = OtpPurpose.REGISTER
 
 
 class ForgotPasswordRequest(BaseModel):

backend/app/services/bgp_detectors.py

@@ -6,6 +6,7 @@ from collections import Counter, defaultdict
 from datetime import UTC, datetime
 from typing import Any
 
+from app.core.enums import BGPStatus
 from app.models.bgp_anomaly import BGPAnomaly
 
 
@@ -127,7 +128,7 @@ def detect_origin_change_anomalies(
                     source=source,
                     anomaly_type=anomaly_type,
                     severity=severity,
-                    status="active",
+                    status=BGPStatus.ACTIVE.value,
                     entity_key=f"{anomaly_type}:{prefix}:{new_origin}",
                     prefix=prefix,
                     origin_asn=sorted(historic)[0] if historic else None,
@@ -197,7 +198,7 @@ def detect_more_specific_burst_anomalies(
                 source=source,
                 anomaly_type="more_specific_burst",
                 severity="high",
-                status="active",
+                status=BGPStatus.ACTIVE.value,
                 entity_key=f"more_specific_burst:{root_prefix}:{len(unique_prefixes)}:{len(related_collectors)}",
                 prefix=sample.get("prefix"),
                 origin_asn=sample.get("origin_asn"),
@@ -267,7 +268,7 @@ def detect_mass_withdrawal_anomalies(
                 source=source,
                 anomaly_type="mass_withdrawal",
                 severity=severity,
-                status="active",
+                status=BGPStatus.ACTIVE.value,
                 entity_key=f"mass_withdrawal:{prefix}:{origin_asn}:{len(related_collectors)}:{count}",
                 prefix=prefix,
                 origin_asn=origin_asn,
@@ -354,7 +355,7 @@ def detect_route_leak_anomalies(
                 source=source,
                 anomaly_type="route_leak_candidate",
                 severity="high" if max_path_length >= dominant_length + 3 else "medium",
-                status="active",
+                status=BGPStatus.ACTIVE.value,
                 entity_key=f"route_leak_candidate:{prefix}:{max_path_length}:{len(related_collectors)}",
                 prefix=prefix,
                 origin_asn=sample_metadata.get("origin_asn"),
@@ -435,7 +436,7 @@ def detect_path_flap_anomalies(
                 source=source,
                 anomaly_type="path_flap",
                 severity=severity,
-                status="active",
+                status=BGPStatus.ACTIVE.value,
                 entity_key=f"path_flap:{prefix}:{transitions}:{len(distinct_paths)}",
                 prefix=prefix,
                 origin_asn=sample_metadata.get("origin_asn"),

backend/app/services/bgp_incidents.py

@@ -9,6 +9,7 @@ from sqlalchemy import select
 from sqlalchemy.ext.asyncio import AsyncSession
 
 from app.core.collected_data_fields import get_record_field
+from app.core.enums import BGPStatus
 from app.models.bgp_anomaly import BGPAnomaly
 from app.models.bgp_incident import BGPIncident
 from app.models.collected_data import CollectedData
@@ -290,7 +291,7 @@ async def create_bgp_incidents_for_anomalies(
             existing.title = title
             existing.summary = summary
             existing.severity = severity
-            existing.status = "active"
+            existing.status = BGPStatus.ACTIVE.value
             existing.confidence = confidence
             existing.started_at = primary.started_at or existing.started_at or datetime.now(UTC)
             existing.ended_at = None
@@ -313,7 +314,7 @@ async def create_bgp_incidents_for_anomalies(
                 title=title,
                 summary=summary,
                 severity=severity,
-                status="active",
+                status=BGPStatus.ACTIVE.value,
                 confidence=confidence,
                 started_at=primary.started_at or datetime.now(UTC),
                 affected_prefixes=prefixes,

backend/app/services/card_image_configuration.py

@@ -0,0 +1,174 @@
+"""Revision-checked image settings; new bundled assets never replace user edits."""
+
+from uuid import uuid4
+
+from fastapi import HTTPException
+from sqlalchemy import select
+from sqlalchemy.dialects.postgresql import insert
+from sqlalchemy.ext.asyncio import AsyncSession
+
+from app.core.request_context import get_request_id
+from app.models.system_log import AuditLog
+from app.models.system_setting import SystemSetting
+from app.models.user import User
+from app.schemas.card_images import CardImageConfiguration
+from app.services.card_image_defaults import (
+    bundled_configuration,
+    previous_bundled_assets,
+    previous_bundled_rules,
+)
+
+CATEGORY = "earth_card_images"
+
+
+def effective_envelope(payload: dict | None) -> dict:
+    defaults, version = bundled_configuration()
+    config = (
+        CardImageConfiguration.model_validate(payload["config"])
+        if payload
+        else defaults.model_copy(deep=True)
+    )
+    if payload:
+        previous = previous_bundled_assets()
+        latest = {asset.id: asset for asset in defaults.assets}
+        for index, asset in enumerate(config.assets):
+            old = next((item for item in previous.get(asset.id, []) if item.url == asset.url), None)
+            # Replace the old bundled source while preserving fields the user
+            # changed (crop, title, enabled state, etc.). Custom URLs are untouched.
+            if old is not None and asset.url == old.url and asset.id in latest:
+                overrides = {
+                    key: value
+                    for key, value in asset.model_dump().items()
+                    if value != getattr(old, key)
+                }
+                config.assets[index] = latest[asset.id].model_copy(update=overrides, deep=True)
+        known_assets = set(payload.get("known_assets", []))
+        asset_ids = {asset.id for asset in config.assets}
+        config.assets.extend(
+            asset.model_copy(deep=True)
+            for asset in defaults.assets
+            if asset.id not in known_assets and asset.id not in asset_ids
+        )
+        prior_rules = previous_bundled_rules()
+        latest_rules = {rule.id: rule for rule in defaults.rules}
+        for index, rule in enumerate(config.rules):
+            old_rule = prior_rules.get(rule.id)
+            if (
+                old_rule
+                and rule.id in latest_rules
+                and rule.target() == old_rule.target()
+                and rule.asset_id == old_rule.asset_id
+            ):
+                overrides = {
+                    key: value
+                    for key, value in rule.model_dump().items()
+                    if value != getattr(old_rule, key)
+                }
+                config.rules[index] = latest_rules[rule.id].model_copy(update=overrides, deep=True)
+        known_rules = set(payload.get("known_rules", []))
+        rule_ids = {rule.id for rule in config.rules}
+        targets = {rule.target() for rule in config.rules}
+        available = {asset.id for asset in config.assets}
+        config.rules.extend(
+            rule.model_copy(deep=True)
+            for rule in defaults.rules
+            if rule.id not in known_rules
+            and rule.id not in rule_ids
+            and rule.target() not in targets
+            and rule.asset_id in available
+        )
+    return {
+        "config": config.model_dump(mode="json"),
+        "revision": f"{version}:{payload['revision'] if payload else 'default'}",
+        "is_default": payload is None,
+    }
+
+
+async def read_configuration(db: AsyncSession) -> dict:
+    record = (
+        await db.execute(select(SystemSetting).where(SystemSetting.category == CATEGORY))
+    ).scalar_one_or_none()
+    return effective_envelope(record.payload if record else None)
+
+
+async def publish_configuration(
+    db: AsyncSession, revision: str, config: CardImageConfiguration, actor: User
+) -> dict:
+    defaults, _ = bundled_configuration()
+    pending = {asset.id for asset in defaults.assets if asset.rights_status == "permission_pending"}
+    if any(
+        asset.id in pending and asset.rights_status != "permission_pending"
+        for asset in config.assets
+    ):
+        raise HTTPException(422, "Bundled image permission is still unverified")
+    initial = {
+        "config": defaults.model_dump(mode="json"),
+        "revision": "default",
+        "known_assets": [],
+        "known_rules": [],
+    }
+    await db.execute(
+        insert(SystemSetting)
+        .values(category=CATEGORY, payload=initial)
+        .on_conflict_do_nothing(index_elements=[SystemSetting.category])
+    )
+    record = (
+        await db.execute(
+            select(SystemSetting).where(SystemSetting.category == CATEGORY).with_for_update()
+        )
+    ).scalar_one()
+    if effective_envelope(record.payload)["revision"] != revision:
+        await db.rollback()
+        raise HTTPException(409, "Configuration changed. Reload before saving your edits.")
+    before = record.payload
+    record.payload = {
+        "config": config.model_dump(mode="json"),
+        "revision": uuid4().hex,
+        "known_assets": [asset.id for asset in defaults.assets],
+        "known_rules": [rule.id for rule in defaults.rules],
+    }
+    db.add(
+        AuditLog(
+            actor_id=actor.id,
+            actor_name=actor.username,
+            action="card_images.update",
+            target_type="system_setting",
+            target_id=CATEGORY,
+            result="success",
+            request_id=get_request_id(),
+            details={"before": before, "after": record.payload},
+        )
+    )
+    await db.commit()
+    return effective_envelope(record.payload)
+
+
+def resolve_image(config: CardImageConfiguration, context: dict[str, str]) -> dict:
+    assets = {asset.id: asset for asset in config.assets}
+    for scope in ("object", "model", "group", "category"):
+        value = context.get("entity_id" if scope == "object" else scope)
+        values = [value] if value else []
+        if scope == "category" and value != "*":
+            values.append("*")
+        for value in values:
+            value = value if scope == "object" else value.casefold()
+            rule = next(
+                (
+                    rule
+                    for rule in config.rules
+                    if rule.enabled
+                    and rule.entity_type == context["entity_type"]
+                    and rule.scope == scope
+                    and rule.value == value
+                ),
+                None,
+            )
+            if not rule:
+                continue
+            match = {"scope": scope, "value": value, "rule_id": rule.id}
+            if rule.suppress:
+                return {"image": None, "match": match, "suppressed": True}
+            asset = assets.get(rule.asset_id)
+            if asset and asset.enabled:
+                return {"image": asset.model_dump(mode="json"), "match": match, "suppressed": False}
+    return {"image": None, "match": None, "suppressed": False}

backend/app/services/card_image_defaults.py

@@ -0,0 +1,108 @@
+"""Read immutable bundled assets without inserting machine-specific database rows."""
+
+import hashlib
+import json
+from functools import lru_cache
+from pathlib import Path
+
+from app.schemas.card_images import CardImageAsset, CardImageConfiguration, CardImageRule
+
+REPO_ROOT = Path(__file__).resolve().parents[3]
+MANIFEST_PATH = REPO_ROOT / "frontend/public/earth/assets/cards/manifest.json"
+
+
+def _asset_from_manifest(item: dict) -> CardImageAsset:
+    pending = item.get("redistribution_status") == "permission_not_verified"
+    return CardImageAsset(
+        id=item["id"],
+        title=item["label"],
+        url=item["url"],
+        width=item["width"],
+        height=item["height"],
+        credit=item["credit"],
+        source_url=item.get("source_page") or "",
+        license=item["license"],
+        license_url=item.get("license_url") or "",
+        kind=item["kind"],
+        rights_status="permission_pending" if pending else "available",
+        enabled=item.get("enabled", not pending),
+        focus_x=item.get("focus_x", 50),
+        focus_y=item.get("focus_y", 50),
+        variants=item.get("variants", {}),
+    )
+
+
+@lru_cache(maxsize=1)
+def previous_bundled_assets() -> dict[str, list[CardImageAsset]]:
+    manifest = json.loads(MANIFEST_PATH.read_bytes())
+    return {
+        key: [CardImageAsset.model_validate(item) for item in value]
+        for key, value in manifest.get("previous_defaults", {}).items()
+    }
+
+
+@lru_cache(maxsize=1)
+def previous_bundled_rules() -> dict[str, CardImageRule]:
+    manifest = json.loads(MANIFEST_PATH.read_bytes())
+    return {
+        key: CardImageRule.model_validate(value)
+        for key, value in manifest.get("previous_rules", {}).items()
+    }
+
+
+@lru_cache(maxsize=1)
+def bundled_configuration() -> tuple[CardImageConfiguration, str]:
+    content = MANIFEST_PATH.read_bytes()
+    manifest = json.loads(content)
+    assets, rules = [], []
+    for item in manifest["assets"]:
+        asset = _asset_from_manifest(item)
+        assets.append(asset)
+        if not asset.enabled:
+            continue
+        hints = item.get("binding_hints", {})
+        for index, target in enumerate(hints.get("defaults", [])):
+            rules.append(
+                CardImageRule(
+                    id=target.get("id", f"builtin-{asset.id}-default-{index}"),
+                    entity_type=target["entity_type"],
+                    scope=target["scope"],
+                    value=target["value"],
+                    label=asset.title,
+                    asset_id=asset.id,
+                )
+            )
+        for index, entity_id in enumerate(hints.get("entity_ids", [])):
+            rules.append(
+                CardImageRule(
+                    id=f"builtin-{asset.id}-entity-{index}",
+                    entity_type=item["entity_type"],
+                    scope="object",
+                    value=entity_id,
+                    label=asset.title,
+                    asset_id=asset.id,
+                )
+            )
+        for norad_id in hints.get("norad_ids", []):
+            rules.append(
+                CardImageRule(
+                    id=f"builtin-{asset.id}-{norad_id}",
+                    entity_type="satellite",
+                    scope="object",
+                    value=str(norad_id),
+                    label=f"{asset.title} · NORAD {norad_id}",
+                    asset_id=asset.id,
+                )
+            )
+        if group := hints.get("constellation_group"):
+            rules.append(
+                CardImageRule(
+                    id=f"builtin-{asset.id}-group",
+                    entity_type="satellite",
+                    scope="group",
+                    value=group,
+                    label=asset.title,
+                    asset_id=asset.id,
+                )
+            )
+    return CardImageConfiguration(assets=assets, rules=rules), hashlib.sha256(content).hexdigest()

backend/app/services/card_image_files.py

@@ -0,0 +1,95 @@
+"""Validated raster uploads and content-addressed, bounded card-size derivatives."""
+
+import hashlib
+import io
+import re
+import warnings
+from uuid import uuid4
+from pathlib import Path
+import os
+
+from fastapi import HTTPException
+from PIL import Image, ImageOps, UnidentifiedImageError
+
+from app.schemas.card_images import CardImageAsset
+from app.services.card_image_defaults import REPO_ROOT
+
+ASSET_DIR = Path(
+    os.environ.get("PLANET_CARD_IMAGE_DIR", str(REPO_ROOT / "data/earth-card-images"))
+).resolve()
+MAX_UPLOAD_BYTES = 12 * 1024 * 1024
+MAX_PIXELS = 32_000_000
+FILE_PATTERN = re.compile(r"^[a-f0-9]{64}-(?:original\.(?:jpg|png|webp)|(?:640|1280)\.webp)$")
+FORMATS = {"JPEG": "jpg", "PNG": "png", "WEBP": "webp"}
+
+
+def uploaded_path(filename: str) -> Path:
+    if not FILE_PATTERN.fullmatch(filename):
+        raise HTTPException(404, "Image not found")
+    return ASSET_DIR / filename
+
+
+def store_image(content: bytes, title: str) -> CardImageAsset:
+    if len(content) > MAX_UPLOAD_BYTES:
+        raise HTTPException(413, "Image must be 12 MiB or smaller")
+    try:
+        with warnings.catch_warnings():
+            warnings.simplefilter("error", Image.DecompressionBombWarning)
+            image = Image.open(io.BytesIO(content))
+            extension = FORMATS.get(image.format)
+            if (
+                not extension
+                or image.width * image.height > MAX_PIXELS
+                or getattr(image, "is_animated", False)
+            ):
+                raise ValueError("Unsupported or oversized raster")
+            image.load()
+            image = ImageOps.exif_transpose(image).convert(
+                "RGBA" if "A" in image.getbands() or "transparency" in image.info else "RGB"
+            )
+    except (
+        UnidentifiedImageError,
+        OSError,
+        ValueError,
+        Image.DecompressionBombError,
+        Image.DecompressionBombWarning,
+    ) as error:
+        raise HTTPException(
+            422, "Use a valid, still JPEG, PNG or WebP image below 32 megapixels"
+        ) from error
+    digest = hashlib.sha256(content).hexdigest()
+    ASSET_DIR.mkdir(parents=True, exist_ok=True)
+    original = f"{digest}-original.{extension}"
+    original_path = uploaded_path(original)
+    if not original_path.exists():
+        temporary = original_path.with_suffix(f".{uuid4().hex}.tmp")
+        temporary.write_bytes(content)
+        temporary.replace(original_path)
+    variants = {}
+    for size, content in render_variants(image).items():
+        filename = f"{digest}-{size}.webp"
+        path = uploaded_path(filename)
+        if not path.exists():
+            temporary = path.with_suffix(f".{uuid4().hex}.tmp")
+            temporary.write_bytes(content)
+            temporary.replace(path)
+        variants[size] = f"/api/v1/earth/card-images/files/{filename}"
+    return CardImageAsset(
+        id=f"upload-{digest[:24]}",
+        title=title or "Image",
+        url=f"/api/v1/earth/card-images/files/{original}",
+        variants=variants,
+        width=image.width,
+        height=image.height,
+    )
+
+
+def render_variants(image: Image.Image) -> dict[str, bytes]:
+    result = {}
+    for size in (640, 1280):
+        resized = image.copy()
+        resized.thumbnail((size, size), Image.Resampling.LANCZOS)
+        buffer = io.BytesIO()
+        resized.save(buffer, format="WEBP", quality=93, method=4)
+        result[str(size)] = buffer.getvalue()
+    return result

backend/app/services/card_image_targets.py

@@ -0,0 +1,180 @@
+"""Bounded catalog queries and type defaults for every Earth object card."""
+
+from sqlalchemy import String, case, cast, func, literal, or_, select
+from sqlalchemy.ext.asyncio import AsyncSession
+
+from app.core.compute_types import (
+    COMPUTE_SITE_TYPES,
+    compute_site_type_expression,
+    compute_record_filter,
+)
+from app.models.bgp_incident import BGPIncident
+from app.models.bgp_collector_location import BGPCollectorLocation
+from app.models.collected_data import CollectedData
+from app.models.earth_interactable import EarthInteractable
+from app.models.vessel import VesselCurrentState
+from app.schemas.card_images import EntityType, MatchScope
+
+SOURCES = {
+    "satellite": ("celestrak_tle", "spacetrack_tle"),
+    "compute_center": ("top500", "epoch_ai_gpu"),
+    "cable": ("arcgis_cables",),
+    "landing_point": ("arcgis_landing_points",),
+}
+# Named constellation families, not inferred hardware models or generations.
+SATELLITE_GROUP_PATTERNS = (
+    ("STARLINK%", "starlink"),
+    ("IRIDIUM%", "iridium-next"),
+    ("BEIDOU-3%", "beidou-3"),
+    ("BEIDOU%", "beidou"),
+    ("QIANFAN%", "qianfan"),
+    ("ONEWEB%", "oneweb"),
+    ("KUIPER%", "kuiper"),
+)
+
+
+def collected_query(entity_type: EntityType):
+    record, metadata = CollectedData, CollectedData.extra_data
+    model = func.coalesce(
+        metadata["satellite_model" if entity_type == "satellite" else "model"].as_string(), ""
+    )
+    category = (
+        compute_site_type_expression(record)
+        if entity_type == "compute_center"
+        else literal(entity_type)
+    )
+    if entity_type == "satellite":
+        group = func.coalesce(
+            func.nullif(func.lower(metadata["constellation_group"].as_string()), ""),
+            case(
+                *[
+                    (func.upper(record.name).like(pattern), value)
+                    for pattern, value in SATELLITE_GROUP_PATTERNS
+                ],
+                else_="",
+            ),
+        )
+        entity_id = metadata["norad_cat_id"].as_string()
+    else:
+        group = func.coalesce(
+            metadata["operator"].as_string(), metadata["organization"].as_string(), ""
+        )
+        entity_id = record.source + ":" + record.source_id
+    return select(
+        record.name.label("name"),
+        cast(entity_id, String).label("entity_id"),
+        model.label("model"),
+        group.label("group"),
+        category.label("category"),
+    ).where(
+        (
+            compute_record_filter(record)
+            if entity_type == "compute_center"
+            else record.source.in_(SOURCES[entity_type])
+        ),
+        record.is_current.is_(True),
+        record.deleted_at.is_(None),
+        metadata["anonymous"].as_boolean().is_not(True),
+    )
+
+
+def target_query(entity_type: EntityType):
+    if entity_type in SOURCES:
+        return collected_query(entity_type)
+    if entity_type == "vessel":
+        model = VesselCurrentState
+        identity, name, group, category = model.mmsi, model.name, model.flag, model.vessel_type_name
+        conditions = []
+    elif entity_type == "bgp":
+        model = BGPIncident
+        identity, name, group, category = (
+            model.incident_key,
+            model.title,
+            model.source,
+            model.incident_type,
+        )
+        conditions = []
+    elif entity_type == "bgp_collector":
+        model = BGPCollectorLocation
+        identity, name, group, category = (
+            model.collector_id,
+            model.collector_id,
+            model.operator,
+            literal("bgp_collector"),
+        )
+        conditions = []
+    else:
+        model = EarthInteractable
+        identity, name, group, category = model.id, model.label, model.layer, model.kind
+        conditions = [model.is_deleted.is_(False)]
+    return select(
+        name.label("name"),
+        cast(identity, String).label("entity_id"),
+        literal("").label("model"),
+        func.coalesce(group, "").label("group"),
+        func.coalesce(category, entity_type).label("category"),
+    ).where(*conditions)
+
+
+def category_context(entity_type: EntityType) -> dict[str, str]:
+    return {
+        "entity_type": entity_type,
+        "entity_id": "",
+        "name": "",
+        "model": "",
+        "group": "",
+        "category": entity_type,
+    }
+
+
+async def get_target(db: AsyncSession, entity_type: EntityType, entity_id: str) -> dict[str, str]:
+    if entity_type == "compute_center" and entity_id.startswith("__category__:"):
+        category = entity_id.partition(":")[2]
+        if category in COMPUTE_SITE_TYPES:
+            return {**category_context(entity_type), "category": category}
+    if entity_id == "__category__":
+        return category_context(entity_type)
+    if entity_type in SOURCES and entity_type != "satellite":
+        source, separator, source_id = entity_id.partition(":")
+        if not separator or not source or not source_id:
+            return category_context(entity_type)
+        # Match the indexed identity columns before projecting JSON metadata.
+        statement = collected_query(entity_type).where(
+            CollectedData.source == source, CollectedData.source_id == source_id
+        )
+    else:
+        catalog = target_query(entity_type).subquery()
+        statement = select(catalog).where(catalog.c.entity_id == entity_id)
+    row = (await db.execute(statement.limit(1))).mappings().first()
+    if not row:
+        # Old map snapshots can outlive a catalog refresh. They may use the type
+        # illustration, but must not claim an unverified exact-object identity.
+        return category_context(entity_type)
+    context = {key: str(value or "") for key, value in row.items()}
+    return {**context, "entity_type": entity_type}
+
+
+async def search_targets(
+    db: AsyncSession, entity_type: EntityType, scope: MatchScope, query: str, limit: int
+) -> list[dict[str, str]]:
+    catalog = target_query(entity_type).subquery()
+    value = catalog.c["entity_id" if scope == "object" else scope]
+    label = catalog.c.name if scope == "object" else value
+    statement = select(value.label("value"), label.label("label")).where(
+        value.is_not(None), value != ""
+    )
+    if query:
+        escaped = query.replace("\\", "\\\\").replace("%", "\\%").replace("_", "\\_")
+        statement = statement.where(
+            or_(value.ilike(f"%{escaped}%", escape="\\"), label.ilike(f"%{escaped}%", escape="\\"))
+        )
+    rows = (await db.execute(statement.distinct().order_by(label, value).limit(limit))).mappings()
+    items = [{"value": row["value"], "label": row["label"] or row["value"]} for row in rows]
+    if scope == "category" and not query:
+        known = {item["value"] for item in items}
+        if entity_type == "compute_center":
+            items.extend(
+                {"value": kind, "label": kind} for kind in COMPUTE_SITE_TYPES if kind not in known
+            )
+        items.insert(0, {"value": "*", "label": "*"})
+    return items

backend/app/services/collectors/aisstream.py

@@ -322,6 +322,21 @@ class AISStreamCollector(BaseCollector):
             last_success_at=now if data else None,
             lag_seconds=max((now - latest_observed_at).total_seconds(), 0),
         )
+        if snapshot_id is not None:
+            from app.models.data_snapshot import DataSnapshot
+
+            snapshot = await db.get(DataSnapshot, snapshot_id)
+            if snapshot:
+                snapshot.record_count = records_added
+                snapshot.status = "success"
+                snapshot.completed_at = now
+                snapshot.summary = {
+                    "created": records_added,
+                    "updated": 0,
+                    "unchanged": 0,
+                    "deleted": 0,
+                    "storage": "ais_raw_observations",
+                }
         await db.commit()
         await self.update_progress(records_added, force=True)
         return records_added

backend/app/services/collectors/base.py

@@ -12,11 +12,11 @@ from sqlalchemy.ext.asyncio import AsyncSession
 
 from app.core.collected_data_fields import build_dynamic_metadata, get_record_field
 from app.core.countries import normalize_country
+from app.core.enums import JobStatus, SnapshotStatus
 from app.core.logging import get_logger
 from app.core.time import to_iso8601_utc
 from app.core.websocket.broadcaster import broadcaster
 from app.services.business_logs import emit_business_log, exception_context
-from app.services.earth_layer_adapters import get_earth_update_layers_for_source
 
 
 logger = get_logger(__name__, service="collector")
@@ -31,6 +31,7 @@ class BaseCollector(ABC):
     frequency_hours: int = 4
     data_type: str = "generic"
     fail_on_empty: bool = False
+    preserve_snapshot_on_failure: bool = False
 
     def __init__(self):
         self._current_task = None
@@ -238,7 +239,7 @@ class BaseCollector(ABC):
             snapshot = await db.get(DataSnapshot, snapshot_id)
             if snapshot:
                 parent_snapshot_id = snapshot.parent_snapshot_id
-                snapshot.status = "cancelled"
+                snapshot.status = SnapshotStatus.CANCELLED.value
                 snapshot.is_current = False
                 snapshot.completed_at = datetime.now(UTC)
                 summary = dict(snapshot.summary or {})
@@ -308,7 +309,7 @@ class BaseCollector(ABC):
             task.datasource_id = datasource_id
             task.source = task.source or self.name
             task.task_type = task.task_type or "collect"
-            task.status = "running"
+            task.status = JobStatus.RUNNING.value
             task.phase = "queued"
             task.started_at = task.started_at or start_time
             task.completed_at = None
@@ -361,6 +362,8 @@ class BaseCollector(ABC):
                 context={"task_id": task_id, "raw_count": len(raw_data)},
             )
             data = self.transform(raw_data)
+            if self.fail_on_empty and not data:
+                raise RuntimeError(f"Collector {self.name} produced no valid records")
             await self._log_collection_event(
                 "collector.phase.transforming.success",
                 "Collector transform phase completed",
@@ -393,7 +396,7 @@ class BaseCollector(ABC):
                 },
             )
 
-            task.status = "success"
+            task.status = JobStatus.SUCCESS.value
             task.phase = "completed"
             task.phase_progress = 100.0
             task.phase_message = "采集完成"
@@ -428,7 +431,7 @@ class BaseCollector(ABC):
             }
         except asyncio.CancelledError:
             await db.rollback()
-            task.status = "cancelled"
+            task.status = JobStatus.CANCELLED.value
             task.phase = "cancelled"
             task.phase_message = "采集已取消"
             task.error_message = "Collection cancelled by operator and rolled back"
@@ -456,7 +459,11 @@ class BaseCollector(ABC):
             raise
         except Exception as e:
             await db.rollback()
-            task.status = "failed"
+            if snapshot_id is not None and self.preserve_snapshot_on_failure:
+                await self._rollback_incomplete_run(
+                    db, task_id=task_id, snapshot_id=snapshot_id, reason=str(e)
+                )
+            task.status = JobStatus.FAILED.value
             task.phase = "failed"
             task.phase_message = str(e)
             task.error_message = str(e)
@@ -464,7 +471,9 @@ class BaseCollector(ABC):
             if snapshot_id is not None:
                 snapshot = await db.get(DataSnapshot, snapshot_id)
                 if snapshot:
-                    snapshot.status = "failed"
+                    snapshot.status = SnapshotStatus.FAILED.value
+                    if self.preserve_snapshot_on_failure:
+                        snapshot.is_current = False
                     snapshot.completed_at = datetime.now(UTC)
                     snapshot.summary = {"error": str(e)}
             await db.commit()
@@ -509,7 +518,7 @@ class BaseCollector(ABC):
                 if snapshot:
                     snapshot.record_count = 0
                     snapshot.summary = {"created": 0, "updated": 0, "unchanged": 0}
-                    snapshot.status = "success"
+                    snapshot.status = SnapshotStatus.SUCCESS.value
                     snapshot.completed_at = datetime.now(UTC)
                     await db.commit()
             return 0
@@ -642,7 +651,7 @@ class BaseCollector(ABC):
             snapshot = await db.get(DataSnapshot, snapshot_id)
             if snapshot:
                 snapshot.record_count = records_added
-                snapshot.status = "success"
+                snapshot.status = SnapshotStatus.SUCCESS.value
                 snapshot.completed_at = datetime.now(UTC)
                 snapshot.summary = {
                     "created": created_count,

backend/app/services/collectors/celestrak.py

@@ -18,21 +18,27 @@ from app.services.business_logs import emit_business_log, exception_context
 from app.services.collectors.base import BaseCollector
 from app.services.collectors.downloads import DownloadHTTPStatusError, ResumableFileDownloader
 
-
 logger = get_logger(__name__, service="collector")
 ACTIVE_GROUP = "active"
 FALLBACK_GROUPS = (
     "starlink",
     "gps-ops",
     "galileo",
-    "glonass",
+    "glo-ops",
     "beidou",
-    "leo",
     "geo",
     "iridium-next",
+    "stations",
+    "visual",
+    "weather",
+    "science",
+    "cubesat",
+    "amateur",
+    "last-30-days",
 )
 FETCH_RETRY_ATTEMPTS = 3
 FETCH_RETRY_BASE_DELAY_SECONDS = 0.8
+CELESTRAK_CACHE_TTL_SECONDS = 2 * 60 * 60
 CELESTRAK_NOT_UPDATED_MARKER = "GP data has not updated since your last successful"
 
 
@@ -45,6 +51,8 @@ class CelesTrakTLECollector(BaseCollector):
     _downloader = ResumableFileDownloader(
         cache_namespace="celestrak",
         default_accept="application/json",
+        preflight_head=False,
+        cache_ttl_seconds=CELESTRAK_CACHE_TTL_SECONDS,
     )
 
     @property
@@ -189,7 +197,9 @@ class CelesTrakTLECollector(BaseCollector):
                     if not is_final_attempt:
                         await asyncio.sleep(FETCH_RETRY_BASE_DELAY_SECONDS * attempt)
 
-        raise RuntimeError(f"CelesTrak active satellite download failed after retries: {last_error}")
+        raise RuntimeError(
+            f"CelesTrak active satellite download failed after retries: {last_error}"
+        )
 
     async def _fetch_fallback_groups(
         self,
@@ -230,17 +240,18 @@ class CelesTrakTLECollector(BaseCollector):
                     )
                 except DownloadHTTPStatusError as exc:
                     if not self._is_not_updated_response(exc):
-                        raise RuntimeError(f"CelesTrak fallback group '{group}' download failed: {exc}") from exc
-                    cached_path = self._downloader.get_cached_file(
+                        raise RuntimeError(
+                            f"CelesTrak fallback group '{group}' download failed: {exc}"
+                        ) from exc
+                    body_path = self._downloader.get_cached_file(
                         group_url,
                         ".json",
                         validate_existing=self._validate_json_file,
                     )
-                    if cached_path is None:
+                    if body_path is None:
                         raise RuntimeError(
                             f"CelesTrak fallback group '{group}' has not updated and no local cached copy is available"
                         ) from exc
-                    body_path = cached_path
 
                 group_records = await self._load_downloaded_payload(
                     body_path,
@@ -341,13 +352,13 @@ class CelesTrakTLECollector(BaseCollector):
                 commit=True,
             )
 
-    @staticmethod
-    def _validate_json_file(path: Path) -> bool:
+    @classmethod
+    def _validate_json_file(cls, path: Path) -> bool:
         try:
-            data = json.loads(path.read_text(encoding="utf-8"))
-        except (OSError, json.JSONDecodeError, UnicodeDecodeError):
+            cls._load_active_payload(path)
+        except RuntimeError:
             return False
-        return isinstance(data, list)
+        return True
 
     async def _log_parse_failure(self, exc: Exception) -> None:
         await emit_business_log(
@@ -368,7 +379,8 @@ class CelesTrakTLECollector(BaseCollector):
             ),
         )
 
-    def _load_active_payload(self, path: Path) -> List[Dict[str, Any]]:
+    @staticmethod
+    def _load_active_payload(path: Path) -> List[Dict[str, Any]]:
         try:
             raw = json.loads(path.read_text(encoding="utf-8"))
         except (OSError, json.JSONDecodeError, UnicodeDecodeError) as exc:
@@ -384,7 +396,9 @@ class CelesTrakTLECollector(BaseCollector):
             else:
                 invalid_count += 1
         if invalid_count:
-            raise RuntimeError(f"CelesTrak active payload contains {invalid_count} invalid record(s)")
+            raise RuntimeError(
+                f"CelesTrak active payload contains {invalid_count} invalid record(s)"
+            )
         if not records:
             raise RuntimeError("CelesTrak active payload contains no satellite records")
         return records

backend/app/services/collectors/compute_base.py

@@ -0,0 +1,82 @@
+"""Quality gates shared by the two compute sources."""
+
+from datetime import datetime
+
+from sqlalchemy import select
+from sqlalchemy.ext.asyncio import AsyncSession
+
+from app.core.countries import normalize_country
+from app.models.compute_center_location import ComputeCenterLocationRecord
+from app.models.data_snapshot import DataSnapshot
+from app.services.collectors.base import BaseCollector
+
+
+class ComputeCollector(BaseCollector):
+    fail_on_empty = True
+    preserve_snapshot_on_failure = True
+
+    async def _create_snapshot(
+        self, db: AsyncSession, task_id: int, data: list[dict], started_at: datetime
+    ) -> int:
+        previous_count = await db.scalar(
+            select(DataSnapshot.record_count)
+            .where(
+                DataSnapshot.source == self.name,
+                DataSnapshot.is_current.is_(True),
+                DataSnapshot.status == "success",
+            )
+            .order_by(DataSnapshot.id.desc())
+            .limit(1)
+        )
+        if previous_count and len(data) < previous_count * 0.6:
+            raise ValueError("Compute source lost over 40% of rows; previous snapshot retained")
+        return await super()._create_snapshot(db, task_id, data, started_at)
+
+    async def _save_data(self, db: AsyncSession, data: list[dict], **kwargs) -> int:
+        if self.name == "top500":
+            await self._copy_verified_locations(db, data)
+        return await super()._save_data(db, data, **kwargs)
+
+    async def _copy_verified_locations(self, db: AsyncSession, data: list[dict]) -> None:
+        """Keep historical location keys; copy only unambiguous exact identities.
+
+        Never migrate by rank: the machine occupying rank 1 changes over time.
+        """
+        names = [item["name"] for item in data]
+        locations = (
+            (
+                await db.execute(
+                    select(ComputeCenterLocationRecord).where(
+                        ComputeCenterLocationRecord.source == self.name,
+                        ComputeCenterLocationRecord.name.in_(names),
+                    )
+                )
+            )
+            .scalars()
+            .all()
+        )
+        existing = {location.source_id for location in locations}
+        for item in data:
+            if item["source_id"] in existing:
+                continue
+            country = normalize_country(item["country"])
+            if not country:
+                continue
+            matches = [
+                location
+                for location in locations
+                if location.name == item["name"]
+                and normalize_country(location.country) == country
+                and (not location.site or location.site == item.get("metadata", {}).get("site"))
+                and not location.source_id.startswith("top500_system_")
+            ]
+            if len(matches) != 1:
+                continue
+            old = matches[0]
+            values = {
+                column.name: getattr(old, column.name)
+                for column in old.__table__.columns
+                if column.name not in {"id", "source_id", "created_at", "updated_at"}
+            }
+            db.add(ComputeCenterLocationRecord(**values, source_id=item["source_id"]))
+            existing.add(item["source_id"])

backend/app/services/collectors/downloads.py

@@ -12,7 +12,6 @@ from typing import Any, Awaitable, Callable
 
 import httpx
 
-
 ProgressCallback = Callable[[int, int | None], Awaitable[None]]
 ValidateCallback = Callable[[Path], bool]
 
@@ -52,10 +51,14 @@ class ResumableFileDownloader:
         user_agent: str = "Planet-Intelligence-System/1.0 (Python/collector)",
         default_accept: str = "*/*",
         cache_root: Path | None = None,
+        preflight_head: bool = True,
+        cache_ttl_seconds: float = 0,
     ) -> None:
         self._cache_dir = (cache_root or default_download_cache_root()) / cache_namespace
         self._user_agent = user_agent
         self._default_accept = default_accept
+        self._preflight_head = preflight_head
+        self._cache_ttl_seconds = cache_ttl_seconds
 
     @staticmethod
     def _cache_key(url: str) -> str:
@@ -93,8 +96,9 @@ class ResumableFileDownloader:
         if not meta_path.exists():
             return {}
         try:
-            return json.loads(meta_path.read_text(encoding="utf-8"))
-        except (json.JSONDecodeError, OSError):
+            data = json.loads(meta_path.read_text(encoding="utf-8"))
+            return data if isinstance(data, dict) else {}
+        except json.JSONDecodeError, OSError:
             return {}
 
     @staticmethod
@@ -111,20 +115,25 @@ class ResumableFileDownloader:
             return last_modified == str(meta.get("last_modified") or "").strip()
         return True
 
+    @staticmethod
+    def _response_info(response: httpx.Response) -> dict[str, Any]:
+        # httpx yields decoded bytes; encoded Content-Length cannot validate them.
+        raw_length = response.headers.get("content-length")
+        encoded = response.headers.get("content-encoding", "identity") != "identity"
+        return {
+            "etag": response.headers.get("etag"),
+            "last_modified": response.headers.get("last-modified"),
+            "content_length": int(raw_length) if raw_length and not encoded else None,
+            "accept_ranges": (response.headers.get("accept-ranges") or "").lower(),
+        }
+
     async def fetch_remote_info(self, client: httpx.AsyncClient, url: str) -> dict[str, Any]:
         try:
             response = await client.head(url)
             if response.status_code >= 400:
                 return {}
-            content_length_raw = response.headers.get("content-length")
-            content_length = int(content_length_raw) if content_length_raw else None
-            return {
-                "etag": response.headers.get("etag"),
-                "last_modified": response.headers.get("last-modified"),
-                "content_length": content_length,
-                "accept_ranges": (response.headers.get("accept-ranges") or "").lower(),
-            }
-        except (httpx.HTTPError, ValueError):
+            return self._response_info(response)
+        except httpx.HTTPError, ValueError:
             return {}
 
     async def download_file(
@@ -139,14 +148,24 @@ class ResumableFileDownloader:
     ) -> Path:
         self._cache_dir.mkdir(parents=True, exist_ok=True)
         final_path, part_path, meta_path = self._cache_paths(url, extension)
+        cached_path = self.get_cached_file(url, extension, validate_existing=validate_existing)
+        if cached_path and 0 <= time.time() - cached_path.stat().st_mtime < self._cache_ttl_seconds:
+            size = cached_path.stat().st_size
+            if progress_callback:
+                await progress_callback(size, size)
+            return cached_path
         meta = self._load_meta(meta_path)
-        remote = await self.fetch_remote_info(client, url)
+        remote = await self.fetch_remote_info(client, url) if self._preflight_head else {}
         expected_size = remote.get("content_length")
 
         if final_path.exists():
             local_size = final_path.stat().st_size
             size_match = expected_size is None or local_size == expected_size
-            if self._validators_match(meta, remote) and size_match:
+            if (
+                (remote.get("etag") or remote.get("last_modified"))
+                and self._validators_match(meta, remote)
+                and size_match
+            ):
                 if validate_existing and not validate_existing(final_path):
                     final_path.unlink(missing_ok=True)
                 else:
@@ -181,9 +200,13 @@ class ResumableFileDownloader:
                 headers["If-Range"] = str(remote.get("last_modified"))
 
         async with client.stream("GET", url, headers=headers) as response:
-            if response.status_code == 304 and final_path.exists():
-                if progress_callback and expected_size and expected_size > 0:
-                    await progress_callback(expected_size, expected_size)
+            if response.status_code == 304:
+                if cached_path is None:
+                    raise RuntimeError(f"Server returned 304 without a valid local cache for {url}")
+                final_path.touch()
+                size = final_path.stat().st_size
+                if progress_callback:
+                    await progress_callback(size, size)
                 return final_path
             if response.status_code >= 400:
                 body = (await response.aread()).decode("utf-8", errors="replace")
@@ -194,12 +217,16 @@ class ResumableFileDownloader:
             else:
                 mode = "wb"
                 resume_from = 0
+                remote = self._response_info(response)
+                expected_size = remote.get("content_length")
 
             downloaded = resume_from
             last_emit_bytes = 0
             last_emit_time = time.monotonic()
             min_emit_bytes = (
-                max(expected_size // 150, 512 * 1024) if expected_size and expected_size > 0 else 1024 * 1024
+                max(expected_size // 150, 512 * 1024)
+                if expected_size and expected_size > 0
+                else 1024 * 1024
             )
 
             with part_path.open(mode) as f:
@@ -230,6 +257,10 @@ class ResumableFileDownloader:
                 f"Resumable download incomplete for {url}: expected={expected_size}, got={final_size}"
             )
 
+        # Never replace the last good payload with an HTML error or invalid JSON.
+        if validate_existing and not validate_existing(part_path):
+            part_path.unlink(missing_ok=True)
+            raise RuntimeError(f"Downloaded file validation failed for {url}")
         part_path.replace(final_path)
         self._save_meta(
             meta_path,
@@ -242,9 +273,6 @@ class ResumableFileDownloader:
             },
         )
 
-        if validate_existing and not validate_existing(final_path):
-            raise RuntimeError(f"Downloaded file validation failed for {url}")
-
         if progress_callback and expected_size and expected_size > 0:
             await progress_callback(expected_size, expected_size)
 

backend/app/services/collectors/epoch_ai.py

@@ -1,119 +1,135 @@
-"""Epoch AI GPU Clusters Collector
-
-Collects data from Epoch AI GPU clusters tracking.
-https://epoch.ai/data/gpu-clusters
-"""
+"""Epoch's official CSV; preserve precision, source dates and anonymous records."""
 
+import csv
+import hashlib
+import io
+import json
+import math
 import re
-from typing import Dict, Any, List
-from datetime import UTC, datetime
-from bs4 import BeautifulSoup
+from urllib.parse import urlparse
+
 import httpx
 
-from app.services.collectors.base import BaseCollector
+from app.core.countries import normalize_country
+from app.services.collectors.compute_base import ComputeCollector
+from app.services.epoch_location import assess_epoch_location
+
+EPOCH_CSV_URL = "https://epoch.ai/data/gpu_clusters.csv"
 
 
-
-class EpochAIGPUCollector(BaseCollector):
+class EpochAIGPUCollector(ComputeCollector):
     name = "epoch_ai_gpu"
     priority = "P0"
     module = "L1"
     frequency_hours = 6
     data_type = "gpu_cluster"
 
-    async def fetch(self) -> List[Dict[str, Any]]:
-        """Fetch Epoch AI GPU clusters data from webpage"""
-        url = self._resolved_url or ""
-
-        async with httpx.AsyncClient(timeout=60.0) as client:
+    async def fetch(self) -> list[dict]:
+        configured = self._resolved_url or EPOCH_CSV_URL
+        parsed = urlparse(configured)
+        url = (
+            EPOCH_CSV_URL
+            if parsed.hostname == "epoch.ai" and not parsed.path.endswith(".csv")
+            else configured
+        )
+        async with httpx.AsyncClient(timeout=60, follow_redirects=True) as client:
             response = await client.get(url)
             response.raise_for_status()
-            return self.parse_response(response.text)
+        return self.parse_response(response.text)
 
-    def parse_response(self, html: str) -> List[Dict[str, Any]]:
-        """Parse Epoch AI webpage to extract GPU cluster data"""
-        data = []
-        soup = BeautifulSoup(html, "html.parser")
+    def parse_response(self, text: str) -> list[dict]:
+        reader = csv.DictReader(io.StringIO(text.lstrip("\ufeff")))
+        required = {"Name", "Country", "Status", "First Operational Date", "16-bit OP/s (log)"}
+        if not required.issubset(reader.fieldnames or []):
+            raise ValueError(
+                "Epoch CSV schema changed or HTML returned; previous snapshot retained"
+            )
+        entries = []
+        seen = set()
+        anonymous_counts = {}
+        for row in reader:
+            name = (row.get("Name") or "").strip()
+            country = normalize_country(row.get("Country"))
+            if not name:
+                raise ValueError("Epoch row has missing identity")
+            source_id = f"epoch_{re.sub(r'[^a-zA-Z0-9]', '_', name.lower())}"
+            anonymous = "anonym" in name.lower()
+            if anonymous:
+                # Public anonymous rows have no facility identity. Preserve
+                # multiplicity within snapshots, without guessing names.
+                digest = hashlib.sha256(json.dumps(row, sort_keys=True).encode()).hexdigest()[:24]
+                anonymous_counts[digest] = anonymous_counts.get(digest, 0) + 1
+                source_id = f"epoch_anonymous_{digest}_{anonymous_counts[digest]}"
+            if source_id in seen:
+                raise ValueError("Epoch identity collision; previous snapshot retained")
+            seen.add(source_id)
+            raw_date = row.get("First Operational Date", "")
+            reference_date = raw_date if re.fullmatch(r"\d{4}-\d{2}-\d{2}", raw_date) else None
+            value = None
+            raw_value = row.get("16-bit OP/s (log)", "")
+            if raw_value:
+                exponent = float(raw_value)
+                if not math.isfinite(exponent) or not 0 <= exponent <= 30:
+                    raise ValueError("Epoch performance exponent out of bounds")
+                value = 10**exponent
+            anonymous = anonymous or (country == "中国" and not row.get("Location"))
+            location = row.get("Location")
+            location_assessment = assess_epoch_location(
+                location,
+                country=country,
+                raw_country=row.get("Country"),
+            )
+            entries.append(
+                {
+                    "source_id": source_id,
+                    "name": name,
+                    "country": country,
+                    "value": value,
+                    "unit": "FLOP/s",
+                    "reference_date": reference_date,
+                    "latitude": None if anonymous else self.coordinate(row.get("latitude"), 90),
+                    "longitude": None if anonymous else self.coordinate(row.get("longitude"), 180),
+                    "metadata": {
+                        "compute_contract": "v1",
+                        "precision": "FP16",
+                        "measurement": "theoretical",
+                        "source_url": EPOCH_CSV_URL,
+                        "raw_reference_date": raw_date,
+                        "status": row.get("Status"),
+                        "certainty": row.get("Certainty"),
+                        "operator": row.get("Owner"),
+                        "gpu_type": row.get("Chip type (primary)"),
+                        "gpu_count": row.get("Chip quantity (primary)"),
+                        "location": location,
+                        "location_precision": location_assessment.precision,
+                        "location_precision_reason": location_assessment.reason,
+                        "raw_country": row.get("Country"),
+                        "raw_performance": raw_value,
+                        "performance_encoding": "log10",
+                        "builds_upon": row.get("Builds Upon"),
+                        "superseded_by": row.get("Superseded by"),
+                        "possible_duplicate": row.get("Possible Duplicate", "").lower() == "true",
+                        "include_in_standard_analysis": row.get(
+                            "Include in Standard Analysis", ""
+                        ).lower()
+                        == "true",
+                        "rounded": country == "中国",
+                        "anonymous": anonymous,
+                        "identity_scope": "snapshot_record" if anonymous else "named_cluster",
+                        "country_unresolved": country is None,
+                        "sources": [
+                            row[f"Source {i}"] for i in range(1, 6) if row.get(f"Source {i}")
+                        ],
+                    },
+                }
+            )
+        return entries
 
-        # Try to find data table on the page
-        tables = soup.find_all("table")
-        for table in tables:
-            rows = table.find_all("tr")
-            for row in rows[1:]:  # Skip header
-                cells = row.find_all(["td", "th"])
-                if len(cells) >= 5:
-                    try:
-                        cluster_name = cells[0].get_text(strip=True)
-                        if not cluster_name or cluster_name in ["Cluster", "System", "Name"]:
-                            continue
-
-                        location_cell = cells[1].get_text(strip=True) if len(cells) > 1 else ""
-                        country, city = self._parse_location(location_cell)
-
-                        perf_cell = cells[2].get_text(strip=True) if len(cells) > 2 else ""
-
-                        entry = {
-                            "source_id": f"epoch_{re.sub(r'[^a-zA-Z0-9]', '_', cluster_name.lower())}",
-                            "name": cluster_name,
-                            "country": country,
-                            "city": city,
-                            "latitude": "",
-                            "longitude": "",
-                            "value": self._parse_performance(perf_cell),
-                            "unit": "TFlop/s",
-                            "metadata": {
-                                "raw_data": perf_cell,
-                            },
-                            "reference_date": datetime.now(UTC).strftime("%Y-%m-%d"),
-                        }
-                        data.append(entry)
-                    except (ValueError, IndexError, AttributeError):
-                        continue
-
-        # If no table found, return sample data
-        if not data:
-            data = self._get_sample_data()
-
-        return data
-
-    def _parse_location(self, location: str) -> tuple:
-        """Parse location string into country and city"""
-        if not location:
-            return "", ""
-        if "," in location:
-            parts = location.rsplit(",", 1)
-            city = parts[0].strip()
-            country = parts[1].strip() if len(parts) > 1 else ""
-            return country, city
-        return location, ""
-
-    def _parse_performance(self, perf: str) -> str:
-        """Parse performance string to extract value"""
-        if not perf:
-            return "0"
-        match = re.search(r"([\d,.]+)\s*(TFlop/s|PFlop/s|GFlop/s)?", perf, re.I)
-        if match:
-            return match.group(1).replace(",", "")
-        match = re.search(r"([\d,.]+)", perf)
-        if match:
-            return match.group(1).replace(",", "")
-        return "0"
-
-    def _get_sample_data(self) -> List[Dict[str, Any]]:
-        """Return sample data for testing when scraping fails"""
-        return [
-            {
-                "source_id": "epoch_sample_1",
-                "name": "Sample GPU Cluster",
-                "country": "United States",
-                "city": "San Francisco, CA",
-                "latitude": "",
-                "longitude": "",
-                "value": "1000",
-                "unit": "TFlop/s",
-                "metadata": {
-                    "note": "Sample data - Epoch AI page structure may vary",
-                },
-                "reference_date": datetime.now(UTC).strftime("%Y-%m-%d"),
-            },
-        ]
+    @staticmethod
+    def coordinate(value: str | None, bound: int) -> float | None:
+        if not value:
+            return None
+        number = float(value)
+        if not math.isfinite(number) or abs(number) > bound:
+            raise ValueError("Epoch coordinate out of bounds")
+        return number

backend/app/services/collectors/news_live_streams.py

@@ -35,7 +35,7 @@ class NewsLiveStreamsCollector(BaseCollector):
     DEFAULT_IPTV_ORG_LOGOS_URL = "https://iptv-org.github.io/api/logos.json"
     DEFAULT_IPTV_ORG_NEWS_CATEGORIES = ("news", "business", "weather")
     DEFAULT_IPTV_ORG_EXCLUDE_CATEGORIES = ("music", "sports", "kids", "entertainment")
-    DEFAULT_IPTV_ORG_MAX_SOURCES = 120
+    DEFAULT_IPTV_ORG_MAX_SOURCES = 0  # Zero keeps the complete matching channel catalog.
 
     async def fetch(self) -> list[dict[str, Any]]:
         request_url = (self._resolved_url or "").strip()
@@ -445,7 +445,7 @@ class NewsLiveStreamsCollector(BaseCollector):
                     "reference_date": datetime.now(UTC).isoformat(),
                 }
             )
-            if len(normalized) >= max_sources:
+            if max_sources > 0 and len(normalized) >= max_sources:
                 break
 
         return normalized

backend/app/services/collectors/peeringdb.py

@@ -11,17 +11,20 @@ To get higher limits, set PEERINGDB_API_KEY environment variable.
 """
 
 import asyncio
-import os
-from typing import Dict, Any, List
 from datetime import UTC, datetime
+import os
+from typing import Any, Dict, List
+from urllib.parse import urlencode
 
 import httpx
-from urllib.parse import urlencode
+
+from app.core.logging import get_logger
 from app.services.collectors.base import HTTPCollector
 
 
 # PeeringDB API key - read from environment variable
 PEERINGDB_API_KEY = os.environ.get("PEERINGDB_API_KEY", "")
+logger = get_logger(__name__, service="collector")
 
 
 class PeeringDBIXPCollector(HTTPCollector):
@@ -39,6 +42,7 @@ class PeeringDBIXPCollector(HTTPCollector):
             "User-Agent": "Planet-Intelligence-System/1.0 (Python/collector)",
             "Accept": "application/json",
         }
+
     @property
     def request_url(self) -> str:
         base = self._resolved_url or self.base_url
@@ -61,7 +65,11 @@ class PeeringDBIXPCollector(HTTPCollector):
                     if response.status_code == 429:
                         # Rate limited - wait and retry with exponential backoff
                         delay = base_delay * (2**attempt)
-                        print(f"PeeringDB rate limited, waiting {delay}s before retry...")
+                        logger.warning_event(
+                            "PeeringDB rate limited; retrying after delay",
+                            event="collector.peeringdb.rate_limited",
+                            context={"delay_seconds": delay, "attempt": attempt + 1},
+                        )
                         await asyncio.sleep(delay)
                         last_error = "Rate limited"
                         continue
@@ -72,13 +80,21 @@ class PeeringDBIXPCollector(HTTPCollector):
             except httpx.HTTPStatusError as e:
                 if e.response.status_code == 429:
                     delay = base_delay * (2**attempt)
-                    print(f"PeeringDB rate limited, waiting {delay}s before retry...")
+                    logger.warning_event(
+                        "PeeringDB rate limited; retrying after delay",
+                        event="collector.peeringdb.rate_limited",
+                        context={"delay_seconds": delay, "attempt": attempt + 1},
+                    )
                     await asyncio.sleep(delay)
                     last_error = "Rate limited"
                     continue
                 raise
 
-        print(f"Warning: PeeringDB collection failed after {max_retries} retries: {last_error}")
+        logger.warning_event(
+            "PeeringDB collection failed after retries",
+            event="collector.peeringdb.retries_exhausted",
+            context={"max_retries": max_retries, "last_error": last_error},
+        )
         return {}
 
     async def fetch(self) -> List[Dict[str, Any]]:
@@ -146,6 +162,7 @@ class PeeringDBNetworkCollector(HTTPCollector):
             "User-Agent": "Planet-Intelligence-System/1.0 (Python/collector)",
             "Accept": "application/json",
         }
+
     @property
     def request_url(self) -> str:
         base = self._resolved_url or self.base_url
@@ -167,7 +184,11 @@ class PeeringDBNetworkCollector(HTTPCollector):
 
                     if response.status_code == 429:
                         delay = base_delay * (2**attempt)
-                        print(f"PeeringDB rate limited, waiting {delay}s before retry...")
+                        logger.warning_event(
+                            "PeeringDB rate limited; retrying after delay",
+                            event="collector.peeringdb.rate_limited",
+                            context={"delay_seconds": delay, "attempt": attempt + 1},
+                        )
                         await asyncio.sleep(delay)
                         last_error = "Rate limited"
                         continue
@@ -178,13 +199,21 @@ class PeeringDBNetworkCollector(HTTPCollector):
             except httpx.HTTPStatusError as e:
                 if e.response.status_code == 429:
                     delay = base_delay * (2**attempt)
-                    print(f"PeeringDB rate limited, waiting {delay}s before retry...")
+                    logger.warning_event(
+                        "PeeringDB rate limited; retrying after delay",
+                        event="collector.peeringdb.rate_limited",
+                        context={"delay_seconds": delay, "attempt": attempt + 1},
+                    )
                     await asyncio.sleep(delay)
                     last_error = "Rate limited"
                     continue
                 raise
 
-        print(f"Warning: PeeringDB collection failed after {max_retries} retries: {last_error}")
+        logger.warning_event(
+            "PeeringDB collection failed after retries",
+            event="collector.peeringdb.retries_exhausted",
+            context={"max_retries": max_retries, "last_error": last_error},
+        )
         return {}
 
     async def fetch(self) -> List[Dict[str, Any]]:
@@ -254,6 +283,7 @@ class PeeringDBFacilityCollector(HTTPCollector):
             "User-Agent": "Planet-Intelligence-System/1.0 (Python/collector)",
             "Accept": "application/json",
         }
+
     @property
     def request_url(self) -> str:
         base = self._resolved_url or self.base_url
@@ -275,7 +305,11 @@ class PeeringDBFacilityCollector(HTTPCollector):
 
                     if response.status_code == 429:
                         delay = base_delay * (2**attempt)
-                        print(f"PeeringDB rate limited, waiting {delay}s before retry...")
+                        logger.warning_event(
+                            "PeeringDB rate limited; retrying after delay",
+                            event="collector.peeringdb.rate_limited",
+                            context={"delay_seconds": delay, "attempt": attempt + 1},
+                        )
                         await asyncio.sleep(delay)
                         last_error = "Rate limited"
                         continue
@@ -286,13 +320,21 @@ class PeeringDBFacilityCollector(HTTPCollector):
             except httpx.HTTPStatusError as e:
                 if e.response.status_code == 429:
                     delay = base_delay * (2**attempt)
-                    print(f"PeeringDB rate limited, waiting {delay}s before retry...")
+                    logger.warning_event(
+                        "PeeringDB rate limited; retrying after delay",
+                        event="collector.peeringdb.rate_limited",
+                        context={"delay_seconds": delay, "attempt": attempt + 1},
+                    )
                     await asyncio.sleep(delay)
                     last_error = "Rate limited"
                     continue
                 raise
 
-        print(f"Warning: PeeringDB collection failed after {max_retries} retries: {last_error}")
+        logger.warning_event(
+            "PeeringDB collection failed after retries",
+            event="collector.peeringdb.retries_exhausted",
+            context={"max_retries": max_retries, "last_error": last_error},
+        )
         return {}
 
     async def fetch(self) -> List[Dict[str, Any]]:

backend/app/services/collectors/spacetrack.py

@@ -1,17 +1,21 @@
-"""Space-Track TLE Collector
+"""Space-Track TLE Collector.
 
 Collects satellite TLE (Two-Line Element) data from Space-Track.org.
 API documentation: https://www.space-track.org/documentation
 """
 
-import json
-from typing import Dict, Any, List
-import httpx
+from typing import Any, Dict, List
 from urllib.parse import urlparse
 
-from app.services.collectors.base import BaseCollector
+import httpx
+
 from app.core.data_sources import get_data_sources_config
+from app.core.logging import get_logger
 from app.core.satellite_tle import build_tle_lines_from_elements
+from app.services.collectors.base import BaseCollector
+
+
+logger = get_logger(__name__, service="collector")
 
 
 class SpaceTrackTLECollector(BaseCollector):
@@ -53,10 +57,16 @@ class SpaceTrackTLECollector(BaseCollector):
         password = settings.SPACETRACK_PASSWORD
 
         if not username or not password:
-            print("SPACETRACK: No credentials configured, using sample data")
+            logger.warning_event(
+                "Space-Track credentials are not configured; using sample data",
+                event="collector.spacetrack.credentials_missing",
+            )
             return self._get_sample_data()
 
-        print(f"SPACETRACK: Attempting to fetch TLE data with username: {username}")
+        logger.info_event(
+            "Space-Track TLE fetch started",
+            event="collector.spacetrack.fetch.start",
+        )
 
         try:
             async with httpx.AsyncClient(
@@ -78,11 +88,17 @@ class SpaceTrackTLECollector(BaseCollector):
                         "password": password,
                     },
                 )
-                print(f"SPACETRACK: Login response status: {login_response.status_code}")
-                print(f"SPACETRACK: Login response URL: {login_response.url}")
+                logger.info_event(
+                    "Space-Track login response received",
+                    event="collector.spacetrack.login.response",
+                    context={"status_code": login_response.status_code},
+                )
 
                 if login_response.status_code == 403:
-                    print("SPACETRACK: Trying alternate login method...")
+                    logger.warning_event(
+                        "Space-Track login returned forbidden; trying alternate method",
+                        event="collector.spacetrack.login.forbidden",
+                    )
 
                     async with httpx.AsyncClient(
                         timeout=120.0,
@@ -90,11 +106,6 @@ class SpaceTrackTLECollector(BaseCollector):
                     ) as alt_client:
                         await alt_client.get(f"{self.site_root}/")
 
-                        form_data = {
-                            "username": username,
-                            "password": password,
-                            "query": "class/gp/NORAD_CAT_ID/25544/format/json",
-                        }
                         alt_login = await alt_client.post(
                             self.login_url,
                             data={
@@ -102,77 +113,59 @@ class SpaceTrackTLECollector(BaseCollector):
                                 "password": password,
                             },
                         )
-                        print(f"SPACETRACK: Alt login status: {alt_login.status_code}")
+                        logger.info_event(
+                            "Space-Track alternate login response received",
+                            event="collector.spacetrack.alt_login.response",
+                            context={"status_code": alt_login.status_code},
+                        )
 
                         if alt_login.status_code == 200:
                             tle_response = await alt_client.get(self.probe_url)
                             if tle_response.status_code == 200:
                                 data = tle_response.json()
-                                print(f"SPACETRACK: Received {len(data)} records via alt method")
+                                logger.info_event(
+                                    "Space-Track alternate query completed",
+                                    event="collector.spacetrack.alt_query.completed",
+                                    context={"record_count": len(data)},
+                                )
                                 return data
 
                 if login_response.status_code != 200:
-                    print(f"SPACETRACK: Login failed, using sample data")
+                    logger.warning_event(
+                        "Space-Track login failed; using sample data",
+                        event="collector.spacetrack.login.failed",
+                        context={"status_code": login_response.status_code},
+                    )
                     return self._get_sample_data()
 
                 tle_response = await client.get(self.probe_url)
-                print(f"SPACETRACK: TLE query status: {tle_response.status_code}")
-
-                if tle_response.status_code != 200:
-                    print(f"SPACETRACK: Query failed, using sample data")
-                    return self._get_sample_data()
-
-                data = tle_response.json()
-                print(f"SPACETRACK: Received {len(data)} records")
-                return data
-        except Exception as e:
-            print(f"SPACETRACK: Error - {e}, using sample data")
-            return self._get_sample_data()
-
-        print(f"SPACETRACK: Attempting to fetch TLE data with username: {username}")
-
-        try:
-            async with httpx.AsyncClient(
-                timeout=120.0,
-                follow_redirects=True,
-                headers={
-                    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36",
-                    "Accept": "application/json, text/html, */*",
-                    "Accept-Language": "en-US,en;q=0.9",
-                },
-            ) as client:
-                # First, visit the main page to get any cookies
-                await client.get(f"{self.site_root}/")
-
-                # Login to get session cookie
-                login_response = await client.post(
-                    self.login_url,
-                    data={
-                        "identity": username,
-                        "password": password,
-                    },
+                logger.info_event(
+                    "Space-Track TLE query response received",
+                    event="collector.spacetrack.query.response",
+                    context={"status_code": tle_response.status_code},
                 )
-                print(f"SPACETRACK: Login response status: {login_response.status_code}")
-                print(f"SPACETRACK: Login response URL: {login_response.url}")
-                print(f"SPACETRACK: Login response body: {login_response.text[:500]}")
-
-                if login_response.status_code != 200:
-                    print(f"SPACETRACK: Login failed, using sample data")
-                    return self._get_sample_data()
-
-                # Query for TLE data (get first 1000 satellites)
-                tle_response = await client.get(self.query_url)
-                print(f"SPACETRACK: TLE query status: {tle_response.status_code}")
 
                 if tle_response.status_code != 200:
-                    print(f"SPACETRACK: Query failed, using sample data")
+                    logger.warning_event(
+                        "Space-Track TLE query failed; using sample data",
+                        event="collector.spacetrack.query.failed",
+                        context={"status_code": tle_response.status_code},
+                    )
                     return self._get_sample_data()
 
                 data = tle_response.json()
-                print(f"SPACETRACK: Received {len(data)} records")
+                logger.info_event(
+                    "Space-Track TLE fetch completed",
+                    event="collector.spacetrack.fetch.completed",
+                    context={"record_count": len(data)},
+                )
                 return data
         except Exception as e:
-            print(f"SPACETRACK: Error - {e}, using sample data")
+            logger.warning_event(
+                "Space-Track TLE fetch failed; using sample data",
+                event="collector.spacetrack.fetch.failed",
+                context={"error": str(e)},
+            )
             return self._get_sample_data()
 
     def transform(self, raw_data: List[Dict[str, Any]]) -> List[Dict[str, Any]]:

backend/app/services/collectors/top500.py

@@ -5,16 +5,18 @@ https://top500.org/lists/top500/
 """
 
 import asyncio
+import math
 import re
+from urllib.parse import urljoin
 from typing import Dict, Any, List
 from bs4 import BeautifulSoup
 import httpx
 
 from app.core.data_sources import get_data_sources_config
-from app.services.collectors.base import BaseCollector
+from app.services.collectors.compute_base import ComputeCollector
 
 
-class TOP500Collector(BaseCollector):
+class TOP500Collector(ComputeCollector):
     name = "top500"
     priority = "P0"
     module = "L1"
@@ -28,7 +30,7 @@ class TOP500Collector(BaseCollector):
         async with httpx.AsyncClient(timeout=60.0, follow_redirects=True) as client:
             response = await client.get(url)
             response.raise_for_status()
-            entries = self.parse_response(response.text)
+            entries = self.parse_response(response.text, str(response.url))
 
             semaphore = asyncio.Semaphore(8)
 
@@ -55,7 +57,7 @@ class TOP500Collector(BaseCollector):
         system_name = link.get_text(" ", strip=True) if link else system_cell.get_text(" ", strip=True)
         detail_url = ""
         if link and link.get("href"):
-            detail_url = f"{str(top500_base_url).rstrip('/')}{link.get('href')}"
+            detail_url = urljoin(str(top500_base_url), link.get("href"))
 
         manufacturer = ""
         if link and link.next_sibling:
@@ -127,10 +129,14 @@ class TOP500Collector(BaseCollector):
 
         return detail_map
 
-    def parse_response(self, html: str) -> List[Dict[str, Any]]:
+    def parse_response(self, html: str, source_url: str = "") -> List[Dict[str, Any]]:
         """Parse TOP500 HTML response"""
         data = []
         soup = BeautifulSoup(html, "html.parser")
+        canonical = soup.find("link", rel="canonical")
+        date_url = source_url or (canonical.get("href", "") if canonical else "") or self._resolved_url or ""
+        date_match = re.search(r"/lists/top500/(\d{4})/(06|11)/", date_url)
+        reference_date = f"{date_match[1]}-{date_match[2]}-01" if date_match else None
 
         # Find the table with TOP500 data
         table = None
@@ -152,6 +158,9 @@ class TOP500Collector(BaseCollector):
             table = soup.find("table", {"class": "top500-table"}) or soup.find("table", {"id": "top500"})
 
         if table:
+            headers = [cell.get_text(" ", strip=True) for cell in table.select("tr")[0].find_all(["th", "td"])]
+            if not reference_date:
+                raise ValueError("TOP500 list date missing; previous snapshot retained")
             rows = table.select("tr")
             for row in rows[1:]:  # Skip header row
                 cells = row.find_all(["td", "th"])
@@ -176,25 +185,32 @@ class TOP500Collector(BaseCollector):
                         cores = cells[2].get_text(strip=True).replace(",", "")
 
                         rmax_text = cells[3].get_text(strip=True)
-                        rmax = self._parse_performance(rmax_text)
+                        rmax = self._parse_performance(rmax_text, headers[3])
 
                         rpeak_text = cells[4].get_text(strip=True)
-                        rpeak = self._parse_performance(rpeak_text)
+                        rpeak = self._parse_performance(rpeak_text, headers[4])
 
                         power = ""
                         if len(cells) >= 6:
                             power = cells[5].get_text(strip=True).replace(",", "")
 
+                        identity = re.search(r"/system/(\d+)", detail_url)
+                        if not identity:
+                            raise ValueError("TOP500 stable system ID missing")
                         entry = {
-                            "source_id": f"top500_{rank}",
+                            "source_id": f"top500_system_{identity[1]}",
                             "name": system_name,
                             "country": country,
                             "city": city,
-                            "latitude": 0.0,
-                            "longitude": 0.0,
+                            "latitude": None,
+                            "longitude": None,
                             "value": str(rmax),
-                            "unit": "PFlop/s",
+                            "unit": "FLOP/s",
                             "metadata": {
+                                "compute_contract": "v1",
+                                "precision": "FP64", "measurement": "LINPACK Rmax",
+                                "source_url": detail_url, "list_url": date_url,
+                                "status": "listed", "rmax_flops": rmax, "rpeak_flops": rpeak,
                                 "rank": rank,
                                 "cores": cores,
                                 "rmax": rmax_text,
@@ -204,15 +220,11 @@ class TOP500Collector(BaseCollector):
                                 "site": site,
                             },
                             "_detail_url": detail_url,
-                            "reference_date": "2025-11-01",
+                            "reference_date": reference_date,
                         }
                         data.append(entry)
                     except (ValueError, IndexError, AttributeError) as e:
-                        continue
-
-        # If scraping failed, return sample data for testing
-        if not data:
-            data = self._get_sample_data()
+                        raise ValueError("Invalid TOP500 row; previous snapshot retained") from e
 
         return data
 
@@ -227,9 +239,14 @@ class TOP500Collector(BaseCollector):
                 return 0.0
         return 0.0
 
-    def _parse_performance(self, text: str) -> float:
+    def _parse_performance(self, text: str, header: str = "") -> float:
         """Parse performance value from text (handles E, P, T suffixes)"""
-        text = text.strip().upper()
+        text = text.strip().upper().replace(",", "")
+        if not re.search(r"[EPTGMK]F", text):
+            unit = re.search(r"\b([EPTGMK])FLOP/?S", header.upper())
+            if not unit:
+                raise ValueError("TOP500 performance unit missing")
+            text = f"{text} {unit[1]}FLOP/S"
         multipliers = {
             "E": 1e18,
             "P": 1e15,
@@ -249,73 +266,9 @@ class TOP500Collector(BaseCollector):
 
         # Try simple float parsing
         try:
-            return float(text.replace(",", ""))
+            value = float(text.replace(",", ""))
+            if not math.isfinite(value) or value < 0:
+                raise ValueError("Invalid TOP500 performance")
+            return value
         except ValueError:
-            return 0.0
-
-    def _get_sample_data(self) -> List[Dict[str, Any]]:
-        """Return sample data for testing when scraping fails"""
-        return [
-            {
-                "source_id": "top500_1",
-                "name": "El Capitan - HPE Cray EX255a, AMD 4th Gen EPYC 24C 1.8GHz, AMD Instinct MI300A",
-                "country": "United States",
-                "city": "Livermore, CA",
-                "latitude": 37.6819,
-                "longitude": -121.7681,
-                "value": "1742.00",
-                "unit": "PFlop/s",
-                "metadata": {
-                    "rank": 1,
-                    "cores": "11039616",
-                    "rmax": "1742.00",
-                    "rpeak": "2746.38",
-                    "power": "29581",
-                    "manufacturer": "HPE",
-                    "site": "DOE/NNSA/LLNL",
-                    "processor": "AMD 4th Gen EPYC 24C 1.8GHz",
-                    "interconnect": "Slingshot-11",
-                    "installation_year": "2025",
-                },
-                "reference_date": "2025-11-01",
-            },
-            {
-                "source_id": "top500_2",
-                "name": "Frontier - HPE Cray EX235a, AMD Optimized 3rd Generation EPYC 64C 2GHz, AMD Instinct MI250X",
-                "country": "United States",
-                "city": "Oak Ridge, TN",
-                "latitude": 36.0107,
-                "longitude": -84.2663,
-                "value": "1353.00",
-                "unit": "PFlop/s",
-                "metadata": {
-                    "rank": 2,
-                    "cores": "9066176",
-                    "rmax": "1353.00",
-                    "rpeak": "2055.72",
-                    "power": "24607",
-                    "manufacturer": "HPE",
-                    "site": "DOE/SC/Oak Ridge National Laboratory",
-                },
-                "reference_date": "2025-11-01",
-            },
-            {
-                "source_id": "top500_3",
-                "name": "Aurora - HPE Cray EX - Intel Exascale Compute Blade, Xeon CPU Max 9470 52C 2.4GHz, Intel Data Center GPU Max",
-                "country": "United States",
-                "city": "Argonne, IL",
-                "latitude": 41.3784,
-                "longitude": -87.8600,
-                "value": "1012.00",
-                "unit": "PFlop/s",
-                "metadata": {
-                    "rank": 3,
-                    "cores": "9264128",
-                    "rmax": "1012.00",
-                    "rpeak": "1980.01",
-                    "power": "38698",
-                    "manufacturer": "Intel",
-                },
-                "reference_date": "2025-11-01",
-            },
-        ]
+            raise ValueError("Invalid TOP500 performance") from None

backend/app/services/collectors/vessel_ais.py

@@ -6,6 +6,7 @@ from typing import Any
 import httpx
 from sqlalchemy.ext.asyncio import AsyncSession
 
+from app.core.enums import SnapshotStatus
 from app.core.time import to_iso8601_utc
 from app.core.websocket.broadcaster import broadcaster
 from app.services.barentswatch import (
@@ -119,6 +120,21 @@ class VesselAISCollector(BaseCollector):
             last_success_at=now if data else None,
             lag_seconds=max((now - latest_observed_at).total_seconds(), 0),
         )
+        if snapshot_id is not None:
+            from app.models.data_snapshot import DataSnapshot
+
+            snapshot = await db.get(DataSnapshot, snapshot_id)
+            if snapshot:
+                snapshot.record_count = records_added
+                snapshot.status = SnapshotStatus.SUCCESS.value
+                snapshot.completed_at = now
+                snapshot.summary = {
+                    "created": records_added,
+                    "updated": 0,
+                    "unchanged": 0,
+                    "deleted": 0,
+                    "storage": "ais_raw_observations",
+                }
         await db.commit()
         await self._broadcast_vessel_snapshot(data)
         await self.update_progress(records_added, force=True)

backend/app/services/compute_catalog.py

@@ -0,0 +1,200 @@
+"""Canonical country and mutually exclusive regional compute participants."""
+
+import re
+from collections.abc import Iterable
+
+from app.core.countries import COUNTRY_CENTROIDS, COUNTRY_ENTRIES
+from app.core.earth_geography_policy import display_country
+
+PALETTE = ("#91baff", "#f0c88a", "#89d2c2", "#c5abdf", "#eda9b3", "#b6c99a")
+DEFAULT_PARTICIPANTS = ("CN", "US", "EUROPE", "APAC", "OTHER")
+
+# Explicit membership prevents a regional total from double-counting a country.
+# APAC deliberately excludes China.
+COMPARISON_GROUPS = (
+    ("CN", "中国", "China", ("CN",), {"latitude": 35.8617, "longitude": 104.1954}),
+    ("US", "美国", "United States", ("US",), {"latitude": 39.8283, "longitude": -98.5795}),
+    (
+        "EUROPE",
+        "欧洲",
+        "Europe",
+        (
+            "GB",
+            "DE",
+            "FR",
+            "CH",
+            "ES",
+            "FI",
+            "HU",
+            "IE",
+            "IT",
+            "LU",
+            "MD",
+            "NL",
+            "PL",
+            "RO",
+            "RU",
+            "UA",
+        ),
+        {"latitude": 54.0, "longitude": 20.0},
+    ),
+    (
+        "APAC",
+        "亚太（不含中国）",
+        "Asia Pacific (excluding China)",
+        ("JP", "KR", "IN", "AU", "AF", "ID", "NP", "PH", "PK", "SG", "TH"),
+        {"latitude": 36.0, "longitude": 130.5},
+    ),
+    (
+        "OTHER",
+        "其他地区",
+        "Other",
+        (
+            "CA",
+            "AO",
+            "EG",
+            "KE",
+            "MZ",
+            "NG",
+            "UG",
+            "ZA",
+            "BR",
+            "CO",
+            "CR",
+            "DO",
+            "MX",
+            "AE",
+            "SY",
+            "TR",
+        ),
+        {"latitude": 12.0, "longitude": 10.0},
+    ),
+)
+
+
+def _country_code(canonical: str, aliases: list[str]) -> str | None:
+    code = next((alias for alias in aliases if re.fullmatch(r"[A-Z]{2}", alias)), None)
+    return "GB" if canonical == "英国" else code
+
+
+def build_country_catalog() -> list[dict]:
+    """Build the policy-projected canonical country drilldown catalog."""
+    countries = []
+    for canonical, aliases in COUNTRY_ENTRIES:
+        if display_country(canonical) != canonical or canonical not in COUNTRY_CENTROIDS:
+            continue
+        code = _country_code(canonical, aliases)
+        if code:
+            countries.append(
+                {
+                    "code": code,
+                    "name": canonical,
+                    "name_en": aliases[0],
+                    "kind": "country",
+                    "members": [code],
+                    "anchor": dict(COUNTRY_CENTROIDS[canonical]),
+                    "anchor_kind": "regional_label",
+                }
+            )
+    return countries
+
+
+COUNTRY_CATALOG = build_country_catalog()
+COUNTRIES = {entry["code"]: entry["name"] for entry in COUNTRY_CATALOG}
+COUNTRY_PARTICIPANTS = {entry["code"]: entry for entry in COUNTRY_CATALOG}
+
+
+def _validate_group_members() -> None:
+    members = [
+        member for _code, _name, _name_en, codes, _anchor in COMPARISON_GROUPS for member in codes
+    ]
+    if len(members) != len(set(members)) or set(members) != set(COUNTRIES):
+        raise ValueError("compute comparison groups must cover country catalog exactly once")
+
+
+_validate_group_members()
+GROUP_BY_MEMBER = {
+    member: code
+    for code, _name, _name_en, members, _anchor in COMPARISON_GROUPS
+    for member in members
+}
+
+
+def build_catalog() -> list[dict]:
+    """Return the five default participants before country drilldown entries."""
+    catalog: list[dict] = []
+    for index, (code, name, name_en, members, anchor) in enumerate(COMPARISON_GROUPS):
+        entry = (
+            dict(COUNTRY_PARTICIPANTS[code])
+            if code in COUNTRY_PARTICIPANTS
+            else {
+                "code": code,
+                "name": name,
+                "name_en": name_en,
+                "kind": "region",
+                "members": list(members),
+                "anchor": dict(anchor),
+                "anchor_kind": "regional_label",
+            }
+        )
+        entry.update(
+            {
+                "members": list(members),
+                "parent_group": name,
+                "parent_group_code": code,
+                "order": index,
+                "default_selected": True,
+                "color": PALETTE[index % len(PALETTE)],
+            }
+        )
+        catalog.append(entry)
+
+    for entry in COUNTRY_CATALOG:
+        if entry["code"] in DEFAULT_PARTICIPANTS:
+            continue
+        country = dict(entry)
+        group_code = GROUP_BY_MEMBER[country["code"]]
+        group_name = next(group[1] for group in COMPARISON_GROUPS if group[0] == group_code)
+        country.update(
+            {
+                "parent_group": group_name,
+                "parent_group_code": group_code,
+                "order": len(catalog),
+                "default_selected": False,
+                "color": PALETTE[len(catalog) % len(PALETTE)],
+            }
+        )
+        catalog.append(country)
+    return catalog
+
+
+CATALOG = build_catalog()
+PARTICIPANTS = {entry["code"]: entry for entry in CATALOG}
+
+
+def participant_members(codes: Iterable[str]) -> dict[str, tuple[str, ...]]:
+    """Resolve country or region participant codes to canonical country codes."""
+    return {code: tuple(PARTICIPANTS[code]["members"]) for code in codes}
+
+
+def source_country_names(codes: Iterable[str]) -> list[str]:
+    """Return unique canonical source labels, respecting China and Serbia ownership."""
+    selected_members = {
+        member for members in participant_members(codes).values() for member in members
+    }
+    selected_names = {COUNTRIES[code] for code in selected_members}
+    return [name for name, _aliases in COUNTRY_ENTRIES if display_country(name) in selected_names]
+
+
+def source_country_participants(codes: Iterable[str]) -> dict[str, str]:
+    """Map stored source-country labels to their requested participant code."""
+    owner_by_country = {
+        member: participant
+        for participant, members in participant_members(codes).items()
+        for member in members
+    }
+    code_by_name = {name: code for code, name in COUNTRIES.items()}
+    return {
+        name: owner_by_country[code_by_name[display_country(name)]]
+        for name in source_country_names(codes)
+    }

backend/app/services/compute_center_locations.py

@@ -3,7 +3,12 @@
 This module is a thin domain wrapper that wires up
 :mod:`app.services.location` for compute centers:
 
-    SourceCoordinates
+    Confirmed stored location -> trustworthy source coordinates -> exact
+    official facility registration -> source city/site coordinates.
+
+Epoch AI coordinates are admitted only at the grain asserted by its official
+``Location`` column. Country/region/empty locations remain unresolved even
+when the CSV happens to contain a latitude/longitude pair.
 
 The online Nominatim step is intentionally reserved for the user-triggered
 ``collect-location`` flow. The regular GeoJSON endpoint runs during Earth
@@ -28,9 +33,12 @@ import httpx
 from sqlalchemy import select
 from sqlalchemy.ext.asyncio import AsyncSession
 
+from app.core.compute_types import compute_record_filter
 from app.core.collected_data_fields import get_record_field
 from app.models.collected_data import CollectedData
 from app.models.compute_center_location import ComputeCenterLocationRecord
+from app.services.compute_center_verified_sites import get_verified_compute_site
+from app.services.epoch_location import assess_epoch_location
 
 from app.services.location import (
     LocationCandidate,
@@ -166,8 +174,13 @@ def set_compute_center_location_cache(
 
 async def refresh_compute_center_location_cache(
     session: AsyncSession,
+    *,
+    as_of: datetime | None = None,
 ) -> dict[str, dict[str, Any]]:
-    result = await session.execute(select(ComputeCenterLocationRecord))
+    query = select(ComputeCenterLocationRecord)
+    if as_of is not None:
+        query = query.where(ComputeCenterLocationRecord.verified_at < as_of)
+    result = await session.execute(query)
     records = result.scalars().all()
     cache = {}
     for record in records:
@@ -350,6 +363,58 @@ class ROROrganizationResolver:
         )
 
 
+class ComputeCenterSourceCoordinatesResolver:
+    """Use source coordinates only at the grain asserted by the source text."""
+
+    name = "source_coordinates"
+
+    def __init__(self) -> None:
+        self._default = SourceCoordinatesResolver()
+
+    def resolve(self, query: LocationQuery) -> ResolverOutput:
+        extra = query.extra or {}
+        if extra.get("source") != "epoch_ai_gpu":
+            return self._default.resolve(query)
+        if query.source_latitude in (None, 0.0) or query.source_longitude in (None, 0.0):
+            return ResolverOutput()
+        assessment = assess_epoch_location(
+            extra.get("source_location"),
+            country=query.country,
+            raw_country=extra.get("source_raw_country"),
+        )
+        if not assessment.is_renderable:
+            return ResolverOutput(
+                attempted_queries=(f"source_coordinates:{assessment.precision}",)
+            )
+        output = self._default.resolve(query)
+        if not output.candidates:
+            return output
+        candidate = output.candidates[0]
+        source_location = extra.get("source_location")
+        source_location_text = (
+            source_location.strip() if isinstance(source_location, str) else ""
+        )
+        source_note = assessment.reason
+        if source_location_text:
+            source_note = f"{source_note}; Epoch Location: {source_location_text}"
+        return ResolverOutput(
+            candidates=(
+                LocationCandidate(
+                    **{
+                        **candidate.__dict__,
+                        "precision": assessment.precision,
+                        "confidence": 1.0 if assessment.precision == "precise" else 0.9,
+                        "source_note": source_note,
+                        "needs_confirmation": assessment.precision != "precise",
+                        "matched_location_name": source_location_text
+                        or candidate.matched_location_name,
+                    }
+                ),
+            ),
+            attempted_queries=output.attempted_queries,
+        )
+
+
 class StoredComputeCenterLocationResolver:
     """Resolve a compute center through the DB-backed current-location cache."""
 
@@ -367,13 +432,26 @@ class StoredComputeCenterLocationResolver:
         longitude = parse_float(stored.get("longitude"))
         if latitude in (None, 0.0) or longitude in (None, 0.0):
             return ResolverOutput()
+        precision = stored.get("precision") or "city"
+        if (
+            extra.get("source") == "epoch_ai_gpu"
+            and stored.get("location_source") == "source_coordinates"
+        ):
+            assessment = assess_epoch_location(
+                extra.get("source_location"),
+                country=query.country,
+                raw_country=extra.get("source_raw_country"),
+            )
+            if not assessment.is_renderable:
+                return ResolverOutput()
+            precision = assessment.precision
         return ResolverOutput(
             candidates=(
                 LocationCandidate(
                     latitude=latitude,
                     longitude=longitude,
                     display_name=stored.get("name") or query.name or "Compute center",
-                    precision=stored.get("precision") or "city",
+                    precision=precision,
                     confidence=float(stored.get("confidence") or 0.85),
                     query=f"stored_compute_center_location::{stored.get('source')}:{stored.get('source_id')}",
                     source=self.name,
@@ -386,6 +464,60 @@ class StoredComputeCenterLocationResolver:
                     matched_location_name=stored.get("site") or stored.get("name") or query.name,
                     location_verified_at=stored.get("verified_at"),
                     suggested_registry_entry=None,
+                    raw_payload={
+                        "stored_location_source": stored.get("location_source"),
+                        "verification_status": stored.get("verification_status"),
+                    },
+                ),
+            )
+        )
+
+
+class VerifiedComputeSiteResolver:
+    """Resolve an exact source identity to an evidence-backed facility site."""
+
+    name = "official_facility_registry"
+
+    def resolve(self, query: LocationQuery) -> ResolverOutput:
+        extra = query.extra or {}
+        source = coerce_str(extra.get("source"))
+        source_id = coerce_str(extra.get("source_id"))
+        if not source or not source_id:
+            return ResolverOutput()
+        site = get_verified_compute_site(source, source_id)
+        if not site:
+            return ResolverOutput()
+        source_urls = [coerce_str(url) for url in site.get("sources", []) if coerce_str(url)]
+        source_note = coerce_str(site.get("source_note"))
+        if source_urls:
+            source_note = f"{source_note} Sources: {'; '.join(source_urls)}".strip()
+        return ResolverOutput(
+            candidates=(
+                LocationCandidate(
+                    latitude=float(site["latitude"]),
+                    longitude=float(site["longitude"]),
+                    display_name=coerce_str(site.get("matched_location_name"))
+                    or query.name
+                    or "Compute center",
+                    precision=coerce_str(site.get("precision")) or "site",
+                    confidence=1.0,
+                    query=f"official_facility_registry::{source}:{source_id}",
+                    source=self.name,
+                    source_note=source_note or None,
+                    matched_fields=("source", "source_id"),
+                    needs_confirmation=False,
+                    city=coerce_str(site.get("city")) or None,
+                    region=None,
+                    country=coerce_str(site.get("country")) or query.country,
+                    matched_location_name=coerce_str(site.get("matched_location_name"))
+                    or None,
+                    location_verified_at=coerce_str(site.get("verified_at")) or None,
+                    suggested_registry_entry=None,
+                    raw_payload={
+                        "address": site.get("address"),
+                        "coordinate_basis": site.get("coordinate_basis"),
+                        "sources": source_urls,
+                    },
                 ),
             )
         )
@@ -400,7 +532,7 @@ def _short_system_name(name: Any) -> str:
     return head or text
 
 
-def _record_context(record: Any, metadata: dict[str, Any]) -> dict[str, str]:
+def _record_context(record: Any, metadata: dict[str, Any]) -> dict[str, Any]:
     name = coerce_str(getattr(record, "name", None))
     return {
         "source": coerce_str(getattr(record, "source", None)),
@@ -417,11 +549,13 @@ def _record_context(record: Any, metadata: dict[str, Any]) -> dict[str, str]:
             or metadata.get("manufacturer")
         ),
         "organization": coerce_str(metadata.get("organization")),
+        "source_location": metadata.get("location"),
+        "source_raw_country": metadata.get("raw_country"),
     }
 
 
 def _context_to_query(
-    context: dict[str, str],
+    context: dict[str, Any],
     *,
     source_lat: float | None = None,
     source_lon: float | None = None,
@@ -444,6 +578,8 @@ def _context_to_query(
             "site": context.get("site") or "",
             "operator": context.get("operator") or "",
             "organization": context.get("organization") or "",
+            "source_location": context.get("source_location"),
+            "source_raw_country": context.get("source_raw_country"),
         },
     )
 
@@ -497,11 +633,36 @@ def _compute_center_query_plan(
     return plan
 
 
+def _compute_center_candidate_sort_key(candidate: LocationCandidate) -> tuple[int, int, float]:
+    stored_origin = (candidate.raw_payload or {}).get("stored_location_source")
+    stored_confirmed = (
+        candidate.source == "stored_compute_center_location"
+        and not candidate.needs_confirmation
+        and stored_origin != "source_coordinates"
+    )
+    if stored_confirmed:
+        source_rank = 0
+    elif candidate.source == "source_coordinates" and candidate.precision == "precise":
+        source_rank = 1
+    elif candidate.source == "official_facility_registry":
+        source_rank = 2
+    elif candidate.source == "source_coordinates":
+        source_rank = 3
+    elif candidate.source == "stored_compute_center_location":
+        source_rank = 4
+    else:
+        source_rank = 5
+    precision_rank = {"precise": 0, "site": 1, "city": 2}.get(candidate.precision, 9)
+    return (source_rank, precision_rank, -float(candidate.confidence or 0))
+
+
 COMPUTE_CENTER_PIPELINE = LocationPipeline(
     [
-        SourceCoordinatesResolver(),
+        ComputeCenterSourceCoordinatesResolver(),
         StoredComputeCenterLocationResolver(),
+        VerifiedComputeSiteResolver(),
     ],
+    sort_key=_compute_center_candidate_sort_key,
     failure_reason=(
         "Could not resolve to city-level coordinates from source coords"
         " or stored compute-center location."
@@ -510,7 +671,8 @@ COMPUTE_CENTER_PIPELINE = LocationPipeline(
 
 COMPUTE_CENTER_COLLECTION_PIPELINE = LocationPipeline(
     [
-        SourceCoordinatesResolver(),
+        ComputeCenterSourceCoordinatesResolver(),
+        VerifiedComputeSiteResolver(),
         ROROrganizationResolver(),
         NominatimResolver(
             query_plan_builder=_compute_center_query_plan,
@@ -518,6 +680,7 @@ COMPUTE_CENTER_COLLECTION_PIPELINE = LocationPipeline(
             geocoder=lambda q: _geocode_online(q),
         ),
     ],
+    sort_key=_compute_center_candidate_sort_key,
     failure_reason=(
         "Could not resolve to city-level coordinates from source coords"
         ", ROR organization lookup, or online geocoding."
@@ -531,6 +694,7 @@ COMPUTE_CENTER_COLLECTION_PIPELINE = LocationPipeline(
 _GEOGRAPHY_MODE_BY_SOURCE = {
     "source_coordinates": "source_coordinates",
     "stored_compute_center_location": "stored_compute_center_location",
+    "official_facility_registry": "official_facility_registry",
     "ror_organization_registry": "ror_organization",
     "nominatim_online_geocode": "online_geocode",
 }
@@ -539,7 +703,7 @@ _GEOGRAPHY_MODE_BY_SOURCE = {
 def _candidate_to_location(
     candidate: LocationCandidate,
     *,
-    context: dict[str, str],
+    context: dict[str, Any],
 ) -> ComputeCenterLocation:
     geography_mode = _GEOGRAPHY_MODE_BY_SOURCE.get(candidate.source, "online_geocode")
     is_estimated = candidate.needs_confirmation or candidate.source.startswith(
@@ -594,7 +758,7 @@ def _candidate_to_location(
 
 def _diagnostic_for(
     record: Any,
-    context: dict[str, str],
+    context: dict[str, Any],
     *,
     failure_reason: str,
     attempted_queries: tuple[str, ...] = (),
@@ -738,7 +902,7 @@ def build_compute_center_location_query(
     organization: str | None = None,
 ) -> LocationQuery:
     name_value = coerce_str(name)
-    context: dict[str, str] = {
+    context: dict[str, Any] = {
         "source": coerce_str(source),
         "source_id": coerce_str(source_id),
         "name": name_value,
@@ -767,7 +931,7 @@ async def seed_compute_center_locations_from_source_coords(
     """Seed stored compute-center locations only from real source coordinates."""
     stmt = (
         select(CollectedData)
-        .where(CollectedData.source.in_(["top500", "epoch_ai_gpu"]))
+        .where(compute_record_filter(CollectedData))
         .where(CollectedData.is_current.is_(True))
     )
     result = await session.execute(stmt)
@@ -783,6 +947,23 @@ async def seed_compute_center_locations_from_source_coords(
         longitude = parse_float(get_record_field(record, "longitude"))
         if latitude in (None, 0.0) or longitude in (None, 0.0):
             continue
+        metadata = record.extra_data or {}
+        precision = "precise"
+        confidence = 1.0
+        needs_confirmation = False
+        source_note = "Seeded from source-provided compute-center coordinates"
+        if source_value == "epoch_ai_gpu":
+            assessment = assess_epoch_location(
+                metadata.get("location"),
+                country=get_record_field(record, "country"),
+                raw_country=metadata.get("raw_country"),
+            )
+            if not assessment.is_renderable:
+                continue
+            precision = assessment.precision
+            confidence = 1.0 if precision == "precise" else 0.9
+            needs_confirmation = precision != "precise"
+            source_note = assessment.reason
         existing = await session.scalar(
             select(ComputeCenterLocationRecord)
             .where(ComputeCenterLocationRecord.source == source_value)
@@ -790,7 +971,6 @@ async def seed_compute_center_locations_from_source_coords(
         )
         if existing:
             continue
-        metadata = record.extra_data or {}
         session.add(
             ComputeCenterLocationRecord(
                 source=source_value,
@@ -802,16 +982,16 @@ async def seed_compute_center_locations_from_source_coords(
                 country=coerce_str(get_record_field(record, "country")) or None,
                 latitude=latitude,
                 longitude=longitude,
-                precision="precise",
-                confidence=1.0,
+                precision=precision,
+                confidence=confidence,
                 location_source="source_coordinates",
-                source_note="Seeded from source-provided compute-center coordinates",
+                source_note=source_note,
                 raw_payload={
                     "record_id": getattr(record, "id", None),
                     "source": source_value,
                     "source_id": source_id,
                 },
-                needs_confirmation=False,
+                needs_confirmation=needs_confirmation,
                 verification_status="source_provided",
                 verified_at=None,
             )

backend/app/services/compute_center_verified_sites.json

@@ -0,0 +1,44 @@
+{
+  "version": 1,
+  "sites": [
+    {
+      "source": "epoch_ai_gpu",
+      "source_ids": ["epoch_jamstec_zettascaler_2_0_gyoukou", "epoch_jamstec_zettascaler_2_2_gyoukou"],
+      "latitude": 35.380924,
+      "longitude": 139.625008,
+      "precision": "site",
+      "location_source": "official_facility_registry",
+      "country": "日本",
+      "city": "Yokohama",
+      "matched_location_name": "JAMSTEC Yokohama Institute for Earth Sciences",
+      "address": "3173-25 Showa-machi, Kanazawa-ku, Yokohama, Kanagawa, Japan",
+      "source_note": "Gyoukou deployment confirmed by JAMSTEC's 2017 release. Campus-level anchor from its official access map center, not a surveyed machine-room or building coordinate.",
+      "verified_at": "2026-09-21T17:06:31Z",
+      "coordinate_basis": "official_access_map_center",
+      "sources": [
+        "https://www.jamstec.go.jp/j/about/press_release/archive/2017/20171114.pdf",
+        "https://www.jamstec.go.jp/e/about/access/yokohama.html"
+      ]
+    },
+    {
+      "source": "epoch_ai_gpu",
+      "source_ids": ["epoch_argonne_nl_sophia__formerly_theta_"],
+      "latitude": 41.718109383287235,
+      "longitude": -87.984498648496,
+      "precision": "site",
+      "location_source": "official_facility_registry",
+      "country": "美国",
+      "city": "Argonne",
+      "matched_location_name": "Argonne Leadership Computing Facility",
+      "address": "9700 South Cass Avenue, Building 240, Argonne, IL 60439, USA",
+      "source_note": "Sophia is an ALCF system. Campus-level anchor from ALCF's official visitor map center; the published address identifies Building 240 but the coordinate is not a surveyed building or rack location.",
+      "verified_at": "2026-09-21T17:06:31Z",
+      "coordinate_basis": "official_access_map_center",
+      "sources": [
+        "https://www.alcf.anl.gov/systems/sophia",
+        "https://docs.alcf.anl.gov/sophia/",
+        "https://www.alcf.anl.gov/about/visiting-alcf"
+      ]
+    }
+  ]
+}

backend/app/services/compute_center_verified_sites.py

@@ -0,0 +1,26 @@
+"""Evidence-backed facility anchors; never match an operator's headquarters by name."""
+
+import json
+from copy import deepcopy
+from functools import lru_cache
+from pathlib import Path
+
+
+@lru_cache(maxsize=1)
+def _site_index() -> dict[tuple[str, str], dict]:
+    payload = json.loads(Path(__file__).with_suffix(".json").read_text(encoding="utf-8"))
+    index = {}
+    for site in payload["sites"]:
+        if not (-90 <= site["latitude"] <= 90 and -180 <= site["longitude"] <= 180):
+            raise ValueError("Invalid verified compute facility coordinate")
+        for source_id in site["source_ids"]:
+            key = (site["source"], source_id)
+            if key in index:
+                raise ValueError("Duplicate verified compute facility identity")
+            index[key] = site
+    return index
+
+
+def get_verified_compute_site(source: str, source_id: str) -> dict | None:
+    site = _site_index().get((source, source_id))
+    return deepcopy(site) if site else None

backend/app/services/compute_configuration.py

@@ -0,0 +1,99 @@
+"""Database overrides with bundled defaults and atomic, revision-checked publishing."""
+
+import hashlib
+from pathlib import Path
+from uuid import uuid4
+
+from app.core.request_context import get_request_id
+from app.models.system_log import AuditLog
+from app.models.system_setting import SystemSetting
+from app.models.user import User
+from app.schemas.compute_configuration import ComputeConfiguration
+from app.services.compute_catalog import COUNTRY_CATALOG
+from fastapi import HTTPException
+from sqlalchemy import select
+from sqlalchemy.dialects.postgresql import insert
+from sqlalchemy.ext.asyncio import AsyncSession
+
+CATEGORY = "compute_comparison"
+DEFAULT_PATH = Path(__file__).with_name("compute_history_defaults.json")
+DEFAULT_CONFIGURATION = ComputeConfiguration.model_validate_json(DEFAULT_PATH.read_text())
+DEFAULT_REVISION = hashlib.sha256(DEFAULT_PATH.read_bytes()).hexdigest()
+
+
+def default_envelope() -> dict:
+    return {
+        "config": DEFAULT_CONFIGURATION.model_dump(mode="json"),
+        "revision": DEFAULT_REVISION,
+        "is_default": True,
+    }
+
+
+async def read_configuration(db: AsyncSession) -> dict:
+    record = (
+        await db.execute(select(SystemSetting).where(SystemSetting.category == CATEGORY))
+    ).scalar_one_or_none()
+    return dict(record.payload) if record else default_envelope()
+
+
+def configuration_response(envelope: dict) -> dict:
+    return {
+        **envelope,
+        "countries": [
+            {key: country[key] for key in ("code", "name", "name_en")}
+            for country in COUNTRY_CATALOG
+        ],
+    }
+
+
+async def publish_configuration(
+    db: AsyncSession,
+    *,
+    revision: str,
+    config: ComputeConfiguration,
+    actor: User,
+    reset: bool = False,
+) -> dict:
+    # The unique category plus row lock also serializes concurrent first writes.
+    await db.execute(
+        insert(SystemSetting)
+        .values(category=CATEGORY, payload=default_envelope())
+        .on_conflict_do_nothing(index_elements=[SystemSetting.category])
+    )
+    record = (
+        await db.execute(
+            select(SystemSetting).where(SystemSetting.category == CATEGORY).with_for_update()
+        )
+    ).scalar_one()
+    if record.payload["revision"] != revision:
+        await db.rollback()
+        raise HTTPException(409, "Configuration changed. Reload before saving your edits.")
+    envelope = (
+        default_envelope()
+        if reset
+        else {
+            "config": config.model_dump(mode="json"),
+            "revision": uuid4().hex,
+            "is_default": reset,
+        }
+    )
+    before = record.payload
+    if reset:
+        await db.delete(record)
+    else:
+        record.payload = envelope
+    # Audit and configuration commit together: a published edit must be attributable.
+    db.add(
+        AuditLog(
+            actor_id=actor.id,
+            actor_name=actor.username,
+            action="compute_comparison.reset" if reset else "compute_comparison.update",
+            target_type="system_setting",
+            target_id=CATEGORY,
+            result="success",
+            request_id=get_request_id(),
+            details={"before": before, "after": envelope},
+        )
+    )
+    await db.commit()
+    return envelope

backend/app/services/compute_history.py

@@ -0,0 +1,122 @@
+"""Versioned, source-backed annual observations for compute comparisons."""
+
+from copy import deepcopy
+from datetime import date
+
+from app.schemas.compute_configuration import ComputeConfiguration
+from app.services.compute_catalog import (
+    CATALOG,
+    DEFAULT_PARTICIPANTS,
+    PARTICIPANTS,
+    participant_members,
+)
+from app.services.compute_configuration import DEFAULT_CONFIGURATION
+
+
+def _participant_value(
+    members: tuple[str, ...], observations: dict[str, float | int], mode: str
+) -> tuple[float | int | None, dict]:
+    covered = [member for member in members if member in observations]
+    missing = [member for member in members if member not in observations]
+    coverage = {
+        "status": "complete" if not missing else "incomplete",
+        "required_members": list(members),
+        "covered_members": covered,
+        "missing_members": missing,
+    }
+    if missing:
+        return None, coverage
+    if mode == "bilateral_share":
+        if (
+            not set(members).issubset({"CN", "US"})
+            or not {"CN", "US"}.issubset(observations)
+            or observations["US"] + observations["CN"] <= 0
+        ):
+            coverage.update(
+                {
+                    "status": "incompatible",
+                    "reason": "bilateral_share_requires_CN_and_US_observations",
+                }
+            )
+            return None, coverage
+        return (
+            round(
+                sum(observations[member] for member in members)
+                / (observations["US"] + observations["CN"])
+                * 100,
+                2,
+            ),
+            coverage,
+        )
+    return sum(observations[member] for member in members), coverage
+
+
+def compute_history(
+    countries: list[str],
+    mode: str = "global_share",
+    config: ComputeConfiguration | None = None,
+) -> dict:
+    """Aggregate only fully covered, same-year compatible observations."""
+    config = config if config is not None else DEFAULT_CONFIGURATION
+    metric = "global_share" if mode == "bilateral_share" else mode
+    annual = {row.year: row for row in config.observations if row.metric == metric}
+    gaps = {row.year: row for row in config.innovation}
+    observed_years = [row.year for row in config.observations] + list(gaps)
+    years = tuple(
+        range(
+            min(observed_years, default=date.today().year),
+            max([date.today().year, *observed_years]) + 1,
+        )
+    )
+    members_by_participant = participant_members(countries)
+    rows = []
+    for year in years:
+        record = annual.get(year)
+        observations = record.values if record else {}
+        values, coverage = {}, {}
+        for code, members in members_by_participant.items():
+            values[code], coverage[code] = _participant_value(members, observations, mode)
+        evidence_id = record.source_id if record else None
+        rows.append(
+            {"year": year, "values": values, "coverage": coverage, "evidence_id": evidence_id}
+        )
+
+    selected_members = {member for members in members_by_participant.values() for member in members}
+    has_innovation_scope = {"CN", "US"}.issubset(selected_members)
+    return {
+        "version": f"compute-history-{config.reviewed_at}",
+        "provenance": "curated_reports",
+        "reviewed_at": config.reviewed_at.isoformat(),
+        "time_basis": "retrospective",
+        "years": years,
+        "mode": mode,
+        "unit": "EFlop/s (FP32)" if mode == "absolute" else "%",
+        "denominator": {"global_share": "global", "bilateral_share": "US+CN", "absolute": None}[
+            mode
+        ],
+        "catalog": deepcopy(CATALOG),
+        "default_participants": list(DEFAULT_PARTICIPANTS),
+        "countries": [deepcopy(PARTICIPANTS[code]) for code in countries],
+        "compute": rows,
+        "innovation": [
+            {
+                "year": year,
+                "value": gaps[year].value if year in gaps and has_innovation_scope else None,
+                "evidence_id": gaps[year].source_id
+                if year in gaps and has_innovation_scope
+                else None,
+            }
+            for year in years
+        ],
+        "evidence": {
+            source.id: source.model_dump(mode="json", exclude={"id"}) for source in config.sources
+        },
+        "innovation_scope": ["US", "CN"],
+        "limitations": [
+            "公开设施样本不代表全国总算力；FP64、FP16 与 FP32 不可直接相加。",
+            "区域值只在全部成员有同年、同指标、可比较观测时汇总；缺失成员时返回 null。",
+            "全球份额不从中美或其他局部成员反推剩余地区。",
+            "年度资料按已配置报告引用展示，观测年与报告年份分列；跨年方法可比性仍需核验。",
+            "创新分差为美国减中国；只有分差，不反推单国得分，不证明算力与创新的因果关系。",
+        ],
+    }

backend/app/services/compute_history_defaults.json

@@ -0,0 +1,125 @@
+{
+  "reviewed_at": "2026-09-22",
+  "sources": [
+    {
+      "id": "caict-2021",
+      "title": "中国算力发展指数白皮书（2021）",
+      "url": "https://www.caict.ac.cn/kxyj/qwfb/bps/202109/P020210918521091309950.pdf",
+      "publisher": "中国信息通信研究院",
+      "report_year": 2021,
+      "published_at": null,
+      "locator": "正文 11 页图 4（2020 年全球份额）"
+    },
+    {
+      "id": "caict-2022",
+      "title": "中国算力发展指数白皮书（2022）",
+      "url": "https://www.caict.ac.cn/kxyj/qwfb/bps/202211/P020221105727522653499.pdf",
+      "publisher": "中国信息通信研究院",
+      "report_year": 2022,
+      "published_at": null,
+      "locator": "正文 13 页图 5（2021 年全球份额）"
+    },
+    {
+      "id": "caict-2023",
+      "title": "中国算力发展指数白皮书（2023）",
+      "url": "https://www.caict.ac.cn/kxyj/qwfb/bps/202309/P020240326630458153765.pdf",
+      "publisher": "中国信息通信研究院",
+      "report_year": 2023,
+      "published_at": null,
+      "locator": "正文 13 页图 5（2022 年全球份额）；16 页（中国总规模）"
+    },
+    {
+      "id": "caict-2024",
+      "title": "先进计算暨算力发展指数蓝皮书（2024年）",
+      "url": "https://www.caict.ac.cn/kxyj/qwfb/bps/202501/P020250117503621662777.pdf",
+      "publisher": "中国信息通信研究院",
+      "report_year": 2024,
+      "published_at": null,
+      "locator": "正文 10 页图 3（2023 年全球份额）；12 页（中国总规模）"
+    },
+    {
+      "id": "innovation-2026",
+      "title": "全球人工智能创新指数报告 2026：科技日报报道",
+      "url": "https://finance.sina.com.cn/tech/roll/2026-07-17/doc-iniickaz8224252.shtml",
+      "publisher": "科技日报（新浪转载）",
+      "report_year": 2026,
+      "published_at": "2026-07-17",
+      "locator": "2023–2025 年美国减中国的指数分差"
+    }
+  ],
+  "observations": [
+    {
+      "year": 2020,
+      "metric": "global_share",
+      "source_id": "caict-2021",
+      "values": {
+        "US": 36,
+        "CN": 31,
+        "JP": 6
+      }
+    },
+    {
+      "year": 2021,
+      "metric": "global_share",
+      "source_id": "caict-2022",
+      "values": {
+        "US": 34,
+        "CN": 33,
+        "JP": 5
+      }
+    },
+    {
+      "year": 2022,
+      "metric": "global_share",
+      "source_id": "caict-2023",
+      "values": {
+        "US": 34,
+        "CN": 33,
+        "JP": 4
+      }
+    },
+    {
+      "year": 2023,
+      "metric": "global_share",
+      "source_id": "caict-2024",
+      "values": {
+        "US": 41,
+        "CN": 31,
+        "JP": 4
+      }
+    },
+    {
+      "year": 2022,
+      "metric": "absolute",
+      "source_id": "caict-2023",
+      "values": {
+        "CN": 302
+      }
+    },
+    {
+      "year": 2023,
+      "metric": "absolute",
+      "source_id": "caict-2024",
+      "values": {
+        "CN": 435
+      }
+    }
+  ],
+  "innovation": [
+    {
+      "year": 2023,
+      "value": 22.02,
+      "source_id": "innovation-2026"
+    },
+    {
+      "year": 2024,
+      "value": 19.96,
+      "source_id": "innovation-2026"
+    },
+    {
+      "year": 2025,
+      "value": 17.95,
+      "source_id": "innovation-2026"
+    }
+  ]
+}

backend/app/services/data_jobs.py

@@ -12,6 +12,7 @@ from sqlalchemy.ext.asyncio import AsyncSession
 
 from app.core.cache import cache
 from app.core.config import settings
+from app.core.enums import JobStatus, JobType, RollbackPolicy
 from app.core.logging import get_logger
 from app.core.time import to_iso8601_utc
 from app.core.websocket.broadcaster import broadcaster
@@ -36,17 +37,17 @@ from app.services.scheduler import sync_datasource_job
 
 logger = get_logger(__name__)
 
-JOB_TYPE_COLLECT = "collect"
-JOB_TYPE_CLEAR_DATA = "clear_data"
-JOB_TYPE_CLEAR_CACHE = "clear_cache"
-JOB_TYPE_EARTH_REFRESH = "earth_refresh"
+JOB_TYPE_COLLECT = JobType.COLLECT.value
+JOB_TYPE_CLEAR_DATA = JobType.CLEAR_DATA.value
+JOB_TYPE_CLEAR_CACHE = JobType.CLEAR_CACHE.value
+JOB_TYPE_EARTH_REFRESH = JobType.EARTH_REFRESH.value
 
-JOB_STATUS_QUEUED = "queued"
-JOB_STATUS_RUNNING = "running"
-JOB_STATUS_CANCELLING = "cancelling"
-JOB_STATUS_SUCCESS = "success"
-JOB_STATUS_FAILED = "failed"
-JOB_STATUS_CANCELLED = "cancelled"
+JOB_STATUS_QUEUED = JobStatus.QUEUED.value
+JOB_STATUS_RUNNING = JobStatus.RUNNING.value
+JOB_STATUS_CANCELLING = JobStatus.CANCELLING.value
+JOB_STATUS_SUCCESS = JobStatus.SUCCESS.value
+JOB_STATUS_FAILED = JobStatus.FAILED.value
+JOB_STATUS_CANCELLED = JobStatus.CANCELLED.value
 
 ACTIVE_JOB_STATUSES = (JOB_STATUS_QUEUED, JOB_STATUS_RUNNING, JOB_STATUS_CANCELLING)
 TERMINAL_JOB_STATUSES = (JOB_STATUS_SUCCESS, JOB_STATUS_FAILED, JOB_STATUS_CANCELLED)
@@ -54,6 +55,9 @@ DATA_WRITE_JOB_TYPES = (JOB_TYPE_COLLECT, JOB_TYPE_CLEAR_DATA, JOB_TYPE_CLEAR_CA
 SOURCE_LOCK_JOB_STATUSES = (JOB_STATUS_RUNNING, JOB_STATUS_CANCELLING)
 QUEUE_POLL_SECONDS = 0.35
 JOB_STALE_LOCK_MINUTES = 90
+ORPHAN_CANCELLING_GRACE_SECONDS = 30
+JOB_RECOVERY_SWEEP_SECONDS = 15
+DATA_DELETE_BATCH_SIZE = 50_000
 DEFAULT_WORKER_CONCURRENCY = 2
 
 RUNNING_DATA_JOB_TASKS: dict[int, asyncio.Task[Any]] = {}
@@ -77,7 +81,7 @@ async def enqueue_datasource_job(
     task_type: str,
     *,
     payload: dict[str, Any] | None = None,
-    rollback_policy: str = "keep_committed_batches",
+    rollback_policy: str = RollbackPolicy.KEEP_COMMITTED_BATCHES.value,
     dedupe_key: str | None = None,
 ) -> CollectionTask:
     if dedupe_key:
@@ -281,6 +285,7 @@ class DataJobWorker:
         self._task: asyncio.Task[None] | None = None
         self._stop_event: asyncio.Event | None = None
         self._running: set[asyncio.Task[Any]] = set()
+        self._last_recovery_sweep_at: datetime | None = None
 
     def start(self) -> None:
         if self._task and not self._task.done():
@@ -303,6 +308,11 @@ class DataJobWorker:
         await self._recover_stale_running_jobs()
         while not self._stop_event.is_set():
             self._running = {task for task in self._running if not task.done()}
+            if (
+                self._last_recovery_sweep_at is None
+                or (_utcnow() - self._last_recovery_sweep_at).total_seconds() >= JOB_RECOVERY_SWEEP_SECONDS
+            ):
+                await self._recover_stale_running_jobs()
             if len(self._running) >= self.concurrency:
                 await asyncio.sleep(QUEUE_POLL_SECONDS)
                 continue
@@ -316,7 +326,9 @@ class DataJobWorker:
             self._running.add(runner)
 
     async def _recover_stale_running_jobs(self) -> None:
+        self._last_recovery_sweep_at = _utcnow()
         cutoff = _utcnow() - timedelta(minutes=JOB_STALE_LOCK_MINUTES)
+        orphan_cancelling_cutoff = _utcnow() - timedelta(seconds=ORPHAN_CANCELLING_GRACE_SECONDS)
         async with async_session_factory() as db:
             result = await db.execute(
                 select(CollectionTask)
@@ -332,6 +344,21 @@ class DataJobWorker:
                 job.error_message = "Marked failed after stale data job lock timeout"
             if stale_jobs:
                 await db.commit()
+            orphan_result = await db.execute(
+                select(CollectionTask)
+                .where(CollectionTask.status == JOB_STATUS_CANCELLING)
+                .where(CollectionTask.locked_at.is_(None))
+                .where(CollectionTask.requested_cancel_at.is_not(None))
+                .where(CollectionTask.requested_cancel_at < orphan_cancelling_cutoff)
+            )
+            for job in orphan_result.scalars().all():
+                if job.id in RUNNING_DATA_JOB_TASKS:
+                    continue
+                await _cancel_task_without_runner(
+                    db,
+                    job,
+                    reason=job.cancel_reason or "cancelled_after_orphaned_runner",
+                )
 
     async def _claim_next_job(self) -> int | None:
         async with async_session_factory() as db:
@@ -477,15 +504,22 @@ async def _run_clear_data_job(db: AsyncSession, task: CollectionTask) -> None:
     await db.commit()
     await _broadcast_task_update(task)
 
-    count_result = await db.execute(
-        select(CollectedData.id).where(CollectedData.source == source)
+    deleted_count = await _delete_table_rows_by_source(
+        db,
+        task,
+        table_name="collected_data",
+        source_column="source",
+        source=source,
+    )
+    derived_deleted_counts = await _clear_derived_datasource_data_in_batches(
+        db,
+        task,
+        source,
+        progress_offset=deleted_count,
     )
-    collected_ids = [row[0] for row in count_result.all()]
-    derived_deleted_counts = await clear_derived_datasource_data(db, source)
-    if collected_ids:
-        await db.execute(CollectedData.__table__.delete().where(CollectedData.id.in_(collected_ids)))
-    deleted_count = len(collected_ids)
     derived_deleted_count = sum(derived_deleted_counts.values())
+    if any(key.startswith("ais_") for key in derived_deleted_counts):
+        await db.execute(text("ANALYZE ais_raw_observations"))
 
     task.records_processed = deleted_count + derived_deleted_count
     task.total_records = task.records_processed
@@ -504,10 +538,99 @@ async def _run_clear_data_job(db: AsyncSession, task: CollectionTask) -> None:
     task.phase = "completed"
     task.phase_message = "数据库数据已清理"
     task.completed_at = _utcnow()
+    datasource = await db.get(DataSource, task.datasource_id)
+    if datasource is not None:
+        datasource.last_status = JOB_STATUS_SUCCESS
+        datasource.last_run_at = task.completed_at
+    await db.execute(
+        DataSnapshot.__table__.update()
+        .where(DataSnapshot.source == source)
+        .values(is_current=False)
+    )
     await db.commit()
     await _broadcast_task_update(task)
 
 
+async def _delete_table_rows_by_source(
+    db: AsyncSession,
+    task: CollectionTask,
+    *,
+    table_name: str,
+    source_column: str,
+    source: str,
+    progress_offset: int = 0,
+) -> int:
+    deleted = 0
+    while True:
+        result = await db.execute(
+            text(
+                f"""
+                WITH doomed AS (
+                    SELECT ctid
+                    FROM {table_name}
+                    WHERE {source_column} = :source
+                    LIMIT :batch_size
+                ),
+                deleted_rows AS (
+                    DELETE FROM {table_name}
+                    USING doomed
+                    WHERE {table_name}.ctid = doomed.ctid
+                    RETURNING 1
+                )
+                SELECT COUNT(*) FROM deleted_rows
+                """
+            ),
+            {"source": source, "batch_size": DATA_DELETE_BATCH_SIZE},
+        )
+        batch_deleted = max(int(result.scalar_one() or 0), 0)
+        if batch_deleted <= 0:
+            break
+        deleted += batch_deleted
+        task.records_processed = progress_offset + deleted
+        task.phase_current = task.records_processed
+        task.phase_unit = "records"
+        task.phase_message = f"正在删除数据：{task.records_processed} 条"
+        await db.commit()
+        await _broadcast_task_update(task)
+    return deleted
+
+
+async def _clear_derived_datasource_data_in_batches(
+    db: AsyncSession,
+    task: CollectionTask,
+    source: str,
+    progress_offset: int = 0,
+) -> dict[str, int]:
+    deleted_counts: dict[str, int] = {}
+    if source in {"barentswatch_vessels", "aisstream_vessels"}:
+        deleted_counts["ais_conflict_records"] = await _delete_table_rows_by_source(
+            db,
+            task,
+            table_name="ais_conflict_records",
+            source_column="selected_source",
+            source=source,
+            progress_offset=progress_offset + sum(deleted_counts.values()),
+        )
+        deleted_counts["ais_source_health"] = await _delete_table_rows_by_source(
+            db,
+            task,
+            table_name="ais_source_health",
+            source_column="source",
+            source=source,
+            progress_offset=progress_offset + sum(deleted_counts.values()),
+        )
+        deleted_counts["ais_raw_observations"] = await _delete_table_rows_by_source(
+            db,
+            task,
+            table_name="ais_raw_observations",
+            source_column="source",
+            source=source,
+            progress_offset=progress_offset + sum(deleted_counts.values()),
+        )
+        return deleted_counts
+    return await clear_derived_datasource_data(db, source)
+
+
 async def _run_clear_cache_job(db: AsyncSession, task: CollectionTask) -> None:
     source = str(task.source or (task.payload or {}).get("source") or "").strip()
     if not source:

backend/app/services/datasource_connectivity.py

@@ -13,6 +13,7 @@ from sqlalchemy import func, select
 
 from app.core.data_sources import get_data_sources_config
 from app.core.datasource_defaults import DEFAULT_DATASOURCES
+from app.core.enums import JobStatus
 from app.models.collected_data import CollectedData
 from app.models.datasource import DataSource
 from app.models.datasource_config import DataSourceConfig
@@ -398,7 +399,7 @@ async def has_collected_data(db, source: str) -> bool:
 
     datasource_result = await db.execute(select(DataSource).where(DataSource.source == source))
     datasource = datasource_result.scalar_one_or_none()
-    return bool(datasource and datasource.last_status == "success")
+    return bool(datasource and datasource.last_status == JobStatus.SUCCESS.value)
 
 
 async def get_builtin_connection_status(

backend/app/services/docs_gatekeeper.py

@@ -6,6 +6,7 @@ from dataclasses import dataclass
 from pathlib import Path
 from typing import Literal
 
+from app.core.enums import UserRole
 from app.models.user import User
 
 DocsAccess = Literal["public", "docs_user", "docs_developer", "docs_admin"]
@@ -43,7 +44,9 @@ DOCS_METADATA: tuple[DocsMetadata, ...] = (
     DocsMetadata("earth-satellite-footprint-policy.md", "earth-satellite-footprint-policy", "docs_developer", "Earth", 13, "智能星球卫星覆盖策略", "Intelligent Planet Satellite Footprint Policy"),
     DocsMetadata("earth-bgp-context.md", "earth-bgp-context", "docs_developer", "Earth", 14, "BGP 态势上下文", "BGP Context"),
     DocsMetadata("earth-interactable-usage.md", "earth-interactable-usage", "docs_developer", "Earth", 16, "智能星球可交互图标接入", "Intelligent Planet Interactable Usage"),
-    DocsMetadata("earth-toolbar-overlay-coordination.md", "earth-toolbar-overlay-coordination", "docs_developer", "Earth", 17, "智能星球工具栏与浮层协同", "Intelligent Planet Toolbar and Overlay Coordination"),
+    DocsMetadata("earth-interactable-clustering.md", "earth-interactable-clustering", "docs_developer", "Earth", 17, "智能星球可交互图标聚类策略", "Intelligent Planet Interactable Clustering"),
+    DocsMetadata("earth-toolbar-overlay-coordination.md", "earth-toolbar-overlay-coordination", "docs_developer", "Earth", 18, "智能星球工具栏与浮层协同", "Intelligent Planet Toolbar and Overlay Coordination"),
+    DocsMetadata("earth-news-sources.md", "earth-news-sources", "docs_developer", "Earth", 19, "智能星球新闻源配置", "Intelligent Planet News Source Configuration"),
     DocsMetadata("frontend-admin-frontend-context.md", "frontend-admin-frontend-context", "docs_developer", "Frontend", 20, "控制台前端结构", "Admin Frontend Context"),
     DocsMetadata("frontend-layout-guidelines.md", "frontend-layout-guidelines", "docs_developer", "Frontend", 21, "前端布局指南", "Frontend Layout Guidelines"),
     DocsMetadata("tactile-ui-components.md", "tactile-ui-components", "docs_developer", "Frontend", 24, "Tactile UI 组件库", "Tactile UI Components"),
@@ -52,6 +55,7 @@ DOCS_METADATA: tuple[DocsMetadata, ...] = (
     DocsMetadata("datasource-collector-settings-connectivity.md", "datasource-collector-settings-connectivity", "docs_developer", "Backend", 32, "数据源、采集器设置与连接验证", "Datasource Collector Settings and Connectivity"),
     DocsMetadata("backend-datasources-api-performance.md", "backend-datasources-api-performance", "docs_developer", "Backend", 33, "数据源 API 性能", "Datasource API Performance"),
     DocsMetadata("data-job-earth-sync-architecture.md", "data-job-earth-sync-architecture", "docs_developer", "Backend", 34, "数据作业与 Outbox 技术架构", "Data Jobs and Outbox Architecture"),
+    DocsMetadata("backend-enum-contracts.md", "backend-enum-contracts", "docs_developer", "Backend", 35, "后端枚举与字符串兼容契约", "Backend Enum and String Compatibility Contract"),
     DocsMetadata("location-pipeline-development.md", "location-pipeline-development", "docs_developer", "Backend", 35, "通用位置估算管线开发说明", "Shared Location Resolution Pipeline Development Guide"),
     DocsMetadata("earth-news-live-streams-collector-format.md", "earth-news-live-streams-collector-format", "docs_developer", "Backend", 36, "新闻直播采集格式", "News Live Streams Collector Format"),
     DocsMetadata("docs-gatekeeper-development.md", "docs-gatekeeper-development", "docs_developer", "Backend", 37, "Docs Gatekeeper 开发说明", "Docs Gatekeeper Development Guide"),
@@ -69,9 +73,9 @@ def get_user_gatekeeper_groups(user: User | None) -> set[str]:
         return set()
 
     role = user.role.value if hasattr(user.role, "value") else str(user.role or "")
-    if role == "super_admin":
+    if role == UserRole.SUPER_ADMIN.value:
         return {"docs_user", "docs_developer", "docs_admin"}
-    if role == "admin":
+    if role == UserRole.ADMIN.value:
         return {"docs_user", "docs_developer", "docs_admin"}
 
     groups = set()

backend/app/services/earth_boundaries.py

@@ -6,8 +6,10 @@ import asyncio
 import hashlib
 import json
 import shutil
+import subprocess
 import sys
 from datetime import UTC, datetime
+from functools import lru_cache
 from pathlib import Path
 from typing import Any
 from uuid import uuid4
@@ -25,8 +27,11 @@ BOUNDARY_MANIFEST_PATH = BOUNDARY_OUTPUT_DIR / "manifest.json"
 PMTILES_ARTIFACT_PATH = (
     REPO_ROOT / "frontend/public/earth/data/boundaries/earth-boundaries-china-pov-v1.pmtiles"
 )
-LEGACY_GEOJSON_PATH = REPO_ROOT / "frontend/public/earth/data/countries-admin0.min.geojson"
+POLICY_FALLBACK_PATH = (
+    REPO_ROOT / "frontend/public/earth/data/countries-admin0.china-pov-v1.geojson"
+)
 POV_POLICY_PATH = REPO_ROOT / "config/earth-boundary-pov-policy.china-v1.json"
+POV_GEOMETRY_PATH = REPO_ROOT / "resources/earth/boundary-pov-geometry.china-v1.json"
 LOCAL_CONFIG_PATH = REPO_ROOT / "config/earth-boundary-sources.local.json"
 EXAMPLE_CONFIG_PATH = REPO_ROOT / "config/earth-boundary-sources.example.json"
 
@@ -150,6 +155,21 @@ def _stable_json_hash(payload: Any) -> str:
     return hashlib.sha256(encoded.encode("utf-8")).hexdigest()
 
 
+@lru_cache(maxsize=16)
+def _read_versioned_asset_cached(
+    path: Path, mtime_ns: int, size_bytes: int
+) -> tuple[dict[str, Any], str]:
+    payload = _read_json(path)
+    return payload, _stable_json_hash(payload)
+
+
+def _versioned_asset(path: Path) -> tuple[dict[str, Any], str]:
+    if not path.exists():
+        return {}, ""
+    stat = path.stat()
+    return _read_versioned_asset_cached(path, stat.st_mtime_ns, stat.st_size)
+
+
 def _artifact_extension(endpoint: str, content_type: str, payload: bytes) -> str:
     suffix = Path(endpoint.split("?", 1)[0]).suffix.lower()
     if suffix in {".json", ".geojson", ".zip", ".pbf"}:
@@ -176,62 +196,37 @@ def _directory_stats(path: Path) -> dict[str, int]:
     return {"file_count": len(files), "size_bytes": sum(item.stat().st_size for item in files)}
 
 
-def _load_source_feature_collection(source: dict[str, Any]) -> dict[str, Any]:
-    path = REPO_ROOT / source["path"]
-    payload = _read_json(path)
-    features = payload.get("features") if isinstance(payload, dict) else None
-    return {
-        "type": "FeatureCollection",
-        "features": features if isinstance(features, list) else [],
-    }
-
-
 def _write_high_precision_geojson_manifest(
     sources: list[dict[str, Any]],
     build_input_hash: str,
     missing_tools: list[str],
 ) -> dict[str, Any]:
-    BOUNDARY_OUTPUT_DIR.mkdir(parents=True, exist_ok=True)
     admin0 = next(source for source in sources if source["kind"] == "admin0-boundaries")
     coastline = next(source for source in sources if source["kind"] == "coastline")
     claim_lines = next(source for source in sources if source["kind"] == "claim-lines")
-
-    admin0_payload = _load_source_feature_collection(admin0)
-    coastline_payload = _load_source_feature_collection(coastline)
-    claim_payload = _load_source_feature_collection(claim_lines)
-    for feature in coastline_payload["features"]:
-        props = feature.setdefault("properties", {})
-        if isinstance(props, dict):
-            props["PLANET_LAYER"] = "coastline"
-
-    base_payload = {
-        "type": "FeatureCollection",
-        "features": [*admin0_payload["features"], *coastline_payload["features"]],
-    }
-    base_path = BOUNDARY_OUTPUT_DIR / "earth-boundaries-high-precision-base.geojson"
-    hover_path = BOUNDARY_OUTPUT_DIR / "earth-boundaries-high-precision-hover.geojson"
-    claim_path = BOUNDARY_OUTPUT_DIR / "earth-boundaries-high-precision-claims.geojson"
-    _write_json(base_path, base_payload)
-    _write_json(hover_path, admin0_payload)
-    _write_json(claim_path, claim_payload)
-
-    manifest = {
-        "version": "natural-earth-v1",
-        "builtAt": _utc_now_iso(),
-        "tileProvider": "geojson-high-precision",
-        "format": "geojson-directory",
-        "buildInputHash": build_input_hash,
-        "base": base_path.name,
-        "hoverIndex": hover_path.name,
-        "claimLine": claim_path.name,
-        "sourceFeatureCount": {
-            "admin0": len(admin0_payload["features"]),
-            "coastline": len(coastline_payload["features"]),
-            "claimLines": len(claim_payload["features"]),
-        },
-        "pmtiles": None,
-        "missingTools": missing_tools,
-    }
+    result = subprocess.run(
+        [
+            sys.executable,
+            str(Path(__file__).resolve().parents[3] / "scripts/build_earth_boundary_china_pov_geojson.py"),
+            "--admin0-source", admin0["path"],
+            "--coastline-source", coastline["path"],
+            "--claims-source", claim_lines["path"],
+            "--output-dir", _relative(BOUNDARY_OUTPUT_DIR),
+            "--build-input-hash", build_input_hash,
+            "--pov-policy", _relative(POV_POLICY_PATH),
+            "--pov-geometry", str(POV_GEOMETRY_PATH),
+        ],
+        cwd=REPO_ROOT,
+        capture_output=True,
+        text=True,
+    )
+    if result.returncode:
+        raise EarthBoundaryBuildError(
+            result.stderr.strip() or "Earth boundary policy build failed",
+            code="policy_build_failed",
+        )
+    manifest = _read_json(BOUNDARY_MANIFEST_PATH)
+    manifest["missingTools"] = missing_tools
     _write_json(BOUNDARY_MANIFEST_PATH, manifest)
     return manifest
 
@@ -291,6 +286,7 @@ def _build_input_hash(source_manifest: dict[str, Any]) -> str:
                 for source in source_manifest.get("sources", [])
             ],
             "pov_policy": source_manifest.get("povPolicy"),
+            "pov_geometry_sha256": _versioned_asset(POV_GEOMETRY_PATH)[1],
             "build_config": BUILD_CONFIG,
         }
     )
@@ -301,7 +297,47 @@ def _has_current_artifacts(boundary_manifest: dict[str, Any], build_input_hash:
         bool(boundary_manifest)
         and boundary_manifest.get("buildInputHash") == build_input_hash
         and boundary_manifest.get("tileProvider") == "pmtiles-mvt"
+        and _policy_verified(boundary_manifest)
         and PMTILES_ARTIFACT_PATH.exists()
+        and _pmtiles_hover_exists(boundary_manifest)
+    )
+
+
+def _policy_verified(manifest: dict[str, Any]) -> bool:
+    policy, policy_hash = _versioned_asset(POV_POLICY_PATH)
+    geometry, geometry_hash = _versioned_asset(POV_GEOMETRY_PATH)
+    coverage = manifest.get("povPolicy") or {}
+    return (
+        policy.get("profile") == "china-pov-v1"
+        and coverage.get("profile") == policy["profile"]
+        and coverage.get("sha256") == policy_hash
+        and coverage.get("reviewedGeometrySourceCommit")
+        == geometry.get("sourceCommit")
+        and coverage.get("geometrySha256") == geometry_hash
+        and coverage.get("applied") is True
+        and {"china-taiwan-penghu", "china-hong-kong", "china-macao", "kosovo"}
+        <= set(coverage.get("appliedRuleIds") or [])
+    )
+
+
+def _geojson_artifacts_exist(manifest: dict[str, Any]) -> bool:
+    paths = [manifest.get("base"), manifest.get("hoverIndex")]
+    if manifest.get("chinaClaims", {}).get("available"):
+        paths.append(manifest.get("claimLine"))
+    for asset in paths:
+        if not isinstance(asset, str) or Path(asset).name != asset:
+            return False
+        if not (BOUNDARY_OUTPUT_DIR / asset).is_file():
+            return False
+    return True
+
+
+def _pmtiles_hover_exists(manifest: dict[str, Any]) -> bool:
+    hover = manifest.get("hoverIndex")
+    return (
+        isinstance(hover, str)
+        and Path(hover).name == hover
+        and (BOUNDARY_OUTPUT_DIR / hover).is_file()
     )
 
 
@@ -313,24 +349,34 @@ def get_boundary_status() -> dict[str, Any]:
     manifest_exists = BOUNDARY_MANIFEST_PATH.exists()
     high_precision_ready = (
         manifest_exists
+        and _policy_verified(boundary_manifest)
         and (
             (
                 boundary_manifest.get("tileProvider") == "pmtiles-mvt"
                 and pmtiles_exists
+                and _pmtiles_hover_exists(boundary_manifest)
+            )
+            or (
+                boundary_manifest.get("tileProvider") == "geojson-high-precision"
+                and _geojson_artifacts_exist(boundary_manifest)
             )
-            or boundary_manifest.get("tileProvider") == "geojson-high-precision"
         )
     )
-    legacy_exists = LEGACY_GEOJSON_PATH.exists()
+    legacy_exists = POLICY_FALLBACK_PATH.exists() and _policy_verified(
+        _versioned_asset(POLICY_FALLBACK_PATH)[0]
+    )
     provider = (
         boundary_manifest.get("tileProvider")
         if high_precision_ready
-        else "legacy-geojson" if legacy_exists else "missing"
+        else "policy-geojson-fallback" if legacy_exists else "missing"
     )
     return {
         "provider": provider,
         "high_precision_ready": high_precision_ready,
         "fallback_available": legacy_exists,
+        "policy_publishable": bool(
+            high_precision_ready and boundary_manifest.get("povPolicy", {}).get("publicationReady")
+        ),
         "config_source": config_source,
         "config_path": _relative(LOCAL_CONFIG_PATH),
         "config_exists": LOCAL_CONFIG_PATH.exists(),
@@ -347,6 +393,8 @@ def get_boundary_status() -> dict[str, Any]:
             "tileProvider": boundary_manifest.get("tileProvider"),
             "buildInputHash": boundary_manifest.get("buildInputHash"),
             "builtAt": boundary_manifest.get("builtAt"),
+            "povPolicy": boundary_manifest.get("povPolicy"),
+            "publicationReady": boundary_manifest.get("povPolicy", {}).get("publicationReady", False),
         },
         "pmtiles": {
             "path": _relative(PMTILES_ARTIFACT_PATH),
@@ -354,9 +402,9 @@ def get_boundary_status() -> dict[str, Any]:
             "size_bytes": PMTILES_ARTIFACT_PATH.stat().st_size if pmtiles_exists else 0,
         },
         "legacy": {
-            "path": _relative(LEGACY_GEOJSON_PATH),
+            "path": _relative(POLICY_FALLBACK_PATH),
             "exists": legacy_exists,
-            "size_bytes": LEGACY_GEOJSON_PATH.stat().st_size if legacy_exists else 0,
+            "size_bytes": POLICY_FALLBACK_PATH.stat().st_size if legacy_exists else 0,
         },
         "source_manifest": {
             "path": _relative(SOURCE_MANIFEST_PATH),
@@ -577,6 +625,8 @@ async def build_boundary_assets(progress_callback: Any = None) -> dict[str, Any]
                 build_input_hash,
                 "--pov-policy",
                 _relative(POV_POLICY_PATH),
+                "--pov-geometry",
+                str(POV_GEOMETRY_PATH),
             ]
         )
         if progress_callback:

backend/app/services/earth_db_change_listener.py

@@ -56,6 +56,8 @@ def build_earth_update_from_db_payload(payload: dict[str, Any]) -> dict[str, Any
     source_has_adapter = bool(get_earth_update_layers_for_source(source))
     effective_source = source if source_has_adapter else (table_name if table_name else source)
     operation = payload.get("operation")
+    if "vessels" in layers and operation in {"DELETE", "TRUNCATE"}:
+        refresh_strategy = "reload"
     update: dict[str, Any] = {
         "event": "earth.layer.changed",
         "action": "database_changed",
@@ -113,6 +115,8 @@ class PendingEarthDbChange:
         operation = payload.get("operation")
         if operation:
             self.operations.add(str(operation))
+            if "vessels" in self.layers and operation in {"DELETE", "TRUNCATE"}:
+                self.refresh_strategy = "reload"
         entity_keys = payload.get("entity_keys")
         if not isinstance(entity_keys, list):
             entity_key = payload.get("entity_key")
@@ -176,6 +180,17 @@ class EarthDbChangeDispatcher:
         if not update:
             return False
 
+        if payload.get("table") == "vessel_current_state" and payload.get("operation") == "DELETE":
+            from app.core.websocket.broadcaster import broadcaster
+
+            keys = payload.get("entity_keys")
+            if not isinstance(keys, list):
+                keys = [payload.get("entity_key")]
+            broadcaster.enqueue_vessel_update({
+                "action": "remove",
+                "vessels": [{"mmsi": key} for key in keys if key is not None],
+            })
+
         source = update["source"]
         pending = self._pending.get(source)
         if pending is None:

backend/app/services/earth_interactables.py

@@ -62,11 +62,11 @@ def interactables_to_geojson(items: list[EarthInteractable]) -> dict[str, Any]:
 def invalidate_interactable_cache(layer: str | None = None) -> int:
     layer_key = str(layer or "*").strip() or "*"
     deleted = earth_layer_cache.delete_pattern(
-        f"{EARTH_LAYER_CACHE_PREFIX}:interactables:layer:{layer_key}*"
+        f"{EARTH_LAYER_CACHE_PREFIX}:interactables:interactable_layer:{layer_key}*"
     )
     if layer_key != "all":
         deleted += earth_layer_cache.delete_pattern(
-            f"{EARTH_LAYER_CACHE_PREFIX}:interactables:layer:all*"
+            f"{EARTH_LAYER_CACHE_PREFIX}:interactables:interactable_layer:all*"
         )
     return deleted
 

backend/app/services/earth_layer_adapters.py

@@ -20,10 +20,12 @@ class EarthLayerAdapter:
 
 EARTH_LAYER_ADAPTERS: tuple[EarthLayerAdapter, ...] = (
     EarthLayerAdapter(
-        sources=frozenset({"barentswatch_vessels", "aisstream_vessels", "vessel_static", "vessel_position", "ais_raw_observations", "ais_source_health"}),
-        tables=frozenset({"vessel_static", "vessel_position", "ais_raw_observations", "ais_source_health"}),
+        sources=frozenset({"barentswatch_vessels", "aisstream_vessels", "vessel_static", "vessel_position", "vessel_current_state", "ais_raw_observations", "ais_source_health"}),
+        tables=frozenset({"vessel_static", "vessel_position", "vessel_current_state", "ais_raw_observations", "ais_source_health"}),
         layers=("vessels",),
         cache_patterns=("vessels*", "summary*"),
+        derived_models=("ais_raw_observations", "ais_conflict_records", "ais_source_health"),
+        refresh_strategy="delta",
     ),
     EarthLayerAdapter(
         sources=frozenset(
@@ -163,17 +165,24 @@ async def clear_derived_datasource_data(db: AsyncSession, source: str) -> dict[s
     from app.models.bgp_anomaly import BGPAnomaly
     from app.models.bgp_incident import BGPIncident
     from app.models.bgp_observation import BGPObservation
+    from app.models.vessel import AISConflictRecord, AISRawObservation, AISSourceHealth
 
     model_by_key: dict[str, Any] = {
         "bgp_observations": BGPObservation,
         "bgp_anomalies": BGPAnomaly,
         "bgp_incidents": BGPIncident,
+        "ais_raw_observations": AISRawObservation,
+        "ais_conflict_records": AISConflictRecord,
+        "ais_source_health": AISSourceHealth,
     }
     deleted_counts: dict[str, int] = {}
     for key in adapter.derived_models:
         model = model_by_key.get(key)
         if model is None:
             continue
-        result = await db.execute(model.__table__.delete().where(model.source == source))
+        if key == "ais_conflict_records":
+            result = await db.execute(model.__table__.delete().where(model.selected_source == source))
+        else:
+            result = await db.execute(model.__table__.delete().where(model.source == source))
         deleted_counts[key] = int(result.rowcount or 0)
     return deleted_counts