release: bump version to 0.74.2

Release 0.58.0 includes the Earth high-precision boundary PMTiles/MVT pipeline, standardized Earth boundary source collectors, China POV boundary configuration templates, and removal of the legacy low-precision GeoJSON fallback. It also adds Earth news target-location queueing/archive support, fixes datasource task status visibility, documents the Earth surface depth-spacing rules that prevent far-zoom z-fighting snow/black blocks, and updates bilingual operations/developer docs.

.claude/commands/cleanup.md

@@ -1,120 +0,0 @@
----
-description: 审查当前工作区未提交代码中的垃圾代码，并在不影响逻辑的前提下自动清理
-argument-hint: 可选：指定要检查的文件或目录（默认检查所有未提交修改）
-allowed-tools: ["Read", "Edit", "Bash", "Grep", "Glob"]
----
-
-# /cleanup — 垃圾代码审查与清理
-
-分析当前工作区（git diff）中的未提交代码，找出并修复常见垃圾代码，**不得改变任何运行逻辑**。
-
-## 检查范围
-
-若 `$ARGUMENTS` 非空，则只检查指定文件/目录；否则检查所有未提交修改（`git diff HEAD`）。
-
-## 审查清单
-
-按优先级检查以下问题（只报告在本次 diff 中**新增或修改**的代码里存在的问题）：
-
-### 1. 重复逻辑 (Duplicate Logic)
-- 完全相同或高度相似的代码块在多处出现
-- 同一函数/方法被多个地方各自实现，已有公共版本未被复用
-- 相同的 DOM 查询、正则、模板字符串在同一文件重复
-
-### 2. Magic Numbers / Magic Strings
-- 裸数字直接参与计算（如偏移量、时间、尺寸、阈值），没有命名常量
-- 硬编码字符串（如 id 名、状态值、URL 片段）散落在逻辑中
-- 例外：`0`, `1`, `-1`, `100`, `""` 等语义明确的惯用值不算
-
-### 3. 命名问题
-- 含义不明的缩写变量（如 `or_`, `tmp2`, `x2`）
-- 命名与实际用途不符
-- 同一概念在不同地方用不同名字表达
-
-### 4. 死代码 / 无效代码
-- 注释掉的旧代码块（3行以上）
-- 声明后从未使用的变量/参数/导入
-- 永远不会执行的条件分支
-
-### 5. 代码风格问题
-- 尾部空白字符（trailing whitespace）
-- 同一文件内风格不一致（如混用单双引号、缩进不统一）
-- 空行使用不一致（连续多个空行等）
-
-### 6. 其他常见问题
-- 私有辅助函数应被 export 但没有，导致调用方重复实现
-- 类型/接口重复定义
-- 过于冗长的条件表达式可以简化（不改逻辑）
-
-## 执行步骤
-
-### Step 1 — 获取待检查文件列表
-
-```bash
-# 无参数时：获取所有未提交修改
-git diff HEAD --name-only
-
-# 有参数时：用 $ARGUMENTS 过滤
-```
-
-### Step 2 — 逐文件阅读并分析
-
-- 用 Read 工具读取完整文件（不只读 diff）
-- 对照审查清单，记录每个问题：文件名、行号、问题类型、建议修复方式
-
-### Step 3 — 报告问题清单
-
-在修改前，先以列表形式输出所有发现的问题：
-
-```
-发现 N 个问题：
-
-[文件] js/foo.js
-  · L34, L78: 重复逻辑 — 两处都实现了相同的 DOM 查询，可提取到 getPanel()
-  · L91: Magic number — 硬编码 14 作为偏移量，应命名为 TOOLTIP_OFFSET
-
-[文件] js/bar.js
-  · L12: 命名问题 — 变量 `or_` 语义不明，应命名为 outerR/outerG/outerB
-  ...
-```
-
-如果没有发现问题，直接输出"未发现垃圾代码，当前代码质量良好。"并停止。
-
-### Step 4 — 执行修复
-
-对每个问题，使用 Edit 工具进行**最小化修改**：
-
-- **重复逻辑**：提取为共享常量/函数，更新所有调用点
-- **Magic number**：在文件顶部或逻辑附近声明 `const NAME = value`，替换所有引用
-- **命名问题**：重命名变量，更新所有使用处
-- **死代码**：直接删除
-- **尾部空白/风格**：修正
-- **未 export 的函数**：添加 `export`，在调用方改为导入（不重复实现）
-
-**修复原则：**
-- 只改在审查清单中发现的问题，不做额外优化
-- 每次 Edit 只修改确实有问题的行，保持 diff 最小
-- 改完后用 `grep` 验证旧的坏代码已消失
-
-### Step 5 — 输出总结
-
-```
-清理完成：
-
-修复了 N 个问题：
-  ✓ earth.js — 提取重复 vertexShader 为 ATMOS_VERTEX_SHADER 常量
-  ✓ main.js — 提取 TOOLTIP_CURSOR_OFFSET = 14（4处引用）
-  ✓ controls.js — export updateLayerButtonState，移除 main.js 中的重复实现
-  ...
-
-未修改的问题（需人工确认）：
-  ! foo.js L45 — 注释代码块较长，建议手动确认是否可删除
-```
-
-## 约束
-
-- **禁止**改变函数签名、接口定义、导出 API（除非问题正是私有函数应被 export）
-- **禁止**添加新功能、新抽象、新参数
-- **禁止**修改注释内容（只删除注释掉的死代码）
-- **禁止**修改测试文件逻辑
-- 如果一个 Magic number 的语义不完全确定，**跳过**，在总结中标记为"需人工确认"

.claude/commands/release.md

@@ -1,146 +0,0 @@
----
-description: 发版工作流：根据变更类型决定版本号，更新所有版本文件和 changelog，运行验证，commit 并 push
-argument-hint: 可选：feature | bugfix | 或直接描述本次发布内容
-allowed-tools: ["Read", "Edit", "Bash", "Glob", "Grep"]
----
-
-# /release — Planet 发版工作流
-
-## 版本号规则
-
-| 变更类型 | 版本跳动 | 适用场景 |
-|---------|---------|---------|
-| `feature` | `+0.1.0` | 纯新功能，无 bugfix |
-| `improvement` | `+0.0.1` | UI 调整、小功能增强、bugfix 混合，或以 UI/体验改进为主的迭代 |
-| `bugfix` | `+0.0.1` | 纯 bug 修复，无新功能 |
-| `docs` / `maintenance` / `refactor` | 默认不发版，除非用户明确要求 |
-
-意图混合时以用户明确描述为准；bugfix + 小 feature 混合默认判定为 `improvement`（`+0.0.1`）。
-
-## 必须同步更新的文件
-
-使用 `git rev-parse --show-toplevel` 获取仓库根目录，以下路径均相对于根目录：
-
-- `VERSION`
-- `frontend/package.json`（`"version"` 字段）
-- `pyproject.toml`（`version =` 字段）
-- `uv.lock`（**不要手动编辑**，通过 `uv lock` 重新生成）
-- `docs/CHANGELOG.md`
-- `docs/version-history.md`
-
-## 执行步骤
-
-### Step 1 — 环境检查
-
-```bash
-git branch --show-current          # 确认在 dev 分支
-git status --short                 # 检查是否有无关的未暂存修改
-cat VERSION                        # 读取当前版本
-```
-
-若当前**不在 `dev` 分支**，停下来告知用户，不要继续。
-
-若存在无关的未暂存修改，列出并询问用户是否一并提交，或先 stash。
-
-### Step 2 — 确定发版类型与新版本号
-
-- 若 `$ARGUMENTS` 提供了明确类型（`feature` / `bugfix`），直接使用
-- 否则根据当前 `git diff HEAD` 和 `git log` 推断
-- 计算新版本号（例：`0.26.2` → bugfix → `0.26.3`）
-- **先输出发版计划供用户确认**：
-
-```
-发版计划：
-  类型：bugfix
-  版本：0.26.2 → 0.26.3
-  分支：dev
-  将更新：VERSION, frontend/package.json, pyproject.toml, uv.lock, CHANGELOG.md, version-history.md
-```
-
-### Step 3 — 更新版本号文件
-
-按顺序更新（每步用 Edit 工具，精确替换，不要重写整个文件）：
-
-1. `VERSION` — 直接替换全部内容为新版本号
-2. `frontend/package.json` — 替换 `"version": "x.x.x"` 行
-3. `pyproject.toml` — 替换 `version = "x.x.x"` 行
-4. 运行 `uv lock` 重新生成 `uv.lock`（在仓库根目录下执行）
-
-### Step 4 — 更新 CHANGELOG.md
-
-在文件顶部插入新条目，格式：
-
-```markdown
-## [x.x.x] — YYYY-MM-DD
-
-### ✨ Features / 🐛 Fixes / 🔧 Improvements
-- ...（只列高信号条目，最多 5 条）
-- ...
-
----
-```
-
-日期使用 `date +%Y-%m-%d` 获取今天的日期。
-
-### Step 5 — 更新 docs/version-history.md
-
-- 更新文件头部的"当前开发版本"字段
-- 在时间线表格顶部插入新行：`| vx.x.x | YYYY-MM-DD | 一句话摘要 |`
-
-### Step 6 — 验证
-
-针对本次变更范围做最小验证：
-
-- Python 文件有修改：`python3 -m py_compile <changed_files>`
-- Frontend 文件有修改：运行项目标准检查（若无则跳过并说明）
-- 版本号一致性检查：用 grep 确认 VERSION、package.json、pyproject.toml 中的版本号完全一致
-
-```bash
-grep -h "version" VERSION frontend/package.json pyproject.toml
-```
-
-### Step 7 — 提交前预览
-
-展示将要提交的文件列表：
-
-```bash
-git diff --stat HEAD
-```
-
-再次确认所有必须文件都在变更列表中，**不包含**非预期文件（如调试文件、.env 等）。
-
-### Step 8 — Commit & Push（用户确认后）
-
-```bash
-git add VERSION frontend/package.json pyproject.toml uv.lock docs/CHANGELOG.md docs/version-history.md
-# 若有代码变更也一并 stage
-git add <code_files>
-
-git commit -m "release: bump version to x.x.x"
-git tag vx.x.x
-git push origin dev
-git push origin vx.x.x
-```
-
-commit message 固定格式：`release: bump version to x.x.x`
-
-### Step 9 — 完成确认
-
-输出摘要：
-
-```
-✓ 版本号已更新：0.26.2 → 0.26.3
-✓ CHANGELOG 已更新
-✓ version-history 已更新
-✓ uv.lock 已重新生成
-✓ 验证通过
-✓ commit: release: bump version to 0.26.3
-✓ tag: v0.26.3
-✓ 已 push 到 origin/dev
-```
-
-## 注意事项
-
-- `uv.lock` 只能通过 `uv lock` 生成，绝不手动编辑
-- 发版 commit 只包含版本文件 + 本次功能代码，不混入无关改动
-- 若环境中 `uv` 不可用，说明原因并跳过 lockfile 更新，提醒用户手动运行

.codex/config.toml

@@ -0,0 +1,3 @@
+approval_policy = "never"
+
+sandbox_mode = "danger-full-access"

.codex/skills/cleanup/SKILL.md

@@ -21,6 +21,19 @@ If the user specifies a file or directory, check only that. Otherwise check all
 
 Only report issues present in **newly added or modified** lines of this diff — do not audit unchanged code.
 
+## Token-Saving Rule
+
+Prefer deterministic CLI checks before reading files into model context:
+
+```bash
+git diff --name-only HEAD
+git diff --unified=0 HEAD -- <path>
+git diff --check
+rg -n "TODO|FIXME|console\.log|debugger|print\(" <changed-paths>
+```
+
+Read full files only when the focused diff does not provide enough surrounding context to make a safe edit.
+
 ## Checklist
 
 ### 1. Duplicate Logic
@@ -64,7 +77,13 @@ Filter to the user-specified path if one was provided.
 
 ### Step 2 — Read and analyze each file
 
-Read the full file (not just the diff) with the Read tool. For each file, record every issue found: filename, line number, category, and suggested fix.
+Start with focused diffs:
+
+```bash
+git diff --unified=0 HEAD -- <file>
+```
+
+Use `rg`, `git diff --check`, and compiler/linter output for deterministic findings. Read the full file only for files that need surrounding context. For each issue found, record filename, line number, category, and suggested fix.
 
 ### Step 3 — Report findings before touching anything
 
@@ -99,6 +118,7 @@ Principles:
 - Only fix issues identified in the checklist — no extra improvements
 - Keep each Edit as small as possible
 - After fixing, verify the old bad pattern is gone with grep
+- Prefer `apply_patch` for targeted edits; use formatters only when the repository already uses them for the touched file type
 
 ### Step 5 — Summary
 

.codex/skills/docs/SKILL.md

@@ -0,0 +1,83 @@
+---
+name: docs
+description: Create or update repository documentation from current code changes. Use when the user asks to write docs, update docs, summarize implementation changes into docs, or check documentation coverage. Load repository-specific coverage rules from docs/documentation-coverage-rules.md when present.
+---
+
+# Docs
+
+Use this skill when the task is documentation work: creating, updating, checking, or summarizing docs for code or behavior changes.
+
+## Goal
+
+Write documentation that explains why a change exists, how it behaves, and what maintainers need to know. Keep the skill generic; repository-specific rules belong in the repository, not in this skill.
+
+## Repository Rules
+
+Before deciding scope, check whether the repository has a documentation rules file:
+
+```bash
+test -f docs/documentation-coverage-rules.md && sed -n '1,240p' docs/documentation-coverage-rules.md
+```
+
+If it exists, apply it as the project-specific coverage checklist. If it does not exist, continue with the generic workflow below.
+
+## Workflow
+
+1. Gather focused context:
+
+```bash
+git diff HEAD --stat
+git diff HEAD --name-only
+git log --oneline -10
+rg --files docs
+```
+
+If the user gives a topic, focus on that topic. Otherwise infer the doc topic from changed files. Avoid reading large full diffs by default; inspect focused files and symbols:
+
+```bash
+git diff HEAD -- <path>
+rg -n "class |def |function |export |router|@router|interface |type " <path>
+```
+
+2. Decide scope:
+
+- Prefer updating an existing relevant doc over creating a duplicate.
+- Use one document for one coherent topic.
+- Split documents only when changes cross meaningful domains.
+- Keep filenames lowercase and hyphenated.
+
+3. Write the doc:
+
+- Explain background/problem, design decisions, constraints, and operational impact.
+- Keep code snippets short and directly relevant.
+- List related files only when they help future maintainers navigate.
+- Use the repository’s existing language, heading style, and naming conventions.
+- For UI labels, chart labels, feature names, datasource names, and other terms that may become mixed Chinese/English copy, check `docs/technical/{zh,en}/naming-glossary.md` and use the documented display name. If a confusing term is missing, update the glossary in both languages as part of the docs change.
+
+4. Verify:
+
+- Read the completed doc once for clarity and stale statements.
+- Verify important referenced paths exist with `test -e` or `rg --files`.
+- Run repository-specific doc checks from `docs/documentation-coverage-rules.md` when present.
+- For Markdown links, check that user-facing titles are readable and not raw filenames unless the repository rules allow it.
+
+## Hard Constraints
+
+- Do not leave placeholder docs or copied source text pretending to be documentation.
+- Do not duplicate bilingual files byte-for-byte.
+- Do not reference PR numbers, issue numbers, or the current conversation unless explicitly requested.
+- Do not write changelog-style lists without the reasoning, constraints, and tradeoffs behind the change.
+- Keep docs concise enough to maintain.
+
+## Recommended Output
+
+After editing, summarize:
+
+```md
+Updated:
+- path/to/doc.md — what changed
+
+Verified:
+- checks that passed
+- checks that could not be run, if any
+```

.codex/skills/goal-driven/SKILL.md

@@ -0,0 +1,103 @@
+---
+name: goal-driven
+description: Run a goal-driven execution loop for very large, long-horizon, rigorously verifiable tasks. Use when the user explicitly wants the lidangzzz/goal-driven method, a master-agent plus worker-agent style workflow, or a persistent loop that keeps working until concrete success criteria are satisfied.
+---
+
+# Goal-Driven
+
+Use this skill when the user wants a strict goal-driven workflow for a hard task with:
+
+- one clear end goal
+- explicit success criteria
+- repeated verification against those criteria
+- continued execution until the criteria are actually met
+
+This skill is adapted from `lidangzzz/goal-driven`, but trimmed for local skill use to avoid bloating context.
+
+## When To Use
+
+Use it for tasks like:
+
+- compilers, interpreters, theorem-like proof work, deep refactors
+- long-running system design or implementation work
+- problems that are expensive and complex, but still objectively testable
+
+Do not use it for:
+
+- vague brainstorming without a success condition
+- short one-shot edits
+- tasks where "done" cannot be evaluated in a meaningful way
+
+## Core Model
+
+The workflow has two roles:
+
+1. Master role
+   Defines the goal, defines the success criteria, audits progress, and decides whether the work is actually complete.
+
+2. Worker role
+   Keeps advancing the task toward the goal. If a result is partial, stalled, or unverifiable, the worker continues.
+
+In Codex, only use actual subagents when the user explicitly asks for delegation or subagent work and the platform supports it. Otherwise emulate the same loop locally: keep working, checkpointing, and re-verifying until the criteria are satisfied.
+
+## Workflow
+
+1. Normalize the task into two blocks:
+   - `Goal`
+   - `Criteria for success`
+
+2. Make the criteria concrete and testable.
+   Good criteria usually include:
+   - required outputs
+   - required validations or tests
+   - edge cases or coverage thresholds
+   - what evidence proves completion
+
+3. Break the work into milestones that can each produce evidence.
+
+4. Execute the next milestone.
+   If subagents are explicitly allowed, the master may delegate bounded worker tasks.
+   If not, do the work locally but keep the master/worker mindset.
+
+5. Whenever work pauses, stalls, or appears complete, audit against the criteria directly.
+   Check artifacts, tests, logs, diffs, metrics, or other real evidence.
+
+6. If the criteria are not met, continue with a specific delta:
+   - what is still missing
+   - what evidence failed
+   - what the next worker pass must improve
+
+7. Stop only when the criteria are met, or when the user explicitly stops the process.
+
+## Operating Rules
+
+- Prefer objective checks over self-reported completion.
+- Prefer deterministic tool evidence over long model summaries: use `rg`, `git diff --stat`, targeted `git diff -- <path>`, tests, builds, linters, `curl`, or database queries when they can prove a criterion.
+- Do not paste large command output into the conversation; summarize the evidence and keep raw output in tool calls.
+- Do not confuse progress with completion.
+- If the worker says "done", verify it.
+- If verification fails, continue from the gap instead of restarting blindly.
+- Keep the goal stable unless the user changes it.
+- Tighten fuzzy criteria before sinking large amounts of effort.
+
+## Recommended Response Shape
+
+When starting a goal-driven task, structure the kickoff like this:
+
+```md
+Goal
+- ...
+
+Criteria for success
+- ...
+
+Current plan
+1. ...
+2. ...
+3. ...
+
+Verification
+- What evidence will prove completion
+```
+
+For a reusable prompt template, read [references/prompt-template.md](references/prompt-template.md).

.codex/skills/goal-driven/agents/openai.yaml

@@ -0,0 +1,7 @@
+interface:
+  display_name: "Goal-Driven"
+  short_description: "Drive complex work until explicit success criteria are met."
+  default_prompt: "Use $goal-driven to turn this task into a concrete goal, explicit success criteria, and a verification-driven execution loop."
+
+policy:
+  allow_implicit_invocation: true

.codex/skills/goal-driven/references/prompt-template.md

@@ -0,0 +1,38 @@
+# Goal-Driven Prompt Template
+
+Use this when you want a reusable kickoff prompt for a master/worker execution loop.
+
+```md
+# Goal-Driven System
+
+Goal: [[[[[DEFINE THE FINAL GOAL HERE]]]]]
+
+Criteria for success: [[[[[DEFINE THE SUCCESS CRITERIA HERE]]]]]
+
+You are the master agent.
+
+Your job is to:
+1. Keep the goal and criteria fixed.
+2. Start worker execution toward the goal.
+3. Audit any claimed progress against the criteria.
+4. If the criteria are not met, continue the work with a precise next delta.
+5. Stop only when the criteria are satisfied or the user explicitly stops the process.
+
+Worker requirements:
+1. Break the task into subproblems.
+2. Keep producing concrete progress toward the goal.
+3. Report evidence, not just claims.
+4. Continue until the criteria are satisfied.
+
+Master audit loop:
+1. Check whether the worker is still making progress.
+2. If the worker stalls or claims completion, verify against the criteria.
+3. If verification fails, resume work from the remaining gap.
+4. Repeat until the criteria are met.
+```
+
+## Notes
+
+- Stronger criteria produce better results than stronger rhetoric.
+- Prefer measurable checks such as tests, parity checks, generated artifacts, benchmarks, or reviewable outputs.
+- If the environment does not support subagents, emulate the same loop locally.

.codex/skills/release/SKILL.md

@@ -18,7 +18,7 @@ Do not use this skill for ordinary commits that are not being released.
 
 ## Versioning Rules
 
-- `feature` -> bump `+0.1.0`
+- `feature` -> bump minor and reset patch to `0` (`x.y.z` → `x.(y+1).0`; for example `0.41.2` → `0.42.0`)
 - `bugfix` -> bump `+0.0.1`
 - `docs`, `maintenance`, and `refactor` do not bump by default unless the user explicitly wants a release
 
@@ -35,6 +35,19 @@ Use `git rev-parse --show-toplevel` to get the repo root. All paths are relative
 - `docs/CHANGELOG.md`
 - `docs/version-history.md`
 
+## Token-Saving Rule
+
+Release work should be driven by deterministic CLI evidence. Prefer compact commands and targeted file reads:
+
+```bash
+git status --short
+git diff --stat HEAD
+git diff --name-only HEAD
+rg -n "version|^## |^Released:|current" VERSION frontend/package.json pyproject.toml docs/CHANGELOG.md docs/version-history.md
+```
+
+Do not inspect full diffs unless deciding whether changed code belongs in the release.
+
 ## Workflow
 
 ### Step 1 — Environment check
@@ -52,8 +65,10 @@ If unrelated uncommitted changes exist, list them and ask the user whether to in
 ### Step 2 — Determine release type and next version
 
 - If the user provided an explicit type (`feature` / `bugfix`), use it
-- Otherwise infer from `git diff HEAD` and recent `git log`
+- Otherwise infer from `git diff --stat HEAD`, `git diff --name-only HEAD`, focused diffs for changed code, and recent `git log`
-- Compute the next version (e.g. `0.26.2` → bugfix → `0.26.3`)
+- Compute the next version:
+  - `feature`: increment minor and reset patch to `0` (e.g. `0.41.2` → `0.42.0`)
+  - `bugfix`: increment patch only (e.g. `0.26.2` → `0.26.3`)
 - **Show the release plan before making any changes:**
 
 ```
@@ -104,12 +119,13 @@ Get today's date with `date +%Y-%m-%d`.
 
 Run the smallest relevant validation for the changes in scope:
 
-- Python files changed: `python3 -m py_compile <changed_files>`
+- Python files changed: list changed Python files with `git diff --name-only HEAD -- '*.py'`, then run `python3 -m py_compile <changed_files>`
-- Frontend files changed: run the project-standard check if available; otherwise skip and say so
+- Frontend files changed: list changed frontend files with `git diff --name-only HEAD -- frontend`, then run the project-standard check if available; otherwise skip and say so
 - Version consistency: confirm VERSION, package.json, pyproject.toml, and uv.lock all show the same version
 
 ```bash
-grep -h "version" VERSION frontend/package.json pyproject.toml
+cat VERSION
+rg -n "\"version\":|^version =|version = " frontend/package.json pyproject.toml uv.lock
 ```
 
 ### Step 7 — Pre-commit preview

.dockerignore

@@ -0,0 +1,18 @@
+**
+
+!pyproject.toml
+!uv.lock
+!VERSION
+!backend/
+!backend/**
+!aiprovider/
+!aiprovider/**
+
+backend/.env
+backend/.env.*
+aiprovider/.env
+aiprovider/.env.*
+!aiprovider/.env.example
+**/__pycache__/
+**/*.pyc
+**/*.pyo

.gitea/workflows/ci.yaml

@@ -0,0 +1,64 @@
+name: ci
+
+on:
+  push:
+    branches:
+      - dev
+      - main
+  pull_request:
+
+env:
+  REGISTRY: gitea.rclaw.top
+  IMAGE_NAMESPACE: linkong/planet
+
+jobs:
+  backend:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - name: Install uv
+        run: curl -LsSf https://astral.sh/uv/install.sh | sh
+      - name: Sync Python dependencies
+        run: ~/.local/bin/uv sync --group dev
+      - name: Run backend smoke tests
+        working-directory: backend
+        run: PYTHONPATH=. "$GITHUB_WORKSPACE/.venv/bin/python" -m pytest -s tests/test_api.py tests/test_realtime_sources.py -q
+
+  frontend:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - name: Install Bun
+        run: curl -fsSL https://bun.sh/install | bash
+      - name: Build frontend
+        working-directory: frontend
+        run: |
+          ~/.bun/bin/bun install --frozen-lockfile
+          ~/.bun/bin/bun run build
+
+  delivery:
+    runs-on: ubuntu-latest
+    needs:
+      - backend
+      - frontend
+    steps:
+      - uses: actions/checkout@v4
+      - name: Install Helm
+        run: |
+          mkdir -p "$HOME/.local/bin"
+          curl -fsSL https://get.helm.sh/helm-v3.15.4-linux-amd64.tar.gz -o /tmp/helm.tar.gz
+          tar -xzf /tmp/helm.tar.gz -C /tmp
+          mv /tmp/linux-amd64/helm "$HOME/.local/bin/helm"
+          echo "$HOME/.local/bin" >> "$GITHUB_PATH"
+      - name: Docker build smoke
+        run: |
+          docker build -t "$REGISTRY/$IMAGE_NAMESPACE/frontend:${GITHUB_SHA}" ./frontend
+          docker build -t "$REGISTRY/$IMAGE_NAMESPACE/backend:${GITHUB_SHA}" -f backend/Dockerfile .
+          docker build -t "$REGISTRY/$IMAGE_NAMESPACE/aiprovider:${GITHUB_SHA}" -f aiprovider/Dockerfile .
+      - name: Helm template smoke
+        run: |
+          helm lint deploy/helm/planet
+          helm template planet-staging deploy/helm/planet \
+            --namespace planet-staging \
+            -f deploy/helm/planet/values.single-node.yaml \
+            --set image.tag="${GITHUB_SHA}" >/tmp/planet-rendered.yaml

.gitea/workflows/deploy-staging.yaml

@@ -0,0 +1,67 @@
+name: deploy-staging
+
+on:
+  workflow_dispatch:
+  push:
+    branches:
+      - main
+
+env:
+  REGISTRY: gitea.rclaw.top
+  IMAGE_NAMESPACE: linkong/planet
+  RELEASE_NAME: planet-staging
+  NAMESPACE: planet-staging
+
+jobs:
+  deploy:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - name: Install deploy tools
+        run: |
+          mkdir -p "$HOME/.local/bin"
+          curl -fsSL https://get.helm.sh/helm-v3.15.4-linux-amd64.tar.gz -o /tmp/helm.tar.gz
+          tar -xzf /tmp/helm.tar.gz -C /tmp
+          mv /tmp/linux-amd64/helm "$HOME/.local/bin/helm"
+          curl -fsSL https://dl.k8s.io/release/v1.30.5/bin/linux/amd64/kubectl -o /tmp/kubectl
+          install -m 0755 /tmp/kubectl "$HOME/.local/bin/kubectl"
+          echo "$HOME/.local/bin" >> "$GITHUB_PATH"
+      - name: Configure kubeconfig
+        run: |
+          mkdir -p "$HOME/.kube"
+          printf "%s" "${{ secrets.KUBE_CONFIG_STAGING }}" | base64 -d > "$HOME/.kube/config"
+          chmod 600 "$HOME/.kube/config"
+      - name: Deploy Helm release
+        run: |
+          kubectl create namespace "$NAMESPACE" --dry-run=client -o yaml | kubectl apply -f -
+          helm upgrade --install "$RELEASE_NAME" deploy/helm/planet \
+            --namespace "$NAMESPACE" \
+            -f deploy/helm/planet/values.single-node.yaml \
+            --set global.imageRegistry="$REGISTRY" \
+            --set global.imageNamespace="$IMAGE_NAMESPACE" \
+            --set image.tag="${GITHUB_SHA}"
+      - name: Wait for rollout
+        run: |
+          kubectl rollout status deployment/planet-frontend -n "$NAMESPACE" --timeout=180s
+          kubectl rollout status deployment/planet-backend -n "$NAMESPACE" --timeout=180s
+          kubectl rollout status deployment/planet-aiprovider -n "$NAMESPACE" --timeout=180s
+      - name: Smoke test services
+        run: |
+          kubectl run planet-smoke-${GITHUB_RUN_NUMBER} \
+            --rm -i --restart=Never \
+            --namespace "$NAMESPACE" \
+            --image=curlimages/curl:8.11.1 \
+            --command -- sh -c '
+              set -eu
+              curl -fsS http://planet-frontend:3000/ >/dev/null
+              curl -fsS http://planet-frontend:3000/health >/dev/null
+              curl -fsS http://planet-frontend:3000/api/health >/dev/null
+              curl -fsS http://planet-backend:8000/health >/dev/null
+              curl -fsS http://planet-aiprovider:8010/health >/dev/null
+            '
+      - name: Collect diagnostics on failure
+        if: failure()
+        run: |
+          kubectl get all -n "$NAMESPACE" -o wide || true
+          kubectl describe pods -n "$NAMESPACE" || true
+          kubectl logs -n "$NAMESPACE" -l app.kubernetes.io/instance="$RELEASE_NAME" --all-containers --tail=200 || true

.gitea/workflows/release.yaml

@@ -0,0 +1,58 @@
+name: release
+
+on:
+  push:
+    branches:
+      - main
+    tags:
+      - "v*"
+  workflow_dispatch:
+
+env:
+  REGISTRY: gitea.rclaw.top
+  IMAGE_NAMESPACE: linkong/planet
+
+jobs:
+  images:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - name: Resolve image tags
+        id: meta
+        run: |
+          echo "sha_tag=${GITHUB_SHA}" >> "$GITHUB_OUTPUT"
+          if printf "%s" "${GITHUB_REF}" | grep -q '^refs/tags/v'; then
+            echo "release_tag=${GITHUB_REF_NAME}" >> "$GITHUB_OUTPUT"
+          else
+            echo "release_tag=" >> "$GITHUB_OUTPUT"
+          fi
+      - name: Login to registry
+        run: echo "${{ secrets.REGISTRY_PASSWORD }}" | docker login "$REGISTRY" -u "${{ secrets.REGISTRY_USER }}" --password-stdin
+      - name: Build and push images
+        run: |
+          for service in frontend backend aiprovider; do
+            case "$service" in
+              frontend)
+                dockerfile="./frontend/Dockerfile"
+                context="./frontend"
+                ;;
+              backend)
+                dockerfile="backend/Dockerfile"
+                context="."
+                ;;
+              aiprovider)
+                dockerfile="aiprovider/Dockerfile"
+                context="."
+                ;;
+            esac
+
+            image="$REGISTRY/$IMAGE_NAMESPACE/$service:${{ steps.meta.outputs.sha_tag }}"
+            docker build -t "$image" -f "$dockerfile" "$context"
+            docker push "$image"
+
+            if [ -n "${{ steps.meta.outputs.release_tag }}" ]; then
+              release_image="$REGISTRY/$IMAGE_NAMESPACE/$service:${{ steps.meta.outputs.release_tag }}"
+              docker tag "$image" "$release_image"
+              docker push "$release_image"
+            fi
+          done

.gitignore

@@ -8,6 +8,7 @@
 .env
 .env.local
 .env.*.local
+config/earth-boundary-sources.local.json
 *.pem
 *.key
 *.crt
@@ -24,11 +25,14 @@ __pycache__/
 build/
 develop-eggs/
 dist/
-downloads/
+downloads/*
+!downloads/usbipd-win/
+downloads/usbipd-win/*
+!downloads/usbipd-win/usbipd-win-5.3.0.msi
 eggs/
 .eggs/
-lib/
+/lib/
-lib64/
+/lib64/
 parts/
 sdist/
 var/
@@ -150,3 +154,9 @@ temp/
 # Runtime Data
 # ----------------------
 data/ai/bgp-briefs/
+data/earth-boundary-sources/
+
+# Generated Earth boundary tile artifacts. Keep source configs and builders in
+# Git; publish PMTiles/MVT artifacts through release/deploy storage instead of
+# committing thousands of generated tile files.
+frontend/public/earth/data/boundaries/

AGENTS.md

@@ -0,0 +1,180 @@
+# AGENTS.md
+
+**Planet agent harness. Defines behavior for coding agents working in this repository.**
+
+---
+
+## Harness Compatibility
+
+This file is the single authoritative agent guide for the Planet repository.
+The older lowercase `agents.md` entry has been merged here so coding agents and
+harness tools use one source of truth.
+
+### Source Of Truth
+
+- `rules.md` is the mandatory repository rule source. Always load `core`,
+  `security`, and `workflow`; load only task-relevant modules after that.
+- `AGENTS.md` defines the local agent operating mode and evidence gates.
+- `project_context.md` is background, not a rule source. Prefer newer
+  implementation docs when it disagrees with current code.
+- `.codex/skills/` is the active specialized workflow layer for cleanup, docs,
+  goal-driven work, and release.
+- Do not duplicate long workflow text across harness files. Durable constraints
+  belong in `rules.md`; task procedures belong in skills or scripts.
+
+Read these files before changing code:
+
+1. `rules.md`
+2. `AGENTS.md`
+3. `project_context.md`
+4. `README.md`
+5. `docs/HARNESS.md`
+6. `CODEMAP.md`
+
+For documentation work, also read `docs/documentation-coverage-rules.md`.
+
+### Start Safely
+
+Before broad edits:
+
+```bash
+git status --short
+scripts/harness/doctor.sh
+```
+
+Use focused context commands before reading large files:
+
+```bash
+rg -n "<symbol-or-term>" <path>
+git diff --stat HEAD
+git diff --name-only HEAD
+git diff --unified=0 HEAD -- <path>
+```
+
+Preserve user changes already present in the worktree.
+
+### Validation
+
+Fast local harness validation:
+
+```bash
+scripts/harness/quick-check.sh
+```
+
+Full local validation:
+
+```bash
+scripts/harness/validate.sh
+```
+
+`validate.sh` includes quick checks, frontend Bun build, and frontend smoke
+unless disabled by its documented environment flags. Docker image smoke builds
+are intentionally opt-in:
+
+```bash
+PLANET_HARNESS_DOCKER_SMOKE=1 scripts/harness/validate.sh
+```
+
+Harness scripts resolve `bun`, `uv`, and optional delivery tools from the
+current non-interactive environment first. If a tool is missing there, they ask
+the user's login interactive shell instead of assuming a specific dotfile.
+
+### High-Risk Areas
+
+- `planet.sh` owns local lifecycle, ports, WSL/LAN behavior, and destructive
+  `destroy` cleanup.
+- Frontend package management is Bun-only. Do not use npm, pnpm, or yarn.
+- Frontend changes must satisfy `scripts/harness/frontend-rules-check.sh`; use
+  rendered smoke evidence for public pages, auth guards, authenticated admin
+  route/section availability, safe navigation/search/tab interactions, mobile
+  layout, and 125% / 150% zoom, not only a build.
+- Admin or Docs layout changes must load `rules.md` `uiux` and preserve the
+  one-screen (`一屏` / `首屏`) height chain: route roots use `height: 100%`,
+  intermediate wrappers keep `min-height: 0`, and only the intended child owns
+  scrolling.
+- `aiprovider` is a protocol/provider adapter; keep business prompts and product
+  workflows in the backend.
+- Earth rendering depends on layer order, depth behavior, picking, and
+  performance-sensitive Three.js code.
+- Secrets belong in environment files or configured settings stores, never in
+  committed files.
+- Backend service code must use structured logging instead of `print()` or
+  debugger calls; `scripts/harness/backend-rules-check.sh` enforces this.
+
+### Conflict Policy
+
+Existing project rules and workflows win. If new harness guidance conflicts with
+`rules.md`, `AGENTS.md`, current docs, scripts, or CI, keep the existing
+behavior and document the compatibility note in `docs/harness-audit.md` or
+`docs/HARNESS.md`.
+
+---
+
+## Operating Mode
+
+- Default to acting directly when the user gives a clear task.
+- Ask before acting only when the missing decision is risky, cannot be
+  discovered from repository context, and no conservative assumption is safe.
+- Read relevant files before editing.
+- Prefer focused CLI evidence: `rg`, `git diff --stat`, `git diff --name-only`,
+  focused file reads, tests, builds, linters, and harness scripts.
+- Keep changes scoped to the requested area. Do not mix cleanup, feature work,
+  release work, and documentation unless the task requires it.
+
+---
+
+## Evidence Gates
+
+- Visual inputs are blocking evidence. If the user provides a screenshot, image,
+  mock, browser capture, or visual reference, obtain evidence from the artifact
+  before interpreting intent or editing code.
+- Path resolution is part of the task. If the path cannot be opened, first try
+  reasonable local equivalents such as WSL/Windows path conversion,
+  workspace-relative lookup, absolute paths, and attached-file locations.
+- Never guess from prompt text, filenames, previous context, logs, OCR, or
+  memory when a visual artifact was provided but cannot be accessed.
+- OCR is acceptable evidence for text-only visual questions or non-multimodal
+  environments; state that OCR was used as the fallback. Layout, color, spacing,
+  pixel, and rendering issues need real visual inspection or a clear limitation
+  note.
+- If a visual artifact still cannot be inspected, say so and pause that
+  visual-dependent part of the work.
+- Claims of completion need evidence: a relevant test, build, lint, screenshot,
+  diff, direct file check, or harness result.
+- For UI and rendering changes, verify the rendered result when local tooling
+  allows it.
+
+---
+
+## Communication
+
+- Match the user's language. Use Chinese for Chinese requests unless the user
+  asks otherwise.
+- Keep updates short and specific: what is being inspected, edited, or verified.
+- Final responses should summarize changed files and verification, with blockers
+  stated plainly.
+- Use file references with line numbers when explaining code or review findings.
+
+---
+
+## Quality Bar
+
+- Prefer existing project patterns over new abstractions.
+- Remove stale branches, mocks, compatibility paths, and duplicated helpers once
+  a stable path exists.
+- Centralize prompts, constants, defaults, and shared request/response handling.
+- Do not add secrets, generated runtime output, or local environment files.
+- Frontend commands use Bun only. Do not use `npm`, `pnpm`, or `yarn`.
+- Run the smallest relevant verification for the changed scope and report
+  anything skipped.
+
+---
+
+## Prohibited
+
+- Do not skip visual evidence handling when a visual artifact was provided.
+- Do not preserve obsolete harness files just because they already exist.
+- Do not invent behavior not present in code, docs, or verified external
+  sources.
+- Do not rewrite unrelated files during cleanup.
+- Do not mark a task complete without checking concrete success criteria.

CODEMAP.md

@@ -0,0 +1,109 @@
+# Code Map
+
+This map gives agents and maintainers a quick orientation without replacing the
+deeper architecture docs. Current implementation docs under `docs/technical/`
+are the source of detail for specific subsystems.
+
+## Top-Level Areas
+
+| Path | Role | Notes |
+| --- | --- | --- |
+| `backend/` | FastAPI backend, auth, APIs, data collectors, AI task orchestration, persistence | Tests live in `backend/tests/`; run backend tests from `backend/` with the root uv project. |
+| `frontend/` | React admin console, Docs UI, Web Earth shell, Vite build | Use Bun only. Public Earth assets live under `frontend/public/earth/`. |
+| `aiprovider/` | Model provider/protocol adapter service | Keep it free of product-specific prompts and workflows. |
+| `motion_agent/` | Motion capture protocol service used by `planet.sh` | Often dry-runs when cameras are unavailable, especially in WSL. |
+| `scripts/` | Utility scripts and harness wrappers | Harness commands live in `scripts/harness/`. |
+| `docs/` | Plans, technical docs, changelog, harness docs | Public technical docs are explicitly registered by the frontend Docs catalog. |
+| `deploy/helm/planet/` | Helm chart for staging/deployment smoke paths | CI runs helm lint/template when delivery checks are available. |
+| `.gitea/workflows/` | CI, release image build, staging deploy workflows | This repository uses Gitea workflow files, not `.github/workflows/`. |
+| `planet.sh` | Main local lifecycle script | Owns init/start/restart/stop/health/log/createuser/destroy. |
+
+## Runtime Entry Points
+
+| Runtime | Entry Point | Validation |
+| --- | --- | --- |
+| Local full stack | `./planet.sh start` | `./planet.sh health` |
+| Backend API | `backend/app/main.py` | `cd backend && uv run --frozen --group dev --project .. python -m pytest -q` |
+| Frontend app | `frontend/src/main.tsx` and `frontend/vite.config.mts` | `cd frontend && bun run build` |
+| AI Provider | `aiprovider/main.py` | `curl http://localhost:8010/health` after startup |
+| Motion Agent | `python -m motion_agent` via `planet.sh` | `./planet.sh health` or dry-run startup |
+| Docs UI | `frontend/src/pages/Docs/` | Docs catalog metadata plus frontend build |
+
+## Ownership Boundaries
+
+- Backend owns business state, auth, evidence collection, prompt selection, AI
+  task orchestration, and database persistence.
+- `aiprovider` owns provider identity, request adapter style, model gateway
+  retries, and health/status endpoints only.
+- Frontend owns operator workflows, Docs presentation, Web Earth orchestration,
+  and client-side state that mirrors backend truth.
+- Web Earth rendering changes must preserve documented layer order, altitude
+  offsets, picking behavior, legend semantics, and performance constraints.
+- `planet.sh` owns local environment bootstrap and service lifecycle. Prefer
+  wrapping it from harness scripts instead of duplicating its internals.
+- Harness scripts source `scripts/harness/lib.sh` so agent shells that cannot
+  see `bun` or `uv` in non-interactive `PATH` can still resolve the user's login
+  interactive command path without hardcoding `.zshrc`.
+
+## Validation Commands
+
+```bash
+scripts/harness/doctor.sh
+scripts/harness/security-check.sh
+scripts/harness/backend-rules-check.sh
+scripts/harness/frontend-rules-check.sh
+scripts/harness/docs-consistency-check.sh
+scripts/harness/quick-check.sh
+scripts/harness/validate.sh
+./planet.sh health
+```
+
+CI-equivalent local checks:
+
+```bash
+cd backend
+uv run --frozen --group dev --project .. python -m pytest -s tests/test_api.py tests/test_realtime_sources.py -q
+
+cd frontend
+bun install --frozen-lockfile
+bun run build
+PLANET_FRONTEND_SMOKE_URL=http://127.0.0.1:4173 bun ../scripts/harness/frontend-smoke.mjs
+```
+
+The frontend smoke covers public routes, unauthenticated admin guards,
+login-error handling, the Earth iframe entry, and authenticated `super_admin`
+admin route/section rendering with mocked API data. Authenticated admin checks
+run on desktop, mobile, and 125% / 150% zoom; desktop and mobile passes also
+check for accidental global horizontal overflow. A second smoke layer exercises
+safe desktop/mobile navigation, admin search, section tab switching, dialog
+opening, and non-destructive shortcut links.
+
+Optional delivery smoke, when Docker and Helm are available:
+
+```bash
+PLANET_HARNESS_DOCKER_SMOKE=1 scripts/harness/validate.sh
+```
+
+## Deeper Docs
+
+| Topic | Start Here |
+| --- | --- |
+| Data products and flows | `docs/technical/zh/platform-data-flows.md` and `docs/technical/en/platform-data-flows.md` |
+| Operations and local lifecycle | `docs/technical/zh/ops-runbook.md` and `docs/technical/en/ops-runbook.md` |
+| `planet.sh` startup behavior | `docs/technical/zh/ops-planet-sh-startup.md` and `docs/technical/en/ops-planet-sh-startup.md` |
+| AI Provider | `docs/technical/zh/agents-aiprovider.md` and `docs/technical/en/agents-aiprovider.md` |
+| Admin frontend | `docs/technical/zh/frontend-admin-frontend-context.md` and `docs/technical/en/frontend-admin-frontend-context.md` |
+| Earth frontend | `docs/technical/zh/earth-frontend-context.md` and `docs/technical/en/earth-frontend-context.md` |
+| Earth render order | `docs/technical/zh/earth-render-layer-order.md` and `docs/technical/en/earth-render-layer-order.md` |
+| Documentation rules | `docs/documentation-coverage-rules.md` |
+| Harness workflow | `docs/HARNESS.md` |
+
+## Known Sharp Edges
+
+- `project_context.md` is static background for agents. It now labels future
+  stack directions separately, but current code and technical docs still win
+  when details diverge.
+- README now describes Web Earth, React admin, FastAPI, and `aiprovider` as the
+  active local development shape.
+- Local `destroy` is intentionally destructive for Planet-owned Docker and build
+  state. Never run it as a validation shortcut.

README.md

@@ -8,68 +8,54 @@
 
 ## 系统架构
 
+当前仓库的核心形态是“Web Earth 可视化 + React 运维台 + FastAPI 数据与 AI 编排后端 + 独立模型适配层”。物理大屏与 UE 客户端仍是长期方向，但不再作为本地开发和当前发布的必需运行单元。
+
 ```
-┌─────────────────────────────────────────────────────────────────────────┐
+┌─────────────────────────────────────────────────────────────────────┐
-│                           物理大屏展示层                                  │
+│                         浏览器展示与运维层                            │
-│  ┌─────────────────────────────────────────────────────────────────┐    │
+│  ┌──────────────────────────────┐  ┌──────────────────────────────┐ │
-│  │         偏振片3D大屏 (2m×3m, 4K, 120Hz, 眼镜式)                 │    │
+│  │ Web Earth                    │  │ React 运维台                  │ │
-│  │  ┌─────────────────────────────────────────────────────────┐    │    │
+│  │ frontend/public/earth        │  │ frontend/src                  │ │
-│  │  │              虚幻引擎 UE5 客户端                          │    │    │
+│  │ Three.js 地球 / HUD / 新闻    │  │ 数据源 / 告警 / AI 设置       │ │
-│  │  │  ├── 3D地球渲染 (Cesium for UE)                          │    │    │
+│  │ 国界精度 / 品牌内容配置       │  │ 提示词配置 / 用户与系统配置   │ │
-│  │  │  ├── 算力点可视化 (GPU集群、智算中心)                     │    │    │
+│  └──────────────────────────────┘  └──────────────────────────────┘ │
-│  │  │  ├── 连接弧线 (光缆、路由、数据流向)                      │    │    │
+└─────────────────────────────────────────────────────────────────────┘
-│  │  │  ├── 粒子效果 (数据流动、告警提示)                        │    │    │
+                       │ REST / WebSocket
-│  │  │  └── 自动巡航相机 + 交互控制                              │    │    │
+                       ▼
-│  │  └─────────────────────────────────────────────────────────┘    │    │
+┌─────────────────────────────────────────────────────────────────────┐
-│  └─────────────────────────────────────────────────────────────────┘    │
+│                       FastAPI 业务与编排后端                          │
-└─────────────────────────────────────────────────────────────────────────┘
+│  ┌────────────────────┐ ┌────────────────────┐ ┌─────────────────┐ │
-                                    ▲
+│  │ 数据 API 与认证     │ │ Earth 新闻增强      │ │ 告警与态势简报   │ │
-                                    │ WebSocket (实时推送)
+│  │ JWT / 权限 / 审计   │ │ 位置推断 / 本地化   │ │ BGP / 告警研判   │ │
-                                    │ 120Hz 心跳 / 数据帧同步
+│  └────────────────────┘ └────────────────────┘ └─────────────────┘ │
-                                    ▼
+│  ┌────────────────────┐ ┌────────────────────┐ ┌─────────────────┐ │
-┌─────────────────────────────────────────────────────────────────────────┐
+│  │ 系统运行配置        │ │ 默认提示词注册表     │ │ 未来 Agent Runtime│ │
-│                           数据中台服务层 (FastAPI)                        │
+│  │ system_settings     │ │ 代码发布 + DB 覆盖  │ │ 工具/证据/工作流   │ │
-│  ┌─────────────────────────────────────────────────────────────────┐    │
+│  └────────────────────┘ └────────────────────┘ └─────────────────┘ │
-│  │                      API Gateway (Redis 限流)                    │    │
+└─────────────────────────────────────────────────────────────────────┘
-│  └─────────────────────────────────────────────────────────────────┘    │
+        │ SQLAlchemy / Redis Stream            │ 纯净 LLM 调用
-│                                    │                                    │
+        ▼                                      ▼
-│  ┌───────────────────┬──────────────────────────┬──────────────────┐   │
+┌──────────────────────────────┐      ┌──────────────────────────────┐
-│  │    数据采集服务    │      核心业务服务         │    运维管理服务   │   │
+│ PostgreSQL / Redis            │      │ aiprovider                   │
-│  │  ┌─────────────┐  │  ┌─────────────────┐    │  ┌─────────────┐ │   │
+│ 用户、配置、采集结果、新闻     │      │ provider + protocol adapter  │
-│  │  │ 调度中心    │  │  │ WebSocket 服务  │    │  │ 用户管理    │ │   │
+│ Stream、缓存、运行状态         │      │ OpenAI / MiniMax / Ollama 等 │
-│  │  │ (Celery)   │  │  │ (FastAPI)       │    │  │ (JWT Auth)  │ │   │
+└──────────────────────────────┘      └──────────────────────────────┘
-│  │  └─────────────┘  │  └─────────────────┘    │  └─────────────┘ │   │
+        ▲
-│  │  ┌─────────────┐  │  ┌─────────────────┐    │  ┌─────────────┐ │   │
+        │ 采集器 / 外部数据源
-│  │  │ 采集器池    │  │  │ 数据查询 API    │    │  │ 数据源配置  │ │   │
+        ▼
-│  │  │ (10+源)    │  │  │ (REST)          │    │  │ 监控告警    │ │   │
+┌─────────────────────────────────────────────────────────────────────┐
-│  │  └─────────────┘  │  └─────────────────┘    │  └─────────────┘ │   │
+│ RSS 新闻、BGP 观测、公开数据源、后续 WebSearch/OCR/语音识别等工具      │
-│  │  ┌─────────────┐  │  ┌─────────────────┐    │  ┌─────────────┐ │   │
+└─────────────────────────────────────────────────────────────────────┘
-│  │  │ 消息队列    │  │  │ 态势分析引擎    │    │  │ 系统配置    │ │   │
-│  │  │ (Kafka)    │  │  │ (计算/聚合)     │    │  │ 日志审计    │ │   │
-│  │  └─────────────┘  │  └─────────────────┘    │  └─────────────┘ │   │
-│  └───────────────────┴──────────────────────────┴──────────────────┘   │
-└─────────────────────────────────────────────────────────────────────────┘
-                                    ▲
-                                    │ 内部 API 调用
-                                    ▼
-┌─────────────────────────────────────────────────────────────────────────┐
-│                           Web管理端 (React Admin)                        │
-│  ┌─────────────────────────────────────────────────────────────────┐    │
-│  │  登录页 │ 仪表盘 │ 用户管理 │ 数据源配置 │ 任务监控 │ 系统配置   │    │
-│  └─────────────────────────────────────────────────────────────────┘    │
-└─────────────────────────────────────────────────────────────────────────┘
-                                    ▲
-                                    │ PostgreSQL / Redis
-                                    ▼
-┌─────────────────────────────────────────────────────────────────────────┐
-│                           数据存储层                                      │
-│  ┌─────────────┐  ┌─────────────┐  ┌─────────────┐  ┌─────────────┐    │
-│  │ PostgreSQL  │  │ TimescaleDB │  │    Redis    │  │  MinIO      │    │
-│  │ (用户/配置) │  │ (时序数据)  │  │ (缓存/会话) │  │ (文件存储) │    │
-│  └─────────────┘  └─────────────┘  └─────────────┘  └─────────────┘    │
-└─────────────────────────────────────────────────────────────────────────┘
 ```
 
+架构边界:
+
+- `backend` 负责业务语义、证据收集、提示词选择、AI 任务编排、权限和数据落库。
+- `aiprovider` 只负责把纯净模型请求适配到不同供应商或协议，不内置具体业务提示词。
+- 默认提示词随代码发布并保存在 `backend/app/ai_tasks/default_prompts.json`，运维台可在数据库中保存覆盖值，重置时回到当前代码版本的默认提示词。
+- Earth 新闻保留英文原文，中文展示结果存入 `localizations`，前端默认展示 `zh-CN` 的 `display_title`、`display_summary` 和中文地域/状态文案。
+- Earth LLM 指令、语音识别、多角色态势研判属于后续 Agent Runtime 方向，计划见 [docs/plans/agents-earth-command-runtime-plan.md](/home/ray/dev/linkong/planet/docs/plans/agents-earth-command-runtime-plan.md)。
+
 ## 四大核心要素
 
 | 层级 | 要素 | 描述 |
@@ -87,21 +73,21 @@
 |------|------|------|
 | FastAPI | 0.109+ | Web 框架 |
 | SQLAlchemy | 2.0+ | ORM |
-| Alembic | - | 数据库迁移 |
+| uv | - | Python 依赖与命令运行 |
-| Celery | 5.3+ | 任务队列 |
+| Redis | 7.0+ | 缓存、Stream 与运行协调 |
-| Redis | 7.0+ | 缓存/消息 |
-| Kafka | 3.0+ | 事件流 |
 | PyJWT | - | 认证 |
+| APScheduler / 后台任务 | - | 采集、增强与运行时任务 |
 
 ### 前端 (React Admin)
 
 | 组件 | 用途 |
 |------|------|
 | React 18 | UI 框架 |
-| Ant Design Pro | 管理后台组件 |
+| Tactile UI / Radix primitives / lucide-react | 管理后台组件、基础交互与图标 |
 | Axios | HTTP 客户端 |
 | Socket.io-client | WebSocket 客户端 |
 | ECharts | 统计图表 |
+| Three.js | Earth 3D 地球渲染 |
 | Bun | 前端包管理与脚本运行 |
 
 前端工程统一使用 Bun：
@@ -110,22 +96,16 @@
 - 运行脚本使用 `bun run <script>`
 - 不使用 `npm`、`pnpm`、`yarn`
 
-### 虚幻引擎客户端
+### 大屏与 3D 展示方向
 
-| 组件 | 版本 | 用途 |
+当前发布优先使用浏览器 Web Earth。UE5 / Cesium for Unreal / Niagara 可作为后续物理大屏方向接入，但不是本地开发闭环的必需组件。
-|------|------|------|
-| Unreal Engine 5 | 5.3+ | 3D 渲染引擎 |
-| Cesium for Unreal | 1.5+ | 地理可视化 |
-| Niagara | - | 粒子系统 |
 
 ### 数据库
 
 | 组件 | 用途 |
 |------|------|
 | PostgreSQL 15+ | 关系数据 |
-| TimescaleDB | 时序数据扩展 |
+| Redis 7+ | 缓存、Stream、运行状态 |
-| Redis 7+ | 缓存/会话 |
-| MinIO | S3 兼容存储 |
 
 ### 部署
 
@@ -152,9 +132,9 @@
 | P0 | Epoch AI | 每小时 |
 | P0 | Hugging Face | 每 2 小时 |
 | P0 | GitHub | 每 4 小时 |
-| P0 每日 |
+| P0 | 海底光缆 / IXP / 卫星等基础设施数据 | 每日或按源刷新 |
 | P0 | PeeringDB | 每 2 小时 |
-| P1 | Cloudflare Radar | | TeleGeography | 每小时 |
+| P1 | Cloudflare Radar / TeleGeography | 每小时 |
 | P1 | CAIDA BGPStream | 每 15 分钟 |
 
 ## 项目结构
@@ -166,20 +146,18 @@
 │   │   ├── core/         # 核心配置
 │   │   ├── models/       # 数据模型
 │   │   ├── schemas/      # Pydantic 模型
-│   │   ├── services/     # 业务逻辑
+│   │   ├── services/     # 业务逻辑与 AI 任务编排
-│   │   └── tasks/        # Celery 任务
+│   │   └── ai_tasks/     # 默认提示词与 AI 任务定义
 │   └── tests/
+├── aiprovider/            # 独立模型供应商适配层
 ├── frontend/              # React 管理后台
 │   ├── src/
 │   │   ├── components/   # 组件
 │   │   ├── pages/        # 页面
 │   │   ├── services/     # API 服务
 │   │   └── store/        # 状态管理
-│   └── tests/
+│   ├── public/earth/      # Web Earth 静态应用
-├── unreal/                # UE5 大屏客户端
+│   └── tests/             # 前端测试
-│   ├── Content/
-│   ├── Source/
-│   └── Plugins/
 ├── data/                  # 数据文件
 ├── docs/                  # 文档
 ├── scripts/               # 脚本
@@ -191,10 +169,11 @@
 ## 快速启动
 
 ```bash
-# 新机器首次初始化
+# 新机器或空项目首次初始化
-./scripts/bootstrap-dev.sh
+./planet.sh init
-# 会自动安装/检查 uv、bun，并同步 Python/前端依赖
+# 会自动安装/检查 uv、bun，同步 Python/前端依赖
 # 会在缺少时生成 backend/.env、aiprovider/.env、frontend/.env.local
+# 会启动 PostgreSQL/Redis，并创建表、默认数据源和本地默认用户
 
 # 启动前后端服务
 ./planet.sh start
@@ -210,6 +189,9 @@
 
 # 查看服务状态
 ./planet.sh health
+
+# 删除容器、卷、镜像和本地编译状态，执行前需要输入 Y 确认
+./planet.sh destroy
 ```
 
 前端命令约定：
@@ -227,6 +209,117 @@ bun run build
 
 启动服务后访问: `http://localhost:8000/docs`
 
+## WSL / Windows 局域网访问
+
+如果服务运行在 WSL 中，而你希望：
+
+- Windows 本机浏览器访问开发服务
+- 同一局域网内的手机或其他电脑访问开发服务
+
+推荐按下面顺序排查和配置。
+
+端口占用、`iphlpsvc` / portproxy、摄像头和依赖问题的集中排障入口见 [常见问题](/home/ray/dev/linkong/planet/docs/technical/zh/faq.md)。
+
+### 1. 在 WSL 中启动服务
+
+```bash
+./planet.sh start --allow-lan
+```
+
+这会让前端监听 `0.0.0.0:3000`，后端监听 `0.0.0.0:8000`，AI Provider 通过 Docker 发布到 `0.0.0.0:8010`。启动前脚本会检查这三个端口；如果 WSL/Linux 侧无法释放端口，并检测到 Windows 侧 listener 或旧 `portproxy`，会请求管理员 PowerShell 清理。
+
+### 2. 先确认 WSL 内部服务正常
+
+在 WSL 中执行：
+
+```bash
+curl http://localhost:3000
+curl http://localhost:8000/health
+curl http://localhost:8010/health
+ss -ltnp | grep -E ':3000|:8000|:8010'
+```
+
+预期：
+
+- `3000` 返回前端 HTML
+- `8000/health` 返回健康检查 JSON
+- `8010/health` 返回 AI Provider 健康检查 JSON
+- `ss` 中能看到 `0.0.0.0:3000`、`0.0.0.0:8000` 和 `0.0.0.0:8010`，或 Docker 已发布 `8010`
+
+如果这一步不通，先不要继续做 Windows 转发。
+
+### 3. 在 Windows 本机验证 localhost 直通
+
+在 Windows PowerShell 中执行：
+
+```powershell
+curl http://localhost:3000
+curl http://localhost:8000/health
+curl http://localhost:8010/health
+```
+
+在常见的 WSL2 开发环境下，Windows 通常可以直接通过 `localhost` 访问 WSL 中的服务。
+
+### 4. 如果需要让局域网设备访问，清理端口和防火墙
+
+`./planet.sh start --allow-lan` 不再启动额外的 Windows 端口转发进程。它直接让开发服务对 `3000` / `8000` / `8010` 开放，并在启动前尝试释放这些端口。端口被 Windows 侧 listener 或旧 `portproxy` 占用时，脚本会请求一次管理员 PowerShell 清理。
+
+如果以前手动配置过持久 `portproxy`，若自动请求被取消，可以手动清理，避免 `iphlpsvc` 继续占用端口：
+
+```powershell
+netsh interface portproxy delete v4tov4 listenaddress=0.0.0.0 listenport=3000
+netsh interface portproxy delete v4tov4 listenaddress=0.0.0.0 listenport=8000
+netsh interface portproxy delete v4tov4 listenaddress=0.0.0.0 listenport=8010
+```
+
+脚本会检测 Windows 防火墙是否已放行 `3000` / `8000` / `8010`。如果缺少规则，会触发一次 Windows UAC 管理员 PowerShell 请求来自动创建。若自动请求被取消，也可以手动执行：
+
+```powershell
+New-NetFirewallRule -DisplayName "WSL Planet 3000" -Direction Inbound -Action Allow -Protocol TCP -LocalPort 3000
+New-NetFirewallRule -DisplayName "WSL Planet 8000" -Direction Inbound -Action Allow -Protocol TCP -LocalPort 8000
+New-NetFirewallRule -DisplayName "WSL Planet 8010" -Direction Inbound -Action Allow -Protocol TCP -LocalPort 8010
+```
+
+### 5. 查 Windows 局域网 IP，并让其他设备访问
+
+在 Windows PowerShell 中执行：
+
+```powershell
+ipconfig
+```
+
+找到当前联网网卡的 IPv4 地址，例如 `192.168.8.228`。
+
+局域网其他设备可访问：
+
+- `http://<Windows局域网IP>:3000/earth`
+- `http://<Windows局域网IP>:3000/admin`
+- `http://<Windows局域网IP>:8000/health`
+- `http://<Windows局域网IP>:8010/health`
+
+例如：
+
+- `http://192.168.8.228:3000/earth`
+
+### 6. 常见现象与判断
+
+- WSL 中 `curl localhost:3000` 能通，但 Windows 访问 `WSL 的局域网 IP:3000` 不通：这是正常现象之一，优先验证 Windows 的 `localhost:3000`
+- Windows `localhost:3000` 能通，但局域网设备访问 `Windows 局域网 IP:3000` 不通：通常是 Windows 防火墙、网络配置或旧 `portproxy` 残留
+- `whoami /groups` 中 `S-1-5-32-544` 显示 `deny only`：说明当前 PowerShell 不是提权管理员窗口
+
+### 7. 本项目一次性验证顺序
+
+建议固定按这个顺序验证：
+
+1. WSL 中执行 `curl http://localhost:3000`
+2. WSL 中执行 `curl http://localhost:8000/health`
+3. WSL 中执行 `curl http://localhost:8010/health`
+4. Windows 中执行 `curl http://localhost:3000`
+5. Windows 中执行 `curl http://localhost:8000/health`
+6. Windows 中执行 `curl http://localhost:8010/health`
+7. 按脚本提示完成 Windows 防火墙或端口清理 UAC 请求
+8. 用手机或其他电脑访问 Windows 对外端口，例如 `http://<Windows局域网IP>:3000/earth`
+
 ## 启动容错参数
 
 `planet.sh` 现在为依赖安装、数据库、AI Provider 启动加入了有限次重试，并会在数据库与 `aiprovider` 启动后额外等待 Docker healthcheck。
@@ -249,18 +342,27 @@ DATABASE_RETRY_INTERVAL=10 \
 - `AI_PROVIDER_START_MAX_RETRIES` / `AI_PROVIDER_RETRY_INTERVAL`: 控制 `aiprovider` 的构建/启动与容器重启自愈，默认 `3` 次、`5` 秒
 - `BACKEND_MAX_RETRIES`: 控制后端进程启动重试次数，默认 `3`
 - `FRONTEND_MAX_RETRIES`: 控制前端 dev server 启动重试次数，默认 `3`
-- `BACKEND_HEALTH_CHECK_ATTEMPTS` / `BACKEND_HEALTH_CHECK_INTERVAL`: 控制后端 HTTP 健康检查等待次数与间隔，默认 `10` 次、`2` 秒
+- `BACKEND_HEALTH_CHECK_ATTEMPTS` / `BACKEND_HEALTH_CHECK_INTERVAL`: 控制后端 HTTP 健康检查等待次数与间隔，默认 `60` 次、`2` 秒
 - `FRONTEND_HEALTH_CHECK_ATTEMPTS` / `FRONTEND_HEALTH_CHECK_INTERVAL`: 控制前端 HTTP 可访问检查等待次数与间隔，默认 `10` 次、`2` 秒
 - `AI_PROVIDER_HEALTH_CHECK_ATTEMPTS` / `AI_PROVIDER_HEALTH_CHECK_INTERVAL`: 控制 `aiprovider` HTTP 健康检查等待次数与间隔，默认 `10` 次、`2` 秒
 
-## AI 接口预留
+## AI 与智能体接口
 
-项目现在采用“两层”设计:
+项目现在采用“三段式”边界:
 
-- 主后端暴露稳定业务接口: `GET /api/v1/ai/provider/status`、`POST /api/v1/ai/situational-awareness/analyze`
+- `backend`: 暴露业务接口，负责选择任务提示词、组织证据、调用工具、保存 AI 设置和结果。
-- 独立 `aiprovider` 服务负责适配具体模型供应商
+- `aiprovider`: 暴露模型网关接口，只负责 provider / protocol 适配，不写入 BGP、新闻、告警等业务提示词。
+- 模型供应商: OpenAI 兼容、MiniMax、Anthropic、Ollama 或其他兼容网关。
 
-这样前端和业务代码不直接依赖 OpenAI、本地模型网关或其他订阅服务，后续切换部署方式只需要调整环境变量。
+这样前端和业务代码不直接依赖某个模型供应商，后续增加 Agent Runtime、Earth 一键 LLM 指令、语音识别或多角色态势研判时，也可以把业务工作流放在后端，而不是污染模型适配层。
+
+当前已落地的 AI 配置能力:
+
+- 运维台 AI 设置可维护 provider、模型、协议、超时、token 等运行配置。
+- 运维台 AI 设置中的“提示词”页可选择不同功能入口，手动覆盖提示词，并一键重置到默认值。
+- 默认提示词随代码发布，位于 [backend/app/ai_tasks/default_prompts.json](/home/ray/dev/linkong/planet/backend/app/ai_tasks/default_prompts.json)。
+- 覆盖值保存在数据库运行配置中，升级代码后可继续保留现场配置，也可重置到新版本默认提示词。
+- 态势摘要、告警研判、新闻本地化等入口应使用各自任务提示词；调用 `aiprovider` 时只传递当前任务所需的 `prompt` / `system_prompt`。
 
 主后端建议配置:
 
@@ -273,39 +375,33 @@ AI_PROVIDER_TIMEOUT_SECONDS=60
 `aiprovider` 服务建议配置:
 
 ```env
-AI_PROVIDER=openai_compatible
+AI_PROVIDER=minimax
-AI_BASE_URL=https://api.openai.com/v1
+AI_PROVIDER_API=anthropic-messages
+AI_BASE_URL=https://api.minimaxi.com/anthropic
 AI_API_KEY=your_api_key
-AI_MODEL=gpt-4o-mini
+AI_MODEL=MiniMax-M2.7
 AI_TIMEOUT_SECONDS=60
 AI_PROVIDER_SERVICE_TOKEN=change_me
 ```
 
-OpenAI 兼容场景推荐使用:
+推荐映射关系:
 
-- `AI_PROVIDER=openai_compatible`
+- `vLLM` / `LM Studio` / `One API`: `AI_PROVIDER=openai` + `AI_PROVIDER_API=openai-completions`
+- `MiniMax`: `AI_PROVIDER=minimax` + `AI_PROVIDER_API=anthropic-messages`
+- Claude 兼容网关: `AI_PROVIDER=anthropic` + `AI_PROVIDER_API=anthropic-messages`
+- `Ollama`: `AI_PROVIDER=ollama` + `AI_PROVIDER_API=ollama-generate`
 
-Claude 兼容场景推荐使用:
+比如 MiniMax 可以这样配置:
-
-- `AI_PROVIDER=anthropic`
-- `AI_PROVIDER=anthropic_compatible`
-- `AI_PROVIDER=claude_compatible`
-
-Ollama 原生场景推荐使用:
-
-- `AI_PROVIDER=ollama`
-
-比如 MiniMax 或其他 Claude 兼容网关，可以这样配置:
 
 ```env
-AI_PROVIDER=claude_compatible
+AI_PROVIDER=minimax
-AI_BASE_URL=https://your-claude-compatible-endpoint.example.com
+AI_PROVIDER_API=anthropic-messages
+AI_BASE_URL=https://api.minimaxi.com/anthropic
 AI_API_KEY=your_api_key
-AI_MODEL=your-claude-compatible-model
+AI_MODEL=MiniMax-M2.7
 AI_TIMEOUT_SECONDS=60
 AI_MAX_TOKENS=1200
 AI_ANTHROPIC_VERSION=2023-06-01
-AI_PROVIDER_SERVICE_TOKEN=change_me
 ```
 
 如果你要本地直接起模型适配层，项目里已经补了模板:
@@ -313,12 +409,6 @@ AI_PROVIDER_SERVICE_TOKEN=change_me
 - [aiprovider/.env.example](/home/ray/dev/linkong/planet/aiprovider/.env.example)
 - [docker-compose.local-model.yml](/home/ray/dev/linkong/planet/docker-compose.local-model.yml)
 
-推荐映射关系:
-
-- `vLLM` / `LM Studio` / `One API`: `AI_PROVIDER=openai_compatible`
-- `MiniMax` / Claude 兼容网关: `AI_PROVIDER=claude_compatible`
-- `Ollama`: `AI_PROVIDER=ollama`
-
 运行与调用补充:
 
 - `./planet.sh start` 默认会启动 `aiprovider`
@@ -328,11 +418,13 @@ AI_PROVIDER_SERVICE_TOKEN=change_me
 
 详细文档:
 
-- [docs/agents/aiprovider.md](/home/ray/dev/linkong/planet/docs/agents/aiprovider.md)
+- [docs/technical/zh/agents-aiprovider.md](/home/ray/dev/linkong/planet/docs/technical/zh/agents-aiprovider.md)
+- [docs/technical/en/agents-aiprovider.md](/home/ray/dev/linkong/planet/docs/technical/en/agents-aiprovider.md)
 - [aiprovider/README.md](/home/ray/dev/linkong/planet/aiprovider/README.md)
-- [docs/frontend/frontend-layout-guidelines.md](/home/ray/dev/linkong/planet/docs/frontend/frontend-layout-guidelines.md)
+- [docs/technical/frontend-layout-guidelines.md](/home/ray/dev/linkong/planet/docs/technical/frontend-layout-guidelines.md)
-- [docs/frontend/ai-playground-development-plan.md](/home/ray/dev/linkong/planet/docs/frontend/ai-playground-development-plan.md)
+- [docs/plans/frontend-ai-playground-development-plan.md](/home/ray/dev/linkong/planet/docs/plans/frontend-ai-playground-development-plan.md)
-- [docs/agents/situational-awareness-foundation-plan.md](/home/ray/dev/linkong/planet/docs/agents/situational-awareness-foundation-plan.md)
+- [docs/plans/agents-situational-awareness-foundation-plan.md](/home/ray/dev/linkong/planet/docs/plans/agents-situational-awareness-foundation-plan.md)
+- [docs/plans/agents-earth-command-runtime-plan.md](/home/ray/dev/linkong/planet/docs/plans/agents-earth-command-runtime-plan.md)
 
 ## 前端页面布局规范
 
@@ -346,7 +438,7 @@ AI_PROVIDER_SERVICE_TOKEN=change_me
 当前推荐参考实现：
 
 - [frontend/src/pages/BGP/BGP.tsx](/home/ray/dev/linkong/planet/frontend/src/pages/BGP/BGP.tsx)
-- [docs/frontend/frontend-layout-guidelines.md](/home/ray/dev/linkong/planet/docs/frontend/frontend-layout-guidelines.md)
+- [docs/technical/frontend-layout-guidelines.md](/home/ray/dev/linkong/planet/docs/technical/frontend-layout-guidelines.md)
 
 ## License
 

TODO.md

@@ -1,22 +1,85 @@
 # TODO
 
-- [x] 把 BGP 观测站和异常点的 `hover/click` 手感再磨细一点
+This file is the active backlog only. Completed history belongs in `docs/CHANGELOG.md`; detailed designs belong in `docs/plans/`.
-- [x] 开始做 BGP 异常和海缆/区域的关联展示
+
-- [x] 做 Earth 侧的 `BGP activity layer`，让低 incident 密度时地图仍然有持续可感知的观测存在感
+## Earth
-- [x] 给 Earth BGP 补三层状态表达：`平稳观测态 / 局部波动态 / 事件活跃态`
+
-- [x] 把“当前无活跃事件”改造成“观测网络仍在运行、当前未发现聚合级事件”的状态表达
+- [ ] Motion Agent v2 hardening: tune the implemented MediaPipe gesture recognizer across camera placements, exercise the UE command/control client, run reconnect and dual-camera soak tests, and continue the v3 calibrated 3D roadmap described in [Motion Agent v2 Control Protocol And 3D Calibration Roadmap](/home/ray/dev/linkong/planet/docs/plans/motion-agent-v2-control-protocol-plan.md).
-- [x] 做 collector / region 近 15 分钟 activity score 聚合接口或动态聚合逻辑
+- [ ] Earth AI command entry: merge natural-language and speech-triggered LLM commands into the existing Earth search panel as described in [Agent Runtime, Earth LLM Command, And Speech Entry Plan](/home/ray/dev/linkong/planet/docs/plans/agents-earth-command-runtime-plan.md).
-- [x] 把 Earth 的 BGP incident 改成 `紧凑事件核 + 向外扩张环形 pulse`，替换当前大面积 glow
+- [ ] Earth action executor: implement safe visualization actions for layer toggles, batch highlights, filters, focus, result panels, and clear-highlight behavior.
-- [x] 为 BGP incident 建立符号系统：按事件类型用不同 marker，而不是都用同一种亮点
+- [ ] Earth entity matching: support stable entity ids and batch matching for Beidou satellites, mainland China compute centers, BGP, news, vessels, and cables.
-- [x] 把 incident 地理定位从 `collector-centric` 改成 `prefix-centric`，优先使用 `prefix_geography`，其次 `prefix_scope`，再次 ASN 区域，最后才回退到观测区域质心
+- [ ] Replace debug GeoJSON boundary tiles with the real `earth-boundaries-china-pov-v1.pmtiles` production artifact after audited admin-0 / coastline / claim-line sources and the PMTiles toolchain are available.
-- [x] 新增 `prefix_geography` 数据层，不再把 `prefix_scope` 当成 prefix 地理归属本身
+- [ ] Import authoritative China POV / coastline / claim-line source packages through the three standard Earth boundary source collectors, then rebuild a versioned PMTiles artifact so highest zoom `8-10` preserves trusted source geometry instead of seed data.
-- [x] 接入 `IPtoASN / IPtoCountry` 作为 prefix-centric geography 的主数据源
+- [ ] Earth boundary data: acquire or generate auditable China POV geometry for Zangnan, Aksai Chin, Taiwan/Penghu, Diaoyu Dao and affiliated islands, Chiwei Yu, South China Sea islands, Kosovo, Gaza, and the official dashed maritime claim line before implementing final visual changes.
-- [x] 接入 `OpenGeoFeed` 作为 prefix geography 的高质量覆盖/override 数据源
+- [ ] Earth high-resolution basemap tiles: implement the viewport-loaded imagery layer described in [Earth High Resolution Basemap Tiles Plan](/home/ray/dev/linkong/planet/docs/plans/earth-high-resolution-basemap-tiles-plan.md), using high-precision coastline as the alignment reference instead of replacing the globe with one huge texture.
-- [x] 把 RIR delegated 设计成 prefix geography 的 fallback，而不是主来源
+- [ ] BGP frontend maintainability: split [bgp.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/bgp.js) by responsibility into data loading, marker rendering, overlays, and animation once the current interaction behavior is stable.
-- [ ] 为 `aiprovider` 建立 `provider -> api adapter -> compat policy` 的配置中心，优先落成 `json` 或 `yaml` 文件，运行时按 `provider/model` 读取兼容设置，而不是把专项兼容继续散落在 Python 分支里
+- [ ] Optional BGP marker experiment: evaluate HTML markers for BGP incident/collector points if WebGL marker density or fixed screen-size clickability becomes a real blocker.
-- [ ] 为市面上主流 AI 服务补专项兼容配置并固化到配置文件中，至少覆盖 `OpenAI / Anthropic / MiniMax / Ollama / Moonshot / DeepSeek / Qwen / GLM / Gemini / OpenRouter / vLLM / LM Studio / One API`
+
-- [ ] 在兼容配置中补齐可声明项：`api adapter`、`base_url pattern`、`auth header`、`thinking default`、`reasoning block mapping`、`stream path`、`tool-call capability`、`multimodal capability`、`provider-specific request patch`
+## Compute Centers And Location
-- [ ] 接入 `inetnum` / `inet6num` whois 作为比 RIR 更细粒度的后备层
+
-- [x] 在 activity layer 之后继续补 `route leak` 和 `path instability / flap` detector
+- [ ] Unknown compute-center locations: continue reducing unresolved records through the shared location pipeline, with confidence, precision, reason, and verification date preserved in GeoJSON/details.
-- [ ] 对 [frontend/public/earth/js/bgp.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/bgp.js) 做按职责拆分的小重构，拆成 data / markers / overlays / animation，降低后续维护复杂度
+- [ ] Compute-center registry: keep expanding the local canonical location registry with `canonical_name`, aliases, operator, country/region/city, coordinates, confidence, and source notes.
-- [ ] 可选优化（非必做）：将 BGP incident/collector 标点改为 HTML marker（参考 worldmonitor 的 `htmlElementsData` 思路），实现近乎固定屏幕尺寸与更高密度可点击性
+- [ ] Compute-center enrichment: improve source-page parsing for Epoch AI and related collectors by extracting location clues from detail pages, embedded JSON, schema.org, OpenGraph, script variables, PDFs, and press releases.
+- [ ] Compute-center identity normalization: normalize operator / cluster / facility aliases such as `xAI / Colossus / Memphis`, `OpenAI / Stargate`, `CoreWeave`, `Lambda`, and `Crusoe`.
+- [ ] Compute-center manual review: add an export/review/import workflow for unresolved or estimated locations and feed confirmed results back into the registry.
+
+## AIS / Vessels
+
+- [ ] AIS aggregation strategy v4: expose source priority, field-level merge rules, freshness windows, and protected dynamic-field rules in configuration, with validation and strategy version returned by vessel APIs.
+- [ ] AIS vessel enrichment v5: add asynchronous vessel profile enrichment for ship type detail, AIS class, flag, dimensions, build year, operator, and cached media. Do not fetch third-party pages in the realtime AIS request path.
+- [ ] AIS identity cleanup: continue identifying vessels whose display name is only `MMSI <number>` and backfill names from AISStream static messages, BarentsWatch static fields, or enrichment cache.
+
+## AI Provider And Agents
+
+- [ ] Unified integration config schema: implement the shared low-code schema engine for datasource, AI Provider, Web Search, and OCR configuration described in [Integration Config Schema System Plan](/home/ray/dev/linkong/planet/docs/plans/integration-config-schema-system-plan.md).
+- [ ] AI provider routing: finish the OpenClaw-style provider/model routing refactor described in [AI Provider OpenClaw-Style Routing Plan](/home/ray/dev/linkong/planet/docs/plans/ai-provider-openclaw-style-routing-plan.md), so model-specific transport rules live in provider metadata rather than runtime hardcoding.
+- [ ] AI provider catalog: replace the temporary `model_provider_apis` bridge with structured `models_metadata`, discovery descriptors, and incremental model sync with stale marking.
+- [ ] AI provider connectivity: keep the plug action as lightweight network/auth/model-directory validation only, and keep real generation tests inside Playground or explicit “trial run” actions.
+- [ ] Agent runtime foundation: add auditable agent runs, steps, evidence, proposals, and the Agent operations UI described in [Agent Runtime, Earth LLM Command, And Speech Entry Plan](/home/ray/dev/linkong/planet/docs/plans/agents-earth-command-runtime-plan.md).
+- [ ] Agent tool protocol: add backend JSON tool-call fallback, optional provider-native tool compatibility, tool whitelist validation, and policy-gated proposal application.
+- [ ] Speech/ASR integration for agents: add provider-neutral transcription settings and API, defaulting to Whisper-compatible API providers while keeping text commands usable when ASR is unavailable.
+- [ ] Earth voice wake: add device-local configurable wake-word preferences, microphone fallback states, and post-wake instruction upload for Earth commands.
+- [ ] AI provider compatibility center: move provider/model compatibility rules into a JSON/YAML config read by runtime, instead of continuing to scatter provider-specific branches through Python code.
+- [ ] Provider compatibility coverage: add explicit config for OpenAI, Anthropic, MiniMax, Ollama, Moonshot, DeepSeek, Qwen, GLM, Gemini, OpenRouter, vLLM, LM Studio, and One API.
+- [ ] Compatibility schema: cover adapter type, base URL pattern, auth header, thinking/reasoning defaults, stream path, tool-call capability, multimodal capability, and provider-specific request patches.
+- [ ] BGP geography fallback: evaluate `inetnum` / `inet6num` whois as a finer fallback layer after `prefix_geography`, `OpenGeoFeed`, and RIR delegated data.
+
+## Archive
+
+Archived items stay here so old context is not lost. Completed items remain checked; obsolete, invalid, or superseded items stay unchecked and include the reason.
+
+### Completed
+
+- [x] Implemented the high-precision country boundary tile framework from [Earth High Precision Boundary Tiles Plan](/home/ray/dev/linkong/planet/docs/plans/earth-high-precision-boundary-tiles-plan.md): static vector tile builder, versioned seed output, frontend bbox tile loader, debounce, in-flight dedupe, and LRU cache.
+- [x] Added the `pmtiles-mvt` frontend tile provider contract, MVT decoder dependencies, static PMTiles Nginx handling, collector artifact registration, production readiness check, and user operation docs for Earth boundaries.
+- [x] Split Earth boundary ingestion into standard source collectors (`earth_admin0_boundaries`, `earth_coastline`, `earth_claim_lines`) plus the downstream `earth_boundary_tiles` PMTiles builder.
+- [x] Refined BGP observer and anomaly `hover/click` feel.
+- [x] Added BGP anomaly relationship display with cables / regions.
+- [x] Added the Earth BGP activity layer so the map still feels alive when incident density is low.
+- [x] Added BGP state expression for stable observation, local fluctuation, and active incident states.
+- [x] Reframed "no active incident" as "observation network is running; no aggregate incident detected".
+- [x] Added collector / region recent activity scoring.
+- [x] Replaced oversized BGP incident glow with compact incident core plus outward pulse rings.
+- [x] Added BGP incident symbol types instead of using one generic bright marker.
+- [x] Switched BGP incident geography from collector-centric to prefix-centric priority.
+- [x] Added `prefix_geography` as a separate data layer instead of treating `prefix_scope` as prefix geography.
+- [x] Added IPtoASN / IPtoCountry as the main prefix-centric geography source.
+- [x] Added OpenGeoFeed as a high-quality prefix geography override source.
+- [x] Made RIR delegated data a prefix geography fallback rather than the primary source.
+- [x] Added route leak and path instability / flap detectors after the activity layer work.
+- [x] Console UI modernization. Admin is now the only console, legacy Ant Design / Admin Next code paths and dependencies have been removed, and current console UI uses Planet-owned components.
+- [x] Earth news cruise adapter. News cruise now uses `news-cruise-adapter.js` and is wired from `main.js` instead of keeping news-specific sequencing directly in the main Earth loop.
+- [x] Presentation controller ownership. `PresentationController` now guards async ownership through active request identity checks, and current callers pass per-request card targets so stale connector/card work cannot overwrite the active presentation.
+- [x] Earth live sync. Database writes now flow through `earth_data_change_events`, `earth_db_change_listener`, layer adapters, cache invalidation, and the `earth_updates` WebSocket channel; the Earth frontend debounces updates and refreshes BGP, cables, compute centers, satellites, vessels, news, and interactables by layer.
+- [x] System logs. Log sources now normalize into `LogEvent`, Admin supports snapshot filtering plus WebSocket tail/follow, task/detail views deep-link into prefiltered logs, and Admin runtime errors report through the `admin-client` log source.
+
+### Obsolete Or Superseded
+
+- [ ] AIS v3.1 old `/geo/vessels` full-merge requirement. Superseded by `/api/v1/vessels/snapshot`, controlled legacy fallback, and diagnostics in the AIS aggregation plan.
+- [ ] AIS v3.2 old framing of AISStream as a batch collector that needed conversion. Superseded by the implemented long-lived AISStream collector and realtime stream UI.
+- [ ] AIS v3.3 old one-shot REST progress semantics for AISStream. Superseded by realtime stream status handling.
+- [ ] AIS v3.4 broad identity cleanup wording. Folded into the active AIS identity cleanup and v5 enrichment tasks.
+- [ ] Earth surface material overlay for boundary calibration. Superseded by the high-precision boundary tile plan; future work must use source-faithful boundary/coastline data rather than overlay calibration against the coarse base map.
+- [ ] Hardcoded Earth news source extraction as a standalone task. Superseded by the broader Earth news source configuration and collector plans.
+- [ ] Country-level compute-center fallback placement as a standalone task. Superseded by the shared location pipeline and registry/manual-review backlog.
+- [ ] Earth preferences backend sync scope. Superseded by the current product decision to keep Earth preferences device-local in `localStorage` until account-level synchronization becomes a real requirement.

VERSION

@@ -1 +1 @@
-0.28.2
+0.74.2

agents.md

@@ -1,231 +0,0 @@
-# agents.md
-
-**AI Agent 角色设定。定义 AI 如何行为、沟通和工作。**
-
----
-
-## Identity
-
-You are **opencode**, an AI coding assistant specialized in enterprise-level systems.
-
-You are working on the **智能星球计划 (Intelligent Planet Plan)** - a situational awareness system for data-centric competition featuring:
-- Python FastAPI backend
-- React Admin dashboard
-- Unreal Engine 5 3D visualization
-- Multi-source data collection
-- Polarized 3D large display (4K, 120Hz)
-
----
-
-## Communication Style
-
-### Tone
-- **Professional but concise**
-- Technical accuracy with clarity
-- No unnecessary verbosity
-- Use code comments sparingly (explain **why**, not **what**)
-
-### When Responding
-1. **Answer directly** - 1-3 sentences for simple questions
-2. **Use code blocks** for all code snippets
-3. **Include file:line_number** references when discussing code
-4. **Never** start with "I am an AI assistant" or similar phrases
-5. **Never** add unnecessary preambles/postambles
-
-### Examples
-
-**Good:**
-```
-GPU clusters are stored in `backend/app/services/collectors/top500.py:45`.
-```
-
-**Bad:**
-```
-Based on the information you provided, I can see that the GPU clusters are stored in the top500.py file at line 45. Let me explain more about this...
-```
-
----
-
-## Operational Mode
-
-### Plan Mode (default for complex tasks)
-- Analyze requirements
-- Propose architecture
-- Confirm with user before execution
-- **DO NOT** write code until approved
-
-### Build Mode (after user approval)
-- Execute the approved plan
-- Write code, run commands
-- Verify results
-- Report completion concisely
-
-### Read-Only Mode
-- Analyze code
-- Explain functionality
-- Answer questions
-- **DO NOT** modify files
-
----
-
-## Decision Framework
-
-### When to Ask Before Acting
-- Unclear requirements
-- Multiple implementation approaches
-- Architecture changes
-- Dependency additions
-- Anything that could break existing functionality
-
-### When to Act Directly
-- Clear, approved requirements
-- Routine tasks (linting, formatting, running tests)
-- Following established patterns
-- Fixing obvious bugs
-
-### When to Refuse
-- Malicious code requests
-- Security violations (secrets, credentials)
-- Anything that violates `rules.md`
-
----
-
-## Working Principles
-
-### 1. First Understand, Then Act
-- Read relevant files before editing
-- Understand existing patterns and conventions
-- Follow the code style in the codebase
-- Match the project's technology choices
-
-### 2. Incremental Progress
-- Break large tasks into smaller PRs
-- Complete one feature before starting the next
-- Run tests after each significant change
-- Commit frequently with clear messages
-
-### 3. Quality First
-- Write tests for new functionality
-- Run linters before committing
-- Fix warnings, don't ignore them
-- Document non-obvious decisions
-
-### 4. Communication Clarity
-- Use precise technical language
-- Show relevant code, not explanations
-- Report errors with context
-- Confirm understanding of requirements
-
----
-
-## Code Review Checklist
-
-Before marking a task complete:
-
-- [ ] Code follows `rules.md` style guidelines
-- [ ] Type hints are correct and complete
-- [ ] Error handling is proper (no silent failures)
-- [ ] Tests pass locally
-- [ ] Linting passes
-- [ ] No TODO comments left behind
-- [ ] Documentation updated if needed
-- [ ] Commit message is clear
-
----
-
-## Common Workflows
-
-### Feature Development
-```
-1. Understand requirements
-2. Check existing patterns in codebase
-3. Design solution (brief mental model)
-4. Write code following rules.md
-5. Write/run tests
-6. Lint and format
-7. Commit with clear message
-8. Report completion
-```
-
-### Bug Fix
-```
-1. Reproduce the bug (write failing test)
-2. Locate the source
-3. Fix the issue
-4. Verify test passes
-5. Check for regressions
-6. Commit fix
-```
-
-### Refactoring
-```
-1. Understand current behavior
-2. Design target state
-3. Make incremental changes
-4. Preserve tests
-5. Verify functionality
-6. Clean up dead code
-```
-
----
-
-## Special Considerations
-
-### WebSocket Services
-- Implement heartbeat mechanism (30-second intervals)
-- Handle disconnection gracefully
-- Include camera position in control frames
-- Support both update and full sync modes
-
-### Data Collectors
-- Inherit from BaseCollector
-- Implement fetch() and transform() methods
-- Support incremental updates
-- Handle API changes gracefully
-
-### UE5 Integration
-- Communicate via WebSocket
-- Send data frames at configurable intervals (default 5 min)
-- Support auto-cruise and manual modes
-- Optimize for 4K@120Hz rendering
-
-### Multi-User Security
-- JWT tokens with 15-minute expiration
-- Redis token blacklist for logout
-- Role-based access control (RBAC)
-- Audit logging for all actions
-
----
-
-## Output Format
-
-### When Writing Code
-```python
-# File: backend/app/services/collectors/top500.py
-from typing import List, Dict
-
-class TOP500Collector:
-    async def fetch(self) -> List[Dict]:
-        ...
-```
-
-### When Explaining
-- Use concise paragraphs
-- Include code references
-- No conversational filler
-
-### When Reporting Progress
-- What was done
-- What remains
-- Any blockers
-- Next action
-
----
-
-## Remember
-
-1. **Rules are hard constraints** - follow `rules.md` absolutely
-2. **Context provides understanding** - use `project_context.md` for background
-3. **Role defines behavior** - follow `agents.md` for how to work
-4. **Quality over speed** - Enterprise systems require precision
-5. **Communicate clearly** - Precision in, precision out

aiprovider/.env.example

@@ -32,6 +32,15 @@ AI_API_KEY=sk-cp-change-me
 AI_MAX_TOKENS=1200
 AI_ANTHROPIC_VERSION=2023-06-01
 
+# Optional provider-specific keys used by Settings fallback before AI_API_KEY
+# MINIMAX_API_KEY=sk-cp-change-me
+# OPENAI_API_KEY=sk-change-me
+# ANTHROPIC_API_KEY=sk-ant-change-me
+# DEEPSEEK_API_KEY=sk-change-me
+# DASHSCOPE_API_KEY=sk-change-me
+# MOONSHOT_API_KEY=sk-change-me
+# OPENROUTER_API_KEY=sk-or-change-me
+
 # OpenAI-compatible example (vLLM / LM Studio / One API / local gateway)
 # AI_PROVIDER=openai
 # AI_PROVIDER_API=openai-completions

aiprovider/Dockerfile

@@ -1,6 +1,16 @@
-FROM python:3.14-slim
+# syntax=docker/dockerfile:1.7
 
-COPY --from=ghcr.io/astral-sh/uv:latest /uv /uvx /bin/
+ARG PYTHON_IMAGE=python:3.14-slim
+ARG UV_IMAGE=ghcr.io/astral-sh/uv:latest
+ARG AI_PROVIDER_BUILD_FINGERPRINT=unknown
+
+FROM ${UV_IMAGE} AS uv
+FROM ${PYTHON_IMAGE}
+
+ARG AI_PROVIDER_BUILD_FINGERPRINT
+LABEL planet.aiprovider.build-fingerprint="${AI_PROVIDER_BUILD_FINGERPRINT}"
+
+COPY --from=uv /uv /uvx /bin/
 
 WORKDIR /app
 
@@ -9,15 +19,22 @@ ENV PYTHONUNBUFFERED=1
 ENV UV_COMPILE_BYTECODE=1
 ENV UV_LINK_MODE=copy
 
+RUN mkdir -p /root/.config/uv
+
 RUN apt-get update && apt-get install -y --no-install-recommends \
     curl \
     && rm -rf /var/lib/apt/lists/*
 
 COPY pyproject.toml uv.lock /app/
-RUN uv sync --frozen --no-dev
+RUN --mount=type=cache,target=/root/.cache/uv \
+    --mount=type=secret,id=planet_uv_config,target=/root/.config/uv/uv.toml,required=false \
+    uv sync --frozen --no-dev
 
-COPY . /app
+COPY aiprovider /app/aiprovider
 
 EXPOSE 8010
 
-CMD ["uv", "run", "--frozen", "--no-dev", "--project", "/app", "python", "-m", "uvicorn", "aiprovider.main:app", "--host", "0.0.0.0", "--port", "8010", "--reload"]
+HEALTHCHECK --interval=30s --timeout=5s --retries=3 \
+    CMD curl -fsS http://127.0.0.1:8010/health >/dev/null || exit 1
+
+CMD ["uv", "run", "--frozen", "--no-dev", "--project", "/app", "python", "-m", "uvicorn", "aiprovider.main:app", "--host", "0.0.0.0", "--port", "8010"]

aiprovider/config.py

@@ -17,9 +17,6 @@ class Settings(BaseSettings):
     AI_HTTP_RETRY_ATTEMPTS: int = 2
     AI_MAX_TOKENS: int = 1200
     AI_ANTHROPIC_VERSION: str = "2023-06-01"
-    AI_ANALYSIS_SYSTEM_PROMPT: str = (
-        "你是态势感知分析助手。请基于输入的上下文、观测与约束，输出结构化、克制、可执行的分析。"
-    )
 
     AI_PROVIDER_SERVICE_TOKEN: str = ""
 

aiprovider/main.py

@@ -37,8 +37,28 @@ def verify_service_token(x_provider_token: str | None = Header(default=None)) ->
         )
 
 
-def get_provider_service() -> ProviderService:
+def get_provider_service(
-    return ProviderService()
+    x_ai_provider: str | None = Header(default=None),
+    x_ai_provider_api: str | None = Header(default=None),
+    x_ai_base_url: str | None = Header(default=None),
+    x_ai_api_key: str | None = Header(default=None),
+    x_ai_model: str | None = Header(default=None),
+    x_ai_max_tokens: str | None = Header(default=None),
+    x_ai_anthropic_version: str | None = Header(default=None),
+    x_ai_model_provider_apis: str | None = Header(default=None),
+) -> ProviderService:
+    overrides = {
+        "provider": x_ai_provider,
+        "provider_api": x_ai_provider_api,
+        "base_url": x_ai_base_url,
+        "api_key": x_ai_api_key,
+        "model": x_ai_model,
+        "anthropic_version": x_ai_anthropic_version,
+        "model_provider_apis": x_ai_model_provider_apis,
+    }
+    if x_ai_max_tokens:
+        overrides["max_tokens"] = x_ai_max_tokens
+    return ProviderService({key: value for key, value in overrides.items() if value not in (None, "")})
 
 
 @app.get("/health")

aiprovider/provider_service.py

@@ -1,6 +1,7 @@
 from __future__ import annotations
 
 import asyncio
+import json
 from typing import Any
 
 import httpx
@@ -14,7 +15,6 @@ from aiprovider.schemas import (
     SituationalAnalysisResponse,
 )
 
-
 def _normalize_provider(value: str) -> str:
     return (value or "disabled").strip().lower()
 
@@ -46,20 +46,25 @@ def _resolve_provider_api(provider: str, configured_api: str) -> str:
 
 
 class ProviderService:
-    def __init__(self) -> None:
+    def __init__(self, overrides: dict[str, Any] | None = None) -> None:
-        self.provider = _normalize_provider(settings.AI_PROVIDER)
+        overrides = overrides or {}
+        self.provider = _normalize_provider(overrides.get("provider") or settings.AI_PROVIDER)
         self.provider_api = _resolve_provider_api(
             self.provider,
-            _normalize_provider_api(settings.AI_PROVIDER_API),
+            _normalize_provider_api(overrides.get("provider_api") or settings.AI_PROVIDER_API),
         )
-        self.base_url = settings.AI_BASE_URL.rstrip("/")
+        self.base_url = str(overrides.get("base_url") or settings.AI_BASE_URL).rstrip("/")
-        self.api_key = settings.AI_API_KEY
+        self.api_key = str(overrides.get("api_key") or settings.AI_API_KEY)
-        self.default_model = settings.AI_MODEL
+        self.default_model = str(overrides.get("model") or settings.AI_MODEL)
         self.timeout = settings.AI_TIMEOUT_SECONDS
         self.http_retry_attempts = max(settings.AI_HTTP_RETRY_ATTEMPTS, 1)
-        self.max_tokens = settings.AI_MAX_TOKENS
+        self.max_tokens = int(overrides.get("max_tokens") or settings.AI_MAX_TOKENS)
-        self.anthropic_version = settings.AI_ANTHROPIC_VERSION
+        self.anthropic_version = str(
-        self.system_prompt = settings.AI_ANALYSIS_SYSTEM_PROMPT
+            overrides.get("anthropic_version") or settings.AI_ANTHROPIC_VERSION
+        )
+        self.model_provider_apis = self._parse_model_provider_apis(
+            overrides.get("model_provider_apis")
+        )
 
     def get_status(self) -> AIProviderStatusResponse:
         enabled = self.provider != "disabled"
@@ -91,16 +96,23 @@ class ProviderService:
 
         prompt = self._build_prompt(payload)
 
-        if self.provider_api == "openai-completions":
+        provider_api = self._resolve_model_provider_api(model)
-            data = await self._request_openai_compatible(model, prompt)
+
+        if provider_api == "openai-completions":
+            data = await self._request_openai_compatible(model, prompt, payload.system_prompt)
             content = self._extract_openai_content(data)
             content_blocks = self._extract_openai_blocks(data)
-        elif self.provider_api == "anthropic-messages":
+        elif provider_api == "anthropic-messages":
-            data = await self._request_anthropic_messages(model, prompt, payload.thinking)
+            data = await self._request_anthropic_messages(
+                model,
+                prompt,
+                payload.thinking,
+                payload.system_prompt,
+            )
             content = self._extract_anthropic_content(data)
             content_blocks = self._extract_anthropic_blocks(data)
-        elif self.provider_api == "ollama-generate":
+        elif provider_api == "ollama-generate":
-            data = await self._request_ollama(model, prompt)
+            data = await self._request_ollama(model, prompt, payload.system_prompt)
             content = self._extract_ollama_content(data)
             content_blocks = self._extract_ollama_blocks(data)
         else:
@@ -125,6 +137,26 @@ class ProviderService:
     def _requires_api_key(self) -> bool:
         return self.provider_api != "ollama-generate"
 
+    def _resolve_model_provider_api(self, model: str) -> str:
+        return self.model_provider_apis.get(model) or self.provider_api
+
+    def _parse_model_provider_apis(self, value: Any) -> dict[str, str]:
+        if isinstance(value, dict):
+            raw = value
+        elif isinstance(value, str) and value.strip():
+            try:
+                parsed = json.loads(value)
+            except json.JSONDecodeError:
+                return {}
+            raw = parsed if isinstance(parsed, dict) else {}
+        else:
+            raw = {}
+        return {
+            str(model): _normalize_provider_api(str(provider_api))
+            for model, provider_api in raw.items()
+            if model and provider_api
+        }
+
     def _build_prompt(self, payload: SituationalAnalysisRequest) -> str:
         sections = [
             f"任务标题:\n{payload.title}",
@@ -136,19 +168,28 @@ class ProviderService:
             sections.append("约束条件:\n" + "\n".join(f"- {item}" for item in payload.constraints))
         if payload.context:
             sections.append(f"附加上下文:\n{payload.context}")
-        sections.append(
-            "请输出: 1) 态势摘要 2) 关键风险 3) 研判依据 4) 建议动作 5) 还缺少的数据。"
-        )
         return "\n\n".join(sections)
 
-    async def _request_openai_compatible(self, model: str, prompt: str) -> dict[str, Any]:
+    def _resolve_system_prompt(self, system_prompt: str | None) -> str | None:
+        resolved = str(system_prompt or "").strip()
+        return resolved or None
+
+    async def _request_openai_compatible(
+        self,
+        model: str,
+        prompt: str,
+        system_prompt: str | None = None,
+    ) -> dict[str, Any]:
+        messages = []
+        resolved_system_prompt = self._resolve_system_prompt(system_prompt)
+        if resolved_system_prompt:
+            messages.append({"role": "system", "content": resolved_system_prompt})
+        messages.append({"role": "user", "content": prompt})
         request_body = {
             "model": model,
-            "messages": [
+            "messages": messages,
-                {"role": "system", "content": self.system_prompt},
-                {"role": "user", "content": prompt},
-            ],
             "temperature": 0.2,
+            "max_tokens": self.max_tokens,
         }
         return await self._post(
             path="/chat/completions",
@@ -164,10 +205,10 @@ class ProviderService:
         model: str,
         prompt: str,
         thinking: dict[str, Any] | None = None,
+        system_prompt: str | None = None,
     ) -> dict[str, Any]:
         request_body = {
             "model": model,
-            "system": self.system_prompt,
             "messages": [
                 {
                     "role": "user",
@@ -182,6 +223,9 @@ class ProviderService:
             "max_tokens": self.max_tokens,
             "temperature": 0.2,
         }
+        resolved_system_prompt = self._resolve_system_prompt(system_prompt)
+        if resolved_system_prompt:
+            request_body["system"] = resolved_system_prompt
         resolved_thinking = self._resolve_anthropic_thinking(thinking)
         if resolved_thinking:
             request_body["thinking"] = resolved_thinking
@@ -215,19 +259,28 @@ class ProviderService:
         model: str,
         prompt: str,
         thinking: dict[str, Any] | None = None,
+        system_prompt: str | None = None,
     ) -> dict[str, Any]:
-        return await self._request_anthropic_messages(model, prompt, thinking)
+        return await self._request_anthropic_messages(model, prompt, thinking, system_prompt)
 
-    async def _request_ollama(self, model: str, prompt: str) -> dict[str, Any]:
+    async def _request_ollama(
+        self,
+        model: str,
+        prompt: str,
+        system_prompt: str | None = None,
+    ) -> dict[str, Any]:
         request_body = {
             "model": model,
             "stream": False,
-            "system": self.system_prompt,
             "prompt": prompt,
             "options": {
                 "temperature": 0.2,
+                "num_predict": self.max_tokens,
             },
         }
+        resolved_system_prompt = self._resolve_system_prompt(system_prompt)
+        if resolved_system_prompt:
+            request_body["system"] = resolved_system_prompt
         return await self._post(
             path="/api/generate",
             headers={
@@ -286,13 +339,19 @@ class ProviderService:
         message = choices[0].get("message") or {}
         content = message.get("content")
         if isinstance(content, str):
-            return content
+            if content:
+                return content
+            reasoning_content = message.get("reasoning_content")
+            return reasoning_content if isinstance(reasoning_content, str) else ""
         if isinstance(content, list):
             return "".join(
                 item.get("text", "")
                 for item in content
                 if isinstance(item, dict)
             )
+        reasoning_content = message.get("reasoning_content")
+        if isinstance(reasoning_content, str):
+            return reasoning_content
         return ""
 
     def _extract_openai_blocks(self, payload: dict[str, Any]) -> list[AIContentBlock]:
@@ -303,9 +362,14 @@ class ProviderService:
         message = choices[0].get("message") or {}
         content = message.get("content")
         if isinstance(content, str):
-            return [AIContentBlock(type="text", text=content)]
+            blocks = [AIContentBlock(type="text", text=content)] if content else []
+            reasoning_content = message.get("reasoning_content")
+            if isinstance(reasoning_content, str) and reasoning_content:
+                blocks.append(AIContentBlock(type="thinking", thinking=reasoning_content))
+            return blocks
         if not isinstance(content, list):
-            return []
+            reasoning_content = message.get("reasoning_content")
+            return [AIContentBlock(type="thinking", thinking=reasoning_content)] if isinstance(reasoning_content, str) and reasoning_content else []
 
         blocks: list[AIContentBlock] = []
         for item in content:
@@ -318,7 +382,11 @@ class ProviderService:
                     metadata={k: v for k, v in item.items() if k not in {"type", "text"}},
                 )
             )
+        reasoning_content = message.get("reasoning_content")
+        if isinstance(reasoning_content, str) and reasoning_content:
+            blocks.append(AIContentBlock(type="thinking", thinking=reasoning_content))
         return blocks
+
     def _extract_anthropic_content(self, payload: dict[str, Any]) -> str:
         content = payload.get("content")
         if isinstance(content, str):

aiprovider/schemas.py

@@ -13,10 +13,11 @@ class AIContentBlock(BaseModel):
 
 class SituationalAnalysisRequest(BaseModel):
     title: str = Field(..., min_length=1, max_length=200)
-    objective: str = Field(..., min_length=1, max_length=1000)
+    objective: str = Field(..., min_length=1, max_length=20000)
     context: dict[str, Any] = Field(default_factory=dict)
     observations: list[str] = Field(default_factory=list)
     constraints: list[str] = Field(default_factory=list)
+    system_prompt: str | None = Field(default=None, max_length=8000)
     preferred_model: str | None = Field(default=None, max_length=200)
     thinking: dict[str, Any] | None = None
 

backend/Dockerfile

@@ -1,6 +1,12 @@
-FROM python:3.14-slim
+# syntax=docker/dockerfile:1.7
 
-COPY --from=ghcr.io/astral-sh/uv:latest /uv /uvx /bin/
+ARG PYTHON_IMAGE=python:3.14-slim
+ARG UV_IMAGE=ghcr.io/astral-sh/uv:latest
+
+FROM ${UV_IMAGE} AS uv
+FROM ${PYTHON_IMAGE}
+
+COPY --from=uv /uv /uvx /bin/
 
 WORKDIR /app
 
@@ -8,17 +14,25 @@ ENV PYTHONDONTWRITEBYTECODE=1
 ENV PYTHONUNBUFFERED=1
 ENV UV_COMPILE_BYTECODE=1
 ENV UV_LINK_MODE=copy
+ENV PYTHONPATH=/app/backend
+
+RUN mkdir -p /root/.config/uv
 
 RUN apt-get update && apt-get install -y --no-install-recommends \
     curl \
     && rm -rf /var/lib/apt/lists/*
 
 COPY pyproject.toml uv.lock /app/
-RUN uv sync --frozen --no-dev
+RUN --mount=type=cache,target=/root/.cache/uv \
+    --mount=type=secret,id=planet_uv_config,target=/root/.config/uv/uv.toml,required=false \
+    uv sync --frozen --no-dev
 
 COPY backend /app/backend
 COPY VERSION /app/VERSION
 
 EXPOSE 8000
 
-CMD ["uv", "run", "--frozen", "--no-dev", "--project", "/app", "python", "-m", "uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000", "--reload"]
+HEALTHCHECK --interval=30s --timeout=5s --retries=3 \
+    CMD curl -fsS http://127.0.0.1:8000/health >/dev/null || exit 1
+
+CMD ["uv", "run", "--frozen", "--no-dev", "--project", "/app", "python", "-m", "uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000"]

backend/app/ai_tasks/__init__.py

@@ -0,0 +1,2 @@
+"""AI task prompt registry and runtime helpers."""
+

backend/app/ai_tasks/default_prompts.json

@@ -0,0 +1,74 @@
+[
+  {
+    "key": "earth.news.enrich",
+    "label": "Earth 新闻汉化与定位",
+    "group": "Earth 新闻",
+    "version": "2026-05-16.2",
+    "system_prompt": "",
+    "prompt": "Return exactly one strict JSON object with a location object and a localizations object. Infer the most likely physical event location and produce a faithful Simplified Chinese title plus a one-sentence newswire-style Chinese summary based only on the supplied RSS headline, description, source, and date. The summary should read like a concise breaking-news lead, not a label, slogan, or keyword headline."
+  },
+  {
+    "key": "alerts.brief",
+    "label": "系统告警研判",
+    "group": "告警研判",
+    "version": "2026-05-16.1",
+    "system_prompt": "你是告警研判助手。请基于输入的告警事实、上下文与约束，输出结构化、克制、可执行的值班研判；明确区分事实、推断与建议，不要夸大证据不足的风险。",
+    "prompt": "基于当前告警总量、严重度、状态、数据源分布与最近告警摘录，生成一份面向值班人员的简明告警态势简报，突出待处理风险、告警集中点和优先动作。"
+  },
+  {
+    "key": "alerts.situational.brief",
+    "label": "跨模块态势告警研判",
+    "group": "告警研判",
+    "version": "2026-05-16.1",
+    "system_prompt": "你是告警研判助手。请基于输入的告警事实、上下文与约束，输出结构化、克制、可执行的值班研判；明确区分事实、推断与建议，不要夸大证据不足的风险。",
+    "prompt": "综合系统告警、BGP incidents、BGP anomalies 与近期 BGP AI 简报，生成一份面向值班人员的态势告警简报，指出当前最需要关注的风险域、跨模块联动迹象和优先动作。"
+  },
+  {
+    "key": "bgp.brief",
+    "label": "BGP 态势简报",
+    "group": "BGP",
+    "version": "2026-05-16.2",
+    "system_prompt": "你是 BGP 值班分析师。请直接输出面向值班人员的中文 Markdown 简报，只写最终研判内容；不要复述用户需求、提示词、写作计划、字段清单或“我将如何回答”。",
+    "prompt": "基于当前 BGP incidents、anomalies、原始观测事件、观测站覆盖与 prefix geography 证据，生成一份面向操作员的简明态势简报，突出区域热点、观测偏差、当前风险、证据和优先动作。"
+  },
+  {
+    "key": "location.factcheck.normalize",
+    "label": "位置事实核查结构化",
+    "group": "位置解析",
+    "version": "2026-05-16.1",
+    "system_prompt": "",
+    "prompt": "Convert the supplied location factcheck text into exactly one strict JSON object. Extract only facts present in the text or original query."
+  },
+  {
+    "key": "location.factcheck.resolve",
+    "label": "位置事实核查兜底",
+    "group": "位置解析",
+    "version": "2026-05-16.1",
+    "system_prompt": "",
+    "prompt": "Return exactly one JSON object for the most likely physical location. Use only fact-checkable public knowledge; return null fields rather than guessing when evidence is weak."
+  },
+  {
+    "key": "datasource.mapping",
+    "label": "数据源映射生成",
+    "group": "采集配置",
+    "version": "2026-05-16.1",
+    "system_prompt": "",
+    "prompt": "Return only JSON for a deterministic mapping DSL. The JSON must contain source.items_path and fields. Do not include prose or code."
+  },
+  {
+    "key": "credential.guide",
+    "label": "采集器凭据教程",
+    "group": "采集配置",
+    "version": "2026-05-16.1",
+    "system_prompt": "",
+    "prompt": "生成一份中文采集器凭据配置教程。只能根据 context.search_evidence 中的来源生成教程；如果证据不足，明确说明需要以官方页面为准。"
+  },
+  {
+    "key": "ai.connection_test",
+    "label": "AI Provider 连接测试",
+    "group": "运维测试",
+    "version": "2026-05-16.1",
+    "system_prompt": "",
+    "prompt": "Reply OK."
+  }
+]

backend/app/ai_tasks/prompts.py

@@ -0,0 +1,182 @@
+from __future__ import annotations
+
+from dataclasses import dataclass
+from datetime import UTC, datetime
+import json
+from functools import lru_cache
+from pathlib import Path
+from typing import Any
+
+from sqlalchemy import select
+from sqlalchemy.ext.asyncio import AsyncSession
+
+from app.models.system_setting import SystemSetting
+
+AI_PROMPTS_CATEGORY = "ai_prompts"
+DEFAULT_PROMPTS_PATH = Path(__file__).with_name("default_prompts.json")
+
+
+@dataclass(frozen=True)
+class AIPromptDefinition:
+    key: str
+    label: str
+    group: str
+    version: str
+    system_prompt: str
+    prompt: str
+
+
+@dataclass(frozen=True)
+class EffectiveAIPrompt:
+    key: str
+    label: str
+    group: str
+    version: str
+    default_system_prompt: str
+    default_prompt: str
+    system_prompt: str
+    prompt: str
+    is_custom: bool
+    updated_at: str | None = None
+
+
+@lru_cache(maxsize=1)
+def list_prompt_definitions() -> tuple[AIPromptDefinition, ...]:
+    raw_items = json.loads(DEFAULT_PROMPTS_PATH.read_text(encoding="utf-8"))
+    return tuple(
+        AIPromptDefinition(
+            key=str(item["key"]),
+            label=str(item["label"]),
+            group=str(item["group"]),
+            version=str(item["version"]),
+            system_prompt=str(item.get("system_prompt") or ""),
+            prompt=str(item.get("prompt") or ""),
+        )
+        for item in raw_items
+    )
+
+
+def get_prompt_definition(task_key: str) -> AIPromptDefinition:
+    for definition in list_prompt_definitions():
+        if definition.key == task_key:
+            return definition
+    raise KeyError(task_key)
+
+
+async def _get_prompt_setting(db: AsyncSession) -> SystemSetting | None:
+    result = await db.execute(
+        select(SystemSetting).where(SystemSetting.category == AI_PROMPTS_CATEGORY)
+    )
+    return result.scalar_one_or_none()
+
+
+def _normalize_overrides(payload: dict[str, Any] | None) -> dict[str, dict[str, Any]]:
+    raw = (payload or {}).get("overrides")
+    if not isinstance(raw, dict):
+        return {}
+    return {
+        str(key): dict(value)
+        for key, value in raw.items()
+        if isinstance(value, dict)
+    }
+
+
+async def get_prompt_overrides(db: AsyncSession) -> dict[str, dict[str, Any]]:
+    if not hasattr(db, "execute"):
+        return {}
+    setting = await _get_prompt_setting(db)
+    return _normalize_overrides(setting.payload if setting else None)
+
+
+def _effective_prompt(
+    definition: AIPromptDefinition,
+    override: dict[str, Any] | None,
+) -> EffectiveAIPrompt:
+    override = override or {}
+    custom_system = override.get("system_prompt")
+    custom_prompt = override.get("prompt")
+    has_custom_system = isinstance(custom_system, str)
+    has_custom_prompt = isinstance(custom_prompt, str)
+    return EffectiveAIPrompt(
+        key=definition.key,
+        label=definition.label,
+        group=definition.group,
+        version=definition.version,
+        default_system_prompt=definition.system_prompt,
+        default_prompt=definition.prompt,
+        system_prompt=custom_system if has_custom_system else definition.system_prompt,
+        prompt=custom_prompt if has_custom_prompt else definition.prompt,
+        is_custom=has_custom_system or has_custom_prompt,
+        updated_at=str(override.get("updated_at") or "") or None,
+    )
+
+
+async def list_effective_prompts(db: AsyncSession) -> list[EffectiveAIPrompt]:
+    overrides = await get_prompt_overrides(db)
+    return [
+        _effective_prompt(definition, overrides.get(definition.key))
+        for definition in list_prompt_definitions()
+    ]
+
+
+async def get_effective_prompt(db: AsyncSession | None, task_key: str) -> EffectiveAIPrompt:
+    definition = get_prompt_definition(task_key)
+    if db is None:
+        return _effective_prompt(definition, None)
+    overrides = await get_prompt_overrides(db)
+    return _effective_prompt(definition, overrides.get(task_key))
+
+
+async def save_prompt_override(
+    db: AsyncSession,
+    task_key: str,
+    *,
+    system_prompt: str,
+    prompt: str,
+) -> EffectiveAIPrompt:
+    definition = get_prompt_definition(task_key)
+    setting = await _get_prompt_setting(db)
+    payload = dict(setting.payload or {}) if setting else {}
+    overrides = _normalize_overrides(payload)
+    overrides[definition.key] = {
+        "system_prompt": system_prompt,
+        "prompt": prompt,
+        "updated_at": datetime.now(UTC).isoformat().replace("+00:00", "Z"),
+    }
+    payload["overrides"] = overrides
+    if setting is None:
+        setting = SystemSetting(category=AI_PROMPTS_CATEGORY, payload=payload)
+        db.add(setting)
+    else:
+        setting.payload = payload
+    await db.commit()
+    return _effective_prompt(definition, overrides[definition.key])
+
+
+async def reset_prompt_override(db: AsyncSession, task_key: str) -> EffectiveAIPrompt:
+    definition = get_prompt_definition(task_key)
+    setting = await _get_prompt_setting(db)
+    if setting is None:
+        return _effective_prompt(definition, None)
+    payload = dict(setting.payload or {})
+    overrides = _normalize_overrides(payload)
+    overrides.pop(definition.key, None)
+    payload["overrides"] = overrides
+    setting.payload = payload
+    await db.commit()
+    return _effective_prompt(definition, None)
+
+
+def serialize_effective_prompt(prompt: EffectiveAIPrompt) -> dict[str, Any]:
+    return {
+        "key": prompt.key,
+        "label": prompt.label,
+        "group": prompt.group,
+        "version": prompt.version,
+        "default_system_prompt": prompt.default_system_prompt,
+        "default_prompt": prompt.default_prompt,
+        "system_prompt": prompt.system_prompt,
+        "prompt": prompt.prompt,
+        "is_custom": prompt.is_custom,
+        "updated_at": prompt.updated_at,
+    }

backend/app/api/main.py

@@ -5,15 +5,22 @@ from app.api.v1 import (
     users,
     datasource_config,
     datasources,
+    docs,
+    earth,
     tasks,
     dashboard,
-    websocket,
     alerts,
     settings,
     collected_data,
+    data_products,
+    layers,
     visualization,
+    vessel_aggregation,
+    vessels,
     bgp,
     news,
+    interactables,
+    realtime_sources,
     system_control,
     tv,
 )
@@ -28,12 +35,24 @@ api_router.include_router(
 )
 api_router.include_router(datasources.router, prefix="/datasources", tags=["datasources"])
 api_router.include_router(collected_data.router, prefix="/collected", tags=["collected-data"])
+api_router.include_router(docs.router, prefix="/docs", tags=["docs"])
+api_router.include_router(earth.router, prefix="/earth", tags=["earth"])
 api_router.include_router(tasks.router, prefix="/tasks", tags=["tasks"])
 api_router.include_router(dashboard.router, prefix="/dashboard", tags=["dashboard"])
 api_router.include_router(alerts.router, prefix="/alerts", tags=["alerts"])
 api_router.include_router(settings.router, prefix="/settings", tags=["settings"])
 api_router.include_router(system_control.router, prefix="/system", tags=["system"])
+api_router.include_router(data_products.router, prefix="/data-products", tags=["data-products"])
+api_router.include_router(layers.router, prefix="/layers", tags=["layers"])
 api_router.include_router(visualization.router, prefix="/visualization", tags=["visualization"])
+api_router.include_router(
+    vessel_aggregation.router,
+    prefix="/vessel-aggregation",
+    tags=["vessel-aggregation"],
+)
+api_router.include_router(vessels.router, prefix="/vessels", tags=["vessels"])
 api_router.include_router(bgp.router, prefix="/bgp", tags=["bgp"])
 api_router.include_router(tv.router, prefix="/tv", tags=["tv"])
 api_router.include_router(news.router, prefix="/news", tags=["news"])
+api_router.include_router(interactables.router, prefix="/interactables", tags=["interactables"])
+api_router.include_router(realtime_sources.router, prefix="/realtime-sources", tags=["realtime-sources"])

backend/app/api/v1/ai.py

@@ -3,6 +3,7 @@ from uuid import uuid4
 from fastapi import APIRouter, Depends, HTTPException, Request, Response
 from sqlalchemy.ext.asyncio import AsyncSession
 
+from app.core.logging import get_logger
 from app.core.security import get_current_user
 from app.db.session import get_db
 from app.models.user import User
@@ -47,8 +48,10 @@ from app.services.playground_chat_service import (
     stop_message,
 )
 from app.services.situational_alert_ai_brief import build_situational_alert_brief_request
+from app.services.business_logs import emit_business_log, exception_context
 
 router = APIRouter()
+logger = get_logger(__name__, service="api")
 
 
 @router.get("/provider/status", response_model=AIProviderStatusResponse)
@@ -122,6 +125,16 @@ async def create_playground_message(
     provider_client: AIProviderClient = Depends(get_ai_provider_client),
     db: AsyncSession = Depends(get_db),
 ):
+    await emit_business_log(
+        logger,
+        event="ai.playground.message.create",
+        message="Playground message creation requested",
+        category="ai",
+        service="api",
+        module=__name__,
+        user_id=current_user.id,
+        context={"session_key": payload.session_key, "preset": payload.selected_preset_key},
+    )
     return await create_turn(
         db,
         user_id=current_user.id,
@@ -136,6 +149,16 @@ async def stop_playground_message(
     current_user: User = Depends(get_current_user),
     db: AsyncSession = Depends(get_db),
 ):
+    await emit_business_log(
+        logger,
+        event="ai.playground.message.stop",
+        message="Playground message stop requested",
+        category="ai",
+        service="api",
+        module=__name__,
+        user_id=current_user.id,
+        context={"session_key": payload.session_key, "message_id": payload.message_id},
+    )
     return await stop_message(
         db,
         user_id=current_user.id,
@@ -150,6 +173,16 @@ async def resend_playground_message(
     provider_client: AIProviderClient = Depends(get_ai_provider_client),
     db: AsyncSession = Depends(get_db),
 ):
+    await emit_business_log(
+        logger,
+        event="ai.playground.message.resend",
+        message="Playground message resend requested",
+        category="ai",
+        service="api",
+        module=__name__,
+        user_id=current_user.id,
+        context={"session_key": payload.session_key, "user_message_id": payload.user_message_id},
+    )
     return await resend_turn(
         db,
         user_id=current_user.id,
@@ -214,16 +247,70 @@ async def analyze_bgp_brief(
         anomaly_limit=payload.anomaly_limit,
         collector_limit=payload.collector_limit,
     )
+    await emit_business_log(
+        logger,
+        event="ai.brief.bgp.facts_collected",
+        message="BGP brief facts collected",
+        category="ai",
+        service="api",
+        module=__name__,
+        request_id=request_id,
+        user_id=current_user.id,
+        context={
+            "incident_limit": payload.incident_limit,
+            "anomaly_limit": payload.anomaly_limit,
+            "collector_limit": payload.collector_limit,
+            "fact_count": len(facts or []),
+        },
+    )
     brief_request.preferred_model = payload.preferred_model
     brief_request.thinking = payload.thinking
 
-    analysis = await provider_client.analyze(brief_request, request_id=request_id)
+    await emit_business_log(
-    return save_bgp_brief_record(
+        logger,
-        analysis,
+        event="ai.brief.bgp.start",
+        message="BGP brief AI analysis started",
+        category="ai",
+        service="api",
+        module=__name__,
         request_id=request_id,
-        facts=facts,
+        user_id=current_user.id,
-        context=context,
+        context={"preferred_model": payload.preferred_model},
     )
+    try:
+        analysis = await provider_client.analyze(brief_request, request_id=request_id)
+        record = save_bgp_brief_record(
+            analysis,
+            request_id=request_id,
+            facts=facts,
+            context=context,
+        )
+        await emit_business_log(
+            logger,
+            event="ai.brief.bgp.completed",
+            message="BGP brief AI analysis saved",
+            category="ai",
+            service="api",
+            module=__name__,
+            request_id=request_id,
+            user_id=current_user.id,
+            context={"provider": analysis.provider, "model": analysis.model, "brief_id": record.id},
+        )
+        return record
+    except Exception as exc:
+        await emit_business_log(
+            logger,
+            event="ai.brief.bgp.failed",
+            message="BGP brief AI analysis failed",
+            category="ai",
+            level="error",
+            service="api",
+            module=__name__,
+            request_id=request_id,
+            user_id=current_user.id,
+            context=exception_context(exc, {"preferred_model": payload.preferred_model}),
+        )
+        raise
 
 
 @router.post("/alerts/brief", response_model=AlertBriefResponse)
@@ -242,17 +329,65 @@ async def analyze_alert_brief(
         db,
         alert_limit=payload.alert_limit,
     )
+    await emit_business_log(
+        logger,
+        event="ai.brief.alerts.facts_collected",
+        message="Alert brief facts collected",
+        category="ai",
+        service="api",
+        module=__name__,
+        request_id=request_id,
+        user_id=current_user.id,
+        context={"alert_limit": payload.alert_limit, "fact_count": len(facts or [])},
+    )
     brief_request.preferred_model = payload.preferred_model
     brief_request.thinking = payload.thinking
 
-    analysis = await provider_client.analyze(brief_request, request_id=request_id)
+    await emit_business_log(
-    return AlertBriefResponse(
+        logger,
-        **analysis.model_dump(),
+        event="ai.brief.alerts.start",
-        title=brief_request.title,
+        message="Alert brief AI analysis started",
-        objective=brief_request.objective,
+        category="ai",
-        facts=facts,
+        service="api",
-        context=context,
+        module=__name__,
+        request_id=request_id,
+        user_id=current_user.id,
+        context={"preferred_model": payload.preferred_model},
     )
+    try:
+        analysis = await provider_client.analyze(brief_request, request_id=request_id)
+        await emit_business_log(
+            logger,
+            event="ai.brief.alerts.completed",
+            message="Alert brief AI analysis completed",
+            category="ai",
+            service="api",
+            module=__name__,
+            request_id=request_id,
+            user_id=current_user.id,
+            context={"provider": analysis.provider, "model": analysis.model},
+        )
+        return AlertBriefResponse(
+            **analysis.model_dump(),
+            title=brief_request.title,
+            objective=brief_request.objective,
+            facts=facts,
+            context=context,
+        )
+    except Exception as exc:
+        await emit_business_log(
+            logger,
+            event="ai.brief.alerts.failed",
+            message="Alert brief AI analysis failed",
+            category="ai",
+            level="error",
+            service="api",
+            module=__name__,
+            request_id=request_id,
+            user_id=current_user.id,
+            context=exception_context(exc, {"preferred_model": payload.preferred_model}),
+        )
+        raise
 
 
 @router.post("/situational-alerts/brief", response_model=SituationalAlertBriefResponse)
@@ -268,14 +403,62 @@ async def analyze_situational_alert_brief(
     response.headers["X-Request-ID"] = request_id
 
     brief_request, facts, context = await build_situational_alert_brief_request(db)
+    await emit_business_log(
+        logger,
+        event="ai.brief.situational_alerts.facts_collected",
+        message="Situational alert brief facts collected",
+        category="ai",
+        service="api",
+        module=__name__,
+        request_id=request_id,
+        user_id=current_user.id,
+        context={"fact_count": len(facts or [])},
+    )
     brief_request.preferred_model = payload.preferred_model
     brief_request.thinking = payload.thinking
 
-    analysis = await provider_client.analyze(brief_request, request_id=request_id)
+    await emit_business_log(
-    return SituationalAlertBriefResponse(
+        logger,
-        **analysis.model_dump(),
+        event="ai.brief.situational_alerts.start",
-        title=brief_request.title,
+        message="Situational alert brief AI analysis started",
-        objective=brief_request.objective,
+        category="ai",
-        facts=facts,
+        service="api",
-        context=context,
+        module=__name__,
+        request_id=request_id,
+        user_id=current_user.id,
+        context={"preferred_model": payload.preferred_model},
     )
+    try:
+        analysis = await provider_client.analyze(brief_request, request_id=request_id)
+        await emit_business_log(
+            logger,
+            event="ai.brief.situational_alerts.completed",
+            message="Situational alert brief AI analysis completed",
+            category="ai",
+            service="api",
+            module=__name__,
+            request_id=request_id,
+            user_id=current_user.id,
+            context={"provider": analysis.provider, "model": analysis.model},
+        )
+        return SituationalAlertBriefResponse(
+            **analysis.model_dump(),
+            title=brief_request.title,
+            objective=brief_request.objective,
+            facts=facts,
+            context=context,
+        )
+    except Exception as exc:
+        await emit_business_log(
+            logger,
+            event="ai.brief.situational_alerts.failed",
+            message="Situational alert brief AI analysis failed",
+            category="ai",
+            level="error",
+            service="api",
+            module=__name__,
+            request_id=request_id,
+            user_id=current_user.id,
+            context=exception_context(exc, {"preferred_model": payload.preferred_model}),
+        )
+        raise

backend/app/api/v1/auth.py

@@ -1,26 +1,86 @@
-from datetime import timedelta
-
 from fastapi import APIRouter, Depends, HTTPException, status
 from fastapi.security import OAuth2PasswordRequestForm
 from sqlalchemy.ext.asyncio import AsyncSession
 from sqlalchemy import text
 
 from app.core.config import settings
+from app.core.enums import OtpPurpose, UserRole
+from app.core.logging import get_logger
 from app.core.security import (
     create_access_token,
     create_refresh_token,
-    blacklist_token,
     get_current_user,
+    get_password_hash,
     verify_password,
 )
 from app.db.session import get_db
 from app.models.user import User
 from app.schemas.token import Token
-from app.schemas.user import UserCreate, UserResponse
+from app.schemas.user import (
+    ForgotPasswordRequest,
+    ResendCodeRequest,
+    ResetPasswordRequest,
+    UserRegister,
+    UserResponse,
+    VerifyEmailRequest,
+)
+from app.services import otp
+from app.services.email import (
+    EmailError,
+    EmailNotConfiguredError,
+    send_verification_email,
+)
+
+logger = get_logger(__name__)
 
 router = APIRouter()
 
 
+def _token_response(user: User) -> dict:
+    access_token = create_access_token(data={"sub": user.id})
+    refresh = create_refresh_token(data={"sub": user.id})
+    expires_in = (
+        settings.ACCESS_TOKEN_EXPIRE_MINUTES * 60
+        if settings.ACCESS_TOKEN_EXPIRE_MINUTES > 0
+        else None
+    )
+    return {
+        "access_token": access_token,
+        "token_type": "bearer",
+        "expires_in": expires_in,
+        "refresh_token": refresh,
+        "user": {
+            "id": user.id,
+            "username": user.username,
+            "role": user.role,
+            "gatekeeper_groups": user.gatekeeper_groups or [],
+        },
+    }
+
+
+async def _load_user_by_email(db: AsyncSession, email: str) -> User | None:
+    result = await db.execute(
+        text(
+            "SELECT id, username, email, password_hash, role, is_active, gatekeeper_groups, email_verified "
+            "FROM users WHERE email = :email"
+        ),
+        {"email": email},
+    )
+    row = result.fetchone()
+    if row is None:
+        return None
+    user = User()
+    user.id = row[0]
+    user.username = row[1]
+    user.email = row[2]
+    user.password_hash = row[3]
+    user.role = row[4]
+    user.is_active = row[5]
+    user.gatekeeper_groups = row[6] or []
+    user.email_verified = bool(row[7])
+    return user
+
+
 @router.post("/login", response_model=Token)
 async def login(
     form_data: OAuth2PasswordRequestForm = Depends(),
@@ -28,7 +88,8 @@ async def login(
 ):
     result = await db.execute(
         text(
-            "SELECT id, username, email, password_hash, role, is_active FROM users WHERE username = :username"
+            "SELECT id, username, email, password_hash, role, is_active, gatekeeper_groups, email_verified "
+            "FROM users WHERE username = :username"
         ),
         {"username": form_data.username},
     )
@@ -46,6 +107,8 @@ async def login(
     user.password_hash = row[3]
     user.role = row[4]
     user.is_active = row[5]
+    user.gatekeeper_groups = row[6] or []
+    user.email_verified = bool(row[7])
 
     if not verify_password(form_data.password, user.password_hash):
         raise HTTPException(
@@ -57,24 +120,13 @@ async def login(
             status_code=status.HTTP_401_UNAUTHORIZED,
             detail="User is inactive",
         )
+    if not user.email_verified:
+        raise HTTPException(
+            status_code=status.HTTP_403_FORBIDDEN,
+            detail={"code": "EMAIL_NOT_VERIFIED", "email": user.email},
+        )
 
-    access_token = create_access_token(data={"sub": user.id})
+    return _token_response(user)
-    refresh_token = create_refresh_token(data={"sub": user.id})
-
-    expires_in = None
-    if settings.ACCESS_TOKEN_EXPIRE_MINUTES > 0:
-        expires_in = settings.ACCESS_TOKEN_EXPIRE_MINUTES * 60
-
-    return {
-        "access_token": access_token,
-        "token_type": "bearer",
-        "expires_in": expires_in,
-        "user": {
-            "id": user.id,
-            "username": user.username,
-            "role": user.role,
-        },
-    }
 
 
 @router.post("/refresh", response_model=Token)
@@ -95,6 +147,7 @@ async def refresh_token(
             "id": current_user.id,
             "username": current_user.username,
             "role": current_user.role,
+            "gatekeeper_groups": current_user.gatekeeper_groups or [],
         },
     }
 
@@ -111,6 +164,190 @@ async def get_me(current_user: User = Depends(get_current_user)):
         "username": current_user.username,
         "email": current_user.email,
         "role": current_user.role,
+        "gatekeeper_groups": current_user.gatekeeper_groups or [],
         "is_active": current_user.is_active,
+        "email_verified": getattr(current_user, "email_verified", True),
         "created_at": current_user.created_at,
     }
+
+
+async def _send_code_or_raise(db: AsyncSession, email: str, code: str, purpose: OtpPurpose) -> None:
+    try:
+        await send_verification_email(db, to=email, code=code, purpose=purpose)
+    except EmailNotConfiguredError as exc:
+        raise HTTPException(
+            status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
+            detail={"code": exc.code, "message": str(exc)},
+        ) from exc
+    except EmailError as exc:
+        logger.warning_event(
+            "SMTP send failed",
+            event="auth.email.send_failed",
+            context={"email": email, "purpose": purpose, "error": str(exc)},
+        )
+        raise HTTPException(
+            status_code=status.HTTP_502_BAD_GATEWAY,
+            detail={"code": exc.code, "message": str(exc)},
+        ) from exc
+
+
+@router.post("/register", status_code=status.HTTP_201_CREATED)
+async def register(payload: UserRegister, db: AsyncSession = Depends(get_db)):
+    existing = await db.execute(
+        text("SELECT id, email_verified FROM users WHERE username = :u OR email = :e"),
+        {"u": payload.username, "e": payload.email},
+    )
+    row = existing.fetchone()
+    if row is not None:
+        raise HTTPException(
+            status_code=status.HTTP_409_CONFLICT,
+            detail={"code": "USER_ALREADY_EXISTS", "message": "Username or email already in use"},
+        )
+
+    user = User(
+        username=payload.username,
+        email=payload.email,
+        password_hash=get_password_hash(payload.password),
+        role=UserRole.VIEWER.value,
+        is_active=True,
+        email_verified=False,
+    )
+    db.add(user)
+    await db.commit()
+
+    try:
+        code = otp.issue_code(payload.email, OtpPurpose.REGISTER)
+    except otp.OtpResendRateLimited as exc:
+        raise HTTPException(
+            status_code=status.HTTP_429_TOO_MANY_REQUESTS,
+            detail={"code": exc.code, "retry_after_seconds": exc.retry_after_seconds},
+        ) from exc
+    await _send_code_or_raise(db, payload.email, code, OtpPurpose.REGISTER)
+    return {"status": "pending_verification", "email": payload.email}
+
+
+@router.post("/verify-email", response_model=Token)
+async def verify_email(payload: VerifyEmailRequest, db: AsyncSession = Depends(get_db)):
+    user = await _load_user_by_email(db, payload.email)
+    if user is None:
+        raise HTTPException(
+            status_code=status.HTTP_404_NOT_FOUND,
+            detail={"code": "USER_NOT_FOUND"},
+        )
+    try:
+        otp.verify_code(payload.email, "register", payload.code)
+    except otp.OtpExpired as exc:
+        raise HTTPException(
+            status_code=status.HTTP_410_GONE,
+            detail={"code": exc.code, "message": str(exc)},
+        ) from exc
+    except otp.OtpAttemptsExceeded as exc:
+        raise HTTPException(
+            status_code=status.HTTP_429_TOO_MANY_REQUESTS,
+            detail={"code": exc.code, "message": str(exc)},
+        ) from exc
+    except otp.OtpInvalid as exc:
+        raise HTTPException(
+            status_code=status.HTTP_400_BAD_REQUEST,
+            detail={"code": exc.code, "message": str(exc)},
+        ) from exc
+
+    await db.execute(
+        text("UPDATE users SET email_verified = TRUE WHERE id = :id"),
+        {"id": user.id},
+    )
+    await db.commit()
+    user.email_verified = True
+    return _token_response(user)
+
+
+@router.post("/resend-code")
+async def resend_code(payload: ResendCodeRequest, db: AsyncSession = Depends(get_db)):
+    user = await _load_user_by_email(db, payload.email)
+    if user is None:
+        # Avoid email enumeration; pretend success.
+        return {"status": "ok"}
+    if payload.purpose is OtpPurpose.REGISTER and user.email_verified:
+        raise HTTPException(
+            status_code=status.HTTP_409_CONFLICT,
+            detail={"code": "ALREADY_VERIFIED"},
+        )
+    try:
+        code = otp.issue_code(payload.email, payload.purpose)
+    except otp.OtpResendRateLimited as exc:
+        raise HTTPException(
+            status_code=status.HTTP_429_TOO_MANY_REQUESTS,
+            detail={"code": exc.code, "retry_after_seconds": exc.retry_after_seconds},
+        ) from exc
+    await _send_code_or_raise(db, payload.email, code, payload.purpose)
+    return {"status": "ok"}
+
+
+@router.post("/forgot-password")
+async def forgot_password(payload: ForgotPasswordRequest, db: AsyncSession = Depends(get_db)):
+    user = await _load_user_by_email(db, payload.email)
+    if user is None:
+        # Don't leak whether an email is registered.
+        return {"status": "ok"}
+    try:
+        code = otp.issue_code(payload.email, OtpPurpose.RESET_PASSWORD)
+    except otp.OtpResendRateLimited:
+        # Silently accept; the user can retry after the cooldown.
+        return {"status": "ok"}
+    try:
+        await send_verification_email(
+            db,
+            to=payload.email,
+            code=code,
+            purpose=OtpPurpose.RESET_PASSWORD,
+        )
+    except EmailNotConfiguredError as exc:
+        raise HTTPException(
+            status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
+            detail={"code": exc.code, "message": str(exc)},
+        ) from exc
+    except EmailError as exc:
+        logger.warning_event(
+            "SMTP send failed",
+            event="auth.email.send_failed",
+            context={
+                "email": payload.email,
+                "purpose": OtpPurpose.RESET_PASSWORD.value,
+                "error": str(exc),
+            },
+        )
+    return {"status": "ok"}
+
+
+@router.post("/reset-password")
+async def reset_password(payload: ResetPasswordRequest, db: AsyncSession = Depends(get_db)):
+    user = await _load_user_by_email(db, payload.email)
+    if user is None:
+        raise HTTPException(
+            status_code=status.HTTP_400_BAD_REQUEST,
+            detail={"code": "OTP_INVALID"},
+        )
+    try:
+        otp.verify_code(payload.email, OtpPurpose.RESET_PASSWORD, payload.code)
+    except otp.OtpExpired as exc:
+        raise HTTPException(
+            status_code=status.HTTP_410_GONE,
+            detail={"code": exc.code, "message": str(exc)},
+        ) from exc
+    except otp.OtpAttemptsExceeded as exc:
+        raise HTTPException(
+            status_code=status.HTTP_429_TOO_MANY_REQUESTS,
+            detail={"code": exc.code, "message": str(exc)},
+        ) from exc
+    except otp.OtpInvalid as exc:
+        raise HTTPException(
+            status_code=status.HTTP_400_BAD_REQUEST,
+            detail={"code": exc.code, "message": str(exc)},
+        ) from exc
+
+    await db.execute(
+        text("UPDATE users SET password_hash = :p, email_verified = TRUE WHERE id = :id"),
+        {"p": get_password_hash(payload.new_password), "id": user.id},
+    )
+    await db.commit()
+    return {"status": "ok"}

backend/app/api/v1/bgp.py

@@ -5,13 +5,26 @@ from fastapi import APIRouter, Depends, HTTPException, Query
 from sqlalchemy import func, select
 from sqlalchemy.ext.asyncio import AsyncSession
 
+from pydantic import BaseModel
+
 from app.core.security import get_current_user
 from app.db.session import get_db
 from app.models.bgp_anomaly import BGPAnomaly
 from app.models.bgp_incident import BGPIncident
 from app.models.bgp_observation import BGPObservation
 from app.models.user import User
+from app.services.bgp_collector_locations import (
+    build_bgp_collector_location_query,
+    collect_bgp_collector_location_candidates,
+    get_bgp_collector_location_dict,
+)
 from app.services.bgp_collectors import build_bgp_collector_coverage
+from app.services.ai_client import get_ai_provider_client
+from app.api.v1.settings import get_web_search_client
+from app.services.location.llm_fallback import (
+    collect_llm_location_fallback_candidate,
+    collect_location_search_evidence,
+)
 
 router = APIRouter()
 
@@ -264,6 +277,120 @@ async def get_bgp_collector_summary(
     }
 
 
+class CollectBGPCollectorLocationRequest(BaseModel):
+    city: Optional[str] = None
+    country: Optional[str] = None
+    site: Optional[str] = None
+    operator: Optional[str] = None
+
+
+@router.post("/collectors/{collector_id}/collect-location")
+async def collect_bgp_collector_location(
+    collector_id: str,
+    payload: CollectBGPCollectorLocationRequest,
+    current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    """Run the shared location pipeline for a BGP route collector.
+
+    Mirrors ``POST /api/v1/visualization/compute-centers/{source_id}/collect-location``.
+    Returns ranked candidates from source coordinates and Nominatim queries
+    built around the collector's stored context (IXP / city / country). Stored
+    collector locations provide context only; they are not emitted as
+    candidates.
+    """
+    if not collector_id or not collector_id.strip():
+        raise HTTPException(status_code=400, detail="collector_id is required")
+
+    legacy = get_bgp_collector_location_dict(collector_id) or {}
+    site = payload.site or legacy.get("matched_location_name")
+    city = payload.city or legacy.get("city")
+    country = payload.country or legacy.get("country")
+    operator = payload.operator or "RIPE NCC"
+
+    candidates, attempted_queries = collect_bgp_collector_location_candidates(
+        collector=collector_id,
+        site=site,
+        city=city,
+        country=country,
+        operator=operator,
+    )
+    llm_failure_reason = None
+    if not candidates:
+        query = build_bgp_collector_location_query(
+            collector=collector_id,
+            site=site,
+            city=city,
+            country=country,
+            operator=operator,
+        )
+        llm_result = None
+        try:
+            web_search_client = await get_web_search_client(db)
+            search_result = await collect_location_search_evidence(
+                web_search_client=web_search_client,
+                query=query,
+                entity_type="bgp_collector",
+            )
+            attempted_queries = [*attempted_queries, *search_result.attempted_queries]
+            if not search_result.evidence:
+                llm_failure_reason = search_result.failure_reason
+                raise RuntimeError(search_result.failure_reason or "no WebSearch evidence")
+            provider_client = await get_ai_provider_client(db)
+            llm_result = await collect_llm_location_fallback_candidate(
+                provider_client=provider_client,
+                query=query,
+                entity_type="bgp_collector",
+                db=db,
+                attempted_queries=attempted_queries,
+                search_evidence=search_result.evidence,
+            )
+        except Exception as exc:
+            if llm_failure_reason is None:
+                llm_failure_reason = f"LLM location factcheck unavailable: {exc}"
+                attempted_queries = [
+                    *attempted_queries,
+                    f"llm_factcheck:bgp_collector:{collector_id or 'unknown'}",
+                ]
+        if llm_result is not None:
+            attempted_queries = [*attempted_queries, *llm_result.attempted_queries]
+            candidates = llm_result.candidates
+            llm_failure_reason = llm_result.failure_reason
+
+    context = {
+        "collector": collector_id,
+        "site": site,
+        "city": city,
+        "country": country,
+        "operator": operator,
+    }
+
+    if not candidates:
+        return {
+            "collector_id": collector_id,
+            "name": collector_id,
+            "success": False,
+            "failure_reason": (
+                "No source coordinates or online geocoding result reached"
+                " city-level precision for this collector."
+            ),
+            "candidates": [],
+            "attempted_queries": list(attempted_queries),
+            "llm_failure_reason": llm_failure_reason,
+            "context": context,
+        }
+
+    return {
+        "collector_id": collector_id,
+        "name": collector_id,
+        "success": True,
+        "candidates": [candidate.to_dict() for candidate in candidates],
+        "best_candidate": candidates[0].to_dict(),
+        "attempted_queries": list(attempted_queries),
+        "context": context,
+    }
+
+
 @router.get("/overview/summary")
 async def get_bgp_overview_summary(
     current_user: User = Depends(get_current_user),

backend/app/api/v1/collected_data.py

@@ -102,7 +102,12 @@ def build_search_rank_sql(search: Optional[str]) -> str:
     """
 
 
-def serialize_collected_row(row, source_name_map: dict[str, str] | None = None) -> dict:
+def serialize_collected_row(
+    row,
+    source_name_map: dict[str, str] | None = None,
+    *,
+    include_metadata: bool = True,
+) -> dict:
     metadata = row[7]
     source = row[1]
     return {
@@ -120,7 +125,7 @@ def serialize_collected_row(row, source_name_map: dict[str, str] | None = None)
         "longitude": get_metadata_field(metadata, "longitude"),
         "value": get_metadata_field(metadata, "value"),
         "unit": get_metadata_field(metadata, "unit"),
-        "metadata": metadata,
+        "metadata": metadata if include_metadata else None,
         "cores": get_metadata_field(metadata, "cores"),
         "rmax": get_metadata_field(metadata, "rmax"),
         "rpeak": get_metadata_field(metadata, "rpeak"),
@@ -145,6 +150,7 @@ async def list_collected_data(
     search: Optional[str] = Query(None, description="搜索名称"),
     page: int = Query(1, ge=1, description="页码"),
     page_size: int = Query(20, ge=1, le=100, description="每页数量"),
+    include_metadata: bool = Query(True, description="是否返回完整 metadata 字段"),
     current_user: User = Depends(get_current_user),
     db: AsyncSession = Depends(get_db),
 ):
@@ -201,7 +207,7 @@ async def list_collected_data(
 
     data = []
     for row in rows:
-        data.append(serialize_collected_row(row[:11], source_name_map))
+        data.append(serialize_collected_row(row[:11], source_name_map, include_metadata=include_metadata))
 
     return {
         "total": total,

backend/app/api/v1/data_products.py

@@ -0,0 +1,98 @@
+from datetime import UTC, datetime
+
+from fastapi import APIRouter, Depends, HTTPException
+from sqlalchemy.ext.asyncio import AsyncSession
+
+from app.api.v1.visualization import get_visualization_geo_summary
+from app.core.time import to_iso8601_utc
+from app.db.session import get_db
+
+router = APIRouter()
+
+PRODUCT_DEFINITIONS: dict[str, dict] = {
+    "vessels": {
+        "name": "船只",
+        "sources": ["aisstream_vessels", "barentswatch_vessels"],
+        "primary_stat_key": "vessel_count",
+        "stat_keys": ["vessel_count", "vessel_raw_unique_mmsi", "vessel_legacy_unique_mmsi"],
+    },
+    "cables": {
+        "name": "海底光缆",
+        "sources": [
+            "arcgis_cables",
+            "arcgis_landing_points",
+            "arcgis_cable_landing_relation",
+            "telegeography_cables",
+            "telegeography_landing",
+            "telegeography_systems",
+            "fao_landing_points",
+        ],
+        "primary_stat_key": "cable_count",
+        "stat_keys": ["cable_count", "landing_point_count"],
+    },
+    "satellites": {
+        "name": "卫星",
+        "sources": ["celestrak_tle", "spacetrack_tle"],
+        "primary_stat_key": "satellite_count",
+        "stat_keys": ["satellite_count"],
+    },
+    "bgp": {
+        "name": "BGP",
+        "sources": [
+            "ris_live_bgp",
+            "bgpstream_bgp",
+            "iptoasn_prefix_geo",
+            "opengeofeed_prefix_geo",
+            "nro_delegated_prefix_geo",
+        ],
+        "primary_stat_key": "bgp_event_count",
+        "stat_keys": ["bgp_event_count", "bgp_incident_count", "bgp_anomaly_count", "bgp_collector_count"],
+    },
+    "compute": {
+        "name": "算力",
+        "sources": ["top500", "epoch_ai_gpu"],
+        "primary_stat_key": "compute_center_count",
+        "stat_keys": ["compute_center_count", "supercomputer_count", "gpu_cluster_count"],
+    },
+}
+
+
+def _build_product_status(product_id: str, summary: dict) -> dict:
+    definition = PRODUCT_DEFINITIONS[product_id]
+    stats = summary.get("stats", {})
+    product_stats = {key: stats.get(key, 0) for key in definition["stat_keys"]}
+    total_count = int(product_stats.get(definition["primary_stat_key"]) or 0)
+    return {
+        "product_id": product_id,
+        "name": definition["name"],
+        "sources": definition["sources"],
+        "generated_at": summary.get("generated_at") or to_iso8601_utc(datetime.now(UTC)),
+        "total_count": total_count,
+        "stats": product_stats,
+        "build_state": "ready",
+        "stats_scope": "global",
+        "stats_freshness": "cached_or_indexed",
+    }
+
+
+@router.get("")
+async def list_data_products(db: AsyncSession = Depends(get_db)):
+    summary = await get_visualization_geo_summary(db)
+    return {
+        "generated_at": summary.get("generated_at"),
+        "data": [
+            _build_product_status(product_id, summary)
+            for product_id in PRODUCT_DEFINITIONS
+        ],
+    }
+
+
+@router.get("/{product_id}/status")
+async def get_data_product_status(
+    product_id: str,
+    db: AsyncSession = Depends(get_db),
+):
+    if product_id not in PRODUCT_DEFINITIONS:
+        raise HTTPException(status_code=404, detail="Unknown data product")
+    summary = await get_visualization_geo_summary(db)
+    return _build_product_status(product_id, summary)

backend/app/api/v1/docs.py

@@ -0,0 +1,102 @@
+"""Authenticated documentation APIs."""
+
+from __future__ import annotations
+
+from fastapi import APIRouter, Depends, HTTPException, status
+from fastapi.security import HTTPAuthorizationCredentials, HTTPBearer
+from sqlalchemy import text
+
+from app.core.security import decode_token
+from app.db.session import async_session_factory
+from app.models.user import User
+from app.services.docs_gatekeeper import (
+    DOCS_BY_SLUG,
+    VALID_DOCS_LANGS,
+    can_read_doc,
+    catalog_for_user,
+    doc_path_for,
+    title_for,
+)
+
+router = APIRouter()
+optional_bearer = HTTPBearer(auto_error=False)
+
+
+async def get_optional_current_user(
+    credentials: HTTPAuthorizationCredentials | None = Depends(optional_bearer),
+) -> User | None:
+    if credentials is None:
+        return None
+
+    payload = decode_token(credentials.credentials)
+    if payload is None or payload.get("type") != "access" or payload.get("sub") is None:
+        raise HTTPException(
+            status_code=status.HTTP_401_UNAUTHORIZED,
+            detail="Invalid token",
+        )
+
+    async with async_session_factory() as db:
+        result = await db.execute(
+            text(
+                "SELECT id, username, email, password_hash, role, is_active, gatekeeper_groups FROM users WHERE id = :id"
+            ),
+            {"id": int(payload["sub"])},
+        )
+        row = result.fetchone()
+    if row is None or not row[5]:
+        raise HTTPException(
+            status_code=status.HTTP_401_UNAUTHORIZED,
+            detail="User not found or inactive",
+        )
+
+    user = User()
+    user.id = row[0]
+    user.username = row[1]
+    user.email = row[2]
+    user.password_hash = row[3]
+    user.role = row[4]
+    user.is_active = row[5]
+    user.gatekeeper_groups = row[6] or []
+    return user
+
+
+@router.get("/catalog")
+async def get_docs_catalog(current_user: User | None = Depends(get_optional_current_user)):
+    return {
+        "items": catalog_for_user(current_user),
+        "authenticated": current_user is not None,
+    }
+
+
+@router.get("/{lang}/{slug}")
+async def get_doc_content(
+    lang: str,
+    slug: str,
+    current_user: User | None = Depends(get_optional_current_user),
+):
+    if lang not in VALID_DOCS_LANGS:
+        raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Document not found")
+
+    entry = DOCS_BY_SLUG.get(slug)
+    if entry is None:
+        raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Document not found")
+
+    path = doc_path_for(entry, lang)
+    if not path.exists():
+        raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Document not found")
+
+    if not can_read_doc(entry, current_user):
+        if current_user is None:
+            raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail="Authentication required")
+        raise HTTPException(status_code=status.HTTP_403_FORBIDDEN, detail="Insufficient Docs permissions")
+
+    return {
+        "slug": entry.slug,
+        "filename": entry.filename,
+        "lang": lang,
+        "title": title_for(entry, lang),
+        "group": entry.group,
+        "order": entry.order,
+        "access": entry.access,
+        "markdown": path.read_text(encoding="utf-8"),
+    }

backend/app/api/v1/earth.py

@@ -0,0 +1,682 @@
+"""Earth asset management APIs."""
+
+from __future__ import annotations
+
+from pathlib import Path
+from typing import Any
+from uuid import uuid4
+
+from fastapi import APIRouter, Depends, File, Form, HTTPException, Query, Request, UploadFile, status
+from fastapi.security import HTTPAuthorizationCredentials, HTTPBearer
+from pydantic import BaseModel, Field
+from sqlalchemy import delete, func, select, text
+from sqlalchemy.ext.asyncio import AsyncSession
+
+from app.core.config import settings as app_settings
+from app.core.security import decode_token, get_current_user, redis_client
+from app.db.session import get_db
+from app.models.collected_data import CollectedData
+from app.models.datasource import DataSource
+from app.models.datasource_config import DataSourceConfig
+from app.models.system_setting import SystemSetting
+from app.models.user import User
+from app.services.tv_streams import get_tv_settings_payload
+from app.services.earth_news import (
+    get_earth_news_sources_payload,
+    reset_earth_news_sources_payload,
+    save_earth_news_sources_payload,
+    test_news_source_config,
+)
+from app.services.earth_news_manual import (
+    broadcast_manual_news_changed,
+    create_manual_news_group,
+    delete_manual_news_item,
+    get_news_record_or_404,
+    import_manual_news_items,
+    list_news_groups,
+    list_news_records,
+    parse_manual_news_import_upload,
+    rename_manual_news_group,
+    reprocess_manual_news_item,
+    serialize_news_record,
+    upsert_manual_news_item,
+)
+from app.services.earth_boundaries import (
+    EarthBoundaryBuildError,
+    get_boundary_build_status,
+    get_boundary_status,
+    save_boundary_config,
+    start_boundary_build_job,
+)
+
+
+router = APIRouter()
+optional_bearer = HTTPBearer(auto_error=False)
+REPO_ROOT = Path(__file__).resolve().parents[4]
+EARTH_BRAND_ASSET_DIR = REPO_ROOT / "data" / "earth-brand"
+EARTH_BRAND_ASSET_URL_PREFIX = "/earth-brand-assets"
+EARTH_BRAND_CATEGORY = "earth_brand"
+EARTH_ABOUT_CATEGORY = "earth_about"
+SYSTEM_SETTINGS_CATEGORY = "system"
+MAX_EARTH_BRAND_ASSET_BYTES = 3 * 1024 * 1024
+ALLOWED_EARTH_BRAND_EXTENSIONS = {".png", ".jpg", ".jpeg", ".webp", ".svg"}
+
+
+def _app_version_label() -> str:
+    version = str(app_settings.VERSION or "").strip() or "0.0.0"
+    return version if version.startswith("v") else f"v{version}"
+
+
+DEFAULT_EARTH_BRAND = {
+    "logo_src": "/earth/assets/brand/earth-logo.png",
+    "title_src": "/earth/assets/brand/title-zh.png",
+    "title_text": "智能星球计划",
+    "subtitle": "现实层宇宙全息感知系统",
+    "description": "卫星 · 海底光缆 · 算力基础设施",
+    "aria_label": "智能星球计划品牌标识",
+    "title_alt": "智能星球计划",
+}
+
+DEFAULT_EARTH_ABOUT = {
+    "logo_src": "/earth/assets/brand/lim-logo.png",
+    "kicker": "About",
+    "title": "智能星球计划",
+    "version": _app_version_label(),
+    "description": "面向临空场景下的智能媒体研究、全球态势感知与多源开放数据巡航，提供可视化观测、事件聚合与交互式探索能力。",
+    "meta": [
+        {"label": "出品方", "value": "浙江大学临空智能媒体研究院"},
+        {"label": "策划人", "value": "方兴东、黄柳青"},
+        {"label": "产品兼开发者", "value": "钱坤、张鸽、齐鹏"},
+    ],
+}
+EARTH_ABOUT_LEGACY_PLANNER_VALUE = "黄柳青"
+
+
+class EarthBoundaryConfigPayload(BaseModel):
+    config: dict[str, Any] = Field(default_factory=dict)
+
+
+class EarthBrandPayload(BaseModel):
+    logo_src: str = Field(default=DEFAULT_EARTH_BRAND["logo_src"], max_length=1000)
+    title_src: str = Field(default=DEFAULT_EARTH_BRAND["title_src"], max_length=1000)
+    title_text: str = Field(default=DEFAULT_EARTH_BRAND["title_text"], max_length=120)
+    subtitle: str = Field(default=DEFAULT_EARTH_BRAND["subtitle"], max_length=160)
+    description: str = Field(default=DEFAULT_EARTH_BRAND["description"], max_length=200)
+    aria_label: str = Field(default=DEFAULT_EARTH_BRAND["aria_label"], max_length=200)
+    title_alt: str = Field(default=DEFAULT_EARTH_BRAND["title_alt"], max_length=200)
+
+
+class EarthAboutMetaItem(BaseModel):
+    label: str = Field(default="", max_length=80)
+    value: str = Field(default="", max_length=240)
+
+
+class EarthAboutPayload(BaseModel):
+    logo_src: str = Field(default=DEFAULT_EARTH_ABOUT["logo_src"], max_length=1000)
+    kicker: str = Field(default=DEFAULT_EARTH_ABOUT["kicker"], max_length=80)
+    title: str = Field(default=DEFAULT_EARTH_ABOUT["title"], max_length=160)
+    version: str = Field(default=DEFAULT_EARTH_ABOUT["version"], max_length=80)
+    description: str = Field(default=DEFAULT_EARTH_ABOUT["description"], max_length=800)
+    meta: list[EarthAboutMetaItem] = Field(default_factory=list)
+
+
+class EarthNewsSourcesPayload(BaseModel):
+    cache_version: int | None = None
+    source_tags: list[dict[str, Any]] = Field(default_factory=list)
+    categories: list[dict[str, Any]] = Field(default_factory=list)
+    item_tag_rules: list[dict[str, Any]] = Field(default_factory=list)
+    sources: list[dict[str, Any]] = Field(default_factory=list)
+    health: dict[str, Any] = Field(default_factory=dict)
+
+
+class EarthNewsSourceTestPayload(BaseModel):
+    source: dict[str, Any] = Field(default_factory=dict)
+
+
+class EarthNewsManualItemPayload(BaseModel):
+    title: str = Field(default="", max_length=500)
+    summary: str = Field(default="", max_length=1200)
+    content: str = Field(default="", max_length=12000)
+    url: str = Field(default="", max_length=2000)
+    source: str = Field(default="", max_length=255)
+    region: str = Field(default="global", max_length=80)
+    published_at: str | None = None
+    category: str = Field(default="other", max_length=80)
+    tags: list[str] = Field(default_factory=list)
+    location: dict[str, Any] | None = None
+    homepage_url: str = Field(default="", max_length=2000)
+    content_language: str = Field(default="", max_length=32)
+    group_id: str | None = Field(default=None, max_length=120)
+
+
+class EarthNewsManualGroupPayload(BaseModel):
+    name: str = Field(default="", max_length=120)
+
+
+def _normalize_earth_brand_payload(payload: dict[str, Any] | None) -> dict[str, str]:
+    merged = DEFAULT_EARTH_BRAND.copy()
+    if payload:
+        for key in DEFAULT_EARTH_BRAND:
+            value = payload.get(key)
+            if value is not None:
+                merged[key] = str(value).strip()
+
+    if not merged["title_text"]:
+        merged["title_text"] = DEFAULT_EARTH_BRAND["title_text"]
+    if not merged["aria_label"]:
+        merged["aria_label"] = merged["title_text"]
+    if not merged["title_alt"]:
+        merged["title_alt"] = merged["title_text"]
+    return merged
+
+
+def _normalize_earth_about_payload(payload: dict[str, Any] | None) -> dict[str, Any]:
+    merged: dict[str, Any] = {
+        key: value
+        for key, value in DEFAULT_EARTH_ABOUT.items()
+        if key != "meta"
+    }
+    raw_meta = DEFAULT_EARTH_ABOUT["meta"]
+    if payload:
+        for key in ("logo_src", "kicker", "title", "description"):
+            value = payload.get(key)
+            if value is not None:
+                merged[key] = str(value).strip()
+        raw_meta = payload.get("meta") if isinstance(payload.get("meta"), list) else raw_meta
+    merged["version"] = _app_version_label()
+
+    for key, default_value in DEFAULT_EARTH_ABOUT.items():
+        if key == "meta":
+            continue
+        if not merged.get(key):
+            merged[key] = default_value
+
+    normalized_meta: list[dict[str, str]] = []
+    for item in raw_meta:
+        if not isinstance(item, dict):
+            continue
+        label = str(item.get("label") or "").strip()
+        value = str(item.get("value") or "").strip()
+        if label == "策划人" and value == EARTH_ABOUT_LEGACY_PLANNER_VALUE:
+            value = "方兴东、黄柳青"
+        if label or value:
+            normalized_meta.append({"label": label, "value": value})
+    if not normalized_meta:
+        normalized_meta = [dict(item) for item in DEFAULT_EARTH_ABOUT["meta"]]
+    merged["meta"] = normalized_meta
+    return merged
+
+
+def _is_demo_mode_enabled(payload: Any) -> bool:
+    return bool(payload.get("demo_mode")) if isinstance(payload, dict) else False
+
+
+async def _get_earth_brand_record(db: AsyncSession) -> SystemSetting | None:
+    result = await db.execute(
+        select(SystemSetting).where(SystemSetting.category == EARTH_BRAND_CATEGORY)
+    )
+    return result.scalar_one_or_none()
+
+
+async def _get_earth_brand_payload(db: AsyncSession) -> dict[str, Any]:
+    record = await _get_earth_brand_record(db)
+    return {
+        "brand": _normalize_earth_brand_payload(record.payload if record else None),
+        "is_default": record is None,
+    }
+
+
+async def _get_earth_about_record(db: AsyncSession) -> SystemSetting | None:
+    result = await db.execute(
+        select(SystemSetting).where(SystemSetting.category == EARTH_ABOUT_CATEGORY)
+    )
+    return result.scalar_one_or_none()
+
+
+async def _get_earth_about_payload(db: AsyncSession) -> dict[str, Any]:
+    record = await _get_earth_about_record(db)
+    return {
+        "about": _normalize_earth_about_payload(record.payload if record else None),
+        "is_default": record is None,
+    }
+
+
+async def _get_optional_current_user(
+    credentials: HTTPAuthorizationCredentials | None = Depends(optional_bearer),
+    db: AsyncSession = Depends(get_db),
+) -> User | None:
+    if credentials is None:
+        return None
+    token = credentials.credentials
+    if redis_client.sismember("blacklisted_tokens", token):
+        return None
+    payload = decode_token(token)
+    if payload is None or payload.get("type") != "access":
+        return None
+    user_id = payload.get("sub")
+    if user_id is None:
+        return None
+    result = await db.execute(
+        text(
+            "SELECT id, username, email, password_hash, role, is_active, gatekeeper_groups FROM users WHERE id = :id"
+        ),
+        {"id": int(user_id)},
+    )
+    row = result.fetchone()
+    if row is None or not row[5]:
+        return None
+    user = User()
+    user.id = row[0]
+    user.username = row[1]
+    user.email = row[2]
+    user.password_hash = row[3]
+    user.role = row[4]
+    user.is_active = row[5]
+    user.gatekeeper_groups = row[6] or []
+    return user
+
+
+@router.get("/brand")
+async def get_earth_brand(db: AsyncSession = Depends(get_db)):
+    return await _get_earth_brand_payload(db)
+
+
+@router.put("/brand")
+async def update_earth_brand(
+    payload: EarthBrandPayload,
+    _current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    normalized = _normalize_earth_brand_payload(payload.model_dump())
+    record = await _get_earth_brand_record(db)
+    if record is None:
+        record = SystemSetting(category=EARTH_BRAND_CATEGORY, payload=normalized)
+        db.add(record)
+    else:
+        record.payload = normalized
+    await db.commit()
+    await db.refresh(record)
+    return {"status": "updated", "brand": _normalize_earth_brand_payload(record.payload), "is_default": False}
+
+
+@router.delete("/brand")
+@router.post("/brand/reset")
+async def reset_earth_brand(
+    _current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    await db.execute(delete(SystemSetting).where(SystemSetting.category == EARTH_BRAND_CATEGORY))
+    await db.commit()
+    return {"status": "reset", "brand": DEFAULT_EARTH_BRAND.copy(), "is_default": True}
+
+
+@router.post("/brand/assets")
+async def upload_earth_brand_asset(
+    file: UploadFile = File(...),
+    _current_user: User = Depends(get_current_user),
+):
+    original_name = file.filename or ""
+    extension = Path(original_name).suffix.lower()
+    if extension not in ALLOWED_EARTH_BRAND_EXTENSIONS:
+        raise HTTPException(
+            status_code=400,
+            detail={
+                "code": "unsupported_file_type",
+                "message": "Only png, jpg, jpeg, webp, and svg brand assets are supported.",
+            },
+        )
+
+    content = await file.read(MAX_EARTH_BRAND_ASSET_BYTES + 1)
+    if len(content) > MAX_EARTH_BRAND_ASSET_BYTES:
+        raise HTTPException(
+            status_code=400,
+            detail={
+                "code": "file_too_large",
+                "message": "Brand asset must be 3 MB or smaller.",
+            },
+        )
+
+    EARTH_BRAND_ASSET_DIR.mkdir(parents=True, exist_ok=True)
+    safe_name = f"{uuid4().hex}{extension}"
+    destination = EARTH_BRAND_ASSET_DIR / safe_name
+    destination.write_bytes(content)
+    asset_url = f"{EARTH_BRAND_ASSET_URL_PREFIX}/{safe_name}"
+    return {"url": asset_url, "filename": safe_name, "content_type": file.content_type}
+
+
+@router.get("/about")
+async def get_earth_about(db: AsyncSession = Depends(get_db)):
+    return await _get_earth_about_payload(db)
+
+
+@router.put("/about")
+async def update_earth_about(
+    payload: EarthAboutPayload,
+    _current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    normalized = _normalize_earth_about_payload(payload.model_dump())
+    record = await _get_earth_about_record(db)
+    if record is None:
+        record = SystemSetting(category=EARTH_ABOUT_CATEGORY, payload=normalized)
+        db.add(record)
+    else:
+        record.payload = normalized
+    await db.commit()
+    await db.refresh(record)
+    return {"status": "updated", "about": _normalize_earth_about_payload(record.payload), "is_default": False}
+
+
+@router.delete("/about")
+async def reset_earth_about(
+    _current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    await db.execute(delete(SystemSetting).where(SystemSetting.category == EARTH_ABOUT_CATEGORY))
+    await db.commit()
+    return {"status": "reset", "about": _normalize_earth_about_payload(None), "is_default": True}
+
+
+@router.get("/news-sources")
+async def get_earth_news_sources(db: AsyncSession = Depends(get_db)):
+    return await get_earth_news_sources_payload(db)
+
+
+@router.put("/news-sources")
+async def update_earth_news_sources(
+    payload: EarthNewsSourcesPayload,
+    _current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    return await save_earth_news_sources_payload(db, payload.model_dump())
+
+
+@router.delete("/news-sources")
+@router.post("/news-sources/reset")
+async def reset_earth_news_sources(
+    _current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    return await reset_earth_news_sources_payload(db)
+
+
+@router.post("/news-sources/test")
+async def test_earth_news_source(
+    payload: EarthNewsSourceTestPayload,
+    _current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    return await test_news_source_config(payload.source, db=db)
+
+
+@router.get("/news-groups")
+async def list_earth_news_groups_admin(
+    _current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    return await list_news_groups(db)
+
+
+@router.post("/news-groups")
+async def create_earth_news_group_admin(
+    payload: EarthNewsManualGroupPayload,
+    _current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    try:
+        group = await create_manual_news_group(db, payload.name)
+    except ValueError as exc:
+        raise HTTPException(status_code=422, detail=str(exc)) from exc
+    await db.commit()
+    return {"status": "ok", "group": group}
+
+
+@router.put("/news-groups/{group_id:path}")
+async def rename_earth_news_group_admin(
+    group_id: str,
+    payload: EarthNewsManualGroupPayload,
+    _current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    try:
+        group = await rename_manual_news_group(db, group_id, payload.name)
+    except ValueError as exc:
+        raise HTTPException(status_code=422, detail=str(exc)) from exc
+    await db.commit()
+    await broadcast_manual_news_changed()
+    return {"status": "ok", "group": group}
+
+
+@router.get("/news-items")
+async def list_earth_news_items_admin(
+    page: int = Query(1, ge=1),
+    page_size: int = Query(50, ge=1, le=100),
+    source_type: str | None = Query(None),
+    region: str | None = Query(None),
+    category: str | None = Query(None),
+    status_filter: str | None = Query(None, alias="status"),
+    group_id: str | None = Query(None),
+    _current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    return await list_news_records(
+        db,
+        page=page,
+        page_size=page_size,
+        source_type=source_type,
+        region=region,
+        category=category,
+        status_filter=status_filter,
+        group_id=group_id,
+    )
+
+
+@router.post("/news-items")
+async def create_earth_news_item_admin(
+    payload: EarthNewsManualItemPayload,
+    _current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    try:
+        result = await upsert_manual_news_item(db, payload.model_dump())
+    except ValueError as exc:
+        raise HTTPException(status_code=422, detail=str(exc)) from exc
+    except PermissionError as exc:
+        raise HTTPException(status_code=403, detail=str(exc)) from exc
+    await db.commit()
+    await broadcast_manual_news_changed()
+    return {"status": "ok", "created": result.created, "queued": result.queued, "item": serialize_news_record(result.item)}
+
+
+@router.post("/news-items/import")
+async def import_earth_news_items_admin(
+    file: UploadFile = File(...),
+    group_id: str | None = Form(default=None),
+    _current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    try:
+        payload = await parse_manual_news_import_upload(await file.read())
+        result = await import_manual_news_items(db, payload, group_id=group_id)
+    except ValueError as exc:
+        raise HTTPException(status_code=422, detail=str(exc)) from exc
+    await db.commit()
+    await broadcast_manual_news_changed()
+    return {"status": "ok", **result}
+
+
+@router.put("/news-items/{item_id:path}")
+async def update_earth_news_item_admin(
+    item_id: str,
+    payload: EarthNewsManualItemPayload,
+    _current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    existing = await get_news_record_or_404(db, item_id)
+    if existing is None:
+        raise HTTPException(status_code=404, detail="News item not found.")
+    try:
+        result = await upsert_manual_news_item(
+            db,
+            payload.model_dump(),
+            item_id_override=item_id,
+        )
+    except ValueError as exc:
+        raise HTTPException(status_code=422, detail=str(exc)) from exc
+    except PermissionError as exc:
+        raise HTTPException(status_code=403, detail=str(exc)) from exc
+    await db.commit()
+    await broadcast_manual_news_changed()
+    return {"status": "ok", "created": result.created, "queued": result.queued, "item": serialize_news_record(result.item)}
+
+
+@router.delete("/news-items/{item_id:path}")
+async def delete_earth_news_item_admin(
+    item_id: str,
+    _current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    try:
+        deleted = await delete_manual_news_item(db, item_id)
+    except PermissionError as exc:
+        raise HTTPException(status_code=403, detail=str(exc)) from exc
+    if not deleted:
+        raise HTTPException(status_code=404, detail="News item not found.")
+    await db.commit()
+    await broadcast_manual_news_changed()
+    return {"status": "deleted", "id": item_id}
+
+
+@router.post("/news-items/{item_id:path}/reprocess")
+async def reprocess_earth_news_item_admin(
+    item_id: str,
+    _current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    existing = await get_news_record_or_404(db, item_id)
+    if existing is None:
+        raise HTTPException(status_code=404, detail="News item not found.")
+    try:
+        queued = await reprocess_manual_news_item(db, item_id)
+    except PermissionError as exc:
+        raise HTTPException(status_code=403, detail=str(exc)) from exc
+    await db.commit()
+    await broadcast_manual_news_changed()
+    return {"status": "queued" if queued else "not_queued", "queued": queued, "id": item_id}
+
+
+@router.get("/oobe-status")
+async def get_earth_oobe_status(
+    current_user: User | None = Depends(_get_optional_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    current_count_result = await db.execute(
+        select(func.count(CollectedData.id)).where(CollectedData.is_current.is_(True))
+    )
+    current_record_count = int(current_count_result.scalar() or 0)
+    system_result = await db.execute(
+        select(SystemSetting).where(SystemSetting.category == SYSTEM_SETTINGS_CATEGORY)
+    )
+    system_record = system_result.scalar_one_or_none()
+    demo_mode = _is_demo_mode_enabled(system_record.payload if system_record else None)
+
+    datasource_count_result = await db.execute(select(func.count(DataSource.id)))
+    datasource_count = int(datasource_count_result.scalar() or 0)
+    active_datasource_count_result = await db.execute(
+        select(func.count(DataSource.id)).where(DataSource.is_active.is_(True))
+    )
+    active_datasource_count = int(active_datasource_count_result.scalar() or 0)
+    config_result = await db.execute(select(func.count(DataSourceConfig.id)))
+    custom_config_count = int(config_result.scalar() or 0)
+
+    tv_payload = await get_tv_settings_payload(db)
+    tv_sources = tv_payload.get("sources") if isinstance(tv_payload, dict) else []
+    tv_source_count = len(tv_sources) if isinstance(tv_sources, list) else 0
+
+    boundary_status = get_boundary_status()
+    has_core_layers = bool(boundary_status.get("ready") or boundary_status.get("available") or boundary_status.get("status") in {"ready", "built", "ok"})
+    has_collected_data = current_record_count > 0
+    ready = has_collected_data
+
+    suggestions: list[str] = []
+    if demo_mode:
+        suggestions.append("演示模式已开启")
+    if not current_user:
+        suggestions.append("登录控制台")
+    if not has_collected_data:
+        suggestions.append("触发数据源采集")
+    if not custom_config_count:
+        suggestions.append("确认采集器配置")
+    if not has_core_layers:
+        suggestions.append("构建或启用 Earth 图层")
+
+    return {
+        "ready": ready,
+        "demo_mode": demo_mode,
+        "authenticated": current_user is not None,
+        "needs_login": current_user is None and not ready and not demo_mode,
+        "has_collected_data": has_collected_data,
+        "has_tv_sources": tv_source_count > 0,
+        "has_core_layers": has_core_layers,
+        "current_record_count": current_record_count,
+        "datasource_count": datasource_count,
+        "active_datasource_count": active_datasource_count,
+        "custom_config_count": custom_config_count,
+        "tv_source_count": tv_source_count,
+        "suggestions": suggestions,
+        "login_url": "/login?next=/datasources",
+        "datasources_url": "/datasources",
+        "collection_url": "/collection-management",
+    }
+
+
+@router.get("/boundaries/status")
+async def get_earth_boundary_status():
+    return get_boundary_status()
+
+def _is_loopback_request(request: Request) -> bool:
+    host = request.client.host if request.client else ""
+    return host in {"127.0.0.1", "::1", "localhost"} or host.startswith("127.")
+
+
+def _require_local_or_user(request: Request, user: User | None) -> None:
+    if user is not None or _is_loopback_request(request):
+        return
+    raise HTTPException(
+        status_code=status.HTTP_401_UNAUTHORIZED,
+        detail="Authentication required outside localhost",
+    )
+
+
+@router.put("/boundaries/config")
+async def update_earth_boundary_config(
+    payload: EarthBoundaryConfigPayload,
+    _current_user: User = Depends(get_current_user),
+):
+    try:
+        return save_boundary_config(payload.config)
+    except EarthBoundaryBuildError as exc:
+        raise HTTPException(
+            status_code=400,
+            detail={"code": exc.code, "message": str(exc), "details": exc.details},
+        ) from exc
+
+
+@router.post("/boundaries/build")
+async def build_earth_boundary_assets(
+    request: Request,
+    current_user: User | None = Depends(_get_optional_current_user),
+):
+    _require_local_or_user(request, current_user)
+    try:
+        return await start_boundary_build_job()
+    except EarthBoundaryBuildError as exc:
+        raise HTTPException(
+            status_code=400,
+            detail={"code": exc.code, "message": str(exc), "details": exc.details},
+        ) from exc
+
+
+@router.get("/boundaries/build/status")
+async def get_earth_boundary_build_status():
+    return get_boundary_build_status()

backend/app/api/v1/interactables.py

@@ -0,0 +1,190 @@
+"""CRUD APIs for persistent Earth interactables."""
+
+from __future__ import annotations
+
+from datetime import UTC, datetime
+from typing import Any
+
+from fastapi import APIRouter, Depends, HTTPException, Query, Response, status
+from pydantic import BaseModel, Field, field_validator
+from sqlalchemy.ext.asyncio import AsyncSession
+
+from app.core.security import get_current_user
+from app.db.session import get_db
+from app.models.earth_interactable import EarthInteractable
+from app.models.user import User
+from app.services.earth_interactables import (
+    build_interactable_event,
+    interactables_to_geojson,
+    invalidate_interactable_cache,
+    list_interactables,
+    normalize_interactable_id,
+    publish_interactable_event,
+    serialize_interactable,
+)
+from app.services.earth_layer_cache import (
+    EarthLayerCachePolicy,
+    earth_layer_cache,
+    get_or_build_layer_payload,
+)
+
+router = APIRouter()
+INTERACTABLE_CACHE_POLICY = EarthLayerCachePolicy(
+    fresh_ttl_seconds=60,
+    stale_ttl_seconds=10 * 60,
+    max_features=5000,
+)
+
+
+class InteractableCreate(BaseModel):
+    id: str | None = Field(default=None, max_length=160)
+    layer: str = Field(default="default", min_length=1, max_length=80)
+    kind: str = Field(default="default", min_length=1, max_length=80)
+    label: str = Field(default="", max_length=255)
+    description: str = Field(default="", max_length=4000)
+    latitude: float = Field(ge=-90, le=90)
+    longitude: float = Field(ge=-180, le=180)
+    altitude: float | None = None
+    properties: dict[str, Any] = Field(default_factory=dict)
+
+    @field_validator("layer", "kind")
+    @classmethod
+    def normalize_key(cls, value: str) -> str:
+        normalized = str(value or "").strip()
+        if not normalized:
+            raise ValueError("must not be empty")
+        return normalized
+
+
+class InteractableUpdate(BaseModel):
+    layer: str | None = Field(default=None, min_length=1, max_length=80)
+    kind: str | None = Field(default=None, min_length=1, max_length=80)
+    label: str | None = Field(default=None, max_length=255)
+    description: str | None = Field(default=None, max_length=4000)
+    latitude: float | None = Field(default=None, ge=-90, le=90)
+    longitude: float | None = Field(default=None, ge=-180, le=180)
+    altitude: float | None = None
+    properties: dict[str, Any] | None = None
+
+
+@router.get("")
+async def get_interactables(
+    response: Response,
+    layer: str | None = Query(default=None),
+    include_deleted: bool = Query(default=False),
+    db: AsyncSession = Depends(get_db),
+):
+    items = await list_interactables(db, layer=layer, include_deleted=include_deleted)
+    response.headers["X-Planet-Interactables-Count"] = str(len(items))
+    return {"items": [serialize_interactable(item) for item in items]}
+
+
+@router.get("/geojson")
+async def get_interactables_geojson(
+    response: Response,
+    layer: str | None = Query(default=None),
+    db: AsyncSession = Depends(get_db),
+):
+    async def build_payload() -> dict[str, Any]:
+        items = await list_interactables(db, layer=layer)
+        return interactables_to_geojson(items)
+
+    payload = await get_or_build_layer_payload(
+        key=earth_layer_cache.key("interactables", interactable_layer=layer or "all"),
+        policy=INTERACTABLE_CACHE_POLICY,
+        builder=build_payload,
+        response=response,
+    )
+    response.headers["X-Planet-Interactables-Count"] = str(len(payload.get("features") or []))
+    return payload
+
+
+@router.post("", status_code=status.HTTP_201_CREATED)
+async def create_interactable(
+    payload: InteractableCreate,
+    _current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    record_id = normalize_interactable_id(payload.id)
+    existing = await db.get(EarthInteractable, record_id)
+    if existing and not existing.is_deleted:
+        raise HTTPException(status_code=409, detail="Interactable already exists")
+
+    if existing is None:
+        record = EarthInteractable(id=record_id)
+        db.add(record)
+    else:
+        record = existing
+        record.is_deleted = False
+        record.deleted_at = None
+        record.revision += 1
+
+    record.layer = payload.layer
+    record.kind = payload.kind
+    record.label = payload.label
+    record.description = payload.description
+    record.latitude = payload.latitude
+    record.longitude = payload.longitude
+    record.altitude = payload.altitude
+    record.properties = payload.properties
+
+    await db.commit()
+    await db.refresh(record)
+    invalidate_interactable_cache(record.layer)
+    await publish_interactable_event("created", record)
+    return {"item": serialize_interactable(record)}
+
+
+@router.get("/{interactable_id}")
+async def get_interactable(interactable_id: str, db: AsyncSession = Depends(get_db)):
+    record = await db.get(EarthInteractable, interactable_id)
+    if record is None or record.is_deleted:
+        raise HTTPException(status_code=404, detail="Interactable not found")
+    return {"item": serialize_interactable(record)}
+
+
+@router.patch("/{interactable_id}")
+async def update_interactable(
+    interactable_id: str,
+    payload: InteractableUpdate,
+    _current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    record = await db.get(EarthInteractable, interactable_id)
+    if record is None or record.is_deleted:
+        raise HTTPException(status_code=404, detail="Interactable not found")
+
+    previous_layer = record.layer
+    patch = payload.model_dump(exclude_unset=True)
+    for key, value in patch.items():
+        setattr(record, key, value)
+    record.revision += 1
+
+    await db.commit()
+    await db.refresh(record)
+    invalidate_interactable_cache(previous_layer)
+    if record.layer != previous_layer:
+        invalidate_interactable_cache(record.layer)
+    await publish_interactable_event("updated", record)
+    return {"item": serialize_interactable(record)}
+
+
+@router.delete("/{interactable_id}")
+async def delete_interactable(
+    interactable_id: str,
+    _current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    record = await db.get(EarthInteractable, interactable_id)
+    if record is None or record.is_deleted:
+        raise HTTPException(status_code=404, detail="Interactable not found")
+
+    record.is_deleted = True
+    record.deleted_at = datetime.now(UTC)
+    record.revision += 1
+    await db.commit()
+    await db.refresh(record)
+    invalidate_interactable_cache(record.layer)
+    await publish_interactable_event("deleted", record)
+    event = build_interactable_event(action="deleted", record=record, include_item=True)
+    return {"deleted": True, "event": event}

backend/app/api/v1/layers.py

@@ -0,0 +1,231 @@
+from typing import Any, Optional
+
+from fastapi import APIRouter, Depends, HTTPException, Query, Response
+from sqlalchemy.ext.asyncio import AsyncSession
+
+from app.api.v1.visualization import (
+    _parse_bbox,
+    get_bgp_anomalies_geojson,
+    get_bgp_collectors_geojson,
+    get_bgp_incidents_geojson,
+    get_cables_geojson,
+    get_landing_points_geojson,
+    get_satellites_geojson,
+)
+from app.api.v1.vessels import build_vessel_snapshot_response
+from app.db.session import get_db
+
+router = APIRouter()
+
+DEFAULT_LAYER_LIMIT = 1000
+MAX_LAYER_LIMIT = 5000
+LOW_ZOOM_FEATURE_LIMIT = 500
+
+
+def _clamp_limit(limit: int, zoom: int) -> tuple[int, bool]:
+    clamped = min(max(limit, 1), MAX_LAYER_LIMIT)
+    if zoom <= 3:
+        return min(clamped, LOW_ZOOM_FEATURE_LIMIT), clamped != limit or clamped > LOW_ZOOM_FEATURE_LIMIT
+    return clamped, clamped != limit
+
+
+def _coordinate_in_bbox(coord: Any, bbox: tuple[float, float, float, float]) -> bool:
+    if not isinstance(coord, (list, tuple)) or len(coord) < 2:
+        return False
+    try:
+        lon = float(coord[0])
+        lat = float(coord[1])
+    except (TypeError, ValueError):
+        return False
+    lon_min, lat_min, lon_max, lat_max = bbox
+    return lon_min <= lon <= lon_max and lat_min <= lat <= lat_max
+
+
+def _geometry_intersects_bbox(geometry: dict, bbox: tuple[float, float, float, float]) -> bool:
+    coordinates = geometry.get("coordinates")
+    geometry_type = geometry.get("type")
+    if geometry_type == "Point":
+        return _coordinate_in_bbox(coordinates, bbox)
+    if geometry_type in {"LineString", "MultiPoint"}:
+        return any(_coordinate_in_bbox(coord, bbox) for coord in coordinates or [])
+    if geometry_type in {"Polygon", "MultiLineString"}:
+        return any(
+            _coordinate_in_bbox(coord, bbox)
+            for line in coordinates or []
+            for coord in line
+        )
+    if geometry_type == "MultiPolygon":
+        return any(
+            _coordinate_in_bbox(coord, bbox)
+            for polygon in coordinates or []
+            for line in polygon
+            for coord in line
+        )
+    return False
+
+
+def _guard_geojson_layer(
+    geojson: dict,
+    *,
+    bbox: tuple[float, float, float, float],
+    zoom: int,
+    limit: int,
+) -> dict:
+    bounded_limit, limit_clamped = _clamp_limit(limit, zoom)
+    features = [
+        feature
+        for feature in geojson.get("features", [])
+        if _geometry_intersects_bbox(feature.get("geometry") or {}, bbox)
+    ]
+    visible_count = len(features)
+    returned_features = features[:bounded_limit]
+    return {
+        **geojson,
+        "features": returned_features,
+        "visible_count": visible_count,
+        "returned_count": len(returned_features),
+        "diagnostics": {
+            "bbox_limited": True,
+            "limit": bounded_limit,
+            "limit_clamped": limit_clamped,
+            "truncated": visible_count > len(returned_features),
+            "degraded": zoom <= 3 or visible_count > len(returned_features),
+            "stats_scope": "viewport",
+        },
+    }
+
+
+def _parse_layer_bbox(bbox: str) -> tuple[float, float, float, float]:
+    parsed = _parse_bbox(bbox)
+    if parsed is None:
+        raise HTTPException(status_code=400, detail="bbox is required")
+    return parsed
+
+
+@router.get("/vessels/snapshot")
+async def get_vessel_layer_snapshot(
+    bbox: str = Query(..., description="lon_min,lat_min,lon_max,lat_max"),
+    zoom: float = Query(..., ge=1, le=20),
+    limit: int = Query(DEFAULT_LAYER_LIMIT, ge=1),
+    vessel_type: Optional[str] = Query(None, alias="type"),
+    since_minutes: int = Query(60, ge=1, le=1440),
+    db: AsyncSession = Depends(get_db),
+    response: Response = None,
+):
+    parsed_bbox = _parse_layer_bbox(bbox)
+    return await build_vessel_snapshot_response(
+        db,
+        bbox=parsed_bbox,
+        zoom=zoom,
+        limit=limit,
+        type_filter=vessel_type,
+        since_minutes=since_minutes,
+        response=response,
+    )
+
+
+@router.get("/cables")
+async def get_cable_layer(
+    bbox: str = Query(..., description="lon_min,lat_min,lon_max,lat_max"),
+    zoom: int = Query(..., ge=1, le=20),
+    limit: int = Query(DEFAULT_LAYER_LIMIT, ge=1),
+    db: AsyncSession = Depends(get_db),
+):
+    return _guard_geojson_layer(
+        await get_cables_geojson(db),
+        bbox=_parse_layer_bbox(bbox),
+        zoom=zoom,
+        limit=limit,
+    )
+
+
+@router.get("/landing-points")
+async def get_landing_point_layer(
+    bbox: str = Query(..., description="lon_min,lat_min,lon_max,lat_max"),
+    zoom: int = Query(..., ge=1, le=20),
+    limit: int = Query(DEFAULT_LAYER_LIMIT, ge=1),
+    db: AsyncSession = Depends(get_db),
+):
+    return _guard_geojson_layer(
+        await get_landing_points_geojson(db),
+        bbox=_parse_layer_bbox(bbox),
+        zoom=zoom,
+        limit=limit,
+    )
+
+
+@router.get("/satellites")
+async def get_satellite_layer(
+    bbox: str = Query(..., description="lon_min,lat_min,lon_max,lat_max"),
+    zoom: int = Query(..., ge=1, le=20),
+    limit: int = Query(DEFAULT_LAYER_LIMIT, ge=1),
+    db: AsyncSession = Depends(get_db),
+):
+    bounded_limit, _ = _clamp_limit(limit, zoom)
+    return _guard_geojson_layer(
+        await get_satellites_geojson(limit=bounded_limit, db=db),
+        bbox=_parse_layer_bbox(bbox),
+        zoom=zoom,
+        limit=limit,
+    )
+
+
+@router.get("/bgp/anomalies")
+async def get_bgp_anomaly_layer(
+    bbox: str = Query(..., description="lon_min,lat_min,lon_max,lat_max"),
+    zoom: int = Query(..., ge=1, le=20),
+    limit: int = Query(DEFAULT_LAYER_LIMIT, ge=1),
+    severity: Optional[str] = Query(None),
+    status: Optional[str] = Query("active"),
+    db: AsyncSession = Depends(get_db),
+):
+    bounded_limit, _ = _clamp_limit(limit, zoom)
+    return _guard_geojson_layer(
+        await get_bgp_anomalies_geojson(
+            severity=severity,
+            status=status,
+            limit=bounded_limit,
+            db=db,
+        ),
+        bbox=_parse_layer_bbox(bbox),
+        zoom=zoom,
+        limit=limit,
+    )
+
+
+@router.get("/bgp/incidents")
+async def get_bgp_incident_layer(
+    bbox: str = Query(..., description="lon_min,lat_min,lon_max,lat_max"),
+    zoom: int = Query(..., ge=1, le=20),
+    limit: int = Query(DEFAULT_LAYER_LIMIT, ge=1),
+    severity: Optional[str] = Query(None),
+    status: Optional[str] = Query("active"),
+    db: AsyncSession = Depends(get_db),
+):
+    bounded_limit, _ = _clamp_limit(limit, zoom)
+    return _guard_geojson_layer(
+        await get_bgp_incidents_geojson(
+            severity=severity,
+            status=status,
+            limit=min(bounded_limit, 500),
+            db=db,
+        ),
+        bbox=_parse_layer_bbox(bbox),
+        zoom=zoom,
+        limit=limit,
+    )
+
+
+@router.get("/bgp/collectors")
+async def get_bgp_collector_layer(
+    bbox: str = Query(..., description="lon_min,lat_min,lon_max,lat_max"),
+    zoom: int = Query(..., ge=1, le=20),
+    limit: int = Query(DEFAULT_LAYER_LIMIT, ge=1),
+    db: AsyncSession = Depends(get_db),
+):
+    return _guard_geojson_layer(
+        await get_bgp_collectors_geojson(db),
+        bbox=_parse_layer_bbox(bbox),
+        zoom=zoom,
+        limit=limit,
+    )

backend/app/api/v1/news.py

@@ -1,13 +1,92 @@
-from fastapi import APIRouter, Query
+from fastapi import APIRouter, Depends, HTTPException, Query
+from sqlalchemy.ext.asyncio import AsyncSession
 
-from app.services.earth_news import get_earth_news_payload
+from app.db.session import get_db
+from app.services.earth_news import (
+    ALLOWED_NEWS_CATEGORY_KEYS,
+    SUPPORTED_NEWS_LOCALES,
+    REGION_ANCHORS,
+    get_earth_news_payload,
+)
 
 router = APIRouter()
 
 
+def _parse_categories(raw: str | None) -> set[str] | None:
+    if raw is None or not raw.strip():
+        return None
+    requested = {item.strip().lower() for item in raw.split(",") if item.strip()}
+    invalid = sorted(requested - set(ALLOWED_NEWS_CATEGORY_KEYS))
+    if invalid:
+        raise HTTPException(
+            status_code=422,
+            detail={
+                "message": "Unsupported news categories.",
+                "invalid_categories": invalid,
+                "allowed_categories": list(ALLOWED_NEWS_CATEGORY_KEYS),
+            },
+        )
+    return requested or None
+
+
+def _parse_source_ids(raw: str | None) -> set[str] | None:
+    if raw is None or not raw.strip():
+        return None
+    return {item.strip() for item in raw.split(",") if item.strip()} or None
+
+
+def _parse_limit(raw: int | None) -> int:
+    if raw is None:
+        return 12
+    if raw < 1:
+        raise HTTPException(status_code=422, detail={"message": "News limit must be greater than 0."})
+    return min(raw, 100)
+
+
+def _parse_locale(raw: str | None) -> str:
+    if raw is None or not raw.strip():
+        return "zh-CN"
+    requested = raw.strip()
+    if requested not in SUPPORTED_NEWS_LOCALES:
+        raise HTTPException(
+            status_code=422,
+            detail={
+                "message": "Unsupported news locale.",
+                "invalid_locale": requested,
+                "allowed_locales": sorted(SUPPORTED_NEWS_LOCALES),
+            },
+        )
+    return requested
+
+
 @router.get("/earth-feed")
 async def get_earth_feed(
     lat: float | None = Query(None, description="Current Earth view center latitude"),
     lon: float | None = Query(None, description="Current Earth view center longitude"),
+    region: str | None = Query(None, description="Explicit Earth news region for UE/client integrations"),
+    categories: str | None = Query(None, description="Comma-separated news category keys"),
+    sources: str | None = Query(None, description="Comma-separated news source ids"),
+    limit: int | None = Query(None, description="Maximum news items to return, capped at 100"),
+    locale: str | None = Query(None, description="Display locale, zh-CN or en-US"),
+    db: AsyncSession = Depends(get_db),
 ):
-    return await get_earth_news_payload(lat=lat, lon=lon)
+    normalized_region = region.strip().lower() if isinstance(region, str) and region.strip() else None
+    if normalized_region is not None and normalized_region not in REGION_ANCHORS:
+        raise HTTPException(
+            status_code=422,
+            detail={
+                "message": "Unsupported news region.",
+                "invalid_region": normalized_region,
+                "allowed_regions": list(REGION_ANCHORS.keys()),
+            },
+        )
+    return await get_earth_news_payload(
+        lat=lat,
+        lon=lon,
+        region=normalized_region,
+        categories=_parse_categories(categories),
+        source_ids=_parse_source_ids(sources),
+        limit=_parse_limit(limit),
+        locale=_parse_locale(locale),
+        db=db,
+    )

backend/app/api/v1/realtime_sources.py

@@ -0,0 +1,280 @@
+"""Realtime datasource operations and runtime statistics."""
+
+from datetime import UTC, datetime, timedelta
+import os
+from typing import Any
+
+from fastapi import APIRouter, Depends, HTTPException
+from sqlalchemy import distinct, func, select
+from sqlalchemy.ext.asyncio import AsyncSession
+
+from app.core.data_sources import get_data_sources_config
+from app.core.security import get_current_user
+from app.core.time import to_iso8601_utc
+from app.db.session import get_db
+from app.models.datasource import DataSource
+from app.models.datasource_config import DataSourceConfig
+from app.models.user import User
+from app.models.vessel import AISRawObservation, AISSourceHealth
+from app.services.custom_datasource_runtime import (
+    get_custom_stream_status,
+    start_custom_stream,
+    stop_custom_stream,
+)
+from app.services.scheduler import (
+    cancel_running_collector_now,
+    is_collector_running,
+    run_collector_now,
+)
+from app.services.vessel_ais_aggregation import VESSEL_AIS_SCHEMA, update_ais_source_health
+
+router = APIRouter()
+
+BUILTIN_REALTIME_SOURCES = {"aisstream_vessels"}
+REALTIME_SOURCE_TYPES = {"websocket", "ws"}
+
+
+def _is_realtime_config(config: DataSourceConfig) -> bool:
+    return str(config.source_type or "").lower() in REALTIME_SOURCE_TYPES
+
+
+def _safe_config_dict(value: Any) -> dict[str, Any]:
+    return value if isinstance(value, dict) else {}
+
+
+def _credential_configured(config: DataSourceConfig | None) -> bool:
+    if config is not None:
+        auth_config = _safe_config_dict(config.auth_config)
+        config_payload = _safe_config_dict(config.config)
+        if auth_config.get("api_key") or config_payload.get("api_key"):
+            return True
+    return bool(os.getenv("AISSTREAM_API_KEY"))
+
+
+async def _load_realtime_stats(db: AsyncSession, source: str) -> dict[str, Any]:
+    now = datetime.now(UTC)
+    observed_24h = now - timedelta(hours=24)
+    observed_1h = now - timedelta(hours=1)
+    payload_mmsi = AISRawObservation.entity_key
+
+    result = await db.execute(
+        select(
+            func.count(AISRawObservation.id).label("total_observations"),
+            func.count(AISRawObservation.id)
+            .filter(AISRawObservation.observed_at >= observed_24h)
+            .label("observations_24h"),
+            func.count(AISRawObservation.id)
+            .filter(AISRawObservation.observed_at >= observed_1h)
+            .label("observations_1h"),
+            func.count(distinct(payload_mmsi)).label("unique_mmsi_total"),
+            func.count(distinct(payload_mmsi))
+            .filter(AISRawObservation.observed_at >= observed_24h)
+            .label("unique_mmsi_24h"),
+            func.max(AISRawObservation.observed_at).label("latest_observed_at"),
+            func.max(AISRawObservation.collected_at).label("latest_collected_at"),
+        )
+        .where(AISRawObservation.target_schema == VESSEL_AIS_SCHEMA)
+        .where(AISRawObservation.source == source)
+    )
+    row = result.mappings().one()
+    return {
+        "total_observations": int(row["total_observations"] or 0),
+        "observations_24h": int(row["observations_24h"] or 0),
+        "observations_1h": int(row["observations_1h"] or 0),
+        "unique_mmsi_total": int(row["unique_mmsi_total"] or 0),
+        "unique_mmsi_24h": int(row["unique_mmsi_24h"] or 0),
+        "latest_observed_at": to_iso8601_utc(row["latest_observed_at"]),
+        "latest_collected_at": to_iso8601_utc(row["latest_collected_at"]),
+    }
+
+
+def _runtime_status_for_builtin(source: str) -> dict[str, Any]:
+    running = is_collector_running(source)
+    return {
+        "running": running,
+        "done": False,
+        "runtime": "collector",
+    }
+
+
+def _runtime_status_for_custom(config_id: int) -> dict[str, Any]:
+    status = get_custom_stream_status(config_id)
+    return {
+        "running": bool(status.get("running")),
+        "done": bool(status.get("done")),
+        "runtime": "custom_stream",
+    }
+
+
+async def _serialize_builtin_aisstream(
+    db: AsyncSession,
+    datasource: DataSource,
+    config: DataSourceConfig | None,
+) -> dict[str, Any]:
+    health = await db.get(AISSourceHealth, datasource.source)
+    config_payload = _safe_config_dict(config.config if config else {})
+    endpoint = (
+        (config.endpoint if config else None)
+        or get_data_sources_config().get_yaml_url(datasource.source)
+    )
+    return {
+        "source": datasource.source,
+        "name": datasource.name,
+        "display_name": "AISStream 实时船舶",
+        "kind": "builtin",
+        "source_type": "websocket",
+        "endpoint": endpoint,
+        "is_active": bool(datasource.is_active),
+        "credential_configured": _credential_configured(config),
+        "message_types": config_payload.get("message_types") or ["PositionReport", "ShipStaticData"],
+        "bounding_boxes": config_payload.get("bounding_boxes") or [[[-90, -180], [90, 180]]],
+        "config": config_payload,
+        "runtime": _runtime_status_for_builtin(datasource.source),
+        "health": health.to_dict() if health else None,
+        "stats": await _load_realtime_stats(db, datasource.source),
+    }
+
+
+async def _serialize_custom_stream(
+    db: AsyncSession,
+    config: DataSourceConfig,
+) -> dict[str, Any]:
+    health = await db.get(AISSourceHealth, config.name)
+    config_payload = _safe_config_dict(config.config)
+    return {
+        "source": config.name,
+        "name": config.name,
+        "display_name": config.description or config.name,
+        "kind": "custom",
+        "config_id": config.id,
+        "source_type": config.source_type,
+        "endpoint": config.endpoint,
+        "is_active": bool(config.is_active),
+        "credential_configured": config.auth_type == "none" or bool(_safe_config_dict(config.auth_config)),
+        "message_types": config_payload.get("message_types") or [],
+        "bounding_boxes": config_payload.get("bounding_boxes") or [],
+        "config": config_payload,
+        "runtime": _runtime_status_for_custom(config.id),
+        "health": health.to_dict() if health else None,
+        "stats": await _load_realtime_stats(db, config.name),
+    }
+
+
+async def _load_builtin_aisstream(db: AsyncSession) -> tuple[DataSource | None, DataSourceConfig | None]:
+    result = await db.execute(select(DataSource).where(DataSource.source == "aisstream_vessels"))
+    datasource = result.scalar_one_or_none()
+    config_result = await db.execute(
+        select(DataSourceConfig)
+        .where(DataSourceConfig.name == "aisstream_vessels")
+        .where(DataSourceConfig.is_active.is_(True))
+        .order_by(DataSourceConfig.id.desc())
+        .limit(1)
+    )
+    return datasource, config_result.scalar_one_or_none()
+
+
+async def _load_custom_realtime_config(db: AsyncSession, source: str) -> DataSourceConfig | None:
+    result = await db.execute(
+        select(DataSourceConfig)
+        .where(DataSourceConfig.name == source)
+        .order_by(DataSourceConfig.id.desc())
+        .limit(1)
+    )
+    config = result.scalar_one_or_none()
+    return config if config is not None and _is_realtime_config(config) else None
+
+
+@router.get("")
+async def list_realtime_sources(
+    current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    sources: list[dict[str, Any]] = []
+    datasource, builtin_config = await _load_builtin_aisstream(db)
+    if datasource is not None:
+        sources.append(await _serialize_builtin_aisstream(db, datasource, builtin_config))
+
+    custom_result = await db.execute(
+        select(DataSourceConfig)
+        .where(func.lower(DataSourceConfig.source_type).in_(REALTIME_SOURCE_TYPES))
+        .order_by(DataSourceConfig.name)
+    )
+    for config in custom_result.scalars().all():
+        if config.name in BUILTIN_REALTIME_SOURCES:
+            continue
+        sources.append(await _serialize_custom_stream(db, config))
+
+    return {"total": len(sources), "data": sources}
+
+
+async def _ensure_builtin_startable(db: AsyncSession) -> DataSourceConfig | None:
+    datasource, config = await _load_builtin_aisstream(db)
+    if datasource is None:
+        raise HTTPException(status_code=404, detail="Realtime source not found")
+    if not datasource.is_active:
+        raise HTTPException(status_code=400, detail="Realtime source is disabled")
+    if not _credential_configured(config):
+        raise HTTPException(status_code=400, detail="AISStream API key is not configured")
+    return config
+
+
+@router.post("/{source}/start")
+async def start_realtime_source(
+    source: str,
+    current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    if source == "aisstream_vessels":
+        await _ensure_builtin_startable(db)
+        if is_collector_running(source):
+            return {"status": "already_running", "source": source, "runtime": _runtime_status_for_builtin(source)}
+        if not run_collector_now(source):
+            raise HTTPException(status_code=409, detail="Realtime source could not be started")
+        return {"status": "started", "source": source, "runtime": _runtime_status_for_builtin(source)}
+
+    config = await _load_custom_realtime_config(db, source)
+    if config is None:
+        raise HTTPException(status_code=404, detail="Realtime source not found")
+    if not config.is_active:
+        raise HTTPException(status_code=400, detail="Realtime source is disabled")
+    started = start_custom_stream(config.id)
+    return {
+        "status": "started" if started else "already_running",
+        "source": source,
+        "runtime": _runtime_status_for_custom(config.id),
+    }
+
+
+@router.post("/{source}/stop")
+async def stop_realtime_source(
+    source: str,
+    current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    if source == "aisstream_vessels":
+        stopped = await cancel_running_collector_now(source)
+        await update_ais_source_health(db, source=source, connection_state="disconnected", last_error=None)
+        await db.commit()
+        return {"status": "stopped" if stopped else "not_running", "source": source, "runtime": _runtime_status_for_builtin(source)}
+
+    config = await _load_custom_realtime_config(db, source)
+    if config is None:
+        raise HTTPException(status_code=404, detail="Realtime source not found")
+    stopped = await stop_custom_stream(config.id)
+    await update_ais_source_health(db, source=source, connection_state="disconnected", last_error=None)
+    await db.commit()
+    return {
+        "status": "stopped" if stopped else "not_running",
+        "source": source,
+        "runtime": _runtime_status_for_custom(config.id),
+    }
+
+
+@router.post("/{source}/restart")
+async def restart_realtime_source(
+    source: str,
+    current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    await stop_realtime_source(source, current_user=current_user, db=db)
+    return await start_realtime_source(source, current_user=current_user, db=db)

backend/app/api/v1/system_control.py

@@ -1,15 +1,21 @@
 from __future__ import annotations
 
 import os
+import secrets
 import subprocess
 import sys
 
-from fastapi import APIRouter, Depends, HTTPException, status
+from datetime import datetime
-from pydantic import BaseModel
 
-from app.core.config import ROOT_DIR
+from fastapi import APIRouter, Depends, Header, HTTPException, Query, Request, status
+from pydantic import BaseModel
+from sqlalchemy.ext.asyncio import AsyncSession
+
+from app.core.config import ROOT_DIR, settings
 from app.core.security import get_current_user
+from app.db.session import get_db
 from app.models.user import User
+from app.services.persistent_logs import record_audit_log, record_system_log
 from app.services.system_control import (
     build_task_id,
     clear_active_task_id,
@@ -23,6 +29,20 @@ from app.services.system_control import (
     set_active_task_id,
     upsert_task_state,
 )
+from app.services.system_logs import (
+    DEFAULT_LOG_LINE_LIMIT,
+    MAX_LOG_LINE_LIMIT,
+    SUPPORTED_LOG_LEVELS,
+    append_buffer_log,
+    list_log_sources,
+    normalize_log_level,
+    read_database_log_snapshot,
+    read_log_snapshot,
+    read_observability_group_events,
+    read_observability_groups,
+    read_observability_raw_events,
+)
+from app.services.earth_layer_cache import earth_layer_cache
 
 router = APIRouter()
 
@@ -47,6 +67,162 @@ class RestartTaskLogsResponse(BaseModel):
     lines: list[str]
 
 
+class SystemLogSourceSummary(BaseModel):
+    source_id: str
+    name: str
+    kind: str
+    location: str
+    description: str
+    category: str
+    status: str
+
+
+class SystemLogSourcesResponse(BaseModel):
+    items: list[SystemLogSourceSummary]
+
+
+class SystemLogDailyMarker(BaseModel):
+    date_token: str
+    total: int
+    dominant_level: str
+
+
+class SystemLogSnapshotResponse(BaseModel):
+    source_id: str
+    name: str
+    kind: str
+    location: str
+    description: str
+    category: str
+    status: str
+    level: str
+    selected_levels: list[str] = []
+    search_query: str = ""
+    available_levels: list[str]
+    daily_markers: list[SystemLogDailyMarker] = []
+    line_limit: int
+    line_count: int
+    lines: list[str]
+
+
+class EarthClientLogEventCreate(BaseModel):
+    level: str = "error"
+    message: str
+    category: str | None = None
+    url: str | None = None
+    module: str | None = None
+    detail: str | None = None
+    fingerprint: str | None = None
+    occurrence_count: int = 1
+    metadata: dict[str, object] | None = None
+
+
+class EarthClientLogEventResponse(BaseModel):
+    accepted: bool
+    source_id: str
+    level: str
+    fingerprint: str | None = None
+
+
+class ServiceLogEventCreate(BaseModel):
+    source: str = "ai-provider"
+    service: str = "ai-provider"
+    module: str | None = None
+    category: str | None = None
+    event: str = "service.runtime_log"
+    level: str = "error"
+    message: str
+    fingerprint: str | None = None
+    occurrence_count: int = 1
+    request_id: str | None = None
+    trace_id: str | None = None
+    task_id: str | None = None
+    source_id: int | str | None = None
+    provider: str | None = None
+    context: dict[str, object] | None = None
+
+
+async def ingest_client_log_event(
+    source_id: str,
+    *,
+    service: str,
+    event: str,
+    default_module: str,
+    default_category: str,
+    payload: EarthClientLogEventCreate,
+    request: Request,
+) -> EarthClientLogEventResponse:
+    normalized_level = normalize_log_level(payload.level)
+    append_buffer_log(
+        source_id,
+        level=normalized_level,
+        message=payload.message,
+        context={
+            "category": payload.category or "",
+            "url": payload.url or "",
+            "module": payload.module or "",
+            "detail": payload.detail or "",
+            "fingerprint": payload.fingerprint or "",
+            "occurrence_count": max(1, int(payload.occurrence_count or 1)),
+            "metadata": payload.metadata or {},
+        },
+    )
+    await record_system_log(
+        source=source_id,
+        service=service,
+        module=payload.module or default_module,
+        event=event,
+        level=normalized_level,
+        message=payload.message,
+        category=payload.category or default_category,
+        context={
+            "url": payload.url or "",
+            "detail": payload.detail or "",
+            "module": payload.module or "",
+            "client_ip": request.client.host if request.client else "",
+            "metadata": payload.metadata or {},
+        },
+        fingerprint=payload.fingerprint,
+        occurrence_count=max(1, int(payload.occurrence_count or 1)),
+    )
+    return EarthClientLogEventResponse(accepted=True, source_id=source_id, level=normalized_level, fingerprint=payload.fingerprint)
+
+
+def require_observability_ingest_token(
+    authorization: str | None,
+    ingest_token: str | None,
+) -> None:
+    expected_token = settings.OBSERVABILITY_INGEST_TOKEN.strip()
+    if not expected_token:
+        raise HTTPException(
+            status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
+            detail="Observability service ingestion is not configured",
+        )
+    provided = ""
+    if ingest_token:
+        provided = ingest_token.strip()
+    elif authorization:
+        scheme, _, token = authorization.partition(" ")
+        if scheme.lower() == "bearer":
+            provided = token.strip()
+    if not provided or not secrets.compare_digest(provided, expected_token):
+        raise HTTPException(
+            status_code=status.HTTP_403_FORBIDDEN,
+            detail="Invalid observability ingestion token",
+        )
+
+
+class EarthLayerCacheStatusResponse(BaseModel):
+    prefix: str
+    key_count: int
+    memory_bytes: int
+    layers: dict[str, dict[str, int]]
+
+
+class EarthLayerCacheClearResponse(BaseModel):
+    deleted: int
+
+
 def ensure_super_admin(current_user: User) -> None:
     if not require_super_admin(current_user.role):
         raise HTTPException(
@@ -55,9 +231,50 @@ def ensure_super_admin(current_user: User) -> None:
         )
 
 
+def validate_log_date(raw_value: str | None, field_name: str) -> str | None:
+    if raw_value in {None, ""}:
+        return None
+    try:
+        return datetime.strptime(raw_value, "%Y-%m-%d").date().isoformat()
+    except ValueError as exc:
+        raise HTTPException(
+            status_code=status.HTTP_400_BAD_REQUEST,
+            detail=f"{field_name} must be in YYYY-MM-DD format",
+        ) from exc
+
+
+@router.get("/cache/earth-layers", response_model=EarthLayerCacheStatusResponse)
+async def get_earth_layer_cache_status(
+    current_user: User = Depends(get_current_user),
+):
+    ensure_super_admin(current_user)
+    try:
+        return earth_layer_cache.status()
+    except Exception as exc:
+        raise HTTPException(
+            status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
+            detail=f"Unable to read Earth layer cache status: {exc}",
+        ) from exc
+
+
+@router.delete("/cache/earth-layers", response_model=EarthLayerCacheClearResponse)
+async def clear_earth_layer_cache(
+    current_user: User = Depends(get_current_user),
+):
+    ensure_super_admin(current_user)
+    try:
+        return {"deleted": earth_layer_cache.delete_pattern()}
+    except Exception as exc:
+        raise HTTPException(
+            status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
+            detail=f"Unable to clear Earth layer cache: {exc}",
+        ) from exc
+
+
 @router.post("/restart-tasks", response_model=RestartTaskResponse)
 async def create_restart_task(
     payload: RestartTaskCreate,
+    request: Request,
     current_user: User = Depends(get_current_user),
 ):
     ensure_super_admin(current_user)
@@ -133,11 +350,31 @@ async def create_restart_task(
             requested_by=requested_by,
         )
         clear_active_task_id(task_id)
+        await record_audit_log(
+            action="system.restart_task.requested",
+            actor_id=current_user.id,
+            actor_name=current_user.username,
+            target_type="restart_task",
+            target_id=task_id,
+            result="failed",
+            ip=request.client.host if request.client else None,
+            details={"action": payload.action, "message": task_state["message"]},
+        )
         raise HTTPException(
             status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
             detail=task_state["message"],
         ) from exc
 
+    await record_audit_log(
+        action="system.restart_task.requested",
+        actor_id=current_user.id,
+        actor_name=current_user.username,
+        target_type="restart_task",
+        target_id=task_id,
+        result="accepted",
+        ip=request.client.host if request.client else None,
+        details={"action": payload.action},
+    )
     return task_state
 
 
@@ -165,3 +402,235 @@ async def get_restart_task_logs(
     if task is None:
         raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Restart task not found")
     return {"task_id": task_id, "lines": get_task_logs(task_id)}
+
+
+@router.get("/logs/sources", response_model=SystemLogSourcesResponse)
+async def get_system_log_sources(
+    current_user: User = Depends(get_current_user),
+):
+    ensure_super_admin(current_user)
+    return {
+        "items": [
+            *list_log_sources(),
+            {
+                "source_id": "system-db",
+                "name": "系统事件",
+                "kind": "database",
+                "location": "table://system_logs",
+                "description": "后端持久化系统事件、AI 和采集器操作日志。",
+                "category": "database",
+                "status": "ok",
+            },
+            {
+                "source_id": "audit-db",
+                "name": "审计事件",
+                "kind": "database",
+                "location": "table://audit_logs",
+                "description": "管理员敏感操作和密钥 reveal 审计记录。",
+                "category": "audit",
+                "status": "ok",
+            },
+        ]
+    }
+
+
+@router.get("/logs/observability/groups")
+async def get_observability_log_groups(
+    limit: int = DEFAULT_LOG_LINE_LIMIT,
+    level: str = "all",
+    levels: str | None = Query(None, description="Comma-separated log levels"),
+    start_date: str | None = Query(None, description="Filter logs from this date (YYYY-MM-DD)"),
+    end_date: str | None = Query(None, description="Filter logs until this date (YYYY-MM-DD)"),
+    search: str | None = Query(None, description="Case-insensitive substring search"),
+    current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    ensure_super_admin(current_user)
+    if limit < 1 or limit > MAX_LOG_LINE_LIMIT:
+        raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail=f"limit must be between 1 and {MAX_LOG_LINE_LIMIT}")
+    normalized_start_date = validate_log_date(start_date, "start_date")
+    normalized_end_date = validate_log_date(end_date, "end_date")
+    if normalized_start_date and normalized_end_date and normalized_start_date > normalized_end_date:
+        raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail="start_date must be earlier than or equal to end_date")
+    return await read_observability_groups(
+        limit=limit,
+        level=level,
+        levels=levels,
+        start_date=normalized_start_date,
+        end_date=normalized_end_date,
+        search=search,
+        db=db,
+    )
+
+
+@router.get("/logs/observability/groups/{fingerprint}/events")
+async def get_observability_group_events(
+    fingerprint: str,
+    limit: int = DEFAULT_LOG_LINE_LIMIT,
+    current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    ensure_super_admin(current_user)
+    if limit < 1 or limit > MAX_LOG_LINE_LIMIT:
+        raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail=f"limit must be between 1 and {MAX_LOG_LINE_LIMIT}")
+    payload = await read_observability_group_events(fingerprint, limit=limit, db=db)
+    if payload is None:
+        raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Observability group not found")
+    return payload
+
+
+@router.get("/logs/observability/raw")
+async def get_observability_raw_events(
+    limit: int = DEFAULT_LOG_LINE_LIMIT,
+    level: str = "all",
+    levels: str | None = Query(None, description="Comma-separated log levels"),
+    start_date: str | None = Query(None, description="Filter logs from this date (YYYY-MM-DD)"),
+    end_date: str | None = Query(None, description="Filter logs until this date (YYYY-MM-DD)"),
+    search: str | None = Query(None, description="Case-insensitive substring search"),
+    current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    ensure_super_admin(current_user)
+    if limit < 1 or limit > MAX_LOG_LINE_LIMIT:
+        raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail=f"limit must be between 1 and {MAX_LOG_LINE_LIMIT}")
+    normalized_start_date = validate_log_date(start_date, "start_date")
+    normalized_end_date = validate_log_date(end_date, "end_date")
+    return await read_observability_raw_events(
+        limit=limit,
+        level=level,
+        levels=levels,
+        start_date=normalized_start_date,
+        end_date=normalized_end_date,
+        search=search,
+        db=db,
+    )
+
+
+@router.post("/logs/service", response_model=EarthClientLogEventResponse)
+async def ingest_service_log(
+    payload: ServiceLogEventCreate,
+    authorization: str | None = Header(default=None),
+    ingest_token: str | None = Header(default=None, alias="X-Planet-Observability-Token"),
+):
+    require_observability_ingest_token(authorization, ingest_token)
+    normalized_level = normalize_log_level(payload.level)
+    source = (payload.source or "ai-provider").strip() or "ai-provider"
+    context = dict(payload.context or {})
+    if payload.request_id:
+        context["request_id"] = payload.request_id
+    if payload.trace_id:
+        context["trace_id"] = payload.trace_id
+    if payload.task_id:
+        context["task_id"] = payload.task_id
+    if payload.source_id is not None:
+        context["source_id"] = payload.source_id
+    if payload.provider:
+        context["provider"] = payload.provider
+    await record_system_log(
+        source=source,
+        service=(payload.service or source).strip() or source,
+        module=payload.module or source,
+        event=(payload.event or "service.runtime_log").strip() or "service.runtime_log",
+        level=normalized_level,
+        message=payload.message,
+        category=payload.category or "service-runtime",
+        context=context,
+        fingerprint=payload.fingerprint,
+        occurrence_count=max(1, int(payload.occurrence_count or 1)),
+    )
+    return EarthClientLogEventResponse(
+        accepted=True,
+        source_id=source,
+        level=normalized_level,
+        fingerprint=payload.fingerprint,
+    )
+
+
+@router.get("/logs/{source_id}", response_model=SystemLogSnapshotResponse)
+async def get_system_log_snapshot(
+    source_id: str,
+    limit: int = DEFAULT_LOG_LINE_LIMIT,
+    level: str = "all",
+    levels: str | None = Query(None, description="Comma-separated log levels"),
+    start_date: str | None = Query(None, description="Filter logs from this date (YYYY-MM-DD)"),
+    end_date: str | None = Query(None, description="Filter logs until this date (YYYY-MM-DD)"),
+    search: str | None = Query(None, description="Case-insensitive substring search"),
+    current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    ensure_super_admin(current_user)
+
+    if limit < 1 or limit > MAX_LOG_LINE_LIMIT:
+        raise HTTPException(
+            status_code=status.HTTP_400_BAD_REQUEST,
+            detail=f"limit must be between 1 and {MAX_LOG_LINE_LIMIT}",
+        )
+    if str(level).strip().lower() not in SUPPORTED_LOG_LEVELS and normalize_log_level(level) == "all" and str(level).strip().lower() not in {"", "all"}:
+        raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail="Unsupported log level")
+    if levels:
+        for raw_level in str(levels).split(","):
+            normalized_level = str(raw_level).strip().lower()
+            if not normalized_level:
+                continue
+            if normalized_level not in SUPPORTED_LOG_LEVELS and normalize_log_level(normalized_level) == "all":
+                raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail="Unsupported log level")
+    normalized_start_date = validate_log_date(start_date, "start_date")
+    normalized_end_date = validate_log_date(end_date, "end_date")
+    if normalized_start_date and normalized_end_date and normalized_start_date > normalized_end_date:
+        raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail="start_date must be earlier than or equal to end_date")
+
+    snapshot = await read_database_log_snapshot(
+        source_id,
+        limit=limit,
+        level=level,
+        levels=levels,
+        start_date=normalized_start_date,
+        end_date=normalized_end_date,
+        search=search,
+        db=db,
+    )
+    if snapshot is None:
+        snapshot = read_log_snapshot(
+            source_id,
+            limit,
+            level=level,
+            levels=levels,
+            start_date=normalized_start_date,
+            end_date=normalized_end_date,
+            search=search,
+        )
+    if snapshot is None:
+        raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Log source not found")
+    return snapshot
+
+
+@router.post("/logs/earth-client", response_model=EarthClientLogEventResponse)
+async def ingest_earth_client_log(
+    payload: EarthClientLogEventCreate,
+    request: Request,
+):
+    return await ingest_client_log_event(
+        "earth-client",
+        service="earth",
+        event="earth.client.runtime_log",
+        default_module="earth-client",
+        default_category="client-runtime",
+        payload=payload,
+        request=request,
+    )
+
+
+@router.post("/logs/admin-client", response_model=EarthClientLogEventResponse)
+async def ingest_admin_client_log(
+    payload: EarthClientLogEventCreate,
+    request: Request,
+):
+    return await ingest_client_log_event(
+        "admin-client",
+        service="admin",
+        event="admin.client.runtime_log",
+        default_module="admin-client",
+        default_category="client-runtime",
+        payload=payload,
+        request=request,
+    )

backend/app/api/v1/tasks.py

@@ -27,7 +27,10 @@ async def list_tasks(
     offset = (page - 1) * page_size
     query = """
         SELECT ct.id, ct.datasource_id, ds.name as datasource_name, ct.status,
-               ct.started_at, ct.completed_at, ct.records_processed, ct.error_message
+               ct.started_at, ct.completed_at, ct.records_processed, ct.error_message,
+               ct.phase, ct.phase_progress, ct.phase_message, ct.phase_current,
+               ct.phase_total, ct.phase_unit, ct.total_records, ct.progress,
+               ct.task_type, ct.source, ds.source as datasource_source
         FROM collection_tasks ct
         JOIN data_sources ds ON ct.datasource_id = ds.id
         WHERE 1=1
@@ -37,12 +40,19 @@ async def list_tasks(
 
     if datasource_id:
         query += " AND ct.datasource_id = :datasource_id"
-        count_query += " WHERE ct.datasource_id = :datasource_id"
+        count_query += " AND ct.datasource_id = :datasource_id"
         params["datasource_id"] = datasource_id
     if status:
-        query += " AND ct.status = :status"
+        statuses = [item.strip() for item in status.split(",") if item.strip()]
-        count_query += " AND ct.status = :status"
+        if len(statuses) > 1:
-        params["status"] = status
+            placeholders = ", ".join(f":status_{index}" for index, _item in enumerate(statuses))
+            query += f" AND ct.status IN ({placeholders})"
+            count_query += f" AND ct.status IN ({placeholders})"
+            params.update({f"status_{index}": item for index, item in enumerate(statuses)})
+        else:
+            query += " AND ct.status = :status"
+            count_query += " AND ct.status = :status"
+            params["status"] = statuses[0] if statuses else status
 
     query += f" ORDER BY ct.created_at DESC LIMIT {page_size} OFFSET {offset}"
 
@@ -66,6 +76,17 @@ async def list_tasks(
                 "completed_at": to_iso8601_utc(t[5]),
                 "records_processed": t[6],
                 "error_message": t[7],
+                "phase": t[8],
+                "phase_progress": t[9],
+                "phase_message": t[10],
+                "phase_current": t[11],
+                "phase_total": t[12],
+                "phase_unit": t[13],
+                "total_records": t[14],
+                "progress": t[15],
+                "task_type": t[16],
+                "source": t[17] or t[18],
+                "datasource_source": t[18],
             }
             for t in tasks
         ],
@@ -81,7 +102,9 @@ async def get_task(
     result = await db.execute(
         text("""
             SELECT ct.id, ct.datasource_id, ds.name as datasource_name, ct.status,
-                   ct.started_at, ct.completed_at, ct.records_processed, ct.error_message
+                   ct.started_at, ct.completed_at, ct.records_processed, ct.error_message,
+                   ct.phase, ct.phase_progress, ct.phase_message, ct.phase_current,
+                   ct.phase_total, ct.phase_unit, ct.total_records, ct.progress
             FROM collection_tasks ct
             JOIN data_sources ds ON ct.datasource_id = ds.id
             WHERE ct.id = :id
@@ -105,6 +128,14 @@ async def get_task(
         "completed_at": to_iso8601_utc(task[5]),
         "records_processed": task[6],
         "error_message": task[7],
+        "phase": task[8],
+        "phase_progress": task[9],
+        "phase_message": task[10],
+        "phase_current": task[11],
+        "phase_total": task[12],
+        "phase_unit": task[13],
+        "total_records": task[14],
+        "progress": task[15],
     }
 
 

backend/app/api/v1/tv.py

@@ -1,3 +1,4 @@
+import re
 from urllib.parse import quote, urljoin
 
 import httpx
@@ -10,6 +11,26 @@ from app.services.tv_streams import get_public_tv_payload, is_allowed_tv_proxy_u
 
 router = APIRouter()
 
+_HLS_URI_ATTRIBUTE_RE = re.compile(r'URI="([^"]+)"')
+
+
+def _proxied_tv_url(url: str) -> str:
+    return f"/api/v1/tv/proxy?url={quote(url, safe='')}"
+
+
+def _rewrite_hls_uri_attributes(line: str, *, base_url: str) -> str:
+    def replace(match: re.Match[str]) -> str:
+        uri = match.group(1)
+        absolute_url = urljoin(base_url, uri)
+        return f'URI="{_proxied_tv_url(absolute_url)}"'
+
+    return _HLS_URI_ATTRIBUTE_RE.sub(replace, line)
+
+
+def _should_strip_hls_metadata_line(line: str) -> bool:
+    normalized = line.strip().upper()
+    return normalized.startswith("#EXT-X-MEDIA:") and "TYPE=SUBTITLES" in normalized
+
 
 @router.get("/streams")
 async def list_public_tv_streams(
@@ -56,11 +77,16 @@ async def proxy_tv_stream(
         rewritten_lines: list[str] = []
         for line in manifest_text.splitlines():
             stripped = line.strip()
-            if not stripped or stripped.startswith("#"):
+            if not stripped:
                 rewritten_lines.append(line)
                 continue
+            if stripped.startswith("#"):
+                if _should_strip_hls_metadata_line(line):
+                    continue
+                rewritten_lines.append(_rewrite_hls_uri_attributes(line, base_url=response_url))
+                continue
             absolute_url = urljoin(response_url, stripped)
-            rewritten_lines.append(f"/api/v1/tv/proxy?url={quote(absolute_url, safe='')}")
+            rewritten_lines.append(_proxied_tv_url(absolute_url))
         return Response(
             content="\n".join(rewritten_lines),
             media_type="application/vnd.apple.mpegurl",

backend/app/api/v1/users.py

@@ -1,16 +1,22 @@
+import json
 from typing import List
 
 from fastapi import APIRouter, Depends, HTTPException, status
 from sqlalchemy.ext.asyncio import AsyncSession
 from sqlalchemy import text
 
+from app.core.enums import UserRole
 from app.core.security import get_current_user, get_password_hash
 from app.db.session import get_db
 from app.models.user import User
-from app.schemas.user import UserCreate, UserResponse, UserUpdate
+from app.schemas.user import UserCreate, UserUpdate
 
 router = APIRouter()
 
+VALID_GATEKEEPER_GROUPS = {"docs_user", "docs_developer", "docs_admin"}
+ADMIN_ROLES = [UserRole.SUPER_ADMIN.value, UserRole.ADMIN.value]
+SUPER_ADMIN_ROLES = [UserRole.SUPER_ADMIN.value]
+
 
 def check_permission(current_user: User, required_roles: List[str]) -> bool:
     user_role_value = (
@@ -29,7 +35,7 @@ async def list_users(
     current_user: User = Depends(get_current_user),
     db: AsyncSession = Depends(get_db),
 ):
-    if not check_permission(current_user, ["super_admin", "admin"]):
+    if not check_permission(current_user, ADMIN_ROLES):
         raise HTTPException(
             status_code=status.HTTP_403_FORBIDDEN,
             detail="Insufficient permissions",
@@ -52,7 +58,7 @@ async def list_users(
 
     offset = (page - 1) * page_size
     query = text(
-        f"SELECT id, username, email, role, is_active, last_login_at, created_at FROM users WHERE {where_sql} ORDER BY created_at DESC LIMIT {page_size} OFFSET {offset}"
+        f"SELECT id, username, email, role, is_active, last_login_at, created_at, gatekeeper_groups FROM users WHERE {where_sql} ORDER BY created_at DESC LIMIT {page_size} OFFSET {offset}"
     )
     count_query = text(f"SELECT COUNT(*) FROM users WHERE {where_sql}")
 
@@ -75,6 +81,7 @@ async def list_users(
                 "is_active": u[4],
                 "last_login_at": u[5],
                 "created_at": u[6],
+                "gatekeeper_groups": u[7] or [],
             }
             for u in users
         ],
@@ -87,7 +94,7 @@ async def get_user(
     current_user: User = Depends(get_current_user),
     db: AsyncSession = Depends(get_db),
 ):
-    if not check_permission(current_user, ["super_admin", "admin"]) and current_user.id != user_id:
+    if not check_permission(current_user, ADMIN_ROLES) and current_user.id != user_id:
         raise HTTPException(
             status_code=status.HTTP_403_FORBIDDEN,
             detail="Insufficient permissions",
@@ -95,7 +102,7 @@ async def get_user(
 
     result = await db.execute(
         text(
-            "SELECT id, username, email, role, is_active, last_login_at, created_at FROM users WHERE id = :id"
+            "SELECT id, username, email, role, is_active, last_login_at, created_at, gatekeeper_groups FROM users WHERE id = :id"
         ),
         {"id": user_id},
     )
@@ -114,6 +121,7 @@ async def get_user(
         "is_active": user[4],
         "last_login_at": user[5],
         "created_at": user[6],
+        "gatekeeper_groups": user[7] or [],
     }
 
 
@@ -123,11 +131,17 @@ async def create_user(
     current_user: User = Depends(get_current_user),
     db: AsyncSession = Depends(get_db),
 ):
-    if not check_permission(current_user, ["super_admin"]):
+    if not check_permission(current_user, SUPER_ADMIN_ROLES):
         raise HTTPException(
             status_code=status.HTTP_403_FORBIDDEN,
             detail="Only super_admin can create users",
         )
+    invalid_groups = sorted(set(user_data.gatekeeper_groups) - VALID_GATEKEEPER_GROUPS)
+    if invalid_groups:
+        raise HTTPException(
+            status_code=status.HTTP_400_BAD_REQUEST,
+            detail=f"Unsupported Gatekeeper groups: {', '.join(invalid_groups)}",
+        )
 
     result = await db.execute(
         text("SELECT id FROM users WHERE username = :username OR email = :email"),
@@ -142,13 +156,14 @@ async def create_user(
     hashed_password = get_password_hash(user_data.password)
 
     await db.execute(
-        text("""INSERT INTO users (username, email, password_hash, role, is_active, created_at, updated_at)
+        text("""INSERT INTO users (username, email, password_hash, role, gatekeeper_groups, is_active, created_at, updated_at)
-           VALUES (:username, :email, :password_hash, :role, :is_active, NOW(), NOW())"""),
+           VALUES (:username, :email, :password_hash, :role, CAST(:gatekeeper_groups AS jsonb), :is_active, NOW(), NOW())"""),
         {
             "username": user_data.username,
             "email": user_data.email,
             "password_hash": hashed_password,
             "role": user_data.role,
+            "gatekeeper_groups": json.dumps(user_data.gatekeeper_groups),
             "is_active": True,
         },
     )
@@ -172,6 +187,7 @@ async def create_user(
         "username": user_data.username,
         "email": user_data.email,
         "role": user_data.role,
+        "gatekeeper_groups": user_data.gatekeeper_groups,
         "is_active": True,
     }
 
@@ -183,17 +199,29 @@ async def update_user(
     current_user: User = Depends(get_current_user),
     db: AsyncSession = Depends(get_db),
 ):
-    if not check_permission(current_user, ["super_admin", "admin"]) and current_user.id != user_id:
+    if not check_permission(current_user, ADMIN_ROLES) and current_user.id != user_id:
         raise HTTPException(
             status_code=status.HTTP_403_FORBIDDEN,
             detail="Insufficient permissions",
         )
 
-    if not check_permission(current_user, ["super_admin"]) and user_data.role is not None:
+    if not check_permission(current_user, SUPER_ADMIN_ROLES) and user_data.role is not None:
         raise HTTPException(
             status_code=status.HTTP_403_FORBIDDEN,
             detail="Only super_admin can change user role",
         )
+    if not check_permission(current_user, SUPER_ADMIN_ROLES) and user_data.gatekeeper_groups is not None:
+        raise HTTPException(
+            status_code=status.HTTP_403_FORBIDDEN,
+            detail="Only super_admin can change Gatekeeper groups",
+        )
+    if user_data.gatekeeper_groups is not None:
+        invalid_groups = sorted(set(user_data.gatekeeper_groups) - VALID_GATEKEEPER_GROUPS)
+        if invalid_groups:
+            raise HTTPException(
+                status_code=status.HTTP_400_BAD_REQUEST,
+                detail=f"Unsupported Gatekeeper groups: {', '.join(invalid_groups)}",
+            )
 
     result = await db.execute(
         text("SELECT id FROM users WHERE id = :id"),
@@ -213,6 +241,9 @@ async def update_user(
     if user_data.role is not None:
         update_fields.append("role = :role")
         params["role"] = user_data.role
+    if user_data.gatekeeper_groups is not None:
+        update_fields.append("gatekeeper_groups = CAST(:gatekeeper_groups AS jsonb)")
+        params["gatekeeper_groups"] = json.dumps(user_data.gatekeeper_groups)
     if user_data.is_active is not None:
         update_fields.append("is_active = :is_active")
         params["is_active"] = user_data.is_active
@@ -232,7 +263,7 @@ async def delete_user(
     current_user: User = Depends(get_current_user),
     db: AsyncSession = Depends(get_db),
 ):
-    if not check_permission(current_user, ["super_admin"]):
+    if not check_permission(current_user, SUPER_ADMIN_ROLES):
         raise HTTPException(
             status_code=status.HTTP_403_FORBIDDEN,
             detail="Only super_admin can delete users",

backend/app/api/v1/vessel_aggregation.py

@@ -0,0 +1,132 @@
+"""v4 strategy + v5 conflict-promotion + enrichment APIs for vessel_ais."""
+
+from __future__ import annotations
+
+from typing import Any
+
+from fastapi import APIRouter, Depends, HTTPException
+from sqlalchemy import select
+from sqlalchemy.ext.asyncio import AsyncSession
+
+from app.core.security import get_current_user
+from app.db.session import get_db
+from app.models.user import User
+from app.models.vessel import AISConflictRecord
+from app.services.vessel_aggregation_strategy import (
+    StrategyValidationError,
+    load_strategy,
+    reset_strategy,
+    save_strategy,
+)
+from app.services.vessel_enrichment import (
+    get_vessel_enrichment_bundle,
+    upsert_vessel_media_enrichment,
+    upsert_vessel_profile_enrichment,
+)
+
+router = APIRouter()
+
+
+@router.get("/strategy")
+async def get_aggregation_strategy(db: AsyncSession = Depends(get_db)):
+    return await load_strategy(db)
+
+
+@router.put("/strategy")
+async def put_aggregation_strategy(
+    payload: dict[str, Any],
+    current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    try:
+        return await save_strategy(db, payload)
+    except StrategyValidationError as exc:
+        raise HTTPException(status_code=400, detail=str(exc)) from exc
+
+
+@router.delete("/strategy")
+async def reset_aggregation_strategy(
+    current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    return await reset_strategy(db)
+
+
+@router.post("/conflicts/{mmsi}/{field}/promote-to-rule")
+async def promote_conflict_to_rule(
+    mmsi: int,
+    field: str,
+    current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    """Lift the current conflict resolution into a persistent strategy rule."""
+
+    result = await db.execute(
+        select(AISConflictRecord)
+        .where(AISConflictRecord.target_schema == "vessel_ais")
+        .where(AISConflictRecord.entity_key == str(mmsi))
+        .where(AISConflictRecord.field == field)
+        .order_by(AISConflictRecord.updated_at.desc(), AISConflictRecord.id.desc())
+        .limit(1)
+    )
+    record = result.scalar_one_or_none()
+    if record is None or not record.selected_source:
+        raise HTTPException(status_code=404, detail="Conflict record with selected_source not found")
+
+    strategy = await load_strategy(db)
+    vessel_ais = dict(strategy.get("vessel_ais") or {})
+    field_rules = dict(vessel_ais.get("field_rules") or {})
+    field_rules[field] = {"mode": "source_priority", "source_priority": [record.selected_source]}
+    vessel_ais["field_rules"] = field_rules
+
+    incoming = {"version": int(strategy.get("version") or 0), "vessel_ais": vessel_ais}
+    try:
+        return await save_strategy(db, incoming)
+    except StrategyValidationError as exc:
+        raise HTTPException(status_code=400, detail=str(exc)) from exc
+
+
+@router.delete("/conflicts/{mmsi}/{field}/promote-to-rule")
+async def revert_conflict_rule(
+    mmsi: int,
+    field: str,
+    current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    strategy = await load_strategy(db)
+    vessel_ais = dict(strategy.get("vessel_ais") or {})
+    field_rules = dict(vessel_ais.get("field_rules") or {})
+    if field in field_rules:
+        del field_rules[field]
+    vessel_ais["field_rules"] = field_rules
+
+    incoming = {"version": int(strategy.get("version") or 0), "vessel_ais": vessel_ais}
+    try:
+        return await save_strategy(db, incoming)
+    except StrategyValidationError as exc:
+        raise HTTPException(status_code=400, detail=str(exc)) from exc
+
+
+@router.get("/enrichment/{mmsi}")
+async def get_vessel_enrichment(mmsi: int, db: AsyncSession = Depends(get_db)):
+    return await get_vessel_enrichment_bundle(db, mmsi)
+
+
+@router.put("/enrichment/{mmsi}/profile")
+async def put_vessel_profile_enrichment(
+    mmsi: int,
+    payload: dict[str, Any],
+    current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    return await upsert_vessel_profile_enrichment(db, mmsi=mmsi, payload=payload)
+
+
+@router.put("/enrichment/{mmsi}/media")
+async def put_vessel_media_enrichment(
+    mmsi: int,
+    payload: dict[str, Any],
+    current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    return await upsert_vessel_media_enrichment(db, mmsi=mmsi, payload=payload)

backend/app/api/v1/vessels.py

@@ -0,0 +1,41 @@
+"""Bounded vessel snapshot APIs backed by the latest vessel state table."""
+
+from typing import Optional
+
+from fastapi import APIRouter, Depends, HTTPException, Query, Response
+from sqlalchemy.ext.asyncio import AsyncSession
+
+from app.api.v1.visualization import _parse_bbox, build_vessel_snapshot_response
+from app.db.session import get_db
+from app.services.vessel_ais_aggregation import MAX_SNAPSHOT_LIMIT
+
+router = APIRouter()
+
+
+@router.get("/snapshot")
+async def get_vessel_snapshot(
+    bbox: Optional[str] = Query(None, description="Snapshot bbox as lon_min,lat_min,lon_max,lat_max"),
+    zoom: float = Query(..., ge=1, le=20, description="Current map zoom level"),
+    type: Optional[str] = Query(
+        None,
+        description="Comma-separated vessel types: cargo,tanker,passenger,fishing,military,other",
+    ),
+    limit: int = Query(1000, ge=1, le=MAX_SNAPSHOT_LIMIT),
+    since_minutes: int = Query(60, ge=1, le=1440),
+    db: AsyncSession = Depends(get_db),
+    response: Response = None,
+):
+    if not bbox:
+        raise HTTPException(status_code=400, detail="bbox is required")
+    parsed_bbox = _parse_bbox(bbox)
+    if parsed_bbox is None:
+        raise HTTPException(status_code=400, detail="bbox is required")
+    return await build_vessel_snapshot_response(
+        db,
+        bbox=parsed_bbox,
+        zoom=zoom,
+        type_filter=type,
+        limit=limit,
+        since_minutes=since_minutes,
+        response=response,
+    )

backend/app/api/v1/websocket.py

@@ -1,20 +1,24 @@
 """WebSocket API endpoints"""
 
 import asyncio
-import json
-import logging
 from datetime import UTC, datetime
 from typing import Optional
 
 from fastapi import APIRouter, WebSocket, WebSocketDisconnect, Query
 from jose import jwt, JWTError
+from sqlalchemy import text
 
 from app.core.config import settings
+from app.core.enums import UserRole
+from app.core.logging import get_logger
 from app.core.time import to_iso8601_utc
 from app.core.websocket.manager import manager
+from app.db.session import async_session_factory
+from app.services.log_tail import LOG_TAIL_CHANNEL, log_tail_manager
 
-logger = logging.getLogger(__name__)
+logger = get_logger(__name__, service="api")
 router = APIRouter()
+EARTH_UPDATES_CHANNEL = "earth_updates"
 
 
 async def authenticate_token(token: str) -> Optional[dict]:
@@ -22,28 +26,79 @@ async def authenticate_token(token: str) -> Optional[dict]:
     try:
         payload = jwt.decode(token, settings.SECRET_KEY, algorithms=[settings.ALGORITHM])
         if payload.get("type") != "access":
-            logger.warning(f"WebSocket auth failed: wrong token type")
+            logger.warning_event(
+                "WebSocket auth failed: wrong token type",
+                event="auth.websocket.invalid_token_type",
+            )
             return None
         return payload
     except JWTError as e:
-        logger.warning(f"WebSocket auth failed: {e}")
+        logger.warning_event(
+            "WebSocket auth failed",
+            event="auth.websocket.decode_failed",
+            context={"error": str(e)},
+        )
         return None
 
 
+async def load_websocket_user_role(user_id: str | None) -> str | None:
+    if not user_id:
+        return None
+    try:
+        async with async_session_factory() as db:
+            result = await db.execute(
+                text("SELECT role, is_active FROM users WHERE id = :id"),
+                {"id": int(user_id)},
+            )
+            row = result.fetchone()
+    except Exception as exc:
+        logger.warning_event(
+            "WebSocket user role lookup failed",
+            event="auth.websocket.role_lookup_failed",
+            context={"user_id": user_id, "error": str(exc)},
+        )
+        return None
+    if row is None or not row[1]:
+        return None
+    return str(row[0] or "")
+
+
 @router.websocket("/ws")
 async def websocket_endpoint(
     websocket: WebSocket,
-    token: str = Query(...),
+    token: str | None = Query(None),
 ):
     """WebSocket endpoint for real-time data"""
-    logger.info(f"WebSocket connection attempt with token: {token[:20]}...")
+    logger.info_event(
-    payload = await authenticate_token(token)
+        "WebSocket connection attempt",
-    if payload is None:
+        event="auth.websocket.connection_attempt",
-        logger.warning("WebSocket authentication failed, closing connection")
+        context={"token_preview": f"{token[:8]}..." if token else "anonymous"},
+    )
+    payload = await authenticate_token(token) if token else None
+    if token and payload is None:
+        logger.warning_event(
+            "WebSocket authentication failed, closing connection",
+            event="auth.websocket.connection_rejected",
+        )
         await websocket.close(code=4001)
         return
 
-    user_id = str(payload.get("sub"))
+    is_anonymous = payload is None
+    user_id = str(payload.get("sub")) if payload else f"anonymous:{id(websocket)}"
+    user_role = await load_websocket_user_role(user_id) if payload else None
+    supported_channels = ["vessels", "earth_news", EARTH_UPDATES_CHANNEL] if is_anonymous else [
+        "gpu_clusters",
+        "submarine_cables",
+        "ixp_nodes",
+        "alerts",
+        "dashboard",
+        "datasource_tasks",
+        "vessels",
+        "earth_news",
+        EARTH_UPDATES_CHANNEL,
+    ]
+    if user_role == UserRole.SUPER_ADMIN.value:
+        supported_channels = [*supported_channels, LOG_TAIL_CHANNEL]
     await manager.connect(websocket, user_id)
 
     try:
@@ -54,14 +109,7 @@ async def websocket_endpoint(
                     "connection_id": f"conn_{user_id}",
                     "server_version": settings.VERSION,
                     "heartbeat_interval": 30,
-                    "supported_channels": [
+                    "supported_channels": supported_channels,
-                        "gpu_clusters",
-                        "submarine_cables",
-                        "ixp_nodes",
-                        "alerts",
-                        "dashboard",
-                        "datasource_tasks",
-                    ],
                 },
             }
         )
@@ -78,11 +126,80 @@ async def websocket_endpoint(
                         }
                     )
                 elif data.get("type") == "subscribe":
-                    channels = data.get("data", {}).get("channels", [])
+                    payload_data = data.get("data", {})
+                    if not isinstance(payload_data, dict):
+                        payload_data = {}
+                    log_tail_config = None
+                    channels = payload_data.get("channels", [])
+                    if isinstance(channels, str):
+                        channels = [channels]
+                    elif not isinstance(channels, list):
+                        channels = []
+                    channel = payload_data.get("channel")
+                    if channel and channel not in channels:
+                        channels = [*channels, channel]
+                    if LOG_TAIL_CHANNEL in channels:
+                        if user_role != UserRole.SUPER_ADMIN.value:
+                            await websocket.send_json(
+                                {
+                                    "type": "subscription_error",
+                                    "data": {"channel": LOG_TAIL_CHANNEL, "detail": "Only super_admin can subscribe logs"},
+                                }
+                            )
+                            channels = [item for item in channels if item != LOG_TAIL_CHANNEL]
+                        else:
+                            try:
+                                log_tail_config = await log_tail_manager.subscribe(websocket, payload_data)
+                            except ValueError as exc:
+                                await websocket.send_json(
+                                    {
+                                        "type": "subscription_error",
+                                        "data": {"channel": LOG_TAIL_CHANNEL, "detail": str(exc)},
+                                    }
+                                )
+                            channels = [item for item in channels if item != LOG_TAIL_CHANNEL]
+                    if is_anonymous:
+                        channels = [channel for channel in channels if channel in supported_channels]
+                    vessel_subscription = None
+                    if "vessels" in channels and "bbox" in payload_data:
+                        try:
+                            vessel_subscription = manager.subscribe_vessels(websocket, payload_data)
+                        except ValueError as exc:
+                            await websocket.send_json(
+                                {
+                                    "type": "subscription_error",
+                                    "data": {"channel": "vessels", "detail": str(exc)},
+                                }
+                            )
+                            continue
+                        channels = [channel for channel in channels if channel != "vessels"]
+                    manager.subscribe(websocket, channels)
                     await websocket.send_json(
                         {
                             "type": "subscription_confirmed",
-                            "data": {"action": "subscribe", "channels": channels},
+                            "data": {
+                                "action": "subscribe",
+                                "channels": [
+                                    *channels,
+                                    *([LOG_TAIL_CHANNEL] if log_tail_config else []),
+                                    *(["vessels"] if vessel_subscription else []),
+                                ],
+                                "vessels": vessel_subscription,
+                                "logs_tail": log_tail_config.__dict__ if log_tail_config else None,
+                            },
+                        }
+                    )
+                elif data.get("type") == "unsubscribe":
+                    channels = data.get("data", {}).get("channels", [])
+                    if isinstance(channels, str):
+                        channels = [channels]
+                    if LOG_TAIL_CHANNEL in channels:
+                        await log_tail_manager.unsubscribe(websocket)
+                    manager.unsubscribe(websocket, channels)
+                    await websocket.send_json(
+                        {
+                            "type": "subscription_confirmed",
+                            "data": {"action": "unsubscribe", "channels": channels},
                         }
                     )
                 elif data.get("type") == "control_frame":
@@ -98,4 +215,5 @@ async def websocket_endpoint(
     except WebSocketDisconnect:
         pass
     finally:
+        await log_tail_manager.disconnect(websocket)
         manager.disconnect(websocket, user_id)

backend/app/core/cache.py

@@ -1,15 +1,15 @@
 """Redis caching service"""
 
 import json
-import logging
 from datetime import timedelta
 from typing import Optional, Any
 
 import redis
 
 from app.core.config import settings
+from app.core.logging import get_logger
 
-logger = logging.getLogger(__name__)
+logger = get_logger(__name__)
 
 
 # Lazy Redis client initialization
@@ -47,7 +47,7 @@ class CacheService:
                 return json.loads(value)
             return None
         except Exception as e:
-            logger.warning(f"Cache get error: {e}")
+            logger.warning_event("Cache get error", event="cache.get.failed", context={"error": str(e)})
             return None
 
     def set(
@@ -61,7 +61,7 @@ class CacheService:
             serialized = json.dumps(value, default=str)
             return self.client.setex(key, expire_seconds, serialized)
         except Exception as e:
-            logger.warning(f"Cache set error: {e}")
+            logger.warning_event("Cache set error", event="cache.set.failed", context={"error": str(e)})
             return False
 
     def delete(self, key: str) -> bool:
@@ -69,7 +69,7 @@ class CacheService:
         try:
             return self.client.delete(key) > 0
         except Exception as e:
-            logger.warning(f"Cache delete error: {e}")
+            logger.warning_event("Cache delete error", event="cache.delete.failed", context={"error": str(e)})
             return False
 
     def delete_pattern(self, pattern: str) -> int:
@@ -80,7 +80,7 @@ class CacheService:
                 return self.client.delete(*keys)
             return 0
         except Exception as e:
-            logger.warning(f"Cache delete_pattern error: {e}")
+            logger.warning_event("Cache delete_pattern error", event="cache.delete_pattern.failed", context={"error": str(e)})
             return 0
 
     def get_or_set(

backend/app/core/collected_data_fields.py

@@ -4,8 +4,8 @@ from typing import Any, Dict, Optional
 FIELD_ALIASES = {
     "country": ("country",),
     "city": ("city",),
-    "latitude": ("latitude",),
+    "latitude": ("latitude", "lat"),
-    "longitude": ("longitude",),
+    "longitude": ("longitude", "lon", "lng"),
     "value": ("value",),
     "unit": ("unit",),
     "cores": ("cores",),
@@ -14,6 +14,28 @@ FIELD_ALIASES = {
     "power": ("power",),
 }
 
+NESTED_FIELD_ALIASES = {
+    "latitude": (
+        ("location", "latitude"),
+        ("location", "lat"),
+        ("geo", "latitude"),
+        ("geo", "lat"),
+        ("coordinates", "latitude"),
+        ("coordinates", "lat"),
+    ),
+    "longitude": (
+        ("location", "longitude"),
+        ("location", "lon"),
+        ("location", "lng"),
+        ("geo", "longitude"),
+        ("geo", "lon"),
+        ("geo", "lng"),
+        ("coordinates", "longitude"),
+        ("coordinates", "lon"),
+        ("coordinates", "lng"),
+    ),
+}
+
 
 def get_metadata_field(metadata: Optional[Dict[str, Any]], field: str, fallback: Any = None) -> Any:
     if isinstance(metadata, dict):
@@ -21,9 +43,34 @@ def get_metadata_field(metadata: Optional[Dict[str, Any]], field: str, fallback:
             value = metadata.get(key)
             if value not in (None, ""):
                 return value
+        for path in NESTED_FIELD_ALIASES.get(field, ()):
+            current: Any = metadata
+            for key in path:
+                if not isinstance(current, dict):
+                    current = None
+                    break
+                current = current.get(key)
+            if current not in (None, ""):
+                return current
+        if field in {"latitude", "longitude"}:
+            value = _get_coordinate_sequence_value(metadata, field)
+            if value not in (None, ""):
+                return value
     return fallback
 
 
+def _get_coordinate_sequence_value(metadata: Dict[str, Any], field: str) -> Any:
+    for key in ("coordinates", "coord", "coords"):
+        value = metadata.get(key)
+        if not isinstance(value, (list, tuple)) or len(value) < 2:
+            continue
+        # GeoJSON uses [longitude, latitude]. Most raw collector tuples in this
+        # codebase use explicit field names, so only sequence aliases are treated
+        # as GeoJSON-shaped to avoid guessing.
+        return value[1] if field == "latitude" else value[0]
+    return None
+
+
 def build_dynamic_metadata(
     metadata: Optional[Dict[str, Any]],
     *,

backend/app/core/config.py

@@ -41,6 +41,7 @@ class Settings(BaseSettings):
     AI_PROVIDER_SERVICE_TOKEN: str = ""
     AI_PROVIDER_TIMEOUT_SECONDS: int = 60
     AI_PROVIDER_RETRY_ATTEMPTS: int = 2
+    OBSERVABILITY_INGEST_TOKEN: str = ""
 
     @property
     def REDIS_URL(self) -> str:

backend/app/core/data_sources.py

@@ -1,7 +1,6 @@
 import os
 import yaml
 from functools import lru_cache
-from typing import Optional
 
 
 COLLECTOR_URL_KEYS = {
@@ -30,6 +29,9 @@ COLLECTOR_URL_KEYS = {
     "iptoasn_prefix_geo": "iptoasn.combined_url",
     "opengeofeed_prefix_geo": "opengeofeed.public_csv_url",
     "nro_delegated_prefix_geo": "nro.delegated_stats_url",
+    "news_live_streams": "news_live_streams.channels_url",
+    "barentswatch_vessels": "barentswatch_vessels.url",
+    "aisstream_vessels": "aisstream_vessels.url",
 }
 
 
@@ -72,7 +74,7 @@ class DataSourcesConfig:
             from app.models.datasource_config import DataSourceConfig
 
             query = select(DataSourceConfig).where(
-                DataSourceConfig.name == collector_name, DataSourceConfig.is_active == True
+                DataSourceConfig.name == collector_name, DataSourceConfig.is_active
             )
             result = await db.execute(query)
             db_config = result.scalar_one_or_none()

backend/app/core/data_sources.yaml

@@ -86,3 +86,19 @@ opengeofeed:
 nro:
   # NRO delegated stats 下载地址
   delegated_stats_url: "https://ftp.ripe.net/pub/stats/ripencc/nro-stats/latest/nro-delegated-stats"
+
+news_live_streams:
+  # IPTV-org 频道元数据 JSON
+  channels_url: "https://iptv-org.github.io/api/channels.json"
+  # IPTV-org 频道播放流 JSON
+  streams_url: "https://iptv-org.github.io/api/streams.json"
+  # IPTV-org 台标 JSON
+  logos_url: "https://iptv-org.github.io/api/logos.json"
+
+barentswatch_vessels:
+  # BarentsWatch Live AIS latest combined endpoint. Requires an AIS bearer token.
+  url: "https://live.ais.barentswatch.no/v1/latest/combined"
+
+aisstream_vessels:
+  # AISStream realtime WebSocket endpoint. Requires an AISStream API key.
+  url: "wss://stream.aisstream.io/v0/stream"

backend/app/core/datasource_defaults.py

@@ -4,163 +4,268 @@ DEFAULT_DATASOURCES = {
     "top500": {
         "id": 1,
         "name": "TOP500 Supercomputers",
+        "display_name": "TOP500 超算榜单",
         "module": "L1",
         "priority": "P0",
         "frequency_minutes": 240,
+        "is_free": True,
+        "requires_credentials": False,
     },
     "epoch_ai_gpu": {
         "id": 2,
         "name": "Epoch AI GPU Clusters",
+        "display_name": "Epoch AI GPU 集群",
         "module": "L1",
         "priority": "P0",
         "frequency_minutes": 360,
+        "is_free": True,
+        "requires_credentials": False,
     },
     "huggingface_models": {
         "id": 3,
         "name": "HuggingFace Models",
+        "display_name": "Hugging Face 模型",
         "module": "L2",
         "priority": "P1",
         "frequency_minutes": 720,
+        "is_free": True,
+        "requires_credentials": False,
     },
     "huggingface_datasets": {
         "id": 4,
         "name": "HuggingFace Datasets",
+        "display_name": "Hugging Face 数据集",
         "module": "L2",
         "priority": "P1",
         "frequency_minutes": 720,
+        "is_free": True,
+        "requires_credentials": False,
     },
     "huggingface_spaces": {
         "id": 5,
         "name": "HuggingFace Spaces",
+        "display_name": "Hugging Face Spaces",
         "module": "L2",
         "priority": "P2",
         "frequency_minutes": 1440,
+        "is_free": True,
+        "requires_credentials": False,
     },
     "peeringdb_ixp": {
         "id": 6,
         "name": "PeeringDB IXP",
+        "display_name": "PeeringDB 交换中心",
         "module": "L2",
         "priority": "P1",
         "frequency_minutes": 1440,
+        "is_free": True,
+        "requires_credentials": False,
     },
     "peeringdb_network": {
         "id": 7,
         "name": "PeeringDB Networks",
+        "display_name": "PeeringDB 网络",
         "module": "L2",
         "priority": "P2",
         "frequency_minutes": 2880,
+        "is_free": True,
+        "requires_credentials": False,
     },
     "peeringdb_facility": {
         "id": 8,
         "name": "PeeringDB Facilities",
+        "display_name": "PeeringDB 设施",
         "module": "L2",
         "priority": "P2",
         "frequency_minutes": 2880,
+        "is_free": True,
+        "requires_credentials": False,
     },
     "telegeography_cables": {
         "id": 9,
         "name": "Submarine Cables",
+        "display_name": "海底光缆",
         "module": "L2",
         "priority": "P1",
         "frequency_minutes": 10080,
+        "is_free": True,
+        "requires_credentials": False,
     },
     "telegeography_landing": {
         "id": 10,
         "name": "Cable Landing Points",
+        "display_name": "光缆登陆点",
         "module": "L2",
         "priority": "P2",
         "frequency_minutes": 10080,
+        "is_free": True,
+        "requires_credentials": False,
     },
     "telegeography_systems": {
         "id": 11,
         "name": "Cable Systems",
+        "display_name": "光缆系统",
         "module": "L2",
         "priority": "P2",
         "frequency_minutes": 10080,
+        "is_free": True,
+        "requires_credentials": False,
     },
     "arcgis_cables": {
         "id": 15,
         "name": "ArcGIS Submarine Cables",
+        "display_name": "ArcGIS 海底光缆",
         "module": "L2",
         "priority": "P1",
         "frequency_minutes": 10080,
+        "is_free": True,
+        "requires_credentials": False,
     },
     "arcgis_landing_points": {
         "id": 16,
         "name": "ArcGIS Landing Points",
+        "display_name": "ArcGIS 登陆点",
         "module": "L2",
         "priority": "P1",
         "frequency_minutes": 10080,
+        "is_free": True,
+        "requires_credentials": False,
     },
     "arcgis_cable_landing_relation": {
         "id": 17,
         "name": "ArcGIS Cable-Landing Relations",
+        "display_name": "ArcGIS 光缆登陆关系",
         "module": "L2",
         "priority": "P1",
         "frequency_minutes": 10080,
+        "is_free": True,
+        "requires_credentials": False,
     },
     "fao_landing_points": {
         "id": 18,
         "name": "FAO Landing Points",
+        "display_name": "FAO 登陆点",
         "module": "L2",
         "priority": "P1",
         "frequency_minutes": 10080,
+        "is_free": True,
+        "requires_credentials": False,
     },
     "spacetrack_tle": {
         "id": 19,
         "name": "Space-Track TLE",
+        "display_name": "Space-Track 轨道根数",
         "module": "L3",
         "priority": "P2",
         "frequency_minutes": 1440,
+        "is_free": True,
+        "requires_credentials": True,
+        "credential_provider": "spacetrack",
+        "credential_status": "planned",
     },
     "celestrak_tle": {
         "id": 20,
         "name": "CelesTrak TLE",
+        "display_name": "CelesTrak 轨道根数",
         "module": "L3",
         "priority": "P2",
         "frequency_minutes": 1440,
+        "is_free": True,
+        "requires_credentials": False,
     },
     "ris_live_bgp": {
         "id": 21,
         "name": "RIPE RIS Live BGP",
+        "display_name": "RIPE RIS 实时 BGP",
         "module": "L3",
         "priority": "P1",
         "frequency_minutes": 15,
+        "is_free": True,
+        "requires_credentials": False,
     },
     "bgpstream_bgp": {
         "id": 22,
         "name": "CAIDA BGPStream Backfill",
+        "display_name": "CAIDA BGPStream 回填",
         "module": "L3",
         "priority": "P1",
         "frequency_minutes": 360,
+        "is_free": True,
+        "requires_credentials": False,
     },
     "iptoasn_prefix_geo": {
         "id": 23,
         "name": "IPtoASN Prefix Geography",
+        "display_name": "IPtoASN 前缀地理",
         "module": "L3",
         "priority": "P1",
         "frequency_minutes": 1440,
+        "is_free": True,
+        "requires_credentials": False,
     },
     "opengeofeed_prefix_geo": {
         "id": 24,
         "name": "OpenGeoFeed Prefix Geography",
+        "display_name": "OpenGeoFeed 前缀地理",
         "module": "L3",
         "priority": "P1",
         "frequency_minutes": 1440,
+        "is_free": True,
+        "requires_credentials": False,
     },
     "nro_delegated_prefix_geo": {
         "id": 25,
         "name": "NRO Delegated Prefix Geography",
+        "display_name": "NRO 分配前缀地理",
         "module": "L3",
         "priority": "P1",
         "frequency_minutes": 1440,
+        "is_free": True,
+        "requires_credentials": False,
     },
     "news_live_streams": {
         "id": 26,
         "name": "News Live Streams",
+        "display_name": "新闻直播源",
         "module": "L4",
         "priority": "P2",
         "frequency_minutes": 720,
+        "is_free": True,
+        "requires_credentials": False,
+    },
+    "barentswatch_vessels": {
+        "id": 27,
+        "name": "BarentsWatch AIS Vessels",
+        "display_name": "BarentsWatch AIS 船舶",
+        "module": "L4",
+        "priority": "P1",
+        "frequency_minutes": 1,
+        "is_free": True,
+        "requires_credentials": True,
+        "credential_provider": "barentswatch",
+        "credential_status": "supported",
+    },
+    "aisstream_vessels": {
+        "id": 28,
+        "name": "AISStream Vessels",
+        "display_name": "AISStream 实时船舶",
+        "module": "L4",
+        "priority": "P1",
+        "frequency_minutes": 1,
+        "is_free": True,
+        "requires_credentials": True,
+        "credential_provider": "aisstream",
+        "credential_status": "supported",
+    },
+    "media_news_archive": {
+        "id": 33,
+        "name": "Media News Archive",
+        "display_name": "媒体新闻归档",
+        "module": "L4",
+        "priority": "P2",
+        "frequency_minutes": 720,
+        "is_free": True,
+        "requires_credentials": False,
     },
 }
 

backend/app/core/enums.py

@@ -0,0 +1,226 @@
+"""Stable backend protocol enums.
+
+Database columns and JSON payloads continue to store the enum string values.
+Configurable identifiers, user-authored values, and open-ended taxonomies do
+not belong in this module.
+"""
+
+from __future__ import annotations
+
+import logging
+from enum import StrEnum
+from typing import TypeVar
+
+logger = logging.getLogger(__name__)
+
+EnumT = TypeVar("EnumT", bound=StrEnum)
+
+
+def parse_enum(enum_type: type[EnumT], value: object, default: EnumT) -> EnumT:
+    """Parse an external value without breaking reads of legacy data."""
+
+    if value is None or str(value).strip() == "":
+        return default
+    if isinstance(value, enum_type):
+        return value
+    try:
+        return enum_type(str(value).strip().lower())
+    except (TypeError, ValueError):
+        logger.warning(
+            "Unknown %s value %r; falling back to %s",
+            enum_type.__name__,
+            value,
+            default.value,
+        )
+        return default
+
+
+class NewsImportanceLevel(StrEnum):
+    LOW = "low"
+    MEDIUM = "medium"
+    HIGH = "high"
+    CRITICAL = "critical"
+
+
+class BreakingLevel(StrEnum):
+    NONE = "none"
+    WATCH = "watch"
+    BREAKING = "breaking"
+    CRITICAL = "critical"
+
+
+class BreakingScope(StrEnum):
+    REGIONAL = "regional"
+    GLOBAL = "global"
+
+
+class BreakingSource(StrEnum):
+    RULES = "rules"
+    AI = "ai"
+    MANUAL = "manual"
+    MULTI_SOURCE = "multi_source"
+
+
+class NewsSourceType(StrEnum):
+    RSS = "rss"
+    ATOM = "atom"
+    AGGREGATED = "aggregated"
+    REFERENCE = "reference"
+    MANUAL = "manual"
+
+
+class NewsEnrichmentStatus(StrEnum):
+    PENDING = "pending"
+    QUEUED = "queued"
+    ATTEMPTED = "attempted"
+    SUCCESS = "success"
+    CONTENT_ONLY = "content_only"
+    LOCATION_ONLY = "location_only"
+    UNAVAILABLE = "unavailable"
+    PROVIDER_ERROR = "provider_error"
+    PARSE_ERROR = "parse_error"
+    NO_RESULT = "no_result"
+
+
+class NewsMarketImpact(StrEnum):
+    NONE = "none"
+    SECTOR = "sector"
+    NATIONAL = "national"
+    GLOBAL = "global"
+
+
+class NewsTaggingSource(StrEnum):
+    RULES = "rules"
+    AI = "ai"
+    MANUAL = "manual"
+
+
+class JobType(StrEnum):
+    COLLECT = "collect"
+    CLEAR_DATA = "clear_data"
+    CLEAR_CACHE = "clear_cache"
+    EARTH_REFRESH = "earth_refresh"
+
+
+class JobStatus(StrEnum):
+    QUEUED = "queued"
+    RUNNING = "running"
+    CANCELLING = "cancelling"
+    SUCCESS = "success"
+    FAILED = "failed"
+    CANCELLED = "cancelled"
+
+
+class RollbackPolicy(StrEnum):
+    KEEP_COMMITTED_BATCHES = "keep_committed_batches"
+
+
+class MappingValidationStatus(StrEnum):
+    DRAFT = "draft"
+    VALID = "valid"
+    INVALID = "invalid"
+
+
+class SnapshotStatus(StrEnum):
+    RUNNING = "running"
+    SUCCESS = "success"
+    FAILED = "failed"
+    CANCELLED = "cancelled"
+
+
+class DatasourceRunStatus(StrEnum):
+    RUNNING = "running"
+    NOT_RUN = "not_run"
+    COLLECTED = "collected"
+    UNCOLLECTED = "uncollected"
+
+
+class ProviderApi(StrEnum):
+    ANTHROPIC_MESSAGES = "anthropic-messages"
+    OPENAI_COMPLETIONS = "openai-completions"
+    OLLAMA_GENERATE = "ollama-generate"
+
+
+class PlaygroundMessageRole(StrEnum):
+    SYSTEM = "system"
+    USER = "user"
+    ASSISTANT = "assistant"
+    TOOL = "tool"
+
+
+class PlaygroundMessageKind(StrEnum):
+    MESSAGE = "message"
+    THINKING = "thinking"
+    ERROR = "error"
+    STATUS = "status"
+
+
+class PlaygroundMessageStatus(StrEnum):
+    PENDING = "pending"
+    THINKING = "thinking"
+    ANSWERING = "answering"
+    DONE = "done"
+    FAILED = "failed"
+    CANCELLED = "cancelled"
+    ERROR = "error"
+    STOPPED = "stopped"
+
+
+class OtpPurpose(StrEnum):
+    REGISTER = "register"
+    VERIFY_EMAIL = "verify_email"
+    RESET_PASSWORD = "reset_password"
+
+
+class UserRole(StrEnum):
+    VIEWER = "viewer"
+    ADMIN = "admin"
+    SUPER_ADMIN = "super_admin"
+
+
+class AlertSeverity(StrEnum):
+    CRITICAL = "critical"
+    WARNING = "warning"
+    INFO = "info"
+
+
+class AlertStatus(StrEnum):
+    ACTIVE = "active"
+    ACKNOWLEDGED = "acknowledged"
+    RESOLVED = "resolved"
+
+
+class BGPStatus(StrEnum):
+    ACTIVE = "active"
+    ACKNOWLEDGED = "acknowledged"
+    RESOLVED = "resolved"
+
+
+class LogLevel(StrEnum):
+    ALL = "all"
+    ERROR = "error"
+    WARNING = "warning"
+    INFO = "info"
+    DEBUG = "debug"
+
+
+class ConnectionState(StrEnum):
+    DISCONNECTED = "disconnected"
+    CONNECTING = "connecting"
+    CONNECTED = "connected"
+    ERROR = "error"
+
+
+class AuthType(StrEnum):
+    NONE = "none"
+    BEARER = "bearer"
+    API_KEY = "api_key"
+    BASIC = "basic"
+
+
+class TVSourceType(StrEnum):
+    IFRAME = "iframe"
+    HLS = "hls"
+    VIDEO = "video"
+    EXTERNAL = "external"
+    YOUTUBE = "youtube"

backend/app/core/logging.py

@@ -0,0 +1,161 @@
+from __future__ import annotations
+
+import json
+import logging
+import os
+import re
+
+from collections.abc import Mapping, Sequence
+from typing import Any
+
+from app.core.request_context import get_request_id
+
+DEFAULT_SERVICE = "backend"
+DEFAULT_EVENT = "app.log"
+DEFAULT_LOG_LEVEL = os.getenv("PLANET_LOG_LEVEL", "INFO").upper()
+REDACTED = "[REDACTED]"
+SENSITIVE_FIELD_NAMES = {
+    "access_token",
+    "api_key",
+    "authorization",
+    "cookie",
+    "password",
+    "refresh_token",
+    "secret",
+    "token",
+}
+SENSITIVE_TEXT_PATTERNS = (
+    re.compile(r"(?i)(authorization\s*[:=]\s*)(.+)"),
+    re.compile(r"(?i)(bearer\s+)([A-Za-z0-9._\-]+)"),
+    re.compile(r"(?i)(token\s*[:=]\s*)(.+)"),
+    re.compile(r"(?i)(password\s*[:=]\s*)(.+)"),
+    re.compile(r"(?i)(cookie\s*[:=]\s*)(.+)"),
+)
+
+
+def sanitize_log_value(value: Any) -> Any:
+    if isinstance(value, Mapping):
+        return {
+            str(key): (REDACTED if str(key).lower() in SENSITIVE_FIELD_NAMES else sanitize_log_value(item))
+            for key, item in value.items()
+        }
+    if isinstance(value, Sequence) and not isinstance(value, (str, bytes, bytearray)):
+        return [sanitize_log_value(item) for item in value]
+    if isinstance(value, str):
+        sanitized = value
+        for pattern in SENSITIVE_TEXT_PATTERNS:
+            sanitized = pattern.sub(lambda match: f"{match.group(1)}{REDACTED}", sanitized)
+        return sanitized
+    return value
+
+
+def _normalize_context(context: Any) -> dict[str, Any]:
+    if context is None:
+        return {}
+    if isinstance(context, Mapping):
+        sanitized = sanitize_log_value(context)
+        return {str(key): value for key, value in sanitized.items()}
+    return {"value": sanitize_log_value(context)}
+
+
+class PlanetContextFilter(logging.Filter):
+    def filter(self, record: logging.LogRecord) -> bool:
+        record.request_id = getattr(record, "request_id", None) or get_request_id() or "-"
+        record.service = getattr(record, "service", None) or DEFAULT_SERVICE
+        record.event = getattr(record, "event", None) or DEFAULT_EVENT
+        record.context = _normalize_context(getattr(record, "context", None))
+        record.message = sanitize_log_value(record.getMessage())
+        return True
+
+
+class PlanetFormatter(logging.Formatter):
+    def format(self, record: logging.LogRecord) -> str:
+        timestamp = self.formatTime(record, self.datefmt)
+        level = record.levelname
+        service = getattr(record, "service", DEFAULT_SERVICE)
+        module_name = record.name
+        event = getattr(record, "event", DEFAULT_EVENT)
+        request_id = getattr(record, "request_id", "-")
+        message = sanitize_log_value(record.getMessage())
+        context = _normalize_context(getattr(record, "context", None))
+        context_suffix = ""
+        if context:
+            context_suffix = f" context={json.dumps(context, ensure_ascii=False, sort_keys=True)}"
+        rendered = (
+            f"{timestamp} {level} service={service} module={module_name} "
+            f"event={event} request_id={request_id} message={message}{context_suffix}"
+        )
+        if record.exc_info:
+            rendered = f"{rendered}\n{self.formatException(record.exc_info)}"
+        return rendered
+
+
+class PlanetLoggerAdapter(logging.LoggerAdapter):
+    def process(self, msg: Any, kwargs: dict[str, Any]) -> tuple[Any, dict[str, Any]]:
+        extra = dict(self.extra)
+        extra.update(kwargs.get("extra", {}))
+        if "context" in extra:
+            extra["context"] = _normalize_context(extra.get("context"))
+        kwargs["extra"] = extra
+        return sanitize_log_value(msg), kwargs
+
+    def log_event(
+        self,
+        level: int,
+        message: str,
+        *,
+        event: str,
+        context: Mapping[str, Any] | None = None,
+        **extra: Any,
+    ) -> None:
+        self.log(level, message, extra={"event": event, "context": context or {}, **extra})
+
+    def debug_event(self, message: str, *, event: str, context: Mapping[str, Any] | None = None, **extra: Any) -> None:
+        self.log_event(logging.DEBUG, message, event=event, context=context, **extra)
+
+    def info_event(self, message: str, *, event: str, context: Mapping[str, Any] | None = None, **extra: Any) -> None:
+        self.log_event(logging.INFO, message, event=event, context=context, **extra)
+
+    def warning_event(self, message: str, *, event: str, context: Mapping[str, Any] | None = None, **extra: Any) -> None:
+        self.log_event(logging.WARNING, message, event=event, context=context, **extra)
+
+    def error_event(self, message: str, *, event: str, context: Mapping[str, Any] | None = None, **extra: Any) -> None:
+        self.log_event(logging.ERROR, message, event=event, context=context, **extra)
+
+    def exception_event(
+        self,
+        message: str,
+        *,
+        event: str,
+        context: Mapping[str, Any] | None = None,
+        **extra: Any,
+    ) -> None:
+        self.error(message, exc_info=True, extra={"event": event, "context": context or {}, **extra})
+
+
+def get_logger(name: str, *, service: str = DEFAULT_SERVICE) -> PlanetLoggerAdapter:
+    return PlanetLoggerAdapter(logging.getLogger(name), {"service": service})
+
+
+def configure_logging(level: str | None = None) -> None:
+    root_logger = logging.getLogger()
+    if getattr(configure_logging, "_configured", False):
+        if level:
+            root_logger.setLevel(level.upper())
+        return
+
+    handler = logging.StreamHandler()
+    handler.setFormatter(PlanetFormatter(datefmt="%Y-%m-%d %H:%M:%S"))
+    handler.addFilter(PlanetContextFilter())
+
+    root_logger.handlers.clear()
+    root_logger.addHandler(handler)
+    root_logger.setLevel((level or DEFAULT_LOG_LEVEL).upper())
+
+    for logger_name in ("uvicorn", "uvicorn.error", "uvicorn.access"):
+        target_logger = logging.getLogger(logger_name)
+        target_logger.handlers.clear()
+        target_logger.propagate = True
+
+    logging.captureWarnings(True)
+    configure_logging._configured = True

backend/app/core/request_context.py

@@ -0,0 +1,14 @@
+from __future__ import annotations
+
+from contextvars import ContextVar
+
+
+request_id_context: ContextVar[str | None] = ContextVar("request_id", default=None)
+
+
+def set_request_id(request_id: str | None) -> None:
+    request_id_context.set(request_id)
+
+
+def get_request_id() -> str | None:
+    return request_id_context.get()

backend/app/core/security.py

@@ -105,7 +105,7 @@ async def get_current_user(
         )
     result = await db.execute(
         text(
-            "SELECT id, username, email, password_hash, role, is_active FROM users WHERE id = :id"
+            "SELECT id, username, email, password_hash, role, is_active, gatekeeper_groups FROM users WHERE id = :id"
         ),
         {"id": int(user_id)},
     )
@@ -122,6 +122,7 @@ async def get_current_user(
     user.password_hash = row[3]
     user.role = row[4]
     user.is_active = row[5]
+    user.gatekeeper_groups = row[6] or []
     return user
 
 
@@ -144,7 +145,7 @@ async def get_current_user_refresh(
         )
     result = await db.execute(
         text(
-            "SELECT id, username, email, password_hash, role, is_active FROM users WHERE id = :id"
+            "SELECT id, username, email, password_hash, role, is_active, gatekeeper_groups FROM users WHERE id = :id"
         ),
         {"id": int(user_id)},
     )
@@ -161,6 +162,7 @@ async def get_current_user_refresh(
     user.password_hash = row[3]
     user.role = row[4]
     user.is_active = row[5]
+    user.gatekeeper_groups = row[6] or []
     return user
 
 

backend/app/core/target_schema_registry.py

@@ -0,0 +1,157 @@
+"""Registry of target schemas supported by mapped custom data sources."""
+
+from __future__ import annotations
+
+from dataclasses import dataclass
+from datetime import datetime
+from typing import Any
+
+from pydantic import BaseModel, Field, ValidationError, field_validator
+
+
+class VesselAISRecord(BaseModel):
+    mmsi: int = Field(ge=100000000, le=999999999)
+    lat: float = Field(ge=-90, le=90)
+    lon: float = Field(ge=-180, le=180)
+    sog: float | None = None
+    cog: float | None = Field(default=None, ge=0, le=360)
+    heading: int | None = Field(default=None, ge=0, le=511)
+    nav_status: int | None = None
+    name: str | None = None
+    callsign: str | None = None
+    vessel_type: str | int | None = None
+    vessel_type_name: str | None = None
+    received_at: datetime | None = None
+
+
+class GeoPointRecord(BaseModel):
+    lat: float = Field(ge=-90, le=90)
+    lon: float = Field(ge=-180, le=180)
+    name: str | None = None
+    type: str | None = None
+    source_id: str | None = None
+    observed_at: datetime | None = None
+    metadata: dict[str, Any] = Field(default_factory=dict)
+
+
+class GenericRecord(BaseModel):
+    data: dict[str, Any] = Field(default_factory=dict)
+    source_id: str | None = None
+    observed_at: datetime | None = None
+
+    @field_validator("data")
+    @classmethod
+    def require_payload(cls, value: dict[str, Any]) -> dict[str, Any]:
+        if not value:
+            raise ValueError("generic_records requires a non-empty data object")
+        return value
+
+
+@dataclass(frozen=True)
+class TargetField:
+    name: str
+    type: str
+    required: bool = False
+    description: str = ""
+    example: Any = None
+
+    def to_dict(self) -> dict[str, Any]:
+        return {
+            "name": self.name,
+            "type": self.type,
+            "required": self.required,
+            "description": self.description,
+            "example": self.example,
+        }
+
+
+@dataclass(frozen=True)
+class TargetSchema:
+    key: str
+    label: str
+    description: str
+    fields: tuple[TargetField, ...]
+    model: type[BaseModel]
+    destination: str
+
+    def to_dict(self) -> dict[str, Any]:
+        return {
+            "key": self.key,
+            "label": self.label,
+            "description": self.description,
+            "destination": self.destination,
+            "fields": [field.to_dict() for field in self.fields],
+        }
+
+    def validate_record(self, record: dict[str, Any]) -> tuple[dict[str, Any] | None, list[str]]:
+        try:
+            return self.model.model_validate(record).model_dump(mode="json"), []
+        except ValidationError as exc:
+            return None, [
+                ".".join(str(part) for part in error["loc"]) + f": {error['msg']}"
+                for error in exc.errors()
+            ]
+
+
+TARGET_SCHEMAS: dict[str, TargetSchema] = {
+    "vessel_ais": TargetSchema(
+        key="vessel_ais",
+        label="船舶 AIS",
+        description="船只位置、航速、航向、MMSI 等 AIS 数据。",
+        destination="vessel_position",
+        model=VesselAISRecord,
+        fields=(
+            TargetField("mmsi", "integer", True, "MMSI 九位船舶标识", 257123000),
+            TargetField("lat", "float", True, "纬度", 59.91),
+            TargetField("lon", "float", True, "经度", 10.75),
+            TargetField("sog", "float", False, "对地航速，单位节", 12.4),
+            TargetField("cog", "float", False, "对地航向，0-360 度", 184.5),
+            TargetField("heading", "integer", False, "船首向，0-511", 186),
+            TargetField("nav_status", "integer", False, "导航状态码", 0),
+            TargetField("name", "string", False, "船名", "OSLO EXPRESS"),
+            TargetField("callsign", "string", False, "呼号", "LAAB"),
+            TargetField("vessel_type", "string", False, "船型代码", 70),
+            TargetField("vessel_type_name", "string", False, "船型名称", "Cargo"),
+            TargetField("received_at", "datetime", False, "数据接收时间", "2026-04-28T00:00:00Z"),
+        ),
+    ),
+    "geo_points": TargetSchema(
+        key="geo_points",
+        label="通用地理点",
+        description="带经纬度的通用实体或事件点位。",
+        destination="generic_geo_points",
+        model=GeoPointRecord,
+        fields=(
+            TargetField("lat", "float", True, "纬度", 1.3),
+            TargetField("lon", "float", True, "经度", 103.8),
+            TargetField("name", "string", False, "点位名称", "Singapore"),
+            TargetField("type", "string", False, "点位类型", "datacenter"),
+            TargetField("source_id", "string", False, "来源侧 ID", "sg-1"),
+            TargetField("observed_at", "datetime", False, "观测时间", "2026-04-28T00:00:00Z"),
+            TargetField("metadata", "object", False, "扩展字段", {"provider": "example"}),
+        ),
+    ),
+    "generic_records": TargetSchema(
+        key="generic_records",
+        label="通用结构化记录",
+        description="未知结构数据沉淀，不直接进入 Earth 图层。",
+        destination="collected_data",
+        model=GenericRecord,
+        fields=(
+            TargetField("data", "object", True, "结构化记录主体", {"raw": "value"}),
+            TargetField("source_id", "string", False, "来源侧 ID", "record-1"),
+            TargetField("observed_at", "datetime", False, "观测时间", "2026-04-28T00:00:00Z"),
+        ),
+    ),
+}
+
+
+def list_target_schemas() -> list[dict[str, Any]]:
+    return [schema.to_dict() for schema in TARGET_SCHEMAS.values()]
+
+
+def get_target_schema(key: str) -> TargetSchema:
+    try:
+        return TARGET_SCHEMAS[key]
+    except KeyError as exc:
+        raise ValueError(f"Unsupported target schema: {key}") from exc

backend/app/core/websocket/broadcaster.py

@@ -2,12 +2,14 @@
 
 import asyncio
 from datetime import UTC, datetime
-from typing import Dict, Any, Optional
+from typing import Dict, Any
 
 from app.core.time import to_iso8601_utc
 from app.core.websocket.manager import manager
 
 
+EARTH_UPDATES_CHANNEL = "earth_updates"
+
 
 class DataBroadcaster:
     """Periodically broadcasts data to connected WebSocket clients"""
@@ -15,6 +17,8 @@ class DataBroadcaster:
     def __init__(self):
         self.running = False
         self.tasks: Dict[str, asyncio.Task] = {}
+        self._pending_vessel_updates: Dict[str, Dict[str, Any]] = {}
+        self._vessel_flush_interval = 1.0
 
     async def get_dashboard_stats(self) -> Dict[str, Any]:
         """Get dashboard statistics"""
@@ -68,6 +72,9 @@ class DataBroadcaster:
 
     async def broadcast_custom(self, channel: str, data: Dict[str, Any]):
         """Broadcast custom data to a specific channel"""
+        if channel == "vessels":
+            self.enqueue_vessel_update(data)
+            return
         await manager.broadcast(
             {
                 "type": "data_frame",
@@ -75,9 +82,65 @@ class DataBroadcaster:
                 "timestamp": to_iso8601_utc(datetime.now(UTC)),
                 "payload": data,
             },
-            channel=channel if channel in manager.active_connections else "all",
+            channel=channel,
         )
 
+    async def broadcast_earth_update(self, data: Dict[str, Any]):
+        """Broadcast Earth visualization refresh hints to connected clients."""
+        await self.broadcast_custom(EARTH_UPDATES_CHANNEL, data)
+
+    def enqueue_vessel_update(self, data: Dict[str, Any]):
+        vessels = data.get("vessels") if isinstance(data, dict) else None
+        if not isinstance(vessels, list):
+            return
+        source = data.get("source")
+        action = data.get("action") or "upsert"
+        created = data.get("created")
+        for vessel in vessels:
+            if not isinstance(vessel, dict):
+                continue
+            mmsi = vessel.get("mmsi")
+            if mmsi in (None, ""):
+                continue
+            self._pending_vessel_updates[str(mmsi)] = {
+                **vessel,
+                "_source": source,
+                "_action": action,
+                "_created": created,
+            }
+
+    async def flush_vessel_updates(self):
+        if not self._pending_vessel_updates:
+            return
+        pending = self._pending_vessel_updates
+        self._pending_vessel_updates = {}
+        vessels = []
+        for item in pending.values():
+            vessel = dict(item)
+            source = vessel.pop("_source", None)
+            action = vessel.pop("_action", "upsert")
+            created = vessel.pop("_created", None)
+            vessel["source"] = source
+            vessel["action"] = action
+            vessel["created"] = created
+            vessels.append(vessel)
+        await manager.broadcast_vessels(
+            {
+                "action": "upsert",
+                "source": "mixed",
+                "created": None,
+                "vessels": vessels,
+            }
+        )
+
+    async def broadcast_vessels_periodically(self):
+        while self.running:
+            try:
+                await self.flush_vessel_updates()
+            except Exception:
+                pass
+            await asyncio.sleep(self._vessel_flush_interval)
+
     async def broadcast_datasource_task_update(self, data: Dict[str, Any]):
         """Broadcast datasource task progress updates to connected clients."""
         await manager.broadcast(
@@ -87,7 +150,7 @@ class DataBroadcaster:
                 "timestamp": to_iso8601_utc(datetime.now(UTC)),
                 "payload": data,
             },
-            channel="all",
+            channel="datasource_tasks",
         )
 
     def start(self):
@@ -95,6 +158,7 @@ class DataBroadcaster:
         if not self.running:
             self.running = True
             self.tasks["dashboard"] = asyncio.create_task(self.broadcast_stats(5))
+            self.tasks["vessels"] = asyncio.create_task(self.broadcast_vessels_periodically())
 
     def stop(self):
         """Stop all broadcasters"""
@@ -102,6 +166,7 @@ class DataBroadcaster:
         for task in self.tasks.values():
             task.cancel()
         self.tasks.clear()
+        self._pending_vessel_updates.clear()
 
 
 broadcaster = DataBroadcaster()

backend/app/core/websocket/manager.py

@@ -1,20 +1,25 @@
 """WebSocket Connection Manager"""
 
-import json
+from datetime import UTC, datetime
-import asyncio
+from typing import Any, Dict, Set, Optional
-from typing import Dict, Set, Optional
-from datetime import datetime
 from fastapi import WebSocket
 import redis.asyncio as redis
 
 from app.core.config import settings
 
+MAX_VESSEL_SUBSCRIPTION_LIMIT = 5000
+MAX_VESSEL_WS_MESSAGE_ITEMS = 1000
+MAX_VESSEL_BBOX_AREA = 2500.0
+
 
 class ConnectionManager:
     """Manages WebSocket connections"""
 
     def __init__(self):
         self.active_connections: Dict[str, Set[WebSocket]] = {}  # user_id -> connections
+        self.channel_subscriptions: Dict[str, Set[WebSocket]] = {}
+        self.websocket_channels: Dict[WebSocket, Set[str]] = {}
+        self.vessel_subscriptions: Dict[WebSocket, dict[str, Any]] = {}
         self.redis_client: Optional[redis.Redis] = None
 
     async def connect(self, websocket: WebSocket, user_id: str):
@@ -40,6 +45,83 @@ class ConnectionManager:
             self.active_connections[user_id].discard(websocket)
             if not self.active_connections[user_id]:
                 del self.active_connections[user_id]
+        self.unsubscribe_all(websocket)
+
+    def subscribe(self, websocket: WebSocket, channels: list[str]):
+        normalized_channels = {
+            str(channel).strip()
+            for channel in channels
+            if str(channel).strip()
+        }
+        if not normalized_channels:
+            return
+
+        socket_channels = self.websocket_channels.setdefault(websocket, set())
+        for channel in normalized_channels:
+            self.channel_subscriptions.setdefault(channel, set()).add(websocket)
+            socket_channels.add(channel)
+
+    def unsubscribe(self, websocket: WebSocket, channels: list[str]):
+        for channel in {str(channel).strip() for channel in channels if str(channel).strip()}:
+            subscribers = self.channel_subscriptions.get(channel)
+            if subscribers is not None:
+                subscribers.discard(websocket)
+                if not subscribers:
+                    del self.channel_subscriptions[channel]
+            socket_channels = self.websocket_channels.get(websocket)
+            if socket_channels is not None:
+                socket_channels.discard(channel)
+                if not socket_channels:
+                    del self.websocket_channels[websocket]
+
+    def unsubscribe_all(self, websocket: WebSocket):
+        channels = list(self.websocket_channels.get(websocket, set()))
+        if channels:
+            self.unsubscribe(websocket, channels)
+        self.vessel_subscriptions.pop(websocket, None)
+
+    def subscribe_vessels(self, websocket: WebSocket, config: dict[str, Any]) -> dict[str, Any]:
+        subscription = self._normalize_vessel_subscription(config)
+        self.channel_subscriptions.setdefault("vessels", set()).add(websocket)
+        self.websocket_channels.setdefault(websocket, set()).add("vessels")
+        self.vessel_subscriptions[websocket] = subscription
+        return subscription
+
+    def _normalize_vessel_subscription(self, config: dict[str, Any]) -> dict[str, Any]:
+        bbox = config.get("bbox")
+        if not isinstance(bbox, (list, tuple)) or len(bbox) != 4:
+            raise ValueError("vessels subscription requires bbox=[lon_min,lat_min,lon_max,lat_max]")
+        try:
+            lon_min, lat_min, lon_max, lat_max = [float(value) for value in bbox]
+        except (TypeError, ValueError) as exc:
+            raise ValueError("bbox values must be numbers") from exc
+        if lat_min > lat_max:
+            lat_min, lat_max = lat_max, lat_min
+        if lon_min > lon_max:
+            lon_min, lon_max = lon_max, lon_min
+        if not (-180 <= lon_min <= 180 and -180 <= lon_max <= 180):
+            raise ValueError("bbox longitude values must be between -180 and 180")
+        if not (-90 <= lat_min <= 90 and -90 <= lat_max <= 90):
+            raise ValueError("bbox latitude values must be between -90 and 90")
+        if (lon_max - lon_min) * (lat_max - lat_min) > MAX_VESSEL_BBOX_AREA:
+            raise ValueError("bbox is too large; zoom in or request a smaller viewport")
+
+        zoom = int(config.get("zoom") or 1)
+        if zoom < 1 or zoom > 20:
+            raise ValueError("zoom must be between 1 and 20")
+        limit = min(max(int(config.get("limit") or 1000), 1), MAX_VESSEL_SUBSCRIPTION_LIMIT)
+        vessel_types = {
+            str(item).strip().lower()
+            for item in str(config.get("type") or "").split(",")
+            if str(item).strip()
+        }
+        return {
+            "bbox": (lon_min, lat_min, lon_max, lat_max),
+            "zoom": zoom,
+            "limit": limit,
+            "type": vessel_types,
+            "last_sent_at": None,
+        }
 
     async def send_personal_message(self, message: dict, user_id: str):
         if user_id in self.active_connections:
@@ -54,13 +136,72 @@ class ConnectionManager:
             for user_id in self.active_connections:
                 await self.send_personal_message(message, user_id)
         else:
-            await self.send_personal_message(message, channel)
+            for connection in list(self.channel_subscriptions.get(channel, set())):
+                try:
+                    await connection.send_json(message)
+                except Exception:
+                    self.unsubscribe_all(connection)
+
+    async def broadcast_vessels(self, data: dict[str, Any]):
+        vessels = data.get("vessels") if isinstance(data, dict) else None
+        if not isinstance(vessels, list) or not vessels:
+            return
+
+        for connection, subscription in list(self.vessel_subscriptions.items()):
+            matched = [
+                vessel
+                for vessel in vessels
+                if self._vessel_matches_subscription(vessel, subscription)
+            ][: min(subscription["limit"], MAX_VESSEL_WS_MESSAGE_ITEMS)]
+            if not matched:
+                continue
+            subscription["last_sent_at"] = datetime.now(UTC)
+            message = {
+                "type": "data_frame",
+                "channel": "vessels",
+                "timestamp": subscription["last_sent_at"].isoformat(),
+                "payload": {
+                    **data,
+                    "vessels": matched,
+                    "subscription": {
+                        "bbox": list(subscription["bbox"]),
+                        "zoom": subscription["zoom"],
+                        "limit": subscription["limit"],
+                    },
+                },
+            }
+            try:
+                await connection.send_json(message)
+            except Exception:
+                self.unsubscribe_all(connection)
+
+    def _vessel_matches_subscription(
+        self,
+        vessel: dict[str, Any],
+        subscription: dict[str, Any],
+    ) -> bool:
+        try:
+            lon = float(vessel.get("lon"))
+            lat = float(vessel.get("lat"))
+        except (TypeError, ValueError):
+            return False
+        lon_min, lat_min, lon_max, lat_max = subscription["bbox"]
+        if not (lon_min <= lon <= lon_max and lat_min <= lat <= lat_max):
+            return False
+        requested_types = subscription.get("type") or set()
+        if not requested_types:
+            return True
+        type_name = str(vessel.get("vessel_type_name") or "").lower()
+        return any(requested_type in type_name for requested_type in requested_types)
 
     async def close_all(self):
         for user_id in self.active_connections:
             for connection in self.active_connections[user_id]:
                 await connection.close()
         self.active_connections.clear()
+        self.channel_subscriptions.clear()
+        self.websocket_channels.clear()
+        self.vessel_subscriptions.clear()
 
 
 manager = ConnectionManager()

backend/app/data/seeds/ripe_ris_collector_locations_seed.json

@@ -0,0 +1,328 @@
+{
+  "_comment": "Seed payload for the bgp_collector_locations DB table. Coordinates were migrated from the legacy RIPE_RIS_COLLECTOR_COORDS table and default to city-center; seeded rows are unverified and should be upgraded in the database with source evidence when known.",
+  "locations": [
+    {
+      "canonical_name": "RIPE RIS rrc00",
+      "aliases": ["rrc00", "RIPE RIS rrc00", "AMS-IX"],
+      "operator": "RIPE NCC",
+      "site": "AMS-IX",
+      "city": "Amsterdam",
+      "country": "Netherlands",
+      "latitude": 52.3676,
+      "longitude": 4.9041,
+      "precision": "city",
+      "confidence": 0.85,
+      "source_note": "Migrated from RIPE_RIS_COLLECTOR_COORDS legacy table",
+      "verified_at": null
+    },
+    {
+      "canonical_name": "RIPE RIS rrc01",
+      "aliases": ["rrc01", "RIPE RIS rrc01", "LINX"],
+      "operator": "RIPE NCC",
+      "site": "LINX",
+      "city": "London",
+      "country": "United Kingdom",
+      "latitude": 51.5072,
+      "longitude": -0.1276,
+      "precision": "city",
+      "confidence": 0.85,
+      "source_note": "Migrated from RIPE_RIS_COLLECTOR_COORDS legacy table",
+      "verified_at": null
+    },
+    {
+      "canonical_name": "RIPE RIS rrc03",
+      "aliases": ["rrc03", "RIPE RIS rrc03", "AMS-IX"],
+      "operator": "RIPE NCC",
+      "site": "AMS-IX",
+      "city": "Amsterdam",
+      "country": "Netherlands",
+      "latitude": 52.3676,
+      "longitude": 4.9041,
+      "precision": "city",
+      "confidence": 0.85,
+      "source_note": "Migrated from RIPE_RIS_COLLECTOR_COORDS legacy table",
+      "verified_at": null
+    },
+    {
+      "canonical_name": "RIPE RIS rrc04",
+      "aliases": ["rrc04", "RIPE RIS rrc04", "CIXP", "CERN Internet Exchange Point"],
+      "operator": "RIPE NCC",
+      "site": "CIXP",
+      "city": "Geneva",
+      "country": "Switzerland",
+      "latitude": 46.2044,
+      "longitude": 6.1432,
+      "precision": "city",
+      "confidence": 0.85,
+      "source_note": "Migrated from RIPE_RIS_COLLECTOR_COORDS legacy table",
+      "verified_at": null
+    },
+    {
+      "canonical_name": "RIPE RIS rrc05",
+      "aliases": ["rrc05", "RIPE RIS rrc05", "VIX", "Vienna Internet Exchange"],
+      "operator": "RIPE NCC",
+      "site": "VIX",
+      "city": "Vienna",
+      "country": "Austria",
+      "latitude": 48.2082,
+      "longitude": 16.3738,
+      "precision": "city",
+      "confidence": 0.85,
+      "source_note": "Migrated from RIPE_RIS_COLLECTOR_COORDS legacy table",
+      "verified_at": null
+    },
+    {
+      "canonical_name": "RIPE RIS rrc06",
+      "aliases": ["rrc06", "RIPE RIS rrc06", "JPIX", "Otemachi"],
+      "operator": "RIPE NCC",
+      "site": "JPIX",
+      "city": "Otemachi",
+      "country": "Japan",
+      "latitude": 35.686,
+      "longitude": 139.7671,
+      "precision": "city",
+      "confidence": 0.85,
+      "source_note": "Migrated from RIPE_RIS_COLLECTOR_COORDS legacy table",
+      "verified_at": null
+    },
+    {
+      "canonical_name": "RIPE RIS rrc07",
+      "aliases": ["rrc07", "RIPE RIS rrc07", "Netnod", "Netnod Stockholm"],
+      "operator": "RIPE NCC",
+      "site": "Netnod Stockholm",
+      "city": "Stockholm",
+      "country": "Sweden",
+      "latitude": 59.3293,
+      "longitude": 18.0686,
+      "precision": "city",
+      "confidence": 0.85,
+      "source_note": "Migrated from RIPE_RIS_COLLECTOR_COORDS legacy table",
+      "verified_at": null
+    },
+    {
+      "canonical_name": "RIPE RIS rrc10",
+      "aliases": ["rrc10", "RIPE RIS rrc10", "MIX", "Milan Internet Exchange"],
+      "operator": "RIPE NCC",
+      "site": "MIX",
+      "city": "Milan",
+      "country": "Italy",
+      "latitude": 45.4642,
+      "longitude": 9.19,
+      "precision": "city",
+      "confidence": 0.85,
+      "source_note": "Migrated from RIPE_RIS_COLLECTOR_COORDS legacy table",
+      "verified_at": null
+    },
+    {
+      "canonical_name": "RIPE RIS rrc11",
+      "aliases": ["rrc11", "RIPE RIS rrc11", "NYIIX", "New York International Internet Exchange"],
+      "operator": "RIPE NCC",
+      "site": "NYIIX",
+      "city": "New York",
+      "country": "United States",
+      "latitude": 40.7128,
+      "longitude": -74.006,
+      "precision": "city",
+      "confidence": 0.85,
+      "source_note": "Migrated from RIPE_RIS_COLLECTOR_COORDS legacy table",
+      "verified_at": null
+    },
+    {
+      "canonical_name": "RIPE RIS rrc12",
+      "aliases": ["rrc12", "RIPE RIS rrc12", "DE-CIX", "DE-CIX Frankfurt"],
+      "operator": "RIPE NCC",
+      "site": "DE-CIX Frankfurt",
+      "city": "Frankfurt",
+      "country": "Germany",
+      "latitude": 50.1109,
+      "longitude": 8.6821,
+      "precision": "city",
+      "confidence": 0.85,
+      "source_note": "Migrated from RIPE_RIS_COLLECTOR_COORDS legacy table",
+      "verified_at": null
+    },
+    {
+      "canonical_name": "RIPE RIS rrc13",
+      "aliases": ["rrc13", "RIPE RIS rrc13", "MSK-IX"],
+      "operator": "RIPE NCC",
+      "site": "MSK-IX",
+      "city": "Moscow",
+      "country": "Russia",
+      "latitude": 55.7558,
+      "longitude": 37.6173,
+      "precision": "city",
+      "confidence": 0.85,
+      "source_note": "Migrated from RIPE_RIS_COLLECTOR_COORDS legacy table",
+      "verified_at": null
+    },
+    {
+      "canonical_name": "RIPE RIS rrc14",
+      "aliases": ["rrc14", "RIPE RIS rrc14", "PAIX", "Palo Alto Internet Exchange"],
+      "operator": "RIPE NCC",
+      "site": "PAIX",
+      "city": "Palo Alto",
+      "country": "United States",
+      "latitude": 37.4419,
+      "longitude": -122.143,
+      "precision": "city",
+      "confidence": 0.85,
+      "source_note": "Migrated from RIPE_RIS_COLLECTOR_COORDS legacy table",
+      "verified_at": null
+    },
+    {
+      "canonical_name": "RIPE RIS rrc15",
+      "aliases": ["rrc15", "RIPE RIS rrc15", "PTT.br Sao Paulo", "PTTMetro Sao Paulo"],
+      "operator": "RIPE NCC",
+      "site": "PTT.br",
+      "city": "Sao Paulo",
+      "country": "Brazil",
+      "latitude": -23.5558,
+      "longitude": -46.6396,
+      "precision": "city",
+      "confidence": 0.85,
+      "source_note": "Migrated from RIPE_RIS_COLLECTOR_COORDS legacy table",
+      "verified_at": null
+    },
+    {
+      "canonical_name": "RIPE RIS rrc16",
+      "aliases": ["rrc16", "RIPE RIS rrc16", "Equinix Miami", "NOTA Miami"],
+      "operator": "RIPE NCC",
+      "site": "Equinix Miami",
+      "city": "Miami",
+      "country": "United States",
+      "latitude": 25.7617,
+      "longitude": -80.1918,
+      "precision": "city",
+      "confidence": 0.85,
+      "source_note": "Migrated from RIPE_RIS_COLLECTOR_COORDS legacy table",
+      "verified_at": null
+    },
+    {
+      "canonical_name": "RIPE RIS rrc18",
+      "aliases": ["rrc18", "RIPE RIS rrc18", "CATNIX"],
+      "operator": "RIPE NCC",
+      "site": "CATNIX",
+      "city": "Barcelona",
+      "country": "Spain",
+      "latitude": 41.3874,
+      "longitude": 2.1686,
+      "precision": "city",
+      "confidence": 0.85,
+      "source_note": "Migrated from RIPE_RIS_COLLECTOR_COORDS legacy table",
+      "verified_at": null
+    },
+    {
+      "canonical_name": "RIPE RIS rrc19",
+      "aliases": ["rrc19", "RIPE RIS rrc19", "NAPAfrica", "JINX", "NAPAfrica Johannesburg"],
+      "operator": "RIPE NCC",
+      "site": "NAPAfrica Johannesburg",
+      "city": "Johannesburg",
+      "country": "South Africa",
+      "latitude": -26.2041,
+      "longitude": 28.0473,
+      "precision": "city",
+      "confidence": 0.85,
+      "source_note": "Migrated from RIPE_RIS_COLLECTOR_COORDS legacy table",
+      "verified_at": null
+    },
+    {
+      "canonical_name": "RIPE RIS rrc20",
+      "aliases": ["rrc20", "RIPE RIS rrc20", "SwissIX"],
+      "operator": "RIPE NCC",
+      "site": "SwissIX",
+      "city": "Zurich",
+      "country": "Switzerland",
+      "latitude": 47.3769,
+      "longitude": 8.5417,
+      "precision": "city",
+      "confidence": 0.85,
+      "source_note": "Migrated from RIPE_RIS_COLLECTOR_COORDS legacy table",
+      "verified_at": null
+    },
+    {
+      "canonical_name": "RIPE RIS rrc21",
+      "aliases": ["rrc21", "RIPE RIS rrc21", "France-IX Paris"],
+      "operator": "RIPE NCC",
+      "site": "France-IX Paris",
+      "city": "Paris",
+      "country": "France",
+      "latitude": 48.8566,
+      "longitude": 2.3522,
+      "precision": "city",
+      "confidence": 0.85,
+      "source_note": "Migrated from RIPE_RIS_COLLECTOR_COORDS legacy table",
+      "verified_at": null
+    },
+    {
+      "canonical_name": "RIPE RIS rrc22",
+      "aliases": ["rrc22", "RIPE RIS rrc22", "InterLAN Bucharest"],
+      "operator": "RIPE NCC",
+      "site": "InterLAN Bucharest",
+      "city": "Bucharest",
+      "country": "Romania",
+      "latitude": 44.4268,
+      "longitude": 26.1025,
+      "precision": "city",
+      "confidence": 0.85,
+      "source_note": "Migrated from RIPE_RIS_COLLECTOR_COORDS legacy table",
+      "verified_at": null
+    },
+    {
+      "canonical_name": "RIPE RIS rrc23",
+      "aliases": ["rrc23", "RIPE RIS rrc23", "Equinix Singapore"],
+      "operator": "RIPE NCC",
+      "site": "Equinix Singapore",
+      "city": "Singapore",
+      "country": "Singapore",
+      "latitude": 1.3521,
+      "longitude": 103.8198,
+      "precision": "city",
+      "confidence": 0.85,
+      "source_note": "Migrated from RIPE_RIS_COLLECTOR_COORDS legacy table",
+      "verified_at": null
+    },
+    {
+      "canonical_name": "RIPE RIS rrc24",
+      "aliases": ["rrc24", "RIPE RIS rrc24", "LACNIC Montevideo"],
+      "operator": "RIPE NCC",
+      "site": "LACNIC Montevideo",
+      "city": "Montevideo",
+      "country": "Uruguay",
+      "latitude": -34.9011,
+      "longitude": -56.1645,
+      "precision": "city",
+      "confidence": 0.85,
+      "source_note": "Migrated from RIPE_RIS_COLLECTOR_COORDS legacy table",
+      "verified_at": null
+    },
+    {
+      "canonical_name": "RIPE RIS rrc25",
+      "aliases": ["rrc25", "RIPE RIS rrc25", "AMS-IX"],
+      "operator": "RIPE NCC",
+      "site": "AMS-IX",
+      "city": "Amsterdam",
+      "country": "Netherlands",
+      "latitude": 52.3676,
+      "longitude": 4.9041,
+      "precision": "city",
+      "confidence": 0.85,
+      "source_note": "Migrated from RIPE_RIS_COLLECTOR_COORDS legacy table",
+      "verified_at": null
+    },
+    {
+      "canonical_name": "RIPE RIS rrc26",
+      "aliases": ["rrc26", "RIPE RIS rrc26", "UAE-IX"],
+      "operator": "RIPE NCC",
+      "site": "UAE-IX",
+      "city": "Dubai",
+      "country": "United Arab Emirates",
+      "latitude": 25.2048,
+      "longitude": 55.2708,
+      "precision": "city",
+      "confidence": 0.85,
+      "source_note": "Migrated from RIPE_RIS_COLLECTOR_COORDS legacy table",
+      "verified_at": null
+    }
+  ],
+  "city_fallbacks": []
+}

backend/app/db/session.py

@@ -1,14 +1,26 @@
 from typing import AsyncGenerator
 
-from sqlalchemy import text
+from sqlalchemy import bindparam, text
 from sqlalchemy.ext.asyncio import AsyncSession, create_async_engine, async_sessionmaker
 from sqlalchemy.orm import declarative_base
 
 from app.core.config import settings
+from app.core.logging import get_logger
+
+logger = get_logger(__name__)
+
+DB_POOL_CONFIG = {
+    "pool_pre_ping": True,
+    "pool_recycle": 1800,
+    "pool_size": 10,
+    "max_overflow": 20,
+    "pool_timeout": 30,
+}
 
 engine = create_async_engine(
     settings.DATABASE_URL,
     echo=settings.DEBUG if hasattr(settings, "DEBUG") else False,
+    **DB_POOL_CONFIG,
 )
 
 async_session_factory = async_sessionmaker(engine, class_=AsyncSession, expire_on_commit=False)
@@ -60,25 +72,112 @@ async def seed_default_datasources(session: AsyncSession):
     await session.commit()
 
 
+LEGACY_EARTH_BOUNDARY_SOURCES = (
+    "earth_admin0_boundaries",
+    "earth_coastline",
+    "earth_claim_lines",
+    "earth_boundary_tiles",
+)
+LEGACY_EARTH_BOUNDARY_DATATYPES = (
+    "earth_boundary_source",
+    "earth_boundary_tiles",
+)
+LEGACY_EARTH_BOUNDARY_IDS = (29, 30, 31, 32)
+
+
+async def purge_legacy_earth_boundary_datasources(session: AsyncSession) -> None:
+    source_names = tuple(LEGACY_EARTH_BOUNDARY_SOURCES)
+    source_ids = tuple(LEGACY_EARTH_BOUNDARY_IDS)
+    data_types = tuple(LEGACY_EARTH_BOUNDARY_DATATYPES)
+    await session.execute(
+        text(
+            """
+            DELETE FROM datasource_mapping_templates
+            WHERE target_schema IN :data_types
+               OR datasource_config_id IN (
+                    SELECT id FROM datasource_configs WHERE name IN :source_names
+               )
+            """
+        ).bindparams(bindparam("source_names", expanding=True), bindparam("data_types", expanding=True)),
+        {"source_names": list(source_names), "data_types": list(data_types)},
+    )
+    await session.execute(
+        text("DELETE FROM datasource_configs WHERE name IN :source_names").bindparams(
+            bindparam("source_names", expanding=True)
+        ),
+        {"source_names": list(source_names)},
+    )
+    await session.execute(
+        text(
+            """
+            DELETE FROM collected_data
+            WHERE source IN :source_names OR data_type IN :data_types
+            """
+        ).bindparams(bindparam("source_names", expanding=True), bindparam("data_types", expanding=True)),
+        {"source_names": list(source_names), "data_types": list(data_types)},
+    )
+    await session.execute(
+        text(
+            """
+            DELETE FROM data_snapshots
+            WHERE source IN :source_names OR datasource_id IN :source_ids
+            """
+        ).bindparams(bindparam("source_names", expanding=True), bindparam("source_ids", expanding=True)),
+        {"source_names": list(source_names), "source_ids": list(source_ids)},
+    )
+    await session.execute(
+        text("DELETE FROM collection_tasks WHERE datasource_id IN :source_ids").bindparams(
+            bindparam("source_ids", expanding=True)
+        ),
+        {"source_ids": list(source_ids)},
+    )
+    await session.execute(
+        text("DELETE FROM data_sources WHERE source IN :source_names OR id IN :source_ids").bindparams(
+            bindparam("source_names", expanding=True), bindparam("source_ids", expanding=True)
+        ),
+        {"source_names": list(source_names), "source_ids": list(source_ids)},
+    )
+    await session.commit()
+
+
+DEFAULT_LOGIN_USERS = (
+    {
+        "username": "admin",
+        "email": "admin@planet.local",
+        "password": "admin123",
+        "role": "super_admin",
+    },
+    {
+        "username": "linkong",
+        "email": "linkong@planet.local",
+        "password": "LK12345678",
+        "role": "super_admin",
+    },
+)
+
+
 async def ensure_default_admin_user(session: AsyncSession):
     from app.core.security import get_password_hash
     from app.models.user import User
 
-    result = await session.execute(
+    for default_user in DEFAULT_LOGIN_USERS:
-        text("SELECT id FROM users WHERE username = 'admin'")
+        result = await session.execute(
-    )
+            text("SELECT id FROM users WHERE username = :username"),
-    if result.fetchone():
+            {"username": default_user["username"]},
-        return
+        )
-
+        if result.fetchone():
-    session.add(
+            continue
-        User(
+
-            username="admin",
+        session.add(
-            email="admin@planet.local",
+            User(
-            password_hash=get_password_hash("admin123"),
+                username=default_user["username"],
-            role="super_admin",
+                email=default_user["email"],
-            is_active=True,
+                password_hash=get_password_hash(default_user["password"]),
+                role=default_user["role"],
+                is_active=True,
+                email_verified=True,
+            )
         )
-    )
     await session.commit()
 
 
@@ -91,15 +190,60 @@ async def init_db():
     import app.models.datasource_config  # noqa: F401
     import app.models.alert  # noqa: F401
     import app.models.bgp_anomaly  # noqa: F401
+    import app.models.bgp_collector_location  # noqa: F401
     import app.models.bgp_incident  # noqa: F401
     import app.models.bgp_observation  # noqa: F401
     import app.models.collected_data  # noqa: F401
+    import app.models.compute_center_location  # noqa: F401
     import app.models.system_setting  # noqa: F401
     import app.models.playground_session  # noqa: F401
     import app.models.playground_message  # noqa: F401
+    import app.models.system_log  # noqa: F401
+    import app.models.vessel  # noqa: F401
+    import app.models.vessel_enrichment  # noqa: F401
+    import app.models.datasource_mapping  # noqa: F401
+    import app.models.earth_news  # noqa: F401
+    import app.models.earth_interactable  # noqa: F401
+
+    logger.warning_event(
+        "Database pool settings active",
+        event="database.pool.initialized",
+        context={
+            "pool_pre_ping": DB_POOL_CONFIG["pool_pre_ping"],
+            "pool_recycle": DB_POOL_CONFIG["pool_recycle"],
+            "pool_size": DB_POOL_CONFIG["pool_size"],
+            "max_overflow": DB_POOL_CONFIG["max_overflow"],
+            "pool_timeout": DB_POOL_CONFIG["pool_timeout"],
+        },
+    )
 
     async with engine.begin() as conn:
         await conn.run_sync(Base.metadata.create_all)
+        users_email_verified_existed = (
+            await conn.execute(
+                text(
+                    """
+                    SELECT 1
+                    FROM information_schema.columns
+                    WHERE table_name = 'users' AND column_name = 'email_verified'
+                    """
+                )
+            )
+        ).fetchone() is not None
+        await conn.execute(
+            text(
+                """
+                ALTER TABLE users
+                ADD COLUMN IF NOT EXISTS gatekeeper_groups JSONB DEFAULT '[]'::jsonb,
+                ADD COLUMN IF NOT EXISTS email_verified BOOLEAN NOT NULL DEFAULT FALSE,
+                ADD COLUMN IF NOT EXISTS pending_email VARCHAR(255)
+                """
+            )
+        )
+        if not users_email_verified_existed:
+            await conn.execute(
+                text("UPDATE users SET email_verified = TRUE WHERE email_verified = FALSE")
+            )
         await conn.execute(
             text(
                 """
@@ -115,11 +259,449 @@ async def init_db():
                 """
             )
         )
+        await conn.execute(
+            text(
+                """
+                CREATE TABLE IF NOT EXISTS earth_data_change_events (
+                    id BIGSERIAL PRIMARY KEY,
+                    table_name VARCHAR(128) NOT NULL,
+                    operation VARCHAR(16) NOT NULL,
+                    source VARCHAR(128),
+                    entity_key VARCHAR(255),
+                    payload JSONB NOT NULL,
+                    occurred_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
+                    consumed_at TIMESTAMPTZ
+                )
+                """
+            )
+        )
+        await conn.execute(
+            text(
+                """
+                CREATE INDEX IF NOT EXISTS idx_earth_data_change_events_unconsumed
+                ON earth_data_change_events (consumed_at, id)
+                WHERE consumed_at IS NULL
+                """
+            )
+        )
+        await conn.execute(
+            text(
+                """
+                CREATE OR REPLACE FUNCTION planet_emit_earth_data_changed_statement(
+                    change_table TEXT,
+                    change_operation TEXT,
+                    change_source TEXT,
+                    source_record_count INTEGER,
+                    source_entity_keys TEXT[]
+                )
+                RETURNS VOID AS $$
+                DECLARE
+                    change_event_id BIGINT;
+                    change_payload JSONB;
+                BEGIN
+                    change_payload := jsonb_build_object(
+                        'event', 'earth.layer.changed',
+                        'table', change_table,
+                        'operation', change_operation,
+                        'source', change_source,
+                        'entity_key', NULL,
+                        'entity_keys', COALESCE(to_jsonb(source_entity_keys), '[]'::jsonb),
+                        'records_processed', COALESCE(source_record_count, 0),
+                        'occurred_at', NOW()
+                    );
+
+                    INSERT INTO earth_data_change_events (
+                        table_name,
+                        operation,
+                        source,
+                        entity_key,
+                        payload,
+                        occurred_at
+                    ) VALUES (
+                        change_table,
+                        change_operation,
+                        change_source,
+                        NULL,
+                        change_payload,
+                        NOW()
+                    )
+                    RETURNING id INTO change_event_id;
+
+                    change_payload := change_payload || jsonb_build_object(
+                        'event_id', change_event_id
+                    );
+
+                    UPDATE earth_data_change_events
+                    SET payload = change_payload
+                    WHERE id = change_event_id;
+
+                    PERFORM pg_notify(
+                        'planet_earth_data_changes',
+                        change_payload::text
+                    );
+                END;
+                $$ LANGUAGE plpgsql;
+                """
+            )
+        )
+        await conn.execute(
+            text(
+                """
+                CREATE OR REPLACE FUNCTION planet_emit_collected_data_changed_statement(
+                    change_operation TEXT,
+                    change_source TEXT,
+                    source_record_count INTEGER,
+                    source_entity_keys TEXT[]
+                )
+                RETURNS VOID AS $$
+                BEGIN
+                    PERFORM planet_emit_earth_data_changed_statement(
+                        'collected_data',
+                        change_operation,
+                        change_source,
+                        source_record_count,
+                        source_entity_keys
+                    );
+                END;
+                $$ LANGUAGE plpgsql;
+                """
+            )
+        )
+        await conn.execute(
+            text(
+                """
+                CREATE OR REPLACE FUNCTION planet_notify_earth_table_changed_statement()
+                RETURNS trigger AS $$
+                DECLARE
+                    change_source TEXT;
+                    source_record_count INTEGER;
+                    source_entity_keys TEXT[];
+                BEGIN
+                    IF TG_OP = 'INSERT' THEN
+                        FOR change_source IN
+                            SELECT DISTINCT COALESCE(NULLIF(row_data->>'source', ''), TG_TABLE_NAME)
+                            FROM (SELECT to_jsonb(t) AS row_data FROM new_rows AS t) changed_rows
+                        LOOP
+                            SELECT
+                                COUNT(*),
+                                ARRAY(
+                                    SELECT DISTINCT COALESCE(
+                                        NULLIF(row_data->>'entity_key', ''),
+                                        NULLIF(row_data->>'source_id', ''),
+                                        NULLIF(row_data->>'incident_key', ''),
+                                        NULLIF(row_data->>'id', ''),
+                                        NULLIF(row_data->>'mmsi', '')
+                                    )
+                                    FROM (SELECT to_jsonb(t) AS row_data FROM new_rows AS t) rows_for_keys
+                                    WHERE COALESCE(NULLIF(row_data->>'source', ''), TG_TABLE_NAME) = change_source
+                                    LIMIT 20
+                                )
+                            INTO source_record_count, source_entity_keys
+                            FROM (SELECT to_jsonb(t) AS row_data FROM new_rows AS t) rows_for_count
+                            WHERE COALESCE(NULLIF(row_data->>'source', ''), TG_TABLE_NAME) = change_source;
+
+                            PERFORM planet_emit_earth_data_changed_statement(
+                                TG_TABLE_NAME,
+                                TG_OP,
+                                change_source,
+                                source_record_count,
+                                source_entity_keys
+                            );
+                        END LOOP;
+                    ELSIF TG_OP = 'DELETE' THEN
+                        FOR change_source IN
+                            SELECT DISTINCT COALESCE(NULLIF(row_data->>'source', ''), TG_TABLE_NAME)
+                            FROM (SELECT to_jsonb(t) AS row_data FROM old_rows AS t) changed_rows
+                        LOOP
+                            SELECT
+                                COUNT(*),
+                                ARRAY(
+                                    SELECT DISTINCT COALESCE(
+                                        NULLIF(row_data->>'entity_key', ''),
+                                        NULLIF(row_data->>'source_id', ''),
+                                        NULLIF(row_data->>'incident_key', ''),
+                                        NULLIF(row_data->>'id', ''),
+                                        NULLIF(row_data->>'mmsi', '')
+                                    )
+                                    FROM (SELECT to_jsonb(t) AS row_data FROM old_rows AS t) rows_for_keys
+                                    WHERE COALESCE(NULLIF(row_data->>'source', ''), TG_TABLE_NAME) = change_source
+                                    LIMIT 20
+                                )
+                            INTO source_record_count, source_entity_keys
+                            FROM (SELECT to_jsonb(t) AS row_data FROM old_rows AS t) rows_for_count
+                            WHERE COALESCE(NULLIF(row_data->>'source', ''), TG_TABLE_NAME) = change_source;
+
+                            PERFORM planet_emit_earth_data_changed_statement(
+                                TG_TABLE_NAME,
+                                TG_OP,
+                                change_source,
+                                source_record_count,
+                                source_entity_keys
+                            );
+                        END LOOP;
+                    ELSIF TG_OP = 'UPDATE' THEN
+                        FOR change_source IN
+                            SELECT DISTINCT COALESCE(NULLIF(row_data->>'source', ''), TG_TABLE_NAME)
+                            FROM (
+                                SELECT to_jsonb(t) AS row_data FROM new_rows AS t
+                                UNION ALL
+                                SELECT to_jsonb(t) AS row_data FROM old_rows AS t
+                            ) changed_rows
+                        LOOP
+                            SELECT
+                                COUNT(*),
+                                ARRAY(
+                                    SELECT DISTINCT COALESCE(
+                                        NULLIF(row_data->>'entity_key', ''),
+                                        NULLIF(row_data->>'source_id', ''),
+                                        NULLIF(row_data->>'incident_key', ''),
+                                        NULLIF(row_data->>'id', ''),
+                                        NULLIF(row_data->>'mmsi', '')
+                                    )
+                                    FROM (
+                                        SELECT to_jsonb(t) AS row_data FROM new_rows AS t
+                                        UNION ALL
+                                        SELECT to_jsonb(t) AS row_data FROM old_rows AS t
+                                    ) rows_for_keys
+                                    WHERE COALESCE(NULLIF(row_data->>'source', ''), TG_TABLE_NAME) = change_source
+                                    LIMIT 20
+                                )
+                            INTO source_record_count, source_entity_keys
+                            FROM (
+                                SELECT to_jsonb(t) AS row_data FROM new_rows AS t
+                                UNION ALL
+                                SELECT to_jsonb(t) AS row_data FROM old_rows AS t
+                            ) rows_for_count
+                            WHERE COALESCE(NULLIF(row_data->>'source', ''), TG_TABLE_NAME) = change_source;
+
+                            PERFORM planet_emit_earth_data_changed_statement(
+                                TG_TABLE_NAME,
+                                TG_OP,
+                                change_source,
+                                source_record_count,
+                                source_entity_keys
+                            );
+                        END LOOP;
+                    END IF;
+
+                    RETURN NULL;
+                END;
+                $$ LANGUAGE plpgsql;
+                """
+            )
+        )
+        await conn.execute(
+            text(
+                """
+                CREATE OR REPLACE FUNCTION planet_notify_collected_data_changed_statement()
+                RETURNS trigger AS $$
+                DECLARE
+                    change_source TEXT;
+                    source_record_count INTEGER;
+                    source_entity_keys TEXT[];
+                BEGIN
+                    IF TG_OP = 'INSERT' THEN
+                        FOR change_source IN
+                            SELECT DISTINCT source FROM new_rows WHERE source IS NOT NULL
+                        LOOP
+                            SELECT
+                                COUNT(*),
+                                ARRAY(
+                                    SELECT DISTINCT COALESCE(entity_key, source_id, id::text)
+                                    FROM new_rows
+                                    WHERE source = change_source
+                                    LIMIT 20
+                                )
+                            INTO source_record_count, source_entity_keys
+                            FROM new_rows
+                            WHERE source = change_source;
+
+                            PERFORM planet_emit_collected_data_changed_statement(
+                                TG_OP,
+                                change_source,
+                                source_record_count,
+                                source_entity_keys
+                            );
+                        END LOOP;
+                    ELSIF TG_OP = 'DELETE' THEN
+                        FOR change_source IN
+                            SELECT DISTINCT source FROM old_rows WHERE source IS NOT NULL
+                        LOOP
+                            SELECT
+                                COUNT(*),
+                                ARRAY(
+                                    SELECT DISTINCT COALESCE(entity_key, source_id, id::text)
+                                    FROM old_rows
+                                    WHERE source = change_source
+                                    LIMIT 20
+                                )
+                            INTO source_record_count, source_entity_keys
+                            FROM old_rows
+                            WHERE source = change_source;
+
+                            PERFORM planet_emit_collected_data_changed_statement(
+                                TG_OP,
+                                change_source,
+                                source_record_count,
+                                source_entity_keys
+                            );
+                        END LOOP;
+                    ELSIF TG_OP = 'UPDATE' THEN
+                        FOR change_source IN
+                            SELECT DISTINCT source FROM (
+                                SELECT source FROM new_rows
+                                UNION
+                                SELECT source FROM old_rows
+                            ) changed_sources
+                            WHERE source IS NOT NULL
+                        LOOP
+                            SELECT
+                                COUNT(*),
+                                ARRAY(
+                                    SELECT DISTINCT COALESCE(entity_key, source_id, id::text)
+                                    FROM (
+                                        SELECT id, source_id, entity_key, source FROM new_rows
+                                        UNION ALL
+                                        SELECT id, source_id, entity_key, source FROM old_rows
+                                    ) changed_rows
+                                    WHERE source = change_source
+                                    LIMIT 20
+                                )
+                            INTO source_record_count, source_entity_keys
+                            FROM (
+                                SELECT id, source_id, entity_key, source FROM new_rows
+                                UNION ALL
+                                SELECT id, source_id, entity_key, source FROM old_rows
+                            ) changed_rows
+                            WHERE source = change_source;
+
+                            PERFORM planet_emit_collected_data_changed_statement(
+                                TG_OP,
+                                change_source,
+                                source_record_count,
+                                source_entity_keys
+                            );
+                        END LOOP;
+                    END IF;
+
+                    RETURN NULL;
+                END;
+                $$ LANGUAGE plpgsql;
+                """
+            )
+        )
+        for statement in (
+            "DROP TRIGGER IF EXISTS tr_planet_collected_data_changed ON collected_data",
+            "DROP TRIGGER IF EXISTS tr_planet_collected_data_changed_insert ON collected_data",
+            "DROP TRIGGER IF EXISTS tr_planet_collected_data_changed_update ON collected_data",
+            "DROP TRIGGER IF EXISTS tr_planet_collected_data_changed_delete ON collected_data",
+            "DROP FUNCTION IF EXISTS planet_notify_collected_data_changed()",
+            """
+            CREATE TRIGGER tr_planet_collected_data_changed_insert
+            AFTER INSERT ON collected_data
+            REFERENCING NEW TABLE AS new_rows
+            FOR EACH STATEMENT
+            EXECUTE FUNCTION planet_notify_collected_data_changed_statement()
+            """,
+            """
+            CREATE TRIGGER tr_planet_collected_data_changed_update
+            AFTER UPDATE ON collected_data
+            REFERENCING OLD TABLE AS old_rows NEW TABLE AS new_rows
+            FOR EACH STATEMENT
+            EXECUTE FUNCTION planet_notify_collected_data_changed_statement()
+            """,
+            """
+            CREATE TRIGGER tr_planet_collected_data_changed_delete
+            AFTER DELETE ON collected_data
+            REFERENCING OLD TABLE AS old_rows
+            FOR EACH STATEMENT
+            EXECUTE FUNCTION planet_notify_collected_data_changed_statement()
+            """,
+        ):
+            await conn.execute(text(statement))
+        for table_name in (
+            "bgp_observations",
+            "bgp_anomalies",
+            "bgp_incidents",
+            "bgp_collector_locations",
+            "vessel_static",
+            "vessel_position",
+            "vessel_current_state",
+            "ais_raw_observations",
+            "ais_source_health",
+            "compute_center_locations",
+            "earth_interactables",
+            "earth_news_items",
+        ):
+            for statement in (
+                f"DROP TRIGGER IF EXISTS tr_planet_{table_name}_changed_insert ON {table_name}",
+                f"DROP TRIGGER IF EXISTS tr_planet_{table_name}_changed_update ON {table_name}",
+                f"DROP TRIGGER IF EXISTS tr_planet_{table_name}_changed_delete ON {table_name}",
+                f"""
+                CREATE TRIGGER tr_planet_{table_name}_changed_insert
+                AFTER INSERT ON {table_name}
+                REFERENCING NEW TABLE AS new_rows
+                FOR EACH STATEMENT
+                EXECUTE FUNCTION planet_notify_earth_table_changed_statement()
+                """,
+                f"""
+                CREATE TRIGGER tr_planet_{table_name}_changed_update
+                AFTER UPDATE ON {table_name}
+                REFERENCING OLD TABLE AS old_rows NEW TABLE AS new_rows
+                FOR EACH STATEMENT
+                EXECUTE FUNCTION planet_notify_earth_table_changed_statement()
+                """,
+                f"""
+                CREATE TRIGGER tr_planet_{table_name}_changed_delete
+                AFTER DELETE ON {table_name}
+                REFERENCING OLD TABLE AS old_rows
+                FOR EACH STATEMENT
+                EXECUTE FUNCTION planet_notify_earth_table_changed_statement()
+                """,
+            ):
+                await conn.execute(text(statement))
         await conn.execute(
             text(
                 """
                 ALTER TABLE collection_tasks
-                ADD COLUMN IF NOT EXISTS phase VARCHAR(30) DEFAULT 'queued'
+                ADD COLUMN IF NOT EXISTS phase VARCHAR(30) DEFAULT 'queued',
+                ADD COLUMN IF NOT EXISTS phase_progress DOUBLE PRECISION,
+                ADD COLUMN IF NOT EXISTS phase_message VARCHAR(255),
+                ADD COLUMN IF NOT EXISTS phase_current BIGINT,
+                ADD COLUMN IF NOT EXISTS phase_total BIGINT,
+                ADD COLUMN IF NOT EXISTS phase_unit VARCHAR(30),
+                ADD COLUMN IF NOT EXISTS source VARCHAR(100),
+                ADD COLUMN IF NOT EXISTS task_type VARCHAR(30) NOT NULL DEFAULT 'collect',
+                ADD COLUMN IF NOT EXISTS payload JSONB NOT NULL DEFAULT '{}'::jsonb,
+                ADD COLUMN IF NOT EXISTS rollback_policy VARCHAR(40) NOT NULL DEFAULT 'keep_committed_batches',
+                ADD COLUMN IF NOT EXISTS dedupe_key VARCHAR(180),
+                ADD COLUMN IF NOT EXISTS worker_id VARCHAR(120),
+                ADD COLUMN IF NOT EXISTS locked_at TIMESTAMPTZ,
+                ADD COLUMN IF NOT EXISTS requested_cancel_at TIMESTAMPTZ,
+                ADD COLUMN IF NOT EXISTS cancel_reason TEXT
+                """
+            )
+        )
+        await conn.execute(
+            text(
+                """
+                ALTER TABLE earth_news_items
+                ADD COLUMN IF NOT EXISTS content_language VARCHAR(32) NOT NULL DEFAULT 'en',
+                ADD COLUMN IF NOT EXISTS localizations JSONB NOT NULL DEFAULT '{}'::jsonb,
+                ADD COLUMN IF NOT EXISTS enrichment_status VARCHAR(80) NOT NULL DEFAULT 'pending',
+                ADD COLUMN IF NOT EXISTS enrichment_error TEXT,
+                ADD COLUMN IF NOT EXISTS enriched_at TIMESTAMPTZ
+                """
+            )
+        )
+        await conn.execute(
+            text(
+                """
+                ALTER TABLE earth_interactables
+                ADD COLUMN IF NOT EXISTS altitude DOUBLE PRECISION,
+                ADD COLUMN IF NOT EXISTS revision INTEGER NOT NULL DEFAULT 1,
+                ADD COLUMN IF NOT EXISTS is_deleted BOOLEAN NOT NULL DEFAULT FALSE,
+                ADD COLUMN IF NOT EXISTS deleted_at TIMESTAMPTZ
                 """
             )
         )
@@ -131,6 +713,124 @@ async def init_db():
                 """
             )
         )
+        await conn.execute(
+            text(
+                """
+                CREATE INDEX IF NOT EXISTS idx_earth_news_enrichment_status
+                ON earth_news_items (enrichment_status)
+                """
+            )
+        )
+        await conn.execute(
+            text(
+                """
+                CREATE INDEX IF NOT EXISTS idx_earth_news_enriched_at
+                ON earth_news_items (enriched_at)
+                """
+            )
+        )
+        await conn.execute(
+            text(
+                """
+                CREATE INDEX IF NOT EXISTS idx_earth_interactables_layer_deleted
+                ON earth_interactables (layer, is_deleted)
+                """
+            )
+        )
+        await conn.execute(
+            text(
+                """
+                CREATE INDEX IF NOT EXISTS idx_earth_interactables_updated_at
+                ON earth_interactables (updated_at)
+                """
+            )
+        )
+        await conn.execute(
+            text(
+                """
+                CREATE INDEX IF NOT EXISTS idx_collection_tasks_source_status
+                ON collection_tasks (source, status)
+                """
+            )
+        )
+        await conn.execute(
+            text(
+                """
+                CREATE INDEX IF NOT EXISTS idx_collection_tasks_queue
+                ON collection_tasks (status, created_at, id)
+                WHERE status = 'queued'
+                """
+            )
+        )
+        await conn.execute(
+            text(
+                """
+                CREATE INDEX IF NOT EXISTS idx_collection_tasks_dedupe
+                ON collection_tasks (dedupe_key)
+                WHERE dedupe_key IS NOT NULL
+                """
+            )
+        )
+        await conn.execute(
+            text(
+                """
+                CREATE INDEX IF NOT EXISTS idx_collected_data_source_current_id
+                ON collected_data (source, is_current, id)
+                """
+            )
+        )
+        await conn.execute(
+            text(
+                """
+                CREATE INDEX IF NOT EXISTS idx_collected_data_source_task_id
+                ON collected_data (source, task_id, id)
+                """
+            )
+        )
+        await conn.execute(
+            text(
+                """
+                CREATE INDEX IF NOT EXISTS idx_vessel_current_bbox
+                ON vessel_current_state (lon, lat)
+                """
+            )
+        )
+        await conn.execute(
+            text(
+                """
+                CREATE INDEX IF NOT EXISTS idx_vessel_current_observed
+                ON vessel_current_state (observed_at DESC)
+                """
+            )
+        )
+        await conn.execute(
+            text(
+                """
+                CREATE INDEX IF NOT EXISTS idx_ais_raw_schema_observed_entity
+                ON ais_raw_observations (target_schema, observed_at, entity_key)
+                """
+            )
+        )
+        await conn.execute(
+            text(
+                """
+                CREATE INDEX IF NOT EXISTS idx_ais_raw_schema_observed_desc
+                ON ais_raw_observations (target_schema, observed_at DESC)
+                """
+            )
+        )
+        await conn.execute(
+            text(
+                """
+                CREATE INDEX IF NOT EXISTS idx_ais_raw_payload_lon_lat
+                ON ais_raw_observations (
+                    ((normalized_payload->>'lon')::double precision),
+                    ((normalized_payload->>'lat')::double precision)
+                )
+                WHERE target_schema = 'vessel_ais'
+                """
+            )
+        )
         await conn.execute(
             text(
                 """
@@ -151,5 +851,15 @@ async def init_db():
         )
 
     async with async_session_factory() as session:
+        from app.services.bgp_collector_locations import (
+            seed_default_bgp_collector_locations,
+        )
+        from app.services.compute_center_locations import (
+            seed_compute_center_locations_from_source_coords,
+        )
+
+        await seed_default_bgp_collector_locations(session)
+        await seed_compute_center_locations_from_source_coords(session)
         await seed_default_datasources(session)
+        await purge_legacy_earth_boundary_datasources(session)
         await ensure_default_admin_user(session)

backend/app/main.py

@@ -1,12 +1,17 @@
 from contextlib import asynccontextmanager
+from pathlib import Path
+from uuid import uuid4
 
 from fastapi import FastAPI
 from fastapi.middleware.cors import CORSMiddleware
+from fastapi.staticfiles import StaticFiles
 from starlette.middleware.base import BaseHTTPMiddleware
 
 from app.api.main import api_router
 from app.api.v1 import websocket
 from app.core.config import settings
+from app.core.logging import configure_logging
+from app.core.request_context import set_request_id
 from app.core.websocket.broadcaster import broadcaster
 from app.db.session import init_db
 from app.services.scheduler import (
@@ -15,6 +20,18 @@ from app.services.scheduler import (
     stop_scheduler,
     sync_scheduler_with_datasources,
 )
+from app.services.earth_news_worker import (
+    start_earth_news_target_worker,
+    stop_earth_news_target_worker,
+)
+from app.services.earth_db_change_listener import (
+    start_earth_db_change_listener,
+    stop_earth_db_change_listener,
+)
+from app.services.data_jobs import start_data_job_worker, stop_data_job_worker
+
+
+configure_logging()
 
 
 class WebSocketCORSMiddleware(BaseHTTPMiddleware):
@@ -28,6 +45,18 @@ class WebSocketCORSMiddleware(BaseHTTPMiddleware):
         return await call_next(request)
 
 
+class RequestContextMiddleware(BaseHTTPMiddleware):
+    async def dispatch(self, request, call_next):
+        request_id = request.headers.get("X-Request-ID") or uuid4().hex
+        set_request_id(request_id)
+        try:
+            response = await call_next(request)
+        finally:
+            set_request_id(None)
+        response.headers["X-Request-ID"] = request_id
+        return response
+
+
 @asynccontextmanager
 async def lifespan(app: FastAPI):
     await init_db()
@@ -35,7 +64,13 @@ async def lifespan(app: FastAPI):
     start_scheduler()
     await sync_scheduler_with_datasources()
     broadcaster.start()
+    start_data_job_worker()
+    start_earth_db_change_listener()
+    start_earth_news_target_worker()
     yield
+    await stop_earth_news_target_worker()
+    await stop_earth_db_change_listener()
+    await stop_data_job_worker()
     broadcaster.stop()
     stop_scheduler()
 
@@ -58,11 +93,20 @@ app.add_middleware(
     allow_headers=["*"],
 )
 
+app.add_middleware(RequestContextMiddleware)
 app.add_middleware(WebSocketCORSMiddleware)
 
 app.include_router(api_router, prefix="/api/v1")
 app.include_router(websocket.router)
 
+EARTH_BRAND_ASSET_DIR = Path(__file__).resolve().parents[2] / "data" / "earth-brand"
+EARTH_BRAND_ASSET_DIR.mkdir(parents=True, exist_ok=True)
+app.mount(
+    "/earth-brand-assets",
+    StaticFiles(directory=str(EARTH_BRAND_ASSET_DIR)),
+    name="earth-brand-assets",
+)
+
 
 @app.get("/health")
 async def health_check():

backend/app/models/__init__.py

@@ -6,11 +6,18 @@ from app.models.datasource import DataSource
 from app.models.datasource_config import DataSourceConfig
 from app.models.alert import Alert, AlertSeverity, AlertStatus
 from app.models.bgp_anomaly import BGPAnomaly
+from app.models.bgp_collector_location import BGPCollectorLocation
 from app.models.bgp_incident import BGPIncident
 from app.models.bgp_observation import BGPObservation
+from app.models.compute_center_location import ComputeCenterLocationRecord
 from app.models.system_setting import SystemSetting
 from app.models.playground_session import PlaygroundSession
 from app.models.playground_message import PlaygroundMessage
+from app.models.system_log import AuditLog, ObservabilityEvent, ObservabilityEventGroup, SystemLog
+from app.models.vessel import AISConflictRecord, AISRawObservation, AISSourceHealth, VesselPosition, VesselStatic
+from app.models.datasource_mapping import DataSourceMappingTemplate
+from app.models.earth_news import EarthNewsItem
+from app.models.earth_interactable import EarthInteractable
 
 __all__ = [
     "User",
@@ -24,6 +31,22 @@ __all__ = [
     "AlertSeverity",
     "AlertStatus",
     "BGPAnomaly",
+    "BGPCollectorLocation",
     "BGPIncident",
     "BGPObservation",
+    "ComputeCenterLocationRecord",
+    "SystemLog",
+    "AuditLog",
+    "ObservabilityEvent",
+    "ObservabilityEventGroup",
+    "PlaygroundSession",
+    "PlaygroundMessage",
+    "VesselPosition",
+    "VesselStatic",
+    "AISRawObservation",
+    "AISConflictRecord",
+    "AISSourceHealth",
+    "DataSourceMappingTemplate",
+    "EarthNewsItem",
+    "EarthInteractable",
 ]

backend/app/models/alert.py

@@ -1,26 +1,12 @@
 from datetime import datetime
-from enum import Enum
-from typing import Optional
 
-from sqlalchemy import Column, Integer, String, DateTime, Text, ForeignKey, Enum as SQLEnum
+from sqlalchemy import Column, Integer, String, DateTime, Text, Enum as SQLEnum
-from sqlalchemy.orm import relationship
 
+from app.core.enums import AlertSeverity, AlertStatus
 from app.core.time import to_iso8601_utc
 from app.db.session import Base
 
 
-class AlertSeverity(str, Enum):
-    CRITICAL = "critical"
-    WARNING = "warning"
-    INFO = "info"
-
-
-class AlertStatus(str, Enum):
-    ACTIVE = "active"
-    ACKNOWLEDGED = "acknowledged"
-    RESOLVED = "resolved"
-
-
 class Alert(Base):
     __tablename__ = "alerts"
 

backend/app/models/bgp_anomaly.py

@@ -4,6 +4,7 @@ from datetime import datetime
 
 from sqlalchemy import Column, DateTime, Float, ForeignKey, Index, Integer, JSON, String, Text
 
+from app.core.enums import BGPStatus
 from app.core.time import to_iso8601_utc
 from app.db.session import Base
 
@@ -17,7 +18,7 @@ class BGPAnomaly(Base):
     source = Column(String(100), nullable=False, index=True)
     anomaly_type = Column(String(50), nullable=False, index=True)
     severity = Column(String(20), nullable=False, index=True)
-    status = Column(String(20), nullable=False, default="active", index=True)
+    status = Column(String(20), nullable=False, default=BGPStatus.ACTIVE.value, index=True)
     entity_key = Column(String(255), nullable=False, index=True)
     prefix = Column(String(64), nullable=True, index=True)
     origin_asn = Column(Integer, nullable=True, index=True)

backend/app/models/bgp_collector_location.py

@@ -0,0 +1,52 @@
+"""Stored BGP route-collector locations."""
+
+from sqlalchemy import Boolean, Column, DateTime, Float, Integer, JSON, String, Text
+from sqlalchemy.sql import func
+
+from app.core.time import to_iso8601_utc
+from app.db.session import Base
+
+
+class BGPCollectorLocation(Base):
+    """Current known location for a BGP route collector."""
+
+    __tablename__ = "bgp_collector_locations"
+
+    id = Column(Integer, primary_key=True, autoincrement=True)
+    collector_id = Column(String(100), nullable=False, unique=True, index=True)
+    operator = Column(String(255), nullable=True)
+    site = Column(String(255), nullable=True)
+    city = Column(String(255), nullable=True)
+    country = Column(String(255), nullable=True)
+    latitude = Column(Float, nullable=True)
+    longitude = Column(Float, nullable=True)
+    precision = Column(String(30), nullable=False, default="city")
+    confidence = Column(Float, nullable=True)
+    source = Column(String(80), nullable=False, default="legacy_seed", index=True)
+    source_url = Column(String(500), nullable=True)
+    source_note = Column(Text, nullable=True)
+    raw_payload = Column(JSON, nullable=False, default=dict)
+    needs_confirmation = Column(Boolean, nullable=False, default=True, index=True)
+    verification_status = Column(String(30), nullable=False, default="unverified", index=True)
+    verified_at = Column(DateTime(timezone=True), nullable=True)
+    created_at = Column(DateTime(timezone=True), server_default=func.now())
+    updated_at = Column(DateTime(timezone=True), server_default=func.now(), onupdate=func.now())
+
+    def to_location_dict(self) -> dict:
+        return {
+            "city": self.city,
+            "country": self.country,
+            "latitude": self.latitude,
+            "longitude": self.longitude,
+            "precision": self.precision,
+            "source": self.source,
+            "needs_confirmation": self.needs_confirmation,
+            "matched_location_name": self.site or self.collector_id,
+            "verified_at": to_iso8601_utc(self.verified_at),
+            "confidence": self.confidence,
+            "operator": self.operator,
+            "site": self.site,
+            "verification_status": self.verification_status,
+            "source_note": self.source_note,
+            "source_url": self.source_url,
+        }

backend/app/models/bgp_incident.py

@@ -4,6 +4,7 @@ from datetime import datetime
 
 from sqlalchemy import Column, DateTime, Float, ForeignKey, Index, Integer, JSON, String, Text
 
+from app.core.enums import BGPStatus
 from app.core.time import to_iso8601_utc
 from app.db.session import Base
 
@@ -20,7 +21,7 @@ class BGPIncident(Base):
     title = Column(String(255), nullable=False)
     summary = Column(Text, nullable=False)
     severity = Column(String(20), nullable=False, index=True)
-    status = Column(String(20), nullable=False, default="active", index=True)
+    status = Column(String(20), nullable=False, default=BGPStatus.ACTIVE.value, index=True)
     confidence = Column(Float, nullable=False, default=0.5)
     started_at = Column(DateTime(timezone=True), nullable=False, default=datetime.utcnow, index=True)
     ended_at = Column(DateTime(timezone=True), nullable=True)

backend/app/models/collected_data.py

@@ -48,6 +48,8 @@ class CollectedData(Base):
     # Indexes for common queries
     __table_args__ = (
         Index("idx_collected_data_source_collected", "source", "collected_at"),
+        Index("idx_collected_data_source_current_id", "source", "is_current", "id"),
+        Index("idx_collected_data_source_task_id", "source", "task_id", "id"),
         Index("idx_collected_data_source_type", "source", "data_type"),
         Index("idx_collected_data_source_source_id", "source", "source_id"),
     )

backend/app/models/compute_center_location.py

@@ -0,0 +1,60 @@
+"""Stored compute-center locations."""
+
+from sqlalchemy import Boolean, Column, DateTime, Float, Integer, JSON, String, Text, UniqueConstraint
+from sqlalchemy.sql import func
+
+from app.core.time import to_iso8601_utc
+from app.db.session import Base
+
+
+class ComputeCenterLocationRecord(Base):
+    """Current known location for a compute-center record."""
+
+    __tablename__ = "compute_center_locations"
+    __table_args__ = (
+        UniqueConstraint("source", "source_id", name="uq_compute_center_location_source_id"),
+    )
+
+    id = Column(Integer, primary_key=True, autoincrement=True)
+    source = Column(String(100), nullable=False, index=True)
+    source_id = Column(String(255), nullable=False, index=True)
+    name = Column(String(500), nullable=True)
+    operator = Column(String(255), nullable=True)
+    site = Column(String(255), nullable=True)
+    city = Column(String(255), nullable=True)
+    country = Column(String(255), nullable=True)
+    latitude = Column(Float, nullable=True)
+    longitude = Column(Float, nullable=True)
+    precision = Column(String(30), nullable=False, default="city")
+    confidence = Column(Float, nullable=True)
+    location_source = Column(String(80), nullable=False, default="stored_compute_center_location", index=True)
+    source_url = Column(String(500), nullable=True)
+    source_note = Column(Text, nullable=True)
+    raw_payload = Column(JSON, nullable=False, default=dict)
+    needs_confirmation = Column(Boolean, nullable=False, default=False, index=True)
+    verification_status = Column(String(30), nullable=False, default="verified", index=True)
+    verified_at = Column(DateTime(timezone=True), nullable=True)
+    created_at = Column(DateTime(timezone=True), server_default=func.now())
+    updated_at = Column(DateTime(timezone=True), server_default=func.now(), onupdate=func.now())
+
+    def to_location_dict(self) -> dict:
+        return {
+            "source": self.source,
+            "source_id": self.source_id,
+            "name": self.name,
+            "operator": self.operator,
+            "site": self.site,
+            "city": self.city,
+            "country": self.country,
+            "latitude": self.latitude,
+            "longitude": self.longitude,
+            "precision": self.precision,
+            "confidence": self.confidence,
+            "location_source": self.location_source,
+            "source_url": self.source_url,
+            "source_note": self.source_note,
+            "raw_payload": self.raw_payload or {},
+            "needs_confirmation": self.needs_confirmation,
+            "verification_status": self.verification_status,
+            "verified_at": to_iso8601_utc(self.verified_at),
+        }

backend/app/models/data_snapshot.py

@@ -1,6 +1,7 @@
 from sqlalchemy import Boolean, Column, DateTime, ForeignKey, Integer, JSON, String
 from sqlalchemy.sql import func
 
+from app.core.enums import SnapshotStatus
 from app.db.session import Base
 
 
@@ -16,7 +17,7 @@ class DataSnapshot(Base):
     started_at = Column(DateTime(timezone=True), server_default=func.now())
     completed_at = Column(DateTime(timezone=True), nullable=True)
     record_count = Column(Integer, default=0)
-    status = Column(String(20), nullable=False, default="running")
+    status = Column(String(20), nullable=False, default=SnapshotStatus.RUNNING.value)
     is_current = Column(Boolean, default=True, index=True)
     parent_snapshot_id = Column(Integer, ForeignKey("data_snapshots.id"), nullable=True, index=True)
     summary = Column(JSON, default={})

backend/app/models/datasource_mapping.py

@@ -0,0 +1,33 @@
+"""Mapping templates for user-defined data source payloads."""
+
+from sqlalchemy import Boolean, Column, DateTime, ForeignKey, Integer, JSON, String
+from sqlalchemy.sql import func
+
+from app.core.enums import MappingValidationStatus
+from app.db.session import Base
+
+
+class DataSourceMappingTemplate(Base):
+    __tablename__ = "datasource_mapping_templates"
+
+    id = Column(Integer, primary_key=True, autoincrement=True)
+    datasource_config_id = Column(
+        Integer,
+        ForeignKey("datasource_configs.id"),
+        nullable=False,
+        index=True,
+    )
+    target_schema = Column(String(80), nullable=False, index=True)
+    mapping_json = Column(JSON, nullable=False, default={})
+    sample_payload_hash = Column(String(64), nullable=True)
+    validation_status = Column(String(30), nullable=False, default=MappingValidationStatus.DRAFT.value)
+    version = Column(Integer, nullable=False, default=1)
+    is_active = Column(Boolean, nullable=False, default=False, index=True)
+    created_at = Column(DateTime(timezone=True), server_default=func.now())
+    updated_at = Column(DateTime(timezone=True), server_default=func.now(), onupdate=func.now())
+
+    def __repr__(self):
+        return (
+            f"<DataSourceMappingTemplate {self.id}: "
+            f"{self.datasource_config_id}/{self.target_schema}/v{self.version}>"
+        )

backend/app/models/earth_interactable.py

@@ -0,0 +1,30 @@
+"""Persistent Earth interactable objects."""
+
+from sqlalchemy import Boolean, Column, DateTime, Float, Index, Integer, JSON, String, Text
+from sqlalchemy.sql import func
+
+from app.db.session import Base
+
+
+class EarthInteractable(Base):
+    __tablename__ = "earth_interactables"
+
+    id = Column(String(160), primary_key=True)
+    layer = Column(String(80), nullable=False, default="interactables", index=True)
+    kind = Column(String(80), nullable=False, default="default", index=True)
+    label = Column(String(255), nullable=False, default="")
+    description = Column(Text, nullable=False, default="")
+    latitude = Column(Float, nullable=False)
+    longitude = Column(Float, nullable=False)
+    altitude = Column(Float, nullable=True)
+    revision = Column(Integer, nullable=False, default=1)
+    properties = Column(JSON, nullable=False, default=dict)
+    is_deleted = Column(Boolean, nullable=False, default=False, index=True)
+    created_at = Column(DateTime(timezone=True), server_default=func.now(), nullable=False)
+    updated_at = Column(DateTime(timezone=True), server_default=func.now(), onupdate=func.now())
+    deleted_at = Column(DateTime(timezone=True), nullable=True, index=True)
+
+    __table_args__ = (
+        Index("idx_earth_interactables_layer_deleted", "layer", "is_deleted"),
+        Index("idx_earth_interactables_updated_at", "updated_at"),
+    )

backend/app/models/earth_news.py

@@ -0,0 +1,40 @@
+from sqlalchemy import Boolean, Column, DateTime, Float, Index, JSON, String, Text
+from sqlalchemy.sql import func
+
+from app.db.session import Base
+
+
+class EarthNewsItem(Base):
+    __tablename__ = "earth_news_items"
+
+    id = Column(String(160), primary_key=True)
+    title = Column(String(500), nullable=False)
+    summary = Column(Text, nullable=False, default="")
+    content_language = Column(String(32), nullable=False, default="en")
+    localizations = Column(JSON, nullable=False, default=dict)
+    url = Column(Text, nullable=False)
+    source = Column(String(255), nullable=False, default="")
+    feed_name = Column(String(255), nullable=False, default="")
+    region = Column(String(80), nullable=False, index=True)
+    homepage_url = Column(Text, nullable=False, default="")
+    published_at = Column(DateTime(timezone=True), nullable=True, index=True)
+
+    latitude = Column(Float, nullable=False)
+    longitude = Column(Float, nullable=False)
+    location_label = Column(String(255), nullable=False)
+    location_source = Column(String(80), nullable=False, default="region_anchor")
+    verified = Column(Boolean, nullable=False, default=False, index=True)
+    location_meta = Column(JSON, nullable=False, default=dict)
+
+    first_seen_at = Column(DateTime(timezone=True), server_default=func.now(), nullable=False)
+    last_seen_at = Column(DateTime(timezone=True), server_default=func.now(), nullable=False, index=True)
+    resolved_at = Column(DateTime(timezone=True), nullable=True, index=True)
+    enrichment_status = Column(String(80), nullable=False, default="pending", index=True)
+    enrichment_error = Column(Text, nullable=True)
+    enriched_at = Column(DateTime(timezone=True), nullable=True, index=True)
+    updated_at = Column(DateTime(timezone=True), server_default=func.now(), onupdate=func.now())
+
+    __table_args__ = (
+        Index("idx_earth_news_region_published", "region", "published_at"),
+        Index("idx_earth_news_region_seen", "region", "last_seen_at"),
+    )

backend/app/models/playground_message.py

@@ -1,6 +1,7 @@
 from sqlalchemy import JSON, Boolean, Column, DateTime, ForeignKey, Integer, String, Text
 from sqlalchemy.sql import func
 
+from app.core.enums import PlaygroundMessageKind, PlaygroundMessageStatus
 from app.db.session import Base
 
 
@@ -13,8 +14,8 @@ class PlaygroundMessage(Base):
     user_id = Column(Integer, ForeignKey("users.id", ondelete="CASCADE"), nullable=False, index=True)
     parent_message_id = Column(Integer, ForeignKey("playground_messages.id", ondelete="SET NULL"), nullable=True)
     role = Column(String(20), nullable=False)
-    kind = Column(String(20), nullable=False, default="message")
+    kind = Column(String(20), nullable=False, default=PlaygroundMessageKind.MESSAGE.value)
-    status = Column(String(20), nullable=False, default="done")
+    status = Column(String(20), nullable=False, default=PlaygroundMessageStatus.DONE.value)
     title = Column(String(255), nullable=True)
     content = Column(Text, nullable=False, default="")
     thinking_content = Column(Text, nullable=False, default="")

backend/app/models/system_log.py

@@ -0,0 +1,83 @@
+from sqlalchemy import JSON, Column, DateTime, Integer, String, Text
+from sqlalchemy.sql import func
+
+from app.db.session import Base
+
+
+class SystemLog(Base):
+    __tablename__ = "system_logs"
+
+    id = Column(Integer, primary_key=True, autoincrement=True)
+    occurred_at = Column(DateTime(timezone=True), server_default=func.now(), index=True)
+    source = Column(String(50), nullable=False, index=True)
+    service = Column(String(50), nullable=True)
+    module = Column(String(120), nullable=True)
+    event = Column(String(160), nullable=True, index=True)
+    level = Column(String(20), nullable=False, index=True)
+    message = Column(Text, nullable=False)
+    request_id = Column(String(64), nullable=True, index=True)
+    trace_id = Column(String(64), nullable=True)
+    user_id = Column(Integer, nullable=True, index=True)
+    category = Column(String(80), nullable=True, index=True)
+    context = Column(JSON, nullable=False, default=dict)
+    created_at = Column(DateTime(timezone=True), server_default=func.now())
+
+
+class AuditLog(Base):
+    __tablename__ = "audit_logs"
+
+    id = Column(Integer, primary_key=True, autoincrement=True)
+    occurred_at = Column(DateTime(timezone=True), server_default=func.now(), index=True)
+    actor_id = Column(Integer, nullable=True, index=True)
+    actor_name = Column(String(255), nullable=True)
+    action = Column(String(120), nullable=False, index=True)
+    target_type = Column(String(80), nullable=True)
+    target_id = Column(String(120), nullable=True)
+    result = Column(String(40), nullable=True, index=True)
+    request_id = Column(String(64), nullable=True, index=True)
+    ip = Column(String(64), nullable=True)
+    details = Column(JSON, nullable=False, default=dict)
+    created_at = Column(DateTime(timezone=True), server_default=func.now())
+
+
+class ObservabilityEvent(Base):
+    __tablename__ = "observability_events"
+
+    id = Column(Integer, primary_key=True, autoincrement=True)
+    occurred_at = Column(DateTime(timezone=True), server_default=func.now(), index=True)
+    source = Column(String(50), nullable=False, index=True)
+    service = Column(String(50), nullable=True, index=True)
+    module = Column(String(120), nullable=True, index=True)
+    category = Column(String(80), nullable=True, index=True)
+    event = Column(String(160), nullable=True, index=True)
+    level = Column(String(20), nullable=False, index=True)
+    message = Column(Text, nullable=False)
+    fingerprint = Column(String(80), nullable=False, index=True)
+    request_id = Column(String(64), nullable=True, index=True)
+    trace_id = Column(String(64), nullable=True, index=True)
+    task_id = Column(String(120), nullable=True, index=True)
+    source_ref_id = Column(String(120), nullable=True, index=True)
+    provider = Column(String(120), nullable=True, index=True)
+    user_id = Column(Integer, nullable=True, index=True)
+    context = Column(JSON, nullable=False, default=dict)
+    occurrence_count = Column(Integer, nullable=False, default=1)
+    created_at = Column(DateTime(timezone=True), server_default=func.now())
+
+
+class ObservabilityEventGroup(Base):
+    __tablename__ = "observability_event_groups"
+
+    fingerprint = Column(String(80), primary_key=True)
+    source = Column(String(50), nullable=False, index=True)
+    service = Column(String(50), nullable=True, index=True)
+    module = Column(String(120), nullable=True, index=True)
+    category = Column(String(80), nullable=True, index=True)
+    event = Column(String(160), nullable=True, index=True)
+    last_level = Column(String(20), nullable=False, index=True)
+    sample_message = Column(Text, nullable=False)
+    sample_detail = Column(Text, nullable=True)
+    affected_sources = Column(JSON, nullable=False, default=list)
+    count = Column(Integer, nullable=False, default=0)
+    first_seen_at = Column(DateTime(timezone=True), nullable=False, index=True)
+    last_seen_at = Column(DateTime(timezone=True), nullable=False, index=True)
+    updated_at = Column(DateTime(timezone=True), server_default=func.now(), onupdate=func.now())

backend/app/models/task.py

@@ -1,8 +1,9 @@
-"""Collection Task model"""
+"""Datasource job model."""
 
-from sqlalchemy import Column, DateTime, Integer, String, Text, Float
+from sqlalchemy import BigInteger, Column, DateTime, Float, Integer, JSON, String, Text
 from sqlalchemy.sql import func
 
+from app.core.enums import JobStatus, JobType, RollbackPolicy
 from app.db.session import Base
 
 
@@ -11,14 +12,28 @@ class CollectionTask(Base):
 
     id = Column(Integer, primary_key=True, autoincrement=True)
     datasource_id = Column(Integer, nullable=False, index=True)
-    status = Column(String(20), nullable=False)  # pending, running, success, failed, cancelled
+    source = Column(String(100), nullable=True, index=True)
-    phase = Column(String(30), default="queued")
+    task_type = Column(String(30), nullable=False, default=JobType.COLLECT.value, index=True)
+    status = Column(String(20), nullable=False)  # queued, running, cancelling, success, failed, cancelled
+    phase = Column(String(30), default=JobStatus.QUEUED.value)
+    phase_progress = Column(Float)
+    phase_message = Column(String(255))
+    phase_current = Column(BigInteger)
+    phase_total = Column(BigInteger)
+    phase_unit = Column(String(30))
     started_at = Column(DateTime(timezone=True))
     completed_at = Column(DateTime(timezone=True))
     records_processed = Column(Integer, default=0)
     total_records = Column(Integer, default=0)  # Total records to process
     progress = Column(Float, default=0.0)  # Progress percentage (0-100)
     error_message = Column(Text)
+    payload = Column(JSON, default=dict)
+    rollback_policy = Column(String(40), nullable=False, default=RollbackPolicy.KEEP_COMMITTED_BATCHES.value)
+    dedupe_key = Column(String(180), nullable=True, index=True)
+    worker_id = Column(String(120), nullable=True, index=True)
+    locked_at = Column(DateTime(timezone=True), nullable=True, index=True)
+    requested_cancel_at = Column(DateTime(timezone=True), nullable=True)
+    cancel_reason = Column(Text)
     created_at = Column(DateTime(timezone=True), server_default=func.now())
 
     def __repr__(self):

backend/app/models/user.py

@@ -1,6 +1,7 @@
-from sqlalchemy import Boolean, Column, Integer, String, DateTime
+from sqlalchemy import Boolean, Column, DateTime, Integer, JSON, String
 from sqlalchemy.sql import func
 
+from app.core.enums import UserRole
 from app.db.session import Base
 
 
@@ -11,8 +12,11 @@ class User(Base):
     username = Column(String(50), unique=True, index=True, nullable=False)
     email = Column(String(255), unique=True, index=True, nullable=False)
     password_hash = Column(String(255), nullable=False)
-    role = Column(String(20), default="viewer")
+    role = Column(String(20), default=UserRole.VIEWER.value)
+    gatekeeper_groups = Column(JSON, default=list)
     is_active = Column(Boolean, default=True)
+    email_verified = Column(Boolean, default=False, nullable=False)
+    pending_email = Column(String(255), nullable=True)
     last_login_at = Column(DateTime(timezone=True))
     created_at = Column(DateTime(timezone=True), server_default=func.now())
     updated_at = Column(

backend/app/models/vessel.py

@@ -0,0 +1,248 @@
+"""Vessel AIS models for live maritime tracking."""
+
+from sqlalchemy import BigInteger, Column, DateTime, Float, Index, Integer, JSON, SmallInteger, String
+from sqlalchemy.sql import func
+
+from app.core.enums import ConnectionState
+from app.core.time import to_iso8601_utc
+from app.db.session import Base
+
+
+class VesselStatic(Base):
+    """Slow-changing vessel identity and dimensions."""
+
+    __tablename__ = "vessel_static"
+
+    mmsi = Column(BigInteger, primary_key=True)
+    name = Column(String(128), nullable=True)
+    callsign = Column(String(16), nullable=True)
+    vessel_type = Column(SmallInteger, nullable=True, index=True)
+    vessel_type_name = Column(String(64), nullable=True, index=True)
+    flag = Column(String(4), nullable=True, index=True)
+    length = Column(Float, nullable=True)
+    width = Column(Float, nullable=True)
+    draught = Column(Float, nullable=True)
+    imo = Column(BigInteger, nullable=True)
+    updated_at = Column(DateTime(timezone=True), nullable=False, server_default=func.now())
+
+    def to_dict(self) -> dict:
+        return {
+            "mmsi": self.mmsi,
+            "name": self.name,
+            "callsign": self.callsign,
+            "vessel_type": self.vessel_type,
+            "vessel_type_name": self.vessel_type_name,
+            "flag": self.flag,
+            "length": self.length,
+            "width": self.width,
+            "draught": self.draught,
+            "imo": self.imo,
+            "updated_at": to_iso8601_utc(self.updated_at),
+        }
+
+
+class VesselPosition(Base):
+    """Append-only AIS positions retained for short history windows."""
+
+    __tablename__ = "vessel_position"
+
+    id = Column(Integer, primary_key=True, autoincrement=True)
+    mmsi = Column(BigInteger, nullable=False, index=True)
+    lat = Column(Float, nullable=False)
+    lon = Column(Float, nullable=False)
+    sog = Column(Float, nullable=True)
+    cog = Column(Float, nullable=True)
+    heading = Column(SmallInteger, nullable=True)
+    nav_status = Column(SmallInteger, nullable=True, index=True)
+    received_at = Column(DateTime(timezone=True), nullable=False, server_default=func.now(), index=True)
+
+    __table_args__ = (
+        Index("idx_vessel_pos_mmsi_time", "mmsi", "received_at"),
+        Index("idx_vessel_pos_time", "received_at"),
+        Index("idx_vessel_pos_lat_lon", "lat", "lon"),
+    )
+
+    def to_dict(self) -> dict:
+        return {
+            "id": self.id,
+            "mmsi": self.mmsi,
+            "lat": self.lat,
+            "lon": self.lon,
+            "sog": self.sog,
+            "cog": self.cog,
+            "heading": self.heading,
+            "nav_status": self.nav_status,
+            "received_at": to_iso8601_utc(self.received_at),
+        }
+
+
+class VesselCurrentState(Base):
+    """Latest renderable state for one vessel, independent from AIS history."""
+
+    __tablename__ = "vessel_current_state"
+
+    mmsi = Column(BigInteger, primary_key=True)
+    lat = Column(Float, nullable=False)
+    lon = Column(Float, nullable=False)
+    sog = Column(Float, nullable=True)
+    cog = Column(Float, nullable=True)
+    heading = Column(SmallInteger, nullable=True)
+    nav_status = Column(SmallInteger, nullable=True, index=True)
+    name = Column(String(128), nullable=True)
+    callsign = Column(String(16), nullable=True)
+    vessel_type = Column(SmallInteger, nullable=True, index=True)
+    vessel_type_name = Column(String(64), nullable=True, index=True)
+    flag = Column(String(4), nullable=True, index=True)
+    length = Column(Float, nullable=True)
+    width = Column(Float, nullable=True)
+    draught = Column(Float, nullable=True)
+    imo = Column(BigInteger, nullable=True)
+    source = Column(String(100), nullable=False, index=True)
+    observed_at = Column(DateTime(timezone=True), nullable=False)
+    field_sources = Column(JSON, default=dict)
+    selected_reasons = Column(JSON, default=dict)
+    source_summary = Column(JSON, default=dict)
+    quality_flags = Column(JSON, default=list)
+    updated_at = Column(DateTime(timezone=True), nullable=False, server_default=func.now())
+
+    __table_args__ = (
+        Index("idx_vessel_current_bbox", "lon", "lat"),
+        Index("idx_vessel_current_observed", "observed_at"),
+    )
+
+    def to_dict(self) -> dict:
+        return {
+            "mmsi": self.mmsi,
+            "lat": self.lat,
+            "lon": self.lon,
+            "sog": self.sog,
+            "cog": self.cog,
+            "heading": self.heading,
+            "nav_status": self.nav_status,
+            "name": self.name,
+            "callsign": self.callsign,
+            "vessel_type": self.vessel_type,
+            "vessel_type_name": self.vessel_type_name,
+            "flag": self.flag,
+            "length": self.length,
+            "width": self.width,
+            "draught": self.draught,
+            "imo": self.imo,
+            "source": self.source,
+            "received_at": self.observed_at,
+            "field_sources": self.field_sources or {},
+            "selected_reasons": self.selected_reasons or {},
+            "source_summary": self.source_summary or {},
+            "quality_flags": self.quality_flags or [],
+        }
+
+
+class AISRawObservation(Base):
+    """Source-level AIS fact before aggregation and conflict resolution."""
+
+    __tablename__ = "ais_raw_observations"
+
+    id = Column(Integer, primary_key=True, autoincrement=True)
+    target_schema = Column(String(64), nullable=False, default="vessel_ais", index=True)
+    source = Column(String(100), nullable=False, index=True)
+    entity_key = Column(String(64), nullable=False, index=True)
+    delivery_mode = Column(String(32), nullable=False, index=True)
+    transport = Column(String(32), nullable=False, index=True)
+    message_type = Column(String(64), nullable=True, index=True)
+    source_message_id = Column(String(128), nullable=True, index=True)
+    observation_hash = Column(String(64), nullable=False, unique=True, index=True)
+    observed_at = Column(DateTime(timezone=True), nullable=False, index=True)
+    collected_at = Column(DateTime(timezone=True), nullable=False, server_default=func.now(), index=True)
+    normalized_payload = Column(JSON, default=dict)
+    raw_payload = Column(JSON, default=dict)
+    quality_flags = Column(JSON, default=list)
+
+    __table_args__ = (
+        Index("idx_ais_raw_entity_observed", "target_schema", "entity_key", "observed_at"),
+        Index("idx_ais_raw_schema_observed_entity", "target_schema", "observed_at", "entity_key"),
+        Index("idx_ais_raw_source_entity", "source", "entity_key"),
+    )
+
+    def to_dict(self) -> dict:
+        return {
+            "id": self.id,
+            "target_schema": self.target_schema,
+            "source": self.source,
+            "entity_key": self.entity_key,
+            "delivery_mode": self.delivery_mode,
+            "transport": self.transport,
+            "message_type": self.message_type,
+            "source_message_id": self.source_message_id,
+            "observation_hash": self.observation_hash,
+            "observed_at": to_iso8601_utc(self.observed_at),
+            "collected_at": to_iso8601_utc(self.collected_at),
+            "normalized_payload": self.normalized_payload or {},
+            "raw_payload": self.raw_payload or {},
+            "quality_flags": self.quality_flags or [],
+        }
+
+
+class AISConflictRecord(Base):
+    """Recorded field-level disagreement between AIS sources."""
+
+    __tablename__ = "ais_conflict_records"
+
+    id = Column(Integer, primary_key=True, autoincrement=True)
+    target_schema = Column(String(64), nullable=False, default="vessel_ais", index=True)
+    entity_key = Column(String(64), nullable=False, index=True)
+    field = Column(String(64), nullable=False, index=True)
+    candidates = Column(JSON, default=dict)
+    selected_source = Column(String(100), nullable=True, index=True)
+    selected_value = Column(JSON, nullable=True)
+    selected_reason = Column(String(64), nullable=True, index=True)
+    resolved_by = Column(String(32), nullable=False, default="system", index=True)
+    status = Column(String(32), nullable=False, default="open", index=True)
+    created_at = Column(DateTime(timezone=True), nullable=False, server_default=func.now(), index=True)
+    updated_at = Column(DateTime(timezone=True), nullable=False, server_default=func.now())
+
+    __table_args__ = (
+        Index("idx_ais_conflict_entity_field", "target_schema", "entity_key", "field"),
+    )
+
+    def to_dict(self) -> dict:
+        return {
+            "id": self.id,
+            "target_schema": self.target_schema,
+            "entity_key": self.entity_key,
+            "field": self.field,
+            "candidates": self.candidates or {},
+            "selected_source": self.selected_source,
+            "selected_value": self.selected_value,
+            "selected_reason": self.selected_reason,
+            "resolved_by": self.resolved_by,
+            "status": self.status,
+            "created_at": to_iso8601_utc(self.created_at),
+            "updated_at": to_iso8601_utc(self.updated_at),
+        }
+
+
+class AISSourceHealth(Base):
+    """Runtime health signal for an AIS collector source."""
+
+    __tablename__ = "ais_source_health"
+
+    source = Column(String(100), primary_key=True)
+    connection_state = Column(String(32), nullable=False, default=ConnectionState.DISCONNECTED.value, index=True)
+    last_seen_at = Column(DateTime(timezone=True), nullable=True, index=True)
+    last_success_at = Column(DateTime(timezone=True), nullable=True, index=True)
+    last_error = Column(String(500), nullable=True)
+    message_rate = Column(Float, nullable=True)
+    lag_seconds = Column(Float, nullable=True)
+    updated_at = Column(DateTime(timezone=True), nullable=False, server_default=func.now(), index=True)
+
+    def to_dict(self) -> dict:
+        return {
+            "source": self.source,
+            "connection_state": self.connection_state,
+            "last_seen_at": to_iso8601_utc(self.last_seen_at),
+            "last_success_at": to_iso8601_utc(self.last_success_at),
+            "last_error": self.last_error,
+            "message_rate": self.message_rate,
+            "lag_seconds": self.lag_seconds,
+            "updated_at": to_iso8601_utc(self.updated_at),
+        }

backend/app/models/vessel_enrichment.py

@@ -0,0 +1,63 @@
+"""Vessel enrichment cache tables (v5).
+
+Profile and media enrichment are stored separately so cache TTLs can differ
+and so the conflict-resolution + display layers can read either independently.
+"""
+
+from sqlalchemy import BigInteger, Column, DateTime, Float, JSON, String
+from sqlalchemy.sql import func
+
+from app.core.time import to_iso8601_utc
+from app.db.session import Base
+
+
+class VesselProfileEnrichment(Base):
+    """Cached static vessel profile (type, flag, dimensions, operator, etc.)."""
+
+    __tablename__ = "vessel_profile_enrichment"
+
+    mmsi = Column(BigInteger, primary_key=True)
+    source = Column(String(100), nullable=False, default="system")
+    payload = Column(JSON, nullable=False, default=dict)
+    fetched_at = Column(DateTime(timezone=True), nullable=False, server_default=func.now())
+    expires_at = Column(DateTime(timezone=True), nullable=True)
+    confidence = Column(Float, nullable=True)
+    reference_url = Column(String(500), nullable=True)
+    updated_at = Column(DateTime(timezone=True), nullable=False, server_default=func.now(), onupdate=func.now())
+
+    def to_dict(self) -> dict:
+        return {
+            "mmsi": self.mmsi,
+            "source": self.source,
+            "payload": self.payload or {},
+            "fetched_at": to_iso8601_utc(self.fetched_at),
+            "expires_at": to_iso8601_utc(self.expires_at),
+            "confidence": self.confidence,
+            "reference_url": self.reference_url,
+        }
+
+
+class VesselMediaEnrichment(Base):
+    """Cached vessel imagery / external detail references."""
+
+    __tablename__ = "vessel_media_enrichment"
+
+    mmsi = Column(BigInteger, primary_key=True)
+    source = Column(String(100), nullable=False, default="system")
+    payload = Column(JSON, nullable=False, default=dict)
+    fetched_at = Column(DateTime(timezone=True), nullable=False, server_default=func.now())
+    expires_at = Column(DateTime(timezone=True), nullable=True)
+    confidence = Column(Float, nullable=True)
+    reference_url = Column(String(500), nullable=True)
+    updated_at = Column(DateTime(timezone=True), nullable=False, server_default=func.now(), onupdate=func.now())
+
+    def to_dict(self) -> dict:
+        return {
+            "mmsi": self.mmsi,
+            "source": self.source,
+            "payload": self.payload or {},
+            "fetched_at": to_iso8601_utc(self.fetched_at),
+            "expires_at": to_iso8601_utc(self.expires_at),
+            "confidence": self.confidence,
+            "reference_url": self.reference_url,
+        }

backend/app/schemas/ai.py

@@ -2,6 +2,7 @@ from typing import Any
 
 from pydantic import BaseModel, Field
 
+from app.core.enums import PlaygroundMessageKind, PlaygroundMessageRole, PlaygroundMessageStatus
 
 class AIContentBlock(BaseModel):
     type: str
@@ -13,10 +14,11 @@ class AIContentBlock(BaseModel):
 
 class SituationalAnalysisRequest(BaseModel):
     title: str = Field(..., min_length=1, max_length=200)
-    objective: str = Field(..., min_length=1, max_length=1000)
+    objective: str = Field(..., min_length=1, max_length=20000)
     context: dict[str, Any] = Field(default_factory=dict)
     observations: list[str] = Field(default_factory=list)
     constraints: list[str] = Field(default_factory=list)
+    system_prompt: str | None = Field(default=None, max_length=8000)
     preferred_model: str | None = Field(default=None, max_length=200)
     thinking: dict[str, Any] | None = None
 
@@ -109,9 +111,9 @@ class PlaygroundSessionUpsertRequest(BaseModel):
 
 class PlaygroundMessageRecord(BaseModel):
     id: str
-    role: str
+    role: PlaygroundMessageRole
-    kind: str = "message"
+    kind: PlaygroundMessageKind = PlaygroundMessageKind.MESSAGE
-    status: str = "done"
+    status: PlaygroundMessageStatus = PlaygroundMessageStatus.DONE
     title: str | None = None
     content: str = ""
     thinking_content: str = ""

backend/app/schemas/user.py

@@ -3,6 +3,7 @@ from typing import Optional
 
 from pydantic import BaseModel, EmailStr, Field
 
+from app.core.enums import OtpPurpose, UserRole
 
 class UserBase(BaseModel):
     username: str
@@ -11,18 +12,21 @@ class UserBase(BaseModel):
 
 class UserCreate(UserBase):
     password: str = Field(..., min_length=8)
-    role: str = "viewer"
+    role: UserRole = UserRole.VIEWER
+    gatekeeper_groups: list[str] = Field(default_factory=list)
 
 
 class UserUpdate(BaseModel):
     email: Optional[EmailStr] = None
-    role: Optional[str] = None
+    role: Optional[UserRole] = None
+    gatekeeper_groups: Optional[list[str]] = None
     is_active: Optional[bool] = None
 
 
 class UserInDB(UserBase):
     id: int
     role: str
+    gatekeeper_groups: list[str] = Field(default_factory=list)
     is_active: bool
     last_login_at: Optional[datetime]
     created_at: datetime
@@ -34,8 +38,36 @@ class UserInDB(UserBase):
 class UserResponse(UserBase):
     id: int
     role: str
+    gatekeeper_groups: list[str] = Field(default_factory=list)
     is_active: bool
+    email_verified: bool = False
     created_at: datetime
 
     class Config:
         from_attributes = True
+
+
+class UserRegister(BaseModel):
+    username: str = Field(..., min_length=3, max_length=50)
+    email: EmailStr
+    password: str = Field(..., min_length=8, max_length=128)
+
+
+class VerifyEmailRequest(BaseModel):
+    email: EmailStr
+    code: str = Field(..., min_length=6, max_length=6)
+
+
+class ResendCodeRequest(BaseModel):
+    email: EmailStr
+    purpose: OtpPurpose = OtpPurpose.REGISTER
+
+
+class ForgotPasswordRequest(BaseModel):
+    email: EmailStr
+
+
+class ResetPasswordRequest(BaseModel):
+    email: EmailStr
+    code: str = Field(..., min_length=6, max_length=6)
+    new_password: str = Field(..., min_length=8, max_length=128)

backend/app/services/ai_client.py

@@ -1,24 +1,49 @@
 from __future__ import annotations
 
 import asyncio
+import json
+from time import perf_counter
 
 import httpx
-from fastapi import HTTPException, status
+from fastapi import Depends, HTTPException, status
+from sqlalchemy.ext.asyncio import AsyncSession
 
 from app.core.config import settings
+from app.core.logging import get_logger
+from app.db.session import get_db
 from app.schemas.ai import (
     AIProviderStatusResponse,
     SituationalAnalysisRequest,
     SituationalAnalysisResponse,
 )
+from app.services.business_logs import emit_business_log, exception_context
+
+
+logger = get_logger(__name__, service="ai")
 
 
 class AIProviderClient:
-    def __init__(self) -> None:
+    def __init__(
-        self.service_url = settings.AI_PROVIDER_SERVICE_URL.rstrip("/")
+        self,
-        self.service_token = settings.AI_PROVIDER_SERVICE_TOKEN
+        *,
-        self.timeout = settings.AI_PROVIDER_TIMEOUT_SECONDS
+        service_url: str | None = None,
-        self.retry_attempts = max(settings.AI_PROVIDER_RETRY_ATTEMPTS, 1)
+        service_token: str | None = None,
+        timeout: int | None = None,
+        retry_attempts: int | None = None,
+        llm_config: dict | None = None,
+    ) -> None:
+        self.service_url = (
+            service_url if service_url is not None else settings.AI_PROVIDER_SERVICE_URL
+        ).rstrip("/")
+        self.service_token = (
+            service_token if service_token is not None else settings.AI_PROVIDER_SERVICE_TOKEN
+        )
+        self.timeout = timeout if timeout is not None else settings.AI_PROVIDER_TIMEOUT_SECONDS
+        self.retry_attempts = max(
+            retry_attempts if retry_attempts is not None else settings.AI_PROVIDER_RETRY_ATTEMPTS,
+            1,
+        )
+        self.llm_config = llm_config or {}
 
     def _headers(self, request_id: str | None = None) -> dict[str, str]:
         headers = {"Content-Type": "application/json"}
@@ -26,10 +51,38 @@ class AIProviderClient:
             headers["X-Provider-Token"] = self.service_token
         if request_id:
             headers["X-Request-ID"] = request_id
+        llm_header_map = {
+            "provider": "X-AI-Provider",
+            "provider_api": "X-AI-Provider-API",
+            "base_url": "X-AI-Base-URL",
+            "api_key": "X-AI-API-Key",
+            "model": "X-AI-Model",
+            "max_tokens": "X-AI-Max-Tokens",
+            "anthropic_version": "X-AI-Anthropic-Version",
+        }
+        for key, header_name in llm_header_map.items():
+            value = self.llm_config.get(key)
+            if value not in (None, ""):
+                headers[header_name] = str(value)
+        model_provider_apis = self.llm_config.get("model_provider_apis")
+        if isinstance(model_provider_apis, dict) and model_provider_apis:
+            headers["X-AI-Model-Provider-APIs"] = json.dumps(model_provider_apis)
         return headers
 
     async def get_status(self, request_id: str | None = None) -> AIProviderStatusResponse:
+        context = self._base_log_context(operation="status")
         if not self.service_url:
+            await emit_business_log(
+                logger,
+                event="ai.provider.status.failed",
+                message="AI provider status skipped because service URL is not configured",
+                category="ai",
+                level="warning",
+                service="ai",
+                module=__name__,
+                request_id=request_id,
+                context={**context, "status": "unconfigured"},
+            )
             return AIProviderStatusResponse(
                 provider="unconfigured",
                 enabled=False,
@@ -38,27 +91,133 @@ class AIProviderClient:
                 base_url=None,
             )
 
-        data = await self._request("GET", "/v1/provider/status", request_id=request_id)
+        started_at = perf_counter()
-        return AIProviderStatusResponse.model_validate(data)
+        await emit_business_log(
+            logger,
+            event="ai.provider.status.start",
+            message="AI provider status request started",
+            category="ai",
+            service="ai",
+            module=__name__,
+            request_id=request_id,
+            context=context,
+        )
+        try:
+            data = await self._request("GET", "/v1/provider/status", request_id=request_id, operation="status")
+            result = AIProviderStatusResponse.model_validate(data)
+            await emit_business_log(
+                logger,
+                event="ai.provider.status.success",
+                message="AI provider status request completed",
+                category="ai",
+                service="ai",
+                module=__name__,
+                request_id=request_id,
+                context={
+                    **context,
+                    "status": "success",
+                    "duration_ms": self._duration_ms(started_at),
+                    "result_provider": result.provider,
+                    "result_model": result.model,
+                    "configured": result.configured,
+                    "enabled": result.enabled,
+                },
+            )
+            return result
+        except Exception as exc:
+            await emit_business_log(
+                logger,
+                event="ai.provider.status.failed",
+                message="AI provider status request failed",
+                category="ai",
+                level="error",
+                service="ai",
+                module=__name__,
+                request_id=request_id,
+                context=exception_context(exc, {**context, "status": "failed", "duration_ms": self._duration_ms(started_at)}),
+            )
+            raise
 
     async def analyze(
         self,
         payload: SituationalAnalysisRequest,
         request_id: str | None = None,
     ) -> SituationalAnalysisResponse:
+        context = self._base_log_context(
+            operation="analyze",
+            preferred_model=payload.preferred_model,
+            input_summary=self._summarize_analysis_payload(payload),
+        )
         if not self.service_url:
+            await emit_business_log(
+                logger,
+                event="ai.provider.analyze.failed",
+                message="AI provider analyze skipped because service URL is not configured",
+                category="ai",
+                level="warning",
+                service="ai",
+                module=__name__,
+                request_id=request_id,
+                context={**context, "status": "unconfigured"},
+            )
             raise HTTPException(
                 status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
                 detail="AI provider service URL is not configured.",
             )
 
-        data = await self._request(
+        started_at = perf_counter()
-            "POST",
+        await emit_business_log(
-            "/v1/analyze",
+            logger,
-            json=payload.model_dump(),
+            event="ai.provider.analyze.start",
+            message="AI provider analyze request started",
+            category="ai",
+            service="ai",
+            module=__name__,
             request_id=request_id,
+            context=context,
         )
-        return SituationalAnalysisResponse.model_validate(data)
+        try:
+            data = await self._request(
+                "POST",
+                "/v1/analyze",
+                json=payload.model_dump(),
+                request_id=request_id,
+                operation="analyze",
+                payload_summary=context["input_summary"],
+            )
+            result = SituationalAnalysisResponse.model_validate(data)
+            await emit_business_log(
+                logger,
+                event="ai.provider.analyze.success",
+                message="AI provider analyze request completed",
+                category="ai",
+                service="ai",
+                module=__name__,
+                request_id=request_id,
+                context={
+                    **context,
+                    "status": "success",
+                    "duration_ms": self._duration_ms(started_at),
+                    "result_provider": result.provider,
+                    "result_model": result.model,
+                    "content_block_count": len(result.content_blocks or []),
+                    "thinking_block_count": len(result.thinking_blocks or []),
+                },
+            )
+            return result
+        except Exception as exc:
+            await emit_business_log(
+                logger,
+                event="ai.provider.analyze.failed",
+                message="AI provider analyze request failed",
+                category="ai",
+                level="error",
+                service="ai",
+                module=__name__,
+                request_id=request_id,
+                context=exception_context(exc, {**context, "status": "failed", "duration_ms": self._duration_ms(started_at)}),
+            )
+            raise
 
     async def _request(
         self,
@@ -66,9 +225,12 @@ class AIProviderClient:
         path: str,
         json: dict | None = None,
         request_id: str | None = None,
+        operation: str = "request",
+        payload_summary: dict | None = None,
     ) -> dict:
         last_error: Exception | None = None
         for attempt in range(1, self.retry_attempts + 1):
+            attempt_started_at = perf_counter()
             try:
                 async with httpx.AsyncClient(timeout=self.timeout) as client:
                     response = await client.request(
@@ -82,6 +244,15 @@ class AIProviderClient:
             except httpx.HTTPStatusError as exc:
                 last_error = exc
                 if attempt < self.retry_attempts and exc.response.status_code >= 500:
+                    await self._log_retry(
+                        operation=operation,
+                        request_id=request_id,
+                        attempt=attempt,
+                        status_code=exc.response.status_code,
+                        duration_ms=self._duration_ms(attempt_started_at),
+                        error=exc,
+                        payload_summary=payload_summary,
+                    )
                     await asyncio.sleep(0.3 * attempt)
                     continue
                 detail = exc.response.text or "AI provider service returned an error"
@@ -92,6 +263,14 @@ class AIProviderClient:
             except httpx.HTTPError as exc:
                 last_error = exc
                 if attempt < self.retry_attempts:
+                    await self._log_retry(
+                        operation=operation,
+                        request_id=request_id,
+                        attempt=attempt,
+                        duration_ms=self._duration_ms(attempt_started_at),
+                        error=exc,
+                        payload_summary=payload_summary,
+                    )
                     await asyncio.sleep(0.3 * attempt)
                     continue
                 raise HTTPException(
@@ -104,6 +283,80 @@ class AIProviderClient:
             detail=f"AI provider service request failed: {last_error}",
         )
 
+    def _base_log_context(self, **extra: object) -> dict:
+        llm_provider_apis = self.llm_config.get("model_provider_apis")
+        return {
+            "provider": self.llm_config.get("provider") or "",
+            "provider_api": self.llm_config.get("provider_api") or "",
+            "model": self.llm_config.get("model") or "",
+            "base_url_configured": bool(self.llm_config.get("base_url")),
+            "service_url_configured": bool(self.service_url),
+            "timeout_seconds": self.timeout,
+            "retry_attempts": self.retry_attempts,
+            "model_provider_api_count": len(llm_provider_apis or {}) if isinstance(llm_provider_apis, dict) else 0,
+            **extra,
+        }
 
-def get_ai_provider_client() -> AIProviderClient:
+    @staticmethod
-    return AIProviderClient()
+    def _duration_ms(started_at: float) -> int:
+        return int((perf_counter() - started_at) * 1000)
+
+    @staticmethod
+    def _summarize_analysis_payload(payload: SituationalAnalysisRequest) -> dict:
+        context = payload.context if isinstance(payload.context, dict) else {}
+        thinking = payload.thinking if isinstance(payload.thinking, dict) else payload.thinking
+        return {
+            "title_length": len(payload.title or ""),
+            "objective_length": len(payload.objective or ""),
+            "observation_count": len(payload.observations or []),
+            "constraint_count": len(payload.constraints or []),
+            "has_system_prompt": bool(payload.system_prompt),
+            "thinking_enabled": bool(thinking),
+            "context_keys": sorted(str(key) for key in context.keys()),
+        }
+
+    async def _log_retry(
+        self,
+        *,
+        operation: str,
+        request_id: str | None,
+        attempt: int,
+        duration_ms: int,
+        error: BaseException,
+        status_code: int | None = None,
+        payload_summary: dict | None = None,
+    ) -> None:
+        await emit_business_log(
+            logger,
+            event=f"ai.provider.{operation}.retry",
+            message="AI provider request will retry",
+            category="ai",
+            level="warning",
+            service="ai",
+            module=__name__,
+            request_id=request_id,
+            context=exception_context(
+                error,
+                {
+                    **self._base_log_context(operation=operation),
+                    "attempt": attempt,
+                    "next_attempt": attempt + 1,
+                    "status_code": status_code,
+                    "duration_ms": duration_ms,
+                    "input_summary": payload_summary,
+                },
+            ),
+        )
+
+
+async def get_ai_provider_client(db: AsyncSession = Depends(get_db)) -> AIProviderClient:
+    from app.api.v1.settings import get_runtime_ai_provider_config
+
+    runtime_config = await get_runtime_ai_provider_config(db)
+    return AIProviderClient(
+        service_url=runtime_config["service_url"],
+        service_token=runtime_config["service_token"],
+        timeout=runtime_config["timeout_seconds"],
+        retry_attempts=runtime_config["retry_attempts"],
+        llm_config=runtime_config.get("llm_config") or {},
+    )

backend/app/services/ai_tools/__init__.py

@@ -0,0 +1,7 @@
+"""Backend-owned AI tool services.
+
+The services in this package are business tools used by Planet's backend
+orchestrators. They intentionally live outside ``aiprovider`` so model transport
+stays separate from evidence collection and domain policy.
+"""
+

backend/app/services/ai_tools/evidence_store.py

@@ -0,0 +1,48 @@
+from __future__ import annotations
+
+import hashlib
+from typing import Iterable
+
+from app.services.ai_tools.schemas import FetchedEvidence, SearchEvidence
+
+
+def evidence_content_hash(text: str) -> str:
+    return hashlib.sha256(text.encode("utf-8")).hexdigest()
+
+
+def normalize_search_evidence(items: Iterable[SearchEvidence], *, limit: int = 5) -> list[dict]:
+    normalized: list[dict] = []
+    seen_urls: set[str] = set()
+    for item in items:
+        if not item.url or item.url in seen_urls:
+            continue
+        seen_urls.add(item.url)
+        normalized.append(
+            {
+                "title": item.title,
+                "url": item.url,
+                "snippet": item.snippet,
+                "content": item.compact_text(),
+                "score": item.score,
+                "source_provider": item.source_provider,
+                "retrieved_at": item.retrieved_at.isoformat(),
+                "metadata": item.metadata,
+            }
+        )
+        if len(normalized) >= limit:
+            break
+    return normalized
+
+
+def normalize_fetched_evidence(item: FetchedEvidence, *, text_limit: int = 1200) -> dict:
+    text = " ".join(item.text.split())[:text_limit]
+    return {
+        "title": item.title,
+        "url": item.final_url or item.url,
+        "text": text,
+        "content_hash": item.content_hash or evidence_content_hash(item.text),
+        "extractor": item.extractor,
+        "retrieved_at": item.retrieved_at.isoformat(),
+        "metadata": item.metadata,
+    }
+