release: bump version to 0.44.0

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

.claude/commands/cleanup.md

@@ -12,6 +12,19 @@ allowed-tools: ["Read", "Edit", "Bash", "Grep", "Glob"]
 
 若 `$ARGUMENTS` 非空，则只检查指定文件/目录；否则检查所有未提交修改（`git diff HEAD`）。
 
+## 节省上下文规则
+
+优先用确定性的 CLI 检查缩小范围，不要一上来把完整文件或大 diff 读入上下文：
+
+```bash
+git diff --name-only HEAD
+git diff --unified=0 HEAD -- <path>
+git diff --check
+rg -n "TODO|FIXME|console\.log|debugger|print\(" <changed-paths>
+```
+
+只有 focused diff 不足以安全判断或修改时，才读取完整文件。
+
 ## 审查清单
 
 按优先级检查以下问题（只报告在本次 diff 中**新增或修改**的代码里存在的问题）：
@@ -59,8 +72,13 @@ git diff HEAD --name-only
 
 ### Step 2 — 逐文件阅读并分析
 
-- 用 Read 工具读取完整文件（不只读 diff）
-- 对照审查清单，记录每个问题：文件名、行号、问题类型、建议修复方式
+先从 focused diff 开始：
+
+```bash
+git diff --unified=0 HEAD -- <file>
+```
+
+用 `rg`、`git diff --check`、编译器或 linter 输出确认确定性问题。只有需要上下文时才用 Read 读取完整文件。对照审查清单，记录每个问题：文件名、行号、问题类型、建议修复方式。
 
 ### Step 3 — 报告问题清单
 
@@ -95,6 +113,7 @@ git diff HEAD --name-only
 - 只改在审查清单中发现的问题，不做额外优化
 - 每次 Edit 只修改确实有问题的行，保持 diff 最小
 - 改完后用 `grep` 验证旧的坏代码已消失
+- 优先做精确补丁；只有仓库已有对应格式化流程时，才运行格式化工具
 
 ### Step 5 — 输出总结
 

.claude/commands/docs.md

@@ -0,0 +1,170 @@
+---
+description: 分析本次 git 变更，在 docs/technical/zh/ 中新建或更新对应的技术文档
+argument-hint: 可选：指定要记录的主题，或留空自动从 git diff 推断
+allowed-tools: ["Read", "Edit", "Write", "Bash", "Glob", "Grep"]
+---
+
+# /docs — 技术文档写入工作流
+
+## 目标
+
+根据当前 git 变更（或用户指定主题）在 `docs/technical/zh/` 中写入或更新技术文档，记录**为什么**这样做，而不只是记录做了什么。
+
+## 执行步骤
+
+### Step 1 — 理解变更范围
+
+```bash
+git diff HEAD --stat          # 变更文件一览
+git diff HEAD --name-only     # 变更文件列表
+git log --oneline -10         # 近期 commit 上下文
+```
+
+若 `$ARGUMENTS` 指定了主题，优先聚焦该主题；否则从文件列表和 diff stat 推断变更主题。不要默认读取完整仓库 diff；只对决定文档主题所需的文件读取 focused diff：
+
+```bash
+git diff HEAD -- <path>
+rg -n "class |def |function |export |router|@router|interface |type " <path>
+```
+
+### Step 2 — 确认文档范围
+
+分析变更，判断：
+
+1. **应写几篇文档**：单一主题写一篇，跨领域变更可拆分（如后端性能优化 + 运维启动脚本分开写）
+2. **是新建还是更新**：检查 `docs/technical/zh/` 中是否已有相关文档
+3. **文档命名**：按 `领域-主题-副题.md` 格式，全小写，用连字符，如：
+   - `backend-datasources-api-performance.md`
+   - `ops-planet-sh-startup.md`
+   - `earth-bgp-context.md`
+
+```bash
+ls docs/technical/zh/          # 查看现有文档
+```
+
+**先输出写作计划供用户确认**（若变更明确且范围小，可直接执行）：
+
+```
+文档计划：
+  新建：docs/technical/zh/ops-planet-sh-startup.md — planet.sh 启动性能优化
+  更新：docs/technical/zh/backend-datasources-api-performance.md — 补充并行化细节
+```
+
+### Step 2.5 — 覆盖范围检查
+
+写文档前必须按变更类型检查配套文档，不要只更新一篇专题文档：
+
+- 用户可见流程变化：更新 `docs/technical/zh/manual.md`，通常也更新 `docs/technical/zh/quickstart.md`。
+- `manual.md`、`quickstart.md` 这类用户手册存在英文版时，同步更新 `docs/technical/en/...`，至少避免英文版与中文版互相矛盾。
+- 控制台页面职责、路由入口、表格/抽屉/设置页行为变化：更新 `docs/technical/zh/frontend-admin-frontend-context.md`。
+- Earth 前端行为、HUD、巡航、图层、图例、交互变化：更新 `docs/technical/zh/earth-frontend-context.md`。
+- 新增 Earth 图层、调整 `renderOrder`、半径/高度偏移、深度策略、拾取策略、legend mode、图层面板顺序或启动加载顺序：更新 `docs/technical/zh/earth-render-layer-order.md`。
+- Earth 图层视觉样式、颜色、图例符号语义变化：若影响样式索引，同步更新 `docs/technical/zh/earth-layer-style-reference.md`。
+- 采集器、数据源、凭证、设置页、连接检查、scheduler、后端 API 变化：更新相关后端文档，优先检查 `docs/technical/zh/backend-collectors.md` 和 datasource/settings 专题文档。
+- 如果某个旧 plan 的假设已经被当前实现推翻，在对应 `docs/plans/*.md` 增加现状修正或更新该段，不要让计划文档继续给出相反方向。
+- 新增 technical 文档后，如果需要被发现，更新 `docs/technical/zh/README.md`。
+- 对本次变更提取旧词做 stale search，例如旧 tab 名、旧路由职责、旧认证假设、改名前 UI 文案：
+
+```bash
+rg -n "旧文案|旧路由职责|旧认证假设" docs/technical docs/plans
+```
+
+### Step 3 — 写文档
+
+遵循以下原则：
+
+**记录 WHY，不只记录 WHAT**
+- 好：`将戳文件从 /tmp 移到 ~/.cache/planet/，因为 WSL 重启后 /tmp 被清空`
+- 差：`修改了 AI_PROVIDER_BUILD_STAMP_FILE 的值`
+
+**必须包含的内容**：
+- 背景/问题：改动之前存在什么问题，为什么要改
+- 核心设计决策及其理由
+- 关键代码片段（用 diff 或 before/after 展示）
+- 相关文件列表
+
+**格式要求**：
+- 使用 `##` 和 `###` 分级，不要超过三级
+- 代码块注明语言（python / bash / typescript / sql）
+- 表格用于对比多个选项或列出参数
+- 中文写作，技术术语保留英文原文
+- `docs/technical/zh/` 中的文档不得用英文原文占位；如果存在 `docs/technical/en/` 对应文件，禁止逐字复制成中文文件
+- 中文文档内部链接应指向 `docs/technical/zh/...`，除非明确引用英文专属文档
+
+**文档结构模板**：
+
+```markdown
+# 标题（说明做了什么）
+
+## 背景
+
+为什么要做这个改动，改动前存在什么问题。
+
+## 核心变更
+
+### 子主题一
+
+before/after 或决策说明 + 关键代码
+
+### 子主题二
+
+...
+
+## 相关文件
+
+- `path/to/file.py` — 简短说明
+```
+
+### Step 4 — 验证
+
+- 读一遍写好的文档，确认逻辑清晰、代码片段无明显错误
+- 用 `rg --files` 或 `test -e` 确认文档中的文件路径在项目中真实存在，避免凭记忆判断：
+- 检查中文文档没有误复制英文版：
+
+```bash
+python - <<'PY'
+from pathlib import Path
+same = []
+for en in sorted(Path("docs/technical/en").glob("*.md")):
+    zh = Path("docs/technical/zh") / en.name
+    if zh.exists() and en.read_text() == zh.read_text():
+        same.append(en.name)
+if same:
+    raise SystemExit("identical en/zh docs: " + ", ".join(same))
+print("no identical en/zh docs")
+PY
+```
+
+- 检查中文文档内部链接没有继续指向无语言目录：
+
+```bash
+rg -n "/home/ray/dev/linkong/planet/docs/technical/(?!zh|en)" docs/technical/zh --pcre2
+```
+
+```bash
+# 对文档中提到的关键路径做快速验证
+ls <mentioned_paths>
+```
+
+如需检查大量链接，优先用确定性提取：
+
+```bash
+rg -n "\]\(([^)]+)\)" docs/technical/zh/<doc>.md
+```
+
+### Step 5 — 完成确认
+
+输出摘要：
+
+```
+✓ 新建：docs/technical/zh/ops-planet-sh-startup.md（约 xxx 字）
+✓ 更新：docs/technical/zh/backend-datasources-api-performance.md
+```
+
+## 注意事项
+
+- 不要写流水账式的"改了 A、改了 B、改了 C"，要写改动背后的约束和权衡
+- 不要在文档中引用 PR 号、issue 号、或当前对话——这些会随时间失效
+- 代码片段保持简洁，只保留说明问题的关键部分，省略无关样板代码
+- 如果某个变更已有文档记录，优先在原文档中追加，而不是新建
+- 文档是给未来的开发者看的，假设读者熟悉项目但不了解这次改动的背景

.claude/commands/goal-driven.md

@@ -0,0 +1,93 @@
+---
+description: 用 goal-driven 方法推动一个复杂任务持续执行，直到明确成功标准被满足
+argument-hint: 建议填写任务目标；若同时给出成功标准更好
+allowed-tools: ["Read", "Edit", "Bash", "Grep", "Glob"]
+---
+
+# /goal-driven — 目标驱动执行模式
+
+使用 `lidangzzz/goal-driven` 的核心思想来推进复杂任务：先固定目标与成功标准，再持续执行和反复验收，直到标准真正满足。
+
+适用场景：
+
+- 长周期实现任务
+- 高复杂度工程任务
+- 可被明确验收的研究、实现、迁移、验证类工作
+
+不适用场景：
+
+- 纯脑暴
+- 无法定义成功标准的模糊任务
+- 很小的一次性修改
+
+## 输入要求
+
+若 `$ARGUMENTS` 只包含目标，没有成功标准，先补全一版可执行的成功标准再开始。
+
+启动时先输出：
+
+```md
+Goal
+- ...
+
+Criteria for success
+- ...
+
+Plan
+1. ...
+2. ...
+3. ...
+
+Verification
+- ...
+```
+
+## 执行规则
+
+1. 先把任务固化为两个核心块：
+   - `Goal`
+   - `Criteria for success`
+
+2. 成功标准必须尽量客观，可验证，可落地。
+   优先写成：
+   - 需要交付什么
+   - 需要通过哪些测试或验证
+   - 如何判断结果真的完成
+
+3. 进入持续执行循环：
+   - 完成一个阶段
+   - 检查当前结果是否满足成功标准
+   - 若未满足，明确剩余差距并继续推进
+
+4. 任何“完成了”“差不多了”“已实现”之类的结论，都必须经过验证，不能直接接受。
+
+5. 如果验证失败：
+   - 明确指出哪条成功标准没满足
+   - 继续工作，不要把阶段性进展误判为完成
+
+6. 只有在以下情况之一才能停止：
+   - 成功标准已满足
+   - 用户明确要求停止
+
+## 执行风格
+
+- 重证据，轻口头判断
+- 优先使用确定性工具证据：`rg`、`git diff --stat`、`git diff -- <path>`、测试、构建、lint、`curl`、数据库查询等能直接证明成功标准的方式
+- 不把大段命令输出粘进回复；保留在工具调用里，回复只总结关键证据
+- 重验收，轻自我感觉
+- 优先用测试、日志、产物、对比结果来证明完成
+- 对长期任务保持“未达标就继续”的节奏
+
+## 简版模板
+
+```md
+Goal: [[[[[在此填写最终目标]]]]]
+
+Criteria for success: [[[[[在此填写成功标准]]]]]
+
+循环执行：
+1. 推进任务
+2. 检查是否满足成功标准
+3. 若未满足，继续工作
+4. 直到满足标准或用户明确停止
+```

.claude/commands/release.md

@@ -28,6 +28,19 @@ allowed-tools: ["Read", "Edit", "Bash", "Glob", "Grep"]
 - `docs/CHANGELOG.md`
 - `docs/version-history.md`
 
+## 节省上下文规则
+
+发版判断应以确定性 CLI 证据为主，优先使用紧凑命令和定点读取：
+
+```bash
+git status --short
+git diff --stat HEAD
+git diff --name-only HEAD
+rg -n "version|^## |^Released:|当前开发版本|current" VERSION frontend/package.json pyproject.toml docs/CHANGELOG.md docs/version-history.md
+```
+
+除非需要判断某个代码变更是否属于本次发版，否则不要读取完整 diff。
+
 ## 执行步骤
 
 ### Step 1 — 环境检查
@@ -45,7 +58,7 @@ cat VERSION                        # 读取当前版本
 ### Step 2 — 确定发版类型与新版本号
 
 - 若 `$ARGUMENTS` 提供了明确类型（`feature` / `bugfix`），直接使用
-- 否则根据当前 `git diff HEAD` 和 `git log` 推断
+- 否则根据 `git diff --stat HEAD`、`git diff --name-only HEAD`、必要的 focused diff 和 `git log` 推断
 - 计算新版本号（例：`0.26.2` → bugfix → `0.26.3`）
 - **先输出发版计划供用户确认**：
 
@@ -91,12 +104,13 @@ cat VERSION                        # 读取当前版本
 
 针对本次变更范围做最小验证：
 
-- Python 文件有修改：`python3 -m py_compile <changed_files>`
-- Frontend 文件有修改：运行项目标准检查（若无则跳过并说明）
+- Python 文件有修改：先用 `git diff --name-only HEAD -- '*.py'` 列出，再运行 `python3 -m py_compile <changed_files>`
+- Frontend 文件有修改：先用 `git diff --name-only HEAD -- frontend` 判断范围，再运行项目标准检查（若无则跳过并说明）
 - 版本号一致性检查：用 grep 确认 VERSION、package.json、pyproject.toml 中的版本号完全一致
 
 ```bash
-grep -h "version" VERSION frontend/package.json pyproject.toml
+cat VERSION
+rg -n "\"version\":|^version =|version = " frontend/package.json pyproject.toml uv.lock
 ```
 
 ### Step 7 — 提交前预览

.codex/config.toml

@@ -0,0 +1,3 @@
+approval_policy = "never"
+
+sandbox_mode = "danger-full-access"

.codex/skills/cleanup/SKILL.md

@@ -21,6 +21,19 @@ If the user specifies a file or directory, check only that. Otherwise check all
 
 Only report issues present in **newly added or modified** lines of this diff — do not audit unchanged code.
 
+## Token-Saving Rule
+
+Prefer deterministic CLI checks before reading files into model context:
+
+```bash
+git diff --name-only HEAD
+git diff --unified=0 HEAD -- <path>
+git diff --check
+rg -n "TODO|FIXME|console\.log|debugger|print\(" <changed-paths>
+```
+
+Read full files only when the focused diff does not provide enough surrounding context to make a safe edit.
+
 ## Checklist
 
 ### 1. Duplicate Logic
@@ -64,7 +77,13 @@ Filter to the user-specified path if one was provided.
 
 ### Step 2 — Read and analyze each file
 
-Read the full file (not just the diff) with the Read tool. For each file, record every issue found: filename, line number, category, and suggested fix.
+Start with focused diffs:
+
+```bash
+git diff --unified=0 HEAD -- <file>
+```
+
+Use `rg`, `git diff --check`, and compiler/linter output for deterministic findings. Read the full file only for files that need surrounding context. For each issue found, record filename, line number, category, and suggested fix.
 
 ### Step 3 — Report findings before touching anything
 
@@ -99,6 +118,7 @@ Principles:
 - Only fix issues identified in the checklist — no extra improvements
 - Keep each Edit as small as possible
 - After fixing, verify the old bad pattern is gone with grep
+- Prefer `apply_patch` for targeted edits; use formatters only when the repository already uses them for the touched file type
 
 ### Step 5 — Summary
 

.codex/skills/docs/SKILL.md

@@ -0,0 +1,136 @@
+---
+name: docs
+description: Analyze current Planet repo changes and create or update technical documentation under docs/technical/zh. Use when the user asks to write docs, update technical docs, summarize implementation changes into documentation, or port the Claude docs-codex workflow into Codex.
+---
+
+# Docs
+
+Use this skill when the user asks to create or update Planet technical documentation, especially under `docs/technical/zh/`.
+
+## Goal
+
+Write or update technical docs that explain why a change exists, not only what files changed.
+
+Default target directory:
+
+- `docs/technical/zh/`
+
+## Workflow
+
+1. Gather change context:
+
+```bash
+git diff HEAD --stat
+git diff HEAD --name-only
+git log --oneline -10
+ls docs/technical/zh/
+```
+
+If the user gives a specific topic, focus on that topic. Otherwise infer the documentation topic from the file list and diff stat. Do **not** read the full repository diff by default; inspect focused diffs only for the files that define the doc topic:
+
+```bash
+git diff HEAD -- <path>
+rg -n "class |def |function |export |router|@router|interface |type " <path>
+```
+
+2. Decide document scope:
+
+- Use one document for one coherent topic.
+- Split documents when the changes cross meaningful domains, such as backend performance and ops startup behavior.
+- Prefer updating an existing relevant doc over creating a duplicate.
+- Name new files as lowercase hyphenated `domain-topic-detail.md`, for example:
+  - `backend-datasources-api-performance.md`
+  - `ops-planet-sh-startup.md`
+  - `earth-bgp-context.md`
+
+3. Apply the documentation coverage checklist before writing:
+
+- User-visible workflow changes must update `docs/technical/zh/manual.md` and usually `docs/technical/zh/quickstart.md`.
+- If an English counterpart exists for user-facing docs such as `manual.md` or `quickstart.md`, update `docs/technical/en/...` enough that it does not contradict the Chinese source.
+- Control console page responsibility changes must update `docs/technical/zh/frontend-admin-frontend-context.md`.
+- Earth frontend behavior changes must update `docs/technical/zh/earth-frontend-context.md`.
+- Earth layer additions, `renderOrder`, altitude/radius offsets, depth strategy, pointer picking, legend modes, or layer panel/startup ordering must update `docs/technical/zh/earth-render-layer-order.md`.
+- Earth layer visual style or legend symbol/color semantics should also update `docs/technical/zh/earth-layer-style-reference.md` when that reference is affected.
+- Collector, datasource, credential, settings, connectivity, scheduler, or API changes must update the relevant backend docs, especially `docs/technical/zh/backend-collectors.md` and any datasource/settings-specific doc.
+- When a change turns an old plan assumption into current behavior, update the relevant `docs/plans/*.md` with a status note instead of leaving contradictory instructions.
+- If adding a new technical document, add it to `docs/technical/zh/README.md` when it should be discoverable from the technical docs index.
+- Search docs for stale terms introduced by the change, for example old tab names, old route responsibilities, obsolete auth assumptions, or renamed UI labels.
+
+4. Write the doc in Chinese:
+
+- Write Chinese prose for `docs/technical/zh/`.
+- Keep technical identifiers, API paths, config keys, code symbols, and standard product names in English where appropriate.
+- Use `##` and `###` headings; avoid going deeper than three levels.
+- Use fenced code blocks with language tags.
+- Use tables when comparing options or listing parameters.
+
+5. Required content:
+
+- Background/problem: what was wrong before and why the change was needed.
+- Core design decisions and rationale.
+- Key code snippets, preferably before/after or focused excerpts.
+- Related files and what each file contributes.
+
+6. Verification:
+
+- Read the completed doc and check that the reasoning is clear.
+- Verify important referenced paths exist.
+- Use `rg --files` or `test -e` for path existence instead of relying on memory.
+- Run a quick duplicate-language check when editing bilingual docs:
+
+```bash
+python - <<'PY'
+from pathlib import Path
+same = []
+for en in sorted(Path("docs/technical/en").glob("*.md")):
+    zh = Path("docs/technical/zh") / en.name
+    if zh.exists() and en.read_text() == zh.read_text():
+        same.append(en.name)
+if same:
+    raise SystemExit("identical en/zh docs: " + ", ".join(same))
+print("no identical en/zh docs")
+PY
+```
+
+Also check that Chinese docs do not link to the old language-less technical docs path:
+
+```bash
+rg -n "/home/ray/dev/linkong/planet/docs/technical/(?!zh|en)" docs/technical/zh --pcre2
+```
+
+This command should return no matches.
+
+If checking many links, prefer deterministic extraction:
+
+```bash
+rg -n "\]\(([^)]+)\)" docs/technical/zh/<doc>.md
+```
+
+Also run focused stale-term searches derived from the change, for example:
+
+```bash
+rg -n "old label|old route purpose|obsolete provider assumption" docs/technical docs/plans
+```
+
+## Hard Constraints
+
+- A file under `docs/technical/zh/` must not be an English source file copied as a placeholder.
+- Do not leave a Chinese doc with only an English title and English first-screen content.
+- When an English counterpart exists in `docs/technical/en/`, never duplicate it byte-for-byte into `docs/technical/zh/`.
+- Internal links inside `docs/technical/zh/` should point to `docs/technical/zh/...` for Chinese docs, unless intentionally linking to an English-only file.
+- Do not reference PR numbers, issue numbers, or the current conversation.
+- Do not write changelog-style lists like "changed A, changed B, changed C" without the constraints and tradeoffs behind those changes.
+- Keep code snippets concise and relevant.
+
+## Recommended Output
+
+After editing, summarize:
+
+```md
+Updated:
+- docs/technical/zh/example.md — what changed
+
+Verified:
+- no identical en/zh docs
+- no language-less docs/technical links in zh docs
+```

.codex/skills/goal-driven/SKILL.md

@@ -0,0 +1,103 @@
+---
+name: goal-driven
+description: Run a goal-driven execution loop for very large, long-horizon, rigorously verifiable tasks. Use when the user explicitly wants the lidangzzz/goal-driven method, a master-agent plus worker-agent style workflow, or a persistent loop that keeps working until concrete success criteria are satisfied.
+---
+
+# Goal-Driven
+
+Use this skill when the user wants a strict goal-driven workflow for a hard task with:
+
+- one clear end goal
+- explicit success criteria
+- repeated verification against those criteria
+- continued execution until the criteria are actually met
+
+This skill is adapted from `lidangzzz/goal-driven`, but trimmed for local skill use to avoid bloating context.
+
+## When To Use
+
+Use it for tasks like:
+
+- compilers, interpreters, theorem-like proof work, deep refactors
+- long-running system design or implementation work
+- problems that are expensive and complex, but still objectively testable
+
+Do not use it for:
+
+- vague brainstorming without a success condition
+- short one-shot edits
+- tasks where "done" cannot be evaluated in a meaningful way
+
+## Core Model
+
+The workflow has two roles:
+
+1. Master role
+   Defines the goal, defines the success criteria, audits progress, and decides whether the work is actually complete.
+
+2. Worker role
+   Keeps advancing the task toward the goal. If a result is partial, stalled, or unverifiable, the worker continues.
+
+In Codex, only use actual subagents when the user explicitly asks for delegation or subagent work and the platform supports it. Otherwise emulate the same loop locally: keep working, checkpointing, and re-verifying until the criteria are satisfied.
+
+## Workflow
+
+1. Normalize the task into two blocks:
+   - `Goal`
+   - `Criteria for success`
+
+2. Make the criteria concrete and testable.
+   Good criteria usually include:
+   - required outputs
+   - required validations or tests
+   - edge cases or coverage thresholds
+   - what evidence proves completion
+
+3. Break the work into milestones that can each produce evidence.
+
+4. Execute the next milestone.
+   If subagents are explicitly allowed, the master may delegate bounded worker tasks.
+   If not, do the work locally but keep the master/worker mindset.
+
+5. Whenever work pauses, stalls, or appears complete, audit against the criteria directly.
+   Check artifacts, tests, logs, diffs, metrics, or other real evidence.
+
+6. If the criteria are not met, continue with a specific delta:
+   - what is still missing
+   - what evidence failed
+   - what the next worker pass must improve
+
+7. Stop only when the criteria are met, or when the user explicitly stops the process.
+
+## Operating Rules
+
+- Prefer objective checks over self-reported completion.
+- Prefer deterministic tool evidence over long model summaries: use `rg`, `git diff --stat`, targeted `git diff -- <path>`, tests, builds, linters, `curl`, or database queries when they can prove a criterion.
+- Do not paste large command output into the conversation; summarize the evidence and keep raw output in tool calls.
+- Do not confuse progress with completion.
+- If the worker says "done", verify it.
+- If verification fails, continue from the gap instead of restarting blindly.
+- Keep the goal stable unless the user changes it.
+- Tighten fuzzy criteria before sinking large amounts of effort.
+
+## Recommended Response Shape
+
+When starting a goal-driven task, structure the kickoff like this:
+
+```md
+Goal
+- ...
+
+Criteria for success
+- ...
+
+Current plan
+1. ...
+2. ...
+3. ...
+
+Verification
+- What evidence will prove completion
+```
+
+For a reusable prompt template, read [references/prompt-template.md](references/prompt-template.md).

.codex/skills/goal-driven/agents/openai.yaml

@@ -0,0 +1,7 @@
+interface:
+  display_name: "Goal-Driven"
+  short_description: "Drive complex work until explicit success criteria are met."
+  default_prompt: "Use $goal-driven to turn this task into a concrete goal, explicit success criteria, and a verification-driven execution loop."
+
+policy:
+  allow_implicit_invocation: true

.codex/skills/goal-driven/references/prompt-template.md

@@ -0,0 +1,38 @@
+# Goal-Driven Prompt Template
+
+Use this when you want a reusable kickoff prompt for a master/worker execution loop.
+
+```md
+# Goal-Driven System
+
+Goal: [[[[[DEFINE THE FINAL GOAL HERE]]]]]
+
+Criteria for success: [[[[[DEFINE THE SUCCESS CRITERIA HERE]]]]]
+
+You are the master agent.
+
+Your job is to:
+1. Keep the goal and criteria fixed.
+2. Start worker execution toward the goal.
+3. Audit any claimed progress against the criteria.
+4. If the criteria are not met, continue the work with a precise next delta.
+5. Stop only when the criteria are satisfied or the user explicitly stops the process.
+
+Worker requirements:
+1. Break the task into subproblems.
+2. Keep producing concrete progress toward the goal.
+3. Report evidence, not just claims.
+4. Continue until the criteria are satisfied.
+
+Master audit loop:
+1. Check whether the worker is still making progress.
+2. If the worker stalls or claims completion, verify against the criteria.
+3. If verification fails, resume work from the remaining gap.
+4. Repeat until the criteria are met.
+```
+
+## Notes
+
+- Stronger criteria produce better results than stronger rhetoric.
+- Prefer measurable checks such as tests, parity checks, generated artifacts, benchmarks, or reviewable outputs.
+- If the environment does not support subagents, emulate the same loop locally.

.codex/skills/release/SKILL.md

@@ -18,7 +18,7 @@ Do not use this skill for ordinary commits that are not being released.
 
 ## Versioning Rules
 
-- `feature` -> bump `+0.1.0`
+- `feature` -> bump minor and reset patch to `0` (`x.y.z` → `x.(y+1).0`; for example `0.41.2` → `0.42.0`)
 - `bugfix` -> bump `+0.0.1`
 - `docs`, `maintenance`, and `refactor` do not bump by default unless the user explicitly wants a release
 
@@ -35,6 +35,19 @@ Use `git rev-parse --show-toplevel` to get the repo root. All paths are relative
 - `docs/CHANGELOG.md`
 - `docs/version-history.md`
 
+## Token-Saving Rule
+
+Release work should be driven by deterministic CLI evidence. Prefer compact commands and targeted file reads:
+
+```bash
+git status --short
+git diff --stat HEAD
+git diff --name-only HEAD
+rg -n "version|^## |^Released:|current" VERSION frontend/package.json pyproject.toml docs/CHANGELOG.md docs/version-history.md
+```
+
+Do not inspect full diffs unless deciding whether changed code belongs in the release.
+
 ## Workflow
 
 ### Step 1 — Environment check
@@ -52,8 +65,10 @@ If unrelated uncommitted changes exist, list them and ask the user whether to in
 ### Step 2 — Determine release type and next version
 
 - If the user provided an explicit type (`feature` / `bugfix`), use it
-- Otherwise infer from `git diff HEAD` and recent `git log`
-- Compute the next version (e.g. `0.26.2` → bugfix → `0.26.3`)
+- Otherwise infer from `git diff --stat HEAD`, `git diff --name-only HEAD`, focused diffs for changed code, and recent `git log`
+- Compute the next version:
+  - `feature`: increment minor and reset patch to `0` (e.g. `0.41.2` → `0.42.0`)
+  - `bugfix`: increment patch only (e.g. `0.26.2` → `0.26.3`)
 - **Show the release plan before making any changes:**
 
 ```
@@ -104,12 +119,13 @@ Get today's date with `date +%Y-%m-%d`.
 
 Run the smallest relevant validation for the changes in scope:
 
-- Python files changed: `python3 -m py_compile <changed_files>`
-- Frontend files changed: run the project-standard check if available; otherwise skip and say so
+- Python files changed: list changed Python files with `git diff --name-only HEAD -- '*.py'`, then run `python3 -m py_compile <changed_files>`
+- Frontend files changed: list changed frontend files with `git diff --name-only HEAD -- frontend`, then run the project-standard check if available; otherwise skip and say so
 - Version consistency: confirm VERSION, package.json, pyproject.toml, and uv.lock all show the same version
 
 ```bash
-grep -h "version" VERSION frontend/package.json pyproject.toml
+cat VERSION
+rg -n "\"version\":|^version =|version = " frontend/package.json pyproject.toml uv.lock
 ```
 
 ### Step 7 — Pre-commit preview

README.md

@@ -227,6 +227,120 @@ bun run build
 
 启动服务后访问: `http://localhost:8000/docs`
 
+## WSL / Windows 局域网访问
+
+如果服务运行在 WSL 中，而你希望：
+
+- Windows 本机浏览器访问开发服务
+- 同一局域网内的手机或其他电脑访问开发服务
+
+推荐按下面顺序排查和配置。
+
+### 1. 在 WSL 中启动服务
+
+```bash
+./planet.sh start --allow-lan
+```
+
+这会让前端监听 `0.0.0.0:3000`，后端监听 `0.0.0.0:8000`。
+
+### 2. 先确认 WSL 内部服务正常
+
+在 WSL 中执行：
+
+```bash
+curl http://localhost:3000
+curl http://localhost:8000/health
+ss -ltnp | grep -E ':3000|:8000'
+```
+
+预期：
+
+- `3000` 返回前端 HTML
+- `8000/health` 返回健康检查 JSON
+- `ss` 中能看到 `0.0.0.0:3000` 和 `0.0.0.0:8000`
+
+如果这一步不通，先不要继续做 Windows 转发。
+
+### 3. 在 Windows 本机验证 localhost 直通
+
+在 Windows PowerShell 中执行：
+
+```powershell
+curl http://localhost:3000
+curl http://localhost:8000/health
+```
+
+在常见的 WSL2 开发环境下，Windows 通常可以直接通过 `localhost` 访问 WSL 中的服务。
+
+### 4. 如果需要让局域网设备访问，再做 Windows 端口转发
+
+注意：下面的命令必须在“以管理员身份运行”的 PowerShell 中执行。
+
+先把 Windows 对外网卡上的 `3000` / `8000` 转发到 Windows 本机 `127.0.0.1`：
+
+```powershell
+netsh interface portproxy delete v4tov4 listenaddress=0.0.0.0 listenport=3000
+netsh interface portproxy delete v4tov4 listenaddress=0.0.0.0 listenport=8000
+
+netsh interface portproxy add v4tov4 listenaddress=0.0.0.0 listenport=3000 connectaddress=127.0.0.1 connectport=3000
+netsh interface portproxy add v4tov4 listenaddress=0.0.0.0 listenport=8000 connectaddress=127.0.0.1 connectport=8000
+```
+
+再放行 Windows 防火墙：
+
+```powershell
+New-NetFirewallRule -DisplayName "WSL Planet 3000" -Direction Inbound -Action Allow -Protocol TCP -LocalPort 3000
+New-NetFirewallRule -DisplayName "WSL Planet 8000" -Direction Inbound -Action Allow -Protocol TCP -LocalPort 8000
+```
+
+检查转发规则是否生效：
+
+```powershell
+netsh interface portproxy show all
+```
+
+预期能看到：
+
+- `0.0.0.0:3000 -> 127.0.0.1:3000`
+- `0.0.0.0:8000 -> 127.0.0.1:8000`
+
+### 5. 查 Windows 局域网 IP，并让其他设备访问
+
+在 Windows PowerShell 中执行：
+
+```powershell
+ipconfig
+```
+
+找到当前联网网卡的 IPv4 地址，例如 `192.168.8.228`。
+
+局域网其他设备可访问：
+
+- `http://<Windows局域网IP>:3000/earth`
+- `http://<Windows局域网IP>:3000/admin`
+
+例如：
+
+- `http://192.168.8.228:3000/earth`
+
+### 6. 常见现象与判断
+
+- WSL 中 `curl localhost:3000` 能通，但 Windows 访问 `WSL 的局域网 IP:3000` 不通：这是正常现象之一，优先验证 Windows 的 `localhost:3000`
+- Windows `localhost:3000` 能通，但局域网设备访问 `Windows 局域网 IP:3000` 不通：通常缺少 `portproxy` 或防火墙放行
+- `whoami /groups` 中 `S-1-5-32-544` 显示 `deny only`：说明当前 PowerShell 不是提权管理员窗口
+
+### 7. 本项目一次性验证顺序
+
+建议固定按这个顺序验证：
+
+1. WSL 中执行 `curl http://localhost:3000`
+2. WSL 中执行 `curl http://localhost:8000/health`
+3. Windows 中执行 `curl http://localhost:3000`
+4. Windows 中执行 `curl http://localhost:8000/health`
+5. 管理员 PowerShell 配置 `portproxy` 和防火墙
+6. 用手机或其他电脑访问 `http://<Windows局域网IP>:3000/earth`
+
 ## 启动容错参数
 
 `planet.sh` 现在为依赖安装、数据库、AI Provider 启动加入了有限次重试，并会在数据库与 `aiprovider` 启动后额外等待 Docker healthcheck。
@@ -328,11 +442,11 @@ AI_PROVIDER_SERVICE_TOKEN=change_me
 
 详细文档:
 
-- [docs/agents/aiprovider.md](/home/ray/dev/linkong/planet/docs/agents/aiprovider.md)
+- [docs/technical/agents-aiprovider.md](/home/ray/dev/linkong/planet/docs/technical/agents-aiprovider.md)
 - [aiprovider/README.md](/home/ray/dev/linkong/planet/aiprovider/README.md)
-- [docs/frontend/frontend-layout-guidelines.md](/home/ray/dev/linkong/planet/docs/frontend/frontend-layout-guidelines.md)
-- [docs/frontend/ai-playground-development-plan.md](/home/ray/dev/linkong/planet/docs/frontend/ai-playground-development-plan.md)
-- [docs/agents/situational-awareness-foundation-plan.md](/home/ray/dev/linkong/planet/docs/agents/situational-awareness-foundation-plan.md)
+- [docs/technical/frontend-layout-guidelines.md](/home/ray/dev/linkong/planet/docs/technical/frontend-layout-guidelines.md)
+- [docs/plans/frontend-ai-playground-development-plan.md](/home/ray/dev/linkong/planet/docs/plans/frontend-ai-playground-development-plan.md)
+- [docs/plans/agents-situational-awareness-foundation-plan.md](/home/ray/dev/linkong/planet/docs/plans/agents-situational-awareness-foundation-plan.md)
 
 ## 前端页面布局规范
 
@@ -346,7 +460,7 @@ AI_PROVIDER_SERVICE_TOKEN=change_me
 当前推荐参考实现：
 
 - [frontend/src/pages/BGP/BGP.tsx](/home/ray/dev/linkong/planet/frontend/src/pages/BGP/BGP.tsx)
-- [docs/frontend/frontend-layout-guidelines.md](/home/ray/dev/linkong/planet/docs/frontend/frontend-layout-guidelines.md)
+- [docs/technical/frontend-layout-guidelines.md](/home/ray/dev/linkong/planet/docs/technical/frontend-layout-guidelines.md)
 
 ## License
 

TODO.md

@@ -20,3 +20,19 @@
 - [x] 在 activity layer 之后继续补 `route leak` 和 `path instability / flap` detector
 - [ ] 对 [frontend/public/earth/js/bgp.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/bgp.js) 做按职责拆分的小重构，拆成 data / markers / overlays / animation，降低后续维护复杂度
 - [ ] 可选优化（非必做）：将 BGP incident/collector 标点改为 HTML marker（参考 worldmonitor 的 `htmlElementsData` 思路），实现近乎固定屏幕尺寸与更高密度可点击性
+- [ ] 保持 Earth 当前这批纯个人偏好设置继续走本地持久化：`旋转模式`、HUD 面板显示/隐藏、`地形透明度` 暂不升级到后端系统设置，避免把设备级偏好过早做成全局配置
+- [ ] 如果后续明确需要“账号级同步 Earth 偏好”，再单独设计 `Earth user preferences`：优先按用户维度而不是全局系统设置保存，并规划 `localStorage -> backend` 的平滑迁移策略
+- [ ] 为 Planet / Earth 补一个可用的日志查看系统：先明确前后端/AI Provider/采集任务的日志入口、最近日志聚合、筛选与 tail 能力，再决定是先做脚本级统一入口还是控制台内置日志面板
+- [ ] 重写控制台 UI，逐步抛弃 Ant Design，建立自有组件体系，并统一采用 `tabler.io` / Tabler Icons 作为控制台主图标库
+- [ ] 把 Earth 态势新闻源从 [earth_news.py](/home/ray/dev/linkong/planet/backend/app/services/earth_news.py) 的硬编码列表抽成可配置目录，优先保持当前“实时聚合”链路不变，只先解决新闻源不可配置的问题
+- [ ] 为 Earth 态势新闻设计后续采集器化方案：明确新闻数据模型、去重策略、区域映射、过期清理和 Earth/AI 复用方式，再决定何时把新闻从实时抓取升级成正式 collector
+- [ ] 为 Earth 地球表面增加一层与基础纹理对齐的材质/纹理 overlay，并在同层叠加国界轮廓参考线；要求国界线与底图稳定对齐，且 hover 到国家轮廓时能高亮当前国家，便于校准地表和增强交互
+- [ ] 把 Earth 新闻接入通用巡航队列：按新闻发生地和时间排序生成巡航目标，巡航聚焦到新闻事件时显示对应新闻卡片，并保持实现边界为“通用巡航层 + 新闻业务适配层”，不要再把新闻逻辑直接耦合回 `main.js` 状态机
+- [ ] 为未知位置的算力中心建立分层坐标补全链路：优先 `精确坐标 > 站点/园区命中 > 城市 > 州/省 > 国家内主要算力城市 > 国家质心`，并把每次回退的 `confidence / reason / precision` 明确写进统一 GeoJSON
+- [ ] 为算力中心补一份可维护的本地位置注册表，例如 `canonical_name / aliases / operator / country / region / city / lat / lon / confidence / source_note`，避免把地点知识长期硬编码在 `visualization.py`
+- [ ] 增强 `epoch_ai_gpu` 和相关算力采集器的源页面解析：即使公开 API 不给坐标，也继续尝试从详情页、HTML、内嵌 JSON、schema.org、OpenGraph、脚本变量和 PDF/新闻稿链接里抽地点线索
+- [ ] 为未知位置算力中心增加外部富化策略评估：可选接入公开知识源或搜索兜底，只抓“站点名/园区名/城市名”级别线索，不直接抓经纬度结论，并把结果作为候选证据而不是真值
+- [ ] 为算力中心建立 `operator / cluster name / facility alias` 归一化层，先解决 `xAI / Colossus / Memphis`、`OpenAI / Stargate`、`CoreWeave`、`Lambda`、`Crusoe` 这类同一对象多种写法导致的地点匹配失败
+- [ ] 为估算位置增加更细的视觉和产品表达：除了问号角标，还要支持 tooltip/详情中的“估算依据”“精度级别”“最后核验时间”，并允许在设置中单独开关“仅看精确位置”
+- [ ] 为国家级估算点设计更合理的落点策略：优先落在“该国主要算力/数据中心城市候选集”而不是几何质心，必要时同国多节点做稳定散列分配，避免大量节点堆在荒漠或海上
+- [ ] 为未知位置算力中心建立人工校验工作流：支持导出待核验清单、记录人工确认结果，并把人工确认反哺到位置注册表，逐步减少问号点比例

VERSION

@@ -1 +1 @@
-0.28.1
+0.44.0

aiprovider/Dockerfile

@@ -1,6 +1,10 @@
-FROM python:3.14-slim
+ARG PYTHON_IMAGE=python:3.14-slim
+ARG UV_IMAGE=ghcr.io/astral-sh/uv:latest
 
-COPY --from=ghcr.io/astral-sh/uv:latest /uv /uvx /bin/
+FROM ${UV_IMAGE} AS uv
+FROM ${PYTHON_IMAGE}
+
+COPY --from=uv /uv /uvx /bin/
 
 WORKDIR /app
 

aiprovider/main.py

@@ -37,8 +37,26 @@ def verify_service_token(x_provider_token: str | None = Header(default=None)) ->
         )
 
 
-def get_provider_service() -> ProviderService:
-    return ProviderService()
+def get_provider_service(
+    x_ai_provider: str | None = Header(default=None),
+    x_ai_provider_api: str | None = Header(default=None),
+    x_ai_base_url: str | None = Header(default=None),
+    x_ai_api_key: str | None = Header(default=None),
+    x_ai_model: str | None = Header(default=None),
+    x_ai_max_tokens: str | None = Header(default=None),
+    x_ai_anthropic_version: str | None = Header(default=None),
+) -> ProviderService:
+    overrides = {
+        "provider": x_ai_provider,
+        "provider_api": x_ai_provider_api,
+        "base_url": x_ai_base_url,
+        "api_key": x_ai_api_key,
+        "model": x_ai_model,
+        "anthropic_version": x_ai_anthropic_version,
+    }
+    if x_ai_max_tokens:
+        overrides["max_tokens"] = x_ai_max_tokens
+    return ProviderService({key: value for key, value in overrides.items() if value not in (None, "")})
 
 
 @app.get("/health")

aiprovider/provider_service.py

@@ -46,19 +46,22 @@ def _resolve_provider_api(provider: str, configured_api: str) -> str:
 
 
 class ProviderService:
-    def __init__(self) -> None:
-        self.provider = _normalize_provider(settings.AI_PROVIDER)
+    def __init__(self, overrides: dict[str, Any] | None = None) -> None:
+        overrides = overrides or {}
+        self.provider = _normalize_provider(overrides.get("provider") or settings.AI_PROVIDER)
         self.provider_api = _resolve_provider_api(
             self.provider,
-            _normalize_provider_api(settings.AI_PROVIDER_API),
+            _normalize_provider_api(overrides.get("provider_api") or settings.AI_PROVIDER_API),
         )
-        self.base_url = settings.AI_BASE_URL.rstrip("/")
-        self.api_key = settings.AI_API_KEY
-        self.default_model = settings.AI_MODEL
+        self.base_url = str(overrides.get("base_url") or settings.AI_BASE_URL).rstrip("/")
+        self.api_key = str(overrides.get("api_key") or settings.AI_API_KEY)
+        self.default_model = str(overrides.get("model") or settings.AI_MODEL)
         self.timeout = settings.AI_TIMEOUT_SECONDS
         self.http_retry_attempts = max(settings.AI_HTTP_RETRY_ATTEMPTS, 1)
-        self.max_tokens = settings.AI_MAX_TOKENS
-        self.anthropic_version = settings.AI_ANTHROPIC_VERSION
+        self.max_tokens = int(overrides.get("max_tokens") or settings.AI_MAX_TOKENS)
+        self.anthropic_version = str(
+            overrides.get("anthropic_version") or settings.AI_ANTHROPIC_VERSION
+        )
         self.system_prompt = settings.AI_ANALYSIS_SYSTEM_PROMPT
 
     def get_status(self) -> AIProviderStatusResponse:

backend/Dockerfile

@@ -1,6 +1,10 @@
-FROM python:3.14-slim
+ARG PYTHON_IMAGE=python:3.14-slim
+ARG UV_IMAGE=ghcr.io/astral-sh/uv:latest
 
-COPY --from=ghcr.io/astral-sh/uv:latest /uv /uvx /bin/
+FROM ${UV_IMAGE} AS uv
+FROM ${PYTHON_IMAGE}
+
+COPY --from=uv /uv /uvx /bin/
 
 WORKDIR /app
 

backend/app/api/v1/datasource_config.py

@@ -1,20 +1,41 @@
 """DataSourceConfig API for user-defined data sources"""
 
-from typing import Optional
+from typing import Any, Optional
 from datetime import datetime
 import base64
+import json
+import re
 from fastapi import APIRouter, Depends, HTTPException, status
 from sqlalchemy import select, func
 from sqlalchemy.ext.asyncio import AsyncSession
 from pydantic import BaseModel, Field
 import httpx
 
+from app.core.target_schema_registry import get_target_schema, list_target_schemas
+from app.core.datasource_defaults import DEFAULT_DATASOURCES
 from app.db.session import get_db
 from app.models.user import User
 from app.models.datasource_config import DataSourceConfig
+from app.models.datasource_mapping import DataSourceMappingTemplate
 from app.core.security import get_current_user
 from app.core.cache import cache
 from app.core.time import to_iso8601_utc
+from app.schemas.ai import SituationalAnalysisRequest
+from app.services.ai_client import AIProviderClient, get_ai_provider_client
+from app.services.datasource_mapping import (
+    MappingError,
+    build_heuristic_mapping,
+    execute_mapping,
+    persist_mapped_records,
+    redact_for_llm,
+    stable_payload_hash,
+)
+from app.services.datasource_connectivity import (
+    get_builtin_connection_status,
+    save_connectivity_success,
+    strip_connectivity_validation,
+    test_builtin_connectivity,
+)
 
 router = APIRouter()
 
@@ -59,6 +80,70 @@ class DataSourceConfigResponse(BaseModel):
         from_attributes = True
 
 
+def _is_builtin_config_name(name: str | None) -> bool:
+    return bool(name and name in DEFAULT_DATASOURCES)
+
+
+async def _ensure_builtin_connection_verified(
+    db: AsyncSession,
+    config_data: DataSourceConfigCreate,
+) -> None:
+    if not _is_builtin_config_name(config_data.name):
+        return
+
+    status_result = await get_builtin_connection_status(
+        db,
+        config_data.name,
+        config_data.endpoint,
+        config_data.auth_type,
+        config_data.headers,
+        config_data.config,
+    )
+    if not status_result.get("connected"):
+        raise HTTPException(
+            status_code=400,
+            detail=status_result.get("message") or "请先完成连接验证，再保存内置采集器配置。",
+        )
+
+
+class CustomSampleRequest(BaseModel):
+    datasource_config_id: Optional[int] = None
+    config: Optional[DataSourceConfigCreate] = None
+    limit_bytes: int = Field(default=200000, ge=1000, le=1000000)
+
+
+class MappingProposeRequest(BaseModel):
+    sample_payload: Any
+    target_schema: str
+    use_ai: bool = True
+
+
+class MappingPreviewRequest(BaseModel):
+    sample_payload: Any
+    target_schema: str
+    mapping_json: dict
+    limit: int = Field(default=20, ge=1, le=100)
+
+
+class MappingTemplateCreate(BaseModel):
+    datasource_config_id: int
+    target_schema: str
+    mapping_json: dict
+    sample_payload: Any | None = None
+    sample_payload_hash: Optional[str] = None
+    validation_status: str = Field(default="draft", pattern="^(draft|valid|invalid)$")
+    is_active: bool = False
+
+
+class MappingTemplateUpdate(BaseModel):
+    target_schema: Optional[str] = None
+    mapping_json: Optional[dict] = None
+    sample_payload: Any | None = None
+    sample_payload_hash: Optional[str] = None
+    validation_status: Optional[str] = Field(default=None, pattern="^(draft|valid|invalid)$")
+    is_active: Optional[bool] = None
+
+
 async def test_endpoint(
     endpoint: str,
     auth_type: str,
@@ -96,6 +181,134 @@ async def test_endpoint(
         }
 
 
+def _build_request_headers(auth_type: str, auth_config: dict, headers: dict) -> dict[str, str]:
+    request_headers = {str(key): str(value) for key, value in (headers or {}).items()}
+    auth_type = str(auth_type or "none").lower()
+    auth_config = auth_config or {}
+
+    if auth_type == "bearer" and auth_config.get("token"):
+        request_headers["Authorization"] = f"Bearer {auth_config['token']}"
+    elif auth_type == "api_key" and auth_config.get("api_key"):
+        location = str(auth_config.get("in") or auth_config.get("location") or "header").lower()
+        if location != "query":
+            key_name = auth_config.get("key_name", "X-API-Key")
+            request_headers[str(key_name)] = str(auth_config["api_key"])
+    elif auth_type == "basic":
+        username = auth_config.get("username", "")
+        password = auth_config.get("password", "")
+        credentials = f"{username}:{password}"
+        encoded = base64.b64encode(credentials.encode()).decode()
+        request_headers["Authorization"] = f"Basic {encoded}"
+    return request_headers
+
+
+def _build_query_params(auth_type: str, auth_config: dict, config: dict) -> dict[str, Any]:
+    params = {}
+    candidate = (config or {}).get("params") or (config or {}).get("query_params")
+    if isinstance(candidate, dict):
+        params.update(candidate)
+
+    auth_type = str(auth_type or "none").lower()
+    auth_config = auth_config or {}
+    if auth_type == "api_key" and auth_config.get("api_key"):
+        location = str(auth_config.get("in") or auth_config.get("location") or "header").lower()
+        if location == "query":
+            key_name = auth_config.get("key_name") or auth_config.get("param_name") or "api_key"
+            params[str(key_name)] = auth_config["api_key"]
+    return params
+
+
+async def fetch_custom_sample_from_config(config: DataSourceConfig, limit_bytes: int) -> Any:
+    request_config = config.config or {}
+    method = str(request_config.get("method") or request_config.get("request_method") or "GET").upper()
+    if method not in {"GET", "POST"}:
+        raise HTTPException(status_code=400, detail="Only GET and POST sample requests are supported.")
+
+    headers = _build_request_headers(config.auth_type, config.auth_config or {}, config.headers or {})
+    params = _build_query_params(config.auth_type, config.auth_config or {}, request_config)
+    timeout = float(request_config.get("timeout", 30))
+    json_body = request_config.get("json_body")
+    if json_body is None and str(request_config.get("body_type") or "").lower() in {"json", ""}:
+        candidate = request_config.get("body")
+        if isinstance(candidate, (dict, list)):
+            json_body = candidate
+
+    async with httpx.AsyncClient(timeout=timeout, follow_redirects=True) as client:
+        response = await client.request(
+            method,
+            config.endpoint,
+            headers=headers,
+            params=params or None,
+            json=json_body,
+        )
+        response.raise_for_status()
+        content = response.content[:limit_bytes]
+        if "application/json" in response.headers.get("content-type", ""):
+            return json.loads(content.decode(response.encoding or "utf-8"))
+        return {"text": content.decode(response.encoding or "utf-8", errors="replace")}
+
+
+def _parse_mapping_from_ai_text(content: str) -> dict[str, Any] | None:
+    if not content:
+        return None
+
+    candidates = [content]
+    fenced = re.findall(r"```(?:json)?\s*(\{.*?\})\s*```", content, flags=re.DOTALL)
+    candidates = fenced + candidates
+    for candidate in candidates:
+        try:
+            parsed = json.loads(candidate)
+        except json.JSONDecodeError:
+            continue
+        if isinstance(parsed, dict) and isinstance(parsed.get("fields"), dict):
+            return parsed
+    return None
+
+
+async def _get_config_for_sample(
+    payload: CustomSampleRequest,
+    db: AsyncSession,
+) -> DataSourceConfig:
+    if payload.datasource_config_id is not None:
+        result = await db.execute(
+            select(DataSourceConfig).where(DataSourceConfig.id == payload.datasource_config_id)
+        )
+        config = result.scalar_one_or_none()
+        if not config:
+            raise HTTPException(status_code=404, detail="Configuration not found")
+        return config
+
+    if payload.config is None:
+        raise HTTPException(status_code=400, detail="datasource_config_id or config is required")
+
+    config_data = payload.config
+    return DataSourceConfig(
+        name=config_data.name,
+        description=config_data.description,
+        source_type=config_data.source_type,
+        endpoint=config_data.endpoint,
+        auth_type=config_data.auth_type,
+        auth_config=config_data.auth_config,
+        headers=config_data.headers,
+        config=config_data.config,
+    )
+
+
+def serialize_mapping_template(template: DataSourceMappingTemplate) -> dict[str, Any]:
+    return {
+        "id": template.id,
+        "datasource_config_id": template.datasource_config_id,
+        "target_schema": template.target_schema,
+        "mapping_json": template.mapping_json,
+        "sample_payload_hash": template.sample_payload_hash,
+        "validation_status": template.validation_status,
+        "version": template.version,
+        "is_active": template.is_active,
+        "created_at": to_iso8601_utc(template.created_at),
+        "updated_at": to_iso8601_utc(template.updated_at),
+    }
+
+
 @router.get("/configs")
 async def list_configs(
     active_only: bool = False,
@@ -132,6 +345,47 @@ async def list_configs(
     }
 
 
+@router.get("/configs/all")
+async def list_all_datasources(
+    current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    """List all data sources: YAML defaults + DB overrides"""
+    from app.core.data_sources import COLLECTOR_URL_KEYS, get_data_sources_config
+
+    config = get_data_sources_config()
+
+    db_query = await db.execute(select(DataSourceConfig))
+    db_configs = {c.name: c for c in db_query.scalars().all()}
+
+    result = []
+    for name, yaml_key in COLLECTOR_URL_KEYS.items():
+        yaml_url = config.get_yaml_url(name)
+        db_config = db_configs.get(name)
+
+        result.append(
+            {
+                "name": name,
+                "default_url": yaml_url,
+                "endpoint": db_config.endpoint if db_config else yaml_url,
+                "is_overridden": db_config is not None and db_config.endpoint != yaml_url
+                if yaml_url
+                else db_config is not None,
+                "is_active": db_config.is_active if db_config else True,
+                "source_type": db_config.source_type if db_config else "http",
+                "auth_type": db_config.auth_type if db_config else "none",
+                "headers": db_config.headers if db_config else {},
+                "config": strip_connectivity_validation(db_config.config if db_config else {}),
+                "config_id": db_config.id if db_config else None,
+                "description": db_config.description
+                if db_config
+                else f"Data source from YAML: {yaml_key}",
+            }
+        )
+
+    return {"total": len(result), "data": result}
+
+
 @router.get("/configs/{config_id}")
 async def get_config(
     config_id: int,
@@ -176,7 +430,7 @@ async def create_config(
         auth_type=config_data.auth_type,
         auth_config=config_data.auth_config,
         headers=config_data.headers,
-        config=config_data.config,
+        config=strip_connectivity_validation(config_data.config),
     )
 
     db.add(config)
@@ -208,6 +462,8 @@ async def update_config(
 
     update_data = config_data.model_dump(exclude_unset=True)
     for field, value in update_data.items():
+        if field == "config":
+            value = strip_connectivity_validation(value)
         setattr(config, field, value)
 
     await db.commit()
@@ -310,38 +566,366 @@ async def test_new_config(
         }
 
 
-@router.get("/configs/all")
-async def list_all_datasources(
+@router.post("/configs/builtin/connection-status")
+async def get_builtin_config_connection_status(
+    config_data: DataSourceConfigCreate,
     current_user: User = Depends(get_current_user),
     db: AsyncSession = Depends(get_db),
 ):
-    """List all data sources: YAML defaults + DB overrides"""
-    from app.core.data_sources import COLLECTOR_URL_KEYS, get_data_sources_config
+    if not _is_builtin_config_name(config_data.name):
+        raise HTTPException(status_code=400, detail="Only built-in datasource configs are supported.")
 
-    config = get_data_sources_config()
+    return await get_builtin_connection_status(
+        db,
+        config_data.name,
+        config_data.endpoint,
+        config_data.auth_type,
+        config_data.headers,
+        config_data.config,
+    )
 
-    db_query = await db.execute(select(DataSourceConfig))
-    db_configs = {c.name: c for c in db_query.scalars().all()}
 
-    result = []
-    for name, yaml_key in COLLECTOR_URL_KEYS.items():
-        yaml_url = config.get_yaml_url(name)
-        db_config = db_configs.get(name)
+@router.post("/configs/builtin/connect")
+async def connect_builtin_config(
+    config_data: DataSourceConfigCreate,
+    current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    if not _is_builtin_config_name(config_data.name):
+        raise HTTPException(status_code=400, detail="Only built-in datasource configs are supported.")
 
-        result.append(
+    result = await test_builtin_connectivity(
+        config_data.name,
+        config_data.endpoint,
+        config_data.auth_type,
+        config_data.headers,
+        config_data.config,
+        db,
+    )
+    if result.get("success") and result.get("checksum"):
+        validation = await save_connectivity_success(
+            db,
+            config_data.name,
+            result["checksum"],
+            result,
+            connected_by="connection_button",
+        )
+        await db.commit()
+        return {
+            **result,
+            "connected": True,
+            "validation": validation,
+        }
+
+    return {
+        **result,
+        "connected": False,
+    }
+
+
+@router.post("/custom/sample")
+async def fetch_custom_sample(
+    payload: CustomSampleRequest,
+    current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    """Fetch a sample payload for a saved or draft custom data source."""
+    config = await _get_config_for_sample(payload, db)
+    try:
+        sample = await fetch_custom_sample_from_config(config, payload.limit_bytes)
+    except httpx.HTTPStatusError as exc:
+        raise HTTPException(
+            status_code=exc.response.status_code,
+            detail=f"Sample request failed: HTTP {exc.response.status_code}",
+        ) from exc
+    except httpx.HTTPError as exc:
+        raise HTTPException(status_code=502, detail=f"Sample request failed: {exc}") from exc
+
+    return {
+        "success": True,
+        "sample_payload": sample,
+        "sample_payload_hash": stable_payload_hash(sample),
+        "redacted_preview": redact_for_llm(sample),
+    }
+
+
+@router.get("/target-schemas")
+async def get_datasource_target_schemas(
+    current_user: User = Depends(get_current_user),
+):
+    """List target schemas available for custom datasource mapping."""
+    return {"data": list_target_schemas()}
+
+
+@router.post("/mappings/propose")
+async def propose_datasource_mapping(
+    payload: MappingProposeRequest,
+    current_user: User = Depends(get_current_user),
+    ai_client: AIProviderClient = Depends(get_ai_provider_client),
+):
+    """Generate a mapping draft for a sample payload and target schema."""
+    schema = get_target_schema(payload.target_schema)
+    redacted_sample = redact_for_llm(payload.sample_payload)
+    fallback_mapping = build_heuristic_mapping(redacted_sample, payload.target_schema)
+
+    ai_error: str | None = None
+    mapping = fallback_mapping
+    generated_by = "heuristic"
+    if payload.use_ai:
+        try:
+            response = await ai_client.analyze(
+                SituationalAnalysisRequest(
+                    title=f"Generate datasource mapping for {schema.key}",
+                    objective=(
+                        "Return only JSON for a deterministic mapping DSL. "
+                        "The JSON must contain source.items_path and fields. "
+                        "Do not include prose or code."
+                    ),
+                    context={
+                        "target_schema": schema.to_dict(),
+                        "sample_payload": redacted_sample,
+                        "mapping_dsl_example": fallback_mapping,
+                    },
+                    observations=[
+                        "Use JSONPath-like paths beginning with $.",
+                        "Never generate executable code.",
+                        "Use field types from the target schema.",
+                    ],
+                    constraints=[
+                        "Return a single JSON object.",
+                        "Do not include credentials or secrets.",
+                        "Mark uncertain optional fields with default null.",
+                    ],
+                )
+            )
+            parsed = _parse_mapping_from_ai_text(response.content)
+            if parsed:
+                mapping = parsed
+                generated_by = "ai_provider"
+            else:
+                ai_error = "AI provider did not return a valid mapping JSON object."
+        except HTTPException as exc:
+            ai_error = str(exc.detail)
+
+    mapping.setdefault("meta", {})
+    if isinstance(mapping["meta"], dict):
+        mapping["meta"].update(
             {
-                "name": name,
-                "default_url": yaml_url,
-                "endpoint": db_config.endpoint if db_config else yaml_url,
-                "is_overridden": db_config is not None and db_config.endpoint != yaml_url
-                if yaml_url
-                else db_config is not None,
-                "is_active": db_config.is_active if db_config else True,
-                "source_type": db_config.source_type if db_config else "http",
-                "description": db_config.description
-                if db_config
-                else f"Data source from YAML: {yaml_key}",
+                "generated_by": generated_by,
+                "requires_review": True,
+                "ai_error": ai_error,
             }
         )
 
-    return {"total": len(result), "data": result}
+    return {
+        "target_schema": schema.to_dict(),
+        "mapping_json": mapping,
+        "sample_payload_hash": stable_payload_hash(payload.sample_payload),
+        "redacted_sample_payload": redacted_sample,
+    }
+
+
+@router.post("/mappings/preview")
+async def preview_datasource_mapping(
+    payload: MappingPreviewRequest,
+    current_user: User = Depends(get_current_user),
+):
+    """Preview deterministic mapping output for a sample payload."""
+    try:
+        preview = execute_mapping(
+            payload.sample_payload,
+            payload.mapping_json,
+            payload.target_schema,
+            limit=payload.limit,
+        )
+    except (MappingError, ValueError) as exc:
+        raise HTTPException(status_code=400, detail=str(exc)) from exc
+
+    return {
+        "success": preview["failed_count"] == 0,
+        "preview": preview,
+        "sample_payload_hash": stable_payload_hash(payload.sample_payload),
+    }
+
+
+@router.get("/mappings")
+async def list_datasource_mappings(
+    datasource_config_id: Optional[int] = None,
+    active_only: bool = False,
+    current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    """List saved mapping templates."""
+    query = select(DataSourceMappingTemplate).order_by(
+        DataSourceMappingTemplate.datasource_config_id,
+        DataSourceMappingTemplate.version.desc(),
+    )
+    if datasource_config_id is not None:
+        query = query.where(DataSourceMappingTemplate.datasource_config_id == datasource_config_id)
+    if active_only:
+        query = query.where(DataSourceMappingTemplate.is_active.is_(True))
+
+    result = await db.execute(query)
+    mappings = result.scalars().all()
+    return {"total": len(mappings), "data": [serialize_mapping_template(item) for item in mappings]}
+
+
+@router.post("/mappings")
+async def create_datasource_mapping(
+    payload: MappingTemplateCreate,
+    current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    """Save a mapping template for a datasource config."""
+    get_target_schema(payload.target_schema)
+    datasource = await db.get(DataSourceConfig, payload.datasource_config_id)
+    if not datasource:
+        raise HTTPException(status_code=404, detail="Configuration not found")
+
+    if payload.sample_payload is not None:
+        try:
+            execute_mapping(payload.sample_payload, payload.mapping_json, payload.target_schema, limit=100)
+        except (MappingError, ValueError) as exc:
+            raise HTTPException(status_code=400, detail=f"Mapping validation failed: {exc}") from exc
+
+    result = await db.execute(
+        select(func.max(DataSourceMappingTemplate.version)).where(
+            DataSourceMappingTemplate.datasource_config_id == payload.datasource_config_id,
+            DataSourceMappingTemplate.target_schema == payload.target_schema,
+        )
+    )
+    next_version = int(result.scalar() or 0) + 1
+
+    if payload.is_active:
+        await db.execute(
+            DataSourceMappingTemplate.__table__.update()
+            .where(DataSourceMappingTemplate.datasource_config_id == payload.datasource_config_id)
+            .values(is_active=False)
+        )
+
+    template = DataSourceMappingTemplate(
+        datasource_config_id=payload.datasource_config_id,
+        target_schema=payload.target_schema,
+        mapping_json=payload.mapping_json,
+        sample_payload_hash=payload.sample_payload_hash
+        or (stable_payload_hash(payload.sample_payload) if payload.sample_payload is not None else None),
+        validation_status=payload.validation_status,
+        version=next_version,
+        is_active=payload.is_active,
+    )
+    db.add(template)
+    await db.commit()
+    await db.refresh(template)
+    return {"message": "Mapping template saved successfully", "data": serialize_mapping_template(template)}
+
+
+@router.put("/mappings/{mapping_id}")
+async def update_datasource_mapping(
+    mapping_id: int,
+    payload: MappingTemplateUpdate,
+    current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    """Update a mapping template in place."""
+    template = await db.get(DataSourceMappingTemplate, mapping_id)
+    if not template:
+        raise HTTPException(status_code=404, detail="Mapping template not found")
+
+    target_schema = payload.target_schema or template.target_schema
+    mapping_json = payload.mapping_json or template.mapping_json
+    get_target_schema(target_schema)
+    if payload.sample_payload is not None:
+        try:
+            execute_mapping(payload.sample_payload, mapping_json, target_schema, limit=100)
+        except (MappingError, ValueError) as exc:
+            raise HTTPException(status_code=400, detail=f"Mapping validation failed: {exc}") from exc
+
+    if payload.is_active is True:
+        await db.execute(
+            DataSourceMappingTemplate.__table__.update()
+            .where(DataSourceMappingTemplate.datasource_config_id == template.datasource_config_id)
+            .where(DataSourceMappingTemplate.id != template.id)
+            .values(is_active=False)
+        )
+
+    template.target_schema = target_schema
+    template.mapping_json = mapping_json
+    if payload.sample_payload_hash is not None:
+        template.sample_payload_hash = payload.sample_payload_hash
+    elif payload.sample_payload is not None:
+        template.sample_payload_hash = stable_payload_hash(payload.sample_payload)
+    if payload.validation_status is not None:
+        template.validation_status = payload.validation_status
+    if payload.is_active is not None:
+        template.is_active = payload.is_active
+
+    await db.commit()
+    await db.refresh(template)
+    return {"message": "Mapping template updated successfully", "data": serialize_mapping_template(template)}
+
+
+@router.post("/{config_id}/run-mapped")
+async def run_mapped_datasource(
+    config_id: int,
+    current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    """Run a saved custom datasource through its active deterministic mapping."""
+    datasource = await db.get(DataSourceConfig, config_id)
+    if not datasource:
+        raise HTTPException(status_code=404, detail="Configuration not found")
+
+    result = await db.execute(
+        select(DataSourceMappingTemplate)
+        .where(DataSourceMappingTemplate.datasource_config_id == config_id)
+        .where(DataSourceMappingTemplate.is_active.is_(True))
+        .order_by(DataSourceMappingTemplate.version.desc())
+        .limit(1)
+    )
+    mapping = result.scalar_one_or_none()
+    if not mapping:
+        raise HTTPException(status_code=404, detail="No active mapping template found")
+
+    try:
+        sample = await fetch_custom_sample_from_config(datasource, 5_000_000)
+        mapped = execute_mapping(sample, mapping.mapping_json, mapping.target_schema)
+    except httpx.HTTPStatusError as exc:
+        raise HTTPException(
+            status_code=exc.response.status_code,
+            detail=f"Datasource request failed: HTTP {exc.response.status_code}",
+        ) from exc
+    except httpx.HTTPError as exc:
+        raise HTTPException(status_code=502, detail=f"Datasource request failed: {exc}") from exc
+    except (MappingError, ValueError) as exc:
+        raise HTTPException(status_code=400, detail=f"Mapping failed: {exc}") from exc
+
+    if mapped["failed_count"] > 0:
+        return {
+            "status": "failed",
+            "datasource_config_id": config_id,
+            "mapping_id": mapping.id,
+            "mapping_version": mapping.version,
+            "target_schema": mapping.target_schema,
+            "mapped_count": mapped["mapped_count"],
+            "failed_count": mapped["failed_count"],
+            "errors": mapped["errors"][:20],
+        }
+
+    written_count = await persist_mapped_records(
+        db,
+        datasource_name=datasource.name,
+        datasource_config_id=datasource.id,
+        target_schema=mapping.target_schema,
+        records=mapped["records"],
+        mapping_version=mapping.version,
+    )
+    return {
+        "status": "success",
+        "datasource_config_id": config_id,
+        "mapping_id": mapping.id,
+        "mapping_version": mapping.version,
+        "target_schema": mapping.target_schema,
+        "fetched_count": mapped["total_items"],
+        "mapped_count": mapped["mapped_count"],
+        "written_count": written_count,
+    }

backend/app/api/v1/datasources.py

@@ -9,6 +9,7 @@ from sqlalchemy.ext.asyncio import AsyncSession
 from app.core.time import to_iso8601_utc
 from app.core.security import get_current_user
 from app.core.data_sources import get_data_sources_config
+from app.core.datasource_defaults import DEFAULT_DATASOURCES
 from app.db.session import get_db
 from app.models.collected_data import CollectedData
 from app.models.data_snapshot import DataSnapshot
@@ -35,6 +36,17 @@ def format_frequency_label(minutes: int) -> str:
     return f"{minutes}m"
 
 
+def datasource_metadata(source: str) -> dict:
+    info = DEFAULT_DATASOURCES.get(source, {})
+    return {
+        "display_name": info.get("display_name") or info.get("name") or source,
+        "is_free": bool(info.get("is_free", True)),
+        "requires_credentials": bool(info.get("requires_credentials", False)),
+        "credential_provider": info.get("credential_provider"),
+        "credential_status": info.get("credential_status", "none"),
+    }
+
+
 def is_due_for_collection(datasource: DataSource, now: datetime) -> bool:
     if datasource.last_run_at is None:
         return True
@@ -72,31 +84,6 @@ async def _load_latest_running_tasks(
     return {task.datasource_id: task for task in result.scalars().all()}
 
 
-async def _load_latest_completed_tasks(
-    db: AsyncSession,
-    datasource_ids: list[int],
-) -> dict[int, CollectionTask]:
-    if not datasource_ids:
-        return {}
-
-    ranked_tasks = (
-        select(
-            CollectionTask.id.label("task_id"),
-            _task_rank_column(CollectionTask.completed_at),
-        )
-        .where(CollectionTask.datasource_id.in_(datasource_ids))
-        .where(CollectionTask.completed_at.isnot(None))
-        .where(CollectionTask.status.in_(("success", "failed", "cancelled")))
-        .subquery()
-    )
-    result = await db.execute(
-        select(CollectionTask)
-        .join(ranked_tasks, CollectionTask.id == ranked_tasks.c.task_id)
-        .where(ranked_tasks.c.row_num == 1)
-    )
-    return {task.datasource_id: task for task in result.scalars().all()}
-
-
 async def _load_latest_task_ids(
     db: AsyncSession,
     datasource_ids: list[int],
@@ -123,21 +110,6 @@ async def _load_latest_task_ids(
     return {datasource_id: task_id for datasource_id, task_id in result.all()}
 
 
-async def _load_datasource_data_counts(
-    db: AsyncSession,
-    sources: list[str],
-) -> dict[str, int]:
-    if not sources:
-        return {}
-
-    result = await db.execute(
-        select(CollectedData.source, func.count(CollectedData.id))
-        .where(CollectedData.source.in_(sources))
-        .group_by(CollectedData.source)
-    )
-    return {source: count for source, count in result.all()}
-
-
 async def _load_datasource_endpoint_overrides(
     db: AsyncSession,
     sources: list[str],
@@ -161,7 +133,7 @@ async def _load_datasource_endpoint_overrides(
 async def _load_datasource_list_context(
     db: AsyncSession,
     datasources: list[DataSource],
-) -> tuple[dict[int, CollectionTask], dict[int, CollectionTask], dict[str, int], dict[str, str]]:
+) -> tuple[dict[int, CollectionTask], dict[str, str]]:
     datasource_ids = [datasource.id for datasource in datasources]
     sources = [datasource.source for datasource in datasources]
 
@@ -185,10 +157,8 @@ async def _load_datasource_list_context(
     if stale_datasource_ids:
         running_tasks = await _load_latest_running_tasks(db, datasource_ids)
 
-    completed_tasks = await _load_latest_completed_tasks(db, datasource_ids)
-    data_counts = await _load_datasource_data_counts(db, sources)
     endpoint_overrides = await _load_datasource_endpoint_overrides(db, sources)
-    return running_tasks, completed_tasks, data_counts, endpoint_overrides
+    return running_tasks, endpoint_overrides
 
 
 async def get_datasource_record(db: AsyncSession, source_id: str) -> Optional[DataSource]:
@@ -401,26 +371,19 @@ async def list_datasources(
 
     collector_list = []
     config = get_data_sources_config()
-    running_tasks, completed_tasks, data_counts, endpoint_overrides = await _load_datasource_list_context(
-        db,
-        datasources,
-    )
+    running_tasks, endpoint_overrides = await _load_datasource_list_context(db, datasources)
     for datasource in datasources:
         running_task = running_tasks.get(datasource.id)
-        last_task = completed_tasks.get(datasource.id)
-        endpoint = endpoint_overrides.get(datasource.source) or config.get_yaml_url(
-            datasource.source,
-        )
-        data_count = data_counts.get(datasource.source, 0)
-
-        last_run_at = datasource.last_run_at or (last_task.completed_at if last_task else None)
-        last_run = to_iso8601_utc(last_run_at)
-        last_status = datasource.last_status or (last_task.status if last_task else None)
+        endpoint = endpoint_overrides.get(datasource.source) or config.get_yaml_url(datasource.source)
+        last_run_at = datasource.last_run_at
+        last_status = datasource.last_status
 
         collector_list.append(
             {
                 "id": datasource.id,
+                "source": datasource.source,
                 "name": datasource.name,
+                **datasource_metadata(datasource.source),
                 "module": datasource.module,
                 "priority": datasource.priority,
                 "frequency": format_frequency_label(datasource.frequency_minutes),
@@ -428,11 +391,9 @@ async def list_datasources(
                 "is_active": datasource.is_active,
                 "collector_class": datasource.collector_class,
                 "endpoint": endpoint,
-                "last_run": last_run,
+                "last_run": to_iso8601_utc(last_run_at),
                 "last_run_at": to_iso8601_utc(last_run_at),
                 "last_status": last_status,
-                "last_records_processed": last_task.records_processed if last_task else None,
-                "data_count": data_count,
                 "is_running": running_task is not None,
                 "task_id": running_task.id if running_task else None,
                 "progress": running_task.progress if running_task else None,
@@ -575,6 +536,7 @@ async def get_datasource(
     return {
         "id": datasource.id,
         "name": datasource.name,
+        **datasource_metadata(datasource.source),
         "module": datasource.module,
         "priority": datasource.priority,
         "frequency": format_frequency_label(datasource.frequency_minutes),

backend/app/api/v1/settings.py

@@ -9,10 +9,36 @@ from sqlalchemy.ext.asyncio import AsyncSession
 
 from app.core.security import get_current_user
 from app.core.time import to_iso8601_utc
+from app.core.config import settings as app_settings
+from app.core.data_sources import get_data_sources_config
+from app.core.datasource_defaults import DEFAULT_DATASOURCES
 from app.db.session import get_db
 from app.models.datasource import DataSource
+from app.models.datasource_config import DataSourceConfig
 from app.models.system_setting import SystemSetting
 from app.models.user import User
+from app.services.barentswatch import (
+    BarentsWatchConfig,
+    check_barentswatch_config,
+    check_barentswatch_connectivity,
+    get_barentswatch_datasource_record,
+    resolve_barentswatch_config,
+)
+from app.services.credential_guides import (
+    generate_credential_guide,
+    get_credential_guide,
+    reset_credential_guide,
+)
+from app.services.datasource_connectivity import (
+    build_builtin_connectivity_checksum,
+    save_connectivity_success,
+)
+from app.services.ai_client import AIProviderClient, get_ai_provider_client
+from app.services.llm_provider_catalog import (
+    get_fallback_llm_provider_preset,
+    list_fallback_llm_provider_presets,
+    refresh_llm_provider_preset,
+)
 from app.services.scheduler import sync_datasource_job
 from app.services.tv_streams import DEFAULT_TV_SETTINGS, get_tv_settings_payload, normalize_tv_settings
 
@@ -39,6 +65,21 @@ DEFAULT_SETTINGS = {
         "password_policy": "medium",
     },
     "tv": DEFAULT_TV_SETTINGS,
+    "external_integrations": {
+        "ai_provider": {
+            "service_url": "",
+            "service_token": "",
+            "provider": "minimax",
+            "provider_api": "anthropic-messages",
+            "base_url": "https://api.minimaxi.com/anthropic",
+            "model": "MiniMax-M2.7",
+            "api_key": "",
+            "max_tokens": 1200,
+            "anthropic_version": "2023-06-01",
+            "timeout_seconds": 60,
+            "retry_attempts": 2,
+        }
+    },
 }
 
 
@@ -96,6 +137,34 @@ class TVSettingsUpdate(BaseModel):
     sources: list[TVStreamSourceUpdate] = Field(default_factory=list)
 
 
+class AIProviderIntegrationUpdate(BaseModel):
+    service_url: str = ""
+    service_token: Optional[str] = None
+    provider: str = Field(default="minimax", max_length=80)
+    provider_api: str = Field(default="anthropic-messages", max_length=80)
+    base_url: str = Field(default="", max_length=500)
+    model: str = Field(default="", max_length=200)
+    api_key: Optional[str] = None
+    max_tokens: int = Field(default=1200, ge=1, le=200000)
+    anthropic_version: str = Field(default="2023-06-01", max_length=40)
+    timeout_seconds: int = Field(default=60, ge=5, le=600)
+    retry_attempts: int = Field(default=2, ge=1, le=10)
+    clear_service_token: bool = False
+    clear_api_key: bool = False
+
+
+class BarentsWatchIntegrationUpdate(BaseModel):
+    endpoint: str = ""
+    client_id: str = ""
+    client_secret: Optional[str] = None
+    clear_client_secret: bool = False
+
+
+class ExternalIntegrationsUpdate(BaseModel):
+    ai_provider: AIProviderIntegrationUpdate
+    barentswatch: BarentsWatchIntegrationUpdate
+
+
 def merge_with_defaults(category: str, payload: Optional[dict]) -> dict:
     merged = deepcopy(DEFAULT_SETTINGS[category])
     if payload:
@@ -146,6 +215,151 @@ async def save_setting_payload(db: AsyncSession, category: str, payload: dict) -
     return merge_with_defaults(category, record.payload)
 
 
+def _mask_secret(value: Optional[str]) -> dict:
+    if not value:
+        return {"configured": False, "preview": ""}
+    text = str(value)
+    if "-" in text:
+        prefix = text.split("-", 1)[0] + "-"
+        preview = prefix + ("*" * max(len(text) - len(prefix), 1))
+    else:
+        prefix_len = min(4, len(text))
+        preview = text[:prefix_len] + ("*" * max(len(text) - prefix_len, 1))
+    return {"configured": True, "preview": preview}
+
+
+async def get_runtime_ai_provider_config(db: AsyncSession) -> dict:
+    runtime_record = await get_setting_record(db, "external_integrations")
+    payload = merge_with_defaults(
+        "external_integrations",
+        runtime_record.payload if runtime_record else None,
+    )
+    ai_payload = payload.get("ai_provider") or {}
+    has_runtime_llm_config = bool(
+        runtime_record
+        and isinstance(runtime_record.payload, dict)
+        and isinstance(runtime_record.payload.get("ai_provider"), dict)
+    )
+    return {
+        "service_url": ai_payload.get("service_url") or app_settings.AI_PROVIDER_SERVICE_URL,
+        "service_token": ai_payload.get("service_token") or app_settings.AI_PROVIDER_SERVICE_TOKEN,
+        "timeout_seconds": int(
+            ai_payload.get("timeout_seconds") or app_settings.AI_PROVIDER_TIMEOUT_SECONDS
+        ),
+        "retry_attempts": int(
+            ai_payload.get("retry_attempts") or app_settings.AI_PROVIDER_RETRY_ATTEMPTS
+        ),
+        "llm_config": {
+            "provider": ai_payload.get("provider") or "minimax",
+            "provider_api": ai_payload.get("provider_api") or "anthropic-messages",
+            "base_url": ai_payload.get("base_url") or "https://api.minimaxi.com/anthropic",
+            "model": ai_payload.get("model") or "MiniMax-M2.7",
+            "api_key": ai_payload.get("api_key") or "",
+            "max_tokens": int(ai_payload.get("max_tokens") or 1200),
+            "anthropic_version": ai_payload.get("anthropic_version") or "2023-06-01",
+        } if has_runtime_llm_config else {},
+    }
+
+
+async def get_barentswatch_config_record(db: AsyncSession) -> Optional[DataSourceConfig]:
+    return await get_barentswatch_datasource_record(db)
+
+
+async def serialize_external_integrations(db: AsyncSession) -> dict:
+    ai_config = await get_runtime_ai_provider_config(db)
+    runtime_setting = await get_setting_record(db, "external_integrations")
+    display_llm_config = ai_config["llm_config"] or DEFAULT_SETTINGS["external_integrations"]["ai_provider"]
+    barentswatch_record = await get_barentswatch_config_record(db)
+    barentswatch_auth = barentswatch_record.auth_config if barentswatch_record else {}
+    barentswatch_auth = barentswatch_auth or {}
+    resolved_barentswatch = await resolve_barentswatch_config(db)
+    return {
+        "ai_provider": {
+            "service_url": ai_config["service_url"],
+            "service_token": _mask_secret(ai_config["service_token"]),
+            "provider": display_llm_config.get("provider") or "minimax",
+            "provider_api": display_llm_config.get("provider_api") or "anthropic-messages",
+            "base_url": display_llm_config.get("base_url") or "https://api.minimaxi.com/anthropic",
+            "model": display_llm_config.get("model") or "MiniMax-M2.7",
+            "api_key": _mask_secret(display_llm_config.get("api_key")),
+            "max_tokens": int(display_llm_config.get("max_tokens") or 1200),
+            "anthropic_version": display_llm_config.get("anthropic_version") or "2023-06-01",
+            "timeout_seconds": ai_config["timeout_seconds"],
+            "retry_attempts": ai_config["retry_attempts"],
+            "source": "runtime" if runtime_setting else "env",
+        },
+        "barentswatch": {
+            "endpoint": resolved_barentswatch.endpoint,
+            "client_id": barentswatch_auth.get("client_id") or resolved_barentswatch.client_id,
+            "client_secret": _mask_secret(
+                barentswatch_auth.get("client_secret") or resolved_barentswatch.client_secret
+            ),
+            "source": resolved_barentswatch.credential_source,
+        },
+    }
+
+
+async def save_external_integrations_payload(
+    db: AsyncSession,
+    update: ExternalIntegrationsUpdate,
+) -> dict:
+    current_payload = await get_setting_payload(db, "external_integrations")
+    current_ai = current_payload.get("ai_provider") or {}
+    ai_payload = {
+        "service_url": update.ai_provider.service_url.strip()
+        or app_settings.AI_PROVIDER_SERVICE_URL,
+        "service_token": current_ai.get("service_token") or "",
+        "provider": update.ai_provider.provider.strip() or "minimax",
+        "provider_api": update.ai_provider.provider_api.strip() or "anthropic-messages",
+        "base_url": update.ai_provider.base_url.strip(),
+        "model": update.ai_provider.model.strip(),
+        "api_key": current_ai.get("api_key") or "",
+        "max_tokens": update.ai_provider.max_tokens,
+        "anthropic_version": update.ai_provider.anthropic_version.strip() or "2023-06-01",
+        "timeout_seconds": update.ai_provider.timeout_seconds,
+        "retry_attempts": update.ai_provider.retry_attempts,
+    }
+    if update.ai_provider.clear_service_token:
+        ai_payload["service_token"] = ""
+    elif update.ai_provider.service_token not in (None, ""):
+        ai_payload["service_token"] = update.ai_provider.service_token
+    if update.ai_provider.clear_api_key:
+        ai_payload["api_key"] = ""
+    elif update.ai_provider.api_key not in (None, ""):
+        ai_payload["api_key"] = update.ai_provider.api_key
+
+    await save_setting_payload(db, "external_integrations", {"ai_provider": ai_payload})
+
+    default_endpoint = get_data_sources_config().get_yaml_url("barentswatch_vessels")
+    barentswatch_record = await get_barentswatch_config_record(db)
+    if barentswatch_record is None:
+        barentswatch_record = DataSourceConfig(
+            name="barentswatch_vessels",
+            description="BarentsWatch Live AIS credentials",
+            source_type="api",
+            endpoint=update.barentswatch.endpoint.strip() or default_endpoint,
+            auth_type="oauth_client",
+            auth_config={},
+            headers={},
+            config={},
+            is_active=True,
+        )
+        db.add(barentswatch_record)
+
+    current_auth = dict(barentswatch_record.auth_config or {})
+    if update.barentswatch.clear_client_secret:
+        current_auth.pop("client_secret", None)
+    elif update.barentswatch.client_secret not in (None, ""):
+        current_auth["client_secret"] = update.barentswatch.client_secret
+    current_auth["client_id"] = update.barentswatch.client_id.strip()
+    barentswatch_record.endpoint = update.barentswatch.endpoint.strip() or default_endpoint
+    barentswatch_record.auth_type = "oauth_client"
+    barentswatch_record.auth_config = current_auth
+    await db.commit()
+
+    return await serialize_external_integrations(db)
+
+
 def format_frequency_label(minutes: int) -> str:
     if minutes % 1440 == 0:
         return f"{minutes // 1440}d"
@@ -155,9 +369,11 @@ def format_frequency_label(minutes: int) -> str:
 
 
 def serialize_collector(datasource: DataSource) -> dict:
+    defaults = DEFAULT_DATASOURCES.get(datasource.source, {})
     return {
         "id": datasource.id,
         "name": datasource.name,
+        "display_name": defaults.get("display_name") or datasource.name,
         "source": datasource.source,
         "module": datasource.module,
         "priority": datasource.priority,
@@ -167,6 +383,10 @@ def serialize_collector(datasource: DataSource) -> dict:
         "last_run_at": to_iso8601_utc(datasource.last_run_at),
         "last_status": datasource.last_status,
         "next_run_at": to_iso8601_utc(datasource.next_run_at),
+        "is_free": bool(defaults.get("is_free", True)),
+        "requires_credentials": bool(defaults.get("requires_credentials", False)),
+        "credential_provider": defaults.get("credential_provider"),
+        "credential_status": defaults.get("credential_status", "none"),
     }
 
 
@@ -243,6 +463,135 @@ async def update_tv_settings(
     return {"status": "updated", "tv": normalize_tv_settings(saved)}
 
 
+@router.get("/integrations")
+async def get_external_integrations(
+    current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    return {"integrations": await serialize_external_integrations(db)}
+
+
+@router.get("/integrations/barentswatch/connectivity")
+async def get_barentswatch_connectivity(
+    current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    return await check_barentswatch_connectivity(db)
+
+
+@router.post("/integrations/barentswatch/connect")
+async def connect_barentswatch_integration(
+    payload: BarentsWatchIntegrationUpdate,
+    current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    current = await resolve_barentswatch_config(db)
+    config = BarentsWatchConfig(
+        endpoint=payload.endpoint.strip() or current.endpoint,
+        client_id=payload.client_id.strip() or current.client_id,
+        client_secret=(
+            ""
+            if payload.clear_client_secret
+            else payload.client_secret or current.client_secret
+        ),
+        credential_source="draft",
+        endpoint_source="draft",
+    )
+    result = await check_barentswatch_config(config)
+    if result.get("success"):
+        checksum, _context = await build_builtin_connectivity_checksum(
+            "barentswatch_vessels",
+            config.endpoint,
+            "none",
+            {},
+            {},
+            db,
+            credential_override={
+                "client_id": config.client_id,
+                "client_secret": config.client_secret,
+            },
+        )
+        validation = await save_connectivity_success(
+            db,
+            "barentswatch_vessels",
+            checksum,
+            result,
+            connected_by="connection_button",
+        )
+        await db.commit()
+        return {**result, "connected": True, "validation": validation}
+    return {**result, "connected": False}
+
+
+@router.get("/credential-guides/{provider}")
+async def read_credential_guide(
+    provider: str,
+    current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    try:
+        return {"guide": await get_credential_guide(db, provider)}
+    except ValueError as exc:
+        raise HTTPException(status_code=404, detail=str(exc)) from exc
+
+
+@router.post("/credential-guides/{provider}/generate")
+async def generate_provider_credential_guide(
+    provider: str,
+    current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+    ai_client: AIProviderClient = Depends(get_ai_provider_client),
+):
+    try:
+        return {"guide": await generate_credential_guide(db, provider, ai_client)}
+    except ValueError as exc:
+        raise HTTPException(status_code=404, detail=str(exc)) from exc
+
+
+@router.post("/credential-guides/{provider}/reset")
+async def reset_provider_credential_guide(
+    provider: str,
+    current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    try:
+        return {"guide": await reset_credential_guide(db, provider)}
+    except ValueError as exc:
+        raise HTTPException(status_code=404, detail=str(exc)) from exc
+
+
+@router.get("/integrations/ai-provider/presets")
+async def get_ai_provider_presets(
+    current_user: User = Depends(get_current_user),
+):
+    return {"data": list_fallback_llm_provider_presets()}
+
+
+@router.post("/integrations/ai-provider/presets/{provider}/refresh")
+async def refresh_ai_provider_preset(
+    provider: str,
+    current_user: User = Depends(get_current_user),
+):
+    try:
+        return {"data": await refresh_llm_provider_preset(provider)}
+    except ValueError as exc:
+        raise HTTPException(status_code=404, detail=str(exc)) from exc
+    except Exception as exc:
+        fallback = get_fallback_llm_provider_preset(provider)
+        fallback["refresh_error"] = str(exc)
+        return {"data": fallback}
+
+
+@router.put("/integrations")
+async def update_external_integrations(
+    payload: ExternalIntegrationsUpdate,
+    current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    saved = await save_external_integrations_payload(db, payload)
+    return {"status": "updated", "integrations": saved}
+
+
 @router.get("/collectors")
 async def get_collector_settings(
     current_user: User = Depends(get_current_user),
@@ -289,6 +638,7 @@ async def get_all_settings(
         "notifications": setting_payloads["notifications"],
         "security": setting_payloads["security"],
         "tv": await get_tv_settings_payload(db),
+        "integrations": await serialize_external_integrations(db),
         "collectors": [serialize_collector(datasource) for datasource in datasources],
         "generated_at": to_iso8601_utc(datetime.now(UTC)),
     }

backend/app/api/v1/system_control.py

@@ -4,12 +4,15 @@ import os
 import subprocess
 import sys
 
-from fastapi import APIRouter, Depends, HTTPException, status
+from datetime import datetime
+
+from fastapi import APIRouter, Depends, HTTPException, Query, Request, status
 from pydantic import BaseModel
 
 from app.core.config import ROOT_DIR
 from app.core.security import get_current_user
 from app.models.user import User
+from app.services.persistent_logs import record_audit_log, record_system_log
 from app.services.system_control import (
     build_task_id,
     clear_active_task_id,
@@ -23,6 +26,15 @@ from app.services.system_control import (
     set_active_task_id,
     upsert_task_state,
 )
+from app.services.system_logs import (
+    DEFAULT_LOG_LINE_LIMIT,
+    MAX_LOG_LINE_LIMIT,
+    SUPPORTED_LOG_LEVELS,
+    append_buffer_log,
+    list_log_sources,
+    normalize_log_level,
+    read_log_snapshot,
+)
 
 router = APIRouter()
 
@@ -47,6 +59,59 @@ class RestartTaskLogsResponse(BaseModel):
     lines: list[str]
 
 
+class SystemLogSourceSummary(BaseModel):
+    source_id: str
+    name: str
+    kind: str
+    location: str
+    description: str
+    category: str
+    status: str
+
+
+class SystemLogSourcesResponse(BaseModel):
+    items: list[SystemLogSourceSummary]
+
+
+class SystemLogDailyMarker(BaseModel):
+    date_token: str
+    total: int
+    dominant_level: str
+
+
+class SystemLogSnapshotResponse(BaseModel):
+    source_id: str
+    name: str
+    kind: str
+    location: str
+    description: str
+    category: str
+    status: str
+    level: str
+    selected_levels: list[str] = []
+    search_query: str = ""
+    available_levels: list[str]
+    daily_markers: list[SystemLogDailyMarker] = []
+    line_limit: int
+    line_count: int
+    lines: list[str]
+
+
+class EarthClientLogEventCreate(BaseModel):
+    level: str = "error"
+    message: str
+    category: str | None = None
+    url: str | None = None
+    module: str | None = None
+    detail: str | None = None
+
+
+class EarthClientLogEventResponse(BaseModel):
+    accepted: bool
+    source_id: str
+    level: str
+
+
 def ensure_super_admin(current_user: User) -> None:
     if not require_super_admin(current_user.role):
         raise HTTPException(
@@ -55,9 +120,22 @@ def ensure_super_admin(current_user: User) -> None:
         )
 
 
+def validate_log_date(raw_value: str | None, field_name: str) -> str | None:
+    if raw_value in {None, ""}:
+        return None
+    try:
+        return datetime.strptime(raw_value, "%Y-%m-%d").date().isoformat()
+    except ValueError as exc:
+        raise HTTPException(
+            status_code=status.HTTP_400_BAD_REQUEST,
+            detail=f"{field_name} must be in YYYY-MM-DD format",
+        ) from exc
+
+
 @router.post("/restart-tasks", response_model=RestartTaskResponse)
 async def create_restart_task(
     payload: RestartTaskCreate,
+    request: Request,
     current_user: User = Depends(get_current_user),
 ):
     ensure_super_admin(current_user)
@@ -133,11 +211,31 @@ async def create_restart_task(
             requested_by=requested_by,
         )
         clear_active_task_id(task_id)
+        await record_audit_log(
+            action="system.restart_task.requested",
+            actor_id=current_user.id,
+            actor_name=current_user.username,
+            target_type="restart_task",
+            target_id=task_id,
+            result="failed",
+            ip=request.client.host if request.client else None,
+            details={"action": payload.action, "message": task_state["message"]},
+        )
         raise HTTPException(
             status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
             detail=task_state["message"],
         ) from exc
 
+    await record_audit_log(
+        action="system.restart_task.requested",
+        actor_id=current_user.id,
+        actor_name=current_user.username,
+        target_type="restart_task",
+        target_id=task_id,
+        result="accepted",
+        ip=request.client.host if request.client else None,
+        details={"action": payload.action},
+    )
     return task_state
 
 
@@ -165,3 +263,92 @@ async def get_restart_task_logs(
     if task is None:
         raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Restart task not found")
     return {"task_id": task_id, "lines": get_task_logs(task_id)}
+
+
+@router.get("/logs/sources", response_model=SystemLogSourcesResponse)
+async def get_system_log_sources(
+    current_user: User = Depends(get_current_user),
+):
+    ensure_super_admin(current_user)
+    return {"items": list_log_sources()}
+
+
+@router.get("/logs/{source_id}", response_model=SystemLogSnapshotResponse)
+async def get_system_log_snapshot(
+    source_id: str,
+    limit: int = DEFAULT_LOG_LINE_LIMIT,
+    level: str = "all",
+    levels: str | None = Query(None, description="Comma-separated log levels"),
+    start_date: str | None = Query(None, description="Filter logs from this date (YYYY-MM-DD)"),
+    end_date: str | None = Query(None, description="Filter logs until this date (YYYY-MM-DD)"),
+    search: str | None = Query(None, description="Case-insensitive substring search"),
+    current_user: User = Depends(get_current_user),
+):
+    ensure_super_admin(current_user)
+
+    if limit < 1 or limit > MAX_LOG_LINE_LIMIT:
+        raise HTTPException(
+            status_code=status.HTTP_400_BAD_REQUEST,
+            detail=f"limit must be between 1 and {MAX_LOG_LINE_LIMIT}",
+        )
+    if str(level).strip().lower() not in SUPPORTED_LOG_LEVELS and normalize_log_level(level) == "all" and str(level).strip().lower() not in {"", "all"}:
+        raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail="Unsupported log level")
+    if levels:
+        for raw_level in str(levels).split(","):
+            normalized_level = str(raw_level).strip().lower()
+            if not normalized_level:
+                continue
+            if normalized_level not in SUPPORTED_LOG_LEVELS and normalize_log_level(normalized_level) == "all":
+                raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail="Unsupported log level")
+    normalized_start_date = validate_log_date(start_date, "start_date")
+    normalized_end_date = validate_log_date(end_date, "end_date")
+    if normalized_start_date and normalized_end_date and normalized_start_date > normalized_end_date:
+        raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail="start_date must be earlier than or equal to end_date")
+
+    snapshot = read_log_snapshot(
+        source_id,
+        limit,
+        level=level,
+        levels=levels,
+        start_date=normalized_start_date,
+        end_date=normalized_end_date,
+        search=search,
+    )
+    if snapshot is None:
+        raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Log source not found")
+    return snapshot
+
+
+@router.post("/logs/earth-client", response_model=EarthClientLogEventResponse)
+async def ingest_earth_client_log(
+    payload: EarthClientLogEventCreate,
+    request: Request,
+):
+    normalized_level = normalize_log_level(payload.level)
+    append_buffer_log(
+        "earth-client",
+        level=normalized_level,
+        message=payload.message,
+        context={
+            "category": payload.category or "",
+            "url": payload.url or "",
+            "module": payload.module or "",
+            "detail": payload.detail or "",
+        },
+    )
+    await record_system_log(
+        source="earth-client",
+        service="earth",
+        module=payload.module or "earth-client",
+        event="earth.client.runtime_log",
+        level=normalized_level,
+        message=payload.message,
+        category=payload.category or "client-runtime",
+        context={
+            "url": payload.url or "",
+            "detail": payload.detail or "",
+            "module": payload.module or "",
+            "client_ip": request.client.host if request.client else "",
+        },
+    )
+    return {"accepted": True, "source_id": "earth-client", "level": normalized_level}

backend/app/api/v1/visualization.py

@@ -4,25 +4,34 @@ Unified API for all visualization data sources.
 Returns GeoJSON format compatible with Three.js, CesiumJS, and Unreal Cesium.
 """
 
-from datetime import UTC, datetime
+from datetime import UTC, datetime, timedelta
 import math
-from fastapi import APIRouter, HTTPException, Depends, Query
+import httpx
+from fastapi import APIRouter, HTTPException, Depends, Query, Response
 from sqlalchemy.ext.asyncio import AsyncSession
 from sqlalchemy import select, func
 from typing import List, Dict, Any, Optional
 
 from app.core.collected_data_fields import get_record_field
+from app.core.countries import get_country_centroid
 from app.core.satellite_tle import build_tle_lines_from_elements
 from app.core.time import to_iso8601_utc
 from app.db.session import get_db
 from app.models.bgp_anomaly import BGPAnomaly
 from app.models.bgp_incident import BGPIncident
 from app.models.collected_data import CollectedData
+from app.models.vessel import VesselPosition, VesselStatic
 from app.services.bgp_collectors import build_bgp_collector_coverage
 from app.services.cable_graph import build_graph_from_data, CableGraph, haversine_distance
 from app.services.collectors.bgp_common import RIPE_RIS_COLLECTOR_COORDS
+from app.services.persistent_logs import record_system_log
+from app.core.logging import get_logger
 
 router = APIRouter()
+logger = get_logger(__name__, service="api")
+TERRAIN_TILE_URL_TEMPLATE = (
+    "https://s3.amazonaws.com/elevation-tiles-prod/terrarium/{z}/{x}/{y}.png"
+)
 
 
 # ============== Converter Functions ==============
@@ -176,6 +185,12 @@ def convert_satellite_to_geojson(records: List[CollectedData]) -> Dict[str, Any]
                 mean_motion=metadata.get("mean_motion"),
             )
 
+        constellation_group = _normalize_satellite_constellation_group(
+            metadata.get("constellation_group"),
+            record.name,
+        )
+        footprint_policy = _get_satellite_footprint_policy(constellation_group)
+
         features.append(
             {
                 "type": "Feature",
@@ -185,6 +200,8 @@ def convert_satellite_to_geojson(records: List[CollectedData]) -> Dict[str, Any]
                     "id": record.id,
                     "norad_cat_id": norad_id,
                     "name": record.name,
+                    "constellation_group": constellation_group,
+                    "footprint_policy": footprint_policy,
                     "international_designator": metadata.get("international_designator"),
                     "epoch": metadata.get("epoch"),
                     "inclination": metadata.get("inclination"),
@@ -205,6 +222,31 @@ def convert_satellite_to_geojson(records: List[CollectedData]) -> Dict[str, Any]
     return {"type": "FeatureCollection", "features": features}
 
 
+def _normalize_satellite_constellation_group(
+    raw_group: Any,
+    name: Optional[str],
+) -> Optional[str]:
+    normalized_group = str(raw_group or "").strip().lower()
+    if normalized_group:
+        return normalized_group
+
+    normalized_name = str(name or "").strip().upper()
+    if normalized_name.startswith("STARLINK"):
+        return "starlink"
+    if normalized_name.startswith("IRIDIUM"):
+        return "iridium-next"
+
+    return None
+
+
+def _get_satellite_footprint_policy(constellation_group: Optional[str]) -> str:
+    if constellation_group == "starlink":
+        return "starlink_ground_footprint"
+    if constellation_group == "iridium-next":
+        return "iridium_coverage_ring"
+    return "none"
+
+
 def _current_collected_data_stmt(source: str):
     return (
         select(CollectedData)
@@ -359,6 +401,317 @@ def convert_gpu_cluster_to_geojson(records: List[CollectedData]) -> Dict[str, An
     return {"type": "FeatureCollection", "features": features}
 
 
+def _parse_float(value: Any) -> Optional[float]:
+    try:
+        if value in (None, ""):
+            return None
+        return float(value)
+    except (TypeError, ValueError):
+        return None
+
+
+COMPUTE_CENTER_COORDINATE_HINTS = (
+    ("el capitan", 37.6819, -121.7681),
+    ("livermore", 37.6819, -121.7681),
+    ("llnl", 37.6819, -121.7681),
+    ("lawrence livermore", 37.6819, -121.7681),
+    ("frontier", 35.9319, -84.3107),
+    ("oak ridge", 35.9319, -84.3107),
+    ("ornl", 35.9319, -84.3107),
+    ("aurora", 41.7130, -87.9820),
+    ("argonne", 41.7130, -87.9820),
+    ("anl", 41.7130, -87.9820),
+    ("fugaku", 34.6953, 135.1974),
+    ("kobe", 34.6953, 135.1974),
+    ("riken", 34.6953, 135.1974),
+    ("summit", 35.9319, -84.3107),
+    ("leonardo", 44.4949, 11.3426),
+    ("bologna", 44.4949, 11.3426),
+    ("alps", 46.0037, 8.9511),
+    ("lugano", 46.0037, 8.9511),
+    ("sunway taihulight", 31.4912, 120.3119),
+    ("wuxi", 31.4912, 120.3119),
+    ("tianhe-2", 23.1291, 113.2644),
+    ("tianhe-2a", 23.1291, 113.2644),
+    ("guangzhou", 23.1291, 113.2644),
+    ("colossus", 35.1495, -90.0490),
+    ("memphis", 35.1495, -90.0490),
+    ("xai", 35.1495, -90.0490),
+)
+
+
+def _normalize_hint_text(*parts: Any) -> str:
+    return " ".join(
+        str(part).strip().lower()
+        for part in parts
+        if part not in (None, "")
+    )
+
+
+def _resolve_compute_center_coordinates(
+    record: CollectedData,
+    metadata: Dict[str, Any],
+) -> Dict[str, Any]:
+    latitude = _parse_float(get_record_field(record, "latitude"))
+    longitude = _parse_float(get_record_field(record, "longitude"))
+    if latitude not in (None, 0.0) and longitude not in (None, 0.0):
+        return {
+            "latitude": latitude,
+            "longitude": longitude,
+            "location_precision": "precise",
+            "geography_mode": "source_coordinates",
+            "is_estimated": False,
+            "estimated_reason": None,
+        }
+
+    hint_text = _normalize_hint_text(
+        record.name,
+        get_record_field(record, "city"),
+        get_record_field(record, "country"),
+        metadata.get("site"),
+        metadata.get("organization"),
+        metadata.get("operator"),
+    )
+    for needle, resolved_latitude, resolved_longitude in COMPUTE_CENTER_COORDINATE_HINTS:
+        if needle in hint_text:
+            return {
+                "latitude": resolved_latitude,
+                "longitude": resolved_longitude,
+                "location_precision": "estimated_site",
+                "geography_mode": "site_hint",
+                "is_estimated": True,
+                "estimated_reason": f"Matched known site hint: {needle}",
+            }
+
+    centroid = get_country_centroid(get_record_field(record, "country"))
+    if centroid:
+        return {
+            "latitude": centroid.get("latitude"),
+            "longitude": centroid.get("longitude"),
+            "location_precision": "estimated_country",
+            "geography_mode": "country_centroid",
+            "is_estimated": True,
+            "estimated_reason": "Estimated from country centroid",
+        }
+
+    return {
+        "latitude": latitude,
+        "longitude": longitude,
+        "location_precision": "unknown",
+        "geography_mode": "unknown",
+        "is_estimated": True,
+        "estimated_reason": "No resolvable location hints",
+    }
+
+
+def _normalize_capacity_band(capacity_value: Optional[float], capacity_unit: str) -> str:
+    if capacity_value is None:
+        return "unknown"
+
+    unit = str(capacity_unit or "").strip().lower()
+    if unit in {"pflop/s", "pflops", "pflop"}:
+        normalized_tflops = capacity_value * 1000
+    elif unit in {"gflop/s", "gflops", "gflop"}:
+        normalized_tflops = capacity_value
+    else:
+        normalized_tflops = capacity_value
+
+    if normalized_tflops >= 1_000_000:
+        return "exascale"
+    if normalized_tflops >= 100_000:
+        return "ultra"
+    if normalized_tflops >= 10_000:
+        return "large"
+    if normalized_tflops > 0:
+        return "regional"
+    return "unknown"
+
+
+def convert_compute_centers_to_geojson(records: List[CollectedData]) -> Dict[str, Any]:
+    """Convert compute infrastructure records into a unified GeoJSON layer."""
+    features = []
+
+    for record in records:
+        metadata = record.extra_data or {}
+        coordinate_info = _resolve_compute_center_coordinates(record, metadata)
+        latitude = coordinate_info.get("latitude")
+        longitude = coordinate_info.get("longitude")
+        site_type = (
+            "supercomputer"
+            if record.source == "top500" or record.data_type == "supercomputer"
+            else "gpu_cluster"
+        )
+        if latitude in (None, 0.0) or longitude in (None, 0.0):
+            continue
+
+        if site_type == "supercomputer":
+            capacity_value = _parse_float(get_record_field(record, "rmax"))
+            capacity_unit = "GFlops"
+        else:
+            capacity_value = _parse_float(get_record_field(record, "value"))
+            capacity_unit = str(get_record_field(record, "unit") or "TFlop/s")
+
+        vendor = (
+            metadata.get("manufacturer")
+            or metadata.get("vendor")
+            or metadata.get("gpu_type")
+        )
+        operator = (
+            metadata.get("organization")
+            or metadata.get("operator")
+            or metadata.get("owner")
+        )
+        rank = metadata.get("rank")
+        if rank in (None, "") and site_type == "supercomputer":
+            rank = get_record_field(record, "rank")
+
+        updated_at = to_iso8601_utc(record.reference_date or record.collected_at)
+
+        features.append(
+            {
+                "type": "Feature",
+                "id": record.id,
+                "geometry": {
+                    "type": "Point",
+                    "coordinates": [longitude or 0, latitude or 0],
+                },
+                "properties": {
+                    "id": record.id,
+                    "source_id": record.source_id,
+                    "name": record.name,
+                    "site_type": site_type,
+                    "country": get_record_field(record, "country"),
+                    "city": get_record_field(record, "city"),
+                    "latitude": latitude,
+                    "longitude": longitude,
+                    "operator": operator,
+                    "vendor": vendor,
+                    "capacity_value": capacity_value,
+                    "capacity_unit": capacity_unit,
+                    "capacity_band": _normalize_capacity_band(capacity_value, capacity_unit),
+                    "rank": rank,
+                    "gpu_count": metadata.get("gpu_count"),
+                    "gpu_type": metadata.get("gpu_type"),
+                    "cores": get_record_field(record, "cores"),
+                    "power": get_record_field(record, "power"),
+                    "source": record.source,
+                    "updated_at": updated_at,
+                    "status": "observed",
+                    "location_precision": coordinate_info.get("location_precision"),
+                    "geography_mode": coordinate_info.get("geography_mode"),
+                    "is_estimated": coordinate_info.get("is_estimated", False),
+                    "estimated_reason": coordinate_info.get("estimated_reason"),
+                    "data_type": "compute_center",
+                    "metadata": metadata,
+                },
+            }
+        )
+
+    return {"type": "FeatureCollection", "features": features}
+
+
+VESSEL_TYPE_FILTERS = {
+    "cargo": lambda props: str(props.get("vessel_type_name", "")).lower() == "cargo"
+    or 70 <= int(props.get("vessel_type") or -1) <= 79,
+    "tanker": lambda props: str(props.get("vessel_type_name", "")).lower() == "tanker"
+    or 80 <= int(props.get("vessel_type") or -1) <= 89,
+    "passenger": lambda props: str(props.get("vessel_type_name", "")).lower() == "passenger"
+    or 60 <= int(props.get("vessel_type") or -1) <= 69,
+    "fishing": lambda props: str(props.get("vessel_type_name", "")).lower() == "fishing"
+    or int(props.get("vessel_type") or -1) == 30,
+    "military": lambda props: str(props.get("vessel_type_name", "")).lower() == "military"
+    or int(props.get("vessel_type") or -1) == 35,
+    "other": lambda props: str(props.get("vessel_type_name", "")).lower()
+    not in {"cargo", "tanker", "passenger", "fishing", "military"},
+}
+
+
+def convert_vessels_to_geojson(rows: List[Any]) -> Dict[str, Any]:
+    features = []
+    for position, static in rows:
+        if position.lat is None or position.lon is None:
+            continue
+        props = {
+            "mmsi": position.mmsi,
+            "name": getattr(static, "name", None) or f"MMSI {position.mmsi}",
+            "callsign": getattr(static, "callsign", None),
+            "imo": getattr(static, "imo", None),
+            "vessel_type": getattr(static, "vessel_type", None),
+            "vessel_type_name": getattr(static, "vessel_type_name", None) or "Other",
+            "flag": getattr(static, "flag", None),
+            "length": getattr(static, "length", None),
+            "width": getattr(static, "width", None),
+            "draught": getattr(static, "draught", None),
+            "sog": position.sog,
+            "cog": position.cog,
+            "heading": position.heading,
+            "nav_status": position.nav_status,
+            "received_at": to_iso8601_utc(position.received_at),
+            "data_type": "vessel",
+        }
+        features.append(
+            {
+                "type": "Feature",
+                "id": position.mmsi,
+                "geometry": {
+                    "type": "Point",
+                    "coordinates": [position.lon, position.lat],
+                },
+                "properties": props,
+            }
+        )
+
+    return {"type": "FeatureCollection", "features": features}
+
+
+def _parse_bbox(value: Optional[str]) -> tuple[float, float, float, float] | None:
+    if not value:
+        return None
+    parts = [part.strip() for part in value.split(",")]
+    if len(parts) != 4:
+        raise HTTPException(status_code=400, detail="bbox must be lon_min,lat_min,lon_max,lat_max")
+    try:
+        lon_min, lat_min, lon_max, lat_max = [float(part) for part in parts]
+    except ValueError as exc:
+        raise HTTPException(status_code=400, detail="bbox values must be numbers") from exc
+    if lat_min > lat_max:
+        lat_min, lat_max = lat_max, lat_min
+    if lon_min > lon_max:
+        lon_min, lon_max = lon_max, lon_min
+    return lon_min, lat_min, lon_max, lat_max
+
+
+def _matches_vessel_type(props: dict[str, Any], requested_types: set[str]) -> bool:
+    if not requested_types:
+        return True
+    for requested_type in requested_types:
+        predicate = VESSEL_TYPE_FILTERS.get(requested_type)
+        if predicate and predicate(props):
+            return True
+    return False
+
+
+def _build_vessel_stats(features: List[dict[str, Any]]) -> dict[str, Any]:
+    by_type: dict[str, int] = {}
+    underway = 0
+    anchored_or_moored = 0
+    for feature in features:
+        props = feature.get("properties", {})
+        vessel_type = str(props.get("vessel_type_name") or "Other")
+        by_type[vessel_type] = by_type.get(vessel_type, 0) + 1
+        nav_status = props.get("nav_status")
+        if nav_status in (1, 5):
+            anchored_or_moored += 1
+        else:
+            underway += 1
+    return {
+        "total": len(features),
+        "by_type": by_type,
+        "underway": underway,
+        "anchored_or_moored": anchored_or_moored,
+    }
+
+
 def convert_bgp_anomalies_to_geojson(
     records: List[BGPAnomaly],
     geography_hints: Optional[Dict[str, Dict[str, Any]]] = None,
@@ -776,15 +1129,41 @@ async def get_cables_geojson(db: AsyncSession = Depends(get_db)):
     except HTTPException:
         raise
     except Exception as e:
+        logger.exception_event(
+            "Failed to build cables GeoJSON response",
+            event="visualization.cables.load_failed",
+            context={"error": str(e)},
+        )
+        await record_system_log(
+            source="backend",
+            service="api",
+            module=__name__,
+            event="visualization.cables.load_failed",
+            level="error",
+            message="Failed to build cables GeoJSON response",
+            category="visualization",
+            context={"error": str(e)},
+        )
         raise HTTPException(status_code=500, detail=f"Internal error: {str(e)}")
 
 
 @router.get("/geo/landing-points")
 async def get_landing_points_geojson(db: AsyncSession = Depends(get_db)):
     try:
-        records = await _load_current_collected_data(db, "arcgis_landing_points")
-        relation_records = await _load_current_collected_data(db, "arcgis_cable_landing_relation")
-        cable_records = await _load_current_collected_data(db, "arcgis_cables")
+        records_by_source = await _load_current_collected_data_by_sources(
+            db,
+            [
+                "arcgis_landing_points",
+                "arcgis_cable_landing_relation",
+                "arcgis_cables",
+            ],
+        )
+        records = records_by_source.get("arcgis_landing_points", [])
+        relation_records = records_by_source.get(
+            "arcgis_cable_landing_relation",
+            [],
+        )
+        cable_records = records_by_source.get("arcgis_cables", [])
 
         city_to_cable_ids_map, cable_id_to_name_map = _build_landing_point_cable_maps(
             relation_records,
@@ -801,9 +1180,68 @@ async def get_landing_points_geojson(db: AsyncSession = Depends(get_db)):
     except HTTPException:
         raise
     except Exception as e:
+        logger.exception_event(
+            "Failed to build landing points GeoJSON response",
+            event="visualization.landing_points.load_failed",
+            context={"error": str(e)},
+        )
+        await record_system_log(
+            source="backend",
+            service="api",
+            module=__name__,
+            event="visualization.landing_points.load_failed",
+            level="error",
+            message="Failed to build landing points GeoJSON response",
+            category="visualization",
+            context={"error": str(e)},
+        )
         raise HTTPException(status_code=500, detail=f"Internal error: {str(e)}")
 
 
+@router.get("/terrain/terrarium/{z}/{x}/{y}.png")
+async def get_terrarium_tile(z: int, x: int, y: int):
+    """Proxy Terrarium elevation tiles through the backend to avoid browser CORS issues."""
+    if z < 0 or x < 0 or y < 0:
+        raise HTTPException(status_code=400, detail="Invalid terrain tile coordinates")
+
+    url = TERRAIN_TILE_URL_TEMPLATE.format(z=z, x=x, y=y)
+
+    try:
+        async with httpx.AsyncClient(
+            timeout=20.0,
+            follow_redirects=True,
+        ) as client:
+            upstream = await client.get(url)
+            upstream.raise_for_status()
+    except httpx.HTTPStatusError as exc:
+        raise HTTPException(
+            status_code=exc.response.status_code,
+            detail=f"Terrain tile upstream error: {exc.response.status_code}",
+        ) from exc
+    except httpx.HTTPError as exc:
+        raise HTTPException(
+            status_code=502,
+            detail=f"Terrain tile fetch failed: {exc}",
+        ) from exc
+
+    cache_control = upstream.headers.get("cache-control") or "public, max-age=86400"
+    etag = upstream.headers.get("etag")
+    last_modified = upstream.headers.get("last-modified")
+    headers = {
+        "Cache-Control": cache_control,
+    }
+    if etag:
+        headers["ETag"] = etag
+    if last_modified:
+        headers["Last-Modified"] = last_modified
+
+    return Response(
+        content=upstream.content,
+        media_type=upstream.headers.get("content-type", "image/png"),
+        headers=headers,
+    )
+
+
 @router.get("/geo/all")
 async def get_all_geojson(db: AsyncSession = Depends(get_db)):
     records_by_source = await _load_current_collected_data_by_sources(
@@ -916,6 +1354,184 @@ async def get_gpu_clusters_geojson(
     }
 
 
+@router.get("/geo/compute-centers")
+async def get_compute_centers_geojson(
+    limit: int = Query(200, ge=1, le=1000),
+    db: AsyncSession = Depends(get_db),
+):
+    """获取统一算力中心 GeoJSON 数据"""
+    records_by_source = await _load_current_collected_data_by_sources(
+        db,
+        ["top500", "epoch_ai_gpu"],
+    )
+    records = _filter_known_records(
+        records_by_source.get("top500", []) + records_by_source.get("epoch_ai_gpu", []),
+    )
+    if limit is not None:
+        records = records[:limit]
+
+    if not records:
+        return {
+            "type": "FeatureCollection",
+            "features": [],
+            "count": 0,
+            "stats": {
+                "total": 0,
+                "supercomputers": 0,
+                "gpu_clusters": 0,
+            },
+        }
+
+    geojson = convert_compute_centers_to_geojson(records)
+    features = geojson.get("features", [])
+    return {
+        **geojson,
+        "count": len(features),
+        "stats": {
+            "total": len(features),
+            "supercomputers": sum(
+                1 for feature in features
+                if feature.get("properties", {}).get("site_type") == "supercomputer"
+            ),
+            "gpu_clusters": sum(
+                1 for feature in features
+                if feature.get("properties", {}).get("site_type") == "gpu_cluster"
+            ),
+        },
+    }
+
+
+@router.get("/geo/vessels")
+async def get_vessels_geojson(
+    bbox: Optional[str] = Query(
+        None,
+        description="Viewport bbox as lon_min,lat_min,lon_max,lat_max",
+    ),
+    type: Optional[str] = Query(
+        None,
+        description="Comma-separated vessel types: cargo,tanker,passenger,fishing,military,other",
+    ),
+    limit: int = Query(5000, ge=1, le=50000),
+    db: AsyncSession = Depends(get_db),
+):
+    """Return latest vessel positions as GeoJSON points."""
+    latest_times = (
+        select(
+            VesselPosition.mmsi.label("mmsi"),
+            func.max(VesselPosition.received_at).label("received_at"),
+        )
+        .group_by(VesselPosition.mmsi)
+        .subquery()
+    )
+    stmt = (
+        select(VesselPosition, VesselStatic)
+        .join(
+            latest_times,
+            (VesselPosition.mmsi == latest_times.c.mmsi)
+            & (VesselPosition.received_at == latest_times.c.received_at),
+        )
+        .outerjoin(VesselStatic, VesselStatic.mmsi == VesselPosition.mmsi)
+        .order_by(VesselPosition.received_at.desc())
+        .limit(limit)
+    )
+
+    parsed_bbox = _parse_bbox(bbox)
+    if parsed_bbox is not None:
+        lon_min, lat_min, lon_max, lat_max = parsed_bbox
+        stmt = stmt.where(
+            VesselPosition.lon >= lon_min,
+            VesselPosition.lon <= lon_max,
+            VesselPosition.lat >= lat_min,
+            VesselPosition.lat <= lat_max,
+        )
+
+    result = await db.execute(stmt)
+    rows = list(result.all())
+    geojson = convert_vessels_to_geojson(rows)
+    requested_types = {
+        item.strip().lower()
+        for item in (type or "").split(",")
+        if item.strip()
+    }
+    if requested_types:
+        geojson["features"] = [
+            feature
+            for feature in geojson.get("features", [])
+            if _matches_vessel_type(feature.get("properties", {}), requested_types)
+        ]
+
+    features = geojson.get("features", [])
+    return {
+        **geojson,
+        "count": len(features),
+        "stats": _build_vessel_stats(features),
+    }
+
+
+@router.get("/vessels/{mmsi}")
+async def get_vessel_detail(mmsi: int, db: AsyncSession = Depends(get_db)):
+    latest_position_stmt = (
+        select(VesselPosition)
+        .where(VesselPosition.mmsi == mmsi)
+        .order_by(VesselPosition.received_at.desc())
+        .limit(1)
+    )
+    static = await db.get(VesselStatic, mmsi)
+    result = await db.execute(latest_position_stmt)
+    position = result.scalar_one_or_none()
+    if position is None:
+        raise HTTPException(status_code=404, detail="Vessel not found")
+    geojson = convert_vessels_to_geojson([(position, static)])
+    return {
+        **(geojson["features"][0]["properties"]),
+        "latitude": position.lat,
+        "longitude": position.lon,
+    }
+
+
+@router.get("/vessels/{mmsi}/track")
+async def get_vessel_track(
+    mmsi: int,
+    hours: int = Query(6, ge=1, le=24),
+    db: AsyncSession = Depends(get_db),
+):
+    cutoff = datetime.now(UTC) - timedelta(hours=hours)
+    result = await db.execute(
+        select(VesselPosition)
+        .where(VesselPosition.mmsi == mmsi)
+        .where(VesselPosition.received_at >= cutoff)
+        .order_by(VesselPosition.received_at.asc())
+    )
+    positions = list(result.scalars().all())
+    if not positions:
+        return {
+            "type": "FeatureCollection",
+            "features": [],
+            "count": 0,
+        }
+
+    return {
+        "type": "FeatureCollection",
+        "features": [
+            {
+                "type": "Feature",
+                "geometry": {
+                    "type": "LineString",
+                    "coordinates": [[position.lon, position.lat] for position in positions],
+                },
+                "properties": {
+                    "mmsi": mmsi,
+                    "hours": hours,
+                    "point_count": len(positions),
+                    "start_at": to_iso8601_utc(positions[0].received_at),
+                    "end_at": to_iso8601_utc(positions[-1].received_at),
+                },
+            }
+        ],
+        "count": 1,
+    }
+
+
 @router.get("/geo/bgp-anomalies")
 async def get_bgp_anomalies_geojson(
     severity: Optional[str] = Query(None),
@@ -971,6 +1587,76 @@ async def get_bgp_collectors_geojson(db: AsyncSession = Depends(get_db)):
     return {**geojson, "count": len(geojson.get("features", []))}
 
 
+@router.get("/geo/summary")
+async def get_visualization_geo_summary(db: AsyncSession = Depends(get_db)):
+    """Return lightweight Earth HUD counts without loading layer GeoJSON payloads."""
+    records_by_source = await _load_current_collected_data_by_sources(
+        db,
+        [
+            "arcgis_cables",
+            "arcgis_landing_points",
+            "celestrak_tle",
+            "top500",
+            "epoch_ai_gpu",
+        ],
+    )
+
+    cables = convert_cable_to_geojson(records_by_source.get("arcgis_cables", []))
+    landing_points = convert_landing_point_to_geojson(
+        records_by_source.get("arcgis_landing_points", []),
+    )
+    satellites = convert_satellite_to_geojson(
+        _filter_known_records(records_by_source.get("celestrak_tle", [])),
+    )
+    compute_centers = convert_compute_centers_to_geojson(
+        _filter_known_records(
+            records_by_source.get("top500", [])
+            + records_by_source.get("epoch_ai_gpu", []),
+        ),
+    )
+    compute_features = compute_centers.get("features", [])
+
+    active_incident_result = await db.execute(
+        select(func.count(BGPIncident.id)).where(BGPIncident.status == "active"),
+    )
+    active_anomaly_result = await db.execute(
+        select(func.count(BGPAnomaly.id)).where(BGPAnomaly.status == "active"),
+    )
+    active_incident_count = int(active_incident_result.scalar() or 0)
+    active_anomaly_count = int(active_anomaly_result.scalar() or 0)
+    bgp_collectors = await build_bgp_collector_coverage(
+        db,
+        source_filter=("ris_live_bgp", "bgpstream_bgp"),
+    )
+    vessel_count_result = await db.execute(
+        select(func.count(func.distinct(VesselPosition.mmsi))),
+    )
+    vessel_count = int(vessel_count_result.scalar() or 0)
+
+    return {
+        "generated_at": to_iso8601_utc(datetime.now(UTC)),
+        "stats": {
+            "cable_count": len(cables.get("features", [])),
+            "landing_point_count": len(landing_points.get("features", [])),
+            "satellite_count": len(satellites.get("features", [])),
+            "compute_center_count": len(compute_features),
+            "vessel_count": vessel_count,
+            "supercomputer_count": sum(
+                1 for feature in compute_features
+                if feature.get("properties", {}).get("site_type") == "supercomputer"
+            ),
+            "gpu_cluster_count": sum(
+                1 for feature in compute_features
+                if feature.get("properties", {}).get("site_type") == "gpu_cluster"
+            ),
+            "bgp_event_count": active_incident_count or active_anomaly_count,
+            "bgp_incident_count": active_incident_count,
+            "bgp_anomaly_count": active_anomaly_count,
+            "bgp_collector_count": len([item for item in bgp_collectors if item.get("collector")]),
+        },
+    }
+
+
 @router.get("/all")
 async def get_all_visualization_data(db: AsyncSession = Depends(get_db)):
     """获取所有可视化数据的统一端点

backend/app/api/v1/websocket.py

@@ -2,7 +2,6 @@
 
 import asyncio
 import json
-import logging
 from datetime import UTC, datetime
 from typing import Optional
 
@@ -10,10 +9,11 @@ from fastapi import APIRouter, WebSocket, WebSocketDisconnect, Query
 from jose import jwt, JWTError
 
 from app.core.config import settings
+from app.core.logging import get_logger
 from app.core.time import to_iso8601_utc
 from app.core.websocket.manager import manager
 
-logger = logging.getLogger(__name__)
+logger = get_logger(__name__, service="api")
 router = APIRouter()
 
 
@@ -22,11 +22,18 @@ async def authenticate_token(token: str) -> Optional[dict]:
     try:
         payload = jwt.decode(token, settings.SECRET_KEY, algorithms=[settings.ALGORITHM])
         if payload.get("type") != "access":
-            logger.warning(f"WebSocket auth failed: wrong token type")
+            logger.warning_event(
+                "WebSocket auth failed: wrong token type",
+                event="auth.websocket.invalid_token_type",
+            )
             return None
         return payload
     except JWTError as e:
-        logger.warning(f"WebSocket auth failed: {e}")
+        logger.warning_event(
+            "WebSocket auth failed",
+            event="auth.websocket.decode_failed",
+            context={"error": str(e)},
+        )
         return None
 
 
@@ -36,10 +43,17 @@ async def websocket_endpoint(
     token: str = Query(...),
 ):
     """WebSocket endpoint for real-time data"""
-    logger.info(f"WebSocket connection attempt with token: {token[:20]}...")
+    logger.info_event(
+        "WebSocket connection attempt",
+        event="auth.websocket.connection_attempt",
+        context={"token_preview": f"{token[:8]}..."},
+    )
     payload = await authenticate_token(token)
     if payload is None:
-        logger.warning("WebSocket authentication failed, closing connection")
+        logger.warning_event(
+            "WebSocket authentication failed, closing connection",
+            event="auth.websocket.connection_rejected",
+        )
         await websocket.close(code=4001)
         return
 

backend/app/core/cache.py

@@ -1,15 +1,15 @@
 """Redis caching service"""
 
 import json
-import logging
 from datetime import timedelta
 from typing import Optional, Any
 
 import redis
 
 from app.core.config import settings
+from app.core.logging import get_logger
 
-logger = logging.getLogger(__name__)
+logger = get_logger(__name__)
 
 
 # Lazy Redis client initialization
@@ -47,7 +47,7 @@ class CacheService:
                 return json.loads(value)
             return None
         except Exception as e:
-            logger.warning(f"Cache get error: {e}")
+            logger.warning_event("Cache get error", event="cache.get.failed", context={"error": str(e)})
             return None
 
     def set(
@@ -61,7 +61,7 @@ class CacheService:
             serialized = json.dumps(value, default=str)
             return self.client.setex(key, expire_seconds, serialized)
         except Exception as e:
-            logger.warning(f"Cache set error: {e}")
+            logger.warning_event("Cache set error", event="cache.set.failed", context={"error": str(e)})
             return False
 
     def delete(self, key: str) -> bool:
@@ -69,7 +69,7 @@ class CacheService:
         try:
             return self.client.delete(key) > 0
         except Exception as e:
-            logger.warning(f"Cache delete error: {e}")
+            logger.warning_event("Cache delete error", event="cache.delete.failed", context={"error": str(e)})
             return False
 
     def delete_pattern(self, pattern: str) -> int:
@@ -80,7 +80,7 @@ class CacheService:
                 return self.client.delete(*keys)
             return 0
         except Exception as e:
-            logger.warning(f"Cache delete_pattern error: {e}")
+            logger.warning_event("Cache delete_pattern error", event="cache.delete_pattern.failed", context={"error": str(e)})
             return 0
 
     def get_or_set(

backend/app/core/data_sources.py

@@ -30,6 +30,8 @@ COLLECTOR_URL_KEYS = {
     "iptoasn_prefix_geo": "iptoasn.combined_url",
     "opengeofeed_prefix_geo": "opengeofeed.public_csv_url",
     "nro_delegated_prefix_geo": "nro.delegated_stats_url",
+    "news_live_streams": "news_live_streams.channels_url",
+    "barentswatch_vessels": "barentswatch_vessels.url",
 }
 
 

backend/app/core/data_sources.yaml

@@ -86,3 +86,15 @@ opengeofeed:
 nro:
   # NRO delegated stats 下载地址
   delegated_stats_url: "https://ftp.ripe.net/pub/stats/ripencc/nro-stats/latest/nro-delegated-stats"
+
+news_live_streams:
+  # IPTV-org 频道元数据 JSON
+  channels_url: "https://iptv-org.github.io/api/channels.json"
+  # IPTV-org 频道播放流 JSON
+  streams_url: "https://iptv-org.github.io/api/streams.json"
+  # IPTV-org 台标 JSON
+  logos_url: "https://iptv-org.github.io/api/logos.json"
+
+barentswatch_vessels:
+  # BarentsWatch Live AIS latest combined endpoint. Requires an AIS bearer token.
+  url: "https://live.ais.barentswatch.no/v1/latest/combined"

backend/app/core/datasource_defaults.py

@@ -4,163 +4,246 @@ DEFAULT_DATASOURCES = {
     "top500": {
         "id": 1,
         "name": "TOP500 Supercomputers",
+        "display_name": "TOP500 超算榜单",
         "module": "L1",
         "priority": "P0",
         "frequency_minutes": 240,
+        "is_free": True,
+        "requires_credentials": False,
     },
     "epoch_ai_gpu": {
         "id": 2,
         "name": "Epoch AI GPU Clusters",
+        "display_name": "Epoch AI GPU 集群",
         "module": "L1",
         "priority": "P0",
         "frequency_minutes": 360,
+        "is_free": True,
+        "requires_credentials": False,
     },
     "huggingface_models": {
         "id": 3,
         "name": "HuggingFace Models",
+        "display_name": "Hugging Face 模型",
         "module": "L2",
         "priority": "P1",
         "frequency_minutes": 720,
+        "is_free": True,
+        "requires_credentials": False,
     },
     "huggingface_datasets": {
         "id": 4,
         "name": "HuggingFace Datasets",
+        "display_name": "Hugging Face 数据集",
         "module": "L2",
         "priority": "P1",
         "frequency_minutes": 720,
+        "is_free": True,
+        "requires_credentials": False,
     },
     "huggingface_spaces": {
         "id": 5,
         "name": "HuggingFace Spaces",
+        "display_name": "Hugging Face Spaces",
         "module": "L2",
         "priority": "P2",
         "frequency_minutes": 1440,
+        "is_free": True,
+        "requires_credentials": False,
     },
     "peeringdb_ixp": {
         "id": 6,
         "name": "PeeringDB IXP",
+        "display_name": "PeeringDB 交换中心",
         "module": "L2",
         "priority": "P1",
         "frequency_minutes": 1440,
+        "is_free": True,
+        "requires_credentials": False,
     },
     "peeringdb_network": {
         "id": 7,
         "name": "PeeringDB Networks",
+        "display_name": "PeeringDB 网络",
         "module": "L2",
         "priority": "P2",
         "frequency_minutes": 2880,
+        "is_free": True,
+        "requires_credentials": False,
     },
     "peeringdb_facility": {
         "id": 8,
         "name": "PeeringDB Facilities",
+        "display_name": "PeeringDB 设施",
         "module": "L2",
         "priority": "P2",
         "frequency_minutes": 2880,
+        "is_free": True,
+        "requires_credentials": False,
     },
     "telegeography_cables": {
         "id": 9,
         "name": "Submarine Cables",
+        "display_name": "海底光缆",
         "module": "L2",
         "priority": "P1",
         "frequency_minutes": 10080,
+        "is_free": True,
+        "requires_credentials": False,
     },
     "telegeography_landing": {
         "id": 10,
         "name": "Cable Landing Points",
+        "display_name": "光缆登陆点",
         "module": "L2",
         "priority": "P2",
         "frequency_minutes": 10080,
+        "is_free": True,
+        "requires_credentials": False,
     },
     "telegeography_systems": {
         "id": 11,
         "name": "Cable Systems",
+        "display_name": "光缆系统",
         "module": "L2",
         "priority": "P2",
         "frequency_minutes": 10080,
+        "is_free": True,
+        "requires_credentials": False,
     },
     "arcgis_cables": {
         "id": 15,
         "name": "ArcGIS Submarine Cables",
+        "display_name": "ArcGIS 海底光缆",
         "module": "L2",
         "priority": "P1",
         "frequency_minutes": 10080,
+        "is_free": True,
+        "requires_credentials": False,
     },
     "arcgis_landing_points": {
         "id": 16,
         "name": "ArcGIS Landing Points",
+        "display_name": "ArcGIS 登陆点",
         "module": "L2",
         "priority": "P1",
         "frequency_minutes": 10080,
+        "is_free": True,
+        "requires_credentials": False,
     },
     "arcgis_cable_landing_relation": {
         "id": 17,
         "name": "ArcGIS Cable-Landing Relations",
+        "display_name": "ArcGIS 光缆登陆关系",
         "module": "L2",
         "priority": "P1",
         "frequency_minutes": 10080,
+        "is_free": True,
+        "requires_credentials": False,
     },
     "fao_landing_points": {
         "id": 18,
         "name": "FAO Landing Points",
+        "display_name": "FAO 登陆点",
         "module": "L2",
         "priority": "P1",
         "frequency_minutes": 10080,
+        "is_free": True,
+        "requires_credentials": False,
     },
     "spacetrack_tle": {
         "id": 19,
         "name": "Space-Track TLE",
+        "display_name": "Space-Track 轨道根数",
         "module": "L3",
         "priority": "P2",
         "frequency_minutes": 1440,
+        "is_free": True,
+        "requires_credentials": True,
+        "credential_provider": "spacetrack",
+        "credential_status": "planned",
     },
     "celestrak_tle": {
         "id": 20,
         "name": "CelesTrak TLE",
+        "display_name": "CelesTrak 轨道根数",
         "module": "L3",
         "priority": "P2",
         "frequency_minutes": 1440,
+        "is_free": True,
+        "requires_credentials": False,
     },
     "ris_live_bgp": {
         "id": 21,
         "name": "RIPE RIS Live BGP",
+        "display_name": "RIPE RIS 实时 BGP",
         "module": "L3",
         "priority": "P1",
         "frequency_minutes": 15,
+        "is_free": True,
+        "requires_credentials": False,
     },
     "bgpstream_bgp": {
         "id": 22,
         "name": "CAIDA BGPStream Backfill",
+        "display_name": "CAIDA BGPStream 回填",
         "module": "L3",
         "priority": "P1",
         "frequency_minutes": 360,
+        "is_free": True,
+        "requires_credentials": False,
     },
     "iptoasn_prefix_geo": {
         "id": 23,
         "name": "IPtoASN Prefix Geography",
+        "display_name": "IPtoASN 前缀地理",
         "module": "L3",
         "priority": "P1",
         "frequency_minutes": 1440,
+        "is_free": True,
+        "requires_credentials": False,
     },
     "opengeofeed_prefix_geo": {
         "id": 24,
         "name": "OpenGeoFeed Prefix Geography",
+        "display_name": "OpenGeoFeed 前缀地理",
         "module": "L3",
         "priority": "P1",
         "frequency_minutes": 1440,
+        "is_free": True,
+        "requires_credentials": False,
     },
     "nro_delegated_prefix_geo": {
         "id": 25,
         "name": "NRO Delegated Prefix Geography",
+        "display_name": "NRO 分配前缀地理",
         "module": "L3",
         "priority": "P1",
         "frequency_minutes": 1440,
+        "is_free": True,
+        "requires_credentials": False,
     },
     "news_live_streams": {
         "id": 26,
         "name": "News Live Streams",
+        "display_name": "新闻直播源",
         "module": "L4",
         "priority": "P2",
         "frequency_minutes": 720,
+        "is_free": True,
+        "requires_credentials": False,
+    },
+    "barentswatch_vessels": {
+        "id": 27,
+        "name": "BarentsWatch AIS Vessels",
+        "display_name": "BarentsWatch AIS 船舶",
+        "module": "L4",
+        "priority": "P1",
+        "frequency_minutes": 1,
+        "is_free": True,
+        "requires_credentials": True,
+        "credential_provider": "barentswatch",
+        "credential_status": "supported",
     },
 }
 

backend/app/core/logging.py

@@ -0,0 +1,161 @@
+from __future__ import annotations
+
+import json
+import logging
+import os
+import re
+
+from collections.abc import Mapping, Sequence
+from typing import Any
+
+from app.core.request_context import get_request_id
+
+DEFAULT_SERVICE = "backend"
+DEFAULT_EVENT = "app.log"
+DEFAULT_LOG_LEVEL = os.getenv("PLANET_LOG_LEVEL", "INFO").upper()
+REDACTED = "[REDACTED]"
+SENSITIVE_FIELD_NAMES = {
+    "access_token",
+    "api_key",
+    "authorization",
+    "cookie",
+    "password",
+    "refresh_token",
+    "secret",
+    "token",
+}
+SENSITIVE_TEXT_PATTERNS = (
+    re.compile(r"(?i)(authorization\s*[:=]\s*)(.+)"),
+    re.compile(r"(?i)(bearer\s+)([A-Za-z0-9._\-]+)"),
+    re.compile(r"(?i)(token\s*[:=]\s*)(.+)"),
+    re.compile(r"(?i)(password\s*[:=]\s*)(.+)"),
+    re.compile(r"(?i)(cookie\s*[:=]\s*)(.+)"),
+)
+
+
+def sanitize_log_value(value: Any) -> Any:
+    if isinstance(value, Mapping):
+        return {
+            str(key): (REDACTED if str(key).lower() in SENSITIVE_FIELD_NAMES else sanitize_log_value(item))
+            for key, item in value.items()
+        }
+    if isinstance(value, Sequence) and not isinstance(value, (str, bytes, bytearray)):
+        return [sanitize_log_value(item) for item in value]
+    if isinstance(value, str):
+        sanitized = value
+        for pattern in SENSITIVE_TEXT_PATTERNS:
+            sanitized = pattern.sub(lambda match: f"{match.group(1)}{REDACTED}", sanitized)
+        return sanitized
+    return value
+
+
+def _normalize_context(context: Any) -> dict[str, Any]:
+    if context is None:
+        return {}
+    if isinstance(context, Mapping):
+        sanitized = sanitize_log_value(context)
+        return {str(key): value for key, value in sanitized.items()}
+    return {"value": sanitize_log_value(context)}
+
+
+class PlanetContextFilter(logging.Filter):
+    def filter(self, record: logging.LogRecord) -> bool:
+        record.request_id = getattr(record, "request_id", None) or get_request_id() or "-"
+        record.service = getattr(record, "service", None) or DEFAULT_SERVICE
+        record.event = getattr(record, "event", None) or DEFAULT_EVENT
+        record.context = _normalize_context(getattr(record, "context", None))
+        record.message = sanitize_log_value(record.getMessage())
+        return True
+
+
+class PlanetFormatter(logging.Formatter):
+    def format(self, record: logging.LogRecord) -> str:
+        timestamp = self.formatTime(record, self.datefmt)
+        level = record.levelname
+        service = getattr(record, "service", DEFAULT_SERVICE)
+        module_name = record.name
+        event = getattr(record, "event", DEFAULT_EVENT)
+        request_id = getattr(record, "request_id", "-")
+        message = sanitize_log_value(record.getMessage())
+        context = _normalize_context(getattr(record, "context", None))
+        context_suffix = ""
+        if context:
+            context_suffix = f" context={json.dumps(context, ensure_ascii=False, sort_keys=True)}"
+        rendered = (
+            f"{timestamp} {level} service={service} module={module_name} "
+            f"event={event} request_id={request_id} message={message}{context_suffix}"
+        )
+        if record.exc_info:
+            rendered = f"{rendered}\n{self.formatException(record.exc_info)}"
+        return rendered
+
+
+class PlanetLoggerAdapter(logging.LoggerAdapter):
+    def process(self, msg: Any, kwargs: dict[str, Any]) -> tuple[Any, dict[str, Any]]:
+        extra = dict(self.extra)
+        extra.update(kwargs.get("extra", {}))
+        if "context" in extra:
+            extra["context"] = _normalize_context(extra.get("context"))
+        kwargs["extra"] = extra
+        return sanitize_log_value(msg), kwargs
+
+    def log_event(
+        self,
+        level: int,
+        message: str,
+        *,
+        event: str,
+        context: Mapping[str, Any] | None = None,
+        **extra: Any,
+    ) -> None:
+        self.log(level, message, extra={"event": event, "context": context or {}, **extra})
+
+    def debug_event(self, message: str, *, event: str, context: Mapping[str, Any] | None = None, **extra: Any) -> None:
+        self.log_event(logging.DEBUG, message, event=event, context=context, **extra)
+
+    def info_event(self, message: str, *, event: str, context: Mapping[str, Any] | None = None, **extra: Any) -> None:
+        self.log_event(logging.INFO, message, event=event, context=context, **extra)
+
+    def warning_event(self, message: str, *, event: str, context: Mapping[str, Any] | None = None, **extra: Any) -> None:
+        self.log_event(logging.WARNING, message, event=event, context=context, **extra)
+
+    def error_event(self, message: str, *, event: str, context: Mapping[str, Any] | None = None, **extra: Any) -> None:
+        self.log_event(logging.ERROR, message, event=event, context=context, **extra)
+
+    def exception_event(
+        self,
+        message: str,
+        *,
+        event: str,
+        context: Mapping[str, Any] | None = None,
+        **extra: Any,
+    ) -> None:
+        self.error(message, exc_info=True, extra={"event": event, "context": context or {}, **extra})
+
+
+def get_logger(name: str, *, service: str = DEFAULT_SERVICE) -> PlanetLoggerAdapter:
+    return PlanetLoggerAdapter(logging.getLogger(name), {"service": service})
+
+
+def configure_logging(level: str | None = None) -> None:
+    root_logger = logging.getLogger()
+    if getattr(configure_logging, "_configured", False):
+        if level:
+            root_logger.setLevel(level.upper())
+        return
+
+    handler = logging.StreamHandler()
+    handler.setFormatter(PlanetFormatter(datefmt="%Y-%m-%d %H:%M:%S"))
+    handler.addFilter(PlanetContextFilter())
+
+    root_logger.handlers.clear()
+    root_logger.addHandler(handler)
+    root_logger.setLevel((level or DEFAULT_LOG_LEVEL).upper())
+
+    for logger_name in ("uvicorn", "uvicorn.error", "uvicorn.access"):
+        target_logger = logging.getLogger(logger_name)
+        target_logger.handlers.clear()
+        target_logger.propagate = True
+
+    logging.captureWarnings(True)
+    configure_logging._configured = True

backend/app/core/request_context.py

@@ -0,0 +1,14 @@
+from __future__ import annotations
+
+from contextvars import ContextVar
+
+
+request_id_context: ContextVar[str | None] = ContextVar("request_id", default=None)
+
+
+def set_request_id(request_id: str | None) -> None:
+    request_id_context.set(request_id)
+
+
+def get_request_id() -> str | None:
+    return request_id_context.get()

backend/app/core/target_schema_registry.py

@@ -0,0 +1,151 @@
+"""Registry of target schemas supported by mapped custom data sources."""
+
+from __future__ import annotations
+
+from dataclasses import dataclass
+from datetime import datetime
+from typing import Any
+
+from pydantic import BaseModel, Field, ValidationError, field_validator
+
+
+class VesselAISRecord(BaseModel):
+    mmsi: int = Field(ge=100000000, le=999999999)
+    lat: float = Field(ge=-90, le=90)
+    lon: float = Field(ge=-180, le=180)
+    sog: float | None = None
+    cog: float | None = Field(default=None, ge=0, le=360)
+    heading: int | None = Field(default=None, ge=0, le=511)
+    name: str | None = None
+    vessel_type: str | int | None = None
+    received_at: datetime | None = None
+
+
+class GeoPointRecord(BaseModel):
+    lat: float = Field(ge=-90, le=90)
+    lon: float = Field(ge=-180, le=180)
+    name: str | None = None
+    type: str | None = None
+    source_id: str | None = None
+    observed_at: datetime | None = None
+    metadata: dict[str, Any] = Field(default_factory=dict)
+
+
+class GenericRecord(BaseModel):
+    data: dict[str, Any] = Field(default_factory=dict)
+    source_id: str | None = None
+    observed_at: datetime | None = None
+
+    @field_validator("data")
+    @classmethod
+    def require_payload(cls, value: dict[str, Any]) -> dict[str, Any]:
+        if not value:
+            raise ValueError("generic_records requires a non-empty data object")
+        return value
+
+
+@dataclass(frozen=True)
+class TargetField:
+    name: str
+    type: str
+    required: bool = False
+    description: str = ""
+    example: Any = None
+
+    def to_dict(self) -> dict[str, Any]:
+        return {
+            "name": self.name,
+            "type": self.type,
+            "required": self.required,
+            "description": self.description,
+            "example": self.example,
+        }
+
+
+@dataclass(frozen=True)
+class TargetSchema:
+    key: str
+    label: str
+    description: str
+    fields: tuple[TargetField, ...]
+    model: type[BaseModel]
+    destination: str
+
+    def to_dict(self) -> dict[str, Any]:
+        return {
+            "key": self.key,
+            "label": self.label,
+            "description": self.description,
+            "destination": self.destination,
+            "fields": [field.to_dict() for field in self.fields],
+        }
+
+    def validate_record(self, record: dict[str, Any]) -> tuple[dict[str, Any] | None, list[str]]:
+        try:
+            return self.model.model_validate(record).model_dump(mode="json"), []
+        except ValidationError as exc:
+            return None, [
+                ".".join(str(part) for part in error["loc"]) + f": {error['msg']}"
+                for error in exc.errors()
+            ]
+
+
+TARGET_SCHEMAS: dict[str, TargetSchema] = {
+    "vessel_ais": TargetSchema(
+        key="vessel_ais",
+        label="船舶 AIS",
+        description="船只位置、航速、航向、MMSI 等 AIS 数据。",
+        destination="vessel_position",
+        model=VesselAISRecord,
+        fields=(
+            TargetField("mmsi", "integer", True, "MMSI 九位船舶标识", 257123000),
+            TargetField("lat", "float", True, "纬度", 59.91),
+            TargetField("lon", "float", True, "经度", 10.75),
+            TargetField("sog", "float", False, "对地航速，单位节", 12.4),
+            TargetField("cog", "float", False, "对地航向，0-360 度", 184.5),
+            TargetField("heading", "integer", False, "船首向，0-511", 186),
+            TargetField("name", "string", False, "船名", "OSLO EXPRESS"),
+            TargetField("vessel_type", "string", False, "船型", "cargo"),
+            TargetField("received_at", "datetime", False, "数据接收时间", "2026-04-28T00:00:00Z"),
+        ),
+    ),
+    "geo_points": TargetSchema(
+        key="geo_points",
+        label="通用地理点",
+        description="带经纬度的通用实体或事件点位。",
+        destination="generic_geo_points",
+        model=GeoPointRecord,
+        fields=(
+            TargetField("lat", "float", True, "纬度", 1.3),
+            TargetField("lon", "float", True, "经度", 103.8),
+            TargetField("name", "string", False, "点位名称", "Singapore"),
+            TargetField("type", "string", False, "点位类型", "datacenter"),
+            TargetField("source_id", "string", False, "来源侧 ID", "sg-1"),
+            TargetField("observed_at", "datetime", False, "观测时间", "2026-04-28T00:00:00Z"),
+            TargetField("metadata", "object", False, "扩展字段", {"provider": "example"}),
+        ),
+    ),
+    "generic_records": TargetSchema(
+        key="generic_records",
+        label="通用结构化记录",
+        description="未知结构数据沉淀，不直接进入 Earth 图层。",
+        destination="collected_data",
+        model=GenericRecord,
+        fields=(
+            TargetField("data", "object", True, "结构化记录主体", {"raw": "value"}),
+            TargetField("source_id", "string", False, "来源侧 ID", "record-1"),
+            TargetField("observed_at", "datetime", False, "观测时间", "2026-04-28T00:00:00Z"),
+        ),
+    ),
+}
+
+
+def list_target_schemas() -> list[dict[str, Any]]:
+    return [schema.to_dict() for schema in TARGET_SCHEMAS.values()]
+
+
+def get_target_schema(key: str) -> TargetSchema:
+    try:
+        return TARGET_SCHEMAS[key]
+    except KeyError as exc:
+        raise ValueError(f"Unsupported target schema: {key}") from exc

backend/app/db/session.py

@@ -5,10 +5,22 @@ from sqlalchemy.ext.asyncio import AsyncSession, create_async_engine, async_sess
 from sqlalchemy.orm import declarative_base
 
 from app.core.config import settings
+from app.core.logging import get_logger
+
+logger = get_logger(__name__)
+
+DB_POOL_CONFIG = {
+    "pool_pre_ping": True,
+    "pool_recycle": 1800,
+    "pool_size": 10,
+    "max_overflow": 20,
+    "pool_timeout": 30,
+}
 
 engine = create_async_engine(
     settings.DATABASE_URL,
     echo=settings.DEBUG if hasattr(settings, "DEBUG") else False,
+    **DB_POOL_CONFIG,
 )
 
 async_session_factory = async_sessionmaker(engine, class_=AsyncSession, expire_on_commit=False)
@@ -97,6 +109,21 @@ async def init_db():
     import app.models.system_setting  # noqa: F401
     import app.models.playground_session  # noqa: F401
     import app.models.playground_message  # noqa: F401
+    import app.models.system_log  # noqa: F401
+    import app.models.vessel  # noqa: F401
+    import app.models.datasource_mapping  # noqa: F401
+
+    logger.warning_event(
+        "Database pool settings active",
+        event="database.pool.initialized",
+        context={
+            "pool_pre_ping": DB_POOL_CONFIG["pool_pre_ping"],
+            "pool_recycle": DB_POOL_CONFIG["pool_recycle"],
+            "pool_size": DB_POOL_CONFIG["pool_size"],
+            "max_overflow": DB_POOL_CONFIG["max_overflow"],
+            "pool_timeout": DB_POOL_CONFIG["pool_timeout"],
+        },
+    )
 
     async with engine.begin() as conn:
         await conn.run_sync(Base.metadata.create_all)

backend/app/main.py

@@ -1,4 +1,5 @@
 from contextlib import asynccontextmanager
+from uuid import uuid4
 
 from fastapi import FastAPI
 from fastapi.middleware.cors import CORSMiddleware
@@ -7,6 +8,8 @@ from starlette.middleware.base import BaseHTTPMiddleware
 from app.api.main import api_router
 from app.api.v1 import websocket
 from app.core.config import settings
+from app.core.logging import configure_logging
+from app.core.request_context import set_request_id
 from app.core.websocket.broadcaster import broadcaster
 from app.db.session import init_db
 from app.services.scheduler import (
@@ -17,6 +20,9 @@ from app.services.scheduler import (
 )
 
 
+configure_logging()
+
+
 class WebSocketCORSMiddleware(BaseHTTPMiddleware):
     async def dispatch(self, request, call_next):
         if request.url.path.startswith("/ws") and request.method == "GET":
@@ -28,6 +34,18 @@ class WebSocketCORSMiddleware(BaseHTTPMiddleware):
         return await call_next(request)
 
 
+class RequestContextMiddleware(BaseHTTPMiddleware):
+    async def dispatch(self, request, call_next):
+        request_id = request.headers.get("X-Request-ID") or uuid4().hex
+        set_request_id(request_id)
+        try:
+            response = await call_next(request)
+        finally:
+            set_request_id(None)
+        response.headers["X-Request-ID"] = request_id
+        return response
+
+
 @asynccontextmanager
 async def lifespan(app: FastAPI):
     await init_db()
@@ -58,6 +76,7 @@ app.add_middleware(
     allow_headers=["*"],
 )
 
+app.add_middleware(RequestContextMiddleware)
 app.add_middleware(WebSocketCORSMiddleware)
 
 app.include_router(api_router, prefix="/api/v1")

backend/app/models/__init__.py

@@ -11,6 +11,9 @@ from app.models.bgp_observation import BGPObservation
 from app.models.system_setting import SystemSetting
 from app.models.playground_session import PlaygroundSession
 from app.models.playground_message import PlaygroundMessage
+from app.models.system_log import SystemLog, AuditLog
+from app.models.vessel import VesselPosition, VesselStatic
+from app.models.datasource_mapping import DataSourceMappingTemplate
 
 __all__ = [
     "User",
@@ -26,4 +29,9 @@ __all__ = [
     "BGPAnomaly",
     "BGPIncident",
     "BGPObservation",
+    "SystemLog",
+    "AuditLog",
+    "VesselPosition",
+    "VesselStatic",
+    "DataSourceMappingTemplate",
 ]

backend/app/models/datasource_mapping.py

@@ -0,0 +1,32 @@
+"""Mapping templates for user-defined data source payloads."""
+
+from sqlalchemy import Boolean, Column, DateTime, ForeignKey, Integer, JSON, String
+from sqlalchemy.sql import func
+
+from app.db.session import Base
+
+
+class DataSourceMappingTemplate(Base):
+    __tablename__ = "datasource_mapping_templates"
+
+    id = Column(Integer, primary_key=True, autoincrement=True)
+    datasource_config_id = Column(
+        Integer,
+        ForeignKey("datasource_configs.id"),
+        nullable=False,
+        index=True,
+    )
+    target_schema = Column(String(80), nullable=False, index=True)
+    mapping_json = Column(JSON, nullable=False, default={})
+    sample_payload_hash = Column(String(64), nullable=True)
+    validation_status = Column(String(30), nullable=False, default="draft")
+    version = Column(Integer, nullable=False, default=1)
+    is_active = Column(Boolean, nullable=False, default=False, index=True)
+    created_at = Column(DateTime(timezone=True), server_default=func.now())
+    updated_at = Column(DateTime(timezone=True), server_default=func.now(), onupdate=func.now())
+
+    def __repr__(self):
+        return (
+            f"<DataSourceMappingTemplate {self.id}: "
+            f"{self.datasource_config_id}/{self.target_schema}/v{self.version}>"
+        )

backend/app/models/system_log.py

@@ -0,0 +1,40 @@
+from sqlalchemy import JSON, Column, DateTime, Integer, String, Text
+from sqlalchemy.sql import func
+
+from app.db.session import Base
+
+
+class SystemLog(Base):
+    __tablename__ = "system_logs"
+
+    id = Column(Integer, primary_key=True, autoincrement=True)
+    occurred_at = Column(DateTime(timezone=True), server_default=func.now(), index=True)
+    source = Column(String(50), nullable=False, index=True)
+    service = Column(String(50), nullable=True)
+    module = Column(String(120), nullable=True)
+    event = Column(String(160), nullable=True, index=True)
+    level = Column(String(20), nullable=False, index=True)
+    message = Column(Text, nullable=False)
+    request_id = Column(String(64), nullable=True, index=True)
+    trace_id = Column(String(64), nullable=True)
+    user_id = Column(Integer, nullable=True, index=True)
+    category = Column(String(80), nullable=True, index=True)
+    context = Column(JSON, nullable=False, default=dict)
+    created_at = Column(DateTime(timezone=True), server_default=func.now())
+
+
+class AuditLog(Base):
+    __tablename__ = "audit_logs"
+
+    id = Column(Integer, primary_key=True, autoincrement=True)
+    occurred_at = Column(DateTime(timezone=True), server_default=func.now(), index=True)
+    actor_id = Column(Integer, nullable=True, index=True)
+    actor_name = Column(String(255), nullable=True)
+    action = Column(String(120), nullable=False, index=True)
+    target_type = Column(String(80), nullable=True)
+    target_id = Column(String(120), nullable=True)
+    result = Column(String(40), nullable=True, index=True)
+    request_id = Column(String(64), nullable=True, index=True)
+    ip = Column(String(64), nullable=True)
+    details = Column(JSON, nullable=False, default=dict)
+    created_at = Column(DateTime(timezone=True), server_default=func.now())

backend/app/models/vessel.py

@@ -0,0 +1,75 @@
+"""Vessel AIS models for live maritime tracking."""
+
+from sqlalchemy import BigInteger, Column, DateTime, Float, Index, Integer, SmallInteger, String
+from sqlalchemy.sql import func
+
+from app.core.time import to_iso8601_utc
+from app.db.session import Base
+
+
+class VesselStatic(Base):
+    """Slow-changing vessel identity and dimensions."""
+
+    __tablename__ = "vessel_static"
+
+    mmsi = Column(BigInteger, primary_key=True)
+    name = Column(String(128), nullable=True)
+    callsign = Column(String(16), nullable=True)
+    vessel_type = Column(SmallInteger, nullable=True, index=True)
+    vessel_type_name = Column(String(64), nullable=True, index=True)
+    flag = Column(String(4), nullable=True, index=True)
+    length = Column(Float, nullable=True)
+    width = Column(Float, nullable=True)
+    draught = Column(Float, nullable=True)
+    imo = Column(BigInteger, nullable=True)
+    updated_at = Column(DateTime(timezone=True), nullable=False, server_default=func.now())
+
+    def to_dict(self) -> dict:
+        return {
+            "mmsi": self.mmsi,
+            "name": self.name,
+            "callsign": self.callsign,
+            "vessel_type": self.vessel_type,
+            "vessel_type_name": self.vessel_type_name,
+            "flag": self.flag,
+            "length": self.length,
+            "width": self.width,
+            "draught": self.draught,
+            "imo": self.imo,
+            "updated_at": to_iso8601_utc(self.updated_at),
+        }
+
+
+class VesselPosition(Base):
+    """Append-only AIS positions retained for short history windows."""
+
+    __tablename__ = "vessel_position"
+
+    id = Column(Integer, primary_key=True, autoincrement=True)
+    mmsi = Column(BigInteger, nullable=False, index=True)
+    lat = Column(Float, nullable=False)
+    lon = Column(Float, nullable=False)
+    sog = Column(Float, nullable=True)
+    cog = Column(Float, nullable=True)
+    heading = Column(SmallInteger, nullable=True)
+    nav_status = Column(SmallInteger, nullable=True, index=True)
+    received_at = Column(DateTime(timezone=True), nullable=False, server_default=func.now(), index=True)
+
+    __table_args__ = (
+        Index("idx_vessel_pos_mmsi_time", "mmsi", "received_at"),
+        Index("idx_vessel_pos_time", "received_at"),
+        Index("idx_vessel_pos_lat_lon", "lat", "lon"),
+    )
+
+    def to_dict(self) -> dict:
+        return {
+            "id": self.id,
+            "mmsi": self.mmsi,
+            "lat": self.lat,
+            "lon": self.lon,
+            "sog": self.sog,
+            "cog": self.cog,
+            "heading": self.heading,
+            "nav_status": self.nav_status,
+            "received_at": to_iso8601_utc(self.received_at),
+        }

backend/app/services/ai_client.py

@@ -3,9 +3,11 @@ from __future__ import annotations
 import asyncio
 
 import httpx
-from fastapi import HTTPException, status
+from fastapi import Depends, HTTPException, status
+from sqlalchemy.ext.asyncio import AsyncSession
 
 from app.core.config import settings
+from app.db.session import get_db
 from app.schemas.ai import (
     AIProviderStatusResponse,
     SituationalAnalysisRequest,
@@ -14,11 +16,27 @@ from app.schemas.ai import (
 
 
 class AIProviderClient:
-    def __init__(self) -> None:
-        self.service_url = settings.AI_PROVIDER_SERVICE_URL.rstrip("/")
-        self.service_token = settings.AI_PROVIDER_SERVICE_TOKEN
-        self.timeout = settings.AI_PROVIDER_TIMEOUT_SECONDS
-        self.retry_attempts = max(settings.AI_PROVIDER_RETRY_ATTEMPTS, 1)
+    def __init__(
+        self,
+        *,
+        service_url: str | None = None,
+        service_token: str | None = None,
+        timeout: int | None = None,
+        retry_attempts: int | None = None,
+        llm_config: dict | None = None,
+    ) -> None:
+        self.service_url = (
+            service_url if service_url is not None else settings.AI_PROVIDER_SERVICE_URL
+        ).rstrip("/")
+        self.service_token = (
+            service_token if service_token is not None else settings.AI_PROVIDER_SERVICE_TOKEN
+        )
+        self.timeout = timeout if timeout is not None else settings.AI_PROVIDER_TIMEOUT_SECONDS
+        self.retry_attempts = max(
+            retry_attempts if retry_attempts is not None else settings.AI_PROVIDER_RETRY_ATTEMPTS,
+            1,
+        )
+        self.llm_config = llm_config or {}
 
     def _headers(self, request_id: str | None = None) -> dict[str, str]:
         headers = {"Content-Type": "application/json"}
@@ -26,6 +44,19 @@ class AIProviderClient:
             headers["X-Provider-Token"] = self.service_token
         if request_id:
             headers["X-Request-ID"] = request_id
+        llm_header_map = {
+            "provider": "X-AI-Provider",
+            "provider_api": "X-AI-Provider-API",
+            "base_url": "X-AI-Base-URL",
+            "api_key": "X-AI-API-Key",
+            "model": "X-AI-Model",
+            "max_tokens": "X-AI-Max-Tokens",
+            "anthropic_version": "X-AI-Anthropic-Version",
+        }
+        for key, header_name in llm_header_map.items():
+            value = self.llm_config.get(key)
+            if value not in (None, ""):
+                headers[header_name] = str(value)
         return headers
 
     async def get_status(self, request_id: str | None = None) -> AIProviderStatusResponse:
@@ -105,5 +136,14 @@ class AIProviderClient:
         )
 
 
-def get_ai_provider_client() -> AIProviderClient:
-    return AIProviderClient()
+async def get_ai_provider_client(db: AsyncSession = Depends(get_db)) -> AIProviderClient:
+    from app.api.v1.settings import get_runtime_ai_provider_config
+
+    runtime_config = await get_runtime_ai_provider_config(db)
+    return AIProviderClient(
+        service_url=runtime_config["service_url"],
+        service_token=runtime_config["service_token"],
+        timeout=runtime_config["timeout_seconds"],
+        retry_attempts=runtime_config["retry_attempts"],
+        llm_config=runtime_config.get("llm_config") or {},
+    )

backend/app/services/barentswatch.py

@@ -0,0 +1,209 @@
+"""BarentsWatch AIS credential resolution and connectivity checks."""
+
+from __future__ import annotations
+
+import os
+import shlex
+from dataclasses import dataclass
+from pathlib import Path
+from typing import Any
+
+import httpx
+from sqlalchemy import select
+from sqlalchemy.ext.asyncio import AsyncSession
+
+from app.core.data_sources import get_data_sources_config
+from app.models.datasource_config import DataSourceConfig
+
+
+BARENTSWATCH_LATEST_URL = "https://live.ais.barentswatch.no/v1/latest/combined"
+BARENTSWATCH_TOKEN_URL = "https://id.barentswatch.no/connect/token"
+BARENTSWATCH_DATASOURCE_NAME = "barentswatch_vessels"
+
+
+@dataclass(frozen=True)
+class BarentsWatchConfig:
+    endpoint: str
+    client_id: str
+    client_secret: str
+    credential_source: str
+    endpoint_source: str
+
+
+def _read_zshrc_env(path: Path | None = None) -> dict[str, str]:
+    zshrc_path = path or Path.home() / ".zshrc"
+    if not zshrc_path.exists():
+        return {}
+
+    values: dict[str, str] = {}
+    for raw_line in zshrc_path.read_text(encoding="utf-8", errors="ignore").splitlines():
+        line = raw_line.strip()
+        if not line or line.startswith("#"):
+            continue
+        if line.startswith("export "):
+            line = line[len("export ") :].strip()
+        if "=" not in line:
+            continue
+
+        key, value = line.split("=", 1)
+        key = key.strip()
+        if not key or not key.replace("_", "").isalnum() or not key[0].isalpha():
+            continue
+
+        try:
+            parsed = shlex.split(value, comments=True, posix=True)
+        except ValueError:
+            parsed = [value.strip().strip("'\"")]
+        if parsed:
+            values[key] = parsed[0]
+    return values
+
+
+def _first_env_value(zshrc_env: dict[str, str], *keys: str) -> tuple[str, str]:
+    for key in keys:
+        value = os.getenv(key)
+        if value:
+            return value, "environment"
+    for key in keys:
+        value = zshrc_env.get(key)
+        if value:
+            return value, "~/.zshrc"
+    return "", ""
+
+
+async def get_barentswatch_datasource_record(db: AsyncSession) -> DataSourceConfig | None:
+    result = await db.execute(
+        select(DataSourceConfig)
+        .where(DataSourceConfig.name == BARENTSWATCH_DATASOURCE_NAME)
+        .where(DataSourceConfig.is_active.is_(True))
+    )
+    return result.scalar_one_or_none()
+
+
+async def resolve_barentswatch_config(db: AsyncSession | None = None) -> BarentsWatchConfig:
+    record = await get_barentswatch_datasource_record(db) if db else None
+    auth_config = dict(record.auth_config or {}) if record else {}
+    config = dict(record.config or {}) if record else {}
+    zshrc_env = _read_zshrc_env()
+
+    env_client_id, env_source = _first_env_value(
+        zshrc_env,
+        "BARENTSWATCH_CLIENT_ID",
+        "BARRENTSWATCH_CLIENT_ID",
+    )
+    env_client_secret, secret_env_source = _first_env_value(
+        zshrc_env,
+        "BARENTSWATCH_CLIENT_SECRET",
+        "BARRENTSWATCH_CLIENT_SECRET",
+    )
+    client_id = auth_config.get("client_id") or config.get("client_id") or env_client_id
+    client_secret = (
+        auth_config.get("client_secret") or config.get("client_secret") or env_client_secret
+    )
+
+    credential_source = ""
+    if auth_config.get("client_id") or auth_config.get("client_secret"):
+        credential_source = "datasource_config"
+    elif config.get("client_id") or config.get("client_secret"):
+        credential_source = "datasource_runtime_config"
+    elif env_source or secret_env_source:
+        credential_source = env_source or secret_env_source
+
+    yaml_endpoint = get_data_sources_config().get_yaml_url(BARENTSWATCH_DATASOURCE_NAME)
+    endpoint = record.endpoint if record and record.endpoint else yaml_endpoint
+    return BarentsWatchConfig(
+        endpoint=endpoint or BARENTSWATCH_LATEST_URL,
+        client_id=str(client_id or ""),
+        client_secret=str(client_secret or ""),
+        credential_source=credential_source or "missing",
+        endpoint_source="datasource_config" if record and record.endpoint else "default",
+    )
+
+
+async def fetch_barentswatch_access_token(
+    client: httpx.AsyncClient,
+    config: BarentsWatchConfig,
+) -> str | None:
+    if not config.client_id or not config.client_secret:
+        return None
+
+    response = await client.post(
+        BARENTSWATCH_TOKEN_URL,
+        data={
+            "client_id": config.client_id,
+            "client_secret": config.client_secret,
+            "scope": "ais",
+            "grant_type": "client_credentials",
+        },
+        headers={"Content-Type": "application/x-www-form-urlencoded"},
+    )
+    response.raise_for_status()
+    payload = response.json()
+    token = payload.get("access_token")
+    return str(token) if token else None
+
+
+async def check_barentswatch_connectivity(db: AsyncSession) -> dict[str, Any]:
+    config = await resolve_barentswatch_config(db)
+    return await check_barentswatch_config(config)
+
+
+async def check_barentswatch_config(config: BarentsWatchConfig) -> dict[str, Any]:
+    if not config.client_id or not config.client_secret:
+        return {
+            "success": False,
+            "stage": "credentials",
+            "message": "未找到 BarentsWatch client id/client secret，请先配置采集器凭证。",
+            "endpoint": config.endpoint,
+            "credential_source": config.credential_source,
+            "settings_tab": "collector_credentials",
+        }
+
+    try:
+        async with httpx.AsyncClient(timeout=20.0) as client:
+            token = await fetch_barentswatch_access_token(client, config)
+            if not token:
+                return {
+                    "success": False,
+                    "stage": "token",
+                    "message": "BarentsWatch token 响应中没有 access_token，请检查凭证。",
+                    "endpoint": config.endpoint,
+                    "credential_source": config.credential_source,
+                    "settings_tab": "collector_credentials",
+                }
+
+            async with client.stream(
+                "GET",
+                config.endpoint,
+                headers={"Authorization": f"Bearer {token}"},
+            ) as response:
+                response.raise_for_status()
+
+        return {
+            "success": True,
+            "stage": "endpoint",
+            "message": "BarentsWatch AIS token 和数据接口均可连通。",
+            "endpoint": config.endpoint,
+            "credential_source": config.credential_source,
+            "endpoint_source": config.endpoint_source,
+        }
+    except httpx.HTTPStatusError as exc:
+        status_code = exc.response.status_code
+        stage = "token" if str(exc.request.url) == BARENTSWATCH_TOKEN_URL else "endpoint"
+        return {
+            "success": False,
+            "stage": stage,
+            "message": f"BarentsWatch {stage} 请求返回 HTTP {status_code}，请检查凭证或接口地址。",
+            "endpoint": config.endpoint,
+            "credential_source": config.credential_source,
+            "settings_tab": "collector_credentials",
+        }
+    except httpx.HTTPError as exc:
+        return {
+            "success": False,
+            "stage": "network",
+            "message": f"BarentsWatch 链路检查失败：{exc.__class__.__name__}",
+            "endpoint": config.endpoint,
+            "credential_source": config.credential_source,
+            "settings_tab": "collector_credentials",
+        }

backend/app/services/collectors/__init__.py

@@ -36,6 +36,7 @@ from app.services.collectors.iptoasn import IPtoASNPrefixGeoCollector
 from app.services.collectors.opengeofeed import OpenGeoFeedPrefixGeoCollector
 from app.services.collectors.nro_delegated import NRODelegatedPrefixGeoCollector
 from app.services.collectors.news_live_streams import NewsLiveStreamsCollector
+from app.services.collectors.vessel_ais import VesselAISCollector
 
 collector_registry.register(TOP500Collector())
 collector_registry.register(EpochAIGPUCollector())
@@ -63,3 +64,4 @@ collector_registry.register(IPtoASNPrefixGeoCollector())
 collector_registry.register(OpenGeoFeedPrefixGeoCollector())
 collector_registry.register(NRODelegatedPrefixGeoCollector())
 collector_registry.register(NewsLiveStreamsCollector())
+collector_registry.register(VesselAISCollector())

backend/app/services/collectors/celestrak.py

@@ -46,6 +46,9 @@ class CelesTrakTLECollector(BaseCollector):
                     if response.status_code == 200:
                         data = response.json()
                         if isinstance(data, list):
+                            for item in data:
+                                if isinstance(item, dict):
+                                    item["_celestrak_group"] = group
                             all_satellites.extend(data)
                             print(f"CelesTrak: Fetched {len(data)} satellites from group '{group}'")
                 except Exception as e:
@@ -78,6 +81,7 @@ class CelesTrakTLECollector(BaseCollector):
                     "name": item.get("OBJECT_NAME", "Unknown"),
                     "reference_date": item.get("EPOCH", ""),
                     "metadata": {
+                        "constellation_group": item.get("_celestrak_group"),
                         "norad_cat_id": item.get("NORAD_CAT_ID"),
                         "international_designator": item.get("OBJECT_ID"),
                         "epoch": item.get("EPOCH"),

backend/app/services/collectors/news_live_streams.py

@@ -1,10 +1,16 @@
 from __future__ import annotations
 
+import asyncio
+import base64
 from datetime import UTC, datetime
 from typing import Any
+from urllib.parse import urlparse
 
 import httpx
+from sqlalchemy import select
 
+from app.core.data_sources import get_data_sources_config
+from app.models.datasource_config import DataSourceConfig
 from app.services.collectors.base import BaseCollector
 
 
@@ -18,52 +24,537 @@ class NewsLiveStreamsCollector(BaseCollector):
     data_type = "news_live_stream"
     fail_on_empty = False
 
+    DEFAULT_TIMEOUT = 45.0
+    DEFAULT_HEADERS = {
+        "User-Agent": "Planet-Intelligence-System/1.0 (Python/collector)",
+        "Accept": "application/json",
+    }
+    RESPONSE_CANDIDATE_KEYS = ("sources", "streams", "channels", "items", "results", "data")
+    DEFAULT_ADAPTER = "iptv_org"
+    DEFAULT_IPTV_ORG_STREAMS_URL = "https://iptv-org.github.io/api/streams.json"
+    DEFAULT_IPTV_ORG_LOGOS_URL = "https://iptv-org.github.io/api/logos.json"
+    DEFAULT_IPTV_ORG_NEWS_CATEGORIES = ("news", "business", "weather")
+    DEFAULT_IPTV_ORG_EXCLUDE_CATEGORIES = ("music", "sports", "kids", "entertainment")
+    DEFAULT_IPTV_ORG_MAX_SOURCES = 120
+
     async def fetch(self) -> list[dict[str, Any]]:
         request_url = (self._resolved_url or "").strip()
         if not request_url:
             return []
 
-        async with httpx.AsyncClient(timeout=45.0, follow_redirects=True) as client:
-            response = await client.get(
+        datasource_config = await self._load_datasource_config()
+        effective_config = self._get_effective_config(datasource_config)
+        adapter = str(effective_config.get("adapter") or "").strip().lower()
+        if adapter == "iptv_org":
+            return await self._fetch_iptv_org(request_url, effective_config)
+
+        request_headers = self._build_request_headers(datasource_config)
+        request_config = self._get_request_config(datasource_config)
+        request_params = self._build_request_params(datasource_config)
+        request_json = self._build_request_json_body(datasource_config)
+        request_data = self._build_request_form_body(datasource_config)
+        timeout = self._get_timeout(datasource_config)
+
+        async with httpx.AsyncClient(timeout=timeout, follow_redirects=True) as client:
+            response = await client.request(
+                request_config["method"],
                 request_url,
-                headers={
-                    "User-Agent": "Planet-Intelligence-System/1.0 (Python/collector)",
-                    "Accept": "application/json",
-                },
+                headers=request_headers,
+                params=request_params or None,
+                json=request_json,
+                data=request_data,
             )
             response.raise_for_status()
-            return self.parse_response(response.json())
+            return self.parse_response(
+                response.json(),
+                response_path=request_config["response_path"],
+            )
 
-    def parse_response(self, response: Any) -> list[dict[str, Any]]:
-        if isinstance(response, dict):
-            candidates = response.get("sources") or response.get("streams") or response.get("data") or []
-        elif isinstance(response, list):
-            candidates = response
+    async def _load_datasource_config(self) -> DataSourceConfig | None:
+        if not self._db_session:
+            return None
+
+        result = await self._db_session.execute(
+            select(DataSourceConfig)
+            .where(DataSourceConfig.name == self.name)
+            .where(DataSourceConfig.is_active.is_(True))
+            .limit(1)
+        )
+        return result.scalar_one_or_none()
+
+    def _get_effective_config(self, datasource_config: DataSourceConfig | None) -> dict[str, Any]:
+        payload = dict(datasource_config.config or {}) if datasource_config else {}
+        if payload:
+            return payload
+
+        yaml_config = get_data_sources_config()
+        return {
+            "adapter": self.DEFAULT_ADAPTER,
+            "streams_url": yaml_config.get_yaml_value("news_live_streams.streams_url")
+            or self.DEFAULT_IPTV_ORG_STREAMS_URL,
+            "logos_url": yaml_config.get_yaml_value("news_live_streams.logos_url")
+            or self.DEFAULT_IPTV_ORG_LOGOS_URL,
+            "news_categories": list(self.DEFAULT_IPTV_ORG_NEWS_CATEGORIES),
+            "exclude_categories": list(self.DEFAULT_IPTV_ORG_EXCLUDE_CATEGORIES),
+            "max_sources": self.DEFAULT_IPTV_ORG_MAX_SOURCES,
+        }
+
+    def _get_request_config(self, datasource_config: DataSourceConfig | None) -> dict[str, Any]:
+        payload = self._get_effective_config(datasource_config)
+        raw_method = payload.get("method") or payload.get("request_method") or "GET"
+        method = str(raw_method).strip().upper() or "GET"
+        if method not in {"GET", "POST"}:
+            method = "GET"
+
+        response_path = payload.get("response_path") or payload.get("payload_path") or payload.get("items_path")
+        if isinstance(response_path, str):
+            response_path = response_path.strip()
         else:
-            candidates = []
+            response_path = None
+
+        return {
+            "method": method,
+            "response_path": response_path or None,
+        }
+
+    def _get_timeout(self, datasource_config: DataSourceConfig | None) -> float:
+        payload = self._get_effective_config(datasource_config)
+        try:
+            return float(payload.get("timeout", self.DEFAULT_TIMEOUT))
+        except (TypeError, ValueError):
+            return self.DEFAULT_TIMEOUT
+
+    def _build_request_headers(self, datasource_config: DataSourceConfig | None) -> dict[str, str]:
+        headers = dict(self.DEFAULT_HEADERS)
+        if datasource_config:
+            headers.update(self._normalize_headers(datasource_config.headers))
+            headers.update(self._build_auth_headers(datasource_config))
+        return headers
+
+    def _build_request_params(self, datasource_config: DataSourceConfig | None) -> dict[str, Any]:
+        params: dict[str, Any] = {}
+        if not datasource_config:
+            return params
+
+        payload = datasource_config.config or {}
+        candidate = payload.get("params") or payload.get("query_params")
+        if isinstance(candidate, dict):
+            params.update(candidate)
+
+        if datasource_config.auth_type == "api_key":
+            auth_config = datasource_config.auth_config or {}
+            if str(auth_config.get("in") or auth_config.get("location") or "header").lower() == "query":
+                api_key = auth_config.get("api_key")
+                key_name = auth_config.get("key_name") or auth_config.get("param_name") or "api_key"
+                if api_key and key_name:
+                    params[str(key_name)] = api_key
+
+        return params
+
+    def _build_request_json_body(self, datasource_config: DataSourceConfig | None) -> Any:
+        if not datasource_config:
+            return None
+
+        payload = datasource_config.config or {}
+        body = payload.get("json_body")
+        if body is None and str(payload.get("body_type") or "").lower() in {"json", ""}:
+            candidate = payload.get("body")
+            if isinstance(candidate, (dict, list)):
+                body = candidate
+        return body
+
+    def _build_request_form_body(self, datasource_config: DataSourceConfig | None) -> Any:
+        if not datasource_config:
+            return None
+
+        payload = datasource_config.config or {}
+        form_body = payload.get("form_body")
+        if form_body is not None:
+            return form_body
+
+        if str(payload.get("body_type") or "").lower() == "form":
+            candidate = payload.get("body")
+            if isinstance(candidate, dict):
+                return candidate
+        return None
+
+    def _normalize_headers(self, headers: Any) -> dict[str, str]:
+        if not isinstance(headers, dict):
+            return {}
+        normalized: dict[str, str] = {}
+        for key, value in headers.items():
+            header_name = str(key).strip()
+            if not header_name or value is None:
+                continue
+            normalized[header_name] = str(value)
+        return normalized
+
+    def _build_auth_headers(self, datasource_config: DataSourceConfig | None) -> dict[str, str]:
+        if not datasource_config:
+            return {}
+
+        auth_type = str(datasource_config.auth_type or "none").lower()
+        auth_config = datasource_config.auth_config or {}
+        if auth_type == "bearer" and auth_config.get("token"):
+            return {"Authorization": f"Bearer {auth_config['token']}"}
+
+        if auth_type == "api_key" and auth_config.get("api_key"):
+            location = str(auth_config.get("in") or auth_config.get("location") or "header").lower()
+            if location == "query":
+                return {}
+            key_name = auth_config.get("key_name") or "X-API-Key"
+            return {str(key_name): str(auth_config["api_key"])}
+
+        if auth_type == "basic":
+            username = str(auth_config.get("username") or "")
+            password = str(auth_config.get("password") or "")
+            encoded = base64.b64encode(f"{username}:{password}".encode()).decode()
+            return {"Authorization": f"Basic {encoded}"}
+
+        return {}
+
+    def _extract_candidates(self, response: Any, response_path: str | None) -> list[Any]:
+        if response_path:
+            extracted = self._extract_from_path(response, response_path)
+            if isinstance(extracted, list):
+                return extracted
+            if isinstance(extracted, dict):
+                for key in self.RESPONSE_CANDIDATE_KEYS:
+                    nested = extracted.get(key)
+                    if isinstance(nested, list):
+                        return nested
+                return [extracted]
+
+        if isinstance(response, dict):
+            for key in self.RESPONSE_CANDIDATE_KEYS:
+                nested = response.get(key)
+                if isinstance(nested, list):
+                    return nested
+            return []
+
+        if isinstance(response, list):
+            return response
+        return []
+
+    def _extract_from_path(self, payload: Any, path: str) -> Any:
+        current = payload
+        for segment in (part.strip() for part in path.split(".") if part.strip()):
+            if isinstance(current, dict):
+                current = current.get(segment)
+                continue
+            if isinstance(current, list):
+                try:
+                    current = current[int(segment)]
+                except (TypeError, ValueError, IndexError):
+                    return None
+                continue
+            return None
+        return current
+
+    def _infer_source_type(self, item: dict[str, Any]) -> str:
+        explicit = str(item.get("source_type") or item.get("type") or "").strip().lower()
+        if explicit in {"iframe", "hls", "video", "external", "youtube"}:
+            return explicit
+
+        youtube_video_id = self._clean_text(
+            item.get("youtube_video_id")
+            or item.get("video_id")
+            or item.get("youtubeVideoId")
+        )
+        youtube_channel = self._clean_text(item.get("youtube_channel") or item.get("channel_handle"))
+        embed_url = self._clean_url(item.get("embed_url") or item.get("embed") or item.get("page_url"))
+        stream_url = self._clean_url(item.get("stream_url") or item.get("stream") or item.get("playback_url") or item.get("hls_url"))
+        homepage_url = self._clean_url(item.get("homepage_url") or item.get("source_url") or item.get("website"))
+
+        if youtube_video_id or youtube_channel:
+            return "youtube"
+        if stream_url.endswith(".m3u8"):
+            return "hls"
+        if stream_url:
+            return "video"
+        if embed_url:
+            parsed = urlparse(embed_url)
+            if "youtube.com" in (parsed.netloc or "") or "youtu.be" in (parsed.netloc or ""):
+                return "youtube"
+            return "iframe"
+        if homepage_url:
+            return "external"
+        return "iframe"
+
+    def _parse_enabled(self, item: dict[str, Any]) -> bool:
+        if "is_enabled" in item:
+            return self._to_bool(item.get("is_enabled"), default=True)
+        if "enabled" in item:
+            return self._to_bool(item.get("enabled"), default=True)
+        if "active" in item:
+            return self._to_bool(item.get("active"), default=True)
+        if "status" in item:
+            status = str(item.get("status") or "").strip().lower()
+            if status in {"disabled", "inactive", "offline"}:
+                return False
+            if status in {"enabled", "active", "online", "live"}:
+                return True
+        return True
+
+    def _to_bool(self, value: Any, *, default: bool) -> bool:
+        if isinstance(value, bool):
+            return value
+        if value in (None, ""):
+            return default
+        if isinstance(value, str):
+            lowered = value.strip().lower()
+            if lowered in {"1", "true", "yes", "on", "enabled", "active", "online", "live"}:
+                return True
+            if lowered in {"0", "false", "no", "off", "disabled", "inactive", "offline"}:
+                return False
+        return bool(value)
+
+    def _clean_text(self, value: Any) -> str:
+        if value is None:
+            return ""
+        return str(value).strip()
+
+    def _clean_url(self, value: Any) -> str:
+        text = self._clean_text(value)
+        if not text:
+            return ""
+        parsed = urlparse(text)
+        if parsed.scheme and parsed.scheme not in {"http", "https"}:
+            return ""
+        if parsed.scheme and not parsed.netloc:
+            return ""
+        return text
+
+    async def _fetch_iptv_org(self, channels_url: str, collector_config: dict[str, Any]) -> list[dict[str, Any]]:
+        streams_url = self._clean_url(collector_config.get("streams_url")) or self.DEFAULT_IPTV_ORG_STREAMS_URL
+        logos_url = self._clean_url(collector_config.get("logos_url")) or self.DEFAULT_IPTV_ORG_LOGOS_URL
+        news_categories = {
+            self._clean_text(value).lower()
+            for value in (collector_config.get("news_categories") or self.DEFAULT_IPTV_ORG_NEWS_CATEGORIES)
+            if self._clean_text(value)
+        }
+        exclude_categories = {
+            self._clean_text(value).lower()
+            for value in (collector_config.get("exclude_categories") or self.DEFAULT_IPTV_ORG_EXCLUDE_CATEGORIES)
+            if self._clean_text(value)
+        }
+        try:
+            max_sources = int(collector_config.get("max_sources", self.DEFAULT_IPTV_ORG_MAX_SOURCES))
+        except (TypeError, ValueError):
+            max_sources = self.DEFAULT_IPTV_ORG_MAX_SOURCES
+
+        timeout = self.DEFAULT_TIMEOUT
+        try:
+            timeout = float(collector_config.get("timeout", self.DEFAULT_TIMEOUT))
+        except (TypeError, ValueError):
+            timeout = self.DEFAULT_TIMEOUT
+
+        async with httpx.AsyncClient(timeout=timeout, follow_redirects=True) as client:
+            channels_payload, streams_payload, logos_payload = await self._gather_iptv_org_payloads(
+                client,
+                channels_url,
+                streams_url,
+                logos_url,
+            )
+
+        channels = channels_payload if isinstance(channels_payload, list) else []
+        streams = streams_payload if isinstance(streams_payload, list) else []
+        logos = logos_payload if isinstance(logos_payload, list) else []
+
+        logo_by_channel = {
+            self._clean_text(item.get("channel")): self._clean_url(item.get("url"))
+            for item in logos
+            if isinstance(item, dict) and self._clean_text(item.get("channel")) and self._clean_url(item.get("url"))
+        }
+
+        streams_by_channel: dict[str, list[dict[str, Any]]] = {}
+        for stream in streams:
+            if not isinstance(stream, dict):
+                continue
+            channel_id = self._clean_text(stream.get("channel"))
+            if not channel_id:
+                continue
+            streams_by_channel.setdefault(channel_id, []).append(stream)
+
+        normalized: list[dict[str, Any]] = []
+        for channel in channels:
+            if not isinstance(channel, dict):
+                continue
+
+            categories = [
+                self._clean_text(value).lower()
+                for value in (channel.get("categories") or [])
+                if self._clean_text(value)
+            ]
+            if news_categories and not any(category in news_categories for category in categories):
+                continue
+            if exclude_categories and any(category in exclude_categories for category in categories):
+                continue
+            if channel.get("is_nsfw") is True:
+                continue
+            if channel.get("closed"):
+                continue
+
+            channel_id = self._clean_text(channel.get("id"))
+            if not channel_id:
+                continue
+
+            stream = self._pick_iptv_org_stream(streams_by_channel.get(channel_id) or [])
+            if not stream:
+                continue
+
+            stream_url = self._clean_url(stream.get("url"))
+            if not stream_url:
+                continue
+
+            name = self._clean_text(channel.get("name")) or channel_id
+            notes_parts = [
+                f"Imported from IPTV-org catalog ({channel_id})",
+                f"Categories: {', '.join(categories)}" if categories else "",
+                f"Quality: {self._clean_text(stream.get('quality'))}" if self._clean_text(stream.get("quality")) else "",
+            ]
+            metadata = {
+                "provider": self._clean_text(channel.get("network")) or "IPTV-org",
+                "region": self._clean_text(channel.get("country")) or "Global",
+                "language": "und",
+                "source_type": "hls" if stream_url.endswith(".m3u8") else "video",
+                "embed_url": "",
+                "stream_url": stream_url,
+                "homepage_url": self._clean_url(channel.get("website")),
+                "poster_url": logo_by_channel.get(channel_id, ""),
+                "youtube_video_id": "",
+                "youtube_channel": "",
+                "sort_order": 400 + len(normalized),
+                "notes": "; ".join(part for part in notes_parts if part),
+                "is_enabled": True,
+                "collector_adapter": "iptv_org",
+                "channel_id": channel_id,
+                "categories": categories,
+                "quality": self._clean_text(stream.get("quality")),
+                "stream_label": self._clean_text(stream.get("label") or stream.get("title")),
+                "stream_referrer": self._clean_text(stream.get("referrer")),
+                "stream_user_agent": self._clean_text(stream.get("user_agent")),
+            }
+
+            normalized.append(
+                {
+                    "source_id": channel_id,
+                    "name": name,
+                    "description": metadata["notes"],
+                    "metadata": metadata,
+                    "reference_date": datetime.now(UTC).isoformat(),
+                }
+            )
+            if len(normalized) >= max_sources:
+                break
+
+        return normalized
+
+    async def _gather_iptv_org_payloads(
+        self,
+        client: httpx.AsyncClient,
+        channels_url: str,
+        streams_url: str,
+        logos_url: str,
+    ) -> tuple[Any, Any, Any]:
+        headers = dict(self.DEFAULT_HEADERS)
+        channels_payload, streams_payload, logos_payload = await asyncio.gather(
+            client.get(channels_url, headers=headers),
+            client.get(streams_url, headers=headers),
+            client.get(logos_url, headers=headers),
+        )
+        channels_payload.raise_for_status()
+        streams_payload.raise_for_status()
+        logos_payload.raise_for_status()
+        return channels_payload.json(), streams_payload.json(), logos_payload.json()
+
+    def _pick_iptv_org_stream(self, streams: list[dict[str, Any]]) -> dict[str, Any] | None:
+        if not streams:
+            return None
+
+        def score(stream: dict[str, Any]) -> tuple[int, int]:
+            url = self._clean_url(stream.get("url"))
+            quality = self._clean_text(stream.get("quality")).lower()
+            quality_score = 0
+            if quality.endswith("p"):
+                try:
+                    quality_score = int(quality[:-1])
+                except ValueError:
+                    quality_score = 0
+            stream_score = 1000 if url.endswith(".m3u8") else 0
+            return stream_score, quality_score
+
+        sorted_streams = sorted(streams, key=score, reverse=True)
+        return sorted_streams[0]
+
+    def parse_response(self, response: Any, *, response_path: str | None = None) -> list[dict[str, Any]]:
+        candidates = self._extract_candidates(response, response_path)
 
         normalized: list[dict[str, Any]] = []
         for index, item in enumerate(candidates):
             if not isinstance(item, dict):
                 continue
 
-            stream_id = item.get("id") or item.get("source_id") or item.get("slug") or f"news-live-{index + 1}"
-            name = str(item.get("name") or item.get("title") or f"News Live {index + 1}").strip()
+            stream_id = (
+                item.get("id")
+                or item.get("source_id")
+                or item.get("slug")
+                or item.get("channel_id")
+                or item.get("code")
+                or f"news-live-{index + 1}"
+            )
+            name = self._clean_text(
+                item.get("name")
+                or item.get("title")
+                or item.get("channel")
+                or item.get("display_name")
+                or f"News Live {index + 1}"
+            )
             if not name:
                 continue
 
+            source_type = self._infer_source_type(item)
+            stream_url = self._clean_url(
+                item.get("stream_url")
+                or item.get("stream")
+                or item.get("playback_url")
+                or item.get("hls_url")
+                or item.get("m3u8_url")
+            )
+            embed_url = self._clean_url(
+                item.get("embed_url")
+                or item.get("embed")
+                or item.get("page_url")
+                or (item.get("url") if source_type == "iframe" else "")
+            )
+            homepage_url = self._clean_url(
+                item.get("homepage_url")
+                or item.get("source_url")
+                or item.get("website")
+                or item.get("url")
+            )
             metadata = {
-                "provider": item.get("provider") or item.get("publisher") or "Collector",
-                "region": item.get("region") or item.get("country") or "Global",
-                "language": item.get("language") or "und",
-                "source_type": item.get("source_type") or "iframe",
-                "embed_url": item.get("embed_url") or item.get("url") or "",
-                "stream_url": item.get("stream_url") or "",
-                "homepage_url": item.get("homepage_url") or item.get("source_url") or "",
-                "poster_url": item.get("poster_url") or "",
+                "provider": self._clean_text(item.get("provider") or item.get("publisher") or item.get("network")) or "Collector",
+                "region": self._clean_text(item.get("region") or item.get("country") or item.get("market")) or "Global",
+                "language": self._clean_text(item.get("language") or item.get("lang") or item.get("locale")) or "und",
+                "source_type": source_type,
+                "embed_url": embed_url,
+                "stream_url": stream_url,
+                "homepage_url": homepage_url,
+                "poster_url": self._clean_url(item.get("poster_url") or item.get("thumbnail_url") or item.get("logo_url")),
+                "youtube_video_id": self._clean_text(
+                    item.get("youtube_video_id")
+                    or item.get("video_id")
+                    or item.get("youtubeVideoId")
+                ),
+                "youtube_channel": self._clean_text(
+                    item.get("youtube_channel")
+                    or item.get("channel_handle")
+                    or item.get("youtubeChannel")
+                ),
                 "sort_order": item.get("sort_order", 200 + index),
-                "notes": item.get("notes") or item.get("description") or "",
-                "is_enabled": item.get("is_enabled", True),
+                "notes": self._clean_text(item.get("notes") or item.get("description") or item.get("summary")),
+                "is_enabled": self._parse_enabled(item),
             }
 
             normalized.append(
@@ -72,7 +563,7 @@ class NewsLiveStreamsCollector(BaseCollector):
                     "name": name,
                     "description": metadata["notes"],
                     "metadata": metadata,
-                    "reference_date": item.get("reference_date", datetime.now(UTC).isoformat()),
+                    "reference_date": item.get("reference_date") or datetime.now(UTC).isoformat(),
                 }
             )
 

backend/app/services/collectors/vessel_ais.py

@@ -0,0 +1,273 @@
+"""BarentsWatch AIS collector for vessel tracking."""
+
+from datetime import UTC, datetime, timedelta
+from typing import Any
+
+import httpx
+from sqlalchemy import delete, select
+from sqlalchemy.ext.asyncio import AsyncSession
+
+from app.models.vessel import VesselPosition, VesselStatic
+from app.services.barentswatch import (
+    BARENTSWATCH_LATEST_URL,
+    fetch_barentswatch_access_token,
+    resolve_barentswatch_config,
+)
+from app.services.collectors.base import BaseCollector
+
+
+VESSEL_TYPE_NAMES = {
+    30: "Fishing",
+    35: "Military",
+    60: "Passenger",
+    70: "Cargo",
+    80: "Tanker",
+}
+
+
+class VesselAISCollector(BaseCollector):
+    """Collect latest AIS positions and append them to vessel tables."""
+
+    name = "barentswatch_vessels"
+    priority = "P1"
+    module = "L4"
+    frequency_hours = 1
+    data_type = "vessel_ais"
+
+    @property
+    def base_url(self) -> str:
+        return self._resolved_url or BARENTSWATCH_LATEST_URL
+
+    async def _get_access_token(self, client: httpx.AsyncClient) -> str | None:
+        config = await resolve_barentswatch_config(self._db_session)
+        return await fetch_barentswatch_access_token(client, config)
+
+    async def fetch(self) -> list[dict[str, Any]]:
+        async with httpx.AsyncClient(timeout=60.0) as client:
+            headers: dict[str, str] = {}
+            token = await self._get_access_token(client)
+            if token:
+                headers["Authorization"] = f"Bearer {token}"
+
+            response = await client.get(self.base_url, headers=headers)
+            if response.status_code == 401 and not token:
+                return self._get_sample_data()
+            response.raise_for_status()
+            payload = response.json()
+
+        if isinstance(payload, list):
+            return [item for item in payload if isinstance(item, dict)]
+        if isinstance(payload, dict):
+            for key in ("features", "data", "items", "vessels"):
+                value = payload.get(key)
+                if isinstance(value, list):
+                    if key == "features":
+                        return [
+                            {
+                                **(item.get("properties") or {}),
+                                "geometry": item.get("geometry"),
+                            }
+                            for item in value
+                            if isinstance(item, dict)
+                        ]
+                    return [item for item in value if isinstance(item, dict)]
+        return self._get_sample_data()
+
+    def transform(self, raw_data: list[dict[str, Any]]) -> list[dict[str, Any]]:
+        transformed = []
+        for item in raw_data:
+            record = self._normalize_record(item)
+            if record:
+                transformed.append(record)
+        return transformed
+
+    async def _save_data(
+        self,
+        db: AsyncSession,
+        data: list[dict[str, Any]],
+        task_id: int | None = None,
+        snapshot_id: int | None = None,
+    ) -> int:
+        now = datetime.now(UTC)
+        records_added = 0
+
+        for index, item in enumerate(data):
+            static = await db.get(VesselStatic, item["mmsi"])
+            if static is None:
+                static = VesselStatic(mmsi=item["mmsi"])
+                db.add(static)
+
+            for field in (
+                "name",
+                "callsign",
+                "vessel_type",
+                "vessel_type_name",
+                "flag",
+                "length",
+                "width",
+                "draught",
+                "imo",
+            ):
+                value = item.get(field)
+                if value not in (None, ""):
+                    setattr(static, field, value)
+            static.updated_at = now
+
+            db.add(
+                VesselPosition(
+                    mmsi=item["mmsi"],
+                    lat=item["lat"],
+                    lon=item["lon"],
+                    sog=item.get("sog"),
+                    cog=item.get("cog"),
+                    heading=item.get("heading"),
+                    nav_status=item.get("nav_status"),
+                    received_at=item.get("received_at") or now,
+                )
+            )
+            records_added += 1
+
+            if (index + 1) % 1000 == 0:
+                await self.update_progress(index + 1, commit=True)
+
+        await db.execute(
+            delete(VesselPosition).where(VesselPosition.received_at < now - timedelta(hours=24))
+        )
+        await db.commit()
+        await self.update_progress(records_added, force=True)
+        return records_added
+
+    def _normalize_record(self, item: dict[str, Any]) -> dict[str, Any] | None:
+        mmsi = _as_int(_pick(item, "mmsi", "MMSI", "Mmsi"))
+        lat = _as_float(_pick(item, "lat", "latitude", "Latitude"))
+        lon = _as_float(_pick(item, "lon", "lng", "longitude", "Longitude"))
+
+        geometry = item.get("geometry")
+        coordinates = geometry.get("coordinates") if isinstance(geometry, dict) else None
+        if (lat is None or lon is None) and isinstance(coordinates, list) and len(coordinates) >= 2:
+            lon = _as_float(coordinates[0])
+            lat = _as_float(coordinates[1])
+
+        if mmsi is None or lat is None or lon is None:
+            return None
+        if not (-90 <= lat <= 90 and -180 <= lon <= 180):
+            return None
+
+        vessel_type = _as_int(_pick(item, "vessel_type", "shipType", "ship_type", "ShipType"))
+        vessel_type_name = (
+            _pick(item, "vessel_type_name", "shipTypeName", "ship_type_name", "VesselTypeName")
+            or _vessel_type_name(vessel_type)
+        )
+        received_at = _parse_datetime(_pick(item, "received_at", "timestamp", "time", "msgtime"))
+
+        return {
+            "mmsi": mmsi,
+            "name": _pick(item, "name", "shipName", "ship_name", "Name"),
+            "callsign": _pick(item, "callsign", "callSign", "CallSign"),
+            "vessel_type": vessel_type,
+            "vessel_type_name": vessel_type_name,
+            "flag": _pick(item, "flag", "country", "Flag"),
+            "length": _as_float(_pick(item, "length", "shipLength", "Length")),
+            "width": _as_float(_pick(item, "width", "shipWidth", "Width")),
+            "draught": _as_float(_pick(item, "draught", "draft", "Draught")),
+            "imo": _as_int(_pick(item, "imo", "IMO", "imoNumber")),
+            "lat": lat,
+            "lon": lon,
+            "sog": _as_float(_pick(item, "sog", "speedOverGround", "SOG")),
+            "cog": _as_float(_pick(item, "cog", "courseOverGround", "COG")),
+            "heading": _as_int(_pick(item, "heading", "trueHeading", "Heading")),
+            "nav_status": _as_int(_pick(item, "nav_status", "navStatus", "NavigationalStatus")),
+            "received_at": received_at,
+        }
+
+    def _get_sample_data(self) -> list[dict[str, Any]]:
+        return [
+            {
+                "mmsi": 257123000,
+                "name": "OSLO TRADER",
+                "lat": 59.91,
+                "lon": 10.73,
+                "sog": 12.4,
+                "cog": 214,
+                "heading": 215,
+                "nav_status": 0,
+                "vessel_type": 70,
+                "vessel_type_name": "Cargo",
+                "flag": "NO",
+                "length": 185,
+            },
+            {
+                "mmsi": 257456000,
+                "name": "NORDIC FJORD",
+                "lat": 60.39,
+                "lon": 5.32,
+                "sog": 0.2,
+                "cog": 82,
+                "heading": 80,
+                "nav_status": 1,
+                "vessel_type": 60,
+                "vessel_type_name": "Passenger",
+                "flag": "NO",
+                "length": 126,
+            },
+        ]
+
+
+def _pick(item: dict[str, Any], *keys: str) -> Any:
+    for key in keys:
+        if key in item and item[key] not in (None, ""):
+            return item[key]
+    return None
+
+
+def _as_float(value: Any) -> float | None:
+    try:
+        if value in (None, ""):
+            return None
+        return float(value)
+    except (TypeError, ValueError):
+        return None
+
+
+def _as_int(value: Any) -> int | None:
+    try:
+        if value in (None, ""):
+            return None
+        return int(float(value))
+    except (TypeError, ValueError):
+        return None
+
+
+def _parse_datetime(value: Any) -> datetime | None:
+    if isinstance(value, datetime):
+        return value if value.tzinfo else value.replace(tzinfo=UTC)
+    if not value:
+        return None
+    if isinstance(value, (int, float)):
+        timestamp = float(value)
+        if timestamp > 10_000_000_000:
+            timestamp /= 1000
+        return datetime.fromtimestamp(timestamp, UTC)
+    if isinstance(value, str):
+        try:
+            parsed = datetime.fromisoformat(value.replace("Z", "+00:00"))
+            return parsed if parsed.tzinfo else parsed.replace(tzinfo=UTC)
+        except ValueError:
+            return None
+    return None
+
+
+def _vessel_type_name(vessel_type: int | None) -> str:
+    if vessel_type is None:
+        return "Other"
+    if 70 <= vessel_type <= 79:
+        return "Cargo"
+    if 80 <= vessel_type <= 89:
+        return "Tanker"
+    if 60 <= vessel_type <= 69:
+        return "Passenger"
+    if vessel_type == 30:
+        return "Fishing"
+    if vessel_type == 35:
+        return "Military"
+    return VESSEL_TYPE_NAMES.get(vessel_type, "Other")

backend/app/services/credential_guides.py

@@ -0,0 +1,165 @@
+"""Credential setup guides for collector integrations."""
+
+from __future__ import annotations
+
+from dataclasses import dataclass
+from typing import Any
+
+from sqlalchemy import select
+
+from app.models.system_setting import SystemSetting
+from app.schemas.ai import SituationalAnalysisRequest
+from app.services.ai_client import AIProviderClient
+
+
+CREDENTIAL_GUIDES_CATEGORY = "collector_credential_guides"
+
+
+@dataclass(frozen=True)
+class CredentialGuideDefault:
+    provider: str
+    title: str
+    prompt: str
+    markdown: str
+
+
+BARENTSWATCH_DEFAULT_GUIDE = CredentialGuideDefault(
+    provider="barentswatch",
+    title="BarentsWatch AIS 凭证获取教程",
+    prompt=(
+        "请生成一份中文教程，指导开发者获取 BarentsWatch Live AIS API 的 "
+        "OAuth client credentials。教程要面向已经有本地开发环境的人，包含注册/登录、"
+        "创建 client、申请或确认 ais scope、复制 client id 和 client secret、"
+        "在系统设置中填写并验证连接、常见失败排查。不要编造具体页面按钮文案，"
+        "必须参考官方 tutorial：https://developer.barentswatch.no/docs/tutorial 。"
+        "必须强调 Live AIS 要选择 AIS-client / AIS - API，而不是普通 API-client。"
+        "如果步骤可能变化，要提醒以 BarentsWatch developer portal 当前页面为准。"
+    ),
+    markdown="""## BarentsWatch AIS 凭证获取
+
+官方教程：https://developer.barentswatch.no/docs/tutorial
+
+1. 先打开上面的 BarentsWatch 官方 tutorial，按官方流程登录或注册开发者账号。
+2. 在 Developer access 页面选择 `AIS - API`，不要选择普通的 `BarentsWatch - API`。
+3. 在 `AIS - API` 下创建用于 Planet 的 AIS client。
+4. 创建时记下你设置的 password / client secret。
+5. 回到 My Page 复制完整 `Client ID`。它通常长得像 `your.email@example.com:client-name`。
+6. 回到 Planet 的 `设置 -> 采集器设置 -> BarentsWatch AIS`，填入 `Client ID` 和 `Client Secret`。
+7. 点击 `连接` 验证 token 和 AIS endpoint 是否可访问。
+8. 连接成功后保存凭证。
+
+### 请求规则
+
+- Token 地址：`https://id.barentswatch.no/connect/token`
+- 请求方式：`POST`
+- Content-Type：`application/x-www-form-urlencoded`
+- Body 必须包含：`grant_type=client_credentials`、`client_id`、`client_secret`、`scope=ais`
+- `client_id`、`client_secret`、`scope`、`grant_type` 都要放在 body，不要放在 header。
+- AIS 数据请求使用 header：`Authorization: Bearer <access_token>`
+
+### 常见排查
+
+- `未找到凭证`：确认 `Client ID` 和 `Client Secret` 已填写，或已经写入 `~/.zshrc`。
+- `HTTP 401/403`：通常是选成了普通 `BarentsWatch - API` client、client secret 错误，或 token 请求没有使用 `scope=ais`。
+- `network` 错误：检查本机是否能访问 `id.barentswatch.no` 和 `live.ais.barentswatch.no`。
+- Endpoint 建议保持默认：`https://live.ais.barentswatch.no/v1/latest/combined`。
+""",
+)
+
+
+DEFAULT_CREDENTIAL_GUIDES = {
+    BARENTSWATCH_DEFAULT_GUIDE.provider: BARENTSWATCH_DEFAULT_GUIDE,
+}
+
+
+async def _get_guide_store(db) -> tuple[SystemSetting | None, dict[str, Any]]:
+    result = await db.execute(
+        select(SystemSetting).where(SystemSetting.category == CREDENTIAL_GUIDES_CATEGORY)
+    )
+    record = result.scalar_one_or_none()
+    payload = dict(record.payload or {}) if record and isinstance(record.payload, dict) else {}
+    return record, payload
+
+
+async def get_credential_guide(db, provider: str) -> dict[str, Any]:
+    default = DEFAULT_CREDENTIAL_GUIDES.get(provider)
+    if default is None:
+        raise ValueError(f"Unsupported credential guide provider: {provider}")
+
+    _record, store = await _get_guide_store(db)
+    custom = store.get(provider) if isinstance(store.get(provider), dict) else None
+    return {
+        "provider": provider,
+        "title": custom.get("title") if custom else default.title,
+        "markdown": custom.get("markdown") if custom else default.markdown,
+        "prompt": default.prompt,
+        "source": "ai" if custom else "default",
+    }
+
+
+async def save_credential_guide(db, provider: str, title: str, markdown: str) -> dict[str, Any]:
+    default = DEFAULT_CREDENTIAL_GUIDES.get(provider)
+    if default is None:
+        raise ValueError(f"Unsupported credential guide provider: {provider}")
+
+    record, store = await _get_guide_store(db)
+    store[provider] = {
+        "title": title or default.title,
+        "markdown": markdown,
+    }
+    if record is None:
+        db.add(SystemSetting(category=CREDENTIAL_GUIDES_CATEGORY, payload=store))
+    else:
+        record.payload = store
+    await db.commit()
+    return await get_credential_guide(db, provider)
+
+
+async def reset_credential_guide(db, provider: str) -> dict[str, Any]:
+    default = DEFAULT_CREDENTIAL_GUIDES.get(provider)
+    if default is None:
+        raise ValueError(f"Unsupported credential guide provider: {provider}")
+
+    record, store = await _get_guide_store(db)
+    if provider in store:
+        store.pop(provider, None)
+        if record is not None:
+            record.payload = store
+            await db.commit()
+    return await get_credential_guide(db, provider)
+
+
+async def generate_credential_guide(
+    db,
+    provider: str,
+    ai_client: AIProviderClient,
+) -> dict[str, Any]:
+    default = DEFAULT_CREDENTIAL_GUIDES.get(provider)
+    if default is None:
+        raise ValueError(f"Unsupported credential guide provider: {provider}")
+
+    response = await ai_client.analyze(
+        SituationalAnalysisRequest(
+            title=f"Generate credential guide for {provider}",
+            objective=default.prompt,
+            context={
+                "provider": provider,
+                "current_default_guide": default.markdown,
+                "product_context": "Planet collector credential settings",
+            },
+            observations=[
+                "Use concise Chinese markdown.",
+                "Prefer stable concepts over brittle UI labels.",
+                "Include verification and troubleshooting steps.",
+            ],
+            constraints=[
+                "Do not ask the user for secrets.",
+                "Do not include fabricated screenshots.",
+                "Return markdown only.",
+            ],
+        )
+    )
+    markdown = response.content.strip()
+    if not markdown:
+        markdown = default.markdown
+    return await save_credential_guide(db, provider, default.title, markdown)

backend/app/services/datasource_connectivity.py

@@ -0,0 +1,384 @@
+"""Connectivity validation helpers for built-in datasource overrides."""
+
+from __future__ import annotations
+
+from datetime import UTC, datetime
+import hashlib
+import json
+import os
+from typing import Any
+
+import httpx
+from sqlalchemy import func, select
+
+from app.core.data_sources import get_data_sources_config
+from app.core.datasource_defaults import DEFAULT_DATASOURCES
+from app.models.collected_data import CollectedData
+from app.models.datasource import DataSource
+from app.models.datasource_config import DataSourceConfig
+from app.models.system_setting import SystemSetting
+from app.services.barentswatch import (
+    _read_zshrc_env,
+    fetch_barentswatch_access_token,
+    resolve_barentswatch_config,
+)
+
+
+CONNECTIVITY_VALIDATION_KEY = "connectivity_validation"
+CONNECTIVITY_STORE_CATEGORY = "datasource_connectivity_validations"
+
+
+def _sha256_json(payload: Any) -> str:
+    encoded = json.dumps(payload, ensure_ascii=False, sort_keys=True, default=str).encode("utf-8")
+    return hashlib.sha256(encoded).hexdigest()
+
+
+def _resolve_spacetrack_credentials() -> tuple[str, str, str]:
+    zshrc_env = _read_zshrc_env()
+    username = os.getenv("SPACETRACK_USERNAME") or zshrc_env.get("SPACETRACK_USERNAME") or ""
+    password = os.getenv("SPACETRACK_PASSWORD") or zshrc_env.get("SPACETRACK_PASSWORD") or ""
+    source = "environment" if os.getenv("SPACETRACK_USERNAME") or os.getenv("SPACETRACK_PASSWORD") else ""
+    if not source and (username or password):
+        source = "~/.zshrc"
+    return username, password, source or "missing"
+
+
+def strip_connectivity_validation(config: dict | None) -> dict:
+    cleaned = dict(config or {})
+    cleaned.pop(CONNECTIVITY_VALIDATION_KEY, None)
+    return cleaned
+
+
+def merge_connectivity_validation(existing_config: dict | None, next_config: dict | None) -> dict:
+    merged = strip_connectivity_validation(next_config)
+    validation = (existing_config or {}).get(CONNECTIVITY_VALIDATION_KEY)
+    if validation:
+        merged[CONNECTIVITY_VALIDATION_KEY] = validation
+    return merged
+
+
+def get_connectivity_validation(config: DataSourceConfig | None) -> dict | None:
+    validation = (config.config or {}).get(CONNECTIVITY_VALIDATION_KEY) if config else None
+    return validation if isinstance(validation, dict) else None
+
+
+async def build_builtin_connectivity_checksum(
+    source: str,
+    endpoint: str,
+    auth_type: str,
+    headers: dict | None,
+    config: dict | None,
+    db=None,
+    credential_override: dict[str, str] | None = None,
+) -> tuple[str, dict[str, Any]]:
+    defaults = DEFAULT_DATASOURCES.get(source, {})
+    credential_provider = defaults.get("credential_provider")
+    credential_fingerprint = ""
+    credential_source = "none"
+    has_credentials = not defaults.get("requires_credentials", False)
+
+    if credential_provider == "barentswatch":
+        if credential_override:
+            client_id = credential_override.get("client_id", "")
+            client_secret = credential_override.get("client_secret", "")
+            credential_source = "draft"
+        else:
+            barentswatch_config = await resolve_barentswatch_config(db)
+            client_id = barentswatch_config.client_id
+            client_secret = barentswatch_config.client_secret
+            credential_source = barentswatch_config.credential_source
+        has_credentials = bool(client_id and client_secret)
+        credential_fingerprint = _sha256_json(
+            {
+                "client_id": client_id,
+                "client_secret": client_secret,
+            }
+        )
+    elif credential_provider == "spacetrack":
+        username, password, credential_source = _resolve_spacetrack_credentials()
+        has_credentials = bool(username and password)
+        credential_fingerprint = _sha256_json(
+            {
+                "username": username,
+                "password": password,
+            }
+        )
+    elif defaults.get("requires_credentials"):
+        credential_source = str(credential_provider or "unsupported")
+
+    checksum_payload = {
+        "source": source,
+        "endpoint": endpoint,
+        "auth_type": "none",
+        "headers": headers or {},
+        "config": strip_connectivity_validation(config),
+        "credential_provider": credential_provider or "none",
+        "credential_fingerprint": credential_fingerprint,
+    }
+    return _sha256_json(checksum_payload), {
+        "requires_credentials": bool(defaults.get("requires_credentials", False)),
+        "credential_provider": credential_provider,
+        "credential_source": credential_source,
+        "has_credentials": has_credentials,
+    }
+
+
+async def test_builtin_connectivity(
+    source: str,
+    endpoint: str,
+    auth_type: str,
+    headers: dict | None,
+    config: dict | None,
+    db=None,
+) -> dict[str, Any]:
+    defaults = DEFAULT_DATASOURCES.get(source)
+    if not defaults:
+        return {
+            "success": False,
+            "message": "未知内置采集器，无法执行连接校验。",
+        }
+
+    checksum, credential_context = await build_builtin_connectivity_checksum(
+        source,
+        endpoint,
+        auth_type,
+        headers,
+        config,
+        db,
+    )
+    if credential_context["requires_credentials"] and not credential_context["has_credentials"]:
+        return {
+            "success": False,
+            "checksum": checksum,
+            "stage": "credentials",
+            "message": "该采集器需要凭证，请先到采集器凭证设置中配置。",
+            "settings_tab": "collector_credentials",
+            **credential_context,
+        }
+    supported_credential_providers = {"barentswatch", "spacetrack"}
+    if (
+        credential_context["requires_credentials"]
+        and credential_context["credential_provider"] not in supported_credential_providers
+    ):
+        return {
+            "success": False,
+            "checksum": checksum,
+            "stage": "credentials",
+            "message": "该采集器的凭证链路尚未接入，暂时无法完成连接校验。",
+            "settings_tab": "collector_credentials",
+            **credential_context,
+        }
+
+    request_headers = {str(key): str(value) for key, value in (headers or {}).items()}
+    request_config = strip_connectivity_validation(config)
+    timeout = float(request_config.get("timeout") or 30)
+    request_endpoint = endpoint
+
+    try:
+        async with httpx.AsyncClient(timeout=timeout, follow_redirects=True) as client:
+            if credential_context["credential_provider"] == "barentswatch":
+                barentswatch_config = await resolve_barentswatch_config(db)
+                token = await fetch_barentswatch_access_token(client, barentswatch_config)
+                if not token:
+                    return {
+                        "success": False,
+                        "checksum": checksum,
+                        "stage": "token",
+                        "message": "凭证可读取，但 token 响应中没有 access_token。",
+                        "settings_tab": "collector_credentials",
+                        **credential_context,
+                    }
+                request_headers["Authorization"] = f"Bearer {token}"
+            elif credential_context["credential_provider"] == "spacetrack":
+                username, password, _source = _resolve_spacetrack_credentials()
+                login_url = "https://www.space-track.org/ajaxauth/login"
+                login_response = await client.post(
+                    login_url,
+                    data={
+                        "identity": username,
+                        "password": password,
+                    },
+                )
+                login_response.raise_for_status()
+
+            started = datetime.now(UTC)
+            async with client.stream("GET", request_endpoint, headers=request_headers) as response:
+                response.raise_for_status()
+                status_code = response.status_code
+            elapsed_ms = (datetime.now(UTC) - started).total_seconds() * 1000
+
+        return {
+            "success": True,
+            "checksum": checksum,
+            "stage": "endpoint",
+            "message": "连接验证成功。",
+            "status_code": status_code,
+            "response_time_ms": elapsed_ms,
+            **credential_context,
+        }
+    except httpx.HTTPStatusError as exc:
+        return {
+            "success": False,
+            "checksum": checksum,
+            "stage": "endpoint",
+            "message": f"连接验证失败：HTTP {exc.response.status_code}",
+            "error": f"HTTP Error: {exc.response.status_code}",
+            **credential_context,
+        }
+    except httpx.HTTPError as exc:
+        return {
+            "success": False,
+            "checksum": checksum,
+            "stage": "network",
+            "message": f"连接验证失败：{exc.__class__.__name__}",
+            "error": str(exc),
+            **credential_context,
+        }
+
+
+def make_success_validation(checksum: str, result: dict[str, Any]) -> dict[str, Any]:
+    return {
+        "checksum": checksum,
+        "status": "success",
+        "validated_at": datetime.now(UTC).isoformat(),
+        "status_code": result.get("status_code"),
+        "credential_source": result.get("credential_source"),
+    }
+
+
+def is_builtin_validation_current(config: DataSourceConfig | None, checksum: str) -> bool:
+    validation = get_connectivity_validation(config)
+    return bool(
+        validation
+        and validation.get("status") == "success"
+        and validation.get("checksum") == checksum
+    )
+
+
+async def get_connectivity_store(db) -> dict[str, Any]:
+    result = await db.execute(
+        select(SystemSetting).where(SystemSetting.category == CONNECTIVITY_STORE_CATEGORY)
+    )
+    record = result.scalar_one_or_none()
+    return dict(record.payload or {}) if record and isinstance(record.payload, dict) else {}
+
+
+async def save_connectivity_success(
+    db,
+    source: str,
+    checksum: str,
+    result: dict[str, Any],
+    *,
+    connected_by: str,
+) -> dict[str, Any]:
+    store = await get_connectivity_store(db)
+    validation = {
+        **make_success_validation(checksum, result),
+        "connected_by": connected_by,
+    }
+    store[source] = validation
+
+    existing = await db.execute(
+        select(SystemSetting).where(SystemSetting.category == CONNECTIVITY_STORE_CATEGORY)
+    )
+    record = existing.scalar_one_or_none()
+    if record is None:
+        db.add(SystemSetting(category=CONNECTIVITY_STORE_CATEGORY, payload=store))
+    else:
+        record.payload = store
+    return validation
+
+
+async def load_builtin_override_config(db, source: str) -> DataSourceConfig | None:
+    result = await db.execute(
+        select(DataSourceConfig)
+        .where(DataSourceConfig.name == source)
+        .where(DataSourceConfig.is_active.is_(True))
+    )
+    return result.scalar_one_or_none()
+
+
+async def get_builtin_effective_candidate(db, source: str) -> dict[str, Any]:
+    override = await load_builtin_override_config(db, source)
+    default_endpoint = get_data_sources_config().get_yaml_url(source)
+    return {
+        "name": source,
+        "endpoint": (override.endpoint if override and override.endpoint else default_endpoint) or "",
+        "auth_type": override.auth_type if override else "none",
+        "headers": override.headers if override else {},
+        "config": strip_connectivity_validation(override.config if override else {}),
+    }
+
+
+async def has_collected_data(db, source: str) -> bool:
+    result = await db.execute(select(func.count(CollectedData.id)).where(CollectedData.source == source))
+    if (result.scalar() or 0) > 0:
+        return True
+
+    datasource_result = await db.execute(select(DataSource).where(DataSource.source == source))
+    datasource = datasource_result.scalar_one_or_none()
+    return bool(datasource and datasource.last_status == "success")
+
+
+async def get_builtin_connection_status(
+    db,
+    source: str,
+    endpoint: str,
+    auth_type: str,
+    headers: dict | None,
+    config: dict | None,
+) -> dict[str, Any]:
+    checksum, credential_context = await build_builtin_connectivity_checksum(
+        source,
+        endpoint,
+        auth_type,
+        headers,
+        config,
+        db,
+    )
+    store = await get_connectivity_store(db)
+    validation = store.get(source)
+    if isinstance(validation, dict) and validation.get("status") == "success":
+        if validation.get("checksum") == checksum:
+            return {
+                "connected": True,
+                "checksum": checksum,
+                "connected_by": validation.get("connected_by") or "connection_button",
+                "message": "当前配置已完成连接验证。",
+                **credential_context,
+            }
+
+    effective = await get_builtin_effective_candidate(db, source)
+    effective_checksum, _ = await build_builtin_connectivity_checksum(
+        source,
+        effective["endpoint"],
+        effective["auth_type"],
+        effective["headers"],
+        effective["config"],
+        db,
+    )
+    if checksum == effective_checksum and await has_collected_data(db, source):
+        return {
+            "connected": True,
+            "checksum": checksum,
+            "connected_by": "collection",
+            "message": "当前配置已有成功采集数据，视为已连接。",
+            **credential_context,
+        }
+
+    if isinstance(validation, dict) and validation.get("status") == "success":
+        return {
+            "connected": False,
+            "checksum": checksum,
+            "connected_by": None,
+            "message": "接口地址或凭证指纹已变化，请重新点击连接验证。",
+            **credential_context,
+        }
+
+    return {
+        "connected": False,
+        "checksum": checksum,
+        "connected_by": None,
+        "message": "当前配置尚未连接，请点击连接验证。",
+        **credential_context,
+    }

backend/app/services/datasource_mapping.py

@@ -0,0 +1,358 @@
+"""Deterministic mapping support for custom data sources."""
+
+from __future__ import annotations
+
+import hashlib
+import json
+import re
+from datetime import UTC, datetime
+from typing import Any
+
+from sqlalchemy.ext.asyncio import AsyncSession
+
+from app.core.target_schema_registry import TargetSchema, get_target_schema
+
+SECRET_KEY_PATTERN = re.compile(
+    r"(token|secret|password|passwd|authorization|api[_-]?key|client[_-]?secret)",
+    re.IGNORECASE,
+)
+
+
+class MappingError(ValueError):
+    """Raised when a mapping definition cannot be executed."""
+
+
+def stable_payload_hash(payload: Any) -> str:
+    encoded = json.dumps(payload, ensure_ascii=False, sort_keys=True, default=str).encode()
+    return hashlib.sha256(encoded).hexdigest()
+
+
+def redact_for_llm(value: Any) -> Any:
+    if isinstance(value, dict):
+        redacted = {}
+        for key, item in value.items():
+            if SECRET_KEY_PATTERN.search(str(key)):
+                redacted[key] = "[REDACTED]"
+            else:
+                redacted[key] = redact_for_llm(item)
+        return redacted
+    if isinstance(value, list):
+        return [redact_for_llm(item) for item in value[:20]]
+    return value
+
+
+def extract_path(payload: Any, path: str | None) -> Any:
+    if not path or path == "$":
+        return payload
+
+    normalized = path.strip()
+    if normalized.startswith("$."):
+        normalized = normalized[2:]
+    elif normalized.startswith("$"):
+        normalized = normalized[1:]
+    normalized = normalized.strip(".")
+    if not normalized:
+        return payload
+
+    current = payload
+    for raw_segment in normalized.split("."):
+        segment = raw_segment.strip()
+        if not segment:
+            continue
+
+        list_all = segment.endswith("[*]")
+        if list_all:
+            segment = segment[:-3]
+
+        index = None
+        match = re.fullmatch(r"(.+)\[(\d+)\]", segment)
+        if match:
+            segment = match.group(1)
+            index = int(match.group(2))
+
+        if segment:
+            if isinstance(current, dict):
+                current = current.get(segment)
+            else:
+                return None
+
+        if list_all:
+            return current if isinstance(current, list) else []
+
+        if index is not None:
+            if not isinstance(current, list) or index >= len(current):
+                return None
+            current = current[index]
+
+    return current
+
+
+def _convert_value(value: Any, target_type: str | None) -> Any:
+    if value is None or target_type in (None, "", "any"):
+        return value
+
+    if target_type == "string":
+        return str(value)
+    if target_type == "integer":
+        return int(value)
+    if target_type == "float":
+        return float(value)
+    if target_type == "boolean":
+        if isinstance(value, bool):
+            return value
+        if isinstance(value, str):
+            return value.strip().lower() in {"1", "true", "yes", "y", "on"}
+        return bool(value)
+    if target_type == "datetime":
+        if isinstance(value, datetime):
+            return value
+        if isinstance(value, (int, float)):
+            return datetime.fromtimestamp(value)
+        if isinstance(value, str):
+            return datetime.fromisoformat(value.replace("Z", "+00:00"))
+        return value
+    if target_type == "object":
+        if isinstance(value, dict):
+            return value
+        raise ValueError("expected object")
+    if target_type == "array":
+        if isinstance(value, list):
+            return value
+        raise ValueError("expected array")
+
+    return value
+
+
+def _apply_enum(value: Any, enum_map: Any) -> Any:
+    if not isinstance(enum_map, dict):
+        return value
+    key = str(value)
+    return enum_map.get(key, enum_map.get(value, value))
+
+
+def _map_one(item: Any, field_mapping: dict[str, Any]) -> tuple[dict[str, Any], list[str]]:
+    output: dict[str, Any] = {}
+    errors: list[str] = []
+
+    for field_name, rule in field_mapping.items():
+        if isinstance(rule, str):
+            rule = {"path": rule}
+        if not isinstance(rule, dict):
+            errors.append(f"{field_name}: mapping rule must be an object or path string")
+            continue
+
+        value = extract_path(item, rule.get("path"))
+        if value is None and "default" in rule:
+            value = rule.get("default")
+        value = _apply_enum(value, rule.get("enum"))
+
+        try:
+            value = _convert_value(value, rule.get("type"))
+        except (TypeError, ValueError) as exc:
+            errors.append(f"{field_name}: failed to convert value {value!r}: {exc}")
+            continue
+
+        if value is not None or rule.get("include_null", False):
+            output[field_name] = value
+
+    return output, errors
+
+
+def execute_mapping(
+    payload: Any,
+    mapping_json: dict[str, Any],
+    target_schema: str | TargetSchema,
+    *,
+    limit: int | None = None,
+) -> dict[str, Any]:
+    schema = get_target_schema(target_schema) if isinstance(target_schema, str) else target_schema
+    source = mapping_json.get("source") or {}
+    fields = mapping_json.get("fields")
+    if not isinstance(fields, dict) or not fields:
+        raise MappingError("mapping_json.fields must be a non-empty object")
+
+    items_path = source.get("items_path") or mapping_json.get("items_path") or "$"
+    items = extract_path(payload, items_path)
+    if isinstance(items, dict):
+        items = [items]
+    elif not isinstance(items, list):
+        items = []
+
+    if limit is not None:
+        items = items[:limit]
+
+    mapped_records: list[dict[str, Any]] = []
+    errors: list[dict[str, Any]] = []
+    for index, item in enumerate(items):
+        mapped, mapping_errors = _map_one(item, fields)
+        validated, validation_errors = schema.validate_record(mapped)
+        all_errors = mapping_errors + validation_errors
+        if all_errors:
+            errors.append({"index": index, "errors": all_errors, "record": mapped})
+            continue
+        if validated is not None:
+            mapped_records.append(validated)
+
+    return {
+        "target_schema": schema.key,
+        "total_items": len(items),
+        "mapped_count": len(mapped_records),
+        "failed_count": len(errors),
+        "records": mapped_records,
+        "errors": errors,
+    }
+
+
+def build_heuristic_mapping(sample_payload: Any, target_schema_key: str) -> dict[str, Any]:
+    schema = get_target_schema(target_schema_key)
+    items_path = "$"
+    sample_item = sample_payload
+    if isinstance(sample_payload, dict):
+        for key in ("data", "items", "results", "features", "vessels"):
+            candidate = sample_payload.get(key)
+            if isinstance(candidate, list) and candidate:
+                items_path = f"$.{key}[*]"
+                sample_item = candidate[0]
+                break
+    elif isinstance(sample_payload, list) and sample_payload:
+        items_path = "$"
+        sample_item = sample_payload[0]
+
+    available = _flatten_keys(sample_item if isinstance(sample_item, dict) else {})
+    fields: dict[str, Any] = {}
+    for field in schema.fields:
+        candidate = _best_field_match(field.name, available)
+        if candidate:
+            fields[field.name] = {"path": f"$.{candidate}", "type": field.type}
+        elif field.name == "data" and target_schema_key == "generic_records":
+            fields[field.name] = {"path": "$", "type": "object"}
+        elif not field.required:
+            fields[field.name] = {"path": f"$.{field.name}", "type": field.type, "default": None}
+
+    return {
+        "source": {"items_path": items_path},
+        "fields": fields,
+        "meta": {
+            "generated_by": "heuristic",
+            "requires_review": True,
+        },
+    }
+
+
+def _flatten_keys(payload: dict[str, Any], prefix: str = "") -> list[str]:
+    keys: list[str] = []
+    for key, value in payload.items():
+        dotted = f"{prefix}.{key}" if prefix else str(key)
+        keys.append(dotted)
+        if isinstance(value, dict):
+            keys.extend(_flatten_keys(value, dotted))
+    return keys
+
+
+def _best_field_match(field_name: str, candidates: list[str]) -> str | None:
+    aliases = {
+        "lat": ("lat", "latitude", "y"),
+        "lon": ("lon", "lng", "longitude", "x"),
+        "mmsi": ("mmsi",),
+        "sog": ("sog", "speed", "speedOverGround"),
+        "cog": ("cog", "course", "courseOverGround"),
+        "received_at": ("received_at", "timestamp", "time", "updated_at"),
+        "observed_at": ("observed_at", "timestamp", "time", "updated_at"),
+        "source_id": ("id", "source_id", "uuid"),
+    }.get(field_name, (field_name,))
+
+    lowered = {candidate.lower(): candidate for candidate in candidates}
+    for alias in aliases:
+        if alias.lower() in lowered:
+            return lowered[alias.lower()]
+    for candidate in candidates:
+        tail = candidate.split(".")[-1].lower()
+        if tail in {alias.lower() for alias in aliases}:
+            return candidate
+    return None
+
+
+def _parse_datetime(value: Any) -> datetime | None:
+    if value is None:
+        return None
+    if isinstance(value, datetime):
+        return value
+    if isinstance(value, str):
+        return datetime.fromisoformat(value.replace("Z", "+00:00"))
+    return None
+
+
+async def persist_mapped_records(
+    db: AsyncSession,
+    *,
+    datasource_name: str,
+    datasource_config_id: int,
+    target_schema: str,
+    records: list[dict[str, Any]],
+    mapping_version: int,
+) -> int:
+    """Persist validated mapped records to the destination for a target schema."""
+    if target_schema == "vessel_ais":
+        from app.models.vessel import VesselPosition
+
+        for record in records:
+            db.add(
+                VesselPosition(
+                    mmsi=record["mmsi"],
+                    lat=record["lat"],
+                    lon=record["lon"],
+                    sog=record.get("sog"),
+                    cog=record.get("cog"),
+                    heading=record.get("heading"),
+                    received_at=_parse_datetime(record.get("received_at")) or datetime.now(UTC),
+                )
+            )
+        await db.commit()
+        return len(records)
+
+    from app.models.collected_data import CollectedData
+
+    collected_at = datetime.now(UTC)
+    for index, record in enumerate(records):
+        if target_schema == "geo_points":
+            source_id = record.get("source_id") or f"{datasource_config_id}:{index}"
+            name = record.get("name")
+            metadata = {
+                "latitude": record.get("lat"),
+                "longitude": record.get("lon"),
+                "type": record.get("type"),
+                "mapping_version": mapping_version,
+                "target_schema": target_schema,
+                **(record.get("metadata") or {}),
+            }
+            reference_date = _parse_datetime(record.get("observed_at"))
+        else:
+            source_id = record.get("source_id") or f"{datasource_config_id}:{index}"
+            name = None
+            metadata = {
+                "data": record.get("data") or {},
+                "mapping_version": mapping_version,
+                "target_schema": target_schema,
+            }
+            reference_date = _parse_datetime(record.get("observed_at"))
+
+        db.add(
+            CollectedData(
+                source=datasource_name,
+                source_id=str(source_id),
+                entity_key=f"{datasource_name}:{source_id}",
+                data_type=target_schema,
+                name=name,
+                title=name,
+                extra_data=metadata,
+                collected_at=collected_at,
+                reference_date=reference_date,
+                is_valid=1,
+                is_current=True,
+                change_type="created",
+                change_summary={},
+            )
+        )
+
+    await db.commit()
+    return len(records)

backend/app/services/earth_news.py

@@ -30,6 +30,14 @@ class RegionProfile:
     accent: str
 
 
+@dataclass(frozen=True)
+class RegionAnchor:
+    region: str
+    label: str
+    latitude: float
+    longitude: float
+
+
 @dataclass(frozen=True)
 class NewsFeedSource:
     id: str
@@ -95,6 +103,39 @@ REGION_PROFILES: dict[str, RegionProfile] = {
     ),
 }
 
+REGION_ANCHORS: dict[str, RegionAnchor] = {
+    "americas": RegionAnchor(
+        region="americas",
+        label="美洲",
+        latitude=37.0902,
+        longitude=-95.7129,
+    ),
+    "europe": RegionAnchor(
+        region="europe",
+        label="欧洲",
+        latitude=50.1109,
+        longitude=8.6821,
+    ),
+    "middle-east-africa": RegionAnchor(
+        region="middle-east-africa",
+        label="中东与非洲",
+        latitude=25.2048,
+        longitude=55.2708,
+    ),
+    "asia-pacific": RegionAnchor(
+        region="asia-pacific",
+        label="亚太",
+        latitude=1.3521,
+        longitude=103.8198,
+    ),
+    "global": RegionAnchor(
+        region="global",
+        label="全球",
+        latitude=20.0,
+        longitude=0.0,
+    ),
+}
+
 
 def _google_news_feed(query: str, *, hl: str, gl: str, ceid: str) -> str:
     return (
@@ -213,6 +254,10 @@ def get_region_profile(region: str) -> RegionProfile:
     return REGION_PROFILES.get(region, REGION_PROFILES["global"])
 
 
+def get_region_anchor(region: str) -> RegionAnchor:
+    return REGION_ANCHORS.get(region, REGION_ANCHORS["global"])
+
+
 def get_sources_for_region(region: str) -> list[NewsFeedSource]:
     return sorted(
         [source for source in NEWS_FEED_SOURCES if source.region in {"global", region}],
@@ -342,6 +387,7 @@ def _serialize_sources(sources: list[NewsFeedSource]) -> list[dict[str, Any]]:
 
 def _serialize_item(item: ParsedNewsItem, *, active_region: str) -> dict[str, Any]:
     published_at = item.published_at
+    anchor = get_region_anchor(item.feed_region)
     return {
         "id": item.id,
         "title": item.title,
@@ -352,6 +398,10 @@ def _serialize_item(item: ParsedNewsItem, *, active_region: str) -> dict[str, An
         "region": item.feed_region,
         "homepage_url": item.homepage_url,
         "published_at": published_at.isoformat().replace("+00:00", "Z") if published_at else None,
+        "latitude": anchor.latitude,
+        "longitude": anchor.longitude,
+        "location_label": anchor.label,
+        "location_inferred": True,
         "is_focus_match": item.feed_region == active_region,
     }
 

backend/app/services/llm_provider_catalog.py

@@ -0,0 +1,150 @@
+"""LLM provider presets used by Settings and the runtime AI provider bridge."""
+
+from __future__ import annotations
+
+from typing import Any
+
+import httpx
+
+MODELS_DEV_URL = "https://models.dev/api.json"
+
+
+FALLBACK_LLM_PROVIDER_PRESETS: dict[str, dict[str, Any]] = {
+    "minimax": {
+        "provider": "minimax",
+        "label": "MiniMax",
+        "provider_api": "anthropic-messages",
+        "base_url": "https://api.minimaxi.com/anthropic",
+        "model": "MiniMax-M2.7",
+        "models": ["MiniMax-M2.7", "MiniMax-M2.7-highspeed", "MiniMax-M2.5", "MiniMax-M2"],
+        "api_key_env": "MINIMAX_API_KEY",
+        "source": "fallback",
+    },
+    "openai": {
+        "provider": "openai",
+        "label": "OpenAI",
+        "provider_api": "openai-completions",
+        "base_url": "https://api.openai.com/v1",
+        "model": "gpt-5.1",
+        "models": ["gpt-5.1", "gpt-5.1-codex", "gpt-4.1", "gpt-4o"],
+        "api_key_env": "OPENAI_API_KEY",
+        "source": "fallback",
+    },
+    "anthropic": {
+        "provider": "anthropic",
+        "label": "Anthropic",
+        "provider_api": "anthropic-messages",
+        "base_url": "https://api.anthropic.com/v1",
+        "model": "claude-sonnet-4-6",
+        "models": ["claude-sonnet-4-6", "claude-opus-4-5", "claude-3-5-haiku-20241022"],
+        "api_key_env": "ANTHROPIC_API_KEY",
+        "source": "fallback",
+    },
+    "deepseek": {
+        "provider": "deepseek",
+        "label": "DeepSeek",
+        "provider_api": "openai-completions",
+        "base_url": "https://api.deepseek.com/v1",
+        "model": "deepseek-chat",
+        "models": ["deepseek-chat", "deepseek-reasoner"],
+        "api_key_env": "DEEPSEEK_API_KEY",
+        "source": "fallback",
+    },
+    "alibaba": {
+        "provider": "alibaba",
+        "label": "Alibaba Qwen / DashScope",
+        "provider_api": "openai-completions",
+        "base_url": "https://dashscope.aliyuncs.com/compatible-mode/v1",
+        "model": "qwen3-max",
+        "models": ["qwen3-max", "qwen3.5-plus", "qwen-max", "qwen-plus"],
+        "api_key_env": "DASHSCOPE_API_KEY",
+        "source": "fallback",
+    },
+    "moonshotai": {
+        "provider": "moonshotai",
+        "label": "Moonshot AI / Kimi",
+        "provider_api": "openai-completions",
+        "base_url": "https://api.moonshot.ai/v1",
+        "model": "kimi-k2.5",
+        "models": ["kimi-k2.5", "kimi-k2-thinking", "kimi-k2-turbo-preview"],
+        "api_key_env": "MOONSHOT_API_KEY",
+        "source": "fallback",
+    },
+    "openrouter": {
+        "provider": "openrouter",
+        "label": "OpenRouter",
+        "provider_api": "openai-completions",
+        "base_url": "https://openrouter.ai/api/v1",
+        "model": "openai/gpt-5.1",
+        "models": ["openai/gpt-5.1", "anthropic/claude-sonnet-4.5", "qwen/qwen3-max"],
+        "api_key_env": "OPENROUTER_API_KEY",
+        "source": "fallback",
+    },
+    "ollama": {
+        "provider": "ollama",
+        "label": "Ollama Local",
+        "provider_api": "ollama-generate",
+        "base_url": "http://127.0.0.1:11434",
+        "model": "qwen2.5:7b",
+        "models": ["qwen2.5:7b", "llama3.1:8b", "mistral:7b"],
+        "api_key_env": "",
+        "source": "fallback",
+    },
+}
+
+MODELS_DEV_PROVIDER_KEYS = {
+    "minimax": "minimax",
+    "openai": "openai",
+    "anthropic": "anthropic",
+    "deepseek": "deepseek",
+    "alibaba": "alibaba",
+    "moonshotai": "moonshotai",
+    "openrouter": "openrouter",
+}
+
+
+def list_fallback_llm_provider_presets() -> list[dict[str, Any]]:
+    return [dict(value) for value in FALLBACK_LLM_PROVIDER_PRESETS.values()]
+
+
+def get_fallback_llm_provider_preset(provider: str) -> dict[str, Any]:
+    key = provider.strip().lower()
+    if key not in FALLBACK_LLM_PROVIDER_PRESETS:
+        raise ValueError(f"Unsupported LLM provider preset: {provider}")
+    return dict(FALLBACK_LLM_PROVIDER_PRESETS[key])
+
+
+async def refresh_llm_provider_preset(provider: str) -> dict[str, Any]:
+    fallback = get_fallback_llm_provider_preset(provider)
+    models_dev_key = MODELS_DEV_PROVIDER_KEYS.get(fallback["provider"])
+    if not models_dev_key:
+        return fallback
+
+    async with httpx.AsyncClient(timeout=15.0, follow_redirects=True) as client:
+        response = await client.get(
+            MODELS_DEV_URL,
+            headers={"User-Agent": "Planet/1.0"},
+        )
+        response.raise_for_status()
+        catalog = response.json()
+
+    upstream = catalog.get(models_dev_key)
+    if not isinstance(upstream, dict):
+        return fallback
+
+    upstream_models = upstream.get("models") if isinstance(upstream.get("models"), dict) else {}
+    model_ids = list(upstream_models.keys())[:80]
+    base_url = upstream.get("api") or fallback["base_url"]
+    if fallback["provider"] == "deepseek" and base_url == "https://api.deepseek.com":
+        base_url = "https://api.deepseek.com/v1"
+
+    refreshed = {
+        **fallback,
+        "label": upstream.get("name") or fallback["label"],
+        "base_url": base_url,
+        "model": model_ids[0] if model_ids else fallback["model"],
+        "models": model_ids or fallback["models"],
+        "api_key_env": (upstream.get("env") or [fallback["api_key_env"]])[0],
+        "source": MODELS_DEV_URL,
+    }
+    return refreshed

backend/app/services/persistent_logs.py

@@ -0,0 +1,86 @@
+from __future__ import annotations
+
+from typing import Any
+
+from app.core.logging import get_logger, sanitize_log_value
+from app.core.request_context import get_request_id
+from app.db.session import async_session_factory
+from app.models.system_log import AuditLog, SystemLog
+
+logger = get_logger(__name__)
+
+
+async def record_system_log(
+    *,
+    source: str,
+    level: str,
+    message: str,
+    service: str | None = None,
+    module: str | None = None,
+    event: str | None = None,
+    request_id: str | None = None,
+    trace_id: str | None = None,
+    user_id: int | None = None,
+    category: str | None = None,
+    context: dict[str, Any] | None = None,
+) -> None:
+    try:
+        async with async_session_factory() as session:
+            session.add(
+                SystemLog(
+                    source=source,
+                    service=service,
+                    module=module,
+                    event=event,
+                    level=level.lower(),
+                    message=str(sanitize_log_value(message)),
+                    request_id=request_id or get_request_id(),
+                    trace_id=trace_id,
+                    user_id=user_id,
+                    category=category,
+                    context=sanitize_log_value(context or {}),
+                )
+            )
+            await session.commit()
+    except Exception:
+        logger.exception_event(
+            "Failed to persist system log",
+            event="system_log.persist.failed",
+            context={"event_name": event, "source": source},
+        )
+
+
+async def record_audit_log(
+    *,
+    action: str,
+    actor_id: int | None = None,
+    actor_name: str | None = None,
+    target_type: str | None = None,
+    target_id: str | None = None,
+    result: str | None = None,
+    request_id: str | None = None,
+    ip: str | None = None,
+    details: dict[str, Any] | None = None,
+) -> None:
+    try:
+        async with async_session_factory() as session:
+            session.add(
+                AuditLog(
+                    actor_id=actor_id,
+                    actor_name=actor_name,
+                    action=action,
+                    target_type=target_type,
+                    target_id=target_id,
+                    result=result,
+                    request_id=request_id or get_request_id(),
+                    ip=ip,
+                    details=sanitize_log_value(details or {}),
+                )
+            )
+            await session.commit()
+    except Exception:
+        logger.exception_event(
+            "Failed to persist audit log",
+            event="audit_log.persist.failed",
+            context={"action": action},
+        )

backend/app/services/scheduler.py

@@ -1,7 +1,6 @@
 """Task Scheduler for running collection jobs."""
 
 import asyncio
-import logging
 from datetime import UTC, datetime, timedelta
 from typing import Any, Dict, Optional
 
@@ -9,13 +8,19 @@ from apscheduler.schedulers.asyncio import AsyncIOScheduler
 from apscheduler.triggers.interval import IntervalTrigger
 from sqlalchemy import select
 
+from app.core.logging import get_logger
 from app.db.session import async_session_factory
 from app.core.time import to_iso8601_utc
 from app.models.datasource import DataSource
 from app.models.task import CollectionTask
 from app.services.collectors.registry import collector_registry
+from app.services.datasource_connectivity import (
+    build_builtin_connectivity_checksum,
+    get_builtin_effective_candidate,
+    save_connectivity_success,
+)
 
-logger = logging.getLogger(__name__)
+logger = get_logger(__name__)
 
 scheduler = AsyncIOScheduler()
 RUNNING_TASK_GUARD_TIMEOUT_MINUTES = 90
@@ -54,7 +59,11 @@ async def _update_next_run_at(datasource: DataSource, session) -> None:
 async def _apply_datasource_schedule(datasource: DataSource, session) -> None:
     collector = collector_registry.get(datasource.source)
     if not collector:
-        logger.warning("Collector not found for datasource %s", datasource.source)
+        logger.warning_event(
+            "Collector not found for datasource",
+            event="collector.schedule.collector_missing",
+            context={"collector_name": datasource.source},
+        )
         return
 
     collector_registry.set_active(datasource.source, datasource.is_active)
@@ -72,13 +81,17 @@ async def _apply_datasource_schedule(datasource: DataSource, session) -> None:
             replace_existing=True,
             kwargs={"collector_name": datasource.source},
         )
-        logger.info(
-            "Scheduled collector: %s (every %sm)",
-            datasource.source,
-            datasource.frequency_minutes,
+        logger.info_event(
+            "Scheduled collector",
+            event="collector.schedule.updated",
+            context={"collector_name": datasource.source, "frequency_minutes": datasource.frequency_minutes},
         )
     else:
-        logger.info("Collector disabled: %s", datasource.source)
+        logger.info_event(
+            "Collector disabled",
+            event="collector.schedule.disabled",
+            context={"collector_name": datasource.source},
+        )
 
     await _update_next_run_at(datasource, session)
 
@@ -87,18 +100,30 @@ async def run_collector_task(collector_name: str):
     """Run a single collector task."""
     collector = collector_registry.get(collector_name)
     if not collector:
-        logger.error("Collector not found: %s", collector_name)
+        logger.error_event(
+            "Collector not found",
+            event="collector.run.collector_missing",
+            context={"collector_name": collector_name},
+        )
         return
 
     async with async_session_factory() as db:
         result = await db.execute(select(DataSource).where(DataSource.source == collector_name))
         datasource = result.scalar_one_or_none()
         if not datasource:
-            logger.error("Datasource not found for collector: %s", collector_name)
+            logger.error_event(
+                "Datasource not found for collector",
+                event="collector.run.datasource_missing",
+                context={"collector_name": collector_name},
+            )
             return
 
         if not datasource.is_active:
-            logger.info("Skipping disabled collector: %s", collector_name)
+            logger.info_event(
+                "Skipping disabled collector",
+                event="collector.run.skipped_disabled",
+                context={"collector_name": collector_name},
+            )
             return
 
         running_result = await db.execute(
@@ -122,10 +147,10 @@ async def run_collector_task(collector_name: str):
                 and (now - started_at) > timedelta(minutes=RUNNING_TASK_GUARD_TIMEOUT_MINUTES)
             )
             if not is_stale:
-                logger.warning(
-                    "Skipping collector %s trigger because task %s is already running",
-                    collector_name,
-                    existing_running.id,
+                logger.warning_event(
+                    "Skipping collector trigger because task is already running",
+                    event="collector.run.skipped_already_running",
+                    context={"collector_name": collector_name, "task_id": existing_running.id},
                 )
                 return
 
@@ -143,31 +168,64 @@ async def run_collector_task(collector_name: str):
                 else stale_reason
             )
             await db.commit()
-            logger.warning(
-                "Marked stale running task %s as failed before rerun of %s",
-                existing_running.id,
-                collector_name,
+            logger.warning_event(
+                "Marked stale running task as failed before rerun",
+                event="collector.run.stale_task_failed",
+                context={"collector_name": collector_name, "task_id": existing_running.id},
             )
 
         try:
             collector._datasource_id = datasource.id
-            logger.info("Running collector: %s (datasource_id=%s)", collector_name, datasource.id)
+            logger.info_event(
+                "Running collector",
+                event="collector.run.started",
+                context={"collector_name": collector_name, "datasource_id": datasource.id},
+            )
             task_result = await collector.run(db)
             datasource.last_run_at = datetime.now(UTC)
             datasource.last_status = task_result.get("status")
+            if datasource.last_status == "success":
+                effective_candidate = await get_builtin_effective_candidate(db, datasource.source)
+                checksum, _credential_context = await build_builtin_connectivity_checksum(
+                    datasource.source,
+                    effective_candidate["endpoint"],
+                    effective_candidate["auth_type"],
+                    effective_candidate["headers"],
+                    effective_candidate["config"],
+                    db,
+                )
+                await save_connectivity_success(
+                    db,
+                    datasource.source,
+                    checksum,
+                    {"status_code": None},
+                    connected_by="collection",
+                )
             await _update_next_run_at(datasource, db)
-            logger.info("Collector %s completed: %s", collector_name, task_result)
+            logger.info_event(
+                "Collector completed",
+                event="collector.run.completed",
+                context={"collector_name": collector_name, "datasource_id": datasource.id, "result": task_result},
+            )
         except asyncio.CancelledError:
             datasource.last_run_at = datetime.now(UTC)
             datasource.last_status = "cancelled"
             await db.commit()
-            logger.warning("Collector %s cancelled by operator", collector_name)
+            logger.warning_event(
+                "Collector cancelled by operator",
+                event="collector.run.cancelled",
+                context={"collector_name": collector_name, "datasource_id": datasource.id},
+            )
             raise
         except Exception as exc:
             datasource.last_run_at = datetime.now(UTC)
             datasource.last_status = "failed"
             await db.commit()
-            logger.exception("Collector %s failed: %s", collector_name, exc)
+            logger.exception_event(
+                "Collector failed",
+                event="collector.run.failed",
+                context={"collector_name": collector_name, "datasource_id": datasource.id, "error": str(exc)},
+            )
 
 
 async def cleanup_stale_running_tasks(max_age_hours: int = 2) -> int:
@@ -194,7 +252,11 @@ async def cleanup_stale_running_tasks(max_age_hours: int = 2) -> int:
 
         if stale_tasks:
             await db.commit()
-            logger.warning("Cleaned up %s stale running collection task(s)", len(stale_tasks))
+            logger.warning_event(
+                "Cleaned up stale running collection tasks",
+                event="collector.cleanup.stale_tasks_cleaned",
+                context={"count": len(stale_tasks)},
+            )
 
         return len(stale_tasks)
 
@@ -203,14 +265,14 @@ def start_scheduler() -> None:
     """Start the scheduler."""
     if not scheduler.running:
         scheduler.start()
-        logger.info("Scheduler started")
+        logger.info_event("Scheduler started", event="scheduler.started")
 
 
 def stop_scheduler() -> None:
     """Stop the scheduler."""
     if scheduler.running:
         scheduler.shutdown(wait=False)
-        logger.info("Scheduler stopped")
+        logger.info_event("Scheduler stopped", event="scheduler.stopped")
 
 
 async def sync_scheduler_with_datasources() -> None:
@@ -271,12 +333,20 @@ def run_collector_now(collector_name: str) -> bool:
     """Run a collector immediately (not scheduled)."""
     collector = collector_registry.get(collector_name)
     if not collector:
-        logger.error("Collector not found: %s", collector_name)
+        logger.error_event(
+            "Collector not found",
+            event="collector.trigger.collector_missing",
+            context={"collector_name": collector_name},
+        )
         return False
 
     existing_task = get_running_collector_task(collector_name)
     if existing_task is not None and not existing_task.done():
-        logger.warning("Collector %s is already running in-memory; skipping duplicate trigger", collector_name)
+        logger.warning_event(
+            "Collector is already running in-memory; skipping duplicate trigger",
+            event="collector.trigger.skipped_already_running",
+            context={"collector_name": collector_name},
+        )
         return False
 
     try:
@@ -289,10 +359,18 @@ def run_collector_now(collector_name: str) -> bool:
                 RUNNING_COLLECTOR_TASKS.pop(collector_name, None)
 
         task.add_done_callback(_cleanup_task)
-        logger.info("Triggered collector: %s", collector_name)
+        logger.info_event(
+            "Triggered collector",
+            event="collector.trigger.started",
+            context={"collector_name": collector_name},
+        )
         return True
     except Exception as exc:
-        logger.error("Failed to trigger collector %s: %s", collector_name, exc)
+        logger.error_event(
+            "Failed to trigger collector",
+            event="collector.trigger.failed",
+            context={"collector_name": collector_name, "error": str(exc)},
+        )
         return False
 
 

backend/app/services/system_control.py

@@ -19,6 +19,10 @@ ALLOWED_ACTIONS: dict[str, dict[str, Any]] = {
         "command": ["./planet.sh", "restart", "-b"],
         "recovery_mode": "backend",
     },
+    "restart-frontend": {
+        "command": ["./planet.sh", "restart", "-f"],
+        "recovery_mode": "frontend",
+    },
     "restart-ai-provider": {
         "command": ["./planet.sh", "restart", "-a"],
         "recovery_mode": "ai-provider",

backend/app/services/system_logs.py

@@ -0,0 +1,532 @@
+from __future__ import annotations
+
+import json
+import re
+import shutil
+import subprocess
+
+from collections import Counter, deque
+from dataclasses import dataclass
+from datetime import UTC, datetime
+from pathlib import Path
+from typing import Any
+
+from app.core.security import redis_client
+
+DEFAULT_LOG_LINE_LIMIT = 200
+MAX_LOG_LINE_LIMIT = 1000
+BUFFER_LOG_LIMIT = 1000
+BUFFER_LOG_TTL_SECONDS = 7 * 24 * 60 * 60
+LOG_BUFFER_KEY_PREFIX = "planet:system_logs"
+
+LOG_LEVEL_ERROR = "error"
+LOG_LEVEL_WARNING = "warning"
+LOG_LEVEL_INFO = "info"
+LOG_LEVEL_DEBUG = "debug"
+LOG_LEVEL_ALL = "all"
+
+SUPPORTED_LOG_LEVELS = {
+    LOG_LEVEL_ALL,
+    LOG_LEVEL_ERROR,
+    LOG_LEVEL_WARNING,
+    LOG_LEVEL_INFO,
+    LOG_LEVEL_DEBUG,
+}
+
+LOG_LEVEL_ALIASES = {
+    "warn": LOG_LEVEL_WARNING,
+    "warning": LOG_LEVEL_WARNING,
+    "err": LOG_LEVEL_ERROR,
+    "error": LOG_LEVEL_ERROR,
+    "info": LOG_LEVEL_INFO,
+    "information": LOG_LEVEL_INFO,
+    "debug": LOG_LEVEL_DEBUG,
+    "trace": LOG_LEVEL_DEBUG,
+    "critical": LOG_LEVEL_ERROR,
+    "fatal": LOG_LEVEL_ERROR,
+}
+
+TIMESTAMP_FORMATS = (
+    "%Y-%m-%d %H:%M:%S.%f",
+    "%Y-%m-%d %H:%M:%S",
+    "%Y-%m-%dT%H:%M:%S.%f",
+    "%Y-%m-%dT%H:%M:%S",
+)
+
+LEVEL_PATTERNS = (
+    ("CRITICAL", LOG_LEVEL_ERROR),
+    ("FATAL", LOG_LEVEL_ERROR),
+    ("ERROR", LOG_LEVEL_ERROR),
+    ("WARNING", LOG_LEVEL_WARNING),
+    ("WARN", LOG_LEVEL_WARNING),
+    ("INFO", LOG_LEVEL_INFO),
+    ("DEBUG", LOG_LEVEL_DEBUG),
+    ("TRACE", LOG_LEVEL_DEBUG),
+)
+
+LEADING_LEVEL_PATTERN = re.compile(
+    r"^\s*(?:\[[^\]]+\]\s*)?(CRITICAL|FATAL|ERROR|WARNING|WARN|INFO|DEBUG|TRACE)\b[:\s-]*",
+    re.IGNORECASE,
+)
+EMBEDDED_LEVEL_PATTERN = re.compile(
+    r"\b(CRITICAL|FATAL|ERROR|WARNING|WARN|INFO|DEBUG|TRACE)\b",
+    re.IGNORECASE,
+)
+CONTROL_CHAR_PATTERN = re.compile(r"[\x00-\x08\x0b\x0c\x0e-\x1f\x7f]")
+
+
+@dataclass(frozen=True)
+class LogSource:
+    source_id: str
+    name: str
+    kind: str
+    location: str
+    description: str
+    category: str
+    status: str = "ok"
+    buffer_key: str | None = None
+    container_name: str | None = None
+
+
+@dataclass
+class StructuredLogEntry:
+    timestamp: datetime | None
+    level: str | None
+    display_line: str
+    raw_line: str
+    search_text: str
+
+
+@dataclass
+class DailyLogMarker:
+    date_token: str
+    total: int
+    dominant_level: str
+
+
+LOG_SOURCES: dict[str, LogSource] = {
+    "backend": LogSource(
+        source_id="backend",
+        name="后端服务",
+        kind="file",
+        location="/tmp/planet_backend.log",
+        description="FastAPI 后端、调度器和采集任务共享日志。",
+        category="service",
+    ),
+    "frontend": LogSource(
+        source_id="frontend",
+        name="前端开发服务",
+        kind="file",
+        location="/tmp/planet_frontend.log",
+        description="控制台与 Earth 前端开发服务输出。",
+        category="service",
+    ),
+    "ai-provider": LogSource(
+        source_id="ai-provider",
+        name="AI Provider",
+        kind="docker",
+        location="docker://planet_aiprovider",
+        description="AI Provider 容器实时输出日志。",
+        category="service",
+        container_name="planet_aiprovider",
+    ),
+    "earth-client": LogSource(
+        source_id="earth-client",
+        name="Earth 浏览器端",
+        kind="buffer",
+        location="redis://planet:system_logs:earth-client",
+        description="Earth 浏览器端上报的运行时错误与关键业务日志。",
+        category="client",
+        buffer_key=f"{LOG_BUFFER_KEY_PREFIX}:earth-client",
+    ),
+}
+
+
+def normalize_log_level(level: str | None) -> str:
+    if level is None:
+        return LOG_LEVEL_ALL
+    normalized = str(level).strip().lower()
+    if normalized in {"", LOG_LEVEL_ALL}:
+        return LOG_LEVEL_ALL
+    return LOG_LEVEL_ALIASES.get(normalized, LOG_LEVEL_ALL)
+
+
+def normalize_log_levels(level: str | None = None, levels: str | None = None) -> tuple[str, ...]:
+    normalized_levels: list[str] = []
+    if levels:
+        for item in str(levels).split(","):
+            normalized = normalize_log_level(item)
+            if normalized != LOG_LEVEL_ALL and normalized not in normalized_levels:
+                normalized_levels.append(normalized)
+    normalized_level = normalize_log_level(level)
+    if normalized_level != LOG_LEVEL_ALL and normalized_level not in normalized_levels:
+        normalized_levels.append(normalized_level)
+    return tuple(normalized_levels)
+
+
+def get_source_status(source: LogSource) -> str:
+    if source.kind == "file":
+        path = Path(source.location)
+        if not path.exists():
+            return "missing"
+        return "ok" if path.stat().st_size > 0 else "empty"
+    if source.kind == "docker":
+        return "ok" if shutil.which("docker") else "docker_unavailable"
+    if source.kind == "buffer":
+        if not source.buffer_key:
+            return "source_unavailable"
+        try:
+            return "ok" if redis_client.llen(source.buffer_key) > 0 else "empty"
+        except Exception:
+            return "source_unavailable"
+    return "source_unavailable"
+
+
+def list_log_sources() -> list[dict[str, str]]:
+    items: list[dict[str, str]] = []
+    for source in LOG_SOURCES.values():
+        items.append(
+            {
+                "source_id": source.source_id,
+                "name": source.name,
+                "kind": source.kind,
+                "location": source.location,
+                "description": source.description,
+                "category": source.category,
+                "status": get_source_status(source),
+            }
+        )
+    return items
+
+
+def get_buffer_log_key(source_id: str) -> str:
+    return f"{LOG_BUFFER_KEY_PREFIX}:{source_id}"
+
+
+def append_buffer_log(
+    source_id: str,
+    *,
+    level: str,
+    message: str,
+    context: dict[str, Any] | None = None,
+) -> None:
+    payload = {
+        "timestamp": datetime.now(tz=UTC).isoformat(),
+        "level": normalize_log_level(level),
+        "message": message,
+        "context": context or {},
+    }
+    buffer_key = get_buffer_log_key(source_id)
+    redis_client.rpush(buffer_key, json.dumps(payload, ensure_ascii=False))
+    redis_client.ltrim(buffer_key, -BUFFER_LOG_LIMIT, -1)
+    redis_client.expire(buffer_key, BUFFER_LOG_TTL_SECONDS)
+
+
+def parse_timestamp(raw_value: str | None) -> datetime | None:
+    if not raw_value:
+        return None
+    candidate = str(raw_value).strip()
+    if not candidate:
+        return None
+    candidate = candidate.replace("Z", "+00:00")
+    try:
+        parsed = datetime.fromisoformat(candidate)
+        return parsed if parsed.tzinfo else parsed.replace(tzinfo=UTC)
+    except ValueError:
+        pass
+    for fmt in TIMESTAMP_FORMATS:
+        try:
+            return datetime.strptime(candidate, fmt).replace(tzinfo=UTC)
+        except ValueError:
+            continue
+    return None
+
+
+def parse_prefixed_timestamp(line: str) -> tuple[datetime | None, str]:
+    stripped = line.strip()
+    if not stripped:
+        return None, ""
+    for prefix_length in (35, 32, 29, 26, 23, 19):
+        if len(stripped) < prefix_length:
+            continue
+        prefix = stripped[:prefix_length]
+        timestamp = parse_timestamp(prefix)
+        if timestamp is not None:
+            return timestamp, stripped[prefix_length:].lstrip()
+    first_token = stripped.split(maxsplit=1)[0]
+    timestamp = parse_timestamp(first_token)
+    if timestamp is not None:
+        remainder = stripped[len(first_token):].lstrip()
+        return timestamp, remainder
+    return None, stripped
+
+
+def infer_log_level_from_text(text: str, *, allow_embedded: bool = True) -> str | None:
+    leading_match = LEADING_LEVEL_PATTERN.match(text)
+    if leading_match:
+        return normalize_log_level(leading_match.group(1))
+
+    if allow_embedded:
+        embedded_match = EMBEDDED_LEVEL_PATTERN.search(text)
+        if embedded_match:
+            return normalize_log_level(embedded_match.group(1))
+        upper_text = text.upper()
+        for pattern, normalized in LEVEL_PATTERNS:
+            if f"{pattern}:" in upper_text or f"{pattern} " in upper_text:
+                return normalized
+    return None
+
+
+def build_display_line(timestamp: datetime | None, level: str | None, message: str) -> str:
+    message_part = message.strip() if message else ""
+    parts = []
+    if timestamp is not None:
+        parts.append(timestamp.astimezone(UTC).strftime("%Y-%m-%d %H:%M:%S"))
+    if level:
+        parts.append(level.upper())
+    if message_part:
+        parts.append(message_part)
+    return " ".join(parts).strip()
+
+
+def sanitize_text_log_line(line: str) -> str:
+    return CONTROL_CHAR_PATTERN.sub("", line)
+
+
+def parse_text_log_entry(line: str) -> StructuredLogEntry:
+    sanitized_line = sanitize_text_log_line(line).rstrip("\n")
+    timestamp, remainder = parse_prefixed_timestamp(sanitized_line)
+    level = infer_log_level_from_text(remainder or sanitized_line, allow_embedded=False)
+    display_line = sanitized_line
+    return StructuredLogEntry(
+        timestamp=timestamp,
+        level=level,
+        display_line=display_line,
+        raw_line=display_line,
+        search_text=display_line.lower(),
+    )
+
+
+def build_buffer_entry(payload: dict[str, Any]) -> StructuredLogEntry:
+    timestamp = parse_timestamp(str(payload.get("timestamp", "")).strip())
+    level = normalize_log_level(payload.get("level"))
+    if level == LOG_LEVEL_ALL:
+        level = None
+    message = str(payload.get("message", "")).strip()
+    context = payload.get("context")
+    context_map = context if isinstance(context, dict) else {}
+    context_fragments = []
+    for key in ("category", "module", "url", "detail"):
+        value = str(context_map.get(key, "")).strip()
+        if value:
+            context_fragments.append(f"{key}={value}")
+    message_with_context = " | ".join([message, *context_fragments]) if context_fragments else message
+    display_line = build_display_line(timestamp, level, message_with_context)
+    search_text = " ".join(
+        [
+            message,
+            json.dumps(context_map, ensure_ascii=False, sort_keys=True),
+            display_line,
+        ]
+    ).lower()
+    return StructuredLogEntry(
+        timestamp=timestamp,
+        level=level,
+        display_line=display_line,
+        raw_line=json.dumps(payload, ensure_ascii=False, sort_keys=True),
+        search_text=search_text,
+    )
+
+
+def read_file_entries(source: LogSource, scan_limit: int) -> list[StructuredLogEntry]:
+    path = Path(source.location)
+    if not path.exists():
+        return []
+    with path.open("r", encoding="utf-8", errors="replace") as handle:
+        recent_lines = deque(handle, maxlen=scan_limit)
+    return [
+        parse_text_log_entry(line)
+        for line in recent_lines
+        if sanitize_text_log_line(line).strip()
+    ]
+
+
+def read_docker_entries(source: LogSource, scan_limit: int) -> list[StructuredLogEntry]:
+    if not shutil.which("docker") or not source.container_name:
+        return []
+    try:
+        completed = subprocess.run(
+            [
+                "docker",
+                "logs",
+                "--timestamps",
+                "--tail",
+                str(scan_limit),
+                source.container_name,
+            ],
+            capture_output=True,
+            text=True,
+            check=False,
+        )
+    except OSError:
+        return []
+    if completed.returncode != 0:
+        return []
+    return [
+        parse_text_log_entry(line)
+        for line in completed.stdout.splitlines()
+        if line.strip()
+    ]
+
+
+def read_buffer_entries(source: LogSource, scan_limit: int) -> list[StructuredLogEntry]:
+    if not source.buffer_key:
+        return []
+    try:
+        raw_items = redis_client.lrange(source.buffer_key, -scan_limit, -1)
+    except Exception:
+        return []
+    entries: list[StructuredLogEntry] = []
+    for raw_item in raw_items:
+        try:
+            payload = json.loads(raw_item)
+        except json.JSONDecodeError:
+            entries.append(parse_text_log_entry(str(raw_item)))
+            continue
+        if isinstance(payload, dict):
+            entries.append(build_buffer_entry(payload))
+        else:
+            entries.append(parse_text_log_entry(str(raw_item)))
+    return entries
+
+
+def read_source_entries(source: LogSource, scan_limit: int) -> list[StructuredLogEntry]:
+    if source.kind == "file":
+        return read_file_entries(source, scan_limit)
+    if source.kind == "docker":
+        return read_docker_entries(source, scan_limit)
+    if source.kind == "buffer":
+        return read_buffer_entries(source, scan_limit)
+    return []
+
+
+def matches_levels(entry: StructuredLogEntry, selected_levels: tuple[str, ...]) -> bool:
+    if not selected_levels:
+        return True
+    return entry.level in selected_levels
+
+
+def matches_date_range(
+    entry: StructuredLogEntry,
+    start_date: str | None,
+    end_date: str | None,
+) -> bool:
+    if not start_date and not end_date:
+        return True
+    if entry.timestamp is None:
+        return False
+    date_token = entry.timestamp.astimezone(UTC).date().isoformat()
+    if start_date and date_token < start_date:
+        return False
+    if end_date and date_token > end_date:
+        return False
+    return True
+
+
+def matches_search(entry: StructuredLogEntry, search: str | None) -> bool:
+    if search is None:
+        return True
+    query = search.strip().lower()
+    if not query:
+        return True
+    return query in entry.search_text
+
+
+def build_daily_log_markers(entries: list[StructuredLogEntry]) -> list[dict[str, Any]]:
+    grouped: dict[str, list[StructuredLogEntry]] = {}
+    for entry in entries:
+        if entry.timestamp is None:
+            continue
+        date_token = entry.timestamp.astimezone(UTC).date().isoformat()
+        grouped.setdefault(date_token, []).append(entry)
+
+    markers: list[DailyLogMarker] = []
+    for date_token, group in sorted(grouped.items()):
+        level_counts = Counter(
+            entry.level
+            for entry in group
+            if entry.level in SUPPORTED_LOG_LEVELS and entry.level != LOG_LEVEL_ALL
+        )
+        dominant_level = LOG_LEVEL_INFO
+        if level_counts:
+            dominant_level = sorted(
+                level_counts.items(),
+                key=lambda item: (
+                    -item[1],
+                    ("error", "warning", "info", "debug").index(item[0]),
+                ),
+            )[0][0]
+        markers.append(
+            DailyLogMarker(
+                date_token=date_token,
+                total=len(group),
+                dominant_level=dominant_level,
+            )
+        )
+    return [marker.__dict__ for marker in markers]
+
+
+def read_log_snapshot(
+    source_id: str,
+    limit: int,
+    *,
+    level: str = LOG_LEVEL_ALL,
+    levels: str | None = None,
+    start_date: str | None = None,
+    end_date: str | None = None,
+    search: str | None = None,
+) -> dict[str, Any] | None:
+    source = LOG_SOURCES.get(source_id)
+    if source is None:
+        return None
+
+    selected_levels = normalize_log_levels(level, levels)
+    search_query = (search or "").strip()
+    scan_limit = max(min(MAX_LOG_LINE_LIMIT * 5, 5000), limit * 5, BUFFER_LOG_LIMIT if source.kind == "buffer" else 1000)
+    all_entries = read_source_entries(source, scan_limit)
+    marker_entries = [
+        entry
+        for entry in all_entries
+        if matches_levels(entry, selected_levels) and matches_search(entry, search_query)
+    ]
+    filtered_entries = [
+        entry
+        for entry in marker_entries
+        if matches_date_range(entry, start_date, end_date)
+    ]
+    visible_entries = filtered_entries[-limit:]
+
+    compatibility_level = selected_levels[0] if len(selected_levels) == 1 else LOG_LEVEL_ALL
+    return {
+        "source_id": source.source_id,
+        "name": source.name,
+        "kind": source.kind,
+        "location": source.location,
+        "description": source.description,
+        "category": source.category,
+        "status": get_source_status(source),
+        "level": compatibility_level,
+        "selected_levels": list(selected_levels),
+        "search_query": search_query,
+        "available_levels": [
+            LOG_LEVEL_ALL,
+            LOG_LEVEL_ERROR,
+            LOG_LEVEL_WARNING,
+            LOG_LEVEL_INFO,
+            LOG_LEVEL_DEBUG,
+        ],
+        "daily_markers": build_daily_log_markers(marker_entries),
+        "line_limit": limit,
+        "line_count": len(visible_entries),
+        "lines": [entry.display_line for entry in visible_entries],
+    }

backend/app/services/tv_streams.py

@@ -17,7 +17,7 @@ TV_LIVE_SOURCE_COLLECTOR = "news_live_streams"
 TV_LIVE_SOURCE_DATA_TYPE = "news_live_stream"
 
 DEFAULT_TV_SETTINGS = {
-        "default_source_id": DEFAULT_TV_SOURCE_ID,
+    "default_source_id": DEFAULT_TV_SOURCE_ID,
     "auto_fallback": True,
     "sources": [
         {
@@ -362,7 +362,7 @@ def _build_collected_tv_source(record: CollectedData, index: int) -> dict[str, A
             "sort_order": metadata.get("sort_order", 200 + index),
             "collector_source": record.source,
             "notes": record.description or metadata.get("notes") or "",
-            "updated_at": to_iso8601_utc(record.updated_at or record.reference_date or datetime.now(UTC)),
+            "updated_at": to_iso8601_utc(record.collected_at or record.reference_date or datetime.now(UTC)),
         },
         index=index,
     )

backend/scripts/system_restart_runner.py

@@ -2,6 +2,7 @@ from __future__ import annotations
 
 import argparse
 import os
+import shlex
 import subprocess
 import sys
 import time
@@ -59,6 +60,8 @@ def wait_for_recovery(action: str) -> tuple[bool, str]:
     recovery_mode = get_action_recovery_mode(action)
     if recovery_mode == "backend":
         return wait_for_http("http://localhost:8000/health"), "backend health recovery"
+    if recovery_mode == "frontend":
+        return wait_for_http("http://localhost:3000"), "frontend entrypoint recovery"
     if recovery_mode == "ai-provider":
         return wait_for_http("http://localhost:8010/health"), "ai provider health recovery"
     if recovery_mode == "database":
@@ -108,8 +111,9 @@ def main() -> int:
         )
         append_task_log(args.task_id, "restart command started")
 
+        shell_command = shlex.join(command)
         completed = subprocess.run(
-            command,
+            ["zsh", "-ic", shell_command],
             cwd=str(ROOT_DIR),
             env=env,
             capture_output=True,

backend/tests/test_api.py

@@ -35,6 +35,7 @@ async def test_health_check():
     data = response.json()
     assert data["status"] == "healthy"
     assert "version" in data
+    assert response.headers["x-request-id"]
 
 
 @pytest.mark.asyncio
@@ -161,6 +162,345 @@ async def test_alerts_endpoint_with_auth(auth_headers):
         app.dependency_overrides.clear()
 
 
+@pytest.mark.asyncio
+async def test_system_log_sources_requires_super_admin(auth_headers):
+    def override_get_current_user():
+        return User(
+            id=1,
+            username="testuser",
+            email="test@example.com",
+            password_hash="hashed",
+            role="admin",
+            is_active=True,
+        )
+
+    app.dependency_overrides = {
+        __import__("app.core.security", fromlist=["get_current_user"]).get_current_user: override_get_current_user,
+    }
+    transport = ASGITransport(app=app)
+    try:
+        async with AsyncClient(transport=transport, base_url="http://test") as client:
+            response = await client.get("/api/v1/system/logs/sources", headers=auth_headers)
+        assert response.status_code == 403
+    finally:
+        app.dependency_overrides.clear()
+
+
+@pytest.mark.asyncio
+async def test_system_log_sources_with_super_admin(auth_headers):
+    def override_get_current_user():
+        return User(
+            id=1,
+            username="root",
+            email="root@example.com",
+            password_hash="hashed",
+            role="super_admin",
+            is_active=True,
+        )
+
+    app.dependency_overrides = {
+        __import__("app.core.security", fromlist=["get_current_user"]).get_current_user: override_get_current_user,
+    }
+    transport = ASGITransport(app=app)
+    try:
+        with patch(
+            "app.api.v1.system_control.list_log_sources",
+            return_value=[
+                {
+                    "source_id": "backend",
+                    "name": "后端服务",
+                    "kind": "file",
+                    "location": "/tmp/planet_backend.log",
+                    "description": "FastAPI 后端、调度器和采集任务共享日志。",
+                    "category": "service",
+                    "status": "ok",
+                }
+            ],
+        ):
+            async with AsyncClient(transport=transport, base_url="http://test") as client:
+                response = await client.get("/api/v1/system/logs/sources", headers=auth_headers)
+        assert response.status_code == 200
+        data = response.json()
+        assert data["items"][0]["source_id"] == "backend"
+    finally:
+        app.dependency_overrides.clear()
+
+
+@pytest.mark.asyncio
+async def test_system_log_snapshot_with_super_admin(auth_headers):
+    def override_get_current_user():
+        return User(
+            id=1,
+            username="root",
+            email="root@example.com",
+            password_hash="hashed",
+            role="super_admin",
+            is_active=True,
+        )
+
+    app.dependency_overrides = {
+        __import__("app.core.security", fromlist=["get_current_user"]).get_current_user: override_get_current_user,
+    }
+    transport = ASGITransport(app=app)
+    try:
+        with patch(
+            "app.api.v1.system_control.read_log_snapshot",
+            return_value={
+                "source_id": "backend",
+                "name": "后端服务",
+                "kind": "file",
+                "location": "/tmp/planet_backend.log",
+                "description": "FastAPI 后端、调度器和采集任务共享日志。",
+                "category": "service",
+                "status": "ok",
+                "level": "all",
+                "selected_levels": [],
+                "search_query": "",
+                "available_levels": ["all", "error", "warning", "info", "debug"],
+                "daily_markers": [],
+                "line_limit": 50,
+                "line_count": 2,
+                "lines": ["line 1", "line 2"],
+            },
+        ):
+            async with AsyncClient(transport=transport, base_url="http://test") as client:
+                response = await client.get("/api/v1/system/logs/backend?limit=50", headers=auth_headers)
+        assert response.status_code == 200
+        data = response.json()
+        assert data["source_id"] == "backend"
+        assert data["line_count"] == 2
+        assert data["lines"] == ["line 1", "line 2"]
+    finally:
+        app.dependency_overrides.clear()
+
+
+@pytest.mark.asyncio
+async def test_system_log_snapshot_supports_level_filter(auth_headers):
+    def override_get_current_user():
+        return User(
+            id=1,
+            username="root",
+            email="root@example.com",
+            password_hash="hashed",
+            role="super_admin",
+            is_active=True,
+        )
+
+    app.dependency_overrides = {
+        __import__("app.core.security", fromlist=["get_current_user"]).get_current_user: override_get_current_user,
+    }
+    transport = ASGITransport(app=app)
+    try:
+        with patch(
+            "app.api.v1.system_control.read_log_snapshot",
+            return_value={
+                "source_id": "backend",
+                "name": "后端服务",
+                "kind": "file",
+                "location": "/tmp/planet_backend.log",
+                "description": "FastAPI 后端、调度器和采集任务共享日志。",
+                "category": "service",
+                "status": "ok",
+                "level": "error",
+                "selected_levels": ["error"],
+                "search_query": "",
+                "available_levels": ["all", "error", "warning", "info", "debug"],
+                "daily_markers": [],
+                "line_limit": 50,
+                "line_count": 1,
+                "lines": ["ERROR: failed"],
+            },
+        ):
+            async with AsyncClient(transport=transport, base_url="http://test") as client:
+                response = await client.get("/api/v1/system/logs/backend?limit=50&level=error", headers=auth_headers)
+        assert response.status_code == 200
+        data = response.json()
+        assert data["level"] == "error"
+        assert data["lines"] == ["ERROR: failed"]
+    finally:
+        app.dependency_overrides.clear()
+
+
+@pytest.mark.asyncio
+async def test_system_log_snapshot_supports_date_range_filter(auth_headers):
+    def override_get_current_user():
+        return User(
+            id=1,
+            username="root",
+            email="root@example.com",
+            password_hash="hashed",
+            role="super_admin",
+            is_active=True,
+        )
+
+    app.dependency_overrides = {
+        __import__("app.core.security", fromlist=["get_current_user"]).get_current_user: override_get_current_user,
+    }
+    transport = ASGITransport(app=app)
+    try:
+        with patch(
+            "app.api.v1.system_control.read_log_snapshot",
+            return_value={
+                "source_id": "backend",
+                "name": "后端服务",
+                "kind": "file",
+                "location": "/tmp/planet_backend.log",
+                "description": "FastAPI 后端、调度器和采集任务共享日志。",
+                "category": "service",
+                "status": "ok",
+                "level": "all",
+                "selected_levels": [],
+                "search_query": "",
+                "available_levels": ["all", "error", "warning", "info", "debug"],
+                "daily_markers": [],
+                "line_limit": 50,
+                "line_count": 1,
+                "lines": ["2026-04-23 INFO: service started"],
+            },
+        ) as mock_read_log_snapshot:
+            async with AsyncClient(transport=transport, base_url="http://test") as client:
+                response = await client.get(
+                    "/api/v1/system/logs/backend?limit=50&start_date=2026-04-20&end_date=2026-04-23",
+                    headers=auth_headers,
+                )
+        assert response.status_code == 200
+        mock_read_log_snapshot.assert_called_once_with(
+            "backend",
+            50,
+            level="all",
+            levels=None,
+            start_date="2026-04-20",
+            end_date="2026-04-23",
+            search=None,
+        )
+    finally:
+        app.dependency_overrides.clear()
+
+
+@pytest.mark.asyncio
+async def test_system_log_snapshot_supports_levels_and_search_filter(auth_headers):
+    def override_get_current_user():
+        return User(
+            id=1,
+            username="root",
+            email="root@example.com",
+            password_hash="hashed",
+            role="super_admin",
+            is_active=True,
+        )
+
+    app.dependency_overrides = {
+        __import__("app.core.security", fromlist=["get_current_user"]).get_current_user: override_get_current_user,
+    }
+    transport = ASGITransport(app=app)
+    try:
+        with patch(
+            "app.api.v1.system_control.read_log_snapshot",
+            return_value={
+                "source_id": "backend",
+                "name": "后端服务",
+                "kind": "file",
+                "location": "/tmp/planet_backend.log",
+                "description": "FastAPI 后端、调度器和采集任务共享日志。",
+                "category": "service",
+                "status": "ok",
+                "level": "all",
+                "selected_levels": ["error", "warning"],
+                "search_query": "timeout",
+                "available_levels": ["all", "error", "warning", "info", "debug"],
+                "daily_markers": [],
+                "line_limit": 50,
+                "line_count": 1,
+                "lines": ["2026-04-23 10:00:00 ERROR timeout"],
+            },
+        ) as mock_read_log_snapshot:
+            async with AsyncClient(transport=transport, base_url="http://test") as client:
+                response = await client.get(
+                    "/api/v1/system/logs/backend?limit=50&levels=error,warning&search=timeout",
+                    headers=auth_headers,
+                )
+        assert response.status_code == 200
+        mock_read_log_snapshot.assert_called_once_with(
+            "backend",
+            50,
+            level="all",
+            levels="error,warning",
+            start_date=None,
+            end_date=None,
+            search="timeout",
+        )
+    finally:
+        app.dependency_overrides.clear()
+
+
+@pytest.mark.asyncio
+async def test_system_log_snapshot_rejects_invalid_date_range(auth_headers):
+    def override_get_current_user():
+        return User(
+            id=1,
+            username="root",
+            email="root@example.com",
+            password_hash="hashed",
+            role="super_admin",
+            is_active=True,
+        )
+
+    app.dependency_overrides = {
+        __import__("app.core.security", fromlist=["get_current_user"]).get_current_user: override_get_current_user,
+    }
+    transport = ASGITransport(app=app)
+    try:
+        async with AsyncClient(transport=transport, base_url="http://test") as client:
+            response = await client.get(
+                "/api/v1/system/logs/backend?start_date=2026-04-31",
+                headers=auth_headers,
+            )
+        assert response.status_code == 400
+        assert "start_date must be in YYYY-MM-DD format" in response.json()["detail"]
+    finally:
+        app.dependency_overrides.clear()
+
+
+@pytest.mark.asyncio
+async def test_ingest_earth_client_log_accepts_public_events():
+    transport = ASGITransport(app=app)
+    try:
+        with patch("app.api.v1.system_control.append_buffer_log") as mock_append_buffer_log:
+            with patch("app.api.v1.system_control.record_system_log", new_callable=AsyncMock) as mock_record_system_log:
+                async with AsyncClient(transport=transport, base_url="http://test") as client:
+                    response = await client.post(
+                        "/api/v1/system/logs/earth-client",
+                        json={
+                            "level": "error",
+                            "message": "登陆点加载失败: 登陆点接口返回 HTTP 500",
+                            "category": "startup-load",
+                            "module": "layer-startup",
+                        },
+                    )
+        assert response.status_code == 200
+        data = response.json()
+        assert data["accepted"] is True
+        assert data["source_id"] == "earth-client"
+        mock_append_buffer_log.assert_called_once()
+        mock_record_system_log.assert_awaited_once()
+        persisted_kwargs = mock_record_system_log.await_args.kwargs
+        assert persisted_kwargs["source"] == "earth-client"
+        assert persisted_kwargs["event"] == "earth.client.runtime_log"
+        assert persisted_kwargs["category"] == "startup-load"
+        assert persisted_kwargs["level"] == "error"
+    finally:
+        app.dependency_overrides.clear()
+
+
+@pytest.mark.asyncio
+async def test_request_id_header_is_echoed_when_provided():
+    transport = ASGITransport(app=app)
+    async with AsyncClient(transport=transport, base_url="http://test") as client:
+        response = await client.get("/health", headers={"X-Request-ID": "planet-test-request"})
+    assert response.status_code == 200
+    assert response.headers["x-request-id"] == "planet-test-request"
+
+
 @pytest.mark.asyncio
 async def test_invalid_token():
     """Test that invalid token is rejected"""
@@ -263,6 +603,8 @@ async def test_ai_situational_analysis_returns_503_when_disabled(auth_headers):
         assert "content_blocks" in data
         assert "text_blocks" in data
         assert "thinking_blocks" in data
+    finally:
+        app.dependency_overrides.clear()
 
 
 @pytest.mark.asyncio
@@ -382,8 +724,6 @@ async def test_save_playground_session_with_auth(auth_headers):
         assert data["state"]["objective"] == "测试目标"
     finally:
         app.dependency_overrides.clear()
-    finally:
-        app.dependency_overrides.clear()
 
 
 @pytest.mark.asyncio

backend/tests/test_datasource_mapping.py

@@ -0,0 +1,199 @@
+from types import SimpleNamespace
+
+import pytest
+from httpx import ASGITransport, AsyncClient
+
+from app.api.v1.datasource_config import get_ai_provider_client
+from app.core.security import get_current_user
+from app.core.target_schema_registry import get_target_schema, list_target_schemas
+from app.main import app
+from app.models.user import User
+from app.services.datasource_mapping import execute_mapping, persist_mapped_records, redact_for_llm
+
+
+SAMPLE_AIS = {
+    "data": [
+        {
+            "mmsi": "257123000",
+            "latitude": "59.91",
+            "longitude": "10.75",
+            "speedOverGround": "12.4",
+            "timestamp": "2026-04-28T00:00:00Z",
+            "api_token": "secret-value",
+        }
+    ]
+}
+
+
+def test_registry_exposes_v1_target_schemas():
+    keys = {schema["key"] for schema in list_target_schemas()}
+
+    assert {"vessel_ais", "geo_points", "generic_records"}.issubset(keys)
+    assert get_target_schema("vessel_ais").destination == "vessel_position"
+
+
+def test_mapping_engine_maps_and_validates_vessel_ais():
+    mapping = {
+        "source": {"items_path": "$.data[*]"},
+        "fields": {
+            "mmsi": {"path": "$.mmsi", "type": "integer"},
+            "lat": {"path": "$.latitude", "type": "float"},
+            "lon": {"path": "$.longitude", "type": "float"},
+            "sog": {"path": "$.speedOverGround", "type": "float"},
+            "received_at": {"path": "$.timestamp", "type": "datetime"},
+        },
+    }
+
+    result = execute_mapping(SAMPLE_AIS, mapping, "vessel_ais")
+
+    assert result["mapped_count"] == 1
+    assert result["failed_count"] == 0
+    assert result["records"][0]["mmsi"] == 257123000
+    assert result["records"][0]["lat"] == 59.91
+
+
+def test_mapping_engine_reports_schema_errors():
+    mapping = {
+        "source": {"items_path": "$.data[*]"},
+        "fields": {
+            "mmsi": {"path": "$.mmsi", "type": "integer"},
+            "lat": {"path": "$.missing_lat", "type": "float"},
+            "lon": {"path": "$.longitude", "type": "float"},
+        },
+    }
+
+    result = execute_mapping(SAMPLE_AIS, mapping, "vessel_ais")
+
+    assert result["mapped_count"] == 0
+    assert result["failed_count"] == 1
+    assert any("lat" in error for error in result["errors"][0]["errors"])
+
+
+def test_redact_for_llm_masks_secret_like_fields():
+    redacted = redact_for_llm(SAMPLE_AIS)
+
+    assert redacted["data"][0]["api_token"] == "[REDACTED]"
+
+
+@pytest.mark.asyncio
+async def test_persist_mapped_records_writes_generic_records():
+    class FakeDB:
+        def __init__(self):
+            self.added = []
+            self.committed = False
+
+        def add(self, value):
+            self.added.append(value)
+
+        async def commit(self):
+            self.committed = True
+
+    db = FakeDB()
+
+    count = await persist_mapped_records(
+        db,
+        datasource_name="custom_weather",
+        datasource_config_id=42,
+        target_schema="generic_records",
+        records=[{"source_id": "row-1", "data": {"temp": 25}}],
+        mapping_version=3,
+    )
+
+    assert count == 1
+    assert db.committed is True
+    assert db.added[0].source == "custom_weather"
+    assert db.added[0].data_type == "generic_records"
+    assert db.added[0].extra_data["mapping_version"] == 3
+
+
+@pytest.mark.asyncio
+async def test_mapping_preview_api_uses_deterministic_engine():
+    def override_get_current_user():
+        return User(
+            id=1,
+            username="testuser",
+            email="test@example.com",
+            password_hash="hashed",
+            role="admin",
+            is_active=True,
+        )
+
+    app.dependency_overrides = {get_current_user: override_get_current_user}
+    transport = ASGITransport(app=app)
+    try:
+        async with AsyncClient(transport=transport, base_url="http://test") as client:
+            response = await client.post(
+                "/api/v1/datasources/mappings/preview",
+                json={
+                    "sample_payload": SAMPLE_AIS,
+                    "target_schema": "vessel_ais",
+                    "mapping_json": {
+                        "source": {"items_path": "$.data[*]"},
+                        "fields": {
+                            "mmsi": {"path": "$.mmsi", "type": "integer"},
+                            "lat": {"path": "$.latitude", "type": "float"},
+                            "lon": {"path": "$.longitude", "type": "float"},
+                        },
+                    },
+                },
+            )
+    finally:
+        app.dependency_overrides.clear()
+
+    assert response.status_code == 200
+    payload = response.json()
+    assert payload["success"] is True
+    assert payload["preview"]["records"][0]["mmsi"] == 257123000
+
+
+@pytest.mark.asyncio
+async def test_mapping_propose_api_redacts_sample_before_ai():
+    seen_context = {}
+
+    class FakeAIClient:
+        async def analyze(self, request, request_id=None):
+            seen_context.update(request.context)
+            return SimpleNamespace(
+                content=(
+                    '{"source":{"items_path":"$.data[*]"},"fields":{'
+                    '"mmsi":{"path":"$.mmsi","type":"integer"},'
+                    '"lat":{"path":"$.latitude","type":"float"},'
+                    '"lon":{"path":"$.longitude","type":"float"}}}'
+                )
+            )
+
+    def override_get_current_user():
+        return User(
+            id=1,
+            username="testuser",
+            email="test@example.com",
+            password_hash="hashed",
+            role="admin",
+            is_active=True,
+        )
+
+    def override_ai_client():
+        return FakeAIClient()
+
+    app.dependency_overrides = {
+        get_current_user: override_get_current_user,
+        get_ai_provider_client: override_ai_client,
+    }
+    transport = ASGITransport(app=app)
+    try:
+        async with AsyncClient(transport=transport, base_url="http://test") as client:
+            response = await client.post(
+                "/api/v1/datasources/mappings/propose",
+                json={
+                    "sample_payload": SAMPLE_AIS,
+                    "target_schema": "vessel_ais",
+                    "use_ai": True,
+                },
+            )
+    finally:
+        app.dependency_overrides.clear()
+
+    assert response.status_code == 200
+    payload = response.json()
+    assert payload["mapping_json"]["meta"]["generated_by"] == "ai_provider"
+    assert seen_context["sample_payload"]["data"][0]["api_token"] == "[REDACTED]"

backend/tests/test_earth_news.py

@@ -0,0 +1,49 @@
+from datetime import UTC, datetime
+
+from app.services.earth_news import ParsedNewsItem, _serialize_item
+
+
+def test_serialize_item_includes_region_anchor_for_cruise():
+    item = ParsedNewsItem(
+        id="google-apac:test",
+        title="Example APAC story",
+        summary="Example summary",
+        url="https://example.com/story",
+        source="Example Source",
+        feed_name="Global Monitor / APAC",
+        feed_region="asia-pacific",
+        homepage_url="https://example.com",
+        published_at=datetime(2026, 4, 23, 2, 30, tzinfo=UTC),
+    )
+
+    payload = _serialize_item(item, active_region="asia-pacific")
+
+    assert payload["latitude"] == 1.3521
+    assert payload["longitude"] == 103.8198
+    assert payload["location_label"] == "亚太"
+    assert payload["location_inferred"] is True
+    assert payload["is_focus_match"] is True
+    assert payload["published_at"] == "2026-04-23T02:30:00Z"
+
+
+def test_serialize_item_falls_back_to_global_anchor():
+    item = ParsedNewsItem(
+        id="custom:test",
+        title="Fallback story",
+        summary="Fallback summary",
+        url="https://example.com/fallback",
+        source="Fallback Source",
+        feed_name="Fallback Feed",
+        feed_region="unknown-region",
+        homepage_url="https://example.com",
+        published_at=None,
+    )
+
+    payload = _serialize_item(item, active_region="americas")
+
+    assert payload["latitude"] == 20.0
+    assert payload["longitude"] == 0.0
+    assert payload["location_label"] == "全球"
+    assert payload["location_inferred"] is True
+    assert payload["is_focus_match"] is False
+    assert payload["published_at"] is None

backend/tests/test_logging.py

@@ -0,0 +1,78 @@
+from __future__ import annotations
+
+import logging
+
+from io import StringIO
+
+from app.core.logging import PlanetContextFilter, PlanetFormatter, get_logger
+from app.core.request_context import set_request_id
+
+
+def _capture_output(callback):
+    stream = StringIO()
+    handler = logging.StreamHandler(stream)
+    handler.setFormatter(PlanetFormatter(datefmt="%Y-%m-%d %H:%M:%S"))
+    handler.addFilter(PlanetContextFilter())
+
+    adapter = get_logger("tests.logging")
+    target_logger = adapter.logger
+    original_handlers = list(target_logger.handlers)
+    original_level = target_logger.level
+    original_propagate = target_logger.propagate
+
+    target_logger.handlers = [handler]
+    target_logger.setLevel(logging.INFO)
+    target_logger.propagate = False
+
+    try:
+        callback(adapter)
+    finally:
+        handler.flush()
+        target_logger.handlers = original_handlers
+        target_logger.setLevel(original_level)
+        target_logger.propagate = original_propagate
+
+    return stream.getvalue()
+
+
+def test_structured_logger_injects_request_id_and_event():
+    set_request_id("req-test-123")
+    try:
+        output = _capture_output(
+            lambda logger: logger.info_event(
+                "collector started",
+                event="collector.run.started",
+                context={"collector_name": "bgp_news"},
+            )
+        )
+    finally:
+        set_request_id(None)
+
+    assert "request_id=req-test-123" in output
+    assert "event=collector.run.started" in output
+    assert "service=backend" in output
+    assert '"collector_name": "bgp_news"' in output
+
+
+def test_structured_logger_redacts_sensitive_text_and_context():
+    set_request_id("req-test-redact")
+    try:
+        output = _capture_output(
+            lambda logger: logger.error_event(
+                "Authorization: Bearer super-secret-token",
+                event="auth.token.failed",
+                context={
+                    "token": "plain-secret",
+                    "nested": {"password": "hunter2"},
+                    "safe": "visible",
+                },
+            )
+        )
+    finally:
+        set_request_id(None)
+
+    assert "super-secret-token" not in output
+    assert "plain-secret" not in output
+    assert "hunter2" not in output
+    assert "[REDACTED]" in output
+    assert '"safe": "visible"' in output

backend/tests/test_system_logs.py

@@ -0,0 +1,217 @@
+from __future__ import annotations
+
+import json
+
+from pathlib import Path
+
+from app.services import system_logs
+
+
+class FakeRedis:
+    def __init__(self) -> None:
+        self.store: dict[str, list[str]] = {}
+
+    def rpush(self, key: str, value: str) -> None:
+        self.store.setdefault(key, []).append(value)
+
+    def ltrim(self, key: str, start: int, end: int) -> None:
+        items = self.store.get(key, [])
+        normalized_end = None if end == -1 else end + 1
+        self.store[key] = items[start:normalized_end]
+
+    def expire(self, key: str, seconds: int) -> None:
+        return None
+
+    def lrange(self, key: str, start: int, end: int) -> list[str]:
+        items = self.store.get(key, [])
+        normalized_end = None if end == -1 else end + 1
+        return items[start:normalized_end]
+
+    def llen(self, key: str) -> int:
+        return len(self.store.get(key, []))
+
+
+def test_read_log_snapshot_uses_structured_buffer_timestamp_level_and_search(monkeypatch):
+    fake_redis = FakeRedis()
+    monkeypatch.setattr(system_logs, "redis_client", fake_redis)
+    monkeypatch.setattr(
+        system_logs,
+        "LOG_SOURCES",
+        {
+            "earth-client": system_logs.LogSource(
+                source_id="earth-client",
+                name="Earth 浏览器端",
+                kind="buffer",
+                location="redis://planet:system_logs:earth-client",
+                description="Earth 浏览器端上报日志",
+                category="client",
+                buffer_key=system_logs.get_buffer_log_key("earth-client"),
+            )
+        },
+    )
+
+    fake_redis.rpush(
+        system_logs.get_buffer_log_key("earth-client"),
+        json.dumps(
+            {
+                "timestamp": "2026-04-22T10:15:30Z",
+                "level": "warning",
+                "message": "news feed degraded",
+                "context": {"module": "news", "detail": "timeout"},
+            },
+            ensure_ascii=False,
+        ),
+    )
+    fake_redis.rpush(
+        system_logs.get_buffer_log_key("earth-client"),
+        json.dumps(
+            {
+                "timestamp": "2026-04-23T06:01:00Z",
+                "level": "error",
+                "message": "landing points failed",
+                "context": {"module": "layer-startup", "detail": "http 500"},
+            },
+            ensure_ascii=False,
+        ),
+    )
+
+    snapshot = system_logs.read_log_snapshot(
+        "earth-client",
+        50,
+        levels="error,warning",
+        start_date="2026-04-23",
+        end_date="2026-04-23",
+        search="landing",
+    )
+
+    assert snapshot is not None
+    assert snapshot["selected_levels"] == ["error", "warning"]
+    assert snapshot["search_query"] == "landing"
+    assert snapshot["line_count"] == 1
+    assert snapshot["lines"][0].startswith("2026-04-23 06:01:00 ERROR landing points failed")
+    assert snapshot["daily_markers"] == [
+        {"date_token": "2026-04-23", "total": 1, "dominant_level": "error"}
+    ]
+
+
+def test_read_log_snapshot_parses_file_timestamp_and_builds_markers(tmp_path: Path, monkeypatch):
+    log_path = tmp_path / "backend.log"
+    log_path.write_text(
+        "\n".join(
+            [
+                "2026-04-22 08:00:00 INFO service booted",
+                "2026-04-23 09:15:00 WARNING disk pressure detected",
+                "2026-04-23 09:16:00 ERROR sync failed",
+                "2026-04-24 10:00:00 DEBUG collector trace",
+            ]
+        ),
+        encoding="utf-8",
+    )
+
+    monkeypatch.setattr(
+        system_logs,
+        "LOG_SOURCES",
+        {
+            "backend": system_logs.LogSource(
+                source_id="backend",
+                name="后端服务",
+                kind="file",
+                location=str(log_path),
+                description="测试文件日志",
+                category="service",
+            )
+        },
+    )
+
+    snapshot = system_logs.read_log_snapshot(
+        "backend",
+        50,
+        levels="warning,error",
+        search="failed",
+    )
+
+    assert snapshot is not None
+    assert snapshot["line_count"] == 1
+    assert snapshot["lines"] == ["2026-04-23 09:16:00 ERROR sync failed"]
+    assert snapshot["daily_markers"] == [
+        {"date_token": "2026-04-23", "total": 1, "dominant_level": "error"}
+    ]
+    assert snapshot["status"] == "ok"
+
+
+def test_append_buffer_log_persists_normalized_level(monkeypatch):
+    fake_redis = FakeRedis()
+    monkeypatch.setattr(system_logs, "redis_client", fake_redis)
+
+    system_logs.append_buffer_log(
+        "earth-client",
+        level="warn",
+        message="feed delayed",
+        context={"module": "news"},
+    )
+
+    stored_items = fake_redis.lrange(system_logs.get_buffer_log_key("earth-client"), 0, -1)
+    payload = json.loads(stored_items[0])
+    assert payload["level"] == "warning"
+    assert payload["message"] == "feed delayed"
+
+
+def test_infer_log_level_prefers_leading_prefix_over_query_string():
+    line = 'INFO: 127.0.0.1 - "GET /api/v1/system/logs/backend?limit=200&level=error&levels=error HTTP/1.1" 200 OK'
+
+    entry = system_logs.parse_text_log_entry(line)
+
+    assert entry.level == "info"
+
+
+def test_parse_text_log_entry_does_not_promote_exception_context_to_error():
+    line = "websockets.exceptions.ConnectionClosedError: sent 1011 (internal error) keepalive ping timeout"
+
+    entry = system_logs.parse_text_log_entry(line)
+
+    assert entry.level is None
+
+
+def test_parse_text_log_entry_still_detects_explicit_error_prefix():
+    line = "ERROR: [Errno 98] Address already in use"
+
+    entry = system_logs.parse_text_log_entry(line)
+
+    assert entry.level == "error"
+
+
+def test_read_log_snapshot_strips_nul_bytes_from_file_lines(tmp_path: Path, monkeypatch):
+    log_path = tmp_path / "backend.log"
+    log_path.write_bytes(
+        (
+            b"INFO: service booted\n"
+            b"ERROR: bind failed\n"
+            + b"\x00" * 32
+            + b"2026-04-23 23:41:32 INFO service=backend message=request served\n"
+        )
+    )
+
+    monkeypatch.setattr(
+        system_logs,
+        "LOG_SOURCES",
+        {
+            "backend": system_logs.LogSource(
+                source_id="backend",
+                name="后端服务",
+                kind="file",
+                location=str(log_path),
+                description="测试文件日志",
+                category="service",
+            )
+        },
+    )
+
+    snapshot = system_logs.read_log_snapshot("backend", 50)
+
+    assert snapshot is not None
+    assert snapshot["line_count"] == 3
+    assert snapshot["lines"] == [
+        "INFO: service booted",
+        "ERROR: bind failed",
+        "2026-04-23 23:41:32 INFO service=backend message=request served",
+    ]

backend/tests/test_vessels.py

@@ -0,0 +1,149 @@
+from datetime import datetime, timedelta, timezone
+
+import pytest
+from httpx import ASGITransport, AsyncClient
+
+from app.api.v1.visualization import convert_vessels_to_geojson
+from app.db.session import get_db
+from app.main import app
+from app.models.vessel import VesselPosition, VesselStatic
+from app.services import barentswatch
+from app.services.collectors.vessel_ais import VesselAISCollector
+
+
+def test_vessel_collector_transforms_barentswatch_like_records():
+    collector = VesselAISCollector()
+    records = collector.transform(
+        [
+            {
+                "mmsi": "257123000",
+                "lat": "59.91",
+                "lon": "10.73",
+                "sog": 12.4,
+                "cog": 214,
+                "nav_status": 0,
+                "shipType": 70,
+                "name": "OSLO TRADER",
+            },
+            {"mmsi": "bad", "lat": 120, "lon": 10},
+        ]
+    )
+
+    assert len(records) == 1
+    assert records[0]["mmsi"] == 257123000
+    assert records[0]["vessel_type_name"] == "Cargo"
+    assert records[0]["lat"] == pytest.approx(59.91)
+
+
+def test_barentswatch_reads_credentials_from_zshrc(tmp_path):
+    zshrc = tmp_path / ".zshrc"
+    zshrc.write_text(
+        "\n".join(
+            [
+                "export BARENTSWATCH_CLIENT_ID='client-from-zshrc'",
+                'export BARENTSWATCH_CLIENT_SECRET="secret-from-zshrc" # local dev credential',
+            ]
+        ),
+        encoding="utf-8",
+    )
+
+    values = barentswatch._read_zshrc_env(zshrc)
+
+    assert values["BARENTSWATCH_CLIENT_ID"] == "client-from-zshrc"
+    assert values["BARENTSWATCH_CLIENT_SECRET"] == "secret-from-zshrc"
+
+
+@pytest.mark.asyncio
+async def test_barentswatch_resolves_config_from_zshrc(tmp_path, monkeypatch):
+    zshrc = tmp_path / ".zshrc"
+    zshrc.write_text(
+        "\n".join(
+            [
+                "export BARENTSWATCH_CLIENT_ID=client-from-zshrc",
+                "export BARENTSWATCH_CLIENT_SECRET=secret-from-zshrc",
+            ]
+        ),
+        encoding="utf-8",
+    )
+    monkeypatch.delenv("BARENTSWATCH_CLIENT_ID", raising=False)
+    monkeypatch.delenv("BARENTSWATCH_CLIENT_SECRET", raising=False)
+    monkeypatch.delenv("BARRENTSWATCH_CLIENT_ID", raising=False)
+    monkeypatch.delenv("BARRENTSWATCH_CLIENT_SECRET", raising=False)
+    monkeypatch.setattr(barentswatch.Path, "home", lambda: tmp_path)
+
+    config = await barentswatch.resolve_barentswatch_config(None)
+
+    assert config.client_id == "client-from-zshrc"
+    assert config.client_secret == "secret-from-zshrc"
+    assert config.credential_source == "~/.zshrc"
+
+
+def test_convert_vessels_to_geojson():
+    position = VesselPosition(
+        mmsi=257123000,
+        lat=59.91,
+        lon=10.73,
+        sog=12.4,
+        cog=214,
+        heading=215,
+        nav_status=0,
+        received_at=datetime(2026, 4, 28, 1, 0, tzinfo=timezone.utc),
+    )
+    static = VesselStatic(
+        mmsi=257123000,
+        name="OSLO TRADER",
+        vessel_type=70,
+        vessel_type_name="Cargo",
+        flag="NO",
+        length=185,
+    )
+
+    payload = convert_vessels_to_geojson([(position, static)])
+
+    assert payload["type"] == "FeatureCollection"
+    assert payload["features"][0]["geometry"]["coordinates"] == [10.73, 59.91]
+    assert payload["features"][0]["properties"]["mmsi"] == 257123000
+    assert payload["features"][0]["properties"]["vessel_type_name"] == "Cargo"
+
+
+@pytest.mark.asyncio
+async def test_vessels_geojson_endpoint_filters_type_and_bbox():
+    now = datetime(2026, 4, 28, 1, 0, tzinfo=timezone.utc)
+    rows = [
+        (
+            VesselPosition(mmsi=1, lat=59.9, lon=10.7, received_at=now),
+            VesselStatic(mmsi=1, name="Cargo Ship", vessel_type=70, vessel_type_name="Cargo"),
+        ),
+        (
+            VesselPosition(mmsi=2, lat=60.3, lon=5.3, received_at=now - timedelta(minutes=1)),
+            VesselStatic(mmsi=2, name="Passenger Ship", vessel_type=60, vessel_type_name="Passenger"),
+        ),
+    ]
+
+    class _Result:
+        def all(self):
+            return rows
+
+    class _FakeSession:
+        async def execute(self, _query):
+            return _Result()
+
+    async def override_get_db():
+        yield _FakeSession()
+
+    app.dependency_overrides[get_db] = override_get_db
+    transport = ASGITransport(app=app)
+    try:
+        async with AsyncClient(transport=transport, base_url="http://test") as client:
+            response = await client.get(
+                "/api/v1/visualization/geo/vessels",
+                params={"bbox": "0,50,20,70", "type": "cargo"},
+            )
+
+        assert response.status_code == 200
+        data = response.json()
+        assert data["count"] == 1
+        assert data["features"][0]["properties"]["name"] == "Cargo Ship"
+        assert data["stats"]["by_type"]["Cargo"] == 1
+    finally:
+        app.dependency_overrides.clear()

backend/tests/test_visualization_compute_centers.py

@@ -0,0 +1,347 @@
+from datetime import datetime, timezone
+
+import pytest
+from httpx import ASGITransport, AsyncClient
+
+from app.api.v1.visualization import convert_compute_centers_to_geojson
+from app.db.session import get_db
+from app.main import app
+from app.models.collected_data import CollectedData
+
+
+def _build_record(
+    *,
+    record_id: int,
+    source: str,
+    data_type: str,
+    name: str,
+    country: str,
+    city: str,
+    latitude: float,
+    longitude: float,
+    metadata: dict,
+):
+    return CollectedData(
+        id=record_id,
+        source=source,
+        data_type=data_type,
+        source_id=f"{source}-{record_id}",
+        name=name,
+        extra_data={
+            "country": country,
+            "city": city,
+            "latitude": latitude,
+            "longitude": longitude,
+            **metadata,
+        },
+        collected_at=datetime(2026, 4, 22, tzinfo=timezone.utc),
+        reference_date=datetime(2026, 4, 21, tzinfo=timezone.utc),
+        is_current=True,
+    )
+
+
+def test_convert_compute_centers_to_geojson_unifies_sources():
+    top500_record = _build_record(
+        record_id=1,
+        source="top500",
+        data_type="supercomputer",
+        name="Frontier",
+        country="United States",
+        city="Oak Ridge",
+        latitude=35.93,
+        longitude=-84.31,
+        metadata={
+            "rank": 1,
+            "manufacturer": "HPE",
+            "organization": "ORNL",
+            "rmax": 1102000.0,
+            "cores": 8730112,
+            "power": 21510.0,
+        },
+    )
+    gpu_record = _build_record(
+        record_id=2,
+        source="epoch_ai_gpu",
+        data_type="gpu_cluster",
+        name="Colossus",
+        country="United States",
+        city="Memphis",
+        latitude=35.15,
+        longitude=-90.05,
+        metadata={
+            "organization": "xAI",
+            "gpu_type": "H100",
+            "gpu_count": 100000,
+            "value": "20000",
+            "unit": "TFlop/s",
+        },
+    )
+
+    payload = convert_compute_centers_to_geojson([top500_record, gpu_record])
+
+    assert payload["type"] == "FeatureCollection"
+    assert len(payload["features"]) == 2
+
+    supercomputer_feature = payload["features"][0]
+    assert supercomputer_feature["properties"]["site_type"] == "supercomputer"
+    assert supercomputer_feature["properties"]["capacity_unit"] == "GFlops"
+    assert supercomputer_feature["properties"]["capacity_band"] == "exascale"
+    assert supercomputer_feature["properties"]["operator"] == "ORNL"
+    assert supercomputer_feature["properties"]["location_precision"] == "precise"
+    assert supercomputer_feature["properties"]["is_estimated"] is False
+
+    gpu_feature = payload["features"][1]
+    assert gpu_feature["properties"]["site_type"] == "gpu_cluster"
+    assert gpu_feature["properties"]["vendor"] == "H100"
+    assert gpu_feature["properties"]["gpu_count"] == 100000
+    assert gpu_feature["properties"]["capacity_band"] == "large"
+    assert gpu_feature["properties"]["location_precision"] == "precise"
+
+
+def test_convert_compute_centers_to_geojson_uses_coordinate_hints():
+    hinted_record = _build_record(
+        record_id=3,
+        source="top500",
+        data_type="supercomputer",
+        name="Frontier",
+        country="United States",
+        city="",
+        latitude=0.0,
+        longitude=0.0,
+        metadata={
+            "organization": "Oak Ridge National Laboratory",
+            "rmax": 1102000.0,
+        },
+    )
+
+    payload = convert_compute_centers_to_geojson([hinted_record])
+
+    assert len(payload["features"]) == 1
+    coords = payload["features"][0]["geometry"]["coordinates"]
+    assert coords[0] == pytest.approx(-84.3107)
+    assert coords[1] == pytest.approx(35.9319)
+    assert payload["features"][0]["properties"]["is_estimated"] is True
+    assert payload["features"][0]["properties"]["location_precision"] == "estimated_site"
+
+
+def test_convert_compute_centers_to_geojson_falls_back_to_country_centroid():
+    centroid_record = _build_record(
+        record_id=4,
+        source="epoch_ai_gpu",
+        data_type="gpu_cluster",
+        name="Unknown Cluster",
+        country="United States",
+        city="",
+        latitude=0.0,
+        longitude=0.0,
+        metadata={
+            "organization": "Unknown Operator",
+            "value": "10000",
+            "unit": "TFlop/s",
+        },
+    )
+
+    payload = convert_compute_centers_to_geojson([centroid_record])
+
+    assert len(payload["features"]) == 1
+    props = payload["features"][0]["properties"]
+    coords = payload["features"][0]["geometry"]["coordinates"]
+    assert coords[0] == pytest.approx(-98.5795)
+    assert coords[1] == pytest.approx(39.8283)
+    assert props["is_estimated"] is True
+    assert props["location_precision"] == "estimated_country"
+    assert props["geography_mode"] == "country_centroid"
+
+
+@pytest.mark.asyncio
+async def test_compute_centers_geojson_endpoint_returns_stats():
+    records = [
+        _build_record(
+            record_id=1,
+            source="top500",
+            data_type="supercomputer",
+            name="Frontier",
+            country="United States",
+            city="Oak Ridge",
+            latitude=35.93,
+            longitude=-84.31,
+            metadata={"rank": 1, "rmax": 1102000.0},
+        ),
+        _build_record(
+            record_id=2,
+            source="epoch_ai_gpu",
+            data_type="gpu_cluster",
+            name="Colossus",
+            country="United States",
+            city="Memphis",
+            latitude=35.15,
+            longitude=-90.05,
+            metadata={"value": "20000", "unit": "TFlop/s"},
+        ),
+    ]
+
+    class _ScalarResult:
+        def __init__(self, rows):
+            self._rows = rows
+
+        def scalars(self):
+            class _Scalars:
+                def __init__(self, rows):
+                    self._rows = rows
+
+                def all(self):
+                    return self._rows
+
+            return _Scalars(self._rows)
+
+    class _FakeSession:
+        async def execute(self, _query):
+            return _ScalarResult(records)
+
+    async def override_get_db():
+        yield _FakeSession()
+
+    app.dependency_overrides[get_db] = override_get_db
+    transport = ASGITransport(app=app)
+    try:
+        async with AsyncClient(transport=transport, base_url="http://test") as client:
+            response = await client.get("/api/v1/visualization/geo/compute-centers")
+
+        assert response.status_code == 200
+        data = response.json()
+        assert data["count"] == 2
+        assert data["stats"]["supercomputers"] == 1
+        assert data["stats"]["gpu_clusters"] == 1
+        assert data["features"][0]["properties"]["data_type"] == "compute_center"
+    finally:
+        app.dependency_overrides.clear()
+
+
+@pytest.mark.asyncio
+async def test_visualization_geo_summary_returns_counts(monkeypatch):
+    records = [
+        _build_record(
+            record_id=1,
+            source="arcgis_cables",
+            data_type="submarine_cable",
+            name="Test Cable",
+            country="",
+            city="",
+            latitude=0,
+            longitude=0,
+            metadata={
+                "route_coordinates": [[[0, 0], [1, 1]]],
+                "status": "active",
+            },
+        ),
+        _build_record(
+            record_id=2,
+            source="arcgis_landing_points",
+            data_type="landing_point",
+            name="Test Landing",
+            country="United States",
+            city="New York",
+            latitude=40.7,
+            longitude=-74.0,
+            metadata={"city_id": 10},
+        ),
+        _build_record(
+            record_id=3,
+            source="celestrak_tle",
+            data_type="satellite_tle",
+            name="TESTSAT",
+            country="",
+            city="",
+            latitude=0,
+            longitude=0,
+            metadata={
+                "norad_cat_id": 12345,
+                "tle_line1": "1 12345U 98067A   24001.00000000  .00000000  00000-0  00000-0 0  9991",
+                "tle_line2": "2 12345  51.6000 100.0000 0001000  10.0000  20.0000 15.50000000    01",
+            },
+        ),
+        _build_record(
+            record_id=4,
+            source="top500",
+            data_type="supercomputer",
+            name="Frontier",
+            country="United States",
+            city="Oak Ridge",
+            latitude=35.93,
+            longitude=-84.31,
+            metadata={"rank": 1, "rmax": 1102000.0},
+        ),
+        _build_record(
+            record_id=5,
+            source="epoch_ai_gpu",
+            data_type="gpu_cluster",
+            name="Colossus",
+            country="United States",
+            city="Memphis",
+            latitude=35.15,
+            longitude=-90.05,
+            metadata={"value": "20000", "unit": "TFlop/s"},
+        ),
+    ]
+
+    class _ScalarResult:
+        def __init__(self, rows=None, scalar_value=None):
+            self._rows = rows or []
+            self._scalar_value = scalar_value
+
+        def scalar(self):
+            return self._scalar_value
+
+        def scalars(self):
+            class _Scalars:
+                def __init__(self, rows):
+                    self._rows = rows
+
+                def all(self):
+                    return self._rows
+
+            return _Scalars(self._rows)
+
+    class _FakeSession:
+        async def execute(self, query):
+            query_text = str(query)
+            if "bgp_incidents" in query_text:
+                return _ScalarResult(scalar_value=2)
+            if "bgp_anomalies" in query_text:
+                return _ScalarResult(scalar_value=3)
+            return _ScalarResult(rows=records)
+
+    async def override_get_db():
+        yield _FakeSession()
+
+    async def _fake_build_bgp_collector_coverage(*_args, **_kwargs):
+        return [
+            {"collector": "rrc00"},
+            {"collector": "rrc01"},
+        ]
+
+    monkeypatch.setattr(
+        "app.api.v1.visualization.build_bgp_collector_coverage",
+        _fake_build_bgp_collector_coverage,
+    )
+
+    app.dependency_overrides[get_db] = override_get_db
+    transport = ASGITransport(app=app)
+    try:
+        async with AsyncClient(transport=transport, base_url="http://test") as client:
+            response = await client.get("/api/v1/visualization/geo/summary")
+
+        assert response.status_code == 200
+        stats = response.json()["stats"]
+        assert stats["cable_count"] == 1
+        assert stats["landing_point_count"] == 1
+        assert stats["satellite_count"] == 1
+        assert stats["compute_center_count"] == 2
+        assert stats["supercomputer_count"] == 1
+        assert stats["gpu_cluster_count"] == 1
+        assert stats["bgp_event_count"] == 2
+        assert stats["bgp_incident_count"] == 2
+        assert stats["bgp_anomaly_count"] == 3
+        assert stats["bgp_collector_count"] == 2
+    finally:
+        app.dependency_overrides.clear()

docker-compose.local-model.yml

@@ -18,6 +18,9 @@ services:
     build:
       context: .
       dockerfile: aiprovider/Dockerfile
+      args:
+        PYTHON_IMAGE: ${PYTHON_IMAGE:-python:3.14-slim}
+        UV_IMAGE: ${UV_IMAGE:-ghcr.io/astral-sh/uv:latest}
     container_name: planet_aiprovider
     ports:
       - "8010:8010"

docker-compose.simple.yml

@@ -5,6 +5,9 @@ services:
     build:
       context: .
       dockerfile: aiprovider/Dockerfile
+      args:
+        PYTHON_IMAGE: ${PYTHON_IMAGE:-python:3.14-slim}
+        UV_IMAGE: ${UV_IMAGE:-ghcr.io/astral-sh/uv:latest}
     container_name: planet_aiprovider
     ports:
       - "8010:8010"

docker-compose.yml

@@ -5,6 +5,9 @@ services:
     build:
       context: .
       dockerfile: aiprovider/Dockerfile
+      args:
+        PYTHON_IMAGE: ${PYTHON_IMAGE:-python:3.14-slim}
+        UV_IMAGE: ${UV_IMAGE:-ghcr.io/astral-sh/uv:latest}
     env_file:
       - ./aiprovider/.env
     container_name: planet_aiprovider

docs/CHANGELOG.md

@@ -8,6 +8,518 @@ This project follows the repository versioning rule:
 - `improvement` -> `+0.0.1`（bugfix + 小功能混合）
 - `bugfix` -> `+0.0.1`
 
+## [0.44.0] — 2026-04-29
+
+### ✨ Highlights
+- 重构数据源与采集器设置边界：数据源页回归目录和采集触发，采集器 endpoint、请求头、凭证与连接验证统一进入设置页
+- BarentsWatch AIS 完整接入凭证解析、连接检查、默认教程、AI 生成教程和船只采集/可视化链路
+- Earth 新增船只图例、缩放反馈胶囊、缩放感知拖拽灵敏度，并将船只渲染性能优化方案沉淀到 plans
+- 仪表盘重启服务新增前端重启 action，并让 runner 通过 `~/.zshrc` 继承本地环境变量
+
+### 🔧 Improvements
+- 采集器连接状态改为基于成功采集或手动连接校验 checksum 判断，避免只依赖前端样式状态
+- 数据源页新增采集中任务标签和任务进度弹窗，内置与自定义数据源统一展示
+- Earth 国界线进一步贴近地表，并补充船只图层渲染顺序、样式和用户手册说明
+- docs skill 与 Claude/Codex 文档流程补齐技术文档和 plans 的职责边界
+
+---
+
+## [0.43.1] — 2026-04-28
+
+### 🐛 Fixes
+- 修正 `planet.sh` 在全量 restart 后启动 AI Provider 时的提示语义，避免把预期内未就绪描述成异常不健康
+
+---
+
+## [0.43.0] — 2026-04-28
+
+### ✨ Highlights
+- 新增 Earth 船舶追踪链路，接入 BarentsWatch AIS 凭证配置、采集器、后端 vessel 模型/API 与前端 Earth 船舶图层
+- 新增自定义数据源映射流程，支持样本抓取、目标 schema、AI 辅助生成映射、预览校验和映射执行
+- Settings 拆分 AI Provider 与采集器凭证配置，DataSources 只保留采集状态、运行参数和必要引导
+
+### 🔧 Improvements
+- AI Provider 支持运行时 LLM 配置、provider preset 下拉与刷新，并在 Playground 中引导到 AI 配置页
+- Markdown 渲染器补齐代码块复制按钮、语言标签、任务列表、图片、自动链接、删除线和文档主题样式
+- Docs 公开导航改为显式元数据白名单，避免开发任务文档自动出现在“其他”分组
+- 将 Codex/Claude cleanup、docs、goal-driven、release 流程补充 CLI-first 约束，并把 `rules.md` 整理成可按模块加载的工程规则
+
+---
+
+## [0.42.2] — 2026-04-28
+
+### 🐛 Fixes
+- Docs 中文模式下补齐左侧分组、文档标题、页头分类与搜索结果分类翻译，并更新文档站品牌标题/副标题文案
+
+---
+
+## [0.42.1] — 2026-04-28
+
+### 🐛 Fixes
+- 修正 release skill 的 feature 版本计算规则：minor 进位时 patch 必须重置为 `0`，例如 `0.41.2` 应发布为 `0.42.0`
+
+---
+
+## [0.42.0] — 2026-04-28
+
+### ✨ Highlights
+- 新增公开 `/docs` 文档站，支持中英文技术文档、使用手册、Quickstart、搜索、目录锚点与浅色/深色/跟随系统主题
+- Earth 在无高清材质时新增轻量 Fresnel 边缘提示，并调整卫星覆盖默认显示与地表材质可读性
+
+### 🔧 Improvements
+- 将技术文档整理为 `docs/technical/zh` 与 `docs/technical/en`，并补充控制台、`planet.sh`、Earth 与公共组件使用说明
+- 新增 `SegmentedControl` 公共滑块组件，支持缩放参数，复用到 docs 语言与主题切换
+- Markdown 渲染器接入自定义滚动条，表格与代码块在深色模式和 overflow 场景下保持可读
+- Docs 搜索结果支持内部滚动、点击外部关闭、重新聚焦恢复上次搜索结果
+- Earth 工具栏展开状态与设置持久化版本迁移继续收口，改善默认面板和快捷关闭行为
+
+---
+
+## [0.41.2] — 2026-04-27
+
+### 🔧 Improvements
+- `planet.sh` 启动链路新增 verbose 滚动输出窗口，并在后端端口占用时打印目标地址和监听进程诊断
+- Docker 构建支持通过 build args 覆盖 Python 与 uv 镜像，方便 Docker Hub 不稳定时切换镜像源
+
+### 🐛 Fixes
+- Earth 海缆登陆点改为基于相机射线与地球遮挡判断可见性，修复旋转后 pin 可见性滞后一帧的问题
+
+### 🔧 Improvements
+- `docker-compose*.yml` 为 AI Provider 构建传入 `PYTHON_IMAGE` / `UV_IMAGE` 参数，默认仍使用官方镜像
+- 后端启动失败遇到 `Address already in use` 时输出 `lsof`、`ss` 与 PID 命令行信息
+- verbose 模式下 AI Provider build、后端与前端启动日志会在 spinner 下方保留最新 5 行滚动展示
+
+---
+
+## [0.41.1] — 2026-04-27
+
+### 🐛 Fixes
+- 修复新闻直播面板设置项持久化失效：`closeTransientMobileOverlays` 通过旁路路径隐藏面板导致下次 persist 快照到错误状态，改为不重新从 DOM 读取面板可见性
+- 修复登陆点 pin 在地球侧面被半截遮挡：改为在接近地平线前（dot < 0.05）主动隐藏，避免深度测试切片
+
+### 🔧 Improvements
+- 将所有画布绘制的图标抽取为 SVG，存入 `frontend/public/earth/assets/icons/`，新增图标规范到 `rules.md`
+
+---
+
+## [0.41.0] — 2026-04-27
+
+### ✨ Highlights
+- Earth 图层系统完成地表到天空的注册顺序与关注优先的面板顺序拆分，支持基座海陆色块、国界、高清材质、云图、地形、算力、BGP、卫星、轨迹与海缆的稳定层级
+- 国界层新增真实行政区轮廓交互与中国/台湾联动高亮，修复高清材质、地形、footprint、卫星与经纬线之间的遮挡和 hover 竞争
+
+### 🔧 Improvements
+- 新增无轮廓基座地图，所有图层关闭时仍保留 `#010609` 海洋与 `#080f1b` 陆地色块
+- 将大气云图抽象为独立图层并接入桌面/移动端图层开关、持久化状态与启动同步
+- 高清材质改为独立纹理覆盖层，地形显示在高清材质上方，并在高清材质关闭/恢复时保持原地形开关意图
+- 补充 Earth 渲染层级与图层样式文档，记录正式图层名、变量名、材质颜色、线宽与 renderOrder
+
+---
+
+## [0.40.5] — 2026-04-26
+
+### 🔧 Improvements
+- 卫星拖尾改用 Instanced screen-space ribbon，单 draw call 渲染所有轨迹段，支持像素级宽度控制
+- Iridium 地面覆盖重写为球面投影径向网格，修复填充光晕不可见问题；新增外圈 LineLoop
+- 搜索面板打开时改用双 rAF 延迟聚焦输入框，确保 CSS 过渡完成后焦点可靠触发
+- 代码清理：提取 `IRIDIUM_OVERLAY_COLOR`、`IRIDIUM_REFERENCE_ALTITUDE_KM` 常量，消除重复三角函数调用
+
+---
+
+## [0.39.0] — 2026-04-24
+
+## [0.40.4] — 2026-04-26
+
+### 🔧 Improvements
+- 新增页面可见性恢复处理，页面从后台切回前台时主动刷新卫星位置，避免累积后台时间在下一帧一次性回放
+- 抽出卫星轨迹状态与轨迹几何清理 helper，统一后台恢复与清空数据时的轨迹重置路径
+
+### 🐛 Fixes
+- 修复页面在后台停留较久后恢复前台时，卫星轨迹因超大 `deltaTime` 突然跳变、拖尾异常拉长的问题
+- 修复后台恢复后首帧仍沿用旧轨迹缓存，导致轨迹与当前卫星位置短时错位的问题
+
+---
+
+## [0.40.3] — 2026-04-25
+
+### 🔧 Improvements
+- 卫星点云升级为自定义 ShaderMaterial，支持 per-point alpha 控制，锁定/悬停卫星从点云中精确隐藏
+- 修复锁定环与自发光选中标记的 depthTest 错误（false → true），消除远端渲染穿透 artifact
+- 新增锁定环悬停态缩放与线宽（LOCKED_RING_HOVER_SCALE / LOCKED_RING_HOVER_LINE_WIDTH）
+- 修复 updateLockedDotWorldTransform / updateLockedHaloWorldTransform 未强制刷新 matrixWorld 导致的位置漂移
+
+---
+
+## [0.40.2] — 2026-04-24
+
+### 🔧 Improvements
+- 卫星点大小随镜头缩放动态调整，拉近变大、拉远变小，响应与相机距离线性对应
+- 调小卫星点默认基础尺寸（dotSize 2.8），缩放范围更合理
+
+---
+
+## [0.40.1] — 2026-04-24
+
+### 🔧 Improvements
+- 卫星选中标记（lockedring / lockeddot / 光晕）颜色统一跟随图例轨道倾角分类配色
+- 修复 Starlink footprint 在特定视角下遮蔽卫星点的渲染顺序问题（Group renderOrder 影响子 Mesh 排序）
+- footprint 材质改为 `depthTest: false` + 相机朝向 limbFade，替代 polygonOffset 深度竞争方案
+- 修复选中海缆时误触发附近卫星高亮（该行为属于 BGP 事件点逻辑，不应用于海缆）
+
+---
+
+## [0.40.0] — 2026-04-24
+
+### ✨ Highlights
+- Earth 卫星 footprint 正式按星座能力分层：Starlink 保留专用地表覆盖，Iridium 改为独立外圈覆盖表达，其它非 Starlink 星座不再误用同一套 footprint
+- Earth 卫星详情卡补齐覆盖能力与当前显示说明，用户现在可以直接看见每颗卫星为什么显示 footprint、为何回退为自身发光
+
+### 🔧 Improvements
+- 后端可视化接口新增并透传 `constellation_group` 与 `footprint_policy`，前端据此执行 capability-gated footprint renderer
+- 新增 Iridium 独立 coverage ring adapter，并继续保留 Starlink 专用 footprint 调校与昼夜可读性增强
+- 新增 Earth 卫星 footprint 策略技术文档，明确 GNSS、generic LEO、GEO 与 Iridium 的显示边界
+
+### 🐛 Fixes
+- 修复前后端对 Iridium footprint policy 命名不一致，导致策略分发语义含混的问题
+- 清理 Starlink footprint 渲染中的未使用常量与过时命名，减少后续继续调校时的歧义
+
+---
+
+## [0.39.0] — 2026-04-24
+
+### ✨ Highlights
+- 后端正式落下统一结构化日志地基：请求上下文、事件名、脱敏与持久化链路开始收口为可扩展的企业级日志体系
+- 系统日志页重构为真正的日志工作台：顶部筛选更紧凑，终端日志区成为主视觉，移动端 Earth 新闻/态势细节交互继续补稳
+
+### 🔧 Improvements
+- 新增 `backend/app/core/logging.py`，统一 `request_id`、`service`、`event` 注入与敏感字段脱敏，并接入后端主入口、调度器、缓存、数据库和可视化链路
+- 系统日志页筛选区重排为更紧凑的两层结构，信息摘要并入终端工具栏 tooltip，日志终端区留出更稳定的按钮避让空间
+- Earth 移动端态势抽屉补齐宽度约束与图例换行规则，新闻详情抽屉在巡航切换时可同步更新标题和摘要
+
+### 🐛 Fixes
+- 修复 `/tmp/planet_backend.log` 中混入空字节时，日志摘要条行数与实际可见日志不一致的问题
+- 修复移动端“态势”tab 在内容渲染后被图例文本撑宽、超出一屏的问题
+- 修复移动端新闻详情抽屉在巡航切换下一条新闻时标题更新但 summary 不同步的问题
+
+---
+
+## [0.38.0] — 2026-04-23
+
+### ✨ Highlights
+- Earth 新闻正式接入通用巡航层：新闻和 BGP 统一进入可配置巡航模块，桌面端与移动端都能在巡航聚焦时展示对应新闻卡片
+- 系统日志页升级为结构化过滤链路：按真实时间戳、结构化级别和字符串检索统一筛选，不再依赖前端或后端从日志文本里猜结果
+
+### 🔧 Improvements
+- 新闻巡航补齐业务适配层：按发生地与时间生成巡航目标，桌面端与移动端统一标题 + summary 卡片风格，并增加连线与打字机摘要展示
+- 日志页筛选体验重排，统一服务源、级别、行数、时间和检索布局，日历标记改为由后端返回的结构化每日聚合结果驱动
+- 后端补充 `system_logs` 结构化解析与多级别精确过滤能力，Earth 浏览器端日志缓冲与系统日志 API 现在走同一套筛选语义
+
+### 🐛 Fixes
+- 修复新闻巡航模块开启后难以关闭、桌面/移动端设置状态互相污染的问题
+- 修复新闻巡航卡片缺少摘要、移动端详情样式不统一、新闻巡航缺少连线的问题
+- 修复日志级别筛选会被访问日志 query string 中的 `level=error` 等参数污染，从而把 `INFO` 行误判为 `ERROR` 的问题
+
+---
+
+## [0.37.2] — 2026-04-23
+
+### ✨ Highlights
+- Earth 图层系统新增经纬线开关，桌面图层面板与移动端抽屉都可直接控制
+
+### 🔧 Improvements
+- 经纬线正式接入 Earth layer registry，复用现有图层切换、移动端图层卡片与设置持久化流
+
+### 🐛 Fixes
+- 修复经纬线只能默认常驻、无法作为独立图层开关控制的问题
+
+---
+
+## [0.37.1] — 2026-04-23
+
+### ✨ Highlights
+- `planet.sh` 后端重启链路修复 `uvicorn --reload` 残留 worker 场景，`restart` 现在能真正替换旧实例
+
+### 🔧 Improvements
+- 收口后端清理逻辑，统一按 `uvicorn` 进程、端口占用进程和进程组执行清理，减少 reload 场景漏杀分支
+
+### 🐛 Fixes
+- 修复部分机器执行 `./planet.sh restart --allow-lan` 后后端仍停留旧实例，导致 `/api/v1/visualization/geo/compute-centers` 返回 `404` 的问题
+
+---
+
+## [0.35.1] — 2026-04-22
+## [0.37.0] — 2026-04-23
+
+### ✨ Highlights
+- Earth 连线系统正式从巡航里解耦成通用 callout connector：桌面端和移动端统一支持对象级锚点、四边切换与临界区边缘滑动
+- BGP 巡航展示继续收口为稳定的“先定位卡片、再连真实锚点、再展示卡片”链路，移动端 popup 与桌面 info panel 的路线规则统一
+
+### 🔧 Improvements
+- connector 配置从 `CRUISE_CONFIG` 拆到独立 `CONNECTOR_CONFIG`，默认类名、动画名和实例命名也全部去 cruise 语义
+- 移动端 popup 增加更稳定的 dock/obstacle 处理，拖动卡片时连线起终点会持续按几何关系自适应刷新
+- Earth 多个图层与控制逻辑继续收口，补充算力中心/BGP 风格对齐、layer panel 与相关交互细节调整
+
+### 🐛 Fixes
+- 修复巡航模式下终点只像“视觉锚点”而不是真实绑定对象的问题，卡片拖动后终点现在会跟随
+- 修复移动端与桌面端多类连线路线异常：压线、反向、临界区折返、起点遮挡事件点等问题
+- 修复对象矩形临界区内连线仍强制中点到中点导致路线像“先钻进 source 内部”再出去的问题
+
+---
+
+## [0.35.1] — 2026-04-22
+## [0.36.0] — 2026-04-22
+
+### ✨ Highlights
+- Earth 新增统一“算力中心”图层：接入超算与 GPU 集群，支持搜索、统计、图例、详情卡与独立图层开关
+- 算力中心支持精确位置与估算位置两种状态，估算点会以问号角标区分，避免数据不全时整批节点在地图上消失
+
+### 🔧 Improvements
+- Earth 详情卡拖拽与地球拖拽交互继续收口，减少拖动卡片和旋转地球时的选中文本与 pointer 竞争
+- `planet.sh` 改为通过独立脚本计算 AI Provider 依赖指纹，降低与根仓库依赖版本文件的无关耦合
+- README 补充 WSL / Windows 局域网访问排查与转发配置说明，便于开发环境联调
+
+### 🐛 Fixes
+- 修复 Earth 算力中心图层在无原始坐标时无法显示的问题，支持站点提示和国家级估算回退
+- 修复信息卡拖拽事件可能被卡片级 stopPropagation 吞掉，导致拖拽流中断的问题
+
+---
+
+## [0.35.1] — 2026-04-22
+
+### ✨ Highlights
+- Earth 统计展示改为统一 `data-earth-stat` 绑定机制，桌面 HUD 和移动端抽屉复用同一套状态更新入口
+
+### 🔧 Improvements
+- 收口海缆、登陆点、卫星、BGP 事件与 BGP 状态的统计写入逻辑，减少后续继续补桌面/移动双写分支的成本
+
+### 🐛 Fixes
+- 修复移动端态势抽屉中的海缆、登陆点与 BGP 统计在图层切换后可能停留旧值的问题
+
+---
+
+## [0.35.0] — 2026-04-22
+
+### ✨ Highlights
+- Earth 移动端底部抽屉系统全面上线：响应式布局自动切换、Tab 导航、手势上拉/下滑开合、惯性速度判定
+- 移动端点击可交互物件（海缆、登陆点、卫星、BGP）后弹出智能定位悬浮卡片，可拖动，点击跳转详情
+
+### 🔧 Improvements
+- 抽屉把手区域缩小至 36px（collapsed 时仅露出把手，不遮挡地球操作区）
+- 抽屉定期弹跳动画提示用户可上拉，5 秒间隔，打开后自动停止
+- 通知胶囊位置调整，不再覆盖品牌 logo
+- 移动端单指旋转、双指捏合缩放地球，触控事件冲突修复（pointer-events 级联）
+
+### 🐛 Fixes
+- 修复移动端抽屉 shell 因 layout 高度（240px+）遮挡地球触控区域，pointer-events 改为按层级精确控制
+- 修复悬浮卡片因 setPointerCapture 在 iOS Safari 抑制合成 click 事件导致无法点击的问题
+
+---
+
+## [0.34.0] — 2026-04-22
+
+### ✨ Highlights
+- Earth 搜索面板正式接入，支持搜索海缆、登陆点、卫星、BGP 事件与观测站，并可直接聚焦到对应对象
+- `planet.sh --allow-lan` 打通 Bun + Vite 的局域网开放链路，启动成功后自动打印推荐访问地址与后端健康检查地址
+
+### 🔧 Improvements
+- 前端开发启动链统一改成 Bun 直接执行 Vite 入口，不再依赖 shell 中额外暴露的 Node 路径
+- Earth 搜索结果接入登陆点详情卡片与对象聚焦，搜索后可直接进入对应详情流
+- `planet.sh` 补充局域网 IPv4 自动识别与推荐地址输出，减少 WSL 局域网调试成本
+
+### 🐛 Fixes
+- 修复 `./planet.sh restart --allow-lan` 全量重启时未把 `--allow-lan` 继续传给 `start()`，导致前端退回本机监听的问题
+- 修复 WSL + Bun 环境下前端偶发因 Vite 启动链不稳定而无法正确监听 `0.0.0.0:3000` 的问题
+
+---
+
+## [0.33.0] — 2026-04-22
+
+### ✨ Highlights
+- `news_live_streams` 采集器默认接入 `iptv-org` 频道目录，并将采集结果稳定并入 Earth TV 直播源列表
+- 数据源页支持直接编辑内置数据源 override，并为内置源提供一键恢复默认配置入口
+
+### 🔧 Improvements
+- `News Live Streams` 现在作为可直接触发的内置默认数据源提供，无需先手工补 override 才能采集
+- TV 播放源菜单会直接区分 `[内置]` 和 `[采集]` 来源，频道来源信息也会同步展示
+- 新增 [earth-news-source-configuration-and-collector-plan.md](/home/ray/dev/linkong/planet/docs/plans/earth-news-source-configuration-and-collector-plan.md)，正式规划 Earth 态势新闻源配置化与后续采集器化路线
+
+### 🐛 Fixes
+- 修复 `news_live_streams` 采集完成后 `/api/v1/tv/streams` 因读取不存在的 `updated_at` 字段而导致默认频道全部消失的问题
+- 修复内置数据源操作列按钮显示不全，以及编辑抽屉中多个 `Collapse` 紧贴的问题
+
+---
+
+## [0.32.0] — 2026-04-22
+
+### ✨ Highlights
+- Earth 设置新增“地球默认大小”持久化项，重置视角、缩放百分比重置和 BGP 巡航视图现在统一复用这一份默认 zoom
+- 卫星焦点层次继续收口：巡航进入 presentation 前不再过早 dim，非焦点卫星改成“降亮度/尾迹/背板”而不是去饱和度
+
+### 🔧 Improvements
+- Earth 设置面板区块和左右留白进一步收紧，整体更贴近 HUD 面板的密度
+- toolbar 展开边界缓存改为按需刷新，减少 document 级 mousemove 期间的重复布局读取
+- Scrollbar 和 ScrollbarOverlay 收窄 observer 范围，减少大表格和动态菜单下的额外刷新成本
+- 更新 [earth-frontend-context.md](/home/ray/dev/linkong/planet/docs/technical/earth-frontend-context.md)，补充默认视图大小已进入 Earth 设置持久化真源
+
+### 🐛 Fixes
+- 修复开启巡航后，尚未进入连线/presentation 时卫星已经整体变暗的问题
+- 修复默认大小重置链路分散在多个入口、实际 reset/cruise/缩放提示不一致的问题
+- 修复开启地形后卫星反馈层与地球背面可见性之间的一组表现问题，保留正面反馈同时恢复背面轨道遮挡
+
+---
+
+## [0.31.3] — 2026-04-22
+
+### ✨ Highlights
+- Earth 图层注册表和启动任务框架继续收口，启动顺序、启动模式、启动提示和任务注册现在都能从统一入口扩展
+- 修复 Earth 普通旋转模式与巡航模式切换时的一组交互回归，同时让卫星/地形/昼夜模式的表现更稳定
+
+### 🔧 Improvements
+- 新增 [layer-startup-tasks.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/layer-startup-tasks.js) 启动任务注册表，支持 `registerLayerStartupTask(id, taskFactory)`，并拆成海缆 / 卫星 / BGP 独立注册函数
+- Earth 图层控制改成注册表驱动，统一承载 `startupPriority`、`startupMode`、`startupLabel`、`startupMessage` 与图层持久化元信息
+- Earth 设置支持持久化图层开关、旋转模式、HUD 面板显示状态、地形透明度与日夜模式，并提供一键重置
+- 更新 [earth-frontend-context.md](/home/ray/dev/linkong/planet/docs/technical/earth-frontend-context.md) 记录图层注册表、启动任务、设置持久化与巡航适配边界
+
+### 🐛 Fixes
+- 修复普通旋转模式下点击海缆 / 卫星 / BGP 后卡片和选中表现会被异常清空的问题
+- 修复巡航模式切回旋转再切回巡航后无法继续自动巡航的问题
+- 修复开启地形后卫星选中反馈层被高海拔区域吞掉的问题，并恢复轨道只在地球前半侧可见
+- 修复关闭日夜模式后地球照明仍沿真实昼夜切换、亮部过曝和偏色的问题，改成更中性的 inspection lighting
+- 修复 toolbar 收起态仍挡住地球交互，以及首帧短暂展开闪现的问题
+
+---
+
+## [0.31.2] — 2026-04-21
+
+### ✨ Highlights
+- Earth 巡航模式重构为“通用巡航队列 + 通用连线动画 + BGP 业务适配”三层结构，后续扩到海缆、卫星或新闻巡航时不必再复制一套 `main.js` 状态机
+- 修复巡航重构后的交互回归：空白点击重新稳定切到下一项，连线按“起点 → 引导线 → 终点”顺序入场
+
+### 🔧 Improvements
+- 新增 [cruise-sequencer.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/cruise-sequencer.js) 统一管理队列推进、停留时长、打断与恢复
+- 新增 [callout-connector.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/callout-connector.js) 统一管理 SVG 连线、折线路径与描边动画
+- 新增 [bgp-cruise-adapter.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/bgp-cruise-adapter.js) 收口 BGP 巡航目标排序、卡片落点、轮询去重与连线适配
+- 更新 [earth-frontend-context.md](/home/ray/dev/linkong/planet/docs/technical/earth-frontend-context.md) 说明新的巡航分层与复用边界
+
+### 🐛 Fixes
+- 修复巡航模式下点击空白处无法稳定跳转到下一项、切回旋转再切回巡航后直接卡住的问题
+- 修复巡航连线被实时重定位覆盖导致“直接出现”而非绘制动画的问题
+- 修复连线动画节点入场节奏不对的问题，改为先出现起点，再绘制连线，最后出现终点
+
+---
+
+## [0.31.1] — 2026-04-21
+
+### ✨ Highlights
+- Earth 图层开关状态统一成可复用的 `active / loading` 状态机，首次启用地形和卫星时不再像按钮失效
+- 文档目录重构为 `docs/technical`、`docs/plans`、`docs/deprecated`，并吸收 `.sisyphus/plans` 中有价值的 Earth / 卫星 / UE5 草案
+
+### 🔧 Improvements
+- 新增 [layer-button-state.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/layer-button-state.js)，统一按钮 tooltip、`aria-busy`、禁用态和状态文本同步
+- 地形图层支持 hover/focus 预热与空闲预热，首次点击等待前移，加载中状态持续可见
+- 卫星图层启用前会立即切换为 `loading` 中间态，请求完成后再切回正常开关表现
+
+### 🐛 Fixes
+- 修复地形首次加载时通知过早消失、开关仍像关闭状态导致用户误判按钮损坏的问题
+- 修复卫星接口较慢时按钮没有任何中间态反馈的问题
+
+---
+
+## [0.31.0] — 2026-04-21
+
+### ✨ Features
+- Earth 新增"巡航展示"模式：自动轮播 BGP 异常事件，逐帧追踪连接线位置，支持外部交互立即中断序列（cancel notifier 模式）
+- 巡航目标事件点高亮显示：hover 外观 + 锁定脉冲动画，并与点击行为统一展示周边受影响卫星与海缆
+- BGP 事件图标新增填充 W 形波动符号（flap 类型），替换原有难以辨认的贝塞尔细线
+- 巡航/点击激活时其余卫星自动降饱和度 + 增加透明度以突出焦点；海缆未受影响时同步变暗
+
+### 🔧 Improvements
+- 修复巡航轮播期间 BGP 事件 polling 刷新导致标记闪烁消失的问题（clearBGPData 延迟到请求完成后执行）
+- 点击与巡航锁定颜色统一为 hover 色（0.92, 0.98, 1.0 全透明），移除锁定态脉冲动画
+- 巡航连接折线转折点从尖角调整为钝角（linkElbowDropPx），提升连线可读性
+
+---
+
+## [0.30.0] — 2026-04-21
+
+### ✨ Features
+- Earth 新增真实地形图层：后端代理 Terrarium DEM 瓦片（`/api/v1/visualization/terrain/terrarium/{z}/{x}/{y}.png`），前端新增 `terrain.js` 负责瓦片拉取、顶点位移与按海拔着色
+- 设置弹窗新增"地形"分组，支持通过滑块实时调整地形图层透明度
+
+### 🔧 Improvements
+- 地形按钮改为异步加载，首次点击显示进度提示并在失败时自动回退
+- 启动阶段改用 `applyImmediateView` 直接应用初始视角，`showStatusMessage` / `queueStatusMessage` 区分即时与队列态状态消息，加载中不再被临时状态打断
+- 控制面板抽取 `applyTerrainUiState` / `getViewRotation` 收敛地形切换与视角旋转的重复 UI 同步逻辑
+
+---
+
+## [0.29.2] — 2026-04-21
+
+### ✨ Highlights
+- Earth 继续收口 HUD 交互与设置面板表现，设置弹窗改成更接近从按钮展开的窗口感，同时加入系统级 admin 入口
+- 修正天球太阳方向与地球受光解耦后的日照逻辑，地表昼夜判断改为按太阳直射点经纬度落到地球贴图坐标
+
+### 🔧 Improvements
+- toolbar 进一步收成更贴近 hub 的浅弓形排列，并统一成与 HUD panel 一致的液态玻璃配色与透明度
+- 设置弹窗与各 HUD panel 继续统一样式、等比缩放和头部基线，设置列表补充系统分组与 admin 跳转
+- 所有 HUD panel 增加更统一的液态玻璃高光与 hover / press 反馈
+
+### 🐛 Fixes
+- 修复设置弹窗仍像旧圆角矩形、标题文案重复和从底边直直飞出的动画问题
+- 修复天球与太阳方向混用显示校准导致中国白天仍落在夜面的日照错误
+
+---
+
+## [0.29.1] — 2026-04-20
+
+### ✨ Highlights
+- Earth 加载状态条改成单一队列式通知面板，加载阶段不再因为步骤文案变化而回缩，也不会被其他通知打断
+- 调整 brand panel 的呈现方式与昼夜/选中态可读性，让品牌区更自然、交互高亮在白天和黑夜里都更稳定
+
+### 🔧 Improvements
+- 移除旧的地球加载浮层结构，统一由 HUD 状态消息承载三点脉冲加载过程
+- brand panel 改为无边框品牌层，仅保留轻微氛围光，不再因为非常规尺寸显得像第五块功能面板
+- 温和收敛地球昼夜材质与主背光强度，保留昼夜辨识度的同时提升白天地表纹理和夜面交互可见性
+
+### 🐛 Fixes
+- 修复加载地球时通知条在步骤切换中反复缩短、其他状态消息抢占加载流程的问题
+- 修复海缆、登陆点和 BGP 选中高亮在黑夜中过暗、在高光中过亮导致难以辨识的问题
+
+---
+
+## [0.29.0] — 2026-04-20
+
+### ✨ Highlights
+- Earth 新增天球层第一版：引入真实全天星图、亮星层与太阳/月亮位置计算，地球场景首次具备可校准的天文背景
+- 地球昼夜分隔升级为更明显的日夜增强效果，夜面、晨昏带和太阳方向联动更容易直接读出来
+
+### 🔧 Improvements
+- 新增 `celestial.js` 模块和 `assets/celestial/` 资源目录，统一管理星图、亮星数据以及太阳/月亮与光照同步
+- 卫星图例改为按倾角分组，严格固定为“赤道轨道 → 低倾角轨道 → 中倾角轨道 → 高倾角轨道 → 逆行轨道”顺序，并全部中文化
+- 图层面板补齐关闭按钮，拖拽脱离左列后不再被流布局 margin 影响，能够真正贴到品牌面板下沿
+
+### 🐛 Fixes
+- 修复天球球壳放大后被相机 far plane 裁剪导致的外层黑环问题
+- 修复图层面板在左侧上移时始终与 brand panel 保持额外间距的问题
+
+---
+
+## [0.28.2] — 2026-04-20
+
+### ✨ Highlights
+- 修正媒体情报面板在 `电视直播 / 态势聚合` tab 间切换时的尺寸记忆逻辑，切回原 tab 后可恢复各自大小状态
+- 清理 `docs/` 根目录遗留的旧路径文档，只保留新的分组目录和归档目录，结束同一文档双路径并存状态
+
+### 🔧 Improvements
+- `media-panel` 切换逻辑改成按 tab 分别记忆尺寸状态，避免 `A -> B -> A` 时继续共用同一套外层尺寸
+- 目录整理真正完成收尾：旧的 `docs/*.md` 平铺计划文档删除，继续以 `docs/agents / earth / backend / frontend / ops / ue5 / deprecated` 为唯一入口
+
+### 🐛 Fixes
+- 修复拉伸 `media-panel` 后切换 tab 时，`news-panel` 高度回退到旧默认值的问题
+- 修复拉伸后切换 tab 导致面板视觉锚点异常的问题，切换时改为围绕当前卡片自身右下角进行尺寸恢复
+
+---
+
 ## [0.28.1] — 2026-04-20
 
 ### ✨ Highlights
@@ -213,7 +725,7 @@ Released: 2026-04-12
 
 - Added [backend/app/api/v1/tv.py](/home/ray/dev/linkong/planet/backend/app/api/v1/tv.py), [backend/app/services/tv_streams.py](/home/ray/dev/linkong/planet/backend/app/services/tv_streams.py), and [backend/app/services/collectors/news_live_streams.py](/home/ray/dev/linkong/planet/backend/app/services/collectors/news_live_streams.py) to provide TV source configuration, public stream payloads, a guarded HLS proxy path, and a collector entry point for future world-news live-source ingestion.
 - Added the Earth TV HUD workspace through [frontend/public/earth/index.html](/home/ray/dev/linkong/planet/frontend/public/earth/index.html), [frontend/public/earth/js/tv.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/tv.js), and [frontend/public/earth/css/tv-panel.css](/home/ray/dev/linkong/planet/frontend/public/earth/css/tv-panel.css), including toolbar access, draggable/closable behavior, resize support, direct video/HLS playback, iframe fallback, and per-channel external-open handling.
-- Added [docs/deprecated/earth-tv-live-module-plan.md](/home/ray/dev/linkong/planet/docs/deprecated/earth-tv-live-module-plan.md) and [docs/earth/news-live-streams-collector-format.md](/home/ray/dev/linkong/planet/docs/earth/news-live-streams-collector-format.md) to document the TV module rollout plan and the expected collector payload format for future curated live-channel ingestion.
+- Added [docs/deprecated/earth-tv-live-module-plan.md](/home/ray/dev/linkong/planet/docs/deprecated/earth-tv-live-module-plan.md) and [docs/earth/technical/news-live-streams-collector-format.md](/home/ray/dev/linkong/planet/docs/technical/earth-news-live-streams-collector-format.md) to document the TV module rollout plan and the expected collector payload format for future curated live-channel ingestion.
 
 ### Improved
 
@@ -299,7 +811,7 @@ Released: 2026-04-10
 
 - Improved [frontend/src/pages/Playground/Playground.tsx](/home/ray/dev/linkong/planet/frontend/src/pages/Playground/Playground.tsx) and [frontend/src/index.css](/home/ray/dev/linkong/planet/frontend/src/index.css) by rebuilding Playground into a true chatbox workflow with persistent history, edit-and-resend behavior, grounded message actions, responsive composer behavior, bottom-stick scrolling, and tighter mobile layout handling.
 - Improved [frontend/src/components/AppLayout/AppLayout.tsx](/home/ray/dev/linkong/planet/frontend/src/components/AppLayout/AppLayout.tsx), [frontend/src/App.tsx](/home/ray/dev/linkong/planet/frontend/src/App.tsx), and [frontend/src/pages/Alerts/Alerts.tsx](/home/ray/dev/linkong/planet/frontend/src/pages/Alerts/Alerts.tsx) by reorganizing navigation around `采集与数据`, `专题观测`, and split alert entries so the app can scale to more observability and situational modules without turning the top-level UI into a single overloaded page.
-- Improved [README.md](/home/ray/dev/linkong/planet/README.md) and [docs/agents/situational-awareness-foundation-plan.md](/home/ray/dev/linkong/planet/docs/agents/situational-awareness-foundation-plan.md) by documenting the current AI/alerts base, planned situational-awareness direction, and the new persistent Playground foundation.
+- Improved [README.md](/home/ray/dev/linkong/planet/README.md) and [docs/agents/situational-awareness-foundation-plan.md](/home/ray/dev/linkong/planet/docs/plans/agents-situational-awareness-foundation-plan.md) by documenting the current AI/alerts base, planned situational-awareness direction, and the new persistent Playground foundation.
 
 ### Fixed
 
@@ -337,7 +849,7 @@ Released: 2026-04-10
 ### Improved
 
 - Improved [rules.md](/home/ray/dev/linkong/planet/rules.md) by adding mandatory release-workflow requirements and a new frontend layout constraint section covering single-screen workspaces, overflow ownership, tab-pane behavior, compact-mode expectations, and readable-card fallbacks.
-- Improved [frontend-layout-guidelines.md](/home/ray/dev/linkong/planet/docs/frontend/frontend-layout-guidelines.md) by summarizing the recurring Earth, Playground, BGP, and admin-layout regressions into concrete constraints for future frontend work, including “prefer scrollbars over unreadable compression” and “do not treat every tab as a table pane.”
+- Improved [frontend-layout-guidelines.md](/home/ray/dev/linkong/planet/docs/technical/frontend-layout-guidelines.md) by summarizing the recurring Earth, Playground, BGP, and admin-layout regressions into concrete constraints for future frontend work, including “prefer scrollbars over unreadable compression” and “do not treat every tab as a table pane.”
 
 ## 0.24.6
 
@@ -354,7 +866,7 @@ Released: 2026-04-10
 - Improved [backend/app/services/bgp_incidents.py](/home/ray/dev/linkong/planet/backend/app/services/bgp_incidents.py) and [backend/app/services/bgp_enrichment.py](/home/ray/dev/linkong/planet/backend/app/services/bgp_enrichment.py) by avoiding historical full-table infrastructure scans, narrowing observation baseline payloads to required columns, and pushing more ASN filtering into the database.
 - Improved [backend/app/api/v1/alerts.py](/home/ray/dev/linkong/planet/backend/app/api/v1/alerts.py), [backend/app/api/v1/dashboard.py](/home/ray/dev/linkong/planet/backend/app/api/v1/dashboard.py), and [backend/app/api/v1/settings.py](/home/ray/dev/linkong/planet/backend/app/api/v1/settings.py) by collapsing several repeated count and settings queries into fewer aggregate or batched reads.
 - Improved [frontend/src/pages/BGP/BGP.tsx](/home/ray/dev/linkong/planet/frontend/src/pages/BGP/BGP.tsx), [frontend/src/index.css](/home/ray/dev/linkong/planet/frontend/src/index.css), and [frontend/src/components/MarkdownRenderer/MarkdownRenderer.tsx](/home/ray/dev/linkong/planet/frontend/src/components/MarkdownRenderer/MarkdownRenderer.tsx) by rebuilding the `AI 简报` tab layout, fixing saved brief scrolling behavior, and extending the renderer to handle tables, separators, and stored metadata comments more gracefully.
-- Improved [docs/frontend/ai-playground-development-plan.md](/home/ray/dev/linkong/planet/docs/frontend/ai-playground-development-plan.md) by explicitly recording that the current BGP brief is only the first-stage summary flow and that regional prefix-geography analysis remains a planned Phase B follow-up.
+- Improved [docs/frontend/plans/ai-playground-development-plan.md](/home/ray/dev/linkong/planet/docs/plans/frontend-ai-playground-development-plan.md) by explicitly recording that the current BGP brief is only the first-stage summary flow and that regional prefix-geography analysis remains a planned Phase B follow-up.
 
 ### Fixed
 
@@ -460,8 +972,8 @@ Released: 2026-04-09
 ### Added
 
 - Added [frontend/src/pages/Playground/Playground.tsx](/home/ray/dev/linkong/planet/frontend/src/pages/Playground/Playground.tsx), introducing the first dedicated AI testing workspace with provider status visibility, prompt/result tabs, and collapsible operator guidance.
-- Added [docs/frontend/frontend-layout-guidelines.md](/home/ray/dev/linkong/planet/docs/frontend/frontend-layout-guidelines.md), documenting the repository standard for one-screen admin workspaces and module-local overflow handling.
-- Added [docs/frontend/ai-playground-development-plan.md](/home/ray/dev/linkong/planet/docs/frontend/ai-playground-development-plan.md), capturing the completed AI gateway/UI work and the next delivery phases for BGP briefs, evidence-first inputs, and future agent runtime expansion.
+- Added [docs/frontend/technical/frontend-layout-guidelines.md](/home/ray/dev/linkong/planet/docs/technical/frontend-layout-guidelines.md), documenting the repository standard for one-screen admin workspaces and module-local overflow handling.
+- Added [docs/frontend/plans/ai-playground-development-plan.md](/home/ray/dev/linkong/planet/docs/plans/frontend-ai-playground-development-plan.md), capturing the completed AI gateway/UI work and the next delivery phases for BGP briefs, evidence-first inputs, and future agent runtime expansion.
 
 ### Improved
 
@@ -566,7 +1078,7 @@ Released: 2026-04-07
 - Added [backend/app/services/ai_client.py](/home/ray/dev/linkong/planet/backend/app/services/ai_client.py), introducing an internal HTTP client for `backend -> aiprovider` calls with request-id propagation and lightweight retry.
 - Added [aiprovider/main.py](/home/ray/dev/linkong/planet/aiprovider/main.py), [aiprovider/provider_service.py](/home/ray/dev/linkong/planet/aiprovider/provider_service.py), and related config/schema files to stand up the dedicated adapter service.
 - Added [aiprovider/.env.example](/home/ray/dev/linkong/planet/aiprovider/.env.example) and [docker-compose.local-model.yml](/home/ray/dev/linkong/planet/docker-compose.local-model.yml) as ready-to-edit local-model templates.
-- Added [docs/agents/aiprovider.md](/home/ray/dev/linkong/planet/docs/agents/aiprovider.md), documenting architecture, configuration, single-machine and multi-machine deployment, and cross-service calling patterns.
+- Added [docs/agents/aiprovider.md](/home/ray/dev/linkong/planet/docs/technical/agents-aiprovider.md), documenting architecture, configuration, single-machine and multi-machine deployment, and cross-service calling patterns.
 - Added a dedicated `重启 AI Provider` control path in [Dashboard.tsx](/home/ray/dev/linkong/planet/frontend/src/pages/Dashboard/Dashboard.tsx), [system_control.py](/home/ray/dev/linkong/planet/backend/app/services/system_control.py), and [system_restart_runner.py](/home/ray/dev/linkong/planet/backend/scripts/system_restart_runner.py).
 
 ### Improved
@@ -692,7 +1204,7 @@ Released: 2026-04-02
 
 - Added a new `IPtoASN Prefix Geography` collector in [iptoasn.py](/home/ray/dev/linkong/planet/backend/app/services/collectors/iptoasn.py) and registered it through [data_sources.yaml](/home/ray/dev/linkong/planet/backend/app/core/data_sources.yaml), [data_sources.py](/home/ray/dev/linkong/planet/backend/app/core/data_sources.py), [datasource_defaults.py](/home/ray/dev/linkong/planet/backend/app/core/datasource_defaults.py), and [collectors/__init__.py](/home/ray/dev/linkong/planet/backend/app/services/collectors/__init__.py).
 - Added country centroid helpers in [countries.py](/home/ray/dev/linkong/planet/backend/app/core/countries.py) so country-level prefix geography can produce map coordinates instead of only labels.
-- Added a dedicated prefix-geography implementation note in [prefix-geography-plan.md](/home/ray/dev/linkong/planet/docs/earth/prefix-geography-plan.md).
+- Added a dedicated prefix-geography implementation note in [prefix-geography-plan.md](/home/ray/dev/linkong/planet/docs/plans/earth-prefix-geography-plan.md).
 - Added recent `15m` collector activity dimensions to BGP coverage output in [bgp_collectors.py](/home/ray/dev/linkong/planet/backend/app/services/bgp_collectors.py) and [visualization.py](/home/ray/dev/linkong/planet/backend/app/api/v1/visualization.py).
 - Added additional BGP detector coverage for `route_leak_candidate` and `path_flap` flows in [test_bgp.py](/home/ray/dev/linkong/planet/backend/tests/test_bgp.py).
 - Added a local Earth cloud texture at [earth_clouds_1024.png](/home/ray/dev/linkong/planet/frontend/public/earth/assets/earth_clouds_1024.png) to avoid remote cloud-map dependency failures.
@@ -707,7 +1219,7 @@ Released: 2026-04-02
 - Improved Earth event animation semantics in [bgp.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/bgp.js) by separating icon pulse from ring expansion so the center marker can breathe while the ring expands independently.
 - Improved Earth texture reliability in [earth.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/earth.js) by switching clouds back to a local static asset under the restored `public/earth` runtime.
 - Improved frontend boot noise in [frontend/index.html](/home/ray/dev/linkong/planet/frontend/index.html) by removing the default Vite favicon request that was generating irrelevant `vite.svg` timeouts during Earth debugging.
-- Improved project planning docs in [bgp-context.md](/home/ray/dev/linkong/planet/docs/earth/bgp-context.md) and [TODO.md](/home/ray/dev/linkong/planet/TODO.md) so the roadmap now explicitly prioritizes `activity layer`, `prefix-centric geography`, and follow-up geofeed/whois work.
+- Improved project planning docs in [bgp-context.md](/home/ray/dev/linkong/planet/docs/technical/earth-bgp-context.md) and [TODO.md](/home/ray/dev/linkong/planet/TODO.md) so the roadmap now explicitly prioritizes `activity layer`, `prefix-centric geography`, and follow-up geofeed/whois work.
 
 ### Fixed
 
@@ -865,7 +1377,7 @@ Released: 2026-03-31
 - Added restart-task Redis helpers and whitelist command mapping in [system_control.py](/home/ray/dev/linkong/planet/backend/app/services/system_control.py).
 - Added detached restart runner orchestration in [system_restart_runner.py](/home/ray/dev/linkong/planet/backend/scripts/system_restart_runner.py).
 - Added `-d` / `--database` support to [planet.sh](/home/ray/dev/linkong/planet/planet.sh) for database-only restarts.
-- Added restart control documentation in [system-service-control.md](/home/ray/dev/linkong/planet/docs/backend/system-service-control.md).
+- Added restart control documentation in [system-service-control.md](/home/ray/dev/linkong/planet/docs/technical/backend-system-service-control.md).
 
 ### Improved
 

docs/agents/agent-architecture-plan.md

@@ -1,647 +0,0 @@
-# Agent Architecture Plan
-
-## Overview
-
-This document defines the agent architecture for Planet.
-
-The architecture is intentionally broader than datasource health checking.
-
-It is designed to support both:
-
-- datasource health governance
-- future situational-awareness workflows
-
-The core idea is to avoid building a one-off "repair broken API links" agent.
-
-Instead, Planet should grow a reusable agent runtime that can:
-
-- collect evidence
-- evaluate signals
-- reason over incomplete information
-- generate proposals
-- produce assessments
-- execute limited actions under policy
-
-
-## Design Goal
-
-Build an agent foundation that can evolve in this order:
-
-1. datasource health checks
-2. datasource repair proposals
-3. signal correlation
-4. situational assessments
-5. controlled runtime actions
-
-This means the architecture should treat datasource health as one use case of the larger agent system, not as the whole system.
-
-
-## Core Principles
-
-1. Separate evidence from reasoning
-
-- raw signals should be gathered first
-- deterministic checks should run before LLM reasoning
-
-2. Agents do not own the defaults
-
-- repository defaults remain human-owned
-- agents operate on runtime state, proposals, and overrides
-
-3. Reasoning and action are different responsibilities
-
-- many agents should be read-only or propose-only
-- only tightly controlled flows may apply changes
-
-4. Shared runtime, specialized roles
-
-- multiple agent roles should share the same object model and orchestration patterns
-- health and situational-awareness agents should not invent incompatible payloads
-
-5. Auditability is mandatory
-
-- every proposal, assessment, and applied action should be attributable
-
-
-## System Layers
-
-Planet agent architecture should be split into four layers.
-
-### 1. Signal Layer
-
-Purpose:
-
-- gather raw evidence from internal and external systems
-
-Example sources:
-
-- collector outputs
-- datasource health checks
-- logs
-- snapshots
-- alerts
-- web search results
-- scraped pages
-- external APIs
-- operator inputs
-
-Responsibilities:
-
-- fetch
-- normalize
-- timestamp
-- tag with source and trust level
-
-This layer should not make high-level judgments.
-
-
-### 2. Evaluation Layer
-
-Purpose:
-
-- perform deterministic analysis
-
-Examples:
-
-- reachability checks
-- schema validation
-- threshold checks
-- time-window comparisons
-- anomaly counters
-- completeness checks
-
-Responsibilities:
-
-- classify signals into machine-readable findings
-- attach deterministic evidence
-
-This layer should avoid LLM dependency whenever possible.
-
-
-### 3. Reasoning Layer
-
-Purpose:
-
-- use LLMs when semantic interpretation or incomplete-information reasoning is needed
-
-Examples:
-
-- endpoint migration inference
-- multi-source event correlation
-- causality hypotheses
-- ambiguity reduction
-- assessment narrative generation
-- action recommendation generation
-
-Responsibilities:
-
-- synthesize evidence
-- produce hypotheses
-- rank confidence
-- explain reasoning boundaries
-
-This is the main place where `aiprovider` and web search are used.
-
-
-### 4. Action Layer
-
-Purpose:
-
-- convert proposals or assessments into controlled system actions
-
-Examples:
-
-- create runtime override
-- create proposal
-- publish alert
-- update operator task queue
-- generate summary artifact
-- trigger follow-up verification
-
-Responsibilities:
-
-- enforce policy
-- enforce approval requirements
-- verify post-action outcomes
-- record audit trails
-
-
-## Architecture Sketch
-
-```mermaid
-flowchart TD
-    A["Collectors / Logs / Snapshots / External APIs"] --> B["Signal Layer"]
-    W["Web Search / Page Fetch / Docs"] --> B
-    B --> C["Evaluation Layer"]
-    C --> D["Findings"]
-    D --> E["Reasoning Layer (LLM + Tools)"]
-    E --> F["Proposals"]
-    E --> G["Assessments"]
-    F --> H["Action Layer"]
-    H --> I["Runtime Overrides / Alerts / Tasks"]
-    H --> J["Verification Loop"]
-    J --> B
-
-    K["Policy Engine"] --> H
-    L["Audit / History Store"] --> H
-    L --> E
-    L --> C
-```
-
-
-## Agent Roles
-
-The first version should define these logical roles.
-
-### 1. Health Agent
-
-Primary use case:
-
-- datasource health governance
-
-Inputs:
-
-- datasource metadata
-- current endpoint
-- latest health records
-- latest failures
-- deterministic findings
-
-Outputs:
-
-- health interpretation
-- repair proposal
-- confidence
-- evidence references
-
-Typical action level:
-
-- propose-only
-
-
-### 2. Correlation Agent
-
-Primary use case:
-
-- identify whether multiple signals describe the same event or related events
-
-Inputs:
-
-- findings from multiple collectors
-- time windows
-- region / ASN / prefix / cable relationships
-- prior incidents
-
-Outputs:
-
-- grouped event candidates
-- correlation rationale
-- confidence per relationship
-
-Typical action level:
-
-- read-only
-
-
-### 3. Assessment Agent
-
-Primary use case:
-
-- produce situational-awareness outputs
-
-Inputs:
-
-- grouped events
-- findings
-- current context
-- historical context
-- operator constraints
-
-Outputs:
-
-- structured assessment
-- risk summary
-- evidence-backed recommendations
-- missing-information list
-
-Typical action level:
-
-- read-only or propose-only
-
-
-### 4. Recovery Agent
-
-Primary use case:
-
-- carry low-risk proposals into controlled runtime actions
-
-Inputs:
-
-- approved proposal
-- policy constraints
-- trusted-domain rules
-- verification checks
-
-Outputs:
-
-- applied override
-- failed application
-- rollback request
-
-Typical action level:
-
-- apply-limited
-
-
-## Shared Object Model
-
-All agents should work on a shared object model.
-
-That prevents the health subsystem and situational-awareness subsystem from drifting into incompatible payloads.
-
-### Signal
-
-Represents a raw observed fact.
-
-Examples:
-
-- a datasource returned HTTP 404
-- a collector returned empty results
-- BGP updates spiked in one region
-- a known endpoint now redirects elsewhere
-
-Suggested shape:
-
-```json
-{
-  "id": "sig_123",
-  "type": "datasource.http_failure",
-  "source": "ris_live_bgp",
-  "occurred_at": "2026-04-08T10:00:00Z",
-  "severity": "medium",
-  "payload": {},
-  "trust": 0.95
-}
-```
-
-
-### Finding
-
-Represents a deterministic or semi-deterministic interpretation of one or more signals.
-
-Examples:
-
-- `schema_changed`
-- `endpoint_unreachable`
-- `data_volume_abnormally_low`
-- `event_cluster_detected`
-
-Suggested shape:
-
-```json
-{
-  "id": "find_123",
-  "type": "datasource.schema_changed",
-  "source_ids": ["sig_123"],
-  "confidence": 0.92,
-  "evidence": [],
-  "details": {}
-}
-```
-
-
-### Proposal
-
-Represents a recommended action, not an already-applied action.
-
-Examples:
-
-- switch endpoint to new URL
-- disable bad override
-- escalate issue for manual review
-
-Suggested shape:
-
-```json
-{
-  "id": "prop_123",
-  "kind": "endpoint_override",
-  "target": "telegeography_cables",
-  "confidence": 0.84,
-  "reason": "Official docs now point to a new API path",
-  "payload": {},
-  "evidence_urls": [],
-  "status": "proposed"
-}
-```
-
-
-### Assessment
-
-Represents a structured situational-awareness output for operators or downstream systems.
-
-Examples:
-
-- current network posture summary
-- incident impact assessment
-- risk and response recommendations
-
-Suggested shape:
-
-```json
-{
-  "id": "assess_123",
-  "scope": "regional-network",
-  "risk_level": "high",
-  "summary": "Regional routing instability is increasing.",
-  "key_risks": [],
-  "evidence": [],
-  "recommendations": [],
-  "missing_data": []
-}
-```
-
-
-## State Machine
-
-The shared orchestration flow should look like this:
-
-```mermaid
-stateDiagram-v2
-    [*] --> Collect
-    Collect --> Validate
-    Validate --> Classify
-    Classify --> Reason
-    Reason --> Propose
-    Reason --> Assess
-    Propose --> Review
-    Review --> Apply
-    Apply --> Verify
-    Verify --> Archive
-    Assess --> Archive
-    Archive --> [*]
-```
-
-Definitions:
-
-- `Collect`: gather signals
-- `Validate`: run deterministic checks
-- `Classify`: create findings
-- `Reason`: invoke LLM reasoning when needed
-- `Propose`: create change proposals
-- `Review`: policy or human approval
-- `Apply`: perform limited runtime action
-- `Verify`: confirm action effect
-- `Archive`: store artifacts and decisions
-
-
-## Permission Model
-
-Each agent role should be assigned one of these action levels.
-
-### `read-only`
-
-Allowed:
-
-- read signals
-- search web
-- fetch pages
-- read internal state
-- generate findings and assessments
-
-Not allowed:
-
-- mutate config
-- write overrides
-- change live runtime behavior
-
-
-### `propose-only`
-
-Allowed:
-
-- everything in `read-only`
-- create proposals
-- create review tasks
-
-Not allowed:
-
-- apply live changes
-
-
-### `apply-limited`
-
-Allowed:
-
-- everything in `propose-only`
-- write approved runtime overrides
-- trigger verification checks
-
-Not allowed:
-
-- mutate repository defaults
-- make destructive data changes
-- bypass policy engine
-
-
-## Runtime Components
-
-The first durable architecture should introduce these components.
-
-### 1. Signal Store
-
-Stores normalized evidence and health outputs.
-
-
-### 2. Finding Store
-
-Stores deterministic classifications that can be reused by multiple agents.
-
-
-### 3. Proposal Store
-
-Stores recommended actions with evidence and confidence.
-
-
-### 4. Assessment Store
-
-Stores structured situational-awareness outputs.
-
-
-### 5. Policy Engine
-
-Decides:
-
-- whether agent may run
-- whether proposal requires review
-- whether proposal may auto-apply
-- whether post-apply verification passed
-
-
-### 6. Override Store
-
-Stores runtime-only configuration changes.
-
-This is where endpoint repairs should live.
-
-
-## Relation To `aiprovider`
-
-`aiprovider` should remain the model gateway.
-
-It should not become the full agent runtime.
-
-Recommended split:
-
-- `aiprovider`
-  - provider adaptation
-  - prompt transport
-  - model execution
-  - protocol compatibility
-
-- agent runtime
-  - orchestration
-  - signal handling
-  - tool selection
-  - proposal generation
-  - policy and audit
-
-This keeps provider concerns and agent behavior concerns separate.
-
-
-## Relation To Datasource Health
-
-Datasource health becomes one vertical slice of this architecture.
-
-Mapping:
-
-- signal:
-  - endpoint unreachable
-  - schema mismatch
-  - bad content type
-- finding:
-  - `failed`
-  - `schema_changed`
-  - `moved_endpoint_suspected`
-- proposal:
-  - runtime override suggestion
-- assessment:
-  - datasource health summary for operators
-
-
-## Relation To Situational Awareness
-
-Future situational-awareness capabilities should reuse the same flow:
-
-- raw telemetry becomes signals
-- anomaly detection becomes findings
-- LLM correlation becomes reasoning
-- operator-facing output becomes assessments
-- policy-approved mitigations become actions
-
-This lets the platform evolve from operational health governance into broader cyber/network posture workflows without changing the architecture.
-
-
-## Suggested Delivery Sequence
-
-### Phase A
-
-- finalize shared object model
-- implement health-oriented signal and finding storage
-
-### Phase B
-
-- implement Health Agent
-- generate proposals only
-
-### Phase C
-
-- implement Assessment Agent
-- expose structured assessments via API
-
-### Phase D
-
-- implement Correlation Agent
-- support multi-source incident grouping
-
-### Phase E
-
-- implement Recovery Agent with policy-gated runtime actions
-
-
-## Recommended First Build
-
-The first build should not try to implement every agent role.
-
-Recommended initial slice:
-
-- shared object model
-- health signals
-- health findings
-- Health Agent
-- proposal generation only
-
-This gives immediate value while preserving the longer-term architecture.
-
-
-## Non-Goals For The First Iteration
-
-- repository YAML auto-rewrites
-- unrestricted autonomous action
-- full incident graph reasoning
-- automatic large-scale remediation
-- agent-owned configuration source of truth
-
-
-## Summary
-
-Planet should treat agents as a reusable runtime for evidence, reasoning, proposals, and assessments.
-
-The datasource health use case is the first practical entrypoint, but the architecture should already assume future situational-awareness expansion.
-
-The safest path is:
-
-- deterministic checks first
-- agent reasoning second
-- proposals before actions
-- runtime overrides instead of default mutation

docs/agents/agent-runtime-roadmap.md

@@ -1,346 +0,0 @@
-# Agent Runtime Roadmap
-
-## Overview
-
-This document connects three existing planning threads into one implementation roadmap:
-
-- `aiprovider` as the model gateway
-- datasource health governance as the first practical agent use case
-- situational awareness as the broader long-term target
-
-Related documents:
-
-- [aiprovider](/home/ray/dev/linkong/planet/docs/agents/aiprovider.md)
-- [datasource-health-plan](/home/ray/dev/linkong/planet/docs/agents/datasource-health-plan.md)
-- [agent-architecture-plan](/home/ray/dev/linkong/planet/docs/agents/agent-architecture-plan.md)
-
-
-## Big Picture
-
-Planet should evolve in layers:
-
-1. stable model gateway
-2. deterministic health and evidence collection
-3. agent runtime for reasoning and proposal generation
-4. situational-awareness assessments and controlled actions
-
-This prevents the system from collapsing into a single giant "AI feature" with unclear boundaries.
-
-
-## Architecture Overview
-
-```mermaid
-flowchart TD
-    U["Frontend / Backend APIs / Operators"] --> B["Planet Backend"]
-    B --> H["Datasource Health Services"]
-    B --> R["Agent Runtime"]
-    R --> P["aiprovider"]
-    P --> M["OpenAI / Anthropic / MiniMax / Ollama / Local Models"]
-
-    C["Collectors / Snapshots / Logs / Alerts / BGP Signals"] --> S["Signal Store"]
-    H --> S
-    S --> E["Evaluation Layer"]
-    E --> F["Findings"]
-    F --> R
-
-    W["Web Search / Page Fetch / Docs Fetch"] --> R
-    R --> PR["Proposals"]
-    R --> AS["Assessments"]
-
-    PR --> O["Runtime Overrides / Review Queue / Tasks"]
-    AS --> SA["Situational Awareness APIs / UI"]
-
-    O --> V["Verification Loop"]
-    V --> S
-```
-
-
-## Role Boundaries
-
-### `aiprovider`
-
-Responsibilities:
-
-- provider compatibility
-- protocol adaptation
-- auth and model transport
-- request/response normalization
-
-Not responsible for:
-
-- agent orchestration
-- business workflows
-- datasource repair policy
-- situational-awareness domain logic
-
-
-### Backend
-
-Responsibilities:
-
-- stable business APIs
-- auth and permissions
-- task orchestration
-- health records
-- proposal and override persistence
-- assessment exposure
-
-
-### Agent Runtime
-
-Responsibilities:
-
-- consume findings and context
-- invoke LLMs via `aiprovider`
-- invoke tools such as web search
-- create proposals
-- create assessments
-- route to policy-controlled action paths
-
-
-## Delivery Sequence
-
-## Stage 1: Gateway Foundation
-
-Status:
-
-- already in place
-
-Delivered by current work:
-
-- `aiprovider`
-- multi-provider compatibility
-- backend AI facade
-- MiniMax / Anthropic-compatible support
-- request-id propagation
-
-Primary outcome:
-
-- the system already has a stable way to call models
-
-
-## Stage 2: Datasource Health MVP
-
-Goal:
-
-- establish deterministic health observability
-
-Key work:
-
-- health check task runner
-- health result table
-- datasource health APIs
-- UI visibility
-- collector endpoint override precedence cleanup
-
-Primary outcome:
-
-- Planet knows which collectors are healthy before asking an LLM anything
-
-
-## Stage 3: Health Agent
-
-Goal:
-
-- let the first agent role operate on health failures
-
-Key work:
-
-- convert health failures into signals/findings
-- invoke agent only for failed or suspicious cases
-- produce repair proposals with evidence and confidence
-
-Primary outcome:
-
-- Planet can suggest endpoint repairs without mutating defaults
-
-
-## Stage 4: Runtime Repair Application
-
-Goal:
-
-- safely apply approved datasource repair proposals
-
-Key work:
-
-- override storage
-- policy-gated apply flow
-- verification after apply
-- rollback path
-
-Primary outcome:
-
-- datasource repair becomes operationally useful without polluting repository defaults
-
-
-## Stage 5: Situational Awareness Assessments
-
-Goal:
-
-- reuse the same runtime for broader operator-facing assessment
-
-Key work:
-
-- normalize telemetry and incident evidence into signals/findings
-- build Assessment Agent
-- expose structured assessments through backend APIs and UI
-
-Primary outcome:
-
-- LLM output becomes evidence-backed situational summary, not just ad hoc chat output
-
-
-## Stage 6: Correlation and Controlled Actions
-
-Goal:
-
-- connect multiple sources into higher-level posture and event groupings
-
-Key work:
-
-- event correlation
-- incident grouping
-- recommendation scoring
-- controlled action routing
-
-Primary outcome:
-
-- Planet becomes a true agent-assisted situational-awareness system
-
-
-## Implementation Tracks
-
-These tracks can progress in parallel, but they should stay loosely coupled.
-
-### Track A: Config and Runtime Resolution
-
-Scope:
-
-- datasource defaults
-- overrides
-- runtime precedence
-- audit trails
-
-First milestone:
-
-- health-safe override layer
-
-
-### Track B: Health and Evidence
-
-Scope:
-
-- deterministic checks
-- failure categorization
-- signal and finding persistence
-
-First milestone:
-
-- datasource health record system
-
-
-### Track C: Agent Runtime
-
-Scope:
-
-- shared object model
-- orchestration flow
-- prompt/tool pipeline
-- policy integration
-
-First milestone:
-
-- Health Agent proposal pipeline
-
-
-### Track D: Situational Awareness
-
-Scope:
-
-- assessment schema
-- multi-source context assembly
-- operator-facing outputs
-
-First milestone:
-
-- structured assessment API
-
-
-## Shared Artifacts
-
-To avoid fragmentation, these artifacts should be shared across all future agent work.
-
-### Shared object model
-
-- `Signal`
-- `Finding`
-- `Proposal`
-- `Assessment`
-
-### Shared orchestration flow
-
-- collect
-- validate
-- classify
-- reason
-- propose or assess
-- review or apply
-- verify
-- archive
-
-### Shared policy model
-
-- read-only
-- propose-only
-- apply-limited
-
-
-## Recommended Next Concrete Steps
-
-1. Build Stage 2 first
-
-- datasource health records
-- deterministic checks
-- no automatic repair
-
-2. Then build Stage 3
-
-- Health Agent
-- proposal generation only
-
-3. Then Stage 4
-
-- override apply flow
-- rollback and verification
-
-4. Only after that start Stage 5
-
-- broader situational-awareness assessment workflows
-
-
-## Why This Order
-
-Because situational-awareness quality depends on reliable upstream data.
-
-If datasource health is weak:
-
-- agent reasoning quality will degrade
-- false explanations will increase
-- assessment trust will drop
-
-So datasource health is not a side task.
-
-It is the first operational foundation for the later situational-awareness system.
-
-
-## Summary
-
-Planet should be built as:
-
-- `aiprovider` for model access
-- backend services for orchestration and persistence
-- datasource health as the first evidence-governance layer
-- agent runtime as the reusable reasoning core
-- situational awareness as the long-term application layer
-
-That path keeps the architecture coherent and lets each phase produce useful functionality without forcing a rewrite later.

docs/ai-playground-development-plan.md

@@ -1,361 +0,0 @@
-# AI Playground Development Plan
-
-## 目标
-
-这份计划用于统一 `aiprovider`、`backend AI facade`、`Playground` 页面，以及后续 `BGP / 告警 / 数据源健康` 等 AI 入口的演进方向。
-
-当前原则：
-
-- `aiprovider` 继续作为独立模型网关
-- `backend` 继续作为稳定业务入口
-- `frontend` 负责测试台和业务 UI
-- 先做“可控、可验证、可解释”的 AI 能力，再逐步引入 agent/tool calling
-
-## 当前已完成
-
-### 1. AI 网关基础层
-
-已完成：
-
-- 独立 `aiprovider` 服务
-- `backend -> aiprovider -> model provider` 调用链
-- `provider/status` 与 `situational-awareness/analyze` 稳定接口
-- `X-Request-ID` 透传
-- 轻量超时与重试
-- MiniMax / Anthropic-compatible / OpenAI-compatible / Ollama 适配
-
-相关文件：
-
-- [backend/app/api/v1/ai.py](/home/ray/dev/linkong/planet/backend/app/api/v1/ai.py)
-- [backend/app/services/ai_client.py](/home/ray/dev/linkong/planet/backend/app/services/ai_client.py)
-- [aiprovider/main.py](/home/ray/dev/linkong/planet/aiprovider/main.py)
-- [aiprovider/provider_service.py](/home/ray/dev/linkong/planet/aiprovider/provider_service.py)
-- [docs/aiprovider.md](/home/ray/dev/linkong/planet/docs/aiprovider.md)
-
-### 2. 本地运行与配置打通
-
-已完成：
-
-- `planet.sh` 启动链路纳入 `aiprovider`
-- `planet.sh` 启动完成后输出 Playground 入口
-- `docker-compose.yml` 为 `aiprovider` 加入 `env_file`
-- `backend/.env` 与 `aiprovider/.env` 两侧 service token 对齐
-- `Playground` 状态缓存，避免页面切换时每次都重新请求 provider 状态
-
-相关文件：
-
-- [planet.sh](/home/ray/dev/linkong/planet/planet.sh)
-- [docker-compose.yml](/home/ray/dev/linkong/planet/docker-compose.yml)
-- [backend/.env.example](/home/ray/dev/linkong/planet/backend/.env.example)
-- [aiprovider/.env.example](/home/ray/dev/linkong/planet/aiprovider/.env.example)
-
-### 3. Playground UI 基础版
-
-已完成：
-
-- 新增前端路由 `/playground`
-- 左侧 `Provider 状态 + 测试说明`
-- 右侧 `请求 / 结果` Tabs
-- `Provider 状态` 支持手动刷新
-- `测试说明` 支持折叠
-- 内部区域采用细滚动条
-- 页面布局开始遵循“单屏工作区 + 模块内部滚动”规范
-
-相关文件：
-
-- [frontend/src/pages/Playground/Playground.tsx](/home/ray/dev/linkong/planet/frontend/src/pages/Playground/Playground.tsx)
-- [frontend/src/App.tsx](/home/ray/dev/linkong/planet/frontend/src/App.tsx)
-- [frontend/src/components/AppLayout/AppLayout.tsx](/home/ray/dev/linkong/planet/frontend/src/components/AppLayout/AppLayout.tsx)
-- [frontend/src/index.css](/home/ray/dev/linkong/planet/frontend/src/index.css)
-
-### 4. 前端布局规范沉淀
-
-已完成：
-
-- 把“一屏工作区、主模块优先、模块内部滚动”的规范文档化
-- 明确 `BGP` 页面为当前参考实现
-
-相关文件：
-
-- [docs/frontend-layout-guidelines.md](/home/ray/dev/linkong/planet/docs/frontend-layout-guidelines.md)
-- [frontend/src/pages/BGP/BGP.tsx](/home/ray/dev/linkong/planet/frontend/src/pages/BGP/BGP.tsx)
-
-## 当前限制
-
-### 1. Playground 还是 prompt playground，不是 agent playground
-
-当前 `Playground` 的 `观察项 / 目标 / 约束条件` 都是人工输入。
-
-模型现在拿到的是：
-
-- 你手工输入的结构化字段
-- 后端传递的少量静态上下文
-
-模型现在拿不到：
-
-- 实时 BGP 事件
-- 真实告警列表
-- 数据源健康状态
-- 自动检索结果
-- tool calling / skills / 自主取数
-
-### 2. `situational-awareness/analyze` 还是通用提示词接口
-
-当前更适合：
-
-- 测试链路
-- 测试模型输出风格
-- 验证不同 provider 是否正常返回
-
-当前还不适合：
-
-- 直接当真实态势系统主入口
-- 让用户手工维护长期分析模板
-- 代替专用业务研判接口
-
-### 3. 还没有可验证的真实业务输入注入
-
-目前最缺的是：
-
-- 从业务系统自动整理“事实输入”
-- 再把这些事实喂给 AI
-
-而不是继续让用户在 Playground 手工输入真实事件摘要。
-
-## 短期计划
-
-### Phase A: Playground 收敛为稳定测试台
-
-目标：
-
-- 保持 Playground 简洁可用
-- 不再继续堆“高级参数”
-
-工作项：
-
-- 继续微调左侧 `Provider 状态` 与 `测试说明` 的空间策略
-- 保持 `请求 / 结果` 为单一主工作区
-- 不引入盲填式高级字段
-- 统一滚动条、卡片、溢出行为
-
-完成标准：
-
-- 笔记本视口下依然可用
-- 各模块标题可见
-- 主要阅读区始终是右侧 Tabs
-
-### Phase B: BGP AI 简报
-
-目标：
-
-- 不再依赖手工填写“观察项”
-- 让系统自动把真实 BGP 数据注入 AI
-- 让 BGP 页面逐步从“摘要汇总”升级为“证据驱动的区域态势分析”
-
-建议实现：
-
-- 新增专用后端接口，例如：
-  - `POST /api/v1/ai/bgp/brief`
-- 后端自动读取：
-  - incidents summary
-  - anomalies
-  - recent events
-  - collector coverage summary
-- 后端将结构化事实注入 `context / observations`
-- 前端在 BGP 页面增加“生成 AI 简报”
-
-当前阶段说明：
-
-- 第一版 `BGP AI 简报` 允许先落地为“值班摘要生成器”
-- 也就是先把 incidents / anomalies / events / collector coverage 自动注入
-- 允许模型先做事实摘要、风险归纳、建议动作
-
-但这不应被视为 Phase B 的最终形态。
-
-Phase B 后续还需要补齐：
-
-- prefix geography 证据注入
-  - `iptoasn`
-  - `opengeofeed`
-  - `nro_delegated`
-- 基于 `affected_regions` 与 prefix geography 的区域聚合
-- 区分“真实区域热度”与“collector coverage 偏差”
-- 对高风险 prefix / ASN 给出更明确的国家、城市、运营商归属线索
-- 让 AI 输出明确回答：
-  - 哪些区域正在异常升温
-  - 哪些结论只是观测站偏差
-  - 当前还缺哪些区域证据
-
-完成标准：
-
-- 用户不需要手工录入 BGP 观察项
-- AI 输出能明确区分“事实”和“研判”
-- AI 不只是复述总量和最近几条事件，还能利用 prefix geography 与 affected regions 做区域态势判断
-- 输出中能明确指出：
-  - 高风险区域
-  - 区域证据来源
-  - collector coverage 偏差对判断的影响
-
-### Phase C: 告警 / 数据源健康 AI 简报
-
-目标：
-
-- 复用同样模式，扩展到其他模块
-
-建议入口：
-
-- `Alerts` 页面：异常与告警摘要
-- `DataSources` 页面：采集失败与健康状态总结
-
-原则：
-
-- 每个业务页优先做“专用 AI 简报”
-- 不优先做“万能大聊天框”
-
-## 中期计划
-
-### 1. Assessment Layer
-
-目标：
-
-- 不只返回自由文本
-- 返回结构化的 assessment
-
-建议输出字段：
-
-- summary
-- key_risks
-- evidence
-- recommendations
-- confidence
-- missing_data
-
-这样后续才能：
-
-- 持久化
-- 回看
-- 对比不同时间的 AI 结论
-- 在 Earth / Dashboard / BGP 页面稳定展示
-
-### 2. Evidence-first Runtime
-
-目标：
-
-- 所有 AI 分析先取真实数据，再调模型
-
-原则：
-
-- 先 evidence
-- 再 prompt
-- 最后才是自由生成
-
-优先要做的不是更强聊天，而是：
-
-- 更稳定的数据注入
-- 更一致的事实模板
-- 更清晰的结果结构
-
-### 3. 按页面提供专用入口
-
-目标：
-
-- 让 AI 成为业务视图的一部分，而不是孤立 playground
-
-优先顺序建议：
-
-1. `BGP` AI 简报
-2. `Alerts` AI 简报
-3. `DataSources` 健康研判
-4. `Dashboard` 总览总结
-
-## 长期计划
-
-### 1. Tool Calling / Agent Runtime
-
-只有在以下基础稳定后再推进：
-
-- 数据源健康信号稳定
-- BGP / Alerts / Datasource evidence 注入稳定
-- assessment 结构稳定
-
-长期可做能力：
-
-- AI 调用受控工具查询业务数据
-- AI 调用检索/web search 做外部验证
-- AI 生成建议而不是直接修改系统
-- 审核后触发受控动作
-
-### 2. 受控动作与闭环
-
-潜在方向：
-
-- 根据健康异常生成修复建议
-- 根据态势变化生成处理建议
-- 进入 review queue
-- 审批后执行
-- 验证结果并形成闭环
-
-### 3. 多模块统一 AI 体验
-
-长期目标不是一个孤立 Playground，而是：
-
-- 每个业务页都有自己的 AI 入口
-- 共享统一的 backend AI facade
-- 共享统一的 assessment 结构
-- 共享统一的 evidence 注入与审计链路
-
-## 设计决策总结
-
-### 为什么保留 `aiprovider`
-
-因为它已经很好地承担了：
-
-- provider 适配
-- 协议兼容
-- service token 边界
-- 独立重启与部署
-
-因此短期内不建议把它并回 `backend`。
-
-### 为什么 Playground 不做成万能聊天页
-
-因为当前更需要的是：
-
-- 稳定测试链路
-- 可验证业务输入
-- 专用分析入口
-
-而不是一个泛化但没有真实数据支撑的聊天框。
-
-### 为什么优先做专用 AI 简报
-
-因为：
-
-- 数据可以自动注入
-- 用户心智更清晰
-- 输出更容易结构化
-- 更容易校验事实与研判是否一致
-
-## 下一步建议
-
-按优先级建议接下来这样做：
-
-1. 稳住 `Playground` 当前布局，不再大幅重做
-2. 在 `BGP` 页面新增专用 “AI 简报” 入口
-3. 后端新增 `BGP brief` 专用接口，自动注入真实数据
-4. 补齐 `BGP brief` 的区域态势证据层
-5. 把 AI 输出逐步从自由文本升级为结构化 assessment
-
-### BGP Brief 后续子项
-
-为避免把“已有 AI 简报”误判成“区域分析已完成”，这里单独记录 `BGP brief` 的后续 backlog：
-
-1. 把高风险 prefix 命中的 `iptoasn / opengeofeed / nro_delegated` 结果注入 brief context
-2. 按国家/城市聚合 active incidents、anomalies、affected prefixes，生成区域热点事实层
-3. 把 collector coverage 与区域热点并排注入，避免模型把观测偏差误判成区域风险
-4. 对高风险 ASN / prefix 追加归属线索，如国家、城市、可能运营商或注册区域
-5. 在输出结构中单独增加：
-   - 区域态势
-   - 证据来源
-   - 观测偏差说明
-   - 缺失区域证据

docs/aiprovider.md

@@ -1,333 +0,0 @@
-# AI Provider Guide
-
-## Overview
-
-`aiprovider` is the model-adapter service for Planet.
-
-It isolates model-vendor details from the main backend so the rest of the system can call a stable business API:
-
-- Caller service -> `planet backend`
-- `planet backend` -> `aiprovider`
-- `aiprovider` -> concrete model provider
-
-The recommended default is:
-
-- External and cross-service callers use `planet backend`
-- Only infrastructure-grade internal jobs call `aiprovider` directly
-
-## Responsibilities
-
-`backend` is responsible for:
-
-- authentication and authorization
-- business-level request shaping
-- stable `/api/v1/ai/...` endpoints
-- internal service-to-service authentication toward `aiprovider`
-
-`aiprovider` is responsible for:
-
-- model protocol adaptation
-- provider selection by `.env`
-- timeout and lightweight retry
-- request tracing via `X-Request-ID`
-
-This now follows an OpenClaw-like seam:
-
-- `AI_PROVIDER` identifies the vendor or logical provider
-- `AI_PROVIDER_API` identifies the wire adapter
-
-That split makes MiniMax, Claude-compatible gateways, and self-hosted OpenAI-compatible services easier to model without overloading one config field.
-
-## Supported Providers
-
-`aiprovider` currently supports these provider identities:
-
-- `openai`
-- `anthropic`
-- `minimax`
-- `ollama`
-
-Supported request adapters:
-
-- `openai-completions`
-- `anthropic-messages`
-- `ollama-generate`
-
-Backward-compatible aliases still accepted:
-
-- `openai_compatible`
-- `anthropic_compatible`
-- `claude_compatible`
-
-Provider mapping:
-
-- `vLLM`, `LM Studio`, `One API`: `AI_PROVIDER=openai`, `AI_PROVIDER_API=openai-completions`
-- `MiniMax`: `AI_PROVIDER=minimax`, `AI_PROVIDER_API=anthropic-messages`
-- Claude-compatible gateways: `AI_PROVIDER=anthropic`, `AI_PROVIDER_API=anthropic-messages`
-- `Ollama`: `AI_PROVIDER=ollama`, `AI_PROVIDER_API=ollama-generate`
-
-## API Surfaces
-
-### Main backend API
-
-Preferred stable entrypoints:
-
-- `GET /api/v1/ai/provider/status`
-- `POST /api/v1/ai/situational-awareness/analyze`
-
-Authentication:
-
-- `Authorization: Bearer <jwt>`
-
-Optional tracing header:
-
-- `X-Request-ID: <caller-generated-id>`
-
-The backend will propagate `X-Request-ID` to `aiprovider` and return the same header in the response.
-
-### AI provider internal API
-
-Internal-only endpoints:
-
-- `GET /v1/provider/status`
-- `POST /v1/analyze`
-
-Authentication:
-
-- `X-Provider-Token: <shared-secret>`
-
-Optional tracing header:
-
-- `X-Request-ID: <caller-generated-id>`
-
-## Request Example
-
-### Call through backend
-
-```bash
-curl -X POST http://localhost:8000/api/v1/ai/situational-awareness/analyze \
-  -H "Authorization: Bearer <access_token>" \
-  -H "X-Request-ID: bgp-incident-20260407-001" \
-  -H "Content-Type: application/json" \
-  -d '{
-    "title": "BGP异常研判",
-    "objective": "总结当前风险并给出处置建议",
-    "observations": [
-      "collector A 在 5 分钟内出现多次 origin 变更",
-      "异常集中在同一地区前缀"
-    ],
-    "constraints": [
-      "不要编造不存在的数据",
-      "区分事实和推断"
-    ],
-    "context": {
-      "source": "bgp-monitor",
-      "severity": "high"
-    }
-  }'
-```
-
-### Call `aiprovider` directly
-
-```bash
-curl -X POST http://localhost:8010/v1/analyze \
-  -H "X-Provider-Token: change_me" \
-  -H "X-Request-ID: ai-batch-job-001" \
-  -H "Content-Type: application/json" \
-  -d '{
-    "title": "链路波动分析",
-    "objective": "给出简要态势摘要和下一步建议",
-    "observations": [
-      "多个节点出现延迟上升"
-    ],
-    "constraints": [
-      "不要假设根因已经确认"
-    ],
-    "context": {
-      "region": "APAC"
-    }
-  }'
-```
-
-## Response Shape
-
-Both backend and `aiprovider` return the same payload shape:
-
-```json
-{
-  "provider": "minimax",
-  "api": "anthropic-messages",
-  "model": "MiniMax-M2.7",
-  "content": "1) 态势摘要 ...",
-  "content_blocks": [],
-  "text_blocks": [],
-  "thinking_blocks": [],
-  "raw_response": {}
-}
-```
-
-Both services also return:
-
-- `X-Request-ID: <id>`
-
-## Configuration
-
-### Backend
-
-Recommended backend `.env`:
-
-```env
-AI_PROVIDER_SERVICE_URL=http://localhost:8010
-AI_PROVIDER_SERVICE_TOKEN=change_me
-AI_PROVIDER_TIMEOUT_SECONDS=60
-AI_PROVIDER_RETRY_ATTEMPTS=2
-```
-
-Reference file:
-
-- [backend/.env.example](/home/ray/dev/linkong/planet/backend/.env.example)
-
-### AI Provider
-
-Reference file:
-
-- [aiprovider/.env.example](/home/ray/dev/linkong/planet/aiprovider/.env.example)
-
-Frontend local reference:
-
-- [frontend/.env.example](/home/ray/dev/linkong/planet/frontend/.env.example)
-
-Common settings:
-
-```env
-SERVICE_NAME=planet-ai-provider
-SERVICE_VERSION=0.1.0
-AI_PROVIDER_SERVICE_TOKEN=change_me
-AI_TIMEOUT_SECONDS=60
-AI_HTTP_RETRY_ATTEMPTS=2
-AI_ANALYSIS_SYSTEM_PROMPT=你是态势感知分析助手。请基于输入的上下文、观测与约束，输出结构化、克制、可执行的分析。
-```
-
-### OpenAI-compatible example
-
-```env
-AI_PROVIDER=openai
-AI_PROVIDER_API=openai-completions
-AI_BASE_URL=http://127.0.0.1:8001/v1
-AI_API_KEY=local-key
-AI_MODEL=your-local-model
-```
-
-### MiniMax CN example
-
-```env
-AI_PROVIDER=minimax
-AI_PROVIDER_API=anthropic-messages
-AI_BASE_URL=https://api.minimaxi.com/anthropic
-AI_API_KEY=sk-cp-xxxxx
-AI_MODEL=MiniMax-M2.7
-AI_MAX_TOKENS=1200
-AI_ANTHROPIC_VERSION=2023-06-01
-```
-
-MiniMax note:
-
-- This follows the same Anthropic Messages request shape as the official MiniMax examples.
-- For MiniMax, `aiprovider` now disables `thinking` by default unless the caller explicitly passes a `thinking` object.
-- This mirrors OpenClaw's caution around MiniMax Anthropic-compatible behavior.
-
-### Anthropic-compatible example
-
-```env
-AI_PROVIDER=anthropic
-AI_PROVIDER_API=anthropic-messages
-AI_BASE_URL=https://your-claude-compatible-endpoint.example.com/anthropic
-AI_API_KEY=your_api_key
-AI_MODEL=your-model
-AI_MAX_TOKENS=1200
-AI_ANTHROPIC_VERSION=2023-06-01
-```
-
-### Ollama example
-
-```env
-AI_PROVIDER=ollama
-AI_PROVIDER_API=ollama-generate
-AI_BASE_URL=http://127.0.0.1:11434
-AI_API_KEY=
-AI_MODEL=qwen2.5:7b
-```
-
-## Deployment Modes
-
-### Single machine
-
-Recommended local flow:
-
-- `backend` on `localhost:8000`
-- `aiprovider` on `localhost:8010`
-- local model gateway on `localhost:11434` or another local port
-
-Helpers already included:
-
-- [planet.sh](/home/ray/dev/linkong/planet/planet.sh)
-- [docker-compose.local-model.yml](/home/ray/dev/linkong/planet/docker-compose.local-model.yml)
-
-### Multi-machine
-
-Example topology:
-
-- app machine: `backend`
-- AI gateway machine: `aiprovider`
-- model machine: local model service or cloud proxy
-
-In that case, this becomes service-to-service HTTP RPC:
-
-- caller -> backend
-- backend -> `http://10.0.0.12:8010`
-- `aiprovider` -> model endpoint
-
-Recommended cross-machine backend config:
-
-```env
-AI_PROVIDER_SERVICE_URL=http://10.0.0.12:8010
-AI_PROVIDER_SERVICE_TOKEN=change_me
-AI_PROVIDER_TIMEOUT_SECONDS=60
-AI_PROVIDER_RETRY_ATTEMPTS=2
-```
-
-Recommended operating rules:
-
-- keep `aiprovider` on a private network
-- protect it with `X-Provider-Token` at minimum
-- always send `X-Request-ID`
-- keep callers on the backend API unless they are infrastructure jobs
-
-## Retry And Failure Behavior
-
-`backend -> aiprovider`:
-
-- retries lightweight network / 5xx failures
-- returns `502` when the provider service is unavailable
-
-`aiprovider -> model provider`:
-
-- retries lightweight network / 5xx failures
-- returns `502` when the model provider is unavailable
-
-This is intentionally conservative. It avoids masking persistent errors while still absorbing short hiccups.
-
-## Operational Notes
-
-- `./planet.sh start` now starts `aiprovider` automatically
-- `./planet.sh restart -a` restarts only `aiprovider`
-- `./planet.sh log -a` tails `aiprovider` logs
-- `./planet.sh health` reports `aiprovider` health
-
-## Recommended Calling Policy
-
-- Frontend and application services: call `backend`
-- Scheduled infra jobs and diagnostics: optionally call `aiprovider`
-- Do not let multiple business services integrate model vendors independently
-
-That keeps provider switching centralized and avoids model-specific drift across the system.

docs/bgp-region-aggregation-plan.md

@@ -1,422 +0,0 @@
-# BGP Region Aggregation Plan
-
-## Goal
-
-This document refines the current BGP `activity layer` into an implementation-ready regional aggregation design.
-
-Primary product goal:
-
-- turn sparse prefix-level observations, anomalies, and incidents into a readable `regional observability layer`
-- keep Earth visually alive during low-incident periods
-- make `incident markers` remain the highest-confidence foreground layer instead of replacing them
-
-This layer is not a new collector, detector, or raw storage table.
-It is an aggregation/view-model layer:
-
-`observations -> enrichment -> anomalies/incidents -> geography mapping -> region aggregation -> Earth/UI activity layer`
-
-## Why This Layer Exists
-
-Current product gap from [bgp-context.md](/home/ray/dev/linkong/planet/docs/bgp-context.md):
-
-- incident density is naturally low
-- anomaly density is higher, but still not enough to keep the globe expressive all the time
-- collector presence alone proves coverage, but does not communicate `where routing is currently active or noisy`
-
-So the missing middle layer is:
-
-- `collectors` show that observation exists
-- `regions` show where activity is building up
-- `incidents` show the specific high-confidence focus events
-
-## Scope
-
-This plan is specifically for:
-
-- a backend aggregation service
-- a summary API for console/stats
-- a GeoJSON API for Earth rendering
-- an Earth background activity layer that supports, but does not replace, incident markers
-
-This plan does not attempt to solve:
-
-- exact prefix geolocation quality
-- polygon-heavy geopolitical visualization
-- persistent materialized region tables in v1
-
-## Region Layer Definition
-
-Recommended conceptual model:
-
-- `region layer` = background situational awareness
-- `incident layer` = focal event markers
-
-That means:
-
-- region activity should answer `where is routing behavior currently active or abnormal`
-- incident markers should answer `which concrete event should the user click`
-
-## Recommended Output Model
-
-Suggested backend output object:
-
-## `BGPRegionActivity`
-
-```json
-{
-  "region_key": "sea",
-  "region_name": "Southeast Asia",
-  "center_lat": 1.3521,
-  "center_lon": 103.8198,
-  "observation_count": 128,
-  "anomaly_count": 9,
-  "incident_count": 2,
-  "activity_score": 17.6,
-  "status": "incident",
-  "affected_prefix_count": 14,
-  "affected_asn_count": 6,
-  "collector_count": 5,
-  "first_seen_at": "2026-04-02T10:00:00Z",
-  "last_seen_at": "2026-04-02T10:12:00Z"
-}
-```
-
-### Fields To Keep In MVP
-
-- `region_key`
-- `region_name`
-- `center_lat`
-- `center_lon`
-- `observation_count`
-- `anomaly_count`
-- `incident_count`
-- `activity_score`
-- `status`
-- `affected_prefix_count`
-- `affected_asn_count`
-- `collector_count`
-- `first_seen_at`
-- `last_seen_at`
-
-### Fields To Delay
-
-These are useful, but not required for the first implementation:
-
-- `bounding_box`
-- `top_incident_types`
-- `top_prefixes`
-- polygon geometry
-
-## Region Definition Strategy
-
-### Recommendation
-
-Use a static region-definition table first.
-
-Examples:
-
-- `north_america`
-- `south_america`
-- `western_europe`
-- `eastern_europe`
-- `east_asia`
-- `southeast_asia`
-- `south_asia`
-- `middle_east`
-- `north_africa`
-- `sub_saharan_africa`
-- `oceania`
-
-Why this is the right v1 choice:
-
-- stable UI semantics
-- strong readability on Earth
-- easier debugging and explanation
-- lower implementation cost than geohash or H3 grids
-
-### Not Recommended For V1
-
-- geohash cell aggregation
-- H3 aggregation
-- fine-grained lat/lon bucket maps
-
-Those are more flexible, but they make the map feel fragmented and less explainable.
-
-## Geography Mapping Strategy
-
-Do not reduce the implementation to only `prefix -> exact geo`.
-
-The region layer should follow the same geography-priority logic already implied by the current BGP direction:
-
-1. `prefix_geography`
-2. `prefix_scope`
-3. `ASN organization region`
-4. `collector centroid` fallback
-
-This matters because exact prefix geography will often be incomplete or approximate.
-The region layer should stay robust even when only partial enrichment is available.
-
-## Backend Design
-
-Recommended new service file:
-
-- [backend/app/services/bgp_regions.py](/home/ray/dev/linkong/planet/backend/app/services/bgp_regions.py)
-
-Suggested responsibilities:
-
-- `map_record_to_region(...)`
-- `aggregate_region_activity(...)`
-- `build_region_geojson(...)`
-- `resolve_activity_status(...)`
-- `compute_activity_score(...)`
-
-### Data Source Inputs
-
-Use a recent rolling window, default `15 minutes`, and aggregate from:
-
-- `BGPObservation`
-- `BGPAnomaly`
-- active `BGPIncident`
-
-### Aggregation Flow
-
-1. query observations in the time window
-2. query anomalies in the same window
-3. query active incidents in the same window or active status set
-4. resolve each record to a best-effort region
-5. accumulate per-region counters
-6. compute score and status
-7. return region activity list
-
-## Status Model
-
-Recommended status buckets:
-
-- `idle`
-- `observing`
-- `anomaly`
-- `incident`
-
-Suggested rule:
-
-```text
-if incident_count > 0: incident
-elif anomaly_count > 0: anomaly
-elif observation_count > 0: observing
-else: idle
-```
-
-This aligns well with the current Earth status language and keeps the visual mapping simple.
-
-## Activity Score
-
-The score should be a tunable heuristic, not a fixed truth model.
-
-Recommended v1 formula:
-
-```text
-activity_score =
-  min(observation_count, 50) * 0.03
-  + anomaly_count * 1.2
-  + incident_count * 5.0
-```
-
-Why cap observations:
-
-- observation volume is usually much larger than anomaly or incident volume
-- uncapped observation counts would overwhelm the score
-- capped observation counts preserve baseline presence without drowning real abnormality
-
-### Practical Guidance
-
-- treat coefficients as configuration-like constants
-- expect to retune after looking at real data
-- keep `incident` weight dominant
-
-## API Design
-
-### 1. Summary/List API
-
-Suggested endpoint:
-
-- `/api/v1/bgp/regions/activity`
-
-Response shape:
-
-```json
-{
-  "window_minutes": 15,
-  "regions": []
-}
-```
-
-Use cases:
-
-- BGP console summaries
-- right-side Earth stats
-- future region list panels
-
-### 2. GeoJSON API
-
-Suggested endpoint:
-
-- `/api/v1/visualization/geo/bgp-regions`
-
-Response shape:
-
-```json
-{
-  "type": "FeatureCollection",
-  "features": []
-}
-```
-
-Each feature should include:
-
-- `geometry`
-  - v1: `Point`
-  - later: optional `Polygon`
-- `properties`
-  - `region_key`
-  - `region_name`
-  - `status`
-  - `activity_score`
-  - `observation_count`
-  - `anomaly_count`
-  - `incident_count`
-  - `affected_prefix_count`
-  - `affected_asn_count`
-  - `collector_count`
-
-## Earth Rendering Plan
-
-Detailed visual layering guidance is expanded in [bgp-earth-rendering-plan.md](/home/ray/dev/linkong/planet/docs/bgp-earth-rendering-plan.md).
-
-### Layer Relationship
-
-- `region layer` = ambient background activity
-- `incident marker` = focal event object
-
-Do not replace incident markers with region markers.
-
-### Region Visual Rules
-
-Suggested mapping:
-
-- `observing`
-  - weak glow
-  - low pulse or no pulse
-- `anomaly`
-  - stronger glow
-  - more visible pulse
-- `incident`
-  - strongest regional emphasis
-  - but still visually secondary to the incident marker itself
-
-### Region Labels
-
-Good v2 enhancement:
-
-- show region name
-- show counts like `2 incidents / 5 anomalies`
-
-This is useful, but should come after the core aggregation and Earth glow layer are working.
-
-## Interaction Model
-
-### Click Region
-
-Recommended detail payload:
-
-- region name
-- observation/anomaly/incident counts in the selected window
-- affected prefix count
-- affected ASN count
-- collector count
-- recent incidents in the region
-
-### Click Incident
-
-Keep the current incident-detail flow.
-
-Interaction should feel hierarchical:
-
-1. region gives situational context
-2. incident gives event focus
-
-## MVP Implementation Order
-
-### Step 1
-
-Define static `REGIONS` in code or config.
-
-### Step 2
-
-Map geography-enriched BGP records into regions using the fallback chain.
-
-### Step 3
-
-Aggregate recent window counts:
-
-- `observation_count`
-- `anomaly_count`
-- `incident_count`
-
-### Step 4
-
-Compute `activity_score` and `status`.
-
-### Step 5
-
-Expose:
-
-- `/api/v1/bgp/regions/activity`
-- `/api/v1/visualization/geo/bgp-regions`
-
-### Step 6
-
-Render region glows on Earth behind incident markers.
-
-## Out Of Scope For MVP
-
-- persistent materialized region tables
-- geohash or H3 support
-- polygon-filled regional overlays
-- detailed top-prefix ranking in the first release
-- complicated scoring personalization
-
-## Risks And Constraints
-
-### Geography Quality
-
-Prefix geography is approximate and incomplete.
-The region layer must tolerate fallback-based placement.
-
-### Query Cost
-
-Dynamic aggregation is the right v1 choice, but repeated short-window queries may eventually need:
-
-- in-process caching
-- scheduled pre-aggregation
-- materialized summaries
-
-### UI Overcrowding
-
-If region glow, collector activity, and incidents all become too strong at once, Earth readability will regress.
-The region layer must remain supportive, not dominant.
-
-## Final Recommendation
-
-The current BGP roadmap should explicitly add:
-
-- `region aggregation` as the concrete implementation of the missing `activity layer`
-
-The recommended product interpretation is:
-
-- `collectors` prove observation coverage
-- `regions` communicate live routing activity and abnormality
-- `incidents` remain the clearest high-confidence event objects
-
-In one sentence:
-
-`region aggregation is not a replacement for incidents; it is the situational background that makes sparse incidents feel legible on Earth.`

docs/collected-data-column-removal-plan.md

@@ -1,207 +0,0 @@
-# collected_data 强耦合列拆除计划
-
-## 背景
-
-当前 `collected_data` 同时承担了两类职责：
-
-1. 通用采集事实表
-2. 少数数据源的宽表字段承载
-
-典型强耦合列包括：
-
-- `country`
-- `city`
-- `latitude`
-- `longitude`
-- `value`
-- `unit`
-
-以及 API 层临时平铺出来的：
-
-- `cores`
-- `rmax`
-- `rpeak`
-- `power`
-
-这些字段并不适合作为统一事实表的长期 schema。  
-推荐方向是：
-
-- 表内保留通用稳定字段
-- 业务差异字段全部归入 `metadata`
-- API 和前端动态读取 `metadata`
-
-## 拆除目标
-
-最终希望 `collected_data` 只保留：
-
-- `id`
-- `snapshot_id`
-- `task_id`
-- `source`
-- `source_id`
-- `entity_key`
-- `data_type`
-- `name`
-- `title`
-- `description`
-- `metadata`
-- `collected_at`
-- `reference_date`
-- `is_valid`
-- `is_current`
-- `previous_record_id`
-- `change_type`
-- `change_summary`
-- `deleted_at`
-
-## 计划阶段
-
-### Phase 1：读取层去依赖
-
-目标：
-
-- API / 可视化 / 前端不再优先依赖宽列表字段
-- 所有动态字段优先从 `metadata` 取
-
-当前已完成：
-
-- 新写入数据时，将 `country/city/latitude/longitude/value/unit` 自动镜像到 `metadata`
-- `/api/v1/collected` 优先从 `metadata` 取动态字段
-- `visualization` 接口优先从 `metadata` 取动态字段
-- 国家筛选已改成只走 `metadata->>'country'`
-- `CollectedData.to_dict()` 已切到 metadata-first
-- 变更比较逻辑已切到 metadata-first
-- 已新增历史回填脚本：
-  [scripts/backfill_collected_data_metadata.py](/home/ray/dev/linkong/planet/scripts/backfill_collected_data_metadata.py)
-- 已新增删列脚本：
-  [scripts/drop_collected_data_legacy_columns.py](/home/ray/dev/linkong/planet/scripts/drop_collected_data_legacy_columns.py)
-
-涉及文件：
-
-- [backend/app/core/collected_data_fields.py](/home/ray/dev/linkong/planet/backend/app/core/collected_data_fields.py)
-- [backend/app/services/collectors/base.py](/home/ray/dev/linkong/planet/backend/app/services/collectors/base.py)
-- [backend/app/api/v1/collected_data.py](/home/ray/dev/linkong/planet/backend/app/api/v1/collected_data.py)
-- [backend/app/api/v1/visualization.py](/home/ray/dev/linkong/planet/backend/app/api/v1/visualization.py)
-
-### Phase 2：写入层去依赖
-
-目标：
-
-- 采集器内部不再把这些字段当作数据库一级列来理解
-- 统一只写：
-  - 通用主字段
-  - `metadata`
-
-建议动作：
-
-1. Collector 内部仍可使用 `country/city/value` 这种临时字段作为采集过程变量
-2. 进入 `BaseCollector._save_data()` 后统一归档到 `metadata`
-3. `CollectedData` 模型中的强耦合列已从 ORM 移除，写入统一归档到 `metadata`
-
-### Phase 3：数据库删列
-
-目标：
-
-- 从 `collected_data` 真正移除以下列：
-  - `country`
-  - `city`
-  - `latitude`
-  - `longitude`
-  - `value`
-  - `unit`
-
-注意：
-
-- `cores / rmax / rpeak / power` 当前本来就在 `metadata` 里，不是表列
-- 这四个主要是 API 平铺字段，不需要数据库删列
-
-## 当前阻塞点
-
-在正式删列前，还需要确认这些地方已经完全不再直接依赖数据库列：
-
-### 1. `CollectedData.to_dict()`
-
-文件：
-
-- [backend/app/models/collected_data.py](/home/ray/dev/linkong/planet/backend/app/models/collected_data.py)
-
-状态：
-
-- 已完成
-
-### 2. 差异计算逻辑
-
-文件：
-
-- [backend/app/services/collectors/base.py](/home/ray/dev/linkong/planet/backend/app/services/collectors/base.py)
-
-状态：
-
-- 已完成
-- 当前已改成比较归一化后的 metadata-first payload
-
-### 3. 历史数据回填
-
-问题：
-
-- 老数据可能只有列值，没有对应 `metadata`
-
-当前方案：
-
-- 在删列前执行一次回填脚本：
-  - [scripts/backfill_collected_data_metadata.py](/home/ray/dev/linkong/planet/scripts/backfill_collected_data_metadata.py)
-
-### 4. 导出格式兼容
-
-文件：
-
-- [backend/app/api/v1/collected_data.py](/home/ray/dev/linkong/planet/backend/app/api/v1/collected_data.py)
-
-现状：
-
-- CSV/JSON 导出已基本切成 metadata-first
-
-建议：
-
-- 删列前再回归检查一次导出字段是否一致
-
-## 推荐执行顺序
-
-1. 保持新数据写入时 `metadata` 完整
-2. 把模型和 diff 逻辑完全切成 metadata-first
-3. 写一条历史回填脚本
-4. 回填后观察一轮
-5. 正式执行删列迁移
-
-## 推荐迁移 SQL
-
-仅在确认全部读取链路已去依赖后执行：
-
-```sql
-ALTER TABLE collected_data
-DROP COLUMN IF EXISTS country,
-DROP COLUMN IF EXISTS city,
-DROP COLUMN IF EXISTS latitude,
-DROP COLUMN IF EXISTS longitude,
-DROP COLUMN IF EXISTS value,
-DROP COLUMN IF EXISTS unit;
-```
-
-## 风险提示
-
-1. 地图类接口对经纬度最敏感
-   必须确保所有地图需要的记录，其 `metadata.latitude/longitude` 已回填完整。
-
-2. 历史老数据如果没有回填，删列后会直接丢失这些信息。
-
-3. 某些 collector 可能仍隐式依赖这些宽字段做差异比较，删列前必须做一次全量回归。
-
-## 当前判断
-
-当前项目已经完成“代码去依赖 + 历史回填 + readiness 检查”。  
-下一步执行顺序建议固定为：
-
-1. 先部署当前代码版本并重启后端
-2. 再做一轮功能回归
-3. 最后执行：
-   `uv run python scripts/drop_collected_data_legacy_columns.py`

docs/collected-data-history-plan.md

@@ -1,402 +0,0 @@
-# 采集数据历史快照化改造方案
-
-## 背景
-
-当前系统的 `collected_data` 更接近“当前结果表”：
-
-- 同一个 `source + source_id` 会被更新覆盖
-- 前端列表页默认读取这张表
-- `collection_tasks` 只记录任务执行状态，不直接承载数据版本语义
-
-这套方式适合管理后台，但不利于后续做态势感知、时间回放、趋势分析和版本对比。  
-如果后面需要回答下面这类问题，当前模型会比较吃力：
-
-- 某条实体在过去 7 天如何变化
-- 某次采集相比上次新增了什么、删除了什么、值变了什么
-- 某个时刻地图上“当时的世界状态”是什么
-- 告警是在第几次采集后触发的
-
-因此建议把采集数据改造成“历史快照 + 当前视图”模型。
-
-## 目标
-
-1. 每次触发采集都保留一份独立快照，历史可追溯。
-2. 管理后台默认仍然只看“当前最新状态”，不增加使用复杂度。
-3. 后续支持：
-   - 时间线回放
-   - 两次采集差异对比
-   - 趋势分析
-   - 按快照回溯告警和地图状态
-4. 尽量兼容现有接口，降低改造成本。
-
-## 结论
-
-不建议继续用以下两种单一模式：
-
-- 直接覆盖旧数据
-  问题：没有历史，无法回溯。
-
-- 软删除旧数据再全量新增
-  问题：语义不清，历史和“当前无效”混在一起，后续统计复杂。
-
-推荐方案：
-
-- 保留历史事实表
-- 维护当前视图
-- 每次采集对应一个明确的快照批次
-
-## 推荐数据模型
-
-### 方案概览
-
-建议拆成三层：
-
-1. `collection_tasks`
-   继续作为采集任务表，表示“这次采集任务”。
-
-2. `data_snapshots`
-   新增快照表，表示“某个数据源在某次任务中产出的一个快照批次”。
-
-3. `collected_data`
-   从“当前结果表”升级为“历史事实表”，每一行归属于一个快照。
-
-同时再提供一个“当前视图”：
-
-- SQL View / 物化视图 / API 查询层封装均可
-- 语义是“每个 `source + source_id` 的最新有效记录”
-
-### 新增表：`data_snapshots`
-
-建议字段：
-
-| 字段 | 类型 | 含义 |
-|---|---|---|
-| `id` | bigint PK | 快照主键 |
-| `datasource_id` | int | 对应数据源 |
-| `task_id` | int | 对应采集任务 |
-| `source` | varchar(100) | 数据源名，如 `top500` |
-| `snapshot_key` | varchar(100) | 可选，业务快照标识 |
-| `reference_date` | timestamptz nullable | 这批数据的参考时间 |
-| `started_at` | timestamptz | 快照开始时间 |
-| `completed_at` | timestamptz | 快照完成时间 |
-| `record_count` | int | 快照总记录数 |
-| `status` | varchar(20) | `running/success/failed/partial` |
-| `is_current` | bool | 当前是否是该数据源最新快照 |
-| `parent_snapshot_id` | bigint nullable | 上一版快照，可用于 diff |
-| `summary` | jsonb | 本次快照统计摘要 |
-
-说明：
-
-- `collection_tasks` 偏“执行过程”
-- `data_snapshots` 偏“数据版本”
-- 一个任务通常对应一个快照，但保留分层更清晰
-
-### 升级表：`collected_data`
-
-建议新增字段：
-
-| 字段 | 类型 | 含义 |
-|---|---|---|
-| `snapshot_id` | bigint not null | 归属快照 |
-| `task_id` | int nullable | 归属任务，便于追查 |
-| `entity_key` | varchar(255) | 实体稳定键，通常可由 `source + source_id` 派生 |
-| `is_current` | bool | 当前是否为该实体最新记录 |
-| `previous_record_id` | bigint nullable | 上一个版本的记录 |
-| `change_type` | varchar(20) | `created/updated/unchanged/deleted` |
-| `change_summary` | jsonb | 字段变化摘要 |
-| `deleted_at` | timestamptz nullable | 对应“本次快照中消失”的实体 |
-
-保留现有字段：
-
-- `source`
-- `source_id`
-- `data_type`
-- `name`
-- `title`
-- `description`
-- `country`
-- `city`
-- `latitude`
-- `longitude`
-- `value`
-- `unit`
-- `metadata`
-- `collected_at`
-- `reference_date`
-- `is_valid`
-
-### 当前视图
-
-建议新增一个只读视图：
-
-`current_collected_data`
-
-语义：
-
-- 对每个 `source + source_id` 只保留最新一条 `is_current = true` 且 `deleted_at is null` 的记录
-
-这样：
-
-- 管理后台继续像现在一样查“当前数据”
-- 历史分析查 `collected_data`
-
-## 写入策略
-
-### 触发按钮语义
-
-“触发”不再理解为“覆盖旧表”，而是：
-
-- 启动一次新的采集任务
-- 生成一个新的快照
-- 将本次结果写入历史事实表
-- 再更新当前视图标记
-
-### 写入流程
-
-1. 创建 `collection_tasks` 记录，状态 `running`
-2. 创建 `data_snapshots` 记录，状态 `running`
-3. 采集器拉取原始数据并标准化
-4. 为每条记录生成 `entity_key`
-   - 推荐：`{source}:{source_id}`
-5. 将本次记录批量写入 `collected_data`
-6. 与上一个快照做比对，计算：
-   - 新增
-   - 更新
-   - 未变
-   - 删除
-7. 更新本批记录的：
-   - `change_type`
-   - `previous_record_id`
-   - `is_current`
-8. 将上一批同实体记录的 `is_current` 置为 `false`
-9. 将本次快照未出现但上一版存在的实体标记为 `deleted`
-10. 更新 `data_snapshots.status = success`
-11. 更新 `collection_tasks.status = success`
-
-### 删除语义
-
-这里不建议真的删记录。  
-建议采用“逻辑消失”模型：
-
-- 历史行永远保留
-- 如果某实体在新快照里消失：
-  - 上一条历史记录补一条“删除状态记录”或标记 `change_type = deleted`
-  - 同时该实体不再出现在当前视图
-
-这样最适合态势感知。
-
-## API 改造建议
-
-### 保持现有接口默认行为
-
-现有接口：
-
-- `GET /api/v1/collected`
-- `GET /api/v1/collected/{id}`
-- `GET /api/v1/collected/summary`
-
-建议默认仍返回“当前视图”，避免前端全面重写。
-
-### 新增历史查询能力
-
-建议新增参数或新接口：
-
-#### 1. 当前/历史切换
-
-`GET /api/v1/collected?mode=current|history`
-
-- `current`：默认，查当前视图
-- `history`：查历史事实表
-
-#### 2. 按快照查询
-
-`GET /api/v1/collected?snapshot_id=123`
-
-#### 3. 快照列表
-
-`GET /api/v1/snapshots`
-
-支持筛选：
-
-- `datasource_id`
-- `source`
-- `status`
-- `date_from/date_to`
-
-#### 4. 快照详情
-
-`GET /api/v1/snapshots/{id}`
-
-返回：
-
-- 快照基础信息
-- 统计摘要
-- 与上一版的 diff 摘要
-
-#### 5. 快照 diff
-
-`GET /api/v1/snapshots/{id}/diff?base_snapshot_id=122`
-
-返回：
-
-- `created`
-- `updated`
-- `deleted`
-- `unchanged`
-
-## 前端改造建议
-
-### 1. 数据列表页
-
-默认仍看当前数据，不改用户使用习惯。
-
-建议新增：
-
-- “视图模式”
-  - 当前数据
-  - 历史数据
-- “快照时间”筛选
-- “只看变化项”筛选
-
-### 2. 数据详情页
-
-详情页建议展示：
-
-- 当前记录基础信息
-- 元数据动态字段
-- 所属快照
-- 上一版本对比入口
-- 历史版本时间线
-
-### 3. 数据源管理页
-
-“触发”按钮文案建议改成更准确的：
-
-- `立即采集`
-
-并在详情里补：
-
-- 最近一次快照时间
-- 最近一次快照记录数
-- 最近一次变化数
-
-## 迁移方案
-
-### Phase 1：兼容式落地
-
-目标：先保留当前页面可用。
-
-改动：
-
-1. 新增 `data_snapshots`
-2. 给 `collected_data` 增加：
-   - `snapshot_id`
-   - `task_id`
-   - `entity_key`
-   - `is_current`
-   - `previous_record_id`
-   - `change_type`
-   - `change_summary`
-   - `deleted_at`
-3. 现有数据全部补成一个“初始化快照”
-4. 现有 `/collected` 默认改查当前视图
-
-优点：
-
-- 前端几乎无感
-- 风险最小
-
-### Phase 2：启用差异计算
-
-目标：采集后可知道本次改了什么。
-
-改动：
-
-1. 写入时做新旧快照比对
-2. 写 `change_type`
-3. 生成快照摘要
-
-### Phase 3：前端态势感知能力
-
-目标：支持历史回放和趋势分析。
-
-改动：
-
-1. 快照时间线
-2. 版本 diff 页面
-3. 地图时间回放
-4. 告警和快照关联
-
-## 唯一性与索引建议
-
-### 建议保留的业务唯一性
-
-在“同一个快照内部”，建议唯一：
-
-- `(snapshot_id, source, source_id)`
-
-不要在整张历史表上强加：
-
-- `(source, source_id)` 唯一
-
-因为历史表本来就应该允许同一实体跨快照存在多条版本。
-
-### 建议索引
-
-- `idx_collected_data_snapshot_id`
-- `idx_collected_data_source_source_id`
-- `idx_collected_data_entity_key`
-- `idx_collected_data_is_current`
-- `idx_collected_data_reference_date`
-- `idx_snapshots_source_completed_at`
-
-## 风险点
-
-1. 存储量会明显增加
-   - 需要评估保留周期
-   - 可以考虑冷热分层
-
-2. 写入复杂度上升
-   - 需要批量 upsert / diff 逻辑
-
-3. 当前接口语义会从“表”变成“视图”
-   - 文档必须同步
-
-4. 某些采集器缺稳定 `source_id`
-   - 需要补齐实体稳定键策略
-
-## 对当前项目的具体建议
-
-结合当前代码，推荐这样落地：
-
-### 短期
-
-1. 先设计并落表：
-   - `data_snapshots`
-   - `collected_data` 新字段
-2. 采集完成后每次新增快照
-3. `/api/v1/collected` 默认查 `is_current = true`
-
-### 中期
-
-1. 在 `BaseCollector._save_data()` 中改成：
-   - 生成快照
-   - 批量写历史
-   - 标记当前
-2. 将 `CollectionTask.id` 关联到 `snapshot.task_id`
-
-### 长期
-
-1. 地图接口支持按 `snapshot_id` 查询
-2. 仪表盘支持“最近一次快照变化量”
-3. 告警支持绑定到快照版本
-
-## 最终建议
-
-最终建议采用：
-
-- 历史事实表：保存每次采集结果
-- 当前视图：服务管理后台默认查询
-- 快照表：承载版本批次和 diff 语义
-
-这样既能保留历史，又不会把当前页面全部推翻重做，是最适合后续做态势感知的一条路径。

docs/datasource-health-plan.md

@@ -1,486 +0,0 @@
-# Datasource Health Plan
-
-## Overview
-
-This document defines a phased plan for datasource health governance.
-
-The goal is to make collectors observable, diagnosable, and recoverable when upstream APIs change, while avoiding unsafe automatic mutation of repository defaults.
-
-The key principle is:
-
-- do not let runtime automation rewrite repository default config
-
-Instead, split responsibilities across:
-
-- default config
-- runtime overrides
-- health check records
-- agent-generated repair proposals
-
-
-## Problem Statement
-
-Collectors currently depend on third-party APIs, data downloads, mirrored JSON files, archive links, and web pages.
-
-These upstream dependencies can fail in several ways:
-
-- endpoint becomes unreachable
-- endpoint still responds but schema changes
-- content-type changes
-- website shuts down or moves
-- mirror link disappears
-- HTML structure changes and scraping fails
-- endpoint requires a new path or new host
-
-We want a system that can:
-
-- detect datasource health degradation early
-- identify likely cause
-- search for updated endpoints when reasonable
-- apply safe runtime fixes without polluting default repo config
-- preserve auditability and rollback
-
-
-## Design Principles
-
-1. Default config is stable
-
-- `backend/app/core/data_sources.yaml` remains the repository baseline.
-- It should be changed intentionally through normal development flow, not by autonomous runtime agents.
-
-2. Runtime fixes are isolated
-
-- Emergency or adaptive fixes should live in a runtime override layer.
-- Overrides should be reversible and auditable.
-
-3. Deterministic checks come first
-
-- Use normal programmatic health checks before using LLMs.
-- Only call an agent when deterministic checks indicate a meaningful failure.
-
-4. Agents suggest before they mutate
-
-- Agents should produce proposals with evidence and confidence.
-- Application of a proposal should be controlled by policy.
-
-5. Every repair is attributable
-
-- Store what changed, why, who or what suggested it, and when it was applied.
-
-
-## Configuration Layers
-
-Recommended runtime precedence:
-
-1. datasource endpoint override
-2. datasource DB endpoint override
-3. repository default YAML
-4. collector internal fallback logic
-
-Definitions:
-
-- repository default YAML:
-  - `backend/app/core/data_sources.yaml`
-  - versioned baseline
-- datasource DB endpoint override:
-  - existing `DataSourceConfig.endpoint`
-  - current runtime override entrypoint
-- datasource endpoint override:
-  - a dedicated new override table
-  - used for health-repair and proposal application
-- collector internal fallback logic:
-  - final defensive fallback
-  - should be minimized over time
-
-
-## Recommended Architecture
-
-### 1. Deterministic Health Checks
-
-Each collector gets a health profile with checks such as:
-
-- endpoint resolves
-- HTTP request succeeds
-- status code is acceptable
-- content-type is expected
-- body parses successfully
-- minimum structural fields exist
-- sample item count is plausible
-- latency is within threshold
-
-Output states:
-
-- `healthy`
-- `degraded`
-- `failed`
-- `schema_changed`
-- `rate_limited`
-- `auth_required`
-
-
-### 2. Agent-Assisted Repair Discovery
-
-Only triggered when deterministic health checks fail or return suspicious structure.
-
-Agent responsibilities:
-
-- search for current official endpoint or replacement path
-- inspect likely upstream documentation or landing pages
-- compare candidate endpoint output to collector expectations
-- produce a repair proposal with confidence and evidence
-
-Agent should not directly modify repository defaults.
-
-
-### 3. Safe Runtime Repair Application
-
-Repair proposals can be:
-
-- reviewed manually
-- auto-applied only under strict low-risk policy
-
-Auto-apply should be limited to cases like:
-
-- same trusted domain
-- highly similar response structure
-- repeated successful verification
-- confidence above threshold
-
-
-## Phased Delivery Plan
-
-## Phase 1: Deterministic Health MVP
-
-Goal:
-
-- build health observability without automated repair
-
-Scope:
-
-- datasource health check task runner
-- datasource health result persistence
-- endpoint reachability + parse checks
-- dashboard or API visibility into health status
-
-Deliverables:
-
-- health check service
-- health check record table
-- status endpoint
-- scheduled or manual check trigger
-
-No agent usage yet.
-
-
-## Phase 2: Agent Repair Proposals
-
-Goal:
-
-- let agent investigate failing sources and propose updated endpoints
-
-Scope:
-
-- invoke agent only when datasource health is `failed` or `schema_changed`
-- web search + page inspection
-- candidate endpoint extraction
-- proposal persistence
-
-Deliverables:
-
-- repair proposal schema
-- proposal generation pipeline
-- confidence and evidence model
-- operator review view or API
-
-Still no automatic config mutation.
-
-
-## Phase 3: Runtime Overrides
-
-Goal:
-
-- allow approved proposals to take effect safely at runtime
-
-Scope:
-
-- add dedicated override storage
-- runtime resolution prefers override over default config
-- proposal application writes override only
-
-Deliverables:
-
-- endpoint override table
-- override-aware resolution logic
-- apply/reject endpoints
-- rollback endpoint
-
-Repository default YAML remains untouched.
-
-
-## Phase 4: Limited Auto-Apply
-
-Goal:
-
-- safely automate a narrow slice of low-risk repairs
-
-Scope:
-
-- policy engine for auto-apply
-- same-domain or trusted-domain checks
-- structure validation
-- staged verification after apply
-
-Deliverables:
-
-- auto-apply rules
-- audit logs
-- automatic post-apply health verification
-- auto-disable or rollback on regression
-
-
-## Data Model Draft
-
-### datasource_health_checks
-
-Purpose:
-
-- store each health evaluation result
-
-Suggested fields:
-
-- `id`
-- `datasource_id`
-- `collector_name`
-- `endpoint_checked`
-- `status`
-- `http_status`
-- `content_type`
-- `latency_ms`
-- `sample_count`
-- `error_message`
-- `details`
-- `checked_at`
-
-`details` can store structured diagnostic data such as:
-
-- parsed fields
-- schema mismatch summary
-- retry count
-- exception class
-
-
-### datasource_repair_proposals
-
-Purpose:
-
-- store agent-generated repair suggestions
-
-Suggested fields:
-
-- `id`
-- `datasource_id`
-- `collector_name`
-- `old_endpoint`
-- `candidate_endpoint`
-- `reason`
-- `confidence`
-- `evidence_urls`
-- `evidence_summary`
-- `status`
-- `created_by`
-- `created_at`
-- `reviewed_at`
-
-Suggested `status` values:
-
-- `proposed`
-- `approved`
-- `rejected`
-- `applied`
-- `expired`
-
-
-### datasource_endpoint_overrides
-
-Purpose:
-
-- runtime endpoint override layer
-
-Suggested fields:
-
-- `id`
-- `datasource_id`
-- `collector_name`
-- `endpoint`
-- `reason`
-- `source`
-- `proposal_id`
-- `enabled`
-- `created_at`
-- `updated_at`
-
-Suggested `source` values:
-
-- `manual`
-- `health-agent`
-- `migration`
-
-
-## API Draft
-
-### Health
-
-- `GET /api/v1/datasources/health`
-- `GET /api/v1/datasources/{id}/health`
-- `POST /api/v1/datasources/{id}/health-check`
-- `POST /api/v1/datasources/health-check-all`
-
-### Repair proposals
-
-- `GET /api/v1/datasources/{id}/repair-proposals`
-- `POST /api/v1/datasources/{id}/repair-proposals/generate`
-- `POST /api/v1/datasources/{id}/repair-proposals/{proposal_id}/approve`
-- `POST /api/v1/datasources/{id}/repair-proposals/{proposal_id}/reject`
-- `POST /api/v1/datasources/{id}/repair-proposals/{proposal_id}/apply`
-
-### Overrides
-
-- `GET /api/v1/datasources/{id}/overrides`
-- `POST /api/v1/datasources/{id}/overrides`
-- `PUT /api/v1/datasources/{id}/overrides/{override_id}`
-- `DELETE /api/v1/datasources/{id}/overrides/{override_id}`
-
-
-## Agent Contract Draft
-
-When deterministic health fails, the agent should receive:
-
-- datasource name
-- collector name
-- current endpoint
-- current failure mode
-- expected response shape summary
-- known trusted domains
-
-Expected output:
-
-```json
-{
-  "status": "proposal",
-  "candidate_endpoint": "https://example.com/api/v2/data",
-  "confidence": 0.86,
-  "reason": "Official docs now point to v2 endpoint",
-  "evidence_urls": [
-    "https://example.com/docs/api",
-    "https://example.com/changelog"
-  ],
-  "notes": "Response shape appears compatible after light field remapping"
-}
-```
-
-The agent should never output "rewrite the default yaml" as its primary action.
-
-
-## Risk Analysis
-
-### Risk: wrong endpoint chosen by agent
-
-Mitigation:
-
-- use trusted-domain allowlists
-- require evidence URLs
-- require confidence threshold
-- add manual review for medium-risk sources
-
-
-### Risk: endpoint responds but schema silently changed
-
-Mitigation:
-
-- deterministic schema checks
-- parse and sample validation
-- content-type checks
-- collector-specific required fields
-
-
-### Risk: automatic runtime override causes hidden drift
-
-Mitigation:
-
-- store all overrides explicitly
-- mark source of override
-- keep default YAML unchanged
-- expose active overrides in API/UI
-
-
-### Risk: persistent bad override breaks data collection
-
-Mitigation:
-
-- allow rollback
-- keep parent/default endpoint visible
-- re-run verification after apply
-- auto-disable override on repeated failure
-
-
-## Operational Policy Recommendations
-
-1. Do not auto-apply for high-value or high-fragility sources initially.
-
-2. Use manual approval for:
-
-- scraped HTML sources
-- unofficial mirrors
-- sources with auth or rate-limit complexity
-- sources with legal or trust ambiguity
-
-3. Allow auto-apply only for:
-
-- same-domain version bumps
-- obvious official migration paths
-- repeated passing verification
-
-4. Expose health + proposal + override state together in one operator view.
-
-
-## Suggested Implementation Order
-
-1. Phase 1
-- health result table
-- deterministic checks
-- API and UI visibility
-
-2. Phase 2
-- proposal table
-- agent prompt/output contract
-- proposal generation job
-
-3. Phase 3
-- runtime override table
-- resolver precedence update
-- apply/reject endpoints
-
-4. Phase 4
-- auto-apply rules
-- rollback policy
-- operator automation
-
-
-## Out Of Scope For The First Iteration
-
-- direct automatic mutation of repository default YAML
-- automatic git commits by repair agents
-- unrestricted autonomous endpoint replacement
-- fully generalized schema remapping engine
-
-
-## Recommended First Milestone
-
-The first milestone should be:
-
-- deterministic datasource health checks
-- persisted results
-- manual visibility
-- no automatic repair
-
-This gives immediate operational value with low risk, and prepares clean inputs for the later agent phase.

docs/datasource-health-stage2-tasks.md

@@ -1,478 +0,0 @@
-# Datasource Health Stage 2 Tasks
-
-## Goal
-
-Stage 2 focuses on the first practical operational layer:
-
-- deterministic datasource health checks
-- persisted health results
-- health visibility through API and UI
-- no agent-assisted repair yet
-
-This stage should make Planet capable of answering:
-
-- which collectors are healthy
-- which collectors are degraded
-- which collectors are failing
-- why they are failing at a basic deterministic level
-
-
-## Scope
-
-Included:
-
-- datasource health data model
-- deterministic health check service
-- manual and scheduled health check triggers
-- health result APIs
-- frontend visibility
-
-Excluded:
-
-- LLM reasoning
-- web-search-based repair proposals
-- automatic endpoint rewriting
-- runtime override application
-
-
-## Delivery Target
-
-At the end of Stage 2, an operator should be able to:
-
-1. see health status for each collector
-2. trigger a health check manually
-3. inspect the latest failure reason
-4. inspect the last checked endpoint
-5. understand whether the problem is:
-   - unreachable
-   - auth-related
-   - rate-limit-related
-   - schema-related
-   - empty-data-related
-
-
-## Work Breakdown
-
-## A. Data Model
-
-### A1. Add datasource health record table
-
-Create a new model, for example:
-
-- `backend/app/models/datasource_health_check.py`
-
-Suggested fields:
-
-- `id`
-- `datasource_id`
-- `collector_name`
-- `endpoint_checked`
-- `status`
-- `http_status`
-- `content_type`
-- `latency_ms`
-- `sample_count`
-- `error_message`
-- `details`
-- `checked_at`
-
-Suggested status enum values:
-
-- `healthy`
-- `degraded`
-- `failed`
-- `schema_changed`
-- `rate_limited`
-- `auth_required`
-- `empty_result`
-
-
-### A2. Add datasource health summary fields
-
-Option A:
-
-- keep summary only in the health check table
-
-Option B:
-
-- also add summary fields on `data_sources`
-
-Recommended first step:
-
-- do not mutate `data_sources` schema yet
-- derive summary from the latest health record
-
-
-### A3. Migration task
-
-Add migration for the health table.
-
-Deliverables:
-
-- migration file
-- model registration
-
-
-## B. Health Check Engine
-
-### B1. Define health check service
-
-Add a new service module, for example:
-
-- `backend/app/services/datasource_health.py`
-
-Responsibilities:
-
-- resolve effective endpoint
-- execute deterministic check
-- classify result
-- persist health record
-
-
-### B2. Define shared result schema
-
-Create a typed result object, for example:
-
-- `HealthCheckResult`
-
-Suggested fields:
-
-- `status`
-- `endpoint_checked`
-- `http_status`
-- `content_type`
-- `latency_ms`
-- `sample_count`
-- `error_message`
-- `details`
-
-
-### B3. Implement base deterministic checks
-
-Every datasource should go through a minimal baseline check:
-
-1. resolve endpoint
-2. perform request
-3. measure latency
-4. inspect status code
-5. inspect content type
-6. inspect body shape
-
-Classification rules:
-
-- network error -> `failed`
-- HTTP 401/403 -> `auth_required`
-- HTTP 429 -> `rate_limited`
-- HTTP 404/410 -> `failed`
-- parse failure -> `schema_changed`
-- zero or suspiciously empty results -> `empty_result` or `degraded`
-- valid parse -> `healthy`
-
-
-### B4. Add collector-aware adapters
-
-Some collectors do not use the same fetch semantics.
-
-Add adapter profiles such as:
-
-- `http_json`
-- `http_csv`
-- `html_scrape`
-- `stream_probe`
-- `auth_session_http`
-
-Initial mapping suggestion:
-
-- `huggingface`, `peeringdb`, `cloudflare` -> `http_json`
-- `fao` -> `http_csv`
-- `top500`, `epoch_ai`, `telegeography live_map` -> `html_scrape`
-- `ris_live` -> `stream_probe`
-- `spacetrack` -> `auth_session_http`
-
-
-### B5. Add sample validation hooks
-
-For each adapter, add a lightweight validation rule.
-
-Examples:
-
-- JSON array length > 0
-- CSV rows > 1
-- HTML page contains expected table or script patterns
-- stream source yields at least one valid event within timeout
-
-
-## C. Persistence and Query Layer
-
-### C1. Save every check run
-
-Each health check should insert a record.
-
-Do not overwrite history in Stage 2.
-
-
-### C2. Add latest-health query helpers
-
-Add helper functions to fetch:
-
-- latest health record by datasource
-- latest failed health record
-- recent health history
-
-
-### C3. Optional retention policy
-
-For Stage 2, retention can be deferred.
-
-If desired, keep only:
-
-- last N records per datasource
-
-
-## D. API Layer
-
-### D1. Add health list endpoint
-
-Suggested endpoint:
-
-- `GET /api/v1/datasources/health`
-
-Returns:
-
-- datasource id
-- collector name
-- current endpoint
-- latest health status
-- last checked time
-- short reason
-
-
-### D2. Add per-datasource health detail endpoint
-
-Suggested endpoint:
-
-- `GET /api/v1/datasources/{id}/health`
-
-Returns:
-
-- latest record
-- recent history
-- detailed classification fields
-
-
-### D3. Add manual health trigger endpoint
-
-Suggested endpoint:
-
-- `POST /api/v1/datasources/{id}/health-check`
-
-Behavior:
-
-- run a health check now
-- persist the result
-- return the new record
-
-
-### D4. Add bulk health trigger endpoint
-
-Suggested endpoint:
-
-- `POST /api/v1/datasources/health-check-all`
-
-Behavior:
-
-- enqueue or run health checks for all active datasources
-
-
-## E. Scheduling
-
-### E1. Add health scheduler task
-
-Decide scheduling strategy.
-
-Recommended first version:
-
-- run collector jobs and health checks separately
-- health checks run on a lower frequency
-
-Suggested frequency:
-
-- every 6h or 12h for most datasources
-- optionally on-demand only in the very first cut
-
-
-### E2. Prevent health check collision with collection
-
-Rules:
-
-- health checks should not disrupt active collection
-- they should use light requests
-- if a collector is currently running, health check may:
-  - skip
-  - or use a lightweight endpoint probe only
-
-
-## F. Frontend
-
-### F1. Add health columns to datasource list
-
-Update:
-
-- `frontend/src/pages/DataSources/DataSources.tsx`
-
-Suggested new columns:
-
-- health status
-- last checked
-- reason summary
-
-
-### F2. Add manual health check action
-
-Per datasource:
-
-- button or dropdown action:
-  - `健康检查`
-
-
-### F3. Add health detail drawer or modal
-
-Show:
-
-- endpoint checked
-- status
-- HTTP status
-- content type
-- sample count
-- error message
-- last few results
-
-
-### F4. Add basic visual language
-
-Suggested colors:
-
-- green -> healthy
-- yellow -> degraded
-- orange -> rate-limited / auth-required
-- red -> failed / schema-changed
-
-
-## G. Observability
-
-### G1. Structured logging
-
-Every health check should log:
-
-- datasource id
-- collector name
-- endpoint
-- status
-- latency
-- failure class
-
-
-### G2. Optional metrics
-
-If metrics are added later, useful counters include:
-
-- health checks total
-- health checks failed
-- schema changes detected
-- rate limited checks
-
-
-## H. Tests
-
-### H1. Unit tests
-
-Add tests for:
-
-- status classification
-- content type classification
-- adapter behavior
-- latest-health query helpers
-
-
-### H2. API tests
-
-Add tests for:
-
-- health endpoints require auth
-- manual trigger endpoint works
-- list endpoint returns latest status
-
-
-### H3. Failure-path tests
-
-Add coverage for:
-
-- HTTP 404
-- HTTP 429
-- invalid JSON
-- empty response
-- parse mismatch
-
-
-## Suggested File Plan
-
-Possible implementation files:
-
-- `backend/app/models/datasource_health_check.py`
-- `backend/app/services/datasource_health.py`
-- `backend/app/schemas/datasource_health.py`
-- `backend/app/api/v1/datasource_health.py`
-- migration file under the project migration system
-
-Likely touched existing files:
-
-- `backend/app/api/main.py`
-- `frontend/src/pages/DataSources/DataSources.tsx`
-- `backend/tests/test_api.py`
-
-
-## Suggested Execution Order
-
-1. Add model and migration
-2. Add service and result schema
-3. Add deterministic adapters
-4. Add manual trigger API
-5. Add list/detail API
-6. Add frontend visibility
-7. Add scheduled checks
-8. Expand tests
-
-
-## Minimal First Milestone
-
-If we want the fastest useful slice, do this first:
-
-1. health table
-2. deterministic check service
-3. manual per-datasource health check API
-4. latest health list API
-5. frontend status badge column
-
-That is enough to start operating the system and will provide the input layer for Stage 3.
-
-
-## Dependency On Later Stages
-
-Stage 2 outputs become direct inputs for Stage 3.
-
-Specifically:
-
-- failed or schema-changed health records become agent triggers
-- health history becomes repair context
-- endpoint_checked becomes proposal baseline
-
-
-## Success Criteria
-
-Stage 2 is done when:
-
-- every active datasource can be health-checked deterministically
-- the latest health state is visible in API and UI
-- operators can manually trigger checks
-- failures are categorized into stable machine-readable statuses
-- no LLM is required for core health visibility

docs/deprecated/README.md

@@ -15,3 +15,8 @@
 - 明确写明“已完成”的计划，优先归档
 - 已被正式实现替代、继续放在 `docs/` 根目录会误导后续开发的计划，归档
 - 仍然指导未来开发、尚未完成或仍有明确执行价值的文档，继续保留在 `docs/`
+
+补充说明：
+
+- 一部分归档文档来自外部或临时工作流草案，例如 sisyphus 生成的初稿
+- 这类文档如果有可用内容，应先吸收到 `docs/plans/` 或 `docs/technical/`，再归档保留来源记录

docs/deprecated/sisyphus-earth-architecture-refactor.md

@@ -1,3 +1,5 @@
+> Archived note: this document was originally created by sisyphus and later reviewed against the main docs set. Useful content has been absorbed into `docs/plans/` where appropriate.
+
 # 地球3D可视化架构重构计划
 
 ## 背景

docs/deprecated/sisyphus-predicted-orbit.md

@@ -1,3 +1,5 @@
+> Archived note: this document was originally created by sisyphus and later reviewed against the main docs set. Useful content has been absorbed into `docs/plans/` where appropriate.
+
 # 卫星预测轨道显示功能
 
 ## TL;DR

docs/deprecated/sisyphus-ue5-client.md

@@ -1,3 +1,5 @@
+> Archived note: this document was originally created by sisyphus and later reviewed against the main docs set. Useful content has been absorbed into `docs/plans/` where appropriate.
+
 # UE5 3D 大屏客户端开发计划
 
 ## 项目概述

docs/deprecated/sisyphus-webgl-instancing-satellites.md

@@ -1,3 +1,5 @@
+> Archived note: this document was originally created by sisyphus and later reviewed against the main docs set. Useful content has been absorbed into `docs/plans/` where appropriate.
+
 # WebGL Instancing 卫星渲染优化计划
 
 ## 背景

docs/earth-module-plan.md

@@ -1,210 +0,0 @@
-# Earth 模块整治计划
-
-## 背景
-
-`planet` 前端中的 Earth 模块是当前最重要的大屏 3D 星球展示能力，但它仍以 legacy iframe 页面形式存在：
-
-- React 页面入口仅为 [Earth.tsx](/home/ray/dev/linkong/planet/frontend/src/pages/Earth/Earth.tsx)
-- 实际 3D 实现位于 [frontend/public/earth](/home/ray/dev/linkong/planet/frontend/public/earth)
-
-当前模块已经具备基础展示能力，但在生命周期、性能、可恢复性、可维护性方面存在明显隐患，不适合长期无人值守的大屏场景直接扩展。
-
-## 目标
-
-本计划的目标不是立刻重写 Earth，而是分阶段把它从“能跑的 legacy 展示页”提升到“可稳定运行、可持续演进的大屏核心模块”。
-
-核心目标：
-
-1. 先止血，解决资源泄漏、重载污染、假性卡顿等稳定性问题
-2. 再梳理数据加载、交互和渲染循环，降低性能风险
-3. 最后逐步从 iframe legacy 向可控模块化架构迁移
-
-## 现阶段主要问题
-
-### 1. 生命周期缺失
-
-- 没有统一 `destroy()` / 卸载清理逻辑
-- `requestAnimationFrame`
-- `window/document/dom listeners`
-- `THREE` geometry / material / texture
-- 运行时全局状态
-  都没有系统回收
-
-### 2. 数据重载不完整
-
-- `reloadData()` 没有彻底清理旧场景对象
-- cable、landing point、satellite 相关缓存与对象存在累积风险
-
-### 3. 渲染与命中检测成本高
-
-- 鼠标移动时频繁创建 `Raycaster` / `Vector2`
-- cable 命中前会重复做 bounding box 计算
-- 卫星每帧计算量偏高
-
-### 4. 状态管理分裂
-
-- 大量依赖 `window.*` 全局桥接
-- 模块之间靠隐式共享状态通信
-- React 外层无法有效感知 Earth 内部状态
-
-### 5. 错误恢复弱
-
-- 数据加载失败主要依赖 `console` 和轻提示
-- 缺少统一重试、降级、局部失败隔离机制
-
-## 分阶段计划
-
-## Phase 1：稳定性止血
-
-目标：
-
-- 不改视觉主形态
-- 优先解决泄漏、卡死、重载污染
-
-### 任务
-
-1. 补 Earth 生命周期管理
-
-- 为 [main.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/main.js) 增加：
-  - `init()`
-  - `destroy()`
-  - `reloadData()`
-  三类明确入口
-- 统一记录并释放：
-  - animation frame id
-  - interval / timeout
-  - DOM 事件监听
-  - `window` 暴露对象
-
-2. 增加场景对象清理层
-
-- 为 cable / landing point / satellite sprite / orbit line 提供统一清理函数
-- reload 前先 dispose 旧对象，再重新加载
-
-3. 增加 stale 状态恢复
-
-- 页面重新进入时，先清理上一次遗留选择态、hover 态、锁定态
-- 避免 iframe reload 后出现旧状态残留
-
-4. 加强失败提示
-
-- 电缆、登陆点、卫星加载拆分为独立状态
-- 某一类数据失败时，其它类型仍可继续显示
-- 提供明确的页面内提示而不是只打 console
-
-### 验收标准
-
-- 页面重复进入 / 离开后内存不持续上涨
-- 连续多次点“重新加载数据”后对象数量不异常增加
-- 单一数据源加载失败时页面不整体失效
-
-## Phase 2：性能优化
-
-目标：
-
-- 控制鼠标交互和动画循环成本
-- 提升大屏长时间运行的稳定帧率
-
-### 任务
-
-1. 复用交互对象
-
-- 复用 `Raycaster`、`Vector2`、中间 `Vector3`
-- 避免 `mousemove` 热路径中频繁 new 对象
-
-2. 优化 cable 命中逻辑
-
-- 提前缓存 cable 中心点 / bounding 数据
-- 移除 `mousemove` 内重复 `computeBoundingBox()`
-- 必要时增加分层命中：
-  - 先粗筛
-  - 再精确相交
-
-3. 改造动画循环
-
-- 使用真实 `deltaTime`
-- 把卫星位置更新、呼吸动画、视觉状态更新拆成独立阶段
-- 为不可见对象减少无意义更新
-
-4. 卫星轨迹与预测轨道优化
-
-- 评估轨迹更新频率
-- 对高开销几何计算增加缓存
-- 限制预测轨道生成频次
-
-### 验收标准
-
-- 鼠标移动时不明显掉帧
-- 中高数据量下动画速度不受帧率明显影响
-- 长时间运行 CPU/GPU 占用更平稳
-
-## Phase 3：架构收编
-
-目标：
-
-- 降低 legacy iframe 架构带来的维护成本
-- 让 React 主应用重新获得对 Earth 模块的控制力
-
-### 任务
-
-1. 抽离 Earth App Shell
-
-- 将数据加载、错误状态、控制面板状态抽到更明确的模块边界
-- 减少 `window.*` 全局依赖
-
-2. 规范模块通信
-
-- 统一 `main / controls / cables / satellites / ui` 的状态流
-- 明确只读配置、运行时状态、渲染对象的职责分层
-
-3. 评估去 iframe 迁移
-
-- 中期可以保留 public/legacy 资源目录
-- 但逐步把 Earth 作为前端内嵌模块而不是完全孤立页面
-
-### 验收标准
-
-- Earth 内部状态不再大量依赖全局变量
-- React 外层可以感知 Earth 加载状态和错误状态
-- 后续功能开发不再必须修改多个 legacy 文件才能完成
-
-## 优先级建议
-
-### P0
-
-- 生命周期清理
-- reload 清理
-- stale 状态恢复
-
-### P1
-
-- 命中检测优化
-- 动画 `deltaTime`
-- 数据加载失败隔离
-
-### P2
-
-- 全局状态收编
-- iframe 架构迁移
-
-## 推荐实施顺序
-
-1. 先做 Phase 1
-2. 再做交互热路径与动画循环优化
-3. 最后再考虑架构迁移
-
-## 风险提示
-
-1. Earth 是 legacy 模块，修复时容易牵一发而动全身
-2. 如果不先补清理逻辑，后续所有性能优化收益都会被泄漏问题吃掉
-3. 如果过早重写而不先止血，短期会影响现有演示稳定性
-
-## 当前建议
-
-最值得马上启动的是一个小范围稳定性 sprint：
-
-- 生命周期清理
-- reload 全量清理
-- 错误状态隔离
-
-这个阶段不追求“更炫”，先追求“更稳”。稳定下来之后，再进入性能和架构层的优化。

docs/earth-tv-live-module-plan.md

@@ -1,117 +0,0 @@
-# Earth 电视直播模块计划
-
-## 目标
-
-为 `Earth` 页面增加一个可配置、可扩展、可拖拽的电视直播模块：
-
-- 后台可配置新闻直播源
-- 默认兜底源为央视 `CCTV-4`
-- 未来可通过采集器接入世界各地新闻直播源
-- Earth 工具栏 `显示控制` 子菜单新增电视按钮
-- 点击后打开一个与其他 HUD 一致的可拖拽/可关闭窗口
-- 窗口内部可播放或承载新闻直播页面
-
-## 设计原则
-
-- 第一阶段先交付“后台可配 + Earth 可用 + 默认可回退”的版本
-- 公开读取接口与后台管理接口分离
-- 手工配置源与采集器源共用统一的前端消费结构
-- Earth 里的电视窗口必须复用现有 HUD 拖拽、关闭、布局最大化逻辑
-- 小屏下优先保证窗口完整显示，超出部分在窗口内部滚动
-
-## 分阶段实现
-
-### Phase 1：后端配置与公开读取
-
-- 在系统设置中新增 `tv` 分类
-- 定义直播源配置结构：
-  - `default_source_id`
-  - `auto_fallback`
-  - `sources[]`
-- 每个直播源至少包含：
-  - `id`
-  - `name`
-  - `provider`
-  - `region`
-  - `language`
-  - `source_type`
-  - `embed_url`
-  - `stream_url`
-  - `homepage_url`
-  - `is_enabled`
-  - `is_fallback`
-  - `sort_order`
-  - `collector_source`
-  - `notes`
-- 默认兜底源使用央视官网 `CCTV-4` 直播页
-- 新增公开读取接口，供 Earth 页面无登录态读取直播源配置
-
-### Phase 2：采集器扩展位
-
-- 新增 `news_live_streams` collector 占位
-- 规范采集器入库数据结构，使其能与后台手工配置源合并
-- TV 公开接口支持合并：
-  - 后台手工配置源
-  - 采集器入库源
-- 保持手工配置源优先级更高，避免采集器覆盖人工兜底配置
-
-### Phase 3：后台配置界面
-
-- 在系统配置页新增 `电视直播` tab
-- 支持：
-  - 查看当前默认源
-  - 开关自动回退
-  - 新增直播源
-  - 编辑直播源
-  - 删除直播源
-  - 启用/禁用直播源
-  - 将某个直播源设为默认源
-- 明确区分：
-  - 手工配置源
-  - 采集器来源
-
-### Phase 4：Earth HUD 集成
-
-- 在 `显示控制` 子菜单加入电视按钮
-- 新增 TV HUD 面板：
-  - 可拖拽
-  - 可关闭
-  - 支持显示/隐藏状态同步
-  - 参与布局最大化与恢复布局
-- 面板内容至少包含：
-  - 当前频道标题
-  - 源切换下拉菜单
-  - 刷新按钮
-  - 打开官网按钮
-  - 播放区域
-
-### Phase 5：播放策略
-
-- 第一版优先支持 `iframe`/嵌入页类直播源
-- 为未来扩展保留：
-  - `hls`
-  - `video`
-  - `external`
-- 如果默认源不可用：
-  - 优先回退到标记为 `is_fallback=true` 的源
-  - 若无明确回退源，则回退到第一个可用源
-- 面板内要有清晰的加载、错误、回退提示
-
-### Phase 6：打磨与清理
-
-- 统一 HUD 风格
-- 小屏下限制窗口尺寸并启用内部滚动
-- 避免窗口超出屏幕
-- 补最小验证
-- 清理临时代码、重复样式和无用资源
-
-## 首版交付定义
-
-当以下条件满足时，认为首版可用：
-
-- 后台可以配置新闻直播源
-- Earth 可以读取并显示默认直播源
-- 工具栏可打开电视窗口
-- 电视窗口可拖拽、可关闭
-- 央视 `CCTV-4` 作为默认兜底源可被使用
-- 代码结构已为后续采集器接入预留统一接口

docs/earth/bgp-context.md

@@ -1,355 +0,0 @@
-# BGP Context
-
-## Current Goal
-
-The BGP module is being evolved from an anomaly-only demo into a layered observability pipeline:
-
-`raw observations -> enrichment -> detectors -> incidents -> console/Earth visualization`
-
-The practical product goal is no longer just to "show incidents on the globe". The current product objective is:
-
-1. keep BGP visually present on Earth even when incident density is low
-2. make incidents clearly feel like a higher-confidence layer than anomalies
-3. show that the observation network is still active even when there are no active incidents
-
-In practice, that means Earth should behave like an observability surface, not only an incident map:
-
-- `collectors` show that observation is happening
-- `activity` shows where routing state is currently active or noisy
-- `incidents` become the highest-confidence focus layer
-
-## Current Backend Architecture
-
-### Data Layers
-
-1. `BGPObservation`
-   - File: `backend/app/models/bgp_observation.py`
-   - Purpose: store normalized raw routing observations from live/history sources.
-   - Typical fields:
-     - `source`
-     - `collector`
-     - `peer_asn`
-     - `peer_ip`
-     - `prefix`
-     - `event_type`
-     - `as_path`
-     - `origin_asn`
-     - `next_hop`
-     - `communities`
-     - `observed_at`
-     - `raw_payload`
-     - `collector_geo`
-     - `ingest_batch_id`
-
-2. `BGPAnomaly`
-   - File: `backend/app/models/bgp_anomaly.py`
-   - Purpose: hold atomic detector outputs.
-   - Current detector output types include:
-     - `origin_change`
-     - `more_specific_burst`
-     - `mass_withdrawal`
-
-3. `BGPIncident`
-   - File: `backend/app/models/bgp_incident.py`
-   - Purpose: aggregate atomic anomalies into incident-level objects for humans and the UI.
-
-### Pipeline
-
-Main flow is currently anchored in:
-
-- `backend/app/services/collectors/bgp_common.py`
-- `backend/app/services/bgp_enrichment.py`
-- `backend/app/services/bgp_detectors.py`
-- `backend/app/services/bgp_incidents.py`
-
-Operational flow:
-
-1. collectors fetch raw BGP data
-2. `normalize_bgp_event()` standardizes payloads
-3. observations are persisted to `bgp_observations`
-4. enrichment augments events with analysis context
-5. detectors create `bgp_anomalies`
-6. incident aggregation rolls anomalies up into `bgp_incidents`
-
-### Current Ingest Sources
-
-1. `RIPE RIS Live`
-   - Collector file: `backend/app/services/collectors/ris_live.py`
-   - Used for realtime observation flow.
-
-2. `CAIDA BGPStream Backfill`
-   - Collector file: `backend/app/services/collectors/bgpstream.py`
-   - Used as history/backfill entry point.
-
-## Current Enrichment Status
-
-Implemented enrichment skeleton in:
-
-- `backend/app/services/bgp_enrichment.py`
-
-Current enrichments:
-
-- prefix family / prefix length
-- supernet / more-specific derivation
-- deduplicated AS path
-- path prepending hints
-- collector region info
-- prefix baseline hints
-- new-origin detection
-- ASN organization profile from PeeringDB where available
-- prefix scope / impacted region hints
-- prefix geography source priority:
-  - `OpenGeoFeed` (override/high confidence)
-  - `IPtoASN` (country-range baseline)
-  - `NRO delegated stats` (registry-allocation fallback)
-
-Current limitation:
-
-- `RPKI` is still placeholder-only and returns `unknown`
-- no real ROA validation source is integrated yet
-- `inetnum` / `inet6num` whois fallback is still pending
-
-## Current API Surface
-
-Primary API file:
-
-- `backend/app/api/v1/bgp.py`
-
-Available endpoints:
-
-- `/api/v1/bgp/events`
-- `/api/v1/bgp/events/summary`
-- `/api/v1/bgp/events/{id}`
-- `/api/v1/bgp/anomalies`
-- `/api/v1/bgp/anomalies/summary`
-- `/api/v1/bgp/anomalies/{id}`
-- `/api/v1/bgp/incidents`
-- `/api/v1/bgp/incidents/summary`
-- `/api/v1/bgp/incidents/{id}`
-
-Visualization GeoJSON endpoints:
-
-- `backend/app/api/v1/visualization.py`
-- `/api/v1/visualization/geo/bgp-collectors`
-- `/api/v1/visualization/geo/bgp-anomalies`
-- `/api/v1/visualization/geo/bgp-incidents`
-
-## Current Earth Behavior
-
-Relevant files:
-
-- `frontend/public/earth/js/bgp.js`
-- `frontend/public/earth/js/main.js`
-- `frontend/public/earth/js/info-card.js`
-- `frontend/public/earth/js/constants.js`
-- `frontend/public/earth/index.html`
-
-Current design:
-
-1. Collectors are always shown when BGP is enabled.
-2. Incident markers are now the primary Earth BGP markers.
-3. If there are no incidents, Earth falls back to anomaly markers.
-4. If there are no anomalies either, collectors still provide presence.
-5. A dedicated `activity layer` now adds:
-   - per-collector recent 15-minute activity halos
-   - clustered regional activity hints derived from active collectors
-6. Incident markers now use:
-   - symbol-driven event cores
-   - outward ring pulses
-   - reduced diffuse glow compared with older Earth builds
-5. The right-side stats now show:
-   - BGP events
-   - collector count
-   - BGP status summary
-
-This is directionally correct, but still incomplete for low-event-density periods. Right now Earth can still feel too quiet when incidents are sparse because the system lacks a dedicated `activity layer` between raw observation and incident focus.
-
-Current BGP status strategy:
-
-- incidents present: show active incident count
-- no incidents but anomalies present: show active anomaly count, plus active observation regions when available
-- no incidents/anomalies but activity present: show `观测网络运行中`
-- no incidents/anomalies but collectors present: show `观测网络运行中 · 当前未发现聚合级事件`
-- no BGP data at all: show `暂无观测数据`
-
-Earth info-card strategy:
-
-- `bgp` card is now incident-centric in wording
-- `bgp_collector` card shows collector location and current event count
-
-## Current Product Gap
-
-The main product gap is not architecture correctness. It is low-density visualization strategy.
-
-Current reality:
-
-- incident count is naturally much lower than anomaly count
-- that is expected, because incidents are aggregated and de-noised
-- but incident-first rendering makes the Earth view look too quiet unless there is another always-available activity layer
-
-Implementation detail for the recommended `activity layer` is expanded in [bgp-region-aggregation-plan.md](/home/ray/dev/linkong/planet/docs/earth/bgp-region-aggregation-plan.md).
-
-So the immediate next milestone is:
-
-`event map -> observability map`
-
-That means Earth needs three simultaneously readable layers:
-
-1. `observation layer`
-   - collectors
-   - recent collector activity
-   - baseline coverage
-2. `activity layer`
-   - recent event density
-   - anomaly/noise hotspots
-   - regional activity scoring
-   - incident presence bonus
-3. `incident layer`
-   - sparse but highly legible, high-confidence event objects
-   - symbol-driven markers
-   - outward ring pulse instead of broad diffuse glow
-
-## Incident Visual Direction
-
-The Earth `incident` layer should not read like a large glowing patch. It should read like a compact, high-confidence event focus.
-
-Design principles:
-
-1. `incident` markers should use a strong primary symbol
-   - the symbol shape should carry type meaning where possible
-   - examples:
-     - `origin_change`: triangle-like warning marker
-     - `mass_withdrawal`: alert/exclamation-style marker
-     - `more_specific_burst`: split/radiating marker
-
-2. emphasis should come from outward ring pulses, not area flooding
-   - use a compact hot core
-   - use one or more expanding ring pulses
-   - avoid broad luminous blobs that make the event center feel vague
-
-3. `collector` and `incident` must stay visually distinct
-   - collectors are observation infrastructure
-   - incidents are extracted event focus
-   - collector activity should stay quieter than incident pulse language
-
-4. calm periods still need observability presence
-   - collectors and activity layers should keep the map alive
-   - once incidents appear, they should clearly dominate nearby BGP visuals
-
-5. incident geography should become `prefix-centric`
-   - collectors should remain evidence sources, not the primary event location
-   - preferred geography priority:
-     - `prefix_geography`
-     - `prefix_scope`
-     - `ASN organization region`
-     - `collector centroid` as final fallback
-   - `prefix_scope` should remain an observation-derived scope hint
-   - a new `prefix_geography` layer should be introduced for actual prefix-centric placement
-
-Reference inspiration:
-
-- `World Monitor`
-  - sparse event symbols
-  - compact centers
-  - ring-like outward pulses
-  - stronger incident legibility than diffuse glow
-
-## Current Console Behavior
-
-Relevant page:
-
-- `frontend/src/pages/BGP/BGP.tsx`
-
-Current BGP console page has three levels:
-
-1. observation summary
-   - total events
-   - collector count
-   - prefix count
-
-2. incident summary and incident table
-
-3. anomaly detail table plus recent observation events
-
-This means the BGP page still has useful signal even when there are zero anomalies.
-
-## Known Product/Engineering Boundaries
-
-1. The current system is still closer to an event board than a full BGP sensing platform.
-2. RIS coverage still needs to expand beyond narrow subscription scope.
-3. BGPStream history is still not full MRT-to-prefix decoded analytics.
-4. Collector geography still depends heavily on static RIPE RIS mappings.
-5. Incident-to-cable/IXP/region association is still weak and early-stage.
-6. Earth currently visualizes logical observation/impact structure, not true physical traffic paths.
-
-## Test Status
-
-BGP-specific tests live in:
-
-- `backend/tests/test_bgp.py`
-
-Verified status at this point:
-
-- `25 passed` for `backend/tests/test_bgp.py`
-- `62 passed` for `backend/tests`
-
-Covered areas include:
-
-- normalization
-- observation serialization
-- enrichment
-- detectors, including route leak candidate and path flap
-- incident aggregation
-- batch anomaly creation
-- BGP events/incidents API
-- summary endpoints
-
-## Most Relevant Files
-
-Backend:
-
-- `backend/app/models/bgp_observation.py`
-- `backend/app/models/bgp_anomaly.py`
-- `backend/app/models/bgp_incident.py`
-- `backend/app/services/collectors/bgp_common.py`
-- `backend/app/services/bgp_enrichment.py`
-- `backend/app/services/bgp_detectors.py`
-- `backend/app/services/bgp_incidents.py`
-- `backend/app/api/v1/bgp.py`
-- `backend/app/api/v1/visualization.py`
-
-Frontend:
-
-- `frontend/src/pages/BGP/BGP.tsx`
-- `frontend/public/earth/js/bgp.js`
-- `frontend/public/earth/js/main.js`
-- `frontend/public/earth/js/info-card.js`
-- `frontend/public/earth/js/constants.js`
-- `frontend/public/earth/index.html`
-
-## Recommended Next Steps
-
-### Next Backend / Detection Priority
-
-1. Integrate real RPKI validation data.
-2. Expand realtime collector coverage and include withdrawals more broadly.
-3. Continue refining route leak and path instability detectors with stronger heuristics.
-
-### Next Correlation / Storytelling Priority
-
-4. Strengthen incident aggregation semantics and titles.
-5. Add weak correlation from incidents to:
-   - cable corridors
-   - landing points
-   - IXPs
-   - other traffic anomaly sources
-6. Refine Earth hover/click handoff between collectors and incidents.
-
-### Next Visualization Priority
-
-7. Refine regional activity scoring so the activity layer is informative without becoming noisy.
-8. Add more incident symbol types as new detectors land.
-9. Add a real prefix geography source:
-   - `IPtoASN / IPtoCountry` as the first practical dataset
-   - `OpenGeoFeed` as a higher-quality override layer
-   - registry/whois only as fallback

docs/earth/bgp-earth-rendering-plan.md

@@ -1,296 +0,0 @@
-# BGP Earth Rendering Plan
-
-## Goal
-
-This document defines how the BGP `region activity layer` and `incident layer` should coexist on Earth without conflicting.
-
-The main question it answers is:
-
-- how to add a regional observability background layer
-- without weakening the current incident-first event focus
-
-## Core Principle
-
-The Earth design should follow a strict semantic hierarchy:
-
-- `collector layer` = observation infrastructure
-- `region activity layer` = background situational awareness
-- `incident layer` = focal high-confidence event objects
-
-In short:
-
-- collectors prove the network is observing
-- regions show where routing behavior is active or abnormal
-- incidents show the concrete event worth clicking
-
-Region aggregation is therefore not a replacement for incident rendering.
-It is the context layer that makes sparse incident markers legible.
-
-## Rendering Hierarchy
-
-Recommended visual stack order:
-
-1. collector network / collector halos
-2. region activity glow
-3. incident markers and incident pulses
-
-This ordering should always hold.
-
-Why:
-
-- collectors should stay visible but quiet
-- regions should create ambient activity presence
-- incidents must remain the first thing users notice as a concrete event
-
-## Role Separation
-
-### Region Layer
-
-The region layer answers:
-
-- where is routing activity building up
-- where is there current noise or instability
-- which part of the world is currently worth looking at
-
-The region layer should feel:
-
-- broad
-- ambient
-- low-frequency
-- contextual
-
-### Incident Layer
-
-The incident layer answers:
-
-- which exact event should the user inspect
-- where is the highest-confidence routing event located right now
-
-The incident layer should feel:
-
-- sharp
-- compact
-- high-contrast
-- intentionally clickable
-
-## Non-Conflict Rules
-
-To avoid visual and semantic conflict, these implementation rules should be treated as hard constraints:
-
-1. region markers must not use the same symbol language as incidents
-2. region emphasis must stay weaker than incident emphasis
-3. region animation frequency must stay lower than incident animation frequency
-4. incident markers must always render above region glows
-5. region layer should support the event, not compete with it
-
-If a user notices the region layer first but misses the incident marker, the region layer is too strong.
-
-If a user only sees isolated incident points and cannot feel broader activity context, the region layer is too weak.
-
-## Region Rendering Rules
-
-The region layer should not be rendered as a second kind of incident point.
-
-Recommended representation:
-
-- diffuse glow
-- halo
-- low-detail pulse
-- soft center, not a sharp icon
-
-### Status Mapping
-
-#### `observing`
-
-- weak glow
-- cool color, such as cyan or blue
-- little to no pulse
-- purpose: keep the globe alive during calm periods
-
-#### `anomaly`
-
-- stronger glow
-- warmer color, such as amber
-- gentle breathing or low-frequency pulse
-- purpose: show that a region is experiencing abnormal routing noise
-
-#### `incident`
-
-- strongest regional background emphasis
-- still clearly weaker than the incident marker itself
-- purpose: lift the surrounding area so the focal event does not feel isolated
-
-### Region Visual Characteristics
-
-Recommended properties:
-
-- large radius
-- low opacity
-- soft edge
-- low-contrast outline or no outline
-- low pulse amplitude
-
-Avoid:
-
-- sharp symbol shapes
-- strong icon silhouettes
-- bright hard-edged centers
-- incident-like pulse language
-
-## Incident Rendering Rules
-
-The incident layer should remain visually sharper and more explicit than region activity.
-
-Recommended qualities:
-
-- clear event symbol
-- compact hot core
-- one or two outward ring pulses
-- high contrast
-- clear click target
-
-The incident layer should read as:
-
-- focal
-- deliberate
-- high-confidence
-
-while the region layer should read as:
-
-- contextual
-- ambient
-- supporting
-
-## Region And Incident In The Same Area
-
-When a region contains one or more incidents:
-
-- the region glow may intensify
-- but the incident marker must remain the dominant local feature
-
-Interpretation should be:
-
-- `region` says this area is in an event state
-- `incident marker` says this is the concrete event object
-
-So a region with `incident` status is not itself the event marker.
-It is the background state around the event.
-
-## Interaction Model
-
-Interaction should also preserve hierarchy.
-
-### Click Region
-
-Open a regional situation view, such as:
-
-- region name
-- observation count
-- anomaly count
-- incident count
-- affected prefix count
-- affected ASN count
-- recent incidents in the region
-
-### Click Incident
-
-Keep the current incident-focused detail interaction.
-
-This creates a natural two-step flow:
-
-1. region gives context
-2. incident gives detail
-
-## Layer Relationship To Existing BGP Elements
-
-### Collector Layer
-
-Collectors should remain:
-
-- quieter than regions
-- more infrastructural than semantic
-- proof of coverage, not proof of incident
-
-### Region Layer
-
-Regions should become:
-
-- the main ambient activity layer
-- the bridge between collectors and incidents
-- the answer to low-density map quietness
-
-### Incident Layer
-
-Incidents should remain:
-
-- the most legible event layer
-- sparse but dominant
-- compact and symbol-driven
-
-## Practical Visual Test
-
-Use this test when tuning the Earth implementation:
-
-### Calm Period
-
-Expected result:
-
-- collectors visible
-- some weak region glows present
-- no region feels alarm-heavy
-- globe still feels alive
-
-### Anomaly Period
-
-Expected result:
-
-- one or more regions brighten noticeably
-- user can sense the active area before clicking
-- still no confusion between region background and incident objects
-
-### Incident Period
-
-Expected result:
-
-- region provides broader context
-- incident marker is the first explicit focal object the eye lands on
-- user can immediately tell both:
-  - which region is active
-  - which specific event to inspect
-
-## Failure Modes To Avoid
-
-### Region Too Strong
-
-Symptoms:
-
-- incident markers disappear into the glow
-- users treat the region center as the main event
-- the map feels like area flooding instead of event focus
-
-### Region Too Weak
-
-Symptoms:
-
-- incident markers still feel isolated
-- low-incident periods still look visually empty
-- users cannot tell where routing activity is generally happening
-
-### Region Uses Incident Language
-
-Symptoms:
-
-- region and incident both look like event markers
-- users cannot distinguish context from event
-
-## Final Design Rule
-
-The desired reading order is:
-
-1. see the specific incident marker
-2. feel the active region around it
-3. understand that collectors and background activity keep the globe alive even during quieter periods
-
-In one sentence:
-
-`incident is the point; region is the field.`

docs/earth/bgp-observability-plan.md

@@ -1,487 +0,0 @@
-# BGP Observability Plan
-
-## Goal
-
-Build a global routing observability capability on top of:
-
-- [RIPE RIS Live](https://ris-live.ripe.net/)
-- [CAIDA BGPStream data access overview](https://bgpstream.caida.org/docs/overview/data-access)
-
-The target is to support:
-
-- real-time routing event ingestion
-- historical replay and baseline analysis
-- anomaly detection
-- Earth big-screen visualization
-
-## Important Scope Note
-
-These data sources expose the BGP control plane, not user traffic itself.
-
-That means the system can infer:
-
-- route propagation direction
-- prefix reachability changes
-- AS path changes
-- visibility changes across collectors
-
-But it cannot directly measure:
-
-- exact application traffic volume
-- exact user packet path
-- real bandwidth consumption between countries or operators
-
-Product wording should therefore use phrases like:
-
-- global routing propagation
-- route visibility
-- control-plane anomalies
-- suspected path diversion
-
-Instead of claiming direct traffic measurement.
-
-## Data Source Roles
-
-### RIS Live
-
-Use RIS Live as the real-time feed.
-
-Recommended usage:
-
-- subscribe to update streams over WebSocket
-- ingest announcements and withdrawals continuously
-- trigger low-latency alerts
-
-Best suited for:
-
-- hijack suspicion
-- withdrawal bursts
-- real-time path changes
-- live Earth event overlay
-
-### BGPStream
-
-Use BGPStream as the historical and replay layer.
-
-Recommended usage:
-
-- backfill time windows
-- build normal baselines
-- compare current events against history
-- support investigations and playback
-
-Best suited for:
-
-- historical anomaly confirmation
-- baseline path frequency
-- visibility baselines
-- postmortem analysis
-
-## Recommended Architecture
-
-```mermaid
-flowchart LR
-  A["RIS Live WebSocket"] --> B["Realtime Collector"]
-  C["BGPStream Historical Access"] --> D["Backfill Collector"]
-  B --> E["Normalization Layer"]
-  D --> E
-  E --> F["data_snapshots"]
-  E --> G["collected_data"]
-  E --> H["bgp_anomalies"]
-  H --> I["Alerts API"]
-  G --> J["Visualization API"]
-  H --> J
-  J --> K["Earth Big Screen"]
-```
-
-## Storage Design
-
-The current project already has:
-
-- [data_snapshot.py](/home/ray/dev/linkong/planet/backend/app/models/data_snapshot.py)
-- [collected_data.py](/home/ray/dev/linkong/planet/backend/app/models/collected_data.py)
-
-So the lowest-risk path is:
-
-1. keep raw and normalized BGP events in `collected_data`
-2. use `data_snapshots` to group each ingest window
-3. add a dedicated anomaly table for higher-value derived events
-
-## Proposed Data Types
-
-### `collected_data`
-
-Use these `source` values:
-
-- `ris_live_bgp`
-- `bgpstream_bgp`
-
-Use these `data_type` values:
-
-- `bgp_update`
-- `bgp_rib`
-- `bgp_visibility`
-- `bgp_path_change`
-
-Recommended stable fields:
-
-- `source`
-- `source_id`
-- `entity_key`
-- `data_type`
-- `name`
-- `reference_date`
-- `metadata`
-
-Recommended `entity_key` strategy:
-
-- event entity: `collector|peer|prefix|event_time`
-- prefix state entity: `collector|peer|prefix`
-- origin state entity: `prefix|origin_asn`
-
-### `metadata` schema for raw events
-
-Store the normalized event payload in `metadata`:
-
-```json
-{
-  "project": "ris-live",
-  "collector": "rrc00",
-  "peer_asn": 3333,
-  "peer_ip": "2001:db8::1",
-  "event_type": "announcement",
-  "prefix": "203.0.113.0/24",
-  "origin_asn": 64496,
-  "as_path": [3333, 64500, 64496],
-  "communities": ["3333:100", "64500:1"],
-  "next_hop": "192.0.2.1",
-  "med": 0,
-  "local_pref": null,
-  "timestamp": "2026-03-26T08:00:00Z",
-  "raw_message": {}
-}
-```
-
-### New anomaly table
-
-Add a new table, recommended name: `bgp_anomalies`
-
-Suggested columns:
-
-- `id`
-- `snapshot_id`
-- `task_id`
-- `source`
-- `anomaly_type`
-- `severity`
-- `status`
-- `entity_key`
-- `prefix`
-- `origin_asn`
-- `new_origin_asn`
-- `peer_scope`
-- `started_at`
-- `ended_at`
-- `confidence`
-- `summary`
-- `evidence`
-- `created_at`
-
-This table should represent derived intelligence, not raw updates.
-
-## Collector Design
-
-## 1. `RISLiveCollector`
-
-Responsibility:
-
-- maintain WebSocket connection
-- subscribe to relevant message types
-- normalize messages
-- write event batches into snapshots
-- optionally emit derived anomalies in near real time
-
-Suggested runtime mode:
-
-- long-running background task
-
-Suggested snapshot strategy:
-
-- one snapshot per rolling time window
-- for example every 1 minute or every 5 minutes
-
-## 2. `BGPStreamBackfillCollector`
-
-Responsibility:
-
-- fetch historical data windows
-- normalize to the same schema as real-time data
-- build baselines
-- re-run anomaly rules on past windows if needed
-
-Suggested runtime mode:
-
-- scheduled task
-- or ad hoc task for investigations
-
-Suggested snapshot strategy:
-
-- one snapshot per historical query window
-
-## Normalization Rules
-
-Normalize both sources into the same internal event model.
-
-Required normalized fields:
-
-- `collector`
-- `peer_asn`
-- `peer_ip`
-- `event_type`
-- `prefix`
-- `origin_asn`
-- `as_path`
-- `timestamp`
-
-Derived normalized fields:
-
-- `as_path_length`
-- `country_guess`
-- `prefix_length`
-- `is_more_specific`
-- `visibility_weight`
-
-## Anomaly Detection Rules
-
-Start with these five rules first.
-
-### 1. Origin ASN Change
-
-Trigger when:
-
-- the same prefix is announced by a new origin ASN not seen in the baseline window
-
-Use for:
-
-- hijack suspicion
-- origin drift detection
-
-### 2. More-Specific Burst
-
-Trigger when:
-
-- a more-specific prefix appears suddenly
-- especially from an unexpected origin ASN
-
-Use for:
-
-- subprefix hijack suspicion
-
-### 3. Mass Withdrawal
-
-Trigger when:
-
-- the same prefix or ASN sees many withdrawals across collectors within a short window
-
-Use for:
-
-- outage suspicion
-- regional incident detection
-
-### 4. Path Deviation
-
-Trigger when:
-
-- AS path length jumps sharply
-- or a rarely seen transit ASN appears
-- or path frequency drops below baseline norms
-
-Use for:
-
-- route leak suspicion
-- unusual path diversion
-
-### 5. Visibility Drop
-
-Trigger when:
-
-- a prefix is visible from far fewer collectors/peers than its baseline
-
-Use for:
-
-- regional reachability degradation
-
-## Baseline Strategy
-
-Use BGPStream historical data to build:
-
-- common origin ASN per prefix
-- common AS path patterns
-- collector visibility distribution
-- normal withdrawal frequency
-
-Recommended baseline windows:
-
-- short baseline: last 24 hours
-- medium baseline: last 7 days
-- long baseline: last 30 days
-
-The first implementation can start with only the 7-day baseline.
-
-## API Design
-
-### Raw event API
-
-Add endpoints like:
-
-- `GET /api/v1/bgp/events`
-- `GET /api/v1/bgp/events/{id}`
-
-Suggested filters:
-
-- `prefix`
-- `origin_asn`
-- `peer_asn`
-- `collector`
-- `event_type`
-- `time_from`
-- `time_to`
-- `source`
-
-### Anomaly API
-
-Add endpoints like:
-
-- `GET /api/v1/bgp/anomalies`
-- `GET /api/v1/bgp/anomalies/{id}`
-- `GET /api/v1/bgp/anomalies/summary`
-
-Suggested filters:
-
-- `severity`
-- `anomaly_type`
-- `status`
-- `prefix`
-- `origin_asn`
-- `time_from`
-- `time_to`
-
-### Visualization API
-
-Add an Earth-oriented endpoint like:
-
-- `GET /api/v1/visualization/geo/bgp-anomalies`
-
-Recommended feature shapes:
-
-- point: collector locations
-- arc: inferred propagation or suspicious path edge
-- pulse point: active anomaly hotspot
-
-## Earth Big-Screen Design
-
-Recommended layers:
-
-### Layer 1: Collector layer
-
-Show known collector locations and current activity intensity.
-
-### Layer 2: Route propagation arcs
-
-Use arcs for:
-
-- origin ASN country to collector country
-- or collector-to-collector visibility edges
-
-Important note:
-
-This is an inferred propagation view, not real packet flow.
-
-### Layer 3: Active anomaly overlay
-
-Show:
-
-- hijack suspicion in red
-- mass withdrawal in orange
-- visibility drop in yellow
-- path deviation in blue
-
-### Layer 4: Time playback
-
-Use `data_snapshots` to replay:
-
-- minute-by-minute route changes
-- anomaly expansion
-- recovery timeline
-
-## Alerting Strategy
-
-Map anomaly severity to the current alert system.
-
-Recommended severity mapping:
-
-- `critical`
-  - likely hijack
-  - very large withdrawal burst
-- `high`
-  - clear origin change
-  - large visibility drop
-- `medium`
-  - unusual path change
-  - moderate more-specific burst
-- `low`
-  - weak or localized anomalies
-
-## Delivery Plan
-
-### Phase 1
-
-- add `RISLiveCollector`
-- normalize updates into `collected_data`
-- create `bgp_anomalies`
-- implement 3 rules:
-  - origin change
-  - more-specific burst
-  - mass withdrawal
-
-### Phase 2
-
-- add `BGPStreamBackfillCollector`
-- build 7-day baseline
-- implement:
-  - path deviation
-  - visibility drop
-
-### Phase 3
-
-- add Earth visualization layer
-- add time playback
-- add anomaly filtering and drilldown
-
-## Practical Implementation Notes
-
-- Start with IPv4 first, then add IPv6 after the event schema is stable.
-- Store the original raw payload in `metadata.raw_message` for traceability.
-- Deduplicate events by a stable hash of collector, peer, prefix, type, and timestamp.
-- Keep anomaly generation idempotent so replay and backfill do not create duplicate alerts.
-- Expect noisy data and partial views; confidence scoring matters.
-
-## Recommended First Patch Set
-
-The first code milestone should include:
-
-1. `backend/app/services/collectors/ris_live.py`
-2. `backend/app/services/collectors/bgpstream.py`
-3. `backend/app/models/bgp_anomaly.py`
-4. `backend/app/api/v1/bgp.py`
-5. `backend/app/api/v1/visualization.py`
-   add BGP anomaly geo endpoint
-6. `frontend/src/pages`
-   add a BGP anomaly list or summary page
-7. `frontend/public/earth/js`
-   add BGP anomaly rendering layer
-
-## Sources
-
-- [RIPE RIS Live](https://ris-live.ripe.net/)
-- [CAIDA BGPStream Data Access Overview](https://bgpstream.caida.org/docs/overview/data-access)

docs/earth/news-live-streams-collector-format.md

@@ -1,97 +0,0 @@
-# News Live Streams Collector Format
-
-`news_live_streams` 采集器面向“频道目录 JSON”输入，而不是直接抓网页。
-
-这样做的目标是：
-
-- 让后台能够稳定接入世界各地新闻直播源
-- 让 `Earth` 页面电视模块始终消费统一结构
-- 便于后续接入类似 `worldmonitor` 那种 YouTube / HLS / iframe 混合频道目录
-
-## 推荐 JSON 结构
-
-```json
-{
-  "sources": [
-    {
-      "id": "bbc-world-news",
-      "name": "BBC World News",
-      "provider": "BBC",
-      "region": "UK",
-      "language": "en",
-      "source_type": "youtube",
-      "youtube_video_id": "dQw4w9WgXcQ",
-      "youtube_channel": "https://www.youtube.com/@BBCNews",
-      "embed_url": "",
-      "stream_url": "",
-      "homepage_url": "https://www.youtube.com/@BBCNews/live",
-      "poster_url": "",
-      "sort_order": 220,
-      "is_enabled": true,
-      "notes": "Primary English global news channel"
-    },
-    {
-      "id": "france24-en",
-      "name": "France 24 English",
-      "provider": "France 24",
-      "region": "France",
-      "language": "en",
-      "source_type": "hls",
-      "stream_url": "https://example.com/live.m3u8",
-      "homepage_url": "https://www.france24.com/en/live",
-      "sort_order": 230,
-      "is_enabled": true
-    },
-    {
-      "id": "cctv4-page",
-      "name": "CCTV-4 中文国际",
-      "provider": "CCTV",
-      "region": "China",
-      "language": "zh-CN",
-      "source_type": "iframe",
-      "embed_url": "https://tv.cctv.com/live/cctv4/",
-      "homepage_url": "https://tv.cctv.com/live/cctv4/",
-      "sort_order": 10,
-      "is_enabled": true
-    }
-  ]
-}
-```
-
-## 字段约定
-
-- `id`: 唯一标识，建议稳定不变
-- `name`: 频道显示名
-- `provider`: 提供方
-- `region`: 国家或地区
-- `language`: 语言代码
-- `source_type`: `iframe` / `hls` / `video` / `external` / `youtube`
-- `embed_url`: 适合 iframe 内嵌的页面
-- `stream_url`: 直接视频流地址
-- `homepage_url`: 官网或频道页
-- `youtube_video_id`: YouTube 直播视频 ID
-- `youtube_channel`: YouTube 频道 handle 或频道 URL
-- `poster_url`: 封面图，可选
-- `sort_order`: 排序值，越小越靠前
-- `is_enabled`: 是否启用
-- `notes`: 简短备注
-
-## 面板行为约定
-
-- `youtube`
-  - 优先使用 `youtube_video_id`
-  - 无法内嵌时至少保留 `youtube_channel` 或 `homepage_url` 供外部打开
-- `hls` / `video`
-  - 优先走 `stream_url`
-- `iframe`
-  - 优先走 `embed_url`
-- `external`
-  - 不尝试内嵌，只保留外部打开
-
-## 当前实现状态
-
-- 后台设置页可以手工维护频道目录
-- `Earth` 电视模块会合并：
-  - 手工配置源
-  - `news_live_streams` 采集器采集源
-- 当前默认兜底源为 `CCTV-4 中文国际`

docs/frontend-layout-guidelines.md

@@ -1,309 +0,0 @@
-# Frontend Layout Guidelines
-
-本项目后台页面默认遵循“单屏工作区”布局规范。目标不是让页面永远不溢出，而是确保在常见桌面视口下：
-
-- 页面主结构能在一屏内看清
-- 用户能同时看到页头、摘要区和主工作区
-- 超出的内容在模块内部滚动，而不是把整页纵向撑爆
-
-当前推荐参考实现：
-
-- [frontend/src/pages/BGP/BGP.tsx](/home/ray/dev/linkong/planet/frontend/src/pages/BGP/BGP.tsx)
-- [frontend/src/index.css](/home/ray/dev/linkong/planet/frontend/src/index.css)
-
-## 核心原则
-
-### 1. 页面优先保证一屏工作区
-
-管理页默认采用：
-
-- 页头：标题、说明、主要操作
-- 主工作区：统计卡、表格、图表、列表、标签页
-
-推荐结构：
-
-```tsx
-<AppLayout>
-  <div className="page-shell">
-    <div className="page-shell__header">...</div>
-    <div className="page-shell__body">...</div>
-  </div>
-</AppLayout>
-```
-
-页面总高度应被限制在 `AppLayout` 内容区内，而不是继续让整个页面自然向下增长。
-
-### 2. 滚动优先发生在模块内部
-
-如果表格、日志、长列表、图表明细超出空间：
-
-- 让卡片内部滚动
-- 让表格内部滚动
-- 让标签页内容区内部滚动
-
-不要默认依赖整个页面滚动去“解决”空间问题。
-
-### 3. 主工作区必须拿到主要空间
-
-页面里最重要的模块必须是视觉和空间上的主角。通常应保证：
-
-- 页头始终可见
-- 摘要区高度被控制
-- 主表格 / 主图表 / 主分析区占据 50% 以上可视高度
-
-如果一个页面有多个大模块，优先顺序是：
-
-1. 先压缩说明区和摘要区
-2. 再把次级模块收进标签页或切换视图
-3. 最后才考虑继续增加整页滚动
-
-### 4. 小屏幕和高缩放必须进入紧凑模式
-
-在窗口高度较低、宽度较窄、或系统缩放较高时，应主动切换紧凑布局，例如：
-
-- 缩小卡片 padding
-- 缩小表头和单元格间距
-- 将摘要区改为更紧凑的单行/横向滚动布局
-- 将次级模块移入标签页、抽屉、折叠区
-
-紧凑模式的目标是保持可用，不是单纯把文字和控件一股脑缩小。
-
-### 5. overflow 责任必须明确
-
-页面中的大块内容必须明确：
-
-- 谁负责占满剩余高度
-- 谁负责裁剪
-- 谁负责滚动
-
-常见要求：
-
-- 父容器链路需要 `min-height: 0`
-- 工作区容器通常需要 `display: flex`
-- 真正的滚动节点要显式 `overflow: auto`
-
-### 6. 卡片不能被压到不可读
-
-历史上我们反复踩到的问题不是“没有滚动条”，而是：
-
-- 卡片被 `flex` 压缩得只剩一小条可视区域
-- 文字能渲染，但读不完整
-- 内容其实存在，却被 `overflow: hidden` 裁掉
-
-因此后续约束是：
-
-- 先保证卡片有可读的最小高度
-- 如果继续压缩会影响阅读，就切换成内部滚动
-- 不要为了“保持一屏”而把正文、表格、描述区压成无法阅读的条状区域
-
-### 7. Tabs 不是天然安全的布局容器
-
-历史上 Tabs 相关回归非常多，典型问题包括：
-
-- 隐藏 tab pane 因为自定义 `display: flex` 而重新露出来
-- 所有 tab 被强行套用同一套高度/overflow 规则
-- 表格 tab 能工作，但 markdown / help / diagnostics tab 被压坏
-
-因此约束是：
-
-- `Tabs` 里的每类内容都要单独定义自己的布局策略
-- 表格 tab 可以是“固定高度 + 内部滚动”
-- 文档/Markdown tab 更适合“tab pane 自身滚动 + 内容正常文档流”
-- 如果覆盖组件库样式，必须同时检查 hidden 状态是否仍然成立
-
-### 8. 摘要区优先进入紧凑模式，而不是挤压正文
-
-历史经验表明，最容易被误处理的是顶部摘要卡：
-
-- 它们经常为了“都放下”被强行压窄
-- 然后正文、表格、AI 结果区一起失去主空间
-
-后续统一约束：
-
-- 小屏或高缩放时，摘要卡优先：
-  - 降低 padding
-  - 改成横向滚动
-  - 改成更紧凑的网格
-- 不要优先牺牲主工作区的可视面积
-
-### 9. 长文档类内容优先保证阅读体验
-
-像下面这些内容，不能直接套用“表格工作区”的逻辑：
-
-- AI 简报
-- 运行日志
-- 原始 JSON
-- 帮助说明
-- 多段描述性文本
-
-这些区域应该优先满足：
-
-- 标题和元信息稳定可见
-- 正文有明确的最小可读高度
-- 正文滚动策略单独定义
-- 支持 Markdown 表格、分隔线、引用、代码块等结构
-
-### 10. 高度关键路径要少包一层
-
-历史上不少滚动问题不是组件本身错，而是多包了一层之后：
-
-- 高度链路断掉
-- `min-height: 0` 没传下去
-- `overflow` 责任被吃掉
-
-因此：
-
-- 对高度关键区域，优先使用最直接的 DOM 结构
-- 使用 `Space`、额外包装 `div`、第三方布局容器时，要确认它们不会改变滚动和高度语义
-- 如果一个区域已经出现“内容明明有，但只剩一条缝”，优先怀疑中间包装层
-
-## 历史坑位总结
-
-从 Earth、Playground、BGP、DataSources 这些页面的 bugfix 可以归纳出几类高频坑：
-
-### 1. 用 `overflow: hidden` 掩盖布局问题
-
-表面上看页面“整齐了”，实际上会导致：
-
-- 内容被裁掉
-- tab 内容只剩一条缝
-- 面板明明渲染成功，但用户看不见
-
-正确做法：
-
-- 让真正的内容节点滚动
-- 不要让上层容器无差别裁剪所有子内容
-
-### 2. 把所有 tab 当成同一种内容
-
-表格、Markdown、帮助卡、日志流的空间需求完全不同。
-
-正确做法：
-
-- 表格：固定工作区 + 内部滚动
-- 文档：普通流式内容 + pane 级滚动
-- 侧边说明：内容驱动高度，不强行拉满
-
-### 3. 只做视觉缩小，不做空间重分配
-
-这会导致：
-
-- 卡片文字被截断
-- 表格只剩 1 到 2 行
-- 按钮和筛选区挤成一团
-
-正确做法：
-
-- 紧凑模式优先重排
-- 横向滚动摘要区
-- 折叠/收纳次级模块
-
-### 4. 父容器高度链不完整
-
-这是最常见的内部滚动失效原因。
-
-检查顺序：
-
-1. 外层是否真的有确定高度
-2. flex 父容器是否带了 `min-height: 0`
-3. 真正滚动节点是否明确 `overflow: auto`
-4. 中间包装层是否偷偷改了布局语义
-
-### 5. UI 状态和显示状态不同步
-
-Earth 相关改动里反复出现：
-
-- 图层隐藏了，但 hover/lock 还在
-- tooltip 还在显示旧对象
-- legend 没跟着切换
-
-这类约束同样适用于后台页面：
-
-- 被隐藏、卸载、切换出视图的内容，不应继续保留活跃交互状态
-
-## 推荐实现模式
-
-### 页面骨架
-
-优先复用项目里已有的通用结构：
-
-- `.dashboard-content-inner`
-- `.page-shell`
-- `.page-shell__header`
-- `.page-shell__body`
-- `.table-scroll-region`
-
-不要每个页面都重新发明一套完全不同的高度和滚动语义。
-
-### 表格工作区
-
-推荐模式：
-
-```tsx
-<Card>
-  <div className="table-scroll-region" ref={tableRegionRef}>
-    <Table
-      pagination={false}
-      scroll={{ x: 1200, y: tableHeight }}
-    />
-  </div>
-</Card>
-```
-
-要求：
-
-- 表格尽量在卡片内部滚动
-- `scroll.y` 应来自实际可用高度估算，而不是完全静态的魔法数字
-- 父容器链路要保证 header、body、content 的 overflow 都在表格内部闭合
-
-### 多模块页面
-
-如果一个页面同时有：
-
-- 摘要卡
-- 表格
-- 异常明细
-- 最近事件
-
-不建议简单纵向堆叠全部模块。优先使用：
-
-- 顶部摘要 + 底部单一主工作区
-- 标签页切换多个次级数据视图
-- 左右分栏，并保证每栏内部独立滚动
-
-## 不推荐的做法
-
-以下模式默认视为不符合本项目页面规范：
-
-- 依赖整页纵向滚动来显示主要工作区
-- 一个页面纵向堆 3 到 4 个大卡片，每个都想完整展示
-- 表格没有内部滚动，导致缩放后只能看到 1 到 2 行数据
-- 父容器缺少 `min-height: 0`，导致内部滚动失效
-- 只做视觉缩小，不处理真正的空间分配
-
-## 页面验收检查清单
-
-提交前至少检查：
-
-- 页头、摘要区、主工作区能否同时出现
-- 主工作区是否拿到了页面中最多的高度
-- 表格或明细溢出时，滚动条是否出现在模块内部
-- 卡片是否被压缩到文字显示不完整；如果会，是否已经切换为内部滚动
-- 浏览器缩放到 `125%` / `150%` 时是否仍可用
-- 低高度窗口下是否还保有合理的可见内容行数
-- Tabs、Card、Table 在 overflow 时是否仍可操作
-- 非表格 tab（Markdown、帮助说明、日志）是否有独立且合理的滚动策略
-
-## 落地顺序
-
-后续新增或重构后台页时，优先按这个顺序设计：
-
-1. 先定义主工作区
-2. 再确定哪些模块必须常驻可见
-3. 最后再做样式和视觉层次
-
-简单说：
-
-- 先保证空间分配正确
-- 再处理滚动边界
-- 最后再做美化

docs/hud-panel-component-plan.md

@@ -1,165 +0,0 @@
-# HUD Panel Component Plan
-
-## Goal
-
-Unify Earth HUD panels into a reusable component layer so new panels can share:
-
-- a consistent shell
-- a consistent header
-- a consistent action-button system
-- a consistent body and collapse pattern
-
-## Scope
-
-Target panels:
-
-- `tv-panel`
-- `news-panel`
-- `legend`
-- `layer-panel`
-- `earth-stats`
-- `info-card`
-- settings modal header/actions
-
-## Component Model
-
-### Base shell
-
-- `.hud-panel`
-- `.hud-panel--compact`
-- `.hud-panel--media`
-- `.hud-panel--collapsed`
-- `.hud-panel-hidden`
-- `.hud-panel.is-dragging`
-- `.hud-panel.is-layout-animating`
-
-### Header
-
-- `.hud-panel__header`
-- `.hud-panel__title-group`
-- `.hud-panel__title`
-- `.hud-panel__subtitle`
-- `.hud-panel__chip`
-- `.hud-panel__actions`
-
-Header baseline rule:
-
-- Header title styling is fixed by the component layer and should not drift per panel
-- Title font size, font weight, letter spacing, line height, text color, and vertical alignment come from the shared header tokens and structure
-- Header divider, border treatment, inner spacing, and title-to-actions alignment are part of the same shared baseline
-- Panel-specific header differences should be limited to explicit variants such as `compact` or `media`, or token overrides with documented intent
-- “Looks close enough” local header overrides should be treated as temporary compatibility code and removed during migration
-
-### Actions
-
-- `.hud-panel__action`
-- `.hud-panel__action--icon`
-- `.hud-panel__action--collapse`
-- `.hud-panel__action--close`
-- `.hud-panel__action--refresh`
-- `.hud-panel__action--external`
-
-Action-button baseline rule:
-
-- Header action buttons must have one fixed default style baseline across all HUD panels
-- Default width behavior, padding, icon size, radius, alignment, hover, and active feedback all come from `.hud-panel__action`
-- Panel-specific differences must be expressed through explicit variants or token overrides, not ad-hoc local button rewrites
-- `close` buttons are part of the same default action system and must not silently fall back to a separate legacy box model
-
-### Body
-
-- `.hud-panel__body`
-- `.hud-panel__body--scroll`
-- `.hud-panel__body--collapsible`
-
-### Collapse behavior
-
-- `.hud-panel--collapsed`
-- `.hud-panel--expand-up`
-- `.hud-panel--expand-down`
-
-Adaptive collapse / expand rule:
-
-- HUD panels support two expansion directions:
-  - top-to-bottom expansion
-  - bottom-to-top expansion
-- Expansion direction should be decided at runtime from available viewport space rather than hardcoded per panel
-- Use:
-  - `d` = available distance from the header anchor to the viewport bottom edge
-  - `h` = expected expanded panel height
-  - buffer = `20px`
-- Collapsed-state direction rule:
-  - if `d > h + 20px`, the next action direction is `expand-up`
-  - if `d <= h + 20px`, the next action direction is `expand-down`
-- To avoid jitter around the threshold, the shared controller should keep a small hysteresis band:
-  - if the current direction is already `up`, keep it until `d <= h`
-  - if the current direction is already `down`, keep it until `d > h + 20px`
-- The opposite edge is still a safety guard:
-  - if the chosen side cannot fit at all, fall back to the other side if it can fit
-  - if neither side fully fits, choose the side with more space and let the body scroll
-- If neither direction fully fits, choose the direction with more available space and let the body scroll
-- Collapse icon direction must match the active expansion direction so the icon always describes the real open/close motion
-- The collapse icon describes the next action, not the current state
-- This mapping is fixed component behavior and must not drift per panel:
-  - collapsed + expand-down => `expand_more`
-  - expanded + expand-down => `expand_less`
-  - collapsed + expand-up => `expand_less`
-  - expanded + expand-up => `expand_more`
-- Panels must not combine icon-name swapping with extra CSS rotation for the same collapse control
-- Expansion direction and icon direction must come from one shared source of truth in the component controller
-- The direction decision should be recomputed when opening, resizing the viewport, or restoring a dragged panel near another edge
-
-## Tokens
-
-Promote panel differences into CSS variables instead of duplicating selectors:
-
-- `--hud-panel-padding`
-- `--hud-header-padding`
-- `--hud-header-gap`
-- `--hud-action-padding`
-- `--hud-action-gap`
-- `--hud-action-icon-size`
-- `--hud-body-gap`
-- `--hud-body-max-height`
-- `--hud-chip-radius`
-- `--hud-title-font-size`
-- `--hud-title-font-weight`
-- `--hud-title-letter-spacing`
-- `--hud-title-line-height`
-- `--hud-title-color`
-- `--hud-header-border-color`
-- `--hud-header-divider-opacity`
-- `--hud-expand-direction`
-
-## Migration Order
-
-1. Build the shared component layer in `frontend/public/earth/css/hud.css`
-2. Migrate `tv-panel` and `news-panel` first as the reference implementation
-3. Migrate `legend` and `layer-panel` into a compact variant
-4. Migrate `earth-stats` and `info-card`
-5. Align settings modal header/actions with the same action system
-6. Remove legacy one-off button selectors after verification
-
-## Guardrails
-
-- Do not change panel behavior and data flow during the first pass
-- Keep old class names temporarily as compatibility hooks
-- Prefer variable overrides over per-panel reimplementation
-- Treat header action-button default styling as fixed component API, not per-panel design space
-- Treat header title typography, border, and divider styling as fixed component API, not per-panel design space
-- Treat collapse direction as a component behavior contract, not a one-off panel trick
-- Treat collapse icon semantics as a component behavior contract, not a per-panel visual preference
-- Verify header alignment and drag/collapse behavior after each migration batch
-
-## First Implementation Batch
-
-Batch 1 should only do:
-
-- shared header structure
-- shared action-button system
-- shared title typography and header border/divider baseline
-- shared collapsible body pattern
-- adaptive collapse direction logic and direction-aware collapse icons
-- migration of `tv-panel` and `news-panel`
-
-That keeps risk low while giving the rest of the HUD a stable target to migrate toward.

docs/plans/README.md

@@ -0,0 +1,41 @@
+# Plans Docs
+
+这里放“未来实施方案和未完成计划”的文档，重点回答：
+
+- 我们准备做什么
+- 为什么要做
+- 分几期做
+- 当前差距和下一步是什么
+
+适合放入这里的内容：
+
+- Earth / BGP / 地形 / 天球实施方案
+- AI Playground 发展计划
+- backend / datasource / agent roadmap
+- UE5 MVP 方案
+
+当前重点入口：
+
+- [earth-mobile-drawer-ui-plan.md](/home/ray/dev/linkong/planet/docs/plans/earth-mobile-drawer-ui-plan.md)
+- [earth-compute-center-bgp-style-plan.md](/home/ray/dev/linkong/planet/docs/plans/earth-compute-center-bgp-style-plan.md)
+- [earth-renderer-architecture-separation-plan.md](/home/ray/dev/linkong/planet/docs/plans/earth-renderer-architecture-separation-plan.md)
+- [earth-country-boundary-overlay-plan.md](/home/ray/dev/linkong/planet/docs/plans/earth-country-boundary-overlay-plan.md)
+- [earth-predicted-orbit-plan.md](/home/ray/dev/linkong/planet/docs/plans/earth-predicted-orbit-plan.md)
+- [earth-webgl-instancing-satellites-plan.md](/home/ray/dev/linkong/planet/docs/plans/earth-webgl-instancing-satellites-plan.md)
+- [earth-real-terrain-plan.md](/home/ray/dev/linkong/planet/docs/plans/earth-real-terrain-plan.md)
+- [earth-news-source-configuration-and-collector-plan.md](/home/ray/dev/linkong/planet/docs/plans/earth-news-source-configuration-and-collector-plan.md)
+- [earth-news-cruise-summary-plan.md](/home/ray/dev/linkong/planet/docs/plans/earth-news-cruise-summary-plan.md)
+- [earth-vessel-rendering-performance-plan.md](/home/ray/dev/linkong/planet/docs/plans/earth-vessel-rendering-performance-plan.md)
+- [frontend-public-docs-site-plan.md](/home/ray/dev/linkong/planet/docs/plans/frontend-public-docs-site-plan.md)
+- [frontend-ai-playground-development-plan.md](/home/ray/dev/linkong/planet/docs/plans/frontend-ai-playground-development-plan.md)
+- [ue5-mvp-fused-plan.md](/home/ray/dev/linkong/planet/docs/plans/ue5-mvp-fused-plan.md)
+
+不适合放入这里的内容：
+
+- 当前代码结构说明
+- 组件现状和实现入口
+- 已经落地的技术上下文说明
+
+这些应放入：
+
+- [docs/technical/README.md](/home/ray/dev/linkong/planet/docs/technical/README.md)

docs/plans/agents-agent-runtime-roadmap.md

@@ -10,9 +10,9 @@ This document connects three existing planning threads into one implementation r
 
 Related documents:
 
-- [aiprovider](/home/ray/dev/linkong/planet/docs/aiprovider.md)
-- [datasource-health-plan](/home/ray/dev/linkong/planet/docs/datasource-health-plan.md)
-- [agent-architecture-plan](/home/ray/dev/linkong/planet/docs/agent-architecture-plan.md)
+- [aiprovider](/home/ray/dev/linkong/planet/docs/technical/agents-aiprovider.md)
+- [datasource-health-plan](/home/ray/dev/linkong/planet/docs/plans/agents-datasource-health-plan.md)
+- [agent-architecture-plan](/home/ray/dev/linkong/planet/docs/plans/agents-agent-architecture-plan.md)
 
 
 ## Big Picture

docs/plans/datasource-custom-api-mapping-plan.md

@@ -0,0 +1,426 @@
+# 自定义 API 数据源与 LLM 映射系统 — 实施计划
+
+**状态**：规划中
+**创建日期**：2026-04-28
+**核心原则**：LLM 辅助生成映射配置；生产采集使用确定性转换引擎
+
+## 已确认决策
+
+| 项目 | 决策 |
+|-----|------|
+| 自定义 API 的定位 | 作为内置数据源的补充入口，不直接等同于 Earth 新功能 |
+| LLM 的职责 | 探索未知 API、分析样本 JSON、生成 mapping 草案 |
+| 采集时是否调用 LLM | 不调用；采集链路必须确定性、可审计、可复现 |
+| 自定义数据如何进入 Earth | 必须映射到已支持的目标 schema，或先进入通用数据沉淀 |
+| 外部凭证放置位置 | Settings / 外部集成统一管理 provider token；DataSources 引用 provider profile |
+| TimescaleDB | 放入 TODO；高频时序数据稳定后再评估迁移 |
+
+---
+
+## 一、背景与问题
+
+当前系统已经有 `datasource_configs`，可以配置自定义数据源的 endpoint、auth、headers、config，也已经有部分 collector 会读取这些配置。但这只能解决“怎么请求数据”，还没有解决以下问题：
+
+- API 返回 JSON 后，如何转换成系统已有领域模型。
+- 自定义数据源是补充已有能力，还是全新数据沉淀。
+- 转换规则由谁生成、谁校验、谁执行。
+- 未知数据是否能自动在 Earth 上展示。
+- 外部 token 是放在全局配置中心，还是放在每个 datasource 下。
+
+专业做法是把“请求配置”“外部凭证”“目标 schema”“字段映射”“采集执行”拆开：
+
+- Settings 管外部集成凭证，例如 AI Provider、BarentsWatch、未来付费 AIS API。
+- DataSources 管具体数据源实例，例如 endpoint、调度频率、目标 schema、mapping 版本。
+- LLM 只在配置阶段辅助生成 mapping，不进入生产采集链路。
+- Earth 只消费明确 schema 的数据，不消费任意未知 JSON。
+
+---
+
+## 二、目标架构
+
+### 2.1 自定义 API 数据源生命周期
+
+```mermaid
+flowchart LR
+    A[配置 endpoint/auth/request] --> B[抓取 sample JSON]
+    B --> C[选择目标 schema]
+    C --> D[LLM 生成 mapping 草案]
+    D --> E[确定性 mapping engine 预览]
+    E --> F[schema validation]
+    F --> G[保存 mapping version]
+    G --> H[scheduler 执行 mapped collector]
+    H --> I[写入目标表或 generic_records]
+```
+
+### 2.2 目标 schema 分层
+
+| schema | 用途 | Earth 可视化 |
+|-------|------|-------------|
+| `vessel_ais` | 船只 AIS 位置、航速、航向、MMSI 等 | 进入船舶图层 |
+| `geo_points` | 通用点位数据，包含经纬度、名称、类型、时间 | 进入通用 geo layer（TODO） |
+| `news_events` | 新闻/事件类数据，带时间、地点、摘要、来源 | 复用新闻/事件链路 |
+| `compute_centers` | 算力中心、机房、数据中心数据 | 复用算力中心图层 |
+| `generic_records` | 未知结构化数据沉淀 | 不直接展示 |
+
+v1 建议优先实现：
+
+- `vessel_ais`
+- `geo_points`
+- `generic_records`
+
+其他 schema 可先在 registry 中预留名称，但不承诺完整落库与可视化。
+
+### 2.3 LLM 的边界
+
+LLM 可以做：
+
+- 根据 API 文档或 sample JSON 解释字段含义。
+- 推荐目标 schema。
+- 生成 mapping JSON 草案。
+- 给出字段置信度和需要人工确认的字段。
+- 帮用户发现分页、数组路径、时间字段、坐标字段。
+
+LLM 不应该做：
+
+- 在正式采集时参与每批数据转换。
+- 生成并执行 Python/JavaScript 代码。
+- 接触 API key、bearer token、basic auth password。
+- 自动创建新的 Earth 图层或数据库表。
+
+---
+
+## 三、后端实施计划
+
+### Phase 1 — Target Schema Registry
+
+新增代码级 registry，统一描述系统支持的目标数据类型。
+
+每个 target schema 至少包含：
+
+- `key`：例如 `vessel_ais`。
+- `label`：前端展示名称。
+- `description`：适用场景。
+- `fields`：字段名、类型、是否必填、说明、示例。
+- `validator`：Pydantic 或等价校验器。
+- `destination`：写入目标，例如 vessel 表、generic_records、future geo layer。
+
+示例概念：
+
+```json
+{
+  "key": "vessel_ais",
+  "fields": [
+    {"name": "mmsi", "type": "integer", "required": true},
+    {"name": "lat", "type": "float", "required": true},
+    {"name": "lon", "type": "float", "required": true},
+    {"name": "sog", "type": "float", "required": false},
+    {"name": "cog", "type": "float", "required": false},
+    {"name": "received_at", "type": "datetime", "required": false}
+  ]
+}
+```
+
+### Phase 2 — Mapping Template Model
+
+新增 mapping 配置持久化表，建议命名为 `datasource_mapping_templates`。
+
+关键字段：
+
+- `id`
+- `datasource_config_id`
+- `target_schema`
+- `mapping_json`
+- `sample_payload_hash`
+- `validation_status`
+- `version`
+- `is_active`
+- `created_at`
+- `updated_at`
+
+`mapping_json` 是声明式 DSL，不允许任意代码执行。
+
+示例：
+
+```json
+{
+  "source": {
+    "items_path": "$.data.vessels[*]"
+  },
+  "fields": {
+    "mmsi": {"path": "$.mmsi", "type": "integer"},
+    "lat": {"path": "$.latitude", "type": "float"},
+    "lon": {"path": "$.longitude", "type": "float"},
+    "sog": {"path": "$.speedOverGround", "type": "float", "default": null},
+    "received_at": {"path": "$.timestamp", "type": "datetime"}
+  }
+}
+```
+
+### Phase 3 — Deterministic Mapping Engine
+
+实现独立 mapping engine，输入 sample/raw payload 和 mapping JSON，输出目标 schema 记录。
+
+v1 支持能力：
+
+- JSONPath/JMESPath 风格路径提取。
+- 数组展开。
+- 默认值。
+- 基础类型转换：string、integer、float、boolean、datetime。
+- 坐标范围校验。
+- 简单枚举映射。
+- 错误收集：缺字段、类型转换失败、路径不存在。
+
+明确不支持：
+
+- 任意表达式执行。
+- 用户提交脚本。
+- LLM runtime 修复。
+
+### Phase 4 — LLM Mapping Assistant API
+
+新增配置阶段 API：
+
+- `POST /api/v1/datasources/custom/sample`
+  - 按 datasource 请求配置抓取 sample JSON。
+- `GET /api/v1/datasources/target-schemas`
+  - 返回可选目标 schema 和字段说明。
+- `POST /api/v1/datasources/mappings/propose`
+  - 输入 sample JSON + target schema，调用 AI provider 生成 mapping 草案。
+- `POST /api/v1/datasources/mappings/preview`
+  - 使用确定性 mapping engine 预览转换结果。
+- `POST /api/v1/datasources/mappings`
+  - 保存 mapping 版本。
+- `PUT /api/v1/datasources/mappings/{id}`
+  - 更新 mapping，生成新版本或覆盖草稿。
+- `POST /api/v1/datasources/{id}/run-mapped`
+  - 手动触发一次 mapped collector。
+
+安全要求：
+
+- `propose` 请求发送给 LLM 前必须脱敏 sample。
+- auth headers、token、password 不进入 prompt。
+- LLM 返回结果必须再经过 mapping schema 校验。
+
+### Phase 5 — Generic Mapped HTTP Collector
+
+新增通用 collector：
+
+- 读取 `DataSourceConfig` 请求配置。
+- 读取 active mapping template。
+- 拉取 API 数据。
+- 使用 mapping engine 转换。
+- 使用 target schema validator 校验。
+- 调用 destination handler 写入目标表或 generic storage。
+- 将失败记录写入错误日志或 dead-letter 结构。
+
+对于 `generic_records`：
+
+- 保存 datasource id。
+- 保存 target schema。
+- 保存 normalized JSON。
+- 保存 raw payload 摘要或 raw reference。
+- 保存采集时间、source timestamp、mapping version。
+
+---
+
+## 四、前端实施计划
+
+### Phase 1 — Settings 外部集成
+
+Settings 中保留统一外部集成配置：
+
+- AI Provider：base URL、model、API key。
+- BarentsWatch：client id/client secret 或 bearer token。
+- 未来付费接口：AISHub、MarineTraffic、VesselFinder 等 provider profile。
+
+DataSources 不直接管理全局 secret，只引用 provider profile。
+
+### Phase 2 — Settings 自定义源向导
+
+自定义数据源配置入口应放在 `/settings` 的“采集器设置”或后续专门的自定义采集器设置区。`/datasources` 保持数据源目录和采集触发职责，不再承载编辑入口。
+
+自定义数据源配置改成向导或右侧 drawer：
+
+1. Request
+   - endpoint
+   - method
+   - auth profile
+   - headers
+   - query/body config
+   - schedule
+2. Sample
+   - 点击抓取 sample
+   - 展示 JSON tree
+   - 支持选择数组根路径
+3. Target Schema
+   - 选择 `vessel_ais`、`geo_points`、`generic_records`
+   - 展示该 schema 必填字段
+4. Mapping Proposal
+   - 调用 LLM 生成 mapping 草案
+   - 显示字段匹配置信度
+   - 标出需要人工确认的字段
+5. Preview
+   - 用确定性 engine 预览前 N 条转换结果
+   - 展示校验错误
+6. Save & Enable
+   - 保存 mapping version
+   - 启用调度或仅保存草稿
+
+### Phase 3 — 运维视图
+
+为 mapped datasource 展示：
+
+- 上次运行时间。
+- 成功记录数。
+- 失败记录数。
+- 当前 mapping version。
+- 目标 schema。
+- 最近错误。
+- 手动运行按钮。
+
+---
+
+## 五、数据库与存储策略
+
+### v1：继续使用 PostgreSQL
+
+PostgreSQL 可以承载当前规模的采集、关系查询、JSONB 沉淀和基础时序查询。v1 不必因为“时序数据”立刻引入 TimescaleDB。
+
+适合继续用 PostgreSQL 的场景：
+
+- 数据量可控。
+- 最近状态查询为主。
+- 历史保留窗口较短。
+- 查询模式还没稳定。
+- 需要快速迭代 schema 与 mapping。
+
+### TODO：TimescaleDB
+
+以下条件满足后，再评估 TimescaleDB：
+
+- AIS、遥测、轨迹类数据达到高频持续写入。
+- 需要按时间窗口做聚合、降采样、retention policy。
+- 单表时间序列查询明显成为瓶颈。
+- 历史轨迹保留从 24h 扩展到数周或数月。
+
+候选迁移对象：
+
+- `vessel_position`
+- future telemetry tables
+- future generic time-series records
+
+备选方案：
+
+- PostgreSQL 原生按天/月分区。
+- TimescaleDB hypertable。
+- 热数据 PostgreSQL，冷数据对象存储。
+
+---
+
+## 六、安全与治理
+
+### Secret 管理
+
+- Settings 中保存 provider credentials。
+- API 返回配置时必须 mask secret。
+- LLM prompt 只能包含脱敏 sample 和 schema 说明。
+- 后续 TODO：引入字段级加密或 KMS。
+
+### Mapping 治理
+
+- 每次 mapping 变更保留版本。
+- active mapping 只能有一个。
+- 允许保存 draft mapping。
+- 运行记录关联 mapping version。
+- 校验失败不能自动启用。
+
+### 错误处理
+
+常见错误类型：
+
+- API 401/403：凭证错误或过期。
+- API 429：限流，需要调整 schedule。
+- JSON path 不存在：上游结构变化。
+- 类型转换失败：mapping 规则错误。
+- schema validation failed：转换结果不满足目标模型。
+
+每次运行需要记录：
+
+- datasource id。
+- mapping version。
+- started_at / finished_at。
+- fetched count。
+- mapped count。
+- written count。
+- failed count。
+- error summary。
+
+---
+
+## 七、测试计划
+
+### Backend Unit Tests
+
+- mapping engine：
+  - path 提取。
+  - 数组展开。
+  - 默认值。
+  - 类型转换。
+  - datetime parse。
+  - 枚举映射。
+  - 缺字段错误。
+- target schema registry：
+  - `vessel_ais` 必填字段校验。
+  - `geo_points` 经纬度范围校验。
+  - `generic_records` 接受未知结构。
+- LLM assistant：
+  - mock provider 返回 mapping。
+  - 验证 secret 不进入 prompt。
+  - 验证非法 mapping 被拒绝。
+
+### Backend Integration Tests
+
+- sample JSON -> propose mapping -> preview -> save mapping。
+- mapped collector 使用保存的 mapping 写入 `generic_records`。
+- `vessel_ais` sample 写入船舶相关目标结构。
+- 上游 JSON 结构变化时，运行失败并记录错误。
+
+### Frontend Tests
+
+- 自定义数据源向导完整流程。
+- 未配置 AI Provider 时，提示去 Settings 配置，但允许手写 mapping。
+- LLM 返回不完整 mapping 时，Preview 阶段显示校验错误。
+- 保存 mapping 后展示 active version 和运行状态。
+
+---
+
+## 八、分期工作量
+
+| 阶段 | 内容 | 估算 |
+|-----|------|------|
+| Phase 0 | 完成本规划、确认 schema registry 设计 | 0.5 天 |
+| Phase 1 | target schema registry + mapping template model | 1–2 天 |
+| Phase 2 | deterministic mapping engine | 2–3 天 |
+| Phase 3 | sample/propose/preview/save API | 2–3 天 |
+| Phase 4 | DataSources 自定义源向导 | 3–5 天 |
+| Phase 5 | generic mapped collector + run history | 2–4 天 |
+| Phase 6 | vessel_ais / geo_points destination handler | 2–4 天 |
+
+---
+
+## 九、当前差距与下一步
+
+当前差距：
+
+- `datasource_configs` 只描述请求配置，不描述目标 schema 和 mapping。
+- 自定义源没有 sample -> schema -> mapping -> preview -> save 的闭环。
+- 生产采集还没有通用 mapped collector。
+- Settings 与 DataSources 的职责边界需要在 UI 上进一步明确。
+- Earth 还没有通用 `geo_points` 图层。
+
+下一步建议：
+
+1. 先实现 target schema registry 和 mapping engine，不急着接 LLM。
+2. 用固定 sample JSON 做 `vessel_ais` 和 `generic_records` 的单元测试。
+3. 再接 LLM propose API，让 LLM 产出的只是 mapping 草案。
+4. 最后做前端向导，把人工确认和 preview 放到启用之前。

docs/plans/earth-bgp-region-aggregation-plan.md

@@ -17,7 +17,7 @@ It is an aggregation/view-model layer:
 
 ## Why This Layer Exists
 
-Current product gap from [bgp-context.md](/home/ray/dev/linkong/planet/docs/earth/bgp-context.md):
+Current product gap from [bgp-context.md](/home/ray/dev/linkong/planet/docs/technical/earth-bgp-context.md):
 
 - incident density is naturally low
 - anomaly density is higher, but still not enough to keep the globe expressive all the time
@@ -290,7 +290,7 @@ Each feature should include:
 
 ## Earth Rendering Plan
 
-Detailed visual layering guidance is expanded in [bgp-earth-rendering-plan.md](/home/ray/dev/linkong/planet/docs/earth/bgp-earth-rendering-plan.md).
+Detailed visual layering guidance is expanded in [bgp-earth-rendering-plan.md](/home/ray/dev/linkong/planet/docs/plans/earth-bgp-earth-rendering-plan.md).
 
 ### Layer Relationship
 

docs/plans/earth-celestial-background-plan.md

@@ -112,6 +112,285 @@
 
 - [Three.js SpriteMaterial](https://threejs.org/docs/pages/SpriteMaterial.html)
 
+## 天球背景资源与星体数据来源
+
+为避免把“视觉背景”和“可计算天体位置”混为一谈，本方案明确分成两类资源：
+
+### 1. 背景资源：全天星图贴图
+
+用于 Phase 1 的“真实天空背景”。
+
+推荐优先来源：
+
+- NASA SVS 的 Tycho 全天星图
+  - [The Tycho Catalog Skymap - Version 2.0](https://svs.gsfc.nasa.gov/3572/)
+- NASA Deep Star Maps 2020
+  - SatelliteMap.space 在 credits 中明确提到其使用了 `NASA Deep Star Maps 2020 - High-resolution star field (1.7 billion stars from Gaia DR2)` 作为星空视觉资源
+  - 这说明行业内成熟实现并不一定直接渲染全部星表点，而很可能先使用一张高质量官方深空星图作为背景层
+- 如需后续替换，也可评估 ESA / Gaia 的全天 sky map 资源
+  - [Gaia DR3 stories](https://www.cosmos.esa.int/web/gaia/dr3-stories)
+
+建议要求：
+
+- 使用官方来源或官方衍生可复用资源
+- 等距矩形投影（equirectangular）
+- 坐标定义尽量明确为赤道坐标展开
+- 分辨率建议至少 `4k`
+- 颜色不要过亮，避免压过 Earth HUD 前景
+- 尽量优先选择官方天文机构已经生产好的深空图，而不是自行拼接低质量星空纹理
+
+建议本地资源目录：
+
+- `frontend/public/earth/assets/celestial/starmap_equatorial_4k.jpg`
+
+### 2. 位置数据：星表与天体计算
+
+用于 Phase 2+ 的“位置正确的星体”。
+
+推荐来源分两层：
+
+- 太阳、月亮位置
+  - 使用 [Astronomy Engine](https://github.com/cosinekitty/astronomy)
+- 恒星位置
+  - 第一优先：Hipparcos / Tycho
+    - [Hipparcos overview](https://www.cosmos.esa.int/web/Hipparcos)
+    - [Hipparcos catalogues](https://www.cosmos.esa.int/web/hipparcos/catalogues)
+  - 第二优先：Gaia
+    - [Gaia DR3 stories](https://www.cosmos.esa.int/web/gaia/dr3-stories)
+
+建议策略：
+
+- V1：背景球壳只用全天星图，不立即生成全量恒星点
+- V2：只挑选亮星（例如星等 `< 5.5`）生成恒星点层
+- V3：如果确实需要更丰富的星场，再逐步扩展到更深星等
+
+这样做的原因：
+
+- 背景球壳负责“天球真实感”
+- 亮星点负责“位置正确、可后续标注和高亮”
+- 不需要一开始就处理数十万甚至数百万颗星
+
+### 3. 对外部成熟实现的参考结论
+
+`SatelliteMap.space` 的公开 credits 提供了一个很有价值的参考样板：
+
+- 图形渲染使用 `TWGL.js`
+- 天文计算使用 `Skyfield` 与 `Astronomia`
+- 星空/天球视觉资源使用 `NASA Deep Star Maps 2020`
+
+这给本项目的启发是：
+
+- “真实感强的天球背景”完全可以先依赖官方高质量深空图
+- “位置正确的动态天体”则应依赖单独的天文计算链路
+- 没有必要在第一版就直接渲染完整星表
+
+因此本项目推荐继续坚持两层拆分：
+
+- 背景层：官方深空图 / 全天星图
+- 计算层：太阳、月亮与后续亮星点
+
+## 如何保证星体位置正确
+
+位置正确不是只看“图看起来像”，而是要统一参考系和转换链路。
+
+### 1. 统一坐标基准
+
+本方案推荐统一使用：
+
+- `J2000` 赤道坐标系作为恒星位置基准
+
+原因：
+
+- Hipparcos / Tycho 资料和大量天文可视化都容易映射到该基准
+- 太阳、月亮也可以通过 Astronomy Engine 转到同一坐标系
+- 这样背景、恒星点、太阳、月亮就能共用一套 sky orientation
+
+### 2. 背景贴图与点位必须使用同一展开逻辑
+
+如果背景球壳使用赤道坐标全天图，那么：
+
+- 亮星点也必须按赤道坐标贴到同一球面方向
+- 太阳/月亮 sprite 也必须按赤道坐标转换后落到同一 world-space
+
+否则会出现：
+
+- 背景银河带是对的
+- 但太阳/月亮或亮星点飘到不匹配的位置
+
+### 3. RA / Dec 到 Three.js 坐标的落点方式
+
+亮星点和日月方向最终都要转成单位球面向量。
+
+概念步骤：
+
+1. 读取赤经 `RA`
+2. 读取赤纬 `Dec`
+3. 转成弧度
+4. 映射到单位球面向量
+5. 再根据 Three.js 当前世界坐标定义做轴向映射
+
+参考公式：
+
+```text
+x = cos(dec) * cos(ra)
+y = sin(dec)
+z = cos(dec) * sin(ra)
+```
+
+实际接入 Three.js 时，需要做一次项目内坐标轴校准：
+
+- 验证 `RA = 0h`
+- 验证 `RA = 6h`
+- 验证北天极
+- 验证银河带主方向
+
+然后确定最终的：
+
+- `x/y/z` 对应 Three.js 哪个轴
+- 是否需要 `z` 取反
+- 是否需要整体再做一个固定 `rotation`
+
+建议把这层显式封装在：
+
+```js
+function equatorialToWorldVector(raRad, decRad)
+```
+
+不要把轴映射散落在不同模块里。
+
+### 4. 背景球壳与恒星点的关系
+
+推荐最终组合：
+
+- 背景层：全天星图球壳
+- 点位层：亮星点
+- 动态层：太阳 / 月亮
+
+这样有三个好处：
+
+- 背景层提供密集真实的天空纹理
+- 亮星点提供位置正确、可扩展的标注基础
+- 太阳/月亮提供与时间相关的真实动态对象
+
+## 数据与资源建议清单
+
+### 推荐首批引入资源
+
+1. 全天星图
+- 来源：NASA Tycho all-sky map
+- 用途：背景球壳纹理
+
+2. 月亮纹理
+- 用途：Phase 4 月相表现
+- 路径建议：
+  - `frontend/public/earth/assets/celestial/moon_albedo_2k.jpg`
+
+3. 太阳 glow 贴图
+- 用途：太阳 sprite halo
+- 路径建议：
+  - `frontend/public/earth/assets/celestial/sun_glow.png`
+
+### 推荐首批数据文件
+
+如果要上亮星层，建议新增一个预处理后的轻量数据文件：
+
+- `frontend/public/earth/assets/celestial/bright-stars.json`
+
+建议字段：
+
+```json
+[
+  {
+    "id": 32349,
+    "name": "Sirius",
+    "raDeg": 101.2875,
+    "decDeg": -16.7161,
+    "mag": -1.46,
+    "colorIndex": 0.00
+  }
+]
+```
+
+建议不要在浏览器里直接吞原始 Gaia 大表，而是先离线裁剪成：
+
+- 只保留亮星
+- 只保留渲染必需字段
+- JSON 或二进制轻量格式
+
+## 资源与数据实施路线
+
+### 路线 A：先做可用版本（推荐）
+
+1. 引入 NASA Tycho 全天图
+   - 或评估替换为更接近 SatelliteMap.space 路线的 `NASA Deep Star Maps 2020`
+2. 实现背景球壳
+3. 用 Astronomy Engine 计算太阳/月亮方向
+4. 暂不做亮星点
+
+优点：
+
+- 最快见效
+- 风险最低
+- 就能明显提升天球真实感
+
+### 路线 B：在 A 基础上增强
+
+1. 离线生成 `bright-stars.json`
+2. 浏览器端渲染亮星点
+3. 后续可加：
+   - 星座线
+   - 亮星名称
+   - 特定星体高亮
+
+优点：
+
+- 背景真实感和“位置正确的可交互星体”同时兼顾
+
+## 代码模块建议细化
+
+### 新增模块
+
+- `frontend/public/earth/js/celestial.js`
+  - 管理天球背景
+  - 管理太阳/月亮
+  - 管理亮星层（后续）
+
+- `frontend/public/earth/js/celestial-data.js`
+  - 资源路径
+  - 星图方向配置
+  - 亮星数据加载（后续）
+
+### 建议函数设计
+
+```js
+export function initCelestialLayer(scene)
+export function updateCelestialLayer(date)
+export function setCelestialVisibility(visible)
+export function disposeCelestialLayer()
+
+function loadStarMapTexture()
+function createSkySphere(texture)
+function createSunSprite()
+function createMoonSprite()
+function getSunEquatorialPosition(date)
+function getMoonEquatorialPosition(date)
+function equatorialToWorldVector(raRad, decRad)
+```
+
+### 推荐后续预处理脚本
+
+如要引入亮星层，建议单独做离线脚本：
+
+- `scripts/build_bright_stars.py`
+
+职责：
+
+- 从 Hipparcos / Tycho 源数据读取
+- 过滤亮星
+- 生成 `bright-stars.json`
+
+这样浏览器端只消费轻量结果，不承担大表解析成本。
+
 ## 分阶段实施
 
 ## Phase 1：真实天球背景